diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000000000000000000000000000000000000..e5434d92311a50c9f1d6ea5aab0816f1eb5e5ccf --- /dev/null +++ b/.gitignore @@ -0,0 +1,130 @@ +# Logs +logs +*.log +npm-debug.log* +yarn-debug.log* +yarn-error.log* +lerna-debug.log* + +# Diagnostic reports (https://nodejs.org/api/report.html) +report.[0-9]*.[0-9]*.[0-9]*.[0-9]*.json + +# Runtime data +pids +*.pid +*.seed +*.pid.lock + +# Directory for instrumented libs generated by jscoverage/JSCover +lib-cov + +# Coverage directory used by tools like istanbul +coverage +*.lcov + +# nyc test coverage +.nyc_output + +# Grunt intermediate storage (https://gruntjs.com/creating-plugins#storing-task-files) +.grunt + +# Bower dependency directory (https://bower.io/) +bower_components + +# node-waf configuration +.lock-wscript + +# Compiled binary addons (https://nodejs.org/api/addons.html) +build/Release + +# Dependency directories +node_modules/ +jspm_packages/ + +# Snowpack dependency directory (https://snowpack.dev/) +web_modules/ + +# TypeScript cache +*.tsbuildinfo + +# Optional npm cache directory +.npm + +# Optional eslint cache +.eslintcache + +# Microbundle cache +.rpt2_cache/ +.rts2_cache_cjs/ +.rts2_cache_es/ +.rts2_cache_umd/ + +# Optional REPL history +.node_repl_history + +# Output of 'npm pack' +*.tgz + +# Yarn Integrity file +.yarn-integrity + +# dotenv environment variables file +.env +.env.test + +# parcel-bundler cache (https://parceljs.org/) +.cache +.parcel-cache + +# Next.js build output +.next +out + +# Nuxt.js build / generate output +.nuxt +dist + +# Gatsby files +.cache/ +# Comment in the public line in if your project uses Gatsby and not Next.js +# https://nextjs.org/blog/next-9-1#public-directory-support +# public + +# vuepress build output +.vuepress/dist + +# Serverless directories +.serverless/ + +# FuseBox cache +.fusebox/ + +# DynamoDB Local files +.dynamodb/ + +# TernJS port file +.tern-port + +# Stores VSCode versions used for testing VSCode extensions +.vscode-test + +# yarn v2 +.yarn/cache +.yarn/unplugged +.yarn/build-state.yml +.yarn/install-state.gz +.pnp.* + +# mac +.DS_Store + +temp-docs + +app/.vitepress/.temp +app/.vitepress/.cache +app/.vitepress/dist +app/.vitepress/cache +app/.vitepress/public/menu/menu.json +app/.vitepress/public/menu/menu-en.json +app/zh/docs/**/* +app/en/docs/**/* \ No newline at end of file diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000000000000000000000000000000000000..3b7b82d0da2db857eda1a798dbd908ea136f07b5 --- /dev/null +++ b/LICENSE @@ -0,0 +1,427 @@ +Attribution-ShareAlike 4.0 International + +======================================================================= + +Creative Commons Corporation ("Creative Commons") is not a law firm and +does not provide legal services or legal advice. Distribution of +Creative Commons public licenses does not create a lawyer-client or +other relationship. Creative Commons makes its licenses and related +information available on an "as-is" basis. Creative Commons gives no +warranties regarding its licenses, any material licensed under their +terms and conditions, or any related information. Creative Commons +disclaims all liability for damages resulting from their use to the +fullest extent possible. + +Using Creative Commons Public Licenses + +Creative Commons public licenses provide a standard set of terms and +conditions that creators and other rights holders may use to share +original works of authorship and other material subject to copyright +and certain other rights specified in the public license below. The +following considerations are for informational purposes only, are not +exhaustive, and do not form part of our licenses. + + Considerations for licensors: Our public licenses are + intended for use by those authorized to give the public + permission to use material in ways otherwise restricted by + copyright and certain other rights. Our licenses are + irrevocable. Licensors should read and understand the terms + and conditions of the license they choose before applying it. + Licensors should also secure all rights necessary before + applying our licenses so that the public can reuse the + material as expected. Licensors should clearly mark any + material not subject to the license. This includes other CC- + licensed material, or material used under an exception or + limitation to copyright. More considerations for licensors: + wiki.creativecommons.org/Considerations_for_licensors + + Considerations for the public: By using one of our public + licenses, a licensor grants the public permission to use the + licensed material under specified terms and conditions. If + the licensor's permission is not necessary for any reason--for + example, because of any applicable exception or limitation to + copyright--then that use is not regulated by the license. Our + licenses grant only permissions under copyright and certain + other rights that a licensor has authority to grant. Use of + the licensed material may still be restricted for other + reasons, including because others have copyright or other + rights in the material. A licensor may make special requests, + such as asking that all changes be marked or described. + Although not required by our licenses, you are encouraged to + respect those requests where reasonable. More_considerations + for the public: + wiki.creativecommons.org/Considerations_for_licensees + +======================================================================= + +Creative Commons Attribution-ShareAlike 4.0 International Public +License + +By exercising the Licensed Rights (defined below), You accept and agree +to be bound by the terms and conditions of this Creative Commons +Attribution-ShareAlike 4.0 International Public License ("Public +License"). To the extent this Public License may be interpreted as a +contract, You are granted the Licensed Rights in consideration of Your +acceptance of these terms and conditions, and the Licensor grants You +such rights in consideration of benefits the Licensor receives from +making the Licensed Material available under these terms and +conditions. + + +Section 1 -- Definitions. + + a. Adapted Material means material subject to Copyright and Similar + Rights that is derived from or based upon the Licensed Material + and in which the Licensed Material is translated, altered, + arranged, transformed, or otherwise modified in a manner requiring + permission under the Copyright and Similar Rights held by the + Licensor. For purposes of this Public License, where the Licensed + Material is a musical work, performance, or sound recording, + Adapted Material is always produced where the Licensed Material is + synched in timed relation with a moving image. + + b. Adapter's License means the license You apply to Your Copyright + and Similar Rights in Your contributions to Adapted Material in + accordance with the terms and conditions of this Public License. + + c. BY-SA Compatible License means a license listed at + creativecommons.org/compatiblelicenses, approved by Creative + Commons as essentially the equivalent of this Public License. + + d. Copyright and Similar Rights means copyright and/or similar rights + closely related to copyright including, without limitation, + performance, broadcast, sound recording, and Sui Generis Database + Rights, without regard to how the rights are labeled or + categorized. For purposes of this Public License, the rights + specified in Section 2(b)(1)-(2) are not Copyright and Similar + Rights. + + e. Effective Technological Measures means those measures that, in the + absence of proper authority, may not be circumvented under laws + fulfilling obligations under Article 11 of the WIPO Copyright + Treaty adopted on December 20, 1996, and/or similar international + agreements. + + f. Exceptions and Limitations means fair use, fair dealing, and/or + any other exception or limitation to Copyright and Similar Rights + that applies to Your use of the Licensed Material. + + g. License Elements means the license attributes listed in the name + of a Creative Commons Public License. The License Elements of this + Public License are Attribution and ShareAlike. + + h. Licensed Material means the artistic or literary work, database, + or other material to which the Licensor applied this Public + License. + + i. Licensed Rights means the rights granted to You subject to the + terms and conditions of this Public License, which are limited to + all Copyright and Similar Rights that apply to Your use of the + Licensed Material and that the Licensor has authority to license. + + j. Licensor means the individual(s) or entity(ies) granting rights + under this Public License. + + k. Share means to provide material to the public by any means or + process that requires permission under the Licensed Rights, such + as reproduction, public display, public performance, distribution, + dissemination, communication, or importation, and to make material + available to the public including in ways that members of the + public may access the material from a place and at a time + individually chosen by them. + + l. Sui Generis Database Rights means rights other than copyright + resulting from Directive 96/9/EC of the European Parliament and of + the Council of 11 March 1996 on the legal protection of databases, + as amended and/or succeeded, as well as other essentially + equivalent rights anywhere in the world. + + m. You means the individual or entity exercising the Licensed Rights + under this Public License. Your has a corresponding meaning. + + +Section 2 -- Scope. + + a. License grant. + + 1. Subject to the terms and conditions of this Public License, + the Licensor hereby grants You a worldwide, royalty-free, + non-sublicensable, non-exclusive, irrevocable license to + exercise the Licensed Rights in the Licensed Material to: + + a. reproduce and Share the Licensed Material, in whole or + in part; and + + b. produce, reproduce, and Share Adapted Material. + + 2. Exceptions and Limitations. For the avoidance of doubt, where + Exceptions and Limitations apply to Your use, this Public + License does not apply, and You do not need to comply with + its terms and conditions. + + 3. Term. The term of this Public License is specified in Section + 6(a). + + 4. Media and formats; technical modifications allowed. The + Licensor authorizes You to exercise the Licensed Rights in + all media and formats whether now known or hereafter created, + and to make technical modifications necessary to do so. The + Licensor waives and/or agrees not to assert any right or + authority to forbid You from making technical modifications + necessary to exercise the Licensed Rights, including + technical modifications necessary to circumvent Effective + Technological Measures. For purposes of this Public License, + simply making modifications authorized by this Section 2(a) + (4) never produces Adapted Material. + + 5. Downstream recipients. + + a. Offer from the Licensor -- Licensed Material. Every + recipient of the Licensed Material automatically + receives an offer from the Licensor to exercise the + Licensed Rights under the terms and conditions of this + Public License. + + b. Additional offer from the Licensor -- Adapted Material. + Every recipient of Adapted Material from You + automatically receives an offer from the Licensor to + exercise the Licensed Rights in the Adapted Material + under the conditions of the Adapter's License You apply. + + c. No downstream restrictions. You may not offer or impose + any additional or different terms or conditions on, or + apply any Effective Technological Measures to, the + Licensed Material if doing so restricts exercise of the + Licensed Rights by any recipient of the Licensed + Material. + + 6. No endorsement. Nothing in this Public License constitutes or + may be construed as permission to assert or imply that You + are, or that Your use of the Licensed Material is, connected + with, or sponsored, endorsed, or granted official status by, + the Licensor or others designated to receive attribution as + provided in Section 3(a)(1)(A)(i). + + b. Other rights. + + 1. Moral rights, such as the right of integrity, are not + licensed under this Public License, nor are publicity, + privacy, and/or other similar personality rights; however, to + the extent possible, the Licensor waives and/or agrees not to + assert any such rights held by the Licensor to the limited + extent necessary to allow You to exercise the Licensed + Rights, but not otherwise. + + 2. Patent and trademark rights are not licensed under this + Public License. + + 3. To the extent possible, the Licensor waives any right to + collect royalties from You for the exercise of the Licensed + Rights, whether directly or through a collecting society + under any voluntary or waivable statutory or compulsory + licensing scheme. In all other cases the Licensor expressly + reserves any right to collect such royalties. + + +Section 3 -- License Conditions. + +Your exercise of the Licensed Rights is expressly made subject to the +following conditions. + + a. Attribution. + + 1. If You Share the Licensed Material (including in modified + form), You must: + + a. retain the following if it is supplied by the Licensor + with the Licensed Material: + + i. identification of the creator(s) of the Licensed + Material and any others designated to receive + attribution, in any reasonable manner requested by + the Licensor (including by pseudonym if + designated); + + ii. a copyright notice; + + iii. a notice that refers to this Public License; + + iv. a notice that refers to the disclaimer of + warranties; + + v. a URI or hyperlink to the Licensed Material to the + extent reasonably practicable; + + b. indicate if You modified the Licensed Material and + retain an indication of any previous modifications; and + + c. indicate the Licensed Material is licensed under this + Public License, and include the text of, or the URI or + hyperlink to, this Public License. + + 2. You may satisfy the conditions in Section 3(a)(1) in any + reasonable manner based on the medium, means, and context in + which You Share the Licensed Material. For example, it may be + reasonable to satisfy the conditions by providing a URI or + hyperlink to a resource that includes the required + information. + + 3. If requested by the Licensor, You must remove any of the + information required by Section 3(a)(1)(A) to the extent + reasonably practicable. + + b. ShareAlike. + + In addition to the conditions in Section 3(a), if You Share + Adapted Material You produce, the following conditions also apply. + + 1. The Adapter's License You apply must be a Creative Commons + license with the same License Elements, this version or + later, or a BY-SA Compatible License. + + 2. You must include the text of, or the URI or hyperlink to, the + Adapter's License You apply. You may satisfy this condition + in any reasonable manner based on the medium, means, and + context in which You Share Adapted Material. + + 3. You may not offer or impose any additional or different terms + or conditions on, or apply any Effective Technological + Measures to, Adapted Material that restrict exercise of the + rights granted under the Adapter's License You apply. + + +Section 4 -- Sui Generis Database Rights. + +Where the Licensed Rights include Sui Generis Database Rights that +apply to Your use of the Licensed Material: + + a. for the avoidance of doubt, Section 2(a)(1) grants You the right + to extract, reuse, reproduce, and Share all or a substantial + portion of the contents of the database; + + b. if You include all or a substantial portion of the database + contents in a database in which You have Sui Generis Database + Rights, then the database in which You have Sui Generis Database + Rights (but not its individual contents) is Adapted Material, + + including for purposes of Section 3(b); and + c. You must comply with the conditions in Section 3(a) if You Share + all or a substantial portion of the contents of the database. + +For the avoidance of doubt, this Section 4 supplements and does not +replace Your obligations under this Public License where the Licensed +Rights include other Copyright and Similar Rights. + + +Section 5 -- Disclaimer of Warranties and Limitation of Liability. + + a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE + EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS + AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF + ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS, + IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION, + WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR + PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS, + ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT + KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT + ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU. + + b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE + TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION, + NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT, + INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES, + COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR + USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN + ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR + DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR + IN PART, THIS LIMITATION MAY NOT APPLY TO YOU. + + c. The disclaimer of warranties and limitation of liability provided + above shall be interpreted in a manner that, to the extent + possible, most closely approximates an absolute disclaimer and + waiver of all liability. + + +Section 6 -- Term and Termination. + + a. This Public License applies for the term of the Copyright and + Similar Rights licensed here. However, if You fail to comply with + this Public License, then Your rights under this Public License + terminate automatically. + + b. Where Your right to use the Licensed Material has terminated under + Section 6(a), it reinstates: + + 1. automatically as of the date the violation is cured, provided + it is cured within 30 days of Your discovery of the + violation; or + + 2. upon express reinstatement by the Licensor. + + For the avoidance of doubt, this Section 6(b) does not affect any + right the Licensor may have to seek remedies for Your violations + of this Public License. + + c. For the avoidance of doubt, the Licensor may also offer the + Licensed Material under separate terms or conditions or stop + distributing the Licensed Material at any time; however, doing so + will not terminate this Public License. + + d. Sections 1, 5, 6, 7, and 8 survive termination of this Public + License. + + +Section 7 -- Other Terms and Conditions. + + a. The Licensor shall not be bound by any additional or different + terms or conditions communicated by You unless expressly agreed. + + b. Any arrangements, understandings, or agreements regarding the + Licensed Material not stated herein are separate from and + independent of the terms and conditions of this Public License. + + +Section 8 -- Interpretation. + + a. For the avoidance of doubt, this Public License does not, and + shall not be interpreted to, reduce, limit, restrict, or impose + conditions on any use of the Licensed Material that could lawfully + be made without permission under this Public License. + + b. To the extent possible, if any provision of this Public License is + deemed unenforceable, it shall be automatically reformed to the + minimum extent necessary to make it enforceable. If the provision + cannot be reformed, it shall be severed from this Public License + without affecting the enforceability of the remaining terms and + conditions. + + c. No term or condition of this Public License will be waived and no + failure to comply consented to unless expressly agreed to by the + Licensor. + + d. Nothing in this Public License constitutes or may be interpreted + as a limitation upon, or waiver of, any privileges and immunities + that apply to the Licensor or You, including from the legal + processes of any jurisdiction or authority. + + +======================================================================= + +Creative Commons is not a party to its public +licenses. Notwithstanding, Creative Commons may elect to apply one of +its public licenses to material it publishes and in those instances +will be considered the “Licensor.” The text of the Creative Commons +public licenses is dedicated to the public domain under the CC0 Public +Domain Dedication. Except for the limited purpose of indicating that +material is shared under a Creative Commons public license or as +otherwise permitted by the Creative Commons policies published at +creativecommons.org/policies, Creative Commons does not authorize the +use of the trademark "Creative Commons" or any other trademark or logo +of Creative Commons without its prior written consent including, +without limitation, in connection with any unauthorized modifications +to any of its public licenses or any other arrangements, +understandings, or agreements concerning use of licensed material. For +the avoidance of doubt, this paragraph does not form part of the +public licenses. + +Creative Commons may be contacted at creativecommons.org. diff --git a/PATH_OWNER_MAPPING.YAML b/PATH_OWNER_MAPPING.YAML new file mode 100644 index 0000000000000000000000000000000000000000..677cd30ea3479765556b8d83ef1166891c848714 --- /dev/null +++ b/PATH_OWNER_MAPPING.YAML @@ -0,0 +1,75 @@ +relations: + - path: + - PATH_OWNER_MAPPING.yaml + - docs/zh/docs/A-Ops + - docs/zh/docs/Administration + - docs/zh/docs/ApplicationDev + - docs/zh/docs/Installation + - docs/zh/docs/K3s + - docs/zh/docs/Kernel + - docs/zh/docs/KubeEdge + - docs/zh/docs/Memory-fabric + - docs/zh/docs/Quickstart + - docs/zh/docs/Releasenotes + - docs/zh/docs/SecHarden + - docs/zh/docs/SystemOptimization + - docs/zh/docs/astream + - docs/zh/docs/certification + - docs/zh/docs/desktop + - docs/zh/docs/ops_guide + - docs/zh/docs/thirdparty_migration + - docs/zh/docs/userguide + - docs/en/docs/A-Ops + - docs/en/docs/ApplicationDev + - docs/en/docs/Pin + - docs/en/docs/GCC + - docs/en/docs/EulerMaker + - docs/zh/docs/NestOS + - docs/zh/docs/ROS + - docs/en/docs/UADK + - docs/zh/docs/oepkgs + - docs/zh/docs/memsafety + - docs/zh/docs/Migration-tools + - docs/zh/docs/CPDS + - docs/zh/docs/PilotGo + - docs/zh/docs/CVE-ease + - docs/zh/docs/GMEM + + owner: + - gitee_id: hujunjune + name: hujun + + - path: + - PATH_OWNER_MAPPING.yaml + - docs/zh/docs/A-Tune + - docs/zh/docs/Container + - docs/zh/docs/DPUOffload + - docs/zh/docs/Gazelle + - docs/zh/docs/HSAK + - docs/zh/docs/KernelLiveUpgrade + - docs/zh/docs/KubeOS + - docs/zh/docs/Kubernetes + - docs/zh/docs/ShangMi + - docs/zh/docs/StratoVirt + - docs/zh/docs/TailorCustom + - docs/zh/docs/Virtualization + - docs/zh/docs/oncn-bwm + - docs/zh/docs/rubik + - docs/zh/docs/secGear + - docs/zh/docs/SysCare + - docs/en/docs/A-Tune + - docs/en/docs/Container + - docs/en/docs/DPUOffload + - docs/zh/docs/Kmesh + - docs/zh/docs/sysBoost + + owner: + - gitee_id: amy_mayun + name: zhuyanting + + - path: + - PATH_OWNER_MAPPING.yaml + - docs/zh/docs/Embedded + owner: + - gitee_id: Youmi + name: x00352700 \ No newline at end of file diff --git a/README.en.md b/README.en.md deleted file mode 100644 index 57d177c634f69207c69c906247eac259c560241f..0000000000000000000000000000000000000000 --- a/README.en.md +++ /dev/null @@ -1,36 +0,0 @@ -# docs - -#### Description -To build and enrich documentation for openEuler project. - -#### Software Architecture -Software architecture description - -#### Installation - -1. xxxx -2. xxxx -3. xxxx - -#### Instructions - -1. xxxx -2. xxxx -3. xxxx - -#### Contribution - -1. Fork the repository -2. Create Feat_xxx branch -3. Commit your code -4. Create Pull Request - - -#### Gitee Feature - -1. You can use Readme\_XXX.md to support different languages, such as Readme\_en.md, Readme\_zh.md -2. Gitee blog [blog.gitee.com](https://blog.gitee.com) -3. Explore open source project [https://gitee.com/explore](https://gitee.com/explore) -4. The most valuable open source project [GVP](https://gitee.com/gvp) -5. The manual of Gitee [https://gitee.com/help](https://gitee.com/help) -6. The most popular members [https://gitee.com/gitee-stars/](https://gitee.com/gitee-stars/) diff --git a/README.md b/README.md deleted file mode 100644 index f4e13b3549095a827c5f9e3fcc431f6cf1cd3d11..0000000000000000000000000000000000000000 --- a/README.md +++ /dev/null @@ -1,37 +0,0 @@ -# docs - -#### 介绍 -To build and enrich documentation for openEuler project. - -#### 软件架构 -软件架构说明 - - -#### 安装教程 - -1. xxxx -2. xxxx -3. xxxx - -#### 使用说明 - -1. xxxx -2. xxxx -3. xxxx - -#### 参与贡献 - -1. Fork 本仓库 -2. 新建 Feat_xxx 分支 -3. 提交代码 -4. 新建 Pull Request - - -#### 特技 - -1. 使用 Readme\_XXX.md 来支持不同的语言，例如 Readme\_en.md, Readme\_zh.md -2. Gitee 官方博客 [blog.gitee.com](https://blog.gitee.com) -3. 你可以 [https://gitee.com/explore](https://gitee.com/explore) 这个地址来了解 Gitee 上的优秀开源项目 -4. [GVP](https://gitee.com/gvp) 全称是 Gitee 最有价值开源项目，是综合评定出的优秀开源项目 -5. Gitee 官方提供的使用手册 [https://gitee.com/help](https://gitee.com/help) -6. Gitee 封面人物是一档用来展示 Gitee 会员风采的栏目 [https://gitee.com/gitee-stars/](https://gitee.com/gitee-stars/) diff --git "a/archive/Kernel/\345\206\205\345\255\230\345\217\257\351\235\240\346\200\247\345\210\206\347\272\247\347\211\271\346\200\247\344\275\277\347\224\250\346\214\207\345\215\227.md" "b/archive/Kernel/\345\206\205\345\255\230\345\217\257\351\235\240\346\200\247\345\210\206\347\272\247\347\211\271\346\200\247\344\275\277\347\224\250\346\214\207\345\215\227.md" new file mode 100644 index 0000000000000000000000000000000000000000..4296a5c3fd05bfba4a99d2fc10db055f50732317 --- /dev/null +++ "b/archive/Kernel/\345\206\205\345\255\230\345\217\257\351\235\240\346\200\247\345\210\206\347\272\247\347\211\271\346\200\247\344\275\277\347\224\250\346\214\207\345\215\227.md" @@ -0,0 +1,353 @@ +# 内存可靠性分级管理 + +[1.1 概述](#概述) + +[1.2 约束限制](#约束限制) + +[1.3 使用方法](#使用方法) + +## 概述 + +本特性可以支撑使用者按照需求分配在对应可靠性的内存上，并对部分可能的UCE或CE故障影响进行一定程度的缓解，达到部分MR内存（address range mirror）的情况下，支撑业务整体可靠性不下降。 + +## 约束限制 + +本章节介绍该特性的通用约束，每个子特性会有具体的约束，在对应的小节中详细说明。 + +### 兼容性限制 + +1. 本特性目前仅适用于ARM64。 +2. 硬件需支持部分MR内存（address range mirror），即通过UEFI标准接口上报属性为EFI_MEMORY_MORE_RELIABLE的内存，普通内存无需额外置位。镜像内存（MR）对应为高可靠内存，普通内存对应为低可靠内存。 +3. 高低可靠内存分级借助内核的内存管理区（zone）来实现，两者无法进行动态流动（即页面不能在不同zone之间移动）。 +4. 不同可靠性的连续物理内存会被分隔到不同的memblock，可能会导致原本可以申请大块连续物理内存的场景在使能内存分级后受到限制。 +5. 使能本特性，需要依赖启动参数kernelcore的取值为“kernelcore=reliable”，与该参数其他取值均不兼容。 + +### 设计规格限制 + +1. 内核态开发时，内存申请操作需要注意： + + - 若内存申请接口支持指定gfp_flag，只有gfp_flag包含__GFP_HIGHMEM且__GFP_MOVABLE的内存申请会强制普通内存区域分配或者将这次内存分配重定向可靠内存区域，其他gfp_flag都不会进行干预。 + + - 从slab/slub/slob申请获取的都是高可靠内存（一次性申请内存大于KMALLOC_MAX_CACHE_SIZE时且gfp_flag指定为普通内存区域时可能申请到低可靠内存）。 + +2. 用户态开发时，内存申请操作需要注意： + + - 更改普通进程属性为关键进程后，实际物理内存分配阶段（page fault）才会使用高可靠内存，此前已分配的内存属性不会改变，反之亦然。因此普通进程被拉起到更改关键进程属性期间申请的内存可能不是高可靠内存。是否生效可以通过查询虚拟地址对应的物理地址是否属于高可靠内存段来验证。 + - Libc库如glibc中chunk等类似机制（ptmalloc、tcmalloc、dpdk）为了提高性能存在使用cache的逻辑，而内存cache会导致用户申请内存与内核内存申请逻辑不能完全对应，普通进程变成关键进程时并不能真正使能（该标记仅仅在内核实际发生内存申请时使能）。 + +3. 当上层业务申请内存的时发现高可靠内存不足（触发zone原生min水线）或者触发对应limit限制，会优先释放pagecache以尝试回收高可靠内存。如果仍然申请不到，内核会根据fallback的开关选择oom或fallback到低可靠内存区域完成内存申请。（fallback指某个内存管理区/节点内存不足时，到其他内存管理区/节点申请内存的情况。） + +4. 类似于NUMA_BALANCING的内存动态迁移机制，可能导致已经分配的高/低可靠内存被迁移到别的节点，由于该迁移操作丢失内存申请的上下文，且目标node可能没有对应可靠性的内存，因此可能导致迁移后的内存可靠性与预期不符。 + +5. 按照用户态高可靠内存用途引入如下三个配置文件： + + - /proc/sys/vm/task_reliable_limit: 关键进程（包含systemd）使用的高可靠内存上限。包含匿名页和文件页。进程使用的shmem也会被统计到其中（包含在匿名页中）。 + + - /proc/sys/vm/reliable_pagecache_max_bytes：全局pagecache使用的高可靠内存软上限。约束普通进程使用的高可靠pagecache的数量，系统默认不限制pagecache使用的高可靠内存的量。高可靠进程和文件系统元数据等场景不受此约束。无论fallback开关是否开启，普通进程触发该上限时，会默认申请低可靠内存，若低可靠内存申请不到，则遵循原生流程处理。 + + - /proc/sys/vm/shmem_reliable_bytes_limit：全局shmem使用的高可靠内存软上限。约束普通进程shmem使用高可靠内存的数量，系统默认不限制shmem使用的高可靠内存的量。高可靠进程不受此约束。关闭fallback时，普通进程触发该上限会导致内存申请失败，但不会OOM（与原生流程一致）。 + + 触及这些值可能会导致内存申请fallback或者OOM。 + + 关键进程在tmpfs或pagecache流程产生缺页引发的内存申请，有可能触发多个limit，多个limit之间交互关系情况详见表格。 + + | 是否触及task_reliable_limit | 是否触及reliable_pagecache_max_bytes或者shmem_reliable_bytes_limit | 内存申请处理策略 | + | --------------------------- | ------------------------------------------------------------ | ------------------------------------------------ | + | 是 | 是 | 优先回收pagecache以满足申请，否则Fallback或者OOM | + | 是 | 否 | 优先回收pagecache以满足申请，否则Fallback或者OOM | + | 否 | 否 | 先高可靠内存，失败Fallback或者OOM | + | 否 | 是 | 先高可靠内存，失败Fallback或者OOM | + + 关键进程会遵循task_reliable_limit的限制，如果task_reliable_limit高于tmpfs或pagecachelimit时，由关键进程产生的pagecache、tmpfs依旧会使用高可靠内存，由此会产生pagecache、tmpfs使用高可靠内存数量高于对应Limit的情况。 + + 当触发task_reliable_limit，如果高可靠filecache低于4M，不会进行同步回收。如果pagecache产生时，高可靠filecache低于4M，那么会fallback到低可靠内存完成申请，如果高于4M那么会优先回收pagecache满足此次申请。但接近4M时，会触发更频繁的cache直接回收，由于cache直接回收锁开销大，会导致高cpu占用率，此时文件读写性能接近裸盘性能。 + +6. 即使系统存在足够申请的高可靠内存，在如下场景下也存在fallback到低可靠内存区域内存申请的场景。 + + - 如果进行无法迁移到其他节点进行内存申请，那么会fallback当前节点的低可靠内存，常用场景举例如下： + - 如果内存申请带上了__GFP_THISNODE（如透明大页申请），代表只能从当前节点申请内存，如果此节点高可靠内存不满足申请情况，那么会尝试从本内存节点的低可靠内存区域进行内存申请。 + - 进程通过taskset、numactl等命令运行在某个包含普通内存节点。 + - 进程在系统内存原生的调度机制下调度到了某个包含普通内存节点。 + - 高可靠内存申请触发高可靠内存使用水线也会导致fallback到低可靠。 + +7. 内存分级fallback关闭时，高可靠内存将不能向低可靠内存扩展，有可能导致用户态应用对内存用量的判断与本特性不兼容，比如通过MemFree判断可用内存量。 + +8. 内存分级fallback开启时，对原生fallback有影响，主要区别在于内存管理区zone与NUMA节点的选择上，列举如下： + +- **普通用户进程**fallback流程将会是： 本节点低可靠内存->远端节点低可靠内存。 +- **关键用户进程**fallback流程将会是：本节点高可靠内存-> 远端节点高可靠内存。如果还未申请到内存且开启了memory reliable的fallback功能，将会额外重试： 本节点低可靠内存-> 远端节点低可靠内存。 + +### 场景限制 + +1. 默认页面大小（PAGE_SIZE）只支持4K页面大小。 +2. Numa Node0上低4G内存必须要为高可靠且高可靠内存大小与低可靠内存大小满足内核使用，否则可能导致系统无法启动。其他node的高可靠内存空间大小无要求，但需注意： + 某个node上没有高可靠内存或者高可靠内存不足，可能导致per-node管理结构位于其他node的高可靠内存上（因为其为内核数据结构需要在高可靠区域），由此会产生内核warning，如vmemmap会产生vmemmap_verify相关告警且存在性能影响。 +3. 本特性部分统计值（比如tmpfs高可靠总量）使用percpu技术进行统计，会有额外开销，计算总和时考虑到减少性能影响。因此存在一定的误差，误差10%内属于正常。 +4. 大页限制： + - 启动阶段静态大页为低可靠内存，运行时申请的静态大页默认为低可靠内存，如果内存申请发生在关键进程上下文，那么申请到的大页为高可靠内存。 + - 透明大页THP场景下，通过扫描进行合并大页（2M页为例）时如果待合并的512个4k页面中某一个为高可靠页面，那么新申请的2M大页会使用高可靠内存，透明大页会导致使用更多高可靠内存。 + - 2M大页预留申请遵循原生的fallback流程，如果当前node缺少低可靠内存，那么会fallback高可靠区间申请高可靠完成内存申请。 + - 启动阶段进行2M大页预留，如果没有指定内存节点，那么会负载均衡到每个内存节点进行大页的预留。如果某个内存节点缺少低可靠内存，那么会遵循原生流程使用高可靠。 +5. 当前仅仅支持正常系统启动场景。部分异常场景内核启动可能与内存分级功能不兼容，如kdump启动阶段（当前kdump已支持自动关闭，其他场景需要上层业务关闭。） +6. SWAP换入换出、内存offline、KSM、cma、giganic page流程下新申请的页面类型没有基于分级内存进行考量，可能出现未定义情况（未定义情况包括高可靠用量统计不准、申请到的内存可靠性等级与预期不符等）。 + +### 性能影响 + +- 物理页申请因分级管理的引入而增加了判断逻辑，会有一些性能影响，具体影响程度与系统状况、申请内存类型、各节点高低可靠内存余量有关。 +- 本特性引入高可靠内存相关用量统计值，会对系统性能产生影响。 +- 触发task_reliable_limit时，会对位于高可靠区域的cache同步回收，会增加CPU占用率。pagecache申请（文件读写操作，比如dd）触发task_reliable_limit的场景下，如果当前高可靠内存可用量（ReliableFileCache视为可用内存）接近4M时，会触发更频繁的cache直接回收，由于cache直接回收锁开销大，会导致高cpu占用率。此时文件读写性能接近裸盘性能。 + +## 使用方法 + +### OS支持内存分级管理 + +#### 概述 + +由于内存按照高低可靠性分为两段，内存的申请释放也需要按照高低可靠来进行分开管理。OS需要能够控制内存申请路径，用户态进程使用低可靠内存，内核态使用高可靠内存。高可靠内存不足时需要能够fallback到低可靠区申请或者直接申请失败。 + +同时对于进程部分内存段的可靠性需求与进程本身的性质，也需要能够支持按需指定申请高低可靠内存。如指定关键进程使用高可靠内存，减少关键进程遇到内存错误的概率。目前内核使用的都是高可靠内存，用户态进程使用的都是低可靠内存。如此会造成一些关键或者核心服务的不稳定，如业务转发进程，如果发生故障，会造成IO中断，影响业务的稳定性。因此需要对这些关键服务特殊处理，使其使用高可靠内存，提高关键进程运行的稳定性。 + +在系统遇到内存错误，OS应对未分配的低可靠内存进行覆盖写，以清除未发现的内存错误。 + +#### 约束限制 + +- **关键进程使用高可靠内存** + + 1. /proc//reliable 接口的滥用可能存在高可靠内存被过多使用的风险。 + 2. 用户态进程 reliable 属性只能在进程被拉起后，通过 proc 接口修改或者直接继承父进程该属性。systemd（pid=1）使用高可靠内存，其 reliable 属性无作用，也不继承，内核态线程reliable属性无效。 + 3. 进程的程序段和数据使用高可靠内存，高可靠不足，使用低可靠启动。 + 4. 普通进程在某些场景也会使用到高可靠内存，如hugetlb、pagecache、vdso、tmpfs等。 + +- **未分配内存覆盖写特性** + + 未分配内存覆盖写特性只能执行一次，不支持并发操作，如果执行会有如下影响： + + 1. 该特性耗时较大，每个 Node 有一个 CPU 被覆盖写线程所占用，其他任务在该 CPU 上无法调度。 + 2. 覆盖写过程需获取 Zone lock，其他业务进程内存申请要等待覆盖写完成，可能导致内存分配不及时。 + 3. 并发执行情况下会排队阻塞，产生更大的延时。 + + 如果机器性能不佳，将有可能触发内核RCU stall或soft lockup警告，以及进程内存申请操作被阻塞。因此请限制该特性在必要时只在物理机环境下使用，虚拟机等场景大概率出现如上现象。 + + 物理机参考数据可见下表（实际耗时与硬件性能、当前系统负载有关系）。 + +表：基于物理机TaiShan 2280 V2空载状态下测试数据 + +| 测试项 | Node 0 | Node 1 | Node 2 | Node 3 | +| ------------- | ------ | ------ | ------ | ------ | +| Free Mem (MB) | 109290 | 81218 | 107365 | 112053 | + +总耗时 3.2s + +#### 使用方法 + +本特性提供较多接口，使能特性并校验只需要步骤1-6即可。 + +1. 配置启动参数“kernelcore=reliable”，代表打开内存分级管理开关，CONFIG_MEMORY_RELIABLE是必要的配置，否则整个系统的内存可靠性分级管理不使能。 + +2. 根据需要，可以通过启动参数reliable_debug=[F][,S][,P]来选择性关闭fallback功能（F）、关闭tmpfs使用高可靠内存（S）以及关闭读写缓存使用高可靠内存（P），默认以上功能都使能。 + +3. 根据BIOS上报的地址段，查找高可靠内存，并进行标记，对于NUMA系统，不一定每个node上都要预留可靠内存，但是node 0上低4G物理空间必须为高可靠的内存，系统启动过程中会申请内存，如果无法分到高可靠内存，则会 fallback 到低可靠内存进行分配（mirror功能自带的fallback逻辑）或导致系统无法启动。如果使用低可靠内存，整个系统都不稳定，所以要保留node0上的高可靠内存且低4G物理空间必须为高可靠的内存。 + +4. 启动后，用户可以通过启动日志判断内存分级是否使能，应出现如下打印： + + ```Shell + mem reliable: init succeed, mirrored memory + ``` + +5. 高可靠内存对应的物理地址段可以通过启动日志来查询，观察efi上报memory map里的属性，带有“MR”的即为高可靠内存段，如下为启动日志节选，其中内存段mem06为高可靠内存，mem07为低可靠内存，其物理地址范围也列举在后（其他方式无法直接查询高低可靠内存地址范围）。 + + ```text + [ 0.000000] efi: mem06: [Conventional Memory| |MR| | | | | | |WB| | | ] range=[0x0000000100000000-0x000000013fffffff] (1024MB) + [ 0.000000] efi: mem07: [Conventional Memory| | | | | | | | |WB| | | ] range=[0x0000000140000000-0x000000083eb6cfff] (28651MB) + ``` + +6. 内核态开发时，对于一个页面struct page，可以通过其所处的zone来判断，ZONE_MOVABLE为低可靠内存区，zone编号小于ZONE_MOVABLE的均为高可靠内存区，判断方式举例如下。 + + ```c + bool page_reliable(struct page *page) + { + if (!mem_reliable_status() || !page) + return false; + return page_zonenum(page) < ZONE_MOVABLE; + } + ``` + + 此外提供的若干接口按照功能点分类列举如下： + + 1. **代码层面判断可靠性是否使能：**在内核模块中通过如下接口来判断，返回true表示内存分级功能真正使能，返回false则未使能。 + + ```c + #include + bool mem_reliable_status(void); + ``` + + 2. **内存热插拔：**如果内核本身使能内存热插拔操作（Logical Memory hot-add），高低可靠内存也支持该操作，操作单位为memory block，与原生流程一致。 + + ```Shell + # 上线内存到高可靠区 + echo online_kernel > /sys/devices/system/memory/auto_online_blocks + # 上线内存到低可靠区 + echo online_movable > /sys/devices/system/memory/auto_online_blocks + ``` + + 3. **动态关闭某项分级管理功能：**使用long类型控制根据每个bit判断内存分级功能开关与关闭某项功能： + + - bit0：内存可靠性分级管理功能。 + - bit1：禁止fallback到低可靠区域。 + - bit2：关闭tmpfs使用高可靠内存。 + - bit3：关闭pagecache使用高可靠内存。 + + 其他bit预留，用于扩展。如需更改，可通过如下proc接口（权限为600），取值范围0-15，（只有当总功能bit0为1时才会处理后续功能，否则直接关闭所有功能）。 + + ```Shell + echo 15 > /proc/sys/vm/reliable_debug + # 关闭所有功能，因为bit0为0 + echo 14 > /proc/sys/vm/reliable_debug + ``` + + 此命令只能用于关闭功能，不能打开。对于已经关闭的功能或者运行时关闭的功能，这个命令不能将其打开。 + + 注：此功能用于逃生使用，仅异常场景或者调测阶段需要关闭内存可靠性特性时配置，禁止作为常规功能直接使用。 + + 4. **查看高可靠内存部分统计信息：**可以通过原生/proc/meminfo来查看，其中： + + - ReliableTotal：内核管理可靠内存总大小。 + - ReliableUsed：系统使用可靠内存总大小，包含系统阶段reserved使用。 + - ReliableBuddyMem：伙伴系统剩余可靠总内存大小。 + - ReliableTaskUsed：表示当前关键用户进程，systemd使用的高可靠内存大小，包括匿名页与文件页。 + - ReliableShmem：表示共享内存高可靠用量，包括共享内存、tmpfs、rootfs使用高可靠内存总大小。 + - ReliableFileCache：表示读写缓存高可靠内存用量。 + + 5. **未分配内存覆盖写：**该功能需要打开配置项。 + + CONFIG_CLEAR_FREELIST_PAGE，并且添加启动参数clear_freelist，两者具备才会使能。通过proc接口触发，取值范围只能为1（权限为0200）。 + + ```Shell + echo 1 > /proc/sys/vm/clear_freelist_pages + ``` + + 注：该特性依赖启动参数clear_freelist，内核对启动参数只匹配前缀，故诸如“clear_freelisttt”也会生效该特性。 + + 为防止误操作，加入内核模块参数cfp_timeout_ms，代表调用覆盖写功能的最长执行时长（超时则没写完也退出），通过sys接口触发，默认取值为2000ms（权限为0644）： + + ```Shell + echo 500 > /sys/module/clear_freelist_page/parameters/cfp_timeout_ms # 设置超时为500ms + ``` + + 6. **查看更改当前进程高低可靠属性：**可以通过/proc//reliable来查看该进程是否是高可靠进程；运行写入，该标识会继承，如果子进程不需要，则手动修改子进程属性；systemd和内核线程不支持该属性的读写；可以写0或者1，默认为0，代表低可靠进程（权限为0644）。 + + ```Shell + # 更改pid=1024的进程为高可靠进程，从更改之后开始进程缺页申请的内存是从高可靠内存区域申请，申请不到有可能fallback到低可靠 + echo 1 > /proc/1024/reliable + ``` + + 7. **设置用户态高可靠进程申请上限：**通过/proc/sys/vm/task_reliable_limit来修改用户态进程申请高可靠内存的上限，对应取值范围为[ReliableTaskUsed, ReliableTotal]，单位为Byte（权限为0644）。需注意： + + - 缺省值为ulong_max，代表不受限制。 + - 当该值为0，可靠进程不可使用高可靠内存，fallback模式下，fallback到低可靠内存区域申请，否则直接OOM。 + - 当该值不为0并且触发该limit, 使能fallback功能，fallback到低可靠内存区域申请内存，不使能fallback功能，则返回OOM。 + +### 读写缓存使用高可靠内存 + +#### 概述 + +Page cache 也叫页缓冲或文件缓冲，在linux读写文件时，它用于缓存文件的逻辑内容，从而加快对磁盘上映像和数据的访问。Page cache申请如果使用低可靠内存，当访问时可能触发UCE导致系统异常。因此，需要将读写缓存（page cache）放到高可靠内存区域，同时为了防止Page cache申请过多（默认无限制）将高可靠内存耗尽，需要对page cache的总量及使用可靠内存总量进行限制。 + +#### 约束限制 + +1. page cache超过限制后，page cache会进行定期回收，如果page cache产生的速度大于page cache回收的速度则无法保证page cache的数量在指定的限制之下。 +2. /proc/sys/vm/reliable_pagecache_max_bytes的使用有一定限制，有部分场景的page cache会强制使用可靠内存，如读文件系统的元数据（inode, dentry等），会导致page cache使用的可靠内存超过接口的限制，可以通过 echo 2 \> /proc/sys/vm/drop_caches 来释放inode和dentry。 +3. page cache使用的高可靠内存超过reliable_pagecache_max_bytes限制时，会默认申请低可靠内存，若低可靠内存申请不到，则遵循原生流程处理。 +4. FileCache的统计会先统计到per cpu的缓存中，当缓存中的值超过阈值时才会加到整个系统中，之后才能在/proc/meminfo中体现，ReliableFileCache在/proc/meminfo中没有上述的阈值，会导致有时ReliableFileCache比FileCache的统计值稍大。 +5. 写缓存场景会被dirty_limit限制（由/proc/sys/vm/dirty_ratio限制，代表单个内存节点脏页百分比），超过阈值会跳过当前zone。对于内存分级而言，由于高低可靠内存处于不同的zone，写缓存有可能触发本节点的fallback，使用本节点的低可靠内存。可以通过echo 100 > /proc/sys/vm/dirty_ratio来取消限制。 +6. 读写缓存使用高可靠内存的特性中会限制page cache的使用量，如下几种情况会导致系统性能受影响： + - 如果page cache的上限值限制的过小，会导致IO增加，影响系统性能。 + - 如果page cache 回收的过于频繁，则可能会导致系统卡顿。 + - 如果page cache超过限制后每次回收量过大，则可能导致系统卡顿。 + +#### 使用方法 + +读写缓存使用高可靠内存默认使能，如需关闭，可通过启动项参数设置reliable_debug=P。且page cache不能无限使用，需要限制page cache的使用量。限制page cache量的功能依赖的config开关为CONFIG_SHRINK_PAGECACHE。 + +/proc/meminfo中的FileCache可以用来查询page cache的使用量，ReliableFileCache可以用来查询page cache中可靠内存的使用量。 + +限制page cache量的功能依赖若干proc接口，接口定义在/proc/sys/vm/下，用来控制page cache的使用量，具体如下表： + +| 接口名称（原生/新增） | 权限 | 说明 | 缺省值 | +| ------------------------------------ | ---- | ------------------------------------------------------------ | ------------------------------------------ | +| cache_reclaim_enable（原生） | 644 | 表示page cache限制的功能的使能开关。 **取值范围：**0 或者 1，输入非法值，返回错误。 **示例：**echo 1 > cache_reclaim_enable | 1 | +| cache_limit_mbytes（新增） | 644 | **含义：**表示cache的上限，以M为单位，最小值0（表示关闭限制功能），最大值为meminfo中的MemTotal值（以M为单位换算后）。 **取值范围：**最小值0（表示关闭限制功能），最大值为内存大小（以M为单位，如free –m看到的值）。 **示例:** echo 1024 \> cache_limit_mbytes **其他：**建议cache上限不要低于总内存的一半，否则cache过小可能影响IO性能。 | 0 | +| cache_reclaim_s（原生） | 644 | **含义：**表示定期触发cache回收的时间，以秒为单位系统会根据当前online的cpu个数来创建工作队列，如果有n个cpu则创建n个工作队列，每个工作队列每隔cache_reclaim_s秒进行一次回收。该参数与cpu上下线功能兼容，如果cpu offline，则会减少工作队列个数，cpu online，则会增加。 **取值范围：**最小值0（表示关闭定期回收功能），最大43200，输入非法值，返回错误。 **示例：**echo 120 \> cache_reclaim_s **其他：**建议定期回收时间设成几分钟的级别（如2分钟），否则频繁回收可能导致系统卡顿。 | 0 | +| cache_reclaim_weight（原生） | 644 | **含义：**表示每次回收的权值，内核每个CPU每次期望回收32 \* cache_reclaim_weight个page。该权值同时作用于page上限触发的回收和定期page cache回收。 **取值范围：**最小值1，最大值100，输入非法值，返回错误。 **示例：**echo 10 \> cache_reclaim_weight **其他：**建议设为10或以下，否则每次回收过多内存时，系统可能卡顿。 | 1 | +| reliable_pagecache_max_bytes（新增） | 644 | **含义：**该接口用于控制page cache中高可靠内存的总量。 **取值范围**：0 \~ 高可靠内存最大值，单位为Bytes，高可靠内存的最大值可以通过/proc/meminfo查询，输入非法值返回错误。 **示例:**echo 4096000 \> reliable_pagecache_max_bytes | unsigned long 类型的最大值，代表不限制用量 | + +### tmpfs使用高可靠内存 + +#### 概述 + +对于使用tmpfs做rootfs，rootfs中存放的操作系统使用的核心文件和数据。但是tmpfs默认使用的是低可靠内存，这样会造成核心文件和数据的不可靠。因此需要tmpfs整体使用高可靠内存。 + +#### 使用方法 + +tmpfs使用高可靠内存默认使能，如需关闭，可通过启动项参数设置reliable_debug=S，可以通过/proc/sys/vm/reliable_debug动态关闭，但不能动态打开。 + +在使能tmpfs使用高可靠内存时，可通过/proc/meminfo中ReliableShmem查看当前tmpfs已经使用的高可靠内存。 + +默认tmpfs使用上限是物理内存的一半(rootfs使用tmpfs时除外)。基于传统的SYS V的共享内存，它的使用同时受/proc/sys/kernel/shmmax以及/proc/sys/kernel/shmall的限制，可以动态配置。同时他们也受tmpfs使用高可靠内存的限制。详见下表. + +| **参数** | **说明** | +|---------------------------------|--------------------------------| +| /proc/sys/kernel/shmmax（原生） | SysV共享内存单个段可使用的大小 | +| /proc/sys/kernel/shmall（原生） | SysV共享内存总的可使用的大小 | + +新增接口 /proc/sys/vm/shmem_reliable_bytes_limit 用户设置系统级别 tmpfs 可用的高可靠大小（单位为Byte），缺省值为LONG_MAX，代表用量不受限。可设置的范围为[0, 系统可靠内存总大小]，权限为644。fallback关闭时，在达到该使用上限时，返回没有内存的错误，fallback开启时会尝试从低可靠区域申请。使用示例： + +```Shell +echo 10000000 > /proc/sys/vm/shmem_reliable_bytes_limit +``` + +### 用户态穿越内核UCE不复位 + +#### 概述 + +按照内存可靠性分级的方案，内核以及关键进程使用高可靠内存。大部分用户态进程使用低可靠内存。系统运行时，涉及大量的用户态与内核态数据交换，数据传入内核态时，即发生低可靠内存上数据拷贝到高可靠内存区域。拷贝动作发生在内核态，如果这时在读取用户态数据时发生UCE错误，即发生内核态消费内存UCE，系统会触发panic。本子特性对部分用户态穿越内核UCE场景提供解决方案，避免系统复位，包括COW场景、copy_to_user场景、copy_from_user场景、get_user场景、put_user场景、coredump场景，其余场景均不支持。 + +#### 约束限制 + +1. ARMv8.2及以上版本支持的RAS特性。 +2. 本特性更改的是同步异常处理策略，因此本特性的生效依赖于内核收到Firmware上报的同步异常。 +3. 内核处理依赖BIOS上报的错误类型，不能处理fatal级别硬件错误，可以处理recoverable级别的硬件错误。 +4. 仅支持COW场景、copy_to_user场景（包含读缓存pagecache）、copy_from_user场景、get_user场景、put_user场景、coredump场景六个用户态穿越内核态场景，其余场景不支持。 +5. 在coredump场景中，因为需要在文件系统的write接口上做UCE容错，本特性只支持常用的三个文件系统：ext4、tmpfs、pipefs，对应的容错接口如下： + - pipefs：copy_page_from_iter + - ext4/tmpfs：iov_iter_copy_from_user_atomic + +#### 使用方法 + +确保内核开启config开关CONFIG_ARCH_HAS_COPY_MC，/proc/sys/kernel/machine_check_safe值为1时代表全场景使能，改为0代表不使能，其他值均为非法。 + +当前各场景容错处理机制如下： + +| 序号 | 场景 | 现象 | 应对措施 | +| ---- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | +| 1 | copy_from/to_user：最基本的用户态穿越，主要涉及syscall，sysctl，procfs的操作 | 如果在拷贝时出现UCE异常，会导致内核复位 | 出现 UCE 异常时，kill 当前进程，内核不主动复位 | +| 2 | get/put_user：用于简单的变量拷贝，主要是netlink的场景用的比较多 | 如果在拷贝时出现UCE异常，会导致内核复位 | 出现 UCE 异常时，kill 当前进程，内核不主动复位 | +| 3 | COW：fork子进程，触发写时拷贝 | 触发写时拷贝，如果出现UCE会导致内核复位 | 出现 UCE 异常时，kill 相关进程，内核不主动复位 已实现，见36节 | +| 4 | 读缓存：用户态使用了低可靠内存，用户态程序读写文件时，操作系统会使用空闲内存缓存硬盘文件，提升性能。但用户态程序对文件的读取会先经过内核访问缓存 | 出现UCE异常，会导致内核复位 | 出现 UCE 错误时，kill 当前进程，内核不主动复位 已实现，见36节 | +| 5 | coredump时内存访问触发UCE | 出现UCE异常，会导致内核复位 | 出现 UCE 错误时，kill 当前进程，内核不主动复位 | +| 6 | 写缓存：写缓存回刷到磁盘时，触发UCE | 回刷缓存其实就是磁盘DMA数据的搬移，如果在此过程中触发UCE，超时结束后，页面写失败，这样会造成数据不一致，进而会导致文件系统不可用，如果是关键的数据，会出现内核复位 | 没有解决方案，不支持，内核会发生复位 | +| 7 | 内核启动参数、模块参数使用的都是高可靠内存 | / | 不支持，且本身风险降低 | +| 8 | relayfs：是一个快速的转发数据的文件系统，用于从内核态转发数据到用户态， | / | 不支持，且本身风险降低 | +| 9 | seq_file：将内核数据通过文件的形式传输到用户态 | / | 不支持，且本身风险降低 | + +由于用户态数据大多使用低可靠内存，本项目只涉及内核态读取用户态数据的场景。Linux系统下用户空间与内核空间数据交换有九种方式， 包括内核启动参数、模块参数与 sysfs、sysctl、syscall（系统调用）、netlink、procfs、seq_file、debugfs和relayfs。另有两种情况，进程创建时，COW（copy on write，写时复制）场景，和读写文件缓存（pagecache）场景。 + +其中sysfs，syscall, netlink, procfs 等方式从用户态传输数据到内核都是通过copy_from_user或者get_user的方式。 + +因此用户态穿越到内核有如下几种场景： + +copy_from_user、get_user、COW、读缓存、写缓存回刷。 + +内核态传到用户态有如下几种场景： + +relayfs、seq_file、copy_to_user、put_user。 diff --git "a/archive/LLM/LLM\347\224\250\346\210\267\346\214\207\345\215\227.md" "b/archive/LLM/LLM\347\224\250\346\210\267\346\214\207\345\215\227.md" new file mode 100644 index 0000000000000000000000000000000000000000..86c8d9a978644a24ca2a131981efd0043b8ba36e --- /dev/null +++ "b/archive/LLM/LLM\347\224\250\346\210\267\346\214\207\345\215\227.md" @@ -0,0 +1,5 @@ +# LLM 用户指南 + +本文档介绍openEuler大语言模型（Large Language Model, LLM）的安装、开发等，帮助用户快速了解并使用LLM。LLM是一种人工智能模型，旨在理解和生成人类语言。它们在大量的文本数据上进行训练，可以执行广泛的任务，包括文本总结、翻译、情感分析等等。 + +本文档适用于使用openEuler系统并希望了解和使用LLM的社区开发者、开源爱好者以及相关合作伙伴。使用人员需要具备基本的Linux操作系统知识。 diff --git "a/archive/LLM/chatglm-cpp\344\275\277\347\224\250\346\214\207\345\215\227.md" "b/archive/LLM/chatglm-cpp\344\275\277\347\224\250\346\214\207\345\215\227.md" new file mode 100644 index 0000000000000000000000000000000000000000..659397166377d5527a68d2bea862f8a28e584419 --- /dev/null +++ "b/archive/LLM/chatglm-cpp\344\275\277\347\224\250\346\214\207\345\215\227.md" @@ -0,0 +1,119 @@ +# chatglm-cpp 使用指南 + +## 介绍 + +chatglm-cpp 是基于 C/C++ 实现的 ChatGLM 大模型接口，可以支持用户在CPU机器上完成开源大模型的部署和使用。 + +chatglm-cpp 支持多个中文开源大模型的部署，如 ChatGLM-6B，ChatGLM2-6B ，Baichuan-13B 等。 + +## 软件架构 + +chatglm-cpp 核心架构分为两层 + +- 模型量化层：可以量化开源模型，减少模型大小； +- 模型启动层：可以启动量化后的模型。 + +特性： + +- 基于 ggml 的 C/C++ 实现； +- 通过 int4/int8 量化、优化的 KV 缓存和并行计算等多种方式加速 CPU 推理； +- 互动界面是流媒体生成，具有打字机效果； +- 无需 GPU，可只用 CPU 运行。 + +## 安装教程 + +### 软硬件要求 + +处理器架构：支持 AArch64 和 X86_64 处理器架构； + +操作系统：openEuler 23.09； + +内存：根据不同开源模型的大小，不低于4G。 + +### 安装组件 + +使用 chatglm-cpp 部署大模型，需要安装 chatglm-cpp 软件包。安装前，请确保已经配置了 openEuler yum 源。 + +1. 安装： + +```bash +yum install chatglm-cpp +``` + +1. 查看是否安装成功： + +```bash +chatglm_cpp_main -h +``` + +若成功显示 help 信息则安装成功。 + +## 使用说明 + +### 不使用容器 + +1. 需要安装 chatglm-cpp 软件包： + +```bash +yum install chatglm-cpp +``` + +2. 需要下载开源大模型，如ChatGLM-6B、ChatGLM2-6B等。并将下载的开源大模型通过chatglm_convert.py进行模型量化： + +```bash +python3 /usr/bin/chatglm_convert.py -i model_path/ -t q4_0 -o chatglm-ggml_1.bin +``` + +其中`model_path`为开源大模型的存放路径，q4_0 为开源大模型量化的精度，chatglm-ggml_1.bin 是输出的量化模型的名称。 + +3. 启动模型，进行对话： + +```bash +chatglm_cpp_main -m model_path -i +``` + +其中`model_path`为量化模型的存放路径。 + +可通过以下命令查看命令行选项用法： + +```bash +chatglm_cpp_main -h +``` + +### 使用容器 + +1. 拉取容器镜像： + +```bash +docker pull hub.oepkgs.net/openeuler/chatglm_image +``` + +2. 运行容器镜像，进行对话： + +```bash +docker run -it --security-opt seccomp=unconfined hub.oepkgs.net/openeuler/chatglm_image +``` + +### 正常启动界面 + +模型启动后的界面如图1所示： + +**图1** 模型启动界面 + +![模型启动界面](figures/chatglm.png) + +## 规格说明 + +本项目可支持在 CPU 级别的机器上进行大模型的部署和推理，但是模型推理速度对硬件仍有一定的要求，硬件配置过低可能会导致推理速度过慢，降低使用效率。 + +表1可作为不同机器配置下推理速度的参考： + +表格中 Q4_0，Q4_1，Q5_0，Q5_1 代表模型的量化精度；ms/token 代表模型的推理速度，含义为每个token推理耗费的毫秒数，该值越小推理速度越快； + +**表1** 模型推理速度的测试数据 + +| ChatGLM-6B | Q4_0 | Q4_1 | Q5_0 | Q5_1 | +|--------------------------------|------|------|------|------| +| ms/token (CPU @ Platinum 8260) | 74 | 77 | 86 | 89 | +| 模型大小 | 3.3G | 3.7G | 4.0G | 4.4G | +| 内存占用 | 4.0G | 4.4G | 4.7G | 5.1G | diff --git a/archive/LLM/figures/chatglm.png b/archive/LLM/figures/chatglm.png new file mode 100644 index 0000000000000000000000000000000000000000..bad255b5fd2ade512d291cdd871c6f7f1262560a Binary files /dev/null and b/archive/LLM/figures/chatglm.png differ diff --git a/archive/LLM/figures/llama.png b/archive/LLM/figures/llama.png new file mode 100644 index 0000000000000000000000000000000000000000..184dbf6f0632565477d45077d926f61da1c6ee5a Binary files /dev/null and b/archive/LLM/figures/llama.png differ diff --git "a/archive/LLM/llama.cpp\344\275\277\347\224\250\346\214\207\345\215\227.md" "b/archive/LLM/llama.cpp\344\275\277\347\224\250\346\214\207\345\215\227.md" new file mode 100644 index 0000000000000000000000000000000000000000..edceb58d034fffbef32c31f35da5af737b13556d --- /dev/null +++ "b/archive/LLM/llama.cpp\344\275\277\347\224\250\346\214\207\345\215\227.md" @@ -0,0 +1,119 @@ +# llama.cpp 使用指南 + +## 介绍 + +llama.cpp 是基于 C/C++ 实现的 LLaMa 英文大模型接口，可以支持用户在CPU机器上完成开源大模型的部署和使用。 + +llama.cpp 支持多个英文开源大模型的部署，如LLaMa，LLaMa2，Vicuna等。 + +## 软件架构 + +llama.cpp 核心架构分为两层： + +- 模型量化层：可以量化开源模型，减少模型大小； +- 模型启动层：可以启动量化后的模型。 + +特性： + +- 基于 ggml的C/C++ 实现 +- 通过 int4/int8 量化、优化的KV缓存和并行计算等多种方式加速 CPU 推理； +- 互动界面是流媒体生成，具有打字机效果； +- 无需 GPU，可只用 CPU 运行。 + +## 安装教程 + +### 软硬件要求 + +处理器架构：支持 AArch64 和 X86_64 处理器架构； + +操作系统：openEuler 23.09； + +内存：根据不同开源模型的大小，不低于 4G 。 + +### 安装组件 + +使用llama.cpp部署大模型，需要安装 llama.cpp 软件包。安装前，请确保已经配置了 openEuler yum 源。 + +1. 安装： + +```bash +yum install llama.cpp +``` + +2. 查看是否安装成功： + +```bash +llama_cpp_main -h +``` + +若成功显示 help 信息则安装成功。 + +## 使用说明 + +### 不使用容器 + +1. 需要安装 llama.cpp 软件包： + +```bash +yum install llama.cpp +``` + +2. 需要下载开源大模型，如LLaMa、LLaMa2等。并将下载的开源大模型通过 llama_convert.py 进行模型量化： + +```bash +python3 /usr/bin/llama_convert.py model_path/ +``` + +其中`model_path`为开源大模型的存放路径。 + +3. 启动模型，进行对话： + +```bash +llama_cpp_main -m model_path --color --ctx_size 2048 -n -1 -ins -b 256 --top_k 10000 --temp 0.2 --repeat_penalty 1.1 -t 8 +``` + +其中`model_path`为量化模型的存放路径。 + +可通过以下命令查看命令行选项用法： + +```bash +llama_cpp_main -h +``` + +### 使用容器 + +1. 拉取容器镜像： + +```bash +docker pull hub.oepkgs.net/openeuler/llama_image +``` + +2. 运行容器镜像，进行对话： + +```bash +docker run -it --security-opt seccomp=unconfined hub.oepkgs.net/openeuler/llama_image +``` + +### 正常启动界面 + +模型启动后的界面如图1所示： + +**图1** 模型启动界面 + +![模型启动界面](figures/llama.png) + +## 规格说明 + +本项目可支持在CPU级别的机器上进行大模型的部署和推理，但是模型推理速度对硬件仍有一定的要求，硬件配置过低可能会导致推理速度过慢，降低使用效率。 + +表1可作为不同机器配置下推理速度的参考： + +表格中 Q4_0，Q4_1，Q5_0，Q5_1 代表模型的量化精度；ms/token 代表模型的推理速度，含义为每个token推理耗费的毫秒数，该值越小推理速度越快； + +**表1** 模型推理速度的测试数据 + +| LLaMa-7B | Q4_0 | Q4_1 | Q5_0 | Q5_1 | +|--------------------------------|------|------|------|------| +| ms/token (CPU @ Platinum 8260) | 55 | 54 | 76 | 83 | +| 模型大小 | 3.5G | 3.9G | 4.3G | 6.7G | +| 内存占用 | 3.9G | 4.2G | 4.5G | 5.0G | diff --git a/archive/Open-Source-Software-Notice/openEuler-Open-Source-Software-Notice.zip b/archive/Open-Source-Software-Notice/openEuler-Open-Source-Software-Notice.zip new file mode 100644 index 0000000000000000000000000000000000000000..9828e4c70f2d1a6465c6fe2e8c6f7966145b2aae Binary files /dev/null and b/archive/Open-Source-Software-Notice/openEuler-Open-Source-Software-Notice.zip differ diff --git a/archive/container.md b/archive/container.md new file mode 100644 index 0000000000000000000000000000000000000000..04cb5efa28685dad8df3747659a0ed2b27144e5b --- /dev/null +++ b/archive/container.md @@ -0,0 +1,18 @@ +# 概述 + +openEuler软件包中同时提供了轻量化容器引擎iSulad与docker engine两种容器引擎。 + +同时根据不同使用场景，提供多种容器形态，包括： + +- 适合大部分通用场景的普通容器 +- 适合强隔离与多租户场景的安全容器 +- 适合使用systemd管理容器内业务场景的系统容器 + +本文档提供容器引擎的安装和使用方法以及各个容器形态的部署使用方法。 + +## 读者对象 + +本文档主要适用于使用openEuler并需要安装容器的用户。用户需要具备以下经验和技能： + +- 熟悉Linux基本操作 +- 对容器有一定了解 diff --git a/archive/public_sys-resources/icon-caution.gif b/archive/public_sys-resources/icon-caution.gif new file mode 100644 index 0000000000000000000000000000000000000000..6e90d7cfc2193e39e10bb58c38d01a23f045d571 Binary files /dev/null and b/archive/public_sys-resources/icon-caution.gif differ diff --git a/archive/public_sys-resources/icon-danger.gif b/archive/public_sys-resources/icon-danger.gif new file mode 100644 index 0000000000000000000000000000000000000000..6e90d7cfc2193e39e10bb58c38d01a23f045d571 Binary files /dev/null and b/archive/public_sys-resources/icon-danger.gif differ diff --git a/archive/public_sys-resources/icon-note.gif b/archive/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/archive/public_sys-resources/icon-note.gif differ diff --git a/archive/public_sys-resources/icon-notice.gif b/archive/public_sys-resources/icon-notice.gif new file mode 100644 index 0000000000000000000000000000000000000000..86024f61b691400bea99e5b1f506d9d9aef36e27 Binary files /dev/null and b/archive/public_sys-resources/icon-notice.gif differ diff --git a/archive/public_sys-resources/icon-tip.gif b/archive/public_sys-resources/icon-tip.gif new file mode 100644 index 0000000000000000000000000000000000000000..93aa72053b510e456b149f36a0972703ea9999b7 Binary files /dev/null and b/archive/public_sys-resources/icon-tip.gif differ diff --git a/archive/public_sys-resources/icon-warning.gif b/archive/public_sys-resources/icon-warning.gif new file mode 100644 index 0000000000000000000000000000000000000000..6e90d7cfc2193e39e10bb58c38d01a23f045d571 Binary files /dev/null and b/archive/public_sys-resources/icon-warning.gif differ diff --git a/archive/uncertain/Container/public_sys-resources/icon-caution.gif b/archive/uncertain/Container/public_sys-resources/icon-caution.gif new file mode 100644 index 0000000000000000000000000000000000000000..6e90d7cfc2193e39e10bb58c38d01a23f045d571 Binary files /dev/null and b/archive/uncertain/Container/public_sys-resources/icon-caution.gif differ diff --git a/archive/uncertain/Container/public_sys-resources/icon-danger.gif b/archive/uncertain/Container/public_sys-resources/icon-danger.gif new file mode 100644 index 0000000000000000000000000000000000000000..6e90d7cfc2193e39e10bb58c38d01a23f045d571 Binary files /dev/null and b/archive/uncertain/Container/public_sys-resources/icon-danger.gif differ diff --git a/archive/uncertain/Container/public_sys-resources/icon-note.gif b/archive/uncertain/Container/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/archive/uncertain/Container/public_sys-resources/icon-note.gif differ diff --git a/archive/uncertain/Container/public_sys-resources/icon-notice.gif b/archive/uncertain/Container/public_sys-resources/icon-notice.gif new file mode 100644 index 0000000000000000000000000000000000000000..86024f61b691400bea99e5b1f506d9d9aef36e27 Binary files /dev/null and b/archive/uncertain/Container/public_sys-resources/icon-notice.gif differ diff --git a/archive/uncertain/Container/public_sys-resources/icon-tip.gif b/archive/uncertain/Container/public_sys-resources/icon-tip.gif new file mode 100644 index 0000000000000000000000000000000000000000..93aa72053b510e456b149f36a0972703ea9999b7 Binary files /dev/null and b/archive/uncertain/Container/public_sys-resources/icon-tip.gif differ diff --git a/archive/uncertain/Container/public_sys-resources/icon-warning.gif b/archive/uncertain/Container/public_sys-resources/icon-warning.gif new file mode 100644 index 0000000000000000000000000000000000000000..6e90d7cfc2193e39e10bb58c38d01a23f045d571 Binary files /dev/null and b/archive/uncertain/Container/public_sys-resources/icon-warning.gif differ diff --git "a/archive/uncertain/HCK/HCK\347\211\271\346\200\247\345\217\212\344\275\277\347\224\250\346\211\213\345\206\214.md" "b/archive/uncertain/HCK/HCK\347\211\271\346\200\247\345\217\212\344\275\277\347\224\250\346\211\213\345\206\214.md" new file mode 100644 index 0000000000000000000000000000000000000000..22450003dcb75042fc12e3ac1f7089f1e911d847 --- /dev/null +++ "b/archive/uncertain/HCK/HCK\347\211\271\346\200\247\345\217\212\344\275\277\347\224\250\346\211\213\345\206\214.md" @@ -0,0 +1,111 @@ +# 特性说明 +HCK(High-performance Computing Kit)是HPC应用的软件底座，是在通用Linux平台上通过定制系统调用，调度优化等手段，为HPC应用提供隔离/低底噪的运行环境，同时兼容Linux生态，达到提高HPC应用性能的目的。在构建内核时需要将选项`CONFIG_PURPOSE_BUILT_KERNEL`开启来使能HCK特性。通过在系统启动阶段隔离出一部分CPU用来运行HPC应用，内核线程和其他用户进程则运行在非隔离核上。 + +# 使用说明 +在内核支持HCK特性前提下，需要在内核启动阶段增加启动参数`pbk_cpus`设置预留的CPU资源，默认不预留。可以通过修改grub的配置文件`grub.cfg`或grub的启动界面设置此参数。 +启动参数的格式为： pbk_cpus=，cpulist支持`","`和`"-"`连接，例如`1,3-4`表示预留CPU1/CPU3和CPU4。HCK特性暂不支持x86架构，需要注意的是，在aarch64架构下不支持预留0核。 +当配置好启动参数后，内核会自动完成CPU资源的隔离预留，系统启动后不需要额外的部署步骤。 + +# 用户态launcher工具使用 + +HCK特性有一个用户态的launcher工具，通过其可以指定程序在特定的隔离核上运行，该工具需要单独安装。 + +## 基本选项 + +参见 `launcher -?`： +``` +Usage: launcher [OPTION...] + +launcher: launch process to pbk domain + +EXAMPLES: + launcher -c 1,2 prog # alloc CPU 1,2 from pbk root domain to run prog + launcher -n 2 prog # alloc 2 CPUs from pbk root domain to run prog + launcher -v prog # prog will only touch pbk + launcher -c 1 prog # alloc CPU 1 and 1M memory + + -c, --cpulist=CPULIST cpulist to run prog + -n, --nr_cpu=NR_CPU nr_cpus to run prog + -v, --pbk_view run prog with pbk view + -?, --help Give this help list + --usage Give a short usage message + +Mandatory or optional arguments to long options are also mandatory or optional +for any corresponding short options. +``` +## 选项详解 + +### launcher -？ + +显示该命令的使用说明。 + +### launcher -c + +指定`cpulist`运行程序，例如： + +``` +launcher -c 1,2 ./test +``` + +launcher将会申请CPU1和CPU2用于执行test程序，如果CPU1和CPU2不是隔离核或者已经被其他程序申请，那么返回失败。 + +存在一种例外情况，例如执行以下操作： + +``` +launcher -c 1,2 ./test1 +launcher -c 1,2 ./test2 +``` + +执行test2时，指定了相同的`cpulist`，此时不会返回失败，而是将test2也运行到CPU1和CPU2上。 + +### launcher -n + +指定CPU数量运行程序，例如 + +``` +launcher -n 1 ./test +``` + +launcher将会从隔离核中申请一个CPU来运行test程序，如果剩余的隔离核不足，则返回失败。 + +### launcher -v + +在CPU隔离后，CPU资源划分为Linux资源和隔离核资源。为了适配mpirun等具备资源识别分配功能的程序，在launcher中提供修改资源视角的功能，例如： + +``` +launcher -v cat /proc/cpuinfo +``` + +此时只会显示隔离核而非所有的CPU。 + +此选项还可以结合`-n`和`-c`选项使用，例如 + +``` +launcher -c 1,2 -v cat /proc/cpuinfo +``` + +此时只会显示CPU1和CPU2的信息。 + +受影响的procfs和sysfs的接口如下： + +procfs: + +``` +/proc/cpuinfo +``` + +sysfs: + +``` +/sys/devices/system/cpu/online +/sys/devices/system/cpu/present +/sys/devices/system/cpu/possible +/sys/devices/system/cpu/cpu/online +/sys/devices/system/node/node/cpumap +/sys/devices/system/node/node/cpulist +/sys/fs/cgroup/cpuset/cpuset.cpus +/sys/fs/cgroup/cpuset/cpuset.effective_cpus +``` + +注：除了cgroup相关的两个接口默认仅显示Linux资源外（这是因为隔离核没有加入到cgroup的cpuset中），其他接口默认都会显示Linux资源+隔离核资源，不受隔离影响。 + diff --git "a/archive/uncertain/Memory-fabric/Memory-Fabric\347\224\250\346\210\267\346\214\207\345\215\227.md" "b/archive/uncertain/Memory-fabric/Memory-Fabric\347\224\250\346\210\267\346\214\207\345\215\227.md" new file mode 100644 index 0000000000000000000000000000000000000000..71945d53012aedf602ac4844613afe56a7840dee --- /dev/null +++ "b/archive/uncertain/Memory-fabric/Memory-Fabric\347\224\250\346\210\267\346\214\207\345\215\227.md" @@ -0,0 +1,96 @@ +# Memory Fabric用户文档 + +- [简介](简介.md) +- [部署](部署.md) +- [启动](启动.md) +- [接口](接口.md) + +# 简介 + +内存池套件是基于内存型介质和新型网络技术，构建高性能的分布式内存资源池，并通过BigMemory、MemPlog、MemKV等语义对应用提供服务。 + +内存池套件使用和管理的资源来自计算节点或者单独资源节点提供的可共享的DRAM/Persistent Memory/内存盘等（不支持普通disk直接互联池化），结合新型互联协议（RDMA，CXL，UBUS）us级传输时延，期望内存池套件通过极低的软件时延和节点CPU开销带来整体性能提升。 + +# 部署 + +Memory Fabric需要根据节点类型、资源分布情况和应用需求做到多场景应用集成部署，具体描述如下： + +- 计算和MF资源共节点时支持client和server同节点部署，如[图1](#fig17349154610267)node1、node2所示。 +- 资源独立提供时也支持client、server分离节点部署，如[图1](#fig17349154610267)node3、node4所示。 +- 支持同节点部署和分离节点部署两种场景混合部署。 +- 节点内支持多client，也支持多server模式部署，如[图1](#fig17349154610267)node2所示。 +- client与同节点的server通信使用IPC，配置支持连接远端server时使用RPC通信。 +- server端接入集群管理服务中，用于统一的节点编号和资源信息广播。 + +**图 1** 集成部署 +![](images/IntegratedDeployment.png) + +# 启动 + +Memory Fabric启动分为server和client两个部分。在节点上优先启动server端，完成集群注册、监控流程，然后启动本地资源注册\(总容量通过配置文件获取\)和通信建链流程，初始化完成后支持本地多client与server建立连接，可对外启动业务服务。 + +# 接口 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

接口

+

说明

+

int BM_Init(char *ockPath, int flags);

+

int BM_InitWithExtAttr(char*ockPath, int flags, void *attr); int BM_Exit(void);

+

初始化

+

带属性的初始化

+

退出

+

int BM_CreateNSSec(const char*poolName, const char *ns, int attribute, const char*secId);

+

int BM_DestroyNSSec(const char *poolName, const char*ns, const char *secId);

+

int BM_QueryNSCache(QueryNsCache*buff, size_t buffLen);

+

Namespace创建和销毁、查询

+

int BM_AllocSec(BmInfo *bminfo, size_t size, int flags, BmAffinity* affinity);

+

int BM_FreeSec(BmInfo *bminfo, int flags);

+

int BM_BatchFreeSec(char*bmIdArray[], int num, int flags, const char *ns, const char*secId);

+

Bigmemory对象申请和释放

+

int BM_ExpandSec(BmInfo *bminfo, size_t size, int flags);

+

bigMemory扩容

+

int BM_MapSec(BmInfo*bminfo, int prot, int flags, void **ptr);

+

int BM_Unmap(char *bmId, void*ptr);

+

bigMemory到连续虚拟空间map和unmap

+

int BM_CopySec(BmInfo *srcBM, size_t srcOffset, BmInfo*dstBM, size_t dstOffset, size_t length, int flags);

+

bigMemory对象间拷贝

+

int BM_SpecificObjSwapInSec(DiskFileDesc *file, BmInfo*bminfo, int flags, size_t offset, size_t length);

+

文件内容换入bigmemory对象

+

int BM_ReadSec(BmInfo *bminfo, size_t offset, size_t length, void*buffer, int flags);

+

int BM_WriteSec(BmInfo *bminfo, size_t offset, size_t length, const void*buffer, int flags);

+

Bigmemory读写

+

int BM_GatherSec(intptr_t *inputInfo, BmOperatorCode operatorCode, int flags, BmInfo*bminfo, size_t *bmLen);

+

聚合操作接口

+
diff --git a/archive/uncertain/Memory-fabric/images/.keep b/archive/uncertain/Memory-fabric/images/.keep new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/archive/uncertain/Memory-fabric/images/IntegratedDeployment.png b/archive/uncertain/Memory-fabric/images/IntegratedDeployment.png new file mode 100644 index 0000000000000000000000000000000000000000..0fc7d1b0a3d3cf31a2da0bff64bf03f576234771 Binary files /dev/null and b/archive/uncertain/Memory-fabric/images/IntegratedDeployment.png differ diff --git "a/archive/uncertain/NfsMultipath/NFS\345\244\232\350\267\257\345\276\204.md" "b/archive/uncertain/NfsMultipath/NFS\345\244\232\350\267\257\345\276\204.md" new file mode 100644 index 0000000000000000000000000000000000000000..a6deafffe98fb1c4400b3a97ed82ddccf8607146 --- /dev/null +++ "b/archive/uncertain/NfsMultipath/NFS\345\244\232\350\267\257\345\276\204.md" @@ -0,0 +1,6 @@ + +# NFS多路径用户指南 + +本文档介绍NFS客户端多路径特性的安装部署与使用方法，以指导用户快速了解并使用NFS多路径。 + +本文档适用于使用openEuler系统并希望了解和使用NFS多路径特性的社区开发者、开源爱好者以及相关合作伙伴。使用人员需要具备基本的NFS知识。 diff --git "a/archive/uncertain/NfsMultipath/\344\275\277\347\224\250\346\226\271\346\263\225.md" "b/archive/uncertain/NfsMultipath/\344\275\277\347\224\250\346\226\271\346\263\225.md" new file mode 100644 index 0000000000000000000000000000000000000000..9ec27431e135e35ec2cb5c21f95179d665d6b435 --- /dev/null +++ "b/archive/uncertain/NfsMultipath/\344\275\277\347\224\250\346\226\271\346\263\225.md" @@ -0,0 +1,39 @@ +# 使用方法 + +## 使用多路径参数挂载 + +```shell +多路径挂载命令 +mount -t nfs -o localaddrs=1.1.1.1~2.2.2.2,remoteaddrs=1.1.1.10~2.2.2.10 1.1.1.1:/test /mnt/fs1 + +新增可选参数localaddrs和remoteaddrs用于指定多路径： +localaddrs表示客户端IP地址； +remoteaddrs表示服务端IP地址； +IP地址支持网段或者多个IP地址，网段使用"-",多个IP地址使用"~". +例如：remoteaddrs=1.1.1.1-1.1.1.3表示指定3个服务端地址，分别为1.1.1.1, 1.1.1.2, 1.1.1.3； +localaddrs=1.1.1.1~1.1.1.3表示指定2个客户端地址，分别为1.1.1.1, 1.1.1.3. + +localaddrs和remoteaddrs均为可选参数，可指定其中的任意一个参数或者两个参数都指定，若都不指定，则未使能多路径功能。 +``` + +## 多路径信息查询 + +```shell +查询多路径信息命令 +cat /proc/fs/nfs/mutipath/conn_info +回显如下所示： +===============================Id:4 count 5 proto 3 start======================== +index:0, client_ip:1.1.1.1, server_ip:1.1.1.10, status:connect, load:3 +index:1, client_ip:1.1.1.1, server_ip:1.1.1.11, status:connect, load:3 +index:2, client_ip:1.1.1.1, server_ip:1.1.1.12, status:connect, load:3 +index:3, client_ip:1.1.1.1, server_ip:1.1.1.13, status:disconnect, load:0 +index:4, client_ip:1.1.1.1, server_ip:1.1.1.14, status:disconnect, load:0 +=========================================Id:4 end================================ +status表示该条链路连接状态，connect表示连接可用，disconnect表示连接不可用 +load表示在该条链路上发送的NFS请求数量。 +``` + +## 注意事项 + +* 客户端或者服务端IP地址最多支持8个IP地址，指定超过8个IP地址无法挂载。 +* 最多支持8条多路径链路。 diff --git "a/archive/uncertain/NfsMultipath/\345\256\211\350\243\205\344\270\216\351\203\250\347\275\262.md" "b/archive/uncertain/NfsMultipath/\345\256\211\350\243\205\344\270\216\351\203\250\347\275\262.md" new file mode 100644 index 0000000000000000000000000000000000000000..c20e95ff44c550b3aa02b538664f3586f7594cc3 --- /dev/null +++ "b/archive/uncertain/NfsMultipath/\345\256\211\350\243\205\344\270\216\351\203\250\347\275\262.md" @@ -0,0 +1,28 @@ +# 安装与部署 + +## 软件要求 + +* 操作系统：openEuler 23.03 + +## 硬件要求 + +* x86_64架构 +* ARM架构 + +## 环境准备 + +*安装openEuler系统，安装方法参考 《[安装指南](../Installation/installation.md)》。 + +## 安装NFS多路径 + +```shell +插入NFS多路径KO，参考命令如下： +modprobe nfs_multipath + +查看NFS多路径KO，参考命令如下： +lsmod | grep nfs_multipath +nfs_multipath 36864 1 +nfs 544768 3 nfs_multipath,nfsv3 +sunrpc 704512 30 nfs_multipath,lockd,nfsv3,nfs_acl,nfs +存在nfs_multipath则表示安装成功 +``` diff --git "a/archive/uncertain/NfsMultipath/\345\270\270\350\247\201\351\227\256\351\242\230\344\270\216\350\247\243\345\206\263\345\212\236\346\263\225.md" "b/archive/uncertain/NfsMultipath/\345\270\270\350\247\201\351\227\256\351\242\230\344\270\216\350\247\243\345\206\263\345\212\236\346\263\225.md" new file mode 100644 index 0000000000000000000000000000000000000000..ed22eafb95d826fd2ada7f3dc7d61596022946e6 --- /dev/null +++ "b/archive/uncertain/NfsMultipath/\345\270\270\350\247\201\351\227\256\351\242\230\344\270\216\350\247\243\345\206\263\345\212\236\346\263\225.md" @@ -0,0 +1,26 @@ +# 常见问题与解决办法 + +## 问题1：指定NFS多路径挂载失败，lsmod无法查询到多路径KO + +```shell +报错信息如下： +mount.nfs: an incorrect mount option was specified +执行lsmod | grep nfs_multipath无法查询到多路径KO。 + +可能原因：未安装NFS多路径。 + +解决方案：参考《安装与部署》章节安装NFS多路径。 +``` + +## 问题2：指定NFS多路径挂载失败 + +```shell +界面报错信息如下： +mount.nfs: an incorrect mount option was specified +/var/log/messages日志报错信息: +Mar 18 23:38:22 localhost kernel: [ 635.016793] [MULTIPATH:nfs_multipath_parse_ip_range] iplist for remote reached 8, more than supported limit 8 + +原因：挂载时指定IP地址个数超过8个。 + +解决方案：NFS挂载时，指定IP地址个数小于等于8个。 +``` diff --git "a/archive/uncertain/NfsMultipath/\350\256\244\350\257\206NFS\345\244\232\350\267\257\345\276\204.md" "b/archive/uncertain/NfsMultipath/\350\256\244\350\257\206NFS\345\244\232\350\267\257\345\276\204.md" new file mode 100644 index 0000000000000000000000000000000000000000..e2048d54e555ea1662ede08a77c82c391f5d5260 --- /dev/null +++ "b/archive/uncertain/NfsMultipath/\350\256\244\350\257\206NFS\345\244\232\350\267\257\345\276\204.md" @@ -0,0 +1,18 @@ +# 认识NFS多路径 + +## 简介 + +网络文件系统 (NFS) 是一种分布式文件系统协议，最初由 Sun Microsystems (Sun) 于 1984 年开发，允许NFS客户端上的用户通过计算机网络访问NFS服务端上文件。随着NFS服务广泛应用于金融，EDA，AI，容器等行业，对NFS的性能和可靠性提出了更高的诉求。传统NFS存在以下缺点： + +* 单个主机上的单个挂载点只能通过一个客户端IP和一个服务端IP进行访问，在客户端和服务端之间存在多条物理链路时，无法发挥多条链路的性能。 +* 由于单个挂载点单条链路的缺陷，单挂载点下的单条链路故障后，无法进行链路切换，导致主机业务中断。 + +NFS多路径的诞生主要解决上述传统NFS在使用过程中遇到的缺陷，提出单个挂载点下客户端和服务端之间建立多条链路，支持IO在多条链路中进行传输，进而提升单个挂载点性能，同时周期性检测链路状态信息，支持链路故障IO快速切换。 + +NFS多路径具有以下功能： + +* NFSv3支持Round Robin链路选路算法，实现多条链路性能均衡。 +* NFSv3和NFSv4支持链路故障快速切换，提升NFS可靠性。 +* 提供链路选路算法注册接口，支持NFS服务端开发者自定义选路算法。 +* 支持周期性链路可用性检测。 +* 支持查看链路状态。 diff --git a/archive/uncertain/Ods-Pipeline/image/.keep b/archive/uncertain/Ods-Pipeline/image/.keep new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/archive/uncertain/Ods-Pipeline/image/build_stage.png b/archive/uncertain/Ods-Pipeline/image/build_stage.png new file mode 100644 index 0000000000000000000000000000000000000000..ecc0a751b9bd114f62b47e8a590452457f79e397 Binary files /dev/null and b/archive/uncertain/Ods-Pipeline/image/build_stage.png differ diff --git a/archive/uncertain/Ods-Pipeline/image/pipeline.png b/archive/uncertain/Ods-Pipeline/image/pipeline.png new file mode 100644 index 0000000000000000000000000000000000000000..2ade49082715c53365cb7fccbde09f0c7e8cf3a7 Binary files /dev/null and b/archive/uncertain/Ods-Pipeline/image/pipeline.png differ diff --git a/archive/uncertain/Ods-Pipeline/image/post_pipeline.png b/archive/uncertain/Ods-Pipeline/image/post_pipeline.png new file mode 100644 index 0000000000000000000000000000000000000000..df2d56b25448b9a01419d93be8f5233876f8ba7c Binary files /dev/null and b/archive/uncertain/Ods-Pipeline/image/post_pipeline.png differ diff --git a/archive/uncertain/Ods-Pipeline/image/post_submit.png b/archive/uncertain/Ods-Pipeline/image/post_submit.png new file mode 100644 index 0000000000000000000000000000000000000000..ff3e6f66cfde597a4625d1ac37022c663330ef4b Binary files /dev/null and b/archive/uncertain/Ods-Pipeline/image/post_submit.png differ diff --git a/archive/uncertain/Ods-Pipeline/image/post_workflow.png b/archive/uncertain/Ods-Pipeline/image/post_workflow.png new file mode 100644 index 0000000000000000000000000000000000000000..8447a2a388ce67c8876285120b4ac90b75ca47a2 Binary files /dev/null and b/archive/uncertain/Ods-Pipeline/image/post_workflow.png differ diff --git a/archive/uncertain/Ods-Pipeline/image/pre_deploy_stage.png b/archive/uncertain/Ods-Pipeline/image/pre_deploy_stage.png new file mode 100644 index 0000000000000000000000000000000000000000..35fea094ba62775d5b80d980d4bc1946e16dd700 Binary files /dev/null and b/archive/uncertain/Ods-Pipeline/image/pre_deploy_stage.png differ diff --git a/archive/uncertain/Ods-Pipeline/image/snapshot.png b/archive/uncertain/Ods-Pipeline/image/snapshot.png new file mode 100644 index 0000000000000000000000000000000000000000..27f8a9d652559dff6a005b56e0d7ec07c0f1be47 Binary files /dev/null and b/archive/uncertain/Ods-Pipeline/image/snapshot.png differ diff --git a/archive/uncertain/Ods-Pipeline/image/test_stage.png b/archive/uncertain/Ods-Pipeline/image/test_stage.png new file mode 100644 index 0000000000000000000000000000000000000000..c94bf4fc8542ec6f5dc6f7c1e4b08f3a6f60259f Binary files /dev/null and b/archive/uncertain/Ods-Pipeline/image/test_stage.png differ diff --git a/archive/uncertain/Ods-Pipeline/image/yaml2jobs.png b/archive/uncertain/Ods-Pipeline/image/yaml2jobs.png new file mode 100644 index 0000000000000000000000000000000000000000..6456d49c0215df0388006d50745a9c33a47ce8c1 Binary files /dev/null and b/archive/uncertain/Ods-Pipeline/image/yaml2jobs.png differ diff --git "a/archive/uncertain/Ods-Pipeline/ods pipeline\347\224\250\346\210\267\346\214\207\345\215\227.md" "b/archive/uncertain/Ods-Pipeline/ods pipeline\347\224\250\346\210\267\346\214\207\345\215\227.md" new file mode 100644 index 0000000000000000000000000000000000000000..7fc2747eb7a17fe63e73d75497e28008a8a5b95b --- /dev/null +++ "b/archive/uncertain/Ods-Pipeline/ods pipeline\347\224\250\346\210\267\346\214\207\345\215\227.md" @@ -0,0 +1,161 @@ +# ods pipeline用户指南 + +## 前言 + +### 概述 + +本文档介绍了ods的pipeline,workflow定义，编写规则，功能说明，使用的方法，以及日志结果查询。 + +### 使用对象 + +- 社区开发者 +- 版本Maintainer +- 性能优化工程师 + +### 术语说明 + +| 名称 | 说明 | +| :------: | :----------------------------------------------------------: | +| DevOps | 是指在一组开发者服务的集合，开发者可以直接使用服务，也可以对服务进行编排。 | +| pipeline | 对于编排好的一系列任务，可被系统执行的语法yaml 文件。 | +| workflow | 为完成一组功能的工作流(如：构建为repo、生成ISO等)，是一个编排好的jobs，可以是被pipeline 引用。 | +| job | 在某个执行机器上具体指定的一组执行接口。 | + +### 修订记录 + +| 文档版本 | 发布日期 | 修改说明 | +| -------- | ---------- | -------- | +| 01 | 2023-06-28 | | + +## 工具概述 + +ods是基于pipeline执行流水线测试任务并输出结果的工具，支持配置化管理pipeline以及workflow，灵活组合workflow，降低人工参与成本。解决任务流中繁琐人工操作，效率低的问题。 + +ods主要提供以下功能: + +- pipeline与workflow管理 + +对用户上传的pipeline.yaml或者workflow.yaml进行管理,并能生成pipeline的快照。 + +- 运行pipeline + +用户可运行指定的pipeline,并生成日志。 + +### 应用场景 + +- 操作系统串行测试 ：构建任务完成后 ，测试在某台机器上完成 + +``` +-提交构建repo任务 +-生成镜像(基于多个repo，生成ISO、image) +-iso做成compass-ci做成compass-ci 支持的rootfs +-提交串行测试任务 +-汇总测试结果 +``` + +- 操作系统并行测试 ：构建任务完成后 ，测试在不同的台机器上完成 + +``` +-构建repo +-生成镜像(基于多个repo，生成ISO、image) +-iso做成compass-ci做成compass-ci 支持的rootfs +-执行并行测试任务 +-汇总测试结果 +``` + +- 操作系统包有更新 ：操作系统本身不变，基于repo 更新rpm，开展测试 + +``` +-构建rpm repo +-加载对应系统的rootfs +-执行测试任务 +-汇总测试结果 +``` + +- 场景测试(mysql, ceph, spark => 构建+部署+冒烟测试) + + +``` +- 构建软件包 +- 加载对应系统的rootfs +- 执行测试任务 +- 输出测试报告 +``` + +### 使用介绍 + +本示例中，下列字段均需根据实际部署情况自行配置。部分字段上文提供了生成方法： + +${HOST_IP}：ods服务宿主机IP + +${ODS_MANAGER_PORT}：ods管理服务对应的端口 + +${ODS_RUNNER_PORT}：ods任务运行服务对应的端口 + +${PIPELINE_NAME}: pipeline的名称 + +${WORKFLOW_NAME}: workflow的名称 + +${SUBMIT_USER}: 提交的task的用户名 + +步骤1:编写并提交workflow.yaml +以下图为例: + +![build_stage](./image/build_stage.png) + +![pre_deploy_stage](./image/pre_deploy_stage.png) + +![test_stage](./image/test_stage.png) + +分别编写了名为build_stage，pre_test_stage，test_stage的workflow。填写内容不限于图中所示，具体与compass-ci处理job参数保持一致。 + +调用接口分别提交: + +![post_workflow](./image/post_workflow.png) + + + +步骤2:编写并提交pipeline.yaml + +以下图为例: + +![pipeline](./image/pipeline.png) + +编写了名为test的pipeline。填写内容不限于图中所示。 + +调用接口提交: + +![post_pipeline](./image/post_pipeline.png) + + + +说明：pipeline提交后，会根据其下包含的workflow名称去寻找对应workflow信息，进行扩展，生成snapshot，snapshot仅可通过更新pipeline的方式更新，解析job的来源是snapshot。 + +snapshot示例： + +![snapshot](./image/snapshot.png) + + + +步骤3:提交执行pipeline + +调用接口执行： + +![post_submit](./image/post_submit.png) + +说明: + +1.对于pipeline下的matrix入参，会做笛卡尔积组合，生成多个task + +2.对于单个job的依赖项，其值为workflow.need + job.depends + job.input去重后的合集，若对应的依赖找不到，跳过该依赖，以上图为例，最终job依赖项见下图: + +![yaml2jobs](./image/yaml2jobs.png) + +3.单个job是否提交，会查看其依赖项是否全部满足，否则不会提交执行。 + + + +步骤4:查看执行结果 + +单个job的执行结果在${HOST_IP}的/result目录下，查找对应的测试套名称，具体日志看其下文件。 + diff --git a/archive/uncertain/Open-Source-Software-Notice/openEuler-Open-Source-Software-Notice.zip b/archive/uncertain/Open-Source-Software-Notice/openEuler-Open-Source-Software-Notice.zip new file mode 100644 index 0000000000000000000000000000000000000000..ab74b7f0aa515f45d656bca9e018d47f7159c3d2 Binary files /dev/null and b/archive/uncertain/Open-Source-Software-Notice/openEuler-Open-Source-Software-Notice.zip differ diff --git "a/archive/uncertain/ROS/ROS\347\224\250\346\210\267\346\214\207\345\215\227.md" "b/archive/uncertain/ROS/ROS\347\224\250\346\210\267\346\214\207\345\215\227.md" new file mode 100644 index 0000000000000000000000000000000000000000..9eb573cd3c915ba1e0c695344c9efb288af3a0a2 --- /dev/null +++ "b/archive/uncertain/ROS/ROS\347\224\250\346\210\267\346\214\207\345\215\227.md" @@ -0,0 +1,5 @@ +# ROS用户指南 + +本文档介绍openEuler系统上ROS（英语：Robot Operating System，一般译为机器人操作系统）的安装部署与使用方法，以指导用户快速了解并使用ROS。 + +本文档适用于使用openEuler系统并希望了解和使用ROS的社区开发者、开源爱好者以及相关合作伙伴。使用人员需要具备基本的Linux操作系统知识。 diff --git a/archive/uncertain/ROS/figures/ROS-ROS2.png b/archive/uncertain/ROS/figures/ROS-ROS2.png new file mode 100644 index 0000000000000000000000000000000000000000..649c0aa93b0a3710f027ecf9df2482920f16301e Binary files /dev/null and b/archive/uncertain/ROS/figures/ROS-ROS2.png differ diff --git a/archive/uncertain/ROS/figures/ROS-demo.png b/archive/uncertain/ROS/figures/ROS-demo.png new file mode 100644 index 0000000000000000000000000000000000000000..184ae905d022e52adbac7fcee59d956903e1ff5c Binary files /dev/null and b/archive/uncertain/ROS/figures/ROS-demo.png differ diff --git a/archive/uncertain/ROS/figures/ROS-release.png b/archive/uncertain/ROS/figures/ROS-release.png new file mode 100644 index 0000000000000000000000000000000000000000..bf7c1cb7b2b0b60ec375613d32e09ecd0a9174d0 Binary files /dev/null and b/archive/uncertain/ROS/figures/ROS-release.png differ diff --git a/archive/uncertain/ROS/figures/ROS2-release.png b/archive/uncertain/ROS/figures/ROS2-release.png new file mode 100644 index 0000000000000000000000000000000000000000..dc606412c467714af1d05c92b244ecfef63664f6 Binary files /dev/null and b/archive/uncertain/ROS/figures/ROS2-release.png differ diff --git a/archive/uncertain/ROS/figures/problem.png b/archive/uncertain/ROS/figures/problem.png new file mode 100644 index 0000000000000000000000000000000000000000..9f690fb99cac9b957a6601b6eca3a011bee12273 Binary files /dev/null and b/archive/uncertain/ROS/figures/problem.png differ diff --git a/archive/uncertain/ROS/figures/ros-humble.png b/archive/uncertain/ROS/figures/ros-humble.png new file mode 100644 index 0000000000000000000000000000000000000000..a6079358d9df9b983d82679af067a634fe5c05c3 Binary files /dev/null and b/archive/uncertain/ROS/figures/ros-humble.png differ diff --git a/archive/uncertain/ROS/figures/turtlesim.png b/archive/uncertain/ROS/figures/turtlesim.png new file mode 100644 index 0000000000000000000000000000000000000000..ebc8368f7e8e6a4b44075ad402b492638d636181 Binary files /dev/null and b/archive/uncertain/ROS/figures/turtlesim.png differ diff --git "a/archive/uncertain/ROS/\344\275\277\347\224\250\346\226\271\346\263\225.md" "b/archive/uncertain/ROS/\344\275\277\347\224\250\346\226\271\346\263\225.md" new file mode 100644 index 0000000000000000000000000000000000000000..753071153f3364d52a496f0f67de31e3e6420f55 --- /dev/null +++ "b/archive/uncertain/ROS/\344\275\277\347\224\250\346\226\271\346\263\225.md" @@ -0,0 +1,56 @@ +# 使用方法 + +## ROS使用 + +ROS 提供了一些实用的命令行工具，可以用于获取不同节点的各类信息，常用的命令如下: + +- rosnode : 操作节点 +- rostopic : 操作话题 +- rosservice : 操作服务 +- rosmsg : 操作msg消息 +- rossrv : 操作srv消息 +- rosparam : 操作参数 + +另请参考: + + + +## ROS2使用 + +```shell +# 命令help +[root@openEuler ~]# ros2 --help +usage: ros2 [-h] Call `ros2 -h` for more detailed usage. ... + +ros2 is an extensible command-line tool for ROS 2. + +optional arguments: + -h, --help show this help message and exit + +Commands: + action Various action related sub-commands + bag Various rosbag related sub-commands + component Various component related sub-commands + daemon Various daemon related sub-commands + doctor Check ROS setup and other potential issues + interface Show information about ROS interfaces + launch Run a launch file + lifecycle Various lifecycle related sub-commands + multicast Various multicast related sub-commands + node Various node related sub-commands + param Various param related sub-commands + pkg Various package related sub-commands + run Run a package specific executable + security Various security related sub-commands + service Various service related sub-commands + test Run a ROS2 launch test + topic Various topic related sub-commands + trace Trace ROS nodes to get information on their execution + wtf Use `wtf` as alias to `doctor` + + Call `ros2 -h` for more detailed usage. +``` + +## 注意事项 + +- 新打开终端需要执行“source /opt/ros/foxy/local_setup.bash”或“source /opt/ros/noetic/setup.bash”命令。 diff --git "a/archive/uncertain/ROS/\345\256\211\350\243\205\344\270\216\351\203\250\347\275\262.md" "b/archive/uncertain/ROS/\345\256\211\350\243\205\344\270\216\351\203\250\347\275\262.md" new file mode 100644 index 0000000000000000000000000000000000000000..e99c9809f19441b1a2c10cb6636be920ea564ac2 --- /dev/null +++ "b/archive/uncertain/ROS/\345\256\211\350\243\205\344\270\216\351\203\250\347\275\262.md" @@ -0,0 +1,54 @@ +# 安装与部署 + +## 软件要求 + +* 操作系统：openEuler 25.03 + +## 硬件要求 + +* x86_64架构、AArch64架构 + +## 环境准备 + +* 安装openEuler 25.03 系统，安装方法参考 《[安装指南](../../../docs/zh/server/installation_upgrade/installation/installation.md)》。 + +## 1. ROS2 + +### 1. ros-humble + +#### 1. 安装ros-humble + +1. 安装ros-humble 软件包 + + ```shell + [root@openEuler ~]# yum install openeuler-ros + [root@openEuler ~]# yum install ros-humble-ros-base ros-humble-xxx 例如安装小乌龟ros-humble-turtlesim + ``` + +2. 执行如下命令，查看安装是否成功。如果回显有对应软件包，表示安装成功。 + + ```shell + [root@openEuler ~]# rpm -q ros-humble + ``` + +#### 2. 测试 ros-humble + +##### 运行小乌龟 + +1. 启动小乌龟 + + ```shell + [root@openEuler ~]# source /opt/ros/humble/setup.bash + [root@openEuler ~]# ros2 run turtlesim turtlesim_node + ``` + +2. 打开小乌龟控制终端 + + ```shell + [root@openEuler ~]# source /opt/ros/humble/setup.bash + [root@openEuler ~]# ros2 run turtlesim turtle_teleop_key + ``` + +3. 在乌龟控制终端中使用方向键控制乌龟运动 + + ![ros-humble](./figures/ros-humble.png) diff --git "a/archive/uncertain/ROS/\345\270\270\350\247\201\351\227\256\351\242\230\344\270\216\350\247\243\345\206\263\346\226\271\346\263\225.md" "b/archive/uncertain/ROS/\345\270\270\350\247\201\351\227\256\351\242\230\344\270\216\350\247\243\345\206\263\346\226\271\346\263\225.md" new file mode 100644 index 0000000000000000000000000000000000000000..a7a4901b6ede1390eb2eb734b9817422b2e99cd6 --- /dev/null +++ "b/archive/uncertain/ROS/\345\270\270\350\247\201\351\227\256\351\242\230\344\270\216\350\247\243\345\206\263\346\226\271\346\263\225.md" @@ -0,0 +1,21 @@ +# 常见问题与解决方法 + +## **问题1** + +![](./figures/problem.png) + +原因：出现该警告的原因在于环境变量中同时存在ROS1、ROS2。 + +解决方法：修改环境变量，避免两个版本冲突。 + +```shell +[root@openEuler ~]# vim /opt/ros/humble/share/ros_environment/environment/1.ros_distro.sh +``` + +```shell +# generated from ros_environment/env-hooks/1.ros_distro.sh.in + +#export ROS_DISTRO=humble +``` + +将里面的内容全部注释即可。 diff --git "a/archive/uncertain/ROS/\350\256\244\350\257\206ROS.md" "b/archive/uncertain/ROS/\350\256\244\350\257\206ROS.md" new file mode 100644 index 0000000000000000000000000000000000000000..fa57fa8b6b484c06b6a5d26229111f838b53714a --- /dev/null +++ "b/archive/uncertain/ROS/\350\256\244\350\257\206ROS.md" @@ -0,0 +1,33 @@ +# 认识ROS + +## 简介 + +ROS 是一个适用于机器人的开源的元操作系统。它提供了操作系统应有的服务，包括硬件抽象，底层设备控制，常用函数的实现，进程间消息传递，以及包管理。它也提供用于获取、编译、编写和跨计算机运行代码所需的工具和库函数。 + +ROS的运行架构是一种使用ROS通信模块实现模块间[P2P](https://zh.wikipedia.org/wiki/對等網路)的松耦合的网络连接的处理架构，它执行若干种类型的通讯，包括： + +1. 基于服务的同步[RPC](https://zh.wikipedia.org/wiki/遠程過程調用)（远程过程调用）通讯； +2. 基于Topic的异步数据流通讯，还有参数服务器上的数据存储。 + +自从2007年ROS开始以来，伴随着机器人技术的大发展，ROS的核心思想和基本软件包逐渐完善并发布了不同的ROS发行版本。下面是当前和历史的ROS发行版列表，表中以绿色标记的行是当前支持的发行版。 + +![ROS 发行版](./figures/ROS-release.png) + +ROS虽然仍是机器人领域的开发利器，但介于最初设计时的局限性，也逐渐暴露出不少问题。比如：实时性差、系统开销大、对Python3支持不友好、没有加密机制安全性不高等问题。不少开发者和研究机构还针对ROS的局限性进行了改良，但这些局部功能的改善往往很难带来整体性能的提升。在ROSCon 2014上，新一代ROS的设计架构（Next-generation ROS: Building on DDS）正式公布，2015年8月31日第一个ROS2.0的alpha版本落地，之后也发布了不同的发行版本。下面是当前和历史的ROS2发行版列表，表中以绿色标记的行是当前支持的发行版。 + +![ROS2 发行版](./figures/ROS2-release.png) + +## 架构 + +ROS总体架构如下图所示： + +![ROS 架构](./figures/ROS-ROS2.png) + +1. OS层 + - ROS1主要构建于Linux系统之上，ROS2带来了改变，支持构建的系统包括Linux、Windows、Mac、RTOS，甚至没有操作系统的裸机。 +2. 中间件 + - ROS中最重要的一个概念就是基于发布/订阅模型的“节点”，可以让开发者并行开发低耦合的功能模块，并且便于二次复用。ROS1的通讯系统基于TCPROS/UDPROS，而ROS2的通讯系统基于DDS。DDS是一种分布式实时系统中数据发布/订阅的标准解决方案，下一小节会具体讲解。ROS2内部提供了DDS的抽象层实现，用户不需要关注底层DDS的提供厂家。 + - 在ROS1的架构中Nodelet和TCPROS/UDPROS是并列的层次，为同一个进程中的多个节点提供一种更优化的数据传输方式。ROS2中也保留了这种数据传输方式，只不过换了一个名字，叫做“Intra-process”，同样也是独立于DDS。 +3. 应用层 + + - ROS1强依赖于ROS Master，可以想像一旦Master宕机，整个系统会面临如何的窘境。但是从右边ROS2的架构中我们可以发现，之前让人耿耿于怀的Master终于消失了，节点之间使用一种称为“Discovery”的发现机制来获取彼此的信息。 diff --git "a/archive/uncertain/ROS/\351\231\204\345\275\225.md" "b/archive/uncertain/ROS/\351\231\204\345\275\225.md" new file mode 100644 index 0000000000000000000000000000000000000000..68d9f8f1f83facefa323a5fc188d9ab090c05c4f --- /dev/null +++ "b/archive/uncertain/ROS/\351\231\204\345\275\225.md" @@ -0,0 +1,3 @@ +# 附录 + +更详细的ROS介绍可访问[ROS wiki](https://wiki.ros.org/)，[ROS docs](http://docs.ros.org/)获取。 diff --git "a/archive/uncertain/SystemOptimization/MySQL\346\200\247\350\203\275\350\260\203\344\274\230\346\214\207\345\215\227.md" "b/archive/uncertain/SystemOptimization/MySQL\346\200\247\350\203\275\350\260\203\344\274\230\346\214\207\345\215\227.md" new file mode 100644 index 0000000000000000000000000000000000000000..9b761a0990f743652aa61f008cf6368b758fd1e5 --- /dev/null +++ "b/archive/uncertain/SystemOptimization/MySQL\346\200\247\350\203\275\350\260\203\344\274\230\346\214\207\345\215\227.md" @@ -0,0 +1,361 @@ +# MySQL性能调优指南 + +## 调优概述 + +### 调优原则 + +性能调优一方面在系统设计之初，需考虑硬件的选择，操作系统的选择，基础软件的选择；另一方面，包括每个子系统的设计，算法选择，如何使用编译器的选项，如何发挥硬件最大的性能等等。 + +在性能优化时，必须遵循一定的原则，否则，有可能得不到正确的调优结果。主要有以下几个方面： + +对性能进行分析时，要多方面分析系统的资源瓶颈所在，因为系统某一方面性能低，也许并非自身造成的，而是其他方面造成的。如CPU利用率是100%时，很可能是内存容量太小，因为CPU忙于处理内存调度。 + +- 一次只对影响性能的某方面的一个参数进行调整，多个参数同时调整的话，很难界定性能的影响是由哪个参数造成的。 + +- 在进行系统性能分析时，性能分析工具本身会占用一定的系统资源，如CPU、内存等，所以分析工具本身运行可能会导致系统某方面的资源瓶颈情况更加严重。 +- 必须保证调优后的程序运行正确。 +- 调优过程是持续的过程，每一次调优的结果都有反馈到后续的版本开发中去。 +- 性能调优不能以牺牲代码的可读性和可维护性为代价。 + +### 调优思路 + +性能优化首先要较为精准的定位问题，分析系统性能瓶颈，然后根据其性能指标以及所处层级选择优化的方式方法。 + +下面介绍MySQL数据库具体的调优思路和分析过程，如下图所示。 + + + +调优分析思路如下： + +- 很多情况下压测流量并没有完全进入到服务端，在网络上可能就会出现由于各种规格（带宽、最大连接数、新建连接数等）限制，导致压测结果达不到预期。 +- 接着看关键指标是否满足要求，如果不满足，需要确定是哪个地方有问题，一般情况下，服务器端问题可能性比较大，也有可能是客户端问题（这种情况比较小）。 +- 对于服务器端问题，需要定位的是硬件相关指标，例如CPU，Memory，Disk I/O，Network I/O，如果是某个硬件指标有问题，需要深入的进行分析。 +- 如果硬件指标都没有问题，需要查看数据库相关指标，例如：等待事件、内存命中率等。 +- 如果以上指标都正常，应用程序的算法、缓冲、缓存、同步或异步可能有问题，需要具体深入的分析。 + +## 硬件调优 + +### 目的 + +针对不同的服务器硬件设备，通过设置BIOS中的一些高级配置选项，可以有效提升服务器性能。 + +### 方法 + +以下方法针对鲲鹏服务器进行调优，X86，例如Intel服务器，可选择保持默认BIOS配置。 + +1. 关闭SMMU（鲲鹏服务器特有）。 +2. 重启服务器过程中，单击**Delete**键进入BIOS，选择“Advanced \> MISC Config”，单击**Enter**键进入。 +3. 将“**Support Smmu**”设置为“Disable” 。 + + 注：此优化项只在非虚拟化场景使用，在虚拟化场景，则开启SMMU。 + +4. 关闭预取 + + - 在BIOS中，选择“Advanced\>MISC Config”，单击**Enter**键进入。 + + - 将“CPU Prefetching Configuration”设置为“Disabled”，单击F10键保存退出。 + +## 操作系统调优 + +### 网卡中断绑核 + +#### 目的 + +通过关闭irqbalance服务，使用手动绑定网卡中断到部分专用核上，隔离网卡中断请求和业务请求，可以有效提升系统的网络性能。 + +#### 方法 + +对于不同的硬件配置，用于绑中断的最佳CPU数量会有差异，例如，鲲鹏920 5250上最佳CPU数量为5，可通过观察这5个CPU的使用情况以决定是否再调整用于绑中断的CPU数量。 + +以下脚本是在华为鲲鹏920 5250处理器+ Huawei TM280 25G网卡上的MySQL的最佳绑中断设置，其中第一个参数`$1`是网卡名称，第二个参数`$2`是队列数目5，第三个参数`$3`是网卡对应的总线名，可以用`ethtool -i <网卡名>`查询出： + +```shell +#!/bin/bash +eth1=$1 +cnt=$2 +bus=$3 +ethtool -L $eth1 combined $cnt + +irq1=`cat /proc/interrupts| grep -E ${bus} | head -n$cnt | awk -F ':' '{print $1}'` +irq1=`echo $irq1` +cpulist=(91 92 93 94 95) # 根据所选定的用于处理中断请求的核修改 +c=0 +forirq in $irq1 +do + echo ${cpulist[c]} "->" $irq + echo ${cpulist[c]} > /proc/irq/$irq/smp_affinity_list + let "c++" +done +``` + +**注：若采用下述的gazelle调优方法，则无需使用本节手段。** + +### NUMA绑核 + +#### 目的 + +通过NUMA绑核，减少跨NUMA访问内存，可以有效提升系统的访存性能。 + +#### 方法 + +基于前一节的网卡中断设置，在鲲鹏920 5250上，MySQL启动命令前设置NUMA绑核范围为剩余其他核，即0-90，其中`$mysql_path`为MySQL的安装路径： + +```shell +numactl -C 0-90 -i 0-3 $mysql_path/bin/mysqld --defaults-file=/etc/my.cnf & +``` + +**注：若采用下述的gazelle调优方法，则无需使用本节手段。** + +### 调度参数调优 + +#### 目的 + +在高负载场景下，CPU利用率并不能达到100%，深入分析每个线程的调度轨迹发现内核在做负载均衡时，经常无法找到一个合适的进程来迁移，导致CPU在间断空闲负载均衡失败，空转浪费CPU资源，通过使能openEuler调度特性STEAL模式，可以进一步提高CPU利用率，从而有效提升系统性能。(**当前该特性仅在openEuler 20.03 SP2版本及之后版本支持**) + +#### 方法 + +1）在/etc/grub2-efi.cfg中内核系统启动项末尾添加参数`sched_steal_node_limit=4`，修改后如下图所示： + + + +修改完成后，reboot重启生效。 + +2）设置STEAL模式 + +重启后设置STEAL调度特性如下： + +```shell +echo STEAL > /sys/kernel/debug/sched_features +``` + +### 大页调优 + +#### 目的 + +TLB（Translation lookaside buffer）为页表（存放虚拟地址的页地址和物理地址的页地址的映射关系）在CPU内部的高速缓存。TLB的命中率越高，页表查询性能就越好。内存页面越大，相同业务场景下的TLB命中率越高，访问效率提高，可以有效提升服务器性能。 + +#### 方法 + +- 调整内核的内存页大小。 + + 通过命令`getconf PAGESIZE`查看当前系统的内存页大小，如果大小是4096(4K)，则可通过修改linux内核的内存页大小来使用更大的内存页，需要在修改内核编译选项后重新编译内核。简要步骤如下： + + 1. 执行`make menuconfig`。 + + 2. 选择PAGESIZE大小为64K（Kernel Features--\>Page size(64KB)）。 + + 3. 编译和安装内核。 + +### Gazelle协议栈调优 + +#### 目的 + +原生内核网络协议栈层次深，开销较大，且系统调用的成本也较高。通过gazelle用户态协议栈替代内核协议栈，且通过hook posix接口，避免系统调用带来的开销，能够大幅提高应用的网络I/O吞吐能力。 + +#### 方法 + +1）安装依赖包。 + +配置openEuler的yum源，直接使用yum命令安装 + +```shell +yum install dpdk libconfig numactl libboundsheck libcap gazelle +``` + +2）使用root权限安装ko。 + +网卡从内核驱动绑为用户态驱动的ko，根据实际情况选择一种。mlx4和mlx5网卡不需要绑定vfio或uio驱动。 + +```shell +#若IOMMU能使用 +modprobe vfio-pci +#若IOMMU不能使用，且VFIO支持noiommu +modprobe vfio enable_unsafe_noiommu_mode=1 +modprobe vfio-pci +#其他情况 +modprobe igb_uio +``` + +3）dpdk绑定网卡。 + +将所使用的业务网卡(下面以enp3s0为例)绑定到步骤2选择的驱动，为用户态网卡驱动提供网卡资源访问接口。 + +```shell +#拔业务网卡enp3s0 +ip link set enp3s0 down + +#使用vfio-pci +dpdk-devbind -b vfio-pci enp3s0 + +#使用igb_uio +dpdk-devbind -b igb_uio enp3s0 +``` + +4）大页内存配置。 + +Gazelle使用大页内存提高效率。使用root权限配置系统预留大页内存，根据实际情况，选择一种页大小，配置足够的大页内存即可。如下默认每个内存节点配置2G大页内存，每个大页 2M： + +```shell +#配置2M大页内存：系统静态分配 2M * 1024*4 = 8G +echo 8192 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages + +#查看配置结果 +grep Huge /proc/meminfo +``` + +5）挂载大页内存。 + +创建一个目录，供lstack进程访问大页内存时使用，操作步骤如下： + +```shell +mkdir -p /mnt/hugepages-gazelle +chmod -R 700 /mnt/hugepages-gazelle +mount -t hugetlbfs nodev /mnt/hugepages-gazelle -o pagesize=2M +``` + +6）应用程序使用gazelle。 + +使用LD_PRELOAD预加载Gazelle动态库，`GAZELLE_BIND_PROCNAME`环境变量用于指定MySQL进程名： + +```shell +GAZELLE_BIND_PROCNAME=mysqld LD_PRELOAD=/usr/lib64/liblstack.so $mysql_path/bin/mysqld --defaults-file=/etc/my.cnf --bind-address=192.168.1.10 & +``` + +其中bind-address为服务端业务网卡ip，需同gazelle配置文件的host_addr保持一致。 + +7）修改gazelle配置文件。 + +使能gazelle需要根据硬件环境及软件需求定制gazelle配置文件/etc/gazelle/lstack.conf，示例如下： + +```shell +dpdk_args=["--socket-mem", "2048,2048,2048,2048", "--huge-dir", "/mnt/hugepages-gazelle", "--proc-type", "primary", "--legacy-mem", "--map-perfect"] + +use_ltran=0 +kni_switch=0 +low_power_mode=0 +listen_shadow=1 + +num_cpus="18,38,58,78 " +host_addr="192.168.1.10" +mask_addr="255.255.255.0" +gateway_addr="192.168.1.1" +devices="aa:bb:cc:dd:ee:ff" +``` + +其中，参数`--socket-mem`表示给每个内存节点分配的内存，默认2048M，例子中表示4个内存节点，每个内存节点分配2G(2048)；参数`--huge-dir`为先前建立的挂载了大页内存的目录；`num_cpus`记录lstack线程绑定的cpu编号，可按NUMA选择CPU。参数`host_addr`、`mask_addr`、`gateway_addr`和`devices`分别表示业务网卡的IP地址、掩码、网关地址和mac地址。 + +更详细的使用指导详见：[Gazelle用户指南]() + +## MySQL调优 + +### 数据库参数调优 + +#### 目的 + +通过调整数据库的参数配置，可以有效提升服务器性能。 + +#### 方法 + +默认配置文件路径为`/etc/my.cnf`，可使用如下配置文件参数启动数据库： + +```shell +[mysqld_safe] +log-error=/data/mysql/log/mysql.log +pid-file=/data/mysql/run/mysqld.pid + +[client] +socket=/data/mysql/run/mysql.sock +default-character-set=utf8 + +[mysqld] +basedir=/usr/local/mysql +tmpdir=/data/mysql/tmp +datadir=/data/mysql/data +socket=/data/mysql/run/mysql.sock +port=3306 +user=root +default_authentication_plugin=mysql_native_password +ssl=0 #关闭ssl +max_connections=2000 #设置最大连接数 +back_log=2048 #设置会话请求缓存个数 +performance_schema=OFF #关闭性能模式 +max_prepared_stmt_count=128000 + +#file +innodb_file_per_table=on #设置每个表一个文件 +innodb_log_file_size=1500M #设置logfile大小 +innodb_log_files_in_group=32 #设置logfile组个数 +innodb_open_files=4000 #设置最大打开表个数 + +#buffers +innodb_buffer_pool_size=230G #设置buffer pool size,一般为服务器内存60% +innodb_buffer_pool_instances=16 #设置buffer pool instance个数，提高并发能力 +innodb_log_buffer_size=64M #设置log buffer size大小 + +#tune +sync_binlog=1 #设置每次sync_binlog事务提交刷盘 +innodb_flush_log_at_trx_commit=1 #每次事务提交时MySQL都会把log buffer的数据写入log file，并且flush(刷到磁盘)中去 +innodb_use_native_aio=1 #开启异步IO +innodb_spin_wait_delay=180 #设置spin_wait_delay 参数，防止进入系统自旋 +innodb_sync_spin_loops=25 #设置spin_loops 循环次数，防止进入系统自旋 +innodb_spin_wait_pause_multiplier=25 #设置spin lock循环随机数 +innodb_flush_method=O_DIRECT #设置innodb数据文件及redo log的打开、刷写模式 +innodb_io_capacity=20000 # 设置innodb 后台线程每秒最大iops上限 +innodb_io_capacity_max=40000 #设置压力下innodb 后台线程每秒最大iops上限 +innodb_lru_scan_depth=9000 #设置page cleaner线程每次刷脏页的数量 +innodb_page_cleaners=16 #设置将脏数据写入到磁盘的线程数 +table_open_cache_instances=32 #设置打开句柄分区数 +table_open_cache=30000 #设置打开表的数量 + +#perf special +innodb_flush_neighbors=0 #检测该页所在区(extent)的所有页，如果是脏页，那么一起进行刷新，SSD关闭该功能 +innodb_write_io_threads=16 #设置写线程数 +innodb_read_io_threads=16 #设置读线程数 +innodb_purge_threads=32 #设置回收已经使用并分配的undo页线程数 +innodb_adaptive_hash_index=0 + +sql_mode=STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION,NO_AUTO_VALUE_ON_ZERO,STRICT_ALL_TABLES +``` + +表1 数据库调优参数 + +| 参数名称 | 参数含义 | 优化建议 | +| --------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | +| innodb_thread_concurrency | InnoDB使用操作系统线程来处理用户的事务请求。 | 建议取缺省值为0，它表示默认情况下不限制线程并发执行的数量。 | +| innodb_read_io_threads | 执行请求队列中的读请求操作的线程数。 | 根据CPU核数及读写比例进一步更改来提高性能。 | +| innodb_write_io_threads | 执行请求队列中的写请求操作的线程数。 | 根据CPU核数及读写比例进一步更改来提高性能。 | +| innodb_buffer_pool_instances | 开启多个内存缓冲池，把需要缓冲的数据hash到不同的缓冲池中，这样可以并行的内存读写 | 建议设置8\~32。 | +| innodb_open_files | 在innodb_file_per_table模式下，限制InnoDB能打开的文件数量。 | 建议此值调大一些，尤其是表特别多的情况。 | +| innodb_buffer_pool_size | 缓存数据和索引的地方。 | 通常建议内存的60%左右。 | +| innodb_log_buffer_size | 缓存重做日志。 | 缺省值是64M，建议通过查看innodb_log_wait，调整innodb_log_buffer_size大小。 | +| innodb_io_capacity | InnoDB 后台线程每秒最大iops上限。 | 建议为IO QPS总能力的75%。 | +| innodb_log_files_in_group | 重做日志组的个数。 | - | +| innodb_log_file_size | 重做日志文件大小。 | 如果存在大量写操作，建议增加日志文件大小，但日志文件过大，会影响数据恢复时间。 如果是非生产环境，测试极限性能时，尽量调大日志文件。 如果是商用场景，需要考虑数据恢复时间，综合折中后设置日志文件大小。 | +| innodb_flush_method | Log和数据刷新磁盘的方法： datasync模式：写数据时，write这一步并不需要真正写到磁盘才算完成（可能写入到操作系统buffer中就会返回完成），真正完成是flush操作，buffer交给操作系统去flush,并且文件的元数据信息也都需要更新到磁盘。 O_DSYNC模式：写日志操作是在write这步完成，而数据文件的写入是在flush这步通过fsync完成。 O_DIRECT模式：数据文件的写入操作是直接从mysql innodb buffer到磁盘的，并不用通过操作系统的缓冲，而真正的完成也是在flush这步,日志还是要经过OS缓冲。 | 建议O_DIRECT模式。 | +| innodb_spin_wait_delay | 控制轮询的间隔。 | 根据真实场景调试，直到看不到spin_lock热点函数等。优化建议180。 | +| innodb_sync_spin_loops | 控制轮询的循环次数。 | 根据真实场景调试，直到看不到spin_lock热点函数等。优化建议25。 | +| innodb_spin_wait_pause_multiplier | 控制轮询间隔随机数。 | 根据真实场景调试，直到看不到spin_lock热点函数等。缺省值50，优化建议25-50。 | +| innodb_lru_scan_depth | LRU列表的可用页数量。 | 缺省值是1024，非生产环境，测试极限性能可以适当调大，减少checkpoint次数。 | +| innodb_page_cleaners | 刷新脏数据的线程数。 | 建议与innodb_buffer_pool_instances相等。 | +| innodb_purge_threads | 回收undo的线程数。 | - | +| innodb_flush_log_at_trx_commit | 不管有没有提交，每秒钟都写到binlog日志里. 每次提交事务，都会把log buffer的内容写到磁盘里去，对日志文件做到磁盘刷新，安全性最好。 每次提交事务，都写到操作系统缓存，由OS刷新到磁盘，性能最好。 | 非生产环境，测试极限性能，可以设置为0。 | +| innodb_doublewrite | 是否开启二次写。 | 非生产环境，测试极限性能，可以设置为0，关闭二次写。 | +| ssl | 是否开启安全连接。 | 安全连接对性能影响较大，非生产环境，测试极限性能，可以设置为0；商用场景，根据客户需求调整。 | +| table_open_cache_instances | MySQL 缓存 table 句柄的分区的个数。 | 建议设置16-32。 | +| table_open_cache | Mysqld打开表的数量。 | 建议设置成30000。 | +| skip_log_bin | 是否开启binlog。 | 非生产环境，测试极限性能在参数文件中增加此参数，关闭binlog选项（添加至配置文件中： skip_log_bin \#log-bin=mysql-bin）。 | +| performance_schema | 是否开启性能模式。 | 非生产环境，测试极限性能设置为OFF，关闭性能模式。 | + +### 数据库内核调优 + +#### 目的 + +数据库内核优化是指通过修改MySQL数据库源码，提升数据库性能。使用数据库内核优化patch，需重新编译数据库。 + +#### 方法 + +MySQL数据库内核优化分为两个不同的场景，一个是OLTP场景，一个是OLAP场景，不同的场景采用不同的优化patch。 + +OLTP场景是指主要面向交易的处理系统，以小的事物及小的查询为主，快速响应用户操作。OLTP内核优化patch参考[MySQL细粒度锁优化](https://support.huaweicloud.com/fg-kunpengdbs/kunpengpatch_20_0004.html)。 + +OLAP场景是指主要对用户当前及历史数据进行分析、查询和生成报表，支持管理决策。OLAP内核优化patch参考[MySQL OLAP并行优化](https://support.huaweicloud.com/fg-kunpengdbs/kunpengolap_20_0002.html)。 diff --git "a/archive/uncertain/SystemOptimization/figures/mysql\350\260\203\344\274\230\346\200\235\350\267\257.png" "b/archive/uncertain/SystemOptimization/figures/mysql\350\260\203\344\274\230\346\200\235\350\267\257.png" new file mode 100644 index 0000000000000000000000000000000000000000..93c82f764024334412c8aca006ce0b4651513e83 Binary files /dev/null and "b/archive/uncertain/SystemOptimization/figures/mysql\350\260\203\344\274\230\346\200\235\350\267\257.png" differ diff --git "a/archive/uncertain/SystemOptimization/figures/pci\347\275\221\345\215\241\346\211\200\345\261\236NUMA NODE.png" "b/archive/uncertain/SystemOptimization/figures/pci\347\275\221\345\215\241\346\211\200\345\261\236NUMA NODE.png" new file mode 100644 index 0000000000000000000000000000000000000000..401028d9f88ea936c4e08bc572aeee573ce84b92 Binary files /dev/null and "b/archive/uncertain/SystemOptimization/figures/pci\347\275\221\345\215\241\346\211\200\345\261\236NUMA NODE.png" differ diff --git "a/archive/uncertain/SystemOptimization/figures/pci\350\256\276\345\244\207\345\217\267.png" "b/archive/uncertain/SystemOptimization/figures/pci\350\256\276\345\244\207\345\217\267.png" new file mode 100644 index 0000000000000000000000000000000000000000..02dab7ffc45389886a5a7aec7222b1a53b62d509 Binary files /dev/null and "b/archive/uncertain/SystemOptimization/figures/pci\350\256\276\345\244\207\345\217\267.png" differ diff --git a/archive/uncertain/SystemOptimization/figures/ring_buffer.png b/archive/uncertain/SystemOptimization/figures/ring_buffer.png new file mode 100644 index 0000000000000000000000000000000000000000..4b4a608150554bf677f503213d0a0227310b0a17 Binary files /dev/null and b/archive/uncertain/SystemOptimization/figures/ring_buffer.png differ diff --git "a/archive/uncertain/SystemOptimization/figures/swapoff\344\277\256\346\224\271\345\211\215\345\220\216\345\257\271\346\257\224.png" "b/archive/uncertain/SystemOptimization/figures/swapoff\344\277\256\346\224\271\345\211\215\345\220\216\345\257\271\346\257\224.png" new file mode 100644 index 0000000000000000000000000000000000000000..080c9f9bd79a0090d0ed962358e9da2457afdc77 Binary files /dev/null and "b/archive/uncertain/SystemOptimization/figures/swapoff\344\277\256\346\224\271\345\211\215\345\220\216\345\257\271\346\257\224.png" differ diff --git "a/archive/uncertain/SystemOptimization/figures/\345\206\205\346\240\270\345\220\257\345\212\250\351\241\271\346\267\273\345\212\240\345\217\202\346\225\260.png" "b/archive/uncertain/SystemOptimization/figures/\345\206\205\346\240\270\345\220\257\345\212\250\351\241\271\346\267\273\345\212\240\345\217\202\346\225\260.png" new file mode 100644 index 0000000000000000000000000000000000000000..30bafb334c64617d4963b6781e8976a08de5b553 Binary files /dev/null and "b/archive/uncertain/SystemOptimization/figures/\345\206\205\346\240\270\345\220\257\345\212\250\351\241\271\346\267\273\345\212\240\345\217\202\346\225\260.png" differ diff --git "a/archive/uncertain/SystemOptimization/figures/\345\210\233\345\273\272raid0.png" "b/archive/uncertain/SystemOptimization/figures/\345\210\233\345\273\272raid0.png" new file mode 100644 index 0000000000000000000000000000000000000000..31fc68e727aa3e1f3e9e29851e13ee2e05568735 Binary files /dev/null and "b/archive/uncertain/SystemOptimization/figures/\345\210\233\345\273\272raid0.png" differ diff --git "a/archive/uncertain/SystemOptimization/figures/\346\237\245\347\234\213LRO\345\217\202\346\225\260\346\230\257\345\220\246\345\274\200\345\220\257.png" "b/archive/uncertain/SystemOptimization/figures/\346\237\245\347\234\213LRO\345\217\202\346\225\260\346\230\257\345\220\246\345\274\200\345\220\257.png" new file mode 100644 index 0000000000000000000000000000000000000000..351e0d41ec47d790d4f3556d840e9c951e480680 Binary files /dev/null and "b/archive/uncertain/SystemOptimization/figures/\346\237\245\347\234\213LRO\345\217\202\346\225\260\346\230\257\345\220\246\345\274\200\345\220\257.png" differ diff --git "a/archive/uncertain/SystemOptimization/figures/\346\267\273\345\212\240\345\206\205\346\240\270\345\220\257\345\212\250\351\241\271.png" "b/archive/uncertain/SystemOptimization/figures/\346\267\273\345\212\240\345\206\205\346\240\270\345\220\257\345\212\250\351\241\271.png" new file mode 100644 index 0000000000000000000000000000000000000000..b674ab4a93e3fb2abd3f30749d96e724fd77019c Binary files /dev/null and "b/archive/uncertain/SystemOptimization/figures/\346\267\273\345\212\240\345\206\205\346\240\270\345\220\257\345\212\250\351\241\271.png" differ diff --git "a/archive/uncertain/SystemOptimization/figures/\347\273\221\346\240\270.png" "b/archive/uncertain/SystemOptimization/figures/\347\273\221\346\240\270.png" new file mode 100644 index 0000000000000000000000000000000000000000..627d0ba137a169c37afa1cc6dd81a2fffd9a0085 Binary files /dev/null and "b/archive/uncertain/SystemOptimization/figures/\347\273\221\346\240\270.png" differ diff --git "a/archive/uncertain/SystemOptimization/figures/\347\273\221\346\240\270\346\210\220\345\212\237\351\252\214\350\257\201.png" "b/archive/uncertain/SystemOptimization/figures/\347\273\221\346\240\270\346\210\220\345\212\237\351\252\214\350\257\201.png" new file mode 100644 index 0000000000000000000000000000000000000000..342e691a50fc63ea8a71fdf752a6df46daafe14c Binary files /dev/null and "b/archive/uncertain/SystemOptimization/figures/\347\273\221\346\240\270\346\210\220\345\212\237\351\252\214\350\257\201.png" differ diff --git a/archive/uncertain/SystemOptimization/overview.md b/archive/uncertain/SystemOptimization/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..a0b9ac93e37a594663b6fd901782afcb5c7f5453 --- /dev/null +++ b/archive/uncertain/SystemOptimization/overview.md @@ -0,0 +1,9 @@ +# 系统分析与调优 + +系统分析与调优包含MySQL性能调优指南和大数据调优指南，其中大数据调优指南具体包括： + +- spark 调优指南 +- hive调优指南 +- hbase调优指南 + +上述指南旨在帮助用户在openEuler上进行调优，包含调优手段、调优建议、调优方向等。 diff --git "a/archive/uncertain/SystemOptimization/\345\244\247\346\225\260\346\215\256\350\260\203\344\274\230\346\214\207\345\215\227.md" "b/archive/uncertain/SystemOptimization/\345\244\247\346\225\260\346\215\256\350\260\203\344\274\230\346\214\207\345\215\227.md" new file mode 100644 index 0000000000000000000000000000000000000000..52d1c1d4da5a07a90b9e49ab4181558063267e0d --- /dev/null +++ "b/archive/uncertain/SystemOptimization/\345\244\247\346\225\260\346\215\256\350\260\203\344\274\230\346\214\207\345\215\227.md" @@ -0,0 +1,808 @@ + +## 硬件调优 + +### 配置BIOS + +#### 目的 + +针对不同的服务器硬件设备，通过设置BIOS中的一些高级配置选项，可以有效提升服务器性能。 + +#### 方法 + +以下方法针对鲲鹏服务器进行调优，X86，例如Intel服务器，可选择保持默认BIOS配置。 + +1. 关闭SMMU（鲲鹏服务器特有）。 +2. 重启服务器过程中，单击**Delete**键进入BIOS，选择“Advanced \> MISC Config”，单击**Enter**键进入。 +3. 将“**Support Smmu**”设置为“**Disable**”。 + + 注：此优化项只在非虚拟化场景使用，在虚拟化场景，则开启SMMU。 + +4. 关闭预取 + + - 在BIOS中，选择“Advanced\>MISC Config”，单击**Enter**键进入。 + + - 将“CPU Prefetching Configuration”设置为“Disabled”，单击F10键保存退出。 + +### 创建RAID0 + +#### 目的 + +磁盘创建RAID可以提供磁盘的整体存取性能，环境上若使用的是LSI 3108/3408/3508系列RAID卡，推荐创建单盘RAID 0/RAID 5，充分利用RAID卡的Cache（LSI 3508卡为2GB），提高读写速率。 + +#### 方法 + +1. 使用storcli64_arm文件检查RAID组创建情况。 + + ```shell + ./storcli64_arm /c0 show + ``` + + 注：storcli64_arm文件放在任意目录下执行皆可。文件下载路径： + +2. 创建RAID 0，命令表示为第2块1.2T硬盘创建RAID 0，其中，c0表示RAID卡所在ID、r0表示组RAID 0。依次类推，对除了系统盘之外的所有盘做此操作。 + + ```shell + ./storcli64_arm /c0 add vd r0 drives=65:1 + ``` + + + +### 开启RAID卡Cache + +#### 目的 + +使用RAID卡的RAWBC（无超级电容）/RWBC（有超级电容）性能更佳。 + +使用storcli工具修改RAID组的Cache设置。Read Policy设置为Read ahead，使用RAID卡的Cache做预读。Write Policy设置为Write back，使用RAID卡的Cache做回写，而不是Write through透写。将IO policy设置为Cached IO，使用RAID卡的Cache缓存IO。 + +#### 方法 + +1. 配置RAID卡的cache，其中vx是卷组号（v0/v1/v2....），根据环境上的实际情况进行设置。 + +```shell +./storcli64_arm /c0/vx set rdcache=RA +./storcli64_arm /c0/vx set wrcache=WB/AWB +./storcli64_arm /c0/vx set iopolicy=Cached +``` + +2. 检查配置是否生效。 + +```shell +./storcli64_arm /c0 show +``` + +### 调整rx_buff + +#### 目的 + +以1822网卡为例，默认的rx_buff配置为2KB，在聚合64KB报文的时候需要多片不连续的内存，使用率较低。该参数可以配置为2/4/8/16KB，修改后可以减少不连续的内存，提高内存使用率。 + +#### 方法 + +1. 查看rx_buff参数值，默认为2。 + + ```shell + cat /sys/bus/pci/drivers/hinic/module/parameters/rx_buff + ``` + +2. 在目录“/etc/modprobe.d/”中增加文件hinic.conf，修改rx_buff值为8。 + + ```shell + options hinic rx_buff=8 + ``` + +3. 重新挂载hinic驱动，使得新参数生效。 + + ```shell + rmmod hinic + modprobe hinic + ``` + +4. 查看rx_buff参数是否更新成功。 + + ```shell + cat /sys/bus/pci/drivers/hinic/module/parameters/rx_buff + ``` + +### 调整Ring Buffer + +#### 目的 + +以1822网卡为例，网卡队列深度Ring Buffer最大支持4096，默认配置为1024，可以增大buffer大小用于提高网卡Ring大小。 + +#### 方法 + +1. 查看Ring Buffer默认大小，假设当前网卡名为enp131s0。 + + ```shell + ethtool -g enp131s0 + ``` + +2. 修改Ring Buffer值为4096。 + + ```shell + ethtool -G enp131s0 rx 4096 tx 4096 + ``` + +3. 查看Ring Buffer值是否更新成功。 + + ```shell + ethtool -g enp131s0 + ``` + + + +4. 减少队列数。 + + ```shell + ethtool -L enp131s0 combined 4 + ethtool -l enp131s0 + ``` + +### 开启LRO + +#### 目的 + +以1822网卡为例，支持LRO（Large Receive Offload），可以打开该选项，并合理配置1822 LRO参数。 + +#### 方法 + +1. 查看LRO参数是否开启，默认为off，假设当前网卡名为enp131s0。 + + ```shell + ethtool -k enp131s0 + ``` + +2. 打开LRO。 + + ```shell + ethtool -K enp131s0 lro on + ``` + +3. 查看LRO参数是否开启。 + + ```shell + ethtool -k enp131s0 + ``` + + + +### 配置网卡中断绑核 + +#### 目的 + +相比使用内核的irqbalance使网卡中断在所有核上进行调度，使用手动绑核将中断固定更能有效提高业务网络收发包的能力。 + +#### 方法 + +1. 关闭irqbalance。 + + 若要对网卡进行绑核操作，则需要关闭irqbalance。 + + 执行如下命令： + + ```shell + systemctl stop irqbalance.service #（停止irqbalance，重启失效） + systemctl disable irqbalance.service #（关闭irqbalance，永久生效） + systemctl status irqbalance.service #（查看irqbalance服务状态是否已关闭） + ``` + +2. 查看网卡pci设备号，假设当前网卡名为enp131s0。 + + ```shell + ethtool -i enp131s0 + ``` + + + +3. 查看pcie网卡所属NUMA node。 + + ```shell + lspci -vvvs + ``` + + + +4. 查看NUMA node对应的core的区间，例如此处在鲲鹏920 5250上就可以绑到48\~63。 + + + +5. 进行中断绑核，1822网卡共有16个队列，将这些中断逐个绑至所在NumaNode的16个Core上（例如此处是绑到NUMA node1对应48-63上）。 + + ```shell + bash smartIrq.sh + ``` + + 脚本内容如下： + + ```shell + #!/bin/bash + irq_list=(`cat /proc/interrupts | grep enp131s0 | awk -F: '{print $1}'`) + cpunum=48 # 修改为所在node的第一个Core + for irq in ${irq_list[@]} + do + echo $cpunum > /proc/irq/$irq/smp_affinity_list + echo `cat /proc/irq/$irq/smp_affinity_list` + (( cpunum+=1 )) + done + ``` + +6. 利用脚本查看是否绑核成功。 + + ```shell + sh irqCheck.sh enp131s0 + ``` + + + + 脚本内容如下： + + ```shell + #!/bin/bash + # 网卡名 + intf=$1 + log=irqSet-`date "+%Y%m%d-%H%M%S"`.log + # 可用的CPU数 + cpuNum=$(cat /proc/cpuinfo |grep processor -c) + # RX TX中断列表 + irqListRx=$(cat /proc/interrupts | grep ${intf} | awk -F':' '{print $1}') + irqListTx=$(cat /proc/interrupts | grep ${intf} | awk -F':' '{print $1}') + # 绑定接收中断rx irq + for irqRX in ${irqListRx[@]} + do + cat /proc/irq/${irqRX}/smp_affinity_list + done + # 绑定发送中断tx irq + for irqTX in ${irqListTx[@]} + do + cat /proc/irq/${irqTX}/smp_affinity_list + done + ``` + +## 操作系统调优 + +### 关闭内存透明大页 + +#### 目的 + +关闭内存透明大页，防止内存泄漏，减少卡顿。 + +#### 方法 + +关闭内存透明大页。 + +```shell +echo never > /sys/kernel/mm/transparent_hugepage/enabled +echo never > /sys/kernel/mm/transparent_hugepage/defrag +``` + +### 关闭swap分区 + +#### 目的 + +Linux的虚拟内存会根据系统负载自动调整，内存页（page）swap到磁盘swap分区会影响测试性能。 + +#### 方法 + +```shell +swapoff -a +``` + +修改前后对比如下。 + + + +## spark组件调优 + +### IO配置项调优 + +- IO队列调整 + + - 目的 + + 使用单队列（软队列）模式，单队列模式在spark测试时会有更佳的性能。 + + - 方法 + + 在/etc/grub2-efi.cfg的内核启动命令行中添加`scsi_mod.use_blk_mq=0`，重启生效。 + + + +- 内核IO参数配置 + + ```shell + #! /bin/bash + + echo 3000 > /proc/sys/vm/dirty_expire_centisecs + echo 500 > /proc/sys/vm/dirty_writeback_centisecs + + echo 15000000 > /proc/sys/kernel/sched_wakeup_granularity_ns + echo 10000000 > /proc/sys/kernel/sched_min_granularity_ns + + systemctl start tuned + sysctl -w kernel.sched_autogroup_enabled=0 + sysctl -w kernel.numa_balancing=0 + + echo 11264 > /proc/sys/vm/min_free_kbytes + echo 60 > /proc/sys/vm/dirty_ratio + echo 5 > /proc/sys/vm/dirty_background_ratio + + list="b c d e f g h i j k l m" #按需修改 + for i in $list + do + echo 1024 > /sys/block/sd$i/queue/max_sectors_kb + echo 32 > /sys/block/sd$i/device/queue_depth + echo 256 > /sys/block/sd$i/queue/nr_requests + echo mq-deadline > /sys/block/sd$i/queue/scheduler + echo 2048 > /sys/block/sd$i/queue/read_ahead_kb + echo 2 > /sys/block/sd$i/queue/rq_affinity + echo 0 > /sys/block/sd$i/queue/nomerges + done + ``` + +### JVM参数和版本适配 + +#### 目的 + +最新版本的JDK对Spark性能进行了优化。 + +#### 方法 + +可在spark-defaults.conf文件中添加以下配置来使用新的JDK，以使用新的JDK参数。 + +```shell +spark.executorEnv.JAVA_HOME /usr/local/jdk8u222-b10 +spark.yarn.appMasterEnv.JAVA_HOME /usr/local/jdk8u222-b10 +spark.executor.extraJavaOptions -XX:+UseNUMA -XX:BoxTypeCachedMax=100000 +spark.yarn.am.extraJavaOptions -XX:+UseNUMA -XX:BoxTypeCachedMax=100000 +``` + +注：X86架构，JVM参数`-XX:BoxTypeCachedMax`不适用。 + +### spark应用参数调优 + +#### 目的 + +在Spark基础配置值的基础上，按照理论公式得到一组较合理的Executor执行参数，使能后在鲲鹏上会带来明显的性能提升。 + +#### 方法 + +- 如果用Spark-Test-Tool工具测试sql1\~sql10场景，打开工具目录下的“script/spark-default.conf”文件，添加以下配置项： + + ```shell + yarn.executor.num 15 + yarn.executor.cores 19 + spark.executor.memory 44G + spark.driver.memory 36G + ``` + +- 如果使用HiBench工具测试wordcount、terasort、bayesian、kmeans场景，打开工具目录下的“conf/spark.conf”文件，可以根据实际环境对运行核数、内存大小做调整。 + + ```shell + yarn.executor.num 15 + yarn.executor.cores 19 + spark.executor.memory 44G + spark.driver.memory 36G + ``` + +### 专用场景优化项 + +#### SQL场景 + +##### sql1-IO密集型SQL + +- 目的 + + sql1是IO密集型场景，可以通过优化IO参数带来最佳性能。 + +- 方法 + + - 对以下IO相关参数进行设置，其中sd\$i指所有参与测试的磁盘名。 + + ```shell + echo 128 > /sys/block/sd$i/queue/nr_requests + echo 512 > /sys/block/sd$i/queue/read_ahead_kb + ``` + + - 对内存脏页参数进行设置。 + + ```shell + /proc/sys/vm/vm.dirty_expire_centisecs 500 + /proc/sys/vm/vm.dirty_writeback_centisecs 100 + ``` + + - 并行度在Spark-Test-Tool/script/spark-default.conf里设置。 + + ```shell + spark.sql.shuffle.partitions 350 + spark.default.parallelism 580 + ``` + + - 该场景其余参数都使用[spark应用参数调优](#spark应用参数调优)中的通用优化值。 + +##### sql2 & sql7 - CPU密集型SQL + +- 目的 + + ​sql2和sql7是CPU密集型场景，可以通过优化spark执行参数带来最佳性能。 + +- 方法 + + - Spark-Test-Tool在配置文件（script/spark-default.conf）中指定的运行核数、内存大小可以根据实际环境来做调整，从而达到最优性能。如对于鲲鹏920 5220处理器，sql2和sql7场景建议以下executor参数。 + + ```shell + yarn.executor.num 42 + yarn.executor.cores 6 + spark.executor.memory 15G + spark.driver.memory 36G + ``` + + - 并行度在Spark-Test-Tool/script/spark-default.conf里设置 + + 针对sql2，设置如下： + + ```shell + spark.sql.shuffle.partitions 150 + spark.default.parallelism 504 + ``` + + 针对sql7，设置如下： + + ```shell + spark.sql.shuffle.partitions 300 + spark.default.parallelism 504 + ``` + +##### sql3-IO + CPU + +- 目的 + + sql3是IO+CPU密集型场景，可以通过优化spark执行参数、调整IO参数来带来最佳性能。 + +- 方法 + + - Spark-Test-Tool在配置文件（script/spark-default.conf）中指定的运行核数、内存大小可以根据实际环境来做调整，来达到最优性能。比如对于鲲鹏920 5220处理器，sql3场景建议以下executor参数。 + + ```shell + yarn.executor.num 30 + yarn.executor.cores 6 + spark.executor.memory 24G + spark.driver.memory 36G + ``` + + - 调整IO预取值，其中sd\$i表示所有参与spark的磁盘名。 + + ```shell + echo 4096 > /sys/block/sd$i/queue/read_ahead_kb + ``` + + - 并行度在Spark-Test-Tool/script/spark-default.conf里设置。 + + ```shell + spark.sql.shuffle.partitions 150 + spark.default.parallelism 360 + ``` + +##### sql4 - CPU密集 + +- 目的 + + sql4是CPU密集型场景，可以优化spark执行参数、调整IO参数来带来最佳性能。 + +- 方法 + + Spark-Test-Tool在配置文件中指定的运行核数、内存大小可以根据实际环境来做调整，来达到最优性能。比如对于鲲鹏920 5220处理器，sql4场景建议以下executor参数。 + + - 打开工具目录下的script/spark-default.conf文件，添加以下配置项： + + ```shell + yarn.executor.num 42 + yarn.executor.cores 6 + spark.executor.memory 15G + spark.driver.memory 36G + ``` + + - 同时调整IO预取值，其中sd\$i表示所有参与spark的磁盘名： + + ```shell + echo 4096 > /sys/block/sd$i/queue/read_ahead_kb + ``` + + - 并行度在Spark-Test-Tool/script/spark-default.conf里设置。 + + ```shell + spark.sql.shuffle.partitions 150 + spark.default.parallelism 504 + ``` + +##### sql5 & sql6 & sql8 & sql9 & sql10 + +- 并行度在Spark-Test-Tool/script/spark-default.conf里设置。 + + ```shell + spark.sql.shuffle.partitions 300 + spark.default.parallelism 504 + ``` + +- 该场景其余参数都使用[spark应用参数调优](#spark应用参数调优)的通用优化值。 + +#### HiBench场景 + +##### Wordcount – IO + CPU密集型 + +- 目的 + + Wordcount是IO+CPU密集型场景，二者均衡，采用单队列的deadline调度算法反而不好，采用多队列的mq-deadline算法并调整相关io参数，能得到较好的性能结果。 + +- 方法 + + - 对以下配置进行修改，其中sd\$i指所有参与测试的磁盘名称： + + ```shell + echo mq-deadline > /sys/block/sd$i/queue/scheduler + echo 512 > /sys/block/sd$i/queue/nr_requests + echo 8192 > /sys/block/sd$i/queue/read_ahead_kb + echo 500 > /proc/sys/vm/dirty_expire_centisecs + echo 100 > /proc/sys/vm/dirty_writeback_centisecs + echo 5 > /proc/sys/vm/dirty_background_ratio + ``` + + - 该场景下采用3-5倍总核数作为数据分片的Partitions和 Parallelism进行数据分片，减小单Task文件大小，对性能有正面提升。可以使用以下分片设置： + + ```shell + spark.sql.shuffle.partitions 300 + spark.default.parallelism 600 + ``` + + - HiBench在配置文件中指定的运行核数、内存大小可以根据实际环境来做调整，来达到最优性能。比如对于鲲鹏920 5220处理器，Wordcount场景建议以下executor参数： + + ```shell + yarn.executor.num 51 + yarn.executor.cores 6 + spark.executor.memory 13G + spark.driver.memory 36G + ``` + +##### Terasort – IO + CPU 密集型 + +- 目的 + + ​Terasort是IO和CPU密集型场景，可以对IO参数和spark执行参数进行调整。另外，Terasort对网络的带宽要求也较高，可以通过优化网络参数，提升系统性能。 + +- 方法 + + - 对以下配置进行修改，其中sd\$i指所有参与测试的磁盘名称。 + + ```shell + echo bfq > /sys/block/sd$i/queue/scheduler + echo 512 > /sys/block/sd$i/queue/nr_requests + echo 8192 > /sys/block/sd$i/queue/read_ahead_kb + echo 4 > /sys/block/sd$i/queue/iosched/slice_idle + echo 500 > /proc/sys/vm/dirty_expire_centisecs + echo 100 > /proc/sys/vm/dirty_writeback_centisecs + ``` + + - 该场景下采用3-5倍总核数作为数据分片的Partitions和 Parallelism进行数据分片，减小单Task文件大小，对性能有正面提升。打开HiBench工具的“conf/spark.conf”文件，可以使用以下分片设置： + + ```shell + spark.sql.shuffle.partitions 1000 + spark.default.parallelism 2000 + ``` + + - 打开HiBench工具的“conf/spark.conf”文件，增加以下executor参数： + + ```shell + yarn.executor.num 27 + yarn.executor.cores 7 + spark.executor.memory 25G + spark.driver.memory 36G + ``` + + - 优化网络参数。 + + ```shell + ethtool -K enp131s0 gro on + ethtool -K enp131s0 tso on + ethtool -K enp131s0 gso on + ethtool -G enp131s0 rx 4096 tx 4096 + ethtool -G enp131s0 rx 4096 tx 4096 + # TM 280网卡最大可支持MTU=9000 + ifconfig enp131s0 mtu 9000 up + ``` + +##### Bayesian – CPU密集型 + +- 目的 + + ​Bayesian是CPU密集型场景，可以对IO参数和spark执行参数进行调整。 + +- 方法 + + - 该场景可以使用以下分片设置： + + ```shell + spark.sql.shuffle.partitions 1000 + spark.default.parallelism 2500 + ``` + + - 打开HiBench工具的“conf/spark.conf”文件，增加以下executor参数： + + ```shell + yarn.executor.num 9 + yarn.executor.cores 25 + spark.executor.memory 73G + spark.driver.memory 36G + ``` + + - 该场景使用以下内核参数： + + ```shell + echo mq-deadline > /sys/block/sd$i/queue/scheduler + echo 0 > /sys/module/scsi_mod/parameters/use_blk_mq + echo 50 > /proc/sys/vm/dirty_background_ratio + echo 80 > /proc/sys/vm/dirty_ratio + echo 500 > /proc/sys/vm/dirty_expire_centisecs + echo 100 > /proc/sys/vm/dirty_writeback_centisecs + ``` + +##### Kmeans – 纯计算密集型 + +- 目的 + + ​Kmeans是CPU密集型场景，可以对IO参数和spark执行参数进行调整。 + +- 方法 + + - 主要是调整spark executor参数适配到一个较优值，该场景可以使用以下分片设置： + + ```shell + spark.sql.shuffle.partitions 1000 + spark.default.parallelism 2500 + ``` + + - 调整IO预取值，其中sd\$i表示所有参与spark的磁盘名： + + ```shell + echo 4096 > /sys/block/sd$i/queue/read_ahead_kb + ``` + + - 打开HiBench工具的“conf/spark.conf”文件，增加以下executor参数： + + ```shell + yarn.executor.num 42 + yarn.executor.cores 6 + spark.executor.memory 15G + spark.driver.memory 36G + spark.locality.wait 10s + ``` + +## Hive组件调优 + +### IO配置项调优 + +主要涉及scheduler、read_ahead_kb、sector_size配置。 + +- scheduler推荐使用mq-deadline，可以达到更高的IO效率，从而提高性能。 + +- 块设备的预读推荐设置为4M，读性能更佳，缺省值一般为128KB。 + +- 块设备的sector_size应与物理盘的扇区大小进行匹配。可通过hw_sector_size、max_hw_sectors_kb、max_sectors_kb三者进行匹配，前两者是从硬件中读取出来的值，第三者是内核块设备的聚合最大块大小，推荐与硬件保持一致，即后两者参数保证一致。 + +对涉及的所有数据盘统一设置如下： + +```shell +list="b c d e f g h i j k l m" #按需修改 +for i in $list +do + echo mq-deadline > /sys/block/sd$i/queue/scheduler + echo 4096 > /sys/block/sd$i/queue/read_ahead_kb + echo 512 > sys/block/sd$i/queue/hw_sector_size + echo 1024 > /sys/block/sd$i/queue/max_hw_sectors_kb + echo 256 > /sys/block/sd$i/queue/max_sectors_kb +done +``` + +### 内存脏页参数调优 + +```shell +echo 500 > /proc/sys/vm/dirty_expire_centisecs +echo 100 > /proc/sys/vm/dirty_writeback_centisecs +``` + +### Hive调优 + +#### 组件参数配置 + +| 组件 | 参数名 | 推荐值 | 修改原因 | +| ----------------------------------------- | --------------------------------------------- | ------------------------------------------------ | ------------------------------------------------------------ | +| Yarn-\>NodeManager Yarn-\>ResourceManager | ResourceManager Java heap size | 1024 | 修改JVM内存大小，保证内存水平较高，减少GC的频率。 **说明：** 非固定值，需要根据GC的释放情况来调大或调小Xms及Xmx的值。 | +|Yarn-\>NodeManager Yarn-\>ResourceManager | NodeManager Java heap size | 1024 | |内存大小，保证内存水平较高，减少GC的频率。 **说明：** 非固定值，需要根据GC的释放情况来调大或调小Xms及Xmx的值。 +| Yarn-\>NodeManager | yarn.nodemanager.resource.cpu-vcores | 与实际数据节点的总物理核数相等。 | 可分配给Container的CPU核数。 | +|Yarn->NodeManager | yarn.nodemanager.resource.memory-mb | 与实际数据节点物理内存总量相等。 | 可分配给Container的内存。 | +| Yarn->NodeManager | yarn.nodemanager.numa-awareness.enabled | true | NodeManager启动Container时的Numa感知，需手动添加。 | +|Yarn->NodeManager | yarn.nodemanager.numa-awareness.read-topology | true | NodeManager的Numa拓扑自动感知，需手动添加。 | +| MapReduce2 | mapreduce.map.memory.mb | 7168 | 一个Map Task可使用的内存上限。 | +|MapReduce2 | mapreduce.reduce.memory.mb | 14336 | 一个Reduce Task可使用的资源上限。 | +| MapReduce2 | mapreduce.job.reduce.slowstart.completedmaps | 0.35 | 当Map完成的比例达到该值后才会为Reduce申请资源。 | +| HDFS-\>NameNode | NameNode Java heap size | 3072 | 修改JVM内存大小，保证内存水平较高，减少GC的频率。 | +| HDFS->NameNode | NameNode new generation size | 384 | |修改JVM内存大小，保证内存水平较高，减少GC的频率。 +|HDFS->NameNode | NameNode maximum new generation size | 384 | |修改JVM内存大小，保证内存水平较高，减少GC的频率。 +|HDFS->NameNode | dfs.namenode.service.handler.count | 32 | NameNode RPC服务端监测DataNode和其他请求的线程数，可适量增加。 | +|HDFS->NameNode | dfs.namenode.handler.count | 1200 | NameNode RPC服务端监测客户端请求的线程数，可适量增加。 | +| HDFS-\>DataNode | dfs.datanode.handler.count | 512 | DataNode服务线程数，可适量增加。 | +| TEZ | tez.am.resource.memory.mb | 7168 | 等同于yarn.scheduler.minimum-allocation-mb，默认7168。 | +| TEZ | tez.runtime.io.sort.mb | SQL1: 32 SQL2: 256 SQL3: 256 SQL4: 128 SQL5: 64 | 根据不同的场景进行调整。 | +|TEZ | tez.am.container.reuse.enabled | true | Container重用开关。 | +|TEZ | tez.runtime.unordered.output.buffer.size-mb | 537 | 10%\* hive.tez.container.size。 | +|TEZ | tez.am.resource.cpu.vcores | 10 | 使用的虚拟CPU数量，默认1，需要手动添加。 | +|TEZ | tez.container.max.java.heap.fraction | 0.85 | 基于Yarn提供的内存，分配给java进程的百分比，默认是0.8，需要手动添加。 | + +#### numa特性开启 + +Yarn组件在3.1.0版本合入的新特性支持，支持Yarn组件在启动Container时使能NUMA感知功能，原理是读取系统物理节点上每个NUMA节点的CPU核、内存容量，使用Numactl命令指定启动container的CPU范围和membind范围，减少跨片访问。 + +1. 安装numactl + +```shell +yum install numactl.aarch64 +``` + +2. 开启NUMA感知 + +``` +yarn.nodemanager.numa-awareness.enabled true +yarn.nodemanager.numa-awareness.read-topology true +``` + +在组件参数配置已列出。 + +## Hbase组件调优 + +### Hbase调优 + +#### 组件参数配置 + +如下参数为本次测试所配置参数，x86计算平台和鲲鹏920计算平台参数仅有Yarn部分参数有差异（差异处表格中有体现），hbase和hdfs采用同一套参数进行测试。 + +| 组件 | 参数名 | 推荐值 | 修改原因 | +| ----------------------------------------- | --------------------------------------------- | -------------------------------- | ------------------------------------------------------------ | +| Yarn-\>NodeManager Yarn-\>ResourceManager | ResourceManager Java heap size | 1024 | 修改JVM内存大小，保证内存水平较高，减少GC的频率。 | +| Yarn-\>NodeManager Yarn-\>ResourceManager | NodeManager Java heap size | 1024 | | +| Yarn-\>NodeManager | yarn.nodemanager.resource.cpu-vcores | 与实际数据节点的总物理核数相等。 | 可分配给Container的CPU核数。 | +|Yarn->NodeManager | yarn.nodemanager.resource.memory-mb | 与实际数据节点物理内存总量相等。 | 可分配给Container的内存。 | +|Yarn->NodeManager | yarn.nodemanager.numa-awareness.enabled | true | NodeManager启动Container时的Numa感知，需手动添加。 | +|Yarn->NodeManager | yarn.nodemanager.numa-awareness.read-topology | true | NodeManager的Numa拓扑自动感知，需手动添加。 | +| MapReduce2 | mapreduce.map.memory.mb | 7168 | 一个Map Task可使用的内存上限。 | +|MapReduce2 | mapreduce.reduce.memory.mb | 14336 | 一个Reduce Task可使用的资源上限。 | +|MapReduce2 | mapreduce.job.reduce.slowstart.completedmaps | 0.35 | 当Map完成的比例达到该值后才会为Reduce申请资源。 | +| HDFS-\>NameNode | NameNode Java heap size | 3072 | 修改JVM内存大小，保证内存水平较高，减少GC的频率。 | +| HDFS-\>NameNode | NameNode new generation size | 384 | |修改JVM内存大小，保证内存水平较高，减少GC的频率。 +| HDFS-\>NameNode | NameNode maximum new generation size | 384 | |修改JVM内存大小，保证内存水平较高，减少GC的频率。 +| HDFS-\>NameNode | dfs.namenode.service.handler.count | 128 | NameNode RPC服务端监测DataNode和其他请求的线程数，可适量增加。 | +| HDFS-\>NameNode | dfs.namenode.handler.count | 1200 | NameNode RPC服务端监测客户端请求的线程数，可适量增加。 | +| HDFS-\>DataNode | dfs.datanode.handler.count | 512 | DataNode服务线程数，可适量增加。 | +| HBase-\>RegionServer | HBase RegionServer Maximum Memory | 31744 | 修改JVM内存大小，保证内存水平较高，减少GC的频率。 | +|HBase-\>RegionServer | hbase.regionserver.handler.count | 150 | RegionServer上的RPC服务器实例数量。 | +|HBase-\>RegionServer | hbase.regionserver.metahandler.count | 150 | RegionServer中处理优先请求的程序实例的数量。 | +| HBase-\>RegionServer | hbase.regionserver.global.memstore.size | 0.4 | 最大JVM堆大小（Java -Xmx设置）分配给MemStore的比例。 | +|HBase-\>RegionServer | hfile.block.cache.size | 0.4 | 数据缓存所占的RegionServer GC -Xmx百分比。 | +|HBase-\>RegionServer | hbase.hregion.memstore.flush.size | 267386880 | Regionserver memstore大小，增大可以减小阻塞。 | + +#### numa特性开启 + +Yarn组件在3.1.0版本合入的新特性支持，支持Yarn组件在启动Container时使能NUMA感知功能，原理是读取系统物理节点上每个NUMA节点的CPU核、内存容量，使用Numactl命令指定启动container的CPU范围和membind范围，减少跨片访问。 + +1. 安装numactl + +```shell +yum install numactl.aarch64 +``` + +2. 开启NUMA感知 + +``` +yarn.nodemanager.numa-awareness.enabled true +yarn.nodemanager.numa-awareness.read-topology true +``` + +在组件参数配置已列出。 diff --git a/archive/uncertain/X-diagnosis/X-diagnosis.md b/archive/uncertain/X-diagnosis/X-diagnosis.md new file mode 100644 index 0000000000000000000000000000000000000000..298451b5c3e9801f3617199ef4994d06a8b73d42 --- /dev/null +++ b/archive/uncertain/X-diagnosis/X-diagnosis.md @@ -0,0 +1,433 @@ +# x-diagnose + +## 概述 + +X-diagnose基于EulerOS维护团队多年运维经验，通过对案例的总结/分析形成的系统运维工具集， +主要功能包含问题定位、系统巡检/监控、ftrace增强、一键收集日志等功能，是一款集成分析、 +流程跟踪、信息定时记录、历史经验固化等功能于一体的OS内核问题定位工具。 + +## 安装x-diagnose + +**(1) 依赖软件** + +* python 3.7+ + +**(2) 下载rpm包** + +``` +rpm -ivh xdiagnose-1.x-x.rpm +``` + +## 1. 命令汇总 + +* xdiag +* xd_tcpreststack +* xd_tcpskinfo +* xd_arpstormcheck +* xd_sysinspect +* xd_scsiiocount +* xd_scsiiotrace + +### 1.0 xdiag + +```shell +usage: xdiag [-h] [--inspect] {tcphandcheck,eftrace,ntrace,hook} ... + +xdiagnose tool + +optional arguments: + -h, --help show this help message and exit + --inspect inspector module + +select module: + {tcphandcheck,eftrace,ntrace,hook} + tcphandcheck tcp_hand_check module + eftrace eftrace module + ntrace net trace module + hook hook module +``` + +**--inspect ：系统异常巡检(可以和select module一起使用)支持如下检测项：** + +* ipv6路由缓存满 +* TIMEWAIT状态连接满 +* arp、连接跟踪满 +* snmp或者stat异常 +* 网卡异常统计pause帧、tx_timeout、drop、error +* bond4异常检测: +1)网卡速率不相等 +2)lacp协商没有成功 +* tcp、udp、ip分片等内存满 +* dns无法解析(gethostbyname) +* cron没法运行 +* ntp时钟不准 +* ip冲突检测 +* cpu冲高检测 +* 磁盘满、inode句柄不足 +* 内存不足、sysctl/sshd配置运行过程中修改 + +**tcphandcheck：跟踪tcp的3次握手阶段经常会出现问题，支持定位如下问题：** + +* 连接队列满 +* bind失败 +* timewait连接复用失败 +* 文件句柄超出导致无法创建socket +* 端口复用场景下连接闪断后seq序号异常导致的无法建链 + +**eftrace** + +#### 概述 + +eftrace是ftrace命令生成的偏移计算辅助工具。用户可以使用eftrace方便地生成不同内核版本下的ftrace命令。 + +#### 使用方法 + +#### (1) 举例 + +生成在协议栈调用`ip_rcv_core`函数时打印源地址为`192.168.56.102`的命令： + +```shell +xdiag eftrace 'p:ip_rcv_core ip_rcv_core srcip=(struct iphdr *)($r0->data)->saddr f:srcip==0x6638a8c0' +``` + +生成在协议栈调用`inet_csk_accept`函数结束时返回值为0的命令： + +```shell +xdiag eftrace 'r:inet_csk_accept inet_csk_accept ret=$retval f:ret==0' +``` + +#### (2) 命令解析 + +* `p:` 表示kprobe event +* `r:` 表示kretprobe event +* `f:` 表示kprobe filter过滤 +* `$rx` 表示函数参数，x为参数位置，第一个参数为`$r0` + +#### (3) 可以使用强制类型转换，以及手动指定偏移 + +```shell +xdiag eftrace 'p:ip_rcv_finish ip_rcv_finish +srcip=(struct iphdr *)($r2->data)->saddr +srcport=(struct tcphdr *)($r2->data + 20)->source' +``` + +在函数`ip_rcv_finish`中，`sk_buff`的`data`成员是`unsigned char *`类型，指向报文的ip头，可以强制转换为`iphdr *`获取ip头的内容。 + +当想获取tcp头的内容时，对`data`进行ip头长度的偏移后可指向tcp头并获取信息。 + +额外的偏移可以直接指定，或者使用`sizeof`的方式获取偏移长度： + +`srcport=(struct tcphdr *)($r2->data + sizeof(struct iphdr))->source` + +**xd_sysinspect** + +#### 参数说明 + +xd_sysinspect [-i interval] [-r rotate] [-d dest] [-z gzip] [-s size] [-c cpu_thresh] [-m mem_thresh] [-o] + +* -i interval: + 收集日志的时间间隔，单位秒 +* -r rotate: + 保留日志的份数 +* -d dest: + 日志文件保存的路径 +* -z gzip: + 用于压缩日志文件的命令，默认gzip +* -s size: + 指定该参数后使用日志文件的大小(MB)进行日志分割，超过设定值后会被压缩保存。不指定该参数默认按照小时压缩分割 +* -o: + 只记录触发CPU、内存阈值门限时的日志。如不指定该参数，则按照时间间隔收集日志 +* -c cpu_thresh: + CPU使用率的阈值，超过阈值、恢复阈值会触发日志记录 +* -m mem_thresh: + 内存使用率的阈值，超过阈值、恢复阈值会触发日志记录 + +#### 日志收集方式 + +* 每个interval在系统内收集一次信息 +* cpu,mem统计以interval为间隔，通过读取/proc下的数据计算使用率 + +#### 使用示例 + +#### 以时间为单位抓取日志 + +`xd_sysinspect -i 30 -r 48` + +* -i 30: + 每30秒收集一次日志 +* -r 48: + 每小时分割一次日志文件，保留48份日志 + +#### 以CPU、内存使用率阈值抓取日志 + +`xd_sysinspect -i 30 -r 20 -s 10 -c 80` + +* -i 30 + CPU、内存检查时间间隔30秒 +* -r 20 + 日志文件保留20份 +* -s 10 + 日志文件分割大小10(MB)。当日志文件达到指定值10MB时会进行分割 +* -c 80 + 指定CPU阈值，CPU使用率达到80%时记录一次日志；当使用率降至阈值以下，并重新冲高超过阈值，会再次记录 + +**ntrace：** + +```shell +usage: xdiag ntrace [-h] [-r READ_FILE] [-w WRITE_FILE] [-t TIMEOUT] [--qlen QLEN] [--cpu_mask CPU_MASK] [-b] [-i INTERFACE] {tcp,udp,icmp} ... + +optional arguments: + -h, --help show this help message and exit + -r READ_FILE, --read_file READ_FILE + read an existing trace file + -w WRITE_FILE, --write_file WRITE_FILE + trace write to a specified file + -t TIMEOUT, --timeout TIMEOUT + specify a running time of process + --cpu_mask CPU_MASK set ftrace cpu tracing_mask + -i INTERFACE, --interface INTERFACE + specify an interface + +select protocol: + {tcp,udp,icmp} + tcp tcp protocol + udp udp protocol + icmp icmp protocol +``` + +**expression** ：指定一个过滤报文的表达式，协议[tcp|udp]，地址[host|src|dst]，端口号[port|sport|dport]，逻辑运算符[and|or]。 +**-r** READFILE：读取一个已存在的trace输出文件，比如/var/log/x-diagnose/rawlog/raw_diag.log +**-w** WRITEFILE：将trace命令日志写入文件 +**-i** INTERFACE：指定抓取的网卡 +**-t** TIMEOUT：运行时间，单位为秒 +**--cpu_mask** CPU_MASK：设置ftrace的cpumask用以跟踪指定的cpu + +***说明***： +由于使用ftrace实现，xdiag下的select module功能模块不能复用 + +**hook：在定位问题时，方便确认各hook点的流程，跟踪这些钩子函数：** + +```shell +Usage: hook [ OPTIONS ] + --dev 网络设备过滤 + --host IP地址过滤 +``` + +### 1.1 xd_tcpreststack + +```shell +Usage: xd_tcpreststack [ OPTIONS ] + -h,--help this message + -t,--time The frequency of the probe/ms + -d,--depth Kernel stack Depth\n +``` + +#### 功能 + +监控tcp协议栈(v4/v6)reset信息。 + +#### -t,--time + +监控的时间间隔，单位ms, 建议保持默认值500ms； + +#### -d,--depth + +内核调用栈深度，默认3层 + +### 1.2 xd_tcpskinfo + +```shell +Usage: xd_tcpskinfo [ OPTIONS ] + -h,--help this message + -a,--addr filter IP addr + -p,--port filter port +``` + +#### 功能 + +查看tcp连接socket关键的信息，ss命令抓的信息不够全部一些关键信息没有包含。该工具总结tcp连接在debug过程中经常需要的信息，用来辅助协议栈问题定位。包括如下信息： + +#### -a,--addr + +IP地址过滤，不区分源地址或者目的地。 + +#### -p,--port + +端口过滤，不区分源端口或者目的端口。 + +### 1.3 xd_arpstormcheck + +```shell +Usage: xd_arpstormcheck [ OPTIONS ] + -h,--help this message + -i,--interval The interval time of the probe/s + -c,--count check count, default 1 + -f,--freq filter freq, $$ times per second +``` + +#### 功能 + +监控当前网络是否发发生网络风暴。 + +#### -i,--interval + +监控的时间间隔，默认1s。 + +#### -c,--count + +总监控的次数，监控次数完成后监控工具自动退出。 + +#### -f,--freq + +监控的告警阈值，每秒收到的报文，超过了此阈值，则告警提示网络风暴相关信息； + +### 1.4 xd_scsiiotrace + +```shell +USAGE: xd_scsiiotrace [--help] [-d h:c:t:l] [-E] + +EXAMPLES: + xd_scsiiotrace # Report all scsi cmnd result + xd_scsiiotrace -E # Report error/timeout scsi cmnd result + xd_scsiiotrace -p 0x8000002 # Parse the scsi cmnd result. + xd_scsiiotrace -d 0:0:0:1 # Trace the scsi device only. + + -d, --device=h:c:t:l Trace this scsi device only + -E, --error Trace error/timeout scsi cmnd. (default trace all + scsi cmnd) + -p, --parse=result Parse the scsi cmnd result.(format hex) + -?, --help Give this help list + --usage Give a short usage message +``` + +#### 功能 + +用于监控scsi命令执行结果: +DRIVER_RESULT： 驱动返回结果 +SCSI_RESULT： SCSI转换后的结果。 +DISPOSION： +1)SUCCESS:成功 +2)NEEDS_RETRY/ADD_TO_MLQUEUE:重新入队列 +3)TIMEOUT_ERROR: 命令超时 + +#### -d,--device + +指定需要监控的设备，默认监控所有。 + +#### -E,--error + +只监控不成功的命令（错误或者超时），默认监控所有命令。 + +#### -p,--parse + +用于解析 DRIVER_RESULT或者SCSI_RESULT值具体含义. 默认显示hex值 + +### 1.5 xd_scsiiocount + +```shell +USAGE: xd_scsiiocount [--help] [-t times] [-d device] [-i interval] + +EXAMPLES: + xd_scsiiocount # report all scsi device I/O scsi cmnd count + xd_scsiiocount -i 10 # print 10 second summaries + xd_scsiiocount -d sdc # Trace sdc only + xd_scsiiocount -t 5 # report times + + -d, --device=device Trace this disk only + -i, --interval=interval refresh interval(secs) + -t, --times=times report scsi device I/O times + -?, --help Give this help list + --usage Give a short usage message +``` + +#### 功能 + +用于监控scsi命令下发的命令统计. + +#### -d,--device + +指定需要监控的设备，默认监控所有。 + +#### -i,--interval + +监控的时间间隔，默认5s。 + +#### -t,--times + +监控的次数. 次数达到后，则结束本次监控 + +### 1.6 xd_ext4fsstat + +```shell +USAGE: xd_ext4fstat [--help] [-t times] [-i interval] [-s SORT] [-o opcode] + +EXAMPLES: + xd_ext4fsstat#Trace file read/write stat for ext4 filesystem + xd_ext4fsstat -i 10#printf 10 second summaries + xd_ext4fsstat -m /mnt/test#Trace the special mount point for ext4 filesystem. + xd_ext4fsstat -s r#Sort the read bytes + xd_ext4fsstat -o r#Trace read only, default read and wriete + xd_ext4fsstat -t 5#Trace 5 times + xd_ext4fsstat -v p#show the pid view + + -c, --clean Clean the trace data + -C, --clear Clear the screen + -i, --interval=INTERVAL Refreash interval(secs), default 5s. + -m, --mnt=MNTPOINT the special mount point + -o, --opcode=OPCODE Trace file r/w, default both. + -p, --pid=PID Trace the pid only + -s, --sort=SORT Sort r/w/wb, default read + -t, --times=TIMES Trace times + -T, --top=TOP show the topN (1~8192) + -v, --view=VIEW p:pids view, f: files view, default file view + -?, --help Give this help list + --usage Give a short usage message +``` + +#### 功能 + +用于监控ext4 文件系统读/写数据量统计. + +#### -c,--clean + +每个周期统计完数据后，历史数据将被清空，重新统计，默认是累积. + +#### -C,--clear + +每个周期显示统结果后，清理屏幕 + +#### -i,--interval + +监控的时间间隔，默认5s. + +#### -m,--mnt + +指定监控特定ext4挂载点数据. + +#### -o,--opcode + +指定监控read或者write,默认两者都监控. + +#### -p,--pid + +指定监控特定pid. + +#### -s,--sort + +指定对监控数据进行排序，默认对read进行排序. + +#### -t,--times + +监控的次数. 次数达到后，则结束本次监控. + +#### -T,--top + +指定只显示top数据，默认显示所有监控到的数据. + +#### -v,--view + +指定显示模式，p:进程模式，f:文件默认，默认只显示文件模式 +注：进程模式下，多进程对同一有写入场景下，显示的writeback数据为文件写入总数据 + 非该单进程写入总数据. + +#### diff --git "a/archive/uncertain/astream/astream\345\272\224\347\224\250\344\272\216MySQL\346\214\207\345\257\274.md" "b/archive/uncertain/astream/astream\345\272\224\347\224\250\344\272\216MySQL\346\214\207\345\257\274.md" new file mode 100644 index 0000000000000000000000000000000000000000..15bbff5179e5d98a38de2b0edaca2af95bcfc31f --- /dev/null +++ "b/archive/uncertain/astream/astream\345\272\224\347\224\250\344\272\216MySQL\346\214\207\345\257\274.md" @@ -0,0 +1,573 @@ +# astream使能MySQL测试步骤 + +## 1 环境要求 + +### 1.1 硬件 + +要求服务端（server）和客户端（client）各一台。 + +| | Server | Client | +| :------- | :-----------------------------: | :-----------------------: | +| CPU | Kunpeng 920-6426 * 2 | Kunpeng 920-6426 * 2 | +| 核数 | 64cores*2 | 64cores*2 | +| 主频 | 2600MHz | 2600MHz | +| 内存大小 | 16 * 32G Samsung 2666 MHz | 16 * 32G Samsung 2666 MHz | +| 网络 | SP580 10GE | SP580 10GE | +| 系统盘 | 1.2T HDD TOSHIBA | 1.12 HDD TOSHIBA | +| 数据盘 | 1.6T ES3000 V5 NVMe PCIe SSD*2 | NA | + +### 1.2 软件 + +| 软件名称 | 版本 | +| :----------: | :----: | +| mysql | 8.0.20 | +| benchmarksql | 5.0 | + +### 1.3 组网 + + + +## 2. Server端部署 + +### 2.1 安装mysql依赖包 + +```shell +yum install -y cmake doxygen bison ncurses-devel openssl-devel libtool tar rpcgen libtirpc-devel bison bc unzip git gcc-c++ libaio libaio-devel numactl +``` + +### 2.2 编译安装mysql + +- 从[官网下载](https://downloads.mysql.com/archives/community/)下载源码包。 + +- 下载优化补丁: [细粒度锁优化特性补丁](https://github.com/kunpengcompute/mysql-server/releases/download/tp_v1.0.0/0001-SHARDED-LOCK-SYS.patch) 、 [NUMA调度补丁](https://github.com/kunpengcompute/mysql-server/releases/download/21.0.RC1.B031/0001-SCHED-AFFINITY.patch) 、 [无锁优化特性补丁](https://github.com/kunpengcompute/mysql-server/releases/download/tp_v1.0.0/0002-LOCK-FREE-TRX-SYS.patch)。 + +- 编译mysql。编译前确保已安装`libaio-devel`包。 + + ```shell + tar zxvf mysql-boost-8.0.20.tar.gz + cd mysql-8.0.20/ + patch -p1 < ../0001-SHARDED-LOCK-SYS.patch + patch -p1 < ../0001-SCHED-AFFINITY.patch + patch -p1 < ../0002-LOCK-FREE-TRX-SYS.patch + cd cmake + make clean + cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local/mysql-8.0.20 -DWITH_BOOST=../boost -DDOWNLOAD_BOOST=1 + make -j 64 + make install + ``` + +### 2.3 配置mysql参数 + +为了使磁盘压力足够大，测试过程中采用**mysql双实例**跑法。其中实例1对应的配置文件为/etc/my-1.cnf，实例2对应的配置文件为/etc/my-2.cnf。 + +- /etc/my-1.cnf + +``` +[mysqld_safe] +log-error=/data/mysql-1/log/mysql.log +pid-file=/data/mysql-1/run/mysqld.pid + +[client] +socket=/data/mysql-1/run/mysql.sock +default-character-set=utf8 + +[mysqld] +server-id=3306 +#log-error=/data/mysql-1/log/mysql.log +#basedir=/usr/local/mysql +socket=/data/mysql-1/run/mysql.sock +tmpdir=/data/mysql-1/tmp +datadir=/data/mysql-1/data +default_authentication_plugin=mysql_native_password +port=3306 +user=root +#innodb_page_size=4k + +max_connections=2000 +back_log=4000 +performance_schema=OFF +max_prepared_stmt_count=128000 +#transaction_isolation=READ-COMMITTED +#skip-grant-tables + +#file +innodb_file_per_table +innodb_log_file_size=2048M +innodb_log_files_in_group=32 +innodb_open_files=10000 +table_open_cache_instances=64 + +#buffers +innodb_buffer_pool_size=150G # 根据系统内存大小可调整 +innodb_buffer_pool_instances=16 +innodb_log_buffer_size=2048M +#innodb_undo_log_truncate=OFF + +#tune +default_time_zone=+8:00 +#innodb_numa_interleave=1 +thread_cache_size=2000 +sync_binlog=1 +innodb_flush_log_at_trx_commit=1 +innodb_use_native_aio=1 +innodb_spin_wait_delay=180 +innodb_sync_spin_loops=25 +innodb_flush_method=O_DIRECT +innodb_io_capacity=30000 +innodb_io_capacity_max=40000 +innodb_lru_scan_depth=9000 +innodb_page_cleaners=16 +#innodb_spin_wait_pause_multiplier=25 + +#perf special +innodb_flush_neighbors=0 +innodb_write_io_threads=24 +innodb_read_io_threads=16 +innodb_purge_threads=32 + +sql_mode=STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION,NO_AUTO_VALUE_ON_ZERO,STRICT_ALL_TABLES + +#skip_log_bin +log-bin=mysql-bin # 开启mysql-bin +binlog_expire_logs_seconds=1800 # 根据需要生成的数据量能够维持长时间运行可调整 +ssl=0 +table_open_cache=30000 +max_connect_errors=2000 +innodb_adaptive_hash_index=0 + +mysqlx=0 +``` + +- /etc/my-2.cnf + +``` +[mysqld_safe] +log-error=/data/mysql-2/log/mysql.log +pid-file=/data/mysql-2/run/mysqld.pid + +[client] +socket=/data/mysql-2/run/mysql.sock +default-character-set=utf8 + +[mysqld] +server-id=3307 +#log-error=/data/mysql-2/log/mysql.log +#basedir=/usr/local/mysql +socket=/data/mysql-2/run/mysql.sock +tmpdir=/data/mysql-2/tmp +datadir=/data/mysql-2/data +default_authentication_plugin=mysql_native_password +port=3307 +user=root +#innodb_page_size=4k + +max_connections=2000 +back_log=4000 +performance_schema=OFF +max_prepared_stmt_count=128000 +#transaction_isolation=READ-COMMITTED +#skip-grant-tables + +#file +innodb_file_per_table +innodb_log_file_size=2048M +innodb_log_files_in_group=32 +innodb_open_files=10000 +table_open_cache_instances=64 + +#buffers +innodb_buffer_pool_size=150G # 根据系统内存大小可调整 +innodb_buffer_pool_instances=16 +innodb_log_buffer_size=2048M +#innodb_undo_log_truncate=OFF + +#tune +default_time_zone=+8:00 +#innodb_numa_interleave=1 +thread_cache_size=2000 +sync_binlog=1 +innodb_flush_log_at_trx_commit=1 +innodb_use_native_aio=1 +innodb_spin_wait_delay=180 +innodb_sync_spin_loops=25 +innodb_flush_method=O_DIRECT +innodb_io_capacity=30000 +innodb_io_capacity_max=40000 +innodb_lru_scan_depth=9000 +innodb_page_cleaners=16 +#innodb_spin_wait_pause_multiplier=25 + +#perf special +innodb_flush_neighbors=0 +innodb_write_io_threads=24 +innodb_read_io_threads=16 +innodb_purge_threads=32 + +sql_mode=STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION,NO_AUTO_VALUE_ON_ZERO,STRICT_ALL_TABLES + +log-bin=mysql-bin +#skip_log_bin # 开启mysql-bin +binlog_expire_logs_seconds=1800 # 根据需要生成的数据量能够维持长时间运行可调整 +ssl=0 +table_open_cache=30000 +max_connect_errors=2000 +innodb_adaptive_hash_index=0 + +mysqlx=0 +``` + +### 2.4 部署mysql + +```shell +#!/bin/bash +systemctl stop firewalld +systemctl disable irqbalance +echo 3 > /proc/sys/vm/drop_caches +mysql=mysql-8.0.20 +prepare_mysql_data() +{ + umount /dev/nvme0n1 + rm -rf /data + mkfs.xfs /dev/nvme0n1 -f + groupadd mysql + useradd -g mysql mysql + mkdir /data + mount /dev/nvme0n1 /data + mkdir -p /data/{mysql-1,mysql-2} + mkdir -p /data/mysql-1/{data,run,share,tmp,log} + mkdir -p /data/mysql-2/{data,run,share,tmp,log} + chown -R mysql:mysql /data + chown -R mysql:mysql /data/mysql-1 + chown -R mysql:mysql /data/mysql-2 + touch /data/mysql-1/log/mysql.log + touch /data/mysql-2/log/mysql.log + chown -R mysql:mysql /data/mysql-1/log/mysql.log + chown -R mysql:mysql /data/mysql-2/log/mysql.log +} +init_mysql() +{ + /usr/local/$mysql/bin/mysqld --defaults-file=/etc/my.cnf --user=root --initialize + /usr/local/$mysql/support-files/mysql.server start + sed -i 's/#skip-grant-tables/skip-grant-tables/g' /etc/my.cnf + /usr/local/$mysql/support-files/mysql.server restart + /usr/local/$mysql/bin/mysql -u root -p123456 < + +重启后，设置开启STEAL模式即可。 + +```shell +echo STEAL > /sys/kernel/debug/sched_features +``` + +### 4.2 关闭测试影响项 + +```shell +#关闭irqbalance +systemctl stop irqbalance.service +systemctl disable irqbalance.service + +#关闭防火墙 +systemctl stop iptables +systemctl stop firewalld +``` + +### 4.3 网卡中断绑核 + +```shell +#服务端绑中断(根据环境替换网卡名称、绑核cpu核) +ethtool -L enp4s0 combined 6 +irq1=`cat /proc/interrupts| grep -E enp4s0 | head -n5 | awk -F ':' '{print $1}'` +cpulist=(61 62 63 64 65 66) ## 自行根据环境调整为需要固定处理网卡中断请求的核 +c=0 +for irq in $irq1 +do +echo ${cpulist[c]} "->" $irq +echo ${cpulist[c]} > /proc/irq/$irq/smp_affinity_list +let "c++" +done +``` + +### 4.4 安装nvme-cli工具 + +nvme-cli是一个用于监控和配置管理NVMe设备的命令行工具。可用于开启NVMe SSD多流功能以及通过log相关的命令获取控制器的各类日志记录等功能。 + +```shell +yum install nvme-cli +``` + +### 4.5 开启NVMe 磁盘的多流特性 + +- 查询NVMe SSD磁盘当前的多流使能情况。 + + ```shell + nvme dir-receive /dev/nvme0n1 -n 0x1 -D 0 -O 1 -H + ``` + + + + + 回显结果表示，当前NVMe SSD支持Stream Directive，即支持开启多流特性，当前的状态为关闭状态。 + +- 启用多流功能。 + + ```shell + modprobe -r nvme + modprobe nvme-core streams=1 + modprobe nvme + ``` + +- 再次查询NVMe SSD磁盘当前的多流使能情况。 + + + + 回显结果表示，当前NVMe SSD已开启多流特性。 + +### 4.6 mysql双实例数据准备 + +为了统一基线测试和多流测试的流程，每次测试前格式化磁盘，统一从/bak目录下拷贝两份数据到两个mysql实例的对应数据目录下。 + +```shell +prepare_mysql_data() +{ + umount /dev/nvme0n1 + rm -rf /data + mkfs.xfs /dev/nvme0n1 -f + mkdir /data + mount /dev/nvme0n1 /data + mkdir -p /data/{mysql-1,mysql-2} + mkdir -p /data/mysql-1/{data,run,share,tmp,log} + mkdir -p /data/mysql-2/{data,run,share,tmp,log} + chown -R mysql:mysql /data + chown -R mysql:mysql /data/mysql-1 + chown -R mysql:mysql /data/mysql-2 + touch /data/mysql-1/log/mysql.log + touch /data/mysql-2/log/mysql.log + chown -R mysql:mysql /data/mysql-1/log/mysql.log + chown -R mysql:mysql /data/mysql-2/log/mysql.log +} + +prepare_mysql_data() +# 格式化后，创建mysql双实例对应的数据目录，即可启动astream +astream -i /data/mysql-1/data /data/mysql-2/data -r rule1.txt rule2.txt# ---->该步骤测试基线时请去掉 +cp -r /bak/* /data/mysql-1/data +cp -r /bak/* /data/mysql-2/data +``` + +然后，查看待测磁盘/dev/nvme0n1的空间占用率情况(`df -h`)是否在60%-70%左右即可。 + +### 4.7 绑核启动mysql服务 + +```shell +#启动mysql双实例 +numactl -C 0-60 -i 0-3 /usr/local/bin/mysqld --defaults-file=/etc/my-1.cnf & +numactl -C 67-127 -i 0-3 /usr/local/bin/mysqld --defaults-file=/etc/my-2.cnf & +``` + +### 4.8 设置定时任务 + +拷贝数据或生成数据成功后，在执行mysql测试前，为了衡量磁盘的写放大水平，在12小时(720mins)的测试过程中，每隔1小时利用定时器`crontab`执行计算磁盘WA的脚本`calculate_wa.sh`。 + +```shell +#!/bin/bash + +source /etc/profile +source ~/.bash_profile + +BASE_PATH=$(cd $(dirname $0);pwd) +diskName=$1 + +echo 0x`/usr/bin/nvme get-log /dev/${diskName}n1 -i 0xc0 -n 0xffffffff -l 800|grep "01c0:"|awk '{print $13$12$11$10$9$8$7$6}'` >> ${BASE_PATH}/host_tmp +echo 0x`/usr/bin/nvme get-log /dev/${diskName}n1 -i 0xc0 -n 0xffffffff -l 800|grep "01d0:"|awk '{print $9$8$7$6$5$4$3$2}'` >> ${BASE_PATH}/gc_tmp + +# IO write counts,unit:4K # +hostWriteHexSectorTemp=`tail -1 ${BASE_PATH}/host_tmp` +# GC write counts,unit 4k # +gcWriteHexSectorTemp=`tail -1 ${BASE_PATH}/gc_tmp` +hostWriteDecSectorTemp=`printf "%llu" ${hostWriteHexSectorTemp}` +gcWriteDecSectorTemp=`printf "%llu" ${gcWriteHexSectorTemp}` +preHostValue=`tail -2 ${BASE_PATH}/host_tmp|head -1` +preGcValue=`tail -2 ${BASE_PATH}/gc_tmp|head -1` +preHostValue=`printf "%llu" ${preHostValue}` +preGcValue=`printf "%llu" ${preGcValue}` + +# IO write counts for a period of time +hostWrittenSector=$(echo ${hostWriteDecSectorTemp}-${preHostValue} | bc -l) +# Gc write counts for a period of time +gcWrittenSector=$(echo ${gcWriteDecSectorTemp}-${preGcValue} | bc -l) +nandSector=$(echo ${hostWrittenSector}+${gcWrittenSector} | bc -l) + +# unit from kB->MB +hostWrittenMB=$((${hostWrittenSector}/256)) +nandWrittenMB=$((${nandSector}/256)) + +# compute the WA +WA=$(echo "scale=5;${nandSector}/${hostWrittenSector}" | bc) +echo $nandWrittenMB $hostWrittenMB $WA >> ${BASE_PATH}/result_WA.txt +``` + +可执行`crontab -e`，加入每隔1小时执行脚本命令的定时任务，命令如下： + +```shell +0 */1 * * * bash /root/calculate_wa.sh nvme0 +``` + +若所测的`NVMe`磁盘盘符名为`/dev/nvme0n1`，则定时任务中脚本的参数传入`nvme0`即可。 + +### 4.9 启动mysql双实例测试 + +执行测试前，确保已使能NVMe SSD磁盘的多流特性。 + +客户端工具根目录下执行测试如下： + +```shell +cd benchmarksql5.0-for-mysql +./runBenchmark.sh props.conf +./runBenchmark.sh props-2.conf +``` + +### 4.10 停止astream进程 + +本步骤测试基线时无需操作，若测试多流结束后，执行如下命令停止astream进程： + +```shell +astream stop +``` + +## 5 测试结果 + +使用定时器脚本的结果，在脚本所在目录的result_WA.txt下输出。每次测试结束后，从中选择最近12条非0数据即可。 + +当磁盘有数据写入时，result_WA.txt中每一行有三个值，其意义如下： + +- 1小时内磁盘实际写入的数据量。 +- 1小时内主机提交的写入量。 +- 当前的磁盘WA，根据附录中的公式即可算出每个小时内磁盘的WA水平。 + +**当前实测，在使用astream情况下，mysql长稳运行后期，NVMe SSD磁盘稳定保持12%的WA下降幅度，即性能较前提升12%**。 + +## 6 附录 + +**写入放大**（英语：Write amplification，简称**WA**）是闪存和固态硬盘（SSD）中一种不良的现象，即磁盘实际写入的数据量是写入数据量的多倍。其公式为： + +$$ +WA=\frac{磁盘实际写入的数据量}{主机提交的写入数据量} +$$ + +一般来说，随着数据的存储以及磁盘的碎片化愈演愈烈，WA的值将越来越大，如果WA的值能够延迟升高，那么将有助于延长磁盘的使用寿命。 \ No newline at end of file diff --git a/archive/uncertain/astream/figures/STEAL.png b/archive/uncertain/astream/figures/STEAL.png new file mode 100644 index 0000000000000000000000000000000000000000..3c6aeab2a69f76353eb02e8b10299488c995d4c1 Binary files /dev/null and b/archive/uncertain/astream/figures/STEAL.png differ diff --git "a/archive/uncertain/astream/figures/\345\244\232\346\265\201\347\211\271\346\200\247\345\267\262\344\275\277\350\203\275.PNG" "b/archive/uncertain/astream/figures/\345\244\232\346\265\201\347\211\271\346\200\247\345\267\262\344\275\277\350\203\275.PNG" new file mode 100644 index 0000000000000000000000000000000000000000..1590dff910f5a35542d2ac06c07addceaf7888bb Binary files /dev/null and "b/archive/uncertain/astream/figures/\345\244\232\346\265\201\347\211\271\346\200\247\345\267\262\344\275\277\350\203\275.PNG" differ diff --git "a/archive/uncertain/astream/figures/\345\244\232\346\265\201\347\211\271\346\200\247\346\234\252\344\275\277\350\203\275.PNG" "b/archive/uncertain/astream/figures/\345\244\232\346\265\201\347\211\271\346\200\247\346\234\252\344\275\277\350\203\275.PNG" new file mode 100644 index 0000000000000000000000000000000000000000..8f08500a44f1dc658870e5aa3702defee004d3b4 Binary files /dev/null and "b/archive/uncertain/astream/figures/\345\244\232\346\265\201\347\211\271\346\200\247\346\234\252\344\275\277\350\203\275.PNG" differ diff --git "a/archive/uncertain/astream/figures/\351\203\250\347\275\262.png" "b/archive/uncertain/astream/figures/\351\203\250\347\275\262.png" new file mode 100644 index 0000000000000000000000000000000000000000..d5ff9f187348eaecd17f2c1b380bb782a38710ad Binary files /dev/null and "b/archive/uncertain/astream/figures/\351\203\250\347\275\262.png" differ diff --git a/archive/uncertain/astream/overview.md b/archive/uncertain/astream/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..82e079959a6890b10f24541b8c307a2ac8041e3c --- /dev/null +++ b/archive/uncertain/astream/overview.md @@ -0,0 +1,3 @@ +# astream用户指南 + +astream是一款延长磁盘使用寿命的工具。它基于Linux提供的inotify机制进行目录监控，同时配合用户针对应用场景定义的流分配规则，为匹配到的文件在创建时设置流信息，而后通过内核透传流信息至已使用多流特性的NVMe SSD磁盘，最终使得文件的存储能够根据流信息的标识进行更优的分类存储，降低磁盘垃圾回收的工作量，从而降低磁盘的写放大水平，最终达到延长磁盘使用寿命的效果。专注于具有明显具有相同或相似生命周期的workload特征的数据库场景应用，如MySQL。 \ No newline at end of file diff --git "a/archive/uncertain/astream/\345\256\211\350\243\205\344\270\216\344\275\277\347\224\250\346\226\271\346\263\225.md" "b/archive/uncertain/astream/\345\256\211\350\243\205\344\270\216\344\275\277\347\224\250\346\226\271\346\263\225.md" new file mode 100644 index 0000000000000000000000000000000000000000..12de748c08c00925a8894526b7f6487d24218371 --- /dev/null +++ "b/archive/uncertain/astream/\345\256\211\350\243\205\344\270\216\344\275\277\347\224\250\346\226\271\346\263\225.md" @@ -0,0 +1,107 @@ +# astream + +## 简介 + +astream是一款延长磁盘使用寿命的工具。它基于Linux提供的inotify机制进行目录监控，同时配合用户针对应用场景定义的流分配规则，为匹配到的文件在创建时设置流信息，而后通过内核透传流信息至已使能多流特性的NVMe SSD磁盘，最终使得文件的存储能够根据流信息的标识进行更优的分类存储，降低磁盘垃圾回收的工作量，从而降低磁盘的写放大水平，最终达到延长磁盘使用寿命。专注于具有明显具有相同或相似生命周期的workload特征的数据库场景应用，如MySQL。 + +## 安装 + +配置openEuler 25.03 的yum源，直接使用yum命令安装 + +```bash +yum install astream +``` + +## 使用方法 + +在介绍astream的使用方法前，首先介绍astream启动所需要的流分配规则文件。 + +### 流分配规则文件与完整示例 + +#### 1.介绍 + +流分配规则文件是用户根据自身对workload特征的理解，针对其中具有相同或相似生命周期的一类数据定义的流信息规则集合的文件。 + +在一个流分配规则文件中，每一行表示定义的一条规则，每条规则示例如下： `^/data/mysql/data/undo 4` 。它表示为路径`/data/mysql/data`下的所有以`undo`为前缀的文件都分配流信息4。 + +#### 2.完整示例 + +如下示例为一个具体的MySQL的流分配规则文件。 + +```bash +^/data/mysql/data/ib_logfile 2 +^/data/mysql/data/ibdata1$ 3 +^/data/mysql/data/undo 4 +^/data/mysql/data/mysql-bin 5 +``` + +该规则文件定义了如下四条规则： + +- 以`/data/mysql/data/ib_logfile`为前缀的文件绝对路径对应的文件配置流信息2 + +- 以`/data/mysql/data/ibdata1`为文件绝对路径对应的文件配置流信息3 + +- 以`/data/mysql/data/undo`为前缀的文件绝对路径对应的文件配置流信息4 + +- 以`/data/mysql/data/mysql-bin`为前缀的文件绝对路径对应的文件配置流信息5 + +## 使用说明 + +### 启动astream守护进程 + +假设规则文件`stream_rule1.txt`和`stream_rule2.txt` 位于`/home`下，则 + +- 监控单目录 + + ```shell + astream -i /data/mysql/data -r /home/stream_rule1.txt + ``` + +- 监控多目录 + + 本工具支持同时监控多个目录，即每个监控目录都需要传入与之匹配的流分配规则文件。 + + 如下示例同时监控两个目录： + + ```shell + astream -i /data/mysql-1/data /data/mysql-2/data -r /home/stream_rule1.txt /home/stream_rule2.txt + ``` + +上述命令中监控以下两个目录： + +- 目录1`/data/mysql-1/data`，对应的流分配规则文件为`/home/stream_rule1.txt`。 +- 目录2`/data/mysql-2/data`，对应的流分配规则文件为`/home/stream_rule2.txt`。 + +## 命令行参数说明 + +```shell +astream [options] +``` + +| 选项 | 选项含义 | 示例 | +| ---- | ------------------------------------------------------------ | ---------------------------------------------------- | +| -h | 显示astream使用说明 | `astream -h` | +| -l | 设置astream监控过程中的日志显示级别, debug(1), info(2), warn(3), error(4) | `astream -i /data/mysql/data -r /home/rule.txt -l 2` | +| -i | 指定当前需要监控的目录，多个目录可以以空格间隔输入 | 该参数配合-r使用, 示例见下 | +| -r | 指定监控目录对应的流分配规则文件，传入的个数取决于-i，互相对应 | `astream -i /data/mysql/data -r /home/rule.txt` | +| stop | 正常停止astream守护进程 | `astream stop` | + +## 约束限制 + +使用astream存在一些约束限制： + +### 功能约束 + +- 仅支持实现了NVMe协议的多流(multi-stream)特性的NVMe SSD。 +- 仅支持分配5个流，其约束受内核定义的常量`BLK_MAX_WRITE_HINTS`所约束，以及NVMe SSD支持的最大流个数有关。 + +### 操作约束 + +- 提供的命令行以root权限运行astream守护进程，在测试过程中，常驻系统。 +- 待测磁盘的IO压力需要足够大，且磁盘空间占用较高，因此写放大恶化越明显，使用本工具的多流收益也就越明显。 + +## 注意事项 + +- 在启动astream守护进程后，不能删除受监控的目录然后创建相同目录，需要重新启动astream； +- 规则文件中支持正则匹配一组文件，用户需具备一定的正则匹配知识。 +- 测试时的NVMe SSD磁盘实现了基于NVMe 1.3协议描述的多流功能。 diff --git a/archive/uncertain/desktop/Install_Cinnamon.md b/archive/uncertain/desktop/Install_Cinnamon.md new file mode 100644 index 0000000000000000000000000000000000000000..561720f01019f52d37e458723c639f23b49c9534 --- /dev/null +++ b/archive/uncertain/desktop/Install_Cinnamon.md @@ -0,0 +1,73 @@ +# 在 openEuler 上安装 Cinnamon + +Cinnamon是运行在类Unix操作系统中最常用的桌面环境，也是一个功能完善、操作简单、界面友好、集使用和开发为一身的桌面环境，还是GNU计划的正式桌面。 + +从用户的角度看，Cinnamon是一个集成桌面环境和应用程序的套件。从开发者的角度看，Cinnamon是一个应用程序开发框架(由数目众多的实用函数库组成)。即使用户不运行Cinnamon桌面环境，用Cinnamon编写的应用程序也可以正常运行。 + +Cinnamon既包含文件管理器，应用商店，文本编辑器等基础软件，也包含系统采样分析，系统日志，软件工程IDE，web浏览器，简洁虚拟机监视器，开发者文档浏览器等高级应用和工具。 + +安装时，建议新建一个管理员用户。 + +## 1. 配置源并更新系统 + +[下载](https://openeuler.org/zh/download/) openEuler ISO镜像并安装系统，更新软件源（需要配置Everything源，以及EPOL源，如下命令是在最小化安装系统的情况下安装Cinnamon） + +```sh +sudo dnf update +``` + +## 2. 安装字库 + +```sh +sudo dnf install dejavu-fonts liberation-fonts gnu-*-fonts google-*-fonts +``` + +## 3. 安装Xorg + +```sh +sudo dnf install xorg-* +``` + +安装时，可能会安装无用的包，可使用如下命令安装必要的xorg相关包。 + +```sh +sudo dnf install xorg-x11-apps xorg-x11-drivers xorg-x11-drv-ati \ + xorg-x11-drv-dummy xorg-x11-drv-evdev xorg-x11-drv-fbdev xorg-x11-drv-intel \ + xorg-x11-drv-libinput xorg-x11-drv-nouveau xorg-x11-drv-qxl \ + xorg-x11-drv-synaptics-legacy xorg-x11-drv-v4l xorg-x11-drv-vesa \ + xorg-x11-drv-vmware xorg-x11-drv-wacom xorg-x11-fonts xorg-x11-fonts-others \ + xorg-x11-font-utils xorg-x11-server xorg-x11-server-utils xorg-x11-server-Xephyr \ + xorg-x11-server-Xspice xorg-x11-util-macros xorg-x11-utils xorg-x11-xauth \ + xorg-x11-xbitmaps xorg-x11-xinit xorg-x11-xkb-utils +``` + +## 4. 安装Cinnamon及组件 + +```sh +sudo dnf install cinnamon cinnamon-control-center cinnamon-desktop \ + cinnamon-menus cinnamon-screensaver cinnamon-session \ + cinnamon-settings-daemon cinnamon-themes cjs \ + nemo nemo-extensions muffin cinnamon-translations inxi \ + perl-XML-Dumper xapps mint-x-icons mint-y-icons mintlocale \ + python3-plum-py caribou mozjs78 python3-pam \ + python3-tinycss2 python3-xapp tint2 gnome-terminal \ + lightdm lightdm-gtk +``` + +## 5. 开机自动启动登录管理器 + +```sh +sudo systemctl enable lightdm +``` + +## 6. 设置系统默认以图形界面登录 + +```sh +sudo systemctl set-default graphical.target +``` + +重启验证 + +```sh +sudo reboot +``` diff --git a/archive/uncertain/desktop/desktop.md b/archive/uncertain/desktop/desktop.md new file mode 100644 index 0000000000000000000000000000000000000000..c6fec706be46c109d4a6bb176952410d55b8a475 --- /dev/null +++ b/archive/uncertain/desktop/desktop.md @@ -0,0 +1,3 @@ +# 桌面环境用户指南 + +本文介绍四种常用的桌面环境（UKUI、DDE、XFCE、GNOME）的安装和使用方法，它们提供美观易用、安全可靠的图形化操作界面，给用户带来更好的体验。 diff --git a/archive/uncertain/desktop/figures/38.png b/archive/uncertain/desktop/figures/38.png new file mode 100644 index 0000000000000000000000000000000000000000..838f5ff0616a83cdf42edb053f4e72b93bfa644e Binary files /dev/null and b/archive/uncertain/desktop/figures/38.png differ diff --git a/archive/uncertain/desktop/figures/39.png b/archive/uncertain/desktop/figures/39.png new file mode 100644 index 0000000000000000000000000000000000000000..12a379403d73a47b2fa564120a28fdb58d188963 Binary files /dev/null and b/archive/uncertain/desktop/figures/39.png differ diff --git a/archive/uncertain/desktop/figures/40.png b/archive/uncertain/desktop/figures/40.png new file mode 100644 index 0000000000000000000000000000000000000000..bf419894eab852b45604966c62fafa71f051c4df Binary files /dev/null and b/archive/uncertain/desktop/figures/40.png differ diff --git a/archive/uncertain/desktop/figures/41.png b/archive/uncertain/desktop/figures/41.png new file mode 100644 index 0000000000000000000000000000000000000000..f94b0ee72e0d4e9277e9b44b4268cfbdb8402104 Binary files /dev/null and b/archive/uncertain/desktop/figures/41.png differ diff --git a/archive/uncertain/desktop/figures/42.png b/archive/uncertain/desktop/figures/42.png new file mode 100644 index 0000000000000000000000000000000000000000..3182e551c4e4b03885bad6339f1de514b3f55f8c Binary files /dev/null and b/archive/uncertain/desktop/figures/42.png differ diff --git a/archive/uncertain/desktop/figures/43.jpg b/archive/uncertain/desktop/figures/43.jpg new file mode 100644 index 0000000000000000000000000000000000000000..26e9244f58ea9800081fd61ae135477f05b21b40 Binary files /dev/null and b/archive/uncertain/desktop/figures/43.jpg differ diff --git a/archive/uncertain/desktop/figures/44.png b/archive/uncertain/desktop/figures/44.png new file mode 100644 index 0000000000000000000000000000000000000000..c3abaecd6e053272d81e0ad9bd183c6858b4f3c5 Binary files /dev/null and b/archive/uncertain/desktop/figures/44.png differ diff --git a/archive/uncertain/desktop/figures/45.png b/archive/uncertain/desktop/figures/45.png new file mode 100644 index 0000000000000000000000000000000000000000..86b051acde857c88479714414f721a7f59cca483 Binary files /dev/null and b/archive/uncertain/desktop/figures/45.png differ diff --git a/archive/uncertain/desktop/figures/46.png b/archive/uncertain/desktop/figures/46.png new file mode 100644 index 0000000000000000000000000000000000000000..d8ec41c87628bf28c9905523f99ae93aebd13614 Binary files /dev/null and b/archive/uncertain/desktop/figures/46.png differ diff --git a/archive/uncertain/desktop/figures/47.jpg b/archive/uncertain/desktop/figures/47.jpg new file mode 100644 index 0000000000000000000000000000000000000000..bf95f03c8ea0f84a878bc63af20972c9da71bc04 Binary files /dev/null and b/archive/uncertain/desktop/figures/47.jpg differ diff --git a/archive/uncertain/desktop/figures/48.png b/archive/uncertain/desktop/figures/48.png new file mode 100644 index 0000000000000000000000000000000000000000..ef21fa1ce1e2e9848a8dca16e692de673df7c6d7 Binary files /dev/null and b/archive/uncertain/desktop/figures/48.png differ diff --git a/archive/uncertain/desktop/figures/49.png b/archive/uncertain/desktop/figures/49.png new file mode 100644 index 0000000000000000000000000000000000000000..3b77668e5a4d1bdb3043c473dff9b36fa7144714 Binary files /dev/null and b/archive/uncertain/desktop/figures/49.png differ diff --git a/archive/uncertain/desktop/figures/50.png b/archive/uncertain/desktop/figures/50.png new file mode 100644 index 0000000000000000000000000000000000000000..b86a55fe4363f56fc18befc9d27025a75ca427ad Binary files /dev/null and b/archive/uncertain/desktop/figures/50.png differ diff --git a/archive/uncertain/desktop/figures/51.png b/archive/uncertain/desktop/figures/51.png new file mode 100644 index 0000000000000000000000000000000000000000..d427ac871dba9c32eb4ffe736d5352f8408da533 Binary files /dev/null and b/archive/uncertain/desktop/figures/51.png differ diff --git a/archive/uncertain/desktop/figures/52.png b/archive/uncertain/desktop/figures/52.png new file mode 100644 index 0000000000000000000000000000000000000000..0ca0a2db05c70bc25f9bb59e82d074f671cfc74e Binary files /dev/null and b/archive/uncertain/desktop/figures/52.png differ diff --git a/archive/uncertain/desktop/figures/53.png b/archive/uncertain/desktop/figures/53.png new file mode 100644 index 0000000000000000000000000000000000000000..76fbc34a1d5621b83c2d8c93222766acad33350d Binary files /dev/null and b/archive/uncertain/desktop/figures/53.png differ diff --git a/archive/uncertain/desktop/figures/54.png b/archive/uncertain/desktop/figures/54.png new file mode 100644 index 0000000000000000000000000000000000000000..49ecae6f8941a118223f3765c23015df074c4983 Binary files /dev/null and b/archive/uncertain/desktop/figures/54.png differ diff --git a/archive/uncertain/desktop/figures/56.png b/archive/uncertain/desktop/figures/56.png new file mode 100644 index 0000000000000000000000000000000000000000..36fee795bfe593b6246c8d6c2bddea9386b06f45 Binary files /dev/null and b/archive/uncertain/desktop/figures/56.png differ diff --git a/archive/uncertain/desktop/figures/57.png b/archive/uncertain/desktop/figures/57.png new file mode 100644 index 0000000000000000000000000000000000000000..539d06b77b058a933cb154c43641d498050986e0 Binary files /dev/null and b/archive/uncertain/desktop/figures/57.png differ diff --git a/archive/uncertain/desktop/figures/58.png b/archive/uncertain/desktop/figures/58.png new file mode 100644 index 0000000000000000000000000000000000000000..396ca16d873e54505bcdbd41d669366eea7f5dee Binary files /dev/null and b/archive/uncertain/desktop/figures/58.png differ diff --git a/archive/uncertain/desktop/figures/59.png b/archive/uncertain/desktop/figures/59.png new file mode 100644 index 0000000000000000000000000000000000000000..9b1de98ac4fe686937ca844d3e9481548a79ce63 Binary files /dev/null and b/archive/uncertain/desktop/figures/59.png differ diff --git a/archive/uncertain/desktop/figures/60.jpg b/archive/uncertain/desktop/figures/60.jpg new file mode 100644 index 0000000000000000000000000000000000000000..033c88aaadd04f7d4058ec2eb5b2c70498319bf7 Binary files /dev/null and b/archive/uncertain/desktop/figures/60.jpg differ diff --git a/archive/uncertain/desktop/figures/61.png b/archive/uncertain/desktop/figures/61.png new file mode 100644 index 0000000000000000000000000000000000000000..8df17062963a3baf92318a12ec34b1378122687b Binary files /dev/null and b/archive/uncertain/desktop/figures/61.png differ diff --git a/archive/uncertain/desktop/figures/62.png b/archive/uncertain/desktop/figures/62.png new file mode 100644 index 0000000000000000000000000000000000000000..ec312d6c0c22018c1745dd866da71ce9be47fbda Binary files /dev/null and b/archive/uncertain/desktop/figures/62.png differ diff --git a/archive/uncertain/desktop/figures/63.jpg b/archive/uncertain/desktop/figures/63.jpg new file mode 100644 index 0000000000000000000000000000000000000000..504f7cf59768f6fd1cd73a115d01fbc4e15a02e1 Binary files /dev/null and b/archive/uncertain/desktop/figures/63.jpg differ diff --git a/archive/uncertain/desktop/figures/63.png b/archive/uncertain/desktop/figures/63.png new file mode 100644 index 0000000000000000000000000000000000000000..86b051acde857c88479714414f721a7f59cca483 Binary files /dev/null and b/archive/uncertain/desktop/figures/63.png differ diff --git a/archive/uncertain/desktop/figures/64.png b/archive/uncertain/desktop/figures/64.png new file mode 100644 index 0000000000000000000000000000000000000000..cbbd2ede047e735c3766e08b04595f08cd72f5b2 Binary files /dev/null and b/archive/uncertain/desktop/figures/64.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-01.png b/archive/uncertain/desktop/figures/Cinnamon-01.png new file mode 100644 index 0000000000000000000000000000000000000000..b4a43e7fa938b2ece73ad749e2b513daa976e7c9 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-01.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-02.png b/archive/uncertain/desktop/figures/Cinnamon-02.png new file mode 100644 index 0000000000000000000000000000000000000000..22413a83d51cb9ee177c0d39147da857868f0aec Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-02.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-03.png b/archive/uncertain/desktop/figures/Cinnamon-03.png new file mode 100644 index 0000000000000000000000000000000000000000..5ccc6d4eef17f2d39841046dcf32b9c00652d1a9 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-03.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-04.png b/archive/uncertain/desktop/figures/Cinnamon-04.png new file mode 100644 index 0000000000000000000000000000000000000000..c5d7073a3d2a37727b83a6e43a684747175efa9d Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-04.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-05.png b/archive/uncertain/desktop/figures/Cinnamon-05.png new file mode 100644 index 0000000000000000000000000000000000000000..e2d0a0a523f10d6bce9f51c5d05f019c595e2625 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-05.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-06.png b/archive/uncertain/desktop/figures/Cinnamon-06.png new file mode 100644 index 0000000000000000000000000000000000000000..61bb128fc2c8902edbfd1d76b28f963817753e32 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-06.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-07.png b/archive/uncertain/desktop/figures/Cinnamon-07.png new file mode 100644 index 0000000000000000000000000000000000000000..ef01eb0001c6db0e3d1c024e51b0594372b9348c Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-07.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-08.png b/archive/uncertain/desktop/figures/Cinnamon-08.png new file mode 100644 index 0000000000000000000000000000000000000000..1049f355939b0121c123adc462dcb6d736e48760 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-08.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-09.png b/archive/uncertain/desktop/figures/Cinnamon-09.png new file mode 100644 index 0000000000000000000000000000000000000000..6dc110900f5375ed9ede276f59ea89ba29e5e737 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-09.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-10.png b/archive/uncertain/desktop/figures/Cinnamon-10.png new file mode 100644 index 0000000000000000000000000000000000000000..33dda6e0d0497c1589743c19a8d041a775b7bb7f Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-10.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-11.png b/archive/uncertain/desktop/figures/Cinnamon-11.png new file mode 100644 index 0000000000000000000000000000000000000000..98570f04a066d550b5971afc94ee1c63d24f6275 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-11.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-12.png b/archive/uncertain/desktop/figures/Cinnamon-12.png new file mode 100644 index 0000000000000000000000000000000000000000..56cc0446efd9d72d97905296fd6f19e9e2ac4047 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-12.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-13.png b/archive/uncertain/desktop/figures/Cinnamon-13.png new file mode 100644 index 0000000000000000000000000000000000000000..a3191cc6b2bb2e418353b76bcf645be954655a20 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-13.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-14.png b/archive/uncertain/desktop/figures/Cinnamon-14.png new file mode 100644 index 0000000000000000000000000000000000000000..d673b9d12c8fb5ccaa0b0f3a35b85f939f1040c8 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-14.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-15.png b/archive/uncertain/desktop/figures/Cinnamon-15.png new file mode 100644 index 0000000000000000000000000000000000000000..518c3dca4b9b58f45f7aee5ef974c3dd41804e1d Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-15.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-16.png b/archive/uncertain/desktop/figures/Cinnamon-16.png new file mode 100644 index 0000000000000000000000000000000000000000..17ac8544dab141ddc8d7d98f1712757efedb5531 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-16.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-17.png b/archive/uncertain/desktop/figures/Cinnamon-17.png new file mode 100644 index 0000000000000000000000000000000000000000..07a62594a1acadceeeaabe0e7cbe63c192f0ab37 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-17.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-18.png b/archive/uncertain/desktop/figures/Cinnamon-18.png new file mode 100644 index 0000000000000000000000000000000000000000..d088bb1075ebd2c76304c5bd400a3892d401dfa0 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-18.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-19.png b/archive/uncertain/desktop/figures/Cinnamon-19.png new file mode 100644 index 0000000000000000000000000000000000000000..65198c16e3bdf0b948ba8da1d05943ea87065dfc Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-19.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-20.png b/archive/uncertain/desktop/figures/Cinnamon-20.png new file mode 100644 index 0000000000000000000000000000000000000000..ac4c66887846509474cd57740079d396f5ffee64 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-20.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-21.png b/archive/uncertain/desktop/figures/Cinnamon-21.png new file mode 100644 index 0000000000000000000000000000000000000000..ecc80c0ee42385907cea276726c891aab06f5ea2 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-21.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-22.png b/archive/uncertain/desktop/figures/Cinnamon-22.png new file mode 100644 index 0000000000000000000000000000000000000000..453a1b96f69ca37cf648e1bfd8a247cbd63f7f1f Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-22.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-23.png b/archive/uncertain/desktop/figures/Cinnamon-23.png new file mode 100644 index 0000000000000000000000000000000000000000..3290e73af52f1e88a435e925d6a17d21e78e287c Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-23.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-24.png b/archive/uncertain/desktop/figures/Cinnamon-24.png new file mode 100644 index 0000000000000000000000000000000000000000..825e58ddc9ec0027f0ff94b1d327eaff3a145357 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-24.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-25.png b/archive/uncertain/desktop/figures/Cinnamon-25.png new file mode 100644 index 0000000000000000000000000000000000000000..7b3cdbe8e85b55dc9ee92569f6d668c9defc76eb Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-25.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-26.png b/archive/uncertain/desktop/figures/Cinnamon-26.png new file mode 100644 index 0000000000000000000000000000000000000000..0d696fa8cbd18910bb16ca2c14996fefb5f0cb79 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-26.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-27.png b/archive/uncertain/desktop/figures/Cinnamon-27.png new file mode 100644 index 0000000000000000000000000000000000000000..7312579e88c211b656a1b6e339373abfca7c8984 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-27.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-28.png b/archive/uncertain/desktop/figures/Cinnamon-28.png new file mode 100644 index 0000000000000000000000000000000000000000..e5cf7e56e5663768e4f2698c77aeaa69c899b291 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-28.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-29.png b/archive/uncertain/desktop/figures/Cinnamon-29.png new file mode 100644 index 0000000000000000000000000000000000000000..d796cb8d80a60281a7f67ec494c76ad14d06ac1b Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-29.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-30-0.png b/archive/uncertain/desktop/figures/Cinnamon-30-0.png new file mode 100644 index 0000000000000000000000000000000000000000..8ea95ca410376dbc26729d0dec8bfc171911e960 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-30-0.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-30-1.png b/archive/uncertain/desktop/figures/Cinnamon-30-1.png new file mode 100644 index 0000000000000000000000000000000000000000..730e9bdba19949c6e118c237af302b492f3d41b8 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-30-1.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-31.png b/archive/uncertain/desktop/figures/Cinnamon-31.png new file mode 100644 index 0000000000000000000000000000000000000000..f80f3c86b92e6e00bf76c0101ce52af503d9b05c Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-31.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-32.png b/archive/uncertain/desktop/figures/Cinnamon-32.png new file mode 100644 index 0000000000000000000000000000000000000000..ed8f74fc04472e45ae6c76a7c2507da5dec28263 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-32.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-33.png b/archive/uncertain/desktop/figures/Cinnamon-33.png new file mode 100644 index 0000000000000000000000000000000000000000..a90dda9e151f9ca48ff6c296ff62676b22a191ae Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-33.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-34.png b/archive/uncertain/desktop/figures/Cinnamon-34.png new file mode 100644 index 0000000000000000000000000000000000000000..77c74765bb7116bf8a95b0164cb1de2d9032e56e Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-34.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-35-0.png b/archive/uncertain/desktop/figures/Cinnamon-35-0.png new file mode 100644 index 0000000000000000000000000000000000000000..4321c9e9a52bcf99bde6d72fae68647ab13510b0 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-35-0.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-35-1.png b/archive/uncertain/desktop/figures/Cinnamon-35-1.png new file mode 100644 index 0000000000000000000000000000000000000000..e6f75a945fc73d5478c6515498c7a6a4781ca04f Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-35-1.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-36.png b/archive/uncertain/desktop/figures/Cinnamon-36.png new file mode 100644 index 0000000000000000000000000000000000000000..a832fe2b440079f53775a2ba8c3115f57a89ab2c Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-36.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-37.png b/archive/uncertain/desktop/figures/Cinnamon-37.png new file mode 100644 index 0000000000000000000000000000000000000000..f091a5e910e7cb16dadd311de1100f8830a0eaf7 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-37.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-38.png b/archive/uncertain/desktop/figures/Cinnamon-38.png new file mode 100644 index 0000000000000000000000000000000000000000..7703dc11f1be241d9fe7e30452e7431c925833d3 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-38.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-39.png b/archive/uncertain/desktop/figures/Cinnamon-39.png new file mode 100644 index 0000000000000000000000000000000000000000..abf29f6ee9aaf13ab1f668a787d459a5ab0cb20c Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-39.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-40.png b/archive/uncertain/desktop/figures/Cinnamon-40.png new file mode 100644 index 0000000000000000000000000000000000000000..7c8ee5f4e78b46a0d762be06f96725ecef5d09b2 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-40.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-41-0.png b/archive/uncertain/desktop/figures/Cinnamon-41-0.png new file mode 100644 index 0000000000000000000000000000000000000000..56ea86556624602f4496b4079335245ac8c875a3 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-41-0.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-41-1.png b/archive/uncertain/desktop/figures/Cinnamon-41-1.png new file mode 100644 index 0000000000000000000000000000000000000000..44afad705b026dd69a9d2d7bf2ef35610720b85d Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-41-1.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-42.png b/archive/uncertain/desktop/figures/Cinnamon-42.png new file mode 100644 index 0000000000000000000000000000000000000000..8270b69ca590c1831fbe54e2d08ced55bccef89d Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-42.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-43.png b/archive/uncertain/desktop/figures/Cinnamon-43.png new file mode 100644 index 0000000000000000000000000000000000000000..48259addbdb8cf18303d2afbcd6cbad77deaf141 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-43.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-44.png b/archive/uncertain/desktop/figures/Cinnamon-44.png new file mode 100644 index 0000000000000000000000000000000000000000..e35a4acce457834d4d8608ee7fba783d596548e2 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-44.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-45.png b/archive/uncertain/desktop/figures/Cinnamon-45.png new file mode 100644 index 0000000000000000000000000000000000000000..d6d5e88587e26552ab4ab4d323eea0ae5ce721d2 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-45.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-46.png b/archive/uncertain/desktop/figures/Cinnamon-46.png new file mode 100644 index 0000000000000000000000000000000000000000..8e41651298319f1b72c3dce5b09cb1323c9a15a7 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-46.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-47.png b/archive/uncertain/desktop/figures/Cinnamon-47.png new file mode 100644 index 0000000000000000000000000000000000000000..eaa54608d956576555603d64ba72e374f147a315 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-47.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-48.png b/archive/uncertain/desktop/figures/Cinnamon-48.png new file mode 100644 index 0000000000000000000000000000000000000000..1ca6cb7b39ab375a69b95a7dfa02fa3acd8bb299 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-48.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-49.png b/archive/uncertain/desktop/figures/Cinnamon-49.png new file mode 100644 index 0000000000000000000000000000000000000000..07fe6abf2ca2d7e8dd5184c760f416d094c4629d Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-49.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-50.png b/archive/uncertain/desktop/figures/Cinnamon-50.png new file mode 100644 index 0000000000000000000000000000000000000000..c5452b655b9100c7d144d9d6e923f29664998ca0 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-50.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-51.png b/archive/uncertain/desktop/figures/Cinnamon-51.png new file mode 100644 index 0000000000000000000000000000000000000000..72644867adbb64db4503ff5345425a32bf1cae6d Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-51.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-52.png b/archive/uncertain/desktop/figures/Cinnamon-52.png new file mode 100644 index 0000000000000000000000000000000000000000..571f6ac34edf149cc1e5bd30a875e4148dd4fdd3 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-52.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-53.png b/archive/uncertain/desktop/figures/Cinnamon-53.png new file mode 100644 index 0000000000000000000000000000000000000000..1c7e9051ca448b017e13c6cbe6be66d2f83475c5 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-53.png differ diff --git a/archive/uncertain/desktop/figures/Cinnamon-54.png b/archive/uncertain/desktop/figures/Cinnamon-54.png new file mode 100644 index 0000000000000000000000000000000000000000..1b61db111ebdcb9bde1bff1cc08a2daed79a9f19 Binary files /dev/null and b/archive/uncertain/desktop/figures/Cinnamon-54.png differ diff --git a/archive/uncertain/desktop/figures/dde-1.png b/archive/uncertain/desktop/figures/dde-1.png new file mode 100644 index 0000000000000000000000000000000000000000..fb1d5177c39262ed182f10a57fdae850d007eeb1 Binary files /dev/null and b/archive/uncertain/desktop/figures/dde-1.png differ diff --git a/archive/uncertain/desktop/figures/dde-2.png b/archive/uncertain/desktop/figures/dde-2.png new file mode 100644 index 0000000000000000000000000000000000000000..be5d296937bd17b9646b32c80934aa76738027af Binary files /dev/null and b/archive/uncertain/desktop/figures/dde-2.png differ diff --git a/archive/uncertain/desktop/figures/icon101-o.svg b/archive/uncertain/desktop/figures/icon101-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..af1c5d3dc0277a6ea59e71efb6ca97bdfc782e8e --- /dev/null +++ b/archive/uncertain/desktop/figures/icon101-o.svg @@ -0,0 +1,3 @@ + + + diff --git a/archive/uncertain/desktop/figures/icon103-o.svg b/archive/uncertain/desktop/figures/icon103-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..c06c885725c569ab8db1fe7d595a7c65f18c5142 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon103-o.svg @@ -0,0 +1,26 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon105-o.svg b/archive/uncertain/desktop/figures/icon105-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..36c49949fa569330b761c2d65518f36c10435508 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon105-o.svg @@ -0,0 +1,111 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon107-o.svg b/archive/uncertain/desktop/figures/icon107-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..fb5a3ea756f6ccb7b3e5c31122a433347a908c96 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon107-o.svg @@ -0,0 +1,50 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon110-o.svg b/archive/uncertain/desktop/figures/icon110-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..7958e3f192061592e002e1e8a1bad06ffa86742c --- /dev/null +++ b/archive/uncertain/desktop/figures/icon110-o.svg @@ -0,0 +1,12 @@ + + + + reboot_normal + Created with Sketch. + + + + + + + \ No newline at end of file diff --git a/archive/uncertain/desktop/figures/icon111-o.svg b/archive/uncertain/desktop/figures/icon111-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..097d16a08d305a8b3f3b2268ab1ea8342e799377 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon111-o.svg @@ -0,0 +1,13 @@ + + + + Right + Created with Sketch. + + + + + + + + \ No newline at end of file diff --git a/archive/uncertain/desktop/figures/icon112-o.svg b/archive/uncertain/desktop/figures/icon112-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..e51628c2b8b10495f3410d219814286696ea2fd5 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon112-o.svg @@ -0,0 +1,15 @@ + + + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon116-o.svg b/archive/uncertain/desktop/figures/icon116-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..4d79cd6dbbbfd3969f4e0ad0ad88e27398853505 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon116-o.svg @@ -0,0 +1,72 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon120-o.svg b/archive/uncertain/desktop/figures/icon120-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..e895c347d16a200aea46b00428b0b9f1a3c94246 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon120-o.svg @@ -0,0 +1,49 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon122-o.svg b/archive/uncertain/desktop/figures/icon122-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..7fb014b5fd6097ca37a84d0b6a27dc982d675c8a --- /dev/null +++ b/archive/uncertain/desktop/figures/icon122-o.svg @@ -0,0 +1,3 @@ + + + diff --git a/archive/uncertain/desktop/figures/icon124-o.svg b/archive/uncertain/desktop/figures/icon124-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..960c0ec096c925213f8953398f0e8e5db3cdaed3 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon124-o.svg @@ -0,0 +1,55 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon125-o.svg b/archive/uncertain/desktop/figures/icon125-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..011c05f4b8f296867cd408a339230323fcbb28dd --- /dev/null +++ b/archive/uncertain/desktop/figures/icon125-o.svg @@ -0,0 +1,9 @@ + + + tips + + + + + + \ No newline at end of file diff --git a/archive/uncertain/desktop/figures/icon126-o.svg b/archive/uncertain/desktop/figures/icon126-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..e0a43b6b8beb434090ac0dd3a8fd68c023f11fce --- /dev/null +++ b/archive/uncertain/desktop/figures/icon126-o.svg @@ -0,0 +1,68 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon127-o.svg b/archive/uncertain/desktop/figures/icon127-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..bed95d35334a8d0151211054236c0bacddcc0dd3 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon127-o.svg @@ -0,0 +1,13 @@ + + + + Up + Created with Sketch. + + + + + + + + \ No newline at end of file diff --git a/archive/uncertain/desktop/figures/icon128-o.svg b/archive/uncertain/desktop/figures/icon128-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..aa727f3f5d5883b3fb83a79c4b98e8b5bfe4ade6 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon128-o.svg @@ -0,0 +1,12 @@ + + + + userswitch_normal + Created with Sketch. + + + + + + + \ No newline at end of file diff --git a/archive/uncertain/desktop/figures/icon132-o.svg b/archive/uncertain/desktop/figures/icon132-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..588ba9d98864ba67a562fa9179f29405f7687aa0 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon132-o.svg @@ -0,0 +1,15 @@ + + + + - + Created with Sketch. + + + + + + + + + + \ No newline at end of file diff --git a/archive/uncertain/desktop/figures/icon133-o.svg b/archive/uncertain/desktop/figures/icon133-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..886d90a83e33497d134bdb3dcc864a5c2df53f20 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon133-o.svg @@ -0,0 +1,13 @@ + + + + + + Created with Sketch. + + + + + + + + \ No newline at end of file diff --git a/archive/uncertain/desktop/figures/icon134-o.svg b/archive/uncertain/desktop/figures/icon134-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..784cf383eb0e8f5c7a57a602047be50ad0a3bc05 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon134-o.svg @@ -0,0 +1,15 @@ + + + + = + Created with Sketch. + + + + + + + + + + \ No newline at end of file diff --git a/archive/uncertain/desktop/figures/icon135-o.svg b/archive/uncertain/desktop/figures/icon135-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..cea628a8f5eb92d10661b690242b6de41ca64816 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon135-o.svg @@ -0,0 +1,15 @@ + + + + ~ + Created with Sketch. + + + + + + + + + + \ No newline at end of file diff --git a/archive/uncertain/desktop/figures/icon136-o.svg b/archive/uncertain/desktop/figures/icon136-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..24aa139ab2fefaee20935551f1af5aef473719ed --- /dev/null +++ b/archive/uncertain/desktop/figures/icon136-o.svg @@ -0,0 +1,12 @@ + + + + poweroff_normal + Created with Sketch. + + + + + + + \ No newline at end of file diff --git a/archive/uncertain/desktop/figures/icon49-o.svg b/archive/uncertain/desktop/figures/icon49-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..72ffb173fdb95e1aff5b0001b08ed6b71122b7f2 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon49-o.svg @@ -0,0 +1,178 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon50-o.svg b/archive/uncertain/desktop/figures/icon50-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..05026802be4718205065d6369e14cc0b6ef05bc7 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon50-o.svg @@ -0,0 +1,15 @@ + + + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon52-o.svg b/archive/uncertain/desktop/figures/icon52-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..23149c05873259cd39721b8ee9c3ab7db86d64c5 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon52-o.svg @@ -0,0 +1,9 @@ + + + attention + + + + + + \ No newline at end of file diff --git a/archive/uncertain/desktop/figures/icon53-o.svg b/archive/uncertain/desktop/figures/icon53-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..50e33489ce984b0acfd621da4a8ef837fdf048c1 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon53-o.svg @@ -0,0 +1,11 @@ + + + + previous + Created with Sketch. + + + + + + \ No newline at end of file diff --git a/archive/uncertain/desktop/figures/icon54-o.svg b/archive/uncertain/desktop/figures/icon54-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..3b599aef4b822c707d2f646405bb00837aed96fd --- /dev/null +++ b/archive/uncertain/desktop/figures/icon54-o.svg @@ -0,0 +1,18 @@ + + + + Backspace + Created with Sketch. + + + + + + + + + + + + + \ No newline at end of file diff --git a/archive/uncertain/desktop/figures/icon56-o.svg b/archive/uncertain/desktop/figures/icon56-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..9f13b6861e3858deec8d57a5301c934acc247069 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon56-o.svg @@ -0,0 +1,19 @@ + + + + Slice 1 + Created with Sketch. + + + + + + + + + + + + + + \ No newline at end of file diff --git a/archive/uncertain/desktop/figures/icon57-o.svg b/archive/uncertain/desktop/figures/icon57-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..e6fbfa1381b76ab3fcd45652b33267a7f6c69bb7 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon57-o.svg @@ -0,0 +1,11 @@ + + + + titlebutton/close_normal + Created with Sketch. + + + + + + \ No newline at end of file diff --git a/archive/uncertain/desktop/figures/icon58-o.svg b/archive/uncertain/desktop/figures/icon58-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..9746dcacfc8e5d4c4b63233801e37418a190fc8f --- /dev/null +++ b/archive/uncertain/desktop/figures/icon58-o.svg @@ -0,0 +1,46 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon62-o.svg b/archive/uncertain/desktop/figures/icon62-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..09f61b446669df2e05a3351d40d8c30879c7b035 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon62-o.svg @@ -0,0 +1,47 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon63-o.svg b/archive/uncertain/desktop/figures/icon63-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..06c03ed99260ffadc681475dad35610aedf67f83 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon63-o.svg @@ -0,0 +1,45 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon66-o.svg b/archive/uncertain/desktop/figures/icon66-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..5793b3846b7fe6a5758379591215b16c7f9e1b52 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon66-o.svg @@ -0,0 +1,43 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon68-o.svg b/archive/uncertain/desktop/figures/icon68-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..a7748052dfa436116d8742dca28f7d90865231ed --- /dev/null +++ b/archive/uncertain/desktop/figures/icon68-o.svg @@ -0,0 +1,23 @@ + + + + deepin-system-monitor + Created with Sketch. + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/archive/uncertain/desktop/figures/icon69-o.svg b/archive/uncertain/desktop/figures/icon69-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..e21dfd00a32a44ee1c8e3882b4ca8239be04690f --- /dev/null +++ b/archive/uncertain/desktop/figures/icon69-o.svg @@ -0,0 +1,26 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon70-o.svg b/archive/uncertain/desktop/figures/icon70-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..b5787a7ffa5ed9519a48c6937c60927fd11fd455 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon70-o.svg @@ -0,0 +1,73 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon71-o.svg b/archive/uncertain/desktop/figures/icon71-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..669a21f143b06cb45ea3f45f7f071809f2cbc8a8 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon71-o.svg @@ -0,0 +1,15 @@ + + + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon72-o.svg b/archive/uncertain/desktop/figures/icon72-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..79067ed9b9ff7912e1742183b461fa056601b9cc --- /dev/null +++ b/archive/uncertain/desktop/figures/icon72-o.svg @@ -0,0 +1,34 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon73-o.svg b/archive/uncertain/desktop/figures/icon73-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..cf6292387f5e790db6ebd66184aabcbb39257ee7 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon73-o.svg @@ -0,0 +1,13 @@ + + + + Down + Created with Sketch. + + + + + + + + \ No newline at end of file diff --git a/archive/uncertain/desktop/figures/icon75-o.svg b/archive/uncertain/desktop/figures/icon75-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..ef6823ccc19858f57374f0b78ad31514e8311be3 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon75-o.svg @@ -0,0 +1,3 @@ + + + diff --git a/archive/uncertain/desktop/figures/icon83-o.svg b/archive/uncertain/desktop/figures/icon83-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..35dd6eacc54a933dc9ebc3f3010edfa7363fecc0 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon83-o.svg @@ -0,0 +1,84 @@ + + + + + + image/svg+xml + + img_upload + + + + + + img_upload + Created with Sketch. + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon84-o.svg b/archive/uncertain/desktop/figures/icon84-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..9bd11b9e7b45b506dd7e1c87d09d545d8f48af06 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon84-o.svg @@ -0,0 +1,15 @@ + + + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon86-o.svg b/archive/uncertain/desktop/figures/icon86-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..5da20233309c43d4fc7b315f441cde476c835c67 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon86-o.svg @@ -0,0 +1,66 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon88-o.svg b/archive/uncertain/desktop/figures/icon88-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..c2570c26575fd14cb5e9d9fe77831d2e8f6c9333 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon88-o.svg @@ -0,0 +1,13 @@ + + + + Left + Created with Sketch. + + + + + + + + \ No newline at end of file diff --git a/archive/uncertain/desktop/figures/icon90-o.svg b/archive/uncertain/desktop/figures/icon90-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..79b5e0a141f7969a8f77ae61f4c240de7187afe9 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon90-o.svg @@ -0,0 +1,12 @@ + + + + lock_normal + Created with Sketch. + + + + + + + \ No newline at end of file diff --git a/archive/uncertain/desktop/figures/icon92-o.svg b/archive/uncertain/desktop/figures/icon92-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..21341b64a832e1935252aa82e7a4e0b083c16eae --- /dev/null +++ b/archive/uncertain/desktop/figures/icon92-o.svg @@ -0,0 +1,12 @@ + + + + logout_normal + Created with Sketch. + + + + + + + \ No newline at end of file diff --git a/archive/uncertain/desktop/figures/icon94-o.svg b/archive/uncertain/desktop/figures/icon94-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..a47044149a02101dbd24a3fdb2f3ead77efca6c1 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon94-o.svg @@ -0,0 +1,54 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon97-o.svg b/archive/uncertain/desktop/figures/icon97-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..4f4670de29d8c86885b5aa806b2c8cdc6fc16dcb --- /dev/null +++ b/archive/uncertain/desktop/figures/icon97-o.svg @@ -0,0 +1,84 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/archive/uncertain/desktop/figures/icon99-o.svg b/archive/uncertain/desktop/figures/icon99-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..e9a3aa60a51404c9390bfbea8d8ff09edc0e2e32 --- /dev/null +++ b/archive/uncertain/desktop/figures/icon99-o.svg @@ -0,0 +1,11 @@ + + + notes + + + + + + + + \ No newline at end of file diff --git a/archive/uncertain/desktop/figures/kubesphere-console.png b/archive/uncertain/desktop/figures/kubesphere-console.png new file mode 100644 index 0000000000000000000000000000000000000000..9c93fbeafe366d78bc05dda6e0e673d2dad8874f Binary files /dev/null and b/archive/uncertain/desktop/figures/kubesphere-console.png differ diff --git a/archive/uncertain/desktop/figures/kubesphere.png b/archive/uncertain/desktop/figures/kubesphere.png new file mode 100644 index 0000000000000000000000000000000000000000..939dcb70202b19c7853cbfd8f27f6e8e4678ce26 Binary files /dev/null and b/archive/uncertain/desktop/figures/kubesphere.png differ diff --git a/archive/uncertain/desktop/kubesphere.md b/archive/uncertain/desktop/kubesphere.md new file mode 100644 index 0000000000000000000000000000000000000000..1328b0f04946df9de5b6dba29f9b7ff8d003107f --- /dev/null +++ b/archive/uncertain/desktop/kubesphere.md @@ -0,0 +1,60 @@ +# KubeSphere 安装指南 + +本文介绍如何在 openEuler 21.09 上安装和部署 Kubernetes 和 KubeSphere 集群。 + +## 什么是 KubeSphere + +[KubeSphere](https://kubesphere.io/) 是在 [Kubernetes](https://kubernetes.io/) 之上构建的面向云原生应用的**分布式操作系统**，完全开源，支持多云与多集群管理，提供全栈的 IT 自动化运维能力，简化企业的 DevOps 工作流。它的架构可以非常方便地使第三方应用与云原生生态组件进行即插即用（plug-and-play）的集成。有关更多信息，请参阅 [KubeSphere 官网](https://kubesphere.com.cn/)。 + +## 前提条件 + +您需要准备一台安装了 openEuler 21.09 的物理机/虚拟机，安装方法参考《[安装指南](../../../docs/zh/Server/InstallationUpgrade/Installation/installation.md)》。 + +## 软件安装 + +1. 安装 KubeKey 。 + + ```bash + yum install kubekey + ``` + + > ![](../../public_sys-resources/icon-note.gif)**说明：** + >开始部署前可预先在集群中每个节点部署 Docker，也可交由 KubeKey 自动部署 (KubeKey 自动部署的 Docker 版本为 20.10.8) 。 + +2. 部署 KubeSphere 集群。 + + ```bash + kk create cluster --with-kubesphere v3.1.1 + ``` + + > **说明：** + >此命令会默认安装 Kubernetes v1.19.8。如需指定 Kubernetes 版本，则需要在命令行后添加 `--with-kubernetes <版本号>`，支持的 Kubernetes 版本包括 `v1.17.9`、`v1.18.8`、 `v.1.19.8`、`v1.19.9`、`v1.20.6`。 + +3. 验证 KubeSphere 集群是否安装成功。 + + ```bash + kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f + ``` + + 若回显为如下，则 KubeSphere 集群安装成功。 + + ![](./figures/kubesphere.png) + + > ![](../../public_sys-resources/icon-note.gif) **说明：** + > 本文档适用于在 x86 环境中安装 KubeSphere。ARM64 环境中需要先安装 Kubernetes 才能部署 KubeSphere。 + +## 访问 KubeSphere Web 控制台 + +**注意取决于您的网络环境，您可能需要配置端口转发规则和防火墙策略，请确保在防火墙规则中放行 `30880` 端口。** + +KubeSphere 集群部署成功后，在浏览器地址栏中输入 `<节点 IP 地址>:30880` 访问 KubeSphere Web 控制台。 + +![kubesphere-console](./figures/kubesphere-console.png) + +## 另请参见 + +[什么是 KubeSphere](https://v3-1.docs.kubesphere.io/zh/docs/introduction/what-is-kubesphere/) + +[在 Linux 上安装 KubeSphere](https://v3-1.docs.kubesphere.io/zh/docs/installing-on-linux/introduction/multioverview/) + +[启用可插拔组件](https://v3-1.docs.kubesphere.io/zh/docs/quick-start/enable-pluggable-components/) diff --git a/docs/en/_toc.yaml b/docs/en/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..38181b9a409e4034bb75b069340e7d1ef79b943c --- /dev/null +++ b/docs/en/_toc.yaml @@ -0,0 +1,8 @@ +label: Document Center +sections: + - href: ./server/_toc.yaml + - href: ./virtualization/_toc.yaml + - href: ./cloud/_toc.yaml + - href: ./edge_computing/_toc.yaml + - href: ./embedded/_toc.yaml + - href: ./tools/_toc.yaml diff --git a/docs/en/cloud/_toc.yaml b/docs/en/cloud/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..de0149c3a46e80ef29d5c2c6c0fe47685787dc60 --- /dev/null +++ b/docs/en/cloud/_toc.yaml @@ -0,0 +1,33 @@ +label: Cloud +sections: + - label: Container Engines + sections: + - href: ./container_engine/isula_container_engine/_toc.yaml + - href: ./container_engine/docker_engine/_toc.yaml + - label: Container Forms + sections: + - href: ./container_form/secure_container/_toc.yaml + - href: ./container_form/system_container/_toc.yaml + - label: Container Runtimes + sections: + - href: ./container_runtime/kuasar/_toc.yaml + - label: Container Image Building + sections: + - href: ./image_builder/isula-build/_toc.yaml + - label: Cloud-Native OS + sections: + - href: ./kubeos/kubeos/_toc.yaml + - label: Cloud Base OS + sections: + - href: ./nestos/nestos/_toc.yaml + - label: Hybrid Deployment + sections: + - href: ./hybrid_deployment/rubik/_toc.yaml + - href: ./hybrid_deployment/oncn-bwm/_toc.yaml + - label: Cluster Deployment + sections: + - href: ./cluster_deployment/kubernetes/_toc.yaml + - href: ./cluster_deployment/isulad+k8s/_toc.yaml + - label: Service Mesh + sections: + - href: ./kmesh/kmesh/_toc.yaml diff --git a/docs/en/cloud/cluster_deployment/isulad+k8s/_toc.yaml b/docs/en/cloud/cluster_deployment/isulad+k8s/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..2137ffcdea9e676fbba6534d81df58e1a383310e --- /dev/null +++ b/docs/en/cloud/cluster_deployment/isulad+k8s/_toc.yaml @@ -0,0 +1,13 @@ +label: iSulad + Kubernetes Cluster Deployment Guide +isManual: true +href: ./isulad+k8s-cluster+deployment.md +description: Deploy a Kubernetes cluster using the iSulad container engine on openEuler. +sections: + - label: Overview + href: ./overview.md + - label: iSulad + Kubernetes Environment Deployment + href: ./isulad+k8s-environment-deploy.md + - label: GitLab Deployment + href: ./gitlab-deploy.md + - label: GitLab Runner Deployment + href: ./gitlab-runner-deploy.md diff --git a/docs/en/cloud/cluster_deployment/isulad+k8s/figures/1.view-required-images.png b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/1.view-required-images.png new file mode 100644 index 0000000000000000000000000000000000000000..74cdae5726cec83d5d74b0b8bd01694fd388e342 Binary files /dev/null and b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/1.view-required-images.png differ diff --git a/docs/en/cloud/cluster_deployment/isulad+k8s/figures/13.view-cert-config.png b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/13.view-cert-config.png new file mode 100644 index 0000000000000000000000000000000000000000..8e9ce44af5a01670add1b8b2f5a7223a8bd0f35d Binary files /dev/null and b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/13.view-cert-config.png differ diff --git a/docs/en/cloud/cluster_deployment/isulad+k8s/figures/14.import-cert.png b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/14.import-cert.png new file mode 100644 index 0000000000000000000000000000000000000000..2a1fdb24d6f5c1c9d44cbce08276289adc5c876c Binary files /dev/null and b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/14.import-cert.png differ diff --git a/docs/en/cloud/cluster_deployment/isulad+k8s/figures/15.register-gitlab-runner.jpg b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/15.register-gitlab-runner.jpg new file mode 100644 index 0000000000000000000000000000000000000000..896f13bdc6411b719283f30d9973973950f27a1c Binary files /dev/null and b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/15.register-gitlab-runner.jpg differ diff --git a/docs/en/cloud/cluster_deployment/isulad+k8s/figures/17.png b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/17.png new file mode 100644 index 0000000000000000000000000000000000000000..86f90a67185f532b362f4710ce8f7615cf40c9e1 Binary files /dev/null and b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/17.png differ diff --git a/docs/en/cloud/cluster_deployment/isulad+k8s/figures/18.dns-config.png b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/18.dns-config.png new file mode 100644 index 0000000000000000000000000000000000000000..46b85396db34577b67679da759b6160ee707dec5 Binary files /dev/null and b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/18.dns-config.png differ diff --git a/docs/en/cloud/cluster_deployment/isulad+k8s/figures/2.calico-config.png b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/2.calico-config.png new file mode 100644 index 0000000000000000000000000000000000000000..d656f86d8ce5e110cf240a58e58b05b42aba8c15 Binary files /dev/null and b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/2.calico-config.png differ diff --git a/docs/en/cloud/cluster_deployment/isulad+k8s/figures/20.yaml.png b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/20.yaml.png new file mode 100644 index 0000000000000000000000000000000000000000..4a609d864f0ca184d94e9108656a8652a6dad55d Binary files /dev/null and b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/20.yaml.png differ diff --git a/docs/en/cloud/cluster_deployment/isulad+k8s/figures/3.png b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/3.png new file mode 100644 index 0000000000000000000000000000000000000000..7394b5f21821ce8d352c2f935c3ea3e490dc0519 Binary files /dev/null and b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/3.png differ diff --git a/docs/en/cloud/cluster_deployment/isulad+k8s/figures/4.gitlab-entrance.jpg b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/4.gitlab-entrance.jpg new file mode 100644 index 0000000000000000000000000000000000000000..d3eb0d59d6dee5051470621a4969651668687789 Binary files /dev/null and b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/4.gitlab-entrance.jpg differ diff --git a/docs/en/cloud/cluster_deployment/isulad+k8s/figures/5.view-password.jpg b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/5.view-password.jpg new file mode 100644 index 0000000000000000000000000000000000000000..2e3902815108e9e91a07c382a4aae090b7cc6fe9 Binary files /dev/null and b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/5.view-password.jpg differ diff --git a/docs/en/cloud/cluster_deployment/isulad+k8s/figures/6.logged-in.png b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/6.logged-in.png new file mode 100644 index 0000000000000000000000000000000000000000..5f4d2c2a9a8bf337263028e859e49499155920b0 Binary files /dev/null and b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/6.logged-in.png differ diff --git a/docs/en/cloud/cluster_deployment/isulad+k8s/figures/7.image.png b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/7.image.png new file mode 100644 index 0000000000000000000000000000000000000000..26c811ae616d2fe86e7b8b75c78ef88aff83616b Binary files /dev/null and b/docs/en/cloud/cluster_deployment/isulad+k8s/figures/7.image.png differ diff --git a/docs/en/cloud/cluster_deployment/isulad+k8s/gitlab-deploy.md b/docs/en/cloud/cluster_deployment/isulad+k8s/gitlab-deploy.md new file mode 100644 index 0000000000000000000000000000000000000000..ea24d8006a11aabc4074d84867a6da7f2fea4519 --- /dev/null +++ b/docs/en/cloud/cluster_deployment/isulad+k8s/gitlab-deploy.md @@ -0,0 +1,311 @@ +# GitLab Deployment + +## Description + +GitLab deployment is required in Scenario 1 (openEuler native deployment CI/CD based on GitLab CI/CD). In Scenario 2 (openEuler native development cluster managed by GitLab CI/CD), skip this step. + +## Preparing the Server + +Prepare a machine running openEuler 20.03 LTS or later versions. + +## Starting GitLab + +Copy the required YAML files to the **/home** directory and start the related pod. +> **Note**: The YAML files related to GitLab can be obtained from the GitLab official site. + +Example YAML files are as follows. Modify them as required. + +gitlab-redis.yaml + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: redis + namespace: default + labels: + name: redis +spec: + selector: + matchLabels: + name: redis + template: + metadata: + name: redis + labels: + name: redis + spec: + containers: + - name: redis + image: 10.35.111.11:5000/redis:latest + imagePullPolicy: IfNotPresent + ports: + - name: redis + containerPort: 6379 + volumeMounts: + - mountPath: /var/lib/redis + name: data + livenessProbe: + exec: + command: + - redis-cli + - ping + initialDelaySeconds: 30 + timeoutSeconds: 5 + readinessProbe: + exec: + command: + - redis-cli + - ping + initialDelaySeconds: 5 + timeoutSeconds: 1 + volumes: + - name: data + emptyDir: {} + +--- +apiVersion: v1 +kind: Service +metadata: + name: redis + namespace: default + labels: + name: redis +spec: + ports: + - name: redis + port: 6379 + targetPort: redis + selector: + name: redis +``` + +gitlab-postgresql.yaml + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: postgresql + namespace: default + labels: + name: postgresql +spec: + selector: + matchLabels: + name: postgresql + template: + metadata: + name: postgresql + labels: + name: postgresql + spec: + containers: + - name: postgresql + image: 10.35.111.11:5000/postgres:13.6 + imagePullPolicy: IfNotPresent + env: + - name: POSTGRES_HOST_AUTH_METHOD + value: trust + - name: DB_USER + value: gitlab + - name: DB_PASS + value: passw0rd + - name: DB_NAME + value: gitlab_production + - name: DB_EXTENSION + value: pg_trgm + ports: + - name: postgres + containerPort: 5432 + volumeMounts: + - mountPath: /var/lib/postgresql + name: data + livenessProbe: + exec: + command: + - pg_isready + - -h + - localhost + - -U + - postgres + initialDelaySeconds: 30 + timeoutSeconds: 5 + readinessProbe: + exec: + command: + - pg_isready + - -h + - localhost + - -U + - postgres + initialDelaySeconds: 5 + timeoutSeconds: 1 + volumes: + - name: data + emptyDir: {} + +--- +apiVersion: v1 +kind: Service +metadata: + name: postgresql + namespace: default + labels: + name: postgresql +spec: + ports: + - name: postgres + port: 5432 + targetPort: postgres + selector: + name: postgresql +``` + +gitlab.yaml + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: gitlab + namespace: default + labels: + name: gitlab +spec: + selector: + matchLabels: + name: gitlab + template: + metadata: + name: gitlab + labels: + name: gitlab + spec: + containers: + - name: gitlab + image: 10.35.111.11:5000/yrzr/gitlab-ce-arm64v8:14.3.2-ce.0 + imagePullPolicy: IfNotPresent + env: + - name: TZ + value: Asia/Shanghai + - name: GITLAB_TIMEZONE + value: Beijing + - name: GITLAB_SECRETS_DB_KEY_BASE + value: long-and-random-alpha-numeric-string + - name: GITLAB_SECRETS_SECRET_KEY_BASE + value: long-and-random-alpha-numeric-string + - name: GITLAB_SECRETS_OTP_KEY_BASE + value: long-and-random-alpha-numeric-string + - name: GITLAB_ROOT_PASSWORD + value: admin321 + - name: GITLAB_ROOT_EMAIL + value: 517554016@qq.com + - name: GITLAB_HOST + value: git.qikqiak.com + - name: GITLAB_PORT + value: "80" + - name: GITLAB_SSH_PORT + value: "22" + - name: GITLAB_NOTIFY_ON_BROKEN_BUILDS + value: "true" + - name: GITLAB_NOTIFY_PUSHER + value: "false" + - name: GITLAB_BACKUP_SCHEDULE + value: daily + - name: GITLAB_BACKUP_TIME + value: 01:00 + - name: DB_TYPE + value: postgres + - name: DB_HOST + value: postgresql + - name: DB_PORT + value: "5432" + - name: DB_USER + value: gitlab + - name: DB_PASS + value: passw0rd + - name: DB_NAME + value: gitlab_production + - name: REDIS_HOST + value: redis + - name: REDIS_PORT + value: "6379" + ports: + - name: http + containerPort: 80 + - name: ssh + containerPort: 22 + volumeMounts: + - mountPath: /home/git/data + name: data + livenessProbe: + httpGet: + path: / + port: 80 + initialDelaySeconds: 180 + timeoutSeconds: 5 + readinessProbe: + httpGet: + path: / + port: 80 + initialDelaySeconds: 5 + timeoutSeconds: 1 + volumes: + - name: data + emptyDir: {} + +--- +apiVersion: v1 +kind: Service +metadata: + name: gitlab + namespace: default + labels: + name: gitlab +spec: + ports: + - name: http + port: 80 + targetPort: http + nodePort: 30852 + - name: ssh + port: 22 + nodePort: 32353 + targetPort: ssh + selector: + name: gitlab + type: NodePort +``` + +Start the containers. + +```shell +kubectl apply -f gitlab-redis.yaml +kubectl apply -f gitlab-postgresql.yaml +kubectl apply -f gitlab.yaml +``` + +Check whether the GitLab pod is set up successfully. + +```shell +kubectl get pod -A -owide +``` + +## Logging in to GitLab + +Log in to the GitLab Web UI. The address is the IP address and the configured port. + +![](figures/4.gitlab-entrance.jpg) +The user name is **root**. The default password can be viewed in the password file in the container. + +```shell +kubectl exec -it gitlab-lab -n default /bin/sh +cat /etc/gitlab/initial_root_password +``` + +![](figures/5.view-password.jpg) + +- After you log in, this page is displayed: + +![](figures/6.logged-in.png) diff --git a/docs/en/cloud/cluster_deployment/isulad+k8s/gitlab-runner-deploy.md b/docs/en/cloud/cluster_deployment/isulad+k8s/gitlab-runner-deploy.md new file mode 100644 index 0000000000000000000000000000000000000000..770f1651a7cd302b8f2750e2c353be55875896f9 --- /dev/null +++ b/docs/en/cloud/cluster_deployment/isulad+k8s/gitlab-runner-deploy.md @@ -0,0 +1,178 @@ +# GitLab Runner Deployment and Testing + +## Images and Software + +The following table lists the images required during installation. The version numbers are for reference only. + +| Image | Version | +|------------------------------------|----------| +| gitlab/gitlab-runner | alpine-v14.4.0 | +| gitlab/gitlab-runner-helper | x86_64-54944146 | + +> If the Internet is unavailable in the environment, download the required images in advance. +> Download the images from the Docker Hub official website . + +## Using gitlab-runner.yaml to Start the Runner Container + +In the **gitlab-runner.yaml** file, change the image name. The following is an example of the **.yaml** file. Modify the file as required. + +```shell +vim gitlab-runner.yaml +``` + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: gitlab-runner + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + name: gitlab-runner + template: + metadata: + labels: + name: gitlab-runner + spec: + containers: + - args: + - run + image: gitlab/gitlab-runner:alpine-v14.4.0 + imagePullPolicy: IfNotPresent + name: gitlab-runner + volumeMounts: + - mountPath: /etc/gitlab-runner + name: config + readOnly: false + - mountPath: /etc/ssl/certs + name: cacerts + readOnly: true + restartPolicy: Always + volumes: + - hostPath: + path: /etc/gitlab-runner + name: config + - hostPath: + path: /etc/ssl/key + name: cacerts + +```shell + +Start the container. + +```shell +# kubectl apply -f gitlab-runner.yaml +# kubectl get pod -A -o wide +``` + +![image](figures/7.image.png) + +## Creating a Container Project That Uses User Certificates for Authentication in GitLab + +1. Click **New project**. + +2. Select **Create blank project**. + +3. Enter a name for the project. + +4. Choose **Settings** > **CI/CD** > **Runners** > **Expand**. + +5. Record the address and token for registering the Runner. + +6. Import certificate files. + + Check and generate certificate files **admin.crt**, **admin.key**, and **ca.crt** on the master node. + + - View certificate information. + + ```shell + # cat /etc/kubernetes/admin.conf + ``` + + ![view-cert-config](figures/13.view-cert-config.png) + + - Generate the encrypted **admin.crt**. + + ```shell + # echo "${client-certificate-data}" | base64 -d > admin.crt + ``` + + - Generate the encrypted **admin.key**. + + ```shell + # echo "${client-key-data}" | base64 -d > admin.key + ``` + + - Obtain the CA certificate on the manager node. + + ```shell + # cp /etc/kubernetes/pki/ca.crt ./ + ``` + +7. Import the three certificate files to the GitLab Runner container on the node where the Runner is running. + + > **Note**: To import the certificate files, check the node where the GitLab Runner is running, copy the certificate files to the node, and run the **isula cp** command to import the certificate files. + + ```shell + # isula cp admin.crt [Container ID]:Storage path + # isula cp admin.key [Container ID]:Storage path + # isula cp ca.crt [Container ID]:Storage path + ``` + + Note: The **isula cp** command can copy only one file at a time. + + ![import-cert](figures/14.import-cert.png) + +## Registering the GitLab Runner + +Perform registration in the GitLab Runner container. Currently, interactive registration is used. Obtain the registration information from GitLab. Choose **GitLab** > **Group runners** > **Settings** > **CI/CD** > **Runners**. + +![register-gitlab-runner](figures/15.register-gitlab-runner.jpg) + +Upload the prepared **gitlab-runner-helper** image to the private image repository in advance, go to the GitLab Runner container, and modify the configuration file. + +```shell +# cd /etc/gitlab-runner +# mkdir kubessl +# cp /home/admin.crt /etc/gitlab-runner/kubessl +# cp /home/ca.crt /etc/gitlab-runner/kubessl +# cp /home/admin.key /etc/gitlab-runner/kubessl +# vim /etc/gitlab-runner/config.toml +``` + +![](figures/17.png) + +## Adding the DNS Record of the GitLab Container to the Manager Node + +1. View the IP address of the GitLab container. + + ```shell + # kubectl get pods -Aowide + ``` + +2. Add the IP address of the GitLab container to the Kubernetes DNS configuration file. + + ```shell + # kubectl edit configmaps coredns -n kube-system + ``` + + ![dns](figures/18.dns-config.png) + +3. Restart the CoreDNS service. + + ```shell + # kubectl scale deployment coredns -n kube-system --replicas=0 + # kubectl scale deployment coredns -n kube-system --replicas=2 + ``` + +## GitLab Running Testing + +Return to the GitLab web IDE and choose **CI/CD** > **Editor** > **Create new CI/CD pipeline**. + +- Compile the YAML file as follows: + +![yaml](figures/20.yaml.png) + +- Choose **Pipelines** and view the status. diff --git a/docs/en/cloud/cluster_deployment/isulad+k8s/isulad+k8s-environment-deploy.md b/docs/en/cloud/cluster_deployment/isulad+k8s/isulad+k8s-environment-deploy.md new file mode 100644 index 0000000000000000000000000000000000000000..c183eb755628ce0bed029d8c632738fc0cf63ca5 --- /dev/null +++ b/docs/en/cloud/cluster_deployment/isulad+k8s/isulad+k8s-environment-deploy.md @@ -0,0 +1,406 @@ +# iSulad+Kubernetes Environment Deployment + +## Preparing Cluster Servers + +Prepare at least 3 machines running openEuler 20.03 LTS or later versions. The following table lists information about the machines. + +| Host Name | IP Address | OS | Role | Component | +|-------|-------------|------------------------|----------|-----------| +| lab1 | 197.xxx.xxx.xxx | openEuler 20.03 LTS SP3 | Control node | iSulad/Kubernetes | +| lab2 | 197.xxx.xxx.xxx | openEuler 20.03 LTS SP3 | Worker node 1 | iSulad/Kubernetes | +| lab3 | 197.xxx.xxx.xxx | openEuler 20.03 LTS SP3 | Worker node 2 | iSulad/Kubernetes | + +## Preparing Images and Software Packages + +The following table lists software packages and images used in the example. The versions are for reference only. + +| Software | Version | +|------------------------------------|----------| +| iSulad | 2.0.17-2 | +| kubernetes-client | 1.20.2-9 | +| kubernetes-kubeadm | 1.20.2-9 | +| kubernetes-kubelet | 1.20.2-9 | + +| Image | Version | +|------------------------------------|----------| +| k8s.gcr.io/kube-proxy | v1.20.2 | +| k8s.gcr.io/kube-apiserver | v1.20.2 | +| k8s.gcr.io/kube-controller-manager | v1.20.2 | +| k8s.gcr.io/kube-scheduler | v1.20.2 | +| k8s.gcr.io/etcd | 3.4.13-0 | +| k8s.gcr.io/coredns | 1.7.0 | +| k8s.gcr.io/pause | 3.2 | +| calico/node | v3.14.2 | +| calico/pod2daemon-flexvol | v3.14.2 | +| calico/cni | v3.14.2 | +| calico/kube-controllers | v3.14.2 | + +> If you perform the deployment in without an Internet connection, download the software packages, dependencies, and images in advance. + +- Download software packages: +- Download images from Docker Hub: + +## Modifying the hosts File + +1. Change the host name of the machine, for example, **lab1**. + + ```shell + hostnamectl set-hostname lab1 + sudo -i + ``` + +2. Configure host name resolution by modifying the **/etc/hosts** file on each machine. + + ```shell + vim /etc/hosts + ``` + +3. Add the following content (IP address and host name) to the **hosts** file: + + ```text + 197.xxx.xxx.xxx lab1 + 197.xxx.xxx.xxx lab2 + 197.xxx.xxx.xxx lab3 + ``` + +## Preparing the Environment + +1. Disable the firewall/ + + ```shell + systemctl stop firewalld + systemctl disable firewalld + ``` + +2. Disable SELinux. + + ```shell + setenforce 0 + ``` + +3. Disable memory swapping. + + ```shell + swapoff -a + sed -ri 's/.*swap.*/#&/' /etc/fstab + ``` + +4. Configure the network and enable forwarding. + + ```shell + $ cat > /etc/sysctl.d/kubernetes.conf <" + ], + "pod-sandbox-image": "k8s.gcr.io/pause:3.2", + "native.umask": "normal", + "network-plugin": "cni", + "cni-bin-dir": "/opt/cni/bin", + "cni-conf-dir": "/etc/cni/net.d", + "image-layer-check": false, + "use-decrypted-key": true, + "insecure-skip-verify-enforce": false, + "cri-runtimes": { + "kata": "io.containerd.kata.v2" + } + } + ``` + +3. Restart the isulad service. + + ```shell + systemctl restart isulad + ``` + +### Loading the isulad Images + +1. Check the required system images. + + ```shell + kubeadm config images list + ``` + + Pay attention to the versions in the output, as shown in the figure. + ![](figures/1.view-required-images.png) + +2. Pull the images using the `isula` command. + + > **Note**: The versions in the following commands are for reference only. Use the versions in the preceding output. + + ```shell + isula pull k8simage/kube-apiserver:v1.20.15 + isula pull k8smx/kube-controller-manager:v1.20.15 + isula pull k8smx/kube-scheduler:v1.20.15 + isula pull k8smx/kube-proxy:v1.20.15 + isula pull k8smx/pause:3.2 + isula pull k8smx/coredns:1.7.0 + isula pull k8smx/etcd:3.4.13-0 + ``` + +3. Modify the tags of the pulled images. + + ```shell + isula tag k8simage/kube-apiserver:v1.20.15 k8s.gcr.io/kube-apiserver:v1.20.15 + isula tag k8smx/kube-controller-manager:v1.20.15 k8s.gcr.io/kube-controller-manager:v1.20.15 + isula tag k8smx/kube-scheduler:v1.20.15 k8s.gcr.io/kube-scheduler:v1.20.15 + isula tag k8smx/kube-proxy:v1.20.15 k8s.gcr.io/kube-proxy:v1.20.15 + isula tag k8smx/pause:3.2 k8s.gcr.io/pause:3.2 + isula tag k8smx/coredns:1.7.0 k8s.gcr.io/coredns:1.7.0 + isula tag k8smx/etcd:3.4.13-0 k8s.gcr.io/etcd:3.4.13-0 + ``` + +4. Remove the old images. + + ```shell + isula rmi k8simage/kube-apiserver:v1.20.15 + isula rmi k8smx/kube-controller-manager:v1.20.15 + isula rmi k8smx/kube-scheduler:v1.20.15 + isula rmi k8smx/kube-proxy:v1.20.15 + isula rmi k8smx/pause:3.2 + isula rmi k8smx/coredns:1.7.0 + isula rmi k8smx/etcd:3.4.13-0 + ``` + +5. View pulled images. + + ```shell + isula images + ``` + +### Installing crictl + +```shell +yum install -y cri-tools +``` + +### Initializing the Master Node + +Initialize the master node. + +```shell +kubeadm init --kubernetes-version v1.20.2 --cri-socket=/var/run/isulad.sock --pod-network-cidr= +``` + +- `--kubernetes-version` indicates the current Kubernetes version. +- `--cri-socket` specifies the engine, that is, isulad. +- `--pod-network-cidr` specifies the IP address range of the pods. + +Enter the following commands as prompted: + +```shell +mkdir -p $HOME/.kube +sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config +sudo chown $(id -u):$(id -g) $HOME/.kube/config +``` + +After the initialization, copy the last two lines of the output and run the copied commands on the nodes to add them to the master cluster. The commands can also be generated using the following command: + +```sh +kubeadm token create --print-join-command +``` + +### Adding Nodes + +Paste the `kubeadm join` command generated on Master, add `--cri-socket=/var/run/isulad.sock` before `--discovery-token-ca-cert-hash`, and then run the command. + +```shell +kubeadm join --token bgyis4.euwkjqb7jwuenwvs --cri-socket=/var/run/isulad.sock --discovery-token-ca-cert-hash sha256:3792f02e136042e2091b245ac71c1b9cdcb97990311f9300e91e1c339e1dfcf6 +``` + +### Installing Calico Network Plugins + +1. Pull Calico images. + + Configure the Calico network plugins on the Master node and pull the required images on each node. + + ```shell + isula pull calico/node:v3.14.2 + isula pull calico/cni:v3.14.2 + isula pull calico/kube-controllers:v3.14.2 + isula pull calico/pod2daemon-flexvol:v3.14.2 + ``` + +2. Download the configuration file on Master. + + ```shell + wget https://docs.projectcalico.org/v3.14/manifests/calico.yaml + ``` + +3. Modify **calico.yaml**. + + ```yaml + # vim calico.yaml + + # Modify the following parameters. + + - name: IP_AUTODERECTION_METHOD + Value: "can-reach=197.3.10.254" + + - name: CALICO_IPV4POOL_IPIP + Value: "CrossSubnet" + ``` + + ![](figures/2.calico-config.png) + + - If the default CNI of the pod is Flannel, add the following content to **flannel.yaml**: + + ```yaml + --iface=enp4s0 + ``` + + ![](figures/3.png) + +4. Create a pod. + + ```shell + kubectl apply -f calico.yaml + ``` + + - If you want to delete the configuration file, run the following command: + + ```shell + kubectl delete -f calico.yaml + ``` + +5. View pod information. + + ```shell + kubectl get pod -A -o wide + ``` + +### Checking the Master Node Information + +```shell +kubectl get nodes -o wide +``` + +To reset a node, run the following command: + +```shell +kubeadm reset +``` diff --git a/docs/en/cloud/cluster_deployment/isulad+k8s/overview.md b/docs/en/cloud/cluster_deployment/isulad+k8s/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..744ccf8b6d1c0a694118d2516ecf877c0698e209 --- /dev/null +++ b/docs/en/cloud/cluster_deployment/isulad+k8s/overview.md @@ -0,0 +1,21 @@ +# iSulad + Kubernetes Cluster Deployment Guide + +This document outlines the process of deploying a Kubernetes cluster with kubeadm on the openEuler OS, configuring a Kubernetes + iSulad environment, and setting up gitlab-runner. It serves as a comprehensive guide for creating a native openEuler development environment cluster. + +The guide addresses two primary scenarios: + +**Scenario 1**: A complete walkthrough for establishing a native openEuler development CI/CD pipeline from scratch using gitlab-ci. +**Scenario 2**: Instructions for integrating an existing native openEuler development execution machine cluster into gitlab-ci. + +For scenario 1, the following steps are required: + +1. Set up the Kubernetes + iSulad environment. +2. Deploy GitLab. +3. Install and test gitlab-runner. + +For scenario 2, where a gitlab-ci platform is already available, the process involves: + +1. Configure the Kubernetes + iSulad environment. +2. Install and test gitlab-runner. + +> **Note**: All operations described in this document must be executed with root privileges. diff --git a/docs/en/cloud/cluster_deployment/kubernetes/_toc.yaml b/docs/en/cloud/cluster_deployment/kubernetes/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..0c469f874e01165ca3d327f87bd491127ef7ae53 --- /dev/null +++ b/docs/en/cloud/cluster_deployment/kubernetes/_toc.yaml @@ -0,0 +1,38 @@ +label: Kubernetes Cluster Deployment Guide +isManual: true +description: >- + This guide offers essential instructions for deploying a reliable and + high-performance Kubernetes cluster on openEuler. +sections: + - label: Overview + href: ./overview.md + - label: Preparing VMs + href: ./preparing-VMs.md + - label: Manual Cluster Deployment + href: ./deploying-a-Kubernetes-cluster-manually.md + sections: + - label: Installing the Kubernetes Software Package + href: ./installing-the-Kubernetes-software-package.md + - label: Preparing Certificates + href: ./preparing-certificates.md + - label: Installing etcd + href: ./installing-etcd.md + - label: Deploying Components on the Control Plane + href: ./deploying-control-plane-components.md + - label: Deploying a Node Component + href: ./deploying-a-node-component.md + - label: Automatic Cluster Deployment + href: ./eggo-automatic-deployment.md + sections: + - label: Tool Introduction + href: ./eggo-tool-introduction.md + - label: Deploying a Cluster + href: ./eggo-deploying-a-cluster.md + - label: Dismantling a Cluster + href: ./eggo-dismantling-a-cluster.md + - label: Running the Test Pod + href: ./running-the-test-pod.md + - label: Kubernetes Cluster Deployment Guide Based on containerd + href: ./kubernetes-containerd.md + - label: Common Issues and Solutions + href: ./kubernetes-faqs.md diff --git a/docs/en/cloud/cluster_deployment/kubernetes/deploying-a-Kubernetes-cluster-manually.md b/docs/en/cloud/cluster_deployment/kubernetes/deploying-a-Kubernetes-cluster-manually.md new file mode 100644 index 0000000000000000000000000000000000000000..330a9610fdaf193c0ae5ce17defe0b5de5f30cdd --- /dev/null +++ b/docs/en/cloud/cluster_deployment/kubernetes/deploying-a-Kubernetes-cluster-manually.md @@ -0,0 +1,18 @@ +# Deploying a Kubernetes Cluster Manually + +**Note: Manual deployment applies only to experimental and learning environments and is not intended for commercial environments.** + +This chapter describes how to deploy a Kubernetes cluster. + +## Environment + +Deploy based on the preceding [VM installation](./preparing-VMs.md) and obtain the following VM list: + +| HostName | MAC | IPv4 | +| ---------- | ----------------- | -------------------| +| k8smaster0 | 52:54:00:00:00:80 | 192.168.122.154/24 | +| k8smaster1 | 52:54:00:00:00:81 | 192.168.122.155/24 | +| k8smaster2 | 52:54:00:00:00:82 | 192.168.122.156/24 | +| k8snode1 | 52:54:00:00:00:83 | 192.168.122.157/24 | +| k8snode2 | 52:54:00:00:00:84 | 192.168.122.158/24 | +| k8snode3 | 52:54:00:00:00:85 | 192.168.122.159/24 | diff --git a/docs/en/cloud/cluster_deployment/kubernetes/deploying-a-node-component.md b/docs/en/cloud/cluster_deployment/kubernetes/deploying-a-node-component.md new file mode 100644 index 0000000000000000000000000000000000000000..7837252a72ddfda522d5dbdaf01d006afbc67f9d --- /dev/null +++ b/docs/en/cloud/cluster_deployment/kubernetes/deploying-a-node-component.md @@ -0,0 +1,381 @@ +# Deploying a Node Component + +This section uses the `k8snode1` node as an example. + +## Environment Preparation + +```bash +# A proxy needs to be configured for the intranet. +$ dnf install -y docker iSulad conntrack-tools socat containernetworking-plugins +$ swapoff -a +$ mkdir -p /etc/kubernetes/pki/ +$ mkdir -p /etc/cni/net.d +$ mkdir -p /opt/cni +# Delete the default kubeconfig file. +$ rm /etc/kubernetes/kubelet.kubeconfig + +## Use iSulad as the runtime ########. +# Configure the iSulad. +cat /etc/isulad/daemon.json +{ + "registry-mirrors": [ + "docker.io" + ], + "insecure-registries": [ + "k8s.gcr.io", + "quay.io" + ], + "pod-sandbox-image": "k8s.gcr.io/pause:3.2",# pause type + "network-plugin": "cni", # If this parameter is left blank, the CNI network plug-in is disabled. In this case, the following two paths become invalid. After the plug-in is installed, restart iSulad. + "cni-bin-dir": "/usr/libexec/cni/", + "cni-conf-dir": "/etc/cni/net.d", +} + +# Add the proxy to the iSulad environment variable and download the image. +cat /usr/lib/systemd/system/isulad.service +[Service] +Type=notify +Environment="HTTP_PROXY=http://name:password@proxy:8080" +Environment="HTTPS_PROXY=http://name:password@proxy:8080" + +# Restart the iSulad and set it to start automatically upon power-on. +systemctl daemon-reload +systemctl restart isulad + + + + +## If Docker is used as the runtime, run the following command: ######## +$ dnf install -y docker +# If a proxy environment is required, configure a proxy for Docker, add the configuration file http-proxy.conf, and edit the following content. Replace name, password, and proxy-addr with the actual values. +$ cat /etc/systemd/system/docker.service.d/http-proxy.conf +[Service] +Environment="HTTP_PROXY=http://name:password@proxy-addr:8080" +$ systemctl daemon-reload +$ systemctl restart docker +``` + +## Creating kubeconfig Configuration Files + +Perform the following operations on each node to create a configuration file: + +```bash +$ kubectl config set-cluster openeuler-k8s \ + --certificate-authority=/etc/kubernetes/pki/ca.pem \ + --embed-certs=true \ + --server=https://192.168.122.154:6443 \ + --kubeconfig=k8snode1.kubeconfig + +$ kubectl config set-credentials system:node:k8snode1 \ + --client-certificate=/etc/kubernetes/pki/k8snode1.pem \ + --client-key=/etc/kubernetes/pki/k8snode1-key.pem \ + --embed-certs=true \ + --kubeconfig=k8snode1.kubeconfig + +$ kubectl config set-context default \ + --cluster=openeuler-k8s \ + --user=system:node:k8snode1 \ + --kubeconfig=k8snode1.kubeconfig + +$ kubectl config use-context default --kubeconfig=k8snode1.kubeconfig +``` + +**Note: Change k8snode1 to the corresponding node name.** + +## Copying the Certificate + +Similar to the control plane, all certificates, keys, and related configurations are stored in the `/etc/kubernetes/pki/` directory. + +```bash +$ ls /etc/kubernetes/pki/ +ca.pem k8snode1.kubeconfig kubelet_config.yaml kube-proxy-key.pem kube-proxy.pem +k8snode1-key.pem k8snode1.pem kube_proxy_config.yaml kube-proxy.kubeconfig +``` + +## CNI Network Configuration + +containernetworking-plugins is used as the CNI plug-in used by kubelet. In the future, plug-ins such as calico and flannel can be introduced to enhance the network capability of the cluster. + +```bash +# Bridge Network Configuration +$ cat /etc/cni/net.d/10-bridge.conf +{ + "cniVersion": "0.3.1", + "name": "bridge", + "type": "bridge", + "bridge": "cnio0", + "isGateway": true, + "ipMasq": true, + "ipam": { + "type": "host-local", + "subnet": "10.244.0.0/16", + "gateway": "10.244.0.1" + }, + "dns": { + "nameservers": [ + "10.244.0.1" + ] + } +} + +# Loopback Network Configuration +$ cat /etc/cni/net.d/99-loopback.conf +{ + "cniVersion": "0.3.1", + "name": "lo", + "type": "loopback" +} +``` + +## Deploying the kubelet Service + +### Configuration File on Which Kubelet Depends + +```bash +$ cat /etc/kubernetes/pki/kubelet_config.yaml +kind: KubeletConfiguration +apiVersion: kubelet.config.k8s.io/v1beta1 +authentication: + anonymous: + enabled: false + webhook: + enabled: true + x509: + clientCAFile: /etc/kubernetes/pki/ca.pem +authorization: + mode: Webhook +clusterDNS: +- 10.32.0.10 +clusterDomain: cluster.local +runtimeRequestTimeout: "15m" +tlsCertFile: "/etc/kubernetes/pki/k8snode1.pem" +tlsPrivateKeyFile: "/etc/kubernetes/pki/k8snode1-key.pem" +``` + +**Note: The IP address of the cluster DNS is 10.32.0.10, which must be the same as the value of service-cluster-ip-range.** + +### Compiling the systemd Configuration File + +```bash +$ cat /usr/lib/systemd/system/kubelet.service +[Unit] +Description=kubelet: The Kubernetes Node Agent +Documentation=https://kubernetes.io/docs/ +Wants=network-online.target +After=network-online.target + +[Service] +ExecStart=/usr/bin/kubelet \ + --config=/etc/kubernetes/pki/kubelet_config.yaml \ + --network-plugin=cni \ + --pod-infra-container-image=k8s.gcr.io/pause:3.2 \ + --kubeconfig=/etc/kubernetes/pki/k8snode1.kubeconfig \ + --register-node=true \ + --hostname-override=k8snode1 \ + --cni-bin-dir="/usr/libexec/cni/" \ + --v=2 + +Restart=always +StartLimitInterval=0 +RestartSec=10 + +[Install] +WantedBy=multi-user.target +``` + +**Note: If iSulad is used as the runtime, add the following configuration:** + +```bash +--container-runtime=remote \ +--container-runtime-endpoint=unix:///var/run/isulad.sock \ +``` + +## Deploying kube-proxy + +### Configuration File on Which kube-proxy Depends + +```bash +cat /etc/kubernetes/pki/kube_proxy_config.yaml +kind: KubeProxyConfiguration +apiVersion: kubeproxy.config.k8s.io/v1alpha1 +clientConnection: + kubeconfig: /etc/kubernetes/pki/kube-proxy.kubeconfig +clusterCIDR: 10.244.0.0/16 +mode: "iptables" +``` + +### Compiling the systemd Configuration File + +```bash +$ cat /usr/lib/systemd/system/kube-proxy.service +[Unit] +Description=Kubernetes Kube-Proxy Server +Documentation=https://kubernetes.io/docs/reference/generated/kube-proxy/ +After=network.target + +[Service] +EnvironmentFile=-/etc/kubernetes/config +EnvironmentFile=-/etc/kubernetes/proxy +ExecStart=/usr/bin/kube-proxy \ + $KUBE_LOGTOSTDERR \ + $KUBE_LOG_LEVEL \ + --config=/etc/kubernetes/pki/kube_proxy_config.yaml \ + --hostname-override=k8snode1 \ + $KUBE_PROXY_ARGS +Restart=on-failure +LimitNOFILE=65536 + +[Install] +WantedBy=multi-user.target +``` + +## Starting a Component Service + +```bash +systemctl enable kubelet kube-proxy +systemctl start kubelet kube-proxy +``` + +Deploy other nodes in sequence. + +## Verifying the Cluster Status + +Wait for several minutes and run the following command to check the node status: + +```bash +$ kubectl get nodes --kubeconfig /etc/kubernetes/pki/admin.kubeconfig +NAME STATUS ROLES AGE VERSION +k8snode1 Ready 17h v1.20.2 +k8snode2 Ready 19m v1.20.2 +k8snode3 Ready 12m v1.20.2 +``` + +## Deploying coredns + +coredns can be deployed on a node or master node. In this document, coredns is deployed on the `k8snode1` node. + +### Compiling the coredns Configuration File + +```bash +$ cat /etc/kubernetes/pki/dns/Corefile +.:53 { + errors + health { + lameduck 5s + } + ready + kubernetes cluster.local in-addr.arpa ip6.arpa { + pods insecure + endpoint https://192.168.122.154:6443 + tls /etc/kubernetes/pki/ca.pem /etc/kubernetes/pki/admin-key.pem /etc/kubernetes/pki/admin.pem + kubeconfig /etc/kubernetes/pki/admin.kubeconfig default + fallthrough in-addr.arpa ip6.arpa + } + prometheus :9153 + forward . /etc/resolv.conf { + max_concurrent 1000 + } + cache 30 + loop + reload + loadbalance +} +``` + +Note: + +- Listen to port 53. +- Configure the Kubernetes plug-in, including the certificate and the URL of kube api. + +### Preparing the service File of systemd + +```bash +cat /usr/lib/systemd/system/coredns.service +[Unit] +Description=Kubernetes Core DNS server +Documentation=https://github.com/coredns/coredns +After=network.target + +[Service] +ExecStart=bash -c "KUBE_DNS_SERVICE_HOST=10.32.0.10 coredns -conf /etc/kubernetes/pki/dns/Corefile" + +Restart=on-failure +LimitNOFILE=65536 + +[Install] +WantedBy=multi-user.target +``` + +### Starting the Service + +```bash +systemctl enable coredns +systemctl start coredns +``` + +### Creating the Service Object of coredns + +```bash +$ cat coredns_server.yaml +apiVersion: v1 +kind: Service +metadata: + name: kube-dns + namespace: kube-system + annotations: + prometheus.io/port: "9153" + prometheus.io/scrape: "true" + labels: + k8s-app: kube-dns + kubernetes.io/cluster-service: "true" + kubernetes.io/name: "CoreDNS" +spec: + clusterIP: 10.32.0.10 + ports: + - name: dns + port: 53 + protocol: UDP + - name: dns-tcp + port: 53 + protocol: TCP + - name: metrics + port: 9153 + protocol: TCP +``` + +### Creating the Endpoint Object of coredns + +```bash +$ cat coredns_ep.yaml +apiVersion: v1 +kind: Endpoints +metadata: + name: kube-dns + namespace: kube-system +subsets: + - addresses: + - ip: 192.168.122.157 + ports: + - name: dns-tcp + port: 53 + protocol: TCP + - name: dns + port: 53 + protocol: UDP + - name: metrics + port: 9153 + protocol: TCP +``` + +### Confirming the coredns Service + +```bash +# View the service object. +$ kubectl get service -n kube-system kube-dns +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +kube-dns ClusterIP 10.32.0.10 53/UDP,53/TCP,9153/TCP 51m +# View the endpoint object. +$ kubectl get endpoints -n kube-system kube-dns +NAME ENDPOINTS AGE +kube-dns 192.168.122.157:53,192.168.122.157:53,192.168.122.157:9153 52m +``` diff --git a/docs/en/cloud/cluster_deployment/kubernetes/deploying-control-plane-components.md b/docs/en/cloud/cluster_deployment/kubernetes/deploying-control-plane-components.md new file mode 100644 index 0000000000000000000000000000000000000000..a9b9bb2faff7c208fe6fb3fb1f02616d5c2f7f18 --- /dev/null +++ b/docs/en/cloud/cluster_deployment/kubernetes/deploying-control-plane-components.md @@ -0,0 +1,357 @@ +# Deploying Components on the Control Plane + +## Preparing the kubeconfig File for All Components + +### kube-proxy + +```bash +kubectl config set-cluster openeuler-k8s --certificate-authority=/etc/kubernetes/pki/ca.pem --embed-certs=true --server=https://192.168.122.154:6443 --kubeconfig=kube-proxy.kubeconfig +kubectl config set-credentials system:kube-proxy --client-certificate=/etc/kubernetes/pki/kube-proxy.pem --client-key=/etc/kubernetes/pki/kube-proxy-key.pem --embed-certs=true --kubeconfig=kube-proxy.kubeconfig +kubectl config set-context default --cluster=openeuler-k8s --user=system:kube-proxy --kubeconfig=kube-proxy.kubeconfig +kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig +``` + +### kube-controller-manager + +```bash +kubectl config set-cluster openeuler-k8s --certificate-authority=/etc/kubernetes/pki/ca.pem --embed-certs=true --server=https://127.0.0.1:6443 --kubeconfig=kube-controller-manager.kubeconfig +kubectl config set-credentials system:kube-controller-manager --client-certificate=/etc/kubernetes/pki/kube-controller-manager.pem --client-key=/etc/kubernetes/pki/kube-controller-manager-key.pem --embed-certs=true --kubeconfig=kube-controller-manager.kubeconfig +kubectl config set-context default --cluster=openeuler-k8s --user=system:kube-controller-manager --kubeconfig=kube-controller-manager.kubeconfig +kubectl config use-context default --kubeconfig=kube-controller-manager.kubeconfig +``` + +### kube-scheduler + +```bash +kubectl config set-cluster openeuler-k8s --certificate-authority=/etc/kubernetes/pki/ca.pem --embed-certs=true --server=https://127.0.0.1:6443 --kubeconfig=kube-scheduler.kubeconfig +kubectl config set-credentials system:kube-scheduler --client-certificate=/etc/kubernetes/pki/kube-scheduler.pem --client-key=/etc/kubernetes/pki/kube-scheduler-key.pem --embed-certs=true --kubeconfig=kube-scheduler.kubeconfig +kubectl config set-context default --cluster=openeuler-k8s --user=system:kube-scheduler --kubeconfig=kube-scheduler.kubeconfig +kubectl config use-context default --kubeconfig=kube-scheduler.kubeconfig +``` + +### admin + +```bash +kubectl config set-cluster openeuler-k8s --certificate-authority=/etc/kubernetes/pki/ca.pem --embed-certs=true --server=https://127.0.0.1:6443 --kubeconfig=admin.kubeconfig +kubectl config set-credentials admin --client-certificate=/etc/kubernetes/pki/admin.pem --client-key=/etc/kubernetes/pki/admin-key.pem --embed-certs=true --kubeconfig=admin.kubeconfig +kubectl config set-context default --cluster=openeuler-k8s --user=admin --kubeconfig=admin.kubeconfig +kubectl config use-context default --kubeconfig=admin.kubeconfig +``` + +### Obtaining the kubeconfig Configuration File + +```bash +admin.kubeconfig kube-proxy.kubeconfig kube-controller-manager.kubeconfig kube-scheduler.kubeconfig +``` + +## Configuration for Generating the Key Provider + +When api-server is started, a key pair `--encryption-provider-config=/etc/kubernetes/pki/encryption-config.yaml` needs to be provided. In this document, a key pair `--encryption-provider-config=/etc/kubernetes/pki/encryption-config.yaml` is generated by using urandom: + +```bash +$ cat generate.bash +#!/bin/bash + +ENCRYPTION_KEY=$(head -c 32 /dev/urandom | base64) + +cat > encryption-config.yaml < [!NOTE]NOTE +> +> - When a cluster is deleted, all data in the cluster is deleted and cannot be restored. Exercise caution when performing this operation. +> - Currently, dismantling a cluster does not delete the containers and the container images. However, if the Kubernetes cluster is configured to install a container engine during the deployment, the container engine will be deleted. As a result, the containers may run abnormally. +> - Some error information may be displayed when dismantling the cluster. Generally, this is caused by the error results returned during the delete operations. The cluster can still be properly dismantled. +> + +You can use the command line to delete the entire cluster. For example, run the following command to delete the k8s-cluster: + +```shell +$ eggo -d cleanup --id k8s-cluster +``` diff --git a/docs/en/cloud/cluster_deployment/kubernetes/eggo-tool-introduction.md b/docs/en/cloud/cluster_deployment/kubernetes/eggo-tool-introduction.md new file mode 100644 index 0000000000000000000000000000000000000000..f640be1ce32265dbdffd8ca06c3c68ce1945421b --- /dev/null +++ b/docs/en/cloud/cluster_deployment/kubernetes/eggo-tool-introduction.md @@ -0,0 +1,429 @@ +# Tool Introduction + +This chapter describes the information related to the automatic deployment tool. You are advised to read this chapter before deployment. + +## Deployment Modes + +The automatic Kubernetes cluster deployment tool provided by openEuler supports one-click deployment using the CLI. The tool provides the following deployment modes: + +- Offline deployment: Prepare all required RPM packages, binary files, plugins, and container images on the local host, pack the packages into a tar.gz file in a specified format, and compile the corresponding YAML configuration file. Then, you can run commands to deploy the cluster in one-click. This deployment mode can be used when the VM cannot access the external network. +- Online deployment: Compile the YAML configuration file. The required RPM packages, binary files, plugins, and container images are automatically downloaded from the Internet during installation and deployment. In this mode, the VM must be able to access the software sources and the image repository on which the cluster depends, for example, Docker Hub. + +## Configurations + +When you use the automatic Kubernetes cluster deployment tool, use the YAML configuration file to describe the cluster deployment information. This section describes the configuration items and provides configuration examples. + +### Configuration Items + +- cluster-id: Cluster name, which must comply with the naming rules for the DNS names. Example: k8s-cluster + +- username: User name used to log in to the hosts using SSH where the Kubernetes cluster is to be deployed. The user name must be identical on all hosts. + +- private-key-path:The path of the key for password-free SSH login. You only need to configure either private-key-path or password. If both are configured, private-key-path is used preferentially. + +- masters: The master node list. It is recommended that each master node is also set as a worker node. Each master node contains the following sub-items. Each master node must be configured with a group of sub-items: + - name: The name of the master node, which is the node name displayed to the Kubernetes cluster. + - ip: The IP address of the master node. + - port: The port for SSH login of the node. The default value is 22. + - arch: CPU architecture of the master node. For example, the value for x86_64 CPUs is amd64. + +- workers: The list of the worker nodes. Each worker node contains the following sub-items. Each worker node must be configured with a group of sub-items: + - name: The name of the worker node, which is the node name displayed to the Kubernetes cluster. + - ip: The IP address of the master node. + - port: The port for SSH login of the node. The default value is 22. + - arch: CPU architecture of the worker node. For example, the value for x86_64 CPUs is amd64. + +- etcds: The list of etcd nodes. If this parameter is left empty, one etcd node is deployed for each master node. Otherwise, only the configured etcd node is deployed. Each etcd node contains the following sub-items. Each etcd node must be configured with a group of sub-items: + - name: The name of the etcd node, which is the node name displayed to the Kubernetes cluster. + - ip: The IP address of the etcd node. + - port: The port for SSH login. + - arch: CPU architecture of the etcd node. For example, the value for x86_64 CPUs is amd64. + +- loadbalance: The loadbalance node list. Each loadbalance node contains the following sub-items. Each loadbalance node must be configured with a group of sub-items: + - name: The name of the loadbalance node, which is the node name displayed to the Kubernetes cluster. + - ip: The IP address of the loadbalance node. + - port: The port for SSH login. + - arch: CPU architecture of the loadbalance node. For example, the value for x86_64 CPUs is amd64. + - bind-port: The listening port of the load balancing service. + +- external-ca: Whether to use an external CA certificate. If yes, set this parameter to true. Otherwise, set this parameter to false. + +- external-ca-path: The path of the external CA certificate file. This parameter takes affect only when external-ca is set to true. + +- service: service information created by Kubernetes. The service configuration item contains the following sub-items: + - cidr: The IP address segment of the service created by Kubernetes. + - dnsaddr: DNS address of the service created by Kubernetes + - gateway: The gateway address of the service created by Kubernetes. + - dns: The configuration item of the CoreDNS created by Kubernetes. The dns configuration item contains the following sub-items: + - corednstype: The deployment type of the CoreDNS created by Kubernetes. The value can be pod or binary. + - imageversion: The CoreDNS image version of the pod deployment type. + - replicas: The number of CoreDNS replicas of the pod deployment type. + +- network: The network configuration of the Kubernetes cluster. The network configuration item contains the following sub-items: + - podcidr: IP address segment of the Kubernetes cluster network. + - plugin: The network plugin deployed in the Kubernetes cluster + - plugin-args: The configuration file path of the network plugin of the Kubernetes cluster network. Example: {"NetworkYamlPath": "/etc/kubernetes/addons/calico.yaml"} + +- apiserver-endpoint: The IP address or domain name of the APIServer service that can be accessed by external systems. If loadbalance is configured, set this parameter to the IP address of the loadbalance node. Otherwise, set this parameter to the IP address of the first master node. + +- apiserver-cert-sans: The IP addresses and domain names that need to be configured in the APIServer certificate. This configuration item contains the following sub-items: + - dnsnames: The array list of the domain names that need to be configured in the APIServer certificate. + - ips: The array list of IP addresses that need to be configured in the APIServer certificate. + +- apiserver-timeout: APIServer response timeout interval. + +- etcd-token: The etcd cluster name. + +- dns-vip: The virtual IP address of the DNS. + +- dns-domain: The DNS domain name suffix. + +- pause-image: The complete image name of the pause container. + +- network-plugin: The type of the network plugin. This parameter can only be set to cni. If this item is not configured, the default Kubernetes network is used. + +- cni-bin-dir: network plugin address. Use commas (,) to separate multiple addresses. For example: /usr/libexec/cni,/opt/cni/bin. + +- runtime: The type of the container runtime. Currently, docker and iSulad are supported. + +- runtime-endpoint: The endpoint of the container runtime. This parameter is optional when runtime is set to docker. + +- registry-mirrors: The mirror site address of the image repository used for downloading container images. + +- insecure-registries: The address of the image repository used for downloading container images through HTTP. + +- config-extra-args: The extra parameters for starting services of each component (such as kube-apiserver and etcd). This configuration item contains the following sub-items: + - name: The component name. The value can be etcd, kube-apiserver, kube-controller-manager, kube-scheduler, kube-proxy or kubelet. + + - extra-args: The extended parameters of the component. The format is key: value. Note that the component parameter corresponding to key must be prefixed with a hyphen (-) or two hyphens (--). + + - open-ports: Configure the ports that need to be enabled additionally. The ports required by Kubernetes do not need to be configured. Other plugin ports need to be configured additionally. + - worker | master | etcd | loadbalance: The type of the node where the ports are enabled. Each configuration item contains one or more port and protocol sub-items. + - port: The port address. + - protocol: The port type. The value can be tcp or udp. + + - install: Configure the detailed information about the installation packages or binary files to be installed on each type of nodes. Note that the corresponding files must be packaged in a tar.gz installation package. The following describes the full configuration. Select the configuration items as needed. + - package-source: The detailed information about the installation package. + - type: The compression type of the installation package. Currently, only tar.gz installation packages are supported. + - dstpath: The path where the installation package is to be decompressed on the peer host. The path must be valid absolute path. + - srcpath: The path for storing the installation packages of different architectures. The architecture must correspond to the host architecture. The path must be a valid absolute path. + - arm64: The path of the installation package of the ARM64 architecture. This parameter is required if any ARM64 node is included in the configuration. + - amd64: The path of the installation package of the AMD64 architecture. This parameter is required if any x86_64 node is included in the configuration. + > ![](./public_sys-resources/icon-note.gif)**NOTE**: + > + > - In the install configuration item, the sub-items of etcd, kubernetes-master, kubernetes-worker, network, loadbalance, container, image, and dns are the same, that is, name, type, dst, schedule, and TimeOut. dst, schedule, and TimeOut are optional. You can determine whether to configure them based on the files to be installed. The following uses the etcd and kubernetes-master nodes as an example. + - etcd: The list of packages or binary files to be installed on etcd nodes. + - name: The names of the software packages or binary files to be installed. If the software package is an installation package, enter only the name and do not specify the version. During the installation, `$name*` is used for identification. Example: etcd. If there are multiple software packages, use commas (,) to separate them. + - type: The type of the configuration item. The value can be pkg, repo, bin, file, dir, image, yaml, or shell. If type is set to repo, configure the repo source on the corresponding node. + - dst: The path of the destination folder. This parameter is required when type is set to bin, file, or dir. It indicates the directory where a file or folder is stored. To prevent users from incorrectly configuring a path and deleting important files during cleanup, this parameter must be set to a path in the whitelist. For details, see "Whitelist Description." + - kubernetes-master: The list of packages or binary files to be installed on the Kubernetes master nodes. + - kubernetes-worker: The list of packages or binary files to be installed on the Kubernetes worker nodes. + - network: The list of packages or binary files to be installed for the network. + - loadbalance: The list of packages or binary files to be installed on the loadbalance nodes. + - container: The list of packages or binary files to be installed for the containers. + - image: The tar package of the container image. + - dns: Kubernetes CoreDNS installation package. If corednstype is set to pod, this parameter is not required. + - addition: The list of additional installation packages or binary files. + - master: The following configurations will be installed on all master nodes. + - name: The name of the software package or binary file to be installed. + - type: The type of the configuration item. The value can be pkg, repo, bin, file, dir, image, yaml, or shell. If type is set to repo, configure the repo source on the corresponding node. + - schedule: Valid only when type is set to shell. This parameter indicates when the user wants to execute the script. The value can be prejoin (before the node is added), postjoin (after the node is added), precleanup (before the node is removed), or postcleanup (after the node is removed). + - TimeOut: The script execution timeout interval. If the execution times out, the process is forcibly stopped. The default value is 30s. + - worker: The configurations will be installed on all worker nodes. The configuration format is the same as that of master under addition. + +### Whitelist Description + +The value of dst under install must match the whitelist rules. Set it to a path in the whitelist or a subdirectory of the path. The current whitelist is as follows: + +- /usr/bin +- /usr/local/bin +- /opt/cni/bin +- /usr/libexec/cni +- /etc/kubernetes +- /usr/lib/systemd/system +- /etc/systemd/system +- /tmp + +### Configuration Example + +The following is an example of the YAML file configuration. As shown in the example, nodes of different types can be deployed on a same host, but the configurations of these nodes must be the same. For example, a master node and a worker node are deployed on test0. + +```yaml +cluster-id: k8s-cluster +username: root +private-key-path: /root/.ssh/private.key +masters: +- name: test0 + ip: 192.168.0.1 + port: 22 + arch: arm64 +workers: +- name: test0 + ip: 192.168.0.1 + port: 22 + arch: arm64 +- name: test1 + ip: 192.168.0.3 + port: 22 + arch: arm64 +etcds: +- name: etcd-0 + ip: 192.168.0.4 + port: 22 + arch: amd64 +loadbalance: + name: k8s-loadbalance + ip: 192.168.0.5 + port: 22 + arch: amd64 + bind-port: 8443 +external-ca: false +external-ca-path: /opt/externalca +service: + cidr: 10.32.0.0/16 + dnsaddr: 10.32.0.10 + gateway: 10.32.0.1 + dns: + corednstype: pod + imageversion: 1.8.4 + replicas: 2 +network: + podcidr: 10.244.0.0/16 + plugin: calico + plugin-args: {"NetworkYamlPath": "/etc/kubernetes/addons/calico.yaml"} +apiserver-endpoint: 192.168.122.222:6443 +apiserver-cert-sans: + dnsnames: [] + ips: [] +apiserver-timeout: 120s +etcd-external: false +etcd-token: etcd-cluster +dns-vip: 10.32.0.10 +dns-domain: cluster.local +pause-image: k8s.gcr.io/pause:3.2 +network-plugin: cni +cni-bin-dir: /usr/libexec/cni,/opt/cni/bin +runtime: docker +runtime-endpoint: unix:///var/run/docker.sock +registry-mirrors: [] +insecure-registries: [] +config-extra-args: + - name: kubelet + extra-args: + "--cgroup-driver": systemd +open-ports: + worker: + - port: 111 + protocol: tcp + - port: 179 + protocol: tcp +install: + package-source: + type: tar.gz + dstpath: "" + srcpath: + arm64: /root/rpms/packages-arm64.tar.gz + amd64: /root/rpms/packages-x86.tar.gz + etcd: + - name: etcd + type: pkg + dst: "" + kubernetes-master: + - name: kubernetes-client,kubernetes-master + type: pkg + kubernetes-worker: + - name: docker-engine,kubernetes-client,kubernetes-node,kubernetes-kubelet + type: pkg + dst: "" + - name: conntrack-tools,socat + type: pkg + dst: "" + network: + - name: containernetworking-plugins + type: pkg + dst: "" + loadbalance: + - name: gd,gperftools-libs,libunwind,libwebp,libxslt + type: pkg + dst: "" + - name: nginx,nginx-all-modules,nginx-filesystem,nginx-mod-http-image-filter,nginx-mod-http-perl,nginx-mod-http-xslt-filter,nginx-mod-mail,nginx-mod-stream + type: pkg + dst: "" + container: + - name: emacs-filesystem,gflags,gpm-libs,re2,rsync,vim-filesystem,vim-common,vim-enhanced,zlib-devel + type: pkg + dst: "" + - name: libwebsockets,protobuf,protobuf-devel,grpc,libcgroup + type: pkg + dst: "" + - name: yajl,lxc,lxc-libs,lcr,clibcni,iSulad + type: pkg + dst: "" + image: + - name: pause.tar + type: image + dst: "" + dns: + - name: coredns + type: pkg + dst: "" + addition: + master: + - name: prejoin.sh + type: shell + schedule: "prejoin" + TimeOut: "30s" + - name: calico.yaml + type: yaml + dst: "" + worker: + - name: docker.service + type: file + dst: /usr/lib/systemd/system/ + - name: postjoin.sh + type: shell + schedule: "postjoin" +``` + +### Installation Package Structure + +For offline deployment, you need to prepare the Kubernetes software package and the related offline installation packages, and store the offline installation packages in a specific directory structure. The directory structure is as follows: + +```shell +package +├── bin +├── dir +├── file +├── image +├── pkg +└── packages_notes.md +``` + +The preceding directories are described as follows: + +- The directory structure of the offline deployment package corresponds to the package types in the cluster configuration file config. The package types include pkg, repo, bin, file, dir, image, yaml and shell. + +- The bin directory stores binary files, corresponding to the bin package type. + +- The dir directory stores the directory that needs to be copied to the target host. You need to configure the dst destination path, corresponding to the dir package type. + +- The file directory stores three types of files: file, yaml, and shell. The file type indicates the files to be copied to the target host, and requires the dst destination path to be configured. The yaml type indicates the user-defined YAML files, which will be applied after the cluster is deployed. The shell type indicates the scripts to be executed, and requires the schedule execution time to be configured. The execution time includes prejoin (before the node is added), postjoin (after the node is added), precleanup (before the node is removed), and postcleanup (after the node is removed). + +- The image directory stores the container images to be imported. The container images must be in a tar package format that is compatible with Docker (for example, images exported by Docker or isula-build). + +- The pkg directory stores the rpm/deb packages to be installed, corresponding to the pkg package type. You are advised to use binary files to facilitate cross-release deployment. + +### Command Reference + +To utilize the cluster deployment tool provided by openEuler, use the eggo command to deploy the cluster. + +#### Deploying the Kubernetes Cluster + +Run the following command to deploy a Kubernetes cluster using the specified YAML configuration: + +**eggo deploy** \[ **-d** ] **-f** *deploy.yaml* + +| Parameter| Mandatory (Yes/No)| Description | +| ------------- | -------- | --------------------------------- | +| --debug \| -d | No| Displays the debugging information.| +| --file \| -f | Yes| Specifies the path of the YAML file for the Kubernetes cluster deployment.| + +#### Adding a Single Node + +Run the following command to add a specified single node to the Kubernetes cluster: + +**eggo** **join** \[ **-d** ] **--id** *k8s-cluster* \[ **--type** *master,worker* ] **--arch** *arm64* **--port** *22* \[ **--name** *master1*] *IP* + +| Parameter| Mandatory (Yes/No) | Description| +| ------------- | -------- | ------------------------------------------------------------ | +| --debug \| -d | No| Displays the debugging information.| +| --id | Yes| Specifies the name of the Kubernetes cluster where the node is to be added.| +| --type \| -t | No| Specifies the type of the node to be added. The value can be master or worker. Use commas (,) to separate multiple types. The default value is worker.| +| --arch \| -a | Yes| Specifies the CPU architecture of the node to be added.| +| --port \| -p | Yes| Specifies the port number for SSH login of the node to be added.| +| --name \| -n | No| Specifies the name of the node to be added.| +| *IP* | Yes| Actual IP address of the node to be added.| + +#### Adding Multiple Nodes + +Run the following command to add specified multiple nodes to the Kubernetes cluster: + +**eggo** **join** \[ **-d** ] **--id** *k8s-cluster* **-f** *nodes.yaml* + +| Parameter| Mandatory (Yes/No) | Description | +| ------------- | -------- | -------------------------------- | +| --debug \| -d | No| Displays the debugging information.| +| --id | Yes| Specifies the name of the Kubernetes cluster where the nodes are to be added.| +| --file \| -f | Yes| Specifies the path of the YAML configuration file for adding the nodes.| + +#### Deleting Nodes + +Run the following command to delete one or more nodes from the Kubernetes cluster: + +**eggo delete** \[ **-d** ] **--id** *k8s-cluster* *node* \[*node...*] + +| Parameter| Mandatory (Yes/No) | Description | +| ------------- | -------- | -------------------------------------------- | +| --debug \| -d | No| Displays the debugging information.| +| --id | Yes| Specifies the name of the cluster where the one or more nodes to be deleted are located.| +| *node* | Yes| Specifies the IP addresses or names of the one or more nodes to be deleted.| + +#### Deleting the Cluster + +Run the following command to delete the entire Kubernetes cluster: + +**eggo cleanup** \[ **-d** ] **--id** *k8s-cluster* \[ **-f** *deploy.yaml* ] + +| Parameter| Mandatory (Yes/No) | Description| +| ------------- | -------- | ------------------------------------------------------------ | +| --debug \| -d | No| Displays the debugging information.| +| --id | Yes| Specifies the name of the Kubernetes cluster to be deleted.| +| --file \| -f | No| Specifies the path of the YAML file for the Kubernetes cluster deletion. If this parameter is not specified, the cluster configuration cached during cluster deployment is used by default. In normal cases, you are advised not to set this parameter. Set this parameter only when an exception occurs.| + +> ![](./public_sys-resources/icon-note.gif)**NOTE**: +> +> - The cluster configuration cached during cluster deployment is recommended when you delete the cluster. That is, you are advised not to set the --file | -f parameter in normal cases. Set this parameter only when the cache configuration is damaged or lost due to an exception. + +#### Querying the Cluster + +Run the following command to query all Kubernetes clusters deployed using eggo: + +**eggo list** \[ **-d** ] + +| Parameter| Mandatory (Yes/No) | Description | +| ------------- | -------- | ------------ | +| --debug \| -d | No| Displays the debugging information.| + +#### Generating the Cluster Configuration File + +Run the following command to quickly generate the required YAML configuration file for the Kubernetes cluster deployment. + +**eggo template** **-d** **-f** *template.yaml* **-n** *k8s-cluster* **-u** *username* **-p** *password* **--etcd** \[*192.168.0.1,192.168.0.2*] **--masters** \[*192.168.0.1,192.168.0.2*] **--workers** *192.168.0.3* **--loadbalance** *192.168.0.4* + +| Parameter| Mandatory (Yes/No) | Description | +| ------------------- | -------- | ------------------------------- | +| --debug \| -d | No| Displays the debugging information.| +| --file \| -f | No| Specifies the path of the generated YAML file.| +| --name \| -n | No| Specifies the name of the Kubernetes cluster.| +| --username \| -u | No| Specifies the user name for SSH login of the configured node.| +| --password \| -p | No| Specifies the password for SSH login of the configured node.| +| --etcd | No| Specifies the IP address list of the etcd nodes.| +| --masters | No| Specifies the IP address list of the master nodes.| +| --workers | No| Specifies the IP address list of the worker nodes.| +| --loadbalance \| -l | No| Specifies the IP address of the loadbalance node.| + +#### Querying the Help Information + +Run the following command to query the help information of the eggo command: + + **eggo help** + +#### Querying the Help Information of Subcommands + +Run the following command to query the help information of the eggo subcommands: + +**eggo deploy | join | delete | cleanup | list | template -h** + +| Parameter| Mandatory (Yes/No) | Description | +| ----------- | -------- | ------------ | +| --help\| -h | Yes| Displays the help information.| diff --git a/docs/en/cloud/cluster_deployment/kubernetes/figures/advertiseAddress.png b/docs/en/cloud/cluster_deployment/kubernetes/figures/advertiseAddress.png new file mode 100644 index 0000000000000000000000000000000000000000..b36e5c4664f2d2e5faaa23128fd4711c11e30179 Binary files /dev/null and b/docs/en/cloud/cluster_deployment/kubernetes/figures/advertiseAddress.png differ diff --git a/docs/en/cloud/cluster_deployment/kubernetes/figures/arch.png b/docs/en/cloud/cluster_deployment/kubernetes/figures/arch.png new file mode 100644 index 0000000000000000000000000000000000000000..650e4a67959136e12e49975196a4f3af28e6170e Binary files /dev/null and b/docs/en/cloud/cluster_deployment/kubernetes/figures/arch.png differ diff --git a/docs/en/cloud/cluster_deployment/kubernetes/figures/flannelConfig.png b/docs/en/cloud/cluster_deployment/kubernetes/figures/flannelConfig.png new file mode 100644 index 0000000000000000000000000000000000000000..dc9e7c665edd02fad16d3e6f4970e3125efcbef8 Binary files /dev/null and b/docs/en/cloud/cluster_deployment/kubernetes/figures/flannelConfig.png differ diff --git a/docs/en/cloud/cluster_deployment/kubernetes/figures/name.png b/docs/en/cloud/cluster_deployment/kubernetes/figures/name.png new file mode 100644 index 0000000000000000000000000000000000000000..dd6ddfdc3476780e8c896bfd5095025507f62fa8 Binary files /dev/null and b/docs/en/cloud/cluster_deployment/kubernetes/figures/name.png differ diff --git a/docs/en/cloud/cluster_deployment/kubernetes/figures/podSubnet.png b/docs/en/cloud/cluster_deployment/kubernetes/figures/podSubnet.png new file mode 100644 index 0000000000000000000000000000000000000000..b368f77dd7dfd7722dcf7751b3e37ec28755e42d Binary files /dev/null and b/docs/en/cloud/cluster_deployment/kubernetes/figures/podSubnet.png differ diff --git a/docs/en/cloud/cluster_deployment/kubernetes/installing-etcd.md b/docs/en/cloud/cluster_deployment/kubernetes/installing-etcd.md new file mode 100644 index 0000000000000000000000000000000000000000..f45e22d16464f59f32c4dec443be4dd5b1d02234 --- /dev/null +++ b/docs/en/cloud/cluster_deployment/kubernetes/installing-etcd.md @@ -0,0 +1,88 @@ +# Installing etcd + +## Preparing the Environment + +Run the following command to enable the port used by etcd: + +```bash +firewall-cmd --zone=public --add-port=2379/tcp +firewall-cmd --zone=public --add-port=2380/tcp +``` + +## Installing the etcd Binary Package + +Currently, the RPM package is used for installation. + +```bash +rpm -ivh etcd*.rpm +``` + +Prepare the directories. + +```bash +mkdir -p /etc/etcd /var/lib/etcd +cp ca.pem /etc/etcd/ +cp kubernetes-key.pem /etc/etcd/ +cp kubernetes.pem /etc/etcd/ +# Disabling SELinux +setenforce 0 +# Disabling the Default Configuration of the /etc/etcd/etcd.conf File +# Commenting Out the Line, for example, ETCD_LISTEN_CLIENT_URLS="http://localhost:2379". +``` + +## Compiling the etcd.service File + +The following uses the `k8smaster0` machine as an example: + +```bash +$ cat /usr/lib/systemd/system/etcd.service +[Unit] +Description=Etcd Server +After=network.target +After=network-online.target +Wants=network-online.target + +[Service] +Type=notify +WorkingDirectory=/var/lib/etcd/ +EnvironmentFile=-/etc/etcd/etcd.conf +# set GOMAXPROCS to number of processors +ExecStart=/bin/bash -c "ETCD_UNSUPPORTED_ARCH=arm64 /usr/bin/etcd --name=k8smaster0 --cert-file=/etc/etcd/kubernetes.pem --key-file=/etc/etcd/kubernetes-key.pem --peer-cert-file=/etc/etcd/kubernetes.pem --peer-key-file=/etc/etcd/kubernetes-key.pem --trusted-ca-file=/etc/etcd/ca.pem --peer-trusted-ca-file=/etc/etcd/ca.pem --peer-client-cert-auth --client-cert-auth --initial-advertise-peer-urls https://192.168.122.154:2380 --listen-peer-urls https://192.168.122.154:2380 --listen-client-urls https://192.168.122.154:2379,https://127.0.0.1:2379 --advertise-client-urls https://192.168.122.154:2379 --initial-cluster-token etcd-cluster-0 --initial-cluster k8smaster0=https://192.168.122.154:2380,k8smaster1=https://192.168.122.155:2380,k8smaster2=https://192.168.122.156:2380 --initial-cluster-state new --data-dir /var/lib/etcd" + +Restart=always +RestartSec=10s +LimitNOFILE=65536 + +[Install] +WantedBy=multi-user.target +``` + +**Note:** + +- The boot setting `ETCD_UNSUPPORTED_ARCH=arm64` needs to be added to ARM64; +- In this document, etcd and Kubernetes control are deployed on the same machine. Therefore, the `kubernetes.pem` and `kubernetes-key.pem` certificates are used to start etcd and Kubernetes control. +- A CA certificate is used in the entire deployment process. etcd can generate its own CA certificate and use its own CA certificate to sign other certificates. However, the certificate signed by the CA certificate needs to be used when the APIServer accesses the etcd client. +- `initial-cluster` needs to be added to all configurations for deploying etcd. +- To improve the storage efficiency of etcd, you can use the directory of the SSD as `data-dir`. + +Start the etcd service. + +```bash +systemctl enable etcd +systemctl start etcd +``` + +Then, deploy other hosts in sequence. + +## Verifying Basic Functions + +```bash +$ ETCDCTL_API=3 etcdctl -w table endpoint status --endpoints=https://192.168.122.155:2379,https://192.168.122.156:2379,https://192.168.122.154:2379 --cacert=/etc/etcd/ca.pem --cert=/etc/etcd/kubernetes.pem --key=/etc/etcd/kubernetes-key.pem ++------------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+ +| ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFTAPPLIED INDEX | ERRORS | ++------------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+ +| https://192.168.122.155:2379 | b50ec873e253ebaa | 3.4.14 | 262 kB | false | false | 819 | 21 | 21 | | +| https://192.168.122.156:2379 | e2b0d126774c6d02 | 3.4.14 | 262 kB | true | false | 819 | 21 | 21 | | +| https://192.168.122.154:2379 | f93b3808e944c379 | 3.4.14 | 328 kB | false | false | 819 | 21 | 21 | | ++------------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+ +``` diff --git a/docs/en/cloud/cluster_deployment/kubernetes/installing-the-Kubernetes-software-package.md b/docs/en/cloud/cluster_deployment/kubernetes/installing-the-Kubernetes-software-package.md new file mode 100644 index 0000000000000000000000000000000000000000..3fbed3d94232fd0741ecd4f0957e20948ca190af --- /dev/null +++ b/docs/en/cloud/cluster_deployment/kubernetes/installing-the-Kubernetes-software-package.md @@ -0,0 +1,11 @@ +# Installing the Kubernetes Software Package + +```bash +dnf install -y docker conntrack-tools socat +``` + +After the EPOL source is configured, you can directly install Kubernetes through DNF. + +```bash +rpm -ivh kubernetes*.rpm +``` diff --git a/docs/en/cloud/cluster_deployment/kubernetes/kubernetes-containerd.md b/docs/en/cloud/cluster_deployment/kubernetes/kubernetes-containerd.md new file mode 100644 index 0000000000000000000000000000000000000000..b4a4f521007057ed4c562a4e9c2582642f4ffb71 --- /dev/null +++ b/docs/en/cloud/cluster_deployment/kubernetes/kubernetes-containerd.md @@ -0,0 +1,308 @@ +# Kubernetes Cluster Deployment Guide Based on containerd + +Starting from version 1.21, Kubernetes no longer supports the Kubernetes+Docker setup for cluster deployment. This guide demonstrates how to quickly set up a Kubernetes cluster using containerd as the container runtime. For custom cluster configurations, consult the [official documentation](https://kubernetes.io/docs/home/). + +## Software Package Installation + +### 1. Installing Required Packages + +```sh +yum install -y containerd +yum install -y kubernetes* +yum install -y cri-tools +``` + +> ![](./public_sys-resources/icon-note.gif)**Note** +> +> - If Docker is already installed on the system, uninstall it before installing containerd to prevent conflicts. + +The required containerd version is 1.6.22-15 or higher. If the installed version is not supported, upgrade to version 1.6.22-15 using the following commands, or perform a manual upgrade. + +```sh +wget --no-check-certificate https://repo.openeuler.org/openEuler-24.03-LTS/update/x86_64/Packages/containerd-1.6.22-15.oe2403.x86_64.rpm +rpm -Uvh containerd-1.6.22-15.oe2403.x86_64.rpm +``` + +The package versions downloaded via `yum` in this guide are: + +```text +1. containerd + - Architecture: x86_64 + - Version: 1.6.22-15 +2. kubernetes - client/help/kubeadm/kubelet/master/node + - Architecture: x86_64 + - Version: 1.29.1-4 +3. cri-tools + - Architecture: X86_64 + - Version: 1.29.0-3 +``` + +### 2. Downloading CNI Components + +```sh +mkdir -p /opt/cni/bin +cd /opt/cni/bin +wget --no-check-certificate https://github.com/containernetworking/plugins/releases/download/v1.5.1/cni-plugins-linux-amd64-v1.5.1.tgz +tar -xzvf ./cni-plugins-linux-amd64-v1.5.1.tgz -C . +``` + +> ![](./public_sys-resources/icon-note.gif)**Note** +> +> - The provided download link is for the AMD64 architecture. Choose the appropriate version based on your system architecture. Other versions are available in the [GitHub repository](https://github.com/containernetworking/plugins/releases/). + +### 3. Downloading CNI Plugin (Flannel) + +```sh +wget https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml --no-check-certificate +``` + +## Environment Configuration + +This section configures the OS environment required for Kubernetes. + +### 1. Setting the Host Name + +```sh +hostnamectl set-hostname nodeName +``` + +### 2. Configuring the Firewall + +**Method 1:** + +Configure firewall rules to open ports for etcd and the API Server, ensuring proper communication between the control plane and worker nodes. + +Open ports for etcd: + +```sh +firewall-cmd --zone=public --add-port=2379/tcp --permanent +firewall-cmd --zone=public --add-port=2380/tcp --permanent +``` + +Open ports for the API Server: + +```sh +firewall-cmd --zone=public --add-port=6443/tcp --permanent +``` + +Apply the firewall rules: + +```sh +firewall-cmd --reload +``` + +> ![](./public_sys-resources/icon-note.gif)**Note** +> +> - Firewall configuration may prevent certain container images from functioning properly. To ensure smooth operation, open the necessary ports based on the images being used. + +**Method 2:** + +Disable the firewall using the following commands: + +```sh +systemctl stop firewalld +systemctl disable firewalld +``` + +### 3. Disabling SELinux + +SELinux security policies may block certain operations within containers, such as writing to specific directories, accessing network resources, or executing privileged operations. This can cause critical services like CoreDNS to fail, resulting in `CrashLoopBackOff` or `Error` states. Disable SELinux using the following commands: + +```sh +setenforce 0 +sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config +``` + +### 4. Disabling Swap + +The Kubernetes scheduler allocates pods to nodes based on available memory and CPU resources. If swap is enabled on a node, the actual physical memory and logically available memory may not align, which can affect the scheduler decisions, leading to node overloading or incorrect scheduling. Therefore, disable swap: + +```sh +swapoff -a +sed -ri 's/.*swap.*/#&/' /etc/fstab +``` + +### 5. Configuring the Network + +Enable IPv6 and IPv4 traffic filtering on bridged networks using iptables, and enable IP forwarding to ensure inter-pod communication across nodes: + +```sh +$ cat > /etc/sysctl.d/k8s.conf << EOF +net.bridge.bridge-nf-call-ip6tables = 1 +net.bridge.bridge-nf-call-iptables = 1 +net.ipv4.ip_forward = 1 +vm.swappiness=0 +EOF +$ modprobe br_netfilter +$ sysctl -p /etc/sysctl.d/k8s.conf +``` + +## Configuring containerd + +This section configures containerd, including setting the pause image, cgroup driver, disabling certificate verification for the `registry.k8s.io` image repository, and configuring a proxy. + +First, generate the default configuration file for containerd and output it to the file specified by `containerd_conf`: + +```sh +containerd_conf="/etc/containerd/config.toml" +mkdir -p /etc/containerd +containerd config default > "${containerd_conf}" +``` + +Configure the pause image: + +```sh +pause_img=$(kubeadm config images list | grep pause | tail -1) +sed -i "/sandbox_image/s#\".*\"#\"${pause_img}\"#" "${containerd_conf}" +``` + +Set the cgroup driver to systemd: + +```sh +sed -i "/SystemdCgroup/s/=.*/= true/" "${containerd_conf}" +``` + +Disable certificate verification for the `registry.k8s.io` image repository: + +```sh +sed -i '/plugins."io.containerd.grpc.v1.cri".registry.configs/a\[plugins."io.containerd.grpc.v1.cri".registry.configs."registry.k8s.io".tls]\n insecure_skip_verify = true' /etc/containerd/config.toml +``` + +Configure the proxy (replace "***" in `HTTP_PROXY`, `HTTPS_PROXY`, and `NO_PROXY` with your proxy information): + +```sh +$ server_path="/etc/systemd/system/containerd.service.d" +$ mkdir -p "${server_path}" +$ cat > "${server_path}"/http-proxy.conf << EOF +[Service] +Environment="HTTP_PROXY=***" +Environment="HTTPS_PROXY=***" +Environment="NO_PROXY=***" +EOF +``` + +Restart containerd to apply the configurations: + +```sh +systemctl daemon-reload +systemctl restart containerd +``` + +## Configuring crictl to Use containerd as the Container Runtime + +```sh +crictl config runtime-endpoint unix:///run/containerd/containerd.sock +crictl config image-endpoint unix:///run/containerd/containerd.sock +``` + +## Configuring kubelet to Use systemd as the Cgroup Driver + +```sh +systemctl enable kubelet.service +echo 'KUBELET_EXTRA_ARGS="--runtime-cgroups=/systemd/system.slice --kubelet-cgroups=/systemd/system.slice"' >> /etc/sysconfig/kubelet +systemctl restart kubelet +``` + +## Creating a Cluster Using Kubeadm (Control Plane Only) + +### 1. Configuring Cluster Information + +```sh +kubeadm config print init-defaults --component-configs KubeletConfiguration >> kubeletConfig.yaml +vim kubeletConfig.yaml +``` + +In the **kubeletConfig.yaml** file, configure the node name, advertise address (`advertiseAddress`), and the CIDR for the Pod network. + +**Modify `name` to match the hostname, consistent with the first step in the environment configuration:** + +![](./figures/name.png) + +**Change `advertiseAddress` to the IP address of the control plane:** + +![](./figures/advertiseAddress.png) + +**Add `podSubnet` under `Networking` to specify the CIDR range:** + +![](./figures/podSubnet.png) + +### 2. Deploying the Cluster + +Use `kubeadm` to deploy the cluster. Many configurations are generated by default (such as authentication certificates). Refer to the [official documentation](https://kubernetes.io/docs/home/) for modifications. + +**Disable the proxy (if applicable):** + +```sh +unset http_proxy https_proxy +``` + +Deploy the cluster using `kubeadm init`: + +```sh +kubeadm init --config kubeletConfig.yaml +``` + +Specify the configuration file for `kubectl`: + +```sh +mkdir -p "$HOME"/.kube +cp -i /etc/kubernetes/admin.conf "$HOME"/.kube/config +chown "$(id -u)":"$(id -g)" "$HOME"/.kube/config +export KUBECONFIG=/etc/kubernetes/admin.conf +``` + +### 3. Deploying the CNI Plugin (Flannel) + +This tutorial uses Flannel as the CNI plugin. Below are the steps to download and deploy Flannel. + +The Flannel used here is downloaded from the `registry-1.docker.io` image repository. To avoid certificate verification issues, configure the image repository to skip certificate verification in the containerd configuration file (**/etc/containerd/config.toml**). + +![](./figures/flannelConfig.png) + +Use `kubectl apply` to deploy the **kube-flannel.yml** file downloaded during the software package installation. + +```sh +kubectl apply -f kube-flannel.yml +``` + +> ![](./public_sys-resources/icon-note.gif)**Note** +> +> The control plane may have taint issues, causing the node status in `kubectl get nodes` to remain "not ready." Refer to the [official documentation](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to remove taints. + +## Joining the Cluster (Worker Nodes Only) + +**Disable the proxy (if applicable):** + +```sh +unset http_proxy https_proxy +``` + +After installing and configuring the environment on worker nodes, join the cluster using the following command: + +```sh +kubeadm join : --token --discovery-token-ca-cert-hash sha256: +``` + +This command is generated after `kubeadm init` completes on the control plane. Alternatively, you can generate it on the control plane using the following commands: + +```sh +$ kubeadm token create # Generate token. +$ openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | \ + openssl dgst -sha256 -hex | sed 's/^.* //' # Get hash. +``` + +After joining, check the status of worker nodes on the control plane using: + +```sh +kubectl get nodes +``` + +If the node status shows "not ready," it may be due to unsuccessful Flannel plugin deployment. In this case, run the locally generated Flannel executable to complete the deployment. + +**Running kubectl Commands on Worker Nodes (Optional):** + +To run `kubectl` commands on worker nodes, copy the control plane configuration file **/etc/kubernetes/admin.conf** to the same directory, then configure it using: + +```sh +export KUBECONFIG=/etc/kubernetes/admin.conf +``` diff --git a/docs/en/cloud/cluster_deployment/kubernetes/kubernetes-faqs.md b/docs/en/cloud/cluster_deployment/kubernetes/kubernetes-faqs.md new file mode 100644 index 0000000000000000000000000000000000000000..30708ab50f270ac09a41ddf5b7106d2e7c8e3a63 --- /dev/null +++ b/docs/en/cloud/cluster_deployment/kubernetes/kubernetes-faqs.md @@ -0,0 +1,13 @@ +# Common Issues and Solutions + +## Issue 1: Kubernetes + Docker Deployment Failure + +Reason: Kubernetes dropped support for Kubernetes + Docker cluster deployments starting from version 1.21. + +Solution: Use cri-dockerd + Docker for cluster deployment, or consider alternatives like containerd or iSulad. + +## Issue 2: Unable to Install Kubernetes RPM Packages via yum on openEuler + +Reason: Installing Kubernetes-related RPM packages requires proper configuration of the EPOL repository in yum. + +Solution: Follow the repository configuration guide provided in [this link](https://forum.openeuler.org/t/topic/768) to set up the EPOL repository in your environment. diff --git a/docs/en/cloud/cluster_deployment/kubernetes/overview.md b/docs/en/cloud/cluster_deployment/kubernetes/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..4b1174b926b9a10a3de4a7ecf97757a64da0c341 --- /dev/null +++ b/docs/en/cloud/cluster_deployment/kubernetes/overview.md @@ -0,0 +1,12 @@ +# Kubernetes Cluster Deployment Guide + +This document describes how to deploy a Kubernetes cluster in binary mode on openEuler. + +Note: All operations in this document are performed using the `root` permission. + +## Cluster Status + +The cluster status used in this document is as follows: + +- Cluster structure: six VMs running the `openEuler 21.09` OS, three master nodes, and three nodes. +- Physical machine: `x86/ARM` server of `openEuler 21.09`. diff --git a/docs/en/cloud/cluster_deployment/kubernetes/preparing-VMs.md b/docs/en/cloud/cluster_deployment/kubernetes/preparing-VMs.md new file mode 100644 index 0000000000000000000000000000000000000000..52b21caf6eb76e8ee4219640855f4adcf538bd68 --- /dev/null +++ b/docs/en/cloud/cluster_deployment/kubernetes/preparing-VMs.md @@ -0,0 +1,150 @@ +# Preparing VMs + +This document describes how to use virt manager to install a VM. Ignore if your VM is prepared. + +## Installing Dependency Tools + +VM installation depends on related tools. The following command is an example for installing the dependency and enabling the libvirtd service. (If a proxy is required, configure the proxy first.) + +```bash +dnf install virt-install virt-manager libvirt-daemon-qemu edk2-aarch64.noarch virt-viewer +systemctl start libvirtd +systemctl enable libvirtd +``` + +## Preparing VM Disk Files + +```bash +dnf install -y qemu-img +virsh pool-define-as vmPool --type dir --target /mnt/vm/images/ +virsh pool-build vmPool +virsh pool-start vmPool +virsh pool-autostart vmPool +virsh vol-create-as --pool vmPool --name master0.img --capacity 200G --allocation 1G --format qcow2 +virsh vol-create-as --pool vmPool --name master1.img --capacity 200G --allocation 1G --format qcow2 +virsh vol-create-as --pool vmPool --name master2.img --capacity 200G --allocation 1G --format qcow2 +virsh vol-create-as --pool vmPool --name node1.img --capacity 300G --allocation 1G --format qcow2 +virsh vol-create-as --pool vmPool --name node2.img --capacity 300G --allocation 1G --format qcow2 +virsh vol-create-as --pool vmPool --name node3.img --capacity 300G --allocation 1G --format qcow2 +``` + +## Enabling Firewall Ports + +**Method 1** + +1. Query a port. + + ```shell + netstat -lntup | grep qemu-kvm + ``` + +2. Enable the VNC firewall port. For example, if the port number starts from 5900, run the following commands: + + ```shell + firewall-cmd --zone=public --add-port=5900/tcp + firewall-cmd --zone=public --add-port=5901/tcp + firewall-cmd --zone=public --add-port=5902/tcp + firewall-cmd --zone=public --add-port=5903/tcp + firewall-cmd --zone=public --add-port=5904/tcp + firewall-cmd --zone=public --add-port=5905/tcp + ``` + +**Method 2** + +Disable the firewall. + +```shell +systemctl stop firewalld +``` + +## Preparing the VM Configuration File + +A VM configuration file is required for creating a VM. For example, if the configuration file is master.xml and the host name of the VM is k8smaster0, the configuration is as follows: + +```bash + cat master.xml + + + k8smaster0 + 8 + 8 + + hvm + /usr/share/edk2/aarch64/QEMU_EFI-pflash.raw + /var/lib/libvirt/qemu/nvram/k8smaster0.fd + + + + + + + + + 1 + + destroy + restart + restart + + /usr/libexec/qemu-kvm + + + + + + + + + + + + + + + + + + + + + + + + + + + + +``` + +The VM configuration must be unique. Therefore, you need to modify the following to ensure that the VM is unique: + +- name: host name of the VM. You are advised to use lowercase letters. In this example, the value is `k8smaster0`. +- nvram: handle file path of the NVRAM, which must be globally unique. In this example, the value is `/var/lib/libvirt/qemu/nvram/k8smaster0.fd`. +- disk source file: VM disk file path. In this example, the value is `/mnt/vm/images/master0.img`. +- mac address of the interface: MAC address of the interface. In this example, the value is `52:54:00:00:00:80`. + +## Installing a VM + +1. Create and start a VM. + + ```shell + virsh define master.xml + virsh start k8smaster0 + ``` + +2. Obtain the VNC port number of the VM. + + ```shell + virsh vncdisplay k8smaster0 + ``` + +3. Use a VM connection tool, such as VNC Viewer, to remotely connect to the VM and perform configurations as prompted. + +4. Set the host name of the VM, for example, k8smaster0. + + ```shell + hostnamectl set-hostname k8smaster0 + ``` diff --git a/docs/en/cloud/cluster_deployment/kubernetes/preparing-certificates.md b/docs/en/cloud/cluster_deployment/kubernetes/preparing-certificates.md new file mode 100644 index 0000000000000000000000000000000000000000..997e10e184894aa05f1d9e091e5c1e1337f70985 --- /dev/null +++ b/docs/en/cloud/cluster_deployment/kubernetes/preparing-certificates.md @@ -0,0 +1,412 @@ +# Preparing Certificates + +**Statement: The certificate used in this document is self-signed and cannot be used in a commercial environment.** + +Before deploying a cluster, you need to generate certificates required for communication between components in the cluster. This document uses the open-source CFSSL as the verification and deployment tool to help users understand the certificate configuration and the association between certificates of cluster components. You can select a tool based on the site requirements, for example, OpenSSL. + +## Building and Installing CFSSL + +The following commands for building and installing CFSSL are for your reference (the CFSSL website access permission is required, and the proxy must be configured first): + +```bash +wget --no-check-certificate https://github.com/cloudflare/cfssl/archive/v1.5.0.tar.gz +tar -zxf v1.5.0.tar.gz +cd cfssl-1.5.0/ +make -j6 +cp bin/* /usr/local/bin/ +``` + +## Generating a Root Certificate + +Compile the CA configuration file, for example, ca-config.json: + +```bash +$ cat ca-config.json | jq +{ + "signing": { + "default": { + "expiry": "8760h" + }, + "profiles": { + "kubernetes": { + "usages": [ + "signing", + "key encipherment", + "server auth", + "client auth" + ], + "expiry": "8760h" + } + } + } +} +``` + +Compile a CA CSR file, for example, ca-csr.json: + +```bash +$ cat ca-csr.json | jq +{ + "CN": "Kubernetes", + "key": { + "algo": "rsa", + "size": 2048 + }, + "names": [ + { + "C": "CN", + "L": "HangZhou", + "O": "openEuler", + "OU": "WWW", + "ST": "BinJiang" + } + ] +} +``` + +Generate the CA certificate and key: + +```bash +cfssl gencert -initca ca-csr.json | cfssljson -bare ca +``` + +The following certificates are obtained: + +```bash +ca.csr ca-key.pem ca.pem +``` + +## Generating the admin Account Certificate + +admin is an account used by K8S for system management. Compile the CSR configuration of the admin account, for example, admin-csr.json: + +```bash +cat admin-csr.json | jq +{ + "CN": "admin", + "key": { + "algo": "rsa", + "size": 2048 + }, + "names": [ + { + "C": "CN", + "L": "HangZhou", + "O": "system:masters", + "OU": "Containerum", + "ST": "BinJiang" + } + ] +} +``` + +Generate a certificate: + +```bash +cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin +``` + +The result is as follows: + +```bash +admin.csr admin-key.pem admin.pem +``` + +## Generating a service-account Certificate + +Compile the CSR configuration file of the service-account account, for example, service-account-csr.json: + +```bash +cat service-account-csr.json | jq +{ + "CN": "service-accounts", + "key": { + "algo": "rsa", + "size": 2048 + }, + "names": [ + { + "C": "CN", + "L": "HangZhou", + "O": "Kubernetes", + "OU": "openEuler k8s install", + "ST": "BinJiang" + } + ] +} +``` + +Generate a certificate: + +```bash +cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -profile=kubernetes service-account-csr.json | cfssljson -bare service-account +``` + +The result is as follows: + +```bash +service-account.csr service-account-key.pem service-account.pem +``` + +## Generating the kube-controller-manager Certificate + +Compile the CSR configuration of kube-controller-manager: + +```bash +{ + "CN": "system:kube-controller-manager", + "key": { + "algo": "rsa", + "size": 2048 + }, + "names": [ + { + "C": "CN", + "L": "HangZhou", + "O": "system:kube-controller-manager", + "OU": "openEuler k8s kcm", + "ST": "BinJiang" + } + ] +} +``` + +Generate a certificate: + +```bash +cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -profile=kubernetes kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager +``` + +The result is as follows: + +```bash +kube-controller-manager.csr kube-controller-manager-key.pem kube-controller-manager.pem +``` + +## Generating the kube-proxy Certificate + +Compile the CSR configuration of kube-proxy: + +```bash +{ + "CN": "system:kube-proxy", + "key": { + "algo": "rsa", + "size": 2048 + }, + "names": [ + { + "C": "CN", + "L": "HangZhou", + "O": "system:node-proxier", + "OU": "openEuler k8s kube proxy", + "ST": "BinJiang" + } + ] +} +``` + +Generate a certificate: + +```bash +cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy +``` + +The result is as follows: + +```bash +kube-proxy.csr kube-proxy-key.pem kube-proxy.pem +``` + +## Generating the kube-scheduler Certificate + +Compile the CSR configuration of kube-scheduler: + +```bash +{ + "CN": "system:kube-scheduler", + "key": { + "algo": "rsa", + "size": 2048 + }, + "names": [ + { + "C": "CN", + "L": "HangZhou", + "O": "system:kube-scheduler", + "OU": "openEuler k8s kube scheduler", + "ST": "BinJiang" + } + ] +} +``` + +Generate a certificate: + +```bash +cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -profile=kubernetes kube-scheduler-csr.json | cfssljson -bare kube-scheduler +``` + +The result is as follows: + +```bash +kube-scheduler.csr kube-scheduler-key.pem kube-scheduler.pem +``` + +## Generating the kubelet Certificate + +The certificate involves the host name and IP address of the server where kubelet is located. Therefore, the configuration of each node is different. The script is compiled as follows: + +```bash +$ cat node_csr_gen.bash + +#!/bin/bash + +nodes=(k8snode1 k8snode2 k8snode3) +IPs=("192.168.122.157" "192.168.122.158" "192.168.122.159") + +for i in "${!nodes[@]}"; do + +cat > "${nodes[$i]}-csr.json" <- + Docker is an open source container engine that facilitates quick application + packaging, deployment, and delivery. +sections: + - label: Overview + href: ./overview.md + - label: Installation and Configuration + href: ./installation-and-configuration-3.md + - label: Container Management + href: ./container-management-1.md + - label: Image Management + href: ./image-management-1.md + - label: Command Reference + href: ./command-reference.md + sections: + - label: Container Engine + href: ./container-engine.md + - label: Container Management + href: ./container-management-2.md + - label: Image Management + href: ./image-management-2.md + - label: Statistics + href: ./statistics.md + - label: Docker Common Docker Issues and Solutions + href: ./docker-faqs.md diff --git a/docs/en/cloud/container_engine/docker_engine/command-reference.md b/docs/en/cloud/container_engine/docker_engine/command-reference.md new file mode 100644 index 0000000000000000000000000000000000000000..d7578e549bf15d14f944ccd1fda94911fead4829 --- /dev/null +++ b/docs/en/cloud/container_engine/docker_engine/command-reference.md @@ -0,0 +1,7 @@ +# Command Reference + +- [Command Reference](#command-reference) + - [Container Engine](#container-engine) + - [Container Management](#container-management-40) + - [Image Management](#image-management-43) + - [Statistics](#statistics) diff --git a/docs/en/cloud/container_engine/docker_engine/container-engine.md b/docs/en/cloud/container_engine/docker_engine/container-engine.md new file mode 100644 index 0000000000000000000000000000000000000000..c8dec68b4cea7ce554e54f377db8b167ef04263b --- /dev/null +++ b/docs/en/cloud/container_engine/docker_engine/container-engine.md @@ -0,0 +1,304 @@ +# Container Engine + +Docker daemon is a system process that resides in the background. Before you run a docker subcommand, start Docker daemon. + +If the Docker daemon is installed using the RPM package or system package management tool, you can run the **systemctl start docker** command to start the Docker daemon. + +The **docker** command supports the following parameters: + +1. To combine parameters of a single character, run the following command: + + ```shell + docker run -t -i busybox /bin/sh + ``` + + The command can be written as follows: + + ```shell + docker run -ti busybox /bin/sh + ``` + +2. **bool** command parameters such as **--icc=true**, are displayed in the command help. If this parameter is not used, the default value displayed in the command help is used. If this parameter is used, the opposite value of the value displayed in the command help is used. In addition, if **--icc** is not added when Docker daemon is started, **--icc=true** is used by default. Otherwise, **--icc=false** is used. +3. Parameters such as **--attach=\[\]** in the command help indicate that these parameters can be set for multiple times. For example: + + ```shell + docker run --attach=stdin --attach=stdout -i -t busybox /bin/sh + ``` + +4. Parameters such as **-a** and **--attach=\[\]** in the command help indicate that the parameter can be specified using either **-a** _value_ or **--attach=**_value_. For example: + + ```shell + docker run -a stdin --attach=stdout -i -t busybox /bin/sh + ``` + +5. Parameters such as **--name=""** can be configured with a character string and can be configured only once. Parameters such as **-c=** can be configured with an integer and can be configured only once. + +**Table 1** Parameters specified during the Docker daemon startup + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Parameter

+

Description

+

--api-cors-header

+

CORS header information for enabling remote API calling. This interface supports the secondary development of upper-layer applications, which sets the CORS header for a remote API.

+

--authorization-plugin=[]

+

Authentication plug-in.

+

-b, --bridge=""

+

Existing bridge device mounting to the docker container. Note: none can be used to disable the network in the container.

+

--bip=""

+

Bridge IP address, which is automatically created using the CIDR address. Note: this parameter cannot be used with -b .

+

--cgroup-parent

+

cgroup parent directory configured for all containers.

+

--config-file=/etc/docker/daemon.json

+

Configuration file for starting Docker daemon.

+

--containerd

+

Socket path of containerd.

+

-D, --debug=false

+

Specifies whether to enable the debugging mode.

+

--default-gateway

+

Default gateway of the container IPv4 address.

+

--default-gateway-v6

+

Default gateway of the container IPv6 address.

+

--default-ulimit=[]

+

Default ulimit value of the container.

+

--disable-legacy-registry

+

Disables the original registry.

+

--dns=[]

+

DNS server of the forcibly used container.

+

Example: --dns 8.8.x.x

+

--dns-opt=[]

+

DNS option.

+

--dns-search=[]

+

Forcibly searches DNS search domain name used by a container.

+

Example: --dns-search example.com

+

--exec-opt=[]

+

Parameter to be executed when a container is started.

+

For example, set the native.umask parameter.

+
#The umask value of the started container is 0022.--exec-opt native.umask=normal 
+# The umask value of the started container is 0027 (default value).
+--exec-opt  native.umask=secure
+

Note: If native.umask is also configured in docker create or docker run command, the configuration in command is used.

+

--exec-root=/var/run/docker

+

Root directory for storing the execution status file.

+

--fixed-cidr=""

+

Fixed IP address (for example, 10.20.0.0/16) of the subnet. The IP address of the subnet must belong to the network bridge.

+

--fixed-cidr-v6

+

Fixed IPv6 address.

+

-G, --group="docker"

+

Group assigned to the corresponding Unix socket in the background running mode. Note: When an empty string is configured for this parameter, the group information is removed.

+

-g, --graph="/var/lib/docker"

+

The root directory for running docker.

+

-H, --host=[]

+

Socket bound in background mode. One or more sockets can be configured using tcp://host:port, unix:///path to socket, fd://* or fd://socketfd. Example:

+

$ dockerd -H tcp://0.0.0.0:2375

+

or

+

$ export DOCKER_HOST="tcp://0.0.0.0:2375"

+

--insecure-registry=[]

+

Registry for insecure connections. By default, the Docker uses TLS certificates to ensure security for all connections. If the registry does not support HTTPS connections or the certificate is issued by an unknown certificate authority of the Docker daemon, you need to configure --insecure-registry=192.168.1.110:5000 when starting the daemon. This parameter needs to be configured if a private registry is used.

+

--image-layer-check=true

+

Image layer integrity check. To enable the function, set this parameter to true. Otherwise, set this parameter to false. If this parameter is not configured, the function is disabled by default.

+

When Docker is started, the image layer integrity is checked. If the image layer is damaged, the related images are unavailable. Docker cannot verify empty files, directories, or link files. Therefore, if the preceding files are lost due to a power failure, the integrity check of Docker image data may fail. When the Docker version changes, check whether the parameter is supported. If not supported, delete it from the configuration file.

+

--icc=true

+

Enables communication between containers.

+

--ip="0.0.0.0"

+

Default IP address used when a container is bound to a port.

+

--ip-forward=true

+

Starts the net.ipv4.ip_forward process of the container.

+

--ip-masq=true

+

Enables IP spoofing.

+

--iptables=true

+

Starts the iptables rules defined by the Docker container.

+

-l, --log-level=info

+

Log level.

+

--label=[]

+

Daemon label, in key=value format.

+

--log-driver=json-file

+

Default log driver of container logs.

+

--log-opt=map[]

+

Log drive parameters.

+

--mtu=0

+

MTU value of the container network. If this parameter is not configured, value of route MTU is used by default. If the default route is not configured, set this parameter to the constant value 1500.

+

-p, --pidfile="/var/run/docker.pid"

+

PID file path of the background process.

+

--raw-logs

+

Logs with all timestamps and without the ANSI color scheme.

+

--registry-mirror=[]

+

Image registry preferentially used by the dockerd.

+

-s, --storage-driver=""

+

Storage driver used when a container is forcibly run.

+

--selinux-enabled=false

+

Enables SELinux. If the kernel version is 3.10.0-862.14 or later, this parameter cannot be set to true.

+

--storage-opt=[]

+

Storage driver parameter. This parameter is valid only when the storage driver is devicemapper. Example: dockerd --storage-opt dm.blocksize=512K

+

--tls=false

+

Enables the TLS authentication.

+

--tlscacert="/root/.docker/ca.pem"

+

Certificate file path that has been authenticated by the CA.

+

--tlscert="/root/.docker/cert.pem"

+

File path of the TLS certificates.

+

--tlskey="/root/.docker/key.pem"

+

File path of TLS keys.

+

--tlsverify=false

+

Verifies the communication between the background processes and the client using TLS.

+

--insecure-skip-verify-enforce

+

Whether to forcibly skip the verification of the certificate host or domain name. The default value is false.

+

--use-decrypted-key=true

+

Whether to use the decryption private key.

+

--userland-proxy=true

+

Whether to use the userland proxy for the container LO device.

+

--userns-remap

+

User namespace-based user mapping table in the container.

+
NOTE:

This parameter is not supported in the current version.

+
+
diff --git a/docs/en/cloud/container_engine/docker_engine/container-management-1.md b/docs/en/cloud/container_engine/docker_engine/container-management-1.md new file mode 100644 index 0000000000000000000000000000000000000000..e820fd47cdb34a7464e25ed6bef6c21d96ed2d7e --- /dev/null +++ b/docs/en/cloud/container_engine/docker_engine/container-management-1.md @@ -0,0 +1,706 @@ +# Container Management + +- [Container Management](#container-management) + - [Creating a Container](#creating-a-container) + - [Creating Containers Using hook-spec](#creating-containers-using-hook-spec) + - [Configuring Health Check During Container Creation](#configuring-health-check-during-container-creation) + - [Stopping and Deleting a Container](#stopping-and-deleting-a-container) + - [Querying Container Information](#querying-container-information) + - [Modification Operations](#modification-operations) + +## Creating a Container + +### Downloading Images + +Only user **root** can run the **docker** command. If you log in as a common user, you need to use the **sudo** command before running the **docker** command. + +```shell +docker pull busybox +``` + +This command is used to download the **busybox:latest** image from the official Docker registry. \(If no tag is specified in the command, the default tag name **latest** is used.\) During the image download, the system checks whether the dependent layer exists locally. If yes, the image download is skipped. When downloading images from a private registry, specify the registry description. For example, if a private registry containing some common images is created and its IP address is **192.168.1.110:5000**, you can run the following command to download the image from the private registry: + +```shell +docker pull 192.168.1.110:5000/busybox +``` + +The name of the image downloaded from the private registry contains the registry address information, which may be too long. Run the **docker tag** command to generate an image with a shorter name. + +```shell +docker tag 192.168.1.110:5000/busybox busybox +``` + +Run the **docker images** command to view the local image list. + +### Running a Simple Application + +```shell +$ docker run busybox /bin/echo "Hello world" +Hello world +``` + +This command uses the **busybox:latest** image to create a container, and executes the **echo "Hello world"** command in the container. Run the following command to view the created container: + +```shell +$ docker ps -l +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +d8c0a3315bc0 busybox"/bin/echo 'Hello wo..." 5 seconds ago Exited (0) 3 seconds ago practical_franklin +``` + +### Creating an Interactive Container + +```shell +$ docker run -it busybox /bin/bash +root@bf22919af2cf:/# ls +bin boot dev etc home lib media mnt opt proc root run sbin srv sys tmp usr var +root@bf22919af2cf:/# pwd +/ +``` + +The **-ti** option allocates a pseudo terminal to the container and uses standard input \(STDIN\) for interaction. You can run commands in the container. In this case, the container is an independent Linux VM. Run the **exit** command to exit the container. + +### Running a Container in the Background + +Run the following command. **-d** indicates that the container is running in the background. **--name=container1** indicates that the container name is **container1**. + +```shell +$ docker run -d --name=container1 busybox /bin/sh -c "while true;do echo hello world;sleep 1;done" +7804d3e16d69b41aac5f9bf20d5f263e2da081b1de50044105b1e3f536b6db1c +``` + +The command output contains the container ID but does not contain **hello world**. In this case, the container is running in the background. You can run the **docker ps** command to view the running container. + +```shell +$ docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +7804d3e16d69 busybox "/bin/sh -c 'while tr" 11 seconds ago Up 10 seconds container1 +``` + +Run the following **docker logs** command to view the output during container running: + +```shell +$ docker logs container1 +hello world +hello world +hello world +... +``` + +### Container Network Connection + +By default, a container can access an external network, while port mapping is required when an external network accesses a container. The following uses how to run the private registry service in Docker as an example. In the following command, **-P** is used to expose open ports in the registry to the host. + +```shell +$ docker run --name=container_registry -d -P registry +cb883f6216c2b08a8c439b3957fb396c847a99079448ca741cc90724de4e4731 +``` + +The container\_registry container has been started, but the mapping between services in the container and ports on the host is not clear. You need to run the **docker port** command to view the port mapping. + +```shell +$ docker port container_registry +5000/tcp -> 0.0.0.0:49155 +``` + +The command output shows that port 5000 in the container is mapped to port 49155 on the host. You can access the registry service by using the host IP address **49155**. Enter **** in the address box of the browser and press **Enter**. The registry version information is displayed. + +When running registry images, you can directly specify the port mapping, as shown in the following: + +```shell +docker run --name=container_registry -d -p 5000:5000 registry +``` + +**-p 5000:5000** is used to map port 5000 in the container to port 5000 on the host. + +### Precautions + +- **Do Not Add -a stdin Independently During Container Startup** + + When starting a container, you must add **-a stdout** or **-a stderr** together with **-a stdin** instead of **-a stdin** only. Otherwise, the device stops responding even after the container exits. + +- **Do Not Use the Long Or Short ID of an Existing Container As the Name of a New Container** + + When creating a container, do not use the long or short ID of the existing container A as the name of the new container B. If the long ID of container A is used as the name of container B, Docker will match container A even though the name of container B is used as the specified target container for operations. If the short ID of container A is used as the name of container B, Docker will match container B even though the short ID of container A is used as the specified target container for operations. This is because Docker matches the long IDs of all containers first. If the matching fails, the system performs exact matching using the value of **container\_name**. If matching failure persists, the container ID is directly matched in fuzzy mode. + +- **Containers That Depend on Standard Input and Output, Such As sh/bash, Must Use the -ti Parameter to Avoid Exceptions** + + Normal case: If you do not use the **-ti** parameter to start a process container such as sh/bash, the container exits immediately. + + The cause of this problem is that Docker creates a stdin that matches services in the container first. If the interactive parameters such as **-ti** are not set, Docker closes pipe after the container is started and the service container process sh/bash exits after stdin is closed. + + Exception: If Docker daemon is forcibly killed in a specific phase \(before pipe is closed\), daemon of the pipe is not closed in time. In this case, the sh/bash process does not exit even without **-ti**. As a result, an exception occurs. You need to manually clear the container. + + After being restarted, daemon takes over the original container stream. Containers without the **-ti** parameter may not be able to process the stream because these containers do not have streams to be taken over in normal cases. In actual services, sh/bash without the **-ti** parameter does not take effect and is seldom used. To avoid this problem, the **-ti** parameter is used to restrict interactive containers. + +- **Container Storage Volumes** + + If you use the **-v** parameter to mount files on the host to a container when the container is started, the inodes of the files may be changed when you run the **vi** or **sed** command to modify the files on the host or in the container. As a result, files on the host and in the container are not synchronized. Do not mount files in the container in this mode \(or do not use together with the **vi** and **sed** commands\). You can also mount the upper-layer directories of the files to avoid exceptions. The **nocopy** option can be used to prevent original files in the mount point directory of a container from being copied to the source directory of the host when Docker mounts volumes. However, this option can be used only when an anonymous volume is mounted and cannot be used in the bind mount scenario. + +- **Do Not Use Options That May Affect the Host** + + The **--privileged** option enables all permissions for a container. On the container, mounting operations can be performed and directories such as **/proc** and **/sys** can be modified, which may affect the host. Therefore, do not use this option for common containers. + + A host-shared namespace, such as the **--pid host**, **--ipc host**, or **--net host** option, can enable a container to share the namespace with the host, which will also affect the host. Therefore, do not use this option. + +- **Do Not Use the Unstable Kernel Memory Cgroup** + + Kernel memory cgroup on the Linux kernel earlier than 4.0 is still in the experimental phase and runs unstably. Therefore, do not use kernel memory cgroup. + + When the **docker run --kernel-memory** command is executed, the following alarm is generated: + + ```text + WARNING: You specified a kernel memory limit on a kernel older than 4.0. Kernel memory limits are experimental on older kernels, it won't work as expected as expected and can cause your system to be unstable. + ``` + +- **blkio-weight Parameter Is Unavailable in the Kernel That Supports blkio Precise Control** + + **--blkio-weight-device** can implement more accurate blkio control in a container. The control requires a specified disk device, which can be implemented through the **--blkio-weight-device** parameter of Docker. In this kernel, Docker does not provide the **--blkio-weight** mode to limit the container blkio. If you use this parameter to create a container, the following error is reported: + + ```text + docker: Error response from daemon: oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:398: container init caused \"process_linux.go:369: setting cgroup config for ready process caused \\\"blkio.weight not supported, use weight_device instead\\\"\"" + ``` + +- **Using --blkio-weight-device in CFQ Scheduling Policy** + + The **--blkio-weight-device** parameter works only when the disk works in the Completely Fair Queuing \(CFQ\) policy. + + You can view the scheduler file \(**/sys/block/**_disk_**/queue/scheduler**\) to obtain the policies supported by the disk and the current policy. For example, you can run the following command to view **sda**. + + ```shell + # cat /sys/block/sda/queue/scheduler noop [deadline] cfq + ``` + + **sda** supports the following scheduling policies: **noop**, **deadline**, and **cfq**, and the **deadline** policy is being used. You can run the **echo** command to change the policy to **cfq**. + + ```shell + # echo cfq > /sys/block/sda/queue/scheduler + ``` + +- **systemd Usage Restrictions in Basic Container Images** + + When containers created from basic images are used, systemd in basic images is used only for system containers. + +### Concurrent Performance + +- There is an upper limit for the message buffer in Docker. If the number of messages exceeds the upper limit, the messages are discarded. Therefore, it is recommended that the number of commands executed concurrently should not exceed 1000. Otherwise, the internal messages in Docker may be lost and the container may fail to be started. +- When containers are concurrently created and restarted, the error message"oci runtime error: container init still running" is occasionally reported. This is because containerd optimizes the performance of the event waiting queue. When a container is stopped, the **runc delete** command is executed to kill the init processes in the container within 1s. If the init processes are not killed within 1s, runC returns this error message. The garbage collection \(GC\) mechanism of containerd reclaims residual resources after **runc delete** is executed at an interval of 10s. Therefore, operations on the container are not affected. If the preceding error occurs, wait for 4 or 5s and restart the container. + +### Security Feature Interpretation + +1. The following describes default permission configuration analysis of Docker. + + In the default configuration of a native Docker, capabilities carried by each default process are as follows: + + ```text + "CAP_CHOWN", + "CAP_DAC_OVERRIDE", + "CAP_FSETID", + "CAP_FOWNER", + "CAP_MKNOD", + "CAP_NET_RAW", + "CAP_SETGID", + "CAP_SETUID", + "CAP_SETFCAP", + "CAP_SETPCAP", + "CAP_NET_BIND_SERVICE", + "CAP_SYS_CHROOT", + "CAP_KILL", + "CAP_AUDIT_WRITE", + ``` + + The default seccomp configuration is a whitelist. If any syscall is not in the whitelist, **SCMP\_ACT\_ERRNO** is returned by default. Different system invoking is enabled for different caps of Docker. If a capability is not in the whitelist, Docker will not assign it to the container by default. + +2. CAP\_SYS\_MODULE + + CAP\_SYS\_MODULE allows a container to insert or remove ko modules. Adding this capability allows the container to escape or even damage the kernel. Namespace provides the maximum isolation for a container. In the ko module, you only need to point its namespace to **init\_nsproxy**. + +3. CAP\_SYS\_ADMIN + + The sys\_admin permission provides the following capabilities for a container: + + - For file system: **mount**, **umount**, and **quotactl** + - For namespace setting: **setns**, **unshare**, and **clone new namespace** + - driver ioctl + - For PCI control: **pciconfig\_read**, **pciconfig\_write**, and **pciconfig\_iobase** + - **sethostname** + +4. CAP\_NET\_ADMIN + + CAP\_NET\_ADMIN allows a container to access network interfaces and sniff network traffic. The container can obtain the network traffic of all containers including the host, which greatly damages network isolation. + +5. CAP\_DAC\_READ\_SEARCH + + CAP\_DAC\_READ\_SEARCH calls the open\_by\_handle\_at and name\_to\_handle\_at system calls. If the host is not protected by SELinux, the container can perform brute-force search for the inode number of the file\_handle structure to open any file on the host, which affects the isolation of the file system. + +6. CAP\_SYS\_RAWIO + + CAP\_SYS\_RAWIO allows a container to write I/O ports to the host, which may cause the host kernel to crash. + +7. CAP\_SYS\_PTRACE + + The ptrace permission for a container provides ptrace process debugging in the container. RunC has fixed this vulnerability. However, some tools, such as nsenter and docker-enter, are not protected. In a container, processes executed by these tools can be debugged to obtain resource information \(such as namespace and fd\) brought by these tools. In addition, ptrace can bypass seccomp, greatly increasing attack risks of the kernel. + +8. Docker capability interface: --cap-add all + + --cap-add all grants all permissions to a container, including the dangerous permissions mentioned in this section, which allows the container to escape. + +9. Do not disable the seccomp feature of Docker. + + Docker has a default seccomp configuration with a whitelist. **sys\_call** that is not in the whitelist is disabled by seccomp. You can disable the seccomp feature by running **--security-opt 'seccomp:unconfined'**. If seccomp is disabled or the user-defined seccomp configuration is used but the filtering list is incomplete, attack risks of the kernel in the container are increased. + +10. Do not set the **/sys** and **/proc** directories to writable. + + The **/sys** and **/proc** directories contain Linux kernel maintenance parameters and device management interfaces. If the write permission is configured for the directories in a container, the container may escape. + +11. Docker open capability: --CAP\_AUDIT\_CONTROL + + The permission allows a container to control the audit system and run the **AUDIT\_TTY\_GET** and **AUDIT\_TTY\_SET** commands to obtain the TTY execution records \(including the **root** password\) recorded in the audit system. + +12. CAP\_BLOCK\_SUSPEND and CAP\_WAKE\_ALARM + + The permission provides a container the capability to block the system from suspending \(epoll\). + +13. CAP\_IPC\_LOCK + + With this permission, a container can break the max locked memory limit in **ulimit** and use any mlock large memory block to cause DoS attacks. + +14. CAP\_SYS\_LOG + + In a container with this permission, system kernel logs can be read by using dmesg to break through kernel kaslr protection. + +15. CAP\_SYS\_NICE + + In a container with this permission, the scheduling policy and priority of a process can be changed, causing DoS attacks. + +16. CAP\_SYS\_RESOURCE + + With this permission, a container can bypass resource restrictions, such as disk space resource restriction, keymaps quantity restriction, and **pipe-size-max** restriction, causing DoS attacks. + +17. CAP\_SYS\_TIME + + In a container with this capability, the time on the host can be changed. + +18. Risk analysis of Docker default capabilities + + The default capabilities of Docker include CAP\_SETUID and CAP\_FSETID. If the host and a container share a directory, the container can set permissions for the binary file in the shared directory. Common users on the host can use this method to elevate privileges. With the CAP\_AUDIT\_WRITE capability, a container can write logs to the host, and the host must be configured with log anti-explosion measures. + +19. Docker and host share namespace parameters, such as **--pid**, **--ipc**, and **--uts**. + + This parameter indicates that the container and host share the namespace. The container can attack the host as the namespace of the container is not isolated from that of the host. For example, if you use **--pid** to share PID namespace with the host, the PID on the host can be viewed in the container, and processes on the host can be killed at will. + +20. **--device** is used to map the sensitive directories or devices of the host to the container. + + The Docker management plane provides interfaces for mapping directories or devices on a host to the container, such as **--device** and **-v**. Do not map sensitive directories or devices on the host to the container. + +## Creating Containers Using hook-spec + +### Principles and Application Scenarios + +Docker supports the extended features of hooks. The execution of hook applications and underlying runC complies with the OCI standards. For details about the standards, visit + +There are three types of hooks: prestart, poststart, and poststop. They are respectively used before applications in the container are started, after the applications are started, and after the applications are stopped. + +### API Reference + +The **--hook-spec** parameter is added to the **docker run** and **create** commands and is followed by the absolute path of the **spec** file. You can specify the hooks to be added during container startup. These hooks will be automatically appended after the hooks that are dynamically created by Docker \(currently only libnetwork prestart hook\) to execute programs specified by users during the container startup or destruction. + +The structure of **spec** is defined as follows: + +```go +// Hook specifies a command that is run at a particular event in the lifecycle of a container +type Hook struct{ + Path string `json:"path"` + Args []string `json:"args,omitempty"` + Env []string `json:"env,omitempty"` + Timeout *int `json:"timeout,omitempty"` +} +// Hooks for container setup and teardown +type Hooks struct{ + // Prestart is a list of hooks to be run before the container process is executed. + // On Linux, they are run after the container namespaces are created. + Prestart []Hook `json:"prestart,omitempty"` + // Poststart is a list of hooks to be run after the container process is started. + Poststart []Hook `json:"poststart,omitempty"` + // Poststop is a list of hooks to be run after the container process exits. + Poststop []Hook `json:"poststop,omitempty"` +} +``` + +- The **Path**, **Args**, and **Env** parameters are mandatory. +- **Timeout** is optional, while you are advised to set this parameter to a value ranging from 1 to 120. The parameter type is int. Floating point numbers are not allowed. +- The content of the **spec** file must be in JSON format as shown in the preceding example. If the format is incorrect, an error is reported. +- Both **docker run --hook-spec /tmp/hookspec.json**_xxx_, and **docker create --hook-spec /tmp/hookspec.json **_xxx_** && docker start**_xxx_ can be used. + +### Customizing Hooks for a Container + +Take adding a NIC during the startup as an example. The content of the **hook spec** file is as follows: + +```json +{ + "prestart": [ + { + "path": "/var/lib/docker/hooks/network-hook", + "args": ["network-hook", "tap0", "myTap"], + "env": [], + "timeout": 5 + } + ], + "poststart":[], + "poststop":[] +} +``` + +Specify prestart hook to add the configuration of a network hook. The path is **/var/lib/docker/hooks/network-hook**. **args** indicates the program parameters. Generally, the first parameter is the program name, and the second parameter is the parameter accepted by the program. For the network-hook program, two parameters are required. One is the name of the NIC on the host, and the other is the name of the NIC in the container. + +- Precautions + 1. The **hook** path must be in the**hooks** folder in the **graph** directory \(**--graph**\) of Docker. Its default value is **/var/lib/docker/hooks**. You can run the **docker info** command to view the root path. + + ```shell + $ docker info + ... + Docker Root Dir: /var/lib/docker + ... + ``` + + This path may change due to the user's manual configuration and the use of user namespaces \(**daemon --userns-remap**\). After the symbolic link of the path is parsed, the parsed path must start with _Docker Root Dir_**/hooks** \(for example, **/var/lib/docker/hooks**\). Otherwise, an error message is displayed. + + 2. The **hook** path must be an absolute path because daemon cannot properly process a relative path. In addition, an absolute path meets security requirements. + 3. The information output by the hook program to stderr is output to the client and affects the container lifecycle \(for example, the container may fail to be started\). The information output to stdout is ignored. + 4. Do not reversely call Docker instructions in hooks. + 5. The execute permission must have been granted on the configured hook execution file. Otherwise, an error is reported during hook execution. + 6. The execution time of the hook operation must be as short as possible. If the prestart period is too long \(more than 2 minutes\), the container startup times out. If the poststop period is too long \(more than 2 minutes\), the container is abnormal. + + The known exceptions are as follows: When the **docker stop** command is executed to stop a container and the clearing operation is performed after 2 minutes, the hook operation is not complete. Therefore, the system waits until the hook operation is complete \(the process holds a lock\). As a result, all operations related to the container stop responding. The operations can be recovered only after the hook operation is complete. In addition, the two-minute timeout processing of the **docker stop** command is an asynchronous process. Therefore, even if the **docker stop** command is successfully executed, the container status is still **up**. The container status is changed to **exited** only after the hook operation is completed. + +- Suggestions + 1. You are advised to set the hook timeout threshold to a value less than 5s. + 2. You are advised to configure only one prestart hook, one poststart hook, and one poststop hook for each container. If too many hooks are configured, the container startup may take a long time. + 3. You are advised to identify the dependencies between multiple hooks. If required, you need to adjust the sequence of the hook configuration files according to the dependencies. The execution sequence of hooks is based on the sequence in the configured **spec** file. + +### Multiple **hook-spec** + +If multiple hook configuration files are available and you need to run multiple hooks, you must manually combine these files into a configuration file and specify the new configuration file by using the **--hook-spec** parameter. Then all hooks can take effect. If multiple **--hook-spec** parameters are configured, only the last one takes effect. + +Configuration examples: + +The content of the **hook1.json** file is as follows: + +```shell +# cat /var/lib/docker/hooks/hookspec.json +{ + "prestart": [ + { + "path": "/var/lib/docker/hooks/lxcfs-hook", + "args": ["lxcfs-hook", "--log", "/var/log/lxcfs-hook.log"], + "env": [] + } + ], + "poststart":[], + "poststop":[] +} +``` + +The content of the **hook2.json** file is as follows: + +```shell +# cat /etc/isulad-tools/hookspec.json +{ + "prestart": [ + { + "path": "/docker-root/hooks/docker-hooks", + "args": ["docker-hooks", "--state", "prestart"], + "env": [] + } + ], + "poststart":[], + "poststop":[ + { + "path": "/docker-root/hooks/docker-hooks", + "args": ["docker-hooks", "--state", "poststop"], + "env": [] + } + ] +} +``` + +The content in JSON format after manual combination is as follows: + +```json +{ + "prestart":[ + { + "path": "/var/lib/docker/hooks/lxcfs-hook", + "args": ["lxcfs-hook", "--log", "/var/log/lxcfs-hook.log"], + "env": [] + }, + { + "path": "/docker-root/hooks/docker-hooks", + "args": ["docker-hooks", "--state", "prestart"], + "env": [] + } + ], + "poststart":[], + "poststop":[ + { + "path": "/docker-root/hooks/docker-hooks", + "args": ["docker-hooks", "--state", "poststop"], + "env": [] + } + ] +} +``` + +Docker daemon reads the binary values of hook in actions such as prestart in the hook configuration files in sequence based on the array sequence and executes the actions. Therefore, you need to identify the dependencies between multiple hooks. If required, you need to adjust the sequence of the hook configuration files according to the dependencies. + +### Customizing Default Hooks for All Containers + +Docker daemon can receive the **--hook-spec** parameter. The semantics of **--hook-spec** is the same as that of **--hook-spec** in **docker create** or **docker run**. You can also add hook configurations to the **/etc/docker/daemon.json** file. + +```json +{ + "hook-spec": "/tmp/hookspec.json" +} +``` + +When a container is running, hooks specified in **--hook-spec** defined by daemon are executed first, and then hooks customized for each container are executed. + +## Configuring Health Check During Container Creation + +Docker provides the user-defined health check function for containers. You can configure the **HEALTHCHECK CMD** option in the Dockerfile, or configure the **--health-cmd** option when a container is created so that commands are periodically executed in the container to monitor the health status of the container based on return values. + +### Configuration Methods + +- Add the following configurations to the Dockerfile file: + + ```text + HEALTHCHECK --interval=5m --timeout=3s --health-exit-on-unhealthy=true \ + CMD curl -f http://localhost/ || exit 1 + ``` + + The configurable options are as follows: + + 1. **--interval=DURATION**: interval between two consecutive command executions. The default value is **30s**. After a container is started, the first check is performed after the interval time. + 2. **--timeout=DURATION**: maximum duration for executing a single check command. If the execution times out, the command execution fails. The default value is **30s**. + 3. **--start-period=DURATION**: container initialization period. The default value is **0s**. During the initialization, the health check is also performed, while the health check failure is not counted into the maximum number of retries. However, if the health check is successful during initialization, the container is considered as started. All subsequent consecutive check failures are counted in the maximum number of retries. + 4. **--retries=N**. maximum number of retries for the health check. The default value is **3**. + 5. **--health-exit-on-unhealthy=BOOLEAN**: whether to kill a container when it is unhealthy. The default value is **false**. + 6. **CMD**: This option is mandatory. If **0** is returned after a command is run in a container, the command execution succeeds. If a value other than **0** is returned, the command execution fails. + + After **HEALTHCHECK** is configured, related configurations are written into the image configurations during image creation. You can run the **docker inspect** command to view the configurations. For example: + + ```json + "Healthcheck": { + "Test": [ + "CMD-SHELL", + "/test.sh" + ] + }, + ``` + +- Configurations during container creation: + + ```shell + docker run -itd --health-cmd "curl -f http://localhost/ || exit 1" --health-interval 5m --health-timeout 3s --health-exit-on-unhealthy centos bash + ``` + + The configurable options are as follows: + + 1. **--health-cmd**: This option is mandatory. If **0** is returned after a command is run in a container, the command execution succeeds. If a value other than **0** is returned, the command execution fails. + 2. **--health-interval**: interval between two consecutive command executions. The default value is **30s**. The upper limit of the value is the maximum value of Int64 \(unit: nanosecond\). + 3. **--health-timeout**: maximum duration for executing a single check command. If the execution times out, the command execution fails. The default value is **30s**. The upper limit of the value is the maximum value of Int64 \(unit: nanosecond\). + 4. **--health-start-period**: container initialization time. The default value is **0s**. The upper limit of the value is the maximum value of Int64 \(unit: nanosecond\). + 5. **--health-retries**: maximum number of retries for the health check. The default value is **3**. The maximum value is the maximum value of Int32. + 6. **--health-exit-on-unhealthy**: specifies whether to kill a container when it is unhealthy. The default value is **false**. + + After the container is started, the **HEALTHCHECK** configurations are written into the container configurations. You can run the **docker inspect** command to view the configurations. For example: + + ```json + "Healthcheck": { + "Test": [ + "CMD-SHELL", + "/test.sh" + ] + }, + ``` + +### Check Rules + +1. After a container is started, the container status is **health:starting**. +2. After the period specified by **start-period**, the **cmd** command is periodically executed in the container at the interval specified by **interval**. That is, after the command is executed, the command will be executed again after the specified period. +3. If the **cmd** command is successfully executed within the time specified by **timeout** and the return value is **0**, the check is successful. Otherwise, the check fails. If the check is successful, the container status changes to **health:healthy**. +4. If the **cmd** command fails to be executed for the number of times specified by **retries**, the container status changes to **health:unhealthy**, and the container continues the health check. +5. When the container status is **health:unhealthy**, the container status changes to **health:healthy** if a check succeeds. +6. If **--health-exit-on-unhealthy** is set, and the container exits due to reasons other than being killed \(the returned exit code is **137**\), the health check takes effect only after the container is restarted. +7. When the **cmd** command execution is complete or times out, Docker daemon will record the start time, return value, and standard output of the check to the configuration file of the container. A maximum of five latest records can be recorded. In addition, the configuration file of the container stores health check parameters. + +Run the **docker ps** command to view the container status. + +```shell +$ docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +7de2228674a2 testimg "bash" About an hour ago Up About an hour (unhealthy) cocky_davinci +``` + +When the container is running, the health check status is written into the container configurations. You can run the **docker inspect** command to view the configurations. + +```json +"Health": { + "Status": "healthy", + "FailingStreak": 0, + "Log": [ + { + "Start": "2018-03-07T07:44:15.481414707-05:00", + "End": "2018-03-07T07:44:15.556908311-05:00", + "ExitCode": 0, + "Output": "" + }, + { + "Start": "2018-03-07T07:44:18.557297462-05:00", + "End": "2018-03-07T07:44:18.63035891-05:00", + "ExitCode": 0, + "Output": "" + }, + ...... +} +``` + +> [!NOTE]NOTE +> +> - A maximum of five health check status records can be stored in a container. The last five records are saved. +> - Only one health check configuration item can take effect in a container at a time. The later items configured in the Dockerfile will overwrite the earlier ones. Configurations during container creation will overwrite those in images. +> - In the Dockerfile, you can set **HEALTHCHECK NONE** to cancel the health check configuration in a referenced image. When a container is running, you can set **--no-healthcheck** to cancel the health check configuration in an image. Do not configure the health check and **--no-healthcheck** parameters at the same time during the startup. +> - After a container with configured health check parameters is started, if Docker daemon exits, the health check is not executed. After Docker daemon is restarted, the container health status changes to **starting**. Afterwards, the check rules are the same as above. +> - If health check parameters are set to **0** during container image creation, the default values are used. +> - If health check parameters are set to **0** during container startup, the default values are used. + +## Stopping and Deleting a Container + +Run the **docker stop** command to stop the container named **container1**. + +```shell +docker stop container1 +``` + +Or run the **docker kill** command to kill and stop the container. + +```shell +docker kill container1 +``` + +After the container is stopped, run the **docker rm** command to delete the container. + +```shell +docker rm container1 +``` + +Or run the **docker rm -f** command to forcibly delete the container. + +```shell +docker rm -f container1 +``` + +### Precautions + +- Do not run the **docker rm -f**_XXX_ command to delete a container. If you forcibly delete a container, the **docker rm** command ignores errors during the process, which may cause residual metadata of the container. If you delete an image in common mode and an error occurs during the deletion process, the deletion fails and no metadata remains. +- Do not run the **docker kill** command. The **docker kill** command sends related signals to service processes in a container. Depending on the signal processing policies of service processes in the container may cause the result that the signal execution cannot be performed as expected. +- A container in the restarting state may not stop immediately when you run the **docker stop** command. If a container uses the restart rules, when the container is in the restarting state, there is a low probability that the **docker stop** command on the container returns immediately. The container will still be restarted with the impact of the restart rule. +- Do not run the **docker restart** command to restart a container with the **--rm** parameter. When a container with the **--rm** parameter exits, the container is automatically deleted. If the container with the **--rm** parameter is restarted, exceptions may occur. For example, if both the **--rm** and **-ti** parameters are added when the container is started, the restart operation cannot be performed on the container, otherwise, the container may stop responding and cannot exit. + +### When Using docker stop/restart to Specify -t and t<0, Ensure That Applications in the Container Can Process Stop Signal + +Stop Principle: \(The stop process is called by **Restart**.\) + +1. The SIGTERM \(15\) signal can be sent to a container for the first time. +2. Wait for a period of time \(**t** entered by the user\). +3. If the container process still exists, send the SIGKILL \(9\) signal to forcibly kill the process. + +The meaning of the input parameter **t** \(unit: s\) is as follows: + +- **t** < 0: Wait for graceful stop. This setting is preferred when users are assured that their applications have a proper stop signal processing mechanism. +- **t** = 0: Do not wait and send **kill -9** to the container immediately. +- **t** \> 0: Wait for a specified period and send **kill -9** to the container if the container does not stop within the specified period. + +Therefore, if **t** is set to a value less than 0 \(for example, **t** = **-1**\), ensure that the container application correctly processes the SIGTERM signal. If the container ignores this signal, the container will be suspended when the **docker stop** command is run. + +### Manually Deleting Containers in the Dead State As the Underlying File System May Be Busy + +When Docker deletes a container, it stops related processes of the container, changes the container status to Dead, and then deletes the container rootfs. When the file system or devicemapper is busy, the last step of deleting rootfs fails. Run the **docker ps -a** command. The command output shows that the container is in the Dead state. Containers in the Dead state cannot be started again. Wait until the file system is not busy and run the **docker rm** command again to delete the containers. + +### In PID namespace Shared Containers, If Child Container Is in pause State, Parent Container Stops Responding and the docker run Command Cannot Be Executed + +When the **--pid** parameter is used to create the parent and child containers that share PID namespace, if any process in the child container cannot exit \(for example, it is in the D or pause state\) when the **docker stop** command is executed, the **docker stop** command of the parent container is waiting. You need to manually recover the process so that the command can be executed normally. + +In this case, run the **docker inspect** command on the container in the pause state to check whether the parent container corresponding to **PidMode** is the container that requires **docker stop**. For the required container, run the **docker unpause** command to cancel the pause state of the child container. Then, proceed to the next step. + +Generally, the possible cause is that the PID namespace corresponding to the container cannot be destroyed due to residual processes. If the problem persists, use Linux tools to obtain the residual processes and locate the cause of the process exit failure in PID namespace. After the problem is solved, the container can exit. + +- Obtain PID namespace ID in a container. + + ```shell + docker inspect --format={{.State.Pid}} CONTAINERID | awk '{print "/proc/"$1"/ns/pid"}' |xargs readlink + ``` + +- Obtain threads in the namespace. + + ```shell + ls -l /proc/*/task/*/ns/pid |grep -F PIDNAMESPACE_ID |awk '{print $9}' |awk -F \/ '{print $5}' + ``` + +## Querying Container Information + +In any case, the container status should not be determined based on whether the **docker** command is successfully returned. To view the container status, you are advised to use the following command: + +```shell +docker inspect +``` + +## Modification Operations + +### Precautions for Starting Multiple Processes in Container Using docker exec + +When the first **docker exec** command executed in a container is the **bash** command, ensure that all processes started by **exec** are stopped before you run the **exit** command. Otherwise, the device may stop responding when you run the **exit** command. To ensure that the process started by **exec** is still running in the background when the **exit** command is run, add **nohup** when starting the process. + +### Usage Conflict Between docker rename and docker stats _container\_name_ + +If you run the **docker stats**_container\_name_ command to monitor a container in real time, after the container is renamed by using **docker rename**, the name displayed after **docker stats** is executed is the original name instead of the renamed one. + +### Failed to Perform docker rename Operation on Container in restarting State + +When the rename operation is performed on a container in the restarting state, Docker modifies the container network configuration accordingly. The container in the restarting state may not be started and the network does not exist. As a result, the rename operation reports an error indicating that the sandbox does not exist. You are advised to rename only containers that are not in the restarting state. + +### docker cp + +1. When you run **docker cp** to copy files to a container, all operations on the container can be performed only after the **docker cp** command is executed. +2. When a container runs as a non-**root** user, and you run the **docker cp** command to copy a non-**root** file on the host to the container, the permission role of the file in the container changes to **root**. Different from the **cp** command, the **docker cp** command changes UIDs and GIDs of the files copied to the container to **root**. + +### docker login + +After the **docker login** command is executed, **user/passwd** encrypted by AES \(256-bit\) is saved in **/root/.docker/config.json**. At the same time, _root_**.docker/aeskey** \(permission 0600\) is generated to decrypt **user/passwd** in **/root/.docker/config.json**. Currently, AES key cannot be updated periodically. You need to manually delete the AES key for updating. After AES key is updated, you need to log in to Docker daemon again to push the AES key no matter whether Docker daemon is restarted. For example: + +```text +root@hello:~/workspace/dockerfile# docker login +Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one. +Username: example Password: +Login Succeeded +root@hello:~/workspace/dockerfile# docker push example/empty +The push refers to a repository [docker.io/example/empty] +547b6288eb33: Layer already exists +latest: digest: sha256:99d4fb4ce6c6f850f3b39f54f8eca0bbd9e92bd326761a61f106a10454b8900b size: 524 +root@hello:~/workspace/dockerfile# rm /root/.docker/aeskey +root@hello:~/workspace/dockerfile# docker push example/empty +WARNING: Error loading config file:/root/.docker/config.json - illegal base64 data at input byte 0 +The push refers to a repository [docker.io/example/empty] +547b6288eb33: Layer already exists +errors: +denied: requested access to the resource is denied +unauthorized: authentication required +root@hello:~/workspace/dockerfile# docker login +Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one. +Username: example +Password: +Login Succeeded +root@hello:~/workspace/dockerfile# docker push example/empty +The push refers to a repository [docker.io/example/empty] +547b6288eb33: Layer already exists +latest: digest: sha256:99d4fb4ce6c6f850f3b39f54f8eca0bbd9e92bd326761a61f106a10454b8900b size: 524 +``` diff --git a/docs/en/cloud/container_engine/docker_engine/container-management-2.md b/docs/en/cloud/container_engine/docker_engine/container-management-2.md new file mode 100644 index 0000000000000000000000000000000000000000..b2512da473f694138f57f0eec9851cee8824c47a --- /dev/null +++ b/docs/en/cloud/container_engine/docker_engine/container-management-2.md @@ -0,0 +1,1256 @@ +# Container Management + +- [Container Management](#container-management) + - [attach](#attach) + - [commit](#commit) + - [cp](#cp) + - [create](#create) + - [diff](#diff) + - [exec](#exec) + - [export](#export) + - [inspect](#inspect) + - [logs](#logs) + - [pause/unpause](#pauseunpause) + - [port](#port) + - [ps](#ps) + - [rename](#rename) + - [restart](#restart) + - [rm](#rm) + - [run](#run) + - [start](#start) + - [stats](#stats) + - [stop](#stop) + - [top](#top) + - [update](#update) + - [wait](#wait) + +Subcommands supported by the current Docker are classified into the following groups by function: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Function

+

Command

+

Description

+

Host environment

+

version

+

Views the Docker version.

+

info

+

Views the Docker system and host environment information.

+

Container-related information

+

Container lifecycle management

+

create

+

Creates a container using an image.

+

run

+

Creates and runs a container using an image.

+

start

+

Starts a stopped container.

+

stop

+

Stops a running container.

+

restart

+

Restarts a container.

+

wait

+

Waits for a container to stop and prints the exit code.

+

rm

+

Deletes a container.

+

Container process management

+

pause

+

Suspends all processes in a container.

+

unpause

+

Resumes a suspended process in a container.

+

top

+

Views processes in a container.

+

exec

+

Executes a process in containers.

+

Container inspection tool

+

ps

+

Views running containers (without attaching any option).

+

logs

+

Displays the log information of a container.

+

attach

+

Connects standard input and output to a container.

+

inspect

+

Returns the bottom-layer information of a container.

+

port

+

Lists the port mappings between containers and hosts.

+

diff

+

Returns the changes made by the container compared with rootfs in the image.

+

cp

+

Copies files between containers and hosts.

+

export

+

Exports the file system in a container in a .tar package.

+

stats

+

Views the resource usage of a container in real time.

+

Images

+

Generates an image.

+

build

+

Creates an image using a Dockerfile.

+

commit

+

Creates an image based on the container rootfs.

+

import

+

Creates an image using the content in the .tar package as the file system.

+

load

+

Loads an image from the .tar package.

+

Image registry

+

login

+

Logs in to a registry.

+

logout

+

Logs out of a registry.

+

pull

+

Pulls an image from the registry.

+

push

+

Pushes an image to the registry.

+

search

+

Searches for an image in the registry.

+

Image management

+

images

+

Displays images in the system.

+

history

+

Displays the change history of an image.

+

rmi

+

Deletes an image.

+

tag

+

Adds a tag to an image.

+

save

+

Saves an image to a .tar package.

+

Others

+

events

+

Obtains real-time events from the Docker daemon.

+

rename

+

Renames a container.

+
+ +Some subcommands have some parameters, such as **docker run**. You can run the **docker **_command _**--help** command to view the help information of the command. For details about the command parameters, see the preceding command parameter description. The following sections describe how to use each command. + +## attach + +Syntax: **docker attach \[**_options_**\]** _container_ + +Function: Attaches an option to a running container. + +Parameter description: + +**--no-stdin=false**: Does not attach any STDIN. + +**--sig-proxy=true**: Proxies all signals of the container, except SIGCHLD, SIGKILL, and SIGSTOP. + +Example: + +```shell +$ sudo docker attach attach_test +root@2988b8658669:/# ls bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var +``` + +## commit + +Syntax: **docker commit \[**_options_**\] **_container _**\[**_repository\[:tag\]_**\]** + +Function: creates an image from a container. + +Parameter description: + +**-a**, **--author=""**: specifies an author. + +**-m**, **--message=""**: specifies the submitted information. + +**-p**, **--pause=true**: pauses the container during submission. + +Example: + +Run the following command to start a container and submit the container as a new image: + +```shell +$ sudo docker commit test busybox:test +sha256:be4672959e8bd8a4291fbdd9e99be932912fe80b062fba3c9b16ee83720c33e1 + +$ sudo docker images +REPOSITORY TAG IMAGE ID CREATED SIZE +busybox latest e02e811dd08f 2 years ago 1.09MB +``` + +## cp + +Syntax: **docker cp \[**_options_**\] **_container_**:**_src\_path_ _dest\_path_**|-** + +**docker cp \[**_options_**\]** _src\_path_**|-** _container_**:**_dest\_path_ + +Function: Copies a file or folder from a path in a container to a path on the host or copies a file or folder from the host to the container: + +Precautions: The **docker cp** command does not support the copy of files in virtual file systems such as **/proc**, **/sys**, **/dev**, and **/tmp** in the container and files in the file systems mounted by users in the container. + +Parameter description: + +**-a**, **--archive**: Sets the owner of the file copied to the container to the **container** user \(**--user**\). + +**-L**, **--follow-link**: Parses and traces the symbolic link of a file. + +Example: + +Run the following command to copy the **/test** directory in the registry container to the **/home/**_aaa_ directory on the host: + +```shell +sudo docker cp registry:/test /home/aaa +``` + +## create + +Syntax: **docker create \[**_options_**\]** _image_ **\[**_command_**\] \[**_arg_**...\]** + +Function: Creates a container using an image file and return the ID of the container. After the container is created, run the **docker start** command to start the container. _options_ are used to configure the container during container creation. Some parameters will overwrite the container configuration in the image file. _command_ indicates the command to be executed during container startup. + +Parameter description: + +**Table 1** Parameter description + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Parameter

+

Description

+

-a --attach=[]

+

Attaches the console to the STDIN, STDOUT, and STDERR of the process in the container.

+

--name=""

+

Name of a container.

+

--add-host=[host:ip]

+

Adds a mapping between the host name and IP address to the /etc/hosts in the container.

+

For example, --add-host=test:10.10.10.10.

+

--annotation

+

Sets annotations for the container. For example, set the native.umask parameter.

+
--annotation native.umask=normal #The umask value of the started container is 0022.
+--annotation native.umask=secure #The umask value of the started container is 0027.
+

If this parameter is not set, the umask configuration in dockerd is used.

+

--blkio-weight

+

Relative weight of blockio, which ranges from 10 to 1000.

+

--blkio-weight-device=[]

+

Blockio weight, which configures the relative weight.

+

-c, --cpu-shares=0

+

Relative weight of the host CPU obtained by the container. This parameter can be used to obtain a higher priority. By default, all containers obtain the same CPU priority.

+

--cap-add=[]

+

Adds Linux functions.

+

--cap-drop=[]

+

Clears Linux functions.

+

--cgroup-parent

+

cgroup parent directory for the container.

+

--cidfile=""

+

Writes the container ID to a specified file.

+

For example: --cidfile=/home/cidfile-test writes the container ID to the /home/cidfile-test file.

+

--cpu-period

+

CPU CFS period.

+

The default value is 100 ms. Generally, --cpu-period and --cpu-quota are used together. For example, --cpu-period=50000 --cpu-quota=25000 indicates that if there is one CPU, the container can obtain 50% of the CPU every 50 ms.

+

--cpus=0.5 has the same effect.

+

--cpu-quota

+

CPU CFS quota. The default value is 0, indicating that there is no restriction on the quota.

+

--cpuset-cpus

+

Number of CPUs (0-3, 0, 1) that can be used by processes in the container. By default, there is no restriction on this parameter.

+

--cpuset-mems

+

Memory nodes (0-3, 0, 1) for running processes in the container. This parameter is valid only for the NUMA system.

+

--device=[]

+

Adds the host device to a container, for example, --device=/dev/sdc:/dev/xvdc:rwm.

+

--dns=[]

+

Forcibly enables the container to use the specified DNS server. For example, --dns=114.114.xxx.xxx indicates that nameserver 114.114.xxx.xxx is written to /etc/resolv.conf of the created container and the original content is overwritten.

+

--dns-opt=[]

+

DNS options.

+

--dns-search=[]

+

Forcibly searches DNS search domain name used by a container.

+

-e, --env=[]

+

Sets environment variable for the container.

+

--env=[KERNEL_MODULES=]:

+

Inserts a specified module into a container. Currently, only the modules on the host can be inserted. After the container is deleted, the modules still reside on the host, and the --hook-spec option must be configured for the container. The following are valid parameter formats:

+

KERNEL_MODULERS=

+

KERNEL_MODULERS=a

+

KERNEL_MODULERS=a,b

+

KERNEL_MODULERS=a,b,

+

--entrypoint=""

+

Overwrites the original entrypoint in the image. The entrypoint is used to set the command executed when the container is started.

+

--env-file=[]

+

Reads environment variables from a file. Multiple environment variables are separated by lines in the file. For example: --env-file=/home/test/env indicates multiple environment variables are stored in the env file.

+

--expose=[]

+

Enables an internal port of a container. The -P option described in the following section maps the enabled port to a port on the host.

+

--group-add=[]

+

Adds a specified container to an additional group.

+

-h, --hostname=""

+

Host name.

+

--health-cmd

+

Container health check command.

+

--health-interval

+

Interval between two consecutive command executions. The default value is 30s.

+

--health-timeout

+

Maximum duration for executing a single check command. If the execution times out, the command fails to be executed. The default value is 30s.

+

--health-start-period

+

Interval between the time when the container is started and the time when the first health check is performed. The default value is 0s.

+

--health-retries

+

Maximum number of retries after a health check fails. The default value is 3.

+

--health-exit-on-unhealthy

+

Specifies whether to stop a container when the container is unhealthy. The default value is false.

+

--host-channel=[]

+

Sets a channel for communication between processes in the container and the host, in host path:container path:rw/ro:size limit format.

+

-i, --interactive=false

+

Enables STDIN even if it is not attached.

+

--ip

+

IPv4 address of a container.

+

--ip6

+

IPv6 address of a container.

+

--ipc

+

IPC namespace of a container.

+

--isolation

+

Container isolation policy.

+

-l, --label=[]

+

Label of a container.

+

--label-file=[]

+

Obtains the label from the file.

+

--link=[]

+

Links to another container. This parameter adds environment variables of the IP address and port number of the linked container to the container and adds a mapping to the /etc/hosts file, for example, --link=name:alias.

+

--log-driver

+

Log driver of a container.

+

--log-opt=[]

+

Log driver option.

+

-m, --memory=""

+

Memory limit of a container. The format is numberoptional unit, and available units are b, k, m, and g. The minimum value of this parameter is 4m.

+

--mac-address

+

MAC address of a container, for example, 92:d0:c6:0a:xx:xx.

+

--memory-reservation

+

Container memory limit. The default value is the same as that of --memory. --memory is a hard limit, and --memory-reservation is a soft limit. When the memory usage exceeds the preset value, the memory usage is dynamically adjusted (the system attempts to reduce the memory usage to a value less than the preset value when reclaiming the memory). However, the memory usage may exceed the preset value. Generally, this parameter can be used together with --memory. The value must be less than the preset value of --memory.

+

--memory-swap

+

Total usage of the common memory and swap partition. -1 indicates no restriction is set on the usage. If this parameter is not set, the swap partition size is twice the value of --memory. That is, the swap partition can use the same amount of memory as --memory.

+

--memory-swappiness=-1

+

Time when the container uses the swap memory. The value ranges from 0 to 100, in percentage.

+

--net="bridge"

+

Network mode of the container. Docker 1.3.0 has the following network modes: bridge, host, none, and container:name|id. The default value is bridge.

+
  • bridge: Creates a network stack on the bridge when the Docker daemon is started.
  • host: Uses the network stack of the host in the container.
  • none: Does not use networks.
  • container:name|id: Reuses the network stack of another container.
+

--no-healthcheck

+

Does not perform health check for a container.

+

--oom-kill-disable

+

Disables the OOM killer. You are advised not to set this parameter if the -m parameter is not set.

+

--oom-score-adj

+

Adjusts the OOM rule of a container. The value ranges from -1000 to 1000.

+

-P, --publish-all=false

+

Maps all enabled ports of a container to host ports. Containers can be accessed through the host ports. You can run the docker port command to view the mapping between container ports and host ports.

+

-p, --publish=[]

+

Maps a port in a container to a port on the host, in IP address:host port:container port | IP address::container port | host port:container port | container port format. If no IP address is configured, accesses of all NICs on the host is listened. If no host port is configured, the host port is automatically allocated.

+

--pid

+

PID namespace of a container.

+

--privileged=false

+

Grants extra permission to a container. If the --privileged option is used, the container can access all devices on the host.

+

--restart=""

+

Configures restart rule when the container exits. Currently, version 1.3.1 supports the following rules:

+
  • no: indicates that the container is not restarted when it is stopped.
  • on-failure: indicates that the container is restarted when the container exit code is not 0. This rule can be used to add the maximum number of restart times, for example, on-failure:5, indicating that the container can be restarted for a maximum of five times.
  • always: indicates the container is exited regardless of the exit code.
+

--read-only

+

Mounts the root file system of the container in read-only mode.

+

--security-opt=[]

+

Container security rule.

+

--shm-size

+

Size of the /dev/shm device. The default value is 64M.

+

--stop-signal=SIGTERM

+

Container stop signal. The default value is SIGTERM.

+

-t, --tty=false

+

Allocates a pseudo terminal.

+

--tmpfs=[]

+

Mounts the tmpfs directory.

+

-u, --user=""

+

User name or user ID.

+

--ulimit=[]

+

ulimit option.

+

--userns

+

User namespace of a container.

+

-v, --volume=[]

+

Mounts a directory of the host to the container, or create a volume in the container. For example, -v /home/test:/home mounts the /home/test directory of the host to the /home directory of the container, and -v /tmp creates the tmp folder in the root directory of the container, the folder can be shared by other containers using the --volumes-from option. The host directory cannot be mounted to the /proc subdirectory of the container. Otherwise, an error is reported when the container is started.

+

--volume-driver

+

Data volume driver of the container. This parameter is optional.

+

--volumes-from=[]

+

Mounts the volume of another container to the current container to share the volume. For example, -volumes-from container_name mounts the volume of container_name to the current container. -v and --volumes-from=[] are two very important options for data backup and live migration.

+

-w, --workdir=""

+

Specifies the working directory of the container.

+
+ +Example: + +Run the following command to create a container named **busybox** and run the **docker start** command to start the container. + +```shell +sudo docker create -ti --name=busybox busybox /bin/bash +``` + +## diff + +Syntax: **docker diff** _container_ + +Function: Checks the differences between containers and determines the changes have been made compared with the container creation. + +Parameter description: none. + +Example: + +```shell +$ sudo docker diff registry +C /root +A /root/.bash_history +A /test +``` + +## exec + +Syntax: **docker exec \[**_options_**\]** _container_ _command_ **\[**_arg..._**\]** + +Function: Runs a command in the container. + +Parameter description: + +**-d** and **--detach=false**: Run in the background. + +**-i** and **--interactive=false**: Keep the STDIN of the container enabled. + +**-t** and **--tty=false**: Allocate a virtual terminal. + +**--privileged**: Executes commands in privilege mode. + +**-u** and **--user**: Specifies the user name or UID. + +Example: + +```shell +$ sudo docker exec -ti exec_test ls +bin etc lib media opt root sbin sys tmp var +dev home lib64 mnt proc run srv test usr +``` + +## export + +Syntax: **docker export** _container_ + +Function: Exports the file system content of a container to STDOUT in .tar format. + +Parameter description: none. + +Example: + +Run the following commands to export the contents of the container named **busybox** to the **busybox.tar** package: + +```shell +$ sudo docker export busybox > busybox.tar +$ ls +busybox.tar +``` + +## inspect + +Syntax: **docker inspect \[**_options_**\] **_container_**|**_image_**\[**_container_|_image..._**\]** + +Function: Returns the underlying information about a container or image. + +Parameter description: + +**-f** and **--format=""**: Output information in a specified format. + +**-s** and **--size**: Display the total file size of the container when the query type is container. + +**--type**: Returns the JSON format of the specified type. + +**-t** and **--time=120**: Timeout interval, in seconds. If the **docker inspect** command fails to be executed within the timeout interval, the system stops waiting and immediately reports an error. The default value is **120**. + +Example: + +1. Run the following command to return information about a container: + + ```shell + $ sudo docker inspect busybox_test + [ + { + "Id": "9fbb8649d5a8b6ae106bb0ac7686c40b3cbd67ec2fd1ab03e0c419a70d755577", + "Created": "2019-08-28T07:43:51.27745746Z", + "Path": "bash", + "Args": [], + "State": { + "Status": "running", + "Running": true, + "Paused": false, + "Restarting": false, + "OOMKilled": false, + "Dead": false, + "Pid": 64177, + "ExitCode": 0, + "Error": "", + "StartedAt": "2019-08-28T07:43:53.021226383Z", + "FinishedAt": "0001-01-01T00:00:00Z" + }, + ...... + ``` + +2. Run the following command to return the specified information of a container in a specified format. The following uses the IP address of the busybox\_test container as an example. + + ```shell + $ sudo docker inspect -f {{.NetworkSettings.IPAddress}} busybox_test + 172.17.0.91 + ``` + +## logs + +Syntax: **docker logs \[**_options_**\]** _container_ + +Function: Captures logs in a container that is in the **running** or **stopped** state. + +Parameter description: + +**-f** and **--follow=false**: Print logs in real time. + +**-t** and **--timestamps=false**: Display the log timestamp. + +**--since**: Displays logs generated after the specified time. + +**--tail="all"**: Sets the number of lines to be displayed. By default, all lines are displayed. + +Example: + +1. Run the following command to check the logs of the jaegertracing container where a jaegertracing service runs: + + ```shell + $ sudo docker logs jaegertracing + {"level":"info","ts":1566979103.3696961,"caller":"healthcheck/handler.go:99","msg":"Health Check server started","http-port":14269,"status":"unavailable"} + {"level":"info","ts":1566979103.3820567,"caller":"memory/factory.go:55","msg":"Memory storage configuration","configuration":{"MaxTraces":0}} + {"level":"info","ts":1566979103.390773,"caller":"tchannel/builder.go:94","msg":"Enabling service discovery","service":"jaeger-collector"} + {"level":"info","ts":1566979103.3908608,"caller":"peerlistmgr/peer_list_mgr.go:111","msg":"Registering active peer","peer":"127.0.0.1:14267"} + {"level":"info","ts":1566979103.3922884,"caller":"all-in-one/main.go:186","msg":"Starting agent"} + {"level":"info","ts":1566979103.4047635,"caller":"all-in-one/main.go:226","msg":"Starting jaeger-collector TChannel server","port":14267} + {"level":"info","ts":1566979103.404901,"caller":"all-in-one/main.go:236","msg":"Starting jaeger-collector HTTP server","http-port":14268} + {"level":"info","ts":1566979103.4577134,"caller":"all-in-one/main.go:256","msg":"Listening for Zipkin HTTP traffic","zipkin.http-port":9411} + ``` + +2. Add **-f** to the command to output the logs of the jaegertracing container in real time. + + ```shell + $ sudo docker logs -f jaegertracing + {"level":"info","ts":1566979103.3696961,"caller":"healthcheck/handler.go:99","msg":"Health Check server started","http-port":14269,"status":"unavailable"} + {"level":"info","ts":1566979103.3820567,"caller":"memory/factory.go:55","msg":"Memory storage configuration","configuration":{"MaxTraces":0}} + {"level":"info","ts":1566979103.390773,"caller":"tchannel/builder.go:94","msg":"Enabling service discovery","service":"jaeger-collector"} + {"level":"info","ts":1566979103.3908608,"caller":"peerlistmgr/peer_list_mgr.go:111","msg":"Registering active peer","peer":"127.0.0.1:14267"} + {"level":"info","ts":1566979103.3922884,"caller":"all-in-one/main.go:186","msg":"Starting agent"} + ``` + +## pause/unpause + +Syntax: **docker pause** _container_ + +**docker unpause** _container_ + +Function: The two commands are used in pairs. The **docker pause** command suspends all processes in a container, and the **docker unpause** command resumes the suspended processes. + +Parameter description: none. + +Example: + +The following uses a container where the docker registry service runs as an example. After the **docker pause** command is executed to pause the process of the container, access of the registry service by running the **curl** command is blocked. You can run the **docker unpause** command to resume the suspended registry service. The registry service can be accessed by running the **curl** command. + +1. Run the following command to start a registry container: + + ```shell + sudo docker run -d --name pause_test -p 5000:5000 registry + ``` + + Run the **curl** command to access the service. Check whether the status code **200 OK** is returned. + + ```shell + sudo curl -v 127.0.0.1:5000 + ``` + +2. Run the following command to stop the processes in the container: + + ```shell + sudo docker pause pause_test + ``` + + Run the **curl** command to access the service to check whether it is blocked and wait until the service starts. + +3. Run the following command to resume the processes in the container: + + ```shell + sudo docker unpause pause_test + ``` + + The cURL access in step 2 is restored and the request status code **200 OK** is returned. + +## port + +Syntax: **docker port **_container_ **\[**_private\_port\[/proto\]_**\]** + +Function: Lists the port mapping of a container or queries the host port where a specified port resides. + +Parameter description: none. + +Example: + +1. Run the following command to list all port mappings of a container: + + ```shell + $ sudo docker port registry + 5000/tcp -> 0.0.0.0.:5000 + ``` + +2. Run the following command to query the mapping of a specified container port: + + ```shell + $ sudo docker port registry 5000 + 0.0.0.0.:5000 + ``` + +## ps + +Syntax:**docker ps \[**_options_**\]** + +Function: Lists containers in different states based on different parameters. If no parameter is added, all running containers are listed. + +Parameter description: + +**-a** and **--all=false**: Display the container. + +**-f** and **--filter=\[\]**: Filter values. The available options are: **exited=**_int_ \(exit code of the container\) **status=**_restarting|running|paused|exited_ \(status code of the container\), for example, **-f status=running**: lists the running containers. + +**-l** and **--latest=false**: List the latest created container. + +**-n=-1**: Lists the latest created _n_ containers. + +**--no-trunc=false**: Displays all 64-bit container IDs. By default, 12-bit container IDs are displayed. + +**-q** and **--quiet=false**: Display the container ID. + +**-s** and **--size=false**: Display the container size. + +Example: + +1. Run the following command to lists running containers: + + ```shell + sudo docker ps + ``` + +2. Run the following command to display all containers: + + ```shell + sudo docker ps -a + ``` + +## rename + +Syntax: **docker rename OLD\_NAME NEW\_NAME** + +Function: Renames a container. + +Example: + +Run the **docker run** command to create and start a container, run the **docker rename** command to rename the container, and check whether the container name is changed. + +```shell +$ sudo docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +b15976967abb busybox:latest "bash" 3 seconds ago Up 2 seconds festive_morse +$ sudo docker rename pedantic_euler new_name +$ sudo docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +b15976967abb busybox:latest "bash" 34 seconds ago Up 33 seconds new_name +``` + +## restart + +Syntax: **docker restart \[**_options_**\]** _container_ **\[**_container..._**\]** + +Function: Restarts a running container. + +Parameter description: + +**-t** and **--time=10**: Number of seconds to wait for the container to stop before the container is killed. If the container has stopped, restart the container. The default value is **10**. + +Example: + +```shell +sudo docker restart busybox +``` + +>[!NOTE]NOTE +>During the container restart, if a process in the **D** or **Z** state exists in the container, the container may fail to be restarted. In this case, you need to analyze the cause of the **D** or **Z** state of the process in the container. Restart the container after the **D** or **Z** state of the process in the container is released. + +## rm + +Syntax: **docker rm \[**_options_**\] **_container_ **\[**_container..._**\]** + +Function: Deletes one or more containers. + +Parameter description: + +**-f** and **--force=false**: Forcibly delete a running container. + +**-l** and **--link=false**: Remove the specified link and do not remove the underlying container. + +**-v** and **--volumes=false**: Remove the volumes associated with the container. + +Example: + +1. Run the following command to delete a stopped container: + + ```shell + sudo docker rm test + ``` + +2. Run the following command to delete a running container: + + ```shell + sudo docker rm -f rm_test + ``` + +## run + +Syntax: **docker run \[**_options_**\] **_image_ **\[**_command_**\] \[**_arg_**...\]** + +Function: Creates a container from a specified image \(if the specified image does not exist, an image is downloaded from the official image registry\), starts the container, and runs the specified command in the container. This command integrates the **docker create**, **docker start**, and **docker exec** commands. + +Parameter description: \(The parameters of this command are the same as those of the **docker create** command. For details, see the parameter description of the **docker create** command. Only the following two parameters are different.\) + +**--rm=false**: Specifies the container to be automatically deleted when it exits. + +**-v**: Mounts a local directory or an anonymous volume to the container. Note: When a local directory is mounted to a container with a SELinux security label, do not add or delete the local directory at the same time. Otherwise, the security label may not take effect. + +**--sig-proxy=true**: Receives proxy of the process signal. SIGCHLD, SIGSTOP, and SIGKILL do not use the proxy. + +Example: + +Run the busybox image to start a container and run the **/bin/sh** command after the container is started: + +```shell +sudo docker run -ti busybox /bin/sh +``` + +## start + +Syntax: **docker start \[**_options_**\]** _container_ **\[**_container_**...\]** + +Function: Starts one or more containers that are not running. + +Parameter description: + +**-a** and **--attach=false**: Attach the standard output and error output of a container to STDOUT and STDERR of the host. + +**-i** and **--interactive=false**: Attach the standard input of the container to the STDIN of the host. + +Example: + +Run the following command to start a container named **busybox** and add the **-i -a** to the command to add standard input and output. After the container is started, directly enter the container. You can exist the container by entering **exit**. + +If **-i -a** is not added to the command when the container is started, the container is started in the background. + +```shell +sudo docker start -i -a busybox +``` + +## stats + +Syntax: **docker stats \[**_options_**\] \[**_container_**...\]** + +Function: Continuously monitors and displays the resource usage of a specified container. \(If no container is specified, the resource usage of all containers is displayed by default.\) + +Parameter description: + +**-a**, and **--all**: Display information about all containers. By default, only running containers are displayed. + +**--no-stream**: Displays only the first result and does not continuously monitor the result. + +Example: + +Run the **docker run** command to start and create a container, and run the **docker stats** command to display the resource usage of the container: + +```shell +$ sudo docker stats +CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS +2e242bcdd682 jaeger 0.00% 77.08MiB / 125.8GiB 0.06% 42B / 1.23kB 97.9MB / 0B 38 +02a06be42b2c relaxed_chandrasekhar 0.01% 8.609MiB / 125.8GiB 0.01% 0B / 0B 0B / 0B 10 +deb9e49fdef1 hardcore_montalcini 0.01% 12.79MiB / 125.8GiB 0.01% 0B / 0B 0B / 0B 9 +``` + +## stop + +Syntax: **docker stop \[**_options_**\]** _container_ **\[**_container_**...\]** + +Function: Sends a SIGTERM signal to a container and then sends a SIGKILL signal to stop the container after a certain period. + +Parameter description: + +**-t** and **--time=10**: Number of seconds that the system waits for the container to exit before the container is killed. The default value is **10**. + +Example: + +```shell +sudo docker stop -t=15 busybox +``` + +## top + +Syntax: **docker top** _container_ **\[**_ps options_**\]** + +Function: Displays the processes running in a container. + +Parameter description: none. + +Example: + +Run the top\_test container and run the **top** command in the container. + +```shell +$ sudo docker top top_test +UID PID PPID C STIME TTY TIME CMD +root 70045 70028 0 15:52 pts/0 00:00:00 bash +``` + +The value of **PID** is the PID of the process in the container on the host. + +## update + +Syntax: **docker update \[**_options_**\]** _container_ **\[**_container_**...\]** + +Function: Hot changes one or more container configurations. + +Parameter description: + +**Table 1** Parameter description + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Parameter

+

Description

+

--accel=[]

+

Configures one or more container accelerators.

+

--blkio-weight

+

Relative weight of the container blockio. The value ranges from 10 to 1000.

+

--cpu-shares

+

Relative weight of the host CPU obtained by the container. This parameter can be used to obtain a higher priority. By default, all containers obtain the same CPU priority.

+

--cpu-period

+

CPU CFS period.

+

The default value is 100 ms. Generally, --cpu-period and --cpu-quota are used together. For example, --cpu-period=50000 --cpu-quota=25000 indicates that if there is one CPU, the container can obtain 50% of the CPU every 50 ms.

+

--cpu-quota

+

CPU CFS quota. The default value is 0, indicating that there is no restriction on the quota.

+

--cpuset-cpus

+

Number of CPUs (0-3, 0, 1) that can be used by processes in the container. By default, there is no restriction on this parameter.

+

--cpuset-mems

+

Memory nodes (0-3, 0, 1) for running processes in the container. This parameter is valid only for the NUMA system.

+

--kernel-memory=""

+

Kernel memory limit of a container. The format is numberoptional unit, and available units are b, k, m, and g.

+

-m, --memory=""

+

Memory limit of a container. The format is numberoptional unit, and available units are b, k, m, and g. The minimum value of this parameter is 4m.

+

--memory-reservation

+

Container memory limit. The default value is the same as that of --memory. --memory is a hard limit, and --memory-reservation is a soft limit. When the memory usage exceeds the preset value, the memory usage is dynamically adjusted (the system attempts to reduce the memory usage to a value less than the preset value when reclaiming the memory). However, the memory usage may exceed the preset value. Generally, this parameter can be used together with --memory. The value must be less than the preset value of --memory.

+

--memory-swap

+

Total usage of the common memory and swap partition. -1 indicates no restriction is set on the usage. If this parameter is not set, the swap partition size is twice the value of --memory. That is, the swap partition can use the same amount of memory as --memory.

+

--restart=""

+

Configures restart rule when the container exits. Currently, version 1.3.1 supports the following rules:

+
  • no: indicates that the container is not restarted when it is stopped.
  • on-failure: indicates that the container is restarted when the container exit code is not 0. This rule can be used to add the maximum number of restart times, for example, on-failure:5, indicating that the container can be restarted for a maximum of five times.
  • always: indicates the container is exited regardless of the exit code.
+

--help

+

Help information.

+
+ +Example: + +Run the following command to change the CPU and memory configurations of the container named **busybox**, including changing the relative weight of the host CPU obtained by the container to **512**, the CPU cores that can be run by processes in the container to **0,1,2,3**, and the memory limit for running the container to **512 m**. + +```shell +sudo docker update --cpu-shares 512 --cpuset-cpus=0,3 --memory 512m ubuntu +``` + +## wait + +Syntax: **docker wait** _container_ **\[**_container..._**\]** + +Function: Waits for a container to stop and print the exit code of the container: + +Parameter description: none. + +Example: + +Run the following command to start a container named **busybox**: + +```shell +sudo docker start -i -a busybox +``` + +Run the **docker wait** command: + +```shell +$ sudo docker wait busybox +0 +``` + +Wait until the busybox container exits. After the busybox container exits, the exit code **0** is displayed. diff --git a/docs/en/cloud/container_engine/docker_engine/docker-faqs.md b/docs/en/cloud/container_engine/docker_engine/docker-faqs.md new file mode 100644 index 0000000000000000000000000000000000000000..9ed94a0c471e704c5f9bf13c64c581e462375f53 --- /dev/null +++ b/docs/en/cloud/container_engine/docker_engine/docker-faqs.md @@ -0,0 +1,5 @@ +# Common Docker Issues and Solutions + +## Issue 1: Additional Mount Point in Docker v18.09.9 Compared to v19.03.0 and Later + +In Docker version 18.09.9, containers have an extra mount point compared to those launched in Docker v19.03.0 and later. This is because the default `ipcmode` in v18.09 is set to `shareable`, which creates an additional `shmpath` mount point. To resolve this, either update the `ipcmode` option to `private` in the Docker configuration file or upgrade to a newer Docker version. diff --git a/docs/en/cloud/container_engine/docker_engine/image-management-1.md b/docs/en/cloud/container_engine/docker_engine/image-management-1.md new file mode 100644 index 0000000000000000000000000000000000000000..cd1dc9e4dbe607907a5aedfeff0abc958fb30aa6 --- /dev/null +++ b/docs/en/cloud/container_engine/docker_engine/image-management-1.md @@ -0,0 +1,56 @@ +# Image Management + +- [Image Management](#image-management) + - [Creating an Image](#creating-an-image) + - [Viewing Images](#viewing-images) + - [Deleting Images](#deleting-images) + +## Creating an Image + +You can use the **docker pull**, **docker build**,**docker commit**, **docker import**, or **docker load** command to create an image. For details about how to use these commands, see [4.6.3 Image Management](image-management-2.md). + +### Precautions + +1. Do not concurrently run the **docker load** and **docker rmi** commands. If both of the following conditions are met, concurrency problems may occur: + + - An image exists in the system. + - The docker rmi and docker load operations are concurrently performed on an image. + + Therefore, avoid this scenario. \(All concurrent operations between the image creation operations such as running the **tag**, **build**, and **load**, and **rmi** commands, may cause similar errors. Therefore, do not concurrently perform these operations with **rmi**.\) + +2. If the system is powered off when docker operates an image, the image may be damaged. In this case, you need to manually restore the image. + + When the docker operates images \(using the **pull**, **load**, **rmi**, **build**, **combine**, **commit**, or **import** commands\), image data operations are asynchronous, and image metadata is synchronous. Therefore, if the system power is off when not all image data is updated to the disk, the image data may be inconsistent with the metadata. Users can view images \(possibly none images\), but cannot start containers, or the started containers are abnormal. In this case, run the **docker rmi** command to delete the image and perform the previous operations again. The system can be recovered. + +3. Do not store a large number of images on nodes in the production environment. Delete unnecessary images in time. + + If the number of images is too large, the execution of commands such as **docker image** is slow. As a result, the execution of commands such as **docker build** or **docker commit** fails, and the memory may be stacked. In the production environment, delete unnecessary images and intermediate process images in time. + +4. When the **--no-parent** parameter is used to build images, if multiple build operations are performed at the same time and the FROM images in the Dockerfile are the same, residual images may exist. There are two cases: + - If FROM images are incomplete, the images generated when images of FROM are running may remain. Names of the residual images are similar to **base\_v1.0.0-app\_v2.0.0**, or they are none images. + - If the first several instructions in the Dockerfile are the same, none images may remain. + +### None Image May Be Generated + +1. A none image is the top-level image without a tag. For example, the image ID of **ubuntu** has only one tag **ubuntu**. If the tag is not used but the image ID is still available, the image ID becomes a none image. +2. An image is protected because the image data needs to be exported during image saving. However, if a deletion operation is performed, the image may be successfully untagged and the image ID may fail to be deleted \(because the image is protected\). As a result, the image becomes a none image. +3. If the system is powered off when you run the **docker pull** command or the system is in panic, a none image may be generated. To ensure image integrity, you can run the **docker rmi** command to delete the image and then restart it. +4. If you run the **docker save** command to save an image and specify the image ID as the image name, the loaded image does not have a tag and the image name is **none**. + +### A Low Probability That Image Fails to Be Built If the Image Is Deleted When Being Built + +Currently, the image build process is protected by reference counting. After an image is built, reference counting of the image is increased by 1 \(holdon operation\). Once the holdon operation is successful, the image will not be deleted. However, there is a low probability that before the holdon operation is performed, the image can still be deleted, causing the image build failure. + +## Viewing Images + +Run the following command to view the local image list: + +```shell +docker images +``` + +## Deleting Images + +### Precautions + +Do not run the **docker rmi -f**_XXX_ command to delete images. If you forcibly delete an image, the **docker rmi** command ignores errors during the process, which may cause residual metadata of containers or images. If you delete an image in common mode and an error occurs during the deletion process, the deletion fails and no metadata remains. diff --git a/docs/en/cloud/container_engine/docker_engine/image-management-2.md b/docs/en/cloud/container_engine/docker_engine/image-management-2.md new file mode 100644 index 0000000000000000000000000000000000000000..1ea675321ad9fd4d009ce2df72e4b24fa22a3714 --- /dev/null +++ b/docs/en/cloud/container_engine/docker_engine/image-management-2.md @@ -0,0 +1,465 @@ +# Image Management + +- [Image Management](#image-management) + - [build](#build) + - [history](#history) + - [images](#images) + - [import](#import) + - [load](#load) + - [login](#login) + - [logout](#logout) + - [pull](#pull) + - [push](#push) + - [rmi](#rmi) + - [save](#save) + - [search](#search) + - [tag](#tag) + +## build + +Syntax: **docker build \[**_options_**\]** _path_ **|** _URL_ **| -** + +Function: Builds an image using the Dockerfile in the specified path. + +Parameter description: Common parameters are as follows. For details about more parameters, see the **docker help build** command section. + +**Table 1** Parameter description + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Parameter

+

Description

+

--force-rm=false

+

Deletes containers generated during the build process even if the build is not successful.

+

--no-cache=false

+

Builds cache without using cache.

+

-q, --quiet=false

+

Prevents the redundant information generation during the build.

+

--rm=true

+

Deletes the container generated during the build after the build is successful.

+

-t, --tag=""

+

Tag name of the image generated during the build.

+

--build-arg=[]

+

Configures the build parameters.

+

--label=[]

+

Image-related parameters. The description of each parameter is similar to that of the create command.

+

--isolation

+

Container isolation method.

+

--pull

+

Obtains the latest image during the build.

+
+ +**Dockerfile Command** + +Dockerfile is used to describe how to build an image and automatically build a container. The format of all **Dockerfile** commands is _instruction_ _arguments_. + +**FROM Command** + +Syntax: **FROM** _image_ or **FROM** _image_:_tag_ + +Function: Specifies a basic image, which is the first command for all Dockerfile files. If the tag of a basic image is not specified, the default tag name **latest** is used. + +**RUN Command** + +Syntax: **RUN** _command_ \(for example, **run in a shell - \`/bin/sh -c\`**\) or + +**RUN \[**_executable_, _param1_, _param2_ ... **\]** \(in the **exec** command format\) + +Function: Runs any command in the image specified by the **FROM** command and then commits the result. The committed image can be used in later commands. The **RUN** command is equivalent to: + +**docker run** _image_ _command_ + +**docker commit** _container\_id_ + +**Remarks** + +The number sign \(\#\) is used to comment out. + +**MAINTAINER Command** + +Syntax: **MAINTAINER**_name_ + +Function: Specifies the name and contact information of the maintenance personnel. + +**ENTRYPOINT Command** + +Syntax: **ENTRYPOINT cmd **_param1 param2..._ or **ENTRYPOINT \[**_"cmd", "param1", "param2"..._**\]** + +Function: Configures the command to be executed during container startup. + +**USER Command** + +Syntax: **USER**_name_ + +Function: Specifies the running user of memcached. + +**EXPOSE Command** + +Syntax: **EXPOSE **_port_** \[**_port_**...\]** + +Function: Enables one or more ports for images. + +**ENV Command** + +Syntax: **ENV**_key value_ + +Function: Configures environment variables. After the environment variables are configured, the **RUN** commands can be subsequently used. + +**ADD Command** + +Syntax: **ADD**_src dst_ + +Function: Copies a file from the _src_ directory to the _dest_ directory of a container. _src_ indicates the relative path of the source directory to be built. It can be the path of a file or directory, or a remote file URL. _dest_ indicates the absolute path of the container. + +**VOLUME Command** + +Syntax: **VOLUME \["**_mountpoint_**"\]** + +Function: Creates a mount point for sharing a directory. + +**WORKDIR Command** + +Syntax: **workdir**_path_ + +Function: Runs the **RUN**, **CMD**, and **ENTRYPOINT** commands to set the current working path. The current working path can be set multiple times. If the current working path is a relative path, it is relative to the previous **WORKDIR** command. + +**CMD command** + +Syntax: **CMD \[**_"executable","param1","param2"_**\]** \(This command is similar to the **exec** command and is preferred.\) + +**CMD \["**_param1_**","**_param2_**"\]** \(The parameters are the default parameters for ENTRYPOINT.\) + +**CMD** _command_ _param1_ _param2_ \(This command is similar to the **shell** command.\) + +Function: A Dockerfile can contain only one CMD command. If there are multiple CMD commands, only the last one takes effect. + +**ONBUILD Commands** + +Syntax: **ONBUILD \[**_other commands_**\]** + +Function: This command is followed by other commands, such as the **RUN** and **COPY** commands. This command is not executed during image build and is executed only when the current image is used as the basic image to build the next-level image. + +The following is a complete example of the Dockerfile command that builds an image with the sshd service installed. + +```text +FROM busybox +ENV http_proxy http://192.168.0.226:3128 +ENV https_proxy https://192.168.0.226:3128 +RUN apt-get update && apt-get install -y openssh-server +RUN mkdir -p /var/run/sshd +EXPOSE 22 +ENTRYPOINT /usr/sbin/sshd -D +``` + +Example: + +1. Run the following command to build an image using the preceding Dockerfile: + + ```shell + sudo docker build -t busybox:latest + ``` + +2. Run the following command to view the generated image: + + ```shell + docker images | grep busybox + ``` + +## history + +Syntax: **docker history \[**_options_**\]** _image_ + +Function: Displays the change history of an image. + +Parameter description: + +-H, --human=true + +**--no-trunc=false**: Does not delete any output. + +**-q** and **--quiet=false**: Display only IDs. + +Example: + +```shell +$ sudo docker history busybox:test +IMAGE CREATED CREATED BY SIZE COMMENT +be4672959e8b 15 minutes ago bash 23B +21970dfada48 4 weeks ago 128MB Imported from - +``` + +## images + +Syntax: **docker images \[**_options_**\] \[**_name_**\]** + +Function: Lists existing images. The intermediate image is not displayed if no parameter is configured. + +Parameter description: + +**-a** and **--all=false**: Display all images. + +**-f** and **--filter=\[\]**: Specify a filtering value, for example, **dangling=true**. + +**--no-trunc=false**: Does not delete any output. + +**-q** and **--quiet=false**: Display only IDs. + +Example: + +```shell +$ sudo docker images +REPOSITORY TAG IMAGE ID CREATED SIZE +busybox latest e02e811dd08f 2 years ago 1.09MB +``` + +## import + +Syntax: **docker import URL|- \[**_repository_**\[**_:tag_**\]\]** + +Function: Imports a .tar package that contains rootfs as an image. This parameter corresponds to the **docker export** command. + +Parameter description: none. + +Example: + +Run the following command to generate a new image for **busybox.tar** exported using the **docker export** command: + +```shell +$ sudo docker import busybox.tar busybox:test +sha256:a79d8ae1240388fd3f6c49697733c8bac4d87283920defc51fb0fe4469e30a4f +$ sudo docker images +REPOSITORY TAG IMAGE ID CREATED SIZE +busybox test a79d8ae12403 2 seconds ago 1.3MB +``` + +## load + +Syntax: **docker load \[**_options_**\]** + +Function: Reloads an image from .tar package obtained by running the **docker save** command. This parameter corresponds to the **docker save** command. + +Parameter description: + +**-i** and **--input=""** can be used. + +Example: + +```shell +$ sudo docker load -i busybox.tar +Loaded image ID: sha256:e02e811dd08fd49e7f6032625495118e63f597eb150403d02e3238af1df240ba +$ sudo docker images +REPOSITORY TAG IMAGE ID CREATED SIZE +busybox latest e02e811dd08f 2 years ago 1.09MB +``` + +## login + +Syntax: **docker login \[**_options_**\] \[**_server_**\]** + +Function: Logs in to an image server. If no server is specified, the system logs in to **** by default. + +Parameter description: + +**-e** and **--email=""**: Email address. + +**-p** and **--password=""**: Password. + +**-u** and **--username=""**: User name. + +Example: + +```shell +sudo docker login +``` + +## logout + +Syntax: **docker logout \[**_server_**\]** + +Function: Logs out of an image server. If no server is specified, the system logs out of **** by default. + +Parameter description: none. + +Example: + +```shell +sudo docker logout +``` + +## pull + +Syntax: **docker pull \[**_options_**\]** _name_**\[**_:tag_**\]** + +Function: Pulls an image from an official or private registry. + +Parameter description: + +**-a** and **--all-tags=false**: Download all images in a registry. \(A registry can be tagged with multiple tags. For example, a busybox registry may have multiple tags, such as **busybox:14.04**, **busybox:13.10**, **busybox:latest**. If **-a** is used, all busybox images with tags are pulled.\) + +Example: + +1. Run the following command to obtain the Nginx image from the official registry: + + ```shell + $ sudo docker pull nginx + Using default tag: latest + latest: Pulling from official/nginx + 94ed0c431eb5: Pull complete + 9406c100a1c3: Pull complete + aa74daafd50c: Pull complete + Digest: sha256:788fa27763db6d69ad3444e8ba72f947df9e7e163bad7c1f5614f8fd27a311c3 + Status: Downloaded newer image for nginx:latest + ``` + + When an image is pulled, the system checks whether the dependent layer exists. If yes, the local layer is used. + +2. Pull an image from a private registry. + + Run the following command to pull the Fedora image from the private registry, for example, the address of the private registry is **192.168.1.110:5000**: + + ```shell + sudo docker pull 192.168.1.110:5000/fedora + ``` + +## push + +Syntax: **docker push** _name_**\[**_:tag_**\]** + +Function: Pushes an image to the image registry. + +Parameter description: none. + +Example: + +1. Run the following command to push an image to the private image registry at 192.168.1.110:5000. +2. Label the image to be pushed. \(The **docker tag** command is described in the following section.\) In this example, the image to be pushed is busybox:sshd. + + ```shell + sudo docker tag ubuntu:sshd 192.168.1.110:5000/busybox:sshd + ``` + +3. Run the following command to push the tagged image to the private image registry: + + ```shell + sudo docker push 192.168.1.110:5000/busybox:sshd + ``` + + During the push, the system automatically checks whether the dependent layer exists in the image registry. If yes, the layer is skipped. + +## rmi + +Syntax: **docker rmi \[**_options_**\] **_image _**\[**_image..._**\]** + +Function: Deletes one or more images. If an image has multiple tags in the image library, only the untag operation is performed when the image is deleted. If the image has only one tag, the dependent layers are deleted in sequence. + +Parameter description: + +**-f** and **--force=false**: Forcibly delete an image. + +**--no-prune=false**: Does not delete parent images without tags. + +Example: + +```shell +sudo docker rmi 192.168.1.110:5000/busybox:sshd +``` + +## save + +Syntax: **docker save \[**_options_**\] **_image _**\[**_image..._**\]** + +Function: Saves an image to a TAR package. The output is **STDOUT** by default. + +Parameter description: + +**-o** and **--output=""**: Save an image to a file rather than STDOUT. + +Example: + +```shell +$ sudo docker save -o nginx.tar nginx:latest +$ ls +nginx.tar +``` + +## search + +Syntax: **docker search**_options_ _TERM_ + +Function: Searches for a specific image in the image registry. + +Parameter description: + +**--automated=false**: Displays the automatically built image. + +**--no-trunc=false**: Does not delete any output. + +**-s** and **--stars=0**: Display only images of a specified star level or higher. + +Example: + +1. Run the following command to search for Nginx in the official image library: + + ```shell + $ sudo docker search nginx + NAME DESCRIPTION STARS OFFICIAL AUTOMATED + nginx Official build of Nginx. 11873 [OK] + jwilder/nginx-proxy Automated Nginx reverse proxy for docker con... 1645 [OK] + richarvey/nginx-php-fpm Container running Nginx + PHP-FPM capable of... 739 [OK] + linuxserver/nginx An Nginx container, brought to you by LinuxS... 74 + bitnami/nginx Bitnami nginx Docker Image 70 [OK] + tiangolo/nginx-rtmp Docker image with Nginx using the nginx-rtmp... 51 [OK] + ``` + +2. Run the following command to search for busybox in the private image library. The address of the private image library must be added during the search. + + ```shell + sudo docker search 192.168.1.110:5000/busybox + ``` + +## tag + +Syntax: **docker tag \[**_options_**\] **_image_**\[**_:tag_**\] \[**_registry host/_**\]\[**_username/_**\]**_name_**\[**_:tag_**\]** + +Function: Tags an image to a registry. + +Parameter description: + +**-f** or **--force=false**: Forcibly replaces the original image when the same tag name exists. + +Example: + +```shell +sudo docker tag busybox:latest busybox:test +``` diff --git a/docs/en/cloud/container_engine/docker_engine/installation-and-configuration-3.md b/docs/en/cloud/container_engine/docker_engine/installation-and-configuration-3.md new file mode 100644 index 0000000000000000000000000000000000000000..780a9f3616da3a5f4fdf32cf1f9a42f1bcc25174 --- /dev/null +++ b/docs/en/cloud/container_engine/docker_engine/installation-and-configuration-3.md @@ -0,0 +1,404 @@ +# Installation and Configuration + +This chapter describes important configurations related to the installation of the open source container Docker. + +## Precautions + +- The root permission is required for installing a Docker container. +- The **docker-engine** RPM package cannot be installed together with the **containerd**, **runc**, or **podman** RPM package. This is because the **docker-engine** RPM package contains all components required for Docker running, including **containerd**, **runc**, and **docker** binary files. Yet the **containerd**, **runc**, and **podman** RPM packages also contain the corresponding binary files. Software package conflicts may occur due to repeated installation. + +## Basic Installation Configuration + +### Daemon Parameter Configuration + +You can add configuration items to the **/etc/docker/daemon.json** file to customize parameters. You can run the **dockerd --help** command to view related configuration items and their usage methods. A configuration example is as follows: + +```shell +$ cat /etc/docker/daemon.json +{ + "debug": true, + "storage-driver": "overlay2", + "storage-opts": ["overlay2.override_kernel_check=true"] +} +``` + +### Daemon Running Directory Configuration + +Re-configuring various running directories and files \(including **--graph** and **--exec-root**\) may cause directory conflicts or file attribute changes, affecting the normal use of applications. + +>[!TIP]NOTICE +>Therefore, the specified directories or files should be used only by Docker to avoid file attribute changes and security issues caused by conflicts. + +- Take **--graph** as an example. When **/new/path/** is used as the new root directory of the daemon, if a file exists in **/new/path/** and the directory or file name conflicts with that required by Docker \(for example, **containers**, **hooks**, and **tmp**\), Docker may update the original directory or file attributes, including the owner and permission. + +>[!TIP]NOTICE +>From Docker 17.05, the **--graph** parameter is marked as **Deprecated** and replaced with the **--data-root** parameter. + +### Daemon Network Configuration + +- After the network segment of the docker0 bridge is specified by using the **--bip** parameter on Docker daemon, if the **--bip** parameter is deleted during the next Docker daemon restart, the docker0 bridge uses the previous value of **--bip**, even if the docker0 bridge is deleted before the restart. The reason is that Docker saves the network configuration and restores the previous configuration by default during the next restart. +- When running the **docker network create** command to concurrently create networks, you can create two networks with the same name. The reason is that Docker networks are distinguished by IDs. The name is only an alias that is easy to identify and may not be unique. +- In the Docker bridge network mode, a Docker container establishes external communication through NAT on the host. When Docker daemon starts a Docker container, a docker-proxy process is started for each port mapped on the host to access the proxy. It is recommended that you map only the necessary ports when using userland-proxy to reduce the resources consumed by the port mapping of docker-proxy. + +### Daemon umask Configuration + +The default **umask** value of the main container process and exec process is **0022**. To meet security specifications and prevent containers from being attacked, the default value of **umask** is changed to **0027** after runC implementation is modified. After the modification, the other groups cannot access new files or directories. + +The default value of **umask** is **0027** when Docker starts a container. You can change the value to **0022** by running the **--exec-opt native.umask=normal** command during container startup. + +>[!TIP]NOTICE +>If **native.umask** is configured in **docker create** or **docker run** command, its value is used. + +For details, see the parameter description in [docker create](./container-management-2.md#create) and [docker run](./container-management-2.md#run). + +### Daemon Start Time + +The Docker service is managed by systemd, which restricts the startup time of each service. If the Docker service fails to be started within the specified time, the possible causes are as follows: + +- If Docker daemon is started for the first time using devicemapper, the Docker daemon needs to perform the initialization operation on the device. This operation, however, will perform a large number of disk I/O operations. When the disk performance is poor or many I/O conflicts exist, the Docker daemon startup may time out. devicemapper needs to be initialized only once and does not need to be initialized again during later Docker daemon startup. +- If the usage of the current system resources is too high, the system responses slowly, all operations in the system slow down, and the startup of the Docker service may time out. +- During the restart, a daemon traverses and reads configuration files and the init layer and writable layer configurations of each container in the Docker working directory. If there are too many containers \(including the created and exited containers\) in the current system and the disk read and write performance is limited, the startup of the Docker service may time out due to the long-time daemon traversing. + +If the service startup times out, you are advised to rectify the fault as follows: + +- Ensure that the container orchestration layer periodically deletes unnecessary containers, especially the exited containers. +- Based on performance requirements of the solution, adjust the cleanup period of the orchestration layer and the start time of the Docker service. + +### Journald Component + +After systemd-journald is restarted, Docker daemon needs to be restarted. Journald obtains the Docker daemon logs through a pipe. If the journald service is restarted, the pipe is disabled. The write operation of Docker logs triggers the SIGPIPE signal, which causes the Docker daemon crash. If this signal is ignored, the subsequent Docker daemon logs may fail to be recorded. Therefore, you are advised to restart Docker daemon after the journald service is restarted or becomes abnormal, ensuring that Docker logs can be properly recorded and preventing status exceptions caused by daemon crash. + +### Firewalld Component + +You need to restart the Docker service after restarting or starting firewalld. + +- When the firewalld service is started, the iptables rules of the current system are cleared. Therefore, if the firewalld service is restarted during Docker daemon startup, the Docker service may fail to insert iptables rules, causing the Docker service startup failure. +- If the firewalld service is restarted after the Docker service is started, or the status of the firewalld service \(service paused or resumed\) is changed, the iptables rules of the Docker service are deleted. As a result, the container with port mapping fails to be created. + +### Iptables Component + +If the **--icc=false** option is added in Docker, the communication between containers can be restricted. However, if the OS has some rules, the communication between containers may not be restricted. For example: + +```text +Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) +... +0 0 ACCEPT icmp -- * * 0.0.0.0/0 0.0.0.0/0 +... +0 0 DROP all -- docker0 docker0 0.0.0.0/0 0.0.0.0/0 +... +``` + +In the **Chain FORWARD** command, the ACCEPT icmp rule is added to DROP. As a result, after the **--icc=false** option is added, containers can be pinged, but the peer end is unreachable if UDP or TCP is used. + +Therefore, if you want to add the **--icc=false** option when using Docker in a container OS, you are advised to clear iptables rules on the host first. + +### Audit Component + +You can configure audit for Docker. However, this configuration is not mandatory. For example: + +```text +-w /var/lib/docker -k docker +-w /etc/docker -k docker +-w /usr/lib/systemd/system/docker.service -k docker +-w /usr/lib/systemd/system/docker.socket -k docker +-w /etc/sysconfig/docker -k docker +-w /usr/bin/docker-containerd -k docker +-w /usr/bin/docker-runc -k docker +-w /etc/docker/daemon.json -k docker +``` + +Configuring audit for Docker brings certain benefits for auditing, while it does not have any substantial effects on attack defense. In addition, the audit configurations cause serious efficiency problems, for example, the system may not respond smoothly. Therefore, exercise caution in the production environment. + +The following uses **-w /var/lib/docker -k docker** as an example to describe how to configure Docker audit. + +```shell +cat /etc/audit/rules.d/audit.rules | grep docker -w /var/lib/docker/ -k docker +auditctl -R /etc/audit/rules.d/audit.rules | grep docker +auditctl -l | grep docker -w /var/lib/docker/ -p rwxa -k docker +``` + +>[!NOTE]NOTE +>**-p \[r|w|x|a\]** and **-w** are used together to monitor the read, write, execution, and attribute changes \(such as timestamp changes\) of the directory. In this case, any file or directory operation in the **/var/lib/docker** directory will be recorded in the **audit.log** file. As a result, too many logs will be recorded in the **audit.log** file, which severely affects the memory or CPU usage of the auditd, and further affects the OS. For example, logs similar to the following will be recorded in the **/var/log/audit/audit.log** file each time the **ls /var/lib/docker/containers** command is executed: + +```text +type=SYSCALL msg=audit(1517656451.457:8097): arch=c000003e syscall=257 success=yes exit=3 a0=ffffffffffffff9c a1=1b955b0 a2=90800 a3=0 items=1 ppid=17821 pid=1925 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts6 ses=4 comm="ls" exe="/usr/bin/ls" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="docker"type=CWD msg=audit(1517656451.457:8097): cwd="/root"type=PATH msg=audit(1517656451.457:8097): item=0 name="/var/lib/docker/containers" inode=1049112 dev=fd:00 mode=040700 ouid=0 ogid=0 rdev=00:00 obj=unconfined_u:object_r:container_var_lib_t:s0 objtype=NORMAL +``` + +### Security Configuration seccomp + +During the container network performance test, it is found that the performance of Docker is lower than that of the native kernel namespace. After seccomp is enabled, system calls \(such as sendto\) are not performed through system\_call\_fastpath. Instead, tracesys is called, which greatly deteriorates the performance. Therefore, you are advised to disable seccomp in container scenarios where services require high performance. For example: + +```shell +docker run -itd --security-opt seccomp=unconfined busybox:latest +``` + +### Do Not Modify Private Directory of Docker Daemon + +Do not modify the root directory used by Docker \(**/var/lib/docker** by default\), the directory during operation \(**/run/docker** by default\), or the files or directories in the two directories. The forbidden operations include deleting files, adding files, creating soft or hard links for the directories or files, or modifying attributes, permissions, or contents of the files. If any modification is required, contact the Euler container team for review. + +### Precautions for Common Users in the Scenario Where a Large Number of Containers Are Deployed + +The maximum number of processes that a common user can create on an OS host can be restricted by creating the **/etc/security/limits.d/20-nproc.conf** file in the system. Similarly, the maximum number of processes that a common user can create in a container is determined by the value in the **/etc/security/limits.d/20-nproc.conf** file in the container image, as shown in the following example: + +```shell +$ cat /etc/security/limits.conf +* soft nproc 4096 +``` + +If an error is reported due to insufficient resources when a large number of containers are deployed by a common user, increase the value **4096** in the **/etc/security/limits.d/20-nproc.conf** file. + +Configure the maximum value based on the maximum capability of the kernel, as shown in the following example: + +```shell +$ sysctl -a | grep pid_max +kernel.pid_max = 32768 +``` + +## Storage Driver Configuration + +This Docker version supports two storage drivers: overlay2 and devicemapper. Since overlay2 has better performance than devicemapper, it is recommended that overlay2 be preferentially used in the production environment. + +### overlay2 Storage Driver Configuration + +#### Configuration Methods + +overlay2 is the default storage driver of Docker. You can also use either of the following methods to explicitly configure the driver: + +- Edit the **/etc/docker/daemon.json** file to explicitly configure the **storage-driver** field. + + ```shell + $ cat /etc/docker/daemon.json + { + "storage-driver": "overlay2" + } + ``` + +- Edit the **/etc/sysconfig/docker-storage** file and explicitly configure the Docker daemon startup parameters. + + ```shell + $ cat /etc/sysconfig/docker-storage + DOCKER_STORAGE_OPTIONS="--storage-driver=overlay2" + ``` + +#### Precautions + +- When you perform lifecycle management operations on some containers, an error may be reported, indicating that the corresponding rootfs or executable file cannot be found. +- If the health check of a container is configured to execute executable files in the container, an error may be reported, which causes the health check failure of the container. + +- When you use overlay2 as the graphdriver and modify an image file in a container for the first time, the modification fails if the file size is greater than the remaining space of the system. Even if a little modification on the file is involved, the whole file must be copied to the upper layer. If the remaining space is insufficient, the modification fails. +- Compared with common file systems, the overlay2 file system has the following behavior differences: + - Kernel version + + overlay2 is compatible only with the native kernel 4.0 or later. You are advised to use the Ext4 file system. + + - Copy-UP performance + + Modifying files at the lower layer triggers file replication to the upper layer. Data block replication and fsync are time-consuming. + + - Rename directories + - The rename system call is allowed only when both the source and the destination paths are at the merged layer. Otherwise, the EXDEV error is reported. + - Kernel 4.10 introduces the redirect directory feature to fix this issue. The corresponding kernel option is **CONFIG\_OVERLAY\_FS\_REDIRECT\_DIR**. + + When overlay2 is used, a file system directory fails to be renamed because the related feature configured in the **/sys/module/overlay/parameters/redirect\_dir** file has been disabled. To use this feature, you need to manually set **/sys/module/overlay/parameters/redirect\_dir** to **Y**. + + - Hard link disconnection + - If there are multiple hard links in the lower-layer directory, writing data to the merged layer will trigger Copy-UP, resulting in hard link disconnection. + - The index feature is introduced in kernel 4.13 to fix this issue. The corresponding kernel option is **CONFIG\_OVERLAY\_FS\_INDEX**. Note that this option is not forward compatible and does not support hot upgrade. + + - Changes of **st\_dev** and **st\_ino** + + After Copy-UP is triggered, you can view only new files at the merged layer, and inodes change. Although **attr** and **xattr** can be replicated, **st\_dev** and **st\_ino** are unique and cannot be replicated. As a result, you can run **stat** and **ls** commands to check inode changes accordingly. + + - fd change + + Before Copy-UP is triggered, you can obtain the descriptor fd1 when opening a file in read-only mode. After Copy-UP is trigger, you can obtain the descriptor fd2 when opening the file with the same name. The two descriptors point to different files. The data written to fd2 is not displayed in fd1. + +#### Abnormal Scenarios + +When a container uses the overlay2 storage driver, mount points may be overwritten. + +#### Abnormal Scenario: Mount Point Being Overwritten + +In the faulty container, there is a mount point in **/var/lib/docker/overlay2**. + +```shell +$ mount -l | grep overlay +overlay on /var/lib/docker/overlay2/844fd3bca8e616572935808061f009d106a8748dfd29a0a4025645457fa21785/merged type overlay (rw,relatime,seclabel,lowerdir=/var/lib/docker/overlay2/l/JL5PZQLNDCIBU3ZOG3LPPDBHIJ:/var/lib/docker/overlay2/l/ELRPYU4JJG4FDPRLZJCZZE4UO6,upperdir=/var/lib/docker/overlay2/844fd3bca8e616572935808061f009d106a8748dfd29a0a4025645457fa21785/diff,workdir=/var/lib/docker/overlay2/844fd3bca8e616572935808061f009d106a8748dfd29a0a4025645457fa21785/work) +/dev/mapper/dm-root on /var/lib/docker/overlay2 type ext4 (rw,relatime,seclabel,data=ordered) +``` + +An error as follows may occur when some Docker commands are executed: + +```shell +$ docker rm 1348136d32 +docker rm: Error response from daemon: driver "overlay2" failed to remove root filesystem for 1348136d32: error while removing /var/lib/docker/overlay2/844fd3bca8e616572935808061f009d106a8748dfd29a0a4025645457fa21785: invalid argument +``` + +You will find that the rootfs of the corresponding container cannot be found on the host. However, this does not mean that the rootfs is lost. The rootfs is overwritten by the mount point in **/var/lib/docker/overlay2**, and services are still running properly. The solutions are as follows: + +- Solution 1 + 1. Run the following command to check the graphdriver used by Docker: + + ```shell + docker info | grep "Storage Driver" + ``` + + 2. Run the following commands to query the current mount point: + + ```shell + # Devicemapper + mount -l | grep devicemapper + # Overlay2 + mount -l | grep overlay2 + ``` + + The output format is _A_ on _B_ type _C_ \(_D_\). + - _A_: block device name or **overlay** + - _B_: mount point + - _C_: file system type + - _D_: mounting attribute + + 3. Run the **umount** command on the mount points \(_B_\) one by one from bottom to top. + 4. Run the **docker restart** command on all the containers or delete all the containers. + 5. Run the following command to restart Docker: + + ```shell + systemctl restart docker + ``` + +- Solution 2 + 1. Migrate services. + 2. Restart nodes. + +### devicemapper Storage Driver Configuration + +If you need to set the storage driver of Docker to devicemapper, you can also use either of the following methods to explicitly configure the driver: + +- Edit the **/etc/docker/daemon.json** file to explicitly configure the **storage-driver** field. + + ```shell + $ cat /etc/docker/daemon.json + { + "storage-driver": "devicemapper" + } + ``` + +- Edit the **/etc/sysconfig/docker-storage** file and explicitly configure the Docker daemon startup parameters. + + ```shell + $ cat /etc/sysconfig/docker-storage + DOCKER_STORAGE_OPTIONS="--storage-driver=devicemapper" + ``` + +#### Precautions + +- To use devicemapper, you must use the direct-lvm mode. For details about the configuration method, refer to . +- When configuring devicemapper, if the system does not have sufficient space for automatic capacity expansion of thinpool, disable the automatic capacity expansion function. +- Do not set both the following two parameters in the **/etc/lvm/profile/docker-thinpool.profile** file to **100**: + + ```text + activation { + thin_pool_autoextend_threshold=80 + thin_pool_autoextend_percent=20 + } + ``` + +- You are advised to add **--storage-opt dm.use\_deferred\_deletion=true** and **--storage-opt dm.use\_deferred\_removal=true** when using devicemapper. +- When devicemapper is used, you are advised to use Ext4 as the container file system. You need to add **--storage-opt dm.fs=ext4** to the configuration parameters of Docker daemon. +- If graphdriver is devicemapper and the metadata files are damaged and cannot be restored, you need to manually restore the metadata files. Do not directly operate or tamper with metadata of the devicemapper storage driver in Docker daemon. +- When the devicemapper LVM is used, if the devicemapper thinpool is damaged due to abnormal power-off, you cannot ensure the data integrity or whether the damaged thinpool can be restored. Therefore, you need to rebuild the thinpool. + +**Precautions for Switching the devicemapper Storage Pool When the User Namespace Feature Is Enabled on Docker Daemon** + +- Generally, the path of the deviceset-metadata file is **/var/lib/docker/devicemapper/metadata/deviceset-metadata** during container startup. +- If user namespaces are used, the path of the deviceset-metadata file is **/var/lib/docker/**_userNSUID.GID_**/devicemapper/metadata/deviceset-metadata**. +- When you use the devicemapper storage driver and the container is switched between the user namespace scenario and common scenario, the **BaseDeviceUUID** content in the corresponding deviceset-metadata file needs to be cleared. In the thinpool capacity expansion or rebuild scenario, you also need to clear the **BaseDeviceUUID** content in the deviceset-metadata file. Otherwise, the Docker service fails to be restarted. + +## Impact of Forcibly Killing Docker Background Processes + +### Semaphores May Be Residual + +When the devicemapper is used as the graphdriver, forcible killing may cause residual semaphores. Docker creates semaphores when performing operations on devicemapper. If daemon is forcibly killed before the semaphores are released, the release may fail. A maximum of one semaphore can be leaked at a time, and the leakage probability is low. However, the Linux OS has an upper limit on semaphores. When the number of semaphore leakage times reaches the upper limit, new semaphores cannot be created. As a result, Docker daemon fails to be started. The troubleshooting method is as follows: + +1. Check the residual semaphores in the system. + + ```shell + $ ipcs + ------ Message Queues -------- + key msqid owner perms used-bytes messages + ------ Shared Memory Segments -------- + key shmid owner perms bytes nattch status + ------ Semaphore Arrays -------- + key semid owner perms nsems + 0x0d4d3358 238977024 root 600 1 + 0x0d4d0ec9 270172161 root 600 1 + 0x0d4dc02e 281640962 root 600 1 + ``` + +2. Run the **dmsetup** command to check semaphores created by devicemapper. The semaphore set is the subset of the system semaphores queried in the previous step. + + ```shell + dmsetup udevcookies + ``` + +3. Check the upper limit of kernel semaphores. The fourth value is the upper limit of the current system semaphores. + + ```shell + $ cat /proc/sys/kernel/sem + 250 32000 32 128 + ``` + + If the number of residual semaphores in step 1 is the same as the upper limit of semaphores in step 3, the number of residual semaphores reaches the upper limit. In this case, Docker daemon cannot be normally started. You can run the following command to increase the upper limit to restart Docker: + + ```shell + echo 250 32000 32 1024 > /proc/sys/kernel/sem + ``` + + You can also run the following command to manually clear the residual devicemapper semaphores. The following describes how to clear the devicemapper semaphores applied one minute ago. + + ```shell + $ dmsetup udevcomplete_all 1 + This operation will destroy all semaphores older than 1 minutes with keys that have a prefix 3405 (0xd4d). + Do you really want to continue? [y/n]: y + 0 semaphores with keys prefixed by 3405 (0xd4d) destroyed. 0 skipped. + ``` + +### NICs May Be Residual + +When a container is started in bridge mode, forcibly killing may cause residual NICs. In bridge network mode, when Docker creates a container, a pair of veths are created on the host, and then the NIC information is saved to the database. If daemon is forcibly killed before the NIC information is saved to the database of Docker, the NIC cannot be associated with Docker and cannot be deleted during the next startup because Docker deletes unused NICs from its database. + +### Failed to Restart a Container + +If container hook takes a long time, and containerd is forcibly killed during container startup, the container start operation may fail. When containerd is forcibly killed during container startup, an error is returned for the Docker start operation. After containerd is restarted, the last startup may still be in the **runc create** execution phase \(executing the user-defined hook may take a long time\). If you run the **docker start** command again to start the container, the following error message may be displayed: + +```text +Error response from daemon: oci runtime error: container with id exists: xxxxxx +``` + +This error is caused by running **runc create** on an existing container \(or being created\). After the **runc create** operation corresponding to the first start operation is complete, the **docker start** command can be successfully executed. + +The execution of hook is not controlled by Docker. In this case, if the container is recycled, the containerd process may be suspended when an unknown hook program is executed. In addition, the risk is controllable \(although the creation of the current container is affected in a short period\). + +- After the first operation is complete, the container can be successfully started again. +- Generally, a new container is created after the container fails to be started. The container that fails to be started cannot be reused. + +In conclusion, this problem has a constraint on scenarios. + +### Failed to Restart the Docker Service + +The Docker service cannot be restarted properly due to frequent startup in a short period The Docker system service is monitored by systemd. If the Docker service is restarted for more than five times within 10s, the systemd service detects the abnormal startup. Therefore, the Docker service is disabled. Docker can respond to the restart command and be normally restarted only when the next period of 10s starts. + +## Impact of System Power-off + +When a system is unexpectedly powered off or system panic occurs, Docker daemon status may not be updated to the disk in time. As a result, Docker daemon is abnormal after the system is restarted. The possible problems include but are not limited to the following: + +- A container is created before the power-off. After the restart, the container is not displayed when the **docker ps -a** command is run, as the file status of the container is not updated to the disk. As a result, daemon cannot obtain the container status after the restart. +- Before the system power-off, a file is being written. After daemon is restarted, the file format is incorrect or the file content is incomplete. As a result, loading fails. +- As Docker database \(DB\) will be damaged during power-off, all DB files in **data-root** will be deleted during node restart. Therefore, the following information created before the restart will be deleted after the restart: + - Network: Resources created through Docker network will be deleted after the node is restarted. + - Volume: Resources created through Docker volume will be deleted after the node is restarted. + - Cache construction: The cache construction information will be deleted after the node is restarted. + - Metadata stored in containerd: Metadata stored in containerd will be recreated when a container is started. Therefore, the metadata stored in containerd will be deleted when the node is restarted. + + > [!NOTE]NOTE + > If you want to manually clear data and restore the environment, you can set the environment variable **DISABLE\_CRASH\_FILES\_DELETE** to **true** to disable the function of clearing DB files when the daemon process is restarted due to power-off. diff --git a/docs/en/cloud/container_engine/docker_engine/overview.md b/docs/en/cloud/container_engine/docker_engine/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..7c1eac673ecaa9b085e23b322d1e98858305e84b --- /dev/null +++ b/docs/en/cloud/container_engine/docker_engine/overview.md @@ -0,0 +1,7 @@ +# Docker Container + +Docker is an open-source Linux container engine that enables quick application packaging, deployment, and delivery. The original meaning of Docker is dork worker, whose job is to pack the goods to the containers, and move containers, and load containers. Similarly, the job of Docker in Linux is to pack applications to containers, and deploy and run applications on various platforms using containers. Docker uses Linux Container technology to turn applications into standardized, portable, and self-managed components, enabling the "build once" and "run everywhere" features of applications. Features of Docker technology include: quick application release, easy application deployment and management, and high application density. + +> ![](./public_sys-resources/icon-note.gif) **Note:** +> +> Root privileges are necessary for installing and operating Docker containers. diff --git a/docs/en/cloud/container_engine/docker_engine/public_sys-resources/icon-note.gif b/docs/en/cloud/container_engine/docker_engine/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/cloud/container_engine/docker_engine/public_sys-resources/icon-note.gif differ diff --git a/docs/en/cloud/container_engine/docker_engine/public_sys-resources/icon-notice.gif b/docs/en/cloud/container_engine/docker_engine/public_sys-resources/icon-notice.gif new file mode 100644 index 0000000000000000000000000000000000000000..86024f61b691400bea99e5b1f506d9d9aef36e27 Binary files /dev/null and b/docs/en/cloud/container_engine/docker_engine/public_sys-resources/icon-notice.gif differ diff --git a/docs/en/cloud/container_engine/docker_engine/statistics.md b/docs/en/cloud/container_engine/docker_engine/statistics.md new file mode 100644 index 0000000000000000000000000000000000000000..379fef5aa97f105b3cf0736d76929530329deb43 --- /dev/null +++ b/docs/en/cloud/container_engine/docker_engine/statistics.md @@ -0,0 +1,102 @@ +# Statistics + +- [Statistics](#statistics) + - [events](#events) + - [info](#info) + - [version](#version) + +## events + +Syntax: **docker events \[**_options_**\]** + +Function: Obtains real-time events from the docker daemon. + +Parameter description: + +**--since=""**: Displays events generated after the specified timestamp. + +**--until=""**: Displays events generated before the specified timestamp. + +Example: + +After the **docker events** command is executed, a container is created and started by running the **docker run** command. create and start events are output. + +```shell +$ sudo docker events +2019-08-28T16:23:09.338838795+08:00 container create 53450588a20800d8231aa1dc4439a734e16955387efb5f259c47737dba9e2b5e (image=busybox:latest, name=eager_wu) +2019-08-28T16:23:09.339909205+08:00 container attach 53450588a20800d8231aa1dc4439a734e16955387efb5f259c47737dba9e2b5e (image=busybox:latest, name=eager_wu) +2019-08-28T16:23:09.397717518+08:00 network connect e2e20f52662f1ee2b01545da3b02e5ec7ff9c85adf688dce89a9eb73661dedaa (container=53450588a20800d8231aa1dc4439a734e16955387efb5f259c47737dba9e2b5e, name=bridge, type=bridge) +2019-08-28T16:23:09.922224724+08:00 container start 53450588a20800d8231aa1dc4439a734e16955387efb5f259c47737dba9e2b5e (image=busybox:latest, name=eager_wu) +2019-08-28T16:23:09.924121158+08:00 container resize 53450588a20800d8231aa1dc4439a734e16955387efb5f259c47737dba9e2b5e (height=48, image=busybox:latest, name=eager_wu, width=210) +``` + +## info + +Syntax: **docker info** + +Function: Displays the Docker system information, including the number of containers, number of images, image storage driver, container execution driver, kernel version, and host OS version. + +Parameter description: none. + +Example: + +```shell +$ sudo docker info +Containers: 4 + Running: 3 + Paused: 0 + Stopped: 1 +Images: 45 +Server Version: 18.09.0 +Storage Driver: devicemapper + Pool Name: docker-thinpool + Pool Blocksize: 524.3kB + Base Device Size: 10.74GB + Backing Filesystem: ext4 + Udev Sync Supported: true + Data Space Used: 11GB + Data Space Total: 51GB + Data Space Available: 39.99GB + Metadata Space Used: 5.083MB + Metadata Space Total: 532.7MB + Metadata Space Available: 527.6MB + Thin Pool Minimum Free Space: 5.1GB + Deferred Removal Enabled: true + Deferred Deletion Enabled: true + Deferred Deleted Device Count: 0 +...... +``` + +## version + +Syntax: **docker version** + +Function: Displays the Docker version information, including the client version, server version, Go version, and OS and Arch information. + +Parameter description: none. + +Example: + +```shell +$ sudo docker version +Client: + Version: 18.09.0 + EulerVersion: 18.09.0.48 + API version: 1.39 + Go version: go1.11 + Git commit: cbf6283 + Built: Mon Apr 1 00:00:00 2019 + OS/Arch: linux/arm64 + Experimental: false + +Server: + Engine: + Version: 18.09.0 + EulerVersion: 18.09.0.48 + API version: 1.39 (minimum version 1.12) + Go version: go1.11 + Git commit: cbf6283 + Built: Mon Apr 1 00:00:00 2019 + OS/Arch: linux/arm64 + Experimental: false +``` diff --git a/docs/en/cloud/container_engine/isula_container_engine/_toc.yaml b/docs/en/cloud/container_engine/isula_container_engine/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..e42df515fc7797f8fd92000ac87bbd774964614c --- /dev/null +++ b/docs/en/cloud/container_engine/isula_container_engine/_toc.yaml @@ -0,0 +1,56 @@ +label: iSulad Container Engine +isManual: true +description: >- + iSula is a C/C++-based container engine known for its lightweight design, + flexibility, simplicity, and high performance. +sections: + - label: Overview + href: ./overview.md + - label: Installation, Upgrade, and Uninstallation + href: ./installation-upgrade-Uninstallation.md + sections: + - label: Installation and Configuration + href: ./installation-configuration.md + - label: Upgrade + href: ./upgrade-methods.md + - label: Uninstallation + href: ./uninstallation.md + - label: User Guide + href: ./application-scenarios.md + sections: + - label: Container Management + href: ./container-management.md + - label: Interconnection with the CNI Network + href: ./interconnection-with-the-cni-network.md + - label: Container Resource Management + href: ./container-resource-management.md + - label: Privileged Container + href: ./privileged-container.md + - label: CRI API v1alpha2 + href: ./cri.md + - label: CRI API v1 + href: ./cri-2.md + - label: Image Management + href: ./image-management.md + - label: Checking the Container Health Status + href: ./checking-the-container-health-status.md + - label: Querying Information + href: ./querying-information.md + - label: Security Features + href: ./security-features.md + - label: Supporting OCI hooks + href: ./supporting-oci-hooks.md + - label: Local Volume Management + href: ./local-volume-management.md + - label: Interconnecting iSulad shim v2 with StratoVirt + href: ./interconnecting-isula-shim-v2-with-stratovirt.md + - label: iSulad Support for cgroup v2 + href: ./isulad-support-cgroup-v2.md + - label: iSulad Support for CDI + href: ./isulad-support-cdi.md + - label: iSulad Support for NRI + href: ./isulad-support-nri.md + - label: Common Issues and Solutions + href: ./isula-faqs.md + - label: Appendix + href: ./appendix.md diff --git a/docs/en/cloud/container_engine/isula_container_engine/appendix.md b/docs/en/cloud/container_engine/isula_container_engine/appendix.md new file mode 100644 index 0000000000000000000000000000000000000000..2daa3e1e5b1fe9a3f060617bed4078c7098f4783 --- /dev/null +++ b/docs/en/cloud/container_engine/isula_container_engine/appendix.md @@ -0,0 +1,713 @@ +# Appendix + +- [Appendix](#appendix) + - [Command Line Parameters](#command-line-parameters) + - [CNI Parameters](#cni-parameters) + +## Command Line Parameters + +**Table 1** login command parameters + + + + + + + + + + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

login

+

  

+

  

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+

-p, --password

+

Specifies the password for logging in to the registry.

+

--password-stdin

+

Specifies the password for obtaining the registry from standard input.

+

-u, --username

+

Specifies the username for logging in to the registry.

+
+ +**Table 2** logout command parameters + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

logout

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+
+ +**Table 3** pull command parameters + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

pull

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+
+ +**Table 4** rmi command parameters + + + + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

rmi

+

  

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+

-f, --force

+

Forcibly removes an image.

+
+ +**Table 5** load command parameters + + + + + + + + + + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

load

+

-H, --host (supported only by iSula)

+

Specifies the iSulad socket file path to be accessed.

+

-i, --input

+

Specifies where to import an image. If the image is of the docker type, the value is the image package path. If the image is of the embedded type, the value is the image manifest path.

+

--tag

+

Uses the image name specified by TAG instead of the default image name. This parameter is supported when the type is set to docker.

+

-t, --type

+

Specifies the image type. The value can be embedded or docker (default value).

+
+ +**Table 6** images command parameters + + + + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

images

+

  

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+

-q, --quit

+

Displays only the image name.

+
+ +**Table 7** inspect command parameters + + + + + + + + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

inspect

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+

-f, --format

+

Outputs using a template.

+

-t, --time

+

Timeout interval, in seconds. If the inspect command fails to query container information within the specified period, the system stops waiting and reports an error immediately. The default value is 120s. If the value is less than or equal to 0, the inspect command keeps waiting until the container information is obtained successfully.

+
+ +## CNI Parameters + +**Table 1** CNI single network parameters + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Parameter

+

Type

+

Mandatory or Not

+

Description

+

cniVersion

+

string

+

Yes

+

CNI version. Only 0.3.0 and 0.3.1 are supported.

+

name

+

string

+

Yes

+

Network name, which is user-defined and must be unique.

+

type

+

string

+

Yes

+

Network type. The following types are supported:

+

underlay_ipvlan

+

overlay_l2

+

underlay_l2

+

vpc-router

+

dpdk-direct

+

phy-direct

+

ipmasp

+

bool

+

No

+

Configures the IP masquerade.

+

ipam

+

structure

+

No

+

For details, see the IPAM parameter definition.

+

ipam.type

+

string

+

No

+

IPAM type. The following types are supported:

+

(1) For underlay_l2, overlay_l2, and vpc-router networking, only the default value distributed_l2 is supported.

+

(2) For underlay_ipvlan networking, the default value is distributed_l2. In the CCN scenario, only null and fixed are supported. In the CCE and FST 5G core scenarios, only null and distributed_l2 are supported.

+

(3) For phy-direct and dpdk-direct networking, the default value is l2, and optional values are null and distributed_l2. In the FST 5G core scenario, only null and distributed_l2 are supported.

+

Description:

+

If the value is out of the range (for example, host-local), Canal automatically sets the value to the default value and no error is returned.

+

null: Canal is not used to manage IP addresses.

+

fixed: fixed IP address, which is used in the CCN scenario.

+

l2: This value is not used in any scenario.

+

distributed_l2: The distributed small subnet is used to manage IP addresses.

+

ipam.subnet

+

string

+

No

+

Subnet information. Canal supports the subnet mask ranging from 8 to 29. The IP address cannot be a multicast address (for example, 224.0.0.0/4), reserved address (240.0.0.0/4), local link address (169.254.0.0/16), or local loop address (127.0.0.0/8).

+

ipam.gateway

+

string

+

No

+

Gateway IP address.

+

ipam.range-start

+

string

+

No

+

Available start IP address.

+

ipam.range-end

+

string

+

No

+

Available end IP address.

+

ipam.routes

+

structure

+

No

+

Subnet list. Each element is a route dictionary. For details, see the route definition.

+

ipam.routes.dst

+

string

+

No

+

Destination network.

+

ipam.routes.gw

+

string

+

No

+

Gateway address.

+

dns

+

structure

+

No

+

Contains some special DNS values.

+

dns.nameservers

+

[]string

+

No

+

NameServers

+

dns.domain

+

string

+

No

+

Domain

+

dns.search

+

[]string

+

No

+

Search

+

dns.options

+

[]string

+

No

+

Options

+

multi_entry

+

int

+

No

+

Number of IP addresses required by a vNIC. The value ranges from 0 to 16. For physical passthrough, a maximum of 128 IP addresses can be applied for a single NIC.

+

backup_mode

+

bool

+

No

+

Active/Standby mode, which is used only for phy-direct and dpdk-direct networking.

+

vlanID

+

int

+

No

+

The value ranges from 0 to 4095. It can be specified through PaaS.

+

vlan_inside

+

bool

+

No

+

The value true indicates that the VLAN function is implemented internally on the node, and the value false indicates that the VLAN function is implemented externally.

+

vxlanID

+

int

+

No

+

The value ranges from 0 to 16777215. It can be specified through PaaS.

+

vxlan_inside

+

bool

+

No

+

The value true indicates that the VLAN function is implemented internally on the node, and the value false indicates that the VLAN function is implemented externally.

+

action

+

string

+

No

+

This parameter can be used only with the special container ID 000000000000.

+

Create: creates a network.

+

Delete: deletes a network.

+

args

+

map[string]interface{}

+

No

+

Key-value pair type.

+

runtimeConfig

+

structure

+

No

+

None

+

capabilities

+

structure

+

No

+

None

+
+ +**Table 2** CNI args parameters + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Parameter

+

Type

+

Mandatory

+

Description

+

K8S_POD_NAME

+

string

+

No

+

Set this parameter when you apply for a fixed IP address (runtimeConfig.ican_caps.fixed_ip is set to true).

+

K8S_POD_NAMESPACE

+

string

+

No

+

Set this parameter when you apply for a fixed IP address (runtimeConfig.ican_caps.fixed_ip is set to true).

+

SECURE_CONTAINER

+

string

+

No

+

Secure container flag.

+

multi_port

+

int

+

No

+

The value ranges from 1 to 8. The default value is 1. Specifies the number of passthrough NICs. Only phy-direct and dpdk-direct networks are supported.

+

phy-direct

+

string

+

No

+

Specifies the NIC to be connected when you create an SR-IOV container network.

+

dpdk-direct

+

string

+

No

+

Specifies the NIC to be connected when you create a DPDK passthrough container network.

+

tenant_id

+

string

+

No

+

Indicates the tenant ID.

+

Only vpc-router networks are supported.

+

vpc_id

+

string

+

No

+

VPC ID.

+

Only vpc-router networks are supported.

+

secret_name

+

string

+

No

+

Specifies the AK/SK object name on the K8S APIServer.

+

Only vpc-router networks are supported.

+

For details, see the configuration of VPC-Router logical networks.

+

IP

+

string

+

No

+

IP address specified by the user, in the format of 192.168.0.10.

+

K8S_POD_NETWORK_ARGS

+

string

+

No

+

Specifies an IP address, in the format of 192.168.0.10. If both IP and K8S_POD_NETWORK_ARGS in args are not empty, the value of K8S_POD_NETWORK_ARGS prevails.

+

INSTANCE_NAME

+

string

+

No

+

INSTANCE ID.

+

Refer to fixed IP addresses that support containers.

+

dist_gateway_disable

+

bool

+

No

+

The value true indicates that no gateway is created, and the value false indicates that a gateway is created.

+

phynet

+

string or []string

+

No

+

Specifies the name of the physical plane to be added. The physical plane name is predefined and corresponds to that in the SNC system. When two plane names are entered, the active and standby planes are supported. Example: phy_net1 or ["phy_net2","phy_net3"]

+

endpoint_policies

+

struct

+

No

+

"endpoint_policies": [

+

{

+

"Type": "",

+

"ExceptionList": [

+

""

+

],

+

"NeedEncap": true,

+

"DestinationPrefix": ""

+

}

+

]

+

port_map

+

struct

+

No

+

On a NAT network, container ports can be advertised to host ports.

+

"port_map": [

+

{

+

"local_port": number,

+

"host_port": number,

+

"protocol": [string...]

+

}...

+

]

+
+ +**Table 3** CNI multiple network parameters + + + + + + + + + + + + + + + + + + + + + + + + +

Parameter

+

Type

+

Mandatory

+

Description

+

cniVersion

+

string

+

Yes

+

CNI version. Only 0.3.0 and 0.3.1 are supported.

+

name

+

string

+

Yes

+

Network name, which is user-defined and must be unique.

+

plugins

+

struct

+

Yes

+

For details, see CNI single network parameters

+
diff --git a/docs/en/cloud/container_engine/isula_container_engine/application-scenarios.md b/docs/en/cloud/container_engine/isula_container_engine/application-scenarios.md new file mode 100644 index 0000000000000000000000000000000000000000..c15762fae637869dfe46f9e6620f49b584583187 --- /dev/null +++ b/docs/en/cloud/container_engine/isula_container_engine/application-scenarios.md @@ -0,0 +1,6 @@ +# User Guide + +This section describes how to use iSulad. + +> ![](./public_sys-resources/icon-note.gif)**Note:** +> All iSulad operations require root privileges. diff --git a/docs/en/cloud/container_engine/isula_container_engine/checking-the-container-health-status.md b/docs/en/cloud/container_engine/isula_container_engine/checking-the-container-health-status.md new file mode 100644 index 0000000000000000000000000000000000000000..5db4a32ce83e5d71b4330e25ecbf498c78ea3a12 --- /dev/null +++ b/docs/en/cloud/container_engine/isula_container_engine/checking-the-container-health-status.md @@ -0,0 +1,68 @@ +# Checking the Container Health Status + +- [Checking the Container Health Status](#checking-the-container-health-status) + - [Scenarios](#scenarios) + - [Configuration Methods](#configuration-methods) + - [Check Rules](#check-rules) + - [Usage Restrictions](#usage-restrictions) + +## Scenarios + +In the production environment, bugs are inevitable in applications provided by developers or services provided by platforms. Therefore, a management system is indispensable for periodically checking and repairing applications. The container health check mechanism adds a user-defined health check function for containers. When a container is created, the **--health-cmd** option is configured so that commands are periodically executed in the container to monitor the health status of the container based on return values. + +## Configuration Methods + +Configurations during container startup: + +```shell +isula run -itd --health-cmd "echo iSulad >> /tmp/health_check_file || exit 1" --health-interval 5m --health-timeout 3s --health-exit-on-unhealthy busybox bash +``` + +The configurable options are as follows: + +- **--health-cmd**: This option is mandatory. If **0** is returned after a command is run in a container, the command execution succeeds. If a value other than **0** is returned, the command execution fails. +- **--health-interval**: interval between two consecutive command executions. The default value is **30s**. The value ranges from **1s** to the maximum value of Int64 \(unit: nanosecond\). If the input parameter is set to **0s**, the default value is used. +- **--health-timeout**: maximum duration for executing a single check command. If the execution times out, the command execution fails. The default value is **30s**. The value ranges from **1s** to the maximum value of Int64 \(unit: nanosecond\). If the input parameter is set to **0s**, the default value is used. Only containers whose runtime is of the LCR type are supported. +- **--health-start-period**: container initialization time. The default value is **0s**. The value ranges from **1s** to the maximum value of Int64 \(unit: nanosecond\). +- **--health-retries**: maximum number of retries for the health check. The default value is **3**. The maximum value is the maximum value of Int32. +- **--health-exit-on-unhealthy**: specifies whether to kill a container when it is unhealthy. The default value is **false**. + +## Check Rules + +1. After a container is started, the container status is **health:starting**. +2. After the period specified by **start-period**, the **cmd** command is periodically executed in the container at the interval specified by **interval**. That is, after the command is executed, the command will be executed again after the specified period. +3. If the **cmd** command is successfully executed within the time specified by **timeout** and the return value is **0**, the check is successful. Otherwise, the check fails. If the check is successful, the container status changes to **health:healthy**. +4. If the **cmd** command fails to be executed for the number of times specified by **retries**, the container status changes to **health:unhealthy**, and the container continues the health check. +5. When the container status is **health:unhealthy**, the container status changes to **health:healthy** if a check succeeds. +6. If **--exit-on-unhealthy** is set, and the container exits due to reasons other than being killed \(the returned exit code is **137**\), the health check takes effect only after the container is restarted. +7. When the **cmd** command execution is complete or times out, Docker daemon will record the start time, return value, and standard output of the check to the configuration file of the container. A maximum of five records can be recorded. In addition, the configuration file of the container stores health check parameters. +8. When the container is running, the health check status is written into the container configurations. You can run the **isula inspect** command to view the status. + +```json +"Health": { + "Status": "healthy", + "FailingStreak": 0, + "Log": [ + { + "Start": "2018-03-07T07:44:15.481414707-05:00", + "End": "2018-03-07T07:44:15.556908311-05:00", + "ExitCode": 0, + "Output": "" + }, + { + "Start": "2018-03-07T07:44:18.557297462-05:00", + "End": "2018-03-07T07:44:18.63035891-05:00", + "ExitCode": 0, + "Output": "" + }, + ...... +} +``` + +## Usage Restrictions + +- A maximum of five health check status records can be stored in a container. The last five records are saved. +- If health check parameters are set to **0** during container startup, the default values are used. +- After a container with configured health check parameters is started, if iSulad daemon exits, the health check is not executed. After iSulad daemon is restarted, the health status of the running container changes to **starting**. Afterwards, the check rules are the same as above. +- If the health check fails for the first time, the health check status will not change from **starting** to **unhealthy** until the specified number of retries \(**--health-retries**\) is reached, or to **healthy** until the health check succeeds. +- The health check function of containers whose runtime is of the Open Container Initiative \(OCI\) type needs to be improved. Only containers whose runtime is of the LCR type are supported. diff --git a/docs/en/cloud/container_engine/isula_container_engine/container-management.md b/docs/en/cloud/container_engine/isula_container_engine/container-management.md new file mode 100644 index 0000000000000000000000000000000000000000..7fdc6b41ddb2134a3fc93c6e4ee8d05d1f91f420 --- /dev/null +++ b/docs/en/cloud/container_engine/isula_container_engine/container-management.md @@ -0,0 +1,1956 @@ +# Container Management + +- [Container Management](#container-management) + - [Creating a Container](#creating-a-container) + - [Starting a Container](#starting-a-container) + - [Running a Container](#running-a-container) + - [Stopping a Container](#stopping-a-container) + - [Forcibly Stopping a Container](#forcibly-stopping-a-container) + - [Removing a Container](#removing-a-container) + - [Attaching to a Container](#attaching-to-a-container) + - [Renaming a Container](#renaming-a-container) + - [Executing a Command in a Running Container](#executing-a-command-in-a-running-container) + - [Querying Information About a Single Container](#querying-information-about-a-single-container) + - [Querying Information About All Containers](#querying-information-about-all-containers) + - [Restarting a Container](#restarting-a-container) + - [Waiting for a Container to Exit](#waiting-for-a-container-to-exit) + - [Viewing Process Information in a Container](#viewing-process-information-in-a-container) + - [Displaying Resource Usage Statistics of a Container](#displaying-resource-usage-statistics-of-a-container) + - [Obtaining Container Logs](#obtaining-container-logs) + - [Copying Data Between a Container and a Host](#copying-data-between-a-container-and-a-host) + - [Pausing a Container](#pausing-a-container) + - [Resuming a Container](#resuming-a-container) + - [Obtaining Event Messages from the Server in Real Time](#obtaining-event-messages-from-the-server-in-real-time) + +## Creating a Container + +### Description + +To create a container, run the **isula create** command. The container engine will use the specified container image to create a read/write layer, or use the specified local rootfs as the running environment of the container. After the creation is complete, the container ID is output as standard output. You can run the **isula start** command to start the container. The new container is in the **inited** state. + +### Usage + +```shell +isula create [OPTIONS] IMAGE [COMMAND] [ARG...] +``` + +### Parameters + +The following table lists the parameters supported by the **create** command. + +**Table 1** Parameter description + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

create

+

  

+

--annotation

+

Sets annotations for the container. For example, set the native.umask parameter.

+
--annotation native.umask=normal #The umask value of the started container is 0022.
+--annotation native.umask=secure #The umask value of the started container is 0027.
+

If this parameter is not set, the umask configuration in iSulad is used.

+

--cap-drop

+

Deletes Linux permissions.

+

--cgroup-parent

+

Specifies the cgroup parent path of the container.

+

--cpuset-cpus

+

Allowed CPUs (for example, 0-3, 0, 1).

+

--cpu-shares

+

CPU share (relative weight).

+

--cpu-quota

+

Limits the CPU CFS quota.

+

--device=[]

+

Adds a device to the container.

+

--dns

+

Adds a DNS server.

+

--dns-opt

+

Adds DNS options.

+

--dns-search

+

Sets the search domain of a container.

+

-e, --env

+

Sets environment variables.

+

--env-file

+

Configures environment variables using a file.

+

--entrypoint

+

Entry point to run when the container is started.

+

--external-rootfs=PATH

+

Specifies a rootfs (a folder or block device) that is not managed by iSulad for the container.

+

--files-limit

+

Limits the number of file handles that can be opened in a container. The value -1 indicates no limit.

+

--group-add=[]

+

Adds additional user groups to the container.

+

--help

+

Displays help information.

+

--health-cmd

+

Command executed in a container.

+

--health-exit-on-unhealthy

+

Determines whether to kill a container when the container is detected unhealthy.

+

--health-interval

+

Interval between two consecutive command executions.

+

--health-retries

+

Maximum number of health check retries.

+

--health-start-period

+

Container initialization interval.

+

--health-timeout

+

Maximum time for executing a single check command.

+

--hook-spec

+

Hook configuration file.

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+

-h, --hostname

+

Container host name.

+

-i, --interactive

+

Enables the standard input of the container even if it is not connected to the standard input of the container.

+

--hugetlb-limit=[]

+

Limits the size of huge-page files, for example, --hugetlb-limit 2MB:32MB.

+

--log-opt=[]

+

Log driver option. By default, the container serial port log function is disabled. You can run the --log-opt disable-log=false command to enable it.

+

-l,--label

+

Sets a label for a container.

+

--lablel-file

+

Sets container labels using files.

+

-m, --memory

+

Memory limit.

+

--memory-reservation

+

Sets the container memory limit. The default value is the same as that of --memory. --memory is a hard limit, and --memory-reservation is a soft limit. When the memory usage exceeds the preset value, the memory usage is dynamically adjusted (the system attempts to reduce the memory usage to a value less than the preset value when reclaiming the memory). However, the memory usage may exceed the preset value. Generally, this parameter can be used together with --memory. The value must be less than the preset value of --memory. The minimum value is 4 MB.

+

--memory-swap

+

Memory swap space, which should be a positive integer. The value -1 indicates no limit.

+

--memory-swappiness

+

The value of swappiness is a positive integer ranging from 0 to 100. The smaller the value is, the less the swap partition is used and the more the memory is used in the Linux system. The larger the value is, the more the swap space is used by the kernel. The default value is -1, indicating that the default system value is used.

+

--mount

+

Mounts the host directory, volume, or file system to the container.

+

--no-healthcheck

+

Disables the health check configuration.

+

--name=NAME

+

Container name.

+

--net=none

+

Connects a container to a network.

+

--pids-limit

+

Limits the number of processes that can be executed in the container. The value -1 indicates no limit.

+

--privileged

+

Grants container extension privileges.

+

-R, --runtime

+

Container runtime. The parameter value can be lcr, which is case insensitive. Therefore, LCR and lcr are equivalent.

+

--read-only

+

Sets the rootfs of a container to read-only.

+

--restart

+

Restart policy upon container exit.

+

For a system container, --restart on-reboot is supported.

+

--storage-opt

+

Configures the storage driver option for a container.

+

-t, --tty

+

Allocates a pseudo terminal.

+

--ulimit

+

Sets the ulimit for a container.

+

-u, --user

+

User name or UID, in the format of [<name|uid>][:<group|gid>].

+

-v, --volume=[]

+

Mounts a volume.

+

--volumes-from=[]

+

Uses the mounting configuration of the specified container.

+
+ +### Constraints + +- When the **--user** or **--group-add** parameter is used to verify the user or group during container startup, if the container uses an OCI image, the verification is performed in the **etc/passwd** and **etc/group** files of the actual rootfs of the image. If a folder or block device is used as the rootfs of the container, the **etc/passwd** and **etc/group** files in the host are verified. The rootfs ignores mounting parameters such as **-v** and **--mount**. That is, when these parameters are used to attempt to overwrite the **etc/passwd** and **etc/group** files, the parameters do not take effect during the search and take effect only when the container is started. The generated configuration is saved in the **iSulad root directory/engine/container ID/start\_generate\_config.json** file. The file format is as follows: + + ```json + { + "uid": 0, + "gid": 8, + "additionalGids": [ + 1234, + 8 + ] + } + ``` + +### Example + +Create a container. + +```shell +$ isula create busybox +fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1 +$ isula ps -a +STATUS PID IMAGE COMMAND EXIT_CODE RESTART_COUNT STARTAT FINISHAT RUNTIME ID NAMES inited - busybox "sh" 0 0 - - lcr fd7376591a9c fd7376591a9c4521... +``` + +## Starting a Container + +### Description + +To start one or more containers, run the **isula start** command. + +### Usage + +```shell +isula start [OPTIONS] CONTAINER [CONTAINER...] +``` + +### Parameters + +The following table lists the parameters supported by the **start** command. + +**Table 1** Parameter description + + + + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

start

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+

-R, --runtime

+

Container runtime. The parameter value can be lcr, which is case insensitive. Therefore, LCR and lcr are equivalent.

+
+ +### Example + +Start a new container. + +```shell +isula start fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1 +``` + +## Running a Container + +### Description + +To create and start a container, run the **isula run** command. You can use a specified container image to create a container read/write layer and prepare for running the specified command. After the container is created, run the specified command to start the container. The **run** command is equivalent to creating and starting a container. + +### Usage + +```shell +isula run [OPTIONS] ROOTFS|IMAGE [COMMAND] [ARG...] +``` + +### Parameters + +The following table lists the parameters supported by the **run** command. + +**Table 1** Parameter description + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

run

+

--annotation

+

Sets annotations for the container. For example, set the native.umask option.

+
--annotation native.umask=normal #The umask value of the started container is 0022.
+--annotation native.umask=secure #The umask value of the started container is 0027.
+

If this parameter is not set, the umask configuration in iSulad is used.

+

--cap-add

+

Adds Linux functions.

+

--cap-drop

+

Deletes Linux functions.

+

--cgroup-parent

+

Specifies the cgroup parent path of the container.

+

--cpuset-cpus

+

Allowed CPUs (for example, 0-3, 0, 1).

+

--cpu-shares

+

CPU share (relative weight).

+

--cpu-quota

+

Limits the CPU CFS quota.

+

-d, --detach

+

Runs the container in the background and displays the container ID.

+

--device=[]

+

Adds a device to the container.

+

--dns

+

Adds a DNS server.

+

--dns-opt

+

Adds DNS options.

+

--dns-search

+

Sets the search domain of a container.

+

-e, --env

+

Sets environment variables.

+

--env-file

+

Configures environment variables using a file.

+

--entrypoint

+

Entry point to run when the container is started.

+

--external-rootfs=PATH

+

Specifies a rootfs (a folder or block device) that is not managed by iSulad for the container.

+

--files-limit

+

Limits the number of file handles that can be opened in the container. The value -1 indicates no limit.

+

--group-add=[]

+

Adds additional user groups to the container.

+

--help

+

Displays help information.

+

--health-cmd

+

Command executed in a container.

+

--health-exit-on-unhealthy

+

Determines whether to kill a container when the container is detected unhealthy.

+

--health-interval

+

Interval between two consecutive command executions.

+

--health-retries

+

Maximum number of health check retries.

+

--health-start-period

+

Container initialization interval.

+

--health-timeout

+

Maximum time for executing a single check command.

+

--hook-spec

+

Hook configuration file.

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+

-h, --hostname

+

Container host name.

+

--hugetlb-limit=[]

+

Limits the size of huge-page files, for example, --hugetlb-limit 2MB:32MB.

+

-i, --interactive

+

Enables the standard input of the container even if it is not connected to the standard input of the container.

+

--log-opt=[]

+

Log driver option. By default, the container serial port log function is disabled. You can run the --log-opt disable-log=false command to enable it.

+

-m, --memory

+

Memory limit.

+

--memory-reservation

+

Sets the container memory limit. The default value is the same as that of --memory. --memory is a hard limit, and --memory-reservation is a soft limit. When the memory usage exceeds the preset value, the memory usage is dynamically adjusted (the system attempts to reduce the memory usage to a value less than the preset value when reclaiming the memory). However, the memory usage may exceed the preset value. Generally, this parameter can be used together with --memory. The value must be less than the preset value of --memory. The minimum value is 4 MB.

+

--memory-swap

+

Memory swap space, which should be a positive integer. The value -1 indicates no limit.

+

--memory-swappiness

+

The value of swappiness is a positive integer ranging from 0 to 100. The smaller the value is, the less the swap partition is used and the more the memory is used in the Linux system. The larger the value is, the more the swap space is used by the kernel. The default value is -1, indicating that the default system value is used.

+

--mount

+

Mounts a host directory to a container.

+

--no-healthcheck

+

Disables the health check configuration.

+

--name=NAME

+

Container name.

+

--net=none

+

Connects a container to a network.

+

--pids-limit

+

Limits the number of processes that can be executed in the container. The value -1 indicates no limit.

+

--privileged

+

Grants container extension privileges.

+

-R, --runtime

+

Container runtime. The parameter value can be lcr, which is case insensitive. Therefore, LCR and lcr are equivalent.

+

--read-only

+

Sets the rootfs of a container to read-only.

+

--restart

+

Restart policy upon container exit.

+

For a system container, --restart on-reboot is supported.

+

--rm

+

Automatically clears a container upon exit.

+

--storage-opt

+

Configures the storage driver option for a container.

+

-t, --tty

+

Allocates a pseudo terminal.

+

--ulimit

+

Sets the ulimit for a container.

+

-u, --user

+

User name or UID, in the format of [<name|uid>][:<group|gid>].

+

-v, --volume=[]

+

Mounts a volume.

+
+ +### Constraints + +- When the parent process of a container exits, the corresponding container automatically exits. +- When a common container is created, the parent process cannot be initiated because the permission of common containers is insufficient. As a result, the container does not respond when you run the **attach** command though it is created successfully. +- If **--net** is not specified when the container is running, the default host name is **localhost**. +- If the **--files-limit** parameter is to transfer a small value, for example, 1, when the container is started, iSulad creates a cgroup, sets the files.limit value, and writes the PID of the container process to the **cgroup.procs** file of the cgroup. At this time, the container process has opened more than one handle. As a result, a write error is reported, and the container fails to be started. +- If both**--mount** and **--volume** exist and their destination paths conflict, **--mount** will be run after **--volume** \(that is, the mount point in **--volume** will be overwritten\). + + Note: The value of the **type** parameter of lightweight containers can be **bind** or **squashfs**. When **type** is set to **squashfs**, **src** is the image path. The value of the **type** parameter of the native Docker can be **bind**, **volume**, and **tmpfs**. + +- The restart policy does not support **unless-stopped**. +- The values returned for Docker and lightweight containers are 127 and 125 respectively in the following three scenarios: + + The host device specified by **--device** does not exist. + + The hook JSON file specified by **--hook-spec** does not exist. + + The entry point specified by **--entrypoint** does not exist. + +- When the **--volume** parameter is used, /dev/ptmx will be deleted and recreated during container startup. Therefore, do not mount the **/dev** directory to that of the container. Use **--device** to mount the devices in **/dev** of the container. +- Do not use the echo option to input data to the standard input of the **run** command. Otherwise, the client will be suspended. The echo value should be directly transferred to the container as a command line parameter. + + ```shell + [root@localhost ~]# echo ls | isula run -i busybox /bin/sh + + + ^C + [root@localhost ~]# + ``` + + The client is suspended when the preceding command is executed because the preceding command is equivalent to input **ls** to **stdin**. Then EOF is read and the client does not send data and waits for the server to exit. However, the server cannot determine whether the client needs to continue sending data. As a result, the server is suspended in reading data, and both parties are suspended. + + The correct execution method is as follows: + + ```shell + [root@localhost ~]# isula run -i busybox ls + bin + dev + etc + home + proc + root + sys + tmp + usr + var + [root@localhost ~]# + ``` + +- If the root directory \(/\) of the host is used as the file system of the container, the following situations may occur during the mounting: + + **Table 2** Mounting scenarios + + + + + + + + + + + + + +

Host Path (Source)

+

Container Path (Destination)

+

/home/test1

+

/mnt/

+

/home/test2

+

/mnt/abc

+
+ + > [!TIP]NOTICE + > Scenario 1: Mount **/home/test1** and then **/home/test2**. In this case, the content in **/home/test1** overwrites the content in **/mnt**. As a result, the **abc** directory does not exist in **/mnt**, and mounting**/home/test2** to **/mnt/abc** fails. + > Scenario 2: Mount **/home/test2** and then **/home/test1**. In this case, the content of **/mnt** is replaced with the content of **/home/test1** during the second mounting. In this way, the content mounted during the first mounting from **/home/test2** to **/mnt/abc** is overwritten. + > The first scenario is not supported. For the second scenario, users need to understand the risk of data access failures. + +- Exercise caution when configuring the **/sys** and **/proc** directories as writable. The **/sys** and **/proc** directories contain interfaces for Linux to manage kernel parameters and devices. Configuring these directories as writable in a container may lead to container escape. +- Exercise caution when configuring containers to share namespaces with the host. For example, using **--pid**, **--ipc**, **--uts**, or **--net** to share namespace spaces between the container and the host eliminates namespace isolation between them. This allows attacks on the host from within the container. For instance, using **--pid** to share the PID namespace with the host enables the container to view and kill processes on the host. +- Exercise caution when using parameters like **--device** or **-v** to mount host resources. Avoid mapping sensitive directories or devices of the host into the container to prevent sensitive information leakage. +- Exercise caution when starting containers with the **--privileged** option. The **--privileged** option grants excessive permissions to the container, which can affect the host configuration. + + > [!TIP]NOTICE + > In high concurrency scenarios \(200 containers are concurrently started\), the memory management mechanism of Glibc may cause memory holes and large virtual memory \(for example, 10 GB\). This problem is caused by the restriction of the Glibc memory management mechanism in the high concurrency scenario, but not by memory leakage. Therefore, the memory consumption does not increase infinitely. You can set the **MALLOC\_ARENA\_MAX** environment variable to reduce the virtual memory and increase the probability of reducing the physical memory. However, this environment variable will cause the iSulad concurrency performance to deteriorate. Set this environment variable based on the site requirements. + > + > To balance performance and memory usage, set MALLOC_ARENA_MAX to 4. (The iSulad performance deterioration on the ARM64 server is controlled by less than 10%.) + > Configuration method: + > 1. To manually start iSulad, run the export MALLOC_ARENA_MAX=4 command and then start the iSulad. + > 2. If systemd manages iSulad, you can modify the /etc/sysconfig/iSulad file by adding MALLOC_ARENA_MAX=4. + +### Example + +Run a new container. + +```shell +$ isula run -itd busybox +9c2c13b6c35f132f49fb7ffad24f9e673a07b7fe9918f97c0591f0d7014c713b +``` + +## Stopping a Container + +### Description + +To stop a container, run the **isula stop** command. The SIGTERM signal is sent to the first process in the container. If the container is not stopped within the specified time \(10s by default\), the SIGKILL signal is sent. + +### Usage + +```shell +isula stop [OPTIONS] CONTAINER [CONTAINER...] +``` + +### Parameters + +The following table lists the parameters supported by the **stop** command. + +**Table 1** Parameter description + + + + + + + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

stop

+

-f, --force

+

Forcibly stops a running container.

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+

-t, --time

+

Time for graceful stop. If the time exceeds the value of this parameter, the container is forcibly stopped.

+
+ +### Constraints + +- If the **t** parameter is specified and the value of **t** is less than 0, ensure that the application in the container can process the stop signal. + + Principle of the Stop command: Send the SIGTERM signal to the container, and then wait for a period of time \(**t** entered by the user\). If the container is still running after the period of time, the SIGKILL signal is sent to forcibly kill the container. + +- The meaning of the input parameter **t** is as follows: + + **t** < 0: Wait for graceful stop. This setting is preferred when users are assured that their applications have a proper stop signal processing mechanism. + + **t** = 0: Do not wait and send **kill -9** to the container immediately. + + **t** \> 0: Wait for a specified period and send **kill -9** to the container if the container does not stop within the specified period. + + Therefore, if **t** is set to a value less than 0 \(for example, **t** = -1\), ensure that the container application correctly processes the SIGTERM signal. If the container ignores this signal, the container will be suspended when the **isula stop** command is run. + +### Example + +Stop a container. + +```shell +$ isula stop fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1 +fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1 +``` + +## Forcibly Stopping a Container + +### Description + +To forcibly stop one or more running containers, run the **isula kill** command. + +### Usage + +```shell +isula kill [OPTIONS] CONTAINER [CONTAINER...] +``` + +### Parameters + +The following table lists the parameters supported by the **kill** command. + +**Table 1** Parameter description + + + + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

kill

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+

-s, --signal

+

Signal sent to the container.

+
+ +### Example + +Kill a container. + +```shell +$ isula kill fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1 +fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1 +``` + +## Removing a Container + +### Description + +To remove a container, run the **isula rm** command. + +### Usage + +```shell +isula rm [OPTIONS] CONTAINER [CONTAINER...] +``` + +### Parameters + +The following table lists the parameters supported by the **rm** command. + +**Table 1** Parameter description + + + + + + + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

rm

+

-f, --force

+

Forcibly removes a running container.

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+

-v, --volume

+

Removes a volume mounted to a container. (Note: Currently, iSulad does not use this function.)

+
+ +### Constraints + +- In normal I/O scenarios, it takes T1 to delete a running container in an empty environment \(with only one container\). In an environment with 200 containers \(without a large number of I/O operations and with normal host I/O\), it takes T2 to delete a running container. The specification of T2 is as follows: T2 = max \{T1 x 3, 5\}s. + +### Example + +Delete a stopped container. + +```shell +$ isula rm fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1 +fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1 +``` + +## Attaching to a Container + +### Description + +To attach standard input, standard output, and standard error of the current terminal to a running container, run the **isula attach** command. Only containers whose runtime is of the LCR type are supported. + +### Usage + +```shell +isula attach [OPTIONS] CONTAINER +``` + +### Parameters + +The following table lists the parameters supported by the **attach** command. + +**Table 1** Parameter description + + + + + + + + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

attach

+

--help

+

Displays help information.

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+

-D, --debug

+

Enables the debug mode.

+
+ +### Constraints + +- For the native Docker, running the **attach** command will directly enter the container. For the iSulad container, you have to run the **attach** command and press **Enter** to enter the container. + +### Example + +Attach to a running container. + +```shell +$ isula attach fd7376591a9c3d8ee9a14f5d2c2e5255b02cc44cddaabca82170efd4497510e1 +/ # +/ # +``` + +## Renaming a Container + +### Description + +To rename a container, run the **isula rename** command. + +### Usage + +```shell +isula rename [OPTIONS] OLD_NAME NEW_NAME +``` + +### Parameters + +The following table lists the parameters supported by the **rename** command. + +**Table 1** Parameter description + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

rename

+

-H, --host

+

Renames a container.

+
+ +### Example + +Rename a container. + +```shell +isula rename my_container my_new_container +``` + +## Executing a Command in a Running Container + +### Description + +To execute a command in a running container, run the **isula exec** command. This command is executed in the default directory of the container. If a user-defined directory is specified for the basic image, the user-defined directory is used. + +### Usage + +```shell +isula exec [OPTIONS] CONTAINER COMMAND [ARG...] +``` + +### Parameters + +The following table lists the parameters supported by the **exec** command. + +**Table 1** Parameter description + + + + + + + + + + + + + + + + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

exec

+

  

+

-d, --detach

+

Runs a command in the background.

+

-e, --env

+

Sets environment variables. (Note: Currently, iSulad does not use this function.)

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+

-i, --interactive

+

Enables the standard input though no connection is set up. (Note: Currently, iSulad does not use this function.)

+

-t, --tty

+

Allocates a pseudo terminal. (Note: Currently, iSulad does not use this function.)

+

-u, --user

+

Logs in to the container as a specified user.

+
+ +### Constraints + +- If no parameter is specified in the **isula exec** command, the **-it** parameter is used by default, indicating that a pseudo terminal is allocated and the container is accessed in interactive mode. +- When you run the **isula exec** command to execute a script and run a background process in the script, you need to use the **nohup** flag to ignore the **SIGHUP** signal. + + When you run the **isula exec** command to execute a script and run a background process in the script, you need to use the **nohup** flag. Otherwise, the kernel sends the **SIGHUP** signal to the process executed in the background when the process \(first process of the session\) exits. As a result, the background process exits and zombie processes occur. + +- After running the **isula exec** command to access the container process, do not run background programs. Otherwise, the system will be suspended. + + To run the **isula exec** command to execute a background process, perform the following steps: + + 1. Run the **isula exec container\_name bash** command to access the container. + 2. After entering the container, run the **script &** command. + 3. Run the **exit** command. The terminal stops responding. + + After the **isula exec** command is executed to enter the container, the background program stops responding because the **isula exec** command is executed to enter the container and run the background while1 program. When Bash exits, the while1 program does not exit and becomes an orphan process, which is taken over by process 1. + The the while1 process is executed by the initial Bash process **fork &exec** of the container. The while1 process copies the file handle of the Bash process. As a result, the handle is not completely closed when the Bash process exits. The console process cannot receive the handle closing event, epoll_wait stops responding, and the process does not exit. + +- Do not run the **isula exec** command in the background. Otherwise, the system may be suspended. + + Run the **isula exec** command in the background as follows: + + Run the **isula exec script &** command in the background, for example, **isula exec container\_name script &,isula exec**. The command is executed in the background. The script continuously displays a file by running the **cat** command. Normally, there is output on the current terminal. If you press **Enter** on the current terminal, the client exits the stdout read operation due to the I/O read failure. As a result, the terminal does not output data. The server continues to write data to the buffer of the FIFO because the process is still displaying files by running the **cat** command. When the buffer is full, the process in the container is suspended in the write operation. + +- When a lightweight container uses the **exec** command to execute commands with pipe operations, you are advised to run the **/bin/bash -c** command. + + Typical application scenarios: + + Run the **isula exec container\_name -it ls /test | grep "xx" | wc -l** command to count the number of xx files in the test directory. The output is processed by **grep** and **wc** through the pipe because **ls /test** is executed with **exec**. The output of **ls /test** executed by **exec** contains line breaks. When the output is processed, the result is incorrect. + + Cause: Run the **ls /test** command using **exec**. The command output contains a line feed character. Run the**| grep "xx" | wc -l** command for the output. The processing result is 2 \(two lines\). + + ```shell + [root@localhost ~]# isula exec -it container ls /test + xx xx10 xx12 xx14 xx3 xx5 xx7 xx9 + xx1 xx11 xx13 xx2 xx4 xx6 xx8 + [root@localhost ~]# + ``` + + Suggestion: When running the **run/exec** command to perform pipe operations, run the **/bin/bash -c** command to perform pipe operations in the container. + + ```shell + [root@localhost ~]# isula exec -it container /bin/sh -c "ls /test | grep "xx" | wc -l" + 15 + [root@localhost ~]# + ``` + +- Do not use the **echo** option to input data to the standard input of the **exec** command. Otherwise, the client will be suspended. The echo value should be directly transferred to the container as a command line parameter. + + ```shell + [root@localhost ~]# echo ls | isula exec 38 /bin/sh + + + ^C + [root@localhost ~]# + ``` + + The client is suspended when the preceding command is executed because the preceding command is equivalent to input **ls** to **stdin**. Then EOF is read and the client does not send data and waits for the server to exit. However, the server cannot determine whether the client needs to continue sending data. As a result, the server is suspended in reading data, and both parties are suspended. + + The correct execution method is as follows: + + ```shell + [root@localhost ~]# isula exec 38 ls + bin dev etc home proc root sys tmp usr var + ``` + +### Example + +Run the echo command in a running container. + +```shell +$ isula exec c75284634bee echo "hello,world" +hello,world +``` + +## Querying Information About a Single Container + +### Description + +To query information about a single container, run the **isula inspect** command. + +### Usage + +```shell +isula inspect [OPTIONS] CONTAINER|IMAGE [CONTAINER|IMAGE...] +``` + +### Parameters + +The following table lists the parameters supported by the **inspect** command. + +**Table 1** Parameter description + + + + + + + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

inspect

+

  

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+

-f, --format

+

Output format.

+

-t, --time

+

Timeout interval, in seconds. If the inspect command fails to query container information within the specified period, the system stops waiting and reports an error immediately. The default value is 120s. If the value is less than or equal to 0, the inspect command keeps waiting until the container information is obtained successfully.

+
+ +### Constraints + +- Lightweight containers do not support the output in \{ \{.State\} \} format but support the output in the \{ \{json .State\} \} format. The **-f** parameter is not supported when the object is an image. + +### Example + +Query information about a container. + +```shell +$ isula inspect c75284634bee +[ + { + "Id": "c75284634beeede3ab86c828790b439d16b6ed8a537550456b1f94eb852c1c0a", + "Created": "2019-08-01T22:48:13.993304927-04:00", + "Path": "sh", + "Args": [], + "State": { + "Status": "running", + "Running": true, + "Paused": false, + "Restarting": false, + "Pid": 21164, + "ExitCode": 0, + "Error": "", + "StartedAt": "2019-08-02T06:09:25.535049168-04:00", + "FinishedAt": "2019-08-02T04:28:09.479766839-04:00", + "Health": { + "Status": "", + "FailingStreak": 0, + "Log": [] + } + }, + "Image": "busybox", + "ResolvConfPath": "", + "HostnamePath": "", + "HostsPath": "", + "LogPath": "none", + "Name": "c75284634beeede3ab86c828790b439d16b6ed8a537550456b1f94eb852c1c0a", + "RestartCount": 0, + "HostConfig": { + "Binds": [], + "NetworkMode": "", + "GroupAdd": [], + "IpcMode": "", + "PidMode": "", + "Privileged": false, + "SystemContainer": false, + "NsChangeFiles": [], + "UserRemap": "", + "ShmSize": 67108864, + "AutoRemove": false, + "AutoRemoveBak": false, + "ReadonlyRootfs": false, + "UTSMode": "", + "UsernsMode": "", + "Sysctls": {}, + "Runtime": "lcr", + "RestartPolicy": { + "Name": "no", + "MaximumRetryCount": 0 + }, + "CapAdd": [], + "CapDrop": [], + "Dns": [], + "DnsOptions": [], + "DnsSearch": [], + "ExtraHosts": [], + "HookSpec": "", + "CPUShares": 0, + "Memory": 0, + "OomScoreAdj": 0, + "BlkioWeight": 0, + "BlkioWeightDevice": [], + "CPUPeriod": 0, + "CPUQuota": 0, + "CPURealtimePeriod": 0, + "CPURealtimeRuntime": 0, + "CpusetCpus": "", + "CpusetMems": "", + "SecurityOpt": [], + "StorageOpt": {}, + "KernelMemory": 0, + "MemoryReservation": 0, + "MemorySwap": 0, + "OomKillDisable": false, + "PidsLimit": 0, + "FilesLimit": 0, + "Ulimits": [], + "Hugetlbs": [], + "HostChannel": { + "PathOnHost": "", + "PathInContainer": "", + "Permissions": "", + "Size": 0 + }, + "EnvTargetFile": "", + "ExternalRootfs": "" + }, + "Mounts": [], + "Config": { + "Hostname": "localhost", + "User": "", + "Env": [ + "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", + "TERM=xterm", + "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" + ], + "Tty": true, + "Cmd": [ + "sh" + ], + "Entrypoint": [], + "Labels": {}, + "Annotations": { + "log.console.file": "none", + "log.console.filerotate": "7", + "log.console.filesize": "1MB", + "rootfs.mount": "/var/lib/isulad/mnt/rootfs", + "native.umask": "secure" + }, + "HealthCheck": { + "Test": [], + "Interval": 0, + "Timeout": 0, + "StartPeriod": 0, + "Retries": 0, + "ExitOnUnhealthy": false + } + }, + "NetworkSettings": { + "IPAddress": "" + } + } +] +``` + +## Querying Information About All Containers + +### Description + +To query information about all containers, run the **isula ps** command. + +### Usage + +```shell +isula ps [OPTIONS] +``` + +### Parameters + +The following table lists the parameters supported by the **ps** command. + +**Table 1** Parameter description + + + + + + + + + + + + + + + + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

ps

+

  

+

  

+

  

+

  

+

-a, --all

+

Displays all containers.

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+

-q, --quiet

+

Displays only the container name.

+

-f, --filter

+

Adds filter criteria.

+

--format

+

Output format.

+

--no-trunc

+

Do not truncate the container ID.

+
+ +### Example + +Query information about all containers. + +```shell +$ isula ps -a + +ID IMAGE STATUS PID COMMAND EXIT_CODE RESTART_COUNT STARTAT FINISHAT RUNTIME NAMES +e84660aa059c rnd-dockerhub.huawei.com/official/busybox running 304765 "sh" 0 0 13 minutes ago - lcr e84660aa059cafb0a77a4002e65cc9186949132b8e57b7f4d76aa22f28fde016 +$ isula ps -a --format "table {{.ID}} {{.Image}}" --no-trunc +ID IMAGE +e84660aa059cafb0a77a4002e65cc9186949132b8e57b7f4d76aa22f28fde016 rnd-dockerhub.huawei.com/official/busybox +``` + +## Restarting a Container + +### Description + +To restart one or more containers, run the **isula restart** command. + +### Usage + +```shell +isula restart [OPTIONS] CONTAINER [CONTAINER...] +``` + +### Parameters + +The following table lists the parameters supported by the **restart** command. + +**Table 1** Parameter description + + + + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

restart

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+

-t, --time

+

Time for graceful stop. If the time exceeds the value of this parameter, the container is forcibly stopped.

+
+ +### Constraints + +- If the **t** parameter is specified and the value of **t** is less than 0, ensure that the application in the container can process the stop signal. + + The restart command first calls the stop command to stop the container. Send the SIGTERM signal to the container, and then wait for a period of time \(**t** entered by the user\). If the container is still running after the period of time, the SIGKILL signal is sent to forcibly kill the container. + +- The meaning of the input parameter **t** is as follows: + + **t** < 0: Wait for graceful stop. This setting is preferred when users are assured that their applications have a proper stop signal processing mechanism. + + **t** = 0: Do not wait and send **kill -9** to the container immediately. + + **t** \> 0: Wait for a specified period and send **kill -9** to the container if the container does not stop within the specified period. + + Therefore, if **t** is set to a value less than 0 \(for example, **t** = -1\), ensure that the container application correctly processes the SIGTERM signal. If the container ignores this signal, the container will be suspended when the **isula stop** command is run. + +### Example + +Restart a container. + +```shell +$ isula restart c75284634beeede3ab86c828790b439d16b6ed8a537550456b1f94eb852c1c0a + c75284634beeede3ab86c828790b439d16b6ed8a537550456b1f94eb852c1c0a +``` + +## Waiting for a Container to Exit + +### Description + +To wait for one or more containers to exit, run the **isula wait** command. Only containers whose runtime is of the LCR type are supported. + +### Usage + +```shell +isula wait [OPTIONS] CONTAINER [CONTAINER...] +``` + +### Parameters + +The following table lists the parameters supported by the **wait** command. + +**Table 1** Parameter description + + + + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

wait

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+

/

+

Blocks until the container stops and displays the exit code.

+
+ +### Example + +Wait for a single container to exit. + +```shell +$ isula wait c75284634beeede3ab86c828790b439d16b6ed8a537550456b1f94eb852c1c0a + 137 +``` + +## Viewing Process Information in a Container + +### Description + +To view process information in a container, run the **isula top** command. Only containers whose runtime is of the LCR type are supported. + +### Usage + +```shell +isula top [OPTIONS] container [ps options] +``` + +### Parameters + +The following table lists the parameters supported by the **top** command. + +**Table 1** Parameter description + + + + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

top

+

  

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+

/

+

Queries the process information of a running container.

+
+ +### Example + +Query process information in a container. + +```shell +$ isula top 21fac8bb9ea8e0be4313c8acea765c8b4798b7d06e043bbab99fc20efa72629c +UID PID PPID C STIME TTY TIME CMD +root 22166 22163 0 23:04 pts/1 00:00:00 sh +``` + +## Displaying Resource Usage Statistics of a Container + +### Description + +To display resource usage statistics in real time, run the **isula stats** command. Only containers whose runtime is of the LCR type are supported. + +### Usage + +```shell +isula stats [OPTIONS] [CONTAINER...] +``` + +### Parameters + +The following table lists the parameters supported by the **stats** command. + +**Table 1** Parameter description + + + + + + + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

stats

+

  

+

  

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+

-a, --all

+

Displays all containers. (By default, only running containers are displayed.)

+

--no-stream

+

Display the first result only. Only statistics in non-stream mode are displayed.

+
+ +### Example + +Display resource usage statistics. + +```shell +$ isula stats --no-stream 21fac8bb9ea8e0be4313c8acea765c8b4798b7d06e043bbab99fc20efa72629c CONTAINER CPU % MEM USAGE / LIMIT MEM % BLOCK I / O PIDS +21fac8bb9ea8 0.00 56.00 KiB / 7.45 GiB 0.00 0.00 B / 0.00 B 1 +``` + +## Obtaining Container Logs + +### Description + +To obtain container logs, run the **isula logs** command. Only containers whose runtime is of the LCR type are supported. + +### Usage + +```shell +isula logs [OPTIONS] [CONTAINER...] +``` + +### Parameters + +The following table lists the parameters supported by the **logs** command. + +**Table 1** Parameter description + + + + + + + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

logs

+

  

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+

-f, --follow

+

Traces log output.

+

--tail

+

Displays the number of log records.

+
+ +### Constraints + +- By default, the container log function is enabled. To disable this function, run the **isula create --log-opt disable-log=true** or **isula run --log-opt disable-log=true** command. + +### Example + +Obtain container logs. + +```shell +$ isula logs 6a144695f5dae81e22700a8a78fac28b19f8bf40e8827568b3329c7d4f742406 +hello, world +hello, world +hello, world +``` + +## Copying Data Between a Container and a Host + +### Description + +To copy data between a host and a container, run the **isula cp** command. Only containers whose runtime is of the LCR type are supported. + +### Usage + +```shell +isula cp [OPTIONS] CONTAINER:SRC_PATH DEST_PATH +isula cp [OPTIONS] SRC_PATH CONTAINER:DEST_PATH +``` + +### Parameters + +The following table lists the parameters supported by the **cp** command. + +**Table 1** Parameter description + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

cp

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+
+ +### Constraints + +- When iSulad copies files, note that the **/etc/hostname**, **/etc/resolv.conf**, and **/etc/hosts** files are not mounted to the host, neither the **--volume** and **--mount** parameters. Therefore, the original files in the image instead of the files in the real container are copied. + + ```shell + [root@localhost tmp]# isula cp b330e9be717a:/etc/hostname /tmp/hostname + [root@localhost tmp]# cat /tmp/hostname + [root@localhost tmp]# + ``` + +- When decompressing a file, iSulad does not check the type of the file or folder to be overwritten in the file system. Instead, iSulad directly overwrites the file or folder. Therefore, if the source is a folder, the file with the same name is forcibly overwritten as a folder. If the source file is a file, the folder with the same name will be forcibly overwritten as a file. + + ```shell + [root@localhost tmp]# rm -rf /tmp/test_file_to_dir && mkdir /tmp/test_file_to_dir + [root@localhost tmp]# isula exec b330e9be717a /bin/sh -c "rm -rf /tmp/test_file_to_dir && touch /tmp/test_file_to_dir" + [root@localhost tmp]# isula cp b330e9be717a:/tmp/test_file_to_dir /tmp + [root@localhost tmp]# ls -al /tmp | grep test_file_to_dir + -rw-r----- 1 root root 0 Apr 26 09:59 test_file_to_dir + ``` + +- iSulad freezes the container during the copy process and restores the container after the copy is complete. + +### Example + +Copy the **/test/host** directory on the host to the **/test** directory on container 21fac8bb9ea8. + +```shell +isula cp /test/host 21fac8bb9ea8:/test +``` + +Copy the **/www** directory on container 21fac8bb9ea8 to the **/tmp** directory on the host. + +```shell +isula cp 21fac8bb9ea8:/www /tmp/ +``` + +## Pausing a Container + +### Description + +To pause all processes in a container, run the **isula pause** command. Only containers whose runtime is of the LCR type are supported. + +### Usage + +```shell +isula pause CONTAINER [CONTAINER...] +``` + +### Parameters + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

pause

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+
+ +### Constraints + +- Only containers in the running state can be paused. +- After a container is paused, other lifecycle management operations \(such as **restart**, **exec**, **attach**, **kill**, **stop**, and **rm**\) cannot be performed. +- After a container with health check configurations is paused, the container status changes to unhealthy. + +### Example + +Pause a running container. + +```shell +$ isula pause 8fe25506fb5883b74c2457f453a960d1ae27a24ee45cdd78fb7426d2022a8bac + 8fe25506fb5883b74c2457f453a960d1ae27a24ee45cdd78fb7426d2022a8bac +``` + +## Resuming a Container + +### Description + +To resume all processes in a container, run the **isula unpause** command. It is the reverse process of **isula pause**. Only containers whose runtime is of the LCR type are supported. + +### Usage + +```shell +isula unpause CONTAINER [CONTAINER...] +``` + +### Parameters + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

pause

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+
+ +### Constraints + +- Only containers in the paused state can be unpaused. + +### Example + +Resume a paused container. + +```shell +$ isula unpause 8fe25506fb5883b74c2457f453a960d1ae27a24ee45cdd78fb7426d2022a8bac + 8fe25506fb5883b74c2457f453a960d1ae27a24ee45cdd78fb7426d2022a8bac +``` + +## Obtaining Event Messages from the Server in Real Time + +### **Description** + +The **isula events** command is used to obtain event messages such as container image lifecycle and running event from the server in real time. Only containers whose runtime type is **lcr** are supported. + +### Usage + +```shell +isula events [OPTIONS] +``` + +### Parameter + + + + + + + + + + + + + + + + + + +

Command

+

Parameter

+

Description

+

events

+

-H, --host

+

Specifies the iSulad socket file path to be accessed.

+

-n, --name

+

Obtains event messages of a specified container.

+

-S, --since

+

Obtains event messages generated since a specified time.

+
+ +### Example + +Run the following command to obtain event messages from the server in real time: + +```shell +isula events +``` diff --git a/docs/en/cloud/container_engine/isula_container_engine/container-resource-management.md b/docs/en/cloud/container_engine/isula_container_engine/container-resource-management.md new file mode 100644 index 0000000000000000000000000000000000000000..bf2af73d84364711b749b9ab24a25237bbd67a9e --- /dev/null +++ b/docs/en/cloud/container_engine/isula_container_engine/container-resource-management.md @@ -0,0 +1,720 @@ +# Container Resource Management + +- [Container Resource Management](#container-resource-management) + - [Sharing Resources](#sharing-resources) + - [Restricting CPU Resources of a Running Container](#restricting-cpu-resources-of-a-running-container) + - [Restricting the Memory Usage of a Running Container](#restricting-the-memory-usage-of-a-running-container) + - [Restricting I/O Resources of a Running Container](#restricting-io-resources-of-a-running-container) + - [Restricting the Rootfs Storage Space of a Container](#restricting-the-rootfs-storage-space-of-a-container) + - [Restricting the Number of File Handles in a Container](#restricting-the-number-of-file-handles-in-a-container) + - [Restricting the Number of Processes or Threads that Can Be Created in a Container](#restricting-the-number-of-processes-or-threads-that-can-be-created-in-a-container) + - [Configuring the ulimit Value in a Container](#configuring-the-ulimit-value-in-a-container) + +## Sharing Resources + +### Description + +Containers or containers and hosts can share namespace information mutually, including PID, network, IPC, and UTS information. + +### Usage + +When running the **isula create/run** command, you can set the namespace parameters to share resources. For details, see the following parameter description table. + +### Parameters + +You can specify the following parameters when running the **isula create/run** command: + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Parameter

+

Description

+

Value Range

+

Mandatory or Not

+

--pid

+

Specifies the PID namespace to be shared.

+

[none, host, container:<containerID>]: none indicates that the namespace is not shared. host indicates that the namespace is shared with the host. container:<containerID> indicates that the namespace is shared with the container.

+

No

+

--net

+

Specifies the network namespace to be shared.

+

[none, host, container:<containerID>]: none indicates that the namespace is not shared. host indicates that the namespace is shared with the host. container:<containerID> indicates that the namespace is shared with the container.

+

No

+

--ipc

+

Specifies the IPC namespace to be shared.

+

[none, host, container:<containerID>]: none indicates that the namespace is not shared. host indicates that the namespace is shared with the host. container:<containerID> indicates that the namespace is shared with the container.

+

No

+

--uts

+

Specifies the UTS namespace to be shared.

+

[none, host, container:<containerID>]: none indicates that the namespace is not shared. host indicates that the namespace is shared with the host. container:<containerID> indicates that the namespace is shared with the container.

+

No

+
+ +### Example + +If two containers need to share the same PID namespace, add **--pid container:** when running the container. For example: + +```shell +isula run -tid --name test_pid busybox sh +isula run -tid --name test --pid container:test_pid busybox sh +``` + +## Restricting CPU Resources of a Running Container + +### Description + +You can set parameters to restrict the CPU resources of a container. + +### Usage + +When running the **isula create/run** command, you can set CPU-related parameters to limit the CPU resources of a container. For details about the parameters and values, see the following table. + +### Parameters + +You can specify the following parameters when running the **isula create/run** command: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Parameter

+

Description

+

Value Range

+

Mandatory or Not

+

--cpu-period

+

Limits the CPU CFS period in a container.

+

64-bit integer

+

No

+

--cpu-quota

+

Limits the CPU CFS quota.

+

64-bit integer

+

No

+

--cpu-shares

+

Limits the CPU share (relative weight).

+

64-bit integer

+

No

+

--cpuset-cpus

+

Limits the CPU nodes.

+

A character string. The value is the number of CPUs to be configured. The value ranges from 0 to 3, or 0 and 1.

+

No

+

--cpuset-mems

+

Limits the memory nodes used by cpuset in the container.

+

A character string. The value is the number of CPUs to be configured. The value ranges from 0 to 3, or 0 and 1.

+

No

+
+ +### Example + +To restrict a container to use a specific CPU, add **--cpuset-cpus number** when running the container. For example: + +```shell +isula run -tid --cpuset-cpus 0,2-3 busybox sh +``` + +>[!NOTE]NOTE +>You can check whether the configuration is successful. For details, see "Querying Information About a Single Container." + +## Restricting the Memory Usage of a Running Container + +### Description + +You can set parameters to restrict the memory usage of a container. + +### Usage + +When running the **isula create/run** command, you can set memory-related parameters to restrict memory usage of containers. For details about the parameters and values, see the following table. + +### Parameters + +You can specify the following parameters when running the **isula create/run** command: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Parameter

+

Description

+

Value Range

+

Mandatory or Not

+

--memory

+

Specifies the upper limit of the memory usage of a container.

+

64-bit integer The value is a non-negative number. The value 0 indicates that no limit is set. The unit can be empty (byte), KB, MB, GB, TB, or PB.

+

No

+

--memory-reservation

+

Specifies the soft upper limit of the memory of a container.

+

64-bit integer The value is a non-negative number. The value 0 indicates that no limit is set. The unit can be empty (byte), KB, MB, GB, TB, or PB.

+

No

+

--memory-swap

+

Specifies the upper limit of the swap memory of the container.

+

64-bit integer The value can be -1 or a non-negative number. The value -1 indicates no limit, and the value 0 indicates that no limit is set. The unit can be empty (byte), KB, MB, GB, TB, or PB.

+

No

+

--kernel-memory

+

Specifies the upper limit of the kernel memory of the container.

+

64-bit integer The value is a non-negative number. The value 0 indicates that no limit is set. The unit can be empty (byte), KB, MB, GB, TB, or PB.

+

No

+
+ +### Example + +To set the upper limit of the memory of a container, add **--memory \[\]** when running the container. For example: + +```shell +isula run -tid --memory 1G busybox sh +``` + +## Restricting I/O Resources of a Running Container + +### Description + +You can set parameters to limit the read/write speed of devices in the container. + +### Usage + +When running the **isula create/run** command, you can set **--device-read-bps/--device-write-bps :\[\]** to limit the read/write speed of devices in the container. + +### Parameters + +When running the **isula create/run** command, set **--device-read/write-bps**. + + + + + + + + + + + + + + +

Parameter

+

Description

+

Value Range

+

Mandatory or Not

+

--device-read-bps/--device-write-bps

+

Limits the read/write speed of devices in the container.

+

64-bit integer The value is a positive integer. The value can be 0, indicating that no limit is set. The unit can be empty (byte), KB, MB, GB, TB, or PB.

+

No

+
+ +### Example + +To limit the read/write speed of devices in the container, add **--device-write-bps/--device-read-bps :\[\]** when running the container. For example, to limit the read speed of the device **/dev/sda** in the container **busybox** to 1 Mbit/s, run the following command: + +```shell +isula run -tid --device-write /dev/sda:1mb busybox sh +``` + +To limit the write speed, run the following command: + +```shell +isula run -tid read-bps /dev/sda:1mb busybox sh +``` + +## Restricting the Rootfs Storage Space of a Container + +### Description + +When the overlay2 storage driver is used on the EXT4 file system, the file system quota of a single container can be set. For example, the quota of container A is set to 5 GB, and the quota of container B is set to 10 GB. + +This feature is implemented by the project quota function of the EXT4 file system. If the kernel supports this function, use the syscall SYS\_IOCTL to set the project ID of a directory, and then use the syscall SYS\_QUOTACTL to set the hard limit and soft limit of the corresponding project ID. + +### Usage + +1. Prepare the environment. + + Ensure that the file system supports the **Project ID** and **Project Quota** attributes, the kernel version is 4.19 or later, and the version of the peripheral package e2fsprogs is 1.43.4-2 or later. + +2. Before mounting overlayfs to a container, set different project IDs for the upper and work directories of different containers and set inheritance options. After overlayfs is mounted to a container, the project IDs and inherited attributes cannot be modified. +3. Set the quota as a privileged user outside the container. +4. Add the following configuration to daemon: + + ```shell + -s overlay2 --storage-opt overlay2.override_kernel_check=true + ``` + +5. Daemon supports the following options for setting default restrictions for containers: + + **--storage-opt overlay2.basesize=128M** specifies the default limit. If **--storeage-opt size** is also specified when you run the **isula run** command, the value of this parameter takes effect. If no size is specified during the daemon process or when you run the **isula run** command, the size is not limited. + +6. Enable the **Project ID** and **Project Quota** attributes of the file system. + - Format and mount the file system. + + ```shell + # mkfs.ext4 -O quota,project /dev/sdb + # mount -o prjquota /dev/sdb /var/lib/isulad + ``` + +### Parameters + +When running the **create/run** command, set **--storage-opt**. + + + + + + + + + + + + + + +

Parameter

+

Description

+

Value Range

+

Mandatory or Not

+

--storage-opt size=${rootfsSize}

+

Restricts the root file system (rootfs) storage space of the container.

+

The size parsed by rootfsSize is a positive 64-bit integer expressed in bytes. You can also set it to ([kKmMgGtTpP])?[iI]?[bB]?$.

+

No

+
+ +### Example + +In the **isula run/create** command, use the existing parameter **--storage-opt size=**_value_ to set the quota. The value is a positive number in the unit of **\[kKmMgGtTpP\]?\[iI\]?\[bB\]?**. If the value does not contain a unit, the default unit is byte. + +```shell +$ [root@localhost ~]# isula run -ti --storage-opt size=10M busybox +/ # df -h +Filesystem Size Used Available Use% Mounted on +overlay 10.0M 48.0K 10.0M 0% / +none 64.0M 0 64.0M 0% /dev +none 10.0M 0 10.0M 0% /sys/fs/cgroup +tmpfs 64.0M 0 64.0M 0% /dev +shm 64.0M 0 64.0M 0% /dev/shm +/dev/mapper/vg--data-ext41 + 9.8G 51.5M 9.2G 1% /etc/hostname +/dev/mapper/vg--data-ext41 + 9.8G 51.5M 9.2G 1% /etc/resolv.conf +/dev/mapper/vg--data-ext41 + 9.8G 51.5M 9.2G 1% /etc/hosts +tmpfs 3.9G 0 3.9G 0% /proc/acpi +tmpfs 64.0M 0 64.0M 0% /proc/kcore +tmpfs 64.0M 0 64.0M 0% /proc/keys +tmpfs 64.0M 0 64.0M 0% /proc/timer_list +tmpfs 64.0M 0 64.0M 0% /proc/sched_debug +tmpfs 3.9G 0 3.9G 0% /proc/scsi +tmpfs 64.0M 0 64.0M 0% /proc/fdthreshold +tmpfs 64.0M 0 64.0M 0% /proc/fdenable +tmpfs 3.9G 0 3.9G 0% /sys/firmware +/ # +/ # dd if=/dev/zero of=/home/img bs=1M count=12 && sync +dm-4: write failed, project block limit reached. +10+0 records in +9+0 records out +10432512 bytes (9.9MB) copied, 0.011782 seconds, 844.4MB/s +/ # df -h | grep overlay +overlay 10.0M 10.0M 0 100% / +/ # +``` + +### Constraints + +1. The quota applies only to the rw layer. + + The quota of overlay2 is for the rw layer of the container. The image size is not included. + +2. The kernel supports and enables this function. + + The kernel must support the EXT4 project quota function. When running **mkfs**, add **-O quota,project**. When mounting the file system, add **-o prjquota**. If any of the preceding conditions is not met, an error is reported when **--storage-opt size=**_value_ is used. + + ```shell + $ [root@localhost ~]# isula run -it --storage-opt size=10Mb busybox df -h + Error response from daemon: Failed to prepare rootfs with error: time="2019-04-09T05:13:52-04:00" level=fatal msg="error creating read- + write layer with ID "a4c0e55e82c55e4ee4b0f4ee07f80cc2261cf31b2c2dfd628fa1fb00db97270f": --storage-opt is supported only for overlay over + xfs or ext4 with 'pquota' mount option" + ``` + +3. Description of the limit of quota: + 1. If the quota is greater than the size of the partition where user **root** of iSulad is located, the file system quota displayed by running the **df** command in the container is the size of the partition where user **root** of iSulad is located, not the specified quota. + 2. **--storage-opt size=0** indicates that the size is not limited and the value cannot be less than 4096. The precision of size is one byte. If the specified precision contains decimal bytes, the decimal part is ignored. For example, if size is set to **0.1**, the size is not limited. \(The value is restricted by the precision of the floating point number stored on the computer. That is, 0.999999999999999999999999999 is equal to 1. The number of digits 9 may vary according to computers. Therefore, 4095.999999999999999999999999999 is equal to 4096.\) Note that running **isula inspect** displays the original command line specified format. If the value contains decimal bytes, you need to ignore the decimal part. + 3. If the quota is too small, for example,**--storage-opt size=4k**, the container may fail to be started because some files need to be created for starting the container. + 4. The **-o prjquota** option is added to the root partition of iSulad when iSulad is started last time. If this option is not added during this startup, the setting of the container with quota created during the last startup does not take effect. + 5. The value range of the daemon quota **--storage-opt overlay2.basesize** is the same as that of **--storage-opt size**. + +4. When **storage-opt** is set to 4 KB, the lightweight container startup is different from that of Docker. + + Use the **storage-opt size=4k** and image **rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest** to run the container. + + Docker fails to be started. + + ```shell + [root@localhost ~]# docker run -itd --storage-opt size=4k rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest + docker: Error response from daemon: symlink /proc/mounts /var/lib/docker/overlay2/e6e12701db1a488636c881b44109a807e187b8db51a50015db34a131294fcf70-init/merged/etc/mtab: disk quota exceeded. + See 'docker run --help'. + ``` + + The lightweight container is started properly and no error is reported. + + ```shell + [root@localhost ~]# isula run -itd --storage-opt size=4k rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest + 636480b1fc2cf8ac895f46e77d86439fe2b359a1ff78486ae81c18d089bbd728 + [root@localhost ~]# isula ps + STATUS PID IMAGE COMMAND EXIT_CODE RESTART_COUNT STARTAT FINISHAT RUNTIME ID NAMES + running 17609 rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest /bin/bash 0 0 2 seconds ago - lcr 636480b1fc2c 636480b1fc2cf8ac895f46e77d86439fe2b359a1ff78486ae81c18d089bbd728 + ``` + + During container startup, if you need to create a file in the **rootfs** directory of the container, the image size exceeds 4 KB, and the quota is set to 4 KB, the file creation will fail. + + When Docker starts the container, it creates more mount points than iSulad to mount some directories on the host to the container, such as **/proc/mounts** and **/dev/shm**. If these files do not exist in the image, the creation will fail, therefore, the container fails to be started. + + When a lightweight container uses the default configuration during container startup, there are few mount points. The lightweight container is created only when the directory like **/proc** or **/sys** does not exist. The image **rnd-dockerhub.huawei.com/official/ubuntu-arm64:latest** in the test case contains **/proc** and **/sys**. Therefore, no new file or directory is generated during the container startup. As a result, no error is reported during the lightweight container startup. To verify this process, when the image is replaced with **rnd-dockerhub.huawei.com/official/busybox-aarch64:latest**, an error is reported when the lightweight container is started because **/proc** does not exist in the image. + + ```shell + [root@localhost ~]# isula run -itd --storage-opt size=4k rnd-dockerhub.huawei.com/official/busybox-aarch64:latest + 8e893ab483310350b8caa3b29eca7cd3c94eae55b48bfc82b350b30b17a0aaf4 + Error response from daemon: Start container error: runtime error: 8e893ab483310350b8caa3b29eca7cd3c94eae55b48bfc82b350b30b17a0aaf4:tools/lxc_start.c:main:404 starting container process caused "Failed to setup lxc, + please check the config file." + ``` + +5. Other description: + + When using iSulad with the quota function to switch data drives, ensure that the data drives to be switched are mounted using the **prjquota** option and the mounting mode of the **/var/lib/isulad/storage/overlay2** directory is the same as that of the **/var/lib/isulad** directory. + + > [!NOTE]NOTE + > Before switching the data drive, ensure that the mount point of **/var/lib/isulad/storage/overlay2** is unmounted. + +## Restricting the Number of File Handles in a Container + +### Description + +You can set parameters to limit the number of file handles that can be opened in a container. + +### Usage + +When running the **isula create/run** command, set the **--files-limit** parameter to limit the number of file handles that can be opened in a container. + +### Parameters + +Set the **--files-limit** parameter when running the **isula create/run** command. + + + + + + + + + + + + + + +

Parameter

+

Description

+

Value Range

+

Mandatory or Not

+

--files-limit

+

Limits the number of file handles that can be opened in a container.

+

64-bit integer The value can be 0 or a negative number, but cannot be greater than 2 to the power of 63 minus 1. The value 0 or a negative number indicates no limit.

+

During container creation, some handles are opened temporarily. Therefore, the value cannot be too small. Otherwise, the container may not be restricted by the file limit. If the value is less than the number of opened handles, the cgroup file cannot be written. It is recommended that the value be greater than 30.

+

No

+
+ +### Example + +When running the container, add **--files-limit n**. For example: + +```shell +isula run -ti --files-limit 1024 busybox bash +``` + +### Constraints + +1. If the **--files-limit** parameter is set to a small value, for example, 1, the container may fail to be started. + + ```shell + [root@localhost ~]# isula run -itd --files-limit 1 rnd-dockerhub.huawei.com/official/busybox-aarch64 + 004858d9f9ef429b624f3d20f8ba12acfbc8a15bb121c4036de4e5745932eff4 + Error response from daemon: Start container error: Container is not running:004858d9f9ef429b624f3d20f8ba12acfbc8a15bb121c4036de4e5745932eff4 + ``` + + Docker will be started successfully, and the value of **files.limit cgroup** is **max**. + + ```shell + [root@localhost ~]# docker run -itd --files-limit 1 rnd-dockerhub.huawei.com/official/busybox-aarch64 + ef9694bf4d8e803a1c7de5c17f5d829db409e41a530a245edc2e5367708dbbab + [root@localhost ~]# docker exec -it ef96 cat /sys/fs/cgroup/files/files.limit + max + ``` + + The root cause is that the startup principles of the lxc and runc processes are different. After the lxc process creates the cgroup, the files.limit value is set, and then the PID of the container process is written into the cgroup.procs file of the cgroup. At this time, the process has opened more than one handle. As a result, an error is reported, and the startup fails. After you create a cgroup by running the **runc** command, the PID of the container process is written to the cgroup.procs file of the cgroup, and then the files.limit value is set. Because more than one handle is opened by the process in the cgroup, the file.limit value does not take effect, the kernel does not report any error, and the container is started successfully. + +## Restricting the Number of Processes or Threads that Can Be Created in a Container + +### Description + +You can set parameters to limit the number of processes or threads that can be created in a container. + +### Usage + +When creating or running a container, use the **--pids-limit** parameter to limit the number of processes or threads that can be created in the container. + +### Parameters + +When running the **create/run** command, set the **--pids-limit** parameter. + + + + + + + + + + + + + + +

Parameter

+

Description

+

Value Range

+

Mandatory or Not

+

--pids-limit

+

Limits the number of file handles that can be opened in a container.

+

64-bit integer The value can be 0 or a negative number, but cannot be greater than 2 to the power of 63 minus 1. The value 0 or a negative number indicates no limit.

+

No

+
+ +### Example + +When running the container, add **--pids-limit n**. For example: + +```shell +isula run -ti --pids-limit 1024 busybox bash +``` + +### Constraints + +During container creation, some processes are created temporarily. Therefore, the value cannot be too small. Otherwise, the container may fail to be started. It is recommended that the value be greater than 10. + +## Configuring the ulimit Value in a Container + +### Description + +You can use parameters to control the resources for executed programs. + +### Usage + +Set the **--ulimit** parameter when creating or running a container, or configure the parameter on the daemon to control the resources for executed programs in the container. + +### Parameters + +Use either of the following methods to configure ulimit: + +1. When running the **isula create/run** command, use **--ulimit =\[:\]** to control the resources of the executed shell program. + + + + + + + + + + + + + + +

Parameter

+

Description

+

Value Range

+

Mandatory or Not

+

--ulimit

+

Limits the resources of the executed shell program.

+

64-bit integer The value of the soft limit must be less than or equal to that of the hard limit. If only the soft limit is specified, the value of the hard limit is equal to that of the soft limit. Some types of resources do not support negative numbers. For details, see the following table.

+

No

+
+ +2. Use daemon parameters or configuration files. + + For details, see **--default-ulimits** in [Configuration Mode](installation-configuration.md#configuration-mode). + + **--ulimit** can limit the following types of resources: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Type

+

Description

+

Value Range

+

core

+

limits the core file size (KB)

+

64-bit integer, without unit. The value can be 0 or a negative number. The value -1 indicates no limit. Other negative numbers are forcibly converted into a large positive integer.

+

cpu

+

max CPU time (MIN)

+

data

+

max data size (KB)

+

fsize

+

maximum filesize (KB)

+

locks

+

max number of file locks the user can hold

+

memlock

+

max locked-in-memory address space (KB)

+

msgqueue

+

max memory used by POSIX message queues (bytes)

+

nice

+

nice priority

+

nproc

+

max number of processes

+

rss

+

max resident set size (KB)

+

rtprio

+

max realtime priority

+

rttime

+

realtime timeout

+

sigpending

+

max number of pending signals

+

stack

+

max stack size (KB)

+

nofile

+

max number of open file descriptors

+

64-bit integer, without unit. The value cannot be negative. A negative number is forcibly converted to a large positive number. In addition, "Operation not permitted" is displayed during the setting.

+
+ +### Example + +When creating or running a container, add **--ulimit =\[:\]**. For example: + +```shell +isula create/run -tid --ulimit nofile=1024:2048 busybox sh +``` + +### Constraints + +The ulimit cannot be configured in the **daemon.json** and **/etc/sysconfig/iSulad** files \(or the iSulad command line\). Otherwise, an error is reported when iSulad is started. diff --git a/docs/en/cloud/container_engine/isula_container_engine/cri-2.md b/docs/en/cloud/container_engine/isula_container_engine/cri-2.md new file mode 100644 index 0000000000000000000000000000000000000000..331628c7cc0438402e033d29b0005359fe793216 --- /dev/null +++ b/docs/en/cloud/container_engine/isula_container_engine/cri-2.md @@ -0,0 +1,205 @@ +# CRI API v1 + +## Overview + +Container Runtime Interface (CRI) is the main protocol used by kublet to communicate with container engines. +Kubernetes 1.25 and earlier versions support CRI v1alpha2 and CRI v1. Kubernetes 1.26 and later versions support only CRI v1. + +iSulad supports both [CRI v1alpha2](cri.md) and CRI v1. +For CRI v1, iSulad supports the functions described in [CRI v1alpha2](cri.md) and new interfaces and fields defined in CRI v1. + +Currently, iSulad supports CRI v1 1.29. The API described on the official website is as follows: + +[https://github.com/kubernetes/cri-api/blob/kubernetes-1.29.0/pkg/apis/runtime/v1/api.proto](https://github.com/kubernetes/cri-api/blob/kubernetes-1.29.0/pkg/apis/runtime/v1/api.proto) + +The API description file used by iSulad is slightly different from the official API. The interfaces in this document prevail. + +## New Fields of CRI v1 + +- **CgroupDriver** + + Enum values for cgroup drivers. + + | Member | Description | + | ------------ | --------------------- | + | SYSTEMD = 0 | systemd-cgroup driver | + | CGROUPFS = 1 | cgroupfs driver | + +- **LinuxRuntimeConfiguration** + + cgroup driver used by the container engine + + | Member | Description | + | -------------------------- | ------------------------------------------------------------- | + | CgroupDriver cgroup_driver | Enum value for the cgroup driver used by the container engine | + +- **ContainerEventType** + + Enum values for container event types + + | Member | Description | + | --------------------------- | ------------------------ | + | CONTAINER_CREATED_EVENT = 0 | Container creation event | + | CONTAINER_STARTED_EVENT = 1 | Container startup event | + | CONTAINER_STOPPED_EVENT = 1 | Container stop event | + | CONTAINER_DELETED_EVENT = 1 | Container deletion event | + +- **SwapUsage** + + Virtual memory usage + + | Member | Description | + | -------------------------------- | ------------------------------ | + | int64 timestamp | Timestamp information | + | UInt64Value swap_available_bytes | Available virtual memory bytes | + | UInt64Value swap_usage_bytes | Used virtual memory bytes | + +## New Interfaces + +### RuntimeConfig + +#### Interface Prototype + +```text +rpc RuntimeConfig(RuntimeConfigRequest) returns (RuntimeConfigResponse) {} +``` + +#### Interface Description + +Obtains the cgroup driver configuration (cgroupfs or systemd-cgroup). + +#### Parameter: RuntimeConfigRequest + +No such field + +#### Returns: RuntimeConfigResponse + +| Return | Description | +| ------------------------------- | ------------------------------------------------------ | +| LinuxRuntimeConfiguration linux | CgroupDriver enum value for cgroupfs or systemd-cgroup | + +### GetContainerEvents + +#### Interface Prototype + +```text +rpc GetContainerEvents(GetEventsRequest) returns (stream ContainerEventResponse) {} +``` + +#### Interface Description + +Obtains the pod lifecycle event stream. + +#### Parameter: GetEventsRequest + +No such field + +#### Returns: ContainerEventResponse + +| Return | Description | +| -------------------------------------------- | ------------------------------------------------------------------ | +| string container_id | Container ID | +| ContainerEventType container_event_type | Container event type | +| int64 created_at | Time when the container event is generated | +| PodSandboxStatus pod_sandbox_status | Status of the pod to which the container belongs | +| repeated ContainerStatus containers_statuses | Status of all containers in the pod to which the container belongs | + +## Change Description + +### CRI V1.29 + +#### [Obtaining the cgroup Driver Configuration](https://github.com/kubernetes/kubernetes/pull/118770) + +`RuntimeConfig` obtains the cgroup driver configuration (cgroupfs or systemd-cgroup). + +#### [GetContainerEvents Supports Pod Lifecycle Events](https://github.com/kubernetes/kubernetes/pull/111384) + +`GetContainerEvents` provides event streams related to the pod lifecycle. + +`PodSandboxStatus` is adjusted accordingly. `ContainerStatuses` is added to provide sandbox content status information. + +#### [ContainerStats Virtual Memory Information](https://github.com/kubernetes/kubernetes/pull/118865) + +The virtual memory usage information `SwapUsage` is added to `ContainerStats`. + +#### [OOMKilled Setting in the Reason Field of ContainerStatus](https://github.com/kubernetes/kubernetes/pull/112977) + +The **Reason** field in **ContainerStatus** should be set to OOMKilled when cgroup out-of-memory occurs. + +#### [Modification of PodSecurityContext.SupplementalGroups Description](https://github.com/kubernetes/kubernetes/pull/113047) + +The description is modified to optimize the comments of **PodSecurityContext.SupplementalGroups**. The behavior that the main UID defined by the container image is not in the list is clarified. + +#### [ExecSync Output Restriction](https://github.com/kubernetes/kubernetes/pull/110435) + +The **ExecSync** return value output is less than 16 MB. + +## User Guide + +### Configuring iSulad to Support CRI V1 + +Configure iSulad to support CRI v1 1.29 used by the new Kubernetes version. + +For CRI v1 1.25 or earlier, the functions of V1alpha2 are the same as those of V1. The new features of CRI v1 1.26 or later are supported only in CRI v1. +The functions and features of this upgrade are supported only in CRI v1. Therefore, you need to enable CRI v1as follows. + +Enable CRI v1. + +Set **enable-cri-v1** in **daemon.json** of iSulad to **true** and restart iSulad. + +```json +{ + "group": "isula", + "default-runtime": "runc", + ... + "enable-cri-v1": true +} +``` + +If iSulad is installed from source, enable the **ENABLE_CRI_API_V1** compile option. + +```bash +cmake ../ -D ENABLE_CRI_API_V1=ON +``` + +### Using RuntimeConfig to Obtain the cgroup Driver Configuration + +#### systemd-cgroup Configuration + +iSulad supports both systemd and cgroupfs cgroup drivers. +By default, cgroupfs is used. You can configure iSulad to support systemd-cgroup. +iSulad supports only systemd-cgroup when the runtime is runc. In the iSulad configuration file **daemon.json**, +set **systemd-cgroup** to **true** and restart iSulad to use the systemd-cgroup driver. + +```json +{ + "group": "isula", + "default-runtime": "runc", + ... + "enable-cri-v1": true, + "systemd-cgroup": true +} +``` + +### Using GetContainerEvents to Generate Pod Lifecycle Events + +#### Pod Events Configuration + +In the iSulad configuration file **daemon.json**, +set **enable-pod-events** to **true** and restart iSulad. + +```json +{ + "group": "isula", + "default-runtime": "runc", + ... + "enable-cri-v1": true, + "enable-pod-events": true +} +``` + +## Constraints + +1. The preceding new features are supported by iSulad only when the container runtime is runc. +2. cgroup out-of-memory (OOM) triggers the deletion of the cgroup path of the container. If iSulad processes the OOM event after the cgroup path is deleted, iSulad cannot capture the OOM event of the container. As a result, the **Reason** field in **ContainerStatus** may be incorrect. +3. iSulad does not support the mixed use of different cgroup drivers to manage containers. After a container is started, the cgroup driver configuration in iSulad should not change. diff --git a/docs/en/cloud/container_engine/isula_container_engine/cri.md b/docs/en/cloud/container_engine/isula_container_engine/cri.md new file mode 100644 index 0000000000000000000000000000000000000000..2175407a4f67993b6efc8b97d1c0723ff0f1e154 --- /dev/null +++ b/docs/en/cloud/container_engine/isula_container_engine/cri.md @@ -0,0 +1,1271 @@ +# CRI API v1alpha2 + +## Description + +CRI API is the container runtime APIs provided by Kubernetes. CRI defines service interfaces for containers and images. iSulad uses CRI API to interconnect with Kubernetes. + +The lifecycle of a container is isolated from that of an image. Therefore, two services are required. CRI API is defined using [Protocol Buffers](https://developers.google.com/protocol-buffers/) and is based on [gRPC](https://grpc.io/). + +Currently, the default CRI API version used by iSulad is v1alpha2. The official API description file is as follows: + +[https://github.com/kubernetes/kubernetes/blob/release-1.14/pkg/kubelet/apis/cri/runtime/v1alpha2/api.proto](https://github.com/kubernetes/kubernetes/blob/release-1.14/pkg/kubelet/apis/cri/runtime/v1alpha2/api.proto), + +iSulad uses the API description file of version 1.14 used by Pass, which is slightly different from the official API. The interfaces in this document prevail. + +> [!NOTE]NOTE +> For the WebSocket streaming service of CRI API, the listening address of the server is 127.0.0.1, and the port number is 10350. The port number can be configured through the `--websocket-server-listening-port` command option or in the **daemon.json** configuration file. + +## Interfaces + +The following tables list the parameters that may be used by the interfaces. Some parameters cannot be configured. + +### Interface Parameters + +- **DNSConfig** + + Specifies the DNS servers and search domains of a sandbox. + + | Member | Description | + | ------------------------ | ------------------------------------------------------------------- | + | repeated string servers | List of DNS servers of the cluster | + | repeated string searches | List of DNS search domains of the cluster | + | repeated string options | List of DNS options. See . | + +- **Protocol** + + Enum values of the protocols. + + | Member | Description | + | ------- | ----------- | + | TCP = 0 | TCP | + | UDP = 1 | UDP | + +- **PortMapping** + + Specifies the port mapping configurations of a sandbox. + + | Member | Description | + | -------------------- | -------------------------------- | + | Protocol protocol | Protocol of the port mapping | + | int32 container_port | Port number within the container | + | int32 host_port | Port number on the host | + | string host_ip | Host IP address | + +- **MountPropagation** + + Enum values for mount propagation. + + | Member | Description | + | --------------------------------- | ------------------------------------------------------------------------------------------------------------ | + | PROPAGATION_PRIVATE = 0 | No mount propagation ("rprivate" in Linux) | + | PROPAGATION_HOST_TO_CONTAINER = 1 | Mounts get propagated from the host to the container ("rslave" in Linux) | + | PROPAGATION_BIDIRECTIONAL = 2 | Mounts get propagated from the host to the container and from the container to the host ("rshared" in Linux) | + +- **Mount** + + Specifies a host volume to mount into a container. (Only files and folders are supported.) + + | Member | Description | + | ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | string container_path | Path in the container | + | string host_path | Path on the host | + | bool readonly | Whether the configuration is read-only in the container. The default value is **false**. | + | bool selinux_relabel | Whether to set the SELinux label (not supported) | + | MountPropagation propagation | Mount propagation configuration. The value can be **0**, **1**, or **2**, corresponding to **rprivate**, **rslave**, or **rshared**. The default value is **0**. | + +- **NamespaceOption** + + | Member | Description | + | ----------------- | ------------------------------------------------ | + | bool host_network | Whether to use the network namespace of the host | + | bool host_pid | Whether to use the PID namespace of the host | + | bool host_ipc | Whether to use the IPC namespace of the host | + +- **Capability** + + Contains information about the capabilities to add or drop. + + | Member | Description | + | --------------------------------- | -------------------- | + | repeated string add_capabilities | Capabilities to add | + | repeated string drop_capabilities | Capabilities to drop | + +- **Int64Value** + + Wrapper of the int64 type. + + | Member | Description | + | ----------- | ------------------ | + | int64 value | Actual int64 value | + +- **UInt64Value** + + Wrapper of the uint64 type. + + | Member | Description | + | ------------ | ------------------- | + | uint64 value | Actual uint64 value | + +- **LinuxSandboxSecurityContext** + + Specifies Linux security options for a sandbox. + + Note that these security options are not applied to containers in the sandbox and may not be applicable to a sandbox without any running process. + + | Member | Description | + | -------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | NamespaceOption namespace_options | Options for namespaces of the sandbox | + | SELinuxOption selinux_options | SELinux options (not supported) | + | Int64Value run_as_user | UID to run sandbox processes | + | bool readonly_rootfs | Whether the root file system of the sandbox is read-only | + | repeated int64 supplemental_groups | User group information of process 1 in the sandbox besides the primary group | + | bool privileged | Whether the sandbox can run a privileged container | + | string seccomp_profile_path | Path of the seccomp configuration file. Valid values are:
// **unconfined**: seccomp is not used.
// **localhost/***\*: path of the configuration file installed in the system.
// *\*:Full path of the configuration file.
//By default, this parameter is not set, which is identical to **unconfined**.| + +- **LinuxPodSandboxConfig** + + Sets configurations related to Linux hosts and containers. + + | Member | Description | + | -------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | + | string cgroup_parent | Parent cgroup path of the sandbox. The runtime can convert it to the cgroupfs or systemd semantics as required. (Not configurable) | + | LinuxSandboxSecurityContext security_context | Security attributes of the sandbox | + | map\ sysctls | Linux sysctls configurations of the sandbox | + +- **PodSandboxMetadata** + + Stores all necessary information for building the sandbox name. The container runtime is encouraged to expose the metadata in its user interface for better user experience. For example, the runtime can construct a unique sandbox name based on the metadata. + + | Member | Description | + | ---------------- | --------------------------------------------------------------------- | + | string name | Sandbox name | + | string uid | Sandbox UID | + | string namespace | Sandbox namespace | + | uint32 attempt | Number of attempts to create the sandbox. The default value is **0**. | + +- **PodSandboxConfig** + + Contains all the required and optional fields for creating a sandbox. + + | Member | Description | + | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | + | PodSandboxMetadata metadata | Metadata of the sandbox. This information uniquely identifies the sandbox, and the runtime should leverage this to ensure correct operation. The runtime may also use this information to improve user experience, such as by constructing a readable sandbox name.| + | string hostname | Host name of the sandbox | + | string log_directory | Directory for storing log files of containers in the sandbox | + | DNSConfig dns_config | DNS configuration of the sandbox | + | repeated PortMapping port_mappings | Port mappings of the sandbox | + | map\ labels | Key-value pairs that may be used to identify a single sandbox or a series of sandboxes | + | map\ annotations | Key-value pair holding arbitrary data. The value cannot be modified and can be queried by using **PodSandboxStatus**. | + | LinuxPodSandboxConfig linux | Options related to the linux host | + +- **PodSandboxNetworkStatus** + + Describes the network status of the sandbox. + + | Member | Description | + | -------------- | -------------------------------------------- | + | string ip | IP address of the sandbox | + | string name | Name of the network interface in the sandbox | + | string network | Name of the additional network | + +- **Namespace** + + Stores namespace options. + + | Member | Description | + | ----------------------- | ----------------------- | + | NamespaceOption options | Linux namespace options | + +- **LinuxPodSandboxStatus** + + Describes the status of the Linux sandbox. + + | Member | Description | + | ----------------------- | ----------------- | + | Namespace**namespaces** | Sandbox namespace | + +- **PodSandboxState** + + Enum values for sandbox states. + + | Member | Description | + | -------------------- | ------------------------------ | + | SANDBOX_READY = 0 | Ready state of the sandbox | + | SANDBOX_NOTREADY = 1 | Non-ready state of the sandbox | + +- **PodSandboxStatus** + + Describes the podsandbox status. + + | Member | Description | + | ----------------------------------------- | -------------------------------------------------------------------------------------- | + | string id | Sandbox ID | + | PodSandboxMetadata metadata | Sandbox metadata | + | PodSandboxState state | Sandbox state | + | int64 created_at | Creation timestamps of the sandbox in nanoseconds | + | repeated PodSandboxNetworkStatus networks | Multi-plane network status of the sandbox | + | LinuxPodSandboxStatus linux | Status specific to Linux sandboxes | + | map\ labels | Key-value pairs that may be used to identify a single sandbox or a series of sandboxes | + | map\ annotations | Key-value pair holding arbitrary data. The value cannot be modified by the runtime. | + +- **PodSandboxStateValue** + + Wrapper of **PodSandboxState**. + + | Member | Description | + | --------------------- | ------------- | + | PodSandboxState state | Sandbox state | + +- **PodSandboxFilter** + + Filtering conditions when listing sandboxes. The intersection of multiple conditions is displayed. + + | Member | Description | + | ----------------------------------- | ------------------------------------------------------------------------------------ | + | string id | Sandbox ID | + | PodSandboxStateValue state | Sandbox state | + | map\ label_selector | Sandbox labels. Only full match is supported. Regular expressions are not supported. | + +- **PodSandbox** + + Minimal data that describes a sandbox. + + | Member | Description | + | -------------------------------- | -------------------------------------------------------------------------------------- | + | string id | Sandbox ID | + | PodSandboxMetadata metadata | Sandbox metadata | + | PodSandboxState state | Sandbox state | + | int64 created_at | Creation timestamps of the sandbox in nanoseconds | + | map\ labels | Key-value pairs that may be used to identify a single sandbox or a series of sandboxes | + | map\ annotations | Key-value pair holding arbitrary data. The value cannot be modified by the runtime | + +- **KeyValue** + + Wrapper of a key-value pair. + + | Member | Description | + | ------------ | ----------- | + | string key | Key | + | string value | Value | + +- **SELinuxOption** + + SELinux labels to be applied to the container. + + | Member | Description | + | ------------ | ----------- | + | string user | User | + | string role | Role | + | string type | Type | + | string level | Level | + +- **ContainerMetadata** + + ContainerMetadata contains all necessary information for building the container name. The container runtime is encouraged to expose the metadata in its user interface for better user experience. For example, the runtime can construct a unique container name based on the metadata. + + | Member | Description | + | -------------- | ----------------------------------------------------------------------- | + | string name | Name of a container | + | uint32 attempt | Number of attempts to create the container. The default value is **0**. | + +- **ContainerState** + + Enum values for container states. + + | Member | Description | + | --------------------- | ---------------------------------- | + | CONTAINER_CREATED = 0 | The container is created | + | CONTAINER_RUNNING = 1 | The container is running | + | CONTAINER_EXITED = 2 | The container is in the exit state | + | CONTAINER_UNKNOWN = 3 | The container state is unknown | + +- **ContainerStateValue** + + Wrapper of ContainerState. + + | Member | Description | + | -------------------- | --------------------- | + | ContainerState state | Container state value | + +- **ContainerFilter** + + Filtering conditions when listing containers. The intersection of multiple conditions is displayed. + + | Member | Description | + | ----------------------------------- | -------------------------------------------------------------------------------------- | + | string id | Container ID | + | PodSandboxStateValue state | Container state | + | string pod_sandbox_id | Sandbox ID | + | map\ label_selector | Container labels. Only full match is supported. Regular expressions are not supported. | + +- **LinuxContainerSecurityContext** + + Security configuration that will be applied to a container. + + | Member | Description | + | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | + | Capability capabilities | Capabilities to add or drop | + | bool privileged | Whether the container is in privileged mode. The default value is **false**. | + | NamespaceOption namespace_options | Namespace options of the container | + | SELinuxOption selinux_options | SELinux context to be optionally applied (**not supported currently**) | + | Int64Value run_as_user | UID to run container processes. Only one of **run_as_user** and **run_as_username** can be specified at a time. **run_as_username** takes effect preferentially. | + | string run_as_username | User name to run container processes. If specified, the user must exist in the container image (that is, in **/etc/passwd** inside the image) and be resolved there by the runtime. Otherwise, the runtime must throw an error.| + | bool readonly_rootfs | Whether the root file system in the container is read-only. The default value is configured in **config.json**. | + | repeated int64 supplemental_groups | List of groups of the first process in the container besides the primary group | + | string apparmor_profile | AppArmor configuration file for the container (**not supported currently**) | + | string seccomp_profile_path | Seccomp configuration file for the container | + | bool no_new_privs | Whether to set the **no_new_privs** flag on the container | + +- **LinuxContainerResources** + + Resource specification for the Linux container. + + | Member | Description | + | --------------------------- | ----------------------------------------------------------------------------- | + | int64 cpu_period | CPU Completely Fair Scheduler (CFS) period. The default value is **0**. | + | int64 cpu_quota | CPU CFS quota. The default value is **0**. | + | int64 cpu_shares | CPU shares (weight relative to other containers). The default value is **0**. | + | int64 memory_limit_in_bytes | Memory limit, in bytes. The default value is **0**. | + | int64 oom_score_adj | oom-killer score. The default value is **0**. | + | string cpuset_cpus | CPU cores to be used by the container. The default value is **""**. | + | string cpuset_mems | Memory nodes to be used by the container. The default value is **""**. | + +- **Image** + + Basic information about a container image. + + | Member | Description | + | ---------------------------- | ------------------------------ | + | string id | Image ID | + | repeated string repo_tags | Image tag name (**repo_tags**) | + | repeated string repo_digests | Image digest information | + | uint64 size | Image size | + | Int64Value uid | UID of the default image user | + | string username | Name of the default image user | + +- **ImageSpec** + + Internal data structure that represents an image. Currently, **ImageSpec** wraps only the container image name. + + | Member | Description | + | ------------ | -------------------- | + | string image | Container image name | + +- **StorageIdentifier** + + Unique identifier of a storage device. + + | Member | Description | + | ----------- | ------------------ | + | string uuid | UUID of the device | + +- **FilesystemUsage** + + | Member | Description | + | ---------------------------- | ------------------------------------------------ | + | int64 timestamp | Timestamp at which the information was collected | + | StorageIdentifier storage_id | UUID of the file system that stores the image | + | UInt64Value used_bytes | Space size used for storing image metadata | + | UInt64Value inodes_used | Number of inodes for storing image metadata | + +- **AuthConfig** + + | Member | Description | + | --------------------- | --------------------------------------------------------------------------------------------- | + | string username | User name used for downloading images | + | string password | Password used for downloading images | + | string auth | Base64-encoded authentication information used for downloading images | + | string server_address | Address of the server for downloaded images (not supported currently) | + | string identity_token | Token information used for authentication with the image repository (not supported currently) | + | string registry_token | Token information used for interaction with the image repository (not supported currently) | + +- **Container** + + Container description information, such as the ID and state. + + | Member | Description | + | -------------------------------- | ----------------------------------------------------------------------------------------- | + | string id | Container ID | + | string pod_sandbox_id | ID of the sandbox to which the container belongs | + | ContainerMetadata metadata | Container metadata | + | ImageSpec image | Image specifications | + | string image_ref | Reference to the image used by the container. For most runtimes, this is an image ID. | + | ContainerState state | Container state | + | int64 created_at | Creation timestamps of the container in nanoseconds | + | map\ labels | Key-value pairs that may be used to identify a single container or a series of containers | + | map\ annotations | Key-value pair holding arbitrary data. The value cannot be modified by the runtime | + +- **ContainerStatus** + + Container status information. + + | Member | Description | + | -------------------------------- | -------------------------------------------------------------------------------------------------------- | + | string id | Container ID | + | ContainerMetadata metadata | Container metadata | + | ContainerState state | Container state | + | int64 created_at | Creation timestamps of the container in nanoseconds | + | int64 started_at | Startup timestamps of the container in nanoseconds | + | int64 finished_at | Exit timestamps of the container in nanoseconds | + | int32 exit_code | Container exit code | + | ImageSpec image | Image specifications | + | string image_ref | Reference to the image used by the container. For most runtimes, this is an image ID. | + | string reason | Brief explanation of why the container is in its current state | + | string message | Human-readable message explaining why the container is in its current state | + | map\ labels | Key-value pairs that may be used to identify a single container or a series of containers | + | map\ annotations | Key-value pair holding arbitrary data. The value cannot be modified by the runtime. | + | repeated Mount mounts | Container mount point information | + | string log_path | Container log file path. The file is in the **log_directory** folder configured in **PodSandboxConfig**. | + +- **ContainerStatsFilter** + + Filtering conditions when listing container states. The intersection of multiple conditions is displayed. + + | Member | Description | + | ----------------------------------- | -------------------------------------------------------------------------------------- | + | string id | Container ID | + | string pod_sandbox_id | Sandbox ID | + | map\ label_selector | Container labels. Only full match is supported. Regular expressions are not supported. | + +- **ContainerStats** + + Filtering conditions when listing container states. The intersection of multiple conditions is displayed. + + | Member | Description | + | ------------------------------ | --------------------------- | + | ContainerAttributes attributes | Container Information | + | CpuUsage cpu | CPU usage | + | MemoryUsage memory | Memory usage | + | FilesystemUsage writable_layer | Usage of the writable layer | + +- **ContainerAttributes** + + Basic information about the container. + + | Member | Description | + | ------------------------------- | ----------------------------------------------------------------------------------------- | + | string id | Container ID | + | ContainerMetadata metadata | Container metadata | + | map\ labels | Key-value pairs that may be used to identify a single container or a series of containers | + | map\ annotations | Key-value pair holding arbitrary data. The value cannot be modified by the runtime. | + +- **CpuUsage** + + Container CPU usage. + + | Member | Description | + | ----------------------------------- | ---------------------------------- | + | int64 timestamp | Timestamp | + | UInt64Value usage_core_nano_seconds | CPU usage duration, in nanoseconds | + +- **MemoryUsage** + + Container memory usage. + + | Member | Description | + | ----------------------------- | ------------ | + | int64 timestamp | Timestamp | + | UInt64Value working_set_bytes | Memory usage | + +- **FilesystemUsage** + + Usage of the writable layer of the container. + + | Member | Description | + | ---------------------------- | ------------------------------------------------------------ | + | int64 timestamp | Timestamp | + | StorageIdentifier storage_id | Writable layer directory | + | UInt64Value used_bytes | Number of bytes occupied by the image at the writable layer | + | UInt64Value inodes_used | Number of inodes occupied by the image at the writable layer | + +- **Device** + + Host volume to mount into a container. + + | Member | Description | + | -------------------- | --------------------------------------------------------------------------------------------------------- | + | string container_path | Mount path within the container | + | string host_path | Mount path on the host | + | string permissions | cgroup permissions of the device (**r** allows the container to read from the specified device; **w** allows the container to write to the specified device; **m** allows the container to create device files that do not yet exist).| + +- **LinuxContainerConfig** + + Configuration specific to Linux containers. + + | Member | Description | + | ---------------------------------------------- | -------------------------------------- | + | LinuxContainerResources resources | Container resource specifications | + | LinuxContainerSecurityContext security_context | Linux container security configuration | + +- **ContainerConfig** + + Required and optional fields for creating a container. + + | Member | Description | + | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | ContainerMetadata metadata | Container metadata. This information uniquely identifies the container, and the runtime should leverage this to ensure correct operation. The runtime may also use this information to improve user experience, such as by constructing a readable container name. (**Required**) | + | ImageSpec image | Image used by the container. (Required) | + | repeated string command | Command to be executed. The default value is **"/bin/sh"**. | + | repeated string args | Arguments of the command to be executed | + | string working_dir | Current working directory of the command to be executed | + | repeated KeyValue envs | Environment variables to set in the container | + | repeated Mount mounts | Mount points in the container | + | repeated Device devices | Devices to be mapped in the container | + | mapstring, labels | Key-value pairs that may be used to index and select individual resources | + | mapstring, annotations | Unstructured key-value map that may be used to store and retrieve arbitrary metadata | + | string log_path | Path relative to **PodSandboxConfig.LogDirectory** for container to store the logs (STDOUT and STDERR) on the host | + | bool stdin | Whether to enable STDIN of the container | + | bool stdin_once | Whether to immediately disconnect all data streams connected to STDIN when a data stream connected to stdin is disconnected (**not supported currently**) | + | bool tty | Whether to use a pseudo terminal to connect to STDIO of the container | + | LinuxContainerConfig linux | Configuration specific to Linux containers | + +- **NetworkConfig** + + Runtime network configuration. + + | Member | Description | + | --------------- | ------------------------- | + | string pod_cidr | CIDR for pod IP addresses | + +- **RuntimeConfig** + + Runtime network configuration. + + | Member | Description | + | ---------------------------- | ----------------------------- | + | NetworkConfig network_config | Runtime network configuration | + +- **RuntimeCondition** + + Runtime condition information. + + | Member | Description | + | -------------- | ----------------------------------------------------------------------------- | + | string type | Runtime condition type | + | bool status | Runtime status | + | string reason | Brief description of the reason for the runtime condition change | + | string message | Human-readable message describing the reason for the runtime condition change | + +- **RuntimeStatus** + + Runtime status. + + | Member | Description | + | ------------------------------------ | -------------------------- | + | repeated RuntimeCondition conditions | Current runtime conditions | + +### Runtime Service + +The runtime service contains interfaces for operating pods and containers, and interfaces for querying the configuration and status of the runtime service. + +#### RunPodSandbox + +#### Interface Prototype + +```protobuf +rpc RunPodSandbox(RunPodSandboxRequest) returns (RunPodSandboxResponse) {} +``` + +#### Interface Description + +Creates and starts a pod sandbox. The sandbox is in the ready state on success. + +#### Precautions + +1. The default image for starting the sandbox is **rnd-dockerhub.huawei.com/library/pause-$\{machine\}:3.0**, where **$\{machine\}** indicates the architecture. On x86\_64, the value of **machine** is **amd64**, on ARM64, the value of **machine** is **aarch64**. Currently, only the **amd64** and **aarch64** images can be downloaded from the rnd-dockerhub repository. If the images do not exist on the host, ensure that the host can download them from the rnd-dockerhub repository. +2. The container names use the field in **PodSandboxMetadata** and are separated by underscores (\_). Therefore, the data in metadata cannot contain underscores. Otherwise, the sandbox runs successfully, but the **ListPodSandbox** interface cannot query the sandbox. + +#### Parameter + +| Member | Description | +| ----------------------- | -------------------------------------------------------------------------------------- | +| PodSandboxConfig config | Sandbox configuration | +| string runtime_handler | Runtime to use for the sandbox. Currently, **lcr** and **kata-runtime** are supported. | + +#### Returns + +| Return | Description | +| --------------------- | --------------------------------------- | +| string pod_sandbox_id | The response data is return on success. | + +#### StopPodSandbox + +#### Interface Prototype + +```protobuf +rpc StopPodSandbox(StopPodSandboxRequest) returns (StopPodSandboxResponse) {} +``` + +#### Interface Description + +Stops the pod sandbox, stops the sandbox container, and reclaims the network resources (such as IP addresses) allocated to the sandbox. If any running container belongs to the sandbox, the container must be forcibly terminated. + +#### Parameter + +| Member | Description | +| --------------------- | ----------- | +| string pod_sandbox_id | Sandbox ID | + +#### Returns + +| Return | Description | +| ------ | ----------- | +| None | None | + +#### RemovePodSandbox + +#### Interface Prototype + +```text +rpc RemovePodSandbox(RemovePodSandboxRequest) returns (RemovePodSandboxResponse) {} +``` + +#### Interface Description + +Removes a sandbox. If there are any running containers in the sandbox, they must be forcibly terminated and removed. This interface must not return an error if the sandbox has already been removed. + +#### Precautions + +1. When a sandbox is deleted, the network resources of the sandbox are not deleted. Before deleting the pod, you must call **StopPodSandbox** to remove the network resources. Ensure that **StopPodSandbox** is called at least once before deleting the sandbox. +2. If the container in a sandbox fails to be deleted when the sandbox is deleted, the sandbox is deleted but the container remains. In this case, you need to manually delete the residual container. + +#### Parameter + +| Member | Description | +| --------------------- | ----------- | +| string pod_sandbox_id | Sandbox ID | + +#### Returns + +| Return | Description | +| ------ | ----------- | +| None | None | + +#### PodSandboxStatus + +#### Interface Prototype + +```text +rpc PodSandboxStatus(PodSandboxStatusRequest) returns (PodSandboxStatusResponse) {} +``` + +#### Interface Description + +Queries the status of the sandbox. If the sandbox does not exist, this interface returns an error. + +#### Parameter + +| Member | Description | +| --------------------- | ---------------------------------------------------------------------------------- | +| string pod_sandbox_id | Sandbox ID | +| bool verbose | Whether to return extra information about the sandbox (not configurable currently) | + +#### Returns + +| Return | Description | +| ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | +| PodSandboxStatus status | Sandbox status information | +| map\ info | Extra information of the sandbox. The **key** can be an arbitrary string, and **value** is in JSON format. **info** can include anything debug information. When **verbose** is set to **true**, **info** cannot be empty (not configurable currently). | + +#### ListPodSandbox + +#### Interface Prototype + +```text +rpc ListPodSandbox(ListPodSandboxRequest) returns (ListPodSandboxResponse) {} +``` + +#### Interface Description + +Returns sandbox information. Conditional filtering is supported. + +#### Parameter + +| Member | Description | +| ----------------------- | -------------------------------- | +| PodSandboxFilter filter | Conditional filtering parameters | + +#### Returns + +| Return | Description | +| ------------------------- | ----------- | +| repeated PodSandbox items | Sandboxes | + +#### CreateContainer + +#### Interface Prototype + +```text +rpc CreateContainer(CreateContainerRequest) returns (CreateContainerResponse) {} +``` + +#### Interface Description + +Creates a container in a PodSandbox. + +#### Precautions + +- **sandbox\_config** in **CreateContainerRequest** is the same as the configuration passed to **RunPodSandboxRequest** to create the PodSandbox. It is passed again for reference. **PodSandboxConfig** is immutable and remains unchanged throughout the lifecycle of a pod. +- The container names use the field in **ContainerMetadata** and are separated by underscores (\_). Therefore, the data in metadata cannot contain underscores. Otherwise, the container runs successfully, but the **ListContainers** interface cannot query the container. +- **CreateContainerRequest** does not contain the **runtime\_handler** field. The runtime type of the created container is the same as that of the corresponding sandbox. + +#### Parameter + +| Member | Description | +| ------------------------------- | --------------------------------------------------------- | +| string pod_sandbox_id | ID of the PodSandbox where the container is to be created | +| ContainerConfig config | Container configuration information | +| PodSandboxConfig sandbox_config | PodSandbox configuration information | + +#### Supplementary Information + +Unstructured key-value map that may be used to store and retrieve arbitrary metadata. Some fields can be transferred through this field because CRI does not provide specific parameters. + +- Customization + + | Custom Key:Value | Description | + | ----------------------- | ---------------------------------------------------------------------------------- | + | cgroup.pids.max:int64_t | Limits the number of processes/threads in a container. (Set **-1** for unlimited.) | + +#### Returns + +| Return | Description | +| ------------------- | --------------------------- | +| string container_id | ID of the created container | + +#### StartContainer + +#### Interface Prototype + +```text +rpc StartContainer(StartContainerRequest) returns (StartContainerResponse) {} +``` + +#### Interface Description + +Starts a container. + +#### Parameter + +| Member | Description | +| ------------------- | ------------ | +| string container_id | Container ID | + +#### Returns + +| Return | Description | +| ------ | ----------- | +| None | None | + +#### StopContainer + +#### Interface Prototype + +```text +rpc StopContainer(StopContainerRequest) returns (StopContainerResponse) {} +``` + +#### Interface Description + +Stops a running container. The graceful stop timeout can be configured. If the container has been stopped, no error can be returned. + +#### Parameter + +| Member | Description | +| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | +| string container_id | Container ID | +| int64 timeout | Waiting time before a container is forcibly stopped. The default value is **0**, indicating that the container is forcibly stopped immediately. | + +#### Returns + +None + +#### RemoveContainer + +#### Interface Prototype + +```text +rpc RemoveContainer(RemoveContainerRequest) returns (RemoveContainerResponse) {} +``` + +#### Interface Description + +Deletes a container. If the container is running, it must be forcibly stopped. If the container has been deleted, no error can be returned. + +#### Parameter + +| Member | Description| +| --------------- --- | ------------- | +| string container_id | Container ID | + +#### Returns + +None + +#### ListContainers + +#### Interface Prototype + +```text +rpc ListContainers(ListContainersRequest) returns (ListContainersResponse) {} +``` + +#### Interface Description + +Returns container information. Conditional filtering is supported. + +#### Parameter + +| Member | Description | +| ---------------------- | -------------------------------- | +| ContainerFilter filter | Conditional filtering parameters | + +#### Returns + +| Return | Description | +| ----------------------------- | ----------- | +| repeated Container containers | Containers | + +#### ContainerStatus + +#### Interface Prototype + +```text +rpc ContainerStatus(ContainerStatusRequest) returns (ContainerStatusResponse) {} +``` + +#### Interface Description + +Returns container status information. If the container does not exist, an error is returned. + +#### Parameter + +| Member | Description | +| ------------------- | ---------------------------------------------------------------------------------------- | +| string container_id | Container ID | +| bool verbose | Whether to display additional information about the sandbox (not configurable currently) | + +#### Returns + +| Return | Description | +| ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | +| ContainerStatus status | Container status information | +| map\ info | Extra information of the sandbox. The **key** can be an arbitrary string, and **value** is in JSON format. **info** can include anything debug information. When **verbose** is set to **true**, **info** cannot be empty (not configurable currently).| + +#### UpdateContainerResources + +#### Interface Prototype + +```text +rpc UpdateContainerResources(UpdateContainerResourcesRequest) returns (UpdateContainerResourcesResponse) {} +``` + +#### Interface Description + +Updates container resource configurations. + +#### Precautions + +- This interface is used exclusively to update the resource configuration of a container, not a pod. +- Currently, the **oom\_score\_adj** configuration of containers cannot be updated. + +#### Parameter + +| Member | Description | +| ----------------------------- | ---------------------------------------- | +| string container_id | Container ID | +| LinuxContainerResources linux | Linux resource configuration information | + +#### Returns + +None + +#### ExecSync + +#### Interface Prototype + +```text +rpc ExecSync(ExecSyncRequest) returns (ExecSyncResponse) {} +``` + +#### Interface Description + +Runs a command synchronously in a container and communicates using gRPC. + +#### Precautions + +This interface runs a single command and cannot open a terminal to interact with the container. + +#### Parameter + +| Member | Description | +| ------------------ | ------------------------------------------------------------------ | +| string container_id | Container ID | +| repeated string cmd | Command to be executed | +| int64 timeout | Timeout interval before a command to be stopped is forcibly terminated, in seconds. The default value is **0**, indicating that there is no timeout limit (**not supported currently**).| + +#### Returns + +| Return | Description | +| --------------- | ------------------------------------------------------------------------------------ | +| bytes stdout | Captures the standard output of the command | +| bytes stderr | Captures the standard error output of the command | +| int32 exit_code | Exit code the command finished with. The default value is **0**, indicating success. | + +#### Exec + +#### Interface Prototype + +```text +rpc Exec(ExecRequest) returns (ExecResponse) {} +``` + +#### Interface Description + +Runs a command in the container, obtains the URL from the CRI server using gRPC, and establishes a persistent connection with the WebSocket server based on the obtained URL to interact with the container. + +#### Precautions + +This interface runs a single command and can open a terminal to interact with the container. One of **stdin**, **stdout**, or **stderr** must be true. If **tty** is true, **stderr** must be false. Multiplexing is not supported. In that case, the outputs of **stdout** and **stderr** are combined into a single stream. + +#### Parameter + +| Member | Description | +| ------------------- | --------------------------------------- | +| string container_id | Container ID | +| repeated string cmd | Command to be executed | +| bool tty | Whether to run the command in a TTY | +| bool stdin | Whether to stream standard input | +| bool stdout | Whether to stream standard output | +| bool stderr | Whether to stream standard error output | + +#### Returns + +| Return | Description | +| ---------- | ------------------------------------------------ | +| string url | Fully qualified URL of the exec streaming server | +| | | + +#### Attach + +#### Interface Prototype + +```text +rpc Attach(AttachRequest) returns (AttachResponse) {} +``` + +#### Interface Description + +Takes over process 1 of the container, obtains the URL from the CRI server using gRPC, and establishes a persistent connection with the WebSocket server based on the obtained URL to interact with the container. + +#### Parameter + +| Member | Description | +| ------------------- | --------------------------------------- | +| string container_id | Container ID | +| bool tty | Whether to run the command in a TTY | +| bool stdin | Whether to stream standard input | +| bool stdout | Whether to stream standard output | +| bool stderr | Whether to stream standard error output | + +#### Returns + +| Return | Description | +| ---------- | -------------------------------------------------- | +| string url | Fully qualified URL of the attach streaming server | + +#### ContainerStats + +#### Interface Prototype + +```text +rpc ContainerStats(ContainerStatsRequest) returns (ContainerStatsResponse) {} +``` + +#### Interface Description + +Returns information about the resources occupied by a single container. Only containers whose runtime type is lcr are supported. + +#### Parameter + +| Member | Description | +| ------------------- | ------------ | +| string container_id | Container ID | + +#### Returns + +| Return | Description | +| -------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | +| ContainerStats stats | Container information. Information about drives and inodes can be returned only for containers started using images in oci format. | + +#### ListContainerStats + +#### Interface Prototype + +```text +rpc ListContainerStats(ListContainerStatsRequest) returns (ListContainerStatsResponse) {} +``` + +#### Interface Description + +Returns information about resources occupied by multiple containers. Conditional filtering is supported. + +#### Parameter + +| Member | Description | +| --------------------------- | -------------------------------- | +| ContainerStatsFilter filter | Conditional filtering parameters | + +#### Returns + +| Return | Description | +| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | +| repeated ContainerStats stats | List of container information. Information about drives and inodes can be returned only for containers started using images in OCI format. | + +#### UpdateRuntimeConfig + +#### Interface Prototype + +```text +rpc UpdateRuntimeConfig(UpdateRuntimeConfigRequest) returns (UpdateRuntimeConfigResponse); +``` + +#### Interface Description + +Provides standard CRI for updating pod CIDR of the network plugin. Currently, the CNI network plugins do not need to update the pod CIDR. Therefore, this interface only records access logs. + +#### Precautions + +This interface does not modify the system management information, but only records logs. + +#### Parameter + +| Member | Description | +| ---------------------------- | -------------------------------------------- | +| RuntimeConfig runtime_config | Information to be configured for the runtime | + +#### Returns + +None + +#### Status + +#### Interface Prototype + +```text +rpc Status(StatusRequest) returns (StatusResponse) {}; +``` + +#### Interface Description + +Obtains the network status of the runtime and pod. When the network status is obtained, the network configuration is updated. + +#### Precautions + +If the network configuration fails to be updated, the original configuration is not affected. The original configuration is overwritten only when the network configuration is updated successfully. + +#### Parameter + +| Member | Description | +| ------------ | --------------------------------------------------------------------------- | +| bool verbose | Whether to display additional runtime information (not supported currently) | + +#### Returns + +| Return | Description | +| ----------------------- | ---------------------------------------------------------------------------------------------------------- | +| RuntimeStatus status | Runtime status | +| map\ info | Additional runtime information. The key of **info** can be any value, and the **value** is in JSON format and can contain any debug information. Additional information is displayed only when **Verbose** is set to **true**.| + +### Image Service + +Provides gRPC APIs for pulling, viewing, and removing images from the image repository. + +#### ListImages + +#### Interface Prototype + +```text +rpc ListImages(ListImagesRequest) returns (ListImagesResponse) {} +``` + +#### Interface Description + +Lists information about existing images. + +#### Precautions + +This interface is a unified interface. Images of embedded format can be queried using **cri images**. However, because embedded images are not in OCI standard, the query result has the following restrictions: + +- The displayed image ID is **digest** of **config** of the image because embedded images do not have image IDs. +- **digest** cannot be displayed because embedded images have only **digest** of **config**, not **digest** of themselves, and **digest** does not comply with OCI specifications. + +#### Parameter + +| Member | Description | +| ---------------- | ----------------------------- | +| ImageSpec filter | Name of images to be filtered | +| | | + +#### Returns + +| Return | Description| +| -------------------- | ------------- | +| repeated Image images | List of images | + +#### ImageStatus + +#### Interface Prototype + +```text +rpc ImageStatus(ImageStatusRequest) returns (ImageStatusResponse) {} +``` + +#### Interface Description + +Queries the details about a specified image. + +#### Precautions + +1. This interface is used to query information about a specified image. If the image does not exist, **ImageStatusResponse** is returned, in which **Image** is **nil**. +2. This interface is a unified interface. Images of embedded format cannot be queried because they do not comply with the OCI specification and lack some fields. + +#### Parameter + +| Member | Description | +| --------------- | ---------------------------------------------------------------------------------------------------------- | +| ImageSpec image | Image name | +| bool verbose | Queries extra information. This parameter is not supported currently and no extra information is returned. | + +#### Returns + +| Return | Description | +| ------------------------- | -------------------------------------------------------------------------------------------------------- | +| Image image | Image information | +| map\ info | Extra image information. This parameter is not supported currently and no extra information is returned. | + +#### PullImage + +#### Interface Prototype + +```text +rpc PullImage(PullImageRequest) returns (PullImageResponse) {} +``` + +#### Interface Description + +Downloads an image. + +#### Precautions + +You can download public images or private images using the username, password, and authentication information. The **server_address**, **identity_token**, and **registry_token** fields in **AuthConfig** are not supported. + +#### Parameter + +| Member | Description | +| ------------------------------- | ---------------------------------------------------------------- | +| ImageSpec image | Name of the image to download | +| AuthConfig auth | Authentication information for downloading a private image | +| PodSandboxConfig sandbox_config | Downloads an Image in the pod context (not supported currently). | + +#### Returns + +| Return | Description | +| ---------------- | -------------------------------------- | +| string image_ref | Information about the downloaded image | +| | | + +#### RemoveImage + +#### Interface Prototype + +```text +rpc RemoveImage(RemoveImageRequest) returns (RemoveImageResponse) {} +``` + +#### Interface Description + +Deletes a specified image. + +#### Precautions + +This interface is a unified interface. Images of embedded format cannot be deleted based on the image ID because they do not comply with the OCI specification and lack some fields. + +#### Parameter + +| Member | Description | +| --------------- | ------------------------------------- | +| ImageSpec image | Name or ID of the image to be deleted | + +#### Returns + +None + +#### ImageFsInfo + +#### Interface Prototype + +```text +rpc ImageFsInfo(ImageFsInfoRequest) returns (ImageFsInfoResponse) {} +``` + +#### Interface Description + +Queries information about the file systems of an image. + +#### Precautions + +The queried information is the file system information in the image metadata. + +#### Parameter + +None + +#### Returns + +| Return | Description | +| ------------------------------------------ | ----------------------------- | +| repeated FilesystemUsage image_filesystems | Image file system information | + +### Constraints + +1. If **log_directory** is configured in **PodSandboxConfig** when a sandbox is created, **log_path** must be specified in **ContainerConfig** when a container of the sandbox is created. Otherwise, the container may fail to be started or even deleted using CRI API. + + The actual **LOGPATH** of the container is **log_directory/log_path**. If **log_path** is not configured, the final **LOGPATH** changes to **log_directory**. + + - If the path does not exist, iSulad creates a soft link pointing to the final path of container logs when starting the container, and **log_directory** becomes a soft link. In this case, there are two situations: + + 1. If **log_path** is not configured for other containers in the sandbox, when other containers are started, **log_directory** is deleted and points to **log_path** of the newly started container. As a result, the logs of the previously started container point to the logs of the container started later. + 2. If **log_path** is configured for other containers in the sandbox, **LOGPATH** of the container is **log_directory/log_path**. Because **log_directory** is a soft link, if **log_directory/log_path** is used as the soft link target to point to the actual log path of the container, the container creation fails. + - If the path exists, iSulad attempts to delete the path (non-recursively) when starting the container. If the path is a folder that contains content, the deletion fails. As a result, the soft link fails to be created and the container fails to be started. When the container is deleted, the same symptom occurs. As a result, the container deletion fails. +2. If **log_directory** is configured in **PodSandboxConfig** when a sandbox is created and **log_path** is configured in **ContainerConfig** when a container is created, the final **LOGPATH** is **log_directory/log_path**. iSulad does not create **LOGPATH** recursively. Therefore, you must ensure that **dirname(LOGPATH)**, that is, the parent directory of the final log directory, exists. +3. If **log_directory** is configured in **PodSandboxConfig** when a sandbox is created, and the same **log_path** is specified in **ContainerConfig** when two or more containers are created or containers in different sandboxes point to the same **LOGPATH**, when the containers are started successfully, the log path of the container that is started later overwrites that of the container that is started earlier. +4. If the image content in the remote image repository changes and the CRI image pulling interface is used to download the image again, the image name and tag of the local original image (if it exists) change to "none." + + Example: + + Local image: + + ```text + IMAGE TAG IMAGE ID SIZE + rnd-dockerhub.huawei.com/pproxyisulad/test latest 99e59f495ffaa 753kB + ``` + + After the **rnd-dockerhub.huawei.com/pproxyisulad/test:latest** image in the remote repository is updated and downloaded again: + + ```text + IMAGE TAG IMAGE ID SIZE + 99e59f495ffaa 753kB + rnd-dockerhub.huawei.com/pproxyisulad/test latest d8233ab899d41 1.42MB + ``` + + Run the `isula images` command. **REF** is displayed as **-**. + + ```text + REF IMAGE ID CREATED SIZE + rnd-dockerhub.huawei.com/pproxyisulad/test:latest d8233ab899d41 2019-02-14 19:19:37 1.42MB + - 99e59f495ffaa 2016-05-04 02:26:41 753kB + ``` + +5. The exec and attach interfaces of iSulad CRI API are implemented using WebSocket. Clients interact with iSulad using the same protocol. When using the exec or attach interface, do not transfer a large amount of data or files over the serial port. The exec or attach interface is used only for basic command interaction. If the user side does not process the data or files in a timely manner, data may be lost. In addition, do not use the exec or attach interface to transfer binary data or files. +6. The iSulad CRI API exec/attach depends on libwebsockets (LWS). It is recommended that the streaming API be used only for persistent connection interaction but not in high-concurrency scenarios, because the connection may fail due to insufficient host resources. It is recommended that the number of concurrent connections be less than or equal to 100. diff --git a/docs/en/cloud/container_engine/isula_container_engine/figures/en-us_image_0183048952.png b/docs/en/cloud/container_engine/isula_container_engine/figures/en-us_image_0183048952.png new file mode 100644 index 0000000000000000000000000000000000000000..fe9074f8fba969795f1e1d40fb879e21d5fc2a7c Binary files /dev/null and b/docs/en/cloud/container_engine/isula_container_engine/figures/en-us_image_0183048952.png differ diff --git a/docs/en/cloud/container_engine/isula_container_engine/image-management.md b/docs/en/cloud/container_engine/isula_container_engine/image-management.md new file mode 100644 index 0000000000000000000000000000000000000000..ed2ee074cca619ab87077f7fefd3036a8d1cd837 --- /dev/null +++ b/docs/en/cloud/container_engine/isula_container_engine/image-management.md @@ -0,0 +1,370 @@ +# Image Management + +- [Image Management](#image-management) + - [Docker Image Management](#docker-image-management) + - [Logging In to a Registry](#logging-in-to-a-registry) + - [Logging Out of a Registry](#logging-out-of-a-registry) + - [Pulling Images from a Registry](#pulling-images-from-a-registry) + - [Deleting Images](#deleting-images) + - [Loading Images](#loading-images) + - [Listing Images](#listing-images) + - [Inspecting Images](#inspecting-images) + - [Two-Way Authentication](#two-way-authentication) + - [Embedded Image Management](#embedded-image-management) + - [Loading Images](#loading-images-1) + - [Listing Images](#listing-images-1) + - [Inspecting Images](#inspecting-images-1) + - [Deleting Images](#deleting-images-1) + +## Docker Image Management + +### Logging In to a Registry + +#### Description + +The **isula login** command is run to log in to a registry. After successful login, you can run the **isula pull** command to pull images from the registry. If the registry does not require a password, you do not need to run this command before pulling images. + +#### Usage + +```shell +isula login [OPTIONS] SERVER +``` + +#### Parameters + +For details about parameters in the **login** command, see Table 1 in [Command Line Parameters](./appendix.md#command-line-parameters). + +#### Example + +```shell +$ isula login -u abc my.csp-edge.com:5000 + +Login Succeeded +``` + +### Logging Out of a Registry + +#### Description + +The **isula logout** command is run to log out of a registry. If you run the **isula pull** command to pull images from the registry after logging out of the system, the image will fail to be pulled because you are not authenticated. + +#### Usage + +```shell +isula logout SERVER +``` + +#### Parameters + +For details about parameters in the **logout** command, see Table 2 in [Command Line Parameters](./appendix.md#command-line-parameters). + +#### Example + +```shell +$ isula logout my.csp-edge.com:5000 +Logout Succeeded +``` + +### Pulling Images from a Registry + +#### Description + +Pull images from a registry to the local host. + +#### Usage + +```shell +isula pull [OPTIONS] NAME[:TAG|@DIGEST] +``` + +#### Parameters + +For details about parameters in the **pull** command, see Table 3 in [Command Line Parameters](./appendix.md#command-line-parameters). + +#### Example + +```shell +$ isula pull localhost:5000/official/busybox +Image "localhost:5000/official/busybox" pulling +Image "localhost:5000/official/busybox@sha256:bf510723d2cd2d4e3f5ce7e93bf1e52c8fd76831995ac3bd3f90ecc866643aff" pulled +``` + +### Deleting Images + +#### Description + +Delete one or more images. + +#### Usage + +```shell +isula rmi [OPTIONS] IMAGE [IMAGE...] +``` + +#### Parameters + +For details about parameters in the **rmi** command, see Table 4 in [Command Line Parameters](./appendix.md#command-line-parameters). + +#### Example + +```shell +$ isula rmi rnd-dockerhub.huawei.com/official/busybox +Image "rnd-dockerhub.huawei.com/official/busybox" removed +``` + +### Loading Images + +#### Description + +Load images from a .tar package. The .tar package must be exported by using the **docker save** command or must be in the same format. + +#### Usage + +```shell +isula load [OPTIONS] +``` + +#### Parameters + +For details about parameters in the **load** command, see Table 5 in [Command Line Parameters](./appendix.md#command-line-parameters). + +#### Example + +```shell +$ isula load -i busybox.tar +Load image from "/root/busybox.tar" success +``` + +### Listing Images + +#### Description + +List all images in the current environment. + +#### Usage + +```shell +isula images +``` + +#### Parameters + +For details about parameters in the **images** command, see Table 6 in [Command Line Parameters](./appendix.md#command-line-parameters). + +#### Example + +```shell +$ isula images +REF IMAGE ID CREATED SIZE +rnd-dockerhub.huawei.com/official/busybox:latest e4db68de4ff2 2019-06-15 08:19:54 1.376 MB +``` + +### Inspecting Images + +#### Description + +After the configuration information of an image is returned, you can use the **-f** parameter to filter the information as needed. + +#### Usage + +```shell +isula inspect [options] CONTAINER|IMAGE [CONTAINER|IMAGE...] +``` + +#### Parameters + +For details about parameters in the **inspect** command, see Table 7 in [Command Line Parameters](./appendix.md#command-line-parameters). + +#### Example + +```shell +$ isula inspect -f "{{json .image.id}}" rnd-dockerhub.huawei.com/official/busybox +"e4db68de4ff27c2adfea0c54bbb73a61a42f5b667c326de4d7d5b19ab71c6a3b" +``` + +### Two-Way Authentication + +#### Description + +After this function is enabled, iSulad and image repositories communicate over HTTPS. Both iSulad and image repositories verify the validity of each other. + +#### Usage + +The corresponding registry needs to support this function and iSulad needs to be configured as follows: + +1. Modify iSulad configuration \(default path: **/etc/isulad/daemon.json**\) and set **use-decrypted-key** to **false**. +2. Place related certificates in the folder named after the registry in the **/etc/isulad/certs.d** directory. For details about how to generate certificates, visit the official Docker website: + - [https://docs.docker.com/engine/security/certificates/](https://docs.docker.com/engine/security/certificates/) + - [https://docs.docker.com/engine/security/https/](https://docs.docker.com/engine/security/https/) + +3. Run the **systemctl restart isulad** command to restart iSulad. + +#### Parameters + +Parameters can be configured in the **/etc/isulad/daemon.json** file or carried when iSulad is started. + +```shell +isulad --use-decrypted-key=false +``` + +#### Example + +Set **use-decrypted-key** to **false**. + +```shell +$ cat /etc/isulad/daemon.json +{ + "group": "isulad", + "graph": "/var/lib/isulad", + "state": "/var/run/isulad", + "engine": "lcr", + "log-level": "ERROR", + "pidfile": "/var/run/isulad.pid", + "log-opts": { + "log-file-mode": "0600", + "log-path": "/var/lib/isulad", + "max-file": "1", + "max-size": "30KB" + }, + "log-driver": "stdout", + "hook-spec": "/etc/default/isulad/hooks/default.json", + "start-timeout": "2m", + "storage-driver": "overlay2", + "storage-opts": [ + "overlay2.override_kernel_check=true" + ], + "registry-mirrors": [ + "docker.io" + ], + "insecure-registries": [ + "rnd-dockerhub.huawei.com" + ], + "pod-sandbox-image": "", + "image-opt-timeout": "5m", + "native.umask": "secure", + "network-plugin": "", + "cni-bin-dir": "", + "cni-conf-dir": "", + "image-layer-check": false, + "use-decrypted-key": false, + "insecure-skip-verify-enforce": false +} +``` + +Place the certificate in the corresponding directory. + +```shell +$ pwd +/etc/isulad/certs.d/my.csp-edge.com:5000 +$ ls +ca.crt tls.cert tls.key +``` + +Restart iSulad. + +```shell +systemctl restart isulad +``` + +Run the **pull** command to download images from the registry: + +```shell +$ isula pull my.csp-edge.com:5000/busybox +Image "my.csp-edge.com:5000/busybox" pulling +Image "my.csp-edge.com:5000/busybox@sha256:f1bdc62115dbfe8f54e52e19795ee34b4473babdeb9bc4f83045d85c7b2ad5c0" pulled +``` + +## Embedded Image Management + +### Loading Images + +#### Description + +Load images based on the **manifest** files of embedded images. The value of **--type** must be set to **embedded**. + +#### Usage + +```shell +isula load [OPTIONS] --input=FILE --type=TYPE +``` + +#### Parameters + +For details about parameters in the **load** command, see Table 5 in [Command Line Parameters](./appendix.md#command-line-parameters). + +#### Example + +```shell +$ isula load -i test.manifest --type embedded +Load image from "/root/work/bugfix/tmp/ci_testcase_data/embedded/img/test.manifest" success +``` + +### Listing Images + +#### Description + +List all images in the current environment. + +#### Usage + +```shell +isula images [OPTIONS] +``` + +#### Parameters + +For details about parameters in the **images** command, see Table 6 in [Command Line Parameters](./appendix.md#command-line-parameters). + +#### Example + +```shell +$ isula images +REF IMAGE ID CREATED SIZE +test:v1 9319da1f5233 2018-03-01 10:55:44 1.273 MB +``` + +### Inspecting Images + +#### Description + +After the configuration information of an image is returned, you can use the **-f** parameter to filter the information as needed. + +#### Usage + +```shell +isula inspect [options] CONTAINER|IMAGE [CONTAINER|IMAGE...] +``` + +#### Parameters + +For details about parameters in the **inspect** command, see Table 7 in [Command Line Parameters](./appendix.md#command-line-parameters). + +#### Example + +```shell +$ isula inspect -f "{{json .created}}" test:v1 +"2018-03-01T15:55:44.322987811Z" +``` + +### Deleting Images + +#### Description + +Delete one or more images. + +#### Usage + +```shell +isula rmi [OPTIONS] IMAGE [IMAGE...] +``` + +#### Parameters + +For details about parameters in the **rmi** command, see Table 4 in [Command Line Parameters](./appendix.md#command-line-parameters). + +#### Example + +```shell +$ isula rmi test:v1 +Image "test:v1" removed +``` diff --git a/docs/en/cloud/container_engine/isula_container_engine/installation-configuration.md b/docs/en/cloud/container_engine/isula_container_engine/installation-configuration.md new file mode 100644 index 0000000000000000000000000000000000000000..178782d1f48858f0e60822f4dbca31bf50054631 --- /dev/null +++ b/docs/en/cloud/container_engine/isula_container_engine/installation-configuration.md @@ -0,0 +1,962 @@ +# Installation and Configuration + +This chapter covers the installation, configuration, upgrade, and removal of iSulad. + +> ![](./public_sys-resources/icon-note.gif) **Note:** +> Root privilege is required for installing, upgrading, or uninstalling iSulad. + + + +- [Installation and Configuration](#installation-and-configuration) + - [Installation Methods](#installation-methods) + - [Deployment Configuration](#deployment-configuration) + + + +## Installation Methods + +iSulad can be installed by running the **yum** or **rpm** command. The **yum** command is recommended because dependencies can be installed automatically. + +This section describes two installation methods. + +- \(Recommended\) Run the following command to install iSulad: + + ```shell + sudo yum install -y iSulad + ``` + +- If the **rpm** command is used to install iSulad, you need to download and manually install the RMP packages of iSulad and all its dependencies. To install the RPM package of a single iSulad \(the same for installing dependency packages\), run the following command: + + ```shell + sudo rpm -ihv iSulad-xx.xx.xx-YYYYmmdd.HHMMSS.gitxxxxxxxx.aarch64.rpm + ``` + +## Deployment Configuration + +### Configuration Mode + +The iSulad server daemon **isulad** can be configured with a configuration file or by running the **isulad --xxx** command. The priority in descending order is as follows: CLI \> configuration file \> default configuration in code. + +> [!NOTE]NOTE +> If systemd is used to manage the iSulad process, modify the **OPTIONS** field in the **/etc/sysconfig/iSulad** file, which functions the same as using the CLI. + +- **CLI** + + During service startup, configure iSulad using the CLI. To view the configuration options, run the following command: + + ```shell + $ isulad --help + lightweight container runtime daemon + + Usage: isulad [global options] + + GLOBAL OPTIONS: + + --authorization-plugin Use authorization plugin + --cgroup-parent Set parent cgroup for all containers + --cni-bin-dir The full path of the directory in which to search for CNI plugin binaries. Default: /opt/cni/bin + --cni-conf-dir The full path of the directory in which to search for CNI config files. Default: /etc/cni/net.d + --default-ulimit Default ulimits for containers (default []) + -e, --engine Select backend engine + -g, --graph Root directory of the iSulad runtime + -G, --group Group for the unix socket(default is isulad) + --help Show help + --hook-spec Default hook spec file applied to all containers + -H, --host The socket name used to create gRPC server + --image-layer-check Check layer integrity when needed + --image-opt-timeout Max timeout(default 5m) for image operation + --insecure-registry Disable TLS verification for the given registry + --insecure-skip-verify-enforce Force to skip the insecure verify(default false) + --log-driver Set daemon log driver, such as: file + -l, --log-level Set log level, the levels can be: FATAL ALERT CRIT ERROR WARN NOTICE INFO DEBUG TRACE + --log-opt Set daemon log driver options, such as: log-path=/tmp/logs/ to set directory where to store daemon logs + --native.umask Default file mode creation mask (umask) for containers + --network-plugin Set network plugin, default is null, support null and cni + -p, --pidfile Save pid into this file + --pod-sandbox-image The image whose network/ipc namespaces containers in each pod will use. (default "rnd-dockerhub.huawei.com/library/pause-${machine}:3.0") + --registry-mirrors Registry to be prepended when pulling unqualified images, can be specified multiple times + --start-timeout timeout duration for waiting on a container to start before it is killed + -S, --state Root directory for execution state files + --storage-driver Storage driver to use(default overlay2) + -s, --storage-opt Storage driver options + --tls Use TLS; implied by --tlsverify + --tlscacert Trust certs signed only by this CA (default "/root/.iSulad/ca.pem") + --tlscert Path to TLS certificate file (default "/root/.iSulad/cert.pem") + --tlskey Path to TLS key file (default "/root/.iSulad/key.pem") + --tlsverify Use TLS and verify the remote + --use-decrypted-key Use decrypted private key by default(default true) + -V, --version Print the version + --websocket-server-listening-port CRI websocket streaming service listening port (default 10350) + ``` + + Example: Start iSulad and change the log level to DEBUG. + + ```shell + isulad -l DEBUG + ``` + +- **Configuration file** + + The iSulad configuration file is **/etc/isulad/daemon.json**. The parameters in the file are described as follows: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Parameter

+

Example

+

Description

+

Remarks

+

-e, --engine

+

"engine": "lcr"

+

iSulad runtime, which is Icr by default.

+

None

+

-G, --group

+

"group": "isulad"

+

Socket group.

+

None

+

--hook-spec

+

"hook-spec": "/etc/default/isulad/hooks/default.json"

+

Default hook configuration file for all containers.

+

None

+

-H, --host

+

"hosts": "unix:///var/run/isulad.sock"

+

Communication mode.

+

In addition to the local socket, the tcp://ip:port mode is supported. The port number ranges from 0 to 65535, excluding occupied ports.

+

--log-driver

+

"log-driver": "file"

+

Log driver configuration.

+

None

+

-l, --log-level

+

"log-level": "ERROR"

+

Log output level.

+

None

+

--log-opt

+

"log-opts": {

+

"log-file-mode": "0600",

+

"log-path": "/var/lib/isulad",

+

"max-file": "1",

+

"max-size": "30KB"

+

}

+

Log-related configuration.

+

You can specify max-file, max-size, and log-path. max-file indicates the number of log files. max-size indicates the threshold for triggering log anti-explosion. If max-file is 1, max-size is invalid. log-path specifies the path for storing log files. The log-file-mode command is used to set the permissions to read and write log files. The value must be in octal format, for example, 0666.

+

--start-timeout

+

"start-timeout": "2m"

+

Time required for starting a container.

+

None

+

--runtime

+

"default-runtime": "lcr"

+

Container runtime, which is lcr by default.

+

If neither the CLI nor the configuration file specifies the runtime, lcr is used by default. The priorities of the three specifying methods are as follows: CLI > configuration file > default value lcr. Currently, lcr and kata-runtime are supported.

+

None

+ 
"runtimes":  {
+        "kata-runtime": {
+          "path": "/usr/bin/kata-runtime",
+          "runtime-args": [
+            "--kata-config",
+            "/usr/share/defaults/kata-containers/configuration.toml"
+          ]
+        }
+    }
+

When starting a container, set this parameter to specify multiple runtimes. Runtimes in this set are valid for container startup.

+

Runtime whitelist of a container. The customized runtimes in this set are valid. kata-runtime is used as the example.

+

-p, --pidfile

+

"pidfile": "/var/run/isulad.pid"

+

File for storing PIDs.

+

This parameter is required only when more than two container engines need to be started.

+

-g, --graph

+

"graph": "/var/lib/isulad"

+

Root directory for iSulad runtimes.

+

-S, --state

+

"state": "/var/run/isulad"

+

Root directory of the execution file.

+

--storage-driver

+

"storage-driver": "overlay2"

+

Image storage driver, which is overlay2 by default.

+

Only overlay2 is supported.

+

-s, --storage-opt

+

"storage-opts": [ "overlay2.override_kernel_check=true" ]

+

Image storage driver configuration options.

+

The options are as follows:

+ 
overlay2.override_kernel_check=true #Ignore the kernel version check.
+    overlay2.size=${size} #Set the rootfs quota to ${size}.
+    overlay2.basesize=${size} #It is equivalent to overlay2.size.
+

--image-opt-timeout

+

"image-opt-timeout": "5m"

+

Image operation timeout interval, which is 5m by default.

+

The value -1 indicates that the timeout interval is not limited.

+

--registry-mirrors

+

"registry-mirrors": [ "docker.io" ]

+

Registry address.

+

None

+

--insecure-registry

+

"insecure-registries": [ ]

+

Registry without TLS verification.

+

None

+

--native.umask

+

"native.umask": "secure"

+

Container umask policy. The default value is secure. The value normal indicates insecure configuration.

+

Set the container umask value.

+

The value can be null (0027 by default), normal, or secure.

+ 
normal #The umask value of the started container is 0022.
+    secure #The umask value of the started container is 0027 (default value).
+

--pod-sandbox-image

+

"pod-sandbox-image": "rnd-dockerhub.huawei.com/library/pause-aarch64:3.0"

+

By default, the pod uses the image. The default value is rnd-dockerhub.huawei.com/library/pause-${machine}:3.0.

+

None

+

--network-plugin

+

"network-plugin": ""

+

Specifies a network plug-in. The value is a null character by default, indicating that no network configuration is available and the created sandbox has only the loop NIC.

+

The CNI and null characters are supported. Other invalid values will cause iSulad startup failure.

+

--cni-bin-dir

+

"cni-bin-dir": ""

+

Specifies the storage location of the binary file on which the CNI plug-in depends.

+

The default value is /opt/cni/bin.

+

--cni-conf-dir

+

"cni-conf-dir": ""

+

Specifies the storage location of the CNI network configuration file.

+

The default value is /etc/cni/net.d.

+

--image-layer-check=false

+

"image-layer-check": false

+

Image layer integrity check. To enable the function, set it to true; otherwise, set it to false. It is disabled by default.

+

When iSulad is started, the image layer integrity is checked. If the image layer is damaged, the related images are unavailable. iSulad cannot verify empty files, directories, and link files. Therefore, if the preceding files are lost due to a power failure, the integrity check of iSulad image data may fail to be identified. When the iSulad version changes, check whether the parameter is supported. If not, delete it from the configuration file.

+

--insecure-skip-verify-enforce=false

+

"insecure-skip-verify-enforce": false

+

Indicates whether to forcibly skip the verification of the certificate host name/domain name. The value is of the Boolean type, and the default value is false. If this parameter is set to true, the verification of the certificate host name/domain name is skipped.

+

The default value is false (not skipped). Note: Restricted by the YAJL JSON parsing library, if a non-Boolean value that meets the JSON format requirements is configured in the /etc/isulad/daemon.json configuration file, the default value used by iSulad is false.

+

--use-decrypted-key=true

+

"use-decrypted-key": true

+

Specifies whether to use an unencrypted private key. The value is of the Boolean type. If this parameter is set to true, an unencrypted private key is used. If this parameter is set to false, the encrypted private key is used, that is, two-way authentication is required.

+

The default value is true, indicating that an unencrypted private key is used. Note: Restricted by the YAJL JSON parsing library, if a non-Boolean value that meets the JSON format requirements is configured in the /etc/isulad/daemon.json configuration file, the default value used by iSulad is true.

+

--tls

+

"tls":false

+

Specifies whether to use TLS. The value is of the Boolean type.

+

This parameter is used only in -H tcp://IP:PORT mode. The default value is false.

+

--tlsverify

+

"tlsverify":false

+

Specifies whether to use TLS and verify remote access. The value is of the Boolean type.

+

This parameter is used only in -H tcp://IP:PORT mode.

+

--tlscacert

+

--tlscert

+

--tlskey

+

"tls-config": {

+

"CAFile": "/root/.iSulad/ca.pem",

+

"CertFile": "/root/.iSulad/server-cert.pem",

+

"KeyFile":"/root/.iSulad/server-key.pem"

+

}

+

TLS certificate-related configuration.

+

This parameter is used only in -H tcp://IP:PORT mode.

+

--authorization-plugin

+

"authorization-plugin": "authz-broker"

+

User permission authentication plugin.

+

Only authz-broker is supported.

+

--cgroup-parent

+

"cgroup-parent": "lxc/mycgroup"

+

Default cgroup parent path of a container, which is of the string type.

+

Specifies the cgroup parent path of a container. If --cgroup-parent is specified on the client, the client parameter prevails.

+

Note: If container A is started before container B, the cgroup parent path of container B is specified as the cgroup path of container A. When deleting a container, you need to delete container B and then container A in sequence. Otherwise, residual cgroup resources exist.

+

--default-ulimits

+

"default-ulimits": {

+

"nofile": {

+

"Name": "nofile",

+

"Hard": 6400,

+

"Soft": 3200

+

}

+

}

+

Specifies the ulimit restriction type, soft value, and hard value.

+

Specifies the restricted resource type, for example, nofile. The two field names must be the same, that is, nofile. Otherwise, an error is reported. The value of Hard must be greater than or equal to that of Soft. If the Hard or Soft field is not set, the default value 0 is used.

+

--websocket-server-listening-port

+

"websocket-server-listening-port": 10350

+

Specifies the listening port of the CRI WebSocket streaming service. The default port number is 10350.

+

Specifies the listening port of the CRI websocket streaming service.

+

If the client specifies --websocket-server-listening-port, the specified value is used. The port number ranges from 1024 to 49151.

+
+ + Example: + + ```shell + $ cat /etc/isulad/daemon.json + { + "group": "isulad", + "default-runtime": "lcr", + "graph": "/var/lib/isulad", + "state": "/var/run/isulad", + "engine": "lcr", + "log-level": "ERROR", + "pidfile": "/var/run/isulad.pid", + "log-opts": { + "log-file-mode": "0600", + "log-path": "/var/lib/isulad", + "max-file": "1", + "max-size": "30KB" + }, + "log-driver": "stdout", + "hook-spec": "/etc/default/isulad/hooks/default.json", + "start-timeout": "2m", + "storage-driver": "overlay2", + "storage-opts": [ + "overlay2.override_kernel_check=true" + ], + "registry-mirrors": [ + "docker.io" + ], + "insecure-registries": [ + "rnd-dockerhub.huawei.com" + ], + "pod-sandbox-image": "", + "image-opt-timeout": "5m", + "native.umask": "secure", + "network-plugin": "", + "cni-bin-dir": "", + "cni-conf-dir": "", + "image-layer-check": false, + "use-decrypted-key": true, + "insecure-skip-verify-enforce": false + } + ``` + + > [!TIP]NOTICE + > The default configuration file **/etc/isulad/daemon.json** is for reference only. Configure it based on site requirements. + +### Storage Description + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

File

+

Directory

+

Description

+

\*

+

/etc/default/isulad/

+

Stores the OCI configuration file and hook template file of iSulad. The file configuration permission is set to 0640, and the sysmonitor check permission is set to 0550.

+

\*

+

/etc/isulad/

+

Default configuration files of iSulad and seccomp.

+

isulad.sock

+

/var/run/

+

Pipe communication file, which is used for the communication between the client and iSulad.

+

isulad.pid

+

/var/run/

+

File for storing the iSulad PIDs. It is also a file lock to prevent multiple iSulad instances from being started.

+

\*

+

/run/lxc/

+

Lock file, which is created during iSulad running.

+

\*

+

/var/run/isulad/

+

Real-time communication cache file, which is created during iSulad running.

+

\*

+

/var/run/isula/

+

Real-time communication cache file, which is created during iSulad running.

+

\*

+

/var/lib/lcr/

+

Temporary directory of the LCR component.

+

\*

+

/var/lib/isulad/

+

Root directory where iSulad runs, which stores the created container configuration, default log path, database file, and mount point.

+

/var/lib/isulad/mnt/: mount point of the container rootfs.

+

/var/lib/isulad/engines/lcr/: directory for storing LCR container configurations. Each container has a directory named after the container.

+
+ +### Constraints + +- In high concurrency scenarios \(200 containers are concurrently started\), the memory management mechanism of Glibc may cause memory holes and large virtual memory \(for example, 10 GB\). This problem is caused by the restriction of the Glibc memory management mechanism in the high concurrency scenario, but not by memory leakage. Therefore, the memory consumption does not increase infinitely. You can set **MALLOC\_ARENA\_MAX** to reducevirtual memory error and increase the rate of reducing physical memory. However, this environment variable will cause the iSulad concurrency performance to deteriorate. Set this environment variable based on the site requirements. + + To balance performance and memory usage, set MALLOC_ARENA_MAX to 4. (The iSulad performance on the ARM64 server is affected by less than 10%.) + + Configuration method: + 1. To manually start iSulad, run the export MALLOC_ARENA_MAX=4 command and then start iSulad. + 2. If systemd manages iSulad, you can modify the /etc/sysconfig/iSulad file by adding MALLOC_ARENA_MAX=4. + +- Precautions for specifying the daemon running directories + + Take **--root** as an example. When **/new/path/** is used as the daemon new root directory, if a file exists in **/new/path/** and the directory or file name conflicts with that required by iSulad \(for example, **engines** and **mnt**\), iSulad may update the original directory or file attributes including the owner and permission. + + Therefore, please note the impact of re-specifying various running directories and files on their attributes. You are advised to specify a new directory or file for iSulad to avoid file attribute changes and security issues caused by conflicts. + +- Log file management: + + > [!TIP]NOTICE + > Log function interconnection: logs are managed by systemd as iSulad is and then transmitted to rsyslogd. By default, rsyslog restricts the log writing speed. You can add the configuration item **$imjournalRatelimitInterval 0** to the **/etc/rsyslog.conf** file and restart the rsyslogd service. + +- Restrictions on command line parameter parsing + + When the iSulad command line interface is used, the parameter parsing mode is slightly different from that of Docker. For flags with parameters in the command line, regardless of whether a long or short flag is used, only the first space after the flag or the character string after the equal sign \(=\) directly connected to the flag is used as the flag parameter. The details are as follows: + + 1. When a short flag is used, each character in the character string connected to the hyphen \(-\) is considered as a short flag. If there is an equal sign \(=\), the character string following the equal sign \(=\) is considered as the parameter of the short flag before the equal sign \(=\). + + **isula run -du=root busybox** is equivalent to **isula run -du root busybox**, **isula run -d -u=root busybox**, or **isula run -d -u root busybox**. When **isula run -du:root** is used, as **-:** is not a valid short flag, an error is reported. The preceding command is equivalent to **isula run -ud root busybox**. However, this method is not recommended because it may cause semantic problems. + + 2. When a long flag is used, the character string connected to **--** is regarded as a long flag. If the character string contains an equal sign \(=\), the character string before the equal sign \(=\) is a long flag, and the character string after the equal sign \(=\) is a parameter. + + ```shell + isula run --user=root busybox + ``` + + or + + ```shell + isula run --user root busybox + ``` + +- After an iSulad container is started, you cannot run the **isula run -i/-t/-ti** and **isula attach/exec** commands as a non-root user. +- When iSulad connects to an OCI container, only kata-runtime can be used to start the OCI container. + +### Daemon Multi-Port Binding + +#### Description + +The daemon can bind multiple UNIX sockets or TCP ports and listen on these ports. The client can interact with the daemon through these ports. + +#### Port + +Users can configure one or more ports in the hosts field in the **/etc/isulad/daemon.json** file, or choose not to specify hosts. + +```json +{ + "hosts": [ + "unix:///var/run/isulad.sock", + "tcp://localhost:5678", + "tcp://127.0.0.1:6789" + ] +} +``` + +Users can also run the **-H** or **--host** command in the **/etc/sysconfig/iSulad** file to configure a port, or choose not to specify hosts. + +```ini +OPTIONS='-H unix:///var/run/isulad.sock --host tcp://127.0.0.1:6789' +``` + +If hosts are not specified in the **daemon.json** file and iSulad, the daemon listens on **unix:///var/run/isulad.sock** by default after startup. + +#### Restrictions + +- Users cannot specify hosts in the **/etc/isulad/daemon.json** and **/etc/sysconfig/iSuald** files at the same time. Otherwise, an error will occur and iSulad cannot be started. + + ```text + unable to configure the isulad with file /etc/isulad/daemon.json: the following directives are specified both as a flag and in the configuration file: hosts: (from flag: [unix:///var/run/isulad.sock tcp://127.0.0.1:6789], from file: [unix:///var/run/isulad.sock tcp://localhost:5678 tcp://127.0.0.1:6789]) + ``` + +- If the specified host is a UNIX socket, the socket must start with **unix://** followed by a valid absolute path. +- If the specified host is a TCP port, the TCP port number must start with **tcp://** followed by a valid IP address and port number. The IP address can be that of the local host. +- A maximum of 10 valid ports can be specified. If more than 10 ports are specified, an error will occur and iSulad cannot be started. + +### Configuring TLS Authentication and Enabling Remote Access + +#### Description + +iSulad is designed in C/S mode. By default, the iSulad daemon process listens only on the local/var/run/isulad.sock. Therefore, you can run commands to operate containers only on the local client iSula. To enable iSula's remote access to the container, the iSulad daemon process needs to listen on the remote access port using TCP/IP. However, listening is performed only by simply configuring tcp ip:port. In this case, all IP addresses can communicate with iSulad by calling **isula -H tcp://**_remote server IP address_**:port**, which may cause security problems. Therefore, it is recommended that a more secure version, namely Transport Layer Security \(TLS\), be used for remote access. + +#### Generating TLS Certificate + +- Example of generating a plaintext private key and certificate + + ```shell + #!/bin/bash + set -e + echo -n "Enter pass phrase:" + read password + echo -n "Enter public network ip:" + read publicip + echo -n "Enter host:" + read HOST + + echo " => Using hostname: $publicip, You MUST connect to iSulad using this host!" + + mkdir -p $HOME/.iSulad + cd $HOME/.iSulad + rm -rf $HOME/.iSulad/* + + echo " => Generating CA key" + openssl genrsa -passout pass:$password -aes256 -out ca-key.pem 4096 + echo " => Generating CA certificate" + openssl req -passin pass:$password -new -x509 -days 365 -key ca-key.pem -sha256 -out ca.pem -subj "/C=CN/ST=zhejiang/L=hangzhou/O=Huawei/OU=iSulad/CN=iSulad@huawei.com" + echo " => Generating server key" + openssl genrsa -passout pass:$password -out server-key.pem 4096 + echo " => Generating server CSR" + openssl req -passin pass:$password -subj /CN=$HOST -sha256 -new -key server-key.pem -out server.csr + echo subjectAltName = DNS:$HOST,IP:$publicip,IP:127.0.0.1 >> extfile.cnf + echo extendedKeyUsage = serverAuth >> extfile.cnf + echo " => Signing server CSR with CA" + openssl x509 -req -passin pass:$password -days 365 -sha256 -in server.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out server-cert.pem -extfile extfile.cnf + echo " => Generating client key" + openssl genrsa -passout pass:$password -out key.pem 4096 + echo " => Generating client CSR" + openssl req -passin pass:$password -subj '/CN=client' -new -key key.pem -out client.csr + echo " => Creating extended key usage" + echo extendedKeyUsage = clientAuth > extfile-client.cnf + echo " => Signing client CSR with CA" + openssl x509 -req -passin pass:$password -days 365 -sha256 -in client.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out cert.pem -extfile extfile-client.cnf + rm -v client.csr server.csr extfile.cnf extfile-client.cnf + chmod -v 0400 ca-key.pem key.pem server-key.pem + chmod -v 0444 ca.pem server-cert.pem cert.pem + ``` + +- Example of generating an encrypted private key and certificate request file + + ```shell + #!/bin/bash + + echo -n "Enter public network ip:" + read publicip + echo -n "Enter pass phrase:" + read password + + # remove certificates from previous execution. + rm -f *.pem *.srl *.csr *.cnf + + + # generate CA private and public keys + echo 01 > ca.srl + openssl genrsa -aes256 -out ca-key.pem -passout pass:$password 2048 + openssl req -subj '/C=CN/ST=zhejiang/L=hangzhou/O=Huawei/OU=iSulad/CN=iSulad@huawei.com' -new -x509 -days $DAYS -passin pass:$password -key ca-key.pem -out ca.pem + + # create a server key and certificate signing request (CSR) + openssl genrsa -aes256 -out server-key.pem -passout pass:$PASS 2048 + openssl req -new -key server-key.pem -out server.csr -passin pass:$password -subj '/CN=iSulad' + + echo subjectAltName = DNS:iSulad,IP:${publicip},IP:127.0.0.1 > extfile.cnf + echo extendedKeyUsage = serverAuth >> extfile.cnf + # sign the server key with our CA + openssl x509 -req -days $DAYS -passin pass:$password -in server.csr -CA ca.pem -CAkey ca-key.pem -out server-cert.pem -extfile extfile.cnf + + # create a client key and certificate signing request (CSR) + openssl genrsa -aes256 -out key.pem -passout pass:$password 2048 + openssl req -subj '/CN=client' -new -key key.pem -out client.csr -passin pass:$password + + # create an extensions config file and sign + echo extendedKeyUsage = clientAuth > extfile.cnf + openssl x509 -req -days 365 -passin pass:$password -in client.csr -CA ca.pem -CAkey ca-key.pem -out cert.pem -extfile extfile.cnf + + # remove the passphrase from the client and server key + openssl rsa -in server-key.pem -out server-key.pem -passin pass:$password + openssl rsa -in key.pem -out key.pem -passin pass:$password + + # remove generated files that are no longer required + rm -f ca-key.pem ca.srl client.csr extfile.cnf server.csr + ``` + +#### APIs + +```json +{ + "tls": true, + "tls-verify": true, + "tls-config": { + "CAFile": "/root/.iSulad/ca.pem", + "CertFile": "/root/.iSulad/server-cert.pem", + "KeyFile":"/root/.iSulad/server-key.pem" + } +} +``` + +#### Restrictions + +The server supports the following modes: + +- Mode 1 \(client verified\): tlsverify, tlscacert, tlscert, tlskey +- Mode 2 \(client not verified\): tls, tlscert, tlskey + +The client supports the following modes: + +- Mode 1 \(verify the identity based on the client certificate, and verify the server based on the specified CA\): tlsverify, tlscacert, tlscert, tlskey +- Mode 2 \(server verified\): tlsverify, tlscacert + +Mode 1 is used for the server, and mode 2 for the client if the two-way authentication mode is used for communication. + +Mode 2 is used for the server and the client if the unidirectional authentication mode is used for communication. + +> [!TIP]NOTICE +> +> - If RPM is used for installation, the server configuration can be modified in the **/etc/isulad/daemon.json** and **/etc/sysconfig/iSulad** files. +> - Two-way authentication is recommended as it is more secure than non-authentication or unidirectional authentication. +> - GRPC open-source component logs are not taken over by iSulad. To view gRPC logs, set the environment variables **gRPC\_VERBOSITY** and **gRPC\_TRACE** as required. + +#### Example + +On the server: + +```shell + isulad -H=tcp://0.0.0.0:2376 --tlsverify --tlscacert ~/.iSulad/ca.pem --tlscert ~/.iSulad/server-cert.pem --tlskey ~/.iSulad/server-key.pem +``` + +On the client: + +```shell + isula version -H=tcp://$HOSTIP:2376 --tlsverify --tlscacert ~/.iSulad/ca.pem --tlscert ~/.iSulad/cert.pem --tlskey ~/.iSulad/key.pem +``` + +### devicemapper Storage Driver Configuration + +To use the devicemapper storage driver, you need to configure a thinpool device which requires an independent block device with sufficient free space. Take the independent block device **/dev/xvdf** as an example. The configuration method is as follows. + +#### Configuring a Thinpool + +1. Stop the iSulad service. + + ```shell + # systemctl stop isulad + ``` + +2. Create a logical volume manager \(LVM\) volume based on the block device. + + ```shell + # pvcreate /dev/xvdf + ``` + +3. Create a volume group based on the created physical volume. + + ```shell + # vgcreate isula /dev/xvdf + Volume group "isula" successfully created: + ``` + +4. Create two logical volumes named **thinpool** and **thinpoolmeta**. + + ```shell + # lvcreate --wipesignatures y -n thinpool isula -l 95%VG + Logical volume "thinpool" created. + ``` + + ```shell + # lvcreate --wipesignatures y -n thinpoolmeta isula -l 1%VG + Logical volume "thinpoolmeta" created. + ``` + +5. Convert the two logical volumes into a thinpool and the metadata used by the thinpool. + + ```shell + # lvconvert -y --zero n -c 512K --thinpool isula/thinpool --poolmetadata isula/thinpoolmeta + + WARNING: Converting logical volume isula/thinpool and isula/thinpoolmeta to + thin pool's data and metadata volumes with metadata wiping. + THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.) + Converted isula/thinpool to thin pool. + ``` + +#### Modifying the iSulad Configuration Files + +1. If iSulad has been used in the environment, back up the running data first. + + ```shell + # mkdir /var/lib/isulad.bk + # mv /var/lib/isulad/* /var/lib/isulad.bk + ``` + +2. Modify configuration files. + + Two configuration methods are provided. Select one based on site requirements. + - Edit the **/etc/isulad/daemon.json** file, set **storage-driver** to **devicemapper**, and set parameters related to the **storage-opts** field. For details about related parameters, see [Parameter Description](#en-us_topic_0222861454_section1712923715282). The following lists the configuration reference: + + ```json + { + "storage-driver": "devicemapper" + "storage-opts": [ + "dm.thinpooldev=/dev/mapper/isula-thinpool", + "dm.fs=ext4", + "dm.min_free_space=10%" + ] + } + ``` + + - Edit **/etc/sysconfig/iSulad** to explicitly specify related iSulad startup parameters. For details about related parameters, see [Parameter Description](#en-us_topic_0222861454_section1712923715282). The following lists the configuration reference: + + ```ini + OPTIONS="--storage-driver=devicemapper --storage-opt dm.thinpooldev=/dev/mapper/isula-thinpool --storage-opt dm.fs=ext4 --storage-opt dm.min_free_space=10%" + ``` + +3. Start iSulad for the settings to take effect. + + ```shell + # systemctl start isulad + ``` + +#### Parameter Description + +For details about parameters supported by storage-opts, see [Table 1](#en-us_topic_0222861454_table3191161993812). + +**Table 1** Parameter description + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Parameter

+

Mandatory or Not

+

Description

+

dm.fs

+

Yes

+

Specifies the type of the file system used by a container. This parameter must be set to ext4, that is, dm.fs=ext4.

+

dm.basesize

+

No

+

Specifies the maximum storage space of a single container. The unit can be k, m, g, t, or p. An uppercase letter can also be used, for example, dm.basesize=50G. This parameter is valid only during the first initialization.

+

dm.mkfsarg

+

No

+

Specifies the additional mkfs parameters when a basic device is created. For example: dm.mkfsarg=-O ^has_journal

+

dm.mountopt

+

No

+

Specifies additional mount parameters when a container is mounted. For example: dm.mountopt=nodiscard

+

dm.thinpooldev

+

No

+

Specifies the thinpool device used for container or image storage.

+

dm.min_free_space

+

No

+

Specifies minimum percentage of reserved space. For example, dm.min_free_space=10% indicates that storage-related operations such as container creation will fail when the remaining storage space falls below 10%.

+
+ +#### Precautions + +- When configuring devicemapper, if the system does not have sufficient space for automatic capacity expansion of thinpool, disable the automatic capacity expansion function. + + To disable automatic capacity expansion, set both **thin\_pool\_autoextend\_threshold** and **thin\_pool\_autoextend\_percent** in the **/etc/lvm/profile/isula-thinpool.profile** file to **100**. + + ```text + activation { + thin_pool_autoextend_threshold=100 + thin_pool_autoextend_percent=100 + } + ``` + +- When devicemapper is used, use Ext4 as the container file system. You need to add **--storage-opt dm.fs=ext4** to the iSulad configuration parameters. +- If graphdriver is devicemapper and the metadata files are damaged and cannot be restored, you need to manually restore the metadata files. Do not directly operate or tamper with metadata of the devicemapper storage driver in Docker daemon. +- When the devicemapper LVM is used, if the devicemapper thinpool is damaged due to abnormal power-off, you cannot ensure the data integrity or whether the damaged thinpool can be restored. Therefore, you need to rebuild the thinpool. + +**Precautions for Switching the devicemapper Storage Pool When the User Namespace Feature Is Enabled on iSula** + +- Generally, the path of the deviceset-metadata file is **/var/lib/isulad/devicemapper/metadata/deviceset-metadata** during container startup. +- If user namespaces are used, the path of the deviceset-metadata file is **/var/lib/isulad/**_userNSUID.GID_**/devicemapper/metadata/deviceset-metadata**. +- When you use the devicemapper storage driver and the container is switched between the user namespace scenario and common scenario, the **BaseDeviceUUID** content in the corresponding deviceset-metadata file needs to be cleared. In the thinpool capacity expansion or rebuild scenario, you also need to clear the **BaseDeviceUUID** content in the deviceset-metadata file. Otherwise, the iSulad service fails to be restarted. diff --git a/docs/en/cloud/container_engine/isula_container_engine/installation-upgrade-Uninstallation.md b/docs/en/cloud/container_engine/isula_container_engine/installation-upgrade-Uninstallation.md new file mode 100644 index 0000000000000000000000000000000000000000..b857929197ba92fb9abb7a7bf8168703ffe02e14 --- /dev/null +++ b/docs/en/cloud/container_engine/isula_container_engine/installation-upgrade-Uninstallation.md @@ -0,0 +1,3 @@ +# Installation, Upgrade and Uninstallation + +This chapter describes how to install, configure, upgrade, and uninstall iSulad. diff --git a/docs/en/cloud/container_engine/isula_container_engine/interconnecting-isula-shim-v2-with-stratovirt.md b/docs/en/cloud/container_engine/isula_container_engine/interconnecting-isula-shim-v2-with-stratovirt.md new file mode 100644 index 0000000000000000000000000000000000000000..58b0b7a6a5380e11805f1a76fde3d4b9ee9d003f --- /dev/null +++ b/docs/en/cloud/container_engine/isula_container_engine/interconnecting-isula-shim-v2-with-stratovirt.md @@ -0,0 +1,219 @@ +# Interconnecting iSula with the shim v2 Secure Container + +## Overview + +shim v2 is a next-generation shim solution. Compared with shim v1, shim v2 features shorter call chains, clearer architecture, and lower memory overhead in multi-service container scenarios. iSula can run secure containers through isulad-shim or containerd-shim-kata-v2. The isulad-shim component is the implementation of the shim v1 solution, and the containerd-shim-kata-v2 component is the implementation of the shim v2 solution in the secure container scenario. This document describes how to interconnect iSula with containerd-shim-kata-v2. + +## Interconnecting with containerd-shim-v2-kata + +### Prerequisites + +Before interconnecting iSula with containerd-shim-v2-kata, ensure that the following prerequisites are met: + +- iSulad, lib-shim-v2, and kata-containers have been installed. +- StratoVirt supports only the devicemapper storage driver. Therefore, you need to configure the devicemapper environment and ensure that the devicemapper storage driver used by iSulad works properly. + +### Environment Setup + +The following describes how to install and configure iSulad and kata-containers. + +#### Installing Dependencies + +Configure the YUM source based on the OS version and install iSulad, lib-shim-v2, and kata-containers as the **root** user. + +```shell +# yum install iSulad +# yum install kata-containers +# yum install lib-shim-v2 +``` + +#### Creating and Configuring a Storage Device + +Prepare a drive, for example, **/dev/sdx**. The drive will be formatted. This section uses the block device **/dev/sda** as an example. + +I. Creating devicemapper + +1. Create a physical volume (PV). + + ```shell + $ pvcreate /dev/sda + Physical volume "/dev/loop0" successfully created. + ``` + +2. Create a volume group (VG). + + ```shell + $ vgcreate isula /dev/sda + Volume group "isula" successfully created + ``` + +3. Create the logical volumes **thinpool** and **thinpoolmeta**. + + ```shell + $ lvcreate --wipesignatures y -n thinpool isula -l 95%VG + Logical volume "thinpool" created. + + $ lvcreate --wipesignatures y -n thinpoolmeta isula -l 1%VG + Logical volume "thinpoolmeta" created. + ``` + +4. Convert the created logical volumes to a thin pool. + + ```shell + $ lvconvert -y --zero n -c 64K \ + --thinpool isula/thinpool \ + --poolmetadata isula/thinpoolmeta + Thin pool volume with chunk size 512.00 KiB can address at most 126.50 TiB of data. + WARNING: Converting isula/thinpool and isula/thinpoolmeta to thin pool's data and metadata volumes with metadata wiping. + THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.) + Converted isula/thinpool and isula/thinpoolmeta to thin pool. + ``` + +5. Configure automatic extension of the thin pool using lvm. + + ```shell + $ touch /etc/lvm/profile/isula-thinpool.profile + $ cat << EOF > /etc/lvm/profile/isula-thinpool.profile + activation { + thin_pool_autoextend_threshold=80 + thin_pool_autoextend_percent=20 + } + EOF + $ lvchange --metadataprofile isula-thinpool isula/thinpool + Logical volume isula/thinpool changed. + ``` + +II. Changing the iSulad Storage Driver Type and Setting the Default Runtime + +Modify the **/etc/isulad/daemon.json** configuration file. Set **default-runtime** to **io.containerd.kata.v2** and **storage-driver** to **devicemapper**. The modification result is as follows: + +```json + { + "default-runtime": "io.containerd.kata.v2", + "storage-driver": "devicemapper", + "storage-opts": [ + "dm.thinpooldev=/dev/mapper/isula-thinpool", + "dm.fs=ext4", + "dm.min_free_space=10%" + ], +} +``` + +III. Making the Configuration Take Effect + +1. Restart the iSulad for the configuration to take effect. + + ```shell + # systemctl daemon-reload + # systemctl restart isulad + ``` + +2. Check whether the iSula storage driver is successfully configured. + + ```shell + # isula info + ``` + + If the following information is displayed, the configuration is successful: + + ```shell + Storage Driver: devicemapper + ``` + +### Interconnection Guide + +This section describes how to interconnect iSula with containerd-shim-kata-v2. + +By default, containerd-shim-kata-v2 uses QEMU as the virtualization component. The following describes how to configure QEMU and StratoVirt. + +#### Using QEMU + +If containerd-shim-kata-v2 uses QEMU as the virtualization component, perform the following operations to interconnect iSula with containerd-shim-kata-v2: + +1. Modify the kata configuration file **/usr/share/defaults/kata-containers/configuration.toml**. + + Set **sandbox_cgroup_with_emulator** to **false**. Currently, shim v2 does not support this function. Other parameters are the same as the kata configuration parameters in shim v1 or use the default values. + + ```toml + sandbox_cgroup_with_emulator = false + ``` + +2. Use the BusyBox image to run the secure container and check whether the used runtime is io.containerd.kata.v2. + + ```bash + $ id=`isula run -tid busybox /bin/sh` + $ isula inspect -f '{{ json .HostConfig.Runtime }}' $id + "io.containerd.kata.v2" + ``` + +3. Verify that the QEMU-based VM process is started. If it is started, QEMU is successfully interconnected with the shim v2 secure container. + + ```bash + ps -ef | grep qemu + ``` + +#### Using StratoVirt + +If containerd-shim-kata-v2 uses StratoVirt as the virtualization component, perform the following operations to interconnect iSula with containerd-shim-kata-v2: + +1. Create the **stratovirt.sh** script in any directory (for example, **/home**) and add the execute permission to the file as the **root** user. + + ```shell + # touch /home/stratovirt.sh + # chmod +x /home/stratovirt.sh + ``` + + The content of **stratovirt.sh** is as follows, which is used to specify the path of StratoVirt: + + ```shell + #!/bin/bash + export STRATOVIRT_LOG_LEVEL=info # set log level which includes trace, debug, info, warn and error. + /usr/bin/stratovirt $@ + ``` + +2. Modify the kata configuration file. Set **hypervisor** of the secure container to **stratovirt**, **kernel** to the absolute path of the StratoVirt kernel image, and **initrd** to the initrd image file of kata-containers (if you use YUM to install kata-containers, the initrd image file is downloaded by default and stored in the **/var/lib/kata/** directory). StratoVirt supports only the devicemapper storage mode, prepare the environment in advance and set iSulad to the devicemapper mode. + + The configurations are as follows: + + ```shell + [hypervisor.stratovirt] + path = "/home/stratovirt.sh" + kernel = "/var/lib/kata/vmlinux.bin" + initrd = "/var/lib/kata/kata-containers-initrd.img" + block_device_driver = "virtio-mmio" + use_vsock = true + enable_netmon = true + internetworking_model="tcfilter" + sandbox_cgroup_with_emulator = false + disable_new_netns = false + disable_block_device_use = false + disable_vhost_net = true + ``` + + To use the vsock function in StratoVirt, enable the vhost_vsock kernel module and check whether the module is successfully enabled. + + ```bash + modprobe vhost_vsock + lsmod |grep vhost_vsock + ``` + + Download the kernel of the required version and architecture and save it to the **/var/lib/kata/** directory. For example, download the [openeuler repo](https://repo.openeuler.org/) of the x86 architecture of openEuler 21.03. + + ```bash + cd /var/lib/kata + wget https://archives.openeuler.openatom.cn/openEuler-21.03/stratovirt_img/x86_64/vmlinux.bin + ``` + +3. Use the BusyBox image to run the secure container and check whether the used runtime is io.containerd.kata.v2. + + ```bash + $ id=`isula run -tid busybox sh` + $ isula inspect -f '{{ json .HostConfig.Runtime }}' $id + "io.containerd.kata.v2" + ``` + +4. Verify that the StratoVirt-based VM process is started. If it is started, StratoVirt is successfully interconnected with the shim v2 secure container. + + ```bash + ps -ef | grep stratovirt + ``` diff --git a/docs/en/cloud/container_engine/isula_container_engine/interconnection-with-the-cni-network.md b/docs/en/cloud/container_engine/isula_container_engine/interconnection-with-the-cni-network.md new file mode 100644 index 0000000000000000000000000000000000000000..37d5b315af41a336f9069b4374a7fdd962d409ca --- /dev/null +++ b/docs/en/cloud/container_engine/isula_container_engine/interconnection-with-the-cni-network.md @@ -0,0 +1,123 @@ +# Interconnection with the CNI Network + +- [Interconnection with the CNI Network](#interconnection-with-the-cni-network) + - [Overview](#overview) + - [Common CNIs](#common-cnis) + - [CNI Network Configuration Description](#cni-network-configuration-description) + - [Adding a Pod to the CNI Network List](#adding-a-pod-to-the-cni-network-list) + - [Removing a Pod from the CNI Network List](#removing-a-pod-from-the-cni-network-list) + - [Usage Restrictions](#usage-restrictions) + +## Overview + +The container runtime interface \(CRI\) is provided to connect to the CNI network, including parsing the CNI network configuration file and adding or removing a pod to or from the CNI network. When a pod needs to support a network through a container network plug-in such as Canal, the CRI needs to be interconnected to Canal so as to provide the network capability for the pod. + +## Common CNIs + +Common CNIs include CNI network configuration items in the CNI network configuration and pod configuration. These CNIs are visible to users. + +- CNI network configuration items in the CNI network configuration refer to those used to specify the path of the CNI network configuration file, path of the binary file of the CNI network plug-in, and network mode. For details, see [Table 1](#en-us_topic_0183259146_table18221919589). +- CNI network configuration items in the pod configuration refer to those used to set the additional CNI network list to which the pod is added. By default, the pod is added only to the default CNI network plane. You can add the pod to multiple CNI network planes as required. + +**Table 1** CNI network configuration items + + + + + + + + + + + + + + + + + + + + + + + + +

Function

+

Command

+

Configuration File

+

Description

+

Path of the binary file of the CNI network plug-in

+

--cni-bin-dir

+

"cni-bin-dir": "",

+

The default value is /opt/cni/bin.

+

Path of the CNI network configuration file

+

--cni-conf-dir

+

"cni-conf-dir": "",

+

The system traverses all files with the extension .conf, .conflist, or .json in the directory. The default value is /etc/cni/net.d.

+

Network mode

+

--network-plugin

+

"network-plugin": "",

+

Specifies a network plug-in. The value is a null character by default, indicating that no network configuration is available and the created sandbox has only the loop NIC. The CNI and null characters are supported. Other invalid values will cause iSulad startup failure.

+
+ +Additional CNI network configuration mode: + +Add the network plane configuration item "network.alpha.kubernetes.io/network" to annotations in the pod configuration file. + +The network plane is configured in JSON format, including: + +- **name**: specifies the name of the CNI network plane. +- **interface**: specifies the name of a network interface. + +The following is an example of the CNI network configuration method: + +```json +"annotations" : { + "network.alpha.kubernetes.io/network": "{\"name\": \"mynet\", \"interface\": \"eth1\"}" + } +``` + +### CNI Network Configuration Description + +The CNI network configuration includes two types, both of which are in the .json file format. + +- Single-network plane configuration file with the file name extension .conf or .json. For details about the configuration items, see [Table 1](#cni-parameters.md#en-us_topic_0184347952_table425023335913) in the appendix. +- Multi-network plane configuration file with the file name extension .conflist. For details about the configuration items, see [Table 3](#cni-parameters.md#en-us_topic_0184347952_table657910563105) in the appendix. + +### Adding a Pod to the CNI Network List + +If **--network-plugin=cni** is configured for iSulad and the default network plane is configured, a pod is automatically added to the default network plane when the pod is started. If the additional network configuration is configured in the pod configuration, the pod is added to these additional network planes when the pod is started. + +**port\_mappings** in the pod configuration is also a network configuration item, which is used to set the port mapping of the pod. To set port mapping, perform the following steps: + +```json +"port_mappings":[ + { + "protocol": 1, + "container_port": 80, + "host_port": 8080 + } +] +``` + +- **protocol**: protocol used for mapping. The value can be **tcp** \(identified by 0\) or **udp** \(identified by 1\). +- **container\_port**: port through which the container is mapped. +- **host\_port**: port mapped to the host. + +### Removing a Pod from the CNI Network List + +When StopPodSandbox is called, the interface for removing a pod from the CNI network list will be called to clear network resources. + +> [!NOTE]NOTE + +1. Before calling the RemovePodSandbox interface, you must call the StopPodSandbox interface at least once. +2. If StopPodSandbox fails to call the CNI, residual network resources may exist. + +## Usage Restrictions + +- Currently, only CNI 0.3.0 and CNI 0.3.1 are supported. In later versions, CNI 0.1.0 and CNI 0.2.0 may need to be supported. Therefore, when error logs are displayed, the information about CNI 0.1.0 and CNI 0.2.0 is reserved. +- name: The value must contain lowercase letters, digits, hyphens \(-\), and periods \(.\) and cannot be started or ended with a hyphen or period. The value can contain a maximum of 200 characters. +- The number of configuration files cannot exceed 200, and the size of a single configuration file cannot exceed 1 MB. +- The extended parameters need to be configured based on the actual network requirements. Optional parameters do not need to be written into the netconf.json file. diff --git a/docs/en/cloud/container_engine/isula_container_engine/isula-faqs.md b/docs/en/cloud/container_engine/isula_container_engine/isula-faqs.md new file mode 100644 index 0000000000000000000000000000000000000000..6eb0cd216772695d26f04cd42b749bf7883947b1 --- /dev/null +++ b/docs/en/cloud/container_engine/isula_container_engine/isula-faqs.md @@ -0,0 +1,21 @@ +# Common Issues and Solutions + +## Issue 1: Changing iSulad Default Runtime to `lxc` Causes Container Startup Error: Failed to Initialize Engine or Runtime + +**Cause**: iSulad uses `runc` as its default runtime. Switching to `lxc` without the required dependencies causes this issue. + +**Solution**: To set `lxc` as the default runtime, install the `lcr` and `lxc` packages. Then, either configure the `runtime` field in the iSulad configuration file to `lcr` or use the `--runtime lcr` flag when launching containers. Avoid uninstalling `lcr` or `lxc` after starting containers, as this may leave behind residual resources during container deletion. + +## Issue 2: Error When Using iSulad CRI V1 Interface: rpc error: code = Unimplemented desc = + +**Cause**: iSulad supports both CRI V1alpha2 and CRI V1 interfaces, with CRI V1alpha2 enabled by default. Using CRI V1 requires explicit configuration. + +**Solution**: Enable the CRI V1 interface by modifying the iSulad configuration file at **/etc/isulad/daemon.json**. + +```json +{ + "enable-cri-v1": true, +} +``` + +When compiling iSulad from source, include the `cmake` option `-D ENABLE_CRI_API_V1=ON` to enable CRI V1 support. diff --git a/docs/en/cloud/container_engine/isula_container_engine/isulad-support-cdi.md b/docs/en/cloud/container_engine/isula_container_engine/isulad-support-cdi.md new file mode 100644 index 0000000000000000000000000000000000000000..e7e3cd7ab551a524f2f898fbb3c018d4cc0ea422 --- /dev/null +++ b/docs/en/cloud/container_engine/isula_container_engine/isulad-support-cdi.md @@ -0,0 +1,120 @@ +# iSulad Support for CDI + +## Overview + +Container Device Interface (CDI) is a container runtime specification used to support third-party devices. + +CDI solves the following problems: +In Linux, only one device node needed to be exposed in a container in the past to enable device awareness of the container. However, as devices and software become more complex, vendors want to perform more operations, such as: + +- Exposing multiple device nodes to a container, mounting files from a runtime namespace to a container, or hiding procfs entries. +- Checking the compatibility between containers and devices. For example, checking whether a container can run on a specified device. +- Performing runtime-specific operations, such as virtual machines and Linux container-based runtimes. +- Performing device-specific operations, such as GPU memory cleanup and FPGA re-programming. + +In the absence of third-party device standards, vendors often have to write and maintain multiple plugins for different runtimes, or even contribute vendor-specific code directly in a runtime. In addition, the runtime does not expose the plugin system in a unified manner (or even not at all), resulting in duplication of functionality in higher-level abstractions (such as Kubernetes device plugins). + +To solve the preceding problem, CDI provides the following features: +CDI describes a mechanism that allows third-party vendors to interact with devices without modifying the container runtime. + +The mechanism is exposed as a JSON file (similar to the container network interface CNI), which allows vendors to describe the operations that the container runtime should perform on the OCI-based container. + +Currently, iSulad supports the [CDI v0.6.0](https://github.com/cncf-tags/container-device-interface/blob/v0.6.0/SPEC.md) specification. + +## Configuring iSulad to Support CDI + +Modify the **daemon.json** file as follows and restart iSulad: + +```json +{ + ... + "enable-cri-v1": true, + "cdi-spec-dirs": ["/etc/cdi", "/var/run/cdi"], + "enable-cdi": true +} +``` + +**cdi-spec-dirs** specifies the directory where CDI specifications are stored. If this parameter is not specified, the default value **/etc/cdi** or **/var/run/cdi** is used. + +## Examples + +### CDI Specification Example + +For details about each field, see [CDI v0.6.0](https://github.com/cncf-tags/container-device-interface/blob/v0.6.0/SPEC.md). + +```bash +$ mkdir /etc/cdi +$ cat > /etc/cdi/vendor.json < + +- [Local Volume Management](#local-volume-management) + - [Overview](#overview) + - [Precautions](#precautions) + - [Usage](#usage) + - [Using the -v Option to Mount Data](#using-the--v-option-to-mount-data) + - [Format](#format) + - [Functions](#functions) + - [Parameter Description](#parameter-description) + - [Examples](#examples) + - [Using the --mount Option to Mount Data](#using-the---mount-option-to-mount-data) + - [Format](#format-1) + - [Functions](#functions-1) + - [Parameter Description](#parameter-description-1) + - [Examples](#examples-1) + - [Reusing the Mounting Configuration in Other Containers](#reusing-the-mounting-configuration-in-other-containers) + - [Format](#format-2) + - [Functions](#functions-2) + - [Parameter Description](#parameter-description-2) + - [Examples](#examples-2) + - [Using the Anonymous Volume in an Image](#using-the-anonymous-volume-in-an-image) + - [Querying a Volume](#querying-a-volume) + - [Format](#format-3) + - [Functions](#functions-3) + - [Parameter Description](#parameter-description-3) + - [Examples](#examples-3) + - [Deleting a Volume](#deleting-a-volume) + - [Format](#format-4) + - [Functions](#functions-4) + - [Parameter Description](#parameter-description-4) + - [Examples](#examples-4) + - [Precautions](#precautions-1) + - [Conflict Combination Rules](#conflict-combination-rules) + - [Differences Between iSula and Docker](#differences-between-isula-and-docker) + + + +## Overview + +After a container managed by iSula is destroyed, all data in the container is destroyed. If you want to retain data after the container is destroyed, a data persistence mechanism is required. iSula allows files, directories, or volumes on a host to be mounted to a container at runtime. You can write the data to be persisted to the mount point in the container. After the container is destroyed, the files, directories, and volumes on the host are retained. If you need to delete a file, directory, or volume on the host, you can manually delete the file or directory, or run the iSula command to delete the volume. Currently, the iSula supports only local volume management. Local volumes are classified into named volumes and anonymous volumes. A volume whose name is specified by a user is called a named volume. If a user does not specify a name for a volume, iSula automatically generates a name (a 64-bit random number) for the volume, that is, an anonymous volume. + +The following describes how to use iSula to manage local volumes. + +## Precautions + +- The volume name contains 2 to 64 characters and complies with the regular expression `^[a-zA-Z0-9][a-zA-Z0-9_.-]{1,63}$`. That is, the first character of the volume name must be a letter or digit, and other characters can be letters, digits, underscores (_), periods (.), and hyphens (-). +- During container creation, if data exists at the mount point of the container corresponding to the volume, the data is copied to the volume by default. If the iSula breaks down or restarts or the system is powered off during the copy process, the data in the volume may be incomplete. In this case, you need to manually delete the volume or the data in the volume to ensure that the data is correct and complete. + +## Usage + +### Using the -v Option to Mount Data + +#### Format + +```shell +isula run -v [SRC:]DST[:MODE,MODE...] IMAGE +``` + +#### Functions + +When you create and run a container, use the -v/--volume option to mount the files, directories, or volumes on the host to the container for data persistence. + +#### Parameter Description + +- SRC: Path of the file, directory, or volume to be mounted on the host. If the value is an absolute path, a file or folder on the host is mounted. If the value is a volume name, a volume is mounted. If this parameter is not specified, an anonymous volume is mounted. If a folder or volume does not exist, iSula creates a folder or volume and then mounts it. +- DST: Mount path in the container. The value must be an absolute path. +- MODE: When the source to be mounted is a directory or file, the valid parameters are ro, rw, z, Z, private, rprivate, slave, rslave, shared, and rshared. Only one parameter of the same type can be configured. If the source is a volume, the valid parameters are ro, rw, z, Z, and nocopy. Only one parameter of the same type can be configured. Use commas (,) to separate multiple attributes. The parameters are described as follows: + +| Parameter | Description | +| -------- | -----------------------------------------------| +| ro | The mount point in the container is mounted in read-only mode. | +| rw | The mount point in the container is mounted in read/write mode. | +| z | If SELinux is enabled, add the SELinux share label during mounting. | +| Z | If SELinux is enabled, add the SELinux private label during mounting. | +| private | The mount point in the container is mounted in private propagation mode. | +| rprivate | The mount point in the container is recursively mounted in private propagation mode. | +| slave | The mount point in the container is mounted in subordinate propagation mode. | +| rslave | The mount point in the container is recursively mounted in subordinate propagation mode. | +| shared | The mount point in the container is mounted in shared propagation mode. | +| rshared | The mount point in the container is recursively mounted in shared propagation mode. | +| nocopy | Data at the mount point is not copied. If this parameter is not set, data is copied by default. In addition, if data already exists in the volume, the data will not be copied. | + +#### Examples + +Run the container based on BusyBox, create or mount a volume named vol to the /vol directory of the container, and set the mount point to read-only. In addition, if data exists at the mount point in the container, the data is not copied. + +```shell +isula run -v vol:/vol:ro,nocopy busybox +``` + +### Using the --mount Option to Mount Data + +#### Format + +```shell +isula run --mount [type=TYPE,][src=SRC,]dst=DST[,KEY=VALUE] busybox +``` + +#### Functions + +When you create and run a container, use the --mount option to mount the files, directories, or volumes on the host to the container for data persistence. + +#### Parameter Description + +- type: Type of data mounted to the container. The value can be bind, volume, or squashfs. If this parameter is not specified, the default value is volume. +- src: Path of the file, directory, or volume to be mounted on the host. If the value is an absolute path, the file or directory on the host is mounted. If the value is a volume name, a volume is mounted. If this parameter is not specified, the volume is an anonymous volume. If a folder or volume does not exist, iSula creates a file or volume and then mounts it. The keyword src is also called source. +- dst: Mount path in the container. The value must be an absolute path. The keyword dst is also called destination or target. +- KEY=VALUE: Parameter of --mount. The values are as follows: + +| KEY | VALUE | +| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| selinux-opts/bind-selinux-opts | z or Z. z indicates that if SELinux is enabled, the SELinux share label is added during mounting. Z indicates that if SELinux is enabled, the SELinux private label is added during mounting. | +| ro/readonly | 0/false indicates that the mount is read/write. 1/true indicates that the mount is read-only. If this parameter is not specified, the mount is read-only. The parameter is supported only when type is set to bind. | +| volume-nocopy | Data at the mount point is not copied. If this parameter is not specified, data is copied by default. In addition, if data already exists in the volume, the data will not be copied. This parameter is supported only when type is set to volume. | + +#### Examples + +Run the container based on BusyBox, create or mount a volume named vol to the /vol directory of the container, and set the mount point to read-only. In addition, if data exists at the mount point in the container, the data is not copied. + +```shell +isula run --mount type=volume,src=vol,dst=/vol,ro=true,volume-nocopy=true busybox +``` + +### Reusing the Mounting Configuration in Other Containers + +#### Format + +```shell +isula run --volumes-from CON1[:MODE] busybox +``` + +#### Functions + +When you create and run a container, use the --volumes-from option to indicate that the mount point configuration includes that of the CON1 container. You can set multiple --volumes-from options. + +#### Parameter Description + +- CON1: Name or ID of the container whose mount point is reused. +- MODE: If the value is ro, the mount point is read-only. If the value is rw, the mount point is read/write. + +#### Examples + +Assume that a container named container1 has been configured with a volume vol1 to the container directory /vol1, and a container named container2 has been configured with a volume vol2 to the container directory /vol2. Run a new container to reuse the mounting configuration of container1 and container2. That is, volume vol1 is mounted to the /vol1 directory of the container, and volume vol2 is mounted to the /vol2 directory of the container. + +```shell +isula run --volumes-from container1 --volumes-from container2 busbyox +``` + +### Using the Anonymous Volume in an Image + +You do not need to perform any configuration to use the anonymous volume in the image. If an anonymous volume is configured in the image, iSula automatically creates an anonymous volume and mounts it to the specified path in the image at container runtime. You can write data to the mount point of an anonymous volume in a container for data persistence. + +### Querying a Volume + +#### Format + +```shell +isula volume ls [OPTIONS] +``` + +#### Functions + +This command is used to query all volumes managed by iSula. + +#### Parameter Description + +Option: + +- -q,--quit: If this parameter is not specified, only the volume driver information and volume name are queried by default. If this parameter is specified, only the volume name is queried. + +#### Examples + +This command is used to query all volumes managed by iSula and return only the volume name. + +```shell +isula volume ls -q +``` + +### Deleting a Volume + +#### Format + +```shell +isula volume rm [OPTIONS] VOLUME [VOLUME...] +isula volume prune [OPTIONS] +``` + +#### Functions + +- rm: deletes a specified volume. If the volume is used by a container, the volume fails to be deleted. +- prune: deletes all volumes that are not used by containers. + +#### Parameter Description + +OPTIONS in the prune command: + +- -f,--force: specifies that the system does not display a message asking you whether to delete the volume. By default, a risk message is displayed. You need to enter y to continue the operation. + +#### Examples + +Delete volumes vol1 and vol2. + +```shell +isula volume rm vol1 vol2 +``` + +Delete all unused volumes in the following format. No risk message is displayed. + +```shell +isula volume prune -f +``` + +### Precautions + +#### Conflict Combination Rules + +If a volume mount point conflict occurs, perform the following operations: + +- If configurations of -v and --mount conflict, a failure message is returned. +- If the configuration obtained from --volumes-from conflicts with the -v or --mount configuration, the configuration is discarded. +- If the anonymous volume configuration in the image conflicts with the -v, --mount, or --volumes-from configuration, the configuration is discarded. + +#### Differences Between iSula and Docker + +| iSula Behavior | Docker Behavior | +| ------------------------------------------- | ------------------------------------------- | +| The volume name can contain a maximum of 64 characters. | The length of the volume name is not limited. | +| If the source to be mounted does not exist, the --mount parameter is created. | If the source to be mounted does not exist, an error is reported. | +| The --mount parameter supports the z or Z parameter configuration in bind-selinux-opts and selinux-opts. | The --mount parameter does not support the parameter configuration in the bind-selinux-opts and selinux-opts. | +| Rules for combining mount point conflicts are not processed. | The anonymous volume specified by -v is processed as the anonymous volume in the image. | +| The volume prune command displays the space that has been reclaimed. | The volume prune command does not display the space that has been reclaimed. | +| -v, --mount, and --volumes-from are configured in hostconfig, and the anonymous volume is configured in config. | The anonymous volume specified by -v is configured in config, and other configurations are configured in hostconfig. | diff --git a/docs/en/cloud/container_engine/isula_container_engine/overview.md b/docs/en/cloud/container_engine/isula_container_engine/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..1f43ba362612eb106b6c415e001823618d8c459e --- /dev/null +++ b/docs/en/cloud/container_engine/isula_container_engine/overview.md @@ -0,0 +1,9 @@ +# iSulad Container Engine + +Compared with Docker, iSulad is a new container solution with a unified architecture design to meet different requirements in the CT and IT fields. Lightweight containers are implemented using C/C++. They are smart, fast, and not restricted by hardware and architecture. With less noise floor overhead, the containers can be widely used. + +[Figure 1](#en-us_topic_0182207099_fig10763114141217) shows the unified container architecture. + +**Figure 1** Unified container architecture + +![](./figures/en-us_image_0183048952.png) diff --git a/docs/en/cloud/container_engine/isula_container_engine/privileged-container.md b/docs/en/cloud/container_engine/isula_container_engine/privileged-container.md new file mode 100644 index 0000000000000000000000000000000000000000..f69f8bd9d03669fb7f577ea491626bb669126166 --- /dev/null +++ b/docs/en/cloud/container_engine/isula_container_engine/privileged-container.md @@ -0,0 +1,235 @@ +# Privileged Container + +- [Privileged Container](#privileged-container) + - [Scenarios](#scenarios) + - [Usage Restrictions](#usage-restrictions) + - [Usage Guide](#usage-guide) + +## Scenarios + +By default, iSulad starts common containers that are suitable for starting common processes. However, common containers have only the default permissions defined by capabilities in the **/etc/default/isulad/config.json** directory. To perform privileged operations \(such as use devices in the **/sys** directory\), a privileged container is required. By using this feature, user **root** in the container has **root** permissions of the host. Otherwise, user **root** in the container has only common user permissions of the host. + +## Usage Restrictions + +Privileged containers provide all functions for containers and remove all restrictions enforced by the device cgroup controller. A privileged container has the following features: + +- Secomp does not block any system call. +- The **/sys** and **/proc** directories are writable. +- All devices on the host can be accessed in the container. + +- All system capabilities will be enabled. + +Default capabilities of a common container are as follows: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Capability Key

+

Description

+

SETPCAP

+

Modifies the process capabilities.

+

MKNOD

+

Allows using the system call mknod() to create special files.

+

AUDIT_WRITE

+

Writes records to kernel auditing logs.

+

CHOWN

+

Modifies UIDs and GIDs of files. For details, see the chown(2).

+

NET_RAW

+

Uses RAW and PACKET sockets and binds any IP address to the transparent proxy.

+

DAC_OVERRIDE

+

Ignores the discretionary access control (DAC) restrictions on files.

+

FOWNER

+

Ignores the restriction that the file owner ID must be the same as the process user ID.

+

FSETID

+

Allows setting setuid bits of files.

+

KILL

+

Allows sending signals to processes that do not belong to itself.

+

SETGID

+

Allows the change of the process group ID.

+

SETUID

+

Allows the change of the process user ID.

+

NET_BIND_SERVICE

+

Allows bounding to a port whose number is smaller than 1024.

+

SYS_CHROOT

+

Allows using the system call chroot().

+

SETFCAP

+

Allows transferring and deleting capabilities to other processes.

+
+ +When a privileged container is enabled, the following capabilities are added: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Capability Key

+

Description

+

SYS_MODULE

+

Loads and unloads kernel modules.

+

SYS_RAWIO

+

Allows direct access to /devport, /dev/mem, /dev/kmem, and original block devices.

+

SYS_PACCT

+

Allows the process BSD audit.

+

SYS_ADMIN

+

Allows executing system management tasks, such as loading or unloading file systems and setting disk quotas.

+

SYS_NICE

+

Allows increasing the priority and setting the priorities of other processes.

+

SYS_RESOURCE

+

Ignores resource restrictions.

+

SYS_TIME

+

Allows changing the system clock.

+

SYS_TTY_CONFIG

+

Allows configuring TTY devices.

+

AUDIT_CONTROL

+

Enables and disables kernel auditing, modifies audit filter rules, and extracts audit status and filtering rules.

+

MAC_ADMIN

+

Overrides the mandatory access control (MAC), which is implemented for the Smack Linux Security Module (LSM).

+

MAC_OVERRIDE

+

Allows MAC configuration or status change, which is implemented for Smack LSM.

+

NET_ADMIN

+

Allows executing network management tasks.

+

SYSLOG

+

Performs the privileged syslog(2) operation.

+

DAC_READ_SEARCH

+

Ignores the DAC access restrictions on file reading and catalog search.

+

LINUX_IMMUTABLE

+

Allows modifying the IMMUTABLE and APPEND attributes of a file.

+

NET_BROADCAST

+

Allows network broadcast and multicast access.

+

IPC_LOCK

+

Allows locking shared memory segments.

+

IPC_OWNER

+

Ignores the IPC ownership check.

+

SYS_PTRACE

+

Allows tracing any process.

+

SYS_BOOT

+

Allows restarting the OS.

+

LEASE

+

Allows modifying the FL_LEASE flag of a file lock.

+

WAKE_ALARM

+

Triggers the function of waking up the system, for example, sets the CLOCK_REALTIME_ALARM and CLOCK_BOOTTIME_ALARM timers.

+

BLOCK_SUSPEND

+

Allows blocking system suspension.

+
+ +## Usage Guide + +iSulad runs the **--privileged** command to enable the privilege mode for containers. Do not add privileges to containers unless necessary. Comply with the principle of least privilege to reduce security risks. + +```shell +isula run --rm -it --privileged busybox +``` diff --git a/docs/en/cloud/container_engine/isula_container_engine/public_sys-resources/icon-note.gif b/docs/en/cloud/container_engine/isula_container_engine/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/cloud/container_engine/isula_container_engine/public_sys-resources/icon-note.gif differ diff --git a/docs/en/cloud/container_engine/isula_container_engine/public_sys-resources/icon-notice.gif b/docs/en/cloud/container_engine/isula_container_engine/public_sys-resources/icon-notice.gif new file mode 100644 index 0000000000000000000000000000000000000000..86024f61b691400bea99e5b1f506d9d9aef36e27 Binary files /dev/null and b/docs/en/cloud/container_engine/isula_container_engine/public_sys-resources/icon-notice.gif differ diff --git a/docs/en/cloud/container_engine/isula_container_engine/querying-information.md b/docs/en/cloud/container_engine/isula_container_engine/querying-information.md new file mode 100644 index 0000000000000000000000000000000000000000..c1d22dfd2bfe84e6e4587f742e4ca5f048306741 --- /dev/null +++ b/docs/en/cloud/container_engine/isula_container_engine/querying-information.md @@ -0,0 +1,93 @@ +# Querying Information + +- [Querying Information](#querying-information) + - [Querying the Service Version](#querying-the-service-version) + - [Querying System-level Information](#querying-system-level-information) + +## Querying the Service Version + +### Description + +The **isula version** command is run to query the version of the iSulad service. + +### Usage + +```shell +isula version +``` + +### Example + +Query the version information. + +```shell +isula version +``` + +If the iSulad service is running properly, you can view the information about versions of the client, server, and **OCI config**. + +```text +Client: + Version: 1.0.31 + Git commit: fa7f9902738e8b3d7f2eb22768b9a1372ddd1199 + Built: 2019-07-30T04:21:48.521198248-04:00 + +Server: + Version: 1.0.31 + Git commit: fa7f9902738e8b3d7f2eb22768b9a1372ddd1199 + Built: 2019-07-30T04:21:48.521198248-04:00 + +OCI config: + Version: 1.0.0-rc5-dev + Default file: /etc/default/isulad/config.json +``` + +If the iSulad service is not running, only the client information is queried and a message is displayed indicating that the connection times out. + +```text +Client: + Version: 1.0.31 + Git commit: fa7f9902738e8b3d7f2eb22768b9a1372ddd1199 + Built: 2019-07-30T04:21:48.521198248-04:00 + +Can not connect with server.Is the iSulad daemon running on the host? +``` + +Therefore, the **isula version** command is often used to check whether the iSulad service is running properly. + +## Querying System-level Information + +### Description + +The **isula info** command is run to query the system-level information, number of containers, and number of images. + +### Usage + +```shell +isula info +``` + +### Example + +Query system-level information, including the number of containers, number of images, kernel version, and operating system \(OS\). + +```shell +$ isula info +Containers: 2 + Running: 0 + Paused: 0 + Stopped: 2 +Images: 8 +Server Version: 1.0.31 +Logging Driver: json-file +Cgroup Driverr: cgroupfs +Hugetlb Pagesize: 2MB +Kernel Version: 4.19 +Operating System: Fedora 29 (Twenty Nine) +OSType: Linux +Architecture: x86_64 +CPUs: 8 +Total Memory: 7 GB +Name: localhost.localdomain +iSulad Root Dir: /var/lib/isulad +``` diff --git a/docs/en/cloud/container_engine/isula_container_engine/security-features.md b/docs/en/cloud/container_engine/isula_container_engine/security-features.md new file mode 100644 index 0000000000000000000000000000000000000000..7d0cd7b80f2f5075ce967a63592b304b32c204c0 --- /dev/null +++ b/docs/en/cloud/container_engine/isula_container_engine/security-features.md @@ -0,0 +1,252 @@ +# Security Features + +- [Security Features](#security-features) + - [Seccomp Security Configuration](#seccomp-security-configuration) + - [Scenarios](#scenarios) + - [Usage Restrictions](#usage-restrictions) + - [Usage Guide](#usage-guide) + - [capabilities Security Configuration](#capabilities-security-configuration) + - [Scenarios](#scenarios-1) + - [Usage Restrictions](#usage-restrictions-1) + - [Usage Guide](#usage-guide-1) + - [SELinux Security Configuration](#selinux-security-configuration) + - [Scenarios](#scenarios-2) + - [Usage Restrictions](#usage-restrictions-2) + - [Usage Guide](#usage-guide-2) + +## Seccomp Security Configuration + +### Scenarios + +Secure computing mode \(seccomp\) is a simple sandboxing mechanism introduced to the Linux kernel from version 2.6.23. In some specific scenarios, you may want to perform some privileged operations in a container without starting the privileged container. You can add **--cap-add** at runtime to obtain some small-scope permissions. For container instances with strict security requirements, th capability granularity may not meet the requirements. You can use some methods to control the permission scope in a refined manner. + +- Example + + In a common container scenario, you can use the **-v** flag to map a directory \(including a binary file that cannot be executed by common users\) on the host to the container. + + In the container, you can add chmod 4777 \(the modification permission of the binary file\) to the S flag bit. In this way, on the host, common users who cannot run the binary file \(or whose running permission is restricted\) can obtain the permissions of the binary file \(such as the root permission\) when running the binary file after the action added to the S flag bit is performed, so as to escalate the permission or access other files. + + In this scenario, if strict security requirements are required, the chmod, fchmod, and fchmodat system calls need to be tailored by using seccomp. + +### Usage Restrictions + +- Seccomp may affect performance. Before setting seccomp, evaluate the scenario and add the configuration only if necessary. + +### Usage Guide + +Use **--security-opt** to transfer the configuration file to the container where system calls need to be filtered. + +```shell +isula run -itd --security-opt seccomp=/path/to/seccomp/profile.json rnd-dockerhub.huawei.com/official/busybox +``` + +>[!NOTE]NOTE + +1. When the configuration file is transferred to the container by using **--security-opt** during container creation, the default configuration file \(**/etc/isulad/seccomp\_default.json**\) is used. +2. When **--security-opt** is set to **unconfined** during container creation, system calls are not filtered for the container. +3. **/path/to/seccomp/profile.json** must be an absolute path. + +#### Obtaining the Default Seccomp Configuration of a Common Container + +- Start a common container \(or a container with **--cap-add**\) and check its default permission configuration. + + ```shell + cat /etc/isulad/seccomp_default.json | python -m json.tool > profile.json + ``` + + The **seccomp** field contains many **syscalls** fields. Then extract only the **syscalls** fields and perform the customization by referring to the customization of the seccomp configuration file. + + ```json + "defaultAction": "SCMP_ACT_ERRNO", + "syscalls": [ + { + "action": "SCMP_ACT_ALLOW", + "name": "accept" + }, + { + "action": "SCMP_ACT_ALLOW", + "name": "accept4" + }, + { + "action": "SCMP_ACT_ALLOW", + "name": "access" + }, + { + "action": "SCMP_ACT_ALLOW", + "name": "alarm" + }, + { + "action": "SCMP_ACT_ALLOW", + "name": "bind" + }, + ]... + ``` + +- Check the seccomp configuration that can be identified by the LXC. + + ```shell + cat /var/lib/isulad/engines/lcr/74353e38021c29314188e29ba8c1830a4677ffe5c4decda77a1e0853ec8197cd/seccomp + ``` + + ```text + ... + waitpid allow + write allow + writev allow + ptrace allow + personality allow [0,0,SCMP_CMP_EQ,0] + personality allow [0,8,SCMP_CMP_EQ,0] + personality allow [0,131072,SCMP_CMP_EQ,0] + personality allow [0,131080,SCMP_CMP_EQ,0] + personality allow [0,4294967295,SCMP_CMP_EQ,0] + ... + ``` + +#### Customizing the Seccomp Configuration File + +When starting a container, use **--security-opt** to introduce the seccomp configuration file. Container instances will restrict the running of system APIs based on the configuration file. Obtain the default seccomp configuration of common containers, obtain the complete template, and customize the configuration file by referring to this section to start the container. + +```shell +isula run --rm -it --security-opt seccomp:/path/to/seccomp/profile.json rnd-dockerhub.huawei.com/official/busybox +``` + +The configuration file template is as follows: + +```json +{ +"defaultAction": "SCMP_ACT_ALLOW", +"syscalls": [ +{ +"name": "syscall-name", +"action": "SCMP_ACT_ERRNO", +"args": null +} +] +} +``` + +>[!TIP]NOTICE +> +> - **defaultAction** and **syscalls**: The types of their corresponding actions are the same, but their values must be different. The purpose is to ensure that each syscall has a default action. Clear definitions in the syscall array shall prevail. As long as the values of **defaultAction** and **action** are different, no action conflicts will occur. The following actions are supported: +> **SCMP\_ACT\_ERRNO**: forbids calling syscalls and displays error information. +> **SCMP\_ACT\_ALLOW**: allows calling syscalls. +> - **syscalls**: array, which can contain one or more syscalls. **args** is optional. +> - **name**: syscalls to be filtered. +> - **args**: array. The definition of each object in the array is as follows: +> +> ```go +> type Arg struct { +> Index uint `json:"index"` // Parameter ID. Take open(fd, buf, len) as an example. The fd corresponds to 0 and buf corresponds to 1. +> Value uint64 `json:"value"` // Value to be compared with the parameter. +> ValueTwo uint64 `json:"value_two"` // It is valid only when Op is set to MaskEqualTo. After the bitwise AND operation is performed on the user-defined value and the value of Value, the result is compared with the value of ValueTwo. If they are the same, the action is executed. +> Op Operator `json:"op"` +> } +> ``` +> +> The value of **Op** in **args** can be any of the following: +> "SCMP\_CMP\_NE": NotEqualTo +> "SCMP\_CMP\_LT": LessThan +> "SCMP\_CMP\_LE": LessThanOrEqualTo +> "SCMP\_CMP\_EQ": EqualTo +> "SCMP\_CMP\_GE": GreaterThanOrEqualTo +> "SCMP\_CMP\_GT": GreaterThan +> "SCMP\_CMP\_MASKED\_EQ": MaskEqualTo + +## capabilities Security Configuration + +### Scenarios + +The capability mechanism is a security feature introduced to Linux kernel after version 2.2. The super administrator permission is controlled at a smaller granularity to prevent the root permission from being used. The root permission is divided based on different domains so that the divided permissions can be enabled or disabled separately. For details about capabilities, see the _Linux Programmer's Manual_ \([capabilities\(7\) - Linux man page](http://man7.org/linux/man-pages/man7/capabilities.7.html)\). + +```shell +man capabilities +``` + +### Usage Restrictions + +- The default capability list \(allowlist\) of the iSulad service, which is carried by common container processes by default, are as follows: + + ```text + "CAP_CHOWN", + "CAP_DAC_OVERRIDE", + "CAP_FSETID", + "CAP_FOWNER", + "CAP_MKNOD", + "CAP_NET_RAW", + "CAP_SETGID", + "CAP_SETUID", + "CAP_SETFCAP", + "CAP_SETPCAP", + "CAP_NET_BIND_SERVICE", + "CAP_SYS_CHROOT", + "CAP_KILL", + "CAP_AUDIT_WRITE" + ``` + +- Default configurations of capabilities include **CAP\_SETUID** and **CAP\_FSETID**. If the host and a container share a directory, the container can set permissions for the binary file in the shared directory. Common users on the host can use this feature to elevate privileges. The container can write **CAP\_AUDIT\_WRITE** to the host, which may cause risks. If the application scenario does not require this capability, you are advised to use **--cap-drop** to delete the capability when starting the container. +- Adding capabilities means that the container process has greater capabilities than before. In addition, more system call APIs are opened. + +### Usage Guide + +iSulad uses **--cap-add** or **--cap-drop** to add or delete specific permissions for a container. Do not add extra permissions to the container unless necessary. You are advised to remove the default but unnecessary permissions from the container. + +```shell +isula run --rm -it --cap-add all --cap-drop SYS_ADMIN rnd-dockerhub.huawei.com/official/busybox +``` + +## SELinux Security Configuration + +### Scenarios + +Security-Enhanced Linux \(SELinux\) is a Linux kernel security module that provides a mechanism for supporting access control security policies. Through Multi-Category Security \(MCS\), iSulad labels processes in containers to control containers' access to resources, reducing privilege escalation risks and preventing further damage. + +### Usage Restrictions + +- Ensure that SELinux is enabled for the host and daemon \(the **selinux-enabled** field in the **/etc/isulad/daemon.json** file is set to **true** or **--selinux-enabled** is added to command line parameters\). +- Ensure that a proper SELinux policy has been configured on the host. container-selinux is recommended. +- The introduction of SELinux affects the performance. Therefore, evaluate the scenario before setting SELinux. Enable the SELinux function for the daemon and set the SELinux configuration in the container only when necessary. +- When you configure labels for a mounted volume, the source directory cannot be a subdirectory of **/**, **/usr**, **/etc**, **/tmp**, **/home**, **/run**, **/var**, **/root**, or **/usr**. + +>[!NOTE]NOTE +> +> - iSulad does not support labeling the container file system. To ensure that the container file system and configuration directory are labeled with the container access permission, run the **chcon** command to label them. +> - If SELinux access control is enabled for iSulad, you are advised to add a label to the **/var/lib/isulad** directory before starting daemon. Files and folders generated in the directory during container creation inherit the label by default. For example: +> +> ```shell +> chcon -R system_u:object_r:container_file_t:s0 /var/lib/isulad +> ``` + +### Usage Guide + +- Enable SELinux for daemon. + + ```shell + isulad --selinux-enabled + ``` + +- Configure SELinux security context labels during container startup. + + **--security-opt="label=user:USER"**: Set the label user for the container. + + **--security-opt="label=role:ROLE"**: Set the label role for the container. + + **--security-opt="label=type:TYPE"**: Set the label type for the container. + + **--security-opt="label=level:LEVEL"**: Set the label level for the container. + + **--security-opt="label=disable"**: Disable the SELinux configuration for the container. + + ```shell + $ isula run -itd --security-opt label=type:container_t --security-opt label=level:s0:c1,c2 rnd-dockerhub.huawei.com/official/centos + 9be82878a67e36c826b67f5c7261c881ff926a352f92998b654bc8e1c6eec370 + ``` + +- Add the selinux label to a mounted volume \(**z** indicates the shared mode\). + + ```shell + $ isula run -itd -v /test:/test:z rnd-dockerhub.huawei.com/official/centos + 9be82878a67e36c826b67f5c7261c881ff926a352f92998b654bc8e1c6eec370 + + $ls -Z /test + system_u:object_r:container_file_t:s0 file + ``` diff --git a/docs/en/cloud/container_engine/isula_container_engine/supporting-oci-hooks.md b/docs/en/cloud/container_engine/isula_container_engine/supporting-oci-hooks.md new file mode 100644 index 0000000000000000000000000000000000000000..66fc0d2af4cc62389c43037c9e43d741ca1bcc29 --- /dev/null +++ b/docs/en/cloud/container_engine/isula_container_engine/supporting-oci-hooks.md @@ -0,0 +1,82 @@ +# Supporting OCI hooks + +- [Supporting OCI hooks](#supporting-oci-hooks) + - [Description](#description) + - [APIs](#apis) + - [Usage Restrictions](#usage-restrictions) + +## Description + +The running of standard OCI hooks within the lifecycle of a container is supported. There are three types of standard hooks: + +- prestart hook: executed after the **isula start** command is executed and before the init process of the container is started. +- poststart hook: executed after the init process is started and before the **isula start** command is returned. +- poststop hook: executed after the container is stopped and before the stop command is returned. + +The configuration format specifications of OCI hooks are as follows: + +- **path**: \(Mandatory\) The value must be a character string and must be an absolute path. The specified file must have the execute permission. +- **args**: \(Optional\) The value must be a character string array. The syntax is the same as that of **args** in **execv**. +- **env**: \(Optional\) The value must be a character string array. The syntax is the same as that of environment variables. The content is a key-value pair, for example, **PATH=/usr/bin**. +- **timeout**: \(Optional\) The value must be an integer that is greater than 0. It indicates the timeout interval for hook execution. If the running time of the hook process exceeds the configured time, the hook process is killed. + +The hook configuration is in JSON format and usually stored in a file ended with **json**. An example is as follows: + +```json +{ + "prestart": [ + { + "path": "/usr/bin/echo", + "args": ["arg1", "arg2"], + "env": [ "key1=value1"], + "timeout": 30 + }, + { + "path": "/usr/bin/ls", + "args": ["/tmp"] + } + ], + "poststart": [ + { + "path": "/usr/bin/ls", + "args": ["/tmp"], + "timeout": 5 + } + ], + "poststop": [ + { + "path": "/tmp/cleanup.sh", + "args": ["cleanup.sh", "-f"] + } + ] +} +``` + +## APIs + +Both iSulad and iSula provide the hook APIs. The default hook configurations provided by iSulad apply to all containers. The hook APIs provided by iSula apply only to the currently created container. + +The default OCI hook configurations provided by iSulad are as follows: + +- Set the configuration item **hook-spec** in the **/etc/isulad/daemon.json** configuration file to specify the path of the hook configuration file. Example: **"hook-spec": "/etc/default/isulad/hooks/default.json"** +- Use the **isulad --hook-spec** parameter to set the path of the hook configuration file. + +The OCI hook configurations provided by iSula are as follows: + +- **isula create --hook-spec**: specifies the path of the hook configuration file in JSON format. +- **isula run --hook-spec**: specifies the path of the hook configuration file in JSON format. + +The configuration for **run** takes effect in the creation phase. + +## Usage Restrictions + +- The path specified by **hook-spec** must be an absolute path. +- The file specified by **hook-spec** must exist. +- The path specified by **hook-spec** must contain a common text file in JSON format. +- The file specified by **hook-spec** cannot exceed 10 MB. +- **path** configured for hooks must be an absolute path. +- The file that is designated by **path** configured for hooks must exist. +- The file that is designated by **path** configured for hooks must have the execute permission. +- The owner of the file that is designated by **path** configured for hooks must be user **root**. +- Only user **root** has the write permission on the file that is designated by **path** configured for hooks. +- The value of **timeout** configured for hooks must be greater than **0**. diff --git a/docs/en/cloud/container_engine/isula_container_engine/uninstallation.md b/docs/en/cloud/container_engine/isula_container_engine/uninstallation.md new file mode 100644 index 0000000000000000000000000000000000000000..aac45aae7ff223156d1f5c4e41eaaf736f1f24d4 --- /dev/null +++ b/docs/en/cloud/container_engine/isula_container_engine/uninstallation.md @@ -0,0 +1,22 @@ +# Uninstallation + +To uninstall iSulad, perform the following operations: + +1. Uninstall iSulad and its dependent software packages. + - If the **yum** command is used to install iSulad, run the following command to uninstall iSulad: + + ```shell + sudo yum remove iSulad + ``` + + - If the **rpm** command is used to install iSulad, uninstall iSulad and its dependent software packages. Run the following command to uninstall an RPM package. + + ```shell + sudo rpm -e iSulad-xx.xx.xx-YYYYmmdd.HHMMSS.gitxxxxxxxx.aarch64.rpm + ``` + +2. Images, containers, volumes, and related configuration files are not automatically deleted. The reference command is as follows: + + ```shell + sudo rm -rf /var/lib/iSulad + ``` diff --git a/docs/en/cloud/container_engine/isula_container_engine/upgrade-methods.md b/docs/en/cloud/container_engine/isula_container_engine/upgrade-methods.md new file mode 100644 index 0000000000000000000000000000000000000000..eb5b28a283714eb80837fe38a2108a2304a09ff8 --- /dev/null +++ b/docs/en/cloud/container_engine/isula_container_engine/upgrade-methods.md @@ -0,0 +1,24 @@ +# Upgrade Methods + +- For an upgrade between patch versions of a major version, for example, upgrading 2.x.x to 2.x.x, run the following command: + + ```shell + sudo yum update -y iSulad + ``` + +- For an upgrade between major versions, for example, upgrading 1.x.x to 2.x.x, save the current configuration file **/etc/isulad/daemon.json**, uninstall the existing iSulad software package, install the iSulad software package to be upgraded, and restore the configuration file. + +> [!NOTE]NOTE +> +> - You can run the **sudo rpm -qa \|grep iSulad** or **isula version** command to check the iSulad version. +> - If you want to manually perform upgrade between patch versions of a major version, run the following command to download the RPM packages of iSulad and all its dependent libraries: +> +> ```shell +> sudo rpm -Uhv iSulad-xx.xx.xx-YYYYmmdd.HHMMSS.gitxxxxxxxx.aarch64.rpm +> ``` +> +> If the upgrade fails, run the following command to forcibly perform the upgrade: +> +> ```shell +> sudo rpm -Uhv --force iSulad-xx.xx.xx-YYYYmmdd.HHMMSS.gitxxxxxxxx.aarch64.rpm +> ``` diff --git a/docs/en/cloud/container_form/secure_container/_toc.yaml b/docs/en/cloud/container_form/secure_container/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..8624dff62e4bc98c843919378b98c5fd734b733b --- /dev/null +++ b/docs/en/cloud/container_form/secure_container/_toc.yaml @@ -0,0 +1,21 @@ +label: Secure Container +isManual: true +description: >- + Secure containers integrate virtualization and container technologies to + provide enhanced isolation. +sections: + - label: Overview + href: ./overview.md + - label: Installation and Deployment + href: ./installation-and-deployment-2.md + - label: Application Scenarios + href: ./application-scenarios-2.md + sections: + - label: Managing the Lifecycle of a Secure Container + href: ./managing-the-lifecycle-of-a-secure-container.md + - label: Configuring Resources for a Secure Container + href: ./configuring-resources-for-a-secure-container.md + - label: Monitoring Secure Containers + href: ./monitoring-secure-containers.md + - label: Appendix + href: ./appendix-2.md diff --git a/docs/en/cloud/container_form/secure_container/appendix-2.md b/docs/en/cloud/container_form/secure_container/appendix-2.md new file mode 100644 index 0000000000000000000000000000000000000000..f4a4a257e2e8bca594993a2f0dcfe27f05f26988 --- /dev/null +++ b/docs/en/cloud/container_form/secure_container/appendix-2.md @@ -0,0 +1,487 @@ +# Appendix + +- [Appendix](#appendix) + - [configuration.toml](#configurationtoml) + - [APIs](#apis) + +## configuration.toml + +> [!NOTE]NOTE +> The value of each field in the **configuration.toml** file is subject to the **configuration.toml** file in the **kata-containers-<**_version_**\>.rpm package**. You cannot set any field in the configuration file. + +```text +[hypervisor.qemu] +path: specifies the execution path of the virtualization QEMU. +kernel: specifies the execution path of the guest kernel. +initrd: specifies the guest initrd execution path. +image: specifies the execution path of the guest image (not applicable). +machine_type: specifies the type of the analog chip. The value is virt for the ARM architecture and pc for the x86 architecture. +kernel_params: specifies the running parameters of the guest kernel. +firmware: specifies the firmware path. If this parameter is left blank, the default firmware is used. +machine_accelerators: specifies an accelerator. +default_vcpus: specifies the default number of vCPUs for each SB/VM. +default_maxvcpus: specifies the default maximum number of vCPUs for each SB/VM. +default_root_ports: specifies the default number of root ports for each SB/VM. +default_bridges: specifies the default number of bridges for each SB/VM. +default_memory: specifies the default memory size of each SB/VM. The default value is 1024 MiB. +memory_slots: specifies the number of memory slots for each SB/VM. The default value is 10. +memory_offset: specifies the memory offset. The default value is 0. +disable_block_device_use: disables the block device from being used by the rootfs of the container. +shared_fs: specifies the type of the shared file system. The default value is virtio-9p. +virtio_fs_daemon: specifies the path of the vhost-user-fs daemon process. +virtio_fs_cache_size: specifies the default size of the DAX cache. +virtio_fs_cache: specifies the cache mode. +block_device_driver: specifies the driver of a block device. +block_device_cache_set: specifies whether to set cache-related options for a block device. The default value is false. +block_device_cache_direct: specifies whether to enable O_DIRECT. The default value is false. +block_device_cache_noflush: specifies whether to ignore device update requests. The default value is false. +enable_iothreads: enables iothreads. +enable_mem_prealloc: enables VM RAM pre-allocation. The default value is false. +enable_hugepages: enables huge pages. The default value is false. +enable_swap: enables the swap function. The default value is false. +enable_debug: enables QEMU debugging. The default value is false. +disable_nesting_checks: disables nested check. +msize_9p = 8192: specifies the number of bytes transmitted in each 9p packet. +use_vsock: uses vsocks to directly communicate with the agent (the prerequisite is that vsocks is supported). The default value is false. +hotplug_vfio_on_root_bus: enables the hot swap of the VFIO device on the root bus. The default value is false. +disable_vhost_net: disables vhost_net. The default value is false. +entropy_source: specifies the default entropy source. +guest_hook_path: specifies the binary path of the guest hook. + +[factory] +enable_template: enables the VM template. The default value is false. +template_path: specifies the template path. +vm_cache_number: specifies the number of VM caches. The default value is 0. +vm_cache_endpoint: specifies the address of the Unix socket used by the VMCache. The default value is /var/run/kata-containers/cache.sock. + +[proxy.kata] +path: specifies the kata-proxy running path. +enable_debug: enables proxy debugging. The default value is false. + +[shim.kata] +path: specifies the running path of kata-shim. +enable_debug: enables shim debugging. The default value is false. +enable_tracing: enables shim opentracing. + +[agent.kata] +enable_debug: enables the agent debugging function. The default value is false. +enable_tracing: enables the agent tracing function. +trace_mode: specifies the trace mode. +trace_type: specifies the trace type. +enable_blk_mount: enables guest mounting of the block device. + +[netmon] +enable_netmon: enables network monitoring. The default value is false. +path: specifies the kata-netmon running path. +enable_debug: enables netmon debugging. The default value is false. + +[runtime] +enable_debug: enables runtime debugging. The default value is false. +enable_cpu_memory_hotplug: enables CPU and memory hot swap. The default value is false. +internetworking_model: specifies the network interconnection mode between VMs and containers. +disable_guest_seccomp: disables the seccemp security mechanism in the guest application. The default value is true. +enable_tracing: enables runtime opentracing. The default value is false. +disable_new_netns: disables network namespace creation for the shim and hypervisor processes. The default value is false. +experimental: enables the experimental feature, which does not support user-defined configurations. +``` + +## APIs + +**Table 1** Commands related to the kata-runtime network + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Command

+

Subcommand

+

File Example

+

Field

+

Description

+

Remarks

+

kata-network

+
NOTE:
  • The kata-network command must be used in groups. Network devices that are not added using kata-runtime kata-network cannot be deleted or listed using kata-runtime kata-network. The reverse is also true.
  • kata-runtime kata-network imports configuration parameters through a file or stdin.
+
+

add-iface

+
NOTE:
  • An interface can be added to only one container.
  • The execution result is subject to the returned value (non-zero return value).
+
+

  

+

{

+

"device":"tap1",

+

"name":"eth1",

+

"IPAddresses":[{"address":"172.17.1.10","mask":"24"}],

+

"mtu":1300,

+

"hwAddr":"02:42:20:6f:a2:80"

+

"vhostUserSocket":"/usr/local/var/run/openvswitch/vhost-user1"

+

}

+

  

+

device

+

Sets the name of the NIC on a host.

+

Mandatory. The value can contain a maximum of 15 characters, including letters, digits, underscores (\_), hyphens (-), and periods (.). It must start with a letter. The device name must be unique on the same host.

+

name

+

Sets the name of the NIC in the container.

+

Mandatory. The value can contain a maximum of 15 characters, including letters, digits, underscores (\_), hyphens (-), and periods (.). It must start with a letter. Ensure that the name is unique in the same sandbox.

+

IPAddresses

+

Sets the IP address of an NIC.

+

Optional.

+

Currently, one IP address can be configured for each NIC. If no IP address is configured for the NIC, no IP address will be configured in the container, either.

+

mtu

+

Sets the MTU of an NIC.

+

Mandatory.

+

The value ranges from 46 to 9600.

+

hwAddr

+

Sets the MAC address of an NIC.

+

Mandatory.

+

vhostUserSocket

+

Sets the DPDK polling socket path.

+

Optional.

+

The path contains a maximum of 128 bytes. The naming rule can contain digits, letters, and hyphens (-). The path name must start with a letter.

+

del-iface

+

{

+

"name":"eth1"

+

}

+

None

+

Deletes an NIC from a container.

+
NOTE:

When deleting a NIC, you can only delete it based on the name field in the NIC container. Kata does not identify other fields.

+
+

list-ifaces

+

None

+

None

+

Queries the NIC list in a container.

+

None

+

add-route

+

{

+

"dest":"172.17.10.10/24",

+

"gateway":"",

+

"device":"eth1"

+

}

+

dest

+

Sets the network segment corresponding to the route.

+

The value is in the format of <ip>/<mask>. <ip> is mandatory.

+

There are three cases:

+

1. Both IP address and mask are configured.

+

2. If only an IP address is configured, the default mask is 32.

+

3. If "dest":"default" is configured, there is no destination by default. In this case, the gateway needs to be configured.

+

gateway

+

Sets the next-hop gateway of the route.

+

When "dest":"default" is configured, the gateway is mandatory. In other cases, this parameter is optional.

+

device

+

Sets the name of the NIC corresponding to the route.

+

Mandatory.

+

The value contains a maximum of 15 characters.

+

del-route

+

{

+

"dest":"172.17.10.10/24"

+

}

+

None

+

Deletes a container routing rule.

+

dest is mandatory, and both device and gateway are optional.

+
NOTE:

Kata performs fuzzy match based on different fields and deletes the corresponding routing rules.

+
+

list-routes

+

None

+

None

+

Queries the route list in a container.

+

None

+
+ +**Table 2** kata-ipvs command line interfaces + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Command

+

Subcommand

+

Field

+

Parameter

+

Sub-parameter

+

Description

+

Remarks

+

kata-ipvs

+

ipvsadm

+

--parameters

+

-A, --add-service

+

-t, --tcp-service

+

-u, --udp-service

+

Virtual service type.

+

Mandatory. You can select --tcp-service or --udp-service. The format is ip:port. The value of port ranges from 1 to 65535.

+

Example:

+
kata-runtime kata-ipvs ipvsadm --parameters "--add-service --tcp-service 172.17.0.7:80 --scheduler rr --persistent 3000" <container-id>
+

-s, --scheduler

+

Load balancing scheduling algorithm.

+

Mandatory. Value range: rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq.

+

-p, --persistent

+

Service duration.

+

Mandatory. The value ranges from 1 to 2678400, in seconds.

+

-E, --edit-service

+

-t, --tcp-service

+

-u, --udp-service

+

Virtual service type.

+

Mandatory. You can select --tcp-service or --udp-service. The format is ip:port. The value of port ranges from 1 to 65535.

+

-s, --scheduler

+

Load balancing scheduling algorithm.

+

Mandatory. Value range: rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq.

+

-p, --persistent

+

Service duration.

+

Mandatory. The value ranges from 1 to 2678400, in seconds.

+

-D, --delete-service

+

-t, --tcp-service

+

-u, --udp-service

+

Virtual service type.

+

Mandatory. You can select --tcp-service or --udp-service. The format is ip:port. The value of port ranges from 1 to 65535.

+

-a, --add-server

+

-t, --tcp-service

+

-u, --udp-service

+

Virtual service type.

+

Mandatory. You can select --tcp-service or --udp-service. The format is ip:port. The value of port ranges from 1 to 65535.

+

Example:

+
kata-runtime kata-ipvs ipvsadm --parameters "--add-server --tcp-service 172.17.0.7:80 --real-server 172.17.0.4:80 --weight 100" <container-id>
+

-r, --real-server

+

Real server address.

+

Mandatory. The format is ip:port. The value of port ranges from 1 to 65535.

+

-w, --weight

+

Weight

+

Optional. The value ranges from 0 to 65535.

+

-e, --edit-server

+

-t, --tcp-service

+

-u, --udp-service

+

Virtual service type.

+

Mandatory. You can select --tcp-service or --udp-service. The format is ip:port. The value of port ranges from 1 to 65535.

+

-r, --real-server

+

Real server address.

+

Mandatory. The format is ip:port. The value of port ranges from 1 to 65535.

+

-w, --weight

+

Weight

+

Optional. The value ranges from 0 to 65535.

+

-d, --delete-server

+

-t, --tcp-service

+

-u, --udp-service

+

Virtual service type.

+

Mandatory. You can select --tcp-service or --udp-service. The format is ip:port. The value of port ranges from 1 to 65535.

+

-r, --real-server

+

Real server address.

+

Mandatory. The format is ip:port. The value of port ranges from 1 to 65535.

+

-L, --list

+

-t, --tcp-service

+

-u, --udp-service

+

Queries virtual service information.

+

Optional.

+

Example:

+
kata-runtime kata-ipvs ipvsadm --parameters "--list --tcp-service ip:port" <container-id>
+

--set

+

--tcp

+

TCP timeout.

+

Mandatory. The value ranges from 0 to 1296000.

+

Example:

+
kata-runtime kata-ipvs ipvsadm --parameters "--set 100 100 200" <container-id>
+

--tcpfin

+

TCP FIN timeout.

+

Mandatory. The value ranges from 0 to 1296000.

+

--udp

+

UDP timeout.

+

Mandatory. The value ranges from 0 to 1296000.

+

--restore

+

-

+

Imports standard inputs in batches.

+

Rule files can be specified.

+

Example:

+
kata-runtime kata-ipvs ipvsadm --restore - < <rule file path> <container-id>
+
NOTE:

By default, the NAT mode is used for adding a single real server. To add real servers in batches, you need to manually add the -m option to use the NAT mode.

+

The following is an example of the rule file content:

+

-A -t 10.10.11.12:100 -s rr -p 3000

+

-a -t 10.10.11.12:100 -r 172.16.0.1:80 -m

+

-a -t 10.10.11.12:100 -r 172.16.0.1:81 -m

+

-a -t 10.10.11.12:100 -r 172.16.0.1:82 -m

+
+

cleanup

+

--parameters

+

-d, --orig-dst

+

Specifies the IP address.

+

Mandatory.

+

Example:

+
kata-runtime kata-ipvs cleanup --parameters "--orig-dst 172.17.0.4 --protonum tcp" <container-id>
+

-p, --protonum

+

Protocol type.

+

Mandatory. The value can be tcp or udp.

+
diff --git a/docs/en/cloud/container_form/secure_container/application-scenarios-2.md b/docs/en/cloud/container_form/secure_container/application-scenarios-2.md new file mode 100644 index 0000000000000000000000000000000000000000..ae340c389d2b26f3d72fdaa42ff2ffc62a193e21 --- /dev/null +++ b/docs/en/cloud/container_form/secure_container/application-scenarios-2.md @@ -0,0 +1,3 @@ +# Application Scenarios + +This section describes how to use a secure container. diff --git a/docs/en/cloud/container_form/secure_container/configuring-resources-for-a-secure-container.md b/docs/en/cloud/container_form/secure_container/configuring-resources-for-a-secure-container.md new file mode 100644 index 0000000000000000000000000000000000000000..b59ee1331bdae66ad043cea584b4282b17e27ee1 --- /dev/null +++ b/docs/en/cloud/container_form/secure_container/configuring-resources-for-a-secure-container.md @@ -0,0 +1,36 @@ +# Configuring Resources for a Secure Container + +- [Configuring Resources for a Secure Container](#configuring-resources-for-a-secure-container) + - [Sharing Resources](#sharing-resources) + - [Limiting Resources](#limiting-resources) + - [Limiting Memory Resources Through the Memory Hotplug Feature](#limiting-memory-resources-through-the-memory-hotplug-feature) + +The secure container runs on a virtualized and isolated lightweight VM. Therefore, resource configuration is divided into two parts: resource configuration for the lightweight VM, that is, host resource configuration; resource configuration for containers in the VM, that is, guest container resource configuration. The following describes resource configuration for the two parts in detail. + +## Sharing Resources + +Because the secure container runs on a virtualized and isolated lightweight VM, resources in some namespaces on the host cannot be accessed. Therefore, **--net host**, **--ipc host**, **--pid host**, and **--uts host** are not supported during startup. + +When a pod is started, all containers in the pod share the same net namespace and ipc namespace by default. If containers in the same pod need to share the pid namespace, you can use Kubernetes to configure the pid namespace. In Kubernetes 1.11, the pid namespace is disabled by default. + +## Limiting Resources + +Limitations on sandbox resources should be configured in **configuration.toml**. +Common fields are: + +- **default_vcpus**: specifies the default number of virtual CPUs. +- **default_maxvcpus**: specifies the max number of virtual CPUs. +- **default_root_ports**: specifies the default number of Root Ports in SB/VM. +- **default_bridges**: specifies the default number of bridges. +- **default_memory**: specifies the size of memory. The default size is 1024 MiB. +- **memory_slots**: specifies the number of memory slots. The default number is **10**. + +## Limiting Memory Resources Through the Memory Hotplug Feature + +Memory hotplug is a key feature for containers to allocate memory dynamically in deployment. As Kata containers are based on VMs, this feature needs support both from VMM and guest kernel. Luckily, it has been fully supported for the current default version of QEMU and guest kernel used by Kata on ARM64. For other VMMs, e.g, Cloud Hypervisor, the enablement work is on the road. Apart from VMM and guest kernel, memory hotplug also depends on ACPI which depends on firmware. On x86, you can boot a VM using QEMU with ACPI enabled directly, because it boots up with firmware implicitly. For ARM64, however, you need specify firmware explicitly. That is to say, if you are ready to run a normal Kata container on ARM64, what you need extra to do is to install the UEFI ROM before using the memory hotplug feature. + +```shell +pushd $GOPATH/src/github.com/kata-containers/tests +sudo .ci/aarch64/install_rom_aarch64.sh +popd +``` diff --git a/docs/en/cloud/container_form/secure_container/figures/relationship-between-the-secure-container-and-peripheral-components.png b/docs/en/cloud/container_form/secure_container/figures/relationship-between-the-secure-container-and-peripheral-components.png new file mode 100644 index 0000000000000000000000000000000000000000..454fc025ecb88fef09472eef7cb29ca7a8164856 Binary files /dev/null and b/docs/en/cloud/container_form/secure_container/figures/relationship-between-the-secure-container-and-peripheral-components.png differ diff --git a/docs/en/cloud/container_form/secure_container/figures/secure-container.png b/docs/en/cloud/container_form/secure_container/figures/secure-container.png new file mode 100644 index 0000000000000000000000000000000000000000..2e8b48bdbd0766ec513e0654212cd16613eff826 Binary files /dev/null and b/docs/en/cloud/container_form/secure_container/figures/secure-container.png differ diff --git a/docs/en/cloud/container_form/secure_container/installation-and-deployment-2.md b/docs/en/cloud/container_form/secure_container/installation-and-deployment-2.md new file mode 100644 index 0000000000000000000000000000000000000000..8b240705ba546435a16dfefee835bb2fb6bea69d --- /dev/null +++ b/docs/en/cloud/container_form/secure_container/installation-and-deployment-2.md @@ -0,0 +1,123 @@ +# Installation and Deployment + +- [Installation and Deployment](#installation-and-deployment) + - [Installation Methods](#installation-methods) + - [Prerequisites](#prerequisites) + - [Installation Procedure](#installation-procedure) + - [Deployment Configuration](#deployment-configuration) + - [Configuring the Docker Engine](#configuring-the-docker-engine) + - [iSulad Configuration](#isulad-configuration) + - [Configuration.toml](#configurationtoml) + +## Installation Methods + +### Prerequisites + +- The root permission is required for installing a Kata container. +- For better performance experience, a Kata container needs to run on the bare metal server and cannot run on VMs. +- A Kata container depends on the following components \(openEuler 1.0 version\). Ensure that the required components have been installed in the environment. To install iSulad, refer to [Installation Configuration](../../container_engine/isula_container_engine/installation-configuration.md). + - docker-engine + - qemu + +### Installation Procedure + +Released Kata container components are integrated in the **kata-containers-**_version_**.rpm** package. You can run the **rpm** command to install the corresponding software. + +```shell +rpm -ivh kata-containers-.rpm +``` + +## Deployment Configuration + +### Configuring the Docker Engine + +To enable the Docker engine to support kata-runtime, perform the following steps to configure the Docker engine: + +1. Ensure that all software packages \(**docker-engine** and **kata-containers**\) have been installed in the environment. +2. Stop the Docker engine. + + ```shell + systemctl stop docker + ``` + +3. Modify the configuration file **/etc/docker/daemon.json** of the Docker engine and add the following configuration: + + ```json + { + "runtimes": { + "kata-runtime": { + "path": "/usr/bin/kata-runtime", + "runtimeArgs": [ + "--kata-config", + "/usr/share/defaults/kata-containers/configuration.toml" + ] + } + } + } + ``` + +4. Restart the Docker engine. + + ```shell + systemctl start docker + ``` + +### iSulad Configuration + +To enable the iSulad to support the new container runtime kata-runtime, perform the following steps which are similar to those for the container engine docker-engine: + +1. Ensure that all software packages \(iSulad and kata-containers\) have been installed in the environment. +2. Stop iSulad. + + ```shell + systemctl stop isulad + ``` + +3. Modify the **/etc/isulad/daemon.json** configuration file of the iSulad and add the following configurations: + + ```json + { + "runtimes": { + "kata-runtime": { + "path": "/usr/bin/kata-runtime", + "runtime-args": [ + "--kata-config", + "/usr/share/defaults/kata-containers/configuration.toml" + ] + } + } + } + ``` + +4. Restart iSulad. + + ```shell + systemctl start isulad + ``` + +### Configuration.toml + +The Kata container provides a global configuration file configuration.toml. Users can also customize the path and configuration options of the Kata container configuration file. + +In the **runtimeArges** field of Docker engine, you can use **--kata-config** to specify a private file. The default configuration file path is **/usr/share/defaults/kata-containers/configuration.toml**. + +The following lists the common fields in the configuration file. For details about the configuration file options, see [configuration.toml](./appendix-2.md#configurationtoml). + +1. hypervisor.qemu + - **path**: specifies the execution path of the virtualization QEMU. + - **kernel**: specifies the execution path of the guest kernel. + - **initrd**: specifies the guest initrd execution path. + - **machine\_type**: specifies the type of the analog chip. The value is **virt** for the ARM architecture and **pc** for the x86 architecture. + - **kernel\_params**: specifies the running parameters of the guest kernel. + +2. proxy.kata + - **path**: specifies the kata-proxy running path. + - **enable\_debug**: enables the debugging function for the kata-proxy process. + +3. agent.kata + - **enable\_blk\_mount**: enables guest mounting of the block device. + - **enable\_debug**: enables the debugging function for the kata-agent process. + +4. runtime + - **enable\_cpu\_memory\_hotplug**: enables CPU and memory hot swap. + - **enable\_debug**: enables debugging for the kata-runtime process. diff --git a/docs/en/cloud/container_form/secure_container/managing-the-lifecycle-of-a-secure-container.md b/docs/en/cloud/container_form/secure_container/managing-the-lifecycle-of-a-secure-container.md new file mode 100644 index 0000000000000000000000000000000000000000..7c0d4ee3ae32ab136517278f35dfa219d641a043 --- /dev/null +++ b/docs/en/cloud/container_form/secure_container/managing-the-lifecycle-of-a-secure-container.md @@ -0,0 +1,99 @@ +# Managing the Lifecycle of a Secure Container + +- [Managing the Lifecycle of a Secure Container](#managing-the-lifecycle-of-a-secure-container) + - [Starting a Secure Container](#starting-a-secure-container) + - [Stopping a Secure Container](#stopping-a-secure-container) + - [Deleting a Secure Container](#deleting-a-secure-container) + - [Running a New Command in the Container](#running-a-new-command-in-the-container) + +## Starting a Secure Container + +You can use the Docker engine or iSulad as the container engine of the secure container. The invoking methods of the two engines are similar. You can select either of them to start a secure container. + +To start a secure container, perform the following steps: + +1. Ensure that the secure container component has been correctly installed and deployed. +2. Prepare the container image. If the container image is busybox, run the following commands to download the container image using the Docker engine or iSulad: + + ```shell + docker pull busybox + ``` + + ```shell + isula pull busybox + ``` + +3. Start a secure container. Run the following commands to start a secure container using the Docker engine and iSulad: + + ```shell + docker run -tid --runtime kata-runtime --network none busybox + ``` + + ```shell + isula run -tid --runtime kata-runtime --network none busybox + ``` + + > [!NOTE]NOTE + > The secure container supports the CNI network only and does not support the CNM network. The **-p** and **--expose** options cannot be used to expose container ports. When using a secure container, you need to specify the **--net=none** option + +4. Start a pod. + 1. Start the pause container and obtain the sandbox ID of the pod based on the command output. Run the following commands to start a pause container using the Docker engine and iSulad: + + ```shell + docker run -tid --runtime kata-runtime --network none --annotation io.kubernetes.docker.type=podsandbox + ``` + + ```shell + isula run -tid --runtime kata-runtime --network none --annotation io.kubernetes.cri.container-type=sandbox + ``` + + 2. Create a service container and add it to the pod. Run the following commands to create a service container using the Docker engine and iSulad: + + ```shell + docker run -tid --runtime kata-runtime --network none --annotation io.kubernetes.docker.type=container --annotation io.kubernetes.sandbox.id= busybox + ``` + + ```shell + isula run -tid --runtime kata-runtime --network none --annotation io.kubernetes.cri.container-type=container --annotation io.kubernetes.cri.sandbox-id= busybox + ``` + + **--annotation** is used to mark the container type, which is provided by the Docker engine and iSulad, but not provided by the open-source Docker engine in the upstream community. + +## Stopping a Secure Container + +- Run the following command to stop a secure container: + + ```shell + docker stop + ``` + +- Stop a pod. + + When stopping a pod, note that the lifecycle of the pause container is the same as that of the pod. Therefore, stop service containers before the pause container. + +## Deleting a Secure Container + +Ensure that the container has been stopped. + +```shell +docker rm +``` + +To forcibly delete a running container, run the **-f** command. + +```shell +docker rm -f +``` + +## Running a New Command in the Container + +The pause container functions only as a placeholder container. Therefore, if you start a pod, run a new command in the service container. The pause container does not execute the corresponding command. If only one container is started, run the following command directly: + +```shell +docker exec -ti +``` + +> [!NOTE]NOTE + +1. If the preceding command has no response because another host runs the **docker restart** or **docker stop** command to access the same container, you can press **Ctrl**+**P**+**Q** to exit the operation. +2. If the **-d** option is used, the command is executed in the background and no error information is displayed. The exit code cannot be used to determine whether the command is executed correctly. diff --git a/docs/en/cloud/container_form/secure_container/monitoring-secure-containers.md b/docs/en/cloud/container_form/secure_container/monitoring-secure-containers.md new file mode 100644 index 0000000000000000000000000000000000000000..f261c9cf810283a6b7c66f2b402e025138185c3b --- /dev/null +++ b/docs/en/cloud/container_form/secure_container/monitoring-secure-containers.md @@ -0,0 +1,56 @@ +# Monitoring Secure Containers + +- [Monitoring Secure Containers](#monitoring-secure-containers) + +## Description + +In kata 2.x, events subcommand is removed and replaced by **kata-runtime metrics**, which can be used to gather metrics associated with infrastructure used to run a sandbox, including virtual machine stats, shim v2 CPU seconds and CPU stat of guest OS and so on. Metrics are organized in a Prometheus compatible format so that they can be easily uploaded to Prometheus when work with kata-monitor. + +## Usage + +```shell +kata-runtime metrics +``` + +## Prerequisites + +The sandbox ID must be the full ID. The sandbox to be queried must be in the **running** state. Otherwise, the following error message will be displayed: "Container ID \(\) does not exist". + +When using annotation to make a container run in a specific sandbox, clients should not use kata-runtime metrics to gather metrics of that container. The correct way is to query the corresponding sandbox. + +This command can be used to query the status of only one container. + +## Example + +```shell +$ kata-runtime metrics e2270357d23f9d3dd424011e1e70aa8defb267d813c3d451db58f35aeac97a04 + +# HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles. +# TYPE go_gc_duration_seconds summary +go_gc_duration_seconds{quantile="0"} 2.656e-05 +go_gc_duration_seconds{quantile="0.25"} 3.345e-05 +go_gc_duration_seconds{quantile="0.5"} 3.778e-05 +go_gc_duration_seconds{quantile="0.75"} 4.657e-05 +go_gc_duration_seconds{quantile="1"} 0.00023001 +go_gc_duration_seconds_sum 0.00898126 +go_gc_duration_seconds_count 195 +# HELP go_goroutines Number of goroutines that currently exist. +# TYPE go_goroutines gauge +go_goroutines 27 +# HELP go_info Information about the Go environment. +# TYPE go_info gauge +go_info{version="go1.17.3"} 1 +# HELP kata_hypervisor_netdev Net devices statistics. +# TYPE kata_hypervisor_netdev gauge +kata_hypervisor_netdev{interface="lo",item="recv_bytes"} 0 +kata_hypervisor_netdev{interface="lo",item="recv_compressed"} 0 +kata_hypervisor_netdev{interface="lo",item="recv_drop"} 0 +kata_hypervisor_netdev{interface="lo",item="recv_errs"} 0 +kata_hypervisor_netdev{interface="lo",item="recv_fifo"} 0 +kata_hypervisor_netdev{interface="lo",item="recv_frame"} 0 +kata_hypervisor_netdev{interface="lo",item="recv_multicast"} 0 +kata_hypervisor_netdev{interface="lo",item="recv_packets"} 0 +kata_hypervisor_netdev{interface="lo",item="sent_bytes"} 0 +kata_hypervisor_netdev{interface="lo",item="sent_carrier"} 0 +kata_hypervisor_netdev{interface="lo",item="sent_colls"} 0 +``` diff --git a/docs/en/cloud/container_form/secure_container/overview.md b/docs/en/cloud/container_form/secure_container/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..43790d2491d0a0d9948e71f54d1bb9467ce33f34 --- /dev/null +++ b/docs/en/cloud/container_form/secure_container/overview.md @@ -0,0 +1,29 @@ +# Secure Container + +## Overview + +The secure container technology is an organic combination of virtualization and container technologies. Compared with a common Linux container, a secure container has better isolation performance. + +Common Linux containers use namespaces to isolate the running environment between processes and use cgroups to limit resources. Essentially, these common Linux containers share the same kernel. Therefore, if a single container affects the kernel intentionally or unintentionally, the containers on the same host will be affected. + +Secure containers are isolated by the virtualization layers. Containers on the same host do not affect each other. + +**Figure 1** Secure container architecture + +![](./figures/secure-container.png) + +Secure containers are closely related to the concept of pod in Kubernetes. Kubernetes is the open-source ecosystem standard for the container scheduling management platform. It defines a group of container runtime interfaces \(CRIs\). + +In the CRI standards, a pod is a logical grouping of one or more containers, which are scheduled together and share interprocess communication \(IPC\) and network namespaces. As the smallest unit for scheduling, a pod must contain a pause container and one or more service containers. The lifecycle of a pause container is the same as that of the pod. + +A lightweight virtual machine \(VM\) in a secure container is a pod. The first container started in the VM is the pause container, and the containers started later are service containers. + +In a secure container, you can start a single container or start a pod. + +[Figure 2](#fig17734185518269) shows the relationship between the secure container and peripheral components. + +**Figure 2** Relationship between the secure container and peripheral components +![](./figures/relationship-between-the-secure-container-and-peripheral-components.png) + +> ![](./public_sys-resources/icon-note.gif) **Note:** +> Root privileges are necessary for installing and operating secure containers. diff --git a/docs/en/cloud/container_form/secure_container/public_sys-resources/icon-note.gif b/docs/en/cloud/container_form/secure_container/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/cloud/container_form/secure_container/public_sys-resources/icon-note.gif differ diff --git a/docs/en/cloud/container_form/system_container/_toc.yaml b/docs/en/cloud/container_form/system_container/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..002cd851585385bd7b6af53b9e252cb8f0e5a86d --- /dev/null +++ b/docs/en/cloud/container_form/system_container/_toc.yaml @@ -0,0 +1,38 @@ +label: System Container +isManual: true +description: >- + System containers tackle the issues of migrating heavy applications and + services to the cloud in scenarios requiring intensive computation, high + performance, and massive concurrency. +sections: + - label: Overview + href: ./overview.md + - label: Installation Guideline + href: ./installation-guideline.md + - label: Usage Guide + href: ./usage-guide.md + sections: + - label: Specifying Rootfs to Create a Container + href: ./specifying-rootfs-to-create-a-container.md + - label: Using systemd to Start a Container + href: ./using-systemd-to-start-a-container.md + - label: Reboot or Shutdown in a Container + href: ./reboot-or-shutdown-in-a-container.md + - label: Configurable Cgroup Path + href: ./configurable-cgroup-path.md + - label: Writable Namespace Kernel Parameters + href: ./writable-namespace-kernel-parameters.md + - label: Shared Memory Channels + href: ./shared-memory-channels.md + - label: Dynamically Loading the Kernel Module + href: ./dynamically-loading-the-kernel-module.md + - label: Environment Variable Persisting + href: ./environment-variable-persisting.md + - label: Maximum Number of Handles + href: ./maximum-number-of-handles.md + - label: Security and Isolation + href: ./security-and-isolation.md + - label: Dynamically Managing Container Resources (syscontainer-tools) + href: ./dynamically-managing-container-resources-syscontainer-tools.md + - label: Appendix + href: ./appendix-1.md diff --git a/docs/en/cloud/container_form/system_container/appendix-1.md b/docs/en/cloud/container_form/system_container/appendix-1.md new file mode 100644 index 0000000000000000000000000000000000000000..75232351e3a83a6fa3c920cc4c110e793b8e88f9 --- /dev/null +++ b/docs/en/cloud/container_form/system_container/appendix-1.md @@ -0,0 +1,91 @@ +# Appendix + +- [Appendix](#appendix) + - [Command Line Interface List](#command-line-interface-list) + +## Command Line Interface List + +This section lists commands in system containers, which are different from those in common containers. For details about other commands, refer to sections related to the iSulad container engine or run the **isula _XXX_ --help** command. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Command

+

Parameters

+

Value Description

+

isula create/run

+

--external-rootfs

+
  • Variable of the string type.
  • Absolute path on the host.
  • Specifies the rootfs of a VM when running a system container.
+

--system-container

+
  • Boolean variable.
  • Specifies whether a container is a system container. In a system container scenario, this function must be enabled.
+

--add-host

+
  • Variable of the string type.
  • Specifies the hosts configuration for a container. The format is hostname:ip. Multiple values can be set.
+

--dns, --dns-option, --dns-search

+
  • Variable of the string type.
  • Specifies the DNS configuration for a container. Multiple values can be set.
+

--ns-change-opt

+
  • Variable of the string type.
  • Container namespace kernel parameter. The value can only be net or ipc. If multiple values are set, separate them with commas (,), for example, --ns-change-opt=net,ipc.
+

--oom-kill-disable

+
  • Boolean variable.
  • Indicates whether to enable the oom-kill-disable function.
+

--shm-size

+
  • Variable of the string type.
  • Sets the size of /dev/shm. The default value is 64 MB. The unit can be byte (B), kilobyte (KB), megabyte (MB), gigabyte (GB), terabyte (TB), or petabyte (PB).
+

--sysctl

+
  • Variable of the string type.
  • Specifies container kernel parameters. The format is key=value. Multiple values can be set. The sysctl whitelist is as follows:
+

kernel.msgmax, kernel.msgmnb, kernel.msgmni, kernel.sem, kernel.shmall, kernel.shmmax, kernel.shmmni, kernel.shm_rmid_forced, kernel.pid_max, net., and fs.mqueue

+
NOTE:

The kernel.pid_max kernel parameter in a container must be able to be namespaced. Otherwise, an error is reported.

+

Parameter restrictions (including the parameter types and value ranges) of the sysctl whitelist in a container must be the same as those of kernel parameters in the physical machine.

+
+

--env-target-file

+
  • Variable of the string type.
  • Specifies the env persistent file path. (The path must be an absolute path and the file must be in the rootfs directory.) The file size cannot exceed 10 MB. If the value of --env conflicts with that of env in the file, the value of --env takes effect.
  • The root directory of the absolute path is the rootfs root directory. That is, to set the file path to /etc/environment in the container, you need to specify env-target-file=/etc/environment only.
+

--cgroup-parent

+
  • Variable of the string type.
  • Specifies the cgroup parent directory of a container. The cgroup root directory is /sys/fs/cgroup/controller.
+

--host-channel

+
  • Variable of the string type.
  • Specifies the memory space shared between the host and a container (tmpfs). The format is as follows:
+

host path:container path:rw/ro:size limit

+

--files-limit

+
  • Variable of the string type.
  • Specifies the maximum number of file handles in a container. The value must be an integer.
+

--user-remap

+
  • Variable of the string type.
  • The parameter format is uid:gid:offset.
+
diff --git a/docs/en/cloud/container_form/system_container/configurable-cgroup-path.md b/docs/en/cloud/container_form/system_container/configurable-cgroup-path.md new file mode 100644 index 0000000000000000000000000000000000000000..c089da0dc83f4572c2b34ade900dc7f8ee096030 --- /dev/null +++ b/docs/en/cloud/container_form/system_container/configurable-cgroup-path.md @@ -0,0 +1,96 @@ +# Configurable Cgroup Path + +- [Configurable Cgroup Path](#configurable-cgroup-path) + +## Function Description + +System containers provide the capabilities of isolating and reserving container resources on hosts. You can use the **--cgroup-parent** parameter to specify the cgroup directory used by a container to another directory, thereby flexibly allocating host resources. For example, if the cgroup parent path of containers A, B, and C is set to **/lxc/cgroup1**, and the cgroup parent path of containers D, E, and F is set to **/lxc/cgroup2**, the containers are divided into two groups through the cgroup paths, implementing resource isolation at the cgroup level. + +## Parameter Description + + + + + + + + + + + +

Command

+

Parameter

+

Value Description

+

isula create/run

+

--cgroup-parent

+
  • Variable of the string type.
  • Specifies the cgroup parent path of the container.
+
+ +In addition to specifying the cgroup parent path for a system container using commands, you can also specify the cgroup paths of all containers by modifying the startup configuration files of the iSulad container engine. + + + + + + + + + + + + +

Configuration File Path

+

Parameter

+

Description

+

/etc/isulad/daemon.json

+

--cgroup-parent

+
  • Variable of the string type.
  • Specifies the default cgroup parent path of the container.
  • Example: "cgroup-parent": "/lxc/mycgroup"
+
+ +## Constraints + +- If the **cgroup parent** parameter is set on both the daemon and client, the value specified on the client takes effect. +- If container A is started before container B, the cgroup parent path of container B is specified as the cgroup path of container A. When deleting a container, you need to delete container B and then container A. Otherwise, residual cgroup resources exist. + +## Example + +Start a system container and specify the **--cgroup-parent** parameter. + +```shell +[root@localhost ~]# isula run -tid --cgroup-parent /lxc/cgroup123 --system-container --external-rootfs /root/myrootfs none init +115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e +``` + +Check the cgroup information of the init process in the container. + +```shell +[root@localhost ~]# isula inspect -f "{{json .State.Pid}}" 11 +22167 +[root@localhost ~]# cat /proc/22167/cgroup +13:blkio:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e +12:perf_event:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e +11:cpuset:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e +10:pids:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e +9:rdma:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e +8:devices:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e +7:hugetlb:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e +6:memory:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e +5:net_cls,net_prio:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e +4:cpu,cpuacct:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e +3:files:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e +2:freezer:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e +1:name=systemd:/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e/init.scope +0::/lxc/cgroup123/115878a4dfc7c5b8c62ef8a4b44f216485422be9a28f447a4b9ecac4609f332e +``` + +The cgroup parent path of the container is set to **/sys/fs/cgroup/**__**/lxc/cgroup123**. + +In addition, you can configure the container daemon file to set the cgroup parent paths for all containers. For example: + +```text +{ + "cgroup-parent": "/lxc/cgroup123", +} +``` + +Restart the container engine for the configuration to take effect. diff --git a/docs/en/cloud/container_form/system_container/dynamically-loading-the-kernel-module.md b/docs/en/cloud/container_form/system_container/dynamically-loading-the-kernel-module.md new file mode 100644 index 0000000000000000000000000000000000000000..e1a35da96e7140bb29d29730f4a277c222c0b5fc --- /dev/null +++ b/docs/en/cloud/container_form/system_container/dynamically-loading-the-kernel-module.md @@ -0,0 +1,52 @@ +# Dynamically Loading the Kernel Module + +## Function Description + +Services in a container may depend on some kernel modules. You can set environment variables to dynamically load the kernel modules required by services in the container to the host before the system container starts. This feature must be used together with isulad-hooks. For details, see **Dynamically Managing Container Resources (syscontainer-tools)**. + +## Parameter Description + + + + + + + + + + + +

Command

+

Parameter

+

Value Description

+

isula create/run

+

-e KERNEL_MODULES=module_name1,module_name

+
  • Variable of the string type.
  • This parameter can be set to multiple modules. Use commas (,) to separate module names.
+
+ +## Constraints + +- If loaded kernel modules are not verified or conflict with existing modules on the host, an unpredictable error may occur on the host. Therefore, exercise caution when loading kernel modules. +- Dynamic kernel module loading transfers kernel modules to be loaded to containers. This function is implemented by capturing environment variables for container startup using isulad-tools. Therefore, this function relies on the proper installation and deployment of isulad-tools. +- Loaded kernel modules need to be manually deleted. + +## Example + +When starting a system container, specify the **-e KERNEL\_MODULES** parameter. After the system container is started, the ip\_vs module is successfully loaded to the kernel. + +```shell +[root@localhost ~]# lsmod | grep ip_vs +[root@localhost ~]# isula run -tid -e KERNEL_MODULES=ip_vs,ip_vs_wrr --hook-spec /etc/isulad-tools/hookspec.json --system-container --external-rootfs /root/myrootfs none init +ae18c4281d5755a1e153a7bff6b3b4881f36c8e528b9baba8a3278416a5d0980 +[root@localhost ~]# lsmod | grep ip_vs +ip_vs_wrr 16384 0 +ip_vs 176128 2 ip_vs_wrr +nf_conntrack 172032 7 xt_conntrack,nf_nat,nf_nat_ipv6,ipt_MASQUERADE,nf_nat_ipv4,nf_conntrack_netlink,ip_vs +nf_defrag_ipv6 20480 2 nf_conntrack,ip_vs +libcrc32c 16384 3 nf_conntrack,nf_nat,ip_vs +``` + +>[!NOTE]NOTE + +- isulad-tools must be installed on the host. +- **--hooks-spec** must be set to **isulad hooks**. diff --git a/docs/en/cloud/container_form/system_container/dynamically-managing-container-resources-syscontainer-tools.md b/docs/en/cloud/container_form/system_container/dynamically-managing-container-resources-syscontainer-tools.md new file mode 100644 index 0000000000000000000000000000000000000000..5d415bea563425d0f350e1814225106e6e085029 --- /dev/null +++ b/docs/en/cloud/container_form/system_container/dynamically-managing-container-resources-syscontainer-tools.md @@ -0,0 +1,480 @@ +# Dynamically Managing Container Resources \(syscontainer-tools\) + +- [Dynamically Managing Container Resources (syscontainer-tools)](#dynamically-managing-container-resources-syscontainer-tools) + - [Device Management](#device-management) + - [NIC Management](#nic-management) + - [Route Management](#route-management) + - [Volume Mounting Management](#volume-mounting-management) + +Resources in common containers cannot be managed. For example, a block device cannot be added to a common container, and a physical or virtual NIC cannot be inserted to a common container. In the system container scenario, the syscontainer-tools can be used to dynamically mount or unmount block devices, network devices, routes, and volumes for containers. + +To use this function, you need to install the syscontainer-tools first. + +```shell +[root@localhost ~]# yum install syscontainer-tools +``` + +## Device Management + +### Function Description + +isulad-tools allows you to add block devices \(such as disks and logical volume managers\) or character devices \(such as GPUs, binners, and FUSEs\) on the host to a container. The devices can be used in the container. For example, you can run the **fdisk** command to format the disk and write data to the file system. If the devices are not required, isulad-tools allows you to delete them from the container and return them to the host. + +### Command Format + +```shell +isulad-tools [COMMAND][OPTIONS] [ARG...] +``` + +In the preceding format: + +**COMMAND**: command related to device management. + +**OPTIONS**: option supported by the device management command. + +**container\_id**: container ID. + +**ARG**: parameter corresponding to the command. + +### Parameter Description + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Command

+

Function Description

+

Option Description

+

Parameter Description

+

add-device

+

Adds block devices or character devices on the host to a container.

+

Supported options are as follows:

+
  • --blkio-weight-device: sets the I/O weight (relative weight, ranging from 10 to 100) of a block device.
  • --device-read-bps: sets the read rate limit for the block device (byte/s).
  • --device-read-iops: sets the read rate limit for the block device (I/O/s).
  • --device-write-bps: sets the write rate limit for the block device (byte/s).
  • --device-write-iops: sets the write rate limit for the block device (I/O/s).
  • --follow-partition: If a block device is a basic block device (primary SCSI block disk), set this parameter to add all partitions of the primary disk.
  • --force: If any block device or character device already exists in the container, use this parameter to overwrite the old block device or character device files.
  • --update-config-only: updates configuration files only and does not add disks.
+

Parameter format: hostdevice[:containerdevice][:permission] [hostdevice[:containerdevice][:permission]]

+

In the preceding format:

+

hostdevice: path on the host for storing a device.

+

containerdevice: path on the container for storing a device.

+

permission: operation permission on a device within the container.

+

remove-device

+

Deletes block devices or character devices from a container and restores them to the host.

+

Supported options are as follows:

+

--follow-partition: If a block device is a basic block device (primary SCSI block disk), set this parameter to delete all partitions of the primary disk in the container, and restore them to the host.

+

Parameter format: hostdevice[:containerdevice] [hostdevice[:containerdevice]]

+

In the preceding format:

+

hostdevice: path on the host for storing a device.

+

containerdevice: path on the container for storing a device.

+

list-device

+

Lists all block devices or character devices in a container.

+

Supported options are as follows:

+
  • --pretty: outputs data in JSON format.
  • --sub-partition: For a primary disk, add this flag to display the primary disk and its sub-partitions.
+

None

+

update-device

+

Updates the disk QoS.

+

Supported options are as follows:

+
  • --device-read-bps: sets the read rate limit for the block device (byte/s). You are advised to set this parameter to a value greater than or equal to 1024.
  • --device-read-iops: sets the read rate limit for the block device (I/O/s).
  • --device-write-bps: sets the write rate limit for the block device (byte/s). You are advised to set this parameter to a value greater than or equal to 1024.
  • --device-write-iops: sets the write rate limit for the block device (I/O/s).
+

None

+
+ +### Constraints + +- You can add or delete devices when container instances are not running. After the operation is complete, you can start the container to view the device status. You can also dynamically add a device when the container is running. +- Do not concurrently run the **fdisk** command to format disks in a container and on the host. Otherwise, the container disk usage will be affected. +- When you run the **add-device** command to add a disk to a specific directory of a container, if the parent directory in the container is a multi-level directory \(for example, **/dev/a/b/c/d/e**\) and the directory level does not exist, isulad-tools will automatically create the corresponding directory in the container. When the disk is deleted, the created parent directory is not deleted. If you run the **add-device** command to add a device to this parent directory again, a message is displayed, indicating that a device already exists and cannot be added. +- When you run the**add-device** command to add a disk or update disk parameters, you need to configure the disk QoS. Do not set the write or read rate limit for the block device \(I/O/s or byte/s\) to a small value. If the value is too small, the disk may be unreadable \(the actual reason is the speed is too slow\), affecting service functions. +- When you run the **--blkio-weight-device** command to limit the weight of a specified block device, if the block device supports only the BFQ mode, an error may be reported, prompting you to check whether the current OS environment supports setting the weight of the BFQ block device. + +### Example + +- Start a system container, and set **hook spec** to the isulad hook execution script. + + ```shell + [root@localhost ~]# isula run -tid --hook-spec /etc/isulad-tools/hookspec.json --system-container --external-rootfs /root/root-fs none init + eed1096c8c7a0eca6d92b1b3bc3dd59a2a2adf4ce44f18f5372408ced88f8350 + ``` + +- Add a block device to a container. + + ```shell + [root@localhost ~]# isulad-tools add-device ee /dev/sdb:/dev/sdb123 + Add device (/dev/sdb) to container(ee,/dev/sdb123) done. + [root@localhost ~]# isula exec ee fdisk -l /dev/sdb123 + Disk /dev/sdb123: 50 GiB, 53687091200 bytes, 104857600 sectors + Units: sectors of 1 * 512 = 512 bytes + Sector size (logical/physical): 512 bytes / 512 bytes + I/O size (minimum/optimal): 512 bytes / 512 bytes + Disklabel type: dos + Disk identifier: 0xda58a448 + + Device Boot Start End Sectors Size Id Type + /dev/sdb123p1 2048 104857599 104855552 50G 5 Extended + /dev/sdb123p5 4096 104857599 104853504 50G 83 Linux + ``` + +- Update the device information. + + ```shell + [root@localhost ~]# isulad-tools update-device --device-read-bps /dev/sdb:10m ee + Update read bps for device (/dev/sdb,10485760) done. + ``` + +- Delete a device. + + ```shell + [root@localhost ~]# isulad-tools remove-device ee /dev/sdb:/dev/sdb123 + Remove device (/dev/sdb) from container(ee,/dev/sdb123) done. + Remove read bps for device (/dev/sdb) done. + ``` + +## NIC Management + +### Function Description + +isulad-tools allows you to insert physical or virtual NICs on the host to a container. If the NICs are not required, isulad-tools allows you to delete them from the container and return them to the host. In addition, the NIC configurations can be dynamically modified. To insert a physical NIC, add the NIC on the host to the container. To insert a virtual NIC, create a veth pair and insert its one end to the container. + +### Command Format + +```shell +isulad-tools [COMMAND][OPTIONS] +``` + +In the preceding format: + +**COMMAND**: command related to NIC management. + +**OPTIONS**: option supported by the NIC management command. + +**container\_id**: container ID. + +### Parameter Description + + + + + + + + + + + + + + + + + + + + + + + +

Command

+

Function Description

+

Option Description

+

add-nic

+

Creates an NIC for a container.

+

Supported options are as follows:

+
  • --type: specifies the NIC type. Only eth and veth are supported.
  • --name: specifies the NIC name. The format is [host:]container. If host is not specified, a random value is used.
  • --ip: specifies the NIC IP address.
  • --mac: specifies the NIC MAC address.
  • --bridge: specifies the network bridge bound to the NIC.
  • --mtu: specifies the MTU value of the NIC. The default value is 1500.
  • --update-config-only: If this flag is set, only configuration files are updated and NICs are not added.
  • --qlen: specifies the value of QLEN. The default value is 1000.
+

remove-nic

+

Deletes NICs from a container and restores them to the host.

+

Supported options are as follows:

+
  • --type: specifies the NIC type.
  • --name: specifies the name of the NIC. The format is [host:]container.
+

list-nic

+

Lists all NICs in a container.

+

Supported options are as follows:

+
  • --pretty: outputs data in JSON format.
  • --filter: outputs filtered data in the specific format, for example, --filter' {"ip":"192.168.3.4/24", "Mtu":1500}'.
+

update-nic

+

Modifies configuration parameters of a specified NIC in a container.

+

Supported options are as follows:

+
  • --name: specifies the name of the NIC in the container. This parameter is mandatory.
  • --ip: specifies the NIC IP address.
  • --mac: specifies the NIC MAC address.
  • --bridge: specifies the network bridge bound to the NIC.
  • --mtu: specifies the MTU value of the NIC.
  • --update-config-only: If this flag is set, configuration files are updated and NICs are not updated.
  • --qlen: specifies the value of QLEN.
+
+ +### Constraints + +- Physical NICs \(eth\) and virtual NICs \(veth\) can be added. +- When adding a NIC, you can also configure the NIC. The configuration parameters include **--ip**, **--mac**, **--bridge**, **--mtu**, **--qlen**. +- A maximum of eight physical NICs can be added to a container. +- If you run the **isulad-tools add-nic** command to add an eth NIC to a container and do not add a hook, you must manually delete the NIC before the container exits. Otherwise, the name of the eth NIC on the host will be changed to the name of that in the container. +- For a physical NIC \(except 1822 VF NIC\), use the original MAC address when running the **add-nic** command. Do not change the MAC address in the container, or when running the **update-nic** command. +- When using the **isulad-tools add-nic** command, set the MTU value. The value range depends on the NIC model. +- When using isulad-tools to add NICs and routes to containers, you are advised to run the **add-nic** command to add NICs and then run the **add-route** command to add routes. When using isulad-tools to delete NICs and routes from a container, you are advised to run the **remove-route** command to delete routes and then run the **remove-nic** command to delete NICs. +- When using isulad-tools to add NICs, add a NIC to only one container. + +### Example + +- Start a system container, and set **hook spec** to the isulad hook execution script. + + ```shell + [root@localhost ~]# isula run -tid --hook-spec /etc/isulad-tools/hookspec.json --system-container --external-rootfs /root/root-fs none init + 2aaca5c1af7c872798dac1a468528a2ccbaf20b39b73fc0201636936a3c32aa8 + ``` + +- Add a virtual NIC to a container. + + ```shell + [root@localhost ~]# isulad-tools add-nic --type "veth" --name abc2:bcd2 --ip 172.17.28.5/24 --mac 00:ff:48:13:xx:xx --bridge docker0 2aaca5c1af7c + Add network interface to container 2aaca5c1af7c (bcd2,abc2) done + ``` + +- Add a physical NIC to a container. + + ```shell + [root@localhost ~]# isulad-tools add-nic --type "eth" --name eth3:eth1 --ip 172.17.28.6/24 --mtu 1300 --qlen 2100 2aaca5c1af7c + Add network interface to container 2aaca5c1af7c (eth3,eth1) done + ``` + + > [!NOTE]NOTE + > When adding a virtual or physical NIC, ensure that the NIC is in the idle state. Adding a NIC in use will disconnect the system network. + +## Route Management + +### Function Description + +isulad-tools can be used to dynamically add or delete routing tables for system containers. + +### Command Format + +```shell +isulad-tools [COMMAND][OPTIONS] [ARG...] +``` + +In the preceding format: + +**COMMAND**: command related to route management. + +**OPTIONS**: option supported by the route management command. + +**container\_id**: container ID. + +**ARG**: parameter corresponding to the command. + +### API Description + + + + + + + + + + + + + + + + + + + + + + + +

Command

+

Function Description

+

Option Description

+

Parameter Description

+

add-route

+

Adds the network routing rules to a container.

+

Supported options are as follows:

+

--update-config-only: If this parameter is configured, configuration files are updated and routing tables are not updated.

+

Parameter format: [{rule1},{rule2}]

+

Example of rule:

+

'[{"dest":"default", "gw":"192.168.10.1"},{"dest":"192.168.0.0/16","dev":"eth0","src":"192.168.1.2"}]'

+
  • dest: target network. If this parameter is left blank, the default gateway is used.
  • src: source IP address of a route.
  • gw: route gateway.
  • dev: network device.
+

remove-route

+

Deletes a route from a container.

+

Supported options are as follows:

+

--update-config-only: If this parameter is configured, only configuration files are updated and routes are not deleted from the container.

+

Parameter format: [{rule1},{rule2}]

+

Example of rule:

+

'[{"dest":"default", "gw":"192.168.10.1"},{"dest":"192.168.0.0/16","dev":"eth0","src":"192.168.1.2"}]'

+
  • dest: target network. If this parameter is left blank, the default gateway is used.
  • src: source IP address of a route.
  • gw: route gateway.
  • dev: network device.
+

list-route

+

Lists all routing rules in a container.

+

Supported options are as follows:

+
  • --pretty: outputs data in JSON format.
  • --filter: outputs filtered data in the specific format, for example, --filter' {"ip":"192.168.3.4/24", "Mtu":1500}'.
+

None

+
+ +### Constraints + +- When using isulad-tools to add NICs and routes to containers, you are advised to run the **add-nic** command to add NICs and then run the **add-route** command to add routes. When using isulad-tools to delete NICs and routes from a container, you are advised to run the **remove-route** command to delete routes and then run the **remove-nic** command to delete NICs. +- When adding a routing rule to a container, ensure that the added routing rule does not conflict with existing routing rules in the container. + +### Example + +- Start a system container, and set **hook spec** to the isulad hook execution script. + + ```shell + [root@localhost ~]# isula run -tid --hook-spec /etc/isulad-tools/hookspec.json --system-container --external-rootfs /root/root-fs none init + 0d2d68b45aa0c1b8eaf890c06ab2d008eb8c5d91e78b1f8fe4d37b86fd2c190b + ``` + +- Use isulad-tools to add a physical NIC to the system container. + + ```shell + [root@localhost ~]# isulad-tools add-nic --type "eth" --name enp4s0:eth123 --ip 172.17.28.6/24 --mtu 1300 --qlen 2100 0d2d68b45aa0 + Add network interface (enp4s0) to container (0d2d68b45aa0,eth123) done + ``` + +- isulad-tools adds a routing rule to the system container. Format example: **\[\{"dest":"default", "gw":"192.168.10.1"\},\{"dest":"192.168.0.0/16","dev":"eth0","src":"192.168.1.2"\}\]**. If **dest** is left blank, its value will be **default**. + + ```shell + [root@localhost ~]# isulad-tools add-route 0d2d68b45aa0 '[{"dest":"172.17.28.0/32", "gw":"172.17.28.5","dev":"eth123"}]' + Add route to container 0d2d68b45aa0, route: {dest:172.17.28.0/32,src:,gw:172.17.28.5,dev:eth123} done + ``` + +- Check whether a routing rule is added in the container. + + ```shell + [root@localhost ~]# isula exec -it 0d2d68b45aa0 route + Kernel IP routing table + Destination Gateway Genmask Flags Metric Ref Use Iface + 172.17.28.0 172.17.28.5 255.255.255.255 UGH 0 0 0 eth123 + 172.17.28.0 0.0.0.0 255.255.255.0 U 0 0 0 eth123 + ``` + +## Volume Mounting Management + +### Function Description + +In a common container, you can set the **--volume** parameter during container creation to mount directories or volumes of the host to the container for resource sharing. However, during container running, you cannot unmount directories or volumes that are mounted to the container, or mount directories or volumes of the host to the container. Only the system container can use the isulad-tools tool to dynamically mount directories or volumes of the host to the container and unmount directories or volumes from the container. + +### Command Format + +```shell +isulad-tools [COMMAND][OPTIONS] [ARG...] +``` + +In the preceding format: + +**COMMAND**: command related to route management. + +**OPTIONS**: option supported by the route management command. + +**container\_id**: container ID. + +**ARG**: parameter corresponding to the command. + +### API Description + +**Table 1** + + + + + + + + + + + + + + + + + + + + + + + + +

Command

+

Function Description

+

Option Description

+

Parameter Description

+

add-path

+

Adds files or directories on the host to a container.

+

None

+

The parameter format is as follows:

+

hostpath:containerpath:permission [hostpath:containerpath:permission ...]

+

In the preceding format:

+

hostpath: path on the host for storing a volume.

+

containerpath: path on the container for storing a volume.

+

permission: operation permission on a mount path within the container.

+

remove-path

+

Deletes directories or files from the container and restores them to the host.

+

None

+

Parameter format: hostpath:containerpath[hostpath:containerpath ]

+

In the preceding format:

+

hostpath: path on the host for storing a volume.

+

containerpath: path on the container for storing a volume.

+

list-path

+

Lists all path directories in a container.

+

Supported options are as follows:

+

--pretty: outputs data in JSON format.

+

None

+
+ +### Constraints + +- When running the **add-path** command, specify an absolute path as the mount path. +- The mount point /.sharedpath is generated on the host after the mount path is specified by running the **add-path** command. +- A maximum of 128 volumes can be added to a container. +- Do not overwrite the root directory \(/\) in a container with the host directory by running the **add-path** command. Otherwise, the function is affected. + +### Example + +- Start a system container, and set **hook spec** to the isulad hook execution script. + + ```shell + [root@localhost ~]# isula run -tid --hook-spec /etc/isulad-tools/hookspec.json --system-container --external-rootfs /root/root-fs none init + e45970a522d1ea0e9cfe382c2b868d92e7b6a55be1dd239947dda1ee55f3c7f7 + ``` + +- Use isulad-tools to mount a directory on the host to a container, implementing resource sharing. + + ```shell + [root@localhost ~]# isulad-tools add-path e45970a522d1 /home/test123:/home/test123 + Add path (/home/test123) to container(e45970a522d1,/home/test123) done. + ``` + +- Create a file in the **/home/test123** directory on the host and check whether the file can be accessed in the container. + + ```shell + [root@localhost ~]# echo "hello world" > /home/test123/helloworld + [root@localhost ~]# isula exec e45970a522d1 bash + [root@localhost /]# cat /home/test123/helloworld + hello world + ``` + +- Use isulad-tools to delete the mount directory from the container. + + ```shell + [root@localhost ~]# isulad-tools remove-path e45970a522d1 /home/test123:/home/test123 + Remove path (/home/test123) from container(e45970a522d1,/home/test123) done + [root@localhost ~]# isula exec e45970a522d1 bash + [root@localhost /]# ls /home/test123/helloworld + ls: cannot access '/home/test123/helloworld': No such file or directory + ``` diff --git a/docs/en/cloud/container_form/system_container/environment-variable-persisting.md b/docs/en/cloud/container_form/system_container/environment-variable-persisting.md new file mode 100644 index 0000000000000000000000000000000000000000..c2348c0b7324d2569af565eaecb6d31d86d80430 --- /dev/null +++ b/docs/en/cloud/container_form/system_container/environment-variable-persisting.md @@ -0,0 +1,48 @@ +# Environment Variable Persisting + +- [Environment Variable Persisting](#environment-variable-persisting) + +## Function Description + +In a system container, you can make the **env** variable persistent to the configuration file in the rootfs directory of the container by specifying the **--env-target-file** interface parameter. + +## Parameter Description + + + + + + + + + + + +

Command

+

Parameter

+

Value Description

+

isula create/run

+

--env-target-file

+
  • Variable of the string type.
  • The env persistent file must be in the rootfs directory and must be an absolute path.
+
+ +## Constraints + +- If the target file specified by **--env-target-file** exists, the size cannot exceed 10 MB. +- The parameter specified by **--env-target-file** must be an absolute path in the rootfs directory. +- If the value of **--env** conflicts with that of **env** in the target file, the value of **--env** prevails. + +## Example + +Start a system container and specify the **env** environment variable and **--env-target-file** parameter. + +```shell +[root@localhost ~]# isula run -tid -e abc=123 --env-target-file /etc/environment --system-container --external-rootfs /root/myrootfs none init +b75df997a64da74518deb9a01d345e8df13eca6bcc36d6fe40c3e90ea1ee088e +[root@localhost ~]# isula exec b7 cat /etc/environment +PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin +TERM=xterm +abc=123 +``` + +The preceding information indicates that the **env** variable \(**abc=123**\) of the container has been made persistent to the **/etc/environment** configuration file. diff --git a/docs/en/cloud/container_form/system_container/installation-guideline.md b/docs/en/cloud/container_form/system_container/installation-guideline.md new file mode 100644 index 0000000000000000000000000000000000000000..1c60ca4ecb1038766531b2b8eb41ae4923c998eb --- /dev/null +++ b/docs/en/cloud/container_form/system_container/installation-guideline.md @@ -0,0 +1,26 @@ +# Installation Guideline + +1. Install the container engine iSulad. + + ```shell + # yum install iSulad + ``` + +2. Install dependent packages of system containers. + + ```shell + # yum install isulad-tools authz isulad-lxcfs-toolkit lxcfs + ``` + +3. Run the following command to check whether iSulad is started: + + ```shell + # systemctl status isulad + ``` + +4. Enable the lxcfs and authz services. + + ```shell + # systemctl start lxcfs + # systemctl start authz + ``` diff --git a/docs/en/cloud/container_form/system_container/maximum-number-of-handles.md b/docs/en/cloud/container_form/system_container/maximum-number-of-handles.md new file mode 100644 index 0000000000000000000000000000000000000000..08764a0a779bfa84941f5fef5bb4a9eaccedd1ec --- /dev/null +++ b/docs/en/cloud/container_form/system_container/maximum-number-of-handles.md @@ -0,0 +1,56 @@ +# Maximum Number of Handles + +- [Maximum Number of Handles](#maximum-number-of-handles) + +## Function Description + +System containers support limit on the number of file handles. File handles include common file handles and network sockets. When starting a container, you can specify the **--files-limit** parameter to limit the maximum number of handles opened in the container. + +## Parameter Description + + + + + + + + + + + +

Command

+

Parameter

+

Value Description

+

isula create/run

+

--files-limit

+

  

+
  • The value cannot be negative and must be an integer.
  • The value 0 indicates that the number is not limited by the parameter. The maximum number is determined by the current kernel files cgroup.
+
+ +## Constraints + +- If the value of **--files-limit** is too small, the system container may fail to run the **exec** command and the error "open temporary files" is reported. Therefore, you are advised to set the parameter to a large value. +- File handles include common file handles and network sockets. + +## Example + +To use **--files-limit** to limit the number of file handles opened in a container, run the following command to check whether the kernel supports files cgroup: + +```shell +[root@localhost ~]# cat /proc/1/cgroup | grep files +10:files:/ +``` + +If **files** is displayed, files cgroup is supported. + +Start the container, specify the **--files-limit** parameter, and check whether the **files.limit** parameter is successfully written. + +```shell +[root@localhost ~]# isula run -tid --files-limit 1024 --system-container --external-rootfs /tmp/root-fs empty init 01e82fcf97d4937aa1d96eb8067f9f23e4707b92de152328c3fc0ecb5f64e91d +[root@localhost ~]# isula exec -it 01e82fcf97d4 bash +[root@localhost ~]# cat /sys/fs/cgroup/files/files.limit +1024 + +``` + +The preceding information indicates that the number of file handles is successfully limited in the container. diff --git a/docs/en/cloud/container_form/system_container/overview.md b/docs/en/cloud/container_form/system_container/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..37861583d3becff0c5d5cb8134bcf5d44af561e7 --- /dev/null +++ b/docs/en/cloud/container_form/system_container/overview.md @@ -0,0 +1,3 @@ +# System Container + +System containers are used for heavyweight applications and cloud-based services in scenarios with re-computing, high performance, and high concurrency. Compared with the VM technology, system containers can directly inherit physical machine features and has better performance and less overhead. In addition, system containers can be allocated more computing units of limited resources, reducing costs. Therefore, system containers can be used to build differentiated product competitiveness and provide computing unit instances with higher computing density, lower price, and better performance. diff --git a/docs/en/cloud/container_form/system_container/public_sys-resources/icon-note.gif b/docs/en/cloud/container_form/system_container/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/cloud/container_form/system_container/public_sys-resources/icon-note.gif differ diff --git a/docs/en/cloud/container_form/system_container/reboot-or-shutdown-in-a-container.md b/docs/en/cloud/container_form/system_container/reboot-or-shutdown-in-a-container.md new file mode 100644 index 0000000000000000000000000000000000000000..b5e9b77f1215e3e9054fe9abb90e92a1eae1774d --- /dev/null +++ b/docs/en/cloud/container_form/system_container/reboot-or-shutdown-in-a-container.md @@ -0,0 +1,75 @@ +# Reboot or Shutdown in a Container + +## Function Description + +The **reboot** and **shutdown** commands can be executed in a system container. You can run the **reboot** command to restart a container, and run the **shutdown** command to stop a container. + +## Parameter Description + + + + + + + + + + + +

Command

+

Parameter

+

Value Description

+

isula create/run

+

--restart

+
  • Variable of the string type.
  • Supported option is as follows:

    on-reboot: restarts the system container.

    +

      

    +
+
+ +## Constraints + +- The shutdown function relies on the actual OS of the container running environment. +- When you run the **shutdown -h now** command to shut down the system, do not open multiple consoles. For example, if you run the **isula run -ti** command to open a console and run the **isula attach** command for the container in another host bash, another console is opened. In this case, the **shutdown** command fails to be executed. + +## Example + +- Specify the **--restart on-reboot** parameter when starting a container. For example: + + ```shell + [root@localhost ~]# isula run -tid --restart on-reboot --system-container --external-rootfs /root/myrootfs none init + 106faae22a926e22c828a0f2b63cf5c46e5d5986ea8a5b26de81390d0ed9714f + ``` + +- In the container, run the **reboot** command. + + ```shell + [root@localhost ~]# isula exec -it 10 bash + [root@localhost /]# reboot + ``` + + Check whether the container is restarted. + + ```shell + [root@localhost ~]# isula exec -it 10 ps aux + USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND + root 1 0.1 0.0 21588 9504 ? Ss 12:11 0:00 init + root 14 0.1 0.0 27024 9376 ? Ss 12:11 0:00 /usr/lib/system + root 17 0.0 0.0 18700 5876 ? Ss 12:11 0:00 /usr/lib/system + dbus 22 0.0 0.0 9048 3624 ? Ss 12:11 0:00 /usr/bin/dbus-d + root 26 0.0 0.0 8092 3012 ? Rs+ 12:13 0:00 ps aux + ``` + +- In the container, run the **shutdown** command. + + ```shell + [root@localhost ~]# isula exec -it 10 bash + [root@localhost /]# shutdown -h now + [root@localhost /]# [root@localhost ~]# + ``` + + Check whether the container is stopped. + + ```shell + [root@localhost ~]# isula exec -it 10 bash + Error response from daemon: Exec container error;Container is not running:106faae22a926e22c828a0f2b63cf5c46e5d5986ea8a5b26de81390d0ed9714f + ``` diff --git a/docs/en/cloud/container_form/system_container/security-and-isolation.md b/docs/en/cloud/container_form/system_container/security-and-isolation.md new file mode 100644 index 0000000000000000000000000000000000000000..266b1ede0d80ffd13bc84dcfa75b59126cbad42e --- /dev/null +++ b/docs/en/cloud/container_form/system_container/security-and-isolation.md @@ -0,0 +1,339 @@ +# Security and Isolation + +- [Security and Isolation](#security-and-isolation) + - [Many-to-Many User Namespaces](#many-to-many-user-namespaces) + - [User Permission Control](#user-permission-control) + - [proc File System Isolation](#proc-file-system-isolation) + +## Many-to-Many User Namespaces + +### Function Description + +User namespaces are used to map user **root** of a container to a common user of the host and allow the processes and user in the container \(that are unprivileged on the host\) to have privilege. This can prevent the processes in the container from escaping to the host and performing unauthorized operations. In addition, after user namespaces are used, the container and host use different UIDs and GIDs. This ensures that user resources in the container such as file descriptors are isolated from those on the host. + +In system containers, you can configure the **--user-remap** API parameter to map user namespaces of different containers to different user namespaces on the host, isolating the user namespaces of containers. + +### Parameter Description + + + + + + + + + + + +

Command

+

Parameter

+

Value Description

+

isula create/run

+

--user-remap

+

The parameter format is uid:gid:offset. The parameter is described as follows:

+
  • uid and gid must be integers greater than or equal to 0.
  • offset must be an integer greater than 0 and less than 65536. The value cannot be too small. Otherwise, the container cannot be started.
  • Either the sum of uid and offset or the sum of gid and offset must be less than or equal to 232 - 1. Otherwise, an error is reported during container startup.
+
+ +### Constraints + +- If **--user-remap** is specified in a system container, the rootfs directory must be accessible to users specified by _uid_ or _gid_ in **--user-remap**. Otherwise, user namespaces of containers cannot access rootfs. As a result, the containers fail to be started. +- All IDs in the container can be mapped to the host rootfs. Some directories or files may be mounted from the host to containers, for example, device files in the **/dev/pts** directory. If _offset_ is too small, the mounting may fail. +- _uid_, _gid_, and _offset_ are controlled by the upper-layer scheduling platform. The container engine only checks the validity of them. +- **--user-remap** is available only in system containers. +- **--user-remap** and **--privileged** cannot be set simultaneously. Otherwise, an error is reported during container startup. +- If _uid_ or _gid_ is set to **0**, **--user-remap** does not take effect. +- If **--user-map** is specified for a system container, ensure that the user corresponding to the specified UID or GID can access the isulad metadata directories (**/var/lib/isulad/**, **/var/lib/isulad/engines/**, and **/var/lib/isulad/engines/lcr**). +- **--user-remap** and **--userns** cannot be specified at the same time. + +### Usage Guide + +> [!NOTE]NOTE +> Before specifying the **--user-remap** parameter, configure an offset value for UIDs and GIDs of all directories and files in rootfs. The offset value should be equal to that for _uid_ and _gid_ in **--user-remap**. +> For example, run the following command to offset UIDs and GIDs of all files in the **dev** directory with 100000: +> chown 100000:100000 dev + +Specify the **--user-remap** parameter when the system container is started. + +```shell +[root@localhost ~]# chmod 751 /var/lib/isulad/ +[root@localhost ~]# chmod 751 /var/lib/isulad/engines/ +[root@localhost ~]# chmod 751 /var/lib/isulad/engines/lcr +[root@localhost ~]# isula run -tid --user-remap 100000:100000:65535 --system-container --external-rootfs /home/root-fs none /sbin/init +eb9605b3b56dfae9e0b696a729d5e1805af900af6ce24428fde63f3b0a443f4a +``` + +Check the /sbin/init process information on the host and in a container. + +```shell +[root@localhost ~]# isula exec eb ps aux | grep /sbin/init +root 1 0.6 0.0 21624 9624 ? Ss 15:47 0:00 /sbin/init +[root@localhost ~]# ps aux | grep /sbin/init +100000 4861 0.5 0.0 21624 9624 ? Ss 15:47 0:00 /sbin/init +root 4948 0.0 0.0 213032 808 pts/0 S+ 15:48 0:00 grep --color=auto /sbin/init +``` + +The owner of the /sbin/init process in the container is user **root**, but the owner of the host is the user whose UID is **100000**. + +Create a file in a container and view the file owner on the host. + +```shell +[root@localhost ~]# isula exec -it eb bash +[root@localhost /]# echo test123 >> /test123 +[root@localhost /]# exit +exit +[root@localhost ~]# ll /home/root-fs/test123 +-rw-------. 1 100000 100000 8 Aug 2 15:52 /home/root-fs/test123 +``` + +The owner of the file that is generated in the container is user **root**, but the file owner displayed on the host is the user whose ID is **100000**. + +## User Permission Control + +### Function Description + +A container engine supports TLS for user identity authentication, which is used to control user permissions. Currently, container engines can connect to the authz plug-in to implement permission control. + +### API Description + +You can configure the startup parameters of the iSulad container engine to specify the permission control plug-in. The default daemon configuration file is **/etc/isulad/daemon.json**. + + + + + + + + + + + +

Parameter

+

Example

+

Description

+

--authorization-plugin

+

"authorization-plugin": "authz-broker"

+

User permission authentication plug-in. Currently, only authz-broker is supported.

+
+ +### Constraints + +- User permission policies need to be configured for authz. The default policy file is **/var/lib/authz-broker/policy.json**. This file can be dynamically modified and the modification will take effect immediately without restarting the plug-in service. +- A container engine can be started by user **root**. If some commands used are enabled for by common users, common users may obtain excessive permissions. Therefore, exercise caution when performing such operations. Currently, running the **container\_attach**, **container\_create**, and **container\_exec\_create** commands may cause risks. +- Some compound operations, such as running **isula exec** and **isula inspect** or running and **isula attach** and **isula inspect**, depend on the permission of **isula inspect**. If a user does not have this permission, an error is reported. +- Using SSL/TLS encryption channels hardens security but also reduces performance. For example, the delay increases, more CPU resources are consumed, and encryption and decryption require higher throughput. Therefore, the number of concurrent executions decreases compared with non-TLS communication. According to the test result, when the ARM server \(Cortex-A72 64-core\) is almost unloaded, TLS is used to concurrently start a container. The maximum number of concurrent executions is 200 to 250. +- If **--tlsverify** is specified on the server, the default path where authentication files store is **/etc/isulad**. The default file names are **ca.pem**, **cert.pem**, and **key.pem**. + +### Example + +1. Ensure that the authz plug-in is installed on the host. If the authz plug-in is not installed, run the following command to install and start the authz plug-in service: + + ```shell + [root@localhost ~]# yum install authz + [root@localhost ~]# systemctl start authz + ``` + +2. To enable this function, configure the container engine and TLS certificate. You can use OpenSSL to generate the required certificate. + + ```shell + #SERVERSIDE + + # Generate CA key + openssl genrsa -aes256 -passout "pass:$PASSWORD" -out "ca-key.pem" 4096 + # Generate CA + openssl req -new -x509 -days $VALIDITY -key "ca-key.pem" -sha256 -out "ca.pem" -passin "pass:$PASSWORD" -subj "/C=$COUNTRY/ST=$STATE/L=$CITY/O=$ORGANIZATION/OU=$ORGANIZATIONAL_UNIT/CN=$COMMON_NAME/emailAddress=$EMAIL" + # Generate Server key + openssl genrsa -out "server-key.pem" 4096 + + # Generate Server Certs. + openssl req -subj "/CN=$COMMON_NAME" -sha256 -new -key "server-key.pem" -out server.csr + + echo "subjectAltName = DNS:localhost,IP:127.0.0.1" > extfile.cnf + echo "extendedKeyUsage = serverAuth" >> extfile.cnf + + openssl x509 -req -days $VALIDITY -sha256 -in server.csr -passin "pass:$PASSWORD" -CA "ca.pem" -CAkey "ca-key.pem" -CAcreateserial -out "server-cert.pem" -extfile extfile.cnf + + #CLIENTSIDE + + openssl genrsa -out "key.pem" 4096 + openssl req -subj "/CN=$CLIENT_NAME" -new -key "key.pem" -out client.csr + echo "extendedKeyUsage = clientAuth" > extfile.cnf + openssl x509 -req -days $VALIDITY -sha256 -in client.csr -passin "pass:$PASSWORD" -CA "ca.pem" -CAkey "ca-key.pem" -CAcreateserial -out "cert.pem" -extfile extfile.cnf + ``` + + If you want to use the preceding content as the script, replace the variables with the configured values. If the parameter used for generating the CA is empty, set it to **"**. **PASSWORD**, **COMMON\_NAME**, **CLIENT\_NAME**, and **VALIDITY** are mandatory. + +3. When starting the container engine, add parameters related to the TLS and authentication plug-in and ensure that the authentication plug-in is running properly. In addition, to use TLS authentication, the container engine must be started in TCP listening mode instead of the Unix socket mode. The configuration on the container daemon is as follows: + + ```json + { + "tls": true, + "tls-verify": true, + "tls-config": { + "CAFile": "/root/.iSulad/ca.pem", + "CertFile": "/root/.iSulad/server-cert.pem", + "KeyFile":"/root/.iSulad/server-key.pem" + }, + "authorization-plugin": "authz-broker" + } + ``` + +4. Configure policies. For the basic authorization process, all policies are stored in the **/var/lib/authz-broker/policy.json** configuration file. The configuration file can be dynamically modified without restarting the plug-in. Only the SIGHUP signal needs to be sent to the authz process. In the file, a line contains one JSON policy object. The following provides policy configuration examples: + + - All users can run all iSuald commands: **\{"name":"policy\_0","users":\[""\],"actions":\[""\]\}** + - Alice can run all iSulad commands: **\{"name":"policy\_1","users":\["alice"\],"actions":\[""\]\}** + - A blank user can run all iSulad commands: **\{"name":"policy\_2","users":\[""\],"actions":\[""\]\}** + - Alice and Bob can create new containers: **\{"name":"policy\_3","users":\["alice","bob"\],"actions":\["container\_create"\]\}** + - service\_account can read logs and run **docker top**: **\{"name":"policy\_4","users":\["service\_account"\],"actions":\["container\_logs","container\_top"\]\}** + - Alice can perform any container operations: **\{"name":"policy\_5","users":\["alice"\],"actions":\["container"\]\}** + - Alice can perform any container operations, but the request type can only be **get**: **\{"name":"policy\_5","users":\["alice"\],"actions":\["container"\], "readonly":true\}** + + > [!NOTE]NOTE + - **actions** supports regular expressions. + - **users** does not support regular expressions. + - A users cannot be repeatedly specified by **users**. That is, a user cannot match multiple rules. + +5. After updating the configurations, configure TLS parameters on the client to connect to the container engine. That is, access the container engine with restricted permissions. + + ```shell + [root@localhost ~]# isula version --tlsverify --tlscacert=/root/.iSulad/ca.pem --tlscert=/root/.iSulad/cert.pem --tlskey=/root/.iSulad/key.pem -H=tcp://127.0.0.1:2375 + ``` + + If you want to use the TLS authentication for default client connection, move the configuration file to **\~/.iSulad** and set the **ISULAD\_HOST** and **ISULAD\_TLS\_VERIFY** variables \(rather than transferring **-H=tcp://$HOST:2375** and -**-tlsverify** during each call\). + + ```shell + [root@localhost ~]# mkdir -pv ~/.iSulad + [root@localhost ~]# cp -v {ca,cert,key}.pem ~/.iSulad + [root@localhost ~]# export ISULAD_HOST=localhost:2375 ISULAD_TLS_VERIFY=1 + [root@localhost ~]# isula version + ``` + +## proc File System Isolation + +### Application Scenario + +Container virtualization is lightweight and efficient, and can be quickly deployed. However, containers are not strongly isolated, which causes great inconvenience to users. Containers have some defects in isolation because the namespace feature of the Linux kernel is not perfect. For example, you can view the proc information on the host \(such as meminfo, cpuinfo, stat, and uptime\) in the proc file system of a container. You can use the lxcfs tool to replace the /proc content of instances in the container with the content in the /proc file system of the host so that services in the container can obtain the correct resource value. + +### API Description + +A system container provides two tool packages: lxcfs and lxcfs-toolkit, which are used together. Lxcfs resides on the host as the daemon process. lxcfs-toolkit mounts the lxcfs file system of the host to containers through the hook mechanism. + +The command line of lxcfs-toolkit is as follows: + +```shell +lxcfs-toolkit [OPTIONS] COMMAND [COMMAND_OPTIONS] +``` + + + + + + + + + + + + + + + + + + + + + + + +

Command

+

Function

+

Parameter

+

remount

+

Remounts lxcfs to containers.

+

--all: remounts lxcfs to all containers.

+

--container-id: remounts lxcfs to a specified container.

+

umount

+

Unmounts lxcfs from containers.

+

--all: unmounts lxcfs from all containers.

+

--container-id: unmounts lxcfs from a specified container.

+

check-lxcfs

+

Checks whether the lxcfs service is running properly.

+

None

+

prestart

+

Mounts the /var/lib/lxcfs directory to the container before the lxcfs service starts.

+

None

+
+ +### Constraints + +- Currently, only the **cpuinfo**, **meminfo**, **stat**, **diskstats**, **partitions**, **swaps**, and **uptime** files in the proc file system are supported. Other files are not isolated from other kernel API file systems \(such as sysfs\). +- After an RPM package is installed, a sample JSON file is generated in **/var/lib/lcrd/hooks/hookspec.json**. To add the log function, you need to add the **--log** configuration during customization. +- The **diskstats** file displays only information about disks that support CFQ scheduling, instead of partition information. Devices in containers are displayed as names in the **/dev** directory. If a device name does not exist, the information is left blank. In addition, the device where the container root directory is located is displayed as **sda**. +- The **slave** parameter is required when lxcfs is mounted. If the **shared** parameter is used, the mount point in containers may be leaked to the host, affecting the host running. +- Lxcfs supports graceful service degradation. If the lxcfs service crashes or becomes unavailable, the **cpuinfo**, **meminfo**, **stat**, **diskstats**, **partitions**, **swaps**and **uptime** files in containers are about host information, and other service functions of containers are not affected. +- Bottom layer of lxcfs depends on the FUSE kernel module and libfuse library. Therefore, the kernel needs to support FUSE. +- Lxcfs supports only the running of 64-bit applications in containers. If a 32-bit application is running in a container, the CPU information \(**cpuinfo**\) read by the application may fail to meet expectations. +- Lxcfs simulates the resource view only of container control groups \(cgroups\). Therefore, system calls \(such as sysconf\) in containers can obtain only host information. Lxcfs cannot implement the kernel isolation. +- The CPU information \(**cpuinfo**\) displayed after lxcfs implements the isolation has the following features: + - **processor**: The value increases from 0. + - **physical id**: The value increases from 0. + - **sibliing**: It has a fixed value of **1**. + - **core id**: It has a fixed value of **0**. + - **cpu cores**: It has a fixed value of **1**. + +### Example + +1. Install the lxcfs and lxcfs-toolkit packages and start the lxcfs service. + + ```shell + [root@localhost ~]# yum install lxcfs lxcfs-toolkit + [root@localhost ~]# systemctl start lxcfs + ``` + +2. After a container is started, check whether the lxcfs mount point exists in the container. + + ```shell + [root@localhost ~]# isula run -tid -v /var/lib/lxc:/var/lib/lxc --hook-spec /var/lib/isulad/hooks/hookspec.json --system-container --external-rootfs /home/root-fs none init + a8acea9fea1337d9fd8270f41c1a3de5bceb77966e03751346576716eefa9782 + [root@localhost ~]# isula exec a8 mount | grep lxcfs + lxcfs on /var/lib/lxc/lxcfs type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) + lxcfs on /proc/cpuinfo type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) + lxcfs on /proc/diskstats type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) + lxcfs on /proc/meminfo type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) + lxcfs on /proc/partitions type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) + lxcfs on /proc/stat type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) + lxcfs on /proc/swaps type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) + lxcfs on /proc/uptime type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) + ``` + +3. Run the **update** command to update the CPU and memory resource configurations of the container and check the container resources. As shown in the following command output, the container resource view displays the actual container resource data instead of data of the host. + + ```shell + [root@localhost ~]# isula update --cpuset-cpus 0-1 --memory 1G a8 + a8 + [root@localhost ~]# isula exec a8 cat /proc/cpuinfo + processor : 0 + BogoMIPS : 100.00 + cpu MHz : 2400.000 + Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid + CPU implementer : 0x41 + CPU architecture: 8 + CPU variant : 0x0 + CPU part : 0xd08 + CPU revision : 2 + + processor : 1 + BogoMIPS : 100.00 + cpu MHz : 2400.000 + Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid + CPU implementer : 0x41 + CPU architecture: 8 + CPU variant : 0x0 + CPU part : 0xd08 + CPU revision : 2 + + [root@localhost ~]# isula exec a8 free -m + total used free shared buff/cache available + Mem: 1024 17 997 7 8 1006 + Swap: 4095 0 4095 + ``` diff --git a/docs/en/cloud/container_form/system_container/shared-memory-channels.md b/docs/en/cloud/container_form/system_container/shared-memory-channels.md new file mode 100644 index 0000000000000000000000000000000000000000..4dadb8c54138f5a6108ea43b0cd6bccf640de824 --- /dev/null +++ b/docs/en/cloud/container_form/system_container/shared-memory-channels.md @@ -0,0 +1,55 @@ +# Shared Memory Channels + +## Function Description + +System containers enable the communication between container and host processes through shared memory. You can set the **--host-channel** parameter when creating a container to allow the host to share the same tmpfs with the container so that they can communicate with each other. + +## Parameter Description + + + + + + + + + + + +

Command

+

Parameter

+

Value Description

+

isula create/run

+

--host-channel

+
  • Variable of the string type. Its format is as follows:
    <host path>:<container path>:<rw/ro>:<size limit>
    +
  • The parameter is described as follows:

    <host path>: path to which tmpfs is mounted on the host, which must be an absolute path.

    +

    <container path>: path to which tmpfs is mounted in a container, which must be an absolute path.

    +

    <rw/ro>: permissions on the file system mounted to the container. The value can only be rw (read and write) or ro (read only). The default value is rw.

    +

    <size limit>: maximum size used by the mounted tmpfs. The minimum value is one 4 KB physical page, and the maximum value is half of the total physical memory in the system. The default value is 64MB.

    +
+
+ +## Constraints + +- The lifecycle of tmpfs mounted on the host starts from the container startup to the container deletion. After a container is deleted and its occupied space is released, the space is removed. +- When a container is deleted, the path to which tmpfs is mounted on the host is deleted. Therefore, an existing directory on the host cannot be used as the mount path. +- To ensure that processes running by non-root users on the host can communicate with containers, the permission for tmpfs mounted on the host is 1777. + +## Example + +Specify the **--host-channel** parameter when creating a container. + +```shell +[root@localhost ~]# isula run --rm -it --host-channel /testdir:/testdir:rw:32M --system-container --external-rootfs /root/myrootfs none init +root@3b947668eb54:/# dd if=/dev/zero of=/testdir/test.file bs=1024 count=64K +dd: error writing '/testdir/test.file': No space left on device +32769+0 records in +32768+0 records out +33554432 bytes (34 MB, 32 MiB) copied, 0.0766899 s, 438 MB/s +``` + +> [!NOTE]NOTE +> +> - If **--host-channel** is used for size limit, the file size is constrained by the memory limit in the container. \(The OOM error may occur when the memory usage reaches the upper limit.\) +> - If a user creates a shared file on the host, the file size is not constrained by the memory limit in the container. +> - If you need to create a shared file in the container and the service is memory-intensive, you can add the value of **--host-channel** to the original value of the container memory limit, eliminating the impact. diff --git a/docs/en/cloud/container_form/system_container/specifying-rootfs-to-create-a-container.md b/docs/en/cloud/container_form/system_container/specifying-rootfs-to-create-a-container.md new file mode 100644 index 0000000000000000000000000000000000000000..72909713176e60d949269b2626a93642f3459905 --- /dev/null +++ b/docs/en/cloud/container_form/system_container/specifying-rootfs-to-create-a-container.md @@ -0,0 +1,45 @@ +# Specifying Rootfs to Create a Container + +## Function Description + +Different from a common container that needs to be started by specifying a container image, a system container is started by specifying a local root file system \(rootfs\) using the **--external-rootfs** parameter. The rootfs contains the operating system environment on which the container depends during running. + +## Parameter Description + + + + + + + + + + + + +

Command

+

Parameter

+

Value Description

+

isula create/run

+

--external-rootfs

+
  • Variable of the string type.
  • Absolute path in the root file system of the container, that is, the path of the rootfs.
+
+ +## Constraints + +- The rootfs directory specified using the **--external-rootfs** parameter must be an absolute path. +- The rootfs directory specified using the **--external-rootfs** parameter must be a complete OS environment including **systemd** package. Otherwise, the container fails to be started. +- When a container is deleted, the rootfs directory specified using **--external-rootfs** is not deleted. +- Containers based on an ARM rootfs cannot run in the x86 environment. Containers based on an x86 rootfs cannot run in the ARM environment. +- You are advised not to start multiple container instances in the same rootfs. That is, one rootfs is used by only one container instance that is in the lifecycle. + +## Example + +Assuming the local rootfs path is **/root/myrootfs**, run the following command to start a system container: + +```shell +# isula run -tid --system-container --external-rootfs /root/myrootfs none init +``` + +> [!NOTE]NOTE +> The rootfs is a user-defined file system. Prepare it by yourself. For example, a rootfs is generated after the TAR package of a container image is decompressed. diff --git a/docs/en/cloud/container_form/system_container/usage-guide.md b/docs/en/cloud/container_form/system_container/usage-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..7abb45215aaffac6812844c02068546f68ab77f2 --- /dev/null +++ b/docs/en/cloud/container_form/system_container/usage-guide.md @@ -0,0 +1,17 @@ +# Usage Guide + +System container functions are enhanced based on the iSula container engine. The container management function and the command format of the function provided by system containers are the same as those provided by the iSula container engine. + +The following sections describe how to use the enhanced functions provided by system containers. For details about other command operations, see iSulad container engine documents. + +The system container functions involve only the **isula create/run** command. Unless otherwise specified, this command is used for all functions. The command format is as follows: + +```shell +isula create/run [OPTIONS] [COMMAND] [ARG...] +``` + +In the preceding format: + +- **OPTIONS**: one or more command parameters. For details about supported parameters, see iSulad container engine [appendix](../../container_engine/isula_container_engine/appendix.md#command-line-parameters). +- **COMMAND**: command executed after a system container is started. +- **ARG**: parameter corresponding to the command executed after a system container is started. diff --git a/docs/en/cloud/container_form/system_container/using-systemd-to-start-a-container.md b/docs/en/cloud/container_form/system_container/using-systemd-to-start-a-container.md new file mode 100644 index 0000000000000000000000000000000000000000..db4ddb666d2d31097f18c17299842301139c00a7 --- /dev/null +++ b/docs/en/cloud/container_form/system_container/using-systemd-to-start-a-container.md @@ -0,0 +1,87 @@ +# Using systemd to Start a Container + +- [Using systemd to Start a Container](#using-systemd-to-start-a-container) + +## Function Description + +The init process started in system containers differs from that in common containers. Common containers cannot start system services through systemd. However, system containers have this capability. You can enable the systemd service by specifying the **--system-container** parameter when starting a system container. + +## Parameter Description + + + + + + + + + + + + +

Command

+

Parameter

+

Value Description

+

isula create/run

+

--system-container

+
  • The value is of a Boolean data type and can be true or false. The default value is true.
  • Specifies whether it is a system container. This function must be enabled.
+
+ +## Constraints + +- The systemd service needs to call some special system APIs, including mount, umount2, unshare, reboot, and name\_to\_handle\_at. Therefore, permissions to call the preceding APIs are enabled for system containers when the privileged container tag is disabled. +- All system containers are started by the init process. The init process does not respond to the SIGTERM signal which indicates normal exit. By default, the **stop** command forcibly kills the container 10 seconds later. If you need a quicker stop, you can manually specify the timeout duration of the **stop** command. +- **--system-container** must be used together with **--external-rootfs**. +- Various services can run in a system container. The **systemctl** command is used to manage the service starting and stopping. Services may depend on each other. As a result, when an exception occurs, some service processes are in the D or Z state so that the container cannot exit properly. +- Some service processes in a system container may affect other operation results. For example, if the NetworkManager service is running in the container, adding NICs to the container may be affected \(the NICs are successfully added but then stopped by the NetworkManger\), resulting in unexpected results. +- Currently, system containers and hosts cannot be isolated by using udev events. Therefore, the **fstab** file cannot be configured. +- The systemd service may conflict with the cgconfig service provided by libcgroup. You are advised to delete the libcgroup-related packages from a container or set **Delegate** of the cgconfig service to **no**. + +## Example + +- Specify the **--system-container** and **--external-rootfs** parameters to start a system container. + + ```shell + [root@localhost ~]# isula run -tid -n systest01 --system-container --external-rootfs /root/myrootfs none init + ``` + +- After the preceding commands are executed, the container is running properly. You can run the **exec** command to access the container and view the process information. The command output indicates that the systemd service has been started. + + ```shell + [root@localhost ~]# isula exec -it systest01 bash + [root@localhost /]# ps -ef + UID PID PPID C STIME TTY TIME CMD + root 1 0 2 06:49 ? 00:00:00 init + root 14 1 2 06:49 ? 00:00:00 /usr/lib/systemd/systemd-journal + root 16 1 0 06:49 ? 00:00:00 /usr/lib/systemd/systemd-network + dbus 23 1 0 06:49 ? 00:00:00 /usr/bin/dbus-daemon --system -- + root 25 0 0 06:49 ? 00:00:00 bash + root 59 25 0 06:49 ? 00:00:00 ps -ef + ``` + +- Run the **systemctl** command in the container to check the service status. The command output indicates that the service is managed by systemd. + + ```shell + [root@localhost /]# systemctl status dbus + ● dbus.service - D-Bus System Message Bus + Loaded: loaded (/usr/lib/systemd/system/dbus.service; static; vendor preset: + disabled) + Active: active (running) since Mon 2019-07-22 06:49:38 UTC; 2min 5 + 8s ago + Docs: man:dbus-daemon(1) + Main PID: 23 (dbus-daemon) + CGroup: /system.slice/dbus.service + └─23 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidf + ile --systemd-activation --syslog-only + + Jul 22 06:49:38 localhost systemd[1]: Started D-Bus System Message Bus. + ``` + +- Run the **systemctl** command in the container to stop or start the service. The command output indicates that the service is managed by systemd. + + ```shell + [root@localhost /]# systemctl stop dbus + Warning: Stopping dbus.service, but it can still be activated by: + dbus.socket + [root@localhost /]# systemctl start dbus + ``` diff --git a/docs/en/cloud/container_form/system_container/writable-namespace-kernel-parameters.md b/docs/en/cloud/container_form/system_container/writable-namespace-kernel-parameters.md new file mode 100644 index 0000000000000000000000000000000000000000..6154883b5d8133b6ab3d55f1e610064517256226 --- /dev/null +++ b/docs/en/cloud/container_form/system_container/writable-namespace-kernel-parameters.md @@ -0,0 +1,88 @@ +# Writable Namespace Kernel Parameters + +- [Writable Namespace Kernel Parameters](#writable-namespace-kernel-parameters) + +## Function Description + +For services running in containers, such as databases, big data, and common applications, some kernel parameters need to be set and adjusted to obtain the optimal performance and reliability. The modification permission of all kernel parameters must be disabled or enabled simultaneously \(by using privileged container\). + +When the modification permission is disabled, only the --sysctl external interface is provided and parameters cannot be flexibly modified in a container. + +When the modification permission is enabled, some kernel parameters are globally valid. If some parameters are modified in a container, all programs on the host will be affected, harming security. + +System containers provide the **--ns-change-opt** parameter, which can be used to dynamically set namespace kernel parameters in a container. The parameter value can be **net** or **ipc**. + +## Parameter Description + + + + + + + + + + + +

Command

+

Parameter

+

Value Description

+

isula create/run

+

--ns-change-opt

+
  • Variable of the string type.
  • The parameter value can be net or ipc.

    net: All namespace parameters in the /proc/sys/net directory are supported.

    +

    ipc: Supported namespace parameters are as follows:

    +

    /proc/sys/kernel/msgmax

    +

    /proc/sys/kernel/msgmnb

    +

    /proc/sys/kernel/msgmni

    +

    /proc/sys/kernel/sem

    +

    /proc/sys/kernel/shmall

    +

    /proc/sys/kernel/shmmax

    +

    /proc/sys/kernel/shmmni

    +

    /proc/sys/kernel/shm_rmid_forced

    +

    /proc/sys/fs/mqueue/msg_default

    +

    /proc/sys/fs/mqueue/msg_max

    +

    /proc/sys/fs/mqueue/msgsize_default

    +

    /proc/sys/fs/mqueue/msgsize_max

    +

    /proc/sys/fs/mqueue/queues_max

    +
  • You can specify multiple namespace configurations and separate them with commas (,). For example, --ns-change-opt=net,ipc.
+
+ +## Constraints + +- If both **--privileged** \(privileged container\) and **--ns-change-opt** are specified during container startup, **--ns-change-opt** does not take effect. + +## Example + +Start a container and set **--ns-change-opt** to **net**. + +```shell +[root@localhost ~]# isula run -tid --ns-change-opt net --system-container --external-rootfs /root/myrootfs none init +4bf44a42b4a14fdaf127616c90defa64b4b532b18efd15b62a71cbf99ebc12d2 +[root@localhost ~]# isula exec -it 4b mount | grep /proc/sys +proc on /proc/sys type proc (ro,nosuid,nodev,noexec,relatime) +proc on /proc/sysrq-trigger type proc (ro,nosuid,nodev,noexec,relatime) +proc on /proc/sys/net type proc (rw,nosuid,nodev,noexec,relatime) +``` + +The mount point **/proc/sys/net** in the container has the **rw** option, indicating that the **net**-related namespace kernel parameters have the read and write permissions. + +Start another container and set **--ns-change-opt** to **ipc**. + +```shell +[root@localhost ~]# isula run -tid --ns-change-opt ipc --system-container --external-rootfs /root/myrootfs none init +c62e5e5686d390500dab2fa76b6c44f5f8da383a4cbbeac12cfada1b07d6c47f +[root@localhost ~]# isula exec -it c6 mount | grep /proc/sys +proc on /proc/sys type proc (ro,nosuid,nodev,noexec,relatime) +proc on /proc/sysrq-trigger type proc (ro,nosuid,nodev,noexec,relatime) +proc on /proc/sys/kernel/shmmax type proc (rw,nosuid,nodev,noexec,relatime) +proc on /proc/sys/kernel/shmmni type proc (rw,nosuid,nodev,noexec,relatime) +proc on /proc/sys/kernel/shmall type proc (rw,nosuid,nodev,noexec,relatime) +proc on /proc/sys/kernel/shm_rmid_forced type proc (rw,nosuid,nodev,noexec,relatime) +proc on /proc/sys/kernel/msgmax type proc (rw,nosuid,nodev,noexec,relatime) +proc on /proc/sys/kernel/msgmni type proc (rw,nosuid,nodev,noexec,relatime) +proc on /proc/sys/kernel/msgmnb type proc (rw,nosuid,nodev,noexec,relatime) +proc on /proc/sys/kernel/sem type proc (rw,nosuid,nodev,noexec,relatime) +proc on /proc/sys/fs/mqueue type proc (rw,nosuid,nodev,noexec,relatime) +``` + +The mount point information of **ipc**-related kernel parameters in the container contains the **rw** option, indicating that the **ipc**-related namespace kernel parameters have the read and write permissions. diff --git a/docs/en/cloud/container_runtime/kuasar/_toc.yaml b/docs/en/cloud/container_runtime/kuasar/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..fb57ab48ed4e2c4ce106f1103c142458e0674ae8 --- /dev/null +++ b/docs/en/cloud/container_runtime/kuasar/_toc.yaml @@ -0,0 +1,14 @@ +label: Kuasar Multi-Sandbox Container Runtime +isManual: true +description: >- + Kuasar enables unified management of diverse sandbox types and supports + multiple leading sandbox isolation technologies used in the industry. +sections: + - label: Overview + href: ./overview.md + - label: Installation and Configuration + href: ./kuasar-install-config.md + - label: Usage Instructions + href: ./kuasar-usage.md + - label: Appendix + href: ./kuasar-appendix.md diff --git a/docs/en/cloud/container_runtime/kuasar/figures/kuasar_arch.png b/docs/en/cloud/container_runtime/kuasar/figures/kuasar_arch.png new file mode 100644 index 0000000000000000000000000000000000000000..69fbf889891c2f678590a929d1f91d5139569c27 Binary files /dev/null and b/docs/en/cloud/container_runtime/kuasar/figures/kuasar_arch.png differ diff --git a/docs/en/cloud/container_runtime/kuasar/kuasar-appendix.md b/docs/en/cloud/container_runtime/kuasar/kuasar-appendix.md new file mode 100644 index 0000000000000000000000000000000000000000..f2d7af6a8de4cb6df136e1976a98be843fa1f3bc --- /dev/null +++ b/docs/en/cloud/container_runtime/kuasar/kuasar-appendix.md @@ -0,0 +1,24 @@ +# Appendix + +Fields in the **/var/lib/kuasar/config_stratovirt.toml** configuration file: + +```text +[sandbox] +log_level: Kuasar log level. The default value is info. + +[hypervisor] +path: path of the StratoVirt binary file +machine_type: the processor type to be simulated (virt for the Arm architecture and q35 for the x86 architecture) +kernel_path: execution path of the guest kernel +image_path: execution path of the guest image +initrd_path: execution path of the guest initrd (Configure either initrd_path or image_path.) +kernel_params: guest kernel parameters +vcpus: default number of vCPUs for each sandbox (default: 1) +memory_in_mb: default memory size of each sandbox (default: 1024 MiB) +block_device_driver: block device driver +debug: whether to enable debug mode +enable_mem_prealloc: whether to enable memory pre-allocation + +[hypervisor.virtiofsd_conf] +path: path of vhost_user_fs +``` diff --git a/docs/en/cloud/container_runtime/kuasar/kuasar-install-config.md b/docs/en/cloud/container_runtime/kuasar/kuasar-install-config.md new file mode 100644 index 0000000000000000000000000000000000000000..2dc04ea5d3daab641e2739bc1bd20d821674868a --- /dev/null +++ b/docs/en/cloud/container_runtime/kuasar/kuasar-install-config.md @@ -0,0 +1,126 @@ +# Installation and Configuration + +## Installation + +### Prerequisites + +- To obtain better performance experience, Kuasar must run on bare metal servers. **Currently, Kuasar cannot run on VMs.** +- The running of Kuasar depends on the following openEuler components. Ensure that the dependent components of the required versions have been installed in the environment. + - iSulad (See [Installation and Configuration](../../container_engine/isula_container_engine/installation-configuration.md) of iSulad.) + - StratoVirt (See [Installing StratoVirt](../../../virtualization/virtualization_platform/stratovirt/install_stratovirt.md)) + +### Procedure + +1. The Kuasar deliverables are included in the **kuasar** RPM package. Run the `yum` command to directly install Kuasar. + + ```sh + yum install kuasar + ``` + +2. Install the CRI command line tool crictl required for starting sandboxes and containers. + + ```sh + # Arm environment + $ wget https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.25.0/crictl-v1.25.0-linux-arm64.tar.gz + $ tar -zxvf crictl-v1.25.0-linux-arm64.tar.gz -C /usr/local/bin + # x86 environment + $ wget https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.25.0/crictl-v1.25.0-linux-amd64.tar.gz + $ tar -zxvf crictl-v1.25.0-linux-amd64.tar.gz -C /usr/local/bin + ``` + +3. Install the CNI plugins required for CRI to configure the network. + + ```sh + $ mkdir -p /opt/cni/bin && mkdir -p /etc/cni/net.d + + # Arm environment + $ wget https://github.com/containernetworking/plugins/releases/download/v1.3.0/cni-plugins-linux-arm64-v1.3.0.tgz + $ tar -zxvf cni-plugins-linux-arm64-v1.3.0.tgz -C /opt/cni/bin/ + # x86 environment + $ wget https://github.com/containernetworking/plugins/releases/download/v1.3.0/cni-plugins-linux-amd64-v1.3.0.tgz + $ tar -zxvf cni-plugins-linux-amd64-v1.3.0.tgz -C /opt/cni/bin/ + ``` + +## Configuration + +### Configuring iSulad + +Modify the iSulad configuration file **/etc/isulad/daemon.json** so that iSulad can invoke the container runtime of the Kuasar VMM type. Add the following information: + +```sh +$ cat /etc/isulad/daemon.json +... + "cri-sandboxers": { + "vmm": { + "name": "vmm", + "address": "/run/vmm-sandboxer.sock" + } + }, + "cri-runtimes": { + "vmm": "io.containerd.vmm.v1" + }, +... +``` + +Restart iSulad. + +```sh +systemctl restart isulad +``` + +### crictl Configuration + +Modify the crictl configuration file **/etc/crictl.yaml** to connect to iSulad. + +```sh +$ cat /etc/crictl.yaml +runtime-endpoint: unix:///var/run/isulad.sock +image-endpoint: unix:///var/run/isulad.sock +timeout: 10 +``` + +### Kuasar configuration + +Modify the configuration file to connect Kuasar to StratoVirt. (You can use the default configuration. For details about the fields in the configuration file, see [Appendix](./kuasar-appendix.md).) + +```sh +$ cat /var/lib/kuasar/config_stratovirt.toml +[sandbox] +log_level = "info" + +[hypervisor] +path = "/usr/bin/stratovirt" +machine_type = "virt,mem-share=on" +kernel_path = "/var/lib/kuasar/vmlinux.bin" +image_path = "" +initrd_path = "/var/lib/kuasar/kuasar.initrd" +kernel_params = "task.log_level=debug task.sharefs_type=virtiofs" +vcpus = 1 +memory_in_mb = 1024 +block_device_driver = "virtio-blk" +debug = true +enable_mem_prealloc = false + +[hypervisor.virtiofsd_conf] +path = "/usr/bin/vhost_user_fs" +``` + +Start the kuasar-vmm service. + +```sh +systemctl start kuasar-vmm +``` + +Check whether the service is running. + +```sh +$ systemctl status kuasar-vmm +● kuasar-vmm.service - Kuasar microVM type sandboxer daemon process + Loaded: loaded (/usr/lib/systemd/system/kuasar-vmm.service; disabled; vendor preset: disabled) + Active: active (running) since Sat 2023-08-26 14:57:08 CST; 1h 25min ago + Main PID: 1000445 (vmm-sandboxer) + Tasks: 99 (limit: 814372) + Memory: 226.4M + CGroup: /system.slice/kuasar-vmm.service + └─ 1000445 /usr/local/bin/vmm-sandboxer --listen /run/vmm-sandboxer.sock --dir /run/kuasar-vmm +``` diff --git a/docs/en/cloud/container_runtime/kuasar/kuasar-usage.md b/docs/en/cloud/container_runtime/kuasar/kuasar-usage.md new file mode 100644 index 0000000000000000000000000000000000000000..e9d186f696e31e43e4d950a10a7b3b78182276b3 --- /dev/null +++ b/docs/en/cloud/container_runtime/kuasar/kuasar-usage.md @@ -0,0 +1,92 @@ +# Usage Instructions + +Start a Kuasar sandbox. + +1. Ensure that Kuasar and related components have been correctly installed and configured. + +2. Prepare the service container image. Assume that the container image is **busybox**. Use the iSula container engine to download the container image. + + ```sh + isula pull busybox + ``` + +3. Prepare the YAML files for the pod and container. The file examples are as follows: + + ```sh + $ cat podsandbox.yaml + metadata: + name: busybox-sandbox + namespace: default + uid: hdishd83djaidwnduwk28bcsc + log_directory: /tmp + linux: + namespaces: + options: {} + + $ cat pod-container.yaml + metadata: + name: busybox + image: + image: docker.io/library/busybox:latest + command: + - top + log_path: busybox.log + ``` + +4. Start a pod. + + ```sh + $ crictl runp --runtime=vmm podsandbox.yaml + 5cbcf744949d8500e7159d6bd1e3894211f475549c0be15d9c60d3c502c7ede3 + ``` + + Check the pod list. The pod is in the **Ready** state. + + ```sh + $ crictl pods + POD ID CREATED STATE NAME NAMESPACE ATTEMPT + 5cbcf744949d8 About a minute ago Ready busybox-sandbox default 1 + ``` + +5. Create a service container in the pod. + + ```sh + $ crictl create 5cbcf744949d8500e7159d6bd1e3894211f475549c0be15d9c60d3c502c7ede3 pod-container.yaml podsandbox.yaml + c11df540f913e57d1e28372334c028fd6550a2ba73208a3991fbcdb421804a50 + ``` + + View the container list. The container is in the **Created** state. + + ```sh + $ crictl ps -a + CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID + c11df540f913e docker.io/library/busybox:latest 15 seconds ago Created busybox 0 5cbcf744949d + ``` + +6. Start the service container. + + ```sh + crictl start c11df540f913e57d1e28372334c028fd6550a2ba73208a3991fbcdb421804a50 + ``` + + Check the container list. The container is in the **Running** state. + + ```sh + $ crictl ps + CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID + c11df540f913e docker.io/library/busybox:latest 2 minutes ago Running busybox 0 5cbcf744949d8 + ``` + + > [!NOTE]NOTE + > You can also run a `crictl run` command to start a pod with a service container. + + ```sh + crictl run -r vmm --no-pull container-config.yaml podsandbox-config.yaml + ``` + +7. Stop and delete the container and the pod. + + ```sh + crictl rm -f c11df540f913e + crictl rmp -f 5cbcf744949d8 + ``` diff --git a/docs/en/cloud/container_runtime/kuasar/overview.md b/docs/en/cloud/container_runtime/kuasar/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..afa460db81425f23562d9ea7a6c1e8918b78f4a7 --- /dev/null +++ b/docs/en/cloud/container_runtime/kuasar/overview.md @@ -0,0 +1,12 @@ +# Kuasar Multi-Sandbox Container Runtime + +## Overview + +Kuasar is a container runtime that supports unified management of multiple types of sandboxes. It supports multiple mainstream sandbox isolation technologies, including the kernel-based native container sandbox, lightweight virtualization technology-based microVM sandbox, application kernel sandbox based on process-level virtualization, and the emerging WebAssembly sandbox. +Based on the Kuasar unified container runtime combined with the iSulad container engine and StratoVirt virtualization engine, openEuler builds lightweight full-stack self-developed secure containers for cloud native scenarios, delivering key competitiveness of ultra-low overhead and ultra-fast startup. + +**Figure 1** Kuasar architecture +![](./figures/kuasar_arch.png) + +> [!NOTE]NOTE +> The root permission is required for installing and using Kuasar. diff --git a/docs/en/cloud/container_runtime/kuasar/public_sys-resources/icon-note.gif b/docs/en/cloud/container_runtime/kuasar/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/cloud/container_runtime/kuasar/public_sys-resources/icon-note.gif differ diff --git a/docs/en/cloud/hybrid_deployment/oncn-bwm/_toc.yaml b/docs/en/cloud/hybrid_deployment/oncn-bwm/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..073b677a33016dc9c65431946cc03481723e71b2 --- /dev/null +++ b/docs/en/cloud/hybrid_deployment/oncn-bwm/_toc.yaml @@ -0,0 +1,6 @@ +label: oncn-bwm User Guide +isManual: true +description: Bandwidth management solution for pods in hybrid service environments. +sections: + - label: Overview + href: ./overview.md diff --git a/docs/en/cloud/hybrid_deployment/oncn-bwm/overview.md b/docs/en/cloud/hybrid_deployment/oncn-bwm/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..e362f7c0934df0ff6c530f7fd8d8630e01adbe00 --- /dev/null +++ b/docs/en/cloud/hybrid_deployment/oncn-bwm/overview.md @@ -0,0 +1,240 @@ +# oncn-bwm User Guide + +## Introduction + +With the rapid development of technologies such as cloud computing, big data, artificial intelligence, 5G, and the Internet of Things (IoT), data center construction becomes more and more important. However, the server resource utilization of the data center is very low, resulting in a huge waste of resources. To improve the utilization of server resources, oncn-bwm emerges. + +oncn-bwm is a pod bandwidth management tool applicable to hybrid deployment of offline services. It properly schedules network resources for nodes based on QoS levels to ensure online service experience and greatly improve the overall network bandwidth utilization of nodes. + +The oncn-bwm tool supports the following functions: + +- Enabling/Disabling/Querying pod bandwidth management +- Setting the pod network priority +- Setting the offline service bandwidth range and online service waterline +- Querying internal statistics + +## Installation + +### Environmental Requirements + +- Operating system: openEuler-24.03-LTS with the Yum repository of openEuler-24.03-LTS + +### Installation Procedure + +Run the following command: + +```shell +yum install oncn-bwm +``` + +## How to Use + +The oncn-bwm tool provides the `bwmcli` command line tool to enable pod bandwidth management or perform related configurations. The overall format of the `bwmcli` command is as follows: + +**bwmcli** \< option(s) > + +> Note: +> +> The root permission is required for running the `bwmcli` command. +> +> Pod bandwidth management is supported only in the outbound direction of a node (packets are sent from the node to other nodes). +> +> Pod bandwidth management cannot be enabled for NICs for which tc qdisc rules have been configured. +> +> Upgrading the oncn-bwm package does not affect the enabling status before the upgrade. Uninstalling the oncn-bwm package disables pod bandwidth management for all NICs. + +### Command Interfaces + +#### Pod Bandwidth Management + +##### Commands and Functions + +| Command Format | Function | +| --------------------------- | ------------------------------------------------------------ | +| **bwmcli -e** \ | Enables pod bandwidth management for a specified NIC.| +| **bwmcli -d** \ | Disables pod bandwidth management for a specified NIC.| +| **bwmcli -p devs** | Queries pod bandwidth management of all NICs on a node.| + +> Note: +> +> - If no NIC name is specified, the preceding commands take effect for all NICs on a node. +> +> - Enable pod bandwidth management before running other `bwmcli` commands. + +##### Examples + +- Enable pod bandwidth management for NICs eth0 and eth1. + + ```shell + # bwmcli -e eth0 -e eth1 + enable eth0 success + enable eth1 success + ``` + +- Disable pod bandwidth management for NICs eth0 and eth1. + + ```shell + # bwmcli -d eth0 -d eth1 + disable eth0 success + disable eth1 success + ``` + +- Query pod bandwidth management of all NICs on a node. + + ```shell + # bwmcli -p devs + eth0 : enabled + eth1 : disabled + eth2 : disabled + docker0 : disabled + lo : disabled + ``` + +#### Pod Network Priority + +##### Commands and Functions + +| Command Format | Function | +| ------------------------------------------------------------ | ------------------------------------------------------------ | +| **bwmcli -s** *path* \ | Sets the network priority of a pod. *path* indicates the cgroup path corresponding to the pod, and *prio* indicates the priority. The value of *path* can be a relative path or an absolute path. The default value of *prio* is **0**. The optional values are **0** and **-1**. The value **0** indicates online services, and the value **-1** indicates offline services.| +| **bwmcli -p** *path* | Queries the network priority of a pod. | + +> Note: +> +> Online and offline network priorities are supported. The oncn-bwm tool controls the bandwidth of pods in real time based on the network priority. The specific policy is as follows: For online pods, the bandwidth is not limited. For offline pods, the bandwidth is limited within the offline bandwidth range. + +##### Examples + +- Set the priority of the pod whose cgroup path is **/sys/fs/cgroup/net_cls/test_online** to **0**. + + ```shell + # bwmcli -s /sys/fs/cgroup/net_cls/test_online 0 + set prio success + ``` + +- Query the priority of the pod whose cgroup path is **/sys/fs/cgroup/net_cls/test_online**. + + ```shell + # bwmcli -p /sys/fs/cgroup/net_cls/test_online + 0 + ``` + +#### Offline Service Bandwidth Range + +| Command Format | Function | +| ---------------------------------- | ------------------------------------------------------------ | +| **bwmcli -s bandwidth** \ | Sets the offline bandwidth for a host or VM. **low** indicates the minimum bandwidth, and **high** indicates the maximum bandwidth. The unit is KB, MB, or GB, and the value range is \[1 MB, 9999 GB].| +| **bwmcli -p bandwidth** | Queries the offline bandwidth of a host or VM. | + +> Note: +> +> - All NICs with pod bandwidth management enabled on a host are considered as a whole, that is, the configured online service waterline and offline service bandwidth range are shared. +> +> - The pod bandwidth configured using `bwmcli` takes effect for all offline services on a node. The total bandwidth of all offline services cannot exceed the bandwidth range configured for the offline services. There is no network bandwidth limit for online services. +> +> - The offline service bandwidth range and online service waterline are used together to limit the offline service bandwidth. When the online service bandwidth is lower than the configured waterline, the offline services can use the configured maximum bandwidth. When the online service bandwidth is higher than the configured waterline, the offline services can use the configured minimum bandwidth. + +##### Examples + +- Set the offline bandwidth to 30 Mbit/s to 100 Mbit/s. + + ```shell + # bwmcli -s bandwidth 30mb,100mb + set bandwidth success + ``` + +- Query the offline bandwidth range. + + ```shell + # bwmcli -p bandwidth + bandwidth is 31457280(B),104857600(B) + ``` + +#### Online Service Waterline + +##### Commands and Functions + +| Command Format | Function | +| ---------------------------------------------- | ------------------------------------------------------------ | +| **bwmcli -s waterline** \ | Sets the online service waterline for a host or VM. *val* indicates the waterline value. The unit is KB, MB, or GB, and the value range is [20 MB, 9999 GB].| +| **bwmcli -p waterline** | Queries the online service waterline of a host or VM. | + +> Note: +> +> - When the total bandwidth of all online services on a host is higher than the waterline, the bandwidth that can be used by offline services is limited. When the total bandwidth of all online services on a host is lower than the waterline, the bandwidth that can be used by offline services is increased. +> - The system determines whether the total bandwidth of online services exceeds or is lower than the configured waterline every 10 ms. Then the system determines the bandwidth limit for offline services based on whether the online bandwidth collected within each 10 ms is higher than the waterline. + +##### Examples + +- Set the online service waterline to 20 MB. + + ```shell + # bwmcli -s waterline 20mb + set waterline success + ``` + +- Query the online service waterline. + + ```shell + # bwmcli -p waterline + waterline is 20971520(B) + ``` + +#### Statistics + +##### Commands and Functions + +| Command Format | Function | +| ------------------- | ------------------ | +| **bwmcli -p stats** | Queries internal statistics.| + +> Note: +> +> - **offline_target_bandwidth**: target bandwidth for offline services. +> +> - **online_pkts**: total number of online service packets after pod bandwidth management is enabled. +> +> - **offline_pkts**: total number of offline service packets after pod bandwidth management is enabled. +> +> - **online_rate**: current online service rate. +> +> - **offline_rate**: current offline service rate. + +##### Examples + +Query internal statistics. + +```shell +# bwmcli -p stats +offline_target_bandwidth: 2097152 +online_pkts: 2949775 +offline_pkts: 0 +online_rate: 602 +offline_rate: 0 +``` + +### Typical Use Case + +To configure pod bandwidth management on a node, perform the following steps: + +```shell +bwmcli -p devs #Query the pod bandwidth management status of the NICs in the system. +bwmcli -e eth0 # Enable pod bandwidth management for the eth0 NIC. +bwmcli -s /sys/fs/cgroup/net_cls/online 0 # Set the network priority of the online service pod to 0 +bwmcli -s /sys/fs/cgroup/net_cls/offline -1 # Set the network priority of the offline service pod to -1. +bwmcli -s bandwidth 20mb,1gb # Set the bandwidth range for offline services. +bwmcli -s waterline 30mb # Set the waterline for online services. +``` + +### Constraints + +1. Only the **root** user is allowed to run the bwmcli command. +2. Currently, this feature supports only two network QoS priorities: offline and online. +3. If the tc qdisc rules have been configured for a NIC, the network QoS function will fail to be enabled for the NIC. +4. After a NIC is removed and then inserted, the original QoS rules will be lost. In this case, you need to manually reconfigure the network QoS function. +5. When you run one command to enable or disable multiple NICs at the same time, if any NIC fails to be operated, operations on subsequent NICs will be stopped. +6. When SELinux is enabled in the environment, if the SELinux policy is not configured for the bwmcli program, some commands (such as setting or querying the waterline, bandwidth, and priority) may fail. You can confirm the failure in SELinux logs. To solve this problem, disable SELinux or configure the SELinux policy for the bwmcli program. +7. Upgrading the software package does not change the enabling status before the upgrade. Uninstalling the software package disables the function for all devices. +8. The NIC name can contain only digits, letters, hyphens (-), and underscores (_). NICs whose names contain other characters cannot be identified. +9. In actual scenarios, bandwidth limiting may cause protocol stack memory overstock. In this case, backpressure depends on transport-layer protocols. For protocols that do not have backpressure mechanisms, such as UDP, packet loss, ENOBUFS, and rate limiting deviation may occur. +10. After using bwmcli to enable the network QoS function of a certain network card, the tc command cannot be used to modify the tc rules of the network card. Otherwise, it may affect the network QoS function of the network card, leading to abnormal functionality. diff --git a/docs/en/cloud/hybrid_deployment/rubik/_toc.yaml b/docs/en/cloud/hybrid_deployment/rubik/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..3bb729eb79d054a67e3acbcaf7c1aa8fefbcdf71 --- /dev/null +++ b/docs/en/cloud/hybrid_deployment/rubik/_toc.yaml @@ -0,0 +1,12 @@ +label: Rubik User Guide +isManual: true +description: QoS-based resource allocation in hybrid servic deployment scenarios +sections: + - label: Overview + href: ./overview.md + - label: Installation and Deployment + href: ./installation-and-deployment.md + - label: HTTP APIs + href: ./http-apis.md + - label: Example of Isolation for Hybrid Deployed Services + href: ./example-of-isolation-for-hybrid-deployed-services.md diff --git a/docs/en/cloud/hybrid_deployment/rubik/example-of-isolation-for-hybrid-deployed-services.md b/docs/en/cloud/hybrid_deployment/rubik/example-of-isolation-for-hybrid-deployed-services.md new file mode 100644 index 0000000000000000000000000000000000000000..b792ea400a5258a8b0ff1af040791d937ddac4d0 --- /dev/null +++ b/docs/en/cloud/hybrid_deployment/rubik/example-of-isolation-for-hybrid-deployed-services.md @@ -0,0 +1,233 @@ +# Example of Isolation for Hybrid Deployed Services + +## Environment Preparation + +Check whether the kernel supports isolation of hybrid deployed services. + +```bash +# Check whether isolation of hybrid deployed services is enabled in the /boot/config- system configuration. +# If CONFIG_QOS_SCHED=y, the function is enabled. Example: +cat /boot/config-5.10.0-60.18.0.50.oe2203.x86_64 | grep CONFIG_QOS +CONFIG_QOS_SCHED=y +``` + +Install the Docker engine. + +```bash +yum install -y docker-engine +docker version +# The following shows the output of docker version. +Client: + Version: 18.09.0 + EulerVersion: 18.09.0.300 + API version: 1.39 + Go version: go1.17.3 + Git commit: aa1eee8 + Built: Wed Mar 30 05:07:38 2022 + OS/Arch: linux/amd64 + Experimental: false + +Server: + Engine: + Version: 18.09.0 + EulerVersion: 18.09.0.300 + API version: 1.39 (minimum version 1.12) + Go version: go1.17.3 + Git commit: aa1eee8 + Built: Tue Mar 22 00:00:00 2022 + OS/Arch: linux/amd64 + Experimental: false +``` + +## Hybrid Deployed Services + +**Online Service ClickHouse** + +Use the clickhouse-benchmark tool to test the performance and collect statistics on performance metrics such as QPS, P50, P90, and P99. For details, see . + +**Offline Service Stress** + +Stress is a CPU-intensive test tool. You can specify the **--cpu** option to start multiple concurrent CPU-intensive tasks to increase the stress on the system. + +## Usage Instructions + +1) Start a ClickHouse container (online service). + +2) Access the container and run the **clickhouse-benchmark** command. Set the number of concurrent queries to **10**, the number of queries to **10000**, and time limit to **30**. + +3) Start a Stress container (offline service) at the same time and concurrently execute 10 CPU-intensive tasks to increase the stress on the environment. + +4) After the **clickhouse-benchmark** command is executed, a performance test report is generated. + +The **test_demo.sh** script for the isolation test for hybrid deployed services is as follows: + +```bash +#!/bin/bash + +with_offline=${1:-no_offline} +enable_isolation=${2:-no_isolation} +stress_num=${3:-10} +concurrency=10 +timeout=30 +output=/tmp/result.json +online_container= +offline_container= + +exec_sql="echo \"SELECT * FROM system.numbers LIMIT 10000000 OFFSET 10000000\" | clickhouse-benchmark -i 10000 -c $concurrency -t $timeout" + +function prepare() +{ + echo "Launch clickhouse container." + online_container=$(docker run -itd \ + -v /tmp:/tmp:rw \ + --ulimit nofile=262144:262144 \ + -p 34424:34424 \ + yandex/clickhouse-server) + + sleep 3 + echo "Clickhouse container launched." +} + +function clickhouse() +{ + echo "Start clickhouse benchmark test." + docker exec $online_container bash -c "$exec_sql --json $output" + echo "Clickhouse benchmark test done." +} + +function stress() +{ + echo "Launch stress container." + offline_container=$(docker run -itd joedval/stress --cpu $stress_num) + echo "Stress container launched." + + if [ $enable_isolation == "enable_isolation" ]; then + echo "Set stress container qos level to -1." + echo -1 > /sys/fs/cgroup/cpu/docker/$offline_container/cpu.qos_level + fi +} + +function benchmark() +{ + if [ $with_offline == "with_offline" ]; then + stress + sleep 3 + fi + clickhouse + echo "Remove test containers." + docker rm -f $online_container + docker rm -f $offline_container + echo "Finish benchmark test for clickhouse(online) and stress(offline) colocation." + echo "===============================clickhouse benchmark==================================================" + cat $output + echo "===============================clickhouse benchmark==================================================" +} + +prepare +benchmark +``` + +## Test Results + +Independently execute the online service ClickHouse. + +```bash +sh test_demo.sh no_offline no_isolation +``` + +The baseline QoS data (QPS/P50/P90/P99) of the online service is as follows: + +```json +{ +"localhost:9000": { +"statistics": { +"QPS": 1.8853412284364512, +...... +}, +"query_time_percentiles": { +...... +"50": 0.484905256, +"60": 0.519641313, +"70": 0.570876148, +"80": 0.632544937, +"90": 0.728295525, +"95": 0.808700418, +"99": 0.873945121, +...... +} +} +} +``` + +Execute the **test_demo.sh** script to start the offline service Stress and run the test with the isolation function disabled. + +```bash +# **with_offline** indicates that the offline service Stress is enabled. +# **no_isolation** indicates that isolation of hybrid deployed services is disabled. +sh test_demo.sh with_offline no_isolation +``` + +**When isolation of hybrid deployed services is disabled**, the QoS data (QPS/P80/P90/P99) of the ClickHouse service is as follows: + +```json +{ +"localhost:9000": { +"statistics": { +"QPS": 0.9424028693636205, +...... +}, +"query_time_percentiles": { +...... +"50": 0.840476774, +"60": 1.304607373, +"70": 1.393591017, +"80": 1.41277543, +"90": 1.430316688, +"95": 1.457534764, +"99": 1.555646855, +...... +} +} +``` + +Execute the **test_demo.sh** script to start the offline service Stress and run the test with the isolation function enabled. + +```bash +# **with_offline** indicates that the offline service Stress is enabled. +# **enable_isolation** indicates that isolation of hybrid deployed services is enabled. +sh test_demo.sh with_offline enable_isolation +``` + +**When isolation of hybrid deployed services is enabled**, the QoS data (QPS/P80/P90/P99) of the ClickHouse service is as follows: + +```json +{ +"localhost:9000": { +"statistics": { +"QPS": 1.8825798759270718, +...... +}, +"query_time_percentiles": { +...... +"50": 0.485725185, +"60": 0.512629901, +"70": 0.55656488, +"80": 0.636395956, +"90": 0.734695906, +"95": 0.804118275, +"99": 0.887807409, +...... +} +} +} +``` + +The following table lists the test results. + +| Service Deployment Mode | QPS | P50 | P90 | P99 | +| -------------------------------------- | ------------- | ------------- | ------------- | ------------- | +| ClickHouse (baseline) | 1.885 | 0.485 | 0.728 | 0.874 | +| ClickHouse + Stress (isolation disabled)| 0.942 (-50%) | 0.840 (-42%) | 1.430 (-49%) | 1.556 (-44%) | +| ClickHouse + Stress (isolation enabled) | 1.883 (-0.11%) | 0.486 (-0.21%) | 0.735 (-0.96%) | 0.888 (-1.58%) | + +When isolation of hybrid deployed services is disabled, the QPS of ClickHouse decreases from approximately 1.9 to 0.9, the service response delay (P90) increases from approximately 0.7s to 1.4s, and the QoS decreases by about 50%. When isolation of hybrid deployed services is enabled, the QPS and response delay (P50/P90/P99) of ClickHouse decrease by less than 2% compared with the baseline, and the QoS remains unchanged. diff --git a/docs/en/cloud/hybrid_deployment/rubik/figures/icon-note.gif b/docs/en/cloud/hybrid_deployment/rubik/figures/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/cloud/hybrid_deployment/rubik/figures/icon-note.gif differ diff --git a/docs/en/cloud/hybrid_deployment/rubik/http-apis.md b/docs/en/cloud/hybrid_deployment/rubik/http-apis.md new file mode 100644 index 0000000000000000000000000000000000000000..f7f4752051ad6b91035d343afda0d89475546edc --- /dev/null +++ b/docs/en/cloud/hybrid_deployment/rubik/http-apis.md @@ -0,0 +1,67 @@ +# HTTP APIs + +## Overview + +The open APIs of Rubik are all HTTP APIs, including the API for setting or updating the pod priority, API for detecting the Rubik availability, and API for querying the Rubik version. + +## APIs + +### API for Setting or Updating the Pod Priority + +Rubik provides the function of setting or updating the pod priority. External systems can call this API to send pod information. Rubik sets the priority based on the received pod information to isolate resources. The API call format is as follows: + +```bash +HTTP POST /run/rubik/rubik.sock +{ + "Pods": { + "podaaa": { + "CgroupPath": "kubepods/burstable/podaaa", + "QosLevel": 0 + }, + "podbbb": { + "CgroupPath": "kubepods/burstable/podbbb", + "QosLevel": -1 + } + } +} +``` + +In the **Pods** settings, specify information about the pods whose priorities need to be set or updated. At least one pod must be specified for each HTTP request, and **CgroupPath** and **QosLevel** must be specified for each pod. The meanings of **CgroupPath** and **QosLevel** are as follows: + +| Item | Value Type| Value Range| Description | +| ---------- | ---------- | ------------ | ------------------------------------------------------- | +| QosLevel | Integer | 0, -1 | pod priority. The value **0** indicates that the service is an online service, and the value **-1** indicates that the service is an offline service. | +| CgroupPath | String | Relative path | cgroup subpath of the pod (relative path in the cgroup subsystem)| + +The following is an example of calling the API: + +```sh +curl -v -H "Accept: application/json" -H "Content-type: application/json" -X POST --data '{"Pods": {"podaaa": {"CgroupPath": "kubepods/burstable/podaaa","QosLevel": 0},"podbbb": {"CgroupPath": "kubepods/burstable/podbbb","QosLevel": -1}}}' --unix-socket /run/rubik/rubik.sock http://localhost/ +``` + +### API for Detecting Availability + +As an HTTP service, Rubik provides an API for detecting whether it is running. + +API format: HTTP/GET /ping + +The following is an example of calling the API: + +```sh +curl -XGET --unix-socket /run/rubik/rubik.sock http://localhost/ping +``` + +If **ok** is returned, the Rubik service is running. + +### API for Querying Version Information + +Rubik allows you to query the Rubik version number through an HTTP request. + +API format: HTTP/GET /version + +The following is an example of calling the API: + +```sh +curl -XGET --unix-socket /run/rubik/rubik.sock http://localhost/version +{"Version":"0.0.1","Release":"1","Commit":"29910e6","BuildTime":"2021-05-12"} +``` diff --git a/docs/en/cloud/hybrid_deployment/rubik/installation-and-deployment.md b/docs/en/cloud/hybrid_deployment/rubik/installation-and-deployment.md new file mode 100644 index 0000000000000000000000000000000000000000..cf92a03de8937869903778604bbf33e4cc7462a5 --- /dev/null +++ b/docs/en/cloud/hybrid_deployment/rubik/installation-and-deployment.md @@ -0,0 +1,198 @@ +# Installation and Deployment + +## Overview + +This chapter describes how to install and deploy the Rubik component. + +## Software and Hardware Requirements + +### Hardware + +* Architecture: x86 or AArch64 +* Drive: 1 GB or more +* Memory: 100 MB or more + +### Software + +* OS: openEuler 22.03-LTS +* Kernel: openEuler 22.03-LTS kernel + +### Environment Preparation + +* Install the openEuler OS. For details, see the [_openEuler Installation Guide_](../../../server/installation_upgrade/installation/installation.md). +* Install and deploy Kubernetes. For details, see the [_Kubernetes Cluster Deployment Guide_](../../cluster_deployment/kubernetes/_menu.md). +* Install the Docker or iSulad container engine. If the iSulad container engine is used, you need to install the isula-build container image building tool. + +## Installing Rubik + +Rubik is deployed on each Kubernetes node as a DaemonSet. Therefore, you need to perform the following steps to install the Rubik RPM package on each node. + +1. Configure the Yum repositories openEuler 22.03-LTS and openEuler 22.03-LTS:EPOL (the Rubik component is available only in the EPOL repository). + + ```conf + # openEuler 22.03-LTS official repository + name=openEuler22.03 + baseurl=https://repo.openeuler.org/openEuler-22.03-LTS/everything/$basearch/ + enabled=1 + gpgcheck=1 + gpgkey=https://repo.openeuler.org/openEuler-22.03-LTS/everything/$basearch/RPM-GPG-KEY-openEuler + ``` + + ```conf + # openEuler 22.03-LTS:EPOL official repository + name=Epol + baseurl=https://repo.openeuler.org/openEuler-22.03-LTS/EPOL/$basearch/ + enabled=1 + gpgcheck=0 + ``` + +2. Install Rubik with **root** permissions. + + ```shell + sudo yum install -y rubik + ``` + +> ![](./figures/icon-note.gif)**Note**: +> +> Files related to Rubik are installed in the **/var/lib/rubik** directory. + +## Deploying Rubik + +Rubik runs as a container in a Kubernetes cluster in hybrid deployment scenarios. It is used to isolate and restrict resources for services with different priorities to prevent offline services from interfering with online services, improving the overall resource utilization and ensuring the quality of online services. Currently, Rubik supports isolation and restriction of CPU and memory resources, and must be used together with the openEuler 22.03-LTS kernel. To enable or disable the memory priority feature (that is, memory tiering for services with different priorities), you need to set the value in the **/proc/sys/vm/memcg_qos_enable** file. The value can be **0** or **1**. The default value **0** indicates that the feature is disabled, and the value **1** indicates that the feature is enabled. + +```bash +sudo echo 1 > /proc/sys/vm/memcg_qos_enable +``` + +### Deploying Rubik DaemonSet + +1. Use the Docker or isula-build engine to build Rubik images. Because Rubik is deployed as a DaemonSet, each node requires a Rubik image. After building an image on a node, use the **docker save** and **docker load** commands to load the Rubik image to each node of Kubernetes. Alternatively, build a Rubik image on each node. The following uses isula-build as an example. The command is as follows: + + ```sh + isula-build ctr-img build -f /var/lib/rubik/Dockerfile --tag rubik:0.1.0 . + ``` + +2. On the Kubernetes master node, change the Rubik image name in the **/var/lib/rubik/rubik-daemonset.yaml** file to the name of the image built in the previous step. + + ```yaml + ... + containers: + - name: rubik-agent + image: rubik:0.1.0 # The image name must be the same as the Rubik image name built in the previous step. + imagePullPolicy: IfNotPresent + ... + ``` + +3. On the Kubernetes master node, run the **kubectl** command to deploy the Rubik DaemonSet so that Rubik will be automatically deployed on all Kubernetes nodes. + + ```sh + kubectl apply -f /var/lib/rubik/rubik-daemonset.yaml + ``` + +4. Run the **kubectl get pods -A** command to check whether Rubik has been deployed on each node in the cluster. (The number of rubik-agents is the same as the number of nodes and all rubik-agents are in the Running status.) + + ```sh + $ kubectl get pods -A + NAMESPACE NAME READY STATUS RESTARTS AGE + ... + kube-system rubik-agent-76ft6 1/1 Running 0 4s + ... + ``` + +## Common Configuration Description + +The Rubik deployed using the preceding method is started with the default configurations. You can modify the Rubik configurations as required by modifying the **config.json** section in the **rubik-daemonset.yaml** file and then redeploy the Rubik DaemonSet. + +This section describes common configurations in **config.json**. + +### Configuration Item Description + +```yaml +# The configuration items are in the config.json section of the rubik-daemonset.yaml file. +{ + "autoConfig": true, + "autoCheck": false, + "logDriver": "stdio", + "logDir": "/var/log/rubik", + "logSize": 1024, + "logLevel": "info", + "cgroupRoot": "/sys/fs/cgroup" +} +``` + +| Item | Value Type| Value Range | Description | +| ---------- | ---------- | ------------------ | ------------------------------------------------------------ | +| autoConfig | Boolean | **true** or **false** | **true**: enables automatic pod awareness.
**false**: disables automatic pod awareness.| +| autoCheck | Boolean | **true** or **false** | **true**: enables pod priority check.
**false**: disables pod priority check.| +| logDriver | String | **stdio** or **file** | **stdio**: prints logs to the standard output. The scheduling platform collects and dumps logs.
**file**: prints files to the log directory specified by **logDir**.| +| logDir | String | Absolute path | Directory for storing logs. | +| logSize | Integer | \[10,1048576] | Total size of logs, in MB. If the total size of logs reaches the upper limit, the earliest logs will be discarded.| +| logLevel | String | **error**, **info**, or **debug**| Log level. | +| cgroupRoot | String | Absolute path | cgroup mount point. | + +### Automatic Configuration of Pod Priorities + +If **autoConfig** is set to **true** in **config.json** to enable automatic pod awareness, you only need to specify the priority using annotations in the YAML file when deploying the service pods. After being deployed successfully, Rubik automatically detects the creation and update of the pods on the current node, and sets the pod priorities based on the configured priorities. + +### Pod Priority Configuration Depending on kubelet + +Automatic pod priority configuration depends on the pod creation event notifications from the API server, which have a certain delay. The pod priority cannot be configured before the process is started. As a result, the service performance may fluctuate. To avoid this problem, you can disable the automatic priority configuration option and modify the kubelet source code, so that pod priorities can be configured using Rubik HTTP APIs after the cgroup of each container is created and before each container process is started. For details about how to use the HTTP APIs, see [HTTP APIs](./http-apis.md). + +### Automatic Verification of Pod Priorities + +Rubik supports consistency check on the pod QoS priority configurations of the current node during startup. It checks whether the configuration in the Kubernetes cluster is consistent with the pod priority configuration of Rubik. This function is disabled by default. You can enable or disable it using the **autoCheck** option. If this function is enabled, Rubik automatically verifies and corrects the pod priority configuration of the current node when it is started or restarted. + +## Configuring Rubik for Online and Offline Services + +After Rubik is successfully deployed, you can modify the YAML file of a service to specify the service type based on the following configuration example. Then Rubik can configure the priority of the service after it is deployed to isolate resources. + +The following is an example of deploying an online Nginx service: + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: nginx + namespace: qosexample + annotations: + volcano.sh/preemptable: "false" # If volcano.sh/preemptable is set to true, the service is an offline service. If it is set to false, the service is an online service. The default value is false. +spec: + containers: + - name: nginx + image: nginx + resources: + limits: + memory: "200Mi" + cpu: "1" + requests: + memory: "200Mi" + cpu: "1" +``` + +## Restrictions + +* The maximum number of concurrent HTTP requests that Rubik can receive is 1,000 QPS. If the number of concurrent HTTP requests exceeds the upper limit, an error is reported. + +* The maximum number of pods in a single request received by Rubik is 100. If the number of pods exceeds the upper limit, an error is reported. + +* Only one set of Rubik instances can be deployed on each Kubernetes node. Multiple sets of Rubik instances may conflict with each other. + +* Rubik does not provide port access and can communicate only through sockets. + +* Rubik accepts only valid HTTP request paths and network protocols: (POST), (GET), and (GET). For details about the functions of HTTP requests, see HTTP APIs(./http-apis.md). + +* Rubik drive requirement: 1 GB or more. + +* Rubik memory requirement: 100 MB or more. + +* Services cannot be switched from a low priority (offline services) to a high priority (online services). For example, if service A is set to an offline service and then to an online service, Rubik reports an error. + +* When directories are mounted to a Rubik container, the minimum permission on the Rubik local socket directory **/run/Rubik** is **700** on the service side. + +* When the Rubik service is available, the timeout interval of a single request is 120s. If the Rubik process enters the T (stopped or being traced) or D (uninterruptible sleep) state, the service becomes unavailable. In this case, the Rubik service does not respond to any request. To avoid this problem, set the timeout interval on the client to avoid infinite waiting. + +* If hybrid deployment is used, the original CPU share function of cgroup has the following restrictions: + + If both online and offline tasks are running on the CPU, the CPU share configuration of offline tasks does not take effect. + + If the current CPU has only online or offline tasks, the CPU share configuration takes effect. diff --git a/docs/en/cloud/hybrid_deployment/rubik/overview.md b/docs/en/cloud/hybrid_deployment/rubik/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..c9d5ca34d91708f2ac023d7c282aa416c1d413e8 --- /dev/null +++ b/docs/en/cloud/hybrid_deployment/rubik/overview.md @@ -0,0 +1,17 @@ +# Rubik User Guide + +## Overview + +Low server resource utilization has always been a recognized challenge in the industry. With the development of cloud native technologies, hybrid deployment of online (high-priority) and offline (low-priority) services becomes an effective means to improve resource utilization. + +In hybrid service deployment scenarios, Rubik can properly schedule resources based on Quality if Service (QoS) levels to greatly improve resource utilization while ensuring the quality of online services. + +Rubik supports the following features: + +- Pod CPU priority configuration +- Pod memory priority configuration + +This document is intended for community developers, open source enthusiasts, and partners who use the openEuler system and want to learn and use Rubik. Users must: + +- Know basic Linux operations. +- Be familiar with basic operations of Kubernetes and Docker/iSulad. diff --git a/docs/en/cloud/image_builder/isula-build/_toc.yaml b/docs/en/cloud/image_builder/isula-build/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..fa0fccea663fb4a6ad96b42dc9ec1ed987a0e8c5 --- /dev/null +++ b/docs/en/cloud/image_builder/isula-build/_toc.yaml @@ -0,0 +1,12 @@ +label: Container Image Building +isManual: true +description: Quick contaienr image creation using Dockerfiles +sections: + - label: Overview + href: ./overview.md + - label: User Guide + href: ./isula-build.md + - label: Common Issues and Solutions + href: ./isula-build-faqs.md + - label: Appendix + href: ./isula-build-appendix.md diff --git a/docs/en/cloud/image_builder/isula-build/figures/isula-build_arch.png b/docs/en/cloud/image_builder/isula-build/figures/isula-build_arch.png new file mode 100644 index 0000000000000000000000000000000000000000..911a9ae6f46988586ab49f15de282948f5470c37 Binary files /dev/null and b/docs/en/cloud/image_builder/isula-build/figures/isula-build_arch.png differ diff --git a/docs/en/cloud/image_builder/isula-build/isula-build-appendix.md b/docs/en/cloud/image_builder/isula-build/isula-build-appendix.md new file mode 100644 index 0000000000000000000000000000000000000000..b6cb4d8f323e6d3373b7282215a1e979b99509d9 --- /dev/null +++ b/docs/en/cloud/image_builder/isula-build/isula-build-appendix.md @@ -0,0 +1,91 @@ +# Appendix + +## Command Line Parameters + +**Table 1** Parameters of the `ctr-img build` command + +| **Command** | **Parameter** | **Description** | +| ------------- | -------------- | ------------------------------------------------------------ | +| ctr-img build | --build-arg | String list, which contains variables required during the build. | +| | --build-static | Key value, which is used to build binary equivalence. Currently, the following key values are included: - build-time: string, which indicates that a fixed timestamp is used to build a container image. The timestamp format is YYYY-MM-DD HH-MM-SS. | +| | -f, --filename | String, which indicates the path of the Dockerfiles. If this parameter is not specified, the current path is used. | +| | --format | String, which indicates the image format **oci** or **docker** (**ISULABUILD_CLI_EXPERIMENTAL** needs to be enabled). | +| | --iidfile | String, which indicates the ID of the image output to a local file. | +| | -o, --output | String, which indicates the image export mode and path.| +| | --proxy | Boolean, which inherits the proxy environment variable on the host. The default value is true. | +| | --tag | String, which indicates the tag value of the image that is successfully built. | +| | --cap-add | String list, which contains permissions required by the **RUN** instruction during the build process.| + +**Table 2** Parameters of the `ctr-img load` command + +| **Command** | **Parameter** | **Description** | +| ------------ | ----------- | --------------------------------- | +| ctr-img load | -i, --input | String, path of the local .tar package to be imported.| + +**Table 3** Parameters of the `ctr-img push` command + +| **Command** | **Parameter** | **Description** | +| ------------ | ----------- | --------------------------------- | +| ctr-img push | -f, --format | String, which indicates the pushed image format **oci** or **docker** (**ISULABUILD_CLI_EXPERIMENTAL** needs to be enabled).| + +**Table 4** Parameters of the `ctr-img rm` command + +| **Command** | **Parameter** | **Description** | +| ---------- | ----------- | --------------------------------------------- | +| ctr-img rm | -a, --all | Boolean, which is used to delete all local persistent images. | +| | -p, --prune | Boolean, which is used to delete all images that are stored persistently on the local host and do not have tags. | + +**Table 5** Parameters of the `ctr-img save` command + +| **Command** | **Parameter** | **Description** | +| ------------ | ------------ | ---------------------------------- | +| ctr-img save | -o, --output | String, which indicates the local path for storing the exported images.| +| ctr-img save | -f, --format | String, which indicates the exported image format **oci** or **docker** (**ISULABUILD_CLI_EXPERIMENTAL** needs to be enabled).| + +**Table 6** Parameters of the `login` command + +| **Command** | **Parameter** | **Description** | +| -------- | -------------------- | ------------------------------------------------------- | +| login | -p, --password-stdin | Boolean, which indicates whether to read the password through stdin. or enter the password in interactive mode. | +| | -u, --username | String, which indicates the username for logging in to the image repository.| + +**Table 7** Parameters of the `logout` command + +| **Command** | **Parameter** | **Description** | +| -------- | --------- | ------------------------------------ | +| logout | -a, --all | Boolean, which indicates whether to log out of all logged-in image repositories. | + +**Table 8** Parameters of the `manifest annotate` command + +| **Command** | **Parameter** | **Description** | +| ----------------- | ------------- | ---------------------------- | +| manifest annotate | --arch | Set architecture | +| | --os | Set operating system | +| | --os-features | Set operating system feature | +| | --variant | Set architecture variant | + +## Communication Matrix + +The isula-build component processes communicate with each other through the Unix socket file. No port is used for communication. + +## File and Permission + +- All isula-build operations must be performed by the **root** user. To perform operations as a non-privileged user, you need to configure the `--group` option. + +- The following table lists the file permissions involved in the running of isula-build. + +| **File Path** | **File/Folder Permission** | **Description** | +| ------------------------------------------- | ------------------- | ------------------------------------------------------------ | +| /usr/bin/isula-build | 550 | Binary file of the command line tool. | +| /usr/bin/isula-builder | 550 | Binary file of the isula-builder process. | +| /usr/lib/systemd/system/isula-build.service | 640 | systemd configuration file, which is used to manage the isula-build service. | +| /usr/isula-build | 650 | Root directory of the isula-builder configuration file. | +| /etc/isula-build/configuration.toml | 600 | General isula-builder configuration file, including the settings of the isula-builder log level, persistency directory, runtime directory, and OCI runtime. | +| /etc/isula-build/policy.json | 600 | Syntax file of the signature verification policy file. | +| /etc/isula-build/registries.toml | 600 | Configuration file of each image repository, including the available image repository list and image repository blacklist. | +| /etc/isula-build/storage.toml | 600 | Configuration file of the local persistent storage, including the configuration of the used storage driver. | +| /etc/isula-build/isula-build.pub | 400 | Asymmetric encryption public key file. | +| /var/run/isula_build.sock | 660 | Local socket of isula-builder. | +| /var/lib/isula-build | 700 | Local persistency directory. | +| /var/run/isula-build | 700 | Local runtime directory. | +| /var/lib/isula-build/tmp/\[build_id\]/isula-build-tmp-*.tar | 644 | Local temporary directory for storing the images when they are exported to iSulad. | diff --git a/docs/en/cloud/image_builder/isula-build/isula-build-faqs.md b/docs/en/cloud/image_builder/isula-build/isula-build-faqs.md new file mode 100644 index 0000000000000000000000000000000000000000..3a325a91b6cba76e1c1811cd6f3630a6078b237a --- /dev/null +++ b/docs/en/cloud/image_builder/isula-build/isula-build-faqs.md @@ -0,0 +1,9 @@ +# Common Issues and Solutions + +## Issue 1: isula-build Image Pull Error: Connection Refused + +When pulling an image, isula-build encounters the error: `pinging container registry xx: get xx: dial tcp host:repo: connect: connection refused`. + +This occurs because the image is sourced from an untrusted registry. + +To resolve this, edit the isula-build registry configuration file located at **/etc/isula-build/registries.toml**. Add the untrusted registry to the `[registries.insecure]` section and restart isula-build. diff --git a/docs/en/cloud/image_builder/isula-build/isula-build.md b/docs/en/cloud/image_builder/isula-build/isula-build.md new file mode 100644 index 0000000000000000000000000000000000000000..e1484710612f83712e35ff7e6752ca45f376e7d5 --- /dev/null +++ b/docs/en/cloud/image_builder/isula-build/isula-build.md @@ -0,0 +1,938 @@ +# Installation + +## Preparations + +To ensure that isula-build can be successfully installed, the following software and hardware requirements must be met: + +- Supported architectures: x86_64 and AArch64 +- Supported OS: openEuler +- You have the permissions of the root user. + +### Installing isula-build + +Before using isula-build to build a container image, you need to install the following software packages: + +#### (Recommended) Method 1: Using Yum + +1. Configure the openEuler Yum source. + +2. Log in to the target server as the root user and install isula-build. + + ```shell + sudo yum install -y isula-build + ``` + +#### Method 2: Using the RPM Package + +1. Obtain an **isula-build-*.rpm** installation package from the openEuler Yum source, for example, **isula-build-0.9.6-4.oe1.x86_64.rpm**. + +2. Upload the obtained RPM software package to any directory on the target server, for example, **/home/**. + +3. Log in to the target server as the root user and run the following command to install isula-build: + + ```shell + sudo rpm -ivh /home/isula-build-*.rpm + ``` + +> ![](./public_sys-resources/icon-note.gif) **Note:** +> +> After the installation is complete, you need to manually start the isula-build service. For details about how to start the service, see [Managing the isula-build Service](isula-build-user-guide.md#managing-the-isula-build-service). + +# Configuring and Managing the isula-build Service + +## Configuring the isula-build Service + +After the isula-build software package is installed, the systemd starts the isula-build service based on the default configuration contained in the isula-build software package on the isula-build server. If the default configuration file on the isula-build server cannot meet your requirements, perform the following operations to customize the configuration file: After the default configuration is modified, restart the isula-build server for the new configuration to take effect. For details, see [Managing the isula-build Service](isula-build-user-guide.md#managing-the-isula-build-service). + +Currently, the isula-build server contains the following configuration file: + +- **/etc/isula-build/configuration.toml**: general isula-builder configuration file, which is used to set the isula-builder log level, persistency directory, runtime directory, and OCI runtime. Parameters in the configuration file are described as follows: + +| Configuration Item | Mandatory or Optional | Description | Value | +| --------- | -------- | --------------------------------- | ----------------------------------------------- | +| debug | Optional | Indicates whether to enable the debug log function. | **true**: Enables the debug log function. **false**: Disables the debug log function. | +| loglevel | Optional | Sets the log level. | debug
info
warn
error | +| run_root | Mandatory | Sets the root directory of runtime data. | For example, **/var/run/isula-build/** | +| data_root | Mandatory | Sets the local persistency directory. | For example, **/var/lib/isula-build/** | +| runtime | Optional | Sets the runtime type. Currently, only **runc** is supported. | runc | +| group | Optional | Sets the owner group for the local socket file **isula_build.sock** so that non-privileged users in the group can use isula-build. | isula | +| experimental | Optional | Indicates whether to enable experimental features. | **true**: Enables experimental features. **false**: Disables experimental features. | + +- **/etc/isula-build/storage.toml**: configuration file for local persistent storage, including the configuration of the storage driver in use. + +| Configuration Item | Mandatory or Optional | Description | +| ------ | -------- | ------------------------------ | +| driver | Optional | Storage driver type. Currently, **overlay2** is supported. | + + For more settings, see [containers-storage.conf.5](https://github.com/containers/storage/blob/main/docs/containers-storage.conf.5.md). + +- **/etc/isula-build/registries.toml**: configuration file for each image repository. + +| Configuration Item | Mandatory or Optional | Description | +| ------------------- | -------- | ------------------------------------------------------------ | +| registries.search | Optional | Search domain of the image repository. Only listed image repositories can be found. | +| registries.insecure | Optional | Accessible insecure image repositories. Listed image repositories cannot pass the authentication and are not recommended. | + + For more settings, see [containers-registries.conf.5](https://github.com/containers/image/blob/main/docs/containers-registries.conf.5.md). + +- **/etc/isula-build/policy.json**: image pull/push policy file. Currently, this file cannot be configured. + +> ![](./public_sys-resources/icon-note.gif) **Note:** +> +> - isula-build supports the preceding configuration file with the maximum size of 1 MB. +> - The persistent working directory dataroot cannot be configured on the memory disk, for example, tmpfs. +> - Currently, only overlay2 can be used as the underlying storage driver. +> - Before setting the `--group` option, ensure that the corresponding user group has been created on a local OS and non-privileged users have been added to the group. After isula-builder is restarted, non-privileged users in the group can use the isula-build function. In addition, to ensure permission consistency, the owner group of the isula-build configuration file directory **/etc/isula-build** is set to the group specified by `--group`. + +## Managing the isula-build Service + +Currently, openEuler uses systemd to manage the isula-build service. The isula-build software package contains the systemd service files. After installing the isula-build software package, you can use the systemd tool to start or stop the isula-build service. You can also manually start the isula-builder software. Note that only one isula-builder process can be started on a node at a time. + +> ![](./public_sys-resources/icon-note.gif) **Note:** +> +> Only one isula-builder process can be started on a node at a time. + +### (Recommended) Using systemd for Management + +You can run the following systemd commands to start, stop, and restart the isula-build service: + +- Run the following command to start the isula-build service: + + ```sh + sudo systemctl start isula-build.service + ``` + +- Run the following command to stop the isula-build service: + + ```sh + sudo systemctl stop isula-build.service + ``` + +- Run the following command to restart the isula-build service: + + ```sh + sudo systemctl restart isula-build.service + ``` + +The systemd service file of the isula-build software installation package is stored in the `/usr/lib/systemd/system/isula-build.service` directory. If you need to modify the systemd configuration of the isula-build service, modify the file and run the following command to make the modification take effect. Then restart the isula-build service based on the systemd management command. + +```sh +sudo systemctl daemon-reload +``` + +### Directly Running isula-builder + +You can also run the `isula-builder` command on the server to start the service. The `isula-builder` command can contain flags for service startup. The following flags are supported: + +- `-D, --debug`: whether to enable the debugging mode. +- `--log-level`: log level. The options are **debug**, **info**, **warn**, and **error**. The default value is **info**. +- `--dataroot`: local persistency directory. The default value is **/var/lib/isula-build/**. +- `--runroot`: runtime directory. The default value is **/var/run/isula-build/**. +- `--storage-driver`: underlying storage driver type. +- `--storage-opt`: underlying storage driver configuration. +- `--group`: sets the owner group for the local socket file **isula_build.sock** so that non-privileged users in the group can use isula-build. The default owner group is **isula**. +- `--experimental`: whether to enable experimental features. + +> ![](./public_sys-resources/icon-note.gif) **Note:** +> +> If the command line parameters contain the same configuration items as those in the configuration file, the command line parameters are preferentially used for startup. + +Start the isula-build service. For example, to specify the local persistency directory **/var/lib/isula-build** and disable debugging, run the following command: + +```sh +sudo isula-builder --dataroot "/var/lib/isula-build" --debug=false +``` + +# Usage Guidelines + +## Prerequisites + +isula-build depends on the executable file **runc** to build the **RUN** instruction in the Dockerfile. Therefore, runc must be pre-installed in the running environment of isula-build. The installation method depends on the application scenario. If you do not need to use the complete docker-engine tool chain, you can install only the docker-runc RPM package. + +```sh +sudo yum install -y docker-runc +``` + +If you need to use a complete docker-engine tool chain, install the docker-engine RPM package, which contains the executable file **runc** by default. + +```sh +sudo yum install -y docker-engine +``` + +> ![](./public_sys-resources/icon-note.gif) **Note:** +> +> Ensure the security of OCI runtime (runc) executable files to prevent malicious replacement. + +## Overview + +The isula-build client provides a series of commands for building and managing container images. Currently, the isula-build client provides the following commands: + +- `ctr-img`: manages container images. The `ctr-img` command contains the following subcommands: + - `build`: builds a container image based on the specified Dockerfile. + - `images`: lists local container images. + - `import`: imports a basic container image. + - `load`: imports a cascade image. + - `rm`: deletes a local container image. + - `save`: exports a cascade image to a local disk. + - `tag`: adds a tag to a local container image. + - `pull`: pulls an image to a local host. + - `push`: pushes a local image to a remote repository. +- `info`: displays the running environment and system information of isula-build. +- `login`: logs in to the remote container image repository. +- `logout`: logs out of the remote container image repository. +- `version`: displays the versions of isula-build and isula-builder. +- `manifest` (experimental): manages the manifest list. + +> ![](./public_sys-resources/icon-note.gif) **Note:** +> +> - The `isula-build completion` and `isula-builder completion` commands are used to generate the bash command completion script. These commands are implicitly provided by the command line framework and is not displayed in the help information. +> - isula-build client does not have any configuration file. To use isula-build experimental features, enable the environment variable **ISULABUILD_CLI_EXPERIMENTAL** on the client using the `export ISULABUILD_CLI_EXPERIMENTAL=enabled` command. + +The following describes how to use these commands in detail. + +## ctr-img: Container Image Management + +The isula-build command groups all container image management commands into the `ctr-img` command. The command format is as follows: + +```shell +isula-build ctr-img [command] +``` + +### build: Container Image Build + +The subcommand build of the `ctr-img` command is used to build container images. The command format is as follows: + +```shell +isula-build ctr-img build [flags] +``` + +The `build` command contains the following flags: + +- `--build-arg`: string list containing variables required during the build process. +- `--build-static`: key value, which is used to build binary equivalence. Currently, the following key values are included: + `- build-time`: string indicating that a container image is built at a specified timestamp. The timestamp format is *YYYY-MM-DD HH-MM-SS*. +- `-f, --filename`: string indicating the path of the Dockerfiles. If this parameter is not specified, the current path is used. +- `--format`: string indicating the image format **oci** or **docker** (**ISULABUILD_CLI_EXPERIMENTAL** needs to be enabled). +- `--iidfile`: string indicating a local file to which the ID of the image is output. +- `-o, --output`: string indicating the image export mode and path. +- `--proxy`: boolean, which inherits the proxy environment variable on the host. The default value is **true**. +- `--tag`: string indicating the tag value of the image that is successfully built. +- `--cap-add`: string list containing permissions required by the **RUN** instruction during the build process. + +**The following describes the flags in detail.** + +#### \--build-arg + +Parameters in the Dockerfile are inherited from the commands. The usage is as follows: + +```sh +$ echo "This is bar file" > bar.txt +$ cat Dockerfile_arg +FROM busybox +ARG foo +ADD ${foo}.txt . +RUN cat ${foo}.txt +$ sudo isula-build ctr-img build --build-arg foo=bar -f Dockerfile_arg +STEP 1: FROM busybox +Getting image source signatures +Copying blob sha256:8f52abd3da461b2c0c11fda7a1b53413f1a92320eb96525ddf92c0b5cde781ad +Copying config sha256:e4db68de4ff27c2adfea0c54bbb73a61a42f5b667c326de4d7d5b19ab71c6a3b +Writing manifest to image destination +Storing signatures +STEP 2: ARG foo +STEP 3: ADD ${foo}.txt . +STEP 4: RUN cat ${foo}.txt +This is bar file +Getting image source signatures +Copying blob sha256:6194458b07fcf01f1483d96cd6c34302ffff7f382bb151a6d023c4e80ba3050a +Copying blob sha256:6bb56e4a46f563b20542171b998cb4556af4745efc9516820eabee7a08b7b869 +Copying config sha256:39b62a3342eed40b41a1bcd9cd455d77466550dfa0f0109af7a708c3e895f9a2 +Writing manifest to image destination +Storing signatures +Build success with image id: 39b62a3342eed40b41a1bcd9cd455d77466550dfa0f0109af7a708c3e895f9a2 +``` + +#### \--build-static + +Specifies a static build. That is, when isula-build is used to build a container image, differences between all timestamps and other build factors (such as the container ID and hostname) are eliminated. Finally, a container image that meets the static requirements is built. + +When isula-build is used to build a container image, assume that a fixed timestamp is given to the build subcommand and the following conditions are met: + +- The build environment is consistent before and after the upgrade. +- The Dockerfile is consistent before and after the build. +- The intermediate data generated before and after the build is consistent. +- The build commands are the same. +- The versions of the third-party libraries are the same. + +For container image build, isula-build supports the same Dockerfile. If the build environments are the same, the image content and image ID generated in multiple builds are the same. + +`--build-static` supports the key-value pair option in the *key=value* format. Currently, the following options are supported: + +- build-time: string, which indicates the fixed timestamp for creating a static image. The value is in the format of *YYYY-MM-DD HH-MM-SS*. The timestamp affects the attribute of the file for creating and modifying the time at the diff layer. + + Example: + + ```sh + sudo isula-build ctr-img build -f Dockerfile --build-static='build-time=2020-05-23 10:55:33' . + ``` + + In this way, the container images and image IDs built in the same environment for multiple times are the same. + +#### \--format + +This option can be used when the experiment feature is enabled. The default image format is **oci**. You can specify the image format to build. For example, the following commands are used to build an OCI image and a Docker image, respectively. + + ```sh + export ISULABUILD_CLI_EXPERIMENTAL=enabled; sudo isula-build ctr-img build -f Dockerfile --format oci . + ``` + + ```sh + export ISULABUILD_CLI_EXPERIMENTAL=enabled; sudo isula-build ctr-img build -f Dockerfile --format docker . + ``` + +#### \--iidfile + +Run the following command to output the ID of the built image to a file: + +```shell +isula-build ctr-img build --iidfile filename +``` + +For example, to export the container image ID to the **testfile** file, run the following command: + + ```sh +sudo isula-build ctr-img build -f Dockerfile_arg --iidfile testfile + ``` + + Check the container image ID in the **testfile** file. + + ```sh +$ cat testfile +76cbeed38a8e716e22b68988a76410eaf83327963c3b29ff648296d5cd15ce7b + ``` + +#### \-o, --output + +Currently, `-o` and `--output` support the following formats: + +- `isulad:image:tag`: directly pushes the image that is successfully built to iSulad, for example, `-o isulad:busybox:latest`. The following restrictions apply: + + - isula-build and iSulad must be on the same node. + - The tag must be configured. + - On the isula-build client, you need to temporarily save the successfully built image as **/var/tmp/isula-build-tmp-%v.tar** and then import it to iSulad. Ensure that the **/var/tmp/** directory has sufficient disk space. + +- `docker-daemon:image:tag`: directly pushes the successfully built image to Docker daemon, for example, `-o docker-daemon:busybox:latest`. The following restrictions apply: +- isula-build and Docker must be on the same node. + - The tag must be configured. + +- `docker://registry.example.com/repository:tag`: directly pushes the successfully built image to the remote image repository in Docker image format, for example, `-o docker://localhost:5000/library/busybox:latest`. + +- `docker-archive:/:image:tag`: saves the successfully built image to the local host in Docker image format, for example, `-o docker-archive:/root/image.tar:busybox:latest`. + +When experiment feature is enabled, you can build image in OCI image format with: + +- `oci://registry.example.com/repository:tag`: directly pushes the successfully built image to the remote image repository in OCI image format(OCI image format should be supported by the remote repository), for example, `-o oci://localhost:5000/library/busybox:latest`. + +- `oci-archive:/:image:tag`: saves the successfully built image to the local host in OCI image format, for example, `-o oci-archive:/root/image.tar:busybox:latest`. + +In addition to the flags, the `build` subcommand also supports an argument whose type is string and meaning is context, that is, the context of the Dockerfile build environment. The default value of this parameter is the current path where isula-build is executed. This path affects the path retrieved by the **ADD** and **COPY** instructions of the .dockerignore file and Dockerfile. + +#### \--proxy + +Specifies whether the container started by the **RUN** instruction inherits the proxy-related environment variables **http_proxy**, **https_proxy**, **ftp_proxy**, **no_proxy**, **HTTP_PROXY**, **HTTPS_PROXY**, and **FTP_PROXY**. The default value is **true**. + +When a user configures proxy-related **ARG** or **ENV** in the Dockerfile, the inherited environment variables will be overwritten. + +> ![](./public_sys-resources/icon-note.gif) **Note:** +> +> If the client and daemon are running on different terminals, the environment variables of the terminal where the daemon is running are inherited. + +#### \--tag + +Specifies the tag of the image stored on the local disk after the image is successfully built. + +#### \--cap-add + +Run the following command to add the permission required by the **RUN** instruction during the build process: + +```shell +isula-build ctr-img build --cap-add ${CAP} +``` + +Example: + +```sh +sudo isula-build ctr-img build --cap-add CAP_SYS_ADMIN --cap-add CAP_SYS_PTRACE -f Dockerfile +``` + +> **Note:** +> +> - A maximum of 100 container images can be concurrently built. +> - isula-build supports Dockerfiles with a maximum size of 1 MB. +> - isula-build supports a .dockerignore file with a maximum size of 1 MB. +> - Ensure that only the current user has the read and write permissions on the Dockerfiles to prevent other users from tampering with the files. +> - During the build, the **RUN** instruction starts the container to build in the container. Currently, isula-build supports the host network only. +> - isula-build only supports the tar compression format. +> - isula-build commits once after each image build stage is complete, instead of each time a Dockerfile line is executed. +> - isula-build does not support cache build. +> - isula-build starts the build container only when the **RUN** instruction is built. +> - Currently, the history function of Docker images is not supported. +> - The stage name can start with a digit. +> - The stage name can contain a maximum of 64 characters. +> - isula-build does not support resource restriction on a single Dockerfile build. If resource restriction is required, you can configure a resource limit on isula-builder. +> - Currently, isula-build does not support a remote URL as the data source of the **ADD** instruction in the Dockerfile. +> - The local tar package exported using the **docker-archive** and **oci-archive** types are not compressed, you can manually compress the file as required. + +### image: Viewing Local Persistent Build Images + +You can run the `images` command to view the images in the local persistent storage. + +```sh +$ sudo isula-build ctr-img images +--------------------------------------- ----------- ----------------- ------------------------ ------------ +REPOSITORY TAG IMAGE ID CREATED SIZE +--------------------------------------- ----------- ----------------- ------------------------ ------------ +localhost:5000/library/alpine latest a24bb4013296 2022-01-17 10:02:19 5.85 MB + 39b62a3342ee 2022-01-17 10:01:12 1.45 MB +--------------------------------------- ----------- ----------------- ------------------------ ------------ +``` + +> ![](./public_sys-resources/icon-note.gif) **Note:** +> +> The image size displayed by running the `isula-build ctr-img images` command may be different from that displayed by running the `docker images` command. When calculating the image size, `isula-build` directly calculates the total size of .tar packages at each layer, while `docker` calculates the total size of files by decompressing the .tar packages and traversing the diff directory. Therefore, the statistics are different. + +### import: Importing a Basic Container Image + +A tar file in rootfs form can be imported into isula-build via the `ctr-img import` command. + +The command format is as follows: + +```shell +isula-build ctr-img import [flags] +``` + +Example: + +```sh +$ sudo isula-build ctr-img import busybox.tar mybusybox:latest +Getting image source signatures +Copying blob sha256:7b8667757578df68ec57bfc9fb7754801ec87df7de389a24a26a7bf2ebc04d8d +Copying config sha256:173b3cf612f8e1dc34e78772fcf190559533a3b04743287a32d549e3c7d1c1d1 +Writing manifest to image destination +Storing signatures +Import success with image id: "173b3cf612f8e1dc34e78772fcf190559533a3b04743287a32d549e3c7d1c1d1" +$ sudo isula-build ctr-img images +--------------------------------------- ----------- ----------------- ------------------------ ------------ +REPOSITORY TAG IMAGE ID CREATED SIZE +--------------------------------------- ----------- ----------------- ------------------------ ------------ +mybusybox latest 173b3cf612f8 2022-01-12 16:02:31 1.47 MB +--------------------------------------- ----------- ----------------- ------------------------ ------------ +``` + +> ![](./public_sys-resources/icon-note.gif) **Note** +> +> isula-build supports the import of container basic images with a maximum size of 1 GB. + +### load: Importing Cascade Images + +Cascade images are images that are saved to the local computer by running the `docker save` or `isula-build ctr-img save` command. The compressed image package contains a layer-by-layer image package named **layer.tar**. You can run the `ctr-img load` command to import the image to isula-build. + +The command format is as follows: + +```shell +isula-build ctr-img load [flags] +``` + +Currently, the following flags are supported: + +- `-i, --input`: path of the local .tar package. + +Example: + +```sh +$ sudo isula-build ctr-img load -i ubuntu.tar +Getting image source signatures +Copying blob sha256:cf612f747e0fbcc1674f88712b7bc1cd8b91cf0be8f9e9771235169f139d507c +Copying blob sha256:f934e33a54a60630267df295a5c232ceb15b2938ebb0476364192b1537449093 +Copying blob sha256:943edb549a8300092a714190dfe633341c0ffb483784c4fdfe884b9019f6a0b4 +Copying blob sha256:e7ebc6e16708285bee3917ae12bf8d172ee0d7684a7830751ab9a1c070e7a125 +Copying blob sha256:bf6751561805be7d07d66f6acb2a33e99cf0cc0a20f5fd5d94a3c7f8ae55c2a1 +Copying blob sha256:c1bd37d01c89de343d68867518b1155cb297d8e03942066ecb44ae8f46b608a3 +Copying blob sha256:a84e57b779297b72428fc7308e63d13b4df99140f78565be92fc9dbe03fc6e69 +Copying blob sha256:14dd68f4c7e23d6a2363c2320747ab88986dfd43ba0489d139eeac3ac75323b2 +Copying blob sha256:a2092d776649ea2301f60265f378a02405539a2a68093b2612792cc65d00d161 +Copying blob sha256:879119e879f682c04d0784c9ae7bc6f421e206b95d20b32ce1cb8a49bfdef202 +Copying blob sha256:e615448af51b848ecec00caeaffd1e30e8bf5cffd464747d159f80e346b7a150 +Copying blob sha256:f610bd1e9ac6aa9326d61713d552eeefef47d2bd49fc16140aa9bf3db38c30a4 +Copying blob sha256:bfe0a1336d031bf5ff3ce381e354be7b2bf310574cc0cd1949ad94dda020cd27 +Copying blob sha256:f0f15db85788c1260c6aa8ad225823f45c89700781c4c793361ac5fa58d204c7 +Copying config sha256:c07ddb44daa97e9e8d2d68316b296cc9343ab5f3d2babc5e6e03b80cd580478e +Writing manifest to image destination +Storing signatures +Loaded image as c07ddb44daa97e9e8d2d68316b296cc9343ab5f3d2babc5e6e03b80cd580478e +``` + +> ![](./public_sys-resources/icon-note.gif) **Note:** +> +> - isula-build allows you to import a container image with a maximum size of 50 GB. +> - isula-build automatically recognizes the image format and loads it from the cascade image file. + +### rm: Deleting a Local Persistent Image + +You can run the `rm` command to delete an image from the local persistent storage. The command format is as follows: + +```shell +isula-build ctr-img rm IMAGE [IMAGE...] [FLAGS] +``` + +Currently, the following flags are supported: + +- `-a, --all`: deletes all images stored locally. +- `-p, --prune`: deletes all images that are stored locally and do not have tags. + +Example: + +```sh +$ sudo isula-build ctr-img rm -p +Deleted: sha256:78731c1dde25361f539555edaf8f0b24132085b7cab6ecb90de63d72fa00c01d +Deleted: sha256:eeba1bfe9fca569a894d525ed291bdaef389d28a88c288914c1a9db7261ad12c +``` + +### save: Exporting Cascade Images + +You can run the `save` command to export the cascade images to the local disk. The command format is as follows: + +```shell +isula-build ctr-img save [REPOSITORY:TAG]|imageID -o xx.tar +``` + +Currently, the following flags are supported: + +- `-f, --format`: which indicates the exported image format: **oci** or **docker** (**ISULABUILD_CLI_EXPERIMENTAL** needs to be enabled) +- `-o, --output`: which indicates the local path for storing the exported images. + +The following example shows how to export an image using *image/tag*: + +```sh +$ sudo isula-build ctr-img save busybox:latest -o busybox.tar +Getting image source signatures +Copying blob sha256:50644c29ef5a27c9a40c393a73ece2479de78325cae7d762ef3cdc19bf42dd0a +Copying blob sha256:824082a6864774d5527bda0d3c7ebd5ddc349daadf2aa8f5f305b7a2e439806f +Copying blob sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef +Copying config sha256:21c3e96ac411242a0e876af269c0cbe9d071626bdfb7cc79bfa2ddb9f7a82db6 +Writing manifest to image destination +Storing signatures +Save success with image: busybox:latest +``` + +The following example shows how to export an image using *ImageID*: + +```sh +$ sudo isula-build ctr-img save 21c3e96ac411 -o busybox.tar +Getting image source signatures +Copying blob sha256:50644c29ef5a27c9a40c393a73ece2479de78325cae7d762ef3cdc19bf42dd0a +Copying blob sha256:824082a6864774d5527bda0d3c7ebd5ddc349daadf2aa8f5f305b7a2e439806f +Copying blob sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef +Copying config sha256:21c3e96ac411242a0e876af269c0cbe9d071626bdfb7cc79bfa2ddb9f7a82db6 +Writing manifest to image destination +Storing signatures +Save success with image: 21c3e96ac411 +``` + +The following example shows how to export multiple images to the same tarball: + +```sh +$ sudo isula-build ctr-img save busybox:latest nginx:latest -o all.tar +Getting image source signatures +Copying blob sha256:eb78099fbf7fdc70c65f286f4edc6659fcda510b3d1cfe1caa6452cc671427bf +Copying blob sha256:29f11c413898c5aad8ed89ad5446e89e439e8cfa217cbb404ef2dbd6e1e8d6a5 +Copying blob sha256:af5bd3938f60ece203cd76358d8bde91968e56491daf3030f6415f103de26820 +Copying config sha256:b8efb18f159bd948486f18bd8940b56fd2298b438229f5bd2bcf4cedcf037448 +Writing manifest to image destination +Storing signatures +Getting image source signatures +Copying blob sha256:e2d6930974a28887b15367769d9666116027c411b7e6c4025f7c850df1e45038 +Copying config sha256:a33de3c85292c9e65681c2e19b8298d12087749b71a504a23c576090891eedd6 +Writing manifest to image destination +Storing signatures +Save success with image: [busybox:latest nginx:latest] +``` + +> [!NOTE]NOTE +> +> - Save exports an image in .tar format by default. If necessary, you can save the image and then manually compress it. +> - When exporting an image using image name, specify the entire image name in the *REPOSITORY:TAG* format. + +### tag: Tagging Local Persistent Images + +You can run the `tag` command to add a tag to a local persistent container image. The command format is as follows: + +```shell +isula-build ctr-img tag / busybox:latest +``` + +Example: + +```sh +$ sudo isula-build ctr-img images +--------------------------------------- ----------- ----------------- -------------------------- ------------ +REPOSITORY TAG IMAGE ID CREATED SIZE +--------------------------------------- ----------- ----------------- -------------------------- ------------ +alpine latest a24bb4013296 2020-05-29 21:19:46 5.85 MB +--------------------------------------- ----------- ----------------- -------------------------- ------------ +$ sudo isula-build ctr-img tag a24bb4013296 alpine:v1 +$ sudo isula-build ctr-img images +--------------------------------------- ----------- ----------------- ------------------------ ------------ +REPOSITORY TAG IMAGE ID CREATED SIZE +--------------------------------------- ----------- ----------------- ------------------------ ------------ +alpine latest a24bb4013296 2020-05-29 21:19:46 5.85 MB +alpine v1 a24bb4013296 2020-05-29 21:19:46 5.85 MB +--------------------------------------- ----------- ----------------- ------------------------ ------------ +``` + +### pull: Pulling an Image To a Local Host + +Run the `pull` command to pull an image from a remote image repository to a local host. Command format: + +```shell +isula-build ctr-img pull REPOSITORY[:TAG] +``` + +Example: + +```sh +$ sudo isula-build ctr-img pull example-registry/library/alpine:latest +Getting image source signatures +Copying blob sha256:8f52abd3da461b2c0c11fda7a1b53413f1a92320eb96525ddf92c0b5cde781ad +Copying config sha256:e4db68de4ff27c2adfea0c54bbb73a61a42f5b667c326de4d7d5b19ab71c6a3b +Writing manifest to image destination +Storing signatures +Pull success with image: example-registry/library/alpine:latest +``` + +### push: Pushing a Local Image to a Remote Repository + +Run the `push` command to push a local image to a remote repository. Command format: + +```shell +isula-build ctr-img push REPOSITORY[:TAG] +``` + +Currently, the following flags are supported: + +- `-f, --format`: indicates the pushed image format **oci** or **docker** (**ISULABUILD_CLI_EXPERIMENTAL** needs to be enabled) + +Example: + +```sh +$ sudo isula-build ctr-img push example-registry/library/mybusybox:latest +Getting image source signatures +Copying blob sha256:d2421964bad195c959ba147ad21626ccddc73a4f2638664ad1c07bd9df48a675 +Copying config sha256:f0b02e9d092d905d0d87a8455a1ae3e9bb47b4aa3dc125125ca5cd10d6441c9f +Writing manifest to image destination +Storing signatures +Push success with image: example-registry/library/mybusybox:latest +``` + +> [!NOTE]NOTE +> +> - Before pushing an image, log in to the corresponding image repository. + +## info: Viewing the Operating Environment and System Information + +You can run the `isula-build info` command to view the running environment and system information of isula-build. The command format is as follows: + +```shell + isula-build info [flags] +``` + +The following flags are supported: + +- `-H, --human-readable`: Boolean. The memory information is printed in the common memory format. The value is 1000 power. +- `-V, --verbose`: Boolean. The memory usage is displayed during system running. + +Example: + +```sh +$ sudo isula-build info -H + General: + MemTotal: 7.63 GB + MemFree: 757 MB + SwapTotal: 8.3 GB + SwapFree: 8.25 GB + OCI Runtime: runc + DataRoot: /var/lib/isula-build/ + RunRoot: /var/run/isula-build/ + Builders: 0 + Goroutines: 12 + Store: + Storage Driver: overlay + Backing Filesystem: extfs + Registry: + Search Registries: + oepkgs.net + Insecure Registries: + localhost:5000 + oepkgs.net + Runtime: + MemSys: 68.4 MB + HeapSys: 63.3 MB + HeapAlloc: 7.41 MB + MemHeapInUse: 8.98 MB + MemHeapIdle: 54.4 MB + MemHeapReleased: 52.1 MB +``` + +## login: Logging In to the Remote Image Repository + +You can run the `login` command to log in to the remote image repository. The command format is as follows: + +```shell + isula-build login SERVER [FLAGS] +``` + +Currently, the following flags are supported: + +```shell + Flags: + -p, --password-stdin Read password from stdin + -u, --username string Username to access registry +``` + +Enter the password through stdin. In the following example, the password in creds.txt is transferred to the stdin of isula-build through a pipe for input. + +```sh + $ cat creds.txt | sudo isula-build login -u cooper -p mydockerhub.io + Login Succeeded +``` + +Enter the password in interactive mode. + +```sh + $ sudo isula-build login mydockerhub.io -u cooper + Password: + Login Succeeded +``` + +## logout: Logging Out of the Remote Image Repository + +You can run the `logout` command to log out of the remote image repository. The command format is as follows: + +```shell + isula-build logout [SERVER] [FLAGS] +``` + +Currently, the following flags are supported: + +```shell + Flags: + -a, --all Logout all registries +``` + +Example: + +```sh + $ sudo isula-build logout -a + Removed authentications +``` + +## version: Querying the isula-build Version + +You can run the `version` command to view the current version information. + +```sh +$ sudo isula-build version +Client: + Version: 0.9.6-4 + Go Version: go1.15.7 + Git Commit: 83274e0 + Built: Wed Jan 12 15:32:55 2022 + OS/Arch: linux/amd64 + +Server: + Version: 0.9.6-4 + Go Version: go1.15.7 + Git Commit: 83274e0 + Built: Wed Jan 12 15:32:55 2022 + OS/Arch: linux/amd64 +``` + +## manifest: Manifest List Management + +The manifest list contains the image information corresponding to different system architectures. You can use the same manifest (for example, **openeuler:latest**) in different architectures to obtain the image of the corresponding architecture. The manifest contains the create, annotate, inspect, and push subcommands. + +> [!NOTE]NOTE +> +> manifest is an experiment feature. When using this feature, you need to enable the experiment options on the client and server. For details, see Client Overview and Configuring Services. + +### create: Manifest List Creation + +The create subcommand of the `manifest` command is used to create a manifest list. The command format is as follows: + +```shell +isula-build manifest create MANIFEST_LIST MANIFEST [MANIFEST...] +``` + +You can specify the name of the manifest list and the remote images to be added to the list. If no remote image is specified, an empty manifest list is created. + +Example: + +```sh +sudo isula-build manifest create openeuler localhost:5000/openeuler_x86:latest localhost:5000/openeuler_aarch64:latest +``` + +### annotate: Manifest List Update + +The `annotate` subcommand of the `manifest` command is used to update the manifest list. The command format is as follows: + +```shell +isula-build manifest annotate MANIFEST_LIST MANIFEST [flags] +``` + +You can specify the manifest list to be updated and the images in the manifest list, and use flags to specify the options to be updated. This command can also be used to add new images to the manifest list. + +Currently, the following flags are supported: + +- --arch: Applicable architecture of the rewritten image. The value is a string. +- --os: Indicates the applicable system of the image. The value is a string. +- --os-features: Specifies the OS features required by the image. This parameter is a string and rarely used. +- --variant: Variable of the image recorded in the list. The value is a string. + +Example: + +```sh +sudo isula-build manifest annotate --os linux --arch arm64 openeuler:latest localhost:5000/openeuler_aarch64:latest +``` + +### inspect: Manifest List Inspect + +The `inspect` subcommand of the `manifest` command is used to query the manifest list. The command format is as follows: + +```shell +isula-build manifest inspect MANIFEST_LIST +``` + +Example: + +```sh +$ sudo isula-build manifest inspect openeuler:latest +{ + "schemaVersion": 2, + "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json", + "manifests": [ + { + "mediaType": "application/vnd.docker.distribution.manifest.v2+json", + "size": 527, + "digest": "sha256:bf510723d2cd2d4e3f5ce7e93bf1e52c8fd76831995ac3bd3f90ecc866643aff", + "platform": { + "architecture": "amd64", + "os": "linux" + } + }, + { + "mediaType": "application/vnd.docker.distribution.manifest.v2+json", + "size": 527, + "digest": "sha256:f814888b4bb6149bd39ba8375a1932fb15071b4dbffc7f76c7b602b06abbb820", + "platform": { + "architecture": "arm64", + "os": "linux" + } + } + ] +} +``` + +### push: Manifest List Push to the Remote Repository + +The manifest subcommand `push` is used to push the manifest list to the remote repository. The command format is as follows: + +```shell +isula-build manifest push MANIFEST_LIST DESTINATION +``` + +Example: + +```sh +sudo isula-build manifest push openeuler:latest localhost:5000/openeuler:latest +``` + +# Directly Integrating a Container Engine + +isula-build can be integrated with iSulad or Docker to import the built container image to the local storage of the container engine. + +## Integration with iSulad + +Images that are successfully built can be directly exported to the iSulad. + +Example: + +```sh +sudo isula-build ctr-img build -f Dockerfile -o isulad:busybox:2.0 +``` + +Specify iSulad in the -o parameter to export the built container image to iSulad. You can query the image using isula images. + +```sh +$ sudo isula images +isula images +REPOSITORY TAG IMAGE ID CREATED SIZE +busybox 2.0 2d414a5cad6d 2020-08-01 06:41:36 5.577 MB +``` + +> ![](./public_sys-resources/icon-note.gif) **Note:** +> +> - It is required that isula-build and iSulad be on the same node. +> - When an image is directly exported to the iSulad, the isula-build client needs to temporarily store the successfully built image as `/var/lib/isula-build/tmp/[build_id]/isula-build-tmp-%v.tar` and then import it to the iSulad. Ensure that the /var/tmp/ directory has sufficient disk space. If the isula-build client process is killed or Ctrl+C is pressed during the export, you need to manually clear the `/var/lib/isula-build/tmp/[build_id]/isula-build-tmp-%v.tar` file. + +## Integration with Docker + +Images that are successfully built can be directly exported to the Docker daemon. + +Example: + +```sh +sudo isula-build ctr-img build -f Dockerfile -o docker-daemon:busybox:2.0 +``` + +Specify docker-daemon in the -o parameter to export the built container image to Docker. You can run the `docker images` command to query the image. + +```sh +$ sudo docker images +REPOSITORY TAG IMAGE ID CREATED SIZE +busybox 2.0 2d414a5cad6d 2 months ago 5.22MB +``` + +> ![](./public_sys-resources/icon-note.gif) **Note:** +> +> The isula-build and Docker must be on the same node. + +# Precautions + +This chapter is something about constraints, limitations and differences with `docker build` when use isula-builder build images. + +## Constraints or Limitations + +1. When export an image to [iSulad](https://gitee.com/openeuler/iSulad/blob/master/README.md/), a tag is necessary. +2. Because the OCI runtime, for example, **runc**, will be called by isula-builder when executing the **RUN** instruction, the integrity of the runtime binary should be guaranteed by the user. +3. DataRoot should not be set to **tmpfs**. +4. **Overlay2** is the only storage driver supported by isula-builder currently. +5. Docker image is the only image format supported by isula-builder currently. +6. You are advised to set file permission of the Dockerfile to **0600** to avoid tampering by other users. +7. Only host network is supported by the **RUN** instruction currently. +8. When export image to a tar package, only tar compression format is supported by isula-builder currently. +9. The base image size is limited to 1 GB when importing a base image using `import`. + +## Differences with "docker build" + +`isula-build` complies with [Dockerfile specification](https://docs.docker.com/engine/reference/builder), but there are also some subtle differences between `isula-builder` and `docker build` as follows: + +1. isula-builder commits after each build stage, but not every line. +2. Build cache is not supported by isula-builder. +3. Only **RUN** instruction will be executed in the build container. +4. Build history is not supported currently. +5. Stage name can be start with a number. +6. The length of the stage name is limited to 64 in `isula-builder`. +7. **ADD** instruction source can not be a remote URL currently. +8. Resource restriction on a single build is not supported. If resource restriction is required, you can configure a resource limit on isula-builder. +9. `isula-builder` add each origin layer tar size to get the image size, but docker only uses the diff content of each layer. So the image size listed by `isula-builder images` is different. +10. Image name should be in the *NAME:TAG* format. For example **busybox:latest**, where **latest** must not be omitted. diff --git a/docs/en/cloud/image_builder/isula-build/overview.md b/docs/en/cloud/image_builder/isula-build/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..16ea3ca52967e28fa00dd9da8ff40186e6a8957a --- /dev/null +++ b/docs/en/cloud/image_builder/isula-build/overview.md @@ -0,0 +1,13 @@ +# Container Image Building + +## Overview + +isula-build is a container image build tool developed by the iSula container team. It allows you to quickly build container images using Dockerfiles. + +The isula-build uses the server/client mode. The isula-build functions as a client and provides a group of command line tools for image build and management. The isula-builder functions as the server to process client management requests, and runs as a daemon process in the background. + +![isula-build architecture](./figures/isula-build_arch.png) + +> ![](./public_sys-resources/icon-note.gif) **Note:** +> +> Currently, isula-build supports OCI image format ([OCI Image Format Specification](https://github.com/opencontainers/image-spec/blob/main/spec.md/)) and Docker image format ([Image Manifest Version 2, Schema 2](https://docs.docker.com/registry/spec/manifest-v2-2/)). Use the `export ISULABUILD_CLI_EXPERIMENTAL=enabled` command to enable the experimental feature for supporting OCI image format. When the experimental feature is disabled, isula-build will take Docker image format as the default image format. Otherwise, isula-build will take OCI image format as the default image format. diff --git a/docs/en/cloud/image_builder/isula-build/public_sys-resources/icon-note.gif b/docs/en/cloud/image_builder/isula-build/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/cloud/image_builder/isula-build/public_sys-resources/icon-note.gif differ diff --git a/docs/en/cloud/kmesh/kmesh/_toc.yaml b/docs/en/cloud/kmesh/kmesh/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..f24dfa1cffba685997c38eb0babbcf1e4e64b863 --- /dev/null +++ b/docs/en/cloud/kmesh/kmesh/_toc.yaml @@ -0,0 +1,16 @@ +label: Kmesh User Guide +isManual: true +description: High-performance service mesh data plane software for openEuler +sections: + - label: Overview + href: ./overview.md + - label: Introduction to Kmesh + href: ./getting-to-know-kmesh.md + - label: Installation and Deployment + href: ./installation-and-deployment.md + - label: Usage + href: ./usage.md + - label: Common Issues and Solutions + href: ./faqs.md + - label: Appendix + href: ./appendixes.md diff --git a/docs/en/cloud/kmesh/kmesh/appendixes.md b/docs/en/cloud/kmesh/kmesh/appendixes.md new file mode 100644 index 0000000000000000000000000000000000000000..fd5059d26b33ff142b15f73c686e347baa9d549a --- /dev/null +++ b/docs/en/cloud/kmesh/kmesh/appendixes.md @@ -0,0 +1,3 @@ +# Appendix + +Learn more about [Kmesh](https://gitee.com/openeuler/Kmesh#kmesh). diff --git a/docs/en/cloud/kmesh/kmesh/faqs.md b/docs/en/cloud/kmesh/kmesh/faqs.md new file mode 100644 index 0000000000000000000000000000000000000000..834d2c4257a933ff8171d62a6694af0ca06ec49c --- /dev/null +++ b/docs/en/cloud/kmesh/kmesh/faqs.md @@ -0,0 +1,23 @@ +# Common Issues and Solutions + +## Issue 1: Kmesh Service Exits with an Error When Started in Cluster Mode without Control Plane IP Address Configuration + +![](./figures/not_set_cluster_ip.png) + +Cause: When operating in cluster mode, Kmesh requires communication with the control plane to fetch configuration details. Without the correct control plane IP address, the service cannot proceed and exits with an error. + +Solution: Follow the cluster mode setup instructions in the [Installation and Deployment](./installation-and-deployment.md) guide to properly configure the control plane IP address. + +## Issue 2: Kmesh Service Displays "Get Kube Config Error!" during Startup + +![](./figures/get_kubeconfig_error.png) + +Cause: In cluster mode, Kmesh attempts to retrieve the control plane IP address from the k8s configuration. If the kubeconfig file path is not set in the environment, the service cannot access the kubeconfig and throws this error. (Note: This issue does not occur if the control plane IP address is manually specified in the Kmesh configuration file.) + +Solution: Set up kubeconfig using the following commands: + +```shell +mkdir -p $HOME/.kube +sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config +sudo chown $(id -u):$(id -g) $HOME/.kube/config +``` diff --git a/docs/en/cloud/kmesh/kmesh/figures/get_kubeconfig_error.png b/docs/en/cloud/kmesh/kmesh/figures/get_kubeconfig_error.png new file mode 100644 index 0000000000000000000000000000000000000000..99087b68c6fafea1506e5f8bd862c371e93bdc97 Binary files /dev/null and b/docs/en/cloud/kmesh/kmesh/figures/get_kubeconfig_error.png differ diff --git a/docs/en/cloud/kmesh/kmesh/figures/kmesh-arch.png b/docs/en/cloud/kmesh/kmesh/figures/kmesh-arch.png new file mode 100644 index 0000000000000000000000000000000000000000..000ec80ff35556199caef6ce78953599c1c52312 Binary files /dev/null and b/docs/en/cloud/kmesh/kmesh/figures/kmesh-arch.png differ diff --git a/docs/en/cloud/kmesh/kmesh/figures/not_set_cluster_ip.png b/docs/en/cloud/kmesh/kmesh/figures/not_set_cluster_ip.png new file mode 100644 index 0000000000000000000000000000000000000000..9c879f37fa93c0f4fe0ab0f6220beff174e5f436 Binary files /dev/null and b/docs/en/cloud/kmesh/kmesh/figures/not_set_cluster_ip.png differ diff --git a/docs/en/cloud/kmesh/kmesh/getting-to-know-kmesh.md b/docs/en/cloud/kmesh/kmesh/getting-to-know-kmesh.md new file mode 100644 index 0000000000000000000000000000000000000000..196bf4a5a43a8e541c1267d3c6ebae923f05a9c7 --- /dev/null +++ b/docs/en/cloud/kmesh/kmesh/getting-to-know-kmesh.md @@ -0,0 +1,38 @@ +# Introduction to Kmesh + +## Introduction + +As the number of cloud-native applications surges, the scale of cloud applications and application SLAs pose high requirements on cloud infrastructure. + +The Kubernetes-based cloud infrastructure can help implement agile deployment and management of applications. However, it does not support application traffic orchestration. The emergence of service mesh makes up for the lack of traffic orchestration in Kubernetes and complements Kubernetes to implement agile cloud application development and O&M. However, with the development of service mesh applications, the current Sidecar-based mesh architecture has obvious performance defects on the data plane, which has become a consensus in the industry. + +* Long delay + Take the typical service mesh Istio as an example. After meshing, the single-hop delay of service access increases by 2.65 ms, which cannot meet the requirements of delay-sensitive applications. + +* High overhead + In Istio, each Sidecar configuration occupies more than 50 MB memory, and the CPU exclusively occupies two cores by default. For large-scale clusters, the overhead is high, reducing the deployment density of service containers. + +Based on the programmable kernel, Kmesh offloads mesh traffic governance to the OS and shortens the data path from 3 hops to 1 hop, greatly shortening the delay of the data plane and accelerating service innovation. + +## Architecture + +The following figure shows the overall architecture of Kmesh. + +![](./figures/kmesh-arch.png) + +Kmesh consists of the following components: + +* kmesh-controller + Kmesh management program, which is responsible for Kmesh lifecycle management, xDS protocol interconnection, and O&M monitoring. + +* kmesh-api + API layer provided by Kmesh for external systems, including orchestration APIs converted by xDS and O&M monitoring channels. + +* kmesh-runtime + Runtime that supports L3 to L7 traffic orchestration implemented in the kernel. + +* kmesh-orchestration + L3 to L7 traffic orchestration implemented based on eBPF, such as routing, gray release, and load balancing. + +* kmesh-probe + O&M monitoring probe, providing E2E monitoring capabilities. diff --git a/docs/en/cloud/kmesh/kmesh/installation-and-deployment.md b/docs/en/cloud/kmesh/kmesh/installation-and-deployment.md new file mode 100644 index 0000000000000000000000000000000000000000..72739e2db7cf63aa9ad63771549105fda66d8177 --- /dev/null +++ b/docs/en/cloud/kmesh/kmesh/installation-and-deployment.md @@ -0,0 +1,99 @@ +# Installation and Deployment + +## Software + +* OS: openEuler 23.09 + +## Hardware + +* x86_64 + +## Preparing the Environment + +* Install the openEuler OS by referring to the [*openEuler Installation Guide*](../../../server/installation_upgrade/installation/installation.md). + +* Root permissions are required for installing Kmesh. + +## Installing Kmesh + +* Install the Kmesh software package. + +```shell +yum install Kmesh +``` + +* Check whether the installation is successful. If the command output contains the name of the software package, the installation is successful. + +```shell +rpm -q Kmesh +``` + +## Deploying Kmesh + +### Cluster Mode + +Before starting Kmesh, configure the IP address of the control plane program (for example, Istiod IP address) in the cluster. + +```json + "clusters": [ + { + "name": "xds-grpc", + "type" : "STATIC", + "connect_timeout": "1s", + "lb_policy": "ROUND_ROBIN", + "load_assignment": { + "cluster_name": "xds-grpc", + "endpoints": [{ + "lb_endpoints": [{ + "endpoint": { + "address":{ + "socket_address": { + "protocol": "TCP", + "address": "192.168.0.1",# Configure the control plane IP address (for example, Istiod IP address). + "port_value": 15010 + } + } + } + }] + }] +``` + +Currently, only the traffic orchestration function is supported in the cluster mode. + +### Local Mode + +Before starting Kmesh, modify `kmesh.service` to enable or disable required functions. + +```shell +# Choose -enable-kmesh and disable ADS. +$ vim /usr/lib/systemd/system/kmesh.service +ExecStart=/usr/bin/kmesh-daemon -enable-kmesh -enable-ads=false +$ systemctl daemon-reload +``` + +To enable mesh acceleration, run the following commands: + +```shell +# Choose -enable-mda and disable ADS. +$ vim /usr/lib/systemd/system/kmesh.service +ExecStart=/usr/bin/kmesh-daemon -enable-mda -enable-ads=false +$ systemctl daemon-reload +``` + +When the Kmesh service is started, the kmesh-daemon program is invoked. For details about how to use the kmesh-daemon program, see [Using kmesh-daemon](./usage.md). + +### Starting Kmesh + +```shell +# Start the Kmesh service. +$ systemctl start kmesh.service +# Check the Kmesh running status. +$ systemctl status kmesh.service +``` + +### Stopping Kmesh + +```shell +# Stop the Kmesh service. +$ systemctl stop kmesh.service +``` diff --git a/docs/en/cloud/kmesh/kmesh/overview.md b/docs/en/cloud/kmesh/kmesh/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..ae920162dd78ed35285865abf55d0d2c0435c215 --- /dev/null +++ b/docs/en/cloud/kmesh/kmesh/overview.md @@ -0,0 +1,5 @@ +# Kmesh User Guide + +This document describes how to install, deploy, and use Kmesh, a high-performance service mesh data plane software provided by openEuler. + +This document is intended for developers, open source enthusiasts, and partners who use the openEuler operating system (OS) and want to learn and use Kmesh. Users must have basic knowledge of the Linux OS. diff --git a/docs/en/cloud/kmesh/kmesh/usage.md b/docs/en/cloud/kmesh/kmesh/usage.md new file mode 100644 index 0000000000000000000000000000000000000000..36ca45afc5eeac9ca454ab8edf6492d412c4b503 --- /dev/null +++ b/docs/en/cloud/kmesh/kmesh/usage.md @@ -0,0 +1,69 @@ +# Usage + +## Using kmesh-daemon + +```shell +# Display help information. +[root@openEuler ~]# kmesh-daemon -h +Usage of kmesh-daemon: + -bpf-fs-path string + bpf fs path (default "/sys/fs/bpf") + -cgroup2-path string + cgroup2 path (default "/mnt/kmesh_cgroup2") + -config-file string + [if -enable-kmesh] deploy in kube cluster (default "/etc/kmesh/kmesh.json") + -enable-ads + [if -enable-kmesh] enable control-plane from ads (default true) + -enable-kmesh + enable bpf kmesh + -service-cluster string + [if -enable-kmesh] TODO (default "TODO") + -service-node string + [if -enable-kmesh] TODO (default "TODO") + +# Enable ADS by default. +[root@openEuler ~]# kmesh-daemon -enable-kmesh + +# Enable ADS and specify the path of the configuration file. +[root@openEuler ~]# kmesh-daemon -enable-kmesh -enable-ads=true -config-file=/examples/kmesh.json + +# Disable ADS. +[root@openEuler ~]# kmesh-daemon -enable-kmesh -enable-ads=false +``` + +## Using kmesh-cmd + +```shell +# Display help information. +[root@openEuler ~]# kmesh-cmd -h +Usage of kmesh-cmd: + -config-file string + input config-resources to bpf maps (default "./config-resources.json") + +# Manually load configurations. +[root@openEuler ~]# kmesh-cmd -config-file=/examples/config-resources.json +``` + +## Using O&M Commands + +```shell +# Display help information. +[root@openEuler ~]# curl http://localhost:15200/help + /help: print list of commands + /options: print config options + /bpf/kmesh/maps: print bpf kmesh maps in kernel + /controller/envoy: print control-plane in envoy cache + /controller/kubernetes: print control-plane in kubernetes cache + +# Read the loaded configurations. +[root@openEuler ~]# curl http://localhost:15200/bpf/kmesh/maps +[root@openEuler ~]# curl http://localhost:15200/options +``` + +## Precautions + +* If `-enable-ads=true` is configured, Kmesh automatically receives orchestration rules from the service mesh control plane. In this case, do not run the `kmesh-cmd` command to deliver rules to avoid duplicated configurations. + +* The `-bpf-fs-path` option specifies the BPF directory of the OS. Data related to the Kmesh BPF program will be stored in this directory. The default directory is `/sys/fs/bpf`. + +* The `-cgroup2-path` option specifies the cgroup directory of the OS. The default directory is `/mnt/kmesh_cgroup2`. diff --git a/docs/en/cloud/kubeos/kubeos/_toc.yaml b/docs/en/cloud/kubeos/kubeos/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..d94e671a881944aec853a74da8bed691799f428b --- /dev/null +++ b/docs/en/cloud/kubeos/kubeos/_toc.yaml @@ -0,0 +1,16 @@ +label: KubeOS User Guide +isManual: true +description: >- + KubeOS is a lightweight OS tailored for containerized workloads. It enables + atomic updates, maintains version uniformity, and simplifies O&M. +sections: + - label: Overview + href: ./overview.md + - label: About KubeOS + href: ./about-kubeos.md + - label: Installation and Deployment + href: ./installation-and-deployment.md + - label: Usage Instructions + href: ./usage-instructions.md + - label: KubeOS Image Creation + href: ./kubeos-image-creation.md diff --git a/docs/en/cloud/kubeos/kubeos/about-kubeos.md b/docs/en/cloud/kubeos/kubeos/about-kubeos.md new file mode 100644 index 0000000000000000000000000000000000000000..ec282d26ef1dfbbe0d4a4da2a62e025241b80a10 --- /dev/null +++ b/docs/en/cloud/kubeos/kubeos/about-kubeos.md @@ -0,0 +1,39 @@ +# About KubeOS + +## Introduction + +Containers and Kubernetes are widely used in cloud scenarios. However, a current manner of managing the containers and the OSs separately usually faces problems of function redundancy and difficult collaboration between scheduling systems. In addition, it is difficult to manage OS versions. Software packages are installed, updated, and deleted separately in OSs of the same version. After a period of time, the OS versions become inconsistent, causing version fragmentation. Besides, the OSs may be tightly coupled with services, making it difficult to upgrade major versions. To solve the preceding problems, openEuler provides KubeOS, a container OS upgrade tool based on openEuler. + +Container OSs are lightweight OSs designed for scenarios where services run in containers. KubeOS connects container OSs as components to Kubernetes, so that the container OSs are in the same position as services. With KubeOS, a Kubernetes cluster manages containers and container OSs in a unified system. + +KubeOS is a Kubernetes operator for controlling the container OS upgrade process and upgrading the container OSs as a whole to implement collaboration between the OS managers and services. Before the container OSs are upgraded, services are migrated to other nodes to reduce the impact on services during OS upgrade and configuration. In this upgrade pattern, the container OSs are upgraded atomically so that the OSs remain synchronized with the expected status. This ensures that the OS versions in the cluster are consistent, preventing version fragmentation. + +## Architecture + +### KubeOS Architecture + +**Figure 1** KubeOS architecture + +![](./figures/kubeos-architecture.png) + +As shown in the preceding figure, KubeOS consists of three components: os-operator, os-proxy, and os-agent. The os-operator and os-proxy components run in containers and are deployed in the Kubernetes cluster. os-agent is not considered a cluster component. Its instances run on worker nodes as processes. + +- os-operator: global container OS manager, which continuously checks the container OS versions of all nodes, controls the number of nodes to be upgraded concurrently based on the configured information, and marks the nodes to be upgraded. + +- os-proxy: OS manager of a single node, which continuously checks the container OS version of the node. If a node is marked as the node to be upgraded by os-operator, the node is locked, the pod is evicted, and the upgrade information is forwarded to os-agent. + +- os-agent: receives information from os-proxy, downloads the container OS image used for upgrade from the OS image server, upgrades the container OS, and restarts the node. + +### File System of a Container OS + +**Figure 2** File system layout of a container OS + +![](./figures/file-system-layout-of-a-container-os.png) + +As shown in the figure, a container OS comprises four partitions: + +- boot partition: GRUB2 file partition. +- Persist partition: stores persistent user data. When the container OS is upgraded, the data in this partition is retained. +- Two root partitions: Container OSs use the dual-partition mode with two root partitions, rootA and rootB. Assume that the container runs the OS stored in the rootA partition after initialization. When the system is upgraded, the new system is downloaded to the rootB partition. GRUB has two boot options: A and B. The default boot option of GRUB is set to B and the node is restarted. After the node is started, the container runs the upgraded OS in the rootB partition. + +The root file system of a container OS is read-only. Users' persistent data is stored in the Persist partition. diff --git a/docs/en/cloud/kubeos/kubeos/figures/file-system-layout-of-a-container-os.png b/docs/en/cloud/kubeos/kubeos/figures/file-system-layout-of-a-container-os.png new file mode 100644 index 0000000000000000000000000000000000000000..add62e72f85b103b7dd5780d2e360049f5f712df Binary files /dev/null and b/docs/en/cloud/kubeos/kubeos/figures/file-system-layout-of-a-container-os.png differ diff --git a/docs/en/cloud/kubeos/kubeos/figures/kubeos-architecture.png b/docs/en/cloud/kubeos/kubeos/figures/kubeos-architecture.png new file mode 100644 index 0000000000000000000000000000000000000000..7834a3793b73c49ddd046502c65335a08f576c30 Binary files /dev/null and b/docs/en/cloud/kubeos/kubeos/figures/kubeos-architecture.png differ diff --git a/docs/en/cloud/kubeos/kubeos/installation-and-deployment.md b/docs/en/cloud/kubeos/kubeos/installation-and-deployment.md new file mode 100644 index 0000000000000000000000000000000000000000..91255b529ef17a54921c8ad66fe94b945cb1e55a --- /dev/null +++ b/docs/en/cloud/kubeos/kubeos/installation-and-deployment.md @@ -0,0 +1,194 @@ +# Installation and Deployment + +This chapter describes how to install and deploy the KubeOS tool. + +- [Installation and Deployment](#installation-and-deployment) + - [Software and Hardware Requirements](#software-and-hardware-requirements) + - [Hardware Requirements](#hardware-requirements) + - [Software Requirements](#software-requirements) + - [Environment Preparation](#environment-preparation) + - [KubeOS Installation](#kubeos-installation) + - [KubeOS Deployment](#kubeos-deployment) + - [Building the os-operator and os-proxy Images](#building-the-os-operator-and-os-proxy-images) + - [Creating a KubeOS VM Image](#creating-a-kubeos-vm-image) + - [Deploying CRD, os-operator, and os-proxy](#deploying-crd-os-operator-and-os-proxy) + +## Software and Hardware Requirements + +### Hardware Requirements + +- Currently, only the x86 and AArch64 architectures are supported. + +### Software Requirements + +- OS: openEuler 24.09 + +### Environment Preparation + +- Install the openEuler system. For details, see the [*openEuler Installation Guide*](../../../server/installation_upgrade/installation/installation.md). +- Install qemu-img, bc, Parted, tar, Yum, Docker, and dosfstools. + +## KubeOS Installation + +To install KubeOS, perform the following steps: + +1. Configure the Yum sources openEuler 24.09 and openEuler 24.09:EPOL: + + ```conf + [openEuler24.09] # openEuler 24.09 official source + name=openEuler24.09 + baseurl=http://repo.openeuler.org/openEuler-24.09/everything/$basearch/ + enabled=1 + gpgcheck=1 + gpgkey=http://repo.openeuler.org/openEuler-24.09/everything/$basearch/RPM-GPG-KEY-openEuler + ``` + +2. Install KubeOS as the **root** user. + + ```shell + # yum install KubeOS KubeOS-scripts -y + ``` + +> ![](./public_sys-resources/icon-note.gif)**NOTE**: +> +> KubeOS is installed in the **/opt/kubeOS** directory, including the os-operator, os-proxy, os-agent binary files, KubeOS image build tools, and corresponding configuration files. + +## KubeOS Deployment + +After KubeOS is installed, you need to configure and deploy it. This section describes how to configure and deploy KubeOS. + +### Building the os-operator and os-proxy Images + +#### Environment Preparation + +Before using Docker to create a container image, ensure that Docker has been installed and configured. + +#### Procedure + +1. Go to the working directory. + + ```shell + cd /opt/kubeOS + ``` + +2. Specify the image repository, name, and version for os-proxy. + + ```shell + export IMG_PROXY=your_imageRepository/os-proxy_imageName:version + ``` + +3. Specify the image repository, name, and version for os-operator. + + ```shell + export IMG_OPERATOR=your_imageRepository/os-operator_imageName:version + ``` + +4. Compile a Dockerfile to build an image. Pay attention to the following points when compiling a Dockerfile: + + - The os-operator and os-proxy images must be built based on the base image. Ensure that the base image is safe. + - Copy the os-operator and os-proxy binary files to the corresponding images. + - Ensure that the owner and owner group of the os-proxy binary file in the os-proxy image are **root**, and the file permission is **500**. + - Ensure that the owner and owner group of the os-operator binary file in the os-operator image are the user who runs the os-operator process in the container, and the file permission is **500**. + - The locations of the os-operator and os-proxy binary files in the image and the commands run during container startup must correspond to the parameters specified in the YAML file used for deployment. + + An example Dockerfile is as follows: + + ```text + FROM your_baseimage + COPY ./bin/proxy /proxy + ENTRYPOINT ["/proxy"] + ``` + + ```text + FROM your_baseimage + COPY --chown=6552:6552 ./bin/operator /operator + ENTRYPOINT ["/operator"] + ``` + + Alternatively, you can use multi-stage builds in the Dockerfile. + +5. Build the images (the os-operator and os-proxy images) to be included in the containers OS image. + + ```shell + # Specify the Dockerfile path of os-proxy. + export DOCKERFILE_PROXY=your_dockerfile_proxy + # Specify the Dockerfile path of os-operator. + export DOCKERFILE_OPERATOR=your_dockerfile_operator + # Build images. + docker build -t ${IMG_OPERATOR} -f ${DOCKERFILE_OPERATOR} . + docker build -t ${IMG_PROXY} -f ${DOCKERFILE_PROXY} . + ``` + +6. Push the images to the image repository. + + ```shell + docker push ${IMG_OPERATOR} + docker push ${IMG_PROXY} + ``` + +### Creating a KubeOS VM Image + +#### Precautions + +- The VM image is used as an example. For details about how to create a physical machine image, see **KubeOS Image Creation**. +- The root permission is required for creating a KubeOS image. +- The RPM sources of the kbimg are the **everything** and **EPOL** repositories of openEuler of a specific version. In the Repo file provided during image creation, you are advised to configure the **everything** and **EPOL** repositories of a specific openEuler version for the Yum source. +- By default, the KubeOS VM image built using the default RPM list is stored in the same path as the kbimg tool. This partition must have at least 25 GiB free drive space. +- When creating a KubeOS image, you cannot customize the file system to be mounted. + +#### Procedure + +Use the **kbimg.sh** script to create a KubeOS VM image. For details about the commands, see **KubeOS Image Creation**. + +To create a KubeOS VM image, perform the following steps: + +1. Go to the working directory. + + ```shell + cd /opt/kubeOS/scripts + ``` + +2. Run `kbming.sh` to create a KubeOS image. The following is a command example: + + ```shell + bash kbimg.sh create vm-image -p xxx.repo -v v1 -b ../bin/os-agent -e '''$1$xyz$RdLyKTL32WEvK3lg8CXID0''' + ``` + + In the command, **xx.repo** indicates the actual Yum source file used for creating the image. You are advised to configure both the **everything** and **EPOL** repositories as Yum sources. + + After the KubeOS image is created, the following files are generated in the **/opt/kubeOS/scripts** directory: + + - **system.img**: system image in raw format. The default size is 20 GB. The size of the root file system partition is less than 2,560 MiB, and the size of the Persist partition is less than 14 GiB. + - **system.qcow2**: system image in QCOW2 format. + - **update.img**: partition image of the root file system that is used for upgrade. + + The created KubeOS VM image can be used only in a VM of the x86 or AArch64 architecture. KubeOS does not support legacy boot in an x86 VM + +### Deploying CRD, os-operator, and os-proxy + +#### Precautions + +- The Kubernetes cluster must be deployed first. For details, see the *openEuler 24.09 Kubernetes Cluster Deployment Guide*. + +- The OS of the worker nodes to be upgraded in the cluster must be the KubeOS built using the method described in the previous section. If it is not, use **system.qcow2** to deploy the VM again. For details about how to deploy a VM, see the *openEuler 24.09 Virtualization User Guide*. Currently, KubeOS does not support the master nodes. Use openEuler 24.09 to deploy the upgrade on the master nodes. +- The YAML files for deploying CustomResourceDefinition (CRD), os-operator, os-proxy, and role-based access control (RBAC) of the OS need to be compiled. +- The os-operator and os-proxy components are deployed in the Kubernetes cluster. os-operator must be deployed as a Deployment, and os-proxy as a DaemonSet. +- Kubernetes security mechanisms, such as the RBAC, pod service account, and security policies, must be deployed. + +#### Procedure + +1. Prepare YAML files used for deploying CRD, RBAC, os-operator, and os-proxy of the OS. For details, see [YAML examples](https://gitee.com/openeuler/KubeOS/tree/master/docs/example/config). The following uses **crd.yaml**, **rbac.yaml**, and **manager.yaml** as examples. + +2. Deploy CRD, RBAC, os-operator, and os-proxy. Assume that the **crd.yaml**, **rbac.yaml**, and **manager.yaml** files are stored in the **config/crd**, **config/rbac**, and **config/manager** directories, respectively. Run the following commands: + + ```shell + kubectl apply -f config/crd + kubectl apply -f config/rbac + kubectl apply -f config/manager + ``` + +3. After the deployment is complete, run the following command to check whether each component is started properly. If **STATUS** of all components is **Running**, the components are started properly. + + ```shell + kubectl get pods -A + ``` diff --git a/docs/en/cloud/kubeos/kubeos/kubeos-image-creation.md b/docs/en/cloud/kubeos/kubeos/kubeos-image-creation.md new file mode 100644 index 0000000000000000000000000000000000000000..a591021ba1fff4b90bf9e3bf4d719463520c58b1 --- /dev/null +++ b/docs/en/cloud/kubeos/kubeos/kubeos-image-creation.md @@ -0,0 +1,169 @@ +# KubeOS Image Creation + +## Introduction + +kbimg is an image creation tool required for KubeOS deployment and upgrade. You can use kbimg to create KubeOS Docker, VM, and physical machine images. + +## Commands + +### Command Format + +**bash kbimg.sh** \[ --help \| -h \] create \[ COMMANDS \] \[ OPTIONS \] + +### Parameter Description + +* COMMANDS + + | Parameter | Description | + | ------------- | ---------------------------------------------- | + | upgrade-image | Generates a OCI image for installation and upgrade.| + | vm-image | Generates a VM image for installation and upgrade. | + | pxe-image | Generates images and files required for physical machine installation. | + +* OPTIONS + + | Option | Description | + | ------------ | ------------------------------------------------------------ | + | -p | Path of the repo file. The Yum source required for creating an image is configured in the repo file. | + | -v | Version of the created KubeOS image. | + | -b | Path of the os-agent binary file. | + | -e | Password of the **root** user of the KubeOS image, which is an encrypted password with a salt value. You can run the OpenSSL or KIWI command to generate the password.| + | -d | Generated or used Docker image. | + | -h --help | Help Information. | + +## Usage Description + +### Precautions + +* The root permission is required for executing **kbimg.sh**. +* Currently, only the x86 and AArch64 architectures are supported. +* The RPM sources of the kbimg are the **everything** and **EPOL** repositories of openEuler of a specific version. In the Repo file provided during image creation, you are advised to configure the **everything** and **EPOL** repositories of a specific openEuler version for the Yum source. + +### Creating a KubeOS OCI Image + +#### Precautions + +* The created OCI image can be used only for subsequent VM or physical machine image creation or upgrade. It cannot be used to start containers. +* If the default RPM list is used to create a KubeOS image, at least 6 GB drive space is required. If the RPM list is customized, the occupied drive space may exceed 6 GB. + +#### Example + +* To configure the DNS, customize the `resolv.conf` file in the `scripts` directory. + +```shell + cd /opt/kubeOS/scripts + touch resolv.conf + vim resolv.conf +``` + +* Create a KubeOS image. + +``` shell +cd /opt/kubeOS/scripts +bash kbimg.sh create upgrade-image -p xxx.repo -v v1 -b ../bin/os-agent -e '''$1$xyz$RdLyKTL32WEvK3lg8CXID0''' -d your_imageRepository/imageName:version +``` + +* After the creation is complete, view the created KubeOS image. + +``` shell +docker images +``` + +### Creating a KubeOS VM Image + +#### Precautions + +* To use a Docker image to create a KubeOS VM image, pull the corresponding image or create a Docker image first and ensure the security of the Docker image. +* The created KubeOS VM image can be used only in a VM of the x86 or AArch64 architecture. +* Currently, KubeOS does not support legacy boot in an x86 VM. +* If the default RPM list is used to create a KubeOS image, at least 25 GB drive space is required. If the RPM list is customized, the occupied drive space may exceed 25 GB. + +#### Example + +* Using the Repo Source + * To configure the DNS, customize the `resolv.conf` file in the `scripts` directory. + + ```shell + cd /opt/kubeOS/scripts + touch resolv.conf + vim resolv.conf + ``` + + * Create a KubeOS VM image. + + ``` shell + cd /opt/kubeOS/scripts + bash kbimg.sh create vm-image -p xxx.repo -v v1 -b ../bin/os-agent -e '''$1$xyz$RdLyKTL32WEvK3lg8CXID0''' + ``` + +* Using a Docker Image + + ``` shell + cd /opt/kubeOS/scripts + bash kbimg.sh create vm-image -d your_imageRepository/imageName:version + ``` + +* Result Description + After the KubeOS image is created, the following files are generated in the **/opt/kubeOS/scripts** directory: + * **system.qcow2**: system image in QCOW2 format. The default size is 20 GiB. The size of the root file system partition is less than 2,020 MiB, and the size of the Persist partition is less than 16 GiB. + * **update.img**: partition image of the root file system used for upgrade. + +### Creating Images and Files Required for Installing KubeOS on Physical Machines + +#### Precautions + +* To use a Docker image to create a KubeOS VM image, pull the corresponding image or create a Docker image first and ensure the security of the Docker image. +* The created image can only be used to install KubeOS on a physical machine of the x86 or AArch64 architecture. +* The IP address specified in the **Global.cfg** file is a temporary IP address used during installation. After the system is installed and started, configure the network by referring to **openEuler 22.09 Administrator Guide** > **Configuring the Network**. +* KubeOS cannot be installed on multiple drives at the same time. Otherwise, the startup may fail or the mounting may be disordered. +* Currently, KubeOS does not support legacy boot in an x86 physical machine. +* If the default RPM list is used to create a KubeOS image, at least 5 GB drive space is required. If the RPM list is customized, the occupied drive space may exceed 5 GB. + +#### Example + +* Modify the `00bootup/Global.cfg` file. All parameters are mandatory. Currently, only IPv4 addresses are supported. The following is a configuration example: + + ```shell + # rootfs file name + rootfs_name=kubeos.tar + # select the target disk to install kubeOS + disk=/dev/sda + # pxe server ip address where stores the rootfs on the http server + server_ip=192.168.1.50 + # target machine temporary ip + local_ip=192.168.1.100 + # target machine temporary route + route_ip=192.168.1.1 + # target machine temporary netmask + netmask=255.255.255.0 + # target machine netDevice name + net_name=eth0 + ``` + +* Using the Repo Source + * To configure the DNS, customize the `resolv.conf` file in the `scripts` directory. + + ```shell + cd /opt/kubeOS/scripts + touch resolv.conf + vim resolv.conf + ``` + + * Create an image required for installing KubeOS on a physical machine. + + ```shell + cd /opt/kubeOS/scripts + bash kbimg.sh create pxe-image -p xxx.repo -v v1 -b ../bin/os-agent -e '''$1$xyz$RdLyKTL32WEvK3lg8CXID0''' + ``` + +* Using a Docker Image + + ``` shell + cd /opt/kubeOS/scripts + bash kbimg.sh create pxe-image -d your_imageRepository/imageName:version + ``` + +* Result Description + + * **initramfs.img**: initramfs image used for boot from PXE. + * **kubeos.tar**: OS used for installation from PXE. diff --git a/docs/en/cloud/kubeos/kubeos/overview.md b/docs/en/cloud/kubeos/kubeos/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..aa615c852e059d2ea061ea75ee0cd9d5264ebfa7 --- /dev/null +++ b/docs/en/cloud/kubeos/kubeos/overview.md @@ -0,0 +1,8 @@ +# KubeOS User Guide + +This document describes how to install, deploy, and use KubeOS in the openEuler system. KubeOS connects the container OS to the scheduling system in standard extension pattern and manages the OS upgrade of nodes in the cluster through the scheduling system. + +This document is intended for community developers, open source enthusiasts, and partners who use the openEuler system and want to learn and use the container OSs. Users must: + +* Know basic Linux operations. +* Understand Kubernetes and Docker. diff --git a/docs/en/cloud/kubeos/kubeos/public_sys-resources/icon-note.gif b/docs/en/cloud/kubeos/kubeos/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/cloud/kubeos/kubeos/public_sys-resources/icon-note.gif differ diff --git a/docs/en/cloud/kubeos/kubeos/usage-instructions.md b/docs/en/cloud/kubeos/kubeos/usage-instructions.md new file mode 100644 index 0000000000000000000000000000000000000000..1bb0220e8fc9c6fb6b5619dbed766b56bbe16ab6 --- /dev/null +++ b/docs/en/cloud/kubeos/kubeos/usage-instructions.md @@ -0,0 +1,517 @@ +# Usage Instructions + + + +- [Usage Instructions](#usage-instructions) + - [Precautions](#precautions) + - [OS CR Parameters](#os-cr-parameters) + - [Upgrade](#upgrade) + - [Settings](#settings) + - [Rollback](#rollback) + - [Appendixes](#appendixes) + - [Settings List](#settings-list) + - [kernel Settings](#kernel-settings) + - [GRUB Settings](#grub-settings) + + +## Precautions + +- General precautions + - KubeOS currently only supports virtual machines (VMs) and physical machines using UEFI with x86 and AArch64 architectures. + - When creating or updating the OS CustomResource (CR) using `kubectl apply` with a YAML file, avoid concurrent `apply` operations. Excessive concurrent requests may overwhelm the kube-apiserver, leading to failures. + - If you configure certificates or keys for the container image registry, ensure that the permissions on these files are set to the minimum necessary. +- Upgrade precautions + - Upgrades are performed as atomic upgrades for all packages. Individual package upgrades are not supported. + - Upgrades use a dual-partition upgrade strategy. Configurations with more than two partitions are not supported. + - Cross-major version upgrades are not currently supported. + - Logs for the upgrade process on a single node can be found in the **/var/log/messages** file on that node. + - Strictly adhere to the provided upgrade and rollback procedures. Deviation from the prescribed order of operations may result in upgrade or rollback failures. + - If you need to configure private images for `ctr` (used by `containerd`) on a node, place the **host.toml** configuration file in the `/etc/containerd/certs.d` directory, following the `ctr` guidelines. + - Upgrades using OCI images and mutual TLS (mTLS) authentication are only supported on openEuler 22.09 and later. + - Features `nodeselector`, `executionmode`, `timewindow`, and `timeinterval` are only supported on openEuler 24.09 and later. + - KubeOS 24.09 is not compatible with previous versions. + +- Configuration Precautions + - Users are responsible for the security and reliability of any custom configurations, particularly persistent configurations such as `kernel.sysctl.persist`, `grub.cmdline.current`, and `grub.cmdline.next`. KubeOS does not validate the effectiveness of these parameters. + - When `opstype` is set to `config`, configurations will not be applied if the specified `osversion` does not match the OS version of the target nodes in the cluster. + - Currently, only temporary kernel parameter configuration (`kernel.sysctl`), persistent kernel parameter configuration (`kernel.sysctl.persist`), and GRUB command line configuration (`grub.cmdline.current` and `grub.cmdline.next`) are supported. + - Persistent configurations are written to the persistent partition and will be retained after upgrades and reboots. Temporary kernel parameter configurations will not be retained after a reboot. + - When configuring `grub.cmdline.current` or `grub.cmdline.next`, if a single parameter is provided (not in the `key=value` format), specify the parameter as the key and leave the value empty. + - When deleting a configuration (`operation=delete`), ensure that the key and value in the `key=value` format match the actual configuration. + - Configuration changes cannot be rolled back. If a rollback is required, modify the configuration version and content and reapply the configuration. + - If a configuration error occurs and a node enters the `config` state, revert the configuration version to the previous version and reapply it. This should return the node to the `idle` state. However, note that parameters successfully configured before the error occurred cannot be reverted. + - When configuring `grub.cmdline.current` or `grub.cmdline.next`, if you need to update an existing parameter in the format of `key=value` to a format with only key and no value, for example, updating `rd.info=0` to `rd.info`, you need to delete `key=value` first, and then add the key in the next configuration. Direct updates or updates and deletions in the same operation are not supported. + +## OS CR Parameters + +Create a custom object of the OS type in the cluster and set the corresponding fields. The OS type comes from the CRD object created in the installation and deployment sections. The following describes the fields. + +- The `imageurl` field specifies the location of the operating system image. This URL must use either the `http` or `https` protocol. For `https`, the image transfer is secure. For `http`, you must set the `flagSafe` parameter to `true`. This explicitly signals that you trust the source and allows the image download to proceed. If `imageurl` uses `http` and `flagSafe` is not `true`, the URL is considered unsafe, the image will not be downloaded, and an error message will appear in the node upgrade log. +- You are advised to use the `https` protocol for security. When using `https`, ensure that the target machines being upgraded have the necessary certificates installed. If you maintain the image server yourself, you must sign the images to guarantee their authenticity and ensure the nodes being upgraded trust your certificate. Place the certificate file in the `/etc/KubeOS/certs` directory. The administrator provides the `imageurl` and is responsible for ensuring the security and validity of this URL. An internal network address is recommended for enhanced security. +- The provider of the container OS image is responsible for its integrity. Verify that you obtain images from a trustworthy source. +- When your cluster uses multiple OS versions (meaning that there are multiple OS instances), each OS must have a distinct `nodeselector`. This ensures that a group of nodes identified by a specific label corresponds to only one OS instance. + - If an OS instance has `nodeselector` set to `all-label`, it will be the only valid instance in the cluster (only nodes matching its criteria will be managed). + - Similarly, only one OS instance can have an unconfigured `nodeselector`. This is because an absent `nodeselector` is interpreted as targeting nodes without any labels. +- `timewinterval` parameter + - When not set, the default value is 15 seconds. + - Setting this parameter to `0` will cause the task dispatch interval of the operator to gradually increase until it reaches 1000 seconds. This behavior is due to rate limiting imposed by the Kubernetes `controller-runtime`. + - In parallel execution mode, `timeinterval` defines the delay between the operator dispatching upgrade/configuration tasks for each batch of nodes. + - In serial execution mode, `timeinterval` represents the delay between the completion of one batch of nodes (upgraded/configured serially) and the dispatch of the next upgrade/configuration task. Within a batch, the interval between individual nodes remains 15 seconds. + - Any update to fields of an OS instance will immediately trigger the operator. + + | Parameter | Type | Description | Usage Notes | Mandatory | + | ---------------- | ------ | ------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------- | + | `imagetype` | string | Type of image used for the upgrade | The value can be `docker`, `containerd`, or `disk` and is valid only for upgrades. **Note:** When the value is `containerd`, the agent prioritizes the `crictl` tool for pulling images. If `crictl` is unavailable, it uses the `ctr` command. When `ctr` is used to pull images from a private repository, configure the repository host information in the **/etc/containerd/certs.d** directory according to the [containerd official documentation](https://github.com/containerd/containerd/blob/main/docs/hosts.md). | Yes | + | `opstype` | string | Operation type (upgrade, configuration, or rollback) | The value can be `upgrade`, `config`, or `rollback`. | Yes | + | `osversion` | string | Target version for the upgrade or rollback | `osversion` must match the target OS version of the nodes (specified in the `PRETTY_NAME` field in the **/etc/os-release** file or the OS version detected by Kubernetes). For example: `KubeOS 1.0.0`. | Yes | + | `maxunavailable` | int | Maximum number of nodes undergoing upgrade/configuration/rollback concurrently | If `maxunavailable` exceeds the actual number of nodes, the operation proceeds with the actual number of nodes. | Yes | + | `containerimage` | string | Container image used for the upgrade | This parameter is only applicable when `imagetype` is a container type. The value can be one of the three container image address formats: `repository/name`, `repository/name@sha256:xxxx`, and `repository/name:tag`. | Yes | + | `imageurl` | string | URL of the drive image used for the upgrade | `imageurl` must include the protocol and supports only `http` or `https`. Example: `https://192.168.122.15/update.img`. Valid only for upgrades using drive images. | Yes | + | `checksum` | string | Checksum (SHA-256) of the drive image used for the upgrade or the digests of the container image | This parameter is valid only for upgrades. | Yes | + | `flagSafe` | bool | Whether the address specified by `imageurl` is safe when the `http` protocol is used | The value must be `true` or `false`. This parameter is valid only when `imageurl` uses the `http` protocol. | Yes | + | `mtls` | bool | Whether the connection to `imageurl` uses two-way HTTPS authentication | The value must be `true` or `false`. This parameter is valid only when `imageurl` uses the `https` protocol. | Yes | + | `cacert` | string | Root certificate file used for HTTPS or two-way HTTPS authentication | This parameter is valid only when `imageurl` uses the `https` protocol. | Required when `imageurl` uses `https` | + | `clientcert` | string | Client certificate file used for two-way HTTPS authentication | This parameter is valid only when two-way HTTPS authentication is used. | Required when `mtls` is `true` | + | `clientkey` | string | Client private key file used for two-way HTTPS authentication | This parameter is valid only when two-way HTTPS authentication is used. | Required when `mtls` is `true` | + | `evictpodforce` | bool | Whether to forcibly evict pods during upgrade/rollback | Must be `true` or `false`. This parameter is valid only for upgrades or rollbacks. | Yes | + | `sysconfigs` | / | Configuration settings | 1. When `opstype` is `config`, only configuration is performed.
2. When `opstype` is `upgrade/rollback`, it indicates post-upgrade/rollback configuration, meaning it takes effect after the upgrade/rollback and subsequent reboot. For detailed field descriptions, see the [Settings](#settings). | Required when `opstype` is `config` | + | `upgradeconfigs` | / | Configuration settings to apply before an upgrade. | This parameter is valid for upgrades or rollbacks and takes effect before the upgrade or rollback operation. For detailed field descriptions, see the [Settings](#settings). | Optional | + | `nodeselector` | string | Label of the nodes targeted for the upgrade/configuration/rollback | This parameter is used to perform operations on nodes with specific labels, rather than all worker nodes in the cluster. The nodes targeted for the operation need to have a label with the `upgrade.openeuler.org/node-selector` key. The `nodeselector` parameter should be set to the value of this label. **Notes:** 1. When this parameter is not set or is set to `no-label`, operations are performed on nodes that do not have the `upgrade.openeuler.org/node-selector` label.
2. When this parameter is set to `""` (an empty string), operations are performed on nodes that have the `upgrade.openeuler.org/node-selector=""` label.
3. To ignore labels and perform operations on all nodes, set this parameter to `all-label`. | Optional | + | `timewindow` | / | Time window during which the upgrade/configuration/rollback can take place. | 1. When specifying a time window, both `starttime` and `endtime` must be specified. That is, they should either both be empty or both be non-empty.
1. Both `starttime` and `endtime` are strings and should be in the `YYYY-MM-DD HH:MM:SS` or `HH:MM:SS` format, and both should follow the same format.
2. When in `HH:MM:SS` format, if `starttime` is less than `endtime`, it is assumed that `starttime` refers to that time on the next day.
3. When `timewindow` is not specified, it defaults to no time window restrictions. | Optional | + | `timeinterval` | int | The time interval between each batch of tasks for the upgrade/configuration/rollback operation. | This parameter is in seconds and defines the time interval between the operator dispatching tasks. If the Kubernetes cluster is busy and cannot immediately respond to the operator's request, the actual interval may be longer than the specified time. | Optional | + | `executionmode` | string | The mode in which the upgrade/configuration/rollback operation is executed. | The value can be `serial` or `parallel`. If this parameter is not set, the operation defaults to parallel mode. | Optional | + +## Upgrade + +1. Create a YAML file and deploy an instance of the OS Custom Resource (CR) in the cluster. This YAML file defines the upgrade process. The following example assumes you save the YAML content to **upgrade_v1alpha1_os.yaml**. + + - Upgrade using a drive image + + ```yaml + apiVersion: upgrade.openeuler.org/v1alpha1 + kind: OS + metadata: + name: os-sample + spec: + imagetype: disk + opstype: upgrade + osversion: edit.os.version + maxunavailable: edit.node.upgrade.number + containerimage: "" + evictpodforce: true/false + imageurl: edit.image.url + checksum: image.checksum + flagSafe: imageurl.safety + mtls: imageurl use mtls or not + cacert: ca certificate + clientcert: client certificate + clientkey: client certificate key + ``` + + - Upgrade using a container image + - Before you can upgrade using a container image, you need to create a container image specifically for the upgrade process. For detailed instructions on how to create this image, see [Creating a KubeOS OCI Image](./kubeos-image-creation.md#creating-a-kubeos-oci-image) in [KubeOS Image Creation](./kubeos-image-creation.md). + + ``` yaml + apiVersion: upgrade.openeuler.org/v1alpha1 + kind: OS + metadata: + name: os-sample + spec: + imagetype: docker + opstype: upgrade + osversion: edit.os.version + maxunavailable: edit.node.upgrade.number + containerimage: container image like repository/name:tag + evictpodforce: true/false + imageurl: "" + checksum: container image digests + flagSafe: false + mtls: true + ``` + + - Using containerd as the container engine + + ```yaml + apiVersion: upgrade.openeuler.org/v1alpha1 + kind: OS + metadata: + name: os-sample + spec: + imagetype: containerd + opstype: upgrade + osversion: edit.os.version + maxunavailable: edit.node.upgrade.number + containerimage: container image like repository/name:tag + evictpodforce: true/false + imageurl: "" + checksum: container image digests + flagSafe: false + mtls: true + ``` + + - Example of upgrading and applying configurations + - This example uses containerd as the container engine. The upgrade method does not affect the configuration process. `upgradeconfigs` are applied before the upgrade. `sysconfigs` are applied after the machine reboots from the upgrade. See [Settings](#settings) for detailed information about the configuration parameters. + - When upgrading and configuring, set the `opstype` field to `upgrade`. + + ```yaml + apiVersion: upgrade.openeuler.org/v1alpha1 + kind: OS + metadata: + name: os-sample + spec: + imagetype: "" + opstype: upgrade + osversion: edit.os.version + maxunavailable: edit.node.upgrade.number + containerimage: "" + evictpodforce: true/false + imageurl: "" + checksum: container image digests + flagSafe: false + mtls: false + sysconfigs: + version: edit.os.version + configs: + - model: kernel.sysctl + contents: + - key: kernel param key1 + value: kernel param value1 + - key: kernel param key2 + value: kernel param value2 + - model: kernel.sysctl.persist + configpath: persist file path + contents: + - key: kernel param key3 + value: kernel param value3 + - key: "" + value: "" + upgradeconfigs: + version: 1.0.0 + configs: + - model: kernel.sysctl + contents: + - key: kernel param key4 + value: kernel param value4 + ``` + + - Example of upgrading specific nodes using `nodeselector`, `timewindow`, `timeinterval`, and `executionmode` + - This example uses containerd as the container engine. The upgrade method does not affect node selection. + - Nodes targeted for upgrade must include the `upgrade.openeuler.org/node-selector` label. The value of `nodeselector` in the YAML file should match the value of this label on the desired nodes. For example, if `nodeselector` is set to `kubeos`, only worker nodes with the `upgrade.openeuler.org/node-selector=kubeos` label will be upgraded. + - `nodeselector`, `timewindow`, `timeinterval`, and `executionmode` are also applicable to configuration and rollback operations. + - Example commands for managing node labels: + + ``` shell + # Add a label to node kubeos-node1 + kubectl label nodes kubeos-node1 upgrade.openeuler.org/node-selector=kubeos-v1 + # Modify the label of node kubeos-node1 + kubectl label --overwrite nodes kubeos-node1 upgrade.openeuler.org/node-selector=kubeos-v2 + # Delete the label from node kubeos-node1 + kubectl label nodes kubeos-node1 upgrade.openeuler.org/node-selector- + # View the labels of all nodes + kubectl get nodes --show-labels + ``` + + - Example YAML file: + + ```yaml + apiVersion: upgrade.openeuler.org/v1alpha1 + kind: OS + metadata: + name: os-sample + spec: + imagetype: containerd + opstype: upgrade + osversion: edit.os.version + maxunavailable: edit.node.upgrade.number + containerimage: container image like repository/name:tag + evictpodforce: true/false + imageurl: "" + checksum: container image digests + flagSafe: false + mtls: true + nodeselector: edit.node.label.key + timewindow: + starttime: "HH::MM::SS/YYYY-MM-DD HH::MM::SS" + endtime: "HH::MM::SS/YYYY-MM-DD HH::MM::SS" + timeinterval: time intervel like 30 + executionmode: serial/parallel + ``` + +2. Check the OS version of nodes that have not been upgraded. + + ```shell + kubectl get nodes -o custom-columns='NAME:.metadata.name,OS:.status.nodeInfo.osImage' + ``` + +3. Deploy the CR instance in the cluster. Nodes will be upgraded based on the parameters specified in the YAML file. + + ```shell + kubectl apply -f upgrade_v1alpha1_os.yaml + ``` + +4. Check the OS version of the nodes again to confirm if the upgrade is complete. + + ```shell + kubectl get nodes -o custom-columns='NAME:.metadata.name,OS:.status.nodeInfo.osImage' + ``` + +5. If you need to perform the upgrade again, modify the corresponding fields in **upgrade_v1alpha1_os.yaml**. + +> ![](./public_sys-resources/icon-note.gif)**Note:** +> +> If you need to perform the upgrade again, modify the `imageurl`, `osversion`, `checksum`, `maxunavailable`, `flagSafe`, or `dockerimage` parameters in **upgrade_v1alpha1_os.yaml**. + +## Settings + +- Settings parameters + + This section describes the configuration parameters using an example YAML file. Your configuration should follow the same indentation as the example: + + ```yaml + apiVersion: upgrade.openeuler.org/v1alpha1 + kind: OS + metadata: + name: os-sample + spec: + imagetype: "" + opstype: config + osversion: edit.os.version + maxunavailable: edit.node.config.number + containerimage: "" + evictpodforce: false + checksum: "" + sysconfigs: + version: edit.sysconfigs.version + configs: + - model: kernel.sysctl + contents: + - key: kernel param key1 + value: kernel param value1 + - key: kernel param key2 + value: kernel param value2 + operation: delete + - model: kernel.sysctl.persist + configpath: persist file path + contents: + - key: kernel param key3 + value: kernel param value3 + - model: grub.cmdline.current + contents: + - key: boot param key1 + - key: boot param key2 + value: boot param value2 + - key: boot param key3 + value: boot param value3 + operation: delete + - model: grub.cmdline.next + contents: + - key: boot param key4 + - key: boot param key5 + value: boot param value5 + - key: boot param key6 + value: boot param value6 + operation: delete + ``` + + Configuration parameters + + | Parameter | Type | Description | Usage Note | Mandatory | + | ---------- | -------- | --------------------------- | ------------------------------------------------------------ | ----------------------- | + | `version` | string | Configuration version | This parameter determines if the configuration should be applied by comparing versions. If `version` is empty (`""` or not set), the comparison will still be performed. Therefore, if `sysconfigs` or `upgradeconfigs` is not configured, the existing `version` will be cleared, triggering the configuration. | Yes | + | `configs` | / | Specific configuration content | This parameter contains a list of specific configuration items. | Yes | + | `model` | string | Configuration type | See the [Settings List](#settings-list) in the appendix for supported configuration types. | Yes | + | `configpath` | string | Configuration file path | This parameter is only effective for the `kernel.sysctl.persist` configuration type. See the [Settings List](#settings-list) in the appendix for the description of the configuration file path. | No | + | `contents` | / | Specific key/value pairs and operation type | This parameter contains a list of specific configuration items. | Yes | + | `key` | string | Parameter name | `key` cannot be empty or contain `=`. You are advised not to configure strings containing spaces or tabs. For specific usage of `key` for each configuration type, see the [Settings List](#settings-list) in the appendix. | Yes | + | `value` | string | Parameter value | `value` cannot be empty for parameters in the `key=value` format. You are advised not to configure strings containing spaces or tabs. For specific usage of `value` for each configuration type, see the [Settings List](#settings-list) in the appendix. | Required for parameters in the `key=value` format | + | `operation` | string | Operation to be performed on the parameter | This parameter is only effective for `kernel.sysctl.persist`, `grub.cmdline.current`, and `grub.cmdline.next` parameter types. The default behavior is to add or update. The value can only be `delete`, which means deleting the existing parameter (the `key=value` must match exactly for deletion). | No | + + - `upgradeconfigs` has the same parameters as `sysconfigs`. `upgradeconfigs` is for configuration before upgrade/rollback and only takes effect in upgrade/rollback scenarios. `sysconfigs` supports both configuration only and configuration after upgrade/rollback reboot. + +- Usage + + 1. Create a YAML file like the **upgrade_v1alpha1_os.yaml** example above and deploy the OS CR instance in the cluster. + + 2. Check the configuration version and node status before applying the configuration (`NODESTATUS` should be `idle`). + + ```shell + kubectl get osinstances -o custom-columns='NAME:.metadata.name,NODESTATUS:.spec.nodestatus,SYSCONFIG:status.sysconfigs.version,UPGRADECONFIG:status.upgradeconfigs.version' + ``` + + 3. Apply the configuration, then check the node status again (`NODESTATUS` should change to `config`). + + ```shell + kubectl apply -f upgrade_v1alpha1_os.yaml + kubectl get osinstances -o custom-columns='NAME:.metadata.name,NODESTATUS:.spec.nodestatus,SYSCONFIG:status.sysconfigs.version,UPGRADECONFIG:status.upgradeconfigs.version' + ``` + + 4. Check the node configuration version again to confirm whether the configuration is complete (`NODESTATUS` should return to `idle`): + + ```shell + kubectl get osinstances -o custom-columns='NAME:.metadata.name,NODESTATUS:.spec.nodestatus,SYSCONFIG:status.sysconfigs.version,UPGRADECONFIG:status.upgradeconfigs.version' + ``` + +- If you need to perform the configuration again, modify the corresponding fields in **upgrade_v1alpha1_os.yaml**. + +## Rollback + +- Scenarios + - When a VM fails to start, you can manually select the previous version from the GRUB boot menu. This method only supports rollback to the previous version. + - When a VM starts successfully and you can access the system, you can use the rollback tool (recommended) or manually select the previous version from the GRUB boot menu. + - You can use the rollback tool in two ways: + 1. Rollback mode: reverts to the previous version. + 2. Upgrade mode: re-upgrades to the previous version. + +- Manual rollback instructions + - Restart the VM and select the second boot option in the GRUB boot menu to roll back to the previous version. + +- Rollback tool instructions + - Rolling back to any version + 1. Modify the YAML configuration file of the OS CR instance (for example, **upgrade_v1alpha1_os.yaml**). Set the relevant fields to the image information of the desired version. The OS category originates from the CRD object created in the installation and deployment document. Refer to the upgrade instructions in the previous section for field descriptions and examples. + 2. After modifying the YAML file, execute the update command. Nodes will then roll back according to the configured field information. + + ```shell + kubectl apply -f upgrade_v1alpha1_os.yaml + ``` + + - Rolling back to the previous version + - To roll back to the previous OS version, modify the **upgrade_v1alpha1_os.yaml** file. Set `osversion` to the previous version and `opstype` to `rollback` to roll back to the previous version (that is, switch to the previous partition). Example YAML file: + + ```yaml + apiVersion: upgrade.openeuler.org/v1alpha1 + kind: OS + metadata: + name: os-sample + spec: + imagetype: "" + opstype: rollback + osversion: KubeOS previous version + maxunavailable: 2 + containerimage: "" + evictpodforce: true/false + imageurl: "" + checksum: "" + flagSafe: false + mtls: true + ``` + + - To roll back to the previous configuration version (note that already configured parameters cannot be rolled back), modify the `upgrade_v1alpha1_os.yaml` file. Set `version` of `sysconfigs/upgradeconfigs` to the previous version. Example YAML file: + + ```yaml + apiVersion: upgrade.openeuler.org/v1alpha1 + kind: OS + metadata: + name: os-sample + spec: + imagetype: "" + opstype: config + osversion: edit.os.version + maxunavailable: edit.node.config.number + containerimage: "" + evictpodforce: true/false + imageurl: "" + checksum: "" + flagSafe: false + mtls: false + sysconfigs: + version: previous config version + configs: + - model: kernel.sysctl + contents: + - key: kernel param key1 + value: kernel param value1 + - key: kernel param key2 + value: kernel param value2 + - model: kernel.sysctl.persist + configpath: persist file path + contents: + - key: kernel param key3 + value: kernel param value3 + ``` + + - After modifying the YAML file and executing the update command, the nodes will roll back based on the configured information. + + ```shell + kubectl apply -f upgrade_v1alpha1_os.yaml + ``` + + - Verify that the rollback was successful. + - To check the container OS version (for OS version rollback), verify container OS version of the node. To check the configuration version (for configuration rollback), verify the node configuration version and that the node status is `idle`. + + ```shell + kubectl get osinstances -o custom-columns='NAME:.metadata.name,NODESTATUS:.spec.nodestatus,SYSCONFIG:status.sysconfigs.version,UPGRADECONFIG:status.upgradeconfigs.version' + ``` + +## Appendixes + +### Settings List + +#### kernel Settings + +- `kernel.sysctl`: temporarily sets kernel parameters. These settings will be lost after a reboot. The key/value pairs represent key/value pairs of kernel parameters. Both keys and values cannot be empty. Keys cannot contain the `=` character. The value of `operation` cannot be `delete`. Example: + + ```yaml + configs: + - model: kernel.sysctl + contents: + - key: user.max_user_namespaces + value: 16384 + - key: net.ipv4.tcp_tw_recycle + value: 0 + operation: delete + ``` + +- `kernel.sysctl.persist`: sets persistent kernel parameters that will be retained after a reboot. The key/value pairs represent key/value pairs of kernel parameters. Both keys and values cannot be empty. Keys cannot contain the `=` character. `configpath` specifies the path to the configuration file, which can be a new file (given that the parent directory exists). If not specified, it defaults to **/etc/sysctl.conf**. Example: + + ```yaml + configs: + - model: kernel.sysctl.persist + configpath : /etc/persist.conf + contents: + - key: user.max_user_namespaces + value: 16384 + - key: net.ipv4.tcp_tw_recycle + value: 0 + operation: delete + ``` + +#### GRUB Settings + +- `grub.cmdline.current/next`: sets the kernel boot parameters in the **grub.cfg** file. These parameters appear on the line resembling the following example in **grub.cfg**: + + ```text + linux /boot/vmlinuz root=/dev/sda2 ro rootfstype=ext4 nomodeset quiet oops=panic softlockup_panic=1 nmi_watchdog=1 rd.shell=0 selinux=0 crashkernel=256M panic=3 + ``` + + - The `grub.cmdline.current/next` settings allow configuration for either the current or the next partition: + + - `grub.cmdline.current`: Configures the boot parameters for the current partition. + - `grub.cmdline.next`: Configures the boot parameters for the next partition. + + - Note: During upgrades/rollbacks, the `current` and `next` partition designations in the configuration (`sysconfigs`) are determined at the time the upgrade/rollback operation is initiated. For instance, if the current partition is `A` and an upgrade is initiated with `grub.cmdline.current` configured in `sysconfigs`, the configuration will still be applied to partition `A` after the reboot, even though it may no longer be the `current` partition. + + - `grub.cmdline.current/next` supports both `key=value` (where `value` cannot be empty) and single `key` formats. If `value` contains an equal sign (for example, `root=UUID=some-uuid`), `key` should be set to all characters before the first `=` and `value` should be set to all characters after the first `=`. Example: + + ```yaml + configs: + - model: grub.cmdline.current + contents: + - key: selinux + value: "0" + - key: root + value: UUID=e4f1b0a0-590e-4c5f-9d8a-3a2c7b8e2d94 + - key: panic + value: "3" + operation: delete + - key: crash_kexec_post_notifiers + - model: grub.cmdline.next + contents: + - key: selinux + value: "0" + - key: root + value: UUID=e4f1b0a0-590e-4c5f-9d8a-3a2c7b8e2d94 + - key: panic + value: "3" + operation: delete + - key: crash_kexec_post_notifiers + ``` diff --git a/docs/en/cloud/nestos/nestos/_toc.yaml b/docs/en/cloud/nestos/nestos/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..cd9dd95d286a62f244f202636ab1ba8fbc1db939 --- /dev/null +++ b/docs/en/cloud/nestos/nestos/_toc.yaml @@ -0,0 +1,12 @@ +label: NestOS User Guide +isManual: true +description: >- + NestOS is a lightweight OS optimized for containerized environments. It + employs dual-partition atomic updates to maintain security and reliability. +sections: + - label: Overview + href: ./overview.md + - label: NestOS for Container User Guide + href: ./nestos-for-container.md + - label: Feature Description + href: ./feature-description.md diff --git a/docs/en/cloud/nestos/nestos/feature-description.md b/docs/en/cloud/nestos/nestos/feature-description.md new file mode 100644 index 0000000000000000000000000000000000000000..f53f9cf28af8028c6aba38da47cee88d52905e27 --- /dev/null +++ b/docs/en/cloud/nestos/nestos/feature-description.md @@ -0,0 +1,100 @@ +# Feature Description + +## Container Technology + +NestOS provides computing resources for applications using a containerized computing environment. Applications share a system kernel and resources, but are invisible to each other. This means that applications are no longer directly installed in the OS. Instead, they run in containers through container engines (for example, Docker, PadMan, and iSula). This greatly reduces the coupling among the OS, applications, and running environment. Compared with the traditional application deployment mode, the NestOS cluster provides more flexible and convenient application deployment, less interference between application running environments , and the easier maintenance of OSs. + +## rpm-ostree + +### System Upgrade + +rpm-ostree is a hybrid image/package system that combines RPM and OSTree. It provides RPM-based software package installation and management, and OSTree-based OS update and upgrade. rpm-ostree sees the two operations as updates to the OS. Each update to the system is similar to a transaction submitted by rpm-ostree. This ensures that the update completely succeeds or fails completely and allows the system to be rolled back to the status before the update. + +When updating the OS, rpm-ostree keeps two bootable deployments: the active deployment and standby deployment. The update is performed to the standby deployment and takes effect only after the OS is restarted and the active and standby deployments are switched over. If an error occurs during software installation or upgrade, the rpm-ostree rollback allows NestOS to revert to the previous deployment. The **/ostree/** and **/boot/** directories of NestOS are the OSTree repository environment and show which OSTree deployment is booted into. + +### Read-Only File System + +In the rpm-ostree file system layout, only the **/etc** and **/var** directories are writable. Any data in the **/var** directory is not touched and is shared across upgrades. During the system upgrade, rpm-ostree takes the new default **/etc** and adds the changes on the top. This means that the upgrades will receive new default files in **/etc**, which is a critical feature. + +In a common OS, some files in the **/var** directory adopt the processing policy "/var to tmpfiles.d". That is, the system reads the configuration files in the **/usr/lib/tmpfiles.d/** directory through systemd-tmpfile-setup.service and creates folders and blank files in the **/var** directory. All configuration files in the **/usr/lib/tmpfiles.d/** directory are provided by related RPM packages. In NestOS, the **/var** directory does not involve the commit layer of rpm-ostree. Each commit layer of rpm-ostree shares a **/var** directory. However, files in the **/var** directory conflict with the ostree transaction update model. As a result, when the software package is installed, rpm-ostree deletes those files. The content in the **/var** directory is generated based on "/var to tmpfiles.d". Therefore, in the **/usr/lib/tmpfiles.d/** directory of the NestOS, besides the configuration files provided by some RPM packages, there is the **pkg-XXX.conf** file generated by rpm-ostree during the installation of the XXX software package (even if the XXX has provided a configuration file). **pkg-XXX.conf** records the folders in the **/var** directory provided by the XXX software package but not the files, as they are deleted when rpm-ostree installs the package. When you need to perform operations on a folder in the **/var** directory provided by the RPM package, for example, deleting a folder, you can only delete the folder temporarily by running the `rm` command. After the system is restarted, the folder still exists. You need to modify the **pkg-XXX.conf** file to permanently delete the folder. + +OSTree is designed to parallel-install multiple versions of multiple independent operating systems. OSTree relies on a new top-level **ostree** directory; it can in fact parallel install inside an existing OS or distribution occupying the physical **/root**. On each client machine, there is an OSTree repository stored in **/ostree/repo**, and a set of deployments stored in **/ostree/deploy/$STATEROOT/$CHECKSUM**. Each deployment is primarily composed of a set of hard links into the repository. This means each version is deduplicated; an upgrade process only costs disk space proportional to the new files, plus some constant overhead. + +The model OSTree emphasizes is that the OS read-only content is kept in **/usr**; it comes with code to create a Linux read-only bind mount to prevent inadvertent corruption. There is exactly one **/var** writable directory shared between each deployment for a given OS. The OSTree core code does not touch content in this directory; it is up to the code in each operating system for how to manage and upgrade state. + +### OS Extensions + +NestOS keeps the base image as simple and small as possible for security and maintainability reasons. However, in some cases it is necessary to add software to the base OS itself. For example, drivers or VPN software are potential candidates because they are harder to containerize. These software packages extend the functionality of the base OS rather than providing runtimes for user applications. For this reason, rpm-ostree treats these packages as extensions. That said, there are no restrictions on which packages you can actually install. By default, packages are downloaded from the openEuler repositories. + +To layer a software package, you need to write a systemd unit that executes the `rpm-ostree` command to install the wanted package. The changes are added to a new deployment, which takes effect after restart. + +## nestos-installer + +nestos-installer helps with NestOS installation. It provides the following functions: + +(1) Installing the OS to a target disk, optionally customizing it with an Ignition configuration or first-boot kernel parameters (`nestos-installer install`) + +(2) Downloading and verify an OS image for various cloud, virtualization, or bare metal platforms (`nestos-installer download`) + +(3) Listing NestOS images available for download (`nestos-installer list-stream`) + +(4) Embed an Ignition configuration in a live ISO image to customize the running system that boots from it (`nestos-installer iso ignition`) + +(5) Wrap an Ignition configuration in an initrd image that can be appended to the live PXE initramfs to customize the running system that boots from it (`nestos-installer pxe ignition`) + +## Zincati + +Zincati is an auto-update agent for NestOS hosts. It works as a client for the Cincinnati service, responsible for monitoring NestOS version changes and automatically updating machines using rpm-ostree. Zincati has the following features: + +(1) Agent for continuous automatic updates, with support for phased rollouts + +(2) Runtime customization via TOML dropins, allowing users to overwrite the default configuration. + +(3) Multiple update strategies + +(4) Local maintenance windows on a weekly schedule for planned upgrades + +(5) Tracks and exposes Zincati internal metrics to Prometheus to ease monitoring tasks across a large fleet of nodes + +(6) Logging with configurable priority levels + +(7) Support for complex update-graphs via Cincinnati protocol + +(8) Support for cluster-wide reboot orchestration, via an external lock-manager + +## System Initialization (Ignition) + +Ignition is a distribution-agnostic provisioning utility that not only installs, but also reads configuration files (in JSON format) to initialize NestOS. Configurable components include storage and file systems, systemd units, and users. + +Ignition runs only once during the first boot of the system (while in the initramfs). Because Ignition runs so early in the boot process, it can re-partition disks, format file systems, create users, and write files before the userspace begins to boot. As a result, systemd services are already written to disk when systemd starts, speeding the time to boot. + +(1) Ignition runs only on the first boot +Ignition is designed to be used as a provisioning tool, not as a configuration management tool. Ignition encourages immutable infrastructure, in which machine modification requires that users discard the old node and re-provision the machine. + +(2) Ignition produces the machine specified or no machine at all +Ignition does what it needs to make the system match the state described in the Ignition configuration. If for any reason Ignition cannot deliver the exact machine that the configuration asked for, Ignition prevents the machine from booting successfully. For example, if the user wanted to fetch the document hosted at **** and write it to disk, Ignition would prevent the machine from booting if it were unable to resolve the given URL. + +(3) Ignition configurations are declarative +Ignition configurations describe the state of a system. Ignition configurations do not list a series of steps that Ignition should take. +Ignition configurations do not allow users to provide arbitrary logic (including scripts for Ignition to run). Users describe which file systems must exist, which files must be created, which users must exist, and more. Any further customization must use systemd services, created by Ignition. + +(4) Ignition configurations should not be written by hand +Ignition configurations were designed to be human readable, but difficult to write, to discourage users from attempting to write configs by hand. Use Butane, or a similar tool, to generate Ignition configurations. + +## Afterburn + +Afterburn is a one-shot agent for cloud-like platforms which interacts with provider-specific metadata endpoints. It is typically used in conjunction with Ignition. + +Afterburn comprises several modules which may run at different times during the lifecycle of an instance. Depending on the specific platform, the following services may run in the initramfs on first boot: + +- setting local hostname + +- injecting network command-line arguments + +The following features are conditionally available on some platforms as systemd service units: + +- installing public SSH keys for local system users + +- retrieving attributes from instance metadata + +- checking in to the provider in order to report a successful boot or instance provisioning diff --git a/docs/en/cloud/nestos/nestos/figures/figure1.png b/docs/en/cloud/nestos/nestos/figures/figure1.png new file mode 100644 index 0000000000000000000000000000000000000000..b4eb9017ed202e854c076802492d8561942dfc88 Binary files /dev/null and b/docs/en/cloud/nestos/nestos/figures/figure1.png differ diff --git a/docs/en/cloud/nestos/nestos/figures/figure2.png b/docs/en/cloud/nestos/nestos/figures/figure2.png new file mode 100644 index 0000000000000000000000000000000000000000..90049769c04e2bd494533da1613e38a5199da3d7 Binary files /dev/null and b/docs/en/cloud/nestos/nestos/figures/figure2.png differ diff --git a/docs/en/cloud/nestos/nestos/nestos-for-container.md b/docs/en/cloud/nestos/nestos/nestos-for-container.md new file mode 100644 index 0000000000000000000000000000000000000000..3ebf0215316b9256ce3ea53b3b111bc9b98e1631 --- /dev/null +++ b/docs/en/cloud/nestos/nestos/nestos-for-container.md @@ -0,0 +1,985 @@ +# NestOS for Container User Guide + +## 1. Introduction to NestOS + +### 1.1 Overview + +NestOS, developed by KylinSoft and incubated in the openEuler community, is a cloud-native OS designed for modern infrastructure. It incorporates advanced technologies like rpm-ostree support and Ignition configuration, featuring a dual-root file system with mutual backup and atomic update capabilities. The system also includes the nestos-assembler tool for streamlined integration and building. Optimized for Kubernetes and OpenStack platforms, NestOS minimizes container runtime overhead, enabling efficient cluster formation and secure operation of large-scale containerized workloads. + +This guide provides a comprehensive walkthrough of NestOS, covering its building, installation, deployment, and usage. It aims to help users maximize the system benefits for rapid and efficient configuration and deployment. + +### 1.2 Application Scenarios and Advantages + +NestOS serves as an ideal foundation for cloud environments centered around containerized applications. It resolves challenges such as fragmented operation and maintenance (O&M) practices and redundant platform development, which arise from the decoupling of container and orchestration technologies from the underlying infrastructure. By ensuring alignment between application services and the base OS, NestOS delivers consistent and streamlined O&M. + +![figure1](./figures/figure1.png) + +## 2. Environment Preparation + +### 2.1 Build Environment Requirements + +#### 2.1.1 Requirements for Building the nestos-assembler Tool + +- Use openEuler for optimal results. +- Ensure at least 5 GB of available drive space. + +#### 2.1.2 Requirements for Building NestOS + +| Category | Requirements | +| :----------: | :---------------------: | +| CPU | 4 vCPUs | +| Memory | 4 GB | +| Drive | Available space > 10 GB | +| Architecture | x86_64 or AArch64 | +| Others | Support for KVM | + +### 2.2 Deployment Configuration Requirements + +| Category | Recommended Configuration | Minimum Configuration | +| :----------: | :-----------------------: | :-------------------: | +| CPU | > 4 vCPU | 1 vCPU | +| Memory | > 4GB | 512 MB | +| Drive | > 20GB | 10 GB | +| Architecture | x86_64, aarch64 | / | + +## 3. Quick Start + +### 3.1 Quick Build + +(1) Obtain the nestos-assembler container image. + +You are advised the openEuler-based base image. For additional details, see [Section 6.1](#61-nestos-assembler-container-image-creation). + +```shell +docker pull hub.oepkgs.net/nestos/nestos-assembler:24.03-LTS.20240903.0-aarch64 +``` + +(2) Create a script named `nosa` and save it to `/usr/local/bin`, then make it executable. + +```shell +#!/bin/bash + +sudo docker run --rm -it --security-opt label=disable --privileged --user=root \ + -v ${PWD}:/srv/ --device /dev/kvm --device /dev/fuse --network=host \ + --tmpfs /tmp -v /var/tmp:/var/tmp -v /root/.ssh/:/root/.ssh/ -v /etc/pki/ca-trust/:/etc/pki/ca-trust/ \ + ${COREOS_ASSEMBLER_CONFIG_GIT:+-v $COREOS_ASSEMBLER_CONFIG_GIT:/srv/src/config/:ro} \ + ${COREOS_ASSEMBLER_GIT:+-v $COREOS_ASSEMBLER_GIT/src/:/usr/lib/coreos-assembler/:ro} \ + ${COREOS_ASSEMBLER_CONTAINER_RUNTIME_ARGS} \ + ${COREOS_ASSEMBLER_CONTAINER:-nestos-assembler:your_tag} "$@" +``` + +Note: Replace the value of `COREOS_ASSEMBLER_CONTAINER` with the actual nestos-assembler container image in your environment. + +(3) Obtain nestos-config. + +Use `nosa init` to initialize the build workspace, pull the build configuration, and create the `nestos-build` directory. Run the following command in this directory: + +```shell +nosa init https://gitee.com/openeuler/nestos-config +``` + +(4) Adjust build configurations. + +nestos-config provides default build configurations, so no additional steps are required. For customization, refer to [Section 5](#5-build-configuration-nestos-config). + +(5) Build NestOS images. + +```shell +# Pull build configurations and update cache. +nosa fetch +# Generate root file system, qcow2, and OCI images. +nosa build +# Generate live ISO and PXE images. +nosa buildextend-metal +nosa buildextend-metal4k +nosa buildextend-live +``` + +For detailed build and deployment steps, refer to [Section 6](#6-build-process). + +### 3.2 Quick Deployment + +Using the NestOS ISO image as an example, boot into the live environment and execute the following command to complete the installation by following the wizard: + +```shell +sudo installnestos +``` + +For alternative deployment methods, see [Section 8](#8-deployment-process). + +## 4. Default Configuration + +| Item | Default Configuration | +| :-------------------------: | :----------------------------------------------: | +| Docker service | Disabled by default, requires manual activation. | +| SSH service security policy | Supports only key-based login by default. | + +## 5. Build Configuration: nestos-config + +### 5.1 Obtaining Configuration + +The repository for nestos-config is located at + +### 5.2 Directory Structure Explanation + +| Directory/File | Description | +| :---------------: | :------------------------------------: | +| live/* | Boot configuration for live ISO builds | +| overlay.d/* | Custom file configurations | +| tests/* | User-defined test case configurations | +| *.repo | Repository configurations | +| .yaml, manifests/ | Main build configurations | + +### 5.3 Key Files + +#### 5.3.1 .repo Files + +.repo files in the directory are used to configure software repositories for building NestOS. + +#### 5.3.2 YAML Configuration Files + +YAML files in the directory provide various configurations for NestOS builds. For details, refer to [Section 5.4](#54-key-fields). + +### 5.4 Key Fields + +| Field | Purpose | +| :------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------- | +| packages-aarch64, packages-x86_64, packages | Scope of software package integration | +| exclude-packages | Blocklist for software package integration | +| remove-from-packages | Files/folders to remove from specified packages | +| remove-files | Files/folders to remove | +| extra-kargs | Additional kernel boot parameters | +| initramfs-args | Initramfs build parameters | +| postprocess | Post-build scripts for the file system | +| default-target | Default target, such as **multi-user.target** | +| rojig.name, releasever | Image-related information (name and version) | +| lockfile-repos | List of repository names available for builds, which must match the repository names in the repo files described in [Section 5.3.1](#531-repo-files) | + +### 5.5 Configurable Items + +#### 5.5.1 Repository Configuration + +(1) Edit the .repo file in the configuration directory and modify its content to the desired software repositories. + +```shell +$ vim nestos-pool.repo +[repo_name_1] +Name=xxx +baseurl = https://ip.address/1 +enabled = 1 + +[repo_name_2] +Name=xxx +baseurl = https://ip.address/2 +enabled = 1 +``` + +(2) Modify the `lockfile-repos` field in the YAML configuration file to include the corresponding repository names. + +Note: The repository name is the content inside `[]` in the repo file, not the `name` field. + +```shell +$ vim manifests/rpmlist.yaml +Modify the `lockfile-repos` field as follows: +lockfile-repos: +- repo_name_1 +- repo_name_2 +``` + +#### 5.5.2 Software Package Customization + +Modify the `packages`, `packages-aarch64`, and `packages-x86_64` fields to add or remove software packages. + +For example, adding `nano` to the `packages` field ensures that the system includes `nano` after installation. + +```shell +$ vim manifests/rpmlist.yaml +packages: +- bootupd +... +- authselect +- nano +... +packages-aarch64: +- grub2-efi-aa64 +packages-x86_64: +- microcode_ctl +- grub2-efi-x64 +``` + +#### 5.5.3 Image Name and Version Customization + +Modify the `releasever` and `rojig.name` fields in the YAML file to control the image version and name. + +```shell +$ vim manifest.yaml + +releasever: "1.0" +rojig: + license: MIT + name: nestos + summary: NestOS stable +``` + +With the above configuration, the built image format will be **nestos-1.0.$(date "+%Y%m%d").$build_num.$type**, where **build_num** is the build count and **type** is the type suffix. + +#### 5.5.4 Image Release Information Customization + +Normally, release information is provided by the integrated release package (e.g., `openeuler-release`). However, you can rewrite the **/etc/os-release** file by adding a **postprocess** script. + +```shell +$ vim manifests/system-configuration.yaml +# Add the following content to postprocess. If the content already exists, simply modify the corresponding release information. +postprocess: + - | + #!/usr/bin/env bash + set -xeuo pipefail + export OSTREE_VERSION="$(tail -1 /etc/os-release)" + date_now=$(date "+%Y%m%d") + echo -e 'NAME="openEuler NestOS"\nVERSION="24.03-LTS"\nID="openeuler"\nVERSION_ID="24.03-LTS"\nPRETTY_NAME="NestOS"\nANSI_COLOR="0;31"\nBUILDID="'${date_now}'"\nVARIANT="NestOS"\nVARIANT_ID="nestos"\n' > /usr/lib/os-release + echo -e $OSTREE_VERSION >> /usr/lib/os-release + cp -f /usr/lib/os-release /etc/os-release +``` + +#### 5.5.5 Custom File Creation + +Add or modify custom files in the **overlay.d** directory. This allows for customization of the image content. + +```shell +mkdir -p overlay.d/15nestos/etc/test/test.txt +echo "This is a test message !" > overlay.d/15nestos/etc/test/test.txt +``` + +Using the above configuration to build the image. After image boot, the content of the corresponding file in the system will match the custom content added above. + +```shell +[root@nosa-devsh ~]# cat /etc/test/test.txt +This is a test message ! +``` + +## 6. Build Process + +NestOS employs a containerized method to bundle the build toolchain into a comprehensive container image called nestos-assembler. + +NestOS enables users to create the nestos-assembler container image, simplifying the process of building various NestOS image formats in any Linux distribution environment, such as within existing CI/CD pipelines. Additionally, users can manage, debug, and automate testing of build artifacts using this image. + +### 6.1 nestos-assembler Container Image Creation + +#### 6.1.1 Prerequisites + +1. Prepare the base container image. + + The nestos-assembler container image must be based on a base image that supports the Yum or DNF package manager. Although it can be created from any distribution base image, using an openEuler base image is recommended to reduce software compatibility issues. + +2. Install required software packages. + + Install Docker, the essential dependency: + + ```shell + dnf install -y docker + ``` + +3. Clone the nestos-assembler source code repository. + +```shell +git clone --depth=1 --single-branch https://gitee.com/openeuler/nestos-assembler.git +``` + +#### 6.1.2 Building the nestos-assembler Container Image + +Using the openEuler container image as the base, build the image with the following command: + +```shell +cd nestos-assembler/ +docker build -f Dockerfile . -t nestos-assembler:your_tag +``` + +### 6.2 nestos-assembler Container Image Usage + +#### 6.2.1 Prerequisites + +1. Prepare the nestos-assembler container image. + + Once the nestos-assembler container image is built following [Section 6.1](#61-nestos-assembler-container-image-creation), it can be managed and distributed via a privately hosted container image registry. Ensure the correct version of the nestos-assembler container image is pulled before initiating the NestOS build. + +2. Create the nosa script. + + To streamline user operations, you can write a `nosa` command script. This is particularly useful as the NestOS build process involves multiple calls to the nestos-assembler container image for executing various commands and configuring numerous parameters. For quick build details, see [Section 3.1](#31-quick-build). + +#### 6.2.2 Usage Instructions + +nestos-assembler commands + +| Command | Description | +| :-------------------: | :-------------------------------------------------------------------------------------: | +| init | Initialize the build environment and configuration. See [Section 6.3](#63-build-environment-preparation) for details. | +| fetch | Fetch the latest software packages to the local cache based on the build configuration. | +| build | Build the ostree commit, which is the core command for building NestOS. | +| run | Directly start a QEMU instance, using the latest build version by default. | +| prune | Clean up historical build versions, retaining the latest three versions by default. | +| clean | Delete all build artifacts. Use the `--all` parameter to also clean the local cache. | +| list | List the versions and artifacts present in the current build environment. | +| build-fast | Quickly build a new version based on the previous build record. | +| push-container | Push the container image artifact to the container image registry. | +| buildextend-live | Build ISO artifacts and PXE images that support the live environment. | +| buildextend-metal | Build raw artifacts for bare metal. | +| buildextend-metal4k | Build raw artifacts for bare metal in native 4K mode. | +| buildextend-openstack | Build QCOW2 artifacts for the OpenStack platform. | +| buildextend-qemu | Build QCOW2 artifacts for QEMU. | +| basearch | Retrieve the current architecture information. | +| compress | Compress artifacts. | +| kola | Automated testing framework | +| kola-run | A wrapper for automated testing that outputs summarized results | +| runc | Mount the current build root file system in a container. | +| tag | Manage build project tags. | +| virt-install | Create an instance for the specified build version. | +| meta | Manage build project metadata. | +| shell | Enter the nestos-assembler container image. | + +### 6.3 Build Environment Preparation + +The NestOS build environment requires a dedicated empty folder as the working directory, supporting multiple builds while preserving and managing historical versions. Before setting up the build environment, ensure the build configuration is prepared (see [Section 5](#5-build-configuration-nestos-config)). + +You are advised to maintain a separate build configuration for each independent build environment. If you plan to build NestOS for various purposes, maintain multiple build configurations and their corresponding directories. This approach allows independent evolution of configurations and clearer version management. + +#### 6.3.1 Initializing the Build Environment + +Navigate to the target working directory and run the following command to initialize the build environment: + +```shell +nosa init https://gitee.com/openeuler/nestos-config +``` + +Initialization is only required for the first build. Subsequent builds can reuse the same environment unless significant changes are made to the build configuration. + +#### 6.3.2 Build Environment Structure + +After initialization, the following folders are created in the working directory: + +**builds**: stores build artifacts and metadata. The **latest** subdirectory is a symbolic link to the most recent build version. + +**cache**: contains cached data pulled from software sources and package lists specified in the build configuration. Historical NestOS ostree repositories are also stored here. + +**overrides**: used to place files or RPM packages that should be added to the rootfs of the final artifact during the build process. + +**src**: holds the build configuration, including nestos-config-related content. + +**tmp**: used during builds and automated testing. In case of errors, you can inspect VM CLI outputs, journal logs, and other debugging information here. + +### 6.4 Build Steps + +The primary steps and reference commands for building NestOS are outlined below. + +![figure2](./figures/figure2.png) + +#### 6.4.1 Initial Build + +For the initial build, the build environment must be initialized. Refer to [Section 6.3](#63-build-environment-preparation) for detailed instructions. + +For subsequent builds, the existing build environment can be reused. Use `nosa list` to check the current versions and corresponding artifacts in the build environment. + +#### 6.4.2 Updating Build Configuration and Cache + +After initializing the build environment, run the following command to update the build configuration and cache: + +```shell +nosa fetch +``` + +This step validates the build configuration and pulls software packages from the configured sources to the local cache. When the build configuration changes or you want to update to the latest software versions, repeat this step. Otherwise, the build may fail or produce unexpected results. + +If significant changes are made to the build configuration and you want to clear the local cache and re-fetch, use: + +```shell +nosa clean --all +``` + +#### 6.4.3 Building the Immutable Root File system + +The core of NestOS, an immutable OS, is its immutable root file system based on ostree technology. Run the following command to build the ostree file system: + +```shell +nosa build +``` + +By default, the `build` command generates the ostree file system and an OCI archive. You can also include `qemu`, `metal`, or `metal4k` to simultaneously build the corresponding artifacts, equivalent to running `buildextend-qemu`, `buildextend-metal`, and `buildextend-metal4k` afterward. + +```shell +nosa build qemu metal metal4k +``` + +To add custom files or RPM packages during the NestOS build, place them in the **rootfs/** or **rpm/** folders under the **overrides** directory before running the `build` command. + +#### 6.4.4 Building Various Artifacts + +After running the `build` command, you can use `buildextend` commands to build different types of artifacts. Details are as follows. + +- Building QCOW2 images: + +```shell +nosa buildextend-qemu +``` + +- Building ISO images with a live environment or PXE boot components: + +```shell +nosa buildextend-metal +nosa buildextend-metal4k +nosa buildextend-live +``` + +- Building QCOW2 images for the OpenStack environment: + +```shell +nosa buildextend-openstack +``` + +- Building container images for container-based updates: + +When the `nosa build` command is executed, an OCI archive format image is also generated. This image can be pushed to a local or remote image registry directly. + +```shell +nosa push-container [container-image-name] +``` + +The remote image registry address must be appended to the container image name, and no `:` should appear except in the tag. If no `:` is detected, the command generates a tag in the format `{latest_build}-{arch}`. Example: + +```shell +nosa push-container registry.example.com/nestos:1.0.20240903.0-x86_64 +``` + +This command supports the following options: + +`--authfile`: specifies the authentication file for logging into the remote image registry. + +`--insecure`: bypasses SSL/TLS verification for self-signed certificates. + +`--transport`: specifies the target image push protocol. The default is `docker`. Supported options: + +- `containers-storage`: pushes to the local storage directory of container engines like Podman and CRIO. +- `dir`: pushes to a specified local directory. +- `docker`: pushes to a private or remote container image registry using the Docker API. +- `docker-archive`: exports an archive file for use with `docker load`. +- `docker-daemon`: pushes to the local storage directory of the Docker container engine. + +### 6.5 Artifacts Acquisition + +Once the build process is complete, the artifacts are stored in the following directory within the build environment: + +```text +builds/{version}/{arch}/ +``` + +For convenience, if you are only interested in the latest build version or are using CI/CD, a **latest** directory symbol link points to the most recent version directory: + +```text +builds/latest/{arch}/ +``` + +To reduce the size of the artifacts for easier transfer, you can compress them using the following command: + +```shell +nosa compress +``` + +Note that compression removes the original files, which may disable some debugging commands. To restore the original files, use the decompression command: + +```shell +nosa uncompress +``` + +### 6.6 Build Environment Maintenance + +Before or after setting up the NestOS environment, you may need to address specific requirements. The following commands are recommended for resolving these issues. + +#### 6.6.1 Cleaning Up Historical or Invalid Build Versions to Free Drive Space + +To clean up historical build versions, run: + +```shell +nosa prune +``` + +To delete all artifacts in the current build environment, run: + +```shell +nosa clean +``` + +If the build configuration has changed software repositories or historical caches are no longer needed, you can completely clear the current build environment cache: + +```shell +nosa clean --all +``` + +#### 6.6.2 Temporarily Running a Build Version Instance for Debugging or Verification + +```shell +nosa run +``` + +Use `--qemu-image` or `--qemu-iso` to specify the boot image address. For additional parameters, refer to `nosa run --help`. + +Once the instance starts, the build environment directory is mounted to **/var/mnt/workdir**, allowing access to the build environment. + +#### 6.6.3 Running Automated Tests + +```shell +nosa kola run +``` + +This command runs predefined test cases. You can also append a specific test case name to execute it individually. + +```shell +nosa kola testiso +``` + +This command performs installation and deployment tests for ISO or PXE live environments, acting as a smoke test for the build process. + +#### 6.6.4 Debugging and Verifying netsos-assembler + +```shell +nosa shell +``` + +This command launches a shell environment within the build toolchain container, enabling you to verify the functionality of the build toolchain environment. + +## 7. Deployment Configuration + +### 7.1 Introduction + +Before you deploy NestOS, it is essential to understand and prepare the necessary configurations. NestOS offers flexible configuration options through Ignition files, which can be managed using Butane. This simplifies automated deployment and environment setup for users. + +This section provides a detailed overview of Butane functionality and usage, along with configuration examples for various scenarios. These configurations will help you quickly set up and run NestOS, ensuring system security and reliability while meeting application needs. Additionally, we will explore how to customize images by pre-integrating Ignition files, enabling efficient configuration and deployment for specific use cases. + +### 7.2 Introduction to Butane + +Butane is a tool that converts human-readable YAML configuration files into NestOS Ignition files. It simplifies the process of writing complex configurations by allowing users to create configuration files in a more readable format, which are then converted into JSON format suitable for NestOS. + +NestOS has adapted Butane by adding support for the `nestos` variant and configuration specification version `v1.0.0`, corresponding to the Ignition configuration specification `v3.3.0`. This ensures configuration stability and compatibility. + +### 7.3 Butane Usage + +To install the Butane package, use the following command: + +```shell +dnf install butane +``` + +Edit **example.yaml** and execute the following command to convert it into an Ignition file **example.ign**. The process of writing YAML files will be explained in detail later: + +```shell +butane example.yaml -o example.ign -p +``` + +### 7.4 Supported Functional Scenarios + +The following configuration examples (**example.yaml**) briefly describe the main functional scenarios and advanced usage methods supported by NestOS. + +#### 7.4.1 Configuring Users, Groups, Passwords, and SSH Keys + +```YAML +variant: nestos +version: 1.0.0 +passwd: + users: + - name: nest + ssh_authorized_keys: + - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHn2eh... + - name: jlebon + groups: + - wheel + ssh_authorized_keys: + - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC5QFS... + - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIveEaMRW... + - name: miabbott + groups: + - docker + - wheel + password_hash: $y$j9T$aUmgEDoFIDPhGxEe2FUjc/$C5A... + ssh_authorized_keys: + - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDTey7R... +``` + +#### 7.4.2 File Operations: Configuring Network Interfaces + +```YAML +variant: nestos +version: 1.0.0 +storage: + files: + - path: /etc/NetworkManager/system-connections/ens2.nmconnection + mode: 0600 + contents: + inline: | + [connection] + id=ens2 + type=ethernet + interface-name=ens2 + [ipv4] + address1=10.10.10.10/24,10.10.10.1 + dns=8.8.8.8; + dns-search= + may-fail=false + method=manual +``` + +#### 7.4.3 Creating Directories, Files, and Symbolic Links with Permissions + +```YAML +variant: nestos +version: 1.0.0 +storage: + directories: + - path: /opt/tools + overwrite: true + files: + - path: /var/helloworld + overwrite: true + contents: + inline: Hello, world! + mode: 0644 + user: + name: dnsmasq + group: + name: dnsmasq + - path: /opt/tools/transmogrifier + overwrite: true + contents: + source: https://mytools.example.com/path/to/archive.gz + compression: gzip + verification: + hash: sha512-00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 + mode: 0555 + links: + - path: /usr/local/bin/transmogrifier + overwrite: true + target: /opt/tools/transmogrifier + hard: false +``` + +#### 7.4.4 Writing systemd Services: Starting and Stopping Containers + +```YAML +variant: nestos +version: 1.0.0 +systemd: + units: + - name: hello.service + enabled: true + contents: | + [Unit] + Description=MyApp + After=network-online.target + Wants=network-online.target + + [Service] + TimeoutStartSec=0 + ExecStartPre=-/bin/podman kill busybox1 + ExecStartPre=-/bin/podman rm busybox1 + ExecStartPre=/bin/podman pull busybox + ExecStart=/bin/podman run --name busybox1 busybox /bin/sh -c ""trap 'exit 0' INT TERM; while true; do echo Hello World; sleep 1; done"" + + [Install] + WantedBy=multi-user.target +``` + +### 7.5 Pre-Integration of Ignition Files + +The NestOS build toolchain enables users to customize images based on specific use cases and requirements. After creating the image, nestos-installer offers various features for customizing image deployment and application, such as pre-integrating Ignition files, pre-allocating installation locations, and modifying kernel parameters. Below, we introduce the main functionalities. + +#### 7.5.1 Pre-Integration of Ignition Files into ISO Images + +Prepare the NestOS ISO image locally and install the nestos-installer package. Edit **example.yaml** and use the Butane tool to convert it into an Ignition file. In this example, we configure a simple username and password (the password must be encrypted; the example uses `qwer1234`), as shown below: + +```YAML +variant: nestos +version: 1.0.0 +passwd: + users: + - name: root + password_hash: "$1$root$CPjzNGH.NqmQ7rh26EeXv1" +``` + +After converting the YAML file into an Ignition file, execute the following command to embed the Ignition file and specify the target drive location. Replace `xxx.iso` with the local NestOS ISO image: + +```shell +nestos-installer iso customize --dest-device /dev/sda --dest-ignition example.ign xxx.iso +``` + +When installing using the ISO image with the embedded Ignition file , NestOS will automatically read the Ignition file and install it to the target drive. Once the progress bar reaches 100%, the system will automatically boot into the installed NestOS environment. Users can log in using the username and password configured in the Ignition file. + +#### 7.5.2 Pre-Integration of Ignition Files into PXE Images + +Prepare the NestOS PXE image locally. See [Section 6.5](#65-artifacts-acquisition) for details on obtaining the components. The remaining steps are the same as above. + +To simplify the process for users, nestos-installer also supports extracting PXE components from an ISO image. Execute the following command, replacing `xxx.iso` with the local NestOS ISO image: + +```shell +nestos-installer iso extract pxe xxx.iso +``` + +This will generate the following output files: + +```text +xxx-initrd.img +xxx-rootfs.img +xxx-vmlinuz +``` + +Execute the following command to pre-integrate the Ignition file and specify the target drive location: + +```shell +nestos-installer pxe customize --dest-device /dev/sda --dest-ignition example.ign xxx-initrd.img --output custom-initrd.img +``` + +Replace `xxx-initrd.img` with `custom-initrd.img` according to the PXE installation method for NestOS. After booting, NestOS will automatically read the Ignition file and install it to the target drive. Once the progress bar reaches 100%, the system will automatically boot into the installed NestOS environment. Users can log in using the username and password configured in the Ignition file. + +## 8. Deployment Process + +### 8.1 Introduction + +NestOS supports multiple deployment platforms and common deployment methods, currently focusing on QCOW2, ISO, and PXE. Compared to general-purpose OS deployments, the main difference lies in how to pass custom deployment configurations characterized by Ignition files. The following sections will introduce these methods in detail. + +### 8.2 Installation Using QCOW2 Images + +#### 8.2.1 Creating a QCOW2 Instance with QEMU + +Prepare the NestOS QCOW2 image and the corresponding Ignition file (see [Section 7](#7-deployment-configuration) for details). Execute the following commands in the terminal: + +```shell +IGNITION_CONFIG="/path/to/example.ign" +IMAGE="/path/to/image.qcow2" +IGNITION_DEVICE_ARG="-fw_cfg name=opt/com.coreos/config,file=${IGNITION_CONFIG}" + +qemu-img create -f qcow2 -F qcow2 -b ${IMAGE} my-nestos-vm.qcow2 +``` + +For the AArch64 environment, execute the following command: + +```shell +qemu-kvm -m 2048 -M virt -cpu host -nographic -drive if=virtio,file=my-nestos-vm.qcow2 ${IGNITION_DEVICE_ARG} -nic user,model=virtio,hostfwd=tcp::2222-:22 -bios /usr/share/edk2/aarch64/QEMU_EFI-pflash.raw +``` + +For the x86_64 environment, execute the following command: + +```shell +qemu-kvm -m 2048 -M pc -cpu host -nographic -drive if=virtio,file=my-nestos-vm.qcow2 ${IGNITION_DEVICE_ARG} -nic user,model=virtio,hostfwd=tcp::2222-:22 +``` + +#### 8.2.2 Creating a QCOW2 Instance with virt-install + +Assuming the libvirt service is running normally and the network uses the default subnet bound to the `virbr0` bridge, you can follow these steps to create a NestOS instance. + +Prepare the NestOS QCOW2 image and the corresponding Ignition file (see [Section 7](#7-deployment-configuration) for details). Execute the following commands in the terminal: + +```shell +IGNITION_CONFIG="/path/to/example.ign" +IMAGE="/path/to/image.qcow2" +VM_NAME="nestos" +VCPUS="4" +RAM_MB="4096" +DISK_GB="10" +IGNITION_DEVICE_ARG=(--qemu-commandline="-fw_cfg name=opt/com.coreos/config,file=${IGNITION_CONFIG}") +``` + +**Note: When using virt-install, the QCOW2 image and Ignition file must be specified with absolute paths.** + +Execute the following command to create the instance: + +```shell +virt-install --connect="qemu:///system" --name="${VM_NAME}" --vcpus="${VCPUS}" --memory="${RAM_MB}" --os-variant="kylin-hostos10.0" --import --graphics=none --disk="size=${DISK_GB},backing_store=${IMAGE}" --network bridge=virbr0 "${IGNITION_DEVICE_ARG[@]} +``` + +### 8.3 Installation Using ISO Images + +Prepare the NestOS ISO image and boot it. The first boot of the NestOS ISO image will default to the Live environment, which is a volatile memory-based environment. + +#### 8.3.1 Installing the OS to the Target Drive Using the nestos-installer Wizard Script + +1. In the NestOS live environment, follow the printed instructions upon first entry. Enter the following command to automatically generate a simple Ignition file and proceed with the installation and reboot: + + ```shell + sudo installnestos + ``` + +2. Follow the terminal prompts to enter the username and password. + +3. Select the target drive installation location. Press **Enter** to use the default option **/dev/sda**. + +4. After completing the above steps, nestos-installer will begin installing NestOS to the target drive based on the provided configuration. Once the progress bar reaches 100%, the system will automatically reboot. + +5. After rebooting, the system will automatically enter NestOS. Press **Enter** at the GRUB menu or wait 5 seconds to boot the system. Log in using the previously configured username and password. The installation is now complete. + +#### 8.3.2 Manually Installing the OS to the Target Drive Using the nestos-installer Command + +1. Prepare the Ignition file **example.ign** (see [Section 7](#7-deployment-configuration) for details). + +2. Follow the printed instructions upon first entry into the NestOS live environment. Enter the following command to begin the installation: + + ```shell + sudo nestos-installer install /dev/sda --ignition-file example.ign + ``` + + If network access is available, the Ignition file can also be retrieved via a URL, for example: + + ```shell + sudo nestos-installer install /dev/sda --ignition-file http://www.example.com/example.ign + ``` + +3. After executing the above command, nestos-installer will begin installing NestOS to the target drive based on the provided configuration. Once the progress bar reaches 100%, the system will automatically reboot. + +4. After rebooting, the system will automatically enter NestOS. Press **Enter** at the GRUB menu or wait 5 seconds to boot the system. Log in using the previously configured username and password. The installation is now complete. + +### 8.4 PXE Deployment + +The PXE installation components for NestOS include the kernel, **initramfs.img**, and **rootfs.img**. These components are generated using the `nosa buildextend-live` command (see [Section 6](#6-build-process) for details). + +1. Use the PXELINUX `KERNEL` command to specify the kernel. A simple example is as follows: + + ```shell + KERNEL nestos-live-kernel-x86_64 + ``` + +2. Use the PXELINUX `APPEND` command to specify the initrd and rootfs. A simple example is as follows: + + ```shell + APPEND initrd=nestos-live-initramfs.x86_64.img,nestos-live-rootfs.x86_64.img + ``` + + **Note: If you have pre-integrated the Ignition file into the PXE components as described in [Section 7.5](#75-pre-integration-of-ignition-files), you only need to replace it here and skip the subsequent steps.** + +3. Specify the installation location. For example, to use **/dev/sda**, append the following to the `APPEND` command: + + ```ini + nestosos.inst.install_dev=/dev/sda + ``` + +4. Specify the Ignition file, which must be retrieved over the network. Append the corresponding URL to the `APPEND` command, for example: + + ```ini + nestos.inst.ignition_url=http://www.example.com/example.ign + ``` + +5. After booting, NestOS will automatically read the Ignition file and install the OS to the target drive. Once the progress bar reaches 100%, the system will automatically boot into the installed NestOS environment. Users can log in using the username and password configured in the Ignition file. + +## 9. Basic Usage + +### 9.1 Introduction + +NestOS employs an OS packaging solution based on ostree and rpm-ostree technologies, setting critical directories to read-only mode to prevent accidental modifications to core system files and configurations. Leveraging the overlay layering concept, it allows users to manage RPM packages on top of the base ostree filesystem without disrupting the initial system architecture. Additionally, it supports building OCI-format images, enabling OS version switching at the granularity of images. + +### 9.2 SSH Connection + +For security reasons, NestOS does not support password-based SSH login by default and only allows key-based authentication. This design enhances system security by mitigating risks associated with password leaks or weak password attacks. + +The method for establishing an SSH connection using keys in NestOS is the same as in openEuler. If users need to temporarily enable password-based login, they can follow these steps: + +1. Edit the additional configuration file of the SSH service: + + ```shell + vi /etc/ssh/sshd_config.d/40-disable-passwords.conf + ``` + +2. Modify the default `PasswordAuthentication` setting as follows: + + ```shell + PasswordAuthentication yes + ``` + +3. Restart the sshd service to temporarily enable password-based SSH login. + +### 9.3 RPM Package Installation + +**Note: Immutable OS discourages installing software packages in the runtime environment. This method is provided only for temporary debugging scenarios. For service requirements that necessitate changes to the integrated package list, rebuild the OS by updating the build configuration.** + +NestOS does not support conventional package managers like Yum or DNF. Instead, it uses rpm-ostree to manage system updates and package installations. rpm-ostree combines the advantages of image-based and package-based management, allowing users to layer and manage RPM packages on top of the base OS without disrupting its initial structure. Use the following command to install an RPM package: + +```shell +rpm-ostree install +``` + +After installation, reboot the OS. The bootloader menu will display two branches, with the first branch being the latest by default: + +```shell +systemctl reboot +``` + +After rebooting, check the system package layering status to confirm that the package has been installed in the current version: + +```shell +rpm-ostree status -v +``` + +### 9.4 Version Rollback + +After an update or RPM package installation, the previous version of the OS deployment remains on the drive. If the update causes issues, users can manually roll back to a previous version using rpm-ostree. The specific process is as follows: + +#### 9.4.1 Temporary Rollback + +To temporarily roll back to a previous OS deployment, hold down the **Shift** key during system boot. When the bootloader menu appears, select the corresponding branch (by default, there are two branches; choose the other one). Before doing this, you can use the following command to view the two existing version branches in the current environment: + +```shell +rpm-ostree status +``` + +#### 9.4.2 Permanent Rollback + +To permanently roll back to a previous OS deployment, run the following command in the current version. This operation will set system deployment of the previous version as the default deployment. + +```shell +rpm-ostree rollback +``` + +Reboot to apply the changes. The default deployment option in the bootloader menu will have changed, eliminating the need for manual switching. + +```shell +systemctl reboot +``` + +## 10. Container Image-Based Updates + +### 10.1 Use Case Description + +NestOS, as a container cloud base OS based on the immutable infrastructure concept, distributes and updates the file system as a whole. This approach brings significant convenience in terms of operations and security. However, in real-world production environments, the officially released versions often fail to meet user requirements. For example, users may want to integrate self-maintained critical foundational components by default or further trim software packages to reduce system runtime overhead based on specific scenarios. Therefore, compared to general-purpose OSs, users have stronger and more frequent customization needs for NestOS. + +nestos-assembler can provide OCI-compliant container images. Beyond simply packaging and distributing the root file system, leveraging the ostree native container feature allows container cloud users to utilize familiar technology stacks. By writing a single ContainerFile (Dockerfile), users can easily build customized images for integrating custom components or subsequent upgrade and maintenance tasks. + +### 10.2 Usage + +#### 10.2.1 Customizing Images + +- Basic steps + +1. Refer to [Section 6](#6-build-process) to build the NestOS container image, and use the `nosa push-container` command to push it to a public or private container image registry. +2. Write a Containerfile (Dockerfile) as shown in the following example: + + ```dockerfile + FROM registry.example.com/nestos:1.0.20240603.0-x86_64 + + # Perform custom build steps, such as installing software or copying self-built components. + # Here, installing the strace package is used as an example. + RUN rpm-ostree install strace && rm -rf /var/cache && ostree container commit + ``` + +3. Run `docker build` or integrate it into CI/CD to build the corresponding image. + + > Note: + > 1. NestOS does not have the yum/dnf package manager. If software packages need to be installed, use the `rpm-ostree install` command to install local RPM packages or software provided in the repository. + > 2. If needed, you can also modify the software source configurations in the `/etc/yum.repo.d/` directory. + > 3. Each meaningful build command should end with `&& ostree container commit`. From the perspective of container image build best practices, it is recommended to minimize the number of RUN layers. + > 4. During the build process, non-/usr or /etc directory contents are cleaned up. Therefore, customization via container images is primarily suitable for software package or component updates. Do not use this method for system maintenance or configuration changes (e.g., adding users with `useradd`). + +#### 10.2.2 Deploying/Upgrading Images + +Assume that the container image built in the above steps is pushed as `registry.example.com/nestos:1.0.20240903.0-x86_64`. + +In an environment where NestOS is already deployed, execute the following command: + +```shell +sudo rpm-ostree rebase ostree-unverified-registry:registry.example.com/nestos:1.0.20240903.0-x86_64 +``` + +Reboot to complete the deployment of the customized version. + +After deployment is complete using the container image method, `rpm-ostree upgrade` will default to updating the source from the ostree update source to the container image address. Subsequently, you can update the container image under the same tag. Using `rpm-ostree upgrade` will detect if the remote image has been updated. If changes are detected, it will pull the latest image and complete the deployment. diff --git a/docs/en/cloud/nestos/nestos/overview.md b/docs/en/cloud/nestos/nestos/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..b6bcb85ccb0c4b4fd3161b419dd02d16257891bc --- /dev/null +++ b/docs/en/cloud/nestos/nestos/overview.md @@ -0,0 +1,3 @@ +# NestOS User Guide + +This document describes the installation, deployment, features, and usage of the NestOS cloud-based operating system. NestOS runs common container engines, such as Docker, iSula, PodMan, and CRI-O, and integrates technologies such as Ignition, rpm-ostree, OCI runtime, and SELinux. NestOS adopts the design principles of dual-system partitions, container technology, and cluster architecture. It can adapt to multiple basic running environments in cloud scenarios.In addition, NestOS optimizes Kubernetes and provides support for platforms such as OpenStack and oVirt for IaaS ecosystem construction. In terms of PaaS ecosystem construction, platforms such as OKD and Rancher are supported for easy deployment of clusters and secure running of large-scale containerized workloads. To download NestOS images, see [NestOS](https://nestos.openeuler.org/). diff --git a/docs/en/devstation/oeGitExt/_toc.yaml b/docs/en/devstation/oeGitExt/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..14c65bb3406456649a69324ac37477cbf5a80061 --- /dev/null +++ b/docs/en/devstation/oeGitExt/_toc.yaml @@ -0,0 +1,10 @@ +label: oeGitExt User Guide +isManual: true +description: Use oeGitExt to perform CLI-based Gitee operations. +sections: + - label: User Guide + href: ./oeGitExt.md + - label: Installation and Uninstallation Guide + href: ./installation-and-uninstallation-guide.md + - label: Usage + href: ./usage.md diff --git a/docs/en/devstation/oeGitExt/installation-and-uninstallation-guide.md b/docs/en/devstation/oeGitExt/installation-and-uninstallation-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..af41515e76a0d9fdebe17cd15354a005cad4f96c --- /dev/null +++ b/docs/en/devstation/oeGitExt/installation-and-uninstallation-guide.md @@ -0,0 +1,31 @@ +# Installation and Uninstallation Guide + +## System Requirements + +- OS: openEuler 25.03 + +## Installation + +```bash +sudo dnf clean all +``` + +```bash +sudo dnf makecache +``` + +```bash +sudo dnf install oegitext +``` + +## Uninstallation + +```bash +sudo dnf remove oegitext +``` + +Then, remove the configuration file. + +```bash +rm ~/.oegitext +``` diff --git a/docs/en/devstation/oeGitExt/oeGitExt.md b/docs/en/devstation/oeGitExt/oeGitExt.md new file mode 100644 index 0000000000000000000000000000000000000000..af96c0cb3ea9de1dbfbe977e325a07e619122f0d --- /dev/null +++ b/docs/en/devstation/oeGitExt/oeGitExt.md @@ -0,0 +1,10 @@ +# oeGitExt User Guide + +## Overview + +oeGitExt enables CLI-based access to manage your Gitee repositories, issues, and pull requests (PRs), including those in the `openeuler` and `src-openeuler` projects. This guide covers installation, uninstallation, and usage instructions. + +This document is intended for Gitee users who should have the following knowledge or skills: + +- Basic understanding of Git and Gitee +- Basic Linux CLI proficiency diff --git a/docs/en/devstation/oeGitExt/usage.md b/docs/en/devstation/oeGitExt/usage.md new file mode 100644 index 0000000000000000000000000000000000000000..8f3cb8731b8e41d83f1bf1427167898adbe20e9e --- /dev/null +++ b/docs/en/devstation/oeGitExt/usage.md @@ -0,0 +1,272 @@ +# Usage + +## 1. Setting Up the Gitee Access Token + +Create a [Gitee personal access token](https://gitee.com/profile/personal_access_tokens) with `user_info`, `projects`, `pull_requests`, and `issues` permissions, then configure the token. + +```bash +oegitext config -token ${access_token} +``` + +## 2. Showing Repository Information + +View command help: + +```bash +oegitext show proj -h +``` + +Command syntax: + +```text +oegitext show proj [OPTIONS] +Available options: + -h/--help Show this help. + -create Show projects created by me. + -p/--pretty Formatted output + -j/--json JSON output + -s/--sort Sort by: created/updated + -d/--direction Sort order: asc/desc + -c/--columns Select output columns. +``` + +### Listing Accessible Projects + +```bash +oegitext show proj +oegitext show proj -p # Formatted output +oegitext show proj -c state,url # Filter specific columns. +oegitext show proj -c state,url -p +``` + +### Listing Projects Created by Me + +```bash +oegitext show proj -create # Formatted output +oegitext show proj -create -p # Show only the state and url columns. +oegitext show proj -create -c state,url +``` + +## 3. Showing Issue Information + +```bash +oegitext show issue -h +``` + +```text +oegitext show issue [-h] [-create] [-p] [-j] [-s {created,updated}] [-d {desc,asc}] [-c COLUMNS] +Available options: + -h, --help Show this help. + -create Show issues created by me. + -p, --pretty Formatted output + -j, --json JSON output + -s, --sort Sort by: created/updated + -d, --direction Sort order: asc/desc + -c, --columns Select output columns. +``` + +### Listing Issues Assigned to Me + +```bash +oegitext show issue # Formatted output +oegitext show issue -p # Show only the state and url columns. +oegitext show issue -c state,url +``` + +### Listing Issues Created by Me + +```bash +oegitext show issue -create # Formatted output +oegitext show issue -create -p # Show only the state and url columns. +oegitext show issue -create -c state,url +``` + +## 4. Showing PR Information + +```bash +oegitext show pr -h +``` + +```text +oegitext show pr [-h] [-oe] -name REPO_NAME [-only] [-p] [-j] [-c COLUMNS] +Available options: + -h, --help Show this help. + -name PR location (owner/repo) + -only Show PRs created by me. + -p, --pretty Formatted output + -j, --json JSON output + -c, --columns Select output columns. +``` + +### Listing PRs of a Repository + +```bash +oegitext show pr -name src-openeuler/vscode # Formatted output +oegitext show pr -name src-openeuler/vscode -p # Show only the state and url columns. +oegitext show pr -name src-openeuler/vscode -c state,url +``` + +## 5. Showing openEuler Repository Information + +```bash +oegitext show repo -h +``` + +```text +oegitext show repo [-h] [-owner {openeuler,src-openeuler}] [-p] [-j] [-c COLUMNS] +Available options: + -h, --help Show this help. + -owner Repository owner (openeuler or src-openeuler, defaulting to openeuler) + -p, --pretty Formatted output + -j, --json JSON output + -c, --columns Select output columns. +``` + +## 6. Forking a Repository + +```bash +oegitext show fork -h +``` + +```text +oegitext fork [-h] -user USER -repo REPO [-org ORG] [-name NAME] [-path PATH] [-show] +Available options: + -h, --help Show this help. + -user USER Namespace path (organization/enterprise/personal path) + -repo REPO Repository path + -org ORG Full organization namespace path (default: forks to personal namespace) + -name NAME Forked repository name (default: same as source) + -path PATH Forked repository path (default: same as source) + -show Display request result. +``` + +## 7. Handling Issues + +```bash +oegitext show issue -h +``` + +```text +oegitext issue [-h] -cmd {create,update,close,open,get} [-user USER] [-repo REPO] [-title TITLE] [-number NUMBER] + [-body BODY] [-show] +Available options: + -h, --help Show this help. + -cmd Issue operation command + -user USER Issue namespace path (organization/enterprise/personal path) + -repo REPO Repository path + -title TITLE Issue title + -number NUMBER Issue number + -body BODY Issue content + -show Display request result. +``` + +### Getting Repository Issues + +```text +oegitext issue -cmd get -user USER -repo REPO -number NUMBER [-show] +``` + +### Creating an Issue + +```text +oegitext issue -cmd create -user USER -repo REPO -title TITLE [-body BODY] [-show] +``` + +### Updating an Issue + +```text +oegitext issue -cmd update -user USER -repo REPO -title TITLE -number NUMBER [-body BODY] [-show] +``` + +### Closing an Issue + +```text +oegitext issue -cmd close -user USER -repo REPO -number NUMBER [-show] +``` + +### Reopening an Issue + +```text +oegitext issue -cmd open -user USER -repo REPO -number NUMBER [-show] +``` + +## 8. Handling PRs + +```bash +oegitext pull -h +``` + +```text +oegitext pull [-h] -cmd {create,update,close,open,review,test,merge,get} [-user USER] [-repo REPO] [-title TITLE] + [-head HEAD] [-base BASE] [-number NUMBER] [-body BODY] [-state STATE] [-show] +Available options: + -h, --help Show this help. + -cmd PR operation command: create, update, close, open, review, test, merge, get + -user USER PR namespace path (organization/enterprise/personal path) + -repo REPO Repository path + -title TITLE PR title + -head HEAD Source branch (format: branch or path_with_namespace:branch) + -base BASE Target branch name + -number NUMBER PR number + -body PR content + -state STATE PR state + -show Display request result. + +``` + +### Getting PR Information + +```text +oegitext pull -cmd get -user USER -repo REPO -number NUMBER [-show] +``` + +### Creating a PR + +```text +oegitext pull -cmd create -user USER -repo REPO -title TITLE -head HEAD -base BASE [-body BODY] [-show] +Available options: + -head HEAD: Pull Request 提交的源分支。格式：branch (master) 或者：path_with_namespace:branch (oschina/gitee:master) + -base BASE: Pull Request 提交目标分支的名称 +``` + +### Updating a PR + +```text +oegitext pull -cmd update -user USER -repo REPO -number NUMBER -body BODY [-show] +``` + +### Closing a PR + +```text +oegitext pull -cmd close -user USER -repo REPO -number NUMBER [-show] +``` + +### Reopening a PR + +```text +oegitext pull -cmd open -user USER -repo REPO -number NUMBER [-show] +``` + +### Reviewing a PR + +```text +oegitext pull -cmd review -user USER -repo REPO -number NUMBER -state {pass,reset} [-show] +Available options: + -state pass: Force review approval. + reset: Reset review status. +``` + +### Testing a PR + +```text +oegitext pull -cmd test -user USER -repo REPO -number NUMBER -state {pass,reset} [-show] +Available options: + -state pass: Force test approval. + reset: Reset test status. +``` + +### Merging a PR + +```text +oegitext pull -cmd merge -user USER -repo REPO -number NUMBER [-show] +``` diff --git a/docs/en/devstation/oedeploy/_toc.yaml b/docs/en/devstation/oedeploy/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..47f509caad32e8ee2088dcc91c7d7c5ca524adab --- /dev/null +++ b/docs/en/devstation/oedeploy/_toc.yaml @@ -0,0 +1,14 @@ +label: oeDeploy User Guide +isManual: true +description: 'oeDeploy is a lightweight deployment tool designed for simplicity.' +sections: + - label: oeDeploy Introduction + href: ./introduction.md + - label: oedp Command Guide + href: ./command.md + - label: oeDeploy Features + href: ./feature.md + - label: oeDeploy Use Cases + href: ./usecase.md + - label: oeDeploy Plugin Development Guide + href: ./develop_guide.md diff --git a/docs/en/devstation/oedeploy/command.md b/docs/en/devstation/oedeploy/command.md new file mode 100644 index 0000000000000000000000000000000000000000..32519dbeba9fe9d7712df9404e40d48e18765a4d --- /dev/null +++ b/docs/en/devstation/oedeploy/command.md @@ -0,0 +1,58 @@ +# oedp Command Reference + +## `oedp` + +| Option | Shortcut | Description | +| ----------- | -------- | -------------------- | +| `--version` | `-v` | Displays the version | + +## `oedp info` + +Displays project details (defaulting to the current directory). + +| Option | Shortcut | Mandatory | Description | +| ------------------ | -------- | --------- | -------------------------------- | +| `--project [path]` | `-p` | No | Specifies the project directory. | +| | | | | + +## `oedp run [action]` + +Executes a project action (defaulting to the current directory). `[action]` refers to a plugin method (query via `oedp info`). + +| Option | Shortcut | Mandatory | Description | +| ------------------ | -------- | --------- | -------------------------------- | +| `--project [path]` | `-p` | No | Specifies the project directory. | +| `--debug` (WIP) | `-d` | No | Runs in debug mode. | + +## `oedp list` + +List available plugins from sources + +| Option | Shortcut | Mandatory | Description | +| ---------------- | -------- | --------- | -------------------------------- | +| `--local [path]` | `-l` | No | Specifies the local source path. | + +## `oedp check [action]` (WIP) + +Validates project action requirements (defaulting to the current directory). + +| Option | Shortcut | Mandatory | Description | +| ------------------ | -------- | --------- | -------------------------------- | +| `--project [path]` | `-p` | No | Specifies the project directory. | + +## `oedp init [plugin]` (WIP) + +Initialize a plugin (`[plugin]` names available via `oedp list`) + +| Option | Shortcut | Mandatory | Description | +| ------------------ | -------- | --------- | --------------------------------------------------------------------------------- | +| `--project [path]` | `-p` | Yes | Specifies the initialization directory. | +| `--local [path]` | `-l` | No | Specifies the local source path. | +| `--force` | `-f` | No | Forces overwrite (deletes all files in the target directory before initializing). | + +## oedp Directory Structure + +- **/var/oedp/log/**: log files +- **/var/oedp/plugin/**: plugin cache +- **/usr/lib/oedp/src/**: source code +- **/etc/oedp/config/**: configuration files diff --git a/docs/en/devstation/oedeploy/develop_guide.md b/docs/en/devstation/oedeploy/develop_guide.md new file mode 100644 index 0000000000000000000000000000000000000000..c721a243fdd3126003256dc6e617df057d8665aa --- /dev/null +++ b/docs/en/devstation/oedeploy/develop_guide.md @@ -0,0 +1,146 @@ +# oeDeploy Plugin Development Guide + +## 1. Introduction to oeDeploy Plugins + +oeDeploy plugins are components that provide automated deployment capabilities within the oeDeploy tool. They achieve automation by converting complex deployment processes into Ansible playbooks. A plugin may integrate multiple deployment actions (such as installation, uninstallation, and environment cleanup), each corresponding to one or more Ansible playbooks. All configurable parameters should be centralized to reduce both user learning costs and developer maintenance efforts. + +## 2. Plugin Directory Structure + +Adhere to the following directory structure: + +```text +{ plugin_name } +|-- main.yaml +|-- config.yaml +|-- doc/ +`-- workspace/ +``` + +| File/Directory | Description | +| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | +| config.yaml | Contains host-related configurations (IP addresses, passwords, keys, and ports) and application deployment settings. Supports Jinja2 templating. | +| main.yaml | Primary configuration file for oedp to identify deployment workflows. | +| doc | Optional. Documentation directory for the plugin. | +| workspace | Deployment capability library for the application. | + +Naming constraints for `plugin_name`: + +- The deployed software name is `name`, and its version is `version`. +- `plugin_name` must follow one of these formats: + - `{name}-{version}` + - `{name}-{version}-xxx` +- **Example**: For a Kubernetes deployment plugin (version 1.31.1), valid names include `kubernetes-1.31.1`, `kubernetes-1.31.1-20240101`, and `kubernetes-1.31.1-offline-20240101`. + +## 3. main.yaml + +The **main.yaml** file records key plugin information including: `name`, `version`, `description`, and `action`. When users run the `oedp info` command, `oedp` reads and parses this file to display detailed information. + +The `action` field should be a dictionary (key-value map), where each key serves as an action name, and its corresponding value contains details about that action. + +Action details are structured as a dictionary. The `description` field provides explanatory text displayed during `oedp info` execution, while the `tasks` field lists the specific steps for the action. These steps are executed sequentially. + +For each step, specify the `playbook` path to execute. Optionally, a `vars` field can define variable file paths. All paths are relative to the `workspace` directory. Additionally, the `scope` field determines the target host group (defaulting to `all`) + +```yaml +name: kubernetes +version: 1.31.1 +description: install kubernetes 1.31.1 +action: + install: + description: install kubernetes + tasks: + - name: install kubernetes + playbook: init-k8s.yml + vars: variables.yml + scope: all + delete: + description: delete kubernetes + tasks: + - name: delete kubernetes + playbook: delete-k8s.yml + vars: variables.yml + scope: all + clean: + description: clean cache files + tasks: + - name: clean cache files + playbook: clean-k8s.yml + scope: all +``` + +## 4. config.yaml + +The **config.yaml** file serves as the user configuration file for the plugin, primarily containing host group settings and other global configurations. It follows Ansible inventory file rules and can be directly passed as an inventory during Ansible playbook execution. + +```yaml +all: + children: + masters: + hosts: + HOST_IP1: # e.g. 192.168.10.1 + ansible_host: HOST_IP1 # e.g. 192.168.10.1 + ansible_port: 22 + ansible_user: root + ansible_password: "" + architecture: amd64 # e.g. [ amd64, arm64 ] + oeversion: 22.03-LTS # e.g. [ 22.03-LTS, 24.03-LTS ] + workers: + hosts: + HOST_IP2: + ansible_host: HOST_IP2 + ansible_port: 22 + ansible_user: root + ansible_password: "" + architecture: amd64 + oeversion: 22.03-LTS + HOST_IP3: + ansible_host: HOST_IP3 + ansible_port: 22 + ansible_user: root + ansible_password: "" + architecture: amd64 + oeversion: 22.03-LTS + vars: + init_cluster_force: "true" # e.g. [ "true", "false" ] (forced cluster initialization) + service_cidr: 10.96.0.0/16 # Service subnet + pod_cidr: 10.244.0.0/16 # Pod IP address subnet + certs_expired: 3650 # Certificate expiration time + has_deployed_containerd: "false" # e.g. [ "true", "false" ] (whether containerd is available) +``` + +## 5. workspace + +The **workspace** directory contains the core deployment scripts. Its structure is flexible but must align with parameters defined in main.yaml and config.yaml. + +The **workspace** directory is treated as the root directory of the entire plugin. + +In this example, the **workspace** directory structure is: + +```text +workspace +|-- roles +| `-- ... +|-- init-k8s.yml +|-- delete-k8s.yml +|-- clean-k8s.yml +|-- variables.yml +`-- ... +``` + +## 6. Plugin Packaging + +To distribute a plugin, package it into `{plugin_name}.tar.gz` format. The compressed file name must match the internal directory name, and extraction should yield only one directory. For example: + +```text +kubernetes-1.31.1.tar.gz +`-- kubernetes-1.31.1 + |-- main.yaml + |-- config.yaml + `-- workspace/ +``` + +In this example, the following command packages the plugin: + +```bash +tar zcvf kubernetes-1.31.1.tar.gz kubernetes-1.31.1/ +``` diff --git a/docs/en/devstation/oedeploy/feature.md b/docs/en/devstation/oedeploy/feature.md new file mode 100644 index 0000000000000000000000000000000000000000..7076f7566818d2ef0cc0b4734f62dcde82327bf4 --- /dev/null +++ b/docs/en/devstation/oedeploy/feature.md @@ -0,0 +1,19 @@ +# Features in Development + +- Debug mode for executing specified tasks +- Plugin initialization functionality for deployment +- Plugin validation module for deployment +- Plugin source management module with automatic download/extraction by plugin name (configurable sources) +- oeDeploy web interface supporting custom deployment task creation, node information management, and visualized deployment workflows + +# Planned Features + +- Plugin marketplace with plugin lists and web-based task creation +- Helm deployment project compatibility + +# Released Features + +## v1.0.1 + +- Ansible integration as the distributed node command orchestration platform +- Playbook-level workflow orchestration support diff --git a/docs/en/devstation/oedeploy/introduction.md b/docs/en/devstation/oedeploy/introduction.md new file mode 100644 index 0000000000000000000000000000000000000000..eada8bbed11e59c2013351f2513e7a2497c22e79 --- /dev/null +++ b/docs/en/devstation/oedeploy/introduction.md @@ -0,0 +1,21 @@ +# Introduction + +oeDeploy is a lightweight software deployment tool designed to help developers quickly and efficiently set up various software environments, supporting both single-node and distributed scenarios. + +- **Universal scenario support**: Compatible with single-node and cluster environments. +- **One-click deployment for mainstream software**: Includes standardized deployment solutions for common software, with ongoing plugin additions to expand capabilities. +- **Flexible plugin management**: Features an extensible architecture, allowing developers to write custom plugins tailored to specific service needs. +- **Efficient development experience**: Supports the CLI and a GUI tool (work in progress), enabling developers to focus on workflow orchestration and core functionality with minimal coding. + +# Download + +- [oedp CLI tool (aarch64)](https://repo.oepkgs.net/openEuler/rpm/openEuler-24.03-LTS/contrib/oedp/aarch64/Packages/oedp-1.0.1-1.oe2503.aarch64.rpm) +- [oedp CLI tool (x86_64)](https://repo.oepkgs.net/openEuler/rpm/openEuler-24.03-LTS/contrib/oedp/x86_64/Packages/oedp-1.0.1-1.oe2503.x86_64.rpm) +- [oeDeploy plugins](https://repo.oepkgs.net/openEuler/rpm/openEuler-24.03-LTS/contrib/oedp/plugins/) + +# Docs + +- [oedp Command Guide](./command.md) +- [oeDeploy Features](./feature.md) +- [oeDeploy Use Cases](./usecase.md) +- [oeDeploy Plugin Development Guide](./develop_guide.md) diff --git a/docs/en/devstation/oedeploy/usecase.md b/docs/en/devstation/oedeploy/usecase.md new file mode 100644 index 0000000000000000000000000000000000000000..699cf930df56e758e43a19715a879fc52e6d55a7 --- /dev/null +++ b/docs/en/devstation/oedeploy/usecase.md @@ -0,0 +1,84 @@ +# oeDeploy Use Cases + +## Use Case 1: One-Click Kubernetes Deployment + +Prepare three virtual machines with 2 cores and 4 GB memory (with Layer-3 network connectivity) running openEuler 24.03 LTS SP1. The goal is to deploy a Kubernetes cluster consisting of 1 master and 2 worker nodes. + +On any node, download and install oeDeploy CLI tool `oedp`. + +```bash +wget https://repo.oepkgs.net/openEuler/rpm/openEuler-24.03-LTS/contrib/oedp/x86_64/Packages/oedp-1.0.1-1.oe2503.x86_64.rpm +yum install -y oedp-1.0.1-1.oe2503.x86_64.rpm +``` + +Download and extract the Kubernetes plugin package (verify that the **kubernetes-1.31.1** directory appears). + +```shell +wget https://repo.oepkgs.net/openEuler/rpm/openEuler-24.03-LTS/contrib/oedp/plugins/kubernetes-1.31.1.tar.gz +tar -zxvf kubernetes-1.31.1.tar.gz +``` + +View plugin details. + +```shell +oedp info -p kubernetes-1.31.1 +``` + +Modify the project configuration file with actual node information. + +```shell +vim kubernetes-1.31.1/config.yaml +``` + +```yaml +all: + children: + masters: + hosts: + 172.27.76.114: # master node IP address + ansible_host: 172.27.76.114 # master node IP address + ansible_port: 22 + ansible_user: root + ansible_password: + architecture: amd64 # amd64 or arm64 + oeversion: 24.03-LTS # 22.03-LTS or 24.03-LTS + workers: + hosts: + 172.27.70.60: # worker node IP address + ansible_host: 172.27.70.60 # worker node IP address + ansible_port: 22 + ansible_user: root + ansible_password: + architecture: amd64 + oeversion: 24.03-LTS + 172.27.72.90: + ansible_host: 172.27.72.90 + ansible_port: 22 + ansible_user: root + ansible_password: + architecture: amd64 + oeversion: 24.03-LTS + vars: + init_cluster_force: "true" + service_cidr: 10.96.0.0/16 + pod_cidr: 10.244.0.0/16 + certs_expired: 3650 + has_deployed_containerd: "false" + ansible_ssh_common_args: "-o StrictHostKeyChecking=no" +``` + +> Note: Ensure SSH connectivity between nodes (supports both password and key-based authentication). If using key-based authentication, omit password configuration. + +Execute automated deployment. + +```shell +oedp run install -p kubernetes-1.31.1 +``` + +To uninstall Kubernetes: + +```shell +oedp run delete -p kubernetes-1.31.1 +``` + +> The `-p` flag specifies the extracted plugin directory. When running `oedp` commands from within the **kubernetes-1.31.1** root directory, omit this parameter. diff --git a/docs/en/edge_computing/_toc.yaml b/docs/en/edge_computing/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..0a4c0f3cacf24ab27605aeaeb1b13a13325fdd7f --- /dev/null +++ b/docs/en/edge_computing/_toc.yaml @@ -0,0 +1,4 @@ +label: Edge Computing +sections: + - href: ./kube_edge/_toc.yaml + - href: ./k3s/_toc.yaml diff --git a/docs/en/edge_computing/k3s/_toc.yaml b/docs/en/edge_computing/k3s/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..a5ff12cc2aa8dbf1a117718797f02c3c006ad147 --- /dev/null +++ b/docs/en/edge_computing/k3s/_toc.yaml @@ -0,0 +1,8 @@ +label: K3s Deployment Guide +isManual: true +description: >- + K3s is a lightweight Kubernetes distribution designed for edge computing and + similar use cases. +sections: + - label: K3s Deployment Guide + href: ./k3s-deployment-guide.md diff --git a/docs/en/edge_computing/k3s/figures/agent-install.png b/docs/en/edge_computing/k3s/figures/agent-install.png new file mode 100644 index 0000000000000000000000000000000000000000..dca1d64ec8aae821393bb715daf4c56b783a68e0 Binary files /dev/null and b/docs/en/edge_computing/k3s/figures/agent-install.png differ diff --git a/docs/en/edge_computing/k3s/figures/check-agent.png b/docs/en/edge_computing/k3s/figures/check-agent.png new file mode 100644 index 0000000000000000000000000000000000000000..aa467713353d70ad513e8ee13ac9d8b6520b7ee0 Binary files /dev/null and b/docs/en/edge_computing/k3s/figures/check-agent.png differ diff --git a/docs/en/edge_computing/k3s/figures/check-server.png b/docs/en/edge_computing/k3s/figures/check-server.png new file mode 100644 index 0000000000000000000000000000000000000000..06343de9a8b0eacb0f6194cf438b2b27af88cae4 Binary files /dev/null and b/docs/en/edge_computing/k3s/figures/check-server.png differ diff --git a/docs/en/edge_computing/k3s/figures/server-install.png b/docs/en/edge_computing/k3s/figures/server-install.png new file mode 100644 index 0000000000000000000000000000000000000000..7d30c8f4f73946c8b0555186c1736492039da731 Binary files /dev/null and b/docs/en/edge_computing/k3s/figures/server-install.png differ diff --git a/docs/en/edge_computing/k3s/figures/set-hostname.png b/docs/en/edge_computing/k3s/figures/set-hostname.png new file mode 100644 index 0000000000000000000000000000000000000000..32564d6159825b6d4131a6b138a493188ce88c6c Binary files /dev/null and b/docs/en/edge_computing/k3s/figures/set-hostname.png differ diff --git a/docs/en/edge_computing/k3s/figures/token.png b/docs/en/edge_computing/k3s/figures/token.png new file mode 100644 index 0000000000000000000000000000000000000000..79e5313bd1d5e707659cd08d4aafdf528b9df8f0 Binary files /dev/null and b/docs/en/edge_computing/k3s/figures/token.png differ diff --git a/docs/en/edge_computing/k3s/figures/yum-install.png b/docs/en/edge_computing/k3s/figures/yum-install.png new file mode 100644 index 0000000000000000000000000000000000000000..0e601a23a5a67e7927f12bc90d1a4137e1a3a567 Binary files /dev/null and b/docs/en/edge_computing/k3s/figures/yum-install.png differ diff --git a/docs/en/edge_computing/k3s/k3s-deployment-guide.md b/docs/en/edge_computing/k3s/k3s-deployment-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..11dd4c263eb74dc90098950b45382a115bead865 --- /dev/null +++ b/docs/en/edge_computing/k3s/k3s-deployment-guide.md @@ -0,0 +1,90 @@ +# K3s Deployment Guide + +## What Is K3s + +K3s is a lightweight Kubernetes distribution that is optimized for edge computing and IoT scenarios. The K3s provides the following enhanced features: + +- Packaged as a single binary file. +- Uses SQLite3-based lightweight storage backend as the default storage mechanism and supports etcd3, MySQL, and PostgreSQL. +- Encapsulated in a simple launcher that handles various complex TLS and options. +- Secure by default and has reasonable default values for lightweight environments. +- Batteries included, providing simple but powerful functions such as local storage providers, service load balancers, Helm controllers, and Traefik Ingress controllers. +- Encapsulates all operations of the Kubernetes control plane in a single binary file and process, capable of automating and managing complex cluster operations including certificate distribution. +- Minimizes external dependencies and requires only kernel and cgroup mounting. + +## Application Scenarios + +K3s is applicable to the following scenarios: + +- Edge computing +- IoT +- Continuous integration +- Development +- ARM +- Embedded Kubernetes + +The resources required for running K3s are small. Therefore, K3s is also suitable for development and test scenarios. In these scenarios, K3s facilitates function verification and problem reproduction by shortening cluster startup time and reducing resources consumed by the cluster. + +## Deploying K3s + +### Preparations + +- Ensure that the host names of the server node and agent node are different. + +You can run the `hostnamectl set-hostname "host name"` command to change the host name. + +![1661829534335](./figures/set-hostname.png) + +- Install K3s on each node using Yum. + + The K3s official website provides binary executable files of different architectures and the **install.sh** script for offline installation. The openEuler community migrates the compilation process of the binary file to the community and releases the compiled RPM package. You can run the `yum` command to download and install K3s. + +![1661830441538](./figures/yum-install.png) + +### Deploying the Server Node + +To install K3s on a single server, run the following command on the server node: + +```shell +INSTALL_K3S_SKIP_DOWNLOAD=true k3s-install.sh +``` + +![1661825352724](./figures/server-install.png) + +### Checking Server Deployment + +![1661825403705](./figures/check-server.png) + +### Deploying the Agent Node + +Query the token value of the server node. The token is stored in the **/var/lib/rancher/k3s/server/node-token** file on the server node. + +> **Note**: +> +> Only the second half of the token is used. + +![1661825538264](./figures/token.png) + +Add agents. Run the following command on each agent node: + +```shell +INSTALL_K3S_SKIP_DOWNLOAD=true K3S_URL=https://myserver:6443 K3S_TOKEN=mynodetoken k3s-install.sh +``` + +> **Note:** +> +> Replace **myserver** with the IP address of the server or a valid DNS, and replace **mynodetoken** with the token of the server node. + +![1661829392357](./figures/agent-install.png) + +### Checking Agent Deployment + +After the installation is complete, run `kubectl get nodes` on the server node to check if the agent node is successfully registered. + +![1661826797319](./figures/check-agent.png) + +A basic K3S cluster is set up. + +### More + +For details about how to use K3s, visit the K3s [official website](https://rancher.com/docs/k3s/latest/en/). diff --git a/docs/en/edge_computing/kube_edge/_toc.yaml b/docs/en/edge_computing/kube_edge/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..6d1d34bc2d89ab72ee406ba710b46720fee6c7b2 --- /dev/null +++ b/docs/en/edge_computing/kube_edge/_toc.yaml @@ -0,0 +1,9 @@ +label: KubeEdge User Guide +isManual: true +href: ./overview.md +description: KubeEdge brings Kubernetes functionalities to edge environments. +sections: + - label: KubeEdge Usage Guide + href: ./kubeedge-usage-guide.md + - label: KubeEdge Deployment Guide + href: ./kubeedge-deployment-guide.md diff --git a/docs/en/edge_computing/kube_edge/kubeedge-deployment-guide.md b/docs/en/edge_computing/kube_edge/kubeedge-deployment-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..5a306908787a285f8a23984b1a0191e0649a8484 --- /dev/null +++ b/docs/en/edge_computing/kube_edge/kubeedge-deployment-guide.md @@ -0,0 +1,224 @@ +# KubeEdge Deployment Guide + +## Description + +### KubeEdge + +KubeEdge is an open source system dedicated to solving problems in edge scenarios. It extends the capabilities of containerized application orchestration and device management to edge devices. Based on Kubernetes, KubeEdge provides core infrastructure support for networks, application deployment, and metadata synchronization between the cloud and the edge. KubeEdge supports MQTT and allows for custom logic to enable communication for the resource-constrained devices at the edge. KubeEdge consists of components deployed on the cloud and edge nodes. The components are now open source. + +> + +### iSulad + +iSulad is a lightweight container runtime daemon designed for IoT and cloud infrastructure. It is lightweight, fast, and is not restricted by hardware specifications or architectures. It is suitable for wide application in various scenarios, such as cloud, IoT, and edge computing. + +> + +## Cluster Overview + +### Component Versions + +| Component | Version | +| ---------- | --------------------------------- | +| OS | openEuler 22.03 | +| Kubernetes | 1.20.2-4 | +| iSulad | 2.0.11 | +| KubeEdge | v1.8.0 | + +### Node Planning Example + +| Node | Location | Components | +| -------------- | -------- | -------------------------------- | +| cloud.kubeedge | Cloud | Kubernetes (Master), iSulad, CloudCore | +| edge.kubeedge | Edge | iSulad, EdgeCore | + +> Note: You can run the `hostnamectl set-hostname [cloud,edge].kubeedge` command to set the cloud and edge node names in advance. + +## Preparations + +### Tool Package Download + +[kubeedge-tools](https://gitee.com/Poorunga/kubeedge-tools) provides complete offline installation packages and deployment scripts for easy and quick KubeEdge cluster deployment even if the node cannot access the Internet. + +```bash +# Download and decompress the kubeedge-tools package on both the cloud and edge nodes. +$ wget -O kubeedge-tools.zip https://gitee.com/Poorunga/kubeedge-tools/repository/archive/master.zip +$ unzip kubeedge-tools.zip + +# Go to the kubeedge-tools directory for all the subsequent operations. +$ cd kubeedge-tools-master +``` + +### Kubernetes Deployment + +Perform the following operations on the cloud node only. + +#### Initializing the Cloud Environment + +```bash +./setup-cloud.sh +``` + +#### Installing Kubernetes + +Use openEuler 22.03 SP2 to install Kubernetes. + +#### Configuring Network for the Cloud Container + +Container Network Interface (CNI) software that provides network for Kubernetes nodes include [flannel](https://github.com/flannel-io/flannel), [Calico](https://github.com/projectcalico/calico), [Cilium](https://github.com/cilium/cilium), and more. If you have not decided which CNI software to use, run the following command to configure network for the cloud container: + +```bash +./install-flannel-cloud.sh +``` + +#### Checking Deployment Status + +```bash +# Check whether the node status is normal (Ready) +$ kubectl get nodes +NAME STATUS ROLES AGE VERSION +cloud.kubeedge Ready control-plane,master 12m v1.20.2 + +# Check whether the Kubernetes components are normal (Running) +$ kubectl get pods -n kube-system +NAME READY STATUS RESTARTS AGE +coredns-74ff55c5b-4ptkh 1/1 Running 0 15m +coredns-74ff55c5b-zqx5n 1/1 Running 0 15m +etcd-cloud.kubeedge 1/1 Running 0 15m +kube-apiserver-cloud.kubeedge 1/1 Running 0 15m +kube-controller-manager-cloud.kubeedge 1/1 Running 0 15m +kube-flannel-cloud-ds-lvh4n 1/1 Running 0 13m +kube-proxy-2tcnn 1/1 Running 0 15m +kube-scheduler-cloud.kubeedge 1/1 Running 0 15m +``` + +## Deployment + +### CloudCore Deployment + +Perform the following operations on the cloud node only. + +#### Initializing the Cluster + +```bash +# Set --advertise-address to the IP address of the cloud node. +$ keadm init --advertise-address="cloud_node_IP_address" --kubeedge-version=1.8.0 +... +CloudCore started +``` + +#### Configuring CloudCore + +```bash +./patch-cloud.sh +``` + +#### Checking Deployment Status + +```bash +# active (running) indicates a normal status +$ systemctl status cloudcore | grep running + Active: active (running) since Fri 2022-03-04 10:54:30 CST; 5min ago +``` + +CloudCore has been deployed on the cloud node. Then, deploy EdgeCore on the edge node. + +### EdgeCore Deployment + +Perform the following operations only on the edge node unless otherwise specified. + +#### Initializing the Edge Environment + +```bash +./setup-edge.sh +``` + +#### Managing the Edge Node + +```bash +# Run the keadm gettoken command on the cloud node. +$ keadm gettoken +96058ab80ffbeb87fe58a79bfb19ea13f9a5a6c3076a17c00f80f01b406b4f7c.eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NDY0NDg4NzF9.1mJegWB7SUVjgf-OvAqILgbZXeMHR9eOzMxpNFc42SI +# Save this token for subsequent steps. + + +# Run the keadm join command on the edge node. +# Set --cloudcore-ipport to the IP address and port number (10000) of the cloud node. Set --token to the token saved in the previous step. +$ keadm join --cloudcore-ipport=clou_node_IP_address:10000 --kubeedge-version=1.8.0 --token=96058ab80ffbeb87fe58a79bfb19ea13f9a5a6c3076a17c00f80f01b406b4f7c.eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NDY0NDg4NzF9.1mJegWB7SUVjgf-OvAqILgbZXeMHR9eOzMxpNFc42SI +... +KubeEdge edgecore is running... +``` + +#### Configuring EdgeCore + +```bash +./patch-edge.sh +``` + +#### Configuring Network for the Edge Container + +If you have not decided which CNI software to use, run the following command to configure network for the edge container: + +```bash +# Run the command on the cloud node. +$ ./install-flannel-edge.sh +``` + +#### Checking Whether the Edge Node is Managed + +```bash +# Run the command on the cloud node. You can see that the edge node is added. +$ kubectl get nodes +NAME STATUS ROLES AGE VERSION +cloud.kubeedge Ready control-plane,master 1h v1.20.2 +edge.kubeedge Ready agent,edge 10m v1.19.3-kubeedge-v1.8.0 +``` + +The KubeEdge cluster has been deployed. Next, let's test the task delivery from the cloud to the edge. + +### Application Deployment + +Perform the following operations on the cloud node only. + +#### Deploying Nginx + +```bash +$ kubectl apply -f yamls/nginx-deployment.yaml +deployment.apps/nginx-deployment created + +# Check whether Nginx is deployed on the edge node and running. +$ kubectl get pod -owide | grep nginx +nginx-deployment-84b99f4bf-jb6sz 1/1 Running 0 30s 10.244.1.2 edge.kubeedge +``` + +#### Testing the Function + +```bash +# Access the IP address of Nginx on the edge node. +$ curl 10.244.1.2:80 + + + +Welcome to nginx! + + + +

Welcome to nginx!

+

If you see this page, the nginx web server is successfully installed and +working. Further configuration is required.

+ +

For online documentation and support please refer to +nginx.org.
+Commercial support is available at +nginx.com.

+ +

Thank you for using nginx.

+ + +``` + +The deployment of KubeEdge is complete. diff --git a/docs/en/edge_computing/kube_edge/kubeedge-usage-guide.md b/docs/en/edge_computing/kube_edge/kubeedge-usage-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..1d32cab4d8aefd8699c001dd58a820e5ccde81d1 --- /dev/null +++ b/docs/en/edge_computing/kube_edge/kubeedge-usage-guide.md @@ -0,0 +1,221 @@ +# KubeEdge Usage Guide + +KubeEdge extends the capabilities of Kubernetes to edge scenarios and provides infrastructure support for the network, application deployment, and metadata synchronization between the cloud and the edge. The usage of KubeEdge is the same as that of Kubernetes. In addition, KubeEdge supports the management and control of edge devices. The following example describes how to use KubeEdge to implement edge-cloud synergy. + +## 1. Preparations + +Example: **KubeEdge Counter Demo** + +The counter is a pseudo device. You can run this demo without any additional physical devices. The counter runs on the edge side. You can use the web interface on the cloud side to control the counter and get the counter value. Click the link below to view the schematic diagram. + +For details, see . + +1. This demo requires the KubeEdge v1.2.1 or later. In this example, the latest KubeEdge v1.8.0 is used. + + ```shell + [root@ke-cloud ~]# kubectl get node + NAME STATUS ROLES AGE VERSION + ke-cloud Ready master 13h v1.20.2 + ke-edge1 Ready agent,edge 64s v1.19.3-kubeedge-v1.8.0 + + # Note: In this document, the edge node ke-edge1 is used for verification. If you perform verification by referring to this document, you need to change the edge node name based on your actual deployment. + ``` + +2. Ensure that the following configuration items are enabled for the Kubernetes API server: + + ```shell + --insecuret-port=8080 + --insecure-bind-address=0.0.0.0 + ``` + + You can modify the `/etc/kubernetes/manifests/kube-apiserver.yaml` file, and then restart the Pod of the Kubernetes API server component to make the modifications take effect. + +3. Download the sample code: + + ```shell + [root@ke-cloud ~]# git clone https://github.com/kubeedge/examples.git $GOPATH/src/github.com/kubeedge/examples + ``` + +## 2. Creating the Device Model and Device + +1. Create the device model. + + ```shell + [root@ke-cloud ~]# cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/crds + [root@ke-cloud crds~]# kubectl create -f kubeedge-counter-model.yaml + ``` + +2. Create the device. + + Modify **matchExpressions** as required. + + ```shell + [root@ke-cloud ~]# cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/crds + [root@ke-cloud crds~]# vim kubeedge-counter-instance.yaml + apiVersion: devices.kubeedge.io/v1alpha1 + kind: Device + metadata: + name: counter + labels: + description: 'counter' + manufacturer: 'test' + spec: + deviceModelRef: + name: counter-model + nodeSelector: + nodeSelectorTerms: + - matchExpressions: + - key: 'kubernetes.io/hostname' + operator: In + values: + - ke-edge1 + + status: + twins: + - propertyName: status + desired: + metadata: + type: string + value: 'OFF' + reported: + metadata: + type: string + value: '0' + + [root@ke-cloud crds~]# kubectl create -f kubeedge-counter-instance.yaml + ``` + +## 3. Deploying the Cloud Application + +1. Modify the code. + + The cloud application **web-controller-app** controls the edge application **pi-counter-app**. The default listening port of the cloud application is 80. Change the port number to 8089. + + ```shell + [root@ke-cloud ~]# cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/web-controller-app + [root@ke-cloud web-controller-app~]# vim main.go + package main + + import ( + "github.com/astaxie/beego" + "github.com/kubeedge/examples/kubeedge-counter-demo/web-controller-app/controller" + ) + + func main() { + beego.Router("/", new(controllers.TrackController), "get:Index") + beego.Router("/track/control/:trackId", new(controllers.TrackController), "get,post:ControlTrack") + + beego.Run(":8089") + } + ``` + +2. Build the image. + + Note: When building the image, copy the source code to the path specified by **GOPATH**. Disable Go modules if they are enabled. + + ```shell + [root@ke-cloud web-controller-app~]# make all + [root@ke-cloud web-controller-app~]# make docker + ``` + +3. Deploy web-controller-app. + + ```shell + [root@ke-cloud ~]# cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/crds + [root@ke-cloud crds~]# kubectl apply -f kubeedge-web-controller-app.yaml + ``` + +## 4. Deploying the Edge Application + +The **pi-counter-app** application on the edge is controlled by the cloud application. The edge application communicates with the MQTT server to perform simple counting. + +1. Modify the code and build the image. + + Change the value of **GOARCH** to **amd64** in `Makefile` to run the container. + + ```shell + [root@ke-cloud ~]# cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/counter-mapper + [root@ke-cloud counter-mapper~]# vim Makefile + .PHONY: all pi-execute-app docker clean + all: pi-execute-app + + pi-execute-app: + GOARCH=amd64 go build -o pi-counter-app main.go + + docker: + docker build . -t kubeedge/kubeedge-pi-counter:v1.0.0 + + clean: + rm -f pi-counter-app + + [root@ke-cloud counter-mapper~]# make all + [root@ke-cloud counter-mapper~]# make docker + ``` + +2. Deploy pi-counter-app. + + ```shell + [root@ke-cloud ~]# cd $GOPATH/src/github.com/kubeedge/examples/kubeedge-counter-demo/crds + [root@ke-cloud crds~]# kubectl apply -f kubeedge-pi-counter-app.yaml + + Note: To prevent Pod deployment from being stuck at `ContainerCreating`, run the docker save, scp, and docker load commands to release the image to the edge. + + [root@ke-cloud ~]# docker save -o kubeedge-pi-counter.tar kubeedge/kubeedge-pi-counter:v1.0.0 + [root@ke-cloud ~]# scp kubeedge-pi-counter.tar root@192.168.1.56:/root + [root@ke-edge1 ~]# docker load -i kubeedge-pi-counter.tar + ``` + +## 5. Trying the Demo + +Now, the KubeEdge Demo is deployed on the cloud and edge as follows: + +```shell +[root@ke-cloud ~]# kubectl get pods -o wide +NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES +kubeedge-counter-app-758b9b4ffd-f8qjj 1/1 Running 0 26m 192.168.1.66 ke-cloud +kubeedge-pi-counter-c69698d6-rb4xz 1/1 Running 0 2m 192.168.1.56 ke-edge1 +``` + +Let's test the running effect of the Demo. + +1. Execute the ON command. + On the web page, select **ON** and click **Execute**. You can run the following command on the edge node to view the execution result: + + ```shell + [root@ke-edge1 ~]# docker logs -f counter-container-id + ``` + +2. Check the counter's STATUS. + On the web page, select **STATUS** and click **Execute**. The current counter status is displayed on the web page. + +3. Execute the OFF command. + On the web page, select **OFF** and click **Execute**. You can run the following command on the edge node to view the execution result: + + ```shell + [root@ke-edge1 ~]# docker logs -f counter-container-id + ``` + +## 6. Others + +1. For more official KubeEdge examples, visit . + + | Name | Description | + | ---------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | + | [LED-RaspBerry-Pi](https://github.com/kubeedge/examples/blob/master/led-raspberrypi/README.md) | Controlling a LED light with Raspberry Pi using KubeEdge platform. | + | [Data Analysis @ Edge](https://github.com/kubeedge/examples/blob/master/apache-beam-analysis/README.md) | Analyzing data at edge by using Apache Beam and KubeEdge. | + | [Security@Edge](https://github.com/kubeedge/examples/blob/master/security-demo/README.md) | Security at edge using SPIRE for identity management. | + | [Bluetooth-CC2650-demo](https://github.com/kubeedge/examples/blob/master/bluetooth-CC2650-demo/README.md) | Controlling a CC2650 SensorTag bluetooth device using KubeEdge platform. | + | [Play Music @Edge through WeChat](https://github.com/kubeedge/examples/blob/master/wechat-demo/README.md) | Play music at edge based on WeChat and KubeEdge. | + | [Play Music @Edge through Web](https://github.com/kubeedge/examples/blob/master/web-demo/README.md) | Play music at edge based on Web and KubeEdge. | + | [Collecting temperature @Edge](https://github.com/kubeedge/examples/blob/master/temperature-demo/README.md) | Collecting temperature at edge based KubeEdge. | + | [Control pseudo device counter and collect data](https://github.com/kubeedge/examples/blob/master/kubeedge-counter-demo/README.md) | Control pseudo device counter and collect data based KubeEdge. | + | [Play Music @Edge through Twitter](https://github.com/kubeedge/examples/blob/master/ke-twitter-demo/README.md) | Play music at edge based on Twitter and KubeEdge. | + | [Control Zigbee @Edge through cloud](https://github.com/kubeedge/examples/blob/master/kubeedge-edge-ai-application/README.md) | Face detection at cloud using OpenCV and using it to control zigbee on edge using KubeEdge. | + +2. Use EdgeMesh to discover edge services. + + + +3. Customize the cloud-edge message route. + + diff --git a/docs/en/edge_computing/kube_edge/overview.md b/docs/en/edge_computing/kube_edge/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..5b0b219c468dfdb9ce346ba1c0b86f61a13ef152 --- /dev/null +++ b/docs/en/edge_computing/kube_edge/overview.md @@ -0,0 +1,3 @@ +# KubeEdge User Guide + +This document describes how to deploy and use the KubeEdge edge computing platform for users and administrators. diff --git a/docs/en/embedded/_toc.yaml b/docs/en/embedded/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..c7e64045a647861d7c6b39557100b0480ab637cc --- /dev/null +++ b/docs/en/embedded/_toc.yaml @@ -0,0 +1,9 @@ +label: Embedded +sections: + - label: openEuler Embedded User Guide + href: >- + https://pages.openeuler.openatom.cn/embedded/docs/build/html/master/index.html + description: >- + openEuler Embedded is a lightweight, secure, real-time OS tailored for + embedded environments, with support for various hardware architectures. + - href: ./uniproton/_toc.yaml diff --git a/docs/en/embedded/uniproton/_toc.yaml b/docs/en/embedded/uniproton/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..e3ed042d8632b161b4cda877eaaf8e4cd8beacaf --- /dev/null +++ b/docs/en/embedded/uniproton/_toc.yaml @@ -0,0 +1,12 @@ +label: UniProton User Guide +isManual: true +description: >- + UniProton is an OS tailored for embedded environments. It offers task and + memory management, interrupt handling, and advanced debugging features. +sections: + - label: Overview + href: ./overview.md + - label: UniProton Feature Design + href: ./uniproton-functions.md + - label: UniProton Interfaces + href: ./uniproton-apis.md diff --git a/docs/en/embedded/uniproton/figures/FCS.png b/docs/en/embedded/uniproton/figures/FCS.png new file mode 100644 index 0000000000000000000000000000000000000000..08e6b0a2156f7dd751ce36e2378a7242e97f89b3 Binary files /dev/null and b/docs/en/embedded/uniproton/figures/FCS.png differ diff --git a/docs/en/embedded/uniproton/figures/MemoryApplication.png b/docs/en/embedded/uniproton/figures/MemoryApplication.png new file mode 100644 index 0000000000000000000000000000000000000000..533dbdfce75046db700ac9cb1560cffaaf204dd3 Binary files /dev/null and b/docs/en/embedded/uniproton/figures/MemoryApplication.png differ diff --git a/docs/en/embedded/uniproton/figures/MemoryRelease.png b/docs/en/embedded/uniproton/figures/MemoryRelease.png new file mode 100644 index 0000000000000000000000000000000000000000..954743ce27dd9b74d9ec4d0f45e1e5185ad696f0 Binary files /dev/null and b/docs/en/embedded/uniproton/figures/MemoryRelease.png differ diff --git a/docs/en/embedded/uniproton/figures/pend_semaphore.png b/docs/en/embedded/uniproton/figures/pend_semaphore.png new file mode 100644 index 0000000000000000000000000000000000000000..59d8159d1ff1cecb43f59cc5d7c5a9900db8e767 Binary files /dev/null and b/docs/en/embedded/uniproton/figures/pend_semaphore.png differ diff --git a/docs/en/embedded/uniproton/figures/post_semaphore.png b/docs/en/embedded/uniproton/figures/post_semaphore.png new file mode 100644 index 0000000000000000000000000000000000000000..fa08d76dafd335b60838dda08db61ccadd8c6b8d Binary files /dev/null and b/docs/en/embedded/uniproton/figures/post_semaphore.png differ diff --git a/docs/en/embedded/uniproton/overview.md b/docs/en/embedded/uniproton/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..708e82ee0920f5755184de937b851a34c3039793 --- /dev/null +++ b/docs/en/embedded/uniproton/overview.md @@ -0,0 +1,7 @@ +# UniProton User Guide + +## Introduction + +UniProton is an operating system (OS) for embedded scenarios provided by the openEuler community. It aims to build a high-quality OS platform that shields underlying hardware differences for upper-layer service software and provides powerful debugging functions. UniProton allows service software to be quickly ported to different hardware platforms, facilitates chip selection, and reduces costs for hardware procurement and software maintenance. + +This document describes the basic functions and APIs of UniProton. diff --git a/docs/en/embedded/uniproton/uniproton-apis.md b/docs/en/embedded/uniproton/uniproton-apis.md new file mode 100644 index 0000000000000000000000000000000000000000..8e8ae10187b8253da0a4f25aa943ae46d8d6598e --- /dev/null +++ b/docs/en/embedded/uniproton/uniproton-apis.md @@ -0,0 +1,3 @@ +# UniProton APIs + +This document is currently not available in English. diff --git a/docs/en/embedded/uniproton/uniproton-functions.md b/docs/en/embedded/uniproton/uniproton-functions.md new file mode 100644 index 0000000000000000000000000000000000000000..3845d308d844162cf91631e0424564ab5ec6a2db --- /dev/null +++ b/docs/en/embedded/uniproton/uniproton-functions.md @@ -0,0 +1,156 @@ +# UniProton Feature Design + + + +- [UniProton Feature Design](#uniproton-feature-design) + - [Task Management](#task-management) + - [Event Management](#event-management) + - [Queue Management](#queue-management) + - [Hard Interrupt Management](#hard-interrupt-management) + - [Memory Management](#memory-management) + - [FSC Memory Algorithm](#fsc-memory-algorithm) + - [Core Idea](#core-idea) + - [Memory Application](#memory-application) + - [Memory Release](#memory-release) + - [Timer Management](#timer-management) + - [Semaphore Management](#semaphore-management) + - [Exception Management](#exception-management) + - [CPU Usage Statistics](#cpu-usage-statistics) + - [STM32F407ZGT6 Development Board Support](#stm32f407zgt6-development-board-support) + - [OpenAMP Hybrid Deployment](#openamp-hybrid-deployment) + - [POSIX Standard APIs](#posix-standard-apis) + +## Task Management + +UniProton is a single-process multi-thread operating system (OS). In UniProton, a task represents a thread. Tasks in UniProton are scheduled in preemption mode instead of time slice rotation scheduling. High-priority tasks can interrupt low-priority tasks. Low-priority tasks can be scheduled only after high-priority tasks are suspended or blocked. + +A total of 32 priorities are defined, with priority 0 being the highest and 31 being the lowest. Multiple tasks can be created in a priority. + +The task management module of UniProton provides the following functions: - Creates, deletes, suspends, resumes, and delays tasks; - Locks and unlocks task scheduling; - Obtains the current task ID; - Obtains and sets task private data; - Query the pending semaphore ID of a specified task; - Query the status, context, and general information of a specified task; - Obtains and sets task priorities; - Adjusts the task scheduling order of a specified priority; - Register and unregister hooks for task creation, deletion, and switching. During initialization, UniProton creates an idle task with the lowest priority by default. When no task is in the running status, the idle task is executed. + +## Event Management + +The event mechanism enables communication between threads. Event communication can only be event notifications and no data is transmitted. + +As an extension of tasks, events allow tasks to communicate with each other. Each task supports 32 event types, each represented by a bit of a 32-bit value. + +UniProton can read current task events and write specified task events. Multiple event types can be read or written at one time. + +## Queue Management + +A queue, also called message queue, is a method commonly used for inter-thread communication to store and transfer data. Data can be written to the head or tail of a queue based on the priority, but can be read only from the head of a queue. + +When creating a queue, UniProton allocates memory space for the queue based on the queue length and message unit size input by the user. The queue control block contains **Head** and **Tail** pointers, which indicate the storage status of data in a queue. **Head** indicates the start position of occupied message nodes in the queue. **Tail** indicates the end position of the occupied message nodes in the queue. + +## Hard Interrupt Management + +A hardware interrupt is a level signal that is triggered by hardware and affects system running. A hardware interrupt is used to notify the CPU of a hardware event. Hardware interrupts include maskable interrupts and non-maskable interrupts (NMIs). + +Hardware interrupts have different internal priorities, but they all have a higher priority than other tasks. When multiple hardware interrupts are triggered at the same time, the hardware interrupt with the highest priority is always responded first. Whether a high-priority hardware interrupt can interrupt a low-priority hardware interrupt that is being executed (that is, nested interrupts) depends on the chip platform. + +The OS creates a tick hardware interrupt during initialization for task delay and software timer purposes. The tick is essentially a hardware timer. + +## Memory Management + +Memory management is to dynamically divide and manage large memory areas allocated by users. When a section of a program needs to use the memory, the program calls the memory application function of the OS to obtain the memory block of a specified size. After using the memory, the program calls the memory release function to release the occupied memory. + +UniProton provides the FSC memory algorithm. The following table lists the advantages, disadvantages, and application scenarios of FSC. + +| Algorithm | Advantages | Disadvantages | Application Scenarios | +| :----------------------------------------------------------- | ------------------------------------------------------------ | ------------------------------ | ------------------------------------ | +| Private FSC algorithm| The memory control block information occupies a small amount of memory. The minimum 4-byte-aligned memory block size can be applied for. Adjacent memory blocks can be quickly split and merged without creating memory fragmentation.| The efficiency of memory application and release is low.| It can flexibly adapt to various product scenarios.| + +The FSC memory algorithm is described as follows: + +### FSC Memory Algorithm + +#### Core Idea + +The size of the requested memory is **uwSize**. If the size is in binary, it is expressed as **0b{0}1xxx**. **{0}** indicates that there may be one or more zeros before **1**. Regardless of the content of following **1** (**xxx**), if **1** is changed to **10** and **xxx** is changed to **0**, **10yyy** is always greater than **1xxx** (**yyy** indicates that the corresponding bits of **xxx** are changed to **0**). + +The subscript of the leftmost 1 can be directly obtained. The subscript values are 0 to 31 from the most significant bit to the least significant bit (BitMap), or 0 to 31 from the least significant bit to the most significant bit (uwSize). If the subscripts of the bits of the 32-bit register are 0 to 31 from the most significant bit to the least significant bit, the subscript of the leftmost 1 of 0x80004000 is 0. Therefore, we can maintain an idle linked list header array (the number of elements does not exceed 31). The subscript of the leftmost 1 of the memory block size is used as the index of the linked list header array. That is, all memory blocks with the same subscript of the leftmost 1 are mounted to the same idle linked list. + +For example, the sizes of idle blocks that can be mounted to the linked list whose index is 2 are 4, 5, 6, and 7, and the sizes of idle blocks that can be mounted to the linked list whose index is N are 2^N to 2^(N+1)-1. + +![](./figures/FCS.png) + +#### Memory Application + +When applying for the memory of uwSize, use assembly instructions to obtain the subscript of the leftmost 1 first. Assume that the subscript is **n**. To ensure that the first idle memory block in the idle linked list meets the uwSize requirement, the search starts from the index n+1. If the idle linked list of index n+1 is not empty, the first idle block in the linked list is used. If the linked list of n+1 is empty, the linked list of n+2 is checked, and so on, until a non-empty linked list is found or the index reaches 31. + +A 32-bit BitMap global variable is defined to prevent the for loop from checking whether the idle linked list is empty recursively. If the idle linked list of n is not empty, the value whose subscript is n of BitMap is set to 1. Otherwise, the value is set to 0. The bit whose subscript is 31 of the BitMap is directly set to 1 during initialization. Therefore, the first non-idle linked list is searched from linked list of n+1. Bits 0 to n of the BitMap copy can be cleared first, and then a subscript of the leftmost 1 of the copy is obtained. If the subscript is not equal to 31, the subscript is the array index of the first non-empty idle linked list. + +All idle blocks are connected in series in the form of a bidirectional idle linked list. If the first idle block obtained from the linked list is large, that is, after a usSize memory block is split, the remaining space can be allocated at least once, The remaining idle blocks are added to the corresponding idle linked list. + +![](./figures/MemoryApplication.png) + +The memory control header records the size of the idle memory block (including the control header itself). The memory control header contains a reused member at the beginning. When a memory block is idle, it is used as a pointer to the next idle memory block. When a memory block is occupied, it stores a magic number, indicating that the memory block is not idle. To prevent the magic number from conflicting with the pointer (same as the address value), the upper and lower four bits of the magic number are 0xf. The start addresses of the allocated memory blocks are 4-byte-aligned. Therefore, no conflict occurs. + +#### Memory Release + +When the memory is released, adjacent idle blocks are combined. First, the validity of the address parameter (**pAddr**) is determined by checking the magic number in the control header. The start address of the control header of the next memory block is obtained by adding the start address to the offset value. If the next memory block is idle, the next memory block is deleted from the idle linked list to which it belongs, and the size of the current memory block is adjusted. + +To quickly find the control header of the previous memory block and determine whether the previous memory block is idle during memory release, a member is added to the memory control header to mark whether the previous memory block is idle. When the memory is applied for, the flag of the next memory block can be set to the occupied state (if the idle memory block is divided into two, and the previous memory block is idle, the flag of the current memory block is set to the idle state). When the memory is released, the flag of the next memory block is set to the idle state. When the current memory is released, if the previous memory block is marked as occupied, the previous memory block does not need to be merged; if the previous memory block is marked as idle, the previous memory block needs to be merged. If a memory block is idle, the flag of the next control block is set to the distance to the current control block. + + ![](./figures/MemoryRelease.png) + +## Timer Management + +UniProton provides the software timer function to meet the requirements of timing services. + +Software timers are based on the tick interrupts. Therefore, the period of a timer must be an integral multiple of the tick. The timeout scanning of the software timer is performed in the tick handler function. + +Currently, the software timer interface can be used to create, start, stop, restart, and delete timers. + +## Semaphore Management + +A semaphore is typically used to coordinate a group of competing tasks to access to critical resources. When a mutex is required, the semaphore is used as a critical resource counter. Semaphores include intra-core semaphores and inter-core semaphores. + +The semaphore object has an internal counter that supports the following operations: + +- Pend: The Pend operation waits for the specified semaphore. If the counter value is greater than 0, it is decreased by 1 and a success message is returned. If the counter value of the semaphore is 0, the requesting task is blocked until another task releases the semaphore. The amount of time the task will wait for the semaphore is user configurable. + +- Post: The Post operation releases the specified semaphore. If no task is waiting for the semaphore, the counter is incremented by 1 and returned. Otherwise, the first task (the earliest blocked task) in the list of tasks pending for this semaphore is woken up. + +The counter value of a semaphore corresponds to the number of available resources. It means mutually exclusive resources remained that could be occupied. The counter value can be: + +- 0, indicating that there is no accumulated post operation, and there may be a task blocked on the semaphore. + +- A positive value, indicating that there are one or more post release operations. + +## Exception Management + +Exception takeover of UniProton is a maintenance and test feature that records as much information as possible when an exception occurs to facilitate subsequent fault locating. In addition, the exception hook function is provided so that users can perform special handling when an exception occurs. The exception takeover feature handles internal exceptions and external hardware exceptions. + +## CPU Usage Statistics + +The system CPU usage (CPU percentage, CPUP) in UniProton refers to the CPU usage of the system within a period of time. It reflects the CPU load and the system running status (idle or busy) in the given period of time. The valid range of the system CPUP is 0 to 10000, in basis points. 10000 indicates that the system is fully loaded. + +The thread CPUP refers to the CPU usage of a single thread. It reflects the thread status, busy or idle, in a period of time. The valid range of the thread CPUP is 0 to 10000, in basis points. 10000 indicates that the process is being executed for a period of time. The total CPUPs of all threads (including interrupts and idle tasks) in a single-core system is 10000. + +The system-level CPUP statistics of UniProton depends on the tick module, which is implemented by tick sampling idle tasks or idle software interrupt counter. + +## STM32F407ZGT6 Development Board Support + +The kernel peripheral startup process and board driver of UniProton supports the STM32F407ZGT6 development board. The directory structure is as follows: + +├─apps # Demo based on the real-time OS of UniProton +│ └─hello_world # hello_world example program +├─bsp # Board-level driver to interconnect with the OS +├─build # Build script to build the final image +├─config # Configuration items to adjust running parameters +├─include # APIs provided by the real-time OS of UniProton +└─libs # Static libraries of the real-time OS of UniProton. The makefile example in the build directory has prepared the reference of the header file and static libraries. + +## OpenAMP Hybrid Deployment + +OpenAMP is an open source software framework designed to standardize the interaction between environments in heterogeneous embedded systems through open source solutions based on asymmetric multi-processing. OpenAMP consists of the following components: + +1. Remoteproc manages the life cycle of the slave core, shared memory, and resources such as buffer and vring used for communication, and initializes RPMsg and virtio. +2. RPMsg enables multi-core communication based on virtio. +3. Virtio, which is a paravirtualization technology, uses a set of virtual I/Os to implement driver communication between the master and slave cores. +4. libmetal shields OS implementation details, provides common user APIs to access devices, and handles device interrupts and memory requests. + +## POSIX Standard APIs + +[UniProton supports POSIX standard APIs](./uniproton-apis.md). diff --git a/docs/en/server/_toc.yaml b/docs/en/server/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..57158c903134fa020bcae081c8f9b1bf64a8c204 --- /dev/null +++ b/docs/en/server/_toc.yaml @@ -0,0 +1,63 @@ +label: Server +sections: + - label: Release Notes + sections: + - href: ./releasenotes/releasenotes/_toc.yaml + - label: Quick Start + sections: + - href: ./quickstart/quickstart/_toc.yaml + - label: Installation and Upgrade + sections: + - href: ./installation_upgrade/installation/_toc.yaml + - href: ./installation_upgrade/upgrade/_toc.yaml + - label: OS Administration + sections: + - href: ./administration/administrator/_toc.yaml + - href: ./administration/sysmaster/_toc.yaml + - href: ./administration/compa_command/_toc.yaml + - label: O&M + sections: + - href: ./maintenance/aops/_toc.yaml + - href: ./maintenance/gala/_toc.yaml + - href: ./maintenance/sysmonitor/_toc.yaml + - href: ./maintenance/kernel_live_upgrade/_toc.yaml + - href: ./maintenance/syscare/_toc.yaml + - href: ./maintenance/common_skills/_toc.yaml + - href: ./maintenance/common_tools/_toc.yaml + - href: ./maintenance/troubleshooting/_toc.yaml + - label: Security + sections: + - href: ./security/secharden/_toc.yaml + - href: ./security/trusted_computing/_toc.yaml + - href: ./security/secgear/_toc.yaml + - href: ./security/cve-ease/_toc.yaml + - href: ./security/cert_signature/_toc.yaml + - href: ./security/sbom/_toc.yaml + - href: ./security/shangmi/_toc.yaml + - label: Memory and Storage + sections: + - href: ./memory_storage/lvm/_toc.yaml + - href: ./memory_storage/etmem/_toc.yaml + - href: ./memory_storage/gmem/_toc.yaml + - href: ./memory_storage/hsak/_toc.yaml + - label: Network + sections: + - href: ./network/network_config/_toc.yaml + - href: ./network/gazelle/_toc.yaml + - label: Performance Optimization + sections: + - href: ./performance/overall/_toc.yaml + - href: ./performance/tuning_framework/_toc.yaml + - href: ./performance/cpu_optimization/_toc.yaml + - href: ./performance/system_optimization/_toc.yaml + - label: Application Development + sections: + - href: ./development/application_dev/_toc.yaml + - href: ./development/gcc/_toc.yaml + - label: High Availability + sections: + - href: ./high_availability/ha/_toc.yaml + - label: Diversified Computing + sections: + - href: ./diversified_computing/dpu_offload/_toc.yaml + - href: ./diversified_computing/dpu_os/_toc.yaml diff --git a/docs/en/server/administration/administrator/_toc.yaml b/docs/en/server/administration/administrator/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..730acb79649d873d2d2dee8b095b23f91ee9e209 --- /dev/null +++ b/docs/en/server/administration/administrator/_toc.yaml @@ -0,0 +1,30 @@ +label: Administrator Guide +isManual: true +href: ./administration.md +description: Common administration operations on openEuler +sections: + - label: Viewing System Information + href: ./viewing-system-information.md + - label: Basic Configuration + href: ./basic-configuration.md + - label: User and User Group Management + href: ./user-and-user-group-management.md + - label: Software Package Management with DNF + href: ./using-dnf-to-manage-software-packages.md + - label: Service Management + href: ./service-management.md + - label: Process Management + href: ./process-management.md + - label: Service Configuration + href: ./configuring-services.md + sections: + - label: Configuring the Repo Server + href: ./configuring-the-repo-server.md + - label: Configuring the FTP Server + href: ./configuring-the-ftp-server.md + - label: Configuring the Web Server + href: ./configuring-the-web-server.md + - label: Setting Up the Database Server + href: ./setting-up-the-database-server.md + - label: Common Issues and Solutions + href: ./common-issues-and-solutions.md diff --git a/docs/en/server/administration/administrator/administration.md b/docs/en/server/administration/administrator/administration.md new file mode 100644 index 0000000000000000000000000000000000000000..0c32e256b1d6e2e2f7b268b47f7466b660f7744b --- /dev/null +++ b/docs/en/server/administration/administrator/administration.md @@ -0,0 +1,4 @@ +# Administrator Guide + +This document provides common administrator operations of the openEuler system to help administrators better use the system. +This document is intended for all administrators who use openEuler. diff --git a/docs/en/server/administration/administrator/basic-configuration.md b/docs/en/server/administration/administrator/basic-configuration.md new file mode 100644 index 0000000000000000000000000000000000000000..670b59612359357904055f7be15931b6be4bd7b9 --- /dev/null +++ b/docs/en/server/administration/administrator/basic-configuration.md @@ -0,0 +1,456 @@ +# Basic Configuration + + + +- [Basic Configuration](#basic-configuration) + - [Setting the System Locale](#setting-the-system-locale) + - [Displaying the Current Locale Status](#displaying-the-current-locale-status) + - [Listing Available Locales](#listing-available-locales) + - [Setting the Locale](#setting-the-locale) + - [Setting the Keyboard Layout](#setting-the-keyboard-layout) + - [Displaying the Current Settings](#displaying-the-current-settings) + - [Listing Available Keyboard Layouts](#listing-available-keyboard-layouts) + - [Setting the Keyboard Layout](#setting-the-keyboard-layout-1) + - [Setting the Date and Time](#setting-the-date-and-time) + - [Using the timedatectl Command](#using-the-timedatectl-command) + - [Displaying the Current Date and Time](#displaying-the-current-date-and-time) + - [Synchronizing the System Clock with a Remote Server](#synchronizing-the-system-clock-with-a-remote-server) + - [Changing the Current Date](#changing-the-current-date) + - [Changing the Current Time](#changing-the-current-time) + - [Changing the Time Zone](#changing-the-time-zone) + - [Using the date Command](#using-the-date-command) + - [Displaying the Current Date and Time](#displaying-the-current-date-and-time-1) + - [Changing the Current Time](#changing-the-current-time-1) + - [Changing the Current Date](#changing-the-current-date-1) + - [Using the hwclock Command](#using-the-hwclock-command) + - [Real-Time Clock and System Clock](#real-time-clock-and-system-clock) + - [Displaying the Current Date and Time](#displaying-the-current-date-and-time-2) + - [Setting the Date and Time](#setting-the-date-and-time-1) + - [Setting kdump](#setting-kdump) + - [Setting the Memory Reserved for kdump](#setting-the-memory-reserved-for-kdump) + - [Parameter Formats of the Memory Reserved for kdump](#parameter-formats-of-the-memory-reserved-for-kdump) + - [Recommended Reserved Memory](#recommended-reserved-memory) + - [Disabling Network Drivers](#disabling-network-drivers) + + + +## Setting the System Locale + +System locale settings are stored in the /etc/locale.conf file and can be modified by the localectl command. These settings are read at system boot by the systemd daemon. + +### Displaying the Current Locale Status + +To display the current locale status, run the following command: + +```shell +localectl status +``` + +Example command output: + +```shell +$ localectl status + System Locale: LANG=zh_CN.UTF-8 + VC Keymap: cn + X11 Layout: cn +``` + +### Listing Available Locales + +To display available locales, run the following command: + +```shell +localectl list-locales +``` + +You can check that by listing all Chinese locales with the following command: + +```shell +$ localectl list-locales | grep zh +zh_CN.UTF-8 +``` + +### Setting the Locale + +To set the language environment, run the following command as the user **root**. In the command, _locale_ indicates the language type to be set. Run the **localectl list-locales** command to obtain the value range. Change the value as required. + +```shell +localectl set-locale LANG=locale +``` + +For example, if you want to use Simplified Chinese as the locale, run the following command as the user **root**: + +```shell +localectl set-locale LANG=zh_CN.UTF-8 +``` + +> [!NOTE]NOTE +> After the modification, log in again or run the command `source /etc/locale.conf` as the user **root** to update the configuration file for the modification to take effect: + +## Setting the Keyboard Layout + +Keyboard layout settings are stored in the /etc/locale.conf file and can be modified by the localectl command. These settings are read at early boot by the systemd daemon. + +### Displaying the Current Settings + +To display the current keyboard layout settings, run the following command: + +```shell +localectl status +``` + +Example command output: + +```shell +$ localectl status + System Locale: LANG=zh_CN.UTF-8 + VC Keymap: cn + X11 Layout: cn +``` + +### Listing Available Keyboard Layouts + +To list all available keyboard layouts that can be configured on openEuler, run the following command: + +```shell +localectl list-keymaps +``` + +For example, the command output of the Chinese keyboard layout is as follows: + +```shell +$ localectl list-keymaps | grep cn +cn +``` + +### Setting the Keyboard Layout + +To set the keyboard layout, run the following command as the user **root**. In the command, _map_ indicates the keyboard layout to be set. Run the **localectl list-keymaps** command to obtain the value range. Change it as required. + +```shell +localectl set-keymap map +``` + +The keyboard layout will be equally applied to graphical user interfaces. + +Then you can verify if your setting was successful by checking the status: + +```shell +$ localectl status + System Locale: LANG=zh_CN.UTF-8 + VC Keymap: cn + X11 Layout: us +``` + +## Setting the Date and Time + +This topic describes how to set the system date, time, and time zone by using timedatectl, date, and hwclock commands. + +### Using the timedatectl Command + +#### Displaying the Current Date and Time + +To display the current date and time, run the following command: + +```shell +timedatectl +``` + +Example command output: + +```shell +$ timedatectl + Local time: Mon 2019-09-30 04:05:00 EDT + Universal time: Mon 2019-09-30 08:05:00 UTC + RTC time: Mon 2019-09-30 08:05:00 + Time zone: China Standard Time (CST), UTC +8 +System clock synchronized: no + NTP service: inactive + RTC in local TZ: no +``` + +#### Synchronizing the System Clock with a Remote Server + +Your system clock can be automatically synchronized with a remote server using the Network Time Protocol (NTP). Run the following command as the user **root** to enable or disable NTP. The value of _boolean_ is **yes** or **no**, indicating that the NTP is enabled or disabled for automatic system clock synchronization. Change the value as required. + +> [!NOTE]NOTE +> If the remote NTP server is enabled to automatically synchronize the system clock, you cannot manually change the date and time. If you need to manually change the date or time, ensure that automatic NTP system clock synchronization is disabled. You can run the **timedatectl set-ntp no** command to disable the NTP service. + +```shell +timedatectl set-ntp boolean +``` + +For example, to enable automatic remote time synchronization, run the following command: + +```shell +timedatectl set-ntp yes +``` + +#### Changing the Current Date + +> [!NOTE]NOTE +> Before changing the date, ensure that automatic NTP system clock synchronization has been disabled. + +Run the following command as the user **root** to change the current date. In the command, _YYYY_ indicates the year, _MM_ indicates the month, and _DD_ indicates the day. Change them as required. + +```shell +timedatectl set-time YYYY-MM-DD +``` + +For example, to change the current date to August 14, 2019, run the following command as the user **root**: + +```shell +timedatectl set-time '2019-08-14' +``` + +#### Changing the Current Time + +> [!NOTE]NOTE +> Before changing the time, ensure that automatic NTP system clock synchronization has been disabled. + +To change the current time, run the following command as the user **root**. In the command, _HH_ indicates the hour, _MM_ indicates the minute, and _SS_ indicates the second. Change them as required. + +```shell +timedatectl set-time HH:MM:SS +``` + +For example, to change the current time to 15:57:24, run the following command: + +```shell +timedatectl set-time 15:57:24 +``` + +#### Changing the Time Zone + +To list all available time zones, run the following command: + +```shell +timedatectl list-timezones +``` + +To change the current time zone, run the following command as the user **root**. In the command, _time\_zone_ indicates the time zone to be set. Change it as required. + +```shell +timedatectl set-timezone time_zone +``` + +Imagine you want to identify which time zone is closest to your present location while you are in Asia. You can check that by listing all available time zones in Asia with the following command: + +```shell +$ timedatectl list-timezones | grep Asia +Asia/Aden +Asia/Almaty +Asia/Amman +Asia/Anadyr +Asia/Aqtau +Asia/Aqtobe +Asia/Ashgabat +Asia/Baghdad +Asia/Bahrain +...... + +Asia/Seoul +Asia/Shanghai +Asia/Singapore +Asia/Srednekolymsk +Asia/Taipei +Asia/Tashkent +Asia/Tbilisi +Asia/Tehran +Asia/Thimphu +Asia/Tokyo +``` + +To change the time zone to Asia/Shanghai, run the following command: + +```shell +timedatectl set-timezone Asia/Shanghai +``` + +### Using the date Command + +#### Displaying the Current Date and Time + +To display the current date and time, run the following command: + +```shell +date +``` + +By default, the **date** command displays the local time. To display the time in Coordinated Universal Time (UTC), run the command with the --utc or -u command line option: + +```shell +date --utc +``` + +You can also customize the format of the displayed information by providing the + "format" option on the command line: + +```shell +date +"format" +``` + +**Table 1** Formatting options + +| Format Option | Description | +| :---- | :---- | +| %H | The hour in the HH format (for example, 17) | +| %M | The minute in the MM format (for example, 37) | +| %S | The second in the SS format (for example, 25) | +| %d | The day of the month in the DD format (for example, 15) | +| %m | The month in the MM format (for example, 07) | +| %Y | The year in the YYYY format (for example, 2019) | +| %Z | The time zone abbreviation (for example, CEST) | +| %F | The full date in the YYYY-MM-DD format (for example, 2019-7-15). This option is equal to %Y-%m-%d | +| %T | The full time in the HH:MM:SS format (for example, 18:30:25). This option is equal to %H:%M:%S | + +Example commands and outputs: + +- To display the current date and time: + + ```shell + $ date + Sat Aug 17 17:26:34 CST 2019 + ``` + +- To display the current date and time in UTC: + + ```shell + $ date --utc + Sat Aug 17 09:26:18 UTC 2019 + ``` + +- To customize the output of the date command: + + ```shell + $ date +"%Y-%m-%d %H:%M" + 2019-08-17 17:24 + ``` + +#### Changing the Current Time + +To change the current time, run the date command with the --set or -s option as the root user: Run the following command as the user **root**. In the command, _HH_ indicates the hour, _MM_ indicates the minute, and _SS_ indicates the second. Change them as required. + +```shell +date --set HH:MM:SS +``` + +By default, the date command sets the local time. To set the system clock in UTC instead, run the command with the --utc or -u command line option: + +```shell +date --set HH:MM:SS --utc +``` + +For example, to change the current time to 23:26:00, run the following command as the user **root**: + +```shell +date --set 23:26:00 +``` + +#### Changing the Current Date + +To change the current date, run the command with the --set or -s command line option. Run the following command as the user **root**. In the command, _YYYY_ indicates the year, _MM_ indicates the month, and _DD_ indicates the day. Change them as required. + +```shell +date --set YYYY-MM-DD +``` + +For example, to change the current date to November 2, 2019, run the following command as the user **root**: + +```shell +date --set 2019-11-02 +``` + +### Using the hwclock Command + +You can run the hwclock command to set the real time clock (RTC). + +#### Real-Time Clock and System Clock + +Linux divides clocks into the following types: + +- System clock: clock of the current Linux kernel. +- Hardware clock RTC: hardware clock of the mainboard powered by the battery. This clock can be set in the **Standard BIOS Feature** option of the BIOS. + +When Linux starts, it reads the RTC and sets the system clock time based on the RTC time. + +#### Displaying the Current Date and Time + +To display the current RTC date and time, run the following command as the user **root**: + +```shell +hwclock +``` + +Example command output: + +```shell +$ hwclock +2019-08-26 10:18:42.528948+08:00 +``` + +#### Setting the Date and Time + +Run the following command as the user **root** to change the date and time of the current hardware. In the command, _dd_ indicates the day, _mm_ indicates the month, _yyyy_ indicates the year, _HH_ indicates the hour, and _MM_ indicates the minute. Change them as required. + +```shell +hwclock --set --date "dd mm yyyy HH:MM" +``` + +For example, to change the current time to 21:17 on October 21, 2019, run the following command: + +```shell +hwclock --set --date "21 Oct 2019 21:17" --utc +``` + +## Setting kdump + +This section describes how to set the memory reserved for kdump and modify parameters in the kdump configuration file. + +### Setting the Memory Reserved for kdump + +#### Parameter Formats of the Memory Reserved for kdump + +The memory reserved for kdump must be added to the bootargs in the **/boot/efi/EFI/openEuler/grub.cfg** configuration file. The memory reserved for kdump has been added to the released openEuler version by default and can be adjusted as required. After adding or modifying the bootargs, restart the system for the setting to take effect. The parameter formats of the memory reserved for kdump are as follows: + +| Bootarg| Description| Default Value| Remarks| +|----------|----------|----------|----------| +| crashkernel=x| If the physical memory size is less than 4 GB, x of the memory is reserved for kdump.| The default value is 512 MB for x86.| This configuration method is used only when the available memory size is less than 4 GB. In this case, ensure that the available contiguous memory is sufficient for reservation.| +| crashkernel=x@y| x of the memory is reserved at the start address of y for kdump.| Unused| Ensure that x of the memory at the start address of y is not reserved for other modules.| +| crashkernel=x,high| If the physical memory size is less than 4 GB, 256 MB memory is reserved. If the physical memory size is greater than 4 GB, x of the memory is reserved for kdump. | The default value is 1024M,high for ARM64.| Ensure that the available physical contiguous memory size is greater than or equal to 256 MB when the memory size is less than 4 GB, and is greater than or equal to x when the memory size is greater than 4 GB. The actual reserved memory size is 256 MB + x. | +| crashkernel=x,low crashkernel=y,high| x of the memory is reserved for kdump when the physical memory size is less than 4 GB, and y of the memory is reserved for kdump when the physical memory size is greater than 4 GB. | Unused| Ensure that the available physical contiguous memory size is greater than or equal to x when the physical memory size is less than 4 GB, and is greater than or equal to y when the physical memory size is greater than 4 GB.| + +### Recommended Reserved Memory + +| Recommended Solution| Reserved Parameter| Description| +|----------|----------|----------| +| General solution| crashkernel=2048M,high| If the memory size is less than 4 GB, 256 MB is reserved for kdump. If the memory size is greater than 4 GB, 2048 MB is reserved for kdump. 256 + 2048 MB in total.| +| Economical solution| crashkernel=1024M,high| If the memory size is less than 4 GB, 256 MB is reserved for kdump. If the memory size is greater than 4 GB, 1024 MB is reserved for kdump. 256 + 1024 MB in total. It is recommended that kdump files not be dumped using the network in scenarios where the system memory size is less than 512 GB. In VM scenarios, you can reduce the reserved memory. You are advised to set crashkernel to 512M or crashkernel to 256M,high.| + +> [!NOTE]NOTE +> If kdump files are not dumped using the network, you need to set the kdump file system not to pack network drivers. Loading the network driver requires a large amount of memory. As a result, the memory reserved for kdump may be insufficient and kdump may fail. Therefore, you are advised to disable network drivers. + +### Disabling Network Drivers + +In the kdump configuration file **/etc/kdump.conf**, the dracut parameters can be used to set the tailored driver module. You can configure the network driver to the tailored driver list to prevent the kdump file system from loading the driver. After the configuration file is modified, restart the kdump service for the modification to take effect. Set the dracut parameters as follows: + +`dracut_args --omit-drivers "mdio-gpi usb_8dev et1011c rt2x00usb bcm-phy-lib mac80211_hwsim rtl8723be rndis_host hns3_cae amd vrf rtl8192cu mt76x02-lib int51x1 ppp_deflate team_mode_loadbalance smsc911x aweth bonding mwifiex_usb hnae dnet rt2x00pci vaser_pci hdlc_ppp marvell rtl8xxxu mlxsw_i2c ath9k_htc rtl8150 smc91x cortina at803x rockchip cxgb4 spi_ks8995 mt76x2u smsc9420 mdio-cavium bnxt_en ch9200 dummy macsec ice mt7601u rtl8188ee ixgbevf net1080 liquidio_vf be2net mlxsw_switchx2 gl620a xilinx_gmii2rgmii ppp_generic rtl8192de sja1000_platform ath10k_core cc770_platform realte igb c_can_platform c_can ethoc dm9601 smsc95xx lg-vl600 ifb enic ath9 mdio-octeon ppp_mppe ath10k_pci cc770 team_mode_activebackup marvell10g hinic rt2x00lib mlx4_en iavf broadcom igc c_can_pci alx rtl8192se rtl8723ae microchip lan78xx atl1c rtl8192c-common almia ax88179_178a qed netxen_nic brcmsmac rt2800usb e1000 qla3xxx mdio-bitbang qsemi mdio-mscc-miim plx_pci ipvlan r8152 cx82310_eth slhc mt76x02-usb ems_pci xen-netfront usbnet pppoe mlxsw_minimal mlxsw_spectrum cdc_ncm rt2800lib rtl_usb hnae3 ath9k_common ath9k_hw catc mt76 hns_enet_drv ppp_async huawei_cdc_ncm i40e rtl8192ce dl2 qmi_wwan mii peak_usb plusb can-dev slcan amd-xgbe team_mode_roundrobin ste10Xp thunder_xcv pptp thunder_bgx ixgbe davicom icplus tap tun smsc75xx smsc dlci hns_dsaf mlxsw_core rt2800mmi softing uPD60620 vaser_usb dp83867 brcmfmac mwifiex_pcie mlx4_core micrel team macvlan bnx2 virtio_net rtl_pci zaurus hns_mdi libcxgb hv_netvsc nicvf mt76x0u teranetics mlxfw cdc_eem qcom-emac pppox mt76-usb sierra_net i40evf bcm87xx mwifiex pegasus rt2x00mmi sja1000 ena hclgevf cnic cxgb4vf ppp_synctty iwlmvm team_mode_broadcast vxlan vsockmon hdlc_cisc rtl8723-common bsd_comp fakelb dp83822 dp83tc811 cicada fm10 8139t sfc hs geneve hclge xgene-enet-v2 cdc_mbim hdlc asix netdevsim rt2800pci team_mode_random lxt ems_usb mlxsw_pci sr9700 mdio-thunder mlxsw_switchib macvtap atlantic cdc_ether mcs7830 nicpf mdi peak_pci atl1e cdc_subset ipvtap btcoexist mt76x0-common veth slip iwldvm bcm7xxx vitesse netconsole epic100 myri10ge r8169 qede microchip_t1 liquidi bnx2x brcmutil mwifiex_sdi mlx5_core rtlwifi vmxnet3 nlmon hns3 hdlc_raw esd_usb2 atl2 mt76x2-common iwlwifi mdio-bcm-unimac national ath rtwpci rtw88 nfp rtl8821ae fjes thunderbolt-net 8139cp atl1 mscc vcan dp83848 dp83640 hdlc_fr e1000e ipheth net_failover aquantia rtl8192ee igbvf rocker intel-xway tg3" --omit "ramdisk network ifcfg qemu-net" --install "chmod" --nofscks` + +## Setting the Disk Scheduling Algorithm + +This section describes how to set the disk scheduling algorithm. + +### Temporarily Modifying the Scheduling Policy + +For example, if all I/O scheduling algorithms are changed to **mq-deadline**, the modification becomes invalid after the system is restarted. + +```shell +echo mq-deadline > /sys/block/sd*/queue/scheduler +``` + +### Permanently Setting the Scheduling Policy + +You can add **elevator=mq-deadline** to the kernel line in the kernel boot configuration file **grub.cfg**. The setting takes effect after the system is restarted. + +```shell +linux /vmlinuz-4.19.90-2003.4.0.0036.oe1.x86_64 root=/dev/mapper/openeuler-root ro resume=/dev/mapper/openeuler-swap rd.lvm.lv=openeuler/root rd.lvm.lv=openeuler/swap quiet crashkernel=512M elevator=mq-deadline +``` diff --git a/docs/en/server/administration/administrator/common-issues-and-solutions.md b/docs/en/server/administration/administrator/common-issues-and-solutions.md new file mode 100644 index 0000000000000000000000000000000000000000..c232d7b8293c508bc1ab7b22756830ecf29b0de5 --- /dev/null +++ b/docs/en/server/administration/administrator/common-issues-and-solutions.md @@ -0,0 +1,395 @@ +# Common Issues and Solutions + +## Issue 1: Why Is the Memory Usage of the libvirtd Service Queried by Running the systemctl and top Commands Different + +### Symptom + +The output of the **systemctl** and **systemd-cgtop** commands shows that the libvirtd service occupies more than 1.5 GB memory, but the output of the **top** command shows that the libvirtd service occupies about 70 MB memory. + +### Possible Cause + +The memory displayed in the services \(including systemctl and systemd-cgtop\) managed by systemd can be obtained from **memory.usage\_in\_bytes** in Cgroup. Running the **top** command is to query the memory information in the **/proc** directory. The query results are different because the statistical method varies. + +Generally, the memory used by service processes has the following types: + +- anon\_rss: anonymous pages in user mode address spaces, for example, memory allocated by calling the malloc function or the mmap function with configured **MAP\_ANONYMOUS**. When the system memory is insufficient, this type of memory can be swapped by the kernel. +- file\_rss: mapped pages in user mode address spaces, including map file \(such as mmap of a specified file\) and map tmpfs \(such as IPC shared memory\). When the system memory is insufficient, the kernel can reclaim these pages. Data may need to be synchronized between the kernel and map file before reclamation. +- file\_cache: file cache \(page in page cache of disk file\), which is generated when a file is read or written. When the system memory is insufficient, the kernel can reclaim these pages. Data may need to be synchronized between the kernel and map file before reclamation. +- buffer pages: belongs to page cache, for example, cache generated when block device files are read. + +anon\_rss and file\_rss belong to the resident set size \(RSS\) of processes, and file\_cache and buffer pages belong to page cache. In brief: + +RSS in the output of the **top** command = anon\_rss + file\_rss; Shared memory \(SHR\) = file\_rss + +**memory.usage\_in\_bytes** in Cgroup = cache + RSS + swap + +In conclusion, the definition of memory usage obtained by running the **systemd** command is different from that obtained by running the **top** command. Therefore, the query results are different. + +## Issue 2: An Error Occurs When stripsize Is Set to 4 During RAID 0 Volume Configuration + +### Symptom + +An error occurs when the **stripsize** parameter is set to **4** during RAID 0 volume configuration. + +### Possible Cause + +The 64 KB page table can be enabled only in the scenario where **stripsize** is set to **64**. + +### Solution + +You do not need to modify the configuration file. When running the **lvcreate** command on openEuler, set **stripesize** to **64** because the minimum supported stripe size is 64 KB. + +## Issue 3: Failed to Compile MariaDB Using rpmbuild + +### Symptom + +When you log in to the system as user **root** and run the **rpmbuild** command to compile the MariaDB source code, the compilation fails and the following information is displayed: + +```text ++ echo 'mysql can'\''t run test as root' +mysql can't run test as root ++ exit 1 +``` + +### Possible Cause + +The MariaDB does not allow user **root** to execute test cases. However, test cases are automatically executed during compilation. As a result, the compilation process is blocked. + +### Solution + +Use a text editor, such as vi, to modify the value of the **runtest** variable in the **mariadb.spec** file. + +Before the modification: + +```text +%global runtest 1 +``` + +After the modification: + +```text +%global runtest 0 +``` + +The modification disables the function of executing test cases during compilation, which does not affect the compilation and the RPM package content after compilation. + +## Issue 4: Failed to Start the SNTP Service Using the Default Configuration + +### Symptom + +The SNTP service fails to be started with the default configuration. + +### Possible Cause + +The domain name of the NTP server is not added to the default configuration. + +### Solution + +Modify the **/etc/sysconfig/sntp** file and add the domain name of the NTP server in China: **0.generic.pool.ntp.org**. + +## Issue 5: Installation Failure Caused by Software Package Conflict, File Conflict, or Missing Software Package + +### Symptom + +Software package conflict, file conflict, or missing software packages may occur during software package installation. As a result, the upgrade is interrupted and the installation fails. The error information about software package conflict, file conflict, and missing software packages is as follows: + +The following is an example of software package conflict error information \(the conflict between **libev-libevent-devel-4.24-11.oe1.aarch64** and **libevent-devel-2.1.11-2.oe1.aarch64** is used as an example\): + +```text +package libev-libevent-devel-4.24-11.oe1.aarch64 conflicts with libevent-devel provided by libevent-devel-2.1.11-2.oe1.aarch64 + - cannot install the best candidate for the job + - conflicting requests +``` + +The following is an example of file conflict error information \(the **/usr/bin/containerd** file conflict is used as an example\): + +```text +Error: Transaction test error: + file /usr/bin/containerd from install of containerd-1.2.0-101.oe1.aarch64 conflicts with file from package docker-engine-18.09.0-100.aarch64 + file /usr/bin/containerd-shim from install of containerd-1.2.0-101.oe1.aarch64 conflicts with file from package docker-engine-18.09.0-100.aarch64 +``` + +The following is an example of the error message indicating that the **blivet-data** software package is missing: + +```text +Error: + Problem: cannot install both blivet-data-1:3.1.1-6.oe1.noarch and blivet-data-1:3.1.1-5.noarch + - package python2-blivet-1:3.1.1-5.noarch requires blivet-data = 1:3.1.1-5, but none of the providers can be installed + - cannot install the best update candidate for package blivet-data-1:3.1.1-5.noarch + - problem with installed package python2-blivet-1:3.1.1-5.noarch(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages) +``` + +### Possible Cause + +- In the software packages provided by openEuler, some software packages have different names but the same functions. As a result, the software packages cannot be installed at the same time. +- In the software packages provided by openEuler, some software packages have different names but the same functions. As a result, the files after installation are the same, causing file conflict. +- Some software packages are depended on by other software packages before the upgrade. After the software packages are upgraded, the software packages that depend on them may fail to be installed due to lack of software packages. + +### Solution + +If a software package conflict occurs, perform the following steps \(the software package conflict in "Symptom" is used as an example\): + +1. According to the error message displayed during the installation, the software package that conflicts with the to-be-installed software package **libev-libevent-devel-4.24-11.oe1.aarch64** is **libevent-devel-2.1.11-2.oe1.aarch64**. +2. Run the **dnf remove** command to uninstall the software package that conflicts with the software package to be installed. + + ```shell + dnf remove libevent-devel-2.1.11-2.oe1.aarch64 + ``` + +3. Perform the installation again. + +If a file conflict occurs, perform the following steps \(the file conflict in "Symptom" is used as an example\): + +1. According to the error message displayed during the installation, the names of the software packages that cause the file conflict are **containerd-1.2.0-101.oe1.aarch64** and **docker-engine-18.09.0-100.aarch64**. +2. Record the names of the software packages that do not need to be installed. The following uses **docker-engine-18.09.0-100.aarch64** as an example. +3. Run the **dnf remove** command to uninstall the software package that does not need to be installed. + + ```shell + dnf remove docker-engine-18.09.0-100.aarch64 + ``` + +4. Perform the installation again. + +If a software package is missing, perform the following steps \(the missed software package in "Symptom" is used as an example\): + +1. Determine the name of the software package to be upgraded \(**blivet-data-1:3.1.1-5.noarch**\) and the name of the dependent software package \(**python2-blivet-1:3.1.1-5.noarch**\) based on the error information displayed during the upgrade. +2. Run the **dnf remove** command to uninstall the software package that depends on the upgrade package or add the **\-\-allowerasing** parameter when upgrading the software package. + - Run the **dnf remove** command to uninstall the software package that depends on the **blivet-data-1:3.1.1-5.noarch** software package. + + ```shell + dnf remove python2-blivet-1:3.1.1-5.noarch + ``` + + - Add the **\-\-allowerasing** parameter when upgrading the software package. + + ```shell + yum update blivet-data-1:3.1.1-5.noarch -y --allowerasing + ``` + +3. Perform the upgrade again. + +### Installing Conflicting Instances + +- A file conflict occurs. + + The **python3-edk2-devel.noarch** file conflicts with the **build.noarch** file due to duplicate file names. + + ```shell + $ yum install python3-edk2-devel.noarch build.noarch + ... + Error: Transaction test error: + file /usr/bin/build conflicts between attempted installs of python3-edk2-devel-202002-3.oe1.noarch and build-20191114-324.4.oe1.noarch + ``` + +## Issue 6: Failed to Downgrade libiscsi + +### Symptom + +libiscsi-1.19.4 or later fails to be downgraded to libiscsi-1.19.3 or earlier. + +```text +Error: +Problem: problem with installed package libiscsi-utils-1.19.0-4.oe1.x86_64 +- package libiscsi-utils-1.19.0-4.oe1.x86_64 requires libiscsi(x86-64) = 1.19.0-4.oe1, but none of the providers can be installed +- cannot install both libiscsi-1.19.0-3.oe1.x86_64 and libiscsi-1.19.0-4.oe1.x86_64 +- cannot install both libiscsi-1.19.0-4.oe1.x86_64 and libiscsi-1.19.0-3.oe1.x86_64 +- conflicting requests +(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages) +``` + +### Possible Cause + +In libiscsi-1.19.3 or earlier, binary files named **iscsi-xxx** are packed into the main package **libiscsi**. However, these binary files introduce improper dependency CUnit. To solve this problem, in libiscsi-1.19.4, these binary files are separated into the **libiscsi-utils** subpackage. The main package is weakly dependent on the subpackage. You can integrate or uninstall the subpackage during image building based on product requirements. If the subpackage is not integrated or is uninstalled, the functions of the **libiscsi** main package are not affected. +When libiscsi-1.19.4 or later is downgraded to libiscsi-1.19.3 or earlier and the **libiscsi-utils** subpackage is installed in the system, because libiscsi-1.19.3 or earlier does not contain **libiscsi-utils**, **libiscsi-utils** will fail to be downgraded. Due to the fact that **libiscsi-utils** depends on the **libiscsi** main package before the downgrade, a dependency problem occurs and the libiscsi downgrade fails. + +### Solution + +Run the following command to uninstall the **libiscsi-utils** subpackage and then perform the downgrade: + +```shell +yum remove libiscsi-utils +``` + +## Issue 7: Failed to Downgrade xfsprogs + +### Symptom + +xfsprogs-5.6.0-2 or later fails to be downgraded to xfsprogs-5.6.0-1 or earlier. + +```text +Error: +Problem: problem with installed package xfsprogs-xfs_scrub-5.6.0-2.oe1.x86_64 +- package xfsprogs-xfs_scrub-5.6.0-2.oe1.x86_64 requires xfsprogs = 5.6.0-2.oe1, but none of the providers can be installed +- cannot install both xfsprogs-5.6.0-1.oe1.x86_64 and xfsprogs-5.6.0-2.oe1.x86_64 +- cannot install both xfsprogs-5.6.0-2.oe1.x86_64 and xfsprogs-5.6.0-1.oe1.x86_64 +- conflicting requests +``` + +### Possible Cause + +In xfsprogs-5.6.0-2, to reduce improper dependencies of the **xfsprogs** main package and separate experimental commands from the main package, the `xfs_scrub*` commands are separated into the **xfsprogs-xfs_scrub** subpackage. The **xfsprogs** main package is weakly dependent on the **xfsprogs-xfs_scrub** sub-package. You can integrate or uninstall the subpackage during image creation based on product requirements. If the subpackage is not integrated or is uninstalled, the functions of the **xfsprogs** main package are not affected. + +When xfsprogs-5.6.0-2 or later is downgraded to xfsprogs-5.6.0-1 or earlier and the **xfsprogs-xfs_scrub** subpackage is installed in the system, because xfsprogs-5.6.0-1 or earlier does not contain **xfsprogs-xfs_scrub**, **xfsprogs-xfs_scrub** will fail to be downgraded. Due to the fact that **xfsprogs-xfs_scrub** depends on the **xfsprogs** main package before the downgrade, a dependency problem occurs and the xfsprogs downgrade fails. + +### Solution + +Run the following command to uninstall the **xfsprogs-xfs_scrub** subpackage and then perform the downgrade: + +```shell +yum remove xfsprogs-xfs_scrub +``` + +## Issue 8: Failed to Downgrade elfutils + +### Symptom + +The dependency is missing. As a result, elfutils failed to be downgraded. + +![](figures/1665628542704.png) + +### Possible Cause + +22.03-LTS, 22.03-LTS-Next: elfutils-0.185-12 + +master: elfutils-0.187-7 + +20.03-LTS-SP1: elfutils-0.180-9 + +In the preceding versions, the **eu-objdump**, **eu-readelf**, and **eu-nm** commands provided by the elfutils main package are split into the elfutils-extra subpackage. When elfutils-extra has been installed in the system and elfutils is downgraded, because an earlier version (such as the version of an earlier branch) cannot provide the corresponding elfutils-extra package, the elfutils-extra subpackage is not downgraded. However, the elfutils-extra subpackage depends on the elfutils package before the downgrade. As a result, the dependency problem cannot be resolved, and the elfutils subpackage fails to be downgraded. + +### Solution + +Run the following command to uninstall the elfutils-extra subpackage and then perform the downgrade: + +```shell +yum remove -y elfutils-extra +``` + +## Issue 9: CPython/Lib Detects CVE-2019-9674: Zip Bomb + +### Symptom + +**Lib/zipfile.py** in Python 3.7.2 or earlier allows remote attackers to create DoS requests using zip bombs, resulting in high resource consumption. + +### Possible Cause + +Remote attackers use zip bombs to cause denial of service, affecting target system services or even crashing the system. A zip bomb is a zip file with a high compression ratio. It may be several MB or dozens of MB in size. However, after decompression, a large amount of data is generated, consuming a large amount of resources. + +### Solution + +Add the alarm information to **zipfile** at . + +## Issue 10: ReDoS Attack Occurs Due to Improper Use of glibc Regular Expressions + +### Symptom + +The regcomp/regexec interface of glibc is used for programming, or the glibc regular expressions, such as grep/sed, are used in shell commands. Improper regular expressions or inputs cause ReDoS attacks (CVE-2019-9192/CVE-2018-28796). +The typical regular expression pattern is the combination of the"reverse reference (\1)" with the "asterisk (*)" (zero match or multiple matches), "plus sign (+)" (one match or multiple matches), or "{m,n}" (minimum match: m; maximum match: n); or the combination of ultra-long character strings with regular expressions. The following is an example: + +```shell +$ echo D | grep -E "$(printf '(\0|)(\\1\\1)*')"Segmentation fault (core dumped) +$ grep -E "$(printf '(|)(\\1\\1)*')" +Segmentation fault (core dumped) +$ echo A | sed '/\(\)\(\1\1\)*/p' +Segmentation fault (core dumped) +$ time python -c 'print "a"*40000' | grep -E "a{1,32767}" +Segmentation fault (core dumped) +$ time python -c 'print "a"*40900' | grep -E "(a)\\1" +Segmentation fault (core dumped) +``` + +### Possible Cause + +A core dump occurs on the process that uses the regular expression. The glibc regular expression is implemented using the NFA/DFA hybrid algorithm. The internal principle is to use a greedy algorithm for recursive query to match as many character strings as possible. The greedy algorithm causes the ReDoS attack when processing the recursive regular expression. + +### Solution + +1. Strict permission control is required to reduce the attack surface. + +2. Ensure that the regular expression is correct. Do not enter an invalid regular expression or a combination of ultra-long character strings with regular expressions (references or asterisks) that may trigger infinite recursion. + + ```text + # ()(\1\1)* + # "a"*400000 + ``` + +3. After a user program detects a process exception, the user program can restart the process to restore services, improving program reliability. + +## Issue 11: An Error Is Reported When gdbm-devel Is Installed or Uninstalled During the Installation and Uninstallation of httpd-devel and apr-util-devel + +### Symptom + +1. An error is reported when gdbm-devel-1.18.1-1 is installed or uninstalled. +2. After the error is rectified, gdbm and gdbm-devel are upgraded to the 1.18.1-2 version. However, the default version of gdbm-devel is still 1.18.1-1 when httpd-devel and apr-util-devel (dependent on gdbm-devel) are installed. As a result, the error persists. + +### Possible Cause + +1. The gdbm-devel-1.18.1-1 package does not contain the help package that provides `info`. As a result, the help package cannot be introduced when gdbm-devel is installed independently, and the following alarm information is displayed: + + ```text + install-info: No such file or directory for /usr/share/info/gdbm.info.gz + ``` + +2. By default, the gdbm-1.18.1-1 main package is installed in the system, but the gdbm-devel package is not installed. The software packages depending on gdbm-devel still match the version of the gdbm main package and install gdbm-devel-1.18.1-1. As a result, the error persists. + +### Solution + +1. Install gdbm-1.18.1-2 to upgrade gdbm. The error is rectified. +2. Upgrade gdbm, and then install gdbm-devel to make it depend on the gdbm of the later version. The error is rectified. + +## Issue 12: An rpmdb Error Is Reported When Running the yum or dnf Command After the System Is Rebooted + +### Symptom + +1. After the system is rebooted, an error is reported when running an RPM-related command `yum` or `dnf` as follows: + error: db5 error(-30973) from dbenv->open: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery + error: cannot open Packages index using db5 - (-30973) + error: cannot open Packages database in /var/lib/rpm + Error: Error: rpmdb open failed + +### Possible Cause + +1. During an installation or upgrade, read and write operations are performed on the **/var/lib/rpm/__db.00\*** file. If an unexpected interruption occurs, such as forced power-off, drive space full, or `kill -9`, the **_db** file will be damaged. An error will be reported when a `dnf` or `yum` command is executed. + +### Solution + +Step 1 Run the `kill -9` command to terminate all running RPM-related commands. +Step 2 Run `rm -rf /var/lib/rpm/__db.00*` to delete all db.00 files. +Step 3 Run the `rpmdb --rebuilddb` command to rebuild the RPM database. + +## Issue 13: Failed to Run `rpmrebuild -d /home/test filesystem` to Rebuild the filesystem Package + +### Symptom + +Failed to run the `rpmrebuild --comment-missing=y --keep-perm -b -d /home/test filesystem-3.16-3.oe1.aarch64` command to rebuild the **filesystem** package. The following information is displayed: + +```text +/usr/lib/rpmrebuild/rpmrebuild.sh:Error:(RpmBuild) Package 'filesystem-3.16-3.oe1.aarch64' build failed. +/usr/lib/rpmrebuild/rpmrebuild.sh:Error: RpmBuild +``` + +### Possible Cause + +The software package creates the directory in the **%pretrans -p** phase, and modify the directory in the **%ghost** phase. If you create a file or directory in the directory and use `rpmrebuild` to build the package, the created file or directory will be included in the package. + +The root cause of the symptom is that **filesystem** creates the **/proc** directory in the **%pretrans** phase and modifies the directory in the **%ghost** phase, but some small processes are dynamically created during system running. As a result, `rpmrebuild` cannot include the processes in the package because they are not files or directories and fails to rebuild the package. + +### Solution + +Do not use `rpmrebuild` to rebuild the **filesystem** package. + +## Issue 14: An Error Is Reported When modprobe or `insmod` Is Executed With the `-f` Option + +### Symptom + +An error is reported when `modprobe -f ` or `insmod -f .ko.xz` is executed. For example, when `insmod -f xfs.ko.xz` is executed, error message **modprobe: ERROR: could not insert 'xfs': Key was rejected by service** is displayed. + +Up till now (2022.09.20, kmod v30), this issue has not been fixed by the kmod community.Linux v5.17 [b1ae6dc](https://github.com/torvalds/linux/commit/b1ae6dc41eaaa98bb75671e0f3665bfda248c3e7) introduced support for compressed kernel modules, which is not supported by kmod. + +### Possible Cause + +`modprobe` and `insmod` use the `finit_module()` system call to load uncompressed ko files. For compressed ko files, kmod uses the `init_module()` system call to decompress them. `init_module()` does not take the ignore check flag. As a result, `mod_verify_sig()` is always executed by the kernel. the `-f` option of `modprobe` and `insmod` changes verification information about the ko file, resulting in verification failure of `mod_verify_sig()`. + +### Solution + +Do not use the `-f` option when running `insmod` or `modprobe` on compressed ko files. diff --git a/docs/en/server/administration/administrator/configuring-services.md b/docs/en/server/administration/administrator/configuring-services.md new file mode 100644 index 0000000000000000000000000000000000000000..e33439eb2e5ad90f0eaa7013ed262100489e21fd --- /dev/null +++ b/docs/en/server/administration/administrator/configuring-services.md @@ -0,0 +1 @@ +# Configuring Services diff --git a/docs/en/server/administration/administrator/configuring-the-ftp-server.md b/docs/en/server/administration/administrator/configuring-the-ftp-server.md new file mode 100644 index 0000000000000000000000000000000000000000..157c48573c74e5ae919b968f79217f4d4a546cae --- /dev/null +++ b/docs/en/server/administration/administrator/configuring-the-ftp-server.md @@ -0,0 +1,503 @@ +# Configuring the FTP Server + +## General Introduction + +### FTP Overview + +File Transfer Protocol \(FTP\) is one of the earliest transmission protocols on the Internet. It is used to transfer files between the server and client. FTP allows users to access files on a remote system using a set of standard commands without logging in to the remote system. In addition, the FTP server provides the following functions: + +- Subscriber classification + + By default, the FTP server classifies users into real users, guest users, and anonymous users based on the login status. The three types of users have different access permissions. Real users have complete access permissions, while anonymous users have only the permission to downloading resources. + +- Command records and log file records + + FTP can use the syslogd to record data, including historical commands and user transmission data \(such as the transmission time and file size\). Users can obtain log information from the /var/log/ directory. + +- Restricting the access scope of users + + FTP can limit the work scope of a user to the home directory of the user. After a user logs in to the system through FTP, the root directory displayed by the system is the home directory of the user. This environment is called change root \(chroot for short\). In this way, users can access only the main directory, but not important directories such as /etc, /home, and /usr/local. This protects the system and keeps the system secure. + +### Port Used by the FTP Server + +The FTP service requires multiple network ports. The server uses the following ports: + +- Command channel. The default port number is 21. +- Data channel. The default port number is 20. + +Port 21 is used to receive connection requests from the FTP client, and port 20 is used by the FTP server to proactively connect to the FTP client. + +### Introduction to vsftpd + +FTP has a long history and uses the unencrypted transmission mode, and is therefore considered insecure. This section describes the Very Secure FTP Daemon \(vsftpd\), to use FTP in a more secure way. + +The vsftpd is introduced to build a security-centric FTP server. The vsftpd is designed with the following features: + +- The startup user of the vsftpd service is a common user who has low system permission. In addition, the vsftpd service uses chroot to change the root directory, preventing the risk of misusing system tools. +- Any vsftpd command that requires high execution permission is controlled by a special upper-layer program. The upper-layer program has low permission and does not affect the system. +- vsftpd integrates most of the extra commands \(such as dir, ls, and cd\) used by FTP. Generally, the system does not need to provide extra commands, which are secure for the system. + +## Using vsftpd + +### Installing vsftpd + +To use the vsftpd service, you need to install the vsftpd software. If the yum source has been configured, run the following command as the root user to install the vsftpd service: + +```shell +dnf install vsftpd +``` + +### Service Management + +To start, stop, or restart the vsftpd service, run the corresponding command as the root user. + +- Starting vsftpd services + + ```shell + systemctl start vsftpd + ``` + + You can run the netstat command to check whether communication port 21 is enabled. If the following information is displayed, the vsftpd service has been enabled. + + ```shell + $ netstat -tulnp | grep 21 + tcp6 0 0 :::21 :::* LISTEN 19716/vsftpd + ``` + + > [!NOTE]NOTE + > If the **netstat** command does not exist, run the **dnf install net-tools** command to install the **net-tools** software and then run the **netstat** command. + +- Stopping the vsftpd services + + ```shell + systemctl stop vsftpd + ``` + +- Restarting the vsftpd service + + ```shell + systemctl restart vsftpd + ``` + +## Configuring vsftpd + +### vsftpd Configuration Files + +You can modify the vsftpd configuration file to control user permissions. [Table 1](#table1541615718372) describes the vsftpd configuration files. You can modify the configuration files as required. You can run the man command to view more parameter meanings. + +**Table 1** vsftpd configuration files + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Configuration File

+

Description

+

/etc/vsftpd/vsftpd.conf

+

Main configuration file of the vsftpd process. The configuration format is Parameter=Parameter value. The parameter and parameter value cannot be empty.

+

You can run the following command to view details about the vsftpd.conf file:

+

man 5 vsftpd.conf

+

/etc/pam.d/vsftpd

+

Pluggable authentication modules (PAMs) are used for identity authentication and restrict some user operations.

+

/etc/vsftpd/ftpusers

+

List of users who are not allowed to use the vsftpd. By default, the system account is also in this file. Therefore, the system account cannot use vsftpd by default.

+

/etc/vsftpd/user_list

+

List of users who are allowed or not allowed to log in to the vsftpd server. Whether the file takes effect depends on the following parameters in the main configuration file vsftpd.conf:

+

userlist_enable: indicates whether to enable the userlist mechanism. The value YES indicates that the userlist mechanism is enabled. In this case, the userlist_deny configuration is valid. The value NO indicates that the userlist mechanism is disabled.

+

userlist_deny: indicates whether to forbid users in the user list to log in. YES indicates that users in the user list are forbidden to log in. NO indicates that users in the command are allowed to log in.

+

For example, if userlist_enable is set to YES and userlist_deny is set to YES, all users in the user list cannot log in.

+

/etc/vsftpd/chroot_list

+

Whether to restrict the user list in the home directory. By default, this file does not exist. You need to create it manually. It is the value of chroot_list_file in the vsftpd.conf file.

+

The function of this parameter is determined by the following parameters in the vsftpd.conf file:

+
  • chroot_local_user: indicates whether to restrict all users to the home directory. The value YES indicates that all users are restricted to the home directory, and the value NO indicates that all users are not restricted to the home directory.
  • chroot_list_enable: indicates whether to enable the list of restricted users. The value YES indicates that the list is enabled, and the value NO indicates that the list is disabled.
+

For example, if chroot_local_user is set to YES, chroot_list_enable is set to YES, and chroot_list_file is set to /etc/vsftpd/chroot_list, all users are restricted to their home directories, and users in chroot_list are not restricted.

+

/usr/sbin/vsftpd

+

Unique execution file of vsftpd.

+

/var/ftp/

+

Default root directory for anonymous users to log in. The root directory is related to the home directory of the ftp user.

+
+ +### Default Configuration Description + +> [!NOTE]NOTE +> The configuration content in this document is for reference only. You can modify the content based on the site requirements \(for example, security hardening requirements\). + +In the openEuler system, vsftpd does not open to anonymous users by default. Run the vim command to view the main configuration file. The content is as follows: + +```shell +$ vim /etc/vsftpd/vsftpd.conf +anonymous_enable=NO +local_enable=YES +write_enable=YES +local_umask=022 +dirmessage_enable=YES +xferlog_enable=YES +connect_from_port_20=YES +xferlog_std_format=YES +listen=NO +listen_ipv6=YES +pam_service_name=vsftpd +userlist_enable=YES +``` + +[Table 2](#table18185162512499) describes the parameters. + +**Table 2** Parameter description + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Parameter

+

Description

+

anonymous_enable

+

Indicates whether to allow anonymous users to log in. YES indicates that anonymous users are allowed to log in; NO indicates that anonymous users are not allowed to log in.

+

local_enable

+

Whether to allow local users to log in. YES indicates that local users are allowed to log in. NO indicates that local users are not allowed to log in.

+

write_enable

+

Whether to allow the login user to have the write permission. YES indicates that the upload and write function is enabled, and NO indicates that the function is disabled.

+

local_umask

+

Indicates the umask value when a local user adds a profile.

+

dirmessage_enable

+

Indicates whether to display the contents that users need to pay attention to when a user accesses a directory. The options are YES (yes) and NO (no).

+

xferlog_enable

+

Indicates whether to record file upload and download operations. The options are YES (record operations) and NO (not record operations).

+

connect_from_port_20

+

Indicates whether port 20 is used for data transmission in port mode. YES indicates that port 20 is used, and NO indicates that port 20 is not used.

+

xferlog_std_format

+

Indicates whether the transfer log file is written in the standard xferlog format. The options are YES (yes) and NO (no).

+

listen

+

Indicates whether the vsftpd service is started in standalone mode. The options are YES (yes) and NO (no).

+

pam_service_name

+

Support for PAM management. The value is a service name, for example, vsftpd.

+

userlist_enable

+

Indicates whether to support account login control in the /etc/vsftpd/user_list file. The options are YES (yes) and NO (no).

+

tcp_wrappers

+

Indicates whether to support the firewall mechanism of the TCP Wrappers. The options are YES (yes) and NO (no).

+

listen_ipv6

+

Indicates whether to listen to IPv6 FTP requests. The options are YES (yes) and NO (no). listen and listen_ipv6 cannot be enabled at the same time.

+
+ +### Setting the Local Time + +#### Overview + +In the openEuler system, vsftpd uses the Greenwich Mean Time \(GMT\) time by default, which may be different from the local time. For example, the GMT time is 8 hours later than the Beijing time. You need to change the GMT time to the local time. Otherwise, the server time and client time are inconsistent, which may cause errors during file upload and download. + +#### Setting Method + +To set the vsftpd time to the local time, perform the following steps as the **root** user: + +1. Open the vsftpd.conf file and change the value of use\_localtime to **YES**. Run the following command: + + ```shell + vim /etc/vsftpd/vsftpd.conf + ``` + + Modify the file contents as follows: + + ```shell + use_localtime=YES + ``` + +2. Restart the vsftpd service. + + ```shell + systemctl restart vsftpd + ``` + +3. Set the vsftpd service to start automatically upon power-on. + + ```shell + systemctl enable vsftpd + ``` + +### Configuring Welcome Information + +You are advised to configure a welcome information file for the vsftpd service. To configure the **welcome.txt** file of the vsftp service, perform the following steps as the **root** user: + +1. Open the vsftpd.conf configuration file, add the welcome information to the file, save the file, and exit. + + ```shell + vim /etc/vsftpd/vsftpd.conf + ``` + + The following configuration lines need to be added: + + ```text + banner_file=/etc/vsftpd/welcome.txt + ``` + +2. Create welcome information. Specifically, open the welcome.txt file, write the welcome information, save the file, and exit. + + ```shell + vim /etc/vsftpd/welcome.txt + ``` + + The following is an example: + + ```text + Welcome to this FTP server! + ``` + +### Configuring the Login Permission of a System Account + +Generally, users need to restrict the login permission of some accounts. You can set the restriction as required. + +By default, vsftpd manages and restricts user identities based on user lists stored in two files. FTP requests from a user in any of the files will be denied. + +- **/etc/vsftpd/user_list** can be used as an allowlist, blocklist, or invalid list, which is determined by the **userlist_enable** and **userlist_deny** parameters. +- **/etc/vsftpd/ftpusers** can be used as a blocklist only, regardless of the parameters. + +## Verifying Whether the FTP Service Is Successfully Set Up + +You can use the FTP client provided by openEuler for verification. The command and output are as follows. Enter the user name \(an existing user in the system\) and password as prompted. If the message "Login successful" is displayed, the FTP server is successfully set up. + +```shell +$ ftp localhost +Trying 127.0.0.1... +Connected to localhost (127.0.0.1). +220-Welcome to this FTP server! +220 +Name (localhost:root): USERNAME +331 Please specify the password. +Password: +230 Login successful. +Remote system type is UNIX. +Using binary mode to transfer files. +ftp> bye +221 Goodbye. +``` + +> [!NOTE]NOTE +> If the **ftp** command does not exist, run the **dnf install ftp** command as the **root** user to install the **ftp** software and then run the **ftp** command. + +## Configuring a Firewall + +To open the FTP service to the Internet, you need to configure the firewall and SElinux as the **root** user. + +```shell +$ firewall-cmd --add-service=ftp --permanent +success +$ firewall-cmd --reload +success +$ setsebool -P ftpd_full_access on +``` + +## File Transmission + +### Overview + +This section describes how to transfer files after the vsftpd service is started. + +### Connecting to the Server + +#### Command Format + +**ftp** \[_hostname_ \| _ip-address_\] + +**hostname** indicates the name of the server, and **ip-address** indicates the IP address of the server. + +#### Requirements + +Run the following command on the command-line interface \(CLI\) of the openEuler OS: + +```shell +ftp ip-address +``` + +Enter the user name and password as prompted. If the following information is displayed after the authentication is successful, the FTP connection is successful. In this case, you have accessed the directory of the connected server. + +```shell +ftp> +``` + +At this prompt, you can enter different commands to perform related operations. + +- Display the current path of the server. + + ```shell + ftp>pwd + ``` + +- Display the local path. You can upload the files in this path to the corresponding location on the FTP server. + + ```shell + ftp>lcd + ``` + +- Exit the current window and return to the local Linux terminal. + + ```shell + ftp>! + ``` + +### Downloading a File + +Generally, the get or mget command is used to download files. + +#### How to Use get + +- Function description: Transfers files from a remote host to a local host. +- Command format: **get** \[_remote-file_\] \[_local-file_\] + + _remote-file_ indicates a remote file, and _local-file_ indicates a local file. + +- For example, run the following command to obtain the /home/openEuler/openEuler.htm file on the remote server to the local directory /home/myopenEuler/ and change the file name to myopenEuler.htm + + ```shell + ftp> get /home/openEuler/openEuler.htm /home/myopenEuler/myopenEuler.htm + ``` + +#### How to Use mget + +- Function description: Receives a batch of files from the remote host to the local host. +- Command format: **mget** \[_remote-file_\] + + _remote-file_ indicates a remote file. + +- For example, to obtain all files in the /home/openEuler/ directory on the server, run the following command: + + ```shell + ftp> cd /home/openEuler/ + ftp> mget *.* + ``` + + > [!NOTE]NOTE + > - In this case, a message is displayed each time a file is downloaded. To block the prompt information, run the **prompt off** command before running the **mget \*.\*** command. + > - The files are downloaded to the current directory on the Linux host. For example, if you run the ftp command in /home/myopenEuler/, all files are downloaded to /home/myopenEuler/. + +### Uploading a File + +Generally, the put or mput command is used to upload files. + +#### How to Use put + +- Function: Transfers a local file to a remote host. +- Command format: **put** \[_local-file_\] \[_remote-file_\] + + _remote-file_ indicates a remote file, and _local-file_ indicates a local file. + +- For example, run the following command to transfer the local myopenEuler.htm file to the remote host /home/openEuler/ and change the file name to openEuler.htm: + + ```shell + ftp> put myopenEuler.htm /home/openEuler/openEuler.htm + ``` + +#### How to Use mput + +- Function: Transfers a batch of files from the local host to a remote host. +- Command format: **mput** \[_local-file_\] + + _local-file_ indicates a local file. + +- For example, run the following command to upload all HTM files in the local directory to the /home/openEuler/ directory on the server: + + ```shell + ftp> cd /home/openEuler/ + ftp> mput *.htm + ``` + +### Deleting a File + +Generally, the **delete** or **mdelete** command is used to delete a file. + +#### How to Use delete + +- Function description: Deletes one or more files from the remote server. +- Command format: **delete** \[_remote-file_\] + + _remote-file_ indicates a remote file. + +- For example, to delete the /home/openEuler/openEuler.htm from the remote server, run the following command: + + ```shell + ftp> cd /home/openEuler/ + ftp> delete openEuler.htm + ``` + +#### How to Use mdelete + +- Function description: Deletes files from a remote server. This function is used to delete files in batches. +- Command format: **mdelete** \[_remote-file_\] + + _remote-file_ indicates a remote file. + +- For example, to delete all files whose names start with **a** from the /home/openEuler/ directory on the remote server, run the following command: + + ```shell + ftp> cd /home/openEuler/ + ftp> mdelete a* + ``` + +### Disconnecting from the Server + +Run the bye command to disconnect from the server. + +```shell +ftp> bye +``` diff --git a/docs/en/server/administration/administrator/configuring-the-repo-server.md b/docs/en/server/administration/administrator/configuring-the-repo-server.md new file mode 100644 index 0000000000000000000000000000000000000000..54b238545079b389410007c97d4051ae3b3d2ad3 --- /dev/null +++ b/docs/en/server/administration/administrator/configuring-the-repo-server.md @@ -0,0 +1,387 @@ +# Configuring the Repository Server + +> [!NOTE]NOTE +> openEuler provides multiple repositories for online usage. For details about the repositories, see [OS Installation](../../releasenotes/releasenotes/os-installation.md). If you cannot obtain the openEuler repository online, you can use the ISO release package provided by openEuler to create a local openEuler repository. This section uses the **openEuler-24.03-LTS-SP1-aarch64-dvd.iso** file as an example. Modify the ISO file as required. + +## Overview + +Create the **openEuler-24.03-LTS-SP1-aarch64-dvd.iso** file provided by openEuler as the repository. The following uses Nginx as an example to describe how to deploy the repository and provide the HTTP service. + +## Creating or Updating a Local Repository + +Mount the openEuler ISO file **openEuler-24.03-LTS-SP1-aarch64-dvd.iso** to create and update a repository. + +### Obtaining the ISO File + +Obtain the openEuler ISO file from the following website: + + + +### Mounting an ISO File to Create a Repository + +Run the **mount** command as the **root** user to mount the ISO file. + +The following is an example: + +```shell +mount /home/openEuler/openEuler-24.03-LTS-SP1-aarch64-dvd.iso /mnt/ +``` + +The mounted mnt directory is as follows: + +```text +. +│── boot.catalog +│── docs +│── EFI +│── images +│── Packages +│── repodata +│── TRANS.TBL +└── RPM-GPG-KEY-openEuler +``` + +In the preceding directory, **Packages** indicates the directory where the RPM package is stored, **repodata** indicates the directory where the repository metadata is stored, and **RPM-GPG-KEY-openEuler** indicates the public key for signing openEuler. + +### Creating a Local Repository + +You can copy related files in the ISO file to a local directory to create a local repository. The following is an example: + +```shell +mount /home/openEuler/openEuler-24.03-LTS-SP1-aarch64-dvd.iso /mnt/ +mkdir -p /home/openEuler/srv/repo/ +cp -r /mnt/Packages /home/openEuler/srv/repo/ +cp -r /mnt/repodata /home/openEuler/srv/repo/ +cp -r /mnt/RPM-GPG-KEY-openEuler /home/openEuler/srv/repo/ +``` + +The local repository directory is as follows: + +```text +. +│── Packages +│── repodata +└── RPM-GPG-KEY-openEuler +``` + +**Packages** indicates the directory where the RPM package is stored, **repodata** indicates the directory where the repository metadata is stored, and **RPM-GPG-KEY-openEuler** indicates the public key for signing openEuler. + +### Updating the Repository + +You can update the repository in either of the following ways: + +- Use the latest ISO file to update the existing repository. The method is the same as that for creating a repository. That is, mount the ISO file or copy the ISO file to the local directory. + +- Add a RPM package to the **Packages** directory of the repository and run the **createrepo** command to update the repository. + + ```shell + createrepo --update --workers=10 ~/srv/repo + ``` + +In this command, **--update** indicates the update, and **--workers** indicates the number of threads, which can be customized. + +> [!NOTE]NOTE +> If the command output contains "createrepo: command not found", run the **dnf install createrepo** command as the **root** user to install the **createrepo** softeware. + +## Deploying the Remote Repository + +Install openEuler OS and deploy the repository using Nginx on openEuler OS. + +### Installing and Configuring Nginx + +1. Download the Nginx tool and install it as the **root** user. + +2. After Nginx is installed, configure /etc/nginx/nginx.conf as the **root** user. + + > [!NOTE]NOTE + > The configuration content in this document is for reference only. You can configure the content based on the site requirements (for example, security hardening requirements). + + ```text + user nginx; + worker_processes auto; # You are advised to set this parameter to **core-1** . + error_log /var/log/nginx/error.log warn; # Log storage location + pid /var/run/nginx.pid; + + events { + worker_connections 1024; + } + + http { + include /etc/nginx/mime.types; + default_type application/octet-stream; + + log_format main '$remote_addr - $remote_user [$time_local] "$request" ' + '$status $body_bytes_sent "$http_referer" ' + '"$http_user_agent" "$http_x_forwarded_for"'; + + access_log /var/log/nginx/access.log main; + sendfile on; + keepalive_timeout 65; + + server { + listen 80; + server_name localhost; # Server name (URL) + client_max_body_size 4G; + root /usr/share/nginx/repo; # Default service directory + + location / { + autoindex on; # Enable the access to lower-layer files in the directory. + autoindex_exact_size on; + autoindex_localtime on; + } + + } + + } + ``` + +### Starting Nginx + +1. Run the following `systemctl` commands as the **root** user to start the Nginx service. + + ```shell + systemctl enable nginx + systemctl start nginx + ``` + +2. You can run the following command to check whether Nginx is started successfully: + + ```shell + systemctl status nginx + ``` + + - [Figure 1](#en-us_topic_0151920971_fd25e3f1d664b4087ae26631719990a71) indicates that the Nginx service is started successfully. + + **Figure 1** The Nginx service is successfully started. + ![](./figures/the-nginx-service-is-successfully-started.png) + + - If the Nginx service fails to be started, view the error information. + + ```shell + systemctl status nginx.service --full + ``` + + **Figure 2** The Nginx service startup fails + ![](./figures/nginx-startup-failure.png) + + As shown in [Figure 2](#en-us_topic_0151920971_f1f9f3d086e454b9cba29a7cae96a4c54), the Nginx service fails to be created because the /var/spool/nginx/tmp/client\_body directory fails to be created. You need to manually create the directory as the **root** user. Solve similar problems as follows: + + ```shell + mkdir -p /var/spool/nginx/tmp/client_body + mkdir -p /var/spool/nginx/tmp/proxy + mkdir -p /var/spool/nginx/tmp/fastcgi + mkdir -p /usr/share/nginx/uwsgi_temp + mkdir -p /usr/share/nginx/scgi_temp + ``` + +### Deploying the Repository + +1. Run the following command as the **root** user to create the **/usr/share/nginx/repo** directory specified in the Nginx configuration file /etc/nginx/nginx.conf: + + ```shell + mkdir -p /usr/share/nginx/repo + ``` + +2. Run the following command as the **root** user to modify the **/usr/share/nginx/repo** directory permission: + + ```shell + chmod -R 755 /usr/share/nginx/repo + ``` + +3. Configure firewall rules as the **root** user to enable the port (port 80) configured for Nginx. + + ```shell + firewall-cmd --add-port=80/tcp --permanent + firewall-cmd --reload + ``` + + Check whether port 80 is enabled as the **root** user. If the output is **yes**, port 80 is enabled. + + ```shell + firewall-cmd --query-port=80/tcp + ``` + + You can also enable port 80 using iptables as the **root** user. + + ```shell + iptables -I INPUT -p tcp --dport 80 -j ACCEPT + ``` + +4. After the Nginx service is configured, you can use the IP address to access the web page, as shown in [Figure 3](#en-us_topic_0151921017_fig1880404110396). + + **Figure 3** Nginx deployment succeeded + ![](./figures/nginx-deployment-succeeded.png) + +5. Use either of the following methods to add the repository to the **/usr/share/nginx/repo** directory: + + - Copy related files in the image to the **/usr/share/nginx/repo** directory as the **root** user. + + ```shell + mount /home/openEuler/openEuler-24.03-LTS-SP1-aarch64-dvd.iso /mnt/ + cp -r /mnt/Packages /usr/share/nginx/repo/ + cp -r /mnt/repodata /usr/share/nginx/repo/ + cp -r /mnt/RPM-GPG-KEY-openEuler /usr/share/nginx/repo/ + chmod -R 755 /usr/share/nginx/repo + ``` + + The **openEuler-24.03-LTS-SP1-aarch64-dvd.iso** file is stored in the **/home/openEuler** directory. + + - Create a soft link for the repository in the **/usr/share/nginx/repo** directory as the **root** user. + + ```shell + ln -s /mnt /usr/share/nginx/repo/os + ``` + + **/mnt** is the created repository, and **/usr/share/nginx/repo/os** points to **/mnt** . + +## Using the Repository + +The repository can be configured as a Yum repository, which is a shell front-end software package manager. Based on the Redhat package manager (RPM), YUM can automatically download the RPM package from the specified server, install the package, and process dependent relationship. It supports one-off installation for all dependent software packages. + +### Configuring Repository as the Yum Repository + +You can configure the built repository as the Yum repository and create the \*\*\*.repo configuration file (the extension .repo is mandatory) in the /etc/yum.repos.d/ directory as the **root** user. You can configure the Yum repository on the local host or HTTP server. + +- Configuring the local Yum repository. + + Create the **openEuler.repo** file in the **/etc/yum.repos.d** directory and use the local repository as the Yum repository. The content of the **openEuler.repo** file is as follows: + + ```text + [base] + name=base + baseurl=file:///home/openEuler/srv/repo + enabled=1 + gpgcheck=1 + gpgkey=file:///home/openEuler/srv/repo/RPM-GPG-KEY-openEuler + ``` + + > [!NOTE]NOTE + > - **repoid** indicates the ID of the software repository. Repoids in all .repo configuration files must be unique. In the example, **repoid** is set to **base**. + > - **name** indicates the string that the software repository describes. + > - **baseurl** indicates the address of the software repository. + > - **enabled** indicates whether to enable the software source repository. The value can be **1** or **0**. The default value is **1**, indicating that the software source repository is enabled. + > - **gpgcheck** indicates whether to enable the GNU privacy guard (GPG) to check the validity and security of sources of RPM packages. **1** indicates GPG check is enabled. **0** indicates the GPG check is disabled. + > - **gpgkey** indicates the public key used to verify the signature. + +- Configuring the Yum repository for the HTTP server + + Create the **openEuler.repo** file in the **/etc/yum.repos.d** directory. + + - If the repository of the HTTP server deployed by the user is used as the Yum repository, the content of **openEuler.repo** is as follows: + + ```text + [base] + name=base + baseurl=http://192.168.139.209/ + enabled=1 + gpgcheck=1 + gpgkey=http://192.168.139.209/RPM-GPG-KEY-openEuler + ``` + + > [!NOTE]NOTE + > 192.168.139.209 is an example. Replace it with the actual IP address. + + - If the openEuler repository provided by openEuler is used as the Yum repository, the content of **openEuler.repo** is as follows (the AArch64-based OS repository is used as an example): + + ```text + [base] + name=base + baseurl=http://repo.openeuler.org/openEuler-24.03-LTS-SP1/OS/aarch64/ + enabled=1 + gpgcheck=1 + gpgkey=http://repo.openeuler.org/openEuler-24.03-LTS-SP1/OS/aarch64/RPM-GPG-KEY-openEuler + ``` + +### Repository Priority + +If there are multiple repositories, you can set the repository priority in the .repo file. If the priority is not set, the default priority is **99** . If the same RPM package exists in the sources with the same priority, the latest version is installed. **1** indicates the highest priority and **99** indicates the lowest priority. The following shows how to set the priority of **openEuler.repo** to **2**. + +```text +[base] +name=base +baseurl=http://192.168.139.209/ +enabled=1 +priority=2 +gpgcheck=1 +gpgkey=http://192.168.139.209/RPM-GPG-KEY-openEuler +``` + +### Related Commands of dnf + +The **dnf** command can automatically parse the dependency between packages during installation and upgrade. The common usage method is as follows: + +```shell +dnf +``` + +Common commands are as follows: + +- Installation + + Run the following command as the **root** user. + + ```shell + dnf install + ``` + +- Upgrade + + Run the following command as the **root** user. + + ```shell + dnf update + ``` + +- Rollback + + Run the following command as the **root** user. + + ```shell + dnf downgrade + ``` + +- Update check + + ```shell + dnf check-update + ``` + +- Uninstallation + + Run the following command as the **root** user. + + ```shell + dnf remove + ``` + +- Query + + ```shell + dnf search + ``` + +- Local installation + + Run the following command as the **root** user. + + ```shell + dnf localinstall + ``` + +- Historical records check + + ```shell + dnf history + ``` + +- Cache records clearing + + ```shell + dnf clean all + ``` + +- Cache update + + ```shell + dnf makecache + ``` diff --git a/docs/en/server/administration/administrator/configuring-the-web-server.md b/docs/en/server/administration/administrator/configuring-the-web-server.md new file mode 100644 index 0000000000000000000000000000000000000000..0affcf1a06a78a0fb4d5c7f6f1b70aae390560c9 --- /dev/null +++ b/docs/en/server/administration/administrator/configuring-the-web-server.md @@ -0,0 +1,519 @@ +# Configuring the Web Server + + + +- [Configuring the Web Server](#configuring-the-web-server) + - [Apache Server](#apache-server) + - [Overview](#overview) + - [Managing httpd](#managing-httpd) + - [Configuration File Description](#configuration-file-description) + - [Management Module and SSL](#management-module-and-ssl) + - [Verifying Whether the Web Service Is Successfully Set Up](#verifying-whether-the-web-service-is-successfully-set-up) + - [Nginx Server](#nginx-server) + - [Overview](#overview-1) + - [Installing Nginx](#installing-nginx) + - [Managing Nginx](#managing-nginx) + - [Configuration File Description](#configuration-file-description-1) + - [Management Modules](#management-modules) + - [Verifying Whether the Web Service Is Successfully Set Up](#verifying-whether-the-web-service-is-successfully-set-up-1) + + + +## Apache Server + +### Overview + +World Wide Web \(Web\) is one of the most commonly used Internet protocols. At present, the web server in the Unix-Like system is mainly implemented through the Apache server software. To operate dynamic websites, LAMP \(Linux + Apache + MySQL + PHP\) is developed. Web services can be combined with multimedia such as text, graphics, images, and audio, and support information transmission through hyperlinks. + +The web server version in the openEuler system is Apache HTTP server 2.4, that is, httpd, which is an open-source web server developed by the Apache Software Foundation. + +### Managing httpd + +#### Overview + +You can use the systemctl tool to manage the httpd service, including starting, stopping, and restarting the service, and viewing the service status. This section describes how to manage the Apache HTTP service. + +#### Prerequisites + +- To use the Apache HTTP service, ensure that the rpm package of the httpd service has been installed in your system. Run the following command as the **root** user to install the rpm package: + + ```shell + dnf install httpd + ``` + + For more information about service management, see [Service Management](./service-management.md). + +- To start, stop, and restart the httpd service, you must have the root permission. + +#### Starting a Service + +- Run the following command to start and run the httpd service: + + ```shell + systemctl start httpd + ``` + +- If you want the httpd service to automatically start when the system starts, the command and output are as follows: + + ```shell + $ systemctl enable httpd + Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service → /usr/lib/systemd/system/httpd.service. + ``` + +> [!NOTE]NOTE +> If the running Apache HTTP server functions as a secure server, a password is required after the system is started. The password is an encrypted private SSL key. + +#### Stopping the Service + +- Run the following command to stop the httpd service: + + ```shell + systemctl stop httpd + ``` + +- If you want to prevent the service from automatically starting during system startup, the command and output are as follows: + + ```shell + $ systemctl disable httpd + Removed /etc/systemd/system/multi-user.target.wants/httpd.service. + ``` + +#### Restarting a Service + +You can restart the service in any of the following ways: + +- Restart the service by running the restart command: + + ```shell + systemctl restart httpd + ``` + + This command stops the ongoing httpd service and restarts it immediately. This command is generally used after a service is installed or when a dynamically loaded module \(such as PHP\) is removed. + +- Reload the configuration. + + ```shell + systemctl reload httpd + ``` + + This command causes the running httpd service to reload its configuration file. Any requests that are currently being processed will be interrupted, causing the client browser to display an error message or re-render some pages. + +- Re-load the configuration without affecting the activation request. + + ```shell + apachectl graceful + ``` + + This command causes the running httpd service to reload its configuration file. Any requests that are currently being processed will continue to use the old configuration file. + +#### Verifying the Service Status + +Check whether the httpd service is running. + +```shell +systemctl is-active httpd +``` + +If active is displayed in the command output, the service is running. + +### Configuration File Description + +After the httpd service is started, it reads the configuration file shown in [Table 1](#table24341012096) by default. + +**Table 1** Configuration file description + + + + + + + + + + + + + +

File

+

Description

+

/etc/httpd/conf/httpd.conf

+

Main configuration files.

+

/etc/httpd/conf.d

+

Secondary directory of configuration files, which are also contained in the main configuration file.

+

The secondary directory of a configuration file is contained in the main configuration file.

+
+ +Although the default configuration can be used in most cases, you need to be familiar with some important configuration items. After the configuration file is modified, run the following command as the **root** user to check the syntax errors that may occur in the configuration file: + +```shell +apachectl configtest +``` + +If the following information is displayed, the syntax of the configuration file is correct: + +```text +Syntax OK +``` + +> [!NOTE]NOTE +> +> - Before modifying the configuration file, back up the original file so that the configuration file can be quickly restored if a fault occurs. +> - The modified configuration file takes effect only after the web service is restarted. + +### Management Module and SSL + +#### Overview + +The httpd service is a modular application that is distributed with many Dynamic Shared Objects \(DSOs\). DSOs can be dynamically loaded or unloaded when running if necessary. These modules are located in the /usr/lib64/httpd/modules/ directory of the server operating system. This section describes how to load and write a module. + +#### Loading a Module + +To load a special DSO module, you can use the load module indication in the configuration file. The modules provided by the independent software package have their own configuration files in the /etc/httpd/conf.modules.d directory. + +For example, to load the asis DSO module, perform the following steps: + +1. In the **/etc/httpd/conf.modules.d/00-optional.conf** file, uncomment the following configuration line as the **root** user: + + ```conf + LoadModule asis_module modules/mod_asis.so + ``` + +2. After the loading is complete, restart the httpd service as the **root** user to reload the configuration file. + + ```shell + systemctl restart httpd + ``` + +3. After the loading is complete, run the httpd -M command as the **root** user to check whether the asis DSO module is loaded. + + ```shell + httpd -M | grep asis + ``` + + If the following information is displayed, the asis DSO module is successfully loaded: + + ```text + asis_module (shared) + ``` + +> [!NOTE]NOTE +> **Common httpd commands** +> +> - httpd -v: views the httpd version number. +> - httpd -l: views the static modules compiled into the httpd program. +> - httpd -M: views the static modules and loaded dynamic modules that have been compiled into the httpd program. + +#### Introduction to SSL + +Secure Sockets Layer \(SSL\) is an encryption protocol that allows secure communication between the server and client. The Transport Layer Security \(TLS\) protocol ensures security and data integrity for network communication. openEuler supports Mozilla Network Security Services \(NSS\) as the security protocol TLS. To load the SSL, perform the following steps: + +1. Install the **mod\_ssl** RPM package as the **root** user. + + ```shell + dnf install mod_ssl + ``` + +2. After the loading is complete, restart the httpd service as the **root** user to reload the configuration file. + + ```shell + systemctl restart httpd + ``` + +3. After the loading is complete, run the **httpd -M** command as the **root** user to check whether the SSL is loaded. + + ```shell + httpd -M | grep ssl + ``` + + If the following information is displayed, the SSL has been loaded successfully. + + ```text + ssl_module (shared) + ``` + +### Verifying Whether the Web Service Is Successfully Set Up + +After the web server is set up, perform the following operations to check whether the web server is set up successfully: + +1. Run the following command as the **root** user to check the IP address of the server: + + ```shell + ip a + ``` + +2. Configure the firewall as the **root** user. + + ```shell + $ firewall-cmd --add-service=http --permanent + success + # firewall-cmd --reload + success + ``` + +3. Verify whether the web server is successfully set up. You can select the Linux or Windows operating system for verification. + - Using the Linux OS + + Run the following command to check whether the web page can be accessed. If the service is successfully set up, the web page can be accessed. + + ```shell + curl http://192.168.1.60 + ``` + + Run the following command to check whether the command output is 0. If the command output is 0, the httpd server is successfully set up. + + ```shell + echo $? + ``` + + - Using the Windows OS + + Open the browser and enter the following address in the address box. If the web page can be accessed, the httpd server is successfully set up. + + + + If the port number is changed, enter the address in the following format: + + + +## Nginx Server + +### Overview + +Nginx is a lightweight web server which also acts as a reverse proxy server and email \(IMAP/POP3\) proxy server. It features low memory usage and strong concurrency capability. Nginx supports FastCGI, SSL, virtual hosts, URL rewrite, Gzip, and extension of many third-party modules. + +### Installing Nginx + +1. Configure the local yum source. For details, see [Configuring the Repo Server](./configuring-the-repo-server.md). +2. Clear the cache. + + ```shell + dnf clean all + ``` + +3. Create a cache. + + ```shell + dnf makecache + ``` + +4. Install the Nginx server as the **root** user. + + ```shell + dnf install nginx + ``` + +5. Check the installed RPM package. + + ```shell + dnf list all | grep nginx + ``` + +### Managing Nginx + +#### Overview + +You can use the systemctl tool to manage the Nginx service, including starting, stopping, and restarting the service, and viewing the service status. This section describes how to manage the Nginx service. + +#### Prerequisites + +- Ensure that the Nginx service has been installed. If not, install it by referring to [Installing Nginx](#installing-nginx). + + For more information about service management, see [Service Management](./service-management.md). + +- To start, stop, and restart the Nginx service, you must have the **root** permission. + +#### Starting a Service + +- Run the following command to start and run the Nginx service: + + ```shell + systemctl start nginx + ``` + +- If you want the Nginx service to automatically start when the system starts, the command and output are as follows: + + ```shell + $ systemctl enable nginx + Created symlink /etc/systemd/system/multi-user.target.wants/nginx.service → /usr/lib/systemd/system/nginx.service. + ``` + +> [!NOTE]NOTE +> If the running Nginx server functions as a secure server, a password is required after the system is started. The password is an encrypted private SSL key. + +#### Stopping the Service + +- Run the following command to stop the Nginx service: + + ```shell + systemctl stop nginx + ``` + +- If you want to prevent the service from automatically starting during system startup, the command and output are as follows: + + ```shell + $ systemctl disable nginx + Removed /etc/systemd/system/multi-user.target.wants/nginx.service. + ``` + +#### Restarting a Service + +You can restart the service in any of the following ways: + +- Restart the service. + + ```shell + systemctl restart nginx + ``` + + This command stops the ongoing Nginx service and restarts it immediately. This command is generally used after a service is installed or when a dynamically loaded module \(such as PHP\) is removed. + +- Reload the configuration. + + ```shell + systemctl reload nginx + ``` + + This command causes the running Nginx service to reload its configuration file. Any requests that are currently being processed will be interrupted, causing the client browser to display an error message or re-render some pages. + +- Smoothly restart Nginx. + + ```shell + kill -HUP PID + ``` + + This command causes the running Nginx service to reload its configuration file. Any requests that are currently being processed will continue to use the old configuration file. + +#### Verifying the Service Status + +Check whether the Nginx service is running. + +```shell +systemctl is-active nginx +``` + +If **active** is displayed in the command output, the service is running. + +### Configuration File Description + +After the Nginx service is started, it reads the configuration file shown in [Table 2](#table24341012096) by default. + +**Table 2** Configuration file description + + + + + + + + + + + + + +

File

+

Description

+

/etc/nginx/nginx.conf

+

Main configuration files.

+

/etc/nginx/conf.d

+

Secondary directory of configuration files, which are also contained in the main configuration file.

+

The secondary directory of a configuration file is contained in the main configuration file.

+
+ +Although the default configuration can be used in most cases, you need to be familiar with some important configuration items. After the configuration file is modified, run the following command as the **root** user to check the syntax errors that may occur in the configuration file: + +```shell +nginx -t +``` + +If the command output contains **syntax is ok**, the syntax of the configuration file is correct. + +> [!NOTE]NOTE +> +> - Before modifying the configuration file, back up the original file so that the configuration file can be quickly restored if a fault occurs. +> - The modified configuration file takes effect only after the web service is restarted. + +### Management Modules + +#### Overview + +The Nginx service is a modular application that is distributed with many Dynamic Shared Objects \(DSOs\). DSOs can be dynamically loaded or unloaded when running if necessary. These modules are located in the **/usr/lib64/nginx/modules/** directory of the server operating system. This section describes how to load and write a module. + +#### Loading a Module + +To load a special DSO module, you can use the load module indication in the configuration file. Generally, the modules provided by independent software packages have their own configuration files in the **/usr/share/nginx/modules** directory. + +The DSO is automatically loaded when the **dnf install nginx** command is used to install the Nginx in the openEuler operating system. + +### Verifying Whether the Web Service Is Successfully Set Up + +After the web server is set up, perform the following operations to check whether the web server is set up successfully: + +1. Run the following command as the **root** user to check the IP address of the server: + + ```shell + ip a + ``` + + If the following information is displayed, the IP address of the server is **192.168.1.60**. + + ```text + enp3s0: flags=4163 mtu 1500 + inet 192.168.1.60 netmask 255.255.255.0 broadcast 192.168.1.255 + inet6 fe80::5054:ff:fe95:499f prefixlen 64 scopeid 0x20 + ether 52:54:00:95:49:9f txqueuelen 1000 (Ethernet) + RX packets 150713207 bytes 49333673733 (45.9 GiB) + RX errors 0 dropped 43 overruns 0 frame 0 + TX packets 2246438 bytes 203186675 (193.7 MiB) + TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 + + enp4s0: flags=4163 mtu 1500 + ether 52:54:00:7d:80:9e txqueuelen 1000 (Ethernet) + RX packets 149937274 bytes 44652889185 (41.5 GiB) + RX errors 0 dropped 1102561 overruns 0 frame 0 + TX packets 0 bytes 0 (0.0 B) + TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 + + lo: flags=73 mtu 65536 + inet 127.0.0.1 netmask 255.0.0.0 + inet6 ::1 prefixlen 128 scopeid 0x10 + loop txqueuelen 1000 (Local Loopback) + RX packets 37096 bytes 3447369 (3.2 MiB) + RX errors 0 dropped 0 overruns 0 frame 0 + TX packets 37096 bytes 3447369 (3.2 MiB) + TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 + ``` + +2. Configure the firewall as the **root** user. + + ```shell + $ firewall-cmd --add-service=http --permanent + success + # firewall-cmd --reload + success + ``` + +3. Verify whether the web server is successfully set up. You can select the Linux or Windows operating system for verification. + - Using the Linux OS + + Run the following command to check whether the web page can be accessed. If the service is successfully set up, the web page can be accessed. + + ```shell + curl http://192.168.1.60 + ``` + + Run the following command to check whether the command output is **0**. If the command output is **0**, the Nginx server is successfully set up. + + ```shell + echo $? + ``` + + - Using the Windows OS + + Open the browser and enter the following address in the address box. If the web page can be accessed, the Nginx server is successfully set up. + + + + If the port number is changed, enter the address in the following format: + + diff --git a/docs/en/server/administration/administrator/figures/1665628542704.png b/docs/en/server/administration/administrator/figures/1665628542704.png new file mode 100644 index 0000000000000000000000000000000000000000..58907e0ed31c79b260be80480d4fd4c27e4e2e24 Binary files /dev/null and b/docs/en/server/administration/administrator/figures/1665628542704.png differ diff --git a/docs/en/server/administration/administrator/figures/creat_datadisk.png b/docs/en/server/administration/administrator/figures/creat_datadisk.png new file mode 100644 index 0000000000000000000000000000000000000000..0dfd6a2802184af6d809c485191ea52452cf28d5 Binary files /dev/null and b/docs/en/server/administration/administrator/figures/creat_datadisk.png differ diff --git a/docs/en/server/administration/administrator/figures/creat_datadisk1.png b/docs/en/server/administration/administrator/figures/creat_datadisk1.png new file mode 100644 index 0000000000000000000000000000000000000000..0dfd6a2802184af6d809c485191ea52452cf28d5 Binary files /dev/null and b/docs/en/server/administration/administrator/figures/creat_datadisk1.png differ diff --git a/docs/en/server/administration/administrator/figures/d1376b2a-d036-41c4-b852-e8368f363b5e-1.png b/docs/en/server/administration/administrator/figures/d1376b2a-d036-41c4-b852-e8368f363b5e-1.png new file mode 100644 index 0000000000000000000000000000000000000000..900cdc07c1f0e844bc48fe2342e83c91a23c24ec Binary files /dev/null and b/docs/en/server/administration/administrator/figures/d1376b2a-d036-41c4-b852-e8368f363b5e-1.png differ diff --git a/docs/en/server/administration/administrator/figures/d1376b2a-d036-41c4-b852-e8368f363b5e.png b/docs/en/server/administration/administrator/figures/d1376b2a-d036-41c4-b852-e8368f363b5e.png new file mode 100644 index 0000000000000000000000000000000000000000..900cdc07c1f0e844bc48fe2342e83c91a23c24ec Binary files /dev/null and b/docs/en/server/administration/administrator/figures/d1376b2a-d036-41c4-b852-e8368f363b5e.png differ diff --git a/docs/en/server/administration/administrator/figures/en-us_image_0230050789.png b/docs/en/server/administration/administrator/figures/en-us_image_0230050789.png new file mode 100644 index 0000000000000000000000000000000000000000..0b785be2a026fe059c6ee41700a971a11cfff7ae Binary files /dev/null and b/docs/en/server/administration/administrator/figures/en-us_image_0230050789.png differ diff --git a/docs/en/server/administration/administrator/figures/en-us_image_0231563132.png b/docs/en/server/administration/administrator/figures/en-us_image_0231563132.png new file mode 100644 index 0000000000000000000000000000000000000000..bb801a9471f3f3541ba96491654f25e2df9ce8bf Binary files /dev/null and b/docs/en/server/administration/administrator/figures/en-us_image_0231563132.png differ diff --git a/docs/en/server/administration/administrator/figures/en-us_image_0231563134.png b/docs/en/server/administration/administrator/figures/en-us_image_0231563134.png new file mode 100644 index 0000000000000000000000000000000000000000..398d15376d29d3aa406abb2e7e065d4625428c4d Binary files /dev/null and b/docs/en/server/administration/administrator/figures/en-us_image_0231563134.png differ diff --git a/docs/en/server/administration/administrator/figures/en-us_image_0231563135.png b/docs/en/server/administration/administrator/figures/en-us_image_0231563135.png new file mode 100644 index 0000000000000000000000000000000000000000..785977142a6bf0e1c1815b82dea73d75fa206a75 Binary files /dev/null and b/docs/en/server/administration/administrator/figures/en-us_image_0231563135.png differ diff --git a/docs/en/server/administration/administrator/figures/en-us_image_0231563136.png b/docs/en/server/administration/administrator/figures/en-us_image_0231563136.png new file mode 100644 index 0000000000000000000000000000000000000000..c274db4d0ca9d8758267a916e19fdef4aa22d0ba Binary files /dev/null and b/docs/en/server/administration/administrator/figures/en-us_image_0231563136.png differ diff --git a/docs/en/server/administration/administrator/figures/example-command-output.png b/docs/en/server/administration/administrator/figures/example-command-output.png new file mode 100644 index 0000000000000000000000000000000000000000..2d77d3dc2934763b5da896a827b9805da34d1c09 Binary files /dev/null and b/docs/en/server/administration/administrator/figures/example-command-output.png differ diff --git a/docs/en/server/administration/administrator/figures/login.png b/docs/en/server/administration/administrator/figures/login.png new file mode 100644 index 0000000000000000000000000000000000000000..d15c2cad98fba16320d587f3c7b0c80f435c5d3a Binary files /dev/null and b/docs/en/server/administration/administrator/figures/login.png differ diff --git a/docs/en/server/administration/administrator/figures/mariadb-logical-architecture.png b/docs/en/server/administration/administrator/figures/mariadb-logical-architecture.png new file mode 100644 index 0000000000000000000000000000000000000000..c4e65e786d918c84bbb14c101b69bc4ad36ccb4b Binary files /dev/null and b/docs/en/server/administration/administrator/figures/mariadb-logical-architecture.png differ diff --git a/docs/en/server/administration/administrator/figures/nginx-deployment-succeeded.png b/docs/en/server/administration/administrator/figures/nginx-deployment-succeeded.png new file mode 100644 index 0000000000000000000000000000000000000000..9ffb2c142defbd690e5407659116bf8e5582ba73 Binary files /dev/null and b/docs/en/server/administration/administrator/figures/nginx-deployment-succeeded.png differ diff --git a/docs/en/server/administration/administrator/figures/nginx-startup-failure.png b/docs/en/server/administration/administrator/figures/nginx-startup-failure.png new file mode 100644 index 0000000000000000000000000000000000000000..c8b855453433796265de42d7ffd0189c7ff9be2b Binary files /dev/null and b/docs/en/server/administration/administrator/figures/nginx-startup-failure.png differ diff --git a/docs/en/server/administration/administrator/figures/postgres.png b/docs/en/server/administration/administrator/figures/postgres.png new file mode 100644 index 0000000000000000000000000000000000000000..e7fc36882718587ec949133fe9892185cb4c2158 Binary files /dev/null and b/docs/en/server/administration/administrator/figures/postgres.png differ diff --git a/docs/en/server/administration/administrator/figures/postgresql-architecture.png b/docs/en/server/administration/administrator/figures/postgresql-architecture.png new file mode 100644 index 0000000000000000000000000000000000000000..cc73eb31f746409efc1e997072bf3d18b013012e Binary files /dev/null and b/docs/en/server/administration/administrator/figures/postgresql-architecture.png differ diff --git a/docs/en/server/administration/administrator/figures/the-nginx-service-is-successfully-started.png b/docs/en/server/administration/administrator/figures/the-nginx-service-is-successfully-started.png new file mode 100644 index 0000000000000000000000000000000000000000..bc6929772fd98fac3494b4436f26910b09818cb7 Binary files /dev/null and b/docs/en/server/administration/administrator/figures/the-nginx-service-is-successfully-started.png differ diff --git a/docs/en/server/administration/administrator/process-management.md b/docs/en/server/administration/administrator/process-management.md new file mode 100644 index 0000000000000000000000000000000000000000..678297725a1df8289f103a08de1940470bbf9f7b --- /dev/null +++ b/docs/en/server/administration/administrator/process-management.md @@ -0,0 +1,349 @@ +# Process Management + +The operating system (OS) manages multiple user requests and tasks. In most cases, the OS comes with only one CPU and one main memory, but multiple tier-2 disks and input/output \(I/O\) devices. Therefore, users have to share resources, but it appears to users that they are exclusively occupying resources. The OS places user tasks, OS tasks, mailing, print tasks, and other pending tasks in a queue and schedules the tasks according to predefined rules. This topic describes how the OS manages processes. + + +- [Process Management](#process-management) + - [Viewing Processes](#viewing-processes) + - [who Command](#who-command) + - [ps Command](#ps-command) + - [top Command](#top-command) + - [kill Command](#kill-command) + - [Scheduling a Process](#scheduling-a-process) + - [Using the at Command to Run Processes at the Scheduled Time](#using-the-at-command-to-run-processes-at-the-scheduled-time) + - [Using the cron Service to Run Commands Periodically](#using-the-cron-service-to-run-commands-periodically) + - [Suspending/Resuming a Process](#suspendingresuming-a-process) + + + +## Viewing Processes + +Linux is a multi-task system and needs to get process information during process management. To manage processes, you need to know the number of processes and their statuses. Multiple commands are available to view processes. + +### who Command + +The `who` command is used to display system user information. For example, before running the `talk` command to establish instant communication with another user, you need to run the `who` command to determine whether the target user is online. In another example, the system administrator can run the `who` command to learn what each login user is doing at the current time. The `who` command is widely seen in system administration since it is easy to use and can return a comprehensive set of accurate user information. + +The following is an example output of the `who` command, where system users and their status are displayed: The use of the `who` command is as follows: + +```shell +$ who +admin tty1 Jul 28 15:55 +admin pts/0 Aug 5 15:46 (192.168.0.110) +admin pts/2 Jul 29 19:52 (192.168.0.110) +root pts/3 Jul 30 12:07 (192.168.0.110) +root pts/4 Jul 31 10:29 (192.168.0.144) +root pts/5 Jul 31 14:52 (192.168.0.11) +root pts/6 Aug 6 10:12 (192.168.0.234) +root pts/8 Aug 6 11:34 (192.168.0.234) +``` + +### ps Command + +The **ps** command is the most basic and powerful command to view process information. The ps command is used to display process information, including which processes are running, terminated, resource-hungry, or stay as zombies. + +The `ps` command is the most basic and powerful command to view process information, including which processes are running, terminated, resource-hungry, or stay as zombies. + +A common scenario is to monitor background processes, which do not interact with your screen, keyboard, and other I/O devices. [Table 1](#en-us_topic_0151921029_t34619d964a3d41ad8694189ec383359c) lists the common `ps` command options. + +**Table 1** Common ps command options + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Option

+

Description

+

-e

+

Displays all processes.

+

-f

+

Full output format.

+

-h

+

Hides column headings in the process information.

+

-l

+

Long output format.

+

-w

+

Wide output format.

+

-a

+

Lists all processes on a terminal, including those of other users.

+

-r

+

Lists only running processes.

+

-x

+

Lists all processes without control terminals.

+
+ +For example, to list all processes on a terminal, run the following command: + +```shell +$ ps -a + PID TTY TIME CMD +12175 pts/6 00:00:00 bash +24526 pts/0 00:00:00 vsftpd +29478 pts/5 00:00:00 ps +32461 pts/0 1-01:58:33 sh +``` + +### top Command + +Both the `top` and `ps` commands can display a list of currently running processes, but the `top` command allows you to update the displayed list of processes by pressing a button repeatedly. If the `top` command is executed in foreground, it exclusively occupies foreground until it is terminated. The `top` command provides real-time visibility into system processor status. You can sort the list of CPU tasks by CPU usage, memory usage, or task execution time. Extensive display customization, such as choosing the columns or sorting method, can be achieved using interactive commands or the customization file. + +[Figure 1](#en-us_topic_0151921029_f289234fcdbac453796200d80e9889cd1) provides an example output of the `top` command. + +**Figure 1** Example command output +![](./figures/example-command-output.png) + +### kill Command + +The `kill` command is used to terminate a process regardless of whether the process is running in foreground or background. It differs from the combo key **Ctrl+C**, which can terminate only foreground processes. The reason for terminating a background process can be heavy use of CPU resources or deadlock. + +The `kill` command sends a signal to terminate running processes. By default, the `TERM` signal is used, terminating all processes incapable of capturing it. To terminate a process capable of capturing the `TERM` signal, use the `KILL` signal \(signal ID: 9\) instead. + +Two types of syntax of the `kill` command: + +```shell +kill [-s signal | -p] [-a] PID... +kill -l [signal] +``` + +The process ID can be retrieved by running the `ps` command. The `-s` option indicates the signal sent to the specified program. The signal details can be viewed by running the `kill -l` command. The `-p` option indicates the specified process ID. + +For example, to terminate the process whose ID is 1409, run the following command as the **root** user: + +```shell +kill -9 1409 +``` + +Example output of the `kill` command with the `-l` option + +```shell +$ kill -l + 1) SIGHUP 2) SIGINT 3) SIGQUIT 4) SIGILL 5) SIGTRAP + 6) SIGABRT 7) SIGBUS 8) SIGFPE 9) SIGKILL 10) SIGUSR1 +11) SIGSEGV 12) SIGUSR2 13) SIGPIPE 14) SIGALRM 15) SIGTERM +16) SIGSTKFLT 17) SIGCHLD 18) SIGCONT 19) SIGSTOP 20) SIGTSTP +21) SIGTTIN 22) SIGTTOU 23) SIGURG 24) SIGXCPU 25) SIGXFSZ +26) SIGVTALRM 27) SIGPROF 28) SIGWINCH 29) SIGIO 30) SIGPWR +31) SIGSYS 34) SIGRTMIN 35) SIGRTMIN+1 36) SIGRTMIN+2 37) SIGRTMIN+3 +38) SIGRTMIN+4 39) SIGRTMIN+5 40) SIGRTMIN+6 41) SIGRTMIN+7 42) SIGRTMIN+8 +43) SIGRTMIN+9 44) SIGRTMIN+10 45) SIGRTMIN+11 46) SIGRTMIN+12 47) SIGRTMIN+13 +48) SIGRTMIN+14 49) SIGRTMIN+15 50) SIGRTMAX-14 51) SIGRTMAX-13 52) SIGRTMAX-12 +53) SIGRTMAX-11 54) SIGRTMAX-10 55) SIGRTMAX-9 56) SIGRTMAX-8 57) SIGRTMAX-7 +58) SIGRTMAX-6 59) SIGRTMAX-5 60) SIGRTMAX-4 61) SIGRTMAX-3 62) SIGRTMAX-2 +63) SIGRTMAX-1 64) SIGRTMAX +``` + +## Scheduling a Process + +The time-consuming and resource-demanding part of maintenance work is often performed at late night. You can schedule relevant processes to get started at the scheduled time instead of staying up all night. The following describes the process scheduling commands. + +### Using the at Command to Run Processes at the Scheduled Time + +#### Function + +The `at` command is used to run a batch of processes \(a series of commands\) at the scheduled time or time and date. + +Syntax of the `at` command: + +```shell +at [-V] [-q queue] [-f filename] [-mldbv] time +at -c job [job...] +``` + +#### Time Format + +The scheduled time can be in any of the following formats: + +- _hh:mm_ today: If _hh:mm_ is earlier than the current time, the selected commands will be run at _hh:mm_ the next day. +- midnight, noon, teatime \(typically at 16:00\), or the like +- 12-hour format followed by am or pm +- Time + date \(_month day_, _mm/dd/yy_, or _dd.mm.yy_\). The scheduled date must follow the scheduled time. + +The scheduled time can also be relative time, which is suitable for scheduling commands that are going to be executed soon. For example, now+_N_ minutes, hours, days, or weeks. _N_ indicates the specified time, which may be a few days or hours. Further, the scheduled time can be words like today, tomorrow, or the like. Here are some examples of the scheduled time. + +Assume that the current time is 12:30 June 7 2019 and you want to run a command at 4:30 pm. The time scheduled by the `at` command can be any of the following: + +```shell + at 4:30pm + at 16:30 + at 16:30 today + at now+4 hours + at now+ 240 minutes + at 16:30 7.6.19 + at 16:30 6/7/19 + at 16:30 Jun 7 +``` + +Although you can select any of the preceding examples according to your preference, absolute time in 24-hour format, such as `at 16:30 6/7/19`, is recommended. + +#### Privileges + +Only commands from standard input or from the file specified by the **-f** option can be scheduled by the `at` command. If the `su` command is executed to switch the OS from user A to user B and then the `at` command is executed at the shell prompt of user B, the `at` command execution result is sent to user B, whereas emails \(if any\) are sent to user A. + +For example, to run the `slocate -u` command at 10 am on June 8, 2019, run the following commands as the **root** user: + +```shell +$ at 10:00 6/8/19 +at> slocate -u +at> +[1]+ Stopped at 10:00 6/8/19 +``` + +When the **at\>** prompt appears, type `slocate -u` and press **Enter**. Repeat the step to add other commands that need to be run at 10 am on 8 June 2019. Then, press **Ctrl+D** to exit the `at` command. + +The administrator is authorized to run the `at` command unconditionally. For other users, their privileges to run the `at` command is defined in the **/etc/at.allow** and **/etc/at.deny** files. + +### Using the cron Service to Run Commands Periodically + +The `at` command can run commands at the scheduled time, but only once. It means that after the commands to be run is specified, the system completes the task at the specified time. If you need to run the commands repeatedly, the **cron** service is a good choice. + +#### Cron Service + +The **cron** service searches the **/var/spool/cron** directory for the **crontab** files named by the user name in the **/etc/passwd** file and loads the search results into memory to execute the commands in the **crontab** files. Each user has a **crontab** file with the same name as the user name. For example, the **crontab** file of the **userexample** user is **/var/spool/cron/userexample**. + +The **cron** service also reads the cron configuration file **/etc/crontab** every minute, which can be edited in various formats. If no **crontab** files are found, the **cron** service enters sleep mode and releases system resources. One minute later, the **cron** service is waken up to repeat the search work and command execution. Therefore, the background process occupies few resources and is wakened up every minute to check whether there are commands to be executed. + +Command execution results are then mailed to users specified by the environment variable `MAILTO` in the **/etc/crontab** file. The **cron** service, once started, does not require manual intervention except when you need to replace the scheduled commands with new ones. + +#### crontab Command + +The `crontab` command is used to install, edit, remove, list, and perform other operations on **crontab** files. Each user has its own **crontab** files and can add commands to be executed to the files. + +Here are common `crontab` command options: + +- crontab -u //Set the **cron** service of a user. This option is required only when the `crontab` command is run by the **root** user. +- crontab -l //List details about the **cron** service of a user. +- crontab -r //Remove the **cron** service of a user. +- crontab -e //Edit the **cron** service of a user. + +For example, to list the **cron** service settings of the **root** user, run the following command: + +```shell +crontab -u root -l +``` + +#### crontab Files + +Enter the commands to be executed and their scheduled time in **crontab** files. Each line in the files contains six fields. The first five fields are the time when the specified command is executed, and the last field is the command to be executed. Fields are separated by spaces or tabs. The format is as follows: + +```shell +minute hour day-of-month month-of-year day-of-week commands +``` + +[Table 2](#en-us_topic_0151921016_t7d97d1204fe249d7ae0a87b4cf9a9353) describes the fields in each line. + +**Table 2** Parameter description + + + + + + + + + + + + + + + + + + + + + + + + + +

Parameter

+

Description

+

minute

+

The minute of the hour at which commands will be executed. Value range: 0-59.

+

hour

+

The hour of the day at which scheduled commands will be executed. Value range: 0-23.

+

day-of-month

+

The day of the month on which scheduled commands will be executed. Value range: 1-31.

+

month-of-year

+

The month of the year in which scheduled commands will be executed. Value range: 1-12.

+

day-of-week

+

The day of the week on which scheduled commands will be executed. Value range: 0-6.

+

commands

+

Scheduled commands.

+
+ +The fields cannot be left unspecified. In addition to numerical values, the following special characters are allowed: asterisk \(\*\), indicating a wildcard value; forward slash \(/\), followed by a numeral value _N_ to indicate that commands will be executed at a regular interval of _N_; hyphen \(-\), used with a range; and comma \(,\), used to separate discrete values. A complete path to the commands must be provided. + +For example, to allow the OS to add **sleepy** to the **/tmp/test.txt** file every two hours from 18 pm to 22 pm, add the following line to a **crontab** file: + +```shell +* 18-22/2 * * * echo "sleepy" >> /tmp/test.txt +``` + +Each time the **cron** service settings of a user are edited, the **cron** service generates a **crontab** file with the same name as the user in the **/var/spool/cron directory**. The **crontab** file can be edited only using the `crontab -e` command. Alternatively, the user can create a file and run the `crontab _filename_` command to import its **cron** settings to the new file. + +For example, to create a **crontab** file for the **userexample** user, perform the following steps: + +1. Create a file using any text editor. Add the commands that need to be executed periodically and the command execution interval to the new file. In this example, the new file is **\~/userexample.cron**. +2. Run the following command as the **root** user to install the new file as the **crontab** file of the **userexample** user: + + ```shell + crontab -u userexample ~/userexample.cron + ``` + +After the new file is installed, you will find a file named **userexample** in the **/var/spool/cron** directory. This file is the required **crontab** file. + +> [!NOTE]NOTE +> Do not restart the **cron** service after a **crontab** file is modified, because the **cron** service, once started, reads the **crontab** file every minute to check whether there are commands that need to be executed periodically. + +#### /etc/crontab File + +The **cron** service reads all files in the **/var/spool/cron** directory and the **/etc/crontab** file every minute. Therefore, you can use the **cron** service by configuring the **/etc/crontab** file. A **crontab** file contains user-specific commands, whereas the **/etc/crontab** file contains system-wide commands. The following is an example of the **/etc/crontab** file. + +```text +SHELL=/bin/sh +PATH=/usr/bin:/usr/sbin:/sbin:/bin:/usr/lib/news/bin +MAILTO=root //If an error occurs or data is output, the data is sent to the account by email. +HOME=/ +# run-parts +01 * * * * root run-parts /etc/cron.hourly //Run scripts in the /etc/cron.hourly directory once an hour. +02 4 * * * root run-parts /etc/cron.daily //Run scripts in the /etc/cron.daily directory once a day. +22 4 * * 0 root run-parts /etc/cron.weekly //Run scripts in the /etc/cron.weekly directory once a week. +42 4 1 * * root run-parts /etc/cron.monthly //Run scripts in the /etc/cron.monthly directory once a month. +``` + +> [!NOTE]NOTE +> If the **run-parts** parameter is deleted, a script name instead of a directory name is used. + +## Suspending/Resuming a Process + +A process can be suspended or resumed by job control, and the process will continue to work from the suspended point after being resumed. To suspend a foreground process, press **Ctrl+Z**. After you press **Ctrl+Z**, the `cat` command is suspended together with the foreground process you want to suspend. You can use the `jobs` command instead to display a list of shell jobs, including their names, IDs, and status. + +To resume a process in foreground or background, run the `fg` or `bg` command, respectively. The process then starts from where it was suspended previously. diff --git a/docs/en/server/administration/administrator/public_sys-resources/icon-caution.gif b/docs/en/server/administration/administrator/public_sys-resources/icon-caution.gif new file mode 100644 index 0000000000000000000000000000000000000000..6e90d7cfc2193e39e10bb58c38d01a23f045d571 Binary files /dev/null and b/docs/en/server/administration/administrator/public_sys-resources/icon-caution.gif differ diff --git a/docs/en/server/administration/administrator/public_sys-resources/icon-note.gif b/docs/en/server/administration/administrator/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/server/administration/administrator/public_sys-resources/icon-note.gif differ diff --git a/docs/en/server/administration/administrator/public_sys-resources/icon-notice.gif b/docs/en/server/administration/administrator/public_sys-resources/icon-notice.gif new file mode 100644 index 0000000000000000000000000000000000000000..86024f61b691400bea99e5b1f506d9d9aef36e27 Binary files /dev/null and b/docs/en/server/administration/administrator/public_sys-resources/icon-notice.gif differ diff --git a/docs/en/server/administration/administrator/service-management.md b/docs/en/server/administration/administrator/service-management.md new file mode 100644 index 0000000000000000000000000000000000000000..0cbd812f3d473f910a50363da6c0458b6163c9ad --- /dev/null +++ b/docs/en/server/administration/administrator/service-management.md @@ -0,0 +1,845 @@ +# Service Management + +This topic describes how to manage your operating system and services using the systemd. + + +- [Service Management](#service-management) + - [Introduction to systemd](#introduction-to-systemd) + - [Systemd Units](#systemd-units) + - [Features](#features) + - [Fast Activation](#fast-activation) + - [On-Demand Activation](#on-demand-activation) + - [Service Lifecycle Management by Cgroups](#service-lifecycle-management-by-cgroups) + - [Mount and Automount Point Management](#mount-and-automount-point-management) + - [Transactional Dependency Management](#transactional-dependency-management) + - [Compatibility with SysVinit Scripts](#compatibility-with-sysvinit-scripts) + - [System State Snapshots and System Restoration](#system-state-snapshots-and-system-restoration) + - [Managing System Services](#managing-system-services) + - [Comparison Between SysVinit and systemd Commands](#comparison-between-sysvinit-and-systemd-commands) + - [Listing Services](#listing-services) + - [Displaying Service Status](#displaying-service-status) + - [Starting a Service](#starting-a-service) + - [Stopping a Service](#stopping-a-service) + - [Restarting a Service](#restarting-a-service) + - [Enabling a Service](#enabling-a-service) + - [Disabling a Service](#disabling-a-service) + - [Changing a Runlevel](#changing-a-runlevel) + - [Targets and Runlevels](#targets-and-runlevels) + - [Viewing the Default Startup Target](#viewing-the-default-startup-target) + - [Viewing All Startup Targets](#viewing-all-startup-targets) + - [Changing the Default Target](#changing-the-default-target) + - [Changing the Current Target](#changing-the-current-target) + - [Changing to Rescue Mode](#changing-to-rescue-mode) + - [Changing to Emergency Mode](#changing-to-emergency-mode) + - [Shutting Down, Suspending, and Hibernating the Operating System](#shutting-down-suspending-and-hibernating-the-operating-system) + - [systemctl Command](#systemctl-command) + - [Shutting Down the Operating System](#shutting-down-the-operating-system) + - [Restarting the Operating System](#restarting-the-operating-system) + - [Suspending the Operating System](#suspending-the-operating-system) + - [Hibernating the Operating System](#hibernating-the-operating-system) + + + +## Introduction to systemd + +The systemd is a system and service manager for Linux operating systems. It is designed to be backward compatible with SysV and LSB init scripts, and provides a number of features such as Socket & D-Bus based activation of services, on-demand activation of daemons, system state snapshots, and mount & automount point management. With systemd, the service control logic and parallelization are refined. + +### Systemd Units + +In systemd, the targets of most actions are units, which are resources systemd know how to manage. Units are categorized by the type of resources they represent and defined in unit configuration files. For example, the avahi.service unit represents the Avahi daemon and is defined in the **avahi.service** file. [Table 1](#en-us_topic_0151921012_t2dcb6d973cc249ed9ccd56729751ca6b) lists available types of systemd units. + +**Table 1** Available types of systemd units + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Unit Type

+

File Extension

+

Description

+

Service unit

+

.service

+

A system service.

+

Target unit

+

.target

+

A group of systemd units.

+

Automount unit

+

.automount

+

A file system automount point.

+

Device unit

+

.device

+

A device file recognized by the kernel.

+

Mount unit

+

.mount

+

A file system mount point.

+

Path unit

+

.path

+

A file or directory in a file system.

+

Scope unit

+

.scope

+

An externally created process.

+

Slice unit

+

.slice

+

A group of hierarchically organized units that manage system processes.

+

Socket unit

+

.socket

+

An inter-process communication socket.

+

Swap unit

+

.swap

+

A swap device or a swap file.

+

Timer unit

+

.timer

+

A systemd timer.

+
+ +All available types of systemd units are located in one of the following directories listed in [Table 2](#en-us_topic_0151921012_t2523a0a9a0c54f9b849e52d1efa0160c). + +**Table 2** Locations of available systemd units + + + + + + + + + + + + + + + + +

Directory

+

Description

+

/usr/lib/systemd/system/

+

Systemd units distributed with installed RPM packages.

+

/run/systemd/system/

+

Systemd units created at runtime.

+

/etc/systemd/system/

+

Systemd units created and managed by the system administrator.

+
+ +## Features + +### Fast Activation + +The systemd provides more aggressive parallelization than UpStart. The use of Socket- and D-Bus based activation reduces the time required to boot the operating system. + +To accelerate system boot, systemd seeks to: + +- Activate only the necessary processes +- Activate as many processes as possible in parallel + +### On-Demand Activation + +During SysVinit initialization, it activates all the possible background service processes that might be used. Users can log in only after all these service processes are activated. The drawbacks in SysVinit are obvious: slow system boot and a waste of system resources. + +Some services may rarely or even never be used during system runtime. For example, CUPS, printing services are rarely used on most servers. SSHD is rarely accessed on many servers. It is unnecessary to spend time on starting these services and system resources. + +systemd can only be activated when a service is requested. If the service request is over, systemd stops. + +### Service Lifecycle Management by Cgroups + +An important role of an init system is to track and manage the lifecycle of services. It can start and stop a service. However, it is more difficult than you could ever imagine to encode an init system into stopping services. + +Service processes often run in background as daemons and sometimes fork twice. In UpStart, the expect stanza in the configuration file must be correctly configured. Otherwise, UpStart is unable to learn a daemon's PID by counting the number of forks. + +Things are made simpler with Cgroups, which have long been used to manage system resource quotas. The ease of use comes largely from its file-system-like user interface. When a parent service creates a child service, the latter inherits all attributes of the Cgroup to which the parent service belongs. This means that all relevant services are put into the same Cgroup. The systemd can find the PIDs of all relevant services simply by traversing their control group and then stop them one by one. + +### Mount and Automount Point Management + +In traditional Linux systems, users can use the **/etc/fstab** file to maintain fixed file system mount points. These mount points are automatically mounted during system startup. Once the startup is complete, these mount points are available. These mount points are file systems critical to system running, such as the **HOME** directory. Like SysVinit, systemd manages these mount points so that they can be automatically mounted at system startup. systemd is also compatible with the **/etc/fstab** file. You can continue to use this file to manage mount points. + +There are times when you need to mount or unmount on demand. For example, a temporary mounting point is required for you to access the DVD content, and the mounting point is canceled \(using the **umount** command\) if you no longer need to access the content, thereby saving resources. This is traditionally achieved using the autofs service. + +The systemd allows automatic mount without a need to install autofs. + +### Transactional Dependency Management + +System boot involves a host of separate jobs, some of which may be dependent on each other. For example, a network file system \(NFS\) can be mounted only after network connectivity is activated. The systemd can run a large number of dependent jobs in parallel, but not all of them. Looking back to the NFS example, it is impossible to mount NFS and activate network at the same time. Before running a job, systemd calculates its dependencies, creates a temporary transaction, and verifies that this transaction is consistent \(all relevant services can be activated without any dependency on each other\). + +### Compatibility with SysVinit Scripts + +Like UpStart, systemd introduces new configuration methods and has new requirements for application development. If you want to replace the currently running initialization system with systemd, systemd must be compatible with the existing program. It is difficult to modify all the service code in any Linux distribution in a short time for the purpose of using systemd. + +The systemd provides features compatible with SysVinit and LSB initscripts. You do not need to modify the existing services and processes in the system. This reduces the cost of migrating the system to systemd, making it possible for users to replace the existing initialization system with systemd. + +### System State Snapshots and System Restoration + +The systemd can be started on demand. Therefore, the running status of the system changes dynamically, and you cannot know the specific services that are running in the system. systemd snapshots enable the current system running status to be saved and restored. + +For example, if services A and B are running in the system, you can run the **systemd** command to create a snapshot for the current system running status. Then stop process A or make any other change to the system, for example, starting process C. After these changes, run the snapshot restoration command of systemd to restore the system to the point at which the snapshot was taken. That is, only services A and B are running. A possible application scenario is debugging. For example, when an exception occurs on the server, a user saves the current status as a snapshot for debugging, and then perform any operation, for example, stopping the service. After the debugging is complete, restore the snapshot. + +## Managing System Services + +The systemd provides the systemctl command to start, stop, restart, view, enable, and disable system services. + +### Comparison Between SysVinit and systemd Commands + +The **systemctl** command from the **systemd** command has the functions similar to the **SysVinit** command. Note that the **service** and **chkconfig** commands are supported in this version. For details, see [Table 3](#en-us_topic_0151920917_ta7039963b0c74b909b72c22cbc9f2e28). You are advised to manage system services by running the **systemctl** command. + +**Table 3** Comparison between SysVinit and systemd commands + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

SysVinit Command

+

systemd Command

+

Description

+

service network start

+

systemctl start network.service

+

Starts a service.

+

service network stop

+

systemctl stop network.service

+

Stops a service.

+

service network restart

+

systemctl restart network.service

+

Restarts a service.

+

service network reload

+

systemctl reload network.service

+

Reloads a configuration file without interrupting an operation.

+

service network condrestart

+

systemctl condrestart network.service

+

Restarts a service only if it is running.

+

service network status

+

systemctl status network.service

+

Checks the service running status.

+

chkconfig network on

+

systemctl enable network.service

+

Enables a service when the service activation time arrives or a trigger condition for enabling the service is met.

+

chkconfig network off

+

systemctl disable network.service

+

Disables a service when the service activation time arrives or a trigger condition for disabling the service is met.

+

chkconfig network

+

systemctl is-enabled network.service

+

Checks whether a service is enabled.

+

chkconfig --list

+

systemctl list-unit-files --type=service

+

Lists all services in each runlevel and checks whether they are enabled.

+

chkconfig network --list

+

ls /etc/systemd/system/*.wants/network.service

+

Lists the runlevels in which a service is enabled and those in which the service is disabled.

+

chkconfig network --add

+

systemctl daemon-reload

+

Used when you need to create a service file or change settings.

+
+ +### Listing Services + +To list all currently loaded services, run the following command: + +```shell +systemctl list-units --type service +``` + +To list all services regardless of whether they are loaded, run the following command \(with the `--all` option\): + +```shell +systemctl list-units --type service --all +``` + +Example list of all currently loaded services: + +```shell +$ systemctl list-units --type service +UNIT LOAD ACTIVE SUB DESCRIPTION +atd.service loaded active running Deferred execution scheduler +auditd.service loaded active running Security Auditing Service +avahi-daemon.service loaded active running Avahi mDNS/DNS-SD Stack +chronyd.service loaded active running NTP client/server +crond.service loaded active running Command Scheduler +dbus.service loaded active running D-Bus System Message Bus +dracut-shutdown.service loaded active exited Restore /run/initramfs on shutdown +firewalld.service loaded active running firewalld - dynamic firewall daemon +getty@tty1.service loaded active running Getty on tty1 +gssproxy.service loaded active running GSSAPI Proxy Daemon +...... +``` + +### Displaying Service Status + +To display the status of a service, run the following command: + +```shell +systemctl status name.service +``` + +[Table 4](#en-us_topic_0151920917_t36cd267d69244ed39ae06bb117ed8e62) describes the parameters in the command output. + +**Table 4** Output parameters + + + + + + + + + + + + + + + + + + + +

Parameter

+

Description

+

Loaded

+

Information on whether the service has been loaded, the absolute path to the service file, and a note of whether the service is enabled.

+

Active

+

Information on whether the service is running and a time stamp.

+

Main PID

+

PID of the service.

+

CGroup

+

Additional information about related control groups.

+
+ +To verify whether a particular service is running, run the following command: + +```shell +systemctl is-active name.service +``` + +The output of the **is-active** command is as follows: + +**Table 5** Output of the is-active command + + + + + + + + + + + + + + + + + + + +

Status

+

Description

+

active(running)

+

One or more services are running in the system.

+

active(exited)

+

A service that ends properly after being executed only once. Currently, no program is running in the system. For example, the quotaon function is performed only when the program is started or mounted.

+

active(waiting)

+

The program needs to wait for other events to continue running. For example, the print queue service is being started, but it needs to be queued (print jobs) so that it can continue to wake up the printer service to perform the next print function.

+

inactive

+

The service is not running.

+
+ +Similarly, to determine whether a particular service is enabled, run the following command: + +```shell +systemctl is-enabled name.service +``` + +The output of the **is-enabled** command is as follows: + +**Table 6** Output of the is-enabled command + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Status

+

Description

+

enabled

+

Has been permanently enabled through Alias= Alias, .wants/, or .requires/ soft link in the /etc/systemd/system/ directory.

+

enabled-runtime

+

Has been temporarily enabled through Alias= Alias, .wants/, or .requires/ soft link in the /run/systemd/system/ directory.

+

linked

+

Although the unit file is not in the standard unit directory, one or more soft links pointing to the unit file exist in the /etc/systemd/system/ permanent directory.

+

linked-runtime

+

Although the unit file is not in the standard unit directory, one or more soft links pointing to the unit file exist in the /run/systemd/system/ temporary directory.

+

masked

+

Has been masked permanently by the /etc/systemd/system/ directory (soft link to /dev/null). Therefore, the start operation fails.

+

masked-runtime

+

Has been masked temporarily by the /run/systemd/systemd/ directory (soft link to /dev/null). Therefore, the start operation fails.

+

static

+

Not enabled. There is no option available for the enable command in the [Install] section of the unit file.

+

indirect

+

Not enabled. But the list of values for the Also= option in the [Install] section of the unit file is not empty (that is, some units in the list may have been enabled), or the unit file has an alias soft link which is not in the Also= list. For a template unit, it indicates that an instance different from DefaultInstance= is enabled.

+

disabled

+

Not enabled. But the [Install] section of the unit file contains options available for the enable command.

+

generated

+

The unit file is dynamically generated by the unit generator. The generated unit file may not be directly enabled, but is implicitly enabled by the unit generator.

+

transient

+

The unit file is dynamically and temporarily generated by the runtime API. The temporary unit may not be enabled.

+

bad

+

The unit file is incorrect or other errors occur. is-enabled does not return this status, but displays an error message. The list-unit-files command may display this unit.

+
+ +For example, to display the status of gdm.service, run the **systemctl status gdm.service** command. + +```shell +# systemctl status gdm.service +gdm.service - GNOME Display Manager + Loaded: loaded (/usr/lib/systemd/system/gdm.service; enabled) + Active: active (running) since Thu 2013-10-17 17:31:23 CEST; 5min ago + Main PID: 1029 (gdm) + CGroup: /system.slice/gdm.service + ├─1029 /usr/sbin/gdm + ├─1037 /usr/libexec/gdm-simple-slave --display-id /org/gno... + └─1047 /usr/bin/Xorg :0 -background none -verbose -auth /r...Oct 17 17:31:23 localhost systemd[1]: Started GNOME Display Manager. +``` + +### Starting a Service + +To start a service, run the following command as the user **root**: + +```shell +systemctl start name.service +``` + +For example, to start the httpd service, run the following command: + +```shell +# systemctl start httpd.service +``` + +### Stopping a Service + +To stop a service, run the following command as the user **root**: + +```shell +systemctl stop name.service +``` + +For example, to stop the Bluetooth service, run the following command: + +```shell +# systemctl stop bluetooth.service +``` + +### Restarting a Service + +To restart a service, run the following command as the user **root**: + +```shell +systemctl restart name.service +``` + +This command stops the selected service in the current session and immediately starts it again. If the selected service is not running, this command starts it too. + +For example, to restart the Bluetooth service, run the following command: + +```shell +# systemctl restart bluetooth.service +``` + +### Enabling a Service + +To configure a service to start automatically at system boot time, run the following command as the user **root**: + +```shell +systemctl enable name.service +``` + +For example, to configure the httpd service to start automatically at system boot time, run the following command: + +```shell +# systemctl enable httpd.service +ln -s '/usr/lib/systemd/system/httpd.service' '/etc/systemd/system/multi-user.target.wants/httpd.service' +``` + +### Disabling a Service + +To prevent a service from starting automatically at system boot time, run the following command as the user **root**: + +```shell +systemctl disable name.service +``` + +For example, to prevent the Bluetooth service from starting automatically at system boot time, run the following command: + +```shell +# systemctl disable bluetooth.service +Removed /etc/systemd/system/bluetooth.target.wants/bluetooth.service. +Removed /etc/systemd/system/dbus-org.bluez.service. +``` + +## Changing a Runlevel + +### Targets and Runlevels + +In systemd, the concept of runlevels has been replaced with systemd targets to improve flexibility. For example, you can inherit an existing target and turn it into your own target by adding other services. [Table 7](#en-us_topic_0151920939_t9af92c282ad240ea9a79fb08d26e8181) provides a complete list of runlevels and their corresponding systemd targets. + +**Table 7** Mapping between runlevels and targets + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Runlevel

+

systemd Target

+

Description

+

0

+

runlevel0.target, poweroff.target

+

The operating system is powered off.

+

1, s, single

+

runlevel1.target, rescue.target

+

The operating system is in single user mode.

+

2, 4

+

runlevel2.target, runlevel4.target, multi-user.target

+

The operating system is in user-defined or domain-specific runlevel (by default, it is equivalent to runlevel 3).

+

3

+

runlevel3.target, multi-user.target

+

The operating system is in non-graphical multi-user mode, and can be accessed from multiple consoles or networks.

+

5

+

runlevel5.target, graphical.target

+

The operating system is in graphical multi-user mode. All the services running at level 3 can be accessed through graphical login.

+

6

+

runlevel6.target, reboot.target

+

The operating system is rebooted.

+

emergency

+

emergency.target

+

Emergency shell.

+
+ +### Viewing the Default Startup Target + +Run the following command to view the default startup target of the system: + +```shell +systemctl get-default +``` + +### Viewing All Startup Targets + +Run the following command to view all startup targets of the system: + +```shell +systemctl list-units --type=target +``` + +### Changing the Default Target + +To change the default target, run the following command as the user **root**: + +```shell +systemctl set-default name.target +``` + +### Changing the Current Target + +To change the current target, run the following command as the user **root**: + +```shell +systemctl isolate name.target +``` + +### Changing to Rescue Mode + +To change the operating system to rescue mode, run the following command as the user **root**: + +```shell +systemctl rescue +``` + +This command is similar to the **systemctl isolate rescue.target** command. After the command is executed, the following information is displayed on the serial port: + +```console +You are in rescue mode. After logging in, type "journalctl -xb" to viewsystem logs, "systemctl reboot" to reboot, "systemctl default" or "exit"to boot into default mode. +Give root password for maintenance +(or press Control-D to continue): +``` + +> [!NOTE]NOTE +> You need to restart the system to enter the normal mode from the rescue mode. + +### Changing to Emergency Mode + +To change the operating system to emergency mode, run the following command as the user **root**: + +```shell +systemctl emergency +``` + +This command is similar to the **systemctl isolate emergency.target** command. After the command is executed, the following information is displayed on the serial port: + +```console +You are in emergency mode. After logging in, type "journalctl -xb" to viewsystem logs, "systemctl reboot" to reboot, "systemctl default" or "exit"to boot into default mode. +Give root password for maintenance +(or press Control-D to continue): +``` + +> [!NOTE]NOTE +> You need to restart the system to enter the normal mode from the emergency mode. + +## Shutting Down, Suspending, and Hibernating the Operating System + +### systemctl Command + +The systemd uses the systemctl command instead of old Linux system management commands to shut down, restart, suspend, and hibernate the operating system. Although previous Linux system management commands are still available in systemd for compatibility reasons, you are advised to use **systemctl** when possible. The mapping relationship is shown in [Table 8](#en-us_topic_0151920964_t3daaaba6a03b4c36be9668efcdb61f3b). + +**Table 8** Mapping between old Linux system management commands and systemctl + + + + + + + + + + + + + + + + + + + + +

Linux Management Command

+

systemctl Command

+

Description

+

halt

+

systemctl halt

+

Shuts down the operating system.

+

poweroff

+

systemctl poweroff

+

Powers off the operating system.

+

reboot

+

systemctl reboot

+

Reboots the operating system.

+
+ +### Shutting Down the Operating System + +To shut down the system and power off the operating system, run the following command as the user **root**: + +```shell +systemctl poweroff +``` + +To shut down the operating system without powering it off, run the following command as the user **root**: + +```shell +systemctl halt +``` + +By default, running either of these commands causes systemd to send an informative message to all login users. To prevent systemd from sending this message, run this command with the **--no\-wall** option. The command is as follows: + +```shell +systemctl --no-wall poweroff +``` + +### Restarting the Operating System + +To restart the operating system, run the following command as the user **root**: + +```shell +systemctl reboot +``` + +By default, running either of these commands causes systemd to send an informative message to all login users. To prevent systemd from sending this message, run this command with the **--no\-wall** option. The command is as follows: + +```shell +systemctl --no-wall reboot +``` + +### Suspending the Operating System + +To suspend the operating system, run the following command as the user **root**: + +```shell +systemctl suspend +``` + +### Hibernating the Operating System + +To hibernate the operating system, run the following command as the user **root**: + +```shell +systemctl hibernate +``` + +To suspend and hibernate the operating system, run the following command as the user **root**: + +```shell +systemctl hybrid-sleep +``` diff --git a/docs/en/server/administration/administrator/setting-up-the-database-server.md b/docs/en/server/administration/administrator/setting-up-the-database-server.md new file mode 100644 index 0000000000000000000000000000000000000000..48ffe210a0c8a8115ffba299f2d7328c4c04dee1 --- /dev/null +++ b/docs/en/server/administration/administrator/setting-up-the-database-server.md @@ -0,0 +1,2169 @@ +# Setting Up the Database Server + + + +- [Setting Up the Database Server](#setting-up-the-database-server) + - [PostgreSQL Server](#postgresql-server) + - [Software Description](#software-description) + - [Configuring the Environment](#configuring-the-environment) + - [Installing, Running, and Uninstalling PostgreSQL](#installing-running-and-uninstalling-postgresql) + - [Managing Database Roles](#managing-database-roles) + - [Managing Databases](#managing-databases) + - [MariaDB Server](#mariadb-server) + - [Software Description](#software-description-1) + - [Configuring the Environment](#configuring-the-environment-1) + - [Installing, Running, and Uninstalling MariaDB Server](#installing-running-and-uninstalling-mariadb-server) + - [Managing Database Users](#managing-database-users) + - [Managing Databases](#managing-databases-1) + - [MySQL Server](#mysql-server) + - [Software Description](#software-description-2) + - [Configuring the Environment](#configuring-the-environment-2) + - [Installing, Running, and Uninstalling MySQL](#installing-running-and-uninstalling-mysql) + - [Managing Database Users](#managing-database-users-1) + - [Managing Databases](#managing-databases-2) + + + +## PostgreSQL Server + +### Software Description + +[Figure 1](#fig26022387391) shows the PostgreSQL architecture and [Table 1](#table62020913417) describes the main processes. + +**Figure 1** PostgreSQL architecture +![](./figures/postgresql-architecture.png) + +**Table 1** Main processes in PostgreSQL + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Process Type

+

Process Name

+

Description

+

Main process

+

Postmaster

+

Postmaster process controls all database instances in general and is responsible for starting and stopping database instances.

+

Resident process

+

Postgres (resident process)

+

This process manages backend resident processes and is also called postmaster. By default, this process listens Unix domain sockets and the 5432 port of TCP/IP and waits for the front end to process the connections. You can change the listening port number in the postgresql.conf file of PostgreSQL.

+

Subprocess

+

Postgres (subprocess)

+

The subprocess determines whether to allow the connection according to the security policy defined by the pg_hba.conf file. According to the security policy, the subprocess rejects certain IP addresses and networks, allows only certain users to connect to the databases, or allows only certain databases to be connected.

+

Postgres receives the query from the front end, searches the database, and returns the results. Sometimes, it also updates the database. The updated data is recorded in transaction logs (WAL logs for PostgreSQL). This method is used when the system is powered off, the server breaks down, or the server is restarted. In addition, the logs can also be used for data recovery in other scenarios. In PostgreSQL 9.0 or later, WAL logs can be transferred to other PostgreSQL systems to replicate database in real-time.

+

Auxiliary processes

+

SysLogger (system log)

+

The main process starts the Syslogger auxiliary process only when logging_collection in the Postgres.conf file is set to on.

+

BgWriter (background write)

+

This process writes dirty pages from the shared memory to the drive. The purpose is to improve the performance of inserting, updating, and deleting data.

+

WALWriter (write-ahead log)

+

This process writes modification operations into drives before data is modified so that the data does not need to be persisted into files in subsequent real-time data updates.

+

PgArch (archive)

+

write-ahead logs (WALs) are recycled. The PgArch process backs up WALs before archiving them. After the entire database is backed up, the Point in Time Recovery (PITR) technology can be used to archive WALs generated after the backup. The database can be restored to any point after the full backup by using the full backup data and the subsequently archived WALs.

+

AutoVacuum (automatic cleanup)

+

In the PostgreSQL database, after a DELETE operation is performed on a table, old data is not immediately deleted. When new data is added, the system creates a data row instead of overwriting the old data. The old data is only marked as deleted and will be cleared only when no other concurrent transactions are reading the data. In this case, the data is cleared by the AutoVacuum process.

+

PgStat (statistics collection)

+

This process collects data statistics. It is used to estimate the cost during query optimization, including the number of insertions update, and deletion operations performed on a table or index, the number of drive block read and write operations, and the number of row read operations. pg_statistic stores the information collected by the PgStat.

+

CheckPoint (checkpoint)

+

A checkpoint is a transaction sequence point set by the system. It is used to ensure that log information before a checkpoint written into the drives.

+
+ +### Configuring the Environment + +> [!NOTE]NOTE +> The following environment configuration is for reference only. Configure the environment based on the site requirements. + +#### Disabling the Firewall and Automatic Startup + +> [!NOTE]NOTE +> It is recommended that firewall be disabled in the test environment to prevent network impact. Configure the firewall based on actual requirements. + +1. Stop the firewall service as the **root** user. + + ```shell + systemctl stop firewalld + ``` + +2. Disable the firewall service as the **root** user. + + ```shell + systemctl disable firewalld + ``` + + > [!NOTE]NOTE + > The automatic startup is automatically disabled as the firewall is disabled. + +#### Disabling SELinux + +1. Modify the configuration file as the **root** user. + + ```shell + sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config + ``` + +#### Creating a User Group and a User + +> [!NOTE]NOTE +> In the server environment, independent users are assigned to each process to implement permission isolation for security purposes. The user group and user are created for the OS, not for the database. + +1. Create a PostgreSQL user or user group as the **root** user. + + ```shell + groupadd postgres + ``` + + ```shell + useradd -g postgres postgres + ``` + +2. Set the postgres user password as the **root** user. \(Enter the password twice for confirmation.\) + + ```shell + passwd postgres + ``` + +#### Creating Data Drives + +> [!NOTE]NOTE +> +> - When testing the ultimate performance, you are advised to attach NVMe SSDs with better I/O performance to create PostgreSQL test instances to avoid the impact of disk I/O on the performance test result. This section uses NVMe SSDs as an example. For details, see Step 1 to Step 4. +> - In a non-performance test, run the following command as the **root** user to create a data directory. Then skip this section. +> `mkdir /data` + +1. Create a file system \(xfs is used as an example as the **root** user. Create the file system based on the site requirements.\). If a file system has been created for a disk, an error will be reported when you run this command. You can use the **-f** parameter to forcibly create a file system. + + ```shell + mkfs.xfs /dev/nvme0n1 + ``` + +2. Create a data directory. + + ```shell + mkdir /data + ``` + +3. Mount disks. + + ```shell + mount -o noatime,nobarrier /dev/nvme0n1 /data + ``` + +#### Data Directory Authorization + +1. Modify the directory permission as the **root** user. + + ```shell + chown -R postgres:postgres /data/ + ``` + +### Installing, Running, and Uninstalling PostgreSQL + +#### Installing PostgreSQL + +1. Configure the local yum source. For details, see [Configuring the Repo Server](./configuring-the-repo-server.md). +2. Clear the cache. + + ```shell + dnf clean all + ``` + +3. Create a cache. + + ```shell + dnf makecache + ``` + +4. Install the PostgreSQL server as the **root** user. + + ```shell + dnf install postgresql-server + ``` + +5. Check the installed RPM package. + + ```shell + rpm -qa | grep postgresql + ``` + +#### Running PostgreSQL + +##### Initializing the Database + +> [!TIP]NOTICE +> Perform this step as the postgres user. + +1. Switch to the created PostgreSQL user. + + ```shell + su - postgres + ``` + +2. Initialize the database. In the command, **/usr/bin** is the directory where the **initdb** command is located. + + ```shell + /usr/bin/initdb -D /data/ + ``` + +##### Starting the Database + +1. Enable the PostgreSQL database. + + ```shell + /usr/bin/pg_ctl -D /data/ -l /data/logfile start + ``` + +2. Check whether the PostgreSQL database process is started properly. + + ```shell + ps -ef | grep postgres + ``` + + If the following information is displayed, the PostgreSQL processes have been started. + + ![](./figures/postgres.png) + +##### Logging In to the Database + +1. Log in to the database. + + ```shell + /usr/bin/psql -U postgres + ``` + + ![](./figures/login.png) + + > [!NOTE]NOTE + > You do not need to enter a password when logging in to the database for the first time. + +##### Configuring the Database Accounts and Passwords + +1. After login, set the postgres user password. + + ```shell + postgres=#alter user postgres with password '123456'; + ``` + + ![](./figures/en-us_image_0230050789.png) + +##### Exiting the Database + +1. Run **\\q** to exit from the database. + + ```shell + postgres=# \q + ``` + +##### Stopping the Database + +1. Stop the PostgreSQL database. + + ```shell + /usr/bin/pg_ctl -D /data/ -l /data/logfile stop + ``` + +#### Uninstalling PostgreSQL + +1. Stop the database as the postgres user. + + ```shell + /usr/bin/pg_ctl -D /data/ -l /data/logfile stop + ``` + +2. Run the **dnf remove postgresql-server** command as the user **root** to uninstall the PostgreSQL database. + + ```shell + dnf remove postgresql-server + ``` + +### Managing Database Roles + +#### Creating a Role + +You can use the **CREATE ROLE** statement or **createuser** command to create a role. The **createuser** command encapsulates the **CREATE ROLE** statement and needs to be executed on the shell GUI instead of the database GUI. + +```pgsql +CREATE ROLE rolename [ [ WITH ] option [ ... ] ]; +``` + +```shell +createuser rolename +``` + +In the preceding information: + +- **rolename**: indicates a role name. +- Parameters of the _option_ are as follows: + - **SUPERUSER \| NOSUPERUSER**: determines whether a new role is a superuser. If this parameter is not specified, the default value **NOSUPERUSER** is used, indicating that the role is not a superuser. + - **CREATEDB \| NOCREATEDB**: specifies whether a role can create a database. If this parameter is not specified, the default value **NOCREATEDB** is used, indicating that the role cannot create a database. + - **CREATEROLE \| NOCREATEROLE**: determines whether a role can create roles. If this parameter is not specified, the default value **NOCREATEROLE** is used, indicating that the role cannot create roles. + - **INHERIT \| NOINHERIT**: determines whether a role inherits the other roles' permissions in the group to which the role belongs. A role with the INHERIT attribute can automatically use any permissions that have been assigned to its direct or indirect group. If this parameter is not specified, the default value **INHERIT** is used. + - **LOGIN \| NOLOGIN**: determines whether a role can log in. A role with the LOGIN attribute can be considered as a user. A role without this attribute can be used to manage database permissions but is not a user. If this attribute is not specified, the default value **NOLOGIN** is used. However, if **CREATE USER** instead of **CREATE ROLE** is used to create a role, the LOGIN attribute is used by default. + - **\[ENCRYPTED \| UNENCRYPTED\] PASSWORD'password'**: password of a role. The password is valid only for roles with the LOGIN attribute. **ENCRYPTED \| UNENCRYPTED**: determines whether to encrypt the password. If this parameter is not specified, the value **ENCRYPTED** is used, that is, the password is encrypted. + - **VALID UNTIL'timestamp'**: specifies the timestamp when the password of a role expires. If this parameter is not specified, the password is permanently valid. + - **IN ROLE rolename1**: lists one or more existing roles. The new role _rolename_ will be added to and become a member of **rolename1**. + - **ROLE rolename2**: lists one or more existing roles. These roles will be automatically added as members of the new role _rolename_. That is, the new role is a user group. + +To run this command, you must have the CREATEROLE permission or is the database superuser. + +##### Example + + Create a role **roleexample1** who can log in. + +```shell +postgres=# CREATE ROLE roleexample1 LOGIN; +``` + + Create a role **roleexample2** with the password **123456**. + +```shell +postgres=# CREATE ROLE roleexample2 WITH LOGIN PASSWORD '123456'; +``` + + Create a role named **roleexample3**. + +```console +[postgres@localhost ~]$ createuser roleexample3 +``` + +#### Viewing Roles + +You can run the **SELECT** statement or the PostgreSQL meta-command **\\du** to view the role. + +```pgsql +SELECT rolename FROM pg_roles; +``` + +```pgsql +\du +``` + +In the preceding command, _rolename_ indicates the role name. + +##### Example + + View the **roleexample1** role. + +```shell +postgres=# SELECT roleexample1 from pg_roles; +``` + + View the existing roles. + +```shell +postgres=# \du +``` + +#### Modifying a Role + +##### Modifying a Username + +Use the **ALTER ROLE** statement to modify an existing role name. + +```pgsql +ALTER ROLE oldrolername RENAME TO newrolename; +``` + +In the preceding information: + +- _oldrolername_: original role name. +- _newrolename_: new role name. + +##### Example of Modifying a User + + Change the role name **roleexample1** to **roleexapme2**. + +```shell +postgres=# ALTER ROLE roleexample1 RENAME TO roleexample2; +``` + +##### Modifying a User Password + +Use the **ALTER ROLE** statement to modify the login password of a role. + +```pgsql +ALTER ROLE rolename PASSWORD 'password' +``` + +In the preceding information: + +- _rolename_: indicates a role name. +- _password_: password. + +##### Example of Modifying the Password of a Role + + Modify the password of **roleexample1** to **456789**. + +```shell +postgres=# ALTER ROLE roleexample1 WITH PASSWORD '456789'; +``` + +#### Deleting a Role + +You can use the **DROP ROLE** statement or **dropuser** command to delete a role. The **dropuser** command encapsulates the **DROP ROLE** statement and needs to be executed on the shell GUI instead of the database GUI. + +```pgsql +DROP ROLE rolename; +``` + +```shell +dropuser rolename +``` + +In the preceding command, _rolename_ indicates the role name. + +##### Example + + Delete the **userexample1** role. + +```shell +postgres=# DROP ROLE userexample1; +``` + + Delete the **userexample2** role. + +```console +[postgres@localhost ~]$ dropuser userexample2 +``` + +#### Role Permissions + +You can use the **GRANT** statement to grant permissions to a role. + +Grant the table operation permission to a role. + +```pgsql +GRANT { { SELECT | INSERT | UPDATE | DELETE | REFERENCES | TRIGGER } [,...] | ALL [ PRIVILEGES ] } ON [ TABLE ] tablename [, ...] TO { rolename | GROUP groupname | PUBLIC } [, ...] [ WITH GRANT OPTION ] +``` + +Grant the sequence operation permission to a role. + +```pgsql +GRANT { { USAGE | SELECT | UPDATE } [,...] | ALL [ PRIVILEGES ] } ON SEQUENCE sequencename [, ...] TO { rolename | GROUP groupname | PUBLIC } [, ...] [ WITH GRANT OPTION ] +``` + +Grant the database operation permission to a role. + +```pgsql +GRANT { { CREATE | CONNECT | TEMPORARY | TEMP } [,...] | ALL [ PRIVILEGES ] } ON DATABASE databasename [, ...] TO { rolename | GROUP groupname | PUBLIC } [, ...] [ WITH GRANT OPTION ] +``` + +Grant the function operation permission to a role. + +```pgsql +GRANT { EXECUTE | ALL [ PRIVILEGES ] } ON FUNCTION funcname ( [ [ argmode ] [ argname ] argtype [, ...] ] ) [, ...] TO { rolename | GROUP groupname | PUBLIC } [, ...] [ WITH GRANT OPTION ] +``` + +Grant the operation permission of the procedural language to a role. + +```pgsql +GRANT { USAGE | ALL [ PRIVILEGES ] } ON LANGUAGE langname [, ...] TO { rolename | GROUP groupname | PUBLIC } [, ...] [ WITH GRANT OPTION ] +``` + +Grant the schema operation permission to a role. + +```pgsql +GRANT { { CREATE | USAGE } [,...] | ALL [ PRIVILEGES ] } ON SCHEMA schemaname [, ...] TO { rolename | GROUP groupname | PUBLIC } [, ...] [ WITH GRANT OPTION ] +``` + +Grant the tablespace operation permission to a role. + +```pgsql +GRANT { CREATE | ALL [ PRIVILEGES ] } ON TABLESPACE tablespacename [, ...] TO { rolename | GROUP groupname | PUBLIC } [, ...] [ WITH GRANT OPTION ] +``` + +Assign the member relationship of rolename1 to rolename2. + +```pgsql +GRANT rolename1 [, ...] TO rolename2 [, ...] [ WITH ADMIN OPTION ] +``` + +In the preceding information: + +- **SELECT**, **INSERT**, **UPDATE**, **DELETE**, **REFERENCES**, **TRIGGER**, **USAGE**, **CREATE**, **CONNECT**, **TEMPORARY**, **TEMP**, **EXECUTE**, and **ALL \[**_PRIVILEGES_**\]** indicate user operation permissions. **ALL \[**_PRIVILEGES_**\]** indicates all permissions, the _PRIVILEGES_ keyword is optional in PostgreSQL, but it is required in strict SQL statements. +- **ON** clause: specifies the object on which the permission is granted. +- **tablename**: table name. +- **TO** clause: specifies the role to which the permission is granted. +- **rolename**, **rolename1**, and **rolename2**: role names. +- **groupname**: name of a role group. +- **PUBLIC**: indicates that the permission is granted to all roles, including users who may be created later. +- **WITH GRANT OPTION**: indicates that the recipient of a permission can grant the permission to others. This option cannot be assigned to PUBLIC. +- **sequencename**: sequence name. +- **databasename**: database name. +- **funcname \(\[\[argmode\] \[argname\] argtype \[, ...\]\]\)**: function name and its parameters. +- **langname**: procedural language name. +- **schemaname**: schema name. +- **tablespacename**: tablespace name. +- **WITH ADMIN OPTION**: A member can assign the member relationship of a role to other roles and cancel the member relationship of other roles. + +##### Example + + Grant the CREATE permission on database1 to userexample. + +```shell +postgres=# GRANT CREATE ON DATABASE database1 TO userexample; +``` + + Grant all permissions on table1 to all users. + +```shell +postgres=# GRANT ALL PRIVILEGES ON TABLE table1 TO PUBLIC; +``` + +#### Deleting User Permissions + +You can use the **REVOKE** statement to revoke the permissions previously granted to one or more roles. + +Revoke the table operation permission from a role. + +```pgsql +REVOKE [ GRANT OPTION FOR ] { { SELECT | INSERT | UPDATE | DELETE | REFERENCES | TRIGGER } [,...] | ALL [ PRIVILEGES ] } ON [ TABLE ] tablename [, ...] FROM { rolename | GROUP groupname | PUBLIC } [, ...] +``` + +Revoke the sequence operation permission from a role. + +```pgsql +REVOKE [ GRANT OPTION FOR ] { { USAGE | SELECT | UPDATE } [,...] | ALL [ PRIVILEGES ] } ON SEQUENCE sequencename [, ...] FROM { rolename | GROUP groupname | PUBLIC } [, ...] [ CASCADE | RESTRICT ] +``` + +Revoke the database operation permission from a role. + +```pgsql +REVOKE [ GRANT OPTION FOR ] { { CREATE | CONNECT | TEMPORARY | TEMP } [,...] | ALL [ PRIVILEGES ] } ON DATABASE databasename [, ...] FROM { rolename | GROUP groupname | PUBLIC } [, ...] [ CASCADE | RESTRICT ] +``` + +Revoke the function operation permission from a role. + +```pgsql +REVOKE [ GRANT OPTION FOR ] { EXECUTE | ALL [ PRIVILEGES ] } ON FUNCTION funcname ( [ [ argmode ] [ argname ] argtype [, ...] ] ) [, ...] FROM { rolename | GROUP groupname | PUBLIC } [, ...] [ CASCADE | RESTRICT ] +``` + +Revoke the procedural language operation permission from a role. + +```pgsql +REVOKE [ GRANT OPTION FOR ] { USAGE | ALL [ PRIVILEGES ] } ON LANGUAGE langname [, ...] FROM { rolename | GROUP groupname | PUBLIC } [, ...] [ CASCADE | RESTRICT ] +``` + +Revoke the schema operation permission from a role. + +```pgsql +REVOKE [ GRANT OPTION FOR ] { { CREATE | USAGE } [,...] | ALL [ PRIVILEGES ] } ON SCHEMA schemaname [, ...] FROM { rolename | GROUP groupname | PUBLIC } [, ...] [ CASCADE | RESTRICT ] +``` + +Revoke the tablespace operation permission from a role. + +```pgsql +REVOKE [ GRANT OPTION FOR ] { CREATE | ALL [ PRIVILEGES ] } ON TABLESPACE tablespacename [, ...] FROM { rolename | GROUP groupname | PUBLIC } [, ...] [ CASCADE | RESTRICT ] +``` + +Revoke the member relationship of rolename1 from rolename2. + +```pgsql +REVOKE [ ADMIN OPTION FOR ] rolename1 [, ...] FROM rolename2 [, ...] [ CASCADE | RESTRICT ] +``` + +In the preceding information: + +- **GRANT OPTION FOR**: The permission cannot be granted to others, but permission itself is not revoked. +- **SELECT**, **INSERT**, **UPDATE**, **DELETE**, **REFERENCES**, **TRIGGER**, **USAGE**, **CREATE**, **CONNECT**, **TEMPORARY**, **TEMP**, **EXECUTE**, and **ALL \[**_PRIVILEGES_**\]** indicate user operation permissions. **ALL \[**_PRIVILEGES_**\]** indicates all permissions, the _PRIVILEGES_ keyword is optional in PostgreSQL, but it is required in strict SQL statements. +- **ON** clause: specifies the object on which the permission is revoked. +- _tablename_: table name. +- **FROM** clause: specifies the role whose permission is revoked. +- _rolename_, _rolename1_, and _rolename2_: role names. +- _groupname_: name of a role group. +- **PUBLIC**: revokes the implicitly defined groups that have all roles. However, this does not mean that all roles lose the permissions. The permissions directly obtained and the permissions obtained through a group are still valid. +- _sequencename_: sequence name. +- **CASCADE**: revokes all dependent permissions. +- **RESTRICT**: does not revoke all dependent permissions. +- _databasename_: database name. +- **funcname \(**_\[\[argmode\] \[argname\] argtype \[, ...\]\]_**\)**: function name and its parameters. +- _langname_: procedural language name. +- _schemaname_: schema name. +- _tablespacename_: tablespace name. +- **ADMIN OPTION FOR**: The transferred authorization is not automatically revoked. + +##### Example + + Grant the CREATE permission on database1 to userexample. + +```shell +postgres=# GRANT CREATE ON DATABASE database1 TO userexample; +``` + + Grant all permissions on table1 to all users. + +```shell +postgres=# GRANT ALL PRIVILEGES ON TABLE table1 TO PUBLIC; +``` + +### Managing Databases + +#### Creating a Database + +You can use the **CREATE DATABASE** statement or the **createdb** command to create a database. The **createdb** command encapsulates the **CREATE DATABASE** statement and needs to be executed on the shell GUI instead of the database GUI. + +```pgsql +CREATE DATABASE databasename; +``` + +```shell +createdb databasename +``` + +In the preceding command, **databasename** indicates the database name. + +To use this command, you must have the CREATEDB permission. + +##### Example + + Create a database named **database1**. + +```shell +postgres=# CREATE DATABASE database1; +``` + +#### Selecting a Database + +Use the **\\c** statement to select a database. + +```pgsql +\c databasename; +``` + +In the preceding command, **databasename** indicates the database name. + +##### Example + + Select the **databaseexample** database. + +```shell +postgres=# \c databaseexample; +``` + +#### Viewing a Database + +Use the **\\l** statement to view the database. + +```pgsql +\l; +``` + +##### Example + + View all databases. + +```shell +postgres=# \l; +``` + +#### Deleting a Database + +You can run the **DROP DATABASE** statement or **dropdb** command to delete a database. The **dropdb** command encapsulates the **DROP DATABASE** statement and needs to be executed on the shell GUI instead of the database GUI. + +> [!WARNING]CAUTION +> Exercise caution when deleting a database. Once a database is deleted, all tables and data in the database will be deleted. + +```pgsql +DROP DATABASE databasename; +``` + +```shell +dropdb databasename +``` + +In the preceding command, **databasename** indicates the database name. + +The **DROP DATABASE** statement deletes the system directory items of the database and the file directories that contain data. + +**DROP DATABASE** can be executed only by the super administrator or database owner. + +##### Example + + Delete the **databaseexample** database. + +```shell +postgres=# DROP DATABASE databaseexample; +``` + +#### Backing Up a Database + +Run the **pg\_dump** command to back up the database and dump the database to a script file or another archive file. + +```shell +pg_dump [option]... [databasename] > outfile +``` + +In the preceding information: + +- _databasename_: database name. If this parameter is not specified, the environment variable **PGDATABASE** is used. If that environment variable is not specified, use the username that initiates the connection. +- _outfile_: database backup file. +- _option_: parameter option of the **pg\_dump** command. Multiple parameters can be separated by spaces. The common parameters of the **pg\_dump** command are as follows: + - **-f, \-\-file**= _filename_: specified output file. If this parameter is ignored, the standard output is used. + - **-d, \-\-dbname**= _databasename_: database to be dumped. + - **-h, \-\-host**= _hostname_: specifies the hostname. + - **-p, \-\-port**= _portnumber_: port number. + - **-U, \-\-username**= _username_: username of the connection. + - **-W, \-\-password**: forces PostgreSQL to prompt for a password before connecting to a database. + +##### Example + + Back up the database1 database of user **postgres** on port **3306** of the host whose IP address is **192.168.202.144** to the **db1.sql** file. + +```shell +[postgres@localhost ~]$ pg_dump -h 192.168.202.144 -p 3306 -U postgres -W database1 > db1.sql +``` + +#### Restoring a Database + +Run the **psql** command to restore the database. + +```shell +psql [option]... [databasename [username]] < infile +``` + +In the preceding information: + +- _databasename_: database name. If this parameter is not specified, the environment variable **PGDATABASE** is used. If that environment variable is not specified, use the username that initiates the connection. +- _username_: name of a user. +- _infile_: **outfile** parameter in the **pg\_dump** command. +- _option_: parameter option of the **psql** command. Multiple parameters can be separated by spaces. The common parameters of the **psql** command are as follows: + - **-f, \-\-file**= _filename_: specified output file. If this parameter is ignored, the standard output is used. + - **-d, \-\-dbname**= _databasename_: database to be dumped. + - **-h, \-\-host**= _hostname_: specifies the hostname. + - **-p, \-\-port**= _portnumber_: port number. + - **-U, \-\-username**= _username_: username of the connection. + - **-W, \-\-password**: forces PostgreSQL to prompt for a password before connecting to a database. + +The **psql** command cannot be used to automatically create the **databasename** database. Therefore, you need to create the **databasename** database before running the **psql** command to restore the database. + +##### Example + + Import the **db1.sql** script file to the newdb database of the postgres user on the host **192.168.202.144** through port **3306**. + +```shell +[postgres@localhost ~]$ createdb newdb +[postgres@localhost ~]$ psql -h 192.168.202.144 -p 3306 -U postgres -W -d newdb < db1.sql +``` + +## MariaDB Server + +### Software Description + +The MariaDB database management system is a branch of MySQL and is maintained by the open-source community. The MariaDB database management system uses the General Public License \(GPL\). MariaDB is designed to be fully compatible with MySQL, including APIs and command lines, so that it can easily replace MySQL. MariaDB also provides many new features. + +[Figure 2](#fig13492418164520) shows the MariaDB architecture. + +**Figure 2** MariaDB logical architecture +![](./figures/mariadb-logical-architecture.png) + +When MariaDB receives a SQL statement, the execution process is as follows: + +1. When a client connects to MariaDB, the hostname, username, and password of the client are authenticated. The authentication function can be implemented as a plug-in. +2. If the login is successful, the client sends SQL commands to the server. The parser parses the SQL statements. +3. The server checks whether the client has the permission to obtain the required resources. +4. If the query has been stored in the query cache, the result is returned immediately. +5. The optimizer will find the fastest execution policy or plan. That is, the optimizer can determine which tables will be read, which indexes will be accessed, and which temporary tables will be used. A good policy can reduce a large number of disk access and sorting operations. +6. Storage engines read and write data and index files. Caches are used to accelerate these operations. Other features such as transactions and foreign keys are processed at the storage engine layer. + +Storage engines manage and control data at the physical layer. They manage data files, data, indexes, and caches, making data management and reading more efficient. Each table has a .frm file that contains table definitions. + +Each storage engine manages and stores data in different ways, and supports different features and performance. For example: + +- MyISAM: suitable for environments with more reads and fewer writes. It does not support transactions and supports full-text indexes. +- noDB: supports transactions, row locks, and foreign keys. +- MEMORY: stores data in the memory. +- CSV: stores data in CSV format. + +### Configuring the Environment + +> [!NOTE]NOTE +> The following environment configuration is for reference only. Configure the environment based on the site requirements. + +#### Disabling the Firewall and Automatic Startup + +> [!NOTE]NOTE +> It is recommended that firewall be disabled in the test environment to prevent network impact. Configure the firewall based on actual requirements. + +1. Stop the firewall service as the **root** user. + + ```shell + systemctl stop firewalld + ``` + +2. Disable the firewall service as the **root** user. + + ```shell + systemctl disable firewalld + ``` + + > [!NOTE]NOTE + > The automatic startup is automatically disabled as the firewall is disabled. + +#### Disabling SELinux + +1. Modify the configuration file as the **root** user. + + ```shell + sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux + ``` + +#### Creating a User Group and a User + +> [!NOTE]NOTE +> In the server environment, independent users are assigned to each process to implement permission isolation for security purposes. The user group and user are created for the OS, not for the database. + +1. Create a MySQL user or user group as the **root** user. + + ```shell + groupadd mysql + ``` + + ```shell + useradd -g mysql mysql + ``` + +2. Set the user password as the **root** user. + + ```shell + passwd mysql + ``` + + Enter the password twice for confirmation. + +#### Creating Data Drives + +> [!NOTE]NOTE +> +> - If a performance test needs to be performed, an independent drive is required for the data directory. You need to format and mount the drive. For details, see Method 1 or Method 2. +> - In a non-performance test, run the following command as the **root** user to create a data directory. Then skip this section. +> `mkdir /data` + +##### Method 1: Using fdisk for Drive Management as the **root** user + +1. Create a partition, for example, **/dev/sdb**. + + ```shell + fdisk /dev/sdb + ``` + +2. Enter **n** and press **Enter**. +3. Enter **p** and press **Enter**. +4. Enter **1** and press **Enter**. +5. Retain the default settings and press **Enter**. +6. Retain the default settings and press **Enter**. +7. Enter **w** and press **Enter**. +8. Create a file system, for example, **xfs**. + + ```shell + mkfs.xfs /dev/sdb1 + ``` + +9. Mount the partition to **/data** for the OS. + + ```shell + mkdir /data + ``` + + ```shell + mount /dev/sdb1 /data + ``` + +10. Run the **vi /etc/fstab** command and edit the **/etc/fstab** file to enable the data drive to be automatically mounted after the system is restarted. For example, add the content in the last line, as shown in the following figure. + + In the last line, **/dev/nvme0n1p1** is only an example. + + ![](./figures/creat_datadisk1.png) + +##### Method 2: Using LVM for Drive Management as the **root** user + +> [!NOTE]NOTE +> Install the LVM2 package in the image as follows: +> Configure the local yum source. For details, see [Configuring the Repository Server](./configuring-the-repo-server.md). If the repository has been configured, skip this step. Then, install LVM2 by running **yum install lvm2**. + +1. Create a physical volume, for example, **sdb**. + + ```shell + pvcreate /dev/sdb + ``` + +2. Create a physical volume group, for example, **datavg**. + + ```shell + vgcreate datavg /dev/sdb + ``` + +3. Create a logical volume, for example, **datalv** of 600 GB. + + ```shell + lvcreate -L 600G -n datalv datavg + ``` + +4. Create a file system. + + ```shell + mkfs.xfs /dev/datavg/datalv + ``` + +5. Create a data directory and mount it. + + ```shell + mkdir /data + ``` + + ```shell + mount /dev/datavg/datalv /data + ``` + +6. Run the **vi /etc/fstab** command and edit the **/etc/fstab** file to enable the data drive to be automatically mounted after the system is restarted. For example, add the content in the last line, as shown in the following figure. + + In the last line, **/dev/datavg/datalv** is only an example. + + ![](./figures/d1376b2a-d036-41c4-b852-e8368f363b5e.png) + +#### Creating a Database Directory and Granting Permissions + +1. In the created data directory **/data**, create directories for processes and grant permissions to the MySQL group or user created as the **root** user. + + ```shell + mkdir -p /data/mariadb + cd /data/mariadb + mkdir data tmp run log + chown -R mysql:mysql /data + ``` + +### Installing, Running, and Uninstalling MariaDB Server + +#### Installing MariaDB + +1. Configure the local yum source. For details, see [Configuring the Repo Server](./configuring-the-repo-server.md). +2. Clear the cache. + + ```shell + dnf clean all + ``` + +3. Create a cache. + + ```shell + dnf makecache + ``` + +4. Install the MariaDB server. + + ```shell + dnf install mariadb-server + ``` + +5. Check the installed RPM package. + + ```shell + rpm -qa | grep mariadb + ``` + +#### Running MariaDB Server + +1. Start the MariaDB server as the **root** user. + + ```shell + systemctl start mariadb + ``` + +2. Initialize the database as the **root** user. + + ```shell + /usr/bin/mysql_secure_installation + ``` + + During the command execution, you need to enter the password of the database user **root**. If no password is set, press **Enter**. Then, set the password as prompted. + +3. Log in to the database. + + ```shell + mysql -u root -p + ``` + + After the command is executed, the system prompts you to enter the password. The password is the one set in [2](#li197143190587). + + > [!NOTE]NOTE + > Run the **\\q** or **exit** command to exit the database. + +#### Uninstalling MariaDB + +1. Stop the database process as the **root** user. + + ```shell + $ ps -ef | grep mysql + # kill -9 PID + ``` + +2. Run the **dnf remove mariadb-server** command as the **root** user to uninstall MariaDB. + + ```shell + dnf remove mariadb-server + ``` + +### Managing Database Users + +#### Creating Users + +Run the **CREATE USER** statement to create one or more users and set corresponding passwords. + +```pgsql +CREATE USER 'username'@'hostname' IDENTIFIED BY 'password'; +``` + +In the preceding information: + +- _username_: name of a user. +- _host_: hostname, that is, the name of the host where the user connects to the database. As a local user, you can set the parameter to **localhost**. If the host name is not specified during user creation, the host name is **%** by default, indicating a group of hosts. +- _password_: password for logging in to the server. The password can be null. If the password is null, the user can log in to the server without entering the password. This method, however, is not recommended because it provides low security. + +To use the **CREATE USER** statement, you must have the INSERT permission on the database or the global CREATE USER permission. + +After a user account is created using the **CREATE USER** statement, a record is added to the user table in the database. If the account to be created exists, an error will occur during statement execution. + +A new user has few permissions and can perform only operations that do not require permissions. For example, a user can run the **SHOW** statement to query the list of all storage engines and character sets. + +##### Example + + Create a local user whose password is 123456 and username is userexample1. + +```pgsql +> CREATE USER 'userexample1'@'localhost' IDENTIFIED BY '123456'; +``` + + Create a user whose password is 123456, username is userexample2, and hostname is 192.168.1.100. + +```pgsql +> CREATE USER 'userexample2'@'192.168.1.100' IDENTIFIED BY '123456'; +``` + +#### Viewing Users + +Run the **SHOW GRANTS** or **SELECT** statement to view one or more users. + +View a specific user: + +```pgsql +SHOW GRANTS [FOR 'username'@'hostname']; +``` + +```pgsql +SELECT USER,HOST,PASSWORD FROM mysql.user WHERE USER='username'; +``` + +View all users: + +```pgsql +SELECT USER,HOST,PASSWORD FROM mysql.user; +``` + +In the preceding information: + +- _username_: name of a user. +- _hostname_: host name. + +##### Example + + View the user userexample1. + +```pgsql +> SHOW GRANTS FOR 'userexample1'@'localhost'; +``` + + View all users in the MySQL database. + +```pgsql +> SELECT USER,HOST,PASSWORD FROM mysql.user; +``` + +#### Modifying Users + +##### Modifying a Username + +Run the **RENAME USER** statement to change one or more existing usernames. + +```pgsql +RENAME USER 'oldusername'@'hostname' TO 'newusername'@'hostname'; +``` + +In the preceding information: + +- _oldusername_: original username. +- _newusername_: new username. +- _hostname_: host name. + +The **RENAME USER** statement is used to rename an existing account. If the original account does not exist in the system or the new account exists, an error will occur when the statement is executed. + +To use the **RENAME USER** statement, you must have the UPDATE permission on the database or the global CREATE USER permission. + +##### Example of Modifying a User + + Change the username **userexample1** to **userexample2** and change the hostname to **localhost**. + +```pgsql +> RENAME USER 'userexample1'@'localhost' TO 'userexample2'@'localhost'; +``` + +##### Modifying a User Password + +Use the **SET PASSWORD** statement to modify the login password of a user. + +```pgsql +SET PASSWORD FOR 'username'@'hostname' = PASSWORD('newpassword'); +``` + +In the preceding information: + +- **FOR 'username'@'hostname'**: specifies the username and hostname whose password is to be changed. This parameter is optional. +- **PASSWORD\('newpassword'\)**: indicates that the **PASSWORD\(\)** function is used to set a new password. That is, the new password must be transferred to the **PASSWORD\(\)** function for encryption. + +> [!WARNING]CAUTION +> The **PASSWORD\(\)** function is a unidirectional encryption function. Once encrypted, the original plaintext cannot be decrypted. + +If the **FOR** clause is not added to the **SET PASSWORD** statement, the password of the current user is changed. + +The **FOR** clause must be given in the format of **'**_username_**'@'**_hostname_**'**, where _username_ indicates the username of the account and _hostname_ indicates the hostname of the account. + +The account whose password is to be changed must exist in the system. Otherwise, an error occurs when the statement is executed. + +##### Example of Changing a User Password + + Change the password of user **userexample** whose hostname is **localhost** to **0123456**. + +```pgsql +> SET PASSWORD FOR 'userexample'@'localhost' = PASSWORD('0123456') ; +``` + +#### Deleting Users + +Use the **DROP USER** statement to delete one or more user accounts and related permissions. + +```pgsql +DROP USER 'username1'@'hostname1' [,'username2'@'hostname2']...; +``` + +> [!WARNING]CAUTION +> The deletion of users does not affect the tables, indexes, or other database objects that they have created, because the database does not record the accounts that have created these objects. + +The **DROP USER** statement can be used to delete one or more database accounts and their original permissions. + +To use the **DROP USER** statement, you must have the DELETE permission on the database or the global CREATE USER permission. + +In the **DROP USER** statement, if the hostname of an account is not specified, the hostname is **%** by default. + +##### Example + + Delete the local user **userexample**. + +```pgsql +> DROP USER 'userexample'@'localhost'; +``` + +#### Granting Permissions to a User + +Run the **GRANT** statement to grant permissions to a new user. + +```pgsql +GRANT privileges ON databasename.tablename TO 'username'@'hostname'; +``` + +In the preceding information: + +- **ON** clause: specifies the object and its level on which the permission is granted. +- **privileges**: indicates the operation permissions of a user, such as **SELECT**, INSERT, and **UPDATE**. To grant all permissions to a user, use **ALL**. +- _databasename_: database name. +- _tablename_: table name. +- **TO** clause: sets the user password and specifies the user to whom the permission is granted. +- _username_: name of a user. +- _hostname_: host name. + +To grant the user the permission to operate all databases and tables, use asterisks \(\*\), for example, **\*.\***. + +If you specify a password for an existing user in the **TO** clause, the new password will overwrite the original password. + +If the permission is granted to a non-existent user, a **CREATE USER** statement is automatically executed to create the user, but the password must be specified for the user. + +##### Example + + Grant the SELECT and INSERT permissions to local user userexample. + +```pgsql +> GRANT SELECT,INSERT ON *.* TO 'userexample'@'localhost'; +``` + +#### Deleting User Permissions + +Run the **REVOKE** statement to delete the permissions of a user, but the user will not be deleted. + +```pgsql +REVOKE privilege ON databasename.tablename FROM 'username'@'hostname'; +``` + +The parameters in the **REVOKE** statement are the same as those in the **GRANT** statement. + +To use the **REVOKE** statement, you must have the global CREATE USER or UPDATE permission for the database. + +##### Example + + Delete the INSERT permission of local user userexample. + +```pgsql +> REVOKE INSERT ON *.* FROM 'userexample'@'localhost'; +``` + +### Managing Databases + +#### Creating a Database + +Run the **CREATE DATABASE** statement to create a database. + +```pgsql +CREATE DATABASE databasename; +``` + +In the preceding command, _databasename_ can be replaced with the database name, which is case insensitive. + +##### Example + + Create a database named **databaseexample**. + +```pgsql +> CREATE DATABASE databaseexample; +``` + +#### Viewing a Database + +Run the **SHOW DATABASES** statement to view a database. + +```pgsql +SHOW DATABASES; +``` + +##### Example + + View all databases. + +```pgsql +> SHOW DATABASES; +``` + +#### Selecting a Database + +Generally, you need to select a target database before creating or querying a table. Use the **USE** statement to select a database. + +```pgsql +USE databasename; +``` + +In the preceding command, **databasename** indicates the database name. + +##### Example + + Select the **databaseexample** database. + +```pgsql +> USE databaseexample; +``` + +#### Deleting a Database + +You can run the **DROP DATABASE** statement to delete a database. + +> [!WARNING]CAUTION +> Exercise caution when deleting a database. Once a database is deleted, all tables and data in the database will be deleted. + +```pgsql +DROP DATABASE databasename; +``` + +In the preceding command, **databasename** indicates the database name. + +The **DROP DATABASE** command is used to delete an existing database. After this command is executed, all tables in the database are deleted, but the user permissions of the database are not automatically deleted. + +To use **DROP DATABASE**, you need the **DROP** permission on the database. + +**DROP SCHEMA** is a synonym of **DROP DATABASE**. + +##### Example + + Delete the **databaseexample** database. + +```pgsql +> DROP DATABASE databaseexample; +``` + +#### Backing Up a Database + +Run the **mysqldump** command as the **root** user to back up the database. + +Back up one or more tables. + +```shell +mysqldump [options] databasename [tablename ...] > outfile +``` + +Back up one or more databases: + +```shell +mysqldump [options] -databases databasename ... > outfile +``` + +Back up all databases: + +```shell +mysqldump [options] -all-databases > outputfile +``` + +In the preceding information: + +- _databasename_: database name. +- _tablename_: name of a data table. +- _outfile_: database backup file. +- _options_: parameter option of the **mysqldump** command. Multiple parameters can be separated by spaces. The common parameters of the **mysqldump** command are as follows: + - **-u, \-\-user**= _username_: specifies the username. + - **-p, \-\-password**\[= _password_\]: specifies the password. + - **-P, \-\-port**= _portnumber_: specifies the port number. + - **-h, \-\-host**= _hostname_: specifies the hostname. + - **-r, \-\-result-file**= _filename_: saves the export result to a specified file, which is equivalent to **\>**. + - **-t**: backs up data only. + - **-d**: backs up the table structure only. + +##### Example + + Back up all the databases of the user **root** on the host **192.168.202.144** through port **3306** to the **alldb.sql** file. + +```shell +mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 --all-databases > alldb.sql +``` + + Back up the db1 database of the user **root** on the host **192.168.202.144** through port **3306** to the **db1.sql** file. + +```shell +mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 --databases db1 > db1.sql +``` + + Back up the tb1 table of the db1 database of the user **root** on the host **192.168.202.144** through port **3306** to the **db1tb1.sql** file. + +```shell +mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 db1 tb1 > db1tb1.sql +``` + + Back up only the table structure of the db1 database of user **root** on port **3306** of the host whose IP address is **192.168.202.144** to the **db1.sql** file. + +```shell +mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 -d db1 > db1.sql +``` + + Back up only the data of the db1 database of the user **root** on the host **192.168.202.144** through port **3306** to the **db1.sql** file. + +```shell +mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 -t db1 > db1.sql +``` + +#### Restoring a Database + +Run the **mysql** command as the **root** user to restore the database. + +Restore one or more tables: + +```shell +mysql -h hostname -P portnumber -u username -ppassword databasename < infile +``` + +In the preceding information: + +- _hostname_: host name. +- _portnumber_: port number. +- _username_: name of a user. +- _password_: password. +- _databasename_: database name. +- _infile_: **outfile** parameter in the **mysqldump** command. + +##### Example + + Restore a database. + +```shell +mysql -h 192.168.202.144 -P 3306 -uroot -p123456 -t db1 < db1.sql +``` + +## MySQL Server + +### Software Description + +MySQL is a relational database management system \(RDBMS\) developed by the Swedish company MySQL AB, which was bought by Sun Microsystems \(now Oracle\). It is one of the most popular Relational Database Management Systems \(RDBMSs\) in the industry, especially for web applications. + +A relational database stores data in different tables instead of in a large data warehouse to improve efficiency and flexibility. + +The Structured Query Language \(SQL\) used by MySQL is the most common standard language for accessing databases. MySQL uses dual-licensing distribution and is available in two editions: Community Edition and Commercial Edition. MySQL is optimal for small or medium-sized websites because of its small size, fast speed, low cost, and especially the open source code. + +### Configuring the Environment + +> [!NOTE]NOTE +> The following environment configuration is for reference only. Configure the environment based on the site requirements. + +#### Disabling the Firewall and Automatic Startup + +> [!NOTE]NOTE +> It is recommended that firewall be disabled in the test environment to prevent network impact. Configure the firewall based on actual requirements. + +1. Stop the firewall service as the **root** user. + + ```shell + systemctl stop firewalld + ``` + +2. Disable the firewall service as the **root** user. + + ```shell + systemctl disable firewalld + ``` + + > [!NOTE]NOTE + > The automatic startup is automatically disabled as the firewall is disabled. + +#### Disabling SELinux + +1. Modify the configuration file as the **root** user. + + ```shell + sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux + ``` + +#### Creating a User Group and a User + +> [!NOTE]NOTE +> In the server environment, independent users are assigned to each process to implement permission isolation for security purposes. The user group and user are created for the OS, not for the database. + +1. Create a MySQL user or user group as the **root** user. + + ```shell + groupadd mysql + ``` + + ```shell + useradd -g mysql mysql + ``` + +2. Set the user password as the **root** user. + + ```shell + passwd mysql + ``` + + Enter the password twice for confirmation. + +#### Creating Data Drives + +> [!NOTE]NOTE +> +> - If a performance test needs to be performed, an independent drive is required for the data directory. You need to format and mount the drive. For details, see Method 1 or Method 2. +> - In a non-performance test, run the following command as the **root** user to create a data directory. Then skip this section. +> `mkdir /data` + +##### Method 1: Using fdisk for Drive Management as the **root** user + +1. Create a partition, for example, **/dev/sdb**. + + ```shell + fdisk /dev/sdb + ``` + +2. Enter **n** and press **Enter**. +3. Enter **p** and press **Enter**. +4. Enter **1** and press **Enter**. +5. Retain the default settings and press **Enter**. +6. Retain the default settings and press **Enter**. +7. Enter **w** and press **Enter**. +8. Create a file system, for example, **xfs**. + + ```shell + mkfs.xfs /dev/sdb1 + ``` + +9. Mount the partition to **/data** for the OS. + + ```shell + mkdir /data + ``` + + ```shell + mount /dev/sdb1 /data + ``` + +10. Run the **vi /etc/fstab** command and edit the **/etc/fstab** file to enable the data drive to be automatically mounted after the system is restarted. For example, add the content in the last line, as shown in the following figure. + + In the last line, **/dev/nvme0n1p1** is only an example. + + ![](./figures/creat_datadisk.png) + +##### Method 2: Using LVM for Drive Management as the **root** user + +> [!NOTE]NOTE +> Install the LVM2 package in the image as follows: +> Configure the local yum source. For details, see [Configuring the Repository Server](./configuring-the-repo-server.md). If the repository has been configured, skip this step. Then, install LVM2 by running **yum install lvm2**. + +1. Create a PV, for example, **sdb**. + + ```shell + pvcreate /dev/sdb + ``` + +2. Create a physical VG, for example, **datavg**. + + ```shell + vgcreate datavg /dev/sdb + ``` + +3. Create an LV, for example, **datalv** of 600 GB. + + ```shell + lvcreate -L 600G -n datalv datavg + ``` + +4. Create a file system. + + ```shell + mkfs.xfs /dev/datavg/datalv + ``` + +5. Create a data directory and mount it. + + ```shell + mkdir /data + ``` + + ```shell + mount /dev/datavg/datalv /data + ``` + +6. Run the **vi /etc/fstab** command and edit the **/etc/fstab** file to enable the data drive to be automatically mounted after the system is restarted. For example, add the content in the last line, as shown in the following figure. + + In the last line, **/dev/datavg/datalv** is only an example. + + ![](./figures/d1376b2a-d036-41c4-b852-e8368f363b5e-1.png) + +#### Creating a Database Directory and Granting Permissions + +1. In the created data directory **/data**, create directories for processes and grant permissions to the MySQL group or user created as the **root** user. + + ```shell + mkdir -p /data/mysql + cd /data/mysql + mkdir data tmp run log + chown -R mysql:mysql /data + ``` + +### Installing, Running, and Uninstalling MySQL + +#### Installing MySQL + +1. Configure the local yum source. For details, see [Configuring the Repo Server](./configuring-the-repo-server.md). +2. Clear the cache. + + ```shell + dnf clean all + ``` + +3. Create a cache. + + ```shell + dnf makecache + ``` + +4. Install the MySQL server as the **root** user. + + ```shell + dnf install mysql-server + ``` + +5. Check the installed RPM package. + + ```shell + rpm -qa | grep mysql-server + ``` + +#### Running MySQL + +1. Modify the configuration file. + 1. Create the **my.cnf** file as the **root** user and change the file paths \(including the software installation path **basedir** and data path **datadir**\) based on the actual situation. + + ```shell + vi /etc/my.cnf + ``` + + Edit the **my.cnf** file as follows: + + ```shell + [mysqld_safe] + log-error=/data/mysql/log/mysql.log + pid-file=/data/mysql/run/mysqld.pid + [mysqldump] + quick + [mysql] + no-auto-rehash + [client] + default-character-set=utf8 + [mysqld] + basedir=/usr/local/mysql + socket=/data/mysql/run/mysql.sock + tmpdir=/data/mysql/tmp + datadir=/data/mysql/data + default_authentication_plugin=mysql_native_password + port=3306 + user=mysql + ``` + + 2. Ensure that the **my.cnf** file is correctly modified. + + ```shell + cat /etc/my.cnf + ``` + + ![](./figures/en-us_image_0231563132.png) + + > [!WARNING]CAUTION + > In the configuration file, **basedir** specifies the software installation path. Change it based on actual situation. + + 3. Change the group and user of the **/etc/my.cnf** file to **mysql:mysql** as the **root** user. + + ```shell + chown mysql:mysql /etc/my.cnf + ``` + +2. Configure environment variables. + 1. Add the path of the MySQL binary files to the **PATH** parameter as the **root** user. + + ```shell + echo export PATH=$PATH:/usr/local/mysql/bin >> /etc/profile + ``` + + > [!WARNING]CAUTION + > In the command, **/usr/local/mysql/bin** is the absolute path of the **bin** files in the MySQL software installation directory. Change it based on actual situation. + + 2. Run the following command as the **root** user to make the environment variables take effect: + + ```shell + source /etc/profile + ``` + +3. Initialize the database as the **root** user. + + > [!NOTE]NOTE + > The second line from the bottom contains the initial password, which will be used when you log in to the database. + + ```shell + $ mysqld --defaults-file=/etc/my.cnf --initialize + 2020-03-18T03:27:13.702385Z 0 [System] [MY-013169] [Server] /usr/local/mysql/bin/mysqld (mysqld 8.0.17) initializing of server in progress as process 34014 + 2020-03-18T03:27:24.112453Z 5 [Note] [MY-010454] [Server] A temporary password is generated for root@localhost: iNat=)#V2tZu + 2020-03-18T03:27:28.576003Z 0 [System] [MY-013170] [Server] /usr/local/mysql/bin/mysqld (mysqld 8.0.17) initializing of server has completed + ``` + + If the command output contains "initializing of server has completed", the database has been initialized. In the command output, "iNat=\)\# V2tZu" in "A temporary password is generated for root@localhost: iNat=\)\# V2tZu" is the initial password. + +4. Start the database. + + > [!WARNING]CAUTION + > Start MySQL as user **mysql** if it is the first time to start the database service. If you start MySQL as user **root**, a message will be displayed indicating that the **mysql.log** file is missing. If you start MySQL as user **mysql**, the **mysql.log** file will be generated in the **/data/mysql/log** directory. No error will be displayed if you start the database as user **root** again. + + 1. Modify the file permission as the **root** user. + + ```shell + chmod 777 /usr/local/mysql/support-files/mysql.server + chown mysql:mysql /var/log/mysql/* + ``` + + 2. Start MySQL as the **root** user. + + ```shell + cp /usr/local/mysql/support-files/mysql.server /etc/init.d/mysql + chkconfig mysql on + ``` + + Start MySQL as user **mysql**. + + ```shell + su - mysql + service mysql start + ``` + +5. Log in to the database. + + > [!NOTE]NOTE + > + > - Enter the initial password generated during database initialization \([3](#li15634560582)\). + > - If MySQL is installed by using an RPM package obtained from the official website, the **mysqld** file is located in the **/usr/sbin** directory. Ensure that the directory specified in the command is correct. + + ```shell + /usr/local/mysql/bin/mysql -uroot -p -S /data/mysql/run/mysql.sock + ``` + + ![](./figures/en-us_image_0231563134.png) + +6. Configure the database accounts and passwords. + 1. After logging in to the database, change the password of user **root** for logging in to the database. + + ```shell + mysql>alter user 'root'@'localhost' identified by "123456"; + ``` + + 2. Create a user **root** for all the other hosts in the domain. + + ```shell + mysql>create user 'root'@'%' identified by '123456'; + ``` + + 3. Grant permissions to the user **root**. + + ```shell + mysql>grant all privileges on *.* to 'root'@'%'; + mysql>flush privileges; + ``` + + ![](./figures/en-us_image_0231563135.png) + +7. Exit the database. + + Run the **\\q** or **exit** command to exit the database. + + ```shell + mysql>exit + ``` + + ![](./figures/en-us_image_0231563136.png) + +#### Uninstalling MySQL + +1. Stop the database process as the **root** user. + + ```shell + $ ps -ef | grep mysql + # kill -9 PID + ``` + +2. Run the **dnf remove mysql** command as the **root** user to uninstall MySQL. + + ```shell + dnf remove mysql + ``` + +### Managing Database Users + +#### Creating Users + +Run the **CREATE USER** statement to create one or more users and set corresponding passwords. + +```pgsql +CREATE USER 'username'@'hostname' IDENTIFIED BY 'password'; +``` + +In the preceding information: + +- _username_: name of a user. +- _host_: hostname, that is, the name of the host where the user connects to the database. As a local user, you can set the parameter to **localhost**. If the host name is not specified during user creation, the host name is **%** by default, indicating a group of hosts. +- _password_: password for logging in to the server. The password can be null. If the password is null, the user can log in to the server without entering the password. This method, however, is not recommended because it provides low security. + +To use the **CREATE USER** statement, you must have the **INSERT** permission on the database or the global **CREATE USER** permission. + +After a user account is created using the **CREATE USER** statement, a record is added to the user table in the database. If the account to be created exists, an error will occur during statement execution. + +A new user has few permissions and can perform only operations that do not require permissions. For example, a user can run the **SHOW** statement to query the list of all storage engines and character sets. + +##### Example + + Create a local user whose password is **123456** and username is **userexample1**. + +```pgsql +> CREATE USER 'userexample1'@'localhost' IDENTIFIED BY '123456'; +``` + + Create a user whose password is **123456**, username is **userexample2**, and hostname is **192.168.1.100**. + +```pgsql +> CREATE USER 'userexample2'@'192.168.1.100' IDENTIFIED BY '123456'; +``` + +#### Viewing Users + +Run the **SHOW GRANTS** or **SELECT** statement to view one or more users. + +View a specific user: + +```pgsql +SHOW GRANTS [FOR 'username'@'hostname']; +``` + +```pgsql +SELECT USER,HOST,PASSWORD FROM mysql.user WHERE USER='username'; +``` + +View all users: + +```pgsql +SELECT USER,HOST FROM mysql.user; +``` + +In the preceding information: + +- _username_: name of a user. +- _hostname_: host name. + +##### Example + + View the user **userexample1**. + +```pgsql +> SHOW GRANTS FOR 'userexample1'@'localhost'; +``` + + View all users in the MySQL database. + +```pgsql +> SELECT USER,HOST FROM mysql.user; +``` + +#### Modifying Users + +##### Modifying a Username + +Run the **RENAME USER** statement to change one or more existing usernames. + +```pgsql +RENAME USER 'oldusername'@'hostname' TO 'newusername'@'hostname'; +``` + +In the preceding information: + +- _oldusername_: original username. +- _newusername_: new username. +- _hostname_: host name. + +The **RENAME USER** statement is used to rename an existing account. If the original account does not exist in the system or the new account exists, an error will occur when the statement is executed. + +To use the **RENAME USER** statement, you must have the **UPDATE** permission on the database or the global **CREATE USER** permission. + +##### Example of Modifying a User + + Change the username **userexample1** to **userexample2** and change the hostname to **localhost**. + +```pgsql +> RENAME USER 'userexample1'@'localhost' TO 'userexample2'@'localhost'; +``` + +##### Modifying a User Password + +Use the **SET PASSWORD** statement to modify the login password of a user. + +```pgsql +SET PASSWORD FOR 'username'@'hostname' = 'newpassword'; +``` + +In the preceding information: + +- **FOR'**_username_**'@'**_hostname_**'**: specifies the username and hostname whose password is to be changed. This parameter is optional. +- _newpassword_: new password. + +If the **FOR** clause is not added to the **SET PASSWORD** statement, the password of the current user is changed. + +The **FOR** clause must be given in the format of **'**_username_**'@'**_hostname_**'**, where _username_ indicates the username of the account and _hostname_ indicates the hostname of the account. + +The account whose password is to be changed must exist in the system. Otherwise, an error occurs when the statement is executed. + +##### Example of Changing a User Password + + Change the password of user **userexample** whose hostname is **localhost** to **0123456**. + +```pgsql +> SET PASSWORD FOR 'userexample'@'localhost' = '0123456'; +``` + +#### Deleting Users + +Use the **DROP USER** statement to delete one or more user accounts and related permissions. + +```pgsql +DROP USER 'username1'@'hostname1' [,'username2'@'hostname2']...; +``` + +> [!WARNING]CAUTION +> The deletion of users does not affect the tables, indexes, or other database objects that they have created, because the database does not record the accounts that have created these objects. + +The **DROP USER** statement can be used to delete one or more database accounts and their original permissions. + +To use the **DROP USER** statement, you must have the **DELETE** permission on the database or the global **CREATE USER** permission. + +In the **DROP USER** statement, if the hostname of an account is not specified, the hostname is **%** by default. + +##### Example + + Delete the local user **userexample**. + +```pgsql +> DROP USER 'userexample'@'localhost'; +``` + +#### Granting Permissions to a User + +Run the **GRANT** statement to grant permissions to a new user. + +```pgsql +GRANT privileges ON databasename.tablename TO 'username'@'hostname'; +``` + +In the preceding information: + +- **ON** clause: specifies the object and level on which the permission is granted. +- _privileges_: indicates the operation permissions of a user, such as **SELECT**, INSERT, and **UPDATE**. To grant all permissions to a user, use **ALL**. +- _databasename_: database name. +- _tablename_: table name. +- **TO** clause: sets the user password and specifies the user to whom the permission is granted. +- _username_: name of a user. +- _hostname_: host name. + +To grant the user the permission to operate all databases and tables, use asterisks \(\*\), for example, **\*.\***. + +If you specify a password for an existing user in the **TO** clause, the new password will overwrite the original password. + +If the permission is granted to a non-existent user, a **CREATE USER** statement is automatically executed to create the user, but the password must be specified for the user. + +##### Example + + Grant the **SELECT** and **INSERT** permissions to local user **userexample**. + +```pgsql +> GRANT SELECT,INSERT ON *.* TO 'userexample'@'localhost'; +``` + +#### Deleting User Permissions + +Run the **REVOKE** statement to delete the permissions of a user, but the user will not be deleted. + +```pgsql +REVOKE privilege ON databasename.tablename FROM 'username'@'hostname'; +``` + +The parameters in the **REVOKE** statement are the same as those in the **GRANT** statement. + +To use the **REVOKE** statement, you must have the global **CREATE USER** or **UPDATE** permission for the database. + +##### Example + + Delete the **INSERT** permission of local user **userexample**. + +```pgsql +> REVOKE INSERT ON *.* FROM 'userexample'@'localhost'; +``` + +### Managing Databases + +#### Creating a Database + +Run the **CREATE DATABASE** statement to create a database. + +```pgsql +CREATE DATABASE databasename; +``` + +In the preceding command, _databasename_ can be replaced with the database name, which is case insensitive. + +##### Example + + Create a database named **databaseexample**. + +```pgsql +> CREATE DATABASE databaseexample; +``` + +#### Viewing a Database + +Run the **SHOW DATABASES** statement to view a database. + +```pgsql +SHOW DATABASES; +``` + +##### Example + + View all databases. + +```pgsql +> SHOW DATABASES; +``` + +#### Selecting a Database + +Generally, you need to select a target database before creating or querying a table. Use the **USE** statement to select a database. + +```pgsql +USE databasename; +``` + +In the preceding command, _databasename_ indicates the database name. + +##### Example + + Select the **databaseexample** database. + +```pgsql +> USE databaseexample; +``` + +#### Deleting a Database + +Run the **DROP DATABASE** statement to delete a database. + +> [!WARNING]CAUTION +> Exercise caution when deleting a database. Once a database is deleted, all tables and data in the database will be deleted. + +```pgsql +DROP DATABASE databasename; +``` + +In the preceding command, _databasename_ indicates the database name. + +The **DROP DATABASE** command is used to delete an existing database. After this command is executed, all tables in the database are deleted, but the user permissions of the database are not automatically deleted. + +To use **DROP DATABASE**, you need the **DROP** permission on the database. + +**DROP SCHEMA** is a synonym of **DROP DATABASE**. + +##### Example + + Delete the **databaseexample** database. + +```pgsql +> DROP DATABASE databaseexample; +``` + +#### Backing Up a Database + +Run the **mysqldump** command as the **root** user to back up the database. + +Back up one or more tables: + +```shell +mysqldump [options] databasename [tablename ...] > outfile +``` + +Back up one or more databases: + +```shell +mysqldump [options] -databases databasename ... > outfile +``` + +Back up all databases: + +```shell +mysqldump [options] -all-databases > outputfile +``` + +In the preceding information: + +- _databasename_: database name. +- _tablename_: name of a data table. +- _outfile_: database backup file. +- _options_: parameter option of the **mysqldump** command. Multiple parameters can be separated by spaces. The common parameters of the **mysqldump** command are as follows: + - **-u, \-\-user**= _username_: specifies the username. + - **-p, \-\-password**\[= _password_\]: specifies the password. + - **-P, \-\-port**= _portnumber_: specifies the port number. + - **-h, \-\-host**= _hostname_: specifies the hostname. + - **-r, \-\-result-file**= _filename_: saves the export result to a specified file, which is equivalent to **\>**. + - **-t**: backs up data only. + - **-d**: backs up the table structure only. + +##### Example + + Back up all the databases of user **root** on port **3306** of the host whose IP address is **192.168.202.144** to the **alldb.sql** file. + +```shell +mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 --all-databases > alldb.sql +``` + + Back up the db1 database of user **root** on port **3306** of the host whose IP address is **192.168.202.144** to the **db1.sql** file. + +```shell +mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 --databases db1 > db1.sql +``` + + Back up the tb1 table of the db1 database of user **root** on port **3306** of the host whose IP address is **192.168.202.144** to the **db1tb1.sql** file. + +```shell +mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 db1 tb1 > db1tb1.sql +``` + + Back up only the table structure of the db1 database of user **root** on port **3306** of the host whose IP address is **192.168.202.144** to the **db1.sql** file. + +```shell +mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 -d db1 > db1.sql +``` + + Back up only the table structure of the db1 database of user **root** on port **3306** of the host whose IP address is **192.168.202.144** to the **db1.sql** file. + +```shell +mysqldump -h 192.168.202.144 -P 3306 -uroot -p123456 -t db1 > db1.sql +``` + +#### Restoring a Database + +Run the **mysql** command as the **root** user to restore the database. + +Restore one or more tables: + +```shell +mysql -h hostname -P portnumber -u username -ppassword databasename < infile +``` + +In the preceding information: + +- _hostname_: host name. +- _portnumber_: port number. +- _username_: name of a user. +- _password_: password. +- _databasename_: database name. +- _infile_: **outfile** parameter in the **mysqldump** command. + +##### Example + + Restore a database. + +```shell +mysql -h 192.168.202.144 -P 3306 -uroot -p123456 -t db1 < db1.sql +``` diff --git a/docs/en/server/administration/administrator/user-and-user-group-management.md b/docs/en/server/administration/administrator/user-and-user-group-management.md new file mode 100644 index 0000000000000000000000000000000000000000..ec7864626b3192a8f4fc098691c165af339b9882 --- /dev/null +++ b/docs/en/server/administration/administrator/user-and-user-group-management.md @@ -0,0 +1,340 @@ +# User and User Group Management + +In Linux, each common user has an account, including the user name, password, and home directory. There are also special users created for specific purposes, and the most important special user is the admin account whose default user name is root. In addition, Linux provides user groups so that each user belongs to at least one group, facilitating permission management. + +The control of users and user groups is a core element of openEuler security management. This topic introduces the user and group management commands and explains how to assign privileges to common users in graphical user interface and on command lines. + + +- [User and User Group Management](#user-and-user-group-management) + - [Managing Users](#managing-users) + - [Adding a User](#adding-a-user) + - [Modifying a User Account](#modifying-a-user-account) + - [Deleting a User](#deleting-a-user) + - [Granting Rights to a Common User](#granting-rights-to-a-common-user) + - [Managing User Groups](#managing-user-groups) + - [Adding a User Group](#adding-a-user-group) + - [Modifying a User Group](#modifying-a-user-group) + - [Deleting a User Group](#deleting-a-user-group) + - [Adding a User to a Group or Removing a User from a Group](#adding-a-user-to-a-group-or-removing-a-user-from-a-group) + - [Changing the Current Group of a User to a Specified Group](#changing-the-current-group-of-a-user-to-a-specified-group) + + + +## Managing Users + +### Adding a User + +#### useradd Command + +Run the **useradd** command as the user **root** to add user information to the system. In the command, _options_ indicates related parameters and _username_ indicates the user name. + +```shell +useradd [options] username +``` + +#### User Information Files + +The following files contain user account information: + +- /etc/passwd: user account information +- /etc/shadow file: user account encryption information +- /etc/group file: group information +- /etc/default/useradd: default configurations +- /etc/login.defs: system wide settings +- /etc/skel: default directory that holds initial configuration files + +#### Example + +For example, to create a user named userexample, run the following command as the user **root**: + +```shell +useradd userexample +``` + +> [!NOTE]NOTE +> If no prompt is displayed, the user is successfully created. After the user is created, run the **passwd** command to assign a password to the user. A new account without a password will be banned. + +To view information about the new user, run the **id** command: + +```shell +$ id userexample +uid=502(userexample) gid=502(userexample) groups=502(userexample) +``` + +To change the password of the userexample, run the following command: + +```shell +passwd userexample +``` + +It is recommended that the new user password meet the complexity requirements. The password complexity requirements are as follows: + +1. A password must contain at least eight characters. +2. A password must contain at least three of the following types: uppercase letters, lowercase letters, digits, and special characters. +3. A password must be different from the account name. +4. A password cannot contain words in the dictionary. + - Querying a dictionary + In the installed openEuler environment, you can run the following command to export the dictionary library file **dictionary.txt**, and then check whether the password is in the dictionary. + + ```shell + cracklib-unpacker /usr/share/cracklib/pw_dict > dictionary.txt + ``` + + - Modifying a dictionary + 1. Modify the exported dictionary library file, and then run the following command to update the dictionary library: + + ```shell + create-cracklib-dict dictionary.txt + ``` + + 2. Run the following command to add another dictionary file **custom.txt** to the original dictionary library. + + ```shell + create-cracklib-dict dictionary.txt custom.txt + ``` + +Then, enter the password and confirm it as prompted: + +```shell +$ passwd userexample +Changing password for user userexample. +New password: +Retype new password: +passwd: all authentication tokens updated successfully. +``` + +> [!NOTE]NOTE +> If the command output contains **BAD PASSWORD: The password fails the dictionary check - it is too simplistic/systematic**, the password is too simple and needs to be reset. + +### Modifying a User Account + +#### Changing a Password + +Common users can change their passwords using the **passwd** command. Only the admin is allowed to use the **passwd username** command to change passwords for other users. + +#### Changing User's Login Shell + +Common users can use the **chsh** command to change their login shell. Only the admin is allowed to run the **chsh username** command to change login shell for other users. + +Users can also run the **usermod** command as the user **root** to modify the shell information. In the command, _new_shell_path_ indicates the target shell path, and _username_ indicates the user name to be modified. Change them based on the site requirements. + +```shell +usermod -s new_shell_path username +``` + +For example, to change the shell of userexample to csh, run the following command: + +```shell +usermod -s /bin/csh userexample +``` + +#### Changing the Home Directory + +- To change the home directory, run the following command as the user **root**. In the command, _new\_home\_directory_ indicates the created target home directory, and _username_ indicates the user name to be changed. Change them based on the site requirements. + + ```shell + usermod -d new_home_directory username + ``` + +- To move the content in the current home directory to a new one, run the usermod command with the -m option: + + ```shell + usermod -d new_home_directory -m username + ``` + +#### Changing a UID + +To change the user ID, run the following command as the user **root**. In the command, _UID_ indicates the target user ID and _username_ indicates the user name. Change them based on the site requirements. + +```shell +usermod -u UID username +``` + +The **usermod** command automatically changes the UID of the owner of files and directories under the user's home directory. However, for files outside the user's home directory, their owners can only be changed using the **chown** command. + +#### Changing Account Expiry Date + +If the shadow password is used, run the following command as the user **root** to change the validity period of an account. In the command, _MM_, _DD_, and _YY_ indicate the month, day, and year, respectively, and _username_ indicates the user name. Change them based on the site requirements. + +```shell +usermod -e MM/DD/YY username +``` + +### Deleting a User + +Run the **userdel** command as the user **root** to delete an existing user. + +For example, run the following command to delete user Test: + +```shell +userdel Test +``` + +If you also need to delete the user's home directory and all contents in the directory, run the **userdel** command with the -r option to delete them recursively. + +> [!NOTE]NOTE +> You are not advised to directly delete a user who has logged in to the system. To forcibly delete a user, run the **userdel -f** _Test_ command. + +### Granting Rights to a Common User + +The **sudo** command allows common users to execute commands that can be executed only by administrator accounts. + +The **sudo** command allows the user specified in the **/etc/sudoers** file to execute the administrator account commands. For example, an authorized common user can run: + +```shell +sudo /usr/sbin/useradd newuserl +``` + +The **sudo** command can specify a common user that has been added to the **/etc/sudoers** file to process tasks as required. + +The information configured in the **/etc/sudoers** file is as follows: + +- Blank lines or comment lines starting with **\#**: Have no specific functions. +- Optional host alias lines: Create the name of a host list. The lines must start with **Host\_Alias**. The host names in the list must be separated by commas \(,\). For example: + + ```shell + Host_Alias linux=ted1,ted2 + ``` + + **ted1** and **ted2** are two host names, which can be called **linux**. + +- Optional user alias lines: Create the name of a user list. The lines must start with **User\_Alias**. The user names in the list must be separated by commas \(,\). The user alias lines have the same format as the host alias lines. +- Optional command alias lines: Create the name of a command list. The lines must start with **Cmnd\_Alias**. The commands in the list must be separated by commas \(,\). +- Optional running mode alias lines: Create the name of a user list. The difference is that such alias can enable a user in the list to run the **sudo** command. +- Necessary declaration lines for user access: + + The declaration syntax for user access is as follows: + + ```shell + user host = [ run as user ] command list + ``` + + Set the user to a real user name or a defined user alias, and set the host to a real host name or a defined host alias. By default, all the commands executed by sudo are executed as user **root**. If you want to use another account, you can specify it. **command list** is either a command list separated by commas \(,\) or a defined command alias. For example: + + ```shell + ted1 ted2=/sbin/shutdown + ``` + + In this example, ted1 can run the shutdown command on ted2. + + ```shell + newuser1 ted1=(root) /usr/sbin/useradd,/usr/sbin/userdel + ``` + + This indicates that newuser1 on the ted1 host can run the **useradd** and **userdel** commands as the user **root**. + + > [!NOTE]NOTE + > - You can define multiple aliases in a line and separate them with colons \(:\). + > - You can add an exclamation mark \(!\) before a command or a command alias to make the command or the command alias invalid. + > - There are two keywords: ALL and NOPASSWD. ALL indicates all files, hosts, or commands, and NOPASSWD indicates that no password is required. + > - By modifying user access, you can change the access permission of a common user to be the same as that of the user **root**. Then, you can grant rights to the common user. + +The following is an example of the **sudoers** file: + +```text +#sudoers files +#User alias specification +User_Alias ADMIN=ted1:POWERUSER=globus,ted2 +#user privilege specification +ADMIN ALL=ALL +POWERUSER ALL=ALL,!/bin/su +``` + +In the preceding information: + +- User\_Alias ADMIN=ted1:POWERUSER=globus,ted2 + + Two aliases ADMIN and POWERUSER are defined. + +- ADMIN ALL=ALL + + ADMIN can run all commands as the user **root** on all hosts. + +- POWERUSER ALL=ALL,!/bin/su + + POWERUSER can run all commands except the **su** command as the user **root** on all hosts. + +## Managing User Groups + +### Adding a User Group + +#### groupadd Command + +Run the **groupadd** command as the **root** user to add user group information to the system. In the command, _options_ indicates related parameters and _groupname_ indicates the group name. + +```shell +groupadd [options] groupname +``` + +#### User Group Information Files + +The following files contain user group information: + +- /etc/gshadow file: user group encryption information +- /etc/group file: group information +- /etc/login.defs: system wide settings + +#### Example + +For example, to create a user group named groupexample, run the following command as the **root** user: + +```shell +groupadd groupexample +``` + +### Modifying a User Group + +#### Changing a GID + +To change the user group ID, run the following command as the **root** user. In the command, _GID_ indicates the target user group ID and _groupname_ indicates the user group name. Change them based on the site requirements. + +```shell +groupmod -g GID groupname +``` + +#### Changing a User Group Name + +To change the user group name, run the following command as the **root** user. In the command, _newgroupname_ indicates the user group new name and _oldgroupname_ indicates the user group name. Change them based on the site requirements. + +```shell +groupmod -n newgroupname oldgroupname +``` + +### Deleting a User Group + +Run the **groupdel** command as the **root** user to delete an existing user group. + +For example, run the following command to delete user group Test: + +```shell +groupdel Test +``` + +> [!NOTE]NOTE +> The user's primary group cannot be directly deleted. To forcibly delete a user's primary group, run the **groupdel -f** _Test_ command. + +### Adding a User to a Group or Removing a User from a Group + +Run the **gpasswd** command as the **root** user to add a user to a group or remove a user from a group. + +For example, run the following command to add the user userexample to the user group Test: + +```shell +gpasswd -a userexample Test +``` + +For example, run the following command to remove the user userexample from the user group Test: + +```shell +gpasswd -d userexample Test +``` + +### Changing the Current Group of a User to a Specified Group + +If a user belongs to multiple user groups, run the **newgrp** command to switch the user to another user group after logging in to the system. Then, the user has the permission of other user groups. + +For example, run the following command to change the current group of the user userexample to the user group Test: + +```shell +newgrp Test +``` diff --git a/docs/en/server/administration/administrator/using-dnf-to-manage-software-packages.md b/docs/en/server/administration/administrator/using-dnf-to-manage-software-packages.md new file mode 100644 index 0000000000000000000000000000000000000000..8898e6600a22545b087e704d4cd4b491cc139b88 --- /dev/null +++ b/docs/en/server/administration/administrator/using-dnf-to-manage-software-packages.md @@ -0,0 +1,514 @@ +# Using DNF to Manage Software Packages + +DNF is a Linux software package management tool used to manage RPM software packages. The DNF can query software package information, obtain software packages from a specified software library, automatically process dependencies to install or uninstall software packages, and update the system to the latest available version. + +> [!NOTE]NOTE +> +> - DNF is fully compatible with YUM and provides YUM-compatible command lines and APIs for extensions and plug-ins. +> - You must have the administrator rights to use the DNF. All commands in this chapter must be executed by the administrator. + +## Configuring the DNF + +### The DNF Configuration File + +The main configuration file of the DNF is **/etc/dnf/dnf.conf** which consists of two parts: + +- The **main** part in the file stores the global settings of the DNF. + +- The **repository** part in the file stores the settings of the software source. You can add zero or more **repository** sections to the file. + +In addition, the **/etc/yum.repos.d** directory stores zero or more repo source files, which define different repositories. + +You can configure a software source by either directly configuring the **/etc/dnf/dnf.conf** file or configuring the .repo file in the **/etc/yum.repos.d** directory. + +#### Configuring the main Part + +The **/etc/dnf/dnf.conf** file contains the **main** part. The following is an example of the configuration file: + +```text +[main] +gpgcheck=1 +installonly_limit=3 +clean_requirements_on_remove=True +best=True +``` + +Common options are as follows: + +**Table 1** main parameter description + + + +| Parameter | Description | +| ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| cachedir | Cache directory for storing RPM packages and database files. | +| keepcache | The options are 1 and 0, indicating whether to cache the RPM packages and header files that have been successfully installed. The default value is 0, indicating that the RPM packages and header files are not cached. | +| debuglevel | Sets debugging information generated by the DNF. The value ranges from 0 to 10. A larger value indicates more detailed debugging information. The default value is 2. The value 0 indicates that the debug information is not displayed. | +| clean\_requirements\_on\_remove | Deletes the dependency items that are no longer used during DNF removal. If the software package is installed through the DNF instead of the explicit user request, the software package can be deleted only through clean\_requirements\_on\_remove, that is, the software package is introduced as a dependency item. The default value is **True**. | +| best | The system always attempts to install the latest version of the upgrade package. If the latest version cannot be installed, the system displays the cause and stops the installation. The default value is **True**. | +| obsoletes | The options are **1** and **0**, indicating whether to allow the update of outdated RPM packages. The default value is 1, indicating that the update is allowed. | +| gpgcheck | The options are **1** and **0**, indicating whether to perform GPG verification. The default value is **1**, indicating that verification is required. | +| plugins | The options are **1** and **0**, indicating that the DNF plug-in is enabled or disabled. The default value is **1**, indicating that the DNF plug-in is enabled. | +| installonly\_limit | Sets the number of packages that can be installed at the same time by running the **installonlypkgs** command. The default value is 3. You are advised not to decrease the value. | + +#### Configuring the repository Part + +The repository part allows you to customize openEuler software source repositories. The name of each repository must be unique. Otherwise, conflicts may occur. You can configure a software source by either directly configuring the **/etc/dnf/dnf.conf** file or configuring the .repo file in the **/etc/yum.repos.d** directory. + +- Configuring the **/etc/dnf/dnf.conf** file + + The following is a minimum configuration example of the \[repository\] section: + + ```text + [repository] + name=repository_name + baseurl=repository_url + ``` + + > [!NOTE]NOTE + > openEuler provides an online image source at [https://repo.openeuler.org/](https://repo.openeuler.org/). For example, if the openEuler 23.09 version is aarch64, the **baseurl** can be set to . + + Common options are as follows: + + **Table 2** repository parameter description + + +| Parameter | Description | +|-----------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| name=repository_name | Name string of a software repository. | +| baseurl=repository_url | Address of the software repository.
- Network location using the HTTP protocol, for example, `http://path/to/repo`
- Network location using the FTP protocol, for example, `ftp://path/to/repo`
- Local path: for example, `file:///path/to/local/repo` | + +- Configuring the .repo file in the **/etc/yum.repos.d** directory + openEuler provides multiple repo sources for users online. For details about the repo sources, see [OS Installation](../../releasenotes/releasenotes/os-installation.md). + + For example, run the following command as the **root** user to add the openeuler repo source to the openEuler.repo file. + + ```shell + vi /etc/yum.repos.d/openEuler.repo + ``` + + ```text + [OS] + name=openEuler-$releasever - OS + baseurl=https://repo.openeuler.org/openEuler-23.09/OS/$basearch/ + enabled=1 + gpgcheck=1 + gpgkey=https://repo.openeuler.org/openEuler-23.09/OS/$basearch/RPM-GPG-KEY-openEuler + ``` + + > [!NOTE]NOTE + > + > - **enabled** indicates whether to enable the software source repository. The value can be **1** or **0**. The default value is **1**, indicating that the software source repository is enabled. + > - **gpgkey** is the public key used to verify the signature. + +#### Displays the Current Configuration + +- To display the current configuration information, run the following command: + + ```shell + dnf config-manager --dump + ``` + +- To display the configuration of a software source, query the repo id: + + ```shell + dnf repolist + ``` + + Run the following command to display the software source configuration of the corresponding ID. In the command, _repository_ indicates the repository ID. + + ```shell + dnf config-manager --dump repository + ``` + +- You can also use a global regular expression to display all matching configurations. + + ```shell + dnf config-manager --dump glob_expression + ``` + +### Creating a Local Software Repository + +To create a local repository of software sources, perform the following steps. + +1. Install the createrepo software package. Run the following command as the root user: + + ```shell + dnf install createrepo + ``` + +2. Copy the required software packages to a directory, for example, **/mnt/local\_repo/**. +3. Run the following command to create a software source: + + ```shell + createrepo /mnt/local_repo + ``` + +### Adding, Enabling, and Disabling Software Sources + +This section describes how to add, enable, and disable the software source repository by running the **dnf config-manager** command. + +#### Adding Software Source + +To define a new software repository, you can add the repository part to the **/etc/dnf/dnf.conf** file or add the .repo file to the **/etc/yum.repos.d/** directory. You are advised to add the .repo file. Each software source has its own .repo file. The following describes how to add the .repo file. + +To add such a source to your system, run the following command as the user **root**. After the command is executed, the corresponding .repo file is generated in the **/etc/yum.repos.d/** directory. In the command, _repository\_url_ indicates the repo source address. For details, see [Table 2](#en-us_topic_0151921080_t7c83ace02ab94e9986c0684f417e3436). + +```shell +dnf config-manager --add-repo repository_url +``` + +#### Enabling a Software Repository + +To enable the software source, run the following command as the user **root**. In the command, _repository_ indicates the repository ID in the new .repo file. You can run the **dnf repolist** command to query the repository ID. + +```shell +dnf config-manager --set-enable repository +``` + +You can also use a global regular expression to enable all matching software sources. In the command, _glob\_expression_ indicates the regular expression used to match multiple repository IDs. + +```shell +dnf config-manager --set-enable glob_expression +``` + +#### Disabling a Software Repository + +To disable a software source, run the following command as the user **root**: + +```shell +dnf config-manager --set-disable repository +``` + +You can also use a global regular expression to disable all matching software sources. + +```shell +dnf config-manager --set-disable glob_expression +``` + +## Managing Software Package + +The DNF enables you to query, install, and delete software packages. + +### Searching for Software Packages + +You can search for the required RPM package by its name, abbreviation, or description. The command is as follows: + +```shell +dnf search term +``` + +The following is an example: + +```shell +$ dnf search httpd +========================================== N/S matched: httpd ========================================== +httpd.aarch64 : Apache HTTP Server +httpd-devel.aarch64 : Development interfaces for the Apache HTTP server +httpd-manual.noarch : Documentation for the Apache HTTP server +httpd-tools.aarch64 : Tools for use with the Apache HTTP Server +libmicrohttpd.aarch64 : Lightweight library for embedding a webserver in applications +mod_auth_mellon.aarch64 : A SAML 2.0 authentication module for the Apache Httpd Server +mod_dav_svn.aarch64 : Apache httpd module for Subversion server +``` + +### Listing Software Packages + +To list all installed and available RPM packages in the system, run the following command: + +```shell +dnf list all +``` + +To list a specific RPM package in the system, run the following command: + +```shell +dnf list glob_expression... +``` + +The following is an example: + +```shell +$ dnf list httpd +Available Packages +httpd.aarch64 2.4.34-8.h5.oe1 Local +``` + +### Displaying RPM Package Information + +To view information about one or more RPM packages, run the following command: + +```shell +dnf info package_name... +``` + +The following is a command example: + +```shell +$ dnf info httpd +Available Packages +Name : httpd +Version : 2.4.34 +Release : 8.h5.oe1 +Arch : aarch64 +Size : 1.2 M +Repo : Local +Summary : Apache HTTP Server +URL : http://httpd.apache.org/ +License : ASL 2.0 +Description : The Apache HTTP Server is a powerful, efficient, and extensible + : web server. +``` + +### Installing an RPM Package + +To install a software package and all its dependencies that have not been installed, run the following command as the user **root**: + +```shell +dnf install package_name +``` + +You can also add software package names to install multiple software packages at the same time. Add the **strict=False** parameter to the **/etc/dnf/dnf.conf** configuration file and run the **dnf** command with `--setopt=strict=0`. Run the following command as the user **root**: + +```shell +dnf install package_name package_name... --setopt=strict=0 +``` + +The following is an example: + +```shell +dnf install httpd +``` + +> [!NOTE]NOTE +> If the RPM package fails to be installed, see [Installation Failure Caused by Software Package Conflict, File Conflict, or Missing Software Package](./common-issues-and-solutions.md#issue-5-installation-failure-caused-by-software-package-conflict-file-conflict-or-missing-software-package). + +### Downloading Software Packages + +To download the software package using the DNF, run the following command as the user **root**: + +```shell +dnf download package_name +``` + +If you need to download the dependency packages that are not installed, add **\-\-resolve**. The command is as follows: + +```shell +dnf download --resolve package_name +``` + +The following is an example: + +```shell +dnf download --resolve httpd +``` + +### Deleting a Software Package + +To uninstall the software package and related dependent software packages, run the following command as the user **root**: + +```shell +dnf remove package_name... +``` + +The following is an example: + +```shell +dnf remove totem +``` + +## Managing Software Package Groups + +A software package set is a group of software packages that serve a common purpose, for example, a system tool set. You can use the DNF to install or delete software package groups, improving operation efficiency. + +### Listing Software Package Groups + +The summary parameter can be used to list the number of all installed software package groups, available groups, and available environment groups in the system. The command is as follows: + +```shell +dnf groups summary +``` + +The following is an example: + +```shell +$ dnf groups summary +Last metadata expiration check: 0:11:56 ago on Sat 17 Aug 2019 07:45:14 PM CST. +Available Groups: 8 +``` + +To list all software package groups and their group IDs, run the following command: + +```shell +dnf group list +``` + +The following is an example: + +```shell +$ dnf group list +Last metadata expiration check: 0:10:32 ago on Sat 17 Aug 2019 07:45:14 PM CST. +Available Environment Groups: + Minimal Install + Custom Operating System + Server +Available Groups: + Development Tools + Graphical Administration Tools + Headless Management + Legacy UNIX Compatibility + Network Servers + Scientific Support + Security Tools + System Tools +``` + +### Displaying the Software Package Group Information + +To list the mandatory and optional packages contained in a software package group, run the following command: + +```shell +dnf group info glob_expression... +``` + +The following is an example of displaying the Development Tools information: + +```shell +$ dnf group info "Development Tools" +Last metadata expiration check: 0:14:54 ago on Wed 05 Jun 2019 08:38:02 PM CST. + +Group: Development Tools + Description: A basic development environment. + Mandatory Packages: + binutils + glibc-devel + make + pkgconf + pkgconf-m4 + pkgconf-pkg-config + rpm-sign + Optional Packages: + expect +``` + +### Installation Software Package Group + +Each software package group has its own name and corresponding group ID. You can use the software package group name or its ID to install the software package. + +To install a software package group, run the following command as the user **root**: + +```shell +dnf group install group_name +``` + +```shell +dnf group install groupid +``` + +For example, to install the software package group of Development Tools, run the following command: + +```shell +dnf group install "Development Tools" +``` + +```shell +dnf group install development +``` + +### Deleting a Software Package Group + +To uninstall a software package group, you can use the group name or ID to run the following command as the user **root**: + +```shell +dnf group remove group_name +``` + +```shell +dnf group remove groupid +``` + +For example, to delete the software package group of Development Tools, run the following command: + +```shell +dnf group remove "Development Tools" +``` + +```shell +dnf group remove development +``` + +## Check and Update + +You can use the DNF to check whether any software package in your system needs to be updated. You can use the DNF to list the software packages to be updated. You can choose to update all packages at a time or update only specified packages. + +### Checking for Update + +To list all currently available updates, run the following command: + +```shell +dnf check-update +``` + +The following is an example: + +```shell +$ dnf check-update +Last metadata expiration check: 0:02:10 ago on Sun 01 Sep 2019 11:28:07 PM CST. + +anaconda-core.aarch64 19.31.123-1.14 updates +anaconda-gui.aarch64 19.31.123-1.14 updates +anaconda-tui.aarch64 19.31.123-1.14 updates +anaconda-user-help.aarch64 19.31.123-1.14 updates +anaconda-widgets.aarch64 19.31.123-1.14 updates +bind-libs.aarch64 32:9.9.4-29.3 updates +bind-libs-lite.aarch64 32:9.9.4-29.3 updates +bind-license.noarch 32:9.9.4-29.3 updates +bind-utils.aarch64 32:9.9.4-29.3 updates +... +``` + +### Upgrade + +To upgrade a single software package, run the following command as the user **root**: + +```shell +dnf update package_name +``` + +For example, to upgrade the RPM package, run the following command: + +```shell +$ dnf update anaconda-gui.aarch64 +Last metadata expiration check: 0:02:10 ago on Sun 01 Sep 2019 11:30:27 PM CST. +Dependencies Resolved +================================================================================ + Package Arch Version Repository Size +================================================================================ +Updating: + anaconda-gui aarch64 19.31.123-1.14 updates 461 k + anaconda-core aarch64 19.31.123-1.14 updates 1.4 M + anaconda-tui aarch64 19.31.123-1.14 updates 274 k + anaconda-user-help aarch64 19.31.123-1.14 updates 315 k + anaconda-widgets aarch64 19.31.123-1.14 updates 748 k + +Transaction Summary +================================================================================ +Upgrade 5 Package + +Total download size: 3.1 M +Is this ok [y/N]: +``` + +Similarly, to upgrade a software package group, run the following command as the user **root**: + +```shell +dnf group update group_name +``` + +### Updating All Packages and Their Dependencies + +To update all packages and their dependencies, run the following command as the user **root**: + +```shell +dnf update +``` diff --git a/docs/en/server/administration/administrator/viewing-system-information.md b/docs/en/server/administration/administrator/viewing-system-information.md new file mode 100644 index 0000000000000000000000000000000000000000..350816bcc2b399cf37456a0aa39e0de81c558b20 --- /dev/null +++ b/docs/en/server/administration/administrator/viewing-system-information.md @@ -0,0 +1,45 @@ +# Viewing System Information + +- View the system information. + + ```shell + cat /etc/os-release + ``` + + For example, the command and output are as follows: + + ```shell + $ cat /etc/os-release + NAME="openEuler" + VERSION="21.09" + ID="openEuler" + VERSION_ID="21.09" + PRETTY_NAME="openEuler 21.09" + ANSI_COLOR="0;31" + ``` + +- View system resource information. + + Run the following command to view the CPU information: + + ```shell + # lscpu + ``` + + Run the following command to view the memory information: + + ```shell + free + ``` + + Run the following command to view the disk information: + + ```shell + fdisk -l + ``` + +- View the real-time system resource information. + + ```shell + top + ``` diff --git a/docs/en/server/administration/compa_command/_toc.yaml b/docs/en/server/administration/compa_command/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..39cf43af4ebbf38765c51ead27f8c3f4b28c6744 --- /dev/null +++ b/docs/en/server/administration/compa_command/_toc.yaml @@ -0,0 +1,11 @@ +label: Compatibility Commands +isManual: true +href: ./overview.md +description: >- + The shell and Linux commands, rebuilt using Rust, maintain compatibility with + native Linux commands. +sections: + - label: utshell User Guide + href: ./utshell-guide.md + - label: utsudo User Guide + href: ./utsudo-user-guide.md diff --git a/docs/en/server/administration/compa_command/figures/-e.png b/docs/en/server/administration/compa_command/figures/-e.png new file mode 100644 index 0000000000000000000000000000000000000000..593e3e0b8a25e35a0fd7586627ab0a7b21769096 Binary files /dev/null and b/docs/en/server/administration/compa_command/figures/-e.png differ diff --git a/docs/en/server/administration/compa_command/figures/-k.png b/docs/en/server/administration/compa_command/figures/-k.png new file mode 100644 index 0000000000000000000000000000000000000000..0ab440bf2d475b3648947c8e874b442a2775c764 Binary files /dev/null and b/docs/en/server/administration/compa_command/figures/-k.png differ diff --git a/docs/en/server/administration/compa_command/figures/-l.png b/docs/en/server/administration/compa_command/figures/-l.png new file mode 100644 index 0000000000000000000000000000000000000000..66c9d4920f4ac9f8755a4720a5984df454897ed6 Binary files /dev/null and b/docs/en/server/administration/compa_command/figures/-l.png differ diff --git a/docs/en/server/administration/compa_command/figures/grep.png b/docs/en/server/administration/compa_command/figures/grep.png new file mode 100644 index 0000000000000000000000000000000000000000..bda07ec516b51bb34553a3686a7b231451f9493f Binary files /dev/null and b/docs/en/server/administration/compa_command/figures/grep.png differ diff --git a/docs/en/server/administration/compa_command/figures/install.png b/docs/en/server/administration/compa_command/figures/install.png new file mode 100644 index 0000000000000000000000000000000000000000..f5f5af55c17d00ff4dc314f9f365d74e7b0b27f0 Binary files /dev/null and b/docs/en/server/administration/compa_command/figures/install.png differ diff --git a/docs/en/server/administration/compa_command/media/commands1.png b/docs/en/server/administration/compa_command/media/commands1.png new file mode 100644 index 0000000000000000000000000000000000000000..4e857746422ed7d794e0e06ee379be30e1222443 Binary files /dev/null and b/docs/en/server/administration/compa_command/media/commands1.png differ diff --git a/docs/en/server/administration/compa_command/media/commands2.png b/docs/en/server/administration/compa_command/media/commands2.png new file mode 100644 index 0000000000000000000000000000000000000000..c840b1752fbe485854d0ef8bc22eb896055538f7 Binary files /dev/null and b/docs/en/server/administration/compa_command/media/commands2.png differ diff --git a/docs/en/server/administration/compa_command/media/install-y.png b/docs/en/server/administration/compa_command/media/install-y.png new file mode 100644 index 0000000000000000000000000000000000000000..88073a4a22ca70078dc84226b980d430757f3307 Binary files /dev/null and b/docs/en/server/administration/compa_command/media/install-y.png differ diff --git a/docs/en/server/administration/compa_command/media/install.png b/docs/en/server/administration/compa_command/media/install.png new file mode 100644 index 0000000000000000000000000000000000000000..eb5b21f176fde88398a914bdd42e1efbef5ca348 Binary files /dev/null and b/docs/en/server/administration/compa_command/media/install.png differ diff --git a/docs/en/server/administration/compa_command/media/uninstall.png b/docs/en/server/administration/compa_command/media/uninstall.png new file mode 100644 index 0000000000000000000000000000000000000000..757dda084316495125f8f47f681e91167ef64724 Binary files /dev/null and b/docs/en/server/administration/compa_command/media/uninstall.png differ diff --git a/docs/en/server/administration/compa_command/overview.md b/docs/en/server/administration/compa_command/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..28f0e23eb6713c9bb5077e567119eb0ee6fca531 --- /dev/null +++ b/docs/en/server/administration/compa_command/overview.md @@ -0,0 +1,3 @@ +# Compatibility Commands + +This document describes the shell and Linux commands re-written in the Rust language. These commands can be used on openEuler and is compatible with native Linux commands. diff --git a/docs/en/server/administration/compa_command/utshell-guide.md b/docs/en/server/administration/compa_command/utshell-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..b0dc60f37acf6c059d6a77ec1c4e4deb9241f581 --- /dev/null +++ b/docs/en/server/administration/compa_command/utshell-guide.md @@ -0,0 +1,200 @@ +# utshell User Guide + +## Introduction + +utshell is a shell compatible with Bash, capable of executing basic built-in commands and starting external commands. It also implements functions such as task, pipe, and signal processing. + +## Installation and Uninstallation + +### Installing utshell + +Run the `rpm` command to install utshell. Assume that openEuler 23.09 is used. + +![](./media/install.png) + +Enter **y** as prompted to install. + +![](./media/install-y.png) + +### Uninstalling utshell + +Run `rpm -e utshell` to uninstall utshell. + +```shell +rpm -e utshell +``` + +![](./media/uninstall.png) + +## Usage + +### Using Common Commands + +In the utshell environment, enter a command to execute. + +utshell has the following built-in commands: + +![](./media/commands1.png) +![](./media/commands2.png) + +### Defining and Using Variables + +#### Defining a Variable + +Use **=** to define a variable. No space is allowed in the expression. + +```shell +var=4 +``` + +#### Using a Variable + +```shell +echo ${var} +``` + +### Defining and Using Arrays + +#### Defining an Array + +```shell +distros=(ubuntu fedora suse "arch linux") +``` + +#### Using an Array + +```shell +echo ${distros[2]} +``` + +### Defining and Using Functions + +#### Defining a Function + +```shell +func() { echo $1; } +``` + +#### Using a Function + +```shell +func 1 +``` + +#### Passing Parameters to a Function + +When calling a function, use a space to separate the function and the parameters. + +```shell +func firstParam secondParam +``` + +In the function body, use **${number}** to represent the parameters, for example, $1 for the first parameter and $2 for the second parameter. For the tenth and subsequent parameters, the number must be enclosed in braces. + +```shell +func() { +echo $1 ${10} # Ten parameters are required. +} +# Call the function. +func 1 2 3 4 5 6 7 8 9 0 +``` + +### Using Logical Conditions + +#### if + +The syntax is as follows: + +```shell +if condition; then +do-if-true; +elif second-condition; then +do-else-if-true +elif third-condition; then +do-else-if-third-true +else +do-else-false +fi +``` + +**condition** can be a command, for example: + +```shell +if [ "$s" = "string" ]; then +echo "string is equivalent to $s" +else +echo "string is not equivalent to $s" +fi +``` + +**condition** can also be a conditional operator. + +Some conditional operators are as follows. + +```shell +-f: Checks whether a file exists and is a regular file. +-d: Checks whether the provided argument is a directory. +-h: Checks whether the provided argument is a symbolic link. +-s: Checks whether a file exists and is not empty. +-r: Checks whether a file is readable. +-w: Checks whether a file is writable. +-x: Checks whether a file is executable. +``` + +The following conditional operators can be used for comparing numbers. + +```shell +-lt: less than +-gt: greater than +-ge: greater than or equal to +-le: less than or equal to +-ne: not equal to +``` + +The following conditional operators can be used for comparing strings. + +```shell +==: Whether two strings are identical. +=: Whether two strings are identical (same as ==). +!=: Whether two strings are different. +-z: Returns true if the string is empty. +-n: Returns true if the string length is not 0. +``` + +### Using Loops + +#### for + +```shell +for number in 1 2 3 4 5 +do +echo $number +done +# When used with a list: +for number in {1..500..2} +do +echo $number +done +``` + +**{1..500..2}** indicates that the start number is 1, the end number is 500 (included), and the step is 2. + +#### until + +```shell +until [condition]; do +commands +done +``` + +When the condition is true, the loop is executed. + +#### while + +```shell +while [ condition ]; do +commands +done +``` + +When the condition is true, the loop is executed. diff --git a/docs/en/server/administration/compa_command/utsudo-user-guide.md b/docs/en/server/administration/compa_command/utsudo-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..1f7f06a10728cab21657f2675f6dd790d0857f43 --- /dev/null +++ b/docs/en/server/administration/compa_command/utsudo-user-guide.md @@ -0,0 +1,79 @@ +# utsudo User Guide + +This document describes how to install and use utsudo. utsudo is fully compatible with sudo in terms of parameter functions and plug-in usage, greatly reducing users' learning costs. + +This document is intended for utsudo developers, testers, and common users. + +## utsudo Introduction + +The utsudo project was initiated in June 2022. It aims to reconstruct sudo using the Rust language. utsudo is an efficient, secure, and flexible privilege escalation tool. The modules of utsudo include the common tool library, overall framework, and plug-in functions. + +## utsudo Installation + +In version 0.0.4, some files of utsudo conflict with those of sudo. Therefore, you need to use `yumdownloader` to download the binary RPM package of utsudo, and then run `rpm` to install the package with conflicts allowed. + +Run `yumdownloader utsudo` to download the utsudo binary RPM package. + +Then, run `sudo rpm -ivh utsudo-0.0.1-0.04.x86_64.rpm --replacefiles` to install utsudo. The execution process is as follows. + +![](./figures/install.png) + +After the installation is complete, run `rpm -qa | grep utsudo` to check whether utsudo is properly installed, as shown in the following figure. + +![](./figures/grep.png) + +As shown in the preceding figure, utsudo has been installed and the version is **0.0.1-0.04**. + +utsudo will be continuously updated in the future. + +## utsudo Usage + +`utsudo` has various options. Some options are as follows. You can run `utsudo -h` for details. + +```shell +-e, --edit Edit a file instead of running a command. +-k, --reset-timestamp Invalidate the timestamp file. +-l, --list List user privileges or check a specific command. Use the option twice for the longer format. +``` + +### `-e` + +The `-e` option is used to edit files. + +`utsudo -e` is equivalent to `sudoedit`. When the command is executed, a common user is used to edit a file. A file in the writable directory of the calling user cannot be edited unless the user is **root**. + +In a directory on which the current user does not have write permission, a file **test.txt** exists on which the current user does not have write permission. When you edit the **test.txt** file as a common user, a message is displayed indicating that you do not have the permission. You can run `utsudo -e` to edit the file. The following figure shows the execution process. + +![](./figures/-e.png) + +As shown in the figure, the content of the **test.txt** file is successfully modified. (**utsudo -e is okay!!** was added in the editor.) + +### `-k` + +The `-k` option invalidates the timestamp. + +By default, you need to enter the password when you run the `utsudo` command for the first time and every five minutes. The `-k` parameter forces the user to enter the password the next time the `utsudo` command is executed. + +![](./figures/-k.png) + +By default, you do not need to enter the password for five minutes after running `utsudo` for the first time. + +However, as shown in the figure, the `utsudo -k` command invalidates the timestamp of the `utsudo` command. + +### `-l` + +The `-l` option displays the commands that the current user can execute by using `utsudo`. + +The execution process is as follows: + +![](./figures/-l.png) + +As shown in the figure, the **test** user can run the following commands: + +```shell +(ALL) ALL +``` + +That is, all commands can be executed by user **test**, indicating that the **/etc/sudoers** file does not restrict the user. + +This section briefly describes how to use `utsudo`. Other functions and options of utsudo are not listed. diff --git a/docs/en/server/administration/sysmaster/_toc.yaml b/docs/en/server/administration/sysmaster/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..1731bf32f1ffd7e360bbf6e55913b60b9d838a54 --- /dev/null +++ b/docs/en/server/administration/sysmaster/_toc.yaml @@ -0,0 +1,19 @@ +label: sysMaster User Guide +isManual: true +href: ./overview.md +description: Server and device management using sysMaster +sections: + - label: Service Management + href: ./service_management.md + sections: + - label: Installation and Deployment + href: ./sysmaster_install_deploy.md + - label: Usage Instructions + href: ./sysmaster_usage.md + - label: Device Management + href: ./device_management.md + sections: + - label: Installation and Deployment + href: ./devmaster_install_deploy.md + - label: Usage Instructions + href: ./devmaster_usage.md diff --git a/docs/en/server/administration/sysmaster/device_management.md b/docs/en/server/administration/sysmaster/device_management.md new file mode 100644 index 0000000000000000000000000000000000000000..0000f4c4ff28be245e065065cc5cb34ced0127d3 --- /dev/null +++ b/docs/en/server/administration/sysmaster/device_management.md @@ -0,0 +1,14 @@ +# Device Management + +The device manager is a bridge between user-mode software and underlying physical devices, supporting the operation of key base software such as lvm2 and NetworkManager. As the device management component of sysMaster, devmaster supports quick startup of sysMaster and ecosystem compatibility of user-mode software. In addition, devmaster provides layered, decoupled, and scalable device management capabilities for common OSs based on the summary and contemplation of mainstream Linux device management solutions. + +devmaster consists of a daemon, a client tool, and a dynamic library. The devmaster daemon utilizes kernel mechanisms such as netlink, inotify, and sysfs to monitor device events and trigger rule processing tasks. The `devctl` client tool and **libs** dynamic library provide a set of CLI commands and public interfaces for debugging rules, controlling daemons, and querying device status. The following figure shows the overall architecture of devmaster. + +**Figure 1 devmaster overall architecture** +![devmaster_architecture](./figures/devmaster_architecture.png) + +devmaster is written in the Rust language to ensure memory safety. The core functions of devmaster are as follows: + +1. Event-driven operations: The queue cache and worker pool mechanisms are used to meet the requirements of highly concurrent device events. In addition, user-mode processes can be dynamically notified of the readiness of devices. +2. Separation of mechanisms and policies: Device processing logic is defined as rules rather than hard-coded in service code, allowing for on-demand customization and flexible combination. +3. Ecosystem compatibility: devmaster is compatible with the udev syntax and udev user-mode broadcast protocol. Existing services can be migrated to the devmaster environment with low costs. diff --git a/docs/en/server/administration/sysmaster/devmaster_install_deploy.md b/docs/en/server/administration/sysmaster/devmaster_install_deploy.md new file mode 100644 index 0000000000000000000000000000000000000000..e12ac33f71bd262de3d8df12c51fbeebecc9b682 --- /dev/null +++ b/docs/en/server/administration/sysmaster/devmaster_install_deploy.md @@ -0,0 +1,68 @@ +# Installation and Deployment + +Currently, devmaster can be used in the VM environment. This section describes the requirements and procedure of devmaster installation and deployment. + +## Software + +* OS: openEuler 23.09 + +## Hardware + +* x86_64 or AArch64 architecture + +## Installation and Deployment + +1. Run the following `yum` command to install the sysmaster-devmaster package: + + ```shell + # yum install sysmaster-devmaster + ``` + +2. Run the following commands to create the default rule file **/etc/devmaster/rules.d/99-default.rules** and the daemon configuration file **/etc/devmaster/config.toml**: + + ```shell + # mkdir -p /etc/devmaster/rules.d + # mkdir -p /etc/devmaster/network.d + # echo "TAG+=\"devmaster\"" > /etc/devmaster/rules.d/99-default.rules + # cat << EOF > /etc/devmaster/config.toml + log_level = "info" + rules_d = ["/etc/devmaster/rules.d"] + network_d = ["/etc/devmaster/network.d"] + max_workers = 1 + log_targets = ["console"] + EOF + ``` + +3. Run the following commands to start the devmaster daemon and export logs to the **/tmp/devmaster.log** file: + + ```shell + # /lib/devmaster/devmaster &>> /tmp/devmaster.log & + ``` + + > ![Note](./public_sys-resources/icon-note.gif)**Note:** + > + > devmaster must be started with the root privilege and cannot be running with udev at the same time. Before starting devmaster, stop the udev service. + > + > If udev is started by sysMaster, run the following command: + + ```shell + # sctl stop udevd.service udevd-control.socket udevd-kernel.socket + ``` + + > If udev is started by systemd, run the following command: + + ```shell + # systemctl stop systemd-udevd.service systemd-udevd systemd-udevd-kernel.socket systemd-udevd-control.socket + ``` + +4. Run the following command to use the `devctl` tool to trigger a device event: + + ```shell + # devctl trigger + ``` + +5. Check the **/run/devmaster/data/** directory. If the device database is generated, the deployment is successful. + + ```shell + # ll /run/devmaster/data/ + ``` diff --git a/docs/en/server/administration/sysmaster/devmaster_usage.md b/docs/en/server/administration/sysmaster/devmaster_usage.md new file mode 100644 index 0000000000000000000000000000000000000000..ac3fc1cf5be0cd8a5a267010a56edb651ffb3ecd --- /dev/null +++ b/docs/en/server/administration/sysmaster/devmaster_usage.md @@ -0,0 +1,254 @@ +# Usage Instructions + +This section describes how to use devmaster, covering daemon configuration, client tool, rule usage, and NIC configuration. + +## Daemon Configuration + +After being started, the devmaster daemon reads the configuration file, adjusts the log level, and sets the rule path based on the configuration file. devmaster has a unique configuration file **/etc/devmaster/config.toml**, which is in TOML format. + +### Configuration Items + +The devmaster configuration file supports the following configuration items: + +- **rules_d**: Rule path. The default value is **\["/etc/devmaster/rules.d"]**. If this item is not specified, there is no default path. Currently, devmaster does not support rule loading priorities. Rule files with the same name in different rule paths will not conflict with each other. Rule files are loaded in the sequence specified by **rules_d**. Rule files in the same directory are loaded in the lexicographical sequence. +- **max_workers**: Maximum number of concurrent worker threads. If this item is not specified, the default value **3** is used. +- **log_level**: Log level. The value can be **debug** or **info**. If this parameter is not specified, the default value **info** is used. +- **network_d**: NIC configuration path. The default value is **\["/etc/devmaster/network.d"]**. If this parameter is not specified, there is no default path. NIC configurations control the behavior of the `net_setup_link` command of devmaster. For details, see [NIC Configuration](#nic-configuration). + +## Client Tool + +`devctl` is the client tool of the devmaster daemon. It is used to control devmaster behaviors, simulate device events, and debug rules. + + ```shell + # devctl --help + devmaster 0.5.0 + parse program arguments + + USAGE: + devctl + + OPTIONS: + -h, --help Print help information + -V, --version Print version information + + SUBCOMMANDS: + monitor Monitor device events from kernel and userspace + kill Kill all devmaster workers + test Send a fake device to devmaster + trigger Trigger a fake device action, then the kernel will report an uevent + test-builtin Test builtin command on a device + help Print this message or the help of the given subcommand(s) + ``` + +Command options: + + `-h, --help`: Displays help information. + + `-V, --version`: Displays version information. + + ``: Subcommand to be executed, including `monitor`, `trigger`, and `test-builtin`. + +The following sections describe the three frequently used subcommands, which are used to monitor device events, trigger device events, and test built-in commands. + +### Monitoring Device Events + +Monitor uevent events reported by the kernel and events sent after devmaster processes devices, which are prefixed with **KERNEL** and **USERSPACE**, respectively. The command is as follows: + + ```shell + # devctl monitor [OPTIONS] + ``` + +Command options: + + `-h, --help`: Displays help information. + +### Triggering Device Events + +Simulate a device action to trigger a kernel uevent event. This operation is used to replay coldplug device events during kernel initialization. The command is as follows: + + ```shell + # devctl trigger [OPTIONS] [DEVICES...] + ``` + +Command options: + + `-h, --help`: Displays help information. + + `-a, --action `: Action type of a device event. + + `-t, --type `: Type of the device to be searched for. The value can be **devices** or **subsystems**. + + `-v, --verbose`: Prints the found devices. + + `-n, --dry-run`: Does not trigger device events. This option can be used together with `--verbose` to view the list of devices in the system. + + `[DEVICES...]`: Devices for which events are triggered. If this item is left blank, events of all devices in the system are triggered. + +### Testing Built-in Commands + +Test the effect of a built-in command on a device. The command is as follows: + + ```shell + # devctl test-builtin [OPTIONS] + ``` + +Command options: + + `-a, --action `: Action type of a device event. The value can be **add**, **change**, **remove**, **move**, **online**, **offline**, **bind**, or **unbind**. + + `-h, --help`: Displays help information. + + ``: Built-in command to be executed. The value can be **blkid**, **input_id**, **kmod**, **net_id**, **net_setup_link**, **path_id**, or **usb_id**. + + ``: Sysfs path of the device. + +## Rule Usage + +devmaster rules consist of a group of rule files. After the devmaster daemon is started, it loads the rule files in lexicographic order based on the rule path specified in the configuration file. + +> ![Note](./public_sys-resources/icon-note.gif)**Note:** +> +> After adding, deleting, or modifying a rule, you need to restart devmaster for the rule to take effect. + +### Rule Examples + +The following describes several common rule examples. For details about the rule syntax, see the [devmaster manual](http://sysmaster.online/man/exts/devmaster/devmaster/). + +#### Example 1: Creating a Soft Link for a Block Device + +Use the `blkid` built-in command to read the UUID of a block device and create a soft link for the block device based on the UUID. + +After an event of a device that has a file system is triggered, a soft link corresponding to the device is generated in the **/dev/test** directory. + +The following uses the block device of the **sda1** partition as an example. + +1. Create the rule file **/etc/devmaster/rules.d/00-persist-storage.rules**. The file content is as follows: + + ```shell + SUBSYSTEM!="block", GOTO="end" + + IMPORT{builtin}=="blkid" + + ENV{ID_FS_UUID_ENC}=="?*", SYMLINK+="test/$env{ID_FS_UUID_ENC}" + + LABEL="end" + ``` + +2. Trigger the **sda1** device event: + + ```shell + # devctl trigger /dev/sda1 + ``` + +3. Check if a soft link pointing to **sda1** exists in the **/dev/test/** directory. If yes, the rule takes effect. + + ```shell + # ll /dev/test/ + total 0 + lrwxrwxrwx 1 root root 7 Sep 6 15:35 06771fe1-39da-42d7-ad3c-236a10d08a7d -> ../sda1 + ``` + +#### Example 2: Renaming a NIC + +Use the `net_id` built-in command to obtain the hardware attributes of the NIC, then run the `net_setup_link` built-in command to select a hardware attribute based on the NIC configuration as the NIC name, and rename the NIC through the **NAME** rule. + +The following uses the **ens33** NIC as an example to test the effect of the NIC renaming rule: + +1. Create the rule file **/etc/devmaster/rules.d/01-netif-rename.rules**. The file content is as follows: + + ```shell + SUBSYSTEM!="net", GOTO="end" + + IMPORT{builtin}=="net_id" + + IMPORT{builtin}=="net_setup_link" + + ENV{ID_NET_NAME}=="?*", NAME="$env{ID_NET_NAME}" + + LABEL="end" + ``` + +2. Create the NIC configuration file **/etc/devmaster/network.d/99-default.link**. The content is as follows: + + ```shell + [Match] + OriginalName = "*" + + [Link] + NamePolicy = ["database", "onboard", "slot", "path"] + ``` + +3. Bring the NIC offline. + + ```shell + # ip link set ens33 down + ``` + +4. Temporarily name the NIC **tmp**: + + ```shell + # ip link set ens33 name tmp + ``` + +5. Trigger the **add** event of the NIC. + + ```shell + # devctl trigger /sys/class/net/tmp --action add + ``` + +6. Check the NIC name. If the NIC name is changed to **ens33**, the rule takes effect. + + ```shell + # ll /sys/class/net/| grep ens33 + lrwxrwxrwx 1 root root 0 Sep 6 11:57 ens33 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:01.0/net/ens33 + ``` + +7. Restore the network connection after activating the NIC. + + ```shell + # ip link set ens33 up + ``` + +> ![Note](./public_sys-resources/icon-note.gif)**Note:** +> +> An activated NIC cannot be renamed. You need to bring it offline first. In addition, the renaming rule of devmaster takes effect only in the **add** event of the NIC. +> + +#### Example 3: Modifying the User Permissions on a Device Node + +The **OPTIONS+="static_node=\** rule enables devmaster to immediately apply the user permissions in this rule to **/dev/\** after devmaster is started. The configuration takes effect immediately after devmaster is restarted. No device event is required. + +1. Create the rule file **/etc/devmaster/rules.d/02-devnode-privilege.rules**. The file content is as follows: + + ```shell + OWNER="root", GROUP="root", MODE="777", OPTIONS+="static_node=tty5" + ``` + +2. After devmaster is restarted, check the user, user group, and permissions of **/dev/tty5**. If the user, user group, and permissions are changed to **root**, **root**, and **rwxrwxrwx**, the rule takes effect. + + ```shell + # ll /dev/tty5 + crwxrwxrwx 1 root root 4, 5 Feb 3 2978748 /dev/tty5 + ``` + +## NIC Configuration + +The NIC renaming function of devmaster is implemented by the built-in commands `net_id` and `net_setup_link` and the NIC configuration file. In the rule file, use `net_id` to obtain the hardware attributes of a NIC, and then use `net_setup_link` to select a NIC attribute as the new NIC name. The `net_setup_link` command controls the NIC naming style for a specific NIC based on the NIC configuration file. This section describes how to use the NIC configuration file. For details about how to rename a NIC, see [Renaming a NIC](#example-2-renaming-a-nic). + +### Default NIC Configurations + +devmaster provides the following default NIC configurations: + + ```toml + [Match] + OriginalName = "*" + + [Link] + NamePolicy = ["onboard", "slot", "path"] + ``` + +The NIC configuration file contains the **\[Match]** matching section and **\[Link]** control section. Each section contains several configuration items. The configuration items in the **\[Match]** section are used to match NICs. When a NIC meets all matching conditions, all configuration items in the **\[Link]** section are applied to the NIC, for example, setting the NIC naming style and adjusting NIC parameters. + +The preceding default NIC configuration indicates that the configuration takes effect on all NICs and checks the NIC naming styles of the **onboard**, **slot**, and **path** styles in sequence. If an available style is found, the NIC is named in this style. + +For details about the NIC configuration, see the [devmaster manual](http://sysmaster.online/man/exts/devmaster/netif_config/#1). diff --git a/docs/en/server/administration/sysmaster/figures/devmaster_architecture.png b/docs/en/server/administration/sysmaster/figures/devmaster_architecture.png new file mode 100644 index 0000000000000000000000000000000000000000..c3f0e4570d84f5ed513e0a02e0759cc3e4fb3db0 Binary files /dev/null and b/docs/en/server/administration/sysmaster/figures/devmaster_architecture.png differ diff --git a/docs/en/server/administration/sysmaster/figures/sysMaster.png b/docs/en/server/administration/sysmaster/figures/sysMaster.png new file mode 100644 index 0000000000000000000000000000000000000000..85f901da2ddc33059c29df1c86b9023516921dbd Binary files /dev/null and b/docs/en/server/administration/sysmaster/figures/sysMaster.png differ diff --git a/docs/en/server/administration/sysmaster/overview.md b/docs/en/server/administration/sysmaster/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..5fe40cb647b35329a96d0777cd8f53323b2548dd --- /dev/null +++ b/docs/en/server/administration/sysmaster/overview.md @@ -0,0 +1,28 @@ +# sysMaster User Guide + +## Overview + +sysMaster is a collection of ultra-lightweight and highly reliable service management programs. It provides an innovative implementation of PID 1 to replace the conventional init process. Written in Rust, sysMaster is equipped with fault monitoring, second-level self-recovery, and quick startup capabilities, which help improve OS reliability and service availability. + +sysMaster supports unified management of processes, containers, and VMs and applies to multiple scenarios such as servers, cloud computing, and embedded systems. + +sysMaster divides the functions of traditional PID 1 into a 1+1+N architecture based on application scenarios. + +As shown in the figure, sysMaster consists of three components: + +- sysmaster-init, which is a new implementation of PID 1, features simplified functions, a thousand lines of code (KLOC), and ultimate reliability. It is applicable to embedded systems with functions such as system initialization, zombie process recycling, and keep-alive monitoring. +- sysmaster-core undertakes the core service management functions and incorporates the reliability framework to enable live updates and quick self-recovery in the event of crashes, ensuring 24/7 service availability. +- sysmaster-exts offers a set of components (such as devMaster for device management and busMaster for bus communication) that deliver key system functions, which are coupled in traditional PID 1. Users can choose the components to use as required. + +**Figure 1** sysMaster architecture + +![sysMaster](./figures/sysMaster.png) + +**sysmaster** and **devmaster** are the two main packages of sysMaster, which are responsible for service management and device management, respectively. + +## Intended Audience + +This document is intended for openEuler users who need to manage services and devices. The users are expected to: + +- Know basic Linux operations. +- Know service configuration and devices. diff --git a/docs/en/server/administration/sysmaster/public_sys-resources/icon-note.gif b/docs/en/server/administration/sysmaster/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/server/administration/sysmaster/public_sys-resources/icon-note.gif differ diff --git a/docs/en/server/administration/sysmaster/service_management.md b/docs/en/server/administration/sysmaster/service_management.md new file mode 100644 index 0000000000000000000000000000000000000000..617ec7d9562ad9ad154009671c245ef242a85aaa --- /dev/null +++ b/docs/en/server/administration/sysmaster/service_management.md @@ -0,0 +1,5 @@ +# Service Management + +Many background programs and processes in Linux, such as web servers, database servers, and mail servers, are started and stopped during system startup and running. sysmaster provides efficient service management commands and configurations to ensure the normal running of the system. + +This document describes the installation and deployment of sysmaster, as well as its features and usage. diff --git a/docs/en/server/administration/sysmaster/sysmaster_install_deploy.md b/docs/en/server/administration/sysmaster/sysmaster_install_deploy.md new file mode 100644 index 0000000000000000000000000000000000000000..91a82913e8715b0188791abefe261d30dee1bc98 --- /dev/null +++ b/docs/en/server/administration/sysmaster/sysmaster_install_deploy.md @@ -0,0 +1,98 @@ +# Installation and Deployment + +sysmaster can be used in containers and VMs. This document uses the AArch64 architecture as an example to describe how to install and deploy sysmaster in both scenarios. + +## Software + +* OS: openEuler 23.09 + +## Hardware + +* x86_64 or AArch64 architecture + +## Installation and Deployment in Containers + +1. Install Docker. + + ```bash + yum install -y docker + systemctl restart docker + ``` + +2. Load the base container image. + + Download the container image. + + ```bash + wget https://repo.openeuler.org/openEuler-23.09/docker_img/aarch64/openEuler-docker.aarch64.tar.xz + xz -d openEuler-docker.aarch64.tar.xz + ``` + + Load the container image. + + ```bash + docker load --input openEuler-docker.aarch64.tar + ``` + +3. Build the container. + + Create a Dockerfile. + + ```bash + cat << EOF > Dockerfile + FROM openeuler-23.09 + RUN yum install -y sysmaster + CMD ["/usr/lib/sysmaster/init"] + EOF + ``` + + Build the container. + + ```bash + docker build -t openeuler-23.09:latest . + ``` + +4. Start and enter the container. + + Start the container. + + ```bash + docker run -itd --privileged openeuler-23.09:latest + ``` + + Obtain the container ID. + + ```bash + docker ps + ``` + + Use the container ID to enter the container. + + ```bash + docker exec -it /bin/bash + ``` + +## Installation and Deployment in VMs + +1. Create an initramfs image. + To avoid the impact of systemd in the initrd phase, you need to create an initramfs image with systemd removed and use this image to enter the initrd procedure. Run the following command: + + ```bash + dracut -f --omit "systemd systemd-initrd systemd-networkd dracut-systemd" /boot/initrd_withoutsd.img + ``` + +2. Add a boot item. + Add a boot item to **grub.cfg**, whose path is **/boot/efi/EFI/openEuler/grub.cfg** in the AArch64 architecture and **/boot/grub2/grub.cfg** in the x86_64 architecture. Back up the original configurations and modify the configurations as follows: + + * **menuentry**: Change **openEuler (6.4.0-5.0.0.13.oe23.09.aarch64) 23.09** to **openEuler 23.09 withoutsd**. + * **linux**: Change **root=/dev/mapper/openeuler-root ro** to **root=/dev/mapper/openeuler-root rw**. + * **linux**: If Plymouth is installed, add **plymouth.enable=0** to disable it. + * **linux**: Add **init=/usr/lib/sysmaster/init**. + * **initrd**: Set to **/initrd_withoutsd.img**. +3. Install sysmaster. + + ```bash + yum install sysmaster + ``` + +4. If the **openEuler 23.09 withoutsd** boot item is displayed after the restart, the configuration is successful. Select **openEuler 23.09 withoutsd** to log in to the VM. diff --git a/docs/en/server/administration/sysmaster/sysmaster_usage.md b/docs/en/server/administration/sysmaster/sysmaster_usage.md new file mode 100644 index 0000000000000000000000000000000000000000..d0ae28310f5d39c1cb5eb17227aa47cb792652b7 --- /dev/null +++ b/docs/en/server/administration/sysmaster/sysmaster_usage.md @@ -0,0 +1,104 @@ +# sysmaster Usage Instructions + +This section provides examples on how to use sysmaster, including: + +* service unit configuration file creation +* unit service management operations, such as starting, stopping, and viewing services + +For more, see the [sysMaster official manual](http://sysmaster.online/man/all/). + +## Unit Configuration File Creation + +You can create unit configuration files in the **/usr/lib/sysmaster/system/** directory. + +### Types of Unit Configuration Files + +Currently, sysmaster supports unit configuration files of the **target**, **socket**, and **service** types. + +* **target**: Encapsulated startup target managed by sysmaster, which is used for grouping units as a synchronization point. sysmaster provides targets for different states. For example, **multi-user.target** indicates that the system has been started. You can use this target to configure services to run in this state. +* **socket**: Encapsulated socket for inter-process communication to support socket-based startup. For example, you can configure a service unit to depend on a socket. When data is written to the socket, sysmaster starts the corresponding service unit. +* **service**: Encapsulated process monitored and controlled by sysmaster. + +### Composition of Unit Configuration Files + +A unit configuration file consists of three sections: + +* **Unit**: common configuration description of the unit, such as the service name, description, and dependencies +* **Install**: description of how the service is installed and started +* **Service** and **Socket**: configurations of different unit types + +### Creating a service Unit + +The **sshd** service is used to remotely log in to the server and run commands and perform operations on the remote terminal. +The following configuration items are used to create an **sshd.service** service unit: + +```bash +[Unit] +Description="OpenSSH server daemon" +Documentation="man:sshd(8) man:sshd_config(5)" +After="sshd-keygen.target" +Wants="sshd-keygen.target" + +[Service] +Type="notify" +EnvironmentFile="-/etc/sysconfig/sshd" +ExecStart="/usr/sbin/sshd -D $OPTIONS" +ExecReload="/bin/kill -HUP $MAINPID" +KillMode="process" +Restart="on-failure" +RestartSec=42 + +[Install] +WantedBy="multi-user.target" +``` + +The configuration items in the example are described as follows: + +* **Description**: Main functions of the unit. +* **Documentation**: Document link of the unit. +* **After**: Unit startup sequence. In the example, **sshd.service** is started after **sshd-keygen.target**. +* **Wants**: Dependency on another unit. In the example, **sshd-keygen.target** is automatically started with **sshd.service**. +* **Type**: How sysmaster starts the service. **notify** indicates that a notification will be sent after the main process is started. +* **EnvironmentFile**: Path of file that stores environment variables to be loaded. +* **ExecStart**: Command executed when the service is started. In the example, `sshd` is executed when **sshd.service** is started. +* **ExecReload**: Command executed to reload the **sshd.service** configurations. +* **KillMode**: How the process is killed when the service process needs to be stopped. **process** indicates that only the main process is killed. +* **Restart**: Whether to restart the service when the service exits or stops in different situations. **on-failure** indicates that the service is restarted when the service exits abnormally. +* **RestartSec**: Amount of time to wait before the service is restarted after the service exits. +* **WantedBy**: Units that depend on **sshd.service**. + +## Unit Service Management + +`sctl` is a CLI tool of sysmaster. It is used to check and control the behavior of the sysmaster server and the status of each service. It can start, stop, restart, and check system services. + +### Starting a Service + +Run the following command to start the **sshd** service and run the commands specified by **ExecStart**: + +```bash +# sctl start sshd.service +``` + +### Stopping a Service + +Run the following command to stop the **sshd** service and kill the process started by **ExecStart**: + +```bash +# sctl stop sshd.service +``` + +### Restarting a Service + +Run the following command to restart the **sshd** service. After the command is executed, the **sshd** service is stopped and then started. + +```bash +# sctl restart sshd.service +``` + +### Checking Service Status + +Run the following command to check the status of the **sshd** service. You can check whether the service is running properly by viewing the service status. + +```bash +# sctl status sshd.service +``` diff --git a/docs/en/server/development/application_dev/_toc.yaml b/docs/en/server/development/application_dev/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..6f8216e326ab7e86b505cd39c6cf973c3f4ff2e4 --- /dev/null +++ b/docs/en/server/development/application_dev/_toc.yaml @@ -0,0 +1,19 @@ +label: Application Development Guide +isManual: true +href: ./application-development.md +description: Application development on openEuler +sections: + - label: Preparing the Development Environment + href: ./preparations-for-development-environment.md + - label: Using GCC for Compilation + href: ./using-gcc-for-compilation.md + - label: Using LLVM/Clang for Compilation + href: ./using-clang-for-compilation.md + - label: Using Make for Compilation + href: ./using-make-for-compilation.md + - label: Using JDK for Compilation + href: ./using-jdk-for-compilation.md + - label: Building an RPM Package + href: ./building-an-rpm-package.md + - label: Common Issues and Solutions + href: ./common-issues-and-solutions.md diff --git a/docs/en/server/development/application_dev/application-development.md b/docs/en/server/development/application_dev/application-development.md new file mode 100644 index 0000000000000000000000000000000000000000..a4865d3a139e27ba40712d3e8e547ca84e3839b4 --- /dev/null +++ b/docs/en/server/development/application_dev/application-development.md @@ -0,0 +1,95 @@ +# Application Development Guide + +This document describes the common tools used for application development and guides users to develop applications based on openEuler. + +## Overview + +This document describes the following four parts to guide users to use openEuler and develop code based on openEuler. + +- Install and use the GCC compiler in the openEuler operating system \(OS\), and complete the development, compilation, and execution of simple code. +- In the openEuler OS, use the JDK built-in tool to compile and execute code. +- Install IntelliJ IDEA in the openEuler OS for Java development. +- Create an RPM package locally or using the Open Build Service \(OBS\). + +## Intended Audience + +This document is intended for all users who use the openEuler OS for code development. You are expected to have the following experience or capabilities: + +- Have basic knowledge of the Linux OS. +- Know how to use Linux command lines. + +## Symbol Conventions + +The symbols that may be found in this document are defined as follows. + + + + + + + + + + + + + + +

Symbol

+

Description

+

![](./figures/en-us_image_0229243712.png)

+

Indicates a potentially hazardous situation which, if not avoided, could result in equipment damage, data loss, performance deterioration, or unanticipated results.

+

NOTICE is used to address practices not related to personal injury.

+

![](./figures/en-us_image_0229243671.png)

+

Supplements the important information in the main text.

+

NOTE is used to address information not related to personal injury, equipment damage, and environment deterioration.

+
+ +## Command Conventions + +**Table 1** Command conventions + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Format

+

Description

+

Boldface

+

Command keywords, which remain unchanged in the commands, are in boldface.

+

Italic

+

Command parameters, which are replaced with actual values in the commands, are in italic.

+

[ ]

+

Items in square brackets are optional.

+

{ x | y | ... }

+

Optional items are grouped in braces and separated by vertical bars. One item is selected.

+

[ x | y | ... ]

+

Optional items are grouped in brackets and separated by vertical bars. One item is selected or no item is selected.

+

{ x | y | ... }\*

+

Optional items are grouped in brackets and separated by vertical bars. A minimum of one or a maximum of all can be selected.

+

[ x | y | ... ]\*

+

Optional items are grouped in brackets and separated by vertical bars. One or more items are selected or no item is selected.

+
diff --git a/docs/en/server/development/application_dev/building-an-rpm-package.md b/docs/en/server/development/application_dev/building-an-rpm-package.md new file mode 100644 index 0000000000000000000000000000000000000000..52f4a0af4b6a5c9690df5ded3763a8f0e0058d05 --- /dev/null +++ b/docs/en/server/development/application_dev/building-an-rpm-package.md @@ -0,0 +1,538 @@ +# Building an RPM Package + +This section describes how to build an RPM software package on a local PC or using OBS. For details, see the openEuler Packaging Guide. + +## Packaging Description + +### Principles + +During RPM packaging, the source code needs to be compiled. The compiled configuration files and binary command files need to be placed in proper positions. The RPM packages need to be tested as required. A workspace is required for these operations. The **rpmbuild** command uses a set of standard workspaces. + +```shell +rpmdev-setuptree +``` + +The **rpmdev-setuptree** command is used to install rpmdevtools. After the command is executed, the **rpmbuild** folder is generated in the **/root** directory \(or the **/home/**_username_ directory for non-root users\). The directory structure is as follows: + +```shell +$ tree rpmbuild +rpmbuild +├── BUILD +├── RPMS +├── SOURCES +├── SPECS +└── SRPMS +``` + +The content is described as follows: + +| Directory | Macro | Name | Function | +|--------------------|--------------|-----------------|-------------------------------------------| +|~/rpmbuild/BUILD |%_builddir |Build directory |The source code package is decompressed and compiled in a subdirectory of this directory. | +|~/rpmbuild/RPMS |%_rpmdir |Standard RPM package directory |The binary RPM package is generated and stored in this directory. | +|~/rpmbuild/SOURCES |%_sourcedir |Source code directory |The source code package (for example, .tar package) and all patches are stored in this directory. | +|~/rpmbuild/SPECS |%_specdir |Spec file directory |The RPM package configuration file (.spec) is stored in this directory. | +|~/rpmbuild/SPECS |%_srcrpmdir |Source RPM package directory |The source RPM (SRPM) package is stored in this directory. | +|~/rpmbuild/BUILDROOT|%_buildrootdir|Installation directory |Files installed during the %install phase is stored in this directory. | + +The **\~/rpmbuild/SPECS** directory contains the configuration file of the RPM package, which is the drawing of the RPM package. This file tells the **rpmbuild** command how to build the RPM package. The **Macro Code** column contains the corresponding directories in the .spec file, which is similar to the macro or global variable in the programming language. + +### Packaging Process + +The packaging process is as follows: + +1. Place the source code in **%\_sourcedir**. +2. Compile the source code in **%\_builddir**. Generally, the source code is compressed and needs to be decompressed first. +3. Install the RPM package. The installation is similar to pre-assembling the software package. Copy the contents \(such as binary files, configuration files, and man files\) that should be contained in the software package to **%\_buildrootdir** and assemble the contents based on the actual directory structure after installation. For example, if binary commands are stored in **/usr/bin**, copy the directory structure to **%\_buildrootdir**. +4. Perform necessary configurations, such as preparations before installation and cleanup after installation. These are configured in the SPEC file to tell the **rpmbuild** command how to build. +5. Check whether the software is running properly. +6. The generated RPM package is stored in **%\_rpmdir**, and the source code package is stored in **%\_srcrpmdir**. + +In the SPEC file, each phase is described as follows: + +| Phase | Directory to Be Read | Directory to Which Data Is Written | Action | +|-------------------|--------------|-----------------|-------------------------------------------| +| %prep | %_sourcedir | %_builddir | Read the source code and patch in the **%_sourcedir** directory. Then, decompress the source code to the **%_builddir** subdirectory and apply all patches. | +| %build | %_builddir | %_builddir |Compile files in the **%_builddir** build directory. Run a command similar to `./configure && make`.| +| %install | %_builddir | %_buildrootdir |Read files in the **%_builddir** build directory and install them to the **%_buildrootdir** directory. These files are generated after the RPM is installed.| +| %check | %_builddir | %_builddir | Check whether the software is running properly. Run a command similar to `make test`.| +| bin | %_buildrootdir | %_rpmdir| Read files in the **%_buildrootdir** final installation directory to create RPM packages in the **%_rpmdir** directory. In this directory, RPM packages of different architectures are stored in different subdirectories. The **noarch** directory stores RPM packages applicable to all architectures. These RPM files are the RPM packages that are finally installed by users. | +| src | %_sourcedir | %_srcrpmdir | Create the source code RPM package (SRPM for short, with the file name extension .src.rpm) and save it to the **%_srcrpmdir** directory. The SRPM package is usually used to review and upgrade software packages. | + +### Packaging Options + +Run the **rpmbuild** command to build the software package. The **rpmbuild** command can be used to build software packages by building .spec, .tar, and source files. + +The format of the **rpmbuild** command is rpmbuild \[_option_...\] + +[Table 1](#table1342946175212) describes the common rpmbuild packaging options. + +**Table 1** rpmbuild Packaging Options + +| Option | Description | +|----------|--------------| +|-bp _specfile_ |Starts build from the **%prep** phase of the _specfile_ (decompress the source code package and install the patch). | +|-bc _specfile_ |Starts build from the **%build** phase of the _specfile_. | +|-bi _specfile_ |Starts build from the **%install** phase of the _specfile_. | +|-bl _specfile_ |Starts check from the **%files** phase of the _specfile_. | +|-ba _specfile_ |Uses the _specfile_ to build the source code package and binary package. | +|-bb _specfile_ |Uses the _specfile_ to build the binary package. | +|-bs _specfile_ |Uses the _specfile_ to build the source code package. | +|-rp _sourcefile_ |Starts build from the **%prep** phase of the _sourcefile_ (decompress the source code package and install the patch). | +|-rc _sourcefile_ |Starts build from the **%build** phase of the _sourcefile_. | +|-ri _sourcefile_ |Starts build from the **%install** phase of the _sourcefile_. | +|-rl _sourcefile_ |Starts build from the **%files** phase of the _sourcefile_. | +|-ra _sourcefile_ |Uses the _sourcefile_ to build the source code package and binary package. | +|-rb _sourcefile_ |Uses the _sourcefile_ to build the binary package. | +|-rs _sourcefile_ |Uses the _sourcefile_ to build the source code package. | +|-tp _tarfile_ |Starts build from the **%prep** phase of the _tarfile_ (decompress the source code package and install the patch). | +|-tc _tarfile_ |Starts build from the **%build** phase of the _tarfile_. | +|-ti _tarfile_ |Starts build from the **%install** phase of the _tarfile_. | +|-ta _tarfile_ |Uses the _tarfile_ to build the source code package and binary package. | +|-tb _tarfile_ |Uses the _tarfile_ to build the binary package. | +|-ts _tarfile_ |Uses the _tarfile_ to build the source code package. | +|--buildroot=_DIRECTORY_ |During the build, uses _DIRECTORY_ to overwrite the default **/root** directory. | +|--clean |Deletes the files in the **BUILD** directory. | +|--nobuild |No actual build steps are performed. It can be used to test the .spec file. | +|--noclean |Skips the %clean phase of the .spec file (even if it does exist). | +|--nocheck |Skips the %check phase of the .spec file (even if it does exist). | +|--dbpath _DIRECTORY_ |Uses the database in _DIRECTORY_ instead of the default directory **/var/lib/rpm**. | +|--root _DIRECTORY_ |Sets _DIRECTORY_ to the highest level. The default value is **/**, indicating the highest level. | +|--recompile _sourcefile_ |Installs the specified source code package _sourcefile_, that is, start preparation, compilation, and installation of the source code package. | +|--rebuild _sourcefile_ |Builds a new binary package based on `--recompile`. When the build is complete, the build directory, source code, and .spec file are deleted. The deletion effect is the same as that of --clean. | +|-?, --help |Displays detailed help information. | +|--version |Displays detailed version information. | + +## Building an RPM Package Locally + +This section uses an example to describe how to build an RPM software package locally. + +### Setting Up the Development Environment + +#### Prerequisites + +You have obtained the **root** permission, and have configured a repo source for openEuler. + +#### Procedure + +You can use the DNF tool to install rpmdevtools, including the **rpmbuild** command and related dependencies \(such as make and gdb\). Run the following command: + +```shell +dnf install rpmdevtools* +``` + +### Creating a Hello World RPM Package + +The following uses the packaging process of the GNU Hello World project as an example. The package contains the most common peripheral components related to the typical Free and Open Source Software \(FOSS\) project, including the configuration, compilation, and installation environments, documents, and internationalization \(i18n\) information. + +#### Obtaining the Source Code + +Run the following command to download the source code of the official example: + +```shell +rpmdev-setuptree +cd ~/rpmbuild/SOURCES +wget http://ftp.gnu.org/gnu/hello/hello-2.10.tar.gz +``` + +#### Editing the SPEC File + +Run the following command to create the .spec file in the **~/rpmbuild/SPECS** directory: + +```shell +cd ~/rpmbuild/SPECS +vi hello.spec +``` + +Write the corresponding content to the file and save the file. The following is an example of the file content. Modify the corresponding fields based on the actual requirements. + +```text +Name: hello +Version: 2.10 +Release: 1 +Summary: The "Hello World" program from GNU +License: GPL-3.0-or-later +URL: https://ftp.gnu.org/gnu/hello +Source0: https://ftp.gnu.org/gnu/hello/%{name}-%{version}.tar.gz + +BuildRequires: gettext + +%description +The "Hello World" program, done with all bells and whistles of a proper FOSS +project, including configuration, build, internationalization, help files, etc. + +%prep +%autosetup -p1 -n %{name}-%{version} + +%build +%configure +%make_build + +%install +%make_install +%find_lang %{name} + +%check +%make_build check + +%files -f %{name}.lang +%doc AUTHORS ChangeLog NEWS README THANKS TODO +%license COPYING +%{_bindir}/hello +%{_mandir}/man1/hello.1* +%{_infodir}/hello.info* + +%changelog +* Thu Dec 26 2019 Your Name - 2.10-1 +- Update to 2.10 + +* Sat Dec 3 2016 Your Name - 2.9-1 +- Update to 2.9 +``` + +- The **Name** tag indicates the software name, the **Version** tag indicates the version number, and the **Release** tag indicates the release number. +- The **Summary** tag is a brief description. The first letter of the tag must be capitalized to prevent the rpmlint tool \(packaging check tool\) from generating alarms. +- The **License** tag describes the protocol version of the software package. The packager is responsible for checking the license status of the software, which can be implemented by checking the source code or license file or communicating with the author. +- The **Group** tag is used to classify software packages by **/usr/share/doc/rpm/GROUPS**. Currently, this tag has been discarded. However, the VIM template still has this tag. You can delete it. However, adding this tag does not affect the system. The **%changelog** tag should contain the log of changes made for each release, especially the description of the upstream security/vulnerability patches. The **%changelog** tag should contain the version string to avoid the rpmlint tool from generating alarms. +- If multiple lines are involved, such as %changelog or %description, start from the next line of the instruction and end with a blank line. +- Some unnecessary lines \(such as BuildRequires and Requires\) can be commented out with a number sign \(\#\) at the beginning of the lines. +- The default values of **%prep**, **%build**, **%install**, and **%files** are retained. + +#### Building an RPM Package + +Run the following command in the directory where the .spec file is located to build the source code, binary files, and software packages that contain debugging information: + +```shell +rpmbuild -ba hello.spec +``` + +Run the following command to view the execution result: + +```shell +$ tree ~/rpmbuild/*RPMS + +/home/testUser/rpmbuild/RPMS +└── aarch64 + ├── hello-2.10-1.aarch64.rpm + ├── hello-debuginfo-2.10-1.aarch64.rpm + └── hello-debugsource-2.10-1.aarch64.rpm +/home/testUser/rpmbuild/SRPMS +└── hello-2.10-1.src.rpm +``` + +## Building an RPM Package Using the OBS + +This section describes how to build RPM software packages using the OBS on the web page or with OSC. There are two methods: + +- Modifying an existing software package: Modify the source code of an existing software package and build the modified source code into an RPM software package. +- Adding a software package: A new software source file is developed from scratch, and the newly developed source file is used to build an RPM software package. + +### OBS Overview + +OBS is a general compilation framework based on the openSUSE distribution. It is used to build source code packages into RPM software packages or Linux images. OBS uses the automatic distributed compilation mode and supports the compilation of images and installation packages of multiple Linux OS distributions \(such as openEuler, SUSE, and Debian\) on multiple architecture platforms \(such as x86 and ARM64\). + +OBS consists of the backend and frontend. The backend implements all core functions. The frontend provides web applications and APIs for interaction with the backend. In addition, OBS provides an API command line client OSC, which is developed in an independent repository. + +OBS uses the project organization software package. Basic permission control, related repository, and build targets \(OS and architecture\) can be defined in the project. A project can contain multiple subprojects. Each subproject can be configured independently to complete a task. + +### Building an RPM Software Package Online + +This section describes how to build an RPM software package online on OBS. + +#### Building an Existing Software Package + +> [!NOTE]NOTE + +- If you use OBS for the first time, register an individual account on the OBS web page. +- With this method, you must copy the modified code and commit it to the code directory before performing the following operations. The code directory is specified in the **\_service** file. + +To modify the source code of the existing software and build the modified source file into an RPM software package on the OBS web client, perform the following steps: + +1. Log in to OBS at [https://build.openeuler.openatom.cn/](https://build.openeuler.openatom.cn/). +2. Click **All Projects**. The **All Projects** page is displayed. +3. Click the project to be modified. The project details page is displayed. For example, click **openEuler:Mainline**. +4. On the project details page, search for the software package to be modified and click the software package name. The software package details page is displayed. +5. Click **Branch package**. In the displayed dialog box, click **Accept**, as shown in [Figure 1](#fig77646143214). + + **Figure 1** **Branch Confirmation** page + ![](./figures/branch-confirmation-page.png) + +6. Click the **\_service** file to go to the editing page, modify the file content, and click **Save**. An example of the **\_service** file content is as follows. _userCodeURL_ and _userCommitID_ indicate the user code path and commission version number or branch, respectively. + + ```xml + + + git + userCodeURL + userCommitID + + + bz2 + *.tar + + + ``` + + > [!NOTE]NOTE + > Click **Save** to save the **\_service** file. OBS downloads the source code from the specified URL to the software directory of the corresponding OBS project based on the **\_service** file description and replaces the original file. For example, the **kernel** directory of the **openEuler:Mainline** project in the preceding example. + +7. After the files are copied and replaced, OBS automatically starts to build the RPM software package. Wait until the build is complete and view the build status in the status bar on the right. + - **succeeded**: The build is successful. You can click **succeeded** to view the build logs, as shown in [Figure 2](#fig10319114217337). + + **Figure 2** **Succeeded** page + ![](./figures/succeeded-page.png) + + - **failed**: The build failed. Click **failed** to view error logs, locate the fault, and rebuild again. + - **unresolvable**: The build is not performed. The possible cause is that the dependency is missing. + - **disabled**: The build is manually closed or is queuing for build. + - **excluded**: The build is prohibited. The possible cause is that the .spec file is missing or the compilation of the target architecture is prohibited in the .spec file. + +#### Adding a Software Package + +To add a new software package on the OBS web page, perform the following steps: + +1. Log in to the OBS console. +2. Select a project based on the dependency of the new software package. That is, click **All Projects** and select the corresponding project, for example, **openEuler:Mainline**. +3. Click a software package in the project. The software package details page is displayed. +4. Click **Branch package**. On the confirmation page that is displayed, click **Accept**. +5. Click **Delete package** to delete the software package in the new subproject, as shown in [Figure 3](#fig18306181103615). + + **Figure 3** Deleting a software package from a subproject + ![](./figures/deleting-a-software-package-from-a-subproject.png) + + > [!NOTE]NOTE + > The purpose of creating a project by using existing software is to inherit the dependency such as the environment. Therefore, you need to delete these files. + +6. Click **Create Package**. On the page that is displayed, enter the software package name, title, and description, and click **Create** to create a software package, as shown in [Figure 4](#fig6762111693811) and [Figure 5](#fig18351153518389). + + **Figure 4** **Create Package** page + ![](./figures/create-package-page.png) + + **Figure 5** Creating a software package + ![](./figures/creating-a-software-package.png) + +7. Click **Add file** to upload the .spec file and the file to be compiled \(specified in the .spec file\), as shown in [Figure 6](#fig1475845284011). + + **Figure 6** **Add file** page + ![](./figures/add-file-page.png) + +8. After the file is uploaded, OBS automatically starts to build the RPM software package. Wait until the build is complete and view the build status in the status bar on the right. + - **succeeded**: The build is successful. You can click **succeeded** to view the build logs. + - **failed**: The build failed. Click **failed** to view error logs, locate the fault, and rebuild again. + - **unresolvable**: The build is not performed. The possible cause is that the dependency is missing. + - **disabled**: The build is manually closed or is queuing for build. + - **excluded**: The build is prohibited. The possible cause is that the .spec file is missing or the compilation of the target architecture is prohibited in the .spec file. + +#### Obtaining the Software Package + +After the RPM software package is built, perform the following operations to obtain the RPM software package on the web page: + +1. Log in to the OBS console. +2. Click **All Projects** and find the project corresponding to the required software package, for example, **openEuler:Mainline**. +3. Click the name of the required software package in the project. The software package details page is displayed, for example, the **kernel** page in the preceding example. + +4. Click the **Repositories** tab. On the software repository management page that is displayed, click **Enable** in **Publish Flag** to enable the RPM software package download function \(the status changes from ![](./figures/en-us_image_0229243704.png) to ![](./figures/en-us_image_0229243702.png)\), as shown in [Figure 7](#fig17480830144217). + + **Figure 7** **Repositories** page + ![](./figures/repositories-page.png) + +5. Click the project name in the **Repository** column. On the RPM software package download page that is displayed, click **Download** on the right of the RPM software package to download the RPM software package, as shown in [Figure 8](#fig12152145615438). + + **Figure 8** RPM software package download page + ![](./figures/rpm-software-package-download-page.png) + +### Building a Software Package Using OSC + +This section describes how to use the OBS command line tool OSC to create a project and build an RPM software package. + +#### Installing and Configuring the OSC + +##### Prerequisites + +You have obtained the **root** permission, and have configured a repo source for openEuler. + +##### Procedure + +1. Install the OSC command line tool and its dependency as the **root** user. + + ```shell + dnf install osc build + ``` + + > [!NOTE]NOTE + > The compilation of RPM software packages depends on build. + +2. Configure the OSC. + 1. Run the following command to open the **\~/.oscrc** file: + + ```shell + vi ~/.oscrc + ``` + + 2. Add the **user** and **pass** fields to **\~/.oscrc**. The values of _userName_ and _passWord_ are the account and password registered on the OBS website \([https://build.openeuler.openatom.cn/](https://build.openeuler.openatom.cn/)\). + + ```text + [general] + apiurl = https://build.openeuler.openatom.cn/ + [https://build.openeuler.openatom.cn/] + user=userName + pass=passWord + ``` + +#### Building an Existing Software Package + +**Creating a Project** + +1. You can copy an existing project to create a subproject of your own. For example, to copy the **zlib** software package in the **openEuler:Mainline** project to the new branch, run the following command: + + ```shell + osc branch openEuler:Mainline zlib + ``` + + If the following information is displayed, a new branch project **home:testUser:branches:openEuler:Mainline** is created for user **testUser**. + + ```console + A working copy of the branched package can be checked out with: + osc co home:testUser:branches:openEuler:Mainline/zlib + ``` + +2. Download the configuration file \(for example, **\_service**\) of the software package to be modified to the local directory. In the preceding command, _testUser_ indicates the account name configured in the **\~/.oscrc** configuration file. Change it based on the actual requirements. + + ```shell + osc co home:testUser:branches:openEuler:Mainline/zlib + ``` + + Information similar to the following is displayed: + + ```console + A home:testUser:branches:openEuler:Mainline + A home:testUser:branches:openEuler:Mainline/zlib + A home:testUser:branches:openEuler:Mainline/zlib/_service + ``` + +3. Go to the local subproject directory and synchronize the remote code of the software package to the local host. + + ```shell + cd home:testUser:branches:openEuler:Mainline/zlib + osc up -S + ``` + + Information similar to the following is displayed: + + ```console + A _service:tar_scm_kernel_repo:0001-Neon-Optimized-hash-chain-rebase.patch + A _service:tar_scm_kernel_repo:0002-Porting-optimized-longest_match.patch + A _service:tar_scm_kernel_repo:0003-arm64-specific-build-patch.patch + A _service:tar_scm_kernel_repo:zlib-1.2.11-optimized-s390.patch + A _service:tar_scm_kernel_repo:zlib-1.2.11.tar.xz + A _service:tar_scm_kernel_repo:zlib-1.2.5-minizip-fixuncrypt.patch + A _service:tar_scm_kernel_repo:zlib.spec + ``` + +**Building an RPM Package** + +1. Rename the source file and add the renamed source file to the temporary storage of OBS. + + ```shell + rm -f _service;for file in `ls | grep -v .osc`;do new_file=${file##*:};mv $file $new_file;done + osc addremove * + ``` + +2. Modify the source code and .spec file, and run the following command to update the file. + + ```shell + osc up + ``` + +3. Synchronize all modifications of the corresponding software package to the OBS server. The following is an example of command. The information after the **-m** parameter indicates the submmission record. + + ```shell + osc ci -m "commit log" + ``` + +4. Run the following command to obtain the repository name and architecture of the current project: + + ```shell + osc repos home:testUser:branches:openEuler:Mainline + ``` + +5. After the modification is committed, OBS automatically compiles the software package. You can run the following command to view the compilation logs of the corresponding repository. In the command, _standard\_aarch64_ and _aarch64_ indicate the repository name and architecture obtained in the command output. + + ```shell + osc buildlog standard_aarch64 aarch64 + ``` + + > [!NOTE]NOTE + > You can also open the created project on the web client to view the build logs. + +#### Adding a Software Package + +To use the OSC tool of OBS to add a new software package, perform the following steps: + +**Creating a Project** + +1. Create a project based on the dependency of the new software package and a proper project. For example, to create a project based on **zlib** of the **openEuler:Mainline** project, run the following command \(**zlib** is any software package in the project\): + + ```shell + osc branch openEuler:Mainline zlib + ``` + +2. Delete unnecessary software packages added during project creation. For example, to delete the **zlib** software package, run the following command: + + ```shell + cd home:testUser:branches:openEuler:Mainline + osc rm zlib + osc commit -m "commit log" + ``` + +3. Create a software package in your own project. For example, to add the **my-first-obs-package** software package, run the following command: + + ```shell + mkdir my-first-obs-package + cd my-first-obs-package + ``` + +**Building an RPM Package** + +1. Add the prepared source file and .spec file to the software package directory. +2. Modify the source code and .spec file, and upload all files of the corresponding software package to the OBS server. The following is a command example. The information after the **-m** parameter is the commission record. + + ```shell + cd home:testUser:branches:openEuler:Mainline + osc add my-first-obs-package + osc ci -m "commit log" + ``` + +3. Run the following command to obtain the repository name and architecture of the current project: + + ```shell + osc repos home:testUser:branches:openEuler:Mainline + ``` + +4. After the modification is committed, OBS automatically compiles the software package. You can run the following command to view the compilation logs of the corresponding repository. In the command, _standard\_aarch64_ and _aarch64_ indicate the repository name and architecture obtained in the command output. + + ```shell + cd home:testUser:branches:openEuler:Mainline/my-first-obs-package + osc buildlog standard_aarch64 aarch64 + ``` + + > [!NOTE]NOTE + > You can also open the created project on the web client to view the build logs. + +#### Obtaining the Software Package + +After the RPM software package is built, run the following command to obtain the RPM software package using the OSC: + +```shell +osc getbinaries home:testUser:branches:openEuler:Mainline my-first-obs-package standard_aarch64 aarch64 +``` + +The parameters in the command are described as follows. You can modify the parameters according to the actual situation. + +- _home:testUser:branches:openEuler:Mainline_: name of the project to which the software package belongs. +- _my-first-obs-package_: name of the software package. +- _standard\_aarch64_: repository name. +- _aarch64_: repository architecture name. + +> [!NOTE]NOTE +> You can also obtain the software package built using OSC from the web page. For details, see [Obtaining the Software Package](#obtaining-the-software-package). diff --git a/docs/en/server/development/application_dev/common-issues-and-solutions.md b/docs/en/server/development/application_dev/common-issues-and-solutions.md new file mode 100644 index 0000000000000000000000000000000000000000..9a409dd841557496c2e9132954a2414af79b02c8 --- /dev/null +++ b/docs/en/server/development/application_dev/common-issues-and-solutions.md @@ -0,0 +1,19 @@ +# Common Issues and Solutions + +## Issue 1: Self-compilation of Some Applications Depending on the java-devel Package Fails + +### Symptom + +The self-compilation of some applications that depend on java-devel fails when the rpmbuild command is executed. + +### Cause Analysis + +To provide OpenJDK features that are updated and compatible with Java applications, the openEuler provides OpenJDK of multiple versions, such as OpenJDK 1.8.0 and OpenJDK 11. The compilation of some applications depends on the java-devel package. When the java-devel package is installed, the system installs java-11-openjdk of a later version by default. As a result, the compilation of these applications fails. + +### Solution + +You need to run the following command to install java-1.8.0-openjdk and then run the `rpmbuild` command to perform self-compilation: + +```shell +# yum install java-1.8.0-openjdk +``` diff --git a/docs/en/server/development/application_dev/figures/add-file-page.png b/docs/en/server/development/application_dev/figures/add-file-page.png new file mode 100644 index 0000000000000000000000000000000000000000..83f0bfaeeb9227bcbb863a93ab8d3535e2b2bc1d Binary files /dev/null and b/docs/en/server/development/application_dev/figures/add-file-page.png differ diff --git a/docs/en/server/development/application_dev/figures/branch-confirmation-page.png b/docs/en/server/development/application_dev/figures/branch-confirmation-page.png new file mode 100644 index 0000000000000000000000000000000000000000..e66cbcd22217b74785381b85128ea61895194882 Binary files /dev/null and b/docs/en/server/development/application_dev/figures/branch-confirmation-page.png differ diff --git a/docs/en/server/development/application_dev/figures/create-package-page.png b/docs/en/server/development/application_dev/figures/create-package-page.png new file mode 100644 index 0000000000000000000000000000000000000000..36ea525856d428b6f88a338202e7cb59b2204fc0 Binary files /dev/null and b/docs/en/server/development/application_dev/figures/create-package-page.png differ diff --git a/docs/en/server/development/application_dev/figures/creating-a-software-package.png b/docs/en/server/development/application_dev/figures/creating-a-software-package.png new file mode 100644 index 0000000000000000000000000000000000000000..f983809e8288f3c2ba7e951b60a3ca3a0f18775a Binary files /dev/null and b/docs/en/server/development/application_dev/figures/creating-a-software-package.png differ diff --git a/docs/en/server/development/application_dev/figures/deleting-a-software-package-from-a-subproject.png b/docs/en/server/development/application_dev/figures/deleting-a-software-package-from-a-subproject.png new file mode 100644 index 0000000000000000000000000000000000000000..a365cd1f46bfb8bec094b79477c0168861a5193b Binary files /dev/null and b/docs/en/server/development/application_dev/figures/deleting-a-software-package-from-a-subproject.png differ diff --git a/docs/en/server/development/application_dev/figures/en-us_image_0229243671.png b/docs/en/server/development/application_dev/figures/en-us_image_0229243671.png new file mode 100644 index 0000000000000000000000000000000000000000..ad5ed3f7beeb01e6a48707c4806606b41d687e22 Binary files /dev/null and b/docs/en/server/development/application_dev/figures/en-us_image_0229243671.png differ diff --git a/docs/en/server/development/application_dev/figures/en-us_image_0229243702.png b/docs/en/server/development/application_dev/figures/en-us_image_0229243702.png new file mode 100644 index 0000000000000000000000000000000000000000..96096879d161f04750a332e5c749a834c49d3173 Binary files /dev/null and b/docs/en/server/development/application_dev/figures/en-us_image_0229243702.png differ diff --git a/docs/en/server/development/application_dev/figures/en-us_image_0229243704.png b/docs/en/server/development/application_dev/figures/en-us_image_0229243704.png new file mode 100644 index 0000000000000000000000000000000000000000..267bc9508f3a065b5b40c367e745f0d8c3ddb5fa Binary files /dev/null and b/docs/en/server/development/application_dev/figures/en-us_image_0229243704.png differ diff --git a/docs/en/server/development/application_dev/figures/en-us_image_0229243712.png b/docs/en/server/development/application_dev/figures/en-us_image_0229243712.png new file mode 100644 index 0000000000000000000000000000000000000000..62ef0decdf6f1e591059904001d712a54f727e68 Binary files /dev/null and b/docs/en/server/development/application_dev/figures/en-us_image_0229243712.png differ diff --git a/docs/en/server/development/application_dev/figures/repositories-page.png b/docs/en/server/development/application_dev/figures/repositories-page.png new file mode 100644 index 0000000000000000000000000000000000000000..b7c04eedf9dd32cf4a9d024a05f5c8b294c76934 Binary files /dev/null and b/docs/en/server/development/application_dev/figures/repositories-page.png differ diff --git a/docs/en/server/development/application_dev/figures/rpm-software-package-download-page.png b/docs/en/server/development/application_dev/figures/rpm-software-package-download-page.png new file mode 100644 index 0000000000000000000000000000000000000000..9f32d6c16d344df6951fc4e6aa027d02dfb9ccb5 Binary files /dev/null and b/docs/en/server/development/application_dev/figures/rpm-software-package-download-page.png differ diff --git a/docs/en/server/development/application_dev/figures/succeeded-page.png b/docs/en/server/development/application_dev/figures/succeeded-page.png new file mode 100644 index 0000000000000000000000000000000000000000..3f10cd1db8bdc9be1ab8b660ef93e8a481c2d6b8 Binary files /dev/null and b/docs/en/server/development/application_dev/figures/succeeded-page.png differ diff --git a/docs/en/server/development/application_dev/preparations-for-development-environment.md b/docs/en/server/development/application_dev/preparations-for-development-environment.md new file mode 100644 index 0000000000000000000000000000000000000000..aa463ad12da5a37b59fe852675a50748da81cf7a --- /dev/null +++ b/docs/en/server/development/application_dev/preparations-for-development-environment.md @@ -0,0 +1,328 @@ +# Preparations for Development Environment + +## Environment Requirements + +- If physical machines (PMs) are used, the minimum hardware requirements of the development environment are described in Table 1. + + **Table 1** Minimum hardware specifications + + | Component | Minimum Hardware Specification | Description | + | ------------ | -------------------------------------------------------------- | ------------------------------------------------------------- | + | Architecture | - AArch64
- x86_64 | - 64-bit Arm architecture
- 64-bit Intel x86 architecture | + | CPU | - Huawei Kunpeng 920 series
- Intel Xeon processor | - | + | Memory | ≥ 4 GB (8 GB or higher recommended for better user experience) | - | + | Hard disk | ≥ 120 GB (for better user experience) | IDE, SATA, SAS interfaces are supported. | + +- If virtual machines (VMs) are used, the minimum virtualization space required for the development environment is described in Table 2. + + **Table 2** Minimum virtualization space + + | Component | Minimum Virtualization Space | Description | + | ------------ | ----------------------------------------------------------------- | ----------- | + | Architecture | - AArch64
- x86_64 | - | + | CPU | Two CPUs | - | + | Memory | ≥ 4 GB (8 GB or higher recommended for better user experience) | - | + | Hard disk | ≥ 32 GB (120 GB or higher recommended for better user experience) | - | + +### OS Requirements + +The openEuler OS is required. + +For details about how to install the openEuler OS, see the _Installation Guide_. On the **SOFTWARE SELECTION** page, select **Development Tools** in the **Add-Ons for Selected Environment** area. + +## Configuring the openEuler Yum Source + +Configure an online Yum source using the online openEuler repo source. Alternatively, configure a local Yum source by mounting an ISO file and creating a local openEuler repo source. + +### Configuring an Online Yum Source by Obtaining the Online openEuler Repo Source + +> [!NOTE]NOTE +> openEuler provides multiple repo sources for users online. For details about the repo sources, see [OS Installation](../../releasenotes/releasenotes/os-installation.md). This section uses the OS repo source file of the AArch64 architecture as an example. + +1. Go to the yum source directory and check the .repo configuration file. + + ```shell + $ cd /etc/yum.repos.d + $ ls + openEuler-xxx.repo + ``` + +2. Edit the **openEuler-xxx.repo** file as the **root** user. Configure the online openEuler repo source as the yum source. + + ```shell + vi openEuler-xxx.repo + ``` + + Edit the **openEuler-xxx.repo** file as follows: + + ```text + [osrepo] + name=osrepo + baseurl=http://repo.openeuler.org/openEuler-{version}/OS/{arch}/ + enabled=1 + gpgcheck=1 + gpgkey=http://repo.openeuler.org/openEuler-{version}/OS/{arch}/RPM-GPG-KEY-openEuler + ``` + + > [!NOTE]NOTE + + - **repoid** indicates the ID of the software repository. Repoids in all .repo configuration files must be unique. In the example, **repoid** is set to **base**. + - **name** indicates the string that the software repository describes. + - **baseurl** indicates the address of the software repository. + - **enabled** indicates whether to enable the software source repository. The value can be **1** or **0**. The default value is **1**, indicating that the software source repository is enabled. + - **gpgcheck** indicates whether to enable the GNU privacy guard (GPG) to check the validity and security of sources of RPM packages. **1** indicates GPG check is enabled. **0** indicates the GPG check is disabled. If this option is not specified, the GPG check is enabled by default. + - **gpgkey** indicates the public key used to verify the signature. + +### Configuring a Local Yum Source by Mounting an ISO File + +> [!NOTE]NOTE +> openEuler provides multiple ISO release packages. For details about each ISO release package, see [OS Installation](../../releasenotes/releasenotes/os-installation.md). This section does not specify the version and architecture of related files. Choose them based on the actual requirements. + +1. Download the ISO release package. + + Download an ISO image using a cross-platform file transfer tool. + + 1. Visit the [openEuler community](https://www.openeuler.org/en/). + 2. Choose **Downloads** > **Community Editions**. + 3. Locate the target version. Then, click **Download**. The download list is displayed. + 4. The download list includes the following architectures: + + - **x86\_64**: ISO of the x86\_64 architecture. + - **AArch64**: ISO of the AArch64 architecture. + - **ARM32**: ISO for embedded devices. + + 5. Click **AArch64** and **Server**. + 6. Choose the required ISO type and click **Download** to download the openEuler release package to the local host. + 7. Click **SHA256** to copy the checksum. Save the checksum as a local verification file. + 8. Log in to the openEuler OS and create a directory for storing the release package and verification file, for example, **~/iso**. + + ```shell + mkdir ~/iso + ``` + + 9. Use a cross-platform file transfer tool (such as WinSCP) to upload the local openEuler release package and verification file to the openEuler OS. + + Run the **wget** command to download the ISO image. + + 1. Visit the [openEuler community](https://www.openeuler.org/en/). + 2. Choose **Downloads** > **Community Editions**. + 3. Locate the target version. Then, click **Download**. The download list is displayed. + 4. The download list includes the following architectures: + + - **x86\_64**: ISO of the x86\_64 architecture. + - **AArch64**: ISO of the AArch64 architecture. + - **ARM32**: ISO for embedded devices. + + 5. Click **AArch64**. + 6. Click **Server**. + 7. Choose the required ISO type, right-click **Download**, and copy the link address. + 8. Right-click **SHA256** and copy the link address. + 9. Log in to the openEuler OS, create a directory for storing the release package and verification file, for example, **~/iso**. Then switch to the directory. + + ```shell + mkdir ~/iso + cd ~/iso + ``` + + 10. Run the **wget** command to remotely download the release package and verification file. In the command, replace **ipaddriso** with the address copied in steps 7. + + ```shell + wget ipaddriso + ``` + +2. Release Package Integrity Check + + 1. Calculate the SHA256 verification value of the openEuler release package. + + ```shell + sha256sum openEuler-xxx-dvd.iso + ``` + + After the command is run, the verification value is displayed. + + 2. Check whether the calculated value is the same as that of the saved SHA256 value. + + If the verification values are consistent, the .iso file is not damaged. If they are inconsistent, the file is damaged and you need to obtain the file again. + +3. Mount the ISO file and configure it as a repo source. + + ```shell + mount /home/iso/openEuler-xxx-dvd.iso /mnt/ + ``` + + ```text + . + │── boot.catalog + │── docs + │── EFI + │── images + │── Packages + │── repodata + │── TRANS.TBL + └── RPM-GPG-KEY-openEuler + ``` + + In the preceding directory, **Packages** indicates the directory where the RPM package is stored, **repodata** indicates the directory where the repo source metadata is stored, and **RPM-GPG-KEY-openEuler** indicates the public key for signing openEuler. + +4. Go to the yum source directory and check the .repo configuration file in the directory. + + ```shell + $ cd /etc/yum.repos.d + $ ls + openEuler-xxx.repo + ``` + +5. Edit the **openEuler-xxx.repo** file as the **root** user. Configure the local openEuler repo source created in step [3](#li6236932222) as the yum source. + + ```shell + vi openEuler-xxx.repo + ``` + + Edit the **openEuler-xxx.repo** file as follows: + + ```text + [localosrepo] + name=localosrepo + baseurl=file:///mnt + enabled=1 + gpgcheck=1 + gpgkey=file:///mnt/RPM-GPG-KEY-openEuler + ``` + +## Installing the Software Packages + +The software required varies in different development environments. However, the installation methods are the same. This section describes how to install common software packages (such as JDK and rpm-build). Some development software, such as GCC and GNU make, is provided by the openEuler OS by default. + +### Installing the JDK Software Package + +1. Run the `dnf list installed | grep jdk` command to check whether the JDK software is installed. + + Check the command output. If the command output contains "jdk", the JDK has been installed. If no such information is displayed, the software is not installed. + +2. Run `dnf clean all` to clear the cache. + +3. Run `dnf makecache` to create the cache. + +4. Run `dnf search jdk | grep jdk` to query the JDK software package that can be installed. + + View the command output and install the **java-{version}-openjdk-devel.aarch64** software package. + +5. Install the JDK software package as the **root** user. Run `dnf install java-{version}-openjdk-devel.aarch64`. + +6. Query information about the JDK software by running `java -version`. + + Check the command output. If the command output contains "openjdk version", the JDK has been correctly installed. + +### Installing the rpm-build Software Package + +1. Run the `dnf list installed | grep rpm-build` command to check whether the rpm-build software is installed. + + Check the command output. If the command output contains "rpm-build", the software has been installed. If no such information is displayed, the software is not installed. + +2. Run `dnf clean all` to clear the cache. + +3. Run `dnf makecache` to create the cache. + +4. Install the rpm-build package as the **root** user. Run `dnf install rpm-build`. + +5. Query the rpm-build software version. Run `rpmbuild --version`. + +## Using the IDE for Java Development + +For small-sized Java applications, you can directly use JDK to compile them to run Java applications. However, for medium- and large-sized Java applications, this method does not meet the development requirements. You can perform the following steps to install and use the IDE to facilitate Java development on the openEuler OS. + +### Overview + +IntelliJ IDEA is a popular Java IDE. You can download the community edition of IntelliJ IDEA for free. Currently, openEuler supports Java development using IntelliJ IDEA, improving the work efficiency of developers. + +### Logging In to the Server Using MobaXterm + +MobaXterm is an excellent SSH client. It has an X Server and can easily solve remote GUI display problems. + +You need to download, install, and start MobaXterm in advance, and then log in to your server in SSH mode to perform the following operations. + +### Setting the JDK Environment + +Before setting JAVA\_HOME, you need to find the installation path of the JDK. You are supported to have installed the JDK. If you have not installed the JDK, install it by referring to the preceding section "Installing the JDK Software Package". + +Run the following commands to view the Java path: + +```shell +$ which java +/usr/bin/java +``` + +Run the following commands to check the directory to which the soft link points: + +```shell +$ ls -la /usr/bin/java +lrwxrwxrwx. 1 root root 22 Mar 6 20:28 /usr/bin/java -> /etc/alternatives/java +$ ls -la /etc/alternatives/java +lrwxrwxrwx. 1 root root 83 Mar 6 20:28 /etc/alternatives/java -> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-1.h2.aarch64/jre/bin/java +``` + +**/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-1.h2.aarch64** indicates the actual JDK path. Run the following commands to set **JAVA\_HOME** and **PATH**: + +```shell +export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-1.h2.aarch64 +export PATH=$JAVA_HOME/bin:$PATH +``` + +### Downloading and Installing the GTK Library + +Run the following command: + +```shell +dnf list installed | grep gtk +``` + +If **gtk2** or **gtk3** is displayed, the GTK library has been installed. In this case, skip this step. Otherwise, run the following command as the **root** user to automatically download and install the GTK library: + +```shell +dnf -y install gtk2 libXtst libXrender xauth +``` + +### Setting X11 Forwarding + +Switch to the SSHD configuration directory. + +```shell +cd ~/.ssh +``` + +If the directory does not exist, run the following command to create it and then switch to it: + +```shell +mkdir ~/.ssh +``` + +Edit the configuration file in the **.ssh** directory and save the file. + +1. Run the **vim** command to open the configuration file. + + ```shell + vim config + ``` + +2. Add the following content to the end of the file and save the file: + + ```text + Host * + ForwardAgent yes + ForwardX11 yes + ``` + +### Downloading and Running IntelliJ IDEA + +After the preceding environment configuration is complete, you can download and run IntelliJ IDEA. The latest version of IntelliJ IDEA is incompatible with openEuler in some functions. You are advised to click [here](https://www.jetbrains.com/idea/download/other.html) to download the Linux package of the 2018 version. Move the downloaded package to the directory where you want to install the software and decompress the package. + +```shell +tar xf ideaIC-2018.3.tar.gz +``` + +After the decompression is complete, switch to the IntelliJ IDEA directory and run IntelliJ IDEA. + +```shell +cd ./idea-IC-183.4284.148 +bin/idea.sh & +``` diff --git a/docs/en/server/development/application_dev/public_sys-resources/icon-note.gif b/docs/en/server/development/application_dev/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/server/development/application_dev/public_sys-resources/icon-note.gif differ diff --git a/docs/en/server/development/application_dev/using-clang-for-compilation.md b/docs/en/server/development/application_dev/using-clang-for-compilation.md new file mode 100644 index 0000000000000000000000000000000000000000..e1c79c9f6bd3616d2f351192d63b457279d742fc --- /dev/null +++ b/docs/en/server/development/application_dev/using-clang-for-compilation.md @@ -0,0 +1,100 @@ +# Using LLVM/Clang for Compilation + +This chapter describes the basic knowledge of LLVM/Clang compilation and provides examples for demonstration. For more information about how to use Clang, run the **clang --help** command. + +- [Using LLVM/Clang for Compilation](#using-llvmclang-for-compilation) + - [Overview](#overview) + - [LLVM/Clang Installation](#llvmclang-installation) + - [Coexistence of Multiple Versions](#coexistence-of-multiple-versions) + - [Example](#example) + +## Overview + +LLVM is a compiler that covers multiple programming languages and target processors. It uses Clang as the compiler and driver of C and C++. Clang can not only compile C and C++ programs into the LLVM intermediate representations (IRs), but also invoke all LLVM optimization passes for code generation until the final binary file is generated. + +## LLVM/Clang Installation + +Install the Clang and LLVM software packages using the Yum source in the openEuler OS. llvm12 is installed by default. + +```shell +yum install llvm +yum install clang +yum install lld // The openEuler Yum source does not contain LLD12. +``` + +Check whether the installation is successful. + +```shell +clang -v +``` + +If the command output contains the Clang version information, the installation is successful. + +## Coexistence of Multiple Versions + +Configure the openEuler LLVM/Clang multi-version support as follows: + +```text +Yum package name: +llvm{-*} +clang{-*} +lld{-*} +bolt{-*} +Example: +clang15 +llvm15-devel + +Installation path: +/usr/lib64/llvm +Example: +/usr/lib64/llvm15 + +Executable files with the - suffix are installed in the /usr/bin directory. +Example: +/usr/bin/clang-15 +/usr/bin/lld-15 +``` + +Currently, the following LLVM/Clang versions are supported: + +```shell +llvm //By default, llvm12 is installed. +llvm-15 +``` + +Install other versions using Yum. + +```shell +yum install llvm15 +yum install clang15 +yum install lld15 +``` + +Check whether the installation is successful. + +```shell +clang-15 -v +``` + +## Example + +Compile and run C and C++ programs. + +```shell +clang [command line flags] hello.c -o hello.o +./hello.o +``` + +```shell +clang++ [command line flags] hello.cpp -o hello.o +./hello.o +``` + +Specify the LLD of LLVM as the linker. If it is not specified, the default LLD is used. + +```shell +clang [command line flags] -fuse-ld=lld hello.c -o hello.o +./hello.o +``` + +For more information, see the [LLVM User Guides](https://llvm.org/docs/UserGuides.html). diff --git a/docs/en/server/development/application_dev/using-gcc-for-compilation.md b/docs/en/server/development/application_dev/using-gcc-for-compilation.md new file mode 100644 index 0000000000000000000000000000000000000000..95901a5484433b71bfece96917bce0663ad1cad5 --- /dev/null +++ b/docs/en/server/development/application_dev/using-gcc-for-compilation.md @@ -0,0 +1,472 @@ +# Using GCC for Compilation + +This chapter describes the basic knowledge of GCC compilation and provides examples for demonstration. For more information about GCC, run the **man gcc** command. + + +- [Using GCC for Compilation](#using-gcc-for-compilation) + - [Overview](#overview) + - [Basics](#basics) + - [File Type](#file-type) + - [Compilation Process](#compilation-process) + - [Compilation Options](#compilation-options) + - [Multi-file Compilation](#multi-file-compilation) + - [Libraries](#libraries) + - [Dynamic Link Library](#dynamic-link-library) + - [Static Link Library](#static-link-library) + - [Examples](#examples) + - [Example for Using GCC to Compile C Programs](#example-for-using-gcc-to-compile-c-programs) + - [Example for Creating and Using a DLL Using GCC](#example-for-creating-and-using-a-dll-using-gcc) + - [Example for Creating and Using an SLL Using GCC](#example-for-creating-and-using-an-sll-using-gcc) + + + +## Overview + +The GNU Compiler Collection \(GCC\) is a powerful and high-performance multi-platform compiler developed by GNU. The GCC compiler can compile and link source programs, assemblers, and target programs of C and C++ into executable files. By default, the GCC software package is installed in the openEuler OS. + +## Basics + +### File Type + +For any given input file, the file type determines which compilation to perform. Table 1 describes the common GCC file types. + +**Table 1** Common GCC file types + +| Extension (Suffix) | Description | +| --- | --- | +| .c | C source code file. | +| .C, .cc, or .cxx | C++ source code file. | +| .m | Objective-C source code file. | +| .s | Assembly language source code file. | +| .i | Preprocessed C source code file. | +| .ii | Preprocessed C++ source code file. | +| .S | Pre-processed assembly language source code file. | +| .h | Header file contained in the program. | +| .o | Target file after compilation. | +| .so | Dynamic link library, which is a special target file. | +| .a | Static link library. | +| .out | Executable files, which do not have a fixed suffix. The system distinguishes executable files from inexecutable files based on file attributes. If the name of an executable file is not given, GCC generates a file named **a.out**. | + +### Compilation Process + +Using GCC to generate executable files from source code files requires preprocessing, compilation, assembly, and linking. + +1. Preprocessing: Preprocess the source program \(such as a **.c** file\) to generate an **.i** file. +2. Compilation: Compile the preprocessed **.i** file into an assembly language to generate an **.s** file. +3. Assemble: Assemble the assembly language file to generate the target file **.o**. +4. Linking: Link the **.o** files of each module to generate an executable program file. + +The **.i**, **.s**, and **.o** files are intermediate or temporary files. If the GCC is used to compile programs in C language at a time, these files will be deleted. + +### Compilation Options + +GCC compilation command format: **gcc** \[_options_\] \[_filenames_\] + +In the preceding information: + +_options_ : compilation options. + +_filenames_ : file name. + +GCC is a powerful compiler. It has many _options_, but most of them are not commonly used. Table 2 describes the common _options_. + +**Table 2** Common GCC compilation options + +| *options* Value | Description | Example | +| --- | --- | --- | +| -c | Compiles and assembles specified source files to generate target files without linking them. It is usually used to compile subprogram files. | # Use the **-c** option to compile the source files **test1.c** and **test2.c**.

gcc -c test1.c test2.c | +| -S | Compiles the specified source file to generate an assembly language file with the .s suffix but without assembling it. | # Use the compiler to preprocess **circle.c**, translate it into assembly language, and store the result in circle.s.

gcc -S circle.c | +| -E | Preprocesses specified source files without compiling them.

By default, the output of the preprocessor is imported to a standard output stream, such as a display. You can use the **-o** option to import it to an output file. | # Export the preprocessing result to the **circle.i** file.

gcc -E circle.c -o circle.i | +| -o *file* | Generates a specified output *file* when an executable file is generated. The name must be different from that of the source file. If this option is not given, GCC generates the preset executable file **a.out**. | # Use the source file as the input file and the executable file as the output file. That is, compile the entire program.

gcc main.c func.c -o app.out | +| -g | Contains standard debugging information in executable programs. | - | +| -L *library\_path* | Adds the *library\_path* to the library file search path list. | - | +| -I *library* | Searches for the specified function *library* during linking.

When GCC is used to compile and link programs, GCC links **libc.a** or **libc.so** by default. However, other libraries (such as non-standard libraries and third-party libraries) need to be manually added. | # Use the **-l** option to link the math library.

gcc main.c -o main.out -lm

NOTE:
The file name of the math library is **libm.a**. The prefix lib and suffix .a are standard, and **m** is the basic name. GCC automatically adds these prefixes and suffixes to the basic name following the **-l** option. In this example, the basic name is **m**. | +| -I *head\_path* | Adds the *head\_path* to the search path list of the header file. | - | +| -static | Performs static compilation and links static libraries. Do not link dynamic libraries. | - | +| -shared | Default option, which can be omitted.
  • A dynamic library file can be generated.
  • During dynamic compilation, the dynamic library is preferentially linked. The static library with the same name is linked only when there is no dynamic library.
| - | +| -fPIC (or -fpic) | Generates location-independent target code that uses a relative address. Generally, the **-static** option is used to generate a dynamic library file from the PIC target file. | - | + +### Multi-file Compilation + +There are two methods provided for compiling multiple source files. + +- Multiple source files are compiled at the same time. All files need to be recompiled during compilation. + + Example: Compile **test1.c** and **test2.c** and link them to the executable file **test**. + + ```shell + gcc test1.c test2.c -o test + ``` + +- Compile each source file, and then link the target files generated after compilation. During compilation, only modified files need to be recompiled. + + For example, compile **test1.c** and **test2.c**, and link the target files **test1.o** and **test2.o** to the executable file **test**. + + ```shell + gcc -c test1.c + gcc -c test2.c + gcc test1.o test2.o -o test + ``` + +## Libraries + +A library is mature and reusable code that has been written for use. Each program depends on many basic underlying libraries. + +The library file name is prefixed with lib and suffixed with .so \(dynamic library\) or .a \(static library\). The middle part is the user-defined library file name, for example, libfoo.so or libfoo.a. Because all library files comply with the same specifications, the **lib** prefix can be omitted when the **-l** option specifies the name of the linked library file. That is, when GCC processes **-lfoo**, the library file **libfoo.so** or **libfoo.a** is automatically linked. When creating a library, you must specify the full file name **libfoo.so** or **libfoo.a**. + +Libraries are classified into static libraries and dynamic libraries based on the linking time. The static library links and packs the target file .o generated by assembly and the referenced library into an executable file in the linking phase. The dynamic library is not linked to the target code when the program is compiled, but is loaded when the program is run. The differences are as follows: + +- The resource usage is different. + + The static library is a part of the generated executable file, while the dynamic library is a separate file. Therefore, the sizes and occupied disk space of the executable files of the static library and dynamic library are different, which leads to different resource usage. + +- The scalability and compatibility are different. + + If the implementation of a function in the static library changes, the executable file must be recompiled. For the executable file generated by dynamic linking, only the dynamic library needs to be updated, and the executable file does not need to be recompiled. + +- The dependency is different. + + The executable file of the static library can run without depending on any other contents, while the executable file of the dynamic library must depend on the dynamic library. Therefore, the static library is convenient to migrate. + +- The loading speeds are different. + + Static libraries are linked together with executable files, while dynamic libraries are linked only when they are loaded or run. Therefore, for the same program, static linking is faster than dynamic linking. + +### Dynamic Link Library + +You can use the **-shared** and **-fPIC** options to create a dynamic link library \(DLL\) with the source file, assembly file, or target file. The **-fPIC** option is used in the compilation phase. This option is used when the target file is generated, so as to generate location-independent code. + +Example 1: Generate a DLL from the source file. + +```shell +gcc -fPIC -shared test.c -o libtest.so +``` + +Example 2: Generate a DLL from the target file. + +```shell +gcc -fPIC -c test.c -o test.o +gcc -shared test.o -o libtest.so +``` + +To link a DLL to an executable file, you need to list the name of the DLL in the command line. + +Example: Compile **main.c** and **libtest.so** into **app.out**. When **app.out** is running, the link library **libtest.so** is dynamically loaded. + +```shell +gcc main.c libtest.so -o app.out +``` + +In this mode, the **libtest.so** file in the current directory is used. + +If you choose to search for a DLL, to ensure that the DLL can be linked when the program is running, you must implement by using one of the following methods: + +- Save the DLL to a standard directory, for example, **/usr/lib**. +- Add the DLL path **libraryDIR** to the environment variable **LD\_LIBRARY\_PATH**. + + ```shell + export LD_LIBRARY_PATH=libraryDIR:$LD_LIBRARY_PATH + ``` + + > [!NOTE]NOTE + > **LD\_LIBRARY\_PATH** is an environment variable of the DLL. If the DLL is not in the default directories \(**/lib** and **/usr/lib**\), you need to specify the environment variable **LD\_LIBRARY\_PATH**. + +- Add the DLL path **libraryDIR** to **/etc/ld.so.conf** and run **ldconfig**, or use the DLL path **libraryDIR** as a parameter to run **ldconfig**. + +```shell +gcc main.c -L libraryDIR -ltest -o app.out +export LD_LIBRARY_PATH=libraryDIR:$LD_LIBRARY_PATH +``` + +### Static Link Library + +To create a static link library \(SLL\), you need to compile the source file to the target file, and then run the **ar** command to compress the target file into an SLL. + +Example: Compile and compress source files **test1.c**, **test2.c**, and **test3.c** into an SLL. + +```shell +gcc -c test1.c test2.c test3.c +ar rcs libtest.a test1.o test2.o test3.o +``` + +The **ar** command is a backup compression command. You can compress multiple files into a backup file \(also called an archive file\) or extract member files from the backup file. The most common use of **ar** is to compress the target files into an SLL. + +The format of the **ar** command to compress the target files into an SLL is as follows: + +ar rcs _Sllfilename_ _Targetfilelist_ + +- _Sllfilename_ : Name of the static library file. +- _Targetfilelist_ : Target file list. +- **r**: replaces the existing target file in the library or adds a new target file. +- **c**: creates a library regardless of whether the library exists. +- **s**: creates the index of the target file. The speed can be improved when a large library is created. + +Example: Create a main.c file to use the SLL. + +```shell +gcc main.c -L libraryDIR -ltest -o test.out +``` + +In the preceding command, **libraryDIR** indicates the path of the libtest.a library. + +## Examples + +### Example for Using GCC to Compile C Programs + +1. Run the **cd** command to go to the code directory. The **~/code** directory is used as an example. The command is as follows: + + ```shell + cd ~/code + ``` + +2. Compile the Hello World program and save it as **helloworld.c**. The following uses the Hello World program as an example. The command is as follows: + + ```shell + vi helloworld.c + ``` + + Code example: + + ```c + #include + int main() + { + printf("Hello World!\n"); + return 0; + } + ``` + +3. Run the following command to compile the code in the code directory: + + ```shell + gcc helloworld.c -o helloworld + ``` + + If no error is reported, the execution is successful. + +4. After the compilation is complete, the helloworld file is generated. Check the compilation result. The following is an example: + + ```shell + $ ./helloworld + Hello World! + ``` + +### Example for Creating and Using a DLL Using GCC + +1. Run the **cd** command to go to the code directory. The **~/code** directory is used as an example. Create the **src**, **lib**, and **include** subdirectories in the directory to store the source file, DLL file, and header file, respectively. + + ```shell + cd ~/code + mkdir src lib include + ``` + +2. Run the **cd** command to go to the **~/code/src** directory and create two functions **add.c** and **sub.c** to implement addition and subtraction, respectively. + + ```shell + cd ~/code/src + vi add.c + vi sub.c + ``` + + The following is an example of the **add.c** code: + + ```c + #include "math.h" + int add(int a, int b) + { + return a+b; + } + ``` + + The following is an example of the **sub.c** code: + + ```c + #include "math.h" + int sub(int a, int b) + { + return a-b; + } + ``` + +3. Compile the source files add.c and sub.c into the DLL libmath.so, and store the DLL in the **~/code/lib** directory. + + ```shell + gcc -fPIC -shared add.c sub.c -o ~/code/lib/libmath.so + ``` + +4. Go to the **~/code/include** directory, create a header file **math.h**, and declare the header file of the function. + + ```shell + cd ~/code/include + vi math.h + ``` + + The following is an example of the **math.h** code: + + ```c + #ifndef __MATH_H_ + #define __MATH_H_ + int add(int a, int b); + int sub(int a, int b); + #endif + ``` + +5. Run the **cd** command to go to the **~/code/src** directory and create a **main.c** function that invokes add\(\) and sub\(\). + + ```shell + cd ~/code/src + vi main.c + ``` + + The following is an example of the **math.c** code: + + ```c + #include + #include "math.h" + int main() + { + int a, b; + printf("Please input a and b:\n"); + scanf("%d %d", &a, &b); + printf("The add: %d\n", add(a,b)); + printf("The sub: %d\n", sub(a,b)); + return 0; + } + ``` + +6. Compile **main.c** and **libmath.so** into **math.out**. + + ```shell + gcc main.c -I ~/code/include -L ~/code/lib -lmath -o math.out + ``` + +7. Add the path of the DLL to the environment variable. + + ```shell + export LD_LIBRARY_PATH=~/code/lib:$LD_LIBRARY_PATH + ``` + +8. Run the following command to execute **math.out**: + + ```shell + ./math.out + ``` + + The command output is as follows: + + ```console + Please input a and b: + 9 2 + The add: 11 + The sub: 7 + ``` + +### Example for Creating and Using an SLL Using GCC + +1. Run the **cd** command to go to the code directory. The **~/code** directory is used as an example. Create the **src**, **lib**, and **include** subdirectories in the directory to store the source file, SLL file, and header file respectively. + + ```shell + cd ~/code + mkdir src lib include + ``` + +2. Run the **cd** command to go to the **~/code/src** directory and create two functions **add.c** and **sub.c** to implement addition and subtraction, respectively. + + ```shell + cd ~/code/src + vi add.c + vi sub.c + ``` + + The following is an example of the **add.c** code: + + ```c + #include "math.h" + int add(int a, int b) + { + return a+b; + } + ``` + + The following is an example of the **sub.c** code: + + ```c + #include "math.h" + int sub(int a, int b) + { + return a-b; + } + ``` + +3. Compile the source files **add.c** and **sub.c** into the target files **add.o** and **sub.o**. + + ```shell + gcc -c add.c sub.c + ``` + +4. Run the **ar** command to compress the **add.o** and **sub.o** target files into the SLL **libmath.a** and save the SLL to the **~/code/lib** directory. + + ```shell + ar rcs ~/code/lib/libmath.a add.o sub.o + ``` + +5. Go to the **~/code/include** directory, create a header file **math.h**, and declare the header file of the function. + + ```shell + cd ~/code/include + vi math.h + ``` + + The following is an example of the **math.h** code: + + ```c + #ifndef __MATH_H_ + #define __MATH_H_ + int add(int a, int b); + int sub(int a, int b); + #endif + ``` + +6. Run the **cd** command to go to the **~/code/src** directory and create a **main.c** function that invokes add\(\) and sub\(\). + + ```shell + cd ~/code/src + vi main.c + ``` + + The following is an example of the **math.c** code: + + ```c + #include + #include "math.h" + int main() + { + int a, b; + printf("Please input a and b:\n"); + scanf("%d %d", &a, &b); + printf("The add: %d\n", add(a,b)); + printf("The sub: %d\n", sub(a,b)); + return 0; + } + ``` + +7. Compile **main.c** and **libmath.a** into **math.out**. + + ```shell + gcc main.c -I ~/code/include -L ~/code/lib -lmath -o math.out + ``` + +8. Run the following command to execute **math.out**: + + ```shell + ./math.out + ``` + + The command output is as follows: + + ```console + Please input a and b: + 9 2 + The add: 11 + The sub: 7 + ``` diff --git a/docs/en/server/development/application_dev/using-jdk-for-compilation.md b/docs/en/server/development/application_dev/using-jdk-for-compilation.md new file mode 100644 index 0000000000000000000000000000000000000000..b95faa6f06b55024f2cd64ac3fe835a0cd7dfc65 --- /dev/null +++ b/docs/en/server/development/application_dev/using-jdk-for-compilation.md @@ -0,0 +1,526 @@ +# Using JDK for Compilation + + + +- [Using JDK for Compilation](#using-jdk-for-compilation) + - [Overview](#overview) + - [Basics](#basics) + - [File Type and Tool](#file-type-and-tool) + - [Java Program Generation Process](#java-program-generation-process) + - [Common JDK Options](#common-jdk-options) + - [Class Library](#class-library) + - [Package Declaration](#package-declaration) + - [Package Reference](#package-reference) + - [Examples](#examples) + - [Compiling a Java Program Without a Package](#compiling-a-java-program-without-a-package) + - [Compiling a Java Program with a Package](#compiling-a-java-program-with-a-package) + + + +## Overview + +A Java Development Kit \(JDK\) is a software package required for Java development. It contains the Java Runtime Environment \(JRE\) and compilation and commissioning tools. On the basis of OpenJDK, openEuler optimizes GC, enhances concurrency stability, and enhances security, improving the performance and stability of Java applications on ARM. + +## Basics + +### File Type and Tool + +For any given input file, the file type determines which tool to use for processing. The common file types and tools are described in [Table 1](#table634145764320) and [Table 2](#table103504146433). + +**Table 1** Common JDK file types + + + + + + + + + + + + + + + + +

Extension (Suffix)

+

Description

+

.java

+

Java source code file.

+

.class

+

Java bytecode file, which is intermediate code irrelevant to any specific machine or OS environment. It is a binary file, which is the target code file generated after the Java source file is compiled by the Java compiler.

+

.jar

+

JAR package of Java files.

+
+ +**Table 2** Common JDK tools + + + + + + + + + + + + + + + + +

Name

+

Description

+

java

+

Java running tool, which is used to run .class bytecode files or .jar files.

+

javac

+

Compiles Java source code files into .class bytecode files.

+

jar

+

Creates and manages JAR files.

+
+ +### Java Program Generation Process + +To generate a program from Java source code files and run the program using Java, compilation and run are required. + +1. Compilation: Use the Java compiler \(javac\) to compile Java source code files \(.java files\) into .class bytecode files. +2. Run: Execute the bytecode files on the Java virtual machine \(JVM\). + +### Common JDK Options + +#### Javac Compilation Options + +The command format for javac compilation is as follows: **javac** \[_options_\] \[_sourcefiles_\] \[_classes_\] \[@_argfiles_\] + +In the preceding information: + +_options_: command options. + +_sourcefiles_: one or more source files to be compiled. + +_classes_: one or more classes to be processed as comments. + +@_argfiles_: one or more files that list options and source files. The **-J** option is not allowed in these files. + +Javac is a Java compiler. It has many _options_, but most of them are not commonly used. [Table 3](#table1342946175212) describes the common options values. + +**Table 3** Common javac options + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

options Value

+

Description

+

Example

+

-d path

+

Path for storing the generated class files.

+

By default, the class files generated after compilation are in the same path as the source file. You can use the -d option to export the class files to the specified path.

+

# Use the -d option to export all class files to the bin directory.

+

javac /src/*.java -d /bin

+

-s path

+

Path for storing the generated source files.

+

-

+

-cp path or -classpath path

+

Searches for the class files required for compilation and specifies the location of the class files.

+

# In the Demo, the getLine() method in the GetStringDemo class needs to be invoked. The .class file compiled by the GetStringDemo class is stored in the bin directory.

+

javac -cp bin Demo.java -d bin

+

-verbose

+

Outputs information about the operations being performed by the compiler, such as loaded class information and compiled source file information.

+

# Display information about the operations that are being performed by the compiler.

+

javac -verbose -cp bin Demo.java

+

-source sourceversion

+

Specifies the location of the input source files to be searched for.

+

-

+

-sourcepath path

+

Searches for source files (Java files) required for compilation and specifies the location of the source files to be searched for, for example, JAR, ZIP, or other directories that contain Java files.

+

-

+

-target targetversion

+

Generates class files of a specific JVM version. The value can be 1.1, 1.2, 1.3, 1.4, 1.5 (or 5), 1.6 (or 6), 1.7 (or 7), or 1.8 (or 8). The default value of targetversion is related to sourceversion of the -source option. The options of sourceversion are as follows:

+
  • 1.2, corresponding to target version 1.4
  • 1.3, corresponding to target version 1.4
  • 1.5, 1.6, 1.7, and unspecified, corresponding to target version 1.8
  • For other values, the values of targetversion and sourceversion are the same.
+

-

+
+ +#### Java Running Options + +The Java running format is as follows: + +Running class file: **java** \[_options_\] _classesname_ \[args\] + +Running Java file: **java** \[_options_\] -jar _filename_ \[args\] + +In the preceding information: + +_options_: command options, which are separated by spaces. + +_classname_: name of the running .class file. + +_filename_: name of the running .jar file. + +args: parameters transferred to the main\(\) function. The parameters are separated by spaces. + +Java is a tool for running Java applications. It has many _options_, but most of them are not commonly used. [Table 4](#table371918587238) describes the common options. + +**Table 4** Common Java running options + + + + + + + + + + + + + + + + +

options Value

+

Description

+

Example

+

-cp path or -classpath path

+

Specifies the location of the file to be run and the class path to be used, including the .jar, .zip, and class file directories.

+

If there are multiple paths, separate them with colons (:).

+

-

+

-verbose

+

Outputs information about the operations being performed by the compiler, such as loaded class information and compiled source file information.

+

# Display information about the operations that are being performed by the compiler.

+

java -verbose -cp bin Demo.java

+
+ +#### JAR Options + +The JAR command format is as follows: **jar** \{c | t | x | u\}\[vfm0M\] \[_jarfile_\] \[_manifest_\] \[-C _dir_\] _file_... + +[Table 5](#table3691718114817) describes the parameters in the **jar** command. + +**Table 5** JAR parameter description + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Parameter

+

Description

+

Example

+

c

+

Creates a JAR package.

+

# Compress the hello.class files in the current directory into Hello.jar. The compression process is not displayed. If the Hello.jar files do not exist, create them. Otherwise, clear the directory.

+

jar cf Hello.jar hello.class

+

t

+

Lists the contents of a JAR package.

+

# List the files contained in Hello.jar.

+

jar tf Hello.jar

+

x

+

Decompresses a JAR package.

+

# Decompress Hello.jar to the current directory. No information is displayed.

+

jar xf Hello.jar

+

u

+

Updates the existing JAR package, for example, add files to the JAR package.

+

-

+

v

+

Generates a detailed report and prints it to the standard output.

+

# Compress the hello.class files in the current directory into Hello.jar and display the compression process. If the Hello.jar files do not exist, create them. Otherwise, clear the directory.

+

jar cvf Hello.jar hello.class

+

f

+

Specifies the name of a JAR package. This parameter is mandatory.

+

-

+

m

+

Specifies the manifest file to be contained.

+

-

+

0

+

If this parameter is not set, the generated JAR package is larger but faster than that generated when this parameter is not set.

+

-

+

M

+

If the manifest file of all items is not generated, this parameter will be ignored.

+

# Compress the hello.class files in the current directory into Hello.jar and display the compression process. If the Hello.jar files do not exist, create them. Otherwise, clear the directory. However, the manifest file is not generated when Hello.jar is created.

+

jar cvfM Hello.jar hello.class

+

jarfile

+

JAR package, which is an auxiliary parameter of the f parameter.

+

-

+

manifest

+

Manifest file in .mf format, which is an auxiliary parameter of the m parameter.

+

-

+

-C dir

+

Runs the jar command in the specified dir. This command can be used only with parameters c and t.

+

-

+

file

+

Specifies the file or path list. All files in the file or path (including those in the recursive path) are compressed into the JAR package or the JAR package is decompressed to the path.

+

# Compress all class files in the current directory into Hello.jar and display the compression process. If the Hello.jar files do not exist, create them. Otherwise, clear the directory.

+

jar cvf Hello.jar *.class

+
+ +## Class Library + +The Java class library is implemented as a package, which is a collection of classes and interfaces. The Java compiler generates a bytecode file for each class, and the file name is the same as the class name. Therefore, conflicts may occur between classes with the same name. In the Java language, a group of classes and interfaces are encapsulated in a package. Class namespaces can be effectively managed by package. Classes in different packages do not conflict even if they have the same name. This solves the problem of conflicts between classes with the same name and facilitates the management of a large number of classes and interfaces. It also ensures the security of classes and interfaces. + +In addition to many packages provided by Java, developers can customize packages by collecting compiled classes and interfaces into a package for future use. + +Before using a custom package, you need to declare the package. + +### Package Declaration + +The declaration format of a package is package pkg1\[.pkg2\[.pkg3...\]\]. + +To declare a package, you must create a directory. The subdirectory name must be the same as the package name. Then declare the package at the beginning of the class file that needs to be placed in the package, indicating that all classes of the file belong to the package. The dot \(.\) in the package declaration indicates the directory hierarchy. If the source program file does not contain the package statement, the package is specified as an anonymous package. An anonymous package does not have a path. Generally, Java still stores the classes in the source file in the current working directory \(that is, the directory where the Java source files are stored\). + +The package declaration statement must be added to the beginning of the source program file and cannot be preceded by comments or spaces. If you use the same package declaration statement in different source program files, you can include the classes in different source program files in the same package. + +### Package Reference + +In Java, there are two methods to use the common classes in the package provided by Java or the classes in the custom package. + +- Add the package name before the name of the class to be referenced. + + For example, name.A obj=new name.A \(\) + + **name** indicates the package name, **A** indicates the class name, and **obj** indicates the object. This string indicates that class **A** in the **name** package is used to define an object **obj** in the program. + + Example: Create a test object of the Test class in the example package. + + ```java + example.Test test = new example.Test(); + ``` + +- Use **import** at the beginning of the file to import the classes in the package. + + The format of the **import** statement is import pkg1\[.pkg2\[.pkg3...\]\].\(classname | \*\). + + **pkg1\[.pkg2\[.pkg3...\]\]** indicates the package level, and **classname** indicates the class to be imported. If you want to import multiple classes from a package, you can use the wildcard \(\*\) instead. + + Example: Import the **Test** class in the **example** package. + + ```java + import example.Test; + ``` + + Example: Import the entire **example** package. + + ```java + import example.*; + ``` + +## Examples + +### Compiling a Java Program Without a Package + +1. Run the **cd** command to go to the code directory. The **~/code** directory is used as an example. The command is as follows: + + ```shell + cd ~/code + ``` + +2. Compile the Hello World program and save it as **HelloWorld.java**. The following uses the Hello World program as an example. The command is as follows: + + ```shell + vi HelloWorld.java + ``` + + Code example: + + ```java + public class HelloWorld { + public static void main(String[] args) { + System.out.println("Hello World"); + } + } + ``` + +3. Run the following command to compile the code in the code directory: + + ```shell + javac HelloWorld.java + ``` + + If no error is reported, the execution is successful. + +4. After the compilation is complete, the HelloWorld.class file is generated. You can run the **java** command to view the result. The following is an example: + + ```shell + $ java HelloWorld + Hello World + ``` + +### Compiling a Java Program with a Package + +1. Run the **cd** command to go to the code directory. The **~/code** directory is used as an example. Create the **~/code/Test/my/example**, **~/code/Hello/world/developers**, and **~/code/Hi/openos/openeuler** subdirectories in the directory to store source files. + + ```shell + cd ~/code + mkdir -p Test/my/example + mkdir -p Hello/world/developers + mkdir -p Hi/openos/openeuler + ``` + +2. Run the **cd** command to go to the **~/code/Test/my/example** directory and create **Test.java**. + + ```shell + cd ~/code/Test/my/example + vi Test.java + ``` + + The following is an example of the Test.java code: + + ```java + package my.example; + import world.developers.Hello; + import openos.openeuler.Hi; + public class Test { + public static void main(String[] args) { + Hello me = new Hello(); + me.hello(); + Hi you = new Hi(); + you.hi(); + } + } + ``` + +3. Run the **cd** command to go to the **~/code/Hello/world/developers** directory and create **Hello.java**. + + ```shell + cd ~/code/Hello/world/developers + vi Hello.java + ``` + + The following is an example of the Hello.java code: + + ```java + package world.developers; + public class Hello { + public void hello(){ + System.out.println("Hello, openEuler."); + } + } + ``` + +4. Run the **cd** command to go to the **~/code/Hi/openos/openeuler** directory and create **Hi.java**. + + ```shell + cd ~/code/Hi/openos/openeuler + vi Hi.java + ``` + + The following is an example of the Hi.java code: + + ```java + package openos.openeuler; + public class Hi { + public void hi(){ + System.out.println("Hi, the global developers."); + } + } + ``` + +5. Run the **cd** command to go to the **~/code** directory and use javac to compile the source file. + + ```shell + cd ~/code + javac -classpath Hello:Hi Test/my/example/Test.java + ``` + + After the command is executed, the **Test.class**, **Hello.class**, and **Hi.class** files are generated in the **~/code/Test/my/example**, **~/code/Hello/world/developers**, and **~/code/Hi/openos/openeuler** directories. + +6. Run the **cd** command to go to the **~/code** directory and run the **Test** program using Java. + + ```shell + cd ~/code + java -classpath Test:Hello:Hi my/example/Test + ``` + + The command output is as follows: + + ```text + Hello, openEuler. + Hi, the global developers. + ``` diff --git a/docs/en/server/development/application_dev/using-make-for-compilation.md b/docs/en/server/development/application_dev/using-make-for-compilation.md new file mode 100644 index 0000000000000000000000000000000000000000..a00b04ec6df7879dbcd1c76bfb0f0e6948961060 --- /dev/null +++ b/docs/en/server/development/application_dev/using-make-for-compilation.md @@ -0,0 +1,372 @@ +# Using Make for Compilation + +This chapter describes the basic knowledge of make compilation and provides examples for demonstration. For more information about Make, run the **man make** command. + + +- [Using Make for Compilation](#using-make-for-compilation) + - [Overview](#overview) + - [Basics](#basics) + - [File Type](#file-type) + - [make Work Process](#make-work-process) + - [make Options](#make-options) + - [Makefiles](#makefiles) + - [Makefile Structure](#makefile-structure) + - [Makefile Contents](#makefile-contents) + - [Examples](#examples) + - [Example of Using Makefile to Implement Compilation](#example-of-using-makefile-to-implement-compilation) + + + +## Overview + +The GNU make utility \(usually abbreviated as make\) is a tool for controlling the generation of executable files from source files. make automatically identifies which parts of the complex program have changed and need to be recompiled. Make uses configuration files called makefiles to control how programs are built. + +## Basics + +### File Type + +[Table 1](#table634145764320) describes the file types that may be used in the makefiles file. + +**Table 1** File types + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Extension (Suffix)

+

Description

+

.c

+

C source code file.

+

.C, .cc, or .cxx

+

C++ source code file.

+

.m

+

Objective-C source code file.

+

.s

+

Assembly language source code file.

+

.i

+

Preprocessed C source code file.

+

.ii

+

Preprocessed C++ source code file.

+

.S

+

Pre-processed assembly language source code file.

+

.h

+

Header file contained in the program.

+

.o

+

Target file after compilation.

+

.so

+

Dynamic link library, which is a special target file.

+

.a

+

Static link library.

+

.out

+

Executable files, which do not have a fixed suffix. The system distinguishes executable files from inexecutable files based on file attributes. If the name of an executable file is not given, GCC generates a file named a.out.

+
+ +### make Work Process + +The process of deploying make to generate an executable file from the source code file is described as follows: + +1. The make command reads the Makefiles, including the files named GNUmakefile, makefile, and Makefile in the current directory, the included makefile, and the rule files specified by the **-f**, **--file**, and **--makefile** options. +2. Initialize variables. +3. Derive implicit rules, analyze dependencies, and create a dependency chain. +4. Determine which targets need to be regenerated based on the dependency chain. +5. Run a command to generate the final file. + +### make Options + +make command format: **make** \[_option_\]... \[_target_\]... + +In the preceding command: + +_option_ : parameter option. + +_target_ : target specified in Makefile. + +[Table 2](#table261872312343) describes the common make options. + +**Table 2** Common make options + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

options Value

+

Description

+

-C dir, --directory=dir

+

Specifies dir as the working directory after the make command starts to run.

+

When there are multiple -C options, the final working directory of make is the relative path of the first directory.

+

-d

+

Displays all debugging information during execution of the make command. You can use the -d option to display all the information during the construction of the dependency chain and the reconstruction of the target.

+

-e, --environment-overrides

+

Overwrites the variable definition with the same name in Makefile with the environment variable definition.

+

-f file, --file=file,

+

--makefile=file

+

Specifies the file as the Makefile for the make command.

+

-h, --help

+

Displays help information.

+

-i, --ignore-errors

+

Ignores the errors occurred during the execution.

+

-k, --keep-going

+

When an error occurs during command execution, the make command is not terminated. The make command executes all commands as many as possible until a critical error occurs.

+

-n, --just-print, --dry-run

+

Simulates the execution of commands (including the commands starting with @) in the actual execution sequence. This command is used only to display the execution process and has no actual execution effect.

+

-o file, --old-file=file, --assume-old=file

+

The specified file does not need to be rebuilt even if its dependency has expired, and no target of this dependency file is rebuilt.

+

-p, --print-data-base

+

Before the command is executed, all data of Makefile read by make and the version information of make are printed. If you only need to print the data, run the make -qp command to view the preset rules and variables before the make command is executed. You can run the make -p -f /dev/null command.

+

-r, --no-builtin-rules

+

Ignores the use of embedded implicit rules and the implicit suffix list of all suffix rules.

+

-R, --no-builtin-variables

+

Ignores embedded hidden variables.

+

-s, --silent, --quiet

+

Cancels the printing during the command execution.

+

-S, --no-keep-going, --stop

+

Cancels the -k option. In the recursive make process, the sub-make inherits the upper-layer command line option through the MAKEFLAGS variable. You can use the -S option in the sub-make to cancel the -k option transferred by the upper-layer command, or cancel the -k option in the system environment variable MAKEFLAGS.

+

-t, --touch

+

Updates the timestamp of all target files to the current system time. Prevents make from rebuilding all outdated target files.

+

-v,--version

+

Displays the make version.

+
+ +## Makefiles + +Make is a tool that uses makefiles for compilation, linking, installation, and cleanup, so as to generate executable files and other related files from source code files. Therefore, makefiles describe the compilation and linking rules of the entire project, including which files need to be compiled, which files do not need to be compiled, which files need to be compiled first, which files need to be compiled later, and which files need to be rebuilt. The makefiles automate project compilation. You do not need to manually enter a large number of source files and parameters each time. + +This chapter describes the structure and main contents of makefiles. For more information about makefiles, run the **info make** command. + +### Makefile Structure + +The makefile file structure is as follows: + +_targets_:_prerequisites_ + +_command_ + +or + +_targets_:_prerequisites_;_command_ + +_command_ + +In the preceding information: + +- _targets_ : targets, which can be target files, executable files, or tags. +- _prerequisites_ : dependency files, which are the files or targets required for generating the _targets_. There can be multiple or none of them. +- _command_ : command \(any shell command\) to be executed by make. Multiple commands are allowed, and each command occupies a line. +- Use colons \(:\) to separate the target files from the dependency files. Press **Tab** at the beginning of each command line. + +The makefile file structure indicates the output target, the object on which the output target depends, and the command to be executed for generating the target. + +### Makefile Contents + +A makefile file consists of the following contents: + +- Explicit rule + + Specify the dependency, such as the file to be generated, dependency file, and generated command. + +- Implicit rule + + Specify the rule that is automatically derived by make. The make command supports the automatic derivation function. + +- Variable definition +- File indicator + + The file indicator consists of three parts: + + - Inclusion of other makefiles, for example, include xx.md + - Selective execution, for example, \#ifdef + - Definition of multiple command lines, for example, define...endef. \(define ... endef\) + +- Comment + + The comment starts with a number sign \(\#\). + +## Examples + +### Example of Using Makefile to Implement Compilation + +1. Run the **cd** command to go to the code directory. The **~/code** directory is used as an example. + + ```shell + cd ~/code + ``` + +2. Create a header file **hello.h** and two functions **hello.c** and **main.c**. + + ```shell + vi hello.h + vi hello.c + vi main.c + ``` + + The following is an example of the **hello.h** code: + + ```c + #pragma once + #include + void hello(); + ``` + + The following is an example of the **hello.c** code: + + ```c + #include "hello.h" + void hello() + { + int i=1; + while(i<5) + { + printf("The %dth say hello.\n", i); + i++; + } + } + ``` + + The following is an example of the **main.c** code: + + ```c + #include "hello.h" + #include + int main() + { + hello(); + return 0; + } + ``` + +3. Create the makefile. + + ```shell + vi Makefile + ``` + + The following provides an example of the makefile content: + + ```text + main:main.o hello.o + gcc -o main main.o hello.o + main.o:main.c + gcc -c main.c + hello.o:hello.c + gcc -c hello.c + clean: + rm -f hello.o main.o main + ``` + +4. Run the **make** command. + + ```shell + make + ``` + + After the command is executed, the commands executed in makefile are printed. If you do not need to print the information, add the **-s** option to the **make** command. + + ```shell + gcc -c main.c + gcc -c hello.c + gcc -o main main.o hello.o + ``` + +5. Execute the ./main target. + + ```shell + ./main + ``` + + After the command is executed, the following information is displayed: + + The 1th say hello. + + The 2th say hello. + + The 3th say hello. + + The 4th say hello. diff --git a/docs/en/server/development/gcc/_toc.yaml b/docs/en/server/development/gcc/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..5d3639522f3f8d316bfe9fb1926cb33577a5ac79 --- /dev/null +++ b/docs/en/server/development/gcc/_toc.yaml @@ -0,0 +1,17 @@ +label: GCC User Guide +isManual: true +href: ./overview.md +description: >- + GCC for openEuler builds upon the open source GCC, with a primary focus on + optimizing C, C++, and Fortran languages. +sections: + - label: Kernel FDO User Guide + href: ./kernel-fdo-user-guide.md + - label: LTO User Guide + href: ./lto-user-guide.md + - label: GCC Basic Performance Optimization User Guide + href: ./gcc-basic-performance-optimization-user-guide.md + - label: Alternative GCC 14 User Guide + href: ./gcc-14-secondary-version-compilation-toolchain-user-guide.md + - label: PIN User Guide + href: ./pin-user-guide.md diff --git a/docs/en/server/development/gcc/gcc-14-secondary-version-compilation-toolchain-user-guide.md b/docs/en/server/development/gcc/gcc-14-secondary-version-compilation-toolchain-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..947bcbc6aed1624cdfb75fbdf15619b051fc230e --- /dev/null +++ b/docs/en/server/development/gcc/gcc-14-secondary-version-compilation-toolchain-user-guide.md @@ -0,0 +1,139 @@ +# Background + +## Overview + +OSs prioritize robustness by adopting time-tested, stable software versions rather than the latest releases. This strategy minimizes instability risks from version changes and maintains system stability throughout the LTS cycle. Consequently, openEuler has chosen GCC 12.3.1 as its baseline for the entire 24.03 LTS lifecycle. + +This decision, however, introduces challenges. Many hardware features rely on the foundational GCC toolchain, and using an older GCC version delays the activation of new features in OS releases. Additionally, some users prefer the latest compiler versions to unlock new capabilities, which often deliver performance gains over older versions. + +To enable diverse computational features and cater to varying user needs for hardware support, openEuler 24.09 introduces the openEuler GCC Toolset. This multi-version GCC compilation toolchain, tailored for openEuler, provides a secondary GCC version higher than the system primary version, offering users a more flexible and efficient compilation environment. With the openEuler GCC Toolset 14, users can seamlessly switch between GCC versions to leverage new hardware features and benefit from the latest GCC optimizations. + +## Solution Design + +### Compilation Toolchain Features + +The GCC compilation toolchain, developed and maintained by GNU, is a collection of open source tools designed to translate high-level language code into machine language. Beyond GCC itself, it includes a range of auxiliary tools and libraries, collectively forming a comprehensive compilation environment. + +1. GCC compiler (such as `gcc`, `g++`, and `gfortran`): + + - Role: The GCC compiler is the heart of the toolchain, handling preprocessing and compilation to transform source code into assembly or intermediate representation. For C++ code, `g++` acts as the C++ frontend, performing compilation and automatically linking C++ standard libraries. + +2. Binutils toolset: + + - Tools: including the linker (`ld`), assembler (`as`), object file viewer (`readelf`), symbol viewer (`nm`), object file format converter (`objcopy`), disassembler (`objdump`), and size viewer (`size`) + - Role: These tools support the compilation process by converting assembly to machine code, linking object files into executables, and inspecting file details. + +3. glibc library: + + - Role: The GNU C Library (glibc) is the standard C library for GNU and Linux systems, providing essential functions like `printf` and `malloc` required for compiling C programs. + +4. Other auxiliary tools: + + - Debugger (`gdb`): assists developers in debugging executables to identify and fix errors. + - Performance Analysis Tool (`gprof`): helps analyze and optimize program performance. + +### Toolchain Selection + +The software components in the toolchain significantly influence compilation results, with GCC, binutils, and glibc being the core elements. Since glibc, the C standard library, is tightly coupled with the OS kernel version, it remains unchanged. This toolchain includes only GCC and binutils to fulfill the needs of secondary version compilation. + +The latest GCC release, gcc-14.2.0, is selected for the openEuler GCC toolset. For binutils, while openEuler 24.09 defaults to version 2.41, the latest GCC 14 recommends binutils-2.42. Thus, binutils-2.42 is chosen for this toolchain. + +The openEuler GCC toolset incorporates gcc-14.2.0 and binutils-2.42 as the secondary version toolchain to ensure compilation environment stability and efficiency while minimizing complexity. This approach balances compilation quality and user experience. The toolchain GCC version will be updated to gcc-14.3.0 upon its release by the upstream community. + +### Architecture Design + +To differentiate from the default toolchain and prevent conflicts, this toolchain is named gcc-toolset-14. Its package names begin with the prefix `gcc-toolset-14-`, followed by the original toolchain package name. To avoid path overlap with the default **/usr** installation path, gcc-toolset-14 is installed in **/opt/openEuler/gcc-toolset-14/**. Additionally, to distinguish it from open source GCC and enable future integration of openEuler community features, the version of gcc-toolset-14-gcc is set to 14.2.1. + +The applications and libraries in gcc-toolset-14 coexist with the system default GCC version without replacing or overwriting it. They are not set as the default or preferred option. To simplify version switching and management, the scl-utils tool is introduced. Its usage and switching methods are outlined below. + +## Installation and Deployment + +### Software Requirements + +- OS: openEuler 24.09 + +### Hardware Requirements + +- AArch64 or X86_64 + +### Installation Methods + +Install the default GCC compiler, gcc-12.3.1, in **/usr**: + +```shell +yum install -y gcc gcc-c++ +``` + +Install the secondary version compilation toolchain, gcc-toolset-14, in **/opt/openEuler/gcc-toolset-14/root/usr/**: + +```shell +yum install -y gcc-toolset-14-gcc* +yum install -y gcc-toolset-14-binutils* +``` + +## Usage + +This solution uses the SCL (Software Collections) tool to manage different versions of the compilation toolchain. + +### SCL + +SCL is a vital Linux tool that enables users to install and use multiple versions of applications and runtime environments safely and conveniently, preventing system conflicts. + +Key benefits of SCL include: + +1. Multi-version coexistence: allows installing and using multiple versions of software libraries, tools, and runtime environments on the same system to meet diverse needs. +2. Avoiding system conflicts: isolates different software versions to prevent conflicts with the system default version. +3. Enhancing development efficiency: provides developers with the latest toolchains and runtime environments, improving productivity. + +### Version Switching Methods + +**Install SCL:** + +```shell +yum install scl-utils scl-utils-build +``` + +**Register gcc-toolset-14:** + +```shell +## Register gcc-toolset-14. +scl register /opt/openEuler/gcc-toolset-14/ + +## Deregister gcc-toolset-14. +scl deregister gcc-toolset-14 +``` + +Use `scl list-collections` to verify that gcc-toolset-14 is successfully registered. + +**Switch to gcc-toolset-14:** + +```shell +scl enable gcc-toolset-14 bash +``` + +This command launches a new bash shell session with tools from gcc-toolset-14, replacing the system defaults. In this session, there is no need to manually switch compiler versions or paths. To exit the gcc-toolset-14 environment, type `exit` to return to the system default version. + +SCL works by automatically setting environment variables for different tool versions. For details, check the **/opt/openEuler/gcc-toolset-14/enable** file, which contains all environment variable configurations for gcc-toolset-14. If SCL is unavailable, use the following methods to switch toolchain versions: + +```shell +## Option 1: Without SCL, use a script to switch the compilation toolchain. +source /opt/openEuler/gcc-toolset-14/enable + +## Option 2: With SCL, use SCL to switch the toolchain and activate the runtime environment. +scl enable gcc-toolset-14 bash +``` + +## Usage Constraints + +### Compilation Scenarios + +- **Primary version**: Use the system default gcc-12.3.1 for standard compilation and building. +- **Secondary version**: When the advanced features of GCC 14 are needed for application building, use SCL to switch the bash environment to the gcc-toolset-14 compilation environment. + +### GCC 14 Secondary Version Usage Instructions + +1. The openEuler GCC toolset 14 secondary compilation toolchain offers two usage methods: + 1) **Dynamic linking**: By default, the `-lstdc++` option is automatically included for dynamic linking. This links the system dynamic library **/usr/lib64/libstdc++.so.6** and the **libstdc++_nonshared.a** static library provided by the GCC 14 secondary version. This static library contains stable C++ features introduced in GCC 14 compared to GCC 12. + 2) **Static linking**: You can use the `-static` option for static linking, which links the full-feature **libstdc++.a** static library provided by the GCC 14 secondary version. The path to this library is **/opt/openEuler/gcc-toolset-14/root/usr/lib/gcc/aarch64-openEuler-linux/14/libstdc++.a**. + +2. By default, builds use dynamic linking, which links the **libstdc++_nonshared.a** static library. To ensure system compatibility, this library only includes officially standardized C++ features. Experimental features like `-fmodules-ts` and `-fmodule-header`, which are part of C++20 module capabilities, are not included in **libstdc++_nonshared.a**. If you need these features, you should use static linking to fully link the GCC 14 secondary version static library. diff --git a/docs/en/server/development/gcc/gcc-basic-performance-optimization-user-guide.md b/docs/en/server/development/gcc/gcc-basic-performance-optimization-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..e65f3458ee487eb261d7f254db1aeac3b8cb4946 --- /dev/null +++ b/docs/en/server/development/gcc/gcc-basic-performance-optimization-user-guide.md @@ -0,0 +1,185 @@ +# GCC Basic Performance Optimization User Guide + +## Introduction + +Compiler performance optimization plays a vital role in enhancing application development efficiency, runtime performance, and maintainability. It is a significant research area in computer science and a critical component of the software development process. GCC for openEuler extends its general compilation optimization capabilities by improving backend performance techniques, including instruction optimization, vectorization enhancements, prefetching improvements, and data flow analysis optimizations. + +## Installation and Deployment + +### Software Requirements + +OS: openEuler 24.09 + +### Hardware Requirements + +AArch64 architecture + +### Software Installation + +Install GCC and related components as required. For example, to install GCC: + +```shell +yum install gcc +``` + +## Usage + +### CRC Optimization + +#### Description + +Detects CRC software loop code and generates efficient hardware instructions. + +#### Usage + +Include the `-floop-crc` option during compilation. + +Note: The `-floop-crc` option must be used alongside `-O3 -march=armv8.1-a`. + +### If-Conversion Enhancement + +#### Description + +Improves If-Conversion optimization by leveraging additional registers to minimize conflicts. + +#### Usage + +This optimization is part of the RTL if-conversion process. Enable it using the following compilation options: + +`-fifcvt-allow-complicated-cmps` + +`--param=ifcvt-allow-register-renaming=[0,1,2]` (The value controls the optimization scope.) + +Note: This optimization requires the `-O2` optimization level and should be used with `--param=max-rtl-if-conversion-unpredictable-cost=48` and `--param=max-rtl-if-conversion-predictable-cost=48`. + +### Multiplication Calculation Optimization + +#### Description + +Optimizes Arm-related instruction merging to recognize 32-bit complex combinations of 64-bit integer multiplication logic and produce efficient 64-bit instructions. + +#### Usage + +Enable the optimization using the `-fuaddsub-overflow-match-all` and `-fif-conversion-gimple` options. + +Note: This optimization requires `-O3` or higher optimization levels. + +### cmlt Instruction Generation Optimization + +#### Description + +Generates `cmlt` instructions for specific arithmetic operations, reducing the instruction count. + +#### Usage + +Enable the optimization using the `-mcmlt-arith` option. + +Note: This optimization requires `-O3` or higher optimization levels. + +### Vectorization Optimization Enhancement + +#### Description + +Identifies and simplifies redundant instructions generated during vectorization, enabling shorter loops to undergo vectorization. + +#### Usage + +Enable the optimization using the parameter `--param=vect-alias-flexible-segment-len=1` (default is 0). + +Note: This optimization requires `-O3` or higher optimization levels. + +### Combined Optimization of min max and uzp1/uzp2 Instructions + +#### Description + +Identifies opportunities to optimize `min max` and `uzp1/uzp2` instructions together, reducing the instruction count to enhance performance. + +#### Usage + +Enable `min max` optimization with the `-fconvert-minmax` option. The `uzp1/uzp2` instruction optimization is automatically enabled at `-O3` or higher levels. + +Note: This optimization requires `-O3` or higher optimization levels. + +### ldp/stp Optimization + +#### Description + +Detects poorly performing `ldp/stp` instructions and splits them into two separate `ldr` and `str` instructions. + +#### Usage + +Enable the optimization using the `-fsplit-ldp-stp` option. Control the search range with the parameter `--param=param-ldp-dependency-search-range=[1,32]` (default is 16). + +Note: This optimization requires `-O1` or higher optimization levels. + +### AES Instruction Optimization + +#### Description + +Identifies AES software algorithm instruction sequences and replaces them with hardware instructions for acceleration. + +#### Usage + +Enable the optimization using the `-fcrypto-accel-aes` option. + +Note: This optimization requires `-O3` or higher optimization levels. + +### Indirect Call Optimization + +#### Description + +Analyzes and optimizes indirect calls in the program, converting them into direct calls where possible. + +#### Usage + +Enable the optimization using the `-ficp -ficp-speculatively` options. + +Note: This optimization must be used with `-O2 -flto -flto-partition=one`. + +### IPA-prefetch + +#### Description + +Detects indirect memory accesses in loops and inserts prefetch instructions to minimize latency. + +#### Usage + +Enable the optimization using the `-fipa-prefetch -fipa-ic` options. + +Note: This optimization must be used with `-O3 -flto`. + +### -fipa-struct-reorg + +#### Description + +Optimizes memory layout by reorganizing the arrangement of structure members to improve cache hit rates. + +#### Usage + +Add the options `-O3 -flto -flto-partition=one -fipa-struct-reorg` to enable the optimization. + +Note: The `-fipa-struct-reorg` option requires `-O3 -flto -flto-partition=one` to be enabled globally. + +### -fipa-reorder-fields + +#### Description + +Optimizes memory layout by reordering structure members from largest to smallest, reducing padding and improving cache hit rates. + +#### Usage + +Add the options `-O3 -flto -flto-partition=one -fipa-reorder-fields` to enable the optimization. + +Note: The `-fipa-reorder-fields` option requires `-O3 -flto -flto-partition=one` to be enabled globally. + +### -ftree-slp-transpose-vectorize + +#### Description + +Enhances data flow analysis for loops with consecutive memory reads by inserting temporary arrays during loop splitting. During SLP vectorization, it introduces transposition analysis for `grouped_stores`. + +#### Usage + +Add the options `-O3 -ftree-slp-transpose-vectorize` to enable the optimization. + +Note: The `-ftree-slp-transpose-vectorize` option requires `-O3` to be enabled. diff --git a/docs/en/server/development/gcc/kernel-fdo-user-guide.md b/docs/en/server/development/gcc/kernel-fdo-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..02c9ec2b151b0e06e05168bd6de78374be9acfd8 --- /dev/null +++ b/docs/en/server/development/gcc/kernel-fdo-user-guide.md @@ -0,0 +1,69 @@ +# Kernel FDO User Guide + +## Overview + +The feedback-directed optimization (FDO) of the kernel allows users to build optimized kernels for different applications to improve the application performance in single-application scenarios. In addition, FDO is integrated GCC for openEuler, and A-FOT provides automatic optimization, enabling users to easily enable FDO. + +## Installation and Deployment + +### Software Requirements + +* OS: openEuler 23.09 + +### Hardware Requirements + +* Architecture: AArch64 or x86_64 + +### Software Installation + +#### Downloading the Kernel Source Code + +Download the kernel source code, A-FOT, and dependency packages. + +```shell +yum install -y kernel-source A-FOT make gcc flex bison elfutils-libelf-devel diffutils openssl-devel dwarves +``` + +Copy the kernel source code. + +```shell +cp -r /usr/src/linux-6.4.0-8.0.0.16.oe2309.aarch64 . +``` + +**Note: Change the version number as required.** + +## Usage + +You can use A-FOT to enable kernel FDO and obtain the optimized kernel by specifying **opt_mode** as **Auto_kernel_PGO**. Other configuration items can be specified on the CLI, for example, `./a-fot --pgo_phase 1`. `-s` and `-n` options can be specified on CLI only. Options related to kernel FDO are as follows. + +| No.| Option (Configuration File)| Description | Default Value | +| ---- | -------------------- | ------------------------------------------------------------ | ------------------------ | +| 1 | config_file | Path of the configuration file. User configurations are read from this file. | ${afot_path}/a-fot.ini | +| 2 | opt_mode | Optimization mode to be executed by the tool. The value can be **AutoFDO**, **AutoPrefetch**, **AutoBOLT**, or **Auto_kernel_PGO**.| AutoPrefetch | +| 3 | pgo_mode | Kernel FDO mode, which can be **arc** (GCOV, using only the arc profile) or **all** (full PGO, using the arc and value profiles). | all | +| 4 | pgo_phase | FDO execution phase. The value can be **1** (kernel instrumentation phase) or **2** (data collection and kernel optimization phase). | 1 | +| 5 | kernel_src | Kernel source code directory. If this option is not specified, the tool automatically downloads the source code. | None (optional) | +| 6 | kernel_name | File name of the kernel build. The tool will add the **-pgoing** or **-pgoed** suffix depending on the phase. | kernel | +| 7 | work_path | Script working directory, which is used to store log files, wrappers, and profiles. | **/opt** (**/tmp** cannot be used.)| +| 8 | run_script | Application execution script. The user needs to write the script, which will be used by the tool to execute the target application.| /root/run.sh | +| 9 | gcc_path | GCC path. | /usr | +| 10 | -s | Silent mode. The tool automatically reboots and switches the kernel to execute the second phase. | None | +| 11 | -n | The tool is not used to compile the kernel. This option applies to the scenario where the execution environment and kernel compilation environment are different. | None | + +After configuring the compilation options, run the following command to use A-FOT to automatically optimize the kernel: + +```shell +a-fot --config_file ./a-fot.ini -s +``` + +**Note: The `-s` option instructs A-FOT to automatically reboot into the compiled kernel. If you do not want the tool to automatically perform this sensitive operation, omit this option. However, you need to manually reboot and perform the second phase (`--pgo_phase 2`).** + +**Note: All paths must be absolute paths.** + +**Note: The kernel of openEuler 23.09 does not support full PGO. Change the value of pgo_mode to arc.** + +## Compatibility + +This section describes the compatibility issues in some special scenarios. This project is in continuous iteration and issues will be fixed as soon as possible. Developers are welcome to join this project. + +* The kernel of openEuler 23.09 does not support full PGO. Change the value of pgo_mode to arc. diff --git a/docs/en/server/development/gcc/lto-user-guide.md b/docs/en/server/development/gcc/lto-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..cd188c1ca5f8c0e3c025f8b6b0562048ed0e926a --- /dev/null +++ b/docs/en/server/development/gcc/lto-user-guide.md @@ -0,0 +1,25 @@ +# Introduction to Link-Time Optimization + +In traditional compilation, GCC compiles and optimizes individual source files (compilation units) to generate .o object files containing assembly code. The linker then processes these `.o` files, resolving symbol tables and performing relocations to create the final executable. However, the linker, which has access to cross-file function call information, operates on assembly code and cannot perform compilation optimizations. Conversely, the compilation stage capable of optimizations lacks global cross-file information. While this approach improves efficiency by recompiling only modified units, it misses many cross-file optimization opportunities. + +Link-Time Optimization (LTO) addresses this limitation by enabling optimizations during the linking phase, leveraging cross-compilation-unit call information. To achieve this, LTO preserves the Intermediate Representation (IR) required for optimizations until linking. During linking, the linker invokes the LTO plugin to perform whole-program analysis, make better optimization decisions, and generate more efficient IR. This optimized IR is then converted into object files with assembly code, and the linker completes the standard linking process. + +# Enabling LTO in Version Builds + +## Background + +Many international communities have adopted LTO in their version builds to achieve better performance and smaller binary sizes. LTO is emerging as a key area for exploring compilation optimization opportunities. Starting with version 24.09, openEuler will introduce LTO in its version builds. + +## Solution + +To enable LTO during package builds, we will add `-flto -ffat-lto-objects` to the global compilation options in the macros of **openEuler-rpm-config**. The `-flto` flag enables Link-Time Optimization, while `-ffat-lto-objects` generates fat object files containing both LTO object information and the assembly information required for regular linking. During the build process, LTO object information is used for optimizations. However, since LTO object files are not compatible across GCC versions, we remove the LTO-related fields from `.o/.a` files before packaging them into `.rpm` files, retaining only the assembly code needed for regular linking. This ensures that static libraries remain unaffected. + +## Scope of Enablement + +Due to the significant differences between LTO and traditional compilation workflows, and to minimize the impact on version quality, LTO is currently enabled for only 500+ packages. The list of these packages is available in **/usr/lib/rpm/%{_vendor}/lto_white_list**. These whitelisted applications have been successfully built and passed their test suites with LTO enabled. The LTO compilation options (`-flto -ffat-lto-objects`) are applied only when building whitelisted applications; otherwise, they are omitted. + +In future innovation releases, we will work with application maintainers to expand the scope of LTO enablement. + +## Notes + +The current hot-patching mechanism is incompatible with LTO, causing hot patches to fail when LTO is enabled. We have agreed with the hot-patching team on a solution, which will be implemented in future releases. diff --git a/docs/en/server/development/gcc/overview.md b/docs/en/server/development/gcc/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..9ac7d9b17c68f7897a0675a7116e671dbc94aafe --- /dev/null +++ b/docs/en/server/development/gcc/overview.md @@ -0,0 +1,3 @@ +# GCC for openEuler User Guide + +The GCC for openEuler compiler is developed based on the open source GNU compiler Collection (GCC). The open source GCC is the de facto standard of cross-platform compilers, and it complies with the GPLv3 license, becoming the most widely used C/C++ compiler on Linux. GCC for openEuler inherits capabilities of the open source GCC. It also has optimizations on C, C++, and Fortran languages and delivers enhanced features such as automatic feedback-directed optimization (FDO), software and hardware collaboration, memory optimization, and automatic vectorization. GCC for openEuler is compatible with a wide range of hardware platforms such as Kunpeng, Phytium, and Loongson, fully unleashing the computing power of these hardware platforms. diff --git a/docs/en/server/development/gcc/pin-user-guide.md b/docs/en/server/development/gcc/pin-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..eaa335aaf8cea0f192bf62a84a3d36b988262b26 --- /dev/null +++ b/docs/en/server/development/gcc/pin-user-guide.md @@ -0,0 +1,134 @@ +# Installation and Deployment + +## Software + +* OS: openEuler 23.03 + +## Hardware + +* x86_64 +* Arm + +## Preparing the Environment + +* Install the openEuler operating system. For details, see the [*openEuler Installation Guide*](../../installation_upgrade/installation/installation.md). + +### Install the dependency + +#### Installing the Software on Which the PIN GCC Client Depends + +```shell +yum install -y grpc +yum install -y grpc-devel +yum install -y grpc-plugins +yum install -y protobuf-devel +yum install -y jsoncpp +yum install -y jsoncpp-devel +yum install -y gcc-plugin-devel +yum install -y llvm-mlir +yum install -y llvm-mlir-devel +yum install -y llvm-devel +``` + +#### Installing the Software on Which the PIN Server Depends + +```shell +yum install -y grpc +yum install -y grpc-devel +yum install -y grpc-plugins +yum install -y protobuf-devel +yum install -y jsoncpp +yum install -y jsoncpp-devel +yum install -y llvm-mlir +yum install -y llvm-mlir-devel +yum install -y llvm-devel +``` + +## Installing PIN + +### rpmbuild + +#### Building the PIN GCC Client + +```shell +git clone https://gitee.com/src-openeuler/pin-gcc-client.git +cd pin-gcc-client +mkdir -p ~/rpmbuild/SOURCES +cp *.path pin-gcc-client.tar.gz ~/rpmbuild/SOURCES +rpmbuild -ba pin-gcc-client.spec +cd ~/rpmbuild/RPMS +rpm -ivh pin-gcc-client.rpm +``` + +#### Building the PIN Server + +```shell +git clone https://gitee.com/src-openeuler/pin-server.git +cd pin-server +mkdir -p ~/rpmbuild/SOURCES +cp *.path pin-server.tar.gz ~/rpmbuild/SOURCES +rpmbuild -ba pin-server.spec +cd ~/rpmbuild/RPMS +rpm -ivh pin-server.rpm +``` + +### Build + +#### Building the PIN GCC Client + +```shell +git clone https://gitee.com/openeuler/pin-gcc-client.git +cd pin-gcc-client +mkdir build +cd build +cmake ../ -DMLIR_DIR=${MLIR_PATH} -DLLVM_DIR=${LLVM_PATH} +make +``` + +#### Building the PIN Server + +```shell +git clone https://gitee.com/openeuler/pin-server.git +cd pin-server +mkdir build +cd build +cmake ../ -DMLIR_DIR=${MLIR_PATH} -DLLVM_DIR=${LLVM_PATH} +make +``` + +# Usage + +You can use `-fplugin` and `-fplugin-arg-libpin_xxx` to enable the Plug-IN (PIN) tool. +Command: + +```shell +$(TARGET): $(OBJS) + $(CXX) -fplugin=${CLIENT_PATH}/build/libpin_gcc_client.so \ + -fplugin-arg-libpin_gcc_client-server_path=${SERVER_PATH}/build/pin_server \ + -fplugin-arg-libpin_gcc_client-log_level="1" \ + -fplugin-arg-libpin_gcc_client-arg1="xxx" +``` + +You can use the `${INSTALL_PATH}/bin/pin-gcc-client.json` file to configure PIN. The configuration options are as follows: + +`path`: path of the executable file of the PIN server. + +`sha256file`: path of the PIN verification file `xxx.sha256`. + +`timeout`: timeout interval for cross-process communication, in milliseconds. + +Compile options: + +`-fplugin`: path of the .so file of the PIN client. + +`-fplugin-arg-libpin_gcc_client-server_path`: path of the executable program of the PIN server. + +`-fplugin-arg-libpin_gcc_client-log_level`: default log level. The value ranges from `0` to `3`. The default value is `1`. + +`-fplugin-arg-libpin_gcc_client-argN`: other parameters that can be specified as required. `argN` indicates the argument required by PIN. + +# Compatibility + +This section describes the compatibility issues in some special scenarios. This project is in continuous iteration and will be fixed as soon as possible. Developers are welcome to join this project. + +* When PIN is enabled in the `-flto` phase, multi-process compilation using `make -j` is not supported. You are advised to use `make -j1` for compilation. diff --git a/docs/en/server/diversified_computing/dpu_offload/_toc.yaml b/docs/en/server/diversified_computing/dpu_offload/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..b35d947fec27f79991a1f10b5131dbc638ab77fa --- /dev/null +++ b/docs/en/server/diversified_computing/dpu_offload/_toc.yaml @@ -0,0 +1,16 @@ +label: libvirt Direct Connection Aggregation Environment Establishment +isManual: true +href: ./libvirt-direct-connection-aggregation-environment-establishment.md +description: >- + DPU offloading feature for container management and its installation and + deployment method on openEuler +sections: + - label: qtfs Shared File System + href: ./qtfs-architecture-and-usage.md + - label: Imperceptible DPU Offload User Guide + href: ./overview.md + sections: + - label: Imperceptible Container Management Plane Offload + href: ./imperceptible-container-management-plane-offload.md + - label: Imperceptible Container Management Plane Offload Deployment Guide + href: ./offload-deployment-guide.md diff --git a/docs/en/server/diversified_computing/dpu_offload/config/client.json b/docs/en/server/diversified_computing/dpu_offload/config/client.json new file mode 100644 index 0000000000000000000000000000000000000000..4aedf4c846914a6bc34dff1988c7794ddb1fa521 --- /dev/null +++ b/docs/en/server/diversified_computing/dpu_offload/config/client.json @@ -0,0 +1,5 @@ +{ + "Protocol": "tcp", + "Ipaddr" : "192.168.10.11", + "Port" : "7777" +} diff --git a/docs/en/server/diversified_computing/dpu_offload/config/prepare.sh b/docs/en/server/diversified_computing/dpu_offload/config/prepare.sh new file mode 100644 index 0000000000000000000000000000000000000000..ccfe9402051a02451644345b39ef6aa2657bfe89 --- /dev/null +++ b/docs/en/server/diversified_computing/dpu_offload/config/prepare.sh @@ -0,0 +1,58 @@ +#!/bin/bash + +mkdir -p /another_rootfs/var/run/docker/containerd +iptables -t nat -N DOCKER + +echo "---------insmod qtfs ko----------" +# TEST_MODE: IP +insmod ${YOUR_PATH}/qtfs.ko qtfs_server_ip=${YOUR_SERVER_IP} qtfs_log_level=INFO # Enter the .ko file path and IP address. +nohup ${YOUR_PATH}/udsproxyd 1 ${YOUR_CLIENT_IP} 12121 ${YOUR_SERVER_IP} 12121 2>&1 & + +# TEST_MODE: vsock +# insmod ${YOUR_PATH}/qtfs.ko qtfs_server_vsock_cid=${YOUR_SERVER_VSOCK_CID} qtfs_log_level=INFO # Enter the .ko file path and IP address. +# nohup ${YOUR_PATH}/udsproxyd 1 ${YOUR_CLIENT_VSOCK_CID} 12121 ${YOUR_SERVER_VSOCK_CID} 12121 2>&1 & + +qtcfg -w udsconnect -x /var/run/rexec +qtcfg -w udsconnect -x /run/rexec + +mkdir /another_rootfs/local_proc/ +mount -t proc proc /another_rootfs/local_proc/ +mount --bind /var/run/ /another_rootfs/var/run/ +mount --bind /var/lib/ /another_rootfs/var/lib/ +mount --bind /etc /another_rootfs/etc +mount -t devtmpfs devtmpfs /another_rootfs/dev/ +mount -t sysfs sysfs /another_rootfs/sys +mkdir -p /another_rootfs/sys/fs/cgroup +mount -t tmpfs tmpfs /another_rootfs/sys/fs/cgroup +list="perf_event freezer files net_cls,net_prio hugetlb pids rdma cpu,cpuacct memory devices blkio cpuset" +for i in $list +do + echo $i + mkdir -p /another_rootfs/sys/fs/cgroup/$i + mount -t cgroup cgroup -o rw,nosuid,nodev,noexec,relatime,$i /another_rootfs/sys/fs/cgroup/$i +done + +mount -t qtfs -o proc /proc /another_rootfs/proc +echo "proc" +mount -t qtfs /sys /another_rootfs/sys +echo "cgroup" + +mkdir -p /another_rootfs/var/lib/docker/containers +mkdir -p /another_rootfs/var/lib/docker/containerd +mkdir -p /another_rootfs/var/lib/docker/overlay2 +mkdir -p /another_rootfs/var/lib/docker/image +mkdir -p /another_rootfs/var/lib/docker/tmp +mount -t qtfs /var/lib/docker/containers /another_rootfs/var/lib/docker/containers +mount -t qtfs /var/lib/docker/containerd /another_rootfs/var/lib/docker/containerd +mount -t qtfs /var/lib/docker/overlay2 /another_rootfs/var/lib/docker/overlay2 +mount -t qtfs /var/lib/docker/image /another_rootfs/var/lib/docker/image +mount -t qtfs /var/lib/docker/tmp /another_rootfs/var/lib/docker/tmp +mkdir -p /another_rootfs/run/containerd/io.containerd.runtime.v1.linux/ +mount -t qtfs /run/containerd/io.containerd.runtime.v1.linux/ /another_rootfs/run/containerd/io.containerd.runtime.v1.linux/ +mkdir -p /another_rootfs/var/run/docker/containerd +mount -t qtfs /run/docker/containerd /another_rootfs/run/docker/containerd +mkdir -p /another_rootfs/var/lib/containerd/io.containerd.runtime.v1.linux +mount -t qtfs /var/lib/containerd/io.containerd.runtime.v1.linux /another_rootfs/var/lib/containerd/io.containerd.runtime.v1.linux + +qtcfg -w udsconnect -x /another_rootfs/var/run/rexec +qtcfg -w udsconnect -x /another_rootfs/run/rexec diff --git a/docs/en/server/diversified_computing/dpu_offload/config/rexec.service b/docs/en/server/diversified_computing/dpu_offload/config/rexec.service new file mode 100644 index 0000000000000000000000000000000000000000..ee9e5e4895adb5c010e3f8d4db6652cfaed3d355 --- /dev/null +++ b/docs/en/server/diversified_computing/dpu_offload/config/rexec.service @@ -0,0 +1,13 @@ +[Unit] +Description=Rexec_server Service +After=network.target + +[Service] +Type=simple +Environment=CMD_NET_ADDR=tcp://0.0.0.0:7777 +ExecStart=/usr/bin/rexec_server +ExecReload=/bin/kill -s HUP $MAINPID +KillMode=process + +[Install] +WantedBy=multi-user.target diff --git a/docs/en/server/diversified_computing/dpu_offload/config/server.json b/docs/en/server/diversified_computing/dpu_offload/config/server.json new file mode 100644 index 0000000000000000000000000000000000000000..1d4a7bbbc1cbf086e18b147f3f27e6a15c2e322e --- /dev/null +++ b/docs/en/server/diversified_computing/dpu_offload/config/server.json @@ -0,0 +1,5 @@ +{ + "Protocol": "tcp", + "Ipaddr" : "0.0.0.0", + "Port" : "7777" +} diff --git a/docs/en/server/diversified_computing/dpu_offload/config/server_start.sh b/docs/en/server/diversified_computing/dpu_offload/config/server_start.sh new file mode 100644 index 0000000000000000000000000000000000000000..fd3655159ddb0fc6069dfa3ab802f4c9f8520c13 --- /dev/null +++ b/docs/en/server/diversified_computing/dpu_offload/config/server_start.sh @@ -0,0 +1,44 @@ +#!/bin/bash + +modprobe overlay +mkdir /var/lib/docker/containers +mkdir -p /var/lib/docker/containers +mkdir -p /var/lib/docker/containerd +mkdir -p /var/lib/docker/overlay2 +mkdir -p /var/lib/docker/tmp +mkdir -p /var/lib/docker/image +mkdir -p /var/run/docker/containerd +mkdir -p /run/containerd/io.containerd.runtime.v1.linux/ +mkdir -p /var/run/docker/netns +mkdir -p /var/lib/containerd/io.containerd.runtime.v1.linux/ +mkdir -p /run/user/0 +touch /var/run/docker/netns/default +# this should be done once +mount --bind /proc/1/ns/net /var/run/docker/netns/default + +function TaskClean() +{ + echo "Now do task clean..." + pkill engine + rmmod qtfs_server + echo "TaskClean done" +} + +trap "TaskClean exit" SIGINT + +mkdir -p /var/run/docker/containerd +mkdir -p /run/containerd/io.containerd.runtime.v1.linux/ + +# TEST_MODE: IP +insmod ${YOUR_PATH}/qtfs_server.ko qtfs_server_ip=${YOUR_SERVER_IP} qtfs_log_level=ERROR +nohup ${YOUR_PATH}/engine 16 1 ${YOUR_SERVER_IP} 12121 ${YOUR_CLIENT_IP} 12121 2>&1 & + +# TEST_MODE: vsock +# insmod ${YOUR_PATH}/qtfs_server.ko qtfs_server_vsock_cid=${YOUR_SERVER_VSOCK_CID} qtfs_log_level=ERROR +# nohup ${YOUR_PATH}/engine 16 1 ${YOUR_SERVER_VSOCK_CID} 12121 ${YOUR_CLIENT_VSOCK_CID} 12121 2>&1 & + +sleep 2 + +qtcfg -w udsconnect -x /var/run/rexec +qtcfg -w udsconnect -x /run/rexec +qtcfg -w udsconnect -x /var/run/containerd diff --git a/docs/en/server/diversified_computing/dpu_offload/config/whitelist b/docs/en/server/diversified_computing/dpu_offload/config/whitelist new file mode 100644 index 0000000000000000000000000000000000000000..b0be45f86276e89fa9fd0827ba06bbb27d158f62 --- /dev/null +++ b/docs/en/server/diversified_computing/dpu_offload/config/whitelist @@ -0,0 +1,8 @@ +kill +taskset +qemu-kvm +rexec_shim +/usr/bin/taskset +/usr/bin/kill +/usr/bin/qemu-kvm +/usr/bin/rexec_shim diff --git a/docs/en/server/diversified_computing/dpu_offload/figures/arch.png b/docs/en/server/diversified_computing/dpu_offload/figures/arch.png new file mode 100644 index 0000000000000000000000000000000000000000..b6a7836fd6fab75009e781ac1ed96c73c352f75b Binary files /dev/null and b/docs/en/server/diversified_computing/dpu_offload/figures/arch.png differ diff --git a/docs/en/server/diversified_computing/dpu_offload/figures/offload-arch.png b/docs/en/server/diversified_computing/dpu_offload/figures/offload-arch.png new file mode 100644 index 0000000000000000000000000000000000000000..b0f7b8587c47838880bcca5d6694f66a16ec0aaf Binary files /dev/null and b/docs/en/server/diversified_computing/dpu_offload/figures/offload-arch.png differ diff --git a/docs/en/server/diversified_computing/dpu_offload/figures/qtfs-arch.png b/docs/en/server/diversified_computing/dpu_offload/figures/qtfs-arch.png new file mode 100644 index 0000000000000000000000000000000000000000..749b007287d8503badcea52036b7a71b06092bc2 Binary files /dev/null and b/docs/en/server/diversified_computing/dpu_offload/figures/qtfs-arch.png differ diff --git a/docs/en/server/diversified_computing/dpu_offload/imperceptible-container-management-plane-offload.md b/docs/en/server/diversified_computing/dpu_offload/imperceptible-container-management-plane-offload.md new file mode 100644 index 0000000000000000000000000000000000000000..b49dd3e781f859574dc65edf1f31a9673c9142ed --- /dev/null +++ b/docs/en/server/diversified_computing/dpu_offload/imperceptible-container-management-plane-offload.md @@ -0,0 +1,35 @@ +# Imperceptible Container Management Plane Offload + +## Overview + +Moore's law ceases to apply in data center and cloud scenarios. The CPU computing power growth rate of general processing units is slowing down, while the network I/O speed and performance keep increasing. As a result, the processing capability of current general-purpose processors cannot meet the I/O processing requirements of the network and drives. In traditional data centers, more and more general-purpose CPU computing power is occupied by I/O and management planes. This part of resource loss is called data center tax. According to AWS statistics, the data center tax may account for more than 30% of the computing power of the data center. + +The data processing unit (DPU) is developed to release the computing resources from the host CPU. The management plane, network, storage, and security capabilities are offloaded to DPUs for acceleration, reducing costs and improving efficiency. Mainstream cloud vendors, such as AWS, Alibaba Cloud, and Huawei Cloud, use self-developed processors to offload the management plane and related data plane, achieving 100% utilization of data center computing resources. + +The management plane processes can be offloaded to the DPU by splitting the component source code. The source code is split into two parts that run independently on the host and DPU based on the function logic. In this way, the component is offloaded. However, this method has the following problems: + +1. The software compatibility of the component is affected. You need to maintain the component and related patches in subsequent version upgrades, which increases the maintenance workload. +2. The offload cannot be inherited by other components. You need to split each component based on code logic analysis. + +To solve these problems, openEuler introduces imperceptible DPU offload. The abstraction layer provided by the OS shields the cross-host access differences between the host and DPU, and enables service processes to be offloaded to the DPU with virtually zero modification. This part of work at the common layer of the OS and is irrelevant to upper-layer services. Other services can also inherit the offload to DPU. + +## Architecture + +### Imperceptible Container Management Plane DPU Offload Architecture + +**Figure 1** Imperceptible Container Management Plane DPU Offload Architecture + +![offload-arch](./figures/offload-arch.png) + +As shown in Figure 1, after the container management plane is offloaded, management processes such as dockerd and kubelet run on the DPU side, and container processes run on the host. The interaction between processes is ensured by the system layer. + +* Communication layer: DPUs and hosts can communicate with each other through PCIe interfaces or networks. A communication interface layer is provided based on underlying physical connections to provide communication interfaces for upper-layer services. + +* qtfs kernel shared file system: The container management plane components kubelet and dockerd interact with container processes through file systems. Management plane tools need to prepare data plane paths to rootfs and volume for container processes. In addition, the proc and cgroup file systems need to be used to control and monitor the resources and status of container processes. For details about qtfs, see [qtfs Shared File System Introduction and Usage](./qtfs-architecture-and-usage.md). + +* User-mode offload environment: You need to use qtfs to prepare the runtime environment for the offloaded management plane, and remotely mount the container management and runtime directories of the host to the DPU. System management file systems such as proc, sys, and cgroup need to be mounted. To prevent damage to the native system functions of the DPU, the preceding mounting operations are performed in the chroot environment. In addition, the management plane (running on the DPU) and container processes (running on the host) have invoking relationships. The rexec remote binary execution tool needs to be used to provide corresponding functions. + +For details about how to offload container management plane, see the [Deployment Guide](./offload-deployment-guide.md). + +> ![](public_sys-resources/icon-note.gif) **NOTE**: +> In this user guide, modifications are performed to the container management plane components and the rexec tool of a specific version. You can modify other versions based on the actual execution environment. The patch provided in this document is for verification only and is not for commercial use. diff --git a/docs/en/server/diversified_computing/dpu_offload/libvirt-direct-connection-aggregation-environment-establishment.md b/docs/en/server/diversified_computing/dpu_offload/libvirt-direct-connection-aggregation-environment-establishment.md new file mode 100644 index 0000000000000000000000000000000000000000..42e5933ad03ca73ab6770efafb8fcd4153026fdf --- /dev/null +++ b/docs/en/server/diversified_computing/dpu_offload/libvirt-direct-connection-aggregation-environment-establishment.md @@ -0,0 +1,358 @@ +# 1 Hardware Preparation + +## Test Mode + +Prepare two physical machines (VMs have not been tested) that can communicate with each other. + +One physical machine functions as the DPU, and the other functions as the host. In this document, DPU and HOST refer to the two physical machines. + +> [!NOTE]NOTE +> In the test mode, network ports are exposed without connection authentication, which is risky and should be used only for internal tests and verification. Do not use this mode in the production environment. + +## vsock mode + +The DPU and HOST are required. The DPU must be able to provide vsock communication through virtio. + +This document describes only the test mode usage. + +# 2 libvirt offload architecture + +![arch](./figures/arch.png) + +# 3 Environment Setup + +## 3.1 qtfs File System Deployment + +For details, visit . + +To establish a qtfs connection, you need to disable the firewall. + +## 3.2 Deploying the udsproxyd Service + +### 3.2.1 Introduction + +udsproxyd is a cross-host Unix domain socket (UDS) proxy service, which needs to be deployed on both the host and DPU. The udsproxyd components on the host and dpu are peers. They implement seamless UDS communication between the host and DPU, which means that if two processes can communicate with each other through UDSs on the same host, they can do the same between the host and DPU. The code of the processes does not need to be modified, only that the client process needs to run with the **LD_PRELOAD=libudsproxy.so** environment variable. As a cross-host Unix socket service, udsproxyd can be used by running with `LD_PRELOAD=libudsproxy.so`. With the support of qtfs, udsproxyd can also be used transparently. You need to configure the allowlist in advance. The specific operations are described later. + +### 3.2.2 Deploying udsproxyd + +Build udsproxyd in the dpu-utilities project: + +```bash +cd qtfs/ipc +make -j UDS_TEST_MODE=1 && make install +``` + +The engine service on the qtfs server has incorporated the udsproxyd feature. You do not need to manually start udsproxyd if the qtfs server is deployed. However, you need to start udsproxyd on the client by running the following command: + +```bash +nohup /usr/bin/udsproxyd 2>&1 & +``` + +Parameters: + +```bash +thread num: number of threads. Currently, only one thread is supported. +addr: IP address of the host. +port: Port used on the host. +peer addr: IP address of the udsproxyd peer. +peer port: port used on the udsproxyd peer. +``` + +Example: + +```bash +nohup /usr/bin/udsproxyd 1 192.168.10.10 12121 192.168.10.11 12121 2>&1 & +``` + +If the qtfs engine service is not started, you can start udsproxyd on the server to test udsproxyd separately. Run the following command: + +```bash +nohup /usr/bin/udsproxyd 1 192.168.10.11 12121 192.168.10.10 12121 2>&1 & +``` + +### 3.2.3 Using udsproxyd + +#### 3.2.3.1 Using udsproxyd Independently + +When starting the client process of the Unix socket application that uses the UDS service, add the **LD_PRELOAD=libudsproxy.so** environment variable to intercept the **connect** API of glibc for UDS interconnection. In the libvirt offload scenario, you can copy **libudsproxy.so**, which will be used by the libvirtd service, to the **/usr/lib64** directory in the chroot directory of libvirt. + +#### 3.2.3.2 Using the udsproxyd Service Transparently + +Configure the UDS service allowlist for qtfs. The allowlist is the sock file address bound to the Unix socket server. For example, the files of the Unix socket server created by the libvirt VM are in the **/var/lib/libvirt** directory. In this case, add the directory path to the allowlist in either of the following ways: + +* Load the allowlist by using the `qtcfg` utility. First compile the utility in **qtfs/qtinfo**. + +Run the following command on the qtfs client: + +```bash +make role=client +make install +``` + +Run the following command on the qtfs server: + +```bash +make role=server +make install +``` + +After `qtcfg` is installed automatically, run `qtcfg` to configure the allowlist. Assume that **/var/lib/libvirt** needs to be added to the allowlist: + +```bash +qtcfg -x /var/lib/libvirt/ +``` + +Query the allowlist: + +```bash +qtcfg -z +``` + +Delete an allowlist entry: + +```bash +qtcfg -y 0 +``` + +The parameter is the index number listed when you query the allowlist. + +* Add an allowlist entry through the configuration file. The configuration file needs to be set before the qtfs or qtfs_server kernel module is loaded. The allowlist is loaded when the kernel modules are initialized. + +> [!NOTE]NOTE +> The allowlist prevents irrelevant Unix sockets from establishing remote connections, causing errors or wasting resources. You are advised to set the allowlist as precisely as possible. For example, in this document, **/var/lib/libvirt** is set in the libvirt scenario. It would be risky to directly add **/var/lib**, **/var**, or the root directory. + +## 3.3 rexec Service Deployment + +### 3.3.1 Introduction + +rexec is a remote execution component developed using the C language. It consists of the rexec client and rexec server. The server is a daemon process, and the client is a binary file. After being started, the client establishes a UDS connection with the server using the udsproxyd service, and the server daemon process starts a specified program on the server machine. During libvirt virtualization offload, libvirtd is offloaded to the DPU. When libvirtd needs to start the QEMU process on the HOST, the rexec client is invoked to remotely start the process. + +### 3.3.2 Deploying rexec + +#### 3.3.2.1 Configuring the Environment Variables and Allowlist + +Configure the rexec server allowlist on the host. Put the **whitelist** file in the **/etc/rexec** directory, and change the file permission to read-only. + +```bash +chmod 400 /etc/rexec/whitelist +``` + +In the test environment, the allowlist is not mandatory. You can disable the allowlist by deleting the **whitelist** file and restarting the rexec_server process. + +After downloading the dpu-utilities code, go to the **qtfs/rexec** directory and run `make && make install` to install all binary files required by rexec (**rexec** and **rexec_server**) to the **/usr/bin** directory. + +Before starting the rexec_server service on the server, check whether the **/var/run/rexec** directory exists. If not, create it. + +```bash +mkdir /var/run/rexec +``` + +#### 3.3.2.2 Starting the Service + +You can start the rexec_server service on the server in either of the following ways. + +* Method 1: + + Configure rexec as a systemd service. + + Add the **[rexec.service](./config/rexec.service)** file to **/usr/lib/systemd/system**. + + Then, use `systemctl` to manage the rexec service. + + Start the service for the first time: + + ```bash + systemctl daemon-reload + + systemctl enable --now rexec + ``` + + Restart the service: + + ```bash + systemctl stop rexec + + systemctl start rexec + ``` + +* Method 2: + + Manually start the service in the background. + + ```bash + nohup /usr/bin/rexec_server 2>&1 & + ``` + +## 3.4 libvirt Service Deployment + +### 3.4.1 Deploying on the HOST + +Install the VM runtime and libvirt. (libvirt is installed to create related directories.) + +```bash +yum install -y qemu libvirt edk2-aarch64 # (required for starting VMs in the Arm environment) +``` + +Put the VM image on the HOST. The VM image will be mounted to the client through qtfs and shared with libvirt. + +### 3.4.2 Deploying on the DPU + +#### 3.4.2.1 Creating the Chroot Environment + +(a) Download the QCOW image from the openEuler official website, for example, openEuler 23.09: . + +(b) Mount the QCOW2 image. + +```bash +cd /root/ + +mkdir p2 new_root_origin new_root + +modprobe nbd maxport=8 + +qemu-nbd -c /dev/nbd0 xxx.qcow2 + +mount /dev/nbd0p2 /root/p2 + +cp -rf /root/p2/* /root/new_root_origin/ + +umount /root/p2 + +qemu-nbd -d /dev/nbd0 +``` + +(c) Now, the root directory of the image is decompressed in **new_root_origin**. Bind mount **new_root** to **new_root_origin** as the mount point for chroot. + +```bash +mount --bind /root/new_root_origin /root/new_root +``` + +#### 3.4.2.2 Installing libvirt + +Compile the source code with a patch. + +(a) Go to the chroot environment and install the compilation environment and common tools. + +```bash +yum groupinstall "Development tools" -y +yum install -y vim meson qemu qemu-img strace edk2-aarch64 tar +``` + +**edk2-aarch64** is required for starting VMs in the Arm environment. + +(b) Install the dependency packages required for libvirt compilation. + +```bash + yum install -y rpcgen python3-docutils glib2-devel gnutls-devel libxml2-devel libpciaccess-devel libtirpc-devel yajl-devel systemd-devel dmidecode glusterfs-api numactl +``` + +(c) Download the libvirt-x.x.x source code package . + +(d) Obtain the libvirt patch: + +. + +(e) Decompress the source code package to a directory in the chroot environment, for example, **/home**, and apply the patch. + +(f) Go to the **libvirt-x.x.x** directory and run the following command: + +```bash +meson build --prefix=/usr -Ddriver_remote=enabled -Ddriver_network=enabled -Ddriver_qemu=enabled -Dtests=disabled -Ddocs=enabled -Ddriver_libxl=disabled -Ddriver_esx=disabled -Dsecdriver_selinux=disabled -Dselinux=disabled +``` + +(g) Complete the installation. + +```bash +ninja -C build install +``` + +#### 3.4.2.3 Starting the libvirtd Service + +To use libvirt direct connection aggregation, you need to start the libvirtd service in the chroot environment, which requires the libvirtd service outside the chroot environment to be stopped. + +(a) Put the [VM jumper script](./scripts/qemu-kvm) in **/usr/bin** and **/usr/libexec** in the chroot environment to replace the **qemu-kvm** binary file. The jumper script will call rexec to start a remote VM. +> [!NOTE]NOTE +> In the XML file of virsh, set **\** under **\** to **qemu-kvm**. If you set **\** to another value, change it to **qemu-kvm** or replace the binary file specified by **\** with the jumper script. The content of the jumper script also needs to be modified accordingly. + +(b) Copy the **libudsproxy.so** file generated during udsproxyd compilation to the **/usr/lib64** directory in the chroot directory. If the udsproxyd service is used by configuring the UDS allowlist of qtfs, you do not need to copy the **libudsproxy.so** file. + +(c) Save the **rexec** binary file generated during rexec compilation to the **/usr/bin** directory of the chroot environment. + +(d) To configure the chroot mounting environment, you need to mount some directories. Use the following scripts: + +* [virt_start.sh](./scripts/virt_start.sh) is the configuration script. In the script, you need to manually change the **qtfs.ko** path to the path of the compiled **.ko** file and set the correct HOST IP address. +* [virt_umount.sh](./scripts/virt_umount.sh) is the configuration revert script. + +(e) The mount directories in the script are based on the examples in this document. You can modify the paths in the script as required. + +(f) After the chroot environment is configured, enter the chroot environment and manually start libvirtd. + +If qtfs is not configured to use the udsproxyd allowlist, run the following commands: + +```bash +LD_PRELOAD=/usr/lib64/libudsproxy.so virtlogd -d +LD_PRELOAD=/usr/lib64/libudsproxy.so libvirtd -d +``` + +If qtfs is configured to use the udsproxyd allowlist, the LD_PRELOAD prefix is not required: + +```bash +virtlogd -d +libvirtd -d +``` + +To check whether the allowlist is configured, run the following command in another terminal that is not in the chroot environment: + +```bash +qtcfg -z +``` + +Check whether the allowlist contains **/var/lib/libvirt**. + +## 3.5 VM Startup + +After the service is deployed, you can manage the VM life cycle from the DPU. + +### 3.5.1 Defining the VM + +(a) Place the VM boot image in a directory on the HOST, for example: + +```bash +/home/VMs/Domain_name +``` + +(b) Use qtfs to mount the directory to the DPU. + +```bash +mount -t qtfs /home/VMs /home/VMs +``` + +(c) In the XML file, **/home/VMs/Domain_name** is used as the boot image. In this way, the same image file is presented to the DPU and HOST (**Domain_name** is the VM **domain**). + +(d) Check whether **\** in the XML file points to the jumper script. + +(e) Define the VM. + +```bash +virsh define xxx.xml +``` + +### 3.5.2 Starting the VM + +```bash +virsh start domain +``` + +# 4 Environment Reset + +Some libvirt directories are shared between the DPU and the HOST. Therefore, you need to unmount these directories before uninstalling the environment. Generally, stop the libvirtd and virtlogd processes and run the **virt_umount.sh** script. If a VM is running on the HOST, stop the VM before unmounting the directories. + +# 5 Common Errors + +1. libvirt compilation failure: Check whether the dependency packages are installed. If an external directory or HOST directory is mounted to the chroot environment, the compilation may fail. In this case, unmount the directory first. + +2. qtfs mounting failure: The engine process on the server is not started or the firewall is not disabled. As a result, the qtfs connection fails. + +3. VM definition failure: Check whether the emulator in the XML file points to the jumper script, whether the VM image has been mounted to the DPU through qtfs, and whether the path is the same as that on the HOST. + +4. VM startup failure: Check whether the libvirtd and virtlogd services are started, whether the rexec service is started, whether the jumper process is started, and whether an error is reported when qemu-kvm is started. diff --git a/docs/en/server/diversified_computing/dpu_offload/offload-deployment-guide.md b/docs/en/server/diversified_computing/dpu_offload/offload-deployment-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..5bc0c03ab107dc7d145e99803602be2a054d0d34 --- /dev/null +++ b/docs/en/server/diversified_computing/dpu_offload/offload-deployment-guide.md @@ -0,0 +1,167 @@ +# Imperceptible Container Management Plane Offload Deployment Guide + +> ![](./public_sys-resources/icon-note.gif)**NOTE**: +> +> In this user guide, modifications are performed to the container management plane components and the rexec tool of a specific version. You can modify other versions based on the actual execution environment. The patch provided in this document is for verification only and is not for commercial use. +> ![](./public_sys-resources/icon-note.gif)**NOTE**: +> +> The communication between shared file systems is implemented through the network. You can perform a simulated offload using two physical machines or VMs connected through the network. +> +> Before the verification, you are advised to set up a Kubernetes cluster and container running environment that can be used properly and offload the management plane process of a single node. You can use a physical machine or VM that is connected to the network as an emulated DPU. + +## Introduction + +Container management plane, that is, management tools of containers such as Kubernetes, dockerd, containerd, and isulad. Container management plane offload is to offload the container management plane from the host where the container is located to another host, that is, the DPU, a set of hardware that has an independent running environment. + +By mounting directories related to container running on the host to the DPU through qtfs, the container management plane tool running on the DPU can access these directories and prepare the running environment for the containers running on the host. To remotely mount the special file systems such as proc and sys, a dedicated rootfs is created as the running environment of Kubernetes and dockerd (referred to as **/another_rootfs**). + +In addition, rexec is used to start and delete containers so that the container management plane and containers can run on two different hosts for remote container management. + +## Related Component Patches + +### rexec + +rexec is a remote execution tool written in the Go language based on the [rexec](https://github.com/docker/libchan/tree/master/examples/rexec) example tool of Docker/libchan. rexec is used to remotely invoke binary files. For ease of use, capabilities such as transferring environment variables and monitoring the exit of original processes are added to rexec. + +To use the rexec tool, run the `CMD_NET_ADDR=tcp://0.0.0.0: rexec_server` command on the server to start the rexec service process, and then run the `CMD_NET_ADDR=tcp://: rexec [command]` on the client`. This instructs rexec_server to execute the command. + +### dockerd + +The changes to dockerd are based on version 18.09. + +In containerd, the part that invokes libnetwork-setkey through hook is commented out. This does not affect container startup. In addition, to ensure the normal use of `docker load`, an error in the `mount` function in **mounter_linux.go** is commented out. + +In the running environment of the container management plane, **/proc** is mounted to the proc file system on the server, and the local proc file system is mounted to **/local_proc**. In dockerd and containerd, **/proc** is changed to **/local_proc** for accessing **/proc/self/xxx**, **/proc/getpid()/xxx**, or related file systems. + +### containerd + +The changes to containerd are based on containerd-1.2-rc.1. + +When obtaining mounting information, **/proc/self/mountinfo** can obtain only the local mounting information of dockerd but cannot obtain that on the server. Therefore, **/proc/self/mountinfo** is changed to **/proc/1/mountinfo** to obtain the mounting information on the server by obtaining the mounting information of process 1 on the server. + +In containerd-shim, the Unix socket that communicates with containerd is changed to TCP. containerd obtains the IP address of the running environment of containerd-shim through the **SHIM_HOST** environment variable, that is, the IP address of the server. The has value of shim is used to generate a port number, which is used as the communication port to start containerd-shim. + +In addition, the original method of sending signals to containerd-shim is changed to the method of remotely invoking the `kill` command to send signals to shim, ensuring that Docker can correctly kill containers. + +### Kubernetes + +kubelet is not modified. The container QoS manager may fail to be configured for the first time. This error does not affect the subsequent pod startup process. + +## Container Management Plane Offload Operation Guide + +Start rexec_server on both the server and client. rexec_server on the server is used to invoke rexec to stat containerd-shim. rexec_server on the client is used to execute invoking of dockerd and containerd by containerd-shim. + +### Server + +Create a folder required by the container management plane, insert **qtfs_server.ko**, and start the engine process. + +In addition, you need to create the rexec script **/usr/bin/dockerd** on the server. + +``` shell +#!/bin/bash +CMD_NET_ADDR=tcp://: rexec /usr/bin/dockerd $* +``` + +### Client + +Prepare a rootfs as the running environment of dockerd and containerd. Use the following script to mount the server directories required by dockerd and containerd to the client. Ensure that the remote directories mounted in the script exist on both the server and client. + +``` shell +#!/bin/bash +mkdir -p /another_rootfs/var/run/docker/containerd +iptables -t nat -N DOCKER +echo "---------insmod qtfs ko----------" +insmod /YOUR/QTFS/PATH/qtfs.ko qtfs_server_ip= qtfs_log_level=INFO + +# The proc file system in the chroot environment is replaced by the proc shared file system of the DPU. The actual proc file system of the local host needs to be mounted to **/local_proc**. +mount -t proc proc /another_rootfs/local_proc/ + +# Bind the chroot internal environment to the external environment to facilitate configuration and running. +mount --bind /var/run/ /another_rootfs/var/run/ +mount --bind /var/lib/ /another_rootfs/var/lib/ +mount --bind /etc /another_rootfs/etc + +mkdir -p /another_rootfs/var/lib/isulad + +# Create and mount the dev, sys, and cgroup file systems in the chroot environment. +mount -t devtmpfs devtmpfs /another_rootfs/dev/ +mount -t sysfs sysfs /another_rootfs/sys +mkdir -p /another_rootfs/sys/fs/cgroup +mount -t tmpfs tmpfs /another_rootfs/sys/fs/cgroup +list="perf_event freezer files net_cls,net_prio hugetlb pids rdma cpu,cpuacct memory devices blkio cpuset" +for i in $list +do + echo $i + mkdir -p /another_rootfs/sys/fs/cgroup/$i + mount -t cgroup cgroup -o rw,nosuid,nodev,noexec,relatime,$i /another_rootfs/sys/fs/cgroup/$i +done + +## common system dir +mount -t qtfs -o proc /proc /another_rootfs/proc +echo "proc" +mount -t qtfs /sys /another_rootfs/sys +echo "cgroup" + +# Mount the shared directory required by the container management plane. +mount -t qtfs /var/lib/docker/containers /another_rootfs/var/lib/docker/containers +mount -t qtfs /var/lib/docker/containerd /another_rootfs/var/lib/docker/containerd +mount -t qtfs /var/lib/docker/overlay2 /another_rootfs/var/lib/docker/overlay2 +mount -t qtfs /var/lib/docker/image /another_rootfs/var/lib/docker/image +mount -t qtfs /var/lib/docker/tmp /another_rootfs/var/lib/docker/tmp +mkdir -p /another_rootfs/run/containerd/io.containerd.runtime.v1.linux/ +mount -t qtfs /run/containerd/io.containerd.runtime.v1.linux/ /another_rootfs/run/containerd/io.containerd.runtime.v1.linux/ +mkdir -p /another_rootfs/var/run/docker/containerd +mount -t qtfs /var/run/docker/containerd /another_rootfs/var/run/docker/containerd +mount -t qtfs /var/lib/kubelet/pods /another_rootfs/var/lib/kubelet/pods +``` + +In**/another_rootfs**, create the following script to support cross-host operations: + +* /another_rootfs/usr/local/bin/containerd-shim + +``` shell +#!/bin/bash +CMD_NET_ADDR=tcp://: /usr/bin/rexec /usr/bin/containerd-shim $* +``` + +* /another_rootfs/usr/local/bin/remote_kill + +``` shell +#!/bin/bash +CMD_NET_ADDR=tcp://: /usr/bin/rexec /usr/bin/kill $* +``` + +* /another_rootfs/usr/sbin/modprobe + +``` shell +#!/bin/bash +CMD_NET_ADDR=tcp://: /usr/bin/rexec /usr/sbin/modprobe $* +``` + +After changing the root directories of dockerd and containerd to the required rootfs, run the following command to start dockerd and containerd: + +* containerd + +``` shell +#!/bin/bash +SHIM_HOST= containerd --config /var/run/docker/containerd/containerd.toml --address /var/run/containerd/containerd.sock +``` + +* dockerd + +``` shell +#!/bin/bash +SHIM_HOST=CMD_NET_ADDR=tcp://: /usr/bin/dockerd --containerd /var/run/containerd/containerd.sock +``` + +* kubelet + +Use the original parameters to start kubelet in the chroot environment. + +Because **/var/run/** is bound to **/another_rootfs/var/run/**, you can use Docker to access the **docker.sock** interface for container management in the regular rootfs. + +The container management plane is offloaded to the DPU. You can run `docker` commands to create and delete containers, or use `kubectl` on the current node to schedule and destroy pods. The actual container service process runs on the host. + +> ![](./public_sys-resources/icon-note.gif)**NOTE**: +> +> This guide describes only the container management plane offload. The offload of container network and data volumes requires additional offload capabilities, which are not included. You can perform cross-node startup of containers that are not configured with network and storage by referring to this guide. diff --git a/docs/en/server/diversified_computing/dpu_offload/overview.md b/docs/en/server/diversified_computing/dpu_offload/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..53ef828f387423867ffc8194bf5cf879c768276c --- /dev/null +++ b/docs/en/server/diversified_computing/dpu_offload/overview.md @@ -0,0 +1,11 @@ +# Imperceptible DPU Offload User Guide + +This document describes the container management plane DPU offload function of openEuler, as well as how to install and deploy it. Through the unified abstraction layer provided by the OS, this function masks the differences in how the container management plane accesses resources across hosts. This makes it possible to offload services from the container management plane to the DPU. + +This document is intended for community developers, open source enthusiasts, and partners who use the openEuler OS and want to learn and use the OS kernel and containers. Users must: + +- Know basic Linux operations. + +- Be familiar with the fundamental mechanisms of the Linux kernel file system. + +- Understand Kubernetes and Docker, as well as how to deploy and use them. diff --git a/docs/en/server/diversified_computing/dpu_offload/public_sys-resources/icon-note.gif b/docs/en/server/diversified_computing/dpu_offload/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/server/diversified_computing/dpu_offload/public_sys-resources/icon-note.gif differ diff --git a/docs/en/server/diversified_computing/dpu_offload/qtfs-architecture-and-usage.md b/docs/en/server/diversified_computing/dpu_offload/qtfs-architecture-and-usage.md new file mode 100644 index 0000000000000000000000000000000000000000..7fe0544418ae42a58daab9eb2e091f945d944416 --- /dev/null +++ b/docs/en/server/diversified_computing/dpu_offload/qtfs-architecture-and-usage.md @@ -0,0 +1,77 @@ +# qtfs Shared File System Architecture and Usage + +## Introduction + +qtfs is a shared file system project. It can be deployed on either a host-DPU hardware architecture or on two hosts. qtfs works in client-server mode, allowing the client to access specified file systems on the server in the same way that local files are accessed. + +qtfs provides the following features: + ++ Mount point propagation + ++ Sharing of special file systems such as proc, sys, and cgroup + ++ Shared read and write of remote files + ++ Remote mounting of server file systems on the client + ++ Customized processing of special files + ++ Remote FIFO, Unix sockets, and epoll that allow the client and server to access the files as if they were like local + ++ Bottom-layer host-DPU communication over the PCIe protocol, outperforming the network + ++ Kernel module development, preventing intrusive modification to the kernel + +## Software Architecture + +![qtfs-arch](./figures/qtfs-arch.png) + +## Installation + +Perform operations in the following qtfs-related directories: + ++ **qtfs**: code of the client kernel module. Compile the client **.ko** file in this directory. + ++ **qtfs_server**: code of the server kernel module. Compile the server **.ko** files and related programs in this directory. + ++ **qtinfo**: diagnosis tool that is used to check the status of file systems and change the log level. + ++ **demo**, **test**, and **doc**: demo programs, test programs, and project documents. + ++ Root directory: code of common modules used by the client and server. + +Configure the kernel compilation environment on two servers (or VMs). + +1. The kernel version must be 5.10 or later. +2. Install the kernel development package by running `yum install kernel-devel`. +3. Assume that the host IP address is 192.168.10.10 and the DPU IP address is 192.168.10.11. + +Install the qtfs server. + +```bash + 1. cd qtfs_server + 2. make clean && make + 3. insmod qtfs_server.ko qtfs_server_ip=192.168.10.10 qtfs_server_port=12345 qtfs_log_level=WARN + 4. nohup ./engine 16 1 192.168.10.10 12121 192.168.10.11 12121 2>&1 & +``` + +Install the qtfs client. + +```bash + 1. cd qtfs + 2. make clean && make + 3. insmod qtfs.ko qtfs_server_ip=192.168.10.10 qtfs_server_port=12345 qtfs_log_level=WARN + 4. cd ../ipc/ + 5. make clean && make && make install + 6. nohup udsproxyd 1 192.168.10.11 12121 192.168.10.10 12121 2>&1 & +``` + +## Usage + +After the installation is complete, mount the server file system to the client. For example: + +```bash + mount -t qtfs / /root/mnt/ +``` + +The file system is visible to the client. Access **/root/mnt** on the client to view and operate files on the server. diff --git a/docs/en/server/diversified_computing/dpu_offload/scripts/qemu-kvm b/docs/en/server/diversified_computing/dpu_offload/scripts/qemu-kvm new file mode 100644 index 0000000000000000000000000000000000000000..e869371be109b57f59709fc23bc5b1cb2002cfbf --- /dev/null +++ b/docs/en/server/diversified_computing/dpu_offload/scripts/qemu-kvm @@ -0,0 +1,3 @@ +#!/bin/bash + +exec /usr/bin/rexec /usr/bin/qemu-kvm $* diff --git a/docs/en/server/diversified_computing/dpu_offload/scripts/virt_start.sh b/docs/en/server/diversified_computing/dpu_offload/scripts/virt_start.sh new file mode 100644 index 0000000000000000000000000000000000000000..06ca194b7a639a947b6e395f116beeba7c897459 --- /dev/null +++ b/docs/en/server/diversified_computing/dpu_offload/scripts/virt_start.sh @@ -0,0 +1,48 @@ +#!/bin/bash +insmod ./qtfs.ko qtfs_server_ip=192.168.10.11 qtfs_log_level=NONE + +systemctl stop libvirtd + +if [ ! -d "/root/new_root/local_proc" ]; then + mkdir -p /root/new_root/local_proc +fi +if [ ! -d "/root/new_root/local" ]; then + mkdir -p /root/new_root/local +fi +mount -t proc proc /root/new_root/local_proc/ +mount -t proc proc /root/new_root/local/proc +mount -t sysfs sysfs /root/new_root/local/sys +mount --bind /var/run/ /root/new_root/var/run/ +mount --bind /var/lib/ /root/new_root/var/lib/ +mount --bind /var/cache/ /root/new_root/var/cache +mount --bind /etc /root/new_root/etc + +mkdir -p /root/new_root/home/VMs/ +mount -t qtfs /home/VMs/ /root/new_root/home/VMs/ + +mount -t qtfs /var/lib/libvirt /root/new_root/var/lib/libvirt + +mount -t devtmpfs devtmpfs /root/new_root/dev/ +mount -t hugetlbfs hugetlbfs /root/new_root/dev/hugepages/ +mount -t mqueue mqueue /root/new_root/dev/mqueue/ +mount -t tmpfs tmpfs /root/new_root/dev/shm + +mount -t sysfs sysfs /root/new_root/sys +mkdir -p /root/new_root/sys/fs/cgroup +mount -t tmpfs tmpfs /root/new_root/sys/fs/cgroup +list="perf_event freezer files net_cls,net_prio hugetlb pids rdma cpu,cpuacct memory devices blkio cpuset" +for i in $list +do + echo $i + mkdir -p /root/new_root/sys/fs/cgroup/$i + mount -t cgroup cgroup -o rw,nosuid,nodev,noexec,relatime,$i /root/new_root/sys/fs/cgroup/$i +done + +## common system dir +mount -t qtfs -o proc /proc /root/new_root/proc +echo "proc" + +mount -t qtfs /sys /root/new_root/sys +echo "cgroup" +mount -t qtfs /dev/pts /root/new_root/dev/pts +mount -t qtfs /dev/vfio /root/new_root/dev/vfio diff --git a/docs/en/server/diversified_computing/dpu_offload/scripts/virt_umount.sh b/docs/en/server/diversified_computing/dpu_offload/scripts/virt_umount.sh new file mode 100644 index 0000000000000000000000000000000000000000..4adddec913c23069c6bffddec0bf1770f8c5ce71 --- /dev/null +++ b/docs/en/server/diversified_computing/dpu_offload/scripts/virt_umount.sh @@ -0,0 +1,20 @@ +#!/bin/bash + +umount /root/new_root/dev/hugepages +umount /root/new_root/etc +umount /root/new_root/home/VMs +umount /root/new_root/local_proc +umount /root/new_root/local/proc +umount /root/new_root/var/lib/libvirt +umount /root/new_root/var/lib +umount /root/new_root/* +umount /root/new_root/dev/pts +umount /root/new_root/dev/mqueue +umount /root/new_root/dev/shm +umount /root/new_root/dev/vfio +umount /root/new_root/dev +rmmod qtfs + +umount /root/new_root/sys/fs/cgroup/* +umount /root/new_root/sys/fs/cgroup +umount /root/new_root/sys diff --git a/docs/en/server/diversified_computing/dpu_os/_toc.yaml b/docs/en/server/diversified_computing/dpu_os/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..d4308bb9509bca32d67421481c8a410132c0656d --- /dev/null +++ b/docs/en/server/diversified_computing/dpu_os/_toc.yaml @@ -0,0 +1,13 @@ +label: DPU-OS +isManual: true +href: ./overview.md +description: >- + This guide outlines the process of creating a DPU-OS image through openEuler + OS customization, including deployment and validation procedures. +sections: + - label: DPU-OS Background and Requirements + href: ./dpu-os-background-and-requirements.md + - label: DPU-OS Tailoring Guide + href: ./dpu-os-tailoring-guide.md + - label: Verification and Deployment + href: ./verification-and-deployment.md diff --git a/docs/en/server/diversified_computing/dpu_os/dpu-os-background-and-requirements.md b/docs/en/server/diversified_computing/dpu_os/dpu-os-background-and-requirements.md new file mode 100644 index 0000000000000000000000000000000000000000..849d841464bf3af5100d9aa7317794ebc6e8722b --- /dev/null +++ b/docs/en/server/diversified_computing/dpu_os/dpu-os-background-and-requirements.md @@ -0,0 +1,67 @@ +# DPU-OS Background and Requirements + +## Overview + +In data center and cloud environments, Moore's Law has reached its limits, leading to a slowdown in the growth of general-purpose CPU computing power. At the same time, network I/O speeds and performance continue to rise, creating a growing disparity between the two. This gap highlights the inability of current general-purpose processors to meet the demands of network, disk, and other I/O processing. In traditional data centers, a significant portion of general-purpose CPU resources is consumed by I/O and management tasks, a phenomenon known as the "Datacenter Tax." AWS estimates that this tax can consume over 30% of a data center's computing power, and in some cases, even more. + +The DPU was introduced to address this issue by offloading management, network, storage, and security tasks from the host CPU to dedicated processor chips. This offloading accelerates processing, reduces costs, and improves efficiency. Leading cloud providers like AWS, Alibaba Cloud, and Huawei Cloud have developed custom chips to handle these offloaded tasks, ensuring that 100% of data center computing resources are available for customer use. + +The DPU market is experiencing rapid growth, driven by strong demand from cloud providers and big data applications. Numerous Chinese DPU startups have also entered the market with innovative products. This growth presents challenges for cloud and big data providers, who must integrate diverse DPU products, and for DPU manufacturers, who must adapt device drivers to customer-specified operating systems. openEuler, a leading open-source operating system in China, addresses these challenges by offering DPU-OS, a solution built on openEuler that bridges the gap between DPU manufacturers and customers. Furthermore, since DPUs rely on their OS to support service acceleration, DPU-OS requires performance optimization. By leveraging openEuler, DPU-related acceleration capabilities can be embedded into DPU-OS, fostering a robust DPU software ecosystem. + +## DPU-OS Requirements Analysis and Design + +### Current State of DPUs and OS Requirements + +DPUs exhibit several key characteristics and challenges: + +- Limited general-purpose processing resources + + DPUs are in the early stages of development, with hardware continuously evolving. Power constraints result in modest hardware specifications. Mainstream DPUs typically feature 8 to 24 CPU cores with limited single-core performance. Memory capacity ranges from 16 to 32GB, and local storage varies from tens to hundreds of gigabytes. The operating system running on DPUs must accommodate these constraints. + +- Varied DPU-OS installation methods + + The diversity of DPU manufacturers and products has led to multiple installation and deployment methods. These include PXE network installation, USB installation, and custom methods such as host-delivered installation images. + +- High performance requirements + + DPU application scenarios demand high performance. Compared to general-purpose server operating systems, DPU-OS may require specific kernel features or functional components. Examples include vDPA for device passthrough and live migration, vendor-specific driver support, seamless DPU process offloading, customized user-space data plane acceleration tools like DPDK/SPDK/OVS, and DPU management and monitoring tools. + +Based on these characteristics, the following requirements for DPU-OS are proposed: + +- Ultra-lightweight DPU-OS installation package + + Trim the openEuler system image to eliminate unnecessary packages and optimize system services to reduce resource overhead. + +- Customization support and tools + + Provide customization configurations and tools to enable customers or DPU manufacturers to tailor the system. openEuler offers an ISO reference implementation. + +- Customized kernel and system for peak performance + + Customize the kernel and drivers to deliver competitive features for DPUs. Enable hardware acceleration through tailored components and optimize system configurations for superior performance. Include DPU-related management and control tools for unified administration. + +### DPU-OS Design + +**Figure 1** Overall Design of DPU-OS + +![dpuos-arch](./figures/dpuos-arch.png) + +As illustrated in Figure 1, DPU-OS is structured into five layers: + +- **Kernel layer**: Customize the kernel configuration to remove non-essential features and modules, creating a lightweight kernel. Enable specific kernel features to deliver high-performance DPU capabilities. + +- **Driver layer**: Trim and customize openEuler native drivers, selecting the minimal required set. Integrate DPU vendor-specific drivers to natively support certain DPU hardware products. + +- **System configuration layer**: Optimize system settings through sysctl and proc configurations to ensure peak performance for DPU-related services. + +- **Peripheral package layer**: Customize and trim openEuler peripheral packages, selecting the minimal set. Provide a suite of DPU-related custom tools. + +- **System service layer**: Streamline native system service startup items to eliminate unnecessary services, minimizing runtime overhead. + +This five-layer design achieves the goal of a lightweight, high-performance DPU-OS. While this is a long-term design heavily reliant on the DPU software and hardware ecosystem, the current phase focuses on trimming using openEuler's imageTailor tool. + +For detailed steps on DPU-OS trimming, refer to the [DPU-OS Tailoring Guide](./dpu-os-tailoring-guide.md). For verification and deployment, consult the [DPU-OS Deployment and Verification Guide](./verification-and-deployment.md). + +> ![](./public_sys-resources/icon-note.gif)**Note**: +> +> Currently, DPU-OS leverages openEuler's existing kernel and peripheral packages, trimmed using the imageTailor tool to produce a lightweight OS installation image. Future development will integrate additional kernel and peripheral package features based on specific needs. diff --git a/docs/en/server/diversified_computing/dpu_os/dpu-os-tailoring-guide.md b/docs/en/server/diversified_computing/dpu_os/dpu-os-tailoring-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..05e25a8519f98d24a5bf508dae7c58a6200d7358 --- /dev/null +++ b/docs/en/server/diversified_computing/dpu_os/dpu-os-tailoring-guide.md @@ -0,0 +1,65 @@ +# DPU-OS Tailoring Guide + +This document explains how to use imageTailor to trim the DPU-OS installation image using configuration files from the [dpu-utilities repository](https://gitee.com/openeuler/dpu-utilities/tree/master/dpuos). Follow these steps: + +## Prepare imageTailor and Required RPM Packages + +Install the imageTailor tool by referring to the [imageTailor User Guide](https://docs.openeuler.org/zh/docs/22.03_LTS/docs/TailorCustom/imageTailor%E4%BD%BF%E7%94%A8%E6%8C%87%E5%8D%97.html) and prepare the necessary RPM packages for tailoring. + +You can use the openEuler installation image as the RPM source. While **openEuler-22.03-LTS-everything-debug-aarch64-dvd.iso** contains a complete set of RPMs, it is large. Alternatively, use the RPMs from **openEuler-22.03-LTS-aarch64-dvd.iso** along with the install-scripts.noarch package. + +Obtain the `install-scripts.noarch` package from the everything repository or download it using yum: + +```bash +yum install -y --downloadonly --downloaddir=./ install-scripts +``` + +## Copy DPUOS Configuration Files + +The imageTailor tool is installed in **/opt/imageTailor** by default. Copy the DPU-OS configuration files to the appropriate paths, selecting the correct architecture directory. The DPU-OS tailoring configuration repository supports x86_64 and aarch64 architectures. + +```bash +cp -rf custom/cfg_dpuos /opt/imageTailor/custom +cp -rf kiwi/minios/cfg_dpuos /opt/imageTailor/kiwi/minios/cfg_dpuos +``` + +## Modify Other Configuration Files + +- Add a line for `dpuos` configuration in **kiwi/eulerkiwi/product.conf**: + +```bash +dpuos PANGEA EMBEDDED DISK GRUB2 install_mode=install install_media=CD install_repo=CD selinux=0 +``` + +- Add a line for `dpuos` configuration in **kiwi/eulerkiwi/minios.conf**: + +```bash +dpuos kiwi/minios/cfg_dpuos yes +``` + +- Add a line for `dpuos` configuration in **repos/RepositoryRule.conf**: + +```bash +dpuos 1 rpm-dir euler_base +``` + +## Set Passwords + +Navigate to **/opt/imageTailor** and update the passwords in the following files: + +- **custom/cfg_dpuos/usr_file/etc/default/grub** + +- **custom/cfg_dpuos/rpm.conf** + +- **kiwi/minios/cfg_dpuos/rpm.conf** + +For password generation and modification, refer to the openEuler imageTailor manual section on [Configuring Initial Passwords](../../../Tools/CommunityTools/ImageCustom/imageTailor/imagetailor-user-guide.md#configuring-initial-passwords). + +## Execute the Tailoring Command + +Run the following command to perform the tailoring. The resulting ISO will be saved in **/opt/imageTailor/result**: + +```bash +cd /opt/imageTailor +./mkdliso -p dpuos -c custom/cfg_dpuos --sec --minios force +``` diff --git a/docs/en/server/diversified_computing/dpu_os/figures/dpuos-arch.png b/docs/en/server/diversified_computing/dpu_os/figures/dpuos-arch.png new file mode 100644 index 0000000000000000000000000000000000000000..453370ab07858a13a6c40f8d22e3f608e9ec6b4c Binary files /dev/null and b/docs/en/server/diversified_computing/dpu_os/figures/dpuos-arch.png differ diff --git a/docs/en/server/diversified_computing/dpu_os/overview.md b/docs/en/server/diversified_computing/dpu_os/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..89d83786b9a29940803a05f959d209dc6d9f1c4c --- /dev/null +++ b/docs/en/server/diversified_computing/dpu_os/overview.md @@ -0,0 +1,11 @@ +# Overview + +This document outlines the background requirements and design principles of DPU-OS. It also details the process of creating a DPU-OS image by customizing the openEuler operating system, along with deployment and verification methods. The feature leverages the openEuler ecosystem to deliver a lightweight, high-performance DPU-OS, offering a reference implementation for data processing unit (DPU) scenarios and users. + +This document targets community developers, DPU vendors, and customers using openEuler who want to explore and adopt DPUs. Users should possess the following skills and knowledge: + +- Proficiency in basic Linux operations. + +- Understanding of Linux system construction and deployment fundamentals. + +- Familiarity with the openEuler imageTailor tool for image customization. diff --git a/docs/en/server/diversified_computing/dpu_os/public_sys-resources/icon-note.gif b/docs/en/server/diversified_computing/dpu_os/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/server/diversified_computing/dpu_os/public_sys-resources/icon-note.gif differ diff --git a/docs/en/server/diversified_computing/dpu_os/verification-and-deployment.md b/docs/en/server/diversified_computing/dpu_os/verification-and-deployment.md new file mode 100644 index 0000000000000000000000000000000000000000..0a468aac1b889ad465589b008fc2acac7fc12c05 --- /dev/null +++ b/docs/en/server/diversified_computing/dpu_os/verification-and-deployment.md @@ -0,0 +1,38 @@ +# Verification and Deployment + +Once DPU-OS is built, it can be installed and deployed for verification. Since DPU hardware is still in its early stages, you can also use VirtualBox to set up a virtual machine for deployment and testing. + +## Deploying DPU-OS on VirtualBox + +This section outlines the steps to install and deploy DPU-OS using the VirtualBox hypervisor. + +### Preparation for Verification + +Before deploying DPU-OS, ensure the following prerequisites are met: + +- Obtain the DPU-OS ISO file. +- Ensure the host machine has VirtualBox installed. + +### Initial Installation and Startup + +#### Creating a Virtual Machine + +Create a new virtual machine in VirtualBox: + +- Configure the virtual machine with at least 2 CPUs and 4GB of RAM. + +- Allocate a virtual disk with a recommended size of 60GB or larger. + +- Enable EFI boot in the system extension properties. + +- In the storage settings, select the local DPU-OS ISO file as the optical drive. + +- Customize other settings such as network or display as needed. + +#### Starting the Virtual Machine + +Start the newly created virtual machine and choose **Install from ISO** to begin the DPU-OS installation. The installation process is automated and requires no manual input. After installation, the system will reboot automatically. + +Select **Boot From Local Disk** to start DPU-OS. Use the password specified during the DPU-OS creation process. + +By following these steps, you can successfully deploy and verify DPU-OS locally. diff --git a/docs/en/server/high_availability/ha/_toc.yaml b/docs/en/server/high_availability/ha/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..e84be387e4ec03eac8c6f5eca76459ddf8532735 --- /dev/null +++ b/docs/en/server/high_availability/ha/_toc.yaml @@ -0,0 +1,8 @@ +label: HA User Guide +isManual: true +description: HA cluster installation and usage +sections: + - label: HA Installation and Deployment + href: ./ha-installation-and-deployment.md + - label: HA Usage Examples + href: ./ha-usage-examples.md diff --git a/docs/en/server/high_availability/ha/figures/HA-add-resource.png b/docs/en/server/high_availability/ha/figures/HA-add-resource.png new file mode 100644 index 0000000000000000000000000000000000000000..ac24895a1247828d248132f6c789ad8ef51a57e4 Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-add-resource.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-apache-show.png b/docs/en/server/high_availability/ha/figures/HA-apache-show.png new file mode 100644 index 0000000000000000000000000000000000000000..c216500910f75f2de1108f6b618c5c08f4df8bae Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-apache-show.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-apache-suc.png b/docs/en/server/high_availability/ha/figures/HA-apache-suc.png new file mode 100644 index 0000000000000000000000000000000000000000..23a7aaa702e3e68190ff7e01a5a673aee2c92409 Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-apache-suc.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-api.png b/docs/en/server/high_availability/ha/figures/HA-api.png new file mode 100644 index 0000000000000000000000000000000000000000..f825fe005705d30809d12df97958cff0e5a80135 Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-api.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-clone-suc.png b/docs/en/server/high_availability/ha/figures/HA-clone-suc.png new file mode 100644 index 0000000000000000000000000000000000000000..4b6099ccc88d4f6f907a0c4563e729ab2a4dece1 Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-clone-suc.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-clone.png b/docs/en/server/high_availability/ha/figures/HA-clone.png new file mode 100644 index 0000000000000000000000000000000000000000..1b09ab73849494f4ffd759fa612ae3c241bd9c1d Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-clone.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-corosync.png b/docs/en/server/high_availability/ha/figures/HA-corosync.png new file mode 100644 index 0000000000000000000000000000000000000000..c4d93242e65c503b6e1b6a457e2517f647984a66 Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-corosync.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-firstchoice-cmd.png b/docs/en/server/high_availability/ha/figures/HA-firstchoice-cmd.png new file mode 100644 index 0000000000000000000000000000000000000000..a265bab07f1d8e46d9d965975be180a8de6c9eb2 Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-firstchoice-cmd.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-firstchoice.png b/docs/en/server/high_availability/ha/figures/HA-firstchoice.png new file mode 100644 index 0000000000000000000000000000000000000000..bd982ddcea55c629c0257fca86051a9ffa77e7b4 Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-firstchoice.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-group-new-suc.png b/docs/en/server/high_availability/ha/figures/HA-group-new-suc.png new file mode 100644 index 0000000000000000000000000000000000000000..437fd01ee83a9a1f65c12838fe56eea8435f6759 Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-group-new-suc.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-group-new-suc2.png b/docs/en/server/high_availability/ha/figures/HA-group-new-suc2.png new file mode 100644 index 0000000000000000000000000000000000000000..4fb933bd761f9808de95a324a50226ff041ebd4f Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-group-new-suc2.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-group-new.png b/docs/en/server/high_availability/ha/figures/HA-group-new.png new file mode 100644 index 0000000000000000000000000000000000000000..9c914d0cc2e14f3220fc4346175961f129efb37b Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-group-new.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-group-suc.png b/docs/en/server/high_availability/ha/figures/HA-group-suc.png new file mode 100644 index 0000000000000000000000000000000000000000..2338580343833ebab08627be3a2efbcdb48aef9e Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-group-suc.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-group.png b/docs/en/server/high_availability/ha/figures/HA-group.png new file mode 100644 index 0000000000000000000000000000000000000000..6897817665dee90c0f8c47c6a3cb4bb09db52d78 Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-group.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-home-page.png b/docs/en/server/high_availability/ha/figures/HA-home-page.png new file mode 100644 index 0000000000000000000000000000000000000000..c9a7a82dc412250d4c0984b3876c6f93c6aca789 Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-home-page.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-login.png b/docs/en/server/high_availability/ha/figures/HA-login.png new file mode 100644 index 0000000000000000000000000000000000000000..65d0ae11ec810da7574ec72bebf6e1b020c94a0d Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-login.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-mariadb-suc.png b/docs/en/server/high_availability/ha/figures/HA-mariadb-suc.png new file mode 100644 index 0000000000000000000000000000000000000000..6f6756c945121715edc623bd9a848bc48ffeb4ca Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-mariadb-suc.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-mariadb.png b/docs/en/server/high_availability/ha/figures/HA-mariadb.png new file mode 100644 index 0000000000000000000000000000000000000000..d29587c8609b9d6aefeb07170901361b5ef8402d Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-mariadb.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-nfs-suc.png b/docs/en/server/high_availability/ha/figures/HA-nfs-suc.png new file mode 100644 index 0000000000000000000000000000000000000000..c0ea6af79e91649f1ad7d97ab6c2a0069a4f4fb8 Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-nfs-suc.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-nfs.png b/docs/en/server/high_availability/ha/figures/HA-nfs.png new file mode 100644 index 0000000000000000000000000000000000000000..f6917938eec2e0431a9891c067475dd0b21c1bd9 Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-nfs.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-pacemaker.png b/docs/en/server/high_availability/ha/figures/HA-pacemaker.png new file mode 100644 index 0000000000000000000000000000000000000000..7681f963f67d2b803fef6fb2c3247384136201f8 Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-pacemaker.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-pcs-status.png b/docs/en/server/high_availability/ha/figures/HA-pcs-status.png new file mode 100644 index 0000000000000000000000000000000000000000..fb150fba9f6258658702b35caacf98076d1fd109 Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-pcs-status.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-pcs.png b/docs/en/server/high_availability/ha/figures/HA-pcs.png new file mode 100644 index 0000000000000000000000000000000000000000..283670d7c3d0961ee1cb41345c2b2a013d7143b0 Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-pcs.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-refresh.png b/docs/en/server/high_availability/ha/figures/HA-refresh.png new file mode 100644 index 0000000000000000000000000000000000000000..c2678c0c2945acbabfbeae0d5de8924a216bbf31 Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-refresh.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-vip-suc.png b/docs/en/server/high_availability/ha/figures/HA-vip-suc.png new file mode 100644 index 0000000000000000000000000000000000000000..313ce56e14f931c78dad4349ed57ab3fd7907f50 Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-vip-suc.png differ diff --git a/docs/en/server/high_availability/ha/figures/HA-vip.png b/docs/en/server/high_availability/ha/figures/HA-vip.png new file mode 100644 index 0000000000000000000000000000000000000000..d8b417df2e64527d3b29d0289756dfbb01bf66ec Binary files /dev/null and b/docs/en/server/high_availability/ha/figures/HA-vip.png differ diff --git a/docs/en/server/high_availability/ha/ha-installation-and-deployment.md b/docs/en/server/high_availability/ha/ha-installation-and-deployment.md new file mode 100644 index 0000000000000000000000000000000000000000..8f3f4a6fe609c16f0fd629280cde0682fa023db7 --- /dev/null +++ b/docs/en/server/high_availability/ha/ha-installation-and-deployment.md @@ -0,0 +1,199 @@ +# HA Installation and Deployment + +This document describes how to install and deploy an HA cluster. + +## Installation and Deployment + +- Prepare the environment: At least two physical machines or VMs with openEuler installed are required. (This section uses two physical machines or VMs as an example.) For details about how to install openEuler 21.03, see the [_openEuler Installation Guide_](../../installation_upgrade/installation/installation.md). + +### Modifying the Host Name and the /etc/hosts File + +- **Note: You need to perform the following operations on both hosts. The following takes one host as an example. IP addresses in this document are for reference only.** + +Before using the HA software, ensure that all host names have been changed and written into the **/etc/hosts** file. + +- Run the following command to change the host name: + +```shell +hostnamectl set-hostname ha1 +``` + +- Edit the **/etc/hosts** file and write the following fields: + +```text +172.30.30.65 ha1 +172.30.30.66 ha2 +``` + +### Configuring the Yum Repository + +After the system is successfully installed, the Yum source is configured by default. The file location is stored in the **/etc/yum.repos.d/openEuler.repo** file. The HA software package uses the following sources: + +```text +[OS] +name=OS +baseurl=http://repo.openeuler.org/openEuler-23.09/OS/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-23.09/OS/$basearch/RPM-GPG-KEY-openEuler + +[everything] +name=everything +baseurl=http://repo.openeuler.org/openEuler-23.09/everything/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-23.09/everything/$basearch/RPM-GPG-KEY-openEuler + +[EPOL] +name=EPOL +baseurl=http://repo.openeuler.org/openEuler-23.09/EPOL/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-23.09/OS/$basearch/RPM-GPG-KEY-openEuler +``` + +### Installing the HA Software Package Components + +```shell +yum install -y corosync pacemaker pcs fence-agents fence-virt corosync-qdevice sbd drbd drbd-utils +``` + +### Setting the hacluster User Password + +```shell +passwd hacluster +``` + +### Modifying the /etc/corosync/corosync.conf File + +```text +totem { + version: 2 + cluster_name: hacluster + crypto_cipher: none + crypto_hash: none +} +logging { + fileline: off + to_stderr: yes + to_logfile: yes + logfile: /var/log/cluster/corosync.log + to_syslog: yes + debug: on + logger_subsys { + subsys: QUORUM + debug: on + } +} +quorum { + provider: corosync_votequorum + expected_votes: 2 + two_node: 1 + } +nodelist { + node { + name: ha1 + nodeid: 1 + ring0_addr: 172.30.30.65 + } + node { + name: ha2 + nodeid: 2 + ring0_addr: 172.30.30.66 + } + } +``` + +### Managing the Services + +#### Disabling the firewall + +1. Stop the firewall. + + ```shell + systemctl stop firewalld + ``` + +2. Change the status of SELINUX in the **/etc/selinux/config** file to disabled. + + ```text + # SELINUX=disabled + ``` + +#### Managing the pcs service + +1. Start the pcs service. + + ```shell + systemctl start pcsd + ``` + +2. Query the pcs service status. + + ```shell + systemctl status pcsd + ``` + + The service is started successfully if the following information is displayed: + + ![](./figures/HA-pcs.png) + +#### Managing the Pacemaker service + +1. Start the Pacemaker service. + + ```shell + systemctl start pacemaker + ``` + +2. Query the Pacemaker service status. + + ```shell + systemctl status pacemaker + ``` + + The service is started successfully if the following information is displayed: + + ![](./figures/HA-pacemaker.png) + +#### Managing the Corosync service + +1. Start the Corosync service. + + ```shell + systemctl start corosync + ``` + +2. Query the Corosync service status. + + ```shell + systemctl status corosync + ``` + + The service is started successfully if the following information is displayed: + + ![](./figures/HA-corosync.png) + +### Performing Node Authentication + +- **Note: Run this command on any node.** + +```shell +pcs host auth ha1 ha2 +``` + +### Accessing the Front-End Management Platform + +After the preceding services are started, open the browser (Chrome or Firefox is recommended) and enter `https://localhost:2224` in the navigation bar. + +- This page is the native management platform. + +![](./figures/HA-login.png) + +For details about how to install the management platform newly developed by the community, see . + +- The following is the management platform newly developed by the community. + +![](./figures/HA-api.png) + +- For how to quickly use an HA cluster and add an instance, see the [HA Usage Example](./ha-usage-examples.md/). diff --git a/docs/en/server/high_availability/ha/ha-usage-examples.md b/docs/en/server/high_availability/ha/ha-usage-examples.md new file mode 100644 index 0000000000000000000000000000000000000000..122d4ae29c8fc731498e9ff2078613f86ea00c38 --- /dev/null +++ b/docs/en/server/high_availability/ha/ha-usage-examples.md @@ -0,0 +1,248 @@ +# HA Usage Examples + +This section describes how to get started with the HA cluster and add an instance. If you are not familiar with HA cluster installation, see [HA Installation and Deployment](./ha-installation-and-deployment.md). + +## Quick Start Guide + +The following operations use the management platform newly developed by the community as an example. + +### Login Page + +The user name is `hacluster`, and the password is the one set on the host by the user. + +![](./figures/HA-api.png) + +### Home page + +After logging in to the system, the main page is displayed. The main page consists of the side navigation bar, the top operation area, the resource node list area, and the node operation floating area. + +The following describes the features and usage of the four areas in detail. + +![](./figures/HA-home-page.png) + +#### Navigation bar + +The side navigation bar consists of two parts: the name and logo of the HA cluster software, and the system navigation. The system navigation consists of three parts: **System**, **Cluster Configurations**, and **Tools**. **System** is the default option and the corresponding item to the home page. It displays the information and operation entries of all resources in the system. **Preference Settings** and **Heartbeat Configurations** are set under **Cluster Configurations**. **Log Download** and **Quick Cluster Operation** are set under **Tools**. These two items are displayed in a pop-up box after you click them. + +#### Top Operation Area + +The current login user is displayed statically. When you hover the mouse cursor on the user icon, the operation menu items are displayed, including **Refresh Settings** and **Log Out**. After you click **Refresh Settings**, the **Refresh Settings** dialog box is displayed with the **Refresh Settings** option. You can set the automatic refresh modes for the system, the options are **Do not refresh automatically**, **Refresh every 5 seconds**, and **Refresh every 10 seconds**. By default, **Do not refresh automatically** is selected. Click **Log Out** to log out and jump to the login page. After that, a re-login is required if you want to continue to access the system. + +![](./figures/HA-refresh.png) + +#### Resource Node List Area + +The resource node list displays the resource information such as **Resource Name**, **Status**, **Resource Type**, **Service**, and **Running Node** of all resources in the system, and the node information such as all nodes in the system and the running status of the nodes. In addition, you can **Add**, **Edit**, **Start**, **Stop**, **Clear**, **Migrate**, **Migrate Back**, **Delete**, and **Associate** the resources. + +#### Node Operation Floating Area + +By default, the node operation floating area is collapsed. When you click a node in the heading of the resource node list, the node operation area is displayed on the right, as shown in the preceding figure. This area consists of the collapse button, the node name, the stop button, and the standby button, and provides the stop and standby operations. Click the arrow in the upper left corner of the area to collapse the area. + +### Preference Settings + +The following operations can be performed using command lines. The following is a simple example. For more command details, run the `pcs --help` command. + +- Through the CLI + + ```shell + # pcs property set stonith-enabled=false + # pcs property set no-quorum-policy=ignore + ``` + + Run the following command to view all configurations: + + ```shell + pcs property + ``` + + ![](./figures/HA-firstchoice-cmd.png) + +- Through the GUI + Clicking **Preference Settings** in the navigation bar, the **Preference Settings** dialog box is displayed. Change the values of **No Quorum Policy** and **Stonith Enabled** from the default values to the values shown in the following figure. Then, click OK. + + ![](./figures/HA-firstchoice.png) + +### Add Resource + +#### Adding Common Resources + +1. Click **Add Common Resource**. The **Create Resource** dialog box is displayed. + All mandatory configuration items of a resource are displayed on the **Basic** page. After you select a resource type on the **Basic** page, other mandatory and optional configuration items of the resource are displayed. + +2. Enter the resource configuration information. + A gray text area is displayed on the right of the dialog box to describe the current configuration item. After all mandatory parameters are set, click **OK** to create a common resource or click **Cancel** to cancel the add operation. + The optional configuration items on the **Instance Attribute**, **Meta Attribute**, or **Operation Attribute** page are optional. If they are not configured, the resource creation process is not affected. You can modify them as required. Otherwise, the default values are used. + +The following uses Apache as an example to describe how to add resources through the CLI and GUI. + +- Through the CLI + + ```shell + # pcs resource create httpd ocf:heartbeat:apache + ``` + + Check the resource running status: + + ```shell + # pcs status + ``` + + ![](./figures/HA-pcs-status.png) + +- Through the GUI + +1. Enter the resource name and resource type, as shown in the following figure. + + ![](./figures/HA-add-resource.png) + +2. If the following information is displayed, the resource is successfully added and started, and runs on a node, for example, ha1. + + ![](./figures/HA-apache-suc.png) +3. Access the Apache page. + + ![](./figures/HA-apache-show.png) + +#### Adding Group Resources + +>**Note:** +> Adding group resources requires at least one common resource in the cluster. + +1. Click **Add Group Resource**. The **Create Resource** dialog box is displayed. + All the parameters on the **Basic** tab page are mandatory. After setting the parameters, click **OK** to add the resource or click **Cancel** to cancel the add operation. + + ![](./figures/HA-group.png) + + > **Notes:** + > Group resources are started in the sequence of child resources. Therefore, you need to select child resources in sequence. + +2. If the following information is displayed, the resource is added successfully. + + ![](./figures/HA-group-suc.png) + +#### Adding Clone Resources + +1. Click **Add Clone Resource**. The **Create Resource** dialog box is displayed. + On the **Basic** page, enter the object to be cloned. The resource name is automatically generated. After entering the object name, click **OK** to add the resource, or click **Cancel** to cancel the add operation. + + ![](./figures/HA-clone.png) + +2. If the following information is displayed, the resource is added successfully. + + ![](./figures/HA-clone-suc.png) + +### Editing Resources + +- Starting a resource: Select a target resource from the resource node list. The target resource must not be running. Start the resource. +- Stopping a resource: Select a target resource from the resource node list. The target resource must be running. Stop the resource. +- Clearing a resource: Select a target resource from the resource node list. Clear the resource. +- Migrating a resource: Select a target resource from the resource node list. The resource must be a common resource or group resource in the running status. Migrate the resource to migrate it to a specified node. +- Migrating back a resource: Select a target resource from the resource node list. The resource must be a migrated resource. Migrate back the resource to clear the migration settings of the resource and migrate the resource back to the original node. After you click **Migrate Back**, the status change of the resource item in the list is the same as that when the resource is started. +- Deleting a resource: Select a target resource from the resource node list. Delete the resource. + +### Setting Resource Relationships + +Resource relationships are used to set restrictions for the target resources. There are three types resource restrictions: resource location, resource collaboration, and resource order. + +- Resource location: sets the running level of the resource on the nodes in the cluster to determine the node where the resource runs during startup or switchover. The running levels are Master Node and Slave 1 in descending order. +- Resource collaboration: indicates whether the target resource and other resources in the cluster run on the same node. **Same Node** indicates that this node must run on the same node as the target resource. **Mutually Exclusive** indicates that this node cannot run on the same node as the target resource. +- Resource order: Set the order in which the target resource and other resources in the cluster are started. **Front Resource** indicates that this resource must be started before the target resource. **Follow-up Resource** indicates that this resource can be started only after the target resource is started. + +## HA MySQL Configuration Example + +### Configuring the Virtual IP Address + +1. On the home page, choose **Add** > **Add Common Resource**, and set the parameters as follows: + + ![](./figures/HA-vip.png) + +2. The resource is successfully created and started, and runs on a node, for example, ha1. +3. The IP address can be pinged and connected. After login, you can perform various operations normally. Resources can be switched to ha2 and can be accessed normally. See the following figure. + ![](./figures/HA-vip-suc.png) + +### Configuring NFS Storage + +Perform the following steps to configure another host as the NFS server: + +1. Install the software package. + + ```shell + # yum install -y nfs-utils rpcbind + ``` + +2. Disable the firewall. + + ```shell + # systemctl stop firewalld && systemctl disable firewalld + ``` + +3. Modify the /etc/selinux/config file to change the status of SELinux to disabled. + + ```shell + # SELINUX=disabled + ``` + +4. Start services. + + ```shell + # systemctl start rpcbind && systemctl enable rpcbind + # systemctl start nfs-server && systemctl enable nfs-server + ``` + +5. Create a shared directory on the server. + + ```shell + # mkdir -p /test + ``` + +6. Modify the NFS configuration file. + + ```shell + # vim /etc/exports + # /test *(rw,no_root_squash) + ``` + +7. Reload the service. + + ```shell + # systemctl reload nfs + ``` + +8. Install the software package on the client. Install MySQL first and then mount NFS to the MySQL data path. + + ```shell + # yum install -y nfs-utils mariadb-server + ``` + +9. On the home page, choose **Add** > **Add Common Resource** and configure the NFS resource as follows: + + ![](./figures/HA-nfs.png) + +10. The resource is successfully created and started, and runs on a node, for example, ha1. The NFS is mounted to the `/var/lib/mysql` directory. The resource is switched to ha2. The NFS is unmounted from ha1 and automatically mounted to ha2. See the following figure. + + ![](./figures/HA-nfs-suc.png) + +### Configuring MySQL + +1. On the home page, choose **Add** > **Add Common Resource** and configure the MySQL resource as follows: + + ![](./figures/HA-mariadb.png) + +2. If the following information is displayed, the resource is successfully added: + + ![](./figures/HA-mariadb-suc.png) + +### Adding the Preceding Resources as a Group Resource + +1. Add the three resources in the resource startup sequence. + + On the home page, choose **Add** > **Add Group Resource** and configure the group resource as follows: + + ![](./figures/HA-group-new.png) + +2. The group resource is successfully created and started. If the command output is the same as that of the preceding common resources, the group resource is successfully added. + + ![](./figures/HA-group-new-suc.png) + +3. Use ha1 as the standby node and migrate the group resource to the ha2 node. The system is running properly. + + ![](./figures/HA-group-new-suc2.png) diff --git a/docs/en/server/installation_upgrade/installation/_toc.yaml b/docs/en/server/installation_upgrade/installation/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..ab766a9fd562fec295cb3c217439fac7983fc368 --- /dev/null +++ b/docs/en/server/installation_upgrade/installation/_toc.yaml @@ -0,0 +1,38 @@ +label: Installation Guide +isManual: true +href: ./installation.md +description: Installing openEuler +sections: + - label: Installation on Servers + href: ./installation-on-servers.md + sections: + - label: Installation Preparations + href: ./installation-preparations.md + - label: Installation Modes + href: ./installation-modes.md + - label: Installation Guide + href: ./installation-guide.md + - label: Using Kickstart for Automatic Installation + href: ./using-kickstart-for-automatic-installation.md + - label: Common Issues and Solutions + href: ./server-installation-common-issues-and-solutions.md + - label: Installation on Raspberry Pi + href: ./install-pi.md + sections: + - label: Installation Preparations + href: ./installation-preparations-1.md + - label: Installation Modes + href: ./installation-modes-1.md + - label: Installation Guide + href: ./installation-guide-1.md + - label: Common Issues and Solutions + href: ./raspi-common-issues-and-solutions.md + - label: More Resources + href: ./more-resources.md + - label: RISC-V Installation Guide + href: ./risc-v.md + sections: + - label: VM Installation + href: ./risc-v-qemu.md + - label: More Resources + href: ./risc-v-more.md diff --git a/docs/en/server/installation_upgrade/installation/figures/Megaraid_IO_Request_uncompleted.png b/docs/en/server/installation_upgrade/installation/figures/Megaraid_IO_Request_uncompleted.png new file mode 100644 index 0000000000000000000000000000000000000000..9f5a9e0f03055c59148830c8f8894196acd6861f Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/Megaraid_IO_Request_uncompleted.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/Partition_expansion.png b/docs/en/server/installation_upgrade/installation/figures/Partition_expansion.png new file mode 100644 index 0000000000000000000000000000000000000000..37a6ef7a2371a9a5518f6d2ce0dc6d36fc71fe1b Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/Partition_expansion.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/bios.png b/docs/en/server/installation_upgrade/installation/figures/bios.png new file mode 100644 index 0000000000000000000000000000000000000000..d5a96738001c5a910174c030af583bb09ff29ce6 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/bios.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/cancle_disk.png b/docs/en/server/installation_upgrade/installation/figures/cancle_disk.png new file mode 100644 index 0000000000000000000000000000000000000000..f1db0f2c524695303f0d8791fcb3c256c75507ad Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/cancle_disk.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/completing-the-automatic-installation.png b/docs/en/server/installation_upgrade/installation/figures/completing-the-automatic-installation.png new file mode 100644 index 0000000000000000000000000000000000000000..f2169685ef202bae133ae74fec620ec64aea46df Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/completing-the-automatic-installation.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/custom_paratition.png b/docs/en/server/installation_upgrade/installation/figures/custom_paratition.png new file mode 100644 index 0000000000000000000000000000000000000000..d2e8c68e6af866e96bf5dd2a2f532de81c59a9d9 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/custom_paratition.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/dialog-box-showing-no-bootable-device.png b/docs/en/server/installation_upgrade/installation/figures/dialog-box-showing-no-bootable-device.png new file mode 100644 index 0000000000000000000000000000000000000000..944c658d621f00b18e4aa75eaca420d76c08715c Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/dialog-box-showing-no-bootable-device.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/drive-icon.png b/docs/en/server/installation_upgrade/installation/figures/drive-icon.png new file mode 100644 index 0000000000000000000000000000000000000000..b41fcb09dfbf805da4863142855e7c2de4bf4c7b Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/drive-icon.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/en-us_image_0213178479.png b/docs/en/server/installation_upgrade/installation/figures/en-us_image_0213178479.png new file mode 100644 index 0000000000000000000000000000000000000000..62ef0decdf6f1e591059904001d712a54f727e68 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/en-us_image_0213178479.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229291243.png b/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229291243.png new file mode 100644 index 0000000000000000000000000000000000000000..2418510f855facae4b47129840894490a1eac7ca Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229291243.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229291247.png b/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229291247.png new file mode 100644 index 0000000000000000000000000000000000000000..d67b599b9ab74017c0800529053befed3efab8a7 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229291247.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229291264.jpg b/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229291264.jpg new file mode 100644 index 0000000000000000000000000000000000000000..3f0a0658e08010f4f453e558a41e31257783b416 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229291264.jpg differ diff --git a/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229291270.png b/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229291270.png new file mode 100644 index 0000000000000000000000000000000000000000..deefef68670d64c131e4c41911a01236158f1dd1 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229291270.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229291272.png b/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229291272.png new file mode 100644 index 0000000000000000000000000000000000000000..e0ad8102bddd886c3bd7a306b088e8a52e2b99c9 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229291272.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229291280.png b/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229291280.png new file mode 100644 index 0000000000000000000000000000000000000000..5754e734c48b23ace2a4fbf1302b820077cd7b71 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229291280.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229291286.png b/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229291286.png new file mode 100644 index 0000000000000000000000000000000000000000..4ffcb081e2c8f82bcc49a65a939f2cd8bd6f949b Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229291286.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229420473.png b/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229420473.png new file mode 100644 index 0000000000000000000000000000000000000000..86c61a4b8e2a5795baff2fc74629924d01d7b97b Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/en-us_image_0229420473.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/en-us_image_0231657950.png b/docs/en/server/installation_upgrade/installation/figures/en-us_image_0231657950.png new file mode 100644 index 0000000000000000000000000000000000000000..bea985ef710c57aeba16600067304b1005ad92e8 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/en-us_image_0231657950.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/enforce-secure-boot.png b/docs/en/server/installation_upgrade/installation/figures/enforce-secure-boot.png new file mode 100644 index 0000000000000000000000000000000000000000..0e40f5fd8d73dbcbad6bdcec5d56d3883d54023a Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/enforce-secure-boot.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/error-message.png b/docs/en/server/installation_upgrade/installation/figures/error-message.png new file mode 100644 index 0000000000000000000000000000000000000000..c5802a2b7a750eed8429ec06c7e4919a3d161a9e Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/error-message.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/figure-10.png b/docs/en/server/installation_upgrade/installation/figures/figure-10.png new file mode 100644 index 0000000000000000000000000000000000000000..45a84ab55ce7c5e51e12bbb6888ccf671a5539ef Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/figure-10.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/figure-11.png b/docs/en/server/installation_upgrade/installation/figures/figure-11.png new file mode 100644 index 0000000000000000000000000000000000000000..d50310a29b8087365b93e41b3a5f4aa1880b1b85 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/figure-11.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/figure-12.png b/docs/en/server/installation_upgrade/installation/figures/figure-12.png new file mode 100644 index 0000000000000000000000000000000000000000..339d3d96f469f54f5b9c0f3b40fb0cd78935180c Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/figure-12.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/figure-13.png b/docs/en/server/installation_upgrade/installation/figures/figure-13.png new file mode 100644 index 0000000000000000000000000000000000000000..3288b925f49c12b0a6c3a96d5b1ca27fad23120a Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/figure-13.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/figure-14.png b/docs/en/server/installation_upgrade/installation/figures/figure-14.png new file mode 100644 index 0000000000000000000000000000000000000000..2113c78fa2977772c55be8ca3b32ddb09ba65f1d Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/figure-14.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/figure-15.png b/docs/en/server/installation_upgrade/installation/figures/figure-15.png new file mode 100644 index 0000000000000000000000000000000000000000..9fd26d5e6247d8e843b79cef0c05ef971c32a4ad Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/figure-15.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/figure-16.png b/docs/en/server/installation_upgrade/installation/figures/figure-16.png new file mode 100644 index 0000000000000000000000000000000000000000..2c3231a61e2ea97c2d6b74868373a1176d3dfc05 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/figure-16.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/figure-17.png b/docs/en/server/installation_upgrade/installation/figures/figure-17.png new file mode 100644 index 0000000000000000000000000000000000000000..8862e92a973e1f52bc7db887f223da880a33d4a3 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/figure-17.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/figure-18.png b/docs/en/server/installation_upgrade/installation/figures/figure-18.png new file mode 100644 index 0000000000000000000000000000000000000000..b5463dddcc1a892edc09d9ff9ac12a7d1751e6ae Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/figure-18.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/figure-19.png b/docs/en/server/installation_upgrade/installation/figures/figure-19.png new file mode 100644 index 0000000000000000000000000000000000000000..ed519042a32bc742d5210f55163de526b9cf17f0 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/figure-19.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/figure-4.png b/docs/en/server/installation_upgrade/installation/figures/figure-4.png new file mode 100644 index 0000000000000000000000000000000000000000..329cc06b70cc26cde85106d40db2c8a17947f937 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/figure-4.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/figure-5.png b/docs/en/server/installation_upgrade/installation/figures/figure-5.png new file mode 100644 index 0000000000000000000000000000000000000000..aac62bc3aa30d89fa3a133a8262ef926bb9319b7 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/figure-5.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/figure-6.png b/docs/en/server/installation_upgrade/installation/figures/figure-6.png new file mode 100644 index 0000000000000000000000000000000000000000..440167f076a2b9e6c4882a808461698d12088e65 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/figure-6.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/figure-7.png b/docs/en/server/installation_upgrade/installation/figures/figure-7.png new file mode 100644 index 0000000000000000000000000000000000000000..a6a6b00852e044d4e624bd1ad3494807bffa1854 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/figure-7.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/figure-8.png b/docs/en/server/installation_upgrade/installation/figures/figure-8.png new file mode 100644 index 0000000000000000000000000000000000000000..eaf7defbe291deae7d5a67993db33b2213a8ea75 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/figure-8.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/figure-9.png b/docs/en/server/installation_upgrade/installation/figures/figure-9.png new file mode 100644 index 0000000000000000000000000000000000000000..fd3e344ab1a8e53ebbf6c8dbb11e5bbf67977a3b Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/figure-9.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/ftp-mode.png b/docs/en/server/installation_upgrade/installation/figures/ftp-mode.png new file mode 100644 index 0000000000000000000000000000000000000000..14b45c69f0b553d55ebcb01064b8f9e2ee591ed8 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/ftp-mode.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/http-mode.png b/docs/en/server/installation_upgrade/installation/figures/http-mode.png new file mode 100644 index 0000000000000000000000000000000000000000..1414062bbd161ac75998d1c098933807a3268412 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/http-mode.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/image-dialog-box.png b/docs/en/server/installation_upgrade/installation/figures/image-dialog-box.png new file mode 100644 index 0000000000000000000000000000000000000000..caeb56bb46f766dd39d66a65e308c591954d32cf Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/image-dialog-box.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/nfs-mode.png b/docs/en/server/installation_upgrade/installation/figures/nfs-mode.png new file mode 100644 index 0000000000000000000000000000000000000000..da913146067340b0db9a94baa4691c5b8497ebae Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/nfs-mode.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/reset_devices.png b/docs/en/server/installation_upgrade/installation/figures/reset_devices.png new file mode 100644 index 0000000000000000000000000000000000000000..70cc2e0138dd48950f4704bd3f1160448d5058a1 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/reset_devices.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/restart-icon.png b/docs/en/server/installation_upgrade/installation/figures/restart-icon.png new file mode 100644 index 0000000000000000000000000000000000000000..a1b02b2dff42c90845d2491192507ea6967352e3 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/restart-icon.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/security.png b/docs/en/server/installation_upgrade/installation/figures/security.png new file mode 100644 index 0000000000000000000000000000000000000000..59ac7bfcef796fc32d0127a9d6095d32cb282fb2 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/security.png differ diff --git a/docs/en/server/installation_upgrade/installation/figures/startparam.png b/docs/en/server/installation_upgrade/installation/figures/startparam.png new file mode 100644 index 0000000000000000000000000000000000000000..b197f4d492213513edf84a99cdb14f186630a828 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/figures/startparam.png differ diff --git a/docs/en/server/installation_upgrade/installation/install-pi.md b/docs/en/server/installation_upgrade/installation/install-pi.md new file mode 100644 index 0000000000000000000000000000000000000000..9c643080454ac812127fbe4c24e2cfbf41cda648 --- /dev/null +++ b/docs/en/server/installation_upgrade/installation/install-pi.md @@ -0,0 +1,3 @@ +# Installation on Raspberry Pi + +This section describes how to install openEuler on Raspberry Pi. Users must have basic knowledge of Linux OS management. diff --git a/docs/en/server/installation_upgrade/installation/installation-guide-1.md b/docs/en/server/installation_upgrade/installation/installation-guide-1.md new file mode 100644 index 0000000000000000000000000000000000000000..a937d46dc42731a3099f60c7d9df23db8b633aa4 --- /dev/null +++ b/docs/en/server/installation_upgrade/installation/installation-guide-1.md @@ -0,0 +1,186 @@ +# Installation Guide + +This section describes how to enable the Raspberry Pi function after [Writing Raspberry Pi Images into the SD card](installation-modes-1.md). + + + +- [Installation Guide](#installation-guide) + - [Starting the System](#starting-the-system) + - [Logging in to the System](#logging-in-to-the-system) + - [Configuring the System](#configuring-the-system) + - [Expanding the Root Directory Partition](#expanding-the-root-directory-partition) + - [Connecting to the Wi-Fi Network](#connecting-to-the-wi-fi-network) + + +## Starting the System + +After an image is written into the SD card, insert the SD card into the Raspberry Pi and power on the SD card. + +For details about the Raspberry Pi hardware, visit the [Raspberry Pi official website](https://www.raspberrypi.org/). + +## Logging in to the System + +You can log in to the Raspberry Pi in either of the following ways: + +1. Local login + + Connect the Raspberry Pi to the monitor (the Raspberry Pi video output interface is Micro HDMI), keyboard, and mouse, and start the Raspberry Pi. The Raspberry Pi startup log is displayed on the monitor. After Raspberry Pi is started, enter the user name **root** and password **openeuler** to log in. + +2. SSH remote login + + By default, the Raspberry Pi uses the DHCP mode to automatically obtain the IP address. If the Raspberry Pi is connected to a known router, you can log in to the router to check the IP address. The new IP address is the Raspberry Pi IP address. + + According to the preceding figure, the IP address of the Raspberry Pi is **192.168.31.109**. You can run the `ssh root@192.168.31.109` command and enter the password `openeuler` to remotely log in to the Raspberry Pi. + +## Configuring the System + +### Expanding the Root Directory Partition + +The space of the default root directory partition is small. Therefore, you need to expand the partition capacity before using it. + +To expand the root directory partition capacity, perform the following procedure: + +1. Run the `fdisk -l` command as the root user to check the drive partition information. The command output is as follows: + + ```shell + # fdisk -l + Disk /dev/mmcblk0: 14.86 GiB, 15931539456 bytes, 31116288 sectors + Units: sectors of 1 * 512 = 512 bytes + Sector size (logical/physical): 512 bytes / 512 bytes + I/O size (minimum/optimal): 512 bytes / 512 bytes + Disklabel type: dos + Disk identifier: 0xf2dc3842 + + Device Boot Start End Sectors Size Id Type + /dev/mmcblk0p1 * 8192 593919 585728 286M c W95 FAT32 (LBA) + /dev/mmcblk0p2 593920 1593343 999424 488M 82 Linux swap / Solaris + /dev/mmcblk0p3 1593344 5044223 3450880 1.7G 83 Linux + ``` + + The drive letter of the SD card is **/dev/mmcblk0**, which contains three partitions: + + - **/dev/mmcblk0p1**: boot partition + - **/dev/mmcblk0p2**: swap partition + - **/dev/mmcblk0p3**: root directory partition + + Here, we need to expand the capacity of `/dev/mmcblk0p3`. + +2. Run the `fdisk /dev/mmcblk0` command as the root user and the interactive command line interface (CLI) is displayed. To expand the partition capacity, perform the following procedure as shown in [Figure 1](#zh-cn_topic_0151920806_f6ff7658b349942ea87f4521c0256c315). + + 1. Enter `p` to check the partition information. + + Record the start sector number of `/dev/mmcblk0p3`. That is, the value in the `Start` column of the `/dev/mmcblk0p3` information. In the example, the start sector number is `1593344`. + + 2. Enter `d` to delete the partition. + + 3. Enter `3` or press `Enter` to delete the partition whose number is `3`. That is, the `/dev/mmcblk0p3`. + + 4. Enter `n` to create a partition. + + 5. Enter `p` or press `Enter` to create a partition of the `Primary` type. + + 6. Enter `3` or press `Enter` to create a partition whose number is `3`. That is, the `/dev/mmcblk0p3`. + + 7. Enter the start sector number of the new partition. That is, the start sector number recorded in Step `1`. In the example, the start sector number is `1593344`. + + > [!NOTE]NOTE + > Do not press **Enter** or use the default parameters. + + 8. Press `Enter` to use the last sector number by default as the end sector number of the new partition. + + 9. Enter `N` without changing the sector ID. + + 10. Enter `w` to save the partition settings and exit the interactive CLI. + + **Figure 1** Expand the partition capacity + ![](./figures/Partition_expansion.png) + +3. Run the `fdisk -l` command as the root user to check the drive partition information and ensure that the drive partition is correct. The command output is as follows: + + ```shell + # fdisk -l + Disk /dev/mmcblk0: 14.86 GiB, 15931539456 bytes, 31116288 sectors + Units: sectors of 1 * 512 = 512 bytes + Sector size (logical/physical): 512 bytes / 512 bytes + I/O size (minimum/optimal): 512 bytes / 512 bytes + Disklabel type: dos + Disk identifier: 0xf2dc3842 + + Device Boot Start End Sectors Size Id Type + /dev/mmcblk0p1 * 8192 593919 585728 286M c W95 FAT32 (LBA) + /dev/mmcblk0p2 593920 1593343 999424 488M 82 Linux swap / Solaris + /dev/mmcblk0p3 1593344 31116287 29522944 14.1G 83 Linux + ``` + +4. Run the `resize2fs /dev/mmcblk0p3` command as the root user to increase the size of the unloaded file system. + +5. Run the `df -lh` command to check the drive space information and ensure that the root directory partition has been expanded. + + > [!NOTE]NOTE + > If the root directory partition is not expanded, run the `reboot` command to restart the Raspberry Pi and then run the `resize2fs /dev/mmcblk0p3` command as the root user. + +### Connecting to the Wi-Fi Network + +To connect to the Wi-Fi network, perform the following procedure: + +1. Check the IP address and network adapter information. + + `ip a` + + Obtain information about the wireless network adapter **wlan0**: + + ```text + 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 + link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 + inet 127.0.0.1/8 scope host lo + valid_lft forever preferred_lft forever + inet6 ::1/128 scope host + valid_lft forever preferred_lft forever + 2: eth0: mtu 1500 qdisc mq state UP group default qlen 1000 + link/ether dc:a6:32:50:de:57 brd ff:ff:ff:ff:ff:ff + inet 192.168.31.109/24 brd 192.168.31.255 scope global dynamic noprefixroute eth0 + valid_lft 41570sec preferred_lft 41570sec + inet6 fe80::cd39:a969:e647:3043/64 scope link noprefixroute + valid_lft forever preferred_lft forever + 3: wlan0: mtu 1500 qdisc fq_codel state DOWN group default qlen 1000 + link/ether e2:e6:99:89:47:0c brd ff:ff:ff:ff:ff:ff + ``` + +2. Scan information about available Wi-Fi networks. + + `nmcli dev wifi` + +3. Connect to the Wi-Fi network. + + Run the `nmcli dev wifi connect SSID password PWD` command as the root user to connect to the Wi-Fi network. + + In the command, `SSID` indicates the SSID of the available Wi-Fi network scanned in the preceding step, and `PWD` indicates the password of the Wi-Fi network. For example, if the `SSID` is `openEuler-wifi`and the password is `12345678`, the command for connecting to the Wi-Fi network is `nmcli dev wifi connect openEuler-wifi password 12345678`. The connection is successful. + + ```text + Device 'wlan0' successfully activated with '26becaab-4adc-4c8e-9bf0-1d63cf5fa3f1'. + ``` + +4. Check the IP address and wireless network adapter information. + + `ip a` + + ```text + 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 + link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 + inet 127.0.0.1/8 scope host lo + valid_lft forever preferred_lft forever + inet6 ::1/128 scope host + valid_lft forever preferred_lft forever + 2: eth0: mtu 1500 qdisc mq state UP group default qlen 1000 + link/ether dc:a6:32:50:de:57 brd ff:ff:ff:ff:ff:ff + inet 192.168.31.109/24 brd 192.168.31.255 scope global dynamic noprefixroute eth0 + valid_lft 41386sec preferred_lft 41386sec + inet6 fe80::cd39:a969:e647:3043/64 scope link noprefixroute + valid_lft forever preferred_lft forever + 3: wlan0: mtu 1500 qdisc fq_codel state UP group default qlen 1000 + link/ether dc:a6:32:50:de:58 brd ff:ff:ff:ff:ff:ff + inet 192.168.31.110/24 brd 192.168.31.255 scope global dynamic noprefixroute wlan0 + valid_lft 43094sec preferred_lft 43094sec + inet6 fe80::394:d086:27fa:deba/64 scope link noprefixroute + valid_lft forever preferred_lft forever + ``` diff --git a/docs/en/server/installation_upgrade/installation/installation-guide.md b/docs/en/server/installation_upgrade/installation/installation-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..85388ec6a32f4ff4ff551f913d0687ff23244522 --- /dev/null +++ b/docs/en/server/installation_upgrade/installation/installation-guide.md @@ -0,0 +1,332 @@ +# Installation Guideline + +This section describes how to install openEuler using a CD/DVD-ROM. The installation process is the same for other installation modes except the boot option. + +## Starting the Installation + +### Booting from the CD/DVD-ROM Drive + +Load the ISO image of openEuler from the CD/DVD-ROM drive of the server and restart the server. The procedure is as follows: + +> ![](./public_sys-resources/icon-note.gif) **Note** +> Before the installation, ensure that the server boots from the CD/DVD-ROM drive preferentially. The following steps describe how to install openEuler using a virtual CD/DVD-ROM drive on the baseboard management controller (BMC). The procedure for installing openEuler from a physical drive is the same as that of a virtual drive. + +1. On the toolbar, click the icon shown in the following figure. + + **Figure 1** Drive icon + ![](./figures/drive-icon.png) + + An image dialog box is displayed, as shown in the following figure. + + **Figure 2** Image dialog box + ![](./figures/image-dialog-box.png) + +2. Select **Image File** and then click **...**. The **Open** dialog box is displayed. +3. Select the image file and click **Open**. In the image dialog box, click **Connect**. If **Connect** changes to **Disconnect**, the virtual CD/DVD-ROM drive is connected to the server. +4. On the toolbar, click the restart icon shown in the following figure to restart the device. + + **Figure 3** Restart icon + ![](./figures/restart-icon.png) + +### Installation Wizard + +A boot menu is displayed after the system is booted using the boot medium. In addition to options for starting the installation program, some other options are available on the boot menu. During system installation, the **Test this media & install openEuler 21.09** mode is used by default. Press the arrow keys on the keyboard to change the selection, and press **Enter** when the desired option is highlighted. + +> ![](./public_sys-resources/icon-note.gif) **Note** + +- If you do not perform any operations within 1 minute, the system automatically selects the default option **Test this media & install openEuler 21.09** and enters the installation page. +- During physical machine installation, if you cannot use the arrow keys to select boot options and the system does not respond after you press **Enter**, click ![](./figures/en-us_image_0229420473.png) on the BMC page and configure **Key & Mouse Reset**. + +**Figure 4** Installation Wizard +![](./figures/figure-4.png) + +Installation wizard options are described as follows: + +- **Install openEuler 21.09**: Install openEuler on your server in GUI mode. + +- **Test this media & install openEuler 21.09**: Default option. Install openEuler on your server in GUI mode. The integrity of the installation medium is checked before the installation program is started. + +- **Troubleshooting**: Troubleshooting mode, which is used when the system cannot be installed properly. In troubleshooting mode, the following options are available: + - **Install openEuler 21.09 in basic graphics mode**: Basic graphics installation mode. In this mode, the video driver is not started before the system starts and runs. + - **Rescue the openEuler system**: Rescue mode, which is used to restore the system. In rescue mode, the installation process is printed to the Virtual Network Computing (VNC) or BMC interface, and the serial port is unavailable. + +On the installation wizard screen, press **e** to go to the parameter editing screen of the selected option, and press **c** to go to the command line interface (CLI). + +### Installation in GUI Mode + +On the installation wizard page, select **Test this media & install openEuler 21.09** to enter the GUI installation mode. + +Perform graphical installation operations using a keyboard. + +- Press **Tab** or **Shift+Tab** to move between GUI controls (such as buttons, area boxes, and check boxes). +- Press the up or down arrow key to move a target in the list. +- Press the left or right arrow key to move between the horizontal toolbar and watch bar. +- Press the spacebar or **Enter** to select or delete highlighted options, expand or collapse a drop-down list. +- Press **Alt+a shortcut key** (the shortcut key varies for different pages) to select the control where the shortcut key is located. The shortcut key can be highlighted (underlined) by holding down **Alt**. + +## Configuring an Installation Program Language + +After the installation starts, the system will prompt the language that is used during the installation process. English is configured by default, as shown in the following figure. Configure another language as required. + +**Figure 5** Selecting a language +![](./figures/figure-5.png) + +After the language is set, click **Continue**. The installation page is displayed. + +If you want to exit the installation, click **Exit**. The message **Are you sure you want to exit the installation program?** is displayed. Click **Yes** in the dialog box to go back to the installation wizard page. + +## Entering the Installation Page + +After the installation program starts, the installation page is displayed, as shown in the following figure. On the page, you can configure the time, language, installation source, network, and storage device. + +Some configuration items are matched with safety symbols. A safety symbol will disappear after the item is configured. Start the installation only when all the safety symbols disappear from the page. + +If you want to exit the installation, click **Exit**. The message **Are you sure you want to exit the installation program?** is displayed. Click **Yes** in the dialog box to go back to the installation wizard page. + +**Figure 6** Installation summary +![](./figures/figure-6.png) + +## Setting the Keyboard Layout + +On the **INSTALLATION SUMMARY** page, click **KEYBOARD**. You can add or delete multiple keyboard layouts in the system. + +- To view the keyboard layout: Select a keyboard layout in the left box and click **keyboard** under the box. +- To test the keyboard layout: Select the keyboard layout in the left box and click keyboard icon in the upper right corner to switch to the desired layout, and then type in the right text box to ensure that the keyboard layout can work properly. + +**Figure 7** Setting the keyboard layout +![](./figures/figure-7.png) + +After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. + +## Setting a System Language + +On the **INSTALLATION SUMMARY** page, click **LANGUAGE SUPPORT** to set the system language, as shown in the following figure. Set another language as required, such as Chinese. + +> ![](./public_sys-resources/icon-note.gif) **Note** +> If you select **Chinese**, the system does not support the display of Chinese characters when you log in to the system using VNC, but supports the display of Chinese characters when you log in to the system using a serial port. When you log in to the system using SSH, whether the system supports the display of Chinese characters depends on the SSH client. If you select **English**, the display is not affected. + +**Figure 8** Setting a system language + +![](./figures/figure-8.png) + +After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. + +## Setting Date and Time + +On the **INSTALLATION SUMMARY** page, click **TIME & DATE**. On the **TIME & DATE** page, set the system time zone, date, and time. + +When setting the time zone, select a region from the drop-down list of **Region** and a city from the drop-down list of **City** at the top of the page, as shown in the following figure. + +If your city is not displayed in the drop-down list, select the nearest city in the same time zone. + +> ![](./public_sys-resources/icon-note.gif) **Note** + +- Before manually setting the time zone, disable the network time synchronization function in the upper right corner. +- If you want to use the network time, ensure that the network can connect to the remote NTP server. For details about how to set the network, see [Setting the Network and Host Name](#setting-the-network-and-host-name). + +**Figure 9** Setting date and time + +![](./figures/figure-9.png) + +After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. + +## Setting the Installation Source + +On the **INSTALLATION SUMMARY** page, click **INSTALLATION SOURCE** to locate the installation source. + +- When you use a complete CD/DVD-ROM for installation, the installation program automatically detects and displays the installation source information. You can use the default settings, as shown in the following figure. + + **Figure 10** Installation source + ![](./figures/figure-10.png) + +- When the network source is used for installation, you need to set the URL of the network source. + + - HTTP or HTTPS mode + + The following figure shows the installation source in HTTP or HTTPS mode: + + ![](./figures/http-mode.png) + + If the HTTPS server uses a private certificate, press **e** on the installation wizard page to go to the parameter editing page of the selected option, and add the **inst.noverifyssl** parameter. + + Enter the actual installation source address, for example, ****, where **openEuler-24.03-LTS-SP1** indicates the version number, and **x86-64** indicates the CPU architecture. Use the actual version number and CPU architecture. + + - FTP mode + + The following figure shows the installation source in FTP mode. Enter the FTP address in the text box. + + ![](./figures/ftp-mode.png) + + You need to set up an FTP server, mount the **openEuler-21.09-x86_64-dvd.iso** image, and copy the mounted files to the shared directory on the FTP server. **x86_64** indicates the CPU architecture. Use the actual image. + + - NFS mode + + The following figure shows the installation source in NFS mode. Enter the NFS address in the text box. + + ![](./figures/nfs-mode.png) + + You need to set up an NFS server, mount the **openEuler-21.09-x86_64-dvd.iso** image, and copy the mounted file to the shared directory on the NFS server. **x86_64** indicates the CPU architecture. Use the actual image. + +During the installation, if you have any questions about configuring the installation source, see [An Exception Occurs During the Selection of the Installation Source](https://gitee.com/openeuler/docs/blob/5232a58d1e76f59c50d68183bdfd3f6dc1603390/docs/en/docs/Installation/faqs.md#an-exception-occurs-during-the-selection-of-the-installation-source). + +After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. + +## Selecting Installation Software + +On the **INSTALLATION SUMMARY** page, click **SOFTWARE SELECTION** to specify the software package to be installed. + +Based on the actual requirements, select **Minimal Install** in the left box and select an add-on in the **Additional software for Selected Environment** area in the right box, as shown in the following figure. + +**Figure 11** Selecting installation software +![](./figures/figure-11.png) + +> ![](./public_sys-resources/icon-note.gif) **Note** + +- In **Minimal Install** mode, not all packages in the installation source will be installed. If the required package is not installed, you can mount the installation source to the local host as a repo source, and use DNF to install the package. +- If you select **Virtualization Host**, the virtualization components QEMU, libvirt, and edk2 are installed by default. You can select whether to install the OVS component in the add-on area. + +After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. + +## Setting the Installation Destination + +On the **INSTALLATION SUMMARY** page, click **INSTALLATION DESTINATION** to select the OS installation disk and partition. + +You can view available local storage devices in the following figure. + +**Figure 12** Setting the installation destination +![](./figures/figure-12.png) + +### Storage Configuration + +On the **INSTALLATION DESTINATION** page, configure storage for system partition. You can either manually configure partitions or select **Automatic** for automatic partitioning. + +> ![](./public_sys-resources/icon-note.gif) **Note** + +- During partitioning, to ensure system security and performance, you are advised to divide the device into the following partitions: **/boot**, **/var**, **/var/log**, **/var/log/audit**, **/home**, and **/tmp**. See [Table 1](#table1). +- If the system is configured with the **swap** partition, the **swap** partition is used when the physical memory of the system is insufficient. Although the **swap** partition can be used to expand the physical memory, if it is used due to insufficient memory, the system response slows and the system performance deteriorates. Therefore, you are not advised to configure it in the system with sufficient physical memory or in the performance sensitive system. +- If you need to split a logical volume group, select **Custom** to manually partition the logical volume group. On the **MANUAL PARTITIONING** page, click **Modify** in the **Volume Group** area to reconfigure the logical volume group. + +**Table 1** Suggested disk partitions + + +| Partition Type | Partition Type | Partition Size | Description | +| --- | --- | --- | --- | +| Primary partition | / | 20 GB | Root directory used to install the OS. | +| Primary partition | /boot | 1 GB | Boot partition. | +| Primary partition | /swap | 16 GB | Swap partition. | +| Logical partition | /home | 1 GB | Stores local user data. | +| Logical partition | /tmp | 10 GB | Stores temporary files. | +| Logical partition | /var | 5 GB | Stores the dynamic data of the daemon process and other system service processes. | +| Logical partition | /var/log | 10 GB | Stores system log data. | +| Logical partition | /var/log/audit | 2 GB | Stores system audit log data. | +| Logical partition | /usr| 5 GB | Stores shared and read-only applications. | +| Logical partition | /var/tmp | 5 GB | Stores temporary files that can be retained during system reboot. | +| Logical partition | /opt | Size of the remaining disk space. | Used to install application software. | + +**Automatic** + +Select **Automatic** if the software is installed in a new storage device or the data in the storage device is not required. After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. + +**Custom** + +If you need to manually partition the disk, click **Custom** and click **Done** in the upper left corner. The following page is displayed. + +On the **MANUAL PARTITIONING** page, you can partition the disk in either of the following ways. After the partitioning is completed, the window shown in the following figure is displayed. + +- Automatic creation: Click **Click here to create them automatically**. The system automatically assigns four mount points according to the available storage space: **/boot**, **/**, **/home**, **/boot/efi**, and **swap**. + +- Manual creation: Click ![](./figures/en-us_image_0229291243.png) to add a mount point. It is recommended that the expected capacity of each mount point not exceed the available space. + + > ![](./public_sys-resources/icon-note.gif) **Note** + > If the expected capacity of the mount point exceeds the available space, the system allocates the remaining available space to the mount point. + +**Figure 13** MANUAL PARTITIONING page +![](./figures/figure-13.png) + +> ![](./public_sys-resources/icon-note.gif) **Note** +> If non-UEFI mode is selected, the **/boot/efi** partition is not required. Otherwise, it is required. + +After the setting is complete, click **Done** in the upper left corner to go back to the **SUMMARY OF CHANGES** page. + +Click **Accept Changes** to go back to the **INSTALLATION SUMMARY** page. + +## Setting the Network and Host Name + +On the **INSTALLATION SUMMARY** page, select **NETWORK & HOST NAME** to configure the system network functions. + +The installation program automatically detects a local access interface. The detected interface is listed in the left box, and the interface details are displayed in the right area, as shown in Figure 14. You can enable or disable a network interface by clicking the switch in the upper right corner of the page. The switch is turned off by default. If the installation source is set to network, turn on the switch. You can also click **Configure** to configure the selected interface. Select **Connect automatically with priority** to enable the NIC automatic startup upon system startup, as shown in Figure 15. + +In the lower left box, enter the host name. The host name can be the fully quantified domain name (FQDN) in the format of *hostname.domain_name* or the brief host name in the format of *hostname*. + +**Figure 14** Setting the network and host name +![](./figures/figure-14.png) + +**Figure 15** Configuring the network +![](./figures/figure-15.png) + +After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. + +## Setting the Root Password + +Select Root Password on the **INSTALLATION SUMMARY** page. The **Root Password** page is displayed, as shown in the following figure. Enter a password based on [Password Complexity](#password-complexity) requirements and confirm the password. + +> ![](./public_sys-resources/icon-note.gif) **Note** + +- The **root** account is used to perform key system management tasks. You are not advised to use the **root** account for daily work or system access. +- If you select **Lock root account** on the **Root Password** page, the **root** account will be disabled. + +**Figure 16** root password + +![](./figures/figure-16.png) + +### Password Complexity + +The password of the **root** user or the password of the new user must meet the password complexity requirements. Otherwise, the password configuration or user creation will fail. The password complexity requirements are as follows: + +1. A password must contain at least eight characters. + +2. A password must contain at least three of the following types: uppercase letters, lowercase letters, digits, and special characters. + +3. A password must be different from the account name. + +4. A password cannot contain words in the dictionary. + + > ![](./public_sys-resources/icon-note.gif) **Note** + > In the installed openEuler environment, you can run the`cracklib-unpacker /usr/share/cracklib/pw_dict > dictionary.txt` command to export the dictionary library file **dictionary.txt**, and then check whether the password is in the dictionary. + +After the settings are completed, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. + +## Creating a User + +Click **User Creation**. Figure 17 shows the page for creating a user. Enter a username and set a password. By clicking **Advanced**, you can also configure a home directory and a user group, as shown in Figure 18. + +**Figure 17** Creating a user +![](./figures/figure-17.png) + +**Figure 18** Advanced user configuration +![](./figures/figure-18.png) + +After configuration, click **Done** in the left-upper corner to switch back to the **INSTALLATION SUMMARY** page. + +## Starting Installation + +On the installation page, after all the mandatory items are configured, the alarm symbols will disappear. Then, you can click **Begin Installation** to install the system. + +## Installation Procedure + +After the installation starts, the overall installation progress and the progress of writing the software package to the system are displayed. See the following figure. + +> [!TIP]NOTICE +> If you click **Exit** or reset or power off the server during the installation, the installation is interrupted and the system is unavailable. In this case, you need to reinstall the system. + +**Figure 19** Installation process +![](./figures/figure-19.png) + +## Completing the Installation + +After openEuler is installed, Click **Reboot** to restart the system. + +> ![](./public_sys-resources/icon-note.gif) **Note** + +- If a physical CD/DVD-ROM is used for installation and it is not automatically ejected during the restart, manually remove it. Then, the openEuler CLI login page is displayed. +- If a virtual CD/DVD-ROM is used for installation, change the server boot option to **Hard Disk** and restart the server. Then, the openEuler CLI login page is displayed. diff --git a/docs/en/server/installation_upgrade/installation/installation-modes-1.md b/docs/en/server/installation_upgrade/installation/installation-modes-1.md new file mode 100644 index 0000000000000000000000000000000000000000..789851e5fc8b1261ef45a65e4bd25cd52c949f16 --- /dev/null +++ b/docs/en/server/installation_upgrade/installation/installation-modes-1.md @@ -0,0 +1,118 @@ +# Installation Modes + +> ![](./public_sys-resources/icon-notice.gif)**NOTE** +> +> - The hardware supports only Raspberry Pi 3B/3B+/4B. +> - The installation is performed by writing images to the SD card. This section describes how to write images on Windows, Linux, and Mac. +> - The image used in this section is the Raspberry Pi image of openEuler. For details about how to obtain the image, see [Installation Preparations](installation-preparations-1.md). + + + +- [Installation Modes](#installation-modes) + - [Writing Images on Windows](#writing-images-on-windows) + - [Formatting the SD Card](#formatting-the-sd-card) + - [Writing Images to the SD Card](#writing-images-to-the-sd-card) + - [Writing Images on Linux](#writing-images-on-linux) + - [Checking Drive Partition Information](#checking-drive-partition-information) + - [Unmounting the SD Card](#unmounting-the-sd-card) + - [Writing Images to the SD Card](#writing-images-to-the-sd-card-1) + - [Writing Images on Mac](#writing-images-on-mac) + - [Checking Drive Partition Information](#checking-drive-partition-information-1) + - [Unmounting the SD Card](#unmounting-the-sd-card-1) + - [Writing Images to the SD Card](#writing-images-to-the-sd-card-2) + + + +## Writing Images on Windows + +This section uses Windows 10 as an example to describe how to write images to the SD card in the Windows environment. + +### Formatting the SD Card + +To format the SD card, perform the following procedures: + +1. Download and install a SD card formatting tool. The following operations use SD Card Formatter as an example. + +2. Start SD Card Formatter. In **Select card**, select the drive letter of the SD card to be formatted. + + If no image has been installed in the SD card, only one drive letter exists. In **Select card**, select the drive letter of the SD card to be formatted. + + If an image has been installed in the SD card, one or more drive letters exist. For example, the SD card corresponds to three drive letters: E, G, and H. In **Select card**, you can select the drive letter E of the boot partition. + +3. In **Formatting options**, select a formatting mode. The default mode is **Quick format**. + +4. Click **Format** to start formatting. A progress bar is displayed to show the formatting progress. + +5. After the formatting is completed, the message "Formatting was successfully completed" is displayed. Click **OK**. + +### Writing Images to the SD Card + +> ![](./public_sys-resources/icon-notice.gif)**NOTE** +> If the compressed image file **openEuler-21.09-raspi-aarch64.img.xz** is obtained, decompress the file to obtain the **openEuler-21.09-raspi-aarch64.img** image file. + +To write the **openEuler-21.09-raspi-aarch64.img** image file to the SD card, perform the following procedures: + +1. Download and install a tool for writing images. The following operations use Win32 Disk Imager as an example. +2. Start Win32 Disk Imager and right-click **Run as administrator**. +3. Select the path of the image file in IMG format from the **Image File** drop-down list box. +4. In **Device**, select the drive letter of the SD card to which the image is written. +5. Click **Write**. A progress bar is displayed to show the progress of writing the image to the SD card. +6. After the write operation is completed, a dialog box is displayed, indicating that the write operation is successfully completed. Click **OK**. + +## Writing Images on Linux + +This section describes how to write images to the SD card in the Linux environment. + +### Checking Drive Partition Information + +Run the ` **fdisk -l** ` command as the **root** user to obtain the drive information of the SD card. For example, the drive partition corresponding to the SD card can be **/dev/sdb**. + +### Unmounting the SD Card + +1. Run the ` **df -lh** ` command to check the mounted volumes. + +2. If the partitions corresponding to the SD card are not mounted, skip this step. If the partitions (for example, /dev/sdb1 and /dev/sdb3) are mounted, run the following commands as the **root** user to unmount them: + + `umount /dev/sdb1` + + `umount /dev/sdb3` + +### Writing Images to the SD Card + +1. If the image obtained is compressed, run the ` **xz -d openEuler-21.09-raspi-aarch64.img.xz** ` command to decompress the compressed file to obtain the **openEuler-21.09-raspi-aarch64.img** image file. Otherwise, skip this step. + +2. Run the following command as the **root** user to write the `openEuler-21.09-raspi-aarch64.img` image to the SD card: + + `dd bs=4M if=openEuler-21.09-raspi-aarch64.img of=/dev/sdb` + + > [!NOTE]NOTE + > Generally, the block size is set to 4 MB. If the write operation fails or the written image cannot be used, you can set the block size to 1 MB and try again. However, the write operation is time-consuming when the block size is set to 1 MB. + +## Writing Images on Mac + +This section describes how to flash images to the SD card in the Mac environment. + +### Checking Drive Partition Information + +Run the ` **diskutil list** ` command as the **root** user to obtain the drive information of the SD card. For example, the drive partition corresponding to the SD card can be **/dev/disk3**. + +### Unmounting the SD Card + +1. Run the ` **df -lh** ` command to check the mounted volumes. + +2. If the partitions corresponding to the SD card are not mounted, skip this step. If the partitions (for example, dev/disk3s1 and /dev/disk3s3) are mounted, run the following commands as the **root** user to unmount them: + + `diskutil umount /dev/disk3s1` + + `diskutil umount /dev/disk3s3` + +### Writing Images to the SD Card + +1. If the image obtained is compressed, run the `xz -d openEuler-21.09-raspi-aarch64.img.xz` command to decompress the compressed file to obtain the **openEuler-21.09-raspi-aarch64.img** image file. Otherwise, skip this step. + +2. Run the following command as the **root** user to write the image `openEuler-21.09-raspi-aarch64.img` to the SD card: + + `dd bs=4m if=openEuler-21.09-raspi-aarch64.img of=/dev/disk3` + + > [!NOTE]NOTE + > Generally, the block size is set to 4 MB. If the write operation fails or the written image cannot be used, you can set the block size to 1 MB and try again. However, the write operation is time-consuming when the block size is set to 1 MB. diff --git a/docs/en/server/installation_upgrade/installation/installation-modes.md b/docs/en/server/installation_upgrade/installation/installation-modes.md new file mode 100644 index 0000000000000000000000000000000000000000..ba3fc96737ed7d0c1d087e3c6bcdbaa3afc3f36d --- /dev/null +++ b/docs/en/server/installation_upgrade/installation/installation-modes.md @@ -0,0 +1,199 @@ +# Installation Modes + +> [!TIP]NOTICE +> +> - Only TaiShan 200 servers and FusionServer Pro rack server are supported. For details about the supported server models, see [Hardware Compatibility](./installation-preparations.md#hardware-compatibility). Only a virtualization platform created by the virtualization components \(openEuler as the host OS and QEMU and KVM provided in the release package\) of openEuler and the x86 virtualization platform of Huawei public cloud are supported. +> - Currently, only installation modes such as DVD-ROM, USB flash drive, network, QCOW2 image, and private image are supported. In addition, only the x86 virtualization platform of Huawei public cloud supports the private image installation mode. + + + +- [Installation Modes](#installation-modes) + - [Installation Through a DVD-ROM](#installation-through-a-dvd-rom) + - [Preparing the Installation Source](#preparing-the-installation-source) + - [Starting the Installation](#starting-the-installation) + - [Installation Through a USB Flash Drive](#installation-through-a-usb-flash-drive) + - [Preparing the Installation Source](#preparing-the-installation-source-1) + - [Starting the Installation](#starting-the-installation-1) + - [Installation Through the Network Using PXE](#installation-through-the-network-using-pxe) + - [Installation Through a QCOW2 Image](#installation-through-a-qcow2-image) + - [Creating a QCOW2 Image](#creating-a-qcow2-image) + - [Starting the Installation](#starting-the-installation-2) + - [Installation Through a Private Image](#installation-through-a-private-image) + - [Creating a Private Image](#creating-a-private-image) + - [Starting the Installation](#starting-the-installation-3) + + + +## Installation Through a DVD-ROM + +This section describes how to create or use a DVD-ROM to install the openEuler. + +### Preparing the Installation Source + +If you have obtained a DVD-ROM, directly install the OS using the DVD-ROM. If you have obtained an ISO file, record the ISO file to a DVD and install the OS using the obtained DVD. + +### Starting the Installation + +Perform the following operations to start the installation: + +> [!NOTE]NOTE +> Set the system to preferentially boot from the DVD-ROM drive. Take BIOS as an example. You need to move the **CD/DVD-ROM Drive** option under **Boot Type Order** to the top. + +1. Disconnect all drives that are not required, such as USB drives. +2. Start your computer system. +3. Insert the installation DVD-ROM into the computer. +4. Restart the computer system. + +After a short delay, a graphical wizard page is displayed, which contains different boot options. If you do not perform any operation within one minute, the installation starts automatically with the default option. + +## Installation Through a USB Flash Drive + +This section describes how to create or use a USB flash drive to install the openEuler. + +### Preparing the Installation Source + +Pay attention to the capacity of the USB flash drive. The USB flash drive must have sufficient space to store the entire image. It is recommended that the USB flash drive have more than 16 GB space. + +1. Connect the USB flash drive to the system and run the **dmesg** command to view related log. At the end of the log, you can view the information generated by the USB flash drive that is just connected. The information is similar to the following: + + ```text + [ 170.171135] sd 5:0:0:0: [sdb] Attached SCSI removable disk + ``` + + > [!NOTE]NOTE + > Take the **sdb** USB flash drive as an example. + +2. Switch to user **root**. When running the **su** command, you need to enter the password. + + ```bash + su - root + ``` + +3. Ensure that the USB flash drive is not mounted. + + ```bash + findmnt /dev/sdb + ``` + + - If no command output is displayed, the file system is not mounted. Go to the next step. + + - If the following information is displayed, the USB flash drive is automatically mounted. + + ```bash + $ findmnt /dev/sdb + TARGET SOURCE FSTYPE OPTIONS + /mnt/iso /dev/sdb iso9660 ro,relatime + ``` + + In this case, you need to run the **umount** command to uninstall the device. + + ```bash + umount /mnt/iso + ``` + +4. Run the **dd** command to write the ISO image to the USB flash drive. + + ```bash + dd if=/path/to/image.iso of=/dev/device bs=blocksize + ``` + + Replace **/path/to/image.iso** with the complete path of the downloaded ISO image file, replace **device** with the device name provided by the **dmesg** command, and set a proper block size \(for example, 512 KB\) to replace **blocksize** to accelerate the write progress. + + For example, if the ISO image file name is **/home/testuser/Downloads/openEuler-21.09-aarch64-dvd.iso** and the detected device name is **sdb**, run the following command: + + ```bash + dd if=/home/testuser/Downloads/openEuler-21.09-aarch64-dvd.iso of=/dev/sdb bs=512k + ``` + + > [!NOTE]NOTE + > As described in ISOLINUX, the ISO 9660 file system created by the **mkisofs** command will boot through BIOS firmware, but only from the CD-ROM, DVD-ROM, or BD. In this case, you need to run the **isohybrid -u your.iso** command to process the ISO file and then run the **dd** command to write the ISO file to the USB flash drive. (This problem affects only the x86 architecture.) + +5. After the image is written, remove the USB flash drive. + + No progress is displayed during the image write process. When the number sign (#) appears again, run the following command to write the data to the drive. Then exit the **root** account and remove the USB flash drive. In this case, you can use the USB drive as the installation source of the system. + + ```bash + sync + ``` + +### Starting the Installation + +Perform the following operations to start the installation: + +> [!NOTE]NOTE +> Set the system to preferentially boot from the USB flash drive. Take the BIOS as an example. You need to move the **USB** option under **Boot Type Order** to the top. + +1. Disconnect all drives that are not required. +2. Open your computer system. +3. Insert the USB flash drive into the computer. +4. Restart the computer system. + +After a short delay, a graphical wizard page is displayed, which contains different boot options. If you do not perform any operation within one minute, the installation program automatically starts the installation. + +## Installation Through the Network Using PXE + +To boot with PXE, you need to properly configure the server and your computer's network interface shall support PXE. + +If the target hardware is installed with a PXE-enabled NIC, configure it to boot the computer from network system files rather than local media \(such as DVD-ROMs\) and execute the Anaconda installation program. + +For installation through the network using PXE, the client uses a PXE-enabled NIC to send a broadcast request for DHCP information and IP address to the network. The DHCP server provides the client with an IP address and other network information, such as the IP address or host name of the DNS and FTP server \(which provides the files required for starting the installation program\), and the location of the files on the server. + +> [!NOTE]NOTE +> The TFTP, DHCP, and HTTP server configurations are not described here. For details, see [Full-automatic Installation Guide](./using-kickstart-for-automatic-installation.md#full-automatic-installation-guide). + +## Installation Through a QCOW2 Image + +This section describes how to create or use a QCOW2 image to install the openEuler. + +### Creating a QCOW2 Image + +1. Install the **qemu-img** software package. + + ```bash + dnf install -y qemu-img + ``` + +2. Run the **create** command of the qemu-img tool to create an image file. The command format is as follows: + + ```bash + qemu-img create -f -o + ``` + + The parameters are described as follows: + + - _imgFormat_: Image format. The value can be **raw** or **qcow2**. + - _fileOption_: File option, which is used to set features of an image file, such as specifying a backend image file, compressing, and encrypting. + - _fileName_: File name. + - _diskSize_: Disk size, which specifies the size of a block disk. The unit can be K, M, G, or T, indicating KiB, MiB, GiB, or TiB. + + For example, to create an image file **openEuler-image.qcow2** whose disk size is 32 GB and format is qcow2, the command and output are as follows: + + ```bash + $ qemu-img create -f qcow2 openEuler-image.qcow2 32G + Formatting 'openEuler-image.qcow2', fmt=qcow2 size=34359738368 cluster_size=65536 lazy_refcounts=off refcount_bits=16 + ``` + +### Starting the Installation + +Perform the following operations to start the installation: + +1. Prepare a QCOW2 image file. +2. Prepare the VM network. +3. Prepare the UEFI boot tool set EDK II. +4. Prepare the VM XML configuration file. +5. Create a VM. +6. Start the VM. + +For details, see the [*Virtualization User Guide*](../../../virtualization/virtualization_platform/virtualization/introduction-to-virtualization.md) + +## Installation Through a Private Image + +This section describes how to create or use a private image to install the openEuler. + +### Creating a Private Image + +For instructions about how to create a private image, see [*Image Management Service User Guide*](https://support.huaweicloud.com/intl/en-us/usermanual-ims/en-us_topic_0013901628.html). + +### Starting the Installation + +For details about how to start the x86 virtualization platform of Huawei public cloud, see [Elastic Cloud Server User Guide](https://support.huaweicloud.com/intl/en-us/wtsnew-ims/index.html). diff --git a/docs/en/server/installation_upgrade/installation/installation-on-servers.md b/docs/en/server/installation_upgrade/installation/installation-on-servers.md new file mode 100644 index 0000000000000000000000000000000000000000..33dbf9a416e60c59d3d99cf34ae308adba1e4736 --- /dev/null +++ b/docs/en/server/installation_upgrade/installation/installation-on-servers.md @@ -0,0 +1,3 @@ +# Installation on Servers + +This section describes how to install openEuler on servers. Users must have basic knowledge of Linux OS management. diff --git a/docs/en/server/installation_upgrade/installation/installation-preparations-1.md b/docs/en/server/installation_upgrade/installation/installation-preparations-1.md new file mode 100644 index 0000000000000000000000000000000000000000..1d8a4c1b158ce51d3fe74eecfa104dfb6fa8d52a --- /dev/null +++ b/docs/en/server/installation_upgrade/installation/installation-preparations-1.md @@ -0,0 +1,102 @@ +# Installation Preparations + +This section describes the compatibility of the hardware and software and the related configurations and preparations required for the installation. + +## Obtaining the Installation Source + +Before installation, obtain the openEuler Raspberry Pi image and its verification file. + +1. Visit [openEuler Repo](https://repo.openeuler.org/). +2. Choose **openEuler 22.03 LTS SP2**. +3. Click **raspi_img**. The download list of Raspberry Pi images is displayed. +4. Click **openEuler-22.03-LTS-SP2-raspi-aarch64.img.xz** to download the openEuler Raspberry Pi image to the local PC. +5. Click **openEuler-22.03-LTS-SP2-raspi-aarch64.img.xz.sha256sum** to download the verification file of the openEuler Raspberry Pi image to the local PC. + +## Verifying the Image Integrity + +### Overview + +During package transmission, to prevent software packages from being incompletely downloaded due to network or storage device problems, you need to verify the integrity of the software packages after obtaining them. Only the software packages that pass the verification can be deployed. + +Compare the verification value recorded in the verification file with the verification value that is manually calculated to determine whether the software package is complete. If the two values are the same, the downloaded file is complete. Otherwise, the downloaded file is incomplete and you need to obtain the software package again. + +### Prerequisites + +Before verifying the integrity of the image file, ensure that the following files are available: + +Image file: **openEuler-22.03-LTS-SP2-raspi-aarch64.img.xz** +Image file: **openEuler-22.03-LTS-SP2-raspi-aarch64.img.xz** + +Verification file: **openEuler-22.03-LTS-SP2-raspi-aarch64.img.xz.sha256sum** +Verification file: **openEuler-22.03-LTS-SP2-raspi-aarch64.img.xz.sha256sum** + +### Procedures + +To verify the file integrity, perform the following procedures: + +1. Obtain the verification value from the verification file. Run the following command: + + ```shell + cat openEuler-22.03-LTS-SP2-raspi-aarch64.img.xz.sha256sum + ``` + +2. Calculate the SHA256 verification value of the file. Run the following command: + + ```shell + sha256sum openEuler-22.03-LTS-SP2-raspi-aarch64.img.xz + ``` + + After the command is executed, the verification value is displayed. + +3. Check whether the verification values obtained from the step 1 and step 2 are consistent. + + If they are consistent, the downloaded file is not damaged. Otherwise, the downloaded file is incomplete and you need to obtain the file again. + +## Installation Requirements + +If the openEuler OS is installed in the Raspberry Pi environment, the Raspberry Pi environment must meet the following requirements. + +### Hardware Compatibility + +Currently, the openEuler Raspberry Pi image supports the 3B, 3B+, and 4B versions. + +### Minimum Hardware Specifications + +[Table 1](#tff48b99c9bf24b84bb602c53229e2542) lists the minimum hardware specifications for the openEuler Raspberry Pi image. + +**Table 1** Minimum hardware specifications + + + + + + + + + + + + + + + + + + + + + + +

Component Name

+

Minimum Hardware Specifications

+

Description

+

Raspberry Pi version

+
  • Raspberry Pi 3B
  • Raspberry Pi 3B+
  • Raspberry Pi 4B
  • Raspberry Pi 400
+

-

+

Memory

+

≥ 2 GB (4 GB or higher recommended for better user experience)

+

-

+

Drive

+

8 GB or higher recommended for better user experience

+

-

+
diff --git a/docs/en/server/installation_upgrade/installation/installation-preparations.md b/docs/en/server/installation_upgrade/installation/installation-preparations.md new file mode 100644 index 0000000000000000000000000000000000000000..0b11851c39bf27023c6f5c20b32ad47cfb2b4471 --- /dev/null +++ b/docs/en/server/installation_upgrade/installation/installation-preparations.md @@ -0,0 +1,107 @@ +# Installation Preparations + +This section describes the compatibility of the hardware and software and the related configurations and preparations required for the installation. + +## Obtaining the Installation Source + +Obtain the openEuler release package and verification file before the installation. + +Perform the following operations to obtain the openEuler release package: + +1. Visit the [openEuler](https://www.openeuler.org/en/) website. +2. Click **Downloads**. +3. Click **Community Editions**. The version list is displayed. +4. Click **Download** on the right of **openEuler 22.03 LTS SP2**. +5. Download the required openEuler release package and the corresponding verification file based on the architecture and scenario. + 1. If the architecture is AArch64: + 1. Click **AArch64**. + 2. If you install openEuler from a local source, download the **Offline Standard ISO** or **Offline Everything ISO** release package **openEuler-22.03-LTS-SP2-(everything-)aarch64-dvd.iso** to the local host. + 3. If you install openEuler through the network, download the **Network Install ISO** release package **openEuler-22.03-LTS-SP2-netinst-aarch64-dvd.iso** to the local host. + 2. If the architecture is x86_64: + 1. Click **x86_64**. + 2. If you install openEuler from a local source, download the **Offline Standard ISO** or **Offline Everything ISO** release package **openEuler-22.03-LTS-SP2-(everything-)x86_64-dvd.iso** to the local host. + 3. If you install openEuler through the network, download the **Network Install ISO** release package **openEuler-22.03-LTS-SP2-netinst-x86_64-dvd.iso** to the local host. + +> [!NOTE]NOTE + +- When the network is available, install the environment on the network because the ISO release package is small. +- The release package of the AArch64 architecture supports the UEFI mode, while the release package of the x86\_64 architecture supports both the UEFI and Legacy modes. + +## Release Package Integrity Check + +>[!NOTE]NOTE +> This section describes how to verify the integrity of the release package in the AArch64 architecture. The procedure for verifying the integrity of the release package in the x86_64 architecture is the same. + +### Introduction + +To prevent the software package from being incompletely downloaded due to network or storage device faults during transmission, you need to verify the integrity of the software package after obtaining it. Only the software package that passes the verification can be installed. + +Compare the verification value recorded in the verification file with the .iso file verification value calculated manually to check whether the software package passes the verification. If the values are consistent, the .iso file is not damaged. Otherwise, the file is damaged and you need to obtain it again. + +### Prerequisites + +Before verifying the integrity of the release package, prepare the following files: + +- ISO file: **openEuler-22.03-LTS-SP2-aarch64-dvd.iso** +- Verification file: Copy and save the **Integrity Check** SHA256 value to a local file. + +### Procedures + +To verify the file integrity, perform the following operations: + +1. Calculate the SHA256 verification value of the file. Run the following command: + + ```shell + sha256sum openEuler-22.03-LTS-SP2-aarch64-dvd.iso + ``` + + After the command is run, the verification value is displayed. + +2. Check whether the calculated value is the same as that of the saved SHA256 value. + + If the verification values are consistent, the .iso file is not damaged. If they are inconsistent, you can confirm that the file is damaged and you need to obtain the file again. + +## Installation Requirements for PMs + +To install the openEuler OS on a PM, the PM must meet the following requirements. + +### Hardware Compatibility + +You need to take hardware compatibility into account during openEuler installation. [Compatibility List](https://www.openeuler.org/en/compatibility/) describes the types of supported servers. + +### Minimum Hardware Specifications + +[Table 2](#tff48b99c9bf24b84bb602c53229e2541) lists the minimum hardware specifications supported by openEuler. + +**Table 2** Minimum hardware specifications + +| Component | Minimum Hardware Specifications | +| :---- | :---- | +| Architecture | AArch64 or x86_64 | +| CPU | Two CPUs | +| Memory | ≥ 4 GB (8 GB or higher recommended for better user experience) | +| Hard disk | ≥ 32 GB (120 GB or higher recommended for better user experience) | + +## Installation Requirements for VMs + +To install the openEuler OS on a VM, the PM must meet the following requirements. + +### Virtualization Platform Compatibility + +When installing openEuler, pay attention to the compatibility of the virtualization platform. Currently, the following virtualization platforms are supported: + +- A virtualization platform created by the virtualization components \(openEuler as the host OS and QEMU and KVM provided in the release package\) of openEuler +- x86 virtualization platform of Huawei public cloud + +### Minimum Virtualization Space + +[Table 3](#tff48b99c9bf24b84bb602c53229e2541) lists the minimum virtualization space required by openEuler. + +**Table 3** Minimum virtualization space + +| Component | Minimum Virtualization Space | +| :---- | :---- | +| Architecture | AArch64 or x86_64 | +| CPU | Two CPUs| +| Memory | ≥ 4 GB (8 GB or higher recommended for better user experience) | +| Hard disk | ≥ 32 GB (120 GB or higher recommended for better user experience) | diff --git a/docs/en/server/installation_upgrade/installation/installation.md b/docs/en/server/installation_upgrade/installation/installation.md new file mode 100644 index 0000000000000000000000000000000000000000..1b71043c393ad163228adad09aea51ac3590dddb --- /dev/null +++ b/docs/en/server/installation_upgrade/installation/installation.md @@ -0,0 +1,5 @@ +# Installation Guide + +This guide describes how to install openEuler. + +This guide is intended for openEuler users with a basic understanding of Linux system management, and is also recommended for administrators, system engineers, and maintenance personnel. diff --git a/docs/en/server/installation_upgrade/installation/more-resources.md b/docs/en/server/installation_upgrade/installation/more-resources.md new file mode 100644 index 0000000000000000000000000000000000000000..fab0df5d93e9e7c5283d8d46368d3c4d6402c887 --- /dev/null +++ b/docs/en/server/installation_upgrade/installation/more-resources.md @@ -0,0 +1,4 @@ +# Reference + +- How to Create a Raspberry Pi Image File +- How to Use Raspberry Pi diff --git a/docs/en/server/installation_upgrade/installation/public_sys-resources/icon-note.gif b/docs/en/server/installation_upgrade/installation/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/public_sys-resources/icon-note.gif differ diff --git a/docs/en/server/installation_upgrade/installation/public_sys-resources/icon-notice.gif b/docs/en/server/installation_upgrade/installation/public_sys-resources/icon-notice.gif new file mode 100644 index 0000000000000000000000000000000000000000..86024f61b691400bea99e5b1f506d9d9aef36e27 Binary files /dev/null and b/docs/en/server/installation_upgrade/installation/public_sys-resources/icon-notice.gif differ diff --git a/docs/en/server/installation_upgrade/installation/raspi-common-issues-and-solutions.md b/docs/en/server/installation_upgrade/installation/raspi-common-issues-and-solutions.md new file mode 100644 index 0000000000000000000000000000000000000000..9fb8fe803d822467d0eb00d76606187152fdd0ec --- /dev/null +++ b/docs/en/server/installation_upgrade/installation/raspi-common-issues-and-solutions.md @@ -0,0 +1,66 @@ +# Common Issues and Solutions + +## Issue 1: Failed to Start the Raspberry Pi + +### Symptom + +After the Raspberry Pi image released by the openEuler is written to the SD card, the Raspberry Pi fails to be started. + +### Cause Analysis + +The possible causes are as follows: + +1. The downloaded image file is incomplete. To avoid this problem, ensure that the image passes the integrity verification. +2. An error occurs when the image is written to the SD card. In most cases, the error occurs when the image is written to the SD card in the Windows environment using the application software. + +### Solution + +Re-write the complete image to the SD card. + +## Issue 2: Failed to Connect to Wi-Fi by Running the nmcli Command + +### Symptom + +Failed to connect to the Wi-Fi network by running the `nmcli dev wifi connect SSID password PWD` command. An error message, for example, `Error: Connection activation failed: (7) Secrets were required, but not provided.`, is displayed. + +### Cause Analysis + +The command to be executed does not have a password. Note that if the password contains special characters, use single quotation marks to quote the password. If you fail to connect to the Wi-Fi network by running the `nmcli` command line, you are advised to use the nmtui utility for connection. + +### Solution + +Run the `nmtui` command to enter the nmtui utility. Perform the following steps to connect to the Wi-Fi network: + +1. Select **Edit a connection** and press **Enter**. The window for editing network connections is displayed. +2. Press the right arrow key on the keyboard to select **Add**, and then press **Enter**. The window for creating a network connection is displayed. +3. Set the connection type to **Wi-Fi**, press the right arrow key on the keyboard to select **Create**, and press **Enter**. The page for editing Wi-Fi connection information is displayed. +4. On the Wi-Fi connection information page, edit the following information. Other information depends on the specific requirements. After the editing is complete, select **OK** and press **Enter** to return to the window for editing network connections. + 1. Enter the name of the Wi-Fi connection in the **Profile name** text box. You can use the default name, for example, **Wi-Fi connection 1**. + 2. Enter **wlan0** in the **Device** text box. + 3. Enter the SSID of the Wi-Fi network to be connected in the **SSID** text box. + 4. In the **Security** area, select the Wi-Fi password encryption mode as required, for example, **WPA & WPA2 Personal**. + 5. Enter the Wi-Fi password in the **Password** text box. + +5. Select **Back** to return to the home screen of the nmtui utility. +6. Select **Activate a connection** and press **Enter**. The window for activating network connections is displayed. +7. Check whether the added Wi-Fi connection is activated. The name of an activated Wi-Fi connection is marked with an asterisk (*). If the Wi-Fi connection is not activated, select the Wi-Fi connection, press the right arrow key on the keyboard to select **Activate**, and press **Enter** to activate the connection. After the activation is complete, select **Back** and press **Enter** to return to the home screen of the nmtui utility. +8. Select **Quit**, press the right arrow key on the keyboard to select **OK**, and press **Enter** to exit the nmtui utility. + +## Issue 3: Failed to Install the TensorFlow and Related Packages + +### Symptom + +Failed to install the TensorFlow and related packages using **yum**. + +### Cause Analysis + +The dependencies of TensorFlow have not been upgraded to the version that adapts to TensorFlow 2.12.1. You need to manually install the dependencies using **pip**. + +### Solution + +1. Run `yumdownloader python3-tensorflow` to download the TensorFlow RPM package. +2. Run `rpm -ivh --nodeps python3-tensorflow` to install the package. +3. Install TensorFlow dependencies. + 1. Use **pip** to install dependencies: `pip3 install tensorflow-estimator==2.12.0 keras==2.12.0 protobuf==3.20.3` + 2. Use **yum** to install other dependencies: `yum install python3-termcolor python3-future python3-numpy python3-six python3-astunparse python3-google-pasta python3-opt-einsum python3-typing-extensions python3-wrapt python3-h5py python3-grpcio python3-absl-py python3-flatbuffers python3-gast` +4. Use **yum** to install related packages. For example, run `yum install python-keras-rl2` to install python-keras-rl2. diff --git a/docs/en/server/installation_upgrade/installation/risc-v-more.md b/docs/en/server/installation_upgrade/installation/risc-v-more.md new file mode 100644 index 0000000000000000000000000000000000000000..7e6cd81b929f3ea58e5ef0ff8c7bc20ee364548e --- /dev/null +++ b/docs/en/server/installation_upgrade/installation/risc-v-more.md @@ -0,0 +1,4 @@ +# Reference + +- Play with OpenEuler on VisionFive +- Play with openEuler on RISC-V platform diff --git a/docs/en/server/installation_upgrade/installation/risc-v-qemu.md b/docs/en/server/installation_upgrade/installation/risc-v-qemu.md new file mode 100644 index 0000000000000000000000000000000000000000..29f5bdf8723d1b9a384e8e98c6963979584afb8b --- /dev/null +++ b/docs/en/server/installation_upgrade/installation/risc-v-qemu.md @@ -0,0 +1,341 @@ +# Installation Guide + +This chapter describes the installation of openEuler using QEMU as an example. For other installation methods, refer to the installation guides for the development boards. + +## Installing QEMU + +### System Environment + +The environments tested so far include WSL2 (Ubuntu 20.04.4 LTS and Ubuntu 22.04.1 LTS) and Ubuntu 22.04.1 live-server LTS. + +## Installing QEMU of the RISC-V Architecture + +Install the qemu-system-riscv64 package provided by the Linux distribution. openEuler 23.09 x86_64 provides QEMU 6.2.0 (**qemu-system-riscv-6.2.0-80.oe2309.x86_64**): + +``` bash +dnf install -y qemu-system-riscv +``` + +QEMU 8.0 and later versions are recommended because they provide a lot of fixes and updates for RISC-V. The following uses QEMU 8.1.2 as an example. + +### Manual Compilation and Installation + +If the provided software package is outdated, you can manually compile and install QEMU. + +``` bash +wget https://download.qemu.org/qemu-8.1.2.tar.xz +tar -xvf qemu-8.1.2.tar.xz +cd qemu-8.1.2 +mkdir res +cd res +sudo apt install libspice-protocol-dev libepoxy-dev libgtk-3-dev libspice-server-dev build-essential autoconf automake autotools-dev pkg-config bc curl gawk git bison flex texinfo gperf libtool patchutils mingw-w64 libmpc-dev libmpfr-dev libgmp-dev libexpat-dev libfdt-dev zlib1g-dev libglib2.0-dev libpixman-1-dev libncurses5-dev libncursesw5-dev meson libvirglrenderer-dev libsdl2-dev -y +../configure --target-list=riscv64-softmmu,riscv64-linux-user --prefix=/usr/local/bin/qemu-riscv64 --enable-slirp +make -j$(nproc) +sudo make install +``` + +The above commands will install QEMU to **/usr/local/bin/qemu-riscv64**. Add **/usr/local/bin/qemu-riscv64/bin** to **$PATH**. + +For compilation and installation on other OSs, including openEuler, see the QEMU official documentation. + +You can refer to the compilation procedure on RHEL or CentOS for the compilation on openEuler, for example: + +``` bash +sudo dnf install -y git glib2-devel libfdt-devel pixman-devel zlib-devel bzip2 ninja-build python3 \ + libaio-devel libcap-ng-devel libiscsi-devel capstone-devel \ + gtk3-devel vte291-devel ncurses-devel \ + libseccomp-devel nettle-devel libattr-devel libjpeg-devel \ + brlapi-devel libgcrypt-devel lzo-devel snappy-devel \ + librdmacm-devel libibverbs-devel cyrus-sasl-devel libpng-devel \ + libuuid-devel pulseaudio-libs-devel curl-devel libssh-devel \ + systemtap-sdt-devel libusbx-devel +curl -LO https://download.qemu.org/qemu-8.1.2.tar.xz +tar -xvf qemu-8.1.2.tar.xz +cd qemu-8.1.2 +mkdir res +cd res +../configure --target-list=riscv64-softmmu,riscv64-linux-user --prefix=/usr/local/bin/qemu-riscv64 +make -j$(nproc) +sudo make install +``` + +## Preparing the openEuler RISC-V Disk Image + +### Disk Image Download + +Download the boot firmware (**fw_payload_oe_uboot_2304.bin**), disk image (**openEuler-23.09-RISC-V-qemu-riscv64.qcow2.xz**) and startup script (**start_vm.sh**). + +### Download Directory + +The current build is located in the [openEuler Repo](https://repo.openeuler.org/openEuler-23.09/virtual_machine_img/riscv64/). You can also visit the [openEuler official website](https://www.openeuler.org/zh/download/) to obtain the image from other mirrored sources. + +### Content Description + +- **fw_payload_oe_uboot_2304.bin**: Boot firmware +- **openEuler-23.09-RISC-V-qemu-riscv64.qcow2.xz**: Compressed disk image for the openEuler RISC-V QEMU virtual machine (VM) +- **openEuler-23.09-RISC-V-qemu-riscv64.qcow2.xz.sha256sum**: Verification file for the compressed disk image. Run `sha256sum -c openEuler-23.09-RISC-V-qemu-riscv64.qcow2.xz.sha256sum` for verification. +- **start_vm.sh**: Official VM startup script + +### (Optional) Copy-On-Write (COW) Disk Configuration + +> The copy-on-write (COW) technology does not make changes to the original image file; the modification is written to another image file. This feature is only supported by the QCOW format in QEMU. Multiple disk images can point to the same image for testing multiple configurations without damaging the original image. + +#### Creating an Image + +Run the following command to create an image, and use the new image when starting the virtual machine below. Assume that the original image is **openEuler-23.09-RISC-V-qemu-riscv64.qcow2**, and the new image is **test.qcow2**. + +``` bash +qemu-img create -o backing_file=openEuler-23.09-RISC-V-qemu-riscv64.qcow2,backing_fmt=qcow2 -f qcow2 test.qcow2 +``` + +#### Viewing Image Information + +``` bash +qemu-img info --backing-chain test.qcow2 +``` + +#### Modifying the Base Image Location + +Run the following command to modify the base image location. Assume that the new base image is **another.qcow2**, and the image to be modified is **test.qcow2**. + +``` bash +qemu-img rebase -b another.qcow2 test.qcow2 +``` + +#### Merging Images + +Merge the modified image into the original image. Assume that the new image is **test.qcow2**. + +``` bash +qemu-img commit test.qcow2 +``` + +#### Expanding Root Partition + +To expand the root partition for more available space, perform the operations below. + +Expand the disk image. + +``` bash +qemu-img resize test.qcow2 +100G +``` + +Output + +``` text +Image resized. +``` + +Start the VM and run the following command to check the disk size. + +``` bash +lsblk +``` + +List the partition information. + +``` bash +fdisk -l +``` + +Modify the root Partition. + +``` bash +fdisk /dev/vda +Welcome to fdisk (util-linux 2.35.2). +Changes will remain in memory only, until you decide to write them. +Be careful before using the write command. + +Command (m for help): p # Display partition information. +Disk /dev/vda: 70 GiB, 75161927680 bytes, 146800640 sectors +Units: sectors of 1 * 512 = 512 bytes +Sector size (logical/physical): 512 bytes / 512 bytes +I/O size (minimum/optimal): 512 bytes / 512 bytes +Disklabel type: dos +Disk identifier: 0x247032e6 + +Device Boot Start End Sectors Size Id Type +/dev/vda1 2048 4194303 4192256 2G e W95 FAT16 (LBA) +/dev/vda2 4194304 83886079 79691776 38G 83 Linux + +Command (m for help): d # Delete the existing partition. +Partition number (1,2, default 2): 2 + +Partition 2 has been deleted. + +Command (m for help): n # Create new partition. +Partition type + p primary (1 primary, 0 extended, 3 free) + e extended (container for logical partitions) +Select (default p): p # Select the primary partition. +Partition number (2-4, default 2): 2 +First sector (4194304-146800639, default 4194304): # The starting block here should be consistent with the /dev/vda2 mentioned above. +Last sector, +/-sectors or +/-size{K,M,G,T,P} (4194304-146800639, default 146800639): # Keep the default and allocate directly to the end. + +Created a new partition 2 of type 'Linux' and of size 68 GiB. +Partition #2 contains a ext4 signature.Do you want to remove the signature? [Y]es/[N]o: n + +Command (m for help): p # Check again. + +Disk /dev/vda: 70 GiB, 75161927680 bytes, 146800640 sectors +Units: sectors of 1 * 512 = 512 bytes +Sector size (logical/physical): 512 bytes / 512 bytes +I/O size (minimum/optimal): 512 bytes / 512 bytes +Disklabel type: dos +Disk identifier: 0x247032e6 + +Device Boot Start End Sectors Size Id Type +/dev/vda1 2048 4194303 4192256 2G e W95 FAT16 (LBA) +/dev/vda2 4194304 146800639 142606336 68G 83 Linux + +Command (m for help): w # Write to the disk. +The partition table has been altered. +Syncing disks. +``` + +Update Disk Information. + +``` bash +resize2fs /dev/vda2 +``` + +## Launching the openEuler RISC-V VM + +### Starting the VM + +- Ensure that the current directory contains **fw_payload_oe_uboot_2304.bin**, the disk image compressed file, and the startup script. +- Decompress the image file with `xz -dk openEuler-23.09-RISC-V-qemu-riscv64.qcow2.xz`. +- Adjust the startup parameters. +- Run the startup script using `$ bash start_vm.sh`. + +### (Optional) Adjusting Startup Parameters + +- **vcpu** specifies the number of QEMU threads, which does not strictly correspond to the number of CPU cores. If the **vcpu** value exceeds the number of CPU cores of the host machine, the execution may be slowed or blocked. The default value is **4**. +- **memory** specifies the size of the VM memory and can be adjusted as needed. The default value is **2**. +- **drive** specifies the path of the virtual disk. If you configure the COW image as mentioned earlier, enter the path of the created image. +- **fw** specifies the path for the U-Boot image. +- **ssh_port** specifies the port forwarded for SSH, defaulting to **12055**. Set it to blank to disable this feature. + +## Logging into the VM + +The script offers support for SSH logins. + +If the VM is exposed to a public network, change the **root** user password immediately after logging in. + +### SSH Login + +Secure Shell (SSH) is an encrypted network transfer protocol, offering a secure transfer environment for network services over an unsecured network. SSH achieves this by creating a secure tunnel to connect SSH clients and servers. The most common use for SSH is for remote OS login, including the remote CLI and remote command execution. While SSH is most frequently used on Unix-like OSs, some of its functions are also available on Windows. In 2015, Microsoft announced they would provide native SSH protocol support in future Windows OSs. The OpenSSH client is available in Windows 10 1803 and later versions. + +- Username: **root** or **openeuler** +- Default Password: **openEuler12#$** +- Login method: See script prompts (or use your preferred SSH client). + +Upon successful login, you will see the following information: + +``` bash +Authorized users only. All activities may be monitored and reported. + +Authorized users only. All activities may be monitored and reported. +Last login: Sun Oct 15 17:19:52 2023 from 10.0.2.2 + +Welcome to 6.4.0-10.1.0.20.oe2309.riscv64 + +System information as of time: Sun Oct 15 19:40:07 CST 2023 + +System load: 0.47 +Processes: 161 +Memory used: .7% +Swap used: 0.0% +Usage On: 11% +IP address: 10.0.2.15 +Users online: 1 + +[root@openeuler ~]# +``` + +### VNC Login + +This method is natively supported by QEMU and is similar to remotely operating a physical machine without sound. + +> Virtual Network Computing (VNC) is a screen sharing and remote operation program that uses the Remote Frame Buffer (RFB) protocol. VNC sends keyboard and mouse actions as well as real-time screen images through the network. +> +> VNC is independent of the OS, which means it can be used across platforms. For instance, a Windows computer can connect to a Linux computer, and vice versa. You can even use VNC through a Java-supported browser on a computer without the client software. + +#### Installing VNC Viewer + +Download [TigerVNC](https://sourceforge.net/projects/tigervnc/files/stable/) or [VNC Viewer](https://www.realvnc.com/en/connect/download/viewer/). + +#### Modifying the Startup Script + +Before the **sleep 2** line in the startup script, add the following content: + +``` bash +vnc_port=12056 +echo -e "\033[37mVNC Port: \033[0m \033[34m"$vnc_port"\033[0m" +cmd="${cmd} -vnc :"$((vnc_port-5900)) +``` + +#### Connecting to VNC + +Launch TigerVNC or VNC Viewer, paste the address, and press **Enter**. The operating interface is similar to that of a physical machine. + +## Modifying the Default Software Source Configuration + +The software source for openEuler 23.09 RISC-V version currently only contains the **\[OS]** and **\[source]** repositories. However, the default configuration file includes other repositories not provided in this RISC-V version. + +Before using a package manager to install software packages, edit the software source configuration to keep only the **\[OS]** and **\[source]** sections. + +Connect to the VM via SSH or VNC and log in as the **root** user (if you log in as a non-privileged user, run the commands with `sudo`). Then, perform the following operations. + +### Modify /etc/yum.repos.d/openEuler.repo + +``` bash +vi /etc/yum.repos.d/openEuler.repo +# or: nano /etc/yum.repos.d/openEuler.repo +``` + +Remove the **\[everything]**, **\[EPOL]**, **\[debuginfo]**, **\[update]**, and **\[update-source]** sections to keep only the **\[OS]** and **\[source]** sections. + +After making the changes, the configurations in **openEuler.repo** should be similar to the following: + +``` text +#generic-repos is licensed under the Mulan PSL v2. +#You can use this software according to the terms and conditions of the Mulan PSL v2. +#You may obtain a copy of Mulan PSL v2 at: +# http://license.coscl.org.cn/MulanPSL2 +#THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR +#IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FIT FOR A PARTICULAR +#PURPOSE. +#See the Mulan PSL v2 for more details. + +[OS] +name=OS +baseurl=http://repo.openeuler.org/openEuler-23.09/OS/$basearch/ +metalink=https://mirrors.openeuler.org/metalink?repo=$releasever/OS&arch=$basearch +metadata_expire=1h +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-23.09/OS/$basearch/RPM-GPG-KEY-openEuler + +[source] +name=source +baseurl=http://repo.openeuler.org/openEuler-23.09/source/ +metalink=https://mirrors.openeuler.org/metalink?repo=$releasever&arch=source +metadata_expire=1h +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-23.09/source/RPM-GPG-KEY-openEuler +``` + +Then, you can use the DNF package manager to install software packages normally. When installing for the first time, it is necessary to import the openEuler GPG key. If you see the following prompt, enter **y** to confirm. + +``` text +retrieving repo key for OS unencrypted from http://repo.openeuler.org/openEuler-23.09/OS/riscv64/RPM-GPG-KEY-openEuler +OS 18 kB/s | 2.1 kB 00:00 +Importing GPG key 0xB25E7F66: + Userid : "private OBS (key without passphrase) " + Fingerprint: 12EA 74AC 9DF4 8D46 C69C A0BE D557 065E B25E 7F66 + From : http://repo.openeuler.org/openEuler-23.09/OS/riscv64/RPM-GPG-KEY-openEuler +Is this ok [y/N]: y +Key imported successfully +``` diff --git a/docs/en/server/installation_upgrade/installation/risc-v.md b/docs/en/server/installation_upgrade/installation/risc-v.md new file mode 100644 index 0000000000000000000000000000000000000000..042190766519991c91849ab2a405ebaf590a64af --- /dev/null +++ b/docs/en/server/installation_upgrade/installation/risc-v.md @@ -0,0 +1,5 @@ +# RISC-V Installation Guide + +This document describes how to install openEuler on RISC-V. + +This document is intended for openEuler users with basic understanding of Linux system management. diff --git a/docs/en/server/installation_upgrade/installation/server-installation-common-issues-and-solutions.md b/docs/en/server/installation_upgrade/installation/server-installation-common-issues-and-solutions.md new file mode 100644 index 0000000000000000000000000000000000000000..d053b9345cbe7193cb49bd0ba3419aa88626357c --- /dev/null +++ b/docs/en/server/installation_upgrade/installation/server-installation-common-issues-and-solutions.md @@ -0,0 +1,344 @@ +# Common Issues and Solutions + +## Issue 1: openEuler Fails to Start After It Is Installed to the Second Drive + +### Symptom + +The OS is installed on the second drive **sdb** during the installation, causing startup failure. + +### Possible Causes + +When openEuler is installed to the second drive, MBR and GRUB are installed to the second drive **sdb** by default. The following two situations may occur: + +1. openEuler installed on the first drive is loaded and started if it is complete. +2. openEuler installed on the first drive fails to be started from hard drives if it is incomplete. + +The preceding two situations occur because the first drive **sda** is booted by default to start openEuler in the BIOS window. If openEuler is not installed on the **sda** drive, system restart fails. + +### Solutions + +This problem can be solved using either of the following two methods: + +- During the openEuler installation, select the first drive or both drives, and install the boot loader on the first drive **sda**. +- After installing openEuler, restart it by modifying the boot option in the BIOS window. + +## Issue 2: openEuler Enters Emergency Mode After It Is Started + +### Symptom + +openEuler enters emergency mode after it is powered on. + +![fig](./figures/en-us_image_0229291264.jpg) + +### Possible Causes + +Damaged OS files result in drive mounting failure, or overpressured I/O results in drive mounting timeout \(threshold: 90s\). + +An unexpected system power-off and low I/O performance of drives may also cause the problem. + +### Solutions + +1. Log in to openEuler as the **root** user. +2. Check and restore files by using the file system check \(fsck\) tool, and restart openEuler. + + > [!NOTE]NOTE + > The fsck tool checks and maintains inconsistent file systems. If the system is powered off or a drive is faulty, run the **fsck** command to check file systems. Run the **fsck.ext3 -h** and **fsck.ext4 -h** commands to view the usage method of the fsck tool. + +If you want to disable the timeout mechanism of drive mounting, add **x-systemd.device-timeout=0** to the **etc/fstab** file. For example: + +```sh +# +# /etc/fstab +# Created by anaconda on Mon Sep 14 17:25:48 2015 +# +# Accessible filesystems, by reference, are maintained under '/dev/disk' +# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info +# +/dev/mapper/openEuler-root / ext4 defaults,x-systemd.device-timeout=0 0 0 +UUID=afcc811f-4b20-42fc-9d31-7307a8cfe0df /boot ext4 defaults,x-systemd.device-timeout=0 0 0 +/dev/mapper/openEuler-home /home ext4 defaults 0 0 +/dev/mapper/openEuler-swap swap swap defaults 0 0 +``` + +## Issue 3: openEuler Fails to Be Reinstalled When an Unactivated Logical Volume Group Exists + +### Symptom + +After a drive fails, openEuler fails to be reinstalled because a logical volume group that cannot be activated exists in openEuler. + +### Possible Causes + +During the installation of openEuler, a logical volume group cannot be activated. + +### Solutions + +Before reinstalling openEuler, restore the abnormal logical volume group to the normal status or clear it. For example: + +- Restore the logical volume group. + 1. Run the following command to clear the active status of the abnormal logical volume group to ensure that the error message "Can't open /dev/sdc exclusively mounted filesystem" is not displayed: + + ```sh + vgchange -a n testvg32947 + ``` + + 2. Run the following command to recreate a physical volume based on the backup file: + + ```sh + pvcreate --uuid JT7zlL-K5G4-izjB-3i5L-e94f-7yuX-rhkLjL --restorefile /etc/lvm/backup/testvg32947 /dev/sdc + ``` + + 3. Run the following command to restore the logical volume group information: + + ```sh + vgcfgrestore testvg32947 + ``` + + 4. Run the following command to reactivate the logical volume group: + + ```sh + vgchange -ay testvg32947 + ``` + +- Run the following commands to clear the logical volume group: + + ```sh + vgchange -a n testvg32947 + vgremove -y testvg32947 + ``` + +## Issue 4: An Exception Occurs During the Selection of the Installation Source + +### Symptom + +After the installation source is selected, the message "Error checking software selection" is displayed. + +### Possible Causes + +This is because the software package dependency in the installation source is abnormal. + +### Solutions + +Check whether the installation source is abnormal. Use the new installation source. + +## Issue 5: Kdump Service Fails to Be Enabled + +### Symptom + +Run the **systemctl status kdump** command. The following information is displayed, indicating that no memory is reserved. + +![fig](./figures/en-us_image_0229291280.png) + +### Possible Causes + +The kdump service requires the system to reserve memory for running the kdump kernel. However, the system does not reserve memory for the kdump service. As a result, the kdump service cannot be started. + +### Solutions + +For the scenario where the OS has been installed + +1. Add **crashkernel=1024M,high** to **/boot/efi/EFI/openEuler/grub.cfg**. +2. Restart the system for configuration to take effect. +3. Run the following command to check the kdump status: + + ```sh + systemctl status kdump + ``` + + If the following information is displayed, the kdump status is **active**, indicating that the kdump service is enabled. No further action is required. + + ![fig](./figures/en-us_image_0229291272.png) + +### Parameter Description + +The following table describes the parameters of the memory reserved for the kdump kernel. + +**Table 1** crashkernel parameters + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Kernel Boot Parameter

+

Description

+

Default Value

+

Remarks

+

crashkernel=X

+

Reserve X of the physical memory for kdump when the physical memory is less than 4 GB.

+

None. You can adjust the value as required.

+

This configuration method is used only when the memory is less than 4 GB. Ensure that the continuous available memory is sufficient.

+

crashkernel=X@Y

+

Reserve X of the memory at the start address Y for kdump.

+

None. You can adjust the value as required.

+

Ensure that the X of the memory at the start address Y is not reserved for other modules.

+

crashkernel=X,high

+

Reserve 256 MB of the physical memory for kdump when the physical memory is less than 4 GB, and X of the physical memory for kdump when the physical memory is greater than or equal to 4 GB.

+

None. You can adjust the value based as required. The recommended value is 1024M,high.

+

Ensure that 256 MB of the memory is reserved for continuous use when the physical memory is less than 4 GB and X of the memory is reserved when the physical memory is greater than or equal to 4 GB. The actual reserved memory size equals 256 MB plus X.

+

crashkernel=X,low

+

crashkernel=Y,high

+

Reserve X of the physical memory for kdump when the physical memory is less than 4 GB and Y of the physical memory for kdump when the physical memory is greater than or equal to 4 GB.

+

None. You can adjust the value as required.

+

Ensure that X of the memory is reserved for continuous use when the physical memory is less than 4 GB and Y of the memory is reserved when the physical memory is greater than or equal to 4 GB. The actual reserved memory size equals X plus Y.

+
+ +## Issue 6: Fails to Select Only One Drive for Reinstallation When openEuler Is Installed on a Logical Volume Consisting of Multiple Drives + +### Symptom + +If openEuler is installed on a logical volume consisting of multiple drives, an error message will be displayed as shown in [Figure 1](#fig115949762617) when you attempt to select one of the drives for reinstallation. + +**Figure 1** Error message +![fig](./figures/error-message.png) + +### Possible Causes + +The previous logical volume contains multiple drives. If you select one of the drives for reinstallation, the logical volume will be damaged. + +### Solutions + +The logical volume formed by multiple drives is equivalent to a volume group. Therefore, you only need to delete the corresponding volume group. + +1. Press **Ctrl**+**Alt**+**F2** to switch to the CLI and run the following command to find the volume group: + + ```sh + vgs + ``` + + ![fig](./figures/en-us_image_0231657950.png) + +2. Run the following command to delete the volume group: + + ```sh + vgremove euleros + ``` + +3. Run the following command to restart the installation program for the modification to take effect: + + ```sh + systemctl restart anaconda + ``` + + > [!NOTE]NOTE + > You can also press **Ctrl**+**Alt**+**F6** to return to the GUI and click **Refresh** in the lower right corner to refresh the storage configuration. + +## Issue 7: openEuler Fails to Be Installed on an x86 PM in UEFI Mode due to Secure Boot Option Setting + +### Symptom + +During the installation of openEuler on an x86 PM in UEFI mode, the system stays at the "No bootable device" page and the installation cannot continue because **secure boot** is set to **enabled** \(by default, it is set to **disabled**\), as shown in [Figure 2](#fig115949762618). + +**Figure 2** Dialog box showing "No bootable device" +![fig](./figures/dialog-box-showing-no-bootable-device.png) + +### Possible Causes + +After **Secure Boot** is set to **Enabled**, the mainboard verifies the boot program and OS. If the boot program and OS are not signed using the corresponding private key, they cannot pass the authentication of the built-in public key on the mainboard. + +### Solutions + +Access the BIOS, set **Secure Boot** to **Disabled**, and reinstall the openEuler. + +1. During the system startup, press **F11** and enter the password **Admin@9000** to access the BIOS. + + ![fig](./figures/bios.png) + +2. Choose **Administer Secure Boot**. + + ![fig](./figures/security.png) + +3. Set **Enforce Secure Boot** to **Disabled**. + + ![fig](./figures/enforce-secure-boot.png) + + > [!NOTE]NOTE + > After **Enforce Secure Boot** is set to **Disabled**, save the settings and exit. Then, reinstall the system. + +## Issue 8: pmie_check Is Reported in the messages Log During openEuler Installation + +### Symptom + +During the OS installation, if you click **Server > Performance tool**, PCP is installed. After the OS is installed and restarted, an error "pmie_check failed in /usr/share/pcp/lib/pmie" is displayed in the **/var/log/messages** log. + +### Possible Causes + +anaconda does not support the installation of SELinux policy module in the chroot environment. During the pcp-selinux installation, the postin script fails to execute the PCP-related SELinux policy module. As a result, an error is reported after the OS is restarted. + +### Solutions + +After the OS is installed and restarted, perform either of the following two operations: + +1. Install SElinux policy module pcpupstream. + + ```sh + /usr/libexec/pcp/bin/selinux-setup /var/lib/pcp/selinux install "pcpupstream" + ``` + +2. Reinstall pcp-selinux + + ```sh + sudo dnf reinstall pcp-selinux + ``` + +## Issue 9: Installation Fails when a User Selects Two Drives with OS Installed and Customizes Partitioning + +### Symptom + +During the OS installation, the OS has been installed on two drives. In this case, if you select one drive for custom partitioning, and click **Cancel** to perform custom partitioning on the other drive, the installation fails. + +![fig](./figures/cancle_disk.png) + +![fig](./figures/custom_paratition.png) + +### Possible Causes + +A user selects a drive for partitioning twice. After the user clicks **Cancel** and then selects the other drive, the drive information is incorrect. As a result, the installation fails. + +### Solutions + +Select the target drive for custom partitioning. Do not frequently cancel the operation. If you have to cancel and select another drive, you are advised to reinstall the OS. + +### Learn More About the Issue at + + + +## Issue 10: vmcore Fails to Be Generated by Kdump on the PM with LSI MegaRAID Card Installed + +### Symptom + +After the Kdump service is deployed, kernel breaks down due to the manual execution of the **echo c > /proc/sysrq-trigger** command or kernel fault. When Kdump enables second kernel, an error "BRCM Debug mfi stat 0x2d, data len requested/completed 0x200/0x0" is reported in the MegaRAID driver, as shown in the following figure. As a result, vmcore fails to be generated. + +![Error information](./figures/Megaraid_IO_Request_uncompleted.png) + +### Possible Causes + +The **reset_devices** parameter is configured by default and is enabled during second kernel startup, making MegaRAID driver or drive faulty. An error is reported when the vmcore file is dumped ana accesses the MegaRAID card. As a result, vmcore fails to be generated. + +### Solutions + +Delete the **reset_devices** parameter in the **etc/sysconfig/kdump** file on a PM, as shown in the following figure. Therefore, the I/O request will be responded when the MegaRAID driver resets the device during the second kernel startup, and vmcore will be successfully generated. + +![Deleting reset_devices](./figures/reset_devices.png) diff --git a/docs/en/server/installation_upgrade/installation/using-kickstart-for-automatic-installation.md b/docs/en/server/installation_upgrade/installation/using-kickstart-for-automatic-installation.md new file mode 100644 index 0000000000000000000000000000000000000000..7ebf40cc9c7d3d610797dbafb1be2760213129cf --- /dev/null +++ b/docs/en/server/installation_upgrade/installation/using-kickstart-for-automatic-installation.md @@ -0,0 +1,366 @@ +# Using Kickstart for Automatic Installation + + + +- [Using Kickstart for Automatic Installation](#using-kickstart-for-automatic-installation) + - [Introduction](#introduction) + - [Overview](#overview) + - [Advantages and Disadvantages](#advantages-and-disadvantages) + - [Background](#background) + - [Semi-automatic Installation Guide](#semi-automatic-installation-guide) + - [Environment Requirements](#environment-requirements) + - [Procedure](#procedure) + - [Full-automatic Installation Guide](#full-automatic-installation-guide) + - [Environment Requirements](#environment-requirements-1) + - [Procedure](#procedure-1) + + + +## Introduction + +### Overview + +You can use the kickstart tool to automatically install the openEuler OS in either of the following ways: + +- Semi-automatic installation: You only need to specify the location of the kickstart file. Kickstart automatically configures OS attributes such as keyboard, language, and partitions. +- Automatic installation: The OS is automatically installed. + +### Advantages and Disadvantages + +[Table 1](#table1388812373315) lists the advantages and disadvantages of semi-automatic installation and full-automatic installation using kickstart. You can select an installation mode as required. + +**Table 1** Advantages and disadvantages + + + + + + + + + + + + + + + + +

Installation Mode

+

Advantage

+

Disadvantage

+

Semi-automatic installation

+

Services such as TFTP, PXE, and DHCP do not need to be prepared.

+

You need to manually specify the path of the kickstart file.

+

Full-automatic installation

+

The OS is installed automatically.

+

Services, such as TFTP, DHCPD, and PXE, need to be configured.

+
+ +### Background + +**Kickstart** + +Kickstart is an unattended installation mode. The principle of kickstart is to record typical parameters that need to be manually entered during the installation and generate the configuration file **ks.cfg**. During the installation, the installation program searches the **ks.cfg** configuration file first for required parameters. If no matching parameters are found, you need to manually configure these parameters. If all required parameters are covered by the kickstart file, automatic installation can be achieved by only specifying the path of the kickstart file. + +Both full-automatic or semi-automatic installation can be achieved by kickstart. + +**PXE** + +Pre-boot Execution Environment \(PXE\)\) works in client/server network mode. The PXE client can obtain an IP address from the DHCP server during the startup and implement client boot and installation through the network based on protocols such as trivial file transfer protocol \(TFTP\). + +**TFTP** + +TFTP is used to transfer simple and trivial files between clients and the server. + +## Semi-automatic Installation Guide + +### Environment Requirements + +The environment requirements for semi-automatic installation of openEuler OS using kickstart are as follows: + +- PM/VM \(For details about how to create VMs, see the documents from corresponding vendors\): includes the computer where kickstart is used for automatic installation and the computer where the kickstart tool is installed. +- Httpd: stores the kickstart file. +- ISO: openEuler-21.09-aarch64-dvd.iso + +### Procedure + +To use kickstart to perform semi-automatic installation of openEuler, perform the following steps: + +**Environment Preparation** + +> [!NOTE]NOTE +> Before the installation, ensure that the firewall of the HTTP server is disabled. Run the following command to disable the firewall: +> +> ```shell +> iptables -F +> ``` + +1. Install httpd and start the service. + + ```shell + # dnf install httpd -y + # systemctl start httpd + # systemctl enable httpd + ``` + +2. Run the following commands to prepare the kickstart file: + + ```shell + # mkdir /var/www/html/ks + #vim /var/www/html/ks/openEuler-ks.cfg ===>The file can be obtained by modifying the **anaconda-ks.cfg** file automatically generated from openEuler OS. + ==================================== + ***Modify the following information as required.*** + #version=DEVEL + ignoredisk --only-use=sda + autopart --type=lvm + # Partition clearing information + clearpart --none --initlabel + # Use graphical install + graphical + # Use CDROM installation media + cdrom + # Keyboard layouts + keyboard --vckeymap=cn --xlayouts='cn' + # System language + lang zh_CN.UTF-8 + + # Network information + network --bootproto=dhcp --device=enp4s0 --ipv6=auto --activate + network --hostname=openeuler.com + # Root password + rootpw --iscrypted $6$fQE83lxEZ48Or4zc$j7/PlUMHn29yTjCD4Fi44WTZL/RzVGxJ/7MGsZMl6QfE3KjIVT7M4UrhFXbafvRq2lUddAFcyWHd5WRmXfEK20 + # Run the Setup Agent on first boot + firstboot --enable + # Do not configure the X Window System + skipx + # System services + services --disabled="chronyd" + # System timezone + timezone Asia/Shanghai --utc --nontp + + %packages + @^minimal-environment + @standard + + %end + + %anaconda + pwpolicy root --minlen=8 --minquality=1 --notstrict --nochanges --notempty + pwpolicy user --minlen=8 --minquality=1 --notstrict --nochanges --emptyok + pwpolicy luks --minlen=8 --minquality=1 --notstrict --nochanges --notempty + %end + + %post + #enable kdump + sed -i "s/ ro / ro crashkernel=1024M,high /" /boot/efi/EFI/openEuler/grub.cfg + %end + ===================================== + ``` + + > [!NOTE]NOTE + > The method of generating the password ciphertext is as follows: + + ```python + # python3 + Python 3.7.0 (default, Apr 1 2019, 00:00:00) + [GCC 7.3.0] on linux + Type "help", "copyright", "credits" or "license" for more information. + >>> import crypt + >>> passwd = crypt.crypt("myPasswd") + >>> print (passwd) + $6$63c4tDmQGn5SDayV$mZoZC4pa9Jdt6/ALgaaDq6mIExiOO2EjzomB.Rf6V1BkEMJDcMddZeGdp17cMyc9l9ML9ldthytBEPVcnboR/0 + ``` + +3. Mount the ISO image file to the CD-ROM drive of the computer where openEuler is to be installed. + + If you want to install openEuler through the NFS, specify the path \(which is **cdrom** by default\) of installation source in the kickstart file. + +**Installing the System** + +1. The installation selection dialog box is displayed. + 1. On the installation wizard page in [Starting the Installation](installation-modes.md#starting-the-installation), select **Install openEuler 21.09** and press **e**. + 2. Add **inst.ks=** to the startup parameters. + + ![startparam.png](https://gitee.com/openeuler/docs/raw/master/docs/zh/docs/Installation/figures/startparam.png "startparam.png") + + 3. Press **Ctrl**+**x** to start the automatic installation. + +2. Verify that the installation is complete. + + After the installation is complete, the system automatically reboots. If the first boot option of the system is set to the CD_ROM, the installation page is displayed again. Shut down the computer and change startup option to start from the hard disk preferentially. + + ![](./figures/completing-the-automatic-installation.png) + +## Full-automatic Installation Guide + +### Environment Requirements + +The environment requirements for full-automatic installation of openEuler using kickstart are as follows: + +- PM/VM \(For details about how to create VMs, see the documents from corresponding vendors\): includes the computer where kickstart is used for automatic installation and the computer where the kickstart tool is installed. +- Httpd: stores the kickstart file. +- TFTP: provides vmlinuz and initrd files. +- DHCPD/PXE: provides the DHCP service. +- ISO: openEuler-21.09-aarch64-dvd.iso + +### Procedure + +To use kickstart to perform full-automatic installation of openEuler, perform the following steps: + +**Environment Preparation** + +> [!NOTE]NOTE +> Before the installation, ensure that the firewall of the HTTP server is disabled. Run the following command to disable the firewall: +> +> ```shell +> iptables -F +> ``` + +1. Install httpd and start the service. + + ```shell + # dnf install httpd -y + # systemctl start httpd + # systemctl enable httpd + ``` + +2. Install and configure TFTP. + + ```shell + # dnf install tftp-server -y + # vim /etc/xinetd.d/tftp + service tftp + { + socket_type = dgram + protocol = udp + wait = yes + user = root + server = /usr/sbin/in.tftpd + server_args = -s /var/lib/tftpboot + disable = no + per_source = 11 + cps = 100 2 + flags = IPv4 + } + # systemctl start tftp + # systemctl enable tftp + # systemctl start xinetd + # systemctl status xinetd + # systemctl enable xinetd + ``` + +3. Prepare the installation source. + + ```shell + # mount openEuler-21.09-aarch64-dvd.iso /mnt + # cp -r /mnt/* /var/www/html/openEuler/ + ``` + +4. Set and modify the kickstart configuration file **openEuler-ks.cfg**. Select the HTTP installation source by referring to [3](#en-us_topic_0229291289_l1692f6b9284e493683ffa2ef804bc7ca). + + ```shell + #vim /var/www/html/ks/openEuler-ks.cfg + ==================================== + ***Modify the following information as required.*** + #version=DEVEL + ignoredisk --only-use=sda + autopart --type=lvm + # Partition clearing information + clearpart --none --initlabel + # Use graphical install + graphical + # Keyboard layouts + keyboard --vckeymap=cn --xlayouts='cn' + # System language + lang zh_CN.UTF-8 + #Use http installation source + url --url=//192.168.122.1/openEuler/ + %post + #enable kdump + sed -i "s/ ro / ro crashkernel=1024M,high /" /boot/efi/EFI/openEuler/grub.cfg + %end + ... + ``` + +5. Modify the PXE configuration file **grub.cfg** as follows. (Note: Currently, openEuler does not support the cfg file in bls format.) + + ```shell + # cp -r /mnt/images/pxeboot/* /var/lib/tftpboot/ + # cp /mnt/EFI/BOOT/grubaa64.efi /var/lib/tftpboot/ + # cp /mnt/EFI/BOOT/grub.cfg /var/lib/tftpboot/ + # ls /var/lib/tftpboot/ + grubaa64.efi grub.cfg initrd.img TRANS.TBL vmlinuz + # vim /var/lib/tftpboot/grub.cfg + set default="1" + + function load_video { + if [ x$feature_all_video_module = xy ]; then + insmod all_video + else + insmod efi_gop + insmod efi_uga + insmod ieee1275_fb + insmod vbe + insmod vga + insmod video_bochs + insmod video_cirrus + fi + } + + load_video + set gfxpayload=keep + insmod gzio + insmod part_gpt + insmod ext2 + + set timeout=60 + + + ### BEGIN /etc/grub.d/10_linux ### + menuentry 'Install openEuler 21.03 ' --class red --class gnu-linux --class gnu --class os { + set root=(tftp,192.168.1.1) + linux /vmlinuz ro inst.geoloc=0 console=ttyAMA0 console=tty0 rd.iscsi.waitnet=0 inst.ks=http://192.168.122.1/ks/openEuler-ks.cfg + initrd /initrd.img + } + ``` + +6. Configure DHCP \(which can be replaced by DNSmasq\). + + ```shell + # dnf install dhcp -y + # + # DHCP Server Configuration file. + # see /usr/share/doc/dhcp-server/dhcpd.conf.example + # see dhcpd.conf(5) man page + # + # vim /etc/dhcp/dhcpd.conf + ddns-update-style interim; + ignore client-updates; + filename "grubaa64.efi"; # location of the pxelinux startup file; + next-server 192.168.122.1; # (IMPORTANT) IP address of the TFTP server; + subnet 192.168.122.0 netmask 255.255.255.0 { + option routers 192.168.111.1; # Gateway address + option subnet-mask 255.255.255.0; # Subnet mask + range dynamic-bootp 192.168.122.50 192.168.122.200; # Dynamic IP address range + default-lease-time 21600; + max-lease-time 43200; + } + # systemctl start dhcpd + # systemctl enable dhcpd + ``` + +**Installing the System** + +1. On the **Start boot option** screen, press **F2** to boot from the PXE and start automatic installation. + + ![](./figures/en-us_image_0229291270.png) + + ![](./figures/en-us_image_0229291286.png) + + ![](./figures/en-us_image_0229291247.png) + +2. The automatic installation window is displayed. +3. Verify that the installation is complete. + + ![](./figures/completing-the-automatic-installation.png) diff --git a/docs/en/server/installation_upgrade/upgrade/_toc.yaml b/docs/en/server/installation_upgrade/upgrade/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..6502f02ee567e269e7bb3d12c582d201f01a0423 --- /dev/null +++ b/docs/en/server/installation_upgrade/upgrade/_toc.yaml @@ -0,0 +1,6 @@ +label: Upgrade Guide +isManual: true +description: Upgrading openEuler +sections: + - label: Upgrade and Downgrade Guide + href: ./openeuler-22.03-lts-upgrade-and-downgrade-guide.md diff --git a/docs/en/server/installation_upgrade/upgrade/images/LTS_version.png b/docs/en/server/installation_upgrade/upgrade/images/LTS_version.png new file mode 100644 index 0000000000000000000000000000000000000000..84132f5825f1bdc743736ab2c65cd23c7ec9afa9 Binary files /dev/null and b/docs/en/server/installation_upgrade/upgrade/images/LTS_version.png differ diff --git a/docs/en/server/installation_upgrade/upgrade/images/SP1_repo.png b/docs/en/server/installation_upgrade/upgrade/images/SP1_repo.png new file mode 100644 index 0000000000000000000000000000000000000000..42336cddfc122937e4ac2a8ce07182fa166ce942 Binary files /dev/null and b/docs/en/server/installation_upgrade/upgrade/images/SP1_repo.png differ diff --git a/docs/en/server/installation_upgrade/upgrade/images/SP1_version.png b/docs/en/server/installation_upgrade/upgrade/images/SP1_version.png new file mode 100644 index 0000000000000000000000000000000000000000..d22f8523c22dbd094d21ccdfb86446062b63b5ea Binary files /dev/null and b/docs/en/server/installation_upgrade/upgrade/images/SP1_version.png differ diff --git a/docs/en/server/installation_upgrade/upgrade/openeuler-22.03-lts-upgrade-and-downgrade-guide.md b/docs/en/server/installation_upgrade/upgrade/openeuler-22.03-lts-upgrade-and-downgrade-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..50254d8c7d67c8a70ef873c6d4bf5127cc3a8bbe --- /dev/null +++ b/docs/en/server/installation_upgrade/upgrade/openeuler-22.03-lts-upgrade-and-downgrade-guide.md @@ -0,0 +1,73 @@ +# Upgrade and Downgrade Guide + +This document describes how to upgrade openEuler 22.03 LTS to openEuler 22.03 LTS SP1. The operations for other versions are similar. + +## 1. OS Installation + +Obtain an openEuler 22.03 LTS image and install the OS by referring to the installation guide. + +View the versions of openEuler and the kernel in the current environment. + +![LTS_version](./images/LTS_version.png) + +## 2. Upgrade Execution + +### 2.1 Adding the openEuler 22.03 LTS SP1 Repositories (openEuler-22.03-LTS-SP1.repo) + +```bash +vi /etc/yum.repos.d/openEuler-22.03-LTS-SP1.repo +``` + +Add information about the following openEuler 22.03 LTS SP1 repositories and save and exit. + +```text +SP1_OS, SP1_everything, SP1_EPOL, SP1_debuginfo, SP1_source, SP1_update +``` + +![SP1_repo](./images/SP1_repo.png) + +### 2.2 Performing the Upgrade + +```bash +dnf update | tee update_log +``` + +Note: + +1. If an error is reported during the upgrade, run `dnf update --skip-broken -x conflict_pkg1 |tee update_log` to avoid the problem. If multiple packages conflict, use the `-x conflict_pkg1 -x conflict_pkg2 -x conflict_pkg3` options to skip the packages and analyze, validate, and update the conflicted packages after the upgrade. +2. Options: + `--allowerasing`: Allow erasing of installed packages to resolve dependencies. + `--skip-broken`: Resolve dependency problems by skipping packages. + `-x`: Used with `--skip-broken` to specify the packages to be skipped. + +### 2.3 Rebooting the OS + +```bash +reboot +``` + +## 3. Upgrade Verification + +View the versions of openEuler and the kernel in the current environment. + +![SP1_version](./images/SP1_version.png) + +## 4. Downgrade Execution + +### 4.1 Performing the Downgrade + +```bash +dnf downgrade | tee downgrade_log +``` + +### 4.2 Rebooting the OS + +```bash +reboot +``` + +## 5. Downgrade Verification + +View the versions of openEuler and the kernel in the current environment. + +![LTS_version](./images/LTS_version.png) diff --git a/docs/en/server/maintenance/aops/_toc.yaml b/docs/en/server/maintenance/aops/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..a2b7bdd5a6f74d89fcc17153b29cd7124629d7b6 --- /dev/null +++ b/docs/en/server/maintenance/aops/_toc.yaml @@ -0,0 +1,23 @@ +label: A-Ops User Guide +isManual: true +href: ./overview.md +description: >- + A-Ops enables quick fault identification and centralized configuration + management. +sections: + - label: A-Ops Deployment + href: ./deploying-aops.md + - label: A-Ops Intelligent Location Framework User Guide + href: ./aops-intelligent-location-framework-user-guide.md + - label: aops-agent Deployment + href: ./deploying-aops-agent.md + - label: Hot Patch DNF Plugin Command Usage + href: ./dnf-plugin-command-usage.md + - label: Configuration Tracing Service + href: ./gala-ragdoll-user-guide.md + - label: Architecture Awareness Service + href: ./architecture-awareness-service-manual.md + - label: Community Hot Patch Creation and Release Process + href: ./community-hotpatch-creation-and-release-process.md + - label: Automated O&M Service + href: ./automated-om-service-manual.md diff --git a/docs/en/server/maintenance/aops/aops-intelligent-location-framework-user-guide.md b/docs/en/server/maintenance/aops/aops-intelligent-location-framework-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..8bcc02fe01c053549e4a200b7cbb544cb02fae60 --- /dev/null +++ b/docs/en/server/maintenance/aops/aops-intelligent-location-framework-user-guide.md @@ -0,0 +1,182 @@ +# AOps 智能定位框架使用手册 + +参照[AOps部署指南](deploying-aops.md)部署AOps前后端服务后，即可使用AOps智能定位框架。 + +下文会从页面的维度进行AOps智能定位框架功能的介绍。 + +## 1. 工作台 + + 该页面为数据看板页面，用户登录后，仍在该页面。 + + ![4911661916984_.pic](./figures/工作台.jpg) + +支持操作： + +- 当前纳管的主机数量 +- 当前所有未确认的告警数量 + +- 每个主机组告警情况的统计 + +- 用户帐户操作 + + - 修改密码 + - 退出登录 +- 业务域和CVE信息暂不支持 + +## 2. 资产管理 + +资产管理分为对主机组进行管理以及对主机进行管理。每个主机在agent侧注册时需指定一个已存在的主机组进行注册，注册完毕后会在前端进行显示。 + +（1）主机组页面： + +![4761661915951_.pic](./figures/主机组.jpg) + +支持如下操作： + +- 主机组添加 +- 主机组删除 +- 查看当前所有主机组 +- 查看每个主机组下的主机信息 + +添加主机组时，需指定主机组的名称和描述。注意：请勿重复名称。 + +![添加主机组](./figures/添加主机组.jpg) + +（2）主机管理页面： + +![主机管理](./figures/主机管理.jpg) + +支持如下操作： + +- 查看主机列表（可根据主机组、管理节点进行筛选，可根据主机名称进行排序） +- 删除主机 +- 点击主机可跳转到主机详情界面 + +（3）主机详细信息界面： + +![主机详情](./figures/主机详情.jpg) + +详情页的上半部分展示了该主机的操作系统及CPU等的基础信息。 + +![插件管理](./figures/插件管理.jpg) + +详情页的下半部分，用户可以看到该主机当前运行的采集插件信息（目前agent只支持gala-gopher插件）。 + +支持如下操作： + +- 查看主机基础信息及插件信息 +- 插件的管理（gala-gopher） + - 插件资源查看 + - 插件的开启和管理 + - gala-gopher的采集探针的开启和关闭 +- 主机场景的识别 + +点击场景识别后，系统会生成该主机的场景，并推荐检测该场景所需开启的插件以及采集项，用户可以根据推荐结果进行插件/探针的调整。 + +注意：修改插件信息如关闭插件或开关探针后，需要点击保存才能生效。 + +![修改插件](./figures/修改插件.png) + +## 3. 智能定位 + +AOps项目的智能定位策略采用内置网络诊断应用作为模板，生成个性化工作流的策略进行检测和诊断。 + +“应用”作为工作流的模板,描述了检测中各步骤的串联情况，内置各步骤中使用的检测模型的推荐逻辑。用户在生成工作流时，可根据各主机的采集项、场景等信息，定制出工作流的详细信息。 + +（1）工作流列表页面： + +![工作流](./figures/工作流.jpg) + +支持操作： + +- 查看当前工作流列表，支持按照主机组、应用和状态进行筛选，并支持分页操作 +- 查看当前应用列表 + +（2）工作流详情页面： + +![工作流详情](./figures/工作流详情.jpg) + +支持操作： + +- 查看工作流所属主机组，主机数量、状态等基础信息 +- 查看单指标检测、多指标检测、集群故障诊断各步骤的详细算法模型信息 +- 修改检测各步骤应用的模型 +- 执行、暂停和删除工作流 + +修改某检测步骤的模型时，用户可根据模型名或标签搜索系统内置的模型库，选中模型后点击应用进行更改。 + +![修改模型](./figures/修改模型.png) + +（3）应用详情页面 + +![app详情](./figures/应用.png) + +支持操作： + +- 查看应用的整体流程 +- 基于应用创建工作流 + +创建工作流时，点击右上角的创建工作流按钮，并在右侧弹出的窗口中输入工作流的名称和描述，选择要检测的主机组。选中主机组后，下方会列出该主机组的所有主机，用户可选中部分主机后移到右侧的列表，最后点击创建，即可在工作流列表中看到新创建的工作流。 + +![app详情](./figures/app详情.jpg) + +![创建工作流](./figures/创建工作流.jpg) + +（4）告警 + +启动工作流后，会根据工作流的执行周期定时触发诊断，每次诊断若结果为异常，则会作为一条告警存入数据库，同时也会反应在前端告警页面中。 + +![告警](./figures/告警.jpg) + +支持操作： + +- 查看当前告警总数 +- 查看各主机组的告警数量 +- 查看告警列表 +- 告警确认 +- 查看告警详情 +- 下载诊断报告 + +告警确认后，将不在列表中显示 + +![告警确认](./figures/告警确认.jpg) + +点击异常详情后，可以根据主机维度查看告警详情，包括异常数据项的展示以及根因节点、根因异常的判断等。 + +![告警详情](./figures/告警详情.jpg) + +## 4. 配置溯源 + +AOps项目的配置溯源用于对目标主机配置文件内容的变动进行检测记录，对于文件配置错误类引发的故障起到很好的支撑作用。 + +### 创建配置域 + +![](./figures/chuangjianyewuyu.png) + +### 添加配置域纳管node + +![](./figures/tianjianode.png) + +### 添加配置域配置 + +![](./figures/xinzengpeizhi.png) + +### 查询预期配置 + +![](./figures/chakanyuqi.png) + +### 删除配置 + +![](./figures/shanchupeizhi.png) + +### 查询实际配置 + +![](./figures/chaxunshijipeizhi.png) + +### 配置校验 + +![](./figures/zhuangtaichaxun.png) + +### 配置同步 + +暂未提供 diff --git a/docs/en/server/maintenance/aops/architecture-awareness-service-manual.md b/docs/en/server/maintenance/aops/architecture-awareness-service-manual.md new file mode 100644 index 0000000000000000000000000000000000000000..e25037375171e6f5a0901c1425caf2e3ea94cc3a --- /dev/null +++ b/docs/en/server/maintenance/aops/architecture-awareness-service-manual.md @@ -0,0 +1,63 @@ +# Architecture Awareness Service Manual + +## Installation + +### Manual Installation + +- Installing using the repo source mounted by Yum. + + Configure the Yum sources **openEuler23.09** and **openEuler23.09:Epol** in the **/etc/yum.repos.d/openEuler.repo** file. + + ```ini + [everything] # openEuler 23.09 officially released repository + name=openEuler23.09 + baseurl=https://repo.openeuler.org/openEuler-23.09/everything/$basearch/ + enabled=1 + gpgcheck=1 + gpgkey=https://repo.openeuler.org/openEuler-23.09/everything/$basearch/RPM-GPG-KEY-openEuler + + [Epol] # openEuler 23.09:Epol officially released repository + name=Epol + baseurl=https://repo.openeuler.org/openEuler-23.09/EPOL/main/$basearch/ + enabled=1 + gpgcheck=1 + gpgkey=https://repo.openeuler.org/openEuler-23.09/OS/$basearch/RPM-GPG-KEY-openEuler + ``` + + Run the following commands to download and install gala-spider and its dependencies. + + ```shell + # A-Ops architecture awareness service, usually installed on the master node + yum install gala-spider + yum install python3-gala-spider + + # A-Ops architecture awareness probe, usually installed on the master node + yum install gala-gopher + ``` + +- Installing using the RPM packages. Download **gala-spider-vx.x.x-x.oe1.aarch64.rpm**, and then run the following commands to install the modules. (`x.x-x` indicates the version. Replace it with the actual version number.) + + ```shell + rpm -ivh gala-spider-vx.x.x-x.oe1.aarch64.rpm + + rpm -ivh gala-gopher-vx.x.x-x.oe1.aarch64.rpm + ``` + +### Installing Using the A-Ops Deployment Service + +#### Editing the Task List + +Modify the deployment task list and enable the steps for gala_spider: + +```yaml +--- +step_list: + ... + gala_gopher: + enable: false + continue: false + gala_spider: + enable: false + continue: false + ... +``` diff --git a/docs/en/server/maintenance/aops/automated-om-service-manual.md b/docs/en/server/maintenance/aops/automated-om-service-manual.md new file mode 100644 index 0000000000000000000000000000000000000000..7277fef3f64a28dbf6f9f035ccba8d1a16e837c9 --- /dev/null +++ b/docs/en/server/maintenance/aops/automated-om-service-manual.md @@ -0,0 +1,64 @@ +# 自动化运维服务使用手册 + +## 主要功能 + +本服务主要包括批量命令执行和批量脚本执行的功能，两者均支持定时任务。 + +### 命令相关 + +命令页面包含命令管理和命令执行两个页签 + +#### 命令管理 + +命令管理界面可以对命令进行增删查改功能： +![](./image/新建命令.png) +![](./image/命令管理界面.png) + +#### 命令执行 + +命令执行界面可以创建命令执行任务： +![](./image/新建命令任务.png) + +命令执行界面可以对创建的任务进行任务信息查看，任务执行，任务结果查看，删除任务等操作： +![](./image/命令执行.png) +任务结果查看： +![](./image/任务结果查看.png) + +### 脚本相关 + +脚本页面包含脚本管理、脚本执行、操作管理三个页签 + +#### 操作管理 + +为了屏蔽操作系统的架构，系统对脚本执行的影响，抽象出操作的概念。一个操作对应一组包括各类架构和系统的脚本，脚本执行时选择主机和操作后，根据主机的系统和架构选择对应的脚本进行执行。 + +操作管理界面可以对操作进行增删查改功能： + +![](./image/操作管理.png) + +#### 脚本管理 + +脚本管理界面可以对脚本进行上传，查询，删除，编辑功能： +![](./image/脚本管理.png) +创建脚本： +![](./image/创建脚本.png) + +#### 脚本执行 + +脚本执行界面可以创建脚本执行任务，创建脚本任务仅能通过选择操作来选择对应脚本： +![](./image/脚本执行.png) +创建脚本任务： +![](./image/创建脚本任务.png) + +### 定时任务 + +定时任务支持指定时间执行和周期执行功能 +单次执行： +![](./image/单次执行.png) +周期执行： +![](./image/周期执行.png) + +### 文件推送 + +文件推送功能支持将脚本推送至指定目录，此类任务不会执行脚本，且推送任务与定时任务互斥： +![](./image/文件推送.png) diff --git a/docs/en/server/maintenance/aops/community-hotpatch-creation-and-release-process.md b/docs/en/server/maintenance/aops/community-hotpatch-creation-and-release-process.md new file mode 100644 index 0000000000000000000000000000000000000000..47a94ff5914049369f052ebafc61dc36a882e311 --- /dev/null +++ b/docs/en/server/maintenance/aops/community-hotpatch-creation-and-release-process.md @@ -0,0 +1,264 @@ +# 社区热补丁制作发布流程 + +## 制作内核态/用户态热补丁 + +### 场景1. 在src-openEuler/openEuler仓下评论pr制作新版本热补丁 + +> [!NOTE]说明 +> 制作内核态热补丁需在**openEuler/kernel**仓评论pr。 +> 制作用户态热补丁需在src-openEuler仓评论pr，现在支持**src-openEuler/openssl，src-openEuler/glibc，src-openEuler/systemd**。 + +#### 1. 在已合入pr下评论制作热补丁 + +- 从src-openeuler仓【支持openssl, glibc, systemd】评论已合入pr制作新版本热补丁。 + +```shell +/makehotpatch [软件包版本号] [patch list] [cve/bug] [issue id] [os_branch] +``` + +命令说明：使用多个patch用','分隔，需注意patch的先后顺序。 + +![image-20230629114903593](./image/src-openEuler仓评论.png) + +- 从openeuler仓【支持kernel】评论已合入pr制作新版本热补丁。 + +```shell +/makehotpatch [软件包版本号] [cve/bug] [issue id] [os_branch] +``` + +![image-20230629142933917](./image/openEuler仓评论.png) + +评论后，门禁触发hotpatch_metadata仓创建热补丁issue以及同步该pr。 + +#### 2. hotpatch_metadata仓自动创建热补丁issue、同步该pr + +pr评论区提示启动热补丁制作流程。 + +![image-20230629143426498](./image/启动热补丁工程流程.png) + +随后，hotpatch_metadata仓自动创建热补丁issue，并在hotpatch_metadata仓同步该pr。 + +> [!NOTE]说明 +> 热补丁issue用于跟踪热补丁制作流程。 +> hotpatch_metadata仓用于触发制作热补丁。 + +![image-20230629144503840](./image/热补丁issue链接和pr链接.png) + +点击查看热补丁issue链接内容。 + +- 热补丁Issue类别为hotpatch。 + +![image-20230607161545732](./image/image-20230607161545732.png) + +点击查看在hotpatch_metadata仓自动创建的pr。 + +![hotpatch-fix-pr](./image/hotpatch-fix-pr.png) + +#### 3. 触发制作热补丁 + +打开hotpatch_metadata仓自动创建的pr，评论区可以查看热补丁制作信息。 + +![img](./image/45515A7F-0EC2-45AA-9B58-AB92DE9B0979.png) + +查看热补丁制作结果。 + +![img](./image/E574E637-0BF3-4F3B-BAE6-04ECBD09D151.png) + +如果热补丁制作失败，可以根据相关日志信息修改pr、评论 /retest直到热补丁可以被成功制作。 + +如果热补丁制作成功，可以通过Download link下载热补丁进行自验。 + +![image-20230608151244425](./image/hotpatch-pr-success.png) + +**若热补丁制作成功，可以对热补丁进行审阅**。 + +### 场景2、从hotpatch_metadata仓提pr修改热补丁 + +> [!NOTE]说明 +> 从hotpatch_metadata仓提pr只能修改还未正式发布的热补丁。 + +#### 1. 提pr + +用户需要手动创建热补丁issue。 + +（1）阅读readme，根据热补丁issue模版创建热补丁。 + +![image-20230612113428096](./image/image-20230612113428096.png) + +> [!NOTE]说明 +> 用户不允许修改热补丁元数据文件中已被正式发布的热补丁的相关内容。 + +pr内容： + +- patch文件。 +- 修改热补丁元数据hotmetadata.xml文件。 + +#### 2. 触发制作热补丁 + +**若热补丁制作成功，可以对热补丁进行审阅**。 + +### 场景3、从hotpatch_metadata仓提pr制作新版本热补丁 + +#### 1. 提pr + +在hotpatch_metadata仓提pr。 + +（1）阅读readme，根据热补丁issue模版创建热补丁。 + +![image-20230612113428096](./image/image-20230612113428096.png) + +pr内容： + +- patch文件。 +- 如果没有相应热补丁元数据hotmetadata.xml文件，则手动创建；否则修改热补丁元数据hotmetadata.xml文件。 + +#### 2. 触发制作热补丁 + +**若热补丁制作成功，可以对热补丁进行审阅**。 + +## 审阅热补丁 + +### 1. 审阅热补丁pr + +确认可发布，合入pr。 + +### 2. pr合入，回填热补丁issue + +在热补丁issue页面补充热补丁路径，包含src.rpm/arm架构/x86架构的rpm包，以及对应hotpatch.xml，用于展示热补丁信息。 + +> [!NOTE]说明 +> 如果一个架构失败，强行合入，也可只发布单架构的包。 + +![img](./image/EF5E0132-6E5C-4DD1-8CB5-73035278E233.png) + +- 热补丁Issue标签为hotpatch。 + +- 查看热补丁元数据内容。 + +热补丁元数据模版： + +> [!NOTE]说明 +> 热补丁元数据用于管理查看热补丁相关历史制作信息。 + +```xml + + + Managing Hot Patch Metadata + + + + src.rpm归档地址 + x86架构debuginfo二进制包归档地址 + arm架构debuginfo二进制包归档地址 + patch文件 + + https://gitee.com/wanghuan158/hot-patch_metadata/issues/I7AE5F + + + + +``` + +```xml + + + Managing Hot Patch Metadata + + + + download_link + download_link + download_link + 0001-PEM-read-bio-ret-failure.patch + + https://gitee.com/wanghuan158/hot-patch_metadata/issues/I7AE5F + + + download_link + download_link + download_link + 0001-PEM-read-bio-ret-failure.patch + + https://gitee.com/wanghuan158/hot-patch_metadata/issues/I7AE5P + + + + +``` + +> [!NOTE]说明 +> 注意：download_link均为repo仓正式的归档链接。 +> 热补丁当前只考虑演进，version 2基于version 1的src继续构建。 + +![image-20230607163358749](./image/image-20230607163358749.png) + +### 3. 关闭相应热补丁Issue + +## 发布热补丁 + +### 1、收集热补丁发布需求 + +在release-management仓库每周update需求收集的issue下方，手动评论start-update命令，此时会收集待发布的热补丁和待发布的修复cve的冷补丁。后台会在hotpatch_meta仓库根据hotpatch标签查找已关闭的热补丁issue。 + +### 2、生成安全公告热补丁信息 + +社区根据收集到的热补丁issue信息，在生成安全公告的同时生成hotpatch字段补丁，过滤已经发布的漏洞补丁。 + +- 在安全公告文件新增HotPatchTree字段，记录和公告相关漏洞的热补丁，每个补丁按架构和CVE字段区分（Type=ProductName 记录分支，Type=ProductArch 记录补丁具体的rpm包）。 + +![](./image/patch-file.PNG) + +### 3、Majun平台上传文件到openEuler官网，同步生成updateinfo.xml文件 + +社区将生成的安全公告上传到openEuler官网，同时基于所收集的热补丁信息生成updateinfo.xml文件。 + +![](./image/hotpatch-xml.PNG) + +updateinfo.xml文件样例： + +```xml + + + + openEuler-SA-2022-1 + An update for mariadb is now available for openEuler-22.03-LTS + Important + openEuler + + + + + + patch-redis-6.2.5-1-HP001.(CVE-2022-24048) + + + openEuler + + patch-redis-6.2.5-1-HP001-1-1.aarch64.rpm + + + patch-redis-6.2.5-1-HP001-1-1.x86_64.rpm + + + patch-redis-6.2.5-1-HP002-1-1.aarch64.rpm + + + patch-redis-6.2.5-1-HP002-1-1.x86_64.rpm + + + + + ... + +``` + +### 4、openEuler官网可以查看更新的热补丁信息，以cve编号划分 + +![image-20230612113626330](./image/image-20230612113626330.png) + +### 5、获取热补丁相关文件 + +社区将热补丁相关文件同步至openEuler的repo源下，可以在各个分支的hotpatch目录下获取相应文件。 + +> [!NOTE]说明 +> openEuler的repo地址： diff --git a/docs/en/server/maintenance/aops/deploying-aops-agent.md b/docs/en/server/maintenance/aops/deploying-aops-agent.md new file mode 100644 index 0000000000000000000000000000000000000000..72bb7a4aacf0f5e0db24214f54253cde4802bbe6 --- /dev/null +++ b/docs/en/server/maintenance/aops/deploying-aops-agent.md @@ -0,0 +1,656 @@ +# Deploying aops-agent + +## 1. Environment Requirements + +One host running on openEuler 20.03 or later + +## 2. Configuration Environment Deployment + +### 2.1 Disabling the Firewall + +```shell +systemctl stop firewalld +systemctl disable firewalld +systemctl status firewalld +``` + +### 2.2 Deploying aops-agent + +1. Run `yum install aops-agent` to install aops-agent based on the Yum source. + +2. Modify the configuration file. Change the value of the **ip** in the agent section to the IP address of the local host. + + ```shell + vim /etc/aops/agent.conf + ``` + + The following uses 192.168.1.47 as an example. + + ```ini + [agent] + ;IP address and port number bound when the aops-agent is started. + ip=192.168.1.47 + port=12000 + + [gopher] + ;Default path of the gala-gopher configuration file. If you need to change the path, ensure that the file path is correct. + config_path=/opt/gala-gopher/gala-gopher.conf + + ;aops-agent log collection configuration + [log] + ;Level of the logs to be collected, which can be set to DEBUG, INFO, WARNING, ERROR, or CRITICAL + log_level=INFO + ;Location for storing collected logs + log_dir=/var/log/aops + ;Maximum size of a log file + max_bytes=31457280 + ;Number of backup logs + backup_count=40 + ``` + +3. Run `systemctl start aops-agent` to start the service. + +### 2.3 Registering with aops-manager + +To identify users and prevent APIs from being invoked randomly, aops-agent uses tokens to authenticate users, reducing the pressure on the deployed hosts. + +For security purposes, the active registration mode is used to obtain the token. Before the registration, prepare the information to be registered on aops-agent and run the `register` command to register the information with aops-manager. No database is configured for aops-agent. After the registration is successful, the token is automatically saved to the specified file and the registration result is displayed on the GUI. In addition, save the local host information to the aops-manager database for subsequent management. + +1. Prepare the **register.json** file. + + Prepare the information required for registration on aops-agent and save the information in JSON format. The data structure is as follows: + + ```JSON + { + // Name of the login user + "web_username":"admin", + // User password + "web_password": "changeme", + // Host name + "host_name": "host1", + // Name of the group to which the host belongs + "host_group_name": "group1", + // IP address of the host where aops-manager is running + "manager_ip":"192.168.1.23", + // Whether to register as a management host + "management":false, + // External port for running aops-manager + "manager_port":"11111", + // Port for running aops-agent + "agent_port":"12000" + } + ``` + + Note: Ensure that aops-manager is running on the target host, for example, 192.168.1.23, and the registered host group exists. + +2. Run `aops_agent register -f register.json`. +3. The registration result is displayed on the GUI. If the registration is successful, the token character string is saved to a specified file. If the registration fails, locate the fault based on the message and log content (**/var/log/aops/aops.log**). + + The following is an example of the registration result: + + - Registration succeeded. + + ```shell + [root@localhost ~]# aops_agent register -f register.json + Agent Register Success + ``` + + - Registration failed. The following uses the aops-manager start failure as an example. + + ```shell + [root@localhost ~]# aops_agent register -f register.json + Agent Register Fail + [root@localhost ~]# + ``` + + - Log content + + ```shell + 2022-09-05 16:11:52,576 ERROR command_manage/register/331: HTTPConnectionPool(host='192.168.1.23', port=11111): Max retries exceeded with url: /manage/host/add (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) + [root@localhost ~]# + ``` + +## 3. Plug-in Support + +### 3.1 gala-gopher + +#### 3.1.1 Introduction + +gala-gopher is a low-load probe framework based on eBPF. It can be used to monitor the CPU, memory, and network status of hosts and collect data. You can configure the collection status of existing probes based on service requirements. + +#### 3.1.2 Deployment + +1. Run `yum install gala-gopher` to install gala-gopher based on the Yum source. +2. Enable probes based on service requirements. You can view information about probes in **/opt/gala-gopher/gala-gopher.conf**. +3. Run `systemctl start gala-gopher` to start the gala-gopher service. + +#### 3.1.3 Others + +For more information about gala-gopher, see . + +## 4. API Support + +### 4.1 List of External APIs + +| No. | API | Type | Description | +| --- | ------------------------------ | ---- | ------------------------------------------------------------------- | +| 1 | /v1/agent/plugin/start | POST | Starts a plug-in. | +| 2 | /v1/agent/plugin/stop | POST | Stops a plug-in. | +| 3 | /v1/agent/application/info | GET | Collects running applications in the target application collection. | +| 4 | /v1/agent/host/info | GET | Obtains host information. | +| 5 | /v1/agent/plugin/info | GET | Obtains the plug-in running information in aops-agent. | +| 6 | /v1/agent/file/collect | POST | Collects content of the configuration file. | +| 7 | /v1/agent/collect/items/change | POST | Changes the running status of plug-in collection items. | + +#### 4.1.1 /v1/agent/plugin/start + +- Description: Starts the plug-in that is installed but not running. Currently, only the gala-gopher plug-in is supported. + +- HTTP request mode: POST + +- Data submission mode: query + +- Request parameter + + | Parameter | Mandatory | Type | Description | + | ----------- | --------- | ---- | ------------ | + | plugin_name | True | str | Plug-in name | + +- Request parameter example + + | Parameter | Value | + | ----------- | ----------- | + | plugin_name | gala-gopher | + +- Response body parameters + + | Parameter | Type | Description | + | --------- | ---- | -------------------------------------------- | + | code | int | Return code | + | msg | str | Information corresponding to the status code | + +- Response example + + ```json + { + "code": 200, + "msg": "xxxx" + } + ``` + +#### 4.1.2 /v1/agent/plugin/stop + +- Description: Stops a running plug-in. Currently, only the gala-gopher plug-in is supported. + +- HTTP request mode: POST + +- Data submission mode: query + +- Request parameter + + | Parameter | Mandatory | Type | Description | + | ----------- | --------- | ---- | ------------ | + | plugin_name | True | str | Plug-in name | + +- Request parameter example + + | Parameter | Value | + | ----------- | ----------- | + | plugin_name | gala-gopher | + +- Response body parameters + + | Parameter | Type | Description | + | --------- | ---- | -------------------------------------------- | + | code | int | Return code | + | msg | str | Information corresponding to the status code | + +- Response example + + ```json + { + "code": 200, + "msg": "xxxx" + } + ``` + +#### 4.1.3 /v1/agent/application/info + +- Description: Collects running applications in the target application collection. Currently, the target application collection contains MySQL, Kubernetes, Hadoop, Nginx, Docker, and gala-gopher. + +- HTTP request mode: GET + +- Data submission mode: query + +- Request parameter + + | Parameter | Mandatory | Type | Description | + | --------- | --------- | ---- | ----------- | + | | | | | + +- Request parameter example + + | Parameter | Value | + | --------- | ----- | + | | | + +- Response body parameters + + | Parameter | Type | Description | + | --------- | ---- | -------------------------------------------- | + | code | int | Return code | + | msg | str | Information corresponding to the status code | + | resp | dict | Response body | + + - resp + + | Parameter | Type | Description | + | --------- | ---------- | -------------------------------- | + | running | List\[str] | List of the running applications | + +- Response example + + ```json + { + "code": 200, + "msg": "xxxx", + "resp": { + "running": [ + "mysql", + "docker" + ] + } + } + ``` + +#### 4.1.4 /v1/agent/host/info + +- Description: Obtains information about the host where aops-agent is installed, including the system version, BIOS version, kernel version, CPU information, and memory information. + +- HTTP request mode: POST + +- Data submission mode: application/json + +- Request parameter + + | Parameter | Mandatory | Type | Description | + | --------- | --------- | ---------- | ----------------------------------------------------------------------------------------------------- | + | info_type | True | List\[str] | List of the information to be collected. Currently, only the CPU, disk, memory, and OS are supported. | + +- Request parameter example + + ```json + ["os", "cpu","memory", "disk"] + ``` + +- Response body parameters + + | Parameter | Type | Description | + | --------- | ---- | -------------------------------------------- | + | code | int | Return code | + | msg | str | Information corresponding to the status code | + | resp | dict | Response body | + + - resp + + | Parameter | Type | Description | + | --------- | ----------- | ------------------ | + | cpu | dict | CPU information | + | memory | dict | Memory information | + | os | dict | OS information | + | disk | List\[dict] | Disk information | + + - cpu + + | Parameter | Type | Description | + | ------------ | ---- | ------------------------- | + | architecture | str | CPU architecture | + | core_count | int | Number of cores | + | l1d_cache | str | L1 data cache size | + | l1i_cache | str | L1 instruction cache size | + | l2_cache | str | L2 cache size | + | l3_cache | str | L3 cache size | + | model_name | str | Model name | + | vendor_id | str | Vendor ID | + + - memory + + | Parameter | Type | Description | + | --------- | ----------- | --------------------------- | + | size | str | Total memory | + | total | int | Number of DIMMs | + | info | List\[dict] | Information about all DIMMs | + + - info + + | Parameter | Type | Description | + | ------------ | ---- | ----------- | + | size | str | Memory size | + | type | str | Type | + | speed | str | Speed | + | manufacturer | str | Vendor | + + - os + + | Parameter | Type | Description | + | ------------ | ---- | -------------- | + | bios_version | str | BIOS version | + | os_version | str | OS version | + | kernel | str | Kernel version | + +- Response example + + ```json + { + "code": 200, + "msg": "operate success", + "resp": { + "cpu": { + "architecture": "aarch64", + "core_count": "128", + "l1d_cache": "8 MiB (128 instances)", + "l1i_cache": "8 MiB (128 instances)", + "l2_cache": "64 MiB (128 instances)", + "l3_cache": "128 MiB (4 instances)", + "model_name": "Kunpeng-920", + "vendor_id": "HiSilicon" + }, + "memory": { + "info": [ + { + "manufacturer": "Hynix", + "size": "16 GB", + "speed": "2933 MT/s", + "type": "DDR4" + }, + { + "manufacturer": "Hynix", + "size": "16 GB", + "speed": "2933 MT/s", + "type": "DDR4" + } + ], + "size": "32G", + "total": 2 + }, + "os": { + "bios_version": "1.82", + "kernel": "5.10.0-60.18.0.50", + "os_version": "openEuler 22.03 LTS" + }, + "disk": [ + { + "capacity": "xxGB", + "model": "xxxxxx" + } + ] + } + } + ``` + +#### 4.1.5 /v1/agent/plugin/info + +- Description: Obtains the plug-in running status of the host. Currently, only the gala-gopher plug-in is supported. + +- HTTP request mode: GET + +- Data submission mode: query + +- Request parameter + + | Parameter | Mandatory | Type | Description | + | --------- | --------- | ---- | ----------- | + | | | | | + +- Request parameter example + + | Parameter | Value | + | --------- | ----- | + | | | + +- Response body parameters + + | Parameter | Type | Description | + | --------- | ----------- | -------------------------------------------- | + | code | int | Return code | + | msg | str | Information corresponding to the status code | + | resp | List\[dict] | Response body | + + - resp + + | Parameter | Type | Description | + | ------------- | ----------- | -------------------------------------------- | + | plugin_name | str | Plug-in name | + | collect_items | list | Running status of plug-in collection items | + | is_installed | str | Information corresponding to the status code | + | resource | List\[dict] | Plug-in resource usage | + | status | str | Plug-in running status | + + - resource + + | Parameter | Type | Description | + | ------------- | ---- | -------------- | + | name | str | Resource name | + | current_value | str | Resource usage | + | limit_value | str | Resource limit | + +- Response example + + ```json + { + "code": 200, + "msg": "operate success", + "resp": [ + { + "collect_items": [ + { + "probe_name": "system_tcp", + "probe_status": "off", + "support_auto": false + }, + { + "probe_name": "haproxy", + "probe_status": "auto", + "support_auto": true + }, + { + "probe_name": "nginx", + "probe_status": "auto", + "support_auto": true + }, + ], + "is_installed": true, + "plugin_name": "gala-gopher", + "resource": [ + { + "current_value": "0.0%", + "limit_value": null, + "name": "cpu" + }, + { + "current_value": "13 MB", + "limit_value": null, + "name": "memory" + } + ], + "status": "active" + } + ] + } + ``` + +#### 4.1.6 /v1/agent/file/collect + +- Description: Collects information such as the content, permission, and owner of the target configuration file. Currently, only text files smaller than 1 MB, without execute permission, and supporting UTF8 encoding can be read. + +- HTTP request mode: POST + +- Data submission mode: application/json + +- Request parameter + + | Parameter | Mandatory | Type | Description | + | --------------- | --------- | ---------- | --------------------------------------------------- | + | configfile_path | True | List\[str] | List of the full paths of the files to be collected | + +- Request parameter example + + ```json + [ "/home/test.conf", "/home/test.ini", "/home/test.json"] + ``` + +- Response body parameters + + | Parameter | Type | Description | + | ------------- | ----------- | --------------------------------------- | + | infos | List\[dict] | File collection information | + | success_files | List\[str] | List of files successfully collected | + | fail_files | List\[str] | List of files that fail to be collected | + + - infos + + | Parameter | Type | Description | + | --------- | ---- | --------------- | + | path | str | File path | + | content | str | File content | + | file_attr | dict | File attributes | + + - file_attr + + | Parameter | Type | Description | + | --------- | ---- | ------------------------------- | + | mode | str | Permission of the file type | + | owner | str | File owner | + | group | str | Group to which the file belongs | + +- Response example + + ```json + { + "infos": [ + { + "content": "this is a test file", + "file_attr": { + "group": "root", + "mode": "0644", + "owner": "root" + }, + "path": "/home/test.txt" + } + ], + "success_files": [ + "/home/test.txt" + ], + "fail_files": [ + "/home/test.txt" + ] + } + ``` + +#### 4.1.7 /v1/agent/collect/items/change + +- Description: Changes the collection status of the plug-in collection items. Currently, only the status of the gala-gopher collection items can be changed. For the gala-gopher collection items, see **/opt/gala-gopher/gala-gopher.conf**. + +- HTTP request mode: POST + +- Data submission mode: application/json + +- Request parameter + + | Parameter | Mandatory | Type | Description | + | ----------- | --------- | ---- | ------------------------------------------------------------ | + | plugin_name | True | dict | Expected modification result of the plug-in collection items | + + - plugin_name + + | Parameter | Mandatory | Type | Description | + | ------------ | --------- | ------ | --------------------------------------------------- | + | collect_item | True | string | Expected modification result of the collection item | + +- Request parameter example + + ```json + { + "gala-gopher":{ + "redis":"auto", + "system_inode":"on", + "tcp":"on", + "haproxy":"auto" + } + } + ``` + +- Response body parameters + + | Parameter | Type | Description | + | --------- | ----------- | -------------------------------------------- | + | code | int | Return code | + | msg | str | Information corresponding to the status code | + | resp | List\[dict] | Response body | + + - resp + + | Parameter | Type | Description | + | ----------- | ---- | -------------------------------------------------------- | + | plugin_name | dict | Modification result of the corresponding collection item | + + - plugin_name + + | Parameter | Type | Description | + | --------- | ---------- | ----------------------------------------------- | + | success | List\[str] | Collection items that are successfully modified | + | failure | List\[str] | Collection items that fail to be modified | + +- Response example + + ```json + { + "code": 200, + "msg": "operate success", + "resp": { + "gala-gopher": { + "failure": [ + "redis" + ], + "success": [ + "system_inode", + "tcp", + "haproxy" + ] + } + } + } + ``` + + ### FAQs + +1. If an error is reported, view the **/var/log/aops/aops.log** file, rectify the fault based on the error message in the log file, and restart the service. + +2. You are advised to run aops-agent in Python 3.7 or later. Pay attention to the version of the Python dependency library when installing it. + +3. The value of **access_token** can be obtained from the **/etc/aops/agent.conf** file after the registration is complete. + +4. To limit the CPU and memory resources of a plug-in, add **MemoryHigh** and **CPUQuota** to the **Service** section in the service file corresponding to the plug-in. + + For example, set the memory limit of gala-gopher to 40 MB and the CPU limit to 20%. + + ```ini + [Unit] + Description=a-ops gala gopher service + After=network.target + + [Service] + Type=exec + ExecStart=/usr/bin/gala-gopher + Restart=on-failure + RestartSec=1 + RemainAfterExit=yes + ;Limit the maximum memory that can be used by processes in the unit. The limit can be exceeded. However, after the limit is exceeded, the process running speed is limited, and the system reclaims the excess memory as much as possible. + ;The option value can be an absolute memory size in bytes (K, M, G, or T suffix based on 1024) or a relative memory size in percentage. + MemoryHigh=40M + ;Set the CPU time limit for the processes of this unit. The value must be a percentage ending with %, indicating the maximum percentage of the total time that the unit can use a single CPU. + CPUQuota=20% + + [Install] + WantedBy=multi-user.target + ``` diff --git a/docs/en/server/maintenance/aops/deploying-aops.md b/docs/en/server/maintenance/aops/deploying-aops.md new file mode 100644 index 0000000000000000000000000000000000000000..60154be4bb161ef1765457058cc59c4944280ed6 --- /dev/null +++ b/docs/en/server/maintenance/aops/deploying-aops.md @@ -0,0 +1,457 @@ +# Deploying A-Ops + +## 1. Environment Requirements + +- Two hosts running on openEuler 24.09 + + These two hosts are used to deploy two modes of the check module: scheduler and executor. Other services, such as MySQL, Elasticsearch, and aops-manager, can be independently deployed on any host. To facilitate operations, deploy these services on host A. + +- More than 8 GB memory + +## 2. Configuring the Deployment Environment + +### Host A + +Deploy the following A-Ops services on host A: aops-tools, aops-manager, aops-check, aops-web, aops-agent, and gala-gopher. + +Deploy the following third-party services on host A: MySQL, Elasticsearch, ZooKeeper, Kafka, and Prometheus. + +The deployment procedure is as follows: + +#### 2.1 Disabling the Firewall + +Disable the firewall on the local host. + +```shell +systemctl stop firewalld +systemctl disable firewalld +systemctl status firewalld +``` + +#### 2.2 Deploying aops-tools + +Install aops-tools. + +```shell +yum install aops-tools +``` + +#### 2.3 Deploying Databases MySQL and Elasticsearch + +##### 2.3.1 Deploying MySQL + +Use the **aops-basedatabase** script installed during aops-tools installation to install MySQL. + +```shell +cd /opt/aops/aops_tools +./aops-basedatabase mysql +``` + +Modify the MySQL configuration file. + +```shell +vim /etc/my.cnf +``` + +Add **bind-address** and set it to the IP address of the local host. + +![1662346986112](./figures/modify-mysql-config-file.png) + +Restart the MySQL service. + +```shell +systemctl restart mysqld +``` + +Connect to the database and set the permission. + +```shell +mysql +show databases; +use mysql; +select user,host from user;// If the value of user is root and the value of host is localhost, the MySQL database can be connected only by the local host but cannot be connected from the external network and by the local software client. +update user set host = '%' where user='root'; +flush privileges;// Refresh the permission. +exit +``` + +##### 2.3.2 Deploying Elasticsearch + +Use the **aops-basedatabase** script installed during aops-tools installation to install Elasticsearch. + +```shell +cd /opt/aops/aops_tools +./aops-basedatabase elasticsearch +``` + +Modify the configuration file. + +Modify the Elasticsearch configuration file. + +```shell +vim /etc/elasticsearch/elasticsearch.yml +``` + +![1662370718890](./figures/elasticsearch-config-2.png) + +![1662370575036](./figures/elasticsearch-config-1.png) + +![1662370776219](./figures/elasticsearch3.png) + +Restart the Elasticsearch service. + +```shell +systemctl restart elasticsearch +``` + +#### 2.4 Deploying aops-manager + +Install aops-manager. + +```shell +yum install aops-manager +``` + +Modify the configuration file. + +```shell +vim /etc/aops/manager.ini +``` + +Change the IP address of each service in the configuration file to the actual IP address. Because all services are deployed on host A, you need to set their IP addresses to the actual IP address of host A. + +```ini +[manager] +ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. +port=11111 +host_vault_dir=/opt/aops +host_vars=/opt/aops/host_vars + +[uwsgi] +wsgi-file=manage.py +daemonize=/var/log/aops/uwsgi/manager.log +http-timeout=600 +harakiri=600 + +[elasticsearch] +ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. +port=9200 +max_es_query_num=10000000 + +[mysql] +ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. +port=3306 +database_name=aops +engine_format=mysql+pymysql://@%s:%s/%s +pool_size=10000 +pool_recycle=7200 + +[aops_check] +ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. +port=11112 +``` + +Start the aops-manager service. + +```shell +systemctl start aops-manager +``` + +#### 2.5 Deploying aops-web + +Install aops-web. + +```shell +yum install aops-web +``` + +Modify the configuration file. Because all services are deployed on host A, set the IP address of each service accessed by aops-web to the actual IP address of host A. + +```shell +vim /etc/nginx/aops-nginx.conf +``` + +The following figure shows the configuration of some services. + +![1662378186528](./figures/web-config.png) + +Enable the aops-web service. + +```shell +systemctl start aops-web +``` + +#### 2.6 Deploying Kafka + +##### 2.6.1 Deploying ZooKeeper + +Install ZooKeeper. + +```shell +yum install zookeeper +``` + +Start the ZooKeeper service. + +```shell +systemctl start zookeeper +``` + +##### 2.6.2 Deploying Kafka + +Install Kafka. + +```shell +yum install kafka +``` + +Modify the configuration file. + +```shell +vim /opt/kafka/config/server.properties +``` + +Change the value of **listeners** to the IP address of the local host. + +![1662381371927](./figures/kafka-config.png) + +Start the Kafka service. + +```shell +cd /opt/kafka/bin +nohup ./kafka-server-start.sh ../config/server.properties & +tail -f ./nohup.out # Check all the outputs of nohup. If the IP address of host A and the Kafka startup success INFO are displayed, Kafka is started successfully. +``` + +#### 2.7 Deploying aops-check + +Install aops-check. + +```shell +yum install aops-check +``` + +Modify the configuration file. + +```shell +vim /etc/aops/check.ini +``` + +Change the IP address of each service in the configuration file to the actual IP address. Because all services are deployed on host A, you need to set their IP addresses to the actual IP address of host A. + +```ini +[check] +ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. +port=11112 +mode=configurable // Configurable mode, which means aops-check is used as the scheduler in common diagnosis mode. +timing_check=on + +[default_mode] +period=30 +step=30 + +[elasticsearch] +ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. +port=9200 + +[mysql] +ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. +port=3306 +database_name=aops +engine_format=mysql+pymysql://@%s:%s/%s +pool_size=10000 +pool_recycle=7200 + +[prometheus] +ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. +port=9090 +query_range_step=15s + +[agent] +default_instance_port=8888 + +[manager] +ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. +port=11111 + +[consumer] +kafka_server_list=192.168.1.1:9092 // Change the service IP address to the actual IP address of host A. +enable_auto_commit=False +auto_offset_reset=earliest +timeout_ms=5 +max_records=3 +task_name=CHECK_TASK +task_group_id=CHECK_TASK_GROUP_ID +result_name=CHECK_RESULT +[producer] +kafka_server_list = 192.168.1.1:9092 // Change the service IP address to the actual IP address of host A. +api_version = 0.11.5 +acks = 1 +retries = 3 +retry_backoff_ms = 100 +task_name=CHECK_TASK +task_group_id=CHECK_TASK_GROUP_ID +``` + +Start the aops-check service in configurable mode. + +```shell +systemctl start aops-check +``` + +#### 2.8 Deploying the Client Services + +aops-agent and gala-gopher must be deployed on the client. For details, see [aops-agent Deployment](deploying-aops-agent.md). + +Note: Before registering a host, you need to add a host group to ensure that the host group to which the host belongs exists. In this example, only host A is deployed and managed. + +#### 2.9 Deploying Prometheus + +Install Prometheus. + +```shell +yum install prometheus2 +``` + +Modify the configuration file. + +```shell +vim /etc/prometheus/prometheus.yml +``` + +Add the gala-gopher addresses of all clients to the monitoring host of Prometheus. + +![1662377261742](./figures/prometheus-config.png) + +Start the Prometheus service: + +```shell +systemctl start prometheus +``` + +#### 2.10 Deploying gala-ragdoll + +The configuration source tracing function of A-Ops depends on gala-ragdoll. Git is used to monitor configuration file changes. + +Install gala-ragdoll. + +```shell +yum install gala-ragdoll # A-Ops configuration source tracing +``` + +Modify the configuration file. + +```shell +vim /etc/ragdoll/gala-ragdoll.conf +``` + +Change the IP address in **collect_address** of the **collect** section to the IP address of host A, and change the values of **collect_api** and **collect_port** to the actual API and port number. + +```ini +[git] +git_dir = "/home/confTraceTest" +user_name = "user_name" +user_email = "user_email" + +[collect] +collect_address = "http://192.168.1.1" // Change the IP address to the actual IP address of host A. +collect_api = "/manage/config/collect" // Change the API to the actual API for collecting configuration files. +collect_port = 11111 // Change the port number to the actual port number of the service. + +[sync] +sync_address = "http://0.0.0.0" +sync_api = "/demo/syncConf" +sync_port = 11114 + + +[ragdoll] +port = 11114 +``` + +Start the gala-ragdoll service. + +```shell +systemctl start gala-ragdoll +``` + +### Host B + +Only aops-check needs to be deployed on host B as the executor. + +#### 2.11 Deploying aops-check + +Install aops-check. + +```shell +yum install aops-check +``` + +Modify the configuration file. + +```shell +vim /etc/aops/check.ini +``` + +Change the IP address of each service in the configuration file to the actual IP address. Change the IP address of the aops-check service deployed on host B to the IP address of host B. Because other services are deployed on host A, change the IP addresses of those services to the IP address of host A. + +```ini +[check] +ip=192.168.1.2 // Change the IP address to the actual IP address of host B. +port=11112 +mode=executor // Executor mode, which means aops-check is used as the executor in normal diagnosis mode. +timing_check=on + +[default_mode] +period=30 +step=30 + +[elasticsearch] +ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. +port=9200 + +[mysql] +ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. +port=3306 +database_name=aops +engine_format=mysql+pymysql://@%s:%s/%s +pool_size=10000 +pool_recycle=7200 + +[prometheus] +ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. +port=9090 +query_range_step=15s + +[agent] +default_instance_port=8888 + +[manager] +ip=192.168.1.1 // Change the service IP address to the actual IP address of host A. +port=11111 + +[consumer] +kafka_server_list=192.168.1.1:9092 // Change the service IP address to the actual IP address of host A. +enable_auto_commit=False +auto_offset_reset=earliest +timeout_ms=5 +max_records=3 +task_name=CHECK_TASK +task_group_id=CHECK_TASK_GROUP_ID +result_name=CHECK_RESULT +[producer] +kafka_server_list = 192.168.1.1:9092 // Change the service IP address to the actual IP address of host A. +api_version = 0.11.5 +acks = 1 +retries = 3 +retry_backoff_ms = 100 +task_name=CHECK_TASK +task_group_id=CHECK_TASK_GROUP_ID +``` + +Start the aops-check service in executor mode. + +```shell +systemctl start aops-check +``` + +The service deployment on the two hosts is complete. diff --git a/docs/en/server/maintenance/aops/dnf-plugin-command-usage.md b/docs/en/server/maintenance/aops/dnf-plugin-command-usage.md new file mode 100644 index 0000000000000000000000000000000000000000..04f32add022a7dbd30411defe37c369b52c4e174 --- /dev/null +++ b/docs/en/server/maintenance/aops/dnf-plugin-command-usage.md @@ -0,0 +1,548 @@ +# DNF Plugin Command Usage + +Install the DNF plugin: + +```shell +dnf install dnf-hotpatch-plugin +``` + +After installing the plugin, you can run DNF commands to use plugin functions related to hot patches, such as hot patch scanning (`dnf hot-updateinfo`), setting and querying (`dnf hotpatch`), and applying (`dnf hotupgrade`). This document describes the usage of the commands. + +## Hot Patch Scanning + +`hot-updateinfo` can scan hot patches and query hot patches for specified CVEs. + +```shell +$ dnf hot-updateinfo list cves [--cve [cve_id]] +General DNF options: + -h, --help, --help-cmd + show command help + --cve CVES, --cves CVES + Include packages needed to fix the given CVE, in updates +``` + +- `--list` + +1. Query the CVEs on the host that can be fixed and their related cold and hot patches. + + ```shell + $ dnf hot-updateinfo list cves + # cve-id level cold-patch hot-patch + Last metadata expiration check: 0:54:46 ago on Thu Mar 16 09:40:27 2023. + CVE-2022-3080 Important/Sec. bind-libs-9.16.23-10.oe2203.aarch64 patch-bind-libs-9.16.23-09-name-1-111.aarch64 + CVE-2021-25220 Moderate/Sec. bind-9.16.23-10.oe2203.aarch64 - + CVE-2022-1886 Critical/Sec. vim-common-8.2-39.oe2203.aarch64 patch-vim-common-8.2-38-name-1-233.aarch64 + CVE-2022-1725 Low/Sec. vim-minimal-8.2-58.oe2203.aarch64 patch-vim-minimal-8.2-57-name-2-11.aarch64 + ``` + +2. Query cold and hot patches for specified CVEs. + + ```shell + $ dnf hot-updateinfo list cves --cve CVE-2022-3080 + # cve-id level cold-patch hot-patch + Last metadata expiration check: 0:54:46 ago on Thu Mar 16 09:40:27 2023. + CVE-2022-3080 Important/Sec. bind-libs-9.16.23-10.oe2203.aarch64 patch-bind-libs-9.16.23-09-name-1-111.aarch64 + ``` + +3. An empty list will be displayed if the CVE does not exist. + + ```shell + $ dnf hot-updateinfo list cves --cve CVE-2022-3089 + # cve-id level cold-patch hot-patch + Last metadata expiration check: 0:54:46 ago on Thu Mar 16 09:40:27 2023. + ``` + +## Hot Patch Statuses + +- A hot patch can be in the following statuses: + + - NOT-APPLIED: The hot patch is not installed. + + - DEACTIVED: The hot patch is installed. + + - ACTIVED: The hot patch is activated. + + - ACCEPT: The hot patch has been accepted and will be applied after a reboot. + + ![Hot patch statuses](./figures/hot_patch_statuses.png) + +## Querying and Changing Hot Patch Statuses + +`hotpatch` can be used to query and convert hot patch statuses. + +```shell +$ dnf hotpatch +General DNF options: + -h, --help, --help-cmd + show command help + --cve CVES, --cves CVES + Include packages needed to fix the given CVE, in updates + +Hotpatch command-specific options: + --list [{cve, cves}] show list of hotpatch + --apply APPLY_NAME apply hotpatch + --remove REMOVE_NAME remove hotpatch + --active ACTIVE_NAME active hotpatch + --deactive DEACTIVE_NAME + deactive hotpatch + --accept ACCEPT_NAME accept hotpatch +``` + +1. `dnf hotpatch --list` lists available hot patches in the system. + + ```shell + $ dnf hotpatch --list + Last metadata expiration check: 0:54:46 ago on Thu Mar 16 09:40:27 2023. + base-pkg/hotpatch status + redis-6.2.5-1/HP001 NOT-APPLIED + redis-6.2.5-1/HP001 NOT-APPLIED + redis-6.2.5-1/HP002 ACTIVED + redis-6.2.5-1/HP002 ACTIVED + ``` + +2. `dnf hotpatch --list cves` queries hot patches related to CVEs. + + ```shell + $ dnf hotpatch --list cves + Last metadata expiration check: 0:54:46 ago on Thu Mar 16 09:40:27 2023. + CVE-id base-pkg/hotpatch status + CVE-2023-1111 redis-6.2.5-1/HP001 NOT-APPLIED + CVE-2023-1112 redis-6.2.5-1/HP001 NOT-APPLIED + CVE-2023-2221 redis-6.2.5-1/HP002 ACTIVED + CVE-2023-2222 redis-6.2.5-1/HP002 ACTIVED + ``` + +3. `dnf hotpatch --list cves --cve ` queries hot patches for specified CVEs. + + ```shell + $ dnf hotpatch --list cves --cve CVE-2023-1111 + Last metadata expiration check: 0:54:46 ago on Thu Mar 16 09:40:27 2023. + CVE-id base-pkg/hotpatch status + CVE-2023-1111 redis-6.2.5-1/HP001 NOT-APPLIED + ``` + +4. An empty list will be displayed if the specified CVE does not exist when running `dnf hotpatch --list cves --cve `. + + ```shell + $ dnf hotpatch --list cves --cve CVE-2023-1 + Last metadata expiration check: 0:54:46 ago on Thu Mar 16 09:40:27 2023. + ``` + +5. `dnf hotpatch --apply ` applies a hot patch. You can run `syscare list` to query the hot patch status after applying the hot patch. For details about hot patch statuses, see the previous section. + + ```shell + $ dnf hotpatch --apply redis-6.2.5-1/HP2 + Last metadata expiration check: 2:38:51 ago on Thu May 25 13:49:28 2023. + Gonna apply this hot patch: redis-6.2.5-1/HP2 + apply hot patch 'redis-6.2.5-1/HP2' succeed + $ syscare list + Uuid Name Status + 25209ddc-b1e4-48e0-b715-e759ec8db401 redis-6.2.5-1/HP2 ACTIVED + ``` + +6. `dnf hotpatch --deactive ` deactivates a hot patch. You can run `syscare list` to query the hot patch status after deactivating the hot patch. For details about hot patch statuses, see the previous section. + + ```shell + $ dnf hotpatch --deactive redis-6.2.5-1/HP2 + Last metadata expiration check: 2:39:10 ago on Thu May 25 13:49:28 2023. + Gonna deactive this hot patch: redis-6.2.5-1/HP2 + deactive hot patch 'redis-6.2.5-1/HP2' succeed + $ syscare list + Uuid Name Status + 25209ddc-b1e4-48e0-b715-e759ec8db401 redis-6.2.5-1/HP2 DEACTIVED + ``` + +7. `dnf hotpatch --remove ` removes a hot patch. You can run `syscare list` to query the hot patch status after removing the hot patch. For details about hot patch statuses, see the previous section. + + ```shell + $ dnf hotpatch --remove redis-6.2.5-1/HP2 + Last metadata expiration check: 2:53:25 ago on Thu May 25 13:49:28 2023. + Gonna remove this hot patch: redis-6.2.5-1/HP2 + remove hot patch 'redis-6.2.5-1/HP2' succeed + $ syscare list + Uuid Name Status + 25209ddc-b1e4-48e0-b715-e759ec8db401 redis-6.2.5-1/HP2 NOT-APPLIED + ``` + +8. `dnf hotpatch --active ` activating a hot patch.You can run `syscare list` to query the hot patch status after activating the hot patch. For details about hot patch statuses, see the previous section. + + ```shell + $ dnf hotpatch --active redis-6.2.5-1/HP2 + Last metadata expiration check: 2:53:37 ago on Thu May 25 13:49:28 2023. + Gonna active this hot patch: redis-6.2.5-1/HP2 + active hot patch 'redis-6.2.5-1/HP2' failed, remain original status. + $ syscare list + Uuid Name Status + 25209ddc-b1e4-48e0-b715-e759ec8db401 redis-6.2.5-1/HP2 ACTIVED + ``` + +9. `dnf hotpatch --accept ` accepts a hot patch. You can run `syscare list` to query the hot patch status after accepting the hot patch. For details about hot patch statuses, see the previous section. + + ```shell + $ dnf hotpatch --accept redis-6.2.5-1/HP2 + Last metadata expiration check: 2:53:25 ago on Thu May 25 13:49:28 2023. + Gonna accept this hot patch: redis-6.2.5-1/HP2 + remove hot patch 'redis-6.2.5-1/HP2' succeed + $ syscare list + Uuid Name Status + 25209ddc-b1e4-48e0-b715-e759ec8db401 redis-6.2.5-1/HP2 ACCEPTED + ``` + +## Applying Hot Patches + +The `hotupgrade` command is used to apply hot patches to fix specified or all CVEs. + +```shell +$ dnf hotupgrade [--cve [cve_id]] [SPEC ...] + +General DNF options: + -h, --help, --help-cmd + show command help + --cve CVES, --cves CVES + Include packages needed to fix the given CVE, in updates + +command-specific options: + SPEC Hotpatch specification +``` + +- Case 1: If hot patches have been installed, `dnf hotupgrade` applies all available hot patches. + +```shell +$ dnf hotupgrade +Last metadata expiration check: 4:04:34 ago on Fri Jun 02 06:33:41 2023. +Gonna apply these hot patches:['patch-redis-6.2.5-1-HP001-1-1.x86_64', 'patch-redis-6.2.5-1-HP002-1-1.x86_64'] +The target package 'redis-6.2.5-1' has a hotpatch 'HP001' applied +Gonna remove these hot patches: ['redis-6.2.5-1/HP001'] +Remove hot patch redis-6.2.5-1/HP001. +Package patch-redis-6.2.5-1-HP001-1-1.x86_64 is already installed. +Package patch-redis-6.2.5-1-HP002-1-1.x86_64 is already installed. +Dependencies resolved. +Nothing to do. +Complete! +Applying hot patch +Apply hot patch succeed: redis-6.2.5-1/HP001. +Apply hot patch failed: redis-6.2.5-1/HP002. +``` + +- Case 2: If hot patches have not been installed, `dnf hotupgrade` installs and applies all available hot patches. + +```shell +$ dnf hotupgrade +Last metadata expiration check: 4:13:16 ago on Fri Jun 02 06:33:41 2023. +Gonna apply these hot patches:['patch-redis-6.2.5-1-HP002-1-1.x86_64', 'patch-redis-6.2.5-1-HP001-1-1.x86_64'] +Package patch-redis-6.2.5-1-HP002-1-1.x86_64 is already installed. +Dependencies resolved. +xxxx(Install messages) +Is this ok [y/N]: y +Downloading Packages: +xxxx(Install process) +Complete! + +Applying hot patch +Apply hot patch succeed: redis-6.2.5-1/HP001. +``` + +- Case 3: `dnf hotupgrade ` upgrades specified hot patches. + +```shell +$ dnf hotupgrade patch-redis-6.2.5-1-HP001-1-1.x86_64 +Last metadata expiration check: 0:07:49 ago on Thu Jun 08 12:03:46 2023. +Package patch-redis-6.2.5-1-HP001-1-1.x86_64 is already installed. +Dependencies resolved. +Nothing to do. +Complete! +Applying hot patch +Apply hot patch succeed: redis-6.2.5-1/HP001. +``` + +- `--cve` + + - Case 1: `dnf hotupgrade --cve ` is used to install and apply a hot patch for a specified CVE. + + ```shell + $ dnf hotupgrade --cve CVE-2021-11 + Last metadata expiration check: xxx + Dependencies resolved. + xxxx(Install messages) + Is this ok [y/N]: y + Downloading Packages: + xxxx(Install process) + Complete! + Applying hot patch + Apply hot patch succeed: redis-6.2.5-1/HP001 + ``` + + - Case 2: `dnf hotupgrade --cve ` is used for a CVE that does not exist. + + ```shell + $ dnf hotupgrade --cve CVE-2021-11 + Last metadata expiration check: xxx + The cve doesnt exist: CVE-2021-11 + Error: No hot patches marked for install. + ``` + + - Case 3: `dnf hotupgrade --cve ` is used to install and apply a hot patch of a higher version for a CVE that has a hot patch of a lower version. The hot patch of the lower version is uninstalled. + + ```shell + $ dnf hotupgrade --cve CVE-2021-22 + Last metadata expiration check: xxx + The target package 'redis-6.2.5-1' has a hotpatch 'HP001' applied + Gonna remove these hot patches: ['redis-6.2.5-1/HP001'] + Is this ok [y/N]: y + Remove hot patch redis-6.2.5-1/HP001 + xxxx (install messages and process install) + Apply hot patch + apply hot patch succeed: redis-6.2.5-1/HP002 + ``` + + - Case 4: `dnf hotupgrade --cve ` is used to install and apply a hot patch for a CVE that has the latest hot patch. + + ```shell + $ dnf hotupgrade --cve CVE-2021-22 + Package patch -redis-6.2.5-1-HP002-1-1.x86_64 is already installed. + Dependencies resolved. + Nothing to do. + Complete! + Applying hot patch + Apply hot patch succeed: redis-6.2.5-1/HP002 + ``` + +- `SPEC` + + ```shell + dnf hotupgrade bind-libs-hotpatch + ``` + +The output of this sub-command is the same as that of `--cve`. + +## Usage Example + +Assume that the repositories of hot and cold patches on this host have been enabled. + +Scan CVEs that can be fixed on the host. + +```shell +$ dnf hot-updateinfo list cves +Last metadata expiration check: 0:00:38 ago on Sat Mar 25 11:53:46 2023. +CVE-2023-22995 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64 - +CVE-2023-26545 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64 - +CVE-2022-40897 Important/Sec. python3-setuptools-59.4.0-5.oe2203sp1.noarch - +CVE-2021-1 Important/Sec. redis-6.2.5-2.x86_64 patch-redis-6.2.5-1-HP001-1-1.x86_64 +CVE-2021-11 Important/Sec. redis-6.2.5-2.x86_64 patch-redis-6.2.5-1-HP001-1-1.x86_64 +CVE-2021-2 Important/Sec. redis-6.2.5-3.x86_64 patch-redis-6.2.5-1-HP002-1-1.x86_64 +CVE-2021-22 Important/Sec. redis-6.2.5-3.x86_64 patch-redis-6.2.5-1-HP002-1-1.x86_64 +CVE-2021-33 Important/Sec. redis-6.2.5-4.x86_64 - +CVE-2021-3 Important/Sec. redis-6.2.5-4.x86_64 - +CVE-2022-38023 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64 - +CVE-2022-37966 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64 - + +``` + +CVE-2021-1, CVE-2021-11, CVE-2021-2, and CVE-2021-22 can be fixed by hot patches. + +Start the Redis service based on the **redis.conf** configuration file. + +```shell +$ sudo redis-server ./redis.conf & +[1] 285075 +285076:C 25 Mar 2023 12:09:51.503 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo +285076:C 25 Mar 2023 12:09:51.503 # Redis version=255.255.255, bits=64, commit=00000000, modified=0, pid=285076, just started +285076:C 25 Mar 2023 12:09:51.503 # Configuration loaded +285076:M 25 Mar 2023 12:09:51.504 * Increased maximum number of open files to 10032 (it was originally set to 1024). +285076:M 25 Mar 2023 12:09:51.504 * monotonic clock: POSIX clock_gettime + _._ + _.-``__ ''-._ + _.-`` `. `_. ''-._ Redis 255.255.255 (00000000/0) 64 bit + .-`` .-```. ```\/ _.,_ ''-._ + ( ' , .-` | `, ) Running in standalone mode + |`-._`-...-` __...-.``-._|'` _.-'| Port: 6380 + | `-._ `._ / _.-' | PID: 285076 + `-._ `-._ `-./ _.-' _.-' + |`-._`-._ `-.__.-' _.-'_.-'| + | `-._`-._ _.-'_.-' | https://redis.io + `-._ `-._`-.__.-'_.-' _.-' + |`-._`-._ `-.__.-' _.-'_.-'| + | `-._`-._ _.-'_.-' | + `-._ `-._`-.__.-'_.-' _.-' + `-._ `-.__.-' _.-' + `-._ _.-' + `-.__.-' + +285076:M 25 Mar 2023 12:09:51.505 # Server initialized +285076:M 25 Mar 2023 12:09:51.505 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. +285076:M 25 Mar 2023 12:09:51.506 * Ready to accept connections + +``` + +Test the function before applying the hot patch. + +```shell +$ telnet 127.0.0.1 6380 +Trying 127.0.0.1... +Connected to 127.0.0.1. +Escape character is '^]'. + +*100 + +-ERR Protocol error: expected '$', got ' ' +Connection closed by foreign host. +``` + +Specify CVE-2021-1 and ensure that the related hot patch is associated and applied. + +```shell +$ dnf hotupgrade --cve CVE-2021-1 +Last metadata expiration check: 0:05:19 ago on Sat Mar 25 11:53:46 2023. +Package patch-redis-6.2.5-1-HP001-1-1.x86_64 is already installed. +Dependencies resolved. +Nothing to do. +Complete! +Applying hot patch +Apply hot patch succeed: redis-6.2.5-1/HP001. +``` + +Run `syscare` to check whether the hot patch has been applied (the status is **ACTIVED**). + +```shell +$ syscare list +Uuid Name Status +cf47649c-b370-4f5a-a914-d2ca4d8f1f3a redis-6.2.5-1/HP001 ACTIVED +``` + +Check whether the CVE has been fixed. Because the **patch-redis-6.2.5-1-HP001-1-1.x86_64** hot patch also fixes CVE-2021-11, CVE-2021-1 and CVE-2021-11 no longer exists. + +```shell +$ dnf hot-updateinfo list cves +Last metadata expiration check: 0:08:48 ago on Sat Mar 25 11:53:46 2023. +CVE-2023-22995 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64 - +CVE-2023-1076 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64 - +CVE-2023-26607 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64 - +CVE-2022-40897 Important/Sec. python3-setuptools-59.4.0-5.oe2203sp1.noarch - +CVE-2021-22 Important/Sec. redis-6.2.5-3.x86_64 patch-redis-6.2.5-1-HP002-1-1.x86_64 +CVE-2021-2 Important/Sec. redis-6.2.5-3.x86_64 patch-redis-6.2.5-1-HP002-1-1.x86_64 +CVE-2021-33 Important/Sec. redis-6.2.5-4.x86_64 - +CVE-2021-3 Important/Sec. redis-6.2.5-4.x86_64 - +CVE-2022-38023 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64 - +CVE-2022-37966 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64 - +``` + +Test the function after applying the hot patch. + +```shell +$ telnet 127.0.0.1 6380 +Trying 127.0.0.1... +Connected to 127.0.0.1. +Escape character is '^]'. + +*100 + +-ERR Protocol error: unauthenticated multibulk length +Connection closed by foreign host. +``` + +Run `syscare` and specify the patch name to manually remove the hot patch. + +```shell +$ syscare remove redis-6.2.5-1/HP001 +$ syscare list +Uuid Name Status +cf47649c-b370-4f5a-a914-d2ca4d8f1f3a redis-6.2.5-1/HP001 NOT-APPLIED +``` + +Scan the CVEs to be fixed on the host. CVE-2021-1 and CVE-2021-11 are displayed. + +```shell +$ dnf hot-updateinfo list cves +Last metadata expiration check: 0:00:38 ago on Sat Mar 25 11:53:46 2023. +CVE-2023-22995 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64 - +CVE-2023-26545 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64 - +CVE-2022-40897 Important/Sec. python3-setuptools-59.4.0-5.oe2203sp1.noarch - +CVE-2021-1 Important/Sec. redis-6.2.5-2.x86_64 patch-redis-6.2.5-1-HP001-1-1.x86_64 +CVE-2021-11 Important/Sec. redis-6.2.5-2.x86_64 patch-redis-6.2.5-1-HP001-1-1.x86_64 +CVE-2021-2 Important/Sec. redis-6.2.5-3.x86_64 patch-redis-6.2.5-1-HP002-1-1.x86_64 +CVE-2021-22 Important/Sec. redis-6.2.5-3.x86_64 patch-redis-6.2.5-1-HP002-1-1.x86_64 +CVE-2021-33 Important/Sec. redis-6.2.5-4.x86_64 - +CVE-2021-3 Important/Sec. redis-6.2.5-4.x86_64 - +CVE-2022-38023 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64 - +CVE-2022-37966 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64 - +``` + +- Case 1 + +Apply hot patch **patch-redis-6.2.5-1-HP002-1-1.x86_64**. + +```shell +$ dnf hotupgrade patch-redis-6.2.5-1-HP002-1-1.x86_64 +Last metadata expiration check: 0:05:19 ago on Sat Mar 25 11:53:46 2023. +Package patch-redis-6.2.5-1-HP002-1-1.x86_64 is already installed. +Dependencies resolved. +Nothing to do. +Complete! +Applying hot patch +Apply hot patch succeed: redis-6.2.5-1/HP002. +``` + +Scan the CVEs to be fixed on the host. Because the cold patch **redis-6.2.5-3.x86_64** corresponding to **patch-redis-6.2.5-1-HP002-1-1.x86_64** is of a higher version than **redis-6.2.5-2.x86_64**, **redis-6.2.5-2.x86_64** also fixes CVE-2021-1, CVE-2021-11, CVE-2021-2, and CVE-2021-22. + +```shell +$ dnf hot-updateinfo list cves +Last metadata expiration check: 0:00:38 ago on Sat Mar 25 11:53:46 2023. +CVE-2023-22995 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64 - +CVE-2023-26545 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64 - +CVE-2022-40897 Important/Sec. python3-setuptools-59.4.0-5.oe2203sp1.noarch - +CVE-2021-33 Important/Sec. redis-6.2.5-4.x86_64 - +CVE-2021-3 Important/Sec. redis-6.2.5-4.x86_64 - +CVE-2022-38023 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64 - +CVE-2022-37966 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64 - +``` + +- Case 2 + +Open the **xxx-updateinfo.xml.gz** file in the **repodata** directory of the hot patch repository. Check the information related to CVE-2021-33 and CVE-2021-3. + +```xml + + openEuler-SA-2022-3 + An update for mariadb is now available for openEuler-22.03-LTS + Important + openEuler + + + + + + patch-redis-6.2.5-2-HP001.(CVE-2022-24048) + + + openEuler + + patch-redis-6.2.5-2-HP001-1-1.aarch64.rpm + + + patch-redis-6.2.5-2-HP001-1-1.x86_64.rpm + + + + +``` + +The format of the **name** field of **package** is **patch-\-\-\-\**. In the example, **patch-redis-6.2.5-2-HP001** requires the source code version of redis-6.2.5-2 to be installed. Check the version of Redis on the host. + +```shell +$ rpm -qa | grep redis +redis-6.2.5-1.x86_64 +``` + +redis-6.2.5-2 is not installed. Therefore, the hot patch will not be displayed. + +```shell +$ dnf hot-updateinfo list cves +Last metadata expiration check: 0:00:38 ago on Sat Mar 25 11:53:46 2023. +CVE-2023-22995 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64 - +CVE-2023-26545 Important/Sec. python3-perf-5.10.0-136.22.0.98.oe2203sp1.x86_64 - +CVE-2022-40897 Important/Sec. python3-setuptools-59.4.0-5.oe2203sp1.noarch - +CVE-2021-33 Important/Sec. redis-6.2.5-4.x86_64 - +CVE-2021-3 Important/Sec. redis-6.2.5-4.x86_64 - +CVE-2022-38023 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64 - +CVE-2022-37966 Important/Sec. samba-client-4.17.2-5.oe2203sp1.x86_64 - +``` diff --git a/docs/en/server/maintenance/aops/figures/add_config.png b/docs/en/server/maintenance/aops/figures/add_config.png new file mode 100644 index 0000000000000000000000000000000000000000..18d71c2e099c19b5d28848eec6a8d11f29ccee27 Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/add_config.png differ diff --git a/docs/en/server/maintenance/aops/figures/add_node.png b/docs/en/server/maintenance/aops/figures/add_node.png new file mode 100644 index 0000000000000000000000000000000000000000..d68f5e12a62548f2ec59374bda9ab07f43b8b5cb Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/add_node.png differ diff --git "a/docs/en/server/maintenance/aops/figures/app\350\257\246\346\203\205.jpg" "b/docs/en/server/maintenance/aops/figures/app\350\257\246\346\203\205.jpg" new file mode 100644 index 0000000000000000000000000000000000000000..bd179be46c9e711d7148ee44dc56f4a7a02f56bf Binary files /dev/null and "b/docs/en/server/maintenance/aops/figures/app\350\257\246\346\203\205.jpg" differ diff --git a/docs/en/server/maintenance/aops/figures/chakanyuqi.png b/docs/en/server/maintenance/aops/figures/chakanyuqi.png new file mode 100644 index 0000000000000000000000000000000000000000..bbead6a91468d5dee570cfdc66faf9a4ab155d7c Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/chakanyuqi.png differ diff --git a/docs/en/server/maintenance/aops/figures/chaxunshijipeizhi.png b/docs/en/server/maintenance/aops/figures/chaxunshijipeizhi.png new file mode 100644 index 0000000000000000000000000000000000000000..d5f6e450fc0e1e246492ca71a6fcd8db572eb469 Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/chaxunshijipeizhi.png differ diff --git a/docs/en/server/maintenance/aops/figures/chuangjianyewuyu.png b/docs/en/server/maintenance/aops/figures/chuangjianyewuyu.png new file mode 100644 index 0000000000000000000000000000000000000000..8849a2fc81dbd14328c6c66c53033164a0b67b52 Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/chuangjianyewuyu.png differ diff --git a/docs/en/server/maintenance/aops/figures/create_service_domain.png b/docs/en/server/maintenance/aops/figures/create_service_domain.png new file mode 100644 index 0000000000000000000000000000000000000000..4f5b8de2d2c4ddb9bfdfba1ac17258a834561e2d Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/create_service_domain.png differ diff --git a/docs/en/server/maintenance/aops/figures/delete_config.png b/docs/en/server/maintenance/aops/figures/delete_config.png new file mode 100644 index 0000000000000000000000000000000000000000..cfea2eb44f7b8aa809404b8b49b4bd2e24172568 Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/delete_config.png differ diff --git a/docs/en/server/maintenance/aops/figures/elasticsearch-config-1.png b/docs/en/server/maintenance/aops/figures/elasticsearch-config-1.png new file mode 100644 index 0000000000000000000000000000000000000000..1b7e0eab093b2f0455b8f3972884e5f757fbec3d Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/elasticsearch-config-1.png differ diff --git a/docs/en/server/maintenance/aops/figures/elasticsearch-config-2.png b/docs/en/server/maintenance/aops/figures/elasticsearch-config-2.png new file mode 100644 index 0000000000000000000000000000000000000000..620dbbda71259e3b6ee6a2efb646a9692adf2456 Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/elasticsearch-config-2.png differ diff --git a/docs/en/server/maintenance/aops/figures/elasticsearch3.png b/docs/en/server/maintenance/aops/figures/elasticsearch3.png new file mode 100644 index 0000000000000000000000000000000000000000..893aae242aa9117c64f323374d4728d230894973 Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/elasticsearch3.png differ diff --git a/docs/en/server/maintenance/aops/figures/hot_patch_statuses.png b/docs/en/server/maintenance/aops/figures/hot_patch_statuses.png new file mode 100644 index 0000000000000000000000000000000000000000..f5f8a3a95705145787e7aaf9c8d1fff404892240 Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/hot_patch_statuses.png differ diff --git a/docs/en/server/maintenance/aops/figures/kafka-config.png b/docs/en/server/maintenance/aops/figures/kafka-config.png new file mode 100644 index 0000000000000000000000000000000000000000..57eb17ccbd2fa63d97f700c29847fac7f08042ff Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/kafka-config.png differ diff --git a/docs/en/server/maintenance/aops/figures/modify-mysql-config-file.png b/docs/en/server/maintenance/aops/figures/modify-mysql-config-file.png new file mode 100644 index 0000000000000000000000000000000000000000..d83425ee0622be329782620318818662b292e88b Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/modify-mysql-config-file.png differ diff --git a/docs/en/server/maintenance/aops/figures/prometheus-config.png b/docs/en/server/maintenance/aops/figures/prometheus-config.png new file mode 100644 index 0000000000000000000000000000000000000000..7c8d0328967e8eb9bc4aa7465a273b9ef5a30b58 Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/prometheus-config.png differ diff --git a/docs/en/server/maintenance/aops/figures/query_actual_config.png b/docs/en/server/maintenance/aops/figures/query_actual_config.png new file mode 100644 index 0000000000000000000000000000000000000000..d5f6e450fc0e1e246492ca71a6fcd8db572eb469 Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/query_actual_config.png differ diff --git a/docs/en/server/maintenance/aops/figures/query_status.png b/docs/en/server/maintenance/aops/figures/query_status.png new file mode 100644 index 0000000000000000000000000000000000000000..a3d0b3294bf6e0eeec50a2c2f8c5059bdc256376 Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/query_status.png differ diff --git a/docs/en/server/maintenance/aops/figures/shanchupeizhi.png b/docs/en/server/maintenance/aops/figures/shanchupeizhi.png new file mode 100644 index 0000000000000000000000000000000000000000..cfea2eb44f7b8aa809404b8b49b4bd2e24172568 Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/shanchupeizhi.png differ diff --git a/docs/en/server/maintenance/aops/figures/sync_conf.png b/docs/en/server/maintenance/aops/figures/sync_conf.png new file mode 100644 index 0000000000000000000000000000000000000000..c8c229bf41b27f1fe6629106957fd5e47851096d Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/sync_conf.png differ diff --git a/docs/en/server/maintenance/aops/figures/tianjianode.png b/docs/en/server/maintenance/aops/figures/tianjianode.png new file mode 100644 index 0000000000000000000000000000000000000000..d68f5e12a62548f2ec59374bda9ab07f43b8b5cb Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/tianjianode.png differ diff --git a/docs/en/server/maintenance/aops/figures/view_expected_config.png b/docs/en/server/maintenance/aops/figures/view_expected_config.png new file mode 100644 index 0000000000000000000000000000000000000000..bbead6a91468d5dee570cfdc66faf9a4ab155d7c Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/view_expected_config.png differ diff --git a/docs/en/server/maintenance/aops/figures/web-config.png b/docs/en/server/maintenance/aops/figures/web-config.png new file mode 100644 index 0000000000000000000000000000000000000000..721335115922e03f255e67e6b775c1ac0cfbbc50 Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/web-config.png differ diff --git a/docs/en/server/maintenance/aops/figures/xinzengpeizhi.png b/docs/en/server/maintenance/aops/figures/xinzengpeizhi.png new file mode 100644 index 0000000000000000000000000000000000000000..18d71c2e099c19b5d28848eec6a8d11f29ccee27 Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/xinzengpeizhi.png differ diff --git a/docs/en/server/maintenance/aops/figures/zhuangtaichaxun.png b/docs/en/server/maintenance/aops/figures/zhuangtaichaxun.png new file mode 100644 index 0000000000000000000000000000000000000000..a3d0b3294bf6e0eeec50a2c2f8c5059bdc256376 Binary files /dev/null and b/docs/en/server/maintenance/aops/figures/zhuangtaichaxun.png differ diff --git "a/docs/en/server/maintenance/aops/figures/\344\270\273\346\234\272\347\256\241\347\220\206.jpg" "b/docs/en/server/maintenance/aops/figures/\344\270\273\346\234\272\347\256\241\347\220\206.jpg" new file mode 100644 index 0000000000000000000000000000000000000000..9f6d8858468c0cc72c1bd395403f064cc63f82bd Binary files /dev/null and "b/docs/en/server/maintenance/aops/figures/\344\270\273\346\234\272\347\256\241\347\220\206.jpg" differ diff --git "a/docs/en/server/maintenance/aops/figures/\344\270\273\346\234\272\347\273\204.jpg" "b/docs/en/server/maintenance/aops/figures/\344\270\273\346\234\272\347\273\204.jpg" new file mode 100644 index 0000000000000000000000000000000000000000..fb5472de6b3d30abf6af73e286f70ac8e1d58c15 Binary files /dev/null and "b/docs/en/server/maintenance/aops/figures/\344\270\273\346\234\272\347\273\204.jpg" differ diff --git "a/docs/en/server/maintenance/aops/figures/\344\270\273\346\234\272\350\257\246\346\203\205.jpg" "b/docs/en/server/maintenance/aops/figures/\344\270\273\346\234\272\350\257\246\346\203\205.jpg" new file mode 100644 index 0000000000000000000000000000000000000000..effd8c29aba14c2e8f301f9f60d8f25ce8c533f0 Binary files /dev/null and "b/docs/en/server/maintenance/aops/figures/\344\270\273\346\234\272\350\257\246\346\203\205.jpg" differ diff --git "a/docs/en/server/maintenance/aops/figures/\344\277\256\346\224\271\346\217\222\344\273\266.png" "b/docs/en/server/maintenance/aops/figures/\344\277\256\346\224\271\346\217\222\344\273\266.png" new file mode 100644 index 0000000000000000000000000000000000000000..ba4a8d4d9aadb7f712bdcb4b193f05f956d38841 Binary files /dev/null and "b/docs/en/server/maintenance/aops/figures/\344\277\256\346\224\271\346\217\222\344\273\266.png" differ diff --git "a/docs/en/server/maintenance/aops/figures/\344\277\256\346\224\271\346\250\241\345\236\213.png" "b/docs/en/server/maintenance/aops/figures/\344\277\256\346\224\271\346\250\241\345\236\213.png" new file mode 100644 index 0000000000000000000000000000000000000000..23ff4e5fddb87ac157b1002a70c47d9b4c76b873 Binary files /dev/null and "b/docs/en/server/maintenance/aops/figures/\344\277\256\346\224\271\346\250\241\345\236\213.png" differ diff --git "a/docs/en/server/maintenance/aops/figures/\345\210\233\345\273\272\345\267\245\344\275\234\346\265\201.jpg" "b/docs/en/server/maintenance/aops/figures/\345\210\233\345\273\272\345\267\245\344\275\234\346\265\201.jpg" new file mode 100644 index 0000000000000000000000000000000000000000..1a2b45e860914a1ac0cfb6908b02fb5cad4cbd60 Binary files /dev/null and "b/docs/en/server/maintenance/aops/figures/\345\210\233\345\273\272\345\267\245\344\275\234\346\265\201.jpg" differ diff --git "a/docs/en/server/maintenance/aops/figures/\345\221\212\350\255\246.jpg" "b/docs/en/server/maintenance/aops/figures/\345\221\212\350\255\246.jpg" new file mode 100644 index 0000000000000000000000000000000000000000..89ac88e154275d4be8179d773e7093f2357f425f Binary files /dev/null and "b/docs/en/server/maintenance/aops/figures/\345\221\212\350\255\246.jpg" differ diff --git "a/docs/en/server/maintenance/aops/figures/\345\221\212\350\255\246\347\241\256\350\256\244.jpg" "b/docs/en/server/maintenance/aops/figures/\345\221\212\350\255\246\347\241\256\350\256\244.jpg" new file mode 100644 index 0000000000000000000000000000000000000000..57844f772853c541f7a1328b007a9b6ae4d5caf0 Binary files /dev/null and "b/docs/en/server/maintenance/aops/figures/\345\221\212\350\255\246\347\241\256\350\256\244.jpg" differ diff --git "a/docs/en/server/maintenance/aops/figures/\345\221\212\350\255\246\350\257\246\346\203\205.jpg" "b/docs/en/server/maintenance/aops/figures/\345\221\212\350\255\246\350\257\246\346\203\205.jpg" new file mode 100644 index 0000000000000000000000000000000000000000..5b4830b47897a0d51be28238a879a70b1de9ca3b Binary files /dev/null and "b/docs/en/server/maintenance/aops/figures/\345\221\212\350\255\246\350\257\246\346\203\205.jpg" differ diff --git "a/docs/en/server/maintenance/aops/figures/\345\267\245\344\275\234\345\217\260.jpg" "b/docs/en/server/maintenance/aops/figures/\345\267\245\344\275\234\345\217\260.jpg" new file mode 100644 index 0000000000000000000000000000000000000000..998b81e3b88d888d0915dcff48dc8cc5df30d91c Binary files /dev/null and "b/docs/en/server/maintenance/aops/figures/\345\267\245\344\275\234\345\217\260.jpg" differ diff --git "a/docs/en/server/maintenance/aops/figures/\345\267\245\344\275\234\346\265\201.jpg" "b/docs/en/server/maintenance/aops/figures/\345\267\245\344\275\234\346\265\201.jpg" new file mode 100644 index 0000000000000000000000000000000000000000..17fb5b13034e1fc5276c68583fed1952415b0b5f Binary files /dev/null and "b/docs/en/server/maintenance/aops/figures/\345\267\245\344\275\234\346\265\201.jpg" differ diff --git "a/docs/en/server/maintenance/aops/figures/\345\267\245\344\275\234\346\265\201\350\257\246\346\203\205.jpg" "b/docs/en/server/maintenance/aops/figures/\345\267\245\344\275\234\346\265\201\350\257\246\346\203\205.jpg" new file mode 100644 index 0000000000000000000000000000000000000000..458e023847bb2ad1f198f5a2dd1691748038137e Binary files /dev/null and "b/docs/en/server/maintenance/aops/figures/\345\267\245\344\275\234\346\265\201\350\257\246\346\203\205.jpg" differ diff --git "a/docs/en/server/maintenance/aops/figures/\345\272\224\347\224\250.png" "b/docs/en/server/maintenance/aops/figures/\345\272\224\347\224\250.png" new file mode 100644 index 0000000000000000000000000000000000000000..aa34bb909ee7c86a95126c13fa532ce93410a931 Binary files /dev/null and "b/docs/en/server/maintenance/aops/figures/\345\272\224\347\224\250.png" differ diff --git "a/docs/en/server/maintenance/aops/figures/\346\217\222\344\273\266\347\256\241\347\220\206.jpg" "b/docs/en/server/maintenance/aops/figures/\346\217\222\344\273\266\347\256\241\347\220\206.jpg" new file mode 100644 index 0000000000000000000000000000000000000000..2258d03976902052aaf39d36b6374fa680b9f8aa Binary files /dev/null and "b/docs/en/server/maintenance/aops/figures/\346\217\222\344\273\266\347\256\241\347\220\206.jpg" differ diff --git "a/docs/en/server/maintenance/aops/figures/\346\267\273\345\212\240\344\270\273\346\234\272\347\273\204.jpg" "b/docs/en/server/maintenance/aops/figures/\346\267\273\345\212\240\344\270\273\346\234\272\347\273\204.jpg" new file mode 100644 index 0000000000000000000000000000000000000000..9fcd24d949e500323e7a466be7cbfaf48d257ad0 Binary files /dev/null and "b/docs/en/server/maintenance/aops/figures/\346\267\273\345\212\240\344\270\273\346\234\272\347\273\204.jpg" differ diff --git a/docs/en/server/maintenance/aops/gala-ragdoll-user-guide.md b/docs/en/server/maintenance/aops/gala-ragdoll-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..aa199fd1b35aaf7cc123520af66b1a312df1a7b0 --- /dev/null +++ b/docs/en/server/maintenance/aops/gala-ragdoll-user-guide.md @@ -0,0 +1,137 @@ +# gala-ragdoll Usage Guide + +## Installation + +### Manual Installation + +- Installing using the repo source mounted by Yum. + + Configure the Yum sources **openEuler23.09** and **openEuler23.09:Epol** in the **/etc/yum.repos.d/openEuler.repo** file. + + ```ini + [everything] # openEuler 23.09 officially released repository + name=openEuler23.09 + baseurl=https://repo.openeuler.org/openEuler-23.09/everything/$basearch/ + enabled=1 + gpgcheck=1 + gpgkey=https://repo.openeuler.org/openEuler-23.09/everything/$basearch/RPM-GPG-KEY-openEuler + + [Epol] # openEuler 23.09:Epol officially released repository + name=Epol + baseurl=https://repo.openeuler.org/openEuler-23.09/EPOL/main/$basearch/ + enabled=1 + gpgcheck=1 + gpgkey=https://repo.openeuler.org/openEuler-23.09/OS/$basearch/RPM-GPG-KEY-openEuler + + ``` + + Run the following commands to download and install gala-ragdoll and its dependencies. + + ```shell + yum install gala-ragdoll # A-Ops configuration source tracing service + yum install python3-gala-ragdoll + + yum install gala-spider # A-Ops architecture awareness service + yum install python3-gala-spider + ``` + +- Installing using the RPM packages. Download **gala-ragdoll-vx.x.x-x.oe1.aarch64.rpm**, and then run the following commands to install the modules. (`x.x-x` indicates the version. Replace it with the actual version number.) + + ```shell + rpm -ivh gala-ragdoll-vx.x.x-x.oe1.aarch64.rpm + ``` + +### Installing Using the A-Ops Deployment Service + +#### Editing the Task List + +Modify the deployment task list and enable the steps for gala_ragdoll: + +```yaml +--- +step_list: + ... + gala_ragdoll: + enable: false + continue: false + ... +``` + +### Configuration File Description + +`/etc/yum.repos.d/openEuler.repo` is the configuration file used to specify the Yum source address. The content of the configuration file is as follows: + +```ini +[OS] +name=OS +baseurl=http://repo.openeuler.org/openEuler-23.09/OS/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-23.09/OS/$basearch/RPM-GPG-KEY-openEuler +``` + +### YANG Model Description + +`/etc/yum.repos.d/openEuler.repo` is expressed using the YANG language. For details, see `gala-ragdoll/yang_modules/openEuler-logos-openEuler.repo.yang`. +The following extended fields are added: + +| Extended Field Name | Extended Field Format| Example| +| ------------ | ---------------------- | ----------------------------------------- | +| path | OS_TYPE:CONFIGURATION_FILE_PATH | openEuler:/etc/yum.repos.d/openEuler.repo | +| type | Configuration file type | ini, key-value, json, text, and more | +| spacer | Spacer between a configuration item and its value | " ", "=", ":", and more | + +Attachment: Learning the YANG language: . + +### Creating Domains using Configuration Source Tracing + +#### Viewing the Configuration File + +gala-ragdoll contains the configuration file of the configuration source tracing. + +```shell +[root@openeuler-development-1-1drnd ~]# cat /etc/ragdoll/gala-ragdoll.conf +[git] // Defines the current Git information, including the directory and user information of the Git repository. +git_dir = "/home/confTraceTestConf" +user_name = "user" +user_email = "email" + +[collect] // The collect interface provided by A-Ops. +collect_address = "http://192.168.0.0:11111" +collect_api = "/manage/config/collect" + +[ragdoll] +port = 11114 +``` + +#### Creating the Configuration Domain + +![](./figures/create_service_domain.png) + +#### Adding Managed Nodes to the Configuration Domain + +![](./figures/add_node.png) + +#### Adding Configurations to the Configuration Domain + +![](./figures/add_config.png) + +#### Querying the Expected Configuration + +![](./figures/view_expected_config.png) + +#### Deleting Configurations + +![](./figures/delete_config.png) + +#### Querying the Actual Configuration + +![](./figures/query_actual_config.png) + +#### Verifying the Configuration + +![](./figures/query_status.png) + +#### Configuration Synchronization + +![](./figures/sync_conf.png) diff --git a/docs/en/server/maintenance/aops/image/45515A7F-0EC2-45AA-9B58-AB92DE9B0979.png b/docs/en/server/maintenance/aops/image/45515A7F-0EC2-45AA-9B58-AB92DE9B0979.png new file mode 100644 index 0000000000000000000000000000000000000000..c810b26ad0c052960dfdf4bfd78e9224ce465318 Binary files /dev/null and b/docs/en/server/maintenance/aops/image/45515A7F-0EC2-45AA-9B58-AB92DE9B0979.png differ diff --git a/docs/en/server/maintenance/aops/image/E574E637-0BF3-4F3B-BAE6-04ECBD09D151.png b/docs/en/server/maintenance/aops/image/E574E637-0BF3-4F3B-BAE6-04ECBD09D151.png new file mode 100644 index 0000000000000000000000000000000000000000..6ef6ef9bd126e6c2007389065bbecc1cfdd97f5b Binary files /dev/null and b/docs/en/server/maintenance/aops/image/E574E637-0BF3-4F3B-BAE6-04ECBD09D151.png differ diff --git a/docs/en/server/maintenance/aops/image/EF5E0132-6E5C-4DD1-8CB5-73035278E233.png b/docs/en/server/maintenance/aops/image/EF5E0132-6E5C-4DD1-8CB5-73035278E233.png new file mode 100644 index 0000000000000000000000000000000000000000..a2a29d2e1b62f7df409e87d03f2525ba8355f77e Binary files /dev/null and b/docs/en/server/maintenance/aops/image/EF5E0132-6E5C-4DD1-8CB5-73035278E233.png differ diff --git a/docs/en/server/maintenance/aops/image/hotpatch-fix-pr.png b/docs/en/server/maintenance/aops/image/hotpatch-fix-pr.png new file mode 100644 index 0000000000000000000000000000000000000000..209c73f7b4522819c52662a9038bdf19a88eacfd Binary files /dev/null and b/docs/en/server/maintenance/aops/image/hotpatch-fix-pr.png differ diff --git a/docs/en/server/maintenance/aops/image/hotpatch-pr-success.png b/docs/en/server/maintenance/aops/image/hotpatch-pr-success.png new file mode 100644 index 0000000000000000000000000000000000000000..48ea807e03c0f8e6efbceacbbc583c6ac3b3c865 Binary files /dev/null and b/docs/en/server/maintenance/aops/image/hotpatch-pr-success.png differ diff --git a/docs/en/server/maintenance/aops/image/hotpatch-xml.PNG b/docs/en/server/maintenance/aops/image/hotpatch-xml.PNG new file mode 100644 index 0000000000000000000000000000000000000000..f1916620d3cc7b1c29059bcc5513fdc7ee94127b Binary files /dev/null and b/docs/en/server/maintenance/aops/image/hotpatch-xml.PNG differ diff --git a/docs/en/server/maintenance/aops/image/image-20230607161545732.png b/docs/en/server/maintenance/aops/image/image-20230607161545732.png new file mode 100644 index 0000000000000000000000000000000000000000..ba6992bea8d2a1d7ca4769ebfdd850b98d1a372f Binary files /dev/null and b/docs/en/server/maintenance/aops/image/image-20230607161545732.png differ diff --git a/docs/en/server/maintenance/aops/image/image-20230607163358749.png b/docs/en/server/maintenance/aops/image/image-20230607163358749.png new file mode 100644 index 0000000000000000000000000000000000000000..191c36b65058ce8dea6bb2f1fe10a85b0177f2cf Binary files /dev/null and b/docs/en/server/maintenance/aops/image/image-20230607163358749.png differ diff --git a/docs/en/server/maintenance/aops/image/image-20230612113428096.png b/docs/en/server/maintenance/aops/image/image-20230612113428096.png new file mode 100644 index 0000000000000000000000000000000000000000..48b59b5e6cb4043703de96066c8d67e85eed4f16 Binary files /dev/null and b/docs/en/server/maintenance/aops/image/image-20230612113428096.png differ diff --git a/docs/en/server/maintenance/aops/image/image-20230612113626330.png b/docs/en/server/maintenance/aops/image/image-20230612113626330.png new file mode 100644 index 0000000000000000000000000000000000000000..9d3621022deb02b267c3eb29315a7fe33c1f095e Binary files /dev/null and b/docs/en/server/maintenance/aops/image/image-20230612113626330.png differ diff --git "a/docs/en/server/maintenance/aops/image/openEuler\344\273\223\350\257\204\350\256\272.png" "b/docs/en/server/maintenance/aops/image/openEuler\344\273\223\350\257\204\350\256\272.png" new file mode 100644 index 0000000000000000000000000000000000000000..29223cbddc39f8fcc0b725a3ed83495709e05f78 Binary files /dev/null and "b/docs/en/server/maintenance/aops/image/openEuler\344\273\223\350\257\204\350\256\272.png" differ diff --git a/docs/en/server/maintenance/aops/image/patch-file.PNG b/docs/en/server/maintenance/aops/image/patch-file.PNG new file mode 100644 index 0000000000000000000000000000000000000000..f587a48c2be945beaadecf44a6d711da14be50c6 Binary files /dev/null and b/docs/en/server/maintenance/aops/image/patch-file.PNG differ diff --git "a/docs/en/server/maintenance/aops/image/src-openEuler\344\273\223\350\257\204\350\256\272.png" "b/docs/en/server/maintenance/aops/image/src-openEuler\344\273\223\350\257\204\350\256\272.png" new file mode 100644 index 0000000000000000000000000000000000000000..3f8fbd534e8f8a48fdd60a5c3f13b33531a4112a Binary files /dev/null and "b/docs/en/server/maintenance/aops/image/src-openEuler\344\273\223\350\257\204\350\256\272.png" differ diff --git "a/docs/en/server/maintenance/aops/image/\344\273\273\345\212\241\347\273\223\346\236\234\346\237\245\347\234\213.png" "b/docs/en/server/maintenance/aops/image/\344\273\273\345\212\241\347\273\223\346\236\234\346\237\245\347\234\213.png" new file mode 100644 index 0000000000000000000000000000000000000000..31fe24f44facaaa62fbeddd3eef0090a3be88908 Binary files /dev/null and "b/docs/en/server/maintenance/aops/image/\344\273\273\345\212\241\347\273\223\346\236\234\346\237\245\347\234\213.png" differ diff --git "a/docs/en/server/maintenance/aops/image/\345\210\233\345\273\272\350\204\232\346\234\254.png" "b/docs/en/server/maintenance/aops/image/\345\210\233\345\273\272\350\204\232\346\234\254.png" new file mode 100644 index 0000000000000000000000000000000000000000..feb95836d056335d9d7ef673acc5fdf39e29bd8e Binary files /dev/null and "b/docs/en/server/maintenance/aops/image/\345\210\233\345\273\272\350\204\232\346\234\254.png" differ diff --git "a/docs/en/server/maintenance/aops/image/\345\210\233\345\273\272\350\204\232\346\234\254\344\273\273\345\212\241.png" "b/docs/en/server/maintenance/aops/image/\345\210\233\345\273\272\350\204\232\346\234\254\344\273\273\345\212\241.png" new file mode 100644 index 0000000000000000000000000000000000000000..e7b1c5fc77c4027f1cdb96941440220db8637e5f Binary files /dev/null and "b/docs/en/server/maintenance/aops/image/\345\210\233\345\273\272\350\204\232\346\234\254\344\273\273\345\212\241.png" differ diff --git "a/docs/en/server/maintenance/aops/image/\345\215\225\346\254\241\346\211\247\350\241\214.png" "b/docs/en/server/maintenance/aops/image/\345\215\225\346\254\241\346\211\247\350\241\214.png" new file mode 100644 index 0000000000000000000000000000000000000000..8020c60843c11e566778a1a03c1fa7516de9dd6b Binary files /dev/null and "b/docs/en/server/maintenance/aops/image/\345\215\225\346\254\241\346\211\247\350\241\214.png" differ diff --git "a/docs/en/server/maintenance/aops/image/\345\220\257\345\212\250\347\203\255\350\241\245\344\270\201\345\267\245\347\250\213\346\265\201\347\250\213.png" "b/docs/en/server/maintenance/aops/image/\345\220\257\345\212\250\347\203\255\350\241\245\344\270\201\345\267\245\347\250\213\346\265\201\347\250\213.png" new file mode 100644 index 0000000000000000000000000000000000000000..1405eced0a14e3956191e111b7c1d588e5b3d27b Binary files /dev/null and "b/docs/en/server/maintenance/aops/image/\345\220\257\345\212\250\347\203\255\350\241\245\344\270\201\345\267\245\347\250\213\346\265\201\347\250\213.png" differ diff --git "a/docs/en/server/maintenance/aops/image/\345\221\250\346\234\237\346\211\247\350\241\214.png" "b/docs/en/server/maintenance/aops/image/\345\221\250\346\234\237\346\211\247\350\241\214.png" new file mode 100644 index 0000000000000000000000000000000000000000..b75743556384fe58690847b3794607ef9a890d6d Binary files /dev/null and "b/docs/en/server/maintenance/aops/image/\345\221\250\346\234\237\346\211\247\350\241\214.png" differ diff --git "a/docs/en/server/maintenance/aops/image/\345\221\275\344\273\244\346\211\247\350\241\214.png" "b/docs/en/server/maintenance/aops/image/\345\221\275\344\273\244\346\211\247\350\241\214.png" new file mode 100644 index 0000000000000000000000000000000000000000..b5c9fbbeb5a4bba5f81d753fa5aa620ad261804c Binary files /dev/null and "b/docs/en/server/maintenance/aops/image/\345\221\275\344\273\244\346\211\247\350\241\214.png" differ diff --git "a/docs/en/server/maintenance/aops/image/\345\221\275\344\273\244\347\256\241\347\220\206\347\225\214\351\235\242.png" "b/docs/en/server/maintenance/aops/image/\345\221\275\344\273\244\347\256\241\347\220\206\347\225\214\351\235\242.png" new file mode 100644 index 0000000000000000000000000000000000000000..c0357fc88d33c8b706203b70016d53629a3db70c Binary files /dev/null and "b/docs/en/server/maintenance/aops/image/\345\221\275\344\273\244\347\256\241\347\220\206\347\225\214\351\235\242.png" differ diff --git "a/docs/en/server/maintenance/aops/image/\346\223\215\344\275\234\347\256\241\347\220\206.png" "b/docs/en/server/maintenance/aops/image/\346\223\215\344\275\234\347\256\241\347\220\206.png" new file mode 100644 index 0000000000000000000000000000000000000000..3a1b8c3accdfb688da2e8e54eb17e86d18ee4d0b Binary files /dev/null and "b/docs/en/server/maintenance/aops/image/\346\223\215\344\275\234\347\256\241\347\220\206.png" differ diff --git "a/docs/en/server/maintenance/aops/image/\346\226\207\344\273\266\346\216\250\351\200\201.png" "b/docs/en/server/maintenance/aops/image/\346\226\207\344\273\266\346\216\250\351\200\201.png" new file mode 100644 index 0000000000000000000000000000000000000000..c449eb18608e0146275f1b9f4ca41d05d48af021 Binary files /dev/null and "b/docs/en/server/maintenance/aops/image/\346\226\207\344\273\266\346\216\250\351\200\201.png" differ diff --git "a/docs/en/server/maintenance/aops/image/\346\226\260\345\273\272\345\221\275\344\273\244.png" "b/docs/en/server/maintenance/aops/image/\346\226\260\345\273\272\345\221\275\344\273\244.png" new file mode 100644 index 0000000000000000000000000000000000000000..50d5bd4ce5499512acf2b8af86445fe6df6ce29f Binary files /dev/null and "b/docs/en/server/maintenance/aops/image/\346\226\260\345\273\272\345\221\275\344\273\244.png" differ diff --git "a/docs/en/server/maintenance/aops/image/\346\226\260\345\273\272\345\221\275\344\273\244\344\273\273\345\212\241.png" "b/docs/en/server/maintenance/aops/image/\346\226\260\345\273\272\345\221\275\344\273\244\344\273\273\345\212\241.png" new file mode 100644 index 0000000000000000000000000000000000000000..792ec4e81017575fd27466a275c0502563808296 Binary files /dev/null and "b/docs/en/server/maintenance/aops/image/\346\226\260\345\273\272\345\221\275\344\273\244\344\273\273\345\212\241.png" differ diff --git "a/docs/en/server/maintenance/aops/image/\347\203\255\350\241\245\344\270\201issue\351\223\276\346\216\245\345\222\214pr\351\223\276\346\216\245.png" "b/docs/en/server/maintenance/aops/image/\347\203\255\350\241\245\344\270\201issue\351\223\276\346\216\245\345\222\214pr\351\223\276\346\216\245.png" new file mode 100644 index 0000000000000000000000000000000000000000..c9f6dc0a0f1a1758bb936b61ec939f8f5eeee633 Binary files /dev/null and "b/docs/en/server/maintenance/aops/image/\347\203\255\350\241\245\344\270\201issue\351\223\276\346\216\245\345\222\214pr\351\223\276\346\216\245.png" differ diff --git "a/docs/en/server/maintenance/aops/image/\350\204\232\346\234\254\346\211\247\350\241\214.png" "b/docs/en/server/maintenance/aops/image/\350\204\232\346\234\254\346\211\247\350\241\214.png" new file mode 100644 index 0000000000000000000000000000000000000000..4ab626ad5949e17a5d486431ae4c0481ca42a442 Binary files /dev/null and "b/docs/en/server/maintenance/aops/image/\350\204\232\346\234\254\346\211\247\350\241\214.png" differ diff --git "a/docs/en/server/maintenance/aops/image/\350\204\232\346\234\254\347\256\241\347\220\206.png" "b/docs/en/server/maintenance/aops/image/\350\204\232\346\234\254\347\256\241\347\220\206.png" new file mode 100644 index 0000000000000000000000000000000000000000..62c60399dc58a79a9ab48a7eb584ce615c11b05c Binary files /dev/null and "b/docs/en/server/maintenance/aops/image/\350\204\232\346\234\254\347\256\241\347\220\206.png" differ diff --git a/docs/en/server/maintenance/aops/overview.md b/docs/en/server/maintenance/aops/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..1dff96bb38202330db97f431a91f0c27919071fd --- /dev/null +++ b/docs/en/server/maintenance/aops/overview.md @@ -0,0 +1,3 @@ +# A-Ops User Guide + +This document describes the A-Ops intelligent O&M framework and how to install and use services such as intelligent location and configuration source tracing, helping you quickly understand and use A-Ops. By using A-Ops, you can reduce the O&M cost of the system cluster, quickly locate system faults, and centrally manage configuration items. diff --git a/docs/en/server/maintenance/common_skills/_toc.yaml b/docs/en/server/maintenance/common_skills/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..72f3ade188ac83a2e01e5d84dec9e1245536708f --- /dev/null +++ b/docs/en/server/maintenance/common_skills/_toc.yaml @@ -0,0 +1,8 @@ +label: Common Skills +isManual: true +description: Common configurations and commands for O&M +sections: + - label: Information Collection + href: ./information-collection.md + - label: Common Configurations + href: ./common-configurations.md diff --git a/docs/en/server/maintenance/common_skills/common-configurations.md b/docs/en/server/maintenance/common_skills/common-configurations.md new file mode 100644 index 0000000000000000000000000000000000000000..c8bd0a7d83de2675177b5ae085f8afc011011a8d --- /dev/null +++ b/docs/en/server/maintenance/common_skills/common-configurations.md @@ -0,0 +1,566 @@ +# Common Configurations + +- [Common Configurations](#common-configurations) + - [Configuring the Network](#configuring-the-network) + - [Managing RPM Packages](#managing-rpm-packages) + - [Configuring SSH](#configuring-ssh) + +## Configuring the Network + +1. Configure the IP address. + + Run the `ip` command to configure an address for the interface. `interface-name` indicates the name of the NIC. + + ```shell + ip addr [ add | del ] address dev interface-name + ``` + +2. Configure a static IP address. + + ```shell + # Configure the static IP address. + ip address add 192.168.0.10/24 dev enp3s0 + + # Run the following command as the root user to query the configuration result: + ip addr show dev enp3s0 + + # The result is as follows: + 2: enp3s0: mtu 1500 qdisc fq_codel state UP group default qlen 1000 + link/ether 52:54:00:aa:ad:4a brd ff:ff:ff:ff:ff:ff + inet 192.168.202.248/16 brd 192.168.255.255 scope global dynamic noprefixroute enp3s0 + valid_lft 9547sec preferred_lft 9547sec + inet 192.168.0.10/24 scope global enp3s0 + valid_lft forever preferred_lft forever + inet6 fe80::32e8:cc22:9db2:f4d4/64 scope link noprefixroute + valid_lft forever preferred_lft forever + ``` + +3. Configure a static route. + + Run the `ip route add` command to add a static route to the routing table and run the `ip route del` command to delete a static route. The common format of the `ip route` command is as follows: + + ```shell + ip route [ add | del | change | append | replace ] destination-address + ``` + + - To add a static route to the host address, run the following command as the **root** user: + + ```shell + ip route add 192.168.2.1 via 10.0.0.1 [dev interface-name] + ``` + + - To add a static route to the network, run the following command as the **root** user: + + ```shell + ip route add 192.168.2.0/24 via 10.0.0.1 [dev interface-name] + ``` + +4. Configure the network using the ifcfg file. + + Modify the **ifcfg-enp4s0** file generated in the **/etc/sysconfig/network-scripts/** directory as the **root** user. The following is an example: + + ```text + TYPE=Ethernet + PROXY_METHOD=none + BROWSER_ONLY=no + BOOTPROTO=none + IPADDR=192.168.0.10 + PREFIX=24 + DEFROUTE=yes + IPV4_FAILURE_FATAL=no + IPV6INIT=yes + IPV6_AUTOCONF=yes + IPV6_DEFROUTE=yes + IPV6_FAILURE_FATAL=no + IPV6_ADDR_GEN_MODE=stable-privacy + NAME=enp4s0static + UUID=xx + DEVICE=enp4s0 + ONBOOT=yes + ``` + +## Managing RPM Packages + +The full name of RPM is RPM Package Manager, which is intended to manage Red Hat software packages. It is used in mainstream distributions such as openEuler, Fedora, Red Hat, Mandriva, SUSE and YellowDog, and distributions developed based on these distributions. + +RPM installs the required software to a set of management programs on the Linux host in database record mode. The software to be installed is compiled and packaged, and the default database record in the packaged software records the dependencies required for the software installation. When a user installs the software on a Linux host, RPM checks whether the dependencies on the Linux host meets the requirements based on the data recorded in it. + +- If yes, install the software. +- If no, do not install the software. + +During the installation, all software information is written into the RPM database for subsequent query, verification, and uninstallation. +![en-us_other_0000001337581224](images/en-us_other_0000001337581224.jpeg) + +1. Default installation path of the RPM packages + + Generally, RPM uses the default installation path. (The default installation path can be queried by running a command and will be described in detail in subsequent sections.) All installation files are distributed to the directories listed in the following table by type. + + RPM installation paths and their meanings + + |Installation Path|Description| + |--|--| + |/etc/|Configuration file installation directory| + |/usr/bin/|Installation directory of the executable commands| + |/usr/lib/|Path for storing the function library used by the program| + |/usr/share/doc|Location where the basic software user manual is saved| + |/usr/share/man/|Path for saving the help file| + + Note: You can manually specify the installation path of RPM, but this method is not recommended. After the installation path is manually specified, all installation files are installed in the specified path, and the command for querying the installation path in the system cannot be used. The command can be identified by the system only after being manually configured. + +2. rpm command options + + **Checking the RPM Signature of the Software Package** + + Before installing the RPM package on a Linux host, check the PGP signature. After ensuring that the signature integrity and source are correct, run the `rpm --checksig` command to verify the validity: + + ```shell + rpm --checksig nano-2.3.1-10.el7.x86_64.rpm + ``` + + **Installing RPM Packages** + + To install RPM packages in Linux, use the `-i` option in the `rpm` command. + + ```shell + rpm -ivh nano-2.3.1-10.el7.x86_64.rpm + ``` + + - `-i`: installs the software package. + - `-v`: displays detailed information. + - `-h`: lists flags during suite installation. + + **Querying an Installed RPM Package** + + To query an RPM package (dnf) installed in the Linux system, use the `-q` option in the `rpm` command. + + ```shell + rpm -q dnf + ``` + + - `-q`: query operation + + If the specified package is not installed, the following error message is displayed: + + ```text + package dnf is not installed + ``` + + **Querying All Installed RPM Packages** + + To query all RPM packages installed in Linux, use the `-qa` option in the `rpm` command. + + ```shell + $ rpm -qa + dracut-config-rescue-055-7.oe2203sp2.x86_64 + parted-3.5-1.oe2203sp2.x86_64 + irqbalance-1.8.0-9.oe2203sp2.x86_64 + ...... + ``` + + Note: When using the `-qa` option, use the pipe character (|) together to improve the search accuracy. + + **Querying Details About an Installed RPM Package** + + Use the `-qi` option in the `rpm` command to query the details of an RPM package installed in the system. + + ```shell + # rpm -qi python3 + Name : python3 + Version : 3.9.9 + Release : 24.oe2203sp2 + Architecture: x86_64 + Install Date: Wed 30 Mar 2022 08:30:23 AM UTC + Group : Unspecified + Size : 35916839 + License : Python + Signature : RSA/SHA1, Wed 30 Mar 2022 03:29:30 AM UTC, Key ID d557065eb25e7f66 + Source RPM : python3-3.9.9-24.oe2203sp2.x86_64.rpm + Build Date : Tue 15 Mar 2022 12:00:00 AM UTC + Build Host : obs-worker1639015616-x86-0001 + Packager : http://openeuler.org + Vendor : http://openeuler.org + URL : https://www.python.org/ + Summary : Interpreter of the Python3 programming language + Description : + Python combines remarkable power with very clear syntax. It has modules, + classes, exceptions, very high level dynamic data types, and dynamic + typing. There are interfaces to many system calls and libraries, as well + as to various windowing systems. New built-in modules are easily written + in C or C++ (or other languages, depending on the chosen implementation). + Python is also usable as an extension language for applications written + in other languages that need easy-to-use scripting or automation interfaces. + + This package Provides python version 3. + ``` + + **Querying All Files in an RPM Package** + + To query the file list of an RPM package that is not installed, use the `-qlp` option in the `rpm` command. + + ```shell + $ rpm -qlp pkgship-2.2.0-10.oe2203sp2.noarch.rpm + /etc/ima/digest_lists.tlv/0-metadata_list-compact_tlv-pkgship-2.2.0-10.oe2203sp2.noarch + /etc/ima/digest_lists/0-metadata_list-compact-pkgship-2.2.0-10.oe2203sp2.noarch + /etc/pkgship/auto_install_pkgship_requires.sh + /etc/pkgship/conf.yaml + /etc/pkgship/package.ini + ...... + ``` + + **Querying RPM Package Dependencies** + + To query the list of dependency packages compiled by a specified RPM package that is not installed, use the `-qRp` option in the `rpm` command. + + ```shell + $ rpm -qRp pkgship-2.2.0-10.oe2203sp2.noarch.rpm + /bin/bash + /bin/sh + /usr/bin/python3 + config(pkgship) = 2.2.0-10.oe2203sp2 + python3 + python3-Flask-Limiter + ...... + ``` + + **Verifying All Installed RPM Packages** + + To verify an installed RPM package, use the `-Va` option in the `rpm` command to compare the information about the files installed in the package with the information about the files obtained from the package metadata stored in the RPM database. + + ```shell + $ rpm -Va + S.5....T. c /root/.bashrc + .......T. c /etc/yum.repos.d/openEuler.repo + S.5....T. c /etc/issue + S.5....T. c /etc/issue.net + S.5....T. c /etc/csh.login + S.5....T. c /etc/profile + .M....G.. g /var/log/lastlog + .M....... c /boot/grub2/grubenv + ...... + ``` + + Output fields of the `rpm -Va` command and their meanings + + |Field|Description| + |--|--| + |S|The file length changes.| + |M|The access permission or type of a file changes.| + |5|The MD5 checksum changes.| + |D|The attributes of a device node change.| + |L|The symbolic link of a file changes.| + |U|The owner of a file, subdirectory, or device node changes.| + |G|The group of a file, subdirectory, or device node changes.| + |T|The last modification time of a file changes.| + + **Querying the RPM Package of a Specific File** + + To query an RPM package that provides a specific binary file on Linux, use the `-qf` option in the `rpm` command. + + ```shell + $ rpm -qf /usr/share/doc/pkgship + pkgship-2.2.0-10.oe2203sp2.noarch.rpm + ``` + + **Querying Files in an Installed RPM Package** + + To query the list of installation files of an RPM package, use the `-ql` option in the `rpm` command. + + ```shell + $ rpm -ql dnf + /etc/bash_completion.d/dnf + /etc/ima/digest_lists.tlv/0-metadata_list-compact_tlv-dnf-4.14.0-14.oe2203sp2.noarch + /etc/ima/digest_lists/0-metadata_list-compact-dnf-dnf-4.14.0-14.oe2203sp2.noarch + /usr/bin/dnf + /usr/lib/systemd/system/dnf-makecache.service + /usr/lib/systemd/system/dnf-makecache.timer + /usr/share/doc/dnf + /usr/share/doc/dnf/AUTHORS + /usr/share/doc/dnf/README.rst + /usr/share/licenses/dnf + /usr/share/licenses/dnf/COPYING + /usr/share/licenses/dnf/PACKAGE-LICENSING + /var/cache/dnf + ``` + + **Querying the Recently Installed RPM Packages** + + Linux is a multi-user OS. During the use of Linux, other users may have installed some software packages. To query the recently installed packages in the system, use the `-qa --last` options in the `rpm` command. + + ```shell + $ rpm -qa --last + ntp-4.2.8p15-11.oe2203sp2.x86_64 + ntpstat-0.6-4.oe2203sp2.noarch + ntp-help-4.2.8p15-11.oe2203sp2.noarch + ``` + + **Querying Only the Documents of the Installed RPM Packages** + + You can obtain the help information of any command from the **Linux Man** page (path for storing **/usr/share/doc/Package\_Name-Version\_Number/docs\*** documents). To query the list of documents associated with the installed RPM packages, use the `-qdf` option in the `rpm` command and enter the binary file path. + + ```shell + $ rpm -qdf /usr/bin/grep + /usr/share/doc/grep/NEWS + /usr/share/doc/grep/README + /usr/share/doc/grep/THANKS + /usr/share/doc/grep/TODO + /usr/share/info/grep.info.gz + /usr/share/man/man1/egrep.1.gz + /usr/share/man/man1/fgrep.1.gz + /usr/share/man/man1/grep.1.gz + ``` + + **Upgrading an Installed RPM Package** + + You can easily upgrade the installed RPM package to the latest version by using the `-Uvh` option and the `rpm` command. + + ```shell + $ rpm -Uvh pkgship-2.2.0-10.oe2203sp2.noarch.rpm + Preparing... ################################# [100%] + ``` + + Note: When the installed RPM package is upgraded, the old RPM package is deleted and the new RPM package is installed. + + **Removing an Installed RPM Package** + + To remove an RPM package installed on the system, use the `-ev` or `-e` option in the `rpm` command. + + ```shell + rpm -ev pkgship + ``` + + **Rebuilding the Damaged RPM Database** + + When you try to update the system using the `yum update` command, you may receive an error message indicating that the RPM database is damaged. If you receive this message, use the `--rebuilddb` option in the `rpm` command to rebuild the database. + + ```shell + rm /var/lib/rpm/__db* + rpm --rebuilddb + ``` + + **Checking Whether Vulnerabilities in Specific Packages Have Been Fixed** + + You can use the `--changelog` option in the `rpm` command and enter the corresponding CVE ID. + + ```shell + rpm -q --changelog python-2.6.6 | grep -i "CVE-2019-9636" + ``` + + **Importing the RPM GPG Key** + + By default, when a new repository is added to the Linux system, the GPG key is automatically imported. You can also use `--import` in the `rpm` command to manually import the RPM GPG key to check the integrity of a package when downloading it from the repository. + + ```shell + rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-OpenEuler-22.03-LTS-SP2 + ``` + +3. DNF commands + + DNF commands + + |Command|Description| + |--|--| + |repolist|Displays the configured software repository source.| + |install|Installs one or more software packages on Linux.| + |upgrade|Upgrades one or more software packages on Linux.| + |list|Lists a software package or a group of software packages.| + |info|Displays detailed information about a package or package group.| + |updateinfo|Displays the bulletin information about a package.| + |search|Searches for the specified character string in the software package details.| + |check-update|Checks for software package update.| + |remove|Removes one or more software packages from the system.| + |reinstall|Reinstalls a package.| + |downgrade|Downgrades a software package.| + |autoremove|Removes all unnecessary software packages that are installed due to dependency relationships.| + |distro-sync|Synchronizes the installed software package to the latest available version.| + |makecache|Creates a metadata cache.| + |repository-package|Runs commands on all software packages in a specified repository.| + |provides|Searches for the software package that provides the specified content.| + |group|Displays or uses group information.| + |history|Displays or uses transaction history.| + |clean|Deletes cached data.| + + **Displaying Configured Software Repositories** + + By default, the `--enabled` option is added to display the enabled software repositories. + + ```shell + $ dnf repolist --enabled + repo id repo name + EPOL EPOL + OS OS + debuginfo debuginfo + everything everything + pkgship_elasticsearch Elasticsearch repository + source source + update update + ``` + + - `--all`: displays all software repositories. + - `--disabled`: displays disabled software repositories. + - `--enabled`: displays enabled repositories (default). + + Installing One or More Software Packages + + You can run the `install` command to install RPM packages. + + ```shell + dnf install software_package + ``` + + Conflicting packages or packages that cannot be installed may exist during software package installation. You can add `--allowerasing` to the command to replace the conflicting packages or `--skip-broken` to skip the packages that cannot be installed. + + ```shell + dnf install software_package [software_package ...] --allowerasing --skip-broken + ``` + + When dnf is used to install a software package, add `--installroot` to set the root directory for installing the software package. + + ```shell + dnf install software_package --installroot software_package_root_directory + ``` + + If you need to temporarily specify a repository source for installation, you can add the `--setopt=reposdir=` option to specify the loading directory of the repository source. + + ```shell + dnf install software_package --setopt=reposdir=repo_source_directory + ``` + + If interactive confirmation is not required during installation, you can add `-y` or `--assumeyes` to enable all software packages to be installed to automatically answer **Yes**. + + ```shell + dnf install software_package -y + ``` + + To install an RPM package by specifying a specific repository source, you can specify the `--repo` or `--enablerepo` option. To achieve the same effect, you can also use the `--disablerepo` option to disable the matched repository source. You are advised to use the `--repo` option to install the RPM package. + + ```shell + dnf install software_package --repo=repo_source_ + ``` + + **Reinstalling a Software Package** + + You can run the `reinstall` command to reinstall a software package in the system. + + ```shell + dnf reinstall software_package + ``` + + **Upgrading One or More Software Packages** + + - You can use the `upgrade` command to upgrade one or more software packages on Linux. + + ```shell + dnf upgrade software_package [software_package ...] + ``` + + - You can also run the `update` command to upgrade one or more software packages. + + ```shell + dnf update software_package [software_package ...] + ``` + + **Downgrading a Software Package** + + If a compatibility problem occurs because the version of a software package is too late, you can downgrade the software package. + + ```shell + dnf downgrade software_package + ``` + + **Listing a Package or a Group of Packages** + + You can run the `list` command to list the software packages installed in the system and the software packages in the configured repository. + + ```shell + dnf list + ``` + + You can add options to filter the displayed package list. + + - `--all`: displays all software packages (default). + - `--available`: displays only available software packages. + - `--installed`: displays only installed software packages. + - `--extras`: displays only additional software packages. + - `--updates`: displays only the software packages to be upgraded. + - `--upgrades`: displays only the software packages to be upgraded. + - `--autoremove`: displays only the software packages to be removed. + - `--recent`: displays the software packages that have been changed recently. + + **Querying Details About a Software Package** + + You can run the `info` command to query details about a software package. + + ```shell + dnf info software_package + ``` + + **Searching for a Software Package** + + If you need to install a software package in the system but you are not sure about the full name of the software package, you can run the `search` command to search for the matched package. + + ```shell + dnf search software_package + ``` + + **Uninstalling One or More Software Packages** + + You can run the `remove` command to remove an expired or duplicate software package. + + ```shell + dnf remove software_package + ``` + + - `--duplicates`: removes installed (duplicate) software packages. + - `--oldinstallonly`: removes expired installation-only software packages. + + **Automatically Removing Software Packages Installed Due to Dependency Relationships** + + You can run the `autoremove` command to remove unnecessary software packages that are installed due to dependency relationships. + + ```shell + dnf autoremove software_package + ``` + +## Configuring SSH + +1. Introduction to the SSH service + + Secure Shell (SSH) is a reliable protocol that ensures the security of remote login sessions and other network services. The SSH protocol can effectively prevent information leakage during remote management. SSH encrypts transferred data to prevent domain name server (DNS) spoofing and IP spoofing. OpenSSH was created as an open source alternative to the proprietary SSH protocol. + +2. Configuring the SSH Service + + ```shell + # Open and modify the /etc/ssh/sshd_config file. + vi /etc/ssh/sshd_config + + # Restart the SSH service. + systemctl restart sshd + + # Check the SSH service status. + systemctl status sshd + ``` + +3. Main options in the SSH service configuration file + + ```text + $ Specify the SSH protocol version. + Protocol 2 + + # Allowed users + AllowUsers xxx + + # Denied users + DenyUser root + + # Configure session timeout. + ClientAliveInterval 120 + + # Disable SSH root login. + PermitRootLogin no + + # Configure or change the SSH port number. + Port 1234 + + # Disable SSH password authentication. + PasswordAuthentication no + ``` diff --git a/docs/en/server/maintenance/common_skills/images/en-us_other_0000001337581224.jpeg b/docs/en/server/maintenance/common_skills/images/en-us_other_0000001337581224.jpeg new file mode 100644 index 0000000000000000000000000000000000000000..2c019b828bdf9c699f203f09ba3542968ff21262 Binary files /dev/null and b/docs/en/server/maintenance/common_skills/images/en-us_other_0000001337581224.jpeg differ diff --git a/docs/en/server/maintenance/common_skills/information-collection.md b/docs/en/server/maintenance/common_skills/information-collection.md new file mode 100644 index 0000000000000000000000000000000000000000..3b2d7e31a29bed9e79c94851e82a155c547459ea --- /dev/null +++ b/docs/en/server/maintenance/common_skills/information-collection.md @@ -0,0 +1,157 @@ +# Information Collection + +## Querying OS Information + +1. Query the OS version. + + ```shell + cat /etc/openEuler-latest + cat /etc/os-release + cat /etc/openEuler-release + ``` + +2. Query the kernel version. + + ```shell + uname -a + ``` + +## Querying Hardware Information + +1. Query CPU statistics. + + ```shell + lscpu + ``` + +2. View CPU parameters. + + ```shell + cat /proc/cpuinfo + ``` + +3. View system memory information. + + ```shell + cat /proc/meminfo + ``` + +4. View memory information. + + ```shell + dmidecode -t memory + ``` + +5. View hard drive and partition distribution. + + ```shell + lsblk + ``` + +6. View details about hard drives and partitions. + + ```shell + fdisk -l + ``` + +7. View NIC information. + + ```shell + lspci | grep -i 'eth' + ``` + +8. View all network interfaces. + + ```shell + ip a + yum install -y net-tools + ifconfig + ``` + +9. View details about a network interface. + + ```shell + ethtool enp7s0 # (enp7s0 is used as an example.) + ``` + +10. View PCI information. + + ```shell + lspci + ``` + +11. View the device tree. + + ```shell + lspci -t + ``` + +12. View BIOS information. + + ```shell + dmidecode -t bios + ``` + +## Querying Software Information + +1. Query details about a software package. + + ```shell + rpm -qi systemd # (systemd is used as an example.) + ``` + +2. View the modules provided by a software package. + + ```shell + rpm -q --provides systemd # (systemd is used as an example.) + ``` + +3. View all installed software packages. + + ```shell + rpm -qa | grep systemd # (systemd is used as an example.) + ``` + +4. View the file list of a software package. + + ```shell + rpm -ql python3-rpm # (python3-rpm is used as an example.) + ``` + +## Viewing OS Logs + +1. View the information and error logs after the system is started. + + ```shell + cat /var/log/messages + ``` + +2. View the security-related logs. + + ```shell + cat /var/log/secure + ``` + +3. View the email-related logs. + + ```shell + cat /var/log/maillog + ``` + +4. View the logs related to scheduled tasks. + + ```shell + cat /var/log/cron + ``` + +5. View the logs related to UUCP and news devices. + + ```shell + cat /var/log/spooler + ``` + +6. View the logs related to the startup and stop of the daemon process. + + ```shell + cat /var/log/boot.log + ``` diff --git a/docs/en/server/maintenance/common_tools/_toc.yaml b/docs/en/server/maintenance/common_tools/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..0a0d39f01d141625d7dafb981fa86f70457c711c --- /dev/null +++ b/docs/en/server/maintenance/common_tools/_toc.yaml @@ -0,0 +1,6 @@ +label: Commonly Used Tools for Location and Demarcation +isManual: true +href: ./commonly-used-tools.md +description: >- + Commonly used tools for location and demarcation, including ftrace, strace, + and kdump diff --git a/docs/en/server/maintenance/common_tools/commonly-used-tools.md b/docs/en/server/maintenance/common_tools/commonly-used-tools.md new file mode 100644 index 0000000000000000000000000000000000000000..81a4da0a40667da4482dc85a6534cfcf35dc0c91 --- /dev/null +++ b/docs/en/server/maintenance/common_tools/commonly-used-tools.md @@ -0,0 +1,204 @@ +# Commonly Used Tools + +- [Commonly Used Tools](#commonly-used-tools) + - [ftrace](#ftrace) + - [strace](#strace) + - [kdump](#kdump) + +## ftrace + +1. ftrace: a debug tool for the Linux kernel space. The kernel provides trace events for you to trace . ftrace can capture events so that you can intuitively view these events and trace kernel functions. +2. Configuration and usage of ftrace: To use ftrace, you need to compile its dependencies into the kernel. By default, openEuler compiles the ftrace option. If the ftrace option is not enabled, you can enable it by choosing **Kernel hacking** > **Tracers** > **Trace syscalls** in **menuconfig**. In addition, you need to compile the debugfs by choosing **Kernel hacking** > **Generic Kernel Debugging Instruments** > **Debug Filesystem**. + +- **Configuring the ftrace function** + +ftrace provides access interfaces for user space through the debugfs. After the debugfs is configured in the kernel, the **/sys/kernel/debug** directory is created. The debugfs is mounted to this directory. If the kernel supports ftrace-related configuration items, a **tracing** directory is created in the debugfs. The debugfs is mounted to this directory. The following figure shows the content of this directory. + +![](./images/en-us_image_0000001322372918.png) + +- **Introduction to the ftrace debugfs interface** + + You can view some control and output files provided by ftrace through the debugfs. The common files are described as follows: + +```shell +available_tracers: available tracers + +current_tracer: running tracer + +available_events: lists all available trace events in the OS + +events: This directory differentiates events by module. + +set_event: lists the events to be traced. + +tracing_on: enables or disables tracing. echo 0 > tracing_on indicates that tracing is disabled, and 1 indicates that tracing is enabled. + +trace: queries trace data. +``` + +- **Available tracers** + +![en-us_image_0000001373373585](./images/en-us_image_0000001373373585.png) + +```shell +function: a function call tracing program that does not require parameters +function_graph: a function call tracer that uses subcalls +``` + +- **Trace events** + +```shell +# Specify the arm_event of the RAS to be traced. +echo ras:arm_event > /sys/kernel/debug/tracing/set_event + +# This file contains the event format and fields to be printed. +cat /sys/kernel/debug/tracing/events/ras/arm_event/format + +# Start tracing. +echo 1 > /sys/kernel/debug/tracing/tracing_on + +# Observe the trace output. +tail -f /sys/kernel/debug/tracing/trace +``` + +![c50cb9df64f4659787c810167c89feb4_1884x257](./images/c50cb9df64f4659787c810167c89feb4_1884x257.png) + +- **Tracing input parameters of kernel functions** + +Trace mmap, which corresponds to the system call **do_mmap**. Output the **addr** input parameter. + +![en-us_image_0000001373379529](./images/en-us_image_0000001373379529.png) + +```shell +# Trace through the kprobe. +echo 'p:probe1 do_mmap addr=%x1' > kprobe_events + +# Enable kprobe. +echo 1 > events/kprobes/probe1/enable + +# Start tracing. +echo 1 > tracing_on + +# View trace data. +``` + +![en-us_image_0000001322379488](./images/en-us_image_0000001322379488.png) + +- **Tracing function calls** + +```shell +# Select a tracing type. +echo function_graph > current_tracer + +# Set the PID of the process to be filtered. +echo set_ftrace_pid + +# Start tracing. +echo 1 > tracing_on + +# View trace data. +``` + +![en-us_image_0000001322219840](./images/en-us_image_0000001322219840.png) + +## strace + +The `strace` command is a diagnosis and debugging tool. You can use the `strace` command to analyze system calls and signal transmission of applications to solve problems or understand the application execution process. + +You can run the `strace -h` command to view the functions provided by strace. + +![en-us_image_0000001322112990](./images/en-us_image_0000001322112990.png) + +The most common usage is to trace a command, trace the forks, print the time, and output the result to the **output** file. + +```shell +strace -f -tt -o output xx +``` + +## kdump + +1. crash/kdump Principles + + kdump is a snapshot of the memory status of the OS running at a certain time point. It helps O&M personnel debug and analyze the cause of system breakdown. kdump is usually used when system breakdown and panic happen. + + The process is as follows. + + ![en-us_image_0000001321685172](./images/en-us_image_0000001321685172.png) + +2. Installing and configuring related tools + + ```shell + # Use Yum to install the corresponding software package. + yum install kernel-debuginfo-$(uname -r) kexec-tools crash -y + + # Set the reserved memory size for **crashkernel**. + vim /etc/default/grub + ``` + + ![en-us_image_0000001372821865](./images/en-us_image_0000001372821865.png) + + ```shell + # Regenerate the grub configuration file. + grub2-mkconfig -o /boot/efi/EFI/openEuler/grub.cfg + reboot + + # Start the kdump service. + systemctl start kdump #Start kdump. + systemctl enable kdump #Set the kdump to start upon system startup. + ``` + +3. Triggering a crash + + Step 1. Retain the default settings of the kernel. When a hard lock or oops occurs, a panic is triggered. + + ![en-us_image_0000001372824637](./images/en-us_image_0000001372824637.png) + + Step 2. Modify the settings. The following commands cam make the settings take effect only once and become invalid after the system is restarted. + + ```shell + # Set a soft lock to trigger a panic. + echo 1 > /proc/sys/kernel/softlockup_panic + + # Trigger a kernel panic when an out of memory (OOM) error occurs. + echo 1 > /proc/sys/vm/panic_on_oom + + # A panic occurs when a process is hung. + echo 1 > /proc/sys/kernel/hung_task_panic + + # Set the timeout interval of the hung task mechanism. + echo 60 > /proc/sys/kernel/kernel.hung_task_timeout_secs + ``` + + Step 3. To make the configuration take effect permanently, write the following parameters to the **/etc/sysctl.conf** file and run the `sysctl -p` command: + + ```shell + kernel.hung_task_panic=1 + kernel.hung_task_timeout_secs=60 + kernel.softlockup_panic=1 + vm.panic_on_oom=1 + ``` + +4. Analyzing the crash + +Step 1. Enable crash debugging. + +Step 2. Find the generated **vmcore** file generated in the **/var/crash** directory. + +Step 3. Run the following command to start crash debugging: + +```shell +crash {vmcore file} {debug kernel vmlinux} +``` + +![en-us_image_0000001372748125](./images/en-us_image_0000001372748125.png) + +The format of the **crash** debugging command is *command args*. *command* indicates the command to be executed, and *args* indicates the parameters required by some debugging commands. + +|Command|Description| +|--|--| +|help|Prints the help information of a command. You can view the supported commands or the help information of a specific command. For example, run `help bt`.| +|bt|Prints the function call stack information.| +|log|Prints the system message buffer. Parameters can be appended, for example, **log**.| +|ps|Displays the process status. **>** indicates that the process is active.| +|dis|Disassembles a specified function or address. Example: `dis -l [func]`| +|mount|Displays information about the current file system.| diff --git a/docs/en/server/maintenance/common_tools/images/c50cb9df64f4659787c810167c89feb4_1884x257.png b/docs/en/server/maintenance/common_tools/images/c50cb9df64f4659787c810167c89feb4_1884x257.png new file mode 100644 index 0000000000000000000000000000000000000000..01081f25627731c56764c196e3fae32d55bc7023 Binary files /dev/null and b/docs/en/server/maintenance/common_tools/images/c50cb9df64f4659787c810167c89feb4_1884x257.png differ diff --git a/docs/en/server/maintenance/common_tools/images/en-us_image_0000001321685172.png b/docs/en/server/maintenance/common_tools/images/en-us_image_0000001321685172.png new file mode 100644 index 0000000000000000000000000000000000000000..a98265bdf251608c0ff394fefe545cd3192bdb28 Binary files /dev/null and b/docs/en/server/maintenance/common_tools/images/en-us_image_0000001321685172.png differ diff --git a/docs/en/server/maintenance/common_tools/images/en-us_image_0000001322112990.png b/docs/en/server/maintenance/common_tools/images/en-us_image_0000001322112990.png new file mode 100644 index 0000000000000000000000000000000000000000..6f4b32bf2b36595abe10f2550cda5714bc355553 Binary files /dev/null and b/docs/en/server/maintenance/common_tools/images/en-us_image_0000001322112990.png differ diff --git a/docs/en/server/maintenance/common_tools/images/en-us_image_0000001322219840.png b/docs/en/server/maintenance/common_tools/images/en-us_image_0000001322219840.png new file mode 100644 index 0000000000000000000000000000000000000000..48b28664df46ddf9aa38c7570bb9e9edb8080ac9 Binary files /dev/null and b/docs/en/server/maintenance/common_tools/images/en-us_image_0000001322219840.png differ diff --git a/docs/en/server/maintenance/common_tools/images/en-us_image_0000001322372918.png b/docs/en/server/maintenance/common_tools/images/en-us_image_0000001322372918.png new file mode 100644 index 0000000000000000000000000000000000000000..5424367c9bc564e713220ba87f963096881833b8 Binary files /dev/null and b/docs/en/server/maintenance/common_tools/images/en-us_image_0000001322372918.png differ diff --git a/docs/en/server/maintenance/common_tools/images/en-us_image_0000001322379488.png b/docs/en/server/maintenance/common_tools/images/en-us_image_0000001322379488.png new file mode 100644 index 0000000000000000000000000000000000000000..8b18cdca066be43b74443498edc5500ea9e1e608 Binary files /dev/null and b/docs/en/server/maintenance/common_tools/images/en-us_image_0000001322379488.png differ diff --git a/docs/en/server/maintenance/common_tools/images/en-us_image_0000001372748125.png b/docs/en/server/maintenance/common_tools/images/en-us_image_0000001372748125.png new file mode 100644 index 0000000000000000000000000000000000000000..5f6326b9415cf766dd8379dbadd5aa1a0dc6861f Binary files /dev/null and b/docs/en/server/maintenance/common_tools/images/en-us_image_0000001372748125.png differ diff --git a/docs/en/server/maintenance/common_tools/images/en-us_image_0000001372821865.png b/docs/en/server/maintenance/common_tools/images/en-us_image_0000001372821865.png new file mode 100644 index 0000000000000000000000000000000000000000..21e8dad1cd90755440cf858523b12c036a91e1ad Binary files /dev/null and b/docs/en/server/maintenance/common_tools/images/en-us_image_0000001372821865.png differ diff --git a/docs/en/server/maintenance/common_tools/images/en-us_image_0000001372824637.png b/docs/en/server/maintenance/common_tools/images/en-us_image_0000001372824637.png new file mode 100644 index 0000000000000000000000000000000000000000..aefb5d83c079e6718ef88fd934b4b496cdc29565 Binary files /dev/null and b/docs/en/server/maintenance/common_tools/images/en-us_image_0000001372824637.png differ diff --git a/docs/en/server/maintenance/common_tools/images/en-us_image_0000001373373585.png b/docs/en/server/maintenance/common_tools/images/en-us_image_0000001373373585.png new file mode 100644 index 0000000000000000000000000000000000000000..c4e5e47c9beca2c7c7630d78916f80eda652b52a Binary files /dev/null and b/docs/en/server/maintenance/common_tools/images/en-us_image_0000001373373585.png differ diff --git a/docs/en/server/maintenance/common_tools/images/en-us_image_0000001373379529.png b/docs/en/server/maintenance/common_tools/images/en-us_image_0000001373379529.png new file mode 100644 index 0000000000000000000000000000000000000000..daa40b49e679668905632f25ff42bf8599ba0ead Binary files /dev/null and b/docs/en/server/maintenance/common_tools/images/en-us_image_0000001373379529.png differ diff --git a/docs/en/server/maintenance/gala/_toc.yaml b/docs/en/server/maintenance/gala/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..4ed8d0c2b4b728f6f1a4430946d6312c74e02831 --- /dev/null +++ b/docs/en/server/maintenance/gala/_toc.yaml @@ -0,0 +1,12 @@ +label: gala-anteater User Guide +isManual: true +description: >- + Smart fault detection, performance data gathering and analysis, and resource + monitoring and management +sections: + - label: gala-anteater User Guide + href: ./using-gala-anteater.md + - label: gala-gopher User Guide + href: ./using-gala-gopher.md + - label: gala-spider User Guide + href: ./using-gala-spider.md diff --git a/docs/en/server/maintenance/gala/figures/attach-process.png b/docs/en/server/maintenance/gala/figures/attach-process.png new file mode 100644 index 0000000000000000000000000000000000000000..f76e8f4513cb45fbece12e6237039c41786b0467 Binary files /dev/null and b/docs/en/server/maintenance/gala/figures/attach-process.png differ diff --git a/docs/en/server/maintenance/gala/figures/deadlock.png b/docs/en/server/maintenance/gala/figures/deadlock.png new file mode 100644 index 0000000000000000000000000000000000000000..d4f863a1a87d7aad3128481c763ee715aefd0a9f Binary files /dev/null and b/docs/en/server/maintenance/gala/figures/deadlock.png differ diff --git a/docs/en/server/maintenance/gala/figures/deadlock2.png b/docs/en/server/maintenance/gala/figures/deadlock2.png new file mode 100644 index 0000000000000000000000000000000000000000..3be42a5a34f90c2f3b351c7077635c580ea847a7 Binary files /dev/null and b/docs/en/server/maintenance/gala/figures/deadlock2.png differ diff --git a/docs/en/server/maintenance/gala/figures/deadlock3.png b/docs/en/server/maintenance/gala/figures/deadlock3.png new file mode 100644 index 0000000000000000000000000000000000000000..5ef1a08394daf6433e10f85a5b3c57df25c3e303 Binary files /dev/null and b/docs/en/server/maintenance/gala/figures/deadlock3.png differ diff --git a/docs/en/server/maintenance/gala/figures/flame_muti_ins.png b/docs/en/server/maintenance/gala/figures/flame_muti_ins.png new file mode 100644 index 0000000000000000000000000000000000000000..5943c7fda223a7fde4d2987ad56af4ffa776bd81 Binary files /dev/null and b/docs/en/server/maintenance/gala/figures/flame_muti_ins.png differ diff --git a/docs/en/server/maintenance/gala/figures/gala-gopher-start-success.png b/docs/en/server/maintenance/gala/figures/gala-gopher-start-success.png new file mode 100644 index 0000000000000000000000000000000000000000..ab16e9d3661db3fd4adc6c605b2d2d08e79fdc1c Binary files /dev/null and b/docs/en/server/maintenance/gala/figures/gala-gopher-start-success.png differ diff --git a/docs/en/server/maintenance/gala/figures/gala-spider-arch.png b/docs/en/server/maintenance/gala/figures/gala-spider-arch.png new file mode 100644 index 0000000000000000000000000000000000000000..c5a0768be63a98ef7ccc4a56996a8c715f7090af Binary files /dev/null and b/docs/en/server/maintenance/gala/figures/gala-spider-arch.png differ diff --git a/docs/en/server/maintenance/gala/figures/gopher-arch.png b/docs/en/server/maintenance/gala/figures/gopher-arch.png new file mode 100644 index 0000000000000000000000000000000000000000..f151965a21d11dd7a3e215cc4ef23d70d059f4b1 Binary files /dev/null and b/docs/en/server/maintenance/gala/figures/gopher-arch.png differ diff --git a/docs/en/server/maintenance/gala/figures/lockcompete1.png b/docs/en/server/maintenance/gala/figures/lockcompete1.png new file mode 100644 index 0000000000000000000000000000000000000000..5848b114e02d09f23303da8cff7aef56216f655f Binary files /dev/null and b/docs/en/server/maintenance/gala/figures/lockcompete1.png differ diff --git a/docs/en/server/maintenance/gala/figures/lockcompete2.png b/docs/en/server/maintenance/gala/figures/lockcompete2.png new file mode 100644 index 0000000000000000000000000000000000000000..ed02a882a145dafeafb76469f328085edecc6775 Binary files /dev/null and b/docs/en/server/maintenance/gala/figures/lockcompete2.png differ diff --git a/docs/en/server/maintenance/gala/figures/lockcompete3.png b/docs/en/server/maintenance/gala/figures/lockcompete3.png new file mode 100644 index 0000000000000000000000000000000000000000..3992edc5b7ea61d8a2aa08ce47f0876b7d2e8cf3 Binary files /dev/null and b/docs/en/server/maintenance/gala/figures/lockcompete3.png differ diff --git a/docs/en/server/maintenance/gala/figures/lockcompete4.png b/docs/en/server/maintenance/gala/figures/lockcompete4.png new file mode 100644 index 0000000000000000000000000000000000000000..049ac49bcc1fb71ea9fe6866bd27e84d0acf42b1 Binary files /dev/null and b/docs/en/server/maintenance/gala/figures/lockcompete4.png differ diff --git a/docs/en/server/maintenance/gala/figures/lockcompete5.png b/docs/en/server/maintenance/gala/figures/lockcompete5.png new file mode 100644 index 0000000000000000000000000000000000000000..8b5cf5aaef43f125abdf3adb8a7f798dd2c86b54 Binary files /dev/null and b/docs/en/server/maintenance/gala/figures/lockcompete5.png differ diff --git a/docs/en/server/maintenance/gala/figures/lockcompete6.png b/docs/en/server/maintenance/gala/figures/lockcompete6.png new file mode 100644 index 0000000000000000000000000000000000000000..c3b1f5f097b9e9bcabf75229eabc6ce8fe126a71 Binary files /dev/null and b/docs/en/server/maintenance/gala/figures/lockcompete6.png differ diff --git a/docs/en/server/maintenance/gala/figures/spider_topology.png b/docs/en/server/maintenance/gala/figures/spider_topology.png new file mode 100644 index 0000000000000000000000000000000000000000..5823a116f384801e1197350f151b4d04ef519ac4 Binary files /dev/null and b/docs/en/server/maintenance/gala/figures/spider_topology.png differ diff --git a/docs/en/server/maintenance/gala/figures/tprofiling-dashboard-detail.png b/docs/en/server/maintenance/gala/figures/tprofiling-dashboard-detail.png new file mode 100644 index 0000000000000000000000000000000000000000..2093808bc4e1654956f6143393757c1244f08f98 Binary files /dev/null and b/docs/en/server/maintenance/gala/figures/tprofiling-dashboard-detail.png differ diff --git a/docs/en/server/maintenance/gala/figures/tprofiling-dashboard.png b/docs/en/server/maintenance/gala/figures/tprofiling-dashboard.png new file mode 100644 index 0000000000000000000000000000000000000000..15f4917f5a0bfcf5dee1f8fe68e65635ffebd85e Binary files /dev/null and b/docs/en/server/maintenance/gala/figures/tprofiling-dashboard.png differ diff --git a/docs/en/server/maintenance/gala/figures/tprofiling-run-arch.png b/docs/en/server/maintenance/gala/figures/tprofiling-run-arch.png new file mode 100644 index 0000000000000000000000000000000000000000..0ad835125a5e7b7f66938543de1e1c9d53706ce4 Binary files /dev/null and b/docs/en/server/maintenance/gala/figures/tprofiling-run-arch.png differ diff --git a/docs/en/server/maintenance/gala/using-gala-anteater.md b/docs/en/server/maintenance/gala/using-gala-anteater.md new file mode 100644 index 0000000000000000000000000000000000000000..9cac21cbff25f2ec88eb9940eb3ff587638bfafc --- /dev/null +++ b/docs/en/server/maintenance/gala/using-gala-anteater.md @@ -0,0 +1,391 @@ +# Using gala-anteater + +gala-anteater is an AI-based operating system exception detection platform. It provides functions such as time series data preprocessing, exception detection, and exception reporting. Based on offline pre-training, online model incremental learning and model update, it can be well adapted to multi-dimensional and multi-modal data fault diagnosis. + +This chapter describes how to deploy and use the gala-anteater service. + +## Installation + +Mount the repositories. + +```conf +[everything] +name=everything +baseurl=http://121.36.84.172/dailybuild/EBS-openEuler-24.09/EBS-openEuler-24.09/everything/$basearch/ +enabled=1 +gpgcheck=0 +priority=1 + +[EPOL] +name=EPOL +baseurl=http://repo.openeuler.org/openEuler-22.03-LTS-SP4/EPOL/main/$basearch/ +enabled=1 +gpgcheck=0 +priority=1 +``` + +Install gala-anteater. + +```bash +yum install gala-anteater +``` + +## Configuration + +> Note: gala-anteater uses a configuration file (**/etc/gala-anteater/config/gala-anteater.yaml**) for its startup settings. + +### Configuration Parameters + +```yaml +Global: + data_source: "prometheus" + +Arangodb: + url: "http://localhost:8529" + db_name: "spider" + +Kafka:f + server: "192.168.122.100" + port: "9092" + model_topic: "gala_anteater_hybrid_model" + rca_topic: "gala_cause_inference" + meta_topic: "gala_gopher_metadata" + group_id: "gala_anteater_kafka" + # auth_type: plaintext/sasl_plaintext, please set "" for no auth + auth_type: "" + username: "" + password: "" + +Prometheus: + server: "localhost" + port: "9090" + steps: "5" + +Aom: + base_url: "" + project_id: "" + auth_type: "token" + auth_info: + iam_server: "" + iam_domain: "" + iam_user_name: "" + iam_password: "" + ssl_verify: 0 + +Schedule: + duration: 1 +``` + +| Parameter | Description | Default Value | +| ----------- | --------------------------------------------------------------------------------------------- | ---------------------------- | +| Global | | | +| data_source | Data source | "prometheus" | +| Arangodb | | | +| url | IP address of the ArangoDB graph database | "" | +| db_name | Name of the ArangoDB database | "spider" | +| Kafka | | | +| server | IP address of the Kafka server. Configure according to the installation node IP address. | | +| port | Port of the Kafka server (for example, 9092) | | +| model_topic | Topic for reporting fault detection results | "gala_anteater_hybrid_model" | +| rca_topic | Topic for reporting root cause analysis results | "gala_cause_inference" | +| meta_topic | Topic for gopher to collect metric data | "gala_gopher_metadata" | +| group_id | Kafka group ID | "gala_anteater_kafka" | +| Prometheus | | | +| server | IP address of the Prometheus server. Configure according to the installation node IP address. | | +| port | Port of the Prometheus server (for example, 9090) | | +| steps | Metric sampling interval | | +| Schedule | | | +| duration | Interval (in minutes) between anomaly detection model executions | 1 | + +## Start + +Start gala-anteater. + +```bash +systemctl start gala-anteater +``` + +### Fault Injection + +gala-anteater is a fault detection and root cause locating module. In the testing phase, you need to inject faults to construct fault scenarios. This allows gala-anteater to obtain information about faulty nodes and the root cause nodes of fault propagation. + +- Fault injection (for reference only) + + ```bash + chaosblade create disk burn --size 10 --read --write --path /var/lib/docker/overlay2/cf0a469be8a84cabe1d057216505f8d64735e9c63159e170743353a208f6c268/merged --timeout 120 + ``` + + ChaosBlade is a fault injection tool that can simulate various faults, including but not limited to drive faults, network faults, and I/O faults. + Note: Injecting different faults will cause corresponding fluctuations in related metrics monitored and reported to the Prometheus module by metric collectors (such as gala-gopher). These fluctuations will be visible in the Prometheus graph. + +### gala-anteater Service Status Query + +If the following information is displayed, the service is started successfully. The startup log is saved to the **logs/anteater.log** file in the current running directory. + +```log +2022-09-01 17:52:54,435 - root - INFO - Run gala_anteater main function... +2022-09-01 17:52:54,436 - root - INFO - Start to try updating global configurations by querying data from Kafka! +2022-09-01 17:52:54,994 - root - INFO - Loads metric and operators from file: xxx\metrics.csv +2022-09-01 17:52:54,997 - root - INFO - Loads metric and operators from file: xxx\metrics.csv +2022-09-01 17:52:54,998 - root - INFO - Start to re-train the model based on last day metrics dataset! +2022-09-01 17:52:54,998 - root - INFO - Get training data during 2022-08-31 17:52:00+08:00 to 2022-09-01 17:52:00+08:00! +2022-09-01 17:53:06,994 - root - INFO - Spends: 11.995422840118408 seconds to get unique machine_ids! +2022-09-01 17:53:06,995 - root - INFO - The number of unique machine ids is: 1! +2022-09-01 17:53:06,996 - root - INFO - Fetch metric values from machine: xxxx. +2022-09-01 17:53:38,385 - root - INFO - Spends: 31.3896164894104 seconds to get get all metric values! +2022-09-01 17:53:38,392 - root - INFO - The shape of training data: (17281, 136) +2022-09-01 17:53:38,444 - root - INFO - Start to execute vae model training... +2022-09-01 17:53:38,456 - root - INFO - Using cpu device +2022-09-01 17:53:38,658 - root - INFO - Epoch(s): 0 train Loss: 136.68 validate Loss: 117.00 +2022-09-01 17:53:38,852 - root - INFO - Epoch(s): 1 train Loss: 113.73 validate Loss: 110.05 +2022-09-01 17:53:39,044 - root - INFO - Epoch(s): 2 train Loss: 110.60 validate Loss: 108.76 +2022-09-01 17:53:39,235 - root - INFO - Epoch(s): 3 train Loss: 109.39 validate Loss: 106.93 +2022-09-01 17:53:39,419 - root - INFO - Epoch(s): 4 train Loss: 106.48 validate Loss: 103.37 +... +2022-09-01 17:53:57,744 - root - INFO - Epoch(s): 98 train Loss: 97.63 validate Loss: 96.76 +2022-09-01 17:53:57,945 - root - INFO - Epoch(s): 99 train Loss: 97.75 validate Loss: 96.58 +2022-09-01 17:53:57,969 - root - INFO - Schedule recurrent job with time interval 1 minute(s). +2022-09-01 17:53:57,973 - apscheduler.scheduler - INFO - Adding job tentatively -- it will be properly scheduled when the scheduler starts +2022-09-01 17:53:57,974 - apscheduler.scheduler - INFO - Added job "partial" to job store "default" +2022-09-01 17:53:57,974 - apscheduler.scheduler - INFO - Scheduler started +2022-09-01 17:53:57,975 - apscheduler.scheduler - DEBUG - Looking for jobs to run +2022-09-01 17:53:57,975 - apscheduler.scheduler - DEBUG - Next wakeup is due at 2022-09-01 17:54:57.973533+08:00 (in 59.998006 seconds) +``` + +## Output Data of Fault Detection + +If gala-anteater detects an exception, it sends the result to `model_topic` of Kafka. The output data format is as follows: + +```json +{ + "Timestamp":1659075600000, + "Attributes":{ + "entity_id":"xxxxxx_sli_1513_18", + "event_id":"1659075600000_1fd37742xxxx_sli_1513_18", + "event_type":"app" + }, + "Resource":{ + "anomaly_score":1.0, + "anomaly_count":13, + "total_count":13, + "duration":60, + "anomaly_ratio":1.0, + "metric_label":{ + "machine_id":"1fd37742xxxx", + "tgid":"1513", + "conn_fd":"18" + }, + "recommend_metrics":{ + "gala_gopher_tcp_link_notack_bytes":{ + "label":{ + "__name__":"gala_gopher_tcp_link_notack_bytes", + "client_ip":"x.x.x.165", + "client_port":"51352", + "hostname":"localhost.localdomain", + "instance":"x.x.x.172:8888", + "job":"prometheus-x.x.x.172", + "machine_id":"xxxxxx", + "protocol":"2", + "role":"0", + "server_ip":"x.x.x.172", + "server_port":"8888", + "tgid":"3381701" + }, + "score":0.24421279500639545 + }, + ... + }, + "metrics":"gala_gopher_ksliprobe_recent_rtt_nsec" + }, + "SeverityText":"WARN", + "SeverityNumber":14, + "Body":"TimeStamp, WARN, APP may be impacting sli performance issues." +} +``` + +## Output Data of Root Cause Locating + +Each faulty node detected triggers root cause locating. Results of root cause locating are sent to `rca_topic` of Kafka. The output data format is as follows: + +```yaml +{ + "Timestamp": 1724287883452, + "event_id": "1721125159975_475ae627-7e88-41ed-8bb8-ff0fee95a69d_l7_3459438_192.168.11.103_192.168.11.102_26_tcp_server_server_http", + "Attributes": { + "event_id": "1721125159975_475ae627-7e88-41ed-8bb8-ff0fee95a69d_l7_3459438_192.168.11.103_192.168.11.102_26_tcp_server_server_http", + "event_source": "root-cause-inference" + }, + "Resource": { + "abnormal_kpi": { + "metric_id": "gala_gopher_l7_latency_sum", + "entity_id": "", + "metric_labels": { + "client_ip": "192.168.11.103", + "comm": "python", + "container_id": "83d0c2f4a7f4", + "container_image": "ba2d060a624e", + "container_name": "/k8s_backend_backend-node2-01-5bcb47fd7c-4jxxs_default_475ae627", + "instance": "192.168.122.102:8888", + "job": "192.168.122.102", + "l4_role": "tcp_server", + "l7_role": "server", + "machine_id": "66086618-3bad-489e-b17d-05245224f29a-192.168.122.102", + "pod": "default/backend-node2-01-5bcb47fd7c-4jxxs", + "pod_id": "475ae627-7e88-41ed-8bb8-ff0fee95a69d", + "pod_namespace": "default", + "protocol": "http", + "server_ip": "192.168.11.102", + "server_port": "26", + "ssl": "no_ssl", + "tgid": "3459438" + }, + "desc": "L7 session averaged latency.", + "score": 0.3498585816683402 + }, + "cause_metrics": [ + { + "metric_id": "gala_gopher_container_cpu_user_seconds_total@4a9fcc23-8ba2-4b0a-bcb0-b1bfd89ed929", + "entity_id": "", + "metric_labels": { + "container_id": "1319ff912a6f", + "container_image": "ba2d060a624e", + "container_name": "/k8s_backend_backend-node3-02-654dd97bf9-s8jg5_default_4a9fcc23", + "instance": "192.168.122.103:8888", + "job": "192.168.122.103", + "machine_id": "494a61be-23cc-4c97-a871-902866e43747-192.168.122.103", + "pod": "default/backend-node3-02-654dd97bf9-s8jg5", + "pod_id": "4a9fcc23-8ba2-4b0a-bcb0-b1bfd89ed929", + "pod_namespace": "default" + }, + "desc": "\u5bb9\u56681s\u5185\u7528\u6237\u6001CPU\u8d1f\u8f7d", + "keyword": "process", + "score": 0.1194249668036936, + "path": [ + { + "pod_id": "4a9fcc23-8ba2-4b0a-bcb0-b1bfd89ed929", + "pod": "default/backend-node3-02-654dd97bf9-s8jg5", + "instance": "192.168.122.103:8888", + "job": "192.168.122.103", + "pod_state": "normal" + }, + { + "pod_id": "475ae627-7e88-41ed-8bb8-ff0fee95a69d", + "pod": "default/backend-node2-01-5bcb47fd7c-4jxxs", + "instance": "192.168.122.102:8888", + "job": "192.168.122.102", + "pod_state": "abnormal" + } + ] + }, + { + "metric_id": "gala_gopher_proc_wchar_bytes@67134fb4-b2a3-43c5-a5b3-b3b463ad7d43", + "entity_id": "", + "metric_labels": { + "cmdline": "python ./backend.py ", + "comm": "python", + "container_id": "de570c7328bb", + "container_image": "ba2d060a624e", + "container_name": "/k8s_backend_backend-node2-02-548c79d989-bnl9g_default_67134fb4", + "instance": "192.168.122.102:8888", + "job": "192.168.122.102", + "machine_id": "66086618-3bad-489e-b17d-05245224f29a-192.168.122.102", + "pgid": "3459969", + "pod": "default/backend-node2-02-548c79d989-bnl9g", + "pod_id": "67134fb4-b2a3-43c5-a5b3-b3b463ad7d43", + "pod_namespace": "default", + "ppid": "3459936", + "start_time": "1139543501", + "tgid": "3459969" + }, + "desc": "\u8fdb\u7a0b\u7cfb\u7edf\u8c03\u7528\u81f3FS\u7684\u5199\u5b57\u8282\u6570", + "keyword": "process", + "score": 0.37121879175399997, + "path": [ + { + "pod_id": "67134fb4-b2a3-43c5-a5b3-b3b463ad7d43", + "pod": "default/backend-node2-02-548c79d989-bnl9g", + "instance": "192.168.122.102:8888", + "job": "192.168.122.102", + "pod_state": "normal" + }, + { + "pod_id": "4a9fcc23-8ba2-4b0a-bcb0-b1bfd89ed929", + "pod": "default/backend-node3-02-654dd97bf9-s8jg5", + "instance": "192.168.122.103:8888", + "job": "192.168.122.103", + "pod_state": "normal" + }, + { + "pod_id": "475ae627-7e88-41ed-8bb8-ff0fee95a69d", + "pod": "default/backend-node2-01-5bcb47fd7c-4jxxs", + "instance": "192.168.122.102:8888", + "job": "192.168.122.102", + "pod_state": "abnormal" + } + ] + }, + { + "metric_id": "gala_gopher_l7_latency_avg@956c70a2-9918-459c-a0a8-39396251f952", + "entity_id": "", + "metric_labels": { + "client_ip": "192.168.11.103", + "comm": "python", + "container_id": "eef1ca1082a7", + "container_image": "ba2d060a624e", + "container_name": "/k8s_backend_backend-node2-03-584f4c6cfd-w4d2b_default_956c70a2", + "instance": "192.168.122.102:8888", + "job": "192.168.122.102", + "l4_role": "tcp_server", + "l7_role": "server", + "machine_id": "66086618-3bad-489e-b17d-05245224f29a-192.168.122.102", + "pod": "default/backend-node2-03-584f4c6cfd-w4d2b", + "pod_id": "956c70a2-9918-459c-a0a8-39396251f952", + "pod_namespace": "default", + "protocol": "http", + "server_ip": "192.168.11.113", + "server_port": "26", + "ssl": "no_ssl", + "tgid": "3460169" + }, + "desc": "L7 session averaged latency.", + "keyword": null, + "score": 0.5624857367147617, + "path": [ + { + "pod_id": "956c70a2-9918-459c-a0a8-39396251f952", + "pod": "default/backend-node2-03-584f4c6cfd-w4d2b", + "instance": "192.168.122.102:8888", + "job": "192.168.122.102", + "pod_state": "abnormal" + }, + { + "pod_id": "4a9fcc23-8ba2-4b0a-bcb0-b1bfd89ed929", + "pod": "default/backend-node3-02-654dd97bf9-s8jg5", + "instance": "192.168.122.103:8888", + "job": "192.168.122.103", + "pod_state": "normal" + }, + { + "pod_id": "475ae627-7e88-41ed-8bb8-ff0fee95a69d", + "pod": "default/backend-node2-01-5bcb47fd7c-4jxxs", + "instance": "192.168.122.102:8888", + "job": "192.168.122.102", + "pod_state": "abnormal" + } + ] + } + ] + }, + "desc": "L7 session averaged latency.", + "top1": "gala_gopher_container_cpu_user_seconds_total@4a9fcc23-8ba2-4b0a-bcb0-b1bfd89ed929\u5f02\u5e38", + "top2": "gala_gopher_proc_wchar_bytes@67134fb4-b2a3-43c5-a5b3-b3b463ad7d43\u5f02\u5e38", + "top3": "gala_gopher_l7_latency_avg@956c70a2-9918-459c-a0a8-39396251f952\u5f02\u5e38", + "keywords": [ + "process", + null + ], + "SeverityText": "WARN", + "SeverityNumber": 13, + "Body": "A cause inferring event for an abnormal event" +} +``` diff --git a/docs/en/server/maintenance/gala/using-gala-gopher.md b/docs/en/server/maintenance/gala/using-gala-gopher.md new file mode 100644 index 0000000000000000000000000000000000000000..9ac9624058d07d5c88528059a876ea5a6cc2c1b4 --- /dev/null +++ b/docs/en/server/maintenance/gala/using-gala-gopher.md @@ -0,0 +1,1119 @@ +# Using gala-gopher + +As a data collection module, gala-gopher provides OS-level monitoring capabilities, supports dynamic probe installation and uninstallation, and integrates third-party probes in a non-intrusive manner to quickly expand the monitoring scope. + +This chapter describes how to deploy and use the gala-gopher service. + +# Installation + +Mount the repositories. + +```basic +[oe-2209] # openEuler 23.09 officially released repository +name=oe2209 +baseurl=http://119.3.219.20:82/openEuler:/23.09/standard_x86_64 +enabled=1 +gpgcheck=0 +priority=1 + +[oe-2209:Epol] # openEuler 23.09: Epol officially released repository +name=oe2209_epol +baseurl=http://119.3.219.20:82/openEuler:/23.09:/Epol/standard_x86_64/ +enabled=1 +gpgcheck=0 +priority=1 +``` + +Install gala-gopher. + +```bash +# yum install gala-gopher +``` + +# Configuration + +## Configuration Description + +The configuration file of gala-gopher is **/opt/gala-gopher/gala-gopher.conf**. The configuration items in the file are described as follows (the parts that do not need to be manually configured are not described): + +The following configurations can be modified as required: + +- `global`: global configuration for gala-gopher. + - `log_file_name`: name of the gala-gopher log file. + - `log_level`: gala-gopher log level (currently not enabled). + - `pin_path`: path for storing the map shared by the eBPF probe (keep the default configuration). +- `metric`: configuration for metric data output. + - `out_channel`: output channel for metrics (`web_server`, `logs`, or `kafka`). If empty, the output channel is disabled. + - `kafka_topic`: topic configuration for Kafka output. +- `event`: configuration for abnormal event output. + - `out_channel`: output channel for events (`logs` or `kafka`). If empty, the output channel is disabled. + - `kafka_topic`: topic configuration for Kafka output. + - `timeout`: reporting interval for the same event. + - `desc_language`: language for event descriptions (`zh_CN` or `en_US`). +- `meta`: configuration for metadata output. + - `out_channel`: output channel for metadata (`logs` or `kafka`). If empty, the output channel is disabled. + - `kafka_topic`: topic configuration for Kafka output. +- `ingress`: probe data reporting configuration (currently unused). + - `interval`: unused. +- `egress`: database reporting configuration (currently unused). + - `interval`: unused. + - `time_range`: unused. +- `imdb`: cache configuration. + - `max_tables_num`: maximum number of cache tables. Each meta file in **/opt/gala-gopher/meta** corresponds to a table. + - `max_records_num`: maximum records per cache table. Each probe typically generates at least one record per observation period. + - `max_metrics_num`: maximum number of metrics per record. + - `record_timeout`: cache table aging time (seconds). Records not updated within this time are deleted. +- `web_server`: `web_server` output channel configuration. + - `port`: listening port. +- `rest_api_server`: + - `port`: listening port for the REST API. + - `ssl_auth`: enables HTTPS encryption and authentication for the REST API (`on` or `off`). Enable in production. + - `private_key`: absolute path to the server's private key file for HTTPS encryption (required if `ssl_auth` is `on`). + - `cert_file`: absolute path to the server's certificate file for HTTPS encryption (required if `ssl_auth` is `on`). + - `ca_file`: absolute path to the CA certificate for client authentication (required if `ssl_auth` is `on`). +- `kafka`: Kafka output channel configuration. + - `kafka_broker`: IP address and port of the Kafka server. + - `batch_num_messages`: number of messages per batch. + - `compression_codec`: message compression type. + - `queue_buffering_max_messages`: maximum number of messages in the producer buffer. + - `queue_buffering_max_kbytes`: maximum size (KB) of the producer buffer. + - `queue_buffering_max_ms`: maximum time (ms) the producer waits for more messages before sending a batch. +- `logs`: `logs` output channel configuration. + - `metric_dir`: path for metric data logs. + - `event_dir`: path for abnormal event logs. + - `meta_dir`: path for metadata logs. + - `debug_dir`: path for gala-gopher runtime logs. + +## Configuration File Example + +- Select the data output channels. + + ```yaml + metric = + { + out_channel = "web_server"; + kafka_topic = "gala_gopher"; + }; + + event = + { + out_channel = "kafka"; + kafka_topic = "gala_gopher_event"; + }; + + meta = + { + out_channel = "kafka"; + kafka_topic = "gala_gopher_metadata"; + }; + ``` + +- Configure Kafka and Web Server. + + ```yaml + web_server = + { + port = 8888; + }; + + kafka = + { + kafka_broker = ":9092"; + }; + ``` + +- Select the probe to be enabled. The following is an example. + + ```yaml + probes = + ( + { + name = "system_infos"; + param = "-t 5 -w /opt/gala-gopher/task_whitelist.conf -l warn -U 80"; + switch = "on"; + }, + ); + extend_probes = + ( + { + name = "tcp"; + command = "/opt/gala-gopher/extend_probes/tcpprobe"; + param = "-l warn -c 1 -P 7"; + switch = "on"; + } + ); + ``` + +# Start + +After the configuration is complete, start gala-gopher. + +```bash +# systemctl start gala-gopher.service +``` + +Query the status of the gala-gopher service. + +```bash +# systemctl status gala-gopher.service +``` + +If the following information is displayed, the service is started successfully: Check whether the enabled probe is started. If the probe thread does not exist, check the configuration file and gala-gopher run log file. + +![](./figures/gala-gopher-start-success.png) + +> Note: The root permission is required for deploying and running gala-gopher. + +# How to Use + +## Deployment of External Dependent Software + +![](./figures/gopher-arch.png) + +As shown in the preceding figure, the green parts are external dependent components of gala-gopher. gala-gopher outputs metric data to Prometheus, metadata and abnormal events to Kafka. gala-anteater and gala-spider in gray rectangles obtain data from Prometheus and Kafka. + +> Note: Obtain the installation packages of Kafka and Prometheus from the official websites. + +### REST Dynamic Configuration Interface + +The web server port is configurable (default is 9999). The URL format is `http://[gala-gopher-node-ip-address]:[port]/[function (collection feature)]`. For example, the URL for the flamegraph is `http://localhost:9999/flamegraph` (the following documentation uses the flamegraph as an example). + +#### Configuring the Probe Monitoring Scope + +Probes are disabled by default and can be dynamically enabled and configured via the API. Taking the flamegraph as an example, the REST API can be used to enable `oncpu`, `offcpu`, and `mem` flamegraph capabilities. The monitoring scope can be configured based on four dimensions: process ID, process name, container ID, and pod. + +Below is an example of an API that simultaneously enables the oncpu and offcpu collection features for the flamegraph: + +```sh +curl -X PUT http://localhost:9999/flamegraph --data-urlencode json=' +{ + "cmd": { + "bin": "/opt/gala-gopher/extend_probes/stackprobe", + "check_cmd": "", + "probe": [ + "oncpu", + "offcpu" + ] + }, + "snoopers": { + "proc_id": [ + 101, + 102 + ], + "proc_name": [ + { + "comm": "app1", + "cmdline": "", + "debugging_dir": "" + }, + { + "comm": "app2", + "cmdline": "", + "debugging_dir": "" + } + ], + "pod_id": [ + "pod1", + "pod2" + ], + "container_id": [ + "container1", + "container2" + ] + } +}' +``` + +A full description of the collection features is provided below: + +| Collection Feature | Description | Sub-item Scope | Monitoring Targets | Startup File | Startup Condition | +| ------------------ | -------------------------------------------------- | ----------------------------------------------------------------------------------------- | ---------------------------------------- | ---------------------------------- | ------------------------- | +| flamegraph | Online performance flamegraph observation | oncpu, offcpu, mem | proc_id, proc_name, pod_id, container_id | $gala-gopher-dir/stackprobe | NA | +| l7 | Application layer 7 protocol observation | l7_bytes_metrics, l7_rpc_metrics, l7_rpc_trace | proc_id, proc_name, pod_id, container_id | $gala-gopher-dir/l7probe | NA | +| tcp | TCP exception and state observation | tcp_abnormal, tcp_rtt, tcp_windows, tcp_rate, tcp_srtt, tcp_sockbuf, tcp_stats, tcp_delay | proc_id, proc_name, pod_id, container_id | $gala-gopher-dir/tcpprobe | NA | +| socket | Socket (TCP/UDP) exception observation | tcp_socket, udp_socket | proc_id, proc_name, pod_id, container_id | $gala-gopher-dir/endpoint | NA | +| io | Block layer I/O observation | io_trace, io_err, io_count, page_cache | NA | $gala-gopher-dir/ioprobe | NA | +| proc | Process system calls, I/O, DNS, VFS observation | base_metrics, proc_syscall, proc_fs, proc_io, proc_dns, proc_pagecache | proc_id, proc_name, pod_id, container_id | $gala-gopher-dir/taskprobe | NA | +| jvm | JVM layer GC, threads, memory, cache observation | NA | proc_id, proc_name, pod_id, container_id | $gala-gopher-dir/jvmprobe | NA | +| ksli | Redis performance SLI (access latency) observation | NA | proc_id, proc_name, pod_id, container_id | $gala-gopher-dir/ksliprobe | NA | +| postgre_sli | PG DB performance SLI (access latency) observation | NA | proc_id, proc_name, pod_id, container_id | $gala-gopher-dir/pgsliprobe | NA | +| opengauss_sli | openGauss access throughput observation | NA | \[ip, port, dbname, user, password] | $gala-gopher-dir/pg_stat_probe.py | NA | +| dnsmasq | DNS session observation | NA | proc_id, proc_name, pod_id, container_id | $gala-gopher-dir/rabbitmq_probe.sh | NA | +| lvs | LVS session observation | NA | NA | $gala-gopher-dir/trace_lvs | lsmod\|grep ip_vs\| wc -l | +| nginx | Nginx L4/L7 layer session observation | NA | proc_id, proc_name, pod_id, container_id | $gala-gopher-dir/nginx_probe | NA | +| haproxy | Haproxy L4/7 layer session observation | NA | proc_id, proc_name, pod_id, container_id | $gala-gopher-dir/trace_haproxy | NA | +| kafka | Kafka producer/consumer topic observation | NA | dev, port | $gala-gopher-dir/kafkaprobe | NA | +| baseinfo | System basic information | cpu, mem, nic, disk, net, fs, proc, host | proc_id, proc_name, pod_id, container_id | system_infos | NA | +| virt | Virtualization management information | NA | NA | virtualized_infos | NA | +| tprofiling | Thread-level performance profiling observation | oncpu, syscall_file, syscall_net, syscall_lock, syscall_sched | proc_id, proc_name | | | + +### Configuring Probe Runtime Parameters + +Probes require additional parameter settings during runtime, such as configuring the sampling period and reporting period for flamegraphs. + +```sh +curl -X PUT http://localhost:9999/flamegraph --data-urlencode json=' +{ + "params": { + "report_period": 180, + "sample_period": 180, + "metrics_type": [ + "raw", + "telemetry" + ] + } +}' +``` + +Detailed runtime parameters are as follows: + +| Parameter | Description | Default & Range | Unit | Supported Monitoring Scope | Supported by gala-gopher | +| ------------------- | ---------------------------------------- | -------------------------------------------------------------- | ------- | ------------------------------------------- | ------------------------ | +| sample_period | Sampling period | 5000, \[100~10000] | ms | io, tcp | Y | +| report_period | Reporting period | 60, \[5~600] | s | ALL | Y | +| latency_thr | Latency reporting threshold | 0, \[10~100000] | ms | tcp, io, proc, ksli | Y | +| offline_thr | Process offline reporting threshold | 0, \[10~100000] | ms | proc | Y | +| drops_thr | Packet loss reporting threshold | 0, \[10~100000] | package | tcp, nic | Y | +| res_lower_thr | Resource percentage lower limit | 0%, \[0%~100%] | percent | ALL | Y | +| res_upper_thr | Resource percentage upper limit | 0%, \[0%~100%] | percent | ALL | Y | +| report_event | Report abnormal events | 0, \[0, 1] | NA | ALL | Y | +| metrics_type | Report telemetry metrics | raw, \[raw, telemetry] | NA | ALL | N | +| env | Working environment type | node, \[node, container, kubenet] | NA | ALL | N | +| report_source_port | Report source port | 0, \[0, 1] | NA | tcp | Y | +| l7_protocol | Layer 7 protocol scope | http, \[http, pgsql, mysql, redis, kafka, mongo, rocketmq, dns] | NA | l7 | Y | +| support_ssl | Support SSL encrypted protocol observation | 0, \[0, 1] | NA | l7 | Y | +| multi_instance | Output separate flamegraphs for each process | 0, \[0, 1] | NA | flamegraph | Y | +| native_stack | Display native language stack (for JAVA processes) | 0, \[0, 1] | NA | flamegraph | Y | +| cluster_ip_backend | Perform Cluster IP backend conversion | 0, \[0, 1] | NA | tcp, l7 | Y | +| pyroscope_server | Set flamegraph UI server address | localhost:4040 | NA | flamegraph | Y | +| svg_period | Flamegraph SVG file generation period | 180, \[30, 600] | s | flamegraph | Y | +| perf_sample_period | Period for collecting stack info in oncpu flamegraph | 10, \[10, 1000] | ms | flamegraph | Y | +| svg_dir | Directory for storing flamegraph SVG files | "/var/log/gala-gopher/stacktrace" | NA | flamegraph | Y | +| flame_dir | Directory for storing raw stack info in flamegraphs | "/var/log/gala-gopher/flamegraph" | NA | flamegraph | Y | +| dev_name | Observed network card/disk device name | "" | NA | io, kafka, ksli, postgre_sli, baseinfo, tcp | Y | +| continuous_sampling | Enable continuous sampling | 0, \[0, 1] | NA | ksli | Y | +| elf_path | Path to the executable file to observe | "" | NA | nginx, haproxy, dnsmasq | Y | +| kafka_port | Kafka port number to observe | 9092, \[1, 65535] | NA | kafka | Y | +| cadvisor_port | Port number for starting cadvisor | 8080, \[1, 65535] | NA | cadvisor | Y | + +### Starting and Stopping Probes + +```sh +curl -X PUT http://localhost:9999/flamegraph --data-urlencode json=' +{ + "state": "running" // optional: running, stopped +}' +``` + +### Constraints and Limitations + +1. The interface is stateless. The settings uploaded each time represent the final runtime configuration for the probe, including state, parameters, and monitoring scope. +2. Monitoring targets can be combined arbitrarily, and the monitoring scope is the union of all specified targets. +3. The startup file must be valid and accessible. +4. Collection features can be enabled partially or fully as needed, but disabling a feature requires disabling it entirely. +5. The monitoring target for opengauss is a DB instance (IP/Port/dbname/user/password). +6. The interface can receive a maximum of 2048 characters per request. + +#### Querying Probe Configurations and Status + +```sh +curl -X GET http://localhost:9999/flamegraph +{ + "cmd": { + "bin": "/opt/gala-gopher/extend_probes/stackprobe", + "check_cmd": "" + "probe": [ + "oncpu", + "offcpu" + ] + }, + "snoopers": { + "proc_id": [ + 101, + 102 + ], + "proc_name": [ + { + "comm": "app1", + "cmdline": "", + "debugging_dir": "" + }, + { + "comm": "app2", + "cmdline": "", + "debugging_dir": "" + } + ], + "pod_id": [ + "pod1", + "pod2" + ], + "container_id": [ + "container1", + "container2" + ] + }, + "params": { + "report_period": 180, + "sample_period": 180, + "metrics_type": [ + "raw", + "telemetry" + ] + }, + "state": "running" +} +``` + +## Introduction to stackprobe + +A performance flamegraph tool designed for cloud-native environments. + +### Features + +- Supports observation of applications written in C/C++, Go, Rust, and Java. +- Call stack supports container and process granularity: For processes within containers, the workload Pod name and container name are marked with `[Pod]` and `[Con]` prefixes at the bottom of the call stack. Process names are prefixed with `[]`, while threads and functions (methods) have no prefix. +- Supports generating SVG format flamegraphs locally or uploading call stack data to middleware. +- Supports generating/uploading flamegraphs for multiple instances based on process granularity. +- For Java processes, flamegraphs can simultaneously display native methods and Java methods. +- Supports multiple types of flamegraphs, including oncpu, offcpu, and mem. +- Supports custom sampling periods. + +### Usage Instructions + +Basic startup command example: Start the performance flamegraph with default parameters. + +```sh +curl -X PUT http://localhost:9999/flamegraph -d json='{ "cmd": {"probe": ["oncpu"] }, "snoopers": {"proc_name": [{ "comm": "cadvisor"}] }, "state": "running"}' +``` + +Advanced startup command example: Start the performance flamegraph with custom parameters. For a complete list of configurable parameters, refer to [Configuring Probe Runtime Parameters](#configuring-probe-runtime-parameters). + +```sh +curl -X PUT http://localhost:9999/flamegraph -d json='{ "cmd": { "check_cmd": "", "probe": ["oncpu", "offcpu", "mem"] }, "snoopers": { "proc_name": [{ "comm": "cadvisor", "cmdline": "", "debugging_dir": "" }, { "comm": "java", "cmdline": "", "debugging_dir": "" }] }, "params": { "perf_sample_period": 100, "svg_period": 300, "svg_dir": "/var/log/gala-gopher/stacktrace", "flame_dir": "/var/log/gala-gopher/flamegraph", "pyroscope_server": "localhost:4040", "multi_instance": 1, "native_stack": 0 }, "state": "running"}' +``` + +Key configuration options explained: + +- **Enabling flamegraph types**: + + Set via the `probe` parameter. Values include `oncpu`, `offcpu`, and `mem`, representing CPU usage time, blocked time, and memory allocation statistics, respectively. + + Example: + + `"probe": ["oncpu", "offcpu", "mem"]` + +- **Setting the period for generating local SVG flamegraphs**: + + Configured via the `svg_period` parameter, in seconds. Default is 180, with an optional range of \[30, 600]. + + Example: + + `"svg_period": 300` + +- **Enabling/disabling stack information upload to Pyroscope**: + + Set via the `pyroscope_server` parameter. The value must include the address and port. If empty or incorrectly formatted, the probe will not attempt to upload stack information. The upload period is 30 seconds. + + Example: + + `"pyroscope_server": "localhost:4040"` + +- **Setting the call stack sampling period**: + + Configured via the `perf_sample_period` parameter, in milliseconds. Default is 10, with an optional range of \[10, 1000]. This parameter only applies to oncpu flamegraphs. + + Example: + + `"perf_sample_period": 100` + +- **Enabling/disabling multi-instance flamegraph generation**: + + Set via the `multi_instance` parameter, with values 0 or 1. Default is 0. A value of 0 merges flamegraphs for all processes, while 1 generates separate flamegraphs for each process. + + Example: + + `"multi_instance": 1` + +- **Enabling/disabling native call stack collection**: + + Set via the `native_stack` parameter, with values 0 or 1. Default is 0. This parameter only applies to Java processes. A value of 0 disables collection of the JVM's native call stack, while 1 enables it. + + Example: + + `"native_stack": 1` + + Visualization: (Left: `"native_stack": 1`, Right: `"native_stack": 0`) + + ![image-20230804172905729](./figures/flame_muti_ins.png) + +### Implementation Plan + +#### 1. User-Space Program Logic + +The program periodically (every 30 seconds) converts kernel-reported stack information from addresses to symbols using the symbol table. It then uses the flamegraph plugin or pyroscope to generate a flame graph from the symbolized call stack. + +The approach to obtaining the symbol table differs based on the code segment type. + +- Kernel Symbol Table: Access **/proc/kallsyms**. + +- Native Language Symbol Table: Query the process virtual memory mapping file (**/proc/{pid}/maps**) to retrieve address mappings for each code segment in the process memory. The libelf library is then used to load the symbol table of the corresponding module for each segment. + +- Java Language Symbol Table: + + Since Java methods are not statically mapped to the process virtual address space, alternative methods are used to obtain the symbolized Java call stack. + +##### Method 1: Perf Observation + +A JVM agent dynamic library is loaded into the Java process to monitor JVM method compilation and loading events. This allows real-time recording of memory address-to-Java symbol mappings, generating the Java process symbol table. This method requires the Java process to be launched with the `-XX:+PreserveFramePointer` option. Its key advantage is that the flame graph can display the JVM call stack, and the resulting Java flame graph can be merged with those of other processes for unified visualization. + +##### Method 2: JFR Observation + +The JVM built-in profiler, Java Flight Recorder (JFR), is dynamically enabled to monitor various events and metrics of the Java application. This is accomplished by loading a Java agent into the Java process, which internally calls the JFR API. This method offers the advantage of more precise and comprehensive collection of Java method call stacks. + +Both Java performance analysis methods can be loaded in real time (without restarting the Java process) and feature low overhead. When stackprobe startup parameters are configured as `"multi_instance": 1` and `"native_stack": 0`, it uses Method 2 to generate the Java process flame graph; otherwise, it defaults to Method 1. + +#### 2. Kernel-Space Program Logic + +The kernel-space functionality is implemented using eBPF. Different flame graph types correspond to distinct eBPF programs. These programs periodically or through event triggers traverse the current user-space and kernel-space call stacks, reporting the results to user space. + +##### 2.1 On-CPU Flame Graph + +A sampling eBPF program is attached to perf software event `PERF_COUNT_SW_CPU_CLOCK` to periodically sample the call stack. + +##### 2.2 Off-CPU Flame Graph + +A sampling eBPF program is attached to process scheduling tracepoint `sched_switch`. This program records the time and process ID when a process is scheduled out and samples the call stack when the process is scheduled back in. + +##### 2.3 Memory Flame Graph + +A sampling eBPF program is attached to page fault tracepoint `page_fault_user`. The call stack is sampled whenever this event is triggered. + +#### 3. Java Language Support + +- stackprobe main process: + + 1. Receives an IPC message to identify the Java process to be observed. + 2. Utilizes the Java agent loading module to inject the JVM agent program into the target Java process: `jvm_agent.so` (for [Method 1](#method-1-perf-observation)) or `JstackProbeAgent.jar` (for [Method 2](#method-2-jfr-observation)). + 3. For Method 1, the main process loads the `java-symbols.bin` file of the corresponding Java process to facilitate address-to-symbol conversion. For Method 2, it loads the `stacks-{flame_type}.txt` file of the corresponding Java process, which can be directly used to generate flame graphs. + +- Java agent loading module: + + 1. Detects a new Java process and copies the JVM agent program to `/proc//root/tmp` in the process space (to ensure visibility to the JVM inside the container during attachment). + 2. Adjusts the ownership of the directory and JVM agent program to match the observed Java process. + 3. Launches the `jvm_attach` subprocess and passes the relevant parameters of the observed Java process. + +- JVM agent program: + + - jvm_agent.so: Registers JVMTI callback functions. + + When the JVM loads a Java method or dynamically compiles a native method, it triggers the callback function. The callback records the Java class name, method name, and corresponding memory address in `/proc//root/tmp/java-data-/java-symbols.bin` within the observed Java process space. + - JstackProbeAgent.jar: Invokes the JFR API. + + Activates JFR for 30 seconds and transforms the JFR statistics into a stack format suitable for flame graphs. The output is saved to `/proc//root/tmp/java-data-/stacks-.txt` in the observed Java process space. For more information, refer to [JstackProbe Introduction](https://gitee.com/openeuler/gala-gopher/blob/dev/src/probes/extends/java.probe/jstack.probe/readme.md). + +- jvm_attach: Dynamically loads the JVM agent program into the JVM of the observed process (based on `sun.tools.attach.LinuxVirtualMachine` from the JDK source code and the `jattach` tool). + + 1. Configures its own namespace (the JVM requires the attaching process and the observed process to share the same namespace for agent loading). + 2. Verifies if the JVM attach listener is active (by checking for the existence of the UNIX socket file `/proc//root/tmp/.java_pid`). + 3. If inactive, creates `/proc//cwd/.attach_pid` and sends a SIGQUIT signal to the JVM. + 4. Establishes a connection to the UNIX socket. + 5. Interprets the response; a value of 0 indicates successful attachment. + + Attachment process diagram: + + ![Attachment process](./figures/attach-process.png) + +### Precautions + +- To achieve the best observation results for Java applications, configure the stackprobe startup options to `"multi_instance": 1` and `"native_stack": 0` to enable JFR observation (JDK8u262+). Otherwise, stackprobe will use perf to generate Java flame graphs. When using perf, ensure that the JVM option `XX:+PreserveFramePointer` is enabled (JDK8 or later). + +### Constraints + +- Supports observation of Java applications based on the hotSpot JVM. + +## Introduction to tprofiling + +tprofiling, a thread-level application performance diagnostic tool provided by gala-gopher, leverages eBPF technology. It monitors key system performance events at the thread level, associating them with detailed event content. This enables real-time recording of thread states and key activities, helping users quickly pinpoint application performance bottlenecks. + +### Features + +From the OS perspective, a running application comprises multiple processes, each containing multiple running threads. tprofiling monitors and records key activities (referred to as **events**) performed by these threads. The tool then presents these events on a timeline in the front-end interface, providing an intuitive view of what each thread is doing at any given moment—whether it is executing on the CPU or blocked by file or network I/O operations. When performance issues arise, analyzing the sequence of key performance events for a given thread enables rapid problem isolation and localization. + +Currently, with its implemented event monitoring capabilities, tprofiling can identify application performance issues such as: + +- File I/O latency and blocking +- Network I/O latency and blocking +- Lock contention +- Deadlocks + +As more event types are added and refined, tprofiling will cover a broader range of application performance problems. + +### Event Observation Scope + +tprofiling currently supports two main categories of system performance events: syscall events and on-CPU events. + +**Syscall Events** + +Application performance often suffers from system resource bottlenecks like excessive CPU usage or I/O wait times. Applications typically access these resources through syscalls. Observation key syscall events helps identify time-consuming or blocking resource access operations. + +The syscall events currently observed by tprofiling are detailed in the [Supported Syscall Events](#supported-system-call-events) section. These events fall into categories such as file operations, network operations, lock operations, and scheduling operations. Examples of observed syscall events include: + +- File Operations + - `read`/`write`: Reading from or writing to disk files or network connections; these operations can be time-consuming or blocking. + - `sync`/`fsync`: Synchronizing file data to disk, which blocks the thread until completion. +- Network Operations + - `send`/`recv`: Reading from or writing to network connections; these operations can be time-consuming or blocking. +- Lock Operations + - `futex`: A syscall related to user-mode lock implementations. A `futex` call often indicates lock contention, potentially causing threads to block. +- Scheduling Operations: These syscall events can change a thread's state, such as yielding the CPU, sleeping, or waiting for other threads. + - `nanosleep`: The thread enters a sleep state. + - `epoll_wait`: The thread waits for I/O events, blocking until an event arrives. + +**on-CPU Events** + +A thread's running state can be categorized as either on-CPU (executing on a CPU core) or off-CPU (not executing). Observation on-CPU events helps identify threads performing time-consuming CPU-bound operations. + +### Event Content + +Thread profiling events include the following information: + +- Event Source: This includes the thread ID, thread name, process ID, process name, container ID, container name, host ID, and host name associated with the event. + + - `thread.pid`: The thread ID. + - `thread.comm`: The thread name. + - `thread.tgid`: The process ID. + - `proc.name`: The process name. + - `container.id`: The container ID. + - `container.name`: The container name. + - `host.id`: The host ID. + - `host.name`: The host name. + +- Event Attributes: These include common attributes and extended attributes. + + - Common Attributes: These include the event name, event type, start time, end time, and duration. + + - `event.name`: The event name. + - `event.type`: The event type, which can be `oncpu`, `file`, `net`, `lock`, or `sched`. + - `start_time`: The event start time, which is the start time of the first event in an aggregated event. See [Aggregated Events](#aggregated-events) for more information. + - `end_time`: The event end time, which is the end time of the last event in an aggregated event. + - `duration`: The event duration, calculated as (`end_time` - `start_time`). + - `count`: The number of aggregated events. + + - Extended Attributes: These provide more detailed information specific to each syscall event. For example, `read` and `write` events for files or network connections include the file path, network connection details, and function call stack. + + - `func.stack`: The function call stack. + - `file.path`: The file path for file-related events. + - `sock.conn`: The TCP connection information for network-related events. + - `futex.op`: The `futex` operation type, which can be `wait` or `wake`. + + Refer to the [Supported Syscall Events](#supported-system-call-events) section for details on the extended attributes supported by each event type. + +### Event Output + +As an eBPF probe extension provided by gala-gopher, tprofiling sends generated system events to gala-gopher for processing. gala-gopher then outputs these events in the openTelemetry format and publishes them as JSON messages to a Kafka queue. Front-end applications can consume these tprofiling events by subscribing to the Kafka topic. + +Here's an example of a thread profiling event output: + +```json +{ + "Timestamp": 1661088145000, + "SeverityText": "INFO", + "SeverityNumber": 9, + "Body": "", + "Resource": { + "host.id": "", + "host.name": "", + "thread.pid": 10, + "thread.tgid": 10, + "thread.comm": "java", + "proc.name": "xxx.jar", + "container.id": "", + "container.name": "", + }, + "Attributes": { + values: [ + { + // common info + "event.name": "read", + "event.type": "file", + "start_time": 1661088145000, + "end_time": 1661088146000, + "duration": 0.1, + "count": 1, + // extend info + "func.stack": "read;", + "file.path": "/test.txt" + }, + { + "event.name": "oncpu", + "event.type": "oncpu", + "start_time": 1661088146000, + "end_time": 1661088147000, + "duration": 0.1, + "count": 1, + } + ] + } +} +``` + +Key fields: + +- `Timestamp`: The timestamp when the event was reported. +- `Resource`: Information about the event source. +- `Attributes`: Event attribute information, containing a `values` list. Each item in the list represents a tprofiling event from the same source and includes the event's attributes. + +### Quick Start + +#### Installation + +tprofiling is an eBPF probe extension for gala-gopher, so you must first install gala-gopher before enabling tprofiling. + +[gala-ops](https://gitee.com/openeuler/gala-docs) provides a demo UI for tprofiling based on Kafka, Logstash, Elasticsearch, and Grafana. You can use the gala-ops deployment tools for quick setup. + +#### Architecture + +![](./figures/tprofiling-run-arch.png) + +Software components: + +- Kafka: An open-source message queue that receives and stores tprofiling events collected by gala-gopher. +- Logstash: A real-time, open-source log collection engine that consumes tprofiling events from Kafka, processes them (filtering, transformation, etc.), and sends them to Elasticsearch. +- Elasticsearch: An open, distributed search and analytics engine that stores the processed tprofiling events for querying and visualization in Grafana. +- Grafana: An open-source visualization tool to query and visualize the collected tprofiling events. Users interact with tprofiling through the Grafana UI to analyze application performance. + +#### Deploying the tprofiling Probe + +First, install gala-gopher as described in the [gala-gopher documentation](https://gitee.com/openeuler/gala-gopher#快速开始). Because tprofiling events are sent to Kafka, configure the Kafka service address during deployment. + +After installing and running gala-gopher, start the tprofiling probe using gala-gopher's HTTP-based dynamic configuration API: + +```sh +curl -X PUT http://:9999/tprofiling -d json='{"cmd": {"probe": ["oncpu", "syscall_file", "syscall_net", "syscall_sched", "syscall_lock"]}, "snoopers": {"proc_name": [{"comm": "java"}]}, "state": "running"}' +``` + +Configuration parameters: + +- ``: The IP address of the node where gala-gopher is deployed. +- `probe`: Under `cmd`, the `probe` configuration specifies the system events that the tprofiling probe monitors. `oncpu`, `syscall_file`, `syscall_net`, `syscall_sched`, and `syscall_lock` correspond to on-CPU events and file, network, scheduling, and lock syscall events, respectively. You can enable only the desired tprofiling event types. +- `proc_name`: Under `snoopers`, the `proc_name` configuration filters the processes to monitor by process name. You can also filter by process ID using the `proc_id` configuration. See [REST Dynamic Configuration Interface](#rest-dynamic-configuration-interface) for details. + +To stop the tprofiling probe, run: + +```sh +curl -X PUT http://:9999/tprofiling -d json='{"state": "stopped"}' +``` + +#### Deploying the Front-End Software + +The tprofiling UI requires Kafka, Logstash, Elasticsearch, and Grafana. Install these components on a management node. You can use the gala-ops deployment tools for quick installation; see the [Online Deployment Documentation](https://gitee.com/openeuler/gala-docs#%E5%9C%A8%E7%BA%BF%E9%83%A8%E7%BD%B2). + +On the management node, obtain the deployment script from the [Online Deployment Documentation](https://gitee.com/openeuler/gala-docs#%E5%9C%A8%E7%BA%BF%E9%83%A8%E7%BD%B2) and run the following command to install Kafka, Logstash, and Elasticsearch with one command: + +```sh +sh deploy.sh middleware -K -E -A -p +``` + +Run the following command to install Grafana: + +```sh +sh deploy.sh grafana -P -E +``` + +#### Usage + +After completing the deployment, access A-Ops by browsing to `http://[deployment_node_management_IP_address]:3000` and logging into Grafana. The default username and password are both **admin**. + +After logging in, find the **ThreadProfiling** dashboard. + +![image-20230628155002410](./figures/tprofiling-dashboard.png) + +Click to enter the tprofiling UI and explore its features. + +![image-20230628155249009](./figures/tprofiling-dashboard-detail.png) + +### Use Cases + +#### Case 1: Deadlock Detection + +![image-20230628095802499](./figures/deadlock.png) + +The above diagram shows the thread profiling results of a deadlock demo process. The pie chart shows that `lock` events (in gray) consume a significant portion of the execution time. The lower section displays the thread profiling results for the entire process, with the vertical axis representing the sequence of profiling events for different threads. The `java` main thread remains blocked. The `LockThd1` and `LockThd2` service threads execute `oncpu` and `file` events, followed by simultaneous, long-duration `lock` events. Hovering over a `lock` event reveals that it triggers a `futex` syscall lasting 60 seconds. + +![image-20230628101056732](./figures/deadlock2.png) + +This suggests potential issues with `LockThd1` and `LockThd2`. We can examine their thread profiling results in the thread view. + +![image-20230628102138540](./figures/deadlock3.png) + +This view displays the profiling results for each thread, with the vertical axis showing the sequence of events. `LockThd1` and `LockThd2` normally execute `oncpu` events, including `file` and `lock` events, periodically. However, around 10:17:00, they both execute a long `futex` event without any intervening `oncpu` events, indicating a blocked state. `futex` is a syscall related to user-space lock implementation, and its invocation often signals lock contention and potential blocking. + +Based on this analysis, a deadlock likely exists between `LockThd1` and `LockThd2`. + +#### Case 2: Lock Contention Detection + +![image-20230628111119499](./figures/lockcompete1.png) + +The above diagram shows the thread profiling results for a lock contention demo process. The process primarily executes `lock`, `net`, and `oncpu` events, involving three service threads. Between 11:05:45 and 11:06:45, the event execution times for all three threads increase significantly, indicating a potential performance problem. We can examine each thread's profiling results in the thread view, focusing on this period. + +![image-20230628112709827](./figures/lockcompete2.png) + +By examining the event sequence for each thread, we can understand their activities: + +- Thread `CompeteThd1`: Periodically triggers short `oncpu` events, performing a calculation task. However, around 11:05:45, it begins triggering long `oncpu` events, indicating a time-consuming calculation. + + ![image-20230628113336435](./figures/lockcompete3.png) + +- Thread `CompeteThd2`: Periodically triggers short `net` events. Clicking on an event reveals that the thread is sending network messages via the `write` syscall, along with the TCP connection details. Similarly, around 11:05:45, it starts executing long `futex` events and becomes blocked, increasing the interval between `write` events. + + ![image-20230628113759887](./figures/lockcompete4.png) + + ![image-20230628114340386](./figures/lockcompete5.png) + +- Thread `tcp-server`: A TCP server that continuously reads client requests via the `read` syscall. Starting around 11:05:45, the `read` event execution time increases, indicating that it is waiting to receive network requests. + + ![image-20230628114659071](./figures/lockcompete6.png) + +Based on this analysis, whenever `CompeteThd1` performs a long `oncpu` operation, `CompeteThd2` calls `futex` and enters a blocked state. Once `CompeteThd1` completes the `oncpu` operation, `CompeteThd2` acquires the CPU and performs the network `write` operation. This strongly suggests lock contention between `CompeteThd1` and `CompeteThd2`. Because `CompeteThd2` is waiting for a lock and cannot send network requests, the `tcp-server` thread spends most of its time waiting for `read` requests. + +### Topics + +#### Supported System Call Events + +When selecting system call events for monitoring, consider these principles: + +1. Choose potentially time-consuming or blocking events, such as file, network, or lock operations, as they involve system resource access. +2. Choose events that affect a thread's running state. + +| Event/Syscall Name | Description | Default Type | Extended Content | +| ------------------ | ------------------------------------------------------------------------------ | ------------ | -------------------------------------- | +| `read` | Reads/writes to drive files or the network; may be time-consuming or blocking. | `file` | `file.path`, `sock.conn`, `func.stack` | +| `write` | Reads/writes to drive files or the network; may be time-consuming or blocking. | `file` | `file.path`, `sock.conn`, `func.stack` | +| `readv` | Reads/writes to drive files or the network; may be time-consuming or blocking. | `file` | `file.path`, `sock.conn`, `func.stack` | +| `writev` | Reads/writes to drive files or the network; may be time-consuming or blocking. | `file` | `file.path`, `sock.conn`, `func.stack` | +| `preadv` | Reads/writes to drive files or the network; may be time-consuming or blocking. | `file` | `file.path`, `sock.conn`, `func.stack` | +| `pwritev` | Reads/writes to drive files or the network; may be time-consuming or blocking. | `file` | `file.path`, `sock.conn`, `func.stack` | +| `sync` | Synchronously flushes files to the drive; blocks the thread until completion. | `file` | `func.stack` | +| `fsync` | Synchronously flushes files to the drive; blocks the thread until completion. | `file` | `file.path`, `sock.conn`, `func.stack` | +| `fdatasync` | Synchronously flushes files to the drive; blocks the thread until completion. | `file` | `file.path`, `sock.conn`, `func.stack` | +| `sched_yield` | Thread voluntarily relinquishes the CPU for rescheduling. | `sched` | `func.stack` | +| `nanosleep` | Thread enters a sleep state. | `sched` | `func.stack` | +| `clock_nanosleep` | Thread enters a sleep state. | `sched` | `func.stack` | +| `wait4` | Thread blocks. | `sched` | `func.stack` | +| `waitpid` | Thread blocks. | `sched` | `func.stack` | +| `select` | Thread blocks and waits for an event. | `sched` | `func.stack` | +| `pselect6` | Thread blocks and waits for an event. | `sched` | `func.stack` | +| `poll` | Thread blocks and waits for an event. | `sched` | `func.stack` | +| `ppoll` | Thread blocks and waits for an event. | `sched` | `func.stack` | +| `epoll_wait` | Thread blocks and waits for an event. | `sched` | `func.stack` | +| `sendto` | Reads/writes to the network; may be time-consuming or blocking. | `net` | `sock.conn`, `func.stack` | +| `recvfrom` | Reads/writes to the network; may be time-consuming or blocking. | `net` | `sock.conn`, `func.stack` | +| `sendmsg` | Reads/writes to the network; may be time-consuming or blocking. | `net` | `sock.conn`, `func.stack` | +| `recvmsg` | Reads/writes to the network; may be time-consuming or blocking. | `net` | `sock.conn`, `func.stack` | +| `sendmmsg` | Reads/writes to the network; may be time-consuming or blocking. | `net` | `sock.conn`, `func.stack` | +| `recvmmsg` | Reads/writes to the network; may be time-consuming or blocking. | `net` | `sock.conn`, `func.stack` | +| `futex` | Often indicates lock contention; the thread may block. | `lock` | `futex.op`, `func.stack` | + +#### Aggregated Events + +tprofiling currently supports two main categories of system performance events: system call events and `oncpu` events. In certain scenarios, `oncpu` events and some system call events (like `read` and `write`) can trigger frequently, generating a large volume of system events. This can negatively impact both the performance of the application being observed and the tprofiling probe itself. + +To improve performance, tprofiling aggregates multiple system events with the same name from the same thread within a one-second interval into a single reported event. Therefore, a tprofiling event is actually an aggregated event containing one or more identical system events. Some attribute meanings differ between aggregated events and real system events: + +- `start_time`: The start time of the first system event in the aggregation. +- `end_time`: Calculated as `start_time + duration`. +- `duration`: The sum of the actual execution times of all system events in the aggregation. +- `count`: The number of system events aggregated. When `count` is 1, the aggregated event is equivalent to a single system event. +- Extended event attributes: The extended attributes of the first system event in the aggregation. + +## Introduction to L7Probe + +Purpose: L7 traffic observation, covering common protocols like HTTP1.X, PG, MySQL, Redis, Kafka, HTTP2.0, MongoDB, and RocketMQ. Supports observation of encrypted streams. + +Scope: Node, container, and Kubernetes pod environments. + +### Code Framework Design + +```text +L7Probe + | --- included // Public header files + | --- connect.h // L7 connect object definition + | --- pod.h // pod/container object definition + | --- conn_tracker.h // L7 protocol tracking object definition + | --- protocol // L7 protocol parsing + | --- http // HTTP1.X L7 message structure definition and parsing + | --- mysql // mysql L7 message structure definition and parsing + | --- pgsql // pgsql L7 message structure definition and parsing + | --- bpf // Kernel bpf code + | --- L7.h // BPF program parses L7 protocol types + | --- kern_sock.bpf.c // Kernel socket layer observation + | --- libssl.bpf.c // OpenSSL layer observation + | --- gossl.bpf.c // Go SSL layer observation + | --- cgroup.bpf.c // Pod lifecycle observation + | --- pod_mng.c // pod/container instance management (detects pod/container lifecycle) + | --- conn_mng.c // L7 Connect instance management (handles BPF observation events, such as Open/Close events, Stats statistics) + | --- conn_tracker.c // L7 traffic tracking (tracks data from BPF observation, such as data generated by send/write, read/recv system events) + | --- bpf_mng.c // BPF program lifecycle management (dynamically opens, loads, attaches, and unloads BPF programs, including uprobe BPF programs) + | --- session_conn.c // Manages JSSE sessions (records the mapping between JSSE sessions and socket connections, and reports JSSE connection information) + | --- L7Probe.c // Main probe program +``` + +### Probe Output + +| Metric Name | Table Name | Metric Type | Unit | Metric Description | +| --------------- | ---------- | ----------- | ---- | ------------------------------------------------------------------------------------------------------------------------------ | +| tgid | N/A | Key | N/A | Process ID of the L7 session. | +| client_ip | N/A | Key | N/A | Client IP address of the L7 session. | +| server_ip | N/A | Key | N/A | Server IP address of the L7 session.
Note: In Kubernetes, Cluster IP addresses can be translated to Backend IP addresses. | +| server_port | N/A | Key | N/A | Server port of the L7 session.
Note: In Kubernetes, Cluster Ports can be translated to Backend Ports. | +| l4_role | N/A | Key | N/A | Role of the L4 protocol (TCP Client/Server or UDP). | +| l7_role | N/A | Key | N/A | Role of the L7 protocol (Client or Server). | +| protocol | N/A | Key | N/A | Name of the L7 protocol (HTTP/HTTP2/MySQL...). | +| ssl | N/A | Label | N/A | Indicates whether the L7 session uses SSL encryption. | +| bytes_sent | l7_link | Gauge | N/A | Number of bytes sent by the L7 session. | +| bytes_recv | l7_link | Gauge | N/A | Number of bytes received by the L7 session. | +| segs_sent | l7_link | Gauge | N/A | Number of segments sent by the L7 session. | +| segs_recv | l7_link | Gauge | N/A | Number of segments received by the L7 session. | +| throughput_req | l7_rpc | Gauge | QPS | Request throughput of the L7 session. | +| throughput_resp | l7_rpc | Gauge | QPS | Response throughput of the L7 session. | +| req_count | l7_rpc | Gauge | N/A | Request count of the L7 session. | +| resp_count | l7_rpc | Gauge | N/A | Response count of the L7 session. | +| latency_avg | l7_rpc | Gauge | ns | Average latency of the L7 session. | +| latency | l7_rpc | Histogram | ns | Latency histogram of the L7 session. | +| latency_sum | l7_rpc | Gauge | ns | Total latency of the L7 session. | +| err_ratio | l7_rpc | Gauge | % | Error rate of the L7 session. | +| err_count | l7_rpc | Gauge | N/A | Error count of the L7 session. | + +### Dynamic Control + +#### Controlling the Scope of Pod Observation + +1. REST request sent to gala-gopher. +2. gala-gopher forwards the request to L7Probe. +3. L7Probe identifies relevant containers based on the Pod information. +4. L7Probe retrieves the CGroup ID (`cpuacct_cgrp_id`) of each container and writes it to the object module (using the `cgrp_add` API). +5. During socket system event processing, the CGroup (`cpuacct_cgrp_id`) of the process is obtained, referencing the Linux kernel code (`task_cgroup`). +6. Filtering occurs during observation via the object module (using the `is_cgrp_exist` API). + +#### Controlling Observation Capabilities + +1. REST request sent to gala-gopher. +2. gala-gopher forwards the request to L7Probe. +3. L7Probe dynamically enables or disables BPF-based observation features (including throughput, latency, tracing, and protocol type detection) based on the request parameters. + +### Observation Points + +#### Kernel Socket System Calls + +TCP-related system calls: + +```c +// int connect(int sockfd, const struct sockaddr *addr, socklen_t addrlen); +// int accept(int sockfd, struct sockaddr *addr, socklen_t *addrlen); +// int accept4(int sockfd, struct sockaddr *addr, socklen_t *addrlen, int flags); +// ssize_t write(int fd, const void *buf, size_t count); +// ssize_t send(int sockfd, const void *buf, size_t len, int flags); +// ssize_t read(int fd, void *buf, size_t count); +// ssize_t recv(int sockfd, void *buf, size_t len, int flags); +// ssize_t writev(int fd, const struct iovec *iov, int iovcnt); +// ssize_t readv(int fd, const struct iovec *iov, int iovcnt); +``` + +TCP and UDP-related system calls: + +```c +// ssize_t sendto(int sockfd, const void *buf, size_t len, int flags, const struct sockaddr *dest_addr, socklen_t addrlen); +// ssize_t recvfrom(int sockfd, void *buf, size_t len, int flags, struct sockaddr *src_addr, socklen_t *addrlen); +// ssize_t sendmsg(int sockfd, const struct msghdr *msg, int flags); +// ssize_t recvmsg(int sockfd, struct msghdr *msg, int flags); +// int close(int fd); +``` + +Important notes: + +1. `read`/`write` and `readv`/`writev` can be confused with regular file I/O. The kernel function `security_socket_sendmsg` is observed to determine if a file descriptor (FD) refers to a socket operation. +2. `sendto`/`recvfrom` and `sendmsg`/`recvmsg` are used by both TCP and UDP. Refer to the manuals below. +3. `sendmmsg`/`recvmmsg` and `sendfile` are not currently supported. + +[sendto manual](https://man7.org/linux/man-pages/man2/send.2.html): If sendto() is used on a connection-mode (SOCK_STREAM, SOCK_SEQPACKET) socket, the arguments dest_addr and addrlen are ignored (and the error EISCONN may be returned when they are not NULL and 0), and the error ENOTCONN is returned when the socket was not actually connected. otherwise, the address of the target is given by dest_addr with addrlen specifying its size. + +`sendto` determines that the protocol is TCP if the `dest_addr` parameter is NULL; otherwise, it is UDP. + +[recvfrom manual](https://linux.die.net/man/2/recvfrom): The recvfrom() and recvmsg() calls are used to receive messages from a socket, and may be used to receive data on a socket whether or not it is connection-oriented. + +`recvfrom` determines that the protocol is TCP if the `src_addr` parameter is NULL; otherwise, it is UDP. + +[sendmsg manual](https://man7.org/linux/man-pages/man3/sendmsg.3p.html): The sendmsg() function shall send a message through a connection-mode or connectionless-mode socket. If the socket is a connectionless-mode socket, the message shall be sent to the address specified by msghdr if no pre-specified peer address has been set. If a peer address has been pre-specified, either themessage shall be sent to the address specified in msghdr (overriding the pre-specified peer address), or the function shall return -1 and set errno to \[EISCONN]. If the socket is connection-mode, the destination address in msghdr shall be ignored. + +`sendmsg` determines that the protocol is TCP if `msghdr->msg_name` is NULL; otherwise, it is UDP. + +[recvmsg manual](https://man7.org/linux/man-pages/man3/recvmsg.3p.html): The recvmsg() function shall receive a message from a connection-mode or connectionless-mode socket. It is normally used with connectionless-mode sockets because it permits the application to retrieve the source address of received data. + +`recvmsg` determines that the protocol is TCP if `msghdr->msg_name` is NULL; otherwise, it is UDP. + +#### libSSL API + +SSL_write + +SSL_read + +#### Go SSL API + +#### JSSE API + +sun/security/ssl/SSLSocketImpl$AppInputStream + +sun/security/ssl/SSLSocketImpl$AppOutputStream + +### JSSE Observation Scheme + +#### Loading the JSSEProbe + +The `l7_load_jsse_agent` function in `main` loads the JSSEProbe. + +It polls processes in the whitelist (`g_proc_obj_map_fd`). If a process is a Java process, it uses `jvm_attach` to load **JSSEProbeAgent.jar** into it. After loading, the Java process outputs observation information to **/tmp/java-data-\/jsse-metrics.txt** at specific points (see [JSSE API](#jsse-api)). + +#### Processing JSSEProbe Messages + +The `l7_jsse_msg_handler` thread handles JSSEProbe messages. + +It polls processes in the whitelist (`g_proc_obj_map_fd`). If a process has a `jsse-metrics` output file, it reads the file line by line, then parses, converts, and reports JSSE read/write information. + +##### 1. Parsing JSSE Read/Write Information + +The `jsse-metrics.txt` output format is: + +```text +|jsse_msg|662220|Session(1688648699909|TLS_AES_256_GCM_SHA384)|1688648699989|Write|127.0.0.1|58302|This is test message| +``` + +It parses the process ID, session ID, time, read/write operation, IP address, port, and payload. + +The parsed information is stored in `session_data_args_s`. + +##### 2. Converting JSSE Read/Write Information + +It converts the information in `session_data_args_s` into `sock_conn` and `conn_data`. + +This conversion queries two hash maps: + +`session_head`: Records the mapping between the JSSE session ID and the socket connection ID. If the process ID and 4-tuple information match, the session and socket connection are linked. + +`file_conn_head`: Records the last session ID of the Java process, in case L7Probe doesn't start reading from the beginning of a request and can't find the session ID. + +##### 3. Reporting JSSE Read/Write Information + +It reports `sock_conn` and `conn_data` to the map. + +## sliprobe Introduction + +`sliprobe` uses eBPF to collect and report container-level service-level indicator (SLI) metrics periodically. + +### Features + +- Collects the total latency and statistical histogram of CPU scheduling events per container. Monitored events include scheduling wait, active sleep, lock/IO blocking, scheduling delay, and long system calls. +- Collects the total latency and statistical histogram of memory allocation events per container. Monitored events include memory reclamation, swapping, and memory compaction. +- Collects the total latency and statistical histogram of BIO layer I/O operations per container. + +### Usage Instructions + +Example command to start `sliprobe`: Specifies a reporting period of 15 seconds and observes SLI metrics for containers `abcd12345678` and `abcd87654321`. + +```shell +curl -X PUT http://localhost:9999/sli -d json='{"params":{"report_period":15}, "snoopers":{"container_id":[{"container_id": "abcd12345678","abcd87654321"}]}, "state":"running"}' +``` + +### Code Logic + +#### Overview + +1. The user-space application receives a list of containers to monitor and stores the inode of each container's `cpuacct` subsystem directory in an eBPF map, sharing it with the kernel. +2. The kernel traces relevant kernel events using eBPF kprobes/tracepoints, determines if the event belongs to a monitored container, and records the event type and timestamp. It aggregates and reports SLI metrics for processes in the same cgroup at regular intervals. +3. The user-space application receives and prints the SLI metrics reported by the kernel. + +#### How SLI Metrics Are Calculated + +##### CPU SLI + +1. **cpu_wait** + + At the `sched_stat_wait` tracepoint, get the `delay` value (second parameter). + +2. **cpu_sleep** + + At the `sched_stat_sleep` tracepoint, get the `delay` value (second parameter). + +3. **cpu_iowait** + + At the `sched_stat_blocked` tracepoint, if the current process is `in_iowait`, get the `delay` value (second parameter). + +4. **cpu_block** + + At the `sched_stat_blocked` tracepoint, if the current process is not `in_iowait`, get the `delay` value (second parameter). + +5. **cpu_rundelay** + + At the `sched_switch` tracepoint, get the `run_delay` value of the next scheduled process (`next->sched_info.run_delay`) from the third parameter `next` and store it in `task_sched_map`. Calculate the difference in `run_delay` between two scheduling events of the same process. + +6. **cpu_longsys** + + At the `sched_switch` tracepoint, get the `task` structure of the next scheduled process from the third parameter `next`. Obtain the number of context switches (`nvcsw+nivcsw`) and user-space execution time (`utime`) from the `task` structure. If the number of context switches and user-space execution time remain the same between two scheduling events of the same process, the process is assumed to be executing a long system call. Accumulate the time the process spends in kernel mode. + +##### MEM SLI + +1. **mem_reclaim** + + Calculate the difference between the return and entry timestamps of the `mem_cgroup_handle_over_high` function. + + Calculate the difference between the timestamps of the `mm_vmscan_memcg_reclaim_end` and `mm_vmscan_memcg_reclaim_begin` tracepoints. + +2. **mem_swapin** + + Calculate the difference between the return and entry timestamps of the `do_swap_page` function. + +3. **mem_compact** + + Calculate the difference between the return and entry timestamps of the `try_to_compact_pages` function. + +##### IO SLI + +1. **bio_latency** + + Calculate the timestamp difference between entering the `bio_endio` function and triggering the `block_bio_queue` tracepoint. + + Calculate the timestamp difference between entering the `bio_endio` function and exiting the `generic_make_request_checks` function. + +## Output Data + +- **Metric** + + Prometheus Server has a built-in Express Browser UI. You can use PromQL statements to query metric data. For details, see [Using the expression browser](https://prometheus.io/docs/prometheus/latest/getting_started/#using-the-expression-browser) in the official document. The following is an example. + + If the specified metric is `gala_gopher_tcp_link_rcv_rtt`, the metric data displayed on the UI is as follows: + + ```basic + gala_gopher_tcp_link_rcv_rtt{client_ip="x.x.x.165",client_port="1234",hostname="openEuler",instance="x.x.x.172:8888",job="prometheus",machine_id="1fd3774xx",protocol="2",role="0",server_ip="x.x.x.172",server_port="3742",tgid="1516"} 1 + ``` + +- **Metadata** + + You can directly consume data from the Kafka topic `gala_gopher_metadata`. The following is an example. + + ```bash + # Input request + ./bin/kafka-console-consumer.sh --bootstrap-server x.x.x.165:9092 --topic gala_gopher_metadata + # Output data + {"timestamp": 1655888408000, "meta_name": "thread", "entity_name": "thread", "version": "1.0.0", "keys": ["machine_id", "pid"], "labels": ["hostname", "tgid", "comm", "major", "minor"], "metrics": ["fork_count", "task_io_wait_time_us", "task_io_count", "task_io_time_us", "task_hang_count"]} + ``` + +- **Abnormal events** + + You can directly consume data from the Kafka topic `gala_gopher_event`. The following is an example. + + ```bash + # Input request + ./bin/kafka-console-consumer.sh --bootstrap-server x.x.x.165:9092 --topic gala_gopher_event + # Output data + {"timestamp": 1655888408000, "meta_name": "thread", "entity_name": "thread", "version": "1.0.0", "keys": ["machine_id", "pid"], "labels": ["hostname", "tgid", "comm", "major", "minor"], "metrics": ["fork_count", "task_io_wait_time_us", "task_io_count", "task_io_time_us", "task_hang_count"]} + ``` diff --git a/docs/en/server/maintenance/gala/using-gala-spider.md b/docs/en/server/maintenance/gala/using-gala-spider.md new file mode 100644 index 0000000000000000000000000000000000000000..1b1afa7d80b875cc011bccb947b8d58bbc780ba7 --- /dev/null +++ b/docs/en/server/maintenance/gala/using-gala-spider.md @@ -0,0 +1,527 @@ +# Using gala-spider + +This chapter describes how to deploy and use gala-spider and gala-inference. + +## gala-spider + +gala-spider provides the OS-level topology drawing function. It periodically obtains the data of all observed objects collected by gala-gopher (an OS-level data collection software) at a certain time point and calculates the topology relationship between them. The generated topology is saved to the graph database ArangoDB. + +### Installation + +Mount the Yum sources. + +```basic +[oe-2209] # openEuler 22.09 officially released repository +name=oe2209 +baseurl=http://119.3.219.20:82/openEuler:/22.09/standard_x86_64 +enabled=1 +gpgcheck=0 +priority=1 + +[oe-2209:Epol] # openEuler 22.09: Epol officially released repository +name=oe2209_epol +baseurl=http://119.3.219.20:82/openEuler:/22.09:/Epol/standard_x86_64/ +enabled=1 +gpgcheck=0 +priority=1 +``` + +Install gala-spider. + +```sh +# yum install gala-spider +``` + +### Configuration + +#### Configuration File Description + +The configuration file of gala-spider is **/etc/gala-spider/gala-spider.yaml**. The configuration items in this file are described as follows: + +- `global`: global configuration information. + - `data_source`: database for collecting observation metrics. Currently, only `prometheus` is supported. + - `data_agent`: agent for collecting observation metrics. Currently, only `gala_gopher` is supported. +- `spider`: spider configuration information. + - `log_conf`: log configuration information. + - `log_path`: log file path. + - `log_level`: level of the logs to be printed. The value can be `DEBUG`, `INFO`, `WARNING`, `ERROR`, or `CRITICAL`. + - `max_size`: log file size, in MB. + - `backup_count`: number of backup log files. +- `storage`: configuration information about the topology storage service. + - `period`: storage period, in seconds, indicating the interval for storing the topology. + - `database`: graph database for storage. Currently, only `arangodb` is supported. + - `db_conf`: configuration information of the graph database. + - `url`: IP address of the graph database server. + - `db_name`: name of the database where the topology is stored. +- `kafka`: Kafka configuration information. + - `server`: Kafka server address. + - `metadata_topic`: topic name of the observed metadata messages. + - `metadata_group_id`: consumer group ID of the observed metadata messages. +- `prometheus`: Prometheus database configuration information. + - `base_url`: IP address of the Prometheus server. + - `instant_api`: API for collecting data at a single time point. + - `range_api`: API for collecting data in a time range. + - `step`: collection time step, which is configured for `range_api`. + +#### Configuration File Example + +```yaml +global: + data_source: "prometheus" + data_agent: "gala_gopher" + +prometheus: + base_url: "http://localhost:9090/" + instant_api: "/api/v1/query" + range_api: "/api/v1/query_range" + step: 1 + +spider: + log_conf: + log_path: "/var/log/gala-spider/spider.log" + # log level: DEBUG/INFO/WARNING/ERROR/CRITICAL + log_level: INFO + # unit: MB + max_size: 10 + backup_count: 10 + +storage: + # unit: second + period: 60 + database: arangodb + db_conf: + url: "http://localhost:8529" + db_name: "spider" + +kafka: + server: "localhost:9092" + metadata_topic: "gala_gopher_metadata" + metadata_group_id: "metadata-spider" +``` + +### Start + +- Run the following command to start gala-spider. + + ```sh + # spider-storage + ``` + +- Use the systemd service to start gala-spider. + + ```sh + # systemctl start gala-spider + ``` + +### How to Use + +#### Deployment of External Dependent Software + +The running of gala-spider depends on multiple external software for interaction. Therefore, before starting gala-spider, you need to deploy the software on which gala-spider depends. The following figure shows the software dependency of gala-spider. + +![gala-spider-arch](./figures/gala-spider-arch.png) + +The dotted box on the right indicates the two functional components of gala-spider. The green parts indicate the external components that gala-spider directly depends on, and the gray rectangles indicate the external components that gala-spider indirectly depends on. + +- **spider-storage**: core component of gala-spider, which provides the topology storage function. + 1. Obtains the metadata of the observation object from Kafka. + 2. Obtains information about all observation object instances from Prometheus. + 3. Saves the generated topology to the graph database ArangoDB. +- **gala-inference**: core component of gala-spider, which provides the root cause locating function. It subscribes to abnormal KPI events from Kafka to trigger the root cause locating process of abnormal KPIs, constructs a fault propagation graph based on the topology obtained from the ArangoDB, and outputs the root cause locating result to Kafka. +- **prometheus**: time series database. The observation metric data collected by the gala-gopher component is reported to Prometheus for further processing. +- **kafka**: messaging middleware, which is used to store the observation object metadata reported by gala-gopher, exception events reported by the exception detection component gala-anteater, and root cause locating results reported by the cause-inference component. +- **arangodb**: graph database, which is used to store the topology generated by spider-storage. +- **gala-gopher**: data collection component. It must be deployed in advance. +- **arangodb-ui**: UI provided by ArangoDB, which can be used to query topologies. + +The two functional components in gala-spider are released as independent software packages. + +**spider-storage**: corresponds to the gala-spider software package in this section. + +**gala-inference**: corresponds to the gala-inference software package. + +For details about how to deploy the gala-gopher software, see [Using gala-gopher](using-gala-gopher.md). This section only describes how to deploy ArangoDB. + +The current ArangoDB version is 3.8.7, which has the following requirements on the operating environment: + +- Only the x86 system is supported. +- GCC 10 or later + +For details about ArangoDB deployment, see [Deployment](https://www.arangodb.com/docs/3.9/deployment.html) in the ArangoDB official document. + +The RPM-based ArangoDB deployment process is as follows: + +1. Configure the Yum sources. + + ```basic + [oe-2209] # openEuler 22.09 officially released repository + name=oe2209 + baseurl=http://119.3.219.20:82/openEuler:/22.09/standard_x86_64 + enabled=1 + gpgcheck=0 + priority=1 + + [oe-2209:Epol] # openEuler 22.09: Epol officially released repository + name=oe2209_epol + baseurl=http://119.3.219.20:82/openEuler:/22.09:/Epol/standard_x86_64/ + enabled=1 + gpgcheck=0 + priority=1 + ``` + +2. Install arangodb3. + + ```sh + # yum install arangodb3 + ``` + +3. Modify the configurations. + + The configuration file of the arangodb3 server is **/etc/arangodb3/arangod.conf**. You need to modify the following configurations: + + - `endpoint`: IP address of the arangodb3 server. + - `authentication`: whether identity authentication is required for accessing the arangodb3 server. Currently, gala-spider does not support identity authentication. Therefore, set `authentication` to `false`. + + The following is an example. + + ```yaml + [server] + endpoint = tcp://0.0.0.0:8529 + authentication = false + ``` + +4. Start arangodb3. + + ```sh + # systemctl start arangodb3 + ``` + +#### Modifying gala-spider Configuration Items + +After the dependent software is started, you need to modify some configuration items in the gala-spider configuration file. The following is an example. + +Configure the Kafka server address. + +```yaml +kafka: + server: "localhost:9092" +``` + +Configure the Prometheus server address. + +```yaml +prometheus: + base_url: "http://localhost:9090/" +``` + +Configure the IP address of the ArangoDB server. + +```yaml +storage: + db_conf: + url: "http://localhost:8529" +``` + +#### Starting the Service + +Run `systemctl start gala-spider` to start the service. Run `systemctl status gala-spider` to check the startup status. If the following information is displayed, the startup is successful: + +```sh +[root@openEuler ~]# systemctl status gala-spider +● gala-spider.service - a-ops gala spider service + Loaded: loaded (/usr/lib/systemd/system/gala-spider.service; enabled; vendor preset: disabled) + Active: active (running) since Tue 2022-08-30 17:28:38 CST; 1 day 22h ago + Main PID: 2263793 (spider-storage) + Tasks: 3 (limit: 98900) + Memory: 44.2M + CGroup: /system.slice/gala-spider.service + └─2263793 /usr/bin/python3 /usr/bin/spider-storage +``` + +#### Output Example + +You can query the topology generated by gala-spider on the UI provided by ArangoDB. The procedure is as follows: + +1. Enter the IP address of the ArangoDB server in the address box of the browser, for example, ****. The ArangoDB UI is displayed. + +2. Click **DB** in the upper right corner of the page to switch to the spider database. + +3. On the **COLLECTIONS** page, you can view the collections of observation object instances and topology relationships stored in different time segments, as shown in the following figure. + + ![spider_topology](./figures/spider_topology.png) + +4. You can query the stored topology using the AQL statements provided by ArangoDB. For details, see the [AQL Documentation](https://www.arangodb.com/docs/3.8/aql/). + +## gala-inference + +gala-inference provides the capability of locating root causes of abnormal KPIs. It uses the exception detection result and topology as the input and outputs the root cause locating result to Kafka. The gala-inference component is archived in the gala-spider project. + +### Installation + +Mount the Yum sources. + +```basic +[oe-2209] # openEuler 22.09 officially released repository +name=oe2209 +baseurl=http://119.3.219.20:82/openEuler:/22.09/standard_x86_64 +enabled=1 +gpgcheck=0 +priority=1 + +[oe-2209:Epol] # openEuler 22.09: Epol officially released repository +name=oe2209_epol +baseurl=http://119.3.219.20:82/openEuler:/22.09:/Epol/standard_x86_64/ +enabled=1 +gpgcheck=0 +priority=1 +``` + +Install gala-inference. + +```sh +# yum install gala-inference +``` + +### Configuration + +#### Configuration File Description + +The configuration items in the gala-inference configuration file **/etc/gala-inference/gala-inference.yaml** are described as follows: + +- `inference`: configuration information about the root cause locating algorithm. + - `tolerated_bias`: tolerable time offset for querying the topology at the exception time point, in seconds. + - `topo_depth`: maximum depth for topology query. + - `root_topk`: top *K* root cause metrics generated in the root cause locating result. + - `infer_policy`: root cause derivation policy, which can be `dfs` or `rw`. + - `sample_duration`: sampling period of historical metric data, in seconds. + - `evt_valid_duration`: valid period of abnormal system metric events during root cause locating, in seconds. + - `evt_aging_duration`: aging period of abnormal metric events during root cause locating, in seconds. +- `kafka`: Kafka configuration information. + - `server`: IP address of the Kafka server. + - `metadata_topic`: configuration information about the observed metadata messages. + - `topic_id`: topic name of the observed metadata messages. + - `group_id`: consumer group ID of the observed metadata messages. + - `abnormal_kpi_topic`: configuration information about abnormal KPI event messages. + - `topic_id`: topic name of the abnormal KPI event messages. + - `group_id`: consumer group ID of the abnormal KPI event messages. + - `abnormal_metric_topic`: configuration information about abnormal metric event messages. + - `topic_id`: topic name of the abnormal metric event messages. + - `group_id`: consumer group ID of the abnormal system metric event messages. + - `consumer_to`: timeout interval for consuming abnormal system metric event messages, in seconds. + - `inference_topic`: configuration information about the output event messages of the root cause locating result. + - `topic_id`: topic name of the output event messages of the root cause locating result. +- `arangodb`: configuration information about the ArangoDB graph database, which is used to query sub-topologies required for root cause locating. + - `url`: IP address of the graph database server. + - `db_name`: name of the database where the topology is stored. +- `log_conf`: log configuration information. + - `log_path`: log file path. + - `log_level`: level of the logs to be printed. The value can be `DEBUG`, `INFO`, `WARNING`, `ERROR`, or `CRITICAL`. + - `max_size`: log file size, in MB. + - `backup_count`: number of backup log files. +- `prometheus`: Prometheus database configuration information, which is used to obtain historical time series data of metrics. + - `base_url`: IP address of the Prometheus server. + - `range_api`: API for collecting data in a time range. + - `step`: collection time step, which is configured for `range_api`. + +#### Configuration File Example + +```yaml +inference: + # Tolerable time offset for querying the topology at the exception time point, in seconds. + tolerated_bias: 120 + topo_depth: 10 + root_topk: 3 + infer_policy: "dfs" + # Unit: second + sample_duration: 600 + # Valid period of abnormal metric events during root cause locating, in seconds. + evt_valid_duration: 120 + # Aging period of abnormal metric events, in seconds. + evt_aging_duration: 600 + +kafka: + server: "localhost:9092" + metadata_topic: + topic_id: "gala_gopher_metadata" + group_id: "metadata-inference" + abnormal_kpi_topic: + topic_id: "gala_anteater_hybrid_model" + group_id: "abn-kpi-inference" + abnormal_metric_topic: + topic_id: "gala_anteater_metric" + group_id: "abn-metric-inference" + consumer_to: 1 + inference_topic: + topic_id: "gala_cause_inference" + +arangodb: + url: "http://localhost:8529" + db_name: "spider" + +log: + log_path: "/var/log/gala-inference/inference.log" + # log level: DEBUG/INFO/WARNING/ERROR/CRITICAL + log_level: INFO + # unit: MB + max_size: 10 + backup_count: 10 + +prometheus: + base_url: "http://localhost:9090/" + range_api: "/api/v1/query_range" + step: 5 +``` + +### Start + +- Run the following command to start gala-inference. + + ```sh + # gala-inference + ``` + +- Use the systemd service to start gala-inference. + + ```sh + # systemctl start gala-inference + ``` + +### How to Use + +#### Dependent Software Deployment + +The running dependency of gala-inference is the same as that of gala-spider. For details, see [Deployment of External Dependent Software](#deployment-of-external-dependent-software). In addition, gala-inference indirectly depends on the running of [gala-spider](#gala-spider) and [gala-anteater](using-gala-anteater.md). Deploy gala-spider and gala-anteater in advance. + +#### Modify configuration items + +Modify some configuration items in the gala-inference configuration file. The following is an example. + +Configure the Kafka server address. + +```yaml +kafka: + server: "localhost:9092" +``` + +Configure the Prometheus server address. + +```yaml +prometheus: + base_url: "http://localhost:9090/" +``` + +Configure the IP address of the ArangoDB server. + +```yaml +arangodb: + url: "http://localhost:8529" +``` + +#### Starting the Service + +Run `systemctl start gala-inference` to start the service. Run `systemctl status gala-inference` to check the startup status. If the following information is displayed, the startup is successful: + +```sh +[root@openEuler ~]# systemctl status gala-inference +● gala-inference.service - a-ops gala inference service + Loaded: loaded (/usr/lib/systemd/system/gala-inference.service; enabled; vendor preset: disabled) + Active: active (running) since Tue 2022-08-30 17:55:33 CST; 1 day 22h ago + Main PID: 2445875 (gala-inference) + Tasks: 10 (limit: 98900) + Memory: 48.7M + CGroup: /system.slice/gala-inference.service + └─2445875 /usr/bin/python3 /usr/bin/gala-inference +``` + +#### Output Example + +When the exception detection module gala-anteater detects a KPI exception, it exports the corresponding abnormal KPI event to Kafka. The gala-inference keeps monitoring the message of the abnormal KPI event. If gala-inference receives the message of the abnormal KPI event, root cause locating is triggered. The root cause locating result is exported to Kafka. You can view the root cause locating result on the Kafka server. The basic procedure is as follows: + +1. If Kafka is installed using the source code, go to the Kafka installation directory. + + ```sh + cd /root/kafka_2.13-2.8.0 + ``` + +2. Run the command for consuming the topic to obtain the output of root cause locating. + + ```sh + ./bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic gala_cause_inference + ``` + + Output example: + + ```json + { + "Timestamp": 1661853360000, + "event_id": "1661853360000_1fd37742xxxx_sli_12154_19", + "Attributes": { + "event_id": "1661853360000_1fd37742xxxx_sli_12154_19" + }, + "Resource": { + "abnormal_kpi": { + "metric_id": "gala_gopher_sli_rtt_nsec", + "entity_id": "1fd37742xxxx_sli_12154_19", + "timestamp": 1661853360000, + "metric_labels": { + "machine_id": "1fd37742xxxx", + "tgid": "12154", + "conn_fd": "19" + } + }, + "cause_metrics": [ + { + "metric_id": "gala_gopher_proc_write_bytes", + "entity_id": "1fd37742xxxx_proc_12154", + "metric_labels": { + "__name__": "gala_gopher_proc_write_bytes", + "cmdline": "/opt/redis/redis-server x.x.x.172:3742", + "comm": "redis-server", + "container_id": "5a10635e2c43", + "hostname": "openEuler", + "instance": "x.x.x.172:8888", + "job": "prometheus", + "machine_id": "1fd37742xxxx", + "pgid": "12154", + "ppid": "12126", + "tgid": "12154" + }, + "timestamp": 1661853360000, + "path": [ + { + "metric_id": "gala_gopher_proc_write_bytes", + "entity_id": "1fd37742xxxx_proc_12154", + "metric_labels": { + "__name__": "gala_gopher_proc_write_bytes", + "cmdline": "/opt/redis/redis-server x.x.x.172:3742", + "comm": "redis-server", + "container_id": "5a10635e2c43", + "hostname": "openEuler", + "instance": "x.x.x.172:8888", + "job": "prometheus", + "machine_id": "1fd37742xxxx", + "pgid": "12154", + "ppid": "12126", + "tgid": "12154" + }, + "timestamp": 1661853360000 + }, + { + "metric_id": "gala_gopher_sli_rtt_nsec", + "entity_id": "1fd37742xxxx_sli_12154_19", + "metric_labels": { + "machine_id": "1fd37742xxxx", + "tgid": "12154", + "conn_fd": "19" + }, + "timestamp": 1661853360000 + } + ] + } + ] + }, + "SeverityText": "WARN", + "SeverityNumber": 13, + "Body": "A cause inferring event for an abnormal event" + } + ``` diff --git a/docs/en/server/maintenance/kernel_live_upgrade/_toc.yaml b/docs/en/server/maintenance/kernel_live_upgrade/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..0a93b9d5f6fed67ff093947b756245cc53a986f5 --- /dev/null +++ b/docs/en/server/maintenance/kernel_live_upgrade/_toc.yaml @@ -0,0 +1,13 @@ +label: Kernel Hot Upgrade Guide +isManual: true +href: ./kernel-live-upgrade.md +description: >- + User-space automation tool that facilitates rapid kernel restarts and program + live migration, enabling kernel hot-swapping functionality +sections: + - label: Installation and Deployment + href: ./installation-and-deployment.md + - label: Usage Guide + href: ./usage-guide.md + - label: Common Problems and Solutions + href: ./common-issues-and-solutions.md diff --git a/docs/en/server/maintenance/kernel_live_upgrade/common-issues-and-solutions.md b/docs/en/server/maintenance/kernel_live_upgrade/common-issues-and-solutions.md new file mode 100644 index 0000000000000000000000000000000000000000..b279c0defae1f100e6b346e00c4078748e0afdce --- /dev/null +++ b/docs/en/server/maintenance/kernel_live_upgrade/common-issues-and-solutions.md @@ -0,0 +1,29 @@ +# Common Problems and Solutions + +## Issue 1: After the `nvwa update` Command Is Executed, the System Is Not Upgraded + +Cause: An error occurs when the running information is retained or the kernel is replaced. + +Solution: View logs to find the error cause. + +## Issue 2: After the Acceleration Feature Is Enabled, the `nvwa` Command Fails to Be Executed + +Cause: NVWA provides many acceleration features, including quick kexec, pin memory, and cpu park. These features involve the cmdline configuration and memory allocation. When selecting the memory, run cat /proc/iomemory to ensure that the selected memory does not conflict with that of other programs. If necessary, run the dmesg command to check whether error logs exist after the feature is enabled. + +## Issue 3: After the Hot Upgrade, the Related Process Is Not Recovered + +Cause: Check whether the nvwa service is running. If the nvwa service is running, the service or process may fail to be recovered. + +Solution: Run the service `nvwa status` command to view the nvwa logs. If the service fails to be started, check whether the service is enabled, and then run the `systemd` command to view the logs of the corresponding service. Further logs are stored in the process or service folder named after the path specified by **criu_dir**. The dump.log file stores the logs generated when the running information is retained, and the restore.log file restores the logs generated for process recovery. + +## Issue 4: The Recovery Fails, and the Log Displays "Can't Fork for 948: File Exists." + +Cause: The kernel hot upgrade tool finds that the PID of the program is occupied during program recovery. + +Solution: The current kernel does not provide a mechanism for retaining PIDs. Related policies are being developed. This restriction will be resolved in later kernel versions. Currently, you can only manually restart related processes. + +## Issue 5: When the `nvwa` Command Is Used to Save and Recover a Simple Program (Hello World), the System Displays a Message Indicating That the Operation Fails or the Program Is Not Running + +Cause: There are many restrictions on the use of CRIU. + +Solution: View the NVWA logs. If the error is related to the CRIU, check the dump.log or restore.log file in the corresponding directory. For details about the usage restrictions related to the CRIU, see [CRIU WiKi](https://criu.org/What_cannot_be_checkpointed). diff --git a/docs/en/server/maintenance/kernel_live_upgrade/installation-and-deployment.md b/docs/en/server/maintenance/kernel_live_upgrade/installation-and-deployment.md new file mode 100644 index 0000000000000000000000000000000000000000..3f802b7ce308139624cae07be09bf2c90827270e --- /dev/null +++ b/docs/en/server/maintenance/kernel_live_upgrade/installation-and-deployment.md @@ -0,0 +1,180 @@ +# Installation and Deployment + +This document describes how to install and deploy the kernel hot upgrade tool. + + + +- [Installation and Deployment](#installation-and-deployment) + - [Hardware and Software Requirements](#hardware-and-software-requirements) + - [Hardware Requirements](#hardware-requirements) + - [Software Requirements](#software-requirements) + - [Environment Preparation](#environment-preparation) + - [Installing the Kernel Hot Upgrade Tool](#installing-the-kernel-hot-upgrade-tool) + - [Deploying the Kernel Hot Upgrade Tool](#deploying-the-kernel-hot-upgrade-tool) + - [Configurations](#configurations) + - [Enabling the Kernel Hot Upgrade Tool](#enabling-the-kernel-hot-upgrade-tool) + + + +## Hardware and Software Requirements + +### Hardware Requirements + +- Currently, only the ARM64 architecture is supported. + +### Software Requirements + +- Operating system: openEuler 22.03 LTS + +## Environment Preparation + +- Install the openEuler system. For details, see the [_openEuler Installation Guide_](../../installation_upgrade/installation/installation.md) + +- The root permission is required for installing the kernel hot upgrade tool. + +## Installing the Kernel Hot Upgrade Tool + +This section describes how to install the kernel live upgrade tool. + +Perform the following steps: + +1. Mount the ISO file of openEuler. + + ```shell + mount openEuler-22.03-LTS-everything-aarch64-dvd.iso /mnt + ``` + +2. Configure the local yum repository. + + ```shell + vim /etc/yum.repos.d/local.repo + ``` + + The configurations are as follows: + + ```conf + [local] + name=local + baseurl=file:///mnt + gpgcheck=1 + enabled=1 + ``` + +3. Import the GPG public key of the RPM digital signature to the system. + + ```shell + rpm --import /mnt/RPM-GPG-KEY-openEuler + ``` + +4. Install the kernel hot upgrade tool. + + ```shell + yum install nvwa -y + ``` + +5. Check whether the installation is successful. If the command output is as follows, the installation is successful. + + ```shell + $ rpm -qa | grep nvwa + nvwa-xxx + ``` + +## Deploying the Kernel Hot Upgrade Tool + +This section describes how to configure and deploy the kernel hot upgrade tool. + +### Configurations + +The configuration files of the kernel hot upgrade tool are stored in **/etc/nvwa**. The configuration files are as follows: + +- nvwa-restore.yaml + + This configuration file is used to instruct the kernel hot upgrade tool to save and recover the process during the kernel hot upgrade. The configuration is as follows: + + - pids + + Specifies the processes that need to be retained and recovered during the NVWA hot upgrade. The processes are identified by process ID (PID). Note that the processes managed by NVWA are automatically recovered after the NVWA service is started. + + - services + + Specifies the services that need to be retained and recovered during NVWA hot upgrade. Compared to PIDs, the kernel hot upgrade tool can directly save and recover the process. For services, the kernel hot upgrade tool depends on the systemd to perform related operations. The service name must be the same as the service name used in systemd. Note that whether the service managed by NVWA needs to be automatically recovered when the NVWA is started depends on whether the service is enabled in the systemd. Currently, only the notify and oneshot service types are supported. + + - restore_net + + Specifies whether the kernel hot upgrade tool is required to save and recover the network configuration. If the network configuration is incorrect, the network may be unavailable after the recovery. This function is disabled by default. + + - enable_quick_kexec + + Used to specify whether to enable the quick kexec feature. quick kexec is a feature launched by the NVWA community to accelerate the kernel restart process. To use this feature, add "quickkexec=128M" to cmdline. 128 indicates the size of the memory allocated to the quick kexec feature. The memory is used to load the kernel and initramfs during the upgrade. Therefore, the size must be greater than the total size of the kernel and initramfs involved in the upgrade. This feature is disabled by default. + + - enable_pin_memory + + Used to specify whether to enable the pin memory feature. pin memory is a feature launched by the NVWA community to accelerate the process storage and recovery process. The pin_memory feature is not supported for multi-process recovery. To use the feature, you need to add "max_pin_pid_num=10 redirect_space_size=2M pinmemory=200M@0x640000000" to cmdline. + + max_pin_pid_num indicates the maximum number of processes that support pin memory recovery. redirect_space_size indicates the reserved memory space required for redirecting physical pages during pin memory recovery. You are advised to set redirect_space_size to 1/100 of the total reserved pin memory. pinmemory indicates the start point and size of the memory segment. The 200 MB space starting from 0x640000000 is the total memory space used by the pin memory. This space should not be used by other programs. + +- Configuration example of **nvwa-restore.yaml** + +```yaml +pids: + - 14109 +services: + - redis +restore_net: false +enable_quick_kexec: true +enable_pin_memory: true +``` + +- **nvwa-server.yaml** + + This file contains the configuration information required during the running of the kernel hot upgrade tool. The details are as follows: + + - criu_dir + + This parameter specifies the directory for storing the information generated when the kernel hot upgrade tool saves the running information. Note that the information may occupy a large amount of disk space. + + - criu_exe + + This parameter specifies the path of the CRIU executable file used by the kernel hot upgrade tool. You are advised not to change the path unless you need to debug the CRIU. + + - kexec_exe + + This parameter specifies the path of the kexec executable file used by the kernel hot upgrade tool. You are advised not to change the path unless you need to debug kexec. + + - systemd_etc + + This parameter specifies the path of the folder used to overwrite the systemd configuration file. The path is determined by the systemd. Generally, you do not need to change the path. + + - log_dir + + This parameter stores the log information generated by the kernel hot upgrade tool. The log module is not enabled currently. For details about how to view logs of the kernel hot upgrade tool, see How to Run. + +- Configuration example of **nvwa-server.yaml** + +```yaml +criu_dir: /var/nvwa/running/ +criu_exe: /usr/sbin/criu +kexec_exe: /usr/sbin/kexec +systemd_etc: /etc/systemd/system/ +log_dir: /etc/nvwa/log/ +``` + +## Enabling the Kernel Hot Upgrade Tool + +The running of the kernel hot upgrade tool depends on the configuration file. After the configuration file is modified, you need to run the kernel hot upgrade tool again. + +After the installation is successful, you can run the systemd commands to operate the kernel hot upgrade tool. + +- Enable NVWA. + + systemctl enable nvwa + +- Start nvwa. + + systemctl start nvwa + +- View the NVWA logs. + + service nvwa status + +- For more usage, see the usage of systemd. diff --git a/docs/en/server/maintenance/kernel_live_upgrade/kernel-live-upgrade.md b/docs/en/server/maintenance/kernel_live_upgrade/kernel-live-upgrade.md new file mode 100644 index 0000000000000000000000000000000000000000..a12557cb441503e148a1e3462d25ac05e1767f32 --- /dev/null +++ b/docs/en/server/maintenance/kernel_live_upgrade/kernel-live-upgrade.md @@ -0,0 +1,14 @@ +# Kernel Live Upgrade Guide + +This document describes how to install, deploy, and use the kernel live upgrade feature on openEuler. This kernel live upgrade feature on openEuler is implemented through quick kernel restart and hot program migration. A user-mode tool is provided to automate this process. + +This document is intended for community developers, open-source enthusiasts, and partners who want to learn about and use the openEuler system and kernel live upgrade. The users are expected to know basics about the Linux operating system. + +## Application Scenario + +The kernel live upgrade is to save and restore the process running data with the second-level end-to-end latency. + +The following two conditions must be met: + +1. The kernel needs to be restarted due to vulnerability fixing or version update. +2. Services running on the kernel can be quickly recovered after the kernel is restarted. diff --git a/docs/en/server/maintenance/kernel_live_upgrade/usage-guide.md b/docs/en/server/maintenance/kernel_live_upgrade/usage-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..c41f6465c9f032e2629670acb9ecd9baba183bef --- /dev/null +++ b/docs/en/server/maintenance/kernel_live_upgrade/usage-guide.md @@ -0,0 +1,113 @@ +# Usage Guide + + + +- [Usage Guide](#usage-guide) + - [Command](#command) + - [Restrictions](#restrictions) + - [NVWA Acceleration Feature Description and Usage](#nvwa-acceleration-feature-description-and-usage) + - [Generated Log Information](#generated-log-information) + + + +## Command + +- `nvwa help` + + Prints the help information. The printed information is as follows: + + ```text + NAME: + nvwa - a tool used for openEuler kernel update. + + USAGE: + nvwa [global options] command [command options] [arguments...] + + VERSION: + 0.1 + + COMMANDS: + update specify kernel version for nvwa to update + init init nvwa running environment + help, h Shows a list of commands or help for one command + + GLOBAL OPTIONS: + --help, -h show help (default: false) + --version, -v print the version (default: false) + ``` + + Note that global options such as `nvwa --help` and `nvwa --version` are not supported due to parameter parsing restrictions. This problem will be resolved in later versions. + To view the NVWA version installed in the system, run the `rpm -qa nvwa` command. + +- `nvwa update ` + + When the kernel is hot upgraded to a version, the NVWA searches for the kernel image and ramfs in the /boot directory. The kernel must be named in the vmlinuz-\ format, and rootfs in the initramfs-\.img format. + + Note that the upgrade may fail. If the upgrade fails, some processes or services that are dumped will stop running. + + You can run the `uname -r` command to obtain the \ of the current version. You can find all existing versions in the /boot directory by referring to the format of the \. + +- `nvwa init` + + Clears the running information generated by NVWA and modifies the systemd configuration. This command is used to clear the running information before the NVWA is executed or after the execution fails. + +## Restrictions + +1. For services that need to be saved using NVWA, you need to set StandardOutput and StandardError in the configuration. The following uses Redis as an example: + + ```conf + [Unit] + Description=Redis persistent key-value database + After=network.target + [Service] + ExecStart=/usr/bin/redis-server /etc/redis.conf --supervised systemd + Type=notify + User=redis + Group=redis + RuntimeDirectory=redis + RuntimeDirectoryMode=0755 + StandardOutput=file:/root/log1.log + StandardError=file:/root/log2.log + [Install] + WantedBy=multi-user.target + ``` + +2. To use the acceleration feature, you need to modify the cmdline and allocate proper memory. For details, see [NVWA Acceleration Feature Description and Usage](#nvwa-acceleration-feature-description-and-usage). + +3. SELINUX needs to be disabled during the running process. + + Theoretically, you need to disable the NVWA service only after you run the NVWA update command and before you restart the system to restore the process. It is recommended that SELinux be disabled during the entire process. + +## NVWA Acceleration Feature Description and Usage + +1. cpu park + + The cpu park command uses the kexec process to make the CPU stay busy waiting, so as to respond to the interrupt request sent by the primary core more quickly, and reduce the status changes. + + To use cpu park, you need to add "cpuparkmem=0x200000000" to cmdline. 0x200000000 is the start address of the memory that is not used by other programs. cpuparkmem occupies the memory space whose size is about 1 MB from this address. + + Note that if the memory is sufficient, it is recommended that the address range be after 4G(0x100000000). The first 4 GB is usually reserved by each system component, which is prone to conflict. + +2. quick kexec + + quick kexec accelerates image loading using kexec. + + To use quick kexec, you need to enable related options in the configuration file. For more information, see [Configurations](./installation-and-deployment.md#configurations). + +3. pin_memory + + pin memory accelerates the storage and recovery of the CRIU. + + To use pin memory, you need to enable related options in the configuration file. For more information, see [Configurations](./installation-and-deployment.md#configurations). + +## Generated Log Information + +The logs generated by the kernel hot upgrade tool consist of two parts: + +- Logs generated during running + + Run the service `nvwa status` command to view logs. + +- Logs generated while retaining the running information + + The logs are stored in the process/service folder in the path specified by **criu_dir**. diff --git a/docs/en/server/maintenance/syscare/_toc.yaml b/docs/en/server/maintenance/syscare/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..f5d608f4205d2149fdffdd1d5af0331d84b12a98 --- /dev/null +++ b/docs/en/server/maintenance/syscare/_toc.yaml @@ -0,0 +1,15 @@ +label: SysCare User Guide +isManual: true +href: ./syscare-user-guide.md +description: Online hot patching +sections: + - label: SysCare Introduction + href: ./syscare-introduction.md + - label: SysCare Installation + href: ./installing-syscare.md + - label: SysCare Usage + href: ./using-syscare.md + - label: Constraints + href: ./constraints.md + - label: Common Issues and Solutions + href: ./common-issues-and-solutions.md diff --git a/docs/en/server/maintenance/syscare/common-issues-and-solutions.md b/docs/en/server/maintenance/syscare/common-issues-and-solutions.md new file mode 100644 index 0000000000000000000000000000000000000000..0203440f4d006419efe239298ca85ce3a09edf81 --- /dev/null +++ b/docs/en/server/maintenance/syscare/common-issues-and-solutions.md @@ -0,0 +1,19 @@ +# Common Issues and Solutions + +## Issue 1: "alloc upatch module memory failed" + +Possible cause: The SELinux constraint is triggered. + +Solution: Manually add policies according to the error. The policies to be added vary according to the actual situation. For details, see . + +## Issue 2: "patch file error 2" + +Possible cause: The patch cannot be detected. + +Solution: Use another patch. + +## Issue 3: "build project error 11" + +Possible cause: The source package fails to be compiled. + +Solution: Run `rpmbuild -ra *.src.rpm` to check if the source package can be compiled and the compilation dependencies are satisfied. diff --git a/docs/en/server/maintenance/syscare/constraints.md b/docs/en/server/maintenance/syscare/constraints.md new file mode 100644 index 0000000000000000000000000000000000000000..2aae6664362212f6464cf75ddd1a7b56c0f8adca --- /dev/null +++ b/docs/en/server/maintenance/syscare/constraints.md @@ -0,0 +1,37 @@ +# Constraints + +## Version Constraints + +OS version: openEuler 22.03 LTS SP1 or SP2 +Architecture: x86 or AArch64 + +## Application Constraints + +Currently, user-mode patches support only Redis and Nginx. + +Note: + +1. Currently, each software needs to be adapted to process the LINE macro. Currently, only Redis and Nginx are adapted. Other software that is not adapted may cause the patch size to be too large. (Parameters will be introduced in the future to support user adaptation.) +2. Each user-mode live patch can contain only one ELF file. To fix multiple bugs, you can pass the patch files of multiple bug fixes to the patch making parameters to make a live patch for multiple bugs. + +## Language Constraints + +Theoretically, patches are compared at the object file level, which is irrelevant to the programming language. Currently, only the C and C++ languages are tested. + +## Others + +- Only 64-bit OSs are supported. +- Only the ELF format can be hot-patched. Interpreted languages are not supported. +- Only GCC and G++ compilers are supported. +- The compiler must support the `-gdwarf`, `-ffunction-sections`, and `-fdata-sections` parameters. +- The debug information must be in the DWARF format. +- Cross compilation is not supported. +- Source files that are in different paths but have the same file name, same global variables, and same functions cannot be recognized. +- Assembly code, including **.S** files and inline assembly code, cannot be modified. +- External symbols (dynamic library dependencies) cannot be added. +- Multiple patches cannot be applied to the same binary file. +- Mixed compilation of C and C++ is not supported. +- C++ exceptions cannot be modified. +- The `-g3` group section compilation option, specific compilation optimization options, and specific GCC plugins are not supported. +- ifunc cannot be added by using `__attribute__((ifunc("foo")))`. +- TLS variables cannot be added by using `__thread int foo`. diff --git a/docs/en/server/maintenance/syscare/figures/syscare_arch.png b/docs/en/server/maintenance/syscare/figures/syscare_arch.png new file mode 100644 index 0000000000000000000000000000000000000000..e8c931ad3ba6743224ffa133808f8b66239ce486 Binary files /dev/null and b/docs/en/server/maintenance/syscare/figures/syscare_arch.png differ diff --git a/docs/en/server/maintenance/syscare/installing-syscare.md b/docs/en/server/maintenance/syscare/installing-syscare.md new file mode 100644 index 0000000000000000000000000000000000000000..9f80648d6b64c45069b40be475c896d28c48141a --- /dev/null +++ b/docs/en/server/maintenance/syscare/installing-syscare.md @@ -0,0 +1,43 @@ +# Installing SysCare + +This chapter describes how to install SysCare on openEuler. + +## Installing SysCare Core Components + +### Minimum Hardware Requirements + +* 2 CPUs (x86_64 or AArch64) +* 4 GB memory +* 100 GB drive + +### Prerequisites + +1. openEuler 23.09 has been installed. + +### Installing from Source + +Clone the SysCare source code and then compile and install SysCare as follows: + +```shell +git clone https://gitee.com/openeuler/syscare.git +cd syscare +mkdir build +cd build +cmake -DCMAKE_INSTALL_PREFIX=/usr -DKERNEL_VERSION=$(uname -r) .. +make +make install +``` + +### Installing SysCare from a Repository + +If the repository source contains SysCare packages, you can use the `dnf` or `yum` command to download and install them. + +```shell +dnf install syscare syscare-kmod syscare-build syscare-build-ebpf +``` + +### Uninstalling SysCare + +```shell +dnf erase syscare syscare-kmod syscare-build syscare-build-ebpf +``` diff --git a/docs/en/server/maintenance/syscare/syscare-introduction.md b/docs/en/server/maintenance/syscare/syscare-introduction.md new file mode 100644 index 0000000000000000000000000000000000000000..17c05d1368dda9dd18380122ac23d1d1cd3c15b0 --- /dev/null +++ b/docs/en/server/maintenance/syscare/syscare-introduction.md @@ -0,0 +1,22 @@ +# Introduction to SysCare + +## Overview + +SysCare is an online live patching tool that automatically fixes bugs and vulnerabilities in OS components, such as kernels, user-mode services, and dynamic libraries. + +![img](./figures/syscare_arch.png) + +## SysCare Functions + +SysCare supports live patching for kernels and user-mode services: + +1. One-click creation + SysCare is a unified environment for both kernel- and user-mode live patches that ignores differences between patches, ensuring they can be created with just one click. +2. Patch lifecycle operations + SysCare provides a unified patch management interface for users to install, activate, uninstall, and query patches. + +## SysCare Technologies + +1. Unified patches: SysCare masks differences in detail when creating patches, providing a unified management tool to improve O&M efficiency. +2. User-mode live patching: SysCare supports live patching of multi-process and multi-thread services in user mode, which takes effect when a process or thread is started or restarted. +3. Lazy mechanism: SysCare fixes the ptrace defect (all kernel calls are ended) and improves the fix success rate. diff --git a/docs/en/server/maintenance/syscare/syscare-user-guide.md b/docs/en/server/maintenance/syscare/syscare-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..ee022874c6c1a67fa3557df60ca150dcf1d9f69f --- /dev/null +++ b/docs/en/server/maintenance/syscare/syscare-user-guide.md @@ -0,0 +1,3 @@ +# SysCare User Guide + +This document describes how to install and use SysCare on openEuler. diff --git a/docs/en/server/maintenance/syscare/using-syscare.md b/docs/en/server/maintenance/syscare/using-syscare.md new file mode 100644 index 0000000000000000000000000000000000000000..80a4d53cb94165bf789ed8f851cbb49e4f956085 --- /dev/null +++ b/docs/en/server/maintenance/syscare/using-syscare.md @@ -0,0 +1,300 @@ +# Using SysCare + +This chapter describes how to use SysCare on openEuler. + +## Prerequisites + +openEuler 22.03 LTS SP2 has been installed. + +## Using SysCare CLI Tools + +You can use `syscare build` to create patches and use `syscare patch` to manage patches, including installing, activating, deactivating, confirming, and uninstalling patches. + +### Creating Patches + +`syscare-build` is used to create patches, for example: + +```shell +syscare build \ + --patch-name "HP001" \ + --source ./redis-6.2.5-1.src.rpm \ + --debuginfo ./redis-debuginfo-6.2.5-1.x86_64.rpm \ + --output ./output \ + ./0001-Prevent-unauthenticated-client-from-easily-consuming.patch +``` + +### Managing Patches + +The pattern for matching a patch name is **TARGET_PACKAGE_NAME/PATCH_NAME**. If **PATCH_NAME** is unique, **TARGET_PACKAGE_NAME/** can be omitted. UUIDs can also be used to identify packages. + +1. Installing a patch: + + ```shell + syscare apply PATCH_NAME + ``` + +2. Activating a patch: + + ```shell + syscare active PATCH_NAME + ``` + +3. Deactivating a patch: + + ```shell + syscare deactive PATCH_NAME + ``` + +4. Uninstalling/removing a patch: + + ```shell + syscare remove PATCH_NAME + ``` + +5. Confirming a patch: + + ```shell + syscare accept patch-name + ``` + +6. Querying the status of a patch: + + ```shell + syscare status PATCH_NAME + ``` + +7. Querying all SysCare patches: + + ```shell + syscare list + ``` + +## Patch Making Module + +### SysCare Patch Making Tool + +`syscare-build` is a CLI tool that creates kernel- and user-mode live patches from RPM packages. Patches are encapsulated into RPM packages. + +### Command Parameters + +```text +Usage: syscare-build [OPTIONS] --patch-name --source --debuginfo ... + +Arguments: + ... Patch file(s) + +Options: + -n, --patch-name Patch name + --patch-arch Patch architecture [default: x86_64] + --patch-version Patch version [default: 1] + --patch-release Patch release [default: 1] + --patch-description Patch description [default: (none)] + --target-name Patch target name + -t, --target-elfname Patch target executable name + --target-arch parch target architecture + --target-epoch Patch target epoch + --target-version Patch target version + --target-release Patch target release + --target-license Patch target license + -s, --source Source package + -d, --debuginfo Debuginfo package + --workdir Working directory [default: .] + -o, --output Generated patch output directory [default: .] + -j, --jobs Parallel build jobs [default: 96] + --skip-compiler-check Skip compiler version check (not recommended) + --skip-cleanup Skip post-build cleanup + -v, --verbose Provide more detailed info + -h, --help Print help information + -V, --version Print version information +``` + +### Parameters + +|Name|Description|Type|Note| +| ---- | ---- | ---- | ---- | +| *\* |Patch file path|String|Mandatory. The value can be multiple valid paths.| + +### Options + +|Name|Description|Type|Note| +| ---- | ---- | ---- | ---- | +|-n, --patch-name *\*|Patch name|String|Mandatory. The value must comply with the RPM package naming convention.| +|--patch-arch *\*|Patch architecture|String|The default value is the current architectures. The value must comply with the RPM package naming convention.| +|--patch-version *\*|Patch version|String|The default value is **1**. The value must comply with the RPM package naming convention.| +|--patch-release *\*|Patch release|Integer|The default value is **1**. The value must comply with the RPM package naming convention.| +|--patch-description *\*|Patch description|String|The default value is **none**.| +|--target-name *\*|Target software RPM package name|String|The default value is determined by the **src.rpm** package specified by `--source`.| +|--target-arch *\*|Target software RPM package architecture|String|The default value is determined by the **src.rpm** package specified by `--source`.| +|--target-epoch *\*|Target software RPM package epoch|String|The default value is determined by the **src.rpm** package specified by `--source`.| +|--target-version *\*|Target software RPM package version|String|The default value is determined by the **src.rpm** package specified by `--source`.| +|--target-release *\*|Target software RPM package release|String|The default value is determined by the **src.rpm** package specified by `--source`.| +|--target-license *\*|Target software RPM package license|String|The default value is determined by the **src.rpm** package specified by `--source`.| +|-s, --source *\*|Target software **src.rpm** package path|String|Mandatory. The value must be a valid path.| +|-d, --debuginfo *\*|Target software **debuginfo** package path|String|Mandatory. The value must be a valid path.| +|--workdir *\*|Temporary directory|String|The default value is the current directory. The value must be a valid path.| +|-o, --output *\*|Patch output directory|String|The default value is the current directory. The value must be a valid path.| +|-j, --jobs *\*|Number of parallel compilation jobs|Integer|The default value is the number of CPU threads| +|--skip-compiler-check|Skip compiler check|Flag|-| +|--skip-cleanup|Skip temporary file cleanup|Flag|-| +|-v, --verbose|Print detail information|Flag|-| +|-h, --help|Print help information|Flag|-| +|-V, --version|Print version information|Flag|-| + +An example command is as follows: + +```shell +syscare build \ + --patch-name "HP001" \ + --patch-description "CVE-2021-32675 - When parsing an incoming Redis Standard Protocol (RESP) request, Redis allocates memory according to user-specified values which determine the number of elements (in the multi-bulk header) and size of each element (in the bulk header). An attacker delivering specially crafted requests over multiple connections can cause the server to allocate significant amount of memory. Because the same parsing mechanism is used to handle authentication requests, this vulnerability can also be exploited by unauthenticated users." \ + --source ./redis-6.2.5-1.src.rpm \ + --debuginfo ./redis-debuginfo-6.2.5-1.x86_64.rpm \ + --output ./output \ + ./0001-Prevent-unauthenticated-client-from-easily-consuming.patch +``` + +### Patch Output + +- A patch package that contains the binary file of SysCare and meta information. This package is used to install the live patch. +- A patch source package that contains the target software source code and the new patch. This package is used to create live patches for new versions. + +Naming rules: + +- Patch package: patch-*TARGET_SOFTWARE_FULL_NAME*-*PATCH_NAME*-*PATCH_VERSION*-*PATCH_RELEASE*.*PATCH_ARCHITECTURE*.rpm +- Patch source code package: *TARGET_SOFTWARE_FULL_NAME*-*PATCH_NAME*-*PATCH_VERSION*-*PATCH_RELEASE*.*PATCH_ARCHITECTURE*.src.rpm + +### Patch Information + +The patch meta information contains the following fields: + +| Field | Description | +| ----------- | ---------------------- | +| uuid | Patch ID | +| name | Patch name | +| version | Patch version | +| release | Patch release | +| arch | Patch architecture | +| type | Patch type | +| target | Target software name | +| target_elf | Name of the executable file of the target software | +| digest | Patch fingerprint | +| license | Target software license | +| description | Patch description | +| patch| Patch file list | + +Example: + +```text +syscare info redis-6.2.5-1/HP001 +uuid: ec503257-aa75-4abc-9045-c4afdd7ae0f2 +name: HP001 +version: 1 +release: 1 +arch: x86_64 +type: UserPatch +target: redis-6.2.5-1 +target_elf: redis-cli, redis-server, redis-benchmark +digest: 31fc7544 +license: BSD and MIT +description: CVE-2021-32675 - When parsing an incoming Redis Standard Protocol (RESP) request, Redis allocates memory according to user-specified values which determine the number of elements (in the multi-bulk header) and size of each element (in the bulk header). An attacker delivering specially crafted requests over multiple connections can cause the server to allocate significant amount of memory. Because the same parsing mechanism is used to handle authentication requests, this vulnerability can also be exploited by unauthenticated users. +patch: +31fc7544 0001-Prevent-unauthenticated-client-from-easily-consuming.patch +``` + +### Patch Making Process + +1. Prepare the source package (source RPM) and debugging information package (debuginfo RPM) of the target software. + + Example: + + ```shell + yumdownloader kernel --source + + yumdownloader kernel --debuginfo + ``` + +2. Ensure that the related software build dependencies are installed. + + Example: + + ```shell + dnf install make gcc bison flex openssl-devel dwarves python3-devel elfutils-libelf-devel + ``` + +3. Run the `syscare-build` command. + + Example: + + ```shell + syscare build \ + --patch-name HP001 \ + --source kernel-5.10.0-60.66.0.91.oe2203.src.rpm \ + --debuginfo kernel-debuginfo-5.10.0-60.66.0.91.oe2203.x86_64.rpm \ + --output output \ + 001-kernel-patch-test.patch + ``` + + During patch making, a temporary folder whose name starts with **syscare-build** is created in the directory specified by `--workdir` (the current directory by default) to store temporary files and build logs. + + Example: + + ```shell + $ ls -l syscare-build.111602/ + total 100 + -rw-r--r--. 1 dev dev 92303 Nov 12 00:00 build.log + drwxr-xr-x. 6 dev dev 4096 Nov 12 00:00 package + drwxr-xr-x. 4 dev dev 4096 Nov 12 00:00 patch + ``` + + Build logs (**build.log**) are generated in the temporary folder. + + ```shell + $ cat syscare-build.111602/build.log | less + ... + ``` + + If the patch is created successfully and `--skip-compiler-check` is not specified, the temporary folder will be deleted after patch making. + +4. Check the build result. + + Example: + + ```shell + $ ls -l + total 189680 + -rw-r--r--. 1 dev dev 194218767 Nov 12 00:00 kernel-5.10.0-60.91.0.115.oe2203-HP001-1-1.x86_64.src.rpm + -rw-r--r--. 1 dev dev 10937 Nov 12 00:00 patch-kernel-5.10.0-60.91.0.115.oe2203-HP001-1-1.x86_64.rpm + ``` + + In the output: + + **patch-kernel-5.10.0-60.91.0.115.oe2203-HP001-1-1.x86_64.rpm** is the patch package. + + **kernel-5.10.0-60.91.0.115.oe2203-HP001-1-1.x86_64.src.rpm** is the patch source package. + +5. Install the patch. + + ```shell + dnf install patch-xxx.rpm + ``` + + After the patch is installed, files in the patch are stored in the **/usr/lib/syscare/patches/target_software_package_name/patch_name** directory + +6. Uninstall the patch. + + ```shell + dnf remove patch-xxx + ``` + + The patch package will be uninstalled when the patch is beyond the **ACTIVED** state. + +### Error Handling + +If an error occurs, see the build logs: + +Error output example: + +```text +... +Building patch, this may take a while +ERROR: Process '/usr/libexec/syscare/upatch-build' exited unsuccessfully, exit_code=255 +``` diff --git a/docs/en/server/maintenance/sysmonitor/_toc.yaml b/docs/en/server/maintenance/sysmonitor/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..da40b574779f3e054bc5d77cf64d3774cacec841 --- /dev/null +++ b/docs/en/server/maintenance/sysmonitor/_toc.yaml @@ -0,0 +1,4 @@ +label: sysmonitor User Guide +isManual: true +href: ./sysmonitor-user-guide.md +description: sysmonitor tracks exceptions in the OS during runtime. diff --git a/docs/en/server/maintenance/sysmonitor/figures/sysmonitor_functions.png b/docs/en/server/maintenance/sysmonitor/figures/sysmonitor_functions.png new file mode 100644 index 0000000000000000000000000000000000000000..e9655456ebce192d196e5f55c5fc09c03fa440d8 Binary files /dev/null and b/docs/en/server/maintenance/sysmonitor/figures/sysmonitor_functions.png differ diff --git a/docs/en/server/maintenance/sysmonitor/sysmonitor-user-guide.md b/docs/en/server/maintenance/sysmonitor/sysmonitor-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..482aa99c8181f4df29374b6601a78217a70c612b --- /dev/null +++ b/docs/en/server/maintenance/sysmonitor/sysmonitor-user-guide.md @@ -0,0 +1,797 @@ +# sysmonitor User Guide + +## Introduction + +The system monitor (sysmonitor) daemon monitors exceptions that occur during OS running and records the exceptions in the system log file **/var/log/sysmonitor.log**. sysmonitor runs as a service. You can run the `systemctl start|stop|restart|reload sysmonitor` command to start, stop, restart, and reload the service. You are advised to deploy sysmonitor to locate system exceptions. + +![](./figures/sysmonitor_functions.png) + +### Precautions + +- sysmonitor cannot run concurrently. +- Ensure that all configuration files are valid. Otherwise, the monitoring service may be abnormal. +- The root privilege is required for sysmonitor service operations, configuration file modification, and log query. The **root** user has the highest permission in the system. When performing operations as the **root** user, follow the operation guide to avoid system management and security risks caused by improper operations. + +### Configuration Overview + +Configuration file **/etc/sysconfig/sysmonitor** of sysmonitor defines the monitoring period of each monitoring item and specifies whether to enable monitoring. Spaces are not allowed between the configuration item, equal sign (=), and configuration value, for example, **PROCESS_MONITOR="on"**. + +Configuration description + +| Item | Description | Mandatory| Default Value | +| ------------------------- | ------------------------------------------------------------ | -------- | -------------------------------------- | +| PROCESS_MONITOR | Whether to enable key process monitoring. The value can be **on** or **off**. | No | on | +| PROCESS_MONITOR_PERIOD | Monitoring period on key processes, in seconds. | No | 3 | +| PROCESS_RECALL_PERIOD | Interval for attempting to restart a key process after the process fails to be recovered, in minutes. The value can be an integer ranging from 1 to 1440.| No | 1 | +| PROCESS_RESTART_TIMEOUT | Timeout interval for recovering a key process service from an exception, in seconds. The value can be an integer ranging from 30 to 300.| No | 90 | +| PROCESS_ALARM_SUPRESS_NUM | Number of alarm suppression times when the key process monitoring configuration uses the alarm command to report alarms. The value is a positive integer.| No | 5 | +| FILESYSTEM_MONITOR | Whether to enable ext3 and ext4 file system monitoring. The value can be **on** or **off**. | No | on | +| DISK_MONITOR | Whether to enable drive partition monitoring. The value can be **on** or **off**. | No | on | +| DISK_MONITOR_PERIOD | Drive monitoring period, in seconds. | No | 60 | +| INODE_MONITOR | Whether to enable drive inode monitoring. The value can be **on** or **off**. | No | on | +| INODE_MONITOR_PERIOD | Drive inode monitoring period, in seconds. | No | 60 | +| NETCARD_MONITOR | Whether to enable NIC monitoring. The value can be **on** or **off**. | No | on | +| FILE_MONITOR | Whether to enable file monitoring. The value can be **on** or **off**. | No | on | +| CPU_MONITOR | Whether to enable CPU monitoring. The value can be **on** or **off**. | No | on | +| MEM_MONITOR | Whether to enable memory monitoring. The value can be **on** or **off**. | No | on | +| PSCNT_MONITOR | Whether to enable process count monitoring. The value can be **on** or **off**. | No | on | +| FDCNT_MONITOR | Whether to enable file descriptor (FD) count monitoring. The value can be **on** or **off**. | No | on | +| CUSTOM_DAEMON_MONITOR | Whether to enable custom daemon item monitoring. The value can be **on** or **off**. | No | on | +| CUSTOM_PERIODIC_MONITOR | Whether to enable custom periodic item monitoring. The value can be **on** or **off**. | No | on | +| IO_DELAY_MONITOR | Whether to enable local drive I/O latency monitoring. The value can be **on** or **off**. | No | off | +| PROCESS_FD_NUM_MONITOR | Whether to enable process FD count monitoring. The value can be **on** or **off**. | No | on | +| PROCESS_MONITOR_DELAY | Whether to wait until all monitoring items are normal when sysmonitor is started. The value can be **on** (wait) or **off** (do not wait).| No | on | +| NET_RATE_LIMIT_BURST | NIC route information printing rate, that is, the number of logs printed per second. | No | 5
Valid range: 0 to 100 | +| FD_MONITOR_LOG_PATH | FD monitoring log file | No | /var/log/sysmonitor.log| +| ZOMBIE_MONITOR | Whether to monitor zombie processes | No | off | +| CHECK_THREAD_MONITOR | Whether to enable internal thread self-healing. The value can be **on** or **off**. | No | on | +| CHECK_THREAD_FAILURE_NUM | Number of internal thread self-healing checks in a period. | No | 3
Valid range: 2 to 10 | + +- After modifying the **/etc/sysconfig/sysmonitor** configuration file, restart the sysmonitor service for the configurations to take effect. +- If an item is not configured in the configuration file, it is enabled by default. +- After the internal thread self-healing function is enabled, if a sub-thread of the monitoring item is suspended and the number of checks in a period exceeds the configured value, the sysmonitor service is restarted for restoration. The configuration is reloaded. The configured key process monitoring and customized monitoring are restarted. If this function affects user experience, you can disable it. + +### Command Reference + +- Start sysmonitor. + +```shell +systemctl start sysmonitor +``` + +- Stop sysmonitor. + +```shell +systemctl stop sysmonitor +``` + +- Restart sysmonitor. + +```shell +systemctl restart sysmonitor +``` + +- Reload sysmonitor for the modified configurations to take effect. + +```shell +systemctl reload sysmonitor +``` + +### Monitoring Logs + +By default, logs is split and dumped to prevent the **sysmonitor.log** file from getting to large. Logs are dumped to a drive directory. In this way, a certain number of logs can be retained. + +The configuration file is **/etc/rsyslog.d/sysmonitor.conf**. Because this rsyslog configuration file is added, after sysmonitor is installed for the first time, you need to restart the rsyslog service to make the sysmonitor log configuration take effect. + +```text +$template sysmonitorformat,"%TIMESTAMP:::date-rfc3339%|%syslogseverity-text%|%msg%\n" + +$outchannel sysmonitor, /var/log/sysmonitor.log, 2097152, /usr/libexec/sysmonitor/sysmonitor_log_dump.sh +if ($programname == 'sysmonitor' and $syslogseverity <= 6) then { +:omfile:$sysmonitor;sysmonitorformat +stop +} + +if ($msg contains 'Time has been changed') then { +:omfile:$sysmonitor;sysmonitorformat +stop +} + +if ($programname == 'sysmonitor' and $syslogseverity > 6) then { +/dev/null +stop +} +``` + +## ext3/ext4 Filesystem Monitoring + +### Introduction + +A fault in the filesystem may trigger I/O operation errors, which further cause OS faults. File system fault detection can detect the faults in real time so that system administrators or users can rectify them in a timely manner. + +### Configuration File Description + +None + +### Exception Logs + +For a file system to which the errors=remount-ro mounting option is added, if the ext3 or ext4 file system is faulty, the following exception information is recorded in the **sysmonitor.log** file: + +```text +info|sysmonitor[127]: loop0 filesystem error. Remount filesystem read-only. +``` + +In other exception scenarios, if the ext3 or ext4 file system is faulty, the following exception information is recorded in the **sysmonitor.log** file: + +```text +info|sysmonitor[127]: fs_monitor_ext3_4: loop0 filesystem error. flag is 1879113728. +``` + +## Key Processing Monitoring + +### Introduction + +Key processes in the system are periodically monitored. When a key process exits abnormally, sysmonitor automatically attempts to recover the key process. If the recovery fails, alarms can be reported. The system administrator can be promptly notified of the abnormal process exit event and whether the process is restarted. Fault locating personnel can locate the time when the process exits abnormally from logs. + +### Configuration File Description + +The configuration file directory is **/etc/sysmonitor/process**. Each process or module corresponds to a configuration file. + +```text +USER=root +NAME=irqbalance +RECOVER_COMMAND=systemctl restart irqbalance +MONITOR_COMMAND=systemctl status irqbalance +STOP_COMMAND=systemctl stop irqbalance +``` + +The configuration items are as follows: + +| Item | Description | Mandatory| Default Value | +| ---------------------- | ------------------------------------------------------------ | -------- | --------------------------------------------------- | +| NAME | Process or module name | Yes | None | +| RECOVER_COMMAND | Recovery command | No | None | +| MONITOR_COMMAND | Monitoring command
If the command output is 0, the process is normal. If the command output is greater than 0, the process is abnormal.| No | pgrep -f $(which xxx)
*xxx* is the process name configured in the **NAME** field.| +| STOP_COMMAND | Stopping command | No | None | +| USER | User name
User for executing the monitoring, recovery, and stopping commands or scripts | No | If this item is left blank, the **root** user is used by default. | +| CHECK_AS_PARAM | Parameter passing switch
If this item is on, the return value of **MONITOR_COMMAND** is transferred to the **RECOVER_COMMAND** command or script as an input parameter. If this item is set to off or other values, the function is disabled.| No | None | +| MONITOR_MODE | Monitoring mode
- **parallel** or **serial**
| No | serial | +| MONITOR_PERIOD | Monitoring period
- Parallel monitoring period
- This item does not take effect when the monitoring mode is **serial**.| No | 3 | +| USE_CMD_ALARM | Alarm mode
If this parameter is set to **on** or **ON**, alarms are reported using the alarm reporting command. | No | None | +| ALARM_COMMAND | Alarm reporting command | No | None | +| ALARM_RECOVER_COMMAND | Alarm recovery command | No | No | + +- After modifying the configuration file for monitoring key processes, run `systemctl reload sysmonitor`. The new configuration takes effect after a monitoring period. +- The recovery command and monitoring command must not block. Otherwise, the monitoring thread of the key process becomes abnormal. +- When the recovery command is executed for more than 90 seconds, the stopping command is executed to stop the process. +- If the recovery command is empty or not configured, the monitoring command does not attempt to recover the key process when detecting that the key process is abnormal. +- If a key process is abnormal and fails to be started for three consecutive times, the process is started based on the period specified by **PROCESS_RECALL_PERIOD** in the global configuration file. +- If the monitored process is not a daemon process, **MONITOR_COMMAND** is mandatory. +- If the configured key service does not exist in the current system, the monitoring does not take effect and the corresponding information is printed in the log. If a fatal error occurs in other configuration items, the default configuration is used and no error is reported. +- The permission on the configuration file is 600. You are advised to set the monitoring item to the **service** type of systemd (for example, **MONITOR_COMMAND=systemctl status irqbalance**). If a process is monitored, ensure that the **NAME** field is an absolute path. +- The restart, reload, and stop of sysmonitor do not affect the monitored processes or services. +- If **USE_CMD_ALARM** is set to **on**, you must ensure the validiy of **ALARM_COMMAND** and **ALARM_RECOVER_COMMAND**. If **ALARM_COMMAND** or **ALARM_RECOVER_COMMAND** is empty or not configured, no alarm is reported. +- The security of user-defined commands, such as the monitoring, recovery, stopping, alarm reporting, and alarm recovery commands, is ensured by users. Commands are executed by the user **root**. You are advised to set the script command permission to be used only by the user **root** to prevent privilege escalation for common users. +- If the length of the monitoring command cannot be greater than 200 characters. Otherwise, the process monitoring fails to be added. +- When the recovery command is set to a systemd service restart command (for example, **RECOVER_COMMAND=systemctl restart irqbalance**), check whether the recovery command conflicts with the open source systemd service recovery mechanism. Otherwise, the behavior of key processes may be affected after exceptions occur. +- The processes started by the sysmonitor service are in the same cgroup as the sysmonitor service, and resources cannot be restricted separately. Therefore, you are advised to use the open source systemd mechanism to recover the processes. + +### Exception Logs + +- **RECOVER_COMMAND** configured + + If a process or module exception is detected, the following exception information is recorded in the **/var/log/sysmonitor.log** file: + + ```text + info|sysmonitor[127]: irqbalance is abnormal, check cmd return 1, use "systemctl restart irqbalance" to recover + ``` + + If the process or module recovers, the following information is recorded in the **/var/log/sysmonitor.log** file: + + ```text + info|sysmonitor[127]: irqbalance is recovered + ``` + +- **RECOVER_COMMAND** not configured + + If a process or module exception is detected, the following exception information is recorded in the **/var/log/sysmonitor.log** file: + + ```text + info|sysmonitor[127]: irqbalance is abnormal, check cmd return 1, recover cmd is null, will not recover + ``` + + If the process or module recovers, the following information is recorded in the **/var/log/sysmonitor.log** file: + + ```text + info|sysmonitor[127]: irqbalance is recovered + ``` + +## File Monitoring + +### Introduction + +If key system files are deleted accidentally, the system may run abnormally or even break down. Through file monitoring, you can learn about the deletion of key files or the addition of malicious files in the system in a timely manner, so that administrators and users can learn and rectify faults in a timely manner. + +### Configuration File Description + +The configuration file is **/etc/sysmonitor/file**. Each monitoring configuration item occupies a line. A monitoring configuration item contains the file (directory) and event to be monitored. The file (directory) to be monitored is an absolute path. The file (directory) to be monitored and the event to be monitored are separated by one or more spaces. + +The file monitoring configuration items can be added to the **/etc/sysmonitor/file.d** directory. The configuration method is the same as that of the **/etc/sysmonitor/file** directory. + +- Due to the log length limit, it is recommended that the absolute path of a file or directory be less than 223 characters. Otherwise, the printed logs may be incomplete. + +- Ensure that the path of the monitored file is correct. If the configured file does not exist or the path is incorrect, the file cannot be monitored. + +- Due to the path length limit of the system, the absolute path of the monitored file or directory must be less than 4096 characters. + +- Directories and regular files can be monitored. **/proc**, **/proc/\***, **/dev**, **/dev/\***, **/sys**, **/sys/\***, pipe files, or socket files cannot be monitored. + +- Only deletion events can be monitored in **/var/log** and **/var/log/\***. + +- If multiple identical paths exist in the configuration file, the first valid configuration takes effect. In the log file, you can see messages indicating that the identical paths are ignored. + +- Soft links cannot be monitored. When a hard link file deletion event is configured, the event is printed only after the file and all its hard links are deleted. + +- When a monitored event occurs after the file monitoring is successfully added, the monitoring log records the absolute path of the configured file. + +- Currently, directories cannot be monitored recursively. The configured directory is monitored but not its subdirectories. + +- The events to be monitored are configured using bitmaps as follows. + +```text + ------------------------------- + | 11~32 | 10 | 9 | 1~8 | + ------------------------------- +``` + +Each bit in the event bitmap represents an event. If bit _n_ is set to 1, the event corresponding to bit _n_ is monitored. The hexadecimal number corresponding to the monitoring bitmap is the event monitoring item written to the configuration file. + +| Item| Description | Mandatory| +| ------ | ------------------ | -------- | +| 1~8 | Reserved | No | +| 9 | File or directory addition event| Yes | +| 10 | File or directory deletion event| Yes | +| 11~32 | Reserved | No | + +- After modifying the file monitoring configuration file, run `systemctl reload sysmonitor`. The new configuration takes effect within 60 seconds. +- Strictly follow the preceding rules to configure events to be monitored. If the configuration is incorrect, the events cannot be monitored. If an event to be monitored in the configuration item is empty, only the deletion event is monitored by default, that is, **0x200**. +- After a file or directory is deleted, the deletion event is reported only when all processes that open the file stop. +- If a monitored a is modified by `vi` or `sed`, "File XXX may have been changed" is recorded in the monitoring log. +- Currently, file addition and deletion events can be monitored, that is, the ninth and tenth bits take effect. Other bits are reserved and do not take effect. If a reserved bit is configured, the monitoring log displays a message indicating that the event monitoring is incorrectly configured. + +**Example** + +Monitor the subdirectory addition and deletion events in **/home**. The lower 12-bit bitmap is 001100000000. The configuration is as follows: + +```text +/home 0x300 +``` + +Monitor the file deletion events of **/etc/ssh/sshd_config**. The lower 12-bit bitmap is 001000000000. The configuration is as follows: + +```text +/etc/sshd/sshd_config 0x200 +``` + +### Exception Logs + +If a configured event occurs to the monitored file, the following information is displayed in the **/var/log/sysmonitor.log** file: + +```text +info|sysmonitor[127]: 1 events queued +info|sysmonitor[127]: 1th events handled +info|sysmonitor[127]: Subfile "111" under "/home" was added. +``` + +## Drive Partition Monitoring + +### Introduction + +The system periodically monitors the drive partitions mounted to the system. When the drive partition usage is greater than or equal to the configured alarm threshold, the system records a drive space alarm. When the drive partition usage falls below the configured alarm recovery threshold, a drive space recovery alarm is recorded. + +### Configuration File Description + +The configuration file is **/etc/sysmonitor/disk**. + +```text +DISK="/var/log" ALARM="90" RESUME="80" +DISK="/" ALARM="95" RESUME="85" +``` + +| Item| Description | Mandatory| Default Value| +| ------ | ---------------------- | -------- | ------ | +| DISK | Mount directory | Yes | None | +| ALARM | Integer indicating the drive space alarm threshold| No | 90 | +| RESUME | Integer indicating the drive space alarm recovery threshold| No | 80 | + +- After modifying the configuration file for drive space monitoring, run `systemctl reload sysmonitor`. The new configuration takes effect after a monitoring period. +- If a mount directory is configured repeatedly, the last configuration item takes effect. +- The value of **ALARM** must be greater than that of **RESUME**. +- Only the mount point or the drive partition of the mount point can be monitored. +- When the CPU usage and I/O usage are high, the `df` command execution may time out. As a result, the drive usage cannot be obtained. +- If a drive partition is mounted to multiple mount points, an alarm is reported for each mount point. + +### Exception Logs + +If a drive space alarm is detected, the following information is displayed in the **/var/log/sysmonitor.log** file: + +```text +warning|sysmonitor[127]: report disk alarm, /var/log used:90% alarm:90% +info|sysmonitor[127]: report disk recovered, /var/log used:4% resume:10% +``` + +## NIC Status Monitoring + +### Introduction + +During system running, the NIC status or IP address may change due to human factors or exceptions. You can monitor the NIC status and IP address changes to detect exceptions in a timely manner and locate exception causes. + +### Configuration File Description + +The configuration file is **/etc/sysmonitor/network**. + +```text +#dev event +eth1 UP +``` + +The following table describes the configuration items. +| Item| Description | Mandatory| Default Value | +| ------ | ------------------------------------------------------------ | -------- | ------------------------------------------------- | +| dev | NIC name | Yes | None | +| event | Event to be monitored. The value can be **UP**, **DOWN**, **NEWADDR**, or **DELADDR**.
- UP: The NIC is up.
- DOWN: The NIC is down.
- NEWADDR: An IP address is added.
- DELADDR: An IP address is deleted.| No | If this item is empty, **UP**, **DOWN**, **NEWADDR**, and **DELADDR** are monitored.| + +- After modifying the configuration file for NIC monitoring, run `systemctl reload sysmonitor` for the new configuration to take effect. +- The **UP** and **DOWN** status of virtual NICs cannot be monitored. +- Ensure that each line in the NIC monitoring configuration file contains less than 4096 characters. Otherwise, a configuration error message will be recorded in the monitoring log. +- By default, all events of all NICs are monitored. That is, if no NIC monitoring is configured, the **UP**, **DOWN**, **NEWADDR**, and **DELADDR** events of all NICs are monitored. +- If a NIC is configured but no event is configured, all events of the NIC are monitored by default. +- The events of route addition can be recorded five times per second. You can change the number of times by setting **NET_RATE_LIMIT_BURST** in **/etc/sysconfig/sysmonitor**. + +### Exception Logs + +If a NIC event is detected, the following information is displayed in the **/var/log/sysmonitor.log** file: + +```text +info|sysmonitor[127]: lo: ip[::1] prefixlen[128] is added, comm: (ostnamed)[1046], parent comm: syst emd[1] +info|sysmonitor[127]: lo: device is up, comm: (ostnamed)[1046], parent comm: systemd[1] +``` + +If a route event is detected, the following information is displayed in the **/var/log/sysmonitor.log** file: + +```text +info|sysmonitor[881]: Fib4 replace table=255 192.168.122.255/32, comm: daemon-init[1724], parent com m: systemd[1] +info|sysmonitor[881]: Fib4 replace table=254 192.168.122.0/24, comm: daemon-init[1724], parent comm: systemd[1] +info|sysmonitor[881]: Fib4 replace table=255 192.168.122.0/32, comm: daemon-init[1724], parent comm: systemd[1] +info|sysmonitor[881]: Fib6 replace fe80::5054:ff:fef6:b73e/128, comm: kworker/1:3[209], parent comm: kthreadd[2] +``` + +## CPU Monitoring + +### Introduction + +The system monitors the global CPU usage or the CPU usage in a specified domain. When the CPU usage exceeds the configured alarm threshold, the system runs the configured log collection command. + +### Configuration File Description + +The configuration file is **/etc/sysmonitor/cpu**. + +When the global CPU usage of the system is monitored, an example of the configuration file is as follows: + +```text +# cpu usage alarm percent +ALARM="90" + +# cpu usage alarm resume percent +RESUME="80" + +# monitor period (second) +MONITOR_PERIOD="60" + +# stat period (second) +STAT_PERIOD="300" + +# command executed when cpu usage exceeds alarm percent +REPORT_COMMAND="" +``` + +When the CPU usage of a specific domain is monitored, an example of the configuration file is as follows: + +```text +# monitor period (second) +MONITOR_PERIOD="60" + +# stat period (second) +STAT_PERIOD="300" + +DOMAIN="0,1" ALARM="90" RESUME="80" +DOMAIN="2,3" ALARM="50" RESUME="40" + +# command executed when cpu usage exceeds alarm percent +REPORT_COMMAND="" +``` + +| Item | Description | Mandatory| Default Value| +| -------------- | ------------------------------------------------------------ | -------- | ------ | +| ALARM | Number greater than 0, indicating the CPU usage alarm threshold | No | 90 | +| RESUME | Number greater than or equal to 0, indicating the CPU usage alarm recovery threshold | No | 80 | +| MONITOR_PERIOD | Monitoring period, in seconds. The value is greater than 0. | No | 60 | +| STAT_PERIOD | Statistical period, in seconds. The value is greater than 0. | No | 300 | +| DOMAIN | CPU IDs in the domain, represented by decimal numbers
- CPU IDs can be enumerated and separated by commas, for example, **1,2,3**. CPU IDs can be specified as a range in the format of _X_-_Y_, for example, **0-2**. The two representations can be used together, for example, **0, 1, 2-3** or **0-1, 2-3**. Spaces or other characters are not allowed.
- Each monitoring domain has an independent configuration item. Each configuration item supports a maximum of 256 CPUs. A CPU ID must be unique in a domain and across domains.| No | None | +| REPORT_COMMAND | Command for collecting logs after the CPU usage exceeds the alarm threshold | No | None | + +- After modifying the configuration file for CPU monitoring, run `systemctl reload sysmonitor`. The new configuration takes effect after a monitoring period. +- The value of **ALARM** must be greater than that of **RESUME**. +- After the CPU domain monitoring is configured, the global average CPU usage of the system is not monitored, and the separately configured **ALARM** and **RESUME** values do not take effect. +- If the configuration of a monitoring domain is invalid, CPU monitoring is not performed at all. +- All CPUs configured in **DOMAIN** must be online. Otherwise, the domain cannot be monitored. +- The command of **REPORT_COMMAND** cannot contain insecure characters such as **&**, **;**, and **>**, and the total length cannot exceed 159 characters. Otherwise, the command cannot be executed. +- Ensure the security and validity of **REPORT_COMMAND**. sysmonitor is responsible only for running the command as the **root** user. +- **REPORT_COMMAND** must not block. When the execution time of the command exceeds 60s, the sysmonitor forcibly stops the execution. +- Even if the CPU usage of multiple domains exceeds the threshold in a monitoring period, **REPORT_COMMAND** is executed only once. + +### Exception Logs + +If a global CPU usage alarm is detected or cleared and the log collection command is configured, the following information is displayed in the **/var/log/sysmonitor.log** file: + +```text +info|sysmonitor[127]: CPU usage alarm: 91.3% +info|sysmonitor[127]: cpu monitor: execute REPORT_COMMAND[sysmoniotrcpu] successfully +info|sysmonitor[127]: CPU usage resume 70.1% +``` + +If a domain average CPU usage alarm is detected or cleared and the log collection command is configured, the following information is displayed in the **/var/log/sysmonitor.log** file: + +```text +info|sysmonitor[127]: CPU 1,2,3 usage alarm: 91.3% +info|sysmonitor[127]: cpu monitor: execute REPORT_COMMAND[sysmoniotrcpu] successfully +info|sysmonitor[127]: CPU 1,2,3 usage resume 70.1% +``` + +## Memory Monitoring + +### Introduction + +Monitors the system memory usage and records logs when the memory usage exceeds or falls below the threshold. + +### Configuration File Description + +The configuration file is **/etc/sysmonitor/memory**. + +```text +# memory usage alarm percent +ALARM="90" + +# memory usage alarm resume percent +RESUME="80" + +# monitor period(second) +PERIOD="60" +``` + +### Configuration Item Description + +| Item| Description | Mandatory| Default Value| +| ------ | ----------------------------- | -------- | ------ | +| ALARM | Number greater than 0, indicating the memory usage alarm threshold | No | 90 | +| RESUME | Number greater than or equal to 0, indicating the memory usage alarm recovery threshold| No | 80 | +| PERIOD | Monitoring period, in seconds. The value is greater than 0. | No | 60 | + +- After modifying the configuration file for memory monitoring, run `systemctl reload sysmonitor`. The new configuration takes effect after a monitoring period. +- The value of **ALARM** must be greater than that of **RESUME**. +- The average memory usage in three monitoring periods is used to determine whether an alarm is reported or cleared. + +### Exception Logs + +If a memory alarm is detected, sysmonitor obtains the **/proc/meminfo** information and prints the information in the **/var/log/sysmonitor.log** file. The information is as follows: + +```text +info|sysmonitor[127]: memory usage alarm: 90% +info|sysmonitor[127]:---------------show /proc/meminfo: --------------- +info|sysmonitor[127]:MemTotal: 3496388 kB +info|sysmonitor[127]:MemFree: 2738100 kB +info|sysmonitor[127]:MemAvailable: 2901888 kB +info|sysmonitor[127]:Buffers: 165064 kB +info|sysmonitor[127]:Cached: 282360 kB +info|sysmonitor[127]:SwapCached: 4492 kB +...... +info|sysmonitor[127]:---------------show_memory_info end. --------------- +``` + +If the following information is printed, sysmonitor runs `echo m > /proc/sysrq-trigger` to export memory allocation information. You can view the information in **/var/log/messages**. + +```text +info|sysmonitor[127]: sysrq show memory ifno in message. +``` + +When the alarm is recovered, the following information is displayed: + +```text +info|sysmonitor[127]: memory usage resume: 4.6% +``` + +## Process and Thread Monitoring + +### Introduction + +Monitors the number of processes and threads. When the total number of processes or threads exceeds or falls below the threshold, a log is recorded or an alarm is reported. + +### Configuration File Description + +The configuration file is **/etc/sysmonitor/pscnt**. + +```text +# number of processes(include threads) when alarm occur +ALARM="1600" + +# number of processes(include threads) when alarm resume +RESUME="1500" + +# monitor period(second) +PERIOD="60" + +# process count usage alarm percent +ALARM_RATIO="90" + +# process count usage resume percent +RESUME_RATIO="80" + +# print top process info with largest num of threads when threads alarm +# (range: 0-1024, default: 10, monitor for thread off:0) +SHOW_TOP_PROC_NUM="10" +``` + +| Item | Description | Mandatory| Default Value| +| ----------------- | ------------------------------------------------------------ | -------- | ------ | +| ALARM | Integer greater than 0, indicating the process count alarm threshold | No | 1600 | +| RESUME | Integer greater than or equal to 0, indicating the process count alarm recovery threshold | No | 1500 | +| PERIOD | Monitoring period, in seconds. The value is greater than 0. | No | 60 | +| ALARM_RATIO | Number greater than 0 and less than or equal to 100. Process count alarm threshold. | No | 90 | +| RESUME_RATIO | Number greater than 0 and less than or equal to 100. Process count alarm recovery threshold, which must be less than **ALARM_RATIO**.| No | 80 | +| SHOW_TOP_PROC_NUM | Whether to use the latest `top` information about threads | No | 10 | + +- After modifying the configuration file for process count monitoring, run `systemctl reload sysmonitor`. The new configuration takes effect after a monitoring period. +- The value of **ALARM** must be greater than that of **RESUME**. +- The process count alarm threshold is the larger between **ALARM** and **ALARM_RATIO** in **/proc/sys/kernel/pid_max**. The alarm recovery threshold is the larger of **RESUME** and **RESUME_RATIO** in **/proc/sys/kernel/pid_max**. +- The thread count alarm threshold is the larger between **ALARM** and **ALARM_RATIO** in **/proc/sys/kernel/threads-max**. The alarm recovery threshold is the larger of **RESUME** and **RESUME_RATIO** in **/proc/sys/kernel/threads-max**. +- The value of **SHOW_TOP_PROC_NUM** ranges from 0 to 1024. 0 indicates that thread monitoring is disabled. A larger value, for example, 1024, indicates that thread alarms will be generated in the environment. If the alarm threshold is high, the performance is affected. You are advised to set this parameter to the default value 10 or a smaller value. If the impact is huge, you are advised to set this parameter to 0 to disable thread monitoring. +- The value of **PSCNT_MONITOR** in **/etc/sysconfig/sysmonitor** and the value of **SHOW_TOP_PROC_NUM** in **/etc/sysmonitor/pscnt** determine whether thread monitoring is enabled. + - If **PSCNT_MONITOR** is on and **SHOW_TOP_PROC_NUM** is set to a valid value, thread monitoring is enabled. + - If **PSCNT_MONITOR** is on and **SHOW_TOP_PROC_NUM** is 0, thread monitoring is disabled. + - If **PSCNT_MONITOR** is off, thread monitoring is disabled. +- When a process count alarm is generated, the system FD usage information and memory information (**/proc/meminfo**) are printed. +- When a thread count alarm is generated, the total number of threads, `top` process information, number of processes in the current environment, number of system FDs, and memory information (**/proc/meminfo**) are printed. +- If system resources are insufficient before a monitoring period ends, for example, the thread count exceeds the maximum number allowed, the monitoring cannot run properly due to resource limitation. As a result, the alarm cannot be generated. + +### Exception Logs + +If a process count alarm is detected, the following information is displayed in the **/var/log/sysmonitor.log** file: + +```text +info|sysmonitor[127]:---------------process count alarm start: --------------- +info|sysmonitor[127]: process count alarm:1657 +info|sysmonitor[127]: process count alarm, show sys fd count: 2592 +info|sysmonitor[127]: process count alarm, show mem info +info|sysmonitor[127]:---------------show /proc/meminfo: --------------- +info|sysmonitor[127]:MemTotal: 3496388 kB +info|sysmonitor[127]:MemFree: 2738100 kB +info|sysmonitor[127]:MemAvailable: 2901888 kB +info|sysmonitor[127]:Buffers: 165064 kB +info|sysmonitor[127]:Cached: 282360 kB +info|sysmonitor[127]:SwapCached: 4492 kB +...... +info|sysmonitor[127]:---------------show_memory_info end. --------------- +info|sysmonitor[127]:---------------process count alarm end: --------------- +``` + +If a process count recovery alarm is detected, the following information is displayed in the **/var/log/sysmonitor.log** file: + +```text +info|sysmonitor[127]: process count resume: 1200 +``` + +If a thread count alarm is detected, the following information is displayed in the **/var/log/sysmonitor.log** file: + +```text +info|sysmonitor[127]:---------------threads count alarm start: --------------- +info|sysmonitor[127]:threads count alarm: 273 +info|sysmonitor[127]:open threads most 10 processes is [top1:pid=1756900,openthreadsnum=13,cmd=/usr/bin/sysmonitor --daemon] +info|sysmonitor[127]:open threads most 10 processes is [top2:pid=3130,openthreadsnum=13,cmd=/usr/lib/gassproxy -D] +..... +info|sysmonitor[127]:---------------threads count alarm end. --------------- +``` + +## System FD Count Monitoring + +### Introduction + +Monitors the number of system FDs. When the total number of system FDs exceeds or is less than the threshold, a log is recorded. + +### Configuration File Description + +The configuration file is **/etc/sysmonitor/sys_fd_conf**. + +```text +# system fd usage alarm percent +SYS_FD_ALARM="80" +# system fd usage alarm resume percent +SYS_FD_RESUME="70" +# monitor period (second) +SYS_FD_PERIOD="600" +``` + +Configuration items: + +| Item | Description | Mandatory| Default Value| +| ------------- | --------------------------------------------------------- | -------- | ------ | +| SYS_FD_ALARM | Integer greater than 0 and less than 100, indicating the alarm threshold of the percentage of the total number of FDs and the maximum number of FDs allowed.| No | 80 | +| SYS_FD_RESUME | Integer greater than 0 and less than 100, indicating the alarm recovery threshold of the percentage of the total number of FDs and the maximum number of FDs allowed.| No | 70 | +| SYS_FD_PERIOD | Integer between 100 and 86400, indicating the monitor period in seconds | No | 600 | + +- After modifying the configuration file for FD count monitoring, run `systemctl reload sysmonitor`. The new configuration takes effect after a monitoring period. +- The value of **SYS_FD_ALARM** must be greater than that of **SYS_FD_RESUME**. If the value is invalid, the default value is used and a log is recorded. + +### Exception Logs + +An FD count alarm is recorded in the monitoring logs when detected. The following information is displayed in the **/var/log/sysmonitor.log** file: + +```text +info|sysmonitor[127]: sys fd count alarm: 259296 +``` + +When a system FD usage alarm is generated, the top three processes that use the most FDs are printed. + +```text +info|sysmonitor[127]:open fd most three processes is:[top1:pid=23233,openfdnum=5000,cmd=/home/openfile] +info|sysmonitor[127]:open fd most three processes is:[top2:pid=23267,openfdnum=5000,cmd=/home/openfile] +info|sysmonitor[127]:open fd most three processes is:[top3:pid=30144,openfdnum=5000,cmd=/home/openfile] +``` + +## Drive Inode Monitoring + +### Introduction + +Periodically monitors the inodes of mounted drive partitions. When the drive partition inode usage is greater than or equal to the configured alarm threshold, the system records a drive inode alarm. When the drive inode usage falls below the configured alarm recovery threshold, a drive inode recovery alarm is recorded. + +### Configuration File Description + +The configuration file is **/etc/sysmonitor/inode**. + +```text +DISK="/" +DISK="/var/log" +``` + +| Item| Description | Mandatory| Default Value| +| ------ | ------------------------- | -------- | ------ | +| DISK | Mount directory | Yes | None | +| ALARM | Integer indicating the drive inode alarm threshold| No | 90 | +| RESUME | Integer indicating the drive inode alarm recovery threshold| No | 80 | + +- After modifying the configuration file for drive inode monitoring, run `systemctl reload sysmonitor`. The new configuration takes effect after a monitoring period. +- If a mount directory is configured repeatedly, the last configuration item takes effect. +- The value of **ALARM** must be greater than that of **RESUME**. +- Only the mount point or the drive partition of the mount point can be monitored. +- When the CPU usage and I/O usage are high, the `df` command execution may time out. As a result, the drive inode usage cannot be obtained. +- If a drive partition is mounted to multiple mount points, an alarm is reported for each mount point. + +### Exception Logs + +If a drive inode alarm is detected, the following information is displayed in the **/var/log/sysmonitor.log** file: + +```text +info|sysmonitor[4570]:report disk inode alarm, /var/log used:90% alarm:90% +info|sysmonitor[4570]:report disk inode recovered, /var/log used:79% alarm:80% +``` + +## Local Drive I/O Latency Monitoring + +### Introduction + +Reads the local drive I/O latency data every 5 seconds and collects statistics on 60 groups of data every 5 minutes. If more than 30 groups of data are greater than the configured maximum I/O latency, the system records a log indicating excessive drive I/O latency. + +### Configuration File Description + +The configuration file is **/etc/sysmonitor/iodelay**. + +```text +DELAY_VALUE="500" +``` + +| Item | Description | Mandatory| Default Value| +| ----------- | -------------------- | -------- | ------ | +| DELAY_VALUE | Maximum drive I/O latency| Yes | 500 | + +### Exception Logs + +If a drive I/O latency alarm is detected, the following information is displayed in the **/var/log/sysmonitor.log** file: + +```text +info|sysmonitor[127]:local disk sda IO delay is too large, I/O delay threshold is 70. +info|sysmonitor[127]:disk is sda, io delay data: 71 72 75 87 99 29 78 ...... +``` + +If a drive I/O latency recovery alarm is detected, the following information is displayed in the **/var/log/sysmonitor.log** file: + +```text +info|sysmonitor[127]:local disk sda IO delay is normal, I/O delay threshold is 70. +info|sysmonitor[127]:disk is sda, io delay data: 11 22 35 8 9 29 38 ...... +``` + +## Zombie Process Monitoring + +### Introduction + +Monitors the number of zombie processes in the system. If the number is greater than the alarm threshold, an alarm log is recorded. When the number drops lower than the recovery threshold, a recovery alarm is reported. + +### Configuration File Description + +The configuration file is **/etc/sysmonitor/zombie**. + +```text +# Ceiling zombie process counts of alarm +ALARM="500" + +# Floor zombie process counts of resume +RESUME="400" + +# Periodic (second) +PERIOD="600" +``` + +| Item| Description | Mandatory| Default Value| +| ------ | ------------------------------- | -------- | ------ | +| ALARM | Number greater than 0, indicating the zombie process count alarm threshold | No | 500 | +| RESUME | Number greater than or equal to 0, indicating the zombie process count recovery threshold| No | 400 | +| PERIOD | Monitoring period, in seconds. The value is greater than 0. | No | 60 | + +### Exception Logs + +If a zombie process count alarm is detected, the following information is displayed in the **/var/log/sysmonitor.log** file: + +```text +info|sysmonitor[127]: zombie process count alarm: 600 +info|sysmonitor[127]: zombie process count resume: 100 +``` + +## Custom Monitoring + +### Introduction + +You can customize monitoring items. The monitoring framework reads the content of the configuration file, parses the monitoring attributes, and calls the monitoring actions to be performed. The monitoring module provides only the monitoring framework. It is not aware of what users are monitoring or how to monitor, and does not report alarms. + +### Configuration File Description + +The configuration files are stored in **/etc/sysmonitor.d/**. Each process or module corresponds to a configuration file. + +```text +MONITOR_SWITCH="on" +TYPE="periodic" +EXECSTART="/usr/sbin/iomonitor_daemon" +PERIOD="1800" +``` + +| Item | Description | Mandatory | Default Value| +| -------------- | ------------------------------------------------------------ | --------------------- | ------ | +| MONITOR_SWITCH | Monitoring switch | No | off | +| TYPE | Custom monitoring item type
**daemon**: background execution
**periodic**: periodic execution| Yes | None | +| EXECSTART | Monitoring command | Yes | None | +| ENVIROMENTFILE | Environment variable file | No | None | +| PERIOD | If the type is **periodic**, this parameter is mandatory and sets the monitoring period. The value is an integer greater than 0.| Yes when the type is **periodic**| None | + +- The absolute path of the configuration file or environment variable file cannot contain more than 127 characters. The environment variable file path cannot be a soft link path. +- The length of the **EXECSTART** command cannot exceed 159 characters. No space is allowed in the key field. +- The execution of the periodic monitoring command cannot time out. Otherwise, the custom monitoring framework will be affected. +- Currently, a maximum of 256 environment variables can be configured. +- The custom monitoring of the daemon type checks whether the `reload` command is delivered or whether the daemon process exits abnormally every 10 seconds. If the `reload` command is delivered, the new configuration is loaded 10 seconds later. If a daemon process exits abnormally, the daemon process is restarted 10 seconds later. +- If the content of the **ENVIROMENTFILE** file changes, for example, an environment variable is added or the environment variable value changes, you need to restart the sysmonitor service for the new environment variable to take effect. +- You are advised to set the permission on the configuration files in the **/etc/sysmonitor.d/** directory to 600. If **EXECSTART** is only an executable file, you are advised to set the permission on the executable file to 550. +- After a daemon process exits abnormally, sysmonitor reloads the configuration file of the daemon process. + +### Exception Logs + +If a monitoring item of the daemon type exits abnormally, the **/var/log/sysmonitor.log** file records the following information: + +```text +info|sysmonitor[127]: custom daemon monitor: child process[11609] name unetwork_alarm exit code[127],[1] times. +``` diff --git a/docs/en/server/maintenance/troubleshooting/_toc.yaml b/docs/en/server/maintenance/troubleshooting/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..0a0a7d6bf6b363ab328df09840081350793c3a0e --- /dev/null +++ b/docs/en/server/maintenance/troubleshooting/_toc.yaml @@ -0,0 +1,4 @@ +label: Troubleshooting +isManual: true +href: ./troubleshooting.md +description: Common troubleshooting methods diff --git a/docs/en/server/maintenance/troubleshooting/troubleshooting.md b/docs/en/server/maintenance/troubleshooting/troubleshooting.md new file mode 100644 index 0000000000000000000000000000000000000000..37e477d1cd18c0979609cd4879bea3cc70eb91ca --- /dev/null +++ b/docs/en/server/maintenance/troubleshooting/troubleshooting.md @@ -0,0 +1,101 @@ +# Troubleshooting + +- [Troubleshooting](#troubleshooting) + - [Triggering kdump Restart](#triggering-kdump-restart) + - [Performing Forcible Restart](#performing-forcible-restart) + - [Restarting the Network](#restarting-the-network) + - [Repairing the File System](#repairing-the-file-system) + - [Manually Dropping Cache](#manually-dropping-cache) + - [Rescue Mode and Single-User Mode](#rescue-mode-and-single-user-mode) + +## Triggering kdump Restart + +```shell +# Write 1 to the sysrq file to enable the SysRq function. After this function is enabled, the kernel will respond to any operation. +echo 1 > /proc/sys/kernel/sysrq + +# Make the system crash. +echo c > /proc/sysrq-trigger +``` + +## Performing Forcible Restart + +You can use either of the following methods to forcibly restart the OS: + +- Manually restart the OS. + +```shell +reboot -f +``` + +- Forcibly power on and off the OS through iBMC. + +## Restarting the Network + +openEuler uses NetworkManager to manage the network. Run the following command to restart the network: + +```shell +systemctl restart NetworkManager +``` + +## Repairing the File System + +After the OS is forcibly powered off and then powered on, the file system may be damaged. When the OS is started, it automatically checks and repairs the file system. If the file system fails to be repaired, you need to run the `fsck` command to scan for and repair the file system. + +```shell +# In this case, the system enters the rescue mode. Check which file system is damaged in the log. +journalctl -xb +# Check whether the partition has been mounted before the repair. +cat /proc/mounts +# Uninstall the directory. +umount xxx +# If the directory cannot be uninstalled, kill the process that occupies the directory. +lsof | grep xxx +kill xxx +# Run the fsck command to rectify the fault. Enter yes or no when prompted. +fsck -y /dev/xxx +``` + +## Manually Dropping Cache + +```shell +#Different values of N can achieve different clearance purposes. According to the Linux kernel document, run the sync command before clearing data. (The drop operation does not release any dirty objects. The sync command writes all unwritten system buffers to drives, including modified inodes, delayed block I/Os, and read/write mapping files. In this way, dirty objects can be reduced so that more objects can be released.) +echo N > /proc/sys/vm/drop_caches + +#Release the page caches. +echo 1 > /proc/sys/vm/drop_caches + +#Release dentries and inodes. +echo 2 > /proc/sys/vm/drop_caches + +#Release the page caches, dentries, and inodes. +echo 3 > /proc/sys/vm/drop_caches +``` + +## Rescue Mode and Single-User Mode + +- Rescue mode + + Mount the openEuler 22.03 LTS SP2 ISO image and enter the rescue mode. + + 1. Select **Troubleshooting**. + 2. Select **Rescue a openEuler system**. + 3. Proceed as prompted. + + ```text + 1)Continue + + 2)Read-only mount + + 3)Skip to shell + + 4)Quit(Reboot) + ``` + +- Single-user mode + + On the login page, enter **e** to go to the grub page, add **init=/bin/sh** to the **linux** line, and press **Ctrl**+**X**. + + 1. Run the `mount -o remount,rw /` command. + 2. Perform operations such as changing the password. + 3. Enter **exit** to exit. diff --git a/docs/en/server/memory_storage/etmem/_toc.yaml b/docs/en/server/memory_storage/etmem/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..981d8c222960e780fb8ea50900984ef75cb2493d --- /dev/null +++ b/docs/en/server/memory_storage/etmem/_toc.yaml @@ -0,0 +1,6 @@ +label: etmem User Guide +isManual: true +description: Expand memory capacity with etmem. +sections: + - label: Using etmem + href: ./etmem-user-guide.md diff --git a/docs/en/server/memory_storage/etmem/etmem-user-guide.md b/docs/en/server/memory_storage/etmem/etmem-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..5490e7171f1b0620127b3d97e6c639dbce8fa0d5 --- /dev/null +++ b/docs/en/server/memory_storage/etmem/etmem-user-guide.md @@ -0,0 +1,773 @@ +# etmem User Guide + +## Introduction + +The development of CPU computing power, particularly lower costs of ARM cores, makes memory cost and capacity become the core frustration that restricts business costs and performance. Therefore, the most pressing issue is how to save memory cost and how to expand memory capacity. + +etmem is a tiered memory expansion technology that uses DRAM+memory compression/high-performance storage media to form tiered memory storage. Memory data is tiered, and cold data is migrated from memory media to high-performance storage media to release memory space and reduce memory costs. + +The tools provided by the etmem software package include the etmem client and the etmemd server. etmemd runs continuously after being launched and implements functions such as recognition and elimination of cold and hot memory of target processes. etmem runs once when called and controls etmemd to respond with different operations based on different command parameters. + +## Compilation + +1. Download the etmem source code. + + ```shell + git clone https://gitee.com/openeuler/etmem.git + ``` + +2. Install the compilation and running dependency. The compilation and running of etmem depend on the libboundscheck component. + + Install the dependency: + + ```bash + yum install libboundscheck + ``` + + Use the `rpm` command to check if the package is installed: + + ```bash + rpm -qi libboundscheck + ``` + +3. Build source code. + + ```bash + cd etmem + mkdir build + cd build + cmake .. + make + ``` + +## Precautions + +### Dependencies + +As a memory expansion tool, etmem needs to rely on kernel features. To identify memory access conditions and support the active writing of memory into the swap partition to achieve the requirement of vertical memory expansion, etmem needs to insert the **etmem_scan** and **etmem_swap** modules at runtime: + +```bash +modprobe etmem_scan +modprobe etmem_swap +``` + +### Restrictions + +The etmem process requires root privileges. The root user has the highest system privileges. When using the root user to perform operations, strictly follow the operation instructions to avoid system management and security risks. + +### Constraints + +- The client and server of etmem must be deployed on the same server. Cross-server communication is not supported. +- etmem can scan target processes whose process name is less than or equal to 15 characters. Supported characters in process names are letters, numbers, periods (.), slashes (/), hyphens (-), and underscores (\_). +- When AEP media is used for memory expansion, it relies on the system being able to correctly recognize the AEP device and initialize the device as a NUMA node. Additionally, the **vm_flags** field in the configuration file can only be configured as **ht**. +- The private commands of the engine are only valid for the corresponding engine and tasks under the engine, such as `showhostpages` and `showtaskpages` supported by cslide. +- In a third-party policy implementations, **fd** in the `eng_mgt_func` interface cannot be written with the **0xff** and **0xfe** characters. +- Multiple different third-party policy dynamic libraries, distinguished by **eng_name** in the configuration file, can be added within a project. +- Concurrent scanning of the same process is prohibited. +- Using the **/proc/xxx/idle_pages** and **/proc/xxx/swap_pages** files is prohibited when **etmem_scan** and **etmem_swap** modules are not loaded. +- The etmem configuration file requires the owner to be the root user, with permissions of 600 or 400. The size of the configuration file cannot exceed 10 MB. +- When etmem injects a third-party policy, the **so** of the third-party policy requires the owner to be the root user, with permissions of 500 or 700. + +## Instructions + +### etmem Configuration Files + +Before running the etmem process, the administrator needs to decide the memory of which processes needs to be expanded, configure the process information in the etmem configuration files, and configure information such as the memory scanning cycle, scanning times, and memory hot and cold thresholds. + +The configuration file examples are included in the source package and stored in the **/etc/etmem** directory. There are three example files: + +```text +/etc/etmem/cslide_conf.yaml +/etc/etmem/slide_conf.yaml +/etc/etmem/thirdparty_conf.yaml +``` + +Contents of the files are as follows: + +```sh +#slide engine example +#slide_conf.yaml +[project] +name=test +loop=1 +interval=1 +sleep=1 +sysmem_threshold=50 +swapcache_high_vmark=10 +swapcache_low_vmark=6 + +[engine] +name=slide +project=test + +[task] +project=test +engine=slide +name=background_slide +type=name +value=mysql +T=1 +max_threads=1 +swap_threshold=10g +swap_flag=yes + +#cslide engine example +#cslide_conf.yaml +[engine] +name=cslide +project=test +node_pair=2,0;3,1 +hot_threshold=1 +node_mig_quota=1024 +node_hot_reserve=1024 + +[task] +project=test +engine=cslide +name=background_cslide +type=pid +name=23456 +vm_flags=ht +anon_only=no +ign_host=no + +#Third-party engine example +#thirdparty_conf.yaml +[engine] +name=thirdparty +project=test +eng_name=my_engine +libname=/usr/lib/etmem_fetch/my_engine.so +ops_name=my_engine_ops +engine_private_key=engine_private_value + +[task] +project=test +engine=my_engine +name=background_third +type=pid +value=12345 +task_private_key=task_private_value +``` + +Fields in the configuration files are described as follows: + +| Item | Description | Mandatory | Contains Parameters | Parameter Range | Example | +| -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------- | ------------------- | -------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| \[project\] | Beginning identifier of the project public configuration section | No | No | N/A | Beginning identifier of the project parameters, indicating that the parameters below are within the range of the project section until another \[xxx\] or the end of the file | +| name | Name of the project | Yes | Yes | String of up to 64 characters | Specifies that the project, engine and task need to be mounted to the specified project during configuration. | +| loop | Number of loops for memory scan | Yes | Yes | 1~120 | loop=3 // Memory is scanned 3 times. | +| interval | Time interval for each memory scan | Yes | Yes | 1~1200 | interval=5 // The interval is 5s. | +| sleep | Time interval for each memory scan+operation | Yes | Yes | 1~1200 | sleep=10 //The interval is 10s | +| sysmem_threshold | Memory swapping threshold. This is a slide engine configuration item. | No | Yes | 0~100 | sysmem_threshold=50 // When available memory is less than 50%, etmem swaps out memory. | +| swapcache_high_wmark | High watermark of swapcache. This is a slide engine configuration item. | No | Yes | 1~100 | swapcache_high_wmark=5 // swapcache can be up to 5% of the system memory. If this ratio is reached, etmem triggers swapcache recycling.
Note: swapcache_high_wmark must be greater than swapcache_low_wmark. | +| swapcache_low_wmark | Low watermark of swapcache. This is a slide engine configuration item. | No | Yes | \[1~swapcache_high_wmark\) | swapcache_low_wmark=3 //When swapcache recycling is triggered, the system recycles the swapcache memory occupancy to less than 3%. | +| \[engine\] | Beginning identifier of the engine public configuration section | No | No | N/A | Beginning identifier of the engine parameters, indicating that the parameters below are within the range of the engine section until another \[xxx\] or the end of the file | +| project | project to which the engine belongs | Yes | Yes | String of up to 64 characters | If a project named test exists, the item can be **project=test**. | +| engine | engine to which the engine belongs | Yes | Yes | slide/cslide/thirdparty | Specifies the policy to use (**slide**, **cslide**, or **thirdparty**) | +| node_pair | Node pair of AEP and DRAM. This is a cslide engine configuration item. | Yes when **engine** is **cslide** | Yes | Pair the node numbers of AEP and DRAM. Separate AEP and DRAM using a comma, and separate each pair using semicolons. | node_pair=2,0;3,1 | +| hot_threshold | Threshold of hot memory watermark. This is a cslide engine configuration item. | Yes when **engine** is **cslide** | Yes | An integer greater than or equal to 0 and less than or equal to INT_MAX | hot_threshold=3 // Memory with less than 3 accesses will be recognized as cold memory. | +| node_mig_quota | Maximum one-way flow when DRAM and AEP migrate mutually. This is a cslide engine configuration item. | Yes when **engine** is **cslide** | Yes | An integer greater than or equal to 0 and less than or equal to INT_MAX | node_mig_quota=1024 // The unit is MB. A maximum of 1024 MB can be migrated from AEP to DRAM or from DRAM to AEP each time. | +| node_hot_reserve | Size of the reserved space for hot memory in DRAM. This is a cslide engine configuration item. | Yes when **engine** is **cslide** | Yes | An integer greater than or equal to 0 and less than or equal to INT_MAX | node_hot_reserve=1024 //The unit is MB. When the hot memory of all VMs is greater than this configuration value, the hot memory will also be migrated to AEP. | +| eng_name | Name of the engine for mounting by task. This is a third-party engine configuration item. | Yes when **engine** is **thirdparty** | Yes | String of up to 64 characters | eng_name=my_engine // When mounting a task to the third-party policy engine, specify **engine=my_engine** in the task. | +| libname | Absolute path to the dynamic library of the third-party policy. This is a third-party engine configuration item. | Yes when **engine** is **thirdparty** | Yes | String of up to 256 characters | libname=/user/lib/etmem_fetch/code_test/my_engine.so | +| ops_name | Name of the operator in the dynamic library of the third-party policy. This is a third-party engine configuration item. | Yes when **engine** is **thirdparty** | Yes | String of up to 256 characters | ops_name=my_engine_ops // Name of the struct for the third-party policy implementation interface | +| engine_private_key | Reserved item for third-party policies to parse private parameters by themselves. This is a third-party engine configuration item. | No | No | Restrict according to the third-party policy's private parameters. | Configure the private engine parameters according to the third-party policy. | +| \[task\] | Beginning identifier of the task public configuration section | No | No | N/A | Beginning identifier of the task parameters, indicating that the parameters below are within the range of the project section until another \[xxx\] or the end of the file | +| project | project to which the task belongs | Yes | Yes | String of up to 64 characters | If a project named test exists, the item can be **project=test**. | +| engine | engine to which the task belongs | Yes | Yes | String of up to 64 characters | Name of the engine to which the task belongs | +| name | Name of the task | Yes | Yes | String of up to 64 characters | name=background1 // The name of the task is background1. | +| type | How the target process is identified | Yes | Yes | pid/name | **pid** specifies to identify by PID. **name** specifies to identify by name. | +| value | Value to be identified for the target process | Yes | Yes | Actual PID/name | Used with **type** to specify the PID or name of the target process. Ensure the configuration is correct and unique. | +| T | Threshold of hot memory watermark. This is a slide engine configuration item. | Yes when **engine** is **slide** | Yes | 0~loop * 3 | T=3 // Memory with less than 3 accesses will be recognized as cold memory. | +| max_threads | Maximum number of threads in the etmem internal thread pool, with each thread handling a process/subprocess memory scan+operation task. This is a slide engine configuration item. | No | Yes | 1~2 * number of cores + 1, the default value is 1. | Controls the number of internal processing threads for the etmemd server without external representation. When the target process has multiple child processes, the larger the item value, the more concurrent executions, but the more resources consumed. | +| vm_flags | Flag of the VMA to be scanned. This is a cslide engine configuration item. | No | Yes | String of up to 256 characters, with different flags separated by spaces. | vm_flags=ht // Scans memory of the VMA whose flag is ht. | +| anon_only | Scans anonymous pages only. This is a cslide engine configuration item. | No | Yes | yes/no | anon_only=no | +| ign_host | Ignores page table scan information on the host. This is a cslide engine configuration item. | No | Yes | yes/no | ign_host=no | +| task_private_key | Reserved for a task of a third-party policy to parse private parameters. This is a third-party engine configuration item. | No | No | Restrict according to the third-party policy's private parameters. | Configure the private task parameters according to the third-party policy. | +| swap_threshold | Process memory swapping threshold. This is a slide engine configuration item. | No | Yes | Absolute value of memory available to the process | swap_threshold=10g // Memory swapping will not be triggered when the process memory is less than 10 GB.
Currently, the unit can only be **g** or **G**. This item is used with **sysmem_threshold**. When system memory is lower than **sysmem_threshold**, memory of processes in the allowlist is checked. | +| swap_flag | Enables process memory swapping. This is a slide engine configuration item. | No | Yes | yes/no | swap_flag=yes | + +### Starting etmemd + +Modify related configuration files before using etmem services. After being started, etmemd stays in the system to operate the memory of the target processes.To start etmemd, you can either run the `etmemd` command or configure a service file for `systemctl` to start etmemd. The latter requires the `mode-systemctl` option. + +#### How to Use + +Run the following command to start etmemd: + +```bash +etmemd -l 0 -s etmemd_socket +``` + +or + +```bash +etmemd --log-level 0 --socket etmemd_socket +``` + +The `0` parameter of option `-l` and the `etmemd_socket` parameter of option `-s` are user-defined parameters and are described as follows. + +#### Command Parameters + +| Option | Description | Mandatory | Contains Parameters | Parameter Range | Example | +| --------------- | ---------------------------------- | -------- | ---------- | --------------------- | ------------------------------------------------------------ | +| -l or \-\-log-level | etmemd log level | No | Yes | 0~3 | 0: debug level
1: info level
2: warning level
3: error level
Logs whose levels are higher than the specified value are printed to **/var/log/message**. | +| -s or \-\-socket | Socket listened by etmemd to interact with the client | Yes | Yes | String of up to 107 characters | Socket listened by etmemd | +| -m or \-\-mode-systemctl| Starts the etmemd service through systemctl | No| No| N/A| The `-m` option needs to be specified in the service file.| +| -h or \-\-help | Prints help information | No | No | N/A | This option prints help information and exit. | + +### Adding and Deleting Projects, Engines, and Tasks Using the etmem Client + +#### Scenario + +1. The administrator adds a project, engine, or task to etmem (a project can contain multiple etmem engines, an engine can contain multiple tasks). + +2. The administrator deletes an existing etmem project, engine, or task (all tasks in a project is stopped before the project is deleted). + +#### Usage + +When etmemd is running normally, run `etmem` with the `obj` option to perform addition and deletion. etmem automatically identifies projects, engines, or tasks according to the content of the configuration file. + +- Add an object. + + ```bash + etmem obj add -f /etc/etmem/slide_conf.yaml -s etmemd_socket + ``` + + or + + ```bash + etmem obj add --file /etc/etmem/slide_conf.yaml --socket etmemd_socket + ``` + +- Delete an object. + + ```bash + etmem obj del -f /etc/etmem/slide_conf.yaml -s etmemd_socket + ``` + + or + + ```bash + etmem obj del --file /etc/etmem/slide_conf.yaml --socket etmemd_socket + ``` + +#### Command Parameters + +| Option | Description | Mandatory | Contains Parameters | Parameter Range | Example | +| ---------------- | -------------------------------------------------------------------------------------------------------------- | --------- | ------------------- | ----------------------------------------------------------------------------------------------------- | ------- | +| -f or \-\-file | Specifies the configuration file of the object. | Yes | Yes | Specify the path. | | +| -s or \-\-socket | Socket used for communication with etmemd, which must be the same as the one specified when etmemd is started. | Yes | Yes | The administrator can use this option to specify an etmemd server when multiple etmemd servers exist. | | + +### Querying, Starting, and Stopping Projects Using the etmem Client + +#### Scenario + +A project is added by using `etmem obj add` and is not deleted by using `etmem obj del`. In this case, the project can be started and stopped. + +1. The administrator starts an added project. + +2. The administrator stops a started project. + +A started project will be stopped if the administrator run `obj del` to delete the project. + +#### Usage + +Added projects can be started and stopped by using `etmem project` commands. + +- Query a project. + + ```bash + etmem project show -n test -s etmemd_socket + ``` + + or + + ```bash + etmem project show --name test --socket etmemd_socket + ``` + +- Start a project. + + ```bash + etmem project start -n test -s etmemd_socket + ``` + + or + + ```bash + etmem project start --name test --socket etmemd_socket + ``` + +- Stop a project. + + ```bash + etmem project stop -n test -s etmemd_socket + ``` + + or + + ```bash + etmem project stop --name test --socket etmemd_socket + ``` + +- Print help information. + + ```bash + etmem project help + ``` + +#### Command Parameters + +| Option | Description | Mandatory | Contains Parameters | Parameter Range | Example | +| ---------------- | -------------------------------------------------------------------------------------------------------------- | --------- | ------------------- | ----------------------------------------------------------------------------------------------------- | ------- | +| -n or \-\-name | Name of the project | Yes | Yes | The project name corresponds to the configuration file. | | +| -s or \-\-socket | Socket used for communication with etmemd, which must be the same as the one specified when etmemd is started. | Yes | Yes | The administrator can use this option to specify an etmemd server when multiple etmemd servers exist. | | + +### Specifying System Memory Swapping Threshold and Process Memory Swapping Using the etmem Client + +Only slide policies support private features. + +- Process or system memory swapping threshold + +It is necessary to consider the timing of etmem memory swapping for optimal performance. Memory swapping is not performed when the system has enough available memory or a process occupies a low amount of memory. Memory swapping threshold can be specified for the system and processes. + +- Process memory swapping + +The memory of I/O latency-sensitive service processes should not be swapped in the storage scenario. In this case, you can disable memory swapping for certain services. + +Process and system memory swapping thresholds and process memory swapping are controlled by the **sysmem_threshold**, **swap_threshold**, and **swap_flag** parameters in the configuration file. For details, see [etmem Configuration Files](#etmem-configuration-files). + +```sh +#slide_conf.yaml +[project] +name=test +loop=1 +interval=1 +sleep=1 +sysmem_threshold=50 + +[engine] +name=slide +project=test + +[task] +project=test +engine=slide +name=background_slide +type=name +value=mysql +T=1 +max_threads=1 +swap_threshold=10g +swap_flag=yes +``` + +#### System Memory Swapping Threshold + +The **sysmem_threshold** parameter is used to set system memory swapping threshold. The value range for **sysmem_threshold** is 0 to 100. If **sysmem_threshold** is set in the configuration file, etmem will swap memory when system memory is lower than **sysmem_threshold**. + +For example: + +1. Compose the configuration according to the example. Set **sysmem_threshold** to **20**. +2. Start the server, add a project to the server, and start the project. + + ```bash + etmemd -l 0 -s monitor_app & + etmem obj add -f etmem_config -s monitor_app + etmem project start -n test -s monitor_app + etmem project show -s monitor_app + ``` + +3. Observe the memory swapping results. etmem swaps memory only when the system available memory is less than 20%. + +#### Process Memory Swapping Threshold + +The **swap_threshold** parameter is used to set process memory swapping threshold. **swap_threshold** is the absolute memory usage of a process in the format of \**g/G**. If **swap_threshold** is set in the configuration file, etmem will not swap memory of the process when the process memory usage is lower then **swap_threshold**. + +For example: + +1. Compose the configuration according to the example. Set **swap_threshold** to **5g**. +2. Start the server, add a project to the server, and start the project. + + ```bash + etmemd -l 0 -s monitor_app & + etmem obj add -f etmem_config -s monitor_app + etmem project start -n test -s monitor_app + etmem project show -s monitor_app + ``` + +3. Observe the memory swapping results. etmem swaps memory only when the process memory usage reaches 5 GB. + +#### Process Memory Swapping + +The **swap_flag** parameter is used to enable the process memory swapping feature. The value of **swap_flag** can be **yes** or **no**. If **swap_flag** is **no** or not configured, etmem swaps memory normally. If **swap_flag** is **yes**, etmem swaps memory of the specified processes only. + +For example: + +1. Compose the configuration according to the example. Set **swap_flag** to **yes**. +2. Flag the memory to be swapped for the service process. + + ```bash + madvise(addr_start, addr_len, MADV_SWAPFLAG) + ``` + +3. Start the server, add a project to the server, and start the project. + + ```bash + etmemd -l 0 -s monitor_app & + etmem obj add -f etmem_config -s monitor_app + etmem project start -n test -s monitor_app + etmem project show -s monitor_app + ``` + +4. Observe the memory swapping results. Only the flagged memory is swapped. Other memory is retained in the DRAM. + +In the process memory page swapping scenario, `ioctl` is added to the original scan interface file **idle_pages** to ensure that VMAs that are not flagged do not participate in memory scanning and swapping. + +Scan management interface: + +- Function prototype + + ```c + ioctl(fd, cmd, void *arg); + ``` + +- Input parameters + 1. fd: file descriptor, which is obtained by opening a file under /proc/pid/idle_pages using the open system call + 2. cmd: controls the scan actions. The following values are supported: + VMA_SCAN_ADD_FLAGS: adds VMA memory swapping flags to scan only flagged VMAs + VMA_SCAN_REMOVE_FLAGS: removes added VMA memory swapping flags + 3. args: integer pointer parameter used to pass a specific mask. The following value is supported: + VMA_SCAN_FLAG: Before the etmem_scan.ko module starts scanning, the walk_page_test interface is called to determine whether the VMA address meets the scanning requirements. If this flag is set, only the VMA addresses that contain specific swap flags are scanned. + +- Return values + 1. 0 if the command succeeds + 2. Other values if the command fails + +- Precautions + Unsupported flags are ignored and do not return errors. + +### Specifying swapcache Memory Recycling Instructions Using the etmem Client + +The user-mode etmem initiates a memory elimination and recycling operation and interacts with the kernel-mode memory recycling module through the **write procfs** interface. The memory recycling module parses the virtual address sent from the user space, obtains the page corresponding to the address, and calls the native kernel interface to swap and recycle the memory corresponding to the page. During memory swapping, swapcache will use some system memory. To further save memory, the swapcache memory recycling feature is added. + +Add **swapcache_high_wmark** and **swapcache_low_wmark** parameters to use the swapcache memory recycling feature. + +- **swapcache_high_wmark**: High system memory water of swapcache +- **swapcache_low_wmark**: Low system memory water of swapcache + +After etmem swaps memory, it checks the swapcache memory occupancy. When the occupancy exceeds the high watermark, an `ioctl` instruction will be issued through **swap_pages** to trigger the swapcache memory recycling and stop when swapcache memory occupancy reaches the low watermark. + +An example configuration file is as follows. For details, see [etmem Configuration Files](#etmem-configuration-files). + +```sh +#slide_conf.yaml +[project] +name=test +loop=1 +interval=1 +sleep=1 +swapcache_high_vmark=5 +swapcache_low_vmark=3 + +[engine] +name=slide +project=test + +[task] +project=test +engine=slide +name=background_slide +type=name +value=mysql +T=1 +max_threads=1 +``` + +During memory swapping, swapcache memory needs to be recycled to further save memory. An `ioctl` interface is added to the original memory swap interface to configure swapcache watermarks and swapcache memory recycling. + +- Function prototype + + ```c + ioctl(fd, cmd, void *arg); + ``` + +- Input parameters + 1. fd: file descriptor, which is obtained by opening a file under /proc/pid/idle_pages using the open system call + 2. cmd: controls the scan actions. The following values are supported: + RECLAIM_SWAPCACHE_ON: enables swapcache memory swapping + RECLAIM_SWAPCACHE_OFF: disables swapcache memory swapping + SET_SWAPCACHE_WMARK: configures swapcache memory watermarks + 3. args: integer pointer parameter used to pass a specific mask. The following value is supported: + Parameters that pass the values of swapcache watermarks + +- Return values + 1. 0 if the command succeeds + 2. Other values if the command fails + +- Precautions + Unsupported flags are ignored and do not return errors. + +### Executing Private Commands and Functions Using the etmem Client + +Only the cslide policy support private commands. + +- `showtaskpages` +- `showhostpages` + +For engines and tasks of engines that use the cslide policy, you can run the commands above to query the page access of tasks and the usage of system huge pages on the host of VMs. + +For example: + +```bash +etmem engine showtaskpages <-t task_name> -n proj_name -e cslide -s etmemd_socket + +etmem engine showhostpages -n proj_name -e cslide -s etmemd_socket +``` + +**Note**: `showtaskpages` and `showhostpages` are supported by the cslide policy only. + +#### Command Parameters + +| Option | Description | Mandatory | Contains Parameters | Parameter Range | Example | +| ------------------- | -------------------------------------------------------------------------------------------------------------- | --------- | ------------------- | ----------------------------------------------------------------------------------------------------- | ------- | +| -n or \-\-proj_name | Name of the project | Yes | Yes | Name of an existing project to run | | +| -s or \-\-socket | Socket used for communication with etmemd, which must be the same as the one specified when etmemd is started. | Yes | Yes | The administrator can use this option to specify an etmemd server when multiple etmemd servers exist. | | +| -e or \-\-engine | Name of the engine to run | Yes | Yes | Name of an existing engine to run | | +| -t or \-\-task_name | Name of the task to run | No | Yes | Name of an existing task to run | | + +### Enabling and Disabling Kernel Swap + +When etmem swaps memory to the drive to expand memory, you can choose to enable the kernel swap feature. You can disable the native kernel swap mechanism to void swapping memory undesirably, resulting in problems with user-mode processes. + +A sys interface is provided to implement such control. A **kobj** named **kernel_swap_enable** is created in **/sys/kernel/mm/swap** to enable and disable kerne swap. The default value of **kernel_swap_enable** is **true**. + +For example: + +```sh +# Enable kernel swap +echo true > /sys/kernel/mm/swap/kernel_swap_enable +or +echo 1 > /sys/kernel/mm/swap/kernel_swap_enable + +# Disable kernel swap +echo false > /sys/kernel/mm/swap/kernel_swap_enable +or +echo 0 > /sys/kernel/mm/swap/kernel_swap_enable + +``` + +### Starting etmem Upon System Startup + +#### Scenario + +You can configure the systemd configuration file to run etmemd as a forking service of systemd. + +#### Usage + +Compose a service configuration file to start etmemd with the `-m` option. For example: + +```bash +etmemd -l 0 -s etmemd_socket -m +``` + +#### Command Parameters + +| Option | Description | Mandatory | Contains Parameters | Parameter Range | Example | +| --------------- | ---------------------------------- | -------- | ---------- | --------------------- | ------------------------------------------------------------ | +| -l or \-\-log-level | etmemd log level | No | Yes | 0~3 | 0: debug level
1: info level
2: warning level
3: error level
Logs whose levels are higher than the specified value are printed to **/var/log/message**. | +| -s or \-\-socket | Socket listened by etmemd to interact with the client | Yes | Yes | String of up to 107 characters | Socket listened by etmemd | +| -m or \-\-mode-systemctl| Starts the etmemd service through systemctl | No| No| N/A| The `-m` option needs to be specified in the service file.| +| -h or \-\-help | Prints help information | No | No | N/A | This option prints help information and exit. | + +### Supporting Third-party Memory Expansion Policies With etmem + +#### Scenario + +etmem provides third-party memory expansion policy registration and module scanning dynamic library and can eliminate memory according to third-party policies. + +You can use the module scanning dynamic library to implement the interface of the struct required for connecting to etmem. + +#### Usage + +To use a third-party memory expansion elimination policy, perform the following steps: + +1. Invoke the scanning interface of the module as required. + +2. Implement the interfaces using the function template provided by the etmem header file and encapsulate them into a struct. + +3. Build a dynamic library of the third-party memory expansion elimination policy. + +4. Specify the **thirdparty** engine in the configuration file. + +5. Enter the names of the library and the interface struct to the corresponding **task** fields in the configuration file. + +Other steps are similar to those of using other engines. + +Interface struct template: + +```c +struct engine_ops { + +/* Parsing private parameters of the engine. Implement the interface if required, otherwise, set it to NULL. */ + +int (*fill_eng_params)(GKeyFile *config, struct engine *eng); + +/* Clearing private parameters of the engine. Implement the interface if required, otherwise, set it to NULL. */ + +void (*clear_eng_params)(struct engine *eng); + +/* Parsing private parameters of the task. Implement the interface if required, otherwise, set it to NULL. */ + +int (*fill_task_params)(GKeyFile *config, struct task *task); + +/* Clearing private parameters of the task. Implement the interface if required, otherwise, set it to NULL. */ + +void (*clear_task_params)(struct task *tk); + +/* Task starting interface */ + +int (*start_task)(struct engine *eng, struct task *tk); + +/* Task stopping interface */ + +void (*stop_task)(struct engine *eng, struct task *tk); + +/* Allocate PID-related private parameters */ + +int (*alloc_pid_params)(struct engine *eng, struct task_pid **tk_pid); + +/* Destroy PID-related private parameters */ + +void (*free_pid_params)(struct engine *eng, struct task_pid **tk_pid); + +/* Support for private commands required by the third-party policy. If this interface is not required, set it to NULL */ + +int (*eng_mgt_func)(struct engine *eng, struct task *tk, char *cmd, int fd); + +}; +``` + +External interfaces of the scanning module: + +| Interface |Description| +| ------------ | --------------------- | +| etmemd_scan_init | Initializes the scanning module| +| etmemd_scan_exit | Exits the scanning module| +| etmemd_get_vmas | Gets the VMAs to be scanned| +| etmemd_free_vmas | Releases VMAs scanned by `etmemd_get_vmas`| +| etmemd_get_page_refs | Scans pages in VMAs| +| etmemd_free_page_refs | Release the page access information list obtained by `etmemd_get_page_refs` | + +In the VM scanning scenario, `ioctl` is added to the original scan interface file **idle_pages** to distinguish the EPT scanning granularity and specify whether to ignore page access flags on the hosts. + +In the process memory page swapping scenario, `ioctl` is added to the original scan interface file **idle_pages** to ensure that VMAs that are not flagged do not participate in memory scanning and swapping. + +Scan management interface: + +- Function prototype + + ```c + ioctl(fd, cmd, void *arg); + ``` + +- Input parameters + 1. fd: file descriptor, which is obtained by opening a file under /proc/pid/idle_pages using the open system call + 2. cmd: controls the scan actions. The following values are supported: + IDLE_SCAN_ADD_FLAG: adds a scanning flag + IDLE_SCAM_REMOVE_FLAGS: removes a scanning flag + VMA_SCAN_ADD_FLAGS: adds VMA memory swapping flags to scan only flagged VMAs + VMA_SCAN_REMOVE_FLAGS: removes added VMA memory swapping flags + 3. args: integer pointer parameter used to pass a specific mask. The following value is supported: + SCAN_AS_HUGE: scans the pages according to the 2 MB granularity to see if the pages have been accessed when scanning the EPT page table. If this parameter is not set, the granularity will be the granularity of the EPT page table itself. + SCAN_IGN_HUGE: ignores page access flags on the hosts when scanning VMs. + VMA_SCAN_FLAG: Before the etmem_scan.ko module starts scanning, the walk_page_test interface is called to determine whether the VMA address meets the scanning requirements. If this flag is set, only the VMA addresses that contain specific swap flags are scanned. + +- Return values + 1. 0 if the command succeeds + 2. Other values if the command fails + +- Precautions + Unsupported flags are ignored and do not return errors. + +An example configuration file is as follows. For details, see [etmem Configuration Files](#etmem-configuration-files). + +```text +#thirdparty +[engine] + +name=thirdparty + +project=test + +eng_name=my_engine + +libname=/user/lib/etmem_fetch/code_test/my_engine.so + +ops_name=my_engine_ops + +engine_private_key=engine_private_value + +[task] + +project=test + +engine=my_engine + +name=background1 + +type=pid + +value=1798245 + +task_private_key=task_private_value +``` + + **Note**: + +You need to use the module scanning dynamic library to implement the interface of the struct required for connecting to etmem. + +**fd** in the `eng_mgt_func` interface cannot be written with the **0xff** and **0xfe** characters. + +Multiple different third-party policy dynamic libraries, distinguished by **eng_name** in the configuration file, can be added within a project. + +### Help Information of the etmem Client and Server + +Run the following command to print help information of the etmem server: + +```bash +etmemd -h +``` + +or: + +```bash +etmemd --help +``` + +Run the following command to print help information of the etmem client: + +```bash +etmem help +``` + +Run the following command to print help information of project, engine, and task operations: + +```bash +etmem obj help +``` + +Run the following command to print help information of projects: + +```bash +etmem project help +``` + +## How to Contribute + +1. Fork this repository. +2. Create a branch. +3. Commit your code. +4. Create a pull request (PR). diff --git a/docs/en/server/memory_storage/gmem/_toc.yaml b/docs/en/server/memory_storage/gmem/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..a568c846d446297e71841dc213bdeb5c1df41a9f --- /dev/null +++ b/docs/en/server/memory_storage/gmem/_toc.yaml @@ -0,0 +1,9 @@ +label: GMEM User Guide +isManual: true +href: ./introduction-to-gmem.md +description: Centralized management for heterogeneous memory interconnections +sections: + - label: Installation and Deployment + href: ./installation-and-deployment.md + - label: Usage Instructions + href: ./usage-instructions.md diff --git a/docs/en/server/memory_storage/gmem/images/GMEM_architecture.png b/docs/en/server/memory_storage/gmem/images/GMEM_architecture.png new file mode 100644 index 0000000000000000000000000000000000000000..59b82d647166525b296529d3e3dfede3eca48f4a Binary files /dev/null and b/docs/en/server/memory_storage/gmem/images/GMEM_architecture.png differ diff --git a/docs/en/server/memory_storage/gmem/installation-and-deployment.md b/docs/en/server/memory_storage/gmem/installation-and-deployment.md new file mode 100644 index 0000000000000000000000000000000000000000..8ce7ad0ccedb69cdd2f9514c4786f8b511b0356c --- /dev/null +++ b/docs/en/server/memory_storage/gmem/installation-and-deployment.md @@ -0,0 +1,116 @@ +# Installation and Deployment + +This section describes how to install and deploy GMEM. + +## Software and Hardware Requirements + +* Kunpeng 920 CPU +* Ascend 910 processor +* openEuler 23.09 + +## Environment Requirements + +* The root permission is required for using and configuring GMEM. +* GMEM can be enabled or disabled only at the system level. +* The administrator must ensure that the GMEM configuration is safe and available. + +## Installing GMEM + +* Prepare files. + + [CANN Community Version History - Ascend Community (hiascend.com)](https://www.hiascend.com/en/software/cann/community-history) + + [Firmware and Driver - Ascend Community (hiascend.com)](https://www.hiascend.com/en/hardware/firmware-drivers/community?product=2&model=19&cann=6.0.1.alpha001&driver=1.0.18.alpha) + + | Source | Software Package | + | ------------------------------------------------------------ | ------------------------------------------------------------ | + | openEuler 23.09 | kernel-6.4.0-xxx.aarch64.rpm
kernel-devel-6.4.0-xxx.aarch64.rpm
libgmem-xxx.aarch64.rpm
libgmem-devel-xxx.aarch64.rpm | + | Ascend community | CANN package:
Ascend-cann-toolkit-xxx-linux.aarch64.rpm
NPU firmware and driver:
Ascend-hdk-910-npu-driver-xxx.aarch64.rpm
Ascend-hdk-910-npu-firmware-xxx.noarch.rpm | + | Contact the maintainers of the GMEM community.
[@yang_yanchao](https://gitee.com/yang_yanchao) email:
[@LemmyHuang](https://gitee.com/LemmyHuang) email: | gmem-example-xxx.aarch64.rpm
mindspore-xxx-linux_aarch64.whl | + +* Install the kernel. + + Ensure that GMEM compilation options are enabled (enabled by default) for the openEuler kernel. + + ```sh + [root@localhost ~]# cat /boot/config-`uname -r` | grep CONFIG_GMEM + CONFIG_GMEM=y + CONFIG_GMEM_DEV=m + + [root@localhost ~]# cat /boot/config-`uname -r` | grep CONFIG_REMOTE_PAGER + CONFIG_REMOTE_PAGER=m + CONFIG_REMOTE_PAGER_MASTER=m + ``` + + Add **gmem=on** to the boot options. + + ```sh + [root@localhost gmem]# cat /proc/cmdline + BOOT_IMAGE=/vmlinuz-xxx root=/dev/mapper/openeuler-root ... gmem=on + ``` + + Configure **transparent_hugepage**. + + ```sh + echo always > /sys/kernel/mm/transparent_hugepage/enabled + ``` + +* Install the user-mode dynamic library libgmem. + + ```sh + yum install libgmem libgmem-devel + ``` + +* Install the CANN framework. + + Install the matching CANN, including the toolkit, driver, and firmware. After the installation is complete, restart the system. + + ```sh + rpm -ivh Ascend-cann-toolkit-xxx-linux.aarch64.rpm + # Use the tool provided by libgmem to install the NPU driver. + sh /usr/local/gmem/install_npu_driver.sh Ascend-hdk-910-npu-driver-xxx.aarch64.rpm + rpm -ivh Ascend-hdk-910-npu-firmware-xxx.noarch.rpm + ``` + + Run the environment configuration script in the **Ascend** directory to configure environment variables. + + ```sh + source /usr/local/Ascend/ascend-toolkit/set_env.sh + ``` + + Check whether the NPU is working properly. + + ```sh + [root@localhost ~]# npu-smi info + +-------------------------------------------------------------------------------------------+ + | npu-smi 22.0.4.1 Version: 22.0.4.1 | + +----------------------+---------------+----------------------------------------------------+ + | NPU Name | Health | Power(W) Temp(C) Hugepages-Usage(page)| + | Chip | Bus-Id | AICore(%) Memory-Usage(MB) HBM-Usage(MB) | + +======================+===============+====================================================+ + | 0 910B | OK | 79.4 82 0 / 0 | + | 0 | 0000:81:00.0 | 0 1979 / 15039 0 / 32768 | + +======================+===============+====================================================+ + ``` + +* Install the gmem-example software package. + + gmem-example updates the host driver, NPU driver, and NPU kernel. After the installation is complete, restart the system for the driver to take effect. + + ```sh + rpm -ivh gmem-example-xxx.aarch64.rpm + ``` + +* Install MindSpore. + + Obtain the correct MindSpore version and install it. After the installation, run the following command to check whether MindSpore functions are normal: + + ```sh + python -c "import mindspore;mindspore.run_check()" + MindSpore version: x.x.x + The result of multiplication calculation is correct, MindSpore has been installed on platform [Ascend] successfully! + ``` + +## Performing Training or Inference + +After installation is complete, you can execute MindSpore-based training or inference directly without any adaptation. diff --git a/docs/en/server/memory_storage/gmem/introduction-to-gmem.md b/docs/en/server/memory_storage/gmem/introduction-to-gmem.md new file mode 100644 index 0000000000000000000000000000000000000000..1841bc0ee7362b1c6a6020a23391215b389f4529 --- /dev/null +++ b/docs/en/server/memory_storage/gmem/introduction-to-gmem.md @@ -0,0 +1,37 @@ +# Introduction to GMEM + +## Introduction + +Memory management on the CPU side is separated from that on the heterogeneous accelerator side. Explicit data migration makes it difficult to balance usability and performance. The high bandwidth memory (HBM) of heterogeneous accelerators is generally insufficient for large language models, and manual swap causes a large performance loss and applies only to dedicated scenarios. A large number of invalid data migrations occur in search & recommendation and big data scenarios, and no efficient memory pooling solution is available. The Heterogeneous Memory Management (HMM) feature of Linux is confronted with complicated programming, unsatisfying performance, and poor portability, and depends greatly on manual tuning. As a result, OS communities are not willing to use HMM. An efficient memory management solution is needed to address the preceding issues on heterogeneous accelerators. + +Generalized Memory Management (GMEM) provides centralized management for heterogeneous memory interconnection. The GMEM APIs allow devices to connect to a unified address space and obtains programming optimization for heterogeneous memory, separating CPU architecture-related implementations from the memory management system of Linux. + +After the memory of the CPU and accelerator is encapsulated into a unified virtual address space, developers do not need to manually migrate the memory between two parallel address spaces. Instead, they only need to use a unified set of application and release functions. The dynamic random access memory (DRAM) of the CPU can even serve as the cache of the accelerator without much overhead. + +## Architecture + +![GMEM-architecture.png](images/GMEM_architecture.png) + +## Application Scenarios + +Foundation model training and inference + +* GMEM implements transparent heterogeneous memory capacity expansion and automatic HBM overcommitment, enabling high-performance and low-threshold training and inference. +* GMEM provides OS-native simplified heterogeneous memory management. With memory overcommitment, the performance of foundation model training is 60% higher than that of NVIDIA. + +Large memory sharing + +* GMEM provides flexible policies for remote access and on-demand memory migration to eliminate memory migration bottlenecks and improve end-to-end performance of search & recommendation and big data applications. + +## Functions + +For driver developers, GMEM provides unified function registration interfaces to reduce repeated work and the size of memory management code and avoid additional vulnerabilities. + +* Interfaces provided by GMEM can simplify the code for the driver to access the physical memory. +* The unified interfaces help avoid vulnerabilities when the driver developer implements the same function repeatedly. + +For users, GMEM provides stronger programmability for AI model and machine learning framework development using accelerators. You do not need to manually manage the data storage location. + +* Memory of the CPU and the accelerator can be accessed through unified memory application and release functions. +* Addresses of both CPU memory and accelerator memory can be mapped to the same virtual address space. +* GMEM encapsulates memory management code, improving the management performance. diff --git a/docs/en/server/memory_storage/gmem/usage-instructions.md b/docs/en/server/memory_storage/gmem/usage-instructions.md new file mode 100644 index 0000000000000000000000000000000000000000..14ae5dfaaf8381ae4092440127fa06d7519eba0e --- /dev/null +++ b/docs/en/server/memory_storage/gmem/usage-instructions.md @@ -0,0 +1,66 @@ +# Usage Instructions + +## Introduction + +GMEM applies for virtual memory that is peer-to-peer accessible through a specific flag and provides some memory optimization semantics externally. Performance can be optimized through the memory semantics. +libgmem is the abstraction layer of the GMEM user API. It encapsulates the preceding memory semantics to simplify user operations. + +## Available APIs + +* Memory application + + GMEM extends `mmap` semantics and adds a MAP_PEER_SHARED flag to apply for heterogeneous memory. When the flag is used, 2 MB-aligned virtual addresses are returned by default. + + ```c + addr = mmap(NULL , size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_PEER_SHARED, -1, 0); + ``` + +* Memory release + + `munmap` is used to release memory of hosts and devices. + + ```c + munmap(addr, size); + ``` + +* Memory semantics + + `FreeEager`: For an address segment within the specified range \[**addr**, **addr** + **size**], `FreeEager` releases a complete page that aligns the page size inwards (the default page size is 2 MB). If no complete page exists in the range, a success message is returned. + + If the API is invoked successfully, 0 is returned. Otherwise, an error code is returned. + + ```c + Prototype: `int gmemFreeEager(unsigned long addr, size_t size, void *stream);` + Usage: `ret = gmemFreeEager(addr, size, stream);` + ``` + + `Prefetch`: For an address segment with the specified range \[**addr**, **addr** + **size**], `Prefetch` prefetches a complete page (covering the entire address segment) whose range is outward aligned with the page size. This ensures that the subsequent access to the virtual memory area initiated by the specified computing unit device **hnid** does not trigger a page fault. + + If the API is invoked successfully, 0 is returned. Otherwise, an error code is returned. + + ```c + Prototype: `int gmemPrefetch(unsigned long addr, size_t size, int hnid, void *stream);` + Usage: `ret = gmemPrefetch(addr, size, hnid, stream);` + ``` + + If the value of **stream** is empty, the invocation is synchronous. Otherwise, the invocation is asynchronous. + +* Other APIs + + Obtain the NUMA ID of the current device. If the API is invoked successfully, the NUMA ID is returned. Otherwise, an error code is returned. + + ```c + Prototype: `int gmemGetNumaId (void);` + Usage: `numaid = gmemGetNumaId ();` + ``` + + Obtain the GMEM statistics of the kernel. + + ```sh + cat /proc/gmemstat + ``` + +## Constraints + +1. GMEM supports only 2 MB huge pages. Therefore, transparent huge pages of the host OS and NPU OS must be enabled to use GMEM. +2. The heterogeneous memory obtained using `MAP_PEER_SHARED` cannot be inherited during forking. diff --git a/docs/en/server/memory_storage/hsak/_toc.yaml b/docs/en/server/memory_storage/hsak/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..c93fb36d4e6d594029de00b679cf2e60dbf46582 --- /dev/null +++ b/docs/en/server/memory_storage/hsak/_toc.yaml @@ -0,0 +1,13 @@ +label: HSAK Developer Guide +isManual: true +href: ./hsak-developer-guide.md +description: >- + HSAK delivers a high-performance IO software stack optimized for new storage + media, offering both high bandwidth and low latency. +sections: + - label: Development with HSAK + href: ./development_with_hsak.md + - label: HSAK Tool Usage + href: ./hsak_tool_usage.md + - label: HSAK C APIs + href: ./hsak_c_apis.md diff --git a/docs/en/server/memory_storage/hsak/development_with_hsak.md b/docs/en/server/memory_storage/hsak/development_with_hsak.md new file mode 100644 index 0000000000000000000000000000000000000000..4a47c498968984c6fede9b08ff465af48b0ec1d0 --- /dev/null +++ b/docs/en/server/memory_storage/hsak/development_with_hsak.md @@ -0,0 +1,231 @@ +# Development with HSAK + +## Instructions + +### **nvme.conf.in** Configuration File + +By default, the HSAK configuration file is located in **/etc/spdk/nvme.conf.in**. You can modify the configuration file based on service requirements. The content of the configuration file is as follows: + +- \[Global\] + +1. **ReactorMask**: cores used for I/O polling. The value is a hexadecimal number and cannot be set to core 0. The bits from the least significant one to the most significant one indicate different CPU cores. For example, 0x1 indicates core 0, and 0x6 indicates cores 1 and 2. This parameter supports a maximum of 34 characters, including the hexadecimal flag **0x**. Each hexadecimal character can be F at most, indicating four cores. Therefore, a maximum of 128 (32 x 4) cores are supported. +2. **LogLevel**: HSAK log print level (**0**: error; **1**: warning; **2**: notice; **3**: info; **4**: debug). +3. **MemSize**: memory occupied by HSAK (The minimum value is 500 MB.) +4. **MultiQ**: whether to enable multi-queue on the same block device. +5. **E2eDif**: DIF type (**1**: half-way protection; **2**: full protection). Drives from different vendors may have different DIF support capabilities. For details, see the documents provided by hardware vendors. +6. **IoStat**: whether to enable the I/O statistics function. The options are **Yes** and **No**. +7. **RpcServer**: whether to start the RPC listening thread. The options are **Yes** and **No**. +8. **NvmeCUSE**: whether to enable the CUSE function. The options are **Yes** and **No**. After the function is enabled, the NVMe character device is generated in the **/dev/spdk** directory. + +- \[Nvme\] + +1. **TransportID**: PCI address and name of the NVMe controller. The format is **TransportID "trtype:PCIe traddr:0000:09:00.0" nvme0**. +2. **RetryCount**: number of retries upon an I/O failure. The value **0** indicates no retry. The maximum value is **255**. +3. **TimeoutUsec**: I/O timeout interval. If this parameter is set to **0** or left blank, no timeout interval is set. The unit is μs. +4. **ActionOnTimeout**: I/O timeout behavior (**None**: prints information only; **Reset**: resets the controller; **abort**: aborts the command). The default value is **None**. + +- \[Reactor\] + +1. **BatchSize**: number of I/Os that can be submitted in batches. The default value is **8**, and the maximum value is **32**. + +### Header File Reference + +HSAK provides two external header files. Include the two files when using HSAK for development. + +1. **bdev_rw.h**: defines the macros, enumerations, data structures, and APIs of the user-mode I/O operations on the data plane. +2. **ublock.h**: defines macros, enumerations, data structures, and APIs for functions such as device management and information obtaining on the management plane. + +### Service Running + +After software development and compilation, you must run the **setup.sh** script to rebind the NVMe drive driver to the user mode before running the software. The script is located in **/opt/spdk** by default. +Run the following commands to change the drive driver's binding mode from kernel to user and reserve 1024 x 2 MB huge pages: + +```shell +[root@localhost ~]# cd /opt/spdk +[root@localhost spdk]# ./setup.sh +0000:3f:00.0 (8086 2701): nvme -> uio_pci_generic +0000:40:00.0 (8086 2701): nvme -> uio_pci_generic +``` + +Run the following commands to restore the drive driver's mode from user to kernel and free the reserved huge pages: + +```shell +[root@localhost ~]# cd /opt/spdk +[root@localhost spdk]# ./setup.sh reset +0000:3f:00.0 (8086 2701): uio_pci_generic -> nvme +0000:40:00.0 (8086 2701): uio_pci_generic -> nvme +``` + +### User-Mode I/O Read and Write Scenarios + +Call HSAK APIs in the following sequence to read and write service data through the user-mode I/O channel: + +1. Initialize the HSAK UIO module. + Call **libstorage_init_module** to initialize the HSAK user-mode I/O channel. + +2. Open a drive block device. + Call **libstorage_open** to open a specified block device. If multiple block devices need to be opened, call this API repeatedly. + +3. Allocate I/O memory. + Call **libstorage_alloc_io_buf** or **libstorage_mem_reserve** to allocate memory. **libstorage_alloc_io_buf** can allocate a maximum of 65 KB I/Os, and **libstorage_mem_reserve** can allocate unlimited memory unless there is no available space. + +4. Perform read and write operations on a drive. + You can call the following APIs to perform read and write operations based on service requirements: + + - libstorage_async_read + - libstorage_async_readv + - libstorage_async_write + - libstorage_async_writev + - libstorage_sync_read + - libstorage_sync_write + +5. Free I/O memory. + Call **libstorage_free_io_buf** or **libstorage_mem_free** to free memory, which must correspond to the API used to allocate memory. + +6. Close a drive block device. + Call **libstorage_close** to close a specified block device. If multiple block devices are opened, call this API repeatedly to close them. + + | API | Description | + | ----------------------- | ----------------------------------------------------------------------------------- | + | libstorage_init_module | Initializes the HSAK module. | + | libstorage_open | Opens a block device. | + | libstorage_alloc_io_buf | Allocates memory from buf_small_pool or buf_large_pool of SPDK. | + | libstorage_mem_reserve | Allocates memory space from the huge page memory reserved by DPDK. | + | libstorage_async_read | Delivers asynchronous I/O read requests (the read buffer is a contiguous buffer). | + | libstorage_async_readv | Delivers asynchronous I/O read requests (the read buffer is a discrete buffer). | + | libstorage_async_write | Delivers asynchronous I/O write requests (the write buffer is a contiguous buffer). | + | libstorage_async_wrtiev | Delivers asynchronous I/O write requests (the write buffer is a discrete buffer). | + | libstorage_sync_read | Delivers synchronous I/O read requests (the read buffer is a contiguous buffer). | + | libstorage_sync_write | Delivers synchronous I/O write requests (the write buffer is a contiguous buffer). | + | libstorage_free_io_buf | Frees the allocated memory to buf_small_pool or buf_large_pool of SPDK. | + | libstorage_mem_free | Frees the memory space that libstorage_mem_reserve allocates. | + | libstorage_close | Closes a block device. | + | libstorage_exit_module | Exits the HSAK module. | + +### Drive Management Scenarios + +HSAK contains a group of C APIs, which can be used to format drives and create and delete namespaces. + +1. Call the C API to initialize the HSAK UIO component. If the HSAK UIO component has been initialized, skip this operation. + + libstorage_init_module + +2. Call corresponding APIs to perform drive operations based on service requirements. The following APIs can be called separately: + + - libstorage_create_namespace + + - libstorage_delete_namespace + + - libstorage_delete_all_namespace + + - libstorage_nvme_create_ctrlr + + - libstorage_nvme_delete_ctrlr + + - libstorage_nvme_reload_ctrlr + + - libstorage_low_level_format_nvm + + - libstorage_deallocate_block + +3. If you exit the program, destroy the HSAK UIO. If other services are using the HSAK UIO, you do not need to exit the program and destroy the HSAK UIO. + + libstorage_exit_module + + | API | Description | + | ------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | + | libstorage_create_namespace | Creates a namespace on a specified controller (the prerequisite is that the controller supports namespace management). | + | libstorage_delete_namespace | Deletes a namespace from a specified controller. | + | libstorage_delete_all_namespace | Deletes all namespaces from a specified controller. | + | libstorage_nvme_create_ctrlr | Creates an NVMe controller based on the PCI address. | + | libstorage_nvme_delete_ctrlr | Destroys an NVMe controller based on the controller name. | + | libstorage_nvme_reload_ctrlr | Automatically creates or destroys the NVMe controller based on the input configuration file. | + | libstorage_low_level_format_nvm | Low-level formats an NVMe drive. | + | libstorage_deallocate_block | Notifies NVMe drives of blocks that can be freed for garbage collection. | + +### Data-Plane Drive Information Query + +The I/O data plane of HSAK provides a group of C APIs for querying drive information. Upper-layer services can process service logic based on the queried information. + +1. Call the C API to initialize the HSAK UIO component. If the HSAK UIO component has been initialized, skip this operation. + + libstorage_init_module + +2. Call corresponding APIs to query information based on service requirements. The following APIs can be called separately: + + - libstorage_get_nvme_ctrlr_info + + - libstorage_get_mgr_info_by_esn + + - libstorage_get_mgr_smart_by_esn + + - libstorage_get_bdev_ns_info + + - libstorage_get_ctrl_ns_info + +3. If you exit the program, destroy the HSAK UIO. If other services are using the HSAK UIO, you do not need to exit the program and destroy the HSAK UIO. + + libstorage_exit_module + + | API | Description | + | ------------------------------- | ------------------------------------------------------------------------ | + | libstorage_get_nvme_ctrlr_info | Obtains information about all controllers. | + | libstorage_get_mgr_info_by_esn | Obtains the management information of the drive corresponding to an ESN. | + | libstorage_get_mgr_smart_by_esn | Obtains the S.M.A.R.T. information of the drive corresponding to an ESN. | + | libstorage_get_bdev_ns_info | Obtains namespace information based on the device name. | + | libstorage_get_ctrl_ns_info | Obtains information about all namespaces based on the controller name. | + +### Management-Plane Drive Information Query + +The management plane component Ublock of HSAK provides a group of C APIs for querying drive information on the management plane. + +1. Call the C API to initialize the HSAK Ublock server. + +2. Call the HSAK UIO component initialization API in another process based on service requirements. + +3. If multiple processes are required to query drive information, initialize the Ublock client. + +4. Call the APIs listed in the following table on the Ublock server process or client process to query information. + +5. After obtaining the block device list, call the APIs listed in the following table to free resources. + +6. If you exit the program, destroy the HSAK Ublock module (the destruction method on the server is the same as that on the client). + + | API | Description | + | ---------------------------- | ------------------------------------------------------------ | + | init_ublock | Initializes the Ublock function module. This API must be called before the other Ublock APIs. A process can be initialized only once because the init_ublock API initializes DPDK. The initial memory allocated by DPDK is bound to the process PID. One PID can be bound to only one memory. In addition, DPDK does not provide an API for freeing the memory. The memory can be freed only by exiting the process. | + | ublock_init | It is the macro definition of the init_ublock API. It can be considered as initializing Ublock to an RPC service. | + | ublock_init_norpc | It is the macro definition of the init_ublock API. It can be considered as initializing Ublock to a non-RPC service. | + | ublock_get_bdevs | Obtains the device list. The obtained device list contains only PCI addresses and does not contain specific device information. To obtain specific device information, call the ublock_get_bdev API. | + | ublock_get_bdev | Obtains information about a specific device, including the device serial number, model, and firmware version. The information is stored in character arrays instead of character strings. | + | ublock_get_bdev_by_esn | Obtains the device information based on the specified ESN, including the serial number, model, and firmware version. | + | ublock_get_SMART_info | Obtains the S.M.A.R.T. information of a specified device. | + | ublock_get_SMART_info_by_esn | Obtains the S.M.A.R.T. information of the device corresponding to an ESN. | + | ublock_get_error_log_info | Obtains the error log information of a device. | + | ublock_get_log_page | Obtains information about a specified log page of a specified device. | + | ublock_free_bdevs | Frees the device list. | + | ublock_free_bdev | Frees device resources. | + | ublock_fini | Destroys the Ublock module. This API destroys the Ublock module and internally created resources. This API must be used together with the Ublock initialization API. | + +### Log Management + +HSAK logs are exported to **/var/log/messages** through syslog by default and managed by the rsyslog service of the OS. If a custom log directory is required, use rsyslog to configure the log directory. + +1. Modify the **/etc/rsyslog.conf** configuration file. + +2. Restart the rsyslog service: + + ```shell + if ($programname == 'LibStorage') then { + action(type="omfile" fileCreateMode="0600" file="/var/log/HSAK/run.log") + stop + } + ``` + +3. Start the HSAK process. The log information is redirected to the target directory. + + ```shell + sysemctl restart rsyslog + ``` + +4. If redirected logs need to be dumped, manually configure log dump in the **/etc/logrotate.d/syslog** file. diff --git a/docs/en/server/memory_storage/hsak/hsak-developer-guide.md b/docs/en/server/memory_storage/hsak/hsak-developer-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..3aa9395bc75c6e693f79546e17b4afb67da59159 --- /dev/null +++ b/docs/en/server/memory_storage/hsak/hsak-developer-guide.md @@ -0,0 +1,47 @@ +# HSAK Developer Guide + +## Overview + +As the performance of storage media such as NVMe SSDs and SCMs is continuously improved, the latency overhead of the media layer in the I/O stack continuously reduces, and the overhead of the software stack becomes a bottleneck. Therefore, the kernel I/O data plane needs to be reconstructed to reduce the overhead of the software stack. HSAK provides a high-bandwidth and low-latency I/O software stack for new storage media, which reduces the overhead by more than 50% compared with the traditional I/O software stack. +The HSAK user-mode I/O engine is developed based on the open-source SPDK. + +1. A unified interface is provided for external systems to shield the differences between open-source interfaces. +2. Enhanced I/O data plane features are added, such as DIF, drive formatting, batch I/O delivery, trim, and dynamic drive addition and deletion. +3. Features such as drive device management, drive I/O monitoring, and maintenance and test tools are provided. + +## Compilation Tutorial + +1. Download the HSAK source code. + + ```shell + git clone + ``` + +2. Install the compilation and running dependencies. + + The compilation and running of HSAK depend on components such as Storage Performance Development Kit (SPDK), Data Plane Development Kit (DPDK), and libboundscheck. + +3. Start the compilation. + + ```shell + cd hsak + mkdir build + cd build + cmake .. + make + ``` + +## Precautions + +### Constraints + +- A maximum of 512 NVMe devices can be used and managed on the same machine. +- When HSAK is enabled to execute I/O-related services, ensure that the system has at least 500 MB continuous idle huge page memory. +- Before enabling the user-mode I/O component to execute services, ensure that the drive management component (Ublock) has been enabled. +- When the drive management component (Ublock) is enabled to execute services, ensure that the system has sufficient continuous idle memory. Each time the Ublock component is initialized, 20 MB huge page memory is allocated. +- Before HSAK is run, **setup.sh** is called to configure huge page memory and unbind the kernel-mode driver of the NVMe device. +- Other interfaces provided by the HSAK module can be used only after libstorage_init_module is successfully executed. Each process can call libstorage_init_module only once. +- After the libstorage_exit_module function is executed, other interfaces provided by HSAK cannot be used. In multi-thread scenarios, exit HSAK after all threads end. +- Only one service can be started for the HSAK Ublock component on a server and supports concurrent access of a maximum of 64 Ublock clients. The Ublock server can process a maximum of 20 client requests per second. +- The HSAK Ublock component must be started earlier than the data plane I/O component and Ublock clients. The command line tool provided by HSAK can be executed only after the Ublock server is started. +- Do not register the function for processing the SIGBUS signal. SPDK has an independent processing function for the signal. If the processing function is overwritten, the registered signal processing function becomes invalid and a core dump occurs. diff --git a/docs/en/server/memory_storage/hsak/hsak_c_apis.md b/docs/en/server/memory_storage/hsak/hsak_c_apis.md new file mode 100644 index 0000000000000000000000000000000000000000..ccb1e774803c91c94e4aa95a760d157b6ba73aa7 --- /dev/null +++ b/docs/en/server/memory_storage/hsak/hsak_c_apis.md @@ -0,0 +1,2521 @@ +# C APIs + +## Macro Definition and Enumeration + +### bdev_rw.h + +#### enum libstorage_ns_lba_size + +1. Prototype + + ```c + enum libstorage_ns_lba_size + { + LIBSTORAGE_NVME_NS_LBA_SIZE_512 = 0x9, + LIBSTORAGE_NVME_NS_LBA_SIZE_4K = 0xc + }; + ``` + +2. Description + + Sector (data) size of a drive. + +#### enum libstorage_ns_md_size + +1. Prototype + + ```c + enum libstorage_ns_md_size + { + LIBSTORAGE_METADATA_SIZE_0 = 0, + LIBSTORAGE_METADATA_SIZE_8 = 8, + LIBSTORAGE_METADATA_SIZE_64 = 64 + }; + ``` + +2. Description + + Metadata size of a drive. + +3. Remarks + + - ES3000 V3 (single-port) supports formatting of five sector types (512+0, 512+8, 4K+64, 4K, and 4K+8). + + - ES3000 V3 (dual-port) supports formatting of four sector types (512+0, 512+8, 4K+64, and 4K). + + - ES3000 V5 supports formatting of five sector types (512+0, 512+8, 4K+64, 4K, and 4K+8). + + - Optane drives support formatting of seven sector types (512+0, 512+8, 512+16,4K, 4K+8, 4K+64, and 4K+128). + +#### enum libstorage_ns_pi_type + +1. Prototype + + ```c + enum libstorage_ns_pi_type + { + LIBSTORAGE_FMT_NVM_PROTECTION_DISABLE = 0x0, + LIBSTORAGE_FMT_NVM_PROTECTION_TYPE1 = 0x1, + LIBSTORAGE_FMT_NVM_PROTECTION_TYPE2 = 0x2, + LIBSTORAGE_FMT_NVM_PROTECTION_TYPE3 = 0x3, + }; + ``` + +2. Description + + Protection type supported by drives. + +3. Remarks + + ES3000 supports only protection types 0 and 3. Optane drives support only protection types 0 and 1. + +#### enum libstorage_crc_and_prchk + +1. Prototype + + ```c + enum libstorage_crc_and_prchk + { + LIBSTORAGE_APP_CRC_AND_DISABLE_PRCHK = 0x0, + LIBSTORAGE_APP_CRC_AND_ENABLE_PRCHK = 0x1, + LIBSTORAGE_LIB_CRC_AND_DISABLE_PRCHK = 0x2, + LIBSTORAGE_LIB_CRC_AND_ENABLE_PRCHK = 0x3, + #define NVME_NO_REF 0x4 + LIBSTORAGE_APP_CRC_AND_DISABLE_PRCHK_NO_REF = LIBSTORAGE_APP_CRC_AND_DISABLE_PRCHK | NVME_NO_REF, + LIBSTORAGE_APP_CRC_AND_ENABLE_PRCHK_NO_REF = LIBSTORAGE_APP_CRC_AND_ENABLE_PRCHK | NVME_NO_REF, + }; + ``` + +2. Description + + - **LIBSTORAGE_APP_CRC_AND_DISABLE_PRCHK**: Cyclic redundancy check (CRC) is performed for the application layer, but not for HSAK. CRC is disabled for drives. + + - **LIBSTORAGE_APP_CRC_AND_ENABLE_PRCHK**: CRC is performed for the application layer, but not for HSAK. CRC is enabled for drives. + + - **LIBSTORAGE_LIB_CRC_AND_DISABLE_PRCHK**: CRC is performed for HSAK, but not for the application layer. CRC is disabled for drives. + + - **LIBSTORAGE_LIB_CRC_AND_ENABLE_PRCHK**: CRC is performed for HSAK, but not for the application layer. CRC is enabled for drives. + + - **LIBSTORAGE_APP_CRC_AND_DISABLE_PRCHK_NO_REF**: CRC is performed for the application layer, but not for HSAK. CRC is disabled for drives. REF tag verification is disabled for drives whose PI TYPE is 1 (Intel Optane P4800). + + - **LIBSTORAGE_APP_CRC_AND_ENABLE_PRCHK_NO_REF**: CRC is performed for the application layer, but not for HSAK. CRC is enabled for drives. REF tag verification is disabled for drives whose PI TYPE is 1 (Intel Optane P4800). + + - If PI TYPE of an Intel Optane P4800 drive is 1, the CRC and REF tag of the metadata area are verified by default. + + - Intel Optane P4800 drives support DIF in 512+8 format but does not support DIF in 4096+64 format. + + - For ES3000 V3 and ES3000 V5, PI TYPE of the drives is 3. By default, only the CRC of the metadata area is performed. + + - ES3000 V3 supports DIF in 512+8 format but does not support DIF in 4096+64 format. ES3000 V5 supports DIF in both 512+8 and 4096+64 formats. + + The summary is as follows: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
E2E Verification ModeCtrl FlagCRC Generator Write ProcessRead Process
Application VerificationCRC for HSAKCRC for DrivesApplication VerificationCRC for HSAKCRC for Drives
Halfway protection0ControllerXXXXXX
1ControllerXXXXX√
2ControllerXXXXXX
3ControllerXXXXX√
Full protection0App√XX√XX
1App√X√√X√
2HSAKX√XX√X
3HSAKX√√X√√
+ +#### enum libstorage_print_log_level + +1. Prototype + + ```c + enum libstorage_print_log_level + { + LIBSTORAGE_PRINT_LOG_ERROR, + LIBSTORAGE_PRINT_LOG_WARN, + LIBSTORAGE_PRINT_LOG_NOTICE, + LIBSTORAGE_PRINT_LOG_INFO, + LIBSTORAGE_PRINT_LOG_DEBUG, + }; + ``` + +2. Description + + Storage Performance Development Kit (SPDK) log print levels: ERROR, WARN, NOTICE, INFO, and DEBUG, corresponding to 0 to 4 in the configuration file. + +#### MAX_BDEV_NAME_LEN + +1. Prototype + + ```c + #define MAX_BDEV_NAME_LEN 24 + ``` + +2. Description + + Maximum length of a block device name. + +#### MAX_CTRL_NAME_LEN + +1. Prototype + + ```c + #define MAX_CTRL_NAME_LEN 16 + ``` + +2. Description + + Maximum length of a controller. + +#### LBA_FORMAT_NUM + +1. Prototype + + ```c + #define LBA_FORMAT_NUM 16 + ``` + +2. Description + + Number of LBA formats supported by a controller. + +#### LIBSTORAGE_MAX_DSM_RANGE_DESC_COUNT + +1. Prototype + + ```c + #define LIBSTORAGE_MAX_DSM_RANGE_DESC_COUNT 256 + ``` + +2. Description + + Maximum number of 16-byte sets in the dataset management command. + +### ublock.h + +#### UBLOCK_NVME_UEVENT_SUBSYSTEM_UIO + +1. Prototype + + ```c + #define UBLOCK_NVME_UEVENT_SUBSYSTEM_UIO 1 + ``` + +2. Description + + This macro is used to define that the subsystem corresponding to the uevent event is the userspace I/O subsystem (UIO) provided by the kernel. When the service receives the uevent event, this macro is used to determine whether the event is a UIO event that needs to be processed. + + The value of the int subsystem member in struct ublock_uevent is **UBLOCK_NVME_UEVENT_SUBSYSTEM_UIO**. Currently, only this value is available. + +#### UBLOCK_TRADDR_MAX_LEN + +1. Prototype + + ```c + #define UBLOCK_TRADDR_MAX_LEN 256 + ``` + +2. Description + + The *Domain:Bus:Device.Function* (**%04x:%02x:%02x.%x**) format indicates the maximum length of the PCI address character string. The actual length is far less than 256 bytes. + +#### UBLOCK_PCI_ADDR_MAX_LEN + +1. Prototype + + ```c + #define UBLOCK_PCI_ADDR_MAX_LEN 256 + ``` + +2. Description + + Maximum length of the PCI address character string. The actual length is far less than 256 bytes. The possible formats of the PCI address are as follows: + + - Full address: **%x:%x:%x.%x** or **%x.%x.%x.%x** + + - When the **Function** value is **0**: **%x:%x:%x** + + - When the **Domain** value is **0**: **%x:%x.%x** or **%x.%x.%x** + + - When the **Domain** and **Function** values are **0**: **%x:%x** or **%x.%x** + +#### UBLOCK_SMART_INFO_LEN + +1. Prototype + + ```c + #define UBLOCK_SMART_INFO_LEN 512 + ``` + +2. Description + + Size of the structure for the S.M.A.R.T. information of an NVMe drive, which is 512 bytes. + +#### enum ublock_rpc_server_status + +1. Prototype + + ```c + enum ublock_rpc_server_status { + // start rpc server or not + UBLOCK_RPC_SERVER_DISABLE = 0, + UBLOCK_RPC_SERVER_ENABLE = 1, + }; + ``` + +2. Description + + Status of the RPC service in HSAK. The status can be enabled or disabled. + +#### enum ublock_nvme_uevent_action + +1. Prototype + + ```c + enum ublock_nvme_uevent_action { + UBLOCK_NVME_UEVENT_ADD = 0, + UBLOCK_NVME_UEVENT_REMOVE = 1, + UBLOCK_NVME_UEVENT_INVALID, + }; + ``` + +2. Description + + Indicates whether the uevent hot swap event is to insert or remove a drive. + +#### enum ublock_subsystem_type + +1. Prototype + + ```c + enum ublock_subsystem_type { + SUBSYSTEM_UIO = 0, + SUBSYSTEM_NVME = 1, + SUBSYSTEM_TOP + }; + ``` + +2. Description + + Type of the callback function, which is used to determine whether the callback function is registered for the UIO driver or kernel NVMe driver. + +## Data Structure + +### bdev_rw.h + +#### struct libstorage_namespace_info + +1. Prototype + + ```c + struct libstorage_namespace_info + { + char name[MAX_BDEV_NAME_LEN]; + uint64_t size; /** namespace size in bytes */ + uint64_t sectors; /** number of sectors */ + uint32_t sector_size; /** sector size in bytes */ + uint32_t md_size; /** metadata size in bytes */ + uint32_t max_io_xfer_size; /** maximum i/o size in bytes */ + uint16_t id; /** namespace id */ + uint8_t pi_type; /** end-to-end data protection information type */ + uint8_t is_active :1; /** namespace is active or not */ + uint8_t ext_lba :1; /** namespace support extending LBA size or not */ + uint8_t dsm :1; /** namespace supports Dataset Management or not */ + uint8_t pad :3; + uint64_t reserved; + }; + ``` + +2. Description + + This data structure contains the namespace information of a drive. + +3. Struct members + + | Member | Description | + | ---------------------------- | ------------------------------------------------------------ | + | char name\[MAX_BDEV_NAME_LEN] | Name of the namespace. | + | uint64_t size | Size of the drive space allocated to the namespace, in bytes. | + | uint64_t sectors | Number of sectors. | + | uint32_t sector_size | Size of each sector, in bytes. | + | uint32_t md_size | Metadata size, in bytes. | + | uint32_t max_io_xfer_size | Maximum size of data in a single I/O operation, in bytes. | + | uint16_t id | Namespace ID. | + | uint8_t pi_type | Data protection type. The value is obtained from enum libstorage_ns_pi_type. | + | uint8_t is_active :1 | Namespace active or not. | + | uint8_t ext_lba :1 | Whether the namespace supports logical block addressing (LBA) in extended mode. | + | uint8_t dsm :1 | Whether the namespace supports dataset management. | + | uint8_t pad :3 | Reserved parameter. | + | uint64_t reserved | Reserved parameter. | + +#### struct libstorage_nvme_ctrlr_info + +1. Prototype + + ```c + struct libstorage_nvme_ctrlr_info { + char name[MAX_CTRL_NAME_LEN]; + char address[24]; + struct { + uint32_t domain; + uint8_t bus; + uint8_t dev; + uint8_t func; + } pci_addr; + uint64_t totalcap; /* Total NVM Capacity in bytes */ + uint64_t unusecap; /* Unallocated NVM Capacity in bytes */ + int8_t sn[20]; /* Serial number */ + uint8_t fr[8]; /* Firmware revision */ + uint32_t max_num_ns; /* Number of namespaces */ + uint32_t version; + uint16_t num_io_queues; /* num of io queues */ + uint16_t io_queue_size; /* io queue size */ + uint16_t ctrlid; /* Controller id */ + uint16_t pad1; + struct { + struct { + uint32_t ms : 16; /* metadata size */ + uint32_t lbads : 8; /* lba data size */ + uint32_t reserved : 8; + } lbaf[LBA_FORMAT_NUM]; + uint8_t nlbaf; + uint8_t pad2[3]; + uint32_t cur_format : 4; + uint32_t cur_extended : 1; + uint32_t cur_pi : 3; + uint32_t cur_pil : 1; + uint32_t cur_can_share : 1; + uint32_t mc_extented : 1; + uint32_t mc_pointer : 1; + uint32_t pi_type1 : 1; + uint32_t pi_type2 : 1; + uint32_t pi_type3 : 1; + uint32_t md_start : 1; + uint32_t md_end : 1; + uint32_t ns_manage : 1; /* Supports the Namespace Management and Namespace Attachment commands */ + uint32_t directives : 1; /* Controller support Directives or not */ + uint32_t streams : 1; /* Controller support Streams Directives or not */ + uint32_t dsm : 1; /* Controller support Dataset Management or not */ + uint32_t reserved : 11; + } cap_info; + }; + ``` + +2. Description + + This data structure contains the controller information of a drive. + +3. Struct members + + | Member | Description | + | ------------------------------------------------------------ | ------------------------------------------------------------ | + | char name\[MAX_CTRL_NAME_LEN] | Controller name. | + | char address\[24] | PCI address, which is a character string. | + | struct
{
uint32_t domain;
uint8_t bus;
uint8_t dev;
uint8_t func;
} pci_addr | PCI address, in segments. | + | uint64_t totalcap | Total capacity of the controller, in bytes. Optane drives are based on the NVMe 1.0 protocol and do not support this parameter. | + | uint64_t unusecap | Free capacity of the controller, in bytes. Optane drives are based on the NVMe 1.0 protocol and do not support this parameter. | + | int8_t sn\[20]; | Serial number of a drive, which is an ASCII character string without **0**. | + | uint8_t fr\[8]; | Drive firmware version, which is an ASCII character string without **0**. | + | uint32_t max_num_ns | Maximum number of namespaces. | + | uint32_t version | NVMe protocol version supported by the controller. | + | uint16_t num_io_queues | Number of I/O queues supported by a drive. | + | uint16_t io_queue_size | Maximum length of an I/O queue. | + | uint16_t ctrlid | Controller ID. | + | uint16_t pad1 | Reserved parameter. | + + Members of the struct cap_info substructure: + + | Member | Description | + | ------------------------------------------------------------ | ------------------------------------------------------------ | + | struct
{
uint32_t ms : 16;
uint32_t lbads : 8;
uint32_t reserved : 8;
}lbaf\[LBA_FORMAT_NUM] | **ms**: metadata size. The minimum value is 8 bytes.
**lbads**: The LBA size is 2^lbads, and the value of **lbads** is greater than or equal to 9. | + | uint8_t nlbaf | Number of LBA formats supported by the controller. | + | uint8_t pad2\[3] | Reserved parameter. | + | uint32_t cur_format : 4 | Current LBA format of the controller. | + | uint32_t cur_extended : 1 | Whether the controller supports LBA in extended mode. | + | uint32_t cur_pi : 3 | Current protection type of the controller. | + | uint32_t cur_pil : 1 | The current protection information (PI) of the controller is located in the first or last eight bytes of the metadata. | + | uint32_t cur_can_share : 1 | Whether the namespace supports multi-path transmission. | + | uint32_t mc_extented : 1 | Whether metadata is transmitted as part of the data buffer. | + | uint32_t mc_pointer : 1 | Whether metadata is separated from the data buffer. | + | uint32_t pi_type1 : 1 | Whether the controller supports protection type 1. | + | uint32_t pi_type2 : 1 | Whether the controller supports protection type 2. | + | uint32_t pi_type3 : 1 | Whether the controller supports protection type 3. | + | uint32_t md_start : 1 | Whether the controller supports protection information in the first eight bytes of metadata. | + | uint32_t md_end : 1 | Whether the controller supports protection information in the last eight bytes of metadata. | + | uint32_t ns_manage : 1 | Whether the controller supports namespace management. | + | uint32_t directives : 1 | Whether the Directives command set is supported. | + | uint32_t streams : 1 | Whether Streams Directives is supported. | + | uint32_t dsm : 1 | Whether Dataset Management commands are supported. | + | uint32_t reserved : 11 | Reserved parameter. | + +#### struct libstorage_dsm_range_desc + +1. Prototype + + ```c + struct libstorage_dsm_range_desc + { + /* RESERVED */ + uint32_t reserved; + + /* NUMBER OF LOGICAL BLOCKS */ + uint32_t block_count; + + /* UNMAP LOGICAL BLOCK ADDRESS */uint64_t lba;}; + ``` + +2. Description + + Definition of a single 16-byte set in the data management command set. + +3. Struct members + + | Member | Description | + | -------------------- | ------------------------ | + | uint32_t reserved | Reserved parameter. | + | uint32_t block_count | Number of LBAs per unit. | + | uint64_t lba | Start LBA. | + +#### struct libstorage_ctrl_streams_param + +1. Prototype + + ```c + struct libstorage_ctrl_streams_param + { + /* MAX Streams Limit */ + uint16_t msl; + + /* NVM Subsystem Streams Available */ + uint16_t nssa; + + /* NVM Subsystem Streams Open */uint16_t nsso; + + uint16_t pad; + }; + ``` + +2. Description + + Streams attribute value supported by NVMe drives. + +3. Struct members + + | Member | Description | + | ------------- | ------------------------------------------------------------ | + | uint16_t msl | Maximum number of Streams resources supported by a drive. | + | uint16_t nssa | Number of Streams resources that can be used by each NVM subsystem. | + | uint16_t nsso | Number of Streams resources used by each NVM subsystem. | + | uint16_t pad | Reserved parameter. | + +#### struct libstorage_bdev_streams_param + +1. Prototype + + ```c + struct libstorage_bdev_streams_param + { + /* Stream Write Size */ + uint32_t sws; + + /* Stream Granularity Size */ + uint16_t sgs; + + /* Namespace Streams Allocated */ + uint16_t nsa; + + /* Namespace Streams Open */ + uint16_t nso; + + uint16_t reserved[3]; + }; + ``` + +2. Description + + Streams attribute value of the namespace. + +3. Struct members + + | Member | Description | + | -------------------- | ------------------------------------------------------------ | + | uint32_t sws | Write granularity with the optimal performance, in sectors. | + | uint16_t sgs | Write granularity allocated to Streams, in sws. | + | uint16_t nsa | Number of private Streams resources that can be used by a namespace. | + | uint16_t nso | Number of private Streams resources used by a namespace. | + | uint16_t reserved\[3] | Reserved parameter. | + +#### struct libstorage_mgr_info + +1. Prototype + + ```c + struct libstorage_mgr_info + { + char pci[24]; + char ctrlName[MAX_CTRL_NAME_LEN]; + uint64_t sector_size; + uint64_t cap_size; + uint16_t device_id; + uint16_t subsystem_device_id; + uint16_t vendor_id; + uint16_t subsystem_vendor_id; + uint16_t controller_id; + int8_t serial_number[20]; + int8_t model_number[40]; + uint8_t firmware_revision[8]; + }; + ``` + +2. Description + + Drive management information (consistent with the drive information used by the management plane). + +3. Struct members + + | Member | Description | + | -------------------------------- | ---------------------------------------------- | + | char pci\[24] | Character string of the drive PCI address. | + | char ctrlName\[MAX_CTRL_NAME_LEN] | Character string of the drive controller name. | + | uint64_t sector_size | Drive sector size. | + | uint64_t cap_size | Drive capacity, in bytes. | + | uint16_t device_id | Drive device ID. | + | uint16_t subsystem_device_id | Drive subsystem device ID. | + | uint16\*t vendor\*id | Drive vendor ID. | + | uint16_t subsystem_vendor_id | Drive subsystem vendor ID. | + | uint16_t controller_id | Drive controller ID. | + | int8_t serial_number\[20] | Drive serial number. | + | int8_t model_number\[40] | Device model. | + | uint8_t firmware_revision\[8] | Firmware version. | + +#### struct **attribute**((packed)) libstorage_smart_info + +1. Prototype + + ```c + /* same with struct spdk_nvme_health_information_page in nvme_spec.h */ + struct __attribute__((packed)) libstorage_smart_info { + /* details of uint8_t critical_warning + * + * union spdk_nvme_critical_warning_state { + * uint8_t raw; + * struct { + * uint8_t available_spare : 1; + * uint8_t temperature : 1; + * uint8_t device_reliability : 1; + * uint8_t read_only : 1; + * uint8_t volatile_memory_backup : 1; + * uint8_t reserved : 3; + * } bits; + * }; + */ + uint8_t critical_warning; + uint16_t temperature; + uint8_t available_spare; + uint8_t available_spare_threshold; + uint8_t percentage_used; + uint8_t reserved[26]; + + /* + * Note that the following are 128-bit values, but are + * defined as an array of 2 64-bit values. + */ + /* Data Units Read is always in 512-byte units. */ + uint64_t data_units_read[2]; + /* Data Units Written is always in 512-byte units. */ + uint64_t data_units_written[2]; + /* For NVM command set, this includes Compare commands. */ + uint64_t host_read_commands[2]; + uint64_t host_write_commands[2]; + /* Controller Busy Time is reported in minutes. */ + uint64_t controller_busy_time[2]; + uint64_t power_cycles[2]; + uint64_t power_on_hours[2]; + uint64_t unsafe_shutdowns[2]; + uint64_t media_errors[2]; + uint64_t num_error_info_log_entries[2]; + + /* Controller temperature related. */ + uint32_t warning_temp_time; + uint32_t critical_temp_time; + uint16_t temp_sensor[8]; + uint8_t reserved2[296]; + }; + + ``` + +2. Description + + This data structure defines the S.M.A.R.T. information of a drive. + +3. Struct members + + | Member | **Description (For details, see the NVMe protocol.)** | + | -------------------------------------- | ------------------------------------------------------------ | + | uint8_t critical_warning | Critical alarm of the controller status. If a bit is set to 1, the bit is valid. You can set multiple bits to be valid. Critical alarms are returned to the host through asynchronous events.
Bit 0: When this bit is set to 1, the redundant space is less than the specified threshold.
Bit 1: When this bit is set to 1, the temperature is higher or lower than a major threshold.
Bit 2: When this bit is set to 1, component reliability is reduced due to major media errors or internal errors.
Bit 3: When this bit is set to 1, the medium has been set to the read-only mode.
Bit 4: When this bit is set to 1, the volatile component of the controller fails. This parameter is valid only when the volatile component exists in the controller.
Bits 5-7: reserved. | + | uint16_t temperature | Temperature of a component. The unit is Kelvin. | + | uint8_t available_spare | Percentage of the available redundant space (0 to 100%). | + | uint8_t available_spare_threshold | Threshold of the available redundant space. An asynchronous event is reported when the available redundant space is lower than the threshold. | + | uint8_t percentage_used | Percentage of the actual service life of a component to the service life of the component expected by the manufacturer. The value **100** indicates that the actual service life of the component has reached to the expected service life, but the component can still be used. The value can be greater than 100, but any value greater than 254 will be set to 255. | + | uint8_t reserved\[26] | Reserved. | + | uint64_t data_units_read\[2] | Number of 512 bytes read by the host from the controller. The value **1** indicates that 1000 x 512 bytes are read, which exclude metadata. If the LBA size is not 512 bytes, the controller converts it into 512 bytes for calculation. The value is expressed in hexadecimal notation. | + | uint64_t data_units_written\[2] | Number of 512 bytes written by the host to the controller. The value **1** indicates that 1000 x 512 bytes are written, which exclude metadata. If the LBA size is not 512 bytes, the controller converts it into 512 bytes for calculation. The value is expressed in hexadecimal notation. | + | uint64_t host_read_commands\[2] | Number of read commands delivered to the controller. | + | uint64_t host_write_commands\[2]; | Number of write commands delivered to the controller. | + | uint64_t controller_busy_time\[2] | Busy time for the controller to process I/O commands. The process from the time the commands are delivered to the time the results are returned to the CQ is busy. The time is expressed in minutes. | + | uint64_t power_cycles\[2] | Number of machine on/off cycles. | + | uint64_t power_on_hours\[2] | Power-on duration, in hours. | + | uint64_t unsafe_shutdowns\[2] | Number of abnormal power-off times. The value is incremented by 1 when CC.SHN is not received during power-off. | + | uint64_t media_errors\[2] | Number of times that the controller detects unrecoverable data integrity errors, including uncorrectable ECC errors, CRC errors, and LBA tag mismatch. | + | uint64_t num_error_info_log_entries\[2] | Number of entries in the error information log within the controller lifecycle. | + | uint32_t warning_temp_time | Accumulated time when the temperature exceeds the warning alarm threshold, in minutes. | + | uint32_t critical_temp_time | Accumulated time when the temperature exceeds the critical alarm threshold, in minutes. | + | uint16_t temp_sensor\[8] | Temperature of temperature sensors 1-8. The unit is Kelvin. | + | uint8_t reserved2\[296] | Reserved. | + +#### libstorage_dpdk_contig_mem + +1. Prototype + + ```c + struct libstorage_dpdk_contig_mem { + uint64_t virtAddr; + uint64_t memLen; + uint64_t allocLen; + }; + ``` + +2. Description + + Description about a contiguous virtual memory segment in the parameters of the callback function that notifies the service layer of initialization completion after the DPDK memory is initialized. + + Currently, 800 MB memory is reserved for HSAK. Other memory is returned to the service layer through **allocLen** in this struct for the service layer to allocate memory for self-management. + + The total memory to be reserved for HSAK is about 800 MB. The memory reserved for each memory segment is calculated based on the number of NUMA nodes in the environment. When there are too many NUMA nodes, the memory reserved on each memory segment is too small. As a result, HSAK initialization fails. Therefore, HSAK supports only the environment with a maximum of four NUMA nodes. + +3. Struct members + + | Member | Description | + | ----------------- | -------------------------------------------------------- | + | uint64_t virtAddr | Start address of the virtual memory. | + | uint64_t memLen | Length of the virtual memory, in bytes. | + | uint64_t allocLen | Available memory length in the memory segment, in bytes. | + +#### struct libstorage_dpdk_init_notify_arg + +1. Prototype + + ```c + struct libstorage_dpdk_init_notify_arg { + uint64_t baseAddr; + uint16_t memsegCount; + struct libstorage_dpdk_contig_mem *memseg; + }; + ``` + +2. Description + + Callback function parameter used to notify the service layer of initialization completion after DPDK memory initialization, indicating information about all virtual memory segments. + +3. Struct members + + | Member | Description | + | ----------------------------------------- | ------------------------------------------------------------ | + | uint64_t baseAddr | Start address of the virtual memory. | + | uint16_t memsegCount | Number of valid **memseg** array members, that is, the number of contiguous virtual memory segments. | + | struct libstorage_dpdk_contig_mem *memseg | Pointer to the memory segment array. Each array element is a contiguous virtual memory segment, and every two elements are discontiguous. | + +#### struct libstorage_dpdk_init_notify + +1. Prototype + + ```c + struct libstorage_dpdk_init_notify { + const char *name; + void (*notifyFunc)(const struct libstorage_dpdk_init_notify_arg *arg); + TAILQ_ENTRY(libstorage_dpdk_init_notify) tailq; + }; + ``` + +2. Description + + Struct used to notify the service layer of the callback function registration after the DPDK memory is initialized. + +3. Struct members + + | Member | Description | + | ------------------------------------------------------------ | ------------------------------------------------------------ | + | const char *name | Name of the service-layer module of the registered callback function. | + | void (*notifyFunc)(const struct libstorage_dpdk_init_notify_arg*arg) | Callback function parameter used to notify the service layer of initialization completion after the DPDK memory is initialized. | + | TAILQ_ENTRY(libstorage_dpdk_init_notify) tailq | Linked list that stores registered callback functions. | + +### ublock.h + +#### struct ublock_bdev_info + +1. Prototype + + ```c + struct ublock_bdev_info { + uint64_t sector_size; + uint64_t cap_size; // cap_size + uint16_t device_id; + uint16_t subsystem_device_id; // subsystem device id of nvme control + uint16_t vendor_id; + uint16_t subsystem_vendor_id; + uint16_t controller_id; + int8_t serial_number[20]; + int8_t model_number[40]; + int8_t firmware_revision[8]; + }; + ``` + +2. Description + + This data structure contains the device information of a drive. + +3. Struct members + + | Member | Description | + | ---------------------------- | ----------------------------------------------- | + | uint64_t sector_size | Sector size of a drive, for example, 512 bytes. | + | uint64_t cap_size | Total drive capacity, in bytes. | + | uint16_t device_id | Device ID. | + | uint16_t subsystem_device_id | Device ID of a subsystem. | + | uint16_t vendor_id | Main ID of the device vendor. | + | uint16_t subsystem_vendor_id | Sub-ID of the device vendor. | + | uint16_t controller_id | ID of the device controller. | + | int8_t serial_number\[20] | Device serial number. | + | int8_t model_number\[40] | Device model. | + | int8_t firmware_revision\[8] | Firmware version. | + +#### struct ublock_bdev + +1. Prototype + + ```c + struct ublock_bdev { + char pci[UBLOCK_PCI_ADDR_MAX_LEN]; + struct ublock_bdev_info info; + struct spdk_nvme_ctrlr *ctrlr; + TAILQ_ENTRY(ublock_bdev) link; + }; + ``` + +2. Description + + The data structure contains the drive information of the specified PCI address, and the structure itself is a node of the queue. + +3. Struct members + + | Member | Description | + | --------------------------------- | ------------------------------------------------------------ | + | char pci\[UBLOCK_PCI_ADDR_MAX_LEN] | PCI address. | + | struct ublock_bdev_info info | Drive information. | + | struct spdk_nvme_ctrlr *ctrlr | Data structure of the device controller. The members in this structure are not open to external systems. External services can obtain the corresponding member data through the SPDK open source interface. | + | TAILQ_ENTRY(ublock_bdev) link | Structure of the pointers before and after a queue. | + +#### struct ublock_bdev_mgr + +1. Prototype + + ```c + struct ublock_bdev_mgr { + TAILQ_HEAD(, ublock_bdev) bdevs; + }; + ``` + +2. Description + + This data structure defines the header structure of a ublock_bdev queue. + +3. Struct members + + | Member | Description | + | -------------------------------- | ----------------------- | + | TAILQ_HEAD(, ublock_bdev) bdevs; | Queue header structure. | + +#### struct **attribute**((packed)) ublock_SMART_info + +1. Prototype + + ```c + struct __attribute__((packed)) ublock_SMART_info { + uint8_t critical_warning; + uint16_t temperature; + uint8_t available_spare; + uint8_t available_spare_threshold; + uint8_t percentage_used; + uint8_t reserved[26]; + /* + + Note that the following are 128-bit values, but are + + defined as an array of 2 64-bit values. + */ + /* Data Units Read is always in 512-byte units. */ + uint64_t data_units_read[2]; + /* Data Units Written is always in 512-byte units. */ + uint64_t data_units_written[2]; + /* For NVM command set, this includes Compare commands. */ + uint64_t host_read_commands[2]; + uint64_t host_write_commands[2]; + /* Controller Busy Time is reported in minutes. */ + uint64_t controller_busy_time[2]; + uint64_t power_cycles[2]; + uint64_t power_on_hours[2]; + uint64_t unsafe_shutdowns[2]; + uint64_t media_errors[2]; + uint64_t num_error_info_log_entries[2]; + /* Controller temperature related. */ + uint32_t warning_temp_time; + uint32_t critical_temp_time; + uint16_t temp_sensor[8]; + uint8_t reserved2[296]; + }; + ``` + +2. Description + + This data structure defines the S.M.A.R.T. information of a drive. + +3. Struct members + + | Member | Description (For details, see the NVMe protocol.) | + | -------------------------------------- | ------------------------------------------------------------ | + | uint8_t critical_warning | Critical alarm of the controller status. If a bit is set to 1, the bit is valid. You can set multiple bits to be valid. Critical alarms are returned to the host through asynchronous events.
Bit 0: When this bit is set to 1, the redundant space is less than the specified threshold.
Bit 1: When this bit is set to 1, the temperature is higher or lower than a major threshold.
Bit 2: When this bit is set to 1, component reliability is reduced due to major media errors or internal errors.
Bit 3: When this bit is set to 1, the medium has been set to the read-only mode.
Bit 4: When this bit is set to 1, the volatile component of the controller fails. This parameter is valid only when the volatile component exists in the controller.
Bits 5-7: reserved. | + | uint16_t temperature | Temperature of a component. The unit is Kelvin. | + | uint8_t available_spare | Percentage of the available redundant space (0 to 100%). | + | uint8_t available_spare_threshold | Threshold of the available redundant space. An asynchronous event is reported when the available redundant space is lower than the threshold. | + | uint8_t percentage_used | Percentage of the actual service life of a component to the service life of the component expected by the manufacturer. The value **100** indicates that the actual service life of the component has reached to the expected service life, but the component can still be used. The value can be greater than 100, but any value greater than 254 will be set to 255. | + | uint8_t reserved\[26] | Reserved. | + | uint64_t data_units_read\[2] | Number of 512 bytes read by the host from the controller. The value **1** indicates that 1000 x 512 bytes are read, which exclude metadata. If the LBA size is not 512 bytes, the controller converts it into 512 bytes for calculation. The value is expressed in hexadecimal notation. | + | uint64_t data_units_written\[2] | Number of 512 bytes written by the host to the controller. The value **1** indicates that 1000 x 512 bytes are written, which exclude metadata. If the LBA size is not 512 bytes, the controller converts it into 512 bytes for calculation. The value is expressed in hexadecimal notation. | + | uint64_t host_read_commands\[2] | Number of read commands delivered to the controller. | + | uint64_t host_write_commands\[2]; | Number of write commands delivered to the controller. | + | uint64_t controller_busy_time\[2] | Busy time for the controller to process I/O commands. The process from the time the commands are delivered to the time the results are returned to the CQ is busy. The value is expressed in minutes. | + | uint64_t power_cycles\[2] | Number of machine on/off cycles. | + | uint64_t power_on_hours\[2] | Power-on duration, in hours. | + | uint64_t unsafe_shutdowns\[2] | Number of abnormal power-off times. The value is incremented by 1 when CC.SHN is not received during power-off. | + | uint64_t media_errors\[2] | Number of unrecoverable data integrity errors detected by the controller, including uncorrectable ECC errors, CRC errors, and LBA tag mismatch. | + | uint64_t num_error_info_log_entries\[2] | Number of entries in the error information log within the controller lifecycle. | + | uint32_t warning_temp_time | Accumulated time when the temperature exceeds the warning alarm threshold, in minutes. | + | uint32_t critical_temp_time | Accumulated time when the temperature exceeds the critical alarm threshold, in minutes. | + | uint16_t temp_sensor\[8] | Temperature of temperature sensors 1-8. The unit is Kelvin. | + | uint8_t reserved2\[296] | Reserved. | + +#### struct ublock_nvme_error_info + +1. Prototype + + ```c + struct ublock_nvme_error_info { + uint64_t error_count; + uint16_t sqid; + uint16_t cid; + uint16_t status; + uint16_t error_location; + uint64_t lba; + uint32_t nsid; + uint8_t vendor_specific; + uint8_t reserved[35]; + }; + ``` + +2. Description + + This data structure contains the content of a single error message in the device controller. The number of errors supported by different controllers may vary. + +3. Struct members + + | Member | Description (For details, see the NVMe protocol.) | + | ----------------------- | ------------------------------------------------------------ | + | uint64_t error_count | Error sequence number, which increases in ascending order. | + | uint16_t sqid | Submission queue identifier for the command associated with an error message. If an error cannot be associated with a specific command, this parameter should be set to **FFFFh**. | + | uint16_t cid | Command identifier associated with an error message. If an error cannot be associated with a specific command, this parameter should be set to **FFFFh**. | + | uint16_t status | Status of a completed command. | + | uint16_t error_location | Command parameter associated with an error message. | + | uint64_t lba | First LBA when an error occurs. | + | uint32_t nsid | Namespace where an error occurs. | + | uint8_t vendor_specific | Log page identifier associated with the page if other vendor-specific error messages are available. The value **00h** indicates that no additional information is available. The valid value ranges from 80h to FFh. | + | uint8_t reserved\[35] | Reserved. | + +#### struct ublock_uevent + +1. Prototype + + ```c + struct ublock_uevent { + enum ublock_nvme_uevent_action action; + int subsystem; + char traddr[UBLOCK_TRADDR_MAX_LEN + 1]; + }; + ``` + +2. Description + + This data structure contains parameters related to the uevent event. + +3. Struct members + + | Member | Description | + | -------------------------------------- | ------------------------------------------------------------ | + | enum ublock_nvme_uevent_action action | Whether the uevent event type is drive insertion or removal through enumeration. | + | int subsystem | Subsystem type of the uevent event. Currently, only **UBLOCK_NVME_UEVENT_SUBSYSTEM_UIO** is supported. If the application receives other values, no processing is required. | + | char traddr\[UBLOCK_TRADDR_MAX_LEN + 1] | PCI address character string in the *Domain:Bus:Device.Function* (**%04x:%02x:%02x.%x**) format. | + +#### struct ublock_hook + +1. Prototype + + ```c + struct ublock_hook + { + ublock_callback_func ublock_callback; + void *user_data; + }; + ``` + +2. Description + + This data structure is used to register callback functions. + +3. Struct members + + | Member | Description | + | ------------------------------------ | ------------------------------------------------------------ | + | ublock_callback_func ublock_callback | Function executed during callback. The type is bool func(void *info, void*user_data). | + | void *user_data | User parameter transferred to the callback function. | + +#### struct ublock_ctrl_iostat_info + +1. Prototype + + ```c + struct ublock_ctrl_iostat_info + { + uint64_t num_read_ops; + uint64_t num_write_ops; + uint64_t read_latency_ms; + uint64_t write_latency_ms; + uint64_t io_outstanding; + uint64_t num_poll_timeout; + uint64_t io_ticks_ms; + }; + ``` + +2. Description + + This data structure is used to obtain the I/O statistics of a controller. + +3. Struct members + + | Member | Description | + | ------------------------- | ------------------------------------------------------------ | + | uint64_t num_read_ops | Accumulated number of read I/Os of the controller. | + | uint64_t num_write_ops | Accumulated number of write I/Os of the controller. | + | uint64_t read_latency_ms | Accumulated read latency of the controller, in ms. | + | uint64_t write_latency_ms | Accumulated write latency of the controller, in ms. | + | uint64_t io_outstanding | Queue depth of the controller. | + | uint64_t num_poll_timeout | Accumulated number of polling timeouts of the controller. | + | uint64_t io_ticks_ms | Accumulated I/O processing latency of the controller, in ms. | + +## API + +### bdev_rw.h + +#### libstorage_get_nvme_ctrlr_info + +1. Prototype + + uint32_t libstorage_get_nvme_ctrlr_info(struct libstorage_nvme_ctrlr_info** ppCtrlrInfo); + +2. Description + + Obtains information about all controllers. + +3. Parameters + + | Parameter | Description | + | ----------------------------------------------- | ------------------------------------------------------------ | + | struct libstorage_nvme_ctrlr_info** ppCtrlrInfo | Output parameter, which returns all obtained controller information.
Note:
Free the memory using the free API in a timely manner. | + +4. Return value + + | Return Value | Description | + | ------------ | ------------------------------------------------------------ | + | 0 | Failed to obtain controller information or no controller information is obtained. | + | > 0 | Number of obtained controllers. | + +#### libstorage_get_mgr_info_by_esn + +1. Prototype + + ```c + int32_t libstorage_get_mgr_info_by_esn(const char *esn, struct libstorage_mgr_info *mgr_info); + ``` + +2. Description + + Obtains the management information about the NVMe drive corresponding to the ESN. + +3. Parameters + + | Parameter | Description | + | ------------------------------------ | ------------------------------------------------------------ | + | const char *esn | ESN of the target device.
Note:
An ESN is a string of a maximum of 20 characters (excluding the end character of the string), but the length may vary according to hardware vendors. For example, if the length is less than 20 characters, spaces are padded at the end of the character string.
| + | struct libstorage_mgr_info *mgr_info | Output parameter, which returns all obtained NVMe drive management information. | + +4. Return value + + | Return Value | Description | + | ------------ | ------------------------------------------------------------ | + | 0 | Succeeded in querying the NVMe drive management information corresponding to an ESN. | + | -1 | Failed to query the NVMe drive management information corresponding to an ESN. | + | -2 | No NVMe drive matching an ESN is obtained. | + +#### libstorage_get_mgr_smart_by_esn + +1. Prototype + + ```c + int32_t libstorage_get_mgr_smart_by_esn(const char *esn, uint32_t nsid, struct libstorage_smart_info *mgr_smart_info); + ``` + +2. Description + + Obtains the S.M.A.R.T. information of the NVMe drive corresponding to an ESN. + +3. Parameters + + | Parameter | Description | + | ------------------------------------ | ------------------------------------------------------------ | + | const char *esn | ESN of the target device.
Note:
An ESN is a string of a maximum of 20 characters (excluding the end character of the string), but the length may vary according to hardware vendors. For example, if the length is less than 20 characters, spaces are padded at the end of the character string.
| + | uint32_t nsid | Specified namespace. | + | struct libstorage_mgr_info *mgr_info | Output parameter, which returns all obtained S.M.A.R.T. information of NVMe drives. | + +4. Return value + + | Return Value | Description | + | ------------ | ------------------------------------------------------------ | + | 0 | Succeeded in querying the S.M.A.R.T. information of the NVMe drive corresponding to an ESN. | + | -1 | Failed to query the S.M.A.R.T. information of the NVMe drive corresponding to an ESN. | + | -2 | No NVMe drive matching an ESN is obtained. | + +#### libstorage_get_bdev_ns_info + +1. Prototype + + ```c + uint32_t libstorage_get_bdev_ns_info(const char* bdevName, struct libstorage_namespace_info** ppNsInfo); + ``` + +2. Description + + Obtains namespace information based on the device name. + +3. Parameters + + | Parameter | Description | + | ------------------------------------------- | ------------------------------------------------------------ | + | const char* bdevName | Device name. | + | struct libstorage_namespace_info** ppNsInfo | Output parameter, which returns namespace information.
Note:
Free the memory using the free API in a timely manner. | + +4. Return value + + | Return Value | Description | + | ------------ | ---------------------------- | + | 0 | The operation failed. | + | 1 | The operation is successful. | + +#### libstorage_get_ctrl_ns_info + +1. Prototype + + ```c + uint32_t libstorage_get_ctrl_ns_info(const char* ctrlName, struct libstorage_namespace_info** ppNsInfo); + ``` + +2. Description + + Obtains information about all namespaces based on the controller name. + +3. Parameters + + | Parameter | Description | + | ------------------------------------------- | ------------------------------------------------------------ | + | const char* ctrlName | Controller name. | + | struct libstorage_namespace_info** ppNsInfo | Output parameter, which returns information about all namespaces.
Note:
Free the memory using the free API in a timely manner. | + +4. Return value + + | Return Value | Description | + | ------------ | ------------------------------------------------------------ | + | 0 | Failed to obtain the namespace information or no namespace information is obtained. | + | > 0 | Number of namespaces obtained. | + +#### libstorage_create_namespace + +1. Prototype + + ```c + int32_t libstorage_create_namespace(const char* ctrlName, uint64_t ns_size, char** outputName); + ``` + +2. Description + + Creates a namespace on a specified controller (the prerequisite is that the controller supports namespace management). + + Optane drives are based on the NVMe 1.0 protocol and do not support namespace management. Therefore, this API is not supported. + + ES3000 V3 and V5 support only one namespace by default. By default, a namespace exists on the controller. To create a namespace, delete the original namespace. + +3. Parameters + + | Parameter | Description | + | -------------------- | ------------------------------------------------------------ | + | const char* ctrlName | Controller name. | + | uint64_t ns_size | Size of the namespace to be created (unit: sector_size). | + | char** outputName | Output parameter, which indicates the name of the created namespace.
Note:
Free the memory using the free API in a timely manner. | + +4. Return value + + | Return Value | Description | + | ------------ | ---------------------------------------------- | + | ≤ 0 | Failed to create the namespace. | + | > 0 | ID of the created namespace (starting from 1). | + +#### libstorage_delete_namespace + +1. Prototype + + ```c + int32_t libstorage_delete_namespace(const char* ctrlName, uint32_t ns_id); + ``` + +2. Description + + Deletes a namespace from a specified controller. Optane drives are based on the NVMe 1.0 protocol and do not support namespace management. Therefore, this API is not supported. + +3. Parameters + + | Parameter | Description | + | -------------------- | ---------------- | + | const char* ctrlName | Controller name. | + | uint32_t ns_id | Namespace ID | + +4. Return value + + | Return Value | Description | + | ------------ | ------------------------------------------------------------ | + | 0 | Deletion succeeded. | + | Other values | Deletion failed.
Note:
Before deleting a namespace, stop I/O operations. Otherwise, the namespace fails to be deleted. | + +#### libstorage_delete_all_namespace + +1. Prototype + + ```c + int32_t libstorage_delete_all_namespace(const char* ctrlName); + ``` + +2. Description + + Deletes all namespaces from a specified controller. Optane drives are based on the NVMe 1.0 protocol and do not support namespace management. Therefore, this API is not supported. + +3. Parameters + + | Parameter | Description | + | -------------------- | ---------------- | + | const char* ctrlName | Controller name. | + +4. Return value + + | Return Value | Description | + | ------------ | ------------------------------------------------------------ | + | 0 | Deletion succeeded. | + | Other values | Deletion failed.
Note:
Before deleting a namespace, stop I/O operations. Otherwise, the namespace fails to be deleted. | + +#### libstorage_nvme_create_ctrlr + +1. Prototype + + ```c + int32_t libstorage_nvme_create_ctrlr(const char *pci_addr, const char *ctrlr_name); + ``` + +2. Description + + Creates an NVMe controller based on the PCI address. + +3. Parameters + + | Parameter | Description | + | ---------------- | ---------------- | + | char *pci_addr | PCI address. | + | char *ctrlr_name | Controller name. | + +4. Return value + + | Return Value | Description | + | ------------ | ------------------- | + | < 0 | Creation failed. | + | 0 | Creation succeeded. | + +#### libstorage_nvme_delete_ctrlr + +1. Prototype + + ```c + int32_t libstorage_nvme_delete_ctrlr(const char *ctrlr_name); + ``` + +2. Description + + Destroys an NVMe controller based on the controller name. + +3. Parameters + + | Parameter | Description | + | ---------------------- | ---------------- | + | const char *ctrlr_name | Controller name. | + + This API can be called only after all delivered I/Os are returned. + +4. Return value + + | Return Value | Description | + | ------------ | ---------------------- | + | < 0 | Destruction failed. | + | 0 | Destruction succeeded. | + +#### libstorage_nvme_reload_ctrlr + +1. Prototype + + ```c + int32_t libstorage_nvme_reload_ctrlr(const char *cfgfile); + ``` + +2. Description + + Adds or deletes an NVMe controller based on the configuration file. + +3. Parameters + + | Parameter | Description | + | ------------------- | ------------------------------- | + | const char *cfgfile | Path of the configuration file. | + + Before using this API to delete a drive, ensure that all delivered I/Os have been returned. + +4. Return value + + | Return Value | Description | + | ------------ | ------------------------------------------------------------ | + | < 0 | Failed to add or delete drives based on the configuration file. (Drives may be successfully added or deleted for some controllers.) | + | 0 | Drives are successfully added or deleted based on the configuration file. | + + > Constraints + + - Currently, a maximum of 36 controllers can be configured in the configuration file. + + - The reload API creates as many controllers as possible. If a controller fails to be created, the creation of other controllers is not affected. + + - In concurrency scenarios, the final drive initialization status may be inconsistent with the input configuration file. + + - If you delete a drive that is delivering I/Os by reloading the drive, I/Os fail. + + - After the controller name (for example, **nvme0**) corresponding to the PCI address in the configuration file is modified, the modification does not take effect after this interface is called. + + - The reload function is valid only when drives are added or deleted. Other configuration items in the configuration file cannot be reloaded. + +#### libstorage_low_level_format_nvm + +1. Prototype + + ```c + int8_t libstorage_low_level_format_nvm(const char* ctrlName, uint8_t lbaf, + enum libstorage_ns_pi_type piType, + bool pil_start, bool ms_extented, uint8_t ses); + ``` + +2. Description + + Low-level formats NVMe drives. + +3. Parameters + + | Parameter | Description | + | --------------------------------- | ------------------------------------------------------------ | + | const char* ctrlName | Controller name. | + | uint8_t lbaf | LBA format to be used. | + | enum libstorage_ns_pi_type piType | Protection type to be used. | + | bool pil_start | The protection information is stored in first eight bytes (1) or last eight bytes (0) of the metadata. | + | bool ms_extented | Whether to format to the extended type. | + | uint8_t ses | Whether to perform secure erase during formatting. Currently, only the value **0** (no-secure erase) is supported. | + +4. Return value + + | Return Value | Description | + | ------------ | ------------------------------------------------- | + | < 0 | Formatting failed. | + | ≥ 0 | LBA format generated after successful formatting. | + + > Constraints + + - This low-level formatting API will clear the data and metadata of the drive namespace. Exercise caution when using this API. + + - It takes several seconds to format an ES3000 drive and several minutes to format an Intel Optane drive. Before using this API, wait until the formatting is complete. If the formatting process is forcibly stopped, the formatting fails. + + - Before formatting, stop the I/O operations on the data plane. If the drive is processing I/O requests, the formatting may fail occasionally. If the formatting is successful, the drive may discard the I/O requests that are being processed. Therefore, before formatting the drive, ensure that the I/O operations on the data plane are stopped. + + - During the formatting, the controller is reset. As a result, the initialized drive resources are unavailable. Therefore, after the formatting is complete, restart the I/O process on the data plane. + + - ES3000 V3 supports protection types 0 and 3, PI start and PI end, and mc extended. ES3000 V3 supports DIF in 512+8 format but does not support DIF in 4096+64 format. + + - ES3000 V5 supports protection types 0 and 3, PI start and PI end, mc extended, and mc pointer. ES3000 V5 supports DIF in both 512+8 and 4096+64 formats. + + - Optane drives support protection types 0 and 1, PI end, and mc extended. Optane drives support DIF in 512+8 format but does not support DIF in 4096+64 format. + + | **Drive Type** | **LBA Format** | **Drive Type** | **LBA Format** | + | ------------------ | ------------------------------------------------------------ | -------------- | ------------------------------------------------------------ | + | Intel Optane P4800 | lbaf0:512+0
lbaf1:512+8
lbaf2:512+16
lbaf3:4096+0
lbaf4:4096+8
lbaf5:4096+64
lbaf6:4096+128 | ES3000 V3, V5 | lbaf0:512+0
lbaf1:512+8
lbaf2:4096+64
lbaf3:4096+0
lbaf4:4096+8 | + +#### LIBSTORAGE_CALLBACK_FUNC + +1. Prototype + + ```c + typedef void (*LIBSTORAGE_CALLBACK_FUNC)(int32_t cb_status, int32_t sct_code, void* cb_arg); + ``` + +2. Description + + Registered HSAK I/O completion callback function. + +3. Parameters + + | Parameter | Description | + | ----------------- | ------------------------------------------------------------ | + | int32_t cb_status | I/O status code. The value **0** indicates success, a negative value indicates system error code, and a positive value indicates drive error code (for different error codes,
see [Appendixes](#Appendixes)). | + | int32_t sct_code | I/O status code type:
0: [GENERIC](#generic)
1: [COMMAND_SPECIFIC](#command_specific)
2: [MEDIA_DATA_INTERGRITY_ERROR](#media_data_intergrity_error)
7: VENDOR_SPECIFIC | + | void* cb_arg | Input parameter of the callback function. | + +4. Return value + + None. + +#### libstorage_deallocate_block + +1. Prototype + + ```c + int32_t libstorage_deallocate_block(int32_t fd, struct libstorage_dsm_range_desc *range, uint16_t range_count, LIBSTORAGE_CALLBACK_FUNC cb, void* cb_arg); + ``` + +2. Description + + Notifies NVMe drives of the blocks that can be released. + +3. Parameters + + | Parameter | Description | + | --------------------------------------- | ------------------------------------------------------------ | + | int32_t fd | Open drive file descriptor. | + | struct libstorage_dsm_range_desc *range | Description of blocks that can be released on NVMe drives.
Note:
This parameter requires **libstorage_mem_reserve** to allocate huge page memory. 4 KB alignment is required during memory allocation, that is, align is set to 4096.
The TRIM range of drives is restricted based on different drives. Exceeding the maximum TRIM range on the drives may cause data exceptions. | + | uint16_t range_count | Number of members in the array range. | + | LIBSTORAGE_CALLBACK_FUNC cb | Callback function. | + | void* cb_arg | Callback function parameter. | + +4. Return value + + | Return Value | Description | + | ------------ | ------------------------------- | + | < 0 | Failed to deliver the request. | + | 0 | Request submitted successfully. | + +#### libstorage_async_write + +1. Prototype + + ```c + int32_t libstorage_async_write(int32_t fd, void *buf, size_t nbytes, off64_t offset, void *md_buf, size_t md_len, enum libstorage_crc_and_prchk dif_flag, LIBSTORAGE_CALLBACK_FUNC cb, void* cb_arg); + ``` + +2. Description + + Delivers asynchronous I/O write requests (the write buffer is a contiguous buffer). + +3. Parameters + + | Parameter | Description | + | -------------------------------------- | ------------------------------------------------------------ | + | int32_t fd | File descriptor of the block device. | + | void *buf | Buffer for I/O write data (four-byte aligned and cannot cross the 4 KB page boundary).
Note:
The LBA in extended mode must contain the metadata memory size. | + | size_t nbytes | Size of a single write I/O, in bytes (an integer multiple of **sector_size**).
Note:
Only the data size is included. LBAs in extended mode do not include the metadata size. | + | off64_t offset | Write offset of the LBA, in bytes (an integer multiple of **sector_size**).
Note:
Only the data size is included. LBAs in extended mode do not include the metadata size. | + | void *md_buf | Metadata buffer. (Applicable only to LBAs in separated mode. Set this parameter to **NULL** for LBAs in extended mode.) | + | size_t md_len | Buffer length of metadata. (Applicable only to LBAs in separated mode. Set this parameter to **0** for LBAs in extended mode.) | + | enum libstorage_crc_and_prchk dif_flag | Whether to calculate DIF and whether to enable drive verification. | + | LIBSTORAGE_CALLBACK_FUNC cb | Registered callback function. | + | void* cb_arg | Parameters of the callback function. | + +4. Return value + + | Return Value | Description | + | ------------ | ---------------------------------------------- | + | 0 | I/O write requests are submitted successfully. | + | Other values | Failed to submit I/O write requests. | + +#### libstorage_async_read + +1. Prototype + + ```c + int32_t libstorage_async_read(int32_t fd, void *buf, size_t nbytes, off64_t offset, void *md_buf, size_t md_len, enum libstorage_crc_and_prchk dif_flag, LIBSTORAGE_CALLBACK_FUNC cb, void* cb_arg); + ``` + +2. Description + + Delivers asynchronous I/O read requests (the read buffer is a contiguous buffer). + +3. Parameters + + | Parameter | Description | + | -------------------------------------- | ------------------------------------------------------------ | + | int32_t fd | File descriptor of the block device. | + | void *buf | Buffer for I/O read data (four-byte aligned and cannot cross the 4 KB page boundary).
Note:
LBAs in extended mode must contain the metadata memory size. | + | size_t nbytes | Size of a single read I/O, in bytes (an integer multiple of **sector_size**).
Note:
Only the data size is included. LBAs in extended mode do not include the metadata size. | + | off64_t offset | Read offset of the LBA, in bytes (an integer multiple of **sector_size**).
Note:
Only the data size is included. The LBA in extended mode does not include the metadata size. | + | void *md_buf | Metadata buffer. (Applicable only to LBAs in separated mode. Set this parameter to **NULL** for LBAs in extended mode.). | + | size_t md_len | Buffer length of metadata. (Applicable only to LBAs in separated mode. Set this parameter to **0** for LBAs in extended mode.). | + | enum libstorage_crc_and_prchk dif_flag | Whether to calculate DIF and whether to enable drive verification. | + | LIBSTORAGE_CALLBACK_FUNC cb | Registered callback function. | + | void* cb_arg | Parameters of the callback function. | + +4. Return value + + | Return Value | Description | + | ------------ | --------------------------------------------- | + | 0 | I/O read requests are submitted successfully. | + | Other values | Failed to submit I/O read requests. | + +#### libstorage_async_writev + +1. Prototype + + ```c + int32_t libstorage_async_writev(int32_t fd, struct iovec *iov, int iovcnt, size_t nbytes, off64_t offset, void *md_buf, size_t md_len, enum libstorage_crc_and_prchk dif_flag, LIBSTORAGE_CALLBACK_FUNC cb, void* cb_arg); + ``` + +2. Description + + Delivers asynchronous I/O write requests (the write buffer is a discrete buffer). + +3. Parameters + + | Parameter | Description | + | -------------------------------------- | ------------------------------------------------------------ | + | int32_t fd | File descriptor of the block device. | + | struct iovec *iov | Buffer for I/O write data.
Note:
LBAs in extended mode must contain the metadata size.
The address must be 4-byte-aligned and the length cannot exceed 4 GB. | + | int iovcnt | Number of buffers for I/O write data. | + | size_t nbytes | Size of a single write I/O, in bytes (an integer multiple of **sector_size**).
Note:
Only the data size is included. LBAs in extended mode do not include the metadata size. | + | off64_t offset | Write offset of the LBA, in bytes (an integer multiple of **sector_size**).
Note:
Only the data size is included. LBAs in extended mode do not include the metadata size. | + | void *md_buf | Metadata buffer. (Applicable only to LBAs in separated mode. Set this parameter to **NULL** for LBAs in extended mode.) | + | size_t md_len | Length of the metadata buffer. (Applicable only to LBAs in separated mode. Set this parameter to **0** for LBAs in extended mode.) | + | enum libstorage_crc_and_prchk dif_flag | Whether to calculate DIF and whether to enable drive verification. | + | LIBSTORAGE_CALLBACK_FUNC cb | Registered callback function. | + | void* cb_arg | Parameters of the callback function. | + +4. Return value + + | Return Value | Description | + | ------------ | ---------------------------------------------- | + | 0 | I/O write requests are submitted successfully. | + | Other values | Failed to submit I/O write requests. | + +#### libstorage_async_readv + +1. Prototype + + ```c + int32_t libstorage_async_readv(int32_t fd, struct iovec *iov, int iovcnt, size_t nbytes, off64_t offset, void *md_buf, size_t md_len, enum libstorage_crc_and_prchk dif_flag, LIBSTORAGE_CALLBACK_FUNC cb, void* cb_arg); + ``` + +2. Description + + Delivers asynchronous I/O read requests (the read buffer is a discrete buffer). + +3. Parameters + + | Parameter | Description | + | -------------------------------------- | ------------------------------------------------------------ | + | int32_t fd | File descriptor of the block device. | + | struct iovec *iov | Buffer for I/O read data.
Note:
LBAs in extended mode must contain the metadata size.
The address must be 4-byte-aligned and the length cannot exceed 4 GB. | + | int iovcnt | Number of buffers for I/O read data. | + | size_t nbytes | Size of a single read I/O, in bytes (an integer multiple of **sector_size**).
Note:
Only the data size is included. LBAs in extended mode do not include the metadata size. | + | off64_t offset | Read offset of the LBA, in bytes (an integer multiple of **sector_size**).
Note:
Only the data size is included. LBAs in extended mode do not include the metadata size. | + | void *md_buf | Metadata buffer. (Applicable only to LBAs in separated mode. Set this parameter to **NULL** for LBAs in extended mode.) | + | size_t md_len | Length of the metadata buffer. (Applicable only to LBAs in separated mode. Set this parameter to **0** for LBAs in extended mode.) | + | enum libstorage_crc_and_prchk dif_flag | Whether to calculate DIF and whether to enable drive verification. | + | LIBSTORAGE_CALLBACK_FUNC cb | Registered callback function. | + | void* cb_arg | Parameters of the callback function. | + +4. Return value + + | Return Value | Description | + | ------------ | --------------------------------------------- | + | 0 | I/O read requests are submitted successfully. | + | Other values | Failed to submit I/O read requests. | + +#### libstorage_sync_write + +1. Prototype + + ```c + int32_t libstorage_sync_write(int fd, const void *buf, size_t nbytes, off_t offset); + ``` + +2. Description + + Delivers synchronous I/O write requests (the write buffer is a contiguous buffer). + +3. Parameters + + | Parameter | Description | + | -------------- | ------------------------------------------------------------ | + | int32_t fd | File descriptor of the block device. | + | void *buf | Buffer for I/O write data (four-byte aligned and cannot cross the 4 KB page boundary).
Note:
LBAs in extended mode must contain the metadata memory size. | + | size_t nbytes | Size of a single write I/O, in bytes (an integer multiple of **sector_size**).
Note:
Only the data size is included. LBAs in extended mode do not include the metadata size. | + | off64_t offset | Write offset of the LBA, in bytes. (an integer multiple of **sector_size**).
Note:
Only the data size is included. LBAs in extended mode do not include the metadata size. | + +4. Return value + + | Return Value | Description | + | ------------ | ---------------------------------------------- | + | 0 | I/O write requests are submitted successfully. | + | Other values | Failed to submit I/O write requests. | + +#### libstorage_sync_read + +1. Prototype + + ```c + int32_t libstorage_sync_read(int fd, const void *buf, size_t nbytes, off_t offset); + ``` + +2. Description + + Delivers synchronous I/O read requests (the read buffer is a contiguous buffer). + +3. Parameters + + | Parameter | Description | + | -------------- | ------------------------------------------------------------ | + | int32_t fd | File descriptor of the block device. | + | void *buf | Buffer for I/O read data (four-byte aligned and cannot cross the 4 KB page boundary).
Note:
LBAs in extended mode must contain the metadata memory size. | + | size_t nbytes | Size of a single read I/O, in bytes (an integer multiple of **sector_size**).
Note:
Only the data size is included. LBAs in extended mode do not include the metadata size. | + | off64_t offset | Read offset of the LBA, in bytes (an integer multiple of **sector_size**).
Note:
Only the data size is included. LBAs in extended mode do not include the metadata size. | + +4. Return value + + | Return Value | Description | + | ------------ | --------------------------------------------- | + | 0 | I/O read requests are submitted successfully. | + | Other values | Failed to submit I/O read requests. | + +#### libstorage_open + +1. Prototype + + ```c + int32_t libstorage_open(const char* devfullname); + ``` + +2. Description + + Opens a block device. + +3. Parameters + + | Parameter | Description | + | ----------------------- | ---------------------------------------- | + | const char* devfullname | Block device name (format: **nvme0n1**). | + +4. Return value + + | Return Value | Description | + | ------------ | ------------------------------------------------------------ | + | -1 | Opening failed. For example, the device name is incorrect, or the number of opened FDs is greater than the number of available channels of the NVMe drive. | + | > 0 | File descriptor of the block device. | + + After the MultiQ function in **nvme.conf.in** is enabled, different FDs are returned if a thread opens the same device for multiple times. Otherwise, the same FD is returned. This attribute applies only to the NVMe device. + +#### libstorage_close + +1. Prototype + + ```c + int32_t libstorage_close(int32_t fd); + ``` + +2. Description + + Closes a block device. + +3. Parameters + + | Parameter | Description | + | ---------- | ------------------------------------------ | + | int32_t fd | File descriptor of an opened block device. | + +4. Return value + + | Return Value | Description | + | ------------ | ----------------------------------------------- | + | -1 | Invalid file descriptor. | + | -16 | The file descriptor is busy. Retry is required. | + | 0 | Close succeeded. | + +#### libstorage_mem_reserve + +1. Prototype + + ```c + void* libstorage_mem_reserve(size_t size, size_t align); + ``` + +2. Description + + Allocates memory space from the huge page memory reserved by the DPDK. + +3. Parameters + + | Parameter | Description | + | ------------ | ----------------------------------- | + | size_t size | Size of the memory to be allocated. | + | size_t align | Aligns allocated memory space. | + +4. Return value + + | Return Value | Description | + | ------------ | -------------------------------------- | + | NULL | Allocation failed. | + | Other values | Address of the allocated memory space. | + +#### libstorage_mem_free + +1. Prototype + + ```c + void libstorage_mem_free(void* ptr); + ``` + +2. Description + + Frees the memory space pointed to by **ptr**. + +3. Parameters + + | Parameter | Description | + | --------- | ---------------------------------------- | + | void* ptr | Address of the memory space to be freed. | + +4. Return value + + None. + +#### libstorage_alloc_io_buf + +1. Prototype + + ```c + void* libstorage_alloc_io_buf(size_t nbytes); + ``` + +2. Description + + Allocates memory from buf_small_pool or buf_large_pool of the SPDK. + +3. Parameters + + | Parameter | Description | + | ------------- | ----------------------------------- | + | size_t nbytes | Size of the buffer to be allocated. | + +4. Return value + + | Return Value | Description | + | ------------ | -------------------------------------- | + | Other values | Start address of the allocated buffer. | + +#### libstorage_free_io_buf + +1. Prototype + + ```c + int32_t libstorage_free_io_buf(void *buf, size_t nbytes); + ``` + +2. Description + + Frees the allocated memory to buf_small_pool or buf_large_pool of the SPDK. + +3. Parameters + + | Parameter | Description | + | ------------- | ---------------------------------------- | + | void *buf | Start address of the buffer to be freed. | + | size_t nbytes | Size of the buffer to be freed. | + +4. Return value + + | Return Value | Description | + | ------------ | ------------------ | + | -1 | Freeing failed. | + | 0 | Freeing succeeded. | + +#### libstorage_init_module + +1. Prototype + + ```c + int32_t libstorage_init_module(const char* cfgfile); + ``` + +2. Description + + Initializes the HSAK module. + +3. Parameters + + | Parameter | Description | + | ------------------- | ------------------------------------ | + | const char* cfgfile | Name of the HSAK configuration file. | + +4. Return value + + | Return Value | Description | + | ------------ | ------------------------- | + | Other values | Initialization failed. | + | 0 | Initialization succeeded. | + +#### libstorage_exit_module + +1. Prototype + + ```c + int32_t libstorage_exit_module(void); + ``` + +2. Description + + Exits the HSAK module. + +3. Parameters + + None. + +4. Return value + + | Return Value | Description | + | ------------ | --------------------------------- | + | Other values | Failed to exit the cleanup. | + | 0 | Succeeded in exiting the cleanup. | + +#### LIBSTORAGE_REGISTER_DPDK_INIT_NOTIFY + +1. Prototype + + ```c + LIBSTORAGE_REGISTER_DPDK_INIT_NOTIFY(_name, _notify) + ``` + +2. Description + + Service layer registration function, which is used to register the callback function when the DPDK initialization is complete. + +3. Parameters + + | Parameter | Description | + | --------- | ------------------------------------------------------------ | + | _name | Name of a module at the service layer. | + | _notify | Prototype of the callback function registered at the service layer: **void (*notifyFunc)(const struct libstorage_dpdk_init_notify_arg *arg);** | + +4. Return value + + None + +### ublock.h + +#### init_ublock + +1. Prototype + + ```c + int init_ublock(const char *name, enum ublock_rpc_server_status flg); + ``` + +2. Description + + Initializes the Ublock module. This API must be called before other Ublock APIs. If the flag is set to **UBLOCK_RPC_SERVER_ENABLE**, that is, Ublock functions as the RPC server, the same process can be initialized only once. + + When Ublock is started as the RPC server, the monitor thread of a server is started at the same time. When the monitor thread detects that the RPC server thread is abnormal (for example, thread suspended), the monitor thread calls the exit function to trigger the process to exit. + + In this case, the product script is used to start the process again. + +3. Parameters + + | Parameter | Description | + | ------------------------------------ | ------------------------------------------------------------ | + | const char *name | Module name. The default value is **ublock**. You are advised to set this parameter to **NULL**. | + | enum ublock_rpc_server_status
flg | Whether to enable RPC. The value can be **UBLOCK_RPC_SERVER_DISABLE** or **UBLOCK_RPC_SERVER_ENABLE**.
If RPC is disabled and the drive is occupied by service processes, the Ublock module cannot obtain the drive information. | + +4. Return value + + | Return Value | Description | + | ------------- | ------------------------------------------------------------ | + | 0 | Initialization succeeded. | + | -1 | Initialization failed. Possible cause: The Ublock module has been initialized. | + | Process exits | Ublock considers that the following exceptions cannot be rectified and directly calls the exit API to exit the process:
- The RPC service needs to be created, but it fails to be created onsite.
- Failed to create a hot swap monitoring thread. | + +#### ublock_init + +1. Prototype + + ```c + #define ublock_init(name) init_ublock(name, UBLOCK_RPC_SERVER_ENABLE) + ``` + +2. Description + + It is the macro definition of the init_ublock API. It can be regarded as initializing Ublock into the required RPC service. + +3. Parameters + + | Parameter | Description | + | --------- | ------------------------------------------------------------ | + | name | Module name. The default value is **ublock**. You are advised to set this parameter to **NULL**. | + +4. Return value + + | Return Value | Description | + | ------------- | ------------------------------------------------------------ | + | 0 | Initialization succeeded. | + | -1 | Initialization failed. Possible cause: The Ublock RPC server module has been initialized. | + | Process exits | Ublock considers that the following exceptions cannot be rectified and directly calls the exit API to exit the process:
- The RPC service needs to be created, but it fails to be created onsite.
- Failed to create a hot swap monitoring thread. | + +#### ublock_init_norpc + +1. Prototype + + ```c + #define ublock_init_norpc(name) init_ublock(name, UBLOCK_RPC_SERVER_DISABLE) + ``` + +2. Description + + It is the macro definition of the init_ublock API and can be considered as initializing Ublock into a non-RPC service. + +3. Parameters + + | Parameter | Description | + | --------- | ------------------------------------------------------------ | + | name | Module name. The default value is **ublock**. You are advised to set this parameter to **NULL**. | + +4. Return value + + | Return Value | Description | + | ------------- | ------------------------------------------------------------ | + | 0 | Initialization succeeded. | + | -1 | Initialization failed. Possible cause: The Ublock client module has been initialized. | + | Process exits | Ublock considers that the following exceptions cannot be rectified and directly calls the exit API to exit the process:
- The RPC service needs to be created, but it fails to be created onsite.
- Failed to create a hot swap monitoring thread. | + +#### ublock_fini + +1. Prototype + + ```c + void ublock_fini(void); + ``` + +2. Description + + Destroys the Ublock module and internally created resources. This API must be used together with the Ublock initialization API. + +3. Parameters + + None. + +4. Return value + + None. + +#### ublock_get_bdevs + +1. Prototype + + ```c + int ublock_get_bdevs(struct ublock_bdev_mgr* bdev_list); + ``` + +2. Description + + Obtains the device list (all NVMe devices in the environment, including kernel-mode and user-mode drivers). The obtained NVMe device list contains only PCI addresses and does not contain specific device information. To obtain specific device information, call ublock_get_bdev. + +3. Parameters + + | Parameter | Description | + | --------------------------------- | ------------------------------------------------------------ | + | struct ublock_bdev_mgr* bdev_list | Output parameter, which returns the device queue. The **bdev_list** pointer must be allocated externally. | + +4. Return value + + | Return Value | Description | + | ------------ | ------------------------------------------ | + | 0 | The device queue is obtained successfully. | + | -2 | No NVMe device exists in the environment. | + | Other values | Failed to obtain the device list. | + +#### ublock_free_bdevs + +1. Prototype + + ```c + void ublock_free_bdevs(struct ublock_bdev_mgr* bdev_list); + ``` + +2. Description + + Releases a device list. + +3. Parameters + + | Parameter | Description | + | --------------------------------- | ------------------------------------------------------------ | + | struct ublock_bdev_mgr* bdev_list | Head pointer of the device queue. After the device queue is cleared, the **bdev_list** pointer is not released. | + +4. Return value + + None. + +#### ublock_get_bdev + +1. Prototype + + ```c + int ublock_get_bdev(const char *pci, struct ublock_bdev *bdev); + ``` + +2. Description + + Obtains information about a specific device. In the device information, the serial number, model, and firmware version of the NVMe device are saved as character arrays instead of character strings. (The return format varies depending on the drive controller, and the arrays may not end with 0.) + + After this API is called, the corresponding device is occupied by Ublock. Therefore, call ublock_free_bdev to free resources immediately after the required service operation is complete. + +3. Parameters + + | Parameter | Description | + | ------------------------ | ------------------------------------------------------------ | + | const char *pci | PCI address of the device whose information needs to be obtained. | + | struct ublock_bdev *bdev | Output parameter, which returns the device information. The **bdev** pointer must be allocated externally. | + +4. Return value + + | Return Value | Description | + | ------------ | ------------------------------------------------------------ | + | 0 | The device information is obtained successfully. | + | -1 | Failed to obtain device information due to incorrect parameters. | + | -11(EAGAIN) | Failed to obtain device information due to the RPC query failure. A retry is required (3s sleep is recommended). | + +#### ublock_get_bdev_by_esn + +1. Prototype + + ```c + int ublock_get_bdev_by_esn(const char *esn, struct ublock_bdev *bdev); + ``` + +2. Description + + Obtains information about the device corresponding to an ESN. In the device information, the serial number, model, and firmware version of the NVMe device are saved as character arrays instead of character strings. (The return format varies depending on the drive controller, and the arrays may not end with 0.) + + After this API is called, the corresponding device is occupied by Ublock. Therefore, call ublock_free_bdev to free resources immediately after the required service operation is complete. + +3. Parameters + + | Parameter | Description | + | ------------------------ | ------------------------------------------------------------ | + | const char *esn | ESN of the device whose information is to be obtained.
Note:
An ESN is a string of a maximum of 20 characters (excluding the end character of the string), but the length may vary according to hardware vendors. For example, if the length is less than 20 characters, spaces are padded at the end of the character string. | + | struct ublock_bdev *bdev | Output parameter, which returns the device information. The **bdev** pointer must be allocated externally. | + +4. Return value + + | Return Value | Description | + | ------------ | ------------------------------------------------------------ | + | 0 | The device information is obtained successfully. | + | -1 | Failed to obtain device information due to incorrect parameters. | + | -11(EAGAIN) | Failed to obtain device information due to the RPC query failure. A retry is required (3s sleep is recommended). | + +#### ublock_free_bdev + +1. Prototype + + ```c + void ublock_free_bdev(struct ublock_bdev *bdev); + ``` + +2. Description + + Frees device resources. + +3. Parameters + + | Parameter | Description | + | ------------------------ | ------------------------------------------------------------ | + | struct ublock_bdev *bdev | Pointer to the device information. After the data in the pointer is cleared, the **bdev** pointer is not freed. | + +4. Return value + + None. + +#### TAILQ_FOREACH_SAFE + +1. Prototype + + ```c + #define TAILQ_FOREACH_SAFE(var, head, field, tvar) + for ((var) = TAILQ_FIRST((head)); + (var) && ((tvar) = TAILQ_NEXT((var), field), 1); + (var) = (tvar)) + ``` + +2. Description + + Provides a macro definition for each member of the secure access queue. + +3. Parameters + + | Parameter | Description | + | --------- | ------------------------------------------------------------ | + | var | Queue node member on which you are performing operations. | + | head | Queue head pointer. Generally, it refers to the object address defined by **TAILQ_HEAD(xx, xx) obj**. | + | field | Name of the struct used to store the pointers before and after the queue in the queue node. Generally, it is the name defined by **TAILQ_ENTRY (xx) name**. | + | tvar | Next queue node member. | + +4. Return value + + None. + +#### ublock_get_SMART_info + +1. Prototype + + ```c + int ublock_get_SMART_info(const char *pci, uint32_t nsid, struct ublock_SMART_info *smart_info); + ``` + +2. Description + + Obtains the S.M.A.R.T. information of a specified device. + +3. Parameters + + | Parameter | Description | + | ------------------------------------ | ------------------------------------------------------------ | + | const char *pci | Device PCI address. | + | uint32_t nsid | Specified namespace. | + | struct ublock_SMART_info *smart_info | Output parameter, which returns the S.M.A.R.T. information of the device. | + +4. Return value + + | Return Value | Description | + | ------------ | ------------------------------------------------------------ | + | 0 | The S.M.A.R.T. information is obtained successfully. | + | -1 | Failed to obtain S.M.A.R.T. information due to incorrect parameters. | + | -11(EAGAIN) | Failed to obtain S.M.A.R.T. information due to the RPC query failure. A retry is required (3s sleep is recommended). | + +#### ublock_get_SMART_info_by_esn + +1. Prototype + + ```c + int ublock_get_SMART_info_by_esn(const char *esn, uint32_t nsid, struct ublock_SMART_info *smart_info); + ``` + +2. Description + + Obtains the S.M.A.R.T. information of the device corresponding to an ESN. + +3. Parameters + + | Parameter | Description | + | --------------------------------------- | ------------------------------------------------------------ | + | const char *esn | Device ESN.
Note:
An ESN is a string of a maximum of 20 characters (excluding the end character of the string), but the length may vary according to hardware vendors. For example, if the length is less than 20 characters, spaces are padded at the end of the character string. | + | uint32_t nsid | Specified namespace. | + | struct ublock_SMART_info
*smart_info | Output parameter, which returns the S.M.A.R.T. information of the device. | + +4. Return value + + | Return Value | Description | + | ------------ | ------------------------------------------------------------ | + | 0 | The S.M.A.R.T. information is obtained successfully. | + | -1 | Failed to obtain SMART information due to incorrect parameters. | + | -11(EAGAIN) | Failed to obtain S.M.A.R.T. information due to the RPC query failure. A retry is required (3s sleep is recommended). | + +#### ublock_get_error_log_info + +1. Prototype + + ```c + int ublock_get_error_log_info(const char *pci, uint32_t err_entries, struct ublock_nvme_error_info *errlog_info); + ``` + +2. Description + + Obtains the error log information of a specified device. + +3. Parameters + + | Parameter | Description | + | ------------------------------------------ | ------------------------------------------------------------ | + | const char *pci | Device PCI address. | + | uint32_t err_entries | Number of error logs to be obtained. A maximum of 256 error logs can be obtained. | + | struct ublock_nvme_error_info *errlog_info | Output parameter, which returns the error log information of the device. For the **errlog_info** pointer, the caller needs to apply for space and ensure that the obtained space is greater than or equal to err_entries x size of (struct ublock_nvme_error_info). | + +4. Return value + + | Return Value | Description | + | ------------------------------------------------------------ | ------------------------------------------------------------ | + | Number of obtained error logs. The value is greater than or equal to 0. | Error logs are obtained successfully. | + | -1 | Failed to obtain error logs due to incorrect parameters. | + | -11(EAGAIN) | Failed to obtain error logs due to the RPC query failure. A retry is required (3s sleep is recommended). | + +#### ublock_get_log_page + +1. Prototype + + ```c + int ublock_get_log_page(const char *pci, uint8_t log_page, uint32_t nsid, void *payload, uint32_t payload_size); + ``` + +2. Description + + Obtains information about a specified device and log page. + +3. Parameters + + | Parameter | Description | + | --------------------- | ------------------------------------------------------------ | + | const char *pci | Device PCI address. | + | uint8_t log_page | ID of the log page to be obtained. For example, **0xC0** and **0xCA** indicate the customized S.M.A.R.T. information of ES3000 V5 drives. | + | uint32_t nsid | Namespace ID. Some log pages support obtaining by namespace while some do not. If obtaining by namespace is not supported, the caller must transfer **0XFFFFFFFF**. | + | void *payload | Output parameter, which stores log page information. The caller is responsible for allocating memory. | + | uint32_t payload_size | Size of the applied payload, which cannot be greater than 4096 bytes. | + +4. Return value + + | Return Value | Description | + | ------------ | ---------------------------------------------------- | + | 0 | The log page is obtained successfully. | + | -1 | Failed to obtain error logs due to parameter errors. | + +#### ublock_info_get_pci_addr + +1. Prototype + + ```c + char *ublock_info_get_pci_addr(const void *info); + ``` + +2. Description + + Obtains the PCI address of the hot swap device. + + The memory occupied by info and the memory occupied by the returned PCI address do not need to be freed by the service process. + +3. Parameters + + | Parameter | Description | + | ---------------- | ------------------------------------------------------------ | + | const void *info | Hot swap event information transferred by the hot swap monitoring thread to the callback function. | + +4. Return value + + | Return Value | Description | + | ------------ | --------------------------------- | + | NULL | Failed to obtain the information. | + | Other values | Obtained PCI address. | + +#### ublock_info_get_action + +1. Prototype + + ```c + enum ublock_nvme_uevent_action ublock_info_get_action(const void *info); + ``` + +2. Description + + Obtains the type of the hot swap event. + + The memory occupied by info does not need to be freed by service process. + +3. Parameters + + | Parameter | Description | + | ---------------- | ------------------------------------------------------------ | + | const void *info | Hot swap event information transferred by the hot swap monitoring thread to the callback function. | + +4. Return value + + | Return Value | Description | + | -------------------------- | ------------------------------------------------------------ | + | Type of the hot swap event | Type of the event that triggers the callback function. For details, see the definition in **5.1.2.6 enum ublock_nvme_uevent_action**. | + +#### ublock_get_ctrl_iostat + +1. Prototype + + ```c + int ublock_get_ctrl_iostat(const char* pci, struct ublock_ctrl_iostat_info *ctrl_iostat); + ``` + +2. Description + + Obtains the I/O statistics of a controller. + +3. Parameters + + | Parameter | Description | + | ------------------------------------------- | ------------------------------------------------------------ | + | const char* pci | PCI address of the controller whose I/O statistics are to be obtained. | + | struct ublock_ctrl_iostat_info *ctrl_iostat | Output parameter, which returns I/O statistics. The **ctrl_iostat** pointer must be allocated externally. | + +4. Return value + + | Return Value | Description | + | ------------ | ------------------------------------------------------------ | + | 0 | Succeeded in obtaining I/O statistics. | + | -1 | Failed to obtain I/O statistics due to invalid parameters or RPC errors. | + | -2 | Failed to obtain I/O statistics because the NVMe drive is not taken over by the I/O process. | + | -3 | Failed to obtain I/O statistics because the I/O statistics function is disabled. | + +#### ublock_nvme_admin_passthru + +1. Prototype + + ```c + int32_t ublock_nvme_admin_passthru(const char *pci, void *cmd, void *buf, size_t nbytes); + ``` + +2. Description + + Transparently transmits the **nvme admin** command to the NVMe device. Currently, only the **nvme admin** command for obtaining the identify parameter is supported. + +3. Parameters + + | Parameter | Description | + | --------------- | ------------------------------------------------------------ | + | const char *pci | PCI address of the destination controller of the **nvme admin** command. | + | void *cmd | Pointer to the **nvme admin** command struct. The struct size is 64 bytes. For details, see the NVMe specifications. Currently, only the command for obtaining the identify parameter is supported. | + | void *buf | Saves the output of the **nvme admin** command. The space is allocated by users and the size is expressed in nbytes. | + | size_t nbytes | Size of the user buffer. The buffer for the identify parameter is 4096 bytes, and that for the command to obtain the identify parameter is 4096 nbytes. | + +4. Return value + + | Return Value | Description | + | ------------ | ------------------------------------------ | + | 0 | The user command is executed successfully. | + | -1 | Failed to execute the user command. | + +# Appendixes + +## GENERIC + +Generic Error Code Reference + +| sc | value | +| ------------------------------------------ | ----- | +| NVME_SC_SUCCESS | 0x00 | +| NVME_SC_INVALID_OPCODE | 0x01 | +| NVME_SC_INVALID_FIELD | 0x02 | +| NVME_SC_COMMAND_ID_CONFLICT | 0x03 | +| NVME_SC_DATA_TRANSFER_ERROR | 0x04 | +| NVME_SC_ABORTED_POWER_LOSS | 0x05 | +| NVME_SC_INTERNAL_DEVICE_ERROR | 0x06 | +| NVME_SC_ABORTED_BY_REQUEST | 0x07 | +| NVME_SC_ABORTED_SQ_DELETION | 0x08 | +| NVME_SC_ABORTED_FAILED_FUSED | 0x09 | +| NVME_SC_ABORTED_MISSING_FUSED | 0x0a | +| NVME_SC_INVALID_NAMESPACE_OR_FORMAT | 0x0b | +| NVME_SC_COMMAND_SEQUENCE_ERROR | 0x0c | +| NVME_SC_INVALID_SGL_SEG_DESCRIPTOR | 0x0d | +| NVME_SC_INVALID_NUM_SGL_DESCIRPTORS | 0x0e | +| NVME_SC_DATA_SGL_LENGTH_INVALID | 0x0f | +| NVME_SC_METADATA_SGL_LENGTH_INVALID | 0x10 | +| NVME_SC_SGL_DESCRIPTOR_TYPE_INVALID | 0x11 | +| NVME_SC_INVALID_CONTROLLER_MEM_BUF | 0x12 | +| NVME_SC_INVALID_PRP_OFFSET | 0x13 | +| NVME_SC_ATOMIC_WRITE_UNIT_EXCEEDED | 0x14 | +| NVME_SC_OPERATION_DENIED | 0x15 | +| NVME_SC_INVALID_SGL_OFFSET | 0x16 | +| NVME_SC_INVALID_SGL_SUBTYPE | 0x17 | +| NVME_SC_HOSTID_INCONSISTENT_FORMAT | 0x18 | +| NVME_SC_KEEP_ALIVE_EXPIRED | 0x19 | +| NVME_SC_KEEP_ALIVE_INVALID | 0x1a | +| NVME_SC_ABORTED_PREEMPT | 0x1b | +| NVME_SC_SANITIZE_FAILED | 0x1c | +| NVME_SC_SANITIZE_IN_PROGRESS | 0x1d | +| NVME_SC_SGL_DATA_BLOCK_GRANULARITY_INVALID | 0x1e | +| NVME_SC_COMMAND_INVALID_IN_CMB | 0x1f | +| NVME_SC_LBA_OUT_OF_RANGE | 0x80 | +| NVME_SC_CAPACITY_EXCEEDED | 0x81 | +| NVME_SC_NAMESPACE_NOT_READY | 0x82 | +| NVME_SC_RESERVATION_CONFLICT | 0x83 | +| NVME_SC_FORMAT_IN_PROGRESS | 0x84 | + +## COMMAND_SPECIFIC + +Error Code Reference for Specific Commands + +| sc | value | +| ------------------------------------------ | ----- | +| NVME_SC_COMPLETION_QUEUE_INVALID | 0x00 | +| NVME_SC_INVALID_QUEUE_IDENTIFIER | 0x01 | +| NVME_SC_MAXIMUM_QUEUE_SIZE_EXCEEDED | 0x02 | +| NVME_SC_ABORT_COMMAND_LIMIT_EXCEEDED | 0x03 | +| NVME_SC_ASYNC_EVENT_REQUEST_LIMIT_EXCEEDED | 0x05 | +| NVME_SC_INVALID_FIRMWARE_SLOT | 0x06 | +| NVME_SC_INVALID_FIRMWARE_IMAGE | 0x07 | +| NVME_SC_INVALID_INTERRUPT_VECTOR | 0x08 | +| NVME_SC_INVALID_LOG_PAGE | 0x09 | +| NVME_SC_INVALID_FORMAT | 0x0a | +| NVME_SC_FIRMWARE_REQ_CONVENTIONAL_RESET | 0x0b | +| NVME_SC_INVALID_QUEUE_DELETION | 0x0c | +| NVME_SC_FEATURE_ID_NOT_SAVEABLE | 0x0d | +| NVME_SC_FEATURE_NOT_CHANGEABLE | 0x0e | +| NVME_SC_FEATURE_NOT_NAMESPACE_SPECIFIC | 0x0f | +| NVME_SC_FIRMWARE_REQ_NVM_RESET | 0x10 | +| NVME_SC_FIRMWARE_REQ_RESET | 0x11 | +| NVME_SC_FIRMWARE_REQ_MAX_TIME_VIOLATION | 0x12 | +| NVME_SC_FIRMWARE_ACTIVATION_PROHIBITED | 0x13 | +| NVME_SC_OVERLAPPING_RANGE | 0x14 | +| NVME_SC_NAMESPACE_INSUFFICIENT_CAPACITY | 0x15 | +| NVME_SC_NAMESPACE_ID_UNAVAILABLE | 0x16 | +| NVME_SC_NAMESPACE_ALREADY_ATTACHED | 0x18 | +| NVME_SC_NAMESPACE_IS_PRIVATE | 0x19 | +| NVME_SC_NAMESPACE_NOT_ATTACHED | 0x1a | +| NVME_SC_THINPROVISIONING_NOT_SUPPORTED | 0x1b | +| NVME_SC_CONTROLLER_LIST_INVALID | 0x1c | +| NVME_SC_DEVICE_SELF_TEST_IN_PROGRESS | 0x1d | +| NVME_SC_BOOT_PARTITION_WRITE_PROHIBITED | 0x1e | +| NVME_SC_INVALID_CTRLR_ID | 0x1f | +| NVME_SC_INVALID_SECONDARY_CTRLR_STATE | 0x20 | +| NVME_SC_INVALID_NUM_CTRLR_RESOURCES | 0x21 | +| NVME_SC_INVALID_RESOURCE_ID | 0x22 | +| NVME_SC_CONFLICTING_ATTRIBUTES | 0x80 | +| NVME_SC_INVALID_PROTECTION_INFO | 0x81 | +| NVME_SC_ATTEMPTED_WRITE_TO_RO_PAGE | 0x82 | + +## MEDIA_DATA_INTERGRITY_ERROR + +Error Code Reference for Medium Exceptions + +| sc | value | +| -------------------------------------- | ----- | +| NVME_SC_WRITE_FAULTS | 0x80 | +| NVME_SC_UNRECOVERED_READ_ERROR | 0x81 | +| NVME_SC_GUARD_CHECK_ERROR | 0x82 | +| NVME_SC_APPLICATION_TAG_CHECK_ERROR | 0x83 | +| NVME_SC_REFERENCE_TAG_CHECK_ERROR | 0x84 | +| NVME_SC_COMPARE_FAILURE | 0x85 | +| NVME_SC_ACCESS_DENIED | 0x86 | +| NVME_SC_DEALLOCATED_OR_UNWRITTEN_BLOCK | 0x87 | diff --git a/docs/en/server/memory_storage/hsak/hsak_tool_usage.md b/docs/en/server/memory_storage/hsak/hsak_tool_usage.md new file mode 100644 index 0000000000000000000000000000000000000000..342c01a55218a6290565e29f403e1249bb3120f6 --- /dev/null +++ b/docs/en/server/memory_storage/hsak/hsak_tool_usage.md @@ -0,0 +1,123 @@ +# Command-Line Interface + +## Command for Querying Drive Information + +### Format + +```shell +libstorage-list [] [] +``` + +### Parameters + +- *commands*: Only **help** is available. **libstorage-list help** is used to display the help information. + +- *device*: specifies the PCI address. The format is **0000:09:00.0**. Multiple PCI addresses are allowed and separated by spaces. If no specific PCI address is set, the command line lists all enumerated device information. + +### Precautions + +- The fault injection function applies only to development, debugging, and test scenarios. Do not use this function on live networks. Otherwise, service and security risks may occur. + +- Before running this command, ensure that the management component (Ublock) server has been started, and the user-mode I/O component (UIO) has not been started or has been correctly started. + +- Drives that are not occupied by the Ublock or UIO component will be occupied during the command execution. If the Ublock or UIO component attempts to obtain the drive control permission, a storage device access conflict may occur. As a result, the command execution fails. + +## Command for Switching Drivers for Drives + +### Format + +```shell +libstorage-shutdown reset [ ...] +``` + +### Parameters + +- **reset**: switches the UIO driver to the kernel-mode driver for a specific drive. + +- *device*: specifies the PCI address, for example, **0000:09:00.0**. Multiple PCI addresses are allowed and separated by spaces. + +### Precautions + +- The **libstorage-shutdown reset** command is used to switch a drive from the user-mode UIO driver to the kernel-mode NVMe driver. + +- Before running this command, ensure that the Ublock server has been started, and the UIO component has not been started or has been correctly started. + +- The **libstoage-shutdown reset** command is risky. Before switching to the NVMe driver, ensure that the user-mode instance has stopped delivering I/Os to the NVMe device, all FDs on the NVMe device have been disabled, and the instance that accesses the NVMe device has exited. + +## Command for Obtaining I/O Statistics + +### Format + +```shell +libstorage-iostat [-t ] [-i ] [-d ] +``` + +### Parameters + +- -**t**: interval, in seconds. The value ranges from 1 to 3600. This parameter is of the int type. If the input parameter value exceeds the upper limit of the int type, the value is truncated to a negative or positive number. + +- -**i**: number of collection times. The minimum value is **1** and the maximum value is *MAX_INT*. If this parameter is not set, information is collected at an interval by default. This parameter is of the int type. If the input parameter value exceeds the upper limit of the int type, the value is truncated to a negative or positive number. + +- -**d**: name of a block device (for example, **nvme0n1**, which depends on the controller name configured in **/etc/spdk/nvme.conf.in**). You can use this parameter to collect performance data of one or more specified devices. If this parameter is not set, performance data of all detected devices is collected. + +### Precautions + +- The I/O statistics configuration is enabled. + +- The process has delivered I/O operations to the drive whose performance information needs to be queried through the UIO component. + +- If no device in the current environment is occupied by service processes to deliver I/Os, the command exits after the message "You cannot get iostat info for nvme device no deliver io" is displayed. + +- When multiple queues are enabled on a drive, the I/O statistics tool summarizes the performance data of multiple queues on the drive and outputs the data in a unified manner. + +- The I/O statistics tool supports data records of a maximum of 8192 drive queues. + +- The I/O statistics are as follows: + + | Device | r/s | w/s | rKB/s | wKB/s | avgrq-sz | avgqu-sz | r_await | w_await | await | svctm | util% | poll-n | + | ----------- | ------------------------------ | ------------------------------- | ----------------------------------- | ------------------------------------ | -------------------------------------- | -------------------------- | --------------------- | ---------------------- | ------------------------------- | --------------------------------------- | ------------------ | -------------------------- | + | Device name | Number of read I/Os per second | Number of write I/Os per second | Number of read I/O bytes per second | Number of write I/O bytes per second | Average size of delivered I/Os (bytes) | I/O depth of a drive queue | I/O read latency (μs) | I/O write latency (μs) | Average read/write latency (μs) | Processing latency of a single I/O (μs) | Device utilization | Number of polling timeouts | + +## Commands for Drive Read/Write Operations + +### Format + +```shell +libstorage-rw [OPTIONS...] +``` + +### Parameters + +1. **COMMAND** parameters + + - **read**: reads a specified logical block from the device to the data buffer (standard output by default). + + - **write**: writes data in a data buffer (standard input by default) to a specified logical block of the NVMe device. + + - **help**: displays the help information about the command line. + +2. **device**: specifies the PCI address, for example, **0000:09:00.0**. + +3. **OPTIONS** parameters + + - **--start-block, -s**: indicates the 64-bit start address of the logical block to be read or written. The default value is **0**. + + - **--block-count, -c**: indicates the number of the logical blocks to be read or written (counted from 0). + + - **--data-size, -z**: indicates the number of bytes of the data to be read or written. + + - **--namespace-id, -n**: indicates the namespace ID of the device. The default value is **1**. + + - **--data, -d**: indicates the data file used for read and write operations (The read data is saved during read operations and the written data is provided during write operations.) + + - **--limited-retry, -l**: indicates that the device controller restarts for a limited number of times to complete device read and write operations. + + - **--force-unit-access, -f**: ensures that read and write operations are completed from the nonvolatile media before the instruction is completed. + + - **--show-command, -v**: displays instruction information before sending a read/write command. + + - **--dry-run, -w**: displays only information about read and write instructions but does not perform actual read and write operations. + + - **--latency. -t**: collects statistics on the end-to-end read and write latency of the CLI. + + - **--help, -h**: displays the help information about related commands. diff --git a/docs/en/server/memory_storage/lvm/_toc.yaml b/docs/en/server/memory_storage/lvm/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..8fc086fd180f34b3a4543eabeee31a97f1fb5804 --- /dev/null +++ b/docs/en/server/memory_storage/lvm/_toc.yaml @@ -0,0 +1,6 @@ +label: Logical Volume Configuration and Management +isManual: true +description: Use LVM to manage drives. +sections: + - label: Managing Drives Through LVM + href: ./managing-drives-through-lvm.md diff --git a/docs/en/server/memory_storage/lvm/managing-drives-through-lvm.md b/docs/en/server/memory_storage/lvm/managing-drives-through-lvm.md new file mode 100644 index 0000000000000000000000000000000000000000..1281426851d143c5c122a56890ba3a448d83afe0 --- /dev/null +++ b/docs/en/server/memory_storage/lvm/managing-drives-through-lvm.md @@ -0,0 +1,575 @@ +# Managing Drives Through LVM + + + +- [Managing Drives Through LVM](#managing-drives-through-lvm) + - [LVM Overview](#lvm-overview) + - [Basic Terms](#basic-terms) + - [Installing the LVM](#installing-the-lvm) + - [Managing PVs](#managing-pvs) + - [Creating a PV](#creating-a-pv) + - [Viewing a PV](#viewing-a-pv) + - [Modifying PV Attributes](#modifying-pv-attributes) + - [Deleting a PV](#deleting-a-pv) + - [Managing VGs](#managing-vgs) + - [Creating a VG](#creating-a-vg) + - [Viewing a VG](#viewing-a-vg) + - [Modifying VG Attributes](#modifying-vg-attributes) + - [Extending a VG](#extending-a-vg) + - [Shrinking a VG](#shrinking-a-vg) + - [Deleting a VG](#deleting-a-vg) + - [Managing LVs](#managing-lvs) + - [Creating an LV](#creating-an-lv) + - [Viewing an LV](#viewing-an-lv) + - [Adjusting the LV Size](#adjusting-the-lv-size) + - [Extending an LV](#extending-an-lv) + - [Shrinking an LV](#shrinking-an-lv) + - [Deleting an LV](#deleting-an-lv) + - [Creating and Mounting a File System](#creating-and-mounting-a-file-system) + - [Creating a File System](#creating-a-file-system) + - [Manually Mounting a File System](#manually-mounting-a-file-system) + - [Automatically Mounting a File System](#automatically-mounting-a-file-system) + + + +## LVM Overview + +Logical Volume Manager \(LVM\) is a mechanism used for managing drive partitions in Linux. By adding a logical layer between drives and file systems, LVM shields the drive partition layout for file systems, thereby improving flexibility in managing drive partitions. + +The procedure of managing a drive through LVM is as follows: + +1. Create physical volumes for a drive. +2. Combine several physical volumes into a volume group. +3. Create logical volumes in the volume group. +4. Create file systems on logical volumes. + +When drives are managed using LVM, file systems are distributed on multiple drives and can be easily resized as needed. Therefore, file system space will no longer be limited by drive capacities. + +### Basic Terms + +- Physical media: refers to physical storage devices in the system, such as drives \(**/dev/hda** and **/dev/sda**\). It is the storage unit at the lowest layer of the storage system. + +- Physical volume \(PV\): refers to a drive partition or device \(such as a RAID\) that has the same logical functions as a drive partition. PVs are basic logical storage blocks of LVM. A PV contains a special label that is stored in the second 512-byte sector by default. It can also be stored in one of the first four sectors. A label contains the universal unique identifier \(UUID\) of the PV, size of the block device, and the storage location of LVM metadata in the device. + +- Volume group \(VG\): consists of PVs and shields the details of underlying PVs. You can create one or more logical volumes within a VG without considering detailed PV information. + +- Logical volume \(LV\): A VG cannot be used directly. It can be used only after being partitioned into LVs. LVs can be formatted into different file systems and can be directly used after being mounted. + +- Physical extent \(PE\): A PE is a small storage unit in a PV. The PE size is the same as the size of the logical extent in the VG. + +- Logical extent \(LE\): An LE is a small storage unit in an LV. In one VG, the LEs of all the LVs have the same size. + +## Installing the LVM + +> [!NOTE]NOTE +> The LVM has been installed on the openEuler OS by default. You can run the **rpm -qa | grep lvm2** command to check whether it is installed. If the command output contains "lvm2", the LVM has been installed. In this case, skip this section. If no information is output, the LVM is not installed. Install it by referring to this section. + +1. Configure the local yum source. For details, see [Configuring the Repo Server](../../administration/administrator/configuring-the-repo-server.md). +2. Clear the cache. + + ```bash + dnf clean all + ``` + +3. Create a cache. + + ```bash + dnf makecache + ``` + +4. Install the LVM as the **root** user. + + ```bash + dnf install lvm2 + ``` + +5. Check the installed RPM package. + + ```bash + rpm -qa | grep lvm2 + ``` + +## Managing PVs + +### Creating a PV + +Run the **pvcreate** command as the **root** user to create a PV. + +```bash +pvcreate [option] devname ... +``` + +In the preceding information: + +- _option_: command parameter options. Common parameter options are as follows: + - **-f**: forcibly creates a PV without user confirmation. + - **-u**: specifies the UUID of the device. + - **-y**: answers yes to all questions. + +- _devname_: specifies the name of the device corresponding to the PV to be created. If multiple PVs need to be created in batches, set this option to multiple device names and separate the names with spaces. + +Example 1: Create PVs based on **/dev/sdb** and **/dev/sdc**. + +```bash +pvcreate /dev/sdb /dev/sdc +``` + +Example 2: Create PVs based on **/dev/sdb1** and **/dev/sdb2**. + +```bash +pvcreate /dev/sdb1 /dev/sdb2 +``` + +### Viewing a PV + +Run the **pvdisplay** command as the **root** user to view PV information, including PV name, VG to which the PV belongs, PV size, PE size, total number of PEs, number of available PEs, number of allocated PEs, and UUID. + +```bash +pvdisplay [option] devname +``` + +In the preceding information: + +- _option_: command parameter options. Common parameter options are as follows: + - **-s**: outputs information in short format. + - **-m**: displays the mapping from PEs to LEs. + +- _devname_: indicates the device corresponding to the PV to be viewed. If no PVs are specified, information about all PVs is displayed. + +Example: Run the following command to display the basic information about the PV **/dev/sdb**: + +```bash +pvdisplay /dev/sdb +``` + +### Modifying PV Attributes + +Run the **pvchange** command as the **root** user to modify the attributes of a PV. + +```bash +pvchange [option] pvname ... +``` + +In the preceding information: + +- _option_: command parameter options. Common parameter options are as follows: + - **-u**: generates a new UUID. + - **-x**: indicates whether PE allocation is allowed. + +- _pvname_: specifies the name of the device corresponding to the PV to be modified. If multiple PVs need to be modified in batches, set this option to multiple device names and separate the names with spaces. + +Example: Run the following command to prohibit PEs on the PV **/dev/sdb** from being allocated. Running `pvdisplay` for a PV that is not added to a VG will return the **Allocatable** attribute with the value **NO**. You need to add the PV to a VG before you can change the attribute. + +```bash +pvchange -x n /dev/sdb +``` + +### Deleting a PV + +Run the **pvremove** command as the **root** user to delete a PV. + +```bash +pvremove [option] pvname ... +``` + +In the preceding information: + +- _option_: command parameter options. Common parameter options are as follows: + - **-f**: forcibly deletes a PV without user confirmation. + - **-y**: answers yes to all questions. + +- _pvname_: specifies the name of the device corresponding to the PV to be deleted. If multiple PVs need to be deleted in batches, set this option to multiple device names and separate the names with spaces. + +Example: Run the following command to delete the PV **/dev/sdb**. If the PV has been added to a VG, you need to delete the VG or remove the PV from the VG in advance. + +```bash +pvremove /dev/sdb +``` + +## Managing VGs + +### Creating a VG + +Run the **vgcreate** command as the **root** user to create a VG. + +```bash +vgcreate [option] vgname pvname ... +``` + +In the preceding information: + +- _option_: command parameter options. Common parameter options are as follows: + - **-l**: specifies the maximum number of LVs that can be created on the VG. + - **-p**: specifies the maximum number of PVs that can be added to the VG. + - **-s**: specifies the PE size of a PV in the VG. + +- _vgname_: name of the VG to be created. +- _pvname_: name of the PV to be added to the VG. + +Example: Run the following command to create VG **vg1** and add the PVs **/dev/sdb** and **/dev/sdc** to the VG. + +```bash +vgcreate vg1 /dev/sdb /dev/sdc +``` + +### Viewing a VG + +Run the **vgdisplay** command as the **root** user to view VG information. + +```bash +vgdisplay [option] [vgname] +``` + +In the preceding information: + +- _option_: command parameter options. Common parameter options are as follows: + - **-s**: outputs information in short format. + - **-A**: displays only attributes of active VGs. + +- _vgname_: name of the VG to be viewed. If no VGs are specified, information about all VGs is displayed. + +Example: Run the following command to display the basic information about VG **vg1**: + +```bash +vgdisplay vg1 +``` + +### Modifying VG Attributes + +Run the **vgchange** command as the **root** user to modify the attributes of a VG. + +```bash +vgchange [option] vgname +``` + +In the preceding information: + +- _option_: command parameter options. Common parameter options are as follows: + - **-a**: sets the active status of the VG. + +- _vgname_: name of the VG whose attributes are to be modified. + +Example: Run the following command to change the status of **vg1** to active. + +```bash +vgchange -ay vg1 +``` + +### Extending a VG + +Run the **vgextend** command as the **root** user to dynamically extend a VG. In this way, the VG size is extended by adding PVs to the VG. + +```bash +vgextend [option] vgname pvname ... +``` + +In the preceding information: + +- _option_: command parameter options. Common parameter options are as follows: + - **dev**: debugging mode. + - **-t**: test only. + +- _vgname_: name of the VG whose size is to be extended. +- _pvname_: name of the PV to be added to the VG. + +Example: Run the following command to add PV **/dev/sdb** to VG **vg1**: + +```bash +vgextend vg1 /dev/sdb +``` + +### Shrinking a VG + +Run the **vgreduce** command as the **root** user to delete PVs from a VG to reduce the VG size. A VG must contain at least one PV. + +```bash +vgreduce [option] vgname pvname ... +``` + +In the preceding information: + +- _option_: command parameter options. Common parameter options are as follows: + - **-a**: If no PVs are specified in the command, all empty PVs are deleted. + - **\-\-removemissing**: deletes lost PVs in the VG to restore the VG to the normal state. + +- _vgname_: name of the VG to be shrunk. +- _pvname_: name of the PV to be deleted from the VG. + +Example: Run the following command to remove PV **/dev/sdb2** from VG **vg1**: + +```bash +vgreduce vg1 /dev/sdb2 +``` + +### Deleting a VG + +Run the **vgremove** command as the **root** user to delete a VG. + +```bash +vgremove [option] vgname +``` + +In the preceding information: + +- _option_: command parameter options. Common parameter options are as follows: + - **-f**: forcibly deletes a VG without user confirmation. + +- _vgname_: name of the VG to be deleted. + +Example: Run the following command to delete VG **vg1**. + +```bash +vgremove vg1 +``` + +## Managing LVs + +### Creating an LV + +Run the **lvcreate** command as the **root** user to create an LV. + +```bash +lvcreate [option] vgname +``` + +In the preceding information: + +- _option_: command parameter options. Common parameter options are as follows: + - **-L**: specifies the size of the LV in kKmMgGtT. + - **-l**: specifies the size of the LV \(number of LEs\). + - **-n**: specifies the name of the LV to be created. + - **-s**: creates a snapshot. + +- _vgname_: name of the VG to be created. + +Example 1: Run the following command to create a 10 GB LV in VG **vg1**. The default LV name is **lvo10**. + +```bash +lvcreate -L 10G vg1 +``` + +Example 2: Run the following command to create a 200 MB LV in VG **vg1** and name the LV **lv1**. + +```bash +lvcreate -L 200M -n lv1 vg1 +``` + +### Viewing an LV + +Run the **lvdisplay** command as the **root** user to view the LV information, including the size of the LV, its read and write status, and snapshot information. + +```bash +lvdisplay [option] [lvname] +``` + +In the preceding information: + +- _option_: command parameter options. Common parameter options are as follows: + - **-v**: displays the mapping from LEs to PEs. + +- _lvname_: device file corresponding to the LV whose attributes are to be displayed. If this option is not set, attributes of all LVs are displayed. + + > [!NOTE]NOTE + > Device files corresponding to LVs are stored in the VG directory. For example, if LV **lv1** is created in VG **vg1**, the device file corresponding to **lv1** is **/dev/vg1/lv1**. + +Example: Run the following command to display the basic information about LV **lv1**: + +```bash +lvdisplay /dev/vg1/lv1 +``` + +### Adjusting the LV Size + +Run the **lvresize** command as the **root** user to increase or reduce the size of an LVM LV. This may cause data loss. Therefore, exercise caution when running this command. + +```bash +lvresize [option] vgname +``` + +In the preceding information: + +- _option_: command parameter options. Common parameter options are as follows: + - **-L**: specifies the size of the LV in kKmMgGtT. + - **-l**: specifies the size of the LV \(number of LEs\). + - **-f**: forcibly adjusts the size of the LV without user confirmation. + +- _lvname_: name of the LV to be adjusted. + +Example 1: Run the following command to increase the size of LV **/dev/vg1/lv1** by 200 MB. + +```bash +lvresize -L +200 /dev/vg1/lv1 +``` + +Example 2: Run the following command to reduce the size of LV **/dev/vg1/lv1** by 200 MB. + +```bash +lvresize -L -200 /dev/vg1/lv1 +``` + +### Extending an LV + +Run the **lvextend** command as the **root** user to dynamically extend the size of an LV online without interrupting the access of applications to the LV. + +```bash +lvextend [option] lvname +``` + +In the preceding information: + +- _option_: command parameter options. Common parameter options are as follows: + - **-L**: specifies the size of the LV in kKmMgGtT. + - **-l**: specifies the size of the LV \(number of LEs\). + - **-f**: forcibly adjusts the size of the LV without user confirmation. + +- _lvname_: device file of the LV whose size is to be extended. + +Example: Run the following command to increase the size of LV **/dev/vg1/lv1** by 100 MB. + +```bash +lvextend -L +100M /dev/vg1/lv1 +``` + +### Shrinking an LV + +Run the **lvreduce** command as the **root** user to reduce the size of an LV. This may delete existing data on the LV. Therefore, confirm whether the data can be deleted before running the command. + +```bash +lvreduce [option] lvname +``` + +In the preceding information: + +- _option_: command parameter options. Common parameter options are as follows: + - **-L**: specifies the size of the LV in kKmMgGtT. + - **-l**: specifies the size of the LV \(number of LEs\). + - **-f**: forcibly adjusts the size of the LV without user confirmation. + +- _lvname_: device file of the LV whose size is to be extended. + +Example: Run the following command to reduce the space of LV **/dev/vg1/lvl** by 100 MB: + +```bash +lvreduce -L -100M /dev/vg1/lv1 +``` + +### Deleting an LV + +Run the **lvremove** command as the **root** user to delete an LV. If the LV has been mounted by running the **mount** command, you need to run the **umount** command to unmount the LV before running the **lvremove** command. + +```bash +lvremove [option] lvname +``` + +In the preceding information: + +- _option_: command parameter options. Common parameter options are as follows: + - **-f**: forcibly deletes an LV without user confirmation. + +- _lvname_: device name of the LV to be deleted. + +Example: Run the following command to delete LV **/dev/vg1/lv1**. + +```bash +lvremove /dev/vg1/lv1 +``` + +## Creating and Mounting a File System + +After creating an LV, you need to create a file system on the LV and mount the file system to the corresponding directory. + +### Creating a File System + +Run the **mkfs** command as the **root** user to create a file system. + +```bash +mkfs [option] lvname +``` + +In the preceding information: + +- _option_: command parameter options. Common parameter options are as follows: + - **-t**: specifies the type of the Linux file system to be created, such as **ext2**, **ext3**, and **ext4**. The default type is **ext2**. + +- _lvname_: name of the LV device file corresponding to the file system to be created. + +Example: Run the following command to create the **ext4** file system on LV **/dev/vg1/lv1**: + +```bash +mkfs -t ext4 /dev/vg1/lv1 +``` + +### Manually Mounting a File System + +The file system that is manually mounted is not valid permanently. It does not exist after the OS is restarted. + +Run the **mount** command as the **root** user to mount a file system. + +```bash +mount lvname mntpath +``` + +In the preceding information: + +- _lvname_: name of the LV device file corresponding to the file system to be mounted. +- _mntpath_: mount path. + +Example: Run the following command to mount LV **/dev/vg1/lv1** to the directory **/mnt/data**. + +```bash +mount /dev/vg1/lv1 /mnt/data +``` + +### Automatically Mounting a File System + +A file system that is automatically mounted does not exist after the OS is restarted. You need to manually mount the file system again. If you perform the following steps as the **root** user after manually mounting the file system, the file system can be automatically mounted after the OS is restarted. + +1. Run the **blkid** command to query the UUID of an LV. The following uses LV **/dev/vg1/lv1** as an example: + + ```bash + blkid /dev/vg1/lv1 + ``` + + Check the command output. It contains the following information in which _uuidnumber_ is a string of digits, indicating the UUID, and _fstype_ indicates the file system type. + + /dev/vg1/lv1: UUID=" _uuidnumber_ " TYPE=" _fstype_ " + +2. Run the **vi /etc/fstab** command to edit the **fstab** file and add the following content to the end of the file: + + ```vim + UUID=uuidnumber mntpath fstype defaults 0 0 + ``` + + In the preceding information: + + - Column 1: indicates the UUID. Enter _uuidnumber_ obtained in [1](#li65701520154311). + - Column 2: indicates the mount directory of the file system. Replace _mntpath_ with the actual value. + - Column 3: indicates the file system format. Enter _fstype_ obtained in [1](#li65701520154311). + - Column 4: indicates the mount option. In this example, **defaults** is used. + - Column 5: indicates the backup option. Enter either **1** \(the system automatically backs up the file system\) or **0** \(the system does not back up the file system\). In this example, **0** is used. + - Column 6: indicates the scanning option. Enter either **1** \(the system automatically scans the file system during startup\) or **0** \(the system does not scan the file system\). In this example, **0** is used. + +3. Verify the automatic mounting function. + 1. Run the **umount** command to unmount the file system. The following uses LV **/dev/vg1/lv1** as an example: + + ```bash + umount /dev/vg1/lv1 + ``` + + 2. Run the following command to reload all content in the **/etc/fstab** file: + + ```bash + mount -a + ``` + + 3. Run the following command to query the file system mounting information \(**/mnt/data** is used as an example\): + + ```bash + mount | grep /mnt/data + ``` + + Check the command output. If the command output contains the following information, the automatic mounting function takes effect: + + ```text + /dev/vg1/lv1 on /mnt/data + ``` diff --git a/docs/en/server/memory_storage/lvm/public_sys-resources/icon-note.gif b/docs/en/server/memory_storage/lvm/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/server/memory_storage/lvm/public_sys-resources/icon-note.gif differ diff --git a/docs/en/server/memory_storage/overview.md b/docs/en/server/memory_storage/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..23bc749dd9e7a8fd33390910f1fe2d4680332ca7 --- /dev/null +++ b/docs/en/server/memory_storage/overview.md @@ -0,0 +1,149 @@ +# Memory Management + +## Basic Concepts + +The memory is an important component of a computer, and is used to temporarily store operation data in the CPU and data exchanged with an external memory such as hardware. In particular, a non-uniform memory access architecture (NUMA) is a memory architecture designed for a multiprocessor computer. The memory access time depends on the location of the memory relative to the processor. In NUMA mode, a processor accesses the local memory faster than the non-local memory (the memory is located in another processor or shared between processors). + +## Memory Monitoring + +1. `free`: displays the system memory status. + Example: + + ```shell + # Display the system memory status in MB. + free -m + ``` + + The output is as follows: + + ```shell + [root@openEuler ~]# free -m + total used free shared buff/cache available + Mem: 2633 436 324 23 2072 2196 + Swap: 4043 0 4043 + ``` + + The fields in the command output are as follows: + + | Field | Description | + | ---------- | ----------------------------------------------------------------------- | + | total | Total memory size. | + | used | Used memory. | + | free | Free memory. | + | shared | Total memory shared by multiple processes. | + | buff/cache | Total number of buffers and caches. | + | available | Estimated available memory to start a new application without swapping. | + +2. `vmstat`: dynamically monitors the system memory and views the system memory usage. + + Example: + + ```shell + # Monitor the system memory and display active and inactive memory. + vmstat -a + ``` + + The output is as follows: + + ```shell + [root@openEuler ~]# vmstat -a + procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- + r b swpd free inact active si so bi bo in cs us sy id wa st + 2 0 520 331980 1584728 470332 0 0 0 2 15 19 0 0 100 0 0 + ``` + + In the command output, the field related to the memory is as follows: + + | Field | Description | + | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | memory | Memory information.
**-swpd**: usage of the virtual memory, in KB.
**-free**: free memory capacity, in KB.
**-inact**: inactive memory capacity, in KB.
**-active**: active memory capacity, in KB. | + +3. `sar`: monitors the memory usage of the system. + + Example: + + ```shell + # Monitor the memory usage in the sampling period in the system. Collect the statistics every two seconds for three times. + sar -r 2 3 + ``` + + The output is as follows: + + ```shell + [root@openEuler ~]# sar -r 2 3 + + 04:02:09 PM kbmemfree kbavail kbmemused %memused kbbuffers kbcached kbcommit %commit kbactive kbinact kb + dirty + 04:02:11 PM 332180 2249308 189420 7.02 142172 1764312 787948 11.52 470404 1584924 + 36 + 04:02:13 PM 332148 2249276 189452 7.03 142172 1764312 787948 11.52 470404 1584924 + 36 + 04:02:15 PM 332148 2249276 189452 7.03 142172 1764312 787948 11.52 470404 1584924 + 36 + Average: 332159 2249287 189441 7.03 142172 1764312 787948 11.52 470404 1584924 + 36 + ``` + + The fields in the command output are described as follows: + + | Field | Description | + | --------- | ------------------------------------------------ | + | kbmemfree | Unused memory space. | + | kbmemused | Used memory space. | + | %memused | Percentage of the used space. | + | kbbuffers | Amount of data stored in the buffer. | + | kbcached | Data access volume in all domains of the system. | + +4. `numactl`: displays the NUMA node configuration and status. + + Example: + + ```shell + # Check the current NUMA configuration. + numactl -H + ``` + + The output is as follows: + + ```shell + [root@openEuler ~]# numactl -H + available: 1 nodes (0) + node 0 cpus: 0 1 2 3 + node 0 size: 2633 MB + node 0 free: 322 MB + node distances: + node 0 + 0: 10 + ``` + + The server contains one NUMA node. The NUMA node that contains four cores and 6 GB memory. + The command also displays the distance between NUMA nodes. The further the distance, the higher the latency of cross-node memory accesses, which should be avoided as much as possible. + + `numastat`: displays NUMA node status. + + ```shell + # Check the NUMA node status. + numastat + ``` + + ```shell + [root@openEuler ~]# numastat + node0 + numa_hit 5386186 + numa_miss 0 + numa_foreign 0 + interleave_hit 17483 + local_node 5386186 + other_node 0 + ``` + + The fields in the `numastat` command output are described as follows: + + | Field | Description | + | -------------- | ----------------------------------------------------------------------------------------------------------------------------------- | + | numa_hit | Number of times that the CPU core accesses the local memory on a node. | + | numa_miss | Number of times that the core of a node accesses the memory of other nodes. | + | numa_foreign | Number of pages that were allocated to the local node but moved to other nodes. Each numa_foreign corresponds to a numa_miss event. | + | interleave_hit | Number of pages of the interleave policy that are allocated to this node. | + | local_node | Size of memory that was allocated to this node by processes on this node. | + | other_node | Size of memory that was allocated to other nodes by processes on this node. | diff --git a/docs/en/server/network/gazelle/_toc.yaml b/docs/en/server/network/gazelle/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..9d2a4f8e3436e0cdcd51ca112475e7e413997df0 --- /dev/null +++ b/docs/en/server/network/gazelle/_toc.yaml @@ -0,0 +1,4 @@ +label: Gazelle User Guide +isManual: true +href: ./gazelle-user-guide.md +description: Improved network I/O throughput for applications diff --git a/docs/en/server/network/gazelle/gazelle-user-guide.md b/docs/en/server/network/gazelle/gazelle-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..66d26ad648dad56512ad4de4d2e89059b06e5da8 --- /dev/null +++ b/docs/en/server/network/gazelle/gazelle-user-guide.md @@ -0,0 +1,285 @@ +# Gazelle User Guide + +## Overview + +Gazelle is a high-performance user-mode protocol stack. It directly reads and writes NIC packets in user mode based on DPDK and transmit the packets through shared hugepage memory, and uses the LwIP protocol stack. Gazelle greatly improves the network I/O throughput of applications and accelerates the network for the databases, such as MySQL and Redis. + +- High Performance + Zero-copy and lock-free packets that can be flexibly scaled out and scheduled adaptively. +- Universality + Compatible with POSIX without modification, and applicable to different types of applications. + +In the single-process scenario where the NIC supports multiple queues, use **liblstack.so** only to shorten the packet path. + +## Installation + +Configure the Yum source of openEuler and run the`yum` command to install Gazelle. + +```sh +yum install dpdk +yum install libconfig +yum install numactl +yum install libboundscheck +yum install libpcap +yum install gazelle +``` + +> NOTE: +> The version of dpdk must be 21.11-2 or later. + +## How to Use + +To configure the operating environment and use Gazelle to accelerate applications, perform the following steps: + +### 1. Installing the .ko File as the root User + +Install the .ko files based on the site requirements to bind NICs to the user-mode driver. + +Bind the NIC from the kernel driver to the user-mode driver. Choose one of the following .ko files based on the site requirements. + +```sh +#If the IOMMU is available +modprobe vfio-pci + +#If the IOMMU is not available and the VFIO supports the no-IOMMU mode +modprobe vfio enable_unsafe_noiommu_mode=1 +modprobe vfio-pci + +#Other cases +modprobe igb_uio +``` + +>NOTE: +You can check whether the IOMMU is enabled based on the BIOS configuration. + +### 2. Binding the NIC Using DPDK + +Bind the NIC to the driver selected in Step 1 to provide an interface for the user-mode NIC driver to access the NIC resources. + +```sh +#Using vfio-pci +dpdk-devbind -b vfio-pci enp3s0 + +#Using igb_uio +dpdk-devbind -b igb_uio enp3s0 +``` + +### 3. Configuring Memory Huge Pages + +Gazelle uses hugepage memory to improve efficiency. You can configure any size for the memory huge pages reserved by the system using the **root** permissions. Each memory huge page requires a file descriptor. If the memory is large, you are advised to use 1 GB huge pages to avoid occupying too many file descriptors. +Select a page size based on the site requirements and configure sufficient memory huge pages. Run the following commands to configure huge pages: + +```sh +#Configuring 1024 2 MB huge pages on node0. The total memory is 2 GB. +echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages + +#Configuring 5 1 GB huge pages on node0. The total memory is 5 GB. +echo 5 > /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages +``` + +>NOTE: +Run the **cat** command to query the actual number of reserved pages. If the continuous memory is insufficient, the number may be less than expected. + +### 4. Mounting Memory Huge Pages + +Create a directory for the lstack process to access the memory huge pages. Run the following commands: + +```sh +mkdir -p /mnt/hugepages-lstack +chmod -R 700 /mnt/hugepages-lstack + +mount -t hugetlbfs nodev /mnt/hugepages-lstack -o pagesize=2M +``` + +### 5. Enabling Gazelle for an Application + +Enable Gazelle for an application using either of the following methods as required. + +- Recompile the application and replace the sockets interface. + +```sh +#Add the Makefile of Gazelle to the application makefile. +-include /etc/gazelle/lstack.Makefile + +#Add the LSTACK_LIBS variable when compiling the source code. +gcc test.c -o test ${LSTACK_LIBS} +``` + +- Use the **LD_PRELOAD** environment variable to load the Gazelle library. + Use the **GAZELLE_BIND_PROCNAME** environment variable to specify the process name, and **LD_PRELOAD** to specify the Gazelle library path. + + ```sh + GAZELLE_BIND_PROCNAME=test LD_PRELOAD=/usr/lib64/liblstack.so ./test + ``` + +- Use the **GAZELLE_THREAD_NAME** environment variable to specify the thread bound to Gazelle. + If only one thread of a multi-thread process meets the conditions for using Gazelle, use **GAZELLE_THREAD_NAME** to specify the thread for using Gazelle. Other threads use kernel-mode protocol stack. + + ```sh + GAZELLE_BIND_PROCNAME=test GAZELLE_THREAD_NAME=test_thread LD_PRELOAD=/usr/lib64/liblstack.so ./test + ``` + +### 6. Configuring Gazelle + +- The **lstack.conf** file is used to specify the startup parameters of lstack. The default path is **/etc/gazelle/lstack.conf**. The parameters in the configuration file are as follows: + +|Options|Value|Remarks| +|:---|:---|:---| +|dpdk_args|--socket-mem (mandatory)
--huge-dir (mandatory)
--proc-type (mandatory)
--legacy-mem
--map-perfect
-d|DPDK initialization parameter. For details, see the DPDK description.
**--map-perfect** is an extended feature. It is used to prevent the DPDK from occupying excessive address space and ensure that extra address space is available for lstack.
The **-d** option is used to load the specified .so library file.| +|listen_shadow| 0/1 | Whether to use the shadow file descriptor for listening. This function is enabled when there is a single listen thread and multiple protocol stack threads.| +|use_ltran| 0/1 | Whether to use ltran. This parameter is no longer supported.| +|num_cpus|"0,2,4 ..."|IDs of the CPUs bound to the lstack threads. The number of IDs is the number of lstack threads (less than or equal to the number of NIC queues). You can select CPUs by NUMA nodes.| +|low_power_mode|0/1|Whether to enable the low-power mode. This parameter is not supported currently.| +|kni_switch|0/1|Whether to enable the rte_kni module. The default value is **0**. This parameter is no longer supported.| +|unix_prefix|"string"|Prefix string of the Unix socket file used for communication between Gazelle processes. By default, this parameter is left blank. The value must be the same as the value of **unix_prefix** in **ltran.conf** of the ltran process that participates in communication, or the value of the **-u** option for `gazellectl`. The value cannot contain special characters and can contain a maximum of 128 characters.| +|host_addr|"192.168.xx.xx"|IP address of the protocol stack, which is also the IP address of the application.| +|mask_addr|"255.255.xx.xx"|Subnet mask.| +|gateway_addr|"192.168.xx.1"|Gateway address.| +|devices|"aa:bb:cc:dd:ee:ff"|MAC address for NIC communication. The NIC is used as the primary bond NIC in bond 1 mode. | +|app_bind_numa|0/1|Whether to bind the epoll and poll threads of an application to the NUMA node where the protocol stack is located. The default value is 1, indicating that the threads are bound.| +|send_connect_number|4|Number of connections for sending packets in each protocol stack loop. The value is a positive integer.| +|read_connect_number|4|Number of connections for receiving packets in each protocol stack loop. The value is a positive integer.| +|rpc_number|4|Number of RPC messages processed in each protocol stack loop. The value is a positive integer.| +|nic_read_num|128|Number of data packets read from the NIC in each protocol stack cycle. The value is a positive integer.| +|bond_mode|-1|Bond mode. Currently, two network ports can be bonded. The default value is -1, indicating that the bond mode is disabled. bond1/4/6 is supported.| +|bond_slave_mac|"aa:bb:cc:dd:ee:ff;AA:BB:CC:DD:EE:FF"|MAC addresses of the bond network ports. Separate the MAC addresses with semicolons (;).| +|bond_miimon|10|Listening interval in bond mode. The default value is 10. The value ranges from 0 to 1500.| +|udp_enable|0/1|Whether to enable the UDP function. The default value is 1.| +|nic_vlan_mode|-1|Whether to enable the VLAN mode. The default value is -1, indicating that the VLAN mode is disabled. The value ranges from -1 to 4095. IDs 0 and 4095 are commonly reserved in the industry and have no actual effect.| +|tcp_conn_count|1500|Maximum number of TCP connections. The value of this parameter multiplied by **mbuf_count_per_conn** is the size of the mbuf pool applied for during initialization. If the value is too small, the startup fails. The value of (**tcp_conn_count** x **mbuf_count_per_conn** x 2048) cannot be greater than the huge page size.| +|mbuf_count_per_conn|170|Number of mbuf required by each TCP connection. The value of this parameter multiplied by **tcp_conn_count** is the size of the mbuf address pool applied for during initialization. If the value is too small, the startup fails. The value of (**tcp_conn_count** x **mbuf_count_per_conn** x 2048) cannot be greater than the huge page size.| + +lstack.conf example: + +```sh +dpdk_args=["--socket-mem", "2048,0,0,0", "--huge-dir", "/mnt/hugepages-lstack", "--proc-type", "primary", "--legacy-mem", "--map-perfect"] + +use_ltran=1 +kni_switch=0 + +low_power_mode=0 + +num_cpus="2,22" +num_wakeup="3,23" + +host_addr="192.168.1.10" +mask_addr="255.255.255.0" +gateway_addr="192.168.1.1" +devices="aa:bb:cc:dd:ee:ff" + +send_connect_number=4 +read_connect_number=4 +rpc_number=4 +nic_read_num=128 +mbuf_pool_size=1024000 +bond_mode=1 +bond_slave_mac="aa:bb:cc:dd:ee:ff;AA:BB:CC:DD:EE:FF" +udp_enable=1 +nic_vlan_mode=-1 +``` + +- The ltran mode is deprecated. If multiple processes are required, try the virtual network mode using SR-IOV network hardware. + +### 7. Starting an Application + +- Start the application. + If the environment variable **LSTACK_CONF_PATH** is not used to specify the configuration file before the application is started, the default configuration file path **/etc/gazelle/lstack.conf** is used. + + ```sh + export LSTACK_CONF_PATH=./lstack.conf + LD_PRELOAD=/usr/lib64/liblstack.so GAZELLE_BIND_PROCNAME=redis-server redis-server redis.conf + ``` + +### 8. APIs + +Gazelle wraps the POSIX interfaces of the application. The code of the application does not need to be modified. + +### 9. Debugging Commands + +```sh +Usage: gazellectl [-h | help] + or: gazellectl lstack {show | set} {ip | pid} [LSTACK_OPTIONS] [time] [-u UNIX_PREFIX] + + where LSTACK_OPTIONS := + show lstack all statistics + -r, rate show lstack statistics per second + -s, snmp show lstack snmp + -c, connetct show lstack connect + -l, latency show lstack latency + -x, xstats show lstack xstats + -k, nic-features show state of protocol offload and other feature + -a, aggregatin [time] show lstack send/recv aggregation + set: + loglevel {error | info | debug} set lstack loglevel + lowpower {0 | 1} set lowpower enable + [time] measure latency time default 1S +``` + +The `-u` option specifies the prefix of the Unix socket for communication between Gazelle processes. The value of this parameter must be the same as that of **unix_prefix** in the **lstack.conf** file. + +**Packet Capturing Tool** +The NIC used by Gazelle is managed by DPDK. Therefore, tcpdump cannot capture Gazelle packets. As a substitute, Gazelle uses gazelle-pdump provided in the dpdk-tools software package as the packet capturing tool. gazelle-pdump uses the multi-process mode of DPDK to share memory with the lstack process. +[Usage](https://gitee.com/openeuler/gazelle/blob/master/doc/pdump.md) + +**Thread Binding** +When the starting a lstack process, you can specify a thread bound to lstack using the environment variable **GAZELLE_THREAD_NAME**. When there are multiple threads in the service process, you can use this variable to specify the thread whose network interface needs to be managed by lstack. Other threads will use the kernel-mode protocol stack. By default, this parameter is left blank, that is, all threads in the process are bound. + +### 10. Precautions + +### 1. Location of the DPDK Configuration File + +For the **root** user, the configuration file is stored in the **/var/run/dpdk** directory after the DPDK is started. +For a non-root user, the path of the DPDK configuration file is determined by the environment variable **XDG_RUNTIME_DIR**. + +- If **XDG_RUNTIME_DIR** is not set, the DPDK configuration file is stored in **/tmp/dpdk**. +- If **XDG_RUNTIME_DIR** is set, the DPDK configuration file is stored in the path specified by **XDG_RUNTIME_DIR**. +- Note that **XDG_RUNTIME_DIR** is set by default on some servers. + +## Restrictions + +Restrictions of Gazelle are as follows: + +### Function Restrictions + +- Blocking **accept()** or **connect()** is not supported. +- A maximum of 1500 TCP connections are supported. +- Currently, only TCP, ICMP, ARP, IPv4, and UDP are supported. +- When a peer end pings Gazelle, the specified packet length must be less than or equal to 14,000 bytes. +- Transparent huge pages are not supported. +- VM NICs do not support multiple queues. + +### Operation Restrictions + +- By default, the command lines and configuration files provided by Gazelle requires **root** permissions. Privilege escalation and changing of file owner are required for non-root users. +- To bind the NIC from user-mode driver back to the kernel driver, you must exit Gazelle first. +- Memory huge pages cannot be remounted to subdirectories created in the mount point. +- The minimum hugepage memory of each application instance protocol stack thread is 800 MB. +- Gazelle supports only 64-bit OSs. +- The `-march=native` option is used when building the x86 version of Gazelle to optimize Gazelle based on the CPU instruction set of the build environment (Intel® Xeon® Gold 5118 CPU @ 2.30GHz). Therefore, the CPU of the operating environment must support the SSE4.2, AVX, AVX2, and AVX-512 instruction set extensions. +- The maximum number of IP fragments is 10 (the maximum ping packet length is 14,790 bytes). TCP does not use IP fragments. +- You are advised to set the **rp_filter** parameter of the NIC to 1 using the `sysctl` command. Otherwise, the Gazelle protocol stack may not be used as expected. Instead, the kernel protocol stack is used. +- The hybrid bonding of multiple types of NICs is not supported. +- The active/standby mode (bond1 mode) supports active/standby switchover only when a fault occurs at the link layer (for example, the network cable is disconnected), but does not support active/standby switchover when a fault occurs at the physical layer (for example, the NIC is powered off or removed). +- If the length of UDP packets to be sent exceeds 45952 (32 x 1436) bytes, increase the value of **send_ring_size** to at least 64. + +## Precautions + +You need to evaluate the use of Gazelle based on application scenarios. + +The ltran mode and kni module is no longer supported due to changes in the dependencies and upstream community. + +**Shared Memory** + +- Current situation: + The memory huge pages are mounted to the **/mnt/hugepages-lstack** directory. During process initialization, files are created in the **/mnt/hugepages-lstack** directory. Each file corresponds to a huge page, and the mmap function is performed on the files. +- Current mitigation measures + The huge page file permission is **600**. Only the owner can access the files. The default owner is the **root** user. Other users can be configured. + Huge page files are locked by DPDK and cannot be directly written or mapped. +- Caution + Malicious processes belonging to the same user imitate the DPDK implementation logic to share huge page memory using huge page files and perform write operations to damage the huge page memory. As a result, the Gazelle program crashes. It is recommended that the processes of a user belong to the same trust domain. + +**Traffic Limit** +Gazelle does not limit the traffic. Users can send packets at the maximum NIC line rate to the network, which may congest the network. + +**Process Spoofing** +Ensure that all lstack processes are trusted. diff --git a/docs/en/server/network/network_config/_toc.yaml b/docs/en/server/network/network_config/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..249c6eb2b934409a1083843b77a069d1d5d74e6b --- /dev/null +++ b/docs/en/server/network/network_config/_toc.yaml @@ -0,0 +1,4 @@ +label: Network Configuration +isManual: true +href: ./network-configuration.md +description: Configuring the IP address, host name, and network binding diff --git a/docs/en/server/network/network_config/network-configuration.md b/docs/en/server/network/network_config/network-configuration.md new file mode 100644 index 0000000000000000000000000000000000000000..e0572cd04369ed7134d6a3bab418bfac34683579 --- /dev/null +++ b/docs/en/server/network/network_config/network-configuration.md @@ -0,0 +1,1331 @@ +# Configuring the Network + +## Configuring an IP Address + +### Using the nmcli Command + +> [!NOTE]NOTE +> The network configuration configured by running the **nmcli** command takes effect immediately and will not be lost after the system restarts. + +#### Introduction to nmcli + +**nmcli** \(NetworkManager Command Line Interface\) is the command-line utility to configure networking through NetworkManager. The basic format of using **nmcli** is as follows: + +```shell +nmcli [OPTIONS] OBJECT { COMMAND | help } +``` + +In the preceding command, **OBJECT** can be one of the following options: **general**, **networking**, **radio**, **connection**, and **device**. **OPTIONS** can be optional options, such as **-t**, **\-\-terse** \(for script processing\),**-p**, **\-\-pretty** \(for human-readable output\), **-h**, and **\-\-help**. For more information, run the **nmcli help** command. + +```shell +nmcli help +``` + +Common commands are listed as follows: + +- To display the general status of NetworkManager, run the following command: + + ```shell + nmcli general status + ``` + +- To display all connections, run the following command: + + ```shell + nmcli connection show + ``` + +- To display the current active connections only, add the **-a** or **\-\-active** option as follows: + + ```shell + nmcli connection show --active + ``` + +- To display the device identified by NetworkManager and its connection status, run the following command: + + ```shell + nmcli device status + ``` + +- To start or stop network interfaces, for example, run the nmcli commands as the **root** user: + + ```shell + nmcli connection up id enp3s0 + nmcli device disconnect enp3s0 + ``` + +#### Device Management + +##### Connecting to a Device + +Run the following command to connect NetworkManager to the corresponding network device. Try to find the proper connection configuration and activate it. + + ```shell + nmcli device connect "$IFNAME" + ``` + +> If the corresponding connection configuration does not exist, NetworkManager creates and activates a configuration file with default settings. + +##### Disconnecting to a Device + +Run the following command to disconnect NetworkManager with the network device and prevent the device from being automatically activated. + + ```shell + nmcli device disconnect "$IFNAME" + ``` + +#### Setting Network Connections + +Run the following command to display all the available network connections: + +```shell +$ nmcli con show + + +NAME UUID TYPE DEVICE +enp4s0 5afce939-400e-42fd-91ee-55ff5b65deab ethernet enp4s0 +enp3s0 c88d7b69-f529-35ca-81ab-aa729ac542fd ethernet enp3s0 +virbr0 ba552da6-f014-49e3-91fa-ec9c388864fa bridge virbr0 +``` + +> [!NOTE]NOTE +> In the command output, **NAME** indicates the connection ID \(name\). + +After a network connection is added, the corresponding configuration file is generated and associated with the corresponding device. To check for available devices, run the following command: + +```shell +$ nmcli dev status + +DEVICE TYPE STATE CONNECTION +enp3s0 ethernet connected enp3s0 +enp4s0 ethernet connected enp4s0 +virbr0 bridge connected virbr0 +lo loopback unmanaged -- +virbr0-nic tun unmanaged -- +``` + +##### Configuring Dynamic IP Connections + +###### Configuring IP Addresses + +When DHCP is used to allocate a network, run the following command to add a network configuration file: + +```shell +nmcli connection add type ethernet con-name connection-name ifname interface-name +``` + +For example, to create a dynamic connection configuration file named **net-test**, run the following command as the **root** user: + +```shell +$ nmcli connection add type ethernet con-name net-test ifname enp3s0 +Connection 'net-test' (a771baa0-5064-4296-ac40-5dc8973967ab) successfully added. +``` + +The NetworkManager sets **connection.autoconnect** to **yes** and saves the setting to the **/etc/sysconfig/network-scripts/ifcfg-net-test** file. In the **/etc/sysconfig/network-scripts/ifcfg-net-test** file, **ONBOOT** is set to **yes**. + +###### Activating a Connection and Checking Device Connection Status + +Run the following command as the **root** user to activate a network connection: + +```shell +$ nmcli con up net-test +Connection successfully activated (D-Bus active path:/org/freedesktop/NetworkManager/ActiveConnection/5) +``` + +Run the following command to check the connection status of devices: + +```shell +$ nmcli device status + +DEVICE TYPE STATE CONNECTION +enp4s0 ethernet connected enp4s0 +enp3s0 ethernet connected net-test +virbr0 bridge connected virbr0 +lo loopback unmanaged -- +virbr0-nic tun unmanaged -- +``` + +##### Configuring Static IP Connections + +###### Configuring IP Addresses + +To add a static IPv4 network connection, run the following command: + +```shell +nmcli connection add type ethernet con-name connection-name ifname interface-name ip4 address gw4 address +``` + +> [!NOTE]NOTE +> To add an IPv6 address and related gateway information, use the **ip6** and **gw6** options. + +For example, to create a static connection configuration file named **net-static**, run the following command as the **root** user: + +```shell +nmcli con add type ethernet con-name net-static ifname enp3s0 ip4 192.168.0.10/24 gw4 192.168.0.254 +``` + +You can also specify the IPv6 address and gateway for the device. The following is an example: + +```shell +$ nmcli con add type ethernet con-name test-lab ifname enp3s0 ip4 192.168.0.10/24 gw4 192.168.0.254 ip6 abbe::**** gw6 2001:***::* +Connection 'net-static' (63aa2036-8665-f54d-9a92-c3035bad03f7) successfully added. +``` + +The NetworkManager sets the internal parameter **ipv4.method** to **manual**, **connection.autoconnect** to **yes**, and writes the setting to the **/etc/sysconfig/network-scripts/ifcfg-my-office** file. In the file, **BOOTPROTO** is set to **none**, and **ONBOOT** is set to **yes**. + +Run the following command as the **root** user to set IPv4 addresses of two DNS servers: + +```shell +nmcli con mod net-static ipv4.dns "*.*.*.* *.*.*.*" +``` + +Run the following command as the **root** user to set IPv6 addresses of two DNS servers: + +```shell +nmcli con mod net-static ipv6.dns "2001:4860:4860::**** 2001:4860:4860::****" +``` + +###### Activating a Connection and Checking Device Connection Status + +Run the following command as the **root** user to activate a network connection: + +```shell +$ nmcli con up net-static ifname enp3s0 +Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/6) +``` + +Run the following command to check the connection status of devices: + +```shell +$ nmcli device status + +DEVICE TYPE STATE CONNECTION +enp4s0 ethernet connected enp4s0 +enp3s0 ethernet connected net-static +virbr0 bridge connected virbr0 +lo loopback unmanaged -- +virbr0-nic tun unmanaged -- +``` + +Run the following command to view the connection details \(with the **-p** and **\-\-pretty** options to add the title and segment to the output\): + +```shell +$ nmcli -p con show net-static +=============================================================================== +Connection profile details (net-static ) +=============================================================================== +connection.id: net-static +connection.uuid: b9f18801-6084-4aee-af28-c8f0598ff5e1 +connection.stable-id: -- +connection.type: 802-3-ethernet +connection.interface-name: enp3s0 +connection.autoconnect: yes +connection.autoconnect-priority: 0 +connection.autoconnect-retries: -1 (default) +connection.multi-connect: 0 (default) +connection.auth-retries: -1 +connection.timestamp: 1578988781 +connection.read-only: no +connection.permissions: -- +connection.zone: -- +connection.master: -- +connection.slave-type: -- +connection.autoconnect-slaves: -1 (default) +connection.secondaries: -- +connection.gateway-ping-timeout: 0 +connection.metered: unknown +connection.lldp: default +connection.mdns: -1 (default) +connection.llmnr: -1 (default) +``` + +##### Adding a Wi-Fi Connection + +You can add the Wi-Fi connection using either of the following methods: + +**Method 1: Connect to the Wi-Fi network using a network port.** + +Connect to the Wi-Fi network specified by the SSID or BSSID. Run the following command to find a matching connection or create a connection, and then activate the connection on the device. + +```shell +nmcli device wifi connect "$SSID" password "$PASSWORD" ifname "$IFNAME" +nmcli --ask device wifi connect "$SSID" +``` + +**Method 2: Connect to the Wi-Fi network using the configuration file.** + +1. Run the following command to check for available Wi-Fi access points: + + ```shell + nmcli dev wifi list + ``` + +2. Run the following command to generate a static IP address configuration that allows Wi-Fi connections automatically allocated by the DNS: + + ```shell + nmcli con add con-name Wifi ifname wlan0 type wifi ssid MyWifi ip4 192.168.100.101/24 gw4 192.168.100.1 + ``` + +3. Run the following command to set a WPA2 password, for example, **answer**: + + ```shell + nmcli con modify Wifi wifi-sec.key-mgmt wpa-psk + nmcli con modify Wifi wifi-sec.psk answer + ``` + +4. Run the following command to change the Wi-Fi status: + + ```shell + nmcli radio wifi [ on | off ] + ``` + +##### Modifying Attributes + +Run the following command to check a specific attribute, for example, mtu: + +```shell +$ nmcli connection show id 'Wifi ' | grep mtu +802-11-wireless.mtu: auto +``` + +Run the following command to modify the attribute: + +```shell +nmcli connection modify id 'Wifi ' 802-11-wireless.mtu 1350 +``` + +Run the following command to confirm the modification: + +```shell +$ nmcli connection show id 'Wifi ' | grep mtu +802-11-wireless.mtu: 1350 +``` + +#### Configuring a Static Route + +- Run the nmcli command to configure a static route for a network connection: + + ```shell + nmcli connection modify enp3s0 +ipv4.routes "192.168.122.0/24 10.10.10.1" + ``` + +- Run the following command to configure the static route using the editor: + + ```shell + $ nmcli con edit type ethernet con-name enp3s0 + ===| nmcli interactive connection editor |=== + Adding a new '802-3-ethernet' connection + Type 'help' or '?' for available commands. + Type 'describe [.]' for detailed property description. + You may edit the following settings: connection, 802-3-ethernet (ethernet), 802-1x, ipv4, ipv6, dcb + nmcli> set ipv4.routes 192.168.122.0/24 10.10.10.1 + nmcli> + nmcli> save persistent + Saving the connection with 'autoconnect=yes'. That might result in an immediate activation of the connection. + Do you still want to save? [yes] yes + Connection 'enp3s0' (1464ddb4-102a-4e79-874a-0a42e15cc3c0) successfully saved. + nmcli> quit + ``` + +### Using the ip Command + +> [!NOTE]NOTE +> The network configuration configured using the **ip** command takes effect immediately, but the configuration will be lost after the system restarts. + +#### Configuring IP Addresses + +Run the **ip** command to configure an IP address for the interface. The command format is as follows, where _interface-name_ indicates the NIC name. + +```shell +ip addr [ add | del ] address dev interface-name +``` + +##### Configuring a Static IP Address + +Run the following command as the **root** user to configure an IP address: + +```shell +ip address add 192.168.0.10/24 dev enp3s0 +``` + +Run the following command as the **root** user to view the configuration result: + +```shell +$ ip addr show dev enp3s0 +2: enp3s0: mtu 1500 qdisc fq_codel state UP group default qlen 1000 + link/ether 52:54:00:aa:ad:4a brd ff:ff:ff:ff:ff:ff + inet 192.168.202.248/16 brd 192.168.255.255 scope global dynamic noprefixroute enp3s0 + valid_lft 9547sec preferred_lft 9547sec + inet 192.168.0.10/24 scope global enp3s0 + valid_lft forever preferred_lft forever + inet6 fe80::32e8:cc22:9db2:f4d4/64 scope link noprefixroute + valid_lft forever preferred_lft forever +``` + +##### Configuring Multiple IP Addresses + +The **ip** command can be used to assign multiple IP addresses to an interface. You can run the **ip** command multiple times as the **root** user to assign IP addresses to an interface. The following is an example: + +```shell +$ ip address add 192.168.2.223/24 dev enp4s0 +$ ip address add 192.168.4.223/24 dev enp4s0 +$ ip addr + +3: enp4s0: mtu 1500 qdisc fq_codel state UP group default qlen 1000 + link/ether 52:54:00:aa:da:e2 brd ff:ff:ff:ff:ff:ff + inet 192.168.203.12/16 brd 192.168.255.255 scope global dynamic noprefixroute enp4s0 + valid_lft 8389sec preferred_lft 8389sec + inet 192.168.2.223/24 scope global enp4s0 + valid_lft forever preferred_lft forever + inet 192.168.4.223/24 scope global enp4s0 + valid_lft forever preferred_lft forever + inet6 fe80::1eef:5e24:4b67:f07f/64 scope link noprefixroute + valid_lft forever preferred_lft forever +``` + +#### Configuring a Static Route + +To add a static route to the routing table, run the **ip route add** command. To delete a route, run the **ip route del** command. The following shows the common format of the **ip route** command: + +```shell +ip route [ add | del | change | append | replace ] destination-address +``` + +To display the current IP routing table, run the **ip route** command as the **root** user. The following is an example: + +```shell +$ ip route + +default via 192.168.0.1 dev enp3s0 proto dhcp metric 100 +default via 192.168.0.1 dev enp4s0 proto dhcp metric 101 +192.168.0.0/16 dev enp3s0 proto kernel scope link src 192.168.202.248 metric 100 +192.168.0.0/16 dev enp4s0 proto kernel scope link src 192.168.203.12 metric 101 +192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown +``` + +To add a static route to the host address, run the following command as the **root** user: + +```shell +ip route add 192.168.2.1 via 10.0.0.1 [dev interface-name] +``` + +In the preceding command, **192.168.2.1** is the IP address in the dot-decimal notation, **10.0.0.1** is the next hop, and _interface-name_ is the exit interface for entering the next hop. + +To add a static route to the network, that is, an IP address that represents an IP address range, run the following command as the **root** user: + +```shell +ip route add 192.168.2.0/24 via 10.0.0.1 [dev interface-name] +``` + +In the preceding command, **192.168.2.1** is the IP address of the target network, _10.0.0.1_ is the network prefix, and _interface-name_ is the NIC name. + +### Configuring the Network Through the ifcfg File + +> [!NOTE]NOTE +> The network configured in the **ifcfg** file does not take effect immediately. After modifying the file (for example, **ifcfg-enp3s0**), you need to run the **nmcli con reload;nmcli con up enp3s0** command as the **root** user to reload the configuration file and activate the connection for the modification to take effect. + +#### Configuring a Static Network + +The following uses the **enp4s0** network interface as an example to describe how to configure a static network by modifying the **ifcfg** file as the **root** user. The **ifcfg-enp4s0** file is generated in the **/etc/sysconfig/network-scripts/** directory. Modify the following parameters in the file: + +```text +TYPE=Ethernet +PROXY_METHOD=none +BROWSER_ONLY=no +BOOTPROTO=none +IPADDR=192.168.0.10 +PREFIX=24 +DEFROUTE=yes +IPV4_FAILURE_FATAL=no +IPV6INIT=yes +IPV6_AUTOCONF=yes +IPV6_DEFROUTE=yes +IPV6_FAILURE_FATAL=no +IPV6_ADDR_GEN_MODE=stable-privacy +NAME=enp4s0static +UUID=08c3a30e-c5e2-4d7b-831f-26c3cdc29293 +DEVICE=enp4s0 +ONBOOT=yes +``` + +#### Configuring a Dynamic Network + +The following uses the **em1** network interface as an example to describe how to configure a dynamic network by modifying the **ifcfg** file. The **ifcfg-em1** file is generated in the **/etc/sysconfig/network-scripts/** directory. Modify the following parameters in the file: + +```text +DEVICE=em1 +BOOTPROTO=dhcp +ONBOOT=yes +``` + +To configure an interface to send different host names to the DHCP server, add the following content to the **ifcfg** file: + +```text +DHCP_HOSTNAME=hostname +``` + +To configure an interface to ignore the routes sent by the DHCP server to prevent network services from updating the /etc/resolv.conf file using the DNS server received from the DHCP server, add the following content to the **ifcfg** file: + +```text +PEERDNS=no +``` + +To configure an interface to use a specific DNS server, set the **PEERDNS** parameter to **no** and add the following content to the **ifcfg** file: + +```text +DNS1=ip-address +DNS2=ip-address +``` + +**ip-address** is the IP address of the DNS server. This allows the network service to update the **/etc/resolv.conf** file using the specified DNS server. + +#### Default Gateway Configuration + +When determining the default gateway, parse the **/etc/sysconfig/network** file and then the **ifcfg** file, and uses the value of **GATEWAY** that is read last as the default route in the routing table. + +In a dynamic network environment, when the NetworkManager is used to manage hosts, you are advised to set the default gateway to DHCP assignment. + +## Configuring a Host Name + +### Introduction + +There are three types of host names: **static**, **transient**, and **pretty**. + +- **static**: Static host name, which can be set by users and saved in the **/etc/hostname** file. +- **transient**: Dynamic host name, which is maintained by the kernel. The initial value is a static host name. The default value is **localhost**. The value can be changed when the DHCP or mDNS server is running. +- **pretty**: Flexible host name, which can be set in any form \(including special characters/blanks\). Static and transient host names are subject to the general domain name restrictions. + +> [!NOTE]NOTE +> Static and transient host names can contain only letters \(a to z and A to Z\), digits \(0 to 9\), hyphens \(-\), and periods \(.\). The host names cannot start or end with a period \(.\) or contain two consecutive periods \(.\). The host name can contain a maximum of 64 characters. + +### Configuring a Host Name by Running the hostnamectl Command + +#### Viewing All Host Names + +Run the following command to view the current host name: + +```shell +hostnamectl status +``` + +> [!NOTE]NOTE +> If no option is specified in the command, the **status** option is used by default. + +#### Setting All Host Names + +Run the following command as the **root** user to set all host names: + +```shell +hostnamectl set-hostname name +``` + +#### Setting a Specific Host Name + +Run the following command as the **root** user to set a specific host name: + +```shell +hostnamectl set-hostname name [option...] +``` + +The option may be one or more of **\-\-pretty**, **\-\-static**, and **\-\-transient**. + +If **\-\-static** or **\-\-transient** is used together with **\-\-pretty**, the host names of the **static** or **transient** type will be simplified to the host names of the **pretty** type with spaces replaced with hyphens \(-\) and special characters deleted. + +When setting a host name of the **pretty** type, use double quotation marks if the host name contains spaces or single quotation marks. An example is as follows: + +```shell +hostnamectl set-hostname "Stephen's notebook" --pretty +``` + +#### Clearing a Specific Host Name + +To clear a specific host name and restore it to the default format, run the following command as the **root** user: + +```shell +hostnamectl set-hostname "" [option...] +``` + +In the preceding command, **""** is a blank character string, and the _option_ may be one or more of **\-\-pretty**, **\-\-static**, and **\-\-transient**. + +#### Remotely Changing a Host Name + +To change the host name in a remote system, run the **hostnamectl** command as the **root** user with the **-H** or **\-\-host** option. + +```shell +hostnamectl set-hostname -H [username]@hostname new_hostname +``` + +In the preceding command, _hostname_ indicates the name of the remote host to be configured, _username_ indicates the user-defined name, and _new\_hostname_ indicates the new host name. **hostnamectl** is used to connect to the remote system through SSH. + +### Configuring a Host Name by Running the nmcli Command + +To query a static host name, run the following command: + +```shell +nmcli general hostname +``` + +To name a static host as **host-server**, run the following command as **root** user: + +```shell +nmcli general hostname host-server +``` + +To enable the system to detect the change of the static host name, run the following command as the **root** user to restart the hostnamed service: + +```shell +systemctl restart systemd-hostnamed +``` + +## Configuring Network Bonding + +### Running the nmcli Command + +- To create a bond named **mybond0**, run the following command: + + ```shell + nmcli con add type bond con-name mybond0 ifname mybond0 mode active-backup + ``` + +- To add a slave interface, run the following command: + + ```shell + nmcli con add type bond-slave ifname enp3s0 master mybond0 + ``` + + To add another slave interface, repeat the preceding command with the new interface name: + + ```shell + $ nmcli con add type bond-slave ifname enp4s0 master mybond0 + Connection 'bond-slave-enp4s0' (05e56afc-b953-41a9-b3f9-0791eb49f7d3) successfully added. + ``` + +- To enable a bond, run the following command to enable the slave interface first: + + ```shell + $ nmcli con up bond-slave-enp3s0 + Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/14) + ``` + + ```shell + $ nmcli con up bond-slave-enp4s0 + Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/15) + ``` + + Then, run the following command to enable the bond: + + ```shell + $ nmcli con up mybond0 + Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/16) + ``` + +### Configuring Network Bonding by Using a Command Line + +#### Checking Whether the Bonding Kernel Module Is Installed + +By default, the bonding kernel module is loaded. To load this module, run the following command as the **root** user: + +```shell +modprobe --first-time bonding +``` + +Run the following command as the **root** user to display the information about the module: + +```shell +modinfo bonding +``` + +For more commands, run the modprobe \-\-help command as the **root** user. + +#### Creating a Channel Bonding Interface + +To create a channel bonding interface, you can create a file named **ifcfg-bondN** in the **/etc/sysconfig/network-scripts/** directory as the **root** user \(replacing N with the actual interface number, for example, 0\). + +Write the corresponding content to the configuration file according to the type of the interface to be bonded, for example, network interface. An example of the interface configuration file is as follows: + +```text +DEVICE=bond0 +NAME=bond0 +TYPE=Bond +BONDING_MASTER=yes +IPADDR=192.168.1.1 +PREFIX=24 +ONBOOT=yes +BOOTPROTO=none +BONDING_OPTS="bonding parameters separated by spaces" +``` + +#### Creating a Slave Interface + +After creating a channel bonding interface, you must add the **MASTER** and **SLAVE** instructions to the configuration file of the slave interface. + +For example, to bind the two network interfaces enp3s0 and enp4s0 in channel mode, the configuration files are as follows: + +```text +TYPE=Ethernet +NAME=bond-slave-enp3s0 +UUID=3b7601d1-b373-4fdf-a996-9d267d1cac40 +DEVICE=enp3s0 +ONBOOT=yes +MASTER=bond0 +SLAVE=yes +``` + +```text +TYPE=Ethernet +NAME=bond-slave-enp4s0 +UUID=00f0482c-824f-478f-9479-abf947f01c4a +DEVICE=enp4s0 +ONBOOT=yes +MASTER=bond0 +SLAVE=yes +``` + +#### Activating Channel Bonding + +To activate channel bonding, you need to enable all the slave interfaces. Run the following command as the **root** user: + +```shell +$ ifup enp3s0 +Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/7) +``` + +```shell +$ ifup enp4s0 +Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/8) +``` + +> [!NOTE]NOTE +> If an interface is in **up** state, run the **ifdown** _enp3s0_ command to change the state to **down**. In the command, _enp3s0_ indicates the actual NIC name. + +After that, enable all the slave interfaces to enable the bonding \(do not set them to **Down**\). + +To enable the NetworkManager to detect the modifications made by the system, run the following command as the **root** user after each modification: + +```shell +nmcli con load /etc/sysconfig/network-scripts/ifcfg-device +``` + +Run the following command as the **root** user to check the status of the bonded interface: + +```shell +$ ip link show + +1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 + link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 +2: enp3s0: mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000 + link/ether 52:54:00:aa:ad:4a brd ff:ff:ff:ff:ff:ff +3: enp4s0: mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000 + link/ether 52:54:00:aa:da:e2 brd ff:ff:ff:ff:ff:ff +4: virbr0: mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000 + link/ether 86:a1:10:fb:ef:07 brd ff:ff:ff:ff:ff:ff +5: virbr0-nic: mtu 1500 qdisc fq_codel master virbr0 state DOWN mode DEFAULT group default qlen 1000 + link/ether 52:54:00:29:35:4c brd ff:ff:ff:ff:ff:ff +``` + +#### Creating Multiple Bondings + +The system creates a channel bonding interface for each bonding, including the **BONDING\_OPTS** instruction. This configuration method allows multiple bonded devices to use different configurations. Perform the following operations to create multiple channel bonding interfaces: + +- Create multiple **ifcfg-bondN** files that contain the **BONDING\_OPTS** instruction so that network scripts can create bonding interfaces as required. +- Create or edit the existing interface configuration file to be bonded, and add the **SLAVE** instruction. +- Use the MASTER instruction to assign the interface to be bonded, that is, the slave interface, to the channel bonding interface. + +The following is an example of the configuration file of a channel bonding interface: + +```text +DEVICE=bondN +NAME=bondN +TYPE=Bond +BONDING_MASTER=yes +IPADDR=192.168.1.1 +PREFIX=24 +ONBOOT=yes +BOOTPROTO=none +BONDING_OPTS="bonding parameters separated by spaces" +``` + +In this example, replace N with the number of the bonded interface. For example, to create two interfaces, you need to create two configuration files **ifcfg-bond0** and **ifcfg-bond1** with correct IP addresses. + +## IPv6 Differences \(vs IPv4\) + +### Restrictions + +- chrony supports global addresses but not link-local addresses. +- Firefox supports the access to the global address through HTTP or HTTPS, but does not support the access to the link-local address. + +### Configuration Description + +#### Setting the MTU of an Interface Device + +##### Overview + +In an IPv6 scenario, the minimum MTU value of the entire routing path is used as the PMTU value of the current link. The source end determines whether to fragment packets based on the PMTU value. Other devices on the entire path do not need to fragment packets. This reduces the load of intermediate routing devices. The minimum value of IPv6 PMTU is 1280. + +##### Setting the MTU of the Interface Device + +If the MTU of an interface configured with an IPv6 address is set to a value smaller than **1280** \(the minimum value of the IPv6 PMTU\), the IPv6 address of the interface will be deleted and cannot be added again. Therefore, in IPv6 scenarios, the MTU of the interface device must be greater than or equal to 1280. Run the following commands as the **root** user to view the details: + +```shell +$ ip addr show enp3s0 +3: enp3s0: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 + link/ether 52:54:00:62:xx:xx brd ff:ff:ff:ff:xx:xx + inet 10.41.125.236/16 brd 10.41.255.255 scope global noprefixroute dynamic enp3s0 + valid_lft 38663sec preferred_lft 38663sec + inet6 2001:222::2/64 scope global + valid_lft forever preferred_lft forever +``` + +```shell +$ ip link set dev enp3s0 mtu 1200 +$ ip addr show enp3s0 +3: enp3s0: mtu 1200 qdisc pfifo_fast state UP group default qlen 1000 + link/ether 52:54:00:62:xx:xx brd ff:ff:ff:ff:xx:xx + inet 10.41.125.236/16 brd 10.41.255.255 scope global noprefixroute dynamic enp3s0 + valid_lft 38642sec preferred_lft 38642sec +``` + +```shell +$ ip addr add 2001:222::2/64 dev enp3s0 +RTNETLINK answers: No buffer space available +``` + +```shell +$ ip link set dev enp3s0 mtu 1500 +$ ip addr show enp3s0 +3: enp3s0: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 + link/ether 52:54:00:62:xx:xx brd ff:ff:ff:ff:xx:xx + inet 10.41.125.236/16 brd 10.41.255.255 scope global noprefixroute dynamic enp3s0 + valid_lft 38538sec preferred_lft 38538sec +``` + +```shell +$ ip addr add 2001:222::2/64 dev enp3s0 +$ ip addr show enp3s0 +3: enp3s0: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 + link/ether 52:54:00:62:xx:xx brd ff:ff:ff:ff:xx:xx + inet 10.41.125.236/16 brd 10.41.255.255 scope global noprefixroute dynamic enp3s0 + valid_lft 38531sec preferred_lft 38531sec + inet6 2001:222::2/64 scope global + valid_lft forever preferred_lft forever +``` + +#### Stateful IPv6 Address Autoconfiguration + +##### Overview + +Both IPv6 and IPv4 addresses can be obtained through DHCP as the **root** user. There are configuration methods for IPv6 address: stateless autoconfiguration and stateful autoconfiguration. + +- Stateless autoconfiguration + + The DHCP server is not required for management. The device obtains the network prefix according to the router advertisement \(RA\), or the prefix of a link-local address is fixed to fe80::. The interface ID is automatically obtained based on the value of IPV6\_ADDR\_GEN\_MODE in the ifcfg file. + + 1. If the value of IPv6\_ADDR\_GEN\_MODE is stable-privacy, the device determines a random interface ID based on the device and network environment. + 2. If the value of IPv6\_ADDR\_GEN\_MODE is EUI64, the device determines the interface ID based on the device MAC address. + +- Stateful autoconfiguration: The DHCP server manages and leases IPv6 addresses from the DHCPv6 server base on the DHCPv6 protocol. + + In stateful autoconfiguration, the DHCPv6 server can classify clients based on the vendor class configured on the clients and assign IPv6 addresses in different address segments to different types of clients. In IPv4 scenarios, the client can use the -V option of the dhclient command to set the vendor-class-identifier field. The DHCP server classifies clients based on the vendor-class-identifier field in the configuration file. In IPv6 scenarios, if the same method is used to classify clients, the classification does not take effect. + + ```shell + dhclient -6 -V + ``` + + This is because DHCPv6 differs greatly from DHCP. The vendor-class-option in DHCPv6 replaces the vendor-class-identifier in DHCP. However, the -V option of dhclient cannot be set to vendor-class-option. + +##### Setting the vendor class for dhclient in Stateful IPv6 Address Autoconfiguration + +- On the client, add the setting of vendor class by using the configuration file. + + Client configuration file \(/etc/dhcp/dhclient6.conf\): The file location can be customized. You need to specify the configuration file using the dhclient -cf option. + + ```text + option dhcp6.vendor-class code 16 = {integer 32, integer 16, string}; + interface "enp3s0" { + send dhcp6.vendor-class ; + } + ``` + + > [!NOTE]NOTE + + - \: a 32-digit integer, indicating the enterprise ID. The enterprise is registered through the IANA. + - \: a 16-digit integer, indicating the length of the vendor class string. + - \: character string of the vendor class to be set, for example, HWHW. + + On the client: + + ```shell + dhclient -6 -cf /etc/dhcp/dhclient6.conf + ``` + +- The DHCPv6 server configuration file \(/etc/dhcp/dhcpd6.conf\) needs to be specified by the dhcpd -cf option. + + ```text + option dhcp6.vendor-class code 16 = {integer 32, integer 16, string}; + subnet6 fc00:4:12:ffff::/64 { + class "hw" { + match if substring ( option dhcp6.vendor-class, 6, 10 ) = "HWHW"; + } + pool6 { + allow members of "hw"; + range6 fc00:4:12:ffff::ff10 fc00:4:12:ffff::ff20; + } + pool6 { + allow unknown clients; + range6 fc00:4:12:ffff::100 fc00:4:12:ffff::120; + } + } + ``` + + > [!NOTE]NOTE + > In substring \(option dhcp6.vendor-class, 6, 10\), the start position of the substring is 6, because the substring contains four bytes of and two bytes of . The end position of the substring is 6+. In this example, the vendor class string is HWHW, and the length of the string is 4. Therefore, the end position of the substring is 6 + 4 = 10. You can specify and as required. + + On the server: + + ```shell + dhcpd -6 -cf /etc/dhcp/dhcpd6.conf + ``` + +#### Kernel Supporting Socket-Related System Calls + +##### Overview + +The length of an IPv6 address is extended to 128 bits, indicating that there are sufficient IPv6 addresses for allocation. Compared with the IPv4 header, the IPv6 header is simplified, and the IPv6 automatic configuration function is enhanced. IPv6 addresses are classified into unicast addresses, multicast addresses, and anycast addresses. Common unicast addresses include link-local addresses, unique local addresses, and global addresses. As there are sufficient global IPv6 addresses, unique local addresses are not used. \(formerly known as site-local addresses, which were discarded in 2004.\) Currently, the mainstream unicast addresses are link-local address and global address. The current kernel supports socket system invoking. The link-local address and global address using unicast addresses are different. + +##### Differences Between the link-local Address and global Address During Socket Invoking + +RFC 2553: Basic Socket Interface Extensions for IPv6 defines the sockaddr\_in6 data structure as follows: + +```c +struct sockaddr_in6 { + uint8_t sin6_len; /* length of this struct */ + sa_family_t sin6_family; /* AF_INET6 */ + in_port_t sin6_port; /* transport layer port # */ + uint32_t sin6_flowinfo; /* IPv6 flow information */ + struct in6_addr sin6_addr; /* IPv6 address */ + uint32_t sin6_scope_id; /* set of interfaces for a scope */ +}; +``` + +> [!NOTE]NOTE +> sin6\_scope\_id: a 32-bit integer. For the link-local address, it identifies the index of the specified interface. For the link-range sin6\_addr, it identifies the index of the specified interface. For the site-range sin6\_addr, it is used as the site identifier \(the site-local address has been discarded\). + +When the link-local address is used for socket communication, the interface index corresponding to the address needs to be specified when the destination address is constructed. Generally, you can use the if\_nametoindex function to convert an interface name into an interface index number. Details are as follows: + +```c +int port = 1234; +int sk_fd; +int iff_index = 0; +char iff_name[100] = "enp3s0"; +char * ll_addr[100] = "fe80::123:456:789"; +struct sockaddr_in6 server_addr; + +memset(&server_addr,0,sizeof(structsockaddr_in6)); +iff_index=if_nametoindex(iff_name); + +server_addr.sin6_family=AF_INET6; +server_addr.sin6_port=htons(port); +server_addr.sin6_scope_id=iff_index; +inet_pton(AF_INET6, ll_addr, &(server_addr.sin6_addr)); + +sk_fd=socket(AF_INET6, SOCK_STREAM, IPPROTO_TCP); +connect(sk_fd, (struct sockaddr *)&server_addr, sizeof(struct sockaddr_in6)); +``` + +#### Persistency Configuration of the IPv4 dhclient Daemon Process + +##### Overview + +When the NetworkManager service is used to manage network services, if the ifcfg- configuration file of an interface is configured to obtain an IP address in DHCP mode, the NetworkManager service starts the dhclient daemon process to obtain an IP address from the DHCP server. + +The dhclient provides the -1 option to determine whether the dhclient process persistently attempts to request an IP address or exits after the request times out before receiving a response from the DHCP server. For the IPv4 dhclient daemon process, you can set PERSISTENT\_DHCLIENT in the ifcfg- configuration file to determine whether to set the persistence of the IPv4 dhclient process. + +##### Restrictions + +1. If the ongoing dhclient process is killed, the network service cannot automatically start it. Therefore, you need to ensure the reliability. +2. If PERSISTENT\_DHCLIENT is configured, ensure that the corresponding DHCP server exists. If no DHCP server is available when the network service is started and the dhclient process continuously attempts to send request packets but does not receive any response, the network service is suspended until the network service times out. The network service starts the IPv4 dhclient processes of multiple NICs in serial mode. If persistency is configured for a NIC but the DHCP server is not ready, the network service will be suspended when obtaining an IPv4 address for the NIC. As a result, the NIC cannot obtain an IPv4 or IPv6 address. + +The preceding restrictions apply to special scenarios. You need to ensure reliability. + +##### Configuration Differences Between IPv4 DHCP and IPv6 DHCPv6 + +You can configure the ifcfg- parameter on an interface to enable IPv4 and IPv6 to dynamically obtain IP addresses using DHCP or DHCPv6. The configuration is as follows: + +```text +BOOTPROTO=none|bootp|dhcp +DHCPV6C=yes|no +PERSISTENT_DHCLIENT=yes|no|1|0 +``` + +- BOOTPROTO: **none** indicates that an IPv4 address is statically configured. **bootp\|dhcp** enables DHCP dhclient to dynamically obtain an IPv4 address. +- DHCPV6C: **no** indicates that an IPv6 address is statically configured, and **yes** indicates that the DHCPv6 dhclient is enabled to dynamically obtain the IPv6 address. +- PERSISTENT\_DHCLIENT: **no\|0** indicates that the IPv4 dhclient process is configured as nonpersistent. If the dhclient sends a request packet to the DHCP server but does not receive any response, the dhclient exits after a period of time and the exit value is 2. **yes\|1** indicates that the IPv4 dhclient process is configured to be persistent. The dhclient process repeatedly sends request packets to the DHCP server. **If PERSISTENT\_DHCLIENT is not configured, dhclient of IPv4 is set to yes\|1 by default.** + + > [!NOTE]NOTE + > The PERSISTENT\_DHCLIENT configuration takes effect only for IPv4 and does not take effect for IPv6-related dhclient -6 processes. By default, the persistence configuration is not performed for IPv6. + +#### Differences Between IPv4 and IPv6 Configuration Using the iproute Command + +##### Overview + +IPv4 and IPv6 are two different protocol standards. Therefore, the iproute commands are different in usage. This section describes the differences between IPv4 and IPv6 commands in the iproute package. + +To run the iproute commands, you must have the root permission. + +##### Lifecycle of an IPv6 Address + + + + + + + + + + + + + + + + + + + +

IPv6 status

+

Description

+

tentative

+

Temporary state: The newly added address is still in the DAD process.

+

preferred

+

Preferred state: The DAD process is complete, but no NA packet is received, indicating that the address does not conflict.

+

deprecated

+

Deprecated state: An address has a validity period (valid_lft or preferred_lft). After preferred_lft expires, the address changes to the deprecated state.

+

The address in this state cannot be used to create a new connection, but the original connection can still be used.

+

invalid

+

Invalid state: If the lease renewal fails after the preferred_lft time expires, the address status is set to invalid after the valid_lft time expires, indicating that the address cannot be used again.

+
+ +Remarks: + +- preferred\_lft: preferred lifetime. The preferred\_lft address has not expired and can be used for normal communication. If there are multiple preferred addresses, the address is selected based on the kernel mechanism. +- valid\_lft: valid lifetime. The address cannot be used for creating new connections within the period of \[preferred\_lft, valid\_lft\]. The existing connections are still valid. + +##### Command ip link + +The commands are as follows: + +```shell +ip link set IFNAME mtu MTU +``` + +The minimum PMTU of IPv6 is 1280. If the MTU is set to a value smaller than 1280, IPv6 addresses will be lost. Other devices cannot ping the IPv6 address. + +##### Command ip addr + +1. The commands are as follows: + + ```shell + ip [-6] addr add IFADDR dev IFNAME + ``` + + You can choose to add the -6 option or not to add the IPv6 address. The ip addr command determines whether the address is an IPv4 address or an IPv6 address based on the address type. + + If the -6 option is specified but IFADDR is an IPv4 address, an error message is returned. + +2. The commands are as follows: + + ```shell + ip [-6] addr add IFADDR dev IFNAME [home|nodad] + ``` + + \[home\|nodad\] is valid only for IPv6 addresses. + + - home: specifies the home address defined in RFC 6275. \(This address is obtained by the mobile node from the home link, and is a permanent address of the mobile node. If the mobile node remains in the same home link, communication between various entities is performed normally.\) + - nodad: indicates that DAD is not performed when this IPv6 address is added. \(RFC 4862\) If multiple interfaces on a device are configured with the same IPv6 address through nodad, the IPv6 address is used in the interface sequence. An IPv6 address with both nodad and non-nodad cannot be added the same interface because the two IP addresses are the same. Otherwise, the message "RTNETLINK answers: File exists" is displayed. + +3. The commands are as follows: + + ```shell + ip [-6] addr del IFADDR dev IFNAME + ``` + + You can choose to add the -6 option or not to delete an IPv6 address. The ip addr del command determines whether an IPv4 address or an IPv6 address is used based on the address type. + +4. The commands are as follows: + + ```shell + ip [-6] addr show dev IFNAME [tentative|-tentative|deprecated|-deprecated|dadfailed|-dadfailed|temporary] + ``` + + - If the -6 option is not specified, both IPv4 and IPv6 addresses are displayed. If the -6 option is specified, only IPv6 addresses are displayed. + - \[tentative\|-tentative\|deprecated\|-deprecated\|dadfailed\|-dadfailed\|temporary\]. These options are only for IPv6. You can filter and view addresses based on the IPv6 address status. + 1. tentative: \(only for IPv6\) lists only the addresses that have not passed duplicate address detection \(DAD\). + 2. -tentative: \(only for IPv6\) lists only the addresses that are not in the DAD process. + 3. deprecated: \(only for IPv6\) lists only the deprecated addresses. + 4. -deprecated: \(only for IPv6\) lists only the addresses that are not deprecated. + 5. dadfailed: \(only for IPv6\) lists only the addresses that fail the DAD. + 6. -dadfailed: \(only for IPv6\) lists only the addresses that do not encounter DAD failures. + 7. temporary: \(only for IPv6\) lists only the temporary addresses. + +##### Command ip route + +1. The commands are as follows: + + ```shell + ip [-6] route add ROUTE [mtu lock MTU] + ``` + + - -6 option: You can add the -6 option or not when adding an IPv6 route. The ip route command determines whether an IPv4 or IPv6 address is used based on the address type. + + - mtu lock MTU: specifies the MTU of the locked route. If the MTU is not locked, the MTU value may be changed by the kernel during the PMTUD process. If the MTU is locked, PMTUD is not attempted. All IPv4 packets are not set with the DF bit and IPv6 packets are segmented based on the MTU. + +2. The commands are as follows: + + ```shell + ip [-6] route del ROUTE + ``` + + You can choose whether to add the -6 option when deleting an IPv6 route. The ip route command determines whether an IPv4 address or an IPv6 address is used based on the address type. + +##### Command ip rule + +1. The commands are as follows: + + ```shell + ip [-6] rule list + ``` + + -6 option: If the -6 option is set, IPv6 policy-based routes are printed. If the -6 option is not set, IPv4 policy-based routes are printed. Therefore, you need to configure the -6 option according to the specific protocol type. + +2. The commands are as follows: + + ```shell + ip [-6] rule [add|del] [from|to] ADDR table TABLE pref PREF + ``` + + -6 option: IPv6-related policy routing entries need to be configured with the -6 option. Otherwise, the error message "Error: Invalid source address." is displayed. Accordingly, the -6 option cannot be set for IPv4-related policy routing entries. Otherwise, the error message "Error: Invalid source address." is displayed. + +#### Configuration Differences of the NetworkManager Service + +##### Overview + +The NetworkManager service uses the ifup/ifdown logical interface definition to perform advanced network settings. Most of the parameters are set in the /etc/sysconfig/network and /etc/sysconfig/network-scripts/ifcfg- configuration files. The former is a global setting, and the latter is a setting of a specified NIC. When the two settings conflict, the latter takes effect. + +##### Configuration Differences + +The configuration differences in /etc/sysconfig/network are as follows: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

IPv4

+

IPv6

+

Description

+

N/A

+

IPV6FORWARDING=yes|no

+

IPv6 forwarding. By default, IPv6 packets are not forwarded.

+

N/A

+

IPV6_AUTOCONF=yes|no

+

If IPv6 forwarding is enabled, the value is no. Otherwise, the value is yes.

+

N/A

+

IPV6_ROUTER=yes|no

+

If IPv6 forwarding is enabled, the value is yes. Otherwise, the value is no.

+

N/A

+

IPV6_AUTOTUNNEL=yes|no

+

Indicates the automatic tunnel mode. The default value is no.

+

GATEWAY

+

IPV6_DEFAULTGW=<IPv6 address[%interface]> (optional)

+

Indicates the default gateway in IPv6.

+

N/A

+

IPV6_DEFAULTDEV=<interface> (optional)

+

Specifies the default forwarding NIC.

+

N/A

+

IPV6_RADVD_PIDFILE=<pid-file> (optional)

+

The default path of ipv6_radvd_pid is /var/run/radvd/radvd.pid.

+

N/A

+

IPV6_RADVD_TRIGGER_ACTION=startstop|reload|restart|SIGHUP (optional)

+

Default radvd trigger action.

+
+ +The differences in /etc/sysconfig/network-scripts/ifcfg- are as follows: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

IPv4

+

IPv6

+

Description

+

IPADDR

+

IPV6ADDR=<IPv6 address>[/<prefix length>]

+

indicates the IP address.

+

PREFIX

+

N/A

+

The network prefix, network alias, and PPP are invalid. The priority is higher than that of NETMASK.

+

NETMASK

+

N/A

+

Indicates the subnet mask. It is used only for the alias and PPP.

+

GATEWAY

+

IPV6_DEFAULTGW=<IPv6 address[%interface]> (optional)

+

Default gateway

+

MTU

+

IPV6_MTU=<MTU of link> (optional)

+

Default MTU

+

IPV4_FAILURE_FATAL=yes|no

+

IPV6_FAILURE_FATAL

+

The default value is no. If this parameter is set to yes, ifup-eth exits when dhclient fails.

+

N/A

+

IPV6_PRIVACY=rfc3041

+

Disabled by default.

+

N/A

+

IPV6INIT=yes|no

+

IPv6 is enabled by default.

+

N/A

+

IPV6FORWARDING=yes|no

+

This function is disabled by default and has been discarded.

+
+ +### FAQs + +#### The iscsi-initiator-utils Does Not Support the fe80 IPv6 Address + +##### Symptom + +When a client uses an IPv6 address to log in to the iSCSI server, run the iscsiadm -m node -p ipv6address -l command. If the global address is used, replace ipv6address in the command example with the global address. However, the link-local address \(IPv6 address starting with fe80\) cannot be used because the current mechanism of iscsi-initiator-utils does not support the link-local address to log in to the iSCSI server. + +##### Possible Cause + +If you log in to the system using the iscsiadm -m node -p fe80::xxxx -l format, a login timeout error is returned. This is because you must specify an interface when using the link-local address. Otherwise, the iscsi\_io\_tcp\_connect function fails to invoke the connect function, and the standard error code 22 is generated. + +If you use the iscsiadm -m node -p fe80::xxxx%enp3s0 -l format for login, the iscsi\_addr\_match function will compare the address fe80::xxxx%enp3s0 with the address fe80::xxxx in the node information returned by the server. The comparison result does not match, causing the login failure. + +Therefore, **the current mechanism of iscsi-initiator-utils does not support login to the iSCSI server using a link-local address.** + +#### The IPv6 Address Is Lost After the NIC Is Down + +##### Symptom + +Run the ip link down+up NIC or ifconfig down+up NIC command to disable the NIC and then enable it to go online. Check the IP address configured on the NIC. It is found that the IPv4 address is not lost but the configured IPv6 address is lost. + +##### Possible Cause + +According to the processing logic in the kernel, if the NIC is set to the down state, all IPv4 and IPv6 addresses will be cleared. After the NIC is set to the up state, the IPv4 address is automatically restored, and the automatically configured IPv6 link-local address on the NIC is also restored. However, other IPv6 addresses are lost by default. To retain these IPv6 addresses, run the **sysctl -w net.ipv6.conf.\< _NIC name_ \>.keep\_addr\_on\_down=1** command. + +#### Taking a Long Time to Add or Delete an IPv6 Address for a Bond Interface with Multiple IPv6 Addresses + +##### Symptom + +When users run the following command to add or delete \(including flush\) an IPv6 address, the waiting time increases linearly along with the number of IPv6 addresses configured on a bond interface. **X** is the least significant 16 bits that dynamically change. For example, it takes about five minutes to add 3000 IPv6 address to or delete them from a bond interface that already has four physical NICs using a single thread, while for a common physical NIC, it takes less than 10 seconds. + +```shell +ip a add/del 192:168::18:X/64 dev DEVICE +``` + +##### Possible Cause + +When an IPv6 address is added to a bond interface, the IPv6 multicast address is generated and synchronized to all physical NICs. The time required increases with the number of IPv6 addresses. As a result, it takes a too long time. + +##### Solution + +The IPv6 multicast address is generated by combining the least significant 24 bits of the IPv6 address and 33-33-ff. If there are too many multicast addresses, it takes a long time to add or delete the address. If there are a few multicast addresses, the time required is not affected. + +It is recommended that you set the least significant 24 bits of the IPv6 address to be the same as the most significant 24 bits of the IPv6 address. In this way, a single NIC can communicate with external devices using only one IP address in a network segment. + +#### Rsyslog Log Transmission Is Delayed in the Scenario Where Both IPv4 and IPv6 Are Used + +##### Symptom + +When both IPv4 and IPv6 addresses are configured in the configuration file of the rsyslog client and the port configurations are the same, there is a possibility that log output is delayed when the server collects logs. + +##### Possible Cause + +The delay is caused by the buffer queue mechanism of rsyslog. By default, rsyslog writes data to a file only when the number of buffer queues reaches a specified value. + +##### Solution + +You can disable the buffer queue mechanism by configuring the Direct mode as the **root** user. Add the following information at the beginning of the new remote transmission configuration file in the /etc/rsyslog.d directory on the rsyslog remote transmission server: + +```text +$ActionQueueType Direct +$MainMsgQueueType Direct +``` + +> [!NOTE]NOTE + +- In direct mode, the queue size is reduced by 1. Therefore, one log is reserved in the queue for the next log output. +- The direct mode degrades the rsyslog performance of the server. diff --git a/docs/en/server/network/network_config/public_sys-resources/icon-note.gif b/docs/en/server/network/network_config/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/server/network/network_config/public_sys-resources/icon-note.gif differ diff --git a/docs/en/server/performance/cpu_optimization/_toc.yaml b/docs/en/server/performance/cpu_optimization/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..75f72d929f4edd6d0aa4154c1e9c5b442479e6ad --- /dev/null +++ b/docs/en/server/performance/cpu_optimization/_toc.yaml @@ -0,0 +1,4 @@ +label: CPU Tuning +sections: + - href: ./sysboost/_toc.yaml + - href: ./kae/_toc.yaml diff --git a/docs/en/server/performance/cpu_optimization/kae/_toc.yaml b/docs/en/server/performance/cpu_optimization/kae/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..33732c2b76dce9c00cdce4d6ef20d98f9b41362d --- /dev/null +++ b/docs/en/server/performance/cpu_optimization/kae/_toc.yaml @@ -0,0 +1,6 @@ +label: Using the Kunpeng Accelerator Engine (KAE) +isManual: true +href: ./using-the-kae.md +description: >- + The KAE acceleration engine minimizes processor usage while enhancing its + efficiency. diff --git a/docs/en/server/performance/cpu_optimization/kae/figures/en-us_image_0231143189.png b/docs/en/server/performance/cpu_optimization/kae/figures/en-us_image_0231143189.png new file mode 100644 index 0000000000000000000000000000000000000000..7656f3aa5f5907f1e9f981c0cb5d44d4fcb84ef3 Binary files /dev/null and b/docs/en/server/performance/cpu_optimization/kae/figures/en-us_image_0231143189.png differ diff --git a/docs/en/server/performance/cpu_optimization/kae/figures/en-us_image_0231143191.png b/docs/en/server/performance/cpu_optimization/kae/figures/en-us_image_0231143191.png new file mode 100644 index 0000000000000000000000000000000000000000..a82d1bcb2b719e3a372f63ae099cb5d52a93b536 Binary files /dev/null and b/docs/en/server/performance/cpu_optimization/kae/figures/en-us_image_0231143191.png differ diff --git a/docs/en/server/performance/cpu_optimization/kae/figures/en-us_image_0231143193.png b/docs/en/server/performance/cpu_optimization/kae/figures/en-us_image_0231143193.png new file mode 100644 index 0000000000000000000000000000000000000000..94614045bddb0871b44d2f6603402f914871ad61 Binary files /dev/null and b/docs/en/server/performance/cpu_optimization/kae/figures/en-us_image_0231143193.png differ diff --git a/docs/en/server/performance/cpu_optimization/kae/figures/en-us_image_0231143195.png b/docs/en/server/performance/cpu_optimization/kae/figures/en-us_image_0231143195.png new file mode 100644 index 0000000000000000000000000000000000000000..05011dbabe2d245c37ec68de646851bf955a2361 Binary files /dev/null and b/docs/en/server/performance/cpu_optimization/kae/figures/en-us_image_0231143195.png differ diff --git a/docs/en/server/performance/cpu_optimization/kae/figures/en-us_image_0231143196.png b/docs/en/server/performance/cpu_optimization/kae/figures/en-us_image_0231143196.png new file mode 100644 index 0000000000000000000000000000000000000000..9bdbac969920af77721980804bd1c5433bea5bc9 Binary files /dev/null and b/docs/en/server/performance/cpu_optimization/kae/figures/en-us_image_0231143196.png differ diff --git a/docs/en/server/performance/cpu_optimization/kae/figures/en-us_image_0231143197.png b/docs/en/server/performance/cpu_optimization/kae/figures/en-us_image_0231143197.png new file mode 100644 index 0000000000000000000000000000000000000000..5ea4eec4002374096d8ac18eb973ed3bf874b632 Binary files /dev/null and b/docs/en/server/performance/cpu_optimization/kae/figures/en-us_image_0231143197.png differ diff --git a/docs/en/server/performance/cpu_optimization/kae/figures/en-us_image_0231143198.png b/docs/en/server/performance/cpu_optimization/kae/figures/en-us_image_0231143198.png new file mode 100644 index 0000000000000000000000000000000000000000..7d6360c150495d204da4b069e6dc62677580888f Binary files /dev/null and b/docs/en/server/performance/cpu_optimization/kae/figures/en-us_image_0231143198.png differ diff --git a/docs/en/server/performance/cpu_optimization/kae/public_sys-resources/icon-note.gif b/docs/en/server/performance/cpu_optimization/kae/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/server/performance/cpu_optimization/kae/public_sys-resources/icon-note.gif differ diff --git a/docs/en/server/performance/cpu_optimization/kae/using-the-kae.md b/docs/en/server/performance/cpu_optimization/kae/using-the-kae.md new file mode 100644 index 0000000000000000000000000000000000000000..eeb067bd0588cba59d1b1ce81138e17f3a8e2c21 --- /dev/null +++ b/docs/en/server/performance/cpu_optimization/kae/using-the-kae.md @@ -0,0 +1,724 @@ +# Using the Kunpeng Accelerator Engine (KAE) + +## Overview + +Kunpeng Accelerator Engine \(KAE\) is a software acceleration library of openEuler, which provides hardware acceleration engine function on the Kunpeng 920 processor. It supports symmetric encryption, asymmetric encryption, and digital signature. It is ideal for accelerating SSL/TLS applications, reducing processor consumption and improving processor efficiency. In addition, users can quickly migrate existing services through the standard OpenSSL interface. + +The KAE supports the following algorithms: + +- Digest algorithm SM3, which supports asynchronous mode. +- Symmetric encryption algorithm SM4, which supports asynchronous, CTR, XTS, and CBC modes. +- Symmetric encryption algorithm AES, which supports asynchronous, ECB, CTR, XTS, and CBC modes. +- Asymmetric algorithm RSA, which supports asynchronous mode and key sizes 1024, 2048, 3072, and 4096. +- Key negotiation algorithm DH, which supports asynchronous mode and key sizes 768, 1024, 1536, 2048, 3072, and 4096. + +## Application Scenarios + +The KAE applies to the following scenarios, as shown in [Table 1](#table11915824163418). + +**Table 1** Application scenarios + + + + + + + + + + + + + + + + + + + +

Scenario

+

Data

+

Big data

+

Stream data

+

Data encryption

+

Block data

+

Intelligent security protection

+

Video stream data

+

Web service

+

Handshake connections

+
+ +## Installing, Running, and Uninstalling the KAE + +### Installing the Accelerator Software Packages + +#### Preparing for Installation + +##### Environment Requirements + +- The accelerator engine is enabled on TaiShan 200 servers. + +> [!NOTE]NOTE + +- You need to import the accelerator license. For details, see section "License Management" in the [TaiShan Rack Server iBMC \(V500 or Later\) User Guide](https://support.huawei.com/enterprise/en/doc/EDOC1100121685/426cffd9?idPath=7919749%257C9856522%257C21782478%257C8060757). +- If the accelerator is used in the physical machine scenario, the SMMU must be disabled. For details, see the [TaiShan 200 Server BIOS Parameter Reference](https://support.huawei.com/enterprise/en/doc/EDOC1100088647). + +- CPU: Kunpeng 920 +- OS: openEuler-21.09-aarch64-dvd.iso + +##### KAE Software Description + +**Table 2** RPM software packages of the KAE + + + + + + + + + + + + + + + + +

Software Package

+

Description

+

kae_driver-version_num-1.OS_type.aarch64.rpm

+

Accelerator driver, including the uacce.ko, hisi_qm.ko, hisi_sec2.ko, and hisi_hpre.ko kernel modules.

+

Algorithms supported: SM3, SM4, AES, RSA, and DH.

+

libwd-version_num-1.OS_type.aarch64.rpm

+

Coverage: libwd.so dynamic link library

+

It provides interfaces for the KAE.

+

libkae-version_num-1.OS_type.aarch64.rpm

+

Dependency: libwd RPM package

+

Coverage: libkae.so dynamic link library

+

Algorithms supported: SM3, SM4, AES, RSA, and DH.

+
+ +#### Installing the Accelerator Software Package + +##### Prerequisites + +- The remote SSH login tool has been installed on the local PC. +- The openEuler OS has been installed. +- The RPM tool is running properly. +- OpenSSL 1.1.1a or a later version has been installed. + + You can run the following commands to query the version number of OpenSSL: + + - openssl version + +##### Procedure + +1. Log in to the openEuler OS CLI as user **root**. +2. Create a directory for storing accelerator engine software packages. +3. Use SSH to copy all accelerator engine software packages to the created directory. +4. In the directory, run the **rpm -ivh** command to install the accelerator engine software packages. + + > [!NOTE]NOTE + > Install the **libwd** package first because the **libkae** package installation depends on the **libwd** package. + + ```shell + rpm -ivh uacce*.rpm hisi*.rpm libwd-*.rpm libkae*.rpm + ``` + + ```text + Verifying... ################################# [100%] + Preparing... ################################# [100%] + checking installed modules + uacce modules start to install + Updating / installing... + 1:uacce-1.2.10-4.oe1 ################################# [ 14%] + uacce modules installed + 2:libwd-1.2.10-3.oe1 ################################# [ 29%] + 3:libkae-1.2.10-3.oe1 ################################# [ 43%] + checking installed modules + hisi_hpre modules start to install + 4:hisi_hpre-1.2.10-4.oe1 ################################# [ 57%] + hisi_hpre modules installed + checking installed modules + hisi_rde modules start to install + 5:hisi_rde-1.2.10-4.oe1 ################################# [ 71%] + hisi_rde modules installed + checking installed modules + hisi_sec2 modules start to install + 6:hisi_sec2-1.2.10-4.oe1 ################################# [ 86%] + hisi_sec2 modules installed + checking installed modules + hisi_zip modules start to install + 7:hisi_zip-1.2.10-4.oe1 ################################# [100%] + hisi_zip modules installed + ``` + +5. Run the **rpm -qa** command to check whether the accelerator software packages have been installed successfully. Run the **rpm -ql** command to check whether files in the software packages are correct. The following is an example: + + ```shell + rpm -qa|grep -E "hisi|uacce|libwd|libkae" + ``` + + ```text + hisi_rde-1.2.10-4.oe1.aarch64 + hisi_sec2-1.2.10-4.oe1.aarch64 + libkae-1.2.10-3.oe1.aarch64 + hisi_hpre-1.2.10-4.oe1.aarch64 + uacce-1.2.10-4.oe1.aarch64 + libwd-1.2.10-3.oe1.aarch64 + hisi_zip-1.2.10-4.oe1.aarch64 + ``` + + ```shell + rpm -ql uacce hisi* libwd* libkae + ``` + + ```text + /lib/modules/4.19.90-2003.4.0.0036.oe1.aarch64/extra/hisi_qm.ko + /lib/modules/4.19.90-2003.4.0.0036.oe1.aarch64/extra/uacce.ko + /etc/modprobe.d/hisi_hpre.conf + /lib/modules/4.19.90-2003.4.0.0036.oe1.aarch64/extra/hisi_hpre.ko + /etc/modprobe.d/hisi_rde.conf + /lib/modules/4.19.90-2003.4.0.0036.oe1.aarch64/extra/hisi_rde.ko + /etc/modprobe.d/hisi_sec2.conf + /lib/modules/4.19.90-2003.4.0.0036.oe1.aarch64/extra/hisi_sec2.ko + /etc/modprobe.d/hisi_zip.conf + /lib/modules/4.19.90-2003.4.0.0036.oe1.aarch64/extra/hisi_zip.ko + /usr/include/warpdrive/config.h + /usr/include/warpdrive/include/uacce.h + /usr/include/warpdrive/smm.h + /usr/include/warpdrive/wd.h + /usr/include/warpdrive/wd_bmm.h + /usr/include/warpdrive/wd_cipher.h + /usr/include/warpdrive/wd_comp.h + /usr/include/warpdrive/wd_dh.h + /usr/include/warpdrive/wd_digest.h + /usr/include/warpdrive/wd_rsa.h + /usr/lib64/libwd.so.1.2.10 + /usr/local/lib/engines-1.1/libkae.so.1.2.10 + ``` + +6. Restart the system or run commands to manually load the accelerator engine drivers to the kernel in sequence, and check whether the drivers are successfully loaded. + + ```shell + modprobe uacce + lsmod | grep uacce + modprobe hisi_qm + lsmod | grep hisi_qm + modprobe hisi_sec2 # Loads the hisi_sec2 driver to the kernel based on the configuration file in /etc/modprobe.d/hisi_sec2.conf. + modprobe hisi_hpre # Loads the hisi_hpre driver to the kernel based on the configuration file in /etc/modprobe.d/hisi_hpre.conf. + ``` + +##### Environment Variables Setup + +Run the following command to export the environment variables \(If you have specified the installation directory, set **/usr/local** to the actual one\): + +```shell +export OPENSSL_ENGINES=/usr/local/lib/engines-1.1 +``` + +##### Post-Installation Check + +Run the **rpm -qa** command to check whether the accelerator engine software packages are successfully installed. + +If the command output contains _software package name_**-**_version number_**-**, the software packages are successfully installed. The following is an example: + +```shell +rpm -qa|grep -E "hisi|uacce|libwd|libkae" +``` + +```text +hisi_rde-1.2.10-4.oe1.aarch64 +hisi_sec2-1.2.10-4.oe1.aarch64 +libkae-1.2.10-3.oe1.aarch64 +hisi_hpre-1.2.10-4.oe1.aarch64 +uacce-1.2.10-4.oe1.aarch64 +libwd-1.2.10-3.oe1.aarch64 +hisi_zip-1.2.10-4.oe1.aarch64 +``` + +#### Required Operations After Installation + +##### Testing the OpenSSL Accelerator Engine + +You can run the following commands to test some accelerator functions. + +- Use the OpenSSL software algorithm to test the RSA performance. + + ```shell + $ ./openssl speed -elapsed rsa2048 + ... + sign verify sign/s verify/s + rsa 2048 bits 0.001384s 0.000035s 724.1 28365.8. + ``` + +- Use the KAE to test the RSA performance. + + ```shell + $ ./openssl speed -elapsed -engine kae rsa2048 + .... + sign verify sign/s verify/s + rsa 2048 bits 0.000355s 0.000022s 2819.0 45478.4 + ``` + +> [!NOTE]NOTE +> After the KAE is used, the signature performance is improved from 724.1 sign/s to 2819 sign/s. + +- Use the OpenSSL software algorithm to test the asynchronous RSA performance. + + ```shell + $ ./openssl speed -elapsed -async_jobs 36 rsa2048 + .... + sign verify sign/s verify/s + rsa 2048 bits 0.001318s 0.000032s 735.7 28555 + ``` + +- Use the KAE to test the asynchronous RSA performance. + + ```shell + $ ./openssl speed -engine kae -elapsed -async_jobs 36 rsa2048 + .... + sign verify sign/s verify/s + rsa 2048 bits 0.000018s 0.000009s 54384.1 105317.0 + ``` + +> [!NOTE]NOTE +> After the KAE is used, the asynchronous RSA signature performance is improved from 735.7 sign/s to 54384.1 sign/s. + +- Use the OpenSSL software algorithm to test the performance of the SM4 CBC mode. + + ```shell + $ ./openssl speed -elapsed -evp sm4-cbc + You have chosen to measure elapsed time instead of user CPU time. + .... + Doing sm4-cbc for 3s on 10240 size blocks: 2196 sm4-cbc's in 3.00s .... + type 51200 bytes 102400 bytes1048576 bytes2097152 bytes4194304 bytes8388608 bytes + sm4-cbc 82312.53k 85196.80k 85284.18k 85000.85k 85284.18k 85261.26k + ``` + +- Use the KAE to test the SM4 CBC mode performance. + + ```shell + $ ./openssl speed -elapsed -engine kae -evp sm4-cbc + engine "kae" set. + You have chosen to measure elapsed time instead of user CPU time. + ... + Doing sm4-cbc for 3s on 1048576 size blocks: 11409 sm4-cbc's in 3.00s + ... + type 51200 bytes 102400 bytes1048576 bytes2097152 bytes4194304 bytes8388608 bytes + sm4-cbc 383317.33k 389427.20k 395313.15k 392954.73k 394264.58k 394264.58k + ``` + +> [!NOTE]NOTE +> After the KAE is used, the SM4 CBC mode performance is improved from 82312.53 kbit/s to 383317.33 kbit/s when the input data block size is 8 MB. + +- Use the OpenSSL software algorithm to test the SM3 mode performance. + + ```shell + $ ./openssl speed -elapsed -evp sm3 + You have chosen to measure elapsed time instead of user CPU time. + Doing sm3 for 3s on 102400 size blocks: 1536 sm3's in 3.00s + .... + type 51200 bytes 102400 bytes1048576 bytes2097152 bytes4194304 bytes8388608 bytes + sm3 50568.53k 52428.80k 52428.80k 52428.80k 52428.80k 52428.80k + ``` + +- Use the KAE to test the SM3 mode performance. + + ```shell + $ ./openssl speed -elapsed -engine kae -evp sm3 + engine "kae" set. + You have chosen to measure elapsed time instead of user CPU time. + Doing sm3 for 3s on 102400 size blocks: 19540 sm3's in 3.00s + .... + type 51200 bytes 102400 bytes 1048576 bytes 2097152 bytes 4194304 bytes 8388608 bytes + sm3 648243.20k 666965.33k 677030.57k 678778.20k 676681.05k 668292.44k + ``` + +> [!NOTE]NOTE +> After the KAE is used, the SM3 algorithm performance is improved from 52428.80 kbit/s to 668292.44 kbit/s when the input data block size is 8 MB. + +- Use the OpenSSL software algorithm to test the asynchronous performance of the AES algorithm in CBC mode. + + ```shell + $ ./openssl speed -elapsed -evp aes-128-cbc -async_jobs 4 + You have chosen to measure elapsed time instead of user CPU time. + Doing aes-128-cbc for 3s on 51200 size blocks: 65773 aes-128-cbc's in 3.00s + Doing aes-128-cbc for 3s on 102400 size blocks: 32910 aes-128-cbc's in 3.00s + .... + type 51200 bytes 102400 bytes1048576 bytes2097152 bytes4194304 bytes8388608 bytes + aes-128-cbc 1122525.87k 1123328.00k 1120578.22k 1121277.27k 1119879.17k 1115684.86k + ``` + +- Use the KEA engine to test the asynchronous performance of the AES algorithm in CBC mode. + + ```shell + $ ./openssl speed -elapsed -evp aes-128-cbc -async_jobs 4 -engine kae + engine "kae" set. + You have chosen to measure elapsed time instead of user CPU time. + Doing aes-128-cbc for 3s on 51200 size blocks: 219553 aes-128-cbc's in 3.00s + Doing aes-128-cbc for 3s on 102400 size blocks: 117093 aes-128-cbc's in 3.00s + .... + type 51200 bytes 102400 bytes1048576 bytes2097152 bytes4194304 bytes8388608 bytes + aes-128-cbc 3747037.87k 3996774.40k 1189085.18k 1196774.74k 1196979.11k 1199570.94k + ``` + +> [!NOTE]NOTE + +- The AES algorithm supports only asynchronous mode when the data length is 256 KB or less. +- After the KAE is used, the AES algorithm performance is improved from 1123328.00 kbit/s to 3996774.40 kbit/s when the input data block size is 100 KB. + +### Upgrading the Accelerator Software Packages + +#### Scenario + +You can run the **rpm -Uvh** command to upgrade the accelerator software. + +#### Procedure + +1. Download the latest accelerator engine software packages from the openEuler community. +2. Use SSH to log in to the Linux CLI as user **root**. +3. Save the downloaded software packages to a directory. +4. In the directory, run the **rpm -Uvh** command to upgrade the accelerator driver package and engine library package. The following is an example: + + The command and output are as follows: + + ![](./figures/en-us_image_0231143189.png) + + ![](./figures/en-us_image_0231143191.png) + +5. Run the **rpm -qa** command to check whether the upgrade is successful. Ensure that the queried version is the latest version. + + ![](./figures/en-us_image_0231143193.png) + + ![](./figures/en-us_image_0231143195.png) + +6. Restart the system or run the following commands to manually uninstall the drivers of the earlier version, load the drivers of the latest version, and check whether the new drivers are successfully loaded. + + ```shell + # Uninstall the existing drivers. + $ lsmod | grep uacce + uacce 262144 3 hisi_hpre,hisi_sec2,hisi_qm + $ + $ rmmod hisi_hpre + $ rmmod hisi_sec2 + $ rmmod hisi_qm + $ rmmod uacce + $ lsmod | grep uacce + $ + # Load the new drivers. + $ modprobe uacce + $ modprobe hisi_qm + $ modprobe hisi_sec2 # Loads the hisi_sec2 driver to the kernel based on the configuration file in /etc/modprobe.d/hisi_sec2.conf. + $ modprobe hisi_hpre # Loads the hisi_hpre driver to the kernel based on the configuration file in /etc/modprobe.d/hisi_hpre.conf. + $ lsmod | grep uacce + uacce 36864 3 hisi_sec2,hisi_qm,hisi_hpre + ``` + +### Uninstalling the Accelerator Software Packages + +#### Scenario + +You do not need the accelerator engine software or you want to install a new one. + +#### Procedure + +1. Use SSH to log in to the Linux CLI as user **root**. +2. Restart the system or run commands to manually uninstall the accelerator drivers loaded to the kernel, and check whether the drivers are successfully uninstalled. + + ```shell + # lsmod | grep uacce + uacce 36864 3 hisi_sec2,hisi_qm,hisi_hpre + # rmmod hisi_hpre + # rmmod hisi_sec2 + # rmmod hisi_qm + # rmmod uacce + # lsmod | grep uacce + # + ``` + +3. Run the **rpm -e** command to uninstall the accelerator engine software packages. The following is an example: + + > [!NOTE]NOTE + > Due to the dependency relationships, the **libkae** package must be uninstalled before the **libwd** package. + + ![](./figures/en-us_image_0231143196.png) + + ![](./figures/en-us_image_0231143197.png) + +4. Run the **rpm -qa \|grep** command to check whether the uninstallation is successful. + + ![](./figures/en-us_image_0231143198.png) + +## Querying Logs + +[Table 3](#table52821836) lists log information related to the accelerator engine. + +**Table 3** Log information + + + + + + + + + + + + + + + + +

Directory

+

File

+

Description

+

/var/log/

+

kae.log

+

By default, the log level of the OpenSSL engine log is error. To set the log level, perform the following procedure:

+
  1. Run export KAE_CONF_ENV=/var/log/.
  2. Create the kae.cnf file in /var/log/.
  3. In the kae.cnf file, configure the content as follows:

    [LogSection]

    +

    debug_level=error #Value: none, error, info, warning or debug

    +
+
NOTE:

In normal cases, you are advised not to enable the info or debug log level. Otherwise, the accelerator performance will deteriorate.

+
+

/var/log/

+

messages/syslog

+
  • Kernel logs are stored in the /var/log/messages directory.
+
NOTE:

Alternatively, you can run the dmesg > /var/log/dmesg.log command to collect driver and kernel logs.

+
+
+ +## Acceleration Engine Application + +> [!NOTE]NOTE +> If you have not purchased the engine license, you are advised not to use the KAE to invoke the corresponding algorithms. Otherwise, the performance of the OpenSSL encryption algorithm may be affected. + +### Example Code for the KAE + +```c +#include + +#include + +/* OpenSSL headers */ + +#include + +#include + +#include + +#include + +int main(int argc, char **argv) + +{ + + /* Initializing OpenSSL */ + + SSL_load_error_strings(); + + ERR_load_BIO_strings(); + + OpenSSL_add_all_algorithms(); + + /*You can use ENGINE_by_id Function to get the handle of the Huawei Accelerator Engine*/ + + ENGINE *e = ENGINE_by_id("kae"); + + /* Enable the accelerator asynchronization function. This parameter is optional. The value 0 indicates disabled, and the value 1 indicates enabled. The asynchronous function is enabled by default. */ + + ENGINE_ctrl_cmd_string(e, "KAE_CMD_ENABLE_ASYNC", "1", 0) + + ENGINE_init(e); + + RSA*rsa=RSA_new_method(e);#Specify the engine for RSA encryption and decryption. + + /*The user code*/ + + ...... + +; + + ENGINE_free(e); + +; + +} +``` + +### Usage of the KAE in the OpenSSL Configuration File openssl.cnf + +Create the **openssl.cnf** file and add the following configuration information to the file: + +```text +openssl_conf=openssl_def +[openssl_def] +engines=engine_section +[engine_section] +kae=kae_section +[kae_section] +engine_id=kae +dynamic_path=/usr/local/lib/engines-1.1/kae.so +KAE_CMD_ENABLE_ASYNC=1 #The value 0 indicates that the asynchronous function is disabled. The value 1 indicates that the asynchronous function is enabled. The asynchronous function is enabled by default. +default_algorithms=ALL +init=1 +``` + +Export the environment variable **OPENSSL\_CONF**. + +```shell +export OPENSSL_CONF=/home/app/openssl.cnf #Path for storing the openssl.cnf file +``` + +The following is an example of the OpenSSL configuration file: + +```c +#include + +#include + +/* OpenSSL headers */ + +#include + +#include + +#include + +#include + +int main(int argc, char **argv) + +{ + + /* Initializing OpenSSL */ + + SSL_load_error_strings(); + + ERR_load_BIO_strings(); + +#Load openssl configure + +OPENSSL_init_crypto(OPENSSL_INIT_LOAD_CONFIG, NULL); OpenSSL_add_all_algorithms(); + + /*You can use ENGINE_by_id Function to get the handle of the Huawei Accelerator Engine*/ + + ENGINE *e = ENGINE_by_id("kae"); + + /*The user code*/ + + ...... + +; + + ENGINE_free(e); + +; +} +``` + +## Troubleshooting + +### Failed to Initialize the Accelerator Engine + +#### Symptom + +The accelerator engine is not completely loaded. + +#### Solution + +1. Check whether the accelerator drivers are loaded successfully. Specifically, run the **lsmod** command to check whether uacce.ko, qm.ko, sgl.ko, hisi\_sec2.ko, hisi\_hpre.ko, hisi\_zip.ko, and hisi\_rde.ko exist. + + ```shell + $ lsmod | grep uacce + uacce 262144 2 hisi_hpre,hisi_qm,hisi_sec2,hisi_zip,hisi_rde + ``` + +2. Check whether the accelerator engine library exists in **/usr/lib64** \(directory for RPM installation\) or **/usr/local/lib** \(directory for source code installation\) and the OpenSSL installation directory, and check whether the correct soft link is established. + + ```shell + $ ll /usr/local/lib/engines-1.1/ |grep kae + # Check whether the KAE has been correctly installed and whether a soft link has been established. If yes, the displayed information is as follows: + lrwxrwxrwx. 1 root root 22 Nov 12 02:33 kae.so -> kae.so.1.0.1 + lrwxrwxrwx. 1 root root 22 Nov 12 02:33 kae.so.0 -> kae.so.1.0.1 + -rwxr-xr-x. 1 root root 112632 May 25 2019 kae.so.1.0.1 + $ + $ ll /usr/lib64/ | grep libwd + # Check whether libwd has been correctly installed and whether a soft link has been established. If yes, the displayed information is as follows: + lrwxrwxrwx. 1 root root 14 Nov 12 02:33 libwd.so -> libwd.so.1.0.1 + lrwxrwxrwx. 1 root root 14 Nov 12 02:33 libwd.so.0 -> libwd.so.1.0.1 + -rwxr-xr-x. 1 root root 137120 May 25 2019 libwd.so.1.0.1 + $ + ``` + +3. Check whether the path of the OpenSSL engine library can be exported by running the **export** command. + + ```shell + $ echo $OPENSSL_ENGINES + $ export OPENSSL_ENGINES=/usr/local/lib/engines-1.1 + $ echo $OPENSSL_ENGINES + /usr/local/lib/engines-1.1 + ``` + +### Failed to Identify Accelerator Devices After the Acceleration Engine Is Installed + +#### Symptom + +After the acceleration engine is installed, the accelerator devices cannot be identified. + +#### Solution + +1. Check whether the device exists in the virtual file system. Normally, the following accelerator devices are displayed: + + ```shell + $ ls -al /sys/class/uacce/ + total 0 + lrwxrwxrwx. 1 root root 0 Nov 14 03:45 hisi_hpre-2 -> ../../devices/pci0000:78/0000:78:00.0/0000:79:00.0/uacce/hisi_hpre-2 + lrwxrwxrwx. 1 root root 0 Nov 14 03:45 hisi_hpre-3 -> ../../devices/pci0000:b8/0000:b8:00.0/0000:b9:00.0/uacce/hisi_hpre-3 + lrwxrwxrwx. 1 root root 0 Nov 17 22:09 hisi_rde-4 -> ../../devices/pci0000:78/0000:78:01.0/uacce/hisi_rde-4 + lrwxrwxrwx. 1 root root 0 Nov 17 22:09 hisi_rde-5 -> ../../devices/pci0000:b8/0000:b8:01.0/uacce/hisi_rde-5 + lrwxrwxrwx. 1 root root 0 Nov 14 08:39 hisi_sec-0 -> ../../devices/pci0000:74/0000:74:01.0/0000:76:00.0/uacce/hisi_sec-0 + lrwxrwxrwx. 1 root root 0 Nov 14 08:39 hisi_sec-1 -> ../../devices/pci0000:b4/0000:b4:01.0/0000:b6:00.0/uacce/hisi_sec-1 + lrwxrwxrwx. 1 root root 0 Nov 17 22:09 hisi_zip-6 -> ../../devices/pci0000:74/0000:74:00.0/0000:75:00.0/uacce/hisi_zip-6 + lrwxrwxrwx. 1 root root 0 Nov 17 22:09 hisi_zip-7 -> ../../devices/pci0000:b4/0000:b4:00.0/0000:b5:00.0/uacce/hisi_zip-7 + ``` + +2. If you want to use the HPRE device but the device is not found in [1](#li1760055514614), check whether the accelerator software is correctly installed by referring to [Failed to Upgrade the Accelerator Drivers](#failed-to-upgrade-the-accelerator-drivers). +3. If the accelerator software is correctly installed, run the **lspci** command to check whether the physical device exists. + + ```shell + $ lspci | grep HPRE + 79:00.0 Network and computing encryption device: Huawei Technologies Co., Ltd. HiSilicon HPRE Engine (rev 21) + b9:00.0 Network and computing encryption device: Huawei Technologies Co., Ltd. HiSilicon HPRE Engine (rev 21) + $ lspci | grep SEC + 76:00.0 Network and computing encryption device: Huawei Technologies Co., Ltd. HiSilicon SEC Engine (rev 21) + b6:00.0 Network and computing encryption device: Huawei Technologies Co., Ltd. HiSilicon SEC Engine (rev 21) + $ lspci | grep RDE + 78:01.0 RAID bus controller: Huawei Technologies Co., Ltd. HiSilicon RDE Engine (rev 21) + b8:01.0 RAID bus controller: Huawei Technologies Co., Ltd. HiSilicon RDE Engine (rev 21) + $ lspci | grep ZIP + 75:00.0 Processing accelerators: Huawei Technologies Co., Ltd. HiSilicon ZIP Engine (rev 21) + b5:00.0 Processing accelerators: Huawei Technologies Co., Ltd. HiSilicon ZIP Engine (rev 21) + $ + ``` + +4. If no physical device is found in [3](#li1560012551369), perform the following operations: + - Check whether the accelerator license has been imported. If no, import the accelerator license. For details, see "License Management" in the [TaiShan Rack Server iBMC \(V500 or Later\) User Guide](https://support.huawei.com/enterprise/en/doc/EDOC1100121685/426cffd9?idPath=7919749%257C9856522%257C21782478%257C8060757). After the accelerator license is imported, power off and restart the iBMC to enable the license. + - Check whether the iBMC and BIOS versions support the accelerator feature. + +### Failed to Upgrade the Accelerator Drivers + +#### Symptom + +After the accelerator drivers are upgraded, the driver version is not changed after the system is restarted. + +#### Possible Cause + +Before the accelerator drivers are upgraded, the system upgrades other driver packages. These driver packages may update the boot file system initramfs, and update the accelerator drivers to initramfs before upgrade. For example, if the NIC driver is updated or initramfs is manually updated, the system loads the accelerator drivers from initramfs first during restart. + +#### Solution + +After the accelerator drivers are upgraded, run the **dracut \-\-force** command to update initramfs again. diff --git a/docs/en/server/performance/cpu_optimization/sysboost/_toc.yaml b/docs/en/server/performance/cpu_optimization/sysboost/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..4883c1ea6598157e99e16817880f9165ffde5f0f --- /dev/null +++ b/docs/en/server/performance/cpu_optimization/sysboost/_toc.yaml @@ -0,0 +1,13 @@ +label: sysBoost User Guide +isManual: true +href: ./sysboost.md +description: >- + Enhance code compatibility with the CPU microarchitecture of the execution + environment to boost program performance. +sections: + - label: Getting to Know sysBoost + href: ./getting-to-know-sysboost.md + - label: Installation and Deployment + href: ./installation-and-deployment.md + - label: Usage Instructions + href: ./usage-instructions.md diff --git a/docs/en/server/performance/cpu_optimization/sysboost/figures/architecture.png b/docs/en/server/performance/cpu_optimization/sysboost/figures/architecture.png new file mode 100644 index 0000000000000000000000000000000000000000..c4d805fe2caf7fe1f2ae38a37b22c39e1e002c6b Binary files /dev/null and b/docs/en/server/performance/cpu_optimization/sysboost/figures/architecture.png differ diff --git a/docs/en/server/performance/cpu_optimization/sysboost/figures/icon-note.gif b/docs/en/server/performance/cpu_optimization/sysboost/figures/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/server/performance/cpu_optimization/sysboost/figures/icon-note.gif differ diff --git a/docs/en/server/performance/cpu_optimization/sysboost/getting-to-know-sysboost.md b/docs/en/server/performance/cpu_optimization/sysboost/getting-to-know-sysboost.md new file mode 100644 index 0000000000000000000000000000000000000000..6666c961d15ce0a5dfac833b6e6a61324db22799 --- /dev/null +++ b/docs/en/server/performance/cpu_optimization/sysboost/getting-to-know-sysboost.md @@ -0,0 +1,61 @@ +# Getting to Know sysBoost + +## Introduction + +sysBoost reorders the code of executable files and dynamic libraries online to adapt the code for the CPU microarchitecture of the operating environment, boosting program performance. + +## Background + +- Large-scale applications use a large number of third-party or self-developed dynamic libraries. A large number of PLT jumps occur during function invoking. As a result, the instructions per cycle (IPC) decreases. + +- Assembly code is large in size and occupies a large amount of memory, resulting in a high iTLB miss rate. Hotspot code segments are scattered. As a result, the I-cache miss rate is high, affecting the CPU pipeline efficiency. + +- Application developers are unfamiliar with the OS and CPU microarchitecture, resulting in high IPC performance optimization costs. + +## sysBoost Design + +### Key technologies + +- Dynamic library merging: Scattered code segments and data segments are merged when the dynamic loader loads dynamic libraries. Huge page memory is used to improve the iTLB hit ratio. + +- PLT jump elimination: When the application code calls a dynamic library function, the execution is redirected to the PLT and then to the actual function. Eliminating PLT jump can improve the IPC. + +- Online reordering of hotspot code segments: By default, code is arranged by dynamic library. The online reordering technology can reorder hotspot code by segment. + +- exec native huge page mechanism: The user-mode huge page mechanism requires specific application configuration and recompilation. The exec native huge page mechanism directly uses huge page memory when the kernel loads the ELF file,without the need for modifying applications. + +### Architecture + +**Figure 1** sysBoost architecture + +![](./figures/architecture.png) + +## sysBoost Features + +- Full static merging: Applications and their dependent dynamic libraries are merged into one binary file, and segment-level reordering is performed. Multiple discrete code segments or data segments are merged into one to improve application performance. + +- Automatic binary file optimization: The sysBoost daemon reads the configuration file to obtain the binary files to be optimized and the corresponding optimization methods, optimizes the binary files based on user requirements, and stores the optimized binary files in RTO files. + +- Huge page preloading of binary code segments/data segments: When the user-mode page table is mapped to the physical memory, huge page (2 MB) mapping can improve performance. However, openEuler does not support huge page mapping of file pages. sysBoost provides the huge page pre-loading function. After binary optimization is complete, sysBoost immediately loads the content to the kernel as a huge page. When an application is started, sysBoost maps the pre-loaded content to the user-mode page table in batches to reduce page faults and memory access delay of the application, thereby improving the application startup speed and running efficiency. + +- Binary exception monitoring: If a bug occurs in the RTO binary file generated by sysBoost, the application may crash. To avoid repeated application starts and crashes and prevent the fault from spreading, sysBoost monitors the processes that load the RTO binary files. If such a process crashes, sysBoost rolls back the optimization by deleting the RTO file and the flag on the original application file. In addition, sysBoost renames the configuration file to prevent optimization from being applied again after the sysBoost service is restarted. + +## Benefits + +### Scenario 1 + +In the Bash test of UnixBench, some common commands and scripts are executed, such as `ls`, `grep`, and `awk`. These commands and scripts usually invoke some system libraries, such as **libc** and **libpthread**. These library files usually need to be dynamically linked, which increases the program startup time and delay. By using the binary file merging technology, these library files can be merged into an executable file, significantly improving the Bash performance and increasing the UnixBench score. + +### Scenario 2 + +The dynamic assembly design of some applications uses a large number of dynamic libraries, which brings the following problems: + +- Indirect function jump and scattered code segments affect CPU execution efficiency. +- The parsing of excessive dynamic library symbols slows down program startup. +- Profile-guided optimization based on a specific service model cannot adapt to different service models. + +Using sysBoost to start large processes during service deployment can effectively solve the preceding problems. + +- The exec huge page mechanism allows the loaded large processes to store code segments and data segments in memory huge pages, reducing the TLB miss rate. +- A large process contains all dynamic library code and application code, eliminating indirect function jumps. +- Service changes are intelligently identified online to regenerate large processes based on appropriate hotspot models. diff --git a/docs/en/server/performance/cpu_optimization/sysboost/installation-and-deployment.md b/docs/en/server/performance/cpu_optimization/sysboost/installation-and-deployment.md new file mode 100644 index 0000000000000000000000000000000000000000..43979e5020e9f7ce2e36f4cecdcf87225c1fb6b5 --- /dev/null +++ b/docs/en/server/performance/cpu_optimization/sysboost/installation-and-deployment.md @@ -0,0 +1,68 @@ +# Installation and Deployment + +## Software and Hardware Requirements + +- Hardware: Kunpeng 920 server + +- Software: openEuler 23.09 + +## Environment Preparation + +- Install the openEuler OS. + +- Obtain root permissions. + +## sysBoost Installation + +To install the sysBoost, perform the following steps (**xxx** in the commands indicate version numbers): + +1. Mount an openEuler ISO image. + + ```sh + # Use the ISO file of the corresponding openEuler version. + mount openEuler-xxx-aarch64-dvd.iso /mnt + ``` + +2. Configure a local Yum source. + + ```sh + vim /etc/yum.repos.d/local.repo + ``` + + The configured contents are as follows: + + ```text + [localosrepo] + name=localosrepo + baseurl=file:///mnt + enabled=1 + gpgcheck=1 + gpgkey=file:///mnt/RPM-GPG-KEY-openEuler + ``` + +3. Install sysBoost. + + ```sh + yum install sysboost -y + ``` + +4. Check whether the installation is successful. + + ```text + # rpm -qa | grep sysboost + sysboost-xxx + # rpm -qa | grep native-turbo + native-turbo-xxx + ``` + + If the preceding information is displayed, the installation is successful. + +5. Install the relocation packages required for combining the ELF files. + + ```sh + yum install bash-relocation-xxx -y + yum install ncurses-relocation-xxx -y + ``` + + > ![](./figures/icon-note.gif) **Note:** + > If the ELF files and their dependency libraries contain the relocation segment, skip this step. diff --git a/docs/en/server/performance/cpu_optimization/sysboost/sysboost.md b/docs/en/server/performance/cpu_optimization/sysboost/sysboost.md new file mode 100644 index 0000000000000000000000000000000000000000..758405a32d143328fdc1a0b677a0a7c2f7194b30 --- /dev/null +++ b/docs/en/server/performance/cpu_optimization/sysboost/sysboost.md @@ -0,0 +1,5 @@ +# sysBoost User Guide + +This document describes how to install and use sysBoost, which is an online ELF reordering optimization software for openEuler. + +This document is intended for developers, open-source enthusiasts, and partners who use the openEuler system and want to know and use sysBoost. You need to have basic knowledge of the Linux OS. diff --git a/docs/en/server/performance/cpu_optimization/sysboost/usage-instructions.md b/docs/en/server/performance/cpu_optimization/sysboost/usage-instructions.md new file mode 100644 index 0000000000000000000000000000000000000000..1f6ef3887f4167f7d6e5bd7adb5de29204523442 --- /dev/null +++ b/docs/en/server/performance/cpu_optimization/sysboost/usage-instructions.md @@ -0,0 +1,94 @@ +# Usage Instructions + +## Overview + +- Root permissions are required for using and configuring sysBoost. +- Only one sysBoost instance can exist. +- The system administrator must ensure the configuration file is correct. + +## Configuration + +### Configuration File Description + +Configuration file directory: **/etc/sysboost.d/** + +**Table 1** Client YAML configuration file + + + + + + + + + + + + + + + + + + + + + + + + + +

Item

+

Description

+

Type

+

Value Range

+

elf_path

+

ELF file to be combined

+

String

+

ELF path supported by sysBoost

+

mode

+

sysBoost running mode

+

String

+

"static"

+

libs

+

Dependency library of the ELF file specified by elf_path. This is optional because sysBoost can automatically detects dependency libraries.

+

String

+

Path of the dependent library of the ELF file supported by sysBoost

+
+ +### Configuration Example + +sysBoost TOML configuration file example: + +```text +# /etc/sysboost.d/bash.toml +elf_path = "/usr/bin/bash" +mode = "static-nolibc" +libs = ["/usr/lib64/libtinfo.so.6"] +``` + +## Usage + +- Start sysBoost. + + ```text + systemctl start sysboost.service + ``` + +- Stop sysBoost. + + ```text + systemctl stop sysboost.service + ``` + +- Query sysBoost status. If there is no text in red, sysBoost is running normally. + + ```text + systemctl status sysboost.service + ``` + +- View logs. If sysBoost fails, see the system logs for details. + + ```text + cat /var/log/messages + ``` diff --git a/docs/en/server/performance/overall/_toc.yaml b/docs/en/server/performance/overall/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..2f17e74f847ac1173baa3a45b96a1819ef0f8ab3 --- /dev/null +++ b/docs/en/server/performance/overall/_toc.yaml @@ -0,0 +1,3 @@ +label: Overview +sections: + - href: ./system_resource/_toc.yaml diff --git a/docs/en/server/performance/overall/system_resource/_toc.yaml b/docs/en/server/performance/overall/system_resource/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..036dfca17311c4be9213c982a9d3819577054bfd --- /dev/null +++ b/docs/en/server/performance/overall/system_resource/_toc.yaml @@ -0,0 +1,4 @@ +label: System Resources and Performance +isManual: true +href: ./system-resources-and-performance.md +description: CPU, memory, I/O, and common performance analysis tools diff --git a/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001335457246.png b/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001335457246.png new file mode 100644 index 0000000000000000000000000000000000000000..325d6a8ce097db0b92b1a883bc4b3d4ad0bc6a49 Binary files /dev/null and b/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001335457246.png differ diff --git a/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001336448570.png b/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001336448570.png new file mode 100644 index 0000000000000000000000000000000000000000..4bd494d78d83fef2e8a89c80e17c9b6db892a2e9 Binary files /dev/null and b/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001336448570.png differ diff --git a/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001337039920.png b/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001337039920.png new file mode 100644 index 0000000000000000000000000000000000000000..40c07e9b6ec27cdbe47d39788736b892f1174cc8 Binary files /dev/null and b/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001337039920.png differ diff --git a/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001384808269.png b/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001384808269.png new file mode 100644 index 0000000000000000000000000000000000000000..be18ecef3a149d5742f18535552f66f26ab34832 Binary files /dev/null and b/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001384808269.png differ diff --git a/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001385585749.png b/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001385585749.png new file mode 100644 index 0000000000000000000000000000000000000000..c13604ab7095c2a7717bde1384f0aea3d53f69e3 Binary files /dev/null and b/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001385585749.png differ diff --git a/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001385611905.png b/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001385611905.png new file mode 100644 index 0000000000000000000000000000000000000000..8c233e40a21e678ddf4115c2e2e80c96e25a60ce Binary files /dev/null and b/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001385611905.png differ diff --git a/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001385905845.png b/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001385905845.png new file mode 100644 index 0000000000000000000000000000000000000000..a6cb8bc4a188ef444919d71f7f16baa06422788b Binary files /dev/null and b/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001385905845.png differ diff --git a/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001386149037.png b/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001386149037.png new file mode 100644 index 0000000000000000000000000000000000000000..da73fead24d8805bb43287f53c757e80ff0d597f Binary files /dev/null and b/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001386149037.png differ diff --git a/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001389098425.png b/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001389098425.png new file mode 100644 index 0000000000000000000000000000000000000000..c63903009ab9ba454f169250632dbec1b3c94467 Binary files /dev/null and b/docs/en/server/performance/overall/system_resource/images/zh-cn_image_0000001389098425.png differ diff --git a/docs/en/server/performance/overall/system_resource/system-resources-and-performance.md b/docs/en/server/performance/overall/system_resource/system-resources-and-performance.md new file mode 100644 index 0000000000000000000000000000000000000000..73d059a7dbd21a4d3cd9488e90ea3ab51d53fd46 --- /dev/null +++ b/docs/en/server/performance/overall/system_resource/system-resources-and-performance.md @@ -0,0 +1,313 @@ +# System Resources and Performance + +## CPU + +### Basic Concepts + +A central processing unit (CPU) is one of main devices of a computer, and a function of the CPU is to interpret computer instructions and process data in computer software. + +1. Physical core: an actual CPU core that can be seen. It has independent circuit components and L1 and L2 caches and can independently execute instructions. A CPU can have multiple physical cores. +2. Logical core: a core that exists at the logical layer in the same physical core. Generally, a physical core corresponds to a thread. However, if hyper-threading is enabled and the number of hyper-threads is *n*, a physical core can be divided into *n* logical cores. + +You can run the **lscpu** command to check the number of CPUs on the server, the number of physical cores in each CPU, and the number of logical cores in each CPU. + +### Common CPU Performance Analysis Tools + +1. **uptime**: prints the average system load. You can view the last three numbers to determine the change trend of the average load. + If the average load is greater than the number of CPUs, the CPUs are insufficient to serve threads and some threads are waiting. If the average load is less than the number of CPUs, there are remaining CPUs. + + ![zh-cn_image_0000001384808269](./images/zh-cn_image_0000001384808269.png) + +2. **vmstat**: dynamically monitors the usage of system resources and checks which phase occupies the most system resources. + You can run the **vmstat -h** command to view command parameters. + Example: + + ```shell + #Monitor the status and update the status every second. + vmstat 1 + ``` + + ![](./images/zh-cn_image_0000001385585749.png) + The fields in the command output are described as follows: + + | Field | Description | + | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | procs | Process information. | + | memory | Memory information. | + | swap | Swap partition information. | + | io | Drive read/write information. | + | system | System information. | + | cpu | CPU information.
**-us**: percentage of the CPU computing time consumed by non-kernel processes.
**-sy**: percentage of the CPU computing time consumed by kernel processes.
**-id**: idle.
**-wa**: percentage of CPU resources consumed by waiting for I/Os.
**-st**: percentage of CPUs stolen by VMs. | + +3. **sar**: analyzes system performance, observes current activities and configurations, and archives and reports historical statistics. + Example: + + ```shell + # Check the overall CPU load of the system. Collect the statistics every 3 seconds for five times. + sar -u 3 5 + ``` + + ![zh-cn_image_0000001336448570](./images/zh-cn_image_0000001336448570.png) + + The fields in the command output are described as follows: + + | Field | Description | + | ------- | ----------------------------------------------------------------------------------------------------------------------- | + | %user | Percentage of the CPU time consumed in user mode. | + | %nice | Percentage of the CPU time consumed by a process whose scheduling priority is changed through **nice** in user mode. | + | %system | Percentage of the CPU time consumed in system mode. | + | %iowait | Percentage of the time consumed by the CPU to wait for drive I/Os in idle state. | + | %steal | Percentage of the time used for waiting for other virtual CPU computing by using virtualization technologies of the OS. | + | %idle | Percentage of CPU idle time. | + +4. **ps**: displays running processes. + + ```shell + # View all processes in the system, and view the PIDs and priorities of the their parent processes. + ps -le + ``` + + ![zh-cn_image_0000001337039920](./images/zh-cn_image_0000001337039920.png) + + ```shell + # View the processes generated by the current shell. + ps -l + ``` + + ![zh-cn_image_0000001385611905](./images/zh-cn_image_0000001385611905.png) + +5. **top**: dynamically and continuously monitors the running status of processes and displays the processes that consume the most CPU resources. + +```shell +top +``` + +![zh-cn_image_0000001335457246](./images/zh-cn_image_0000001335457246.png) + +## Memory + +### Basic Concepts + +The memory is an important component of a computer, and is used to temporarily store operation data in the CPU and data exchanged with an external memory such as hardware. In particular, a non-uniform memory access architecture (NUMA) is a memory architecture designed for a multiprocessor computer. The memory access time depends on the location of the memory relative to the processor. In NUMA mode, a processor accesses the local memory faster than the non-local memory (the memory is located in another processor or shared between processors). + +### Common Memory Analysis Tools and Methods + +1. **free**: displays the system memory status. + Example: + + ```shell + # Display the system memory status in MB. + free -m + ``` + + The output is as follows: + + ```text + total used free shared buff/cache available + Mem: 2633 436 324 23 2072 2196 + Swap: 4043 0 4043 + ``` + + The fields in the command output are described as follows: + + | Field | Description | + | ---------- | ----------------------------------------------------------------------- | + | total | Total memory size. | + | used | Used memory. | + | free | Free memory. | + | shared | Total memory shared by multiple processes. | + | buff/cache | Total number of buffers and caches. | + | available | Estimated available memory to start a new application without swapping. | + +2. **vmstat**: dynamically monitors the system memory and views the system memory usage. + + Example: + + ```shell + # Monitor the system memory and display active and inactive memory. + vmstat -a + ``` + + The output is as follows: + + ```text + procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- + r b swpd free inact active si so bi bo in cs us sy id wa st + 2 0 520 331980 1584728 470332 0 0 0 2 15 19 0 0 100 0 0 + ``` + + In the command output, the field related to the memory is described as follows: + + | Field | Description | + | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | memory | Memory information.
**-swpd**: usage of the virtual memory, in KB.
**-free**: free memory capacity, in KB.
**-inact**: inactive memory capacity, in KB.
**-active**: active memory capacity, in KB. | + +3. **sar**: monitors the memory usage of the system. + + Example: + + ```shell + # Monitor the memory usage in the sampling period in the system. Collect the statistics every two seconds for three times. + sar -r 2 3 + ``` + + The output is as follows: + + ```text + 04:02:09 PM kbmemfree kbavail kbmemused %memused kbbuffers kbcached kbcommit %commit kbactive kbinact kb + dirty + 04:02:11 PM 332180 2249308 189420 7.02 142172 1764312 787948 11.52 470404 1584924 + 36 + 04:02:13 PM 332148 2249276 189452 7.03 142172 1764312 787948 11.52 470404 1584924 + 36 + 04:02:15 PM 332148 2249276 189452 7.03 142172 1764312 787948 11.52 470404 1584924 + 36 + Average: 332159 2249287 189441 7.03 142172 1764312 787948 11.52 470404 1584924 + 36 + ``` + + The fields in the command output are described as follows: + + | Field | Description | + | --------- | ------------------------------------------------ | + | kbmemfree | Unused memory space. | + | kbmemused | Used memory space. | + | %memused | Percentage of the used space. | + | kbbuffers | Amount of data stored in the buffer. | + | kbcached | Data access volume in all domains of the system. | + +4. **numactl**: displays the NUMA node configuration and status. + + Example: + + ```shell + # Check the current NUMA configuration. + numactl -H + ``` + + The output is as follows: + + ```text + available: 1 nodes (0) + node 0 cpus: 0 1 2 3 + node 0 size: 2633 MB + node 0 free: 322 MB + node distances: + node 0 + 0: 10 + ``` + + The server contains one NUMA node, which consists of four CPU cores, each has about 6 GB memory. + The output also shows distances between nodes. The greater the distance, the larger the latency of corss-NUMA node memory accesses. Applications should not access memory across NUMA nodes frequently. + + **numastat**: displays the NUMA node status. + + ```shell + # Check the NUMA node status. + numastat + ``` + + ```text + node0 + numa_hit 5386186 + numa_miss 0 + numa_foreign 0 + interleave_hit 17483 + local_node 5386186 + other_node 0 + ``` + + The fields in the **numstat** command output are described as follows: + + | Field | Description | + | --------- | --------------------------------------------------------------------------- | + | numa_hit | Number of times that the CPU core accesses the local memory on a node. | + | numa_miss | Number of times that the core of a node accesses the memory of other nodes. | + +## I/O + +### Basic Concepts + +I/O indicates input/output. Input refers to the operation of receiving signals or data by the system, and output refers to the operation of sending signals or data from the system. For a combination of CPU and main memory, any information incoming to or outgoing from the CPU/memory combination is considered as I/Os. + +### Common I/O Performance Analysis Tools + +1. **iostat**: reports statistics about all online drives. + + Example: + + ```shell + # Display the drive information in KB. Collect the statistics every second and for 100 seconds. + iostat -d -k -x 100 + ``` + + ![zh-cn_image_0000001385905845](./images/zh-cn_image_0000001385905845.png) + + The fields in the command output are described as follows: + + | Field | Description | + | -------- | ---------------------------------------------------------------------------------- | + | Device | Name of the monitoring device. | + | r/s | Number of read requests completed by the device per second (after combination). | + | rKB/s | Number of KBs read from the drive per second. | + | rrqm/s | Number of read operations merged into the request queue per second. | + | %rrqm | Percentage of read requests merged before they are sent to the device. | + | r_await | Average time consumed by each read request. | + | rareq-sz | Average size of read requests sent to the device, in KB. | + | w/s | Number of write requests completed by the device per second (after combination). | + | wKB/s | Number of KBs written to the drive per second. | + | wrqm/s | Number of write operations merged into the request queue per second. | + | %wrqm | Percentage of write requests merged before they are sent to the device. | + | w_await | Average time consumed by each write request. | + | wareq-sz | Average size of write requests sent to the device, in KB. | + | d/s | Number of discard requests processed by the device per second. | + | dKB/s | Number of sectors (KB) discarded by the device per second. | + | drqm/s | Number of discard requests merged into the device queue per second. | + | %drqm | Percentage of discard requests merged before they are sent to the device. | + | d_await | Average time for sending discard requests to the device to be served. | + | dareq-sz | Average size of discard requests sent to the device, in KB. | + | f/s | Number of refresh requests completed by the device per second (after combination). | + | f_await | Average time for sending refresh requests to the device to be served. | + | aqu-sz | Average queue length of requests sent to the device. | + | %util | Percentage of the I/O operation time, that is, the usage. | + +2. **sar**: displays the read and write performance of the system drive. + + Example: + + ```shell + # Display the usage status of all hard drives in the system in the sampling period. Collect the statistics every 3 seconds for five times. + sar -d 3 5 + ``` + + ![zh-cn_image_0000001386149037](./images/zh-cn_image_0000001386149037.png) + + The fields in the command output are described as follows: + + | Field | Description | + | ------- | ----------------------------------------------------------------------------------------------- | + | tps | Total number of transfers sent to the physical device per second. | + | rKB/s | Number of KBs read from the device per second. | + | wKB/s | Number of KBs written to the device per second. | + | dKB/s | Number of KBs discarded by the device per second. | + | areq-sz | Average size (KB) of I/O requests sent to the device. | + | aqu-sz | Average queue length of requests sent to the device. | + | await | Average time for sending I/O requests to the device to be served. | + | %util | Percentage of the time used to send I/O requests to the device (bandwidth usage of the device). | + +3. vmstat + + ```shell + # Run the vmstat command to monitor and report drive statistics. + vmstat -d + ``` + + ![zh-cn_image_0000001389098425](./images/zh-cn_image_0000001389098425.png) + + The fields in the command output are described as follows: + + | Field | Description | + | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | reads | **-total**: total number of reads that have been successfully completed.
**-merged**: number of merged reads (resulting in one I/O).
**-sectors**: sectors from which data is successfully read.
**-ms**: number of milliseconds spent on reading data. | + | writes | **-total**: total number of writes that have been successfully completed.
**-merged**: merged writes (resulting in one I/O).
**-sectors**: sectors to which data is successfully written.
**-ms**: number of milliseconds spent on writing data. | + | IO | **-cur**: number of I/O operations in progress. **-sec**: total number of seconds spent on I/O. | diff --git a/docs/en/server/performance/system_optimization/_toc.yaml b/docs/en/server/performance/system_optimization/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..2683e30512ba909f1f55b5c84f8f51eb7cb9de9a --- /dev/null +++ b/docs/en/server/performance/system_optimization/_toc.yaml @@ -0,0 +1,3 @@ +label: System Optimization +sections: + - href: ./atune/_toc.yaml diff --git a/docs/en/server/performance/system_optimization/atune/_toc.yaml b/docs/en/server/performance/system_optimization/atune/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..784ea24220e40b0077cda52606ca0f281d96c9b4 --- /dev/null +++ b/docs/en/server/performance/system_optimization/atune/_toc.yaml @@ -0,0 +1,17 @@ +label: A-Tune User Guide +isManual: true +href: ./a-tune.md +description: Optimized openEuler performance through AI-powered, automated tuning +sections: + - label: Getting to Know A-Tune + href: ./getting-to-know-a-tune.md + - label: Installation and Deployment + href: ./installation-and-deployment.md + - label: Usage Instructions + href: ./usage-instructions.md + - label: Native-Turbo + href: ./native-turbo.md + - label: Common Issues and Solutions + href: ./common-issues-and-solutions.md + - label: Appendix + href: ./appendix.md diff --git a/docs/en/server/performance/system_optimization/atune/a-tune.md b/docs/en/server/performance/system_optimization/atune/a-tune.md new file mode 100644 index 0000000000000000000000000000000000000000..b481797f5c97d2c5e477fe1a0c7a4b92f646d7b3 --- /dev/null +++ b/docs/en/server/performance/system_optimization/atune/a-tune.md @@ -0,0 +1,5 @@ +# A-Tune User Guide + +This document describes how to install and use A-Tune, which is a performance self-optimization software for openEuler. + +This document is intended for developers, open-source enthusiasts, and partners who use the openEuler system and want to know and use A-Tune. You need to have basic knowledge of the Linux OS. diff --git a/docs/en/server/performance/system_optimization/atune/appendix.md b/docs/en/server/performance/system_optimization/atune/appendix.md new file mode 100644 index 0000000000000000000000000000000000000000..81568dbd22322b5003f19aa0d953edfc519004f3 --- /dev/null +++ b/docs/en/server/performance/system_optimization/atune/appendix.md @@ -0,0 +1,22 @@ +# Appendix + +- [Appendix](#appendix) + - [Acronyms and Abbreviations](#acronyms-and-abbreviations) + +## Acronyms and Abbreviations + +**Table 1** Terminology + + + + + + + + + +

Term

+

Description

+

profile

+

Set of optimization items and optimal parameter configuration.

+
diff --git a/docs/en/server/performance/system_optimization/atune/common-issues-and-solutions.md b/docs/en/server/performance/system_optimization/atune/common-issues-and-solutions.md new file mode 100644 index 0000000000000000000000000000000000000000..38a9d93493919182e8bcc05f45740b3f50aae64f --- /dev/null +++ b/docs/en/server/performance/system_optimization/atune/common-issues-and-solutions.md @@ -0,0 +1,52 @@ +# Common Issues and Solutions + +## Issue 1: An error occurs when the **train** command is used to train a model, and the message "training data failed" is displayed + +Cause: Only one type of data is collected by using the **collection**command. + +Solution: Collect data of at least two data types for training. + +## Issue 2: atune-adm cannot connect to the atuned service + +Possible cause: + +1. Check whether the atuned service is started and check the atuned listening address. + + ```shell + systemctl status atuned + netstat -nap | grep atuned + ``` + +2. The firewall blocks the atuned listening port. +3. The HTTP proxy is configured in the system. As a result, the connection fails. + +Solution: + +1. If the atuned service is not started, run the following command to start the service: + + ```shell + systemctl start atuned + ``` + +2. Run the following command on the atuned and atune-adm servers to allow the listening port to receive network packets. In the command, **60001** is the listening port number of the atuned server. + + ```shell + iptables -I INPUT -p tcp --dport 60001 -j ACCEPT + iptables -I INPUT -p tcp --sport 60001 -j ACCEPT + ``` + +3. Run the following command to delete the HTTP proxy or disable the HTTP proxy for the listening IP address without affecting services: + + ```shell + no_proxy=$no_proxy, Listening_IP_address + ``` + +## Issue 3: The atuned service cannot be started, and the message "Job for atuned.service failed because a timeout was exceeded." is displayed + +Cause: The hosts file does not contain the localhost information. + +Solution: Add localhost to the line starting with **127.0.0.1** in the **/etc/hosts** file. + +```text +127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 +``` diff --git a/docs/en/server/performance/system_optimization/atune/figures/en-us_image_0214540398.png b/docs/en/server/performance/system_optimization/atune/figures/en-us_image_0214540398.png new file mode 100644 index 0000000000000000000000000000000000000000..cea2292307b57854aa629ec102a5bc1b16d244a0 Binary files /dev/null and b/docs/en/server/performance/system_optimization/atune/figures/en-us_image_0214540398.png differ diff --git a/docs/en/server/performance/system_optimization/atune/figures/en-us_image_0227497000.png b/docs/en/server/performance/system_optimization/atune/figures/en-us_image_0227497000.png new file mode 100644 index 0000000000000000000000000000000000000000..3df66e5f25177cba7fe65cfb859fab860bfb7b46 Binary files /dev/null and b/docs/en/server/performance/system_optimization/atune/figures/en-us_image_0227497000.png differ diff --git a/docs/en/server/performance/system_optimization/atune/figures/en-us_image_0227497343.png b/docs/en/server/performance/system_optimization/atune/figures/en-us_image_0227497343.png new file mode 100644 index 0000000000000000000000000000000000000000..a8654b170295b4b0be3c37187e4b227ca635fbc0 Binary files /dev/null and b/docs/en/server/performance/system_optimization/atune/figures/en-us_image_0227497343.png differ diff --git a/docs/en/server/performance/system_optimization/atune/figures/en-us_image_0231122163.png b/docs/en/server/performance/system_optimization/atune/figures/en-us_image_0231122163.png new file mode 100644 index 0000000000000000000000000000000000000000..c61c39c5f5119d84c6799b1e17285a7fe313639f Binary files /dev/null and b/docs/en/server/performance/system_optimization/atune/figures/en-us_image_0231122163.png differ diff --git a/docs/en/server/performance/system_optimization/atune/figures/en-us_image_0245342444.png b/docs/en/server/performance/system_optimization/atune/figures/en-us_image_0245342444.png new file mode 100644 index 0000000000000000000000000000000000000000..10f0fceb42c00c80ef49decdc0c480eb04c2ca6d Binary files /dev/null and b/docs/en/server/performance/system_optimization/atune/figures/en-us_image_0245342444.png differ diff --git a/docs/en/server/performance/system_optimization/atune/figures/picture1.png b/docs/en/server/performance/system_optimization/atune/figures/picture1.png new file mode 100644 index 0000000000000000000000000000000000000000..624d148b98bc9890befbecc53f29d6a4890d06af Binary files /dev/null and b/docs/en/server/performance/system_optimization/atune/figures/picture1.png differ diff --git a/docs/en/server/performance/system_optimization/atune/figures/picture4.png b/docs/en/server/performance/system_optimization/atune/figures/picture4.png new file mode 100644 index 0000000000000000000000000000000000000000..c576fd0369008e847e6943d6f99351caccf9f3e5 Binary files /dev/null and b/docs/en/server/performance/system_optimization/atune/figures/picture4.png differ diff --git a/docs/en/server/performance/system_optimization/atune/getting-to-know-a-tune.md b/docs/en/server/performance/system_optimization/atune/getting-to-know-a-tune.md new file mode 100644 index 0000000000000000000000000000000000000000..78c84dd0df7194bca6669cbfcb8ee47ad437d49a --- /dev/null +++ b/docs/en/server/performance/system_optimization/atune/getting-to-know-a-tune.md @@ -0,0 +1,68 @@ +# Getting to Know A-Tune + +- [Getting to Know A-Tune](#getting-to-know-a-tune) + - [Introduction](#introduction) + - [Architecture](#architecture) + - [Supported Features and Service Models](#supported-features-and-service-models) + +## Introduction + +An operating system \(OS\) is basic software that connects applications and hardware. It is critical for users to adjust OS and application configurations and make full use of software and hardware capabilities to achieve optimal service performance. However, numerous workload types and varied applications run on the OS, and the requirements on resources are different. Currently, the application environment composed of hardware and software involves more than 7000 configuration objects. As the service complexity and optimization objects increase, the time cost for optimization increases exponentially. As a result, optimization efficiency decreases sharply. Optimization becomes complex and brings great challenges to users. + +Second, as infrastructure software, the OS provides a large number of software and hardware management capabilities. The capability required varies in different scenarios. Therefore, capabilities need to be enabled or disabled depending on scenarios, and a combination of capabilities will maximize the optimal performance of applications. + +In addition, the actual business embraces hundreds and thousands of scenarios, and each scenario involves a wide variety of hardware configurations for computing, network, and storage. The lab cannot list all applications, business scenarios, and hardware combinations. + +To address the preceding challenges, openEuler launches A-Tune. + +A-Tune is an AI-based engine that optimizes system performance. It uses AI technologies to precisely profile business scenarios, discover and infer business characteristics, so as to make intelligent decisions, match with the optimal system parameter configuration combination, and give recommendations, ensuring the optimal business running status. + +![](figures/en-us_image_0227497000.png) + +## Architecture + +The following figure shows the A-Tune core technical architecture, which consists of intelligent decision-making, system profile, and interaction system. + +- Intelligent decision-making layer: consists of the awareness and decision-making subsystems, which implements intelligent awareness of applications and system optimization decision-making, respectively. +- System profile layer: consists of the feature engineering and two-layer classification model. The feature engineering is used to automatically select service features, and the two-layer classification model is used to learn and classify service models. +- Interaction system layer: monitors and configures various system resources and executes optimization policies. + +![](figures/en-us_image_0227497343.png) + +## Supported Features and Service Models + +### Supported Features + +[Table 1](#table1919220557576) describes the main features supported by A-Tune, feature maturity, and usage suggestions. + +**Table 1** Feature maturity + + + +| Feature | Maturity | Usage Suggestion | +| --------------------------------------------------------- | -------- | ---------------- | +| Auto optimization of 15 applications in 11 workload types | Tested | Pilot | +| User-defined profile and service models | Tested | Pilot | +| Automatic parameter optimization | Tested | Pilot | + +### Supported Service Models + +Based on the workload characteristics of applications, A-Tune classifies services into 11 types. For details about the bottleneck of each type and the applications supported by A-Tune, see [Table 2](#table2819164611311). + +**Table 2** Supported workload types and applications + + + +| Service category | Type | Bottleneck | Supported Application | +| ------------------ | -------------------- | ------------------------------------------------------------ | ----------------------------------- | +| default | Default type | Low resource usage in terms of cpu, memory, network, and I/O | N/A | +| webserver | Web application | Bottlenecks of cpu and network | Nginx, Apache Traffic Server | +| database | Database | Bottlenecks of cpu, memory, and I/O | Mongodb, Mysql, Postgresql, Mariadb | +| big_data | Big data | Bottlenecks of cpu and memory | Hadoop-hdfs, Hadoop-spark | +| middleware | Middleware framework | Bottlenecks of cpu and network | Dubbo | +| in-memory_database | Memory database | Bottlenecks of memory and I/O | Redis | +| basic-test-suite | Basic test suite | Bottlenecks of cpu and memory | SPECCPU2006, SPECjbb2015 | +| hpc | Human genome | Bottlenecks of cpu, memory, and I/O | Gatk4 | +| storage | Storage | Bottlenecks of network, and I/O | Ceph | +| virtualization | Virtualization | Bottlenecks of cpu, memory, and I/O | Consumer-cloud, Mariadb | +| docker | Docker | Bottlenecks of cpu, memory, and I/O | Mariadb | diff --git a/docs/en/server/performance/system_optimization/atune/installation-and-deployment.md b/docs/en/server/performance/system_optimization/atune/installation-and-deployment.md new file mode 100644 index 0000000000000000000000000000000000000000..1ae296aea325239f8bd99598d5456699d24453e6 --- /dev/null +++ b/docs/en/server/performance/system_optimization/atune/installation-and-deployment.md @@ -0,0 +1,524 @@ +# Installation and Deployment + +This chapter describes how to install and deploy A-Tune. + +- [Installation and Deployment](#installation-and-deployment) + - [Software and Hardware Requirements](#software-and-hardware-requirements) + - [Hardware Requirement](#hardware-requirement) + - [Software Requirement](#software-requirement) + - [Environment Preparation](#environment-preparation) + - [A-Tune Installation](#a-tune-installation) + - [Installation Modes](#installation-modes) + - [Installation Procedure](#installation-procedure) + - [A-Tune Deployment](#a-tune-deployment) + - [Overview](#overview) + - [Example](#example) + - [Example](#example-1) + - [Starting A-Tune](#starting-a-tune) + - [Starting A-Tune engine](#starting-a-tune-engine) + +## Software and Hardware Requirements + +### Hardware Requirement + +- Huawei Kunpeng 920 processor + +### Software Requirement + +- OS: openEuler 22.03 + +## Environment Preparation + +For details about installing an openEuler OS, see the [_openEuler Installation Guide_](../../../installation_upgrade/installation/installation.md). + +## A-Tune Installation + +This section describes the installation modes and methods of the A-Tune. + +### Installation Modes + +A-Tune can be installed in single-node, distributed, and cluster modes. + +- Single-node mode + + The client and server are installed on the same system. + +- Distributed mode + + The client and server are installed on different systems. + +- Cluster mode + A cluster consists of a client and more than one servers. + +The installation modes are as follows: + +![](./figures/en-us_image_0231122163.png) + +### Installation Procedure + +To install the A-Tune, perform the following steps: + +1. Mount an openEuler ISO image. + + ```shell + mount openEuler-22.03-LTS-everything-x86_64-dvd.iso /mnt + ``` + + > Use the **everything** ISO image. + +2. Configure the local Yum source. + + ```shell + vim /etc/yum.repos.d/local.repo + ``` + + The configured contents are as follows: + + ```conf + [local] + name=local + baseurl=file:///mnt + gpgcheck=1 + enabled=1 + ``` + +3. Import the GPG public key of the RPM digital signature to the system. + + ```shell + rpm --import /mnt/RPM-GPG-KEY-openEuler + ``` + +4. Install an A-Tune server. + + > [!NOTE]NOTE + > In this step, both the server and client software packages are installed. For the single-node deployment, skip **Step 5**. + + ```shell + yum install atune -y + yum install atune-engine -y + ``` + +5. For a distributed mode, install an A-Tune client on associated server. + + ```shell + yum install atune-client -y + ``` + +6. Check whether the installation is successful. + + ```shell + $ rpm -qa | grep atune + atune-client-xxx + atune-db-xxx + atune-xxx + atune-engine-xxx + ``` + + If the preceding information is displayed, the installation is successful. + +## A-Tune Deployment + +This section describes how to deploy A-Tune. + +### Overview + +The configuration items in the A-Tune configuration file **/etc/atuned/atuned.cnf** are described as follows: + +- A-Tune service startup configuration (modify the parameter values as required). + + - **protocol**: Protocol used by the gRPC service. The value can be **unix** or **tcp**. **unix** indicates the local socket communication mode, and **tcp** indicates the socket listening port mode. The default value is **unix**. + - **address**: Listening IP address of the gRPC service. The default value is **unix socket**. If the gRPC service is deployed in distributed mode, change the value to the listening IP address. + - **port**: Listening port of the gRPC server. The value ranges from 0 to 65535. If **protocol** is set to **unix**, you do not need to set this parameter. + - **connect**: IP address list of the nodes where the A-Tune is located when the A-Tune is deployed in a cluster. IP addresses are separated by commas (,). + - **rest_host**: Listening address of the REST service. The default value is localhost. + - **rest_port**: Listening port of the REST service. The value ranges from 0 to 65535. The default value is 8383. + - **engine_host**: IP address for connecting to the A-Tune engine service of the system. + - **engine_port**: Port for connecting to the A-Tune engine service of the system. + - **sample_num**: Number of samples collected when the system executes the analysis process. The default value is 20. + - **interval**: Interval for collecting samples when the system executes the analysis process. The default value is 5s. + - **grpc_tls**: Indicates whether to enable SSL/TLS certificate verification for the gRPC service. By default, this function is disabled. After grpc_tls is enabled, you need to set the following environment variables before running the **atune-adm** command to communicate with the server: + - export ATUNE_TLS=yes + - export ATUNED_CACERT=\ + - export ATUNED_CLIENTCERT=\ + - export ATUNED_CLIENTKEY=\ + - export ATUNED_SERVERCN=server + - **tlsservercafile**: Path of the gPRC server's CA certificate. + - **tlsservercertfile**: Path of the gPRC server certificate. + - **tlsserverkeyfile**: Path of the gPRC server key. + - **rest_tls**: Indicates whether to enable SSL/TLS certificate verification for the REST service. This function is enabled by default. + - **tlsrestcacertfile**: Path of the server's CA certificate of the REST service. + - **tlsrestservercertfile**: Path of the server certificate of the REST service. + - **tlsrestserverkeyfile**: Indicates the key path of the REST service. + - **engine_tls**: Indicates whether to enable SSL/TLS certificate verification for the A-Tune engine service. This function is enabled by default.. + - **tlsenginecacertfile**: Path of the client CA certificate of the A-Tune engine service. + - **tlsengineclientcertfile**: Client certificate path of the A-Tune engine service. + - **tlsengineclientkeyfile**: Client key path of the A-Tune engine service. + +- System information + + System is the parameter information required for system optimization. You must modify the parameter information according to the actual situation. + + - **disk**: Disk information to be collected during the analysis process or specified disk during disk optimization. + - **network**: NIC information to be collected during the analysis process or specified NIC during NIC optimization. + - **user**: User name used for ulimit optimization. Currently, only the user **root** is supported. + +- Log information + + Change the log level as required. The default log level is info. Log information is recorded in the **/var/log/messages** file. + +- Monitor information + + Hardware information that is collected by default when the system is started. + +- Tuning information + + Tuning is the parameter information required for offline tuning. + + - **noise**: Evaluation value of Gaussian noise. + - **sel_feature**: Indicates whether to enable the function of generating the importance ranking of offline tuning parameters. By default, this function is disabled. + +#### Example + +```conf +#################################### server ############################### + # atuned config + [server] + # the protocol grpc server running on + # ranges: unix or tcp + protocol = unix + + # the address that the grpc server to bind to + # default is unix socket /var/run/atuned/atuned.sock + # ranges: /var/run/atuned/atuned.sock or ip address + address = /var/run/atuned/atuned.sock + + # the atune nodes in cluster mode, separated by commas + # it is valid when protocol is tcp + # connect = ip01,ip02,ip03 + + # the atuned grpc listening port + # the port can be set between 0 to 65535 which not be used + # port = 60001 + + # the rest service listening port, default is 8383 + # the port can be set between 0 to 65535 which not be used + rest_host = localhost + rest_port = 8383 + + # the tuning optimizer host and port, start by engine.service + # if engine_host is same as rest_host, two ports cannot be same + # the port can be set between 0 to 65535 which not be used + engine_host = localhost + engine_port = 3838 + + # when run analysis command, the numbers of collected data. + # default is 20 + sample_num = 20 + + # interval for collecting data, default is 5s + interval = 5 + + # enable gRPC authentication SSL/TLS + # default is false + # grpc_tls = false + # tlsservercafile = /etc/atuned/grpc_certs/ca.crt + # tlsservercertfile = /etc/atuned/grpc_certs/server.crt + # tlsserverkeyfile = /etc/atuned/grpc_certs/server.key + + # enable rest server authentication SSL/TLS + # default is true + rest_tls = true + tlsrestcacertfile = /etc/atuned/rest_certs/ca.crt + tlsrestservercertfile = /etc/atuned/rest_certs/server.crt + tlsrestserverkeyfile = /etc/atuned/rest_certs/server.key + + # enable engine server authentication SSL/TLS + # default is true + engine_tls = true + tlsenginecacertfile = /etc/atuned/engine_certs/ca.crt + tlsengineclientcertfile = /etc/atuned/engine_certs/client.crt + tlsengineclientkeyfile = /etc/atuned/engine_certs/client.key + + + #################################### log ############################### + [log] + # either "debug", "info", "warn", "error", "critical", default is "info" + level = info + + #################################### monitor ############################### + [monitor] + # with the module and format of the MPI, the format is {module}_{purpose} + # the module is Either "mem", "net", "cpu", "storage" + # the purpose is "topo" + module = mem_topo, cpu_topo + + #################################### system ############################### + # you can add arbitrary key-value here, just like key = value + # you can use the key in the profile + [system] + # the disk to be analysis + disk = sda + + # the network to be analysis + network = enp189s0f0 + + user = root + + #################################### tuning ############################### + # tuning configs + [tuning] + noise = 0.000000001 + sel_feature = false +``` + +The configuration items in the configuration file **/etc/atuned/engine.cnf** of the A-Tune engine are described as follows: + +- Startup configuration of the A-Tune engine service (modify the parameter values as required). + + - **engine_host**: Listening address of the A-Tune engine service. The default value is localhost. + - **engine_port**: Listening port of the A-Tune engine service. The value ranges from 0 to 65535. The default value is 3838. + - **engine_tls**: Indicates whether to enable SSL/TLS certificate verification for the A-Tune engine service. This function is enabled by default. + - **tlsenginecacertfile**: Path of the server CA certificate of the A-Tune engine service. + - **tlsengineservercertfile**: Path of the server certificate of the A-Tune engine service. + - **tlsengineserverkeyfile**: Server key path of the A-Tune engine service. + +- Log information + + Change the log level as required. The default log level is info. Log information is recorded in the **/var/log/messages** file. + +#### Example + +```conf +#################################### engine ############################### + [server] + # the tuning optimizer host and port, start by engine.service + # if engine_host is same as rest_host, two ports cannot be same + # the port can be set between 0 to 65535 which not be used + engine_host = localhost + engine_port = 3838 + + # enable engine server authentication SSL/TLS + # default is true + engine_tls = true + tlsenginecacertfile = /etc/atuned/engine_certs/ca.crt + tlsengineservercertfile = /etc/atuned/engine_certs/server.crt + tlsengineserverkeyfile = /etc/atuned/engine_certs/server.key + + #################################### log ############################### + [log] + # either "debug", "info", "warn", "error", "critical", default is "info" + level = info +``` + +## Starting A-Tune + +After A-Tune is installed, you need to configure the A-Tune service before starting it. + +- Configure the A-Tune service. + Modify the network adapter and drive information in the **atuned.cnf** configuration file. + > Note: + > + > If atuned is installed through `make install`, the network adapter and drive information in the configuration file is automatically updated to the default devices on the machine. To collect data from other devices, perform the following steps to configure atuned. + + Run the following command to search for the network adapter that needs to be specified for optimization or data collection, and change the value of **network** in the **/etc/atuned/atuned.cnf** file to the specified network adapter: + + ```shell + ip addr + ``` + + Run the following command to search for the drive that need to be specified for optimization or data collection, and change the value of **disk** in the **/etc/atuned/atuned.cnf** file to the specified drive: + + ```shell + fdisk -l | grep dev + ``` + +- About the certificate: + The A-Tune engine and client use the gRPC communication protocol. Therefore, you need to configure a certificate to ensure system security. For information security purposes, A-Tune does not provide a certificate generation method. You need to configure a system certificate by yourself. + If security is not considered, set **rest_tls** and **engine_tls** in the **/etc/atuned/atuned.cnf** file to **false**, set **engine_tls** in the **/etc/atuned/engine.cnf** file to **false**. + A-Tune is not liable for any consequences incurred if no security certificate is configured. + +- Start the atuned service. + + ```shell + systemctl start atuned + ``` + +- Query the atuned service status. + + ```shell + systemctl status atuned + ``` + + If the following command output is displayed, the service is started successfully: + + ![](./figures/en-us_image_0214540398.png) + +## Starting A-Tune Engine + +To use AI functions, you need to start the A-Tune engine service. + +- Start the atune-engine service. + + ```shell + systemctl start atune-engine + ``` + +- Query the atune-engine service status. + + ```shell + systemctl status atune-engine + ``` + + If the following command output is displayed, the service is started successfully: + + ![](./figures/en-us_image_0245342444.png) + +## Distributed Deployment + +### Purpose of Distributed Deployment + +A-Tune supports distributed deployment to implement distributed architecture and on-demand deployment. The components of A-Tune can be deployed separately. Lightweight component deployment has little impact on services and avoids installing too many dependencies to reduce the system load. + +This document describes only a common deployment mode: deploying the client and server on the same node and deploying the engine module on another node. For details about other deployment modes, contact A-Tune developers. + +**Deployment relationship** +![](figures/picture1.png) + +### Configuration File + +In distributed deployment mode, you need to configure the write the IP address and port number of the engine in the configuration file so that other components can access the engine component through the IP address. + +1. Modify the **/etc/atuned/atuned.cnf** file on the server node. + + - Change the values of **engine_host** and **engine_port** in line 34 to the IP address and port number of the engine node. For the deployment in the preceding figure, the values are **engine_host = 192.168.0.1 engine_port = 3838**. + - Change the values of **rest_tls** and **engine_tls** in lines 49 and 55 to **false**. Otherwise, you need to apply for and configure certificates. You do not need to configure SSL certificates in the test environment. However, you need to configure SSL certificates in the production environment to prevent security risks. + +2. Modify the**/etc/atuned/engine.cnf** file on the engine node. + + - Change the values of **engine_host** and **engine_port** in lines 17 and 18 to the IP address and port number of the engine node. For the deployment in the preceding figure, the value are **engine_host = 192.168.0.1 engine_port = 3838**. + - Change the value of **engine_tls** in line 22 to **false**. + +3. After modifying the configuration file, restart the service for the modification to take effect. + + - Run the `systemctl restart atuned command` on the server node. + - Run the `systemctl restart atune-engine` command on the engine node. + +4. (Optional) Run the `tuning` command in the **A-Tune/examples/tuning/compress** folder. + + - For details, see **A-Tune/examples/tuning/compress/README**. + - Run the `atune-adm tuning --project compress --detail compress_client.yaml` command. + - This step is to check whether the distributed deployment is successful. + +### Precautions + +1. This document does not describe how to configure the authentication certificates. You can set **rest_tls** or **engine_tls** in the **atuned.cnf** and **engine.cnf** files to **false** if necessary. +2. After modifying the configuration file, restart the service. Otherwise, the modification does not take effect. +3. Do not enable the proxy when using A-Tune. +4. The **disk** and **network** items of the **\[system]** section in the **atuned.cnf** file need to be modified. For details about how to modify the items, see the [A-Tune User Guide](https://gitee.com/gaoruoshu/A-Tune/blob/master/Documentation/UserGuide/A-Tune%E7%94%A8%E6%88%B7%E6%8C%87%E5%8D%97.md). + +### Example + +#### atuned.cnf + +```conf +# ...... + +# the tuning optimizer host and port, start by engine.service +# if engine_host is same as rest_host, two ports cannot be same +# the port can be set between 0 to 65535 which not be used +engine_host = 192.168.0.1 +engine_port = 3838 + +# ...... +``` + +#### engine.cnf + +```bash +[server] +# the tuning optimizer host and port, start by engine.service +# if engine_host is same as rest_host, two ports cannot be same +# the port can be set between 0 to 65535 which not be used +engine_host = 192.168.0.1 +engine_port = 3838 +``` + +## Cluster Deployment + +### Purpose of Cluster Deployment + +To support fast tuning in multi-node scenarios, A-Tune supports dynamic tuning of parameter settings on multiple nodes at the same time. In this way, you do not need to tune each node separately, improving tuning efficiency. +Cluster deployment mode consists of one master node and several agent nodes. The client and server are deployed on the master node to receive commands and interact with the engine. Other nodes receive instructions from the master node and configure the parameters of the current node. + +**Deployment relationship** + ![](figures/picture4.png) + +In the preceding figure, the client and server are deployed on the node whose IP address is 192.168.0.0. Project files are stored on this node. Other nodes do not contain project files. +The master node communicates with the agent nodes through TCP. Therefore, you need to modify the configuration file. + +### Modifications to atuned.cnf + +1. Set the value of **protocol** to **tcp**. +2. Set the value of **address** to the IP address of the current node. +3. Set the value of **connect** to the IP addresses of all nodes. The first IP address is the IP address of the master node, and the subsequent IP addresses are the IP addresses of agent nodes. Use commas (,) to separate the IP addresses. +4. During debugging, you can set **rest_tls** and **engine_tls** to **false**. +5. Perform the same modification on the **atuned.cnf** files of all the master and agent nodes. + +### Precautions + +1. The values of **engine_host** and **engine_port** must be consistent in the **engine.cnf** file and the **atuned.cnf** file on the server. +2. This document does not describe how to configure the authentication certificates. You can set **rest_tls** or **engine_tls** in the **atuned.cnf** and **engine.cnf** files to **false** if necessary. +3. After modifying the configuration file, restart the service. Otherwise, the modification does not take effect. +4. Do not enable the proxy when using A-Tune. + +### Example + +#### atuned.cnf + +```conf +# ...... + +[server] +# the protocol grpc server running on +# ranges: unix or tcp +protocol = tcp + +# the address that the grpc server to bind to +# default is unix socket /var/run/atuned/atuned.sock +# ranges: /var/run/atuned/atuned.sock or ip address +address = 192.168.0.0 + +# the atune nodes in cluster mode, separated by commas +# it is valid when protocol is tcp +connect = 192.168.0.0,192.168.0.1,192.168.0.2,192.168.0.3 + +# the atuned grpc listening port +# the port can be set between 0 to 65535 which not be used +port = 60001 + +# the rest service listening port, default is 8383 +# the port can be set between 0 to 65535 which not be used +rest_host = localhost +rest_port = 8383 + +# the tuning optimizer host and port, start by engine.service +# if engine_host is same as rest_host, two ports cannot be same +# the port can be set between 0 to 65535 which not be used +engine_host = 192.168.1.1 +engine_port = 3838 + +# ...... +``` + +#### engine.cnf + +```bash +[server] +# the tuning optimizer host and port, start by engine.service +# if engine_host is same as rest_host, two ports cannot be same +# the port can be set between 0 to 65535 which not be used +engine_host = 192.168.1.1 +engine_port = 3838 +``` + +Note: For details about the **engine.cnf** file, see the configuration file for distributed deployment. diff --git a/docs/en/server/performance/system_optimization/atune/native-turbo.md b/docs/en/server/performance/system_optimization/atune/native-turbo.md new file mode 100644 index 0000000000000000000000000000000000000000..0abd1b3e503143f89e99faedd06cd0ac17a42110 --- /dev/null +++ b/docs/en/server/performance/system_optimization/atune/native-turbo.md @@ -0,0 +1,54 @@ +# Native-Turbo + +## Overview + +The code segment and data segment of a large program can reach hundreds of MB, and the TLB miss rate of key service processes is high. The size of the kernel page table affects the performance. + +To facilitate the use of huge pages, the Native-Turbo feature enables the system to automatically use huge pages when loading programs. Huge pages can be used for code segments and data segments. + +## How to Use + +1. Enable the feature. + + This feature has two levels of switches. `sysctl fs.exec-use-hugetlb` determines whether to enable this feature in the system. (This command can only be run by the **root** user. The value `0` indicates that this feature is disabled, and the value `1` indicates that this feature is enabled. Other values are invalid.) + + If not enabled, this feature will not be used even if users set environment variables because the kernel will ignore related processes. + + After this feature is enabled, common users can use the environment variable `HUGEPAGE_PROBE` to determine whether to use huge pages for running programs. If the value is `1`, huge pages are used. If the value is not set, huge pages are not used. + + ```shell + sysctl fs.exec-use-hugetlb=1 # The main program uses huge pages. + export HUGEPAGE_PROBE=1 # The dynamic library uses huge pages. + ``` + + You can also configure the environment variable `LD_HUGEPAGE_LIB=1` to force all segments to use huge pages. + +2. Mark the segments that need to use huge pages. By default, all segments are marked. `-x` only marks code segments. `-d` clears existing marks. + + ```shell + hugepageedit [-x] [-d] app + ``` + + This tool is provided by the glibc-devel package. + +3. Run the application. + + ./app + +## Restrictions + +1. The program and dynamic library must be compiled in 2 MB alignment mode by adding the following GCC compilation parameters: + + ```shell + -zcommon-page-size=0x200000 -zmax-page-size=0x200000 + ``` + +2. Sufficient huge pages must be reserved before use. Otherwise, the program will fail to be executed. + + If the cgroup is used, pay attention to the `hugetlb` limit. If the limit is less than the number of required huge pages, the system may break down during running. + +3. The size of the process page table is changed to 2 MB. Therefore, the parameters invoked by the system such as `mprotect` must be aligned by 2 MB. Otherwise, the execution will fail. + +4. The LibcarePlus hot patch mechanism is not supported. + +5. Huge pages cannot be shared among multiple processes because they will consume multiple times of memory. diff --git a/docs/en/server/performance/system_optimization/atune/public_sys-resources/icon-note.gif b/docs/en/server/performance/system_optimization/atune/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/server/performance/system_optimization/atune/public_sys-resources/icon-note.gif differ diff --git a/docs/en/server/performance/system_optimization/atune/usage-instructions.md b/docs/en/server/performance/system_optimization/atune/usage-instructions.md new file mode 100644 index 0000000000000000000000000000000000000000..eeb1a37e6ad2f6cb3dd8bdbde7adce3068057095 --- /dev/null +++ b/docs/en/server/performance/system_optimization/atune/usage-instructions.md @@ -0,0 +1,757 @@ +# Usage Instructions + +You can use functions provided by A-Tune through the CLI client atune-adm. This chapter describes the functions and usage of the A-Tune client. + +- [Usage Instructions](#usage-instructions) + - [Overview](#overview) + - [Querying Workload Types](#querying-workload-types) + - [list](#list) + - [Workload Type Analysis and Auto Optimization](#workload-type-analysis-and-auto-optimization) + - [analysis](#analysis) + - [User-defined Model](#user-defined-model) + - [define](#define) + - [collection](#collection) + - [train](#train) + - [undefine](#undefine) + - [Querying Profiles](#querying-profiles) + - [info](#info) + - [Updating a Profile](#updating-a-profile) + - [update](#update) + - [Activating a Profile](#activating-a-profile) + - [profile](#profile) + - [Rolling Back Profiles](#rolling-back-profiles) + - [rollback](#rollback) + - [Updating Database](#updating-database) + - [upgrade](#upgrade) + - [Querying System Information](#querying-system-information) + - [check](#check) + - [Automatic Parameter Optimization](#automatic-parameter-optimization) + - [Tuning](#tuning) + +## Overview + +- You can run the **atune-adm help/--help/-h** command to query commands supported by atune-adm. +- The **define**, **update**, **undefine**, **collection**, **train**, and **upgrade**commands do not support remote execution. +- In the command format, brackets \(\[\]\) indicate that the parameter is optional, and angle brackets \(<\>\) indicate that the parameter is mandatory. The actual parameters prevail. + +## Querying Workload Types + +### list + +#### Function + +Query the supported profiles, and the values of Active. + +#### Format + +**atune-adm list** + +#### Example + +```shell +# atune-adm list + +Support profiles: ++------------------------------------------------+-----------+ +| ProfileName | Active | ++================================================+===========+ +| arm-native-android-container-robox | false | ++------------------------------------------------+-----------+ +| basic-test-suite-euleros-baseline-fio | false | ++------------------------------------------------+-----------+ +| basic-test-suite-euleros-baseline-lmbench | false | ++------------------------------------------------+-----------+ +| basic-test-suite-euleros-baseline-netperf | false | ++------------------------------------------------+-----------+ +| basic-test-suite-euleros-baseline-stream | false | ++------------------------------------------------+-----------+ +| basic-test-suite-euleros-baseline-unixbench | false | ++------------------------------------------------+-----------+ +| basic-test-suite-speccpu-speccpu2006 | false | ++------------------------------------------------+-----------+ +| basic-test-suite-specjbb-specjbb2015 | false | ++------------------------------------------------+-----------+ +| big-data-hadoop-hdfs-dfsio-hdd | false | ++------------------------------------------------+-----------+ +| big-data-hadoop-hdfs-dfsio-ssd | false | ++------------------------------------------------+-----------+ +| big-data-hadoop-spark-bayesian | false | ++------------------------------------------------+-----------+ +| big-data-hadoop-spark-kmeans | false | ++------------------------------------------------+-----------+ +| big-data-hadoop-spark-sql1 | false | ++------------------------------------------------+-----------+ +| big-data-hadoop-spark-sql10 | false | ++------------------------------------------------+-----------+ +| big-data-hadoop-spark-sql2 | false | ++------------------------------------------------+-----------+ +| big-data-hadoop-spark-sql3 | false | ++------------------------------------------------+-----------+ +| big-data-hadoop-spark-sql4 | false | ++------------------------------------------------+-----------+ +| big-data-hadoop-spark-sql5 | false | ++------------------------------------------------+-----------+ +| big-data-hadoop-spark-sql6 | false | ++------------------------------------------------+-----------+ +| big-data-hadoop-spark-sql7 | false | ++------------------------------------------------+-----------+ +| big-data-hadoop-spark-sql8 | false | ++------------------------------------------------+-----------+ +| big-data-hadoop-spark-sql9 | false | ++------------------------------------------------+-----------+ +| big-data-hadoop-spark-tersort | false | ++------------------------------------------------+-----------+ +| big-data-hadoop-spark-wordcount | false | ++------------------------------------------------+-----------+ +| cloud-compute-kvm-host | false | ++------------------------------------------------+-----------+ +| database-mariadb-2p-tpcc-c3 | false | ++------------------------------------------------+-----------+ +| database-mariadb-4p-tpcc-c3 | false | ++------------------------------------------------+-----------+ +| database-mongodb-2p-sysbench | false | ++------------------------------------------------+-----------+ +| database-mysql-2p-sysbench-hdd | false | ++------------------------------------------------+-----------+ +| database-mysql-2p-sysbench-ssd | false | ++------------------------------------------------+-----------+ +| database-postgresql-2p-sysbench-hdd | false | ++------------------------------------------------+-----------+ +| database-postgresql-2p-sysbench-ssd | false | ++------------------------------------------------+-----------+ +| default-default | false | ++------------------------------------------------+-----------+ +| docker-mariadb-2p-tpcc-c3 | false | ++------------------------------------------------+-----------+ +| docker-mariadb-4p-tpcc-c3 | false | ++------------------------------------------------+-----------+ +| hpc-gatk4-human-genome | false | ++------------------------------------------------+-----------+ +| in-memory-database-redis-redis-benchmark | false | ++------------------------------------------------+-----------+ +| middleware-dubbo-dubbo-benchmark | false | ++------------------------------------------------+-----------+ +| storage-ceph-vdbench-hdd | false | ++------------------------------------------------+-----------+ +| storage-ceph-vdbench-ssd | false | ++------------------------------------------------+-----------+ +| virtualization-consumer-cloud-olc | false | ++------------------------------------------------+-----------+ +| virtualization-mariadb-2p-tpcc-c3 | false | ++------------------------------------------------+-----------+ +| virtualization-mariadb-4p-tpcc-c3 | false | ++------------------------------------------------+-----------+ +| web-apache-traffic-server-spirent-pingpo | false | ++------------------------------------------------+-----------+ +| web-nginx-http-long-connection | true | ++------------------------------------------------+-----------+ +| web-nginx-https-short-connection | false | ++------------------------------------------------+-----------+ +``` + +> [!NOTE]NOTE +> If the value of Active is **true**, the profile is activated. In the example, the profile of web-nginx-http-long-connection is activated. + +## Workload Type Analysis and Auto Optimization + +### analysis + +#### Function + +Collect real-time statistics from the system to identify and automatically optimize workload types. + +#### Format + +**atune-adm analysis** \[OPTIONS\] + +#### Parameter Description + +- OPTIONS + +| Parameter | Description | +| ------------------------ | ---------------------------------------------------------------------------------------------- | +| --model, -m | New model generated after user self-training | +| --characterization, -c | Use the default model for application identification and do not perform automatic optimization | +| --times value, -t value | Time duration for data collection | +| --script value, -s value | File to be executed | + +#### Example + +- Use the default model for application identification. + + ```shell + # atune-adm analysis --characterization + ``` + +- Use the default model to identify applications and perform automatic tuning. + + ```shell + # atune-adm analysis + ``` + +- Use the user-defined training model for recognition. + + ```shell + # atune-adm analysis --model /usr/libexec/atuned/analysis/models/new-model.m + ``` + +## User-defined Model + +A-Tune allows users to define and learn new models. To define a new model, perform the following steps: + +1. Run the **define** command to define a new profile. +2. Run the **collection** command to collect the system data corresponding to the application. +3. Run the **train** command to train the model. + +### define + +#### Function + +Add a user-defined application scenarios and the corresponding profile tuning items. + +#### Format + +**atune-adm define** \ \ \ \ + +#### Example + +Add a profile whose service_type is **test_service**, application_name is **test_app**, scenario_name is **test_scenario**, and tuning item configuration file is **example.conf**. + +```shell +# atune-adm define test_service test_app test_scenario ./example.conf +``` + +The **example.conf** file can be written as follows (the following optimization items are optional and are for reference only). You can also run the **atune-adm info** command to view how the existing profile is written. + +```ini + [main] + # list its parent profile + [kernel_config] + # to change the kernel config + [bios] + # to change the bios config + [bootloader.grub2] + # to change the grub2 config + [sysfs] + # to change the /sys/* config + [systemctl] + # to change the system service status + [sysctl] + # to change the /proc/sys/* config + [script] + # the script extension of cpi + [ulimit] + # to change the resources limit of user + [schedule_policy] + # to change the schedule policy + [check] + # check the environment + [tip] + # the recommended optimization, which should be performed manunaly +``` + +### collection + +#### Function + +Collect the global resource usage and OS status information during service running, and save the collected information to a CSV output file as the input dataset for model training. + +> [!NOTE]NOTE +> +> - This command depends on the sampling tools such as perf, mpstat, vmstat, iostat, and sar. +> - Currently, only the Kunpeng 920 CPU is supported. You can run the **dmidecode -t processor** command to check the CPU model. + +#### Format + +**atune-adm collection** + +#### Parameter Description + +- OPTIONS + +| Parameter | Description | +| ----------------- | ----------------------------------------------------------------------------------------------------- | +| --filename, -f | Name of the generated CSV file used for training: *name*-*timestamp*.csv | +| --output_path, -o | Path for storing the generated CSV file. The absolute path is required. | +| --disk, -b | Disk used during service running, for example, /dev/sda. | +| --network, -n | Network port used during service running, for example, eth0. | +| --app_type, -t | Mark the application type of the service as a label for training. | +| --duration, -d | Data collection time during service running, in seconds. The default collection time is 1200 seconds. | +| --interval, -i | Interval for collecting data, in seconds. The default interval is 5 seconds. | + +#### Example + +```shell +# atune-adm collection --filename name --interval 5 --duration 1200 --output_path /home/data --disk sda --network eth0 --app_type test_service-test_app-test_scenario +``` + +> Note: +> +> In the example, data is collected every 5 seconds for a duration of 1200 seconds. The collected data is stored as the *name* file in the **/home/data** directory. The application type of the service is defined by the `atune-adm define` command, which is **test_service-test_app-test_scenario** in this example. +> The data collection interval and duration can be specified using the preceding command options. + +### train + +#### Function + +Use the collected data to train the model. Collect data of at least two application types during training. Otherwise, an error is reported. + +#### Format + +**atune-adm train** + +#### Parameter Description + +- OPTIONS + + | Parameter | Description | + | ----------------- | ------------------------------------------------------ | + | --data_path, -d | Path for storing CSV files required for model training | + | --output_file, -o | Model generated through training | + +#### Example + +Use the CSV file in the **data** directory as the training input. The generated model **new-model.m** is stored in the **model** directory. + +```shell +# atune-adm train --data_path /home/data --output_file /usr/libexec/atuned/analysis/models/new-model.m +``` + +### undefine + +#### Function + +Delete a user-defined profile. + +#### Format + +**atune-adm undefine** + +#### Example + +Delete the user-defined profile. + +```shell +# atune-adm undefine test_service-test_app-test_scenario +``` + +## Querying Profiles + +### info + +#### Function + +View the profile content. + +#### Format + +**atune-adm info** + +#### Example + +View the profile content of web-nginx-http-long-connection. + +```shell +# atune-adm info web-nginx-http-long-connection + +*** web-nginx-http-long-connection: + +# +# nginx http long connection A-Tune configuration +# +[main] +include = default-default + +[kernel_config] +#TODO CONFIG + +[bios] +#TODO CONFIG + +[bootloader.grub2] +iommu.passthrough = 1 + +[sysfs] +#TODO CONFIG + +[systemctl] +sysmonitor = stop +irqbalance = stop + +[sysctl] +fs.file-max = 6553600 +fs.suid_dumpable = 1 +fs.aio-max-nr = 1048576 +kernel.shmmax = 68719476736 +kernel.shmall = 4294967296 +kernel.shmmni = 4096 +kernel.sem = 250 32000 100 128 +net.ipv4.tcp_tw_reuse = 1 +net.ipv4.tcp_syncookies = 1 +net.ipv4.ip_local_port_range = 1024 65500 +net.ipv4.tcp_max_tw_buckets = 5000 +net.core.somaxconn = 65535 +net.core.netdev_max_backlog = 262144 +net.ipv4.tcp_max_orphans = 262144 +net.ipv4.tcp_max_syn_backlog = 262144 +net.ipv4.tcp_timestamps = 0 +net.ipv4.tcp_synack_retries = 1 +net.ipv4.tcp_syn_retries = 1 +net.ipv4.tcp_fin_timeout = 1 +net.ipv4.tcp_keepalive_time = 60 +net.ipv4.tcp_mem = 362619 483495 725238 +net.ipv4.tcp_rmem = 4096 87380 6291456 +net.ipv4.tcp_wmem = 4096 16384 4194304 +net.core.wmem_default = 8388608 +net.core.rmem_default = 8388608 +net.core.rmem_max = 16777216 +net.core.wmem_max = 16777216 + +[script] +prefetch = off +ethtool = -X {network} hfunc toeplitz + +[ulimit] +{user}.hard.nofile = 102400 +{user}.soft.nofile = 102400 + +[schedule_policy] +#TODO CONFIG + +[check] +#TODO CONFIG + +[tip] +SELinux provides extra control and security features to linux kernel. Disabling SELinux will improve the performance but may cause security risks. = kernel +disable the nginx log = application +``` + +## Updating a Profile + +You can update the existing profile as required. + +### update + +#### Function + +Update the original tuning items in the existing profile to the content in the **new.conf** file. + +#### Format + +**atune-adm update** + +#### Example + +Change the tuning item of the profile named **test_service-test_app-test_scenario** to **new.conf**. + +```shell +# atune-adm update test_service-test_app-test_scenario ./new.conf +``` + +## Activating a Profile + +### profile + +#### Function + +Manually activate the profile to make it in the active state. + +#### Format + +**atune-adm profile** + +#### Parameter Description + +For details about the profile name, see the query result of the list command. + +#### Example + +Activate the profile corresponding to the web-nginx-http-long-connection. + +```shell +# atune-adm profile web-nginx-http-long-connection +``` + +## Rolling Back Profiles + +### rollback + +#### Functions + +Roll back the current configuration to the initial configuration of the system. + +#### Format + +**atune-adm rollback** + +#### Example + +```shell +# atune-adm rollback +``` + +## Updating Database + +### upgrade + +#### Function + +Update the system database. + +#### Format + +**atune-adm upgrade** + +#### Parameter Description + +- DB\_FILE + + New database file path. + +#### Example + +The database is updated to **new\_sqlite.db**. + +```shell +# atune-adm upgrade ./new_sqlite.db +``` + +## Querying System Information + +### check + +#### Function + +Check the CPU, BIOS, OS, and NIC information. + +#### Format + +**atune-adm check** + +#### Example + +```shell +# atune-adm check + cpu information: + cpu:0 version: Kunpeng 920-6426 speed: 2600000000 HZ cores: 64 + cpu:1 version: Kunpeng 920-6426 speed: 2600000000 HZ cores: 64 + system information: + DMIBIOSVersion: 0.59 + OSRelease: 4.19.36-vhulk1906.3.0.h356.eulerosv2r8.aarch64 + network information: + name: eth0 product: HNS GE/10GE/25GE RDMA Network Controller + name: eth1 product: HNS GE/10GE/25GE Network Controller + name: eth2 product: HNS GE/10GE/25GE RDMA Network Controller + name: eth3 product: HNS GE/10GE/25GE Network Controller + name: eth4 product: HNS GE/10GE/25GE RDMA Network Controller + name: eth5 product: HNS GE/10GE/25GE Network Controller + name: eth6 product: HNS GE/10GE/25GE RDMA Network Controller + name: eth7 product: HNS GE/10GE/25GE Network Controller + name: docker0 product: +``` + +## Automatic Parameter Optimization + +A-Tune provides the automatic search capability with the optimal configuration, saving the trouble of manually configuring parameters and performance evaluation. This greatly improves the search efficiency of optimal configurations. + +### Tuning + +#### Function + +Use the specified project file to search the dynamic space for parameters and find the optimal solution under the current environment configuration. + +#### Format + +**atune-adm tuning** \[OPTIONS\] + +> [!NOTE]NOTE +Before running the command, ensure that the following conditions are met: + +1. The YAML configuration file on the server has been edited and stored in the **/etc/atuned/tuning/** directory of the atuned service. +2. The YAML configuration file of the client has been edited and stored on the atuned client. + +#### Parameter Description + +- OPTIONS + +| Parameter | Description | +| ------------- | ----------------------------------------------------------- | +| --restore, -r | Restores the initial configuration before tuning. | +| --project, -p | Specifies the project name in the YAML file to be restored. | +| --restart, -c | Perform tuning based on historical tuning results. | +| --detail, -d | Print detailed information about the tuning process. | + +> [!NOTE]NOTE +> If this parameter is used, the -p parameter must be followed by a specific project name and the YAML file of the project must be specified. + +- **PROJECT\_YAML**: YAML configuration file of the client. + +#### Configuration Description + +**Table 1** YAML file on the server + +| Name | Description | Type | Value Range | +| ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | ----------- | +| project | Project name. | Character string | - | +| startworkload | Script for starting the service to be optimized. | Character string | - | +| stopworkload | Script for stopping the service to be optimized. | Character string | - | +| maxiterations | Maximum number of optimization iterations, which is used to limit the number of iterations on the client. Generally, the more optimization iterations, the better the optimization effect, but the longer the time required. Set this parameter based on the site requirements. | Integer | >10 | +| object | Parameters to be optimized and related information.
For details about the object configuration items, see Table 2. | | | + +**Table 2** Description of object configuration items + +| Name | Description | Type | Value Range | +| ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | ------------------------------------------------------------------------------- | +| name | Parameter to be optimized. | Character string | - | +| desc | Description of parameters to be optimized. | Character string | - | +| get | Script for querying parameter values. | - | - | +| set | Script for setting parameter values. | - | - | +| needrestart | Specifies whether to restart the service for the parameter to take effect. | Enumeration | **true** or **false** | +| type | Parameter type. Currently, the **discrete** and **continuous** types are supported. | Enumeration | **discrete** or **continuous** | +| dtype | This parameter is available only when type is set to **discrete**. Currently, **int**, **float** and **string** are supported. | Enumeration | int, float, string | +| scope | Parameter setting range. This parameter is valid only when type is set to discrete and **dtype** is set to **int** or **float**, or **type** is set to **continuous**. | Integer/Float | The value is user-defined and must be within the valid range of this parameter. | +| step | Parameter value step, which is used when **dtype** is set to **int** or **float**. | Integer/Float | This value is user-defined. | +| items | Enumerated value of which the parameter value is not within the scope. This is used when **dtype** is set to **int** or **float**. | Integer/Float | The value is user-defined and must be within the valid range of this parameter. | +| options | Enumerated value range of the parameter value, which is used when **dtype** is set to **string**. | Character string | The value is user-defined and must be within the valid range of this parameter. | + +**Table 3** Description of configuration items of a YAML file on the client + +| Name | Description | Type | Value Range | +| --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | ------------------------------------------------- | +| project | Project name, which must be the same as that in the configuration file on the server. | Character string | - | +| engine | Tuning algorithm. | Character string | "random", "forest", "gbrt", "bayes", "extraTrees" | +| iterations | Number of optimization iterations. | Integer | ≥ 10 | +| random_starts | Number of random iterations. | Integer | < iterations | +| feature_filter_engine | Parameter search algorithm, which is used to select important parameters. This parameter is optional. | Character string | "lhs" | +| feature_filter_cycle | Parameter search cycles, which is used to select important parameters. This parameter is used together with feature_filter_engine. | Integer | - | +| feature_filter_iters | Number of iterations for each cycle of parameter search, which is used to select important parameters. This parameter is used together with feature_filter_engine. | Integer | - | +| split_count | Number of evenly selected parameters in the value range of tuning parameters, which is used to select important parameters. This parameter is used together with feature_filter_engine. | Integer | - | +| benchmark | Performance test script. | - | - | +| evaluations | Performance test evaluation index.
For details about the evaluations configuration items, see Table 4. | - | - | + +**Table 4** Description of evaluations configuration item + +| Name | Description | Type | Value Range | +| --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | ---------------------------- | +| name | Evaluation index name. | Character string | - | +| get | Script for obtaining performance evaluation results. | - | - | +| type | Specifies a **positive** or **negative** type of the evaluation result. The value **positive** indicates that the performance value is minimized, and the value **negative** indicates that the performance value is maximized. | Enumeration | **positive** or **negative** | +| weight | Weight of the index. The value ranges from 0 to 100. | Integer | 0-100 | +| threshold | Minimum performance requirement of the index. | Integer | User-defined | + +#### Example + +The following is an example of the YAML file configuration on a server: + +```yaml +project: "compress" +maxiterations: 500 +startworkload: "" +stopworkload: "" +object : + - + name : "compressLevel" + info : + desc : "The compresslevel parameter is an integer from 1 to 9 controlling the level of compression" + get : "cat /root/A-Tune/examples/tuning/compress/compress.py | grep 'compressLevel=' | awk -F '=' '{print $2}'" + set : "sed -i 's/compressLevel=\\s*[0-9]*/compressLevel=$value/g' /root/A-Tune/examples/tuning/compress/compress.py" + needrestart : "false" + type : "continuous" + scope : + - 1 + - 9 + dtype : "int" + - + name : "compressMethod" + info : + desc : "The compressMethod parameter is a string controlling the compression method" + get : "cat /root/A-Tune/examples/tuning/compress/compress.py | grep 'compressMethod=' | awk -F '=' '{print $2}' | sed 's/\"//g'" + set : "sed -i 's/compressMethod=\\s*[0-9,a-z,\"]*/compressMethod=\"$value\"/g' /root/A-Tune/examples/tuning/compress/compress.py" + needrestart : "false" + type : "discrete" + options : + - "bz2" + - "zlib" + - "gzip" + dtype : "string" +``` + +The following is an example of the YAML file configuration on a client: + +```yaml +project: "compress" +engine : "gbrt" +iterations : 20 +random_starts : 10 + +benchmark : "python3 /root/A-Tune/examples/tuning/compress/compress.py" +evaluations : + - + name: "time" + info: + get: "echo '$out' | grep 'time' | awk '{print $3}'" + type: "positive" + weight: 20 + - + name: "compress_ratio" + info: + get: "echo '$out' | grep 'compress_ratio' | awk '{print $3}'" + type: "negative" + weight: 80 +``` + +#### Example + +- Download test data. + + ```shell + wget http://cs.fit.edu/~mmahoney/compression/enwik8.zip + ``` + +- Prepare the tuning environment. + + Example of **prepare.sh**: + + ```shell + #!/usr/bin/bash + if [ "$#" -ne 1 ]; then + echo "USAGE: $0 the path of enwik8.zip" + exit 1 + fi + + path=$( + cd "$(dirname "$0")" + pwd + ) + + echo "unzip enwik8.zip" + unzip "$path"/enwik8.zip + + echo "set FILE_PATH to the path of enwik8 in compress.py" + sed -i "s#compress/enwik8#$path/enwik8#g" "$path"/compress.py + + echo "update the client and server yaml files" + sed -i "s#python3 .*compress.py#python3 $path/compress.py#g" "$path"/compress_client.yaml + sed -i "s# compress/compress.py# $path/compress.py#g" "$path"/compress_server.yaml + + echo "copy the server yaml file to /etc/atuned/tuning/" + cp "$path"/compress_server.yaml /etc/atuned/tuning/ + ``` + + Run the script. + + ```shell + sh prepare.sh enwik8.zip + ``` + +- Run the `tuning` command to tune the parameters. + + ```shell + atune-adm tuning --project compress --detail compress_client.yaml + ``` + +- Restore the configuration before running `tuning`. **compress** indicates the project name in the YAML file. + + ```shell + atune-adm tuning --restore --project compress + ``` diff --git a/docs/en/server/performance/tuning_framework/_toc.yaml b/docs/en/server/performance/tuning_framework/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..a174e3e13f4e36ba269c05c15110524dc9b5bcae --- /dev/null +++ b/docs/en/server/performance/tuning_framework/_toc.yaml @@ -0,0 +1,3 @@ +label: Tuning Framework +sections: + - href: ./oeaware/_toc.yaml diff --git a/docs/en/server/performance/tuning_framework/oeaware/_toc.yaml b/docs/en/server/performance/tuning_framework/oeaware/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..3db9a88e1ef65683d8211bc58c2f780a1b36ab3d --- /dev/null +++ b/docs/en/server/performance/tuning_framework/oeaware/_toc.yaml @@ -0,0 +1,6 @@ +label: oeAware User Guide +isManual: true +href: ./oeaware-user-guide.md +description: >- + Intelligent activation of system tuning features based on dynamic sensing of + system behavior diff --git a/docs/en/server/performance/tuning_framework/oeaware/figures/dep-failed.png b/docs/en/server/performance/tuning_framework/oeaware/figures/dep-failed.png new file mode 100644 index 0000000000000000000000000000000000000000..afb4750135657876b455978bf9d8f5eff36be91e Binary files /dev/null and b/docs/en/server/performance/tuning_framework/oeaware/figures/dep-failed.png differ diff --git a/docs/en/server/performance/tuning_framework/oeaware/figures/dep.png b/docs/en/server/performance/tuning_framework/oeaware/figures/dep.png new file mode 100644 index 0000000000000000000000000000000000000000..91388d6a860f032c86c0559b232f2d5ef55a40f8 Binary files /dev/null and b/docs/en/server/performance/tuning_framework/oeaware/figures/dep.png differ diff --git a/docs/en/server/performance/tuning_framework/oeaware/figures/dependency.png b/docs/en/server/performance/tuning_framework/oeaware/figures/dependency.png new file mode 100644 index 0000000000000000000000000000000000000000..0cd087fb0c9095e63aa76e0d2464a92225af2399 Binary files /dev/null and b/docs/en/server/performance/tuning_framework/oeaware/figures/dependency.png differ diff --git a/docs/en/server/performance/tuning_framework/oeaware/oeaware-user-guide.md b/docs/en/server/performance/tuning_framework/oeaware/oeaware-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..a088ae635bfb1bdb2826ed062253cc91eef2c6a6 --- /dev/null +++ b/docs/en/server/performance/tuning_framework/oeaware/oeaware-user-guide.md @@ -0,0 +1,521 @@ +# oeAware User Guide + +## Introduction + +oeAware is a framework for implementing low-load collection, sensing, and tuning on openEuler. It aims to intelligently enable optimization features after dynamically detecting system behaviors. Traditional optimization features run independently and are statically enabled or disabled. oeAware divides optimization into three layers: collection, sensing, and tuning. Each layer is associated through subscription and is developed as plugins. + +## Plugin Description + +**Plugin definition**: Each plugin corresponds to an .so file. Plugins are classified into collection plugins, sensing plugins, and tuning plugins. + +**Instance definition**: The scheduling unit in the service is instance. A plugin contains multiple instances. For example, a collection plugin includes multiple collection items, and each collection item is an instance. + +**Dependencies Between Instances** + +Before running an instance, ensure that the dependency between the instances is met. + +![img](./figures/dependency.png) + +- A collection instance does not depend on any other instance. + +- A sensing instance depends on a collection instance and other sensing instances. + +- A tuning instance depends on a collection instance, sensing instance, and other tuning instances. + +## Installation + +Configure the openEuler Yum repository and run the `yum` commands to install oeAware. on openEuler 22.03 LTS SP4, oeAware has been installed by default. + +```shell +yum install oeAware-manager +``` + +### Service Startup + +Run the `systemd` command to start the service. + +```shell +systemctl start oeaware +``` + +Skip this step + +Configuration file path: **/etc/oeAware/config.yaml** + +```yaml +log_path: /var/log/oeAware # Log storage path +log_level: 1 # Log level. 1: DUBUG; 2: INFO; 3: WARN; 4: ERROR. +enable_list: # Plugins are enabled by default. + - name: libtest.so # Configure the plugin and enable all instances of the plugin. + - name: libtest1.so # Configure the plugin and enable the specified plugin instances. + instances: + - instance1 + - instance2 + ... + ... +plugin_list: # Downloaded packages are supported. + - name: test #The name must be unique. If the name is repeated, the first occurrence is used. + description: hello world + url: https://gitee.com/openeuler/oeAware-manager/raw/master/README.md #url must not be empty. + ... +``` + +After modifying the configuration file, run the following commands to restart the service: + +```shell +systemctl daemon-reload +systemctl restart oeaware +``` + +## Usage + +Start the oeaware service. Then, manage plugins and instances using the `oeawarectl` command, which supports loading, unloading, and querying plugins, along with enabling, disabling, and querying instances. + +### Plugin Loading + +By default, the service loads the plugins in the plugin storage paths. + +Collection plugin path: /usr/lib64/oeAware-plugin/collector + +Sensing plugin path: /usr/lib64/oeAware-plugin/scenario + +Tuning plugin path: /usr/lib64/oeAware-plugin/tune + +You can also manually load the plugins. + +```shell +oeawarectl -l | --load -t | --type # plugin type can be collector, scenario, or tune +``` + +Example + +```shell +[root@localhost ~]# oeawarectl -l libthread_collect.so -t collector +Plugin loaded successfully. +``` + +If the operation fails, an error description is returned. + +### Plugin Unloading + +```shell +oeawarectl -r | --remove +``` + +Example + +```shell +[root@localhost ~]# oeawarectl -r libthread_collect.so +Plugin remove successfully. +``` + +If the operation fails, an error description is returned. + +### Plugin Query + +#### Querying Plugin Status + +```shell +oeawarectl -q # Query all loaded plugins. +oeawarectl --query # Query a specified plugin. +``` + +Example + +```shell +[root@localhost ~]# oeawarectl -q +Show plugins and instances status. +------------------------------------------------------------ +libthread_collector.so + thread_collector(available, close) # Plugin instance and status +libpmu.so + pmu_cycles_sampling(available, close) + pmu_cycles_counting(available, close) + pmu_uncore_counting(available, close) + pmu_spe_sampling(available, close) +libthread_tune.so + thread_tune(available, close) +libthread_scenario.so + thread_scenario(available, close) +------------------------------------------------------------ +format: +[plugin] + [instance]([dependency status], [running status]) +dependency status: available means satisfying dependency, otherwise unavailable. +running status: running means that instance is running, otherwise close. +``` + +If the operation fails, an error description is returned. + +#### Querying Plugin Dependencies + +```shell +oeawarectl -Q # Query the dependency graph of loaded instances. +oeawarectl --query-dep= # Query the dependency graph of a specified instance. +``` + +A **dep.png** file will be generated in the current directory to display the dependencies. + +Example + +Relationship diagram when dependencies are met + +![img](./figures/dep.png) + +Relationship diagram when dependencies are not met + +![img](./figures/dep-failed.png) + +If the operation fails, an error description is returned. + +### Enabling Plugins + +#### Enabling a Plugin Instance + +```shell +oeawarectl -e | --enable +``` + +If the operation fails, an error description is returned. + +#### Disabling a Plugin Instance + +```shell +oeawarectl -d | --disable +``` + +If the operation fails, an error description is returned. + +### Downloading and Installing Plugins + +Use the `--list` command to query the RPM packages that can be downloaded and installed plugins. + +```shell +oeawarectl --list +``` + +The query result is as follows: + +```shell +Supported Packages: # Downloadable packages +[name1] # plugin_list configured in config +[name2] +... +Installed Plugins: # Installed plugins +[name1] +[name2] +... +``` + +Use the `--install` command to download and install the RPM package. + +```shell +oeawarectl -i | --install # Name of a package queried using --list (package in Supported Packages) +``` + +If the operation fails, an error description is returned. + +### Help + +Use the `--help` command for help information. + +```shell +usage: oeawarectl [options]... + options + -l|--load [plugin] load plugin and need plugin type. + -t|--type [plugin_type] assign plugin type. there are three types: + collector: collection plugin. + scenario: awareness plugin. + tune: tune plugin. + -r|--remove [plugin] remove plugin from system. + -e|--enable [instance] enable the plugin instance. + -d|--disable [instance] disable the plugin instance. + -q query all plugins information. + --query [plugin] query the plugin information. + -Q query all instances dependencies. + --query-dep [instance] query the instance dependency. + --list the list of supported plugins. + -i|--install [plugin] install plugin from the list. + --help show this help message. +``` + +## Plugin Development + +### Common Data Structures of Plugins + +```c +struct DataBuf { + int len; + void *data; +}; +``` + +**struct DataBuf** is the data buffer. + +- **data**: specific data. **data** is an array. The data type can be defined as required. +- len: size of **data**. + +```c +struct DataRingBuf { + const char *instance_name; + int index; + uint64_t count; + struct DataBuf *buf; + int buf_len; +}; +``` + +**struct DataRingBuf** facilitates data transfer between plugins, primarily utilizing a circular buffer. + +- **instance_name**: instance of the incoming data. For instance, when data reaches a perception plugin, it distinguishes which collection item belongs to which collection plugin. + +- **index**: current data write position. For example, after each data collection, the index increments. + +- **count**: execution count of the instance, continuously accumulating. + +- **buf**: data buffer. Some collection items require multiple samplings before the perception plugin processes them, so the buf array stores these samples. + +- **buf_len**: size of the data buffer. Once the buffer is initialized, **buf_len** remains constant. + +```C +struct Param { + const struct DataRingBuf **ring_bufs; + int len; +}; +``` + +- **ring_bufs**: data required by the instance, sourced from other instances. +- **len**: length of the **ring_bufs** array. + +### Instance Interfaces + +```C +struct Interface { + const char* (*get_version)(); + /* The instance name is a unique identifier in the system. */ + const char* (*get_name)(); + const char* (*get_description)(); + /* Specifies the instance dependencies, which is used as the input information + * for instance execution. + */ + const char* (*get_dep)(); + /* Instance scheduling priority. In a uniform time period, a instance with a + * lower priority is scheduled first. + */ + int (*get_priority)(); + int (*get_type)(); + /* Instance execution period. */ + int (*get_period)(); + bool (*enable)(); + void (*disable)(); + const struct DataRingBuf* (*get_ring_buf)(); + void (*run)(const struct Param*); +}; +``` + +```c +int get_instance(Interface **interface); +``` + +Every plugin includes a **get_instance** function to provide instances to the framework. + +Obtaining the version number + +1. Interface definition + + ```c + char* (*get_version)(); + ``` + +2. Interface description + +3. Parameter description + +4. Return value description + + The specific version number is returned. This interface is reserved. + +Obtaining the instance name + +1. Interface definition + + ```c + char* (*get_name)(); + ``` + +2. Interface description + + Obtains the name of an instance. When you run the `-q` command on the client, the instance name is displayed. In addition, you can run the `--enable` command to enable the instance. + +3. Parameter description + +4. Return value description + + The name of the instance is returned. Ensure that the instance name is unique. + +Obtaining description information + +1. Interface definition + + ```c + char* (*get_description)(); + ``` + +2. Interface description + +3. Parameter description + +4. Return value description + + The detailed description is returned. This interface is reserved. + +Obtaining the type + +1. Interface definition + + ```c + char* (*get_type)(); + ``` + +2. Interface description + +3. Parameter description + +4. Return value description + + The specific type information is returned. This interface is reserved. + +Obtaining the sampling period + +1. Interface definition + + ```c + int (*get_cycle)(); + ``` + +2. Interface description + + Obtains the sampling period. Different collection items can use different collection periods. + +3. Parameter description + +4. Return value description + + The specific sampling period is returned. The unit is ms. + +Obtaining dependencies + +1. Interface definition + + ```c + char* (*get_dep)(); + ``` + +2. Interface description + +3. Parameter description + +4. Return value description + + Information about the dependent instances is returned. This interface is reserved. + +Enabling an instance + +1. Interface definition + + ```c + void (*enable)(); + ``` + +2. Interface description + + Enables an instance. + +3. Parameter description + +4. Return value description + +Disabling an instance + +1. Interface definition + + ```c + void (*disable)(); + ``` + +2. Interface description + + Disables an instance. + +3. Parameter description + +4. Return value description + +Obtaining the data buffer + +1. Interface definition + + ```c + const DataRingBuf* (*get_ring_buf)(); + ``` + +2. Interface description + + Obtains the buffer management pointer of the collection data (the memory is applied for by the plugin). The pointer is used by sensing plugins. + +3. Parameter description + +4. Return value description + + The **struct DataRingBuf** management pointer is returned. + +Executing an instance + +1. Interface definition + + ```c + void (*run)(const Param*); + ``` + +2. Interface description + + Runs at regular intervals according to the execution cycle. + +3. Parameter description + + Contains the data necessary for the instance to execute. + +4. Return value description + +## Supported Plugins + +- **libpmu.so**: collects PMU-related data. +- **libthread_collector.so**: gathers thread information within the system. +- **libthread_scenario.so**: monitors details of a specific thread. +- **libthread_tune.so**: enhances UnixBench performance. +- **libsmc_tune.so**: enables SMC acceleration for seamless TCP protocol performance improvements. +- **libtune_numa.so**: optimizes cross-NUMA node memory access to boost system performance. + +## Constraints + +### Function Constraints + +By default, oeAware integrates the libkperf module for collecting Arm microarchitecture information. This module can be called by only one process at a time. If this module is called by other processes or the perf command is used, conflicts may occur. + +### Operation Constraints + +Currently, only the **root** user can operate oeAware. + +## Notes + +The user group and permission of the oeAware configuration file and plugins are strictly verified. Do not modify the permissions and user group of oeAware-related files. + +Permissions: + +- Plugin files: 440 + +- Client executable file: 750 + +- Server executable file: 750 + +- Service configuration file: 640 diff --git a/docs/en/server/quickstart/quickstart/_toc.yaml b/docs/en/server/quickstart/quickstart/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..20432c67e0a1bcb059016167ccc54d65740244f0 --- /dev/null +++ b/docs/en/server/quickstart/quickstart/_toc.yaml @@ -0,0 +1,6 @@ +label: Quick Start +isManual: true +description: Quickly install and use openEuler. +sections: + - label: Quick Start + href: ./quick-start.md diff --git a/docs/en/server/quickstart/quickstart/figures/Installation_wizard.png b/docs/en/server/quickstart/quickstart/figures/Installation_wizard.png new file mode 100644 index 0000000000000000000000000000000000000000..fc3a96c0cd4b5a2ece94a0b3fc484720440adace Binary files /dev/null and b/docs/en/server/quickstart/quickstart/figures/Installation_wizard.png differ diff --git a/docs/en/server/quickstart/quickstart/figures/advanced-user-configuration.png b/docs/en/server/quickstart/quickstart/figures/advanced-user-configuration.png new file mode 100644 index 0000000000000000000000000000000000000000..59a188aece92ad19cc9b42f69e235d9a9d4f702a Binary files /dev/null and b/docs/en/server/quickstart/quickstart/figures/advanced-user-configuration.png differ diff --git a/docs/en/server/quickstart/quickstart/figures/creating-a-user.png b/docs/en/server/quickstart/quickstart/figures/creating-a-user.png new file mode 100644 index 0000000000000000000000000000000000000000..0e2befb0832d1167f5ffdcafdf7d9952d9ccdfbe Binary files /dev/null and b/docs/en/server/quickstart/quickstart/figures/creating-a-user.png differ diff --git a/docs/en/server/quickstart/quickstart/figures/drive-icon.png b/docs/en/server/quickstart/quickstart/figures/drive-icon.png new file mode 100644 index 0000000000000000000000000000000000000000..b41fcb09dfbf805da4863142855e7c2de4bf4c7b Binary files /dev/null and b/docs/en/server/quickstart/quickstart/figures/drive-icon.png differ diff --git a/docs/en/server/quickstart/quickstart/figures/en-us_image_0229420473.png b/docs/en/server/quickstart/quickstart/figures/en-us_image_0229420473.png new file mode 100644 index 0000000000000000000000000000000000000000..86c61a4b8e2a5795baff2fc74629924d01d7b97b Binary files /dev/null and b/docs/en/server/quickstart/quickstart/figures/en-us_image_0229420473.png differ diff --git a/docs/en/server/quickstart/quickstart/figures/image-dialog-box.png b/docs/en/server/quickstart/quickstart/figures/image-dialog-box.png new file mode 100644 index 0000000000000000000000000000000000000000..caeb56bb46f766dd39d66a65e308c591954d32cf Binary files /dev/null and b/docs/en/server/quickstart/quickstart/figures/image-dialog-box.png differ diff --git a/docs/en/server/quickstart/quickstart/figures/installation-process.png b/docs/en/server/quickstart/quickstart/figures/installation-process.png new file mode 100644 index 0000000000000000000000000000000000000000..2d219c7605ee75e73dffba1e2dd7c277968d4801 Binary files /dev/null and b/docs/en/server/quickstart/quickstart/figures/installation-process.png differ diff --git a/docs/en/server/quickstart/quickstart/figures/installation-summary.png b/docs/en/server/quickstart/quickstart/figures/installation-summary.png new file mode 100644 index 0000000000000000000000000000000000000000..d5ca555a2b2291e139b67098a7c23d29b23b8b24 Binary files /dev/null and b/docs/en/server/quickstart/quickstart/figures/installation-summary.png differ diff --git a/docs/en/server/quickstart/quickstart/figures/password-of-the-root-account.png b/docs/en/server/quickstart/quickstart/figures/password-of-the-root-account.png new file mode 100644 index 0000000000000000000000000000000000000000..fe65e73a81e25e5fa90a13af707165911e7fa459 Binary files /dev/null and b/docs/en/server/quickstart/quickstart/figures/password-of-the-root-account.png differ diff --git a/docs/en/server/quickstart/quickstart/figures/restart-icon.png b/docs/en/server/quickstart/quickstart/figures/restart-icon.png new file mode 100644 index 0000000000000000000000000000000000000000..a1b02b2dff42c90845d2491192507ea6967352e3 Binary files /dev/null and b/docs/en/server/quickstart/quickstart/figures/restart-icon.png differ diff --git a/docs/en/server/quickstart/quickstart/figures/selecting-a-language.png b/docs/en/server/quickstart/quickstart/figures/selecting-a-language.png new file mode 100644 index 0000000000000000000000000000000000000000..eaeb26ca208778822bf591782a689569339c3552 Binary files /dev/null and b/docs/en/server/quickstart/quickstart/figures/selecting-a-language.png differ diff --git a/docs/en/server/quickstart/quickstart/figures/selecting-installation-software.png b/docs/en/server/quickstart/quickstart/figures/selecting-installation-software.png new file mode 100644 index 0000000000000000000000000000000000000000..c246e997d787d0d6a0439dcaf8780a09a9b72ca7 Binary files /dev/null and b/docs/en/server/quickstart/quickstart/figures/selecting-installation-software.png differ diff --git a/docs/en/server/quickstart/quickstart/figures/setting-the-boot-device.png b/docs/en/server/quickstart/quickstart/figures/setting-the-boot-device.png new file mode 100644 index 0000000000000000000000000000000000000000..42455bcd651b98a08b012b275d5f170daf07ac59 Binary files /dev/null and b/docs/en/server/quickstart/quickstart/figures/setting-the-boot-device.png differ diff --git a/docs/en/server/quickstart/quickstart/figures/setting-the-installation-destination.png b/docs/en/server/quickstart/quickstart/figures/setting-the-installation-destination.png new file mode 100644 index 0000000000000000000000000000000000000000..224f165b222598aa140187bdfa9b1e75af36c0c5 Binary files /dev/null and b/docs/en/server/quickstart/quickstart/figures/setting-the-installation-destination.png differ diff --git a/docs/en/server/quickstart/quickstart/public_sys-resources/icon-note.gif b/docs/en/server/quickstart/quickstart/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/server/quickstart/quickstart/public_sys-resources/icon-note.gif differ diff --git a/docs/en/server/quickstart/quickstart/public_sys-resources/icon-notice.gif b/docs/en/server/quickstart/quickstart/public_sys-resources/icon-notice.gif new file mode 100644 index 0000000000000000000000000000000000000000..86024f61b691400bea99e5b1f506d9d9aef36e27 Binary files /dev/null and b/docs/en/server/quickstart/quickstart/public_sys-resources/icon-notice.gif differ diff --git a/docs/en/server/quickstart/quickstart/quick-start.md b/docs/en/server/quickstart/quickstart/quick-start.md new file mode 100644 index 0000000000000000000000000000000000000000..e0fbddd201ba637e171fe14c56a089821a7b0e6b --- /dev/null +++ b/docs/en/server/quickstart/quickstart/quick-start.md @@ -0,0 +1,347 @@ +# Quick Start + +This document uses openEuler 22.03 LTS SP2 installed on the TaiShan 200 server as an example to describe how to quickly install and use openEuler. For details about the installation requirements and methods, see the [Installation Guide](../../installation_upgrade/installation/installation.md). + +## Making Preparations + +- Hardware Compatibility + + [Table 1](#table14948632047) describes the types of supported servers. + + **Table 1** Supported servers + + + + + + + + + + + + + + + + + +

Server Type

+

Server Name

+

Server Model

+

Rack server

+

TaiShan 200

+

2280 balanced model

+

Rack server

+

FusionServer Pro

+

FusionServer Pro 2288H V5

+
+ + > [!NOTE]NOTE + > 2280 balanced model, FusionServer Pro 2288H V5: The server must be configured with the Avago SAS3508 RAID controller card and the LOM-X722 NIC. + +- Minimum Hardware Specifications + + [Table 2](#tff48b99c9bf24b84bb602c53229e2541) lists the minimum hardware specifications supported by openEuler. + + **Table 2** Minimum hardware specifications + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Component

Minimum Hardware Specifications

Description

Architecture

  • AArch64
  • x86_64
  • 64-bit Arm architecture
  • 64-bit Intel x86 architecture

CPU

  • Huawei Kunpeng 920 series
  • Intel ® Xeon® processor

-

Memory

≥ 4 GB (8 GB or higher recommended for better user experience)

-

Drive

≥ 120 GB (for better user experience)

  • IDE, SATA, and SAS drives are supported.
  • A driver software is required to use the NVMe drive with the DIF feature. Contact the drive manufacturer if the feature is not available.

+ +## Obtaining the Installation Source + +Perform the following operations to obtain the openEuler release package: + +1. Visit the [openEuler](https://www.openeuler.org/en/) website. +2. Click **Downloads**. +3. Click **Community Editions**. The version list is displayed. +4. Click **Download** on the right of **openEuler 23.09**. +5. Download the required openEuler release package and the corresponding verification file based on the architecture and scenario. + + - If the AArch64 architecture is used: + + 1. Click **AArch64**. + 2. For local installation, download the **Offline Standard ISO** or **Offline Everything ISO** release package **openEuler-22.03-LTS-SP2-aarch64-dvd.iso** to the local host. + 3. For network installation, download the **Network Install ISO** release package **openEuler-22.03-LTS-SP2-netinst-aarch64-dvd.iso** to the local host. + + - If the x86\_64 architecture is used: + + 1. Click **x86_64**. + 2. For local installation, download the **Offline Standard ISO** or **Offline Everything ISO** release package **openEuler-22.03-LTS-SP2-x86_64-dvd.iso** to the local host. + 3. For network installation, download the **Network Install ISO** release package **openEuler-22.03-LTS-SP2-netinst-x86_64-dvd.iso** to the local host. + +> [!NOTE]NOTE + +- When the network is available, install the environment on the network because the ISO release package is small. +- The release package of the AArch64 architecture supports the UEFI mode, while the release package of the x86\_64 architecture supports both the UEFI and Legacy modes. + +## Checking the Release Package Integrity + +> [!NOTE]NOTE +> This section describes how to verify the integrity of the release package in the AArch64 architecture. The procedure for verifying the integrity of the release package in the x86\_64 architecture is the same. + +### Overview + +To prevent incomplete download of the software package due to network or storage device problems during the transmission, verify the integrity of the software package after obtaining it. Only the software package that passes the verification can be deployed. + +Compare the verification value recorded in the verification file with the verification value of the ISO file calculated manually to determine whether the software package is complete. If the two values are the same, the ISO file is complete. Otherwise, the ISO file integrity is damaged. In this case, obtain the ISO release package again. + +### Prerequisites + +The following files need to be prepared: + +ISO file: openEuler-22.03-LTS-SP2-aarch64-dvd.iso + +Verification file: Copy and save the SHA256 value corresponding to the ISO file. + +1. Calculate the SHA256 verification value of the file. Run the following command: + + ```sh + # sha256sum openEuler-22.03-LTS-SP2-aarch64-dvd.iso + ``` + +2. Check whether the values calculated in step 1 and step 2 are the same. + + If the values are consistent, the ISO file is not damaged. Otherwise, the file is damaged and you need to obtain it again. + If the values are consistent, the ISO file is not damaged. Otherwise, the file is damaged and you need to obtain it again. + +## Starting Installation + +1. Log in to the iBMC WebUI. + + For details, see [TaiShan 200 Server User Guide (Model 2280)](https://support.huawei.com/enterprise/en/doc/EDOC1100093459). + +2. Choose **Configuration** from the main menu, and select **Boot Device** from the navigation tree. The **Boot Device** page is displayed. + + Set **Effective** and **Boot Medium** to **One-time** and **DVD-ROM**, respectively, and click **Save**, as shown in [Figure 1](#fig1011938131018). + + **Figure 1** Setting the boot device + ![fig](./figures/setting-the-boot-device.png) + +3. Choose **Remote Console** from the main menu. The **Remote Console** page is displayed. + + Select an integrated remote console as required to access the remote virtual console, for example, **Java Integrated Remote Console (Shared)**. + +4. On the toolbar, click the icon shown in the following figure. + + **Figure 2** Drive icon + ![fig](./figures/drive-icon.png) + + An image dialog box is displayed, as shown in the following figure. + + **Figure 3** Image dialog box + ![fig](./figures/image-dialog-box.png) + +5. Select **Image File** and then click **Browse**. The **Open** dialog box is displayed. + +6. Select the image file and click **Open**. In the image dialog box, click **Connect**. If **Connect** changes to **Disconnect**, the virtual CD/DVD-ROM drive is connected to the server. + +7. On the toolbar, click the restart icon shown in the following figure to restart the device. + + **Figure 4** Restart icon + ![fig](./figures/restart-icon.png) + +8. A boot menu is displayed after the system restarts, as shown in [Figure 5](#fig1648754873314). + + > [!NOTE]NOTE + + - If you do not perform any operations within 1 minute, the system automatically selects the default option **Test this media \& install openEuler 22.03-LTS-SP2** and enters the installation page. + - During physical machine installation, if you cannot use the arrow keys to select boot options and the system does not respond after you press **Enter**, click ![fig](./figures/en-us_image_0229420473.png) on the BMC page and configure **Key \& Mouse Reset**. + + **Figure 5** Installation wizard + ![fig](./figures/Installation_wizard.png) + +9. On the installation wizard page, press **Enter** to select the default option **Test this media \& install openEuler 22.03-LTS-SP2** to enter the GUI installation page. + +## Performing Installation + +After entering the GUI installation page, perform the following operations to install the system: + +1. Set an installation language. The default language is English. You can change the language based on the site requirements, as shown in [Figure 6](#fig874344811484). + + **Figure 6** Selecting a language + ![fig](./figures/selecting-a-language.png) + +2. On the **INSTALLATION SUMMARY** page, set configuration items based on the site requirements. + + - A configuration item with an alarm symbol must be configured. When the alarm symbol disappears, you can perform the next operation. + - A configuration item without an alarm symbol is configured by default. + - You can click **Begin Installation** to install the system only when all alarms are cleared. + + **Figure 7** Installation summary + ![fig](./figures/installation-summary.png) + + 1. Select **Software Selection** to set configuration items. + + Based on the site requirements, select **Minimal Install** on the left box and select an add-on in the **Add-Ons for Selected Environment** area on the right, as shown in [Figure 8](#fig1133717611109). + + **Figure 8** Selecting installation software + ![fig](./figures/selecting-installation-software.png) + + > [!NOTE]NOTE + + - In **Minimal Install** mode, not all packages in the installation source are installed. If a required package is not installed, you can mount the installation source to the local host as a repo source, and use DNF to install the package. + - If you select **Virtual Host**, the virtualization components QEMU, libvirt, and edk2 are installed by default. You can select whether to install the OVS component in the add-on area. + + After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. + + 2. Select **Installation Destination** to set configuration items. + + On the **INSTALLATION DESTINATION** page, select a local storage device. + + > [!TIP]NOTICE + > The NVMe data protection feature is not supported because the NVMe drivers built in the BIOSs of many servers are of earlier versions. (Data protection: Format disk sectors to 512+N or 4096+N bytes.) Therefore, when selecting a proper storage medium, do not select an NVMe SSD with data protection enabled as the system disk. Otherwise, the OS may fail to boot. + > Users can consult the server vendor about whether the BIOS supports NVMe disks with data protection enabled as system disks. If you cannot confirm whether the BIOS supports NVMe disks with data protection enabled as system disks, you are not advised to use an NVMe disk to install the OS, or you can disable the data protection function of an NVMe disk to install the OS. + + You also need to configure the storage to partition the system. You can either manually configure partitions or select **Automatic** for automatic partitioning. Select **Automatic** if the software is installed in a new storage device or the data in the storage device is not required, as shown in [Figure 9](#fig153381468101). + + **Figure 9** Setting the installation destination + ![fig](./figures/setting-the-installation-destination.png) + + > [!NOTE]NOTE + > + > - During partitioning, to ensure system security and performance, you are advised to divide the device into the following partitions: **/boot**, **/var**, **/var/log**, **/var/log/audit**, **/home**, and **/tmp**. + > - If the system is configured with the **swap** partition, the **swap** partition is used when the physical memory of the system is insufficient. Although the **swap** partition can be used to expand the physical memory, if the **swap** partition is used due to insufficient memory, the system response slows and the system performance deteriorates. Therefore, you are not advised to configure the **swap** partition in a system with sufficient physical memory or a performance-sensitive system. + > - If you need to split a logical volume group, select **Custom** to manually partition the logical volume group. On the **MANUAL PARTITIONING** page, click **Modify** in the **Volume Group** area to reconfigure the logical volume group. + + After the setting is complete, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. + + 3. Select **Root Password** and set the root password. + + On the **ROOT PASSWORD** page, enter a password that meets the [Password Complexity](#password-complexity) requirements and confirm the password, as shown in [Figure 10](#zh-cn_topic_0186390266_zh-cn_topic_0122145909_fig1323165793018). + + > [!NOTE]NOTE + + - The **root** account is used to perform key system management tasks. You are not advised to use the **root** account for daily work or system access. + - If you select **Lock root account** on the **Root Password** page, the **root** account will be disabled. + + **Password Complexity** + + The password of the **root** user or a new user must meet the password complexity requirements. Otherwise, the password setting or user creation will fail. The password must meet the following requirements: + + 1. Contains at least eight characters. + 2. Contains at least three of the following: uppercase letters, lowercase letters, digits, and special characters. + 3. Different from the user name. + 4. Not allowed to contain words in the dictionary. + + > [!NOTE]NOTE + > In the openEuler environment, you can run the `cracklib-unpacker /usr/share/cracklib/pw_dict > dictionary.txt` command to export the dictionary library file **dictionary.txt**. You can check whether the password is in this dictionary. + + **Figure 10** root password + ![fig](./figures/password-of-the-root-account.png) + + After the settings are completed, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. + + 4. Select **Create a User** and set the parameters. + + [Figure 11](#zh-cn_topic_0186390266_zh-cn_topic_0122145909_fig1237715313319) shows the page for creating a user. Enter the user name and set the password. The password complexity requirements are the same as those of the root password. In addition, you can set the home directory and user group by clicking **Advanced**, as shown in [Figure 12](#zh-cn_topic_0186390266_zh-cn_topic_0122145909_fig1237715313319). + + **Figure 11** Creating a user + ![fig](./figures/creating-a-user.png) + + **Figure 12** Advanced user configuration + ![fig](./figures/advanced-user-configuration.png) + + After the settings are completed, click **Done** in the upper left corner to go back to the **INSTALLATION SUMMARY** page. + + 5. Set other configuration items. You can use the default values for other configuration items. + +3. Click **Start the Installation** to install the system, as shown in [Figure 13](#zh-cn_topic_0186390266_zh-cn_topic_0122145909_fig1237715313319). + + **Figure 13** Starting the installation + + ![fig](./figures/installation-process.png) + +4. After the installation is completed, restart the system. + + openEuler has been installed. Click **Reboot** to restart the system. + +## Viewing System Information + +After the system is installed and restarted, the system CLI login page is displayed. Enter the username and password set during the installation to log in to openEuler and view the following system information. For details about system management and configuration, see the [openEuler 22.03-LTS-SP2 Administrator Guide](../../administration/administrator/administration.md). + +- Run the following command to view the system information: + + ```sh + cat /etc/os-release + ``` + + ```sh + cat /etc/os-release + ``` + +- View system resource information. + + Run the following command to view the CPU information: + + ```sh + # lscpu + ``` + + Run the following command to view the memory information: + + ```sh + # free + ```` + + Run the following command to view the disk partition information: + + ```sh + # fdisk -l + ``` + + Run the following command to view the CPU information: + + ```sh + # lscpu + ``` + + Run the following command to view the memory information: + + ```sh + # free + ```` + + Run the following command to view the disk partition information: + + ```sh + # fdisk -l + ``` + +- Run the following command to view the IP address: + + ```sh + # ip addr + ``` diff --git a/docs/en/server/releasenotes/releasenotes/_toc.yaml b/docs/en/server/releasenotes/releasenotes/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..bcf4529294d76dcfcc455ad869d6c18fe83a688d --- /dev/null +++ b/docs/en/server/releasenotes/releasenotes/_toc.yaml @@ -0,0 +1,29 @@ +label: Release Notes +isManual: true +href: ./release-notes.md +description: Release notes for openEuler 25.03 +sections: + - label: Introduction + href: ./introduction.md + - label: Terms of Use + href: ./terms-of-use.md + - label: User Notice + href: ./user-notice.md + - label: Account List + href: ./account-list.md + - label: OS Installation + href: ./os-installation.md + - label: Key Features + href: ./key-features.md + - label: Known Issues + href: ./known-issues.md + - label: Resolved Issues + href: ./resolved-issues.md + - label: Common Vulnerabilities and Exposures (CVEs) + href: ./cve.md + - label: Source Code + href: ./source-code.md + - label: Contribution + href: ./contribution.md + - label: Acknowledgment + href: ./acknowledgment.md diff --git a/docs/en/server/releasenotes/releasenotes/account-list.md b/docs/en/server/releasenotes/releasenotes/account-list.md new file mode 100644 index 0000000000000000000000000000000000000000..f31dceb5e3927466715cc79a2f425125d6858e0e --- /dev/null +++ b/docs/en/server/releasenotes/releasenotes/account-list.md @@ -0,0 +1,6 @@ +# Account List + +| User Name| Default Password | Function | User Status| Login Mode | Remarks | +| ------ | ------------- | ------------------ | -------- | ------------------ | ------------------------------------------------------------ | +| root | openEuler12#$ | Default user of the VM image| Enabled | Remote login | This account is used to log in to the VM installed using the openEuler VM image. | +| root | openEuler#12 | GRUB2 login | Enabled | Local login and remote login| GRand UnifiedBootloader (GRUB) is used to boot different systems, such as Windows and Linux.
GRUB2 is an upgraded version of GRUB. When the system is started, you can modify startup parameters on the GRUB2 GUI. To ensure that the system startup parameters are modified with authorization, you need to encrypt the GRUB2 GUI. The GRUB2 GUI can be modified only when you enter the correct GRUB2 password.| diff --git a/docs/en/server/releasenotes/releasenotes/acknowledgment.md b/docs/en/server/releasenotes/releasenotes/acknowledgment.md new file mode 100644 index 0000000000000000000000000000000000000000..b469785cbeda8f4b08b6f8c55988288913d45e56 --- /dev/null +++ b/docs/en/server/releasenotes/releasenotes/acknowledgment.md @@ -0,0 +1,3 @@ +# Acknowledgment + +We sincerely thank all the members who participated in and assisted in the openEuler project. It is your hard work to make the version released successfully and provide the possibility for the better development of openEuler. diff --git a/docs/en/server/releasenotes/releasenotes/contribution.md b/docs/en/server/releasenotes/releasenotes/contribution.md new file mode 100644 index 0000000000000000000000000000000000000000..6dc7fa17ba5428383aaee02d51862a4aa094cd72 --- /dev/null +++ b/docs/en/server/releasenotes/releasenotes/contribution.md @@ -0,0 +1,21 @@ +# Contribution + +As an openEuler user, you can contribute to the openEuler community in multiple ways. For details about how to contribute to the community, see [How to Contribute](https://openeuler.org/en/community/contribution/). Here, some methods are listed for reference. + +## Special Interest Groups \(SIGs\) + +openEuler brings together people of common interest to form different special interest groups \(SIGs\). For details about existing SIGs, see the [SIG list](https://openeuler.org/en/sig/sig-list/). + +You are welcome to join an existing SIG or create a SIG. For details about how to create a SIG, see the [SIG Management Procedure](https://gitee.com/openeuler/community/blob/master/en/technical-committee/governance/README.md). + +## Mail List and Tasks + +You are welcome to actively help users solve problems raised in the [mail list](https://openeuler.org/en/community/mailing-list/) and issues \(including [code repository issues](https://gitee.com/organizations/openeuler/issues) and [software package repository issues](https://gitee.com/organizations/src-openeuler/issues)\). In addition, you can submit an issue. All these will help the openEuler community to develop better. + +## Documents + +We invite you to contribute to the community by submitting code and feedback on problems and difficulties, or suggestions on improving the usability and integrity of documents. For example, problems in obtaining software or documents and difficulties in using the system. Welcome to pay attention to and improve the documentation module of the [openEuler community](https://openeuler.org/en/). + +## IRC + +openEuler has also opened a channel in IRC as an additional channel to provide community support and interaction. For details, see [openEuler IRC](https://gitee.com/openeuler/community/blob/master/en/communication/IRCs.md). diff --git a/docs/en/server/releasenotes/releasenotes/cve.md b/docs/en/server/releasenotes/releasenotes/cve.md new file mode 100644 index 0000000000000000000000000000000000000000..482c72e54fb3d21b29bd010625c531ccc72efc4e --- /dev/null +++ b/docs/en/server/releasenotes/releasenotes/cve.md @@ -0,0 +1,3 @@ +# Common Vulnerabilities and Exposures (CVEs) + +For details about the CVEs involved in the version, see the [CVE list](https://www.openeuler.org/en/security/cve/). diff --git a/docs/en/server/releasenotes/releasenotes/introduction.md b/docs/en/server/releasenotes/releasenotes/introduction.md new file mode 100644 index 0000000000000000000000000000000000000000..565577db870e6aaa1fe7df096667f8714c5cd4dd --- /dev/null +++ b/docs/en/server/releasenotes/releasenotes/introduction.md @@ -0,0 +1,3 @@ +# Introduction + +openEuler is an open source operating system. The current openEuler kernel is based on Linux and supports Kunpeng and other processors. It fully unleashes the potential of computing chips. As an efficient, stable, and secure open source OS built by global open source contributors, openEuler applies to database, big data, cloud computing, and artificial intelligence \(AI\) scenarios. In addition, openEuler community is an open source community for global OSs. Through community cooperation, openEuler builds an innovative platform, builds a unified and open OS that supports multiple processor architectures, and promotes the prosperity of the software and hardware application ecosystem. diff --git a/docs/en/server/releasenotes/releasenotes/key-features.md b/docs/en/server/releasenotes/releasenotes/key-features.md new file mode 100644 index 0000000000000000000000000000000000000000..98ebb361d59550394b171bc467b66ef0528e79a8 --- /dev/null +++ b/docs/en/server/releasenotes/releasenotes/key-features.md @@ -0,0 +1,289 @@ +# Key Features + +## AI + +AI is redefining OSs by powering intelligent development, deployment, and O&M. openEuler supports general-purpose architectures like Arm, x86, and RISC-V, and next-gen AI processors like NVIDIA and Ascend. Further, openEuler is equipped with extensive AI capabilities that have made it a preferred choice for diversified computing power. + +- **OS for AI**: openEuler offers an efficient development and runtime environment that containerizes software stacks of AI platforms with out-of-the-box availability. It also provides various AI frameworks to facilitate AI development. + - openEuler supports TensorFlow, PyTorch, and MindSpore frameworks and software development kits (SDKs) of major computing architectures, such as Compute Architecture for Neural Networks (CANN) and Compute Unified Architecture (CUDA), to make it easy to develop and run AI applications. + - Environment setup is further simplified by containerizing software stacks. openEuler provides three types of container images: + 1. SDK images: Use openEuler as the base image and install the SDK of a computing architecture, for example, Ascend CANN and NVIDIA CUDA. + 2. AI framework images: Use an SDK image as the base and install AI framework software, such as PyTorch and TensorFlow. You can use an AI framework image to quickly build a distributed AI framework, such as Ray. + 3. Model application images: Provide a complete set of toolchains and model applications. + For details, see the [openEuler AI Container Image User Guide](https://gitee.com/openeuler/docs/blob/stable2-22.03_LTS_SP3/docs/en/docs/AI/ai_container_image_user_guide.md). + - The sysHax LLM heterogeneous acceleration runtime enhances model inference performance in single-server, multi-device setups by optimizing Kunpeng + *x*PU (GPU/NPU) resource synergy. + 1. Dynamic resource allocation: Intelligently balances Kunpeng CPU and xPU workloads to maximize compute efficiency. + 2. CPU inference acceleration: Improves throughput via NUMA-aware scheduling, parallelized matrix operations, and SVE-optimized inference operators. +- **AI for OS**: AI makes openEuler more intelligent. The openEuler Copilot System, an intelligent Q&A platform, is developed using foundation models and openEuler data. It assists in code generation, problem analysis, and system O&M. + - AI application development framework: Foundation model applications are key to enterprise AI adoption. Combining retrieval-augmented generation (RAG) with foundation models effectively addresses gaps in domain-specific data. To support this, the openEuler community has developed an AI application development framework that provides intelligent tools for individual and enterprise developers, enabling rapid AI application development through streamlined workflows. It lowers technical barriers, improves efficiency and data quality, and meets diverse needs for data processing, model training, and content management. Its core features include: + 1. Multi-path enhanced RAG for improved Q&A accuracy: Overcomes limitations of traditional native RAG (low accuracy, weak guidance) using techniques like corpus governance, prompt rewriting, and multi-path retrieval comparison. + 2. Document processing and optimization: Supports incremental corpus updates, deduplication, sensitive data masking, and standardization (such as summarization, code annotation, and case organization). + 3. Embedding model fine-tuning: Enables rapid tuning and evaluation of embedding models (such as BGE models) for domain-specific performance gains. + - Intelligent Q&A: The openEuler Copilot System is accessible via web or shell. + 1. Workflow scheduling: + - Atomic agent operations: Multiple agent operations can be combined into a multi-step workflow that is internally ordered and associated, and is executed as an inseparable atomic operation. + - Real-time data processing: Data generated in each step of the workflow can be processed immediately and then transferred to the next step. + - Intelligent interaction: When the openEuler Copilot System receives a vague or complex user instruction, it proactively asks the user to clarify and provide more details. + 2. Task recommendation: + - Intelligent response: The openEuler Copilot System can analyze the semantic information entered. + - Intelligent guidance: The openEuler Copilot System comprehensively analyzes the execution status, function requirements, and associated tasks of the current workflow to provide next-step operation suggestions. + 3. RAG: The RAG technology used by openEuler Copilot System supports more document formats and content scenarios, and enhances Q&A quality while adding little system load. + 4. Corpus governance: Corpus governance is a core RAG capability. It imports corpuses into the knowledge base in a supported format using fragment relationship extraction, fragment derivative construction, and optical character recognition (OCR). This increases the retrieval hit rate. + For details, see the [openEuler Copilot System Intelligent Q&A Service User Guide](https://gitee.com/openeuler/docs/blob/stable2-22.03_LTS_SP4/docs/en/docs/AI/EulerCopilot_user_guide.md). + - Intelligent tuning: The openEuler Copilot System supports the intelligent shell entry. + Through this entry, you can interact with the openEuler Copilot System using a natural language and perform heuristic tuning operations such as performance data collection, system performance analysis, and system performance tuning. + - Intelligent diagnosis: + 1. Inspection: The Inspection Agent checks for abnormalities of designated IP addresses and provides an abnormality list that contains associated container IDs and abnormal metrics (such as CPU and memory). + 2. Demarcation: The Demarcation Agent analyzes and demarcates a specified abnormality contained in the inspection result and outputs the top 3 metrics of the root cause. + 3. Location: The Detection Agent performs profiling location analysis on the root cause, and provides useful hotspot information such as the stack, system time, and performance metrics related to the root cause. + - Intelligent vulnerability patching: The openEuler intelligent vulnerability patching tool provides automated vulnerability management and repair capabilities for the openEuler kernel repository. This feature analyzes the impact of vulnerabilities on openEuler versions using the `/analyze` command and enables minute-level automated patch pull request (PR) creation via the `/create_pr` command. + - Intelligent container images: The openEuler Copilot System can invoke environment resources through a natural language, assist in pulling container images for local physical resources, and establish a development environment suitable for debugging on existing compute devices. This system supports three types of containers, and container images have been released on Docker Hub. You can manually pull and run these container images. + 1. SDK layer: encapsulates only the component libraries that enable AI hardware resources, such as CUDA and CANN. + 2. SDKs + training/inference frameworks: accommodates TensorFlow, PyTorch, and other frameworks (for example, tensorflow2.15.0-cuda12.2.0 and pytorch2.1.0.a1-cann7.0.RC1) in addition to the SDK layer. + 3. SDKs + training/inference frameworks + LLMs: encapsulates several models (for example, llama2-7b and chatglm2-13b) based on the second type of containers. + +## openEuler Embedded + +openEuler 25.03 Embedded is designed for embedded applications, offering significant progress in southbound and northbound ecosystems and infrastructure. openEuler Embedded provides a closed loop framework often found in operational technology (OT) applications such as manufacturing and robotics, whereby innovations help optimize its embedded system software stack and ecosystem. For southbound compatibility, the "openEuler Embedded Ecosystem Expansion Initiative" has strengthened hardware support including Kunpeng 920 and TaishanPi, while achieving successful adaptation of STMicroelectronics' STM32MP257 high-performance microprocessor for industry applications through collaboration with MYIR. On the northbound front, capabilities have been enriched with industrial middleware and graphical middleware solutions, enabling practical implementations in manufacturing automation and robotics. Looking ahead, openEuler Embedded will collaborate with community partners, users, and developers to expand support for new processor architectures like LoongArch, enhance southbound hardware compatibility, and advance key capabilities including industrial middleware, embedded AI, edge computing, and simulation systems to establish a comprehensive embedded software platform solution. + +- **Southbound ecosystem**: openEuler Embedded Linux supports mainstream processor architectures like AArch64, x86_64, AArch32, and RISC-V, and will extend support to LoongArch in the future. openEuler 24.03 and later versions have a rich southbound ecosystem and support chips from Raspberry Pi, HiSilicon, Rockchip, Renesas, TI, Phytium, StarFive, and Allwinner. +- **Embedded virtualization base**: openEuler Embedded uses an elastic virtualization base that enables multiple OSs to run on a system-on-a-chip (SoC). The base incorporates a series of technologies including bare metal, embedded virtualization, lightweight containers, LibOS, trusted execution environment (TEE), and heterogeneous deployment. + 1. The bare metal hybrid deployment solution runs on OpenAMP to manage peripherals by partition at a high performance level; however, it delivers poor isolation and flexibility. This solution supports the hybrid deployment of UniProton/Zephyr/RT-Thread and openEuler Embedded Linux. + 2. Partitioning-based virtualization is an industrial-grade hardware partition virtualization solution that runs on Jailhouse. It offers superior performance and isolation but inferior flexibility. This solution supports the hybrid deployment of UniProton/Zephyr/FreeRTOS and openEuler Embedded Linux or of OpenHarmony and openEuler Embedded Linux. + 3. Real-time virtualization is available as two community hypervisors, ZVM (for real-time VM monitoring) and Rust-Shyper (for Type-I embedded VM monitoring). +- **MICA deployment framework**: The MICA deployment framework is a unified environment that masks the differences between technologies that comprise the embedded elastic virtualization base. The multi-core capability of hardware combines the universal Linux OS and a dedicated real-time operating system (RTOS) to make full use of all OSs. The MICA deployment framework covers lifecycle management, cross-OS communication, service-oriented framework, and multi-OS infrastructure. + - Lifecycle management provides operations to load, start, suspend, and stop the client OS. + - Cross-OS communication uses a set of communication mechanisms between different OSs based on shared memory. + - Service-oriented framework enables different OSs to provide their own services. For example, Linux provides common file system and network services, and the RTOS provides real-time control and computing. + - Multi-OS infrastructure integrates OSs through a series of mechanisms, covering resource expression and allocation and unified build. + The MICA deployment framework provides the following functions: + - Lifecycle management and cross-OS communication for openEuler Embedded Linux and the RTOS (Zephyr or UniProton) in bare metal mode + - Lifecycle management and cross-OS communication for openEuler Embedded Linux and the RTOS (FreeRTOS or Zephyr) in partitioning-based virtualization mode +- **Northbound ecosystem**: Over 600 common embedded software packages can be built using openEuler.Soft real-time kernel helps respond to soft real-time interrupts within microseconds. The distributed soft bus system (DSoftBus) of openEuler Embedded integrates the DSoftBus and point-to-point authentication module of OpenHarmony, implementing interconnection between openEuler-based embedded devices and OpenHarmony-based devices as well as between openEuler-based embedded devices. With iSula containers, openEuler and other OS containers can be deployed on embedded devices to simplify application porting and deployment. Embedded container images can be compressed to 5 MB, and can be easily deployed into the OS on another container. +- **UniProton**: UniProton is an RTOS that features ultra-low latency and flexible MICA deployments. It is suited for industrial control because it supports both microcontroller units and multi-core CPUs. UniProton provides the following capabilities: + - Compatible with processor architectures like Cortex-M, AArch64, x86_64, and riscv64, and supports M4, RK3568, RK3588, x86_64, Hi3093, Raspberry Pi 4B, Kunpeng 920, Ascend 310, and Allwinner D1s. + - Connects with openEuler Embedded Linux on Raspberry Pi 4B, Hi3093, RK3588, and x86_64 devices in bare metal mode. + - Can be debugged using GDB on openEuler Embedded Linux. + +## epkg + +epkg is a new software package manager that supports the installation and use of non-service software packages. It solves version compatibility issues so that users can install and run software of different versions on the same OS by using simple commands to create, enable, and switch between environments. + +- **Multi-version compatibility**: Enables installation of multiple software versions, resolving version conflicts. +- **Flexible installation modes**: Supports both privileged (system-wide) and unprivileged (user-specific) installations, enabling minimal-footprint deployments and self-contained installations. +- **Environment management**: Facilitates environment lifecycle operations (create, delete, activate, register, and view), supporting multiple environments with distinct software repositories. Enables multi-environment version control, with runtime registration for multiple environments and exclusive environment activation for development debugging. +- **Environment rollback**: Maintains operational history tracking and provides state restoration capabilities, allowing recovery from misoperations or faulty package installations. +- **Package management**: Implements core package operations (install, remove, and query) with RPM/DNF-level functionality parity, meeting daily usage requirements for typical users and scenarios. + +## GCC Compilation and Linking Acceleration + +To improve the compilation efficiency of openEuler software packages and enhance CI pipeline and developer productivity, optimization techniques for C/C++ components are implemented through compiler and linker enhancements. The combination of GCC 12.3 with profile-guided optimization (PGO) and link time optimization (LTO), alongside the modern mold linker, reduced the total compilation time for the top 90+ software packages by approximately 9.5%. The following key capabilities are supported: + +1. GCC 12.3 is configured to generate binaries with PGO and LTO, accelerating the compilation process. +2. Applications specified in the [allowlist](https://gitee.com/src-openeuler/openEuler-rpm-config/blob/openEuler-25.03/0002-Enable-mold-links-through-whitelist.patch#L49) automatically switch to the mold linker to optimize linking efficiency. + +## Kernel Innovations + +openEuler 25.03 runs on Linux kernel 6.6 and inherits the competitive advantages of community versions and innovative features released in the openEuler community. + +- **Kernel replication**: This feature optimizes Linux kernel performance bottlenecks in non-uniform memory access (NUMA) architectures. Research shows critical data center applications like Apache, MySQL, and Redis experience significant performance impacts from kernel operations: kernel execution accounts for 61% of application CPU cycles, 57% of total instructions executed, 61% of I-cache misses, and 46% of I-TLB misses. Traditional Linux kernels restrict code segments, read-only data segments, and kernel page tables (**swapper_pg_dir**) to primary NUMA nodes without migration capability. This forces frequent cross-NUMA operations during system calls when multi-threaded applications are deployed across multiple NUMA nodes, increasing memory access latency and degrading system performance. The kernel replication feature extends the **pgd** global page directory table in **mm_struct** by automatically creating NUMA-local replicas of kernel code, data segments, and page tables during kernel initialization. This mechanism maps identical kernel virtual addresses to physical addresses within their respective NUMA nodes, enhancing memory locality and reducing cross-NUMA overhead. The implementation supports vmalloc, dynamic module loading, dynamic instruction injection mechanisms (Kprobe, KGDB, and BPF), security features (KPTI, KASLR, and KASAN), and 64 KB huge pages. A new boot-time cmdline configuration option (disabled by default) enables dynamic control for compatibility management. This feature benefits high-concurrency, multi-threaded server workloads. +- **HAOC 3.0 security feature**: Hardware-assisted OS compartmentalization (HAOC) leverages x86 and Arm processor capabilities to implement a dual-architecture kernel design. It creates isolated execution environments (IEE) within the kernel to prevent attackers from performing lateral movement and privilege escalation. The current version establishes IEE as a protected domain where sensitive resources can be incrementally isolated. These resources become accessible exclusively through controlled IEE interfaces, preventing unauthorized access by standard kernel code. + +## NestOS + +NestOS is a community cloud OS that uses nestos-assembler for quick integration and build. It runs rpm-ostree and Ignition tools over a dual rootfs and atomic update design, and enables easy cluster setup in large-scale containerized environments. Compatible with Kubernetes and OpenStack, NestOS also reduces container overheads. + +- **Out-of-the-box availability**: integrates popular container engines such as iSulad, Docker, and Podman to provide lightweight and tailored OSs for the cloud. +- **Easy configuration**: uses the Ignition utility to install and configure a large number of cluster nodes with a single configuration. +- **Secure management**: runs rpm-ostree to manage software packages and works with the openEuler software package source to ensure secure and stable atomic updates. +- **Hitless node updating**: uses Zincati to provide automatic node updates and reboot without interrupting services. +- **Dual rootfs**: executes dual rootfs for active/standby switchovers, to ensure integrity and security during system running. + +## oeAware Enhancements + +oeAware is a framework that provides low-load collection, sensing, and tuning upon detecting defined system behaviors on openEuler. The framework divides the tuning process into three layers: collection, sensing, and tuning. Each layer is associated through subscription and developed as plugins, overcoming the limitations of traditional tuning techniques that run independently and are statically enabled or disabled. +Every oeAware plugin is a dynamic library that utilizes oeAware interfaces. The plugins comprise multiple instances that each contains several topics and deliver collection or sensing results to other plugins or external applications for tuning and analysis purposes. openEuler 25.03 introduces the transparent_hugepage_tune and preload_tune plugins. + +- The SDK enables subscription to plugin topics, with a callback function handling data from oeAware. This allows external applications to create tailored functionalities, such as cross-cluster information collection or local node analysis. +- The Performance monitoring unit (PMU) information collection plugin gathers performance records from the system PMU. +- The Docker information collection plugin retrieves specific parameter details about the Docker environment. +- The system information collection plugin captures kernel parameters, thread details, and resource information (CPU, memory, I/O, network) from the current environment. +- The thread sensing plugin monitors key information about threads. +- The evaluation plugin examines system NUMA and network information during service operations, suggesting optimal tuning methods. +- The system tuning plugins comprise stealtask for enhanced CPU tuning, smc_tune which leverages shared memory communication in the kernel space to boost network throughput and reduce latency, xcall_tune which bypasses non-essential code paths to minimize system call processing overhead, transparent_hugepage_tune which enables transparent huge pages to boost the TLB hit ratio, and preload_tune which seamlessly loads dynamic libraries. +- The Docker tuning plugin addresses CPU performance issues during sudden load spikes by utilizing the CPU burst feature. +- smc_tune: SMC acceleration must be enabled before the server-client connection is established. This feature is most effective in scenarios with numerous persistent connections. +- Docker tuning is not compatible with Kubernetes containers. +- xcall_tune: The **FAST_SYSCALL** kernel configuration option must be activated. + +## A-Ops with CVE Fixes and Configuration Source Tracing + +A-Ops empowers intelligent O&M through interactive dialogs and wizard-based operations. The intelligent interactive dialogs, featuring CVE prompts and fixes, configuration source tracing, configuration exception tracing, and configuration baseline synchronization, enable the O&M assistant to streamline routine O&M operations. +A-Ops integrates the intelligent O&M assistant based on the openEuler Intelligent Interaction Platform for intelligent CVE fixing and configuration source tracing. + +- CVE fixing: A-Ops displays cluster CVE status, prompts high-score and high-severity CVEs, and offers corresponding fixes. You can apply these fixes and check results using the assistant or WebUI. +- Configuration source tracing: You can use the assistant to find the machines with abnormal baseline configurations. The assistant shows these machines and incorrect configuration items. It then intelligently gives you summaries and suggests fixes. You can correct the configurations using the assistant or WebUI. + +## k8s-install + +k8s-install is an online utility designed to provision cloud-native infrastructure on a wide range of Linux distributions and architectures. It also serves as a tool for creating offline installation packages. It supports installation, deployment, and secure updates of cloud-native infrastructure suites across multiple dimensions with just a few clicks, greatly reducing deployment and adaptation time while ensuring a standardized and traceable workflow. Currently, the following issues are present: + +- openEuler suffers from outdated cloud-native toolchain versions and lacks maintenance for multiple version baselines (such as Kubernetes 1.20, 1.25, and 1.29) within the same release. Consequently, released branches cannot be updated to major versions, requiring users to independently adapt and maintain later versions to meet business requirements. +- Service parties commonly use tools like Ansible to deploy cloud infrastructure, often relying on non-standard packages, static binaries, and tarballs instead of distribution-managed packages. This practice inherently lacks support for secure CVE fixes, thereby posing security risks. +- Version synchronization between offline and online installations is challenging. Furthermore, upgrading or modifying offline packages is difficult. +- The lack of standardized installation and deployment processes results in inconsistent component versions, leading to incompatibilities and configuration differences that make issue resolution time-consuming and root cause analysis difficult. +- The installer detects, installs, and updates the runC, containerd, Docker, and Kubernetes components and their dependent system libraries. +- The configuration library stores configuration file templates for Docker and Kubernetes software. +- The package library stores RPM packages for various versions and architectures of runC, containerd, Docker, Kubernetes, and their dependent system libraries. +- The image library stores images required for Kubernetes software startup, such as various versions of kube-apiserver, kube-scheduler, etcd, and coredns. It also includes images for basic network plugins like Flannel. +- The publisher encapsulates the latest code scripts, RPM packages, images, and configurations to create online and offline installation packages. Written in Bash, the main k8s-install program does not need to be compiled or linked. Its online installation package is encapsulated into an RPM package, built using spec files. + +## k8s-install Installers + +k8s-install is a tool used to install and securely update cloud-native infrastructure. +Version adaptation: openEuler suffers from outdated cloud-native toolchain versions from the upstream and released branches cannot be updated to major versions, requiring users to independently adapt and maintain later versions to meet business requirements. k8s-install supports multiple baseline versions to meet service requirements, preventing deployment failures or function exceptions caused by version incompatibilities. +Improved deployment efficiency and standardization: The lack of standardized installation and deployment processes across departments or projects leads to inconsistent component versions, resulting in frequent adaptation issues and time-consuming resolutions. k8s-install enables standard deployment, ensuring component version compatibility, reducing fault locating time, and improving overall deployment efficiency. +Enhanced security and maintainability: Service parties often deploy static binaries and tarballs, which lack support for secure CVE fixes. k8s-install can fix CVEs a timely manner, ensuring system security and stability. In addition, the code for all components has been committed to both the company's internal repository and the openEuler repository, which facilitates version tracing and fault locating and enhances system maintainability. +Promoting open source and collaboration: By establishing and actively maintaining a repository within the openEuler community, k8s-install promotes technology sharing, fosters the growth of the community ecosystem, attracts more developers, enhances project influence, and promotes the continuous progress of cloud-native technologies. +The installers provides the following core functions: + +- Multi-version support: It supports multiple baseline Kubernetes versions, including 1.20, 1.25, and 1.29, to meet the version requirements of various business scenarios and enable on-demand deployment. +- Multi-architecture support: With compatibility for various architectures including x86_64, AArch64, and LoongArch64, it is suitable for diverse hardware environments, thereby expanding its application scope. +- Multi-component management: It integrates installation and configuration of Go, runC, containerd, Docker, Kubernetes, and related components, streamlining the deployment of complex components and improving efficiency. +- Online and offline deployment: An online installer k8s-install and an offline installer k8s-install-offline are available. Combined with the **publish.sh** publisher, these installers ensure flexible and stable deployment across various network conditions. + +## k8s-install Publisher + +**publish.sh** is the publisher in the k8s-install tool chain. It has the following advantages: + +- Ensuring offline deployment: In network-restricted or offline environments, such as certain data centers or specialized production setups, direct access to online repositories is not possible. **publish.sh** can generate complete offline installation packages, ensuring successful cloud-native infrastructure deployment in these scenarios and broadening the application scope of the tools. +- Efficient version iteration and release management: With the continuous updates of the k8s-install tool and its components, **publish.sh** enables automated build, test, and release processes. This enhances the efficiency of version iteration, ensures timely and accurate delivery of new versions to users, and facilitates the ongoing evolution of the system. +- Improving the stability and reliability of resource acquisition: Online repositories can face issues with package or image availability due to network fluctuations or delayed updates. **publish.sh** fetches resources from official or trusted online repositories and ensures their stability and reliability through integration and testing, preventing deployment failures caused by resource issues. +- Facilitating multi-team collaboration and resource synchronization: In large projects, different teams may manage various components or modules. **publish.sh** can integrate and publish the updates from each team, ensuring resource consistency across teams. It facilitates collaboration and improves overall project progress and quality. + +**Functions** + +- Offline package generation and release: It pulls the latest software packages and images from online Yum and image repositories, combines them with the latest configuration files and installer, and packages them into an offline **.tgz** installation package to meet the deployment needs of offline environments. +- Online code update and release: It uploads the updated code to the Git repository, selects the configuration library and installer for source code packaging, uploads it to the OBS server for official compilation after local build testing, and publishes it to the Yum repository to achieve online resource update and synchronization. + +## Trace IO + +Trace IO (TrIO) is designed to optimize the on-demand loading of container images using EROFS over fscache. It achieves this by accurately tracking I/Os during container startup and efficiently orchestrating I/Os into container images to improve the cold startup process of containers. Compared with existing container image loading solutions, TrIO can significantly reduce the cold startup latency of container jobs and improve bandwidth utilization. TrIO comprises both kernel-space and user-space modules. The kernel-space module includes adaptations within the EROFS file system. The user-space module provides tools for capturing I/O traces during container runtime and offers an adaptation modification guidance based on Nydus snapshotter. This allows container users to leverage TrIO without modifying containerd and runC, ensuring compatibility with existing container management tools. +The core advantage of TrIO lies in its ability to aggregate I/O operations during on-demand container loading. By orchestrating the runtime I/O traces of container jobs, TrIO accurately fetches the necessary I/O data during container execution. This greatly improves the efficiency of pulling image data during container startup, thereby achieving low latency. +TrIO's functionality comprises two main aspects: capturing container runtime I/Os and utilizing the runtime I/Os during container startup. Container runtime I/Os are captured by using eBPF to trace I/O operations in the file system. This allows for obtaining the I/O read requests during container job startup, and orchestrating the corresponding data to build a minimal runtime image. During container startup, a custom snapshotter plugin module pulls the minimal runtime image using large I/O operations and imports it into the kernel. Subsequently, all I/O operations during container job execution will preferentially be read from this minimal runtime image. +Compared with the existing on-demand container loading solutions, TrIO has the following advantages: + +- No I/O amplification: TrIO accurately captures runtime I/Os and use them for job startup. It ensures that I/Os are not amplified during container job startup. +- I/O aggregation: During container job startup, TrIO uses large I/O operations to pull all the necessary data for the startup process to the container node at once. This improves the efficiency of loading image data while reducing startup latency. + +## Kuasar Integration with virtCCA + +The Kuasar confidential container leverages the virtCCA capability of Kunpeng 920 processors. It connects northbound to the iSulad container engine and southbound to Kunpeng virtCCA hardware, enabling seamless integration of Kunpeng confidential computing with the cloud-native technology stack. +Kuasar fully utilizes the advantages of the Sandboxer architecture to deliver a high-performance, low-overhead confidential container runtime. Kuasar-sandboxer integrates the virtCCA capability of openEuler QEMU to manage the lifecycle of confidential sandboxes, allowing users to create confidential sandboxes on confidential hardware and ensuring containers run within a trusted execution environment (TEE). +Kuasar-task offers a Task API for iSulad to manage lifecycles of containers within secure sandboxes. Container images are securely pulled into encrypted sandbox memory through Kuasar-task's image retrieval capability. + +**Technical Constraints** + +1. Remote attestation support of Kuasar is planned for integration via secGear in the SP versions of openEuler 24.03 LTS. +2. Image encryption/decryption capabilities will be added after secGear integration. + +**Feature Description** + +Kuasar has expanded its capabilities to include confidential container support while maintaining existing secure container functionality. You can enable this feature through iSulad runtime configuration. + +- Native integration with the iSulad container engine preserves Kubernetes ecosystem compatibility. +- Hardware-level protection via Kunpeng virtCCA technology ensures confidential workloads are deployed in trusted environments. + +## vKernel for Advanced Container Isolation + +The virtual kernel (vKernel) architecture represents a breakthrough in container isolation, addressing the inherent limitations of shared-kernel architectures while preserving container performance efficiency. +vKernel creates independent system call tables and file permission tables to enhance foundational security. It implements isolated kernel parameters, enabling containers to customize both macro-level resource policies and micro-level resource configurations. By partitioning kernel data ownership, leveraging hardware features to protect kernel privilege data, and building isolated kernel page tables for user data protection, vKernel further reinforces security. Future iterations will explore kernel data related to performance interference to strengthen container performance isolation capabilities. + +## secGear with Secure Key Hosting for Confidential Container Images + +The remote attestation service of secGear provides secure key hosting capabilities for confidential container images, establishing a management system that encompasses secure key storage, dynamic fine-grained authorization, and cross-environment collaborative distribution. By integrating zero-trust policies and automated auditing capabilities, secGear ensures data confidentiality and operational traceability while optimizing the balance between key governance and operational costs. This delivers a unified "encrypt by default, decrypt on demand" security framework for cloud-native environments. +secGear combines remote attestation technologies to build a layered key hosting architecture. + +**Attestation service** + +A centralized key hosting server leverages the remote attestation mechanism of TEEs to securely store and manage image encryption keys throughout their lifecycle. It offers authorized users granular policy configuration interfaces for tailored access control. + +**Attestation agent** + +Lightweight attestation agent components deployed within confidential compute nodes expose local RESTful APIs. The confidential container runtime invokes these APIs to validate the integrity of the confidential execution environment and establish secure dynamic sessions with the server, enabling encrypted key transmission. + +## RA-TLS + +RA-TLS integrates remote attestation of confidential computing into TLS negotiation procedures, ensuring secure transmission of sensitive data into TEEs while simplifying secure channel establishment for confidential computing workloads, thereby lowering adoption barriers. + +**One-way authentication** + +In deployments where TLS servers operate within confidential environments and clients reside in regular environments, RA-TLS validates the legitimacy of the server confidential environment and applications through remote attestation before TLS key negotiation. + +**Two-way authentication** + +For scenarios where both TLS servers and clients operate within confidential environments, RA-TLS enforces mutual verification of peer environments and applications via remote attestation before TLS key negotiation. + +**Technical Constraints** + +Confidential computing environments must maintain network accessibility (such as virtCCA-enabled configurations). + +## openAMDC for High-Performance In-Memory Data Caching and KV Storage + +openAMDC stands for open advanced in-memory data cache. It stores and caches data in memory to accelerate access, enhance application concurrency, minimize latency, and can serve as both a message broker and in-memory database. + +**Feature Description** + +- Core capabilities: openAMDC, compatible with the Redis Serialization Protocol (RESP), delivers comprehensive caching for strings, lists, hashes, and sets while supporting active-standby, cluster, and sentinel deployment options. +- Architectural features: openAMDC employs a multi-threaded architecture to significantly enhance in-memory caching performance, while integrating a hot-cold data tiering mechanism to enable hybrid memory-drive storage. + 1. Multi-thread architecture: During initialization, openAMDC spawns multiple worker threads, each running an event loop for network monitoring. By enabling SO_REUSEPORT for socket listeners, kernel-level load balancing is implemented across threads sharing the same port. This approach eliminates resource contention from shared listening sockets through dedicated per-thread socket queues, substantially improving concurrency throughput. + 2. Data exchange architecture: Built upon the multi-threaded foundation, openAMDC implements data exchange capabilities supporting hybrid memory-drive storage, effectively optimizing total cost of ownership while maintaining performance efficiency. + +## OpenStack Antelope + +OpenStack is an open source project that provides a cloud computing management platform. It aims to deliver scalable and flexible cloud computing services to support private and public cloud environments. + +**Feature Description** + +OpenStack offers a series of services and tools to help build and manage public, private, and hybrid clouds. The service types include: + +- **Compute service**: creates, manages, and monitors VMs. It empowers users to quickly create, deploy, and destroy VMs and container instances, enabling flexible management and optimal utilization of computing resources. +- **Storage service**: provides object storage, block storage, file storage, and other storage. Block storage services, such as Cinder, allow users to dynamically allocate and manage persistent block storage devices, such as VM drives. Object storage services, such as Swift, provide a scalable and distributed object storage solution, facilitating storage of large amounts of unstructured data. +- Network service: empowers users to create, manage, and monitor virtual networks, and provides capabilities for topology planning, subnet management, and security group configuration. These features enable building of complex network structures while ensuring security and reliability. +- **Identity authentication service**: provides comprehensive identity management and access control capabilities, including user, role, and permissions management. It ensures secure access and management of cloud resources while safeguarding data confidentiality and integrity. +- **Image service**: enables image creation, management, and sharing through image uploading, downloading, and deletion. Users can perform management operations on images with ease and quickly deploy VM instances. +- **Orchestration service**: automates application deployment and management, and facilitates service collaboration and integration. Orchestration services like Heat help streamline application deployment and management by automatically perform related tasks based on user-defined templates. + +## openEuler DevStation + +openEuler DevStation is a Linux desktop OS built for developers, streamlining workflows while ensuring ecosystem compatibility. The latest release delivers major upgrades across three dimensions: supercharged toolchain, smarter GUI, and extended hardware support. These improvements create a more powerful, secure, and versatile development platform. + +**Feature Description** + +- Developer-centric community toolchain + + 1. Comprehensive development suite: Pre-configured with VSCodium (an open source, telemetry-free IDE) and development environments for major languages including Python, Java, Go, Rust, and C/C++. + 2. Enhanced tool ecosystem: Features innovative tools like oeDeploy for seamless deployment, epkg for extended package management, DevKit utilities, and an AI-powered coding assistant, delivering complete workflow support from environment configuration to production-ready code. + 3. oeDevPlugin Extension: A specialized VSCodium plugin for openEuler developers, providing, visual issue/PR dashboards, quick repository cloning and PR creation, automated code quality checks (such as license headers, formatting), real-time community task tracking. + 4. Intelligent assistant: Generates code from natural language prompts, creates API documentation with few clicks, and explains Linux commands, with a privacy-focused offline operation mode. + +- Enhanced GUI and productivity suite + + 1. Smart navigation and workspace: Features an adaptive navigation bar that intelligently organizes shortcuts for development tools, system utilities, and common applications—all with customizable workspace layouts. + 2. Built-in productivity applications: Comes with the Thunderbird email client pre-installed for seamless office workflows. + +- Hardware compatibility upgrades + + 1. Notebook-ready support: Comprehensive compatibility with modern laptop components, including precision touchpads, Wi-Fi 6/Bluetooth stacks, and multi-architectural drivers, delivering 20% faster AI and rendering workloads. + 2. Raspberry Pi DevStation image: Provides an Arm-optimized development environment out of the box, featuring a lightweight desktop environment with pre-installed IoT development tools (VScodium and oeDevPlugin) and accelerated performance for Python scientific computing libraries like NumPy and pandas. + +## oeDeploy for Simplified Software Deployment + +oeDeploy revolutionizes software deployment as a lightweight yet powerful tool that accelerates environment setup across single-node and distributed systems with unmatched efficiency. + +**Feature Description** + +- Universal deployment: Seamlessly handles both standalone and clustered deployments through automation, eliminating manual processes and slashing setup times. +- Pre-built software solutions: Comes with optimized deployment solutions for industry-standard software, with continuous expansion through a growing plugin ecosystem. +- Customizable architecture: Features an open plugin framework that empowers developers to build tailored deployment solutions aligned with their unique technical requirements. +- Developer-centric design: Combines robust CLI capabilities with upcoming GUI tools and a plugin marketplace, letting developers concentrate on innovation rather than infrastructure. diff --git a/docs/en/server/releasenotes/releasenotes/known-issues.md b/docs/en/server/releasenotes/releasenotes/known-issues.md new file mode 100644 index 0000000000000000000000000000000000000000..0a0ad3b3076b08d2113b7b40b33f10522d8c6d15 --- /dev/null +++ b/docs/en/server/releasenotes/releasenotes/known-issues.md @@ -0,0 +1,10 @@ +# Known Issues + +| No. | Issue ID | Description | Severity | Impact Analysis | Mitigation Measures | Historical Discovery Scenarios | +| --- | ----------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------- | ------------------------------ | +| 1 | [I5LZXD](https://gitee.com/src-openEuler/openldap/issues/I5LZXD) | Build issue with openldap in openEuler:22.09 | Minor | Test case failures occur during the build process. This is a test case design issue with limited impact. The problem can be temporarily resolved by using sleep to wait for operations to complete, though it may still fail under high load. | Skip the affected test cases and monitor the upstream community for a fix. | | +| 2 | [I5NLZI](https://gitee.com/src-openEuler/dde/issues/I5NLZI) | Abnormal icon display in the launcher (openEuler 22.09 rc2) | Minor | Only affects icon display in the DDE desktop launcher, with no functional impact. The usability issue is manageable. | Switch themes to avoid the issue. | | +| 3 | [I5P5HM](https://gitee.com/src-openEuler/afterburn/issues/I5P5HM) | Uninstallation error: `Failed to stop afterburn-sshkeys@.service` (22.09_RC3_EPOL, arm/x86) | Minor | | | | +| 4 | [I5PQ3O](https://gitee.com/src-openEuler/openmpi/issues/I5PQ3O) | Execution error with `ompi-clean -v -d` (openEuler-22.09-RC3) | Major | This package is specific to NestOS, with limited usage. It is enabled by default for the **core** user in NestOS, with minimal impact on the server version. | No mitigation measures have been provided by the SIG. | | +| 5 | [I5Q2FE](https://gitee.com/src-openEuler/udisks2/issues/I5Q2FE) | Build issue with udisks2 in openEuler:22.09 | Minor | Test case failures occur during the build process. The issue has not been reproduced in long-term local builds, and the environment was not retained. | Monitor the community build success rate. | | +| 6 | [I5SJ0R](https://gitee.com/src-openEuler/podman/issues/I5SJ0R) | Execution error with `podman create --blkio-weight-device /dev/loop0:123:15 fedora ls` (22.09RC5, arm/x86) | Minor | The blkio-weight feature is supported in the 4.xx kernel but not in the 5.10 version. | Track the upgrade of the podman component. | | diff --git a/docs/en/server/releasenotes/releasenotes/os-installation.md b/docs/en/server/releasenotes/releasenotes/os-installation.md new file mode 100644 index 0000000000000000000000000000000000000000..fbd928f05ee00e3e7be80abb61c8dfdd237876a0 --- /dev/null +++ b/docs/en/server/releasenotes/releasenotes/os-installation.md @@ -0,0 +1,161 @@ +# OS Installation + +## Release Files + +The openEuler release files include [ISO release package](https://www.openeuler.org/en/download/archive/), [VM images](http://repo.openeuler.org/openEuler-22.09/), [container images](http://repo.openeuler.org/openEuler-22.09/), [embedded images](http://repo.openeuler.org/openEuler-22.09/), and [repo sources](http://repo.openeuler.org/openEuler-22.09/). [Table 1](#table8396719144315) describes the ISO release packages. [Table 3](#table1276911538154) describes the container images. [Table 5](#table953512211576) describes the repo sources, which are convenient for online use. + +**Table 1** ISO release packages + + + +| Name | Description | +| ---------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | +| openEuler-22.09-aarch64-dvd.iso | Base installation ISO file of the AArch64 architecture, including the core components for running the minimum system. | +| openEuler-22.09-everything-aarch64-dvd.iso | Full installation ISO file of the AArch64 architecture, including all components for running the entire system. | +| openEuler-22.09-everything-debug-aarch64-dvd.iso | ISO file for openEuler debugging in the AArch64 architecture, including the symbol table information required for debugging. | +| openEuler-22.09-x86\_64-dvd.iso | Base installation ISO file of the x86\_64 architecture, including the core components for running the minimum system. | +| openEuler-22.09-everything-x86\_64-dvd.iso | Full installation ISO file of the x86\_64 architecture, including all components for running the entire system. | +| openEuler-22.09-everything-debuginfo-x86\_64-dvd.iso | ISO file for openEuler debugging in the x86\_64 architecture, including the symbol table information required for debugging. | +| openEuler-22.09-source-dvd.iso | ISO file of the openEuler source code. | +| openEuler-21.09-edge-aarch64-dvd.iso | Edge ISO file in the AArch64 architecture, including the core components for running the minimum system. | +| openEuler-21.09-edge-x86\_64-dvd.iso | Edge ISO file in the x86\_64 architecture, including the core components for running the minimum system. | +| openEuler-21.09-Desktop-aarch64-dvd.iso | Desktop ISO file in the AArch64 architecture, including the minimum software set for running the development desktop. | +| openEuler-21.09-Desktop-x86\_64-dvd.iso | Desktop ISO file in the x86\_64 architecture, including the minimum software set for running the development desktop. | + +**Table 2** VM images + + + +| Name | Description | +| -------------------------------- | -------------------------------------------------- | +| openEuler-22.09-aarch64.qcow2.xz | VM image of openEuler in the AArch64 architecture. | +| openEuler-22.09-x86\_64.qcow2.xz | VM image of openEuler in the x86\_64 architecture. | + +> [!NOTE]NOTE +> The default password of the **root** user of the VM image is **openEuler12#$**. Change the password upon the first login. + +**Table 3** Container images + + + +| Name | Description | +| ------------------------------- | ---------------------------------------------------------- | +| openEuler-docker.aarch64.tar.xz | AContainer image of openEuler in the AArch64 architecture. | +| openEuler-docker.x86\_64.tar.xz | Container image of openEuler in the x86\_64 architecture. | + +**Table 4** Embedded images + +| Name | Description | +| -------------------------------------- | ------------------------------------------------------------ | +| arm64/aarch64-std/zImage | Kernel image that supports QEMU in the AArch64 architecture. | +| arm64/aarch64-std/\*toolchain-22.09.sh | Development and compilation toolchain in the AArch64 architecture. | +| arm64/aarch64-std/\*rootfs.cpio.gz | File system that supports QEMU in the AArch64 architecture. | +| arm32/arm-std/zImage | Kernel image that supports QEMU in the ARM architecture. | +| arm32/arm-std/\*toolchain-22.09.sh | Development and compilation toolchain in the ARM architecture. | +| arm32/arm-std/\*rootfs.cpio.gz | File system that supports QEMU in the ARM architecture. | +| source-list/manifest.xml | Manifest of source code used for building. | + +**Table 5** Repo sources + + + +| Name | Description | +| --------------------- | ------------------------------------------ | +| ISO | Stores ISO images. | +| OS | Stores basic software package sources. | +| debuginfo | Stores debugging package sources. | +| docker\_img | Stores container images. | +| virtual\_machine\_img | Stores VM images. | +| embedded\_img | Stores embedded images. | +| everything | Stores full software package sources. | +| extras | Stores extended software package sources. | +| source | Stores source code software package. | +| update | Stores update software package sources. | +| EPOL | Stores extended openEuler package sources. | + +## Minimum Hardware Specifications + +[Table 6](#zh-cn_topic_0182825778_tff48b99c9bf24b84bb602c53229e2541) lists the minimum hardware specifications for installing openEuler 22.09. + +**Table 6** Minimum hardware requirements + +| Component | Minimum Hardware Specification | +| ---------- | -------------------------------------------------------------------------------------- | +| CPU | Kunpeng 920 (AArch64)/ x86\_64 (later than Skylake)

x86-64 (later than Skylake) | +| Memory | ≥ 8 GB | +| Hard drive | ≥ 120 GB | + +## Hardware Compatibility + +[Table 7](#zh-cn_topic_0227922427_table39822012) describes the typical configurations of servers and components supported by openEuler. openEuler will support more servers in the future. Partners and developers are welcome to participate in the contribution and verification. For details about the servers supported by openEuler, see [Compatibility List](https://www.openeuler.org/en/compatibility/). + +**Table 7** Supported servers and configurations + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Vendor

+

Server Name

+

Server Model

+

Component

+

Configuration

+

Huawei

+

TaiShan 200

+

2280 (balanced model)

+

CPU

+

HiSilicon Kunpeng 920

+

Memory

+

4 x 32 GB 2933MHz

+

RAID card

+

LSI SAS3508

+

Network

+

TM210

+

Huawei

+

FusionServer Pro

+

2288H V5 (rack server)

+

CPU

+

Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz

+

Memory

+

4 x 32 GB 2400MHz

+

RAID card

+

LSI SAS3508

+

Network

+

X722

+
diff --git a/docs/en/server/releasenotes/releasenotes/public_sys-resources/icon-note.gif b/docs/en/server/releasenotes/releasenotes/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/server/releasenotes/releasenotes/public_sys-resources/icon-note.gif differ diff --git a/docs/en/server/releasenotes/releasenotes/release-notes.md b/docs/en/server/releasenotes/releasenotes/release-notes.md new file mode 100644 index 0000000000000000000000000000000000000000..bd10870b01e4e96e4a6864b73463e456efc17a45 --- /dev/null +++ b/docs/en/server/releasenotes/releasenotes/release-notes.md @@ -0,0 +1,3 @@ +# Release Notes + +This document is the release notes of openEuler 22.09. diff --git a/docs/en/server/releasenotes/releasenotes/resolved-issues.md b/docs/en/server/releasenotes/releasenotes/resolved-issues.md new file mode 100644 index 0000000000000000000000000000000000000000..9a3f7ccfb0e54e0c67438559fae8431322fd51fb --- /dev/null +++ b/docs/en/server/releasenotes/releasenotes/resolved-issues.md @@ -0,0 +1,15 @@ +# Resolved Issues + +For the complete issue list, see [Issues](https://gitee.com/organizations/src-openeuler/issues). + +For the complete list kernel related commits, see [Commits](https://gitee.com/openeuler/kernel/commits/openEuler-21.03). + +For details about resolved issues, see [Table 1](#table249714911433). + +**Table 1** Resolved issues + +| ISSUE | Description | +| :----------------------------------------------------------- | :----------------------------------------------------------- | +|[I5J302](https://gitee.com/open_euler/dashboard?issue_id=I5J302)|\[Installation conflict arm/x86_64]openEuler 22.09: fwupd conflicts with dbxtool during installation| +|[I5J36Q](https://gitee.com/open_euler/dashboard?issue_id=I5J36Q)|\[Installation conflict arm/x86_64]openEuler 22.09: python3-wrapt conflicts during installation | +|[I5J3K1](https://gitee.com/open_euler/dashboard?issue_id=I5J3K1)|\[Installation conflict arm/x86_64]openEuler 22.09 : mariadb conflicts with mysql during installation| diff --git a/docs/en/server/releasenotes/releasenotes/source-code.md b/docs/en/server/releasenotes/releasenotes/source-code.md new file mode 100644 index 0000000000000000000000000000000000000000..f4ddca12ae04ba59655fe5cceb1e1e0af1c21e26 --- /dev/null +++ b/docs/en/server/releasenotes/releasenotes/source-code.md @@ -0,0 +1,8 @@ +# Source Code + +openEuler contains two code repositories: + +- Code repository: [https://gitee.com/openeuler](https://gitee.com/openeuler) +- Software package repository: [https://gitee.com/src-openeuler](https://gitee.com/src-openeuler) + +The openEuler release packages also provide the source ISO files. For details, see [OS Installation](./os-installation.md). diff --git a/docs/en/server/releasenotes/releasenotes/terms-of-use.md b/docs/en/server/releasenotes/releasenotes/terms-of-use.md new file mode 100644 index 0000000000000000000000000000000000000000..15862c99d7e0c212eccd6e07ece03d498e4b296a --- /dev/null +++ b/docs/en/server/releasenotes/releasenotes/terms-of-use.md @@ -0,0 +1,13 @@ +# Terms of Use + +**Copyright © 2025 openEuler Community** + +Your replication, use, modification, and distribution of this document are governed by the Creative Commons License Attribution-ShareAlike 4.0 International Public License \(CC BY-SA 4.0\). You can visit [https://creativecommons.org/licenses/by-sa/4.0/](https://creativecommons.org/licenses/by-sa/4.0/) to view a human-readable summary of \(and not a substitute for\) CC BY-SA 4.0. For the complete CC BY-SA 4.0, visit [https://creativecommons.org/licenses/by-sa/4.0/legalcode](https://creativecommons.org/licenses/by-sa/4.0/legalcode). + +**Trademarks and Permissions** + +All trademarks and registered trademarks mentioned in the documents are the property of their respective holders. The use of the openEuler trademark must comply with the [Use Specifications of the openEuler Trademark](https://www.openeuler.org/en/other/brand/). + +**Disclaimer** + +This document is used only as a guide. Unless otherwise specified by applicable laws or agreed by both parties in written form, all statements, information, and recommendations in this document are provided "AS IS" without warranties, guarantees or representations of any kind, including but not limited to non-infringement, timeliness, and specific purposes. diff --git a/docs/en/server/releasenotes/releasenotes/user-notice.md b/docs/en/server/releasenotes/releasenotes/user-notice.md new file mode 100644 index 0000000000000000000000000000000000000000..0047a4e9f26b66d36431bfc7311afd03351a882a --- /dev/null +++ b/docs/en/server/releasenotes/releasenotes/user-notice.md @@ -0,0 +1,5 @@ +# User Notice + +- The version number counting rule of openEuler is changed from openEuler _x.x_ to openEuler _year_._month_. For example, openEuler 21.03 indicates that the version is released in March 2021. +- The [Python core team](https://www.python.org/dev/peps/pep-0373/#update) has stopped maintaining Python 2 in January 2020. Python 2 reached end of maintenance (EOM) on December 31, 2020. In 2021, openEuler 21.03 fixed only the critical CVEs related to Python 2. +- From openEuler 22.03 LTS, only Python 3 is supported. Please switch to Python 3 to use the OS. diff --git a/docs/en/server/security/cert_signature/_toc.yaml b/docs/en/server/security/cert_signature/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..e1c81b24b46db0f9319a6fcb7e5dea7f35cc4b28 --- /dev/null +++ b/docs/en/server/security/cert_signature/_toc.yaml @@ -0,0 +1,11 @@ +label: Overview of Certificates and Signatures +isManual: true +href: ./overview_of_certificates_and_signatures.md +description: >- + The openEuler signature platform provides signing services to ensure the + integrity of system files. +sections: + - label: Introduction to Signature Certificates + href: ./introduction_to_signature_certificates.md + - label: Secure Boot + href: ./secure_boot.md diff --git a/docs/en/server/security/cert_signature/figures/cert-tree.png b/docs/en/server/security/cert_signature/figures/cert-tree.png new file mode 100644 index 0000000000000000000000000000000000000000..930a664600b31140c3939b1abd005cc2572cdbf9 Binary files /dev/null and b/docs/en/server/security/cert_signature/figures/cert-tree.png differ diff --git a/docs/en/server/security/cert_signature/figures/mokutil-db.png b/docs/en/server/security/cert_signature/figures/mokutil-db.png new file mode 100644 index 0000000000000000000000000000000000000000..82dbe6e04cafe3e9ac039ba19acd5996d4cf2259 Binary files /dev/null and b/docs/en/server/security/cert_signature/figures/mokutil-db.png differ diff --git a/docs/en/server/security/cert_signature/figures/mokutil-sb-off.png b/docs/en/server/security/cert_signature/figures/mokutil-sb-off.png new file mode 100644 index 0000000000000000000000000000000000000000..f3018c9fd0236e9c2cf560f0da3827ed2a877f6d Binary files /dev/null and b/docs/en/server/security/cert_signature/figures/mokutil-sb-off.png differ diff --git a/docs/en/server/security/cert_signature/figures/mokutil-sb-on.png b/docs/en/server/security/cert_signature/figures/mokutil-sb-on.png new file mode 100644 index 0000000000000000000000000000000000000000..449b6774dc61a601cf884845fbd0be5d314108e1 Binary files /dev/null and b/docs/en/server/security/cert_signature/figures/mokutil-sb-on.png differ diff --git a/docs/en/server/security/cert_signature/figures/mokutil-sb-unsupport.png b/docs/en/server/security/cert_signature/figures/mokutil-sb-unsupport.png new file mode 100644 index 0000000000000000000000000000000000000000..525c72f78b897ffaba0d356406ab9d9e64024d91 Binary files /dev/null and b/docs/en/server/security/cert_signature/figures/mokutil-sb-unsupport.png differ diff --git a/docs/en/server/security/cert_signature/introduction_to_signature_certificates.md b/docs/en/server/security/cert_signature/introduction_to_signature_certificates.md new file mode 100644 index 0000000000000000000000000000000000000000..3720dea42fdad92c6b1cae08087d1e6713307d60 --- /dev/null +++ b/docs/en/server/security/cert_signature/introduction_to_signature_certificates.md @@ -0,0 +1,46 @@ +# Introduction to Signature Certificates + +openEuler supports two signature mechanisms: openPGP and CMS, which are used for different file types. + +| File Type | Signature Type | Signature Format| +| --------------- | ------------ | -------- | +| EFI files | authenticode | CMS | +| Kernel module files | modsig | CMS | +| IMA digest lists| modsig | CMS | +| RPM software packages | RPM | openPGP | + +## openPGP Certificate Signing + +openEuler uses openPGP certificates to sign RPM software packages. The signature certificates are released with the OS image. You can obtain certificates used by openEuler in either of the following ways: + +Method 1: Download the certificate from the repository. For example, download the certificate of openEuler 24.03 LTS from the following address: + +```text +https://repo.openeuler.org/openEuler-24.03-LTS/OS/aarch64/RPM-GPG-KEY-openEuler +``` + +Method 2: Log in to the system and obtain the file from the specified path. + +```shell +cat /etc/pki/rpm-gpg/RPM-GPG-KEY-openEuler +``` + +## CMS Certificate Signing + +The openEuler signature platform uses a three-level certificate chain to manage signature private keys and certificates. + +![](./figures/cert-tree.png) + +Certificates of different levels have different validity periods. The current plan is as follows: + +| Type| Validity Period| +| -------- | ------ | +| Root certificate | 30 years | +| Level-2 certificate| 10 years | +| Level-3 certificate| 3 years | + +The openEuler root certificate can be downloaded from the community certificate center. + +```text +https://www.openeuler.org/en/security/certificate-center/ +``` diff --git a/docs/en/server/security/cert_signature/overview_of_certificates_and_signatures.md b/docs/en/server/security/cert_signature/overview_of_certificates_and_signatures.md new file mode 100644 index 0000000000000000000000000000000000000000..5b34fb2790887ffa71ef519565d902264e69afd3 --- /dev/null +++ b/docs/en/server/security/cert_signature/overview_of_certificates_and_signatures.md @@ -0,0 +1,29 @@ +# Overview of Certificates and Signatures + +## Overview + +Digital signature is an important technology for protecting the integrity of OSs. By adding signatures to key system components and verifying the signatures in subsequent processes such as component loading and running, you can effectively check component integrity and prevent security problems caused by component tampering. Multiple system integrity protection mechanisms are supported in the industry to protect the integrity of different types of components in each phase of system running. Typical technical mechanisms include: + +- Secure boot +- Kernel module signing +- Integrity measurement architecture (IMA) +- RPM signature verification + +The preceding integrity protection security mechanisms depend on signatures (usually integrated in the component release phase). However, open source communities generally lack signature private keys and certificate management mechanisms. Therefore, Linux distributions released by open source communities generally do not provide default signatures or use only private keys temporarily generated in the build phase for signatures. Usually, these integrity protection security mechanisms can be enabled only after users or downstream OSVs perform secondary signing, which increases the cost of security functions and reduces usability. + +## Solution + +The openEuler community infrastructure supports the signature service. The signature platform manages signature private keys and certificates in a unified manner and works with the EulerMaker build platform to automatically sign key files during the software package build process of the community edition. Currently, the following file types are supported: + +- EFI files +- Kernel module files +- IMA digest lists +- RPM software packages + +## Constraints + +The signature service of the openEuler community has the following constraints: + +- Currently, only official releases of the openEuler community can be signed. Private builds cannot be signed. +- Currently, only EFI files related to OS secure boot can be signed, including shim, GRUB, and kernel files. +- Currently, only the kernel module files provided by the kernel software package can be signed. diff --git a/docs/en/server/security/cert_signature/secure_boot.md b/docs/en/server/security/cert_signature/secure_boot.md new file mode 100644 index 0000000000000000000000000000000000000000..fb8f217924ac306bfaac7dea46457a48c2edc009 --- /dev/null +++ b/docs/en/server/security/cert_signature/secure_boot.md @@ -0,0 +1,58 @@ +# Secure Boot + +## Overview + +Secure Boot relies on public and private key pairs to sign and verify components in the booting process. During booting, the previous component authenticates the digital signature of the next component. If the authentication is successful, the next component runs. If the authentication fails, the booting stops. Secure Boot ensures the integrity of each component during system booting and prevents unauthenticated components from being loaded and running, preventing security threats to the system and user data. + +Components to be authenticated and loaded in sequence in Secure Boot are BIOS, shim, GRUB, and vmlinuz (kernel image). + +Related EFI startup components are signed by the openEuler signature platform using signcode. The public key certificate is integrated into the signature database by the BIOS. During the boot, the BIOS verifies the shim. The shim and GRUB components obtain the public key certificate from the signature database of the BIOS and verify the next-level components. + +## Background and Solutions + +In earlier openEuler versions, secure boot components are not signed. Therefore, the secure boot function cannot be directly used to ensure the integrity of system components. + +In openEuler 22.03 LTS SP3 and later versions, openEuler uses the community signature platform to sign OS components, including the GRUB and vmlinuz components, and integrates the community signature root certificate in the shim component. + +For the shim component, to facilitate end-to-end secure boot, the signature platform of the openEuler community is used for signature. After external CAs officially operate the secure boot component signature service, the signatures of these CAs will be integrated into the shim module of openEuler. + +## Usage + +### Obtaining the openEuler Certificate + +To obtain the openEuler root certificate, visit and download it from the **Certificate Center** directory. + +The root certificate name on the web page are **openEuler Shim Default CA** and **default-x509ca.cert**. + +### Operation in the BIOS + +Import the openEuler root certificate to the certificate database of the BIOS and enable secure boot in the BIOS. + +For details about how to import the BIOS certificate and enable secure boot, see the documents provided by the BIOS vendor. + +### Operation in the OS + +Check the certificate information in the database: `mokutil -db` + +![](./figures/mokutil-db.png) +Note: There is a large amount of certificate information. Only some important information is displayed in the screenshot. + +Check the secure boot status: `mokutil --sb` + +- **SecureBoot disabled**: Secure boot is disabled. + +![](./figures/mokutil-sb-off.png) + +- **SecureBoot enabled**: Secure boot is enabled. + +![](./figures/mokutil-sb-on.png) + +- **not supported**: The system does not support secure boot. + +![](./figures/mokutil-sb-unsupport.png) + +## Constraints + +- **Software**: The OS must be booted in UEFI mode. +- **Architecture**: Arm or x86 +- **Hardware**: The BIOS must support the verification function related to secure boot. diff --git a/docs/en/server/security/cve-ease/_toc.yaml b/docs/en/server/security/cve-ease/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..1fc91df78465285668ff15efa9009c41389cfa6f --- /dev/null +++ b/docs/en/server/security/cve-ease/_toc.yaml @@ -0,0 +1,8 @@ +label: CVE-ease Design Overview +isManual: true +description: Quickly respond to system vulnerabilities. +sections: + - label: CVE-ease Design Overview + href: ./cve-ease-design-overview.md + - label: CVE-ease Introduction and Installation + href: ./cve-ease-introduction-and-installation.md diff --git a/docs/en/server/security/cve-ease/cve-ease-design-overview.md b/docs/en/server/security/cve-ease/cve-ease-design-overview.md new file mode 100644 index 0000000000000000000000000000000000000000..52bc28b7403ec053dbb9f2b6b776913fdd70af13 --- /dev/null +++ b/docs/en/server/security/cve-ease/cve-ease-design-overview.md @@ -0,0 +1,33 @@ +# CVE-ease Design Overview + +## 1. Overview + +Common Vulnerabilities and Exposures (CVEs) plays a vital role in ensuring system security and stability, making effective handling and management of CVE data essential. To address this, the CVE-ease vulnerability management system was developed, enabling real-time acquisition, management, and reporting of vulnerability information. + +## 2. Key Features + +CVE-ease provides the following core functionalities: + +- Acquisition and analysis of CVE information +- Organization and storage of CVE data +- Access to historical and real-time CVE details +- Real-time tracking of CVE status +- Immediate broadcasting of CVE updates + +## 3. Module Functions + +![](./figures/CVE-ease_desigin_table.png) + +### 3.1 CVE Acquisition + +During operation, CVE-ease periodically scrapes CVE information from disclosure websites. Before scraping, the system scans the CVE database to create an index of existing CVE IDs and verify connectivity with CVE platforms. The process begins by retrieving the IDs of the latest disclosed CVEs, followed by fetching detailed descriptions based on these IDs. If the data is successfully retrieved, the process concludes. Otherwise, the system retries until successful. + +### 3.2 CVE Information Organization and Storage + +Scraped CVE data is structured and stored in a database according to predefined formats, with feature values calculated. If a scraped CVE ID is not present in the database, it is added directly. If the ID exists, the system compares feature values and updates the entry if discrepancies are found. + +### 3.3 Viewing Historical and Real-Time CVE Information + +Users can query specific CVE details through the interactive interface. By default, the system displays the 10 most recent CVEs. Users can customize queries to retrieve historical CVE data based on criteria such as CVE score or year. + +![](./figures/CVE-ease_function.png) diff --git a/docs/en/server/security/cve-ease/cve-ease-introduction-and-installation.md b/docs/en/server/security/cve-ease/cve-ease-introduction-and-installation.md new file mode 100644 index 0000000000000000000000000000000000000000..672af6be5adb416f086e88b6de17e1041f454cd4 --- /dev/null +++ b/docs/en/server/security/cve-ease/cve-ease-introduction-and-installation.md @@ -0,0 +1,566 @@ +# CVE-ease + +## Project Overview + +CVE-ease is a dedicated platform for CVE information, aggregating data from community releases and delivering timely notifications via email, WeChat, DingTalk, and other channels. Users can access detailed CVE information on the platform, such as vulnerability descriptions, affected scopes, and remediation recommendations. This enables them to choose appropriate fixes tailored to their systems. + +The platform is designed to help users swiftly identify and address vulnerabilities, thereby improving system security and stability. + +CVE-ease is an **independent innovation initiative by CTYun**. Open sourced in the openEuler community, it strictly adheres to the **Mulan PSL2** license. We welcome community contributions to the project, working together to create a secure, stable, and reliable ecosystem for domestic operating systems. + +Open source details: + +- This repository **strictly** complies with the [Mulan Permissive Software License, Version 2](http://license.coscl.org.cn/MulanPSL2). +- **This repository meets the open source guidelines of China Telecom Cloud Technology Co., Ltd, having undergone thorough review and preparation to present a high-quality open source project with complete documentation and resources**. +- The repository is managed by designated personnel from the company, ensuring **long-term maintenance for LTS versions** and ongoing development support. + +## Software Architecture + +CVE-ease is a platform dedicated to CVE information, structured around four core modules: CVE crawler, CVE analyzer, CVE notifier, and CVE frontend. Below is an overview of functionality and design of each module. + +- **CVE crawler** + +This module collects CVE information from multiple data sources provided by the openEuler community and stores it in relational databases like MySQL. These data sources are primarily managed by the cve-manager project, which supports fetching CVE details from NVD, CNNVD, CNVD, RedHat, Ubuntu, and Debian. CVE-ease employs Python-based crawler scripts, each tailored to a specific data source. These scripts can be executed on a schedule or manually, formatting the scraped raw CVE data and storing it for further analysis. + +- **CVE analyzer** + +This module processes CVE information by parsing, categorizing, and scoring it. Written in Python, the analyzer script periodically retrieves raw CVE data from the relational database and performs tasks such as extracting basic attributes (such as ID, title, description), categorizing the impact scope (such as OS, software packages), scoring severity (such as CVSS scores), and matching remediation suggestions (such as patch links). The processed structured data is then stored in SQL format for future queries and presentation. + +- **CVE notifier** + +This module sends CVE notifications to users via email, WeChat, DingTalk, and other channels based on their subscription preferences. The Python-based notifier script periodically retrieves structured CVE data from the MySQL database, filters it according to user-configured impact scopes, generates appropriate notification content for different channels, and invokes APIs to deliver notifications. The script also logs sending results and feedback, updating subscription statuses in the database. + +- **CVE frontend** + +This module offers a user-friendly CLI terminal command, enabling users to view, search, and subscribe to CVE information. + +The architecture of CVE-ease is designed to create an efficient, flexible, and scalable platform, providing users with timely and accurate security vulnerability intelligence. + +## Development Roadmap + +1. Adapt repodata to support multiple OSVs. +2. Add MOTD login broadcast functionality. +3. Enhance the DNF plugin to include patching capabilities. +4. Implement automatic patching for specific packages. +5. Introduce specific package awareness features. +6. ... + +**We highly value your feedback on the development direction of CVE-ease. If you have any suggestions or ideas, feel free to share them with us. Your input is greatly appreciated!** + +## Installation Guide + +CVE-ease is in fast-paced development, offering installation methods such as direct installation, container installation, and RPM package installation. + +### Direct Installation + +```shell +git clone https://gitee.com/openeuler/cve-ease cve-ease.git +cd cve-ease.git/cve-ease +make install +``` + +### Container Installation + +```shell +git clone https://gitee.com/openeuler/cve-ease cve-ease.git +cd cve-ease.git/cve-ease +make run-in-docker +``` + +### RPM Package Installation + +```shell +git clone https://gitee.com/openeuler/cve-ease cve-ease.git +cd cve-ease.git/cve-ease +make gensrpm +cd .. +rpm -ivh *.src.rpm +cd ~/rpmbuild +rpmbuild -ba SPECS/cve-ease.spec +cd RPMS/noarch +rpm -ivh *.rpm +``` + +## Usage Guide + +### Help Information + +- Running the `cve-ease` command without options displays the help menu. +- The `cve-ease` command includes multiple subcommands, organized into `basic`, `info`, and `notifier` categories. +- Use the `help` subcommand to view detailed information for each command category. + +```shell +# cve-ease + +Available commands: + +basic commands: + config Print cve-ease config + daemon Run as daemon without interactive + motd Motd info manager + service Service manager + +info commands: + cve OpenEuler CVE info + cvrf OpenEuler CVRF info + db Database manager + help List available commands + logger Logger config + repodata Repodata info + rpm Rpm info + sa OpenEuler security notice info + +notifier commands: + dingding Notifier of dingding + feishu Notifier of feishu + mail163 Notifier of mail163 + mailqq Notifier of mailqq + wecom Notifier of wecom + +Try "cve-ease --help" for help about global gconfig +Try "cve-ease help" to get all available commands +Try "cve-ease --help" for help about the gconfig of a particular command +Try "cve-ease help " to get commands under a particular category +Available commands are: basic, info, notifier + +# cve-ease help info +Available commands: + +info commands: + cve OpenEuler CVE info + cvrf OpenEuler CVRF info + db Database manager + help List available commands + logger Logger config + repodata Repodata info + rpm Rpm info + sa OpenEuler security notice info + +Try "cve-ease --help" for help about global gconfig +Try "cve-ease help" to get all available commands +Try "cve-ease --help" for help about the gconfig of a particular command +Try "cve-ease help " to get commands under a particular category +Available commands are: basic, info, notifier +``` + +### Configuration File + +The configuration file is located at `/etc/cve-ease/cve-ease.cfg`. + +```ini +[main] +pid_file_path = /var/log/cve-ease/cve-ease.pid +lock_file_path = /var/log/cve-ease/cve-ease.lock + +# log configuration + +# debug/ error(default) / warn +log_level = debug +log_file_path = /var/log/cve-ease/cve-ease.log +log_maxbytes = 10240 +log_backup_num = 30 + +# sql configuration +db_type = sqlite +db_file_path = /usr/share/cve-ease/cve-ease.db +db_user = +db_password = +db_host = +db_port = +product = openEuler-23.09 +expiration_days = 14 + +# notifier +notifier_record_num = 9 + +# filter +focus_on = kernel,systemd,openssh,openssl + +[wecom] +enabled = 1 +# https://developer.work.weixin.qq.com/document/path/91770?version=4.0.19.6020&platform=win +# https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=fe9eae1f-xxxx-4ae3-xxxx-ecf9f77abba6 + +update_key = 2142ef2a-d99d-417d-8c31-b550b0fcb4e3 +status_key = 2142ef2a-d99d-417d-8c31-b550b0fcb4e3 + + +[dingding] +enabled = 1 +# just for test +update_key = 81907155a6cc88004e1ed6bcdd86c68d5b21565ed59d549ca031abc93d90d9cb +status_key = 81907155a6cc88004e1ed6bcdd86c68d5b21565ed59d549ca031abc93d90d9cb + + +[feishu] +enabled = 1 +# just for test +update_key = 5575739b-f59d-48db-b737-63672b2c32ab +status_key = 5575739b-f59d-48db-b737-63672b2c32ab + + +[mail163] +enabled = 0 +mail_sender = xxxxxxx@163.com +mail_recver = xxxxxxx@163.com +mail_smtp_token = xxxxxx + + +[mailqq] +enabled = 0 +mail_sender = xxxxxxx@qq.com +mail_recver = xxxxxxx@qq.com +mail_smtp_token = xxxxxxxx +``` + +### CVE-ease + +The CVE-ease service consists of two files, **cve-ease.service** and **cve-ease.timer**, utilizing the systemd timer functionality for scheduled execution. + +```ini +# /usr/lib/systemd/system/cve-ease.timer +# CTyunOS cve-ease: MulanPSL2 +# +# This file is part of cve-ease. +# + +[Unit] +Description=CTyunOS cve-ease Project +Documentation=https://gitee.com/openeuler/cve-ease + +[Timer] +OnBootSec=1m +OnUnitActiveSec=10m +RandomizedDelaySec=10 + +[Install] +WantedBy=timers.target +``` + +```shell +# systemctl enable --now cve-ease.timer +Created symlink /etc/systemd/system/timers.target.wants/cve-ease.timer → /usr/lib/systemd/system/cve-ease.timer. +# systemctl status cve-ease.timer +● cve-ease.timer - CTyunOS cve-ease Project + Loaded: loaded (/usr/lib/systemd/system/cve-ease.timer; enabled; vendor preset: disabled) + Active: active (waiting) since Sat 2023-03-18 17:55:53 CST; 5s ago + Trigger: Sat 2023-03-18 18:05:55 CST; 9min left + Docs: https://gitee.com/openeuler/cve-ease + +Mar 18 17:55:53 56d941221b41 systemd[1]: Started CTyunOS cve-ease Project. +# systemctl status cve-ease.service +● cve-ease.service - CTyunOS cve-ease project + Loaded: loaded (/usr/lib/systemd/system/cve-ease.service; disabled; vendor preset: disabled) + Active: inactive (dead) since Sat 2023-03-18 17:55:56 CST; 5s ago + Docs: https://gitee.com/openeuler/cve-ease + Process: 196 ExecStart=/usr/bin/cve-ease daemon (code=exited, status=0/SUCCESS) + Main PID: 196 (code=exited, status=0/SUCCESS) + +Mar 18 17:55:53 56d941221b41 systemd[1]: Starting CTyunOS cve-ease project... +Mar 18 17:55:56 56d941221b41 systemd[1]: cve-ease.service: Succeeded. +Mar 18 17:55:56 56d941221b41 systemd[1]: Started CTyunOS cve-ease project. +``` + +### basic Commands + +#### config + +```shell +Usage: cve-ease config +(Specify the --help global option for a list of other help options) + +Options: + -h, --help show this help message and exit + -r, --rawdata print raw config file content +``` + +```shell +cve-ease config # Display the configuration file path and active settings. +cve-ease config -r # Display the configuration file path and raw data. +``` + +#### daemon + +- The `daemon` command acts as the entry point for the systemd service and is typically not run manually. +- The service is executed periodically by the systemd timer associated with cve-ease. + +```ini +# /usr/lib/systemd/system/cve-ease.service +# CTyunOS cve-ease: MulanPSL2 +# +# This file is part of cve-ease. +# + +[Unit] +Description=CTyunOS cve-ease project +Documentation=https://gitee.com/openeuler/cve-ease + +[Service] +Type=oneshot +ExecStart=/usr/bin/cve-ease daemon + +[Install] +WantedBy=multi-user.target +``` + +#### motd + +- TODO. + +#### service + +`service` command options for controlling the cve-ease service: + +```shell +Usage: cve-ease service +(Specify the --help global option for a list of other help options) + +Options: + -h, --help show this help message and exit + -k, --kill kill cve-ease service + -r, --restart restart cve-ease service + -s, --status get cve-ease service status + -v, --verbose show verbose output +``` + +```shell +cve-ease service -k # Pause the cve-ease service +cve-ease service -r # Restart the cve-ease service +cve-ease service -s # Query the cve-ease service status +``` + +### info Commands + +#### cve + +Retrieve CVE data from the openEuler community in the [openEuler Security Center](https://www.openeuler.org/en/security/cve/). + +```shell +Usage: cve-ease cve +(Specify the --help global option for a list of other help options) + +Options: + -h, --help show this help message and exit + -r, --rawdata get cve cache and print raw data without write db + -m, --makecache get cve cache + -l, --list list all cve info + -t, --total get cve info statistics + -v, --verbose show verbose output +``` + +```shell +cve-ease cve -m # Collect CVE data and store it in the database. +cve-ease cve -l # Fetch CVE data from the database and format it for display. +cve-ease cve -t # Retrieve and show CVE statistics from the database. +cve-ease cve -r # Gather CVE data and display it in raw form (without saving to the database). +``` + +#### sa + +Retrieve security advisory (SA) data from the openEuler community in the [openEuler Security Center](https://www.openeuler.org/en/security/cve/). + +```shell +Usage: cve-ease sa +(Specify the --help global option for a list of other help options) + +Options: + -h, --help show this help message and exit + -r, --rawdata get sa cache and print raw data without write db + -m, --makecache get sa cache + -l, --list list all sa info + -t, --total get sa info statistics + -v, --verbose show verbose output +``` + +```shell +cve-ease sa -m # Collect SA data and store it in the database. +cve-ease sa -l # Fetch SA data from the database and format it for display. +cve-ease sa -t # Retrieve and show SA statistics from the database. +cve-ease sa -r # Gather SA data and display it in raw form (without saving to the database). +``` + +#### cvrf + +Common Vulnerability Reporting Framework (CVRF)-related commands: + +```shell +cve-ease cvrf -m # Collect CVRF data and store it in the database. +cve-ease cvrf -l # Fetch CVRF data from the database and format it for display. +cve-ease cvrf -t # Retrieve and show CVRF statistics from the database. +``` + +#### rpm + +```shell +Usage: cve-ease rpm +(Specify the --help global option for a list of other help options) + +Options: + -h, --help show this help message and exit + -l, --list list all rpm info + -v, --verbose show verbose output +``` + +```shell +cve-ease rpm -l # Use the RPM interface to retrieve and display details of installed RPM packages in the system. +``` + +#### repodata + +```shell +Usage: cve-ease repodata +(Specify the --help global option for a list of other help options) + +Options: + -h, --help show this help message and exit + -m, --makecache cache repodata to database + -p PRODUCT, --product=PRODUCT + specific product (work with --check) + --osv=OSV specific osv rpm release + -t, --total get total rpm statistics + -l, --list list all rpm + -c, --check check repo cve + -v, --verbose show verbose output +``` + +```bash +cve-ease repodata -p ctyunos2 -m # Set ctyunos2 as the OSV version, cache its repository metadata, and store it in the database. +cve-ease repodata --osv ctyunos2 -p openEuler-23.09 -c # Compare the ctyunos2 repository with the openEuler 23.09 repository. +cve-ease repodata -l # Display the package details available in the database. +cve-ease repodata -t # Fetch and show statistics for the repository data in the database. +``` + +#### logger + +```shell +Usage: cve-ease logger +(Specify the --help global option for a list of other help options) + +Options: + -h, --help show this help message and exit + -l, --list list all logger info + -t, --total get logger statistics + -v, --verbose show verbose output +``` + +#### db + +```shell +Usage: cve-ease db +(Specify the --help global option for a list of other help options) + +Options: + -h, --help show this help message and exit + -p, --purge purge db and recreate it (Danger Operation) + -s, --stats get database statistics + -v, --verbose show verbose output +``` + +### notifier Commands + +#### wecom + +```shell +Usage: cve-ease wecom +(Specify the --help global option for a list of other help options) + +Options: + -h, --help show this help message and exit + -t, --test run test + -v, --verbose show verbose output + -c CONTENT, --content=CONTENT + show verbose output +``` + +```shell +cve-ease wecom -t # Send a test message to a WeCom group. +cve-ease wecom -t -c 'helloworld' # Send a custom test message to a WeCom group. +``` + +#### dingding + +```shell +Usage: cve-ease dingding +(Specify the --help global option for a list of other help options) + +Options: + -h, --help show this help message and exit + -t, --test run test + -v, --verbose show verbose output + -c CONTENT, --content=CONTENT + show verbose output +``` + +```shell +cve-ease dingding -t # Send a test message to a DingTalk group. +cve-ease dingding -t -c 'helloworld' # Send a custom test message to a DingTalk group. +``` + +#### feishu + +```shell +Usage: cve-ease feishu +(Specify the --help global option for a list of other help options) + +Options: + -h, --help show this help message and exit + -t, --test run test + -v, --verbose show verbose output + -c CONTENT, --content=CONTENT + show verbose output +``` + +```shell +cve-ease feishu -t # Send a test message to a Lark group. +cve-ease feishu -t -c 'helloworld' # Send a custom test message to a Lark group. +``` + +#### mail163 + +```shell +Usage: cve-ease mail163 +(Specify the --help global option for a list of other help options) + +Options: + -h, --help show this help message and exit + -t, --test run test + -v, --verbose show verbose output + -c CONTENT, --content=CONTENT + show verbose output +``` + +```shell +cve-ease mail163 -t # Send a test message to a 163 mail address. +cve-ease mail163 -t -c 'helloworld' # Send a custom test message to a 163 mail address. +``` + +#### mailqq + +```shell +Usage: cve-ease mailqq +(Specify the --help global option for a list of other help options) + +Options: + -h, --help show this help message and exit + -t, --test run test + -v, --verbose show verbose output + -c CONTENT, --content=CONTENT + show verbose output +``` + +```shell +cve-ease mailqq -t # Send a test message to a QQ mail address. +cve-ease mailqq -t -c 'helloworld' # Send a custom test message to a QQ mail address. +``` + +## How to Contribute + +1. Fork the repository. +2. Since the project is in fast-paced development with only the master branch active, make your changes on the master branch and submit them. +3. Create a pull request (PR) with a clear description of its functionality and purpose, along with relevant test cases. +4. Notify the repository maintainer to review your PR. + +## Core Developers and Contact Details + +- You Yifeng - [Gitee Private Message](https://gitee.com/youyifeng) +- Wu Kaishun - [Gitee Private Message](https://gitee.com/wuzimo) diff --git a/docs/en/server/security/cve-ease/figures/CVE-ease_desigin_table.png b/docs/en/server/security/cve-ease/figures/CVE-ease_desigin_table.png new file mode 100644 index 0000000000000000000000000000000000000000..c02a3569bca30fd225c048360e66a2cf052bc84e Binary files /dev/null and b/docs/en/server/security/cve-ease/figures/CVE-ease_desigin_table.png differ diff --git a/docs/en/server/security/cve-ease/figures/CVE-ease_function.png b/docs/en/server/security/cve-ease/figures/CVE-ease_function.png new file mode 100644 index 0000000000000000000000000000000000000000..3f4119eddcfd5eef088123895c5b39a040f0526e Binary files /dev/null and b/docs/en/server/security/cve-ease/figures/CVE-ease_function.png differ diff --git a/docs/en/server/security/sbom/_toc.yaml b/docs/en/server/security/sbom/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..3b949f21fc2bdc2f22f98ab14cebec91c57240d7 --- /dev/null +++ b/docs/en/server/security/sbom/_toc.yaml @@ -0,0 +1,8 @@ +label: SBOM User Guide +isManual: true +description: >- + The SBOM provides a unique identification of software components and their + contents. +sections: + - label: Introduction to SBOM + href: ./sbom.md diff --git a/docs/en/server/security/sbom/figures/image.png b/docs/en/server/security/sbom/figures/image.png new file mode 100644 index 0000000000000000000000000000000000000000..b4bfa78fee5662ed919d3f2fe76fa407f20f9ec9 Binary files /dev/null and b/docs/en/server/security/sbom/figures/image.png differ diff --git a/docs/en/server/security/sbom/sbom.md b/docs/en/server/security/sbom/sbom.md new file mode 100644 index 0000000000000000000000000000000000000000..61cf9ba6663be1a3e9f3eb705a00bb0448010bcc --- /dev/null +++ b/docs/en/server/security/sbom/sbom.md @@ -0,0 +1,50 @@ +# 1. Introduction to SBOM + +A Software Bill of Materials (SBOM) serves as a formal, machine-readable inventory that uniquely identifies software components and their contents. Beyond basic identification, it tracks copyright and licensing details. Organizations use SBOM to enhance supply chain transparency, and it is rapidly becoming a mandatory deliverable in software distribution. + +# 2. SBOM Core Requirements + +The National Telecommunications and Information Administration (NTIA) has established baseline requirements for SBOM implementation. These essential data elements enable component tracking throughout the software supply chain and serve as the foundation for extended features such as license tracking and vulnerability monitoring. + +| Core Field | Definition | +| ------------------------------- | ------------------------------------------------------------ | +| Supplier | Entity responsible for component creation and identification | +| Component | Official designation of the software unit | +| Version | Tracking identifier for component iterations | +| Other identifiers | Supplementary reference keys | +| Dependencies | Mapping of component relationships and inclusions | +| SBOM author | Entity generating the SBOM documentation | +| Timestamp | SBOM generation date and time | +| **Recommended Optional Fields** | | +| Component hash | Digital fingerprint for security verification | +| Lifecycle phase | Development stage at SBOM creation | + +# 3. openEuler SBOM Implementation + +openEuler's SBOM framework incorporates extensive metadata tracking through SPDX, including: + +| Base Field | SPDX Path | +| ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Supplier | document->packages->supplier | +| Name | document->packages->name | +| Version | document->packages->versionInfo (epoch:version-release in openEuler) | +| Other identifiers | document->packages->externalRefs->purl | +| Dependencies | document->packages->externalRefs->purl | +| SBOM author | document->creationInfo->creators | +| Timestamp | document->creationInfo->created | +| Component hash | document->packages->checksums | +| Lifecycle phase | Not supported | +| Other relationships | Internal subcomponents: document->packages->externalRefs(category:PROVIDE_MANAGER)->purl
Runtime dependencies: document->relationships(relationshipType:DEPENDS_ON) | +| License info | document->packages->licenseDeclared document->packages->licenseConcluded | +| Copyright info | document->packages->copyrightText | +| Upstream community | document->packages->externalRefs(category:SOURCE_MANAGER)->url | +| Patch information | Patch files: document->files(fileTypes:SOURCE)
Patch relationships: document->relationships(relationshipType:PATCH_APPLIED) | +| Component source | document->packages->downloadLocation | +| Component details | document->packages->description document->packages->summary | +| Website/Blog | document->packages->homepage | + +# 4. SBOM Structure + +The system uses RPM packages as the fundamental unit for SBOM generation and analysis. + +![](./figures/image.png) diff --git a/docs/en/server/security/secgear/_toc.yaml b/docs/en/server/security/secgear/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..6b82ce495c3848353d45dd7988bd9b454853f71d --- /dev/null +++ b/docs/en/server/security/secgear/_toc.yaml @@ -0,0 +1,17 @@ +label: secGear Developer Guide +isManual: true +href: ./secgear.md +Description: Build applications with secGear to safeguard data during cloud operations. +sections: + - label: Introduction to secGear + href: ./introduction-to-secgear.md + - label: secGear Installation + href: ./secgear-installation.md + - label: API Reference + href: ./api-reference.md + - label: Developer Guide + href: ./developer-guide.md + - label: secGear Tools + href: ./using-secgear-tools.md + - label: Application Scenarios + href: ./application-scenarios.md diff --git a/docs/en/server/security/secgear/api-reference.md b/docs/en/server/security/secgear/api-reference.md new file mode 100644 index 0000000000000000000000000000000000000000..9ada319006d9739e2fc12a39972fd9b9462cb42d --- /dev/null +++ b/docs/en/server/security/secgear/api-reference.md @@ -0,0 +1,357 @@ +# API Reference + +The secGear unified programming framework for confidential computing consists of the TEE and REE. This section describes the APIs required for developing applications. In addition to these APIs, the TEE inherits the open-source POSIC APIs of ARM TrustZone and Intel SGX. + +## cc_enclave_create + +Creates an enclave API. + +**Function:** + +Initialization API. The function calls different TEE creation functions based on the type to initialize the enclave context in different TEE solutions. This API is called by the REE. + +> ![](./public_sys-resources/icon-note.gif) **Note:** +> Due to Intel SGX restrictions, memory mapping contention exists when multiple thread invoke cc_enclave_create concurrently. As a result, the creation of the enclave API may fail. Avoid concurrent invocations of cc_enclave_create in your code. + +**Function Declaration:** + +```c +cc_enclave_result_t cc_enclave_create(const char* path, enclave_type_t type, uint32_t version,uint32_t flags,const enclave_features_t* features,uint32_t features_count, + cc_enclave_t ** enclave); +``` + +**Parameters:** + +- Path: input parameter, which specifies a path of the enclave to be loaded. +- Type: input parameter, which specifies the TEE solution, for example, SGX_ENCLAVE_TYPE, GP_ENCLAVE_TYPE and AUTO_ENCLAVE_TYPE. +- version: input parameter, which specifies the enclave engine version. Currently, there is only one version, and the value is 0. +- Flags: input parameter, which specifies the running status of the enclave. For example, SECGEAR_DEBUG_FLAG indicates the debugging status, and SECGEAR_SIMULATE_FLAG indicates the simulation status (not supported currently). +- features: input parameter, which specifies some features supported by the enclave, for example, PCL and switchless of the SGX. This parameter is not supported currently. Set it to NULL. +- features_count: input parameter, which specifies the number of features. This parameter is not supported currently. Set it to 0. +- enclave: output parameter, which specifies the created enclave context. + +**Return Values:** + +- CE_SUCCESS: The authentication information is verified successfully. +- CE_ERROR_INVALID_PARAMETER: The input parameter is incorrect. +- CE_ERROR_OUT_OF_MEMORY: No memory is available. +- CC_FAIL: Common failure. +- CC_ERROR_UNEXPECTED: Unexpected error. +- CC_ERROR_ENCLAVE_MAXIMUM: The number of enclaves created by a single app reaches the maximum. +- CC_ERROR_INVALID_PATH: The secure binary path is invalid. +- CC_ERROR_NO_FIND_REGFUNC: The enclave search fails. + +## cc_enclave_destroy + +Destroys the enclave API. + +**Function:** + +This API is called by the REE to call the exit functions of different TEEs to release the created enclave entities. + +**Function Declaration:** + +```c +cc_enclave_result_t cc_enclave_destroy (cc_enclave_t ** enclave); +``` + +**Parameter:** + +- enclave: input parameter, which specifies the context of the created enclave. + +**Return Values:** + +- CE_SUCCESS: The authentication information is verified successfully. +- CE_ERROR_INVALID_PARAMETER: The input parameter is incorrect. +- CE_ERROR_OUT_OF_MEMORY: No memory is available. +- CC_ERROR_NO_FIND_UNREGFUNC: The enclave search fails. +- CC_FAIL: common failure. +- CC_ERROR_UNEXPECTED: unexpected error. + +## cc_malloc_shared_memory + +Creates the shared memory. + +**Functions** + +After the switchless feature is enabled, this API is called by the REE to create the shared memory that can be accessed by both the TEE and REE. + +**Function Declaration:** + +```c +void *cc_malloc_shared_memory(cc_enclave_t *enclave, size_t size); +``` + +**Parameters:** + +- enclave: input parameter, which indicates the context handle of the secure environment. Different platforms have different shared memory models. To ensure cross-platform interface consistency, this parameter is used only on the ARM platform and is ignored on the SGX platform. +- size: input parameter, which indicates the size of the shared memory. + +**Return Values:** + +- NULL: Failed to apply for the shared memory. +- Other values: start address of the created shared memory. + +## cc_free_shared_memory + +Releases the shared memory. + +**Functions** + +This API is called by the REE to release the shared memory after the switchless feature is enabled. + +**Function Declaration:** + +```c +cc_enclave_result_t cc_free_shared_memory(cc_enclave_t *enclave, void *ptr); +``` + +**Parameters:** + +- enclave: input parameter, which indicates the context handle of the secure environment. Different platforms have different shared memory models. To ensure cross-platform interface consistency, this parameter is used only on the ARM platform (the value of this parameter must be the same as the value of enclave passed when cc_malloc_shared_memory is invoked). It is ignored on the SGX platform. +- ptr: input parameter, which indicates the shared memory address returned by cc_malloc_shared_memory. + +**Return Values:** + +- CC_ERROR_BAD_PARAMETERS: invalid input parameter. +- CC_ERROR_INVALID_HANDLE: The enclave is invalid or the input enclave does not match the enclave corresponding to the ptr. (It takes effect only on the ARM platform. The SGX platform ignores the enclave and therefore does not check the enclave.) +- CC_ERROR_NOT_IMPLEMENTED: The API is not implemented. +- CC_ERROR_SHARED_MEMORY_START_ADDR_INVALID: ptr is not the shared memory address returned by cc_malloc_shared_memory (valid only on the ARM platform). +- CC_ERROR_OUT_OF_MEMORY: insufficient memory (valid only on the ARM platform). +- CC_FAIL: common failure. +- CC_SUCCESS: success + +## cc_enclave_generate_random + +Generates random numbers. + +**Function:** + +Generate a secure random number for the password on the TEE. + +**Function Declaration:** + +```c +cc_enclave_result_t cc_enclave_generate_random(void *buffer, size_t size) +``` + +**Parameters:** + +- buffer: input parameter, which specifies the buffer for generating random numbers. +- size: input parameter, which specifies the buffer length. + +**Return Values:** + +- CE_OK: Authentication information is verified successfully. +- CE_ERROR_INVALID_PARAMETER: incorrect input parameter. +- CE_ERROR_OUT_OF_MEMORY: no memory is available. + +## cc_enclave_seal_data + +Ensures data persistence. + +**Function:** + +This API is called by the TEE to encrypt the internal data of the enclave so that the data can be persistently stored outside the enclave. + +**Function Declaration:** + +```c +cc_enclave_result_t cc_enclave_seal_data(uint8_t *seal_data, uint32_t seal_data_len, + + cc_enclave_sealed_data_t *sealed_data, uint32_t sealed_data_len, + + uint8_t *additional_text, uint32_t additional_text_len) +``` + +**Parameters:** + +- seal_data: input parameter, which specifies the data to be encrypted. +- seal_data_len: input parameter, which specifies the length of the data to be encrypted. +- sealed_data: output parameter, which specifies the encrypted data processing handle. +- sealed_data_len: output parameter, which specifies the length of the encrypted ciphertext. +- additional_text: input parameter, which specifies the additional message required for encryption. +- additional_text_len: input parameter, which specifies the additional message length. + +**Return Values:** + +- CE_SUCCESS: Data encryption succeeds. +- CE_ERROR_INVALID_PARAMETER: incorrect input parameter. +- CE_ERROR_OUT_OF_MEMORY: no memory is available. +- CC_ERROR_SHORT_BUFFER: The input buffer is too small. +- CC_ERROR_GENERIC: Common bottom-layer hardware error. + +## cc_enclave_unseal_data + +Decrypts data. + +**Function:** + +This API is called by the TEE to decrypt the data sealed by the enclave and import the external persistent data back to the enclave. + +**Function Declaration:** + +```c +cc_enclave_result_t cc_enclave_unseal_data(cc_enclave_sealed_data_t *sealed_data, + + uint8_t *decrypted_data, uint32_t *decrypted_data_len, + + uint8_t *additional_text, uint32_t *additional_text_len) +``` + +**Parameters:** + +- sealed_data: input parameter, which specifies the handle of the encrypted data. +- decrypted_data: output parameter, which specifies the buffer of the decrypted ciphertext data. +- decrypted_data_len: output parameter, which specifies the length of the decrypted ciphertext. +- additional_text: output parameter, which specifies an additional message after decryption. +- additional_text_len: output parameter, which specifies the length of the additional message after decryption. + +**Return Values:** + +- CE_SUCCESS: Data decryption is successful. +- CE_ERROR_INVALID_PARAMETER: incorrect input parameter. +- CE_ERROR_OUT_OF_MEMORY: no memory is available. +- CC_ERROR_SHORT_BUFFER: The input buffer is too small. +- CC_ERROR_GENERIC: common bottom-layer hardware error. + +## cc_enclave_get_sealed_data_size + +Obtains the size of the encrypted data. + +**Function:** + +Obtain the size of the sealed_data data. This API can be called by the TEE and REE to allocate the decrypted data space. + +**Function Declaration:** + +```c +uint32_t cc_enclave_get_sealed_data_size(const uint32_t add_len, const uint32_t seal_data_len); +``` + +**Parameters:** + +- add_len: input parameter, which specifies the additional message length. +- sealed_data_len: input parameter, which specifies the length of the encrypted information. + +**Return Values:** + +- UINT32_MAX: Parameter error or function execution error. +- others: The function is successfully executed, and the return value is the size of the sealed_data structure. + +## cc_enclave_get_encrypted_text_size + +Obtains the length of an encrypted message. + +**Function:** + +This API is called by the TEE to obtain the length of the encrypted message in the encrypted data. + +**Function Declaration:** + +```c +uint32_t cc_enclave_get_encrypted_text_size(const cc_enclave_sealed_data_t *sealed_data); +``` + +**Parameter:** + +- sealed_data: input parameter, which specifies the handle of the encrypted data + +**Return Values:** + +- UINT32_MAX: Parameter error or function execution error. +- others: The function is executed successfully, and the return value is the length of the encrypted message in sealed_data. + +## cc_enclave_get_add_text_size + +Obtains the length of an additional message. + +**Function:** + +This API is called by the TEE to obtain the length of the additional message in the encrypted data. + +**Function Declaration:** + +```c +uint32_t cc_enclave_get_add_text_size(const cc_enclave_sealed_data_t *sealed_data); +``` + +**Parameter:** + +- sealed_data: input parameter, handle of the encrypted data. + +**Return Values:** + +- UINT32_MAX: Parameter error or function execution error. +- others: The function is successfully executed, and the return value is the length of the additional message in sealed_data. + +## cc_enclave_memory_in_enclave + +Performs security memory check. + +**Function:** + +This API is called by the TEE to check whether the memory addresses of the specified length belong to the TEE. + +**Function Declaration:** + +```c +bool cc_enclave_memory_in_enclave(const void *addr, size_t size) +``` + +**Parameters:** + +- *addr: input parameter, which specifies the memory address to be verified. +- size: input parameter, which specifies the length to be verified starting from the memory address. + +**Return Values:** + +- true: The memory in the specified zone is in the secure zone. +- false: Some or all memory in the specified area is not within the secure range. + +## cc_enclave_memory_out_enclave + +Performs security memory check. + +**Function:** + +This API is called by the TEE to check whether the memory addresses of the specified length belong to the REE. + +**Function Declaration:** + +```c +bool cc_enclave_memory_out_enclave(const void *addr, size_t size) +``` + +**Parameters:** + +- *addr: input parameter, which specifies the memory address to be verified. +- size: input parameter, length to be verified starting from the memory address. + +**Return Values:** + +- true: The memory of the specified area is in the non-secure area. +- false: Some or all of the memory in the specified zone is in the secure area. + +## PrintInfo + +Prints messages. + +**Function:** + +Print TEE logs. This API outputs the information that the TEE user wants to print. The input logs are stored in the REE /var/log/secgear/secgear.log. + +**Function Declaration:** + +```c +void PrintInfo(int level, const char *fmt, ...); +``` + +**Parameters:** + +- level: log print level, which is an input parameter. The value can be PRINT_ERROR, PRINT_WARNING, PRINT_STRACE, and PRINT_DEBUG. +- fmt: Input parameter, and a character to be output. + +**Return Value:** + +- None diff --git a/docs/en/server/security/secgear/application-scenarios.md b/docs/en/server/security/secgear/application-scenarios.md new file mode 100644 index 0000000000000000000000000000000000000000..13a9b7588f9b77c4a2c36b9a366ad54016bbf2dc --- /dev/null +++ b/docs/en/server/security/secgear/application-scenarios.md @@ -0,0 +1,96 @@ +# Application Scenarios + +This chapter describes confidential computing solutions in typical scenarios with examples, helping you understand the application scenarios of secGear and build confidential computing solutions based on your services. + +## TEE-based BJCA Cryptographic Module + +Driven by policies and services, the cryptographic application assurance infrastructure has been evolving towards virtualization. As services are migrated to the cloud, a brand-new cryptographic delivery mode needs to be built to integrate cryptographic services, cloud services, and service applications. Under such circumstance, Beijing Certificate Authority (BJCA) launches a TEE-based cryptographic module. BJCA can not only use the Kunpeng-based TEEs to build compliant cryptographic computing modules to support cryptographic cloud service platforms, but also build a confidential computing platform based on Kunpeng hosts to provide high-speed ubiquitous, elastically deployed, and flexibly scheduled cryptographic services for various scenarios such as cloud computing, privacy computing, and edge computing. The endogenous cryptographic module based on Kunpeng processors has become a revolutionary innovative solution in the cryptographic industry, and becomes a new starting point for endogenous trusted cryptographic computing. + +### Status Quo + +In conventional cryptographic modules, algorithm protocols and processed data are privacy data. Migrating cryptographic modules to the cloud has security risks. + +### Solution + +![](./figures/BJCA_Crypto_Module.png) + +The figure shows a TEE-based cryptographic module solution. secGear can divide the cryptographic module into two parts: management service and algorithm protocol. + +- Management service: runs on the REE to provide cryptographic services for the external world and forward requests to the TEE for processing. +- Algorithm protocol: runs on the TEE to encrypt and decrypt user data. + +Cryptographic services may have highly concurrent requests with large data volumes. The switchless feature of secGear reduces the context switches and data copies typically required for processing a large number of requests between the REE and TEE. + +## TEE-based Fully-Encrypted GaussDB + +Cloud databases have become an important growth point for database services in the future. Most traditional database service vendors are accelerating the provision of better cloud database services. However, cloud databases face more complex and diversified risks than traditional databases. Application vulnerabilities, system configuration errors, and malicious administrators may pose great risks to data security and privacy. + +### Status Quo + +The deployment network of cloud databases changes from a private environment to an open environment. The system O&M role is divided into service administrators and O&M administrators. Service administrators have service management permissions and belong to the enterprise service provider. O&M administrators belong to the cloud service provider. Although being defined to be responsible only for system O&M management, the database O&M administrator still has full permissions to use data. The database O&M administrator can access or even tamper with data with O&M management permissions or privilege escalation. In addition, due to the open environment and blurring of network boundaries, user data is more fully exposed to attackers in the entire service process, no matter in transfer, storage, O&M, or running. Therefore, in cloud database scenarios, how to solve the third-party trust problem and how to protect data security more reliably are facing greater challenges than traditional databases. Data security and privacy leakage are top concerns of cloud databases. + +### Solution + +To address the preceding challenges, the TEE-based fully-encrypted GaussDB (openGauss) is designed as follows: Users hold data encryption and decryption keys, data is stored in ciphertext in the entire life cycle of the database service, and query operations are completed in the TEE of the database service. + +![](./figures/secret_gaussdb.png) + +The figure shows the TEE-based fully-encrypted database solution. The fully-encrypted database has the following features: + +1. Data files are stored in ciphertext and plaintext key information is not stored. +2. The database data key is stored on the client. +3. When the client initiates a query request, the REE executes the encrypted SQL syntax on the server to obtain related ciphertext records and sends them to the TEE. +4. The client encrypts and transfers the database data key to the server TEE through the secure channel of secGear. The database data key is decrypted in the TEE and used to decrypt the ciphertext records into plaintext records. The SQL statement is executed to obtain the query result. Then the query result is encrypted using the database data key and sent back to the client. + +In step 3, when a large number of concurrent database requests are sent, frequent calls between the REE and TEE will be triggered and a large amount of data needs to be transferred. As a result, the performance deteriorates sharply. The switchless feature of secGear helps reduce context switches in calls and data copies, improving the performance. + +## TEE-based openLooKeng Federated SQL + +openLooKeng federated SQL is a type of cross-DC query. The typical scenario is as follows. There are three DCs: central DC A, edge DC B, and edge DC C. The openLooKeng cluster is deployed in the three DCs. When receiving a cross-domain query request, DC A delivers an execution plan to each DC. After the openLookeng clusters in edge DCs B and C complete computing, the result is transferred to the openLookeng cluster in DC A over the network to complete aggregation computing. + +### Status Quo + +In the preceding solution, the computing result is transferred between openLookeng clusters in different DCs, avoiding insufficient network bandwidth and solving the cross-domain query problem to some extent. However, the computing result is obtained from the original data and may contain sensitive information. As a result, security and compliance risks exist when data is transferred out of the domain. How do we protect the computing results of the edge DCs during aggregation computing and ensure that the computing results are available but invisible in the central DC? + +### Solution + +In DC A, the openLookeng cluster splits the aggregation computing logic and operators into independent modules and deploys them in the Kunpeng-based TEE. The computing results of the edge DCs are transferred to the TEE of DC A through the secure channel. All data is finally aggregated and computed in the TEE. In this way, the computing results of the edge DCs are protected from being obtained or tampered with by privileged or malicious programs in the REE of DC A during aggregation computing. + +![](./figures/openLooKeng.png) + +The figure shows the TEE-based federated SQL solution. The query process is as follows: + +1. A user delivers a cross-domain query request in DC A. The coordinator of openLooKeng splits and delivers the execution plan to its worker nodes and the coordinators of edge DCs based on the query SQL statement and data distribution. Then the coordinators of edge DCs deliver the execution plan to their worker nodes. +2. Each worker node executes the plan to obtain the local computing result. +3. Edge DCs encrypt their computing results through the secure channel of secGear, transfer the results to the REE of DC A over the Internet, forward the results to the TEE, and decrypt the results in the TEE. +4. DC A performs aggregation computing on the computing results of DCs A, B, and C in the TEE, obtains a final execution result, and returns the result to the user. + +In step 4, when there are a large number of query requests, the REE and TEE will be frequently invoked and a large amount of data is copied. As a result, the performance deteriorates. The switchless feature of secGear is optimized to reduce context switches and data copies to improve the performance. + +## TEE-based MindSpore Feature Protection + +Vertical federated learning (VFL) is an important branch of federated learning. When multiple parties have features about the same set of users, VFL can be used for collaborative training. + +![](./figures/Mindspore_original.png) + +### Status Quo + +The figure shows the data processing flow of the traditional solution. + +1. A party that has features is also called a follower, while a party that has labels is also called a leader. Each follower inputs its features to its bottom model to obtain the intermediate result, and then sends the intermediate result to the leader. +2. The leader uses its labels and the intermediate results of followers to train the top model, and then sends the computed gradient back to the followers to train their bottom models. + +This solution prevents followers from directly uploading their raw data out of the domain, thereby protecting data privacy. However, attackers may derive user information from the uploaded intermediate results, causing privacy leakage risks. Therefore, a stronger privacy protection solution is required for intermediate results and gradients to meet security compliance requirements. + +### Solution + +Based on the security risks and solutions in the previous three scenarios, confidential computing is a good choice to make intermediate results "available but invisible" out of the domain. + +![](./figures/Mindspore.png) + +The figure shows the TEE-based VFL feature protection solution. The data processing process is as follows: + +1. Followers encrypt their intermediate results through the secure channel of secGear and transfer the results to the leader. After receiving the results, the leader transfers them to the TEE and decrypts them through the secure channel in the TEE. +2. In the TEE, the intermediate results are input to the computing module at the federated split layer to compute the result. + +In this process, the plaintext intermediate results of followers exist only in the TEE memory, which is inaccessible to the leader, like a black box. diff --git a/docs/en/server/security/secgear/developer-guide.md b/docs/en/server/security/secgear/developer-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..2629b1b880e930d3cfb4fa58437563775ffc03db --- /dev/null +++ b/docs/en/server/security/secgear/developer-guide.md @@ -0,0 +1,93 @@ +# Developer Guide + +This chapter provides an example of using secGear to develop a C language program helloworld, helping you understand how to use secGear to develop applications. + +## Downloading Examples + +```shell +git clone https://gitee.com/openeuler/secGear.git +``` + +## Directory Structure + +```shell +cd examples/helloworld + +#Directory structure: +├── helloworld +│ ├── CMakeLists.txt +│ ├── enclave +│ │ ├── CMakeLists.txt +│ │ ├── Enclave.config.xml +│ │ ├── Enclave.lds +│ │ ├── hello.c +│ │ ├── manifest.txt +│ │ └── config_cloud.ini +│ ├── helloworld.edl +│ └── host +│ ├── CMakeLists.txt +│ └── main.c +``` + +The code body consists of three parts: + +- **main.c**: REE program +- **helloworld.edl**: header file of the APIs called by the REE and TEE +- **hello.c**: TEE program + +## Preparations + +In addition to the preceding three parts, there are compilation project file (**CMakeLists.txt**) and developer licenses (**Enclave.config.xml**/**Enclave.lds** of Intel SGX and **manifest.txt**/**config_cloud.ini** of Kunpeng). + +> ![](./public_sys-resources/icon-note.gif)NOTE: +> +> - The Kunpeng developer license needs to be [applied for from the Huawei service owner](https://www.hikunpeng.com/document/detail/en/kunpengcctrustzone/fg-tz/kunpengtrustzone_04_0009.html). +> - Because Intel SGX is debugged in debug mode, you do not need to apply for a developer license currently. If the remote attestation service of Intel is required for commercial use, you need to [apply for a license from Intel](https://www.intel.com/content/www/us/en/developer/tools/software-guard-extensions/request-license.html). + +After the application is successful, the developer license file is obtained and needs to be stored in the corresponding code directory. + +## Development Procedure + +Reconstructing a confidential computing application based on secGear is similar to independently extracting functional modules. The procedure is as follows: Identify sensitive data processing logic, extract it into an independent library, deploy it in the TEE, and define APIs provided by the REE in the EDL file. + +The following figure shows the development procedure. + +1. Develop the main function and APIs in the REE, manage the enclave, and call functions in the TEE. +2. Develop the EDL file (similar to the C language header file that defines the interaction APIs between the REE and TEE). +3. Develop TEE APIs. +4. Call the code generation tool codegener to automatically generate the interaction source code between the REE and TEE based on the EDL file and compile the source code to the binary files of the REE and TEE. The REE logic directly calls the corresponding API of the TEE without considering the automatically generated interaction code, reducing the development cost. +5. Call the signing tool to sign binary files in the TEE to implement trusted boot of the TEE program. + +![](./figures/develop_step.png) + +## Build and Run + +### Arm Environment + +```shell +// clone secGear repository +git clone https://gitee.com/openeuler/secGear.git + +// build secGear and examples +cd secGear +source environment +mkdir debug && cd debug && cmake -DENCLAVE=GP .. && make && sudo make install + +// run helloworld +/vendor/bin/secgear_helloworld +``` + +### x86 Environment + +```shell +// clone secGear repository +git clone https://gitee.com/openeuler/secGear.git + +// build secGear and examples +cd secGear +source /opt/intel/sgxsdk/environment && source environment +mkdir debug && cd debug && cmake .. && make && sudo make install + +// run helloworld +./examples/helloworld/host/secgear_helloworld +``` diff --git a/docs/en/server/security/secgear/figures/BJCA_Crypto_Module.png b/docs/en/server/security/secgear/figures/BJCA_Crypto_Module.png new file mode 100644 index 0000000000000000000000000000000000000000..3144e38d02872d2618ac6b8e4473504613b57261 Binary files /dev/null and b/docs/en/server/security/secgear/figures/BJCA_Crypto_Module.png differ diff --git a/docs/en/server/security/secgear/figures/Mindspore.png b/docs/en/server/security/secgear/figures/Mindspore.png new file mode 100644 index 0000000000000000000000000000000000000000..5a1fd1788d4ada4166c44444e14672cf8c14bd15 Binary files /dev/null and b/docs/en/server/security/secgear/figures/Mindspore.png differ diff --git a/docs/en/server/security/secgear/figures/Mindspore_original.png b/docs/en/server/security/secgear/figures/Mindspore_original.png new file mode 100644 index 0000000000000000000000000000000000000000..280c050df982d8806b5ed4293a3c0aec90042906 Binary files /dev/null and b/docs/en/server/security/secgear/figures/Mindspore_original.png differ diff --git a/docs/en/server/security/secgear/figures/develop_step.png b/docs/en/server/security/secgear/figures/develop_step.png new file mode 100644 index 0000000000000000000000000000000000000000..a7b6e1842c61382cf9bad136eebab04eb6750c0e Binary files /dev/null and b/docs/en/server/security/secgear/figures/develop_step.png differ diff --git a/docs/en/server/security/secgear/figures/openLooKeng.png b/docs/en/server/security/secgear/figures/openLooKeng.png new file mode 100644 index 0000000000000000000000000000000000000000..9f9e249830140fd65e26ebbc637d60adf8a20901 Binary files /dev/null and b/docs/en/server/security/secgear/figures/openLooKeng.png differ diff --git a/docs/en/server/security/secgear/figures/secGear_arch.png b/docs/en/server/security/secgear/figures/secGear_arch.png new file mode 100644 index 0000000000000000000000000000000000000000..06620dfd30614ded62190354c1ec0b0b2c497c01 Binary files /dev/null and b/docs/en/server/security/secgear/figures/secGear_arch.png differ diff --git a/docs/en/server/security/secgear/figures/secret_gaussdb.png b/docs/en/server/security/secgear/figures/secret_gaussdb.png new file mode 100644 index 0000000000000000000000000000000000000000..3556c3727f6f723f620e5c451e5fec563dfde1aa Binary files /dev/null and b/docs/en/server/security/secgear/figures/secret_gaussdb.png differ diff --git a/docs/en/server/security/secgear/introduction-to-secgear.md b/docs/en/server/security/secgear/introduction-to-secgear.md new file mode 100644 index 0000000000000000000000000000000000000000..cca03b5f35904d53d833541ef531d59493aa5ec0 --- /dev/null +++ b/docs/en/server/security/secgear/introduction-to-secgear.md @@ -0,0 +1,165 @@ +# Introduction to secGear + +## Overview + +With the rapid development of cloud computing, more and more enterprises deploy computing services on the cloud. The security of user data on the third-party cloud infrastructure is facing great challenges. Confidential computing is a technology that uses hardware-based trusted execution environments (TEEs) to protect confidentiality and integrity of data in use. It relies on the bottom-layer hardware to build the minimum trust dependency, which removes the OS, hypervisor, infrastructure, system administrator, and service provider from the trusted entity list as unauthorized entities to reduce potential risks. There are various confidential computing technologies (such as Intel SGX, Arm TrustZone, and RISC-V Keystone) and software development kits (SDKs) in the industry and the application ecosystem of different TEEs are isolated from each other, which brings high development and maintenance costs to confidential computing application developers. To help developers quickly build confidential computing solutions that protect data security on the cloud, openEuler launches the unified confidential computing programming framework secGear. + +## Architecture + +![](./figures/secGear_arch.png) + +The architecture of secGear consists of three layers that form the foundation of openEuler confidential computing software. + +- Base layer: The unified layer of the confidential computing SDK provides APIs for different TEEs, enabling different architectures to share the same set of source code. +- Middleware layer: The general component layer provides confidential computing software for users to quickly build confidential computing solutions. +- Server layer: The confidential computing service layer runs dedicated solutions for typical situations. + +## Key Features + +### Switchless + +#### Pain Points + +After a conventional application is reconstructed using confidential computing, the rich execution environment (REE) logic frequently invokes the TEE logic or the REE frequently exchanges large data blocks with the TEE. Each call between the REE and TEE requires context switching among the REE user mode, REE kernel mode, driver, TEE kernel mode, and TEE user mode. When large blocks of data are exchanged during the call, multiple memory copies are generated. In addition, the interaction performance between the REE and TEE deteriorates due to factors such as the size limit of underlying data blocks, which severely affects the implementation of confidential computing applications. + +#### Solution + +Switchless is a technology that uses shared memory to reduce the number of context switches and data copies between the REE and TEE to optimize the interaction performance. + +#### How to Use + +1. Enable switchless when creating an enclave. + + The configuration items of switchless are described as follows: + + ```c + typedef struct { + uint32_t num_uworkers; + uint32_t num_tworkers; + uint32_t switchless_calls_pool_size; + uint32_t retries_before_fallback; + uint32_t retries_before_sleep; + uint32_t parameter_num; + uint32_t workers_policy; + uint32_t rollback_to_common; + cpu_set_t num_cores; + } cc_sl_config_t; + ``` + + | Configuration Item | Description | + | -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | num_uworkers | Number of proxy worker threads in the REE, which are used to make switchless out calls (OCALLs). Currently, this field takes effect only on the SGX platform and can be configured on the Arm platform. However, because the Arm platform does not support OCALLs, the configuration does not take effect on the Arm platform.
Specifications:
Arm: maximum value: **512**; minimum value: **1**; default value: **8** (used when this field is set to **0**).
SGX: maximum value: **4294967295**; minimum value: **1**. | + | num_tworkers | Number of proxy worker threads in the TEE, which are used to make switchless enclave calls (ECALLs).
Specifications:
Arm: maximum value: **512**; minimum value: **1**; default value: **8** (used when this field is set to **0**).
SGX: maximum value: **4294967295**; minimum value: **1**. | + | switchless_calls_pool_size | Size of the switchless call pool. The pool can contain **switchless_calls_pool_size** x 64 switchless calls. For example, if **switchless_calls_pool_size=1**, 64 switchless calls are contained in the pool.
Specifications:
Arm: maximum value: **8**; minimum value: **1**; default value: **1** (used when this field is set to **0**).
SGX: maximum value: **8**; minimum value: **1**; default value: **1** (used when **switchless_calls_pool_size** is set to **0**). | + | retries_before_fallback | After the **pause** assembly instruction is executed for **retries_before_fallback** times, if the switchless call is not made by the proxy worker thread on the other side, the system rolls back to the switch call mode. This field takes effect only on the SGX platform.
Specifications:
SGX: maximum value: **4294967295**; minimum value: **1**; default value: **20000** (used when this field is set to **0**). | + | retries_before_sleep | After the **pause** assembly instruction is executed for **retries_before_sleep** times, if the proxy worker thread does not receive any task, the proxy worker thread enters the sleep state. This field takes effect only on the SGX platform.
Specifications:
SGX: maximum value: **4294967295**; minimum value: **1**; default value: **20000** (used when this field is set to **0**). | + | parameter_num | Maximum number of parameters supported by a switchless function. This field takes effect only on the Arm platform.
Specifications:
Arm: maximum value: **16**; minimum value: **0**. | + | workers_policy | Running mode of the switchless proxy thread. This field takes effect only on the Arm platform.
Specifications:
Arm:
**WORKERS_POLICY_BUSY**: The proxy thread always occupies CPU resources regardless of whether there are tasks to be processed. This mode applies to scenarios that require high performance and extensive system software and hardware resources.
**WORKERS_POLICY_WAKEUP**: The proxy thread wakes up only when there is a task. After the task is processed, the proxy thread enters the sleep state and waits to be woken up by a new task. | + | rollback_to_common | Whether to roll back to a common call when an asynchronous switchless call fails. This field takes effect only on the Arm platform.
Specifications:
Arm:
**0**: No. If the operation fails, only the error code is returned.
Other values: Yes. If the operation fails, an asynchronous switchless call is rolled back to a common call and the return value of the common call is returned. | + | num_cores | Number of cores for TEE core binding
Specifications: The maximum value is the number of cores in the environment. | + +1. Add the **transition_using_threads** flag when defining the API in the enclave description language (EDL) file. + + ```ocaml + enclave { + include "secgear_urts.h" + from "secgear_tstdc.edl" import *; + from "secgear_tswitchless.edl" import *; + trusted { + public int get_string([out, size=32]char *buf); + public int get_string_switchless([out, size=32]char *buf) transition_using_threads; + }; + }; + ``` + +### Secure Channel + +#### Pain Points + +When requesting the confidential computing service on the cloud, the data owner needs to upload the data to be processed to the TEE on the cloud for processing. Because the TEE is not connected to the network, the data needs to be transferred to the REE over the network in plaintext and then transferred to the TEE from the REE. The data plaintext is exposed in the REE memory, which poses security risks. + +#### Solution + +A secure channel is a technology that combines confidential computing remote attestation to implement secure key negotiation between the data owner and the TEE on the cloud. It negotiates a sessionkey owned only by the data owner and the TEE on the cloud. Then the sessionkey is used to encrypt user data transferred over the network. After receiving the ciphertext data, the REE transfers the data to the TEE for decryption and processing. + +#### How to Use + +The secure channel is provided as a library and consists of the client, host, and enclave, which are icalled by the client, server client application (CA), and server trusted application (TA) of the service program respectively. + +| Module | Header File | Library File | Dependency | +|------------|--------------------------|-----------------------|---------| +| Client | secure_channel_client.h | libcsecure_channel.so | OpenSSL| +| Host | secure_channel_host.h | libusecure_channel.so | OpenSSL| +| Enclave| secure_channel_enclave.h | libtsecure_channel.so | TEE and TEE software stack | + +##### APIs + +| API | Header File and Library | Function | Remarks| +|----------------------------------------------------------------------------------------------------------------------------------------------|-----------------------|--------------|----| +| cc_sec_chl_client_init | secure_channel_client.h libcsecure_channel.so | Initializes the secure channel on the client. | Before calling this API, initialize the network connection and message sending hook function in the **ctx** parameter. | +| cc_sec_chl_client_fini | secure_channel_client.h libcsecure_channel.so | Destroys the secure channel on the client. | Instructs the server to destroy the local client information and local secure channel information. | +| cc_sec_chl_client_callback | secure_channel_client.h libcsecure_channel.so | Function for processing secure channel negotiation messages.| Processes messages sent from the server to the client during secure channel negotiation. This API is called when messages are received on the client. | +| cc_sec_chl_client_encrypt | secure_channel_client.h libcsecure_channel.so | Encryption API of the secure channel on the client. | None | +| cc_sec_chl_client_decrypt | secure_channel_client.h libcsecure_channel.so | Decryption API of the secure channel on the client. | None | +| int (*cc_conn_opt_funcptr_t)(void*conn, void *buf, size_t count); | secure_channel.h | Prototype of the message sending hook function. | Implemented by the client and server to specify the secure channel negotiation message type. It sends secure channel negotiation messages to the peer end. | +| cc_sec_chl_svr_init | secure_channel_host.h libusecure_channel.so | Initializes the secure channel on the server. | Before calling this API, initialize **enclave_ctx** in **ctx**. | +| cc_sec_chl_svr_fini | secure_channel_host.h libusecure_channel.so | Destroys the secure channel on the server. | Destroys information about the secure channel on the server and all clients. | +| cc_sec_chl_svr_callback | secure_channel_host.h libusecure_channel.so | Function for processing secure channel negotiation messages. | Processes messages sent from the client to the server during security channel negotiation. This API is called when messages are received on the server. Before calling this API, you need to initialize the network connection to the client and the message sending hook function. For details, see [examples](https://gitee.com/openeuler/secGear/blob/master/examples/secure_channel/host/server.c#:~:text=conn_ctx.conn_kit.send). | +| cc_sec_chl_enclave_encrypt | secure_channel_enclave.h libtsecure_channel.so | Encryption API of the secure channel on the enclave. | None | +| cc_sec_chl_enclave_decrypt | secure_channel_enclave.h libtsecure_channel.so | Decryption API of the secure channel on the enclave. | None| + +##### Precautions + +A secure channel encapsulates only the key negotiation process and encryption and decryption APIs, but does not establish any network connection. The negotiation process reuses the network connection of the service. The network connection between the client and server is established and maintained by the service. The message sending hook function and network connection pointer are transferred during the initialization of the secure channel on the client and the server. +For details, see [secure channel examples](https://gitee.com/openeuler/secGear/tree/master/examples/secure_channel). + +### Remote Attestation + +#### Challenges + +As confidential computing technologies advance, several major platforms have emerged, including Arm Trustzone/CCA, Intel SGX/TDX, QingTian Enclave, and Hygon CSV. Solutions often involve multiple confidential computing hardware platforms, sometimes requiring collaboration between different TEEs. Remote attestation is a crucial part of the trust chain in any confidential computing technology. However, each technology has its own attestation report format and verification process. This forces users to integrate separate verification workflows for each TEE, increasing complexity and hindering the adoption of new TEE types. + +#### Solution + +The unified remote attestation framework of secGear addresses the key components related to remote attestation in confidential computing, abstracting away the differences between different TEEs. It provides two components: attestation agent and attestation service. The agent is integrated by users to obtain attestation reports and connect to the attestation service. The service can be deployed independently and supports the verification of iTrustee and virtCCA remote attestation reports. + +#### Feature Description + +The unified remote attestation framework focuses on confidential computing functionalities, while service deployment and operation capabilities are provided by third-party deployment services. The key features of the unified remote attestation framework are as follows: + +- Report verification plugin framework: Supports runtime compatibility with attestation report verification for different TEE platforms, such as iTrustee, virtCCA, and CCA. It also supports the extension of new TEE report verification plugins. +- Certificate baseline management: Supports the management of baseline values of Trusted Computing Bases (TCB) and Trusted Applications (TA) as well as public key certificates for different TEE types. Centralized deployment on the server ensures transparency for users. +- Policy management: Provides default policies for ease of use and customizable policies for flexibility. +- Identity token: Issues identity tokens for different TEEs, endorsed by a third party for mutual authentication between different TEE types. +- Attestation agent: Supports connection to attestation service/peer-to-peer attestation, compatible with TEE report retrieval and identity token verification. It is easy to integrate, allowing users to focus on their service logic. + +Two modes are supported depending on the usage scenario: peer-to-peer verification and attestation service verification. + +Attestation service verification process: + +1. The user (regular node or TEE) initiates a challenge to the TEE platform. +2. The TEE platform obtains the TEE attestation report through the attestation agent and returns it to the user. +3. The user-side attestation agent forwards the report to the remote attestation service. +4. The remote attestation service verifies the report and returns an identity token in a unified format endorsed by a third party. +5. The attestation agent verifies the identity token and parses the attestation report verification result. +6. Upon successful verification, a secure connection is established. + +Peer-to-peer verification process (without the attestation service): + +1. The user initiates a challenge to the TEE platform, which then returns the attestation report to the user. +2. The user uses a local peer-to-peer TEE verification plugin to verify the report. + +> ![](./public_sys-resources/icon-note.gif) **Note:** +> The attestation agent varies depending on whether peer-to-peer verification or remote attestation service verification is used. Users can select the desired mode during compilation by specifying the appropriate option, enabling the attestation agent to support either the attestation service or peer-to-peer mode. + +#### Application Scenarios + +In scenarios like finance and AI, where confidential computing is used to protect the security of privacy data during runtime, remote attestation is a technical means to verify the legitimacy of the confidential computing environment and applications. secGear provides components that are easy to integrate and deploy, helping users quickly enable confidential computing remote attestation capabilities. + +## Acronyms and Abbreviations + +| Acronym/Abbreviation | Full Name | +| -------------------- | ----------------------------- | +| REE | rich execution environment | +| TEE | trusted execution environment | +| EDL | enclave description language | diff --git a/docs/en/server/security/secgear/public_sys-resources/icon-note.gif b/docs/en/server/security/secgear/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/server/security/secgear/public_sys-resources/icon-note.gif differ diff --git a/docs/en/server/security/secgear/secgear-installation.md b/docs/en/server/security/secgear/secgear-installation.md new file mode 100644 index 0000000000000000000000000000000000000000..b6435b7f96843ac1b98d8721eef12db1a3444ec7 --- /dev/null +++ b/docs/en/server/security/secgear/secgear-installation.md @@ -0,0 +1,109 @@ +# secGear Installation + +## Arm Environment + +### Environment Requirements + +#### Hardware + +| Item | Version | +| ------ | --------------------------------------------------- | +| Server| TaiShan 200 server (model 2280) | +| Mainboard | Kunpeng board | +| BMC | 1711 board (model BC82SMMAB); firmware version: 3.01.12.49 or later| +| CPU | Kunpeng 920 processor (model 7260, 5250, or 5220) | +| Chassis | No special requirements; an 8- or 12-drive chassis recommended | + +> [!NOTE]NOTE +> Ensure that the TrustZone feature kit has been preconfigured on the server. That is, the TEE OS, TEE OS boot key, BMC, BIOS, and license have been preconfigured on the server. +> For common servers, the TrustZone feature cannot be enabled only by upgrading the BMC, BIOS, and TEE OS firmware. +> By default, the TrustZone feature is disabled on the server. For details about how to enable the TrustZone feature on the server, see BIOS settings. + +### Environment Preparation + +For details, see [Environment Requirements](https://www.hikunpeng.com/document/detail/en/kunpengcctrustzone/fg-tz/kunpengtrustzone_20_0018.html) and [Procedure](https://www.hikunpeng.com/document/detail/en/kunpengcctrustzone/fg-tz/kunpengtrustzone_20_0019.html) on the Kunpeng official website. + +### Installation + +1. Configure the openEuler Yum repository. You can configure an online Yum repository (see the example below) or configure a local Yum repository by mounting an ISO file. + + ```shell + vi /etc/yum.repo/openEuler.repo + [osrepo] + name=osrepo + baseurl=http://repo.openeuler.org/openEuler-22.03-LTS/everything/aarch64/ + enabled=1 + gpgcheck=1 + gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS/everything/aarch64/RPM-GPG-KEY-openEuler + ``` + +2. Install secGear. + + ```shell + #Install the compiler. + yum install cmake ocaml-dune + + #Install secGear. + yum install secGear-devel + + #Check whether the installations are successful. If the command output is as follows, the installations are successful. + rpm -qa | grep -E 'secGear|itrustee|ocaml-dune' + itrustee_sdk-xxx + itrustee_sdk-devel-xxx + secGear-xxx + secGear-devel-xxx + ocaml-dune-xxx + ``` + +## x86 Environment + +### Environment Requirements + +#### Hardware + +Processor that supports the Intel SGX feature + +### Environment Preparation + +Purchase a device that supports the Intel SGX feature and enable the SGX feature by referring to the BIOS setting manual of the device. + +### Installation + +1. Configure the openEuler Yum repository. You can configure an online Yum repository (see the example below) or configure a local Yum repository by mounting an ISO file. + + ```shell + vi openEuler.repo + [osrepo] + name=osrepo + baseurl=http://repo.openeuler.org/openEuler-22.03-LTS/everything/x86_64/ + enabled=1 + gpgcheck=1 + gpgkey=http://repo.openeuler.org/openEuler-22.03-LTS/everything/x86_64/RPM-GPG-KEY-openEuler + ``` + +2. Install secGear. + + ```shell + # Install the compiler. + yum install cmake ocaml-dune + + # Install secGear. + yum install secGear-devel + + # Check whether the installations are successful. If the command output is as follows, the installations are successful. + rpm -qa | grep -E 'secGear|ocaml-dune|sgx' + secGear-xxx + secGear-devel-xxx + ocaml-dune-xxx + libsgx-epid-xxx + libsgx-enclave-common-xxx + libsgx-quote-ex-xxx + libsgx-aesm-launch-plugin-xxx + libsgx-uae-service-xxx + libsgx-ae-le-xxx + libsgx-urts-xxx + sgxsdk-xxx + sgx-aesm-service-xxx + linux-sgx-driver-xxx + libsgx-launch-xxx + ``` diff --git a/docs/en/server/security/secgear/secgear.md b/docs/en/server/security/secgear/secgear.md new file mode 100644 index 0000000000000000000000000000000000000000..d2458fc7dc5fed47bd88d20337261da379020d2e --- /dev/null +++ b/docs/en/server/security/secgear/secgear.md @@ -0,0 +1,5 @@ +# secGear Developer Guide + +This document describes the architecture, features, installation, developer guide, and application scenarios of secGear, a unified confidential computing programming framework provided by openEuler. + +This document is intended for developers, open source enthusiasts, and partners who use the openEuler operating system (OS) and want to learn and use secGear. Users must have basic knowledge of the Linux OS. diff --git a/docs/en/server/security/secgear/using-secgear-tools.md b/docs/en/server/security/secgear/using-secgear-tools.md new file mode 100644 index 0000000000000000000000000000000000000000..9650186ef995de4b54023c55de61c4b0e1291861 --- /dev/null +++ b/docs/en/server/security/secgear/using-secgear-tools.md @@ -0,0 +1,149 @@ +# secGear Tools + +secGear provides a tool set to facilitate application development. This document describes the tools and how to use them. + +## Code Generation Tool: codegener + +### Overview + +secGear codegener is a tool developed based on Intel SGX SDK edger8r. It is used to parse the EDL file to generate intermediate C code, that is, to assist in generating code that is called between the TEE and REE. + +The EDL file format defined by secGear codegener is the same as that defined by Intel SGX SDK edger8r, but the complete syntax definition of Intel is not supported: + +- The public can be used only in methods. Functions without public are declared as private by default. +- Switchless calls from the REE to the TEE and from the TEE to the REE are not supported. +- The Outside Call (OCALL) does not support some calling modes (such as cdecl, stdcall, and fastcall). + +The EDL file syntax is similar to the C language syntax. The following describes parts different from the C language syntax: + +| Member | Description | +| ----------------------- | ------------------------------------------------------------ | +| include "my_type.h" | Uses the type defined in the external inclusion file. | +| trusted | Declares that secure functions are available on the trusted application (TA) side. | +| untrusted | Declares that insecure functions are available on the TA side. | +| return_type | Defines the return value type. | +| parameter_type | Defines the parameter type. | +| \[in, size = len] | For the ECALL, this parameter indicates that data needs to be transferred from the REE to the TEE. For the OCALL, this parameter is required for the pointer type, and size indicates the buffer that is actually used. | +| \[out, size = len] | For the ECALL, this parameter indicates that data needs to be transferred from the TEE to the REE. For the OCALL, this parameter needs to be used for the pointer type, and size indicates the buffer that is actually used.| + +### Usage Instructions + +#### Command Format + +The format of the codegen command is as follows: + +- x86_64 architecture: + +**codegen_x86_64** < --trustzone \| --sgx > \[--trusted-dir \ \| **--untrusted-dir** \\| --trusted \| --untrusted ] edlfile + +ARM architecture + +**codegen_arm64** < --trustzone \| --sgx > \[--trusted-dir \ \| **--untrusted-dir** \\| --trusted \| --untrusted ] edlfile + +#### Parameter Description + +The parameters are described as follows: + +| Parameter | Mandatory/Optional | Description | +| ----------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| --trustzone \| --sgx | Mandatory | Generates the API function corresponding to the confidential computing architecture only in the current command directory. If no parameter is specified, the SGX API function is generated by default. | +| --search-path \ | Optional | Specifies the search path of the file that the EDL file to be converted depends on. | +| --use-prefix | Optional | Adds a prefix to the proxy function name. The prefix is the name of the EDL file. | +| --header-only | Optional | Specifies that the code generation tool generates only header files. | +| --trusted-dir \ | Optional | Specifies the directory where the generated TEE auxiliary code is stored. If this parameter is not specified, the current path is used by default. | +| --untrusted-dir \ | Optional | Specifies the directory where the auxiliary code for generating insecure functions is located. | +| --trusted | Optional | Generates TEE auxiliary code. | +| --untrusted | Optional | Generates REE auxiliary code. | +| edlfile | Mandatory | EDL file to be converted, for example, hello.edl. | + +#### Examples + +- Convert *helloworld.edl* to generate TEE auxiliary code in *enclave-directory* and generate REE auxiliary code in *host-directory*. An example command is as follows: + +```shell +codegen_x86_64 --sgx --trusted-dir enclave-directory --untrusted-dir host-directory helloworld.edl +``` + +- Convert *helloworld.edl* to generate TEE auxiliary code in the current directory. The following is a command example for not generating REE auxiliary code: + +```shell +codegen_x86_64 --sgx --trusted helloworld.edl +``` + +- Convert *helloworld.edl* to generate REE auxiliary code in the current directory. The following is a command example that does not generate TEE auxiliary code: + +```shell +codegen_x86_64 --sgx --untrusted helloworld.edl +``` + +- Convert *helloworld.edl*. An example of the command for generating TEE and REE auxiliary code in the current directory is as follows: + +```shell +codegen_x86_64 --sgx helloworld.edl +``` + +## Signature Tool: sign_tool + +### Overview + +secGear sign_tool is a command line tool, including the compilation tool chain and signature tool, which are used for enclave signing. The sign_tool has two signature modes: + +- Single-step signature: applies only to the debugging mode. +- Two-step signature: applies to the commercial scenario. Obtain the signature private key from a third-party platform or an independent security device to sign the enclave. + +### Operation Instructions + +#### Format + +The sign_tool contains the sign command (for signing the enclave) and the digest command (for generating the digest value). Command format: + +**sign_tool.sh -d** \[sign \| digest] **-x** \ **-i** \ **-p** \ **-s** \ \[OPTIONS] **-o** \ + +#### Parameter Description + +| sign Command Parameter | Description | Mandatory/Optional | +| -------------- | -------------------------------------------------------------| -------------------------------------------- | +| -a \ | api_level, which identifies the GP API version of the iTrustee TA. The default value is 1. | Optional | +| -c \ | Configuration file | Optional | +| -d \ | Specifies the operation (sign or digest) to be performed by the signature tool. | Only the sign operation is performed in single-step mode. In two-step mode, the digest operation must be performed before the sign operation. | +| -e \ | Public key certificate of the device, which is used to protect the AES key for encrypting rawdata (mandatory for iTrustee). | This parameter is mandatory only for the iTrustee type. | +| -f \ | OTRP_FLAG, which determines whether to support the OTRP standard protocol. The default value is 0. | Optional | +| -i \ | Library file to be signed. | Mandatory | +| -k \ | Private key (PEM file) required for one-step signature. | This parameter is mandatory only for the SGX type. | +| -m \ | Security configuration file manifest.txt, which is configured by users. | Only the iTrustee type is mandatory. | +| -o \ | Output file. | Mandatory | +| -p \ | Public key certificate (PEM file) of the signature server required for two-step signing. | Mandatory | +| -s \ | Signed digest value required for two-step signing. | Mandatory | +| -t \ | TA_TYPA, which identifies TA binary format of the iTrustee. The default value is 1. | Optional | +| -x \ | enclave type (sgx or trustzone) | Mandatory | +| -h | Prints the help information. | Optional | + +#### Single-Step Signature + +Set the enclave type is SGX, sign the test.enclave, and generate the signature file signed.enclave. The following is an example: + +```shell +sign_tool.sh -d sign -x sgx -i test.enclave -k private_test.pem -o signed.enclave +``` + +#### Two-Step Signature + +The following uses SGX as an example to describe the two-step signature procedure: + +1. Generate digest value. + + Use the sign_tool to generate the digest value digest.data and the temporary intermediate file signdata. The file is used when the signature file is generated and is automatically deleted after being signed. Example: + + ```shell + sign_tool.sh -d digest -x sgx -i input -o digest.data + ``` + +2. Send digest.data to the signature authority or platform and obtain the corresponding signature. + +3. Use the obtained signature to generate the signed dynamic library signed.enclave. + + ```shell + sign_tool.sh -d sign -x sgx-i input -p pub.pem -s signature -o signed.enclave + ``` + +Note: To release an official version of applications supported by Intel SGX, you need to apply for an Intel whitelist. For details about the process, see the Intel document at . diff --git a/docs/en/server/security/secharden/_toc.yaml b/docs/en/server/security/secharden/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..8b25ed44f7cf4d63a9b23c09dc5d76ad29bc2acb --- /dev/null +++ b/docs/en/server/security/secharden/_toc.yaml @@ -0,0 +1,28 @@ +label: Security Hardening Guide +isManual: true +href: ./secharden.md +description: Security hardening techniques and tools +sections: + - label: OS Hardening Overview + href: ./os-hardening-overview.md + - label: Security Configuration Description + href: ./security-configuration-benchmark.md + - label: Security Hardening Guide + href: ./security-hardening-guide.md + sections: + - label: Account Passwords + href: ./account-passwords.md + - label: Authentication and Authorization + href: ./authentication-and-authorization.md + - label: System Services + href: ./system-services.md + - label: File Permissions + href: ./file-permissions.md + - label: Kernel Parameters + href: ./kernel-parameters.md + - label: SELinux Configuration + href: ./selinux-configuration.md + - label: Security Hardening Tools + href: ./security-hardening-tools.md + - label: Appendix + href: ./appendix.md diff --git a/docs/en/server/security/secharden/account-passwords.md b/docs/en/server/security/secharden/account-passwords.md new file mode 100644 index 0000000000000000000000000000000000000000..5286bbaeaf15d7d170e1193fff58a5bb21def8b9 --- /dev/null +++ b/docs/en/server/security/secharden/account-passwords.md @@ -0,0 +1,329 @@ +# Account Passwords + +- [Account Passwords](#account-passwords) + - [Shielding System Accounts](#shielding-system-accounts) + - [Restricting Account Permissions on the su Command](#restricting-account-permissions-on-the-su-command) + - [Setting Password Complexity](#setting-password-complexity) + - [Setting the Password Validity Period](#setting-the-password-validity-period) + - [Setting Password Encryption Algorithms](#setting-password-encryption-algorithms) + - [Locking an Account After Three Login Failures](#locking-an-account-after-three-login-failures) + - [Hardening the su Command](#hardening-the-su-command) + +## Shielding System Accounts + +### Description + +Accounts excluding user accounts are system accounts. System accounts cannot be used for logins or performing other operations. Therefore, system accounts must be shielded. + +### Implementation + +Modify the shell of a system account to **/sbin/nologin**. + +```shell +usermod -L -s /sbin/nologin $systemaccount +``` + +> [!NOTE]NOTE +> _$systemaccount_ indicates the system account. + +## Restricting Account Permissions on the su Command + +### Description + +The **su** command is used to switch user accounts. To improve system security, only the user **root** and users in the **wheel** group can use the **su** command. + +### Implementation + +Modify the **/etc/pam.d/su** file as follows: + +```text +auth required pam_wheel.so use_uid +``` + +**Table 1** Configuration item in pam\_wheel.so + + + + + + + + + + +

Item

+

Description

+

use_uid

+

UID of the current account.

+
+ +## Setting Password Complexity + +### Description + +You can set the password complexity requirements by modifying the corresponding configuration file. You are advised to set the password complexity based on the site requirements. + +### Implementation + +The password complexity is implemented by the **pam\_pwquality.so** and **pam\_pwhistory.so** modules in the **/etc/pam.d/password-auth** and **/etc/pam.d/system-auth** files. You can modify the configuration items of the two modules to change the password complexity requirements. + +### Example + +This section provides an example for configuring password complexity. + +**Password Complexity Requirements** + +1. Contains at least eight characters. +2. Contains at least three types of the following characters: + + - At least one lowercase letter + + - At least one uppercase letter + + - At least one digit + + - At least one space or one of the following special characters: \` \~ ! @ \# $ % ^ & \* \( \) - \_ = + \\ \| \[ \{ \} \] ; : ' " , < . \> / ? + +3. Cannot be the same as an account name or the account name in reverse order. +4. Cannot be the last five passwords used. + +**Implementation** + +Add the following content to the first two lines of the **password** configuration item in the **/etc/pam.d/password-auth** and **/etc/pam.d/system-auth** files: + +```text +password requisite pam_pwquality.so minlen=8 minclass=3 enforce_for_root try_first_pass local_users_only retry=3 dcredit=0 ucredit=0 lcredit=0 ocredit=0 +password required pam_pwhistory.so use_authtok remember=5 enforce_for_root +``` + +**Configuration Item Description** + +For details about the configuration items of **pam\_pwquality.so** and **pam\_pwhistory.so**, see [Table 1](#table201221044172117) and [Table 2](#table1212544452120), respectively. + +**Table 1** Configuration items in pam\_pwquality.so + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Item

+

Description

+

minlen=8

+

A password must contain at least eight characters.

+

minclass=3

+

A password must contain at least three of the following types: uppercase letters, lowercase letters, digits, and special characters.

+

ucredit=0

+

A password contains any number of uppercase letters.

+

lcredit=0

+

A password contains any number of lowercase letters.

+

dcredit=0

+

A password contains any number of digits.

+

ocredit=0

+

A password contains any number of special characters.

+

retry=3

+

Each time a maximum of three password changes is allowed.

+

enforce_for_root

+

This configuration is also effective for user root.

+
+ +**Table 2** Configuration items in pam\_pwhistory.so + + + + + + + + + + + + + +

Item

+

Description

+

remember=5

+

A password must be different from the last five passwords used.

+

enforce_for_root

+

This configuration is also effective for user root.

+
+ +## Setting the Password Validity Period + +### Description + +To ensure system security, you are advised to set the password validity period and notify users to change passwords before the passwords expire. + +### Implementation + +The password validity period is set by modifying the **/etc/login.defs** file. [Table 1](#en-us_topic_0152100281_t77b5d0753721450c81911c18b74e82eb) describes the hardening items. All hardening items in the table are in the **/etc/login.defs** file. You can directly modify the items in the configuration file. + +**Table 1** Configuration items in login.defs + + + + + + + + + + + + + + + + + + + + + + + + +

Item

+

Description

+

Suggestion

+

Configured as Suggested

+

PASS_MAX_DAYS

+

Maximum validity period of a password.

+

90

+

No

+

PASS_MIN_DAYS

+

Minimum interval between password changes.

+

0

+

No

+

PASS_WARN_AGE

+

Number of days before the password expires.

+

7

+

No

+
+ +> [!NOTE]NOTE +> The **login.defs** file is used to set restrictions on user accounts, such as setting the maximum password validity period and maximum length. The configuration in this file is invalid for the user **root**. If the **/etc/shadow** file contains the same items, the **/etc/shadow** configuration takes precedence over the **/etc/login.defs** configuration. When a user attempts to log in after the password expires, the user will be informed of the password expiry and is required to change the password. If the user does not change the password, the user cannot access the system. + +## Setting Password Encryption Algorithms + +### Description + +For system security, passwords cannot be stored in plaintext in the system and must be encrypted. The passwords that do not need to be restored must be encrypted using irreversible algorithms. Set the password encryption algorithm to SHA-512. This item has been set by default in openEuler. The preceding settings can effectively prevent password disclosure and ensure password security. + +### Implementation + +To set the password encryption algorithm, add the following configuration to the **/etc/pam.d/password-auth** and **/etc/pam.d/system-auth** files: + +```text +password sufficient pam_unix.so sha512 shadow nullok try_first_pass use_authtok +``` + +**Table 1** Configuration items in pam\_unix.so + + + + + + + + + + +

Item

+

Description

+

sha512

+

The SHA-512 algorithm is used for password encryption.

+
+ +## Locking an Account After Three Login Failures + +### Description + +To ensure user system security, you are advised to set the maximum number of failed login attempts \(three attempts are recommended\) and the automatic unlocking time \(300 seconds are recommended\) for a locked account. + +If an account is locked, any input is invalid but does not reset the locking countdown timer. Records of the user's invalid inputs are cleared once unlocked. The preceding settings protect passwords from being forcibly cracked and improve system security. + +> [!NOTE]NOTE +> By default, the maximum number of failed login attempts is 3 in openEuler. After an account is locked, the automatic unlock time is 60 seconds. + +### Implementation + +The password complexity is set by modifying the **/etc/pam.d/password-auth** and **/etc/pam.d/system-auth** files. The maximum number of failed login attempts is set to **3**, and the unlocking time after an account is locked is set to **300** seconds. The configuration is as follows: + +```text +auth required pam_faillock.so preauth audit deny=3 even_deny_root unlock_time=300 +auth [default=die] pam_faillock.so authfail audit deny=3 even_deny_root unlock_time=300 +auth sufficient pam_faillock.so authsucc audit deny=3 even_deny_root unlock_time=300 +``` + +**Table 1** Configuration items in pam\_faillock.so + + + + + + + + + + + + + + + + + + + +

Item

+

Description

+

authfail

+

Captures account login failure events.

+

deny=3

+

A user account will be locked after three failed login attempts.

+

unlock_time=300

+

A locked common user account is automatically unlocked after 300 seconds.

+

even_deny_root

+

This configuration is also effective for user root.

+
+ +## Hardening the su Command + +### Description + +To enhance system security and prevent the environment variables of the current user from being brought into other environments when you run the **su** command to switch to another user, this item has been configured by default in openEuler. The **PATH** variable is always initialized when the **su** command is used to switch users. + +### Implementation + +Modify the **/etc/login.defs** file. The configuration is as follows: + +```text +ALWAYS_SET_PATH=yes +``` diff --git a/docs/en/server/security/secharden/appendix.md b/docs/en/server/security/secharden/appendix.md new file mode 100644 index 0000000000000000000000000000000000000000..f57674a77570540cfbbaa7b76f26772b1d9320d3 --- /dev/null +++ b/docs/en/server/security/secharden/appendix.md @@ -0,0 +1,31 @@ +# Appendix + +This chapter describes the file permissions and **umask** values. + +- [Appendix](#appendix) + - [Permissions on Files and Directories](#permissions-on-files-and-directories) + - [umask Values](#umask-values) + +## Permissions on Files and Directories + +Permission on files and directories in Linux specifies the users who can access and perform operations on files and directories and the access and operation modes. Permissions on files and directories include read only, write only, and execute. + +The following types of users can access files and directories: + +- File creator +- Users in the same group as a file creator +- Users not in the same group as a file creator + +An example of permission on files and directories is described as follows: + +If the permission on **/usr/src** is set to **755** which is 111101101 in binary mode, permissions for each type of users are described as follows: + +- The left-most **111** indicates that the file owner can read, write, and execute the file. +- The middle **101** indicates the group users can read and execute but cannot write the file. +- The right-most **101** indicates that other users can read and execute but cannot write the file. + +## umask Values + +When a user creates a file or directory, the file or directory has a default permission. The default permission is specified by the **umask** value. + +The **umask** value is the complement of the permission value. The actual permission value is obtained by subtracting the **umask** value from the default maximum permission value. The default maximum permission of a file is readable and writable. The default maximum permission of a directory is readable, writable, and executable. The default permission of a file is 666 minus the **umask** value. The default permission of a directory is 777 minus the **umask** value. diff --git a/docs/en/server/security/secharden/authentication-and-authorization.md b/docs/en/server/security/secharden/authentication-and-authorization.md new file mode 100644 index 0000000000000000000000000000000000000000..9baa3855c4f11c0bec78420214cc0c592c43fd45 --- /dev/null +++ b/docs/en/server/security/secharden/authentication-and-authorization.md @@ -0,0 +1,145 @@ +# Authentication and Authorization + +## Setting a Warning for Remote Network Access + +### Description + +A warning for remote network access is configured and displayed for users who attempt to remotely log in to the system. The warning indicates the penalty for authorized access and is used to threaten potential attackers. When the warning is displayed, system architecture and other system information are hidden to protect the system from being attacked. + +### Implementation + +This setting can be implemented by modifying the **/etc/issue.net** file. Replace the original content in the **/etc/issue.net** file with the following information \(which has been set by default in openEuler\): + +```text +Authorized users only. All activities may be monitored and reported. +``` + +## Forestalling Unauthorized System Restart by Pressing Ctrl+Alt+Delete + +### Description + +By default, you can restart the system by pressing **Ctrl**+**Alt**+**Delete**. You are advised to disable this function to prevent data loss due to misoperations. + +### Implementation + +To disable the feature of restarting the system by pressing **Ctrl**+**Alt**+**Delete**, perform the following steps: + +1. Run the following commands to delete the two **ctrl-alt-del.target** files: + + ```shell + rm -f /etc/systemd/system/ctrl-alt-del.target + rm -f /usr/lib/systemd/system/ctrl-alt-del.target + ``` + +2. Change **\#CtrlAltDelBurstAction=reboot-force** to **CtrlAltDelBurstAction=none** in the **/etc/systemd/system.conf** file. +3. Run the following command to restart systemd for the modification to take effect. Note that running the command may cause system services to be unavailable or restarted temporarily. In addition, you must be the **root** user or a user with the sudo permission to perform this operation. + + ```shell + systemctl daemon-reexec + ``` + +## Setting an Automatic Exit Interval for Shell + +### Description + +An unattended shell is prone to listening or attacks. Therefore, it is advised that a mechanism be configured to ensure that a shell can automatically exit when it does not run for a period. + +### Implementation + +At the end of file **/etc/profile**, set the **TMOUT** field \(unit: second\) that specifies the interval for automatic exit as follows: + +```shell +export TMOUT=300 +``` + +## Setting the Default umask Value for Users to 0077 + +### Description + +The **umask** value is used to set default permission on files and directories. A smaller **umask** value indicates that group users or other users have incorrect permission, which brings system security risks. Therefore, the default **umask** value must be set to **0077** for all users, that is, the default permission on user directories is **700** and the permission on user files is **600**. The **umask** value indicates the complement of a permission. For details about how to convert the **umask** value to a permission, see [umask Values](./appendix.md#umask-values). + +> [!NOTE]NOTE +> By default, the **umask** value of the openEuler user is set to **0022**. + +### Implementation + +1. Add **umask 0077** to the **/etc/bashrc** file and all files in the **/etc/profile.d/** directory. + + ```shell + echo "umask 0077" >> $FILE + ``` + + > [!NOTE]NOTE + > _$FILE_ indicates the file name, for example, echo "umask 0077" \>\> /etc/bashrc. + +2. Set the ownership and group of the **/etc/bashrc** file and all files in the **/etc/profile.d/** directory to **root**. + + ```shell + chown root.root $FILE + ``` + + > [!NOTE]NOTE + > _$FILE_ indicates the file name, for example, **chown root.root /etc/bashrc**. + +## Setting the GRUB2 Encryption Password + +### Description + +GRand Unified Bootloader \(GRUB\) is an operating system boot manager used to boot different systems \(such as Windows and Linux\). GRUB2 is an upgraded version of GRUB. + +When starting the system, you can modify the startup parameters of the system on the GRUB2 screen. To ensure that the system startup parameters are not modified randomly, you need to encrypt the GRUB2 screen. The startup parameters can be modified only when the correct GRUB2 password is entered. + +> [!NOTE]NOTE +> The default password of GRUB2 is **openEuler\#12**. You are advised to change the default password upon the first login and periodically update the password. If the password is leaked, startup item configurations may be modified, causing the system startup failure. + +### Implementation + +1. Run the **grub2-mkpasswd-pbkdf2** command to generate an encrypted password. + + > [!NOTE]NOTE + > SHA-512 is used as the GRUB2 encryption algorithm. + + ```shell + $ grub2-mkpasswd-pbkdf2 + Enter password: + Reenter password: + PBKDF2 hash of your password is + grub.pbkdf2.sha512.10000.5A45748D892672FDA02DD3B6F7AE390AC6E6D532A600D4AC477D25C7D087644697D8A0894DFED9D86DC2A27F4E01D925C46417A225FC099C12DBD3D7D49A7425.2BD2F5BF4907DCC389CC5D165DB85CC3E2C94C8F9A30B01DACAA9CD552B731BA1DD3B7CC2C765704D55B8CD962D2AEF19A753CBE9B8464E2B1EB39A3BB4EAB08 + ``` + + > [!NOTE]NOTE + > Enter the same password in the **Enter password** and **Reenter password** lines. + > After **openEuler\#12** is encrypted by **grub2-mkpasswd-pbkdf2**, the output is **grub.pbkdf2.sha512.10000.5A45748D892672FDA02DD3B6F7AE390AC6E6D532A600D4AC477D25C7D087644697D8A0894DFED9D86DC2A27F4E01D925C46417A225FC099C12DBD3D7D49A7425.2BD2F5BF4907DCC389CC5D165DB85CC3E2C94C8F9A30B01DACAA9CD552B731BA1DD3B7CC2C765704D55B8CD962D2AEF19A753CBE9B8464E2B1EB39A3BB4EAB08**. The ciphertext is different each time. + +2. Open **/boot/efi/EFI/openEuler/grub.cfg** in a vi editor. In different modes, the paths of the **grub.cfg** file are different. See the note below. Append the following fields to the beginning of **/boot/efi/EFI/openEuler/grub.cfg**. + + ```text + set superusers="root" + password_pbkdf2 root grub.pbkdf2.sha512.10000.5A45748D892672FDA02DD3B6F7AE390AC6E6D532A600D4AC477D25C7D087644697D8A0894DFED9D86DC2A27F4E01D925C46417A225FC099C12DBD3D7D49A7425.2BD2F5BF4907DCC389CC5D165DB85CC3E2C94C8F9A30B01DACAA9CD552B731BA1DD3B7CC2C765704D55B8CD962D2AEF19A753CBE9B8464E2B1EB39A3BB4EAB08 + ``` + + > [!NOTE]NOTE + + - In different modes, the paths of the **grub.cfg** file are different: In the UEFI mode of the x86 architecture, the path is **/boot/efi/EFI/openEuler/grub.cfg**. In the Legacy BIOS mode of the x86 architecture, the path is **/boot/grub2/grub.cfg**. In the aarch64 architecture, the path is **/boot/efi/EFI/openEuler/grub.cfg**. + - The **superusers** field is used to set the account name of the super GRUB2 administrator. + - The first parameter following the **password\_pbkdf2** field is the GRUB2 account name, and the second parameter is the encrypted password of the account. + +## Setting the Secure Single-user Mode + +### Description + +When you log in to the system as user **root** in single-user mode, if the **root** password is not set, high security risks exist. + +### Implementation + +This setting can be implemented by modifying the **/etc/sysconfig/init** file. Set **SINGLE** to **SINGLE=/sbin/sulogin**. + +## Disabling Interactive Startup + +### Description + +With interactive guidance, console users can disable audit, firewall, or other services, which compromises system security. Users can disable interactive startup to improve security. This item is disabled by default in openEuler. + +### Implementation + +This setting can be implemented by modifying the **/etc/sysconfig/init** file. Set **PROMPT** to **no**. diff --git a/docs/en/server/security/secharden/file-permissions.md b/docs/en/server/security/secharden/file-permissions.md new file mode 100644 index 0000000000000000000000000000000000000000..6bedea21b7aa566950bc91317cdb676a76129b9c --- /dev/null +++ b/docs/en/server/security/secharden/file-permissions.md @@ -0,0 +1,233 @@ +# File Permissions + +- [File Permissions](#file-permissions) + - [Setting the Permissions on and Ownership of Files](#setting-the-permissions-on-and-ownership-of-files) + - [Deleting Unowned Files](#deleting-unowned-files) + - [Removing a Symbolic Link to /dev/null](#removing-a-symbolic-link-to-devnull) + - [Setting the umask Value for a Daemon](#setting-the-umask-value-for-a-daemon) + - [Adding a Sticky Bit Attribute to Globally Writable Directories](#adding-a-sticky-bit-attribute-to-globally-writable-directories) + - [Disabling the Globally Writable Permission on Unauthorized Files](#disabling-the-globally-writable-permission-on-unauthorized-files) + - [Restricting Permissions on the at Command](#restricting-permissions-on-the-at-command) + - [Restricting Permissions on the cron Command](#restricting-permissions-on-the-cron-command) + - [Restricting Permissions on the sudo Command](#restricting-permissions-on-the-sudo-command) + +## Setting the Permissions on and Ownership of Files + +### Description + +In Linux, all objects are processed as files. Even a directory will be processed as a large file containing many files. Therefore, the most important thing in Linux is the security of files and directories. Their security is ensured by permissions and owners. + +By default, the permissions and ownership of common directories, executable files, and configuration files in the system are set in openEuler. + +### Implementation + +The following uses the **/bin** directory as an example to describe how to change the permission and ownership of a file: + +- Modify the file permission. For example, set the permission on the **/bin** directory to **755**. + + ```shell + chmod 755 /bin + ``` + +- Change the ownership of the file. For example, set the ownership and group of the **/bin** directory to **root:root**. + + ```shell + chown root:root /bin + ``` + +## Deleting Unowned Files + +### Description + +When deleting a user or group, the system administrator may forget to delete the files of the user or group. If the name of a new user or group is the same as that of the deleted user or group, the new user or group will own files on which it has no permission. You are advised to delete these files. + +### Implementation + +Delete the file whose user ID does not exist. + +1. Search for the file whose user ID does not exist. + + ```shell + find / -nouser + ``` + +2. Delete the found file. In the preceding command, _filename_ indicates the name of the file whose user ID does not exist. + + ```shell + rm -f filename + ``` + +Delete the file whose group ID does not exist. + +1. Search for the file whose user ID does not exist. + + ```shell + find / -nogroup + ``` + +2. Delete the found file. In the preceding command, _filename_ indicates the name of the file whose group ID does not exist. + + ```shell + rm -f filename + ``` + +## Removing a Symbolic Link to /dev/null + +### Description + +A symbolic link to **/dev/null** may be used by malicious users. This affects system security. You are advised to delete these symbolic links to improve system security. + +### Special Scenario + +After openEuler is installed, symbolic links to **/dev/null** may exist. These links may have corresponding functions. \(Some of them are preconfigured and may be depended by other components.\) Rectify the fault based on the site requirements. For details, see [Implementation](#en-us_topic_0152100319_s1b24647cdd834a8eaca3032611baf072). + +For example, openEuler supports UEFI and legacy BIOS installation modes. The GRUB packages supported in the two boot scenarios are installed by default. If you select the legacy BIOS installation mode, a symbolic link **/etc/grub2-efi.cfg** is generated. If you select the UEFI installation mode, a symbolic link **/etc/grub2.cfg** is generated. You need to process these symbolic links based on the site requirements. + +### Implementation + +1. Run the following command to search for symbolic links to **/dev/null**: + + ```shell + find dirname -type l -follow 2>/dev/null + ``` + + > [!NOTE]NOTE + > _dir__name_ indicates the directory to be searched. Normally, key system directories, such as **/bin**, **/boot**, **/usr**, **/lib64**, **/lib**, and **/var**, need to be searched. + +2. If these symbolic links are useless, run the following command to delete them: + + ```shell + rm -f filename + ``` + + > [!NOTE]NOTE + > _filename_ indicates the file name obtained in [Step 1](#en-us_topic_0152100319_l4dc74664c4fb400aaf91fb314c4f9da6). + +## Setting the umask Value for a Daemon + +### Description + +The **umask** value is used to set default permission on files and directories. If the **umask** value is not specified, the file has the globally writable permission. This brings risks. A daemon provides a service for the system to receive user requests or network customer requests. To improve the security of files and directories created by the daemon, you are advised to set **umask** to **0027**. The **umask** value indicates the complement of a permission. For details about how to convert the **umask** value to a permission, see [umask Values](./appendix.md#umask-values). + +> [!NOTE]NOTE +> By default, the **umask** value of the daemon is set to **0022** in openEuler. + +### Implementation + +In configuration file **/etc/sysconfig/init**, add **umask 0027** as a new row. + +## Adding a Sticky Bit Attribute to Globally Writable Directories + +### Description + +Any user can delete or modify a file or directory in a globally writable directory, which leads to unauthorized file or directory deletion. Therefore, the sticky bit attribute is required for globally writable directories. + +### Implementation + +1. Search for globally writable directories. + + ```shell + find / -type d -perm -0002 ! -perm -1000 -ls | grep -v proc + ``` + +2. Add the sticky bit attribute to globally writable directories. _dirname_ indicates the name of the directory that is found. + + ```shell + chmod +t dirname + ``` + +## Disabling the Globally Writable Permission on Unauthorized Files + +### Description + +Any user can modify globally writable files, which affects system integrity. + +### Implementation + +1. Search for all globally writable files. + + ```shell + find / -type d ( -perm -o+w ) | grep -v proc + find / -type f ( -perm -o+w ) | grep -v proc + ``` + +2. View the settings of files \(excluding files and directories with sticky bits\) listed in step 1, and delete the files or disable the globally writable permission on them. Run the following command to remove the permission. In the command, _filename_ indicates the file name. + + ```shell + chmod o-w filename + ``` + + > [!NOTE]NOTE + > You can run the following command to check whether the sticky bit is set for the file or directory. If the command output contains the **T** flag, the file or directory is with a sticky bit. In the command, _filename_ indicates the name of the file or directory to be queried. + + ```shell + ls -l filename + ``` + +## Restricting Permissions on the at Command + +### Description + +The **at** command is used to create a scheduled task. Users who can run the **at** command must be specified to protect the system from being attacked. + +### Implementation + +1. Delete the **/etc/at.deny** file. + + ```shell + rm -f /etc/at.deny + ``` + +2. Run the following commands to create the **/etc/at.allow** file and change its ownership to **root:root**. + + ```shell + touch /etc/at.allow + chown root:root /etc/at.allow + ``` + +3. Set that only user **root** can operate file **/etc/at.allow**. + + ```shell + chmod og-rwx /etc/at.allow + ``` + +## Restricting Permissions on the cron Command + +### Description + +The **cron** command is used to create a routine task. Users who can run the **cron** command must be specified to protect the system from being attacked. + +### Implementation + +1. Delete the **/etc/cron.deny** file. + + ```shell + rm -f /etc/at.deny + ``` + +2. Run the following commands to create the **/etc/cron.allow** file and change its ownership to **root:root**: + + ```shell + touch /etc/cron.allow + chown root:root /etc/cron.allow + ``` + +3. Set that only user **root** can operate file **/etc/cron.allow**. + + ```shell + chmod og-rwx /etc/cron.allow + ``` + +## Restricting Permissions on the sudo Command + +### Description + +A common user can use the **sudo** command to run commands as the user **root**. To harden system security, it is necessary to restrict permissions on the **sudo** command. Only user **root** can use the **sudo** command. By default, openEuler does not restrict the permission of non-root users to run the sudo command. + +### Implementation + +Modify the **/etc/sudoers** file to restrict permissions on the **sudo** command. Comment out the following configuration line: + +```text +#%wheel ALL=(ALL) ALL +``` diff --git a/docs/en/server/security/secharden/kernel-parameters.md b/docs/en/server/security/secharden/kernel-parameters.md new file mode 100644 index 0000000000000000000000000000000000000000..f2d4816884fa477d89c62721db651de7c22432df --- /dev/null +++ b/docs/en/server/security/secharden/kernel-parameters.md @@ -0,0 +1,228 @@ +# Kernel Parameters + +- [Kernel Parameters](#kernel-parameters) + - [Hardening the Security of Kernel Parameters](#hardening-the-security-of-kernel-parameters) + +## Hardening the Security of Kernel Parameters + +### Description + +Kernel parameters specify the status of network configurations and application privileges. The kernel provides system control which can be fine-tuned or configured by users. This function can improve the security of the OS by controlling configurable kernel parameters. For example, you can fine-tune or configure network options to improve system security. + +### Implementation + +1. Write the hardening items in [Table 1](#en-us_topic_0152100187_t69b5423c26644b26abe94d88d38878eb) to the **/etc/sysctl.conf** file. + + > [!NOTE]NOTE + > Record security hardening items as follows: + + ```ini + net.ipv4.icmp_echo_ignore_broadcasts = 1 + net.ipv4.conf.all.rp_filter = 1 + net.ipv4.conf.default.rp_filter = 1 + ``` + + **Table 1** Policies for hardening the security of kernel parameters + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Item

+

Description

+

Suggestion

+

Configured as Suggested

+

net.ipv4.icmp_echo_ignore_broadcasts

+

Specifies whether ICMP broadcast packets are accepted. They are not accepted according to the hardening policy.

+

1

+

Yes

+

net.ipv4.conf.all.rp_filter

+

Specifies whether the actual source IP address used by a data packet is related to a routing table and whether the data packet receives responses through interfaces. The item is enabled according to the hardening policy.

+

1

+

Yes

+

net.ipv4.conf.default.rp_filter

+

1

+

Yes

+

net.ipv4.ip_forward

+

The IP forwarding function prevents unauthorized IP address packets from being transferred to a network. The item is disabled according to the hardening policy.

+

0

+

Yes

+

net.ipv4.conf.all.accept_source_route

+

accept_source_route indicates that a packet sender can specify a path for sending the packet and a path for receiving a response. The item is disabled according to the hardening policy.

+

0

+

Yes

+

net.ipv4.conf.default.accept_source_route

+

0

+

Yes

+

net.ipv4.conf.all.accept_redirects

+

Specifies whether a redirected ICMP packet is sent. The packet is not sent according to the hardening policy.

+

0

+

Yes

+

net.ipv4.conf.default.accept_redirects

+

0

+

Yes

+

net.ipv6.conf.all.accept_redirects

+

0

+

Yes

+

net.ipv6.conf.default.accept_redirects

+

0

+

Yes

+

net.ipv4.conf.all.send_redirects

+

Specifies whether a redirected ICMP packet is sent to another server. This item is enabled only when the host functions as a route. The item is disabled according to the hardening policy.

+

0

+

Yes

+

net.ipv4.conf.default.send_redirects

+

0

+

Yes

+

net.ipv4.icmp_ignore_bogus_error_responses

+

Fake ICMP packets are not recorded to logs, which saves disk space. The item is enabled according to the hardening policy.

+

1

+

Yes

+

net.ipv4.tcp_syncookies

+

SYN attack is a DoS attack that forces system restart by occupying system resources. TCP-SYN cookie protection is enabled according to the hardening policy.

+

1

+

Yes

+

kernel.dmesg_restrict

+

Hardens dmesg messages. Only the administrator is allowed to view the messages.

+

1

+

Yes

+

kernel.sched_autogroup_enabled

+

Determines whether the kernel automatically groups and schedules threads. After this item is enabled, scheduling groups compete for time slices, and threads in a scheduling group compete for the time slices allocated to the scheduling group. The item is disabled according to the hardening policy.

+

0

+

No

+

kernel.sysrq

+

Disables the magic key.

+
NOTE:

You are advised to disable the magic key so that commands cannot be directly passed to the kernel.

+
+

0

+

Yes

+

net.ipv4.conf.all.secure_redirects

+

Specifies whether redirected ICMP messages sent from any servers or from gateways listed in the default gateway list are accepted. Redirected ICMP messages are received from any servers according to the hardening policy.

+

0

+

Yes

+

net.ipv4.conf.default.secure_redirects

+

0

+

Yes

+
+ +2. Run the following command to load the kernel parameters set in the **sysctl.conf** file: + + ``` + sysctl -p /etc/sysctl.conf + ``` + +### Other Security Suggestions + +- **net.ipv4.icmp\_echo\_ignore\_all**: ignores ICMP requests. + + For security purposes, you are advised to enable this item. The default value is **0**. Set the value to **1** to enable this item. + + After this item is enabled, all incoming ICMP Echo request packets will be ignored, which will cause failure to ping the target host. Determine whether to enable this item based on your actual networking condition. + +- **net.ipv4.conf.all.log\_martians/net.ipv4.conf.default.log\_martians**: logs spoofed, source routed, and redirect packets. + + For security purposes, you are advised to enable this item. The default value is **0**. Set the value to **1** to enable this item. + + After this item is enabled, data from forbidden IP addresses will be logged. Too many new logs will overwrite old logs because the total number of logs allowed is fixed. Determine whether to enable this item based on your actual usage scenario. + +- **net.ipv4.tcp\_timestamps**: disables tcp\_timestamps. + + For security purposes, you are advised to disable tcp\_timestamps. The default value is **1**. Set the value to **0** to disable tcp\_timestamps. + + After this item is disabled, TCP retransmission timeout will be affected. Determine whether to disable this item based on the actual usage scenario. + +- **net.ipv4.tcp\_max\_syn\_backlog**: determines the number of queues that is in SYN\_RECV state. + + This parameter determines the number of queues that is in SYN\_RECV state. When this number is exceeded, new TCP connection requests will not be accepted. This to some extent prevents system resource exhaustion. Configure this parameter based on your actual usage scenario. diff --git a/docs/en/server/security/secharden/os-hardening-overview.md b/docs/en/server/security/secharden/os-hardening-overview.md new file mode 100644 index 0000000000000000000000000000000000000000..9e6e8fa69978b185307b7ec9c6486c0341e3919c --- /dev/null +++ b/docs/en/server/security/secharden/os-hardening-overview.md @@ -0,0 +1,118 @@ +# OS Hardening Overview + +- [OS Hardening Overview](#os-hardening-overview) + - [Security Hardening Purpose](#security-hardening-purpose) + - [Security Hardening Solution](#security-hardening-solution) + - [Security Hardening Impacts](#security-hardening-impacts) + +This chapter describes the purpose and solution of openEuler system hardening. + +## security-hardening-purpose + +The OS, as the core of the information system, manages hardware and software resources and is the basis of information system security. Applications must depend on the OS to ensure the integrity, confidentiality, availability, and controllability of information. Without the OS security protection, protective methods against hackers and virus attacks at other layers cannot meet the security requirements. + +Therefore, security hardening is essential for an OS. Security hardening helps build a dynamic and complete security system, enhance product security, and improve product competitiveness. + +## security-hardening-solution + +This section describes the openEuler security hardening solution, including the hardening method and items. + +### Security Hardening Method + +You can manually modify security hardening configurations or run commands to harden the system, or use the security hardening tool to modify security hardening items in batches. The openEuler security hardening tool runs as openEuler-security.service. When the system is started for the first time, the system automatically runs the service to execute the default hardening policy, and automatically set the service not to start as the system starts. + +You can modify the **security.conf** file and use the security hardening tool to implement user-defined security hardening. + +## security hardening impacts + +Security hardening on file permissions and account passwords may change user habits, affecting system usability. For details about common hardening items that affect system usability, see [Table 1](#en-us_topic_0152100325_ta4a48f54ff2849ada7845e2380209917). + +**Table 1** Security hardening impacts + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Item

+

Suggestion

+

Impact

+

Configured By Default

+

Timeout setting on the text-based user interface (TUI)

+

When the TUI is idle for a long period of time, it automatically exits.

+
NOTE:

When a user logs in to the system using SSH, the timeout period is determined by the smaller value of the TMOUT field in the /etc/profile file and the ClientAliveInterval field in the /etc/ssh/sshd_config file. You are advised to set this parameter to 300 seconds.

+
+

If you do not perform any operation on the TUI for a long time, TUI automatically exits.

+

No

+

Password complexity

+

The password is a string containing at least eight characters chosen from three or four of the following types: uppercase letters, lowercase letters, digits, and special characters.

+

All passwords must comply with the complexity requirements.

+

No

+

Password retry limits

+

If a user fails to enter the correct password for three consecutive times when logging in to the OS, the user account will be locked for 60 seconds.

+

After the account is locked, the user can log in to the system only after 60 seconds.

+

Yes

+

Default umask value

+

The default umask value of all users is set to 077 so that the default permission on files created by users is 600 and the default permission on directories is 700.

+

Users must modify the permission on specified files or directories as required.

+

No

+

Password validity period

+

The password validity period can be modified in the /etc/login.defs file and is set to 90 days by default. It can be modified in any time. An expiration notification will be displayed seven days before a password is to expire.

+

When a user attempts to log in after the password expires, the user will be informed of the password expiry and is required to change the password. If the user does not change the password, the user cannot access the system.

+

No

+

su permission control

+

The su command is used to switch user accounts. To improve system security, only the user root and users in the wheel group can use the su command.

+

Common users can successfully run the su command only after joining in the wheel group.

+

Yes

+

Disabling user root from logging in using SSH

+

Set the value of the PermitRootLogin field in the /etc/ssh/sshd_config file to no. In this way, user root cannot directly log in to the system using SSH.

+

You need to log in to the system as a common user in SSH mode and then switch to user root.

+

No

+

Strong SSH encryption algorithm

+

The MACs and Ciphers configurations of SSH services support the CTR and SHA2 algorithms and do not support the CBC, MD5, and SHA1 algorithms.

+

Some early Xshell and PuTTY versions on Windows do not support aes128-ctr, aes192-ctr, aes256-ctr, hmac-sha2-256, and hmac-sha2-512 algorithms. Ensure that the latest PuTTY (0.63 or later) and Xshell (5.0 or later) are used.

+

Yes

+
diff --git a/docs/en/server/security/secharden/public_sys-resources/icon-note.gif b/docs/en/server/security/secharden/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/server/security/secharden/public_sys-resources/icon-note.gif differ diff --git a/docs/en/server/security/secharden/secharden.md b/docs/en/server/security/secharden/secharden.md new file mode 100644 index 0000000000000000000000000000000000000000..6539f6bd1eb52da6f2076345facd9c4759538a79 --- /dev/null +++ b/docs/en/server/security/secharden/secharden.md @@ -0,0 +1,5 @@ +# Security Hardening Guide + +This document describes how to perform security hardening for openEuler. + +This document is intended for administrators who need to perform security hardening for openEuler. You must be familiar with the OS security architecture and technologies. diff --git a/docs/en/server/security/secharden/security-configuration-benchmark.md b/docs/en/server/security/secharden/security-configuration-benchmark.md new file mode 100644 index 0000000000000000000000000000000000000000..3d11a99607f7df7c1ce1002b45884307ef6fdfea --- /dev/null +++ b/docs/en/server/security/secharden/security-configuration-benchmark.md @@ -0,0 +1,3 @@ +# openEuler Security Configuration Description + +For details, see the [openEuler security configuration description](https://gitee.com/openeuler/security-committee/tree/master/secure-configuration-benchmark). diff --git a/docs/en/server/security/secharden/security-hardening-guide.md b/docs/en/server/security/secharden/security-hardening-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..1863f6a1230fb77085954fada3892ee4c8282f82 --- /dev/null +++ b/docs/en/server/security/secharden/security-hardening-guide.md @@ -0,0 +1,3 @@ +# Security Hardening Guide + +You can modify the hardening policy configuration file or script to harden the system. This chapter describes the hardening items, whether the items are hardened by default, and how to perform security hardening. diff --git a/docs/en/server/security/secharden/security-hardening-tools.md b/docs/en/server/security/secharden/security-hardening-tools.md new file mode 100644 index 0000000000000000000000000000000000000000..2bb1f5feaecde25e651915bbc66b8232a5984f3c --- /dev/null +++ b/docs/en/server/security/secharden/security-hardening-tools.md @@ -0,0 +1,122 @@ +# Security Hardening Tools + +## Security Hardening Procedure + +### Overview + +You need to modify the **usr-security.conf** file so that the security hardening tool can set hardening policies based on the **usr-security.conf** file. This section describes rules for modifying the **usr-security.conf** file. For details about the configurable security hardening items, see [Security Hardening Guide](security-hardening-guide.md). + +### Precautions + +- After modifying the items, restart the security hardening service for the modification to take effect. For details about how to restart the service, see [Hardening Items Taking Effect](#hardening-items-taking-effect). +- When modifying security hardening items, you only need to modify the **/etc/openEuler\_security/usr-security.conf** file. You are not advised to modify the **/etc/openEuler\_security/security.conf** file. The **security.conf** file contains basic hardening items which are executed only once. +- After the security hardening service is restarted for the configuration to take effect, the configuration in effect cannot be deleted by deleting the corresponding hardening items from the **usr-security.conf** file and restarting the security hardening service. +- Security hardening operations are recorded in the **/var/log/openEuler-security.log** file. + +### Configuration Format + +Each line in the **usr-security.conf** file indicates a configuration item. The configuration format varies according to the configuration content. The following describes the format of each configuration item. + +> [!NOTE]NOTE + +- All configuration items start with an execution ID. The execution ID is a positive integer and can be customized. +- Contents of a configuration item are separated by an at sign \(@\). +- If the actual configuration content contains an at sign \(@\), use two at signs \(@@\) to distinguish the content from the separator. For example, if the actual content is **xxx@yyy**, set this item to **xxx@@yyy**. Currently, an at sign \(@\) cannot be placed at the beginning or end of the configuration content. + +- **d**: comment + + Format: _Execution ID_**@d@**_Object file_**@**_Match item_ + + Function: Comment out lines starting with the match item \(the line can start with a space\) in an object file by adding a number sign \(\#\) at the beginning of the line. + + Example: If the execution ID is **401**, comment out lines starting with **%wheel** in the **/etc/sudoers** file. + + ```sh + 401@d@/etc/sudoers@%wheel + ``` + +- **m**: replacement + + Format: _Execution ID_**@m@**_Object file_**@**_Match item_**@**_Target value_ + + Function: Replace lines starting with the match item \(the line can start with a space\) in an object file with _match item_ and _target value_. If the match line starts with spaces, the spaces will be deleted after the replacement. + + Example: If the execution ID is **101**, replace lines starting with **Protocol** in the **/etc/ssh/sshd\_config** file with **Protocol 2**. The spaces after **Protocol** are matched and replaced. + + ```sh + 101@m@/etc/ssh/sshd_config@Protocol @2 + ``` + +- **sm**: accurate modification + + Format: _Execution ID_**@sm@**_Object file_**@**_Match item_**@**_Target value_ + + Function: Replace lines starting with the match item \(the line can start with a space\) in an object file with _match item_ and _target value_. If the match line starts with spaces, the spaces are retained after the replacement. This is the difference between **sm** and **m**. + + Example: If the execution ID is **201**, replace lines starting with **size** in the **/etc/audit/hzqtest** file with **size 2048**. + + ```sh + 201@sm@/etc/audit/hzqtest@size@ 2048 + ``` + +- **M**: subitem modification + + Format: _Execution ID_**@M@**_Object file_**@**_Match item_**@**_Match subitem__\[@Value of the match subitem\]_ + + Function: Match lines starting with the match item \(the line can start with a space\) in an object file and replace the content starting with the match subitem in these lines with the _match subitem_ and _value of the match subitem_. The value of the match subitem is optional. + + Example: If the execution ID is **101**, find lines starting with **key** in the file and replace the content starting with **key2** in these lines with **key2value2**. + + ```sh + 101@M@file@key@key2@value2 + ``` + +- **systemctl**: service management + + Format: _Execution ID_**@systemctl@**_Object service_**@**_Operation_ + + Function: Use **systemctl** to manage object services. The value of **Operation** can be **start**, **stop**, **restart**, or **disable**. + + Example: If the execution ID is **218**, stop the **cups.service**. This provides the same function as running the **systemctl stop cups.service** command. + + ```sh + 218@systemctl@cups.service@stop + ``` + +- Other commands + + Format: _Execution ID_**@**_Command_**@**_Object file_ + + Function: Run the corresponding command, that is, run the command line _Command_ _Object file_. + + Example 1: If the execution ID is **402**, run the **rm -f** command to delete the **/etc/pki/ca-trust/extracted/pem/email-ca-bundle.pem** file. + + ```sh + 402@rm -f @/etc/pki/ca-trust/extracted/pem/email-ca-bundle.pem + ``` + + Example 2: If the execution ID is **215**, run the **touch** command to create the **/etc/cron.allow** file. + + ```sh + 215@touch @/etc/cron.allow + ``` + + Example 3: If the execution ID is **214**, run the **chown** command to change the owner of the **/etc/at.allow** file to **root:root**. + + ```sh + 214@chown root:root @/etc/at.allow + ``` + + Example 4: If the execution ID is **214**, run the **chmod** command to remove the **rwx** permission of the group to which the owner of the**/etc/at.allow** file belongs and other non-owner users. + + ```sh + 214@chmod og-rwx @/etc/at.allow + ``` + +## Hardening Items Taking Effect + +After modifying the **usr-security.conf** file, run the following command for the new configuration items to take effect: + +```sh +systemctl restart openEuler-security.service +``` diff --git a/docs/en/server/security/secharden/selinux-configuration.md b/docs/en/server/security/secharden/selinux-configuration.md new file mode 100644 index 0000000000000000000000000000000000000000..edc2ef688ec0dfe60452611d45c11273f2d3b9ac --- /dev/null +++ b/docs/en/server/security/secharden/selinux-configuration.md @@ -0,0 +1,273 @@ +# SELinux Configuration + +## Overview + +Discretionary access control \(DAC\) determines whether a resource can be accessed based on users, groups, and other permissions. It does not allow the system administrator to create comprehensive and fine-grained security policies. SELinux \(Security-Enhanced Linux\) is a module of the Linux kernel and a security subsystem of Linux. SELinux implements mandatory access control \(MAC\). Each process and system resource has a special security label. In addition to the principles specified by the DAC, the SELinux needs to determine whether each type of process has the permission to access a type of resource. + +By default, openEuler uses SELinux to improve system security. SELinux has three modes: + +- **permissive**: The SELinux outputs alarms but does not forcibly execute the security policies. +- **enforcing**: The SELinux security policies are forcibly executed. +- **disabled**: The SELinux security policies are not loaded. + +## Configuration Description + +- Obtain the SELinux running status: + + ```shell + $ getenforce + Enforcing + ``` + +- Set the enforcing mode when SELinux is enabled: + + ```shell + $ setenforce 1 + $ getenforce + Enforcing + ``` + +- Set the permissive mode when SELinux is enabled: + + ```shell + $ setenforce 0 + $ getenforce + Permissive + ``` + +- Disable SELinux when SELinux is enabled. (A reboot is required.) + 1. Modify the SELinux configuration file **/etc/selinux/config** and set **SELINUX=disabled**. + + ```shell + $ cat /etc/selinux/config | grep "SELINUX=" + SELINUX=disabled + ``` + + 2. Reboot the system. + + ```shell + reboot + ``` + + 3. The status is changed successfully. + + ```shell + $ getenforce + Disabled + ``` + +- Set the permissive mode when SELinux is disabled: + 1. Modify the SELinux configuration file **/etc/selinux/config** and set **SELINUX=permissive**. + + ```shell + $ cat /etc/selinux/config | grep "SELINUX=" + SELINUX=permissive + ``` + + 2. Create the **.autorelabel** file in the root directory. + + ```shell + touch /.autorelabel + ``` + + 3. Reboot the system. The system will restart twice. + + ```shell + reboot + ``` + + 4. The status is changed successfully. + + ```shell + $ getenforce + Permissive + ``` + +- Set the enforcing mode when SELinux is disabled: + 1. Set SELinux to the permissive mode. + 2. Modify the SELinux configuration file **/etc/selinux/config** and set **SELINUX=enforcing**. + + ```shell + $ cat /etc/selinux/config | grep "SELINUX=" + SELINUX=enforcing + ``` + + 3. Reboot the system. + + ```shell + reboot + ``` + + 4. The status is changed successfully. + + ```shell + $ getenforce + Enforcing + ``` + +## SELinux Commands + +- Query the SELinux status. **SELinux status** indicates the SELinux status. **enabled** indicates that SELinux is enabled, and **disabled** indicates that SELinux is disabled. **Current mode** indicates the current mode of the SELinux. + + ```shell + $ sestatus + SELinux status: enabled + SELinuxfs mount: /sys/fs/selinux + SELinux root directory: /etc/selinux + Loaded policy name: targeted + Current mode: enforcing + Mode from config file: enforcing + Policy MLS status: enabled + Policy deny_unknown status: allowed + Memory protection checking: actual (secure) + Max kernel policy version: 31 + ``` + +## Adding Policies + +- Obtain and add missing policies based on the audit logs. (The audit service must be enabled and SELinux access denial logs must exist in audit logs.) + 1. Check whether the audit logs contain SELinux access denial logs. Use the actual audit log path. + + ```shell + grep avc /var/log/audit/audit.log* + ``` + + 2. Query missing rules. + + ```shell + audit2allow -a /var/log/audit/audit.log* + ``` + + 3. Generate a policy module based on the missing rule and name it **demo**. + + ```shell + $ audit2allow -a /var/log/audit/audit.log* -M demo + ******************** IMPORTANT *********************** + To make this policy package active, execute: + semodule -i demo.pp + ``` + + 4. Load the **demo** policy module. + + ```shell + semodule -i demo.pp + ``` + +- Compose and add the SELinux policy module. + 1. Compose the FC file (if the security context of file creation is involved). + + ```shell + $ cat demo.fc + /usr/bin/example -- system_u:object_r:example_exec_t:s0 + /resource -- system_u:object_r:resource_file_t:s0 + ``` + + 2. Compose the TE file (example). + + ```shell + $ cat demo.te + module demo 1.0; + require + { + role unconfined_r; + role system_r; + type user_devpts_t; + type root_t; + attribute file_type; + attribute domain; + class dir { getattr search add_name create open remove_name rmdir setattr write }; + class file { entrypoint execute getattr open read map setattr write create }; + class process { sigchld rlimitinh siginh transition setcap getcap }; + class unix_stream_socket { accept bind connect listen recvfrom sendto listen create lock read write getattr setattr getopt setopt append shutdown ioctl connectto }; + class capability { chown dac_override dac_read_search }; + class chr_file { append getattr ioctl read write }; + }; + role unconfined_r types example_t; + role system_r types example_t; + type example_exec_t, file_type; + type resource_file_t, file_type; + type example_t, domain; + allow example_t user_devpts_t : chr_file { append getattr ioctl read write }; + allow example_t file_type : dir { getattr search }; + allow example_t example_exec_t : file { entrypoint execute getattr map open read }; + allow domain example_exec_t : file { execute getattr map open read }; + allow example_t example_exec_t : process { sigchld }; + allow domain example_t : process { rlimitinh siginh transition }; + allow example_t resource_file_t : file { create getattr open read setattr write }; + allow example_t root_t : dir { add_name create getattr open remove_name rmdir search setattr write }; + allow example_t example_t : unix_stream_socket { accept append bind connect create getattr getopt ioctl listen listen lock read recvfrom sendto setattr setopt shutdown write }; + allow example_t domain : unix_stream_socket { connectto }; + allow example_t example_t : capability { chown dac_override dac_read_search }; + allow example_t example_t : process { getcap setcap }; + type_transition domain example_exec_t : process example_t; + type_transition example_t root_t : file resource_file_t "resource"; + ``` + + 3. Compile **demo.te** as **demo.mod**. + + ```shell + checkmodule -Mmo demo.mod demo.te + ``` + + 4. Package **demo.mod** and **demo.fc** as a policy module file. + + ```shell + semodule_package -m demo.mod -f demo.fc -o demo.pp + ``` + + 5. Load the policy module. + + ```shell + semodule -i demo.pp + ``` + + 6. Delete the loaded policy module file. + + ```shell + $ semodule -r demo + libsemanage.semanage_direct_remove_key: Removing last demo module (no other demo module exists at another priority). + ``` + +## Function Verification + +- SELinux adopts an whitelist mechanism. Modules that are not configured with proper policies may fail to run properly due to lack of permissions. It is necessary to verify the functions of the modules and configure reasonable rules. + 1. Check whether the audit service is enabled: + + ```shell + systemctl status auditd + ``` + + 2. Set the SELinux mode to permissive. (Alarms are printed, but SELinux polices are not enforced. For details, see [Configuration Description](#configuration-description).) + + ```shell + $ getenforce + Permissive + ``` + + 3. Execute all function cases of the test module and check the SELinux access denial logs in the audit logs. + + ```shell + grep avc /var/log/audit/audit.log* + ``` + + 4. Analyze access denial logs and filter out missing rules. + + ```text + type=AVC msg=audit(1596161643.271:1304): avc: denied { read } for pid=1782603 comm="smbd" name=".viminfo" dev="dm-0" ino=2488208 scontext=system_u:system_r:smbd_t:s0 tcontext=staff_u:object_r:user_home_t:s0 tclass=file permissive=1 + Indicates that the smbd process (security context: system_u:system_r:smbd_t:s0) is denied the permission to read the .viminfo file (security context: staff_u:object_r:user_home_t:s0). + permissive=1 indicates that the permissive mode is running. This log records only the operations that are not forbidden. + ``` + + 5. Supplement the missing rules by referring to [Adding Policies](#adding-policies). + +## Precautions + +- Before enabling SELinux, you are advised to upgrade selinux-policy to the latest version using DNF. Otherwise, applications may fail to run properly. For example: + + ```shell + dnf update selinux-policy -y + ``` + +- If the system cannot be started due to improper SELinux configuration (for example, a policy is deleted by mistake or no proper rule or security context is configured), you can add **selinux=0** to the startup parameters to disable SELinux. + +- After SELinux is enabled, permission check is performed on access behaviors, which affects the operating system performance to some extent (related to the frequency of access operations in the running environment). diff --git a/docs/en/server/security/secharden/system-services.md b/docs/en/server/security/secharden/system-services.md new file mode 100644 index 0000000000000000000000000000000000000000..c4a02e4ca3b238194cdb326bf7f3d39d1937c816 --- /dev/null +++ b/docs/en/server/security/secharden/system-services.md @@ -0,0 +1,455 @@ +# System Services + +## Hardening the SSH Service + +### Description + +The Secure Shell \(SSH\) is a reliable security protocol for remote logins and other network services. SSH prevents information disclosure during remote management. SSH encrypts transferred data to prevent domain name server \(DNS\) spoofing and IP spoofing. OpenSSH was created as an open source alternative to the proprietary SSH protocol. + +Hardening the SSH service is to modify configurations of the SSH service to set the algorithm and authentication parameters when the system uses the OpenSSH protocol, improving the system security. [Table 1](#en-us_topic_0152100390_ta2fdb8e4931b4c1a8f502b3c7d887b95) describes the hardening items, recommended hardening values, and default policies. + +### Implementation + +To harden a server, perform the following steps: + +1. Open the configuration file **/etc/ssh/sshd\_config** of the SSH service on the server, and modify or add hardening items and values in the file. +2. Save the **/etc/ssh/sshd\_config** file. +3. Run the following command to restart the SSH service: + + ```shell + systemctl restart sshd + ``` + +To harden a client, perform the following steps: + +1. Open the configuration file **/etc/ssh/ssh\_config** of the SSH service on the client, and modify or add hardening items and values in the file. +2. Save the **/etc/ssh/ssh\_config** file. +3. Run the following command to restart the SSH service: + + ```shell + systemctl restart sshd + ``` + +### Hardening Items + +- Server hardening policies + + All SSH service hardening items are stored in the **/etc/ssh/sshd\_config** configuration file. For details about the server hardening items, hardening suggestions, and whether the hardening items are configured as suggested, see [Table 1](#en-us_topic_0152100390_ta2fdb8e4931b4c1a8f502b3c7d887b95). + + **Table 1** SSH hardening items on a server + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Item

+

Description

+

Suggestion

+

Configured as Suggested

+

Protocol

+

SSH protocol version.

+

2

+

Yes

+

SyslogFacility

+

Log type of the SSH service. The item is set to AUTH, indicating authentication logs.

+

AUTH

+

Yes

+

LogLevel

+

Level for recording SSHD logs.

+

VERBOSE

+

Yes

+

X11Forwarding

+

Specifies whether a GUI can be used after login using SSH.

+

no

+

Yes

+

MaxAuthTries

+

Maximum number of authentication attempts.

+

3

+

No

+

PubkeyAuthentication

+

Specifies whether public key authentication is allowed.

+

yes

+

Yes

+

RSAAuthentication

+

Specifies whether only RSA security authentication is allowed.

+

yes

+

Yes

+

IgnoreRhosts

+

Specifies whether the rhosts and shosts files are used for authentication. The rhosts and shosts files record the names of the servers that support remote access and related login names.

+

yes

+

Yes

+

RhostsRSAAuthentication

+

Specifies whether the RSA algorithm security authentication based on the rhosts file is used. The rhosts file records the names of the servers that support remote access and related login names.

+

no

+

Yes

+

HostbasedAuthentication

+

Specifies whether host-based authentication is used. Host-based authentication indicates that any user of a trusted client can use the SSH service.

+

no

+

Yes

+

PermitRootLogin

+

Specifies whether to allow user root to log in to the system using SSH.

+
NOTE:

If you want to log in to the system using SSH as user root, set the value of the PermitRootLogin field in the /etc/ssh/sshd_config file to yes.

+
+

no

+

No

+

PermitEmptyPasswords

+

Specifies whether accounts with empty passwords can log in.

+

no

+

Yes

+

PermitUserEnvironment

+

Specifies whether to resolve the environment variables set in ~/.ssh/environment and ~/.ssh/authorized_keys.

+

no

+

Yes

+

Ciphers

+

Encryption algorithm of SSH data transmission.

+

aes128-ctr,aes192-ctr,aes256-ctr,chacha20-poly1305@openssh.com,aes128-gcm@openssh.com,aes256-gcm@openssh.com

+

Yes

+

ClientAliveCountMax

+

Timeout count. After the server sends a request, if the number of times that the client does not respond reaches a specified value, the server automatically disconnects from the client.

+

0

+

No

+

Banner

+

File of the prompt information displayed before and after SSH login.

+

/etc/issue.net

+

Yes

+

MACs

+

Hash algorithm for SSH data verification.

+

hmac-sha2-512,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha2-256-etm@openssh.com

+

Yes

+

StrictModes

+

Specifies whether to check the permission on and ownership of the home directory and rhosts file before SSH receives login requests.

+

yes

+

Yes

+

UsePAM

+

Specifies whether to use PAM for login authentication.

+

yes

+

Yes

+

AllowTcpForwarding

+

Specifies whether to allow TCP forwarding.

+

no

+

Yes

+

Subsystem sftp /usr/libexec/openssh/sftp-server

+

SFTP log record level, which records the INFO level and authentication logs.

+

-l INFO -f AUTH

+

Yes

+

AllowAgentForwarding

+

Specifies whether to allow SSH Agent forwarding.

+

no

+

Yes

+

GatewayPorts

+

Specifies whether SSH can connect to ports on the forwarding client.

+

no

+

Yes

+

PermitTunnel

+

Specifies whether Tunnel devices are allowed.

+

no

+

Yes

+

KexAlgorithms

+

SSH key exchange algorithms.

+

curve25519-sha256,curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256

+

Yes

+

LoginGraceTime

+

Time limit for users passing the authentication. 0 indicates no limit. The default value is 60 seconds.

+

60

+

No

+
+ + > [!NOTE]NOTE + > By default, the messages displayed before and after SSH login are saved in the **/etc/issue.net** file. The default information in the **/etc/issue.net** file is **Authorized users only.** **All activities may be monitored and reported.** + +- Client hardening policies + + All SSH service hardening items are stored in the **/etc/ssh/ssh\_config** configuration file. For details about the client hardening items, hardening suggestions, and whether the hardening items are configured as suggested, see [Table 2](#en-us_topic_0152100390_tb289c5a6f1c7420ab4339187f9018ea4). + + **Table 2** SSH hardening items on a client + + + + + + + + + + + + + + + + + + + +

Item

+

Description

+

Suggestion

+

Configured as Suggested

+

KexAlgorithms

+

SSH key exchange algorithms.

+

ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256

+

No

+

VerifyHostKeyDNS

+

Specifies whether to verify HostKey files by using DNS or SSHFP.

+

ask

+

No

+
+ + > [!NOTE]NOTE + > Third-party clients and servers that use the Diffie-Hellman algorithm are required to allow at least 2048-bit connection. + +### Other Security Suggestions + +- The SSH service only listens on specified IP addresses. + + For security purposes, you are advised to only listen on required IP addresses rather than listen on 0.0.0.0 when using the SSH service. You can specify the IP addresses that SSH needs to listen on in the ListenAddress configuration item in the **/etc/ssh/sshd\_config** file. + + 1. Open and modify the **/etc/ssh/sshd\_config** file. + + ```shell + vi /etc/ssh/sshd_config + ``` + + The following information indicates that the bound listening IP address is **192.168.1.100**. You can change the listening IP address based on the site requirements. + + ```text + ... + ListenAddress 192.168.1.100 + ... + ``` + + 2. Restart the SSH service. + + ```shell + systemctl restart sshd.service + ``` + +- SFTP users are restricted from access to upper-level directories. + + SFTP is a secure FTP designed to provide secure file transfer over SSH. Users can only use dedicated accounts to access SFTP for file upload and download, instead of SSH login. In addition, directories that can be accessed over SFTP are limited to prevent directory traversal attacks. The configuration process is as follows: + + > [!NOTE]NOTE + > In the following configurations, **sftpgroup** is an example user group name, and **sftpuser** is an example username. + + 1. Create an SFTP user group. + + ```shell + groupadd sftpgroup + ``` + + 2. Create an SFTP root directory. + + ```shell + mkdir /sftp + ``` + + 3. Modify the ownership of and permission on the SFTP root directory. + + ```shell + chown root:root /sftp + chmod 755 /sftp + ``` + + 4. Create an SFTP user. + + ```shell + useradd -g sftpgroup -s /sbin/nologin sftpuser + ``` + + 5. Set the password of the SFTP user. + + ```shell + passwd sftpuser + ``` + + 6. Create a directory used to store files uploaded by the SFTP user. + + ```shell + mkdir /sftp/sftpuser + ``` + + 7. Modify the ownership of and permission on the upload directory of the SFTP user. + + ```shell + chown root:root /sftp/sftpuser + chmod 777 /sftp/sftpuser + ``` + + 8. Modify the **/etc/ssh/sshd\_config** file. + + ```shell + vi /etc/ssh/sshd_config + ``` + + Modify the following information: + + ```text + #Subsystem sftp /usr/libexec/openssh/sftp-server -l INFO -f AUTH + Subsystem sftp internal-sftp -l INFO -f AUTH + ... + + Match Group sftpgroup + ChrootDirectory /sftp/%u + ForceCommand internal-sftp + ``` + + > [!NOTE]NOTE + - **%u** is a wildcard character. Enter **%u** to represent the username of the current SFTP user. + - The following content must be added to the end of the **/etc/ssh/sshd\_config** file: + + ```text + Match Group sftpgroup + ChrootDirectory /sftp/%u + ForceCommand internal-sftp + ``` + + 9. Restart the SSH service. + + ```shell + systemctl restart sshd.service + ``` + +- Remotely execute commands using SSH. + + When a command is executed remotely through OpenSSH, TTY is disabled by default. If a password is required during command execution, the password is displayed in plain text. To ensure password input security, you are advised to add the **-t** option to the command. Example: + + ```shell + ssh -t testuser@192.168.1.100 su + ``` + + > [!NOTE]NOTE + > **192.168.1.100** is an example IP address, and **testuser** is an example username. diff --git a/docs/en/server/security/shangmi/_toc.yaml b/docs/en/server/security/shangmi/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..19b8b91d715c8da4394172a2d6665796e2791db0 --- /dev/null +++ b/docs/en/server/security/shangmi/_toc.yaml @@ -0,0 +1,25 @@ +label: ShangMi Overview +isManual: true +href: ./overview.md +description: Enable SM algorithms for key OS security features. +sections: + - label: Drive Encryption + href: ./drive-encryption.md + - label: Kernel Module Signing + href: ./kernel-module-signing.md + - label: Algorithm Library + href: ./algorithm-library.md + - label: File Integrity Protection + href: ./file-integrity-protection.md + - label: User Identity Authentication + href: ./user-identity-authentication.md + - label: Certificates + href: ./certificates.md + - label: Secure Boot + href: ./secure-boot.md + - label: SSH Stack + href: ./ssh-stack.md + - label: TLCP Stack + href: ./tlcp-stack.md + - label: RPM Signature Verification + href: ./rpm-signature-verification.md diff --git a/docs/en/server/security/shangmi/algorithm-library.md b/docs/en/server/security/shangmi/algorithm-library.md new file mode 100644 index 0000000000000000000000000000000000000000..ca377c053521b70466745a8961caedfe461ff54f --- /dev/null +++ b/docs/en/server/security/shangmi/algorithm-library.md @@ -0,0 +1,194 @@ +# Algorithm Library + +## OpenSSL Cryptographic Interface + +OpenSSL is a common cryptographic algorithm library software that supports SM2, SM3, and SM4 algorithms. You can invoke the encryption and decryption functions of SM series cryptographic algorithms through command lines or APIs. + +### Prerequisites + +OpenSSL 1.1.1m-6 or later + +```shell +$ rpm -qa openssl +openssl-1.1.1m-6.oe2209.x86_64 +``` + +### How to Use + +#### Scenario 1: Using the CLI to Call Cryptographic Algorithms + +1. SM2 public key algorithm + + Generate an SM2 private key. + + ```shell + openssl ecparam -genkey -name SM2 -out priv.key + ``` + + Generate a public key based on the private key. + + ```shell + $ openssl ec -in priv.key -pubout -out pub.key + read EC key + writing EC key + ``` + + Use the SM2 algorithm to sign the file and set the message digest algorithm to SM3. + + ```shell + openssl dgst -sm3 -sign priv.key -out data.sig data + ``` + + Use the public key to verify the signature. + + ```shell + $ openssl dgst -sm3 -verify pub.key -signature data.sig data + Verified OK + ``` + +2. SM3 message digest algorithm + + Use the SM3 algorithm for data digest. + + ```shell + $ openssl dgst -sm3 data + SM3(data)= a794922bb9f0a034257f6c7090a3e8429801a42d422c21f1473e83b7f7eac385 + ``` + +3. SM4 symmetric cipher algorithm + + Use the SM4 algorithm to encrypt data. **-K** and **-iv** specify the key value and IV value used for encryption, respectively. Generally, the key value and IV value need to be randomly generated. + + ```shell + openssl enc -sm4 -in data -K 123456789ABCDEF0123456789ABCDEF0 -iv 123456789ABCDEF0123456789ABCDEF0 -out data.enc + ``` + + Use the SM4 algorithm to decrypt data. + + ```shell + openssl enc -d -sm4 -in data.enc -K 123456789ABCDEF0123456789ABCDEF0 -iv 123456789ABCDEF0123456789ABCDEF0 -out data.raw + ``` + + Compare the encrypted and decrypted data. The results are consistent. + + ```shell + diff data data.raw + ``` + +#### Scenario 2: Using APIs to Call Cryptographic Algorithms + +You can directly install openssl-help and query the **man** manual. + +```shell +yum install openssl-help +man sm2 +man EVP_sm3 +man EVP_sm4_cbc +``` + +## Kernel Cryptographic Interface + +### Overview + +The cryptographic algorithms of the Linux kernel is managed by the crypto framework. Different algorithm implementations can be registered and invoked in the crypto framework. Kernel 5.10 provided by openEuler supports ShangMi (SM) series cryptographic algorithms (SM2, SM3, and SM4). The SM2 and SM3 algorithms are compiled in the kernel by default, and the SM4 algorithm is provided as a kernel module. + +### Prerequisites + +Kernel 5.10.0-106 or later + +```shell +# rpm -qa kernel +kernel-5.10.0-106.1.0.55.oe2209.x86_64 +``` + +### How to Use + +#### Scenario 1: Querying the Cryptographic Algorithms Supported by the Kernel + +Use **/proc/crypto** to query the registered SM series cryptographic algorithms. By default, the SM2 and SM3 algorithms are loaded. + +```shell +$ cat /proc/crypto | grep sm3 -A8 +name : sm3 +driver : sm3-generic +module : kernel +priority : 100 +refcnt : 1 +selftest : passed +internal : no +type : shash +blocksize : 64 +digestsize : 32 + +$ cat /proc/crypto | grep sm2 -A6 +name : sm2 +driver : sm2-generic +module : kernel +priority : 100 +refcnt : 1 +selftest : passed +internal : no +type : akcipher +``` + +By default, the SM4 algorithm is not loaded. You need to insert the corresponding module first. + +```shell +$ modprobe sm4-generic +$ cat /proc/crypto | grep sm4 -A8 +name : sm4 +driver : sm4-generic +module : sm4_generic +priority : 100 +refcnt : 1 +selftest : passed +internal : no +type : cipher +blocksize : 16 +min keysize : 16 +max keysize : 16 +``` + +#### Scenario 2: Calling Algorithm APIs + +The method of calling SM series cryptographic algorithms is the same as that of calling other algorithms of the same type. For details, see the Linux kernel document. + +```text +https://www.kernel.org/doc/html/v5.10/crypto/userspace-if.html +``` + +#### Scenario 3: Optimizing Algorithm Performance Through Instruction Sets + +The crypto framework allows registration of algorithm implementations related to the architecture. The algorithm performance can be optimized through a specific instruction set. Currently, the kernel 5.10 of openEuler supports algorithm performance optimization using the following instruction sets. + +| Driver | Instruction Set | Priority| +| -------------------------------- | ---------------------- | ------ | +| sm4-neon(ecb/cbc/cfb/ctr) | ARM64-NEON | 200 | +| sm3-avx | x86-AVX | 300 | +| sm4-aesni-avx (ecb/cbc/cfb/ctr) | x86-AVX | 400 | +| sm4-aesni-avx 2(ecb/cbc/cfb/ctr) | x86-AVX2 | 500 | + +When multiple instances of the same algorithm are registered, the default algorithm is selected based on the registered priority of each algorithm instance. A larger **priority** value indicates a higher priority. The priority of a pure software algorithm (with the suffix **-generic**) is fixed to **100**. By default, the performance optimization through instruction sets is disabled for the SM series cryptographic algorithms and is provided for users in the form of a kernel module. For example, to enable the AVX instruction set optimization of the SM3 algorithm, do as follows: + +```shell +$ modprobe sm3-avx +$ cat /proc/crypto | grep sm3 -A8 +name : sm3 +driver : sm3-avx +module : sm3_avx_x86_64 +priority : 300 +refcnt : 1 +selftest : passed +internal : no +type : shash +blocksize : 64 +digestsize : 32 + +...... +``` + +#### Notes + +1. The prerequisite for enabling algorithm instruction set optimization is that the CPU supports the corresponding instruction set. You can query the instruction set supported by the CPU by calling **/proc/cpuinfo**. +2. Calling a specific instruction set has certain overhead. Therefore, it cannot be ensured that the performance optimized by the instruction set is higher than that of software implementation in all scenarios. +3. The optimization through some instruction sets has certain restrictions. For example, the NEON instruction set has optimization effect only in the encryption mode that supports parallel computing. diff --git a/docs/en/server/security/shangmi/certificates.md b/docs/en/server/security/shangmi/certificates.md new file mode 100644 index 0000000000000000000000000000000000000000..559afd304eab48f4aa81dcabd5d974d05b910925 --- /dev/null +++ b/docs/en/server/security/shangmi/certificates.md @@ -0,0 +1,82 @@ +# Certificates + +## Overview + +A ShangMi (SM) certificate is a digital certificate that complies with standards such as *Public Key Cryptographic Algorithm SM2 Based on Elliptic Curves* and *Digital Certificate Format Based on SM2 Algorithm*. The OpenSSL software released by openEuler supports SM certificates. + +## Prerequisites + +OpenSSL 1.1.1m-6 or later + +```shell +$ rpm -qa openssl +openssl-1.1.1m-6.oe2209.x86_64 +``` + +## How to Use + +### Scenario 1: Generating an SM Certificate + +1. Generate a private key for SM2 signing. + + ```shell + openssl ecparam -genkey -name SM2 -out sm2.key + ``` + +2. Generate a signing request. + + ```shell + openssl req -new -sm3 -key sm2.key -out sm2.csr + ``` + +3. Generate an SM certificate. You can use **-extfile** to specify the certificate configuration file. + + ```shell + openssl x509 -req -days 3650 -signkey sm2.key -in sm2.csr -out sm2.crt + ``` + +4. View the certificate information. + + ```shell + openssl x509 -text -in sm2.crt + ``` + +### Scenario 2: Building a Certificate Chain + +#### Using the x509 Command (Generally Used for Function Tests) + +1. Generate a private key for SM2 signing and a signing request. + + ```shell + openssl ecparam -genkey -name SM2 -out sm2.key + openssl req -new -sm3 -key sm2.key -out sm2.csr + ``` + +2. Generate a level-2 certificate based on the level-1 certificate. You can use **-extfile** to specify the certificate configuration file. + + ```shell + openssl x509 -req -sm3 -CAcreateserial -CA ca.crt -CAkey ca.key -in sm2.csr -out sm2.crt + ``` + +#### Using the CA Configuration File (Generally Used in Formal Scenarios) + +1. Prepare the configuration file for generating the certificate. You can use **apps/openssl.cnf** in the OpenSSL source code directory. + +2. Generate a self-signed CA certificate. (The following commands use OpenSSL 1.1.1 as an example. In OpenSSL 3.0.0 and later, change the value of the `-newkey` option to **sm2:SM2.pem**.) + + ```shell + openssl ecparam -name SM2 -out SM2.pem + openssl req -config ./openssl.cnf -nodes -keyout CA.key -newkey ec:SM2.pem -new -out CA.csr + openssl x509 -sm3 -req -days 30 -in CA.csr -extfile ./openssl.cnf -extensions v3_ca -signkey CA.key -out CA.crt + ``` + +3. Generate a level-2 certificate. (The following commands use OpenSSL 1.1.1 as an example. In OpenSSL 3.0.9 and later, change the value of the `-newkey` option to **sm2:SM2.pem**.) + + ```shell + openssl req -config ./openssl.cnf -nodes -keyout SS.key -newkey ec:SM2.pem -new -out SS.csr + openssl x509 -sm3 -req -days 30 -in SS.csr -CA CA.crt -CAkey CA.key -extfile ./openssl.cnf -extensions v3_req -out SS.crt -CAcreateserial + ``` + +### Scenario 3: Generating a TLCP Communication Certificate + +For details, see chapter "TLCP Stack." diff --git a/docs/en/server/security/shangmi/drive-encryption.md b/docs/en/server/security/shangmi/drive-encryption.md new file mode 100644 index 0000000000000000000000000000000000000000..4e260107885b0559064c3c6f6bf823110e3b26a8 --- /dev/null +++ b/docs/en/server/security/shangmi/drive-encryption.md @@ -0,0 +1,90 @@ +# Drive Encryption + +## Overview + +Drive encryption protects the storage confidentiality of important data. Data is encrypted based on a specified encryption algorithm and then written to drives. This feature mainly involves the user-mode tool cryptsetup and the kernel-mode module dm-crypt. Currently, the drive encryption feature provided by the openEuler OS supports ShangMi (SM) series cryptographic algorithms. Parameters are as follows: + +- Encryption modes: luks2 and plain; +- Key length: 256 bits; +- Message digest algorithm: SM3; +- Encryption algorithm: sm4-xts-plain64. + +## Prerequisites + +1. Kernel 5.10.0-106 or later + + ```shell + $ rpm -qa kernel + kernel-5.10.0-106.1.0.55.oe2209.x86_64 + ``` + +2. cryptsetup 2.4.1-1 or later + + ```shell + $ rpm -qa cryptsetup + cryptsetup-2.4.1-1.oe2209.x86_64 + ``` + +## How to Use + +A drive is formatted in a specified encryption mode and mapped to **/dev/mapper** as a dm device. Subsequent drive read and write operations are performed through the dm device. Data encryption and decryption are performed in kernel mode and are not perceived by users. The procedure is as follows: + +1. Format the drive and map the drive as a dm device. + + a. luks2 mode + + Set the encryption mode to luks2, encryption algorithm to sm4-xts-plain64, key length to 256 bits, and message digest algorithm to SM3. + + ```shell + # cryptsetup luksFormat /dev/sdd -c sm4-xts-plain64 --key-size 256 --hash sm3 + # cryptsetup luksOpen /dev/sdd crypt1 + ``` + + b. plain mode + + Set the encryption mode to plain, encryption algorithm to sm4-xts-plain64, key length to 256 bits, and message digest algorithm to SM3. + + ```shell + # cryptsetup plainOpen /dev/sdd crypt1 -c sm4-xts-plain64 --key-size 256 --hash sm3 + ``` + +2. After the mapping is successful, run the **lsblk** command to view the device information. + + ```shell + # lsblk + NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS + ...... + sdd 8:48 0 50G 0 disk + └─crypt1 253:3 0 50G 0 crypt + ...... + ``` + +3. Perform I/O read and write operations on the encrypted device. + + Deliver I/Os to raw drives. + + ```shell + # dd if=/dev/random of=/dev/mapper/crypt1 bs=4k count=10240 + ``` + + Deliver I/Os through the file system. + + ```shell + # mkfs.ext4 /dev/mapper/crypt1 + # mount /dev/mapper/crypt1 /mnt/crypt/ + # dd if=/dev/random of=/mnt/crypt/tmp bs=4k count=10240 + ``` + +4. Disable device mapping. + + If a file system is mounted, unmount it first. + + ```shell + # umount /mnt/crypt + ``` + + Closes a device. + + ```shell + # cryptsetup close crypt1 + ``` diff --git a/docs/en/server/security/shangmi/file-integrity-protection.md b/docs/en/server/security/shangmi/file-integrity-protection.md new file mode 100644 index 0000000000000000000000000000000000000000..409b28e2e2c57638c9b5e48390a0ccd0482bf18c --- /dev/null +++ b/docs/en/server/security/shangmi/file-integrity-protection.md @@ -0,0 +1,512 @@ +# File Integrity Protection + +## Kernel Integrity Measurement Architecture (IMA) + +IAM is a mandatory access control subsystem provided by the Linux kernel. It measures and verifies file integrity by adding check hooks to file access system calls. + +### Prerequisites + +1. The openEuler kernel compilation environment has been prepared. For details, see . +2. You are advised to select the latest kernel source code for compilation. +3. The kernel SM2 root certificate has been generated (for appraisal mode only). + + ```sh + # Generate a certificate configuration file. (Other fields in the configuration file can be defined as required.) + echo 'subjectKeyIdentifier=hash' > ima.cfg + echo 'authorityKeyIdentifier=keyid,issuer' >> ima.cfg + echo 'keyUsage=digitalSignature,nonRepudiation' >> ima.cfg + # Generate a private key for SM2 signing. + # openssl ecparam -genkey -name SM2 -out ima.key + # Generate a signing request. + # openssl req -new -sm3 -key ima.key -out ima.csr + # Generate an SM2 certificate. + # openssl x509 -req -days 3650 -extfile ima.cfg -signkey ima.key -in ima.csr -out ima.crt + ``` + +4. The level-2 IMA certificate has been generated. + + ```sh + # Create a certificate configuration file. + echo 'subjectKeyIdentifier=hash' > ima.cfg + echo 'authorityKeyIdentifier=keyid,issuer' >> ima.cfg + # Generate a private key. + openssl ecparam -genkey -name SM2 -out ima.key + # Generate a signing request. + openssl req -new -sm3 -key ima.key -out ima.csr + # Generate a level-2 certificate based on the level-1 certificate. + openssl x509 -req -sm3 -CAcreateserial -CA ca.crt -CAkey ca.key -extfile ima.cfg -in ima.csr -out ima.crt + # Convert to the DER format. + openssl x509 -outform DER -in ima.crt -out x509_ima.der + ``` + +5. The root certificate has been placed in the kernel source code directory, and **CONFIG_SYSTEM_TRUSTED_KEYS** has been modified to compile the certificate to the kernel trusted key. + + ```sh + $ cp /path/to/ca.crt . + $ make openeuler_defconfig + $ cat .config | grep CONFIG_SYSTEM_TRUSTED_KEYS + CONFIG_SYSTEM_TRUSTED_KEYS="ca.crt" + ``` + +6. The kernel has been compiled and installed. + +```sh +make -j64 +make modules_install +make install +``` + +### Usage + +#### Scenario 1: Native IMA + +##### IMA-Measurement + +Configure the IMA policy and message digest algorithm, disable the IMA-appraisal mode, and restart the system. + +```sh +ima_policy=tcb ima_hash=sm3 ima_appraise=off +``` + +Check the measurement log. It is found that the IMA measures all protected files and the message digest algorithm is SM3. + +```sh +cat /sys/kernel/security/ima/ascii_runtime_measurements +10 601989730f01fb4688bba92d0ec94340cd90757f ima-sig sm3:0000000000000000000000000000000000000000000000000000000000000000 boot_aggregate +10 dc0a98316b03ab15edd2b8daae75a0d64bca7c56 ima-sig sm3:3c62ee3c13ee32d7a287e04c843c03ebb428a5bb3dd83561efffe9b08444be22 /usr/lib/systemd/systemd +10 1d0a5140e3924e2542963ad887a80def0aa8acac ima-sig sm3:4d3b83e143bd9d5288ef099eff4d01444947516d680165c6dd08cd5900768032 /usr/lib64/ld-linux-x86-64.so.2 +...... +``` + +##### IMA-Appraisal (Hash) + +Configure the IMA policy and message digest algorithm, enable the IMA-appraisal fix mode, and restart the system. + +```sh +ima_policy=appraise_tcb ima_hash=sm3 ima_appraise=fix +``` + +**appraise_tcb** indicates to appraise all files owned by the **root** user. + +Perform an open operation on all files to be appraised to automatically mark the .ima extension. + +```sh +find / -fstype ext4 -type f -uid 0 -exec dd if='{}' of=/dev/null count=0 status=none \; +``` + +After the marking is complete, you can see that all files with the .ima extension of the SM3 message digest algorithm are marked. + +```sh +getfattr -m - -d -e hex /bin/bash +getfattr: Removing leading '/' from absolute path names +# file: bin/bash +security.ima=0x0411a794922bb9f0a034257f6c7090a3e8429801a42d422c21f1473e83b7f7eac385 +security.selinux=0x73797374656d5f753a6f626a6563745f723a7368656c6c5f657865635f743a733000 + +openssl dgst -sm3 /bin/bash +SM3(/bin/bash)= a794922bb9f0a034257f6c7090a3e8429801a42d422c21f1473e83b7f7eac385 +``` + +Enable the enforce mode and restart the system. The system can run properly. + +```sh +ima_policy=appraise_tcb ima_hash=sm3 ima_appraise=enforce +``` + +##### IMA-Appraisal (Signing) + +**Prerequisites** + +1. The SM root certificate has been preset in the kernel. +2. The ima-evm-utils software package whose version is later than or equal to the specified version has been installed. + +```sh +$ rpm -qa ima-evm-utils +ima-evm-utils-1.3.2-4.oe2209.x86_64 +``` + +Generate a level-2 IMA certificate. + +```sh +# Create a certificate configuration file. +echo 'subjectKeyIdentifier=hash' > ima.cfg +echo 'authorityKeyIdentifier=keyid,issuer' >> ima.cfg +# Generate a private key. +openssl ecparam -genkey -name SM2 -out ima.key +# Generate a signing request. +openssl req -new -sm3 -key ima.key -out ima.csr +# Generate a level-2 certificate based on the level-1 certificate. +openssl x509 -req -sm3 -CAcreateserial -CA ca.crt -CAkey ca.key -extfile ima.cfg -in ima.csr -out ima.crt +# Convert to the DER format. +openssl x509 -outform DER -in ima.crt -out x509_ima.der +``` + +Place the IMA certificate in the **/etc/keys** directory and run the **dracut** command to create initrd again. + +```sh +mkdir -p /etc/keys +cp x509_ima.der /etc/keys +echo 'install_items+=" /etc/keys/x509_ima.der "' >> /etc/dracut.conf +dracut -f +``` + +Sign the files to be protected. For example, sign all executable files of the **root** user in the **/usr/bin** directory. + +```sh +find /usr/bin -fstype ext4 -type f -executable -uid 0 -exec evmctl -a sm3 ima_sign --key /path/to/ima.key '{}' \; +``` + +Enable the enforce mode and restart the system. The system can run properly. + +```sh +ima_policy=appraise_tcb ima_hash=sm3 ima_appraise=enforce +``` + +Check the protection effect of the signing mode. + +```sh +# getfattr -m - -d /bin/echo +getfattr: Removing leading '/' from absolute path names +# file: bin/echo +security.ima=0sAwIRNJFkBQBIMEYCIQDLBg/bYlrkBqSaXNQMyK7rhiZj+qRiKdu+0fqW8lSmPQIhAJY2qSZJ0HgSu7kygydrS4MCC0KTK59nUkvISenZAUCo +security.selinux="system_u:object_r:bin_t:s0" + +# echo 123 >> /bin/echo +-bash: /bin/echo: Permission denied +``` + +**Note:** + +For each file under the IMA protection, you need to use either the hash mode or the signing mode to mark the integrity information. Generally, files that may change (such as data files and configuration files) are marked in hash mode, and files that do not change (such as executable files and dynamic link libraries) are marked in signing mode. + +#### Scenario 2: IMA Digest Lists Mode + +##### Generating SM3 Digest Lists + +Configure **-a sm3** when calling **gen_digest_lists** to generate SM3 digest lists. + +```sh +gen_digest_lists -a sm3 -t metadata -f compact -i l:policy -o add -p -1 -m immutable -i I:/usr/bin/bash -d -i i: +gen_digest_lists -a sm3 -t metadata -f compact -i l:policy -o add -p -1 -m immutable -i I:/usr/bin/bash -d -i i: -T +``` + +##### Configuring the SM3 Message Digest Algorithm + +The overall procedure is the same as that for enabling the IMA Digest Lists feature. The only difference is that the **ima_hash** startup parameter is set to **sm3**. The following gives an example: + +```sh +# Log mode +ima_template=ima-sig ima_policy="exec_tcb|appraise_exec_tcb|appraise_exec_immutable" initramtmpfs ima_hash=sm3 ima_appraise=log evm=allow_metadata_writes evm=x509 ima_digest_list_pcr=11 ima_appraise_digest_list=digest +# enforce mode +ima_template=ima-sig ima_policy="exec_tcb|appraise_exec_tcb|appraise_exec_immutable" initramtmpfs ima_hash=sm3 ima_appraise=enforce-evm evm=allow_metadata_writes evm=x509 ima_digest_list_pcr=11 ima_appraise_digest_list=digest +``` + +After the configuration is complete, restart the system and query the measurement log. The default algorithm in the measurement log is changed to SM3. + +```sh +$ cat /sys/kernel/security/ima/ascii_runtime_measurements +...... +11 9e32183b5b1da72c6ff4298a44026e3f9af510c9 ima-sig sm3:5a2d81cd135f41e73e0224b9a81c3d8608ccde8caeafd5113de959ceb7c84948 /usr/bin/upload_digest_lists +11 f3b9264761dbaeaf637d08b86cc3655e8f3380f7 ima-sig sm3:cc6faecee9976c12279dab1627a78ef36f6998c65779f3b846494ac5fe5493a1 /usr/libexec/rpm_parser +11 dc0a98316b03ab15edd2b8daae75a0d64bca7c56 ima-sig sm3:3c62ee3c13ee32d7a287e04c843c03ebb428a5bb3dd83561efffe9b08444be22 /usr/lib/systemd/systemd +...... +``` + +##### Verifying Digest Lists Using the SM2 Certificates + +**Prerequisites** + +1. The SM root certificate has been preset in the kernel. +2. The digest-list-tools and ima-evm-utils software packages of the specified versions or later have been installed. + +```sh +$ rpm -qa ima-evm-utils +ima-evm-utils-1.3.2-4.oe2209.x86_64 +$ rpm -qa digest-list-tools +digest-list-tools-0.3.95-10.oe2209.x86_64 +``` + +**Procedure** + +1. Generate level-2 IMA and EVM certificates (sub-certificates of the SM root certificate preset in the kernel). + + ```sh + # Create a certificate configuration file. + echo 'subjectKeyIdentifier=hash' > ima.cfg + echo 'authorityKeyIdentifier=keyid,issuer' >> ima.cfg + # Generate a private key. + openssl ecparam -genkey -name SM2 -out ima.key + # Generate a signing request. + openssl req -new -sm3 -key ima.key -out ima.csr + # Generate a level-2 certificate based on the level-1 certificate. + openssl x509 -req -sm3 -CAcreateserial -CA ca.crt -CAkey ca.key -extfile ima.cfg -in ima.csr -out ima.crt + # Convert to the DER format. + openssl x509 -outform DER -in ima.crt -out x509_ima.der + openssl x509 -outform DER -in ima.crt -out x509_evm.der + ``` + +2. Place the IMA and EVM certificates in the **/etc/keys** directory and run the **dracut** command to create initrd again. + + ```sh + mkdir -p /etc/keys + cp x509_ima.der /etc/keys + cp x509_evm.der /etc/keys + echo 'install_items+=" /etc/keys/x509_ima.der /etc/keys/x509_evm.der "' >> /etc/dracut.conf + dracut -f -e xattr + ``` + +3. Enable the IMA Digest Lists function. After the restart, check whether the certificates are imported to the IMA and EVM key rings. + + ```sh + $ cat /proc/keys + ...... + 024dee5e I------ 1 perm 1f0f0000 0 0 keyring .evm: 1 + ...... + 3980807f I------ 1 perm 1f0f0000 0 0 keyring .ima: 1 + ...... + ``` + +4. Sign the IMA digest lists using the private keys corresponding to the IMA and EVM certificates. The signed IMA digest lists can be imported to the kernel. + + ```sh + # Use **evmctl** to sign the digest lists. + evmctl ima_sign --key /path/to/ima.key -a sm3 0-metadata_list-compact-tree-1.8.0-2.oe2209.x86_64 + # Check the extension after signing. + getfattr -m - -d 0-metadata_list-compact-tree-1.8.0-2.oe2209.x86_64 + # file: 0-metadata_list-compact-tree-1.8.0-2.oe2209.x86_64 + security.ima=0sAwIRNJFkBQBHMEUCIQCzdKVWdxw1hoVm9lgZB6sl+sxapptUFNjqHt5XZD87hgIgBMuZqBdrcNm7fXq/reQw7rzY/RN/UXPrIOxrVvpTouw= + security.selinux="unconfined_u:object_r:admin_home_t:s0" + # Import the signed digest lists file to the kernel. + echo /root/tree/etc/ima/digest_lists/0-metadata_list-compact-tree-1.8.0-2.oe2209.x86_64 > /sys/kernel/security/ima/digest_list_data + # Check the measurement log. You can view the import record of the digest lists. + cat /sys/kernel/security/ima/ascii_runtime_measurements + 11 43b6981f84ba2725d05e91f19577cedb004adffb ima-sig sm3:b9430bbde2b7f30e935d91e29ab6778b6a825a2c3e5e7255895effb8747b7c1a /root/tree/etc/ima/digest_lists/0-metadata_list-compact-tree-1.8.0-2.oe2209.x86_64 0302113491640500473045022100b374a556771c35868566f6581907ab25facc5aa69b5414d8ea1ede57643f3b86022004cb99a8176b70d9bb7d7abfade430eebcd8fd137f5173eb20ec6b56fa53a2ec + ``` + +**Notes:** + +1. By default, the hash algorithm used in the digest lists provided by openEuler is SHA256. When the IMA digest lists measurement algorithm is set to SM3, you must remove the digest lists from the **/etc/ima/digest_lists** directory, generate new digest lists, and sign the digest lists. Otherwise, an error occurs during file integrity check. The procedure is as follows: + + ```sh + # Reset the SELinux tag of a disk (perform this operation when IMA extension verification and SELinux are enabled). + fixfiles -F restore + # Generate digest lists for all files (you can also specify the files). + gen_digest_lists -a sm3 -t metadata -f compact -i l:policy -o add -p -1 -m immutable -i I:/ -d /etc/ima/digest_lists -i i: + # (Optional) Change the name of the generated digest lists file. + mv /etc/ima/digest_lists/0-metadata_list-compact- /etc/ima/digest_lists/0-metadata_list-compact-everything-sm3 + # Sign + evmctl ima_sign --key /path/to/ima.key -a sm3 /etc/ima/digest_lists/0-metadata_list-compact-everything-sm3 + # Re-create initrd. + dracut -f -e xattr + ``` + +2. For the software packages released by openEuler, the IMA digest lists file is provided by default. The default message digest algorithm is SHA256. Therefore, when the message digest algorithm is changed to SM3, the digest lists released by openEuler cannot be imported, and the following message may be displayed during software package installation: + + ```text + Cannon parse /etc/ima/digest_lists/0-metadata_list-rpm-...... + ``` + +# Lightweight Intrusion Detection (AIDE) + +AIDE is a lightweight intrusion detection tool. It checks file integrity to detect malicious intrusions to the system in a timely manner. The AIDE database can use hash algorithms such as SHA256 and SHA512 to create a verification code or hash value for each file in ciphertext. The AIDE provided by openEuler supports the SM3 algorithm in addition to the open source software. + +## Prerequisites + +AIDE 0.17.4-1 or later + +```sh +$ rpm -qa aide +aide-0.17.4-1.oe2209.x86_64 +``` + +## Usage + +Add the SM3 algorithm to the **/etc/aide.conf** configuration file. + +```text +...... +FIPSR = p+i+n+u+g+s+m+c+acl+selinux+xattrs+sha256+sm3 +...... +DATAONLY = p+n+u+g+s+acl+selinux+xattrs+sha256+sm3 +...... +``` + +Initialize the database and save the database as the benchmark. + +Initialize the database. + +```sh +aide -c /etc/aide.conf -i +``` + +The example output is as follows: + +```text +AIDE initialized database at /var/lib/aide/aide.db.new.gz + +Number of entries: 64249 + +--------------------------------------------------- +The attributes of the (uncompressed) database(s): +--------------------------------------------------- + +/var/lib/aide/aide.db.new.gz + MD5 : a7y5ErdpBAezV2iGdaVleg== + SHA1 : u7W7jxomFtZn8rwMlkIRCN0r7iQ= + SHA256 : 88Kw5b2yJ9bejwO+NqT6lyAieno+K0+W + BPVBjXcUl08= + SHA512 : WyOIgRxk9SeSoktF6BYVV0tRL7nGNDKQ + A9QyxVCgzg+PwPMV7tzxmwOZI/dB64pP + vQ/D2jqJdf3NS2PHMI4yvg== + RMD160 : qTEPs2SIxPm3iEwsCnwvp9hR4s4= + TIGER : 0HgLucmhCcB56bxOMj+j1Kuja8UIsFRg + CRC32 : VKE1TA== + WHIRLPOOL : JSA35/NmkMOkUWEpcZJf3PR1UUz5WcLG + AmBKPkao3fzQUsLMTJizCV4CwAE0G/Yc + KX0mpW5vx+gk3njya8rAvA== + GOST : yKjiytOwRr3bJcFsxnJ310t1FY6YE3HB + YNT8XP93xpc= + STRIBOG256: 9bzS+5j4ZAoU/P7v3tkKOWn4ZfggcX28 + 9dLQVhaiJtQ= + STRIBOG512: 9LLXgqsRIRiXP2WOrOJt1qhx6psfbACd + un+GTVmu441quX4zaaPIIG9lzDMBAqMg + hZx5DlxsQj3YjMezSUsXLg== + SM3 : Vwii+uw3Ge5Hh3eo1KOombxn2jWgyYRX + ZdyCRZqWZ/E= + + +End timestamp: 2022-08-12 09:01:25 +0800 (run time: 2m 43s) +``` + +Save the database as the benchmark. + +```sh +mv /var/lib/aide/aide.db.new.gz /var/lib/aide/aide.db.gz +``` + +### Scenario 1: Detecting Changes of Protected Files + +```sh +$ aide -c /etc/aide.conf --check +--------------------------------------------------- +Detailed information about changes: +--------------------------------------------------- + +File: /boot/config-5.10.0-106.3.0.57.oe2209.aarch64 + Size : 182936 | 182938 + Mtime : 2022-08-04 08:00:00 +0800 | 2022-08-12 09:05:34 +0800 + Ctime : 2022-08-11 01:42:44 +0800 | 2022-08-12 09:05:34 +0800 + SHA256 : ae0fOzf7U+e/evTZKpk6JQa00kvSkc5J | gOlhcUgnZWhcyJYMEPxCYccXwFr9lERX + vMTX5Ysh+1k= | KK3O/ytfR/g= + SHA512 : zAPIxIAM7YvJPjwl/PH23leBb/HiO0Rf | p+WxVOZ6DX323rHDRF864w297yh7POk6 + PlRME7yvpzFZk/5BrNe2ofQWR/0sFu1m | 11dOzahlKTWpAKaexC/u+4REiCzjl1rm + JsDSy8m57wzCpJA9iUFq1g== | eb/kd3Xgp1LoKwn49mtqxw== + SM3 : CW0GnITxNeGeYOCAm4xfu78Vqm+wLp/Z | GWq/3nXL16tMYyxyFD/HTZbvJi2h+ttg + cOmXmIKJT4Q= | 6d8XmSHu26A= +``` + +### Scenario 2: Updating the Database + +Update the database. After the update, the database file is **/var/lib/aide/aide.db.new.gz**. + +```sh +$ aide -c /etc/aide.conf --update +--------------------------------------------------- +Detailed information about changes: +--------------------------------------------------- + +File: /boot/config-5.10.0-106.3.0.57.oe2209.aarch64 + Size : 182936 | 182938 + Mtime : 2022-08-04 08:00:00 +0800 | 2022-08-12 09:05:34 +0800 + Ctime : 2022-08-11 01:42:44 +0800 | 2022-08-12 09:05:34 +0800 + SHA256 : ae0fOzf7U+e/evTZKpk6JQa00kvSkc5J | gOlhcUgnZWhcyJYMEPxCYccXwFr9lERX + vMTX5Ysh+1k= | KK3O/ytfR/g= + SHA512 : zAPIxIAM7YvJPjwl/PH23leBb/HiO0Rf | p+WxVOZ6DX323rHDRF864w297yh7POk6 + PlRME7yvpzFZk/5BrNe2ofQWR/0sFu1m | 11dOzahlKTWpAKaexC/u+4REiCzjl1rm + JsDSy8m57wzCpJA9iUFq1g== | eb/kd3Xgp1LoKwn49mtqxw== + SM3 : CW0GnITxNeGeYOCAm4xfu78Vqm+wLp/Z | GWq/3nXL16tMYyxyFD/HTZbvJi2h+ttg + cOmXmIKJT4Q= | 6d8XmSHu26A= +``` + +### Scenario 3: Comparing Databases + +Configure two databases in **/etc/aide.conf**. + +```conf +# The location of the database to be read. +database_in=file:@@{DBDIR}/aide.db.gz +database_new=file:@@{DBDIR}/aide.db.new.gz +``` + +Compare the databases. + +```sh +$ aide -c /etc/aide.conf --compare +--------------------------------------------------- +Detailed information about changes: +--------------------------------------------------- + +File: /boot/config-5.10.0-106.3.0.57.oe2209.aarch64 + Size : 182936 | 182938 + Mtime : 2022-08-04 08:00:00 +0800 | 2022-08-12 09:05:34 +0800 + Ctime : 2022-08-11 01:42:44 +0800 | 2022-08-12 09:05:34 +0800 + SHA256 : ae0fOzf7U+e/evTZKpk6JQa00kvSkc5J | gOlhcUgnZWhcyJYMEPxCYccXwFr9lERX + vMTX5Ysh+1k= | KK3O/ytfR/g= + SHA512 : zAPIxIAM7YvJPjwl/PH23leBb/HiO0Rf | p+WxVOZ6DX323rHDRF864w297yh7POk6 + PlRME7yvpzFZk/5BrNe2ofQWR/0sFu1m | 11dOzahlKTWpAKaexC/u+4REiCzjl1rm + JsDSy8m57wzCpJA9iUFq1g== | eb/kd3Xgp1LoKwn49mtqxw== + SM3 : CW0GnITxNeGeYOCAm4xfu78Vqm+wLp/Z | GWq/3nXL16tMYyxyFD/HTZbvJi2h+ttg + cOmXmIKJT4Q= | 6d8XmSHu26A= + +--------------------------------------------------- +The attributes of the (uncompressed) database(s): +--------------------------------------------------- + +/var/lib/aide/aide.db.gz + MD5 : a7y5ErdpBAezV2iGdaVleg== + SHA1 : u7W7jxomFtZn8rwMlkIRCN0r7iQ= + SHA256 : 88Kw5b2yJ9bejwO+NqT6lyAieno+K0+W + BPVBjXcUl08= + SHA512 : WyOIgRxk9SeSoktF6BYVV0tRL7nGNDKQ + A9QyxVCgzg+PwPMV7tzxmwOZI/dB64pP + vQ/D2jqJdf3NS2PHMI4yvg== + RMD160 : qTEPs2SIxPm3iEwsCnwvp9hR4s4= + TIGER : 0HgLucmhCcB56bxOMj+j1Kuja8UIsFRg + CRC32 : VKE1TA== + WHIRLPOOL : JSA35/NmkMOkUWEpcZJf3PR1UUz5WcLG + AmBKPkao3fzQUsLMTJizCV4CwAE0G/Yc + KX0mpW5vx+gk3njya8rAvA== + GOST : yKjiytOwRr3bJcFsxnJ310t1FY6YE3HB + YNT8XP93xpc= + STRIBOG256: 9bzS+5j4ZAoU/P7v3tkKOWn4ZfggcX28 + 9dLQVhaiJtQ= + STRIBOG512: 9LLXgqsRIRiXP2WOrOJt1qhx6psfbACd + un+GTVmu441quX4zaaPIIG9lzDMBAqMg + hZx5DlxsQj3YjMezSUsXLg== + SM3 : Vwii+uw3Ge5Hh3eo1KOombxn2jWgyYRX + ZdyCRZqWZ/E= + +/var/lib/aide/aide.db.new.gz + MD5 : sKt4dVDKY/8A9EY/X4Ue2A== + SHA1 : hagLXMv7G+KbEKh861kjjFSYpRw= + SHA256 : HTHF7k5U294ECjCLneoZ3o8bH6PYgY5u + AzoIyCacZp4= + SHA512 : 5gWi7K/ztWMl7H+PK1doV/tWDHmaE2m/ + ndRXGR7b5J3v82Jv2HeJPoOt5A4Z/9FH + 5H+uCLYaHwRleyalyy5Wew== + RMD160 : uMM1HtAbfz+G3Y9Z+rVR4qjdqcQ= + TIGER : OTHdXNQOxnHnOl6C9M3czSC42+SeZAZA + CRC32 : T9G1Tw== + WHIRLPOOL : FRMnQ2wHgylsTmpKE8RwdUvkzXucHwu1 + W9ZkUrxoXeci2g7jIgoMmpoeDPhH73qz + nZ7fKj1lStSpiUGD5KPeWA== + GOST : haeO5dhT+t34C1GJf+2dc3q1GMN71FqB + kqoiODo+j2o= + STRIBOG256: lgZUZhhd9JfMOXgNzYptapqagwgmvdM+ + 7uWzJsmOxoY= + STRIBOG512: PA6jksprS37xQzHm1ZIvLR9ROa+FxoiF + /xbAe0pSi4lMXXzABrPKkjyK0WtjxFvx + 07Poj2iDwNNcUJWekbaEXA== + SM3 : R5/HXng5MNvrjoCh8/JzrWle1IO8ggsR + P5i2ePX5BpY= +``` diff --git a/docs/en/server/security/shangmi/kernel-module-signing.md b/docs/en/server/security/shangmi/kernel-module-signing.md new file mode 100644 index 0000000000000000000000000000000000000000..d9ced9bdcff475b349284b1f790f52fbccfe288c --- /dev/null +++ b/docs/en/server/security/shangmi/kernel-module-signing.md @@ -0,0 +1,107 @@ +# Kernel Module Signing + +## Overview + +The kernel module signing facility is an important mechanism for protecting Linux kernel security. Signature information is added to the end of the kernel module file in a certain format, and the system checks whether the signature matches the public key preset in the kernel when the kernel module is loaded. In this way, the authenticity and integrity of the kernel module file are ensured. + +## Prerequisites + +1. The openEuler kernel compilation environment has been prepared. For details, see . +2. In openEuler kernel 5.10, the ShangMi (SM) series cryptographic algorithms are supported for kernel module signing. You are advised to select the latest kernel 5.10 source code for compilation. +3. The SM2 private key and certificate used for kernel module signing have been generated. The reference commands using OpenSSL are as follows: + +```shell +# Generate a certificate configuration file. (Other fields in the configuration file can be defined as required.) +$ echo 'subjectKeyIdentifier=hash' > mod.cfg +# Generate a private key for SM2 signing. +$ openssl ecparam -genkey -name SM2 -out mod.key +# Generate a signing request. +$ openssl req -new -sm3 -key mod.key -out mod.csr +# Generate an SM2 certificate. +$ openssl x509 -req -days 3650 -extfile mod.cfg -signkey mod.key -in mod.csr -out mod.crt +``` + +## How to Use + +### Scenario 1: Automatic Signing + +Write the certificate and private key to the **mod.pem** file. + +```shell +cat /path/to/mod.key > mod.pem +cat /path/to/mod.crt >> mod.pem +``` + +Use the SM3 algorithm to sign the kernel module in the kernel compilation options. + +```shell +make openeuler_defconfig +make menuconfig +``` + +Choose **Enable loadable module support** > **Sign modules with SM3** on the GUI. + +```shell +Which hash algorithm should modules be signed with? (Sign modules with SM3) +``` + +Configure **Cryptographic API** > **Certificates for signature checking** to read the private key and certificate used for kernel signing from **mod.pem**. + +```text +(mod.pem) File name or PKCS#11 URI of module signing key +``` + +Build the kernel. + +```shell +make -j64 +make modules_install +make install +``` + +Run the **modinfo** command to check the signature information of the kernel module. + +```shell +$ modinfo /usr/lib/modules/5.10.0/kernel/crypto/sm4.ko +filename: /usr/lib/modules/5.10.0/kernel/crypto/sm4.ko +license: GPL v2 +description: Generic SM4 library +srcversion: 371050FDB8BF9878D9B5B9B +depends: +retpoline: Y +intree: Y +name: sm4 +vermagic: 5.10.0 SMP mod_unload modversions +sig_id: PKCS#7 +signer: Internet Widgits Pty Ltd +sig_key: 33:0B:96:3E:1F:C1:CA:28:98:72:F5:AE:FF:3F:A4:F3:50:5D:E1:87 +sig_hashalgo: sm3 +signature: 30:45:02:21:00:81:96:8D:40:CE:7F:7D:AE:3A:4B:CC:DC:9A:F2:B4: + 16:87:3E:C3:DC:77:ED:BC:6E:F5:D8:F3:DD:77:2B:D4:05:02:20:3B: + 39:5A:89:9D:DC:27:83:E8:D8:B4:75:86:FF:33:2B:34:33:D0:90:76: + 32:4D:36:88:84:34:31:5C:83:63:6B +``` + +### Scenario 2: Manual Signing + +Call **sign_file** in the kernel source code directory to sign the specified kernel module. + +```shell +./scripts/sign-file sm3 /path/to/mod.key /path/to/mod.crt +``` + +Other steps are the same as those in scenario 1. + +### Scenario 3: Module Loading Verification + +Add **module.sig_enforce** to the kernel startup parameters to enable forcible signature verification for the kernel module. + +```text +linux /vmlinuz-5.10.0-106.1.0.55.oe2209.x86_64 root=/dev/mapper/openeuler-root ro resume=/dev/mapper/openeuler-swap rd.lvm.lv=openeuler/root rd.lvm.lv=openeuler/swap crashkernel=512M module.sig_enforce +``` + +After the system is restarted, only the kernel modules that pass the certificate verification can be loaded. + +```shell +# insmod /usr/lib/modules/5.10.0/kernel/crypto/sm4.ko +``` diff --git a/docs/en/server/security/shangmi/overview.md b/docs/en/server/security/shangmi/overview.md new file mode 100644 index 0000000000000000000000000000000000000000..d450b81a20cfc57fdef580cfa4cb0f1a2dce3021 --- /dev/null +++ b/docs/en/server/security/shangmi/overview.md @@ -0,0 +1,29 @@ +# Overview + +ShangMi (SM) algorithms are commercial-grade cryptographic technologies. Cryptographic algorithms form the backbone of security technologies in information systems. Globally, widely adopted algorithms include RSA, AES, and SHA256. In parallel, China has developed a suite of cryptographic algorithms that cater to mainstream application scenarios. Among these, SM2, SM3, and SM4 are particularly prominent in OSs. + +| Algorithm | Publicly Available | Type | Application Scenarios | +| --------- | ------------------ | --------------------- | --------------------------------------------------------------------- | +| SM2 | Yes | Asymmetric encryption | Digital signatures, key exchange, encryption/decryption, PKI systems | +| SM3 | Yes | Hash algorithm | Integrity protection, one-way encryption, and other general scenarios | +| SM4 | Yes | Symmetric encryption | Encrypted storage, secure transmission | + +Additionally, other publicly available algorithms like SM9 and ZUC, as well as non-public algorithms such as SM1 and SM7, are part of the ecosystem. Notably, all publicly available Chinese algorithms have been integrated into ISO/IEC standards, gaining international recognition. China has also established a series of cryptographic technical specifications and application standards, including commercial cryptographic certificate standards and the TLCP protocol stack. These collectively form China's commercial cryptographic standard system, which guides the development of the cryptographic security industry. + +The SM features for the openEuler OS aims to enable SM series cryptographic algorithms for key security features of the OS and provide cryptographic services such as the SM series cryptographic algorithm library, certificates, and secure transmission protocols for upper-layer applications. + +Currently, the following SM features are supported: + +1. SM2, SM3, and SM4 algorithms are supported in the user-mode algorithm libraries, such as OpenSSL and libgcrypt. +2. SM2, SM3, and SM4 cipher suites are supported in OpenSSH. +3. The SM Transport Layer Cryptography Protocol (TLCP) stack is supported in OpenSSL. +4. SM3 and SM4 algorithms are supported for disk encryption (dm-crypt/cryptsetup). +5. The SM3 algorithm is supported for password encryption in user identity authentication (pam/libuser/shadow). +6. The SM3 algorithm is supported for data digest in intrusion detection (AIDE). +7. SM2, SM3, and SM4 algorithms are supported in the kernel cryptographic framework (crypto) and algorithm performance optimization using instruction sets such as AVX/CE/NEON is allowed. +8. The SM3 message digest algorithm and SM2 certificate are supported in Integrity Measurement Architecture and Extended Verification Module (IMA/EVM) of the kernel. +9. The SM2 certificate is supported in kernel module signing and module signature verification. +10. SM4-CBC and SM4-GCM algorithms are supported in Kernel Transport Layer Security (KTLS). +11. SM3 and SM4 algorithms are supported in Kunpeng Accelerator Engine (KAE). +12. UEFI secure boot supports the SM3 digest algorithm and SM2 digital signatures. +13. RPM supports the SM2 encryption/decryption algorithm and SM3 digest algorithm for signing and verification. diff --git a/docs/en/server/security/shangmi/rpm-signature-verification.md b/docs/en/server/security/shangmi/rpm-signature-verification.md new file mode 100644 index 0000000000000000000000000000000000000000..c0df357090c3133907ed134f3a8fdd370cf012be --- /dev/null +++ b/docs/en/server/security/shangmi/rpm-signature-verification.md @@ -0,0 +1,99 @@ +# RPM Signature Verification + +## Overview + +openEuler employs RPM for package management, adhering to the openPGP signature specification. openEuler 24.03 LTS SP1 enhances the open source RPM by adding support for SM2/3 algorithm-based signature generation and verification. + +The following packages have been enhanced for SM algorithm capabilities: + +- GnuPG: The `gpg` CLI tool now supports generating SM signatures. +- RPM: RPM can now invoke `gpg` commands and openSSL APIs for SM signature generation and verification. +- openSSL: SM signature verification is supported (already supported in the open source version). + +## Prerequisites + +1. The following or later versions of gnupg2, libgcrypt, and rpm packages must be installed: + + ```sh + $ rpm -qa libgcrypt + libgcrypt-1.10.2-3.oe2403sp1.x86_64 + + $ rpm -qa gnupg2 + gnupg2-2.4.3-5.oe2403sp1.x86_64 + + $ rpm -qa rpm + rpm-4.18.2-20.oe2403sp1.x86_64 + ``` + +2. ECDSA signing and verification are limited to SM2. + +## Usage + +1. Generate a key. + + Method 1: + + ```sh + gpg --full-generate-key --expert + ``` + + Method 2: + + ```sh + gpg --quick-generate-key sm2p256v1 + ``` + + You will be prompted to enter a password. This password is required for subsequent key operations or signing. Pressing Enter without entering a password means no password is set. + +2. Export the certificate. + + ```sh + gpg -o --export + ``` + +3. Enable the macro for SM3 hash algorithm and SM2 algorithm. + + ```sh + $ vim /usr/lib/rpm/macros + %_enable_sm2p256v1_sm3_algo 1 + ``` + +4. Import the certificate into the RPM database. + + ```sh + rpm --import + ``` + +5. Write the macros required for signing. + + ```sh + $ vim ~/.rpmmacros + %_signature gpg + %_gpg_path /root/.gnupg + %_gpg_name + %_gpgbin /usr/bin/gpg2 + + %__gpg_sign_cmd %{shescape:%{__gpg}} \ + gpg --no-verbose --no-armor --no-secmem-warning --passphrase-file /root/passwd \ + %{?_gpg_digest_algo:--digest-algo=%{_gpg_digest_algo}} \ + %{?_gpg_sign_cmd_extra_args} \ + %{?_gpg_name:-u %{shescape:%{_gpg_name}}} \ + -sbo %{shescape:%{?__signature_filename}} \ + %{?__plaintext_filename:-- %{shescape:%{__plaintext_filename}}} + ``` + + `%__gpg_sign_cmd` includes the default configuration with the addition of `--passphrase-file /root/passwd`. The **passwd** file contains the password. This addition is required only If a password is set in step 1. + +6. Generate a RPM package signature. + + ```sh + rpmsign --addsign + ``` + +7. Verify the RPM package signature. + + ```sh + rpm -Kv + ``` + + If the output shows "Header V4 ECDSA/SM3 Signature" and "OK," the signature verification is successful. diff --git a/docs/en/server/security/shangmi/secure-boot.md b/docs/en/server/security/shangmi/secure-boot.md new file mode 100644 index 0000000000000000000000000000000000000000..3dbfde2eaa8f2db1c482d1a39fff8feac1d0443b --- /dev/null +++ b/docs/en/server/security/shangmi/secure-boot.md @@ -0,0 +1,185 @@ +# Secure Boot + +Secure Boot is a standard feature defined in the UEFI specification. It verifies the signature of each component level by level during system startup to ensure the integrity and authenticity of the boot sequence. The UEFI Secure Boot of the Linux system includes the following three procedures: + +1. The BIOS uses its built-in certificate to verify the signature of the shim component. +2. The shim component uses its built-in certificate to verify the signature of the grub component. +3. The grub component verifies the signature of the kernel component through the interface provided by the shim component. + +openEuler adds support for ShangMi (SM) algorithms to the pesign EFI signature tool and its nss algorithm library. That is, openEuler supports SM3 for hash calculation, SM2 for signing and verifying EFI files. In this way, the secure boot of openEuler can be implemented using SM algorithms. + +## Constraints + +- The openEuler shim component supports SM-based Secure Boot, including verification of the GRUB signature by shim, and verification of the kernel signature by GRUB. The verification of the shim signature depends on the BIOS. +- An ARM64/x86 physical machine that supports UEFI Secure Boot is required. +- The pesign tool can sign signatures with a maximum level of 2. +- Currently, the pesign tool can only generate signatures but cannot verify signatures. + +## Preparations + +1. The following software packages (or their later versions) must be installed: + + ```shell + openssl-1.1.1m-15.oe2203.aarch64 + nss-3.72.0-4.oe2203.aarch64 + pesign-115-2.oe2203.aarch64 + shim-15.6-7.oe2203.aarch64 + crypto-policies-20200619-3.git781bbd4.oe2203.noarch + ``` + +2. Download the source code of the openEuler shim component. Ensure that the version in the spec file is later than 15.6-7. + + ```shell + git clone https://gitee.com/src-openeuler/shim.git -b openEuler-22.03-LTS-SP1 --depth 1 + ``` + +3. Install software packages required for building the shim component: + + ```shell + yum install elfutils-libelf-devel gcc gnu-efi gnu-efi-devel openssl-devel make git rpm-build + ``` + +4. Check whether the SM3 algorithm is enabled for nss. If not, modify the file content as follows: + + ```shell + cat /usr/share/crypto-policies/DEFAULT/nss.txt | grep SM3 + config="disallow=ALL allow=HMAC-SHA256:HMAC-SHA1:HMAC-SHA384:HMAC-SHA512:CURVE25519:SECP256R1:SECP384R1:SECP521R1:aes256-gcm:chacha20-poly1305:aes256-cbc:aes128-gcm:aes128-cbc:SHA256:SHA384:SHA512:SHA224:SHA1:ECDHE-RSA:ECDHE-ECDSA:RSA:DHE-RSA:ECDSA:RSA-PSS:RSA-PKCS:tls-version-min=tls1.0:dtls-version-min=dtls1.0:DH-MIN=1023:DSA-MIN=2048:RSA-MIN=2048:SM3" + ``` + +## Generation of Keys and Certificates + +1. Generate the key and certificate for signing the shim component. The shim signature is verified by the BIOS. As most BIOSs do not support SM algorithms, the RSA algorithm is used. For BIOSs that support SM algorithms you can generate the SM2 key and certificate by referring to the next step. + + ```shell + openssl genrsa -out rsa.key 4096 + openssl req -new -key rsa.key -out rsa.csr -subj '/C=AA/ST=BB/O=CC/OU=DD/CN=secure boot BIOS' + openssl x509 -req -days 365 -in rsa.csr -signkey rsa.key -out rsa.crt + openssl x509 -in rsa.crt -out rsa.der -outform der + ``` + +2. Generate the SM2 key and certificate for signing the GRUB and kernel components. + + ```shell + openssl ecparam -genkey -name SM2 -out sm2.key + openssl req -new -sm3 -key sm2.key -out sm2.csr -subj '/C=AA/ST=BB/O=CC/OU=DD/CN=secure boot shim' + openssl x509 -req -days 3650 -signkey sm2.key -in sm2.csr -out sm2.crt + openssl x509 -in sm2.crt -out sm2.der -outform der + ``` + +3. Create an NSS database and import the keys and certificates generated in the preceding two steps to the NSS database. + + ```shell + # The NSS database is organized in the form of directories. The storage location can be customized. + mkdir certdb + certutil -N -d certdb + # Import the SM2 and RSA certificates to the NSS database and name them sm2 and rsa respectively. + certutil -A -n sm2 -d certdb -t CT,CT,CT -i sm2.crt + certutil -A -n rsa -d certdb -t CT,CT,CT -i rsa.crt + # To import the SM2 and RSA keys to the NSS database, compress them into PKCS 12 files. + openssl pkcs12 -export -out rsa.p12 -inkey rsa.key -in rsa.crt + openssl pkcs12 -export -out sm2.p12 -inkey sm2.key -in sm2.crt + pk12util -d certdb -i rsa.p12 + pk12util -d certdb -i sm2.p12 + ``` + +## shim Component Building + +1. Go to the shim source code directory, modify the configuration variables in shim.spec to enable the support for SM algorithms, and specify the built-in SM2 certificate. + + ```text + %global enable_sm 1 + %global vendor_cert /path/to/sm2.der + ``` + +2. Build the shim software package. + + ```shell + rpmbuild -ba shim.spec --define "_sourcedir $PWD" + ``` + +3. Install the built shim software package. + + ```shell + rpm -Uvh ~/rpmbuild/RPMS/aarch64/shim-xxx.rpm + ``` + +## SM Signature for UEFI Files + +1. Sign the shim component with the RSA key and certificate and replace the original one. + + ```shell + # ARM64 + pesign -n certdb -c rsa -s -i /boot/efi/EFI/openEuler/shimaa64.efi -o shimaa64.efi.signed + cp shimaa64.efi.signed /boot/efi/EFI/openEuler/shimaa64.efi + # x86 + pesign -n certdb -c rsa -s -i /boot/efi/EFI/openEuler/shimx64.efi -o shimx64.efi.signed + cp shimx64.efi.signed /boot/efi/EFI/openEuler/shimx64.efi + ``` + +2. Sign the GRUB component with the SM2 key and certificate and replace the original one. + + ```shell + # ARM64 + pesign -n certdb -c sm2 -s -i /boot/efi/EFI/openEuler/grubaa64.efi -o grubaa64.efi.signed -d sm3 + cp grubaa64.efi.signed /boot/efi/EFI/openEuler/grubaa64.efi + # x86 + pesign -n certdb -c sm2 -s -i /boot/efi/EFI/openEuler/grubx64.efi -o grubx64.efi.signed -d sm3 + cp grubx64.efi.signed /boot/efi/EFI/openEuler/grubx64.efi + ``` + +3. Sign the kernel component with the SM2 key and certificate and replace the original one. (Note that the file name contains the actual version number.) + + ```shell + # For the ARM64 architecture,you need to decompress and sign the component, and compress it again. + cp /boot/vmlinuz-5.10.0-126.0.0.66.oe2203.aarch64 vmlinuz-5.10.0-126.0.0.66.oe2203.aarch64.gz + gzip -d vmlinuz-5.10.0-126.0.0.66.oe2203.aarch64.gz + pesign -n certdb -c sm2 -s -i vmlinuz-5.10.0-126.0.0.66.oe2203.aarch64 -o vmlinuz-5.10.0-126.0.0.66.oe2203.aarch64.signed -d sm3 + gzip vmlinuz-5.10.0-126.0.0.66.oe2203.aarch64.signed + cp vmlinuz-5.10.0-126.0.0.66.oe2203.aarch64.signed.gz /boot/vmlinuz-5.10.0-126.0.0.66.oe2203.aarch64 + # x86 + pesign -n certdb -c sm2 -s -i /boot/vmlinuz-5.10.0-126.0.0.66.oe2203.x86_64 -o vmlinuz-5.10.0-126.0.0.66.oe2203.x86_64.signed -d sm3 + cp vmlinuz-5.10.0-126.0.0.66.oe2203.x86_64.signed /boot/vmlinuz-5.10.0-126.0.0.66.oe2203.x86_64 + ``` + +4. Check the signature information. The following uses shim and GRUB as examples: + + ```shell + pesign -S -i /boot/efi/EFI/openEuler/grubaa64.efi + pesign -S -i /boot/efi/EFI/openEuler/shimaa64.efi + ``` + +## Secure Boot + +Enter the BIOS, import the certificate for signing the shim component, and enable the Secure Boot option. The operation method varies depending on the BIOS. The following uses the Kunpeng 2280 v2 server as an example: + +1. Place the RSA certificate for signing the shim component in the **/boot/efi/EFI/openEuler** directory. + + ```shell + cp rsa.der /boot/efi/EFI/openEuler + ``` + +2. Restart the system. +3. Enter BIOS to enable Secure Boot: + + ```text + Setup > Security > Secure Boot > Enable + ``` + +4. Set the Secure Boot mode to custom: + + ```text + Setup > Security > Secure Boot Certificate Configuration > Secure Boot Mode > Custom + ``` + +5. Import the Secure Boot certificate: + + ```text + Setup > Security > Secure Boot Certificate Configuration > Options Related to Secure Boot Custom Mode > DB Options > Import Signature > Add Signature by File > Select rsa.der > Save and exit. + ``` + +6. Save the configuration and restart the system. The system is started successfully. Secure Boot is enabled. + + ```shell + mokutil --sb-state + SecureBoot enabled + ``` diff --git a/docs/en/server/security/shangmi/ssh-stack.md b/docs/en/server/security/shangmi/ssh-stack.md new file mode 100644 index 0000000000000000000000000000000000000000..219a551288267a76272aa8a1f046b34f2e1567f0 --- /dev/null +++ b/docs/en/server/security/shangmi/ssh-stack.md @@ -0,0 +1,52 @@ +# SSH Stack + +## Overview + +The OpenSSH component is a Secure Shell Protocol (SSH) component implemented based on libcrypto of OpenSSL in C language. The main function is remote login to ensure the integrity and reliability of encrypted information over an unsecured network. The SSH server and client configuration items provided by openEuler involve key exchange, public key authentication, symmetric encryption, and integrity authentication. The values of these configuration items can be ShangMi (SM) Cipher Suites (including SM2, SM3, and SM4 algorithms). + +## Prerequisites + +OpenSSH 8.8p1-5 or later + +```shell +$ rpm -qa | grep openssh +openssh-8.8p1-5.oe2209.x86_64 +openssh-server-8.8p1-5.oe2209.x86_64 +openssh-clients-8.8p1-5.oe2209.x86_64 +``` + +## How to Use + +### Scenario 1: Remote Login + +1. On the client, call **ssh-keygen** to generate a user key, which is saved as **\~/.ssh/id_sm2** and **\~/.ssh/id_sm2.pub** by default. Then, send **\~/.ssh/id_sm2.pub** from the client to the server. (You can also run the **ssh-copy-id** command to send the file.) + + ```shell + ssh-keygen -t sm2 -m PEM + ``` + +2. On the server, call **ssh-keygen** to generate a host key and add the public key sent by the client to the authorized key file list. (If you run the **ssh-copy-id** command, the public key is automatically written.) + + ```shell + ssh-keygen -t sm2 -m PEM -f /etc/ssh/ssh_host_sm2_key + cat /path/to/id_sm2.pub >> ~/.ssh/authorized_keys + ``` + +3. On the server, modify the **/etc/ssh/sshd_config** file to support login using SM series cryptographic algorithms. The following table lists the SM configuration items. + + | Description | Configuration Item | SM Value | + | ------------------------------------------------------------------------------------ | ---------------------- | ------------------------- | + | Authentication key for the host key and public key (configurable only on the server) | HostKeyAlgorithms | /etc/ssh/ssh_host_sm2_key | + | Host key and public key authentication algorithm | HostKeyAlgorithms | sm2 | + | Key exchange algorithm | KexAlgorithms | sm2-sm3 | + | Symmetric cryptographic algorithm | Ciphers | sm4-ctr | + | Integrity check algorithm | MACs | hmac-sm3 | + | User public key authentication algorithm | PubkeyAcceptedKeyTypes | sm2 | + | Authentication key for the user public key (configurable only on the client) | IdentityFile | ~/.ssh/id_sm2 | + | Hash algorithm used for printing key fingerprints | FingerprintHash | sm3 | + +4. On the client, configure the SM series cryptographic algorithms to complete the login. You can enable the SM Cipher Suites on the client by running commands or modifying the configuration file. The following shows how to log in using the CLI: + + ```shell + ssh -o PreferredAuthentications=publickey -o HostKeyAlgorithms=sm2 -o PubkeyAcceptedKeyTypes=sm2 -o Ciphers=sm4-ctr -o MACs=hmac-sm3 -o KexAlgorithms=sm2-sm3 -i ~/.ssh/id_sm2 [remote-ip] + ``` diff --git a/docs/en/server/security/shangmi/tlcp-stack.md b/docs/en/server/security/shangmi/tlcp-stack.md new file mode 100644 index 0000000000000000000000000000000000000000..30e6b9f92ae6e93050e1e6a03ce5cf5e3f4b21be --- /dev/null +++ b/docs/en/server/security/shangmi/tlcp-stack.md @@ -0,0 +1,518 @@ +# TLCP Stack + +## Overview + +Transport Layer Cryptography Protocol (TLCP) is a secure communication protocol that complies with the *GB/T 38636-2020 Information security technology-Transport layer cryptography protocol (TLCP)*. TLCP requires an encryption certificate and a signature certificate with their corresponding private keys in communication. Based on the open source version, the OpenSSL software released after openEuler 22.09 supports the TLCP and provides the following functions: + +- Double certificates for SM series cryptographic algorithms based on TLCP +- ECC_SM4_CBC_SM3 and ECDHE_SM4_CBC_SM3 cipher suites +- SM2 certificates + +## Prerequisites + +The version of the OpenSSL software installed in the openEuler operating system is later than 1.1.1m-4. + +```shell +$ rpm -qa openssl +openssl-1.1.1m-6.oe2209.x86_64 +``` + +## How to Use + +### Scenario 1: Generating Two SM2 Certificates + +According to the TLCP, two certificates are required for communication: signature certificate and encryption certificate. The signature certificate is used for identity authentication, and the encryption certificate is used for data encryption during key negotiation. CA is short for Certificate Authority. A certificate is generated only after the CSR certificate request file is issued by the CA. The following is a reference case that describes how to use a self-signed CA certificate to issue an entity certificate: + +1. Prepare the configuration file **openssl.cnf** required for generating the certificates. The reference content is as follows (modified based on the OpenSSL source code **apps/openssl.cnf**): + + ```conf + # This definition stops the following lines choking if HOME isn't + # defined. + HOME = . + + # Extra OBJECT IDENTIFIER info: + #oid_file = $ENV::HOME/.oid + oid_section = new_oids + + # To use this configuration file with the "-extfile" option of the + # "openssl x509" utility, name here the section containing the + # X.509v3 extensions to use: + # extensions = + # (Alternatively, use a configuration file that has only + # X.509v3 extensions in its main [= default] section.) + + [ new_oids ] + + # We can add new OIDs in here for use by 'ca', 'req' and 'ts'. + # Add a simple OID like this: + # testoid1=1.2.3.4 + # Or use config file substitution like this: + # testoid2=${testoid1}.5.6 + + # Policies used by the TSA examples. + tsa_policy1 = 1.2.3.4.1 + tsa_policy2 = 1.2.3.4.5.6 + tsa_policy3 = 1.2.3.4.5.7 + + #################################################################### + [ ca ] + default_ca = CA_default # The default ca section + + #################################################################### + [ CA_default ] + + dir = ./demoCA # Where everything is kept + certs = $dir/certs # Where the issued certs are kept + crl_dir = $dir/crl # Where the issued crl are kept + database = $dir/index.txt # database index file. + #unique_subject = no # Set to 'no' to allow creation of + # several certs with same subject. + new_certs_dir = $dir/newcerts # default place for new certs. + + certificate = $dir/cacert.pem # The CA certificate + serial = $dir/serial # The current serial number + crlnumber = $dir/crlnumber # the current crl number + # must be commented out to leave a V1 CRL + crl = $dir/crl.pem # The current CRL + private_key = $dir/private/cakey.pem# The private key + + x509_extensions = usr_cert # The extensions to add to the cert + + # Comment out the following two lines for the "traditional" + # (and highly broken) format. + name_opt = ca_default # Subject Name options + cert_opt = ca_default # Certificate field options + + # Extension copying option: use with caution. + # copy_extensions = copy + + # Extensions to add to a CRL. Note: Netscape communicator chokes on V2 CRLs + # so this is commented out by default to leave a V1 CRL. + # crlnumber must also be commented out to leave a V1 CRL. + # crl_extensions = crl_ext + + default_days = 365 # how long to certify for + default_crl_days= 30 # how long before next CRL + default_md = default # use public key default MD + preserve = no # keep passed DN ordering + + # A few difference way of specifying how similar the request should look + # For type CA, the listed attributes must be the same, and the optional + # and supplied fields are just that :-) + policy = policy_match + + # For the CA policy + [ policy_match ] + countryName = match + stateOrProvinceName = match + organizationName = match + organizationalUnitName = optional + commonName = supplied + emailAddress = optional + + # For the 'anything' policy + # At this point in time, you must list all acceptable 'object' + # types. + [ policy_anything ] + countryName = optional + stateOrProvinceName = optional + localityName = optional + organizationName = optional + organizationalUnitName = optional + commonName = supplied + emailAddress = optional + + #################################################################### + [ req ] + default_bits = 2048 + default_keyfile = privkey.pem + distinguished_name = req_distinguished_name + attributes = req_attributes + x509_extensions = v3_ca # The extensions to add to the self signed cert + + # Passwords for private keys if not present they will be prompted for + # input_password = secret + # output_password = secret + + # This sets a mask for permitted string types. There are several options. + # default: PrintableString, T61String, BMPString. + # pkix : PrintableString, BMPString (PKIX recommendation before 2004) + # utf8only: only UTF8Strings (PKIX recommendation after 2004). + # nombstr : PrintableString, T61String (no BMPStrings or UTF8Strings). + # MASK:XXXX a literal mask value. + # WARNING: ancient versions of Netscape crash on BMPStrings or UTF8Strings. + string_mask = utf8only + + # req_extensions = v3_req # The extensions to add to a certificate request + + [ req_distinguished_name ] + countryName = Country Name (2 letter code) + countryName_default = AU + countryName_min = 2 + countryName_max = 2 + + stateOrProvinceName = State or Province Name (full name) + stateOrProvinceName_default = Some-State + + localityName = Locality Name (eg, city) + + 0.organizationName = Organization Name (eg, company) + 0.organizationName_default = Internet Widgits Pty Ltd + + # we can do this but it is not needed normally :-) + #1.organizationName = Second Organization Name (eg, company) + #1.organizationName_default = World Wide Web Pty Ltd + + organizationalUnitName = Organizational Unit Name (eg, section) + #organizationalUnitName_default = + + commonName = Common Name (e.g. server FQDN or YOUR name) + commonName_max = 64 + + emailAddress = Email Address + emailAddress_max = 64 + + # SET-ex3 = SET extension number 3 + + [ req_attributes ] + challengePassword = A challenge password + challengePassword_min = 4 + challengePassword_max = 20 + + unstructuredName = An optional company name + + [ usr_cert ] + + # These extensions are added when 'ca' signs a request. + + # This goes against PKIX guidelines but some CAs do it and some software + # requires this to avoid interpreting an end user certificate as a CA. + + basicConstraints=CA:FALSE + + # Here are some examples of the usage of nsCertType. If it is omitted + # the certificate can be used for anything *except* object signing. + + # This is OK for an SSL server. + # nsCertType = server + + # For an object signing certificate this would be used. + # nsCertType = objsign + + # For normal client use this is typical + # nsCertType = client, email + + # and for everything including object signing: + # nsCertType = client, email, objsign + + # This is typical in keyUsage for a client certificate. + # keyUsage = nonRepudiation, digitalSignature, keyEncipherment + + # This will be displayed in Netscape's comment listbox. + nsComment = "OpenSSL Generated Certificate" + + # PKIX recommendations harmless if included in all certificates. + subjectKeyIdentifier=hash + authorityKeyIdentifier=keyid,issuer + + # This stuff is for subjectAltName and issuerAltname. + # Import the email address. + # subjectAltName=email:copy + # An alternative to produce certificates that aren't + # deprecated according to PKIX. + # subjectAltName=email:move + + # Copy subject details + # issuerAltName=issuer:copy + + #nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem + #nsBaseUrl + #nsRevocationUrl + #nsRenewalUrl + #nsCaPolicyUrl + #nsSslServerName + + # This is required for TSA certificates. + # extendedKeyUsage = critical,timeStamping + + [ v3_req ] + + # Extensions to add to a certificate request + + basicConstraints = CA:FALSE + keyUsage = nonRepudiation, digitalSignature + + [ v3enc_req ] + + # Extensions to add to a certificate request + + basicConstraints = CA:FALSE + keyUsage = keyAgreement, keyEncipherment, dataEncipherment + + [ v3_ca ] + + # Extensions for a typical CA + + + # PKIX recommendation. + + subjectKeyIdentifier=hash + + authorityKeyIdentifier=keyid:always,issuer + + basicConstraints = critical,CA:true + + # Key usage: this is typical for a CA certificate. However since it will + # prevent it being used as an test self-signed certificate it is best + # left out by default. + keyUsage = cRLSign, keyCertSign + + # Some might want this also + # nsCertType = sslCA, emailCA + + # Include email address in subject alt name: another PKIX recommendation + # subjectAltName=email:copy + # Copy issuer details + # issuerAltName=issuer:copy + + # DER hex encoding of an extension: beware experts only! + # obj=DER:02:03 + # Where 'obj' is a standard or added object + # You can even override a supported extension: + # basicConstraints= critical, DER:30:03:01:01:FF + + [ crl_ext ] + + # CRL extensions. + # Only issuerAltName and authorityKeyIdentifier make any sense in a CRL. + + # issuerAltName=issuer:copy + authorityKeyIdentifier=keyid:always + + [ proxy_cert_ext ] + # These extensions should be added when creating a proxy certificate + + # This goes against PKIX guidelines but some CAs do it and some software + # requires this to avoid interpreting an end user certificate as a CA. + + basicConstraints=CA:FALSE + + # Here are some examples of the usage of nsCertType. If it is omitted + # the certificate can be used for anything *except* object signing. + + # This is OK for an SSL server. + # nsCertType = server + + # For an object signing certificate this would be used. + # nsCertType = objsign + + # For normal client use this is typical + # nsCertType = client, email + + # and for everything including object signing: + # nsCertType = client, email, objsign + + # This is typical in keyUsage for a client certificate. + # keyUsage = nonRepudiation, digitalSignature, keyEncipherment + + # This will be displayed in Netscape's comment listbox. + nsComment = "OpenSSL Generated Certificate" + + # PKIX recommendations harmless if included in all certificates. + subjectKeyIdentifier=hash + authorityKeyIdentifier=keyid,issuer + + # This stuff is for subjectAltName and issuerAltname. + # Import the email address. + # subjectAltName=email:copy + # An alternative to produce certificates that aren't + # deprecated according to PKIX. + # subjectAltName=email:move + + # Copy subject details + # issuerAltName=issuer:copy + + #nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem + #nsBaseUrl + #nsRevocationUrl + #nsRenewalUrl + #nsCaPolicyUrl + #nsSslServerName + + # This really needs to be in place for it to be a proxy certificate. + proxyCertInfo=critical,language:id-ppl-anyLanguage,pathlen:3,policy:foo + + #################################################################### + [ tsa ] + + default_tsa = tsa_config1 # the default TSA section + + [ tsa_config1 ] + + # These are used by the TSA reply generation only. + dir = ./demoCA # TSA root directory + serial = $dir/tsaserial # The current serial number (mandatory) + crypto_device = builtin # OpenSSL engine to use for signing + signer_cert = $dir/tsacert.pem # The TSA signing certificate + # (optional) + certs = $dir/cacert.pem # Certificate chain to include in reply + # (optional) + signer_key = $dir/private/tsakey.pem # The TSA private key (optional) + signer_digest = sha256 # Signing digest to use. (Optional) + default_policy = tsa_policy1 # Policy if request did not specify it + # (optional) + other_policies = tsa_policy2, tsa_policy3 # acceptable policies (optional) + digests = sha1, sha256, sha384, sha512 # Acceptable message digests (mandatory) + accuracy = secs:1, millisecs:500, microsecs:100 # (optional) + clock_precision_digits = 0 # number of digits after dot. (optional) + ordering = yes # Is ordering defined for timestamps? + # (optional, default: no) + tsa_name = yes # Must the TSA name be included in the reply? + # (optional, default: no) + ess_cert_id_chain = no # Must the ESS cert id chain be included? + # (optional, default: no) + ess_cert_id_alg = sha1 # algorithm to compute certificate + # identifier (optional, default: sha1) + ``` + +2. Generate a self-signed CA certificate. + + ```shell + openssl ecparam -name SM2 -out SM2.pem + openssl req -config ./openssl.cnf -nodes -subj '/C=AA/ST=BB/O=CC/OU=DD/CN=root ca' -keyout CA.key -newkey ec:SM2.pem -new -out CA.csr + openssl x509 -sm3 -req -days 30 -in CA.csr -extfile ./openssl.cnf -extensions v3_ca -signkey CA.key -out CA.crt + ``` + +3. Generate the server signature certificate and encryption certificate. + + ```shell + openssl req -config ./openssl.cnf -nodes -subj '/C=AA/ST=BB/O=CC/OU=DD/CN=server sign' -keyout SS.key -newkey ec:SM2.pem -new -out SS.csr + openssl x509 -sm3 -req -days 30 -in SS.csr -CA CA.crt -CAkey CA.key -extfile ./openssl.cnf -extensions v3_req -out SS.crt -CAcreateserial + openssl req -config ./openssl.cnf -nodes -subj '/C=AA/ST=BB/O=CC/OU=DD/CN=server enc' -keyout SE.key -newkey ec:SM2.pem -new -out SE.csr + openssl x509 -sm3 -req -days 30 -in SE.csr -CA CA.crt -CAkey CA.key -extfile ./openssl.cnf -extensions v3enc_req -out SE.crt -CAcreateserial + ``` + +4. Generate the client signature certificate and encryption certificate. + + ```shell + openssl req -config ./openssl.cnf -nodes -subj '/C=AA/ST=BB/O=CC/OU=DD/CN=client sign' -keyout CS.key -newkey ec:SM2.pem -new -out CS.csr + openssl x509 -sm3 -req -days 30 -in CS.csr -CA CA.crt -CAkey CA.key -extfile ./openssl.cnf -extensions v3_req -out CS.crt -CAcreateserial + openssl req -config ./openssl.cnf -nodes -subj '/C=AA/ST=BB/O=CC/OU=DD/CN=client enc' -keyout CE.key -newkey ec:SM2.pem -new -out CE.csr + openssl x509 -sm3 -req -days 30 -in CE.csr -CA CA.crt -CAkey CA.key -extfile ./openssl.cnf -extensions v3enc_req -out CE.crt -CAcreateserial + ``` + +### Scenario 2: Using the OpenSSL CLI to Verify the TLCP Stack + +The **s_server/s_client** tool provided by OpenSSL can be used to test the TLCP. + +```shell +# Start the server. +openssl s_server -verify 5 -accept 4433 \ + -cert SS.crt \ + -key SS.key \ + -dcert SE.crt \ + -dkey SE.key \ + -CAfile CA.crt + +# Start the client. +openssl s_client -verify 5 -connect 127.0.0.1:4433 \ + -cert CS.crt \ + -key CS.key \ + -dcert CE.crt \ + -dkey CE.key \ + -CAfile CA.crt -tlcp +``` + +### Scenario 3: Using OpenSSL APIs + +Reference code on the server: + +```cpp +int main() { + // Define the variables. + SSL_CTX *ctx = NULL; + const char *sign_cert_file = "SS.crt"; + const char *sign_key_file = "SS.key"; + const char *enc_cert_file = "SE.crt"; + const char *enc_key_file = "SE.key"; + + // Generate the context. + ctx = SSL_CTX_new(TLS_server_method()); + + // Load the signature certificate and encryption certificate and their private keys. + if (!SSL_CTX_use_gm_certificate_file(ctx, sign_cert_file, SSL_FILETYPE_PEM, SSL_USAGE_SIG)) + goto err; + + if (!SSL_CTX_use_gm_PrivateKey_file(ctx, sign_key_file, SSL_FILETYPE_PEM, SSL_USAGE_SIG)) + goto err; + + if (!SSL_CTX_use_gm_certificate_file(ctx, enc_cert_file, SSL_FILETYPE_PEM, SSL_USAGE_ENC)) + goto err; + + if (!SSL_CTX_use_gm_PrivateKey_file(ctx, enc_key_file, SSL_FILETYPE_PEM, SSL_USAGE_ENC)) + goto err; + + SSL_CTX_set_options(ctx, SSL_OP_ENCCERT_SECOND_POSITION); + + // The subsequent procedure is the same as that of the standard TLS. + SSL *ssl = SSL_new(ctx); +} +``` + +Client reference code: + +```cpp +int main() { + // Define the variables. + SSL_CTX *ctx = NULL; + const char *sign_cert_file = "CS.crt"; + const char *sign_key_file = "CS.key"; + const char *enc_cert_file = "CE.crt"; + const char *enc_key_file = "CE.key"; + + // Generate the context. + ctx = SSL_CTX_new(TLCP_client_method()); + + // Load the signature certificate and encryption certificate and their private keys. + if (!SSL_CTX_use_gm_certificate_file(ctx, sign_cert_file, SSL_FILETYPE_PEM, SSL_USAGE_SIG)) + goto err; + + if (!SSL_CTX_use_gm_PrivateKey_file(ctx, sign_key_file, SSL_FILETYPE_PEM, SSL_USAGE_SIG)) + goto err; + + if (!SSL_CTX_use_gm_certificate_file(ctx, enc_cert_file, SSL_FILETYPE_PEM, SSL_USAGE_ENC)) + goto err; + + if (!SSL_CTX_use_gm_PrivateKey_file(ctx, enc_key_file, SSL_FILETYPE_PEM, SSL_USAGE_ENC)) + goto err; + + // Set the cipher suite to ECC-SM4-CBC-SM3 or ECDHE-SM4-CBC-SM3. + // This step is not mandatory. By default, ECC-SM4-CBC-SM3 is preferred. + if(SSL_CTX_set_cipher_list(ctx, "ECC-SM4-CBC-SM3") <= 0) + goto err; + + // The subsequent procedure is the same as that of the standard TLS. + SSL *ssl = SSL_new(ctx); +} +``` + +# KTLS Uninstallation + +## Overview + +The Linux kernel protocol stack implements only the TCP/IP model and does not support the SSL/TLS session layer protocol. Currently, TLS encryption and decryption are implemented in user mode. However, in some scenarios, for example, when the kernel sends a file, multiple cross-state copies are generated, causing performance overhead. Therefore, the kernel implements KTLS. That is, the encryption context can be configured for the socket to offload data encryption to the kernel mode or lower-layer hardware. + +The KTLS feature of the openEuler kernel 5.10 supports SM series cryptographic algorithms. Currently, the SM4-GCM and SM4-CCM algorithms are supported. + +## Prerequisites + +Kernel 5.10.0-106 or later + +```shell +$ rpm -qa kernel +kernel-5.10.0-106.1.0.55.oe2209.x86_64 +``` + +## How to Use + +The method of calling SM series cryptographic algorithms is the same as that of calling other algorithms of the same type. For details, see the [Linux kernel document](https://www.kernel.org/doc/html/v5.10/networking/tls.html). diff --git a/docs/en/server/security/shangmi/user-identity-authentication.md b/docs/en/server/security/shangmi/user-identity-authentication.md new file mode 100644 index 0000000000000000000000000000000000000000..241d2e6a1d14059a937f63bf0d5e1cb9cc413279 --- /dev/null +++ b/docs/en/server/security/shangmi/user-identity-authentication.md @@ -0,0 +1,131 @@ +# User Identity Authentication + +Operating systems usually use the password mechanism to authenticate users. openEuler provides user password management components, such as PAM, passwd, shadow, and libuser. After a user password is set, it needs to be encrypted for storage. Generally, the hash algorithm is used for encryption. The user password management component provided by openEuler supports the ShangMi 3 (SM3) cryptographic hash algorithm. + +## Using PAM to Encrypt User Passwords + +### Overview + +PAM is a pluggable authentication module of the system that provides an authentication mechanism for upper-layer applications. PAM released by openEuler supports user password encryption using the SM3 algorithm. + +### Prerequisites + +1. PAM 1.5.2-2 or later + + ```shell + $ rpm -qa pam + pam-1.5.2-2.oe2209.x86_64 + ``` + +2. libxcrypt 4.4.26-2 or later + + ```shell + $ rpm -qa libxcrypt + pam-4.4.26-2.oe2209.x86_64 + ``` + +### How to Use + +1. Open the **/etc/pam.d/password-auth** and **/etc/pam.d/system-auth** files, locate the line starting with **password sufficient pam_unix.so**, and change the algorithm field in the line to **sm3**. + + ```shell + $ cat /etc/pam.d/password-auth + ...... + password sufficient pam_unix.so sm3 shadow nullok try_first_pass use_authtok + ...... + + $ cat /etc/pam.d/system-auth + ...... + password sufficient pam_unix.so sm3 shadow nullok try_first_pass use_authtok + ...... + ``` + +2. After the configuration is modified, the password changed by running the **passwd** command or the password created by a new user is encrypted using the SM3 algorithm. The encryption result starts with **sm3** and is stored in **/etc/shadow**. + + ```shell + $ passwd testuser + Changing password for user testuser. + New password: + Retype new password: + passwd: all authentication tokens updated successfully. + $ cat /etc/shadow | grep testuser + testuser:$sm3$wnY86eyUlB5946gU$99LlMr0ddeZNDqnB2KRxn9f30SFCCvMv1WN1cFdsKJ2:19219:0:90:7:35:: + ``` + +### Notes + +1. By default, the SHA512 algorithm is used. After the SM3 algorithm is used, the existing user passwords are not affected. The cryptographic algorithm takes effect only after the passwords are changed. +2. If PAM and libxcrypt need to be downgraded to non-SM versions and existing user passwords are encrypted using the SM3 algorithm, modify the configuration first. + Set the algorithm to a non-SM algorithm, change the user passwords, and downgrade the software to a non-SM version. Otherwise, these users cannot log in to the system. + +## Using Shadow to Encrypt User Passwords + +### Overview + +Shadow is a common user management component in Linux. It provides commands such as **chpasswd**, **chgpasswd**, and **newusers**. The shadow component provided by openEuler supports the SM3 algorithm in user management. By default, Shadow uses the PAM security authentication mechanism. Therefore, the SM3 algorithm supported by Shadow affects only the **chpasswd** and **chgpasswd** commands. + +### Prerequisites + +Shadow 4.9-4 or later + +```shell +$ rpm -qa shadow +shadow-4.9-4.oe2209.x86_64 +``` + +### How to Use + +1. By default, **chpasswd** uses the PAM configuration. Use **-c** to specify the SM3 algorithm. The encryption result starts with **sm3** and is stored in **/etc/shadow**. + + ```shell + $ echo testuser:testPassword |chpasswd -c SM3 + $ cat /etc/shadow | grep testuser + testuser:$sm3$moojQQeBfdGOrL14$NqjckLHlk3ICs1cx.0rKZwRHafjVlqksdSJqfx9eYh6:19220:0:99999:7::: + ``` + +2. By default, **chgpasswd** uses the PAM configuration. Use **-c** to specify the SM3 algorithm. The encryption result starts with **sm3** and is stored in **/etc/shadow**. + + ```shell + $ echo testGroup:testPassword |chpasswd -c SM3 + $ cat /etc/gshadow | grep testGroup + testGroup:$sm3$S3h3X6U6KsXg2Gkc$LFCAnKbi6JItarQz4Y/Aq9/hEbEMQXq9nQ4rY1j9BY9:: + ``` + +### Notes + +By default, Shadow uses the PAM security authentication mechanism. When **-c** is used to specify the encryption algorithm in related commands, the PAM mechanism is not used. + +## Using libuser to Encrypt User Passwords + +### Overview + +The libuser library implements a standardized interface for operating and managing users and group accounts. This library is encapsulated and provides the command line interface (CLI) and Python interface for managing users and groups. User passwords can be encrypted using algorithms such as DES, MD5, Blowfish, SHA256, and SHA512. libuser released by openEuler supports the SM3 algorithm. + +### Prerequisites + +libuser 0.63-3 or later + +```shell +$ rpm -qa libuser +libuser-0.63-3.oe2209.x86_64 +``` + +### How to Use + +1. Configure **crypt_style=sm3** in the **\[defaults]** section in **/etc/libuser.conf**. + + ```shell + $ cat /etc/libuser.conf + ...... + [defaults] + crypt_style = sm3 + ...... + ``` + +2. When you run the **lusermod**, **lpasswd**, or **luseradd** command to set a user password, the default password encryption algorithm is SM3. The encryption result starts with **sm3** and is stored in **/etc/shadow**. + + ```shell + # luseradd testuser -P Test@123 + # cat /etc/shadow | grep testuser + testuser:$sm3$1IJtoN6zlBDCiPKC$5oxscBTgiquPAEmZWGNDVqTPrboHJw3fFSohjF6sONB:18862:0:90:7:35:: + ``` diff --git a/docs/en/server/security/trusted_computing/_toc.yaml b/docs/en/server/security/trusted_computing/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..266b740eb45ff505b7e1821ff965efc432e0cce3 --- /dev/null +++ b/docs/en/server/security/trusted_computing/_toc.yaml @@ -0,0 +1,15 @@ +label: Trusted Computing +isManual: true +href: ./trusted-computing.md +description: Definition and key concepts of trusted computing +sections: + - label: Kernel Integrity Measurement Architecture (IMA) + href: ./ima.md + - label: Dynamic Integrity Measurement (DIM) + href: ./dim.md + - label: Remote Attestation (Kunpeng Security Library) + href: ./remote-attestation-kunpeng-security-library.md + - label: Trusted Platform Control Module + href: ./tpcm.md + - label: Common Issues and Solutions + href: ./trusted-computing-common-issues-and-solutions.md diff --git a/docs/en/server/security/trusted_computing/dim.md b/docs/en/server/security/trusted_computing/dim.md new file mode 100644 index 0000000000000000000000000000000000000000..015be8aee1c9597e36a40e8ba2ca39ae5f3a07a7 --- /dev/null +++ b/docs/en/server/security/trusted_computing/dim.md @@ -0,0 +1,727 @@ +# Dynamic Integrity Measurement (DIM) + +This section describes the DIM feature and its usage. + +## Context + +With the development of the IT industry, information systems are facing an increasing number of security risks. Information systems run a large amount of software, some of which is inevitably vulnerable. Once exploited by attackers, these vulnerabilities could severely damage system services, resulting in data leakage and service unavailability. + +Most software attacks are accompanied by integrity damage, such as malicious process execution, configuration file tampering, and backdoor implantation. Therefore, protection technologies are proposed in the industry. Key data is measured and verified during system startup to ensure that the system can run properly. However, popular integrity protection technologies (such as secure boot and file integrity measurement) cannot protect memory data during process running. If an attacker somehow modifies the instructions of a process, the process may be hijacked or implanted with a backdoor, which is highly destructive and stealthy. To defend against such attacks, the DIM technology is proposed to measure and protect key data in the memory of a running process. + +## Terminology + +Static baseline: baseline measurement data generated by parsing the binary file of the measurement target + +Dynamic baseline: result of the first measurement on the measurement target + +Measurement policy: configuration information for measuring the target + +Measurement log: list of measurement information, including the measurement targets and measurement results + +## Description + +The DIM feature measures key data (such as code sections and data sections) in the memory during program running and compares the measurement result with the baseline value to determine whether the memory data has been tampered with. In this way, DIM can detect attacks and take countermeasures. + +### Function Scope + +- Currently, DIM supports only AArch64 and x86 architectures. +- Currently, DIM supports measurement of the following key memory data: + - Code section of a user-mode process (the section whose attribute is **PT_LOAD** and permission is **RX** in the ELF file, and the virtual memory area whose permission is **RX** after the corresponding process is loaded) + - Kernel module code section, whose start address and length are **core_layout.base** and **core_layout.text_size** respectively in the **struct module** structure corresponding to the kernel module. + - Kernel code section, corresponding to **\_stext** to **\_etext** (Addresses that may change due to the kernel static key mechanism are skipped.) +- DIM can work with the following hardware: + - The measurement result can be extended to the Platform Configuration Register (PCR) of Trusted Platform Module (TPM) 2.0 to connect to the remote attestation service. + +### Technical Limitations + +- For user-mode processes, only mapped code sections of files can be measured. Anonymous code sections cannot be measured. +- Kernel hot patches cannot be measured. +- Measurement can only be triggered proactively. If a file is attacked and recovered during two measurement processes, the attack cannot be identified. +- Proactive modification of code sections, such as relocation of code sections, self-modification, and hot patches, will be identified as attacks. +- For kernel and kernel module measurement, the measurement result when dynamic baseline establishment is triggered is used as the measurement baseline value. The static baseline value only serves as a fixed identifier. +- The measurement target must have been loaded to the memory when dynamic baseline establishment is triggered (for example, process running or kernel module loading). Otherwise, the object cannot be measured. +- If the TPM PCR needs to be used to verify measurement logs, the DIM module cannot be removed. Otherwise, the measurement logs will be cleared and cannot match the PCR. + +> [!NOTE]NOTE +> After DIM is enabled, system performance is affected in the following aspects: + +- Loading the DIM feature and managing baseline data and measurement logs consume system memory. The impact depends on the protection policy configuration. +- Hashing is performed during DIM running, which consumes CPU resources. The impact depends on the size of the data to be measured. +- Resources will be locked and semaphores need to be obtained during DIM running, which may cause other concurrent processes to wait. + +### Specification Constraints + +| Item | Value | +| ----------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ | +| Maximum size of a policy file, static baseline file, signature file, or certificate file | 10MB | +| Maximum number of tampering measurement logs recorded for a measurement target during multiple measurement periods after a dynamic baseline is established. | 10 | +| Maximum number of measurement policies that can be stored in **/etc/dim/policy** | 10,000 | + +### Architecture Description + +DIM includes the dim_tools and dim software packages, which contain the following components: + +| Software Package | Component | Description | +| --------- | ---------------- | ------------------------------------------------------------ | +| dim_tools | dim_gen_baseline| User-mode component for generating the static baseline, which is required for generating the dynamic measurement. The baseline data is imported when DIM runs.| +| dim | dim_core | Kernel module for executing the core dynamic measurement logic, including policy parsing, static baseline parsing, dynamic baseline establishment, measurement execution, measurement logging, and TPM extension operations, to measure key memory data.| +| dim | dim_monitor | Kernel module for measuring and protecting dim_core code sections and key data to prevent DIM function failures caused by attacks on dim_core. | + +The following figure shows the overall architecture: + +![](./figures/dim_architecture.jpg) + +### Key Procedures + +Both the dim_core and dim_monitor modules provide the memory data measurement function, including the following core processes: + +- Dynamic baseline process: The dim_core module reads and parses the policy and static baseline file, measures the code section of the target process, stores the measurement result as a dynamic baseline in the memory, compares the dynamic baseline data with the static baseline data, and records the comparison result in measurement logs. The dim_monitor module measures the code sections and key data of the dim_core module, uses the data as the dynamic baseline, and records measurement logs. +- Dynamic measurement process: The dim_core and dim_monitor modules measure the target and compare the measurement result with the dynamic baseline. If the measurement result is inconsistent with the dynamic baseline, the dim_core and dim_monitor modules record the result in measurement logs. + +### Interfaces + +#### Interface Files + +| Path | Description | +| ------------------------------- | ------------------------------------------------------------ | +| /etc/dim/policy | Measurement policy file | +| /etc/dim/policy.sig | Measurement policy signature file, which is used to store the signature information of the policy file and is used when the signature verification function is enabled| +| /etc/dim/digest_list/*.hash | Static baseline file, which is used to store measurement baseline values | +| /etc/dim/digest_list/*.hash.sig | Static baseline signature file, which is used to store the signature information of the static baseline file and is used when the signature verification function is enabled| +| /etc/keys/x509_dim.der | Certificate file, which is used to verify the signature information of the policy file and static baseline file and is used when the signature verification function is enabled| +| /sys/kernel/security/dim | DIM file system directory, which is generated after the DIM kernel module is loaded and provides kernel interfaces for operating the DIM function| + +#### File Formats + +1. Measurement policy file format + + The lines, each representing a measurement policy, are in plaintext and are separated by Unix newline (LF) characters. The following configuration formats are supported: + + 1. User-mode process code section measurement + + ```text + measure obj=BPRM_TEXT path= + ``` + + 2. Kernel module code section measurement + + ```text + measure obj=MODULE_TEXT name= + ``` + + 3. Kernel measurement + + ```text + measure obj=KERNEL_TEXT + ``` + + **Example:** + + ```shell + # cat /etc/dim/policy + measure obj=BPRM_TEXT path=/usr/bin/bash + measure obj=BPRM_TEXT path=/usr/lib64/libc.so.6 + measure obj=MODULE_TEXT name=ext4 + measure obj=KERNEL_TEXT + ``` + +2. Static baseline file format + + The lines, each representing a static baseline, are in plaintext and are separated by Unix newline (LF) characters. The following configuration formats are supported: + + 1. User-mode process baseline + + ```text + dim USER sha256:6ae347be2d1ba03bf71d33c888a5c1b95262597fbc8d00ae484040408a605d2b + ``` + + 2. Kernel module baseline + + ```text + dim KERNEL sha256:a18bb578ff0b6043ec5c2b9b4f1c5fa6a70d05f8310a663ba40bb6e898007ac5 / + ``` + + 3. Kernel baseline + + ```text + dim KERNEL sha256:2ce2bc5d65e112ba691c6ab46d622fac1b7dbe45b77106631120dcc5441a3b9a + ``` + + **Example:** + + ```text + dim USER sha256:6ae347be2d1ba03bf71d33c888a5c1b95262597fbc8d00ae484040408a605d2b /usr/bin/bash + dim USER sha256:bc937f83dee4018f56cc823f5dafd0dfedc7b9872aa4568dc6fbe404594dc4d0 /usr/lib64/libc.so.6 + dim KERNEL sha256:a18bb578ff0b6043ec5c2b9b4f1c5fa6a70d05f8310a663ba40bb6e898007ac5 6.4.0-1.0.1.4.oe2309.x86_64/dim_monitor + dim KERNEL sha256:2ce2bc5d65e112ba691c6ab46d622fac1b7dbe45b77106631120dcc5441a3b9a 6.4.0-1.0.1.4.oe2309.x86_64 + ``` + +3. Measurement log format + + The lines, each representing a measurement log, are in plaintext and are separated by Unix newline (LF) characters. The format is as follows: + + ```text + : + ``` + + **Example:** + + 1. The code section of the bash process is measured. The measurement result is consistent with the static baseline. + + ```text + 12 0f384a6d24e121daf06532f808df624d5ffc061e20166976e89a7bb24158eb87 sha256:db032449f9e20ba37e0ec4a506d664f24f496bce95f2ed972419397951a3792e /usr/bin.bash [static baseline] + ``` + + 2. The code section of the bash process is measured. The measurement result is inconsistent with the static baseline. + + ```text + 12 0f384a6d24e121daf06532f808df624d5ffc061e20166976e89a7bb24158eb87 sha256:db032449f9e20ba37e0ec4a506d664f24f496bce95f2ed972419397951a3792e /usr/bin.bash [tampered] + ``` + + 3. The code section of the ext4 kernel module is measured. No static baseline is found. + + ```text + 12 0f384a6d24e121daf06532f808df624d5ffc061e20166976e89a7bb24158eb87 sha256:db032449f9e20ba37e0ec4a506d664f24f496bce95f2ed972419397951a3792e ext4 [no static baseline] + ``` + + 4. dim_monitor measures dim_core and records the measurement result of the baseline. + + ```text + 12 660d594ba050c3ec9a7cdc8cf226c5213c1e6eec50ba3ff51ff76e4273b3335a sha256:bdab94a05cc9f3ad36d29ebbd14aba8f6fd87c22ae580670d18154b684de366c dim_core.text [dynamic baseline] + 12 28a3cefc364c46caffca71e7c88d42cf3735516dec32796e4883edcf1241a7ea sha256:0dfd9656d6ecdadc8ec054a66e9ff0c746d946d67d932cd1cdb69780ccad6fb2 dim_core.data [dynamic baseline] + ``` + +4. Certificate and signature file formats + +The files are in the common format. For details, see [Enabling Signature Verification](#enabling-signature-verification). + +#### Kernel Module Parameters + +1. dim_core parameters + + | Parameter | Description | Value Range | Default Value | + | -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------ | ------------- | + | measure_log_capacity | Maximum number of measurement logs recorded by dim_core. When this value is reached, dim_core stops recording measurement logs. | 100-UINT_MAX (64-bit OS) | 100000 | + | measure_schedule | Scheduling time after a process or module is measured, in milliseconds. The value 0 indicates that measurement is not scheduled. | 0-1000 | 0 | + | measure_interval | Automatic measurement interval, in minutes. The value 0 indicates that automatic measurement is not enabled. | 0-525600 | 0 | + | measure_hash | Measurement hash algorithm | sha256, sm3 | sha256 | + | measure_pcr | The TPM PCR to which the measurement result is extended. The value 0 indicates that the measurement result is not extended. The value must be an actual TPM PCR number. | 0-128 | 0 | + | signature | Whether to enable the policy file and signature verification. The value 0 indicates that they are disabled, and the value 1 indicates that they are enabled. | 0, 1 | 0 | + + **Example:** + + ```shell + insmod /path/to/dim_core.ko measure_log_capacity=10000 measure_schedule=10 measure_pcr=12 + modprobe dim_core measure_log_capacity=10000 measure_schedule=10 measure_pcr=12 + ``` + +2. dim_monitor parameters + + | Parameter | Description | Value Range | Default Value| + | -------------------- | ------------------------------------------------------------ | ------------------------ | ------ | + | measure_log_capacity | Maximum number of measurement logs recorded by dim_monitor. When this value is reached, dim_monitor stops recording measurement logs.| 100-UINT_MAX (64-bit OS)| 100000 | + | measure_hash | Measurement hash algorithm | sha256, sm3 | sha256 | + | measure_pcr | The TPM PCR to which the measurement result is extended. The value 0 indicates that the measurement result is not extended. | 0-128 | 0 | + + **Example:** + + ```shell + insmod /path/to/dim_monitor.ko measure_log_capacity=10000 measure_hash=sm3 + modprobe dim_monitor measure_log_capacity=10000 measure_hash=sm3 + ``` + +#### Kernel Interfaces + +1. dim_core interface + + | Interface | Attribute| Function | Example | + | -------------------------- | ---- | ------------------------------------------------------------ | --------------------------------------------------------- | + | measure | Write-only| Write **1** to trigger dynamic measurement. If the operation is successful, **0** is returned. If the operation fails, an error code is returned. | echo 1 > /sys/kernel/security/dim/measure | + | baseline_init | Write-only| Write **1** to trigger dynamic baseline establishment. If the operation is successful, **0** is returned. If the operation fails, an error code is returned. | echo 1 > /sys/kernel/security/dim/baseline_init | + | ascii_runtime_measurements | Read-only| Read the interface to query measurement logs. | cat /sys/kernel/security/dim/ascii_runtime_measurements | + | runtime_status | Read-only| Read the interface to query the status information. If the operation fails, an error code is returned. | cat /sys/kernel/security/dim/runtime_status | + | interval | Read/Write| Write a numeric string to set the automatic measurement interval (the value range is the same as that of **measure_interval**). Read the interface to query the current automatic measurement interval. If the query fails, an error code is returned.| echo 10 > /sys/kernel/security/dim/interval
cat /sys/kernel/security/dim/interval | + + **dim_core Statuses** + + The possible statuses of dim_core are as follows: + + - DIM_NO_BASELINE: dim_core is loaded but no operation is performed. + - DIM_BASELINE_RUNNING: The dynamic baseline is being established. + - DIM_MEASURE_RUNNING: Dynamic measurement is being performed. + - DIM_PROTECTED: The dynamic baseline has been established and is protected. + - DIM_ERROR: An error occurs during dynamic baseline establishment or dynamic measurement. You need to rectify the error and trigger dynamic baseline establishment or dynamic measurement again. + +2. dim_monitor interfaces + + | Interface | Attribute| Description | Example | + | ---------------------------------- | ---- | ---------------------------------------------- | ------------------------------------------------------------ | + | monitor_run | Write-only| Write **1** to trigger measurement. If the operation is successful, **0** is returned. If the operation fails, an error code is returned.| echo 1 > /sys/kernel/security/dim/monitor_run | + | monitor_baseline | Write-only| Write **1** to trigger baseline establishment. If the operation is successful, **0** is returned. If the operation fails, an error code is returned.| echo 1 > /sys/kernel/security/dim/monitor_baseline | + | monitor_ascii_runtime_measurements | Read-only| Read the interface to query measurement logs. | cat /sys/kernel/security/dim/monitor_ascii_runtime_measurements | + | monitor_status | Read-only| Read the interface to query the status information. If the operation fails, an error code is returned. | cat /sys/kernel/security/dim/monitor_status | + + **dim_monitor Statuses** + + - ready: dim_monitor is loaded but no operation is performed. + - running: The dynamic baseline is being established or dynamic measurement is being performed. + - error: An error occurs during dynamic baseline establishment or dynamic measurement. You need to rectify the error and trigger dynamic baseline establishment or dynamic measurement again. + - protected: The dynamic baseline has been established and is protected. + +#### User-Mode Tool Interface + +See for the details of the `dim_gen_baseline` CLI interface. + +## Usage + +### Installing and Uninstalling DIM + +**Prerequisites**: + +- OS: openEuler 23.09 or later +- Kernel: openEuler kernel 5.10 or 6.4 + +Install dim_tools and dim software packages. The following uses openEuler 23.09 as an example: + +```shell +# yum install -y dim_tools dim +``` + +After the software packages are installed, the DIM kernel components are not loaded by default. You can run the following commands to load or unload them: + +```shell +# modprobe dim_core or insmod /path/to/dim_core.ko +# modprobe dim_monitor or insmod /path/to/dim_monitor.ko +# rmmod dim_monitor +# rmmod dim_core +``` + +After the components are loaded successfully, you can run the following commands to query the components: + +```shell +# lsmod | grep dim_core +dim_core 77824 1 dim_monitor +# lsmod | grep dim_monitor +dim_monitor 36864 0 +``` + +Unload the KO files before uninstalling the RPM package. + +```shell +# rmmod dim_monitor +# rmmod dim_core +# rpm -e dim +``` + +> [!NOTE]NOTE +> dim_monitor must be loaded after dim_core and removed before dim_core. +> You can also install DIM from source. For details, see . + +### Measuring Code Sections of User-Mode Processes + +**Prerequisites**: + +- The dim_core module is loaded successfully. + +- A user-mode resident measurement target program has been prepared. This section uses **/opt/dim/demo/dim_test_demo** as an example. + + ```shell + # /opt/dim/demo/dim_test_demo & + ``` + +**Step 1**: Generate a static baseline for the binary file corresponding to the measurement target process. + +```shell +# mkdir -p /etc/dim/digest_list +# dim_gen_baseline /opt/dim/demo/dim_test_demo -o /etc/dim/digest_list/test.hash +``` + +**Step 2**: Configure a measurement policy. + +```shell +# echo "measure obj=BPRM_TEXT path=/opt/dim/demo/dim_test_demo" > /etc/dim/policy +``` + +**Step 3**: Trigger dynamic baseline establishment. + +```shell +# echo 1 > /sys/kernel/security/dim/baseline_init +``` + +**Step 4**: Query measurement logs. + +```shell +# cat /sys/kernel/security/dim/ascii_runtime_measurements +0 e9a79e25f091e03a8b3972b1a0e4ae2ccaed1f5652857fe3b4dc947801a6913e sha256:02e28dff9997e1d81fb806ee5b784fd853eac8812059c4dba7c119c5e5076989 /opt/dim/demo/dim_test_demo [static baseline] +``` + +If the preceding measurement log is displayed, the target process is measured successfully and the measurement result is consistent with the static baseline. + +**Step 5**: Trigger dynamic measurement. + +```shell +# echo 1 > /sys/kernel/security/dim/measure +``` + +After the measurement is complete, you can perform **Step 4** to query the measurement logs. If the measurement result is consistent with the dynamic baseline, the measurement logs are not updated. Otherwise, an exception measurement log is added. If an attacker attempts to tamper with the target program (for example, by modifying and recompiling the code) and restart the target program, for example: + +```shell +# pkill dim_test_demo +# /opt/dim/demo/dim_test_demo & +``` + +Trigger the measurement again and query the measurement logs. You can see a measurement log marked with "tampered." + +```shell +# echo 1 > /sys/kernel/security/dim/measure +# cat /sys/kernel/security/dim/ascii_runtime_measurements +0 e9a79e25f091e03a8b3972b1a0e4ae2ccaed1f5652857fe3b4dc947801a6913e sha256:02e28dff9997e1d81fb806ee5b784fd853eac8812059c4dba7c119c5e5076989 /opt/dim/demo/dim_test_demo [static baseline] +0 08a2f6f2922ad3d1cf376ae05cf0cc507c2f5a1c605adf445506bc84826531d6 sha256:855ec9a890ff22034f7e13b78c2089e28e8d217491665b39203b50ab47b111c8 /opt/dim/demo/dim_test_demo [tampered] +``` + +### Measuring Code Sections of Kernel Modules + +**Prerequisites**: + +- The dim_core module is loaded successfully. + +- A measurement target kernel module has been prepared. This section uses dim_test_module as an example, whose path is **/opt/dim/demo/dim_test_module.ko**. + + ```shell + # insmod /opt/dim/demo/dim_test_module.ko + ``` + +> [!NOTE]NOTE +> The kernel version of the environment where the module is compiled must be the same as the current kernel version. Run the following command to confirm: +> +> ```shell +> # modinfo dim_monitor.ko | grep vermagic | grep "$(uname -r)" +> vermagic: 6.4.0-1.0.1.4.oe2309.x86_64 SMP preempt mod_unload modversions +> ``` + +The first field of the vermagic kernel module information must be the same as the current kernel version. + +**Step 1**: Generate a static baseline for the measurement target kernel module. + +```shell +# mkdir -p /etc/dim/digest_list +# dim_gen_baseline /opt/dim/demo/dim_test_module.ko -o /etc/dim/digest_list/test.hash +``` + +**Step 2**: Configure a measurement policy. + +```shell +# echo "measure obj=MODULE_TEXT name=dim_test_module" > /etc/dim/policy +``` + +**Step 3**: Trigger dynamic baseline establishment. + +```shell +# echo 1 > /sys/kernel/security/dim/baseline_init +``` + +**Step 4**: Query measurement logs. + +```shell +# cat /sys/kernel/security/dim/ascii_runtime_measurements +0 9603a9d5f87851c8eb7d2619f7abbe28cb8a91f9c83f5ea59f036794e23d1558 sha256:9da4bccc7ae1b709deab8f583b244822d52f3552c93f70534932ae21fac931c6 dim_test_module [static baseline] +``` + +The preceding measurement log indicates that dim_test_module is successfully measured and the current measurement result is used as the baseline value for subsequent measurement. The hash value in the measurement log is not the actual measurement value. + +**Step 5**: Trigger dynamic measurement. + +```shell +echo 1 > /sys/kernel/security/dim/measure +``` + +After the measurement is complete, you can perform **Step 4** to query the measurement logs. If the measurement result is consistent with the dynamic baseline, the measurement logs are not updated. Otherwise, an exception measurement log is added. If an attacker attempts to tamper with the target kernel module (for example, by modifying and recompiling the code) and reload the module, for example: + +```shell +rmmod dim_test_module +insmod /opt/dim/demo/dim_test_module.ko +``` + +Trigger the measurement again and query the measurement logs. You can see a measurement log marked with "tampered." + +```shell +# cat /sys/kernel/security/dim/ascii_runtime_measurements +0 9603a9d5f87851c8eb7d2619f7abbe28cb8a91f9c83f5ea59f036794e23d1558 sha256:9da4bccc7ae1b709deab8f583b244822d52f3552c93f70534932ae21fac931c6 dim_test_module [static baseline] +0 6205915fe63a7042788c919d4f0ff04cc5170647d7053a1fe67f6c0943cd1f40 sha256:4cb77370787323140cb572a789703be1a4168359716a01bf745aa05de68a14e3 dim_test_module [tampered] +``` + +### Measuring Code Sections of the Kernel + +**Prerequisites**: + +- The dim_core module is loaded successfully. + +**Step 1**: generate a static baseline for the kernel. + +```shell +# mkdir -p /etc/dim/digest_list +# dim_gen_baseline -k "$(uname -r)" -o /etc/dim/digest_list/test.hash /boot/vmlinuz-6.4.0-1.0.1.4.oe2309.x86_64 +``` + +> [!NOTE]NOTE +> The file name **/boot/vmlinuz-6.4.0-1.0.1.4.oe2309.x86_64** is used as an example. + +**Step 2**: Configure a DIM policy. + +```shell +# echo "measure obj=KERNEL_TEXT" > /etc/dim/policy +``` + +**Step 3**: Trigger dynamic baseline establishment. + +```shell +# echo 1 > /sys/kernel/security/dim/baseline_init +``` + +**Step 4**: Query measurement logs. + +```shell +# cat /sys/kernel/security/dim/ascii_runtime_measurements +0 ef82c39d767dece1f5c52b31d1e8c7d55541bae68a97542dda61b0c0c01af4d2 sha256:5f1586e95b102cd9b9f7df3585fe13a1306cbd464f2ebe47a51ad34128f5d0af 6.4.0-1.0.1.4.oe2309.x86_64 [static baseline] +``` + +The preceding measurement log indicates that the kernel is successfully measured and the current measurement result is used as the baseline value for subsequent measurement. The hash value in the measurement log is not the actual measurement value. + +**Step 5**: Trigger dynamic measurement. + +```shell +# echo 1 > /sys/kernel/security/dim/measure +``` + +After the measurement is complete, you can perform **Step 4** to query the measurement logs. If the measurement result is consistent with the dynamic baseline, the measurement logs are not updated. Otherwise, an exception measurement log is added. + +### Measuring the dim_core Module + +**Prerequisites**: + +- The dim_core and dim_monitor modules are loaded successfully. +- The measurement policy has been configured. + +**Step 1**: Trigger dynamic baseline establishment for dim_core. + +```shell +# echo 1 > /sys/kernel/security/dim/baseline_init +``` + +**Step2**: Trigger dynamic baseline establishment for dim_monitor. + +```shell +# echo 1 > /sys/kernel/security/dim/monitor_baseline +``` + +**Step 3**: Query dim_monitor measurement logs. + +```shell +# cat /sys/kernel/security/dim/monitor_ascii_runtime_measurements +0 c1b0d9909ddb00633fc6bbe7e457b46b57e165166b8422e81014bdd3e6862899 sha256:35494ed41109ebc9bf9bf7b1c190b7e890e2f7ce62ca1920397cd2c02a057796 dim_core.text [dynamic baseline] +0 9be7121cd2c215d454db2a8aead36b03d2ed94fad0fbaacfbca83d57a410674f sha256:f35d20aae19ada5e633d2fde6e93133c3b6ae9f494ef354ebe5b162398e4d7fa dim_core.data [dynamic baseline] +``` + +The preceding measurement log indicates that dim_core is successfully measured and the current measurement result is used as the baseline value for subsequent measurement. +> [!NOTE]NOTE +> If you skip dynamic baseline establishment and directly perform measurement, "tampered" is displayed in the log. + +**Step 4**: Trigger dynamic measurement of dim_monitor. + +```shell +# echo 1 > /sys/kernel/security/dim/monitor_run +``` + +If the measurement result is consistent with the dynamic baseline, the measurement logs are not updated. Otherwise, an exception measurement log is added. If dynamic baseline establishment for dim_core is triggered again after the policy is modified, the measurement target changes, and the baseline data managed by dim_core also changes. As a result, the dim_monitor measurement result changes. + +```shell +# echo "measure obj=BPRM_TEXT path=/usr/bin/bash" > /etc/dim/policy +# echo 1 > /sys/kernel/security/dim/baseline_init +``` + +Trigger the measurement of dim_monitor again and query the measurement logs. You can see a measurement log marked with "tampered." + +```shell +# echo 1 > /sys/kernel/security/dim/monitor_run +# cat /sys/kernel/security/dim/monitor_ascii_runtime_measurements +0 c1b0d9909ddb00633fc6bbe7e457b46b57e165166b8422e81014bdd3e6862899 sha256:35494ed41109ebc9bf9bf7b1c190b7e890e2f7ce62ca1920397cd2c02a057796 dim_core.text [dynamic baseline] +0 9be7121cd2c215d454db2a8aead36b03d2ed94fad0fbaacfbca83d57a410674f sha256:f35d20aae19ada5e633d2fde6e93133c3b6ae9f494ef354ebe5b162398e4d7fa dim_core.data [dynamic baseline] +0 6a60d78230954aba2e6ea6a6b20a7b803d7adb405acbb49b297c003366cfec0d sha256:449ba11b0bfc6146d4479edea2b691aa37c0c025a733e167fd97e77bbb4b9dab dim_core.data [tampered] +``` + +### Extending to the TPM PCR + +**Prerequisites**: + +- The TPM 2.0 has been installed in the system. After the following command is executed, the command output is not empty: + + ```shell + # ls /dev/tpm* + /dev/tpm0 /dev/tpmrm0 + ``` + +- The tpm2-tools software package has been installed in the system. After the following command is executed, the command output is not empty: + + ```shell + # rpm -qa tpm2-tools + ``` + +- The measurement policy and static baseline have been configured. + +**Step 1**: Load the dim_core and dim_monitor modules and set the numbers of the PCRs to which the measurement results are extended. In this example, PCR 12 is specified for the dim_core measurement result, and PCR 13 is specified for the dim_monitor measurement result. + +```shell +# modprobe dim_core measure_pcr=12 +# modprobe dim_monitor measure_pcr=13 +``` + +**Step 2**: Trigger baseline establishment for dim_core and dim_monitor. + +```shell +# echo 1 > /sys/kernel/security/dim/baseline_init +# echo 1 > /sys/kernel/security/dim/monitor_baseline +``` + +**Step 3**: Check the measurement logs. Each log is marked with the corresponding TPM PCR number. + +```shell +# cat /sys/kernel/security/dim/ascii_runtime_measurements +12 2649c414d1f9fcac1c8d0df8ae7b1c18b5ea10a162b957839bdb8f8415ec6146 sha256:83110ce600e744982d3676202576d8b94cea016a088f99617767ddbd66da1164 /usr/lib/systemd/systemd [static baseline] +# cat /sys/kernel/security/dim/monitor_ascii_runtime_measurements +13 72ee3061d5a80eb8547cd80c73a80c3a8dc3b3e9f7e5baa10f709350b3058063 sha256:5562ed25fcdf557efe8077e231399bcfbcf0160d726201ac8edf7a2ca7c55ad0 dim_core.text [dynamic baseline] +13 8ba44d557a9855c03bc243a8ba2d553347a52c1a322ea9cf8d3d1e0c8f0e2656 sha256:5279eadc235d80bf66ba652b5d0a2c7afd253ebaf1d03e6e24b87b7f7e94fa02 dim_core.data [dynamic baseline] +``` + +**Step 4**: Check the TPM PCRs. Extended values have been written to the corresponding PCRs. + +```shell +# tpm2_pcrread sha256 | grep "12:" + 12: 0xF358AC6F815BB29D53356DA2B4578B4EE26EB9274E553689094208E444D5D9A2 +# tpm2_pcrread sha256 | grep "13:" + 13: 0xBFB9FF69493DEF9C50E52E38B332BDA8DE9C53E90FB96D14CD299E756205F8EA +``` + +### Enabling Signature Verification + +**Prerequisites**: + +- A public key certificate and a signature private key have been prepared. The signature algorithm is RSA, the hash algorithm is sha256, and the certificate format is DER. You can also run the following commands to generate the files: + + ```shell + # openssl genrsa -out dim.key 4096 + # openssl req -new -sha256 -key dim.key -out dim.csr -subj "/C=AA/ST=BB/O=CC/OU=DD/CN=DIM Test" + # openssl x509 -req -days 3650 -signkey dim.key -in dim.csr -out dim.crt + # openssl x509 -in dim.crt -out dim.der -outform DER + ``` + +- The measurement policy has been configured. + +**Step 1**: Save the DER certificate as **/etc/keys/x509_dim.der**. + +```shell +# mkdir -p /etc/keys +# cp dim.der /etc/keys/x509_dim.der +``` + +**Step 2**: Sign the policy file and static baseline file. The signature file name must be the original file name suffixed with **.sig**. + +```shell +# openssl dgst -sha256 -out /etc/dim/policy.sig -sign dim.key /etc/dim/policy +# openssl dgst -sha256 -out /etc/dim/digest_list/test.hash.sig -sign dim.key /etc/dim/digest_list/test.hash +``` + +**Step 3**: Load the dim_core module and enable the signature verification function. + +```shell +modprobe dim_core signature=1 +``` + +Now, the policy file and static baseline file can be loaded only after they pass the signature verification. +The baseline establishment will fail if it is triggered after the policy file is modified: + +```shell +# echo "" >> /etc/dim/policy +# echo 1 > /sys/kernel/security/dim/baseline_init +-bash: echo: write error: Key was rejected by service +``` + +> [!NOTE]NOTE +> If the signature verification of a static baseline file fails, dim_core skips the parsing of the file without causing a baseline establishment failure. + +### Configuring Measurement Algorithms + +**Prerequisites**: + +- The measurement policy has been configured. + +**Step 1**: Load the dim_core and dim_monitor modules and configure the measurement algorithm. The following uses the SM3 algorithm as an example. + +```shell +# modprobe dim_core measure_hash=sm3 +# modprobe dim_monitor measure_hash=sm3 +``` + +**Step 2**: Configure a policy that establishes a static baseline of the SM3 algorithm for the measurement target program. + +```shell +# echo "measure obj=BPRM_TEXT path=/opt/dim/demo/dim_test_demo" > /etc/dim/policy +# dim_gen_baseline -a sm3 /opt/dim/demo/dim_test_demo -o /etc/dim/digest_list/test.hash +``` + +**Step 3**: Trigger baseline establishment. + +```shell +# echo 1 > /sys/kernel/security/dim/baseline_init +# echo 1 > /sys/kernel/security/dim/monitor_baseline +``` + +**Step 4**: Check the measurement logs. Each log is marked with the corresponding hash algorithm. + +```shell +# cat /sys/kernel/security/dim/ascii_runtime_measurements +0 585a64feea8dd1ec415d4e67c33633b97abb9f88e6732c8a039064351d24eed6 sm3:ca84504c02bef360ec77f3280552c006ce387ebb09b49b316d1a0b7f43039142 /opt/dim/demo/dim_test_demo [static baseline] +# cat /sys/kernel/security/dim/monitor_ascii_runtime_measurements +0 e6a40553499d4cbf0501f32cabcad8d872416ca12855a389215b2509af76e60b sm3:47a1dae98182e9d7fa489671f20c3542e0e154d3ce941440cdd4a1e4eee8f39f dim_core.text [dynamic baseline] +0 2c862bb477b342e9ac7d4dd03b6e6705c19e0835efc15da38aafba110b41b3d1 sm3:a4d31d5f4d5f08458717b520941c2aefa0b72dc8640a33ee30c26a9dab74eae9 dim_core.data [dynamic baseline] +``` + +### Configuring Automatic Measurement + +**Prerequisites**: + +- The measurement policy has been configured. + +**Method 1**: Load the dim_core module and set the automatic measurement interval to 1 minute. + +```shell +modprobe dim_core measure_interval=1 +``` + +After the module is loaded, dynamic baseline establishment is automatically triggered. Then, dynamic measurement is triggered every minute. + +> [!NOTE]NOTE +> In this case, do not configure the measurement policy for dim_core to measure its own code sections. Otherwise, false alarms may occur. +> In addition, you need to configure **/etc/dim/policy** in advance. Otherwise, the module fails to be loaded when **measure_interval** is set to **1**. + +**Method 2**: After the dim_core module is loaded, you can configure the measurement interval through the kernel module interface. In the following example, the measurement interval is set to 1 minute. + +```shell +modprobe dim_core +echo 1 > /sys/kernel/security/dim/interval +``` + +In this case, the measurement is not triggered immediately. One minute later, dynamic baseline establishment, or dynamic measurement, is triggered. Subsequently, dynamic measurement is triggered every minute. + +### Configuring the Measurement Scheduling Time + +**Prerequisites**: + +- The measurement policy has been configured. + +Load the dim_core module and set the measurement scheduling time to 10 ms. + +```shell +modprobe dim_core measure_schedule=10 +``` + +When the dynamic baseline establishment or dynamic measurement is triggered, dim_core schedules the CPU to release 10 ms each time a process is measured. diff --git a/docs/en/server/security/trusted_computing/figures/RA-arch-1.png b/docs/en/server/security/trusted_computing/figures/RA-arch-1.png new file mode 100644 index 0000000000000000000000000000000000000000..0ad1375a27cd61abf9f06518dbe1c01554623efd Binary files /dev/null and b/docs/en/server/security/trusted_computing/figures/RA-arch-1.png differ diff --git a/docs/en/server/security/trusted_computing/figures/RA-arch-2.png b/docs/en/server/security/trusted_computing/figures/RA-arch-2.png new file mode 100644 index 0000000000000000000000000000000000000000..19c7a1ee60422eb13d8a300514f78a63d1640394 Binary files /dev/null and b/docs/en/server/security/trusted_computing/figures/RA-arch-2.png differ diff --git a/docs/en/server/security/trusted_computing/figures/TPCM.png b/docs/en/server/security/trusted_computing/figures/TPCM.png new file mode 100644 index 0000000000000000000000000000000000000000..50882fb08433ee3ce187b3846bd6ec4a9f6d6818 Binary files /dev/null and b/docs/en/server/security/trusted_computing/figures/TPCM.png differ diff --git a/docs/en/server/security/trusted_computing/figures/dim_architecture.jpg b/docs/en/server/security/trusted_computing/figures/dim_architecture.jpg new file mode 100644 index 0000000000000000000000000000000000000000..b0561e749098f906f1eeef06366569941602a1ca Binary files /dev/null and b/docs/en/server/security/trusted_computing/figures/dim_architecture.jpg differ diff --git a/docs/en/server/security/trusted_computing/figures/ima-modsig.png b/docs/en/server/security/trusted_computing/figures/ima-modsig.png new file mode 100644 index 0000000000000000000000000000000000000000..c3e54e27b6ce30bd21e97908b6168a73f318c117 Binary files /dev/null and b/docs/en/server/security/trusted_computing/figures/ima-modsig.png differ diff --git a/docs/en/server/security/trusted_computing/figures/ima_digest_list_flow.png b/docs/en/server/security/trusted_computing/figures/ima_digest_list_flow.png new file mode 100644 index 0000000000000000000000000000000000000000..11711ca21c6b327c3d347ad4c389d037a6c2c6ae Binary files /dev/null and b/docs/en/server/security/trusted_computing/figures/ima_digest_list_flow.png differ diff --git a/docs/en/server/security/trusted_computing/figures/ima_digest_list_pkg.png b/docs/en/server/security/trusted_computing/figures/ima_digest_list_pkg.png new file mode 100644 index 0000000000000000000000000000000000000000..8a2128add583d3c25ee5f281bb882c94f23b97c7 Binary files /dev/null and b/docs/en/server/security/trusted_computing/figures/ima_digest_list_pkg.png differ diff --git a/docs/en/server/security/trusted_computing/figures/ima_priv_key.png b/docs/en/server/security/trusted_computing/figures/ima_priv_key.png new file mode 100644 index 0000000000000000000000000000000000000000..c939b8e2e8bcd30869f938161ea1edbccd9c89c4 Binary files /dev/null and b/docs/en/server/security/trusted_computing/figures/ima_priv_key.png differ diff --git a/docs/en/server/security/trusted_computing/figures/ima_rpm.png b/docs/en/server/security/trusted_computing/figures/ima_rpm.png new file mode 100644 index 0000000000000000000000000000000000000000..6c4b620ded02ee96357eb587890555af5a319e51 Binary files /dev/null and b/docs/en/server/security/trusted_computing/figures/ima_rpm.png differ diff --git a/docs/en/server/security/trusted_computing/figures/ima_secure_boot.png b/docs/en/server/security/trusted_computing/figures/ima_secure_boot.png new file mode 100644 index 0000000000000000000000000000000000000000..85b959ff1da0f4bcf919f6fea712a0c053f7ad01 Binary files /dev/null and b/docs/en/server/security/trusted_computing/figures/ima_secure_boot.png differ diff --git a/docs/en/server/security/trusted_computing/figures/ima_sig_verify.png b/docs/en/server/security/trusted_computing/figures/ima_sig_verify.png new file mode 100644 index 0000000000000000000000000000000000000000..e0d7ff55ab93dca65763881ba4ff136b85521123 Binary files /dev/null and b/docs/en/server/security/trusted_computing/figures/ima_sig_verify.png differ diff --git a/docs/en/server/security/trusted_computing/figures/ima_tpm.png b/docs/en/server/security/trusted_computing/figures/ima_tpm.png new file mode 100644 index 0000000000000000000000000000000000000000..931440ebdb8a8c993a2f9ef331b214b40d8f9535 Binary files /dev/null and b/docs/en/server/security/trusted_computing/figures/ima_tpm.png differ diff --git a/docs/en/server/security/trusted_computing/figures/ima_trusted_measurement.png b/docs/en/server/security/trusted_computing/figures/ima_trusted_measurement.png new file mode 100644 index 0000000000000000000000000000000000000000..e64224fdf4b99429aeabb87947de2c9f23f1df14 Binary files /dev/null and b/docs/en/server/security/trusted_computing/figures/ima_trusted_measurement.png differ diff --git a/docs/en/server/security/trusted_computing/figures/trusted_chain.png b/docs/en/server/security/trusted_computing/figures/trusted_chain.png new file mode 100644 index 0000000000000000000000000000000000000000..034f0f092f41fb500ee4122339c447d10d4138ec Binary files /dev/null and b/docs/en/server/security/trusted_computing/figures/trusted_chain.png differ diff --git a/docs/en/server/security/trusted_computing/ima.md b/docs/en/server/security/trusted_computing/ima.md new file mode 100644 index 0000000000000000000000000000000000000000..b466ba105977c49b575566e4b108502df9560c9f --- /dev/null +++ b/docs/en/server/security/trusted_computing/ima.md @@ -0,0 +1,1161 @@ +# Kernel Integrity Measurement Architecture (IMA) + +## Overview + +### Introduction to IMA + +IMA is a kernel subsystem that measures files accessed through system calls like `execve()`, `mmap()`, and `open()` based on custom policies. These measurements can be used for **local or remote attestation** or compared against reference values to **control file access**. + +IMA operates in two main modes: + +- Measurement: This mode observes the integrity of files. When protected files are accessed, measurement records are added to the measurement log in kernel memory. If the system has a Trusted Platform Module (TPM), the measurement digest can also be extended into the TPM platform configuration registers (PCRs) to ensure the integrity of the measurement data. This mode does not restrict file access but provides recorded file information to upper-layer applications for remote attestation. +- Appraisal: This mode verifies file integrity, preventing access to unknown or tampered files. It uses cryptographic methods like hashes, signatures, and hash-based message authentication codes (HMACs) to validate file contents. If verification fails, the file is inaccessible to any process. This feature enhances system resilience by isolating compromised files and preventing further damage during an attack. + +In summary, the measurement mode acts as a passive observer, while the appraisal mode enforces strict access control, blocking any file that fails integrity checks. + +### Introduction to EVM + +EVM, or Extended Verification Module, builds on IMA capabilities. While IMA protects file contents, EVM extends this protection to file attributes such as `uid`, `security.ima`, and `security.selinux`. + +### Introduction to IMA Digest Lists + +IMA digest lists enhance the native integrity protection mechanism of the kernel in openEuler, addressing key limitations of the native IMA/EVM mechanism: + +**File access performance degradation due to extension to TPM** + +In IMA measurement mode, each measurement requires accessing the TPM. Since the TPM operates at low speeds, typically using the Serial Peripheral Interface (SPI) protocol with clock frequencies in the tens of MHz, system call performance suffers. + +![](./figures/ima_tpm.png) + +**File access performance degradation due to asymmetric operations** + +In IMA appraisal mode, immutable files are protected using a signature mechanism. Each file verification requires signature validation, and the complexity of asymmetric operations further degrades system call performance. + +![](./figures/ima_sig_verify.png) + +**Decreased efficiency and security from complex deployment** + +In IMA appraisal mode, deployment requires the system to first enter fix mode to mark IMA/EVM extended attributes, then switch to appraisal mode for startup. Additionally, upgrading protected files necessitates rebooting into fix mode to update files and extended attributes. This process reduces deployment efficiency and exposes keys in the runtime environment, compromising security. + +![](./figures/ima_priv_key.png) + +IMA digest lists address these issues by managing baseline digest values for multiple files through a single hash list file. This file consolidates the baseline digest values of files (such as all executables in a software package) for centralized management. The baseline digest values can include file content digests (for IMA mode) and file extended attribute digests (for EVM mode). This consolidated file is the IMA digest list file. + +![](./figures/ima_digest_list_pkg.png) + +When the IMA digest list feature is enabled, the kernel maintains a hash allowlist pool to store digest values from imported IMA digest list files. It also provides interfaces via securityfs for importing, deleting, and querying these files. + +In measurement mode, imported digest list files must undergo measurement and extension to TPM before being added to the allowlist pool. If the digest value of a target file matches the allowlist pool, no additional measurement logging or extension to TPM is required. In appraisal mode, imported digest list files must pass signature verification before being added to the allowlist pool. The digest value of the accessed target file is then matched against the allowlist pool to determine the appraisal result. + +![](./figures/ima_digest_list_flow.png) + +Compared to the native Linux IMA/EVM mechanism, the IMA digest list extension enhances security, performance, and usability for better practical implementation: + +- Security: IMA digest lists are distributed with software packages. During installation, digest lists are imported simultaneously, ensuring baseline values originate from the software distributor (like the openEuler community). This eliminates the need to generate baseline values in the runtime environment, establishing a complete trust chain. +- Performance: The IMA digest list mechanism operates on a per-digest-list basis, reducing TPM access and asymmetric operation frequency to 1/n (where n is the average number of file hashes managed by a single digest list). This improves system call and boot performance. +- Usability: The IMA digest list mechanism supports out-of-the-box functionality, allowing the system to enter appraisal mode immediately after installation. It also enables software package installation and upgrades in appraisal mode without requiring fix mode for file marking, facilitating rapid deployment and seamless updates. + +It is worth noting that, unlike the native IMA/EVM, the IMA digest list stores baseline values for measurement/appraisal in kernel memory. This assumes that kernel memory cannot be tampered with by unauthorized entities. Therefore, the IMA digest list relies on additional security mechanisms (including kernel module secure boot and runtime memory measurement) to safeguard kernel memory integrity. + +However, both the native IMA mechanism and the IMA digest list extension are only components of the system security chain. Neither can independently ensure system security. Security is inherently a systematic, defense-in-depth engineering effort. + +## Interface Description + +### Kernel Boot Parameters + +The openEuler IMA/EVM mechanism provides the following kernel boot parameters. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ParameterValueFunction
ima_appraiseenforce-evmEnable IMA appraisal enforce mode (EVM enabled).
log-evmEnable IMA appraisal log mode (EVM enabled).
enforceEnable IMA appraisal enforce mode.
logEnable IMA appraisal log mode.
offDisable IMA appraisal.
ima_appraise_digest_listdigestEnable IMA+EVM appraisal based on digest lists (comparing file content and extended attributes).
digest-nometadataEnable IMA appraisal based on digest lists (comparing file content only).
evmx509Enable portable signature-based EVM directly (regardless of EVM certificate loading)
completePrevent modification of EVM mode via securityfs after boot.
allow_metadata_writesAllow file metadata modifications without EVM interception.
ima_hashsha256/sha1/...Specify the IMA measurement hash algorithm.
ima_templateimaSpecify the IMA measurement template (d or n).
ima-ngSpecify the IMA measurement template (d-ng or n-ng), default template.
ima-sigSpecify the IMA measurement template (d-ng, n-ng, or sig).
ima_policyexec_tcbMeasure all files accessed via execution or mapping, including loaded kernel modules, firmware, and kernel files.
tcbExtend exec_tcb policy to measure files accessed with uid=0 or euid=0.
secure_bootAppraise all loaded kernel modules, firmware, and kernel files, using IMA signature mode.
appraise_exec_tcbExtend secure_boot policy to appraise all files accessed via execution or mapping.
appraise_tcbAppraise all files owned by uid=0.
appraise_exec_immutableUsed with the appraise_exec_tcb policy, making executable file extended attributes immutable.
ima_digest_list_pcr10Extend IMA measurement results based on digest list in PCR 10, disable native IMA measurement.
11Extend IMA measurement results based on digest list in PCR 11, disable native IMA measurement.
+11Extend IMA measurement results based on digest list in PCR 11, extend native IMA measurement results in PCR 10.
ima_digest_db_sizenn[M]Set kernel digest list size limit (0 to 64 MB), defaulting to 16 MB if not configured. ("Not configured" means to omit the parameter, not leaving the value cannot blank like ima_digest_db_size=.)
ima_capacity-1 to 2147483647Set the kernel measurement log entry limit, defaulting to 100,000. -1 means no limit.
initramtmpfsNoneSupport tmpfs in initrd to carry file extended attributes.
+ +Based on user scenarios, the following parameter combinations are recommended: + +**(1) Native IMA measurement** + +```ini +# Native IMA measurement + custom policy +# No configuration required. This is enabled by default. +# Native IMA measurement + TCB default policy +ima_policy="tcb" +``` + +**(2) IMA measurement based on digest list** + +```ini +# Digest list IMA measurement + custom policy +ima_digest_list_pcr=11 ima_template=ima-ng initramtmpfs +# Digest list IMA measurement + default policy +ima_digest_list_pcr=11 ima_template=ima-ng ima_policy="exec_tcb" initramtmpfs +``` + +**(3) IMA appraisal based on digest list, protecting file content only** + +```ini +# IMA appraisal + log mode +ima_appraise=log ima_appraise_digest_list=digest-nometadata ima_policy="appraise_exec_tcb" initramtmpfs +# IMA appraisal + enforce mode +ima_appraise=enforce ima_appraise_digest_list=digest-nometadata ima_policy="appraise_exec_tcb" initramtmpfs +``` + +**(4) IMA appraisal based on digest list, protecting file content and extended attributes** + +```ini +# IMA appraisal + log mode +ima_appraise=log-evm ima_appraise_digest_list=digest ima_policy="appraise_exec_tcb|appraise_exec_immutable" initramtmpfs evm=x509 evm=complete +# IMA appraisal + enforce mode +ima_appraise=enforce-evm ima_appraise_digest_list=digest ima_policy="appraise_exec_tcb|appraise_exec_immutable" initramtmpfs evm=x509 evm=complete +``` + +> ![](./public_sys-resources/icon-note.gif) **Note:** +> All four parameter sets above can be used individually, but only digest list-based measurement and appraisal modes can be combined, such as (2) with (3) or (2) with (4). + +### securityfs Interface Description + +The securityfs interfaces provided by openEuler IMA are located in the **/sys/kernel/security** directory. Below are the interface names and their descriptions. + +| Path | Permissions | Description | +| :----------------------------- | :---------- | :---------------------------------------------------------------------- | +| ima/policy | 600 | Display or import IMA policies. | +| ima/ascii_runtime_measurement | 440 | Display IMA measurement logs in ASCII format. | +| ima/binary_runtime_measurement | 440 | Display IMA measurement logs in binary format. | +| ima/runtime_measurement_count | 440 | Display the count of IMA measurement log entries. | +| ima/violations | 440 | Display the number of abnormal IMA measurement logs. | +| ima/digests_count | 440 | Display the total number of digests in the system hash table (IMA+EVM). | +| ima/digest_list_data | 200 | Add digest lists. | +| ima/digest_list_data_del | 200 | Delete digest lists. | +| evm | 660 | Query or set EVM mode. | + +The **/sys/kernel/security/evm** interface supports the following values: + +- `0`: EVM is not initialized. +- `1`: Use HMAC (symmetric encryption) to verify extended attribute integrity. +- `2`: Use public key signature verification (asymmetric encryption) to verify extended attribute integrity. +- `6`: Disable extended attribute integrity verification. + +### Digest List Management Tool Description + +The digest-list-tools package includes tools for generating and managing IMA digest list files. The primary CLI tools are as follows. + +#### gen_digest_lists + +The `gen_digest_lists` tool allows users to generate digest lists. The command options are defined below. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
OptionValueFunction
-d<path>Specify the directory to store the generated digest list files. The directory must be valid.
-fcompactSpecify the format of the generated digest list files. Currently, only the compact format is supported.
-i<option arg>:<option value>Define the target file range for generating digest lists. Specific parameters are listed below.
I:<path>Specify the absolute path of files for which digest lists will be generated. If a directory is specified, recursive generation is performed.
E:<path>Specify paths or directories to exclude.
F:<path>Specify paths or directories to generate digest lists for all files under them (ignores the filtering effect of e: when combined with e:).
e:Generate digest lists only for executable files.
l:policyMatch file security contexts from the SELinux policy instead of reading them directly from file extended attributes.
i:Include the file digest value in the calculated extended attribute information when generating metadata-type digest lists (required).
M:Allow explicit specification of file extended attribute information (requires use with the rpmbuild command).
u:Use the list file name specified by the L: parameter as the name of the generated digest list file (requires use with the rpmbuild command).
L:<path>Specify the path to the list file, which contains the information data required to generate digest lists (requires use with the rpmbuild command).
-oaddSpecify the operation for generating digest lists. Currently, only the add operation is supported, which adds the digest list to the file.
-p-1Specify the position in the file where the digest list will be written. Currently, only -1 is supported.
-tfileGenerate digest lists only for file content.
metadataGenerate digest lists for both file content and extended attributes.
-TN/AIf this option is not used, digest list files are generated. Otherwise, TLV digest list files are generated.
-A<path>Specify the relative root directory to truncate the file path prefix for path matching and SELinux label matching.
-mimmutableSpecify the modifiers attribute for the generated digest list files. Currently, only immutable is supported. In enforce/enforce-evm mode, digest lists can only be opened in read-only mode.
-hN/APrint help information.
+ +**Usage examples** + +- Scenario 1: Generate a digest list/TLV digest list for a single file. + + ```shell + gen_digest_lists -t metadata -f compact -i l:policy -o add -p -1 -m immutable -i I:/usr/bin/ls -d ./ -i i: + gen_digest_lists -t metadata -f compact -i l:policy -o add -p -1 -m immutable -i I:/usr/bin/ls -d ./ -i i: -T + ``` + +- Scenario 2: Generate a digest list/TLV digest list for a single file and specify a relative root directory. + + ```shell + gen_digest_lists -t metadata -f compact -i l:policy -o add -p -1 -m immutable -i I:/usr/bin/ls -A /usr/ -d ./ -i i: + gen_digest_lists -t metadata -f compact -i l:policy -o add -p -1 -m immutable -i I:/usr/bin/ls -A /usr/ -d ./ -i i: -T + ``` + +- Scenario 3: Recursively generate a digest list/TLV digest list for files in a directory. + + ```shell + gen_digest_lists -t metadata -f compact -i l:policy -o add -p -1 -m immutable -i I:/usr/bin/ -d ./ -i i: + gen_digest_lists -t metadata -f compact -i l:policy -o add -p -1 -m immutable -i I:/usr/bin/ -d ./ -i i: -T + ``` + +- Scenario 4: Recursively generate a digest list/TLV digest list for executable files in a directory. + + ```shell + gen_digest_lists -t metadata -f compact -i l:policy -o add -p -1 -m immutable -i I:/usr/bin/ -d ./ -i i: -i e: + gen_digest_lists -t metadata -f compact -i l:policy -o add -p -1 -m immutable -i I:/usr/bin/ -d ./ -i i: -i e: -T + ``` + +- Scenario 5: Recursively generate a digest list/TLV digest list for files in a directory, excluding specific subdirectories. + + ```shell + gen_digest_lists -t metadata -f compact -i l:policy -o add -p -1 -m immutable -i I:/usr/ -d ./ -i i: -i E:/usr/bin/ + gen_digest_lists -t metadata -f compact -i l:policy -o add -p -1 -m immutable -i I:/usr/ -d ./ -i i: -i E:/usr/bin/ -T + ``` + +- Scenario 6: In an `rpmbuild` callback script, generate a digest list by reading the list file passed by `rpmbuild`. + + ```shell + gen_digest_lists -i M: -t metadata -f compact -d $DIGEST_LIST_DIR -i l:policy \ + -i i: -o add -p -1 -m immutable -i L:$BIN_PKG_FILES -i u: \ + -A $RPM_BUILD_ROOT -i e: \ + -i E:/usr/src \ + -i E:/boot/efi \ + -i F:/lib \ + -i F:/usr/lib \ + -i F:/lib64 \ + -i F:/usr/lib64 \ + -i F:/lib/modules \ + -i F:/usr/lib/modules \ + -i F:/lib/firmware \ + -i F:/usr/lib/firmware + + gen_digest_lists -i M: -t metadata -f compact -d $DIGEST_LIST_DIR.tlv \ + -i l:policy -i i: -o add -p -1 -m immutable -i L:$BIN_PKG_FILES -i u: \ + -T -A $RPM_BUILD_ROOT -i e: \ + -i E:/usr/src \ + -i E:/boot/efi \ + -i F:/lib \ + -i F:/usr/lib \ + -i F:/lib64 \ + -i F:/usr/lib64 \ + -i F:/lib/modules \ + -i F:/usr/lib/modules \ + -i F:/lib/firmware \ + -i F:/usr/lib/firmware + ``` + +#### manage_digest_lists + +The `manage_digest_lists` tool is designed to parse and convert binary-format TLV digest list files into a human-readable text format. Below are the command options. + +| Option | Value | Function | +| ------ | ------------ | ------------------------------------------------------------------------------------------------------------ | +| -d | \ | Specify the directory containing the TLV digest list files. | +| -f | \ | Specify the name of the TLV digest list file. | +| -p | dump | Define the operation type. Currently, only `dump` is supported, which parses and prints the TLV digest list. | +| -v | N/A | Print verbose details. | +| -h | N/A | Display help information. | + +**Usage example** + +View TLV digest list information. + +```ini +manage_digest_lists -p dump -d /etc/ima/digest_lists.tlv/ +``` + +## File Format Description + +### IMA Policy File Syntax Description + +The IMA policy file is a text-based file that can include multiple rule statements separated by newline characters (`\n`). Each rule statement must begin with an **action** keyword, followed by one or more **filter conditions**: + +```text + [filter condition 2] [filter condition 3]... +``` + +The action keyword defines the specific action for the policy. Only one action is allowed per policy. Refer to the table below for specific actions (note that the `action=` prefix can be omitted in practice, for example, use `dont_measure` instead of `action=dont_measure`). + +Supported filter conditions include: + +- `func`: indicates the type of file to be measured or appraised. It is typically used with `mask`. Only one `func` is allowed per policy. + - `FILE_CHECK` can only be paired with `MAY_EXEC`, `MAY_WRITE`, or `MAY_READ`. + - `MODULE_CHECK`, `MMAP_CHECK`, and `BPRM_CHECK` can only be paired with `MAY_EXEC`. + - Other combinations will not take effect. + +- `mask`: specifies the operation on the file that triggers measurement or appraisal. Only one `mask` is allowed per policy. + +- `fsmagic`: represents the hexadecimal magic number of the file system type, as defined in **/usr/include/linux/magic.h** (by default, all file systems are measured unless excluded using `dont_measure` or `dont_appraise`). + +- `fsuuid`: represents the 16-character hexadecimal string of the system device UUID. + +- `objtype`: specifies the file security type. Only one file type is allowed per policy. Compared to `func`, `objtype` offers finer granularity. For example, `obj_type=nova_log_t` refers to files with the SELinux type `nova_log_t`. + +- `uid`: specifies the user (by user ID) performing the operation on the file. Only one `uid` is allowed per policy. + +- `fowner`: specifies the file owner (by user ID). Only one `fowner` is allowed per policy. + +The keywords are detailed as follows. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
KeywordValueDescription
actionmeasureEnable IMA measurement.
actiondont_measureDisable IMA measurement.
actionappraiseEnable IMA appraisal.
actiondont_appraiseDisable IMA appraisal.
actionauditEnable auditing.
funcFILE_CHECKFiles to be opened
funcMODULE_CHECKKernel module files to be loaded
funcMMAP_CHECKShared library files to be mapped into process memory
funcBRPM_CHECKFiles to be executed (excluding script files opened via programs like /bin/hash)
funcPOLICY_CHECKIMA policy files to be imported
funcFIRMWARE_CHECKFirmware to be loaded into memory
funcDIGEST_LIST_CHECKDigest list files to be loaded into the kernel
funcKEXEC_KERNEL_CHECKKernel to be switched to using kexec
maskMAY_EXECExecute a file.
maskMAY_WRITEWrite to a file.
maskMAY_READRead a file.
maskMAY_APPENDAppend to a file.
fsmagicfsmagic=xxxHexadecimal magic number representing the file system type
fsuuidfsuuid=xxx16-character hexadecimal string representing the system device UUID
fownerfowner=xxxUser ID of the file owner
uiduid=xxxUser ID of the user performing the operation on the file
obj_typeobj_type=xxx_tFile type based on SELinux labels
pcrpcr=<num>PCR in TPM for extending measurements (defaulting to 10)
appraise_typeimasigIMA appraisal based on signatures
appraise_typemeta_immutableAppraisal based on file extended attributes with signatures (supporting digest lists)
+ +## Usage Instructions + +> ![](./public_sys-resources/icon-note.gif) **Note:** +> Native IMA/EVM is an open source Linux feature. This section offers a concise overview of its basic usage. For further details, consult the open source wiki: +> + +### Native IMA Usage Instructions + +#### IMA Measurement Mode + +To enable IMA measurement, configure the measurement policy. + +**Step 1:** Specify the measurement policy by configuring boot parameters or manually. For example, configure the IMA policy via boot parameters as follows: + +```ini +ima_policy="tcb" +``` + +Alternatively, manually configure the IMA policy like this: + +```shell +echo "measure func=BPRM_CHECK" > /sys/kernel/security/ima/policy +``` + +**Step 2:** Reboot the system. You can then check the measurement log in real time to monitor the current measurement status: + +```shell +cat /sys/kernel/security/ima/ascii_runtime_measurements +``` + +#### IMA Appraisal Mode + +Enter fix mode, complete IMA labeling for files, and then enable log or enforce mode. + +**Step 1:** Configure boot parameters and reboot to enter fix mode: + +```ini +ima_appraise=fix ima_policy=appraise_tcb +``` + +**Step 2:** Generate IMA extended attributes for all files requiring appraisal: + +For immutable files (such as binary program files), use signature mode to write the signature of the file digest value into the IMA extended attribute. For example (where **/path/to/ima.key** is the signing private key matching the IMA certificate): + +```shell +find /usr/bin -fstype ext4 -type f -executable -uid 0 -exec evmctl -a sha256 ima_sign --key /path/to/ima.key '{}' \; +``` + +For mutable files (such as data files), use hash mode to write the file digest value into the IMA extended attribute. IMA supports an automatic labeling mechanism, where accessing the file in fix mode will generate the IMA extended attribute: + +```shell +find / -fstype ext4 -type f -uid 0 -exec dd if='{}' of=/dev/null count=0 status=none \; +``` + +To verify if the file has been successfully labeled with the IMA extended attribute (`security.ima`), use: + +```shell +getfattr -m - -d /sbin/init +``` + +**Step 3:** Configure boot parameters, switch the IMA appraisal mode to log or enforce, and reboot the system: + +```ini +ima_appraise=enforce ima_policy=appraise_tcb +``` + +### IMA Digest List Usage Instructions + +#### Prerequisites + +Before using the IMA digest list feature, install the ima-evm-utils and digest-list-tools packages: + +```shell +yum install ima-evm-utils digest-list-tools +``` + +#### Mechanism Overview + +##### Digest List Files + +After installing RPM packages released by openEuler, digest list files are automatically generated in the **/etc/ima** directory. The following types of files exist: + +**/etc/ima/digest_lists/0-metadata_list-compact-\** + +This is the IMA digest list file, generated using the `gen_digest_lists` command (see [gen_digest_lists](#gen_digest_lists) for details). This binary file contains header information and a series of SHA256 hash values, representing the digest values of legitimate file contents and file extended attributes. Once measured or appraised, this file is imported into the kernel, and IMA digest list measurement or appraisal is performed based on the allowlist digest values in this file. + +**/etc/ima/digest_lists/0-metadata_list-rpm-\** + +This is the RPM digest list file, **essentially the header information of the RPM package**. After the RPM package is installed, if the IMA digest list file does not contain a signature, the RPM header information is written into this file, and the signature of the header information is written into the `security.ima` extended attribute. This allows the authenticity of the RPM header information to be verified through the signature. Since the RPM header information includes the digest value of the digest list file, indirect verification of the digest list is achieved. + +**/etc/ima/digest_lists/0-parser_list-compact-libexec** + +This is the IMA parser digest list file, storing the digest value of the **/usr/libexec/rpm_parser** file. This file is used to establish a trust chain from the RPM digest list to the IMA digest list. The kernel IMA digest list mechanism performs special verification on processes generated by this file. If the process is confirmed to be the `rpm_parser` program, all digest lists imported by it are trusted without requiring signature verification. + +**/etc/ima/digest_lists.sig/0-metadata_list-compact-\.sig** + +This is the signature file for the IMA digest list. If this file is included in the RPM package, its content is written into the `security.ima` extended attribute of the corresponding RPM digest list file during the RPM installation phase. This enables signature verification during the IMA digest list import phase. + +**/etc/ima/digest_lists.tlv/0-metadata_list-compact_tlv-\** + +This is the TLV digest list file, typically generated alongside the IMA digest list file for target files. It stores the integrity information of the target files (such as file content digest values and file extended attributes). The purpose of this file is to assist users in querying or restoring the integrity information of target files. + +##### Digest List File Signing Methods + +In IMA digest list appraisal mode, the IMA digest list file must undergo signature verification before it can be imported into the kernel and used for subsequent file whitelist matching. The IMA digest list file supports the following signing methods. + +**(1) IMA extended attribute signature** + +This is the native IMA signing mechanism, where the signature information is stored in the `security.ima` extended attribute in a specific format. It can be generated and added using the `evmctl` command: + +```shell +evmctl ima_sign --key /path/to/ima.key -a sha256 +``` + +Alternatively, the `-f` parameter can be added to store the signature and header information in a separate file: + +```shell +evmctl ima_sign -f --key /path/to/ima.key -a sha256 +``` + +When IMA digest list appraisal mode is enabled, you can directly write the path of a a digest list file to the kernel interface to import or delete it. This process automatically triggers appraisal, performing signature verification on the digest list file content based on the `security.ima` extended attribute: + +```shell +# Import the IMA digest list file. +echo > /sys/kernel/security/ima/digest_list_data +# Delete the IMA digest list file. +echo > /sys/kernel/security/ima/digest_list_data_del +``` + +**(2) IMA digest list appended signature (default in openEuler 24.03 LTS)** + +Starting with openEuler 24.03 LTS, IMA-specific signing keys are supported, and Cryptographic Message Syntax (CMS) signing is used. Since the signature information includes a certificate chain, it may exceed the length limit for the `security.ima` extended attribute. Therefore, a signature appending method similar to kernel module insertion is adopted: + + + +The signing mechanism is as follows: + +1. Append the CMS signature information to the end of the IMA digest list file. + +2. Fill in the structure and append it to the end of the signature information. The structure is defined as follows: + + ```c + struct module_signature { + u8 algo; /* Public-key crypto algorithm [0] */ + u8 hash; /* Digest algorithm [0] */ + u8 id_type; /* Key identifier type [PKEY_ID_PKCS7] */ + u8 signer_len; /* Length of signer's name [0] */ + u8 key_id_len; /* Length of key identifier [0] */ + u8 __pad[3]; + __be32 sig_len; /* Length of signature data */ + }; + ``` + +3. Add the `"~Module signature appended~\n"` magic string. + +A reference script for this step is as follows: + +```shell +#!/bin/bash +DIGEST_FILE=$1 # Path to the IMA digest list file +SIG_FILE=$2 # Path to save the IMA digest list signature information +OUT=$3 # Output path for the digest list file after adding the signature information + +cat $DIGEST_FILE $SIG_FILE > $OUT +echo -n -e "\x00\x00\x02\x00\x00\x00\x00\x00" >> $OUT +echo -n -e $(printf "%08x" "$(ls -l $SIG_FILE | awk '{print $5}')") | xxd -r -ps >> $OUT +echo -n "~Module signature appended~" >> $OUT +echo -n -e "\x0a" >> $OUT +``` + +**(3) Reusing RPM signatures (default in openEuler 22.03 LTS)** + +openEuler 22.03 LTS supports reusing the RPM signing mechanism to sign IMA digest list files. This aims to address the lack of dedicated IMA signing keys in the version. Users do not need to be aware of this signing process. When an RPM package contains an IMA digest list file but no IMA digest list signature file, this signing mechanism is automatically used. The core principle is to verify the IMA digest list through the RPM package header information. + +For RPM packages released by openEuler, each package file can consist of two parts: + +- **RPM header information:** Stores RPM package attribute fields, including the package name and file digest list. The integrity of this information is ensured by the RPM header signature. +- **RPM files:** The actual files installed into the system, including IMA digest list files generated during the build phase. + + + +During RPM package installation, if the RPM process detects that the digest list file in the package does not contain a signature, it creates an RPM digest list file in the **/etc/ima** directory, writes the RPM header information into the file content, and writes the RPM header signature into the `security.ima` extended attribute of the file. This allows indirect verification and import of the IMA digest list through the RPM digest list. + +##### IMA Digest List Import + +When IMA measurement mode is enabled, importing IMA digest list files does not require signature verification. The file path can be directly written to the kernel interface to import or delete the digest list: + +```shell +# Import the IMA digest list file. +echo > /sys/kernel/security/ima/digest_list_data +# Delete the IMA digest list file. +echo > /sys/kernel/security/ima/digest_list_data_del +``` + +When IMA appraisal mode is enabled, importing a digest list requires signature verification. Depending on the signing method, there are two import approaches. + +**Direct import** + +For IMA digest list files that already contain signature information (IMA extended attribute signature or IMA digest list appended signature), the file path can be directly written to the kernel interface to import or delete the digest list. This process automatically triggers appraisal, performing signature verification on the digest list file content based on the `security.ima` extended attribute: + +```shell +# Import the IMA digest list file. +echo > /sys/kernel/security/ima/digest_list_data +# Delete the IMA digest list file. +echo > /sys/kernel/security/ima/digest_list_data_del +``` + +**Using `upload_digest_lists` for import** + +For IMA digest list files that reuse RPM signatures, the `upload_digest_lists` command must be used for import. The specific commands are as follows (note that the specified path should point to the corresponding RPM digest list): + +```shell +# Import the IMA digest list file. +upload_digest_lists add +# Delete the IMA digest list file. +upload_digest_lists del +``` + +This process is relatively complex and requires the following prerequisites: + +1. The system has already imported the digest lists (including IMA digest lists and IMA PARSER digest lists) from the `digest_list_tools` package released by openEuler. + +2. An IMA appraisal policy for application execution (`BPRM_CHECK` policy) has been configured. + +#### Operation Guide + +##### Automatic Digest List Generation During RPM Build + +The openEuler RPM toolchain supports the `%__brp_digest_list` macro, which is configured as follows: + +```text +%__brp_digest_list /usr/lib/rpm/brp-digest-list %{buildroot} +``` + +When this macro is configured, the **/usr/lib/rpm/brp-digest-list** script is invoked when a user runs the `rpmbuild` command to build a package. This script handles the generation and signing of digest lists. By default, openEuler generates digest lists for critical files such as executables, dynamic libraries, and kernel modules. YOU can also modify the script to customize the scope of digest list generation and specify signing keys. The following example uses a user-defined signing key (**/path/to/ima.key**) to sign the digest list. + +```shell +...... (line 66) +DIGEST_LIST_TLV_PATH="$DIGEST_LIST_DIR.tlv/0-metadata_list-compact_tlv-$(basename $BIN_PKG_FILES)" +[ -f $DIGEST_LIST_TLV_PATH ] || exit 0 + +chmod 644 $DIGEST_LIST_TLV_PATH +echo $DIGEST_LIST_TLV_PATH + +evmctl ima_sign -f --key /path/to/ima.key -a sha256 $DIGEST_LIST_PATH &> /dev/null +chmod 400 $DIGEST_LIST_PATH.sig +mkdir -p $DIGEST_LIST_DIR.sig +mv $DIGEST_LIST_PATH.sig $DIGEST_LIST_DIR.sig +echo $DIGEST_LIST_DIR.sig/0-metadata_list-compact-$(basename $BIN_PKG_FILES).sig +``` + +##### IMA Digest List Measurement + +Enable IMA digest list measurement using the following steps: + +**Step 1:** Configure the boot parameters to enable IMA measurement. The process is similar to **native IMA measurement**, but a specific TPM PCR must be configured for measurement. An example boot parameter is as follows: + +```ini +ima_policy=exec_tcb ima_digest_list_pcr=11 +``` + +**Step 2:** Import the IMA digest list. For example, using the digest list of the Bash package: + +```shell +echo /etc/ima/digest_lists/0-metadata_list-compact-bash-5.1.8-6.oe2203sp1.x86_64 > /sys/kernel/security/ima/digest_list_data +``` + +The IMA digest list measurement logs can be queried as follows: + +```shell +cat /sys/kernel/security/ima/ascii_runtime_measurements +``` + +After importing the IMA digest list, if the digest value of the measured file is included in the IMA digest list, no additional measurement logs will be recorded. + +##### IMA Digest List Appraisal + +###### Boot With the Default Policy + +You can configure the `ima_policy` parameter in the boot parameters to specify the IMA default policy. After IMA initialization during kernel startup, the default policy is immediately applied for appraisal. You can enable IMA digest list appraisal using the following steps: + +**Step 1:** Execute the `dracut` command to write the digest list file into initrd: + +```shell +dracut -f -e xattr +``` + +**Step 2:** Configure the boot parameters and IMA policy. Typical configurations are as follows: + +```ini +# IMA appraisal in log/enforce mode based on digest lists, protecting only file content, with the default policy set to appraise_exec_tcb +ima_appraise=log ima_appraise_digest_list=digest-nometadata ima_policy="appraise_exec_tcb" initramtmpfs module.sig_enforce +ima_appraise=enforce ima_appraise_digest_list=digest-nometadata ima_policy="appraise_exec_tcb" initramtmpfs module.sig_enforce +# IMA appraisal in log/enforce mode based on digest lists, protecting file content and extended attributes, with the default policy set to appraise_exec_tcb+appraise_exec_immutable +ima_appraise=log-evm ima_appraise_digest_list=digest ima_policy="appraise_exec_tcb|appraise_exec_immutable" initramtmpfs evm=x509 evm=complete module.sig_enforce +ima_appraise=enforce-evm ima_appraise_digest_list=digest ima_policy="appraise_exec_tcb|appraise_exec_immutable" initramtmpfs evm=x509 evm=complete module.sig_enforce +``` + +Reboot the system to enable IMA digest list appraisal. During the boot process, the IMA policy will take effect, and the IMA digest list files will be automatically imported. + +###### Boot Without a Default Policy + +You can omit the `ima_policy` parameter in the boot parameters, indicating that no default policy is applied during system startup. The IMA appraisal mechanism will wait for the you to import a policy before becoming active. + +**Step 1:** Configure the boot parameters. Typical configurations are as follows: + +```ini +# IMA appraisal in log/enforce mode based on digest lists, protecting only file content, with no default policy +ima_appraise=log ima_appraise_digest_list=digest-nometadata initramtmpfs +ima_appraise=enforce ima_appraise_digest_list=digest-nometadata initramtmpfs +# IMA appraisal in log/enforce mode based on digest lists, protecting file content and extended attributes, with no default policy +ima_appraise=log-evm ima_appraise_digest_list=digest initramtmpfs evm=x509 evm=complete +ima_appraise=enforce-evm ima_appraise_digest_list=digest initramtmpfs evm=x509 evm=complete +``` + +Reboot the system. At this point, since no policy is configured, IMA appraisal will not be active. + +**Step 2:** Import the IMA policy by writing the full path of the policy file to the kernel interface: + +```shell +echo /path/to/policy > /sys/kernel/security/ima/policy +``` + +> ![](./public_sys-resources/icon-note.gif) **Note:** +> The policy must include certain fixed rules. Refer to the following policy templates. +> For openEuler 22.03 LTS (reusing RPM signatures): + +```ini +# Do not appraise access behavior for the securityfs file system. +dont_appraise fsmagic=0x73636673 +# Other user-defined dont_appraise rules +...... +# Appraise imported IMA digest list files. +appraise func=DIGEST_LIST_CHECK appraise_type=imasig +# Appraise all files opened by the /usr/libexec/rpm_parser process. +appraise parser appraise_type=imasig +# Appraise executed applications (triggering appraisal for /usr/libexec/rpm_parser execution, additional conditions such as SELinux labels can be added). +appraise func=BPRM_CHECK appraise_type=imasig +# Other user-defined appraise rules +...... +``` + +> For openEuler 24.03 LTS (IMA extended attribute signatures or appended signatures): + +```ini +# User-defined dont_appraise rules +...... +# Appraise imported IMA digest list files. +appraise func=DIGEST_LIST_CHECK appraise_type=imasig|modsig +# Other user-defined appraise rules. +...... +``` + +**Step 3:** Import the IMA digest list files. Different import methods are required for digest lists with different signing methods. + +> For openEuler 22.03 LTS (IMA digest lists reusing RPM signatures): + +```ini +# Import digest lists from the digest_list_tools package +echo /etc/ima/digest_lists/0-metadata_list-compact-digest-list-tools-0.3.95-13.x86_64 > /sys/kernel/security/ima/digest_list_data +echo /etc/ima/digest_lists/0-parser_list-compact-libexec > /sys/kernel/security/ima/digest_list_data +# Import other RPM digest lists +upload_digest_lists add /etc/ima/digest_lists +# Check the number of imported digest list entries +cat /sys/kernel/security/ima/digests_count +``` + +> For openEuler 24.03 LTS (IMA digest lists with appended signatures): + +```shell +find /etc/ima/digest_lists -name "0-metadata_list-compact-*" -exec echo {} > /sys/kernel/security/ima/digest_list_data \; +``` + +##### Software Upgrade + +After the IMA digest list feature is enabled, files covered by IMA protection require synchronized updates to their digest lists during upgrades. For RPM packages released by openEuler, the addition, update, and deletion of digest lists within the RPM packages are automatically handled during package installation, upgrade, or removal, without requiring manual user intervention. For user-maintained non-RPM packages, manual import of digest lists is necessary. + +##### User Certificate Import + +You can import custom certificates to measure or appraise software not released by openEuler. The openEuler IMA appraisal mode supports signature verification using certificates from the following two key rings: + +- **builtin_trusted_keys**: root certificates pre-configured during kernel compilation +- **ima**: imported via **/etc/keys/x509_ima.der** in initrd, which must be a subordinate certificate of any certificate in the `builtin_trusted_keys` key ring + +**Steps to import a root certificate into the builtin_trusted_keys key ring:** + +**Step 1:** Generate a root certificate using the `openssl` command: + +```shell +echo 'subjectKeyIdentifier=hash' > root.cfg +openssl genrsa -out root.key 4096 +openssl req -new -sha256 -key root.key -out root.csr -subj "/C=AA/ST=BB/O=CC/OU=DD/CN=openeuler test ca" +openssl x509 -req -days 3650 -extfile root.cfg -signkey root.key -in root.csr -out root.crt +openssl x509 -in root.crt -out root.der -outform DER +``` + +**Step 2:** Obtain the openEuler kernel source code, using the latest OLK-5.10 branch as an example: + +```shell +git clone https://gitee.com/openeuler/kernel.git -b OLK-5.10 +``` + +**Step 3:** Navigate to the source code directory and copy the root certificate into it: + +```shell +cd kernel +cp /path/to/root.der . +``` + +Modify the `CONFIG_SYSTEM_TRUSTED_KEYS` option in the config file: + +```shell +CONFIG_SYSTEM_TRUSTED_KEYS="./root.crt" +``` + +**Step 4:** Compile and install the kernel (steps omitted; ensure digest lists are generated for kernel modules). + +**Step 5:** Verify certificate import after rebooting: + +```shell +keyctl show %:.builtin_trusted_keys +``` + +**Steps to import a subordinate certificate into the ima key ring (requires the root certificate to be imported into the builtin_trusted_keys key ring first):** + +**Step 1:** Generate a subordinate certificate based on the root certificate using the `openssl` command: + +```shell +echo 'subjectKeyIdentifier=hash' > ima.cfg +echo 'authorityKeyIdentifier=keyid,issuer' >> ima.cfg +echo 'keyUsage=digitalSignature' >> ima.cfg +openssl genrsa -out ima.key 4096 +openssl req -new -sha256 -key ima.key -out ima.csr -subj "/C=AA/ST=BB/O=CC/OU=DD/CN=openeuler test ima" +openssl x509 -req -sha256 -CAcreateserial -CA root.crt -CAkey root.key -extfile ima.cfg -in ima.csr -out ima.crt +openssl x509 -outform DER -in ima.crt -out x509_ima.der +``` + +**Step 2:** Copy the IMA certificate to the **/etc/keys** directory: + +```shell +mkdir -p /etc/keys/ +cp x509_ima.der /etc/keys/ +``` + +**Step 3:** Package initrd, embedding the IMA certificate and digest lists into the initrd image: + +```shell +echo 'install_items+=" /etc/keys/x509_ima.der "' >> /etc/dracut.conf +dracut -f -e xattr +``` + +**Step 4:** Verify certificate import after rebooting: + +```shell +keyctl show %:.ima +``` + +#### Typical Use Cases + +Depending on the operating mode, IMA digest lists can be applied to trusted measurement scenarios and user-space secure boot scenarios. + +##### Trusted Measurement Scenario + +The trusted measurement scenario primarily relies on the IMA digest list measurement mode, where the kernel and hardware root of trust (RoT) like the TPM jointly measure critical files. This is combined with a remote attestation toolchain to prove the trusted state of current system files: + +![](./figures/ima_trusted_measurement.png) + +**Runtime phase** + +- During software package deployment, digest lists are imported synchronously. IMA measures the digest lists and records the measurement logs (synchronously extended to the TPM). + +- When an application is executed, IMA measurement is triggered. If the file digest matches the allowlist, it is ignored. Otherwise, the measurement log is recorded (synchronously extended to the TPM). + +**Attestation phase (industry standard process)** + +- The remote attestation server sends an attestation request, and the client returns the IMA measurement logs along with the signed TPM PCR values. + +- The remote attestation server sequentially verifies the PCR (signature verification), measurement logs (PCR replay), and file measurement information (comparison with local baseline values), reporting the results to the security center. + +- The security management center takes corresponding actions, such as event notification or node isolation. + +##### User-Space Secure Boot Scenario + +The user-space secure boot scenario primarily relies on the IMA digest list appraisal mode, similar to secure boot. It aims to perform integrity checks on executed applications or accessed critical files. If the check fails, access is denied: + +![](./figures/ima_secure_boot.png) + +**Runtime phase** + +- During application deployment, digest lists are imported. After the kernel verifies the signature, the digest values are loaded into the kernel hash table as an allowlist. + +- When an application is executed, IMA verification is triggered. The file hash value is calculated, and if it matches the baseline value, access is allowed. Otherwise, the event is logged or access is denied. + +## Appendix + +### Kernel Compilation Options + +The compilation options provided by native IMA/EVM and their descriptions are as follows. + +| Compilation Option | Functionality | +| :------------------------------- | :------------------------------------------- | +| CONFIG_INTEGRITY | Enable IMA/EVM compilation. | +| CONFIG_INTEGRITY_SIGNATURE | Enable IMA signature verification. | +| CONFIG_INTEGRITY_ASYMMETRIC_KEYS | Enable IMA asymmetric signature verification. | +| CONFIG_INTEGRITY_TRUSTED_KEYRING | Enable IMA/EVM keyring. | +| CONFIG_INTEGRITY_AUDIT | Compile IMA audit module. | +| CONFIG_IMA | Enable IMA. | +| CONFIG_IMA_WRITE_POLICY | Allow updating IMA policies during runtime. | +| CONFIG_IMA_MEASURE_PCR_IDX | Allow specifying IMA measurement PCR index. | +| CONFIG_IMA_LSM_RULES | Allow configuring LSM rules. | +| CONFIG_IMA_APPRAISE | Enable IMA appraisal. | +| IMA_APPRAISE_BOOTPARAM | Enable IMA appraisal boot parameters. | +| CONFIG_EVM | Enable EVM. | + +The compilation options provided by the openEuler IMA digest list feature and their descriptions are as follows (enabled by default in openEuler kernel compilation). + +| Compilation Option | Functionality | +| :----------------- | :----------------------------- | +| CONFIG_DIGEST_LIST | Enable the IMA digest list feature. | + +### IMA Digest List Root Certificate + +In openEuler 22.03, the RPM key pair is used to sign IMA digest lists. To ensure the IMA feature is usable out-of-the-box, the openEuler kernel compilation process imports the RPM root certificate (PGP certificate) into the kernel by default. This includes the OBS certificate used in older versions and the openEuler certificate introduced in openEuler 22.03 LTS SP1: + +```shell +# cat /proc/keys | grep PGP +1909b4ad I------ 1 perm 1f030000 0 0 asymmetri private OBS b25e7f66: PGP.rsa b25e7f66 [] +2f10cd36 I------ 1 perm 1f030000 0 0 asymmetri openeuler fb37bc6f: PGP.rsa fb37bc6f [] +``` + +Since the current kernel does not support importing PGP subkeys, and the openEuler certificate uses subkey signing, the openEuler kernel preprocesses the certificate before compilation by extracting the subkey and importing it into the kernel. The specific process can be found in the [**process_pgp_certs.sh**](https://gitee.com/src-openeuler/kernel/blob/openEuler-22.03-LTS-SP1/process_pgp_certs.sh) script in the kernel package repository. + +Starting from openEuler 24.03, dedicated IMA certificates are supported. For details, refer to the relevant section in [Introduction to Signature Certificates](../cert_signature/introduction_to_signature_certificates.md). + +If you do not intend to use the IMA digest list feature or prefer other keys for signing/verification, you can remove the related code and implement your own kernel root certificate configuration. diff --git a/docs/en/server/security/trusted_computing/public_sys-resources/icon-note.gif b/docs/en/server/security/trusted_computing/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/server/security/trusted_computing/public_sys-resources/icon-note.gif differ diff --git a/docs/en/server/security/trusted_computing/remote-attestation-kunpeng-security-library.md b/docs/en/server/security/trusted_computing/remote-attestation-kunpeng-security-library.md new file mode 100644 index 0000000000000000000000000000000000000000..9d6cb0b519514e2261d6ce2569c6ee41bb02d6cf --- /dev/null +++ b/docs/en/server/security/trusted_computing/remote-attestation-kunpeng-security-library.md @@ -0,0 +1,417 @@ +# Remote Attestation (Kunpeng Security Library) + +## Introduction + +This project develops basic security software components running on Kunpeng processors. In the early stage, the project focuses on trusted computing fields such as remote attestation to empower security developers in the community. + +## Software Architecture + +On the platform without TEE enabled, this project can provide the platform remote attestation feature, and its software architecture is shown in the following figure: + +![img](./figures/RA-arch-1.png) + +On the platform that has enabled TEE, this project can provide TEE remote attestation feature, and its software architecture is shown in the following figure: + +![img](./figures/RA-arch-2.png) + +## Installation and Configuration + +1. Run the following command to use the RPM package of the Yum installation program: + + ```shell + yum install kunpengsecl-ras kunpengsecl-rac kunpengsecl-rahub kunpengsecl-qcaserver kunpengsecl-attester kunpengsecl-tas kunpengsecl-devel + ``` + +2. Prepare the database environment. Go to the `/usr/share/attestation/ras` directory and run the p`prepare-database-env.sh` script to automatically configure the database environment. + +3. The configuration files required for program running are stored in three paths: current path `./config.yaml`, home path `${HOME}/.config/attestation/ras(rac)(rahub)(qcaserver)(attester)(tas)/config.yaml`, and system path `/etc/attestation/ras(rac)(rahub)(qcaserver)(attester)(tas)/config.yaml`. + +4. (Optional) To create a home directory configuration file, run the `prepare-ras(rac)(hub)(qca)(attester)(tas)conf-env.sh` script in `/usr/share/attestation/ras(rac)(rahub)(qcaserver)(attester)(tas)` after installing the RPM package. + +## Options + +### RAS Boot Options + +Run the `ras` command to start the RAS program. Note that you need to provide the ECDSA public key in the current directory and name it `ecdsakey.pub`. Options are as follows: + +```console + -H --https HTTP/HTTPS mode switch. The default value is https(true), false=http. + -h --hport RESTful API port listened by RAS in HTTPS mode. + -p, --port string Client API port listened by RAS. + -r, --rest string RESTful API port listened by RAS in HTTP mode. + -T, --token Generates a verification code for test and exits. + -v, --verbose Prints more detailed RAS runtime log information. + -V, --version Prints the RAS version and exits. +``` + +### RAC Boot Options + +Run the `sudo raagent` command to start the RAC program. Note that the sudo permission is required to enable the physical TPM module. Options are as follows: + +```console + -s, --server string Specifies the RAS service port to be connected. + -t, --test Starts in test mode. + -v, --verbose Prints more detailed RAC runtime log information. + -V, --version Prints the RAC version and exits. + -i, --imalog Specifies the path of the IMA file. + -b, --bioslog Specifies the path of the BIOS file. + -T, --tatest Starts in TA test mode. +``` + +**Note:** +>1.To use TEE remote attestation feature, you must start RAC not in TA test mode. And place the uuid, whether to use TCB, mem_hash and img_hash of the TA to be attestated sequentially in the **talist** file under the RAC execution path. At the same time, pre install the **libqca.so** and **libteec.so** library provided by the TEE team. The format of the **talist** file is as follows: +> +>```text +>e08f7eca-e875-440e-9ab0-5f381136c600 false ccd5160c6461e19214c0d8787281a1e3c4048850352abe45ce86e12dd3df9fde 46d5019b0a7ffbb87ad71ea629ebd6f568140c95d7b452011acfa2f9daf61c7a +>``` +> +>2.To not use TEE remote attestation feature, you must copy the **libqca.so** and **libteec.so** library in `${DESTDIR}/usr/share/attestation/qcaserver` path to `/usr/lib` or `/usr/lib64` path, and start RAC in TA test mode. + +### QCA Boot Options + +Run the `${DESTDIR}/usr/bin/qcaserver` command to start the QCA program. Note that to start QTA normally, the full path of qcaserver must be used, and the CA path parameter in QTA needs to be kept the same as the path. Options are as follows: + +```console + -C, --scenario int Sets the application scenario of the program, The default value is sce_no_as(0), 1=sce_as_no_daa, 2=sce_as_with_daa. + -S, --server string Specifies the open server address/port. +``` + +### ATTESTER Boot Options + +Run the `attester` command to start the ATTESTER program. Options are as follows: + +```console + -B, --basevalue string Sets the base value file read path + -M, --mspolicy int Sets the measurement strategy, which defaults to -1 and needs to be specified manually. 1=compare only img-hash values, 2=compare only hash values, and 3=compare both img-hash and hash values at the same time. + -S, --server string Specifies the address of the server to connect to. + -U, --uuid int Specifies the trusted apps to verify. + -V, --version Prints the program version and exit. + -T, --test Reads fixed nonce values to match currently hard-coded trusted reports. +``` + +### TAS Boot Options + +Run the `tas` command to start the TAS program. Options are as follows: + +```console + -T, --token Generates a verification code for test and exits. +``` + +**Note:** +>1.To enable the TAS, you must configure the private key for TAS. Run the following command to modify the configuration file in the home directory: +> +>```shell +>$ cd ${HOME}/.config/attestation/tas +>$ vim config.yaml +> # The values of the following DAA_GRP_KEY_SK_X and DAA_GRP_KEY_SK_Y are for testing purposes only. +> # Be sure to update their contents to ensure safety before normal use. +>tasconfig: +> port: 127.0.0.1:40008 +> rest: 127.0.0.1:40009 +> akskeycertfile: ./ascert.crt +> aksprivkeyfile: ./aspriv.key +> huaweiitcafile: ./Huawei IT Product CA.pem +> DAA_GRP_KEY_SK_X: 65a9bf91ac8832379ff04dd2c6def16d48a56be244f6e19274e97881a776543c65a9bf91ac8832379ff04dd2c6def16d48a56be244f6e19274e97881a776543c +> DAA_GRP_KEY_SK_Y: 126f74258bb0ceca2ae7522c51825f980549ec1ef24f81d189d17e38f1773b56126f74258bb0ceca2ae7522c51825f980549ec1ef24f81d189d17e38f1773b56 +>``` +> +>Then enter `tas` to start TAS program. +> +>2.In an environment with TAS, in order to improve the efficiency of QCA's certificate configuration process, not every boot needs to access the TAS to generate the certificate, but through the localized storage of the certificate. That is, read the certification path configured in `config.yaml` on QCA side, check if a TAS-issued certificate has been saved locally through the `func hasAKCert(s int) bool` function. If the certificate is successfully read, there is no need to access TAS. If the certificate cannot be read, you need to access TAS and save the certificate returned by TAS locally. + +## API Definition + +### RAS APIs + +To facilitate the administrator to manage the target server, RAS and the user TA in the TEE deployed on the target server, the following APIs are designed for calling: + +| API | Method | +| --------------------------------- | --------------------------- | +| / | GET | +| /{id} | GET, POST, DELETE | +| /{from}/{to} | GET | +| /{id}/reports | GET | +| /{id}/reports/{reportid} | GET, DELETE | +| /{id}/basevalues | GET | +| /{id}/newbasevalue | POST | +| /{id}/basevalues/{basevalueid} | GET, POST, DELETE | +| /{id}/ta/{tauuid}/status | GET | +| /{id}/ta/{tauuid}/tabasevalues | GET | +| /{id}/ta/{tauuid}/tabasevalues/{tabasevalueid} | GET, POST, DELETE | +| /{id}/ta/{tauuid}/newtabasevalue | POST | +| /{id}/ta/{tauuid}/tareports | GET | +| /{id}/ta/{tauuid}/tareports/{tareportid} | GET, POST, DELETE | +| /{id}/basevalues/{basevalueid} | GET, DELETE | +| /version | GET | +| /config | GET, POST | +| /{id}/container/status | GET | +| /{id}/device/status | GET | + +The usage of the preceding APIs is described as follows: + +To query information about all servers, use `/`. + +```shell +curl -X GET -H "Content-Type: application/json" http://localhost:40002/ +``` + +*** +To query detailed information about a target server, use the GET method of `/{id}`. **{id}** is the unique ID allocated by RAS to the target server. + +```shell +curl -X GET -H "Content-Type: application/json" http://localhost:40002/1 +``` + +*** +To modify information about the target server, use the POST method of `/{id}`. `$AUTHTOKEN` is the identity verification code automatically generated by running the `ras -T` command. + +```go +type clientInfo struct { + Registered *bool `json:"registered"` // Registration status of the target server + IsAutoUpdate *bool `json:"isautoupdate"`// Target server base value update policy +} +``` + +```shell +curl -X POST -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/1 -d '{"registered":false, "isautoupdate":false}' +``` + +*** +To delete a target server, use the DELETE method of `/{id}`. + +**Note:** +>This method does not delete all information about the target server. Instead, it sets the registration status of the target server to `false`. + +```shell +curl -X DELETE -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/1 +``` + +*** +To query information about all servers in a specified range, use the GET method of `/{from}/{to}`. + +```shell +curl -X GET -H "Content-Type: application/json" http://localhost:40002/1/9 +``` + +*** +To query all trust reports of the target server, use the GET method of `/{id}/reports`. + +```shell +curl -X GET -H "Content-Type: application/json" http://localhost:40002/1/reports +``` + +*** +To query details about a specified trust report of the target server, use the GET method of `/{id}/reports/{reportid}`. **{reportid}** indicates the unique ID assigned by RAS to the trust report of the target server. + +```shell +curl -X GET -H "Content-Type: application/json" http://localhost:40002/1/reports/1 +``` + +*** +To delete a specified trust report of the target server, use the DELETE method of `/{id}/reports/{reportid}`. + +**Note:** +>This method will delete all information about the specified trusted report, and the report cannot be queried through the API. + +```shell +curl -X DELETE -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/1/reports/1 +``` + +*** +To query all base values of the target server, use the GET method of `/{id}/reports/{reportid}`. + +```shell +curl -X GET -H "Content-Type: application/json" http://localhost:40002/1/basevalues +``` + +*** +To add a base value to the target server, use the POST method of `/{id}/newbasevalue`. + +```go +type baseValueJson struct { + BaseType string `json:"basetype"` // Base value type + Uuid string `json:"uuid"` // ID of a container or device + Name string `json:"name"` // Base value name + Enabled bool `json:"enabled"` // Whether the base value is available + Pcr string `json:"pcr"` // PCR value + Bios string `json:"bios"` // BIOS value + Ima string `json:"ima"` // IMA value + IsNewGroup bool `json:"isnewgroup"` // Whether this is a group of new reference values +} +``` + +```shell +curl -X POST -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/1/newbasevalue -d '{"name":"test", "basetype":"host", "enabled":true, "pcr":"testpcr", "bios":"testbios", "ima":"testima", "isnewgroup":true}' +``` + +*** +To query details about a specified base value of a target server, use the get method of `/{id}/basevalues/{basevalueid}`. **{basevalueid}** indicates the unique ID allocated by RAS to the specified base value of the target server. + +```shell +curl -X GET -H "Content-Type: application/json" http://localhost:40002/1/basevalues/1 +``` + +*** +To change the availability status of a specified base value of the target server, use the POST method of `/{id}/basevalues/{basevalueid}`. + +```shell +curl -X POST -H "Content-type: application/json" -H "Authorization: $AUTHTOKEN" http://localhost:40002/1/basevalues/1 -d '{"enabled":true}' +``` + +*** +To delete a specified base value of the target server, use the DELETE method of `/{id}/basevalues/{basevalueid}`. + +**Note:** +>This method will delete all the information about the specified base value, and the base value cannot be queried through the API. + +```shell +curl -X DELETE -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/1/basevalues/1 +``` + +To query the trusted status of a specific user TA on the target server, use the GET method of the `"/{id}/ta/{tauuid}/status"` interface. Where {id} is the unique identification number assigned by RAS to the target server, and {tauuid} is the identification number of the specific user TA. + +```shell +curl -X GET -H "Content-type: application/json" -H "Authorization: $AUTHTOKEN" http://localhost:40002/1/ta/test/status +``` + +*** +To query all the baseline value information of a specific user TA on the target server, use the GET method of the `"/{id}/ta/{tauuid}/tabasevalues"` interface. + +```shell +curl -X GET -H "Content-type: application/json" http://localhost:40002/1/ta/test/tabasevalues +``` + +*** +To query the details of a specified base value for a specific user TA on the target server, use the GET method of the `"/{id}/ta/{tauuid}/tabasevalues/{tabasevalueid}"` interface. where {tabasevalueid} is the unique identification number assigned by RAS to the specified base value of a specific user TA on the target server. + +```shell +curl -X GET -H "Content-type: application/json" http://localhost:40002/1/ta/test/tabasevalues/1 +``` + +*** +To modify the available status of a specified base value for a specific user TA on the target server, use the `POST` method of the `"/{id}/ta/{tauuid}/tabasevalues/{tabasevalueid}"` interface. + +```shell +curl -X POST -H "Content-type: application/json" -H "Authorization: $AUTHTOKEN" http://localhost:40002/1/ta/test/tabasevalues/1 --data '{"enabled":true}' +``` + +*** +To delete the specified base value of a specific user TA on the target server, use the `DELETE` method of the `"/{id}/ta/{tauuid}/tabasevalues/{tabasevalueid}"` interface. + +**Note:** +>This method will delete all information about the specified base value, and the base value cannot be queried through the API. + +```shell +curl -X DELETE -H "Content-type: application/json" -H "Authorization: $AUTHTOKEN" -k http://localhost:40002/1/ta/test/tabasevalues/1 +``` + +*** +To add a baseline value to a specific user TA on the target server, use the `POST` method of the `"/{id}/ta/{tauuid}/newtabasevalue"` interface. + +```go +type tabaseValueJson struct { + Uuid string `json:"uuid"` // the identification number of the user TA + Name string `json:"name"` // base value name + Enabled bool `json:"enabled"` // whether a baseline value is available + Valueinfo string `json:"valueinfo"` // mirror hash value and memory hash value +} +``` + +```shell +curl -X POST -H "Content-Type: application/json" -H "Authorization: $AUTHTOKEN" -k http://localhost:40002/1/ta/test/newtabasevalue -d '{"uuid":"test", "name":"testname", "enabled":true, "valueinfo":"test info"}' +``` + +*** +To query the target server for all trusted reports for a specific user TA, use the `GET` method of the `"/{id}/ta/{tauuid}/tareports"` interface. + +```shell +curl -X GET -H "Content-type: application/json" http://localhost:40002/1/ta/test/tareports +``` + +*** +To query the details of a specified trusted report for a specific user TA on the target server, use the `GET` method of the `"/{id}/ta/{tauuid}/tareports/{tareportid}"` interface. Where {tareportid} is the unique identification number assigned by RAS to the specified trusted report of a specific user TA on the target server. + +```shell +curl -X GET -H "Content-type: application/json" http://localhost:40002/1/ta/test/tareports/2 +``` + +*** +To delete the specified trusted report of a specific user TA on the target server, use the `DELETE` method of the `"/{id}/ta/{tauuid}/tareports/{tareportid}"` interface. + +**Note:** +>This method will delete all information of the specified trusted report, and the report cannot be queried through the API. + +```shell +curl -X DELETE -H "Content-type: application/json" http://localhost:40002/1/ta/test/tareports/2 +``` + +*** +To obtain the version information of the program, use the GET method of `/version`. + +```shell +curl -X GET -H "Content-Type: application/json" http://localhost:40002/version +``` + +*** +To query the configuration information about the target server, RAS, or database, use the GET method of `/config`. + +```shell +curl -X GET -H "Content-Type: application/json" http://localhost:40002/config +``` + +*** +To modify the configuration information about the target server, RAS, or database, use the POST method of /config. + +```go +type cfgRecord struct { + // Target server configuration + HBDuration string `json:"hbduration" form:"hbduration"` + TrustDuration string `json:"trustduration" form:"trustduration"` + DigestAlgorithm string `json:"digestalgorithm" form:"digestalgorithm"` + // RAS configuration + MgrStrategy string `json:"mgrstrategy" form:"mgrstrategy"` + ExtractRules string `json:"extractrules" form:"extractrules"` + IsAllupdate *bool `json:"isallupdate" form:"isallupdate"` + LogTestMode *bool `json:"logtestmode" form:"logtestmode"` +} +``` + +```shell +curl -X POST -H "Authorization: $AUTHTOKEN" -H "Content-Type: application/json" http://localhost:40002/config -d '{"hbduration":"5s","trustduration":"20s","DigestAlgorithm":"sha256"}' +``` + +### TAS APIs + +To facilitate the administrator's management of TAS for remote control, the following API is designed for calling: + +| API | Method | +| --------------------| ------------------| +| /config | GET, POST | + +To query the configuration information, use the GET method of the `/config` interface. + +```shell +curl -X GET -H "Content-Type: application/json" http://localhost:40009/config +``` + +*** +To modify the configuration information, use the POST method of the `/config` interface. + +```shell +curl -X POST -H "Content-Type: application/json" -H "Authorization: $AUTHTOKEN" http://localhost:40009/config -d '{"basevalue":"testvalue"}' +``` + +**Note:** +>Currently, only the base value in the configuration information of TAS is supported for querying and modifying. + +## FAQs + +1. Why cannot RAS be started after it is installed? + + > In the current RAS design logic, after the program is started, it needs to search for the `ecdsakey.pub` file in the current directory and read the file as the identity verification code for accessing the program. If the file does not exist in the current directory, an error is reported during RAS boot. + >> Solution 1: Run the `ras -T` command to generate a test token. The `ecdsakey.pub` file is generated. + >> Solution 2: After deploying the oauth2 authentication service, save the verification public key of the JWT token generator as `ecdsakey.pub`. + +2. Why cannot RAS be accessed through REST APIs after it is started? + + > RAS is started in HTTPS mode by default. Therefore, you need to provide a valid certificate for RAS to access it. However, RAS started in HTTP mode does not require a certificate. diff --git a/docs/en/server/security/trusted_computing/tpcm.md b/docs/en/server/security/trusted_computing/tpcm.md new file mode 100644 index 0000000000000000000000000000000000000000..e92a4bbb8183fb04dc992708257fcbce2e10a9a2 --- /dev/null +++ b/docs/en/server/security/trusted_computing/tpcm.md @@ -0,0 +1,37 @@ +# Trusted Platform Control Module + +## Background + +Trusted computing has undergone continuous development and improvement in the past 40 years and has become an important branch of information security. Trusted computing technologies have developed rapidly in recent years and have solved the challenges in Trusted Computing 2.0—integration of trusted systems and existing systems, trusted management, and simplification of trusted application development. These technical breakthroughs form Trusted Computing 3.0, that is, trusted computing based on an active immune system. Compared with the passive plug-in architecture of the previous generation, Trusted Computing 3.0 proposes a new trusted system framework based on self-controlled cryptography algorithms, control chips, trusted software, trusted connections, policy management, and secure and trusted protection applications, implementing trust across the networks. + +The trusted platform control module (TPCM) is a base and core module that can be integrated into a trusted computing platform to establish and ensure a trust source. As one of the innovations in Trusted Computing 3.0 and the core of active immunity, TPCM implements active control over the entire platform. + +The TPCM-based Trusted Computing 3.0 architecture consists of the protection module and the computing module. On the one hand, based on the Trusted Cryptography Module (TPM), the TPCM main control firmware measures the reliability of the protection and computing modules, as well as their firmware. On the other hand, the Trusted Software Base (TSB) measures the reliability of system software and application software. In addition, the TPCM management platform verifies the reliability measurement and synchronizes and manages the trust policies. + +## Feature Description + +The overall system design consists of the protection module, computing module, and trusted management center software, as shown in the following figure. + +![](./figures/TPCM.png) + +- Trusted management center: This centralized management platform, provided by a third-party vendor, formulates, delivers, maintains, and stores protection policies and reference values for trusted computing nodes. +- Protection module: This module operates independently of the computing module and provides trusted computing protection functions that feature active measurement and active control to implement security protection during computing. The protection module consists of the TPCM main control firmware, TCB, and TCM. As a key module for implementing trust protection in a trusted computing node, the TPCM can be implemented in multiple forms, such as cards, chips, and IP cores. It contains a CPU and memory, firmware, and software such as an OS and trusted function components. The TPCM operates alongside the computing module and works according to the built-in protection policy to monitor the trust of protected resources, such as hardware, firmware, and software of the computing module. The TPCM is the Root of Trust in a trusted computing node. + +- Computing module: This module includes hardware, an OS, and application layer software. The running of the OS can be divided into the boot phase and the running phase. In the boot phase, GRUB2 and shim of openEuler support the reliability measurement capability, which protects boot files such as shim, GRUB2, kernel, and initramfs. In the running phase, openEuler supports the deployment of the trusted verification agent (provided by third-party vendor HTTC). The agent sends data to the TPCM for trusted measurement and protection in the running phase. + +The TPCM interacts with other components as follows: + +1. The TPCM hardware, firmware, and software provide an operating environment for the TSB. The trusted function components of the TPCM provide support for the TSB to implement measurement, control, support, and decision-making based on the policy library interpretation requirements. +2. The TPCM accesses the TCM for trusted cryptography functions to complete computing tasks such as trusted verification, measurement, and confidential storage, and provides services for TCM access. +3. The TPCM connects to the trusted management center through the management interface to implement protection policy management and trusted report processing. +4. The TPCM uses the built-in controller and I/O port to interact with the controller of the computing module through the bus to actively monitor the computing module. +5. The built-in protection agent in the OS of the computing module obtains the code and data related to the preset protection object and provides them to the TPCM. The TPCM forwards the monitoring information to the TSB, and the TSB analyzes and processes the information according to the policy library. + +## Constraints + +Supported server: TaiShan 200 server (model 2280) +Supported BMC card: BC83SMMC + +## Application Scenarios + +The TPCM enables a complete trust chain to ensure that the OS boots into a trusted computing environment. diff --git a/docs/en/server/security/trusted_computing/trusted-computing-common-issues-and-solutions.md b/docs/en/server/security/trusted_computing/trusted-computing-common-issues-and-solutions.md new file mode 100644 index 0000000000000000000000000000000000000000..463e8d5444ad7ebf67cf7de0e4b468fbe52b2ec6 --- /dev/null +++ b/docs/en/server/security/trusted_computing/trusted-computing-common-issues-and-solutions.md @@ -0,0 +1,219 @@ +# Common Issues and Solutions + +## Issue 1: System Fails to Boot After IMA Appraisal Enforce Mode Is Enabled with the Default Policy + +The default IMA policy may include checks for critical file access processes such as application execution and kernel module loading. If access to these critical files fails, the system may fail to boot. Common causes include: + +1. The IMA verification certificate is not imported into the kernel, causing the digest list to fail verification. +2. The digest list file is not correctly signed, leading to verification failure. +3. The digest list file is not imported into the initrd, preventing the digest list from being loaded during the boot process. +4. The digest list file does not match the application, causing the application to fail matching the imported digest list. + +Enter the system in log mode to locate and fix the issue. Reboot the system, enter the GRUB menu, and modify the boot parameters to start in log mode: + +```ini +ima_appraise=log +``` + +After the system boots, follow the steps below to troubleshoot. + +**Step 1:** Check the IMA certificates in the key ring. + +```shell +keyctl show %:.builtin_trusted_keys +``` + +For openEuler LTS versions, at least the following kernel certificates should exist (for other versions, reference based on their release dates): + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
VersionCertificate
openEuler 22.03 LTSprivate OBS b25e7f66
openEuler 22.03 LTS SP1/2/3private OBS b25e7f66
openeuler <openeuler@compass-ci.com> b675600b
openEuler 22.03 LTS SP4private OBS b25e7f66
openeuler <openeuler@compass-ci.com> b675600b
openeuler <openeuler@compass-ci.com> fb37bc6f
openEuler 24.03openEuler kernel ICA 1: 90bb67eb4b57eb62bf6f867e4f56bd4e19e7d041
+ +If you have imported other kernel root certificates, use the `keyctl` command to confirm whether the certificates were successfully imported. By default, openEuler does not use the IMA key ring. If you are using it, check whether the user certificates exist in the IMA key ring with the following command: + +```shell +keyctl show %:.ima +``` + +If the issue is that the certificate was not correctly imported, refer to [User Certificate Import](./ima.md#user-certificate-import) for troubleshooting. + +**Step 2:** Check if the digest list contains signature information. + +Query the digest list files in the current system with the following command: + +```shell +ls /etc/ima/digest_lists | grep '_list-compact-' +``` + +For each digest list file, ensure that **one of the following three** signature conditions is met: + +1. The digest list file has a corresponding **RPM digest list file**, and the `security.ima` extended attribute of the **RPM digest list file** contains a signature value. For example, for the bash package digest list, the digest list file path is: + + ```text + /etc/ima/digest_lists/0-metadata_list-compact-bash-5.1.8-6.oe2203sp1.x86_64 + ``` + + The RPM digest list path is: + + ```text + /etc/ima/digest_lists/0-metadata_list-rpm-bash-5.1.8-6.oe2203sp1.x86_64 + ``` + + Check the RPM digest list signature by ensuring the `security.ima` extended attribute is not empty: + + ```shell + getfattr -n security.ima /etc/ima/digest_lists/0-metadata_list-rpm-bash-5.1.8-6.oe2203sp1.x86_64 + ``` + +2. The `security.ima` extended attribute of the digest list file is not empty: + + ```shell + getfattr -n security.ima /etc/ima/digest_lists/0-metadata_list-compact-bash-5.1.8-6.oe2203sp1.x86_64 + ``` + +3. The digest list file contains signature information at the end. Verify if the file content ends with the `~Module signature appended~` magic string (supported in openEuler 24.03 LTS and later versions): + + ```shell + tail -c 28 /etc/ima/digest_lists/0-metadata_list-compact-kernel-6.6.0-28.0.0.34.oe2403.x86_64 + ``` + +If the issue is that the digest list does not contain signature information, refer to [Digest List File Signing Methods](./ima.md#digest-list-file-signing-methods) for troubleshooting. + +**Step 3:** Verify the correctness of the digest list signature. + +After ensuring that the digest list contains signature information, also ensure that the digest list is signed with the correct private key, meaning the signing private key matches the certificate in the kernel. In addition to manually checking the private key, users can check the dmesg logs or audit logs (default path: **/var/log/audit/audit.log**) for signature verification failures. A typical log output is as follows: + +```ini +type=INTEGRITY_DATA msg=audit(1722578008.756:154): pid=3358 uid=0 auid=0 ses=1 subj=unconfined_u:unconfined_r:haikang_t:s0-s0:c0.c1023 op=appraise_data cause=invalid-signature comm="bash" name="/root/0-metadata_list-compact-bash-5.1.8-6.oe2203sp1.x86_64" dev="dm-0" ino=785161 res=0 errno=0UID="root" AUID="root" +``` + +If the issue is incorrect signature information, refer to [Digest List File Signing Methods](./ima.md#digest-list-file-signing-methods) for troubleshooting. + +**Step 4:** Check if the digest list file is imported into the initrd. + +Query whether the digest list file exists in the current initrd with the following command: + +```shell +lsinitrd | grep 'etc/ima/digest_lists' +``` + +If no digest list file is found, users need to recreate the initrd and verify that the digest list is successfully imported: + +```shell +dracut -f -e xattr +``` + +**Step 5:** Verify that the IMA digest list matches the application. + +Refer to [Issue 2](#issue-2-file-execution-fails-after-ima-appraisal-enforce-mode-is-enabled). + +## Issue 2: File Execution Fails After IMA Appraisal Enforce Mode Is Enabled + +After IMA appraisal enforce mode is enabled, if the content or extended attributes of a file configured with IMA policies are incorrect (for example, they do not match the imported digest list), file access may be denied. Common causes include: + +1. The digest list was not successfully imported (refer to [Issue 1](#issue-1-system-fails-to-boot-after-ima-appraisal-enforce-mode-is-enabled-with-the-default-policy) for details). +2. The file content or attributes have been tampered with. + +For scenarios where file execution fails, first ensure that the digest list file has been successfully imported into the kernel. Check the number of digest lists to determine the import status: + +```shell +cat /sys/kernel/security/ima/digests_count +``` + +Next, use the audit logs (default path: **/var/log/audit/audit.log**) to identify which file failed verification and the reason. A typical log output is as follows: + +```ini +type=INTEGRITY_DATA msg=audit(1722811960.997:2967): pid=7613 uid=0 auid=0 ses=1 subj=unconfined_u:unconfined_r:haikang_t:s0-s0:c0.c1023 op=appraise_data cause=IMA-signature-required comm="bash" name="/root/test" dev="dm-0" ino=814424 res=0 errno=0UID="root" AUID="root" +``` + +After identifying the file that failed verification, compare it with the TLV digest list to determine the cause of tampering. For scenarios where extended attribute verification is not enabled, only compare the SHA256 hash value of the file with the `IMA digest` entry in the TLV digest list. For scenarios where extended attribute verification is enabled, also compare the current file attributes with the extended attributes displayed in the TLV digest list. + +Once the cause of the issue is determined, resolve it by restoring the file content and attributes or regenerating the digest list for the file, signing it, and importing it into the kernel. + +## Issue 3: Errors Occur During Packages Installation Across openEuler 22.03 LTS SP Versions After IMA Appraisal Mode Is Enabled + +After IMA appraisal mode is enabled, installing packages from different SP versions of openEuler 22.03 LTS triggers the import of IMA digest lists. This process includes a signature verification step, where the kernel uses its certificates to verify the digest list signatures. Due to changes in signing certificates during the evolution of openEuler, backward compatibility issues may arise in certain cross-SP-version installation scenarios (there are no forward compatibility issues, meaning newer kernels can verify older IMA digest list files without problems). + +You are advised to ensure that the following signing certificates are present in the current kernel: + +```shell +# keyctl show %:.builtin_trusted_keys +Keyring + 566488577 ---lswrv 0 0 keyring: .builtin_trusted_keys + 383580336 ---lswrv 0 0 \_ asymmetric: openeuler b675600b + 453794670 ---lswrv 0 0 \_ asymmetric: private OBS b25e7f66 + 938520011 ---lswrv 0 0 \_ asymmetric: openeuler fb37bc6f +``` + +If any certificates are missing, you are advised to upgrade the kernel to the latest version: + +```shell +yum update kernel +``` + +openEuler 24.03 LTS and later versions include dedicated IMA certificates and support certificate chain verification, ensuring the certificate lifecycle covers the entire LTS version. + +## Issue 4: IMA Digest List Import Fails Despite Correct Signatures After IMA Digest List Appraisal Mode Is Enabled + +The IMA digest list import process includes a verification mechanism. If a digest list fails signature verification during import, the digest list import functionality is disabled, preventing even correctly signed digest lists from being imported afterward. Check the dmesg logs for the following message to confirm if this is the cause: + +```shell +# dmesg +ima: 0-metadata_list-compact-bash-5.1.8-6.oe2203sp1.x86_64 not appraised, disabling digest lists lookup for appraisal +``` + +If such a log is present, a digest list file with an incorrect signature was imported while IMA digest list appraisal mode was enabled, causing the functionality to be disabled. In this case, reboot the system and fix the incorrect digest list signature information. + +## Issue 5: Importing User-Defined IMA Certificates Fails in openEuler 24.03 LTS and Later Versions + +Linux kernel 6.6 introduced additional field validation restrictions for importing certificates. Certificates imported into the IMA key ring must meet the following constraints (following the X.509 standard format): + +- It must be a digital signature certificate, meaning the `keyUsage=digitalSignature` field must be set. +- It must not be a CA certificate, meaning the `basicConstraints=CA:TRUE` field must not be set. +- It must not be an intermediate certificate, meaning the `keyUsage=keyCertSign` field must not be set. + +## Issue 6: kdump Service Fails to Start After IMA Appraisal Mode Is Enabled + +After IMA appraisal enforce mode is enabled, if the IMA policy includes the following `KEXEC_KERNEL_CHECK` rule, the kdump service may fail to start: + +```shell +appraise func=KEXEC_KERNEL_CHECK appraise_type=imasig +``` + +The reason is that in this scenario, all files loaded via `kexec` must undergo integrity verification. As a result, the kernel restricts the loading of kernel image files by kdump to the `kexec_file_load` system call. This can be enabled by modifying the **/etc/sysconfig/kdump** configuration file: + +```shell +KDUMP_FILE_LOAD="on" +``` + +Additionally, the `kexec_file_load` system call itself performs signature verification on the files. Therefore, the kernel image file being loaded must contain a valid secure boot signature, and the current kernel must include the corresponding verification certificate. diff --git a/docs/en/server/security/trusted_computing/trusted-computing.md b/docs/en/server/security/trusted_computing/trusted-computing.md new file mode 100644 index 0000000000000000000000000000000000000000..79eaffe99da1aef7d416af9bd0863fc54d851972 --- /dev/null +++ b/docs/en/server/security/trusted_computing/trusted-computing.md @@ -0,0 +1,27 @@ +# Trusted Computing + +## Trusted Computing Basics + +### What Is Trusted Computing + +The definition of being trusted varies with international organizations. + +1. Trusted Computing Group (TCG): + + An entity that is trusted always achieves the desired goal in an expected way. + +2. International Organization for Standardization (ISO) and International Electrotechnical Commission (IEC) (1999): + + The components, operations, or processes involved in computing are predictable under any conditions and are resistant to viruses and a certain degree of physical interference. + +3. IEEE Computer Society Technical Committee on Dependable Computing: + + Being trusted means that the services provided by the computer system can be proved to be reliable, and mainly refers to the reliability and availability of the system. + +In short, being trusted means that the system operates according to a pre-determined design and policy. + +A trusted computing system consists of a root of trust, a trusted hardware platform, operating system (OS), and application. The basic idea of the system is to create a trusted computing base (TCB) first, and then establish a trust chain that covers the hardware platform, OS, and application. In the trust chain, authentication is performed from the root to the next level, extending trust level by level and building a secure and trusted computing environment. + +![](./figures/trusted_chain.png) + +Unlike traditional security approaches that reactively tackle threats—such as identifying and removing viruses individually—trusted computing employs an allowlist strategy. This ensures that only verified kernels, kernel modules, and applications can operate on the system. Any program that is modified or unrecognized is automatically blocked from execution. diff --git a/docs/en/tools/_toc.yaml b/docs/en/tools/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..ce2b2cd69d4505e5ed043c697db421e045c37c89 --- /dev/null +++ b/docs/en/tools/_toc.yaml @@ -0,0 +1,9 @@ +label: Tools +sections: + - href: ./community_tools/_toc.yaml + - href: ./devops/_toc.yaml + - href: ./ai/_toc.yaml + - href: ./desktop/_toc.yaml + - href: ./cloud/_toc.yaml + - href: ./maintenance/_toc.yaml + - href: ./security/_toc.yaml diff --git a/docs/en/tools/ai/_toc.yaml b/docs/en/tools/ai/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..44df1715bb8629ef6d45aa82ea46d6d7946c4d0f --- /dev/null +++ b/docs/en/tools/ai/_toc.yaml @@ -0,0 +1,5 @@ +label: AI +sections: + - href: ./openeuler_copilot_system/_toc.yaml + - href: ./ai_large_model_service_images_userguide/_toc.yaml + - href: ./ai_container_image_userguide/_toc.yaml diff --git a/docs/en/tools/ai/ai_container_image_userguide/_toc.yaml b/docs/en/tools/ai/ai_container_image_userguide/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..f54323bafa73da4a4f36d20da0ef1f6601914dc5 --- /dev/null +++ b/docs/en/tools/ai/ai_container_image_userguide/_toc.yaml @@ -0,0 +1,8 @@ +label: AI Container Image User Guide +isManual: true +description: >- + openEuler AI container images include AI frameworks and other software, + streamlining AI application development and usage. +sections: + - label: AI Container Image User Guide + href: ./ai-container-image-user-guide.md diff --git a/docs/en/tools/ai/ai_container_image_userguide/ai-container-image-user-guide.md b/docs/en/tools/ai/ai_container_image_userguide/ai-container-image-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..2fc2bf95735167d439f398b4310fe4c0e1d8ddaa --- /dev/null +++ b/docs/en/tools/ai/ai_container_image_userguide/ai-container-image-user-guide.md @@ -0,0 +1,110 @@ +# openEuler AI Container Image User Guide + +## Overview + +The openEuler AI container images encapsulate SDKs for different hardware computing power and software such as AI frameworks and foundation model applications. Start a container using one of the images, and you can use or develop AI applications in your environment. This greatly reduces the time required for application deployment and environment configuration. + +## Obtaining Images + +openEuler has released container images for the Ascend and NVIDIA platforms. Click the links below to download: + +- [openeuler/cann](https://hub.docker.com/r/openeuler/cann) + Stores SDK images for installing CANN software on the openEuler base image in the Ascend environment. + +- [openeuler/cuda](https://hub.docker.com/r/openeuler/cuda) + Stores SDK images for installing CUDA software on the openEuler base image in the NVIDIA environment. + +- [openeuler/pytorch](https://hub.docker.com/r/openeuler/pytorch) + Stores the AI framework image for installing PyTorch based on the SDK image. + +- [openeuler/tensorflow](https://hub.docker.com/r/openeuler/tensorflow) + Stores the AI framework image for installing TensorFlow based on the SDK image. + +- [openeuler/llm](https://hub.docker.com/r/openeuler/tensorrt) + Stores model application images for installing foundation model applications and toolchains based on the AI framework image. + +For details about AI container image classification and image tag specifications, see [oEEP-0014](https://gitee.com/openeuler/TC/blob/master/oEEP/oEEP-0014%20openEuler%20AI%E5%AE%B9%E5%99%A8%E9%95%9C%E5%83%8F%E8%BD%AF%E4%BB%B6%E6%A0%88%E8%A7%84%E8%8C%83.md). + +The size of an AI container image is large. You are advised to run the following command to pull the image to the local environment before starting the container: + +```sh +docker pull image:tag +``` + +In the command, `image` indicates the repository name, for example, `openeuler/cann`, and `tag` indicates the tag of the target image. After the image is pulled, you can start the container. Note that you must have Docker installed before running the `docker pull` command. + +## Starting a Container + +1. Install Docker. For details about how to install Docker, see [Install Docker Engine](https://docs.docker.com/engine/install/). Alternatively, run the following command to install: + + ```sh + yum install -y docker + ``` + + or + + ```sh + apt-get install -y docker + ``` + +2. Installing nvidia-container in the NVIDIA Environment + + (1) Configure the Yum or APT repository. + - For Yum: + + ```sh + curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | \ + sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo + ``` + + - For APT: + + ```sh + curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg + ``` + + ```sh + curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \ + sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \ + sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list + ``` + + (2) Install nvidia-container-toolkit and nvidia-container-runtime. + + ```sh + # For Yum + yum install -y nvidia-container-toolkit nvidia-container-runtime + ``` + + ```sh + # For APT + apt-get install -y nvidia-container-toolkit nvidia-container-runtime + ``` + + (3) Configure Docker. + + ```sh + nvidia-ctk runtime configure --runtime=docker + systemctl restart docker + ``` + + Skip this step in the non-NVIDIA environment. + +3. Ensure that the correct driver and firmware are installed. You can obtain the correct versions from [NVIDIA](https://www.nvidia.com/) or [Ascend](https://www.hiascend.com/) official site. If the driver and firmware are installed, run the `npu-smi` command on the Ascend platform or run the `nvidia-smi` command on the NVIDIA platform. If the hardware information is correctly displayed, the installed version is correct. + +4. After the preceding operations are complete, run the `docker run` command to start the container. + +```sh +# In the Ascend environment +docker run --rm --network host \ + --device /dev/davinci0:/dev/davinci0 \ + --device /dev/davinci_manager --device /dev/devmm_svm --device /dev/hisi_hdc \ + -v /usr/local/dcmi:/usr/local/dcmi -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \ + -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \ + -ti image:tag +``` + +```sh +# In the NVIDIA environment +docker run --gpus all -d -ti image:tag +``` diff --git a/docs/en/tools/ai/ai_large_model_service_images_userguide/_toc.yaml b/docs/en/tools/ai/ai_large_model_service_images_userguide/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..963de021cd933024cad7eec710c8ff69cdb5c410 --- /dev/null +++ b/docs/en/tools/ai/ai_large_model_service_images_userguide/_toc.yaml @@ -0,0 +1,6 @@ +label: LLM Service Image User Guide +isManual: true +href: ./llm-service-image-user-guide.md +description: >- + openEuler provides container images to support large language models (LLMs) + such as Baichuan, ChatGLM, and iFLYTEK Spark. diff --git a/docs/en/tools/ai/ai_large_model_service_images_userguide/llm-service-image-user-guide.md b/docs/en/tools/ai/ai_large_model_service_images_userguide/llm-service-image-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..192955425e4a07dbd7e6c82969488ee08e8b5ac9 --- /dev/null +++ b/docs/en/tools/ai/ai_large_model_service_images_userguide/llm-service-image-user-guide.md @@ -0,0 +1,102 @@ +# Container Images for Large Language Models + +openEuler provides container images to support large language models (LLMs) such as Baichuan, ChatGLM, and iFLYTEK Spark. + +The provided container images come with pre-installed dependencies for both CPU and GPU environments, ensuring a seamless out-of-the-box experience. + +## Pulling the Image (CPU Version) + +```bash +docker pull openeuler/llm-server:1.0.0-oe2203sp3 +``` + +## Pulling the Image (GPU Version) + +```bash +docker pull icewangds/llm-server:1.0.0 +``` + +## Downloading the Model + +Download the model and convert it to GGUF format. + +```bash +# Install Hugging Face Hub. +pip install huggingface-hub + +# Download the model you want to deploy. +export HF_ENDPOINT=https://hf-mirror.com +huggingface-cli download --resume-download baichuan-inc/Baichuan2-13B-Chat --local-dir /root/models/Baichuan2-13B-Chat --local-dir-use-symlinks False + +# Convert the model to GGUF format. +cd /root/models/ +git clone https://github.com/ggerganov/llama.cpp.git +python llama.cpp/convert-hf-to-gguf.py ./Baichuan2-13B-Chat +# Path to the generated GGUF model: /root/models/Baichuan2-13B-Chat/ggml-model-f16.gguf +``` + +## Launch + +Docker v25.0.0 or above is required. + +To use a GPU image, you must install nvidia-container-toolkit. Detailed installation instructions are available in the official NVIDIA documentation: [Installing the NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html). + +**docker-compose.yaml** file content: + +```yaml +version: '3' +services: + model: + image: : # Image name and tag + restart: on-failure:5 + ports: + - 8001:8000 # Listening port number. Change "8001" to modify the port. + volumes: + - /root/models:/models # LLM mount directory + environment: + - MODEL=/models/Baichuan2-13B-Chat/ggml-model-f16.gguf # Model file path inside the container + - MODEL_NAME=baichuan13b # Custom model name + - KEY=sk-12345678 # Custom API Key + - CONTEXT=8192 # Context size + - THREADS=8 # Number of CPU threads, required only for CPU deployment + deploy: # GPU resources, required only for GPU deployment + resources: + reservations: + devices: + - driver: nvidia + count: all + capabilities: [gpu] +``` + +```bash +docker-compose -f docker-compose.yaml up +``` + +`docker run` command: + +```bash +# For CPU deployment +docker run -d --restart on-failure:5 -p 8001:8000 -v /root/models:/models -e MODEL=/models/Baichuan2-13B-Chat/ggml-model-f16.gguf -e MODEL_NAME=baichuan13b -e KEY=sk-12345678 openeuler/llm-server:1.0.0-oe2203sp3 + +# For GPU deployment +docker run -d --gpus all --restart on-failure:5 -p 8001:8000 -v /root/models:/models -e MODEL=/models/Baichuan2-13B-Chat/ggml-model-f16.gguf -e MODEL_NAME=baichuan13b -e KEY=sk-12345678 icewangds/llm-server:1.0.0 +``` + +## Testing + +Call the LLM interface to test the deployment. A successful return indicates successful deployment of the LLM service. + +```bash +curl -X POST http://127.0.0.1:8001/v1/chat/completions \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer sk-12345678" \ + -d '{ + "model": "baichuan13b", + "messages": [ + {"role": "system", "content": "You are a openEuler community assistant, please answer the following question."}, + {"role": "user", "content": "Who are you?"} + ], + "stream": false, + "max_tokens": 1024 + }' +``` diff --git a/docs/en/tools/ai/openeuler_copilot_system/_toc.yaml b/docs/en/tools/ai/openeuler_copilot_system/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..e7fcb531a698493dc932f1e98889e41500eb91e1 --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/_toc.yaml @@ -0,0 +1,45 @@ +label: openEuler Intelligent Interaction Platform +isManual: true +href: ./readme.md +description: Deploy and use openEuler Intelligent Interaction Platform. +sections: + - label: User Guide + sections: + - label: Web Client + sections: + - label: Introduction + href: ./user_guide/web_client/introduction.md + - label: Registration and Login + href: ./user_guide/web_client/registration-and-login.md + - label: Intelligent Q&A Guide + href: ./user_guide/web_client/intelligent-q-and-a-guide.md + - label: Intelligent Plugin Overview + href: ./user_guide/web_client/intelligent-plugin-overview.md + - label: CLI Client + sections: + - label: Obtaining the API Key + href: ./user_guide/cli_client/obtaining-the-api-key.md + - label: CLI Assistant Guide + href: ./user_guide/cli_client/cli-assistant-guide.md + - label: Intelligent Tuning + href: ./user_guide/cli_client/intelligent-tuning.md + - label: Intelligent Diagnosis + href: ./user_guide/cli_client/intelligent-diagnosis.md + - label: Deployment Guide + sections: + - label: Network Environment Deployment Guide + href: ./deployment_guide/network-environment-deployment-guide.md + - label: Offline Environment Deployment Guide + href: ./deployment_guide/offline-environment-deployment-guide.md + - label: Local Asset Library Setup Guide + href: ./deployment_guide/local-asset-library-setup-guide.md + sections: + - label: Intelligent Tuning + href: >- + ./deployment_guide/plugin_deployment_guide/intelligent_tuning/plugin-intelligent-tuning-deployment-guide.md + - label: Intelligent Diagnosis + href: >- + ./deployment_guide/plugin_deployment_guide/intelligent_diagnosis/plugin-intelligent-diagnosis-deployment-guide.md + - label: AI Container Stack + href: >- + ./deployment_guide/plugin_deployment_guide/ai_container_stack/plugin-ai-container-stack-deployment-guide.md diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/local-asset-library-setup-guide.md b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/local-asset-library-setup-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..4fa97bffd8f24214a60fed0c2d61045fcc1212f1 --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/local-asset-library-setup-guide.md @@ -0,0 +1,406 @@ +# 本地资产库构建指南 + +- RAG 是一个检索增强的模块，该指南主要是为rag提供命令行的方式进行数据库管理、资产管理、资产库管理和语料资产管理； + 对于数据库管理提供了清空数据库、初始化数据库等功能； + 对于资产管理提供了资产创建、资产查询和资产删除等功能； + 对于资产库管理提供了资产库创建、资产库查询和资产库删除等功能； + 对于语料资产管理提供了语料上传、语料查询和语料删除等功能。 +- 当前指南面向管理员进行编写，对于管理员而言，可以拥有多个资产，一个资产包含多个资产库（不同资产库的使用的向量化模型可以不同），一个资产库对应一个语料资产。 +- 本地语料上传指南是用户构建项目专属语料的指导，当前支持 docx、pdf、markdown、txt 和 xlsx 文件上传，推荐使用 docx 格式上传。 + +## 准备工作 + +- RAG 中关于语料上传目录挂载的配置： + +将本地语料保存到服务器的目录，例如 /home/docs 目录，且将 /home/data 目录权限设置为755 + +```bash +# 设置本地存放文档目录权限为755 +chmod -R 755 /home/docs +``` + +将文件存放的源目录映射至 RAG 容器目标目录，源目录的配置在 中，下面是文件中具体配置映射源目录的配置方法： + +![配置映射源目录](./pictures/local-asset-library-setup/配置映射源目录.png) + +中间层的配置（链接源目录和目标目录的配置）在 中，下面是文件中具体映射中间层的配置方法： + +![配置映射中间层](./pictures/local-asset-library-setup/配置映射中间层.png) + +目标目录的配置在 中，下面是文件中具体映射目标目录的配置方法： + +![配置映射目标目录](./pictures/local-asset-library-setup/配置映射目标目录.png) + +- 更新 Copilot 服务： + + ```bash + root@openeuler:/home/EulerCopilot/euler-copilot-helm/chart# helm upgrade -n euler-copilot service . + # 请注意：service是服务名，可根据实际修改 + ``` + +- 进入到 RAG 容器： + + ```bash + root@openeuler:~# kubectl -n euler-copilot get pods + NAME READY STATUS RESTARTS AGE + framework-deploy-service-bb5b58678-jxzqr 2/2 Running 0 16d + mysql-deploy-service-c7857c7c9-wz9gn 1/1 Running 0 17d + pgsql-deploy-service-86b4dc4899-ppltc 1/1 Running 0 17d + rag-deploy-service-5b7887644c-sm58z 2/2 Running 0 110m + redis-deploy-service-f8866b56-kj9jz 1/1 Running 0 17d + vectorize-deploy-service-57f5f94ccf-sbhzp 2/2 Running 0 17d + web-deploy-service-74fbf7999f-r46rg 1/1 Running 0 2d + # 进入rag pod + root@openeuler:~# kubectl -n euler-copilot exec -it rag-deploy-service-5b7887644c-sm58z -- bash + ``` + +- 设置 PYTHONPATH + + ```bash + # 设置PYTHONPATH + export PYTHONPATH=$(pwd) + ``` + +## 上传语料 + +### 查看脚本帮助信息 + +```bash +python3 scripts/rag_kb_manager.pyc --help +usage: rag_kb_manager.pyc [-h] --method + {init_database_info,init_rag_info,init_database,clear_database,create_kb,del_kb,query_kb,create_kb_asset,del_kb_asset,query_kb_asset,up_corpus,del_corpus,query_corpus,stop_corpus_uploading_job} + [--database_url DATABASE_URL] [--vector_agent_name VECTOR_AGENT_NAME] [--parser_agent_name PARSER_AGENT_NAME] + [--rag_url RAG_URL] [--kb_name KB_NAME] [--kb_asset_name KB_ASSET_NAME] [--corpus_dir CORPUS_DIR] + [--corpus_chunk CORPUS_CHUNK] [--corpus_name CORPUS_NAME] [--up_chunk UP_CHUNK] + [--embedding_model {TEXT2VEC_BASE_CHINESE_PARAPHRASE,BGE_LARGE_ZH,BGE_MIXED_MODEL}] [--vector_dim VECTOR_DIM] + [--num_cores NUM_CORES] + +optional arguments: + -h, --help show this help message and exit + --method {init_database_info,init_rag_info,init_database,clear_database,create_kb,del_kb,query_kb,create_kb_asset,del_kb_asset,query_kb_asset,up_corpus,del_corpus,query_corpus,stop_corpus_uploading_job} + 脚本使用模式，有init_database_info(初始化数据库配置)、init_database(初始化数据库)、clear_database（清除数据库）、create_kb(创建资产)、 + del_kb(删除资产)、query_kb(查询资产)、create_kb_asset(创建资产库)、del_kb_asset(删除资产库)、query_kb_asset(查询 + 资产库)、up_corpus(上传语料,当前支持txt、html、pdf、docx和md格式)、del_corpus(删除语料)、query_corpus(查询语料)和 + stop_corpus_uploading_job(上传语料失败后，停止当前上传任务) + --database_url DATABASE_URL + 语料资产所在数据库的url + --vector_agent_name VECTOR_AGENT_NAME + 向量化插件名称 + --parser_agent_name PARSER_AGENT_NAME + 分词插件名称 + --rag_url RAG_URL rag服务的url + --kb_name KB_NAME 资产名称 + --kb_asset_name KB_ASSET_NAME + 资产库名称 + --corpus_dir CORPUS_DIR + 待上传语料所在路径 + --corpus_chunk CORPUS_CHUNK + 语料切割尺寸 + --corpus_name CORPUS_NAME + 待查询或者待删除语料名 + --up_chunk UP_CHUNK 语料单次上传个数 + --embedding_model {TEXT2VEC_BASE_CHINESE_PARAPHRASE,BGE_LARGE_ZH,BGE_MIXED_MODEL} + 初始化资产时决定使用的嵌入模型 + --vector_dim VECTOR_DIM + 向量化维度 + --num_cores NUM_CORES + 语料处理使用核数 +``` + +### 具体操作 + +以下出现的命令中带**初始化**字段需要在进行资产管理前按指南中出现的相对顺序进行执行，命令中带**可重复**执字段的在后续过程中可以反复执行，命令中带**注意**字段的需谨慎执行。 + +### 步骤1：配置数据库和 RAG 信息 + +- #### 配置数据库信息（初始化） + +```bash +python3 scripts/rag_kb_manager.pyc --method init_database_info --database_url postgresql+psycopg2://postgres:123456@{dabase_url}:{databse_port}/postgres +``` + +**注意：** + +**{dabase_url}**为 k8s 集群内访问 postgres 服务的 url，请根据具体情况修改，一般为 **{postgres_servive_name}-{{ .Release.Name }}.\.svc.cluster.local** 格式，其中 **{postgres_servive_name}** 可以从 找到： + +![k8s集群中postgres服务的名称](./pictures/local-asset-library-setup/k8s集群中postgres服务的名称.png) + +**{{ .Release.Name }}**和**\** 为部署服务时helm安装应用时指定的 **my-release-name** 以及 **my-namespace**，一条 helm 安装应用的命令如下所示： + +```bash +helm install my-release-name --namespace my-namespace path/to/chart +``` + +**database_port** 的信息可以在 中查看，以下为字段所在位置（一般为5432）： + +![postgres服务端口](./pictures/local-asset-library-setup/postgres服务端口.png) + +数据库信息配置命令执行命令完成之后会在 scripts/config 下出现 database_info.json 文件，文件内容如下： + +```bash +{"database_url": "postgresql+psycopg2://postgres:123456@{dabase_url}:{databse_port}/postgres"} +``` + +下面是命令执行成功的截图： + +![数据库配置信息成功](./pictures/local-asset-library-setup/数据库配置信息成功.png) + +- #### 配置rag信息（初始化） + +```bash +python3 scripts/rag_kb_manager.pyc --method init_rag_info --rag_url http://{rag_url}:{rag_port} +``` + +**{rag_url}** 为 0.0.0.0，**{rag_port}** 可以从 中获取（一般为8005）： + +![rag_port](./pictures/local-asset-library-setup/rag_port.png) + +数据库信息配置命令执行命令完成之后会在 scripts/config 下出现 rag_info.json 文件，文件内容如下： + +```bash +{"rag_url": "http://{rag_url}:{rag_port}"} +``` + +下面是命令执行成功的截图： + +![rag配置信息成功](./pictures/local-asset-library-setup/rag配置信息成功.png) + +### 步骤2：初始化数据库 + +- #### 初始化数据库表格 + +```bash +python3 scripts/rag_kb_manager.pyc --method init_database +# 注意： +# 对于特殊关系型数据库可指定插件参数'--vector_agent_name VECTOR_AGENT_NAME'和 '--parser_agent_name PARSER_AGENT_NAME'；其中VECTOR_AGENT_NAME默认为vector, PARSER_AGENT_NAME默认为zhparser +``` + +命令执行完成之后可以进入数据库容器查看表格是否创建成功，首先获取命名空间中的所有节点名称： + +```bash +# 获取命名空间中的所有pod节点 +kubectl get pods -n euler-copilot +``` + +结果如下： + +![获取数据库pod名称](./pictures/local-asset-library-setup/获取数据库pod名称.png) + +使用下面命令进入数据库： + +```bash +kubectl exec -it pgsql-deploy-b4cc79794-qn8zd -n euler-copilot -- bash +``` + +进入容器后使用下面命令进入数据库： + +```bash +root@pgsql-deploy-b4cc79794-qn8zd:/tmp# psql -U postgres +``` + +再使用\dt查看数据库初始化情况，出现下面内容表示数据库初始化成功： + +![数据库初始化](./pictures/local-asset-library-setup/数据库初始化.png) + +- #### 清空数据库（注意） + + 假设您想清空 RAG 产生的所有数据库数据，可以使用下面命令（**此命令会清空整个数据库，需谨慎操作！**）。 + +```bash +python3 scripts/rag_kb_manager.pyc --method clear_database +# 清空数据库请谨慎操作 +``` + +### 步骤3：创建资产 + + 下列指令若不指定 kb_name，则默认资产名为 default_test（ps：Copilot 不允许存在两个同名的资产）： + +- #### 创建资产（可重复） + +```bash +python3 scripts/rag_kb_manager.pyc --method create_kb --kb_name default_test +``` + +创建资产成功会有以下提示： + +![创建资产成功](./pictures/local-asset-library-setup/创建资产成功.png) + +创建同名资产会有以下提示： + +![重复创建资产失败](./pictures/local-asset-library-setup/重复创建资产失败.png) + +- #### 删除资产（可重复） + +```bash +python3 scripts/rag_kb_manager.pyc --method del_kb --kb_name default_test +``` + +删除资产成功会出现以下提示（会将资产下的所有资产库和语料资产全部删除）： + +![删除资产成功](./pictures/local-asset-library-setup/删除资产成功.png) + +对于不存在的资产进行删除，会出现以下提示： + +![删除不存在的资产失败](./pictures/local-asset-library-setup/删除不存在的资产失败.png) + +- #### 查询资产（可重复） + +```bash +python3 scripts/rag_kb_manager.pyc --method query_kb +``` + +查询资产成功会出现下面内容： + +![查询资产](./pictures/local-asset-library-setup/查询资产.png) + +对于无资产的情况下查询资产会出现以下内容： + +![无资产时查询资产](./pictures/local-asset-library-setup/无资产时查询资产.png) + +### 步骤4：创建资产库 + +下列指令若不指定资产名（kb_name）和资产库名（kb_asset_name），则默认资产名为 default_test 和资产库名 default_test_asset（ps：Copilot 同一个资产下不允许存在两个同名的资产库）： + +- #### 创建资产库（可重复） + +```bash +python3 scripts/rag_kb_manager.pyc --method create_kb_asset --kb_name default_test --kb_asset_name default_test_asset +# 创建属于default_test的资产库 +``` + +对于创建资产库成功会出现以下内容： + +![资产库创建成功](./pictures/local-asset-library-setup/资产库创建成功.png) + +对于指定不存在的资产库创建资产会出现以下内容： + +![指定不存在的资产创建资产库失败](./pictures/local-asset-library-setup/指定不存在的资产创建资产库失败.png) + +对于同一个资产下重复创建同名资产库会出现以下内容： + +![创建资产库失败由于统一资产下存在同名资产库](./pictures/local-asset-library-setup/创建资产库失败由于统一资产下存在同名资产库.png) + +- #### 删除资产库（可重复） + +```bash +python3 scripts/rag_kb_manager.pyc --method del_kb_asset --kb_name default_test --kb_asset_name default_test_asset +``` + +对于删除资产库成功会出现以下内容： + +![资产库删除成功](./pictures/local-asset-library-setup/资产库删除成功png.png) + +对于删除不存在的资产库失败会出现以下内容： + +![资产下不存在对应资产库](./pictures/local-asset-library-setup/删除资产库失败，资产下不存在对应资产库.png) + +对于删除不存在的资产下的资产库会出现以下内容： + +![不存在资产](./pictures/local-asset-library-setup/资产库删除失败，不存在资产.png) + +- #### 查询资产库（可重复） + +```bash +python3 scripts/rag_kb_manager.pyc --method query_kb_asset --kb_name default_test +# 注意：资产是最上层的，资产库属于资产，且不能重名 +``` + +对于查询资产库成功会出现以下内容： + +![资产下查询资产库成功](./pictures/local-asset-library-setup/资产下查询资产库成功.png) + +对于资产内无资产库的情况下查询资产库会出现以下内容： + +![资产下未查询到资产库](./pictures/local-asset-library-setup/资产下未查询到资产库.png) + +对于查询不存在的资产下的资产库会出现以下内容： + +![不存在资产](./pictures/local-asset-library-setup/资产库查询失败，不存在资产.png) + +### 步骤5：上传语料 + +下列指令若不指定资产名（kb_name）和资产库名（kb_asset_name），则默认资产名为 default_test 和资产库名 default_test_asset，对于删除语料命令需要指定完整的语料名称（语料统一为 docx 格式保存在数据库中，可以通过查询语料命令查看已上传的文档名称）；对于查询语料命令可以不指定语料名称（corpus_name），此时默认查询所有语料，可以指定部分或者完整的语料名，此时通过模糊搜索匹配数据库内相关的语料名称。 + +- 上传语料 + +```bash +python3 scripts/rag_kb_manager.pyc --method up_corpus --corpus_dir ./scripts/docs/ --kb_name default_test --kb_asset_name default_test_asset +# 注意： +# 1. RAG容器用于存储用户语料的目录路径是'./scripts/docs/'。在执行相关命令前，请确保该目录下已有本地上传的语料。 +# 2. 若语料已上传但查询未果，请检查宿主机上的待向量化语料目录（位于/home/euler-copilot/docs）的权限设置。 +# 为确保无权限问题影响，您可以通过运行chmod 755 /home/euler-copilot/docs命令来赋予该目录最大访问权限。 +``` + +对于语料上传成功会出现以下内容： + +![语料上传成功](./pictures/local-asset-library-setup/语料上传成功.png) + +对于语料具体的分割和上传情况可以在 logs/app.log 下查看，内容如下： + +![查看文档产生片段总数和上传成功总数](./pictures/local-asset-library-setup/查看文档产生片段总数和上传成功总数.png) + +- 删除语料 + +```bash +python3 scripts/rag_kb_manager.pyc --method del_corpus --corpus_name abc.docx --kb_name default_test --kb_asset_name default_test_asset +# 上传的文件统一转换为docx +``` + +对于语料删除成功会出现以下内容： + +![删除语料](./pictures/local-asset-library-setup/删除语料.png) + +对于删除不存在的语料会出现以下内容： + +![语料删除失败](./pictures/local-asset-library-setup/语料删除失败，未查询到相关语料.png) + +- 查询语料 + +```bash +# 查询指定名称的语料： +python3 scripts/rag_kb_manager.pyc --method query_corpus --corpus_name 语料名.docx +# 查询所有语料： +python3 scripts/rag_kb_manager.pyc --method query_corpus +``` + +对于查询所有语料会出现以下内容： + +![查询全部语料](./pictures/local-asset-library-setup/查询全部语料.png) + +- 停止上传任务 + +```bash +python3 scripts/rag_kb_manager.pyc --method stop_corpus_uploading_job +``` + +对于某些极端条件下（例如内存受限），上传语料失败，需要执行上面shell命令用于清除语料上传失败的缓存。 + +## 网页端查看语料上传进度 + +您可以灵活设置端口转发规则，通过执行如下命令将容器端口映射到主机上的指定端口，并在任何设备上通过访问 `http://<主机IP>:<映射端口>`（例如 ）来查看语料上传的详细情况。 + +```bash +kubectl port-forward rag-deploy-service-5b7887644c-sm58z 3000:8005 -n euler-copilot --address=0.0.0.0 +# 注意: 3000是主机上的端口，8005是rag的容器端口，可修改映射到主机上的端口 +``` + +## 验证上传后效果 + +上传语料成功之后你可以通过以下命令直接与 RAG 交互，来观察语料是否上传成功。 + +```bash +curl -k -X POST "http://{rag_url}:{rag_port}/kb/get_answer" -H "Content-Type: application/json" -d '{ \ + "question": "question", \ + "kb_sn": "kb_name", \ + "fetch_source": true, \ + "top_k": 3 \ +}' +``` + +- `question`：问题 + +- `kb_sn`：资产库名称 + +- `fetch_source`：是否返回关联片段以及片段来源，`false` 代表不返回，`true` 代表返回 + +- `top_k`：关联语料片段个数，需要大于等于3 diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/network-environment-deployment-guide.md b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/network-environment-deployment-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..09ba6f5338ed8e8eca13a40d1f44a3fa6a40c9ad --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/network-environment-deployment-guide.md @@ -0,0 +1,617 @@ +# 网络环境部署指南 + +## 介绍 + +openEuler Copilot System 是一款智能问答工具，使用 openEuler Copilot System 可以解决操作系统知识获取的便捷性，并且为OS领域模型赋能开发者及运维人员。作为获取操作系统知识，使能操作系统生产力工具 (如 A-Ops / A-Tune / x2openEuler / EulerMaker / EulerDevOps / StratoVirt / iSulad 等)，颠覆传统命令交付方式，由传统命令交付方式向自然语义进化，并结合智能体任务规划能力，降低开发、使用操作系统特性的门槛。 + +### 组件介绍 + +| 组件 | 端口 | 说明 | +| ----------------------------- | --------------- | -------------------- | +| euler-copilot-framework | 8002 (内部端口) | 智能体框架服务 | +| euler-copilot-web | 8080 | 智能体前端界面 | +| euler-copilot-rag | 8005 (内部端口) | 检索增强服务 | +| euler-copilot-vectorize-agent | 8001 (内部端口) | 文本向量化服务 | +| mysql | 3306 (内部端口) | MySQL数据库 | +| redis | 6379 (内部端口) | Redis数据库 | +| postgres | 5432 (内部端口) | 向量数据库 | +| secret_inject | 无 | 配置文件安全复制工具 | + +## 环境要求 + +### 软件要求 + +| 类型 | 版本要求 | 说明 | +|------------| -------------------------------------|--------------------------------------| +| 操作系统 | openEuler 22.03 LTS 及以上版本 | 无 | +| K3s | >= v1.30.2，带有 Traefik Ingress 工具 | K3s 提供轻量级的 Kubernetes 集群，易于部署和管理 | +| Helm | >= v3.15.3 | Helm 是一个 Kubernetes 的包管理工具，其目的是快速安装、升级、卸载 openEuler Copilot System 服务 | +| python | >=3.9.9 | python3.9.9 以上版本为模型的下载和安装提供运行环境 | + +### 硬件要求 + +| 类型 | 硬件要求 | +|----------------| -----------------------------| +| 服务器 | 1台 | +| CPU | 鲲鹏或x86_64，>= 32 cores | +| RAM | >= 64GB | +| 存储 | >= 500 GB | +| GPU | Tesla V100 16GB，4张 | +| NPU | 910ProB、910B | + +注意： + +1. 若无 GPU 或 NPU 资源，建议通过调用 OpenAI 接口的方式来实现功能。(接口样例： 参考链接：[API-KEY的获取与配置](https://help.aliyun.com/zh/dashscope/developer-reference/acquisition-and-configuration-of-api-key?spm=a2c4g.11186623.0.0.30e7694eaaxxGa)) +2. 调用第三方 OpenAI 接口的方式不需要安装高版本的 python (>=3.9.9) +3. 英伟达 GPU 对 Docker 的支持必需要新版本 Docker (>= v25.4.0) +4. 如果k8s集群环境，则不需要单独安装k3s，要求version >= 1.28 + +### 部署视图 + +![部署图](./pictures/部署视图.png) + +## 获取 openEuler Copilot System + +- 从 openEuler Copilot System 的官方Git仓库 [euler-copilot-framework](https://gitee.com/openeuler/euler-copilot-framework) 下载最新的部署仓库 +- 如果您正在使用 Kubernetes，则不需要安装 k3s 工具。 + +```bash +# 下载目录以 home 为例 +cd /home +``` + +```bash +git clone https://gitee.com/openeuler/euler-copilot-framework.git +``` + +## 环境准备 + +设备需联网并符合 openEuler Copilot System 的最低软硬件要求。确认服务器、硬件、驱动等准备就绪后，即可开始环境准备工作。为了顺利进行后续操作，请按照指引，先进入我 +们的脚本部署目录，并且按照提供的操作步骤和脚本路径依次执行，以确保初始化成功。 + +```bash +# 进入部署脚本目录 +cd /home/euler-copilot-framework/euler-copilot-helm/scripts && tree +``` + +```bash +. +├── check_env.sh +├── download_file.sh +├── get_log.sh +├── install_tools.sh +└── prepare_docker.sh +``` + +| 序号 | 步骤内容 | 相关指令 | 说明 | +|-------------- |----------|---------------------------------------------|------------------------------------------ | +|1| 环境检查 | `bash check_env.sh` | 主要对服务器的主机名、DNS、防火墙设置、磁盘剩余空间大小、网络、检查SELinux的设置 | +|2| 文件下载 | `bash download_file.sh` | 模型bge-reranker-large、bge-mixed-mode下载 | +|3| 安装部署工具 | `bash install_tools.sh v1.30.2+k3s1 v3.15.3 cn` | 安装helm、k3s工具。注意：cn的使用是使用镜像站，可以去掉不用 | +|4| 大模型准备 | 提供第三方 OpenAI 接口或基于硬件本都部署大模型 | 本地部署大模型可参考附录部分 | + +## 安装 + +您的环境现已就绪，接下来即可启动 openEuler Copilot System 的安装流程。 + +- 下载目录以home为例，进入 openEuler Copilot System 仓库的 Helm 配置文件目录 + + ```bash + cd /home/euler-copilot-framework && ll + ``` + + ```bash + total 28 + drwxr-xr-x 3 root root 4096 Aug 28 17:45 docs/ + drwxr-xr-x 5 root root 4096 Aug 28 17:45 euler-copilot-helm/ + ``` + +- 查看euler-copilot-helm的目录 + + ```bash + tree euler-copilot-helm + ``` + + ```bash + euler-copilot-helm/chart + ├── databases + │   ├── Chart.yaml + │   ├── configs + │   ├── templates + │   └── values.yaml + ├── authhub + │   ├── Chart.yaml + │   ├── configs + │   ├── templates + │   └── values.yaml + └── euler_copilot + ├── Chart.yaml + ├── configs + ├── templates + │   ├── NOTES.txt + │   ├── rag + │   ├── vectorize + │   └── web + └── values.yaml + ``` + +### 1. 安装数据库 + +- 编辑 values.yaml + + ```bash + cd euler-copilot-helm/chart/databases + ``` + + 仅需修改镜像tag为对应架构，其余可不进行修改 + + ```bash + vim values.yaml + ``` + +- 创建命名空间 + + ```bash + kubectl create namespace euler-copilot + ``` + + 设置环境变量 + + ```bash + export KUBECONFIG=/etc/rancher/k3s/k3s.yaml + ``` + +- 安装数据库 + + ```bash + helm install -n euler-copilot databases . + ``` + +- 查看 pod 状态 + + ```bash + kubectl -n euler-copilot get pods + ``` + + ```bash + pgsql-deploy-databases-86b4dc4899-ppltc 1/1 Running 0 17d + redis-deploy-databases-f8866b56-kj9jz 1/1 Running 0 17d + mysql-deploy-databases-57f5f94ccf-sbhzp 2/2 Running 0 17d + ``` + +- 若服务器之前部署过 mysql，则可预先清除下 pvc，再部署 databases。 + + ```bash + # 获取pvc + kubectl -n euler-copilot get pvc + ``` + + ```bash + # 删除pvc + kubectl -n euler-copilot delete pvc mysql-pvc + ``` + +### 2. 安装鉴权平台Authhub + +- 编辑 values.yaml + + ```bash + cd euler-copilot-helm/chart/authhub + ``` + + 请结合 YAML 中的注释中的\[必填]项进行修改 + + ```bash + vim values.yaml + ``` + + - 注意： + 1. authHub 需要域名，可预先申请域名或在 'C:\Windows\System32\drivers\etc\hosts' 下配置。 + authhub和euler-copilot必须是同一个根域名的两个子域名, 例如authhub.test.com和eulercopilot.test.com + 2. 修改tag为对应架构的tag; + +- 安装 AuthHub + + ```bash + helm install -n euler-copilot authhub . + ``` + + AuthHub 默认账号 `administrator`, 密码 `changeme` + +- 查看 pod 状态 + + ```bash + kubectl -n euler-copilot get pods + ``` + + ```bash + NAME READY STATUS RESTARTS AGE + authhub-backend-deploy-authhub-64896f5cdc-m497f 2/2 Running 0 16d + authhub-web-deploy-authhub-7c48695966-h8d2p 1/1 Running 0 17d + pgsql-deploy-databases-86b4dc4899-ppltc 1/1 Running 0 17d + redis-deploy-databases-f8866b56-kj9jz 1/1 Running 0 17d + mysql-deploy-databases-57f5f94ccf-sbhzp 2/2 Running 0 17d + ``` + +- 登录 AuthHub + + AuthHub 的域名以 `authhub.test.com` 为例，浏览器输入`https://authhub.test.com`, 登录界面如下图所示： + + ![部署图](./pictures/authhub登录界面.png) + +- 创建应用eulercopilot + + ![部署图](./pictures/创建应用界面.png) + 点击创建应用，输入应用名称、应用主页和应用回调地址（登录后回调地址），参考如下： + - 应用名称：eulercopilot + - 应用主页： + - 应用回调地址： + - 应用创建好后会生成 Client ID 和 Client Secret，将生成的 Client ID 和 Client Secret 配置到应用里，以 eulercopilot 为例，创建应用后在配置文件中添加配置 `euler-copilot-helm/chart/euler_copilot/values.yaml` 中添加配置 + + ![部署图](./pictures/创建应用成功界面.png) + +### 2. 安装 openEuler Copilot System + +- 编辑 values.yaml + + ```bash + cd euler-copilot-helm/chart/euler_copilot + ``` + + 请结合 YAML 中的注释中的\[必填]项进行修改 + + ```bash + vim values.yaml + ``` + + - 注意： + 1. 查看系统架构，并修改values.yaml中的tag； + 2. 修改values.yaml中的globals的domain为EulerCopilot域名，并配置大模型的相关信息 + 3. 手动创建`docs_dir`、`plugin_dir`、`models`三个文件挂载目录 + 4. 修改values.yaml中framework章节的web_url和oidc设置 + 5. 如果部署插件，则需要配置用于Function Call的模型，此时必须有GPU环境用于部署sglang，可参考附件 + +- 安装 openEuler Copilot System + + ```bash + helm install -n euler-copilot service . + ``` + +- 查看 Pod 状态 + + ```bash + kubectl -n euler-copilot get pods + ``` + + 镜像拉取过程可能需要大约一分钟的时间，请耐心等待。部署成功后，所有 Pod 的状态应显示为 Running。 + + ```bash + NAME READY STATUS RESTARTS AGE + authhub-backend-deploy-authhub-64896f5cdc-m497f 2/2 Running 0 16d + authhub-web-deploy-authhub-7c48695966-h8d2p 1/1 Running 0 17d + pgsql-deploy-databases-86b4dc4899-ppltc 1/1 Running 0 17d + redis-deploy-databases-f8866b56-kj9jz 1/1 Running 0 17d + mysql-deploy-databases-57f5f94ccf-sbhzp 2/2 Running 0 17d + framework-deploy-service-bb5b58678-jxzqr 2/2 Running 0 16d + rag-deploy-service-5b7887644c-sm58z 2/2 Running 0 110m + vectorize-deploy-service-57f5f94ccf-sbhzp 2/2 Running 0 17d + web-deploy-service-74fbf7999f-r46rg 1/1 Running 0 2d + ``` + + 注意：如果 Pod 状态出现失败，建议按照以下步骤进行排查 + + 注意：如果 Pod 状态出现失败，建议按照以下步骤进行排查 + + 1. 查看 Kubernetes 集群的事件 (Events)，以获取更多关于 Pod 失败的上下文信息 + + ```bash + kubectl -n euler-copilot get events + ``` + + 2. 查看镜像拉取是否成功 + + ```bash + k3s crictl images + ``` + + 3. 检查 RAG 的 Pod 日志，以确定是否有错误信息或异常行为。 + + ```bash + kubectl logs rag-deploy-service-5b7887644c-sm58z -n euler-copilot + ``` + + 4. 验证 Kubernetes 集群的资源状态，检查服务器资源或配额是否足够，资源不足常导致 Pod 镜像服拉取失败。 + + ```bash + df -h + ``` + + 5. 如果未拉取成且镜像大小为0，请检查是否是 k3s 版本未满足要求，低于 v1.30.2 + + ```bash + k3s -v + ``` + + 6. 确认 values.yaml 中 framework 的 OIDC 设置是否正确配置，以确保身份验证和授权功能正常工作。 + + ```bash + vim /home/euler-copilot-framework/euler-copilot-helm/chart/euler_copilot/values.yaml + ``` + +## 验证安装 + +恭喜您，openEuler Copilot System 的部署已完成！现在，您可以开启智能问答的非凡体验之旅了。 +请在浏览器中输入 或 （其中 port 默认值为8080，若更改则需相应调整）访问 openEuler Copilot System 网页，并尝试进行问答体验。 + +![Web 界面](./pictures/WEB界面.png) + +## 安装插件 + +详细信息请参考文档 [插件部署指南](./plugin_deployment_guide/) + +## 构建专有领域智能问答 + +### 1. 构建 openEuler 专业知识领域的智能问答 + +1. 修改 values.yaml 的 pg 的镜像仓为 `pg-data` +2. 修改 values.yaml 的 rag 部分的字段 `knowledgebaseID: openEuler_2bb3029f` +3. 将 `vim euler-copilot-helm/chart/databases/templates/pgsql/pgsql-deployment.yaml` 的 volumes 相关字段注释 +4. 进入 `cd euler-copilot-helm/chart/databases`，执行更新服务 `helm upgrade -n euler-copilot databases .` +5. 进入 `cd euler-copilot-helm/chart/euler_copilot`，执行更新服务 `helm upgrade -n euler-copilot service .` +6. 进入网页端进行 openEuler 专业知识领域的问答 + +### 2. 构建项目专属知识领域智能问答 + +详细信息请参考文档 [本地资产库构建指南](local-asset-library-setup-guide.md) + +## 附录 + +### 大模型准备 + +#### GPU 环境 + +参考以下方式进行部署 + +1. 下载模型文件： + + ```bash + huggingface-cli download --resume-download Qwen/Qwen1.5-14B-Chat --local-dir Qwen1.5-14B-Chat + ``` + +2. 创建终端 control + + ```bash + screen -S control + ``` + + ```bash + python3 -m fastchat.serve.controller + ``` + + - 按 Ctrl A+D 置于后台 + +3. 创建新终端 api + + ```bash + screen -S api + ``` + + ```bash + python3 -m fastchat.serve.openai_api_server --host 0.0.0.0 --port 30000 --api-keys sk-123456 + ``` + + - 按 Ctrl A+D 置于后台 + - 如果当前环境的 Python 版本是 3.12 或者 3.9 可以创建 python3.10 的 conda 虚拟环境 + + ```bash + mkdir -p /root/py310 + ``` + + ```bash + conda create --prefix=/root/py310 python==3.10.14 + ``` + + ```bash + conda activate /root/py310 + ``` + +4. 创建新终端 worker + + ```bash + screen -S worker + ``` + + ```bash + screen -r worker + ``` + + 安装 fastchat 和 vllm + + ```bash + pip install fschat vllm + ``` + + 安装依赖： + + ```bash + pip install fschat[model_worker] + ``` + + ```bash + python3 -m fastchat.serve.vllm_worker --model-path /root/models/Qwen1.5-14B-Chat/ --model-name qwen1.5 --num-gpus 8 --gpu-memory-utilization=0.7 --dtype=half + ``` + + - 按 Ctrl A+D 置于后台 + +5. 按照如下方式配置文件，并更新服务。 + + ```bash + vim euler-copilot-helm/chart/euler_copilot/values.yaml + ``` + + 修改如下部分 + + ```yaml + llm: + # 开源大模型，OpenAI兼容接口 + openai: + url: "http://$(IP):30000" + key: "sk-123456" + model: qwen1.5 + max_tokens: 8192 + ``` + +#### NPU 环境 + +NPU 环境部署可参考链接 [MindIE安装指南](https://www.hiascend.com/document/detail/zh/mindie/10RC2/whatismindie/mindie_what_0001.html) + +## FAQ + +### 1. huggingface 使用报错？ + +```text +File "/usr/lib/python3.9/site-packages/urllib3/connection.py", line 186, in _new_conn +raise NewConnectionError( +urllib3.exceptions.eanconectionError: : Failed to establish a new conmection: [Errno 101] Network is unreachable +``` + +- 解决办法 + +```bash +pip3 install -U huggingface_hub +``` + +```bash +export HF_ENDPOINT=https://hf-mirror.com +``` + +### 2. 如何在 RAG 容器中调用获取问答结果的接口？ + +- 请先进入到 RAG 对应 Pod + +```bash +curl -k -X POST "http://localhost:8005/kb/get_answer" -H "Content-Type: application/json" -d '{ \ + "question": "", \ + "kb_sn": "default_test", \ + "fetch_source": true }' +``` + +### 3. 执行 `helm upgrade` 报错 + +```text +Error: INSTALLATI0N FAILED: Kubernetes cluster unreachable: Get "http:/localhost:880/version": dial tcp [:1:8089: connect: connection refused +``` + +或者 + +```text +Error: UPGRADE FAILED: Kubernetes cluster unreachable: the server could not find the requested resource +``` + +- 解决办法 + +```bash +export KUBECONFIG=/etc/rancher/k3s/k3s.yaml +``` + +### 4. 无法查看 Pod 日志？ + +```text +[root@localhost euler-copilot]# kubectl logs rag-deployservice65c75c48d8-44vcp-n euler-copilotDefaulted container "rag" out of: rag.rag-copy secret (init)Error from server: Get "https://172.21.31.11:10250/containerlogs/euler copilot/rag deploy"service 65c75c48d8-44vcp/rag": Forbidden +``` + +- 解决办法 + 如果设置了代理，需要将本机的网络 IP 从代理中剔除 + +```bash +cat /etc/systemd/system/k3s.service.env +``` + +```text +http_proxy="http://172.21.60.51:3128" +https_proxy="http://172.21.60.51:3128" +no_proxy=172.21.31.10 # 代理中剔除本机IP +``` + +### 5. GPU环境部署大模型时出现无法流式回复？ + +在服务执行 curl 大模型失败，但是将 `"stream": true` 改为 `"stream": false`就可以 curl 通？ + +```bash +curl http://localhost:30000/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer sk-123456" -d '{ +"model": "qwen1.5", +"messages": [ +{ +"role": "system", +"content": "你是情感分析专家，你的任务是xxxx" +}, +{ +"role": "user", +"content": "你好" +} +], +"stream": true, +"n": 1, +"max_tokens": 32768 +}' +``` + +- 解决办法： + +```bash +pip install Pydantic=1.10.13 +``` + +### 6. 如何部署sglang？ + +```bash +# 1. 激活 Conda 环境, 并激活 Python 3.10 的 Conda 环境。假设你的环境名为 `myenv`： +conda activate myenv + +# 2. 在激活的环境中，安装 sglang[all] 和 flashinfer +pip install sglang[all]==0.3.0 +pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.4/ + +# 3. 启动服务器 +python -m sglang.launch_server --served-model-name Qwen2.5-32B --model-path Qwen2.5-32B-Instruct-AWQ --host 0.0.0.0 --port 8001 --api-key sk-12345 --mem-fraction-static 0.5 --tp 8 +``` + +- 验证安装 + +```bash +pip show sglang +pip show flashinfer +``` + +- 注意： + +1. API Key：请确保 `--api-key` 参数中的 API 密钥是正确的 +2. 模型路径： 确保 `--model-path` 参数中的路径是正确的，并且模型文件存在于该路径下。 +3. CUDA 版本：确保你的系统上安装了 CUDA 12.1 和 PyTorch 2.4，因为 `flashinfer` 包依赖于这些特定版本。 +4. 线程池大小：根据你的GPU资源和预期负载调整线程池大小。如果你有 8 个 GPU，那么可以选择 --tp 8 来充分利用这些资源。 + +### 7. 如何 curl embedding + +```bash +curl -k -X POST http://$IP:8001/embedding \ + -H "Content-Type: application/json" \ + -d '{"texts": ["sample text 1", "sample text 2"]}' +# $IP为vectorize的Embedding的内网地址 +``` + +### 8. 如何生成证书 + +```bash +下载地址： https://github.com/FiloSottile/mkcert/releases +# 1. 下载 mkcert +# x86_64 +wget https://github.com/FiloSottile/mkcert/releases/download/v1.4.4/mkcert-v1.4.4-linux-amd64 +# arm64 +wget https://github.com/FiloSottile/mkcert/releases/download/v1.4.4/mkcert-v1.4.4-linux-arm64 +# 2. 执行下面的命令生成秘钥 +mkcert -install +# mkcert 可直接接域名或 IP, 生成证书和秘钥 +mkcert example.com +# 3. 将证书和秘钥拷贝到 `/home/euler-copilot-framework_openeuler/euler-copilot-helm/chart_ssl/traefik-secret.yaml` 中, 并执行下面命令使其生效。 +kubectl apply -f traefik-secret.yaml +``` diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/offline-environment-deployment-guide.md b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/offline-environment-deployment-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..2b9f7d6fe9a0c4bb0f16fefb69a8875f1625962e --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/offline-environment-deployment-guide.md @@ -0,0 +1,733 @@ +# 无网络环境下部署指南 + +## 介绍 + +openEuler Copilot System 是一款智能问答工具，使用 openEuler Copilot System 可以解决操作系统知识获取的便捷性，并且为OS领域模型赋能开发者及运维人员。作为获取操作系统知识，使能操作系统生产力工具 (如 A-Ops / A-Tune / x2openEuler / EulerMaker / EulerDevOps / StratoVirt / iSulad 等)，颠覆传统命令交付方式，由传统命令交付方式向自然语义进化，并结合智能体任务规划能力，降低开发、使用操作系统特性的门槛。 + +### 组件介绍 + +| 组件 | 端口 | 说明 | +| ----------------------------- | --------------- | -------------------- | +| euler-copilot-framework | 8002 (内部端口) | 智能体框架服务 | +| euler-copilot-web | 8080 | 智能体前端界面 | +| euler-copilot-rag | 8005 (内部端口) | 检索增强服务 | +| euler-copilot-vectorize-agent | 8001 (内部端口) | 文本向量化服务 | +| mysql | 3306 (内部端口) | MySQL数据库 | +| redis | 6379 (内部端口) | Redis数据库 | +| postgres | 5432 (内部端口) | 向量数据库 | +| secret_inject | 无 | 配置文件安全复制工具 | + +## 环境要求 + +### 软件要求 + +| 类型 | 版本要求 | 说明 | +|------------| -------------------------------------|--------------------------------------| +| 操作系统 | openEuler 22.03 LTS 及以上版本 | 无 | +| K3s | >= v1.30.2，带有 Traefik Ingress 工具 | K3s 提供轻量级的 Kubernetes 集群，易于部署和管理 | +| Helm | >= v3.15.3 | Helm 是一个 Kubernetes 的包管理工具，其目的是快速安装、升级、卸载 openEuler Copilot System 服务 | +| python | >=3.9.9 | python3.9.9 以上版本为模型的下载和安装提供运行环境 | + +### 硬件要求 + +| 类型 | 硬件要求 | +|----------------| -----------------------------| +| 服务器 | 1台 | +| CPU | 鲲鹏或x86_64，>= 32 cores | +| RAM | >= 64GB | +| 存储 | >= 500 GB | +| GPU | Tesla V100 16GB，4张 | +| NPU | 910ProB、910B | + +注意： + +1. 若无 GPU 或 NPU 资源，建议通过调用 OpenAI 接口的方式来实现功能。(接口样例：) +2. 调用第三方 OpenAI 接口的方式不需要安装高版本的 python (>=3.9.9) +3. 英伟达 GPU 对 Docker 的支持必需要新版本 Docker (>= v25.4.0) + +### 部署视图 + +![部署图](./pictures/部署视图.png) + +## 获取 openEuler Copilot System + +- 从 openEuler Copilot System 的官方Git仓库 [euler-copilot-framework](https://gitee.com/openeuler/euler-copilot-framework) 下载最新的部署仓库 +- 如果您正在使用 Kubernetes，则不需要安装 k3s 工具。 + + ```bash + # 下载目录以 home 为例 + cd /home + ``` + + ```bash + git clone https://gitee.com/openeuler/euler-copilot-framework.git + ``` + +## 环境准备 + +如果您的服务器、硬件、驱动等全部就绪，即可启动环境初始化流程，以下部署步骤在无公网环境执行。 + +### 1. 环境检查 + +环境检查主要是对服务器的主机名、DNS、防火墙设置、磁盘剩余空间大小、网络、检查 SELinux 的设置。 + +- 主机名设置 + 在Shell中运行如下命令： + + ```bash + cat /etc/hostname + echo "主机名" > /etc/hostname + ``` + +- 系统DNS设置：需要给当前主机设置有效的DNS +- 防火墙设置 + + ```bash + # 查看防火墙状态 + systemctl status firewalld + # 查看防火墙列表 + firewall-cmd --list-all + # 关闭防火墙 + systemctl stop firewalld + systemctl disable firewalld + ``` + +- SELinux设置 + + ```bash + # 需要关闭selinux，可以临时关闭或永久关闭 + # 永久关闭SELinux + sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config + # 临时关闭 + setenforce 0 + ``` + +### 2. 文件下载 + +- 模型文件 bge-reranker-large、bge-mixed-model 下载 [模型文件下载链接](https://repo.oepkgs.net/openEuler/rpm/openEuler-22.03-LTS/contrib/EulerCopilot/) + + ```bash + mkdir -p /home/EulerCopilot/models + cd /home/EulerCopilot/models + # 将需要下载的bge文件放置在models目录 + wget https://repo.oepkgs.net/openEuler/rpm/openEuler-22.03-LTS/contrib/EulerCopilot/bge-mixed-model.tar.gz + wget https://repo.oepkgs.net/openEuler/rpm/openEuler-22.03-LTS/contrib/EulerCopilot/bge-reranker-large.tar.gz + ``` + +- 下载分词工具 text2vec-base-chinese-paraphrase [分词工具下载链接](https://repo.oepkgs.net/openEuler/rpm/openEuler-22.03-LTS/contrib/EulerCopilot/) + + ```bash + mkdir -p /home/EulerCopilot/text2vec + cd /home/EulerCopilot/text2vec + wget https://repo.oepkgs.net/openEuler/rpm/openEuler-22.03-LTS/contrib/EulerCopilot/text2vec-base-chinese-paraphrase.tar.gz + ``` + +- 镜像包下载 + - x86或arm架构的EulerCopilot服务的各组件镜像单独提供 + +### 3. 安装部署工具 + +#### 3.1 安装 Docker + +如需要基于 GPU/NPU 部署大模型，需要检查 Docker 版本是否满足>= v25.4.0 ，如不满足，请升级 Docker 版本 + +#### 3.2 安装 K3s 并导入镜像 + +- 安装 SELinux 配置文件 + + ```bash + yum install -y container-selinux selinux-policy-base + # packages里有k3s-selinux-0.1.1-rc1.el7.noarch.rpm的离线包 + rpm -i https://rpm.rancher.io/k3s-selinux-0.1.1-rc1.el7.noarch.rpm + ``` + +- x86 架构安装 k3s + + ```bash + # 在有网络的环境上获取k3s相关包，以v1.30.3+k3s1示例 + wget https://github.com/k3s-io/k3s/releases/download/v1.30.3%2Bk3s1/k3s + wget https://github.com/k3s-io/k3s/releases/download/v1.30.3%2Bk3s1/k3s-airgap-images-amd64.tar.zst + cp k3s /usr/local/bin/ + cd /var/lib/rancher/k3s/agent + mkdir images + cp k3s-airgap-images-arm64.tar.zst /var/lib/rancher/k3s/agent/images + # packages里有k3s-install.sh的离线包 + curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh + INSTALL_K3S_SKIP_DOWNLOAD=true ./k3s-install.sh + export KUBECONFIG=/etc/rancher/k3s/k3s.yaml + ``` + +- arm 架构安装 k3s + + ```bash + # 在有网络的环境上获取k3s相关包，以v1.30.3+k3s1示例 + wget https://github.com/k3s-io/k3s/releases/download/v1.30.3%2Bk3s1/k3s-arm64 + wget https://github.com/k3s-io/k3s/releases/download/v1.30.3%2Bk3s1/k3s-airgap-images-arm64.tar.zst + cp k3s-arm64 /usr/local/bin/k3s + cd /var/lib/rancher/k3s/agent + mkdir images + cp k3s-airgap-images-arm64.tar.zst /var/lib/rancher/k3s/agent/images + # packages里有k3s-install.sh的离线包 + curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh + INSTALL_K3S_SKIP_DOWNLOAD=true ./k3s-install.sh + export KUBECONFIG=/etc/rancher/k3s/k3s.yaml + ``` + +- 导入镜像 + + ```bash + # 导入已下载的镜像文件 + k3s ctr image import $(镜像文件) + ``` + +#### 3.3 安装 Helm 工具 + +- x86_64 架构 + + ```bash + wget https://get.helm.sh/helm-v3.15.0-linux-amd64.tar.gz + tar -xzf helm-v3.15.0-linux-amd64.tar.gz + mv linux-amd64/helm /usr/sbin + rm -rf linux-amd64 + ``` + +- arm64 架构 + + ```bash + wget https://get.helm.sh/helm-v3.15.0-linux-arm64.tar.gz + tar -xzf helm-v3.15.0-linux-arm64.tar.gz + mv linux-arm64/helm /usr/sbin + rm -rf linux-arm64 + ``` + +#### 3.4 大模型准备 + +提供第三方openai接口或基于硬件本都部署大模型，本地部署大模型可参考附录部分。 + +## 安装 + +您的环境现已就绪，接下来即可启动 openEuler Copilot System 的安装流程。 + +- 下载目录以home为例，进入 openEuler Copilot System 仓库的 Helm 配置文件目录 + + ```bash + cd /home/euler-copilot-framework && ll + ``` + + ```bash + total 28 + drwxr-xr-x 3 root root 4096 Aug 28 17:45 docs/ + drwxr-xr-x 5 root root 4096 Aug 28 17:45 euler-copilot-helm/ + ``` + +- 查看euler-copilot-helm的目录 + + ```bash + tree euler-copilot-helm + ``` + + ```bash + euler-copilot-helm/chart + ├── databases + │ ├── Chart.yaml + │ ├── configs + │ ├── templates + │ └── values.yaml + ├── authhub + │ ├── Chart.yaml + │ ├── configs + │ ├── templates + │ └── values.yaml + └── euler_copilot + ├── Chart.yaml + ├── configs + ├── templates + │ ├── NOTES.txt + │ ├── rag + │ ├── vectorize + │ └── web + └── values.yaml + ``` + +### 1. 安装数据库 + +- 编辑 values.yaml + + ```bash + cd euler-copilot-helm/chart/databases + ``` + + 仅需修改镜像tag为对应架构，其余可不进行修改 + + ```bash + vim values.yaml + ``` + +- 创建命名空间 + + ```bash + kubectl create namespace euler-copilot + ``` + + 设置环境变量 + + ```bash + export KUBECONFIG=/etc/rancher/k3s/k3s.yaml + ``` + +- 安装数据库 + + ```bash + helm install -n euler-copilot databases . + ``` + +- 查看 pod 状态 + + ```bash + kubectl -n euler-copilot get pods + ``` + + ```bash + pgsql-deploy-databases-86b4dc4899-ppltc 1/1 Running 0 17d + redis-deploy-databases-f8866b56-kj9jz 1/1 Running 0 17d + mysql-deploy-databases-57f5f94ccf-sbhzp 2/2 Running 0 17d + ``` + +- 若服务器之前部署过 mysql，则可预先清除下 pvc，再部署 databases。 + + ```bash + # 获取pvc + kubectl -n euler-copilot get pvc + ``` + + ```bash + # 删除pvc + kubectl -n euler-copilot delete pvc mysql-pvc + ``` + +### 2. 安装鉴权平台Authhub + +- 编辑 values.yaml + + ```bash + cd euler-copilot-helm/chart/authhub + ``` + + 请结合 YAML 中的注释中的\[必填]项进行修改 + + ```bash + vim values.yaml + ``` + + - 注意： + 1. authHub 需要域名，可预先申请域名或在 'C:\Windows\System32\drivers\etc\hosts' 下配置。 + authhub和euler-copilot必须是同一个根域名的两个子域名, 例如authhub.test.com和 + eulercopilot.test.com + 2. 修改tag为对应架构的tag; + +- 安装 AuthHub + + ```bash + helm install -n euler-copilot authhub . + ``` + + AuthHub 默认账号 `administrator`, 密码 `changeme` + +- 查看 pod 状态 + + ```bash + kubectl -n euler-copilot get pods + ``` + + ```bash + NAME READY STATUS RESTARTS AGE + authhub-backend-deploy-authhub-64896f5cdc-m497f 2/2 Running 0 16d + authhub-web-deploy-authhub-7c48695966-h8d2p 1/1 Running 0 17d + pgsql-deploy-databases-86b4dc4899-ppltc 1/1 Running 0 17d + redis-deploy-databases-f8866b56-kj9jz 1/1 Running 0 17d + mysql-deploy-databases-57f5f94ccf-sbhzp 2/2 Running 0 17d + ``` + +- 登录 AuthHub + + AuthHub 的域名以 `authhub.test.com` 为例，浏览器输入`https://authhub.test.com`, 登录界面如下图所示： + + ![部署图](./pictures/authhub登录界面.png) + +- 创建应用eulercopilot + + ![部署图](./pictures/创建应用界面.png) + + 点击创建应用，输入应用名称、应用主页和应用回调地址（登录后回调地址），参考如下： + - 应用名称：eulercopilot + - 应用主页： + - 应用回调地址： + - 应用创建好后会生成 Client ID 和 Client Secret，将生成的 Client ID 和 Client Secret 配置到应用里，以 eulercopilot 为例，创建应用后在配置文件中添加配置 `euler-copilot-helm/chart/euler_copilot/values.yaml` 中添加配置 + + ![部署图](./pictures/创建应用成功界面.png) + +### 2. 安装 openEuler Copilot System + +- 编辑 values.yaml + + ```bash + cd euler-copilot-helm/chart/euler_copilot + ``` + + 请结合 YAML 中的注释中的\[必填]项进行修改 + + ```bash + vim values.yaml + ``` + + - 注意： + 1. 查看系统架构，并修改values.yaml中的tag； + 2. 修改values.yaml中的globals的domain为EulerCopilot域名，并配置大模型的相关信息 + 3. 手动创建`docs_dir`、`plugin_dir`、`models`三个文件挂载目录 + 4. 修改values.yaml中framework章节的web_url和oidc设置 + 5. 如果部署插件，则需要配置用于Function Call的模型，此时必须有GPU环境用于部署sglang，可参考附件 + +- 安装 openEuler Copilot System + + ```bash + helm install -n euler-copilot service . + ``` + +- 查看 Pod 状态 + + ```bash + kubectl -n euler-copilot get pods + ``` + + 镜像拉取过程可能需要大约一分钟的时间，请耐心等待。部署成功后，所有 Pod 的状态应显示为 Running。 + + ```bash + NAME READY STATUS RESTARTS AGE + authhub-backend-deploy-authhub-64896f5cdc-m497f 2/2 Running 0 16d + authhub-web-deploy-authhub-7c48695966-h8d2p 1/1 Running 0 17d + pgsql-deploy-databases-86b4dc4899-ppltc 1/1 Running 0 17d + redis-deploy-databases-f8866b56-kj9jz 1/1 Running 0 17d + mysql-deploy-databases-57f5f94ccf-sbhzp 2/2 Running 0 17d + framework-deploy-service-bb5b58678-jxzqr 2/2 Running 0 16d + rag-deploy-service-5b7887644c-sm58z 2/2 Running 0 110m + vectorize-deploy-service-57f5f94ccf-sbhzp 2/2 Running 0 17d + web-deploy-service-74fbf7999f-r46rg 1/1 Running 0 2d + ``` + + 注意：如果 Pod 状态出现失败，建议按照以下步骤进行排查 + + 1. 查看 Kubernetes 集群的事件 (Events)，以获取更多关于 Pod 失败的上下文信息 + + ```bash + kubectl -n euler-copilot get events + ``` + + 2. 查看镜像拉取是否成功 + + ```bash + k3s crictl images + ``` + + 3. 检查 RAG 的 Pod 日志，以确定是否有错误信息或异常行为。 + + ```bash + kubectl logs rag-deploy-service-5b7887644c-sm58z -n euler-copilot + ``` + + 4. 验证 Kubernetes 集群的资源状态，检查服务器资源或配额是否足够，资源不足常导致 Pod 镜像服拉取失败。 + + ```bash + df -h + ``` + + 5. 如果未拉取成且镜像大小为0，请检查是否是 k3s 版本未满足要求，低于 v1.30.2 + + ```bash + k3s -v + ``` + + 6. 确认 values.yaml 中 framework 的 OIDC 设置是否正确配置，以确保身份验证和授权功能正常工作。 + + ```bash + vim /home/euler-copilot-framework/euler-copilot-helm/chart/euler_copilot/values.yaml + ``` + +## 验证安装 + +恭喜您，openEuler Copilot System 的部署已完成！现在，您可以开启智能问答的非凡体验之旅了。 +请在浏览器中输入 或 （其中 port 默认值为8080，若更改则需相应调整）访问 openEuler Copilot System 网页，并尝试进行问答体验。 + +![Web 界面](./pictures/WEB界面.png) + +## 安装插件 + +详细信息请参考文档 [插件部署指南](./plugin_deployment_guide/) + +## 构建专有领域智能问答 + +### 1. 构建 openEuler 专业知识领域的智能问答 + +1. 修改 values.yaml 的 pg 的镜像仓为 `pg-data` +2. 修改 values.yaml 的 rag 部分的字段 `knowledgebaseID: openEuler_2bb3029f` +3. 将 `vim euler-copilot-helm/chart/databases/templates/pgsql/pgsql-deployment.yaml` 的 volumes 相关字段注释 +4. 进入 `cd euler-copilot-helm/chart/databases`，执行更新服务 `helm upgrade -n euler-copilot databases .` +5. 进入 `cd euler-copilot-helm/chart/euler_copilot`，执行更新服务 `helm upgrade -n euler-copilot service .` +6. 进入网页端进行 openEuler 专业知识领域的问答 + +### 2. 构建项目专属知识领域智能问答 + +详细信息请参考文档 [本地资产库构建指南](local-asset-library-setup-guide.md) + +## 附录 + +### 大模型准备 + +#### GPU 环境 + +参考以下方式进行部署 + +1. 下载模型文件： + + ```bash + huggingface-cli download --resume-download Qwen/Qwen1.5-14B-Chat --local-dir Qwen1.5-14B-Chat + ``` + +2. 创建终端 control + + ```bash + screen -S control + ``` + + ```bash + python3 -m fastchat.serve.controller + ``` + + - 按 Ctrl A+D 置于后台 + +3. 创建新终端 api + + ```bash + screen -S api + ``` + + ```bash + python3 -m fastchat.serve.openai_api_server --host 0.0.0.0 --port 30000 --api-keys sk-123456 + ``` + + - 按 Ctrl A+D 置于后台 + - 如果当前环境的 Python 版本是 3.12 或者 3.9 可以创建 python3.10 的 conda 虚拟环境 + + ```bash + mkdir -p /root/py310 + ``` + + ```bash + conda create --prefix=/root/py310 python==3.10.14 + ``` + + ```bash + conda activate /root/py310 + ``` + +4. 创建新终端 worker + + ```bash + screen -S worker + ``` + + ```bash + screen -r worker + ``` + + 安装 fastchat 和 vllm + + ```bash + pip install fschat vllm + ``` + + 安装依赖： + + ```bash + pip install fschat[model_worker] + ``` + + ```bash + python3 -m fastchat.serve.vllm_worker --model-path /root/models/Qwen1.5-14B-Chat/ --model-name qwen1.5 --num-gpus 8 --gpu-memory-utilization=0.7 --dtype=half + ``` + + - 按 Ctrl A+D 置于后台 + +5. 按照如下方式配置文件，并更新服务。 + + ```bash + vim euler-copilot-helm/chart/euler_copilot/values.yaml + ``` + + 修改如下部分 + + ```yaml + llm: + # 开源大模型，OpenAI兼容接口 + openai: + url: "http://$(IP):30000" + key: "sk-123456" + model: qwen1.5 + max_tokens: 8192 + ``` + +#### NPU 环境 + +NPU 环境部署可参考链接 [MindIE安装指南](https://www.hiascend.com/document/detail/zh/mindie/10RC2/whatismindie/mindie_what_0001.html) + +## FAQ + +### 1. huggingface 使用报错？ + +```text +File "/usr/lib/python3.9/site-packages/urllib3/connection.py", line 186, in _new_conn +raise NewConnectionError( +urllib3.exceptions.eanconectionError: : Failed to establish a new conmection: [Errno 101] Network is unreachable +``` + +- 解决办法 + +```bash +pip3 install -U huggingface_hub +``` + +```bash +export HF_ENDPOINT=https://hf-mirror.com +``` + +### 2. 如何在 RAG 容器中调用获取问答结果的接口？ + +- 请先进入到 RAG 对应 Pod + +```bash +curl -k -X POST "http://localhost:8005/kb/get_answer" -H "Content-Type: application/json" -d '{ \ + "question": "", \ + "kb_sn": "default_test", \ + "fetch_source": true }' +``` + +### 3. 执行 `helm upgrade` 报错 + +```text +Error: INSTALLATI0N FAILED: Kubernetes cluster unreachable: Get "http:/localhost:880/version": dial tcp [:1:8089: connect: connection refused +``` + +或者 + +```text +Error: UPGRADE FAILED: Kubernetes cluster unreachable: the server could not find the requested resource +``` + +- 解决办法 + +```bash +export KUBECONFIG=/etc/rancher/k3s/k3s.yaml +``` + +### 4. 无法查看 Pod 日志？ + +```text +[root@localhost euler-copilot]# kubectl logs rag-deployservice65c75c48d8-44vcp-n euler-copilotDefaulted container "rag" out of: rag.rag-copy secret (init)Error from server: Get "https://172.21.31.11:10250/containerlogs/euler copilot/rag deploy"service 65c75c48d8-44vcp/rag": Forbidden +``` + +- 解决办法 + 如果设置了代理，需要将本机的网络 IP 从代理中剔除 + +```bash +cat /etc/systemd/system/k3s.service.env +``` + +```text +http_proxy="http://172.21.60.51:3128" +https_proxy="http://172.21.60.51:3128" +no_proxy=172.21.31.10 # 代理中剔除本机IP +``` + +### 5. GPU环境部署大模型时出现无法流式回复？ + +在服务执行 curl 大模型失败，但是将 `"stream": true` 改为 `"stream": false`就可以 curl 通？ + +```bash +curl http://localhost:30000/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer sk-123456" -d '{ +"model": "qwen1.5", +"messages": [ +{ +"role": "system", +"content": "你是情感分析专家，你的任务是xxxx" +}, +{ +"role": "user", +"content": "你好" +} +], +"stream": true, +"n": 1, +"max_tokens": 32768 +}' +``` + +- 解决办法： + +```bash +pip install Pydantic=1.10.13 +``` + +### 6. 如何部署sglang？ + +```bash +# 1. 激活 Conda 环境, 并激活 Python 3.10 的 Conda 环境。假设你的环境名为 `myenv`： +conda activate myenv + +# 2. 在激活的环境中，安装 sglang[all] 和 flashinfer +pip install sglang[all]==0.3.0 +pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.4/ + +# 3. 启动服务器 +python -m sglang.launch_server --served-model-name Qwen2.5-32B --model-path Qwen2.5-32B-Instruct-AWQ --host 0.0.0.0 --port 8001 --api-key sk-12345 --mem-fraction-static 0.5 --tp 8 +``` + +- 验证安装 + +```bash +pip show sglang +pip show flashinfer +``` + +- 注意： + +1. API Key：请确保 `--api-key` 参数中的 API 密钥是正确的 +2. 模型路径： 确保 `--model-path` 参数中的路径是正确的，并且模型文件存在于该路径下。 +3. CUDA 版本：确保你的系统上安装了 CUDA 12.1 和 PyTorch 2.4，因为 `flashinfer` 包依赖于这些特定版本。 +4. 线程池大小：根据你的GPU资源和预期负载调整线程池大小。如果你有 8 个 GPU，那么可以选择 --tp 8 来充分利用这些资源。 + +### 7. 如何 curl embedding + +```bash +curl -k -X POST http://$IP:8001/embedding \ + -H "Content-Type: application/json" \ + -d '{"texts": ["sample text 1", "sample text 2"]}' +# $IP为vectorize的Embedding的内网地址 +``` + +### 8. 如何生成证书？ + +```bash +下载地址： https://github.com/FiloSottile/mkcert/releases +# 1. 下载 mkcert +# x86_64 +wget https://github.com/FiloSottile/mkcert/releases/download/v1.4.4/mkcert-v1.4.4-linux-amd64 +# arm64 +wget https://github.com/FiloSottile/mkcert/releases/download/v1.4.4/mkcert-v1.4.4-linux-arm64 + +# 2. 执行下面的命令生成秘钥 +mkcert -install +# mkcert 可直接接域名或 IP, 生成证书和秘钥 +mkcert example.com + +# 3. 将证书和秘钥拷贝到 /home/euler-copilot-framework_openeuler/euler-copilot-helm/chart_ssl/traefik-secret.yaml 中, 并执行下面命令使其生效。 +kubectl apply -f traefik-secret.yaml +``` diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/WEB\347\225\214\351\235\242.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/WEB\347\225\214\351\235\242.png" new file mode 100644 index 0000000000000000000000000000000000000000..bb9be4e33ce470865fe5a07decbc056b9ee4e9bb Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/WEB\347\225\214\351\235\242.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/authhub\347\231\273\345\275\225\347\225\214\351\235\242.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/authhub\347\231\273\345\275\225\347\225\214\351\235\242.png" new file mode 100644 index 0000000000000000000000000000000000000000..341828b1b6f728888d1dd52eec755033680155da Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/authhub\347\231\273\345\275\225\347\225\214\351\235\242.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/k8s\351\233\206\347\276\244\344\270\255postgres\346\234\215\345\212\241\347\232\204\345\220\215\347\247\260.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/k8s\351\233\206\347\276\244\344\270\255postgres\346\234\215\345\212\241\347\232\204\345\220\215\347\247\260.png" new file mode 100644 index 0000000000000000000000000000000000000000..473a0006c9710c92375e226a760c3a79989312f9 Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/k8s\351\233\206\347\276\244\344\270\255postgres\346\234\215\345\212\241\347\232\204\345\220\215\347\247\260.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/postgres\346\234\215\345\212\241\347\253\257\345\217\243.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/postgres\346\234\215\345\212\241\347\253\257\345\217\243.png" new file mode 100644 index 0000000000000000000000000000000000000000..cfee6d88da56bc939886caece540f7de8cf77bbc Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/postgres\346\234\215\345\212\241\347\253\257\345\217\243.png" differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/rag_port.png b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/rag_port.png new file mode 100644 index 0000000000000000000000000000000000000000..b1d93f9c9d7587aa88a27d7e0bf185586583d438 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/rag_port.png differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/rag\351\205\215\347\275\256\344\277\241\346\201\257\346\210\220\345\212\237.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/rag\351\205\215\347\275\256\344\277\241\346\201\257\346\210\220\345\212\237.png" new file mode 100644 index 0000000000000000000000000000000000000000..fec3cdaa2b260e50f5523477da3e58a9e14e2130 Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/rag\351\205\215\347\275\256\344\277\241\346\201\257\346\210\220\345\212\237.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\345\210\233\345\273\272\350\265\204\344\272\247\345\272\223\345\244\261\350\264\245\347\224\261\344\272\216\347\273\237\344\270\200\350\265\204\344\272\247\344\270\213\345\255\230\345\234\250\345\220\214\345\220\215\350\265\204\344\272\247\345\272\223.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\345\210\233\345\273\272\350\265\204\344\272\247\345\272\223\345\244\261\350\264\245\347\224\261\344\272\216\347\273\237\344\270\200\350\265\204\344\272\247\344\270\213\345\255\230\345\234\250\345\220\214\345\220\215\350\265\204\344\272\247\345\272\223.png" new file mode 100644 index 0000000000000000000000000000000000000000..624459821de4542b635eeffa115eeba780929a4e Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\345\210\233\345\273\272\350\265\204\344\272\247\345\272\223\345\244\261\350\264\245\347\224\261\344\272\216\347\273\237\344\270\200\350\265\204\344\272\247\344\270\213\345\255\230\345\234\250\345\220\214\345\220\215\350\265\204\344\272\247\345\272\223.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\345\210\233\345\273\272\350\265\204\344\272\247\346\210\220\345\212\237.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\345\210\233\345\273\272\350\265\204\344\272\247\346\210\220\345\212\237.png" new file mode 100644 index 0000000000000000000000000000000000000000..3104717bfa8f6615ad6726577a24938bc29884b2 Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\345\210\233\345\273\272\350\265\204\344\272\247\346\210\220\345\212\237.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\345\210\240\351\231\244\344\270\215\345\255\230\345\234\250\347\232\204\350\265\204\344\272\247\345\244\261\350\264\245.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\345\210\240\351\231\244\344\270\215\345\255\230\345\234\250\347\232\204\350\265\204\344\272\247\345\244\261\350\264\245.png" new file mode 100644 index 0000000000000000000000000000000000000000..454b9fdfa4b7f209dc370f78677a2f4e71ea49be Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\345\210\240\351\231\244\344\270\215\345\255\230\345\234\250\347\232\204\350\265\204\344\272\247\345\244\261\350\264\245.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\345\210\240\351\231\244\350\257\255\346\226\231.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\345\210\240\351\231\244\350\257\255\346\226\231.png" new file mode 100644 index 0000000000000000000000000000000000000000..d52d25d4778f6db2d2ec076d65018c40cd1da4d3 Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\345\210\240\351\231\244\350\257\255\346\226\231.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\345\210\240\351\231\244\350\265\204\344\272\247\345\272\223\345\244\261\350\264\245\357\274\214\350\265\204\344\272\247\344\270\213\344\270\215\345\255\230\345\234\250\345\257\271\345\272\224\350\265\204\344\272\247\345\272\223.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\345\210\240\351\231\244\350\265\204\344\272\247\345\272\223\345\244\261\350\264\245\357\274\214\350\265\204\344\272\247\344\270\213\344\270\215\345\255\230\345\234\250\345\257\271\345\272\224\350\265\204\344\272\247\345\272\223.png" new file mode 100644 index 0000000000000000000000000000000000000000..82ed79c0154bd8e406621440c4e4a7caaab7e06e Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\345\210\240\351\231\244\350\265\204\344\272\247\345\272\223\345\244\261\350\264\245\357\274\214\350\265\204\344\272\247\344\270\213\344\270\215\345\255\230\345\234\250\345\257\271\345\272\224\350\265\204\344\272\247\345\272\223.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\345\210\240\351\231\244\350\265\204\344\272\247\346\210\220\345\212\237.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\345\210\240\351\231\244\350\265\204\344\272\247\346\210\220\345\212\237.png" new file mode 100644 index 0000000000000000000000000000000000000000..7dd2dea945f39ada1d7dd053d150a995b160f203 Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\345\210\240\351\231\244\350\265\204\344\272\247\346\210\220\345\212\237.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\346\214\207\345\256\232\344\270\215\345\255\230\345\234\250\347\232\204\350\265\204\344\272\247\345\210\233\345\273\272\350\265\204\344\272\247\345\272\223\345\244\261\350\264\245.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\346\214\207\345\256\232\344\270\215\345\255\230\345\234\250\347\232\204\350\265\204\344\272\247\345\210\233\345\273\272\350\265\204\344\272\247\345\272\223\345\244\261\350\264\245.png" new file mode 100644 index 0000000000000000000000000000000000000000..be89bdfde2518bba3941eee5d475f52ad9124343 Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\346\214\207\345\256\232\344\270\215\345\255\230\345\234\250\347\232\204\350\265\204\344\272\247\345\210\233\345\273\272\350\265\204\344\272\247\345\272\223\345\244\261\350\264\245.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\346\225\260\346\215\256\345\272\223\345\210\235\345\247\213\345\214\226.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\346\225\260\346\215\256\345\272\223\345\210\235\345\247\213\345\214\226.png" new file mode 100644 index 0000000000000000000000000000000000000000..27530840aaa5382a226e1ed8baea883895d9d75e Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\346\225\260\346\215\256\345\272\223\345\210\235\345\247\213\345\214\226.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\346\225\260\346\215\256\345\272\223\351\205\215\347\275\256\344\277\241\346\201\257\346\210\220\345\212\237.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\346\225\260\346\215\256\345\272\223\351\205\215\347\275\256\344\277\241\346\201\257\346\210\220\345\212\237.png" new file mode 100644 index 0000000000000000000000000000000000000000..aa04e6f7f0648adfca1240c750ca5b79b88da5f9 Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\346\225\260\346\215\256\345\272\223\351\205\215\347\275\256\344\277\241\346\201\257\346\210\220\345\212\237.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\346\227\240\350\265\204\344\272\247\346\227\266\346\237\245\350\257\242\350\265\204\344\272\247.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\346\227\240\350\265\204\344\272\247\346\227\266\346\237\245\350\257\242\350\265\204\344\272\247.png" new file mode 100644 index 0000000000000000000000000000000000000000..74905172c0c0a0acc4c4d0e35efd2493dc421c4e Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\346\227\240\350\265\204\344\272\247\346\227\266\346\237\245\350\257\242\350\265\204\344\272\247.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\346\237\245\347\234\213\346\226\207\346\241\243\344\272\247\347\224\237\347\211\207\346\256\265\346\200\273\346\225\260\345\222\214\344\270\212\344\274\240\346\210\220\345\212\237\346\200\273\346\225\260.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\346\237\245\347\234\213\346\226\207\346\241\243\344\272\247\347\224\237\347\211\207\346\256\265\346\200\273\346\225\260\345\222\214\344\270\212\344\274\240\346\210\220\345\212\237\346\200\273\346\225\260.png" new file mode 100644 index 0000000000000000000000000000000000000000..432fbfcd02f6d2220e7d2a8512aee893d67be24d Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\346\237\245\347\234\213\346\226\207\346\241\243\344\272\247\347\224\237\347\211\207\346\256\265\346\200\273\346\225\260\345\222\214\344\270\212\344\274\240\346\210\220\345\212\237\346\200\273\346\225\260.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\346\237\245\350\257\242\345\205\250\351\203\250\350\257\255\346\226\231.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\346\237\245\350\257\242\345\205\250\351\203\250\350\257\255\346\226\231.png" new file mode 100644 index 0000000000000000000000000000000000000000..a4f4ea8a3999a9ab659ccd9ea39b80b21ff46e84 Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\346\237\245\350\257\242\345\205\250\351\203\250\350\257\255\346\226\231.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\346\237\245\350\257\242\350\265\204\344\272\247.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\346\237\245\350\257\242\350\265\204\344\272\247.png" new file mode 100644 index 0000000000000000000000000000000000000000..675b40297363664007f96948fb21b1cb90d6beea Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\346\237\245\350\257\242\350\265\204\344\272\247.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\216\267\345\217\226\346\225\260\346\215\256\345\272\223pod\345\220\215\347\247\260.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\216\267\345\217\226\346\225\260\346\215\256\345\272\223pod\345\220\215\347\247\260.png" new file mode 100644 index 0000000000000000000000000000000000000000..8fc0c988e8b3830c550c6be6e42b88ac13448d1a Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\216\267\345\217\226\346\225\260\346\215\256\345\272\223pod\345\220\215\347\247\260.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\257\255\346\226\231\344\270\212\344\274\240\346\210\220\345\212\237.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\257\255\346\226\231\344\270\212\344\274\240\346\210\220\345\212\237.png" new file mode 100644 index 0000000000000000000000000000000000000000..5c897e9883e868bf5160d92cb106ea4e4e9bc356 Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\257\255\346\226\231\344\270\212\344\274\240\346\210\220\345\212\237.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\257\255\346\226\231\345\210\240\351\231\244\345\244\261\350\264\245\357\274\214\346\234\252\346\237\245\350\257\242\345\210\260\347\233\270\345\205\263\350\257\255\346\226\231.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\257\255\346\226\231\345\210\240\351\231\244\345\244\261\350\264\245\357\274\214\346\234\252\346\237\245\350\257\242\345\210\260\347\233\270\345\205\263\350\257\255\346\226\231.png" new file mode 100644 index 0000000000000000000000000000000000000000..407e49b929b7ff4cf14703046a4ba0bfe1bb441e Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\257\255\346\226\231\345\210\240\351\231\244\345\244\261\350\264\245\357\274\214\346\234\252\346\237\245\350\257\242\345\210\260\347\233\270\345\205\263\350\257\255\346\226\231.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\265\204\344\272\247\344\270\213\346\234\252\346\237\245\350\257\242\345\210\260\350\265\204\344\272\247\345\272\223.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\265\204\344\272\247\344\270\213\346\234\252\346\237\245\350\257\242\345\210\260\350\265\204\344\272\247\345\272\223.png" new file mode 100644 index 0000000000000000000000000000000000000000..45ab521ec5f5afbd81ad54f023aae3b7a867dbf2 Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\265\204\344\272\247\344\270\213\346\234\252\346\237\245\350\257\242\345\210\260\350\265\204\344\272\247\345\272\223.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\265\204\344\272\247\344\270\213\346\237\245\350\257\242\350\265\204\344\272\247\345\272\223\346\210\220\345\212\237.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\265\204\344\272\247\344\270\213\346\237\245\350\257\242\350\265\204\344\272\247\345\272\223\346\210\220\345\212\237.png" new file mode 100644 index 0000000000000000000000000000000000000000..90ed5624ae93ff9784a750514c53293df4e961f0 Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\265\204\344\272\247\344\270\213\346\237\245\350\257\242\350\265\204\344\272\247\345\272\223\346\210\220\345\212\237.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\265\204\344\272\247\345\272\223\345\210\233\345\273\272\346\210\220\345\212\237.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\265\204\344\272\247\345\272\223\345\210\233\345\273\272\346\210\220\345\212\237.png" new file mode 100644 index 0000000000000000000000000000000000000000..7b2cc38a931c9c236517c14c86fa93e3eb2b6dcd Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\265\204\344\272\247\345\272\223\345\210\233\345\273\272\346\210\220\345\212\237.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\265\204\344\272\247\345\272\223\345\210\240\351\231\244\345\244\261\350\264\245\357\274\214\344\270\215\345\255\230\345\234\250\350\265\204\344\272\247.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\265\204\344\272\247\345\272\223\345\210\240\351\231\244\345\244\261\350\264\245\357\274\214\344\270\215\345\255\230\345\234\250\350\265\204\344\272\247.png" new file mode 100644 index 0000000000000000000000000000000000000000..1365a8d69467dec250d3451ac63e2615a2194c18 Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\265\204\344\272\247\345\272\223\345\210\240\351\231\244\345\244\261\350\264\245\357\274\214\344\270\215\345\255\230\345\234\250\350\265\204\344\272\247.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\265\204\344\272\247\345\272\223\345\210\240\351\231\244\346\210\220\345\212\237png.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\265\204\344\272\247\345\272\223\345\210\240\351\231\244\346\210\220\345\212\237png.png" new file mode 100644 index 0000000000000000000000000000000000000000..1bd944264baa9369e6f8fbfd04cabcd12730c0e9 Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\265\204\344\272\247\345\272\223\345\210\240\351\231\244\346\210\220\345\212\237png.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\265\204\344\272\247\345\272\223\346\237\245\350\257\242\345\244\261\350\264\245\357\274\214\344\270\215\345\255\230\345\234\250\350\265\204\344\272\247.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\265\204\344\272\247\345\272\223\346\237\245\350\257\242\345\244\261\350\264\245\357\274\214\344\270\215\345\255\230\345\234\250\350\265\204\344\272\247.png" new file mode 100644 index 0000000000000000000000000000000000000000..58bcd320e145dd29d9e5d49cb6d86964ebb83b51 Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\350\265\204\344\272\247\345\272\223\346\237\245\350\257\242\345\244\261\350\264\245\357\274\214\344\270\215\345\255\230\345\234\250\350\265\204\344\272\247.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\351\205\215\347\275\256\346\230\240\345\260\204\344\270\255\351\227\264\345\261\202.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\351\205\215\347\275\256\346\230\240\345\260\204\344\270\255\351\227\264\345\261\202.png" new file mode 100644 index 0000000000000000000000000000000000000000..809b785b999b6663d9e9bd41fed953925093d6bd Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\351\205\215\347\275\256\346\230\240\345\260\204\344\270\255\351\227\264\345\261\202.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\351\205\215\347\275\256\346\230\240\345\260\204\346\272\220\347\233\256\345\275\225.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\351\205\215\347\275\256\346\230\240\345\260\204\346\272\220\347\233\256\345\275\225.png" new file mode 100644 index 0000000000000000000000000000000000000000..62ba5f6615f18deb3d5a71fd68ee8c929638d814 Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\351\205\215\347\275\256\346\230\240\345\260\204\346\272\220\347\233\256\345\275\225.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\351\205\215\347\275\256\346\230\240\345\260\204\347\233\256\346\240\207\347\233\256\345\275\225.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\351\205\215\347\275\256\346\230\240\345\260\204\347\233\256\346\240\207\347\233\256\345\275\225.png" new file mode 100644 index 0000000000000000000000000000000000000000..d32c672fafcb0ef665bda0bcfdce19d2df44db01 Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\351\205\215\347\275\256\346\230\240\345\260\204\347\233\256\346\240\207\347\233\256\345\275\225.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\351\207\215\345\244\215\345\210\233\345\273\272\350\265\204\344\272\247\345\244\261\350\264\245.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\351\207\215\345\244\215\345\210\233\345\273\272\350\265\204\344\272\247\345\244\261\350\264\245.png" new file mode 100644 index 0000000000000000000000000000000000000000..a5ecd6b65abc97320e7467f00d82ff1fd9bf0e44 Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/local-asset-library-setup/\351\207\215\345\244\215\345\210\233\345\273\272\350\265\204\344\272\247\345\244\261\350\264\245.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/\345\210\233\345\273\272\345\272\224\347\224\250\346\210\220\345\212\237\347\225\214\351\235\242.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/\345\210\233\345\273\272\345\272\224\347\224\250\346\210\220\345\212\237\347\225\214\351\235\242.png" new file mode 100644 index 0000000000000000000000000000000000000000..a871907f348317e43633cf05f5241cb978476fb4 Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/\345\210\233\345\273\272\345\272\224\347\224\250\346\210\220\345\212\237\347\225\214\351\235\242.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/\345\210\233\345\273\272\345\272\224\347\224\250\347\225\214\351\235\242.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/\345\210\233\345\273\272\345\272\224\347\224\250\347\225\214\351\235\242.png" new file mode 100644 index 0000000000000000000000000000000000000000..d82c736a94b106a30fd8d1f7b781f9e335bb441f Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/\345\210\233\345\273\272\345\272\224\347\224\250\347\225\214\351\235\242.png" differ diff --git "a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/\351\203\250\347\275\262\350\247\206\345\233\276.png" "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/\351\203\250\347\275\262\350\247\206\345\233\276.png" new file mode 100644 index 0000000000000000000000000000000000000000..181bf1d2ddbe15cfd296c27df27d865bdbce8d69 Binary files /dev/null and "b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/pictures/\351\203\250\347\275\262\350\247\206\345\233\276.png" differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/ai_container_stack/Compatibility-AI-Infra/flows/get_all_docker_images_flow.yaml b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/ai_container_stack/Compatibility-AI-Infra/flows/get_all_docker_images_flow.yaml new file mode 100644 index 0000000000000000000000000000000000000000..d1c4332203be24d3395d45eee2b1620b18d6f06c --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/ai_container_stack/Compatibility-AI-Infra/flows/get_all_docker_images_flow.yaml @@ -0,0 +1,15 @@ +name: get_all_supported_AI_docker_images +description: "获取所有支持的docker容器镜像,输入为空，输出为支持的AI容器镜像列表，包括名字、tag、registry、repository" +steps: + - name: start + call_type: api + params: + endpoint: GET /docker/images + next: list2markdown + - name: list2markdown + call_type: llm + params: + user_prompt: | + 当前已有的docker容器及tag为：{data}。请将这份内容输出为markdown表格，表头为registry、repository、image_name、tag,请注意如果一个容器镜像有多个tag版本，请分多行展示。 +next_flow: + - docker_pull_specified_AI_docker_images \ No newline at end of file diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/ai_container_stack/Compatibility-AI-Infra/flows/pull_images_flow.yaml b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/ai_container_stack/Compatibility-AI-Infra/flows/pull_images_flow.yaml new file mode 100644 index 0000000000000000000000000000000000000000..277677924f152672e5f0b02305733347900d4e4b --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/ai_container_stack/Compatibility-AI-Infra/flows/pull_images_flow.yaml @@ -0,0 +1,15 @@ +name: docker_pull_specified_AI_docker_images +description: "从dockerhub拉取指定的docker容器镜像，输入为容器镜像的名字和tag" +steps: + - name: start + call_type: api + params: + endpoint: POST /docker/pull + next: extract_key + - name: extract_key + call_type: extract + params: + keys: + - data.shell +next_flow: + - docker_run_specified_AI_docker_images \ No newline at end of file diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/ai_container_stack/Compatibility-AI-Infra/flows/run_images_flow.yaml b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/ai_container_stack/Compatibility-AI-Infra/flows/run_images_flow.yaml new file mode 100644 index 0000000000000000000000000000000000000000..54fe3ca39d9fe16b3c1bbcc506b7cf6f0e673ea9 --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/ai_container_stack/Compatibility-AI-Infra/flows/run_images_flow.yaml @@ -0,0 +1,13 @@ +name: docker_run_specified_AI_docker_images +description: "运行指定的容器镜像，输入为容器镜像的名字和tag" +steps: + - name: start + call_type: api + params: + endpoint: POST /docker/run + next: extract_key + - name: extract_key + call_type: extract + params: + keys: + - data.shell diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/ai_container_stack/Compatibility-AI-Infra/openapi.yaml b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/ai_container_stack/Compatibility-AI-Infra/openapi.yaml new file mode 100644 index 0000000000000000000000000000000000000000..b46bf07f044302169c6c02f4f61be22f2fb5657f --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/ai_container_stack/Compatibility-AI-Infra/openapi.yaml @@ -0,0 +1,190 @@ +openapi: 3.0.2 +info: + title: compatibility-ai-infra + version: 0.1.0 +servers: + - url: http://ai-infra-service.compatibility-ai-infra.svc.cluster.local:8101 +paths: + /docker/images: + get: + description: 获取所有支持的AI容器信息，返回容器名字和tag + responses: + '200': + description: Successful Response + content: + application/json: + schema: + $ref: '#/components/schemas/ResponseData' + /docker/pull: + post: + description: 输入容器镜像名字和容器镜像tag，返回拉取该容器的shell命令 + requestBody: + content: + application/json: + schema: + $ref: '#/components/schemas/PullDockerImages' + required: true + responses: + '200': + description: Successful Response + content: + application/json: + schema: + $ref: '#/components/schemas/ResponseData' + '422': + description: Validation Error + content: + application/json: + schema: + $ref: '#/components/schemas/HTTPValidationError' + /docker/run: + post: + description: 输入容器名字和tag，返回运行该容器的shell命令 + requestBody: + content: + application/json: + schema: + $ref: '#/components/schemas/RunDockerImages' + required: true + responses: + '200': + description: Successful Response + content: + application/json: + schema: + $ref: '#/components/schemas/ResponseData' + '422': + description: Validation Error + content: + application/json: + schema: + $ref: '#/components/schemas/HTTPValidationError' +components: + schemas: + HTTPValidationError: + description: HTTP校验错误 + type: object + properties: + detail: + title: Detail + type: array + items: + $ref: '#/components/schemas/ValidationError' + PullDockerImages: + description: 生成容器拉取命令的接口的入参 + required: + - image_name + - image_tag + type: object + properties: + image_name: + description: 容器镜像的名字,不要包含转义符 + type: string + enum: + - cann + - oneapi-runtime + - oneapi-basekit + - llm-server + - mlflow + - llm + - tensorflow + - pytorch + - cuda + image_tag: + description: 容器镜像的tag，不要包含转义符 + type: string + enum: + - "8.0.RC1-oe2203sp4" + - "cann7.0.RC1.alpha002-oe2203sp2" + - "2024.2.0-oe2403lts" + - "1.0.0-oe2203sp3" + - "2.11.1-oe2203sp3" + - "2.13.1-oe2203sp3" + - "chatglm2_6b-pytorch2.1.0.a1-cann7.0.RC1.alpha002-oe2203sp2" + - "llama2-7b-q8_0-oe2203sp2" + - "chatglm2-6b-q8_0-oe2203sp2" + - "fastchat-pytorch2.1.0.a1-cann7.0.RC1.alpha002-oe2203sp2" + - "tensorflow2.15.0-oe2203sp2" + - "tensorflow2.15.0-cuda12.2.0-devel-cudnn8.9.5.30-oe2203sp2" + - "pytorch2.1.0-oe2203sp2" + - "pytorch2.1.0-cuda12.2.0-devel-cudnn8.9.5.30-oe2203sp2" + - "pytorch2.1.0.a1-cann7.0.RC1.alpha002-oe2203sp2" + - "cuda12.2.0-devel-cudnn8.9.5.30-oe2203sp2" + ResponseData: + description: 接口返回值的固定格式 + required: + - code + - message + - data + type: object + properties: + code: + description: 状态码 + type: integer + message: + description: 状态信息 + type: string + data: + description: 返回数据 + type: any + RunDockerImages: + description: 生成容器运行命令的接口的入参 + required: + - image_name + - image_tag + type: object + properties: + image_name: + description: 容器镜像的名字，不要包含转义符 + type: string + enum: + - cann + - oneapi-runtime + - oneapi-basekit + - llm-server + - mlflow + - llm + - tensorflow + - pytorch + - cuda + image_tag: + description: 容器镜像的tag，不要包含转义符 + type: string + enum: + - "8.0.RC1-oe2203sp4" + - "cann7.0.RC1.alpha002-oe2203sp2" + - "2024.2.0-oe2403lts" + - "1.0.0-oe2203sp3" + - "2.11.1-oe2203sp3" + - "2.13.1-oe2203sp3" + - "chatglm2_6b-pytorch2.1.0.a1-cann7.0.RC1.alpha002-oe2203sp2" + - "llama2-7b-q8_0-oe2203sp2" + - "chatglm2-6b-q8_0-oe2203sp2" + - "fastchat-pytorch2.1.0.a1-cann7.0.RC1.alpha002-oe2203sp2" + - "tensorflow2.15.0-oe2203sp2" + - "tensorflow2.15.0-cuda12.2.0-devel-cudnn8.9.5.30-oe2203sp2" + - "pytorch2.1.0-oe2203sp2" + - "pytorch2.1.0-cuda12.2.0-devel-cudnn8.9.5.30-oe2203sp2" + - "pytorch2.1.0.a1-cann7.0.RC1.alpha002-oe2203sp2" + - "cuda12.2.0-devel-cudnn8.9.5.30-oe2203sp2" + ValidationError: + description: 接口的入参校验错误时返回的内容格式 + required: + - loc + - msg + - type + type: object + properties: + loc: + title: Location + type: array + items: + anyOf: + - type: string + - type: integer + msg: + title: Message + type: string + type: + title: Error Type + type: string \ No newline at end of file diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/ai_container_stack/Compatibility-AI-Infra/plugin.json b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/ai_container_stack/Compatibility-AI-Infra/plugin.json new file mode 100644 index 0000000000000000000000000000000000000000..6136093d2313bd85ae2f2244adef96d48dad90bd --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/ai_container_stack/Compatibility-AI-Infra/plugin.json @@ -0,0 +1,6 @@ +{ + "id": "ai_docker_images", + "name": "AI容器镜像", + "description": "该插件接受用户的输入，检查当前支持哪些AI容器，拉取容器，运行容器", + "predefined_question": "查看当前支持哪些AI容器，拉取指定的容器，运行指定的容器" +} \ No newline at end of file diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/ai_container_stack/plugin-ai-container-stack-deployment-guide.md b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/ai_container_stack/plugin-ai-container-stack-deployment-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..5f4b1e5cb319e53e7d080d9b1257d6d9e101969a --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/ai_container_stack/plugin-ai-container-stack-deployment-guide.md @@ -0,0 +1,35 @@ +# AI容器栈部署指南 + +## 准备工作 + ++ 提前安装 [openEuler Copilot System 命令行（智能 Shell）客户端](../../../user_guide/cli_client/cli-assistant-guide.md) + ++ 修改 /xxxx/xxxx/values.yaml 文件的 `euler-copilot-tune` 部分，将 `enable` 字段改为 `True` + +```yaml +enable: True +``` + ++ 更新环境 + +```bash +helm upgrade euler-copilot . +``` + ++ 检查 Compatibility-AI-Infra 目录下的 openapi.yaml 中 `servers.url` 字段，确保AI容器服务的启动地址被正确设置 + ++ 获取 `$plugin_dir` 插件文件夹的路径，该变量位于 euler-copilot-helm/chart/euler_copilot/values.yaml 中的 `framework` 模块 + ++ 如果插件目录不存在，需新建该目录 + ++ 将该目录下的 Compatibility-AI-Infra 文件夹放到 `$plugin_dir` 中 + +```bash +cp -r ./Compatibility-AI-Infra $PLUGIN_DIR +``` + ++ 重建 framework pod，重载插件配置 + +```bash +kubectl delete pod framework-xxxx -n 命名空间 +``` diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_diagnosis/euler-copilot-rca/flows/demarcation.yaml b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_diagnosis/euler-copilot-rca/flows/demarcation.yaml new file mode 100644 index 0000000000000000000000000000000000000000..6831bdea203e1ffd360f765e5f85ebdce704a437 --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_diagnosis/euler-copilot-rca/flows/demarcation.yaml @@ -0,0 +1,18 @@ +name: demarcation +description: 该工具的作用为针对已知异常事件进行定界分析。需从上下文中获取start_time（开始时间），end_time（结束时间），container_id（容器ID） +steps: + - name: start + call_type: api + params: + endpoint: POST /demarcation + next: report_gen + - name: report_gen + call_type: llm + params: + system_prompt: 你是一个系统智能助手,擅长分析系统的故障现象,最终生成分析报告。 + user_prompt: | + 您是一个专业的运维人员，擅长分析系统的故障现象,最终生成分析报告。当前异常检测结果为{data}。 + 将root_causes_metric_top3内容输出为表格形式，并为每个根因指标进行标号。 + 整个分析报告应该符合markdown规范 +next_flow: + - detection \ No newline at end of file diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_diagnosis/euler-copilot-rca/flows/detection.yaml b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_diagnosis/euler-copilot-rca/flows/detection.yaml new file mode 100644 index 0000000000000000000000000000000000000000..836c71423d63248cd84fe20593d6f848c9b35363 --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_diagnosis/euler-copilot-rca/flows/detection.yaml @@ -0,0 +1,10 @@ +name: detection +description: 该工具的作用为针对已知容器ID和指标，执行profiling分析任务，得到任务ID。需从上下文中获取container_id（容器ID）和三个metric（指标）的其中一个。 +steps: + - name: start + call_type: api + params: + endpoint: POST /detection + next: end + - name: end + call_type: none diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_diagnosis/euler-copilot-rca/flows/inspection.yaml b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_diagnosis/euler-copilot-rca/flows/inspection.yaml new file mode 100644 index 0000000000000000000000000000000000000000..afaefe31106c5ec2016fb3f030fb363950b62516 --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_diagnosis/euler-copilot-rca/flows/inspection.yaml @@ -0,0 +1,16 @@ +name: inspection +description: 该工具的作用为在指定机器上对容器进行异常事件检测。需从上下文中获取start_time（开始时间），end_time（结束时间），machine_id（机器IP） +steps: + - name: start + call_type: api + params: + endpoint: POST /inspection + next: list2markdown + - name: list2markdown + call_type: llm + params: + user_prompt: | + 您是一个专业的运维人员，擅长分析系统的故障现象,最终生成分析报告。当前的异常检测结果为{data}。请将anomaly_events_times_list的信息，输出为表格形式。整个分析报告请符合markdown规范。 + +next_flow: + - demarcation \ No newline at end of file diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_diagnosis/euler-copilot-rca/flows/show_profiling.yaml b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_diagnosis/euler-copilot-rca/flows/show_profiling.yaml new file mode 100644 index 0000000000000000000000000000000000000000..b82172eb272e6c0679dd32582e18e4ecda7dc2bf --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_diagnosis/euler-copilot-rca/flows/show_profiling.yaml @@ -0,0 +1,36 @@ +name: show_profiling +description: 根据已知的智能诊断任务ID（task_id），获取报告的原始数据。随后根据原始数据，生成详细的报告。 +steps: + - name: start + call_type: api + params: + endpoint: POST /show_profiling + next: report_gen + - name: report_gen + call_type: llm + params: + system_prompt: | + 你是一个数据分析和性能分析的专家，请按以下的模板分析出应用的性能瓶颈： + + 1.分析topStackSelf字段中自身耗时排名前3的函数调用栈，分析结果中应该包含函数的耗时信息、函数调用栈的解释说明。 + 2.分析topStackTotal字段中总耗时排名前3的函数调用栈，分析结果中应该包含函数的耗时信息、函数调用栈的解释说明。 + 3.总结前两步的分析结果，并给出影响应用性能的瓶颈所在，同时给出建议。 + user_prompt: | + 现有定界分析结果：{data} + 上面提供了一个JSON对象，它包含了应用程序的Profiling分析报告。该JSON对象包括如下几个字段： + + - traceEvents：它是一个事件列表，列表中的每一项表示一个事件，每个事件以字典格式存储，事件的主要内容解释如下： + - cat 字段：表示事件的分类，它的值包括 syscall、python_gc、sample、pthread_sync，oncpu。其中，syscall 表示这是一个系统调用事件；python_gc 表示这是一个Python垃圾回收事件；sample表示这是一个cpu调用栈采样事件；oncpu表示这是一个OnCPU事件，它说明了pid字段所代表的进程正在占用cpu。 + - name字段：表示事件的名称； + - pid字段：表示事件的进程ID； + - tid字段：表示事件所在的线程ID； + - ts字段：表示事件发生的开始时间，它是一个时间戳格式，单位是微秒； + - dur字段：表示事件的耗时，单位是微秒； + - sf字段：表示事件的函数调用栈，内容是以分号（;）分隔的函数名列表，分号左边是调用方的函数名，分号右边是被调用的函数名。 + - args字段：表示每个事件特有的信息，内容主要包括：count字段，表示事件发生的计数；thread.name字段，表示事件所在的线程的名称；cpu字段，表示采样的cpu编号。 + - topStackSelf：表示应用程序在执行CPU操作期间，自身耗时排名前10的函数调用栈列表。自身耗时是指函数调用栈自身的耗时。列表中的每一项内容说明如下： + - stack：用字符串表示的一个函数调用栈，内容是以分号（;）分隔的函数名列表，分号左边是调用方的函数名，分号右边是被调用的函数名。 + - self_time：stack表示的函数调用栈的自身耗时，单位是毫秒。 + - topStackTotal：表示应用程序在执行CPU操作期间，总耗时排名前10的函数调用栈列表，总耗时是指函数调用栈累积的耗时，它包含了自身耗时。列表中的每一项内容说明如下： + - stack：用字符串表示的一个函数调用栈，内容是以分号（;）分隔的函数名列表，分号左边是调用方的函数名，分号右边是被调用的函数名。 + - total_time：stack表示的函数调用栈的总耗时，单位是毫秒。 \ No newline at end of file diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_diagnosis/euler-copilot-rca/openapi.yaml b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_diagnosis/euler-copilot-rca/openapi.yaml new file mode 100644 index 0000000000000000000000000000000000000000..9ebf2715d5ff61cd86150cfa9b208c2c48a2afa3 --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_diagnosis/euler-copilot-rca/openapi.yaml @@ -0,0 +1,255 @@ +openapi: 3.0.2 +info: + title: 智能诊断 + version: 1.0.0 +servers: + - url: http://192.168.10.31:20030 +paths: + /inspection: + post: + description: 对指定机器进行异常检测，返回异常事件 + requestBody: + content: + application/json: + schema: + $ref: '#/components/schemas/InspectionRequestData' + required: true + responses: + '200': + description: Successful Response + content: + application/json: + schema: {} + '422': + description: Validation Error + content: + application/json: + schema: + $ref: '#/components/schemas/HTTPValidationError' + /demarcation: + post: + description: 对指定容器进行异常定界 + requestBody: + content: + application/json: + schema: + $ref: '#/components/schemas/DemarcationRequestData' + required: true + responses: + '200': + description: Successful Response + content: + application/json: + schema: {} + '422': + description: Validation Error + content: + application/json: + schema: + $ref: '#/components/schemas/HTTPValidationError' + /detection: + post: + description: 根据定界结果指标进行定位 + requestBody: + content: + application/json: + schema: + $ref: '#/components/schemas/DetectionRequestData' + required: true + responses: + '200': + description: Successful Response + content: + application/json: + schema: {} + '422': + description: Validation Error + content: + application/json: + schema: + $ref: '#/components/schemas/HTTPValidationError' + /show_profiling: + post: + description: 根据任务ID，获取Profiling结果 + requestBody: + content: + application/json: + schema: + type: object + description: 请求数据 + required: + - task_id + properties: + task_id: + type: string + description: 任务ID，为UUID类型 + responses: + '200': + description: Successful Response + content: + application/json: + schema: + $ref: "#/components/schemas/ShowProfilingResponse" + '422': + description: Validation Error + content: + application/json: + schema: + $ref: '#/components/schemas/HTTPValidationError' +components: + schemas: + HTTPValidationError: + type: object + description: HTTP 校验错误 + properties: + detail: + type: array + items: + $ref: '#/components/schemas/ValidationError' + title: Detail + InspectionRequestData: + type: object + description: 巡检接口入参 + required: + - machine_id + - start_time + - end_time + properties: + machine_id: + description: 机器IP。如果给定的信息没有指定任何机器IP，则默认为“default_0.0.0.0”。 + type: string + title: Machine_ID + default: default_0.0.0.0 + start_time: + description: 根据给定的信息提取出开始时间，如果给定的信息不包含开始时间，开始时间可以设置为当前时间往前推2分钟，最终解析出的时间以'%Y-%m-%d %H:%M:%S'格式输出 + type: string + title: Start_Time + default: '' + end_time: + description: 根据给定的信息提取出结束时间，如果给定的信息不包含结束时间，结束时间可以设置为当前时间，最终解析出的时间以'%Y-%m-%d %H:%M:%S'格式输出 + type: string + title: End_Time + default: '' + DemarcationRequestData: + type: object + description: 定界接口入参 + required: + - start_time + - end_time + - container_id + properties: + start_time: + description: 根据给定的信息提取出开始时间，如果给定的信息不包含开始时间，开始时间可以设置为当前时间往前推2分钟，最终解析出的时间以'%Y-%m-%d %H:%M:%S'格式输出 + type: string + title: Start_Time + default: '' + end_time: + description: 根据给定的信息提取出结束时间，如果给定的信息不包含结束时间，结束时间可以设置为当前时间，最终解析出的时间以'%Y-%m-%d %H:%M:%S'格式输出 + type: string + title: End_Time + default: '' + container_id: + description: 结合问题中指定的具体异常事件，根据给定信息提取容器ID + type: string + title: Container_ID + default: '' + DetectionRequestData: + type: object + description: 定位接口入参 + required: + - container_id + - metric + properties: + container_id: + description: 结合问题中指定的具体指标或者指标号，根据给定信息提取容器ID + type: string + title: Container_ID + default: '' + metric: + description: 结合问题中的具体指标或者指标号，根据给定信息提取具体指标值作为metric + type: string + title: Metric + default: '' + ShowProfilingResponse: + type: object + description: show profiling 的返回结果 + properties: + traceEvents: + type: array + items: + type: object + properties: + cat: + type: string + description: Event category (syscall, python_gc, sample, pthread_sync, oncpu) + name: + type: string + description: Event name + pid: + type: integer + format: int32 + description: Process ID + tid: + type: integer + format: int32 + description: Thread ID + ts: + type: integer + format: int64 + description: Timestamp of the event start in microseconds + dur: + type: integer + format: int32 + description: Duration of the event in microseconds + sf: + type: string + description: Call stack represented as a list of function names separated by semicolons + args: + type: object + additionalProperties: true + description: Additional event-specific information such as count, thread.name, and cpu + topStackSelf: + type: array + items: + type: object + properties: + stack: + type: string + description: Call stack represented as a list of function names separated by semicolons + self_time: + type: number + format: int + description: Exclusive time spent in the call stack in milliseconds + topStackTotal: + type: array + items: + type: object + properties: + stack: + type: string + description: Call stack represented as a list of function names separated by semicolons + total_time: + type: number + format: int + description: Total inclusive time spent in the call stack in milliseconds + ValidationError: + type: object + required: + - loc + - msg + - type + title: ValidationError + properties: + loc: + type: array + items: + anyOf: + - type: string + - type: integer + title: Location + msg: + type: string + title: Message + type: + type: string + title: Error Type \ No newline at end of file diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_diagnosis/euler-copilot-rca/plugin.json b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_diagnosis/euler-copilot-rca/plugin.json new file mode 100644 index 0000000000000000000000000000000000000000..b0ef2fd7aa0c13ad626a01d0fc7a4bf010ab3178 --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_diagnosis/euler-copilot-rca/plugin.json @@ -0,0 +1,5 @@ +{ + "id": "rca", + "name": "智能诊断", + "description": "该插件具备以下功能：巡检，定界，定位" +} \ No newline at end of file diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_diagnosis/plugin-intelligent-diagnosis-deployment-guide.md b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_diagnosis/plugin-intelligent-diagnosis-deployment-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..20b7ff3b322789c8bff6db6c537c21a7b98b24de --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_diagnosis/plugin-intelligent-diagnosis-deployment-guide.md @@ -0,0 +1,189 @@ +# 智能诊断部署指南 + +## 准备工作 + ++ 提前安装 [openEuler Copilot System 命令行（智能 Shell）客户端](../../../user_guide/cli_client/cli-assistant-guide.md) + ++ 被诊断机器不能安装 crictl 和 isula，只能有 docker 一个容器管理工具 + ++ 在需要被诊断的机器上安装 gala-gopher 和 gala-anteater + +## 部署 gala-gopher + +### 1. 准备 BTF 文件 + +**如果Linux内核支持 BTF，则不需要准备 BTF 文件。**可以通过以下命令来查看Linux内核是否已经支持 BTF： + +```bash +cat /boot/config-$(uname -r) | grep CONFIG_DEBUG_INFO_BTF +``` + +如果输出结果为`CONFIG_DEBUG_INFO_BTF=y`，则表示内核支持BTF。否则表示内核不支持BTF。 +如果内核不支持BTF，需要手动制作BTF文件。步骤如下： + +1. 获取当前Linux内核版本的 vmlinux 文件 + + vmlinux 文件存放在 `kernel-debuginfo` 包里面，存放路径为 `/usr/lib/debug/lib/modules/$(uname -r)/vmlinux`。 + + 例如，对于 `kernel-debuginfo-5.10.0-136.65.0.145.oe2203sp1.aarch64`，对应的vmlinux路径为`/usr/lib/debug/lib/modules/5.10.0-136.65.0.145.oe2203sp1.aarch64/vmlinux`。 + +2. 制作 BTF 文件 + + 基于获取到 vmlinux 文件来制作 BTF 文件。这一步可以在自己的环境里操作。首先，需要安装相关的依赖包： + + ```bash + # 说明：dwarves 包中包含 pahole 命令，llvm 包中包含 llvm-objcopy 命令 + yum install -y llvm dwarves + ``` + + 执行下面的命令行，生成 BTF 文件。 + + ```bash + kernel_version=4.19.90-2112.8.0.0131.oe1.aarch64 # 说明：这里需要替换成目标内核版本，可通过 uname -r 命令获取 + pahole -J vmlinux + llvm-objcopy --only-section=.BTF --set-section-flags .BTF=alloc,readonly --strip-all vmlinux ${kernel_version}.btf + strip -x ${kernel_version}.btf + ``` + + 生成的 BTF 文件名称为`.btf`格式，其中 ``为目标机器的内核版本，可通过 `uname -r` 命令获取。 + +### 2. 下载 gala-gopher 容器镜像 + +#### 在线下载 + +gala-gopher 容器镜像已归档到 仓库中，可通过如下命令获取。 + +```bash +# 获取 aarch64 架构的镜像 +docker pull hub.oepkgs.net/a-ops/gala-gopher-profiling-aarch64:latest +# 获取 x86_64 架构的镜像 +docker pull hub.oepkgs.net/a-ops/gala-gopher-profiling-x86_64:latest +``` + +#### 离线下载 + +若无法通过在线下载的方式下载容器镜像，可联系我（何秀军 00465007）获取压缩包。 + +拿到压缩包后，放到目标机器上，解压并加载容器镜像，命令行如下： + +```bash +tar -zxvf gala-gopher-profiling-aarch64.tar.gz +docker load < gala-gopher-profiling-aarch64.tar +``` + +### 3. 启动 gala-gopher 容器 + +容器启动命令： + +```shell +docker run -d --name gala-gopher-profiling --privileged --pid=host --network=host -v /:/host -v /etc/localtime:/etc/localtime:ro -v /sys:/sys -v /usr/lib/debug:/usr/lib/debug -v /var/lib/docker:/var/lib/docker -v /tmp/$(uname -r).btf:/opt/gala-gopher/btf/$(uname -r).btf -e GOPHER_HOST_PATH=/host gala-gopher-profiling-aarch64:latest +``` + +启动配置参数说明： + ++ `-v /tmp/$(uname -r).btf:/opt/gala-gopher/btf/$(uname -r).btf` ：如果内核支持 BTF，则删除该配置即可。如果内核不支持 BTF，则需要将前面准备好的 BTF 文件拷贝到目标机器上，并将 `/tmp/$(uname -r).btf` 替换为对应的路径。 ++ `gala-gopher-profiling-aarch64-0426` ：gala-gopher容器对应的tag，替换成实际下载的tag。 + +探针启动： + ++ `container_id` 为需要观测的容器 id ++ 分别启动 sli 和 container 探针 + +```bash +curl -X PUT http://localhost:9999/sli -d json='{"cmd":{"check_cmd":""},"snoopers":{"container_id":[""]},"params":{"report_period":5},"state":"running"}' +``` + +```bash +curl -X PUT http://localhost:9999/container -d json='{"cmd":{"check_cmd":""},"snoopers":{"container_id":[""]},"params":{"report_period":5},"state":"running"}' +``` + +探针关闭 + +```bash +curl -X PUT http://localhost:9999/sli -d json='{"state": "stopped"}' +``` + +```bash +curl -X PUT http://localhost:9999/container -d json='{"state": "stopped"}' +``` + +## 部署 gala-anteater + +源码部署： + +```bash +# 请指定分支为 930eulercopilot +git clone https://gitee.com/GS-Stephen_Curry/gala-anteater.git +``` + +安装部署请参考 +(请留意python版本导致执行setup.sh install报错) + +镜像部署： + +```bash +docker pull hub.oepkgs.net/a-ops/gala-anteater:2.0.2 +``` + +`/etc/gala-anteater/config/gala-anteater.yaml` 中 Kafka 和 Prometheus 的 `server` 和 `port` 需要按照实际部署修改，`model_topic`、`meta_topic`、`group_id` 自定义 + +```yaml +Kafka: + server: "xxxx" + port: "xxxx" + model_topic: "xxxx" # 自定义，与rca配置中保持一致 + meta_topic: "xxxx" # 自定义，与rca配置中保持一致 + group_id: "xxxx" # 自定义，与rca配置中保持一致 + # auth_type: plaintext/sasl_plaintext, please set "" for no auth + auth_type: "" + username: "" + password: "" + +Prometheus: + server: "xxxx" + port: "xxxx" + steps: "5" +``` + +gala-anteater 中模型的训练依赖于 gala-gopher 采集的数据，因此请保证 gala-gopher 探针正常运行至少24小时，在运行 gala-anteater。 + +## 部署 gala-ops + +每个中间件的大致介绍： + +kafka ： 一个数据库中间件， 分布式数据分流作用， 可以配置为当前的管理节点。 + +prometheus：性能监控， 配置需要监控的生产节点 ip list。 + +直接通过yum install安装kafka和prometheus，可参照安装脚本 + +只需要参照其中 kafka 和 prometheus 的安装即可 + +## 部署 euler-copilot-rca + +镜像拉取 + +```bash +docker pull hub.oepkgs.net/a-ops/euler-copilot-rca:0.9.1 +``` + ++ 修改 `config/config.json` 文件，配置 gala-gopher 镜像的 `container_id` 以及 `ip`，Kafka 和 Prometheus 的 `ip` 和 `port`（与上述 gala-anteater 配置保持一致） + +```yaml +"gopher_container_id": "xxxx", # gala-gopher的容器id + "remote_host": "xxxx" # gala-gopher的部署机器ip + }, + "kafka": { + "server": "xxxx", + "port": "xxxx", + "storage_topic": "usad_intermediate_results", + "anteater_result_topic": "xxxx", + "rca_result_topic": "xxxx", + "meta_topic": "xxxx" + }, + "prometheus": { + "server": "xxxx", + "port": "xxxx", + "steps": 5 + }, +``` diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_tuning/euler-copilot-tune/flows/data_collection.yaml b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_tuning/euler-copilot-tune/flows/data_collection.yaml new file mode 100644 index 0000000000000000000000000000000000000000..d2718f0dd059f3a8a34d02cbc67436c6fc274a28 --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_tuning/euler-copilot-tune/flows/data_collection.yaml @@ -0,0 +1,15 @@ +name: data_collection +description: 采集某一指定ip主机的系统性能指标 +steps: + - name: start + call_type: api + params: + endpoint: POST /performance_metric + next: show_data + - name: show_data + call_type: llm + params: + user_prompt: | + 当前采集到系统性能指标为：{data}, 输出内容请符合markdown规范。 +next_flow: + - performance_analysis \ No newline at end of file diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_tuning/euler-copilot-tune/flows/performance_analysis.yaml b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_tuning/euler-copilot-tune/flows/performance_analysis.yaml new file mode 100644 index 0000000000000000000000000000000000000000..07e2a2ada9c54568be3f3bf13c5b2223e615037a --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_tuning/euler-copilot-tune/flows/performance_analysis.yaml @@ -0,0 +1,15 @@ +name: performance_analysis +description: 分析性能指标并生成性能分析报告 +steps: + - name: start + call_type: api + params: + endpoint: POST /performance_report + next: extract_key + - name: extract_key + call_type: extract + params: + keys: + - data.output +next_flow: + - performance_tuning \ No newline at end of file diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_tuning/euler-copilot-tune/flows/performance_tuning.yaml b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_tuning/euler-copilot-tune/flows/performance_tuning.yaml new file mode 100644 index 0000000000000000000000000000000000000000..e938a0bf1bd83f971c4eaaff2d447a150fcf5560 --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_tuning/euler-copilot-tune/flows/performance_tuning.yaml @@ -0,0 +1,13 @@ +name: performance_tuning +description: 基于性能能分析报告，生成操作系统和Mysql应用的性能优化建议,结果以shell脚本的形式返回 +steps: + - name: start + call_type: api + params: + endpoint: POST /optimization_suggestion + next: extract_key + - name: extract_key + call_type: extract + params: + keys: + - data.script \ No newline at end of file diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_tuning/euler-copilot-tune/openapi.yaml b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_tuning/euler-copilot-tune/openapi.yaml new file mode 100644 index 0000000000000000000000000000000000000000..18ede5a988fdc06c9de09ff0f2b7077554bedbff --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_tuning/euler-copilot-tune/openapi.yaml @@ -0,0 +1,147 @@ +openapi: 3.0.2 +info: + title: 智能诊断 + version: 1.0.0 +servers: + - url: http://euler-copilot-tune.euler-copilot.svc.cluster.local:8100 +paths: + /performance_metric: + post: + description: 对指定机器进行性能指标采集，返回指标值 + requestBody: + content: + application/json: + schema: + $ref: '#/components/schemas/PerformanceMetricRequestData' + required: true + responses: + '200': + description: Successful Response + content: + application/json: + schema: {} + '422': + description: Validation Error + content: + application/json: + schema: + $ref: '#/components/schemas/HTTPValidationError' + /performance_report: + post: + description: 基于采集到的指标，对指定机器进行性能诊断，生成性能分析报告 + requestBody: + content: + application/json: + schema: + $ref: '#/components/schemas/PerformanceReportRequestData' + required: true + responses: + '200': + description: Successful Response + content: + application/json: + schema: {} + '422': + description: Validation Error + content: + application/json: + schema: + $ref: '#/components/schemas/HTTPValidationError' + /optimization_suggestion: + post: + description: 根据性能分析报告，以及指定的机器应用信息，生成调优建议 + requestBody: + content: + application/json: + schema: + $ref: '#/components/schemas/OptimizationSuggestionRequestData' + required: true + responses: + '200': + description: Successful Response + content: + application/json: + schema: {} + '422': + description: Validation Error + content: + application/json: + schema: + $ref: '#/components/schemas/HTTPValidationError' +components: + schemas: + HTTPValidationError: + type: object + description: HTTP 校验错误 + properties: + detail: + type: array + items: + $ref: '#/components/schemas/ValidationError' + OptimizationSuggestionRequestData: + type: object + description: 生成优化建议的接口的入参 + required: + - app + - ip + properties: + app: + type: string + description: 应用名称 + default: mysql + enum: + - mysql + - none + ip: + type: string + description: 点分十进制的ipv4地址, 例如192.168.10.43 + example: "192.168.10.43" + PerformanceMetricRequestData: + type: object + description: 性能指标采集的接口的入参 + required: + - app + - ip + properties: + ip: + type: string + description: 点分十进制的ipv4地址, 例如192.168.10.43 + example: "192.168.10.43" + app: + type: string + description: App + default: none + enum: + - mysql + - none + PerformanceReportRequestData: + type: object + description: 生成性能报告接口的入参 + required: + - ip + properties: + ip: + type: string + description: 点分十进制的ipv4地址, 例如192.168.10.43 + example: "192.168.10.43" + ValidationError: + type: object + required: + - loc + - msg + - type + title: ValidationError + properties: + loc: + type: array + items: + anyOf: + - type: string + - type: integer + title: Location + msg: + type: string + title: Message + type: + type: string + title: Error Type \ No newline at end of file diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_tuning/euler-copilot-tune/plugin.json b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_tuning/euler-copilot-tune/plugin.json new file mode 100644 index 0000000000000000000000000000000000000000..c4b95f57e6169a93dcaf7c08e2d328f5be6bf893 --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_tuning/euler-copilot-tune/plugin.json @@ -0,0 +1,6 @@ +{ + "id": "tune", + "name": "智能性能优化", + "description": "该插件具备以下功能：采集系统性能指标，分析系统性能，推荐系统性能优化建议", + "automatic_flow": false +} \ No newline at end of file diff --git a/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_tuning/plugin-intelligent-tuning-deployment-guide.md b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_tuning/plugin-intelligent-tuning-deployment-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..aaf336d1e2fbedcbe5ace5a48c69fa8113a76371 --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/deployment_guide/plugin_deployment_guide/intelligent_tuning/plugin-intelligent-tuning-deployment-guide.md @@ -0,0 +1,131 @@ +# 智能调优部署指南 + +## 准备工作 + ++ 提前安装 [openEuler Copilot System 命令行（智能 Shell）客户端](../../../user_guide/cli_client/cli-assistant-guide.md) + ++ 被调优机器需要为 openEuler 25.03 + ++ 在需要被调优的机器上安装依赖 + +```bash +yum install -y sysstat perf +``` + ++ 被调优机器需要开启 SSH 22端口 + +## 编辑配置文件 + +修改values.yaml文件的tune部分，将 `enable` 字段改为 `True` ，并配置大模型设置、 +Embedding模型文件地址、以及需要调优的机器和对应机器上的 mysql 的账号名以及密码 + +```bash +vim /home/euler-copilot-framework/euler-copilot-helm/chart/agents/values.yaml +``` + +```yaml +tune: + # 【必填】是否启用智能调优Agent + enabled: true + # 镜像设置 + image: + # 镜像仓库。留空则使用全局设置。 + registry: "" + # 【必填】镜像名称 + name: euler-copilot-tune + # 【必填】镜像标签 + tag: "0.9.1" + # 拉取策略。留空则使用全局设置。 + imagePullPolicy: "" + # 【必填】容器根目录只读 + readOnly: false + # 性能限制设置 + resources: {} + # Service设置 + service: + # 【必填】Service类型，ClusterIP或NodePort + type: ClusterIP + nodePort: + # 大模型设置 + llm: + # 【必填】模型地址（需要包含v1后缀） + url: + # 【必填】模型名称 + name: "" + # 【必填】模型API Key + key: "" + # 【必填】模型最大Token数 + max_tokens: 8096 + # 【必填】Embedding模型文件地址 + embedding: "" + # 待优化机器信息 + machine: + # 【必填】IP地址 + ip: "" + # 【必填】Root用户密码 + # 注意：必需启用Root用户以密码形式SSH登录 + password: "" + # 待优化应用设置 + mysql: + # 【必填】数据库用户名 + user: "root" + # 【必填】数据库密码 + password: "" +``` + +## 安装智能调优插件 + +```bash +helm install -n euler-copilot agents . +``` + +如果之前有执行过安装，则按下面指令更新插件服务 + +```bash +helm upgrade-n euler-copilot agents . +``` + +如果 framework未重启，则需要重启framework配置 + +```bash +kubectl delete pod framework-deploy-service-bb5b58678-jxzqr -n eulercopilot +``` + +## 测试 + ++ 查看 tune 的 pod 状态 + + ```bash + NAME READY STATUS RESTARTS AGE + authhub-backend-deploy-authhub-64896f5cdc-m497f 2/2 Running 0 16d + authhub-web-deploy-authhub-7c48695966-h8d2p 1/1 Running 0 17d + pgsql-deploy-databases-86b4dc4899-ppltc 1/1 Running 0 17d + redis-deploy-databases-f8866b56-kj9jz 1/1 Running 0 17d + mysql-deploy-databases-57f5f94ccf-sbhzp 2/2 Running 0 17d + framework-deploy-service-bb5b58678-jxzqr 2/2 Running 0 16d + rag-deploy-service-5b7887644c-sm58z 2/2 Running 0 110m + vectorize-deploy-service-57f5f94ccf-sbhzp 2/2 Running 0 17d + web-deploy-service-74fbf7999f-r46rg 1/1 Running 0 2d + tune-deploy-agents-5d46bfdbd4-xph7b 1/1 Running 0 2d + ``` + ++ pod启动失败排查办法 + + 检查 euler-copilot-tune 目录下的 openapi.yaml 中 `servers.url` 字段，确保调优服务的启动地址被正确设置 + + 检查 `$plugin_dir` 插件文件夹的路径是否配置正确，该变量位于 `euler-copilot-helm/chart/euler_copilot/values.yaml` 中的 `framework`模块，如果插件目录不存在，需新建该目录，并需要将该目录下的 euler-copilot-tune 文件夹放到 `$plugin_dir` 中。 + + 检查sglang的地址和key填写是否正确，该变量位于 `vim /home/euler-copilot-framework/euler-copilot-helm/chart/euler_copilot/values.yaml` + + ```yaml + # 用于Function Call的模型 + scheduler: + # 推理框架类型 + backend: sglang + # 模型地址 + url: "" + # 模型 API Key + key: "" + # 数据库设置 + ``` + ++ 命令行客户端使用智能调优 + + 具体使用可参考 [openEuler Copilot System 命令行（智能插件：智能调优）](../../../user_guide/cli_client/intelligent-tuning.md) diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/cli-assistant-guide.md b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/cli-assistant-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..d965d51e293c8304710c0469f1da4b605db7b32e --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/cli-assistant-guide.md @@ -0,0 +1,169 @@ +# 命令行助手使用指南 + +## 简介 + +openEuler Copilot System 命令行助手是一个命令行（Shell）AI 助手，您可以通过它来快速生成 Shell 命令并执行，从而提高您的工作效率。除此之外，基于 Gitee AI 在线服务的标准版本还内置了 openEuler 的相关知识，可以助力您学习与使用 openEuler 操作系统。 + +## 环境要求 + +- 操作系统：openEuler 22.03 LTS SP3，或者 openEuler 24.03 LTS 及以上版本 +- 命令行软件： + - Linux 桌面环境：支持 GNOME、KDE、DDE 等桌面环境的内置终端 + - 远程 SSH 链接：支持兼容 xterm-256 与 UTF-8 字符集的终端 + +## 安装 + +openEuler Copilot System 命令行助手支持通过 OEPKGS 仓库进行安装。 + +### 配置 OEPKGS 仓库 + +```bash +sudo dnf config-manager --add-repo https://repo.oepkgs.net/openeuler/rpm/`sed 's/release //;s/[()]//g;s/ /-/g' /etc/openEuler-release`/extras/`uname -m` +``` + +```bash +sudo dnf clean all +``` + +```bash +sudo dnf makecache +``` + +### 安装命令行助手 + +```bash +sudo dnf install eulercopilot-cli +``` + +若遇到 `Error: GPG check FAILED` 错误，使用 `--nogpgcheck` 跳过检查。 + +```bash +sudo dnf install --nogpgcheck eulercopilot-cli +``` + +## 初始化 + +```bash +copilot --init +``` + +然后根据提示输入 API Key 完成配置。 + +![shell-init](./pictures/shell-init.png) + +初次使用前请先退出终端或重新连接 SSH 会话使配置生效。 + +- **查看助手帮助页面** + + ```bash + copilot --help + ``` + + ![shell-help](./pictures/shell-help.png) + +## 使用 + +在终端中输入问题，按下 `Ctrl + O` 提问。 + +### 快捷键 + +- 输入自然语言问题后，按下 `Ctrl + O` 可以直接向 AI 提问。 +- 直接按下 `Ctrl + O` 可以自动填充命令前缀 `copilot`，输入参数后按下 `Enter` 即可执行。 + +### 智能问答 + +命令行助手初始化完成后，默认处于智能问答模式。 +命令提示符**左上角**会显示当前模式。 +若当前模式不是“智能问答”，执行 `copilot -c` (`copilot --chat`) 切换到智能问答模式。 + +![chat-ask](./pictures/shell-chat-ask.png) + +AI 回答完毕后，会根据历史问答生成推荐问题，您可以复制、粘贴到命令行中进行追问。输入追问的问题后，按下 `Enter` 提问。 + +![chat-next](./pictures/shell-chat-continue.png) + +![chat-next-result](./pictures/shell-chat-continue-result.png) + +智能问答模式下支持连续追问，每次追问最多可以关联3条历史问答的上下文。 + +输入 `exit` 可以退出智能问答模式，回到 Linux 命令行。 + +![chat-exit](./pictures/shell-chat-exit.png) + +- 若问答过程中遇到程序错误，可以按下 `Ctrl + C` 立即退出当前问答，再尝试重新提问。 + +### Shell 命令 + +AI 会根据您的问题返回 Shell 命令，openEuler Copilot System 命令行助手可以解释、编辑或执行这些命令，并显示命令执行结果。 + +![shell-cmd](./pictures/shell-cmd.png) + +命令行助手会自动提取 AI 回答中的命令，并显示相关操作。您可以通过键盘上下键选择操作，按下 `Enter` 确认。 + +![shell-cmd-interact](./pictures/shell-cmd-interact.png) + +#### 解释 + +如果 AI 仅返回了一条命令，选择解释后会直接请求 AI 解释命令，并显示回答。 +若 AI 回答了多条命令，选择后会显示命令列表，您每次可以选择**一条**请求 AI 解释。 + +![shell-cmd-explain-select](./pictures/shell-cmd-explain-select.png) + +完成解释后，您可以继续选择其他操作。 + +![shell-cmd-explain-result](./pictures/shell-cmd-explain-result.png) + +#### 编辑 + +![shell-cmd-edit](./pictures/shell-cmd-edit.png) + +选择一条命令进行编辑，编辑完成后按下 `Enter` 确认。 + +![shell-cmd-edit-result](./pictures/shell-cmd-edit-result.png) + +完成编辑后，您可以继续编辑其他命令或选择其他操作。 + +#### 执行 + +如果 AI 仅返回了一条命令，选择执行后会直接执行命令，并显示执行结果。 +若 AI 回答了多条命令，选择后会显示命令列表，您每次可以选择**多条**命令来执行。 + +您可以通过键盘上下键移动光标，按下 `空格键` 选择命令，按下 `Enter` 执行所选命令。 +被选中的命令会显示**蓝色高亮**，如图所示。 + +![shell-cmd-exec-multi-select](./pictures/shell-cmd-exec-multi-select.png) + +若不选择任何命令，直接按下 `Enter`，则会跳过执行命令，直接进入下一轮问答。 + +按下 `Enter` 后，被选中的命令会从上到下依次执行。 + +![shell-cmd-exec-result](./pictures/shell-cmd-exec-result.png) + +若执行过程中遇到错误，命令行助手会显示错误信息，并**终止执行命令**，进入下一轮问答。 +您可以在下一轮问答中提示 AI 更正命令，或要求 AI 重新生成命令。 + +### 智能插件 + +在 Linux 命令行中执行 `copilot -p` (`copilot --plugin`) 切换到智能插件模式。 + +![shell-plugin](./pictures/shell-plugin.png) + +输入问题并按下 `Ctrl + O` 提问后，从列表中选择插件，按下 `Enter` 调用插件回答问题。 + +![shell-plugin-select](./pictures/shell-plugin-select.png) + +![shell-plugin-result](./pictures/shell-plugin-result.png) + +## 卸载 + +```bash +sudo dnf remove eulercopilot-cli +``` + +然后使用以下命令删除配置文件。 + +```bash +rm ~/.config/eulercopilot/config.json +``` + +卸载完成后请重启终端或重新连接 SSH 会话使配置还原。 diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/intelligent-diagnosis.md b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/intelligent-diagnosis.md new file mode 100644 index 0000000000000000000000000000000000000000..eb999cb5483620450b2e2aea77a818382aeca2a4 --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/intelligent-diagnosis.md @@ -0,0 +1,50 @@ +# 智能插件：智能诊断 + +部署智能诊断工具后，可以通过 openEuler Copilot System 智能体框架实现对本机进行诊断。 +在智能诊断模式提问，智能体框架服务可以调用本机的诊断工具诊断异常状况、分析并生成报告。 + +## 操作步骤 + +**步骤1** 切换到“智能插件”模式 + +```bash +copilot -p +``` + +![切换到智能插件模式](./pictures/shell-plugin-diagnose-switch-mode.png) + +**步骤2** 异常事件检测 + +```bash +帮我进行异常事件检测 +``` + +按下 `Ctrl + O` 键提问，然后在插件列表中选择“智能诊断”。 + +![异常事件检测](./pictures/shell-plugin-diagnose-detect.png) + +**步骤3** 查看异常事件详情 + +```bash +查看 XXX 容器的异常事件详情 +``` + +![查看异常事件详情](./pictures/shell-plugin-diagnose-detail.png) + +**步骤4** 执行异常事件分析 + +```bash +请对 XXX 容器的 XXX 指标执行 profiling 分析 +``` + +![异常事件分析](./pictures/shell-plugin-diagnose-profiling.png) + +**步骤5** 查看异常事件分析报告 + +等待 5 至 10 分钟，然后查看分析报告。 + +```bash +查看 对应的 profiling 报告 +``` + +![执行优化脚本](./pictures/shell-plugin-diagnose-report.png) diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/intelligent-tuning.md b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/intelligent-tuning.md new file mode 100644 index 0000000000000000000000000000000000000000..b5c40581668ae4f6074043e62a93b2c4b240e5b3 --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/intelligent-tuning.md @@ -0,0 +1,53 @@ +# 智能插件：智能调优 + +部署智能调优工具后，可以通过 openEuler Copilot System 智能体框架实现对本机进行调优。 +在智能调优模式提问，智能体框架服务可以调用本机的调优工具采集性能指标，并生成性能分析报告和性能优化建议。 + +## 操作步骤 + +**步骤1** 切换到“智能调优”模式 + +```bash +copilot -t +``` + +![切换到智能调优模式](./pictures/shell-plugin-tuning-switch-mode.png) + +**步骤2** 采集性能指标 + +```bash +帮我进行性能指标采集 +``` + +![性能指标采集](./pictures/shell-plugin-tuning-metrics-collect.png) + +**步骤3** 生成性能分析报告 + +```bash +帮我生成性能分析报告 +``` + +![性能分析报告](./pictures/shell-plugin-tuning-report.png) + +**步骤4** 生成性能优化建议 + +```bash +请生成性能优化脚本 +``` + +![性能优化脚本](./pictures/shell-plugin-tuning-script-gen.png) + +**步骤5** 选择“执行命令”，运行优化脚本 + +![执行优化脚本](./pictures/shell-plugin-tuning-script-exec.png) + +- 脚本内容如图： + ![优化脚本内容](./pictures/shell-plugin-tuning-script-view.png) + +## 远程调优 + +如果需要对其他机器进行远程调优，请在上文示例的问题前面加上对应机器的 IP 地址。 + +例如：`请对 192.168.1.100 这台机器进行性能指标采集。` + +进行远程调优前请确保目标机器已部署智能调优工具，同时请确保 openEuler Copilot System 智能体框架能够访问目标机器。 diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/obtaining-the-api-key.md b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/obtaining-the-api-key.md new file mode 100644 index 0000000000000000000000000000000000000000..01381a772743299de24d58a7a94bf0a180f77d29 --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/obtaining-the-api-key.md @@ -0,0 +1,28 @@ +# 获取 API Key + +## 前言 + +openEuler Copilot System 命令行助手使用 API Key 来验证用户身份，并获取 API 访问权限。 +因此，开始使用前，您需要先获取 API Key。 + +## 注意事项 + +- 请妥善保管您的 API Key，不要泄露给他人。 +- API Key 仅用于命令行助手与 DevStation 桌面端，不用于其他用途。 +- 每位用户仅可拥有一个 API Key，重复创建 API Key 将导致旧密钥失效。 +- API Key 仅在创建时显示一次，请务必及时保存。若密钥丢失，您需要重新创建。 +- 若您在使用过程中遇到“请求过于频繁”的错误，您的 API Key 可能已被他人使用，请及时前往官网刷新或撤销 API Key。 + +## 获取方法 + +1. 登录 [openEuler Copilot System (Gitee AI) 官网](https://eulercopilot.gitee.com)。 +2. 点击右上角头像，选择“API KEY”。 +3. 点击“新建”按钮。 +4. **请立即保存 API Key，它仅在创建时显示一次，请勿泄露给他人。** + +## 管理 API Key + +1. 登录 [openEuler Copilot System (Gitee AI) 官网](https://eulercopilot.gitee.com)。 +2. 点击右上角头像，选择“API KEY”。 +3. 点击“刷新”按钮，刷新 API Key；点击“撤销”按钮，撤销 API Key。 + - 刷新 API Key 后，旧密钥失效，请立即保存新生成的 API Key。 diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-chat-ask.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-chat-ask.png new file mode 100644 index 0000000000000000000000000000000000000000..00d5cf5ecf894dd62366ec086bf96eae532f0b5d Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-chat-ask.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-chat-continue-result.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-chat-continue-result.png new file mode 100644 index 0000000000000000000000000000000000000000..f30f9fe7a015e775742bc184b8ac75790dc482fa Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-chat-continue-result.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-chat-continue.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-chat-continue.png new file mode 100644 index 0000000000000000000000000000000000000000..7e4801504fd53fab989574416e6220c4fa3f1d38 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-chat-continue.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-chat-exit.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-chat-exit.png new file mode 100644 index 0000000000000000000000000000000000000000..0bb81190a3039f6c5a311b365376ec230c1ad4b5 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-chat-exit.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd-edit-result.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd-edit-result.png new file mode 100644 index 0000000000000000000000000000000000000000..c5e6f8245e7d66cdbe5370f18d15a791a33a517a Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd-edit-result.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd-edit.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd-edit.png new file mode 100644 index 0000000000000000000000000000000000000000..bb6209373a6d2a1881728bee352e7c3b46cc91d7 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd-edit.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd-exec-multi-select.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd-exec-multi-select.png new file mode 100644 index 0000000000000000000000000000000000000000..2dda108a39af54fc15a4ff8c0dca107de38b9cf0 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd-exec-multi-select.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd-exec-result.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd-exec-result.png new file mode 100644 index 0000000000000000000000000000000000000000..f4fff6a62b8b4220b52fdf55b133f2ba37850569 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd-exec-result.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd-explain-result.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd-explain-result.png new file mode 100644 index 0000000000000000000000000000000000000000..707dd36aa7c7eadae4f29254cf5fc18ce877f597 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd-explain-result.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd-explain-select.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd-explain-select.png new file mode 100644 index 0000000000000000000000000000000000000000..bf58b69e241ea11a6945f21e3fc69d22a401be2e Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd-explain-select.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd-interact.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd-interact.png new file mode 100644 index 0000000000000000000000000000000000000000..00bb3a288fbd2fb962b08f34fbe90c733afe0343 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd-interact.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd.png new file mode 100644 index 0000000000000000000000000000000000000000..619172c8ed60a7b536364944a306fbf76fcbfb1f Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-cmd.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-help.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-help.png new file mode 100644 index 0000000000000000000000000000000000000000..97d0dedd3f7b1c749bc5fded471744923d766b8b Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-help.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-init.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-init.png new file mode 100644 index 0000000000000000000000000000000000000000..bbb2257eb1ff2bfec36110409fc6c55a26386c9e Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-init.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-diagnose-detail.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-diagnose-detail.png new file mode 100644 index 0000000000000000000000000000000000000000..7bd624e025eaae4b77c603d88bf1b9ad5e235fe7 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-diagnose-detail.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-diagnose-detect.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-diagnose-detect.png new file mode 100644 index 0000000000000000000000000000000000000000..2b38259ff0c1c7045dbff9abf64f36a109a3377b Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-diagnose-detect.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-diagnose-profiling.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-diagnose-profiling.png new file mode 100644 index 0000000000000000000000000000000000000000..0e63c01f35dbc291f805b56de749eac09e0a079d Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-diagnose-profiling.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-diagnose-report.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-diagnose-report.png new file mode 100644 index 0000000000000000000000000000000000000000..c16f0184a2ad3d2468466b33d0e861d2a31bc4e2 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-diagnose-report.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-diagnose-switch-mode.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-diagnose-switch-mode.png new file mode 100644 index 0000000000000000000000000000000000000000..165c6c453353b70c3e1e2cb07d7f43d5ee3525e3 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-diagnose-switch-mode.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-result.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-result.png new file mode 100644 index 0000000000000000000000000000000000000000..3e3f45a974a0700d209f7d30af89eb2050a392d6 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-result.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-select.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-select.png new file mode 100644 index 0000000000000000000000000000000000000000..13959203c77eaa9f41051897cf9e847ff3642a8a Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-select.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-tuning-metrics-collect.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-tuning-metrics-collect.png new file mode 100644 index 0000000000000000000000000000000000000000..4d5678b7f77b05d48552fcb9656f4a4372dbbe61 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-tuning-metrics-collect.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-tuning-report.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-tuning-report.png new file mode 100644 index 0000000000000000000000000000000000000000..01daaa9a84c13158a95afddffeb8a7e3303f1e76 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-tuning-report.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-tuning-script-exec.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-tuning-script-exec.png new file mode 100644 index 0000000000000000000000000000000000000000..0b694c3fba6918ef39cca977b2072b2913d12b95 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-tuning-script-exec.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-tuning-script-gen.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-tuning-script-gen.png new file mode 100644 index 0000000000000000000000000000000000000000..6e95551767e213f59669d03fd4cceba05801a983 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-tuning-script-gen.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-tuning-script-view.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-tuning-script-view.png new file mode 100644 index 0000000000000000000000000000000000000000..c82c77bf6f4e4e19f400395aaadc9f99dc8d373c Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-tuning-script-view.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-tuning-switch-mode.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-tuning-switch-mode.png new file mode 100644 index 0000000000000000000000000000000000000000..0f06c803ea3621a0f4fb83bbbe731e2bb4bba788 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin-tuning-switch-mode.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin.png new file mode 100644 index 0000000000000000000000000000000000000000..4c1afd306a6aee029f5bda38aa7b1fce57227e31 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/cli_client/pictures/shell-plugin.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/intelligent-plugin-overview.md b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/intelligent-plugin-overview.md new file mode 100644 index 0000000000000000000000000000000000000000..3a37dc9384dcc2080ceb7a687e94e9700e4513eb --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/intelligent-plugin-overview.md @@ -0,0 +1,19 @@ +# 智能插件 + +## 使用方法 + +1. 如图所示，在输入框左上角可以选择插件，点击显示插件列表。 + + ![智能插件](./pictures/plugin-list.png) + +2. 勾选一个插件，然后提问。 + + ![智能插件](./pictures/plugin-selected.png) + +3. 等待服务响应，查看返回结果。 + + 智能插件模式下，推荐问题将置顶推荐的工作流，蓝色文字为对应插件名称，点击后可快捷追问。 + + ![智能插件](./pictures/plugin-suggestion.png) + + ![智能插件](./pictures/plugin-result.png) diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/intelligent-q-and-a-guide.md b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/intelligent-q-and-a-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..00145dfbf739d761d12876ae79e8d9bf84a24b0d --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/intelligent-q-and-a-guide.md @@ -0,0 +1,134 @@ +# 智能问答使用指南 + +## 开始对话 + +在对话区下侧输入框即可输入对话想要提问的内容，输入 `Shift + Enter` 可进行换行，输入 `Enter` 即可发送对话提问内容，或者单击“发送”也可发送对话提问内容。 + +> **说明** +> 对话区位于页面的主体部分，如图 1 所示。 + +- 图 1 对话区 + ![对话区](./pictures/chat-area.png) + +### 多轮连续对话 + +openEuler Copilot System 智能问答支持多轮连续对话。只需要在同一个对话中继续追问即可使用，如图 2 所示。 + +- 图 2 多轮对话 + ![多轮对话](./pictures/context-support.png) + +### 重新生成 + +如遇到 AI 生成的内容有误或不完整的特殊情况，可以要求 AI 重新回答问题。单击对话回答左下侧的“重新生成”文字，可让 openEuler Copilot System 重新回答用户问题，重新回答后，在对话回答右下侧，会出现回答翻页的图标![向前翻页](./pictures/icon-arrow-prev.png)和![向后翻页](./pictures/icon-arrow-next.png)，单击![向前翻页](./pictures/icon-arrow-prev.png)或![向后翻页](./pictures/icon-arrow-next.png)可查看不同的回答，如图 3 所示。 + +- 图 3 重新生成 + ![重新生成](./pictures/regenerate.png) + +### 推荐问题 + +在 AI 回答的下方，会展示一些推荐的问题，单击即可进行提问，如图 4 所示。 + +- 图 4 推荐问题 + ![推荐问题](./pictures/recommend-questions.png) + +## 管理对话 + +> [!TIP]须知 +> +> 对话管理区页面左侧。 + +### 新建对话 + +单击“新建对话”按钮即可新建对话，如图 5 所示。 + +- 图 5 新建对话 + ![新建对话](./pictures/new-chat.png) + +### 对话历史记录搜索 + +在页面左侧历史记录搜索输入框输入关键词，然后单击![icon-search](./pictures/icon-search.png)即可进行对话历史记录搜索如图 6 所示。 + +- 图 6 对话历史记录搜索 + ![对话历史记录搜索](./pictures/search-history.png) + +### 对话历史记录单条管理 + +历史记录的列表位于历史记录搜索栏的下方，在每条对话历史记录的右侧，单击![编辑](./pictures/icon-edit.png)即可编辑对话历史记录的名字，如图 7 所示。 + +- 图 7 重命名历史记录 + ![重命名历史记录](./pictures/rename-session.png) + +在对话历史记录名字重新书写完成后，单击右侧![确认](./pictures/icon-confirm.png)即可完成重命名，或者单击右侧![取消](./pictures/icon-cancel.png)放弃本次重命名，如图 8 所示。 + +- 图 8 完成/取消重命名历史记录 + ![完成/取消重命名历史记录](./pictures/rename-session-confirmation.png) + +另外，单击对话历史记录右侧的删除图标，如图 9 所示，即可对删除单条对话历史记录进行二次确认，在二次确认弹出框，如图 10 所示，单击“确认”，可确认删除单条对话历史记录，或者单击“取消”，取消本次删除。 + +- 图 9 删除单条历史记录 + ![删除单条历史记录](./pictures/delete-session.png) + +- 图 10 删除单条历史记录二次确认 + ![删除单条历史记录二次确认](./pictures/delete-session-confirmation.png) + +### 对话历史记录批量删除 + +首先单击“批量删除”，如图 11 所示。 + +- 图 11 批量删除 + ![批量删除](./pictures/bulk-delete.png) + +然后可对历史记录进行选择删除，如图 12 所示。单击“全选”，即对所有历史记录选中，单击单条历史记录或历史记录左侧的选择框，可对单条历史记录进行选中。 + +- 图 12 批量删除历史记录选择 + ![批量删除历史记录选择](./pictures/bulk-delete-multi-select.png) + +最后需要对批量删除历史记录进行二次确认，如图 13 所示，单击“确认”，即删除，单击“取消”，即取消本次删除。 + +- 图 13 批量删除二次确认 + ![批量删除二次确认](./pictures/bulk-delete-confirmation.png) + +## 反馈与举报 + +在对话记录区，对话回答的右下侧，可进行对话回答反馈，如图 14 所示，单击![满意](./pictures/icon-thumb-up.png)，可给对话回答点赞；单击![不满意](./pictures/icon-thumb-down.png)，可以给对话回答反馈不满意的原因。 + +- 图 14 点赞和不满意反馈 + ![点赞和不满意反馈](./pictures/feedback.png) + +对于反馈不满意原因，如图 15 所示，在单击![不满意](./pictures/icon-thumb-down.png)之后，对话机器人会展示反馈内容填写的对话框，可选择相关的不满意原因的选项。 + +- 图 15 回答不满意反馈 + ![回答不满意反馈](./pictures/feedback-illegal.png) + +其中单击选择“存在错误信息”，需要填写参考答案链接和描述，如图 16 所示。 + +- 图 16 回答不满意反馈——存在错误信息 + ![回答不满意反馈——存在错误信息](./pictures/feedback-misinfo.png) + +### 举报 + +如果发现 AI 返回的内容中有违规信息，可以点击右下角按钮举报，如图 17 所示。点击举报后选择举报类型并提交，若没有合适的选项，请选择“其他”并输入原因，如图 18 所示。 + +- 图 17 举报按钮 + ![举报1](./pictures/report.png) + +- 图 18 选择举报类型 + ![举报2](./pictures/report-options.png) + +## 查看服务协议和隐私政策 + +单击文字“服务协议”，即可查看服务协议，单击文字“隐私政策”，即可查看隐私政策，如图 19、图 20 所示。 + +- 图 19 服务协议和隐私政策入口 + ![服务协议和隐私政策入口](./pictures/privacy-policy-entry.png) + +- 图 20 服务协议和隐私政策 + ![服务协议和隐私政策](./pictures/privacy-policy.png) + +## 附录 + +### 用户信息导出说明 + +#### 具体说明 + +openEuler Copilot System 后台存在用户信息导出功能，如用户需要，需主动通过 邮箱联系我们，运维会将导出的用户信息通过邮箱回送给用户。 diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/introduction.md b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/introduction.md new file mode 100644 index 0000000000000000000000000000000000000000..afd47b84eab80faadce20970b8881eb515996805 --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/introduction.md @@ -0,0 +1,67 @@ +# 前言 + +## 概述 + +本文档介绍了 openEuler Copilot System 的使用方法，对 openEuler Copilot System 线上服务的 Web 界面的各项功能做了详细介绍，同时提供了常见的问题解答，详细请参考对应手册。 + +## 读者对象 + +本文档主要适用于 openEuler Copilot System 的使用人员。使用人员必须具备以下经验和技能： + +- 熟悉 openEuler 操作系统相关情况。 +- 有 AI 对话使用经验。 + +## 修改记录 + +| 文档版本 | 发布日期 | 修改说明 | +|--------|------------|----------------| +| 03 | 2024-09-19 | 更新新版界面。 | +| 02 | 2024-05-13 | 优化智能对话操作指引。 | +| 01 | 2024-01-28 | 第一次正式发布。 | + +## 介绍 + +### 免责声明 + +- 使用过程中涉及的非工具本身验证功能所用的用户名和密码，不作他用，且不会被保存在系统环境中。 +- 在您进行对话或操作前应当确认您为应用程序的所有者或已获得所有者的充足授权同意。 +- 对话结果中可能包含您所分析应用的内部信息和相关数据，请妥善管理。 +- 除非法律法规或双方合同另有规定，openEuler 社区对分析结果不做任何明示或暗示的声明和保证,不对分析结果的适销性、满意度、非侵权性或特定用途适用性等作出任何保证或者承诺。 +- 您根据分析记录所采取的任何行为均应符合法律法规的要求，并由您自行承担风险。 +- 未经所有者授权，任何个人或组织均不得使用应用程序及相关分析记录从事任何活动。openEuler 社区不对由此造成的一切后果负责，亦不承担任何法律责任。必要时，将追究其法律责任。 + +### openEuler Copilot System 简介 + +openEuler Copilot System 是一个基于 openEuler 操作系统的人工智能助手，可以帮助用户解决各种技术问题，提供技术支持和咨询服务。它使用了最先进的自然语言处理技术和机器学习算法，能够理解用户的问题并提供相应的解决方案。 + +### 场景内容 + +1. OS 领域通用知识：openEuler Copilot System 可以咨询 Linux 常规知识、上游信息和工具链介绍和指导。 +2. openEuler 专业知识：openEuler Copilot System 可以咨询 openEuler 社区信息、技术原理和使用指导。 +3. openEuler 扩展知识：openEuler Copilot System 可以咨询 openEuler 周边硬件特性知识和ISV、OSV相关信息。 +4. openEuler 应用案例：openEuler Copilot System 可以提供 openEuler 技术案例、行业应用案例。 +5. shell 命令生成：openEuler Copilot System 可以帮助用户生成单条 shell 命令或者复杂命令。 + +总之，openEuler Copilot System 可以应用于各种场景，帮助用户提高工作效率和了解 Linux、openEuler 等的相关知识。 + +### 访问和使用 + +openEuler Copilot System 通过网址访问 Web 网页进行使用。账号注册与登录请参考[注册与登录](./registration-and-login.md)。使用方法请参考[智能问答使用指南](./intelligent-q-and-a-guide.md)。 + +### 界面说明 + +#### 界面分区 + +openEuler Copilot System 界面主要由如图 1 所示的区域组成，各个区域的作用如表 1 所示。 + +- 图 1 openEuler Copilot System 界面 + ![Copilot 界面](./pictures/main-page-sections.png) + +- 表 1 openEuler Copilot System 首页界面分区说明 + +| 区域 | 名称 | 说明 | +|-----|------------|----------------------------------------------------------------| +| 1 | 设置管理区 | 提供账号登录和退出操作入口和明亮/黑暗模式切换功能 | +| 2 | 对话管理区 | 用于用户新建对话、对话历史记录管理和对话历史记录批量删除操作 | +| 3 | 对话区 | 用于用户和 openEuler Copilot System 的对话聊天 | +| 4 | 服务协议和隐私政策区 | 提供查看服务协议和隐私政策入口 | diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/bulk-delete-confirmation.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/bulk-delete-confirmation.png new file mode 100644 index 0000000000000000000000000000000000000000..33230200fbe9f1e0fa72c27f51b8786192aa14f2 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/bulk-delete-confirmation.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/bulk-delete-multi-select.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/bulk-delete-multi-select.png new file mode 100644 index 0000000000000000000000000000000000000000..96d8201681c4a7772c815a2b9183a0efca9179c2 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/bulk-delete-multi-select.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/bulk-delete.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/bulk-delete.png new file mode 100644 index 0000000000000000000000000000000000000000..929230cd06cc792b633ab183155225926d2c300d Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/bulk-delete.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/chat-area.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/chat-area.png new file mode 100644 index 0000000000000000000000000000000000000000..752f18ad4bd85aaa1132c50cc4c7b7dc159aec91 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/chat-area.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/context-support.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/context-support.png new file mode 100644 index 0000000000000000000000000000000000000000..0bd5f091d0eff34d9b5f36eec6df63b191656daa Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/context-support.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/delete-session-confirmation.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/delete-session-confirmation.png new file mode 100644 index 0000000000000000000000000000000000000000..efd07828e97de46c9660c162ef553362765d5577 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/delete-session-confirmation.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/delete-session.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/delete-session.png new file mode 100644 index 0000000000000000000000000000000000000000..596af33f7be41d456a57e6a297820530f8485f34 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/delete-session.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/feedback-illegal.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/feedback-illegal.png new file mode 100644 index 0000000000000000000000000000000000000000..b6e84ba45977d911db960da97bdff714624ba18c Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/feedback-illegal.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/feedback-misinfo.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/feedback-misinfo.png new file mode 100644 index 0000000000000000000000000000000000000000..cc5505226add1e6fbde7b93ff09877038e8cfdce Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/feedback-misinfo.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/feedback.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/feedback.png new file mode 100644 index 0000000000000000000000000000000000000000..9fe1c27acb57d4d24a26c8dde61ee4272f954e46 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/feedback.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/gitee-login-click2signup.jpg b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/gitee-login-click2signup.jpg new file mode 100644 index 0000000000000000000000000000000000000000..dde8fbe201a44c116e58c3d435737f1a6a3f6f34 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/gitee-login-click2signup.jpg differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/gitee-login.jpg b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/gitee-login.jpg new file mode 100644 index 0000000000000000000000000000000000000000..ac922094fd513e3f8642f885351f541200e6450b Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/gitee-login.jpg differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/gitee-signup.jpg b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/gitee-signup.jpg new file mode 100644 index 0000000000000000000000000000000000000000..57e473466cba423be0d6f76814b5a0656804a884 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/gitee-signup.jpg differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-arrow-next.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-arrow-next.png new file mode 100644 index 0000000000000000000000000000000000000000..1a36c84e0965f9dbf1f90e9a3daadcd1a2560951 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-arrow-next.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-arrow-prev.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-arrow-prev.png new file mode 100644 index 0000000000000000000000000000000000000000..eb667e93cc6d51aa191a0ac7607e72d4d6923cbc Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-arrow-prev.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-cancel.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-cancel.png new file mode 100644 index 0000000000000000000000000000000000000000..34d4454b6f92ee12db6841dafe0e94a12c3b9584 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-cancel.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-confirm.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-confirm.png new file mode 100644 index 0000000000000000000000000000000000000000..1d650f8192e04fae8f7b7c08cd527227c91b833a Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-confirm.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-edit.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-edit.png new file mode 100644 index 0000000000000000000000000000000000000000..f7b28aa605b5e899855a261d641d27a2674703eb Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-edit.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-search.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-search.png new file mode 100644 index 0000000000000000000000000000000000000000..7902923196c3394ae8eafaf5a2b6fdf7f19b1f40 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-search.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-thumb-down.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-thumb-down.png new file mode 100644 index 0000000000000000000000000000000000000000..cda14d196d92898da920ed64ad37fa9dd124c775 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-thumb-down.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-thumb-up.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-thumb-up.png new file mode 100644 index 0000000000000000000000000000000000000000..c75ce44bff456e24bc19040c18e4e644bbb77bd1 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-thumb-up.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-user.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-user.png new file mode 100644 index 0000000000000000000000000000000000000000..e6b06878b76d9e6d268d74070539b388129fa8c4 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/icon-user.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/login-popup.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/login-popup.png new file mode 100644 index 0000000000000000000000000000000000000000..4ac4116f72aa56c81affdb31b806325966331aa9 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/login-popup.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/logout.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/logout.png new file mode 100644 index 0000000000000000000000000000000000000000..e2288c35d89d598f3bb8d939bdf6a9d125bcae83 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/logout.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/main-page-sections.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/main-page-sections.png new file mode 100644 index 0000000000000000000000000000000000000000..155b68928177de0785f4705d2df14c0233b24743 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/main-page-sections.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/new-chat.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/new-chat.png new file mode 100644 index 0000000000000000000000000000000000000000..176bb3e1e932caa758a56540345218c57ee2ff20 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/new-chat.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/plugin-list.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/plugin-list.png new file mode 100644 index 0000000000000000000000000000000000000000..2745f7d82a21cd9eba139898f5ea0c5ab979037f Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/plugin-list.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/plugin-result.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/plugin-result.png new file mode 100644 index 0000000000000000000000000000000000000000..7056aebeecba8760e0ca2773348cce0a0b8167f1 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/plugin-result.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/plugin-selected.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/plugin-selected.png new file mode 100644 index 0000000000000000000000000000000000000000..9182ffa57db9da349cb36186a7b3cb035b51b8aa Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/plugin-selected.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/plugin-suggestion.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/plugin-suggestion.png new file mode 100644 index 0000000000000000000000000000000000000000..bb416881550349000f61b0c1bd3dd540878bd6ad Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/plugin-suggestion.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/privacy-policy-entry.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/privacy-policy-entry.png new file mode 100644 index 0000000000000000000000000000000000000000..d7efce3e6e8d477ef47a1bc8a9bba0d087cf8058 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/privacy-policy-entry.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/privacy-policy.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/privacy-policy.png new file mode 100644 index 0000000000000000000000000000000000000000..dc22c50de7f9d2dc3e0bf523175e7915c91c630f Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/privacy-policy.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/recommend-questions.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/recommend-questions.png new file mode 100644 index 0000000000000000000000000000000000000000..076ec7092af7fe7987e5dc7c864a6b9f8b2b1160 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/recommend-questions.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/regenerate.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/regenerate.png new file mode 100644 index 0000000000000000000000000000000000000000..655c9d5002df4a17aaf84e8780fff4a0118c6c01 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/regenerate.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/rename-session-confirmation.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/rename-session-confirmation.png new file mode 100644 index 0000000000000000000000000000000000000000..d64708bd57d53deafdc5ddbb70d9deaeaca0d132 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/rename-session-confirmation.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/rename-session.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/rename-session.png new file mode 100644 index 0000000000000000000000000000000000000000..73e7e19c5ac8e8035df0e4b553a9b78ff5c9a051 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/rename-session.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/report-options.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/report-options.png new file mode 100644 index 0000000000000000000000000000000000000000..8a54fd2598d51fc40b57052f404dd830cf621f4d Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/report-options.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/report.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/report.png new file mode 100644 index 0000000000000000000000000000000000000000..471bcbe8614fc8bab4dcc1805fa1bf4574990fc8 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/report.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/search-history.png b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/search-history.png new file mode 100644 index 0000000000000000000000000000000000000000..2239d14a7aa8bc13a7b8d3ec71ba9ed71b95e850 Binary files /dev/null and b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/pictures/search-history.png differ diff --git a/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/registration-and-login.md b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/registration-and-login.md new file mode 100644 index 0000000000000000000000000000000000000000..405ebdb4f397f1cabc06a15ce70902765de3ce69 --- /dev/null +++ b/docs/en/tools/ai/openeuler_copilot_system/user_guide/web_client/registration-and-login.md @@ -0,0 +1,68 @@ +# 登录 openEuler Copilot System + +本章节以 Windows 10 操作系统安装的 Chrome 121 浏览器为例介绍登录 openEuler Copilot System 界面的操作步骤。 + +## 浏览器要求 + +浏览器要求如表 1 所示。 + +- 表 1 浏览器要求 + +| 浏览器类型 | 最低版本 | 推荐版本 | +| ----- | ----- | ----- | +| Google Chrome | 72 | 121 或更高版本 | +| Mozilla Firefox | 89 | 122 或更高版本 | +| Apple Safari | 11.0 | 16.3 或更高版本 | + +## 申请访问权限 + +访问 openEuler Copilot System 在线环境，需要依照[【GITEE AI】openEuler Copilot System 在线环境体验申请教程](https://gitee.com/openeuler/euler-copilot-framework/issues/IARUWT?from=project-issue)申请访问权限 + +## 操作步骤 + +> [!TIP]须知 +> openEuler Copilot System 线上服务 (Gitee AI) 账号和 Gitee 官网账号是通用的。 + +**步骤1** 打开本地 PC 机的浏览器，在地址栏输入 [https://ai.gitee.com/apps/zhengw99/openEulerCopilotSystem](https://ai.gitee.com/apps/zhengw99/openEulerCopilotSystem)，按 `Enter`。在未登录状态，进入 openEuler Copilot System，会出现登录提示弹出框，如图 1 所示。 + +- 图 1 未登录 + ![未登录](./pictures/login-popup.png) + +**步骤2** 登录 openEuler Copilot System（已注册账号）。 + +打开登录界面，如图 2 所示。 + +- 图 2 登录 openEuler Copilot System + ![登录 openEuler Copilot System](./pictures/gitee-login.jpg) + +## 注册 openEuler Copilot System 账号 + +> **前提条件** +> 未注册 Gitee 账号。 + +**步骤1** 进入登录页，单击“点此注册”，如图 3 所示。 + +- 图 3 点此注册 + ![点此注册](./pictures/gitee-login-click2signup.jpg) + +**步骤2** 进入账号注册页，根据页面提示填写相关内容，如图 4 所示。 + +- 图 4 账号注册 + ![账号注册](./pictures/gitee-signup.jpg) + +**步骤3** 按页面要求填写账号信息后，单击“立即注册”，即可注册成功。注册后即可返回登录。 + +## 退出登录 + +> **前提条件** +> 已登录 openEuler Copilot System + +**步骤1** 单击![退出登录](./pictures/icon-user.png)后，会出现“退出登录”下拉框，如图 5 所示。 + +> **说明** +> 账号管理区位于页面的右上角部分，如图 5 所示。 + +- 图 5 账号管理区 + ![账号管理区](./pictures/logout.png) + +**步骤2** 单击“退出登录”即可退出登录，如图 5 所示。 diff --git a/docs/en/tools/cloud/_toc.yaml b/docs/en/tools/cloud/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..efa5bf99eff6632e506e11eb4063e2ee98300f31 --- /dev/null +++ b/docs/en/tools/cloud/_toc.yaml @@ -0,0 +1,5 @@ +label: Cloud Native +sections: + - href: ./ctinspector/_toc.yaml + - href: ./cpds/_toc.yaml + - href: ./pilotgo/_toc.yaml diff --git a/docs/en/tools/cloud/cpds/_toc.yaml b/docs/en/tools/cloud/cpds/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..5e7b46c6af6ddccd5e3a1f6c54b507ed6b099894 --- /dev/null +++ b/docs/en/tools/cloud/cpds/_toc.yaml @@ -0,0 +1,11 @@ +label: CPDS User Guide +isManual: true +href: ./cpds-user-guide.md +description: CPDS for container fault and sub-health status monitoring +sections: + - label: CPDS Introduction + href: ./cpds-introduction.md + - label: Installation and Deployment + href: ./installation-and-deployment.md + - label: Usage Instructions + href: ./usage-instructions.md diff --git a/docs/en/tools/cloud/cpds/cpds-introduction.md b/docs/en/tools/cloud/cpds/cpds-introduction.md new file mode 100644 index 0000000000000000000000000000000000000000..91b647eaca4008e39d77a6697a26e52f88a347f8 --- /dev/null +++ b/docs/en/tools/cloud/cpds/cpds-introduction.md @@ -0,0 +1,57 @@ +# CPDS Overview + +## Introduction + +CPDS (Container Problem Detect System), developed by Beijing Linx Software Corp., is a fault detection system for container clusters. It monitors and identifies container top faults and sub-health conditions. + +## Key Features + +**1. Cluster information collection** + +The system uses node agents on host machines, leveraging systemd, initv, and eBPF technologies to monitor key container services. It collects data on node networks, kernels, drive LVM, and other critical metrics. It also tracks application status, resource usage, system function execution, and I/O operations within containers for anomalies. + +**2. Cluster exception detection** + +The system gathers raw data from cluster nodes and applies predefined rules to detect anomalies, extracting essential information. It uploads both detection results and raw data online while ensuring data persistence. + +**3. Node and service container fault/sub-health diagnosis** + +Using exception detection data, the system diagnoses faults or sub-health conditions in nodes and service containers. Analysis results are stored persistently, and a UI layer enables real-time and historical diagnosis data access. + +## System Architecture + +CPDS comprises four components, as illustrated below. The system follows a microservices architecture, with components interacting via APIs. + +![Architecture](images/architecture.png) + +- [cpds-agent](https://gitee.com/openeuler/cpds-agent): Collects raw data about containers and systems from cluster nodes. + +- [cpds-detector](https://gitee.com/openeuler/cpds-detector): Analyzes node data based on exception rules to detect abnormalities. + +- [cpds-analyzer](https://gitee.com/openeuler/cpds-analyzer): Diagnoses node health using configured rules to assess current status. + +- [cpds-dashboard](https://gitee.com/openeuler/cpds-dashboard): Provides a web interface for node health visualization and diagnostic rule configuration. + +## Supported Fault Detection + +CPDS detects the following fault conditions. + +| No. | Fault Detection Item | +| --- | -------------------------------------------------------------------- | +| 1 | Container service functionality | +| 2 | Container node agent functionality | +| 3 | Container group functionality | +| 4 | Node health detection functionality | +| 5 | Log collection functionality | +| 6 | Drive usage exceeding 85% | +| 7 | Network issues | +| 8 | Kernel crashes | +| 9 | Residual LVM drive issues | +| 10 | CPU usage exceeding 85% | +| 11 | Node monitoring functionality | +| 12 | Container memory allocation failures | +| 13 | Container memory allocation timeouts | +| 14 | Container network response timeouts | +| 15 | Slow container drive read/write operations | +| 16 | Zombie child processes in container applications | +| 17 | Child process and thread creation failures in container applications | diff --git a/docs/en/tools/cloud/cpds/cpds-user-guide.md b/docs/en/tools/cloud/cpds/cpds-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..3a234a92804c23ab94fad06623776c351ff30300 --- /dev/null +++ b/docs/en/tools/cloud/cpds/cpds-user-guide.md @@ -0,0 +1,3 @@ +# Overview + +This document outlines the installation, deployment, and usage of CPDS. diff --git a/docs/en/tools/cloud/cpds/images/architecture.png b/docs/en/tools/cloud/cpds/images/architecture.png new file mode 100644 index 0000000000000000000000000000000000000000..0b5e2f24446e79e1e51d6a2288d7cca72d28e0d2 Binary files /dev/null and b/docs/en/tools/cloud/cpds/images/architecture.png differ diff --git "a/docs/en/tools/cloud/cpds/images/cpds-page/\345\216\237\345\247\213\346\225\260\346\215\256\345\233\276\350\241\250.png" "b/docs/en/tools/cloud/cpds/images/cpds-page/\345\216\237\345\247\213\346\225\260\346\215\256\345\233\276\350\241\250.png" new file mode 100644 index 0000000000000000000000000000000000000000..3f3929f2cb29a8a211852af1d468322d52b6e1af Binary files /dev/null and "b/docs/en/tools/cloud/cpds/images/cpds-page/\345\216\237\345\247\213\346\225\260\346\215\256\345\233\276\350\241\250.png" differ diff --git "a/docs/en/tools/cloud/cpds/images/cpds-page/\345\216\237\345\247\213\346\225\260\346\215\256\346\243\200\347\264\242.png" "b/docs/en/tools/cloud/cpds/images/cpds-page/\345\216\237\345\247\213\346\225\260\346\215\256\346\243\200\347\264\242.png" new file mode 100644 index 0000000000000000000000000000000000000000..da9ab5d92c314be3e97560c449b8e55ff6cc44aa Binary files /dev/null and "b/docs/en/tools/cloud/cpds/images/cpds-page/\345\216\237\345\247\213\346\225\260\346\215\256\346\243\200\347\264\242.png" differ diff --git "a/docs/en/tools/cloud/cpds/images/cpds-page/\345\270\203\345\261\200.png" "b/docs/en/tools/cloud/cpds/images/cpds-page/\345\270\203\345\261\200.png" new file mode 100644 index 0000000000000000000000000000000000000000..be9a66c364e92b3376766c59f3cebeebe123daec Binary files /dev/null and "b/docs/en/tools/cloud/cpds/images/cpds-page/\345\270\203\345\261\200.png" differ diff --git "a/docs/en/tools/cloud/cpds/images/cpds-page/\346\227\266\351\227\264\350\214\203\345\233\264\351\200\211\346\213\251.png" "b/docs/en/tools/cloud/cpds/images/cpds-page/\346\227\266\351\227\264\350\214\203\345\233\264\351\200\211\346\213\251.png" new file mode 100644 index 0000000000000000000000000000000000000000..f4abd49e0bf2f62b6bea4968bf13a131f859918a Binary files /dev/null and "b/docs/en/tools/cloud/cpds/images/cpds-page/\346\227\266\351\227\264\350\214\203\345\233\264\351\200\211\346\213\251.png" differ diff --git "a/docs/en/tools/cloud/cpds/images/cpds-page/\346\237\245\347\234\213\345\216\237\345\247\213\346\225\260\346\215\256.png" "b/docs/en/tools/cloud/cpds/images/cpds-page/\346\237\245\347\234\213\345\216\237\345\247\213\346\225\260\346\215\256.png" new file mode 100644 index 0000000000000000000000000000000000000000..a6d27210ed11f2c2ae66068d0ebd74121fa62437 Binary files /dev/null and "b/docs/en/tools/cloud/cpds/images/cpds-page/\346\237\245\347\234\213\345\216\237\345\247\213\346\225\260\346\215\256.png" differ diff --git "a/docs/en/tools/cloud/cpds/images/cpds-page/\346\237\245\347\234\213\350\247\204\345\210\231.png" "b/docs/en/tools/cloud/cpds/images/cpds-page/\346\237\245\347\234\213\350\247\204\345\210\231.png" new file mode 100644 index 0000000000000000000000000000000000000000..f896cdb44a15c527fec3a8df8e6149673f194aeb Binary files /dev/null and "b/docs/en/tools/cloud/cpds/images/cpds-page/\346\237\245\347\234\213\350\247\204\345\210\231.png" differ diff --git "a/docs/en/tools/cloud/cpds/images/cpds-page/\346\267\273\345\212\240\350\247\204\345\210\231.png" "b/docs/en/tools/cloud/cpds/images/cpds-page/\346\267\273\345\212\240\350\247\204\345\210\231.png" new file mode 100644 index 0000000000000000000000000000000000000000..599665e676bc623604bee376e883524838dd663a Binary files /dev/null and "b/docs/en/tools/cloud/cpds/images/cpds-page/\346\267\273\345\212\240\350\247\204\345\210\231.png" differ diff --git "a/docs/en/tools/cloud/cpds/images/cpds-page/\350\212\202\347\202\271\345\201\245\345\272\267.png" "b/docs/en/tools/cloud/cpds/images/cpds-page/\350\212\202\347\202\271\345\201\245\345\272\267.png" new file mode 100644 index 0000000000000000000000000000000000000000..75adbc5d24a46edfe914b2c7e1297882388058fd Binary files /dev/null and "b/docs/en/tools/cloud/cpds/images/cpds-page/\350\212\202\347\202\271\345\201\245\345\272\267.png" differ diff --git "a/docs/en/tools/cloud/cpds/images/cpds-page/\350\212\202\347\202\271\345\256\271\345\231\250\345\201\245\345\272\267\347\233\221\346\216\247.png" "b/docs/en/tools/cloud/cpds/images/cpds-page/\350\212\202\347\202\271\345\256\271\345\231\250\345\201\245\345\272\267\347\233\221\346\216\247.png" new file mode 100644 index 0000000000000000000000000000000000000000..9ba876561699f46b49f6bc0cc8815d0b5806087b Binary files /dev/null and "b/docs/en/tools/cloud/cpds/images/cpds-page/\350\212\202\347\202\271\345\256\271\345\231\250\345\201\245\345\272\267\347\233\221\346\216\247.png" differ diff --git "a/docs/en/tools/cloud/cpds/images/cpds-page/\350\212\202\347\202\271\346\246\202\350\247\210-\346\214\211\351\222\256.png" "b/docs/en/tools/cloud/cpds/images/cpds-page/\350\212\202\347\202\271\346\246\202\350\247\210-\346\214\211\351\222\256.png" new file mode 100644 index 0000000000000000000000000000000000000000..e0ceb89269e6f8a161a2613ae9c542199161e1c8 Binary files /dev/null and "b/docs/en/tools/cloud/cpds/images/cpds-page/\350\212\202\347\202\271\346\246\202\350\247\210-\346\214\211\351\222\256.png" differ diff --git "a/docs/en/tools/cloud/cpds/images/cpds-page/\350\212\202\347\202\271\346\246\202\350\247\210.png" "b/docs/en/tools/cloud/cpds/images/cpds-page/\350\212\202\347\202\271\346\246\202\350\247\210.png" new file mode 100644 index 0000000000000000000000000000000000000000..6f5fd9a5728bb416638feba8174cfb499e5e8f7a Binary files /dev/null and "b/docs/en/tools/cloud/cpds/images/cpds-page/\350\212\202\347\202\271\346\246\202\350\247\210.png" differ diff --git "a/docs/en/tools/cloud/cpds/images/cpds-page/\350\212\202\347\202\271\347\211\251\347\220\206\350\265\204\346\272\220\347\233\221\346\216\247.png" "b/docs/en/tools/cloud/cpds/images/cpds-page/\350\212\202\347\202\271\347\211\251\347\220\206\350\265\204\346\272\220\347\233\221\346\216\247.png" new file mode 100644 index 0000000000000000000000000000000000000000..c0253de34176db4b23e1491a86bb7892966a7c96 Binary files /dev/null and "b/docs/en/tools/cloud/cpds/images/cpds-page/\350\212\202\347\202\271\347\211\251\347\220\206\350\265\204\346\272\220\347\233\221\346\216\247.png" differ diff --git "a/docs/en/tools/cloud/cpds/images/cpds-page/\350\257\212\346\226\255\347\273\223\346\236\234.png" "b/docs/en/tools/cloud/cpds/images/cpds-page/\350\257\212\346\226\255\347\273\223\346\236\234.png" new file mode 100644 index 0000000000000000000000000000000000000000..ffec25e8ff163efc47c38b23fc180ce8188d38dc Binary files /dev/null and "b/docs/en/tools/cloud/cpds/images/cpds-page/\350\257\212\346\226\255\347\273\223\346\236\234.png" differ diff --git "a/docs/en/tools/cloud/cpds/images/cpds-page/\351\233\206\347\276\244-\345\256\271\345\231\250\345\201\245\345\272\267\347\233\221\346\216\247.png" "b/docs/en/tools/cloud/cpds/images/cpds-page/\351\233\206\347\276\244-\345\256\271\345\231\250\345\201\245\345\272\267\347\233\221\346\216\247.png" new file mode 100644 index 0000000000000000000000000000000000000000..5924d73d2b659d92387a19521fd0692d07601046 Binary files /dev/null and "b/docs/en/tools/cloud/cpds/images/cpds-page/\351\233\206\347\276\244-\345\256\271\345\231\250\345\201\245\345\272\267\347\233\221\346\216\247.png" differ diff --git "a/docs/en/tools/cloud/cpds/images/cpds-page/\351\233\206\347\276\244\346\246\202\350\247\210.png" "b/docs/en/tools/cloud/cpds/images/cpds-page/\351\233\206\347\276\244\346\246\202\350\247\210.png" new file mode 100644 index 0000000000000000000000000000000000000000..351614ce5254748c4d233a2189f3f4a71d2f546a Binary files /dev/null and "b/docs/en/tools/cloud/cpds/images/cpds-page/\351\233\206\347\276\244\346\246\202\350\247\210.png" differ diff --git "a/docs/en/tools/cloud/cpds/images/cpds-page/\351\233\206\347\276\244\347\211\251\347\220\206\350\265\204\346\272\220\347\233\221\346\216\247.png" "b/docs/en/tools/cloud/cpds/images/cpds-page/\351\233\206\347\276\244\347\211\251\347\220\206\350\265\204\346\272\220\347\233\221\346\216\247.png" new file mode 100644 index 0000000000000000000000000000000000000000..5256783e5f907232f3cbd3655b8ee81c03c0ffdd Binary files /dev/null and "b/docs/en/tools/cloud/cpds/images/cpds-page/\351\233\206\347\276\244\347\211\251\347\220\206\350\265\204\346\272\220\347\233\221\346\216\247.png" differ diff --git "a/docs/en/tools/cloud/cpds/images/cpds-page/\351\233\206\347\276\244\347\212\266\346\200\201-\346\246\202\350\247\210.png" "b/docs/en/tools/cloud/cpds/images/cpds-page/\351\233\206\347\276\244\347\212\266\346\200\201-\346\246\202\350\247\210.png" new file mode 100644 index 0000000000000000000000000000000000000000..751887e223d0a323feee7baef3202ed23e6ce5de Binary files /dev/null and "b/docs/en/tools/cloud/cpds/images/cpds-page/\351\233\206\347\276\244\347\212\266\346\200\201-\346\246\202\350\247\210.png" differ diff --git a/docs/en/tools/cloud/cpds/installation-and-deployment.md b/docs/en/tools/cloud/cpds/installation-and-deployment.md new file mode 100644 index 0000000000000000000000000000000000000000..009a7b5cddf181e0ac93274a204baf8ecaba8658 --- /dev/null +++ b/docs/en/tools/cloud/cpds/installation-and-deployment.md @@ -0,0 +1,258 @@ +# Installation and Deployment + +This chapter provides a step-by-step guide to installing and deploying CPDS. + +## Installing CPDS + +This section covers the steps to install CPDS components. + +1. Install cpds-agent. + + > cpds-agent gathers raw data from nodes and can be installed independently on multiple nodes. + + ```shell + yum install cpds-agent + ``` + +2. Install cpds-detector. + + ```shell + yum install cpds-detector + ``` + +3. Install cpds-analyzer. + + ```shell + yum install cpds-analyzer + ``` + +4. Install cpds-dashboard. + + ```shell + yum install cpds-dashboard + ``` + +5. Install Cpds. + + ```shell + yum install Cpds + ``` + +# Deployment of CPDS + +This section explains the configuration and deployment of CPDS. + +## Configuration Overview + +### cpds-agent Configuration + +cpds-agent collects node network information by sending ICMP packets to a specified IP address. The `net_diagnostic_dest` field must specify a reachable IP address, not the local node IP address. You are advised to set the master node IP address on worker nodes and any worker node IP address on the master. + +```bash +vim /etc/cpds/agent/config.json +``` + +```json +{ + "expose_port":"20001", # Port to listen on + "log_cfg_file": "/etc/cpds/agent/log.conf", + "net_diagnostic_dest": "192.30.25.18" # Destination IP address for ICMP packets +} +``` + +### Prometheus Configuration + +CPDS uses Prometheus to collect raw data generated by cpds-agent. cpds-agent opens port 20001 by default. Edit the Prometheus configuration file to connect to cpds-agent for data collection. + +```bash +vim /etc/prometheus/prometheus.yml +``` + +```yaml +global: + scrape_interval: 2s + evaluation_interval: 3s +scrape_configs: + - job_name: "cpds" + static_configs: + - targets: ["cpds-agent1:port","cpds-agent2:port","..."] # IP addresses and ports of deployed cpds-agent instances +``` + +### cpds-detector Configuration + +```bash +vim /etc/cpds/detector/config.yml +``` + +```yaml +generic: + bindAddress: "127.0.0.1" # Address to listen on + port: 19091 # Port to listen on + +database: + host: "127.0.0.1" # Database IP address + port: 3306 # Database port + username: root # Database username + password: root # Database password + maxOpenConnections: 123 # Maximum number of connections + +prometheus: + host: "127.0.0.1" # Detector IP address + port: 9090 # Prometheus port + +log: + fileName: "/var/log/cpds/cpds-detector/cpds-detector.log" + level: "warn" + maxAge: 15 + maxBackups: 100 + maxSize: 100 + localTime: true + compress: true +``` + +### cpds-analyzer Configuration + +```bash +vim /etc/cpds/analyzer/config.yml +``` + +```yaml +generic: + bindAddress: "127.0.0.1" # Address to listen on + port: 19091 # Port to listen on + +database: + host: "127.0.0.1" # Database IP address + port: 3306 # Database port + username: root # Database username + password: root # Database password + maxOpenConnections: 123 # Maximum number of connections + +detector: + host: "127.0.0.1" # Detector IP address + port: 19092 # Detector port + +log: + fileName: "/var/log/cpds/cpds-analyzer/cpds-analyzer.log" + level: "warn" + maxAge: 15 + maxBackups: 100 + maxSize: 100 + localTime: true +``` + +### cpds-dashboard Configuration + +```bash +vim /etc/nginx/conf.d/cpds-ui.conf +``` + +```conf +server { + listen 10119; + + location / { + root /etc/cpds/cpds-ui/; + index index.html index.htm; + } + + location /api/ { + proxy_pass http://127.0.0.1:19091; # Backend analyzer IP address and port + } + + location /websocket/ { + proxy_pass http://127.0.0.1:19091; # Backend analyzer IP address and port + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-Proto http; + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "Upgrade"; + } +} +``` + +# Starting CPDS + +This section outlines the steps to start CPDS. + +## Disabling the Firewall + +```shell +systemctl stop firewalld +systemctl disable firewalld +``` + +Set the SELINUX status to `disabled` in the **/etc/selinux/config** file. + +```conf +SELINUX=disabled +``` + +Restart the system to apply the changes. + +## Initializing the Database + +1. Start the database service. + + ```shell + systemctl start mariadb.service + systemctl enable mariadb.service + ``` + +2. Initialize the database with root privileges. + + ```shell + /usr/bin/mysql_secure_installation + ``` + + > During the process, you will be prompted to enter the **root** user password for the database. If no password is set, press **Enter** and follow the prompts to configure the settings. + +3. Configure database connection permissions. + + ```shell + mysql -u root -p + ``` + + Enter the password set in the previous step when prompted. + + ```shell + GRANT ALL PRIVILEGES ON *.* TO 'username'@'%' IDENTIFIED BY 'password' WITH GRANT OPTION; + ``` + + > Replace `username` with the database username and `password` with the corresponding password. + + For example: + + ```shell + mysql -u root -p + Enter password: + Welcome to the MariaDB monitor. Commands end with ; or \g. + Your MariaDB connection id is 5 + Server version: 10.5.16-MariaDB MariaDB Server + + Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. + + Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. + + MariaDB [(none)]> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'root' WITH GRANT OPTION; + Query OK, 0 rows affected (0.014 sec) + ``` + +### Starting the Service + +```shell +systemctl start Cpds.service +systemctl enable Cpds.service +``` + +Start cpds-agent on all nodes. + +```shell +systemctl start cpds-agent +systemctl enable cpds-agent +``` + +### Accessing the Frontend Management Platform + +Once the services are running, open a browser and navigate to **** to access the frontend management platform. diff --git a/docs/en/tools/cloud/cpds/usage-instructions.md b/docs/en/tools/cloud/cpds/usage-instructions.md new file mode 100644 index 0000000000000000000000000000000000000000..58112f4fc72c682bd3d80df0e0ee4d6a98869d4c --- /dev/null +++ b/docs/en/tools/cloud/cpds/usage-instructions.md @@ -0,0 +1,261 @@ +# 使用手册 + + + +- [介绍](#介绍) +- [页面及功能说明](#页面及功能说明) + - [页面布局](#页面布局) + - [概览](#概览) + - [监控告警](#监控告警) + - [集群状态](#集群状态) + - [集群状态-概览](#集群状态-概览) + - [集群状态-物理资源监控](#集群状态-物理资源监控) + - [集群状态-容器健康监控](#集群状态-容器健康监控) + - [节点健康](#节点健康) + - [节点健康-概览](#节点健康-概览) + - [节点健康-物理资源监控](#节点健康-物理资源监控) + - [节点健康-容器健康监控](#节点健康-容器健康监控) + - [健康诊断](#健康诊断) + - [诊断结果](#诊断结果) + - [原始数据检索](#原始数据检索) + - [原始数据图表](#原始数据图表) + - [规则管理](#规则管理) + - [查看规则](#查看规则) + - [添加规则](#添加规则) + + + +## 介绍 + +CPDS(Container Problem Detect System)容器故障检测系统，是由北京凝思软件股份有限公司设计并开发的容器集群故障检测系统，该软件系统实现了对容器 TOP 故障、亚健康检测的监测与识别。 + +主要分为四个子模块： + +1. 信息采集组件 cpds-agent：本组件根据cpds-detetor(异常检测组件)需要的数据进行相应采集。 +2. 异常检测组件 cpds-detector：本组件根据cpds-analyzer(容器故障/亚健康诊断组件)下发的异常规则，对集群各节点原始数据进行分析，检测节点是否存在异常。 +3. 故障/亚健康诊断组件 cpds-analyzer：本组件根据cpds-dashboard(用户交互组件)下发的诊断规则，对cpds-detector(异常检测组件)收集的异常数据进行处理，判断集群节点是否处于容器故障/亚健康状态。 +4. 用户交互组件 cpds-dashboard：本组件从 cpds-analyzer(故障/亚健康诊断)组件中获取诊断结果数据，并以实时查看、离线查看的形式进行可视化诊断结果展示，便于容器集群运维人员进行分析与策略制定下发。 + +## 页面及功能说明 + +### 页面布局 + +CPDS页面布局分为导航栏、导航菜单、操作区。 + +> 页面布局如下图 + +![页面布局](./images/cpds-page/布局.png) + +| 序号 | 名称 | 说明 | +| ---- | -------- | ------------------------------------------------------------------------------ | +| 1 | 导航菜单 | 导航菜单包含 CPDS 所有功能，选择不同菜单项后，右侧操作区将显示对应的操作页面。 | +| 2 | 导航栏 | 用于指示用户当前页面位于导航树的位置。 | +| 3 | 操作区 | 显示当前操作信息，提供操作功能。 | + +### 概览 + +概览页面可查看整个集群的状态信息，包括集群容器健康状态、集群节点状态、集资源用量、节点监控状态、诊断结果。查看概览流程如下图所示： + +![集群概览](./images/cpds-page/集群概览.png) + +| 名称 | 说明 | +| ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | +| 容器健康状态 | 显示集群中运行中的容器个数占全部容器个数的百分比，并显示全部容器、运行中的容器、停止的容器的个数。 | +| 集群节点状态 | 显示在线节点占全部节点的百分比，并显示全部节点、在线节点、离线节点的个数。 | +| 集群资源用量 | 显示集群 CUP、内容、磁盘的使用的量、总量和使用百分比。 | +| 节点监控状态 | 显示集群节点的 ip 地址、节点状态、节点运行容器数量占比。点击下方的查看更多，会跳转至“监控告警-节点健康”，可以查看更详细的节点信息。 | +| 诊断结果 | 显示触发规则的名称、当前状态、规则第一次触发的时间，以及后续触发的最新时间。点击下方的查看更多，会跳转至“健康诊断-诊断结果”，查看更详细的诊断结果。 | + +### 监控告警 + +监控告警能够对集群、节点的物理资源、容器状态进行监控。 + +#### 集群状态 + +显示集群主机在线状态，提供物理资源监控和容器健康监控。 + +##### 集群状态-概览 + +查看集群信息和节点信息，集群信息包括集群容器健康状态、集群节点状态、集资源用量。查看集群信息流程如下所示： + +1. 点击左侧导航菜单“监控告警”→“集群状态”，选择“概览”标签页，进入概览页面。如下图所示： + ![集群状态-概览](./images/cpds-page/集群状态-概览.png) + + | 名称 | 说明 | + | ------------ | -------------------------------------------------------------------------------------------------- | + | 容器健康状态 | 显示集群中运行中的容器个数占全部容器个数的百分比，并显示全部容器、运行中的容器、停止的容器的个数。 | + | 集群节点状态 | 显示在线节点占全部节点的百分比，并显示全部节点、在线节点、离线节点的个数。 | + | 资源使用情况 | 显示集群 CUP、内容、磁盘的使用的量和总量。 | + | 节点监控状态 | 详见 [节点健康](#节点健康)。 | + +##### 集群状态-物理资源监控 + +点击左侧导航菜单“监控告警”→“集群状态”，选择“物理资源监控”标签页，物理资源监控页面内容如下图所示。 +![集群物理资源监控](./images/cpds-page/集群物理资源监控.png) + +> 其中点击查询时间范围按钮可选择查询数据的时间范围，如下图所示。 +> ![时间范围选择](./images/cpds-page/时间范围选择.png) + +下面将对物理资源监控内容进行说明。 + +| 名称 | 说明 | +| ------------------ | ------------------------------------------------------ | +| 集群总 CPU 使用率 | 集群 CPU 使用百分比 | +| 集群总内存使用率 | 集群内存使用百分比 | +| 集群总磁盘使用率 | 集群磁盘使用百分比 | +| 集群iowait | 集群CPU等待I/O设备完成输入输出操作而处于空闲状态的时间 | +| 网络iops | 集群网卡每秒接收和发送数据包总数 | +| 网络网速 | 集群网卡每秒接收和发送数据大小 | +| 网络丢包率 | 集群单位时间内网卡丢失数据包占总数据包的百分比 | +| 网络错误率 | 集群单位时间内网卡出现错误的数据包占总数据包的百分比 | +| 网络重传率 | 集群单位时间内重传数据包占总数据包的百分比 | +| 集群总磁盘吞吐速率 | 集群磁盘每秒完成读写操作的数据量 | +| 磁盘 iops | 集群磁盘每秒完成读写操作的次数 | + +##### 集群状态-容器健康监控 + +点击左侧导航菜单“监控告警”→“集群状态”，选择“容器健康监控”标签页，该页面显示集群容器健康监控信息如下图所示： +![集群-容器健康监控](./images/cpds-page/集群-容器健康监控.png) + +下面将对容器健康监控内容进行说明。 + +| 名称 | 说明 | +| -------------- | -------------------------------------- | +| 容器CPU使用率 | 容器 CPU 使用量占集群 CPU 总量的百分比 | +| 容器磁盘使用率 | 容器磁盘使用量占集群磁盘总量的百分比 | +| 容器流量 | 容器每秒网卡接收/发送的数据量 | +| 容器内存使用率 | 容器内存使用量占集群内存总量的百分比 | + +#### 节点健康 + +显示各节点主机在线状态及架构信息，提供物理资源监控和容器健康监控。 +> 节点健康主页面如下图所示： + +![节点健康](./images/cpds-page/节点健康.png) + +##### 节点健康-概览 + +点击左侧导航菜单“监控告警”→“节点健康”，点击表格中节点对应的 ip 地址进入节点概览页面。 +> 节点概览页面如下图所示： + +![节点概览](./images/cpds-page/节点概览.png) + +> 点击如下图三个组件，可刷新或切换曲线图展示的数据内容。 + +![节点概览-按钮](./images/cpds-page/节点概览-按钮.png) + +##### 节点健康-物理资源监控 + +点击左侧导航菜单“监控告警”→“节点健康”，点击表格中节点对应的 ip 地址进入节点概览页面，选择“物理资源监控”标签页，物理资源监控页面内容如下图所示： +![节点物理资源监控](./images/cpds-page/节点物理资源监控.png) + +下面将对物理资源监控内容进行说明。 + +| 名称 | 说明 | +| ---------------- | ------------------------------------------------------ | +| 节点 CPU 使用率 | 节点 CPU 使用百分比 | +| 节点内存使用率 | 节点内存使用百分比 | +| 节点磁盘使用率 | 节点磁盘使用百分比 | +| 节点iowait | 节点CPU等待I/O设备完成输入输出操作而处于空闲状态的时间 | +| 节点网络iops | 节点网卡每秒接收和发送数据包总数 | +| 节点网络网速 | 节点网卡每秒接收和发送数据大小 | +| 节点网络丢包率 | 节点单位时间内网卡丢失数据包占总数据包的百分比 | +| 节点网络错误率 | 节点单位时间内网卡出现错误的数据包占总数据包的百分比 | +| 节点网络重传率 | 节点单位时间内重传数据包占总数据包的百分比 | +| 节点磁盘吞吐速率 | 节点磁盘每秒完成读写操作的数据量 | +| 节点磁盘 iops | 节点磁盘每秒完成读写操作的次数 | + +##### 节点健康-容器健康监控 + +点击左侧导航菜单“监控告警”→“节点健康”，点击表格中节点对应的 ip 地址进入节点概览页面，选择“容器健康监控”标签页，容器健康监控页面如下图所示： +![节点容器健康监控](./images/cpds-page/节点容器健康监控.png) + +> 该页面可根据容器状态、容器名称进行排序。 + +节点容器健康监控数据说明如下表： + +| 名称 | 说明 | +| -------- | -------------------------------------------------------------- | +| 容器名称 | 容器的完整id | +| 容器状态 | 容器的运行状态，包括：运行中、已创建、停止等待、暂停共四个状态 | +| CPU用量 | 容器CPU使用率 | +| 内存用量 | 容器内存用量 | +| 出站流量 | 容器网卡对外发送数据大小 | +| 如站流量 | 容器网卡对接收数据大小 | + +### 健康诊断 + +利用故障/亚健康检测规则，对各节点原始数据进行计算分析，得出诊断结果，提供诊断结果列表。支持诊断原始数据查看，显示诊断时所使用的原始数据，支持对时间进行过滤，显示不同时间段原始数据的值，并提供图表展示原始数据的变化规律。 + +#### 诊断结果 + +将规则列表中的规则拿来进行判断，满足判断条件的规则将被加入到诊断结果列表中。规则信息详见 [规则管理](#规则管理)。 +查看诊断结果列表流程如下： + +1. 点击左侧导航菜单“健康诊断”→“诊断结果”，进入诊断结果页面。如下图所示： + ![诊断结果](./images/cpds-page/诊断结果.png) + +2. 可在左上角输入规则名称对诊断结果进行筛选。 + +3. 点击查看原始数据，可以查看最近 10 分钟内原始数据的变化规律，如下图所示： + ![查看原始数据](./images/cpds-page/查看原始数据.png) + +4. 点击删除可以删除对应的诊断结果。 + +#### 原始数据检索 + +支持诊断原始数据查看，显示诊断时所使用的原始数据，支持对时间进行过滤，显示不同时间段原始数据的值，并提供图表展示原始数据的变化规律。页面布局如下图所示： + ![原始数据检索](./images/cpds-page/原始数据检索.png) + +功能说明如下表所示： + +| 名称 | 说明 | +| ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- | +| 原始数据查询 | 利用表达式对原始数据进行查询，可以设置时间选择器对时间进行过滤。可以查询到一段时间内原始数据的变化规律。 | +| 容器状态 | 当利用表达式成功查询原始数据后，查询记录将被记录到表格中，如果超过 10 条不同表达式记录，最先查询的记录将被删除。如果是相同表达式，那么记录会被覆盖。 | + +##### 原始数据图表 + +原始数据图表如下图所示： + ![原始数据图表](./images/cpds-page/原始数据图表.png) + +图表信息说明如下表： + +| 名称 | 说明 | +| -------------- | ------------------------------------------ | +| 监控指标 | 显示内容为查询的表达式 | +| 原始数据曲线图 | 显示该表达式在一段时间内查询结果的变化规律 | +| 原始数据表格 | 显示当前时间，查询结果的字段以及值 | + +### 规则管理 + +#### 查看规则 + +支持故障/亚健康检测规则列表查看、创建、编辑、删除功能，列表包括规则名、表达式、告警级别和亚健康、故障比较规则值信息。查看规则流程如下： + +1. 点击左侧导航菜单“规则管理”→“查看规则”，进入规则列表页面，如下图所示： + ![查看规则](./images/cpds-page/查看规则.png) + +2. 通过左上角输入规则名称，点击搜索可以对规则进行过滤。 +3. 点击删除可以删除对应规则。 + +#### 添加规则 + +击左侧导航菜单“规则管理”→“查看规则”，进入规则列表页面，点击添加规则或者编辑，如下图所示： + ![添加规则](./images/cpds-page/添加规则.png) + +添加/编辑规则内容有如下几点限制： + +1. 规则名称，只能包含数字、英文字母、下划线。 +2. 表达式必须符合PromQL语法，参考[prometheus官方文档](https://prometheus.io/docs/prometheus/latest/querying/basics/)。 +3. 亚健康阈值只能输入数字类型。 +4. 故障康阈值只能输入数字类型。 + +> 阈值只能在对应的比较条件选择之后才能输入。 +> 当比较条件选择之后，对应的阈值必须填写入。 +> 亚健康比较条件、故障比较条件二者必须选择一个，或者两个都选。 + +## 注意事项 + +1. 默认规则，规则名称为node_etcd_service、node_kube_apiserver、node_kube_controller_manager、node_kube_proxynode_kube_scheduler的规则表达式中的ip需要自行更换为实际ip。 +2. 当前版本CPDS只支持对docker容器运行时的故障检测。 diff --git a/docs/en/tools/cloud/ctinspector/_toc.yaml b/docs/en/tools/cloud/ctinspector/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..b494f8914feb9e814d378e65f2baece426dd9f4b --- /dev/null +++ b/docs/en/tools/cloud/ctinspector/_toc.yaml @@ -0,0 +1,12 @@ +label: CTinspector Introduction +isManual: true +description: >- + CTinspector enables precise diagnosis of runtime performance bottlenecks and + system faults. +sections: + - label: Installation and Deployment + href: ./ctinspector-introduction.md + - label: Installation and Deployment + href: ./installation-and-deployment.md + - label: Usage Instructions + href: ./usage-instructions.md diff --git a/docs/en/tools/cloud/ctinspector/ctinspector-introduction.md b/docs/en/tools/cloud/ctinspector/ctinspector-introduction.md new file mode 100644 index 0000000000000000000000000000000000000000..3fe26197753f175c18f491c28ced200755037952 --- /dev/null +++ b/docs/en/tools/cloud/ctinspector/ctinspector-introduction.md @@ -0,0 +1,48 @@ +# CTinspector Introduction + +## Overview + +CTinspector is a language VM framework developed by China Telecom e-Cloud Technology Co., Ltd. based on the eBPF instruction set. CTinspector enables quick expansion of application instances to diagnose network performance bottlenecks, storage I/O hotspots, and load balancing issues, ensuring stable and timely diagnosis during system running. + +Before CTinspector introduces the O&M and problem analysis of the cloud base system, the OVS O&M and ACL configuration efficiency is relatively low, and some functions are not supported. + +* The filtering field needed by the maintenance personnel is not implemented, or the AND or NOT condition expression is not supported. + +* Many commands in the system have similar filtering requirements, such as CT flow tables, OpenFlow flow tables, and offload flow tables. Developing command parameters for each flow table is a heavy development burden. + +* Stateful filtering, for example, viewing the flow table that matches the most packets, cannot be implemented based on command parameters. Traditional filtering rules are for individual flow tables. The relationships between flow tables cannot be established. + +## Architecture + +CTinspector uses a packet VM of the eBPF instruction set. The minimum size of the packet VM is 256 bytes, covering registers, segments (stack, code, and data), and page tables. The packet VM supports independent migration, in which the packet VM code can invoke the migrate kernel function to migrate to a specified node. It also supports resumable execution, that is, once migrated, the packet VM continues to execute the next instruction from the position where it has been interrupted on the previous node. + +![](./figures/CT-package-vm.png) + +The overall architecture of CTinspector is as follows: + +![](./figures/CTinspector-arch.png) + +The CTinspector framework comprises the following components: + +* **eBPF compiler/JIT**: + The eBPF compiler compiles C code into eBPF binary code, and JIT compiles eBPF instructions into +machine code. + +* **eBPF linker/loader**: + loads and links library functions, that is, kernel functions. + +* **Runner**: + executes the eBPF VM, including loading registers, code segments, and stacks, and mapping data segments. + +* **Scheduler**: + determines when to execute the eBPF VM, including determining the VM status and dependency wait conditions. + +* **Basic kernel functions**: + basic library functions, such as transporter, memory mapper, fork, and join_meeting. + +* **Extended kernel functions**: + custom library functions provided by each hook point in addition to the core functions +provided by the eBPF VM runner. + +* **Memory mapper**: + maps application data to the eBPF VM to ensure the eBPF program can read and write application data. diff --git a/docs/en/tools/cloud/ctinspector/figures/CT-package-vm.png b/docs/en/tools/cloud/ctinspector/figures/CT-package-vm.png new file mode 100644 index 0000000000000000000000000000000000000000..bb1ad48a6f28f39b73776b67804332036c32bdce Binary files /dev/null and b/docs/en/tools/cloud/ctinspector/figures/CT-package-vm.png differ diff --git a/docs/en/tools/cloud/ctinspector/figures/CTinspector-arch.png b/docs/en/tools/cloud/ctinspector/figures/CTinspector-arch.png new file mode 100644 index 0000000000000000000000000000000000000000..82f647b7c0a311c8af597ce3fabb3cdf93e5afcc Binary files /dev/null and b/docs/en/tools/cloud/ctinspector/figures/CTinspector-arch.png differ diff --git a/docs/en/tools/cloud/ctinspector/figures/migrate_node_1.png b/docs/en/tools/cloud/ctinspector/figures/migrate_node_1.png new file mode 100644 index 0000000000000000000000000000000000000000..3d7ddb16959cf83235703f564d002f95396f1963 Binary files /dev/null and b/docs/en/tools/cloud/ctinspector/figures/migrate_node_1.png differ diff --git a/docs/en/tools/cloud/ctinspector/figures/migrate_node_2.png b/docs/en/tools/cloud/ctinspector/figures/migrate_node_2.png new file mode 100644 index 0000000000000000000000000000000000000000..99448ced22a6cd34d393ea31cff0ef67d43ec028 Binary files /dev/null and b/docs/en/tools/cloud/ctinspector/figures/migrate_node_2.png differ diff --git a/docs/en/tools/cloud/ctinspector/installation-and-deployment.md b/docs/en/tools/cloud/ctinspector/installation-and-deployment.md new file mode 100644 index 0000000000000000000000000000000000000000..55c4e14b94cdc4b4cd4953151d64b9d066c7aaa8 --- /dev/null +++ b/docs/en/tools/cloud/ctinspector/installation-and-deployment.md @@ -0,0 +1,42 @@ +# Installation and Deployment + +## Software Requirements + +* OS: openEuler 23.09 + +## Hardware Requirements + +* x86_64 architecture + +## Environment Preparation + +* Install openEuler by referring to [Installation Guide](../../../server/installation_upgrade/installation/installation.md) + +* CTinspector installation requires **root** permissions. + +## CTinspector Installation + +* Install the CTinspector framework software package. + +```shell +yum install ctinspector +``` + +* Check whether the installation is successful. If the corresponding software package is displayed in the output, the installation is successful. + +```shell +rpm -q ctinspector +``` + +* Check whether the core dynamic library **libebpf_vm_executor.so** or main program **vm_test** is installed. + +```shell +rpm -ql ctinspector +/usr/bin/vm_test +/usr/include/ctinspector/ebpf_vm_functions.h +/usr/include/ctinspector/ebpf_vm_simulator.h +/usr/include/ctinspector/ebpf_vm_transport_rdma.h +/usr/include/ctinspector/list.h +/usr/include/ctinspector/ub_list.h +/usr/lib64/libebpf_vm_executor.so +``` diff --git a/docs/en/tools/cloud/ctinspector/usage-instructions.md b/docs/en/tools/cloud/ctinspector/usage-instructions.md new file mode 100644 index 0000000000000000000000000000000000000000..f2aae495057bd4fb920a4e6f9f0506d2193c5a0c --- /dev/null +++ b/docs/en/tools/cloud/ctinspector/usage-instructions.md @@ -0,0 +1,45 @@ +# Usage + +## NIC Configuration + +```shell +# Change the MTU of the NIC. +ifconfig ens33 mtu 4200 + +# Add an RXE interface to ens33 for the IB function. +rdma link add rxe_0 type rxe netdev ens33 + +``` + +## Application Development + +Use relevant APIs to develop a scenario-specific application. Build the application as a binary ELF file based on the eBPF instruction set. Take **vm_migrate** of the provided **ebpf_example** for example. **vm_migrate** calls the CTinspector framework and can migrate package VMs between nodes in a resumable manner. + +```text +# Compose the Makefile and set the eBPF instruction set. + +CFLAGS=-O2 -fno-inline -emit-llvm -I/usr/include/ctinspector/ +LINKFLAGS=-march=bpf -filetype=obj + +all: vm_migrate.o + +vm_migrate.o: + clang $(CFLAGS) -c migrate.c -o - | llc $(LINKFLAGS) -o vm_migrate.o + +clean: + rm -f vm_migrate.o +``` + +```shell +# Run make to build the application. +make +clang -O2 -fno-inline -emit-llvm -I/usr/include/ctinspector/ -c migrate.c -o - | llc -march=bpf -filetype=obj -o vm_migrate.o +``` + +## Application Running + +Running **vm_migrate** on node 1. +![](./figures/migrate_node_1.png) + +Running the CTinspector main program on node 2. +![](./figures/migrate_node_2.png) diff --git a/docs/en/tools/cloud/pilotgo/_toc.yaml b/docs/en/tools/cloud/pilotgo/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..7031e3b10298d3243a613357d400a9cb1c264516 --- /dev/null +++ b/docs/en/tools/cloud/pilotgo/_toc.yaml @@ -0,0 +1,7 @@ +label: PilotGo User Guide +isManual: true +href: ./pilotgo-introduction.md +Description: Manage hosts, permissions, alarms, and other tasks using PilotGo. +sections: + - label: Usage Instructions + href: ./usage-instructions.md diff --git "a/docs/en/tools/cloud/pilotgo/figures/G\346\217\222\344\273\2661.png" "b/docs/en/tools/cloud/pilotgo/figures/G\346\217\222\344\273\2661.png" new file mode 100644 index 0000000000000000000000000000000000000000..5c7eaa4cde3364c70ca6bff24c768edad986a59c Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/G\346\217\222\344\273\2661.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/G\346\217\222\344\273\2662.png" "b/docs/en/tools/cloud/pilotgo/figures/G\346\217\222\344\273\2662.png" new file mode 100644 index 0000000000000000000000000000000000000000..45437297fb46749b9f840f45e38cc3e5c4d0d595 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/G\346\217\222\344\273\2662.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/G\346\217\222\344\273\2663.png" "b/docs/en/tools/cloud/pilotgo/figures/G\346\217\222\344\273\2663.png" new file mode 100644 index 0000000000000000000000000000000000000000..d120fdc034f2c588c222837e8316a33cda339e22 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/G\346\217\222\344\273\2663.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/G\346\217\222\344\273\2664.png" "b/docs/en/tools/cloud/pilotgo/figures/G\346\217\222\344\273\2664.png" new file mode 100644 index 0000000000000000000000000000000000000000..1e2ed031ac525d8a69c98c9f143b3edece72be77 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/G\346\217\222\344\273\2664.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/P\346\217\222\344\273\2661.png" "b/docs/en/tools/cloud/pilotgo/figures/P\346\217\222\344\273\2661.png" new file mode 100644 index 0000000000000000000000000000000000000000..f4a923729e62fb321931342ec56238b568dbf16e Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/P\346\217\222\344\273\2661.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/P\346\217\222\344\273\2662.png" "b/docs/en/tools/cloud/pilotgo/figures/P\346\217\222\344\273\2662.png" new file mode 100644 index 0000000000000000000000000000000000000000..d54a04a42afa0f0ae7d37fb2eef88943e4b402f5 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/P\346\217\222\344\273\2662.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/P\346\217\222\344\273\2663.png" "b/docs/en/tools/cloud/pilotgo/figures/P\346\217\222\344\273\2663.png" new file mode 100644 index 0000000000000000000000000000000000000000..a85aad4547a6dc8b6d55d50524c69c92668e54a6 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/P\346\217\222\344\273\2663.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/P\346\217\222\344\273\2664.png" "b/docs/en/tools/cloud/pilotgo/figures/P\346\217\222\344\273\2664.png" new file mode 100644 index 0000000000000000000000000000000000000000..c56bcc5248a53f9d5daeadaddb998d69ef154c4e Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/P\346\217\222\344\273\2664.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\344\277\256\346\224\271\345\257\206\347\240\2011.png" "b/docs/en/tools/cloud/pilotgo/figures/\344\277\256\346\224\271\345\257\206\347\240\2011.png" new file mode 100644 index 0000000000000000000000000000000000000000..a51096f17e336fc0917bce7be08ff69ec2604562 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\344\277\256\346\224\271\345\257\206\347\240\2011.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\344\277\256\346\224\271\345\257\206\347\240\2012.png" "b/docs/en/tools/cloud/pilotgo/figures/\344\277\256\346\224\271\345\257\206\347\240\2012.png" new file mode 100644 index 0000000000000000000000000000000000000000..f26d9ddf85da2d5955ce8f9d338fd1bb036b1132 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\344\277\256\346\224\271\345\257\206\347\240\2012.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\344\277\256\346\224\271\345\257\206\347\240\2013.png" "b/docs/en/tools/cloud/pilotgo/figures/\344\277\256\346\224\271\345\257\206\347\240\2013.png" new file mode 100644 index 0000000000000000000000000000000000000000..b3ffd4507aab3a85b3ab8e775bc1ab4c1efcfda3 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\344\277\256\346\224\271\345\257\206\347\240\2013.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\344\277\256\346\224\271\350\212\202\347\202\2711.png" "b/docs/en/tools/cloud/pilotgo/figures/\344\277\256\346\224\271\350\212\202\347\202\2711.png" new file mode 100644 index 0000000000000000000000000000000000000000..4a127fafef22d62f326e38075173f53f244acfa7 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\344\277\256\346\224\271\350\212\202\347\202\2711.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\344\277\256\346\224\271\350\212\202\347\202\2712.png" "b/docs/en/tools/cloud/pilotgo/figures/\344\277\256\346\224\271\350\212\202\347\202\2712.png" new file mode 100644 index 0000000000000000000000000000000000000000..8a097306b1dbf7ce5c6cb14e9c84ff7f59079dfb Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\344\277\256\346\224\271\350\212\202\347\202\2712.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\344\277\256\346\224\271\350\212\202\347\202\2713.png" "b/docs/en/tools/cloud/pilotgo/figures/\344\277\256\346\224\271\350\212\202\347\202\2713.png" new file mode 100644 index 0000000000000000000000000000000000000000..1e517062c17505a2ec0905863934e5e0a5e47c36 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\344\277\256\346\224\271\350\212\202\347\202\2713.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\210\233\345\273\272\346\211\271\346\254\2411.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\210\233\345\273\272\346\211\271\346\254\2411.png" new file mode 100644 index 0000000000000000000000000000000000000000..ee14b990e8ab6cf0c71bef1a40cb74cd2919e2fc Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\210\233\345\273\272\346\211\271\346\254\2411.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\210\233\345\273\272\346\211\271\346\254\2412.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\210\233\345\273\272\346\211\271\346\254\2412.png" new file mode 100644 index 0000000000000000000000000000000000000000..1f5a1658552227a88cf07f592e048c4bc1005286 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\210\233\345\273\272\346\211\271\346\254\2412.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\210\233\345\273\272\346\211\271\346\254\2413.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\210\233\345\273\272\346\211\271\346\254\2413.png" new file mode 100644 index 0000000000000000000000000000000000000000..4066752952e177ca2bb14b61a86d44ff1efc11f6 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\210\233\345\273\272\346\211\271\346\254\2413.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\210\233\345\273\272\346\211\271\346\254\2414.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\210\233\345\273\272\346\211\271\346\254\2414.png" new file mode 100644 index 0000000000000000000000000000000000000000..ade3fb143ac6a0186985b63c5505afef9666e57e Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\210\233\345\273\272\346\211\271\346\254\2414.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\210\233\345\273\272\346\226\207\344\273\2661.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\210\233\345\273\272\346\226\207\344\273\2661.png" new file mode 100644 index 0000000000000000000000000000000000000000..74889505efa10bf45d699d9c8ec19c81cd63ef4f Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\210\233\345\273\272\346\226\207\344\273\2661.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\210\233\345\273\272\346\226\207\344\273\2662.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\210\233\345\273\272\346\226\207\344\273\2662.png" new file mode 100644 index 0000000000000000000000000000000000000000..0a0f563aa9efd21a789058b76dc88e5e0208a996 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\210\233\345\273\272\346\226\207\344\273\2662.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\210\233\345\273\272\346\226\207\344\273\2663.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\210\233\345\273\272\346\226\207\344\273\2663.png" new file mode 100644 index 0000000000000000000000000000000000000000..e7dfcf189d030a4bffa1ce92885e27e3fab7ecde Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\210\233\345\273\272\346\226\207\344\273\2663.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\346\211\271\346\254\2411.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\346\211\271\346\254\2411.png" new file mode 100644 index 0000000000000000000000000000000000000000..e360587420e42233933a9bb27ad31a62557374f0 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\346\211\271\346\254\2411.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\346\211\271\346\254\2412.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\346\211\271\346\254\2412.png" new file mode 100644 index 0000000000000000000000000000000000000000..0efb93e8dd16f855b444d6a5891be38fdebe92c7 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\346\211\271\346\254\2412.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\346\211\271\346\254\2413.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\346\211\271\346\254\2413.png" new file mode 100644 index 0000000000000000000000000000000000000000..2263d7c359bc58451f9382693b98c15cae4fb273 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\346\211\271\346\254\2413.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\346\234\272\345\231\2501.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\346\234\272\345\231\2501.png" new file mode 100644 index 0000000000000000000000000000000000000000..74c10a8dee0fb08e4ac39d73c3389b9a2262c143 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\346\234\272\345\231\2501.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\346\234\272\345\231\2502.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\346\234\272\345\231\2502.png" new file mode 100644 index 0000000000000000000000000000000000000000..d4e467dd0b6fbd9d13a928deebfa8cca1a515c61 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\346\234\272\345\231\2502.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\346\234\272\345\231\2503.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\346\234\272\345\231\2503.png" new file mode 100644 index 0000000000000000000000000000000000000000..1bb38a09498d5a0d8c96aef1ce7b39f8bbb43207 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\346\234\272\345\231\2503.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\347\224\250\346\210\2671.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\347\224\250\346\210\2671.png" new file mode 100644 index 0000000000000000000000000000000000000000..c0599cd9d3679c2c16debcbf46b85b1328130104 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\347\224\250\346\210\2671.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\347\224\250\346\210\2672.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\347\224\250\346\210\2672.png" new file mode 100644 index 0000000000000000000000000000000000000000..96a3636ed380608616fccb672017ef363108d529 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\347\224\250\346\210\2672.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\350\212\202\347\202\2712.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\350\212\202\347\202\2712.png" new file mode 100644 index 0000000000000000000000000000000000000000..e739b14f7b60794065a9ec8a9b2478b2f0b37dd0 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\350\212\202\347\202\2712.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\350\212\202\347\202\2713.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\350\212\202\347\202\2713.png" new file mode 100644 index 0000000000000000000000000000000000000000..d8c8967d525a68515a7ce651f7d30169654bd784 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\350\212\202\347\202\2713.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\350\247\222\350\211\2621.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\350\247\222\350\211\2621.png" new file mode 100644 index 0000000000000000000000000000000000000000..cf3d51f7ab12f241f8a93223631406d0c1b99ab4 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\350\247\222\350\211\2621.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\350\247\222\350\211\2622.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\350\247\222\350\211\2622.png" new file mode 100644 index 0000000000000000000000000000000000000000..b41055b466720578ca9282ff31589b6e147e8ada Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\350\247\222\350\211\2622.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\350\247\222\350\211\2623.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\350\247\222\350\211\2623.png" new file mode 100644 index 0000000000000000000000000000000000000000..661ed75def31a49cbf6043493c1805d65c83a83b Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\210\240\351\231\244\350\247\222\350\211\2623.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\212\237\350\203\275\346\250\241\345\235\227.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\212\237\350\203\275\346\250\241\345\235\227.png" new file mode 100644 index 0000000000000000000000000000000000000000..86782bfc46f42a051b56f457cd46fad60cad3332 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\212\237\350\203\275\346\250\241\345\235\227.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\217\230\346\233\264\351\203\250\351\227\2501.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\217\230\346\233\264\351\203\250\351\227\2501.png" new file mode 100644 index 0000000000000000000000000000000000000000..23c2d754679c0a374d89c26596669e9bbbebf2f6 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\217\230\346\233\264\351\203\250\351\227\2501.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\345\217\230\346\233\264\351\203\250\351\227\2502.png" "b/docs/en/tools/cloud/pilotgo/figures/\345\217\230\346\233\264\351\203\250\351\227\2502.png" new file mode 100644 index 0000000000000000000000000000000000000000..0efb1384611e7f5b4cb1370e626a238908567dbb Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\345\217\230\346\233\264\351\203\250\351\227\2502.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\211\271\351\207\217\344\270\213\345\217\2211.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\211\271\351\207\217\344\270\213\345\217\2211.png" new file mode 100644 index 0000000000000000000000000000000000000000..387df3d4cd301fe677e663c6a919abf093efba87 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\211\271\351\207\217\344\270\213\345\217\2211.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\211\271\351\207\217\344\270\213\345\217\2212.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\211\271\351\207\217\344\270\213\345\217\2212.png" new file mode 100644 index 0000000000000000000000000000000000000000..ca5e64cbf7d0aeabcececacea125585484e873ca Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\211\271\351\207\217\344\270\213\345\217\2212.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\211\271\351\207\217\345\215\270\350\275\2751.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\211\271\351\207\217\345\215\270\350\275\2751.png" new file mode 100644 index 0000000000000000000000000000000000000000..4bc4ca6f620619fe10a81205a939535f83e772c2 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\211\271\351\207\217\345\215\270\350\275\2751.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\211\271\351\207\217\345\215\270\350\275\2752.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\211\271\351\207\217\345\215\270\350\275\2752.png" new file mode 100644 index 0000000000000000000000000000000000000000..68467232ca5bd65a03eccc4fc3fb8a5e95529ddf Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\211\271\351\207\217\345\215\270\350\275\2752.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\211\271\351\207\217\346\223\215\344\275\2341.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\211\271\351\207\217\346\223\215\344\275\2341.png" new file mode 100644 index 0000000000000000000000000000000000000000..5cee721e3c0ce14f666a85cd3acb27b57684f077 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\211\271\351\207\217\346\223\215\344\275\2341.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\344\270\213\345\217\2211.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\344\270\213\345\217\2211.png" new file mode 100644 index 0000000000000000000000000000000000000000..d5d54a3679b9a183dbc8eddacf881a8c30c0967b Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\344\270\213\345\217\2211.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\344\270\213\345\217\2212.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\344\270\213\345\217\2212.png" new file mode 100644 index 0000000000000000000000000000000000000000..d639180465474d529758cb83e98b8bd44c409e47 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\344\270\213\345\217\2212.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\344\270\213\345\217\2213.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\344\270\213\345\217\2213.png" new file mode 100644 index 0000000000000000000000000000000000000000..87082b54be5d405f859bd17b558b061e08565f0c Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\344\270\213\345\217\2213.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\345\210\240\351\231\2443.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\345\210\240\351\231\2443.png" new file mode 100644 index 0000000000000000000000000000000000000000..a8f2dd996fb826ace2a657ae330aaf36d7c1b884 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\345\210\240\351\231\2443.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\345\216\206\345\217\262\347\211\210\346\234\254.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\345\216\206\345\217\262\347\211\210\346\234\254.png" new file mode 100644 index 0000000000000000000000000000000000000000..74f5e745836607702d69f97939b8629446dc0d71 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\345\216\206\345\217\262\347\211\210\346\234\254.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\345\233\236\346\273\2321.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\345\233\236\346\273\2321.png" new file mode 100644 index 0000000000000000000000000000000000000000..8a7e6dfd18608275d496de46cc157bdcfcc1ffa4 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\345\233\236\346\273\2321.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\345\233\236\346\273\2322.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\345\233\236\346\273\2322.png" new file mode 100644 index 0000000000000000000000000000000000000000..0ceef0dcacc27149d2feb2eff3a7902af1c13186 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\345\233\236\346\273\2322.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\345\233\236\346\273\2323.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\345\233\236\346\273\2323.png" new file mode 100644 index 0000000000000000000000000000000000000000..69b4cda58e7962c11e40bdac7555afb9428941b2 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\345\233\236\346\273\2323.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\345\233\236\346\273\2324.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\345\233\236\346\273\2324.png" new file mode 100644 index 0000000000000000000000000000000000000000..79281449c580ef3059dc30329416e6fb564fb5ae Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\226\207\344\273\266\345\233\236\346\273\2324.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\345\206\205\346\240\270\344\277\256\346\224\2711.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\345\206\205\346\240\270\344\277\256\346\224\2711.png" new file mode 100644 index 0000000000000000000000000000000000000000..ae23a49e9ef1d9c2be390a4715f83457c05dce69 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\345\206\205\346\240\270\344\277\256\346\224\2711.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\345\206\205\346\240\270\344\277\256\346\224\2712.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\345\206\205\346\240\270\344\277\256\346\224\2712.png" new file mode 100644 index 0000000000000000000000000000000000000000..344f95e052c876e312043099b36267e4e9544e5c Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\345\206\205\346\240\270\344\277\256\346\224\2712.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\345\206\205\346\240\270\344\277\256\346\224\2713.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\345\206\205\346\240\270\344\277\256\346\224\2713.png" new file mode 100644 index 0000000000000000000000000000000000000000..1f108d6f224f30a5973b4ddbe7e8d551d8e1f9c5 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\345\206\205\346\240\270\344\277\256\346\224\2713.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\346\234\215\345\212\241\345\201\234\346\255\242.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\346\234\215\345\212\241\345\201\234\346\255\242.png" new file mode 100644 index 0000000000000000000000000000000000000000..c482e8389f10bca2f1ad43545af169d6dd26b1a5 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\346\234\215\345\212\241\345\201\234\346\255\242.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\346\234\215\345\212\241\345\220\257\345\212\250.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\346\234\215\345\212\241\345\220\257\345\212\250.png" new file mode 100644 index 0000000000000000000000000000000000000000..3d8674a65895b1138ca2826b2496f17c81e5818b Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\346\234\215\345\212\241\345\220\257\345\212\250.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\346\234\215\345\212\241\351\207\215\345\220\257.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\346\234\215\345\212\241\351\207\215\345\220\257.png" new file mode 100644 index 0000000000000000000000000000000000000000..a77c72630b6ab284232f7584d4f688e243439960 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\346\234\215\345\212\241\351\207\215\345\220\257.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\347\273\210\347\253\257.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\347\273\210\347\253\257.png" new file mode 100644 index 0000000000000000000000000000000000000000..2a7e5cbb1366030517ceeacc8a1459a764ac98eb Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\347\273\210\347\253\257.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\347\273\210\347\253\2571.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\347\273\210\347\253\2571.png" new file mode 100644 index 0000000000000000000000000000000000000000..d3130734e2fb884c74209411dbb647d88e575a8f Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\347\273\210\347\253\2571.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\350\275\257\344\273\266\345\214\205\345\215\270\350\275\275.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\350\275\257\344\273\266\345\214\205\345\215\270\350\275\275.png" new file mode 100644 index 0000000000000000000000000000000000000000..cc74a97dcf92ca3eb57b8cf7b2319e73cf10c099 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\350\275\257\344\273\266\345\214\205\345\215\270\350\275\275.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\350\275\257\344\273\266\345\214\205\345\256\211\350\243\2052.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\350\275\257\344\273\266\345\214\205\345\256\211\350\243\2052.png" new file mode 100644 index 0000000000000000000000000000000000000000..b24a22cbafc042b7d4cb234708a161a4b6910048 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\234\272\345\231\250\350\275\257\344\273\266\345\214\205\345\256\211\350\243\2052.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\267\273\345\212\240\347\224\250\346\210\2671.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\267\273\345\212\240\347\224\250\346\210\2671.png" new file mode 100644 index 0000000000000000000000000000000000000000..e5f5631e6ca19f8498fa2b030613b0a75d7168f1 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\267\273\345\212\240\347\224\250\346\210\2671.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\267\273\345\212\240\347\224\250\346\210\2672.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\267\273\345\212\240\347\224\250\346\210\2672.png" new file mode 100644 index 0000000000000000000000000000000000000000..017c47fdc9974c3a9ee5758c05512eb0b01a929c Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\267\273\345\212\240\347\224\250\346\210\2672.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\267\273\345\212\240\350\247\222\350\211\2621.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\267\273\345\212\240\350\247\222\350\211\2621.png" new file mode 100644 index 0000000000000000000000000000000000000000..a51db5c136e8d6baf61187d8882d4b02758cb056 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\267\273\345\212\240\350\247\222\350\211\2621.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\346\267\273\345\212\240\350\247\222\350\211\2622.png" "b/docs/en/tools/cloud/pilotgo/figures/\346\267\273\345\212\240\350\247\222\350\211\2622.png" new file mode 100644 index 0000000000000000000000000000000000000000..a352b27353c2513f55cad32d968b1095de96eb23 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\346\267\273\345\212\240\350\247\222\350\211\2622.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\347\224\250\346\210\267\345\257\274\345\205\2451.png" "b/docs/en/tools/cloud/pilotgo/figures/\347\224\250\346\210\267\345\257\274\345\205\2451.png" new file mode 100644 index 0000000000000000000000000000000000000000..7b7c230d9942bd9fceaeb2fbb23b3e16255b2505 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\347\224\250\346\210\267\345\257\274\345\205\2451.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\347\224\250\346\210\267\345\257\274\345\205\2452.png" "b/docs/en/tools/cloud/pilotgo/figures/\347\224\250\346\210\267\345\257\274\345\205\2452.png" new file mode 100644 index 0000000000000000000000000000000000000000..dad2779f6ddb6577a636fe8fb6050aeec69ee2ad Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\347\224\250\346\210\267\345\257\274\345\205\2452.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\347\224\250\346\210\267\345\257\274\345\205\2453.png" "b/docs/en/tools/cloud/pilotgo/figures/\347\224\250\346\210\267\345\257\274\345\205\2453.png" new file mode 100644 index 0000000000000000000000000000000000000000..88d855f0e0f48d3da3523d59df9e2358fb49a92c Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\347\224\250\346\210\267\345\257\274\345\205\2453.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\347\224\250\346\210\267\345\257\274\345\207\2721.png" "b/docs/en/tools/cloud/pilotgo/figures/\347\224\250\346\210\267\345\257\274\345\207\2721.png" new file mode 100644 index 0000000000000000000000000000000000000000..6198f25e96b6f782e042a1e1c36b0bef897ca064 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\347\224\250\346\210\267\345\257\274\345\207\2721.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\347\224\250\346\210\267\345\257\274\345\207\2722.png" "b/docs/en/tools/cloud/pilotgo/figures/\347\224\250\346\210\267\345\257\274\345\207\2722.png" new file mode 100644 index 0000000000000000000000000000000000000000..c55645090a3475c117b2e5805b42bad57a90dfd0 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\347\224\250\346\210\267\345\257\274\345\207\2722.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\347\231\273\345\275\225.png" "b/docs/en/tools/cloud/pilotgo/figures/\347\231\273\345\275\225.png" new file mode 100644 index 0000000000000000000000000000000000000000..6eb0106de32bd3d9da30d194035f129e3083791a Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\347\231\273\345\275\225.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\346\211\271\346\254\2411.png" "b/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\346\211\271\346\254\2411.png" new file mode 100644 index 0000000000000000000000000000000000000000..068b66d65a0f63fabd9f4cd78b46aafbbd1eb8b7 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\346\211\271\346\254\2411.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\346\211\271\346\254\2413.png" "b/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\346\211\271\346\254\2413.png" new file mode 100644 index 0000000000000000000000000000000000000000..a469a8798beecb882e5823132f442ee1eaf5cb21 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\346\211\271\346\254\2413.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\346\226\207\344\273\2661.png" "b/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\346\226\207\344\273\2661.png" new file mode 100644 index 0000000000000000000000000000000000000000..50b5f27cc9cecee17b7758683f61bf21544e8c3b Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\346\226\207\344\273\2661.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\346\226\207\344\273\2662.png" "b/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\346\226\207\344\273\2662.png" new file mode 100644 index 0000000000000000000000000000000000000000..1362aac595643c19f924cf92098bf43abf75c78e Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\346\226\207\344\273\2662.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\346\226\207\344\273\2663.png" "b/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\346\226\207\344\273\2663.png" new file mode 100644 index 0000000000000000000000000000000000000000..ffa2ed188539c7aa0f95cd6beb21d07c0ed6fc84 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\346\226\207\344\273\2663.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\347\224\250\346\210\2671.png" "b/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\347\224\250\346\210\2671.png" new file mode 100644 index 0000000000000000000000000000000000000000..36cdb73c8cffc40e7e9d6831691183cdfb481649 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\347\224\250\346\210\2671.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\347\224\250\346\210\2672.png" "b/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\347\224\250\346\210\2672.png" new file mode 100644 index 0000000000000000000000000000000000000000..7391fda93795f334f7674c98c811bf93919e99a0 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\347\224\250\346\210\2672.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\350\247\222\350\211\2621.png" "b/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\350\247\222\350\211\2621.png" new file mode 100644 index 0000000000000000000000000000000000000000..d752d16e201a493d71feee178f6a9ca4541df5ed Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\350\247\222\350\211\2621.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\350\247\222\350\211\2622.png" "b/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\350\247\222\350\211\2622.png" new file mode 100644 index 0000000000000000000000000000000000000000..25c650b0393a73ba5b40f3409a760e420881dcfe Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\347\274\226\350\276\221\350\247\222\350\211\2622.png" differ diff --git "a/docs/en/tools/cloud/pilotgo/figures/\351\207\215\347\275\256\345\257\206\347\240\2011.png" "b/docs/en/tools/cloud/pilotgo/figures/\351\207\215\347\275\256\345\257\206\347\240\2011.png" new file mode 100644 index 0000000000000000000000000000000000000000..0f33a7a9476814caf942edb428b55a8aa31e3d91 Binary files /dev/null and "b/docs/en/tools/cloud/pilotgo/figures/\351\207\215\347\275\256\345\257\206\347\240\2011.png" differ diff --git a/docs/en/tools/cloud/pilotgo/pilotgo-introduction.md b/docs/en/tools/cloud/pilotgo/pilotgo-introduction.md new file mode 100644 index 0000000000000000000000000000000000000000..d1c8fd2b0b77879ac8762fadb08a2a9487151931 --- /dev/null +++ b/docs/en/tools/cloud/pilotgo/pilotgo-introduction.md @@ -0,0 +1,37 @@ +# PilotGo介绍 + +PilotGo 是 openEuler 社区原生孵化的运维管理平台，采用插件式架构设计，功能模块轻量化组合、独立迭代演进，同时保证核心功能稳定；同时使用插件来增强平台功能、并打通不同运维组件之间的壁垒，实现了全局的状态感知及自动化流程。 + +## 功能描述 + +PilotGo 核心功能模块包括： + +* 用户管理：支持按照组织结构分组管理，支持导入已有平台账号，迁移方便； + +* 权限管理：支持基于RBAC的权限管理，灵活可靠； + +* 主机管理：状态前端可视化、直接执行软件包管理、服务管理、内核参数调优、简单易操作； + +* 批次管理：支持运维操作并发执行，稳定高效； + +* 日志审计：跟踪记录用户及插件的变更操作，方便问题回溯及安全审计； + +* 告警管理：平台异常实时感知； + +* 插件功能：支持扩展平台功能，插件联动，自动化能力倍增，减少人工干预。 + +![本地路径](./figures/功能模块.png) + +当前OS发布版本还集成了以下插件： + +* Prometheus：托管Prometheus监控组件，自动化下发及配置node-exporter监控数据采集，对接平台告警功能； + +![本地路径](./figures/P插件3.png) + +* Grafana：集成Grafana可视化平台，提供美观易用的指标监控面板功能。 + +![本地路径](./figures/G插件4.png) + +## 应用场景 + +PiotGo可用于典型的服务器集群管理场景，支持大批量的服务器集群基本管理及监控；通过集成对应的业务功能插件，还可实现业务集群的统一平台管理，例如Mysql数据库集群、redis数据缓存集群、nginx网关集群等。 diff --git a/docs/en/tools/cloud/pilotgo/usage-instructions.md b/docs/en/tools/cloud/pilotgo/usage-instructions.md new file mode 100644 index 0000000000000000000000000000000000000000..dc6369e3ceb6dd098fb670bc89541eba6a5feb0c --- /dev/null +++ b/docs/en/tools/cloud/pilotgo/usage-instructions.md @@ -0,0 +1,327 @@ +# PilotGo平台使用手册 + +PilotGo 是一个 openEuler 社区原生的运维管理平台，采用插件式开发，增强平台的扩展性、并打通不同运维组件之间的壁垒。PilotGo 核心功能包括：集群管理、批次管理、主机管理、用户管理、权限管理、主机监控、运维审计等。 + +## 1 PilotGo安装与配置 + +PilotGo可以单机部署也可以采用集群式部署。安装之前先关闭防火墙。 + +### 1.1 PilotGo-server 安装与配置 + +安装mysql； +安装redis，设置redis密码（修改）,运行命令： + +`dnf install redis6` + +`vim /etc/redis/redis.conf` + +`#requirepass foobared去掉注释，foobared改为自己的密码` + +`bind 0.0.0.0` + +启动MySQL和redis服务，然后执行： + +`dnf install PilotGo-server` + +修改/opt/PilotGo/server/config_server.yaml里面mysql和redis的配置信息，启动服务： + +`systemctl start PilotGo-server` + +访问页面： + +### 1.2 PilotGo-agent安装与配置 + +执行以下命令进行安装： + +`dnf install PilotGo-agent` + +修改/opt/PilotGo/agent/config_agent.yaml里面的ip信息，启动服务： + +`systemctl start PilotGo-agent` + +### 1.3 PilotGo插件安装与配置 + +详情见3 插件使用手册 + +## 2 PilotGo平台使用说明 + +### 2.1 首次登录 + +#### 2.1.1 用户登录页面 + +用户登录页面如图所示，输入正确的用户名和密码登录系统。默认用户名为，默认密码为admin，首次登录之后建议先修改密码。![本地路径](./figures/登录.png) + +### 2.2 用户模块 + +#### 2.2.1 创建用户 + +创建用户的方式又两种，一种是手动创建单个用户，另外一种是批量导入多个用户。 + +##### 2.2.1.1 创建单个用户 + +1. 具有创建用户权限的用户成功登录之后点击左侧导航栏中的用户管理； +2. 点击页面右上角的添加按钮； +3. 在页面中输入用户名、密码、邮箱，选择部门和角色类型，并点击确定按钮；![本地路径](./figures/添加用户1.png) +4. 页面弹框提示“添加用户成功”，并显示新创建的用户信息，表示创建用户成功。![本地路径](./figures/添加用户2.png) + +##### 2.2.1.2 批量导入多个用户 + +1. 具有创建用户权限的用户成功登录之后点击左侧导航栏中的用户管理； +2. 击页面的批量导入按钮，选择文件点击打开按钮；![本地路径](./figures/用户导入1.png) +3. 显示用户信息则完成用户导入。![本地路径](./figures/用户导入2.png)![本地路径](./figures/用户导入3.png) + +#### 2.2.2 修改用户信息及密码 + +##### 2.2.2.1 修改用户信息 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的用户管理； +2. 找到用户信息，点击操作栏中的编辑按钮； +3. 在页面中输入要修改的用户信息，并点击确定按钮；![本地路径](./figures/编辑用户1.png) +4. 页面弹框提示“用户信息修改成功”，并显示修改后的用户信息。![本地路径](./figures/编辑用户2.png) + +##### 2.2.2.2 修改密码 + +修改密码有两种方式，第一是用户知道密码登录系统后自己修改，第二是用户忘记密码，由管理员登录系统后重置此用户密码，重置默认密码为邮箱@符号的前半部分。 + +###### 2.2.2.2.1 手动修改密码 + +1. 用户登录系统后点击右上角的人像图标和修改密码；![本地路径](./figures/修改密码1.png) +2. 连续输入两次新密码，点击确定按钮；![本地路径](./figures/修改密码2.png) +3. 页面弹框提示“修改成功”。![本地路径](./figures/修改密码3.png) + +###### 2.2.2.2.2 重置密码 + +1. 管理员登录成功后点击左侧导航栏中的用户管理； +2. 找到用户信息，点击操作栏中的重置密码按钮； +3. 用户使用默认密码可以登录系统。![本地路径](./figures/重置密码1.png) + +#### 2.2.3 删除用户 + +1. 管理员登录成功后点击左侧导航栏中的用户管理； +2. 点击页面小方块选择要删除的用户； +3. 点击页面右上角的删除按钮，并点击确定；![本地路径](./figures/删除用户1.png) +4. 页面弹框提示“用户删除成功”，并用户管理页面不显示删除用户的信息。![本地路径](./figures/删除用户2.png) + +#### 2.2.4 导出用户 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的用户管理； +2. 点击页面的导出按钮；![本地路径](./figures/用户导出1.png) +3. 浏览器显示下载进度，成功下载后打开xlsx文件查看信息。![本地路径](./figures/用户导出2.png) + +### 2.3 角色模块 + +#### 2.3.1 添加角色 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的角色管理； +2. 点击页面的添加按钮； +3. 输入角色名和描述信息，并点击确定按钮；![本地路径](./figures/添加角色1.png) +4. 页面弹框提示“新增角色成功”，并页面显示新添加的角色信息。![本地路径](./figures/添加角色2.png) + +### 2.3.2 修改角色 + +#### 2.3.2.1 修改角色信息 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的角色管理； +2. 点击对应角色的编辑按钮； +3. 输入新的角色名和描述信息，并点击确定按钮；![本地路径](./figures/添加角色1.png) +4. 页面弹框提示“角色信息修改成功”，并页面显示修改后的角色信息。![本地路径](./figures/编辑角色2.png) + +#### 2.3.2.2 修改角色权限 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的角色管理； +2. 点击对应角色的变更按钮； +3. 选择相应的权限，点击重置按钮可以清空所选权限，并点击确定按钮；![本地路径](./figures/编辑角色1.png) +4. 页面弹框提示“角色权限变更成功”。![本地路径](./figures/编辑角色2.png) + +### 2.3.3 删除角色 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的角色管理； +2. 点击对应角色的删除按钮，并点击确定；![本地路径](./figures/删除角色1.png)![本地路径](./figures/删除角色2.png) +3. 页面弹框提示“角色删除成功”，并不显示删除的角色信息。![本地路径](./figures/删除角色3.png) + +### 2.4 部门树模块 + +#### 2.4.1 修改部门节点 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的系统和机器列表； +2. 在部门节点对应位置点击修改符号，输入节点名字并点击确定；![本地路径](./figures/修改节点1.png)![本地路径](./figures/修改节点2.png) +3. 页面弹框提示“修改成功”，并显示修改后的部门节点信息。![本地路径](./figures/修改节点3.png) + +#### 2.4.2 删除部门节点 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的系统和机器列表； +2. 在部门节点对应位置点击删除符号并点击确定；![本地路径](./figures/修改节点1.png)![本地路径](./figures/删除节点2.png) +3. 页面弹框提示“删除成功”，并不显示删除节点的信息。![本地路径](./figures/删除节点3.png) + +### 2.5 配置库模块 + +#### 2.5.1 添加 repo 配置文件 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的库配置文件； +2. 点击页面的新增按钮；![本地路径](./figures/创建文件1.png) +3. 输入文件名、文件类型、文件路径、描述和内容等信息，文件名必须以.repo结尾，文件路径必须正确，文件内容要符合repo文件的格式，并点击确定按钮；![本地路径](./figures/创建文件2.png) +4. 页面弹框提示“文件保存成功”；并显示新增的repo配置文件信息。![本地路径](./figures/创建文件3.png) + +#### 2.5.2 修改 repo 配置文件 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的库配置文件； +2. 找到要修改的repo文件，点击对应的编辑按钮；![本地路径](./figures/编辑文件1.png) +3. 输入修改后的文件名、文件类型、文件路径、描述和内容等信息，并点击确定按钮；![本地路径](./figures/编辑文件2.png) +4. 页面弹框提示“配置文件修改成功”；并显示修改后的repo配置文件信息。![本地路径](./figures/编辑文件3.png) + +#### 2.5.3 删除 repo 配置文件 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的库配置文件； +2. 选择要删除的文件，点击页面的删除按钮，并点击确定；![本地路径](./figures/删除角色1.png)![本地路径](./figures/删除角色2.png) +3. 页面弹框提示“存储的文件已从数据库删除”，且页面不显示删除的repo配置文件信息。![本地路径](./figures/文件删除3.png) + +#### 2.5.4 下发 repo 配置文件 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的库配置文件； +2. 找到要下发的文件，点击页面的下发按钮，选择要下发的批次，并点击确定；![本地路径](./figures/文件下发1.png)![本地路径](./figures/文件下发2.png) +3. 页面弹框提示“配置文件下发成功”。![本地路径](./figures/文件下发3.png) + +#### 2.5.5 回滚 repo 配置文件历史版本 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的库配置文件； +2. 找到要回滚的文件，点击页面的历史版本按钮；![本地路径](./figures/文件历史版本.png) +3. 选择要回滚的版本，点击回滚按钮并点击确定；![本地路径](./figures/文件回滚1.png)![本地路径](./figures/文件回滚2.png) +4. 页面弹框提示“已回退到历史版本”，历史版本页面增加一条“-latest”记录。![本地路径](./figures/文件回滚3.png)![本地路径](./figures/文件回滚4.png) + +### 2.6 批次模块 + +#### 2.6.1 创建批次 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的系统和创建批次； +2. 点击机器所在的部门名字，在备选项中选择0个或多个机器ip(点击ip前面的方框)，若选择一个或多个部门的所有机器可以点击部门列表的方框，并点击备选项中的部门名称，选择完成后点击向右的箭头；![本地路径](./figures/创建批次1.png) +3. 输入批次名称和描述，并点击创建按钮；![本地路径](./figures/创建批次2.png) +4. 页面弹框提示“批次入库成功”，并批次页面显示新创建的批次信息。![本地路径](./figures/创建批次3.png)![本地路径](./figures/创建批次4.png) + +#### 2.6.2 修改批次 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的批次； +2. 点击对应批次的编辑按钮；![本地路径](./figures/编辑批次1.png) +3. 输入新的批次名称和备注信息，并点击确定按钮；![本地路径](./figures/编辑文件2.png) +4. 页面弹框提示“批次修改成功”，并显示修改后的批次信息。![本地路径](./figures/编辑批次3.png) + +#### 2.6.3 删除批次 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的批次； +2. 选择要删除的批次，点击删除按钮并点击确定；![本地路径](./figures/删除批次1.png)![本地路径](./figures/删除批次2.png) +3. 页面弹框提示“批次删除成功”，并不显示删除批次的信息。![本地路径](./figures/删除批次3.png) + +#### 2.6.4 批量安装软件包 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的批次，并点击批次名称；![本地路径](./figures/批量操作1.png) +2. 点击右上角的rpm下发按钮，在搜索框输入软件包的名称，并点击下发按钮；![本地路径](./figures/批量下发1.png) +3. 页面弹框提示“软件包安装成功”，agent端可以查到下发的rpm包。![本地路径](./figures/批量下发2.png) + +#### 2.6.5 批量卸载软件包 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的批次，并点击批次名称；![本地路径](./figures/批量操作1.png) +2. 点击右上角的rpm卸载按钮，在搜索框输入软件包的名称，并点击卸载按钮；![本地路径](./figures/批量卸载1.png) +3. 页面弹框提示“软件包卸载成功”，agent端无此软件包。![本地路径](./figures/批量卸载2.png) + +### 2.7 机器模块 + +#### 2.7.1 删除机器 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的系统和机器列表； +2. 选择要删除的机器，点击删除按钮并点击确定；![本地路径](./figures/删除机器1.png)![本地路径](./figures/删除机器2.png) +3. 页面弹框提示“机器删除成功”，并不显示删除机器的信息。![本地路径](./figures/删除机器3.png) + +#### 2.7.2 变更机器部门 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的系统和机器列表； +2. 选择要变更部门的机器，点击变更部门按钮； +3. 核对变更部门机器ip的信息，选择新的部门，并点击确定；![本地路径](./figures/变更部门1.png) +4. 页面弹框提示“机器部门修改成功”，并显示变更后的信息。![本地路径](./figures/变更部门2.png) + +#### 2.7.3 修改机器内核参数 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的系统和机器列表； +2. 点击要查看信息的机器ip，并点击内核参数信息栏目；![本地路径](./figures/机器内核修改1.png) +3. 输入要查找的内核，点击修改，输入参数值并点击确定；![本地路径](./figures/机器内核修改2.png) +4. 页面显示修改进度，成功后显示100%。![本地路径](./figures/机器内核修改3.png) + +#### 2.7.4 启动机器服务 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的系统和机器列表； +2. 点击要查看信息的机器ip，并点击服务信息栏目； +3. 在搜索框输入要启动的服务名称，并点击启动按钮； +4. 页面显示软件包名、执行动作、执行结果进度条信息。![本地路径](./figures/机器服务启动.png) + +#### 2.7.5 重启机器服务 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的系统和机器列表； +2. 点击要查看信息的机器ip，并点击服务信息栏目； +3. 在搜索框输入要重启的服务名称，并点击重启按钮； +4. 页面显示软件包名、执行动作、执行结果进度条信息。![本地路径](./figures/机器服务重启.png) + +#### 2.7.6 停止机器服务 + +1. 具有该权限的用户成功登录，点击左侧导航栏中的系统和机器列表； +2. 点击要查看信息的机器ip，并点击服务信息栏目； +3. 在搜索框输入要启动的服务名称，并点击停止按钮； +4. 页面显示软件包名、执行动作、执行结果进度条信息。![本地路径](./figures/机器服务停止.png) + +#### 2.7.7 安装软件包 + +1. 有该权限的用户成功登录，点击左侧导航栏中的系统和机器列表； +2. 点击要查看信息的机器ip，并点击软件包信息栏目； +3. 在搜索框输入软件包的名称，并点击安装按钮； +4. 页面显示repo名称、repo地址信息，并页面显示软件包名、执行动作、结果等信息。![本地路径](./figures/机器软件包安装2.png) + +#### 2.7.8 卸载软件包 + +1. 有该权限的用户成功登录，点击左侧导航栏中的系统和机器列表； +2. 点击要查看信息的机器ip，并点击软件包信息栏目； +3. 在搜索框输入软件包的名称，并点击卸载按钮； +4. 页面显示repo名称、repo地址信息，并页面显示软件包名、执行动作、结果等信息。![本地路径](./figures/机器软件包卸载.png) + +#### 2.7.9 连接机器终端 + +1. 有该权限的用户成功登录，点击左侧导航栏中的系统和机器列表； +2. 点击要查看信息的机器ip，并点击终端信息栏目； +3. 输入ip地址和机器密码，点击连接按钮；![本地路径](./figures/机器终端1.png) +4. 页面显示终端窗口。![本地路径](./figures/机器终端.png) + +## 3 PilotGo平台插件使用说明 + +### 3.1 Grafana插件使用说明 + +1. 在任意一台服务器上执行dnf install PilotGo-plugin-grafana grafana； +2. 将/opt/PilotGo/plugin/grafana/config.yaml文件中ip地址修改为本机真实ip，修改/etc/grafana/grafana.ini文件一下信息： + + ```shell + root_url = http://真实ip:9999/plugin/grafana + + serve_from_sub_path = true + + allow_embedding = true + ``` + +3. 重启两个服务，执行以下命令： + + ```shell + systemctl restart grafana-server + + systemctl start PilotGo-plugin-grafana + ``` + +4. 成功登录pilotgo平台，点击左侧导航栏中的插件管理，点击添加插件按钮，填写插件名称和服务地址，并点击确定；![本地路径](./figures/G插件1.png) +5. 页面增加一条插件管理数据，导航栏增加一个插件按钮。![本地路径](./figures/G插件2.png)![本地路径](./figures/G插件3.png) + +### 3.2 Prometheus插件使用说明 + +1. 在任意一台服务器上执行dnf install PilotGo-plugin-prometheus； +2. 将/opt/PilotGo/plugin/prometheus/server/config.yml文件中ip地址修改为本机真实ip和mysql服务地址； +3. 重启服务，执行以下命令： + + ```shell + systemctl start PilotGo-plugin-prometheusX + ``` + +4. 成功登录pilotgo平台，点击左侧导航栏中的插件管理，点击添加插件按钮，填写插件名称和服务地址，并点击确定；![本地路径](./figures/P插件1.png) +5. 页面增加一条插件管理数据，导航栏增加一个插件按钮。![本地路径](./figures/P插件2.png)![本地路径](./figures/P插件3.png) +6. 在页面选择机器ip和监控时间，展示机器数据面板。![本地路径](./figures/P插件4.png) diff --git a/docs/en/tools/community_tools/_toc.yaml b/docs/en/tools/community_tools/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..d5a6c3a66c9b8b9f923d3ce5d3c86f1994767446 --- /dev/null +++ b/docs/en/tools/community_tools/_toc.yaml @@ -0,0 +1,23 @@ +label: Community Tools +sections: + - label: Image Creation + sections: + - href: ./image_custom/isocut/_toc.yaml + - href: ./image_custom/image_tailor/_toc.yaml + - label: GCC User Guide + sections: + - href: ../../server/development/gcc/_toc.yaml + - label: Performance Optimization + sections: + - href: ../../server/performance/system_optimization/atune/_toc.yaml + - href: ../../server/performance/tuning_framework/oeaware/_toc.yaml + - label: Migration + sections: + - href: ./migration/migration_tools/_toc.yaml + - label: Virtualization + sections: + - href: ./virtualization/euler_launcher/_toc.yaml + - label: Package Management + sections: + - href: ./epkg/epkg_use/_toc.yaml + - href: ./epkg/autopkg/_toc.yaml diff --git a/docs/en/tools/community_tools/epkg/autopkg/_toc.yaml b/docs/en/tools/community_tools/epkg/autopkg/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..866aab085d15ad39aa2f58f6990a1ce6fce7b7a8 --- /dev/null +++ b/docs/en/tools/community_tools/epkg/autopkg/_toc.yaml @@ -0,0 +1,6 @@ +label: autopkg User Guide +isManual: true +href: ./autopkg.md +description: >- + Streamlined process of packaging software source code into compatible packages + for openEuler diff --git a/docs/en/tools/community_tools/epkg/autopkg/autopkg.md b/docs/en/tools/community_tools/epkg/autopkg/autopkg.md new file mode 100644 index 0000000000000000000000000000000000000000..6a0baa0b86cc570ab289e29f1fd1eb1270f040e8 --- /dev/null +++ b/docs/en/tools/community_tools/epkg/autopkg/autopkg.md @@ -0,0 +1,166 @@ +# Overview + +This software streamlines package integration for the openEuler community by automating the bulk import of source code repositories from public platforms like GitHub. It detects package dependencies and generates binary files automatically, eliminating the need for manual package writing and maintenance. With built-in support for cMake, autotools, Meson, Maven, and Python build systems, the software substantially increases the success rate of end-to-end package integration. + +## Installation and Uninstallation + +### 1. Installation + +Download the source code from the repository. + +```bash +git clone https://gitee.com/qiu-tangke/autopkg.git -b ${branch} +``` + +Navigate to the repository directory and install the software using `pip`. This is compatible with openeuler 22.03 LTS and newer versions. For other versions, ensure that a Python 3.8 or higher environment is available. + +```bash +pip install dist/autopkg-***-py3-none-any.whl +``` + +### 2. Uninstallation + +```bash +pip uninstall autopkg +``` + +## Quick Start + +### 1. Environment Preparation + +The software must run on the host machine and requires Docker container support. Prepare a Docker image of the openEuler OS using the following methods. + +#### Method 1: Direct Image Acquisition from Source Repository + +```bash +arch=$(uname -m) +if [ "$arch" == "aarch64" ]; then + wget https://cache-openeuler.obs.cn-north-4.myhuaweicloud.com/52f2b17e15ceeefecf5646d7711df7e94691ea1adb11884b926532ae52ab3c22/autopkg-latest_aarch64.tar.xz + docker load < autopkg-latest_aarch64.tar.xz +elif [ "$arch" == "x86_64" ]; then + wget https://cache-openeuler.obs.cn-north-4.myhuaweicloud.com/710a5f18188efc70bfa0119d0b35dcbb62cab911c9eb77b86dc6aebdbbfc69de/autopkg-latest_x86-64.tar.xz + docker load < autopkg-latest_x86-64.tar.xz +else + echo "Error: The system architecture is neither aarch64 nor x86_64, it is $arch." +fi +``` + +#### Method 2: Manual Image Construction via Commands (in Case Method 1 Fails) + +```bash +arch=$(uname -m) +wget "https://repo.huaweicloud.com/openeuler/openEuler-23.03/docker_img/${arch}/openEuler-docker.${arch}.tar.xz" +docker load < "openEuler-docker.${arch}.tar.xz" +docker run -dti --privileged --name=autopkg_working --network=host openEuler-23.03:latest +docker exec -ti ${container_id} bash # The following commands are executed inside the container. +yum install -y git make gcc cmake python3-pip ruby ruby-devel rubygems-devel npm maven automake perl wget curl meson +cat >> /root/phase.sh << EOF +#/usr/bin/env bash + +prep +build +install +EOF +exit # Exit the container. +docker commit ${container_id} > autopkg:latest # Save the container modifications. +docker tag ${new_image_id} autopkg:latest # Name and tag the image. +``` + +### 2. Command Line + +```bash +autopkg --help +-g,--git-url: Provide the git repository URL, for example, 'https://***.git'. +-t,--tar-url: Provide the tar package URL. +-d,--dir: Specify the local repository path. +-n,--name: Specify the package name, used for interface request information. +-v,--version: Specify the version, used with "-n". +-l,--language: Specify the language, used with "-n". +-o,--output: Set the output file path. +-b,--build: Enable debug log mode. +-c,--config: Provide directly usable configuration information. +``` + +### 3. Common Commands + +#### A. Specifying Local Repository Path + +```bash +autopkg -d ${package_dir} -o ${output_path} +``` + +![](./images/dir_test.PNG) + +#### B. Specifying Source Package URL + +```bash +autopkg -t ${tar_url} -o ${output_path} +``` + +![](./images/tar_url_test.PNG) + +#### C. Specifying Package Name without Compilation + +```bash +autopkg -n ${name} -v ${version} -l ${language} -o ${output_path} +``` + +![](./images/name_test.PNG) + +## Output File Description + +Upon successful package compilation, the system generates **package.yaml**, **phase.sh**, and **{package_name}.epkg**. If compilation is skipped, only **package.yaml** and **phase.sh** are produced. The output path is determined by the `--output` parameter, defaulting to **/tmp/autopkg/output**. + +### 1. package.yaml (Example: Jekyll, Ruby Compilation) + +Basic information of the package + +```yaml +meta: + summary: No detailed summary available + description: | + # [Jekyll](https://jekyllrb.com/) +name: jekyll +version: 4.3.3 +homepage: https://localhost:8080/jekyll-0.0.1.tar.gz +license: MIT +source: + '0': https://localhost:8080/jekyll-0.0.1.tar.gz # For local repositories, the URL is simulated by the local service +release: 0 +buildRequires: +- ruby +- ruby-devel +- rubygems-devel +``` + +### 2. phase.sh (Example: Jekyll, Ruby Compilation) + +Build script for the package + +```bash +#!/usr/bin/env bash + +prep() { + cd /root/workspace +} + +build() { + if [ -f *.gemspec ]; then + gem build *.gemspec + fi + mkdir -p usr/ + gem install -V --local --build-root usr --force --document=ri,rdoc *.gem +} + +install() { + rm -rf /opt/buildroot + mkdir /opt/buildroot + cp -r usr/ /opt/buildroot +} +``` + +### 3. ***.epkg + +Installation package of the software. + +![](./images/local_epkg.PNG) diff --git a/docs/en/tools/community_tools/epkg/autopkg/images/dir_test.PNG b/docs/en/tools/community_tools/epkg/autopkg/images/dir_test.PNG new file mode 100644 index 0000000000000000000000000000000000000000..3d223e1c3f7aca150b724746b43a931f77e6c6d9 Binary files /dev/null and b/docs/en/tools/community_tools/epkg/autopkg/images/dir_test.PNG differ diff --git a/docs/en/tools/community_tools/epkg/autopkg/images/local_epkg.PNG b/docs/en/tools/community_tools/epkg/autopkg/images/local_epkg.PNG new file mode 100644 index 0000000000000000000000000000000000000000..7f5ecdf2948a661ab2d513f008ae72758706fe28 Binary files /dev/null and b/docs/en/tools/community_tools/epkg/autopkg/images/local_epkg.PNG differ diff --git a/docs/en/tools/community_tools/epkg/autopkg/images/name_test.PNG b/docs/en/tools/community_tools/epkg/autopkg/images/name_test.PNG new file mode 100644 index 0000000000000000000000000000000000000000..95e5cbfc916919e0996172cd4c55433c55eed4c2 Binary files /dev/null and b/docs/en/tools/community_tools/epkg/autopkg/images/name_test.PNG differ diff --git a/docs/en/tools/community_tools/epkg/autopkg/images/tar_url_test.PNG b/docs/en/tools/community_tools/epkg/autopkg/images/tar_url_test.PNG new file mode 100644 index 0000000000000000000000000000000000000000..80a49875c4eacbe74da42a859d7d9df251533fb8 Binary files /dev/null and b/docs/en/tools/community_tools/epkg/autopkg/images/tar_url_test.PNG differ diff --git a/docs/en/tools/community_tools/epkg/epkg_use/_toc.yaml b/docs/en/tools/community_tools/epkg/epkg_use/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..e76bf9a526352bdad88fd3042542e0e903f2f955 --- /dev/null +++ b/docs/en/tools/community_tools/epkg/epkg_use/_toc.yaml @@ -0,0 +1,4 @@ +label: epkg User Guide +isManual: true +href: ./epkg-package-manager-usage-guidemd +description: epkg for package management diff --git a/docs/en/tools/community_tools/epkg/epkg_use/epkg-package-manager-usage-guide.md b/docs/en/tools/community_tools/epkg/epkg_use/epkg-package-manager-usage-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..d88d3e2949955d7fb4be45cf9389bb966ea70394 --- /dev/null +++ b/docs/en/tools/community_tools/epkg/epkg_use/epkg-package-manager-usage-guide.md @@ -0,0 +1,320 @@ +# epkg User Guide + +## Introduction + +This document explains how to initialize the working environment for the epkg package manager and how to use its basic features. All operation results in this document are demonstrated using a non-root user as an example. +Note: Currently, epkg packages are only compatible with the AArch64 architecture, and support for other architectures will be expanded in the future. + +## Quick Start + +The following examples demonstrate how to install different versions of software packages. + +```bash +# Install epkg using curl. +# During installation, you can choose between user/global installation modes to install epkg for the current user or all users. +# Only the root user can use the global installation mode. +wget https://repo.oepkgs.net/openeuler/epkg/rootfs/epkg-installer.sh +sh epkg-installer.sh + +# Uninstall epkg. +wget https://repo.oepkgs.net/openeuler/epkg/rootfs/epkg-uninstaller.sh +sh epkg-uninstaller.sh + +# Initialize epkg. +epkg init +bash // Re-execute .bashrc to update the PATH + +# Create environment 1. +epkg env create t1 +epkg install tree +tree --version +which tree + +# View repositories. +[root@vm-4p64g ~]# epkg repo list +------------------------------------------------------------------------------------------------------------------------------------------------------ +channel | repo | url +------------------------------------------------------------------------------------------------------------------------------------------------------ +openEuler-22.03-LTS-SP3 | OS | https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-22.03-LTS-SP3/OS/aarch64/ +openEuler-24.09 | everything | https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-24.09/everything/aarch64/ +openEuler-24.09 | OS | https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-24.09/OS/aarch64/ +------------------------------------------------------------------------------------------------------------------------------------------------------ + +# Create environment 2, specify a repository. +epkg env create t2 --repo openEuler-22.03-LTS-SP3 +epkg install tree +tree --version +which tree + +# Switch back to environment 1. +epkg env activate t1 +``` + +## epkg Usage + +```bash +Usage: + epkg install PACKAGE + epkg install [--env ENV] PACKAGE (under development) + epkg remove [--env ENV] PACKAGE (under development) + epkg upgrade [PACKAGE] (under development) + + epkg search PACKAGE (under development) + epkg list (under development) + + epkg env list + epkg env create|remove ENV + epkg env activate ENV + epkg env deactivate ENV + epkg env register|unregister ENV + epkg env history ENV (under development) + epkg env rollback ENV (under development) +``` + +Package installation: + +```bash +epkg env create $env # Create an environment. +epkg install $package # Install a package in the environment. +epkg env create $env2 --repo $repo # Create environment 2, specify a repository. +epkg install $package # Install a package in environment 2. +``` + +Package building: + +```bash +epkg build ${yaml_path}/$pkg_name.yaml +``` + +### Installing Software + +Function description: + +Install software in the current environment (confirm the current environment before operation). + +Command: + +```shell +epkg install ${package_name} +``` + +Example output: + +```shell +[root@2d785c36ee2e /]# epkg env activate t1 +Add common to path +Add t1 to path +Environment 't1' activated. +Environment 't1' activated. +[root@2d785c36ee2e /]# epkg install tree +EPKG_ENV_NAME: t1 +Caching repodata for: "OS" +Cache for "OS" already exists. Skipping... +Caching repodata for: "OS" +Cache for "OS" already exists. Skipping... +Caching repodata for: "everything" +Cache for "everything" already exists. Skipping... +start download https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-24.09/everything/aarch64//store/FF/FFCRTKRFGFQ6S2YVLOSUF6PHSMRP7A2N__ncurses-libs__6.4__8.oe2409.epkg +############################################################################################################################################################################################################### 100.0% +start download https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-24.09/everything/aarch64//store/D5/D5BOEFTRBNV3E4EXBVXDSRNTIGLGWVB7__glibc-all-langpacks__2.38__34.oe2409.epkg +############################################################################################################################################################################################################### 100.0% +start download https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-24.09/everything/aarch64//store/VX/VX6SUOPGEVDWF6E5M2XBV53VS7IXSFM5__openEuler-repos__1.0__3.3.oe2409.epkg +############################################################################################################################################################################################################### 100.0% +start download https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-24.09/everything/aarch64//store/LO/LO6RYZTBB2Q7ZLG6SWSICKGTEHUTBWUA__libselinux__3.5__3.oe2409.epkg +############################################################################################################################################################################################################### 100.0% +start download https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-24.09/everything/aarch64//store/EP/EPIEEK2P5IUPO4PIOJ2BXM3QPEFTZUCT__basesystem__12__3.oe2409.epkg +############################################################################################################################################################################################################### 100.0% +start download https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-24.09/everything/aarch64//store/2G/2GYDDYVWYYIDGOLGTVUACSBHYVRCRJH3__setup__2.14.5__2.oe2409.epkg +############################################################################################################################################################################################################### 100.0% +start download https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-24.09/everything/aarch64//store/HC/HCOKXTWQQUPCFPNI7DMDC6FGSDOWNACC__glibc__2.38__34.oe2409.epkg +############################################################################################################################################################################################################### 100.0% +start download https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-24.09/everything/aarch64//store/OJ/OJQAHJTY3Y7MZAXETYMTYRYSFRVVLPDC__glibc-common__2.38__34.oe2409.epkg +############################################################################################################################################################################################################### 100.0% +start download https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-24.09/everything/aarch64//store/FJ/FJXG3K2TSUYXNU4SES2K3YSTA3AHHUMB__tree__2.1.1__1.oe2409.epkg +############################################################################################################################################################################################################### 100.0% +start download https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-24.09/everything/aarch64//store/KD/KDYRBN74LHKSZISTLMYOMTTFVLV4GPYX__readline__8.2__2.oe2409.epkg +############################################################################################################################################################################################################### 100.0% +start download https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-24.09/everything/aarch64//store/MN/MNJPSSBS4OZJL5EB6YKVFLMV4TGVBUBA__tzdata__2024a__2.oe2409.epkg +############################################################################################################################################################################################################### 100.0% +start download https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-24.09/everything/aarch64//store/S4/S4FBO2SOMG3GKP5OMDWP4XN5V4FY7OY5__bash__5.2.21__1.oe2409.epkg +############################################################################################################################################################################################################### 100.0% +start download https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-24.09/everything/aarch64//store/EJ/EJGRNRY5I6XIDBWL7H5BNYJKJLKANVF6__libsepol__3.5__3.oe2409.epkg +############################################################################################################################################################################################################### 100.0% +start download https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-24.09/everything/aarch64//store/TZ/TZRQZRU2PNXQXHRE32VCADWGLQG6UL36__bc__1.07.1__12.oe2409.epkg +############################################################################################################################################################################################################### 100.0% +start download https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-24.09/everything/aarch64//store/WY/WYMBYMCARHXD62ZNUMN3GQ34DIWMIQ4P__filesystem__3.16__6.oe2409.epkg +############################################################################################################################################################################################################### 100.0% +start download https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-24.09/everything/aarch64//store/KQ/KQ2UE3U5VFVAQORZS4ZTYCUM4QNHBYZ7__openEuler-release__24.09__55.oe2409.epkg +############################################################################################################################################################################################################### 100.0% +start download https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-24.09/everything/aarch64//store/HD/HDTOK5OTTFFKSTZBBH6AIAGV4BTLC7VT__openEuler-gpg-keys__1.0__3.3.oe2409.epkg +############################################################################################################################################################################################################### 100.0% +start download https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-24.09/everything/aarch64//store/EB/EBLBURHOKKIUEEFHZHMS2WYF5OOKB4L3__pcre2__10.42__8.oe2409.epkg +############################################################################################################################################################################################################### 100.0% +start download https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-24.09/everything/aarch64//store/YW/YW5WTOMKY2E5DLYYMTIDIWY3XIGHNILT__info__7.0.3__3.oe2409.epkg +############################################################################################################################################################################################################### 100.0% +start download https://repo.oepkgs.net/openeuler/epkg/channel/openEuler-24.09/everything/aarch64//store/E4/E4KCO6VAAQV5AJGNPW4HIXDHFXMR4EJV__ncurses-base__6.4__8.oe2409.epkg +############################################################################################################################################################################################################### 100.0% +start install FFCRTKRFGFQ6S2YVLOSUF6PHSMRP7A2N__ncurses-libs__6.4__8.oe2409 +start install D5BOEFTRBNV3E4EXBVXDSRNTIGLGWVB7__glibc-all-langpacks__2.38__34.oe2409 +start install VX6SUOPGEVDWF6E5M2XBV53VS7IXSFM5__openEuler-repos__1.0__3.3.oe2409 +start install LO6RYZTBB2Q7ZLG6SWSICKGTEHUTBWUA__libselinux__3.5__3.oe2409 +start install EPIEEK2P5IUPO4PIOJ2BXM3QPEFTZUCT__basesystem__12__3.oe2409 +start install 2GYDDYVWYYIDGOLGTVUACSBHYVRCRJH3__setup__2.14.5__2.oe2409 +start install HCOKXTWQQUPCFPNI7DMDC6FGSDOWNACC__glibc__2.38__34.oe2409 +start install OJQAHJTY3Y7MZAXETYMTYRYSFRVVLPDC__glibc-common__2.38__34.oe2409 +start install FJXG3K2TSUYXNU4SES2K3YSTA3AHHUMB__tree__2.1.1__1.oe2409 +start install KDYRBN74LHKSZISTLMYOMTTFVLV4GPYX__readline__8.2__2.oe2409 +start install MNJPSSBS4OZJL5EB6YKVFLMV4TGVBUBA__tzdata__2024a__2.oe2409 +start install S4FBO2SOMG3GKP5OMDWP4XN5V4FY7OY5__bash__5.2.21__1.oe2409 +start install EJGRNRY5I6XIDBWL7H5BNYJKJLKANVF6__libsepol__3.5__3.oe2409 +start install TZRQZRU2PNXQXHRE32VCADWGLQG6UL36__bc__1.07.1__12.oe2409 +start install WYMBYMCARHXD62ZNUMN3GQ34DIWMIQ4P__filesystem__3.16__6.oe2409 +start install KQ2UE3U5VFVAQORZS4ZTYCUM4QNHBYZ7__openEuler-release__24.09__55.oe2409 +start install HDTOK5OTTFFKSTZBBH6AIAGV4BTLC7VT__openEuler-gpg-keys__1.0__3.3.oe2409 +start install EBLBURHOKKIUEEFHZHMS2WYF5OOKB4L3__pcre2__10.42__8.oe2409 +start install YW5WTOMKY2E5DLYYMTIDIWY3XIGHNILT__info__7.0.3__3.oe2409 +start install E4KCO6VAAQV5AJGNPW4HIXDHFXMR4EJV__ncurses-base__6.4__8.oe2409 +``` + +### Listing Environments + +Function description: + +List all environments in epkg (under the `$EPKG_ENVS_ROOT` directory) and indicate the current environment. + +Command: + +```shell +epkg env list +``` + +Example output: + +```shell +[small_leek@19e784a5bc38 bin]# epkg env list +Available environments(sort by time): +w1 +main +common +You are in [main] now +``` + +### Creating an Environment + +Function description: + +Create a new environment. After successful creation, the new environment is activated by default, but is not globally registered. + +Command: + +```shell +epkg env create ${env_name} +``` + +Example output: + +```shell +[small_leek@b0e608264355 bin]# epkg env create work1 +YUM --installroot directory structure created successfully in: /root/.epkg/envs/work1/profile-1 +Environment 'work1' added to PATH. +Environment 'work1' activated. +Environment 'work1' created. +``` + +### Activating an Environment + +Function description: + +Activate the specified environment, refresh `EPKG_ENV_NAME` and `RPMDB_DIR` (used to point to `--dbpath` when software is installed into the specified environment), refresh `PATH` to include the specified environment and the common environment, and set the specified environment as the first priority. + +Command: + +```shell +epkg env activate ${env_name} +``` + +Example output: + +```shell +[small_leek@9d991d463f89 bin]# epkg env activate main +Environment 'main' activated +``` + +### Deactivating an Environment + +Function description: + +Deactivate the specified environment, refresh `EPKG_ENV_NAME` and `RPMDB_DIR`, refresh `PATH`, and default to the main environment. + +Command: + +```shell +epkg env deactivate ${env_name} +``` + +Example output: + +```shell +[small_leek@398ec57ce780 bin]# epkg env deactivate w1 +Environment 'w1' deactivated. +``` + +### Registering an Environment + +Function description: + +Register the specified environment, persistently refresh `PATH` to include all registered environments in epkg, and set the specified environment as the first priority. + +Command: + +```shell +epkg env register ${env_name} +``` + +Example output: + +```shell +[small_leek@5042ae77dd75 bin]# epkg env register lkp +EPKG_ACTIVE_ENV: +Environment 'lkp' has been registered to PATH. +``` + +### Unregistering an Environment + +Function description: + +Unregister the specified environment, persistently refresh `PATH` to include all registered environments in epkg except the specified one. + +Command: + +```shell +epkg env unregister ${env_name} +``` + +Example output: + +```shell +[small_leek@69393675945d /]# epkg env unregister w4 +EPKG_ACTIVE_ENV: +Environment 'w4' has been unregistered from PATH. +``` + +### Building an epkg Package + +Function description: + +Build an epkg package using the YAML file provided by autopkg. + +Command: + +```shell +epkg build ${yaml_path}/$pkg_name.yaml +``` + +Example output: + +```shell +[small_leek@69393675945d /]# epkg build /root/epkg/build/test/tree/package.yaml +pkg_hash: fbfqtsnza9ez1zk0cy23vyh07xfzsydh, dir: /root/.cache/epkg/build-workspace/result +Compress success: /root/.cache/epkg/build-workspace/epkg/fbfqtsnza9ez1zk0cy23vyh07xfzsydh__tree__2.1.1__0.oe2409.epkg +``` diff --git a/docs/en/tools/community_tools/image_custom/image_tailor/_toc.yaml b/docs/en/tools/community_tools/image_custom/image_tailor/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..5dbdc173516c89476e93f9c665c6ef4d7809b223 --- /dev/null +++ b/docs/en/tools/community_tools/image_custom/image_tailor/_toc.yaml @@ -0,0 +1,4 @@ +label: imageTailor User Guide +isManual: true +href: ./imagetailor-user-guide.md +description: OS image tailoring by removing unneeded packages or files diff --git a/docs/en/tools/community_tools/image_custom/image_tailor/figures/flowchart.png b/docs/en/tools/community_tools/image_custom/image_tailor/figures/flowchart.png new file mode 100644 index 0000000000000000000000000000000000000000..d3a71e8bfdb886222151cea3b2a3c0e8d8eae64a Binary files /dev/null and b/docs/en/tools/community_tools/image_custom/image_tailor/figures/flowchart.png differ diff --git a/docs/en/tools/community_tools/image_custom/image_tailor/imagetailor-user-guide.md b/docs/en/tools/community_tools/image_custom/image_tailor/imagetailor-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..71d48c496bcf745d9c63e2c3b4aaa5fc3591628d --- /dev/null +++ b/docs/en/tools/community_tools/image_custom/image_tailor/imagetailor-user-guide.md @@ -0,0 +1,916 @@ +# imageTailor User Guide + +- [imageTailor User Guide](#imagetailor-user-guide) + - [Introduction](#introduction) + - [Installation](#installation) + - [Software and Hardware Requirements](#software-and-hardware-requirements) + - [Obtaining the Installation Package](#obtaining-the-installation-package) + - [Installing imageTailor](#installing-imagetailor) + - [Directory Description](#directory-description) + - [Image Customization](#image-customization) + - [Overall Process](#overall-process) + - [Customizing Service Packages](#customizing-service-packages) + - [Setting a Local repository](#setting-a-local-repository) + - [Adding Files](#adding-files) + - [Adding RPM Packages](#adding-rpm-packages) + - [Adding Hook Scripts](#adding-hook-scripts) + - [Configuring System Parameters](#configuring-system-parameters) + - [Configuring Host Parameters](#configuring-host-parameters) + - [Configuring Initial Passwords](#configuring-initial-passwords) + - [Configuring Partitions](#configuring-partitions) + - [Configuring the Network](#configuring-the-network) + - [Configuring Kernel Parameters](#configuring-kernel-parameters) + - [Creating an Image](#creating-an-image) + - [Command Description](#command-description) + - [Image Creation Guide](#image-creation-guide) + - [Tailoring Time Zones](#tailoring-time-zones) + - [Customization Example](#customization-example) + +## Introduction + +In addition to the kernel, an operating system contains various peripheral packages. These peripheral packages provide functions of a general-purpose operating system but also cause the following problems: + +- A large number of resources (such as memory, disks, and CPUs) are occupied, resulting in low system performance. +- Unnecessary functions increase the development and maintenance costs. + +To address these problems, openEuler provides the imageTailor tool for tailoring and customization images. You can tailor unnecessary peripheral packages in the OS image or add service packages or files as required. This tool includes the following functions: + +- System package tailoring: Tailors system commands, libraries, and drivers based on the list of RPM packages to be installed. +- System configuration modification: Configures the host name, startup services, time zone, network, partitions, drivers to be loaded, and kernel version. +- Software package addition: Adds custom RPM packages or files to the system. + +## Installation + +This section uses openEuler 22.03 LTS in the AArch64 architecture as an example to describe the installation method. + +### Software and Hardware Requirements + +The software and hardware requirements of imageTailor are as follows: + +- The architecture is x86_64 or AArch64. + +- The OS is openEuler 22.03 LTS (the kernel version is 5.10 and the Python version is 3.9, which meet the tool requirements). + +- The root directory **/** of the host to run the tool have at least 40 GB space. + +- The Python version is 3.9 or later. + +- The kernel version is 5.10 or later. + +- The SElinux service is disabled. + + ```shell + $ sudo setenforce 0 + $ getenforce + Permissive + ``` + +### Obtaining the Installation Package + +Download the openEuler release package to install and use imageTailor. + +1. Obtain the ISO image file and the corresponding verification file. + + The image must be an everything image. Assume that the image is to be stored in the **root** directory. Run the following commands: + + ```shell + cd /root/temp + wget https://repo.openeuler.org/openEuler-22.03-LTS/ISO/aarch64/openEuler-22.03-LTS-everything-aarch64-dvd.iso + wget https://repo.openeuler.org/openEuler-22.03-LTS/ISO/aarch64/openEuler-22.03-LTS-everything-aarch64-dvd.iso.sha256sum + ``` + +2. Obtain the verification value in the sha256sum verification file. + + ```shell + cat openEuler-22.03-LTS-everything-aarch64-dvd.iso.sha256sum + ``` + +3. Calculate the verification value of the ISO image file. + + ```shell + sha256sum openEuler-22.03-LTS-everything-aarch64-dvd.iso + ``` + +4. Compare the verification value in the sha256sum file with that of the ISO image. If they are the same, the file integrity is verified. Otherwise, the file integrity is damaged. You need to obtain the file again. + +### Installing imageTailor + +The following uses openEuler 22.03 LTS in AArch64 architecture as an example to describe how to install imageTailor. + +1. Ensure that openEuler 22.03 LTS (or a running environment that meets the requirements of imageTailor) has been installed on the host. + + ```shell + $ cat /etc/openEuler-release + openEuler release 22.03 LTS + ``` + +2. Create a **/etc/yum.repos.d/local.repo** file to configure the Yum source. The following is an example of the configuration file. **baseurl** indicates the directory for mounting the ISO image. + + ```shell + [local] + name=local + baseurl=file:///root/imageTailor_mount + gpgcheck=0 + enabled=1 + ``` + +3. Run the following commands as the **root** user to mount the image to the **/root/imageTailor_mount** directory as the Yum source (ensure that the value of **baseurl** is the same as that configured in the repo file and the disk space of the directory is greater than 20 GB): + + ```shell + mkdir /root/imageTailor_mount + sudo mount -o loop /root/temp/openEuler-22.03-LTS-everything-aarch64-dvd.iso /root/imageTailor_mount/ + ``` + +4. Make the Yum source take effect. + + ```shell + yum clean all + yum makecache + ``` + +5. Install the imageTailor tool as the **root** user. + + ```shell + sudo yum install -y imageTailor + ``` + +6. Run the following command as the **root** user to verify that the tool has been installed successfully: + + ```shell + $ cd /opt/imageTailor/ + $ sudo ./mkdliso -h + ------------------------------------------------------------------------------------------------------------- + Usage: mkdliso -p product_name -c configpath [--minios yes|no|force] [-h] [--sec] + Options: + -p,--product Specify the product to make, check custom/cfg_yourProduct. + -c,--cfg-path Specify the configuration file path, the form should be consistent with custom/cfg_xxx + --minios Make minios: yes|no|force + --sec Perform security hardening + -h,--help Display help information + + Example: + command: + ./mkdliso -p openEuler -c custom/cfg_openEuler --sec + + help: + ./mkdliso -h + ------------------------------------------------------------------------------------------------------------- + ``` + +### Directory Description + +After imageTailor is installed, the directory structure of the tool package is as follows: + +```shell +[imageTailor] + |-[custom] + |-[cfg_openEuler] + |-[usr_file] // Stores files to be added. + |-[usr_install] //Stores hook scripts to be added. + |-[all] + |-[conf] + |-[hook] + |-[cmd.conf] // Configures the default commands and libraries used by an ISO image. + |-[rpm.conf] // Configures the list of RPM packages and drivers installed by default for an ISO image. + |-[security_s.conf] // Configures security hardening policies. + |-[sys.conf] // Configures ISO image system parameters. + |-[kiwi] // Basic configurations of imageTailor. + |-[repos] //RPM sources for obtaining the RPM packages required for creating an ISO image. + |-[security-tool] // Security hardening tool. + |-mkdliso // Executable script for creating an ISO image. +``` + +## Image Customization + +This section describes how to use the imageTailor tool to package the service RPM packages, custom files, drivers, commands, and libraries to the target ISO image. + +### Overall Process + +The following figure shows the process of using imageTailor to customize an image. + +![](./figures/flowchart.png) + +The steps are described as follows: + +- Check software and hardware environment: Ensure that the host for creating the ISO image meets the software and hardware requirements. + +- Customize service packages: Add RPM packages (including service RPM packages, commands, drivers, and library files) and files (including custom files, commands, drivers, and library files). + + - Adding service RPM packages: Add RPM packages to the ISO image as required. For details, see [Installation](#installation). + - Adding custom files: If you want to perform custom operations such as hardware check, system configuration check, and driver installation when the target ISO system is installed or started, you can compile custom files and package them to the ISO image. + - Adding drivers, commands, and library files: If the RPM package source of openEuler does not contain the required drivers, commands, or library files, you can use imageTailor to package the corresponding drivers, commands, or library files into the ISO image. + +- Configure system parameters: + + - Configuring host parameters: To ensure that the ISO image is successfully installed and started, you need to configure host parameters. + - Configuring partitions: You can configure service partitions based on the service plan and adjust system partitions. + - Configuring the network: You can set system network parameters as required, such as the NIC name, IP address, and subnet mask. + - Configuring the initial password: To ensure that the ISO image is successfully installed and started, you need to configure the initial passwords of the **root** user and GRUB. + - Configuring kernel parameters: You can configure the command line parameters of the kernel as required. + +- Configure security hardening policies. + + imageTailor provides default security hardening policies. You can modify **security_s.conf** (in the ISO image customization phase) to perform secondary security hardening on the system based on service requirements. For details, see the [Security Hardening Guide](https://docs.openeuler.org/en/docs/22.03_LTS/docs/SecHarden/secHarden.html). + +- Create an ISO image: + + Use the imageTailor tool to create an ISO image. + +### Customizing Service Packages + +You can pack service RPM packages, custom files, drivers, commands, and library files into the target ISO image as required. + +#### Setting a Local repository + +To customize an ISO image, you must set a repository in the **/opt/imageTailor/repos/euler_base/** directory. This section describes how to set a local repository. + +1. Download the ISO file released by openEuler. (The RPM package of the everything image released by the openEuler must be used.) + + ```shell + cd /opt + wget https://repo.openeuler.org/openEuler-22.03-LTS/ISO/aarch64/openEuler-22.03-LTS-everything-aarch64-dvd.iso + ``` + +2. Create a mount directory **/opt/openEuler_repo** and mount the ISO file to the directory. + + ```shell + $ sudo mkdir -p /opt/openEuler_repo + $ sudo mount openEuler-22.03-LTS-everything-aarch64-dvd.iso /opt/openEuler_repo + mount: /opt/openEuler_repo: WARNING: source write-protected, mounted read-only. + ``` + +3. Copy the RPM packages in the ISO file to the **/opt/imageTailor/repos/euler_base/** directory. + + ```shell + $ sudo rm -rf /opt/imageTailor/repos/euler_base && sudo mkdir -p /opt/imageTailor/repos/euler_base + $ sudo cp -ar /opt/openEuler_repo/Packages/* /opt/imageTailor/repos/euler_base + $ sudo chmod -R 644 /opt/imageTailor/repos/euler_base + $ sudo ls /opt/imageTailor/repos/euler_base|wc -l + 2577 + $ sudo umount /opt/openEuler_repo && sudo rm -rf /opt/openEuler_repo + $ cd /opt/imageTailor + ``` + +#### Adding Files + +You can add files to an ISO image as required. The file types include custom files, drivers, commands, or library file. Store the files to the **/opt/imageTailor/custom/cfg_openEuler/usr_file** directory. + +##### Precautions + +- The commands to be packed must be executable. Otherwise, imageTailor will fail to pack the commands into the ISO. + +- The file stored in the **/opt/imageTailor/custom/cfg_openEuler/usr_file** directory will be generated in the root directory of the ISO. Therefore, the directory structure of the file must be a complete path starting from the root directory so that imageTailor can place the file in the correct directory. + + For example, if you want **file1** to be in the **/opt** directory of the ISO, create an **opt** directory in the **usr_file** directory and copy **file1** to the **opt** directory. For example: + + ```shell + $ pwd + /opt/imageTailor/custom/cfg_openEuler/usr_file + + $ tree + . + ├── etc + │   ├── default + │   │   └── grub + │   └── profile.d + │   └── csh.precmd + └── opt + └── file1 + + 4 directories, 3 files + ``` + +- The paths in **/opt/imageTailor/custom/cfg_openEuler/usr_file** must be real paths. For example, the paths do not contain soft links. You can run the `realpath` or `readlink -f` command to query the real path. + +- If you need to invoke a custom script in the system startup or installation phase, that is, a hook script, store the file in the **hook** directory. + +#### Adding RPM Packages + +##### Procedure + +To add RPM packages (drivers, commands, or library files) to an ISO image, perform the following steps: + +> [!NOTE]NOTE +> +> - The **rpm.conf** and **cmd.conf** files are stored in the **/opt/imageTailor/custom/cfg_openEuler/** directory. +> - The RPM package tailoring granularity below indicates **sys_cut='no'**. For details about the cutout granularity, see [Configuring Host Parameters](#configuring-host-parameters). +> - If no local repository is configured, configure a local repository by referring to [Setting a Local repository](#setting-a-local-repository). +> + +1. Check whether the **/opt/imageTailor/repos/euler_base/** directory contains the RPM package to be added. + + - If yes, go to step 2. + - If no, go to step 3. + +2. Configure the RPM package information in the **\** section in the **rpm.conf** file. + + - For the RPM package tailoring granularity, no further action is required. + - For other tailoring granularities, go to step 4. + +3. Obtain the RPM package and store it in the **/opt/imageTailor/custom/cfg_openEuler/usr_rpm** directory. If the RPM package depends on other RPM packages, store the dependency packages to this directory because the added RPM package and its dependent RPM packages must be packed into the ISO image at the same time. + + - For the RPM package tailoring granularity, go to step 4. + - For other tailoring granularities, no further action is required. + +4. Configure the drivers, commands, and library files to be retained in the RPM package in the **rpm.conf** and **cmd.conf** files. If there are common files to be tailored, configure them in the **\\** section in the **cmd.conf** file. + +##### Configuration File Description + +| Operation | Configuration File| Section | +| :----------- | :----------- | :----------------------------------------------------------- | +| Adding drivers | rpm.conf | \
\
\
Note: The **driver_name** is the relative path of **/lib/modules/{kernel_version_number}/kernel/**.| +| Adding commands | cmd.conf | \
\
\ | +| Adding library files | cmd.conf | \
\
\ | +| Deleting other files| cmd.conf | \
\
\
Note: The file name must be an absolute path.| + +**Example** + +- Adding drivers + + ```shell + + + + + ...... + + ``` + +- Adding commands + + ```shell + + + + + ...... + + ``` + +- Adding library files + + ```shell + + + + + + ``` + +- Deleting other files + + ```shell + + + + + + ``` + +#### Adding Hook Scripts + +A hook script is invoked by the OS during startup and installation to execute the actions defined in the script. The directory for storing hook scripts of imageTailor is **custom/cfg_openEuler/usr_install/hook directory**, which has different subdirectories. Each subdirectory represents an OS startup or installation phase. Store the scripts based on the phases in which the scripts are invoked. + +##### Script Naming Rule + +The script name must start with **S+number** (the number must be at least two digits). The number indicates the execution sequence of the hook script. Example: **S01xxx.sh** + +> [!NOTE]NOTE +> +> The scripts in the **hook** directory are executed using the `source` command. Therefore, exercise caution when using the `exit` command in the scripts because the entire installation script exits after the `exit` command is executed. + +##### Description of hook Subdirectories + +| Subdirectory | Script Example | Time for Execution | Description | +| :-------------------- | :---------------------| :------------------------------- | :----------------------------------------------------------- | +| insmod_drv_hook | N/A | After OS drivers are loaded | N/A | +| custom_install_hook | S01custom_install.sh | After the drivers are loaded, that is, after **insmod_drv_hook** is executed| You can customize the OS installation process by using a custom script.| +| env_check_hook | S01check_hw.sh | Before the OS installation initialization | The script is used to check hardware specifications and types before initialization.| +| set_install_ip_hook | S01set_install_ip.sh | When network configuration is being performed during OS installation initialization. | You can customize the network configuration by using a custom script.| +| before_partition_hook | S01checkpart.sh | Before partitioning | You can check correctness of the partition configuration file by using a custom script.| +| before_setup_os_hook | N/A | Before the repo file is decompressed | You can customize partition mounting.
If the decompression path of the installation package is not the root partition specified in the partition configuration, customize partition mounting and assign the decompression path to the input global variable.| +| before_mkinitrd_hook | S01install_drv.sh | Before the `mkinitrd` command is run | The hook script executed before running the `mkinitrd` command when **initrd** is saved to the disk. You can add and update driver files in **initrd**.| +| after_setup_os_hook | N/A | After OS installation | After the installation is complete, you can perform custom operations on the system files, such as modifying **grub.cfg**.| +| install_succ_hook | N/A | When the OS is successfully installed | The scripts in this subdirectory are used to parse the installation information and send information of whether the installation succeeds.**install_succ_hook** cannot be set to **install_break**.| +| install_fail_hook | N/A | When the OS installation fails | The scripts in this subdirectory are used to parse the installation information and send information of whether the installation succeeds.**install_fail_hook** cannot be set to **install_break**.| + +### Configuring System Parameters + +Before creating an ISO image, you need to configure system parameters, including host parameters, initial passwords, partitions, network, compilation parameters, and system command line parameters. + +#### Configuring Host Parameters + +The **\ \** section in the **/opt/imageTailor/custom/cfg_openEuler/sys.conf** file is used to configure common system parameters, such as the host name and kernel boot parameters. + +The default configuration provided by openEuler is as follows. You can modify the configuration as required. + +```shell + + sys_service_enable='ipcc' + sys_service_disable='cloud-config cloud-final cloud-init-local cloud-init' + sys_utc='yes' + sys_timezone='' + sys_cut='no' + sys_usrrpm_cut='no' + sys_hostname='Euler' + sys_usermodules_autoload='' + sys_gconv='GBK' + +``` + +The parameters are described as follows: + +- sys_service_enable + + This parameter is optional. Services enabled by the OS by default. Separate multiple services with spaces. If you do not need to add a system service, use the default value **ipcc**. Pay attention to the following during the configuration: + + - Default system services cannot be deleted. + - You can configure service-related services, but the repository must contain the service RPM package. + - By default, only the services configured in this parameter are enabled. If a service depends on other services, you need to configure the depended services in this parameter. + +- sys_service_disable + + This parameter is optional. Services that are not allowed to automatically start upon system startup. Separate multiple services with spaces. If no system service needs to be disabled, leave this parameter blank. + +- sys_utc + + (Mandatory) Indicates whether to use coordinated universal time (UTC) time. The value can be **yes** or **no**. The default value is **yes**. + +- sys_timezone + + This parameter is optional. Sets the time zone. The value can be a time zone supported by openEuler, which can be queried in the **/usr/share/zoneinfo/zone.tab** file. + +- sys_cut + + (Mandatory) Indicates whether to tailor the RPM packages. The value can be **yes**, **no**, or **debug**.**yes** indicates that the RPM packages are tailored. **no** indicates that the RPM packages are not tailored (only the RPM packages in the **rpm.conf** file is installed). **debug** indicates that the RPM packages are tailored but the `rpm` command is retained for customization after installation. The default value is **no**. + + > [!NOTE]NOTE + > imageTailor installs the RPM package added by the user, deletes the files configured in the **\** section of the **cmd.conf** file, and then deletes the commands, libraries, and drivers that are not configured in **cmd.conf** or **rpm.conf**. + > When **sys_cut='yes'** is configured, imageTailor does not support the installation of the `rpm` command. Even if the `rpm` command is configured in the **rpm.conf** file, the configuration does not take effect. + +- sys_usrrpm_cut + + (Mandatory) Indicates whether to tailor the RPM packages added by users to the **/opt/imageTailor/custom/cfg_openEuler/usr_rpm** directory. The value can be **yes** or **no**. The default value is **no**. + + - **sys_usrrpm_cut='yes'**: imageTailor installs the RPM packages added by the user, deletes the file configured in the **\** section in the **cmd.conf** file, and then deletes the commands, libraries, and drivers that are not configured in **cmd.conf** or **rpm.conf**. + + - **sys_usrrpm_cut='no'**: imageTailor installs the RPM packages added by the user but does not delete the files in the RPM packages. + +- sys_hostname + + (Mandatory) Host name. After the OS is deployed in batches, you are advised to change the host name of each node to ensure that the host name of each node is unique. + + The host name must be a combination of letters, digits, and hyphens (-) and must start with a letter or digit. Letters are case sensitive. The value contains a maximum of 63 characters. The default value is **Euler**. + +- sys_usermodules_autoload + + (Optional) Driver loaded during system startup. When configuring this parameter, you do not need to enter the file extension **.ko**. If there are multiple drivers, separate them by space. By default, this parameter is left blank, indicating that no additional driver is loaded. + +- sys_gconv + + (Optional) This parameter is used to tailor **/usr/lib/gconv** and **/usr/lib64/gconv**. The options are as follows: + + - **null**/**NULL**: indicates that this parameter is not configured. If **sys_cut='yes'** is configured, **/usr/lib/gconv** and **/usr/lib64/gconv** will be deleted. + - **all**/**ALL**: keeps **/usr/lib/gconv** and **/usr/lib64/gconv**. + - **xxx,xxx**: keeps the corresponding files in the **/usr/lib/gconv** and **/usr/lib64/gconv** directories. If multiple files need to be kept, use commas (,) to separate them. + +- sys_man_cut + + (Optional) Indicates whether to tailor the man pages. The value can be **yes** or **no**. The default value is **yes**. + +> [!NOTE]NOTE +> If both **sys_cut** and **sys_usrrpm_cut** are configured, **sys_cut** is used. The following rules apply: + +- sys_cut='no' + +No matter whether **sys_usrrpm_cut** is set to **yes** or **no**, the system RPM package tailoring granularity is used. That is, imageTailor installs the RPM packages in the repository and the RPM packages in the **usr_rpm** directory, however, the files in the RPM package are not deleted. Even if some files in the RPM packages are not required, imageTailor will delete them. + +- sys_cut='yes' + +1. sys_usrrpm_cut='no' + + System RPM package tailoring granularity: imageTailor deletes files in the RPM packages in the repositories as configured. + +2. sys_usrrpm_cut='yes' + + System and user RPM package tailoring granularity: imageTailor deletes files in the RPM packages in the repositories and the **usr_rpm** directory as configured. + +#### Configuring Initial Passwords + +The **root** and GRUB passwords must be configured during OS installation. Otherwise, you cannot log in to the OS as the **root** user after the OS is installed using the tailored ISO image. This section describes how to configure the initial passwords. + +> [!NOTE]NOTE +> You must configure the initial **root** and GRUB passwords manually. + +##### Configuring the Initial Password of the root User + +###### Introduction + +The initial password of the **root** user is stored in the **/opt/imageTailor/custom/cfg_openEuler/rpm.conf** file. You can modify this file to set the initial password of the **root** user. + +> [!NOTE]NOTE +> +> - If the `--minios yes/force` parameter is required when you run the `mkdliso` command to create an ISO image, you need to enter the corresponding information in the **/opt/imageTailor/kiwi/minios/cfg_minios/rpm.conf** file. + +The default configuration of the initial password of the **root** user in the **/opt/imageTailor/custom/cfg_openEuler/rpm.conf** file is as follows. Add a password of your choice. + +```xml + + + +``` + +The parameters are described as follows: + +- **group**: group to which the user belongs. +- **pwd**: ciphertext of the initial password. The encryption algorithm is SHA-512. Replace **${pwd}** with the actual ciphertext. +- **home**: home directory of the user. +- **name**: name of the user to be configured. + +###### Modification Method + +Before creating an ISO image, you need to change the initial password of the **root** user. The following describes how to set the initial password of the **root** user (**root** permissions are required): + +1. Add a user for generating a password, for example, **testUser**. + + ```shell + sudo useradd testUser + ``` + +2. Set the password of **testUser**. Run the following command and set the password as prompted: + + ```shell + $ sudo passwd testUser + Changing password for user testUser. + New password: + Retype new password: + passwd: all authentication tokens updated successfully. + ``` + +3. View the **/etc/shadow** file. The content following **testUser** (string between two colons) is the ciphertext of the password. + + ``` shell script + $ sudo cat /etc/shadow | grep testUser + testUser:$6$YkX5uFDGVO1VWbab$jvbwkZ2Kt0MzZXmPWy.7bJsgmkN0U2gEqhm9KqT1jwQBlwBGsF3Z59heEXyh8QKm3Qhc5C3jqg2N1ktv25xdP0:19052:0:90:7:35:: + ``` + +4. Copy and paste the ciphertext to the **pwd** field in the **/opt/imageTailor/custom/cfg_openEuler/rpm.conf** file. + + ``` shell script + + + + ``` + +5. If the `--minios yes/force` parameter is required when you run the `mkdliso` command to create an ISO image, configure the **pwd** field of the corresponding user in **/opt/imageTailor/kiwi/minios/cfg_minios/rpm.conf**. + + ``` shell script + + + + ``` + +##### Configuring the Initial GRUB Password + +The initial GRUB password is stored in the **/opt/imageTailor/custom/cfg_openEuler/usr_file/etc/default/grub** file. Modify this file to configure the initial GRUB password. If the initial GRUB password is not configured, the ISO image will fail to be created. + +> [!NOTE]NOTE: + +- The **root** permissions are required for configuring the initial GRUB password. +- The default user corresponding to the GRUB password is **root**. +- The `grub2-set-password` command must exist in the system. If the command does not exist, install it in advance. + +1. Run the following command and set the GRUB password as prompted: + + ```shell + $ sudo grub2-set-password -o ./ + Enter password: + Confirm password: + grep: .//grub.cfg: No such file or directory + WARNING: The current configuration lacks password support! + Update your configuration with grub2-mkconfig to support this feature. + ``` + +2. After the command is executed, the **user.cfg** file is generated in the current directory. The content starting with **grub.pbkdf2.sha512** is the encrypted GRUB password. + + ```shell + $ sudo cat user.cfg + GRUB2_PASSWORD=grub.pbkdf2.sha512.10000.CE285BE1DED0012F8B2FB3DEA38782A5B1040FEC1E49D5F602285FD6A972D60177C365F1 + B5D4CB9D648AD4C70CF9AA2CF9F4D7F793D4CE008D9A2A696A3AF96A.0AF86AB3954777F40D324816E45DD8F66CA1DE836DC7FBED053DB02 + 4456EE657350A27FF1E74429546AD9B87BE8D3A13C2E686DD7C71D4D4E85294B6B06E0615 + ``` + +3. Copy the preceding ciphertext and add the following configuration to the **/opt/imageTailor/custom/cfg_openEuler/usr_file/etc/default/grub** file: + + ```shell + GRUB_PASSWORD="grub.pbkdf2.sha512.10000.CE285BE1DED0012F8B2FB3DEA38782A5B1040FEC1E49D5F602285FD6A972D60177C365F1 + B5D4CB9D648AD4C70CF9AA2CF9F4D7F793D4CE008D9A2A696A3AF96A.0AF86AB3954777F40D324816E45DD8F66CA1DE836DC7FBED053DB02 + 4456EE657350A27FF1E74429546AD9B87BE8D3A13C2E686DD7C71D4D4E85294B6B06E0615" + ``` + +#### Configuring Partitions + +If you want to adjust system partitions or service partitions, modify the **\** section in the **/opt/imageTailor/custom/cfg_openEuler/sys.conf** file. + +> [!NOTE]NOTE +> +> - System partition: partition for storing the OS. +> - Service partition: partition for service data. +> - The type of a partition is determined by the content it stores, not the size, mount path, or file system. +> - Partition configuration is optional. You can manually configure partitions after OS installation. + +The format of **\** is as follows: + +disk_ID mount_path partition _size partition_type file_system \[Secondary formatting flag] + +The default configuration is as follows: + +```shell + +hd0 /boot 512M primary ext4 yes +hd0 /boot/efi 200M primary vfat yes +hd0 / 30G primary ext4 +hd0 - - extended - +hd0 /var 1536M logical ext4 +hd0 /home max logical ext4 + +``` + +The parameters are described as follows: + +- disk_ID: + ID of a disk. Set this parameter in the format of **hd***x*, where *x* indicates the *x*th disk. + + > [!NOTE]NOTE + > + > Partition configuration takes effect only when the disk can be recognized. + +- mount_path: + Mount path to a specified partition. You can configure service partitions and adjust the default system partition. If you do not mount partitions, set this parameter to **-**. + + > [!NOTE]NOTE + > + > - You must configure the mount path to **/**. You can adjust mount paths to other partitions according to your needs. + > - When the UEFI boot mode is used, the partition configuration in the x86_64 architecture must contain the mount path **/boot**, and the partition configuration in the AArch64 architecture must contain the mount path **/boot/efi**. + +- partition_size: + The value types are as follows: + + - G/g: The unit of a partition size is GB, for example, 2G. + - M/m: The unit of a partition size is MB, for example, 300M. + - T/t: The unit of a partition size is TB, for example, 1T. + - MAX/max: The rest space of a hard disk is used to create a partition. This value can only be assigned to the last partition. + + > [!NOTE]NOTE + > + > - A partition size value cannot contain decimal numbers. If there are decimal numbers, change the unit of the value to make the value an integer. For example, 1.5 GB should be changed to 1536 MB. + > - When the partition size is set to **MAX**/**max**, the size of the remaining partition cannot exceed the limit of the supported file system type (the default file system type is **ext4**, and the maximum size is **16T**). + +- partition_type: + The values of partition types are as follows: + + - primary: primary partitions + - extended: extended partition (configure only *disk_ID* for this partition) + - logical: logical partitions + +- file_system: + Currently, **ext4** and **vfat** file systems are supported. + +- Secondary formatting flag: + Indicates whether to format the disk during secondary installation. This parameter is optional. + + - The value can be **yes** or **no**. The default value is **no**. + + > [!NOTE]NOTE + > + > Secondary formatting indicates that openEuler has been installed on the disk before this installation. If the partition table configuration (partition size, mount point, and file type) used in the previous installation is the same as that used in the current installation, this flag can be used to configure whether to format the previous partitions, except the **/boot** and **/** partitions. If the target host is installed for the first time, this flag does not take effect, and all partitions with specified file systems are formatted. + +#### Configuring the Network + +The system network parameters are stored in **/opt/imageTailor/custom/cfg_openEuler/sys.conf**. You can modify the network parameters of the target ISO image, such as the NIC name, IP address, and subnet mask, by configuring **\\** in this file. + +The default network configuration in the **sys.conf** file is as follows. **netconfig-0** indicates the **eth0** NIC. If you need to configure an additional NIC, for example, **eth1**, add **\\** to the configuration file and set the parameters of **eth1**. + +```shell + +BOOTPROTO="dhcp" +DEVICE="eth0" +IPADDR="" +NETMASK="" +STARTMODE="auto" + +``` + +The following table describes the parameters. + +| Parameter | Mandatory or Not| Value | Description | +| :-------- | -------- | :------------------------------------------------ | :----------------------------------------------------------- | +| BOOTPROTO | Yes | none / static / dhcp | **none**: No protocol is used for boot, and no IP address is assigned.
**static**: An IP address is statically assigned.
**dhcp**: An IP address is dynamically obtained using the dynamic host configuration protocol (DHCP).| +| DEVICE | Yes | Example: **eth1** | NIC name. | +| IPADDR | Yes | Example: **192.168.11.100** | IP address.
This parameter must be configured only when the value of **BOOTPROTO** is **static**.| +| NETMASK | Yes | - | Subnet mask.
This parameter must be configured only when the value of **BOOTPROTO** is **static**.| +| STARTMODE | Yes | manual / auto / hotplug / ifplugd / nfsroot / off | NIC start mode.
**manual**: A user runs the `ifup` command on a terminal to start an NIC.
**auto**/**hotplug**/**ifplug**/**nfsroot**: An NIC is started when the OS identifies it.
**off**: An NIC cannot be started in any situations.
For details about the parameters, run the `man ifcfg` command on the host that is used to create the ISO image.| + +#### Configuring Kernel Parameters + +To ensure stable and efficient running of the system, you can modify kernel command line parameters as required. For an OS image created by imageTailor, you can modify the **GRUB_CMDLINE_LINUX** configuration in the **/opt/imageTailor/custom/cfg_openEuler/usr_file/etc/default/grub** file to modify the kernel command line parameters. The default settings of the kernel command line parameters in **GRUB_CMDLINE_LINUX** are as follows: + +```shell +GRUB_CMDLINE_LINUX="net.ifnames=0 biosdevname=0 crashkernel=512M oops=panic softlockup_panic=1 reserve_kbox_mem=16M crash_kexec_post_notifiers panic=3 console=tty0" +``` + +The meanings of the configurations are as follows (for details about other common kernel command line parameters, see related kernel documents): + +- net.ifnames=0 biosdevname=0 + + Name the NIC in traditional mode. + +- crashkernel=512M + + The memory space reserved for kdump is 512 MB. + +- oops=panic panic=3 + + The kernel panics when an oops error occurs, and the system restarts 3 seconds later. + +- softlockup_panic=1 + + The kernel panics when a soft-lockup is detected. + +- reserve_kbox_mem=16M + + The memory space reserved for Kbox is 16 MB. + +- console=tty0 + + Specifies **tty0** as the output device of the first virtual console. + +- crash_kexec_post_notifiers + + After the system crashes, the function registered with the panic notification chain is called first, and then kdump is executed. + +### Creating an Image + +After customizing the operating system, you can use the `mkdliso` script to create the OS image file. The OSimage created using imageTailor is an ISO image file. + +#### Command Description + +##### Syntax + +**mkdliso -p openEuler -c custom/cfg_openEuler \[--minios yes\|no\|force] \[--sec] \[-h]** + +##### Parameter Description + +| Parameter| Mandatory| Description | Value Range | +| -------- | -------- | ------------------------------------------------------------ | ------------------------------------------------------------ | +| -p | Yes | Specifies the product name. | **openEuler** | +| c | Yes | Specifies the relative path of the configuration file. | **custom/cfg_openEuler** | +| --minios | No | Specifies whether to create the **initrd** file that is used to boot the system during system installation. | The default value is **yes**.
**yes**: The **initrd** file will be created when the command is executed for the first time. When a subsequent `mkdliso` is executed, the system checks whether the **initrd** file exists in the **usr_install/boot** directory using sha256 verification. If the **initrd** file exists, it is not created again. Otherwise, it is created.
**no**: The **initrd** file is not created. The **initrd** file used for system boot and running is the same.
**force**: The **initrd** file will be created forcibly, regardless of whether it exists in the **usr_install/boot** directory or not.| +| --sec | No | Specifies whether to perform security hardening on the generated ISO file.
If this parameter is not specified, the user should undertake the resultant security risks| N/A | +| -h | No | Obtains help information. | N/A | + +#### Image Creation Guide + +To create an ISO image using`mkdliso`, perform the following steps: + +> [!NOTE]NOTE: + +- The absolute path to `mkdliso` must not contain spaces. Otherwise, the ISO image creation will fail. +- In the environment for creating the ISO image, the value of **umask** must be set to **0022**. + +1. Run the `mkdliso` command as the **root** user to generate the ISO image file. The following command is used for reference: + + ```shell + # sudo /opt/imageTailor/mkdliso -p openEuler -c custom/cfg_openEuler --sec + ``` + + After the command is executed, the created files are stored in the **/opt/imageTailor/result/{date}** directory, including **openEuler-aarch64.iso** and **openEuler-aarch64.iso.sha256**. + +2. Verify the integrity of the ISO image file. Assume that the date and time is **2022-03-21-14-48**. + + ```shell + cd /opt/imageTailor/result/2022-03-21-14-48/ + sha256sum -c openEuler-aarch64.iso.sha256 + ``` + + If the following information is displayed, the ISO image creation is complete. + + ```text + openEuler-aarch64.iso: OK + ``` + + If the following information is displayed, the image is incomplete. The ISO image file is damaged and needs to be created again. + + ```shell + openEuler-aarch64.iso: FAILED + sha256sum: WARNING: 1 computed checksum did NOT match + ``` + +3. View the logs. + + After an image is created, you can view logs as required (for example, when an error occurs during image creation). When an image is created for the first time, the corresponding log file and security hardening log file are compressed into a TAR package (the log file is named in the format of **sys_custom_log_{Date}.tar.gz**) and stored in the **result/log directory**. Only the latest 50 compressed log packages are stored in this directory. If the number of compressed log packages exceeds 50, the earliest files will be overwritten. + +### Tailoring Time Zones + +After the customized ISO image is installed, you can tailor the time zones supported by the openEuler system as required. This section describes how to tailor the time zones. + +The information about time zones supported by openEuler is stored in the time zone folder **/usr/shre/zoninfo**. You can run the following command to view the time zone information: + +```shell +$ ls /usr/share/zoneinfo/ +Africa/ America/ Asia/ Atlantic/ Australia/ Etc/ Europe/ +Pacific/ zone.tab +``` + +Each subfolder represents an area. The current areas include continents, oceans, and **Etc**. Each area folder contains the locations that belong to it. Generally, a location is a city or an island. + +All time zones are in the format of *area/location*. For example, if China Standard Time is used in southern China, the time zone is Asia/Shanghai (location may not be the capital). The corresponding time zone file is: + +```texta +/usr/share/zoneinfo/Asia/Shanghai +``` + +If you want to tailor some time zones, delete the corresponding time zone files. + +### Customization Example + +This section describes how to use imageTailor to create an ISO image. + +1. Check whether the environment used to create the ISO meets the requirements. + + ``` shell + $ cat /etc/openEuler-release + openEuler release 22.03 LTS + ``` + +2. Ensure that the root directory has at least 40 GB free space. + + ```shell + $ df -h + Filesystem Size Used Avail Use% Mounted on + ...... + /dev/vdb 196G 28K 186G 1% / + ``` + +3. Install the imageTailor tailoring tool. For details, see [Installation](#installation). + + ```shell + $ sudo yum install -y imageTailor + $ ll /opt/imageTailor/ + total 88K + drwxr-xr-x. 3 root root 4.0K Mar 3 08:00 custom + drwxr-xr-x. 10 root root 4.0K Mar 3 08:00 kiwi + -r-x------. 1 root root 69K Mar 3 08:00 mkdliso + drwxr-xr-x. 2 root root 4.0K Mar 9 14:48 repos + drwxr-xr-x. 2 root root 4.0K Mar 9 14:48 security-tool + ``` + +4. Configure a local repository. + + ```shell + $ wget https://repo.openeuler.org/openEuler-22.03-LTS/ISO/aarch64/openEuler-22.03-LTS-everything-aarch64-dvd.iso + $ sudo mkdir -p /opt/openEuler_repo + $ sudo mount openEuler-22.03-LTS-everything-aarch64-dvd.iso /opt/openEuler_repo + mount: /opt/openEuler_repo: WARNING: source write-protected, mounted read-only. + $ sudo rm -rf /opt/imageTailor/repos/euler_base && sudo mkdir -p /opt/imageTailor/repos/euler_base + $ sudo cp -ar /opt/openEuler_repo/Packages/* /opt/imageTailor/repos/euler_base + $ sudo chmod -R 644 /opt/imageTailor/repos/euler_base + $ sudo ls /opt/imageTailor/repos/euler_base|wc -l + 2577 + $ sudo umount /opt/openEuler_repo && sudo rm -rf /opt/openEuler_repo + $ cd /opt/imageTailor + ``` + +5. Change the **root** and GRUB passwords. + + Replace **\${pwd}** with the encrypted password by referring to [Configuring Initial Passwords](#configuring-initial-passwords). + + ```shell + $ cd /opt/imageTailor/ + $ sudo vi custom/cfg_openEuler/usr_file/etc/default/grub + GRUB_PASSWORD="${pwd1}" + $ + $ sudo vi kiwi/minios/cfg_minios/rpm.conf + + + + $ + $ sudo vi custom/cfg_openEuler/rpm.conf + + + + ``` + +6. Run the tailoring command. + + ```shell + $ sudo rm -rf /opt/imageTailor/result + $ sudo ./mkdliso -p openEuler -c custom/cfg_openEuler --minios force + ...... + Complete release iso file at: result/2022-03-09-15-31/openEuler-aarch64.iso + move all mkdliso log file to result/log/sys_custom_log_20220309153231.tar.gz + $ ll result/2022-03-09-15-31/ + total 889M + -rw-r--r--. 1 root root 889M Mar 9 15:32 openEuler-aarch64.iso + -rw-r--r--. 1 root root 87 Mar 9 15:32 openEuler-aarch64.iso.sha256 + ``` diff --git a/docs/en/tools/community_tools/image_custom/image_tailor/public_sys-resources/icon-note.gif b/docs/en/tools/community_tools/image_custom/image_tailor/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/tools/community_tools/image_custom/image_tailor/public_sys-resources/icon-note.gif differ diff --git a/docs/en/tools/community_tools/image_custom/isocut/_toc.yaml b/docs/en/tools/community_tools/image_custom/isocut/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..0e12216d3c04089e2f8dfb61f90b0a28bea03218 --- /dev/null +++ b/docs/en/tools/community_tools/image_custom/isocut/_toc.yaml @@ -0,0 +1,4 @@ +label: isocut User Guide +isManual: true +href: ./isocut-user-guide.md +description: Customize openEuler ISO images. diff --git a/docs/en/tools/community_tools/image_custom/isocut/figures/lack_pack.png b/docs/en/tools/community_tools/image_custom/isocut/figures/lack_pack.png new file mode 100644 index 0000000000000000000000000000000000000000..a4b7f1da15da70f63a86aae360e89017c2b98f2d Binary files /dev/null and b/docs/en/tools/community_tools/image_custom/isocut/figures/lack_pack.png differ diff --git a/docs/en/tools/community_tools/image_custom/isocut/isocut-user-guide.md b/docs/en/tools/community_tools/image_custom/isocut/isocut-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..7d8508fc9486df1c4d9e4166fc6de61501bbc164 --- /dev/null +++ b/docs/en/tools/community_tools/image_custom/isocut/isocut-user-guide.md @@ -0,0 +1,327 @@ +# isocut User Guide + +- [Introduction](#introduction) +- [Software and Hardware Requirements](#software-and-hardware-requirements) +- [Installation](#Installation) +- [Tailoring and Customizing an Image](#tailoring-and-customizing-an-image) + - [Command Description](#command-description) + - [Software Package Source](#software-package-source) + - [Operation Guide](#operation-guide) + +## Introduction + +The size of an openEuler image is large, and the process of downloading or transferring an image is time-consuming. In addition, when an openEuler image is used to install the OS, all RPM packages contained in the image are installed. You cannot choose to install only the required software packages. + +In some scenarios, you do not need to install the full software package provided by the image, or you need to install additional software packages. Therefore, openEuler provides an image tailoring and customization tool. You can use this tool to customize an ISO image that contains only the required RPM packages based on an openEuler image. The software packages can be the ones contained in an official ISO image or specified in addition to meet custom requirements. + +This document describes how to install and use the openEuler image tailoring and customization tool. + +## Software and Hardware Requirements + +The hardware and software requirements of the computer to make an ISO file using the openEuler tailoring and customization tool are as follows: + +- The CPU architecture is AArch64 or x86_64. +- The operating system is openEuler 20.03 LTS SP3. +- You are advised to reserve at least 30 GB drive space for running the tailoring and customization tool and storing the ISO image. + +## Installation + +The following uses openEuler 20.03 LTS SP3 on the AArch64 architecture as an example to describe how to install the ISO image tailoring and customization tool. + +1. Ensure that openEuler 20.03 LTS SP3 has been installed on the computer. + + ```shell + $ cat /etc/openEuler-release + openEuler release 20.03 (LTS-SP3) + ``` + +2. Download the ISO image (must be an **everything** image) of the corresponding architecture and save it to any directory (it is recommended that the available space of the directory be greater than 20 GB). In this example, the ISO image is saved to the **/home/isocut_iso** directory. + + The download address of the AArch64 image is as follows: + + + + > **Note:** + > The download address of the x86_64 image is as follows: + > + > + +3. Create a **/etc/yum.repos.d/local.repo** file to configure the Yum source. The following is an example of the configuration file. **baseurl** is the directory for mounting the ISO image. + + ```shell + [local] + name=local + baseurl=file:///home/isocut_mount + gpgcheck=0 + enabled=1 + ``` + +4. Run the following command as the **root** user to mount the image to the **/home/isocut_mount** directory (ensure that the mount directory is the same as **baseurl** configured in the **repo** file) as the Yum source: + + ```shell + sudo mount -o loop /home/isocut_iso/openEuler-20.03-LTS-SP3-everything-aarch64-dvd.iso /home/isocut_mount + ``` + +5. Make the Yum source take effect. + + ```shell + yum clean all + yum makecache + ``` + +6. Install the image tailoring and customization tool as the **root** user. + + ```shell + sudo yum install -y isocut + ``` + +7. Run the following command as the **root** user to verify that the tool has been installed successfully: + + ```shell + $ sudo isocut -h + Checking input ... + usage: isocut [-h] [-t temporary_path] [-r rpm_path] [-k file_path] source_iso dest_iso + + Cut openEuler iso to small one + + positional arguments: + source_iso source iso image + dest_iso destination iso image + + optional arguments: + -h, --help show this help message and exit + -t temporary_path temporary path + -r rpm_path extern rpm packages path + -k file_path kickstart file + ``` + +## Tailoring and Customizing an Image + +This section describes how to use the image tailoring and customization tool to create an image by tailoring or adding RPM packages to an openEuler image. + +### Command Description + +#### Format + +Run the `isocut` command to use the image tailoring and customization tool. The command format is as follows: + +**isocut** \[ --help \| -h \] \[ -t \ ] \[ -r \ ] \[ -k \ ] \ \ + +#### Parameter Description + +| Parameter | Mandatory | Description | +| ---------------- | --------- | ----------------------------------------------------------------------------------------------------------------------------------- | +| --help \| -h | No | Queries the help information about the command. | +| -t \<*temp_path*> | No | Specifies the temporary directory *temp_path* for running the tool, which is an absolute path. The default value is **/tmp**. | +| -r \<*rpm_path*> | No | Specifies the path of the RPM packages to be added to the ISO image. | +| -k \<*file_path*> | No | Specifies the kickstart template path if kickstart is used for automatic installation. | +| *source_iso* | Yes | Path and name of the ISO source image to be tailored. If no path is specified, the current path is used by default. | +| *dest_iso* | Yes | Specifies the path and name of the new ISO image created by the tool. If no path is specified, the current path is used by default. | + +### Software Package Source + +The RPM packages of the new image can be: + +- Packages contained in an official ISO image. In this case, the RPM packages to be installed are specified in the configuration file **/etc/isocut/rpmlist**. The configuration format is *software_package_name.architecture*. For example, **kernel.aarch64**. + +- Specified in addition. In this case, use the `-r` parameter to specify the path in which the RPM packages are stored when running the `isocut` command and add the RPM package names to the **/etc/isocut/rpmlist** configuration file. (See the name format above.) + + > [!NOTE]NOTE + > + > - When customizing an image, if an RPM package specified in the configuration file cannot be found, the RPM package will not be added to the image. + > - If the dependency of the RPM package is incorrect, an error may be reported when running the tailoring and customization tool. + +### kickstart Functions + +You can use kickstart to install images automatically by using the `-k` parameter to specify a kickstart file when running the `isocut` command. + +The isocut tool provides a kickstart template (**/etc/isocut/anaconda-ks.cfg**). You can modify the template as required. + +#### Modifying the kickstart Template + +If you need to use the kickstart template provided by the isocut tool, perform the following modifications: + +- Configure the root user password and the GRUB2 password in the **/etc/isocut/anaconda-ks.cfg** file. Otherwise, the automatic image installation will pause during the password setting process, waiting for you to manually enter the passwords. +- If you want to specify additional RPM packages and use kickstart for automatic installation, specify the RPM packages in the **%packages** field in both the **/etc/isocut/rpmlist** file and the kickstart file. + +See the next section for details about how to modify the kickstart file. + +##### Configuring Initial Passwords + +###### Setting the Initial Password of the **root** User + +Set the initial password of the **root** user as follows in the **/etc/isocut/anaconda-ks.cfg** file. Replace **${pwd}** with the encrypted password. + +```shell +rootpw --iscrypted ${pwd} +``` + +Obtain the initial password of the **root** user as follows (**root** permissions are required): + +1. Add a user for generating the password, for example, **testUser**. + + ```shell + sudo useradd testUser + ``` + +2. Set the password for the **testUser** user. Run the following command to set the password as prompted: + + ```shell + $ sudo passwd testUser + Changing password for user testUser. + New password: + Retype new password: + passwd: all authentication tokens updated successfully. + ``` + +3. View the **/etc/shadow** file to obtain the encrypted password. The encrypted password is the string between the two colons (:) following the **testUser** user name. (******* is used as an example.) + + ```shell + $ sudo cat /etc/shadow | grep testUser + testUser:***:19052:0:90:7:35:: + ``` + +4. Run the following command to replace the **pwd** field in the **/etc/isocut/anaconda-ks.cfg** file with the encrypted password (replace __***__ with the actual password): + + ```shell + rootpw --iscrypted *** + ``` + +###### Configuring the Initial GRUB2 Password + +Add the following configuration to the **/etc/isocut/anaconda-ks.cfg** file to set the initial GRUB2 password: Replace **${pwd}** with the encrypted password. + +```shell +%addon com_huawei_grub_safe --iscrypted --password='${pwd}' +%end +``` + +> ![](./public_sys-resources/icon-note.gif)NOTE: +> +> - The **root** permissions are required for configuring the initial GRUB password. +> - The default user corresponding to the GRUB password is **root**. +> +> - The `grub2-set-password` command must exist in the system. If the command does not exist, install it in advance. + +1. Run the following command and set the GRUB2 password as prompted: + + ```shell + $ sudo grub2-set-password -o ./ + Enter password: + Confirm password: + grep: .//grub.cfg: No such file or directory + WARNING: The current configuration lacks password support! + Update your configuration with grub2-mkconfig to support this feature. + ``` + +2. After the command is executed, the **user.cfg** file is generated in the current directory. The content starting with **grub.pbkdf2.sha512** is the encrypted GRUB2 password. + + ```shell + $ sudo cat user.cfg + GRUB2_PASSWORD=grub.pbkdf2.sha512.*** + ``` + +3. Add the following information to the **/etc/isocut/anaconda-ks.cfg** file. Replace ******* with the encrypted GRUB2 password. + + ```shell + %addon com_huawei_grub_safe --iscrypted --password='grub.pbkdf2.sha512.***' + %end + ``` + +##### Configuring the %packages Field + +If you want to specify additional RPM packages and use kickstart for automatic installation, specify the RPM packages in the **%packages** field in both the **/etc/isocut/rpmlist** file and the kickstart file. + +This section describes how to specify RPM packages in the **/etc/isocut/anaconda-ks.cfg** file. + +The default configurations of **%packages** in the **/etc/isocut/anaconda-ks.cfg** file are as follows: + +```shell +%packages --multilib --ignoremissing +acl.aarch64 +aide.aarch64 +...... +NetworkManager.aarch64 +%end +``` + +Add specified RPM packages to the **%packages** configurations in the following format: + +*software_package_name.architecture*. For example, **kernel.aarch64**. + +```shell +%packages --multilib --ignoremissing +acl.aarch64 +aide.aarch64 +...... +NetworkManager.aarch64 +kernel.aarch64 +%end +``` + +### Operation Guide + +> [!NOTE]NOTE +> +> - Do not modify or delete the default configuration items in the **/etc/isocut/rpmlist** file. +> - All `isocut` operations require **root** permissions. +> - The source image to be tailored can be a basic image or **everything** image. In this example, the basic image **openEuler-20.03-LTS-SP3-aarch64-dvd.iso** is used. +> - In this example, assume that the new image is named **new.iso** and stored in the **/home/result** directory, the temporary directory for running the tool is **/home/temp**, and the additional RPM packages are stored in the **/home/rpms** directory. + +1. Open the configuration file **/etc/isocut/rpmlist** and specify the RPM packages to be installed (from the official ISO image). + + ```shell + sudo vi /etc/isocut/rpmlist + ``` + +2. Ensure that the space of the temporary directory for running the image tailoring and customization tool is greater than 8 GB. The default temporary directory is**/tmp**. You can also use the `-t` parameter to specify another directory as the temporary directory. The path of the directory must be an absolute path. In this example, the **/home/temp** directory is used. The following command output indicates that the available drive space of the **/home** directory is 38 GB, which meets the requirements. + + ```shell + $ df -h + Filesystem Size Used Avail Use% Mounted on + devtmpfs 1.2G 0 1.2G 0% /dev + tmpfs 1.5G 0 1.5G 0% /dev/shm + tmpfs 1.5G 23M 1.5G 2% /run + tmpfs 1.5G 0 1.5G 0% /sys/fs/cgroup + /dev/mapper/openeuler_openeuler-root 69G 2.8G 63G 5% / + /dev/sda2 976M 114M 796M 13% /boot + /dev/mapper/openeuler_openeuler-home 61G 21G 38G 35% /home + ``` + +3. Tailor and customize the image. + + **Scenario 1**: All RPM packages of the new image are from the official ISO image. + + ```shell + $ sudo isocut -t /home/temp /home/isocut_iso/openEuler-20.03-LTS-SP3-aarch64-dvd.iso /home/result/new.iso + Checking input ... + Checking user ... + Checking necessary tools ... + Initing workspace ... + Copying basic part of iso image ... + Downloading rpms ... + Finish create yum conf + finished + Regenerating repodata ... + Checking rpm deps ... + Getting the description of iso image ... + Remaking iso ... + Adding checksum for iso ... + Adding sha256sum for iso ... + ISO cutout succeeded, enjoy your new image "/home/result/new.iso" + isocut.lock unlocked ... + ``` + + If the preceding information is displayed, the custom image **new.iso** is successfully created. + + **Scenario 2**: The RPM packages of the new image are from the official ISO image and additional packages in **/home/rpms**. + + ```shell + sudo isocut -t /home/temp -r /home/rpms /home/isocut_iso/openEuler-20.03-LTS-SP3-aarch64-dvd.iso /home/result/new.iso + ``` + + **Scenario 3**: The kickstart file is used for automatic installation. You need to modify the **/etc/isocut/anaconda-ks.cfg** file. + + ```shell + sudo isocut -t /home/temp -k /etc/isocut/anaconda-ks.cfg /home/isocut_iso/openEuler-20.03-LTS-SP3-aarch64-dvd.iso /home/result/new.iso + ``` diff --git a/docs/en/tools/community_tools/image_custom/isocut/public_sys-resources/icon-note.gif b/docs/en/tools/community_tools/image_custom/isocut/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/tools/community_tools/image_custom/isocut/public_sys-resources/icon-note.gif differ diff --git a/docs/en/tools/community_tools/migration/migration_tools/_toc.yaml b/docs/en/tools/community_tools/migration/migration_tools/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..9455b02a48de1d504c4966e5a82048c1395bbb06 --- /dev/null +++ b/docs/en/tools/community_tools/migration/migration_tools/_toc.yaml @@ -0,0 +1,4 @@ +label: migration-tools User Guide +isManual: true +href: ./migration-tools-user-guide.md +description: Migration from other OSs (CentOS 7/8) to UnionTech OS Server diff --git a/docs/en/tools/community_tools/migration/migration_tools/figures/environment-check.png b/docs/en/tools/community_tools/migration/migration_tools/figures/environment-check.png new file mode 100644 index 0000000000000000000000000000000000000000..b03c4da5ba24e345a3614cd2c7d7e3b52983ad1a Binary files /dev/null and b/docs/en/tools/community_tools/migration/migration_tools/figures/environment-check.png differ diff --git a/docs/en/tools/community_tools/migration/migration_tools/figures/home-page.png b/docs/en/tools/community_tools/migration/migration_tools/figures/home-page.png new file mode 100644 index 0000000000000000000000000000000000000000..2fb66ae7dc8336d6e38437ba79175fe1c2207a5d Binary files /dev/null and b/docs/en/tools/community_tools/migration/migration_tools/figures/home-page.png differ diff --git a/docs/en/tools/community_tools/migration/migration_tools/figures/kernel.png b/docs/en/tools/community_tools/migration/migration_tools/figures/kernel.png new file mode 100644 index 0000000000000000000000000000000000000000..ecd5bbb3cf306e46da3448de46c4f9fc2e03eed2 Binary files /dev/null and b/docs/en/tools/community_tools/migration/migration_tools/figures/kernel.png differ diff --git a/docs/en/tools/community_tools/migration/migration_tools/figures/license.png b/docs/en/tools/community_tools/migration/migration_tools/figures/license.png new file mode 100644 index 0000000000000000000000000000000000000000..41eb3b6aa755619b94965f8060a27c1940e6936e Binary files /dev/null and b/docs/en/tools/community_tools/migration/migration_tools/figures/license.png differ diff --git a/docs/en/tools/community_tools/migration/migration_tools/figures/migration-check.png b/docs/en/tools/community_tools/migration/migration_tools/figures/migration-check.png new file mode 100644 index 0000000000000000000000000000000000000000..776e9cafdf7e569cd33e1abd47217aa47c86f134 Binary files /dev/null and b/docs/en/tools/community_tools/migration/migration_tools/figures/migration-check.png differ diff --git a/docs/en/tools/community_tools/migration/migration_tools/figures/migration-complete.png b/docs/en/tools/community_tools/migration/migration_tools/figures/migration-complete.png new file mode 100644 index 0000000000000000000000000000000000000000..c832bb723ea5400aa2fe1f932f1f5dcb5a3d5065 Binary files /dev/null and b/docs/en/tools/community_tools/migration/migration_tools/figures/migration-complete.png differ diff --git a/docs/en/tools/community_tools/migration/migration_tools/figures/migration-confirmation.png b/docs/en/tools/community_tools/migration/migration_tools/figures/migration-confirmation.png new file mode 100644 index 0000000000000000000000000000000000000000..69567eabd886befe43fb8a512e7b6cde87fd0937 Binary files /dev/null and b/docs/en/tools/community_tools/migration/migration_tools/figures/migration-confirmation.png differ diff --git a/docs/en/tools/community_tools/migration/migration_tools/figures/migration-in-progress.png b/docs/en/tools/community_tools/migration/migration_tools/figures/migration-in-progress.png new file mode 100644 index 0000000000000000000000000000000000000000..e5bae64aaa303a6e7950ec0fdeb46a66bfaa70a3 Binary files /dev/null and b/docs/en/tools/community_tools/migration/migration_tools/figures/migration-in-progress.png differ diff --git a/docs/en/tools/community_tools/migration/migration_tools/figures/migration-start.png b/docs/en/tools/community_tools/migration/migration_tools/figures/migration-start.png new file mode 100644 index 0000000000000000000000000000000000000000..d8a2b58cd5ce8e559bd82c14400e16b4635d0ec3 Binary files /dev/null and b/docs/en/tools/community_tools/migration/migration_tools/figures/migration-start.png differ diff --git a/docs/en/tools/community_tools/migration/migration_tools/figures/migration-tools-conf.png b/docs/en/tools/community_tools/migration/migration_tools/figures/migration-tools-conf.png new file mode 100644 index 0000000000000000000000000000000000000000..80520c44a86172c9f18e3d50930e0fcc25f411bb Binary files /dev/null and b/docs/en/tools/community_tools/migration/migration_tools/figures/migration-tools-conf.png differ diff --git a/docs/en/tools/community_tools/migration/migration_tools/figures/openeuler-migration-complete.png b/docs/en/tools/community_tools/migration/migration_tools/figures/openeuler-migration-complete.png new file mode 100644 index 0000000000000000000000000000000000000000..20c5a4afaf2b06fae865137b9d4efd53326a9611 Binary files /dev/null and b/docs/en/tools/community_tools/migration/migration_tools/figures/openeuler-migration-complete.png differ diff --git a/docs/en/tools/community_tools/migration/migration_tools/figures/prompt.png b/docs/en/tools/community_tools/migration/migration_tools/figures/prompt.png new file mode 100644 index 0000000000000000000000000000000000000000..224a79aece026a4fa2b003753f40fc0f1ebd4d0d Binary files /dev/null and b/docs/en/tools/community_tools/migration/migration_tools/figures/prompt.png differ diff --git a/docs/en/tools/community_tools/migration/migration_tools/figures/repo.png b/docs/en/tools/community_tools/migration/migration_tools/figures/repo.png new file mode 100644 index 0000000000000000000000000000000000000000..78437bbc839bff8906b535989bbac0c38a6263b7 Binary files /dev/null and b/docs/en/tools/community_tools/migration/migration_tools/figures/repo.png differ diff --git a/docs/en/tools/community_tools/migration/migration_tools/figures/user-check.png b/docs/en/tools/community_tools/migration/migration_tools/figures/user-check.png new file mode 100644 index 0000000000000000000000000000000000000000..8ec0b5518f37532eedb5897bc368d9b58a1ccafc Binary files /dev/null and b/docs/en/tools/community_tools/migration/migration_tools/figures/user-check.png differ diff --git a/docs/en/tools/community_tools/migration/migration_tools/migration-tools-user-guide.md b/docs/en/tools/community_tools/migration/migration_tools/migration-tools-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..819845fe31eb071accbf7babc2c468de22f7644d --- /dev/null +++ b/docs/en/tools/community_tools/migration/migration_tools/migration-tools-user-guide.md @@ -0,0 +1,245 @@ +# migration-tools User Guide + +## Introduction + +This document outlines the usage of server migration software (migration-tools) for seamless migration from CentOS 7 and CentOS 8 systems to UnionTech OS Server (UOS). +The software features a web-based interface that simplifies the migration process through an intuitive graphical environment. + +### Deployment Method + +Install the server component on an openEuler 23.09 server and deploy the agent component on CentOS 7/CentOS 8 servers targeted for migration. + +#### Supported Systems for Migration + +1. Migration from AMD64 and AArch64 CentOS systems to UOS is supported. You need to prepare a complete repository for the target system before migration. + +2. openEuler migration: Only migration from CentOS 7.4 CUI to openEuler 20.03 LTS SP1 is supported. + +3. Systems with i686 architecture RPM packages should not be migrated as this will lead to migration failure. + +| Source System | Target System | Software Repository | +| ----------------- | ----------------------- | ------------------------------- | +| CentOS 7.4 CUI | openEuler 20.03 LTS SP1 | openEuler public repository | +| CentOS 7.0 to 7.7 | UOS 1002a | UOS 1002a (complete repository) | +| CentOS 8.0 to 8.2 | UOS 1050a | UOS 1050a (complete repository) | + +### Usage Instructions + +#### Installation and Configuration + +##### Installing migration-tools-server + +- Disable the firewall. + + ``` shell + systemctl stop firewalld + ``` + +- Install migration-tools-server. + + ``` shell + yum install migration-tools-server -y + ``` + +- Edit the configuration file. + + ``` shell + vim /etc/migration-tools/migration-tools.conf + ``` + + ![Configuration File](./figures/migration-tools-conf.png) + +- Restart the migration-tools-server service. + + ``` shell + systemctl restart migration-tools-server + ``` + +- Distribute the agent package. Choose the appropriate agent package based on the migration system version. + + For CentOS 7 series: + + Replace `xx.xx.xx.xx` with the migration machine's IP address. + + ``` shell + scp -r /usr/lib/migration-tools-server/agent-rpm/el7 root@xx.xx.xx.xx:/root + ``` + + For CentOS 8 series: + + ``` shell + scp -r /usr/lib/migration-tools-server/agent-rpm/el8 root@xx.xx.xx.xx:/root + ``` + +#### Migrating to openEuler + +> **Note:** openEuler migration currently supports only standalone script-based migration. + +- Distribute the migration script from the server to the agent. + + ``` shell + cd /usr/lib/migration-tools-server/ut-Migration-tools-0.1/centos7/ + scp openeuler/centos72openeuler.py root@10.12.23.106:/root + ``` + +- Install the required dependencies for migration. + + ``` shell + yum install python3 dnf rsync yum-utils -y + ``` + +- Begin the migration process. + + ``` shell + python3 centos7/openeuler/centos72openeuler.py + ``` + +- The system will automatically reboot after migration, and the process will be complete upon restart. + + ![openEuler Migration Complete](./figures/openeuler-migration-complete.png) + +#### Migrating to UOS + +##### Installing migration-tools-agent + +On the CentOS machine to be migrated, follow these steps: + +> **Note:** Currently, migration-tools only supports migration from CentOS 7.4 CUI to openEuler 20.03 LTS SP1. + +- Disable the firewall. + + ``` shell + systemctl stop firewalld + ``` + +- Install epel-release (some dependencies are included in the epel repository). + + ``` shell + yum install epel-release -y + ``` + +- Install the migration-tools-agent package (for CentOS 7 series, install the package corresponding to the architecture). + + For CentOS 7: + + ``` shell + cd /root/el7/x86_64 + yum install ./* -y + ``` + + For CentOS 8: + + ``` shell + cd /root/el8/ + yum install ./* -y + ``` + +- Edit the configuration file. + + ``` shell + vim /etc/migration-tools/migration-tools.conf + ``` + + ![Configuration File](./figures/migration-tools-conf.png) + +- Restart the migration-tools-agent service. + + ``` shell + systemctl restart migration-tools-agent + ``` + +##### UOS Migration Steps + +- Access the web interface. + + Once both the server and agent services are running, open a browser (Chrome is recommended) and navigate to `https://server_IP_address:9999`. + + ![Home Page](./figures/home-page.png) + +- Click "I have read and agree to this agreement," then proceed by clicking "Next." + ![License Agreement](./figures/license.png) + +- Review the migration prompt page and click "Next." + ![Prompt](./figures/prompt.png) + +- The environment check page will verify the system version and available disk space. Click "Next" once the check is complete. + +> **Note:** If the check stalls, ensure the agent firewall is disabled and both server and agent services are active. Refresh the browser to restart the check. + +![Environment Check](./figures/environment-check.png) + +- The user check page will validate the username and password. Using the root user is recommended. Click "Next" to initiate the check, and the system will automatically proceed to the repository configuration page upon completion. + + ![User Check](./figures/user-check.png) + +Repository Configuration Page: + +- Enter the appropriate repository path based on the system to be migrated. + + CentOS 7: 1002a, CentOS 8: 1050a + +- Ensure the repository is complete; otherwise, the migration will fail. + +- Only one repository path needs to be entered in the input field. + +![Repo](./figures/repo.png) + +- After entering the repository, click "Next." Once the repository connectivity check is complete, proceed to the kernel version selection page. Select the 4.19 kernel and click "Next." + + ![Kernel](./figures/kernel.png) + +- The migration environment check page compares software package differences before and after migration and generates a report. After the check, you can export the report. + + > **Note:** The check typically takes about one hour. Please wait patiently. + + ![Migration Check](./figures/migration-check.png) + +- After the check, click "Next." A confirmation window for system migration will appear. Ensure the system is backed up, then click "Confirm" to start the migration. + + ![Migration Confirmation](./figures/migration-confirmation.png) + +- After clicking "Confirm," the system migration page will appear. + + ![Migration Start](./figures/migration-start.png) + +- Click "View Details" to monitor the migration progress. + + ![Migration in Progress](./figures/migration-in-progress.png) + +- Once migration is complete, the page will redirect to the completion page. From here, you can export the migration analysis report and logs. + +- The exported files can be found in the **/var/tmp/uos-migration/** directory on the server. Unzip the files to view them. + + ![Migration Complete](./figures/migration-complete.png) + +- After migration, manually restart the agent machine and verify the migration status. + +###### Verification Steps + +Run the following command to verify if the OS has been successfully migrated to the target version. + +``` shell +uosinfo +``` + +If the output matches the expected information below, the migration is successful. + +1002a: + +``` shell +################################################# +Release: UnionTech OS Server release 20 (kongli) +Kernel : 4.19.0-91.77.97.uelc20.x86_64 +Build : UnionTech OS Server 20 1002c 20211228 x86_64 +################################################# +``` + +1050a: + +``` shell +################################################# +Release: UnionTech OS Server release 20 (kongzi) +Kernel : 4.19.0-91.82.88.uelc20.x86_64 +Build : UnionTech OS Server 20 1050a 20220214 x86_64 +################################################# +``` diff --git a/docs/en/tools/community_tools/virtualization/euler_launcher/_toc.yaml b/docs/en/tools/community_tools/virtualization/euler_launcher/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..ccfe9d49cf10e2135a40bfa6ebda56e06473e6ce --- /dev/null +++ b/docs/en/tools/community_tools/virtualization/euler_launcher/_toc.yaml @@ -0,0 +1,9 @@ +label: EulerLauncher User Guide +isManual: true +href: ./overall.md +description: High-performance development resources on mainstream desktop OSs. +sections: + - label: Installing and Running EulerLauncher on Windows + href: ./win-user-manual.md + - label: Installing and Running EulerLauncher on macOS + href: ./mac-user-manual.md diff --git a/docs/en/tools/community_tools/virtualization/euler_launcher/images/mac-content.jpg b/docs/en/tools/community_tools/virtualization/euler_launcher/images/mac-content.jpg new file mode 100644 index 0000000000000000000000000000000000000000..7db445ef7e6b068454e510bfd74f4e56ff523063 Binary files /dev/null and b/docs/en/tools/community_tools/virtualization/euler_launcher/images/mac-content.jpg differ diff --git a/docs/en/tools/community_tools/virtualization/euler_launcher/images/mac-install.jpg b/docs/en/tools/community_tools/virtualization/euler_launcher/images/mac-install.jpg new file mode 100644 index 0000000000000000000000000000000000000000..07945fa95430a28993b65410881a248a62845bcb Binary files /dev/null and b/docs/en/tools/community_tools/virtualization/euler_launcher/images/mac-install.jpg differ diff --git a/docs/en/tools/community_tools/virtualization/euler_launcher/images/mac-start.jpg b/docs/en/tools/community_tools/virtualization/euler_launcher/images/mac-start.jpg new file mode 100644 index 0000000000000000000000000000000000000000..d950b29c99438703a5dd670953c9afc30feb9494 Binary files /dev/null and b/docs/en/tools/community_tools/virtualization/euler_launcher/images/mac-start.jpg differ diff --git a/docs/en/tools/community_tools/virtualization/euler_launcher/images/mac-terminal.jpg b/docs/en/tools/community_tools/virtualization/euler_launcher/images/mac-terminal.jpg new file mode 100644 index 0000000000000000000000000000000000000000..d0f3082cc64658afd888a02ca578f76857578f83 Binary files /dev/null and b/docs/en/tools/community_tools/virtualization/euler_launcher/images/mac-terminal.jpg differ diff --git a/docs/en/tools/community_tools/virtualization/euler_launcher/images/mac-visudo.jpg b/docs/en/tools/community_tools/virtualization/euler_launcher/images/mac-visudo.jpg new file mode 100644 index 0000000000000000000000000000000000000000..7466ef4fec0f928ca0c0255ba59b628251216eb1 Binary files /dev/null and b/docs/en/tools/community_tools/virtualization/euler_launcher/images/mac-visudo.jpg differ diff --git a/docs/en/tools/community_tools/virtualization/euler_launcher/mac-user-manual.md b/docs/en/tools/community_tools/virtualization/euler_launcher/mac-user-manual.md new file mode 100644 index 0000000000000000000000000000000000000000..979f9ea1d3a8eb33b2437d312266fa9a5728447c --- /dev/null +++ b/docs/en/tools/community_tools/virtualization/euler_launcher/mac-user-manual.md @@ -0,0 +1,234 @@ +# Installing and Running EulerLauncher on macOS + +## Preparations + +### Installing Homebrew + +Homebrew is the software package manager for macOS, which provides many practical functions, such as installation, uninstallation, update, viewing, and search. It allows you to use a simple command to manage packages without considering dependencies and file paths. + +On the macOS desktop, press **Shift**+**Command**+**U** to open the **Utilities** folder in **Go** and find **Terminal.app**. + +![](./images/mac-terminal.jpg) + +Install Homebrew based on the network status. + +Run the following command to install Homebrew: + +``` Shell +/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" +``` + +Alternatively, run the following command to install Homebrew: + +``` Shell +/bin/zsh -c "$(curl -fsSL https://gitee.com/cunkai/HomebrewCN/raw/master/Homebrew.sh)" +``` + +### Installing QEMU and Wget + +EulerLauncher depends on QEMU to run on macOS, and image download depends on Wget. Homebrew can be used to easily download and manage such software. Run the following command to install QEMU and Wget: + +``` Shell +brew install qemu +brew install wget +``` + +### Configuring the sudo Password-Free Permission + +EulerLauncher depends on QEMU to run on macOS. To improve user network experience, [vmnet framework][1] of macOS is used to provide VM network capabilities. Currently, administrator permissions are required for using vmnet. When using the QEMU backend to create VMs with vmnet network devices, you need to enable the administrator permission. EulerLauncher automatically uses the `sudo` command to implement this process during startup. Therefore, you need to configure the `sudo` password-free permission for the current user. If you do not want to perform this configuration, please stop using EulerLauncher. + +1. On the macOS desktop, press **Shift**+**Command**+**U** to open the **Utilities** folder in **Go** and find **Terminal.app**. + + ![](./images/mac-terminal.jpg) + +2. Enter `sudo visudo` in the terminal to modify the sudo configuration file. Note that you may be required to enter the password in this step. Enter the password as prompted. + +3. Find and replace `%admin ALL=(ALL) ALL` with `%admin ALL=(ALL) NOPASSWD: ALL`. + + ![](./images/mac-visudo.jpg) + +4. Press **ESC** and enter **:wq** to save the settings. + +## Installing EulerLauncher + +EulerLauncher supports macOS Ventura for Apple Silicon and x86 architectures. [Download the latest version of EulerLauncher][1] for macOS and decompress it to the desired location. + +The directory generated after the decompression contains the following files: + +![](./images/mac-content.jpg) + +**install.exe** is the installation file, which is used to install the support files required by EulerLauncher to the specified location. **EulerLauncher.dmg** is the disk image of the main program. + +1. This operation requires the `sudo` permission. You need to [configure the sudo password-free permission](#configuring-the-sudo-password-free-permission) first. +2. Install the support files. Double-click the **install** file and wait until the program execution is completed. + +3. Configure EulerLauncher. + + - Check the locations of QEMU and Wget. The name of the QEMU binary file varies according to the architecture. Select the correct name (Apple Silicon: **qemu-system-aarch64**; Intel: **qemu-system-x86_64**) as required. + + ``` Shell + which wget + which qemu-system-{host_arch} + ``` + + Reference output: + + ```shell + /opt/homebrew/bin/wget + /opt/homebrew/bin/qemu-system-aarch64 + ``` + + Record the paths, which will be used in the following steps. + + - Open the **eulerlauncher.conf** file and configure it. + + ```shell + sudo vi /Library/Application\ Support/org.openeuler.eulerlauncher/eulerlauncher.conf + ``` + + EulerLauncher configurations are as follows: + + ```ini + [default] + log_dir = # Log file location (xxx.log) + work_dir = # EulerLauncher working directory, which is used to store VM images and VM files. + wget_dir = # Path of the Wget executable file. Set this parameter based on the previous step. + qemu_dir = # Path of the QEMU executable file. Set this parameter based on the previous step. + debug = True + + [vm] + cpu_num = 1 # Number of CPUs of the VM. + memory = 1024 #Memory size of the VM, in MB. Do not set this parameter to a value greater than 2048 for M1 users. + ``` + + Save the modifications and exit. + +4. Install **EulerLauncher.app**. + + - Double-click **EulerLauncher.dmg**. In the displayed window, drag **EulerLauncher.app** to **Applications** to complete the installation. You can find **EulerLauncher.app** in applications. + + ![](./images/mac-install.jpg) + +## Using EulerLauncher + +1. Find **EulerLauncher.app** in applications and click to start the program. + +2. EulerLauncher needs to access the network. When the following dialog box is displayed, click **Allow**. + + ![](./images/mac-start.jpg) + +3. Currently, EulerLauncher can be accessed only in CLI mode. Open **Terminal.app** and use the CLI to perform operations. + +### Operations on Images + +1. List available images. + + ```Shell + eulerlauncher images + ``` + + There are two types of EulerLauncher images: remote images and local images. Only local images in the **Ready** state can be used to create VMs. Remote images can be used only after being downloaded. You can also load a downloaded local image to EulerLauncher. For details, see the following sections. + +2. Download a remote image. + + ```Shell + eulerlauncher download-image 23.09 + + Downloading: 23.09, this might take a while, please check image status with "images" command. + ``` + + The image download request is an asynchronous request. The download is completed in the background. The time required depends on your network status. The overall image download process includes download, decompression, and format conversion. During the download, you can run the `image` command to view the download progress and image status at any time. + + ```Shell + eulerlauncher images + ``` + + When the image status changes to **Ready**, the image is downloaded successfully. The image in the **Ready** state can be used to create VMs. + + ```Shell + eulerlauncher images + ``` + +3. Load a local image. + + Load a custom image or an image downloaded to the local host to EulerLauncher to create a custom VM. + + ```Shell + eulerlauncher load-image --path {image_file_path} IMAGE_NAME + ``` + + The supported image formats are *xxx***.qcow2.xz** and *xxx***.qcow2**. + + Example: + + ```Shell + eulerlauncher load-image --path /opt/openEuler-23.09-x86_64.qcow2.xz 2309-load + + Loading: 2309-load, this might take a while, please check image status with "images" command. + ``` + + Load the **openEuler-23.09-x86_64.qcow2.xz** file in the **/opt** directory to the EulerLauncher system and name it **2309-load**. Similar to the download command, the load command is also an asynchronous command. You need to run the image list command to query the image status until the image status is **Ready**. Compared with directly downloading an image, loading an image is much faster. + + ```Shell + eulerlauncher images + ...... + + eulerlauncher images + ...... + ``` + +4. Delete an image. + + Run the following command to delete an image from the EulerLauncher system: + + ```Shell + eulerlauncher delete-image 2309-load + + Image: 2309-load has been successfully deleted. + ``` + +### Operations on VMs + +1. List VMs. + + ```shell + eulerlauncher list + + +----------+-----------+---------+---------------+ + | Name | Image | State | IP | + +----------+-----------+---------+---------------+ + | test1 | 2309-load | Running | 172.22.57.220 | + +----------+-----------+---------+---------------+ + | test2 | 2309-load | Running | N/A | + +----------+-----------+---------+---------------+ + ``` + + If the VM IP address is **N/A** and the VM status is **Running**, the VM is newly created and the network configuration is not complete. Configuring the network takes several seconds. You can obtain the VM information again later. + +2. Log in to a VM. + + If an IP address has been assigned to a VM, you can run the `ssh` command to log in to the VM. + + ```Shell + ssh root@{instance_ip} + ``` + + If the official image provided by the openEuler community is used, the default username is **root** and the default password is **openEuler12#$**. + +3. Create a VM. + + ```Shell + eulerlauncher launch --image {image_name} {instance_name} + ``` + + Use `--image` to specify an image and a VM name. + +4. Delete a VM. + + ```Shell + eulerlauncher delete-instance {instance_name} + ``` + +Delete a specified VM based on the VM name. + +[1]: https://developer.apple.com/documentation/vmnet diff --git a/docs/en/tools/community_tools/virtualization/euler_launcher/overall.md b/docs/en/tools/community_tools/virtualization/euler_launcher/overall.md new file mode 100644 index 0000000000000000000000000000000000000000..bb672f4915b578d19f5364b367384e634b8b37d5 --- /dev/null +++ b/docs/en/tools/community_tools/virtualization/euler_launcher/overall.md @@ -0,0 +1,18 @@ +# EulerLauncher + +EulerLauncher is a developer tool set incubated by the technical operation team and infrastructure team of the openEuler community. It integrates virtualization technologies (such as LXD, HyperV, and Virtualization Framework) in mainstream desktop operating systems (OSs). It utilizes VMs and container images officially released by the openEuler community to provide developers with unified development resource (such as VMs and containers) provisioning and management experience on Windows, macOS, and Linux, improving the convenience and stability of using the openEuler development environment on mainstream desktop OSs, as well as developer experience. + +## Background + +The convenience and stability of development resources (such as VMs and containers) provided by mainstream desktop OSs are important factors that affect the experience of openEuler developers, especially for individuals and university developers with limited development resources. Common VM management platforms have many limitations. For example, VirtualBox requires developers to download a large ISO image and install the OS, WSL cannot provide a real openEuler kernel, most VM management software does not fully support Apple Sillicon chips, and many software needs to be paid, greatly reducing developers' work efficiency. + +EulerLauncher provides a convenient, easy-to-use, and unified developer tool set on mainstream desktop OSs such as Windows, macOS, and Linux (planned). It supported the x86_64 and AArch64 hardware architectures, including Apple Sillicon chips. It also supports virtual hardware acceleration capabilities for different platforms, providing developers with high-performance development resources. EulerLauncher allows users to use VMs and container images (planned) released by the openEuler community, Daily Build images provided by the openEuler community, and other custom images that meet requirements, providing developers with multiple choices. + +## Quick Start + +For macOS users, see [Installing and Running EulerLauncher on macOS][1]. + +For Windows users, see [Installing and Running EulerLauncher in Windows][2]. + +[1]: ./mac-user-manual.md +[2]: ./win-user-manual.md diff --git a/docs/en/tools/community_tools/virtualization/euler_launcher/win-user-manual.md b/docs/en/tools/community_tools/virtualization/euler_launcher/win-user-manual.md new file mode 100644 index 0000000000000000000000000000000000000000..aa74f01b92d287da7c163e96f49ff6d862e47b32 --- /dev/null +++ b/docs/en/tools/community_tools/virtualization/euler_launcher/win-user-manual.md @@ -0,0 +1,159 @@ +# Installing and Running EulerLauncher on Windows + +EulerLauncher currently supports Windows 10/11. [Download the latest version of EulerLauncher][1] for Windows and decompress it to the desired location. +Right-click **config-env.bat** and choose **Run as administrator** from the shortcut menu. This script adds the current directory to the system environment variable `PATH`. If you know how to configure environment variables or the configuration script is incorrect, you can manually add the directory where the current script is located and the **qemu-img** subdirectory to the system environment variable `PATH`. + +To run EulerLauncher on Windows, you need to connect EulerLauncher to the Hyper-V virtualization backend. Hyper-V is a Microsoft hardware virtualization product that provides better performance for VMs on Windows. Before running EulerLauncher, check whether Hyper-V is enabled in your system. For details about how to check and enable Hyper-V, see [Install Hyper-V][2] or other network resources. + +The directory generated after the decompression contains the following files: + +- **eulerlauncherd.exe**: main process of EulerLauncher. It is a daemon process running in the background. It interacts with various virtualization backends and manages the life cycles of VMs, containers, and images. +- **eulerlauncher.exe**: EulerLauncher CLI client. You can use this client to interact with the eulerlauncherd daemon process and perform operations on VMs and images. +- **eulerlauncher.conf**: EulerLauncher configuration file, which must be stored in the same directory as **eulerlauncherd.exe**. Configure the file as follows: + +```ini +[default] +# Configure the directory for storing log files. +log_dir = D:\eulerlauncher-workdir\logs +# Whether to enable the debug log level. +debug = True +# Configure the EulerLauncher working directory. +work_dir = D:\eulerlauncher-workdir +# Configure the EulerLauncher image directory. The image directory is relative to the working directory. +image_dir = images +# Configure the VM file directory of EulerLauncher. The VM file directory is relative to the working directory. +instance_dir = instances +``` + +After the configuration is complete, right-click **eulerlauncherd.exe** and choose **Run as administrator** from the shortcut menu. **eulerlauncherd.exe** runs as a daemon process in the background. + +Start PowerShell or Command Prompt to prepare for the corresponding operation. + +## Exiting the eulerlauncherd Background Process on Windows + +After **eulerlauncherd.exe** is executed, the eulerlauncherd icon is displayed in the system tray in the lower right corner. Right-click the icon and choose **Exit EulerLauncher** from the shortcut menu to exit the eulerlauncherd background process. + +## Operations on Images + +1. List available images. + + ```PowerShell + eulerlauncher.exe images + + +-----------+----------+--------------+ + | Images | Location | Status | + +-----------+----------+--------------+ + | 22.03-LTS | Remote | Downloadable | + | 21.09 | Remote | Downloadable | + | 2203-load | Local | Ready | + +-----------+----------+--------------+ + ``` + + There are two types of EulerLauncher images: remote images and local images. Only local images in the **Ready** state can be used to create VMs. Remote images can be used only after being downloaded. You can also load a downloaded local image to EulerLauncher. For details, see the following sections. + +2. Download a remote image. + + ```PowerShell + eulerlauncher.exe download-image 23.09 + + Downloading: 23.09, this might take a while, please check image status with "images" command. + ``` + + The image download request is an asynchronous request. The download is completed in the background. The time required depends on your network status. The overall image download process includes download, decompression, and format conversion. During the download, you can run the `image` command to view the download progress and image status at any time. + + ```PowerShell + eulerlauncher.exe images + ``` + + When the image status changes to **Ready**, the image is downloaded successfully. The image in the **Ready** state can be used to create VMs. + + ```PowerShell + eulerlauncher.exe images + ``` + +3. Load a local image. + + Load a custom image or an image downloaded to the local host to EulerLauncher to create a custom VM. + + ```PowerShell + eulerlauncher.exe load-image --path {image_file_path} IMAGE_NAME + ``` + + The supported image formats are *xxx***.qcow2.xz** and *xxx***.qcow2**. + + Example: + + ```PowerShell + eulerlauncher.exe load-image --path D:\openEuler-23.09-x86_64.qcow2.xz 2309-load + + Loading: 2309-load, this might take a while, please check image status with "images" command. + ``` + + Load the **openEuler-23.09-x86_64.qcow2.xz** file in the **D:\\** directory to the EulerLauncher system and name it **2309-load**. Similar to the download command, the load command is also an asynchronous command. You need to run the image list command to query the image status until the image status is **Ready**. Compared with directly downloading an image, loading an image is much faster. + + ```PowerShell + eulerlauncher.exe images + + ...... + + eulerlauncher images + + ...... + ``` + +4. Delete an image. + + Run the following command to delete an image from the EulerLauncher system: + + ```PowerShell + eulerlauncher.exe delete-image 2309-load + + Image: 2309-load has been successfully deleted. + ``` + +## Operations on VMs + +1. List VMs. + + ```Powershell + eulerlauncher.exe list + + +----------+-----------+---------+---------------+ + | Name | Image | State | IP | + +----------+-----------+---------+---------------+ + | test1 | 2309-load | Running | 172.22.57.220 | + +----------+-----------+---------+---------------+ + | test2 | 2309-load | Running | N/A | + +----------+-----------+---------+---------------+ + ``` + + If the VM IP address is **N/A** and the VM status is **Running**, the VM is newly created and the network configuration is not complete. Configuring the network takes several seconds. You can obtain the VM information again later. + +2. Log in to a VM. + + If an IP address has been assigned to a VM, you can run the `ssh` command to log in to the VM. + + ```PowerShell + ssh root@{instance_ip} + ``` + + If the official image provided by the openEuler community is used, the default username is **root** and the default password is **openEuler12#$**. + +3. Create a VM. + + ```PowerShell + eulerlauncher.exe launch --image {image_name} {instance_name} + ``` + + Use `--image` to specify an image and a VM name. + +4. Delete a VM. + + ```PowerShell + eulerlauncher.exe delete-instance {instance_name} + ``` + + Delete a specified VM based on the VM name. + +[1]: https://gitee.com/openeuler/eulerlauncher/releases +[2]: https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/quick-start/enable-hyper-v diff --git a/docs/en/tools/desktop/.keep b/docs/en/tools/desktop/.keep new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/docs/en/tools/desktop/_toc.yaml b/docs/en/tools/desktop/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..d183fccae2e3ce151ff12a76a5b57899fa5abd27 --- /dev/null +++ b/docs/en/tools/desktop/_toc.yaml @@ -0,0 +1,7 @@ +label: Desktop Environments +sections: + - href: ./gnome/_toc.yaml + - href: ./ukui/_toc.yaml + - href: ./dde/_toc.yaml + - href: ./kiran/_toc.yaml + - href: ./xfce/_toc.yaml diff --git a/docs/en/tools/desktop/dde/_toc.yaml b/docs/en/tools/desktop/dde/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..ad0382e929d3ca0ecfc0829deccb76741fabddf0 --- /dev/null +++ b/docs/en/tools/desktop/dde/_toc.yaml @@ -0,0 +1,10 @@ +label: DDE User Guide +isManual: true +description: Install and use DDE. +sections: + - label: DDE Installation + href: ./dde-installation.md + - label: DDE User Guide + href: ./dde-user-guide.md + - label: Common Issues and Solutions + href: ./dde-common-issues-and-solutions.md diff --git a/docs/en/tools/desktop/dde/dde-common-issues-and-solutions.md b/docs/en/tools/desktop/dde/dde-common-issues-and-solutions.md new file mode 100644 index 0000000000000000000000000000000000000000..480c34548b0d9f1fed584a3e3c3555ee01674c32 --- /dev/null +++ b/docs/en/tools/desktop/dde/dde-common-issues-and-solutions.md @@ -0,0 +1,19 @@ +# Common Issues and Solutions + +## Issue 1: After DDE Is Installed, Why Are the Computer and Recycle Bin Icons Not Displayed on the Desktop When I Log in as the **root** User + +### Issue + +After the DDE is installed, the computer and recycle bin icon is not displayed on the desktop when a user logs in as the **root** user. + +![img](./figures/dde-1.png) + +### Cause + +The **root** user is created before the DDE is installed. During the installation, the DDE does not add desktop icons for existing users. This issue does not occur if the user is created after the DDE is installed. + +### Solution + +Right-click the icon in the launcher and choose **Send to Desktop**. The icon functions the same as the one added by DDE. + +![img](./figures/dde-2.png) diff --git a/docs/en/tools/desktop/dde/dde-installation.md b/docs/en/tools/desktop/dde/dde-installation.md new file mode 100644 index 0000000000000000000000000000000000000000..80b1e26de882060a6597193ebfc5c6c31d4390b5 --- /dev/null +++ b/docs/en/tools/desktop/dde/dde-installation.md @@ -0,0 +1,39 @@ +# DDE Installation + +## Introduction + +DDE is a powerful desktop environment developed by UnionTech. It contains dozens of self-developed desktop applications. + +## Procedure + +1. [Download](https://openeuler.org/en/download/) the openEuler ISO file and install the OS. +2. Update the software source. + + ```bash + sudo dnf update + ``` + +3. Install DDE. + + ```bash + sudo dnf install dde + ``` + +4. Set the system to start with the graphical interface. + + ```bash + sudo systemctl set-default graphical.target + ``` + +5. Reboot the system. + + ```bash + sudo reboot + ``` + +6. After the reboot is complete, use the user created during the installation process or the **openeuler** user to log in to the desktop. + + > DDE does not allow login as the **root** user. + > DDE has a built-in **openeuler** user whose password is **openeuler**. + +Now you can use DDE. diff --git a/docs/en/tools/desktop/dde/dde-user-guide.md b/docs/en/tools/desktop/dde/dde-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..b33fab70dd22c3096fb3a71a6163b52866f3d996 --- /dev/null +++ b/docs/en/tools/desktop/dde/dde-user-guide.md @@ -0,0 +1,849 @@ +# DDE Desktop Environment + +## Overview + +DDE desktop environment is an elegant, secure, reliable and easy to use GUI comprised of the desktop, dock, launcher and control center. Acting as the key basis for our operating system, its main interface is shown as below. + +![](./figures/43.jpg) + +### Getting Started + +When you enter DDE for the very first time, a welcome program will automatically start. You can watch the introduction video, select your desktop style and icon theme, and learn more about the system functions. + +![](./figures/46.png) + +## Desktop + +Desktop is the main screen you see after logging in. On the desktop, you can create a new file/folder, sort files, open in terminal, set wallpaper and screensaver and etc. You can also add shortcuts for applications on desktop by using [Send to desktop](#set-app-shortcut). + +![](./figures/41.png) + +### Create New Folder/Document + +Just as in File Manager, you can create a new folder/document on the desktop, or do some operations for the files on it. + +- Right-click the desktop, select **New folder** and enter the name for it. +- Right-click the desktop, select **New document**, select the type and enter its name. + +Right-click a file or folder on the desktop, and use the features of File Manager as below: + +| Function | Description | +| ---------------- | ------------------------------------------------------------ | +| Open with | Select an app to open it. | +| Cut | Move it to another location. | +| Copy | Copy it to another location. | +| Rename | Change its name. | +| Delete | Delete and move it to the trash. | +| Create link | Create a shortcut of the file or folder. | +| Tag information | Add a tag. | +| Compress/Extract | Compress the file or folder, or extract the compressed file. | +| Properties | View the basic info, share it or change the permission. | + +### Sort Files + +Sort the files on your desktop to make it organized and fit your needs. + +1. Right-click the desktop. +2. Click **Sort by**, you can: + +- Click **Name** to display files in the name sequence. +- Click **Size** to display files in the size sequence. +- Click **Type** to display files in type. +- Click **Time modified** to display files in the order of last modified date. + +> ![](./figures/icon125-o.svg)Tips: *Check **Auto arrange**, icons on the desktop will be listed in order automatically, and if an icon is removed, another one will fill in the blank.* + +### Adjust Icon Size + +1. Right-click the desktop. +2. Click **Icon size**, and choose a proper size. + +> ![](./figures/icon125-o.svg)Tips: *Press **Ctrl** + ![](./figures/icon134-o.svg)/![](./figures/icon132-o.svg) scrolling mouse wheel to adjust icon size on the desktop and in Launcher.* + +### Set Display + +You can set display scaling, screen resolution, brightness and so on from the desktop. + +1. Right-click the desktop. +2. Click **Display Settings** to open the settings in Control Center. + +> ![](./figures/icon99-o.svg)Notes: *For specific operations, please refer to [Display](#Display).* + +### Change Wallpaper + +Select some elegant and fashionable wallpapers to beautify your desktop and make it distinctive. + +1. Right-click the desktop. +2. Click **Wallpaper and Screensaver** to preview all the wallpapers. +3. Click your favorite one and it will apply in your desktop and screen lock. +4. You can also choose **Only desktop**, **Only lock screen**, or both. + +![](./figures/63.jpg) + +> ![](./figures/icon125-o.svg)Tips: *You can also set your favorite picture as wallpaper in an image viewer.* + +### Clipboard + +All the texts, pictures and documents cut and copied by the current user after login are displayed in the clipboard, which can be copied quickly by double-clicking the clipboard. The clipboard is cleared automatically after logout and shutdown. + +1. Use the shortcuts **Ctrl**+**Alt**+ **V** to wake up the clipboard. + +2. Double-click in the clipboard to copy the current content quickly and the corresponding block will be moved to the top of the clipboard. + +3. Select the target destination to paste it. + +4. Click![](./figures/icon57-o.svg)to delete the current content and click **Clear All** to clear the clipboard. + + ![](./figures/40.png) + +## Dock + +Dock is at the bottom of the desktop by default to help you quickly open frequently-used applications, which includes Launcher, applications, system tray, and plugins. In the dock, you can open launcher, show the desktop, enter the workspaces, open and exit apps, set input methods, adjust the volume, connect to the network, view the calendar and enter the shutdown interface, and so on. + +### Icons on Dock + +In the Dock, there are icons of Launcher, applications, system tray, and plugins. + +![](./figures/45.png) + +| Icon | Description | +| ---- | ---- | +| ![](./figures/icon66-o.svg) | Launcher - click to view all the installed applications. | +| ![](./figures/icon69-o.svg) | Click to show the desktop. | +| ![](./figures/icon63-o.svg) | File Manager - click to view files and folders on the disk. | +| ![](./figures/icon62-o.svg) | Calendar - view dates and create new schedules. | +| ![](./figures/icon58-o.svg) | Control Center - click to check or change system settings. | +| ![](./figures/icon101-o.svg) | Notification Center - show all notifications from the system and applications. | +| ![](./figures/icon103-o.svg) | Onboard virtual keyboard. | +| ![](./figures/icon122-o.svg) | Click to enter the shutdown interface. | +| ![](./figures/icon126-o.svg) | Trash. | + +> ![](./figures/icon125-o.svg)Tips: *In Efficient Mode, you can click the right side of Dock to show the desktop. Move the cursor to the running app in the Dock and you will see its preview window.* + +### Switch Display Mode + +There are two display modes of Dock: fashion mode and efficient mode, icon sizes are different in them. + +![](./figures/46.png) + +![](./figures/63.png) + +You can switch the display modes by the following operations: + +1. Right-click the Dock and select **Mode**. +2. Select the display mode. + +### Change Dock Location + +You can place Dock on any direction of your desktop. + +1. Right-click the Dock and select **Location**. +2. Select a location. + +### Change Dock Height + +Drag the top edge to increase or decrease the height. + +### Show/Hide Plugins + +1. Right-click the Dock and select **Plugins**. +2. On the submenu, you can check or uncheck **Trash, Power, Show Desktop, Onboard**, and **Datetime** to show or hide the corresponding icon in the Dock. + +### View Notifications + +When there are system or application notifications, they will be shown in the middle of the screen. If there are buttons in the message, click buttons to do the actions; if there are not, click the message to close it. + +![](./figures/51.png) + +Click notification in Dock to view all the notifications. + +### View Date and Time + +- Hover the cursor over the Time icon in Dock to view the current time, date and day of the week. +- Click the Time icon to open Calendar. + +### Enter Shutdown Interface + +There are two ways to enter the shutdown interface: + +- Click ![](./figures/icon122-o.svg) in Dock. +- Click ![](./figures/icon136-o.svg) at the bottom right corner of Launcher mini mode. + +| Function | Description | +| ------------------------------------------------------------ | ------------------------------------------------------------ | +| Shut down ![](./figures/icon136-o.svg) | Shut down the computer. | +| Reboot ![](./figures/icon110-o.svg) | Restart the computer. | +| Lock ![](./figures/icon90-o.svg) | Lock the computer with the password. Or press **Super** + **L** to lock it. | +| Switch user ![](./figures/icon128-o.svg) | Log in with another user account. | +| Log out ![](./figures/icon92-o.svg) | End all the processes and initialize the system. | +| Start system monitor ![](./figures/icon68-o.svg) | View the running processes and end the one you want. | + +> ![](./figures/icon99-o.svg)Notes: ![](./figures/icon128-o.svg) *will be shown if there are multiple accounts in the system.* + +### Trash + +You can find all deleted files in the trash, which can be restored or emptied. + +#### Restore Files + +You can restore deleted files in Trash or press **Ctrl** + **Z** to restore the lately deleted files. + +1. Select the file in the trash. +2. Right-click the file and select **Restore**. +3. The file will be in its original path. + +> ![](./figures/icon52-o.svg)Attention: *If the original folder of the file has been deleted, the deleted file will be restored to a new folder automatically created.* + +#### Empty Trash + +In the trash, click **Empty** to permanently delete all the files in the trash. + +## Launcher + +Launcher ![](./figures/icon66-o.svg) helps you manage all the installed applications, where you can quickly find an application by category navigation or by a search. + +> ![](./figures/icon125-o.svg)Tips: *You can view newly installed applications in Launcher. The newly-installed ones are followed with a blue dot.* + +### Switch Launcher Modes + +There are two display modes of Launcher: fullscreen mode and mini mode. Click the icon at the upper right corner to switch modes. + +Both modes support searching applications and sending them to the desktop or Dock. + +The mini mode also supports opening File Manager, Control Center and shutdown interface directly. + +![](./figures/47.jpg) +![](./figures/52.png) + +### Sort Applications + +In fullscreen mode, all applications in Launcher are listed by the installation time by default. You can sort the application icons as the ways below: + +- Hover the cursor over an application icon, hold down the left key of mouse, drag and drop the application icon to arrange it freely. +- Click the category icon ![](./figures/icon56-o.svg) on the upper left in Launcher to arrange the icons by category. + +![](./figures/60.jpg) + +In mini mode, applications are displayed according to using frequency by default. + +### Find Applications + +In Launcher, you can scroll up and down to find an application, or locate it with the category navigation. + +If you already know the application name, just search for it. + +### Set App Shortcut + +The shortcut offers a method to run applications easily and quickly. + +#### Create App Shortcut + +Send the application icon to the desktop or Dock to facilitate the follow-up operations. + +In Launcher, right-click an app icon and you can: + +- Select **Send to desktop** to create a shortcut on the desktop. +- Select **Send to dock** to fix the application icon in Dock. + +![](./figures/58.png) + +> ![](./figures/icon99-o.svg)Notes: *You can drag the application icon from Launcher to Dock. But you cannot drag and drop the application while it is running. Then you can right-click the application icon in Dock and select **Dock** to fix it in order to open it quickly for the next time.* + +#### Delete Shortcut + +Delete a shortcut from the desktop directly, or remove it from Dock or Launcher. + +**Remove the shortcut from Dock:** + +- Hold down the left key of mouse, drag and drop the icon away from Dock. +- You cannot drag and drop the application icon while it is running. Then you can right-click the application icon in Dock and select **Undock** to remove it from Dock. + +**Remove the shortcut from Launcher:** + +In Launcher, right-click the icon and you can: + +- Select **Remove from desktop** to delete the shortcut from the desktop. +- Select **Remove from dock** to remove the application icon from Dock. + +> ![](./figures/icon99-o.svg)Notes: *The above operations only delete the shortcut rather than uninstall the applications.* + +### Run Applications + +For the applications whose shortcuts have been created on the desktop or Dock, you can open them in the following ways: + +- Double-click the desktop icon or right-click it and select **Open**. +- Click the application icon in Dock or right-click it and select **Open**. + +To open the application only shown in Launcher, click the icon or right-click it and select **Open**. + +> ![](./figures/icon125-o.svg)Tips: *For the frequently-used applications, right-click the app icon and select **Add to startup** to run it when the computer boots.* + +## Control Center + +You can manage the system settings in Control Center, including account management, network settings, date and time, personalization, display settings, etc. After entering the desktop environment, click ![](./figures/icon58-o.svg) to open Control Center. + +### Homepage Introduction + +The homepage of Control Center provides several setting modules and click one to enter the detailed settings. + +![](./figures/42.png) + +Once you open a setting module in Control Center, the navigation appears on the left. Click the left navigation to quickly switch to other settings. + +![](./figures/39.png) + +#### Title Bar + +The title bar contains the back button, search box, main menu and the window buttons. + +- Back button: Click ![](./figures/icon53-o.svg) to go back to the homepage. +- Search box: Input a keyword and search the related settings. +- Main menu: Click ![](./figures/icon83-o.svg) to enter the main menu where you can set the window theme, view the manual and exit. + +### Accounts + +You have already created an account when installing the system. Here you can modify account settings or create a new one. + +![](./figures/38.png) + +#### Create New Account + +1. On the homepage of Control Center, click ![](./figures/icon49-o.svg). +2. Click ![](./figures/icon50-o.svg). +3. Input a username and a password twice. +4. Click **Create**. +5. Input the password of the current user account in the authentication dialog box, and the new account will be added to the account list. + +#### Change Account Avatar + +1. On the homepage of Control Center, click ![](./figures/icon49-o.svg). +2. Click an existing account in the list. +3. Click the user avatar. +4. Select a avatar or upload a local avatar. + +#### Set Full Name + +The account full name is shown in account list and system login interface and you can set it as needed. + +1. On the homepage of Control Center, click ![](./figures/icon49-o.svg). +2. Click an existing account in the list. +3. Click ![](./figures/icon75-o.svg) after **Full Name**, and input a name. + +#### Change Password + +1. On the homepage of Control Center, click ![](./figures/icon49-o.svg). + +2. Click the current account. + +3. Click **Change Password**. + +4. Input a new password twice and confirm. + +#### Delete Account + +1. On the homepage of Control Center, click ![](./figures/icon49-o.svg). +2. Click an account that's not logged in. +3. Click **Delete Account**. +4. Click **Delete** in the pop-up window. + +> ![](./figures/icon52-o.svg)Attention: *The logged in account cannot be deleted.* + +#### Privilege + +The first account has administrator privilege when you install the system. All other accounts you add after that are common users. One account can be grouped in many user groups. + +##### Group setting + +When you add or modify accounts, you can: + +- Select a group existing in the system. +- Select the group with the same name as the current user. +- Select the group with the same name as another user when the account was previously added. + +### Display + +Set screen resolution, brightness, direction and display scaling properly to have the best visual effect. + +![](./figures/44.png) + +#### Single Screen Settings + +##### Change Resolution + +1. On the homepage of Control Center, click ![](./figures/icon72-o.svg). +2. Click **Resolution**. +3. Select a proper resolution in the list. +4. Click **Save**. + +##### Adjust Brightness + +1. On the homepage of Control Center, click ![](./figures/icon72-o.svg). +2. Click **Brightness**. + - Drag the slider to set screen brightness. + - Switch on **Night Shift**, the screen hue will be auto-adjusted according to your location. + - Switch on **Auto Brightness**, the monitor will change the brightness automatically according to ambient light (shown only if PC has a light sensor). + +##### Change Refresh Rate + +1. On the homepage of Control Center, click ![](./figures/icon72-o.svg). +2. Click **Refresh Rate**. +3. Select a proper one, and click **Save**. + +##### Change Display Direction + +1. On the homepage of Control Center, click ![](./figures/icon72-o.svg). +2. Click ![](./figures/icon112-o.svg). +3. Every time you click, the screen will rotate 90 degrees counterclockwise. +4. To restore to the original direction, click the right button to exit; to use the current direction, press **Ctrl**+ **S** to save it. + +#### Multiple Screen Settings + +Expand your desktop by multiple screens! Use VGA/HDMI/DP cable to connect your computer to other display devices. + +1. On the homepage of Control Center, click ![](./figures/icon72-o.svg). +2. Click **Multiple Displays**. +3. Select a display mode: + - **Duplicate**: display the same image on other screens. + - **Extend**: expand the desktop across the screens. + - **Customize**: customize the display settings for multiple screens. + +In multiple displays, press **Super** + **P** to show its OSD. + +Operations are as follows: + +1. Hold **Super** and press **P** or click to select the options. +2. Release the keys, the selected mode will take into effect. + +>![](./figures/icon99-o.svg)Notes: *When the multiple displays are in the extend mode, only the main screen supports desktop icon display, right-click menu operation and other functions, while the sub-screens do not.* + +##### Custom Settings + +1. On the homepage of Control Center, click ![](./figures/icon72-o.svg). +2. Click **Multiple Displays** > **Customize**. +3. Click **Recognize**. +4. Choose **Merge** or **Split** the screens, specify the main screen, set the resolution and refresh rate, and rotate screen if you want. +5. Click **Save**. + +> ![](./figures/icon99-o.svg)Notes: *"Merge" means duplicate mode, "Split" means extend mode.* + +### Default Application Settings + +If you have installed several applications with similar functions, such as text editor, choose one of them to be the default application to open that type of file. + +![](./figures/39.png) + +#### Set Default Application + +1. Right-click the file, choose **Open with** > **Set default program**. +2. Select one application, **Set as default** is checked by default, and click **Confirm**. +3. The application will automatically be added to the default application list in Control Center. + +#### Change Default Application + +1. On the homepage of Control Center, click ![](./figures/icon70-o.svg). +2. Select a file type. +3. Select another one in the list as the default application. + +#### Add Default Application + +1. On the homepage of Control Center, click ![](./figures/icon70-o.svg). +2. Select a file type. +3. Click ![](./figures/icon50-o.svg) below to add a desktop file (usually at /usr/share/applications) or a specified binary file as the default application. +4. The application will be added to the list and set as default application automatically. + +#### Delete Default Application + +In the default application list, you can only delete the applications you added. To remove other applications from the list, the only way is to uninstall them. Once uninstalled, they will automatically be deleted from the list. + +To delete the default applications you have added, do as below: + +1. On the homepage of Control Center, click ![](./figures/icon70-o.svg). +2. Select a file type. +3. Click ![](./figures/icon57-o.svg) after the application name to delete it. + +### Personalization Settings + +You can set theme, accent color, font, change the appearance of the desktop and windows to your favorite style. + +![](./figures/56.png) + +#### Set Window Theme + +1. On the homepage of Control Center, click ![](./figures/icon105-o.svg). +2. Click **General**. +3. Select one window theme, which will be used as system theme. + +> ![](./figures/icon99-o.svg)Notes: *"Auto" means changing window theme automatically according to the sunset and sunrise time. After sunrise, it is light theme; after sunset, it is dark theme.* + +#### Change Accent Color + +Accent color refers to the color used when you select one option or file in the system. + +1. On the homepage of Control Center, click ![](./figures/icon105-o.svg). +2. Click **General**. +3. Pick a color under **Accent Color** and view its effects. + +#### Set Icon Theme + +1. On the homepage of Control Center, click ![](./figures/icon105-o.svg). +2. Click **Icon Theme** and select an icon style. + +#### Set Cursor Theme + +1. On the homepage of Control Center, click ![](./figures/icon105-o.svg). +2. Click **Cursor Theme** and select a set of cursors. + +#### Change Font + +1. On the homepage of Control Center, click ![](./figures/icon105-o.svg). +2. Click **Font**. +3. Set the font and font size for the system. + +### Network Settings + +After login, you need to connect to a network first and then surf the Internet! + +> ![](./figures/icon125-o.svg)Tips: *Check your network status by hovering over or clicking the network icon in Dock.* + +![](./figures/54.png) + +#### Wired Network + +Wired network is secure and stable, which makes it the most common way to connect to the Internet. After your router is set, connect both ends of the network cable to the computer and router to connect to a wired network. + +1. Plug the cable into the network slot of a computer. +2. Plug another end of the cable into the router or network port. +3. On the homepage of Control Center, click ![](./figures/icon97-o.svg). +4. Click **Wired Network** to enter the setting page of wired network. +5. Switch on **Wired Network Adapter** to enable wired network. +6. If it is successfully connected to the network, there will be a prompt "Wired Connection connected". + +You can also edit and add a new wired network in the setting page. + +#### Mobile Network + +If you are at a place without network, mobile network adapter is a useful tool to help you connect to the Internet as long as the place is covered by telephone signals. + +1. Plug the mobile network adapter into your computer USB port. +2. Your computer will auto connect to the network. +3. On the homepage of Control Center, click ![](./figures/icon97-o.svg). +4. Click **Mobile Network** to view the detailed network info. + +#### DSL/PPPoE Connections + +DSL is a dial-up connection using a standard phone line and analog modem to access the Internet. Configure the modem, plug the telephone line into the network interface of the computer, create a broadband dial-up connection, and enter the user name and password provided by the operator to dial up the Internet. + +##### Create a PPPoE Connection + +1. On the homepage of Control Center, click ![](./figures/icon97-o.svg). +2. Click **DSL**. +3. Click ![](./figures/icon50-o.svg). +4. Enter the name, your account and password the operator provides. +5. Click **Save**. The connection will automatically start. + +#### VPN + +VPN is a virtual private network. Its main function is to establish a private network on the public network for encrypted communication. Whether you are on a business trip or working at home, you can use VPN to access intranet resources as long as you can access the Internet. You can also use VPN to speed up access to websites in other countries. + +1. On the homepage of Control Center, click ![](./figures/icon97-o.svg). +2. Click **VPN**, and click ![](./figures/icon50-o.svg) or ![](./figures/icon84-o.svg). +3. Select the VPN protocol type, and enter the name, gateway, account, password and other information. (Importing VPN will automatically fill in information) +4. Click **Save**, the system will try to connect VPN network automatically. +5. You can export the VPN settings to backup or share with other users. + +> ![](./figures/icon99-o.svg)Notes: *If you don't want to use the VPN as the default routing, but only want it to take effect on specific network resources, switch on **Only applied in corresponding resources**.* + +#### System Proxy + +1. On the homepage of Control Center, click ![](./figures/icon97-o.svg). +2. Click **System Proxy**. + +- Click **None** and **Save** to disable the proxy. +- Click **Manual** and input the address and port of proxy servers. +- Click **Auto** and input a URL to configure the proxy info. + +#### Application Proxy + +1. On the homepage of Control Center, click ![](./figures/icon97-o.svg). +2. Click **Application Proxy**. +3. Select a proxy type, and fill in the IP address, port, etc. +4. Click **Save** to save the proxy settings. + +> ![](./figures/icon99-o.svg)Notes: *After being configured, run Launcher, right-click any application's icon and check **Use a proxy**, and then the application will be opened by proxy.* + +#### Network Info + +You can view MAC, IP address, gateway and other network info in network details. + +1. On the homepage of Control Center, click ![](./figures/icon97-o.svg). +2. Click **Network Details**. +3. View the network info of the current network. + +### Sound Settings + +Set your speaker and microphone properly to make you hear more comfortable and make clearer recordings. + +![](./figures/61.png) + +#### Output + +1. On the homepage of Control Center, click ![](./figures/icon116-o.svg). + +2. Click **Output** to: + + - Select output device type from the dropdown list after **Output Device**. + + - Drag the slider to adjust output volume and left/right balance. + - Switch on **Volume Boost**, the volume could be adjustable from 0~150% (the former range is 0~100%). + +#### Input + +1. On the homepage of Control Center, click ![](./figures/icon116-o.svg). +2. Click **Input** to: + - Select input device type from the dropdown list after **Input Device**. + - Adjust input volume by dragging the slider. + - You can enable **Automatic Noise Suppression** by clicking the button after "Automatic Noise Suppression". + +> ![](./figures/icon125-o.svg)Tips: *Usually, you need to turn up the input volume to make sure that you can hear the sound of the sound source, but the volume should not be too high, because it will cause distortion of the sound. Here is how to set input volume: Speak to your microphone at a normal volume and view "Input Level". If the indicator changes obviously according to the volume, then the input volume is at a proper level.* + +#### System Sound Effects + +1. On the homepage of Control Center, click ![](./figures/icon116-o.svg). +2. Click **Sound Effects**, check the options you want to switch on the sound when the corresponding event occurs. + +> ![](./figures/icon125-o.svg)Tips: *Click to listen to the sound effect.* + +### Date and Time + +Set your timezone properly to have correct date and time. You can also change them manually. + +![](./figures/62.png) + +#### Change Timezone + +You have selected the timezone during system installation and do as follows to change it. + +1. On the homepage of Control Center, click ![](./figures/icon124-o.svg). +2. Click **Timezone List**. +3. Click **Change System Timezone** and select a timezone by searching or clicking on the map. +4. Click **Confirm**. + +#### Add Timezone + +Add another timezone to see the date and time there. + +1. On the homepage of Control Center, click ![](./figures/icon124-o.svg). +2. Click **Timezone List**. +3. Click ![](./figures/icon50-o.svg), select a timezone by searching or clicking on the map. +4. Click **Add**. + +#### Delete Timezone + +1. On the homepage of Control Center, click ![](./figures/icon124-o.svg). +2. Click **Timezone List**. +3. Click **Edit** after "Timezone List". +4. Click ![](./figures/icon71-o.svg) to remove the timezone. + +#### Change Date and Time + +Note that the auto-sync function will be disabled after changing date and time manually. + +1. On the homepage of Control Center, click ![](./figures/icon124-o.svg). +2. Click **Time Settings**. + - Switch on/off **Auto Sync**. + - Enter the correct date and time. +3. Click **Confirm**. + +#### Set Time Format + +Setting the format of time and date is supported. + +1. On the homepage of Control Center, click ![](./figures/icon124-o.svg). +2. Click **Time Format** to set the first day of week, long date, short date, long time, and short time. + +### Power Management + +Power management helps you to improve system safety. + +![](./figures/57.png) + +#### Time to Suspend + +1. On the homepage of Control Center, click ![](./figures/icon107-o.svg). +2. Click **Plugged In**. +3. Set the time to suspend. + +#### Time to Lock Screen + +1. On the homepage of Control Center, click ![](./figures/icon107-o.svg). +2. Click **Plugged In**. +3. Set the time to lock screen. + +#### Power button settings + +1. On the homepage of Control Center, click ![](./figures/icon107-o.svg). +2. Click **Plugged In**. +3. You can select **Shut down, Suspend, Hibernate, Turn off the monitor, Do nothing** from the drop-down list after **When pressing the power button**. + +Any operation done here will take effect immediately. At the same time, the system will notify the user that the power button setting is changed. + +### Mouse + +Mouse is common computer input device. Using the mouse, you can make the operation easier and faster. + +![](./figures/53.png) + +#### General Settings + +1. On the homepage of Control Center, click ![](./figures/icon94-o.svg). +2. Click **General**. +3. Switch on **Left Hand**, and adjust **Scrolling Speed**, **Double-click Speed**. + +> ![](./figures/icon99-o.svg)Notes: *If "Left Hand" is enabled, left-click and right-click of the mouse exchange.* + +#### Mouse + +After inserting or connecting the mouse, make relevant settings in the Control Center to make it more in line with your usage habits. + +1. On the homepage of Control Center, click ![](./figures/icon94-o.svg). +2. Click **Mouse**. +3. Adjust **Pointer Speed**, which helps you to control the speed at which the pointer moves as the mouse moves. +4. Switch on **Natural Scrolling**/**Mouse Acceleration** if you want. + +> ![](./figures/icon99-o.svg)Notes: +> +> - *Turn on the mouse acceleration to improve the accuracy of the pointer. The moving distance of the mouse pointer on the screen will increase according to the acceleration of the moving speed. It can be turned on or off according to the usage.* +> - *If Natural Scrolling is enabled, when you scroll down, the page will scroll down, when you scroll up, the page will scroll up as well.* + +### Keyboard and Language + +Set keyboard properties and select your keyboard layout to keep your typing habit. You can also adjust the keyboard layout according to the country and language, change system language, and customize shortcuts here. + +![](./figures/59.png) + +#### Keyboard Properties + +1. On the homepage of Control Center, click ![](./figures/icon86-o.svg). +2. Click **General**. +3. Adjust **Repeat Delay**/**Repeat Rate**. +4. Click "Test here" and hold down a key to test the repeat rate. +5. Switch on **Numeric Keypad** and **Caps Lock Prompt** if you want. + +#### Keyboard Layout + +Set the keyboard layout to customize the keyboard for the current language. When you press a key on the keyboard, the keyboard layout controls which characters are displayed on the screen. After changing the keyboard layout, the characters on the screen may not match the characters on the keyboard keys. + +You have set a keyboard layout during system installation, but you can add more for other purposes. + +![](./figures/50.png) + +##### Add Keyboard Layout + +1. On the homepage of Control Center, click ![](./figures/icon86-o.svg). +2. Click **Keyboard Layout**. +3. Click ![](./figures/icon50-o.svg). Click a keyboard layout to add it. + +##### Delete Keyboard Layout + +1. On the homepage of Control Center, click ![](./figures/icon86-o.svg). +2. Click **Keyboard Layout**. +3. Click **Edit**. +4. Click ![](./figures/icon71-o.svg) to delete keyboard layout. + +##### Switch Keyboard Layout + +1. On the homepage of Control Center, click ![](./figures/icon86-o.svg). +2. Click **Keyboard Layout**. +3. Click the layout you want to switch to. +4. After successful switching, the layout will be marked with a check. + +> ![](./figures/icon125-o.svg)Tips: *You can also select one or more shortcuts to switch the keyboard layouts in order. Select **Applies to** to make the keyboard layout after switching be applied to the whole system or current application.* + +#### System Language + +The system language is the language you selected when you installed the system by default, which can be changed at any time. + +##### Add System Language + +Add multiple languages into the list to change language conveniently. + +1. On the homepage of Control Center, click ![](./figures/icon86-o.svg). +2. Click **System Language**. +3. Click ![](./figures/icon50-o.svg) to enter the language list. +4. Select the language you want, and it will be added into system language list automatically. + +##### Change System Language + +1. On the homepage of Control Center, click ![](./figures/icon86-o.svg). +2. Click **System Language**. +3. Select the language you want to switch to, and the language package will be installed automatically. +4. After being successfully installed, log out and log in again to view the changes. + +> ![](./figures/icon52-o.svg)Attention: *The keyboard layout may also be changed in the process of switching the system language. Please make sure that you select a correct keyboard layout to enter the login password.* + +#### Shortcuts + +The shortcut list includes all shortcuts in the system. View, modify and customize the shortcuts here as you want. + +![](./figures/59.png) + +##### View Shortcuts + +1. On the homepage of Control Center, click ![](./figures/icon86-o.svg). +2. Click **Shortcuts**. +3. You can search or view the default shortcuts for system, window and workspace. + +##### Modify Shortcuts + +1. On the homepage of Control Center, click ![](./figures/icon86-o.svg). +2. Click **Shortcuts**. +3. Click the shortcut you want to modify. +4. Press new shortcut to change it. + +> ![](./figures/icon125-o.svg)Tips: *To disable a shortcut, please press ![](./figures/icon54-o.svg) on the keyboard. To cancel modifying, press **Esc** or click Restore Defaults at the bottom.* + +##### Customize Shortcuts + +1. On the homepage of Control Center, click ![](./figures/icon86-o.svg). +2. Click **Shortcuts**. +3. Click ![](./figures/icon50-o.svg). +4. Enter the name, command and shortcut. +5. Click **Add**. +6. After being successfully added, click **Edit**. +7. Click ![](./figures/icon71-o.svg) to delete the custom shortcut. + +> ![](./figures/icon125-o.svg)Tips: *To change the shortcut, click it and press a new shortcut to change it directly. To edit the name and command of the custom shortcut, click **Edit** > ![](./figures/icon75-o.svg) near the shortcut name to enter the shortcut settings.* + +### System Info + +You can view system version, authorization info, hardware info, and the agreements here. + +![](./figures/48.png) + +#### About This PC + +1. On the homepage of Control Center, click ![](./figures/icon120-o.svg). +2. Under **About This PC**, you can view system version, authorization and hardware information. +3. If the system has not been activated, click **Activate** to activate the system. + +#### Edition License + +1. On the homepage of Control Center, click ![](./figures/icon120-o.svg). +2. View the system edition license under **Edition License**. + +#### End User License Agreement + +1. On the homepage of Control Center, click ![](./figures/icon120-o.svg). +2. View the End User License Agreement under **End User License Agreement**. + +## Keyboard Interaction + +You can use the keyboard to switch between various interface areas, select objects and perform operations. + +| Key | Function | +| :----------------------------------------------------------- | :----------------------------------------------------------- | +| **Tab** | Switch between different areas or dialog buttons. | +| ![](./figures/icon127-o.svg) ![](./figures/icon73-o.svg) ![](./figures/icon88-o.svg) ![](./figures/icon111-o.svg) | Used to select different objects in the same area. Press ![](./figures/icon111-o.svg) to enter the lower menu and ![](./figures/icon88-o.svg) to return to the upper menu. Press![](./figures/icon127-o.svg)and ![](./figures/icon73-o.svg) to switch between up and down. | +| **Enter** | Execute the selected operation. | +| **Space** | Preview the selected object in File Manager; start and pause the playback in Music and Movie; expand the drop-down options in the drop-down list (The enter key is also available.). | +| **Ctrl**+**M** | Open the right-click menu. | diff --git a/docs/en/tools/desktop/dde/figures/38.png b/docs/en/tools/desktop/dde/figures/38.png new file mode 100644 index 0000000000000000000000000000000000000000..838f5ff0616a83cdf42edb053f4e72b93bfa644e Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/38.png differ diff --git a/docs/en/tools/desktop/dde/figures/39.png b/docs/en/tools/desktop/dde/figures/39.png new file mode 100644 index 0000000000000000000000000000000000000000..12a379403d73a47b2fa564120a28fdb58d188963 Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/39.png differ diff --git a/docs/en/tools/desktop/dde/figures/40.png b/docs/en/tools/desktop/dde/figures/40.png new file mode 100644 index 0000000000000000000000000000000000000000..bf419894eab852b45604966c62fafa71f051c4df Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/40.png differ diff --git a/docs/en/tools/desktop/dde/figures/41.png b/docs/en/tools/desktop/dde/figures/41.png new file mode 100644 index 0000000000000000000000000000000000000000..f94b0ee72e0d4e9277e9b44b4268cfbdb8402104 Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/41.png differ diff --git a/docs/en/tools/desktop/dde/figures/42.png b/docs/en/tools/desktop/dde/figures/42.png new file mode 100644 index 0000000000000000000000000000000000000000..3182e551c4e4b03885bad6339f1de514b3f55f8c Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/42.png differ diff --git a/docs/en/tools/desktop/dde/figures/43.jpg b/docs/en/tools/desktop/dde/figures/43.jpg new file mode 100644 index 0000000000000000000000000000000000000000..26e9244f58ea9800081fd61ae135477f05b21b40 Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/43.jpg differ diff --git a/docs/en/tools/desktop/dde/figures/44.png b/docs/en/tools/desktop/dde/figures/44.png new file mode 100644 index 0000000000000000000000000000000000000000..c3abaecd6e053272d81e0ad9bd183c6858b4f3c5 Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/44.png differ diff --git a/docs/en/tools/desktop/dde/figures/45.png b/docs/en/tools/desktop/dde/figures/45.png new file mode 100644 index 0000000000000000000000000000000000000000..86b051acde857c88479714414f721a7f59cca483 Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/45.png differ diff --git a/docs/en/tools/desktop/dde/figures/46.png b/docs/en/tools/desktop/dde/figures/46.png new file mode 100644 index 0000000000000000000000000000000000000000..d8ec41c87628bf28c9905523f99ae93aebd13614 Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/46.png differ diff --git a/docs/en/tools/desktop/dde/figures/47.jpg b/docs/en/tools/desktop/dde/figures/47.jpg new file mode 100644 index 0000000000000000000000000000000000000000..bf95f03c8ea0f84a878bc63af20972c9da71bc04 Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/47.jpg differ diff --git a/docs/en/tools/desktop/dde/figures/48.png b/docs/en/tools/desktop/dde/figures/48.png new file mode 100644 index 0000000000000000000000000000000000000000..ef21fa1ce1e2e9848a8dca16e692de673df7c6d7 Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/48.png differ diff --git a/docs/en/tools/desktop/dde/figures/50.png b/docs/en/tools/desktop/dde/figures/50.png new file mode 100644 index 0000000000000000000000000000000000000000..b86a55fe4363f56fc18befc9d27025a75ca427ad Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/50.png differ diff --git a/docs/en/tools/desktop/dde/figures/51.png b/docs/en/tools/desktop/dde/figures/51.png new file mode 100644 index 0000000000000000000000000000000000000000..d427ac871dba9c32eb4ffe736d5352f8408da533 Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/51.png differ diff --git a/docs/en/tools/desktop/dde/figures/52.png b/docs/en/tools/desktop/dde/figures/52.png new file mode 100644 index 0000000000000000000000000000000000000000..0ca0a2db05c70bc25f9bb59e82d074f671cfc74e Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/52.png differ diff --git a/docs/en/tools/desktop/dde/figures/53.png b/docs/en/tools/desktop/dde/figures/53.png new file mode 100644 index 0000000000000000000000000000000000000000..76fbc34a1d5621b83c2d8c93222766acad33350d Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/53.png differ diff --git a/docs/en/tools/desktop/dde/figures/54.png b/docs/en/tools/desktop/dde/figures/54.png new file mode 100644 index 0000000000000000000000000000000000000000..49ecae6f8941a118223f3765c23015df074c4983 Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/54.png differ diff --git a/docs/en/tools/desktop/dde/figures/56.png b/docs/en/tools/desktop/dde/figures/56.png new file mode 100644 index 0000000000000000000000000000000000000000..36fee795bfe593b6246c8d6c2bddea9386b06f45 Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/56.png differ diff --git a/docs/en/tools/desktop/dde/figures/57.png b/docs/en/tools/desktop/dde/figures/57.png new file mode 100644 index 0000000000000000000000000000000000000000..539d06b77b058a933cb154c43641d498050986e0 Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/57.png differ diff --git a/docs/en/tools/desktop/dde/figures/58.png b/docs/en/tools/desktop/dde/figures/58.png new file mode 100644 index 0000000000000000000000000000000000000000..396ca16d873e54505bcdbd41d669366eea7f5dee Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/58.png differ diff --git a/docs/en/tools/desktop/dde/figures/59.png b/docs/en/tools/desktop/dde/figures/59.png new file mode 100644 index 0000000000000000000000000000000000000000..9b1de98ac4fe686937ca844d3e9481548a79ce63 Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/59.png differ diff --git a/docs/en/tools/desktop/dde/figures/60.jpg b/docs/en/tools/desktop/dde/figures/60.jpg new file mode 100644 index 0000000000000000000000000000000000000000..033c88aaadd04f7d4058ec2eb5b2c70498319bf7 Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/60.jpg differ diff --git a/docs/en/tools/desktop/dde/figures/61.png b/docs/en/tools/desktop/dde/figures/61.png new file mode 100644 index 0000000000000000000000000000000000000000..8df17062963a3baf92318a12ec34b1378122687b Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/61.png differ diff --git a/docs/en/tools/desktop/dde/figures/62.png b/docs/en/tools/desktop/dde/figures/62.png new file mode 100644 index 0000000000000000000000000000000000000000..ec312d6c0c22018c1745dd866da71ce9be47fbda Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/62.png differ diff --git a/docs/en/tools/desktop/dde/figures/63.jpg b/docs/en/tools/desktop/dde/figures/63.jpg new file mode 100644 index 0000000000000000000000000000000000000000..504f7cf59768f6fd1cd73a115d01fbc4e15a02e1 Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/63.jpg differ diff --git a/docs/en/tools/desktop/dde/figures/63.png b/docs/en/tools/desktop/dde/figures/63.png new file mode 100644 index 0000000000000000000000000000000000000000..86b051acde857c88479714414f721a7f59cca483 Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/63.png differ diff --git a/docs/en/tools/desktop/dde/figures/dde-1.png b/docs/en/tools/desktop/dde/figures/dde-1.png new file mode 100644 index 0000000000000000000000000000000000000000..fb1d5177c39262ed182f10a57fdae850d007eeb1 Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/dde-1.png differ diff --git a/docs/en/tools/desktop/dde/figures/dde-2.png b/docs/en/tools/desktop/dde/figures/dde-2.png new file mode 100644 index 0000000000000000000000000000000000000000..be5d296937bd17b9646b32c80934aa76738027af Binary files /dev/null and b/docs/en/tools/desktop/dde/figures/dde-2.png differ diff --git a/docs/en/tools/desktop/dde/figures/icon101-o.svg b/docs/en/tools/desktop/dde/figures/icon101-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..af1c5d3dc0277a6ea59e71efb6ca97bdfc782e8e --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon101-o.svg @@ -0,0 +1,3 @@ + + + diff --git a/docs/en/tools/desktop/dde/figures/icon103-o.svg b/docs/en/tools/desktop/dde/figures/icon103-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..c06c885725c569ab8db1fe7d595a7c65f18c5142 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon103-o.svg @@ -0,0 +1,26 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon105-o.svg b/docs/en/tools/desktop/dde/figures/icon105-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..36c49949fa569330b761c2d65518f36c10435508 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon105-o.svg @@ -0,0 +1,111 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon107-o.svg b/docs/en/tools/desktop/dde/figures/icon107-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..fb5a3ea756f6ccb7b3e5c31122a433347a908c96 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon107-o.svg @@ -0,0 +1,50 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon110-o.svg b/docs/en/tools/desktop/dde/figures/icon110-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..7958e3f192061592e002e1e8a1bad06ffa86742c --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon110-o.svg @@ -0,0 +1,12 @@ + + + + reboot_normal + Created with Sketch. + + + + + + + \ No newline at end of file diff --git a/docs/en/tools/desktop/dde/figures/icon111-o.svg b/docs/en/tools/desktop/dde/figures/icon111-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..097d16a08d305a8b3f3b2268ab1ea8342e799377 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon111-o.svg @@ -0,0 +1,13 @@ + + + + Right + Created with Sketch. + + + + + + + + \ No newline at end of file diff --git a/docs/en/tools/desktop/dde/figures/icon112-o.svg b/docs/en/tools/desktop/dde/figures/icon112-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..e51628c2b8b10495f3410d219814286696ea2fd5 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon112-o.svg @@ -0,0 +1,15 @@ + + + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon116-o.svg b/docs/en/tools/desktop/dde/figures/icon116-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..4d79cd6dbbbfd3969f4e0ad0ad88e27398853505 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon116-o.svg @@ -0,0 +1,72 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon120-o.svg b/docs/en/tools/desktop/dde/figures/icon120-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..e895c347d16a200aea46b00428b0b9f1a3c94246 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon120-o.svg @@ -0,0 +1,49 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon122-o.svg b/docs/en/tools/desktop/dde/figures/icon122-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..7fb014b5fd6097ca37a84d0b6a27dc982d675c8a --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon122-o.svg @@ -0,0 +1,3 @@ + + + diff --git a/docs/en/tools/desktop/dde/figures/icon124-o.svg b/docs/en/tools/desktop/dde/figures/icon124-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..960c0ec096c925213f8953398f0e8e5db3cdaed3 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon124-o.svg @@ -0,0 +1,55 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon125-o.svg b/docs/en/tools/desktop/dde/figures/icon125-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..011c05f4b8f296867cd408a339230323fcbb28dd --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon125-o.svg @@ -0,0 +1,9 @@ + + + tips + + + + + + \ No newline at end of file diff --git a/docs/en/tools/desktop/dde/figures/icon126-o.svg b/docs/en/tools/desktop/dde/figures/icon126-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..e0a43b6b8beb434090ac0dd3a8fd68c023f11fce --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon126-o.svg @@ -0,0 +1,68 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon127-o.svg b/docs/en/tools/desktop/dde/figures/icon127-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..bed95d35334a8d0151211054236c0bacddcc0dd3 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon127-o.svg @@ -0,0 +1,13 @@ + + + + Up + Created with Sketch. + + + + + + + + \ No newline at end of file diff --git a/docs/en/tools/desktop/dde/figures/icon128-o.svg b/docs/en/tools/desktop/dde/figures/icon128-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..aa727f3f5d5883b3fb83a79c4b98e8b5bfe4ade6 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon128-o.svg @@ -0,0 +1,12 @@ + + + + userswitch_normal + Created with Sketch. + + + + + + + \ No newline at end of file diff --git a/docs/en/tools/desktop/dde/figures/icon132-o.svg b/docs/en/tools/desktop/dde/figures/icon132-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..588ba9d98864ba67a562fa9179f29405f7687aa0 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon132-o.svg @@ -0,0 +1,15 @@ + + + + - + Created with Sketch. + + + + + + + + + + \ No newline at end of file diff --git a/docs/en/tools/desktop/dde/figures/icon134-o.svg b/docs/en/tools/desktop/dde/figures/icon134-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..784cf383eb0e8f5c7a57a602047be50ad0a3bc05 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon134-o.svg @@ -0,0 +1,15 @@ + + + + = + Created with Sketch. + + + + + + + + + + \ No newline at end of file diff --git a/docs/en/tools/desktop/dde/figures/icon136-o.svg b/docs/en/tools/desktop/dde/figures/icon136-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..24aa139ab2fefaee20935551f1af5aef473719ed --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon136-o.svg @@ -0,0 +1,12 @@ + + + + poweroff_normal + Created with Sketch. + + + + + + + \ No newline at end of file diff --git a/docs/en/tools/desktop/dde/figures/icon49-o.svg b/docs/en/tools/desktop/dde/figures/icon49-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..72ffb173fdb95e1aff5b0001b08ed6b71122b7f2 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon49-o.svg @@ -0,0 +1,178 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon50-o.svg b/docs/en/tools/desktop/dde/figures/icon50-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..05026802be4718205065d6369e14cc0b6ef05bc7 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon50-o.svg @@ -0,0 +1,15 @@ + + + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon52-o.svg b/docs/en/tools/desktop/dde/figures/icon52-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..23149c05873259cd39721b8ee9c3ab7db86d64c5 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon52-o.svg @@ -0,0 +1,9 @@ + + + attention + + + + + + \ No newline at end of file diff --git a/docs/en/tools/desktop/dde/figures/icon53-o.svg b/docs/en/tools/desktop/dde/figures/icon53-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..50e33489ce984b0acfd621da4a8ef837fdf048c1 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon53-o.svg @@ -0,0 +1,11 @@ + + + + previous + Created with Sketch. + + + + + + \ No newline at end of file diff --git a/docs/en/tools/desktop/dde/figures/icon54-o.svg b/docs/en/tools/desktop/dde/figures/icon54-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..3b599aef4b822c707d2f646405bb00837aed96fd --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon54-o.svg @@ -0,0 +1,18 @@ + + + + Backspace + Created with Sketch. + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/en/tools/desktop/dde/figures/icon56-o.svg b/docs/en/tools/desktop/dde/figures/icon56-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..9f13b6861e3858deec8d57a5301c934acc247069 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon56-o.svg @@ -0,0 +1,19 @@ + + + + Slice 1 + Created with Sketch. + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/en/tools/desktop/dde/figures/icon57-o.svg b/docs/en/tools/desktop/dde/figures/icon57-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..e6fbfa1381b76ab3fcd45652b33267a7f6c69bb7 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon57-o.svg @@ -0,0 +1,11 @@ + + + + titlebutton/close_normal + Created with Sketch. + + + + + + \ No newline at end of file diff --git a/docs/en/tools/desktop/dde/figures/icon58-o.svg b/docs/en/tools/desktop/dde/figures/icon58-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..9746dcacfc8e5d4c4b63233801e37418a190fc8f --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon58-o.svg @@ -0,0 +1,46 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon62-o.svg b/docs/en/tools/desktop/dde/figures/icon62-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..09f61b446669df2e05a3351d40d8c30879c7b035 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon62-o.svg @@ -0,0 +1,47 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon63-o.svg b/docs/en/tools/desktop/dde/figures/icon63-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..06c03ed99260ffadc681475dad35610aedf67f83 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon63-o.svg @@ -0,0 +1,45 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon66-o.svg b/docs/en/tools/desktop/dde/figures/icon66-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..5793b3846b7fe6a5758379591215b16c7f9e1b52 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon66-o.svg @@ -0,0 +1,43 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon68-o.svg b/docs/en/tools/desktop/dde/figures/icon68-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..a7748052dfa436116d8742dca28f7d90865231ed --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon68-o.svg @@ -0,0 +1,23 @@ + + + + deepin-system-monitor + Created with Sketch. + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/en/tools/desktop/dde/figures/icon69-o.svg b/docs/en/tools/desktop/dde/figures/icon69-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..e21dfd00a32a44ee1c8e3882b4ca8239be04690f --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon69-o.svg @@ -0,0 +1,26 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon70-o.svg b/docs/en/tools/desktop/dde/figures/icon70-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..b5787a7ffa5ed9519a48c6937c60927fd11fd455 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon70-o.svg @@ -0,0 +1,73 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon71-o.svg b/docs/en/tools/desktop/dde/figures/icon71-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..669a21f143b06cb45ea3f45f7f071809f2cbc8a8 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon71-o.svg @@ -0,0 +1,15 @@ + + + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon72-o.svg b/docs/en/tools/desktop/dde/figures/icon72-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..79067ed9b9ff7912e1742183b461fa056601b9cc --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon72-o.svg @@ -0,0 +1,34 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon73-o.svg b/docs/en/tools/desktop/dde/figures/icon73-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..cf6292387f5e790db6ebd66184aabcbb39257ee7 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon73-o.svg @@ -0,0 +1,13 @@ + + + + Down + Created with Sketch. + + + + + + + + \ No newline at end of file diff --git a/docs/en/tools/desktop/dde/figures/icon75-o.svg b/docs/en/tools/desktop/dde/figures/icon75-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..ef6823ccc19858f57374f0b78ad31514e8311be3 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon75-o.svg @@ -0,0 +1,3 @@ + + + diff --git a/docs/en/tools/desktop/dde/figures/icon83-o.svg b/docs/en/tools/desktop/dde/figures/icon83-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..35dd6eacc54a933dc9ebc3f3010edfa7363fecc0 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon83-o.svg @@ -0,0 +1,84 @@ + + + + + + image/svg+xml + + img_upload + + + + + + img_upload + Created with Sketch. + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon84-o.svg b/docs/en/tools/desktop/dde/figures/icon84-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..9bd11b9e7b45b506dd7e1c87d09d545d8f48af06 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon84-o.svg @@ -0,0 +1,15 @@ + + + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon86-o.svg b/docs/en/tools/desktop/dde/figures/icon86-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..5da20233309c43d4fc7b315f441cde476c835c67 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon86-o.svg @@ -0,0 +1,66 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon88-o.svg b/docs/en/tools/desktop/dde/figures/icon88-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..c2570c26575fd14cb5e9d9fe77831d2e8f6c9333 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon88-o.svg @@ -0,0 +1,13 @@ + + + + Left + Created with Sketch. + + + + + + + + \ No newline at end of file diff --git a/docs/en/tools/desktop/dde/figures/icon90-o.svg b/docs/en/tools/desktop/dde/figures/icon90-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..79b5e0a141f7969a8f77ae61f4c240de7187afe9 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon90-o.svg @@ -0,0 +1,12 @@ + + + + lock_normal + Created with Sketch. + + + + + + + \ No newline at end of file diff --git a/docs/en/tools/desktop/dde/figures/icon92-o.svg b/docs/en/tools/desktop/dde/figures/icon92-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..21341b64a832e1935252aa82e7a4e0b083c16eae --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon92-o.svg @@ -0,0 +1,12 @@ + + + + logout_normal + Created with Sketch. + + + + + + + \ No newline at end of file diff --git a/docs/en/tools/desktop/dde/figures/icon94-o.svg b/docs/en/tools/desktop/dde/figures/icon94-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..a47044149a02101dbd24a3fdb2f3ead77efca6c1 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon94-o.svg @@ -0,0 +1,54 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon97-o.svg b/docs/en/tools/desktop/dde/figures/icon97-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..4f4670de29d8c86885b5aa806b2c8cdc6fc16dcb --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon97-o.svg @@ -0,0 +1,84 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/en/tools/desktop/dde/figures/icon99-o.svg b/docs/en/tools/desktop/dde/figures/icon99-o.svg new file mode 100644 index 0000000000000000000000000000000000000000..e9a3aa60a51404c9390bfbea8d8ff09edc0e2e32 --- /dev/null +++ b/docs/en/tools/desktop/dde/figures/icon99-o.svg @@ -0,0 +1,11 @@ + + + notes + + + + + + + + \ No newline at end of file diff --git a/docs/en/tools/desktop/gnome/_toc.yaml b/docs/en/tools/desktop/gnome/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..690ce0cd6f30cc2962e94d864d6b806cdda510e6 --- /dev/null +++ b/docs/en/tools/desktop/gnome/_toc.yaml @@ -0,0 +1,8 @@ +label: Gnome User Guide +isManual: true +description: Install and use Gnome. +sections: + - label: GNOME Installation + href: ./gnome-installation.md + - label: GNOME User Guide + href: ./gnome-user-guide.md diff --git a/docs/en/tools/desktop/gnome/figures/gnome-1.png b/docs/en/tools/desktop/gnome/figures/gnome-1.png new file mode 100644 index 0000000000000000000000000000000000000000..ed57060770957f304a3fb7ca993241d56e90f541 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-1.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-10.png b/docs/en/tools/desktop/gnome/figures/gnome-10.png new file mode 100644 index 0000000000000000000000000000000000000000..94af842ca7d0de47db4d0030a0741d0cae634a21 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-10.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-11.png b/docs/en/tools/desktop/gnome/figures/gnome-11.png new file mode 100644 index 0000000000000000000000000000000000000000..022c764002542196b723eadaaaae080b3afc1d0f Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-11.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-12.png b/docs/en/tools/desktop/gnome/figures/gnome-12.png new file mode 100644 index 0000000000000000000000000000000000000000..4255aa3e1629b2af94ec59ae0fe346d91da8ba61 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-12.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-13.png b/docs/en/tools/desktop/gnome/figures/gnome-13.png new file mode 100644 index 0000000000000000000000000000000000000000..f6bad1c09c36bdef3ed4dd48c14e98c03a230cc7 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-13.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-14.png b/docs/en/tools/desktop/gnome/figures/gnome-14.png new file mode 100644 index 0000000000000000000000000000000000000000..a661a4e759ff3107fc9bfa5f664a86f77051dfcf Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-14.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-15.png b/docs/en/tools/desktop/gnome/figures/gnome-15.png new file mode 100644 index 0000000000000000000000000000000000000000..2e5a26c33b2cb432d4b7a79af8407b5b13592d09 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-15.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-16.png b/docs/en/tools/desktop/gnome/figures/gnome-16.png new file mode 100644 index 0000000000000000000000000000000000000000..178d5e836b69168c441676c4e77721e22f460981 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-16.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-17.png b/docs/en/tools/desktop/gnome/figures/gnome-17.png new file mode 100644 index 0000000000000000000000000000000000000000..2ea9f9e1914bb90193689e3d35e48918dcc7c019 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-17.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-18.png b/docs/en/tools/desktop/gnome/figures/gnome-18.png new file mode 100644 index 0000000000000000000000000000000000000000..55b5c07ea89fca246600ff7ea0ff66f03427ddcb Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-18.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-19.png b/docs/en/tools/desktop/gnome/figures/gnome-19.png new file mode 100644 index 0000000000000000000000000000000000000000..b9f78d01b726078d25d900e4ef9f9ffb1bdc9075 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-19.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-2.png b/docs/en/tools/desktop/gnome/figures/gnome-2.png new file mode 100644 index 0000000000000000000000000000000000000000..cf86813c71dd47584c5f4d9c0d6fec29813c9dc9 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-2.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-20.png b/docs/en/tools/desktop/gnome/figures/gnome-20.png new file mode 100644 index 0000000000000000000000000000000000000000..37133665e2025c5267c3bf1ea742bc7295d0cb59 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-20.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-21.png b/docs/en/tools/desktop/gnome/figures/gnome-21.png new file mode 100644 index 0000000000000000000000000000000000000000..0d0bc17b2a973d6e035b3d08097e0ad6138ed786 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-21.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-22.png b/docs/en/tools/desktop/gnome/figures/gnome-22.png new file mode 100644 index 0000000000000000000000000000000000000000..4967a95e8c1fcf4fa5e6a799933149365e447725 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-22.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-23.png b/docs/en/tools/desktop/gnome/figures/gnome-23.png new file mode 100644 index 0000000000000000000000000000000000000000..ac39542d77bdfe64b1c3d0119870cbd5e0136d17 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-23.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-24.png b/docs/en/tools/desktop/gnome/figures/gnome-24.png new file mode 100644 index 0000000000000000000000000000000000000000..e4572c436249b329643a6777ddf6a2852fcab5a6 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-24.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-25.png b/docs/en/tools/desktop/gnome/figures/gnome-25.png new file mode 100644 index 0000000000000000000000000000000000000000..93ac3582bd0dc5a273614e20b89ea654fe02ff9d Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-25.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-26.png b/docs/en/tools/desktop/gnome/figures/gnome-26.png new file mode 100644 index 0000000000000000000000000000000000000000..98f349dab192b4b6a297d4907b9156c5d6240652 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-26.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-27.png b/docs/en/tools/desktop/gnome/figures/gnome-27.png new file mode 100644 index 0000000000000000000000000000000000000000..ea319f8df41e1bdbccb34e597ccdb3c6d21a3727 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-27.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-28.png b/docs/en/tools/desktop/gnome/figures/gnome-28.png new file mode 100644 index 0000000000000000000000000000000000000000..b2f1db5ea27cdfada82ba9572395598df2ecf648 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-28.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-29.png b/docs/en/tools/desktop/gnome/figures/gnome-29.png new file mode 100644 index 0000000000000000000000000000000000000000..fc2f6a8a2bbc56af1b657a5a25359aa15183125e Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-29.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-3.png b/docs/en/tools/desktop/gnome/figures/gnome-3.png new file mode 100644 index 0000000000000000000000000000000000000000..3ac1eb3b0d6cd8a84b0c408f745e40db6d845ca8 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-3.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-30.png b/docs/en/tools/desktop/gnome/figures/gnome-30.png new file mode 100644 index 0000000000000000000000000000000000000000..6799ad80c89443267a31c0b2d3b059cc9dd8aceb Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-30.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-31.png b/docs/en/tools/desktop/gnome/figures/gnome-31.png new file mode 100644 index 0000000000000000000000000000000000000000..39ebf079f7a11c28359440111b548bf2d7be5aaa Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-31.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-32.png b/docs/en/tools/desktop/gnome/figures/gnome-32.png new file mode 100644 index 0000000000000000000000000000000000000000..28a77475cf3a1176a0c5d01418e1833fef627cc7 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-32.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-33.png b/docs/en/tools/desktop/gnome/figures/gnome-33.png new file mode 100644 index 0000000000000000000000000000000000000000..cecde6d0ab299f69ad95f25ff7d3f7130da09d02 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-33.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-34.png b/docs/en/tools/desktop/gnome/figures/gnome-34.png new file mode 100644 index 0000000000000000000000000000000000000000..a760c501d86debdb81a89ef3a70b694e22d0e4da Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-34.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-35.png b/docs/en/tools/desktop/gnome/figures/gnome-35.png new file mode 100644 index 0000000000000000000000000000000000000000..ab46f383f1b8f2c740effff3c59ad224b9e5025b Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-35.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-36.png b/docs/en/tools/desktop/gnome/figures/gnome-36.png new file mode 100644 index 0000000000000000000000000000000000000000..e248144b99bb65943446f6c9fbd6ad45c11ddd58 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-36.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-37.png b/docs/en/tools/desktop/gnome/figures/gnome-37.png new file mode 100644 index 0000000000000000000000000000000000000000..7181726fb3d074298e41ee59ca14c9be68884aad Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-37.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-38.png b/docs/en/tools/desktop/gnome/figures/gnome-38.png new file mode 100644 index 0000000000000000000000000000000000000000..b5c02d20fbad894fa3702b6274b047ef26ea1a10 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-38.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-39.png b/docs/en/tools/desktop/gnome/figures/gnome-39.png new file mode 100644 index 0000000000000000000000000000000000000000..0c85bb5f72d7625e683409c3d4605b56f74d746b Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-39.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-4.png b/docs/en/tools/desktop/gnome/figures/gnome-4.png new file mode 100644 index 0000000000000000000000000000000000000000..0bce0130188f00f30c67a4a8d4904ad8419428da Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-4.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-40.png b/docs/en/tools/desktop/gnome/figures/gnome-40.png new file mode 100644 index 0000000000000000000000000000000000000000..e0c5a085776933b08d94cf51ae22d52af0a68ca0 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-40.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-41.png b/docs/en/tools/desktop/gnome/figures/gnome-41.png new file mode 100644 index 0000000000000000000000000000000000000000..0fc42fc339480115cc588fee59983faf4652fc80 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-41.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-42.png b/docs/en/tools/desktop/gnome/figures/gnome-42.png new file mode 100644 index 0000000000000000000000000000000000000000..fa18531c9e9ec6ee9dcb9e7c5046ee41bcfa454f Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-42.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-43.png b/docs/en/tools/desktop/gnome/figures/gnome-43.png new file mode 100644 index 0000000000000000000000000000000000000000..aadb23eda46dc831a56935a38f9a7d0c9534db89 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-43.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-44.png b/docs/en/tools/desktop/gnome/figures/gnome-44.png new file mode 100644 index 0000000000000000000000000000000000000000..8747e97f510cfd64abf520e099b5abeceb056970 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-44.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-45.png b/docs/en/tools/desktop/gnome/figures/gnome-45.png new file mode 100644 index 0000000000000000000000000000000000000000..a0841f2360ad016c15444ff913a4a7b437ee047e Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-45.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-46.png b/docs/en/tools/desktop/gnome/figures/gnome-46.png new file mode 100644 index 0000000000000000000000000000000000000000..d1815b118b98b523c1c97d14a69292528248878c Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-46.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-47.png b/docs/en/tools/desktop/gnome/figures/gnome-47.png new file mode 100644 index 0000000000000000000000000000000000000000..73c8deaf7bf8c3fca34fec443e9b60d13910732b Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-47.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-48.png b/docs/en/tools/desktop/gnome/figures/gnome-48.png new file mode 100644 index 0000000000000000000000000000000000000000..6414ceafc991a94815324d362918b06e849d952e Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-48.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-49.png b/docs/en/tools/desktop/gnome/figures/gnome-49.png new file mode 100644 index 0000000000000000000000000000000000000000..040a7a235cc3dca8dfae6d89f1d28bb308a42391 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-49.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-5.png b/docs/en/tools/desktop/gnome/figures/gnome-5.png new file mode 100644 index 0000000000000000000000000000000000000000..30076d824d8dde3206ff012538e1691df3a3e5ed Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-5.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-50.png b/docs/en/tools/desktop/gnome/figures/gnome-50.png new file mode 100644 index 0000000000000000000000000000000000000000..05951779983fa2c198afca908f6c54cbc35f557a Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-50.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-51.png b/docs/en/tools/desktop/gnome/figures/gnome-51.png new file mode 100644 index 0000000000000000000000000000000000000000..995f82f57a7828926ceea5dbaaf9f79ec453c1ab Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-51.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-52.png b/docs/en/tools/desktop/gnome/figures/gnome-52.png new file mode 100644 index 0000000000000000000000000000000000000000..6607be66dc61953729cf5106b4d39aa724b8bf76 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-52.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-53.png b/docs/en/tools/desktop/gnome/figures/gnome-53.png new file mode 100644 index 0000000000000000000000000000000000000000..b1e851620eadab0376be79fcc12c0d8d3d943ec3 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-53.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-54.png b/docs/en/tools/desktop/gnome/figures/gnome-54.png new file mode 100644 index 0000000000000000000000000000000000000000..c9e773a1aec444feedfbfb3dce8ca9745ccb9f9e Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-54.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-55.png b/docs/en/tools/desktop/gnome/figures/gnome-55.png new file mode 100644 index 0000000000000000000000000000000000000000..bd935df19ada2bbbf6f9f4b58d020f73fbdbbed0 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-55.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-56.png b/docs/en/tools/desktop/gnome/figures/gnome-56.png new file mode 100644 index 0000000000000000000000000000000000000000..4475f73a68296d19cf0010b1f848aa3cbcd82858 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-56.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-57.png b/docs/en/tools/desktop/gnome/figures/gnome-57.png new file mode 100644 index 0000000000000000000000000000000000000000..24073cd56613e3fce7f3a7e0f057a12d8d8f3077 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-57.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-58.png b/docs/en/tools/desktop/gnome/figures/gnome-58.png new file mode 100644 index 0000000000000000000000000000000000000000..19112ae3f1f4ee9e0d7b2191ddec2d14ad171d67 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-58.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-59.png b/docs/en/tools/desktop/gnome/figures/gnome-59.png new file mode 100644 index 0000000000000000000000000000000000000000..4c4225e9208ad3d38e1ec9e846d6b7c5e89e3991 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-59.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-6.png b/docs/en/tools/desktop/gnome/figures/gnome-6.png new file mode 100644 index 0000000000000000000000000000000000000000..fa53206b21811a75382f6aee788c1cd8a20d9ba7 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-6.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-7.png b/docs/en/tools/desktop/gnome/figures/gnome-7.png new file mode 100644 index 0000000000000000000000000000000000000000..fe38b39118bb79bef4412ed0f08c472ad145980e Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-7.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-8.png b/docs/en/tools/desktop/gnome/figures/gnome-8.png new file mode 100644 index 0000000000000000000000000000000000000000..769939c5583d02d014ded8065eaa7ac6aeb81cd4 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-8.png differ diff --git a/docs/en/tools/desktop/gnome/figures/gnome-9.png b/docs/en/tools/desktop/gnome/figures/gnome-9.png new file mode 100644 index 0000000000000000000000000000000000000000..b699942966fe2fe444e86231638f917f524328d9 Binary files /dev/null and b/docs/en/tools/desktop/gnome/figures/gnome-9.png differ diff --git a/docs/en/tools/desktop/gnome/gnome-installation.md b/docs/en/tools/desktop/gnome/gnome-installation.md new file mode 100644 index 0000000000000000000000000000000000000000..d0d79345e8c0bfe4b0d0b4c196b35697157be026 --- /dev/null +++ b/docs/en/tools/desktop/gnome/gnome-installation.md @@ -0,0 +1,123 @@ +# Installing GNOME on openEuler + +GNOME is a desktop environment for Unix-like operating systems. As the officially released desktop of GNU Project, GNOME provides a comprehensive, easy-to-use, and user-friendly desktop environment for application usage and development. + +For users, GNOME is a suite that integrates the desktop environment and applications. For developers, GNOME is an application development framework, consisting of a large number of function libraries. Applications written in GNOME can run properly even if users do not run the GNOME desktop environment. + +GNOME includes basic software such as the file manager, app store, and text editor, and advanced applications and tools such as system sampling analysis, system logs, software engineering IDE, web browser, simple VM monitor, and developer document browser. + +You are advised to create an administrator during the installation. + +1. [Download](https://www.openeuler.org/en/) the openEuler ISO image, install the system, and update the software source. The Everything and EPOL sources need to be configured. The following command is used to install GNOME in minimum installation mode. + + ```shell + sudo dnf update + ``` + +2. Install font libraries. + + ```shell + sudo dnf install dejavu-fonts liberation-fonts gnu-*-fonts google-*-fonts + ``` + +3. Install Xorg. + + ```shell + sudo dnf install xorg-* + ``` + + In this case, many unnecessary packages may be installed. You can run the following commands to install the required Xorg packages: + + ```shell + sudo dnf install xorg-x11-apps xorg-x11-drivers xorg-x11-drv-ati \ + xorg-x11-drv-dummy xorg-x11-drv-evdev xorg-x11-drv-fbdev xorg-x11-drv-intel \ + xorg-x11-drv-libinput xorg-x11-drv-nouveau xorg-x11-drv-qxl \ + xorg-x11-drv-synaptics-legacy xorg-x11-drv-v4l xorg-x11-drv-vesa \ + xorg-x11-drv-vmware xorg-x11-drv-wacom xorg-x11-fonts xorg-x11-fonts-others \ + xorg-x11-font-utils xorg-x11-server xorg-x11-server-utils xorg-x11-server-Xephyr \ + xorg-x11-server-Xspice xorg-x11-util-macros xorg-x11-utils xorg-x11-xauth \ + xorg-x11-xbitmaps xorg-x11-xinit xorg-x11-xkb-utils + ``` + +4. Install GNOME and it components. + + ```shell + sudo dnf install adwaita-icon-theme atk atkmm at-spi2-atk at-spi2-core baobab \ + abattis-cantarell-fonts cheese clutter clutter-gst3 clutter-gtk cogl dconf \ + dconf-editor devhelp eog epiphany evince evolution-data-server file-roller folks \ + gcab gcr gdk-pixbuf2 gdm gedit geocode-glib gfbgraph gjs glib2 glibmm24 \ + glib-networking gmime30 gnome-autoar gnome-backgrounds gnome-bluetooth \ + gnome-boxes gnome-builder gnome-calculator gnome-calendar gnome-characters \ + gnome-clocks gnome-color-manager gnome-contacts gnome-control-center \ + gnome-desktop3 gnome-disk-utility gnome-font-viewer gnome-getting-started-docs \ + gnome-initial-setup gnome-keyring gnome-logs gnome-menus gnome-music \ + gnome-online-accounts gnome-online-miners gnome-photos gnome-remote-desktop \ + gnome-screenshot gnome-session gnome-settings-daemon gnome-shell \ + gnome-shell-extensions gnome-software gnome-system-monitor gnome-terminal \ + gnome-tour gnome-user-docs gnome-user-share gnome-video-effects \ + gnome-weather gobject-introspection gom grilo grilo-plugins \ + gsettings-desktop-schemas gsound gspell gssdp gtk3 gtk4 gtk-doc gtkmm30 \ + gtksourceview4 gtk-vnc2 gupnp gupnp-av gupnp-dlna gvfs json-glib libchamplain \ + libdazzle libgdata libgee libgnomekbd libgsf libgtop2 libgweather libgxps libhandy \ + libmediaart libnma libnotify libpeas librsvg2 libsecret libsigc++20 libsoup \ + mm-common mutter nautilus orca pango pangomm libphodav python3-pyatspi \ + python3-gobject rest rygel simple-scan sushi sysprof tepl totem totem-pl-parser \ + tracker3 tracker3-miners vala vte291 yelp yelp-tools \ + yelp-xsl zenity + ``` + +5. Enable GNOME Display Manager (GDM). + + ```shell + sudo systemctl enable gdm + ``` + +6. Set the default login mode to GUI. + + ```shell + sudo systemctl set-default graphical.target + ``` + + Reboot the device for configuration verification. + + ```shell + sudo reboot + ``` + +7. If GDM cannot work: + +Disable GDM if it is installed by default. + +```shell +sudo systemctl disable gdm +``` + +Install LightDM instead. + +```shell +sudo dnf install lightdm lightdm-gtk +``` + +Set the default desktop to GNOME as the root user. + +```shell +echo 'user-session=gnome' >> /etc/lightdm/lightdm.conf.d/60-lightdm-gtk-greeter.conf +``` + +Enable LightDM. + +```shell +sudo systemctl enable lightdm +``` + +Set the default login mode to GUI. + +```shell +sudo systemctl set-default graphical.target +``` + +Reboot the device for configuration verification. + +```shell +sudo reboot +``` diff --git a/docs/en/tools/desktop/gnome/gnome-user-guide.md b/docs/en/tools/desktop/gnome/gnome-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..9e3ce4ed07ace5d9956eb7eff8ffad01768dc3fb --- /dev/null +++ b/docs/en/tools/desktop/gnome/gnome-user-guide.md @@ -0,0 +1,384 @@ +# GNOME User Guide + +## 1. Overview + +GNOME is a desktop environment for Unix-like operating systems. As the officially desktop of GNU Project, GNOME aims to build a comprehensive, easy-to-use, and user-friendly desktop environment for Unix or Unix-like operating systems based on free software. + +GNOME provides the following functional components: + +ATK: accessibility toolkit. + +Bonobo: component framework to compound documents. + +GObject: object-oriented framework in C language. + +GConf: system for storing configuration settings of apps. + +GNOME VFS: virtual file system. + +GNOME Keyring: security system. + +GNOME Print: software for printing documents. + +GStreamer: multimedia framework of GNOME. + +GTK+: building toolkit. + +Cairo: complex 2D graphics library. + +Human Interface Guidelines: software development documents provided by Sun Microsystems to facilitate GNOME usage. + +LibXML: XML library designed for GNOME. + +ORBit: CORBA Object Request Broker (ORB) that makes software componentized. + +Pango: library for i18n text arrangement and transformation. + +Metacity: window manager. + +This document describes how to use GNOME. + +The following figure shows the GUI. + +![](./figures/gnome-1.png) + +## 2. Desktop + +### 2.1 Desktop + +The GNOME desktop is clean because it does not display any files or directories. Only the left, middle, and right parts of the top bar on the desktop have entry options. They are the activity entry, message notification entry, and system status entry. + +![](./figures/gnome-2.png) + +### 2.2 Shortcut Menu + +After you right-click in the blank area on the desktop, a shortcut menu shown in the following figure is displayed, providing users with some shortcut functions. + +![](./figures/gnome-3.png) + +The following table describes the shortcuts. + +| Shortcut| Description| +| :------------ | :------------ | +| Change Background| Changes the image displayed on the desktop.| +| Display Settings| Sets the resolution, screen rotation, and night light.| +| Settings| Navigates to system settings.| + +## 3. Top Bar on the Desktop + +### 3.1 Activities + +The **Activities** entry is located in the upper left corner of the desktop. It contains app favorites, lists of all apps and active apps, a multi-view switchover function, and an indicator to the current active app. + +#### 3.1.1 App Favorites + +![](./figures/gnome-4.png) + +You can right-click an app icon in **Favorites** and choose **Remove from Favorites** from the shortcut menu to remove the app from **Favorites**. + +#### 3.1.2 List of All Apps + +To display the list of all apps, click the ![](./figures/gnome-5.png) icon under the app favorites folder. + +![](./figures/gnome-6.png) + +Similarly, you can right-click an app icon in the app list and choose **Add to Favorites** from the shortcut menu to add the app to **Favorites**. + +If there are so many apps and you know their names, you can enter an app name in the search box to search for it. + +![](./figures/gnome-7.png) + +#### 3.1.3 List of Active Apps + +Active apps, that is, running apps are displayed one by one after the last app in **Favorites**. There is a white dot under the icon of each active app. + +![](./figures/gnome-8.png) + +If you right-click an active app, operations that can be performed on the app are displayed. The operations vary with apps. Take **Screenshot** as an example. See the following figure. + +![](./figures/gnome-9.png) + +#### 3.1.4 Multi-View Switchover + +As you view the active app list, the active apps are displayed on the right of the list in multi-view mode. + +![](./figures/gnome-10.png) + +When you move the cursor to the right of the multi-view page, the vertical bar on the right becomes wider to display the window and desktop of the current active app. You can click the desktop image to switch back to the desktop. + +![Figure 10 Multi-view switch 2-big](./figures/gnome-11.png) + +If you click another app, it will be displayed on the top of the vertical bar. + +#### 3.1.5 Indicator to the Current Active App + +The indicator to the current active app is displayed on the right of **Activities**. You can click the indicator to display the operations that can be performed on the app. The operations vary with the apps. Take **Terminal** as an example. See the following figure. + +![](./figures/gnome-12.png) + +You can click **Preferences** to set the terminal preferences. + +### 3.2 Message Notification + +The message notification entry is located in the middle of the top bar on the desktop, including message notification, calendar, clock, and weather. + +![](./figures/gnome-13.png) + +#### 3.2.1 Message Notification + +If you set an alarm or countdown timer in **Clocks**, messages will be displayed on the left of the notification pane when the timer expires. The detailed information about the to-do items set in **Calendar** are also displayed on the left of the notification pane, and the summary information is displayed below the calendar on the right. + +![](./figures/gnome-14.png) + +You can click **Do Not Disturb** to close pop-up notifications on the desktop. + +#### 3.2.2 Calendar + +As shown in the preceding figure, the calendar is displayed on the right, and there is a dot under the date of a to-do item. You can click the date to view the summary about a to-do item at the bottom of the calendar. + +#### 3.2.3 Clock and Weather + +You can also add the clock and weather to areas under the calendar. Clicking the **World Clocks** area will invoke the **Clocks** app, and clicking the **Weather** area will invoke the **Weather** app. + +![](./figures/gnome-15.png) + +### 3.3 System Status + +The system status entry is located in the upper right corner of the desktop. It contains multiple options, as described in the following table. + +| Option| Description| +| :------------ | :------------ | +| Sound| Volume slider| +| Ethernet| Ethernet cards and their connections| +| Location In Use| Location of the system| +| Settings| System settings| +| Lock| Immediate screen lock. A password is required to unlock the screen.| +| Power Off/Log Out| Suspension, shutdown, restart, and logout| + +![](./figures/gnome-16.png) + +The system status displayed here varies according to different settings and system configurations, such as Wi-Fi, Bluetooth, and battery. System statuses can also be appended to the left of the upper right corner by other apps, such as the input source display in the preceding figure. + +#### 3.3.1 Sound + +Quickly adjust the volume. To further set the sound, open the system settings. + +#### 3.3.2 Network + +Quickly enable or disable the network. To further configure the network, open the system settings. + +![](./figures/gnome-17.png) + +#### 3.3.3 Location Service + +Quick enable or disable the location service. To further set the location, open the system settings. + +![](./figures/gnome-18.png) + +#### 3.3.4 Settings + +It is one of the convenient entries to system settings. + +![](./figures/gnome-19.png) + +You can set a large number of system-related options in the **Settings** window, which are shown in the left pane of the preceding and following figures. + +![](./figures/gnome-20.png) + +The settings are also dynamically extended. For example, if the hardware where the system is located has Wi-Fi, the Wi-Fi item is displayed. Some important settings are described in the following sections. + +#### 3.3.4 Lock + +If you click **Lock**, the screen is locked and turns black. When you move the cursor, the screen turns on immediately. You can press any key to access the login page and enter the password to log in to the system again. The following figure shows the lock screen. + +![](./figures/gnome-21.png) + +#### 3.3.4 Power-off/Logout + +The actions include suspension, power-off, restart, and logout. The difference between suspension and locking is that a black screen is directly displayed after suspension. You need to use the keyboard to wake up the login page, which takes a longer time than screen locking. Logout is to log out the current user and return to the login page without a black screen. You can use the same or another user account to log in again. + +![](./figures/gnome-22.png) + +The following figure shows the user login page. + +![](./figures/gnome-23.png) + +After the locking and suspension is waked up, the lock screen is displayed first. You need to press a key or click the screen to enter the user login page. The login page is directly displayed after the logout and restart. + +## 4. Common System Settings and App Examples + +### 4.1 Examples of System Settings + +There are four entries to system settings: + +Right-click on the desktop and choose **Settings**. + +Click the system status entry in the upper right corner and choose **Settings**. + +Click the **Activities** entry in the upper left corner and choose **Settings**. + +On the **Terminal**, run the **gnome-control-center** command. + +#### 4.1.1 Network + +![](./figures/gnome-19.png) + +Wired networks are displayed here. You can click the button to enable or disable a network. You can also set the VPN and network proxy. + +Click the gear icon on the right of an Ethernet connection to view details, and modify or remove the connection. + +![](./figures/gnome-24.png) + +Change the connection name. + +![](./figures/gnome-25.png) + +Change the IP address obtaining mode (**Automatic** or **Manual**), and add the DNS and a route. + +![](./figures/gnome-26.png) + +You can also click the plus sign (+) above the gear icon to create a connection. The settings of the new connection are similar to those shown in preceding figures. The prerequisite is that the Ethernet port exists. + +#### 4.1.2 Displays + +You can set the fixed resolution on the **Displays** tab page. If the resolution of your hardware system is not included, set it on the command line. Then, the newly set resolution will be displayed here. + +![](./figures/gnome-27.png) + +Select a resolution and click **Keep Changes** to make the settings take effect. + +![](./figures/gnome-28.png) + +Some displays allow you to rotate the screen vertically, for example, to view the text at the bottom of the screen at a time. The **Orientation** here also provides such support. + +![](./figures/gnome-29.png) + +#### 4.1.3 Keyboard Shortcuts + +You can set keyboard shortcuts to perform shortcut operations, such as quickly opening the home folder, camera, or browser. GNOME does not provide a shortcut for starting the **Terminal**. You can set a default one. + +View existing shortcut settings in scrolling mode or search for shortcuts. + +![](./figures/gnome-30.png) + +Clicking a disabled item, such as the home folder and web browser, triggers shortcut settings. + +![](./figures/gnome-31.png) + +![](./figures/gnome-32.png) + +Effect after the setting is successful. + +![](./figures/gnome-33.png) + +Scroll the keyboard shortcuts page to the bottom and click + to add a shortcut for opening the **Terminal**. + +![](./figures/gnome-34.png) + +![](./figures/gnome-35.png) + +![](./figures/gnome-36.png) + +![](./figures/gnome-37.png) + +Now, you can press **Ctrl+Alt+T** to open the **Terminal**. Settings of the home folder and web browser are similar. + +![](./figures/gnome-38.png) + +#### 4.1.4 Region and Language + +The system can be switched between multiple languages, even if a language is not selected during system installation. + +![](./figures/gnome-39.png) + +You can click **Language** and **Formats** to change the language from Chinese to English, and click **Restart**. You need to log in to the system again and restart the session for the language settings to take effect. + +![](./figures/gnome-40.png) + +![](./figures/gnome-41.png) + +![](./figures/gnome-42.png) + +Click the gear icon on the right of **Input Sources** to view the keyboard shortcuts and input source options. You can click the plus sign (+) to add an input source. + +![](./figures/gnome-43.png) + +When you use the shortcut to switch the input method, you can view the change in the system status area in the upper right corner. + +![](./figures/gnome-44.png) + +#### 4.1.5 Users + +You can add and delete users on the **Users** GUI. For a non-root user, you need to click **Unlock** and enter the password of the super user to display the complete information. + +![](./figures/gnome-45.png) + +Click **Password** to change the password of the current user. + +![](./figures/gnome-46.png) + +Click **Account Activity** to view the login status of the user in this week. + +![](./figures/gnome-47.png) + +Click **Add User** in the upper right corner to add a user and set the password when adding the user or when logging in to the system as the new user. To log in to the system as a new user, log out of the system and then log in as the new user. The new user can be removed by clicking **Remove User**. The current login user cannot be removed. + +![](./figures/gnome-48.png) + +### 4.2 Application Examples + +#### 4.2.1 Files + +The binary file name of the **Files** app is **nautilus**. You can create, modify, move, save, and delete files in the file system displayed in **Files**. + +![](./figures/gnome-49.png) + +#### 4.2.2 Terminal + +The running **Terminal** is a special process under the GNOME login session. It functions as a console and is a new session in essence. It can perform almost all the tasks that the console can do, and it is what Linux would be without a graphical interface. + +![](./figures/gnome-50.png) + +In the **Preferences** dialog box, you can set the font, character spacing, and theme background. + +#### 4.2.3 Software + +In **Software**, you can search for and install many free open source apps, and view and uninstall installed apps. + +![](./figures/gnome-51.png) + +![](./figures/gnome-52.png) + +#### 4.2.4 Browser + +GNOME has a built-in browser named **Web**. Its interface and functions are simpler than those of Chrome or Firefox, but supports common functions, such as bookmarks, search engine settings, history, and file download. + +![](./figures/gnome-53.png) + +#### 4.2.5 System Monitor + +It is similar to the Task Manager in Windows operating systems, on which you can view the process name, user, and usage of CPU and memory resources. This monitor is dynamic, but its change effect is much worse than that of running the top command. + +![](./figures/gnome-54.png) + +You can also view the usage trend of important components such as the CPU, memory, and network. + +![](./figures/gnome-55.png) + +#### 4.2.6 Text Editor + +A text editor is required for creating, modifying, and saving files. In its **Preferences** dialog box, you can set the font, tab width, theme, and plug-ins. + +![](./figures/gnome-56.png) + +#### 4.2.7 Sysprof + +Sysprof samples and presents a system, including the software and hardware, and is used to locate system performance problems, for example, app startup freezing and system response delay. You can select the project to be traced and click **Record** to start sampling. + +![](./figures/gnome-57.png) + +![](./figures/gnome-58.png) + +After the sampling is stopped, the result provides abundant information for diagnosis and analysis. + +![](./figures/gnome-59.png) diff --git a/docs/en/tools/desktop/kiran/_toc.yaml b/docs/en/tools/desktop/kiran/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..7fc08a0a7d0353741a4a8b6d330de980bdefc62d --- /dev/null +++ b/docs/en/tools/desktop/kiran/_toc.yaml @@ -0,0 +1,8 @@ +label: Kiran User Guide +isManual: true +description: Install and use Kiran. +sections: + - label: Kiran Installation + href: ./kiran-installation.md + - label: Kiran User Guide + href: ./kiran-user-guide.md diff --git a/docs/en/tools/desktop/kiran/figures/kiran-1.png b/docs/en/tools/desktop/kiran/figures/kiran-1.png new file mode 100644 index 0000000000000000000000000000000000000000..59b632062ba3ff6e26c550567e858eb4dfdfc780 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-1.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-10.png b/docs/en/tools/desktop/kiran/figures/kiran-10.png new file mode 100644 index 0000000000000000000000000000000000000000..18cfa3074af1f4b8d49d064a77b016f24ab8c17c Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-10.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-11.png b/docs/en/tools/desktop/kiran/figures/kiran-11.png new file mode 100644 index 0000000000000000000000000000000000000000..b58fbb7ce8a798d5355855a4ac0638540df74d9e Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-11.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-12.png b/docs/en/tools/desktop/kiran/figures/kiran-12.png new file mode 100644 index 0000000000000000000000000000000000000000..920d0c7112be6bed509773413de36506d748b822 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-12.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-13.png b/docs/en/tools/desktop/kiran/figures/kiran-13.png new file mode 100644 index 0000000000000000000000000000000000000000..f6632732bd2e8a10d0cda2bd0550f43741a7ba97 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-13.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-14.png b/docs/en/tools/desktop/kiran/figures/kiran-14.png new file mode 100644 index 0000000000000000000000000000000000000000..52eae7cc40fe4f7c6b2a8fe9744209a1fcbc30d8 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-14.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-15.png b/docs/en/tools/desktop/kiran/figures/kiran-15.png new file mode 100644 index 0000000000000000000000000000000000000000..5496c56ca72983780b9785d2d15c4008fb73aeef Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-15.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-16.png b/docs/en/tools/desktop/kiran/figures/kiran-16.png new file mode 100644 index 0000000000000000000000000000000000000000..6125b257245aa89f9b6592ed5b14a95d5699076e Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-16.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-17.png b/docs/en/tools/desktop/kiran/figures/kiran-17.png new file mode 100644 index 0000000000000000000000000000000000000000..d8a4cb88017efe9f41f78ffc2f9de06dedcc1b23 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-17.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-18.png b/docs/en/tools/desktop/kiran/figures/kiran-18.png new file mode 100644 index 0000000000000000000000000000000000000000..0cb0c50d15597998fbd4cf3db2d1d0f9ec3c920e Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-18.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-19.png b/docs/en/tools/desktop/kiran/figures/kiran-19.png new file mode 100644 index 0000000000000000000000000000000000000000..58ef2d33a52cf6404ea03b6a2d37f8d8b8391539 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-19.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-2.png b/docs/en/tools/desktop/kiran/figures/kiran-2.png new file mode 100644 index 0000000000000000000000000000000000000000..088bf53c1e763924e7cee46d0cdac98ad0a9d5e2 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-2.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-20.png b/docs/en/tools/desktop/kiran/figures/kiran-20.png new file mode 100644 index 0000000000000000000000000000000000000000..e8608485553033eb2ae141162e4300fa48c578cd Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-20.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-21.png b/docs/en/tools/desktop/kiran/figures/kiran-21.png new file mode 100644 index 0000000000000000000000000000000000000000..4d4c0ff304bdfbc8e715d2e756315a005c008336 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-21.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-22.png b/docs/en/tools/desktop/kiran/figures/kiran-22.png new file mode 100644 index 0000000000000000000000000000000000000000..6778d5a40a82e699da9531f4727a196d1442b9ae Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-22.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-23.png b/docs/en/tools/desktop/kiran/figures/kiran-23.png new file mode 100644 index 0000000000000000000000000000000000000000..fc1d5e284eb299a771c5abbfdff611270ddf2449 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-23.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-24.png b/docs/en/tools/desktop/kiran/figures/kiran-24.png new file mode 100644 index 0000000000000000000000000000000000000000..a3ed57f9e9c300a65f867d29a44f287405a0509c Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-24.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-25.png b/docs/en/tools/desktop/kiran/figures/kiran-25.png new file mode 100644 index 0000000000000000000000000000000000000000..694e6173dfbf1fda8d07670a8e3daf4fbeb263ac Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-25.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-26.png b/docs/en/tools/desktop/kiran/figures/kiran-26.png new file mode 100644 index 0000000000000000000000000000000000000000..3b6ae2eeff3aae39107f15b60c5bb14ffc787cd8 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-26.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-27.png b/docs/en/tools/desktop/kiran/figures/kiran-27.png new file mode 100644 index 0000000000000000000000000000000000000000..3b6ae2eeff3aae39107f15b60c5bb14ffc787cd8 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-27.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-28.png b/docs/en/tools/desktop/kiran/figures/kiran-28.png new file mode 100644 index 0000000000000000000000000000000000000000..01ff3a8f47248d96c714e78b80fd81cd1ed16e0f Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-28.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-29.png b/docs/en/tools/desktop/kiran/figures/kiran-29.png new file mode 100644 index 0000000000000000000000000000000000000000..c5ad5b4438eae441f6086ce5e1aae2e6755aa12a Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-29.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-3.png b/docs/en/tools/desktop/kiran/figures/kiran-3.png new file mode 100644 index 0000000000000000000000000000000000000000..e1399424c52eee8804f9433c9e9bf203950008c6 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-3.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-30.png b/docs/en/tools/desktop/kiran/figures/kiran-30.png new file mode 100644 index 0000000000000000000000000000000000000000..c1efc1e3931a129affd5dfcea9e319556e492f04 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-30.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-31.png b/docs/en/tools/desktop/kiran/figures/kiran-31.png new file mode 100644 index 0000000000000000000000000000000000000000..c5ad5b4438eae441f6086ce5e1aae2e6755aa12a Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-31.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-32.png b/docs/en/tools/desktop/kiran/figures/kiran-32.png new file mode 100644 index 0000000000000000000000000000000000000000..fd900ec891b09313a7c558c61213b1816b803034 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-32.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-33.png b/docs/en/tools/desktop/kiran/figures/kiran-33.png new file mode 100644 index 0000000000000000000000000000000000000000..64ba70b08ed63c6e0942478d61e36a8c443f0604 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-33.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-34.png b/docs/en/tools/desktop/kiran/figures/kiran-34.png new file mode 100644 index 0000000000000000000000000000000000000000..4b869e7d172e2f2889d487157b92204a28a8dc4e Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-34.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-35.png b/docs/en/tools/desktop/kiran/figures/kiran-35.png new file mode 100644 index 0000000000000000000000000000000000000000..9b383f3c84964b4fc34c4d8e75400325f93908bc Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-35.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-36.png b/docs/en/tools/desktop/kiran/figures/kiran-36.png new file mode 100644 index 0000000000000000000000000000000000000000..0b16632852c5024e2c6ec4fbd49513e3b7a2b146 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-36.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-37.png b/docs/en/tools/desktop/kiran/figures/kiran-37.png new file mode 100644 index 0000000000000000000000000000000000000000..2be3cc3b2528260c579b59f529e7a5663f1cc779 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-37.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-38.png b/docs/en/tools/desktop/kiran/figures/kiran-38.png new file mode 100644 index 0000000000000000000000000000000000000000..fc1ffaf3aa920f922357f6d48700f42974600d77 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-38.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-39.png b/docs/en/tools/desktop/kiran/figures/kiran-39.png new file mode 100644 index 0000000000000000000000000000000000000000..fd0e5add782b6c9cf4a8b9f6473c96641c39bd1d Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-39.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-4.png b/docs/en/tools/desktop/kiran/figures/kiran-4.png new file mode 100644 index 0000000000000000000000000000000000000000..bd318280b403912ab4846b694592d580b9e5d242 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-4.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-40.png b/docs/en/tools/desktop/kiran/figures/kiran-40.png new file mode 100644 index 0000000000000000000000000000000000000000..083031058ff47dc1550881d3a9f189861d3e8563 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-40.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-41.png b/docs/en/tools/desktop/kiran/figures/kiran-41.png new file mode 100644 index 0000000000000000000000000000000000000000..582893929e2c10a96c49696411bbed3ea9fd7c55 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-41.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-42.png b/docs/en/tools/desktop/kiran/figures/kiran-42.png new file mode 100644 index 0000000000000000000000000000000000000000..eede1243506ccd309ee707465f56c31581dd8554 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-42.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-43.png b/docs/en/tools/desktop/kiran/figures/kiran-43.png new file mode 100644 index 0000000000000000000000000000000000000000..4ea9f45ed8f327fce426352c4ae7fbf06cbefc84 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-43.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-44.png b/docs/en/tools/desktop/kiran/figures/kiran-44.png new file mode 100644 index 0000000000000000000000000000000000000000..c86a100005f89dbb9b24055e42d716205d47399e Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-44.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-45.png b/docs/en/tools/desktop/kiran/figures/kiran-45.png new file mode 100644 index 0000000000000000000000000000000000000000..c5b5d75f972e594587f3393c8d384dcd76e7477e Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-45.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-46.png b/docs/en/tools/desktop/kiran/figures/kiran-46.png new file mode 100644 index 0000000000000000000000000000000000000000..e9a28632c62de95d8ea2d436ba9bc705ff980991 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-46.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-47.png b/docs/en/tools/desktop/kiran/figures/kiran-47.png new file mode 100644 index 0000000000000000000000000000000000000000..a3606e3c899f944eb84d206d98cedc3377197c97 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-47.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-48.png b/docs/en/tools/desktop/kiran/figures/kiran-48.png new file mode 100644 index 0000000000000000000000000000000000000000..b69202c9a83bfc2c835ab166ef0fc2455bb4bcd3 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-48.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-49.png b/docs/en/tools/desktop/kiran/figures/kiran-49.png new file mode 100644 index 0000000000000000000000000000000000000000..d739e6107fd80ecd741dacaaf9dfb868afc61e37 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-49.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-5.png b/docs/en/tools/desktop/kiran/figures/kiran-5.png new file mode 100644 index 0000000000000000000000000000000000000000..154dd54d43b5b98682eb798518046e72fc7e3f83 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-5.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-50.png b/docs/en/tools/desktop/kiran/figures/kiran-50.png new file mode 100644 index 0000000000000000000000000000000000000000..96957676afc9f66bcc4b63c5e39eb8890f108015 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-50.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-6.png b/docs/en/tools/desktop/kiran/figures/kiran-6.png new file mode 100644 index 0000000000000000000000000000000000000000..927b475d6687d60f04fed8a535b2225a8f4b23f7 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-6.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-7.png b/docs/en/tools/desktop/kiran/figures/kiran-7.png new file mode 100644 index 0000000000000000000000000000000000000000..254ef11f36d958f6ef7c70853e5f61032f825463 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-7.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-8.png b/docs/en/tools/desktop/kiran/figures/kiran-8.png new file mode 100644 index 0000000000000000000000000000000000000000..29b5845d2fa94cba92719b8649a5e86c926ea911 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-8.png differ diff --git a/docs/en/tools/desktop/kiran/figures/kiran-9.png b/docs/en/tools/desktop/kiran/figures/kiran-9.png new file mode 100644 index 0000000000000000000000000000000000000000..46bcfdd0e1e88ad0f0ade4a3990c3ac5d66060e7 Binary files /dev/null and b/docs/en/tools/desktop/kiran/figures/kiran-9.png differ diff --git a/docs/en/tools/desktop/kiran/kiran-installation.md b/docs/en/tools/desktop/kiran/kiran-installation.md new file mode 100644 index 0000000000000000000000000000000000000000..4aab96decb8c766339c2913d7c671e27d81c61c9 --- /dev/null +++ b/docs/en/tools/desktop/kiran/kiran-installation.md @@ -0,0 +1,31 @@ +# Kiran Installation + +## Introduction + +Kiran desktop environment, developed by Kylinsec, is a stable, efficient, and easy-to-use desktop environment oriented towards user and market requirements. Kiran supports x86 and AArch64 architectures. + +## Procedure + +You are advised to install Kiran as the **root** user or a newly created administrator. + +1. Download the openEuler 22.09 ISO file and install the OS. + +2. Update the software repository. + +```shell +sudo dnf update +``` + +1. Install kiran-desktop. + +```shell +sudo dnf -y install kiran-desktop +``` + +1. Set the system to start with the graphical interface, and then restart the system using the `reboot` command. + +```shell +systemctl set-default graphical.target +``` + +After the reboot is complete, log in to the Kiran desktop. diff --git a/docs/en/tools/desktop/kiran/kiran-user-guide.md b/docs/en/tools/desktop/kiran/kiran-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..7780aa7b5621ec35fa8976c19bb4fafa3cb6dd6a --- /dev/null +++ b/docs/en/tools/desktop/kiran/kiran-user-guide.md @@ -0,0 +1,298 @@ +# Kiran Desktop Environment + +## 1. Overview + +Kiran desktop environment is a stable, efficient, and easy-to-use desktop environment oriented towards user and market requirements. It consists of the desktop, taskbar, tray, control center, and window management components. This document describes how to use the Kiran desktop. + +## 2. Desktop + +### 2.1. Login Screen + +After the installation is complete, restart the system. After the system is started, enter the user name and password to log in to the system. The login screen displays the time, date, power button, and soft keyboard button. The adaptive UI supports screen zooming and multi-screen display. The login dialog box can be switched between screens following the mouse pointer. + +![Figure 1 Login screen](figures/kiran-1.png) + +### 2.2. Main Screen + +Enter the correct user name and password to log in to the system. The main screen is displayed, as shown in the following figure: + +![Figure 2-Main screen ](figures/kiran-2.png) + +Several icons are displayed on the desktop, such as **Computer**, **Home** folder, and **Trash**. The panel, located at the bottom of the screen, allows you to launch applications and switch between virtual desktops. +A desktop is a working area of a user. You will perform operations and run applications on the desktop. You can place the files and applications on the desktop for easy access. Double-click the icons to run the corresponding applications or open the files. You can drag, add, or delete desktop icons. Desktop icons allow you to complete your work more conveniently. + +![Figure 3-Computer](figures/kiran-3.png) **Computer**: Double-click to display all the local and remote disks and folders accessed from this computer. + +![Figure 4-Home folder](figures/kiran-4.png) **Home** folder: Double-click to display the contents in the home directory of the current user. + +![Figure 5- Trash](figures/kiran-5.png) **Trash**: Deleted files are temporarily stored in Trash. + +Shortcut menu: Right-click on the desktop to display the shortcut menu, which provides shortcuts for icon management, folder creation, document creation, desktop background settings, and theme settings. + +**Create Folder**: Creates a folder. + +**Create Launcher...**: Creates a launcher. + +**Create Document**: Creates an empty plain-text file. + +**Open Terminal...**: Opens the terminal application. + +**Organize Desktop by Name**: Sorts desktop files by name. + +**Keep Aligned**: If this option is selected, the desktop icons are aligned to the grid. + +**Change Desktop Background**: Opens Background to change the background picture of the desktop or lock screen. + +### 2.3 Panel + +The panel is usually located at the bottom of the screen and includes the start menu button, quick launch area, icons of frequently used applications and desktop applets, and taskbar that displays the currently running application. +When you hover the mouse pointer over an icon for several seconds, a white dialog box is displayed, describing the function of the icon. + +![Figure 6-System panel](figures/kiran-6.png) + +## 3. Taskbar + +Taskbar: displays running applications or opened documents. You can click an item on the taskbar to maximize or minimize the selected application window. You can right-click an item and choose Maximize, Minimize, or Close the application window from the shortcut menu. + +| Component | Description | +| :------------------------------------------------------ | :------------------------------------------------------------------------------------------------------------------ | +| ![Figure 7-Start Menu](figures/kiran-7.png) | Start Menu button: Similar to the Start button in Windows. When you click it, the cascaded start menu is displayed. | +| ![Figure 8 Workspace button](figures/kiran-8.png) | Click to display the workspaces. | +| ![Figure 9 File browser button](figures/kiran-9.png) | Click to start the file browser to view and manage files. | +| ![Figure 10 Terminal button](figures/kiran-10.png) | Click to start the terminal. | +| ![Figure 11 Web browser button](figures/kiran-11.png) | Click to start the Firefox browser. | +| ![Figure 12 Network control icon](figures/kiran-12.png) | Displays the current network status. Click to modify the network configuration. | +| ![Figure 13-Clock button](figures/kiran-13.png) | Displays the current date and time. You can customize the display style as required. | + +## 4. Control Center + +### 4.1. Start Menu Settings + +Choose **Start Menu** > **Control Center** > **Start Menu** Settings. +You can set the display mode and opacity of the start menu, as shown in the following figure: + +![Figure 14-Start Menu Settings](figures/kiran-14.png) + +The appearance start menu changes based on the opacity and display mode, as shown in the following figure: + +![Figure 15-Start menu](figures/kiran-15.png) + +### 4.2. Greeter Settings + +Choose **Start Menu** > **Control Center** > **Greeter Settings**. +In the Kiran desktop, you can set the login screen appearance by choosing **Greeter Settings** in **Control Center**, including the background image of the login screen, whether to enable automatic login, zoom ratio, whether to allow login by entering the user name, and whether to display the user list, as shown in the following figure: + +![Figure 16-Greeter Settings](figures/kiran-16.png) + +You can also set automatic login. Set the user name and delay for automatic login. After the system is restarted, the user automatically logs in without entering the password. + +![Figure 17 Autologin settings](figures/kiran-17.png) + +### 4.3 Display Settings + +Display attribute customization is required for every desktop environment. The Kiran desktop provides a powerful tool for customizing display attributes. You can choose **Start Menu** > **Control Center** > **Display Settings** to open the **Display Settings** window, as shown in the following figure: + +![Figure 18-Display Settings](figures/kiran-18.png) + +You can set the screen rotation, resolution, refresh rate, zoom rate, and flip. After the settings are complete, click **Apply**. + +### 4.4 Mouse Settings + +Configure the mouse by selecting **Kiran Cpanel Mouse** in **Control Center**. You can select left-hand or right-hand mode, adjust the mouse pointer speed, set whether to scroll naturally, and set whether to enable the middle button emulation by pressing the left and right button simultaneously. The following figure shows the normal mouse setting window: + +![Figure 19-Kiran Cpanel Mouse](figures/kiran-19.png) + +### 4.5. Account Manager + +Account Manager is an easy-to-use tool for managing users and user groups. You can use this tool to: + +1. Add users and set user attributes. +2. Modify user attributes. +3. View user attributes. +4. Delete users. + +User attributes include the user name, password, and login shell. User group attribute indicates the users in the user group. + +#### 4.5.1 Starting Account Manager + +In **Control Center**, choose **Account Manager** to start the account management tool, as shown in the following figure: + +![Figure 20 Account Manager](figures/kiran-20.png) + +In the window, you can see the user list on the left and the detailed information on the right. Currently, all users in the system except the root user are listed. Click a user on the left. The detailed information about the user is displayed, including the user ID and user type. +Click **Create new user**. On the page that is displayed on the right, enter the user name, user type, and password, and change the avatar as required. After setting the attributes, click **Confirm**. + +![Figure 21 Creating an account](figures/kiran-21.png) + +**Note**: If you have set the minimum length of a password (for example, four digits), you must enter a password of at least four digits. Otherwise, the system will not accept the password. + +Click the avatar area to change the avatar. The system has built-in avatars for you to select. You can also add your own avatar and click Confirm to save the settings. + +![Figure 22-Change avatar](figures/kiran-22.png) + +#### 4.5.2. Deleting a User + +Click the user to be deleted in the left area and click **Delete** on the toolbar on the right, as shown in the following figures: + +![Figure 23 Deleting a user](figures/kiran-23.png) +![Figure 24-Confirming the deletion](figures/kiran-24.png) + +In the displayed dialog box, click **No** to cancel the deletion, or click **Yes** to confirm the deletion. + +#### 4.5.3. Advanced Settings + +Choose **Create new user**, enter the user name and password, and then choose **Advanced Settings**. In the displayed dialog box, set the login shell, user ID, and user home directory. + +![Figure 25-Advanced Settings](figures/kiran-25.png) + +### 4.6. Appearance + +Display attribute customization is required for every desktop environment. The Kiran desktop provides a powerful tool for customizing display attributes. Appearance is a tool that provides unified configuration and management for the desktop background, theme, and font of the system. +Choose **Start Menu** > **Control Center** > **Appearance**. The **Appearance** window is displayed, as shown in the following figure: + +![Figure 26-Appearance Preferences](figures/kiran-26.png) + +#### 4.6.1. Theme + +Theme can be used to set the style of the dialog boxes, menus, system panels, and icons in a unified manner or separately according to your preference. + +1. Theme Settings + The system provides multiple themes by default. You can view the theme information in the **Theme** tab page. Click the theme in the **Theme** tab page to set the system theme, as shown in the following figure: + + ![Figure 27 Theme settings](figures/kiran-27.png) + +2. Customizing a Theme + You can click Customize... to customize a theme based on your preferences, as shown in the following figure. Customization options include controls, color, window border, icons, and pointer. + + ![Figure 28-Customize theme](figures/kiran-28.png) + +#### 4.6.2. Background + +You can set the desktop background, including the color and style. + +1. Background image settings + As shown in the following figure, click a wallpaper in the wallpaper area to set it as the desktop wallpaper. + + ![Figure 29-Background Settings](figures/kiran-29.png) + +2. Style + You can choose how the wallpaper fits the screen by choosing a style from the drop-down list. The styles include tile, zoom, center, scale, stretch, and span. + +3. Adding and removing wallpapers + You can click **Add...** to add your own wallpaper, as shown in the following figure: + + ![Figure 30-Add wallpaper](figures/kiran-30.png) + + Click **Open** to add the wallpaper. + You can also click **Remove** to remove wallpapers that you do not like. Simply select a wallpaper and click **Remove**. + +4. Desktop background color filling settings + You can set a color as the background. In the wallpaper tab page, choose **No Desktop Background** to use a color as the background. + The color filling styles include solid color, horizontal gradient, and vertical gradient. + + ![Figure 31-Background color filling](figures/kiran-31.png) + +#### 4.6.3. Font + +1. Font Settings + +You can set the fonts of the GUI of the system. The font styles include application, document, desktop, window title, and fixed width fonts. + +![Figure 32-Font settings](figures/kiran-32.png) + +1. Font rendering and details settings + +Font rendering settings: You can choose one of the following font rendering styles: monochrome, best shapes, best contrast, and subpixel smoothing. +By default, **best shapes** is used, as shown in the following figure: + +![Figure 33 Font rendering settings](figures/kiran-33.png) + +1. Font Details Settings + You can click **Details...** to set the font details. Details settings include resolution, smoothing, hinting, and subpixel order. + +![Figure 34-Font details setting](figures/kiran-34.png) + +You can choose whether to display icons in menus and on buttons. + +![Figure 35-Icon display settings](figures/kiran-35.png) + +## 5. Desktop Applications + +### 5.1. Text Editor + +To launch the text editor, click **Start Menu**> **All applications** > **Utilities**> **Pluma**. You can also start the text editor by entering **pluma** in the shell prompt. +A text editor is one of the most commonly used tools in all computer systems. Whether to you are creating a plain text file, data file, or source program, you need to use an editor. The text editor is used to view and modify plain text files. Plain text files, such as system logs and configuration files, are common text files that do not contain fonts or style formats. + +![Figure 36-Text editor](figures/kiran-36.png) + +### 5.2. Terminal + +In the desktop environment, you can use the Terminal application to enter the command line interface. To start Terminal, choose **Start Menu** > **All applications** > **Utilities** > **Terminal**, or click the icon on the desktop panel. + +![Figure 37-Terminal](figures/kiran-37.png) + +### 5.3. Firefox + +To launch Firefox, click **Start Menu** > **All applications** > **Network** > **Firefox**. +Firefox is a free and open source web browser. It uses the Gecko rendering engine and supports multiple operating systems, such as Windows, Mac OS X, and GNU/Linux. Firefox is small in size, fast in speed, and has other advanced features, such as tabbed browsing, faster loading speed, pop-up blocker, customizable toolbar, extension management, better search features, and a convenient sidebar. + +![Figure 38 Firefox](figures/kiran-38.png) + +### 5.4 Screenshot Tool + +Choose **Start Menu** > **All applications** > **Graphics** > **Screenshot tool** to start the screenshot tool. +Screenshot tool is a small and flexible screenshot software of the Kiran desktop. The operation UI is simple and easy to use. When the software is started, the icon of the screenshot tool is added to the tray. + +![Figure 39-Screenshot icon in the tray](figures/kiran-39.png) + +Click the icon to display the screenshot interface. You can select the screenshot area. You can right-click the icon and choose Open Launcher to set the capture area and delay. + +![Figure 40-Screenshot UI](figures/kiran-40.png) +![Figure 41-Launcher UI](figures/kiran-41.png) + +In the displayed dialog box, click **√** to save the file to the desktop, or choose Options and select a custom save location, as shown in the following figure: + +![Figure 42-Screenshot process](figures/kiran-42.png) + +### 5.5 Network Settings + +The Kiran desktop uses NetworkManager as the network configuration tool. NetworkManager can set, configure, and manage various network types, and provides advanced support for mobile broadband devices, Bluetooth, and IPv6 protocol. Choose Start Menu > **Control Center** > **Advanced Network Configuration**, or right-click the network icon in the lower right corner of the desktop and choose **Edit Connections...**, as shown in the following figure: + +![Figure 43-NetworkManager](figures/kiran-43.png) + +Wired connection settings: +Select the current NIC. For example, **ens33** is the NIC of the current system. Select the NIC and click the edit button. The NIC editing dialog box is displayed: + +![Figure 44 Editing an NIC](figures/kiran-44.png) + +IPv4 Settings are frequently used. In this example, DHCP is selected to obtain the IP address and DNS server. The system automatically obtains an IP address for the user. +When you need to manually enter the IP address, select **Manual** from the **Method** drop-down list, as shown in the following figure: + +![Figure 45 IPv4 settings](figures/kiran-45.png) + +Click **Add**, enter the IP address, subnet mask, gateway, and DNS server, as shown in the following figure: + +![Figure 46 Setting the network IP address and DNS](figures/kiran-46.png) + +Enter the IP address, subnet mask, gateway, and DNS, and Click **Save**. Click the network icon in the lower right corner of the desktop, choose **Disconnect** to disconnect from the network, and then reconnect to the network. + +--- + +### 5.6. Time and Date Manager + +To set the date and time, select **Time And Date Manager** in **Control Center**, or click the date area in the lower right corner of the desktop. The following window is displayed: + +![Figure 47 Time And Date Manager](figures/kiran-47.png) + +Automatic synchronization: Enable **Automatic synchronization** and connect to the Internet to automatically synchronize the date and time. +Time zone settings: Click **Change Time Zone**, select a time zone from the list on the right, and then click **Save**. + +![Figure 48 Change Time Zone](figures/kiran-48.png) + +Manually set the time: Disable **Automatic synchronization** and click **Set Time Manually** to manually set the year, month, day, and time. After the modification is complete, click **Save**. + +![Figure 49 Set Time Manually](figures/kiran-49.png) + +Modifying the date format: Click **Time date format setting** to modify the date format. You can set the long and short date display formats, time format, and whether to display seconds. + +![Figure 50 Time date format setting](figures/kiran-50.png) diff --git a/docs/en/tools/desktop/ukui/_toc.yaml b/docs/en/tools/desktop/ukui/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..6489bd1bfaf4ba173ccb0b818faeae28b7e24d84 --- /dev/null +++ b/docs/en/tools/desktop/ukui/_toc.yaml @@ -0,0 +1,8 @@ +label: UKUI User Guide +isManual: true +description: Install and use UKUI. +sections: + - label: UKUI Installation + href: ./ukui-installation.md + - label: UKUI User Guide + href: ./ukui-user-guide.md diff --git a/docs/en/tools/desktop/ukui/figures/1.png b/docs/en/tools/desktop/ukui/figures/1.png new file mode 100644 index 0000000000000000000000000000000000000000..40af4242eebb440a76c749a8d970d50cd7b89bf4 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/1.png differ diff --git a/docs/en/tools/desktop/ukui/figures/10.png b/docs/en/tools/desktop/ukui/figures/10.png new file mode 100644 index 0000000000000000000000000000000000000000..e588ffbe3d8d7b66d92ae8f2b4bcec7c80d0592c Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/10.png differ diff --git a/docs/en/tools/desktop/ukui/figures/11.png b/docs/en/tools/desktop/ukui/figures/11.png new file mode 100644 index 0000000000000000000000000000000000000000..1989a5bb08155f920363e154e68bb148715c7e9e Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/11.png differ diff --git a/docs/en/tools/desktop/ukui/figures/12.png b/docs/en/tools/desktop/ukui/figures/12.png new file mode 100644 index 0000000000000000000000000000000000000000..cb6346161182d2cfeaf3818d5ec518ddb11c732e Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/12.png differ diff --git a/docs/en/tools/desktop/ukui/figures/13.png b/docs/en/tools/desktop/ukui/figures/13.png new file mode 100644 index 0000000000000000000000000000000000000000..0a7def1fb66c90da62acde799eaffca97e3b5396 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/13.png differ diff --git a/docs/en/tools/desktop/ukui/figures/14.png b/docs/en/tools/desktop/ukui/figures/14.png new file mode 100644 index 0000000000000000000000000000000000000000..3a27a66d57e284775420d467f90dcc02889bbffe Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/14.png differ diff --git a/docs/en/tools/desktop/ukui/figures/15.png b/docs/en/tools/desktop/ukui/figures/15.png new file mode 100644 index 0000000000000000000000000000000000000000..370bea32abcaa8a2b06a1a61c1455d4b35f43474 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/15.png differ diff --git a/docs/en/tools/desktop/ukui/figures/16.png b/docs/en/tools/desktop/ukui/figures/16.png new file mode 100644 index 0000000000000000000000000000000000000000..812ee462669c5263ef4bffc49ca4f9b6af4541c6 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/16.png differ diff --git a/docs/en/tools/desktop/ukui/figures/17.png b/docs/en/tools/desktop/ukui/figures/17.png new file mode 100644 index 0000000000000000000000000000000000000000..36e524b806874fa3788f5e4dcd78350686281107 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/17.png differ diff --git a/docs/en/tools/desktop/ukui/figures/18.png b/docs/en/tools/desktop/ukui/figures/18.png new file mode 100644 index 0000000000000000000000000000000000000000..51b32442980aa60646f77dabd53ade74f55891fe Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/18.png differ diff --git a/docs/en/tools/desktop/ukui/figures/19.png b/docs/en/tools/desktop/ukui/figures/19.png new file mode 100644 index 0000000000000000000000000000000000000000..c9457d09aa9f1662b2c9e4550cdbdb9f57dd020e Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/19.png differ diff --git a/docs/en/tools/desktop/ukui/figures/2.png b/docs/en/tools/desktop/ukui/figures/2.png new file mode 100644 index 0000000000000000000000000000000000000000..97917cc245484a43bec8562757d920a06f123121 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/2.png differ diff --git a/docs/en/tools/desktop/ukui/figures/20.png b/docs/en/tools/desktop/ukui/figures/20.png new file mode 100644 index 0000000000000000000000000000000000000000..b0943189920d7a541d35da27340593ea93f92a17 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/20.png differ diff --git a/docs/en/tools/desktop/ukui/figures/21.png b/docs/en/tools/desktop/ukui/figures/21.png new file mode 100644 index 0000000000000000000000000000000000000000..e590c22c0ea28906b5f4ea7ccbc6ab11e47ad173 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/21.png differ diff --git a/docs/en/tools/desktop/ukui/figures/22.png b/docs/en/tools/desktop/ukui/figures/22.png new file mode 100644 index 0000000000000000000000000000000000000000..03a548b1ffb1f0ad53cfa5387af2721af90bca81 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/22.png differ diff --git a/docs/en/tools/desktop/ukui/figures/23.png b/docs/en/tools/desktop/ukui/figures/23.png new file mode 100644 index 0000000000000000000000000000000000000000..834c492094715cde1c02c91752ecabfe7921ed62 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/23.png differ diff --git a/docs/en/tools/desktop/ukui/figures/24.png b/docs/en/tools/desktop/ukui/figures/24.png new file mode 100644 index 0000000000000000000000000000000000000000..1881e868b74a60888b319576fa38fb4af92ba75c Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/24.png differ diff --git a/docs/en/tools/desktop/ukui/figures/25.png b/docs/en/tools/desktop/ukui/figures/25.png new file mode 100644 index 0000000000000000000000000000000000000000..f38839725d27a3486984d152e5d9de305364fbd2 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/25.png differ diff --git a/docs/en/tools/desktop/ukui/figures/26.png b/docs/en/tools/desktop/ukui/figures/26.png new file mode 100644 index 0000000000000000000000000000000000000000..6d7957119133ecb98b1b6b104e54a3a4647ec2a5 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/26.png differ diff --git a/docs/en/tools/desktop/ukui/figures/27.png b/docs/en/tools/desktop/ukui/figures/27.png new file mode 100644 index 0000000000000000000000000000000000000000..3e4733717fdc5172d6479b393005219e65e96df4 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/27.png differ diff --git a/docs/en/tools/desktop/ukui/figures/28.png b/docs/en/tools/desktop/ukui/figures/28.png new file mode 100644 index 0000000000000000000000000000000000000000..a77772e818e3f6c11acac3b9cfa18bad14a0a48c Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/28.png differ diff --git a/docs/en/tools/desktop/ukui/figures/29.png b/docs/en/tools/desktop/ukui/figures/29.png new file mode 100644 index 0000000000000000000000000000000000000000..c4f58ffe5855295268298448744e5aadbdc55276 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/29.png differ diff --git a/docs/en/tools/desktop/ukui/figures/3.png b/docs/en/tools/desktop/ukui/figures/3.png new file mode 100644 index 0000000000000000000000000000000000000000..fbb76b336957020ed6867d908e0a8bdcfc953c52 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/3.png differ diff --git a/docs/en/tools/desktop/ukui/figures/30.png b/docs/en/tools/desktop/ukui/figures/30.png new file mode 100644 index 0000000000000000000000000000000000000000..d91adefba1753959e90ccf4aa1501ac08d7144bd Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/30.png differ diff --git a/docs/en/tools/desktop/ukui/figures/31.png b/docs/en/tools/desktop/ukui/figures/31.png new file mode 100644 index 0000000000000000000000000000000000000000..0abef09ab438f5f8cfb68090993f55c493b8c15e Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/31.png differ diff --git a/docs/en/tools/desktop/ukui/figures/32.png b/docs/en/tools/desktop/ukui/figures/32.png new file mode 100644 index 0000000000000000000000000000000000000000..d567cfbacc07a9eb46ff2c54a68432f45e034e94 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/32.png differ diff --git a/docs/en/tools/desktop/ukui/figures/33.png b/docs/en/tools/desktop/ukui/figures/33.png new file mode 100644 index 0000000000000000000000000000000000000000..7b5896e2884520672c0bd88d68471b45a09c56fe Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/33.png differ diff --git a/docs/en/tools/desktop/ukui/figures/34.png b/docs/en/tools/desktop/ukui/figures/34.png new file mode 100644 index 0000000000000000000000000000000000000000..81bc9480fbbd81a97c559d7a6a74274deeab2bd1 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/34.png differ diff --git a/docs/en/tools/desktop/ukui/figures/35.png b/docs/en/tools/desktop/ukui/figures/35.png new file mode 100644 index 0000000000000000000000000000000000000000..ab2399847a643a87279337704e23fea7609bb211 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/35.png differ diff --git a/docs/en/tools/desktop/ukui/figures/36.png b/docs/en/tools/desktop/ukui/figures/36.png new file mode 100644 index 0000000000000000000000000000000000000000..536981609b9ae5d32be56bec612f2b3446146184 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/36.png differ diff --git a/docs/en/tools/desktop/ukui/figures/37.png b/docs/en/tools/desktop/ukui/figures/37.png new file mode 100644 index 0000000000000000000000000000000000000000..e39aa03587642dc1f8622fff515b05a9a3085b28 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/37.png differ diff --git a/docs/en/tools/desktop/ukui/figures/4.png b/docs/en/tools/desktop/ukui/figures/4.png new file mode 100644 index 0000000000000000000000000000000000000000..5078e36aca713706d2cf08a3ebecdc3769951899 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/4.png differ diff --git a/docs/en/tools/desktop/ukui/figures/5.png b/docs/en/tools/desktop/ukui/figures/5.png new file mode 100644 index 0000000000000000000000000000000000000000..2976a745cfaede26594d6daa01cfc18d18b1de8b Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/5.png differ diff --git a/docs/en/tools/desktop/ukui/figures/6.png b/docs/en/tools/desktop/ukui/figures/6.png new file mode 100644 index 0000000000000000000000000000000000000000..275c23872f2353f007371672714902babcc3db53 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/6.png differ diff --git a/docs/en/tools/desktop/ukui/figures/7.png b/docs/en/tools/desktop/ukui/figures/7.png new file mode 100644 index 0000000000000000000000000000000000000000..4d397959ac7f6d166ef5a3b7084bd5c3c93b475f Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/7.png differ diff --git a/docs/en/tools/desktop/ukui/figures/8.png b/docs/en/tools/desktop/ukui/figures/8.png new file mode 100644 index 0000000000000000000000000000000000000000..8ade274092d7b3e461c96d7909a9d89d3a944f09 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/8.png differ diff --git a/docs/en/tools/desktop/ukui/figures/9.png b/docs/en/tools/desktop/ukui/figures/9.png new file mode 100644 index 0000000000000000000000000000000000000000..f7b2215404929346f1a814b0b1d6d482559c08b5 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/9.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon1.png b/docs/en/tools/desktop/ukui/figures/icon1.png new file mode 100644 index 0000000000000000000000000000000000000000..9bac00355cf4aa57d32287fd4271404f6fd3fd4d Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon1.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon10-o.png b/docs/en/tools/desktop/ukui/figures/icon10-o.png new file mode 100644 index 0000000000000000000000000000000000000000..d6c56d1a64c588d86f8fe510c74e5a7c4cb810d4 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon10-o.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon11-o.png b/docs/en/tools/desktop/ukui/figures/icon11-o.png new file mode 100644 index 0000000000000000000000000000000000000000..47a1f2cb7f99b583768c7cbd7e05a57f302dbe8a Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon11-o.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon12-o.png b/docs/en/tools/desktop/ukui/figures/icon12-o.png new file mode 100644 index 0000000000000000000000000000000000000000..f1f0f59dd3879461a0b5bc0632693a4a4124def3 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon12-o.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon13-o.png b/docs/en/tools/desktop/ukui/figures/icon13-o.png new file mode 100644 index 0000000000000000000000000000000000000000..c05a981b29d8ad11c6682f796f79b4cafd0f088b Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon13-o.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon14-o.png b/docs/en/tools/desktop/ukui/figures/icon14-o.png new file mode 100644 index 0000000000000000000000000000000000000000..b21deee4d98593d93fb5f72158d2d78f3d3f1cb9 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon14-o.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon15-o.png b/docs/en/tools/desktop/ukui/figures/icon15-o.png new file mode 100644 index 0000000000000000000000000000000000000000..1827a20e9da4d28e35e8ab2eae739b2fec37b385 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon15-o.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon16.png b/docs/en/tools/desktop/ukui/figures/icon16.png new file mode 100644 index 0000000000000000000000000000000000000000..f271594dda9d3ad0f038c9d719dd68c3e82c59f1 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon16.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon17.png b/docs/en/tools/desktop/ukui/figures/icon17.png new file mode 100644 index 0000000000000000000000000000000000000000..dbe58b89347c857920bce25f067fbd11c308e502 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon17.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon18.png b/docs/en/tools/desktop/ukui/figures/icon18.png new file mode 100644 index 0000000000000000000000000000000000000000..1827a20e9da4d28e35e8ab2eae739b2fec37b385 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon18.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon19-o.png b/docs/en/tools/desktop/ukui/figures/icon19-o.png new file mode 100644 index 0000000000000000000000000000000000000000..47a1f2cb7f99b583768c7cbd7e05a57f302dbe8a Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon19-o.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon2.png b/docs/en/tools/desktop/ukui/figures/icon2.png new file mode 100644 index 0000000000000000000000000000000000000000..9101e4b386df065a87d422bc5a0b287528ea5ec7 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon2.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon26-o.png b/docs/en/tools/desktop/ukui/figures/icon26-o.png new file mode 100644 index 0000000000000000000000000000000000000000..2293a893caf6d89c3beb978598fe7f281e68e7d5 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon26-o.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon27-o.png b/docs/en/tools/desktop/ukui/figures/icon27-o.png new file mode 100644 index 0000000000000000000000000000000000000000..abbab8e40f7e3ca7c2a6f28ff78f08f15117828e Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon27-o.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon28-o.png b/docs/en/tools/desktop/ukui/figures/icon28-o.png new file mode 100644 index 0000000000000000000000000000000000000000..e40d45fc0a9d2af93280ea14e01512838bb3c3dc Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon28-o.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon3.png b/docs/en/tools/desktop/ukui/figures/icon3.png new file mode 100644 index 0000000000000000000000000000000000000000..930ee8909e89e3624c581f83d713af271cd96c75 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon3.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon30-o.png b/docs/en/tools/desktop/ukui/figures/icon30-o.png new file mode 100644 index 0000000000000000000000000000000000000000..e40d45fc0a9d2af93280ea14e01512838bb3c3dc Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon30-o.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon31-o.png b/docs/en/tools/desktop/ukui/figures/icon31-o.png new file mode 100644 index 0000000000000000000000000000000000000000..25959977f986f433ddf3d66935f8d2c2bc6ed86b Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon31-o.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon32.png b/docs/en/tools/desktop/ukui/figures/icon32.png new file mode 100644 index 0000000000000000000000000000000000000000..25959977f986f433ddf3d66935f8d2c2bc6ed86b Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon32.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon33.png b/docs/en/tools/desktop/ukui/figures/icon33.png new file mode 100644 index 0000000000000000000000000000000000000000..88ed145b25f6f025ad795ceb012500e0944cb54c Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon33.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon34.png b/docs/en/tools/desktop/ukui/figures/icon34.png new file mode 100644 index 0000000000000000000000000000000000000000..8247f52a3424c81b451ceb318f4a7979a5eddece Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon34.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon35.png b/docs/en/tools/desktop/ukui/figures/icon35.png new file mode 100644 index 0000000000000000000000000000000000000000..7c656e9030b94809a57c7e369921e6a585f3574c Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon35.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon36.png b/docs/en/tools/desktop/ukui/figures/icon36.png new file mode 100644 index 0000000000000000000000000000000000000000..7d29d173e914dfff48245d3d3a4d42575ce2d1db Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon36.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon37.png b/docs/en/tools/desktop/ukui/figures/icon37.png new file mode 100644 index 0000000000000000000000000000000000000000..58be4c621b6638115153e361801deb9ee06634d8 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon37.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon38.png b/docs/en/tools/desktop/ukui/figures/icon38.png new file mode 100644 index 0000000000000000000000000000000000000000..0c861ccb891f4fb5e533eb7f7151a8fce1571f17 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon38.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon39.png b/docs/en/tools/desktop/ukui/figures/icon39.png new file mode 100644 index 0000000000000000000000000000000000000000..b1ba1f347452d0cd1c06c6c51d2cdf5aea5e490b Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon39.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon4.png b/docs/en/tools/desktop/ukui/figures/icon4.png new file mode 100644 index 0000000000000000000000000000000000000000..548dc8b648edb73ff1dd8a0266e8479203e72ca0 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon4.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon40.png b/docs/en/tools/desktop/ukui/figures/icon40.png new file mode 100644 index 0000000000000000000000000000000000000000..9c29dd1e9a1bf22c36abf51cb18fa9e47b455fab Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon40.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon41.png b/docs/en/tools/desktop/ukui/figures/icon41.png new file mode 100644 index 0000000000000000000000000000000000000000..9e8aea527a2119433fffec5a8800ebfa4fa5062f Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon41.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon42-o.png b/docs/en/tools/desktop/ukui/figures/icon42-o.png new file mode 100644 index 0000000000000000000000000000000000000000..25959977f986f433ddf3d66935f8d2c2bc6ed86b Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon42-o.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon43-o.png b/docs/en/tools/desktop/ukui/figures/icon43-o.png new file mode 100644 index 0000000000000000000000000000000000000000..284bdd551baf25beb4143013402e77a1a4c60ccb Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon43-o.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon44-o.png b/docs/en/tools/desktop/ukui/figures/icon44-o.png new file mode 100644 index 0000000000000000000000000000000000000000..810f4d784ee140dbf562e67a0d3fd391272626a5 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon44-o.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon45-o.png b/docs/en/tools/desktop/ukui/figures/icon45-o.png new file mode 100644 index 0000000000000000000000000000000000000000..3e528ce2c98284f020ae4912a853f5864526396b Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon45-o.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon46-o.png b/docs/en/tools/desktop/ukui/figures/icon46-o.png new file mode 100644 index 0000000000000000000000000000000000000000..ec6a3ca0fe57016f3685981ed518493ceea1c855 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon46-o.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon47-o.png b/docs/en/tools/desktop/ukui/figures/icon47-o.png new file mode 100644 index 0000000000000000000000000000000000000000..6eeaba98d908775bd363a8ffcec27c3b6a214013 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon47-o.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon5.png b/docs/en/tools/desktop/ukui/figures/icon5.png new file mode 100644 index 0000000000000000000000000000000000000000..e4206b7b584bf0702c7cb2f03a3a41e20bfba844 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon5.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon6.png b/docs/en/tools/desktop/ukui/figures/icon6.png new file mode 100644 index 0000000000000000000000000000000000000000..88ced3587e9a42b145fe11393726f40aba9d1b2c Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon6.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon7.png b/docs/en/tools/desktop/ukui/figures/icon7.png new file mode 100644 index 0000000000000000000000000000000000000000..05fe8aa38c84ca0c0c99b0b005ddec2f2ba42f4a Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon7.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon8.png b/docs/en/tools/desktop/ukui/figures/icon8.png new file mode 100644 index 0000000000000000000000000000000000000000..01543c3e0f5e96a023b4e1f0859a03e3a0dafd56 Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon8.png differ diff --git a/docs/en/tools/desktop/ukui/figures/icon9.png b/docs/en/tools/desktop/ukui/figures/icon9.png new file mode 100644 index 0000000000000000000000000000000000000000..a07c9ab8e51decd9a3bca8c969d2ae95bd68512c Binary files /dev/null and b/docs/en/tools/desktop/ukui/figures/icon9.png differ diff --git a/docs/en/tools/desktop/ukui/ukui-installation.md b/docs/en/tools/desktop/ukui/ukui-installation.md new file mode 100644 index 0000000000000000000000000000000000000000..df992cb81e772eeecd010ad4bb5e9ef3040e9a4f --- /dev/null +++ b/docs/en/tools/desktop/ukui/ukui-installation.md @@ -0,0 +1,28 @@ +# UKUI Installation + +UKUI is a Linux desktop built by the KylinSoft software team over the years, primarily based on GTK and QT. Compared to other UI interfaces, UKUI is easy to use. The components of UKUI are small and low coupling, can run alone without relying on other suites. It can provide user a friendly and efficient experience. + +UKUI supports both x86_64 and aarch64 architectures. + +You are advised to create an administrator user before installing UKUI. + +1. Download openEuler ISO and update the software source. + + ```shell + sudo dnf update + ``` + +2. Install UKUI. + + ```shell + sudo dnf install ukui + ``` + +3. If you want to set the system to start with the graphical interface after confirming the installation, run the following command and reboot the system (`reboot`). + + ```shell + systemctl set-default graphical.target + ``` + +UKUI is constantly updated. Please check the latest installation method: +[https://gitee.com/openeuler/ukui](https://gitee.com/openeuler/ukui) diff --git a/docs/en/tools/desktop/ukui/ukui-user-guide.md b/docs/en/tools/desktop/ukui/ukui-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..b83ee3007186d46635695105c1c407887cd4c65f --- /dev/null +++ b/docs/en/tools/desktop/ukui/ukui-user-guide.md @@ -0,0 +1,380 @@ +# UKUI Desktop Environment + +## Overview + +Desktop Environment is the basis for the user's operation on the graphical interface, and provides multiple functions including taskbar, start menu, etc. The main interface is shown in figure below. + +![Fig. 1 Desktop main interface-big](./figures/1.png) + +## Desktop + +### Desktop Icons + +The system places three icons Computer, Recycle Bin and Personal by default, and double click the left mouse button to open the page. The functions are shown in table below. + +| Icon | Description | +| :------------ | :------------ | +| ![](./figures/icon1.png) | Computer: Show the drives and hardwares connected to the machine| +| ![](./figures/icon2.png) | Recycle Bin: Show documents that have been diverted| +| ![](./figures/icon3.png) | Personal: Show personal home directory| + +In addition, right-clicking "Computer" and selecting "Properties", it can show the current system version, kernel version, activation and other related information. + +![Fig. 2 "Computer" - "Properties"-big](./figures/2.png) + +### Right-click Menu + +Right-click on the desktop blank and a menu appears as shown in figure below, providing the users with some shortcut features. + +![Fig. 3 Right-click Menu](./figures/3.png) + +Some of the options are described in table below. + +| Option | Description | +| :------------ | :------------ | +| New | Create new folders, text documents, WPS files | +| View type | Four view types are available: small icon, medium icon, large icon, super large icon | +| Sort by | Four ways to arrange documents according to name, type of document, size and date of modification | + +## Taskbar + +### Basic Function + +Taskbar is located at the bottom and includes the Start Menu, Multi View Switch, File Browser, Firefox Web Browser, WPS, and Tray Menu. + +![Fig. 4 Taskbar](./figures/4.png) + +| Component | Description | +| :------------ | :------------ | +|![](./figures/icon4.png)| Start menu: Open the system menu to find applications and files | +|![](./figures/icon5.png)| Multi View Switch: Operate in multiple workspaces without interfering with each other | +|![](./figures/icon6.png)| File Browser: Browse and manage documents in the system | +|![](./figures/icon7.png)| Firefox Web Browser: Provide a convenient and safe way to access the Internet | +|![](./figures/icon8.png)| WPS: Realize the most common functions of office software such as text, forms, presentations, and more | +|Window Display Area| The blank part in the middle of the horizontal bar. Display running programs, opened documents, and allow users to close the windows, top the windows, etc | +|![](./figures/icon9.png)| Tray Menu: Include settings for voice, Kylin weather,input method, internet connection, notification center, date, and night mode | +|Show Desktop| The button is on the far right. Minimize all windows on the desktop and return to the desktop; Clicking again will restore the windows | + +#### Multi View Switch + +Click the icon "![](./figures/icon10-o.png)" on the taskbar to enter the interface shown in figure below, and select the operation area that users need to work on at the moment in multiple work areas. + +![Fig. 5 Multi View Switch-big](./figures/5.png) + +#### Preview Window + +Users move the mouse over the app icon in the taskbar, and then a small preview window will be shown if this app has already been opened. + +Hover over the specified window as shown below for hover state, the window will be slightly fuzzy glass effect (left), the rest of the window as default Status (right). + +![Fig. 6 Taskbar - Preview Window](./figures/6.png) + +Users can close the application by right-clicking on the app icon in the taskbar. + +![Fig. 7 Taskbar - Right-click Preview](./figures/7.png) + +#### Sidebar + +The sidebar is located at the right of the entire desktop. Click the icon "![](./figures/icon11-o.png)" in the taskbar tray menu to open the storage menu, and click the icon "![](./figures/icon12-o.png)" in Sidebar to pop up the sidebar as shown in figure below. + +The sidebar consists of two parts: Notification Center, Clipboard and Widget. + +![Fig. 8 Sidebar without message status-big](./figures/8.png) + +##### Notification Center + +Notification center will display a list of recent important and newest information. + +Select "Clear" in the upper right corner to clear the list of information; Select "Setting" in the upper right corner to go to the notification settings in the control center, and users can set which applications can show information and the quantity of information. + +![Fig. 9 Notification Center-big](./figures/9.png) + +Workspace at right side can be set to fold by applications. + +![Fig. 10 Fold messages by applications-big](./figures/10.png) + +Icon "![](./figures/icon13-o.png)" at the top right corner of the sidebar can store unimportant information. When the messages are more than 999+, it will be shown as the form of ![](./figures/icon14-o.png) which means limitless. + +![Fig. 11 Message Organizer](./figures/11.png) + +##### Clipboard + +Clipboard can save the contents those were recently selected to copy or cut, and users can operate them by using the icons in Table. + +![Fig. 12 Clipboard](./figures/12.png) + +Clicking "![](./figures/icon15-o.png)", users can edit the the contents of the clipboard. + +![Fig. 13 edit the content](./figures/13.png) + +| Icon | Description | Icon | Description | +| :------------------------ | :----------------- | :------------------------ | :--------------- | +| ![](./figures/icon16.png) | Copy the content | ![](./figures/icon18.png) | Edit the content | +| ![](./figures/icon17.png) | Delete the content | | | + +The second label of the clipboard is the small plug-in that contains alarm clock, notebook, user feedback. + +![Fig. 14 Plug-in](./figures/14.png) + +#### Tray Menu + +##### Storage Menu + +Click "![](./figures/icon19-o.png)" at the tray menu to open the storage menu. + +It contains Kylin Weather, Input Method, Bluetooth, USB, etc. + +![Fig. 15 Storage Menu](./figures/15.png) + +##### Input Method + +The taskbar input method defaults to Sogou input method. Use the shortcut key "Ctrl+Space" to switch it out, and the "Shift" key to switch between Chinese and English modes. + +![Fig. 16 Input Method](./figures/16.png) + +##### USB + +When the USB is inserted into the computer, it will be automatically read the data inside. + +Click "![](./figures/icon26-o.png)" to open the window as shown in figure below. + +When users need to umount the USB, please click the icon "![](./figures/icon27-o.png)". + +![Fig. 17 The status of USB](./figures/17.png) + +##### Power Supply + +Click the icon "![](./figures/icon28-o.png)": + +When no power supply is detected. + +![Fig. 18 No Power Supply](./figures/18.png) + +When power supply is detected. + +![Fig. 19 Have Power Supply](./figures/19.png) + +Users right-click the icon "![](./figures/icon30-o.png)" of power manager to open the power setting menu. + +It provides two setting options: adjust screen brightness, and set power and sleep. + +![Fig. 20 Power Manager](./figures/20.png) + +If the power manager pops up a"low battery" window, users can click to turn on the power save mode, and the power manager will set the machine to run in this mode immediately. + +![Fig. 21 Power Saving Mode](./figures/21.png) + +##### Network + +Users can choose wired or wireless network connections by clicking the icon "![](./figures/icon31-o.png)" of network manager. + +| Icon | Description | Icon | Description | +| :------------------------ | :----------------- | :------------------------ | :--------------------- | +| ![](./figures/icon32.png) | Connected | ![](./figures/icon37.png) | Unconnected | +| ![](./figures/icon33.png) | Connection limited | ![](./figures/icon38.png) | Locked | +| ![](./figures/icon34.png) | Connecting | ![](./figures/icon39.png) | Wifi connected | +| ![](./figures/icon35.png) | Wifi unconnected | ![](./figures/icon40.png) | Wificonnection limited | +| ![](./figures/icon36.png) | Wifi locked | ![](./figures/icon41.png) | Wifi connecting | + +![Fig. 22 Network Connection](./figures/22.png) + +- Wired Network + In the wired network connection interface, click on the wired network plan to expand. Details of the network. + + ![Fig. 23 Wired Network](./figures/23.png) + +- Wireless Network + Click the switch button in the upper right corner to turn on the wireless network connection, and select the WiFi from the list of available wireless networks. Enter the password to access the Internet. + + ![Fig. 24 Wireless Network](./figures/24.png) + +- Network Setting + Right-click the icon "![](./figures/icon42-o.png)" of network manager to pop up the setting menu. + + ![Fig. 25 Wired Network Setting](./figures/25.png) + + Click network setting to go to the setting window immediately. + + ![Fig. 26 Network Setting](./figures/26.png) + +##### Volume + +Click the icon "![](./figures/icon43-o.png)" to open the volume window, and there provides three modes. + +- Mini Mode + It only displays the volume of the speaker. + + ![Fig. 27 Mini Mode](./figures/27.png) + +- According to Equipment + It contains input equipment and output equipment. + + ![Fig. 28 According to Equipment List](./figures/28.png) + +- According to Application + It contains system volume and other applications' volume. + + ![Fig. 29 According to Application List](./figures/29.png) + +##### Calendar + +Click the date&time on the taskbar to open the calendar window. + +Users can view the day's information by filtering the year, month, day. The date will be displayed in large letters, with the time, the week, the festival,and the lunar calendar. Taboos can be seen by checking. + +![Fig. 30 Calendar-big](./figures/30.png) + +##### Night Mode + +Click the icon "![](./figures/icon44-o.png)" on the Taskbar and then the system changes to the night mode. + +#### Advanced Setting + +Right-click the Taskbar to open the menu. + +![Fig. 31 Right-Clicking Menu](./figures/31.png) + +Users can set the lauserst of taskbar according to "Taskbar Settings". + +## Window + +### Window Manager + +The functions provided as shown in Table. + +| Function | Description | +| :--------| :----------| +| Title Bar | Show the title name of current window | +| Minimize/Maximize/Close | The three icon buttons at the right of the title bar correspond to minimize, maximize and close respectively | +| Side Sliding | Users can scroll up and down to view the page by the slider at the right of the window | +| Stack | Allow overlap among windows | +| Drag and Drop | Long press the left mouse button at the title bar to move the window to any position | +| Resize | Move the mouse to the corner of the window and long press the left button to resize the window | + +### Window Switch + +There are three ways to switch windows: + +- Click the window title on the Taskbar. + +- Click the different window at the desktop. + +- Use shortcut keys **Alt + Tab**. + +## Start Menu + +### Basic Function + +Click the button to open the "Start Menu". + +It provides sliding bar. + +![Fig. 32 Start Menu](./figures/32.png) + +#### Category Menu at right side + +When the mouse is over the right side of the start menu, it will appear a pre-expanded cue bar. Clicking to expand, and then three categories are showing at the right side by default: "Common Software", "Alphabetical Category", and "Functional category". + +- All Software: List all software, recently used software will be displayed on the top of this page. + +- Alphabetical Category: List all software by first letter. + +- Functional category: List all software by their functions. + +Users can click the button at top right corner to view fullscreen menu mode. + +![Fig. 33 Fullscreen Menu-big](./figures/33.png) + +#### Function Button at right side + +It provides User Avatar, Computer, Control Center and Shutdown four options. + +##### User Avatar + +Click "![](./figures/icon45-o.png)" to view user's information. + +##### Computer + +Click "![](./figures/icon46-o.png)" to open personal home folder + +##### Control Center + +Click "![](./figures/icon47-o.png)" to go to the control center. + +##### Shutdown + +###### Lock Screen + +When users do not need to use the computer temporarily, the lock screen can be selected (without affecting the current running state of the system) to prevent misoperations. And input the password to re-enter the system. + + The system will automatically lock the screen after a period of idle time by default. + +![Fig. 34 Lock Screen-big](./figures/34.png) + +###### Switch Users & Log Out + +When users want to select another user to log in using the computer, users can select "Log out" or "Switch user". + +At this point, the system will close all running applications; Therefore, please save the current jobs before performing this action. + +###### Shutdown & Reboot + +There are two ways: + +1)"Start Menu" > "Power" > "Shutdown" + +It will pop up a dialog box, and users can choose shutdown or reboot as needed. + +![Fig. 35 Shutdown Dialog Box-big](./figures/35.png) + +2)"Start Menu" > right side menu of the "Shutdown" button > "Shutdown"/"Reboot" + +The system will shutdown or reboot immediately without popping up the dialog box. + +### Advanced Setting + +Right-clicking Start Menu, it provides lock screen, switch user, log out, reboot, and shutdown five shortcut options. + +### Applications + +Users can search apps in the search box by key words. As shown in figure below, the result will show up automatically with the input. + +![Fig. 36 Search Apps](./figures/36.png) + +Right-clicking one app in the Start Menu, the right-click menu popping up. + +![Fig. 37 Right-click Menu](./figures/37.png) + +The options are described in table below. + +| Option | Description | +| :------| :--------| +| Attach to "All Software" |Add the selected software to the top of the list of All Software| +| Attach to Taskbar |Generate icon for the application on the Taskbar| +| Add to Desktop Shortcut |Generate shortcut icon for the application on the desktop| +| Uninstall |Remove the application| + +## FAQ + +### Failed to Login to the System after Locking the Screen + +- Switch to character terminal by **Ctrl + Alt + F2**. + +- Input the user-name and passwd to login to the system. + +- Do "sudo rm -rf ~/.Xauthority". + +- Switch to graphical interface by **Ctrl + Alt + F1**, and input the passwd. + +## Appendix + +### Shortcut Key + +| Shortcut Key | Function | +| :------------------ | :------------------ | +| F5 | Refresh the desktop | +| F1 | Open the user-guide | +| Alt + Tab | Switch the window | +| win | Open the Start Menu | +| Ctrl + Alt + L | Lock Screen | +| Ctrl + Alt + Delete | Log out | diff --git a/docs/en/tools/desktop/xfce/_toc.yaml b/docs/en/tools/desktop/xfce/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..eba922efed433dc412be05afd850add234afeb00 --- /dev/null +++ b/docs/en/tools/desktop/xfce/_toc.yaml @@ -0,0 +1,10 @@ +label: Xfce User Guide +isManual: true +description: Install and use Xfce +sections: + - label: Xfce Installation + href: ./xfce-installation.md + - label: Xfce User Guide + href: ./xfce-user-guide.md + - label: Xfce Common Issues and Solutions + href: ./xfce-common-issues-and-solutions.md diff --git a/docs/en/tools/desktop/xfce/figures/xfce-1.png b/docs/en/tools/desktop/xfce/figures/xfce-1.png new file mode 100644 index 0000000000000000000000000000000000000000..0e478b9f10ddf3210d5f5fada2e45329e2d1d028 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-1.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-2.png b/docs/en/tools/desktop/xfce/figures/xfce-2.png new file mode 100644 index 0000000000000000000000000000000000000000..33a946d988d499a1e98cb43968b72119bd48d7a5 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-2.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-3.png b/docs/en/tools/desktop/xfce/figures/xfce-3.png new file mode 100644 index 0000000000000000000000000000000000000000..020356f0c981fac2aafe33c8e997efbf01af9253 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-3.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-4.png b/docs/en/tools/desktop/xfce/figures/xfce-4.png new file mode 100644 index 0000000000000000000000000000000000000000..21369e366322955023b427e7a2ae63fd29b387e5 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-4.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-5.png b/docs/en/tools/desktop/xfce/figures/xfce-5.png new file mode 100644 index 0000000000000000000000000000000000000000..1f7807877f775fe6aa32652a29ef833e48e1a6ee Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-5.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-6.png b/docs/en/tools/desktop/xfce/figures/xfce-6.png new file mode 100644 index 0000000000000000000000000000000000000000..e5376fcfd1737234a885d4d95649cd996005cf0c Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-6.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-7.png b/docs/en/tools/desktop/xfce/figures/xfce-7.png new file mode 100644 index 0000000000000000000000000000000000000000..b7a94df356b7b9f7dca3d305d066ec854406aaab Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-7.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-71.png b/docs/en/tools/desktop/xfce/figures/xfce-71.png new file mode 100644 index 0000000000000000000000000000000000000000..11d1618c907d4bb18de1eb68e42e9b98d92d91c3 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-71.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-8.png b/docs/en/tools/desktop/xfce/figures/xfce-8.png new file mode 100644 index 0000000000000000000000000000000000000000..f6f97d9a173105cb6a72e4b8c48deab25ecac898 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-8.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-81.png b/docs/en/tools/desktop/xfce/figures/xfce-81.png new file mode 100644 index 0000000000000000000000000000000000000000..b97c9a81c2a07efe361e6dc6ee8bed5db445ecfa Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-81.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-811.png b/docs/en/tools/desktop/xfce/figures/xfce-811.png new file mode 100644 index 0000000000000000000000000000000000000000..58233638eca203d917081d6a9ac5003474cbf60b Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-811.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-812.png b/docs/en/tools/desktop/xfce/figures/xfce-812.png new file mode 100644 index 0000000000000000000000000000000000000000..0fc975f75da95dce8a3e5a098d024578335c9426 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-812.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-813.png b/docs/en/tools/desktop/xfce/figures/xfce-813.png new file mode 100644 index 0000000000000000000000000000000000000000..4d399468c74355cbaa765380720cb9561e95f834 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-813.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-814.png b/docs/en/tools/desktop/xfce/figures/xfce-814.png new file mode 100644 index 0000000000000000000000000000000000000000..c09fd6524a20ba04e0fca30307d35fa05e79c1f4 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-814.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-82.png b/docs/en/tools/desktop/xfce/figures/xfce-82.png new file mode 100644 index 0000000000000000000000000000000000000000..170deb5fb43f4e924d5ba4eba94a02c341d31515 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-82.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-821.png b/docs/en/tools/desktop/xfce/figures/xfce-821.png new file mode 100644 index 0000000000000000000000000000000000000000..c5c1f3567dccda3d0d49ae445612d5b9ba27e09a Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-821.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-83.png b/docs/en/tools/desktop/xfce/figures/xfce-83.png new file mode 100644 index 0000000000000000000000000000000000000000..95e4844c0ece09819d3e9f1e8457bbf371b1282e Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-83.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-831.png b/docs/en/tools/desktop/xfce/figures/xfce-831.png new file mode 100644 index 0000000000000000000000000000000000000000..6456dd02f0281a5ec8d752ba5b95be581bcbfa09 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-831.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-832.png b/docs/en/tools/desktop/xfce/figures/xfce-832.png new file mode 100644 index 0000000000000000000000000000000000000000..2932aaacf71fa53f1d0c10340df3aebcc016e991 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-832.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-84.png b/docs/en/tools/desktop/xfce/figures/xfce-84.png new file mode 100644 index 0000000000000000000000000000000000000000..e0435c2edf9f68d193cff036215f32c259d378f0 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-84.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-841.png b/docs/en/tools/desktop/xfce/figures/xfce-841.png new file mode 100644 index 0000000000000000000000000000000000000000..c2c06346d4a296bfbe7836139cd943baa1ce6ea5 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-841.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-842.png b/docs/en/tools/desktop/xfce/figures/xfce-842.png new file mode 100644 index 0000000000000000000000000000000000000000..101bf6923e3780617d33dde04b92232ca7f87b42 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-842.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-85.png b/docs/en/tools/desktop/xfce/figures/xfce-85.png new file mode 100644 index 0000000000000000000000000000000000000000..21b39638fe4c83e0da5cdc69ecad9b7a22718a55 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-85.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-851.png b/docs/en/tools/desktop/xfce/figures/xfce-851.png new file mode 100644 index 0000000000000000000000000000000000000000..893064ca10399a683afbcb3752266d93b0a79a51 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-851.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-86.png b/docs/en/tools/desktop/xfce/figures/xfce-86.png new file mode 100644 index 0000000000000000000000000000000000000000..35e8a99e31e4a49eb64b24cfbab825111e40f709 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-86.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-861.png b/docs/en/tools/desktop/xfce/figures/xfce-861.png new file mode 100644 index 0000000000000000000000000000000000000000..affc46c874991a3b289e15072e06ba6566c099b1 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-861.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-87.png b/docs/en/tools/desktop/xfce/figures/xfce-87.png new file mode 100644 index 0000000000000000000000000000000000000000..47524c21d57c887c3398ea53a675f89e9f92113f Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-87.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-9.png b/docs/en/tools/desktop/xfce/figures/xfce-9.png new file mode 100644 index 0000000000000000000000000000000000000000..5586c4f62cc161665b91a56ad23b2320901901c0 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-9.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-91.png b/docs/en/tools/desktop/xfce/figures/xfce-91.png new file mode 100644 index 0000000000000000000000000000000000000000..ee69879bb4ad66405b045af5e3965e275fe8eabf Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-91.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-911.png b/docs/en/tools/desktop/xfce/figures/xfce-911.png new file mode 100644 index 0000000000000000000000000000000000000000..b49416558e9ab844fda2026b76e2e900ac106842 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-911.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-92.png b/docs/en/tools/desktop/xfce/figures/xfce-92.png new file mode 100644 index 0000000000000000000000000000000000000000..78dd6313c603aad9ebd37fe68e06f98b2a3b331e Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-92.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-921.png b/docs/en/tools/desktop/xfce/figures/xfce-921.png new file mode 100644 index 0000000000000000000000000000000000000000..5eb6f40df9ca73e11b9b9fa5079496ac0c36857b Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-921.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-93.png b/docs/en/tools/desktop/xfce/figures/xfce-93.png new file mode 100644 index 0000000000000000000000000000000000000000..06ac80c152fefbe1ad2ba1c989f6acfbbaf1a992 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-93.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-931.png b/docs/en/tools/desktop/xfce/figures/xfce-931.png new file mode 100644 index 0000000000000000000000000000000000000000..a156e5cf14ae154b93e845ff1bd5bc6ba12c9beb Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-931.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-94.png b/docs/en/tools/desktop/xfce/figures/xfce-94.png new file mode 100644 index 0000000000000000000000000000000000000000..f48064ff5902c4ea740ccba9a1640cbca27b5b72 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-94.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-941.png b/docs/en/tools/desktop/xfce/figures/xfce-941.png new file mode 100644 index 0000000000000000000000000000000000000000..f7904da12dc807836acfb9d6f24b8d9b976a2fdc Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-941.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-95.png b/docs/en/tools/desktop/xfce/figures/xfce-95.png new file mode 100644 index 0000000000000000000000000000000000000000..bda965b15a859e4cccf4b80f62875f79eb3470fd Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-95.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-951.png b/docs/en/tools/desktop/xfce/figures/xfce-951.png new file mode 100644 index 0000000000000000000000000000000000000000..6521a28275d2b63c12b47604c7afc926f7938697 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-951.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-96.png b/docs/en/tools/desktop/xfce/figures/xfce-96.png new file mode 100644 index 0000000000000000000000000000000000000000..29ce24923477065b98cacf603f185113e9959069 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-96.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-961.png b/docs/en/tools/desktop/xfce/figures/xfce-961.png new file mode 100644 index 0000000000000000000000000000000000000000..874fa200f4e63b690261d7827f3c73cf70861b32 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-961.png differ diff --git a/docs/en/tools/desktop/xfce/figures/xfce-962.png b/docs/en/tools/desktop/xfce/figures/xfce-962.png new file mode 100644 index 0000000000000000000000000000000000000000..bb84e35e43e992bc68b053a0da760bd5aa8b0270 Binary files /dev/null and b/docs/en/tools/desktop/xfce/figures/xfce-962.png differ diff --git a/docs/en/tools/desktop/xfce/xfce-common-issues-and-solutions.md b/docs/en/tools/desktop/xfce/xfce-common-issues-and-solutions.md new file mode 100644 index 0000000000000000000000000000000000000000..21b68776fb1a55872cd96fbbce896f578960d648 --- /dev/null +++ b/docs/en/tools/desktop/xfce/xfce-common-issues-and-solutions.md @@ -0,0 +1,7 @@ +# Common Issues and Solutions + +## Issue 1: Why Is the Background Color of the LightDM Login Page Black + +The login page is black because `background` is not set in the default configuration file **/etc/lightdm/lightdm-gtk-greeter.conf** of lightdm-gtk. + +Set `background=/usr/share/backgrounds/xfce/xfce-blue.jpg` in the `greeter` section at the end of the configuration file, and then run the `systemctl restart lightdm` command. diff --git a/docs/en/tools/desktop/xfce/xfce-installation.md b/docs/en/tools/desktop/xfce/xfce-installation.md new file mode 100644 index 0000000000000000000000000000000000000000..80941b2a8d5e9702edcdc6d7ee4c5178695c97a0 --- /dev/null +++ b/docs/en/tools/desktop/xfce/xfce-installation.md @@ -0,0 +1,70 @@ +# Xfce Installation + +Xfce is a lightweight Linux desktop. In the current version, all components have been updated from GTK2 to GTK3 and from D-Dbus Glib to GDBus. Most components support GObject Introspection (GI), which is used to generate and parse the API meta information of the C program library, so that the dynamic language (or managed language) can be bound to the program library based on C + GObject. In the current version, user experience is optimized, new features are added, and a large number of bugs are fixed. Xfce occupies fewer memory and CPU resources than other UIs (GNOME and KDE), providing smoother and more efficient user experience. + +Xfce supports the x86\_64 and AArch64 architectures. + +You are advised to create an administrator during the installation. + +1. [Download](https://openeuler.org/en/download/)the openEuler ISO image and install the system. Run the following command to update the software source. You are advised to configure the Everything source and the EPOL source. This document describes how to install Xfce in the minimum installation scenario. + + ```shell + sudo dnf update + ``` + +2. Run the following command to install the font library: + + ```shell + sudo dnf install dejavu-fonts liberation-fonts gnu-*-fonts google-*-fonts + ``` + +3. Run the following command to install Xorg: + + ```shell + sudo dnf install xorg-* + ``` + +4. Run the following command to install Xfce: + + ```shell + sudo dnf install xfwm4 xfdesktop xfce4-* xfce4-*-plugin *fonts + ``` + +5. Run the following command to install the login manager: + + ```shell + sudo dnf install lightdm lightdm-gtk + ``` + +6. Run the following command to start Xfce using the login manager: + + ````shell + sudo systemctl start lightdm + ```` + + After the login manager is started, choose **Xfce Session** in the upper right corner and enter the user name and password to log in. + +7. Run the following command to set the GUI to start upon system boot: + + ```shell + sudo systemctl enable lightdm + sudo systemctl set-default graphical.target + ``` + + If GDM is installed by default, you are advised to disable GDM. + + ```shell + systemctl disable gdm + ``` + +8. Restart the server. + + ```shell + sudo reboot + ``` + +9. FAQs + - **Why Is the Background Color of the LightDM Login Page Black?** + + The login page is black because `background` is not set in the default configuration file **/etc/lightdm/lightdm-gtk-greeter.conf** of lightdm-gtk. + Set `background=/usr/share/backgrounds/xfce/xfce-blue.jpg` in the `greeter` section at the end of the configuration file, and then run the `systemctl restart lightdm` command. diff --git a/docs/en/tools/desktop/xfce/xfce-user-guide.md b/docs/en/tools/desktop/xfce/xfce-user-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..c74a522f43d0ba2583bcf128fb4726051a02ab39 --- /dev/null +++ b/docs/en/tools/desktop/xfce/xfce-user-guide.md @@ -0,0 +1,238 @@ +# Xfce User Guide + +## 1. Overview + +Xfce is a lightweight desktop environment running on Unix-like operating systems. Xfce provides multiple functional components, including all applications. This document describes how to use Xfce. + +The following figure shows the WebUI. + +![Figure 1 Main screen of the desktop](./figures/xfce-1.png) + +## 2. Desktop + +### 2.1 Desktop Icons + +By default, icons such as the file system, main folder, and mount directory are placed. You can double-click the icons to open the page. + +![Figure 2 Default desktop icons](./figures/xfce-2.png) + +### 2.2 Shortcut Menu + +Right-click in the blank area on the desktop. The shortcut menus are displayed, as shown in the following figure. + +![Figure 3 Shortcut menu](./figures/xfce-3.png) + +The following table describes some options. + +| Parameter| Description| +|:----------|:----------| +| Open in New Window| Open the **Desktop** directory of the login user.| +| Create Launcher| Create a launcher.| +| Create URL Link| Create a URL link.| +| Create Folder| Create a folder.| +| Create Document| Create a text file.| +| Open Terminal Here| Create a terminal.| +| Arrange Desktop Icons| Automatically arrange desktop icons.| +| Desktop Settings| Set the background, menus, and icons.| +| Properties| Set desktop properties, such as the general, logo, and permission.| +| Applications| Applications| + +## 3. Taskbar + +### 3.1 Basic Functions + +The taskbar is located at the top, including application, window display area, multi-view switch, and tray menus. + +![Figure 4 Taskbar](./figures/xfce-4.png) + +| Component| Description| +|:----------|:----------| +| Applications| Display all applications and settings, and allow you to search for applications and settings.| +| Window display area| The blank area in the middle of the horizontal bar, which displays running programs or opened documents. You can minimize, maximize, close, or pin the window to the top.| +| Switching views| Perform operations in multiple workspaces without interfering with each other.| +| Tray| Set the network connection, sound, power, notification center, calendar, and login user actions.| + +#### 3.1.1 Applications + +![Figure 5 All applications](./figures/xfce-5.png) + +#### 3.1.2 Window Display Area + +![Figure 6 Window display area](./figures/xfce-6.png) + +#### 3.1.3 Multi-View Switching + +Click ![](./figures/xfce-7.png) in the taskbar to enter the corresponding work area. + +For example, you can use the mouse to switch among multiple workspaces to select the operation area that you want to work in. + +![Figure 7 Switching among multiple views](./figures/xfce-71.png) + +#### 3.1.4 Tray + +![Figure 8 Tray menu](./figures/xfce-8.png) + +##### 3.1.4.1 Network + +You can click ![](./figures/xfce-81.png) on the taskbar and select a network connection mode as required. + +![Figure 9 Network connection page](./figures/xfce-811.png) + +Network settings dialog box + +Right-click the network icon ![](./figures/xfce-81.png) on the taskbar. The network setting menu is displayed. + +![Figure 10 Network settings](./figures/xfce-812.png) + +Click **Edit Connections**. The network setting dialog box is displayed. + +![Figure 11 Network setting dialog box](./figures/xfce-813.png) + +Double-click the specified network connection, for example, **enp1s0**. The page for setting the connection is displayed. + +![Figure 12 Setting the wired network](./figures/xfce-814.png) + +##### 3.1.4.2 Volume + +Click the volume icon ![](./figures/xfce-82.png) on the taskbar to open the sound page. + +![Figure 13 Volume setting window](./figures/xfce-821.png) + +##### 3.1.4.3 Power supply + +Click ![](./figures/xfce-83.png) on the taskbar. + +![Figure 14 Power supply devices](./figures/xfce-831.png) + +You can click **Power Manager Settings** to configure the display and nodes. + +![Figure 15 Setting the power manager](./figures/xfce-832.png) + +##### 3.1.4.4 Notification Center + +Click ![](./figures/xfce-84.png) on the taskbar. + +![Figure 16 Notification center](./figures/xfce-841.png) + +You can disable the notification function by selecting **Do not disturb**. + +The notification center displays the latest important information list. You can click **Clear log** to clear the information list. + +You can click **Notification settings** to go to the notification setting page of the control panel and set the applications to be displayed and the number of messages to be displayed. + +![Figure 17 Notification center](./figures/xfce-842.png) + +##### 3.1.4.5 Calendar + +You can click the date and time on the taskbar to display the calendar window and view the calendar, month calendar, and annual calendar. + +You can choose a year, a month and a day to view the information of a specific day. + +![Figure 18 Calendar-big](./figures/xfce-85.png) + +Right-click the time and date on the taskbar and click **Properties** to set the time. + +![Figure 19 Date setting](./figures/xfce-851.png) + +##### 3.1.4.6 Advanced Settings + +Right-click the taskbar and choose **Panel** from the shortcut menu. + +![Figure 20 Shortcut menu on the taskbar](./figures/xfce-86.png) + +You can set the layout of the taskbar and add or delete items. + +![Figure 21 Shortcut menu on the taskbar](./figures/xfce-861.png) + +##### 3.1.4.7 Login User Actions + +Click the login user on the task bar to view related actions. + +![Figure 22 Actions of a login user](./figures/xfce-87.png) + +###### 3.1.4.7.1 Lock Screen + +If you use the computer currently, you can lock the screen (which does not affect the current running status of the system) to prevent misoperations. After locking the screen, you can enter the password to log in to the system again. + +By default, the system automatically locks the screen after a period of idle time. + +###### 3.1.4.7.2 Switch User + +If you want to log in to the computer as another user, choose **Switch User**. + +Then, the system closes all running applications. Therefore, before performing this operation, save the current work. + +###### 3.1.4.7.3 Suspend + +For the sake of environmental protection and energy saving, you can select **Suspend****.** + +After that, the related data is read into the memory. Do not switch the power supply. + +###### 3.1.4.7.3 Shut Down + +You can choose **Shut Down** to shut down a computer. + +Before performing this operation, save the current work. + +###### 3.1.4.7.3 Log Out + +To log out of the GUI, click **Log Out**. + +Then, the system closes all running applications. Therefore, before performing this operation, save the current work. + +## 4. Shortcut Operation Bar + +### 4.1 Basic Functions + +The shortcut operation bar is located at the bottom, including the icons for displaying all the desktops, terminals, file managers, web browsers, application search, and user home directories. + +![Figure 23 Shortcut operation bar](./figures/xfce-9.png) + +| Component| Description | +|:----------|:----------| +| Show Desktop| Hide all windows and show the desktop. Click again to restore the window.| +| Terminal| Open a terminal.| +| File Manager| Open a file manager.| +| Web Browser| Open a web browser.| +| Application Finder| Open the application search window.| +| User Home Directory| Open the home directory of the login user.| + +#### 4.1.1 Show Desktop + +Click ![](./figures/xfce-91.png) on the shortcut operation bar to display the desktop. + +![Figure 24 Showing the desktop](./figures/xfce-911.png) + +#### 4.1.2 Terminal + +Click ![](./figures/xfce-92.png) on the shortcut operation bar to open a terminal. + +![Figure 25 Terminal-big](./figures/xfce-921.png) + +#### 4.1.3 File Manager + +You can click the ![](./figures/xfce-93.png) icon on the shortcut operation bar to open a file manager. + +![Figure 26 File manager](./figures/xfce-931.png) + +#### 4.1.4 Web Browser + +You can click the ![](./figures/xfce-94.png) icon on the shortcut operation bar to open a web browser. + +![Figure 27 Web browser](./figures/xfce-941.png) + +#### 4.1.5 Application Finder + +You can click the ![](./figures/xfce-95.png) icon on the shortcut operation bar to open an application program search interface. + +![Figure 28 Searching for an application](./figures/xfce-951.png) + +#### 4.1.6 User Home Directory + +Click ![](./figures/xfce-96.png) on the shortcut operation bar and click **Open File**. The user home directory page is displayed. + +![Figure 29 User home directory](./figures/xfce-961.png) + +Click the ![](./figures/xfce-96.png) icon on the shortcut operation bar, and then click **Open in Terminal** to open a terminal. The current directory is the home directory of the user. + +![Figure 30 User home directory](./figures/xfce-962.png) diff --git a/docs/en/tools/devops/_toc.yaml b/docs/en/tools/devops/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..a7b6721a2acd9d377b3d301159abfe5a39bf3290 --- /dev/null +++ b/docs/en/tools/devops/_toc.yaml @@ -0,0 +1,8 @@ +label: Community Services +sections: + - label: Source Code Management + sections: + - href: ./code_manage/patch_tracking/_toc.yaml + - label: Package Management + sections: + - href: ./package_manage/pkgship/_toc.yaml diff --git a/docs/en/tools/devops/code_manage/patch_tracking/_toc.yaml b/docs/en/tools/devops/code_manage/patch_tracking/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..7ce870ea4f97feea3192feff34f6c12dd4c25b2b --- /dev/null +++ b/docs/en/tools/devops/code_manage/patch_tracking/_toc.yaml @@ -0,0 +1,8 @@ +label: patch-tracking +isManual: true +description: Package patch management +sections: + - label: patch-tracking + href: ./patch-tracking.md + - label: Common Issues and Solutions + href: ./common-issues-and-solutions.md diff --git a/docs/en/tools/devops/code_manage/patch_tracking/common-issues-and-solutions.md b/docs/en/tools/devops/code_manage/patch_tracking/common-issues-and-solutions.md new file mode 100644 index 0000000000000000000000000000000000000000..5c20243357b920b4befa1343450c21687c7c39d0 --- /dev/null +++ b/docs/en/tools/devops/code_manage/patch_tracking/common-issues-and-solutions.md @@ -0,0 +1,15 @@ +# Common Issues and Solutions + +## Issue 1: Connection Refused upon Access to api.github.com + +### Symptom + +During the operation of patch-tracking, the following error message may occur: + +```text +Sep 21 22:00:10 localhost.localdomain patch-tracking[36358]: 2020-09-21 22:00:10,812 - patch_tracking.util.github_api - WARNING - HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /user (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) +``` + +### Cause Analysis + +The preceding problem is caused by the unstable network access between patch-tracking and GitHub API. Ensure that patch-tracking is operating in a stable network environment (for example, [Huawei Cloud Elastic Cloud Server](https://www.huaweicloud.com/intl/en-us/product/ecs.html)). diff --git a/docs/en/tools/devops/code_manage/patch_tracking/images/Maintainer.jpg b/docs/en/tools/devops/code_manage/patch_tracking/images/Maintainer.jpg new file mode 100644 index 0000000000000000000000000000000000000000..da0d5f1b5d928eca3a0d63795f59c55331136065 Binary files /dev/null and b/docs/en/tools/devops/code_manage/patch_tracking/images/Maintainer.jpg differ diff --git a/docs/en/tools/devops/code_manage/patch_tracking/images/PatchTracking.jpg b/docs/en/tools/devops/code_manage/patch_tracking/images/PatchTracking.jpg new file mode 100644 index 0000000000000000000000000000000000000000..e12afd6227c18c333f289b9aa71abf608d8058a0 Binary files /dev/null and b/docs/en/tools/devops/code_manage/patch_tracking/images/PatchTracking.jpg differ diff --git a/docs/en/tools/devops/code_manage/patch_tracking/patch-tracking.md b/docs/en/tools/devops/code_manage/patch_tracking/patch-tracking.md new file mode 100644 index 0000000000000000000000000000000000000000..599c3c5f046f5c97412f9ded7be0025e4a9de02a --- /dev/null +++ b/docs/en/tools/devops/code_manage/patch_tracking/patch-tracking.md @@ -0,0 +1,300 @@ +# patch-tracking + +## Overview + +During the development of the openEuler release, the latest code of each software package in the upstream community needs to be updated in a timely manner to fix function bugs and security issues, preventing the openEuler release from defects and vulnerabilities. + +This tool manages the patches for software packages, proactively monitors the patches submitted by the upstream community, automatically generates patches, submits issues to the corresponding Maintainer, and verifies basic patch functions to reduce the verification workload and help the Maintainer make decisions quickly. + +## Architecture + +### C/S Architecture + +patch-tracking uses the C/S architecture. + +patch-tracking is located in the server. It executes patch tracking tasks, including maintaining tracking items, identifying branch code changes in the upstream repository and generating patch files, and submitting issues and PRs to Gitee. In addition, patch-tracking provides RESTful APIs for adding, deleting, modifying, and querying tracking items. + +patch-tracking-cli is a command line tool located in the client. It invokes the RESTful APIs of patch-tracking to add, delete, modify, and query tracking items. + +### Core Procedure + +a. Patch tracking service procedure + +The procedure for handling the submitted patch is as follows: + +1. Add the tracking item using the command line tool. +2. Automatically obtain patch files from the upstream repository (for example, GitHub) that is configured for the tracking item. +3. Create a temporary branch and submit the obtained patch file to the temporary branch. +4. Automatically submit an issue to the corresponding repository and generate the PR associated with the issue. + +![PatchTracking](./images/PatchTracking.jpg) + +b. Procedure for the Maintainer to handle the submitted patch + +The procedure for handling the submitted patch is as follows: + +1. The Maintainer analyzes the PR. +2. Execute the continuous integration (CI). After the CI is successfully executed, determine whether to merge the PR. + +![Maintainer](./images/Maintainer.jpg) + +### Data structure + +- Tracking table + +| No. | Name | Description | Type | Key | Is Null Allowed | +| --- | --------------- | ----------------------------------------------------------------------- | ------- | ------- | --------------- | +| 1 | id | Sequence number of the tracking item of the self-added patch | int | - | No | +| 2 | version_control | Version control system type of the upstream SCM | String | - | No | +| 3 | scm_repo | Upstream SCM repository address | String | - | No | +| 4 | scm_branch | Upstream SCM tracking branch | String | - | No | +| 5 | scm_commit | Latest Commit ID processed by the upstream code | String | - | Yes | +| 6 | repo | Address of the Gitee repository where the package source code is stored | String | Primary | No | +| 7 | branch | Branch of the Gitee repository where the package source code is stored | String | Primary | No | +| 8 | enabled | Indicating whether to start tracking | Boolean | - | No | + +- Issue table + +| No. | Name | Description | Type | Key | Is Null Allowed | +| --- | ------ | ----------------------------------------------------------------------- | ------ | ------- | --------------- | +| 1 | issue | Issue No. | String | Primary | No | +| 2 | repo | Address of the Gitee repository where the package source code is stored | String | - | No | +| 3 | branch | Branch of the Gitee repository where the package source code is stored | String | - | No | + +## Tool Deployment + +### Downloading Software + +The repo source is officially released at . + +The RPM package can be obtained from . Choose the correct version, **everything**, correct architecture, **Packages**, then select the RPM package to download. + +### Installing the Tool + +Method 1: Install patch-tracking from the repo source. + +1. Use DNF to mount the repo source (The repo source of 21.03 or later is required. For details, see the [Application Development Guide](https://openeuler.org/zh/docs/21.03/docs/ApplicationDev/%E5%BC%80%E5%8F%91%E7%8E%AF%E5%A2%83%E5%87%86%E5%A4%87.html)). Run the following command to download and install patch-tracking and its dependencies. + +2. Run the following command to install `patch-tracking`: + + ```shell + dnf install patch-tracking + ``` + +Method 2: Install patch-tracking using the RPM package. + +1. Install the required dependencies. + + ```shell + dnf install python3-uWSGI python3-flask python3-Flask-SQLAlchemy python3-Flask-APScheduler python3-Flask-HTTPAuth python3-requests python3-pandas + ``` + +2. `patch-tracking-1.0.0-1.oe1.noarch.rpm` is used as an example. Run the following command to install patch-tracking. + + ```shell + rpm -ivh patch-tracking-1.0.0-1.oe1.noarch.rpm + ``` + +### Generating a Certificate + +Run the following command to generate a certificate: + +```shell +openssl req -x509 -days 3650 -subj "/CN=self-signed" \ +-nodes -newkey rsa:4096 -keyout self-signed.key -out self-signed.crt +``` + +Copy the generated `self-signed.key` and `self-signed.crt` files to the **/etc/patch-tracking** directory. + +### Configuring Parameters + +Configure the corresponding parameters in the configuration file. The path of the configuration file is `/etc/patch-tracking/settings.conf`. + +1. Configure the service listening address. + + ```text + LISTEN = "127.0.0.1:5001" + ``` + +2. GitHub Token is used to access the repository information hosted in the upstream open source software repository of GitHub. For details about how to create a GitHub token, see [Creating a personal access token](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token). + + ```text + GITHUB_ACCESS_TOKEN = "" + ``` + +3. For a repository that is hosted on Gitee and needs to be tracked, configure a Gitee Token with the repository permission to submit patch files, issues, and PRs. + + ```text + GITEE_ACCESS_TOKEN = "" + ``` + +4. Scan the database as scheduled to detect whether new or modified tracking items exist and obtain upstream patches for the detected tracking items. Set the interval of scanning and the unit is second. + + ```text + SCAN_DB_INTERVAL = 3600 + ``` + +5. When the command line tool is running, you need to enter the user name and password hash value for the authentication for the POST interface. + + ```text + USER = "admin" + PASSWORD = "" + ``` + + > The default value of `USER` is `admin`. + + Run the following command to obtain the password hash value. **Test@123** is the configured password. + + ```shell + generate_password Test@123 + ``` + + > The password hash value must meet the following complexity requirements: + > + > - The length is more than or equal to 6 bytes. + > - The password must contain uppercase letters, lowercase letters, digits, and special characters (**~!@#%\^\*-\_=+**). + + Add the password hash value to the quotation marks of `PASSWORD = ""`. + +### Starting the Patch Tracking Service + +You can use either of the following methods to start the service: + +- Using systemd. + + ```shell + systemctl start patch-tracking + ``` + +- Running the executable program. + + ```shell + /usr/bin/patch-tracking + ``` + +## Tool Usage + +### Adding a Tracking Item + +You can associate the software repository and branch to be tracked with the corresponding upstream open source software repository and branch in any of the following ways: + +- Using CLI + + Parameter description: + + > `--user`: User name to be authenticated for the POST interface. It is the same as the USER parameter in the **settings.conf** file. + > `--password`: Password to be authenticated for the POST interface. It is the password string corresponding to the PASSWORD hash value in the **settings.conf** file. + > `--server`: URL for starting the patch tracking service, for example, 127.0.0.1:5001. + > `--version\_control`: Control tool of the upstream repository version. Only GitHub is supported. + > `--repo`: Name of the repository to be tracked, in the format of organization/repository. + > + > `--branch`: Branch name of the repository to be tracked. + > `--scm\_repo`: Name of the upstream repository to be tracked, in the GitHub format of organization/repository. + > `--scm\_branch`: Branch of the upstream repository to be tracked. + > `--scm_commit`: Commit from which the tracking starts. By default, the tracking starts from the latest commit. + > `--enabled`: Indicates whether to automatically track the repository. + + For example: + + ```shell + patch-tracking-cli add --server 127.0.0.1:5001 --user admin --password Test@123 --version_control github --repo testPatchTrack/testPatch1 --branch master --scm_repo BJMX/testPatch01 --scm_branch test --enabled true + ``` + +- Using a Specified File + + Parameter description: + + > `--server`: URL for starting the patch tracking service, for example, 127.0.0.1:5001. + > `--user`: User name to be authenticated for the POST interface. It is the same as the USER parameter in the **settings.conf** file. + > `--password`: Password to be authenticated for the POST interface. It is the password string corresponding to the PASSWORD hash value in the **settings.conf** file. + > `--file`: YAML file path. + + Add the information about the repository, branch, version management tool, and whether to enable monitoring to the YAML file (for example, **tracking.yaml**). The file path is used as the command of the `--file` to invoke the input parameters. + + For example: + + ```shell + patch-tracking-cli add --server 127.0.0.1:5001 --user admin --password Test@123 --file tracking.yaml + ``` + + The format of the YAML file is as follows. The content on the left of the colon (:) cannot be modified, and the content on the right of the colon (:) needs to be set based on the site requirements. + + ```shell + version_control: github + scm_repo: xxx/xxx + scm_branch: master + repo: xxx/xxx + branch: master + enabled: true + ``` + + > version\_control: Control tool of the upstream repository version. Only GitHub is supported. + > scm\_repo: Name of the upstream repository to be tracked, in the GitHub format of organization/repository. + > scm\_branch: Branch of the upstream repository to be tracked. + > repo: Name of the repository to be tracked, in the format of organization/repository. + > branch: Branch name of the repository to be tracked. + > enabled: Indicates whether to automatically track the repository. + +- Using a Specified Directory + + Place multiple `xxx.yaml` files in a specified directory, such as the `test_yaml`, and run the following command to record the tracking items of all YAML files in the specified directory. + + Parameter description: + + > `--user`: User name to be authenticated for the POST interface. It is the same as the USER parameter in the **settings.conf** file. + > `--password`: Password to be authenticated for the POST interface. It is the password string corresponding to the PASSWORD hash value in the **settings.conf** file. + > `--server`: URL for starting the patch tracking service, for example, 127.0.0.1:5001. + > `--dir`: Path where the YAML file is stored. + + ```shell + patch-tracking-cli add --server 127.0.0.1:5001 --user admin --password Test@123 --dir /home/Work/test_yaml/ + ``` + +### Querying a Tracking Item + +Parameter description: + +> `--server`: (Mandatory) URL for starting the patch tracking service, for example, 127.0.0.1:5001. +> `--table`: (Mandatory) Table to be queried. +> `--repo`: (Optional) repo to be queried. Query all content in the table if this parameter is not configured. +> `--branch`: (Optional) Branch to be queried. + +```shell +patch-tracking-cli query --server --table tracking +``` + +The website can be accessed properly. + +```shell +patch-tracking-cli query --server 127.0.0.1:5001 --table tracking +``` + +### Querying the Generated Issue + +```shell +patch-tracking-cli query --server --table issue +``` + +For example: + +```shell +patch-tracking-cli query --server 127.0.0.1:5001 --table issue +``` + +### Deleting a Tracking Item + +```shell +patch-tracking-cli delete --server SERVER --user USER --password PWD --repo REPO [--branch BRANCH] +``` + +For example: + +```shell +patch-tracking-cli delete --server 127.0.0.1:5001 --user admin --password Test@123 --repo testPatchTrack/testPatch1 --branch master +``` + +> You can delete a single piece of data from a specified repo or branch. You can also delete data of all branches in a specified repo. + +### Checking Issues and PRs on Gitee + +Log in to Gitee and check the software project to be tracked. On the Issues and Pull Requests tab pages of the project, you can see the item named in `[patch tracking] TIME`, for example, the `[patch tracking] 20200713101548`. This item is the issue and PR of the patch file that is just generated. diff --git a/docs/en/tools/devops/package_manage/pkgship/_toc.yaml b/docs/en/tools/devops/package_manage/pkgship/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..a5736a5dfddf5b2fc5e95638e11bb52319dceb52 --- /dev/null +++ b/docs/en/tools/devops/package_manage/pkgship/_toc.yaml @@ -0,0 +1,6 @@ +label: pkgship +isManual: true +href: ./pkgship.md +description: >- + Software packages dependency lookup, lifecycle management, patch tracking, and + more. diff --git a/docs/en/tools/devops/package_manage/pkgship/images/packagemanagement.png b/docs/en/tools/devops/package_manage/pkgship/images/packagemanagement.png new file mode 100644 index 0000000000000000000000000000000000000000..6d314e2c6ad6bafd321d9f76cd6aa5f17a8cb394 Binary files /dev/null and b/docs/en/tools/devops/package_manage/pkgship/images/packagemanagement.png differ diff --git a/docs/en/tools/devops/package_manage/pkgship/images/panel.png b/docs/en/tools/devops/package_manage/pkgship/images/panel.png new file mode 100644 index 0000000000000000000000000000000000000000..150eb8c8229f9e8cb47706f3b82f07516a505076 Binary files /dev/null and b/docs/en/tools/devops/package_manage/pkgship/images/panel.png differ diff --git a/docs/en/tools/devops/package_manage/pkgship/pkgship.md b/docs/en/tools/devops/package_manage/pkgship/pkgship.md new file mode 100644 index 0000000000000000000000000000000000000000..2d44c69434932f938ed1be7b6ec1a85edf99866c --- /dev/null +++ b/docs/en/tools/devops/package_manage/pkgship/pkgship.md @@ -0,0 +1,435 @@ +# pkgship + + + +- [pkgship](#pkgship) + - [Introduction](#introduction) + - [Architecture](#architecture) + - [Using the Software Online](#using-the-software-online) + - [Downloading the Software](#downloading-the-software) + - [Operating Environment](#operating-environment) + - [Installing the Tool](#installing-the-tool) + - [Configuring Parameters](#configuring-parameters) + - [Starting and Stopping the Service](#starting-and-stopping-the-service) + - [Using the Tool](#using-the-tool) + - [Viewing and Dumping Logs](#viewing-and-dumping-logs) + - [pkgship-panel](#pkgship-panel) + + + +## Introduction + +The pkgship is a query tool used to manage the dependency of OS software packages and provide a complete dependency graph. The pkgship provides functions such as software package dependency query and lifecycle management. + +1. Software package basic information query: Allow community personnel to quickly obtain information about the name, version, and description of the software package. +2. Software package dependency query: Allow community personnel to understand the impact on software when software packages are introduced, updated, or deleted. + +## Architecture + +The system uses the Flask-RESTful development mode. The following figure shows the architecture: + +![avatar](./images/packagemanagement.png) + +## Using the Software Online + +pkgship provides a [public online service](https://pkgmanage.openeuler.org/Packagemanagement). You can directly use pkgship online if you do not need to customize your query. + +To use a custom data source, install, configure, and use pkgship by referring to the following sections. + +## Downloading the Software + +- The repo source is officially released at: +- You can obtain the source code at: +- You can obtain the RPM package at: + +## Operating Environment + +- Hardware configuration: + +| Item| Recommended Specification| +|----------|----------| +| CPU| 8 cores| +| Memory| 32 GB (minimum: 4 GB)| +| Dive space| 20 GB| +| Network bandwidth| 300 Mbit/s| +| I/O| 375 MB/s| + +- Software configuration: + +| Name| Specifications| +|----------|----------| +| Elasticsearch| 7.10.1. Single-node and cluster deployment is available.| +| Redis| 5.0.4 or later is recommended. You are advised to set the size to 3/4 of the memory.| +| Python| 3.8 or later.| + +## Installing the Tool + +>Note: The software can run in Docker. In openEuler 21.09, due to environment restrictions, use the **--privileged** parameter when creating a Docker. Otherwise, the software fails to be started. This document will be updated after the adaptation. + +**1. Installing the pkgship** + +You can use either of the following methods to install the pkgship: + +- Method 1: Mount the repo source using DNF. + + Use DNF to mount the repo source where the pkgship is located (for details, see the [Application Development Guide](../../../../server/development/application_dev/application-development.md)). Then run the following command to download and install the pkgship and its dependencies: + + ```bash + dnf install pkgship + ``` + +- Method 2: Install the RPM package. Download the RPM package of the pkgship and run the following command to install the pkgship (x.x-x indicates the version number and needs to be replaced with the actual one): + + ```bash + rpm -ivh pkgship-x.x-x.oe1.noarch.rpm + ``` + + Or + + ```bash + dnf install pkgship-x.x-x.oe1.noarch.rpm + ``` + +**2. Installing Elasticsearch and Redis** + +If Elasticsearch or Redis is not installed in the environment, you can execute the automatic installation script after the pkgship is installed. + +The default script path is as follows: + +```bash +/etc/pkgship/auto_install_pkgship_requires.sh +``` + +Run the following command: + +```bash +/bin/bash auto_install_pkgship_requires.sh elasticsearch +``` + +Or + +```bash +/bin/bash auto_install_pkgship_requires.sh redis +``` + +**3. Adding a User After the Installation** + +After the pkgship software is installed, the system automatically creates a user named **pkgshipuser** and a user group named **pkgshipuser**. They will be used when the service is started and running. + +## Configuring Parameters + +1. Configure the parameters in the configuration file. The default configuration file of the system is stored in **/etc/pkgship/package.ini**. Modify the configuration file as required. + + ```bash + vim /etc/pkgship/package.ini + ``` + + ```ini + [SYSTEM-System Configuration] + ; Path for storing the .yaml file imported during database initialization. The .yaml file records the location of the imported .sqlite file. + init_conf_path=/etc/pkgship/conf.yaml + + ; Service query port + query_port=8090 + + ; Service query IP address + query_ip_addr=127.0.0.1 + + ; Address of the remote service. The command line can directly call the remote service to complete the data request. + remote_host=https://api.openeuler.org/pkgmanage + + ; Directory for storing temporary files during initialization and download. The directory will not be occupied for a long time. It is recommended that the available space be at least 1 GB. + temporary_directory=/opt/pkgship/tmp/ + + [LOG-Logs] + ; Service log storage path + log_path=/var/log/pkgship/ + + ; Log level. The options are as follows: + ; INFO DEBUG WARNING ERROR CRITICAL + log_level=INFO + + ; Maximum size of a service log file. If the size of a service log file exceeds the value of this parameter, the file is automatically compressed and dumped. The default value is 30 MB. + max_bytes=31457280 + + ; Maximum number of backup log files. The default value is 30. + backup_count=30 + + [UWSGI-Web Server Configuration] + ; Operation log path + daemonize=/var/log/pkgship-operation/uwsgi.log + ; Size of data transmitted between the front end and back end + buffer-size=65536 + ; Network connection timeout interval + http-timeout=600 + ; Service response time + harakiri=600 + + [REDIS-Cache Configuration] + ; The address of the Redis cache server can be the released domain or IP address that can be accessed. + ; The default link address is 127.0.0.1. + redis_host=127.0.0.1 + + ; Port number of the Redis cache server. The default value is 6379. + redis_port=6379 + + ; Maximum number of connections allowed by the Redis server at a time. + redis_max_connections=10 + + [DATABASE-Database] + ; Database access address. The default value is the IP address of the local host. + database_host=127.0.0.1 + + ; Database access port. The default value is 9200. + database_port=9200 + ``` + +2. Create a YAML configuration file to initialize the database. The **conf.yaml** file is stored in the **/etc/pkgship/** directory by default. The pkgship reads the name of the database to be created and the SQLite file to be imported based on this configuration. You can also configure the repo address of the SQLite file. An example of the **conf.yaml** file is as follows: + + ```yaml + dbname: oe20.03 #Database name + src_db_file: /etc/pkgship/repo/openEuler-20.09/src #Local path of the source package + bin_db_file: /etc/pkgship/repo/openEuler-20.09/bin #Local path of the binary package + priority: 1 #Database priority + + dbname: oe20.09 + src_db_file: https://repo.openeuler.org/openEuler-20.09/source #Repo source of the source package + bin_db_file: https://repo.openeuler.org/openEuler-20.09/everything/aarch64 #Repo source of the binary package + priority: 2 + ``` + + > To change the storage path, change the value of **init\_conf\_path** in the **package.ini** file. + > + > The SQLite file path cannot be configured directly. + > + > The value of **dbname** can contain only lowercase letters, digits, periods (.), hyphens (-), underscores (_), and plus signs (+), and must start and end with lower case letters or digits. + +## Starting and Stopping the Service + +The pkgship can be started and stopped in two modes: systemctl mode and pkgshipd mode. In systemctl mode, the automatic startup mechanism can be stopped when an exception occurs. You can run any of the following commands: + +```bash +systemctl start pkgship.service # Start the service. + +systemctl stop pkgship.service # Stop the service. + +systemctl restart pkgship.service # Restart the service. +``` + +```bash +pkgshipd start # Start the service. + +pkgshipd stop # Stop the service. +``` + +> Only one mode is supported in each start/stop period. The two modes cannot be used at the same time. +> +> The pkgshipd startup mode can be used only by the **pkgshipuser** user. +> +> If the **systemctl** command is not supported in the Docker environment, run the **pkgshipd** command to start or stop the service. + +## Using the Tool + +1. Initialize the database. + + > Application scenario: After the service is started, to query the package information and dependency in the corresponding database (for example, oe20.03 and oe20.09), you need to import the SQLite (including the source code library and binary library) generated by the **createrepo** to the service. Then insert the generated JSON body of the package information into the corresponding database of Elasticsearch. The database name is the value of d**bname-source/binary** generated based on the value of **dbname** in the **conf.yaml** file. + + ```bash + pkgship init [-filepath path] + ``` + + > Parameter description: + > **-filepath**: (Optional) Specifies the path of the initialization configuration file **config.yaml.** You can use either a relative path or an absolute path. If no parameter is specified, the default configuration is used for initialization. + +2. Query a single package. + + You can query details about a source package or binary package (**packagename**) in the specified **database** table. + + > Application scenario: You can query the detailed information about the source package or binary package in a specified database. + + ```bash + pkgship pkginfo $packageName $database [-s] + ``` + + > Parameter description: + > **packagename**: (Mandatory) Specifies the name of the software package to be queried. + > **database**: (Mandatory) Specifies the database name. + > + > **-s**: (Optional) Specifies that the source package `src` is to be queried by `-s`. If this parameter is not specified, the binary package information of `bin` is queried by default. + +3. Query all packages. + + Query information about all packages in the database. + + > Application scenario: You can query information about all software packages in a specified database. + + ```bash + pkgship list $database [-s] + ``` + + > Parameter description: + > **database**: (Mandatory) Specifies the database name. + > **-s**: (Optional) Specifies that the source package `src` is to be queried by `-s`. If this parameter is not specified, the binary package information of `bin` is queried by default. + +4. Query the installation dependency. + + Query the installation dependency of the binary package (**binaryName**). + + > Application scenario: When you need to install the binary package A, you need to install B, the installation dependency of A, and C, the installation dependency of B, etc. A can be installed only after all the installation dependencies are installed in the system. Therefore, before installing the binary package A, you may need to query all installation dependencies of A. You can run the following command to query multiple databases based on the default priority of the platform, and to customize the database query priority. + + ```bash + pkgship installdep [$binaryName $binaryName1 $binaryName2...] [-dbs] [db1 db2...] [-level] $level + ``` + + > Parameter description: + > **binaryName**: (Mandatory) Specifies the name of the dependent binary package to be queried. Multiple packages can be transferred. + > + > **-dbs:** (Optional) Specifies the priority of the database to be queried. If this parameter is not specified, the database is queried based on the default priority. + > + > **-level**: (Optional) Specifies the dependency level to be queried. If this parameter is not specified, the default value **0** is used, indicating that all levels are queried. + +5. Query the compilation dependency. + + Query all compilation dependencies of the source code package (**sourceName**). + + > Application scenario: To compile the source code package A, you need to install B, the compilation dependency package of A. To install B, you need to obtain all installation dependency packages of B. Therefore, before compiling the source code package A, you need to query the compilation dependencies of A and all installation dependencies of these compilation dependencies. You can run the following command to query multiple databases based on the default priority of the platform, and to customize the database query priority. + + ```bash + pkgship builddep [$sourceName $sourceName1 $sourceName2..] -dbs [db1 db2 ..] [-level] $level + ``` + + > Parameter description: + > **sourceName**: (Mandatory) Specifies the name of the source package on which the compilation depends. Multiple packages can be queried. + > + > **-dbs:** (Optional) Specifies the priority of the database to be queried. If this parameter is not specified, the database is queried based on the default priority. + > + > **-level**: (Optional) Specifies the dependency level to be queried. If this parameter is not specified, the default value **0** is used, indicating that all levels are queried. + +6. Query the self-compilation and self-installation dependencies. + + Query the installation and compilation dependencies of a specified binary package (**binaryName**) or source package (**sourceName**). In the command, **\[pkgName]** indicates the name of the binary package or source package to be queried. When querying a binary package, you can query all installation dependencies of the binary package, and the compilation dependencies of the source package corresponding to the binary package, as well as all installation dependencies of these compilation dependencies. When querying a source package, you can query its compilation dependency, and all installation dependencies of the compilation dependency, as well as all installation dependencies of the binary packages generated by the source package. In addition, you can run this command together with the corresponding parameters to query the self-compilation dependency of a software package and the dependency of a subpackage. + + > Application scenario: If you want to introduce a new software package based on the existing version library, you need to introduce all compilation and installation dependencies of the software package. You can run this command to query these two dependency types at the same time to know the packages introduced by the new software package, and to query binary packages and source packages. + + ```bash + pkgship selfdepend [$pkgName1 $pkgName2 $pkgName3 ..] [-dbs] [db1 db2..] [-b] [-s] [-w] + ``` + + > Parameter description: + > + > **pkgName**: (Mandatory) Specifies the name of the software package on which the installation depends. Multiple software packages can be transferred. + > + > **-dbs:** (Optional) Specifies the priority of the database to be queried. If this parameter is not specified, the database is queried based on the default priority. + > + > **-b**: (Optional) Specifies that the package to be queried is a binary package. If this parameter is not specified, the source package is queried by default. + > + > **-s**: (Optional) If **-s** is specified, all installation dependencies, compilation dependencies (that is, compilation dependencies of the source package on which compilation depends), and installation dependencies of all compilation dependencies of the software package are queried. If **-s** is not added, all installation dependencies and layer-1 compilation dependencies of the software package, as well as all installation dependencies of layer-1 compilation dependencies, are queried. + > + > **-w**: (Optional) If **-w** is specified, when a binary package is introduced, the query result displays the source package corresponding to the binary package and all binary packages generated by the source package. If **-w** is not specified, only the corresponding source package is displayed in the query result when a binary package is imported. + +7. Query dependency. + Query the packages that depend on the software package (**pkgName**) in a database (**dbName**). + + > Application scenario: You can run this command to query the software packages that will be affected by the upgrade or deletion of the software source package A. This command displays the source packages (for example, B) that depend on the binary packages generated by source package A (if it is a source package or the input binary package for compilation). It also displays the binary packages (for example, C1) that depend on A for installation. Then, it queries the source package (for example, D) that depend on the binary package generated by B C1 for compilation and the binary package (for example E1) for installation. This process continues until it traverses the packages that depend on the binary packages. + + ```bash + pkgship bedepend dbName [$pkgName1 $pkgName2 $pkgName3] [-w] [-b] [-install/build] + ``` + + > Parameter description: + > + > **dbName**: (Mandatory) Specifies the name of the repository whose dependency needs to be queried. Only one repository can be queried each time. + > + > **pkgName**: (Mandatory) Specifies the name of the software package to be queried. Multiple software packages can be queried. + > + > **-w**: (Optional) If **-w** is not specified, the query result does not contain the subpackages of the corresponding source package by default. If **\[-w]** is specified after the command, not only the dependency of binary package C1 is queried, but also the dependency of other binary packages (such as C2 and C3) generated by source package C corresponding to C1 is queried. + > + > **-b**: (Optional) Specifies `-b` and indicates that the package to be queried is a binary package. By default, the source package is queried. + > + > **-install/build**: (Optional) `-install` indicates that installation dependencies are queried. `-build` indicates that build dependencies are queried. By default, all dependencies are queried. `-install` and `-build` are exclusive to each other. + +8. Query the database information. + + > Application scenario: Check which databases are initialized in Elasticsearch. This function returns the list of initialized databases based on the priority. + + `pkgship dbs` + +9. Obtain the version number. + + > Application scenario: Obtain the version number of the pkgship software. + + `pkgship -v` + +## Viewing and Dumping Logs + +**Viewing Logs** + +When the pkgship service is running, two types of logs are generated: service logs and operation logs. + +1. Service logs: + + Path: **/var/log/pkgship/log\_info.log**. You can customize the path through the **log\_path** field in the **package.ini** file. + + Function: This log records the internal running of the code to facilitate fault locating. + + Permission: The permissions on the path and the log file are 755 and 644, respectively. Common users can view the log file. + +2. Operation logs: + + Path: **/var/log/pkgship-operation/uwsgi.log**. You can customize the path through the **daemonize** field in the **package.ini** file. + + Function: This log records user operation information, including the IP address, access time, URL, and result, to facilitate subsequent queries and record attacker information. + + Permission: The permissions on the path and the log file are 700 and 644, respectively. Only the **root** and **pkgshipuser** users can view the log file. + +**Dumping Logs** + +1. Service log dumping: + + - Dumping mechanism + + Use the dumping mechanism of the logging built-in function of Python to back up logs based on the log size. + + > The items are used to configure the capacity and number of backups of each log in the **package.ini** file. + + ```ini + ; Maximum capacity of each file, the unit is byte, default is 30M + max_bytes=31457280 + + ; Number of old logs to keep;default is 30 + backup_count=30 + ``` + + - Dumping process + + After a log is written, if the size of the log file exceeds the configured log capacity, the log file is automatically compressed and dumped. The compressed file name is **log\_info.log.***x***.gz**, where *x* is a number. A smaller number indicates a later backup. + + When the number of backup log files reaches the threshold, the earliest backup log file is deleted and the latest compressed log file is backed up. + +2. Operation log dumping: + + - Dumping mechanism + + A script is used to dump data by time. Data is dumped once a day and is retained for 30 days. Customized configuration is not supported. + + > The script is stored in **/etc/pkgship/uwsgi\_logrotate.sh**. + + - Dumping process + + When the pkgship is started, the script for dumping data runs in the background. From the startup, dumping and compression are performed every other day. A total of 30 compressed files are retained. The compressed file name is **uwsgi.log-20201010*x*.zip**, where *x* indicates the hour when the file is compressed. + + After the pkgship is stopped, the script for dumping data is stopped and data is not dumped . When the pkgship is started again, the script for dumping data is executed again. + +## pkgship-panel + +### Introduction + +pkgship-panel integrates software package build information and maintenance information so that version maintenance personnel can quickly identify abnormal software packages and notify the package owners to solve the problems, ensuring build project stability and improving the OS build success rate. + +### Architecture + +![](images/panel.png) + +### Using the Tool + +The data source of pkgship-panel cannot be configured. You are advised to use the [pkgship-panel official website](https://pkgmanage.openeuler.org/Infomanagement). diff --git a/docs/en/tools/maintenance/_toc.yaml b/docs/en/tools/maintenance/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..421eba7ff2ce320be80968ce6f5ecaf3ada7fdc4 --- /dev/null +++ b/docs/en/tools/maintenance/_toc.yaml @@ -0,0 +1,8 @@ +label: O&M +sections: + - label: Hot Patch Creation + sections: + - href: ../../server/maintenance/syscare/_toc.yaml + - label: System Monitoring + sections: + - href: ../../server/maintenance/sysmonitor/_toc.yaml diff --git a/docs/en/tools/security/_toc.yaml b/docs/en/tools/security/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..96ba8d6a24747cc17ce88dbba757a210155ef0ed --- /dev/null +++ b/docs/en/tools/security/_toc.yaml @@ -0,0 +1,3 @@ +label: Security +sections: + - href: ../../server/security/secgear/_toc.yaml diff --git a/docs/en/virtualization/_toc.yaml b/docs/en/virtualization/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..0c4603212d1bbc2784f69006d2e497b40c0d8fff --- /dev/null +++ b/docs/en/virtualization/_toc.yaml @@ -0,0 +1,8 @@ +label: Virtualization +sections: + - label: Virtualization Platforms + sections: + - href: ./virtualization_platform/virtualization/_toc.yaml + - href: ./virtualization_platform/stratovirt/_toc.yaml + - href: >- + https://gitee.com/openeuler/openstack-docs/blob/openEuler-25.03/docs/zh/_toc.yaml diff --git a/docs/en/virtualization/virtualization_platform/stratovirt/_toc.yaml b/docs/en/virtualization/virtualization_platform/stratovirt/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..a0adbd5f07354c260fec62f210450da9ae7e4e04 --- /dev/null +++ b/docs/en/virtualization/virtualization_platform/stratovirt/_toc.yaml @@ -0,0 +1,23 @@ +label: StratoVirt User Guide +isManual: true +description: >- + StratoVirt serves as an enterprise-level virtualization platform tailored for + cloud data centers in the computing sector. It enables a unified architecture + to accommodate VMs, containers, and serverless environments. +sections: + - label: StratoVirt Introduction + href: ./stratovirt_introduction.md + - label: StratoVirt Installation + href: ./install_stratovirt.md + - label: Environment Preparation + href: ./prepare_env.md + - label: VM Configuration + href: ./vm_configuration.md + - label: VM Management + href: ./vm_management.md + - label: Interconnection with the iSula Secure Container + href: ./interconnect_isula.md + - label: Interconnection with libvirt + href: ./interconnect_libvirt.md + - label: StratoVirt VFIO User Guide + href: ./stratovirt_vfio_instructions.md diff --git a/docs/en/virtualization/virtualization_platform/stratovirt/figures/StratoVirt_architecture.jpg b/docs/en/virtualization/virtualization_platform/stratovirt/figures/StratoVirt_architecture.jpg new file mode 100644 index 0000000000000000000000000000000000000000..93f1697131dd2a6f8c010def9f42ad067b9b0bd9 Binary files /dev/null and b/docs/en/virtualization/virtualization_platform/stratovirt/figures/StratoVirt_architecture.jpg differ diff --git a/docs/en/virtualization/virtualization_platform/stratovirt/install_stratovirt.md b/docs/en/virtualization/virtualization_platform/stratovirt/install_stratovirt.md new file mode 100644 index 0000000000000000000000000000000000000000..a03c22afa255d98efa61becdc2f1d29a794ca544 --- /dev/null +++ b/docs/en/virtualization/virtualization_platform/stratovirt/install_stratovirt.md @@ -0,0 +1,32 @@ +# Installing StratoVirt + +## Software and Hardware Requirements + +### Minimum Hardware Requirements + +- Processor architecture: Only the AArch64 and x86_64 processor architectures are supported. AArch64 requires ARMv8 or a later version that supports virtualization extension. x86_64 requires VT-x support. + +- 2-core CPU +- 4 GiB memory +- 16 GiB available drive space + +### Software Requirements + +Operating system: openEuler 21.03 + +## Component Installation + +To use StratoVirt virtualization, it is necessary to install StratoVirt. Before the installation, ensure that the openEuler Yum source has been configured. + +1. Run the following command as user **root** to install the StratoVirt component: + + ```shell + # yum install stratovirt + ``` + +2. Check whether the installation is successful. + + ```shell + $ stratovirt -version + StratoVirt 2.1.0 + ``` diff --git a/docs/en/virtualization/virtualization_platform/stratovirt/interconnect_isula.md b/docs/en/virtualization/virtualization_platform/stratovirt/interconnect_isula.md new file mode 100644 index 0000000000000000000000000000000000000000..b461f56b1ff5dce06d97eb6fedee7c29257242fd --- /dev/null +++ b/docs/en/virtualization/virtualization_platform/stratovirt/interconnect_isula.md @@ -0,0 +1,220 @@ +# Interconnection with the iSula Secure Container + +## Overview + +To provide a better isolation environment for containers and improve system security, you can interconnect StratoVirt with iSula secure containers. + +## Connecting to the iSula Secure Container + +### Prerequisites + +iSulad and kata-containers have been installed, and iSulad supports the containerd-kata-shim-v2 container runtime and devicemapper storage driver. + +The following describes how to install and configure iSulad and kata-containers. + +1. Configure the Yum source and install iSulad and kata-containers as the **root** user. + + ```shell + # yum install iSulad + # yum install kata-containers + ``` + +2. Create and configure a storage device. + + You need to plan the drive, for example, **/dev/sdxx**, which will be formatted. + + ```shell + # pvcreate /dev/sdxx + # vgcreate isulaVG0 /dev/sdxx + # lvcreate --wipesignatures y -n thinpool isulaVG0 -l 95%VG + # lvcreate --wipesignatures y -n thinpoolmeta isulaVG0 -l 1%VG + # lvconvert -y --zero n -c 512K --thinpool isulaVG0/thinpool --poolmetadata isulaVG0/thinpoolmeta + ``` + + Add the following information to the **/etc/lvm/profile/isulaVG0-thinpool.profile** configuration file: + + ```text + activation { + thin_pool_autoextend_threshold=80 + thin_pool_autoextend_percent=20 + } + ``` + + Modify **storage-driver** and **storage-opts** in the **/etc/isulad/daemon.json** configuration file as follows. Set the default storage driver type **overlay** to **devicemapper**. + + ```json + "storage-driver": "devicemapper", + "storage-opts": [ + "dm.thinpooldev=/dev/mapper/isulaVG0-thinpool", + "dm.fs=ext4", + "dm.min_free_space=10%" + ], + ``` + +3. Restart **isulad**. + + ```shell + # systemctl daemon-reload + # systemctl restart isulad + ``` + +4. Check whether the iSula storage driver is successfully configured. + + ```shell + # isula info + ``` + + If the following information is displayed, the configuration is successful: + + ```text + Storage Driver: devicemapper + ``` + +### Interconnection Guide + +This section describes how to interconnect StratoVirt with kata-containers to access the iSula container ecosystem. + +#### Connecting to a Lightweight VM + +1. Modify the kata configuration file. Its default path is **/usr/share/defaults/kata-containers/configuration.toml**. You can also configure the file by referring to **configuration-stratovirt.toml** in the same directory. Modify the **hypervisor** type of the secure container to **stratovirt**, **kernel** to the absolute path of the kernel image of kata-containers, and **initrd** to the **initrd** image file of kata-containers. (If you use Yum to install kata-containers, the two image files are downloaded and stored in the **/var/lib/kata/** directory by default. You can also use other images during the configuration.) + + The modified configurations are as follows: + + ```shell + [hypervisor.stratovirt] + path = "/usr/bin/stratovirt" + kernel = "/var/lib/kata/kernel" + initrd = "/var/lib/kata/kata-containers-initrd.img" + machine_type = "microvm" + block_device_driver = "virtio-mmio" + use_vsock = true + enable_netmon = true + internetworking_model="tcfilter" + sandbox_cgroup_with_emulator = false + disable_new_netns = false + disable_block_device_use = false + disable_vhost_net = true + ``` + +2. Run the `isula` command with **root** permissions to start the BusyBox secure container and interconnect StratoVirt with it. + + ```shell + # isula run -tid --runtime "io.containerd.kata.v2" --net=none --name test busybox:latest sh + ``` + +3. Run the `isula ps` command to check whether the secure container **test** is running properly. Then run the following command to access the container: + + ```shell + # isula exec -ti test sh + ``` + +4. Use a VM snapshot to accelerate startup of the secure container and reduce the VM memory overhead. + + Modify the kata configuration file **configuration.toml** and set **enable_template** to **true** to allow the VM to start by creating a snapshot. + + ```shell + [factory] + # VM templating support. Once enabled, new VMs are created from template + # using vm cloning. They will share the same initial kernel, initramfs and + # agent memory by mapping it readonly. It helps speeding up new container + # creation and saves a lot of memory if there are many kata containers running + # on the same host. + # + # When disabled, new VMs are created from scratch. + # + # Note: Requires "initrd=" to be set ("image=" is not supported). + # + # Default false + enable_template = true + ``` + + After the **enable_template** configuration item is set to **true**, kata-containers checks whether a snapshot file exists in the default path (**/run/vc/vm/template**) during secure container creation. If yes, kata-containers starts the VM using the snapshot file. If no, kata-containers creates a VM snapshot and start the VM using the snapshot file. + +5. Use the security component Ozone to further enhance the isolation of secure containers. + + Modify the kata configuration file **configuration.toml** and set the configuration item **ozone_path** to the path of the Ozone executable file. (If StratoVirt is installed using Yum, the Ozone executable file is stored in the **/usr/bin** directory by default.) After this item is configured, the Ozone security sandbox function is enabled to protect the VM against attacks after the virtualization layer isolation is broken and further enhance the isolation of StratoVirt secure containers. + + ```shell + # Path for the ozone specific to stratovirt + # If the ozone path is set, stratovirt will be launched in + # ozone secure environment. It is disabled by default. + ozone_path = "/usr/bin/ozone" + ``` + + You can now run container commands in the **test** container. + +#### Connecting to a Standard VM + +To use a StratoVirt standard VM as the sandbox of a secure container, you need to modify some other configurations. + +1. The configurations are as follows: + + ```shell + [hypervisor.stratovirt] + path = "/usr/bin/stratovirt" + kernel = "/var/lib/kata/kernel" + initrd = "/var/lib/kata/kata-containers-initrd.img" + # x86_64 architecture + machine_type = "q35" + # AArch64 architecture + machine_type = "virt" + block_device_driver = "virtio-blk" + pcie_root_port = 2 + use_vsock = true + enable_netmon = true + internetworking_model = "tcfilter" + sandbox_cgroup_with_emulator = false + disable_new_netns = false + disable_block_device_use = false + disable_vhost_net = true + ``` + + In the configurations above, modify the VM type according to the architecture of the host machine. Change the value of **block_device_driver** to **virtio-blk**. StratoVirt supports only devices hot-plugged to the root port. Set a proper value of **pcie_root_port** based on the number of devices to be hot-plugged. + +2. Install the firmware required for starting a standard VM. + + x86_64 architecture: + + ```shell + # yum install -y edk2-ovmf + ``` + + AArch64 architecture: + + ```shell + # yum install -y edk2-aarch64 + ``` + +3. Build and replace the binary file of kata-containers 2.x. + + Currently, a StratoVirt standard VMs can only be used as the sandbox of a kata-containers 2.x container (corresponding to the openEuler-21.09 branch in the kata-containers repository). You need to download and compile the kata-containers source code and replace the **containerd-shim-kata-v2** binary file in the **/usr/bin** directory. + + ```shell + # mkdir -p /root/go/src/github.com/ + # cd /root/go/src/github.com/ + # git clone https://gitee.com/src-openeuler/kata-containers.git + # cd kata-containers + # git checkout openEuler-21.09 + # ./apply-patches + # cd src/runtime + # make + ``` + + Back up the kata binary file in the **/usr/bin/** directory and replace it with the compiled binary file **containerd-shim-kata-v2**. + + ```shell + # cp /usr/bin/containerd-shim-kata-v2 /usr/bin/containerd-shim-kata-v2.bk + # cp containerd-shim-kata-v2 /usr/bin/containerd-shim-kata-v2 + ``` + +4. Run the `isula` command with **root** permissions to start the BusyBox secure container and interconnect StratoVirt with it. + + ```shell + # isula run -tid --runtime "io.containerd.kata.v2" --net=none --name test busybox:latest sh + ``` + +5. Run the `isula ps` command to check whether the secure container **test** is running properly. Then run the following command to access the container: + + ```shell + # isula exec -ti test sh + ``` diff --git a/docs/en/virtualization/virtualization_platform/stratovirt/interconnect_libvirt.md b/docs/en/virtualization/virtualization_platform/stratovirt/interconnect_libvirt.md new file mode 100644 index 0000000000000000000000000000000000000000..188d0cbb9fbf5b858d0d5139cdf4c2bac1ac0af8 --- /dev/null +++ b/docs/en/virtualization/virtualization_platform/stratovirt/interconnect_libvirt.md @@ -0,0 +1,683 @@ +# Interconnection with libvirt + +## Overview + +libvirt is an upper-layer software that manages different types of Hypervisors using different drivers and provides unified and stable APIs. + +In cloud scenarios, libvirt is widely used to manage large numbers of VMs. To facilitate the deployment, orchestration, and management of large-scale StratoVirt VMs, StratoVirt interconnects with libvirt through the libvirt northbound interface. In this case, you can use an XML file of libvirt to describe a VM, including the VM name, CPU, and disks. + +This chapter describes the XML configurations supported by the StratoVirt platform and how to use the `virsh` command to manage VMs. + +## Prerequisites + +To interconnect StratoVirt with libvirt, the host must meet the following requirements: + +- The Yum source has been correctly configured. +- libvirt has been correctly installed and started. +- StratoVirt has been correctly installed. + +## VM Configuration + +The libvirt tool uses an XML file to describe features about a VM, including the VM name, CPUs, memory, disks, and NICs. You can manage the VM by modifying the XML configuration file. + +Before interconnecting StratoVirt with libvirt, configure the XML file first. This section describes the supported XML configuration items and configuration methods during interconnection between StratoVirt and libvirt. + +> ![](./public_sys-resources/icon-note.gif)**NOTE** +> +> Before using libvirt to manage StratoVirt VMs, pay attention to the features supported by StratoVirt, including mutually exclusive relationships between features, and feature prerequisites and specifications. For details, see [Configuring VMs](./vm_configuration.md) mode. + +### VM Description + +A VM XML file must contain the two basic elements that describe the VM: **domain** and **name**. + +#### Elements + +- **domain**: root element of the VM configuration, which is used to configure the Hypervisor type that runs the StratoVirt VM. + + Attribute **type**: type of **domain**. In StratoVirt, the value is **kvm**. + +- **name**: VM name. + + A VM name contains a maximum of 255 characters, consisting of digits, letters, underscores, hyphens, and colons. Names of VMs on the same host must be unique. + +#### Configuration Example + +Assume that the VM name is StratoVirt. The following is the example: + +```shell + + StratoVirt + ... + +``` + +### Virtual CPU and Memory + +This section describes how to configure virtual CPUs and memory. + +#### Elements + +- **vcpu**: number of virtual processors. + +- **memory**: size of the virtual memory. + + Attribute **unit**: memory unit. The value can be **KiB** (210 bytes), **MiB** (220 bytes), **GiB** (230 bytes), or **TiB** (240 bytes). + + > ![](./public_sys-resources/icon-note.gif)**NOTE** + > + > StratoVirt does not support the CPU topology. Do not set this item. + +#### Configuration Example + +The following is an example of configuring 8 GiB memory and four virtual CPUs: + +```xml + + ... + 4 + 8 + ... + +``` + +### VM Devices + +This section describes how to use the XML file to configure VM devices, including disk, NIC, RNG (random number generator), balloon, console, and vsock devices. + +#### Disks + +#### Elements + +- Attribute **type**: type of the backend storage medium. In StraroVirt, the value is **file**. + + Attribute **device**: type of the storage medium presented to the VM. In StraroVirt, the value is **disk**. + +- **driver**: details about the backend driver. + + Attribute **type**: disk format type. In StraroVirt, the value is **raw**. Currently, StratoVirt supports only **raw** disks. + + Attribute **iothread**: iothread configured for the disk. The value is the iothread ID. Before configuring the disk iothread, use the **iothread** element to configure the iothread quantity. + +- **source**: backend storage medium. + + Attribute **file**: disk path. + +- **target**: details about the backend driver. + + Attribute **dev**: disk name. + + Attribute **bus**: disk device type. In StraroVirt, the value is **virtio**. + +- **iotune**: disk I/O feature. + + Attribute **total_iops_sec**: disk IOPS. + +- **address**: attribute of the bus to which the device is to be mounted. + + Attribute **type**: bus type. In StratoVirt, the value is **pci**. + + Attribute **domain**: domain of the VM. + + Attribute **bus**: ID of the bus to which the device is to be mounted. + + Attribute **slot**: ID of the slot to which the device is to be mounted. The value range is \[0, 31]. + + Attribute **function**: ID of the function to which the device is to be mounted. The value range is \[0, 7]. + +#### Configuration Example + +Set the disk path to **/home/openEuler-21.09-stratovirt.img**, iothread quantity to **1**, disk iothread to **iothread1**, and IOPS to **10000**, and mount the disk to the PCI bus whose bus ID is 1, slot ID is 0, and function ID is 0. The following is the example: + +```xml + + ... + 1 + + + + + + + 10000 + +
+ + ... + + +``` + +#### Network Devices + +#### Elements + +- **interface**: network interface. + + Attribute **type**: network device type. + +- **mac**: virtual NIC address. + + Attribute **address**: virtual NIC address. + +- **source**: backend network bridge. + + Attribute **bridge**: network bridge. + +- **target**: backend NIC. + + Attribute **dev**: backend tap device. + +- **model**: virtual NIC type. + + Attribute **type**: virtual NIC type. In StratoVirt, the value is **virtio**. + +- **driver**: whether to enable the vhost. + + Attribute **name**: If **name** is set to **qemu**, the virtio-net device is used. If **driver** is not configured or **name** is set to **vhost**, the vhost-net device is used. + +#### Configuration Example + +Before configuring the network, [configure the Linux bridge](../Virtualization/environment-preparation.md#preparing-the-vm-network) first. Set the MAC address to **de:ad:be:ef:00:01** and network bridge to **br0**. Use the virtio-net device, and mount it to the PCI bus whose bus ID is 2, slot ID is 0, and function ID is 0. The following is the example: + +```xml + + ... + + + + + + +
+ + ... + + +``` + +#### Balloon Devices + +#### Elements + +- **memballoon**: balloon device type. + + Attribute **model**: type of the balloon device. In StratoVirt, the value is **virtio**. + +- **alias**: alias of the balloon device. + + Attribute **name**: ID of the balloon device. + + Attribute **autodeflate**: auto deflate feature. The options are **on** and **off**. + +#### Configuration Example + +Configure the balloon device, enable the auto deflate feature, and mount it to the PCI bus whose bus ID is 3, slot ID is 0, and function ID is 0. The following is the example: + +```xml + + ... + + + + +
+ + ... + + +``` + +#### Console Devices + +The console device is mounted to the virtio-serial bus. Therefore, you need to create a virtio-serial device when creating a console device. + +> ![](./public_sys-resources/icon-note.gif)**NOTE** +> +> The console device of StratoVirt does not support the multi-port feature. Each VM can be configured with only one console device. + +#### Elements + +- **controller**: controller. + + Attribute **type**: controller type. The value is **virtio-serial**. + +- **alias**: alias. + + Attribute **name**: device ID. + +- **console**: console device. + + Attribute **type**: redirection mode of the console device. The following redirection modes are supported: **pty**, **file**, and **unix**. + +- **target**: configuration of the console device. + + Attribute **type**: console device type. In StratoVirt, the value is **virtio**. + +#### Configuration Example + +Set the redirection mode to **pty** and mount the console device to the PCI bus whose bus ID is 4, slot ID is 0, and function ID is 0. The following is the example: + +```xml + + ... + + + +
+ + + + + + ... + + +``` + +#### RNG Devices + +#### Elements + +- **rng**: RNG device. + + Attribute **model**: type of the RNG device. In StratoVirt, the value is **virtio**. + +- **rate**: rate at which the RNG device generates random numbers. + + Attribute **period**: period of random number generation, in milliseconds. Currently, the StratoVirt does not allow you to set the period value. The default value (1000 milliseconds) is used. + + Attribute **bytes**: maximum number of bytes generated in a period. + +- **backend**: RNG device backend. The value is the path of the RNG device on the host. + + Attribute **model**: type of the backend device. In StratoVirt, the value is **random**. + +#### Configuration Example + +Configure that a maximum of 1234 bytes are generated within 1000 ms. The path of the RNG device on the host is **/dev/random**, and the device is mounted to the PCI bus whose bus ID is 5, slot ID is 0, and function ID is 0. The following is the example: + +```xml + + ... + + + + /dev/random +
+ + ... + + +``` + +#### vsock Devices + +#### Elements + +- **vsock**: vsock device. + + Attribute **model**: type of the vsock device. In StratoVirt, the value is **virtio**. + +- **cid**: CID of the vsock device. + + Attribute **address**: sets the CID value. + +#### Configuration Example + +Set **cid** to **8** and mount the device to the PCI bus whose bus ID is 6, slot ID is 0, and function ID is 0. The following is the example: + +```xml + + ... + + + +
+ + ... + + +``` + +### System Architecture Configuration + +The XML file also contains some architecture-related configurations, such as the pflash and mainboard. + +#### Elements + +- **os**: defines VM startup parameters. + + Child element **type**: VM type. Attribute **arch** indicates the architecture and **machine** indicates the mainboard type. In StratoVirt, the AArch64 architecture supports only the virt mainboard, and the x86_64 architecture supports only the Q35 mainboard. + + Child element **kernel**: kernel path. + + Child element **cmdline**: command line startup parameters. + + Child element **loader**: loading firmware. Attribute **readonly** indicates that the firmware is read-only and **type** indicates the firmware type. In StratoVirt, the type value is **pflash**. + +- **features**: features supported by Hypervisors. + + Child element **acpi**: whether to support ACPI. The ACPI feature is used in StratoVirt, so it must be configured. + + Child element **gic**: interrupt processor specified for ARM processors. Attribute **version** indicates the GIC version. In StratoVirt, the value is **3**. + +#### Configuration Example + +Set the CPU architecture of the VM to ARM and the mainboard to **virt**. The startup command is `console=ttyAMA0 root=/dev/vda reboot=k panic=1 rw`. The path of pflash is **/usr/share/edk2/aarch64/QEMU_EFI-pflash.raw**, which is read-only. The kernel path is **/home/std-vmlinuxz**. The following is the example: + +```xml + + ... + + hvm + /home/std-vmlinuxz + console=ttyAMA0 root=/dev/vda reboot=k panic=1 rw + `/usr/share/edk2/aarch64/QEMU_EFI-pflash.raw` + + ... + +``` + +### Huge Page Memory + +#### Elements + +- **memoryBacking**: configures the memory information. + +- **hugepages**: configures memory huge pages. + +- **page**: configures huge pages. + + Attribute **size**: size of huge memory pages. + + Attribute **unit**: unit of the huge page size. + +#### Configuration Example + +The following is an example of configuring 2 MiB huge pages: + +```xml + + ... + + + + + + ... + +``` + +### Configuration Examples + +#### x86 Configuration Example + +Configure a server named StratoVirt with 8 GiB memory, 1 GiB huge pages, and four vCPUs. Its architecture is x86_64 and the mainboard type is Q35. The following is a configuration example of the corresponding XML file: + +```xml + + StratoVirt + 8 + + + + + + + 4 + + 1 + + hvm + /path/to/standard_vm_kernel + console=hvc0 root=/dev/vda reboot=k panic=1 rw + /path/to/pflash + /path/to/OVMF_VARS + + + + + + /path/to/StratoVirt_binary_file + + + + + + + + + + + + + + + + 1000 + +
+ + + + + + +
+ + + + +
+ + + + + + + + +
+ + + + + /path/to/random_file +
+ + + + +
+ + + +``` + +#### ARM Configuration Example + +Configure a server named StratoVirt with 8 GiB memory, 1 GiB huge pages, four vCPUs. Its architecture is AArch64 and the mainboard type is virt. The configuration example of the corresponding XML file is as follows: + +```xml + + StratoVirt + 8 + + + + + + + 4 + + 1 + + hvm + /path/to/standard_vm_kernel + console=ttyAMA0 root=/dev/vda reboot=k panic=1 rw + /path/to/pflash + + + + + + + /path/to/StratoVirt_binary_file + + + + + + +
+ + 1000 + + + + + + + +
+ + + + +
+ + + + + + + + +
+ + + + + /path/to/random_file +
+ + + + +
+ + + +``` + +## VM Management + +libvirt uses `virsh` commands to manage VMs. After the StratoVirt platform is interconnected with libvirt, only the following commands for interaction with StratoVirt are supported: + +- `create`: creates a VM. + +- `suspend`: suspends a VM. + +- `resume`: resumes a VM. + +- `destroy`: destroys a VM. + +- `console`: logs in to a VM through the console. + +- `define`: creates a VM base on the XML configuration. + +- `undefine`: deletes a defined VM. + +- `start`: starts a stopped VM. + +> ![](./public_sys-resources/icon-note.gif)**NOTE** +> +> The interface for interconnecting with StratoVirt has been implemented in libvirt. However, the commands for stopping and restarting the VM is not supported. Use `virsh -c stratovirt:///system 'command'` to call the stratovirt_driver interface of libvirt, and `virsh -c stratovirt:///system` to access the virsh CLI. In this case, stratovirt_driver is connected. + +### VM Lifecycle Management + +If you have created a VM configuration file named **StratoVirt** in st.xml format, you can use the following commands for VM lifecycle management: + +- Create a VM. + + ```shell + virsh create st.xml + /// + virsh -c stratovirt:///system create st.xml + ``` + + After the VM is created, you can run the `virsh list` or `virsh -c stratovirt:///system list` command to check whether a VM named **StratoVirt** in the **running** status exists. + +- Create a VM base on the XML configuration. + + ```shell + virsh define st.xml + /// + virsh -c stratovirt:///system create st.xml + ``` + + After the VM is created, you can run the `virsh list` or `virsh -c stratovirt:///system list` command to check whether a VM named **StratoVirt** in the **shut off** status exists. + +- Start a stopped VM. + + ```shell + virsh start StratoVirt + /// + virsh -c stratovirt:///system start StratoVirt + ``` + + After the VM is started, you can run the `virsh list` or `virsh -c stratovirt:///system list` command to check whether the status of VM **StratoVirt** is changed from **shut off** to **running**. + +- Suspend a VM. + + ```shell + virsh suspend StratoVirt + /// + virsh -c stratovirt:///system suspend StratoVirt + ``` + + After the VM is suspended, you can run the `virsh list` or `virsh -c stratovirt:///system list` command to check whether the status of VM **StratoVirt** is **paused**. + +- Resume a VM. + + ```shell + virsh resume StratoVirt + /// + virsh -c stratovirt:///system resume StratoVirt + ``` + + After the VM is resumed, you can run the `virsh list` or `virsh -c stratovirt:///system list` command to check whether the status of VM **StratoVirt** is **running**. + +- Destroy a VM. + + ```shell + virsh destroy StratoVirt + /// + virsh -c stratovirt:///system destroy StratoVirt + ``` + + After the VM is destroyed, you can run the `virsh list` or `virsh -c stratovirt:///system list` command to check whether VM **StratoVirt** exists. + +- Delete a defined VM + + ```shell + virsh undefine StratoVirt + /// + virsh -c stratovirt:///system undefine StratoVirt + ``` + + After the defined VM is deleted, you can run the `virsh list` or `virsh -c stratovirt:///system list` command to check whether VM **StratoVirt** in the **shut off** status exists. If VM **StratoVirt** is in the **running** status, after it is destroyed, it no longer enters the **shut off** status. + +### VM Login + +After the VM is created, you can run the `virsh console` or `virsh -c stratovirt:///system console` command to log in to it. If the VM name is **StratoVirt**, run the following command: + +```shell +virsh console StratoVirt +/// +virsh -c stratovirt:///system console StratoVirt +``` + +> ![](./public_sys-resources/icon-note.gif)**NOTE** +> +> To use the `virsh console` command, set the redirection type of the console device to **pty** in the XML file. diff --git a/docs/en/virtualization/virtualization_platform/stratovirt/prepare_env.md b/docs/en/virtualization/virtualization_platform/stratovirt/prepare_env.md new file mode 100644 index 0000000000000000000000000000000000000000..00ae755ef50f021b5278b9294baf3fc09855256a --- /dev/null +++ b/docs/en/virtualization/virtualization_platform/stratovirt/prepare_env.md @@ -0,0 +1,164 @@ +# Environment Preparation + +## Usage + +- StratoVirt can run on VMs with the x86_64 or AArch64 processor architecture. +- You are advised to compile, debug, and deploy StratoVirt on openEuler 22.03 LTS. +- StratoVirt can run with non-root permissions. + +## Environment Requirements + +The following are required in the environment for running StratoVirt: + +- /dev/vhost-vsock device (for implementing MMIO) +- nmap tool +- Kernel and rootfs images + +## Preparing Devices and Tools + +- To run StratoVirt, the MMIO device must be implemented. Therefore, before running StratoVirt, ensure that the **/dev/vhost-vsock** device exists. + + Check whether the device exists. + + ```shell + $ ls /dev/vhost-vsock + /dev/vhost-vsock + ``` + + If the device does not exist, run the following command to generate it: + + ```shell + modprobe vhost_vsock + ``` + +- To use QMP commands, install the nmap tool first. After configuring the Yum source, run the following command to install the tool: + + ```shell + # yum install nmap + ``` + +## Preparing Images + +### Creating the Kernel Image + +StratoVirt of the current version supports only the PE kernel image of the x86_64 and AArch64 platforms. The kernel image in PE format can be generated by using the following method: + +1. Run the following commands to obtain the kernel source code of openEuler: + + ```shell + git clone https://gitee.com/openeuler/kernel.git + cd kernel + ``` + +2. Run the following command to check and switch to the kernel version openEuler-22.03-LTS: + + ```shell + git checkout openEuler-22.03-LTS + ``` + +3. Configure and compile the Linux kernel. You are advised to use the [recommended configuration file](https://gitee.com/openeuler/stratovirt/tree/master/docs/kernel_config)). Copy the file to the kernel directory, rename it to **.config**, and run the `make olddefconfig` command to update to the latest default configuration (otherwise, you may need to manually select options for subsequent compilation). Alternatively, you can run the following command to configure the kernel as prompted. The system may display a message indicating that specific dependencies are missing. Run the `yum install` command to install the dependencies as prompted. + + ```shell + make menuconfig + ``` + +4. Run the following command to create and convert the kernel image to the PE format. The converted image is **vmlinux.bin**. + + ```shell + make -j vmlinux && objcopy -O binary vmlinux vmlinux.bin + ``` + +5. If you want to use the kernel in bzImzge format on the x86 platform, run the following command: + + ```shell + make -j bzImage + ``` + +## Creating the Rootfs Image + +The rootfs image is a file system image. When StratoVirt is started, the ext4 image with **init** can be loaded. To create an ext4 rootfs image, perform the following steps: + +1. Prepare a file with a proper size (for example, create a file with the size of 10 GB in **/home**). + + ```shell + cd /home + dd if=/dev/zero of=./rootfs.ext4 bs=1G count=10 + ``` + +2. Create an empty ext4 file system on this file. + + ```shell + mkfs.ext4 ./rootfs.ext4 + ``` + +3. Mount the file image. Create the **/mnt/rootfs** directory and mount **rootfs.ext4** to the directory as user **root**. + + ```shell + $ mkdir /mnt/rootfs + # Return to the directory where the file system is created, for example, **/home**. + $ cd /home + $ sudo mount ./rootfs.ext4 /mnt/rootfs && cd /mnt/rootfs + ``` + +4. Obtain the latest alpine-mini rootfs of the corresponding processor architecture. + + - If the AArch64 processor architecture is used, you can get the latest rootfs from the [alpine](http://dl-cdn.alpinelinux.org/alpine/latest-stable/releases/). For example, alpine-minirootfs-3.21.0-aarch64.tar.gz, the reference commands are as follows: + + ```shell + wget http://dl-cdn.alpinelinux.org/alpine/latest-stable/releases/aarch64/alpine-minirootfs-3.21.0-aarch64.tar.gz + tar -zxvf alpine-minirootfs-3.21.0-aarch64.tar.gz + rm alpine-minirootfs-3.21.0-aarch64.tar.gz + ``` + + - If the x86_64 processor architecture is used, you can get the latest rootfs from the [alpine](http://dl-cdn.alpinelinux.org/alpine/latest-stable/releases/). For example, alpine-minirootfs-3.21.0-x86_64.tar.gz, the reference commands are as follows: + + ```shell + wget http://dl-cdn.alpinelinux.org/alpine/latest-stable/releases/x86_64/alpine-minirootfs-3.21.0-x86_64.tar.gz + tar -zxvf alpine-minirootfs-3.21.0-x86_64.tar.gz + rm alpine-minirootfs-3.21.0-x86_64.tar.gz + ``` + +5. Run the following commands to create a simple **/sbin/init** for the ext4 file image: + + ```shell + $ rm sbin/init; touch sbin/init && cat > sbin/init < /sys/bus/pci/devices/0000:03:00.0/driver/unbind + ``` + + Finally bind the PCI device to the vfio-pci driver. + + ```shell + lspci -ns 0000:03:00.0 |awk -F':| ' '{print $5" "$6}' > /sys/bus/pci/drivers/vfio-pci/new_id + ``` + + After the NIC is bound to the vfio-pci driver, the NIC information cannot be queried on the host. Only the PCI device information can be queried. + +## VFIO Device Passthrough + +### Introduction + +The VFIO is a user-mode device driver solution provided by the kernel. The VFIO driver can securely present capabilities such as device I/O, interrupt, and DMA to user space. After StratoVirt uses the VFIO device passthrough solution, the I/O performance of VMs is greatly improved. + +### Using VFIO Passthrough + +StratoVirt interconnects with libvirt to enable you to manage and configure VMs by modifying corresponding XML files. The following describes how to enable VFIO passthrough by modifying the XML file of a VM. + +**Step 1** Modify the XML file. + +(1) Run the following command on the host to query the CPU architecture information: + + ```shell + # uname -m + ``` + +(2) For the AArch64 and x86_64 architectures, [download](https://gitee.com/openeuler/stratovirt/tree/master/docs) the StratoVirt XML file **stratovirt_aarch64.xml** or **stratovirtvirt_x86.xml** and save it to any directory, for example, **/home**. + + ```shell + # cp stratovirt/docs/stratovirt_$arch.xml /home + ``` + +(3) Modify the VFIO configuration in the XML file based on the site requirements. **bus**, **slot**, and **function** specify the PCI device bound to the vfio-pci driver. The related configurations are as follows: + +```shell + + + + +
+ + +``` + +In the preceding example, the device type is PCI, and **managed='yes'** indicates that libvirt unbinds the PCI device from the host and rebinds it to the vfio-pci driver. In the**source** item, the **domain**, **bus**, **slot**, and **function** of the VFIO passthrough device are configured. + +**Step 2** Create and log in to a VM using the libvirt command line. + +```shell +# virsh create stratovirt_$arch.xml +# virsh list --all +Id Name State +-------------------- +1 StratoVirt running +# virsh console 1 +``` + +**Step 3** View and use the VFIO passthrough NIC on the VM. + +(1) Check the NIC information before configuration. + + ```shell + # ip a + 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 + link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 + inet 127.0.0.1/8 scope host lo + valid_lft forever preferred_lft forever + 2: enp1s0: mtu 1500 qdisc noop state DOWN group default qlen 1000 + link/ether 72:b8:51:9d:d1:27 brd ff:ff:ff:ff:ff:ff + ``` + +(2) Dynamically configure the IP address of the NIC. + + ```shell + # dhclient + ``` + +(3) Check whether the IP address is configured successfully. + + ```shell + # ip a + 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 + link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 + inet 127.0.0.1/8 scope host lo + valid_lft forever preferred_lft forever + 2: enp1s0: mtu 1500 qdisc mq state UP group default qlen 1000 + link/ether 72:b8:51:9d:d1:27 brd ff:ff:ff:ff:ff:ff + inet 192.168.1.3/16 brd 192.168.255.255 scope global dynamic enp1s0 + valid_lft 86453sec preferred_lft 86453sec + ``` + + The preceding command output indicates that the IP address 192.168.1.3 is successfully assigned and the VM can directly use the configured NIC. + + Note: If the passthrough NIC is not connected to a physical network, network information cannot be obtained. + +### Unbinding the VFIO Driver + +To unbind a passthrough NIC from a VM, log in to the host and run the following command to bind the NIC to the host again.**hinic** indicates the NIC driver type. + +```shell +# echo 0000:03:00.0 > /sys/bus/pci/drivers/vfio-pci/unbind +# echo 0000:03:00.0 > /sys/bus/pci/drivers/hinic/bind +``` + +Note: Before binding a VFIO driver, you can run the **ethtool -i enp0** command on the host to obtain the NIC driver type.**enp0** indicates the name of the corresponding NIC. + +## SR-IOV Passthrough + +### Introduction + +When VFIO passthrough is enabled, VMs can directly access hardware, but each device can be exclusively used by only one VM. The SR-IOV passthrough technology can virtualize a physical function (PF) into multiple virtual functions (VFs) and directly pass the VFs to different VMs. This technology increases the number of available devices. + +### Procedure + +**Step 1** Create multiple VFs. + +The **sriov_numvfs** file is used to describe the count of VFs provided by SR-IOV and is stored in **/sys/bus/pci/devices/domain\:bus\:slot.function/**. For example, for the device whose bus ID is 03, slot ID is 00, and function ID is 0 in the preceding example, you can run the following command to create four VFs: + +```shell +# echo 4 > /sys/bus/pci/devices/0000\:03\:00.0/sriov_numvfs +``` + +**Step 2** Verify that the VFs are successfully created. + +```shell +# lspci -v | grep "Eth" | grep 1822 +``` + +If the following information is displayed, four VFs 03:00.1, 03:00.2, 03:00.3, and 03:00.4 are successfully created: + +```shell +03:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45) +03:00.1 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) +03:00.2 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) +03:00.3 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) +03:00.4 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) +``` + +**Step 3** All the created VFs can be passed to VMs. The method for using an SR-IOV device is the same as that for using a common PCI device. diff --git a/docs/en/virtualization/virtualization_platform/stratovirt/vm_configuration.md b/docs/en/virtualization/virtualization_platform/stratovirt/vm_configuration.md new file mode 100644 index 0000000000000000000000000000000000000000..1a6edbdbc61d51936ba2fab15beb7a9dbd169076 --- /dev/null +++ b/docs/en/virtualization/virtualization_platform/stratovirt/vm_configuration.md @@ -0,0 +1,686 @@ +# VM Configuration + +## Overview + +With StratoVirt, you can use command line parameters to specify VM configurations. Alternatively, you can interconnect StratoVirt with libvirt and use XML files to configure VMs. This chapter describes the command-line configuration mode. + +> ![](./public_sys-resources/icon-note.gif)**NOTE** +> +> In this document, **/path/to/socket** indicates the socket file in the user-defined path. +> +> In openEuler 21.09 and later versions, JSON files are not supported. + +## Specifications + +StratoVirt supports lightweight and standard VMs. + +- Lightweight VMs use the lightweight microVM mainboard and the MMIO bus. +- Standard VMs support standard startup. They use the Q35 mainboard on x86 platforms, and the virt mainboard and PCI bus on AArch64 platforms. + +### Lightweight VMs + +- Number of VM CPUs: \[1, 254] +- VM memory size: \[128 MiB, 512 GiB]. The default memory size is 256 MiB. +- Number of VM disks (including hot plugged-in disks): \[0, 6] +- Number of VM NICs (including hot plugged-in NICs): \[0, 2] +- The VM console device supports only single way connection. +- If the host CPU architecture is x86_64, a maximum of 11 MMIO devices can be configured. However, you are advised to configure a maximum of two other devices except disks and NICs. On the AArch64 platform, a maximum of 160 MMIO devices can be configured. You are advised to configure a maximum of 12 other devices except disks and NICs. + +### Standard VMs + +- Number of VM CPUs: \[1, 254] +- VM memory size: \[128 MiB, 512 GiB]. The default memory size is 256 MiB. +- The VM console device supports only single way connection. +- Only one console device is supported. +- A maximum of 32 PCI devices are supported. +- PCI bus to which the PCI device is mounted: slot ID \[0, 32); function ID \[0, 8). + +## Minimal Configuration + +The minimum configuration for running StratoVirt is as follows: + +- Use the Linux kernel image in PE or bzImage format (x86_64 only). +- Set the rootfs image as the virtio-blk device and add it to kernel parameters. +- Use QMP to control StratoVirt. +- To use a serial port for login, add one to the kernel startup command line. The standard model on the AArch64 platform is ttyAMA0, and the model used in other scenarios is ttyS0. + +## Configuration Description + +### **Command Format** + +The format of the command configured by running cmdline is as follows: + +**$ /path/to/stratovirt** *- \[Parameter 1] \[Option]-\[Parameter 2] \[Option]...* + +### **Usage Instructions** + +1. To ensure that the socket required by QMP can be created, run the following command to clear the environment: + + ```shell + $rm [parameter] *[user-defined socket file path]* + ``` + +2. Run the cmdline command. + + ```shell + $/path/to/stratovirt - *[Parameter 1] [Parameter option] - [Parameter 2] [Parameter option]*... + ``` + +### Basic Parameters + +The following table lists the basic configuration information. + +| Parameter| Option| Description| +| ---------------- | ----------------------------------------------- | ------------------------------------------------------------ | +| -name | *VMname* | Configures the VM name (a string of 1 to 255 characters).| +| -kernel | /path/to/vmlinux.bin| Configures the kernel image.| +| -append | console=ttyS0 root=/dev/vda reboot=k panic=1 rw | Configures the kernel command line parameter. For lightweight VMs, **console** is fixed at **ttyS0** (irrelevant to the platform architecture). For the standard x86_64 virtualization platform, **console** is default to **ttyS0**. For the AArch64 platform, **console** is default to **ttyAMA0**. If the virtio-console device is configured but the serial port device is not configured, set **console** to **hvc0** (irrelevant to the architecture).| +| -initrd | /path/to/initrd.img | Configures the initrd file.| +| -smp | \[cpus=]n\[,maxcpus=,sockets=,dies=,clusters=,cores=,threads=]| **cpus** specifies the number of CPUs with the value range of \[1,254]. **maxcpus** specifies the maximum number of CPUs with the value range of \[1,254]. **sockets**, **dies**, **clusters**, **cores**, and **threads** specifies the number of sockets, dies, clusters, cores, and threads respectively. The values of **sockets**, **cores**, and **threads**, if not specified, depend on the value of **maxcpus**. The values satisfy the following relationship: **maxcpus**=**sockets** x **dies** x **clusters** x **cores** x **threads**.| +| -m | Memory size (MiB/GiB). The default unit is MiB.| Configures the memory size. The value range is \[128 MiB, 512 GiB]. The default memory size is 256 MiB.| +| -qmp | unix:/path/to/socket,server,nowait | Configures QMP. Before running QMP, ensure that the socket file does not exist.| +| -D | /path/to/logfile | Configures the log file.| +| -pidfile | /path/to/pidfile | Configures the pid file. This parameter must be used together with **-daemonize**. Ensure that the pid file does not exist before running the script.| +| -disable-seccomp | N/A| Disables Seccomp. Seccomp is enabled by default.| +| -daemonize | N/A| Enables daemon processes.| + +### VM Types + +You can run the **-machine** parameter to specify the type of the VM to be started. + +Parameters: + +- **type**: VM startup type. The value is **MicroVm** for lightweight virtualization, **q35** for standard virtualization on the x86_64 platform, and **virt** for standard virtualization on the AArch64 platform. +- **dump-guest-core** (optional): whether to dump the VM memory when a process panics. +- **mem-share** (optional): whether to share memory with other processes. + +### Disk Configuration + +VM disk configuration includes the following configuration items: + +- **drive_id**: disk ID. +- **path_on_host**: disk path. +- **serial_num** (optional): serial number of the disk. +- **read_only** (optional): whether the disk is read-only. +- **direct** (optional): whether to open the disk in O_DIRECT mode. +- **iothread** (optional): iothread attribute. +- **throttling.iops-total** (optional): disk QoS for limiting disk I/O operations. +- **if** (optional): driver type. The default value is **none**. The block device is **none**. +- **bus**: bus to which the device is to be mounted. +- **addr**: IDs of the slot and function to which the device is to be mounted. +- **multifunction** (optional): whether to enable PCI multi-function. +- **bootindex** (optional, only for standard machine): boot priority of the block device. If this parameter is not set, the priority is default to the lowest. The value ranges from 0 to 255. A smaller value indicates a higher priority. + +#### Disk Configuration Modes + +Disk configuration consists of two steps: driver configuration and block device configuration. + +The lightweight VM configuration format is as follows: + +```conf +-drive id=drive_id,file=path_on_host[,readonly=off][,direct=off][,throttling.iops-total=200][,if=none] +-device virtio-blk-device,drive=drive_id[,iothread=iothread1][,serial=serial_num] +``` + +The standard VM configuration format is as follows: + +```conf +-drive id=drive_id,file=path_on_host[,readonly=off][,direct=off][,throttling.iops-total=200][,if=none] +-device virtio-blk-pci,drive=drive_id,bus=pcie.0,addr=0x3.0x0[,iothread=iothread1,][serial=serial_num][,multifunction=on][,bootindex=1] +``` + +The following describes the **throttling.iops-total** and **iothread** configuration items: + +#### Disk QoS + +##### Introduction + +QoS is short for quality of service. In cloud scenarios, multiple VMs are started on a single host. Because the total disk access bandwidth of the host is limited, when a VM has heavy disk access pressure, it will occupy the access bandwidth of other VMs. As a result, the I/O performance of other VMs will be affected. To reduce the impact between VMs, you can configure QoS to limit the disk access rate of the VMs. + +##### Precautions + +- Currently, QoS supports the configuration of disk IOPS. +- The value range of IOPS is \[0, 1000000]. The value **0** indicates that the IOPS is not limited. The actual IOPS does not exceed the preset value or the upper limit of the actual backend disk performance. +- Only the average IOPS can be limited. Instantaneous burst traffic cannot be limited. + +##### Configuration Methods + +Usage: + +**CLI** + +```conf +-drive xxx,throttling.iops-total=200 +``` + +Parameters: + +- **throttling.iops-total**: I/O delivery speed of the disk on a VM after IOPS is configured. It does not exceed the value of this parameter. +- *xxx*: other settings of the disk. + +#### **iothread** + +For details about the iothread configuration, see [iothread Configuration](#iothread-configuration). + +### NIC Configuration + +VM NIC configuration includes the following configuration items: + +- **idv**: unique device ID. +- **tap**: tap device. +- **ifname**: name of the tap device on the host. +- **mac** (optional): MAC address of the VM. +- **iothread** (optional): iothread attribute of the disk. For details about the iothread configuration of the NIC, see [iothread Configuration](#iothread-configuration). + +#### Configuration Methods + +> ![](./public_sys-resources/icon-note.gif)**NOTE**: +> +> Before using the network, run the following commands to configure the host bridge and tap device: +> +> ```shell +> brctl addbr qbr0 +> ip tuntap add tap0 mode tap +> brctl addif qbr0 tap0 +> ifconfig qbr0 up; ifconfig tap0 up +> ifconfig qbr0 192.168.0.1 +> ``` + +1. Configure virtio-net. (\[] indicates an optional parameter.) + + Lightweight VMs: + + ```Conf + -netdev tap,id=netdevid,ifname=host_dev_name[,vhostfd=2] + -device virtio-net-device,netdev=netdevid,id=netid[,iothread=iothread1,mac=12:34:56:78:9A:BC] + ``` + + Standard VMs: + + ```Conf + -netdev tap,id=netdevid,ifname=host_dev_name[,vhostfd=2] + -device virtio-net-pci,netdev=netdevid,id=netid,bus=pcie.0,addr=0x2.0x0[,multifunction=on,iothread=iothread1,mac=12:34:56:78:9A:BC] + ``` + +2. Configure vhost-net. + + Lightweight VMs: + + ```conf + -netdev tap,id=netdevid,ifname=host_dev_name,vhost=on[,vhostfd=2] + -device virtio-net-device,netdev=netdevid,id=netid[,iothread=iothread1,mac=12:34:56:78:9A:BC] + ``` + + Standard VMs: + + ```conf + -netdev tap,id=netdevid,ifname=host_dev_name,vhost=on[,vhostfd=2] + -device virtio-net-pci,netdev=netdevid,id=netid,bus=pcie.0,addr=0x2.0x0[,multifunction=on,iothread=iothread1,mac=12:34:56:78:9A:BC] + ``` + +### chardev Configuration + +Redirect I/Os from the Guest to chardev on the host. The chardev backend type can be **stdio**, **pty**, **socket**, or **file**.**file** can be set only during output. The configuration items are as follows: + +- **id**: unique device ID. +- **backend**: redirection type. +- **path**: path of the device redirection file. This parameter is required only for **socket** and **file** devices. +- **server**: uses chardev as a server. This parameter is required only for **socket** devices. +- **nowait**: The expected status is disconnected. This parameter is required only for **socket** devices. + +When chardev is used, a console file is created and used. Therefore, ensure that the console file does not exist before starting StratoVirt. + +#### Configuration Methods + +```conf +-chardev backend,id=chardev_id[,path=path,server,nowait] +``` + +### Serial Port Configuration + +A serial port is a VM device used to transmit data between hosts and VMs. To use a serial port, configure **console** to **ttyS0** in the kernel command line, and to **ttyAMA0** for standard startup on the AArch64 platform. The configuration items are as follows: + +- **chardev**: redirected chardev device. +- **backend**, **path**, **server**, and **nowait**: The meanings of these parameters are the same as those in **chardev**. + +#### Configuration Methods + +```conf +-serial chardev:chardev_id +``` + +Or: + +```conf +-chardev backend[,path=path,server,nowait] +``` + +### Console Device Configuration + +virtio-console is a universal serial port device used for data transmission between hosts and VMs. If only the console device is configured and I/O operations are performed through the console device, set **console** to **hvc0** in the kernel startup parameters. The console device has the following configuration items: + +- **id**: device ID. +- **path**: path of virtio console files. +- **socket**: redirection in socket mode. +- **chardev**: redirected chardev device. + +#### Configuration Methods + +The console configuration consists of three steps: specify virtio-serial, create a character device, and then create a virtconsole device. + +Lightweight VMs: + +```conf +-device virtio-serial-device[,id=virtio-serial0] +-chardev socket,path=socket_path,id=virtioconsole1,server,nowait +-device virtconsole,chardev=virtioconsole1,id=console_id +``` + +Standard VMs: + +```conf +-device virtio-serial-pci,bus=pcie.0,addr=0x1.0x0[,multifunction=on,id=virtio-serial0] +-chardev socket,path=socket_path,id=virtioconsole1,server,nowait +-device virtconsole,chardev=virtioconsole1,id=console_id +``` + +### vsock Device Configuration + +The vsock is also a device for communication between hosts and VMs. It is similar to the console but has better performance. The configuration items are as follows: + +- **id**: unique device ID. +- **guest_cid**: unique context ID. + +#### Configuration Methods + +Lightweight VMs: + +```conf +-device vhost-vsock-device,id=vsock_id,guest-cid=3 +``` + +Standard VMs: + +```conf +-device vhost-vsock-pci,id=vsock_id,guest-cid=3,bus=pcie.0,addr=0x1.0x0[,multifunction=on] +``` + +### Memory Huge Page Configuration + +#### Introduction + +StratoVirt supports the configuration of huge pages for VMs. Compared with the traditional 4 KB memory page mode, huge page memory can effectively reduce the number of TLB misses and page fault interrupts, significantly improving the performance of memory-intensive services. + +#### Precautions + +- The directory to which the huge pages are mounted must be an absolute path. +- Memory huge pages can be configured only during startup. +- Only static huge pages are supported. +- Configure huge pages on the host before use. +- To use the huge page feature, ensure that the VM memory size is an integer multiple of *huge page size*. + +#### Mutually Exclusive Features + +- If the huge page feature is configured, the balloon feature does not take effect. + +#### Configuration Methods + +##### Configuring Huge Pages on the Host + +###### Mounting + +Mount the huge page file system to a specified directory. `/path/to/hugepages` is the user-defined empty directory. + +```shell +mount -t hugetlbfs hugetlbfs /path/to/hugepages +``` + +###### Setting the Number of Huge Pages + +- Set the number of static huge pages. `num` indicates the specified number. + + ```shell + sysctl vm.nr_hugepages=num + ``` + +- Query huge page statistics. + + ```shell + cat /proc/meminfo | grep Hugepages + ``` + + To view statistics about huge pages of other sizes, view the related information in the `/sys/kernel/mm/hugepages/hugepages-*/` directory. + +> ![](./public_sys-resources/icon-note.gif)**NOTE**: +> +> Configure the StratoVirt memory specifications and huge pages based on the huge page usage. If the huge page resources are insufficient, the VM fails to be started. + +#### Adding Huge Page Configuration When Starting StratoVirt + +- CLI + + ```shell + -mem-path /page/to/hugepages + ``` + + In the preceding command, `/page/to/hugepages` indicates the directory to which the huge page file system is mounted. Only absolute paths are supported. + +
+ +> ![](./public_sys-resources/icon-note.gif)**NOTE**: +> +> **Typical configuration**: Set **mem-path** in the StratoVirt command line to the *huge page file system mount directory*. The StratoVirt huge page feature is recommended for the typical configuration. + +### iothread Configuration + +#### Introduction + +After a VM with the iothread configuration is started on StratoVirt, threads independent of the main thread are started on the host. These independent threads can be used to process I/O requests of devices, improving the device I/O performance and reducing the impact on message processing on the management plane. + +#### Precautions + +- A maximum of eight iothreads can be configured. +- The iothread attribute can be configured for disks and NICs. +- iothreads occupy CPU resources of the host. When the I/O pressure is high in a VM, the CPU resources occupied by a single iothread depend on the disk access speed. For example, a common SATA disk occupies less than 20% CPU resources. + +#### Creating an iothread + +**CLI** + +```shell +-object iothread,id=iothread1 -object iothread,id=iothread2 +``` + +Parameters: + +- **id**: identifies an iothread. This ID can be set to the iothread attribute of the disk or NIC. If iothread is configured in the startup parameter, the thread with the specified ID is started on the host after the VM is started. + +#### Configuring the iothread Attribute for a Disk or NIC + +**CLI-based configurations** + +Lightweight VMs: + +Disks + +```conf +-device virtio-blk-device xxx,iothread=iothread1 +``` + +NICs + +```conf +-device virtio-net-device xxx,iothread=iothread2 +``` + +Standard VMs: + +Disks + +```conf +-device virtio-blk-pci xxx,iothread=iothread1 +``` + +NICs + +```conf +-device virtio-net-pci xxx,iothread=iothread2 +``` + +Parameters: + +1. **iothread**: Set this parameter to the iothread ID, indicating the thread that processes the I/O of the device. +2. *xxx*: other configurations of the disk or NIC. + +### Balloon Device Configuration + +#### Introduction + +During running of a VM, the balloon driver in it occupies or releases memory to dynamically adjust the VM's available memory, achieving memory elasticity. + +#### Precautions + +- Before enabling balloon, ensure that the page size of the guest is the same as that of the host. +- The balloon feature must be enabled for the guest kernel. +- When memory elastic scaling is enabled, slight frame freezing may occur in the VM and the memory performance may deteriorate. + +#### Mutually Exclusive Features + +- This feature is mutually exclusive with huge page memory. +- In the x86 architecture, the number of interrupts is limited. Therefore, the total number of balloon devices and other virtio devices cannot exceed 11. By default, six block devices, two net devices, and one serial port device are used. + +#### Specifications + +- Each VM can be configured with only one balloon device. + +#### Configuration Methods + +Lightweight VMs: + +```conf +-device virtio-balloon-device[,deflate-on-oom=true|false][,free-page-reporting=true|false] +``` + +Standard VMs: + +```conf +-device virtio-balloon-pci,bus=pcie.0,addr=0x4.0x0[,deflate-on-oom=true|false][,free-page-reporting=true|false][,multifunction=on|off] +``` + +![](./public_sys-resources/icon-note.gif)**NOTE** + +1. The value of **deflate-on-oom** is of the Boolean type, indicating whether to enable the auto deflate feature. When this feature is enabled, if the balloon device has reclaimed some memory, it automatically releases the memory to the guest when the guest requires the memory. If this feature is disabled, the memory is not automatically returned. +2. The value of **free-page-reporting** is of the Boolean type, indicating whether to release free pages from kernel reporting. This feature can be used to reuse memory. +3. When running the QMP command to reclaim the VM memory, ensure that the VM has sufficient memory to keep basic running. Otherwise, some operations may time out and the VM cannot apply for idle memory. +4. If the huge page feature is enabled in the VM, the balloon device cannot reclaim the memory occupied by the huge pages. + +> If **deflate-on-oom** is set to **false**, when the guest memory is insufficient, the balloon device does not automatically release the memory. As a result, the guest OOM may occur, the processes may be killed, and even the VM cannot run properly. + +### RNG Configuration + +#### Introduction + +Virtio RNG is a paravirtualized random number generator that generates hardware random numbers for the guest. + +#### Configuration Methods + +Virtio RNG can be configured as the Virtio MMIO device or Virtio PCI device. To configure the Virtio RNG device as a Virtio MMIO device, run the following command: + +```conf +-object rng-random,id=objrng0,filename=/path/to/random_file +-device virtio-rng-device,rng=objrng0,max-bytes=1234,period=1000 +``` + +To configure the Virtio RNG device as a Virtio PCI device, run the following command: + +```conf +-object rng-random,id=objrng0,filename=/path/to/random_file +-device virtio-rng-pci,rng=objrng0,max-bytes=1234,period=1000,bus=pcie.0,addr=0x1.0x0,id=rng-id[,multifunction=on] +``` + +Parameters: + +- **filename**: path of the character device used to generate random numbers on the host, for example, **/dev/random**. +- **period**: period for limiting the read rate of random number characters, in milliseconds. +- **max-bytes**: maximum number of bytes of a random number generated by a character device within a period. +- **bus**: name of the bus to which the Virtio RNG device is mounted. +- **addr**: address of the Virtio RNG device. The parameter format is **addr=***slot.function*, where *slot* and *function* indicate the slot number and function number of the device respectively. The slot number and function number are hexadecimal numbers. The function number of the Virtio RNG device is **0x0**. + +#### Precautions + +- If **period** and **max-bytes** are not configured, the read rate of random number characters is not limited. +- Otherwise, the value range of **max-bytes/period\*1000** is \[64, 1000000000]. It is recommended that the value be not too small to prevent the rate of obtaining random number characters from being too slow. +- Only the average number of random number characters can be limited, and the burst traffic cannot be limited. +- If the guest needs to use the Virtio RNG device, the guest kernel requires the following configurations: **CONFIG_HW_RANDOM=y**, **CONFIG_HW_RANDOM_VIA=y**, and **CONFIG_HW_RANDOM_VIRTIO=y**. +- When configuring the Virtio RNG device, check whether the entropy pool is sufficient to avoid VM freezing. For example, if the character device path is **/dev/random**, you can check **/proc/sys/kernel/random/entropy_avail** to view the current entropy pool size. When the entropy pool is full, the entropy pool size is **4096**. Generally, the value is greater than 1000. + +### VNC Configuration + +#### Introduction + +You can log in to the VM through the VNC client and interact with the remote VM system through mouse and keyboard operations on the on VNC desktop. + +#### Precautions + +- Only standard VMs support VNC. +- Only clients that use RFB of version 3.3 to 3.8 can connect to VNC. +- Only one client can be connected at the same time. Follow-up clients connections will result in failure. +- The VNC feature supports only the ARM architecture. + +#### Mutually Exclusive Features + +- The VNC feature and live migration are mutually exclusive. + +#### Specifications + +- Only one VNC server can be configured on each VM. + +#### Configuration Methods + +Standard VMs: + +```shell +-vnc 0.0.0.0:11 +``` + +![](./public_sys-resources/icon-note.gif)**NOTE** + +1. The Pixman library is required for rendering graphics. Install the **pixman.rpm** and **pixman-devel.rpm** packages of the Pixman library on the VM. +2. A USB controller, a mouse and a keyboard are required to perform mouse and keyboard operations. +3. A display device needs to be configured, such as virtio-gpu and ramfb. + +### USB Keyboard and Mouse Configuration + +#### Introduction + +StratoVirt supports USB keyboards and mice. You can remotely connect to the VM through VNC and operate in the VM GUI using a USB keyboard and mouse. Because USB devices are mounted to the USB controller, you need to configure a USB controller in the CLI in advance. + +#### Precautions + +- Only standard VMs support USB keyboards and mice. + +#### Mutually Exclusive Features + +- USB keyboards and mice and live migration are mutually exclusive. + +#### Specifications + +- Only one USB controller, one USB keyboard, and one USB mouse can be configured for each VM. + +#### Configuration Methods + +Add the following option to the StratoVirt startup command to configure the USB controller: + +```conf +-device nec-usb-xhci,id=xhci,bus=pcie.0,addr=0xa.0x0 +``` + +Parameters: + +- `id`: unique ID of the device +- `bus`: bus to mount the device +- `addr`: slot and function numbers for mounting the device + +The configured `bus` and `addr` values cannot conflict with other PCI devices. Otherwise, the VM will fail to start. + +Add the following option to the StratoVirt startup command to configure the USB keyboard: + +```conf +-device usb-bkd,id=kbd +``` + +Parameters: + +- `id`: unique ID of the device + +Add the following option to the StratoVirt startup command to configure the USB mouse: + +```conf +-device usb-tablet,id=tablet +``` + +Parameters: + +- `id`: unique ID of the device + +### virtio-gpu Device Configuration + +#### Introduction + +virtio-gpu devices can be configured for standard VMs for graphics display. + +#### Precautions + +- Only 2D graphics can be displayed. +- Recommended `max_hostmem` (usable memory on the host) is equal to or larger than 256MiB to ensure enough resolution. +- `max_outputs` (number of screens) cannot exceed 16. +- Live migration is not supported. + +#### Specifications + +- Only one virtio-gpu device can be configured for each VM. + +#### Configuration Methods + +Standard VMs: + +```conf +-device virtio-gpu-pci,id=XX,bus=pcie.0,addr=0x2.0x0[,max_outputs=XX][,edid=true|false][,xres=XX][,yres=XX][,max_hostmem=XX] +``` + +Parameters: + +- `max_outputs`: number of screens to be supported by the device. The recommended value is **1**, the maximum value is 16. +- `edid`: whether EDID is supported by the device. The recommended value is **true**. The VM kernel will verify whether the device supports EDID. +- `xres/yres`: horizontal/vertical size of the login window. +- `max_hostmem`: usable memory for the device on the host in bytes + +## Configuration Examples + +### Lightweight VMs + +This section provides an example of the minimum configuration for creating a lightweight VM. + +1. Log in to the host and delete the socket file to ensure that the QMP can be created. + + ```shell + rm -f /tmp/stratovirt.socket + ``` + +2. Run StratoVirt. + + ```shell + $ /path/to/stratovirt \ + -kernel /path/to/vmlinux.bin \ + -append console=ttyS0 root=/dev/vda rw reboot=k panic=1 \ + -drive file=/home/rootfs.ext4,id=rootfs,readonly=false \ + -device virtio-blk-device,drive=rootfs \ + -qmp unix:/tmp/stratovirt.socket,server,nowait \ + -serial stdio + ``` + + After the running is successful, the VM is created and started based on the specified configuration parameters. + +### Standard VMs + +This section provides an example of the minimum configuration for creating a standard VM on the ARM platform. + +1. Delete the socket file to ensure that QMP can be created. + + ```shell + rm -f /tmp/stratovirt.socket + ``` + +2. Run StratoVirt. + + ```shell + $ /path/to/stratovirt \ + -kernel /path/to/vmlinux.bin \ + -append console=ttyAMA0 root=/dev/vda rw reboot=k panic=1 \ + -drive file=/path/to/edk2/code_storage_file,if=pflash,unit=0[,readonly=true] \ + -drive file=/path/to/edk2/data_storage_file,if=pflash,unit=1, \ + -drive file=/home/rootfs.ext4,id=rootfs,readonly=false \ + -device virtio-blk-device,drive=rootfs,bus=pcie.0,addr=0x1 \ + -qmp unix:/tmp/stratovirt.socket,server,nowait \ + -serial stdio + ``` diff --git a/docs/en/virtualization/virtualization_platform/stratovirt/vm_management.md b/docs/en/virtualization/virtualization_platform/stratovirt/vm_management.md new file mode 100644 index 0000000000000000000000000000000000000000..453a42050f3691d347a94ee98c53d6f4dd0049e1 --- /dev/null +++ b/docs/en/virtualization/virtualization_platform/stratovirt/vm_management.md @@ -0,0 +1,733 @@ +# VM Management + +## Overview + +StratoVirt allows you to query VM information and manage VM resources and lifecycle with QMP. To query the information about a VM, connect to the VM first. + +## Querying VM Information + +### Introduction + +StratoVirt can be used to query the VM status, vCPU topology, and vCPU online status. + +### Querying VM Status + +Run the **query-status** command to query the running status of a VM. + +- Usage: + + **{ "execute": "query-status" }** + +- Example: + +```shell +<- { "execute": "query-status" } +-> { "return": { "running": true,"singlestep": false,"status": "running" } +``` + +### Querying Topology Information + +Run the **query-cpus** command to query the topologies of all CPUs. + +- Usage: + + **{ "execute": "query-cpus" }** + +- Example: + +```shell +<- { "execute": "query-cpus" } +-> {"return":[{"CPU":0,"arch":"x86","current":true,"halted":false,"props":{"core-id":0,"socket-id":0,"thread-id":0},"qom_path":"/machine/unattached/device[0]","thread_id":8439},{"CPU":1,"arch":"x86","current":true,"halted":false,"props":{"core-id":0,"socket-id":1,"thread-id":0},"qom_path":"/machine/unattached/device[1]","thread_id":8440}]} +``` + +### Querying vCPU Online Status + +Run the **query-hotpluggable-cpus** command to query the online/offline statuses of all vCPUs. + +- Usage: + +**{ "execute": "query-hotpluggable-cpus" }** + +- Example: + +```shell +<- { "execute": "query-hotpluggable-cpus" } +-> {"return":[{"props":{"core-id":0,"socket-id":0,"thread-id":0},"qom-path":"/machine/unattached/device[0]","type":"host-x86-cpu","vcpus-count":1},{"props":{"core-id":0,"socket-id":1,"thread-id":0},"qom-path":"/machine/unattached/device[1]","type":"host-x86-cpu","vcpus-count":1}]} +``` + +Online vCPUs have the `qom-path` item, while offline vCPUs do not. + +## Managing VM Lifecycle + +### Introduction + +StratoVirt can manage the lifecycle of a VM, including starting, stopping, resuming, and exiting the VM. + +### Creating and Starting a VM + +Use the command line parameters to specify the VM configuration, and create and start a VM. + +- When using the command line parameters to specify the VM configuration, run the following command to create and start the VM: + +```shell +$/path/to/stratovirt - *[Parameter 1] [Parameter option] - [Parameter 2] [Parameter option]*... +``` + +> ![](./public_sys-resources/icon-note.gif) +> +> After the lightweight VM is started, there are two NICs: eth0 and eth1. The two NICs are reserved for hot plugging: eth0 first and then eth1. Currently, only two virtio-net NICs can be hot plugged. + +### Connecting to a VM + +StratoVirt uses QMP to manage VMs. To stop, resume, or exit a VM, connect it the StratoVirt through QMP first. + +Open a new CLI (CLI B) on the host and run the following command to connect to the api-channel as the **root** user: + +```shell +# ncat -U /path/to/socket +``` + +After the connection is set up, you will receive a greeting message from StratoVirt, as shown in the following: + +```shell +{"QMP":{"version":{"qemu":{"micro":1,"minor":0,"major":4},"package":""},"capabilities":[]}} +``` + +You can now manage the VM by entering the QMP commands in CLI B. + +> ![](./public_sys-resources/icon-note.gif) +> +> QMP provides **stop**, **cont**, **quit**, and **query-status** commands to manage and query VM statuses. +> +> All QMP commands for managing VMs are entered in CLI B. `<-` indicates the command input, and `->` indicates the QMP returned result. + +### Stopping a VM + +QMP provides the **stop** command to stop a VM, that is, to stop all vCPUs of the VM. The command syntax is as follows: + +**{"execute":"stop"}** + +**Example:** + +The **stop** command and the command output are as follows: + +```shell +<- {"execute":"stop"} +-> {"event":"STOP","data":{},"timestamp":{"seconds":1583908726,"microseconds":162739}} +-> {"return":{}} +``` + +### Resuming a VM + +QMP provides the **cont** command to resume a stopped VM, that is, to resume all vCPUs of the VM. The command syntax is as follows: + +**{"execute":"cont"}** + +**Example:** + +The **cont** command and the command output are as follows: + +```shell +<- {"execute":"cont"} +-> {"event":"RESUME","data":{},"timestamp":{"seconds":1583908853,"microseconds":411394}} +-> {"return":{}} +``` + +### Exiting a VM + +QMP provides the **quit** command to exit a VM, that is, to exit the StratoVirt process. The command syntax is as follows: + +**{"execute":"quit"}** + +**Example:** + +```shell +<- {"execute":"quit"} +-> {"return":{}} +-> {"event":"SHUTDOWN","data":{"guest":false,"reason":"host-qmp-quit"},"timestamp":{"ds":1590563776,"microseconds":519808}} +``` + +## Managing VM Resources + +### Hot-Pluggable Disks + +StratoVirt allows you to adjust the number of disks when a VM is running. That is, you can add or delete VM disks without interrupting services. + +**Note** + +- For a standard VM, the **CONFIG_HOTPLUG_PCI_PCIE=y** configuration must be enabled for the VM kernel. + +- For a standard VM, devices can be hot added to the root port. The root port device must be configured before the VM is started. + +- You are not advised to hot swap a device when the VM is being started, stopped, or under high internal pressure. Otherwise, the VM may become abnormal because the drivers on the VM cannot respond in a timely manner. + +#### Hot Adding Disks + +**Usage:** + +Lightweight VM: + +```shell +{"execute": "blockdev-add", "arguments": {"node-name": "drive-0", "file": {"driver": "file", "filename": "/path/to/block"}, "cache": {"direct": true}, "read-only": false}} +{"execute": "device_add", "arguments": {"id": "drive-0", "driver": "virtio-blk-mmio", "addr": "0x1"}} +``` + +Standard VM: + +```shell +{"execute": "blockdev-add", "arguments": {"node-name": "drive-0", "file": {"driver": "file", "filename": "/path/to/block"}, "cache": {"direct": true}, "read-only": false}} +{"execute":"device_add", "arguments":{"id":"drive-0", "driver":"virtio-blk-pci", "drive": "drive-0", "addr":"0x0", "bus": "pcie.1"}} +``` + +**Parameters:** + +- For a lightweight VM, the value of **node-name** in **blockdev-add** must be the same as that of **id** in **device_add**. For example, the values of **node-name** and **id** are both **drive-0** as shown above. + +- For a standard VM, the value of **drive** must be the same as that of **node-name** in **blockdev-add**. + +- **/path/to/block** is the image path of the hot added disks. It cannot be the path of the disk image that boots the rootfs. + +- For a lightweight VM, the value of **addr**, starting from **0x0**, is mapped to a virtio device on the VM. **0x0** is mapped to **vda**, **0x1** is mapped to **vdb**, and so on. To be compatible with the QMP protocol, **addr** can be replaced by **lun**, but **lun=0** is mapped to the **vdb** of the guest machine. For a standard VM, the value of **addr** must be **0x0**. + +- For a standard VM, **bus** indicates the name of the bus to mount the device. Currently, the device can be hot added only to the root port device. The value of **bus** must be the ID of the root port device. + +- For a lightweight VM, StratoVirt supports a maximum of six virtio-blk disks. Note this when hot adding disks. For a standard VM, the maximum number of hot added disks depends on the number of root port devices. + +**Example:** + +Lightweight VM: + +```shell +<- {"execute": "blockdev-add", "arguments": {"node-name": "drive-0", "file": {"driver": "file", "filename": "/path/to/block"}, "cache": {"direct": true}, "read-only": false}} +-> {"return": {}} +<- {"execute": "device_add", "arguments": {"id": "drive-0", "driver": "virtio-blk-mmio", "addr": "0x1"}} +-> {"return": {}} +``` + +Standard VM: + +```shell +<- {"execute": "blockdev-add", "arguments": {"node-name": "drive-0", "file": {"driver": "file", "filename": "/path/to/block"}, "cache": {"direct": true}, "read-only": false}} +-> {"return": {}} +<- {"execute":"device_add", "arguments":{"id":"drive-0", "driver":"virtio-blk-pci", "drive": "drive-0", "addr":"0x0", "bus": "pcie.1"}} +-> {"return": {}} +``` + +#### Hot Removing Disks + +**Usage:** + +Lightweight VM: + +```shell +{"execute": "device_del", "arguments": {"id":"drive-0"}} +``` + +Standard VM: + +```shell +{"execute": "device_del", "arguments": {"id":"drive-0"}} +{"execute": "blockdev-del", "arguments": {"node-name": "drive-0"}} +``` + +**Parameters:** + +**id** indicates the ID of the disk to be hot removed. + +- **node-name** indicates the backend name of the disk. + +**Example:** + +Lightweight VM: + +```shell +<- {"execute": "device_del", "arguments": {"id": "drive-0"}} +-> {"event":"DEVICE_DELETED","data":{"device":"drive-0","path":"drive-0"},"timestamp":{"seconds":1598513162,"microseconds":367129}} +-> {"return": {}} +``` + +Standard VM: + +```shell +<- {"execute": "device_del", "arguments": {"id":"drive-0"}} +-> {"return": {}} +-> {"event":"DEVICE_DELETED","data":{"device":"drive-0","path":"drive-0"},"timestamp":{"seconds":1598513162,"microseconds":367129}} +<- {"execute": "blockdev-del", "arguments": {"node-name": "drive-0"}} +-> {"return": {}} +``` + +A **DEVICE_DELETED** event indicates that the device is removed from StratoVirt. + +### Hot-Pluggable NICs + +StratoVirt allows you to adjust the number of NICs when a VM is running. That is, you can add or delete VM NICs without interrupting services. + +**Note** + +- For a standard VM, the **CONFIG_HOTPLUG_PCI_PCIE=y** configuration must be enabled for the VM kernel. + +- For a standard VM, devices can be hot added to the root port. The root port device must be configured before the VM is started. + +- You are not advised to hot swap a device when the VM is being started, stopped, or under high internal pressure. Otherwise, the VM may become abnormal because the drivers on the VM cannot respond in a timely manner. + +#### Hot Adding NICs + +**Preparations (Requiring the root Permission)** + +1. Create and enable a Linux bridge. For example, if the bridge name is **qbr0**, run the following command: + + ```shell + # brctl addbr qbr0 + # ifconfig qbr0 up + ``` + +2. Create and enable a tap device. For example, if the tap device name is **tap0**, run the following command: + + ```shell + # ip tuntap add tap0 mode tap + # ifconfig tap0 up + ``` + +3. Add the tap device to the bridge. + + ```shell + # brctl addif qbr0 tap0 + ``` + +**Usage:** + +Lightweight VM: + +```shell +{"execute":"netdev_add", "arguments":{"id":"net-0", "ifname":"tap0"}} +{"execute":"device_add", "arguments":{"id":"net-0", "driver":"virtio-net-mmio", "addr":"0x0"}} +``` + +Standard VM: + +```shell +{"execute":"netdev_add", "arguments":{"id":"net-0", "ifname":"tap0"}} +{"execute":"device_add", "arguments":{"id":"net-0", "driver":"virtio-net-pci", "addr":"0x0", "netdev": "net-0", "bus": "pcie.1"}} +``` + +**Parameters:** + +- For a lightweight VM, **id** in **netdev_add** must be the same as that in **device_add**. **ifname** is the name of the backend tap device. + +- For a standard VM, the value of **netdev** must be the value of **id** in **netdev_add**. + +- For a lightweight VM, the value of **addr**, starting from **0x0**, is mapped to an NIC on the VM. **0x0** is mapped to **eth0**, **0x1** is mapped to **eth1**. For a standard VM, the value of **addr** must be **0x0**. + +- For a standard VM, **bus** indicates the name of the bus to mount the device. Currently, the device can be hot added only to the root port device. The value of **bus** must be the ID of the root port device. + +- For a lightweight VM, StratoVirt supports a maximum of two virtio-net NICs. Therefore, pay attention to the specification restrictions when hot adding in NICs. For a standard VM, the maximum number of hot added disks depends on the number of root port devices. + +**Example:** + +Lightweight VM: + +```shell +<- {"execute":"netdev_add", "arguments":{"id":"net-0", "ifname":"tap0"}} +-> {"return": {}} +<- {"execute":"device_add", "arguments":{"id":"net-0", "driver":"virtio-net-mmio", "addr":"0x0"}} +-> {"return": {}} +``` + +**addr:0x0** corresponds to **eth0** in the VM. + +Standard VM: + +```shell +<- {"execute":"netdev_add", "arguments":{"id":"net-0", "ifname":"tap0"}} +-> {"return": {}} +<- {"execute":"device_add", "arguments":{"id":"net-0", "driver":"virtio-net-pci", "addr":"0x0", "netdev": "net-0", "bus": "pcie.1"}} +-> {"return": {}} +``` + +#### Hot Removing NICs + +**Usage:** + +Lightweight VM: + +```shell +{"execute": "device_del", "arguments": {"id": "net-0"}} +``` + +Standard VM: + +```shell +{"execute": "device_del", "arguments": {"id":"net-0"}} +{"execute": "netdev_del", "arguments": {"id": "net-0"}} +``` + +**Parameters:** + +**id**: NIC ID, for example, **net-0**. + +- **id** in **netdev_del** indicates the backend name of the NIC. + +**Example:** + +Lightweight VM: + +```shell +<- {"execute": "device_del", "arguments": {"id": "net-0"}} +-> {"event":"DEVICE_DELETED","data":{"device":"net-0","path":"net-0"},"timestamp":{"seconds":1598513339,"microseconds":97310}} +-> {"return": {}} +``` + +Standard VM: + +```shell +<- {"execute": "device_del", "arguments": {"id":"net-0"}} +-> {"return": {}} +-> {"event":"DEVICE_DELETED","data":{"device":"net-0","path":"net-0"},"timestamp":{"seconds":1598513339,"microseconds":97310}} +<- {"execute": "netdev_del", "arguments": {"id": "net-0"}} +-> {"return": {}} +``` + +A **DEVICE_DELETED** event indicates that the device is removed from StratoVirt. + +### Hot-swappable Pass-through Devices + +You can add or delete the passthrough devices of a StratoVirt standard VM when it is running. + +**Note** + +- The **CONFIG_HOTPLUG_PCI_PCIE=y** configuration must be enabled for the VM kernel. + +- Devices can be hot added to the root port. The root port device must be configured before the VM is started. + +- You are not advised to hot swap a device when the VM is being started, stopped, or under high internal pressure. Otherwise, the VM may become abnormal because the drivers on the VM cannot respond in a timely manner. + +#### Hot Adding Pass-through Devices + +**Usage:** + +```shell +{"execute":"device_add", "arguments":{"id":"vfio-0", "driver":"vfio-pci", "bus": "pcie.1", "addr":"0x0", "host": "0000:1a:00.3"}} +``` + +**Parameters:** + +- **id** indicates the ID of the hot added device. + +- **bus** indicates the name of the bus to mount the device. + +- **addr** indicates the slot and function numbers to mount the device. Currently, **addr** must be set to **0x0**. + +- **host** indicates the domain number, bus number, slot number, and function number of the passthrough device on the host machine. + +**Example:** + +```shell +<- {"execute":"device_add", "arguments":{"id":"vfio-0", "driver":"vfio-pci", "bus": "pcie.1", "addr":"0x0", "host": "0000:1a:00.3"}} +-> {"return": {}} +``` + +#### Hot Removing Pass-through Devices + +**Usage:** + +```shell +{"execute": "device_del", "arguments": {"id": "vfio-0"}} +``` + +**Parameters:** + +- **id** indicates the ID of the device to be hot removed, which is specified when the device is hot added. + +**Example:** + +```shell +<- {"execute": "device_del", "arguments": {"id": "vfio-0"}} +-> {"return": {}} +-> {"event":"DEVICE_DELETED","data":{"device":"vfio-0","path":"vfio-0"},"timestamp":{"seconds":1614310541,"microseconds":554250}} +``` + +A **DEVICE_DELETED** event indicates that the device is removed from StratoVirt. + +## Using Ballon Devices + +The balloon device is used to reclaim idle memory from a VM. It called by running the QMP command. + +**Usage:** + +```shell +{"execute": "balloon", "arguments": {"value": 2147483648}} +``` + +**Parameters:** + +- **value**: size of the guest memory to be set. The unit is byte. If the value is greater than the memory value configured during VM startup, the latter is used. + +**Example:** + +The memory size configured during VM startup is 4 GiB. If the idle memory of the VM queried by running the free command is greater than 2 GiB, you can run the QMP command to set the guest memory size to 2147483648 bytes. + +```shell +<- {"execute": "balloon", "arguments": {"value": 2147483648}} +-> {"return": {}} +``` + +Query the actual memory of the VM: + +```shell +<- {"execute": "query-balloon"} +-> {"return":{"actual":2147483648}} +``` + +## Using VM Memory Snapshots + +### Introduction + +A VM memory snapshot stores the device status and memory information of a VM in a snapshot file. If the VM is damaged, you can use the snapshot to restore it to the time when the snapshot was created, improving system reliability. + +StratoVirt allows you to create snapshots for stopped VMs and create VMs in batches with a snapshot file as the VM template. As long as a snapshot is created after a VM is started and enters the user mode, the quick startup can skip the kernel startup and user-mode service initialization phases and complete the VM startup in milliseconds. + +### Mutually Exclusive Features + +Memory snapshots cannot be created or used for VMs that are configured with the following devices or use the following features: + +- vhost-net device +- VFIO passthrough device +- Balloon device +- Huge page memory feature +- mem-shared feature +- memory backend file **mem-path** + +### Creating a Snapshot + +For StratoVirt VMs, perform the following steps to create a storage snapshot: + +1. Create and start a VM. + +2. Run the QMP command on the host to stop the VM. + + ```shell + <- {"execute":"stop"} + -> {"event":"STOP","data":{},"timestamp":{"seconds":1583908726,"microseconds":162739}} + -> {"return":{}} + ``` + +3. Confirm that the VM is stopped. + + ```shell + <- {"execute":"query-status"} + -> {"return":{"running":true,"singlestep":false,"status":"paused"}} + ``` + +4. Run the following QMP command to create a VM snapshot in a specified absolute path, for example, **/path/to/template**: + + ```shell + <- {"execute":"migrate", "arguments":{"uri":"file:/path/to/template"}} + -> {"return":{}} + ``` + +5. Check whether the snapshot is successfully created. + + ```shell + <- {"execute":"query-migrate"} + ``` + + If "{"return":{"status":"completed"}}" is displayed, the snapshot is successfully created. + + If the snapshot is created successfully, the `memory` and `state` directories are generated in the specified path **/path/to/template**. The `state` file contains VM device status information, and the `memory` file contains VM memory data. The size of the `memory` file is close to the configured VM memory size. + +### Querying Snapshot Status + +There are five statuses in the snapshot process. + +- `None`: The snapshot resource is not ready. +- `Setup`: The snapshot resource is ready. You can create a snapshot. +- `Active`: The snapshot is being created. +- `Completed`: The snapshot is created successfully. +- `Failed`: The snapshot fails to be created. + +You can run the `query-migrate` QMP command on the host to query the status of the current snapshot. For example, if the VM snapshot is created successfully, the following output is displayed: + +```shell +<- {"execute":"query-migrate"} +-> {"return":{"status":"completed"}} +``` + +### Restoring a VM + +#### Precautions + +- The following models support the snapshot and boot from snapshot features: + - microvm + - Q35 (x86_64) + - virt (AArch64) +- When a snapshot is used for restoration, the configured devices must be the same as those used when the snapshot is created. +- If a microVM is used and the disk/NIC hot plugging-in feature is enabled before the snapshot is taken, you need to configure the hot plugged-in disks or NICs in the startup command line during restoration. + +#### Restoring a VM from a Snapshot File + +**Command Format** + +```shell +stratovirt -incoming URI +``` + +**Parameters** + +**URI**: snapshot path. The current version supports only the `file` type, followed by the absolute path of the snapshot file. + +**Example** + +Assume that the VM used for creating a snapshot is created by running the following command: + +```shell +$ stratovirt \ + -machine microvm \ + -kernel /path/to/kernel \ + -smp 1 -m 1024 \ + -append "console=ttyS0 pci=off reboot=k quiet panic=1 root=/dev/vda" \ + -drive file=/path/to/rootfs,id=rootfs,readonly=off,direct=off \ + -device virtio-blk-device,drive=rootfs \ + -qmp unix:/path/to/socket,server,nowait \ + -serial stdio +``` + +Then, the command for restoring the VM from the snapshot (assume that the snapshot storage path is **/path/to/template**) is as follows: + +```shell +$ stratovirt \ + -machine microvm \ + -kernel /path/to/kernel \ + -smp 1 -m 1024 \ + -append "console=ttyS0 pci=off reboot=k quiet panic=1 root=/dev/vda" \ + -drive file=/path/to/rootfs,id=rootfs,readonly=off,direct=off \ + -device virtio-blk-device,drive=rootfs \ + -qmp unix:/path/to/another_socket,server,nowait \ + -serial stdio \ + -incoming file:/path/to/template +``` + +## VM Live Migration + +### Introduction + +StratoVirt provides the VM live migration capability, that is, migrating a VM from one server to another without interrupting VM services. + +VM live migration can be used in the following scenarios: + +- When a server is overloaded, the VM live migration technology can be used to migrate VMs to another physical server for load balancing. +- When a server needs maintenance, VMs on the server can be migrated to another physical server without interrupting services. +- When a server is faulty and hardware needs to be replaced or the networking needs to be adjusted, VMs on the server can be migrated to another physical machine to prevent VM service interruption. + +### Live Migration Operations + +This section describes how to live migrate a VM. + +#### Preparing for Live Migration + +1. Log in to the host where the source VM is located as the **root** user and run the following command to start the source VM. Modify the parameters as required: + + ```shell + ./stratovirt \ + -machine q35 \ + -kernel ./vmlinux.bin \ + -append "console=ttyS0 pci=off reboot=k quiet panic=1 root=/dev/vda" \ + -drive file=path/to/rootfs,id=rootfs,readonly=off,direct=off \ + -device virtio-blk-pci,drive=rootfs,id=rootfs,bus=pcie.0,addr=0 \ + -qmp unix:path/to/socket1,server,nowait \ + -serial stdio \ + ``` + +2. Log in to the host where the target VM is located as the **root** user and run the following command to start the target VM. The parameters must be consistent with those of the source VM: + + ```shell + ./stratovirt \ + -machine q35 \ + -kernel ./vmlinux.bin \ + -append "console=ttyS0 pci=off reboot=k quiet panic=1 root=/dev/vda" \ + -drive file=path/to/rootfs,id=rootfs,readonly=off,direct=off \ + -device virtio-blk-pci,drive=rootfs,id=rootfs,bus=pcie.0,addr=0 \ + -qmp unix:path/to/socket2,server,nowait \ + -serial stdio \ + -incoming tcp:192.168.0.1:4446 \ + ``` + +> ![](public_sys-resources/icon-note.gif)**NOTE:** +> +> - The parameters for starting the target VM must be consistent with those for starting the source VM: +> - To change the data transmission mode for live migration from TCP to the UNIX socket protocol, change the `-incoming tcp:192.168.0.1:4446` parameter for starting the target VM to `-incoming unix:/tmp/stratovirt-migrate.socket`. However, the UNIX socket protocol supports only live migration between different VMs on the same physical host. + +#### Starting Live Migration + +On the host where the source VM is located, run the following command to start the VM live migration task: + +```shell +$ ncat -U path/to/socket1 +-> {"QMP":{"version":{"StratoVirt":{"micro":1,"minor":0,"major":0},"package":""},"capabilities":[]}} +<- {"execute":"migrate", "arguments":{"uri":"tcp:192.168.0.1:4446"}} +-> {"return":{}} +``` + +> ![](public_sys-resources/icon-note.gif)**NOTE:** +> +> If the UNIX socket protocol is used for live migration transmission, change `"uri":"tcp:192.168.0.1:4446"` in the command to `"uri":"unix:/tmp/stratovirt-migrate.socket"`. + +#### Finishing Live Migration + +After the `QMP` command is executed, the VM live migration task starts. If no live migration error log is generated, the source VM is migrated to the target VM, and the source VM is automatically destroyed. + +#### Canceling Live Migration + +During live migration, the migration may take a long time or the load of the host where the target VM is located changes. In this case, you need to adjust the migration policy. StratoVirt provides the feature of canceling live migration. + +To cancel live migration, log in to the host where the source VM is located and run the following `QMP` command: + +```shell +$ ncat -U path/to/socket1 +-> {"QMP":{"version":{"StratoVirt":{"micro":1,"minor":0,"major":0},"package":""},"capabilities":[]}} +<- {"execute":"migrate_cancel"} +-> {"return":{}} +``` + +If the live migration task on the target VM stops and the log indicates that the live migration is canceled, the live migration task is canceled successfully. + +#### Querying the Live Migration Status + +A live migration task can be in the following statuses: + +- **None**: Resources, such as vCPUs, memory, and devices, are not ready for live migration. +- **Setup**: Live migration resources have been prepared and live migration can be performed. +- **Active**: The VM is being live migrated. +- **Completed**: Live migration is complete. +- **Failed**: Live migration fails. + +The following `QMP` command queries completed live migration tasks: + +```shell +$ ncat -U path/to/socket +-> {"QMP":{"version":{"StratoVirt":{"micro":1,"minor":0,"major":0},"package":""},"capabilities":[]}} +<- {"execute":"query-migrate"} +-> {"return":{"status":"completed"}} +``` + +### Constraints + +StratoVirt supports live migration of the following standard VM boards: + +- Q35 (x86_64) +- virt (AArch64) + +The following devices and features do not support live migration: + +- vhost-net device +- vhost-user-net device +- virtio balloon device +- vfio device +- Shared backend storage +- Shared memory (back-end memory feature) + +The following command parameters for starting the source and target VMs must be the same: + +- virtio-net: MAC address +- device: BDF number +- smp +- m diff --git a/docs/en/virtualization/virtualization_platform/virtualization/_toc.yaml b/docs/en/virtualization/virtualization_platform/virtualization/_toc.yaml new file mode 100644 index 0000000000000000000000000000000000000000..8f83ae18ad26a72831f2d4818c5a5038d609ad9d --- /dev/null +++ b/docs/en/virtualization/virtualization_platform/virtualization/_toc.yaml @@ -0,0 +1,37 @@ +label: Virtualization User Guide +isManual: true +description: Virtualization technologies for creating and managing VMs on openEuler. +sections: + - label: Introduction to Virtualization + href: ./introduction-to-virtualization.md + - label: Installing Virtualization Components + href: ./virtualization-installation.md + - label: Environment Preparation + href: ./environment-preparation.md + - label: VM Configuration + href: ./vm-configuration.md + - label: Managing VMs + href: ./managing-vms.md + - label: VM Live Migration + href: ./vm-live-migration.md + - label: System Resource Management + href: ./system-resource-management.md + - label: Managing Devices + href: ./managing-devices.md + - label: VM Maintainability Management + href: ./vm-maintainability-management.md + - label: Best Practices + href: ./best-practices.md + - label: Tool Guide + href: ./tool-guide.md + sections: + - label: vmtop + href: ./vmtop.md + - label: LibcarePlus + href: ./libcareplus.md + - label: Skylark VM Hybrid Deployment + href: ./skylark.md + - label: Common Issues and Solutions + href: ./faq.md + - label: Appendix + href: ./appendix.md diff --git a/docs/en/virtualization/virtualization_platform/virtualization/appendix.md b/docs/en/virtualization/virtualization_platform/virtualization/appendix.md new file mode 100644 index 0000000000000000000000000000000000000000..6c9a781909da8c95fca096b383cfbecd84b98480 --- /dev/null +++ b/docs/en/virtualization/virtualization_platform/virtualization/appendix.md @@ -0,0 +1,143 @@ +# Appendix + +- [Appendix](#appendix) + - [Terminology \& Acronyms and Abbreviations](#terminology--acronyms-and-abbreviations) + +## Terminology & Acronyms and Abbreviations + +For the terminology & acronyms and abbreviation used in this document, see [Table 1](#table201236162279) and [Table 2](#table1423422319271). + +**Table 1** Terminology + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Term

+

Description

+

AArch64

+

AArch64 is an execution state of the ARMv8 architecture. AArch64 is not only an extension of the 32-bit ARM architecture, but also a brand new architecture in ARMv8 that uses the brand new A64 instruction set.

+

Domain

+

A collection of configurable resources, including memory, vCPUs, network devices, and disk devices. Run the VM in the domain. A domain is allocated with virtual resources and can be independently started, stopped, and restarted.

+

Libvirt

+

A set of tools used to manage virtualization platforms, including KVM, QEMU, Xen, and other virtualization platforms.

+

Guest OS

+

The OS running on the VM.

+

Host OS

+

The OS of the virtual physical machine.

+

Hypervisor

+

Virtual machine monitor (VMM), is an intermediate software layer that runs between a basic physical server and an OS. It allows multiple OSs and applications to share hardware.

+

VM

+

A complete virtual computer system that is constructed by using the virtualization technology and simulating the functions of a complete computer hardware system through software.

+
+ +**Table 2** Acronyms and abbreviations + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Acronyms and abbreviations

+

Full spelling

+

Full name

+

Description

+

NUMA

+

Non-Uniform Memory Access Architecture

+

Non Uniform Memory Access Architecture

+

NUMA is a memory architecture designed for multi-processor computers. Under NUMA, a processor accesses its own local memory faster than accessing non-local memory (the memory is located on another processor, or the memory shared between processors).

+

KVM

+

Kernel-based Virtual Machine

+

Kernel-based VM

+

KVM is a kernel-based VM. It is a kernel module of Linux and makes Linux a hypervisor.

+

OVS

+

Open vSwitch

+

Open vSwitch

+

OVS is a high-quality multi-layer vSwitch that uses the open-source Apache 2.0 license protocol.

+

QEMU

+

Quick Emulator

+

Quick Emulator

+

QEMU is a general-purpose, open-source emulator that implements hardware virtualization.

+

SMP

+

Symmetric Multi-Processor

+

Symmetric Multi-Processor

+

SMP is a multi-processor computer hardware architecture. Currently, most processor systems use a symmetric multi-processor architecture. The architecture system has multiple processors, each processor shares the memory subsystem and bus structure.

+

UEFI

+

Unified Extensible Firmware Interface

+

Unified Extensible Firmware Interface

+

A standard that describes new interfaces in detail. This interface is used by the OS to automatically load the prestart operation environment to an OS.

+

VM

+

Virtual Machine

+

VM

+

A complete virtual computer system that is constructed by using the virtualization technology and simulating the functions of a complete computer hardware system through software.

+

VMM

+

Virtual Machine Monitor

+

VM Monitor

+

An intermediate software layer that runs between a basic physical server and an OS. It allows multiple OSs and applications to share hardware.

+
diff --git a/docs/en/virtualization/virtualization_platform/virtualization/best-practices.md b/docs/en/virtualization/virtualization_platform/virtualization/best-practices.md new file mode 100644 index 0000000000000000000000000000000000000000..94949adc8845969e5d38a25780a1314b06bef595 --- /dev/null +++ b/docs/en/virtualization/virtualization_platform/virtualization/best-practices.md @@ -0,0 +1,624 @@ +# Best Practices + +## Performance Best Practices + +### Halt-Polling + +#### Overview + +If compute resources are sufficient, the halt-polling feature can be used to enable VMs to obtain performance similar to that of physical machines. If the halt-polling feature is not enabled, the host allocates CPU resources to other processes when the vCPU exits due to idle timeout. When the halt-polling feature is enabled on the host, the vCPU of the VM performs polling when it is idle. The polling duration depends on the actual configuration. If the vCPU is woken up during the polling, the vCPU can continue to run without being scheduled from the host. This reduces the scheduling overhead and improves the VM system performance. + +> [!NOTE]NOTE +> The halt-polling mechanism ensures that the vCPU thread of the VM responds in a timely manner. However, when the VM has no load, the host also performs polling. As a result, the host detects that the CPU usage of the vCPU is high, but the actual CPU usage of the VM is not high. + +#### Instructions + +The halt-polling feature is enabled by default. You can dynamically change the halt-polling time of vCPU by modifying the **halt\_poll\_ns** file. The default value is **500000**, in ns. + +For example, to set the polling duration to 400,000 ns, run the following command: + +```shell +# echo 400000 > /sys/module/kvm/parameters/halt_poll_ns +``` + +### I/O Thread Configuration + +#### Overview + +By default, QEMU main threads handle backend VM read and write operations on the KVM. This causes the following issues: + +- VM I/O requests are processed by a QEMU main thread. Therefore, the single-thread CPU usage becomes the bottleneck of VM I/O performance. +- The QEMU global lock \(qemu\_global\_mutex\) is used when VM I/O requests are processed by the QEMU main thread. If the I/O processing takes a long time, the QEMU main thread will occupy the global lock for a long time. As a result, the VM vCPU cannot be scheduled properly, affecting the overall VM performance and user experience. + +You can configure the I/O thread attribute for the virtio-blk disk or virtio-scsi controller. At the QEMU backend, an I/O thread is used to process read and write requests of a virtual disk. The mapping relationship between the I/O thread and the virtio-blk disk or virtio-scsi controller can be a one-to-one relationship to minimize the impact on the QEMU main thread, enhance the overall I/O performance of the VM, and improve user experience. + +#### Instructions + +To use I/O threads to process VM disk read and write requests, you need to modify VM configurations as follows: + +- Configure the total number of high-performance virtual disks on the VM. For example, set **** to **4** to control the total number of I/O threads. + + ```xml + + VMName + 4194304 + 4194304 + 4 + 4 + ``` + +- Configure the I/O thread attribute for the virtio-blk disk. **** indicates I/O thread IDs. The IDs start from 1 and each ID must be unique. The maximum ID is the value of ****. For example, to allocate I/O thread 2 to the virtio-blk disk, set parameters as follows: + + ```xml + + + + +
+ + ``` + +- Configure the I/O thread attribute for the virtio-scsi controller. For example, to allocate I/O thread 2 to the virtio-scsi controller, set parameters as follows: + + ```xml + + + +
+ + ``` + +- Bind I/O threads to a physical CPU. + + Binding I/O threads to specified physical CPUs does not affect the resource usage of vCPU threads. **** indicates I/O thread IDs, and **** indicates IDs of the bound physical CPUs. + + ```xml + + + + + ``` + +### Raw Device Mapping + +#### Overview + +When configuring VM storage devices, you can use configuration files to configure virtual disks for VMs, or connect block devices \(such as physical LUNs and LVs\) to VMs for use to improve storage performance. The latter configuration method is called raw device mapping \(RDM\). Through RDM, a virtual disk is presented as a small computer system interface \(SCSI\) device to the VM and supports most SCSI commands. + +RDM can be classified into virtual RDM and physical RDM based on backend implementation features. Compared with virtual RDM, physical RDM provides better performance and more SCSI commands. However, for physical RDM, the entire SCSI disk needs to be mounted to a VM for use. If partitions or logical volumes are used for configuration, the VM cannot identify the disk. + +#### Instructions + +VM configuration files need to be modified for RDM. The following is a configuration example. + +- Virtual RDM + + The following is an example of mounting the SCSI disk **/dev/sdc** on the host to the VM as a virtual raw device: + + ```xml + + + ... + + + + + +
+ + ... + + + ``` + +- Physical RDM + + The following is an example of mounting the SCSI disk **/dev/sdc** on the host to the VM as a physical raw device: + + ```xml + + + ... + + + + + +
+ + ... + + + ``` + +### kworker Isolation and Binding + +#### Overview + +kworker is a per-CPU thread implemented by the Linux kernel. It is used to execute workqueue requests in the system. kworker threads will compete for physical core resources with vCPU threads, resulting in virtualization service performance jitter. To ensure that the VM can run stably and reduce the interference of kworker threads on the VM, you can bind kworker threads on the host to a specific CPU. + +#### Instructions + +You can modify the **/sys/devices/virtual/workqueue/cpumask** file to bind tasks in the workqueue to the CPU specified by **cpumasks**. Masks in **cpumask** are in hexadecimal format. For example, if you need to bind kworker to CPU0 to CPU7, run the following command to change the mask to **ff**: + +```shell +# echo ff > /sys/devices/virtual/workqueue/cpumask +``` + +### HugePage Memory + +#### Overview + +Compared with traditional 4 KB memory paging, openEuler also supports 2 MB/1 GB memory paging. HugePage memory can effectively reduce TLB misses and significantly improve the performance of memory-intensive services. openEuler uses two technologies to implement HugePage memory. + +- Static HugePages + + The static HugePage requires that a static HugePage pool be reserved before the host OS is loaded. When creating a VM, you can modify the XML configuration file to specify that the VM memory is allocated from the static HugePage pool. The static HugePage ensures that all memory of a VM exists on the host as the HugePage to ensure physical continuity. However, the deployment difficulty is increased. After the page size of the static HugePage pool is changed, the host needs to be restarted for the change to take effect. The size of a static HugePage can be 2 MB or 1 GB. + +- THP + + If the transparent HugePage \(THP\) mode is enabled, the VM automatically selects available 2 MB consecutive pages and automatically splits and combines HugePages when allocating memory. When no 2 MB consecutive pages are available, the VM selects available 64 KB \(AArch64 architecture\) or 4 KB \(x86\_64 architecture\) pages for allocation. By using THP, users do not need to be aware of it and 2 MB HugePages can be used to improve memory access performance. + +If VMs use static HugePages, you can disable THP to reduce the overhead of the host OS and ensure stable VM performance. + +#### Instructions + +- Configure static HugePages. + + Before creating a VM, modify the XML file to configure a static HugePage for the VM. + + ```xml + + + + + + ``` + + The preceding XML segment indicates that a 1 GB static HugePage is configured for the VM. + + ```xml + + + + + + ``` + + The preceding XML segment indicates that a 2 MB static HugePage is configured for the VM. + +- Configure the THP. + + Dynamically enable the THP through sysfs. + + ```shell + # echo always > /sys/kernel/mm/transparent_hugepage/enabled + ``` + + Dynamically disable the THP. + + ```shell + # echo never > /sys/kernel/mm/transparent_hugepage/enabled + ``` + +### PV-qspinlock + +#### Overview + +PV-qspinlock optimizes the spin lock in the virtual scenario of CPU overcommitment. It allows the hypervisor to set the vCPU in the lock context to the block state and wake up the corresponding vCPU after the lock is released. In this way, pCPU resources can be better used in the overcommitment scenario, and the compilation application scenario is optimized to reduce the compilation duration. + +#### Instructions + +Modify the **/boot/efi/EFI/openEuler/grub.cfg** configuration file of the VM, add **arm_pvspin** to the startup parameter in the command line, and restart the VM for the modification to take effect. After PV-qspinlock takes effect, run the **dmesg** command on the VM. The following information is displayed: + +```shell +[ 0.000000] arm-pv: PV qspinlocks enabled +``` + +> [!NOTE]NOTE +> PV-qspinlock is supported only when the operating systems of the host machine and VM are both openEuler 20.09 or later and the VM kernel compilation option **CONFIG_PARAVIRT_SPINLOCKS** is set to **y** (default value for openEuler). + +### Guest-Idle-Haltpoll + +#### Overview + +To ensure fairness and reduce power consumption, when the vCPU of the VM is idle, the VM executes the WFx/HLT instruction to exit to the host machine and triggers context switchover. The host machine determines whether to schedule other processes or vCPUs on the physical CPU or enter the energy saving mode. However, overheads of switching between a virtual machine and a host machine, additional context switching, and IPI wakeup are relatively high, and this problem is particularly prominent in services where sleep and wakeup are frequently performed. The Guest-Idle-Haltpoll technology indicates that when the vCPU of a VM is idle, the WFx/HLT is not executed immediately and VM-exit occurs. Instead, polling is performed on the VM for a period of time. During this period, the tasks of other vCPUs that share the LLC on the vCPU are woken up without sending IPI interrupts. This reduces the overhead of sending and receiving IPI interrupts and the overhead of VM-exit, thereby reducing the task wakeup latency. + +> [!NOTE]NOTE +The execution of the **idle-haltpoll** command by the vCPU on the VM increases the CPU overhead of the vCPU on the host machine. Therefore, it is recommended that the vCPU exclusively occupy physical cores on the host machine when this feature is enabled. + +#### Procedure + +The Guest-Idle-Haltpoll feature is disabled by default. The following describes how to enable this feature. + +1. Enable the Guest-Idle-Haltpoll feature. + - If the processor architecture of the host machine is x86, you can configure hint-dedicated in the XML file of the VM on the host machine to enable this feature. In this way, the status that the vCPU exclusively occupies the physical core can be transferred to the VM through the VM XML configuration. The host machine ensures the status of the physical core exclusively occupied by the vCPU. + + ```xml + + ... + + + ... + + + + ... + + ``` + + Alternatively, set **cpuidle\_haltpoll.force** to **Y** in the kernel startup parameters of the VM to forcibly enable the function. This method does not require the host machine to configure the vCPU to exclusively occupy the physical core. + + ```ini + cpuidle_haltpoll.force=Y + ``` + + - If the processor architecture of the host machine is AArch64, this feature can be enabled only by configuring **cpuidle\_haltpoll.force=Y haltpoll.enable=Y** in the VM kernel startup parameters. + + ```ini + cpuidle_haltpoll.force=Y haltpoll.enable=Y + ``` + +2. Check whether the Guest-Idle-Haltpoll feature takes effect. Run the following command on the VM. If **haltpoll** is returned, the feature has taken effect. + + ```shell + # cat /sys/devices/system/cpu/cpuidle/current_driver + ``` + +3. (Optional) Set the Guest-Idle-Haltpoll parameter. + + The following configuration files are provided in the **/sys/module/haltpoll/parameters/** directory of the VM. You can adjust the configuration parameters based on service characteristics. + + - **guest\_halt\_poll\_ns**: a global parameter that specifies the maximum polling duration after the vCPU is idle. The default value is **200000** (unit: ns). + - **guest\_halt\_poll\_shrink**: a divisor that is used to shrink the current vCPU **guest\_halt\_poll\_ns** when the wakeup event occurs after the **global guest\_halt\_poll\_ns** time. The default value is **2**. + - **guest\_halt\_poll\_grow**: a multiplier that is used to extend the current vCPU **guest\_halt\_poll\_ns** when the wakeup event occurs after the current vCPU **guest\_halt\_poll\_ns** and before the global **guest\_halt\_poll\_ns**. The default value is **2**. + - **guest\_halt\_poll\_grow\_start**: When the system is idle, the **guest\_halt\_poll\_ns** of each vCPU reaches 0. This parameter is used to set the initial value of the current vCPU **guest\_halt\_poll\_ns** to facilitate scaling in and scaling out of the vCPU polling duration. The default value is **50000** (unit: ns). + - **guest\_halt\_poll\_allow\_shrink**: a switch that is used to enable vCPU **guest\_halt\_poll\_ns** scale-in. The default value is **Y**. (**Y** indicates enabling the scale-in; **N** indicates disabling the scale-in.) + + You can run the following command as the **root** user to change the parameter values. In the preceding command, _value_ indicates the parameter value to be set, and _configFile_ indicates the corresponding configuration file. + + ```shell + # echo value > /sys/module/haltpoll/parameters/configFile + ``` + + For example, to set the global guest\_halt\_poll\_ns to 200000 ns, run the following command: + + ```shell + # echo 200000 > /sys/module/haltpoll/parameters/guest_halt_poll_ns + ``` + +## Security Best Practices + +### Libvirt Authentication + +#### Overview + +When a user uses libvirt remote invocation but no authentication is performed, any third-party program that connects to the host's network can operate VMs through the libvirt remote invocation mechanism. This poses security risks. To improve system security, openEuler provides the libvirt authentication function. That is, users can remotely invoke a VM through libvirt only after identity authentication. Only specified users can access the VM, thereby protecting VMs on the network. + +#### Enabling Libvirt Authentication + +By default, the libvirt remote invocation function is disabled on openEuler. This following describes how to enable the libvirt remote invocation and libvirt authentication functions. + +1. Log in to the host. +2. Modify the libvirt service configuration file **/etc/libvirt/libvirtd.conf** to enable the libvirt remote invocation and libvirt authentication functions. For example, to enable the TCP remote invocation that is based on the Simple Authentication and Security Layer \(SASL\) framework, configure parameters by referring to the following: + + ```ini + #Transport layer security protocol. The value 0 indicates that the protocol is disabled, and the value 1 indicates that the protocol is enabled. You can set the value as needed. + listen_tls = 0 + #Enable the TCP remote invocation. To enable the libvirt remote invocation and libvirt authentication functions, set the value to 1. + listen_tcp = 1 + #User-defined protocol configuration for TCP remote invocation. The following uses sasl as an example. + auth_tcp = "sasl" + ``` + +3. Modify the **/etc/sasl2/libvirt.conf** configuration file to set the SASL mechanism and SASLDB. + + ```ini + #Authentication mechanism of the SASL framework. + mech_list: digest-md5 + #Database for storing usernames and passwords + sasldb_path: /etc/libvirt/passwd.db + ``` + +4. Add the user for SASL authentication and set the password. Take the user **userName** as an example. The command is as follows: + + ```shell + # saslpasswd2 -a libvirt userName + Password: + Again (for verification): + ``` + +5. Modify the **/etc/sysconfig/libvirtd** configuration file to enable the libvirt listening option. + + ```ini + LIBVIRTD_ARGS="--listen" + ``` + +6. Restart the libvirtd service to make the modification to take effect. + + ```shell + # systemctl restart libvirtd + ``` + +7. Check whether the authentication function for libvirt remote invocation takes effect. Enter the username and password as prompted. If the libvirt service is successfully connected, the function is successfully enabled. + + ```shell + # virsh -c qemu+tcp://192.168.0.1/system + Please enter your authentication name: openeuler + Please enter your password: + Welcome to virsh, the virtualization interactive terminal. + + Type: 'help' for help with commands + 'quit' to quit + + virsh # + ``` + +#### Managing SASL + +The following describes how to manage SASL users. + +Query an existing user in the database. + +```shell +# sasldblistusers2 -f /etc/libvirt/passwd.db +user@localhost.localdomain: userPassword +``` + +Delete a user from the database. + +```shell +# saslpasswd2 -a libvirt -d user +``` + +### qemu-ga + +#### Overview + +QEMU guest agent \(qemu-ga\) is a daemon running within VMs. It allows users on a host OS to perform various management operations on the guest OS through outband channels provided by QEMU. The operations include file operations \(open, read, write, close, seek, and flush\), internal shutdown, VM suspend \(suspend-disk, suspend-ram, and suspend-hybrid\), and obtaining of VM internal information \(including the memory, CPU, NIC, and OS information\). + +In some scenarios with high security requirements, qemu-ga provides the blacklist function to prevent internal information leakage of VMs. You can use a blacklist to selectively shield some functions provided by qemu-ga. + +> [!NOTE]NOTE +> The qemu-ga installation package is **qemu-guest-agent-**_xx_**.rpm**. It is not installed on openEuler by default. _xx_ indicates the actual version number. + +#### Procedure + +To add a qemu-ga blacklist, perform the following steps: + +1. Log in to the VM and ensure that the qemu-guest-agent service exists and is running. + + ```shell + # systemctl status qemu-guest-agent |grep Active + Active: active (running) since Wed 2018-03-28 08:17:33 CST; 9h ago + ``` + +2. Query which **qemu-ga** commands can be added to the blacklist: + + ```shell + # qemu-ga --blacklist ? + guest-sync-delimited + guest-sync + guest-ping + guest-get-time + guest-set-time + guest-info + ... + ``` + +3. Set the blacklist. Add the commands to be shielded to **--blacklist** in the **/usr/lib/systemd/system/qemu-guest-agent.service** file. Use spaces to separate different commands. For example, to add the **guest-file-open** and **guest-file-close** commands to the blacklist, configure the file by referring to the following: + + ```ini + [Service] + ExecStart=-/usr/bin/qemu-ga \ + --blacklist=guest-file-open guest-file-close + ``` + +4. Restart the qemu-guest-agent service. + + ```shell + # systemctl daemon-reload + # systemctl restart qemu-guest-agent + ``` + +5. Check whether the qemu-ga blacklist function takes effect on the VM, that is, whether the **--blacklist** parameter configured for the qemu-ga process is correct. + + ```shell + # ps -ef|grep qemu-ga|grep -E "blacklist=|b=" + root 727 1 0 08:17 ? 00:00:00 /usr/bin/qemu-ga --method=virtio-serial --path=/dev/virtio-ports/org.qemu.guest_agent.0 --blacklist=guest-file-open guest-file-close guest-file-read guest-file-write guest-file-seek guest-file-flush -F/etc/qemu-ga/fsfreeze-hook + ``` + + > [!NOTE]NOTE + > For more information about qemu-ga, visit [https://wiki.qemu.org/Features/GuestAgent](https://wiki.qemu.org/Features/GuestAgent). + +### sVirt Protection + +#### Overview + +In a virtualization environment that uses the discretionary access control \(DAC\) policy only, malicious VMs running on hosts may attack the hypervisor or other VMs. To improve security in virtualization scenarios, openEuler uses sVirt for protection. sVirt is a security protection technology based on SELinux. It is applicable to KVM virtualization scenarios. A VM is a common process on the host OS. In the hypervisor, the sVirt mechanism labels QEMU processes corresponding to VMs with SELinux labels. In addition to types which are used to label virtualization processes and files, different categories are used to label different VMs. Each VM can access only file devices of the same category. This prevents VMs from accessing files and devices on unauthorized hosts or other VMs, thereby preventing VM escape and improving host and VM security. + +#### Enabling sVirt Protection + +1. Enable SELinux on the host. + + 1. Log in to the host. + 2. Enable the SELinux function on the host. + + - Modify the system startup parameter file **grub.cfg** to set **selinux** to **1**. + + ```ini + selinux=1 + ``` + + - Modify **/etc/selinux/config** to set the **SELINUX** to **enforcing**. + + ```ini + SELINUX=enforcing + ``` + + 3. Restart the host. + + ```shell + # reboot + ``` + +2. Create a VM where the sVirt function is enabled. + + 1. Add the following information to the VM configuration file: + + ```xml + + ``` + + Or check whether the following configuration exists in the file: + + ```xml + + ``` + + 2. Create a VM. + + ```shell + # virsh define openEulerVM.xml + ``` + +3. Check whether sVirt is enabled. + Run the following command to check whether sVirt protection has been enabled for the QEMU process of the running VM. If **svirt\_t:s0:c** exists, sVirt protection has been enabled. + + ```shell + # ps -eZ|grep qemu |grep "svirt_t:s0:c" + system_u:system_r:svirt_t:s0:c200,c947 11359 ? 00:03:59 qemu-kvm + system_u:system_r:svirt_t:s0:c427,c670 13790 ? 19:02:07 qemu-kvm + ``` + +### VM Trusted Boot + +#### Overview + +Trusted boot includes measure boot and remote attestation. The measure boot function is mainly provided by virtualization component. The remote attestation function is enabled by users who install related software (RA client) on VMs and set up the RA server. + +The two basic elements for measure boot are the root of trust (RoT) and chain of trust. The basic idea is to establish a RoT in the computer system. The trustworthiness of the RoT is ensured by physical security, technical security, and management security, that is, CRTM (Core Root of Trust for Measurement). A chain of trust is established, starting from the RoT to the BIOS/BootLoader, operating system, and then to the application. The measure boot and trust is performed by one level to the previous level. Finally, the trust is extended to the entire system. The preceding process looks like a chain, so it is called a chain of trust. + +The CRTM is the root of the measure boot and the first component of the system startup. No other code is used to check the integrity of the CRTM. Therefore, as the starting point of the chain of trust, it must be an absolutely trusted source of trust. The CRTM needs to be technically designed as a segment of read-only or strictly restricted code to defend against BIOS attacks and prevent remote injection of malicious code or modification of startup code at the upper layer of the operating system. In a physical host, the CPU microcode is used as the CRTM. In a virtualization environment, the sec part of the vBIOS is generally the CRTM. + +During startup, the previous component measures (calculates the hash value) the next component, and then extends the measurement value to the trusted storage area, for example, the PCR of the TPM. The CRTM measurement BootLoader extends the measurement value to the PCR, and the BootLoader measurement OS extends the measurement value to the PCR. + +#### Configuring the vTPM Device to Enable Measurement Startup + +##### Installing the swtpm and libtpms Software + +swtpm provides a TPM emulator (TPM 1.2 and TPM 2.0) that can be integrated into a virtualization environment. So far, it has been integrated into QEMU and serves as a prototype system in RunC. swtpm uses libtpms to provide TPM1.2 and TPM2.0 simulation functions. +Currently, openEuler 21.03 provides the libtpms and swtpm sources. You can run the yum command to install them. + +```shell +# yum install libtpms swtpm swtpm-devel swtpm-tools +``` + +##### Configuring the vTPM Device for the VM + +1. Add the following configuration to the VM configuration file: + + ```xml + + ... + + ... + + + + ... + + ... + + ``` + + > [!NOTE]NOTE + > Currently, trusted boot of VMs on the AArch64 architecture of openEuler 20.09 does not support the ACPI feature. Therefore, do not configure the ACPI feature for VMs. Otherwise, vTPM devices cannot be identified after VMs are started. If the AArch64 architecture is used in versions earlier than openEuler 22.09, set **tpm model** to **<tpm model='tpm-tis-device'>**. + +2. Create a VM. + + ```shell + # virsh define MeasuredBoot.xml + ``` + +3. Start the VM. + + Before starting the VM, run the **chmod** command to grant the following permissions to the **/var/lib/swtpm-localca/** directory. Otherwise, libvirt cannot start swtpm. + + ```shell + # chmod -R 777 /var/lib/swtpm-localca/ + # + # virsh start MeasuredbootVM + ``` + +##### Confirming that the Measure Boot Is Successfully Enabled + +The vBIOS determines whether to enable the measure boot function. Currently, the vBIOS in openEuler 20.09 has the measure boot capability. If the host machine uses the edk2 component of another version, check whether the edk2 component supports the measure boot function. + +Log in to the VM as the **root** user and check whether the TPM driver, tpm2-tss protocol stack, and tpm2-tools are installed on the VM. +By default, the tpm driver (tpm_tis.ko), tpm2-tss protocol stack, and tpm2-tools are installed in openEuler 21.03. If another OS is used, run the following command to check whether the driver and related tools are installed: + +```shell +# lsmod |grep tpm +# tpm_tis 16384 0 +# +# yum list installed | grep -E 'tpm2-tss|tpm2-tools' +# +# yum install tpm2-tss tpm2-tools +``` + +You can run the **tpm2_pcrread** (**tpm2_pcrlist** in tpm2_tools of earlier versions) command to list all PCR values. + +```shell +# tpm2_pcrread +sha1 : + 0 : fffdcae7cef57d93c5f64d1f9b7f1879275cff55 + 1 : 5387ba1d17bba5fdadb77621376250c2396c5413 + 2 : b2a83b0ebf2f8374299a5b2bdfc31ea955ad7236 + 3 : b2a83b0ebf2f8374299a5b2bdfc31ea955ad7236 + 4 : e5d40ace8bb38eb170c61682eb36a3020226d2c0 + 5 : 367f6ea79688062a6df5f4737ac17b69cd37fd61 + 6 : b2a83b0ebf2f8374299a5b2bdfc31ea955ad7236 + 7 : 518bd167271fbb64589c61e43d8c0165861431d8 + 8 : af65222affd33ff779780c51fa8077485aca46d9 + 9 : 5905ec9fb508b0f30b2abf8787093f16ca608a5a + 10 : 0000000000000000000000000000000000000000 + 11 : 0000000000000000000000000000000000000000 + 12 : 0000000000000000000000000000000000000000 + 13 : 0000000000000000000000000000000000000000 + 14 : 0000000000000000000000000000000000000000 + 15 : 0000000000000000000000000000000000000000 + 16 : 0000000000000000000000000000000000000000 + 17 : ffffffffffffffffffffffffffffffffffffffff + 18 : ffffffffffffffffffffffffffffffffffffffff + 19 : ffffffffffffffffffffffffffffffffffffffff + 20 : ffffffffffffffffffffffffffffffffffffffff + 21 : ffffffffffffffffffffffffffffffffffffffff + 22 : ffffffffffffffffffffffffffffffffffffffff + 23 : 0000000000000000000000000000000000000000 +sha256 : + 0 : d020873038268904688cfe5b8ccf8b8d84c1a2892fc866847355f86f8066ea2d + 1 : 13cebccdb194dd916f2c0c41ec6832dfb15b41a9eb5229d33a25acb5ebc3f016 + 2 : 3d458cfe55cc03ea1f443f1562beec8df51c75e14a9fcf9a7234a13f198e7969 + 3 : 3d458cfe55cc03ea1f443f1562beec8df51c75e14a9fcf9a7234a13f198e7969 + 4 : 07f9074ccd4513ef1cafd7660f9afede422b679fd8ad99d25c0659eba07cc045 + 5 : ba34c80668f84407cd7f498e310cc4ac12ec6ec43ea8c93cebb2a688cf226aff + 6 : 3d458cfe55cc03ea1f443f1562beec8df51c75e14a9fcf9a7234a13f198e7969 + 7 : 65caf8dd1e0ea7a6347b635d2b379c93b9a1351edc2afc3ecda700e534eb3068 + 8 : f440af381b644231e7322babfd393808e8ebb3a692af57c0b3a5d162a6e2c118 + 9 : 54c08c8ba4706273f53f90085592f7b2e4eaafb8d433295b66b78d9754145cfc + 10 : 0000000000000000000000000000000000000000000000000000000000000000 + 11 : 0000000000000000000000000000000000000000000000000000000000000000 + 12 : 0000000000000000000000000000000000000000000000000000000000000000 + 13 : 0000000000000000000000000000000000000000000000000000000000000000 + 14 : 0000000000000000000000000000000000000000000000000000000000000000 + 15 : 0000000000000000000000000000000000000000000000000000000000000000 + 16 : 0000000000000000000000000000000000000000000000000000000000000000 + 17 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff + 18 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff + 19 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff + 20 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff + 21 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff + 22 : ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff + 23 : 0000000000000000000000000000000000000000000000000000000000000000 +``` diff --git a/docs/en/virtualization/virtualization_platform/virtualization/environment-preparation.md b/docs/en/virtualization/virtualization_platform/virtualization/environment-preparation.md new file mode 100644 index 0000000000000000000000000000000000000000..537fee1f4d17b52be825fe19c3362bd97a5f71a7 --- /dev/null +++ b/docs/en/virtualization/virtualization_platform/virtualization/environment-preparation.md @@ -0,0 +1,368 @@ +# Environment Preparation + +- [Environment Preparation](#environment-preparation) + - [Preparing a VM Image](#preparing-a-vm-image) + - [Preparing the VM Network](#preparing-the-vm-network) + - [Preparing Boot Firmware](#preparing-boot-firmware) + +## Preparing a VM Image + +### Overview + +A VM image is a file that contains a virtual disk that has been installed and can be used to start the OS. VM images are in different formats, such as raw and qcow2. Compared with the raw format, the qcow2 format occupies less space and supports features such as snapshot, copy-on-write, AES encryption, and zlib compression. However, the performance of the qcow2 format is slightly lower than that of the raw format. The qemu-img tool is used to create image files. This section uses the qcow2 image file as an example to describe how to create a VM image. + +### Creating an Image + +To create a qcow2 image file, perform the following steps: + +1. Install the **qemu-img** software package. + + ```shell + yum install -y qemu-img + ``` + +2. Run the **create** command of the qemu-img tool to create an image file. The command format is as follows: + + ```shell + qemu-img create -f -o + ``` + + The parameters are described as follows: + + - _imgFormat_: Image format. The value can be **raw** or **qcow2**. + - _fileOption_: File option, which is used to set features of an image file, such as specifying a backend image file, compression, and encryption. + - _fileName_: File name. + - _diskSize_: Disk size, which specifies the size of a block disk. The unit can be K, M, G, or T, indicating KiB, MiB, GiB, or TiB. + + For example, to create an image file openEuler-image.qcow2 whose disk size is 4 GB and format is qcow2, the command and output are as follows: + + ```shell + $ qemu-img create -f qcow2 openEuler-image.qcow2 4G + Formatting 'openEuler-image.qcow2', fmt=qcow2 size=4294967296 cluster_size=65536 lazy_refcounts=off refcount_bits=16 + ``` + +### Changing the Image Disk Space + +If a VM requires larger disk space, you can use the qemu-img tool to change the disk space of the VM image. The method is as follows: + +1. Run the following command to query the disk space of the VM image: + + ```shell + qemu-img info + ``` + + For example, if the command and output for querying the disk space of the openEuler-image.qcow2 image are as follows, the disk space of the image is 4 GiB. + + ```shell + $ qemu-img info openEuler-image.qcow2 + image: openEuler-image.qcow2 + file format: qcow2 + virtual size: 4.0G (4294967296 bytes) + disk size: 196K + cluster_size: 65536 + Format specific information: + compat: 1.1 + lazy refcounts: false + refcount bits: 16 + corrupt: false + ``` + +2. Run the following command to change the image disk space. In the command, _imgFileName_ indicates the image name, and **+** and **-** indicate the image disk space to be increased and decreased, respectively. The unit is KB, MB, GB, and T, indicating KiB, MiB, GiB, and TiB, respectively. + + ```shell + qemu-img resize [+|-] + ``` + + For example, to expand the disk space of the openEuler-image.qcow2 image to 24 GiB, that is, to add 20 GiB to the original 4 GiB, the command and output are as follows: + + ```shell + $ qemu-img resize openEuler-image.qcow2 +20G + Image resized. + ``` + +3. Run the following command to check whether the image disk space is changed successfully: + + ```shell + qemu-img info + ``` + + For example, if the openEuler-image.qcow2 image disk space has been expanded to 24 GiB, the command and output are as follows: + + ```shell + $ qemu-img info openEuler-image.qcow2 + image: openEuler-image.qcow2 + file format: qcow2 + virtual size: 24G (25769803776 bytes) + disk size: 200K + cluster_size: 65536 + Format specific information: + compat: 1.1 + lazy refcounts: false + refcount bits: 16 + corrupt: false + ``` + +## Preparing the VM Network + +### Overview + +To enable the VM to communicate with external networks, you need to configure the network environment for the VM. KVM virtualization supports multiple types of bridges, such as Linux bridge and Open vSwitch bridge. As shown in [Figure 1](#fig1785384714917), the data transmission path is **VM \> virtual NIC device \> Linux bridge or Open vSwitch bridge \> physical NIC**. In addition to configuring virtual NICs \(vNICs\) for VMs, creating a bridge for a host is the key to connecting to a virtualized network. + +This section describes how to set up a Linux bridge and an Open vSwitch bridge to connect a VM to the network. You can select a bridge type based on the site requirements. + +**Figure 1** Virtual network structure + +![](./figures/virtual-network-structure.png) + +### Setting Up a Linux Bridge + +The following describes how to bind the physical NIC eth0 to the Linux bridge br0. + +1. Install the **bridge-utils** software package. + + The Linux bridge is managed by the brctl tool. The corresponding installation package is bridge-utils. The installation command is as follows: + + ```shell + yum install -y bridge-utils + ``` + +2. Create bridge br0. + + ```shell + brctl addbr br0 + ``` + +3. Bind the physical NIC eth0 to the Linux bridge. + + ```shell + brctl addif br0 eth0 + ``` + +4. After eth0 is connected to the bridge, the IP address of eth0 is set to 0.0.0.0. + + ```shell + ifconfig eth0 0.0.0.0 + ``` + +5. Set the IP address of br0. + - If a DHCP server is available, set a dynamic IP address through the dhclient. + + ```shell + dhclient br0 + ``` + + - If no DHCP server is available, configure a static IP address for br0. For example, set the static IP address to 192.168.1.2 and subnet mask to 255.255.255.0. + + ```shell + ifconfig br0 192.168.1.2 netmask 255.255.255.0 + ``` + +### Setting Up an Open vSwitch Bridge + +The Open vSwitch bridge provides more convenient automatic orchestration capabilities. This section describes how to install network virtualization components to set up an Open vSwitch bridge. + +**1. Install the Open vSwitch component.** + +If the Open vSwitch is used to provide virtual network, you need to install the Open vSwitch network virtualization component. + +1. Install the Open vSwitch component. + + ```shell + yum install -y openvswitch + ``` + +2. Start the Open vSwitch service. + + ```shell + systemctl start openvswitch + ``` + +**2. Check whether the installation is successful.** + +1. Check whether the openvswitch component is successfully installed. If the installation is successful, the software package information is displayed. The command and output are as follows: + + ```shell + $ rpm -qi openvswitch + Name : openvswitch + Version : 2.11.1 + Release : 1 + Architecture: aarch64 + Install Date: Thu 15 Aug 2019 05:08:35 PM CST + Group : System Environment/Daemons + Size : 6051185 + License : ASL 2.0 + Signature : (none) + Source RPM : openvswitch-2.11.1-1.src.rpm + Build Date : Thu 08 Aug 2019 05:24:46 PM CST + Build Host : armbuild10b247b121b105 + Relocations : (not relocatable) + Vendor : Nicira, Inc. + URL : http://www.openvswitch.org/ + Summary : Open vSwitch daemon/database/utilities + Description : + Open vSwitch provides standard network bridging functions and + support for the OpenFlow protocol for remote per-flow control of + traffic. + ``` + +2. Check whether the Open vSwitch service is started successfully. If the service is in the **Active** state, the service is started successfully. You can use the command line tool provided by the Open vSwitch. The command and output are as follows: + + ```shell + $ systemctl status openvswitch + ● openvswitch.service - LSB: Open vSwitch switch + Loaded: loaded (/etc/rc.d/init.d/openvswitch; generated) + Active: active (running) since Sat 2019-08-17 09:47:14 CST; 4min 39s ago + Docs: man:systemd-sysv-generator(8) + Process: 54554 ExecStart=/etc/rc.d/init.d/openvswitch start (code=exited, status=0/SUCCESS) + Tasks: 4 (limit: 9830) + Memory: 22.0M + CGroup: /system.slice/openvswitch.service + ├─54580 ovsdb-server: monitoring pid 54581 (healthy) + ├─54581 ovsdb-server /etc/openvswitch/conf.db -vconsole:emer -vsyslog:err -vfile:info --remote=punix:/var/run/openvswitch/db.sock --private-key=db:Open_vSwitch,SSL,private_key --certificate> + ├─54602 ovs-vswitchd: monitoring pid 54603 (healthy) + └─54603 ovs-vswitchd unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/open> + ``` + +**3. Set up an Open vSwitch bridge** + +The following describes how to set up an Open vSwitch layer-1 bridge br0. + +1. Create the Open vSwitch bridge br0. + + ```shell + ovs-vsctl add-br br0 + ``` + +2. Add the physical NIC eth0 to br0. + + ```shell + ovs-vsctl add-port br0 eth0 + ``` + +3. After eth0 is connected to the bridge, the IP address of eth0 is set to 0.0.0.0. + + ```shell + ifconfig eth0 0.0.0.0 + ``` + +4. Assign an IP address to OVS bridge br0. + - If a DHCP server is available, set a dynamic IP address through the dhclient. + + ```shell + dhclient br0 + ``` + + - If no DHCP server is available, configure a static IP address for br0, for example, 192.168.1.2. + + ```shell + ifconfig br0 192.168.1.2 + ``` + +## Preparing Boot Firmware + +### Overview + +The boot mode varies depending on the architecture. x86 servers support the Unified Extensible Firmware Interface \(UEFI\) and BIOS boot modes, and AArch64 servers support only the UEFI boot mode. By default, boot files corresponding to the BIOS mode have been installed on openEuler. No additional operations are required. This section describes how to install boot files corresponding to the UEFI mode. + +The Unified Extensible Firmware Interface \(UEFI\) is a new interface standard used for power-on auto check and OS boot. It is an alternative to the traditional BIOS. EDK II is a set of open source code that implements the UEFI standard. In virtualization scenarios, the EDK II tool set is used to start a VM in UEFI mode. Before using the EDK II tool, you need to install the corresponding software package before starting a VM. This section describes how to install the EDK II tool. + +### Installation Methods + +If the UEFI mode is used, the tool set EDK II needs to be installed. The installation package for the AArch64 architecture is **edk2-aarch64**, and that for the x86 architecture is **edk2-ovmf**. This section uses the AArch64 architecture as an example to describe the installation method. For the x86 architecture, you only need to replace **edk2-aarch64** with **edk2-ovmf**. + +1. Run the following command to install the **edk** software package: + + In the AArch64 architecture, the **edk2** package name is **edk2-aarch64**. + + ```shell + yum install -y edk2-aarch64 + ``` + + In the x86\_64 architecture, the **edk2** package name is **edk2-ovmf**. + + ```shell + yum install -y edk2-ovmf + ``` + +2. Run the following command to check whether the **edk** software package is successfully installed: + + In the AArch64 architecture, the command is as follows: + + ```shell + rpm -qi edk2-aarch64 + ``` + + If information similar to the following is displayed, the **edk** software package is successfully installed: + + ```text + Name : edk2-aarch64 + Version : 20180815gitcb5f4f45ce + Release : 1.oe3 + Architecture: noarch + Install Date: Mon 22 Jul 2019 04:52:33 PM CST + Group : Applications/Emulators + ``` + + In the x86\_64 architecture, the command is as follows: + + ```shell + rpm -qi edk2-ovmf + ``` + + If information similar to the following is displayed, the **edk** software package is successfully installed: + + ```text + Name : edk2-ovmf + Version : 201908 + Release : 6.oe1 + Architecture: noarch + Install Date: Thu 19 Mar 2020 09:09:06 AM CST + ``` + +## Configuring a Non-root User + +### Overview + +openEuler virtualization uses virsh to manage VMs. If you want to use the virsh commands to manage VMs as a non-root user, you need to perform related configurations before using the commands. This section provides the configuration guide. + +### Operation Guide + +To allow a non-root user to run the virsh commands to manage VMs, perform the following operations (replace **userName** in the following commands with the actual non-root user name): + +1. Log in to the host as the **root** user. + +2. Add the non-root user to the libvirt user group. + + ```shell + usermod -a -G libvirt userName + ``` + +3. Switch to the non-root user. + + ```shell + su userName + ``` + +4. Configure environment variables for the non-root user. Run the **vim** command to open the ~/.bashrc file: + + ```shell + vim ~/.bashrc + ``` + + Add the following content to the end of the file and save the file: + + ```shell + export LIBVIRT_DEFAULT_URI="qemu:///system" + ``` + + Run the following command for the configuration to take effect: + + ```shell + source ~/.bashrc + ``` + +5. Add the following content to the domain root element in the XML configuration file of the VM so that the qemu-kvm process can access the disk image files. + + ```xml + + ``` diff --git a/docs/en/virtualization/virtualization_platform/virtualization/faq.md b/docs/en/virtualization/virtualization_platform/virtualization/faq.md new file mode 100644 index 0000000000000000000000000000000000000000..b50334284b57d5d91d7f2627e92affe01470241a --- /dev/null +++ b/docs/en/virtualization/virtualization_platform/virtualization/faq.md @@ -0,0 +1,13 @@ +# Common Issues and Solutions + +## Issue 1: QEMU Hot Patch Created with LibcarePlus Fails to Load + +The problem arises when the QEMU version does not match the hot patch version. To resolve this, download the source code for the corresponding QEMU version and ensure the environments for creating the hot patch and building the QEMU package are identical. The buildID can verify consistency. If users lack the QEMU build environment, they can **build and install the package themselves**, then use the buildID from `/usr/libexec/qemu-kvm` in the self-built package. + +## Issue 2: Hot Patch Created with LibcarePlus Is Loaded but Not Effective + +This occurs because certain functions are not supported, including dead loops, non-exiting functions, recursive functions, initialization functions, inline functions, and functions shorter than 5 bytes. To address this, verify if the patched function falls under these constraints. + +## Issue 3: The First Result Displayed by the kvmtop Tool Is Calculated from Two Samples with a 0.05-Second Interval, Resulting in Significant Fluctuations + +This issue stems from a defect in the open-source top framework, and there is currently no solution available. diff --git a/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP1.png b/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP1.png new file mode 100644 index 0000000000000000000000000000000000000000..536e0618a3ab5b70937292205242a08237e34712 Binary files /dev/null and b/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP1.png differ diff --git a/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP2.png b/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP2.png new file mode 100644 index 0000000000000000000000000000000000000000..0557c8782960188dbe9d84a1d0e66c9b45d2b303 Binary files /dev/null and b/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP2.png differ diff --git a/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP3.png b/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP3.png new file mode 100644 index 0000000000000000000000000000000000000000..326fcf1e8d5e3c795ebcde286d8e0fef14bec7d1 Binary files /dev/null and b/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP3.png differ diff --git a/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP4.png b/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP4.png new file mode 100644 index 0000000000000000000000000000000000000000..bc77c038e1e3a5ec30d7ba4f805ca937792e9327 Binary files /dev/null and b/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP4.png differ diff --git a/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP5.png b/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP5.png new file mode 100644 index 0000000000000000000000000000000000000000..0f22b3cbd84f7c93f74898a926bc3e32f231667f Binary files /dev/null and b/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP5.png differ diff --git a/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP6.png b/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP6.png new file mode 100644 index 0000000000000000000000000000000000000000..08235013ca71f1ec51e9af2f143629d1a6132fe9 Binary files /dev/null and b/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP6.png differ diff --git a/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP7.png b/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP7.png new file mode 100644 index 0000000000000000000000000000000000000000..f934521d59dd4a75449fcb2ca8abc54045b9102b Binary files /dev/null and b/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP7.png differ diff --git a/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP8.png b/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP8.png new file mode 100644 index 0000000000000000000000000000000000000000..9a8158e3378bf25dee05b892cc60f424542455d7 Binary files /dev/null and b/docs/en/virtualization/virtualization_platform/virtualization/figures/CertEnrollP8.png differ diff --git a/docs/en/virtualization/virtualization_platform/virtualization/figures/OSBootFlow.png b/docs/en/virtualization/virtualization_platform/virtualization/figures/OSBootFlow.png new file mode 100644 index 0000000000000000000000000000000000000000..f496c5675c72359e5160384c766a11399b04bfa6 Binary files /dev/null and b/docs/en/virtualization/virtualization_platform/virtualization/figures/OSBootFlow.png differ diff --git a/docs/en/virtualization/virtualization_platform/virtualization/figures/SecureBootFlow.png b/docs/en/virtualization/virtualization_platform/virtualization/figures/SecureBootFlow.png new file mode 100644 index 0000000000000000000000000000000000000000..d639975800752c6eca6765a416c256a4752fb590 Binary files /dev/null and b/docs/en/virtualization/virtualization_platform/virtualization/figures/SecureBootFlow.png differ diff --git a/docs/en/virtualization/virtualization_platform/virtualization/figures/kvm-architecture.png b/docs/en/virtualization/virtualization_platform/virtualization/figures/kvm-architecture.png new file mode 100644 index 0000000000000000000000000000000000000000..74cc91f2944b4ed5404edf036b1d71cd84df7e29 Binary files /dev/null and b/docs/en/virtualization/virtualization_platform/virtualization/figures/kvm-architecture.png differ diff --git a/docs/en/virtualization/virtualization_platform/virtualization/figures/status-transition-diagram.png b/docs/en/virtualization/virtualization_platform/virtualization/figures/status-transition-diagram.png new file mode 100644 index 0000000000000000000000000000000000000000..f0e9bb814b155e4abde372c93677a97f207b5475 Binary files /dev/null and b/docs/en/virtualization/virtualization_platform/virtualization/figures/status-transition-diagram.png differ diff --git a/docs/en/virtualization/virtualization_platform/virtualization/figures/virtual-network-structure.png b/docs/en/virtualization/virtualization_platform/virtualization/figures/virtual-network-structure.png new file mode 100644 index 0000000000000000000000000000000000000000..4d57a184352c1a3558eeac56499888b2b98b31f1 Binary files /dev/null and b/docs/en/virtualization/virtualization_platform/virtualization/figures/virtual-network-structure.png differ diff --git a/docs/en/virtualization/virtualization_platform/virtualization/figures/virtualized-architecture.png b/docs/en/virtualization/virtualization_platform/virtualization/figures/virtualized-architecture.png new file mode 100644 index 0000000000000000000000000000000000000000..2e8b01628fb51bb6cc1162d6158259192506bc3a Binary files /dev/null and b/docs/en/virtualization/virtualization_platform/virtualization/figures/virtualized-architecture.png differ diff --git a/docs/en/virtualization/virtualization_platform/virtualization/introduction-to-virtualization.md b/docs/en/virtualization/virtualization_platform/virtualization/introduction-to-virtualization.md new file mode 100644 index 0000000000000000000000000000000000000000..f5cf39971257f40254f288fcdc9513cda2c3a380 --- /dev/null +++ b/docs/en/virtualization/virtualization_platform/virtualization/introduction-to-virtualization.md @@ -0,0 +1,80 @@ +# Introduction to Virtualization + +## Overview + +In computer technologies, virtualization is a resource management technology. It abstracts various physical resources \(such as processors, memory, disks, and network adapters\) of a computer, converts the resources, and presents the resources for segmentation and combination into one or more computer configuration environments. This resource management technology breaks the inseparable barrier of the physical structure, and makes these resources not restricted by the architecture or geographical or physical configuration of the existing resources after virtualization.. In this way, users can better leverage the computer hardware resources and maximize the resource utilization. + +Virtualization enables multiple virtual machines \(VMs\) to run on a physical server. VMs share resources such as processors, memory, and I/O devices of physical machines, but are logically isolated from each other. In the virtualization technology, the physical server is called a host machine, the VM running on the host machine is called a guest, and the operating system \(OS\) running on the VM is called a guest OS. A layer of software, called the virtualization layer, exists between a host machine and a VM to simulate virtual hardware. This virtualization layer is called a VM monitor, as shown in the following figure. + +**Figure 1** Virtualized architecture +![](./figures/virtualized-architecture.png) + +## Virtualized Architecture + +Currently, mainstream virtualization technologies are classified into two types based on the implementation structure of the Virtual Machine Monitor \(VMM\): + +- Hypervisor model + + In this model, VMM is considered as a complete operating system \(OS\) and has the virtualization function. VMM directly manages all physical resources, including processors, memory, and I/O devices. + +- Host model + + In this model, physical resources are managed by a host OS, which is a traditional OS, such as Linux and Windows. The host OS does not provide the virtualization capability. The VMM that provides the virtualization capability runs on the host OS as a driver or software of the system. The VMM invokes the host OS service to obtain resources and simulate the processor, memory, and I/O devices. The virtualization implementation of this model includes KVM and Virtual Box. + +Kernel-based Virtual Machine \(KVM\) is a kernel module of Linux. It makes Linux a hypervisor. [Figure 2](#fig310953013541) shows the KVM architecture. KVM does not simulate any hardware device. It is used to enable virtualization capabilities provided by the hardware, such as Intel VT-x, AMD-V, ARM virtualization extensions. The user-mode QEMU simulates the mainboard, memory, and I/O devices. The user-mode QEMU works with the KVM module to simulate VM hardware. The guest OS runs on the hardware simulated by the QEMU and KVM. + +**Figure 2** KVM architecture +![](./figures/kvm-architecture.png) + +## Virtualization Components + +Virtualization components provided in the openEuler software package: + +- KVM: provides the core virtualization infrastructure to make the Linux system a hypervisor. Multiple VMs can run on the same host at the same time. +- QEMU: simulates a processor and provides a set of device models to work with KVM to implement hardware-based virtualization simulation acceleration. +- Libvirt: provides a tool set for managing VMs, including unified, stable, and open application programming interfaces \(APIs\), daemon process \(libvirtd\), and default command line management tool \(virsh\). +- Open vSwitch: provides a virtual network tool set for VMs, supports programming extension and standard management interfaces and protocols \(such as NetFlow, sFlow, IPFIX, RSPAN, CLI, LACP, and 802.1ag\). + +## Virtualization Characteristics + +Virtualization has the following characteristics: + +- Partition + + Virtualization can logically divide software on a physical server to run multiple VMs \(virtual servers\) with different specifications. + +- Isolation + + Virtualization can simulate virtual hardware and provide hardware conditions for VMs to run complete OSs. The OSs of each VM are independent and isolated from each other. For example, if the OS of a VM breaks down due to a fault or malicious damage, the OSs and applications of other VMs are not affected. + +- Encapsulation + + Encapsulation is performed on a per VM basis. The excellent encapsulation capability makes VMs more flexible than physical machines. Functions such as live migration, snapshot, and cloning of VMs can be realized, implementing quick deployment and automatic O&M of data centers. + +- Hardware-irrelevant + + After being abstracted by the virtualization layer, VMs are not directly bound to underlying hardware and can run on other servers without being modified. + +## Virtualization Advantages + +Virtualization brings the following benefits to infrastructure of the data center: + +- Flexibility and scalability + + Users can dynamically allocate and reclaim resources based to meet dynamic service requirements. In addition, users can plan different VM specifications based on product requirements and adjust the scale without changing the physical resource configuration. + +- Higher availability and better O&M methods + + Virtualization provides O&M methods such as live migration, snapshot, live upgrade, and automatic DR. Physical resources can be deleted, upgraded, or changed without affecting users, improving service continuity and implementing automatic O&M. + +- Security hardening + + Virtualization provides OS-level isolation and hardware-based processor operation privilege-level control. Compared with simple sharing mechanisms, virtualization provides higher security and implements controllable and secure access to data and services. + +- High resource utilization + + Virtualization supports dynamic sharing of physical resources and resource pools, improving resource utilization. + +## openEuler Virtualization + +openEuler provides KVM virtualization components that support the AArch64 and x86\_64 processor architectures. diff --git a/docs/en/virtualization/virtualization_platform/virtualization/libcareplus.md b/docs/en/virtualization/virtualization_platform/virtualization/libcareplus.md new file mode 100644 index 0000000000000000000000000000000000000000..396e1e5b1555b171643c2a74bdc5266179c36cda --- /dev/null +++ b/docs/en/virtualization/virtualization_platform/virtualization/libcareplus.md @@ -0,0 +1,401 @@ +# LibcarePlus + + + +- [LibcarePlus](#libcareplus) + - [Overview](#overview) + - [Hardware and Software Requirements](#hardware-and-software-requirements) + - [Precautions and Constraints](#precautions-and-constraints) + - [Installing LibcarePlus](#installing-libcareplus) + - [Software Installation Dependencies](#software-installation-dependencies) + - [Installing LibcarePlus](#installing-libcareplus-1) + - [Creating LibcarePlus Hot Patches](#creating-libcareplus-hot-patches) + - [Introduction](#introduction) + - [Manual Creation](#manual-creation) + - [Creation Through a Script](#creation-through-a-script) + - [Applying the LibcarePlus Hot Patch](#applying-the-libcareplus-hot-patch) + - [Preparation](#preparation) + - [Loading the Hot Patch](#loading-the-hot-patch) + - [Querying a Hot Patch](#querying-a-hot-patch) + - [Uninstalling the Hot Patch](#uninstalling-the-hot-patch) + + + +## Overview + +LibcarePlus is a hot patch framework for user-mode processes. It can perform hot patch operations on target processes running on the Linux system without restarting the processes. Hot patches can be used to fix CVEs and urgent bugs that do not interrupt application services. + +## Hardware and Software Requirements + +The following software and hardware requirements must be met to use LibcarePlus on openEuler: + +- Currently, the x86 and ARM64 architectures are supported. + +- LibcarePlus can run on any Linux distribution that supports **libunwind**, **elfutils**, and **binutils**. +- LibcarePlus uses the **ptrace()** system call, which requires the kernel configuration option enabled for the corresponding Linux distribution. +- LibcarePlus needs the symbol table of the original executable file when creating a hot patch. Do not strip the symbol table too early. +- On the Linux OS where SELinux is enabled, manually adapt the SELinux policies. + +## Precautions and Constraints + +When using LibcarePlus, comply with the following hot patch specifications and constraints: + +- Only the code written in the C language is supported. The assembly language is not supported. +- Only user-mode programs are supported. Dynamic library patches are not supported. +- The code file name must comply with the C language identifier naming specifications. That is, the code file name consists of letters (A-Z and a-z), digits (0-9), and underscores (_) but the first character cannot be a digit. Special characters such as hyphens (-) and dollar signs ($) are not allowed. +- Incremental patches are supported. Multiple patches can be installed on a process. However, you need to design the patch installation and uninstallation management. Generally, the installation and uninstallation comply with the first-in, last-out (FILO) rule. +- Automatic patch loading is not natively supported. You can design an automatic patch loading method for a specific process. +- Patch query is supported. +- The static function patch is restricted by the symbol table that can find the function in the system. +- Hot patches are process-specific. That is, a hot patch of a dynamic library can be applied only to process that invoke the dynamic library. +- The number of patches that can be applied to a process is limited by the range of the jump instruction and the size of the hole in the virtual memory address space. Generally, up to 512 patches can be applied to a process. +- Thread local storage (TLS) variables of the initial executable (IE) model can be modified. +- Symbols defined in a patch cannot be used in subsequent patches. +- Hot patches are not supported in the following scenarios: + - Infinite loop function, non-exit function, inline function, initialization function, and non-maskable interrupt (NMI) function + - Replacing global variables + - Functions less than 5 bytes + - Modifying the header file + - Adding or deleting the input and output parameters of the target function + - Changing (adding, deleting, or modifying) data structure members + - Modifying the C files that contain GCC macros such as __LINE__ and __FILE__ + - Modifying the Intel vector assembly instruction + +## Installing LibcarePlus + +### Software Installation Dependencies + +The LibcarePlus running depends on **libunwind**, **elfutils**, and **binutils**. On the openEuler system configured with the Yum repo, you can run the following commands to install the software on which LibcarePlus depends: + +``` shell +# yum install -y binutils elfutils elfutils-libelf-devel libunwind-devel +``` + +#### Installing LibcarePlus + +```shell +# yum install libcareplus libcareplus-devel -y +``` + +Check whether LibcarePlus is installed. + +``` shell +# libcare-ctl -h +usage: libcare-ctl [options] [args] + +Options: + -v - verbose mode + -h - this message + +Commands: + patch - apply patch to a user-space process + unpatch- unapply patch from a user-space process + info - show info on applied patches +``` + +## Creating LibcarePlus Hot Patches + +### Introduction + +LibcarePlus hot patch creation methods: + +- Manual creation +- Creation through a script + +The process of manually creating a hot patch is complex. For a project with a large amount of code, for example, QEMU, it is extremely difficult to manually create a hot patch. You are advised to use the script provided by LibcarePlus to generate a hot patch file with one click. + +#### Manual Creation + +The following takes the original file **foo.c** and the patch file **bar.c** as examples to describe how to manually create a hot patch. + +1. Prepare the original file and patch file written in the C language. For example, **foo.c** and **bar.c**. + +
+ Expand foo.c +

+ + ``` c + // foo.c + #include + #include + + void print_hello(void) + { + printf("Hello world!\n"); + } + + int main(void) + { + while (1) { + print_hello(); + sleep(1); + } + } + ``` + +

+
+ +
+ Expand bar.c +

+ + ``` c + // bar.c + #include + #include + + void print_hello(void) + { + printf("Hello world %s!\n", "being patched"); + } + + int main(void) + { + while (1) { + print_hello(); + sleep(1); + } + } + ``` + +

+
+ +2. Build the original file and patch file to obtain the assembly files **foo.s** and **bar.s**. + + ``` shell + # gcc -S foo.c + # gcc -S bar.c + # ls + bar.c bar.s foo.c foo.s + ``` + +3. Run `kpatch_gensrc` to compare **foo.s** and **bar.s** and generate the **foobar.s** file that contains the assembly content of the original file and the differences. + + ``` shell + # sed -i 's/bar.c/foo.c/' bar.s + # kpatch_gensrc --os=rhel6 -i foo.s -i bar.s -o foobar.s --force-global + ``` + + By default, `kpatch_gensrc` compares the original files in the same C language. Therefore, before the comparison, you need to run the `sed` command to change the file name **bar.c** in the patch assembly file **bar.s** to the original file name **foo.c**. Call `kpatch_gensrc` to specify the input files as **foo.s** and **bar.s** and the output file as **foobar.s**. + +4. Build the assembly file **foo.s** in the original file and the generated assembly file **foobar.s** to obtain the executable files **foo** and **foobar**. + + ``` shell + # gcc -o foo foo.s + # gcc -o foobar foobar.s -Wl,-q + ``` + + The **-Wl, -q** linker options reserve the relocation sections in **foobar**. + +5. Use `kpatch_strip` to remove the duplicate content from the executables **foo** and **foobar** and reserve the content required for creating hot patches. + + ``` shell + # kpatch_strip --strip foobar foobar.stripped + # kpatch_strip --rel-fixup foo foobar.stripped + # strip --strip-unneeded foobar.stripped + # kpatch_strip --undo-link foo foobar.stripped + ``` + + The options in the preceding command are described as follows: + + - **--strip** removes useless sections for patch creation from **foobar**. + - **--rel-fixup** repairs the address of the variables and functions accessed in the patch. + - **strip --strip-unneeded** removes the useless symbol information for hot patch relocation. + - **--undo-link** changes the symbol address in a patch from absolute to relative. + +6. Create a hot patch file. + + After the preceding operations, the contents required for creating the hot patch are obtained. Run the `kpatch_make` command to input parameters Build ID of the original executable file and **foobar.stripped** (output file of `kpatch_strip`) to `kpatch_make` to generate a hot patch file. + + ``` shell + # str=$(readelf -n foo | grep 'Build ID') + # substr=${str##* } + # kpatch_make -b $substr -i 0001 foobar.stripped -o foo.kpatch + # ls + bar.c bar.s foo foobar foobar.s foobar.stripped foo.c foo.kpatch foo.s + ``` + + The final hot patch file **foo.kpatch** whose patch ID is **0001** is obtained. + +#### Creation Through a Script + +This section describes how to use LibcarePlus built-in **libcare-patch-make** script to create a hot patch file. The original file **foo.c** and patch file **bar.c** are used as an example. + +1. Run the `diff` command to generate the comparison file of **foo.c** and **bar.c**. + + ``` shell + # diff -up foo.c bar.c > foo.patch + ``` + + The content of the **foo.patch** file is as follows: + +
+ Expand foo.patch +

+ + ``` diff + --- foo.c 2020-12-09 15:39:51.159632075 +0800 + +++ bar.c 2020-12-09 15:40:03.818632220 +0800 + @@ -1,10 +1,10 @@ + -// foo.c + +// bar.c + #include + #include + + void print_hello(void) + { + - printf("Hello world!\n"); + + printf("Hello world %s!\n", "being patched"); + } + + int main(void) + ``` + +

+
+ +2. Write the **makefile** for building **foo.c** as follows: + +
+ Expand makefile +

+ + ``` makefile + all: foo + + foo: foo.c + $(CC) -o $@ $< + + clean: + rm -f foo + + install: foo + mkdir $$DESTDIR || : + cp foo $$DESTDIR + ``` + +

+
+ +3. After the **makefile** is done, directly call `libcare-patch-make`. If `libcare-patch-make` asks you which file to install the patch, enter the original file name, as shown in the following: + + ``` shell + # libcare-patch-make --clean -i 0001 foo.patch + rm -f foo + BUILDING ORIGINAL CODE + /usr/local/bin/libcare-cc -o foo foo.c + INSTALLING ORIGINAL OBJECTS INTO /libcareplus/test/lpmake + mkdir $DESTDIR || : + cp foo $DESTDIR + applying foo.patch... + can't find file to patch at input line 3 + Perhaps you used the wrong -p or --strip option? + The text leading up to this was: + -------------------------- + |--- foo.c 2020-12-10 09:43:04.445375845 +0800 + |+++ bar.c 2020-12-10 09:48:36.778379648 +0800 + -------------------------- + File to patch: foo.c + patching file foo.c + BUILDING PATCHED CODE + /usr/local/bin/libcare-cc -o foo foo.c + INSTALLING PATCHED OBJECTS INTO /libcareplus/test/.lpmaketmp/patched + mkdir $DESTDIR || : + cp foo $DESTDIR + MAKING PATCHES + Fixing up relocation printf@@GLIBC_2.2.5+fffffffffffffffc + Fixing up relocation print_hello+0 + patch for /libcareplus/test/lpmake/foo is in /libcareplus/test/patchroot/700297b7bc56a11e1d5a6fb564c2a5bc5b282082.kpatch + ``` + + After the command is executed, the output indicates that the hot patch file is in the **patchroot** directory of the current directory, and the executable file is in the **lpmake** directory. By default, the Build ID is used to name a hot patch file generated by a script. + +## Applying the LibcarePlus Hot Patch + +This following uses the original file **foo.c** and patch file **bar.c** as an example to describe how to use the LibcarePlus hot patch. + +### Preparation + +Before using the LibcarePlus hot patch, prepare the original executable file **foo** and hot patch file **foo.kpatch**. + +### Loading the Hot Patch + +The procedure for applying the LibcarePlus hot patch is as follows: + +1. In the first shell window, run the executable file to be patched: + + ``` shell + # ./lpmake/foo + Hello world! + Hello world! + Hello world! + ``` + +2. In the second shell window, run the `libcare-ctl` command to apply the hot patch: + + ``` shell + # libcare-ctl -v patch -p $(pidof foo) ./patchroot/BuildID.kpatch + ``` + + If the hot patch is applied successfully, the following information is displayed in the second shell window: + + ``` shell + 1 patch hunk(s) have been successfully applied to PID '10999' + ``` + + The following information is displayed for the target process running in the first shell window: + + ``` shell + Hello world! + Hello world! + Hello world being patched! + Hello world being patched! + ``` + +### Querying a Hot Patch + +The procedure for querying a LibcarePlus hot patch is as follows: + +1. Run the following command in the second shell window: + + ```shell + # libcare-ctl info -p $(pidof foo) + + ``` + + If a hot patch is installed, the following information is displayed in the second shell window: + + ```shell + Pid: 551763 + Target: foo + Build id: df05a25bdadd282812d3ee5f0a460e69038575de + Applied patch number: 1 + Patch id: 0001 + ``` + +### Uninstalling the Hot Patch + +The procedure for uninstalling the LibcarePlus hot patch is as follows: + +1. Run the following command in the second shell window: + + ``` shell + # libcare-ctl unpatch -p $(pidof foo) -i 0001 + ``` + + If the hot patch is uninstalled successfully, the following information is displayed in the second shell window: + + ``` shell + 1 patch hunk(s) were successfully cancelled from PID '10999' + ``` + +2. The following information is displayed for the target process running in the first shell window: + + ``` shell + Hello world being patched! + Hello world being patched! + Hello world! + Hello world! + ``` diff --git a/docs/en/virtualization/virtualization_platform/virtualization/managing-devices.md b/docs/en/virtualization/virtualization_platform/virtualization/managing-devices.md new file mode 100644 index 0000000000000000000000000000000000000000..42546a08f540aa9d58fc6033a7440a87d587635f --- /dev/null +++ b/docs/en/virtualization/virtualization_platform/virtualization/managing-devices.md @@ -0,0 +1,797 @@ +# Managing Devices + +- [Managing Devices](#managing-devices) + - [Configuring a PCIe Controller for a VM](#configuring-a-pcie-controller-for-a-vm) + - [Overview](#overview) + - [Configuring the PCIe Root, PCIe Root Port, and PCIe-PCI-Bridge](#configuring-the-pcie-root-pcie-root-port-and-pcie-pci-bridge) + - [Managing Virtual Disks](#managing-virtual-disks) + - [Managing vNICs](#managing-vnics) + - [Configuring a Virtual Serial Port](#configuring-a-virtual-serial-port) + - [Managing Device Passthrough](#managing-device-passthrough) + - [PCI Passthrough](#pci-passthrough) + - [SR-IOV Passthrough](#sr-iov-passthrough) + - [Managing VM USB](#managing-vm-usb) + - [Configuring USB Controllers](#configuring-usb-controllers) + - [Configuring a USB Passthrough Device](#configuring-a-usb-passthrough-device) + - [Storing Snapshots](#storing-snapshots) + - [Overview](#overview-7) + - [Procedure](#procedure-4) + - [Configuring Disk I/O Suspension](#configuring-disk-io-suspension) + - [Introduction](#introduction) + - [Overview](#overview-8) + - [Application Scenarios](#application-scenarios) + - [Precautions and Restrictions](#precautions-and-restrictions) + - [Disk I/O Suspension Configuration](#disk-io-suspension-configuration) + - [Using the QEMU CLI](#using-the-qemu-cli) + - [Using an XML Configuration File](#using-an-xml-configuration-file) + +## Configuring a PCIe Controller for a VM + +### Overview + +The NIC, disk controller, and PCIe pass-through devices in a VM must be mounted to a PCIe root port. Each root port corresponds to a PCIe slot. The devices mounted to the root port support hot swap, but the root port does not support hot swap. Therefore, users need to consider the hot swap requirements and plan the maximum number of PCIe root ports reserved for the VM. Before the VM is started, the root port is statically configured. + +### Configuring the PCIe Root, PCIe Root Port, and PCIe-PCI-Bridge + +The VM PCIe controller is configured using the XML file. The **model** corresponding to PCIe root, PCIe root port, and PCIe-PCI-bridge in the XML file are **pcie-root**, **pcie-root-port**, and **pcie-to-pci-bridge**, respectively. + +- Simplified configuration method + + Add the following contents to the XML file of the VM. Other attributes of the controller are automatically filled by libvirt. + + ```xml + + + + + + + ``` + + The **pcie-root** and **pcie-to-pci-bridge** occupy one **index** respectively. Therefore, the final **index** is the number of required **root ports + 1**. + +- Complete configuration method + + Add the following contents to the XML file of the VM: + + ```xml + + + + +
+ + + +
+ + + + +
+ + + ``` + + In the preceding contents: + + - The **chassis** and **port** attributes of the root port must be in ascending order. Because a PCIe-PCI-bridge is inserted in the middle, the **chassis** number skips **2**, but the **port** numbers are still consecutive. + - The **address function** of the root port ranges from **0\*0** to **0\*7**. + - A maximum of eight functions can be mounted to each slot. When the slot is full, the slot number increases. + + The complete configuration method is complex. Therefore, the simplified one is recommended. + +## Managing Virtual Disks + +### Overview + +Virtual disk types include virtio-blk, virtio-scsi, and vhost-scsi. virtio-blk simulates a block device, and virtio-scsi and vhost-scsi simulate SCSI devices. + +- virtio-blk: It can be used for common system disk and data disk. In this configuration, the virtual disk is presented as **vd\[a-z\]** or **vd\[a-z\]\[a-z\]** in the VM. +- virtio-scsi: It is recommended for common system disk and data disk. In this configuration, the virtual disk is presented as **sd\[a-z\]** or **sd\[a-z\]\[a-z\]** in the VM. +- vhost-scsi: It is recommended for the virtual disk that has high performance requirements. In this configuration, the virtual disk is presented as **sd\[a-z\]** or **sd\[a-z\]\[a-z\]** on the VM. + +### Procedure + +For details about how to configure a virtual disk, see **VM Configuration** > **Network Devices**. This section uses the virtio-scsi disk as an example to describe how to attach and detach a virtual disk. + +- Attach a virtio-scsi disk. + + Run the **virsh attach-device** command to attach the virtio-scsi virtual disk. + + ```shell + # virsh attach-device + ``` + + The preceding command can be used to attach a disk to a VM online. The disk information is specified in the **attach-device.xml** file. The following is an example of the **attach-device.xml** file: + + ```shell + ### attach-device.xml ### + + + + + +
+ + ``` + + The disk attached by running the preceding commands becomes invalid after the VM is shut down and restarted. If you need to permanently attach a virtual disk to a VM, run the **virsh attach-device** command with the **--config** parameter. + +- Detach a virtio-scsi disk. + + If a disk attached online is no longer used, run the **virsh detach** command to dynamically detach it. + + ```shell + # virsh detach-device + ``` + + **detach-device.xml** specifies the XML information of the disk to be detached, which must be the same as the XML information during dynamic attachment. + +## Managing vNICs + +### Overview + +The vNIC types include virtio-net, vhost-net, and vhost-user. After creating a VM, you may need to attach or detach a vNIC. openEuler supports NIC hot swap, which can change the network throughput and improve system flexibility and scalability. + +### Procedure + +For details about how to configure a virtual NIC, see [3.2.4.2 Network Devices](./vm-configuration.md#network-devices). This section uses the vhost-net NIC as an example to describe how to attach and detach a vNIC. + +- Attach the vhost-net NIC. + + Run the **virsh attach-device** command to attach the vhost-net vNIC. + + ```shell + # virsh attach-device + ``` + + The preceding command can be used to attach a vhost-net NIC to a running VM. The NIC information is specified in the **attach-device.xml** file. The following is an example of the **attach-device.xml** file: + + ```shell + ### attach-device.xml ### + + + + + + + + ``` + + The vhost-net NIC attached using the preceding commands becomes invalid after the VM is shut down and restarted. If you need to permanently attach a vNIC to a VM, run the **virsh attach-device** command with the **--config** parameter. + +- Detach the vhost-net NIC. + + If a NIC attached online is no longer used, run the **virsh detach** command to dynamically detach it. + + ```shell + # virsh detach-device + ``` + + **detach-device.xml** specifies the XML information of the vNIC to be detached, which must be the same as the XML information during dynamic attachment. + +## Configuring a Virtual Serial Port + +### Overview + +In a virtualization environment, VMs and host machines need to communicate with each other to meet management and service requirements. However, in the complex network architecture of the cloud management system, services running on the management plane and VMs running on the service plane cannot communicate with each other at layer 3. As a result, service deployment and information collection are not fast enough. Therefore, a virtual serial port is required for communication between VMs and host machines. You can add serial port configuration items to the XML configuration file of a VM to implement communication between VMs and host machines. + +### Procedure + +The Linux VM serial port console is a pseudo terminal device connected to the host machine through the serial port of the VM. It implements interactive operations on the VM through the host machine. In this scenario, the serial port needs to be configured in the pty type. This section describes how to configure a pty serial port. + +- Add the following virtual serial port configuration items under the **devices** node in the XML configuration file of the VM: + + ```xml + + + + + + ``` + +- Run the **virsh console** command to connect to the pty serial port of the running VM. + + ```shell + # virsh console + ``` + +- To ensure that no serial port message is missed, use the **--console** option to connect to the serial port when starting the VM. + + ```shell + # virsh start --console + ``` + +## Managing Device Passthrough + +The device passthrough technology enables VMs to directly access physical devices. The I/O performance of VMs can be improved in this way. + +Currently, the VFIO passthrough is used. It can be classified into PCI passthrough and SR-IOV passthrough based on device type. + +### PCI Passthrough + +PCI passthrough directly assigns a physical PCI device on the host to a VM. The VM can directly access the device. PCI passthrough uses the VFIO device passthrough mode. The PCI passthrough configuration file in XML format for a VM is as follows: + +```xml + + + +
+ + +
+ +``` + +**Table 1** Device configuration items for PCI passthrough + +| Parameter | Description | Value | +|---|---|---| +| hostdev.source.address.domain | Domain ID of the PCI device on the host OS. | ≥ 0 | +| hostdev.source.address.bus | Bus ID of the PCI device on the host OS. | ≥ 1 | +| hostdev.source.address.slot | Device ID of the PCI device on the host OS. | ≥ 0 | +| hostdev.source.address.function | Function ID of the PCI device on the host OS. | ≥ 0 | +| hostdev.driver.name | Backend driver of PCI passthrough. This parameter is optional. | **vfio** (default value) | +| hostdev.rom | Whether the VM can access the ROM of the passthrough device. | This parameter can be set to **on** or **off**. The default value is **on**.
- **on**: indicates that the VM can access the ROM of the passthrough device. For example, if a VM with a passthrough NIC needs to boot from the preboot execution environment (PXE), or a VM with a passthrough Host Bus Adapter (HBA) card needs to boot from the ROM, you can set this parameter to **on**.
- **off**: indicates that the VM cannot access the ROM of the passthrough device. | +| hostdev.address.type | Device type displayed on the guest, which must be the same as the actual device type. | **pci** (default configuration) | +| hostdev.address.domain | Domain number of the device displayed on the guest. | 0x0000 | +| hostdev.address.bus | Bus number of the device displayed on the guest. | **0x00** (default configuration). This parameter can only be set to the bus number configured in section "Configuring a PCIe Controller for a VM." | +| hostdev.address.slot | Slot number of the device displayed on the guest. | The slot number range is \[0x03,0x1e]
Note: - The first slot number 0x00 is occupied by the system, the second slot number 0x01 is occupied by the IDE controller and USB controller, and the third slot number 0x02 is occupied by the video. - The last slot number 0x1f is occupied by the pvchannel. | +| hostdev.address.function | Function number of the device displayed on the guest. | **0x0** (default configuration): The function number range is \[0x0,0x7] | + +> [!NOTE]NOTE +> VFIO passthrough is implemented by IOMMU group. Devices are divided to IOMMU groups based on access control services (ACS) on hardware. Devices in the same IOMMU group can be assigned to only one VM. If multiple functions on a PCI device belong to the same IOMMU group, they can be directly assigned to only one VM as well. + +### SR-IOV Passthrough + +#### Overview + +Single Root I/O Virtualization (SR-IOV) is a hardware-based virtualization solution. With the SR-IOV technology, a physical function (PF) can provide multiple virtual functions (VFs), and each VF can be directly assigned to a VM. This greatly improves hardware resource utilization and I/O performance of VMs. A typical application scenario is SR-IOV passthrough for NICs. With the SR-IOV technology, a physical NIC (PF) can function as multiple VF NICs, and then the VFs can be directly assigned to VMs. + +> [!NOTE]NOTE +> +> - SR-IOV requires the support of physical hardware. Before using SR-IOV, ensure that the hardware device to be directly assigned supports SR-IOV and the device driver on the host OS works in SR-IOV mode. +> - The following describes how to query the NIC model: +> In the following command output, values in the first column indicate the PCI numbers of NICs, and **19e5:1822** indicates the vendor ID and device ID of the NIC. +> +> ```shell +> # lspci | grep Ether +> 05:00.0 Ethernet controller: Device 19e5:1822 (rev 45) +> 07:00.0 Ethernet controller: Device 19e5:1822 (rev 45) +> 09:00.0 Ethernet controller: Device 19e5:1822 (rev 45) +> 0b:00.0 Ethernet controller: Device 19e5:1822 (rev 45) +> 81:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01) +> 81:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01) +> ``` + +#### Procedure + +To configure SR-IOV passthrough for a NIC, perform the following steps: + +1. Enable the SR-IOV mode for the NIC. + 1. Ensure that VF driver support provided by the NIC supplier exists on the guest OS. Otherwise, VFs in the guest OS cannot work properly. + 2. Enable the SMMU/IOMMU support in the BIOS of the host OS. The enabling method varies depending on the servers of different vendors. For details, see the help documents of the servers. + 3. Configure the host driver to enable the SR-IOV VF mode. The following uses the Hi1822 NIC as an example to describe how to enable 16 VFs. + + ```shell + echo 16 > /sys/class/net/ethX/device/sriov_numvfs + ``` + +2. Obtain the PCI BDF information of PFs and VFs. + 1. Run the following command to obtain the NIC resource list on the current board: + + ```shell + # lspci | grep Eth + 03:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45) + 04:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45) + 05:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45) + 06:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (4*25GE) (rev 45) + 7d:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Device a222 (rev 20) + 7d:00.1 Ethernet controller: Huawei Technologies Co., Ltd. Device a222 (rev 20) + 7d:00.2 Ethernet controller: Huawei Technologies Co., Ltd. Device a221 (rev 20) + 7d:00.3 Ethernet controller: Huawei Technologies Co., Ltd. Device a221 (rev 20) + ``` + + 2. Run the following command to view the PCI BDF information of VFs: + + ```shell + # lspci | grep "Virtual Function" + 03:00.1 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) + 03:00.2 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) + 03:00.3 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) + 03:00.4 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) + 03:00.5 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) + 03:00.6 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) + 03:00.7 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) + 03:01.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) + 03:01.1 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) + 03:01.2 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family Virtual Function (rev 45) + ``` + + 3. Select an available VF and write its configuration to the VM configuration file based on its BDF information. For example, the bus ID of the device **03:00.1** is **03**, its slot ID is **00**, and its function ID is **1**. + +3. Identify and manage the mapping between PFs and VFs. + 1. Identify VFs corresponding to a PF. The following uses PF 03.00.0 as an example: + + ```shell + # ls -l /sys/bus/pci/devices/0000\:03\:00.0/ + ``` + + The following symbolic link information is displayed. You can obtain the VF IDs (virtfnX) and PCI BDF IDs based on the information. + + 2. Identify the PF corresponding to a VF. The following uses VF 03:00.1 as an example: + + ```shell + # ls -l /sys/bus/pci/devices/0000\:03\:00.1/ + ``` + + The following symbolic link information is displayed. You can obtain PCI BDF IDs of the PF based on the information. + + ```shell + lrwxrwxrwx 1 root root 0 Mar 28 22:44 physfn -> ../0000:03:00.0 + ``` + + 3. Obtain names of NICs corresponding to the PFs or VFs. For example: + + ```shell + # ls /sys/bus/pci/devices/0000:03:00.0/net + eth0 + ``` + + 4. Set the MAC address, VLAN, and QoS information of VFs to ensure that the VFs are in the **Up** state before passthrough. The following uses VF 03:00.1 as an example. The PF is eth0 and the VF ID is **0**. + + ```shell + # ip link set eth0 vf 0 mac 90:E2:BA:21:XX:XX #Sets the MAC address. + # ifconfig eth0 up + # ip link set eth0 vf 0 rate 100 #Sets the VF outbound rate, in Mbit/s. + # ip link show eth0 #Views the MAC address, VLAN ID, and QoS information to check whether the configuration is successful. + ``` + +4. Mount the SR-IOV NIC to the VM. + + When creating a VM, add the SR-IOV passthrough configuration item to the VM configuration file. + + ```xml + + + +
+ + + + + + ``` + + **Table 1** SR-IOV configuration options + + | Parameter | Description | Value | + | ------------------------------- | --------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | hostdev.managed | Two modes for libvirt to process PCI devices. | **no**: default value. The passthrough device is managed by the user.
**yes**: The passthrough device is managed by libvirt. Set this parameter to **yes** in the SR-IOV passthrough scenario. | + | hostdev.source.address.bus | Bus ID of the PCI device on the host OS. | ≥ 1 | + | hostdev.source.address.slot | Device ID of the PCI device on the host OS. | ≥ 0 | + | hostdev.source.address.function | Function ID of the PCI device on the host OS. | ≥ 0 | + + > [!NOTE]NOTE + > Disabling the SR-IOV function: + > To disable the SR-IOV function after the VM is stopped and no VF is in use, run the following command: + > The following uses the Hi1822 NIC corresponding network interface name: eth0) as an example: + + ```shell + echo 0 > /sys/class/net/eth0/device/sriov_numvfs + ``` + +#### Configuring SR-IOV Passthrough for the HPRE Accelerator + +The accelerator engine is a hardware acceleration solution provided by TaiShan 200 servers. The HPRE accelerator is used to accelerate SSL/TLS applications. It significantly reduces processor consumption and improves processor efficiency. +On the Kunpeng server, you need to pass through the VFs of the HPRE accelerator on the host to the VM for internal services of the VM. + +**Table 1** HPRE accelerator description + +| | Description | +|-------------|-----------------------------------------------------------------------------------------------------| +| Device name | Hi1620 on-chip RSA/DH security algorithm accelerator (HPRE engine) | +| Description | Modular exponentiation, RSA key pair operation, DH calculation, and some large-number auxiliary operations (modular exponentiation, modular multiplication, modulo operation, multiplication, modular inversion, prime number test, and mutual prime test)| +| VendorID | 0x19E5 | +| PF DeviceID | 0xA258 | +| VF DeviceID | 0xA259 | +| Maximum number of VFs | An HPRE PF supports a maximum of 63 VFs. | + +> [!NOTE]NOTE +> When a VM is using a VF device, the driver on the host cannot be uninstalled, and the accelerator does not support hot swap. +> VF operation (If **VFNUMS** is **0**, the VF is disabled, and **hpre_num** is used to identify a specific accelerator device): +> +> ```shell +> echo $VFNUMS > /sys/class/uacce/hisi_hpre-$hpre_num/device/sriov_numvfs +> ``` + +### vDPA Passthrough + +#### Overview + +vDPA passthrough connects a device on a host to the vDPA framework, uses the vhost-vdpa driver to present a character device, and configures the character device for VMs to use. vDPA passthrough drives can serve as system or data drives for VMs and support hot expansion of data drives. + +vDPA passthrough provides the similar I/O performance as VFIO passthrough, provides flexibility of VirtIO devices, and supports live migration of vDPA passthrough devices. + +With the SR-IOV solution, vDPA passthrough can virtualize a physical NIC (PF) into multiple NICs (VFs), and then connect the VFs to the vDPA framework for VMs to use. + +#### Procedure + +To configure vDPA passthrough, perform the following steps as user **root**: + +1. Create and configure VFs. For details, see steps 1 to 3 in SR-IOV passthrough. The following uses **virtio-net** devices as an example (**08:00.6** and **08:00.7** are PFs, and the others are created VFs): + + ```shell + # lspci | grep -i Eth | grep Virtio + 08:00.6 Ethernet controller: Virtio: Virtio network device + 08:00.7 Ethernet controller: Virtio: Virtio network device + 08:01.1 Ethernet controller: Virtio: Virtio network device + 08:01.2 Ethernet controller: Virtio: Virtio network device + 08:01.3 Ethernet controller: Virtio: Virtio network device + 08:01.4 Ethernet controller: Virtio: Virtio network device + 08:01.5 Ethernet controller: Virtio: Virtio network device + 08:01.6 Ethernet controller: Virtio: Virtio network device + 08:01.7 Ethernet controller: Virtio: Virtio network device + 08:02.0 Ethernet controller: Virtio: Virtio network device + 08:02.1 Ethernet controller: Virtio: Virtio network device + 08:02.2 Ethernet controller: Virtio: Virtio network device + ``` + +2. Unbind the VF drivers and bind the vDPA driver of the hardware vendor. + + ```shell + echo 0000:08:01.1 > /sys/bus/pci/devices/0000\:08\:01.1/driver/unbind + echo 0000:08:01.2 > /sys/bus/pci/devices/0000\:08\:01.2/driver/unbind + echo 0000:08:01.3 > /sys/bus/pci/devices/0000\:08\:01.3/driver/unbind + echo 0000:08:01.4 > /sys/bus/pci/devices/0000\:08\:01.4/driver/unbind + echo 0000:08:01.5 > /sys/bus/pci/devices/0000\:08\:01.5/driver/unbind + echo -n "1af4 1000" > /sys/bus/pci/drivers/vender_vdpa/new_id + ``` + +3. After vDPA devices are bound, you can run the `vdpa` command to query the list of devices managed by vDPA. + + ```shell + # vdpa mgmtdev show + pci/0000:08:01.1: + supported_classes net + pci/0000:08:01.2: + supported_classes net + pci/0000:08:01.3: + supported_classes net + pci/0000:08:01.4: + supported_classes net + pci/0000:08:01.5: + supported_classes net + ``` + +4. After the vDPA devices are created, create the vhost-vDPA devices. + + ```shell + vdpa dev add name vdpa0 mgmtdev pci/0000:08:01.1 + vdpa dev add name vdpa1 mgmtdev pci/0000:08:01.2 + vdpa dev add name vdpa2 mgmtdev pci/0000:08:01.3 + vdpa dev add name vdpa3 mgmtdev pci/0000:08:01.4 + vdpa dev add name vdpa4 mgmtdev pci/0000:08:01.5 + ``` + +5. After the vhost-vDPA devices are created, you can run the `vdpa` command to query the vDPA device list or run the `libvirt` command to query the vhost-vDPA device information. + + ```shell + # vdpa dev show + vdpa0: type network mgmtdev pci/0000:08:01.1 vendor_id 6900 max_vqs 3 max_vq_size 256 + vdpa1: type network mgmtdev pci/0000:08:01.2 vendor_id 6900 max_vqs 3 max_vq_size 256 + vdpa2: type network mgmtdev pci/0000:08:01.3 vendor_id 6900 max_vqs 3 max_vq_size 256 + vdpa3: type network mgmtdev pci/0000:08:01.4 vendor_id 6900 max_vqs 3 max_vq_size 256 + vdpa4: type network mgmtdev pci/0000:08:01.5 vendor_id 6900 max_vqs 3 max_vq_size 256 + + # virsh nodedev-list vdpa + vdpa_vdpa0 + vdpa_vdpa1 + vdpa_vdpa2 + vdpa_vdpa3 + vdpa_vdpa4 + + # virsh nodedev-dumpxml vdpa_vdpa0 + + vdpa_vdpa0 + /sys/devices/pci0000:00/0000:00:0c.0/0000:08:01.1/vdpa0 + pci_0000_08_01_1 + + vhost_vdpa + + + /dev/vhost-vdpa-0 + + + ``` + +6. Mount a vDPA device to the VM. + + When creating a VM, add the item for the vDPA passthrough device to the VM configuration file: + + ```xml + + + + + + ``` + + **Table 4** vDPA configuration description + + | Parameter | Description | Value | + | ------------------ | ---------------------------------------------------- | ----------------- | + | hostdev.source.dev | Path of the vhost-vDPA character device on the host. | /dev/vhost-vdpa-x | + + > [!NOTE]NOTE + > The procedures of creating and configuring VFs and binding the vDPA drivers vary with the design of hardware vendors. Follow the procedure of the corresponding vendor. + +## Managing VM USB + +To facilitate the use of USB devices such as USB key devices and USB mass storage devices on VMs, openEuler provides the USB device passthrough function. Through USB passthrough and hot-swappable interfaces, you can configure USB passthrough devices for VMs, or hot swap USB devices when VMs are running. + +### Configuring USB Controllers + +#### Overview + +A USB controller is a virtual controller that provides specific USB functions for USB devices on VMs. To use USB devices on a VM, you must configure USB controllers for the VM. Currently, openEuler supports the following types of USB controllers: + +- Universal host controller interface (UHCI): also called the USB 1.1 host controller specification. +- Enhanced host controller interface (EHCI): also called the USB 2.0 host controller specification. +- Extensible host controller interface (xHCI): also called the USB 3.0 host controller specification. + +#### Precautions + +- The host server must have USB controller hardware and modules that support USB 1.1, USB 2.0, and USB 3.0 specifications. +- You need to configure USB controllers for the VM by following the order of USB 1.1, USB 2.0, and USB 3.0. +- An xHCI controller has eight ports and can be mounted with a maximum of four USB 3.0 devices and four USB 2.0 devices. An EHCI controller has six ports and can be mounted with a maximum of six USB 2.0 devices. A UHCI controller has two ports and can be mounted with a maximum of two USB 1.1 devices. +- On each VM, only one USB controller of the same type can be configured. +- USB controllers cannot be hot swapped. +- If the USB 3.0 driver is not installed on a VM, the xHCI controller may not be identified. For details about how to download and install the USB 3.0 driver, refer to the official description provided by the corresponding OS distributor. +- To ensure the compatibility of the OS, set the bus ID of the USB controller to **0** when configuring a USB tablet for the VM. The tablet is mounted to the USB 1.1 controller by default. + +#### Configuration Methods + +The following describes the configuration items of USB controllers for a VM. You are advised to configure USB 1.1, USB 2.0, and USB 3.0 to ensure the VM is compatible with three types of devices. + +The configuration item of the USB 1.1 controller (UHCI) in the XML configuration file is as follows: + +```xml + + +``` + +The configuration item of the USB 2.0 controller (EHCI) in the XML configuration file is as follows: + +```xml + + +``` + +The configuration item of the USB 3.0 controller (xHCI) in the XML configuration file is as follows: + +```xml + + +``` + +### Configuring a USB Passthrough Device + +#### Overview + +After USB controllers are configured for a VM, a physical USB device on the host can be mounted to the VM through device passthrough for the VM to use. In the virtualization scenario, in addition to static configuration, hot swapping the USB device is supported. That is, the USB device can be mounted or unmounted when the VM is running. + +#### Precautions + +- A USB device can be assigned to only one VM. +- A VM with a USB passthrough device does not support live migration. +- VM creation fails if no USB passthrough devices exist in the VM configuration file. +- Forcibly hot removing a USB storage device that is performing read or write operation may damage files in the USB storage device. + +#### Configuration Description + +The following describes the configuration items of a USB device for a VM. + +Description of the USB device in the XML configuration file: + +```xml + + +
+ +
+ +``` + +- **
**: *m_ indicates the USB bus address on the host, and _n* indicates the device ID. +- **
**: indicates that the USB device is to be mounted to the USB controller specified on the VM. *x_ indicates the controller ID, which corresponds to the index number of the USB controller configured on the VM. _y* indicates the port address. When configuring a USB passthrough device, you need to set this parameter to ensure that the controller to which the device is mounted is as expected. + +#### Configuration Methods + +To configure USB passthrough, perform the following steps: + +1. Configure USB controllers for the VM. For details, see [Configuring USB Controllers](#configuring-usb-controllers). +2. Query information about the USB device on the host. + + Run the **lsusb** command (the **usbutils** software package needs to be installed) to query the USB device information on the host, including the bus address, device address, device vendor ID, device ID, and product description. For example: + + ```shell + # lsusb + ``` + + ```text + Bus 008 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub + Bus 007 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub + Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub + Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub + Bus 006 Device 002: ID 0bda:0411 Realtek Semiconductor Corp. + Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub + Bus 005 Device 003: ID 136b:0003 STEC + Bus 005 Device 002: ID 0bda:5411 Realtek Semiconductor Corp. + Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub + Bus 001 Device 003: ID 12d1:0003 Huawei Technologies Co., Ltd. + Bus 001 Device 002: ID 0bda:5411 Realtek Semiconductor Corp. + Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub + Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub + ``` + +3. Prepare the XML description file of the USB device. Before hot removing the device, ensure that the USB device is not in use. Otherwise, data may be lost. +4. Run the hot swapping commands. + + Take a VM whose name is **openEulerVM** as an example. The corresponding configuration file is **usb.xml**. + + - Hot adding of the USB device takes effect only for the current running VM. After the VM is restarted, hot add the USB device again. + + ```shell + # virsh attach-device openEulerVM usb.xml --live + ``` + + - Complete persistency configurations for hot adding of the USB device. After the VM is restarted, the USB device is automatically assigned to the VM. + + ```shell + # virsh attach-device openEulerVM usb.xml --config + ``` + + - Hot removing of the USB device takes effect only for the current running VM. After the VM is restarted, the USB device with persistency configurations is automatically assigned to the VM. + + ```shell + # virsh detach-device openEulerVM usb.xml --live + ``` + + - Complete persistency configurations for hot removing of the USB device. + + ```shell + # virsh detach-device openEulerVM usb.xml --config + ``` + +## Storing Snapshots + +### Overview + +The VM system may be damaged due to virus damage, system file deletion by mistake, or incorrect formatting. As a result, the system cannot be started. To quickly restore a damaged system, openEuler provides the storage snapshot function. openEuler can create a snapshot that records the VM status at specific time points without informing users (usually within a few seconds). The snapshot can be used to restore the VM to the status when the snapshots were taken. For example, a damaged system can be quickly restored with the help of snapshots, which improves system reliability. + +> [!NOTE]NOTE +> Currently, storage snapshots can be QCOW2 and RAW images only. Block devices are not supported. + +### Procedure + +To create VM storage snapshots, perform the following steps: + +1. Log in to the host and run the **virsh domblklist** command to query the disk used by the VM. + + ```shell + # virsh domblklist openEulerVM + Target Source + --------------------------------------------- + vda /mnt/openEuler-image.qcow2 + ``` + +2. Run the following command to create the VM disk snapshot **openEuler-snapshot1.qcow2**: + + ```shell + # virsh snapshot-create-as --domain openEulerVM --disk-only --diskspec vda,snapshot=external,file=/mnt/openEuler-snapshot1.qcow2 --atomic + Domain snapshot 1582605802 created + ``` + +3. Run the following command to query disk snapshots: + + ```shell + # virsh snapshot-list openEulerVM + Name Creation Time State + --------------------------------------------------------- + 1582605802 2020-02-25 12:43:22 +0800 disk-snapshot + ``` + +## Configuring Disk I/O Suspension + +### Introduction + +#### Overview + +When a storage fault occurs (for example, the storage link is disconnected), the I/O error of the physical disk is sent to the VM front end through the virtualization layer. The VM receives the I/O error. As a result, the user file system in the VM may change to the read-only state. In this case, the VM needs to be restarted or the user needs to manually restore the file system, which brings extra workload to users. + +In this case, the virtualization platform provides the disk I/O suspension capability. That is, when the storage device is faulty, the VM I/Os are suspended when being delivered to the host. During the suspension period, no I/O error is returned to the VM. In this way, the file system in the VM does not change to the read-only state due to I/O errors. Instead, the file system is suspended. At the same time, the VM backend retries I/Os based on the specified suspension interval. If the storage fault is rectified within the suspension period, the suspended I/Os can be flushed to disks, and the internal file system of the VM automatically recovers without restarting the VM. If the storage fault is not rectified within the suspension period, an error is reported to the VM and the user is notified. + +#### Application Scenarios + +A cloud disk that may cause storage plane link disconnection is used as the backend of the virtual disk. + +#### Precautions and Restrictions + +- Only virtio-blk or virtio-scsi virtual disks support disk I/O suspension. + +- Generally, the backend of the virtual disk where the disk I/Os are suspended is the cloud disk whose storage plane link may be disconnected. + +- Disk I/O suspension can be enabled for read and write I/O errors separately. The retry interval and timeout interval for read and write I/O errors of the same disk are the same. + +- The disk I/O suspension retry interval does not include the actual read/write I/O overhead on the host. That is, the actual interval between two I/O retries is greater than the configured I/O error retry interval. + +- Disk I/O suspension cannot distinguish the specific type of I/O errors (such as storage link disconnection, bad sector, or reservation conflict). As long as the hardware returns an I/O error, suspension is performed. + +- When disk I/O suspension occurs, the internal I/Os of the VM are not returned, the system commands for accessing the disk, such as fdisk, are suspended, and the services that depend on the returned commands are suspended. + +- When disk I/O suspension occurs, the I/Os cannot be flushed to the disk. As a result, the VM may not be gracefully shut down. In this case, you need to forcibly shut down the VM. + +- When disk I/O suspension occurs, the disk data cannot be read. As a result, the VM cannot be restarted. In this case, you need to forcibly stop the VM, wait until the storage fault is rectified, and then restart the VM. + +- After a storage fault occurs, the following problems cannot be solved although disk I/O suspension occurs: + + 1. Failed to execute advanced storage features. + + Advanced features include: virtual disk hot swap, virtual disk creation, VM startup, VM shutdown, VM forcible shutdown, VM hibernation, VM wakeup, VM storage live migration, VM storage live migration cancellation, VM storage snapshot creation, VM storage snapshot combination, VM disk capacity query, online disk capacity expansion, virtual CD-ROM drive insertion, and CD-ROM drive ejection from the VM. + + 2. Failed to execute the VM lifecycle. + +- When a live migration is initiated for a VM configured with disk I/O suspension, the disk I/O suspension configuration must be the same as that of the source host in the XML configuration of the destination disk. + +### Disk I/O Suspension Configuration + +#### Using the QEMU CLI + +To enable disk I/O suspension, configure `werror=retry` `rerror=retry` on the virtual disk device. To configure the retry policy, configure `retry_interval` and `retry_timeout`. `retry_interval` indicates the I/O retry interval. The value ranges from 0 to `MAX_LONG`, in milliseconds. If this parameter is not set, the default value 1000 ms is used. `retry_timeout` indicates the I/O retry timeout interval. The value ranges from 0 to `MAX_LONG`, in milliseconds. The value **0** indicates that no timeout occurs. If this parameter is not set, the default value **0** is used. + +The disk I/O suspension configuration of the virtio-blk disk is as follows: + +```shell +-drive file=/path/to/your/storage,format=raw,if=none,id=drive-virtio-disk0,cache=none,aio=native \ +-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,\ +drive=drive-virtio-disk0,id=virtio-disk0,write-cache=on,\ +werror=retry,rerror=retry,retry_interval=2000,retry_timeout=10000 +``` + +The disk I/O suspension configuration of the virtio-scsi disk is as follows: + +```shell +-drive file=/path/to/your/storage,format=raw,if=none,id=drive-scsi0-0-0-0,cache=none,aio=native \ +-device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,\ +device_id=drive-scsi0-0-0-0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,write-cache=on,\ +werror=retry,rerror=retry,retry_interval=2000,retry_timeout=10000 +``` + +#### Using an XML Configuration File + +To enable disk I/O suspension, configure `error_policy='retry'` `rerror_policy='retry'` in the disk XML configuration. Configure `retry_interval` and `retry_timeout`. retry_interval indicates the I/O retry interval. The value ranges from 0 to `MAX_LONG`, in milliseconds. If this parameter is not set, the default value 1000 ms is used. retry_timeout indicates the I/O retry timeout interval. The value ranges from 0 to `MAX_LONG`, in milliseconds. The value **0** indicates that no timeout occurs. If this parameter is not set, the default value **0** is used. + +The disk I/O suspension XML configuration of the virtio-blk disk is as follows: + +```xml + + + + + + +``` + +The disk I/O suspension XML configuration of the virtio-scsi disk is as follows: + +```xml + + + + + +
+ +``` diff --git a/docs/en/virtualization/virtualization_platform/virtualization/managing-vms.md b/docs/en/virtualization/virtualization_platform/virtualization/managing-vms.md new file mode 100644 index 0000000000000000000000000000000000000000..e20840afb5f6170b7ef3681ac6ec39dc3eb48944 --- /dev/null +++ b/docs/en/virtualization/virtualization_platform/virtualization/managing-vms.md @@ -0,0 +1,801 @@ +# Managing VMs + +- [Managing VMs](#managing-vms) + - [VM Life Cycle](#vm-life-cycle) + - [Introduction](#introduction) + - [Management Commands](#management-commands) + - [Example](#example) + - [Modifying VM Configurations Online](#modifying-vm-configurations-online) + - [Querying VM Information](#querying-vm-information) + - [Logging In to a VM](#logging-in-to-a-vm) + - [Logging In Using VNC Passwords](#logging-in-using-vnc-passwords) + - [Configuring VNC TLS Login](#configuring-vnc-tls-login) + - [VM Secure Boot](#vm-secure-boot) + - [General Introduction](#general-introduction) + - [Secure Boot Practice](#secure-boot-practice) + +## VM Life Cycle + +### Introduction + +#### Overview + +To leverage hardware resources and reduce costs, users need to properly manage VMs. This section describes basic operations during the VM lifecycle, such as creating, using, and deleting VMs. + +#### VM Status + +A VM can be in one of the following status: + +- **undefined**: The VM is not defined or created. That is, libvirt considers that the VM does not exist. +- **shut off**: The VM has been defined but is not running, or the VM is terminated. +- **running**: The VM is running. +- **paused**: The VM is suspended and its running status is temporarily stored in the memory. The VM can be restored to the running status. +- **saved**: Similar to the **paused** status, the running state is stored in a persistent storage medium and can be restored to the running status. +- **crashed**: The VM crashes due to an internal error and cannot be restored to the running status. + +#### Status Transition + +VMs in different status can be converted, but certain rules must be met. [Figure 1](#fig671014583483) describes the common rules for transiting the VM status. + +**Figure 1** Status transition diagram +![](./figures/status-transition-diagram.png) + +#### VM ID + +In libvirt, a created VM instance is called a **domain**, which describes the configuration information of resources such as the CPU, memory, network device, and storage device of the VM. On a host, each domain has a unique ID, which is represented by the VM **Name**, **UUID**, and **Id**. For details, see [Table 1](#table84397266483). During the VM lifecycle, an operation can be performed on a specific VM by using a VM ID. + +**Table 1** Domain ID description + + + + + + + + + + + + + + + +

ID

+

Description

+

Name

+

VM name

+

UUID

+

Universally unique identifier

+

Id

+

VM running ID

+
NOTE:

The ID is not displayed for a powered off VM.

+
+
+ +> [!NOTE]NOTE +> Run the **virsh** command to query the VM ID and UUID. For details, see [Querying VM Information](#querying-vm-information). + +### Management Commands + +#### Overview + +You can use the **virsh** command tool to manage the VM lifecycle. This section describes the commands related to the lifecycle. + +#### Prerequisites + +- Before performing operations on a VM, you need to query the VM status to ensure that the operations can be performed. For details about the conversion between status, see [Status Transition](#status-transition). +- You have administrator rights. +- The VM XML configuration files are prepared. + +#### Command Usage + +You can run the **virsh** command to manage the VM lifecycle. The command format is as follows: + +```shell +virsh +``` + +The parameters are described as follows: + +- _operate_: manages VM lifecycle operations, such as creating, deleting, and starting VMs. +- _obj_: specifies the operation object, for example, the VM to be operated. +- _options_: command option. This parameter is optional. + +[Table 2](#table389518422611) describes the commands used for VM lifecycle management. _VMInstance_ indicates the VM name, VM ID, or VM UUID, _XMLFile_ indicates the XML configuration file of the VM, and _DumpFile_ indicates the dump file. Change them based on the site requirements. + +**Table 2** VM Lifecycle Management Commands + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Command

+

Description

+

virsh define <XMLFile>

+

Define a persistent VM. After the definition is complete, the VM is shut down and is considered as a domain instance.

+

virsh create <XMLFile>

+

Create a temporary VM. After the VM is created, it is in the running status.

+

virsh start <VMInstance>

+

Start the VM.

+

virsh shutdown <VMInstance>

+

Shut down the VM. Start the VM shutdown process. If the VM fails to be shut down, forcibly stop it.

+

virsh destroy <VMInstance>

+

Forcibly stop the VM.

+

virsh reboot <VMInstance>

+

Reboot the VM.

+

virsh save <VMInstance> <DumpFile>

+

Dump the VM running status to a file.

+

virsh restore <DumpFile>

+

Restore the VM from the VM status dump file.

+

virsh suspend <VMInstance>

+

Suspend the VM to make the VM in the paused status.

+

virsh resume <VMInstance>

+

Resume the VM and restore the VM in the paused status to the running status.

+

virsh undefine <VMInstance>

+

After a persistent VM is destroyed, the VM lifecycle ends and no more operations can be performed on the VM.

+
+ +### Example + +This section provides examples of commands related to VM life cycle management. + +- Create a VM. + + The VM XML configuration file is **openEulerVM.xml**. The command and output are as follows: + + ```shell + $ virsh define openEulerVM.xml + Domain openEulerVM defined from openEulerVM.xml + ``` + +- Start a VM. + + Run the following command to start the _openEulerVM_: + + ```shell + $ virsh start openEulerVM + Domain openEulerVM started + ``` + +- Reboot a VM. + + Run the following command to reboot the _openEulerVM_: + + ```shell + $ virsh reboot openEulerVM + Domain openEulerVM is being rebooted + ``` + +- Shut down a VM. + + Run the following command to shut down the _openEulerVM_: + + ```shell + $ virsh shutdown openEulerVM + Domain openEulerVM is being shutdown + ``` + +- Destroy a VM. + - If the **nvram** file is not used during the VM startup, run the following command to destroy the VM: + + ```shell + virsh undefine + ``` + + - If the **nvram** file is used during the VM startup, run the following command to specify the **nvram** processing policy when destroying the VM: + + ```shell + virsh undefine + ``` + + _strategy_ indicates the policy for destroying a VM. The values can be: + + --**nvram**: delete the corresponding **nvram** file when destroying a VM. + + --**keep-nvram**: destroy a VM but retain the corresponding **nvram** file. + + For example, to delete the _openEulerVM_ and its **nvram** file, run the following command: + + ```shell + $ virsh undefine openEulerVM --nvram + Domain openEulerVM has been undefined + ``` + +## Modifying VM Configurations Online + +### Overview + +After a VM is created, users can modify VM configurations. This process is called online modification of VM configuration. After the configuration is modified online, the new VM configuration file is persistent and takes effect after the VM is shut down and restarted. + +The format of the command for modifying VM configuration is as follows: + +```shell +virsh edit +``` + +The **virsh edit** command is used to edit the XML configuration file corresponding to **domain** to update VM configuration. **virsh edit** uses the **vi** program as the default editor. You can specify the editor type by modifying the environment variable _EDITOR_ or _VISUAL_. By default, **virsh edit** preferentially uses the text editor specified by the environment variable _VISUAL_. + +### Procedure + +1. \(Optional\) Set the editor of the **virsh edit** command to **vim**. + + ```shell + export VISUAL=vim + ``` + +2. Run the **virsh edit** command to open the XML configuration file of the _openEulerVM_. + + ```shell + virsh edit openEulerVM + ``` + +3. Modify the VM configuration file. +4. Save the VM configuration file and exit. +5. Shut down the VM. + + ```shell + virsh shutdown openEulerVM + ``` + +6. Start the VM for the modification to take effect. + + ```shell + virsh start openEulerVM + ``` + +## Querying VM Information + +### Overview + +The libvirt provides a set of command line tools to query VM information. This section describes how to use commands to obtain VM information. + +### Prerequisites + +To query VM information, the following requirements must be met: + +- The libvirtd service is running. + +- Only the administrator has the permission to execute command line. + +### Querying VM Information on a Host + +- Query the list of running and paused VMs on a host. + + ```shell + virsh list + ``` + + For example, the following command output indicates that three VMs exist on the host. **openEulerVM01** and **openEulerVM02** are running, and **openEulerVM03** is paused. + + ```text + Id Name State + ---------------------------------------------------- + 39 openEulerVM01 running + 40 openEulerVM02 running + 69 openEulerVM03 paused + ``` + +- Query the list of VM information defined on a host. + + ```shell + virsh list --all + ``` + + For example, the following command output indicates that four VMs are defined on the current host. **openEulerVM01** is running, **openEulerVM02** is paused, and **openEulerVM03** and **openEulerVM04** are shut down. + + ```text + Id Name State + ---------------------------------------------------- + 39 openEulerVM01 running + 69 openEulerVM02 paused + - openEulerVM03 shut off + - openEulerVM04 shut off + ``` + +### Querying Basic VM Information + +Libvirt component provides a group of commands for querying the VM status, including the VM running status, device information, and scheduling attributes. For details, see [Table 3](#table10582103963816). + +**Table 3** Querying basic VM information + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Information to be queried

+

Command line

+

Description

+

Basic information

+

virsh dominfo <VMInstance>

+

The information includes the VM ID, UUID, and VM specifications.

+

Current status

+

virsh domstate <VMInstance>

+

You can use the --reason option to query the reason why the VM changes to the current status.

+

Scheduling information

+

virsh schedinfo <VMInstance>

+

The information includes the vCPU share.

+

Number of vCPUs

+

virsh vcpucount <VMInstance>

+

Number of vCPUs of the VM.

+

Virtual block device status

+

virsh domblkstat <VMInstance>

+

To query the name of a block device, run the virsh domblklist command.

+

vNIC status

+

virsh domifstat <VMInstance>

+

To query the NIC name, run the virsh domiflist command.

+

I/O thread

+

virsh iothreadinfo <VMInstance>

+

VM I/O thread and CPU affinity.

+
+ +### Example + +- Run the **virsh dominfo** command to query the basic information about a created VM. The query result shows that the VM ID is **5**, UUID is **ab472210-db8c-4018-9b3e-fc5319a769f7**, memory size is 8 GiB, and the number of vCPUs is 4. + + ```shell + $ virsh dominfo openEulerVM + Id: 5 + Name: openEulerVM + UUID: ab472210-db8c-4018-9b3e-fc5319a769f7 + OS Type: hvm + State: running + CPU(s): 4 + CPU time: 6.8s + Max memory: 8388608 KiB + Used memory: 8388608 KiB + Persistent: no + Autostart: disable + Managed save: no + Security model: none + Security DOI: 0 + ``` + +- Run the **virsh domstate** command to query the VM status. The query result shows that VM **openEulerVM** is running. + + ```shell + $ virsh domstate openEulerVM + running + ``` + +- Run **virsh schedinfo** to query the VM scheduling information. The query result shows that the CPU reservation share of the VM is 1024. + + ```shell + $ virsh schedinfo openEulerVM + Scheduler : posix + cpu_shares : 1024 + vcpu_period : 100000 + vcpu_quota : -1 + emulator_period: 100000 + emulator_quota : -1 + global_period : 100000 + global_quota : -1 + iothread_period: 100000 + iothread_quota : -1 + ``` + +- Run the **virsh vcpucount** command to query the number of vCPUs. The query result shows that the VM has four CPUs. + + ```shell + $ virsh vcpucount openEulerVM + maximum live 4 + current live 4 + ``` + +- Run the **virsh domblklist** command to query the VM disk information. The query result shows that the VM has two disks. sda is a virtual disk in qcow2 format, and sdb is a cdrom device. + + ```shell + $ virsh domblklist openEulerVM + Target Source + --------------------------------------------------------------------- + sda /home/openeuler/vm/openEuler_aarch64.qcow2 + sdb /home/openeuler/vm/openEuler-23.09-aarch64-dvd.iso + ``` + +- Run the **virsh domiflist** command to query the VM NIC information. The query result shows that the VM has one NIC, the backend is vnet0, which is on the br0 bridge of the host. The MAC address is 00:05:fe:d4:f1:cc. + + ```shell + $ virsh domiflist openEulerVM + Interface Type Source Model MAC + ------------------------------------------------------- + vnet0 bridge br0 virtio 00:05:fe:d4:f1:cc + ``` + +- Run the **virsh iothreadinfo** command to query the VM I/O thread information. The query result shows that the VM has five I/O threads, which are scheduled on physical CPUs 7-10. + + ```shell + $ virsh iothreadinfo openEulerVM + IOThread ID CPU Affinity + --------------------------------------------------- + 3 7-10 + 4 7-10 + 5 7-10 + 1 7-10 + 2 7-10 + ``` + +## Logging In to a VM + +This section describes how to log in to a VM using VNC. + +### Logging In Using VNC Passwords + +#### Overview + +After the OS is installed on a VM, you can remotely log in to the VM using VNC to manage the VM. + +#### Prerequisites + +Before logging in to a VM using a client, such as RealVNC or TightVNC, ensure that: + +- You have obtained the IP address of the host where the VM resides. +- The environment where the client resides can access the network of the host. +- You have obtained the VNC listening port of the VM. This port is automatically allocated when the client is started. Generally, the port number is **5900 + x** \(_x_ is a positive integer and increases in ascending order based on the VM startup sequence. **5900** is invisible to users.\) +- If a password has been set for the VNC, you also need to obtain the VNC password of the VM. + + > [!NOTE]NOTE + > To set a password for the VM VNC, edit the XML configuration file of the VM. That is, add the **passwd** attribute to the **graphics** element and set the attribute value to the password to be configured. For example, to set the VNC password of the VM to **n8VfjbFK**, configure the XML file as follows: + + ```xml + + + + ``` + +#### Procedure + +#### Procedure + +1. Query the VNC port number used by the VM. For example, if the VM name is _openEulerVM_, run the following command: + + ```shell + $ virsh vncdisplay openEulerVM + :3 + ``` + + > [!NOTE]NOTE + > To log in to the VNC, you need to configure firewall rules to allow the connection of the VNC port. The reference command is as follows, where _X_ is **5900 + Port number**, for example, **5903**. + + ```shell + firewall-cmd --zone=public --add-port=X/tcp + ``` + +2. Start the VncViewer software and enter the IP address and port number of the host. The format is **host IP address:port number**, for example, **10.133.205.53:3**. +3. Click **OK** and enter the VNC password \(optional\) to log in to the VM VNC. + +### Configuring VNC TLS Login + +#### Overview + +By default, the VNC server and client transmit data in plaintext. Therefore, the communication content may be intercepted by a third party. To improve security, openEuler allows the VNC server to configure the Transport Layer Security \(TLS\) mode for encryption and authentication. TLS implements encrypted communication between the VNC server and client to prevent communication content from being intercepted by third parties. + +> [!NOTE]NOTE +> +> - To use the TLS encryption authentication mode, the VNC client must support the TLS mode \(for example, TigerVNC\). Otherwise, the VNC client cannot be connected. +> - The TLS encryption authentication mode is configured at the host level. After this feature is enabled, the TLS encryption authentication mode is enabled for the VNC clients of all VMs running on the host. + +#### Procedure + +To enable the TLS encryption authentication mode for the VNC, perform the following steps: + +1. Log in to the host where the VNC server resides, and edit the corresponding configuration items in the **/etc/libvirt/qemu.conf** configuration file of the server. The configuration is as follows: + + ```conf + vnc_listen = "x.x.x.x" # "x.x.x.x" indicates the listening IP address of the VNC. Set this parameter based on the site requirements. The VNC server allows only the connection requests from clients whose IP addresses are in this range. + vnc_tls = 1 # If this parameter is set to 1, VNC TLS is enabled. + vnc_tls_x509_cert_dir = "/etc/pki/libvirt-vnc" # Specify /etc/pki/libvirt-vnc as the path for storing the certificate. + vnc_tls_x509_verify = 1 #If this parameter is set to 1, the X509 certificate is used for TLS authentication. + ``` + +2. Create a certificate and a private key file for the VNC. The following uses GNU TLS as an example. + + > [!NOTE]NOTE + > To use GNU TLS, install the gnu-utils software package in advance. + + 1. Create a certificate file issued by the Certificate Authority \(CA\). + + ```shell + certtool --generate-privkey > ca-key.pem + ``` + + 2. Create a self-signed public and private key for the CA certificate. _Your organization name_ indicates the organization name, which is specified by the user. + + ```shell + $ cat > ca.info< server.info< server-key.pem + certtool --generate-certificate \ + --load-ca-certificate ca-cert.pem \ + --load-ca-privkey ca-key.pem \ + --load-privkey server-key.pem \ + --template server.info \ + --outfile server-cert.pem + ``` + + In the preceding generated file, **server-key.pem** is the private key of the VNC server, and **server-cert.pem** is the public key of the VNC server. + + 4. Issue a certificate to the VNC client. + + ```shell + $ cat > client.info< client-key.pem + certtool --generate-certificate \ + --load-ca-certificate ca-cert.pem \ + --load-ca-privkey ca-key.pem \ + --load-privkey client-key.pem \ + --template client.info \ + --outfile client-cert.pem + ``` + + In the preceding generated file, **client-key.pem** is the private key of the VNC client, and **client-cert.pem** is the public key of the VNC client. The generated public and private key pairs need to be copied to the VNC client. + +3. Shut down the VM to be logged in to and restart the libvirtd service on the host where the VNC server resides. + + ```shell + systemctl restart libvirtd + ``` + +4. Save the generated server certificate to the specified directory on the VNC server and grant the read and write permissions on the certificate only to the current user. + + ```shell + sudo mkdir -m 750 /etc/pki/libvirt-vnc + cp ca-cert.pem /etc/pki/libvirt-vnc/ca-cert.pem + cp server-cert.pem /etc/pki/libvirt-vnc/server-cert.pem + cp server-key.pem /etc/pki/libvirt-vnc/server-key.pem + chmod 0600 /etc/pki/libvirt-vnc/* + ``` + +5. Copy the generated client certificates **ca-cert.pem**, **client-cert.pem**, and **client-key.pem** to the VNC client. After the TLS certificate of the VNC client is configured, you can use VNC TLS to log in to the VM. + + > [!NOTE]NOTE + > - For details about how to configure the VNC client certificate, see the usage description of each client. + > - For details about how to log in to the VM, see [Logging In Using VNC Passwords](#logging-in-using-vnc-passwords). + +## VM Secure Boot + +### General Introduction + +#### Overview + +Secure boot uses public and private key pairs to sign and validate boot components. During the startup, the previous component validates the digital signature of the next component. If the validation is successful, the next component starts. If the validation fails, the startup fails. Secure boot is used to detect whether the firmware and software during startup of the device are tampered with to prevent malware from intrusion and modification. Secure boot ensures the integrity of each component during system startup and prevents unauthorized components from being loaded and running, thereby preventing security threats to the system and user data. Secure boot is implemented based on the UEFI boot mode. It is not supported by the legacy boot mode. According to UEFI specifications, some reliable public keys can be built in the mainboard before delivery. Any operating system or hardware drivers that you want to load on this mainboard must be authenticated by these public keys. The secure boot of a physical machine is implemented by the physical BIOS, while the secure boot of a VM is simulated by software. The process of the VM secure boot is the same as that of the host secure boot, both complying with the open-source UEFI specifications. The UEFI on the virtualization platform is provided by the edk component. When a VM starts, QEMU maps the UEFI image to the memory to simulate the firmware startup process for the VM. Secure boot is a security protection capability provided by edk during the VM startup to protect the OS kernel of the VM from being tampered with. The sequence of signature validation for the secure boot is as follows: UEFI BIOS->shim->GRUB->vmlinuz (signature validation is passed and loaded in sequence). + +| English | Acronyms and Abbreviations | Description | +| :----- | :----- | :----- | +| Secure boot | - | Secure boot indicates that a component validates the digital signature of the next component during startup. If the validation is successful, the component runs. If the validation fails, the component stops running. It ensures the integrity of each component during system startup. | +| Platform key | PK | Platform key is owned by the OEM vendor and must be RSA2048 or stronger. The PK establishes a trusted relationship between the platform owner and the platform firmware. The platform owner registers the PKpub, public key of the PK, with the platform firmware. The platform owner can use the PKpriv, private part of the PK, to change the ownership of the platform or register the KEK key. | +| Key exchange key | KEK | Key exchange key creates a trusted relationship between the platform firmware and the OS. Each OS and third-party application that communicates with the platform firmware register the KEKpub, public part of the KEK key, in the platform firmware. | +| Database trustlist | DB | Database trustlist stores and validates the keys of components such as shim, GRUB, and vmlinuz. | +| Database blocklist | DBx | Database blocklist stores revoked keys. | + +#### Function Description + +The VM secure boot feature is implemented based on the edk open-source project. In non-secure boot mode, the basic Linux process is as follows: + +**Figure 2** System startup process + +![](./figures/OSBootFlow.png) + +In secure boot mode, the first component loaded after UEFI BIOS starts is shim in the system image. By interacting with UEFI BIOS, shim obtains the key stored in the variable DB of UEFI BIOS to validate GRUB. After GRUB is loaded, the key and the authentication API are also called to validate the kernel. The Linux boot process is as follows: + +**Figure 3** Secure boot process + +![](./figures/SecureBootFlow.png) + +The secure boot feature involves multiple key scenarios. Based on the scenario analysis and system breakdown, the secure boot feature involves the following subsystems: UEFI BIOS validating shim, shim validating GRUB, and GRUB validating kernel. When UEFI BIOS validates shim, if the validation is successful, shim is started. If the validation fails, an error message is displayed and shim fails to start. Shim needs to use the private key for signature during image compilation and creation, and the public key certificate needs to be imported to the variable area DB of UEFI BIOS. After shim is started, validate the startup of GRUB. If the validation is successful, GRUB is started. If the validation fails, an error message is displayed and GRUB fails to start. GRUB needs to be signed during image compilation and creation. The public and private key pairs are the same as those of shim. After GRUB is started, it calls the key and the authentication API key registered in UEFI BIOS to validate the kernel. If the validation is successful, GRUB starts the kernel. If the validation fails, an error message is displayed. GRUB needs to sign the image during compilation and creation and uses the public and private key pair that is the same as that of shim. + +#### Constraints + +- Running on the UEFI BIOS that does not support secure boot does not affect existing functions and services. +- The secure boot feature depends on the UEFI BIOS and takes effect only when the UEFI supports this feature. +- When secure boot is enabled in the UEFI BIOS, the system cannot be started if the related components have no signature or the signature is incorrect. +- If secure boot is disabled in the UEFI BIOS, the validation function during the boot process is disabled. +- The second half of the secure boot validation chain, that is, shim->GRUB->kernel, guides the kernel to start. This part of the validation chain is implemented by the OS image. If the OS does not support guiding the kernel for secure boot, the VM secure boot fails. +- Currently, the x86 architecture do not provide nvram file configuration to configure VM secure boot. + +### Secure Boot Practice + +VM secure boot depends on UEFI BIOS. The UEFI BIOS image is installed using the edk rpm package. This section uses AArch64 as an example to describe how to configure VM secure boot. + +#### Configuring VM + +The components in the edk rpm package are installed in the /usr/share/edk2/aarch64 directory, including `QEMU_EFI-pflash.raw` and `vars-template-pflash.raw`. The following describes the XML configuration of the UEFI BIOS during VM startup. + +```xml + + hvm + /usr/share/edk2/aarch64/QEMU_EFI-pflash.raw + /path/to/QEMU-VARS.fd + +``` + +In the preceding configuration, /usr/share/edk2/aarch64/QEMU_EFI-pflash.raw indicates the path of the UEFI BIOS image. /usr/share/edk2/aarch64/vars-template-pflash.raw is the path of the NVRAM image template, and /path/to/QEMU-VARS.fd is the path of the NVRAM image file of the current VM, which is used to store environment variables in the UEFI BIOS. + +#### Importing Certificate + +The certificate for VM secure boot is imported from the BIOS page. Before importing the certificate, you need to import the certificate file to the VM. You can mount the directory where the certificate file is located to the VM by mounting a disk. For example, you can create an image that contains the certificate and mount the image in the XML configuration file of the VM. + +Create a certificate file image. + +```shell +dd of='/path/to/data.img' if='/dev/zero' bs=1M count=64 +mkfs.vfat -I /path/to/data.img +mkdir /path/to/mnt +mount path/to/data.img /path/to/mnt/ +cp -a /path/to/certificates/* /path/to/mnt/ +umount /path/to/mnt/ +``` + +In the preceding command, /path/to/certificates/ indicates the path where the certificate file is located, /path/to/data.img indicates the path where the certificate file image is located, and /path/to/mnt/ indicates the image mounting path. + +Mount the image in the XML file of the VM. + +```xml + + + + + + + + + +``` + +Start the VM and import the PK certificate. The procedure is as follows (the procedure for importing the KEK certificate is the same as that for importing the DB certificate): + +After the VM is started, press F2 to go to the BIOS screen. + +**Figure 4** BIOS screen + +![](./figures/CertEnrollP1.png) + +**Figure 5** Device Manager + +![](./figures/CertEnrollP2.png) + +**Figure 6** Custom Secure Boot Options + +![](./figures/CertEnrollP3.png) + +**Figure 7** PK Options + +![](./figures/CertEnrollP4.png) + +**Figure 8** Enrolling PK + +![](./figures/CertEnrollP5.png) + +In the File Explorer window, many disk directories are displayed, including the certificate file directory mounted through the disk. + +**Figure 9** File Explorer + +![](./figures/CertEnrollP6.png) + +Select the PK certificate to be imported in the disk directory. + +**Figure 10** Disk where the certificate is stored + +![](./figures/CertEnrollP7.png) + +**Figure 11** Selecting **Commit Changes and Exit** to save the imported certificate + +![](./figures/CertEnrollP8.png) + +After the certificate is imported, the UEFI BIOS writes the certificate information and secure boot attributes to the NVRAM configuration file /path/to/QEMU-VARS.fd. Upon the next startup, the VM reads related configurations from the /path/to/QEMU-VARS.fd file, initializes certificate information and secure boot attributes, automatically imports the certificate, and enables secure boot. Similarly, you can use /path/to/QEMU-VARS.fd as the UEFI BIOS boot configuration template file of other VMs with the same configuration. Modify the nvram template field so that the certificate is automatically imported and the secure boot option is enabled when other VMs are started. The VM XML configuration is modified as follows: + +```xml + + hvm + /usr/share/edk2/aarch64/QEMU_EFI-pflash.raw + + +``` + +#### Secure Boot Observation + +After the VM is correctly configured and the PK, KEK, and DB certificates are imported, the VM runs in secure boot mode. You can configure the serial port log file in the VM configuration file in XML format to check whether the VM is in the secure boot mode. The following figure shows how to configure the serial port log file. + +```xml + + + +``` + +After the OS image is successfully loaded to the VM, if "UEFI Secure Boot is enabled" is displayed in the serial port log file, the VM is in the secure boot state. diff --git a/docs/en/virtualization/virtualization_platform/virtualization/public_sys-resources/icon-caution.gif b/docs/en/virtualization/virtualization_platform/virtualization/public_sys-resources/icon-caution.gif new file mode 100644 index 0000000000000000000000000000000000000000..6e90d7cfc2193e39e10bb58c38d01a23f045d571 Binary files /dev/null and b/docs/en/virtualization/virtualization_platform/virtualization/public_sys-resources/icon-caution.gif differ diff --git a/docs/en/virtualization/virtualization_platform/virtualization/public_sys-resources/icon-note.gif b/docs/en/virtualization/virtualization_platform/virtualization/public_sys-resources/icon-note.gif new file mode 100644 index 0000000000000000000000000000000000000000..6314297e45c1de184204098efd4814d6dc8b1cda Binary files /dev/null and b/docs/en/virtualization/virtualization_platform/virtualization/public_sys-resources/icon-note.gif differ diff --git a/docs/en/virtualization/virtualization_platform/virtualization/skylark.md b/docs/en/virtualization/virtualization_platform/virtualization/skylark.md new file mode 100644 index 0000000000000000000000000000000000000000..20abdd7168fc354deaca22e1f7eeeb07e7933dc6 --- /dev/null +++ b/docs/en/virtualization/virtualization_platform/virtualization/skylark.md @@ -0,0 +1,198 @@ +# Skylark + +- [Skylark](#skylark) + - [Skylark Introduction](#skylark-introduction) + - [Architecture and Features](#architecture-and-features) + - [Skylark Installation](#skylark-installation) + - [Skylark Configuration](#skylark-configuration) + - [Skylark Usage](#skylark-usage) + - [Best Practices](#best-practices) + +## Skylark Introduction + +### Scenario + +With the rapid growth of the cloud computing market, cloud vendors are increasing their investment in cloud infrastructure. However, the industry still faces the problem of low resource utilization. Improving resource utilization has become an important technical subject. This document describes openEuler Skylark, as well as how to install and use it. + +### Overview + +Hybrid deployment of services of different priorities is a typical and effective method to improve resource utilization. Services can be classified into high-priority and low-priority services based on latency sensitivity. When high-priority services compete with low-priority services for resources, resources are preferentially provided for high-priority services. Therefore, the core technology of service hybrid deployment is resource isolation control, which involves kernel-mode basic resource isolation and user-mode QoS control. + +This document describes the user-mode QoS control technology provided by Skylark of openEuler 22.09. In Skylark, the priority granularity is VMs. That is, a priority attribute is added to each VM. Resources are isolated and controlled based on VMs. Skylark is a QoS-aware resource scheduler in hybrid deployment scenarios. It improves physical machine resource utilization while ensuring the QoS of high-priority VMs. + +For details about how to better use the priority feature of Skylark in actual application scenarios, see [Best Practices](#best-practices). + +## Architecture and Features + +### Overall Architecture + +The core class of Skylark is `QoSManager`. Class members include data collection class instances, QoS analysis class instances, QoS control class instances, and task scheduling class instances. + +- `DataCollector`: data collection class. It has the `HostInfo` and `GuestInfo` members, which collect host information and VM information, respectively. +- `PowerAnalyzer`: power consumption analysis class, which analyzes power consumption interference and low-priority VMs to be restricted. +- `CpuController`: CPU bandwidth control class, which limits the CPU bandwidth of low-priority VMs. +- `CacheMBWController`: last-level cache (LLC) and memory bandwidth control class, which limits the LLC and memory bandwidth of low-priority VMs. +- `BackgroundScheduler`: task scheduling class, which periodically drives the preceding modules to continuously manage QoS. + +After checking the host environment, Skylark creates a daemon process. The daemon has a main scheduling thread and one or more job threads. + +- The main scheduling thread is unique. It connects to libvirt, creates and initializes the `QosManager` class instance, and then starts to drive the Job threads. +- Each Job thread periodically executes a QoS management task. + +### Power Consumption Interference Control + +Compared with non-hybrid deployment, host resource utilization is higher in hybrid deployment scenarios. High utilization means high power consumption. When the power consumption exceeds the thermal design power (TDP) of the server, CPU frequency reduction is triggered. When the power consumption exceeds the preset TDP (that is, a TDP hotspot occurs), Skylark limits the CPU bandwidth of low-priority VMs to reduce the power consumption of the entire system and ensure the QoS of high-priority VMs. + +During initialization, Skylark sets the power consumption interference control attributes based on the related configuration values in [Skylark Configuration](#skylark-configuration). In each control period, host information and control attributes are comprehensively analyzed to determine whether TDP hotspots occur. If a hotspot occurs, Skylark analyzes the low-priority VMs whose CPU bandwidth needs to be limited based on the VM information. + +### LLC/MB Interference Control + +Skylark can limit the LLC and memory bandwidth of low-priority VMs. Currently, only static allocation is supported. Skylark uses the **/sys/fs/resctrl** interface provided by the OS to implement the limitation. + +1. Skylark creates the **low_prio_machine** folder in the **/sys/fs/resctrl** directory and writes the PID of the low-priority VM to the **/sys/fs/resctrl/low_prio_machine/tasks** file. +2. Skylark allocates LLC ways and memory bandwidth for low-priority VMs based on the LLC/MB configuration items in [Skylark Configuration](#skylark-configuration). The configuration items are written into the **/sys/fs/resctrl/low_prio_machine/schemata** file. + +### CPU Interference Control + +In hybrid deployment scenarios, low-priority VMs generate CPU time slice interference and hardware hyper-threading (SMT) interference on high-priority VMs. + +- When threads of high- and low-priority VMs are running on the same minimum CPU topology unit (core or SMT execution unit), they compete for CPU time slices. +- When threads of high- and low-priority VMs are running on different SMT execution units of the same CPU core at the same time, they compete for resources in the core shared by the SMT execution units. + +CPU interference control includes CPU time slice interference control and SMT interference control, which are implemented based on the **QOS_SCHED** and **SMT_EXPELLER** features provided by the kernel, respectively. + +- The **QOS_SCHED** feature enables high-priority VM threads on a single CPU core or SMT execution unit to suppress low-priority VM threads, eliminating CPU time slice interference. +- The **SMT_EXPELLER** feature enables high-priority VM threads to suppress low-priority VM threads on different SMT execution units of the same CPU core, eliminating SMT interference. + +During initialization, Skylark sets the **cpu.qos_level** field of the slice level corresponding to the low-priority VM under the cgroup CPU subcontroller to -1 to enable the preceding kernel features. By doing this, the kernel controls CPU-related interference without the intervention of Skylark. + +## Skylark Installation + +### Hardware Requirements + +Processor architecture: AArch64 or x86_64 + +- For Intel processors, the RDT function must be supported. +- For the AArch64 architecture, only Kunpeng 920 processor is supported, and the BIOS must be upgraded to 1.79 or later to support the MPAM function. + +### Software Requirements + +- python3, python3-APScheduler, and python3-libvirt +- systemd 249-32 or later +- libvirt 1.0.5 or later +- openEuler kernel 5.10.0 or later. + +### Installation Procedure + +You are advised to install the Skylark component using Yum for automatic processing of the software dependencies: + +```shell +# yum install -y skylark +``` + +Check whether the Skylark is successfully installed. If the installation is successful, the skylarkd background service status is displayed: + +```shell +# systemctl status skylarkd +``` + +(Optional) Enable the Skylark service to automatically start upon system startup: + +```shell +# systemctl enable skylarkd +``` + +## Skylark Configuration + +After the Skylark component is installed, you can modify the configuration file if the default configuration does not meet your requirements. The Skylark configuration file is stored in **/etc/sysconfig/skylarkd**. The following describes the configuration items in the configuration file. + +### Logs + +- The **LOG_LEVEL** parameter is a character string used to set the minimum log level. The supported log levels are **critical > error > warning > info > debug**. Logs whose levels are lower than **LOG_LEVEL** are not recorded in the log file **/var/log/skylark.log**. Skylark backs up logs every seven days for a maximum of four times. (When the number of backup times reaches the limit, the oldest logs are deleted.) The backup log is saved as **/var/log/skylark.log. %Y- %m- %d**. + +### Power Consumption Interference Control + +- **POWER_QOS_MANAGEMENT** is a boolean value used to control whether to enable power consumption QoS management. Only x86 processors support this function. This function is useful if the CPU usage of VMs on the host can be properly limited. + +- **TDP_THRESHOLD** is a floating point number used to control the maximum power consumption of a VM. When the power consumption of the host exceeds **TDP * TDP_THRESHOLD**, a TDP hotspot occurs, and a power consumption control operation is triggered. The value ranges from 0.8 to 1, with the default value being 0.98. + +- **FREQ_THRESHOLD** is a floating point number used to control the minimum CPU frequency when a TDP hotspot occurs on the host. The value ranges from 0.8 to 1, with the default value being 0.98. + 1. When the frequency of some CPUs is lower than **max_freq * FREQ_THRESHOLD**, Skylark limits the CPU bandwidth of low-priority VMs running on these CPUs. + 2. If such a CPU does not exist, Skylark limits the CPU bandwidth of some low-priority VMs based on the CPU usage of low-priority VMs. + +- **QUOTA_THRESHOLD** is a floating point number used to control the CPU bandwidth that a restricted low-priority VM can obtain (CPU bandwidth before restriction x **QUOTA_THRESHOLD**). The value ranges from 0.8 to 1, with the default value being 0.9. + +- **ABNORMAL_THRESHOLD** is an integer used to control the number of low-priority VM restriction periods. The value ranges from 1 to 5, with the default value being 3. + 1. In each power consumption control period, if a low-priority VM is restricted, its number of remaining restriction periods is updated to **ABNORMAL_THRESHOLD**. + 2. Otherwise, its number of remaining restriction periods decreases by 1. When the number of remaining restriction periods of the VM is 0, the CPU bandwidth of the VM is restored to the value before the restriction. + +### LLC/MB Interference Control + +Skylark's interference control on LLC/MB depends on the RDT/MPAM function provided by hardware. For Intel x86_64 processors, **rdt=cmt,mbmtotal,mbmlocal,l3cat,mba** needs to be added to kernel command line parameters. For Kunpeng 920 processors, **mpam=acpi** needs to be added to kernel command line parameters. + +- **MIN_LLC_WAYS_LOW_VMS** is an integer used to control the number of LLC ways that can be accessed by low-priority VMs. The value ranges from 1 to 3, with the default value being 2. During initialization, Skylark limits the numfer of accessible LLC ways for low-priority VMs to this value. + +- **MIN_MBW_LOW_VMS** is a floating point number used to control the memory bandwidth ratio available to low-priority VMs. The value ranges from 0.1 to 0.2, with the default value being 0.1. Skylark limits the memory bandwidth of low-priority VMs based on this value during initialization. + +## Skylark Usage + +### Starting the Service + +Start Skylark for the first time: + +```shell +# systemctl start skylarkd +``` + +Restart Skylark (a service restart is required after modifying the configuration file): + +```shell +# systemctl restart skylarkd +``` + +### Creating VMs + +Skylark uses the **partition** tag in the XML configuration file of a VM to identify the VM priority. + +To create a low-priority VM, configure the XML file as follows: + +```xml + + ... + + /low_prio_machine + + ... + +``` + +To create a high-priority VM, configure the XML file as follows: + +```xml + + ... + + /high_prio_machine + + ... + +``` + +The subsequent VM creation process is the same as the normal process. + +### Running VMs + +Skylark detects VM creation events, manages VMs of different priorities, and performs automatic QoS management based on CPU, power consumption, and LLC/MB resources. + +## Best Practices + +### VM Service Recommendation + +- High-priority VMs are suitable for latency-sensitive services, such as web services, high-performance databases, real-time rendering, and AI inference. +- Low-priority VMs are suitable for non-latency-sensitive services, such as video encoding, big data processing, offline rendering, and AI training. + +### CPU Binding Configuration + +To ensure optimal performance of high-priority VMs, you are advised to bind each vCPU of high-priority VMs to a physical CPU. To enable low-priority VMs to make full use of idle physical resources, you are advised to bind vCPUs of low-priority VMs to CPUs that are bound to high-priority VMs. + +To ensure that low-priority VMs are scheduled when high-priority VMs occupy CPU resources for a long time, you are advised to reserve a small number of for low-priority VMs. diff --git a/docs/en/virtualization/virtualization_platform/virtualization/system-resource-management.md b/docs/en/virtualization/virtualization_platform/virtualization/system-resource-management.md new file mode 100644 index 0000000000000000000000000000000000000000..a681bec04c17111cf11da3a97d1b4023cbc36183 --- /dev/null +++ b/docs/en/virtualization/virtualization_platform/virtualization/system-resource-management.md @@ -0,0 +1,480 @@ +# system Resource Management + +The `libvirt` command manages VM system resources, such as vCPU and virtual memory resources. + +Before you start: + +- Ensure that the libvirtd daemon is running on the host. +- Run the `virsh list --all` command to check that the VM has been defined. + +## Managing vCPU + +### CPU Shares + +#### Overview + +In a virtualization environment, multiple VMs on the same host compete for physical CPUs. To prevent some VMs from occupying too many physical CPU resources and affecting the performance of other VMs on the same host, you need to balance the vCPU scheduling of VMs to prevent excessive competition for physical CPUs. + +The CPU share indicates the total capability of a VM to compete for physical CPU computing resources. You can set **cpu\_shares** to specify the VM capacity to preempt physical CPU resources. The value of **cpu\_shares** is a relative value without a unit. The CPU computing resources obtained by a VM are the available computing resources of physical CPUs \(excluding reserved CPUs\) allocated to VMs based on the CPU shares. Adjust the CPU shares to ensure the service quality of VM CPU computing resources. + +#### Procedure + +Change the value of **cpu\_shares** allocated to the VM to balance the scheduling between vCPUs. + +- Check the current CPU share of the VM. + + ```shell + $ virsh schedinfo + Scheduler : posix + cpu_shares : 1024 + vcpu_period : 100000 + vcpu_quota : -1 + emulator_period: 100000 + emulator_quota : -1 + global_period : 100000 + global_quota : -1 + iothread_period: 100000 + iothread_quota : -1 + ``` + +- Online modification: Run the `virsh schedinfo` command with the `--live` parameter to modify the CPU share of a running VM. + + ```shell + virsh schedinfo --live cpu_shares= + ``` + + For example, to change the CPU share of the running _openEulerVM_ from **1024** to **2048**, run the following commands: + + ```shell + $ virsh schedinfo openEulerVM --live cpu_shares=2048 + Scheduler : posix + cpu_shares : 2048 + vcpu_period : 100000 + vcpu_quota : -1 + emulator_period: 100000 + emulator_quota : -1 + global_period : 100000 + global_quota : -1 + iothread_period: 100000 + iothread_quota : -1 + ``` + + The modification of the **cpu\_shares** value takes effect immediately. The running time of the _openEulerVM_ is twice the original running time. However, the modification will become invalid after the VM is shut down and restarted. + +- Permanent modification: Run the `virsh schedinfo` command with the `--config` parameter to change the CPU share of the VM in the libvirt internal configuration. + + ```shell + virsh schedinfo --config cpu_shares= + ``` + + For example, run the following command to change the CPU share of _openEulerVM_ from **1024** to **2048**: + + ```shell + $ virsh schedinfo openEulerVM --config cpu_shares=2048 + Scheduler : posix + cpu_shares : 2048 + vcpu_period : 0 + vcpu_quota : 0 + emulator_period: 0 + emulator_quota : 0 + global_period : 0 + global_quota : 0 + iothread_period: 0 + iothread_quota : 0 + ``` + + The modification on **cpu\_shares** does not take effect immediately. Instead, the modification takes effect after the _openEulerVM_ is started next time and takes effect permanently. The running time of the _openEulerVM_ is twice that of the original VM. + +### Binding the QEMU Process to a Physical CPU + +#### Overview + +You can bind the QEMU main process to a specific physical CPU range, ensuring that VMs running different services do not interfere with adjacent VMs. For example, in a typical cloud computing scenario, multiple VMs run on one physical machine, and they carry diversified services, causing different degrees of resource occupation. To avoid interference of a VM with dense-storage I/O to an adjacent VM, storage processes that process I/O of different VMs need to be completely isolated. The QEMU main process handles frontend and backend services. Therefore, isolation needs to be implemented. + +#### Procedure + +Run the `virsh emulatorpin` command to bind the QEMU main process to a physical CPU. + +- Check the range of the physical CPU bound to the QEMU process: + + ```shell + $ virsh emulatorpin openEulerVM + emulator: CPU Affinity + ---------------------------------- + *: 0-63 + ``` + + This indicates that the QEMU main process corresponding to VM **openEulerVM** can be scheduled on all physical CPUs of the host. + +- Online binding: Run the `virsh emulatorpin` command with the `--live` parameter to modify the binding relationship between the QEMU process and the running VM. + + ```shell + $ virsh emulatorpin openEulerVM --live 2-3 + + $ virsh emulatorpin openEulerVM + emulator: CPU Affinity + ---------------------------------- + *: 2-3 + ``` + + The preceding commands bind the QEMU process corresponding to VM **openEulerVM** to physical CPUs **2** and **3**. That is, the QEMU process is scheduled only on the two physical CPUs. The binding relationship takes effect immediately but becomes invalid after the VM is shut down and restarted. + +- Permanent binding: Run the `virsh emulatorpin` command with the `--config` parameter to modify the binding relationship between the VM and the QEMU process in the libvirt internal configuration. + + ```shell + $ virsh emulatorpin openEulerVM --config 0-3,^1 + + $ virsh emulatorpin euler + emulator: CPU Affinity + ---------------------------------- + *: 0,2-3 + ``` + + The preceding commands bind the QEMU process corresponding to VM **openEulerVM** to physical CPUs **0**, **2** and **3**. That is, the QEMU process is scheduled only on the three physical CPUs. The modification of the binding relationship does not take effect immediately. Instead, the modification takes effect after the next startup of the VM and takes effect permanently. + +### Adjusting the vCPU Binding Relationship + +#### Overview + +The vCPU of a VM is bound to a physical CPU. That is, the vCPU is scheduled only on the bound physical CPU to improve VM performance in specific scenarios. For example, in a NUMA system, vCPUs are bound to the same NUMA node to prevent cross-node memory access and VM performance deterioration. If the vCPU is not bound, by default, the vCPU can be scheduled on any physical CPU. The specific binding policy is determined by the user. + +#### Procedure + +Run the `virsh vcpupin` command to adjust the binding relationship between vCPUs and physical CPUs. + +- View the vCPU binding information of the VM. + + ```shell + $ virsh vcpupin openEulerVM + VCPU CPU Affinity + ---------------------- + 0 0-63 + 1 0-63 + 2 0-63 + 3 0-63 + ``` + + This indicates that all vCPUs of VM **openEulerVM** can be scheduled on all physical CPUs of the host. + +- Online adjustment: Run the `vcpu vcpupin` command with the `--live` parameter to modify the vCPU binding relationship of a running VM. + + ```shell + $ virsh vcpupin openEulerVM --live 0 2-3 + + $ virsh vcpupin euler + VCPU CPU Affinity + ---------------------- + 0 2-3 + 1 0-63 + 2 0-63 + 3 0-63 + ``` + + The preceding commands bind vCPU **0** of VM **openEulerVM** to pCPU **2** and pCPU **3**. That is, vCPU **0** is scheduled only on the two physical CPUs. The binding relationship takes effect immediately but becomes invalid after the VM is shut down and restarted. + +- Permanent adjustment: Run the `virsh vcpupin` command with the `--config` parameter to modify the vCPU binding relationship of the VM in the libvirt internal configuration. + + ```shell + $ virsh vcpupin openEulerVM --config 0 0-3,^1 + + $ virsh vcpupin openEulerVM + VCPU CPU Affinity + ---------------------- + 0 0,2-3 + 1 0-63 + 2 0-63 + 3 0-63 + ``` + + The preceding commands bind vCPU **0** of VM **openEulerVM** to physical CPUs **0**, **2**, and **3**. That is, vCPU **0** is scheduled only on the three physical CPUs. The modification of the binding relationship does not take effect immediately. Instead, the modification takes effect after the next startup of the VM and takes effect permanently. + +### CPU Hotplug + +#### Overview + +CPU hotplug allows you to increase or decrease the number of CPUs for a running VM without affecting services on it. When the internal service pressure rises to a level where existing CPUs become saturated, CPU hotplug can dynamically boost the computing power of a VM, guaranteeing stable service throughput. CPU hotplug also enables the removal of unused computing resources during low service load, minimizing computing costs. + +Note: CPU hotplug is added for the AArch64 architecture in openEuler 24.03 LTS. However, the new implementation of the mainline community is not compatible with that of earlier openEuler versions. Therefore, the guest OS must match the host OS. That is, the guest and host machines must both run openEuler 24.03 LTS or later versions, or versions earlier than openEuler 24.03 LTS. + +#### Constraints + +- For processors using the AArch64 architecture, the specified VM chipset type \(machine\) needs to be virt-4.1 or a later version when a VM is created. For processors using the x86\_64 architecture, the specified VM chipset type \(machine\) needs to be pc-i440fx-1.5 or a later version when a VM is created. +- The initial CPU of an AArch64 VM cannot be hot removed. +- When configuring Guest NUMA, you need to configure the vCPUs that belong to the same socket in the same vNode. Otherwise, the VM may be soft locked up after the CPU is hot added or removed, which may cause the VM panic. +- VMs do not support CPU hotplug during migration, hibernation, wake-up, or snapshot. +- Whether the hot added CPU can automatically go online depends on the VM OS logic rather than the virtualization layer. +- CPU hot add is restricted by the maximum number of CPUs supported by the Hypervisor and GuestOS. +- When a VM is being started, stopped, or restarted, the hot added CPU may become invalid. However, the hot added CPU takes effect after the VM is restarted. +- CPU hotplug may time out when a VM is starting, shutting down, or restarting. Retry when the VM is in the normal running state. +- During VM CPU hotplug, if the number of added or removed CPUs is not an integer multiple of the number of cores in the VM CPU topology configuration item, the CPU topology displayed in the VM may be disordered. You are advised to add or remove CPUs whose number is an integer multiple of the number of cores each time. +- If the hot added or removed CPU needs to take effect online and is still valid after the VM is restarted, the `--config` and `--live` options need to be transferred to the `virsh setvcpus` interface to persist the hot added or removed CPU. + +#### Procedure + +**VM XML Configuration** + +1. To use the CPU hot add function, configure the number of CPUs, the maximum number of CPUs supported by the VM, and the VM chipset type when creating the VM. (For the AArch64 architecture, the virt-4.2 or a later version is required. For the x86\_64 architecture, the pc-i440fx-1.5 or later version is required. The AArch64 VM is used as an example. The configuration template is as follows: + + ```xml + + ... + n + + hvm + + ... + + ``` + + > ![](./public_sys-resources/icon-note.gif)**Note** + > + > - The value of placement must be static. + > - m indicates the current number of CPUs on the VM, that is, the default number of CPUs after the VM is started. n indicates the maximum number of CPUs that can be hot added to a VM. The value cannot exceed the maximum CPU specifications supported by the Hypervisor or GuestOS. n is greater than or equal to m. + + For example, if the current number of CPUs of a VM is 4 and the maximum number of hot added CPUs is 64, the XML configuration is as follows: + + ```xml + + ... + 64 + + hvm + + ... + ``` + +**Hot Adding and Bringing CPUs Online** + +1. If the hot added CPU needs to be automatically brought online, create the udev rules file /etc/udev/rules.d/99-hotplug-cpu.rules in the VM as user root and define the udev rules in the file. The following is an example: + + ```text + ### automatically online hot-plugged cpu + ACTION=="add", SUBSYSTEM=="cpu", ATTR{online}="1" + ``` + + > ![](./public_sys-resources/icon-note.gif)**Note** + > If you do not use the udev rules, you can use the root permission to manually bring the hot added CPU online by running the following commands: + + ```shell + for i in `grep -l 0 /sys/devices/system/cpu/cpu*/online` + do + echo 1 > $i + done + ``` + +2. Use the virsh tool to hot add CPUs to the VM. For example, to set the number of CPUs after hot adding to 6 on the VM named openEulerVM and make the hot add take effect online, run the following command: + + ```shell + virsh setvcpus openEulerVM 6 --live + ``` + + > ![](./public_sys-resources/icon-note.gif)**Note** + > The format for running the `virsh setvcpus` command to hot add VM CPUs is as follows: + + ```shell + virsh setvcpus [--config] [--live] + ``` + + > - `domain`: Parameter, which is mandatory. Specifies the name of a VM. + > - `count`: Parameter, which is mandatory. Specifies the number of target CPUs, that is, the number of CPUs after hot adding. + > - `--config`: Option, which is optional. This parameter is still valid when the VM is restarted. + > - `--live`: Option, which is optional. The configuration takes effect online. + +**Hot Removing CPUs** + +Use the virsh tool to hot remove CPUs from the VM. For example, to set the number of CPUs after hot removal to 4 on the VM named openEulerVM, run the following command: + +```shell +virsh setvcpus openEulerVM 4 --live +``` + +> ![](./public_sys-resources/icon-note.gif)**Note** +> The format for running the `virsh setvcpus` command to hot remove VM CPUs is as follows: +> +> ```shell +> virsh setvcpus [--config] [--live] +> ``` +> +> - `domain`: Parameter, which is mandatory. Specifies the name of a VM. +> - `count`: Parameter, which is mandatory. Specifies the number of target CPUs, that is, the number of CPUs after hot removal. +> - `--config`: Option, which is optional. This parameter is still valid when the VM is restarted. +> - `--live`: Option, which is optional. The configuration takes effect online. + +## Managing Virtual Memory + +### Introduction to NUMA + +Traditional multi-core computing uses the symmetric multi-processor \(SMP\) mode. Multiple processors are connected to a centralized memory and I/O bus. All processors can access only the same physical memory. Therefore, the SMP system is also referred to as a uniform memory access \(UMA\) system. Uniformity means that a processor can only maintain or share a unique value for each data record in memory at any time. Obviously, the disadvantage of SMP is its limited scalability, because when the memory and the I/O interface are saturated, adding a processor cannot obtain higher performance. + +The non-uniform memory access architecture \(NUMA\) is a distributed memory access mode. In this mode, a processor can access different memory addresses at the same time, which greatly improves concurrency. With this feature, a processor is divided into multiple nodes, each of which is allocated a piece of local memory space. The processors of all nodes can access all physical memories, but the time required for accessing the memory on the local node is much shorter than that on a remote node. + +### Configuring Host NUMA + +To improve VM performance, you can specify NUMA nodes for a VM using the VM XML configuration file before the VM is started so that the VM memory is allocated to the specified NUMA nodes. This feature is usually used together with the vCPU to prevent the vCPU from remotely accessing the memory. + +#### Procedure + +- Check the NUMA topology of the host. + + ```shell + $ numactl -H + available: 4 nodes (0-3) + node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 + node 0 size: 31571 MB + node 0 free: 17095 MB + node 1 cpus: 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 + node 1 size: 32190 MB + node 1 free: 28057 MB + node 2 cpus: 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 + node 2 size: 32190 MB + node 2 free: 10562 MB + node 3 cpus: 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 + node 3 size: 32188 MB + node 3 free: 272 MB + node distances: + node 0 1 2 3 + 0: 10 15 20 20 + 1: 15 10 20 20 + 2: 20 20 10 15 + 3: 20 20 15 10 + ``` + +- Add the **numatune** field to the VM XML configuration file to create and start the VM. For example, to allocate NUMA node 0 on the host to the VM, configure parameters as follows: + + ```xml + + + + ``` + + If the vCPU of the VM is bound to the physical CPU of **node 0**, the performance deterioration caused by the vCPU accessing the remote memory can be avoided. + + > [!NOTE]NOTE + > + > - The sum of memory allocated to the VM cannot exceed the remaining available memory of the NUMA node. Otherwise, the VM may fail to start. + > - You are advised to bind the VM memory and vCPU to the same NUMA node to avoid the performance deterioration caused by vCPU access to the remote memory. For example, bind the vCPU to NUMA node 0 as well. + +### Configuring Guest NUMA + +Many service software running on VMs is optimized for the NUMA architecture, especially for large-scale VMs. openEuler provides the Guest NUMA feature to display the NUMA topology in VMs. You can identify the structure to optimize the performance of service software and ensure better service running. + +When configuring guest NUMA, you can specify the location of vNode memory on the host to implement memory block binding and vCPU binding so that the vCPU and memory on the vNode are on the same physical NUMA node. + +#### Procedure + +After Guest NUMA is configured in the VM XML configuration file, you can view the NUMA topology on the VM. **** is mandatory for Guest NUMA. + +```xml + + + + + + + + + + + [...] + + + + + + +``` + +> [!NOTE]NOTE +> +> - **** provides the NUMA topology function for VMs. **cell id** indicates the vNode ID, **cpus** indicates the vCPU ID, and **memory** indicates the memory size on the vNode. +> - If you want to use Guest NUMA to provide better performance, configure <**numatune\>** and **** so that the vCPU and memory are distributed on the same physical NUMA node. +> - **cellid** in **** corresponds to **cell id** in ****. **mode** can be set to **strict** \(apply for memory from a specified node strictly. If the memory is insufficient, the application fails.\), **preferred** \(apply for memory from a node first. If the memory is insufficient, apply for memory from another node\), or **interleave** \(apply for memory from a specified node in cross mode\).; **nodeset** indicates the specified physical NUMA node. +> - In ****, you need to bind the vCPU in the same **cell id** to the physical NUMA node that is the same as the **memnode**. + +### Memory Hot Add + +#### Overview + +In virtualization scenarios, the memory, CPU, and external devices of VMs are simulated by software. Therefore, the memory can be adjusted online for VMs at the virtualization bottom layer. In the current openEuler version, memory can be added to a VM online. If the physical memory of a VM is insufficient and the VM cannot be shut down, you can use this feature to add physical memory resources to the VM. + +#### Constraints + +- For processors using the AArch64 architecture, the specified VM chipset type \(machine\) needs to be virt-4.2 or a later version when a VM is created.For processors using the x86 architecture, the specified VM chipset type \(machine\) needs to be a later version than pc-i440fx-1.5 when a VM is created. +- Guest NUMA on which the memory hot add feature depends needs to be configured on the VM. Otherwise, the memory hot add process cannot be completed. +- When hot adding memory, you need to specify the ID of Guest NUMA node to which the new memory belongs. Otherwise, the memory hot add fails. +- The VM kernel should support memory hot add. Otherwise, the VM cannot identify the newly added memory or the memory cannot be brought online. +- For a VM that uses hugepages, the capacity of the hot added memory should be an integral multiple of hugepagesz. Otherwise, the hot add fails. +- The hot added memory size should be an integral multiple of the Guest physical memory block size (block\_size\_bytes). Otherwise, the VM cannot go online. The value of block\_size\_bytes can be obtained using the lsmem command in Guest. +- After n pieces of virtio-net NICs are configured, the maximum number of hot add times is set to min\{max\_slot, 64 - n\} to reserve slots for NICs. +- The vhost-user device and the memory hot add feature are mutually exclusive. A VM configured with the vhost-user device does not support memory hot add. After the memory is hot added to a VM, the vhost-user device cannot be hot added. +- If the VM OS is Linux, ensure that the initial memory is greater than or equal to 4 GB. +- If the VM OS is Windows, the first hot added memory needs to be specified to Guest NUMA node0. Otherwise, the hot added memory cannot be identified by the VM. +- In passthrough scenarios, memory needs to be allocated in advance. Therefore, it is normal that the startup and hot add of memory are slower than those of common VMs (especially large-specification VMs). +- It is recommended that the ratio of the available memory to the hot added memory be at least 1:32. That is, at least 1 GB available memory is required for the VM with 32 GB hot added memory. If the ratio is less than 1:32, the VM may be suspended. +- Whether the hot added memory can automatically go online depends on the VM OS logic. You can manually bring the memory online or configure the udev rules to automatically bring the memory online. + +#### Procedure + +**VM XML Configuration** + +1. To use the memory hot add function, configure the maximum hot add memory size and reserved slot number, and configure the Guest NUMA topology when creating a VM. + + For example, run the following command to configure 32 GB initial memory for a VM, reserve 256 slots, set the memory upper limit to 1 TB, and configure two NUMA nodes: + + ```xml + + 32 + 1024 + + + + + + + + .... + ``` + +> ![](./public_sys-resources/icon-note.gif)**Note** +> In the preceding information, +> the value of slots in the maxMemory field indicates the reserved memory slots. The maximum value is 256. +> maxMemory indicates the maximum physical memory supported by the VM. +> For details about how to configure Guest NUMA, see "Configuring Guest NUMA." + +**Hot Adding and Bringing Memory Online** + +1. If the hot added memory needs to be automatically brought online, create the udev rules file /etc/udev/rules.d/99-hotplug-memory.rules in the VM as user root and define the udev rules in the file. The following is an example: + + ```text + ### automatically online hot-plugged memory + ACTION=="add", SUBSYSTEM=="memory", ATTR{state}="online" + ``` + +2. Create a memory description XML file based on the size of the memory to be hot added and the Guest NUMA node of the VM. + + For example, to hot add 1 GB memory to NUMA node0, run the following command: + + ```xml + + + 1024 + 0 + + + ``` + +3. Run the virsh attach-device command to hot add memory to the VM. In the command, openEulerVM indicates the VM name, memory.xml indicates the description file of the hot added memory, and --live indicates that the hot added memory takes effect online. You can also run the --config command to persist the hot added memory to the VM XML file. + + ```shell + virsh attach-device openEulerVM memory.xml --live + ``` + + > ![](./public_sys-resources/icon-note.gif)**Note** + > If you do not use the udev rules, you can use the root permission to manually bring the hot added memory online by running the following command: + + ```shell + for i in `grep -l offline /sys/devices/system/memory/memory*/state` + do + echo online > $i + done + ``` diff --git a/docs/en/virtualization/virtualization_platform/virtualization/tool-guide.md b/docs/en/virtualization/virtualization_platform/virtualization/tool-guide.md new file mode 100644 index 0000000000000000000000000000000000000000..aecca3d63a94c825cb0dbf1cccc2d45d8ff87bba --- /dev/null +++ b/docs/en/virtualization/virtualization_platform/virtualization/tool-guide.md @@ -0,0 +1,3 @@ +# Tool Guide + +To help users better use virtualization, openEuler provides a set of tools, including vmtop and LibcarePlus. This section describes how to install and use these tools. diff --git a/docs/en/virtualization/virtualization_platform/virtualization/virtualization-installation.md b/docs/en/virtualization/virtualization_platform/virtualization/virtualization-installation.md new file mode 100644 index 0000000000000000000000000000000000000000..9cbbf1645eed11afb1c2254b7d2c8cc967432d03 --- /dev/null +++ b/docs/en/virtualization/virtualization_platform/virtualization/virtualization-installation.md @@ -0,0 +1,150 @@ +# Installing Virtualization Components + +This chapter describes how to install virtualization components in openEuler. + +- [Installing Virtualization Components](#installing-virtualization-components) + - [Minimum Hardware Requirements](#minimum-hardware-requirements) + - [Installing Core Virtualization Components](#installing-core-virtualization-components) + - [Installation Methods](#installation-methods) + - [Prerequisites](#prerequisites) + - [Procedure](#procedure) + - [Verifying the Installation](#verifying-the-installation) + +## Minimum Hardware Requirements + +The minimum hardware requirements for installing virtualization components on openEuler are as follows: + +- AArch64 processor architecture: ARMv8 or later, supporting virtualization expansion +- x86\_64 processor architecture, supporting VT-x +- 2-core CPU +- 4 GB memory +- 16 GB available disk space + +## Installing Core Virtualization Components + +### Installation Methods + +#### Prerequisites + +- The yum source has been configured. For details, see the _openEuler 21.03 Administrator Guide_. +- Only the administrator has permission to perform the installation. + +#### Procedure + +1. Install the QEMU component. + + ```shell + # yum install -y qemu + ``` + + > ![](./public_sys-resources/icon-caution.gif) Notice: + > By default, the QEMU component runs as user qemu and user group qemu. If you are not familiar with Linux user group and user permission management, you may encounter insufficient permission when creating and starting VMs. You can use either of the following methods to solve this problem: + > Method 1: Modify the QEMU configuration file. Run the `sudo vim /etc/libvirt/qemu.conf` command to open the QEMU configuration file, find `user = "root"` and `group = "root"`, uncomment them (delete `#`), save the file, and exit. + > Method 2: Change the owner of the VM files. Ensure that user qemu has the permission to access the folder where VM files are stored. Run the `sudo chown qemu:qemu xxx.qcow2` command to change the owner of the VM files that need to be read and written. + +2. Install the libvirt component. + + ```shell + # yum install -y libvirt + ``` + +3. Start the libvirtd service. + + ```shell + # systemctl start libvirtd + ``` + +> [!NOTE]NOTE +> The KVM module is integrated in the openEuler kernel and does not need to be installed separately. + +### Verifying the Installation + +1. Check whether the kernel supports KVM virtualization, that is, check whether the **/dev/kvm** and **/sys/module/kvm** files exist. The command and output are as follows: + + ```shell + # ls /dev/kvm + /dev/kvm + ``` + + ```shell + # ls /sys/module/kvm + parameters uevent + ``` + + If the preceding files exist, the kernel supports KVM virtualization. If the preceding files do not exist, KVM virtualization is not enabled during kernel compilation. In this case, you need to use the Linux kernel that supports KVM virtualization. + +2. Check whether QEMU is successfully installed. If the installation is successful, the QEMU software package information is displayed. The command and output are as follows: + + ```shell + # rpm -qi qemu + Name : qemu + Epoch : 2 + Version : 4.0.1 + Release : 10 + Architecture: aarch64 + Install Date: Wed 24 Jul 2019 04:04:47 PM CST + Group : Unspecified + Size : 16869484 + License : GPLv2 and BSD and MIT and CC-BY + Signature : (none) + Source RPM : qemu-4.0.0-1.src.rpm + Build Date : Wed 24 Jul 2019 04:03:52 PM CST + Build Host : localhost + Relocations : (not relocatable) + URL : http://www.qemu.org + Summary : QEMU is a generic and open source machine emulator and virtualizer + Description : + QEMU is a generic and open source processor emulator which achieves a good + emulation speed by using dynamic translation. QEMU has two operating modes: + + * Full system emulation. In this mode, QEMU emulates a full system (for + example a PC), including a processor and various peripherals. It can be + used to launch different Operating Systems without rebooting the PC or + to debug system code. + * User mode emulation. In this mode, QEMU can launch Linux processes compiled + for one CPU on another CPU. + + As QEMU requires no host kernel patches to run, it is safe and easy to use. + ``` + +3. Check whether libvirt is successfully installed. If the installation is successful, the libvirt software package information is displayed. The command and output are as follows: + + ```shell + # rpm -qi libvirt + Name : libvirt + Version : 5.5.0 + Release : 1 + Architecture: aarch64 + Install Date: Tue 30 Jul 2019 04:56:21 PM CST + Group : Unspecified + Size : 0 + License : LGPLv2+ + Signature : (none) + Source RPM : libvirt-5.5.0-1.src.rpm + Build Date : Mon 29 Jul 2019 08:14:57 PM CST + Build Host : 71e8c1ce149f + Relocations : (not relocatable) + URL : https://libvirt.org/ + Summary : Library providing a simple virtualization API + Description : + Libvirt is a C toolkit to interact with the virtualization capabilities + of recent versions of Linux (and other OSes). The main package includes + the libvirtd server exporting the virtualization support. + ``` + +4. Check whether the libvirt service is started successfully. If the service is in the **Active** state, the service is started successfully. You can use the virsh command line tool provided by the libvirt. The command and output are as follows: + + ```shell + # systemctl status libvirtd + ● libvirtd.service - Virtualization daemon + Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled) + Active: active (running) since Tue 2019-08-06 09:36:01 CST; 5h 12min ago + Docs: man:libvirtd(8) + https://libvirt.org + Main PID: 40754 (libvirtd) + Tasks: 20 (limit: 32768) + Memory: 198.6M + CGroup: /system.slice/libvirtd.service + ─40754 /usr/sbin/libvirtd + + ``` diff --git a/docs/en/virtualization/virtualization_platform/virtualization/vm-configuration.md b/docs/en/virtualization/virtualization_platform/virtualization/vm-configuration.md new file mode 100644 index 0000000000000000000000000000000000000000..9f60d5900f35effb3c76e7c569cf73d8f810dfe1 --- /dev/null +++ b/docs/en/virtualization/virtualization_platform/virtualization/vm-configuration.md @@ -0,0 +1,821 @@ +# VM Configuration + +- [VM Configuration](#vm-configuration) + - [Introduction](#introduction) + - [VM Description](#vm-description) + - [vCPU and Virtual Memory](#vcpu-and-virtual-memory) + - [Virtual Device Configuration](#virtual-device-configuration) + - [Storage Devices](#storage-devices) + - [Network Devices](#network-devices) + - [Bus Configuration](#bus-configuration) + - [Other Common Devices](#other-common-devices) + - [Configurations Related to the System Architecture](#configurations-related-to-the-system-architecture) + - [Other Common Configuration Items](#other-common-configuration-items) + - [XML Configuration File Example](#xml-configuration-file-example) + +## Introduction + +### Overview + +Libvirt tool uses XML files to describe a VM feature, including the VM name, CPU, memory, disk, NIC, mouse, and keyboard. You can manage a VM by modifying configuration files. This section describes the elements in the XML configuration file to help users configure VMs. + +### Format + +The VM XML configuration file uses domain as the root element, which contains multiple other elements. Some elements in the XML configuration file can contain corresponding attributes and attribute values to describe VM information in detail. Different attributes of the same element are separated by spaces. + +The basic format of the XML configuration file is as follows. In the format, **label** indicates the label name, **attribute** indicates the attribute, and **value** indicates the attribute value. Change them based on the site requirements. + +```xml + + VMName + 8 + 4 + + + + + +``` + +### Process + +1. Create an XML configuration file with domain root element. +2. Use the name tag to specify a unique VM name based on the naming rule. +3. Configure system resources such as the virtual CPU \(vCPU\) and virtual memory. +4. Configure virtual devices. + 1. Configure storage devices. + 2. Configure network devices. + 3. Configure the external bus structure. + 4. Configure external devices such as the mouse. + +5. Save the XML configuration file. + +## VM Description + +### Overview + +This section describes how to configure the VM **domain** root element and VM name. + +### Elements + +- **domain**: Root element of a VM XML configuration file, which is used to configure the type of the hypervisor that runs the VM. + + **type**: Type of a domain in virtualization. In the openEuler virtualization, the attribute value is **kvm**. + +- **name**: VM name. + + The VM name is a unique character string on the same host. The VM name can contain only digits, letters, underscores \(\_\), hyphens \(-\), and colons \(:\), but cannot contain only digits. The VM name can contain a maximum of 64 characters. + +### Configuration Example + +For example, if the VM name is **openEuler**, the configuration is as follows: + +```xml + + openEuler + ... + +``` + +## vCPU and Virtual Memory + +### Overview + +This section describes how to configure the vCPU and virtual memory. + +### Elements + +- **vcpu**: The number of virtual processors. +- **memory**: The size of the virtual memory. + + **unit**: The memory unit. The value can be KiB \(210 bytes\), MiB \(220 bytes\), GiB \(230 bytes\), or TiB \(240 bytes\). + +- **cpu**: The mode of the virtual processor. + + **mode**: The mode of the vCPU. + + - **host-passthrough**: indicates that the architecture and features of the virtual CPU are the same as those of the host. + + - **custom**: indicates that the architecture and features of the virtual CPU are configured by the **cpu** element. + + Sub-element **topology**: A sub-element of the element cpu, used to describe the topology structure of a vCPU mode. + + - The attributes **socket**, **cores**, and **threads** of the sub-element topology describe the number of CPU sockets of a VM, the number of processor cores included in each CPU socket, and the number of threads included in each processor core, respectively. The attribute value is a positive integer, and the product of the three values equals the number of vCPUs. + - The ARM architecture supports the virtual hyper-threading function. The virtual CPU hot add and the virtual hyper-threading function are mutually exclusive. + Sub-element **model**: A sub-element of the element cpu, used to describe the CPU model when **mode** is custom. + + Sub-element **feature**: A sub-element of the element cpu, used to enable/disable a CPU feature when **mode** is custom. The attribute **name** describes the name of the CPU feature. And whether enable the CPU feature is controlled by the attribute **policy**: + + - **force**: force enable the CPU feature regardless of it being supported by host CPU. + + - **require**: enable the CPU feature. + + - **optional**: the CPU feature will be enabled if and only if it is supported by host CPU. + + - **disable**: disable the CPU feature. + + - **forbid**: disable the CPU feature and guest creation will fail if the feature is supported by host CPU. + +### Configuration Example + +For example, if the number of vCPUs is 4, the processing mode is host-passthrough, the virtual memory is 8 GiB, the four CPUs are distributed in two CPU sockets, and hyperthreading is not supported, the configuration is as follows: + +```xml + + ... + 4 + 8 + + + +... + +``` + +If the virtual memory is 8 GiB, the number of vCPUs is 4, the processing mode is custom, the CPU model is Kunpeng-920, and pmull is disabled, the configuration is as follows: + +```xml + + ... + 4 + 8 + + Kunpeng-920 + + + ... + +``` + +## Virtual Device Configuration + +The VM XML configuration file uses the **devices** elements to configure virtual devices, including storage devices, network devices, buses, and mouse devices. This section describes how to configure common virtual devices. + +### Storage Devices + +#### Overview + +This section describes how to configure virtual storage devices, including floppy disks, disks, and CD-ROMs and their storage types. + +#### Elements + +The XML configuration file uses the **disk** element to configure storage devices. [Table 1](#table14200183410353) describes common **disk** attributes. [Table 2](#table4866134925114) describes common subelements and their attributes. + +**Table 1** Common attributes of the **disk** element + + + + + + + + + + + + + + + + + + + +
ElementAttributeDescriptionAttribute Value and Description
disktypeSpecifies the type of the backend storage medium.block: block device
file: file device
dir: directory path
network: network disk
deviceSpecifies the storage medium to be presented to the VM.disk: disk (default)
floppy: floppy disk
cdrom: CD-ROM
+ +**Table 2** Common subelements and attributes of the **disk** element + +| Subelement | Subelement Description | Attribute Description | +| ---------- | ------------------------------------------------------------ | ------------------------------------------------------------ | +| source | Specifies the backend storage medium, which corresponds to the type specified by the **type** attribute of the **disk** element. | **file**: file type. The value is the fully qualified path of the corresponding file.
**dev**: block type. The value is the fully qualified path of the corresponding host device.
**dir**: directory type. The value is the fully qualified path of the disk directory.
**protocol**: protocol in use.
**name**: RBD disk name. The format is as follows: $pool/$volume .**host name**: mon address.
**port**: port of the mon address.
| +| driver | Details about the specified backend driver | **type**: disk format type. The value can be **raw** or **qcow2**, which must be the same as that of source.
**io**: disk I/O mode. The options are **native** and **threads**.
**cache**: disk cache mode. The value can be **none**, **writethrough**, **writeback**, or **directsync**.
**iothread**: I/O thread allocated to the disk.
**error_policy**: processing policy when an I/O write error occurs. The value can be stop, report, ignore, enospace, or retry.
**rerror_policy**: processing policy when an I/O read error occurs. The value can be stop, report, ignore, enospac, or retry.
**retry_interval**: I/O retry interval. The value ranges from 0 to MAX_INT, in milliseconds. This parameter can be set only when error_policy or rerror_policy is set to retry.
**retry_timeout**: I/O retry timeout interval. The value ranges from 0 to MAX_INT, in milliseconds. This parameter can be set only when error_policy or rerror_policy is set to retry. | +| target | The bus and device that a disk presents to a VM. | **dev**: specifies the logical device name of a disk, for example, sd\[a-p] for SCSI, SATA, and USB buses and hd\[a-d] for IDE disks.

**bus**: specifies the type of a disk. Common types include scsi, usb, sata, and virtio. | +| boot | The disk can be used as the boot disk. | **order**: specifies the disk startup sequence. | +| readonly | The disk is read-only and cannot be modified by the VM. Generally, it is used together with the CD-ROM drive. | - | + +#### Configuration Example + +After the VM image is prepared according to [Preparing a VM Image](./environment-preparation.md#preparing-a-vm-image), you can use the following XML configuration file to configure the virtual disk for the VM. + +For example, this example configures two I/O threads for the virtual machine, one for a block disk device, one for an optical disc device, and one for an RBD disk, and the first I/O thread is allocated to the block disk device for use. The backend medium of the disk device is in qcow2 format and is used as the preferred boot disk. +Before using an RBD disk, ensure that the qemu-block-rbd driver is installed. Run the following command as the **root** user to install the driver: + +```sh +yum install qemu-block-rbd +``` + +Configuration example: + +```xml + + ... + 2 + + + + + + + + + + + + + + + + + + + + + + ... + + +``` + +### Network Devices + +#### Overview + +The XML configuration file can be used to configure virtual network devices, including the ethernet mode, bridge mode, and vhostuser mode. This section describes how to configure vNICs. + +#### Elements + +In the XML configuration file, the element **interface** is used, and its attribute **type** indicates the mode of the vNIC. The options are **ethernet**, **bridge**, and **vhostuser**. The following uses the vNIC in bridge mode as an example to describe its subelements and attributes. + +**Table 1** Common subelements of a vNIC in bridge mode + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Subelement

+

Subelement Description

+

Attribute and Description

+

mac

+

The mac address of the vNIC.

+

address: specifies the mac address. If this parameter is not set, the system automatically generates a mac address.

+

target

+

Name of the backend vNIC.

+

dev: name of the created backend tap device.

+

source

+

Specify the backend of the vNIC.

+

bridge: used together with the bridge mode. The value is the bridge name.

+

boot

+

The NIC can be used for remote startup.

+

order: specifies the startup sequence of NICs.

+

model

+

Indicates the type of a vNIC.

+

type: virtio is usually used for the NIC in bridge mode.

+

virtualport

+

Port type

+

type: If an OVS bridge is used, set this parameter to openvswitch.

+

driver

+

Backend driver type

+

name: driver name. The value is vhost.

+

queues: the number of NIC queues.

+
+ +#### Configuration Example + +- After creating the Linux bridge br0 by referring to [Preparing a VM Image](./environment-preparation.md#preparing-a-vm-image), configure a vNIC of the VirtIO type bridged on the br0 bridge. The corresponding XML configuration is as follows: + + ```xml + + ... + + + + + + ... + + + ``` + +- If an OVS network bridge is created according to [Preparing a VM Image](./environment-preparation.md#preparing-a-vm-image), configure a VirtIO vNIC device that uses the vhost driver and has four queues. + + ```xml + + ... + + + + + + + + ... + + + ``` + +### Bus Configuration + +#### Overview + +The bus is a channel for information communication between components of a computer. An external device needs to be mounted to a corresponding bus, and each device is assigned a unique address \(specified by the subelement **address**\). Information exchange with another device or a central processing unit \(CPU\) is completed through the bus network. Common device buses include the ISA bus, PCI bus, USB bus, SCSI bus, and PCIe bus. + +The PCIe bus is a typical tree structure and has good scalability. The buses are associated with each other by using a controller. The following uses the PCIe bus as an example to describe how to configure a bus topology for a VM. + +> [!NOTE]NOTE +> The bus configuration is complex. If the device topology does not need to be precisely controlled, the default bus configuration automatically generated by libvirt can be used. + +#### Elements + +In the XML configuration of libvirt, each controller element \(**controller**\) represents a bus, and one or more controllers or devices can be mounted to one controller depending on the VM architecture. This topic describes common attributes and subelements. + +**controller**: controller element, which indicates a bus. + +- Attribute **type**: bus type, which is mandatory for the controller. The common values are **pci**, **usb**, **scsi**, **virtio-serial**, **fdc**, and **ccid**. +- Attribute **index**: bus number of the controller \(the number starts from 0\), which is mandatory for the controller. This attribute can be used in the **address** element. +- Attribute **model**: specific model of the controller, which is mandatory for the controller. The available values are related to the value of **type**. For details about the mapping and description, see [Table 1](#table191911761111). +- Subelement **address**: mount location of a device or controller on the bus network. + - Attribute **type**: device address type. The common values are **pci**, **usb**, or **drive**. The attribute varies according to the **type** of the **address**. For details about the common **type** attribute value and the corresponding **address** attribute, see [Table 2](#table1200165711314). + +- Subelement **model**: name of a controller model. + - Attribute **name**: name of a controller model, which corresponds to the **model** attribute in the parent element controller. + +**Table 1** Mapping between the common values of **type** and **model** for the controller. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Value of Type

+

Value of Model

+

Introduction

+

pci

+

pcie-root

+

PCIe root node, which can be used to mount PCIe devices or controllers.

+

pcie-root-port

+

Only one slot can be used to mount a PCIe device or controller.

+

pcie-to-pci-bridge

+

PCIe-to-PCI bridge controller, which can be used to mount PCI devices.

+

usb

+

ehci

+

USB 2.0 controller, which can be used to mount USB 2.0 devices.

+

nec-xhci

+

USB 3.0 controller, which can be used to mount USB 3.0 devices.

+

scsi

+

virtio-scsi

+

VirtIO SCSI controller, which can be used to mount block devices, such as disks and CD-ROMs.

+

virtio-serial

+

virtio-serial

+

VirtIO serial port controller, which can be used to mount serial port devices, such as a pty serial port.

+
+ +**Table 2** Attributes of the **address** element in different devices. + + + + + + + + + + + + + + + + + + + +

Value of Type

+

Description

+

Address

+

pci

+

The address type is PCI address, indicating the mount location of the device on the PCI bus network.

+

domain: domain ID of the PCI device.

+

bus: bus number of the PCI device.

+

slot: device number of the PCI device.

+

function: function number of the PCI device.

+

multifunction: (optional) specifies whether to enable the multifunction function.

+

usb

+

The address type is USB address, indicating the location of the device on the USB bus.

+

bus: bus number of the USB device.

+

port: port number of the USB device.

+

drive

+

The address type is storage device address, indicating the owning disk controller and its position on the bus.

+

controller: the number of the owning controller.

+

bus: channel number of the device output.

+

target: target number of the storage device.

+

unit: lun number of the storage device.

+
+ +#### Configuration Example + +This example shows the topology of a PCIe bus. Three PCIe-Root-Port controllers are mounted to the PCIe root node \(BUS 0\). The multifunction function is enabled for the first PCIe-Root-Port controller \(BUS 1\). A PCIe-to-PCI-bridge controller is mounted to the first PCIe-Root-Port controller to form a PCI bus \(BUS 3\). A virtio-serial device and a USB 2.0 controller are mounted to the PCI bus. A SCSI controller is mounted to the second PCIe-Root-Port controller \(BUS 2\). No device is mounted to the third PCIe-Root-Port controller \(BUS 0\). The configuration details are as follows: + +```xml + + ... + + + +
+ + +
+ + + +
+ + +
+ + +
+ + +
+ + +
+ + ... + + +``` + +### Other Common Devices + +#### Overview + +In addition to storage devices and network devices, some external devices need to be specified in the XML configuration file. This section describes how to configure these elements. + +#### Elements + +- **serial**: serial port device + + Attribute **type**: specifies the serial port type. The common attribute values are **pty**, **tcp**, **pipe**, and **file**. + +- **video**: media device + + Attribute **type**: media device type. The common attribute value of the AArch64 architecture is **virtio**, and that of the x86\_64 architecture is **vga** or **cirrus**. + + Subelement **model**: subelement of **video**, which is used to specify the media device type. + + In the subelement **model**, if **type** is set to **vga**, a Video Graphics Array \(VGA\) video card is configured. **vram** indicates the size of the video RAM, in KB by default. + + For example, if a 16 MB VGA video card is configured for an x86\_64 VM, configuration in the XML file is as follows. In the example, the value of **vram** is the size of video RAM, in KB by default. + + ```xml + + ``` + +- **input**: output device + + **type** attribute: specifies the type of the output device. The common attribute values are **tablet** and **keyboard**, indicating that the output device is the tablet and keyboard respectively. + + **bus**: specifies the bus to be mounted. The common attribute value is **USB**. + +- **emulator**: emulator application path +- **graphics**: graphics device + + **type** attribute: specifies the type of a graphics device. The common attribute value is **vnc**. + + **listen** attribute: specifies the IP address to be listened to. + +#### Configuration Example + +For example, in the following example, the VM emulator path, pty serial port, VirtIO media device, USB tablet, USB keyboard, and VNC graphics device are configured. + +> [!NOTE]NOTE +> When **type** of **graphics** is set to **VNC**, you are advised to set the **passwd** attribute, that is, the password for logging in to the VM using VNC. + +```xml + + ... + + /usr/libexec/qemu-kvm + + + + + + ... + + +``` + +## Configurations Related to the System Architecture + +### Overview + +The XML configuration file contains configurations related to the system architecture, which cover the mainboard, CPU, and some features related to the architecture. This section describes meanings of these configurations. + +### Elements + +- **os**: defines VM startup parameters. + + Subelement **type**: specifies the VM type. The attribute **arch** indicates the architecture type, for example, AArch64. The attribute **machine** indicates the type of VM chipset. Supported chipset type can be queried by running the **qemu-kvm -machine ?** command. For example, the AArch64 architecture supports the **virt** type. + + Subelement **loader**: specifies the firmware to be loaded, for example, the UEFI file provided by the EDK. The **readonly** attribute indicates whether the file is read-only. The value can be **yes** or **no**. The **type** attribute indicates the **loader** type. The common values are **rom** and **pflash**. + + Subelement **nvram**: specifies the path of the **nvram** file, which is used to store the UEFI startup configuration. + +- **features**: Hypervisor controls some VM CPU/machine features, such as the advanced configuration and power interface \(ACPI\) and the GICv3 interrupt controller specified by the ARM processor. + +### Example for AArch64 Architecture + +The VM is of the **aarch64** type and uses **virt** chipset. The VM configuration started using UEFI is as follows: + +```xml + + ... + + hvm + /usr/share/edk2/aarch64/QEMU_EFI-pflash.raw + /var/lib/libvirt/qemu/nvram/openEulerVM.fd + + ... + +``` + +Configure ACPI and GIC V3 interrupt controller features for the VM. + +```xml + + + + +``` + +### Example for x86\_64 Architecture + +The x86\_64 architecture supports both BIOS and UEFI boot modes. If **loader** is not configured, the default BIOS boot mode is used. The following is a configuration example in which the UEFI boot mode and Q35 chipsets are used. + +```xml + + ... + + hvm + /usr/share/edk2/ovmf/OVMF.fd + + ... + +``` + +## Other Common Configuration Items + +### Overview + +In addition to system resources and virtual devices, other elements need to be configured in the XML configuration file. This section describes how to configure these elements. + +### Elements + +- **iothreads**: specifies the number of **iothread**, which can be used to accelerate storage device performance. + +- **on\_poweroff**: action taken when a VM is powered off. +- **on\_reboot**: action taken when a VM is rebooted. +- **on\_crash**: action taken when a VM is on crash. +- **clock**: indicates the clock type. + + **offset** attribute: specifies the VM clock synchronization type. The value can be **localtime**, **utc**, **timezone**, or **variable**. + +### Configuration Example + +Configure two **iothread** for the VM to accelerate storage device performance. + +```xml +2 +``` + +Destroy the VM when it is powered off. + +```xml +destroy +``` + +Restart the VM. + +```xml +restart +``` + +Restart the VM when it is crashed. + +```xml +restart +``` + +The clock uses the **utc** synchronization mode. + +```xml + +``` + +## XML Configuration File Example + +### Overview + +This section provides XML configuration files of a basic AArch64 VM and a x86\_64 VM as two examples for reference. + +### Example 1 + +An XML configuration file of AArch64 VM, which contains basic elements. The following is a configuration example: + +```xml + + openEulerVM + 8 + 4 + + hvm + /usr/share/edk2/aarch64/QEMU_EFI-pflash.raw + /var/lib/libvirt/qemu/nvram/openEulerVM.fd + + + + + + + + + 1 + + destroy + restart + restart + + /usr/libexec/qemu-kvm + + + + + + + + + + + + + + + + + + + + + + + + + + + +``` + +### Example 2 + +An XML configuration file of x86\_64 VM, which contains basic elements and bus elements. The following is a configuration example: + +```xml + + openEulerVM + 8388608 + 8388608 + 4 + 1 + + hvm + + + + + + + + + destroy + restart + restart + + /usr/libexec/qemu-kvm + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ + + + + + +