From 21c11309059c0a735655ed6dc2a894959f9a00b1 Mon Sep 17 00:00:00 2001 From: neige214 <1316541565@qq.com> Date: Tue, 21 Dec 2021 16:51:07 +0800 Subject: [PATCH] Intership opt --- .../optimized-README/A-Tune-Collector-en.md | 210 +++++++++ .../optimized-README/A-Tune-UI-en.md | 123 +++++ .../optimized-README/A-Tune-en.md | 211 +++++++++ .../optimized-README/Anbox-en.md | 167 +++++++ .../optimized-README/AvxToNeon-en.md | 76 +++ .../optimized-README/attest-tools-en.md | 276 +++++++++++ .../optimized-README/auto_py2to3-en.rst | 81 ++++ .../optimized-README/avocado-vt-en.rst | 23 + .../optimized-README/bishengjdk-11-en.md | 27 ++ .../optimized-README/bishengjdk-8-en.md | 27 ++ .../optimized-README/bishengjdk-riscv-en.md | 9 + .../optimized-README/ci-bot-en.md | 63 +++ .../optimized-README/clibcni-en.md | 16 + .../optimized-README/cloudnative-en.md | 156 +++++++ .../optimized-README/community-en-opt.md | 57 +++ .../optimized-README/community-issue-en.md | 24 + .../optimized-README/compass-ci-en.md | 213 +++++++++ .../digest-list-tools-en-opt.md | 437 ++++++++++++++++++ .../optimized-README/docs-en.md | 58 +++ 19 files changed, 2254 insertions(+) create mode 100644 translation-projects/README-optimization/optimized-README/A-Tune-Collector-en.md create mode 100644 translation-projects/README-optimization/optimized-README/A-Tune-UI-en.md create mode 100644 translation-projects/README-optimization/optimized-README/A-Tune-en.md create mode 100644 translation-projects/README-optimization/optimized-README/Anbox-en.md create mode 100644 translation-projects/README-optimization/optimized-README/AvxToNeon-en.md create mode 100644 translation-projects/README-optimization/optimized-README/attest-tools-en.md create mode 100644 translation-projects/README-optimization/optimized-README/auto_py2to3-en.rst create mode 100644 translation-projects/README-optimization/optimized-README/avocado-vt-en.rst create mode 100644 translation-projects/README-optimization/optimized-README/bishengjdk-11-en.md create mode 100644 translation-projects/README-optimization/optimized-README/bishengjdk-8-en.md create mode 100644 translation-projects/README-optimization/optimized-README/bishengjdk-riscv-en.md create mode 100644 translation-projects/README-optimization/optimized-README/ci-bot-en.md create mode 100644 translation-projects/README-optimization/optimized-README/clibcni-en.md create mode 100644 translation-projects/README-optimization/optimized-README/cloudnative-en.md create mode 100644 translation-projects/README-optimization/optimized-README/community-en-opt.md create mode 100644 translation-projects/README-optimization/optimized-README/community-issue-en.md create mode 100644 translation-projects/README-optimization/optimized-README/compass-ci-en.md create mode 100644 translation-projects/README-optimization/optimized-README/digest-list-tools-en-opt.md create mode 100644 translation-projects/README-optimization/optimized-README/docs-en.md diff --git a/translation-projects/README-optimization/optimized-README/A-Tune-Collector-en.md b/translation-projects/README-optimization/optimized-README/A-Tune-Collector-en.md new file mode 100644 index 0000000..ec1eab5 --- /dev/null +++ b/translation-projects/README-optimization/optimized-README/A-Tune-Collector-en.md @@ -0,0 +1,210 @@ +English | [简体中文](./README.md) +# A-Tune-Collector + +#### Introduction +The A-Tune-Collector is used to collect various system resources. It can also be used as a collector of the A-Tune projects. + +#### Installation + +If the collector is used for the A-Tune project. Please run the following command to install the collector. + +``` +python3 setup.py install +``` + +#### Instructions + +Collection Command Format + +``` +cd atune_collector +python3 collect_data.py [OPTIONS] +``` + +Parameter Description + +| Parameter | Description | +| ------------ | ------------------------------------------------------------ | +| --config, -c | JSON file to be parsed. The JSON file is used to configure the system resource information to be collected. If this parameter is not specified. The **/etc/atune_collector/collect_data.json** file is read by default. | + +Example + +- Use the default resource collection configuration file. + + ``` + python3 collect_data.py + ``` + +- Use the specified resource collection configuration file. + + ``` + python3 collect_data.py -c collect_data.json + ``` + +Configuration Description + +Table 1 **collect_data.json** file + +| Parameter | Description | Type | Value Range | +| ---------------- | ------------------------------------------------------------ | ---------------- | ----------- | +| network | NIC to be collected. | Character string | - | +| block | Disk to be collected. | Character string | - | +| sample_num | Sample number to be collected. | Integer | > 0 | +| interval | Interval for collecting data, in seconds. | Integer | > 0 | +| output_dir | Path for storing collected data. | Character string | - | +| workload_type | Application load type of the collection environment. The default value is **default**. | Character string | - | +| collection_items | Table 2 lists the system parameters to be collected. | List | - | + +Table 2 Description of the **collection_items** configuration + +| Parameter | Description | Type | Value Range | +| --------- | ------------------------------------------------------------ | ---------------- | ----------- | +| name | Name of the item to be collected. | Character string | - | +| module | Category of the item to be collected. The category must match the definition of the corresponding collection module. | Character string | - | +| purpose | Type of the item to be collected. The type must match the definition of the corresponding collection module. | Character string | - | +| metrics | Indicators of the item to be collected. | List | - | +| threshold | Threshold of the item to be collected. | Integer | - | + +Example + +The following is an example of the **collect_data.json** file. + +``` +{ + "network": "eth0", + "block": "sda", + "sample_num": 20, + "interval": 5, + "output_dir": "/var/atuned/collect_data", + "workload_type": "default", + "collection_items": [ + { + "name": "cpu", + "module": "CPU", + "purpose": "STAT", + "metrics": [ + "usr", + "nice", + "sys", + "iowait", + "irq", + "soft", + "steal", + "guest", + "util", + "cutil" + ], + "threshold": 30 + }, + { + "name": "storage", + "module": "STORAGE", + "purpose": "STAT", + "metrics": [ + "rs", + "ws", + "rMBs", + "wMBs", + "rrqm", + "wrqm", + "rareq-sz", + "wareq-sz", + "r_await", + "w_await", + "util", + "aqu-sz" + ] + }, + { + "name": "network", + "module": "NET", + "purpose": "STAT", + "metrics": [ + "rxkBs", + "txkBs", + "rxpcks", + "txpcks", + "ifutil" + ] + }, + { + "name": "network-err", + "module": "NET", + "purpose": "ESTAT", + "metrics": [ + "errs", + "util" + ] + }, + { + "name": "mem.band", + "module": "MEM", + "purpose": "BANDWIDTH", + "metrics": [ + "Total_Util" + ] + }, + { + "name": "perf", + "module": "PERF", + "purpose": "STAT", + "metrics": [ + "IPC", + "CACHE-MISS-RATIO", + "MPKI", + "ITLB-LOAD-MISS-RATIO", + "DTLB-LOAD-MISS-RATIO", + "SBPI", + "SBPC" + ] + }, + { + "name": "vmstat", + "module": "MEM", + "purpose": "VMSTAT", + "metrics": [ + "procs.b", + "memory.swpd", + "io.bo", + "system.in", + "system.cs", + "util.swap", + "util.cpu", + "procs.r" + ] + }, + { + "name": "sys.task", + "module": "SYS", + "purpose": "TASKS", + "metrics": [ + "procs", + "cswchs" + ] + }, + { + "name": "sys.ldavg", + "module": "SYS", + "purpose": "LDAVG", + "metrics": [ + "runq-sz", + "plist-sz", + "ldavg-1", + "ldavg-5" + ] + }, + { + "name": "file.util", + "module": "SYS", + "purpose": "FDUTIL", + "metrics": [ + "fd-util" + ] + } + ] +} +``` + +#### Related Information + +A-Tune project：https://gitee.com/openeuler/A-Tune diff --git a/translation-projects/README-optimization/optimized-README/A-Tune-UI-en.md b/translation-projects/README-optimization/optimized-README/A-Tune-UI-en.md new file mode 100644 index 0000000..e98f68a --- /dev/null +++ b/translation-projects/README-optimization/optimized-README/A-Tune-UI-en.md @@ -0,0 +1,123 @@ +English | [简体中文](./README.md) + +## Introduction + +**A-Tune-UI** is a web project that relies on [A-Tune](https://gitee.com/openeuler/A-Tune). It only supports openEuler-20.09 or later. + + + +## Installation & Usage + +### Approach 1: Install and use locally + +#### 1. Prepare + +Some packages required for the UI, such as **nodejs** and **npm**, can only be found in **openeuler-everything.iso**. +Before the installation, add **openeuler-everything** to your Yum repo: + +1. Open [openeuler.org](https://openeuler.org/en/) and choose **Download** > **Software Packages**. On the page that is displayed, locate the desired image source and click **Download**. +2. Click **everything** and click the desired architecture (`x86_64/aarch64`). The page address is the Yum repo URL. +3. Configure the Yum repo using the URL. + +#### 2. Install + +#### 2-1. Install using the shell script + +```bash +sh install.sh +``` + +This script will clone node-sass from Gitee by default. Users can change the URL of the node-sass package by themselves. Here is the command: +```bash +sh install.sh [git_url] +# For example, you can use the GitHub URL to clone code. +# sh install.sh https://github.com/sass/node-sass.git +``` + +If the installation fails, please try to install it manually. + + +#### 2-2. Install manually + +##### 1) Install dependent system software packages + +```bash +yum install -y npm nodejs gcc-c++ make patch +``` + +##### 2) Install dependent packages for NPM + +```bash +npm ci +``` +##### 3) Compile the node-sass package + +```bash +git clone -b v5 --recursive https://github.com/sass/node-sass.git +cd node-sass +git am arm-support.patch +npm i +node scripts/build -f +``` + +##### 4) Move node-sass into A-Tune-UI + +```bash +mv node-sass A-Tune-UI/node_modules +``` + +#### 3. (Optional) Change the IP address + +If your web page does not open in the local host, change your web IP address as follows: + +```bash +hostname -I # To get host IP +``` +Open the **package.json** file and replace **localhost** with your host IP address in line 10. + +#### 4. Run + +```bash +npm run start +``` +This command will return the URL of the web page. + +**Note:** This project is still in the development phase. If you encounter any problem during installation or running, please locate the problem by referring to section 4 "FAQs" in the [A-Tune-UI Operation Guide](./Documentation/A-Tune-UI操作指南.md). If the problem persists, submit an issue in the code repository. + +### Approach 2: Install and use the docker image + +#### 1. Prepare + +Before installation, make sure Docker and wget tools have already been installed: (You do not need to get all code. Just get the Dockerfile.) +```bash +yum install -y docker wget +wget https://gitee.com/openeuler/A-Tune-UI/Dockerfile +``` + +#### 2. Generate the Docker image + +```bash +docker build --network=host -t atune-ui:latest . +``` + +#### 3. Run + +```bash +docker run -p ::8080 -e ENG_HOST= -e ENG_PORT= atune-ui +``` + +**Note:** + +- **local_ip** should be your IP address and **local_port** should be a port that has not been used. After running, you can open the web page by using the URL: http://:. +- **engine-host** and **engine-port** are the same as the IP address and port information you set in the A-Tune engine.cnf file. + +> **Note:** If A-Tune is not running, you can still get the web page but there is no data on your page. + + +## Related Information + +##### A-Tune +A-Tune project: https://gitee.com/openeuler/A-Tune + +##### A-Tune-Collector +A-Tune-Collector project: https://gitee.com/openeuler/A-Tune-Collector diff --git a/translation-projects/README-optimization/optimized-README/A-Tune-en.md b/translation-projects/README-optimization/optimized-README/A-Tune-en.md new file mode 100644 index 0000000..1981435 --- /dev/null +++ b/translation-projects/README-optimization/optimized-README/A-Tune-en.md @@ -0,0 +1,211 @@ + + +English | [简体中文](./README-zh.md) + +## Introduction to A-Tune + +**A-Tune** is an OS tuning engine powered by artificial intelligence (AI). A-Tune makes use of AI to help the OS understand services, simplify IT system tuning, and improve application performance. + + +I. A-Tune Installation +------------ + +Supported OS: openEuler 20.03 LTS or later + +### Method 1 (applicable to common users): Use the default A-Tune of openEuler. + +```bash +yum install -y atune +``` +For openEuler 20.09 or later, an atune-engine is needed. + +```bash +yum install -y atune-engine +``` + +### Method 2 (applicable to developers): Use the source code of the local repository for installation. + +#### 1. Install dependent system software packages. +```bash +yum install -y golang-bin python3 perf sysstat hwloc-gui +``` + +#### 2. Install Python dependency packages. + +#### 2.1 Install dependency packages for the A-Tune service. +```bash +yum install -y python3-dict2xml python3-flask-restful python3-pandas python3-scikit-optimize python3-xgboost python3-pyyaml +``` +Or +```bash +pip3 install dict2xml Flask-RESTful pandas scikit-optimize xgboost scikit-learn pyyaml +``` + +#### 2.2 (Optional) Install dependency packages for the database. +If you already have the database application installed and want to save A-Tune collection and tuning data to the database, you must also install the following packages: +```bash +yum install -y python3-sqlalchemy python3-cryptography +``` +Or +```bash +pip3 install sqlalchemy cryptography +``` +To use the database, you must also install the database application's dependencies using one of the methods listed below. +| **Database** | **Install Using yum** | **Install Using pip** | +| ------------ | ------------------------------- | --------------------- | +| PostgreSQL | yum install -y python3-psycopg2 | pip3 install psycopg2 | +#### 3. Download the source code. +```bash +git clone https://gitee.com/openeuler/A-Tune.git +``` + +#### 4. Compile. +```bash +cd A-Tune +make +``` + +#### 5. Install. +```bash +make collector-install +make install +``` + +II. Quick Guide +------------ + +### 1. Configure the A-Tune service. + +#### Change the network and disk configuration in the atuned.cnf file. + +You can run the following command to query the NIC that needs to be specified for data collection or optimization and change the value of the network configuration item in the **/etc/atuned/atuned.cnf** file to the specified NIC. + +```bash +ip addr +``` + +You can run the following command to query the disk that needs to be specified for data collection or optimization and change the value of the disk configuration item in the **/etc/atuned/atuned.cnf** file to the specified disk. + +```bash +fdisk -l | grep dev +``` + +### 2. Manage the A-Tune service. + +#### Load and start the atuned and atune-engine services. + +```bash +systemctl daemon-reload +systemctl start atuned +systemctl start atune-engine +``` + +#### Check the status of the atuned and atune-engine services. + +```bash +systemctl status atuned +systemctl status atune-engine +``` + +### 3. Generate AI models. + +You can save the collected data to the **A-Tune/analysis/dataset** directory and run the model generation tool to update the AI model in the **A-Tune/analysis/models** directory. + +**Format** + +python3 generate_models.py + +**Parameter Description** + +- OPTIONS + +| Parameter | Description | +| ---------------- | ------------------------------------------------------------ | +| --csv_path, -d | Path for storing CSV files required for model training. The default directory is **A-Tune/analysis/dataset**. | +| --model_path, -m | Path for storing the new models generated during training. The default path is **A-Tune/analysis/models**. | +| --select, -s | Indicates whether to generate feature models. The default value is **false**. | +| --search, -g | Indicates whether to enable parameter space search. The default value is **false**. | + +Example: + +``` +python3 generate_models.py +``` + +### 4. Run the atune-adm commands. + +#### list command + +This command is used to list the supported profiles as well as active profiles. + +Format: + +atune-adm list + +Example: + +```bash +atune-adm list +``` + +#### profile command + +This command is used to activate the profile to make it in the active state. + +Format: + +atune-adm profile + +Example: Activate the profile corresponding to the web-nginx-http-long-connection. + +```bash +atune-adm profile web-nginx-http-long-connection +``` + +#### analysis command (online static tuning) + +This command is used to collect real-time statistics from the system to identify and optimize workload types. + +Format: + +atune-adm analysis [OPTIONS] + +Example 1: Use the default model to identify applications and perform automatic tuning. + +```bash +atune-adm analysis +``` + +Example 2: Use the user-defined model for recognition. + +```bash +atune-adm analysis --model /usr/libexec/atuned/analysis/models/new-model.m +``` + +#### tuning command (offline dynamic tuning) + +Use the specified project file to search the dynamic space for parameters and find the optimal solution under the current environment configuration. + +Format: + +atune-adm tuning [OPTIONS] + +Example: See [the A-Tune offline tuning example](./examples/tuning). Each example has a corresponding README guide. + +For details about other commands, see the atune-adm help information or [A-Tune User Guide](./Documentation/UserGuide/A-Tune-User-Guide.md). + +III. Web UI +-------- + +[A-Tune-UI](https://gitee.com/openeuler/A-Tune-UI) is a web project based on A-Tune. Please check A-Tune-UI [README](https://gitee.com/openeuler/A-Tune-UI/blob/master/README.en.md) for details. + +IV. How to Contribute +-------- + +We welcome new contributors to take part in the project, and we are happy to provide guidance for new contributors. Please sign [CLA](https://openeuler.org/en/cla.html) before contribution. + +### Mailing list +If you have any question, please contact [A-Tune](https://mailweb.openeuler.org/postorius/lists/a-tune.openeuler.org/). + +### Routine meeting +The SIG meeting is hold at 10:00-12:00 AM on Friday every two weeks. Please send your issues to the [A-Tune](https://mailweb.openeuler.org/postorius/lists/a-tune.openeuler.org/) mailing list. diff --git a/translation-projects/README-optimization/optimized-README/Anbox-en.md b/translation-projects/README-optimization/optimized-README/Anbox-en.md new file mode 100644 index 0000000..c9f0320 --- /dev/null +++ b/translation-projects/README-optimization/optimized-README/Anbox-en.md @@ -0,0 +1,167 @@ +# Anbox + +This code repository is forked from [https://github.com/anbox/anbox](https://github.com/anbox/anbox).And it will be maintained under the openEuler android middleware community. We aim to run anbox on Arm PC natively. + +Anbox is a container-based approach to boot a full Android system on a regular GNU/Linux system like Ubuntu. In other words: Anbox will let you run Android on your Linux system without the slowness of virtualization. + +## Overview + +Anbox uses Linux namespaces (user, pid, uts, net, mount, ipc) to run a full Android system in a container and provide Android applications on any GNU/Linux-based platform. + +The Android inside the container has no direct access to any hardware. All hardware access is going through the anbox daemon on the host. We're reusing what Android implemented within the QEMU-based emulator for OpenGL ES accelerated rendering. The Android system inside the container uses different pipes to communicate with the host system and sends all hardware access commands through these. + +For more details, have a look at the following documentation pages: + + * [Android Hardware OpenGL ES emulation design overview](https://android.googlesource.com/platform/external/qemu/+/emu-master-dev/android/android-emugl/DESIGN) + * [Android QEMU fast pipes](https://android.googlesource.com/platform/external/qemu/+/emu-master-dev/android/docs/ANDROID-QEMU-PIPE.TXT) + * [The Android "qemud" multiplexing daemon](https://android.googlesource.com/platform/external/qemu/+/emu-master-dev/android/docs/ANDROID-QEMUD.TXT) + * [Android qemud services](https://android.googlesource.com/platform/external/qemu/+/emu-master-dev/android/docs/ANDROID-QEMUD-SERVICES.TXT) + +Anbox is currently suited for the desktop use case but can be used on mobile operating systems like Ubuntu Touch, Sailfish OS, or Lune OS too. However, because Android application mapping is presently desktop-specific, further work is required to support stacked window user interfaces as well. + +The Android runtime environment ships with a minimal customized Android system image based on the [Android Open Source Project](https://source.android.com/). The used image is currently based on Android 7.1.1. + +## Supported Linux Distributions + +At the moment we officially support the following Linux distributions: + + * Ubuntu 16.04 (xenial) + * Ubuntu 18.04 (bionic) + * UOS + +However, all other distributions should work as well as long as they provide the mandatory kernel modules (see kernel/). + + * [Release notes](docs/release-notes/anbox-release-notes.md) + + +## Build from source + +### Requirements + +It is recommended that the machine run on the ARM64 architecture. + +To build the Anbox runtime itself there is nothing special to know. We're using cmake as a build system. A few build dependencies need to be present on your host system: + + * libdbus + * google-mock + * google-test + * libboost + * libboost-filesystem + * libboost-log + * libboost-iostreams + * libboost-program-options + * libboost-system + * libboost-test + * libboost-thread + * libcap + * libsystemd + * mesa (libegl1, libgles2) + * libsdl2 + * libprotobuf + * protobuf-compiler + * lxc (>= 3.0) + * libasound + +On an Ubuntu system, you can install all build dependencies with the following command: + +``` +$ sudo apt install build-essential cmake cmake-data debhelper dbus google-mock \ + libboost-dev libboost-filesystem-dev libboost-log-dev libboost-iostreams-dev \ + libboost-program-options-dev libboost-system-dev libboost-test-dev \ + libboost-thread-dev libcap-dev libsystemd-dev libegl1-mesa-dev \ + libgles2-mesa-dev libglm-dev libgtest-dev liblxc1 \ + libproperties-cpp-dev libprotobuf-dev libsdl2-dev libsdl2-image-dev lxc-dev \ + pkg-config protobuf-compiler libasound2-dev +``` +We recommend Ubuntu 18.04 (bionic) with **GCC 7.x** as your build environment. + +On a UOS system you can install all build dependencies with the following command: + +``` +$ sudo apt install gcc libncurses-dev bison flex libssl-dev cmake dkms build-essential \ + cmake-data debhelper dbus google-mock libboost-dev libboost-filesystem-dev libboost-log-dev \ + libboost-iostreams-dev libboost-program-options-dev libboost-thread-dev libcap-dev \ + libsystemd-dev libegl1-mesa-dev libgles2-mesa-dev libglm-dev libgtest-dev liblxc1 \ + libproperties-cpp-dev libprotobuf-dev libsdl2-dev libsdl2-image-dev lxc-dev libdw-dev \ + libbfd-dev libdwarf-dev pkg-config protobuf-compiler libboost-test-dev +``` + +### Build + +``` +$ mkdir -p /home/compile/ +$ cd /home/compile/ +$ git clone https://gitee.com/openeuler/anbox +``` + + * [Apply SDL patch](docs/apply_SDL_patch.md) + * [Install binder & ashmem module](docs/kernel_module.md) + +Afterward, you can build Anbox with + +``` +$ cd /home/compile/anbox +$ mkdir build +$ cd build +$ cmake .. -DCMAKE_CXX_FLAGS="-DENABLE_TOUCH_INPUT -Wno-error=implicit-fallthrough \ + -Wno-error=missing-field-initializers" -DCMAKE_BUILD_TYPE=Release -DWerror=OFF +$ make -j8 +``` + +If you want to choose a version, please do: + +``` +$ git clone https://gitee.com/openeuler/anbox -b anbox-v1.0-rc3 +``` + +The version anbox-v1.0-rc3 will update in the future, make sure it matches the version of AOSP(see build-android.md). + + +A simple command will install the necessary bits into your system: + +``` +$ sudo make install +``` + +## Run Anbox + +Step1: Start container manager, run the commands as the ROOT user and you must have an android.img. + +``` +$ bash /usr/local/bin/anbox-bridge.sh start +$ anbox container-manager --android-image=//android.img \ + --data-path=//anbox-data --privileged --daemon & +``` + +Step2: Start session manager, run the commands as the NON-ROOT user, and run the shell terminal opened from desktop UI. + +``` +$ export EGL_PLATFORM=x11 +$ export EGL_LOG_LEVEL=fatal +$ anbox session-manager --gles-driver=host & +``` + +Wait for the 30s, open the Android APP in "other applications" in the app menu. + +## Documentation + +You will find more documentation for Anbox in the *docs* subdirectory of the project source. + +Interesting things to have a look at + + * [Runtime Setup](docs/runtime-setup.md) + +## Reporting bugs + +If you have found an issue with Anbox, please [file a bug](https://gitee.com/openeuler/anbox/issues). + +## Get in Touch + +Find maintainer here: +https://gitee.com/openeuler/community/tree/master/sig/sig-android-middleware + +## Copyright and Licensing + +Anbox reuses code from other projects like the Android QEMU emulator. These projects are available in the external/ subdirectory with the licensing terms included. + +The Anbox source itself, if not stated differently in the relevant source files, is licensed under the terms of the GPLv3 license. diff --git a/translation-projects/README-optimization/optimized-README/AvxToNeon-en.md b/translation-projects/README-optimization/optimized-README/AvxToNeon-en.md new file mode 100644 index 0000000..e8a5dac --- /dev/null +++ b/translation-projects/README-optimization/optimized-README/AvxToNeon-en.md @@ -0,0 +1,76 @@ +# AvxToNeon + +#### Description +AvxToNeon is a system acceleration library for porting from x86 architecture to Kunpeng architecture. +When applications using Intel intrinsic instructions are ported from the x86 architecture to Kunpeng architecture, the instructions need to be further developed because the Arm64 instruction names and functions are different from that of x86. As a result, huge porting workloads are caused. In this project, the frequently used AVX instructions are encapsulated as independent modules to reduce repeated development workload. The AVX instructions are replaced with related NEON SIMD instructions, while the instruction names and functions remain unchanged. Users can invoke the corresponding instructions by importing related header files into the application software. + +#### Software Architecture +- data: folder of test data +- tests: folder of test source code +- avx2neon.h: external header file +- avx512intrin.h: internal header file +- avxintrin.h: internal header file +- emmintrin.h: internal header file +- immintrin.h: internal header file +- typedefs.h: internal header file +- supportedlist.md: list of supported interfaces + +#### Instructions + +1. Get AvxToNeon source code +2. Mask immintrin.h +3. Include avx2neon.h in your application +4. Add compilation options, ARCH_CFLAGS = -march=armv8-a+fp+simd+crc + +#### License + +It is licensed under the [APACHE LICENSE, VERSION 2.0](https://www.apache.org/licenses/LICENSE-2.0). For details, see the license file. + +#### Test + +This project also provides interface test cases for developers. The logic implementation code of test cases is located in the tests directory, and the input data and expected output of the test cases are in the data directory. Use the following commands to perform test cases: + +``` +(1) cd tests +(2) make +(3) ./test +``` + +After the **test** command is executed, information similar to the following is displayed on the console: + +``` +Running Test MM512_CASTPS128_PS512 + +... + +Running Test MM256_SET_EPI32 + +AVX2NEONTest Complete: Passed 265 tests: Failed 0 +``` + + All the instructions provided in this project have been verified on CentOS Linux release 7.6.1810 (AltArch) and EulerOS V2.0SP8, and GCC 7.3, GCC 4.8.5, and GCC 9.2.0. + +#### More Information + +For more information, visit + + + +If you have questions or comments, we encourage you to create an issue on gitee. You calsolse contact the developer's team directly by sending an email to + + [kunpengcompute@huawei.com](mailto:kunpengcompute@huawei.com). + +#### Obtaining Code + +If you wish to get the source code of functions listed in supportedlist.md, you can send an email to [kunpengcompute@huawei.com](mailto:kunpengcompute@huawei.com). + +#### Contribution + +1. Fork the repository +2. Create Feat_xxx branch +3. Commit your code +4. Create Pull Request + +#### Copyright + +Copyright © 2020 Huawei Corporation. All rights reserved. diff --git a/translation-projects/README-optimization/optimized-README/attest-tools-en.md b/translation-projects/README-optimization/optimized-README/attest-tools-en.md new file mode 100644 index 0000000..4792a0f --- /dev/null +++ b/translation-projects/README-optimization/optimized-README/attest-tools-en.md @@ -0,0 +1,276 @@ +# attest-tools + +## INTRODUCTION + +Managing the complete lifecycle of remote attestation can be very complicated. A TSS library performs only part of the operations necessary for the creation and verification of TPM objects. Other open-source solutions, such as IBM Attestation Client-Server (ACS), have fixed purposes and for this reason, they are difficult to customize and integrate into other solutions. + +attest-tools is a set of libraries and tools that can be used for managing the complete lifecycle of remote attestation. Its advantages are: + +- library API: main functions are exposed as a library: applications wanting to provide Trusted Computing services can simply link this library; + +- modular design: variable parts, such as event log parsers or verifiers, are modularized; attest-tools functionality can be easily extended with third-party modules that support new event log format or new verifiers; + +- simplicity: data to be processed and the status of the processing are stored in contexts created by the applications; library functions directly operate on these contexts; + +- completeness: the library provides functions for all phases of the remote attestation lifecycle; it supports both implicit and explicit remote attestation. + + + +## EXPLICIT vs IMPLICIT ATTESTATION + +Explicit attestation means establishing a communication between a target system and a verifier to determine whether the former satisfies the requirements of the latter. It is called explicit because the goal of the communication is explicitly to attest to a system. + +Implicit attestation means that attestation is not the primary purpose of the communication, but requirements can be verified as part of the establishment or execution of another protocol (e.g. TLS). This alternative should be preferable to the explicit one, as it is easier to integrate into legacy products. On the other end, implicit attestation introduces additional challenges such as having a fixed representation of the system state that can be translated into one or multiple Platform Configuration Register values. + +attest-tool allows verifiers to perform both explicit and implicit attestation. + + + +## SOFTWARE ARCHITECTURE + +attest-tools are composed of different libraries, RA client/server, and TLS client/server. + + + +---------------------+ +---------------------+ + | attest-tools client | | attest-tools server | + +---------------------+ +---------------------- + +----------+ +---------------------+ +---------------------+ + | SKAE lib | | enroll client lib | | enroll server lib | + +----------+ +---------------------+ +---------------------+ + +-------------------------------------------------------------------------+ + | Application API (north-bound) | + | +--------------------+ +---------------------+ +------+ +-----+ +-----+ | + | | | | | | util | | pcr | | ctx | | + | | Event log | | Verifier | +------+ +-----+ +-----+ | + | | | | | +----------+ +-----+ | + | | | | | | ctx_json | | tss | | + | +--------------------+ +---------------------+ +----------+ +-----+ | + | base lib | + | Developer API (south-bound) | + +-------------------------------------------------------------------------+ + +--------------------+ +---------------------+ + | event log parsers | | event log verifiers | + | (plugins) | | (plugins) | + +--------------------+ +---------------------+ + + +### Base lib - libattest.so + +The base library is the main library of attest-tool and provides basic services for the other more complex libraries. Essentially, it is +responsible to manage the data and verifier contexts, and verifying the TPM structure TPMS_ATTEST (which might contain a quote or a certified info). The base library exposes a north-bound interface (Application API), intended to be used by applications, and a south-bound interface (Developer API), intended to be used by developers willing to extend the library functionality. + + +### SKAE lib - libskae.so + +The SKAE library is the library that manages the Subject Key Attestation Evidence X.509 extension defined by TCG: + +https://trustedcomputinggroup.org/wp-content/uploads/IWG_SKAE_Extension_1-00.pdf + +It offers functions for the creation and the verification of the data inside the extension. + + +### Enroll client lib - libenroll_client.so + +The enrollment library for clients is responsible for performing the preliminary steps necessary for remote attestation: obtaining an AK certificate from the Privacy CA (implemented by attest_tools_server), for obtaining a certificate for generated TPM keys (which can be used with openssl_tpm2_engine), and for verifying a quote. + + +### Enroll server lib - libenroll_server.so + +The enrollment library for servers is responsible to verify the requests sent by clients and issuing certificates. The function to issue certificates can be replaced with functions from applications' developers. + + +### Event log parsers - libeventlog_*.so + +These libraries implement parsers for event logs. Currently, the following parsers are available: + +- BIOS event log (TCG v1.2 and v2.0); + +- IMA event log (ima-ng and ima-sig templates, little endian). + + +### Event log verifiers - libverifier_*.so + +These libraries implement verifiers for event logs. Each verifier might take requirements from remote attestation verifiers, that must be satisfied by the system being attested. Currently, the following verifiers are available: + +- BIOS: no checks are done at the moment; verifiers must explicitly specify 'always-true' as a requirement to skip verification; + +- IMA boot aggregate: calculates the digest of PCRs 0-7 obtained by simulating the PCR extend operation with digests from the provided event logs, and compares the result with the file digest in the first entry of the IMA measurement list; + +- IMA cp: obtains the path of the files from the IMA measurements list that don't have a signature, so that they can be transferred from systems being attested to remote attestation verifiers; + +- IMA sig: verifies the signature of files by using the content of /etc/keys/x509_ima.der (must be provided) as certificate; verifier must accept the public key by providing the certificate common name as a requirement (in the future, it must check the validity of the certificate); + +- IMA policy: checks that the loaded IMA policy is one of the pre-defined types (currently, only the 'exec-policy' type is defined); the desired policy type must be specified by the verifier as a requirement; + +- EVM key: checks that the EVM key is securely generated by the TPM (this probably will change, as it seems that the TCG specification doesn't allow the sensitive data origin bit to be set for sealed data blobs); it also checks the PolicyPCR policy by using the event logs supplied by the system being attested and the PCR selection provided by the verifier as a requirement. + + +### RA client - attest_ra_client + +Contacts RA server for AK/TLS certificate and for verifying a quote. Iusesse TCP/IP for communication. + + +### RA server - attest_ra_server + +Processes requests from RA client It uses TCP/IP for communication. + + +### TLS client - attest_tls_client + +It establishes a TLS communication with the TLS server. Before establishing TLS, it exchanges attestation data with the TLS server, so that both client and server certificates (the SKAE extension) can be verified. + + +### TLS server - attest_tls_server + +It receives requests from the TLS client. Before establishing TLS, it exchanges attestation data with the TLS clients, so that both client and server certificates (the SKAE extension) can be verified. + + + +### DATA AND VERIFIER CONTEXTS + +Passing data required to perform remote attestation might not be always possible. For example, to perform the verification of the SKAE extension in a certificate, an application might pass the skae_callback() function as an argument to SSL_CTX_set_verify(). OpenSSL defines the callback function, and new parameters cannot be introduced. + +To overcome this issue, the data and verifier contexts concept has been defined. A data context is a data structure that contains a linked list of data pointers and length for each type of defined information (for example a Privacy CA certificate, or the public part of an AK). The reader can have a look to include/ctx.h for more details. + +A verifier context contains the list of requirements provided by a remote attestation verifier (these requirements are passed to the verifier's plugins mentioned above). It also contains a log for each verification step executed and its status. If a verification step failed, the first log contains the reason for the failure, while previous logs (created by the callers of the failed function) have as reason ' failed'. + +For the data context, the base library provides functions to add binary data from buffers and files. It also provides functions to import/print JSON strings. Support for other data formats might be provided. + +For the verifier context, the base library provides functions to set verifier requirements, to set the mask of PCRs to check, and to print the logs with the list and result of the executed verification steps. + +An application might pass a NULL pointer as data or verifier context. In this case, a global context (defined in the library) is used. This is the only way to use the skae_callback() for OpenSSL, as it is not possible to add a data and verifier context as parameters. + + + +## USE CASES + +This section provides an overview of how to attest tools can accomplish the most common tasks. The reader can modify these examples to implement his scenario. + + +### Create an AK and request a certificate: + +#### Preliminary Steps (on the server) +1) use existing CA or generate a new custom CA by executing: +``` +$ generate_demoCA demoCA +``` +Update /etc/ssl/openssl.cnf, with: +``` +[ CA_default ] +dir = +... +unique_subject = no +... +copy_extensions = copy +... +input_password = +``` +2) configure a TPM (a software TPM is sufficient) +3) install openssl_tpm2_engine + +#### Preliminary Steps (on the client) +The following steps assume that a software TPM is provisioned by libvirt for the VM. + +1) Copy swtpm certificates from /var/lib/swtpm-localca in the host to the VM: +``` +cp issuercert.pem swtpm-localca-rootca-cert.pem /etc/attest-tools/ek_ca_certs +``` + +#### Steps (on the server) +1) execute: +``` +$ attest_ra_server -r /etc/attest-tools/req_examples/req-dummy.json +``` + +#### Steps (on the client) +1) execute: +``` +$ attest_ra_client -a -s +``` + +### Create a TPM key not bound to any PCR, save attestation data to attest.txt, and request a certificate: + +#### Steps (on the client) +1) execute: +``` +$ attest_ra_client -k -s -r attest.txt +``` + +### Perform implicit RA: + +#### Steps (on the server) +1) execute: +``` +$ attest_tls_server -e -k /etc/attest-tools/tls_key.pem \ + -c /etc/attest-tools/tls_key_cert.pem \ + -d /etc/attest-tools/tls_key_ca_cert.pem -a attest.txt +``` + +#### Steps (on the client) +1) execute: +``` +$ attest_tls_client -S -V -s \ + -d /etc/attest-tools/tls_key_ca_cert.pem \ + -r /etc/attest-tools/req_examples/req-dummy.json +``` + +### Create a TPM key bound to PCRs 0-9,10 and request a certificate: + +#### Preliminary Steps (on the client) +1) ensure that the client has a BIOS event log accessible from /sys/kernel/security/tpm0/binary_bios_measurements +2) ensure that the client has an IMA event log accessible from /sys/kernel/security/ima/binary_runtime_measurements and that no IMA policy is loaded + +#### Steps (on the server) +1) execute: +``` +$ attest_ra_server -r /etc/attest-tools/req_examples/req-bios-ima.json \ + -p 0,1,2,3,4,5,6,7,8,9,10 +``` + +#### Steps (on the client) +1) run: +``` +$ attest_ra_client -k -s -r attest.txt -b -i \ + -p 0,1,2,3,4,5,6,7,8,9,10 +``` + +### Perform implicit RA: + +#### Steps (on the server) +1) execute: + +``` +$ attest_tls_server -e -k /etc/attest-tools/tls_key.pem \ + -c /etc/attest-tools/tls_key_cert.pem \ + -d /etc/attest-tools/tls_key_ca_cert.pem -a attest.txt +``` + +#### Steps (on the client) +1) execute: +``` +$ attest_tls_client -S -V -s + -d /etc/attest-tools/tls_key_ca_cert.pem \ + -r /etc/attest-tools/req_examples/req-bios-ima.json \ + -p 0,1,2,3,4,5,6,7,8,9,10 +``` + +### Perform explicit RA: + +#### Steps (on the client) +1) run: +``` +$ attest_ra_client -q -s -b -i \ + -p 0,1,2,3,4,5,6,7,8,9,10 +``` + +### Update PCR and perform again explicit RA: + +#### Steps (on the client) +1) run: +``` +$ tsspcrextend -halg sha1 -ha 10 -ic "test" +$ attest_ra_client -q -s -b -i \ + -p 0,1,2,3,4,5,6,7,8,9,10 +``` + +This time RA should fail. diff --git a/translation-projects/README-optimization/optimized-README/auto_py2to3-en.rst b/translation-projects/README-optimization/optimized-README/auto_py2to3-en.rst new file mode 100644 index 0000000..e454afb --- /dev/null +++ b/translation-projects/README-optimization/optimized-README/auto_py2to3-en.rst @@ -0,0 +1,81 @@ +========== +auto_py2to3 +========== + +On January 1, 2020, the Python 2 codebase was frozen. From that day on, there was no further porting of Python 2, which made the language and runtime environment obsolete. +Core developer Nick Coghlan explained in the FAQ, which ended "the core development team maintained Python 2 and 3 as reference interpreters for about 13 years." +The final version of Python 2 is currently in the beta testing and release candidate phase, and the final production version python 2.7.18 is expected to be released in April 2020. +Although most people in the Python community believe that Python needs urgent changes-especially since the much-needed Unicode support already exists. +But many people are frustrated that Python 2 code works well. Therefore, code migration is needed, and the ultimate goal of the library is to realize automated and fast code migration and provide automated testing. +In this process, although there may be some unsatisfactory aspects in the code migration process, the library will continue to be iterated and maintained. + + +Features +------------ +* Testing setup with unit test and python setup.py test or pytest +* Command-line interface using Click + +Architecture +------------ + +* Relying on the official 2to3 as a technical tool, encapsulating executable files for later use. +* Provide multiple functions through the command line to process project code. + +Version Support +------------ +* Python 2.x to 3.x + +* Linux: build/passing +* Windows: build/passing + +Functions +------------ +The main functions are as follows: + +1. Determine whether to enable automatic conversion of Python2 code to Python3 +2. Determine whether to keep the Python2 code backup +3. Determine whether to open the conversion code text comparison +4. Determine whether the version of the library that the project depends on is suitable for the current Python environment. + + +Usage Example +------------ +.. image:: https://gitee.com/weihaitong/auto_py2to3/raw/master/example/ticketGrabbingExample-commadTest%20processing.png + +Get Started! +------------ +Ready to contribute or use? Here's how to set up `auto_py2to3` for local development. + +1. Fork the `auto_py2to3` repo on Gitee. +2. Clone your fork locally:: + + $ git clone https://gitee.com/weihaitong/auto_py2to3.git + +3. This is how you use this tool:: + + $ cd auto_py2to3/ + $ python setup.py install + $ py2to3 --help + +4. If you want to contribute to this project, you can create a branch for local development:: + + $ git checkout -b name-of-your-bugfix-or-feature + + Now you can make your changes locally. + +5. Commit your changes and push your branch to Gitee:: + + $ git add . + $ git commit -m "Your detailed description of your changes." + $ git push origin name-of-your-bugfix-or-feature + +6. Submit a pull request through the Gitee website. + +Pull Request Guidelines +----------------------- + +Before you submit a pull request, check that it meets these guidelines: + +1. The pull request should include tests. +2. If the pull request adds functionality, the docs should be updated. Put your new functionality into a function with a docstring, and add the feature to the list in README.rst. +3. The pull request should work for Python 3.x. and make sure that the tests pass for all supported Python versions. diff --git a/translation-projects/README-optimization/optimized-README/avocado-vt-en.rst b/translation-projects/README-optimization/optimized-README/avocado-vt-en.rst new file mode 100644 index 0000000..1801cd4 --- /dev/null +++ b/translation-projects/README-optimization/optimized-README/avocado-vt-en.rst @@ -0,0 +1,23 @@ +Avocado VT Plugin +================= + +Avocado-VT is a compatibility plugin that lets you execute virtualization-related tests (then known as virt-test), with all conveniences provided by Avocado. + +Gettings started with Avocado +============================= + +First of all, make sure you have installed Avocado. + +You can check the Avocado online documentation at: + +https://avocado-framework.readthedocs.io/en/latest/guides/user/chapters/installing.html + +Getting started with Avocado-VT +=============================== + +The recommended way of installing Avocado-VT is through RPM packages. + +Documentation on how to install it can be found at: + +http://avocado-vt.readthedocs.org/en/latest/GetStartedGuide.html + diff --git a/translation-projects/README-optimization/optimized-README/bishengjdk-11-en.md b/translation-projects/README-optimization/optimized-README/bishengjdk-11-en.md new file mode 100644 index 0000000..28abb59 --- /dev/null +++ b/translation-projects/README-optimization/optimized-README/bishengjdk-11-en.md @@ -0,0 +1,27 @@ +## Bisheng JDK 11 + +As the in-house OpenJDK customized version at Huawei, Bisheng JDK runs on more than 500 products within Huawei, has accumulated a large number of usage scenarios and java developers' problems and demands, solved many problems encountered in the actual operation of the business. And typically optimized on the ARM architecture, Bisheng JDK is suited for big data applications with high performance. Bison JDK 11 currently supports Linux/AArch64 and Linux/x86_64 platforms. As a "friendly fork" project under the same license as the upstream OpenJDK project, Bisheng JDK will continue to contribute to the OpenJDK community all along. + +The latest binary builds can be downloaded from [releases](https://www.hikunpeng.com/en/developer/devkit/compiler?data=JDK). + +The User Guide && Features are available, please see [wiki](https://gitee.com/openeuler/bishengjdk-11/wikis/Home?sort_id=2879167). + +## Platform Support + +Bisheng JDK currently supports `Linux/AArch64, Linux/x86_64`. + +## Build Instructions + +About the build instructions for Bisheng JDK, please see [wiki](https://gitee.com/openeuler/bishengjdk-11/wikis/Bisheng%20JDK%2011%20Source%20Code%20Building%20Description?sort_id=3919529). + +## License + +Bisheng JDK is licensed under GPLv2 with Classpath Exception. For the details, see [License](https://gitee.com/openeuler/bishengjdk-11/blob/master/LICENSE). + +## Installation + +See the [Installation Guide](https://gitee.com/openeuler/bishengjdk-11/wikis/Bisheng%20JDK%2011%20Installation%20Guide?sort_id=2879162). + +## How to contribute + +Contributions are welcomed. Please [send an email to dev](https://openeuler.org/zh/community/mailing-list) or [create a new issue](https://gitee.com/openeuler/bishengjdk-11/issues) if you find a bug or have new ideas. diff --git a/translation-projects/README-optimization/optimized-README/bishengjdk-8-en.md b/translation-projects/README-optimization/optimized-README/bishengjdk-8-en.md new file mode 100644 index 0000000..9c5ffac --- /dev/null +++ b/translation-projects/README-optimization/optimized-README/bishengjdk-8-en.md @@ -0,0 +1,27 @@ +## Bisheng JDK 8 + +As the in-house OpenJDK customized version at Huawei, Bisheng JDK runs on more than 500 products within Huawei, has accumulated a large number of usage scenarios and java developers' problems and demands, solved many problems encountered in the actual operation of the business. And typically optimized on the ARM architecture, Bisheng JDK is suited for big data applications with high performance. Bisheng JDK 8 is compatible with the Java SE standard and currently supports Linux/AArch64 and Linux/x86_64 platforms. As a "friendly fork" project under the same license as the upstream OpenJDK project, Bisheng JDK will continue to contribute to the OpenJDK community all along. + +The latest binary builds can be downloaded from [releases](https://www.hikunpeng.com/en/developer/devkit/compiler?data=JDK). + +The User Guide && Features are available, please see [wiki](https://gitee.com/openeuler/bishengjdk-11/wikis/Home?sort_id=2879167). + +## Platform Support + +Bisheng JDK currently supports `Linux/AArch64, Linux/x86_64`. + +## Build Instructions + +About the build instructions for Bisheng JDK, please see [wiki](https://gitee.com/openeuler/bishengjdk-11/wikis/Bisheng%20JDK%2011%20Source%20Code%20Building%20Description?sort_id=3919529). + +## License + +Bisheng JDK is licensed under GPLv2 with Classpath Exception. For the details, see [License](https://gitee.com/openeuler/bishengjdk-11/blob/master/LICENSE). + +## Installation + +See the [Installation Guide](https://gitee.com/openeuler/bishengjdk-11/wikis/Bisheng%20JDK%2011%20Installation%20Guide?sort_id=2879162). + +## How to contribute + +Contributions are welcomed. Please [send an email to dev](https://openeuler.org/zh/community/mailing-list) or [create a new issue](https://gitee.com/openeuler/bishengjdk-11/issues) if you find a bug or have new ideas. diff --git a/translation-projects/README-optimization/optimized-README/bishengjdk-riscv-en.md b/translation-projects/README-optimization/optimized-README/bishengjdk-riscv-en.md new file mode 100644 index 0000000..0af1cff --- /dev/null +++ b/translation-projects/README-optimization/optimized-README/bishengjdk-riscv-en.md @@ -0,0 +1,9 @@ +# Welcome to the JDK! + +For build instructions, please read the [online documentation](https://openjdk.java.net/groups/build/doc/building.html) or either of these files: + +- [doc/building.html](doc/building.html) (html version) +- [doc/building.md](doc/building.md) (markdown version) + +Visit for more information about the OpenJDK Community and the JDK. + diff --git a/translation-projects/README-optimization/optimized-README/ci-bot-en.md b/translation-projects/README-optimization/optimized-README/ci-bot-en.md new file mode 100644 index 0000000..4b563b3 --- /dev/null +++ b/translation-projects/README-optimization/optimized-README/ci-bot-en.md @@ -0,0 +1,63 @@ +# ci-bot + +## Introduction + +This repository is used to address the code of the openEuler ci bot. + +## Architecture + + + +## Prerequisites + +You'll need to set up a MySQL Database before you are getting started. +Here is an example to create a Database instance: + +* Setup MySQL instance by the Huawei Cloud Relational Database Service (RDS) +* Login in MySQL with your account and password +* Create database instance by running the following command: + ``` + CREATE DATABASE cibot; + ``` + The information of the database instance will be used in the following Installation. + +## Config +### environment variables +Some sensitive configuration items support reading from environment variables. +You can set the following environment variables in your OS: + +* GITEE_TOKEN +* WEBHOOK_SECRET +* DATABASE_HOST +* DATABASE_PORT +* DATABASE_USERNAME +* DATABASE_PASSWORD +### label config +If you want to clear some tags when the pull request source branch changes, you can configure it in the configuration file(config.yaml). +For example, see config.yaml delLabels fields.Description: + + * kind,sig,openeuler-cla,priority Delete labels beginning with kind,sig,openeuler-cla or priority. + * lgtm Delete labels lgtm or beginning with lgtm-. + * Except for the above description items, other labels will be judged as equal. +### extraLgtmCountRequired config +If you want to set the number of lgtm tags for a separate repository or organization, you can configure this configuration item. The configuration item is a list, and the list element contains the following configuration items: + + * lcrType Indicates whether the configuration is for the repository or the organization + * lcrName Configure the spatial address of the repository or organization + * lcrCount Number of lgtm tags + +**If your configuration is for a specific repository, you should configure the full path instead of just the repository space address. For example: lcrName:openEuler/ci-bot** + +## Getting Started + +* [Getting Started on Locally](deploy/locally/README.md) +* [Getting Started on CCE](deploy/cce/README.md) + +## Command Help + +See the [Command Help](https://gitee.com/openeuler/community/blob/master/en/sig-infrastructure/command.md) file for details. +> For the Chinese version, please refer to [命令帮助](https://gitee.com/openeuler/community/blob/master/zh/sig-infrastructure/command.md). + +## License + +See the [LICENSE](LICENSE) file for details. diff --git a/translation-projects/README-optimization/optimized-README/clibcni-en.md b/translation-projects/README-optimization/optimized-README/clibcni-en.md new file mode 100644 index 0000000..67cd549 --- /dev/null +++ b/translation-projects/README-optimization/optimized-README/clibcni-en.md @@ -0,0 +1,16 @@ +# clibcni + +CNI (Container Network Interface) is a Cloud Native Computing Foundation project. +clibcni is a library used by [iSulad](https://gitee.com/openeuler/iSulad) to configure network interfaces in containers, following +the specification of CNI. + +## How to Contribute + +We always welcome new contributors. And we are happy to provide guidance for the new contributors. +iSulad follows the kernel coding conventions. You can find a detailed introduction at: + +- https://www.kernel.org/doc/html/v4.10/process/coding-style.html + +## Licensing + +clibcni is licensed under the Mulan PSL v2. diff --git a/translation-projects/README-optimization/optimized-README/cloudnative-en.md b/translation-projects/README-optimization/optimized-README/cloudnative-en.md new file mode 100644 index 0000000..b081719 --- /dev/null +++ b/translation-projects/README-optimization/optimized-README/cloudnative-en.md @@ -0,0 +1,156 @@ +# CloudNative SIG + +## Mission and Vision +CloudNative SIG is committed to providing a convenient and easy-to-use cloud-native infrastructure for users, developers, and enthusiasts, providing a simple and efficient cloud-native application development hosting environment to build cloud-native ecology. + +## Work Objectives and Scope + +- Responsible for community cloud-native technology decisions +- Responsible for the planning, maintenance, and upgrade of cloud-native-related packages +- Respond to user feedback from software within the SIG group promptly to resolve related issues + +## SIG Documents +- [Documents](docs/README.md) + +# SIG Meetings +- Every Tuesday from 4:00-5:30 p.m +- Declare the issue by mail + + +# Members + +### Maintainers + +| [@caihaomin](https://gitee.com/caihaomin) | [@yangzhao_kl](https://gitee.com/yangzhao_kl) | [@jianminw](https://gitee.com/jianminw) | [@weibaohui](https://gitee.com/weibaohui) | [@biannm](https://gitee.com/biannm) | [@pixiake](https://gitee.com/pixiake) | [@duguhaotian](https://gitee.com/duguhaotian) | +| ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | +| 图片名称 | @yangzhao_kl | @jianminw | @weibaohui | @biannm | @pixiake | @duguhaotian | + +### Committers + +| **[@jingxiaolu](https://gitee.com/jingxiaolu)** | [@leizhongkai](https://gitee.com/leizhongkai) | [@jing-rui](https://gitee.com/jing-rui) | [@flyflyflypeng](https://gitee.com/flyflyflypeng) | [@lifeng2221dd1](https://gitee.com/lifeng2221dd1) | +| ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | +| @jingxiaolu | @leizhongkai | @jing-rui | @flyflyflypeng | @lifeng2221dd1 | + + +## Contacts + +- Maillist: cloudnative@openeuler.org + + +## Repositories + +### Orchestration & Management + +| Project | Version | State | Category | +| -------------------------------------------------------- | ------- | ---------- | -------------------------------- | +| [kubernetes](https://gitee.com/src-openeuler/kubernetes) | 1.20.2 | Introduced | Scheduling & Orchestration | +| [etcd](https://gitee.com/src-openeuler/etcd) | 3.4.14 | Introduced | Coordination & Service Discovery | +| [coredns](https://gitee.com/src-openeuler/coredns) | 1.7.0 | Introduced | Coordination & Service Discovery | + + +### Container Runtime + +| Project | Version | State | Category | +| ------------------------------------------------------------ | ------------ | ---------- | ---------------- | +| [iSulad](https://gitee.com/src-openeuler/iSulad) | 2.0.9 | Introduced | Container Engine | +| [lxc](https://gitee.com/src-openeuler/lxc) | 4.0.3 | Introduced | Runtime | +| [lcr](https://gitee.com/src-openeuler/lcr) | 2.0.5 | Introduced | Runtime | +| [clibcni](https://gitee.com/src-openeuler/clibcni) | 2.0.4 | Introduced | Networking | +| [docker](https://gitee.com/src-openeuler/docker) | 18.09.0 | Introduced | Container Engine | +| [containerd](https://gitee.com/src-openeuler/containerd) | 1.2.0 | Introduced | Runtime | +| [runc](https://gitee.com/src-openeuler/runc) | 1.0.0-rc3 | Introduced | Runtime | +| [kata-containers](https://gitee.com/src-openeuler/kata-containers) | 2.1.1/1.11.1 | Introduced | Runtime | +| [libnetwork](https://gitee.com/src-openeuler/libnetwork) | 0.8.0-dev | Introduced | Networking | +| [containernetworking-plugins](https://gitee.com/src-openeuler/containernetworking-plugins) | 0.8.6 | Introduced | Networking | +| [cri-o](https://gitee.com/src-openeuler/cri-o) | | Created | Runtime | +| [cri-tools](https://gitee.com/src-openeuler/cri-tools) | | Created | Container Engine | +| [gvisor](https://gitee.com/src-openeuler/gvisor) | | Created | Runtime | +| [crun](https://gitee.com/src-openeuler/crun) | | Created | Runtime | +| [catatonit](https://gitee.com/src-openeuler/catatonit) | | Created | Runtime | + + +### App Definition & Development + +| Project | Version | State | Category | +| ------------------------------------------------------------ | ------- | ---------- | ------------------------------------ | +| [isula-build](https://gitee.com/src-openeuler/isula-build) | 0.9.5 | Introduced | Application Definition & Image Build | +| [operator-manager](https://gitee.com/openeuler/operator-manager) | 2 | Incubating | Application Definition & Image Build | +| [docker-compose](https://gitee.com/src-openeuler/docker-compose) | 1.22.0 | Introduced | Application Definition & Image Build | +| [podman](https://gitee.com/src-openeuler/podman) | 1.0.10 | Introduced | Application Definition & Image Build | +| [skopeo](https://gitee.com/src-openeuler/skopeo) | 1.1.0 | Introduced | Application Definition & Image Build | +| [kubevirt](https://gitee.com/src-openeuler/kubevirt) | | Created | Application Definition & Image Build | + + +### Observability & Analysis + +| Project | Version | State | Category | +| ------------------------------------------------------------ | ------- | ---------- | ---------- | +| [prometheus](https://gitee.com/src-openeuler/prometheus) | 2.20.0 | Introduced | Monitoring | +| [promu](https://gitee.com/src-openeuler/promu) | 0.7.0 | Incubating | Monitoring | +| [cadvisor](https://gitee.com/src-openeuler/cadvisor) | 0.37.0 | Incubating | Monitoring | +| [alertmanager](https://gitee.com/src-openeuler/alertmanager) | 0.21.0 | Incubating | Monitoring | +| [node_exporter](https://gitee.com/src-openeuler/node_exporter) | 1.0.1 | Introduced | Monitoring | +| [virt-what](https://gitee.com/src-openeuler/virt-what) | 1.2 | Introduced | Monitoring | +| [gluster_exporter](https://gitee.com/src-openeuler/gluster_exporter) | 0.2.7 | Incubating | Monitoring | +| [haproxy_exporter](https://gitee.com/src-openeuler/haproxy_exporter) | | Created | Monitoring | +| [influxdb_exporter](https://gitee.com/src-openeuler/influxdb_exporter) | | Created | Monitoring | +| [memcached_exporter](https://gitee.com/src-openeuler/memcached_exporter) | | Created | Monitoring | +| [pushgateway](https://gitee.com/src-openeuler/pushgateway) | 1.4.1 | Incubating | Monitoring | + + +### Installation & Deployment + +| Project | Version | State | Category | +| ---------------------------------------- | ------------ | ---------- | ------------------------------------ | +| [eggo](https://gitee.com/openeuler/eggo) | v1.0.0-alpha | Incubating | Kubernetes deployment by one command | + + +### Provisioning + +| Project | Version | State | Category | +| ------------------------------------------------------------ | ------- | ------- | -------------------------- | +| [harbor](https://gitee.com/src-openeuler/harbor) | | Created | Container Registry | +| [containers-common](https://gitee.com/src-openeuler/containers-common) | | Created | Automation & Configuration | + +### Container OS + +| Project | Version | State | Remarks | +| -------------------------------------------- | ------- | ---------- | -------------------------------------- | +| [NestOS](https://gitee.com/openeuler/NestOS) | | Incubating | A cloudify OS drivied from CoreOS | +| [KubeOS](https://gitee.com/openeuler/KubeOS) | | Incubating | A Kubernetes OS simplifies OS updating | + + +### Dependent Component + +| Project | Version | State | Remarks | +| ------------------------------------------------------------ | ------- | ---------- | -------------------------------- | +| [protobuf](https://gitee.com/src-openeuler/protobuf) | 3.14.0 | Introduced | Dependent Component | +| [iSulad-img](https://gitee.com/openeuler/iSulad-img) | 2.0.1 | Introduced | iSulad 1.x Image Management Tool | +| [busybox](https://gitee.com/src-openeuler/busybox) | 1.31.1 | Introduced | Dependent Component | +| [kata_integration](https://gitee.com/src-openeuler/kata_integration) | 1.0.0 | Introduced | Kata Build Project | +| [docker-client-java](https://gitee.com/src-openeuler/docker-client-java) | 8.11.7 | Introduced | Dependent Component | +| [container-selinux](https://gitee.com/src-openeuler/container-selinux) | 2.138 | Introduced | Docker Selinux Policy | +| [jboss-parent](https://gitee.com/src-openeuler/jboss-parent) | 20-1 | Introduced | Dependent Component | +| [libevhtp](https://gitee.com/src-openeuler/libevhtp) | 1.2.18 | Introduced | Dependent Component | +| [libcgroup](https://gitee.com/src-openeuler/libcgroup) | 0.42.2 | Introduced | Dependent Component | + + +### Note + +| State | Remarks | +| ----------- | ------------------------------------------------------------ | +| Introduced | The component has been introduced into the openEuler distribution. | +| Introducing | The project has been created in openEuler and the code is being introduced from upstream repository, and being introduced to openEuler distribution. | +| Incubating | The project is incubating by CloudNative SIG and under heavy developing. Not introduced to openEuler release yet. | +| Created | The project is created in openEuler but no any code is introduced from upstream repostitory, which is help wanted. | +| Requested | The project is requested by not introduced yet in the openEuler. | + + +### Rodemap + +- 2021.01: The SIG is established +- 2021.06.30 (20.03 SP2): Automatic Installation Set; Container Image Repository; Lightweight Virtualization Stratovirt; iSulad shimV2 +- 2021.09.30 (21.09 Innovation): OLM/Operator/helm Hub; Container OS; Confidential Container; Application Sandbox WASM; High-performance Container Storage +- 2021.12.30 (20.03 SP2): isula-build supports the Operator Framework/SDK; Volcano scheduling algorithm; Stratovirt Standard VM +- 2022.3.30 (22.03 LTS): Cloudnative Application Market; Cloudnative Application Development Suite; A-OPS servitization; openDonau ; High-performance Data Plane Framework; Ebpf Network Acceleration +- ... diff --git a/translation-projects/README-optimization/optimized-README/community-en-opt.md b/translation-projects/README-optimization/optimized-README/community-en-opt.md new file mode 100644 index 0000000..e6c6c0a --- /dev/null +++ b/translation-projects/README-optimization/optimized-README/community-en-opt.md @@ -0,0 +1,57 @@ +# openEuler Community +English | [简体中文](./README_cn.md) + +Welcome to openEuler Community. + + +## Introduction + +The Community repo stores all the information about openEuler Community, including governance, how to contribute, SIGs(project teams), Communications, etc. + +## What you can find here + +- [Introduction to openEuler Community](/en/governance) +- Governance + - [Security Committee](/en/security-committee) + - [Technical Committee](/en/technical-committee) + - [SIGs](https://www.openeuler.org/en/sig/sig-list/) +- [Live](https://www.openeuler.org/en/interaction/live-list/)&[Meetups](https://www.openeuler.org/en/interaction/salon-list/)&[Summit](https://www.openeuler.org/en/interaction/summit-list/) +- [Contribution Guideline](https://www.openeuler.org/en/community/contribution/) + +## How to contribute + +When the openEuler community is updated, the information here should be updated as well. If you would like to help update the information in this repo, you are very appreciated. + +Please read [How to contribute](CONTRIBUTING.md) to get detailed guidance. + +## Maintainers + +### [TC members](/en/technical-committee) have the rights over the following folders + ./repository + ./sig + +### The Blog team have the rights over [openEuler Blog](https://gitee.com/openeuler/website-v2/tree/master/web-ui/docs/en/blog) +1. Xing Zhu (@shredstar), since February, 2020 +2. Perl(@pearl405), since February, 2020 +3. ZhengyuhangHans(@ZhengyuhangHans), since February 2020 + +### The rest of the folders under this repo +1. Fred Li(@zerodefect), since December 2019 +2. 杨丽(@yangli69393), since December 2019 +3. blueskycs2c(@blueskycs2c), since March 2020 + +## Contributors +1. freesky-edward(@freesky-edward), since August 2019 +2. xiangxinyong(@xiangxinyong), since August 2019 +3. blueskycs2c(@blueskycs2c), since February 2020 +4. Ivye(@Ivye), since March, 2020 + +## Contact + +Mail: community@openeuler.org + +IRC: #openeuler-community + +## Meeting + +Find the meeting schedule [here](/meeting_records/README.md). diff --git a/translation-projects/README-optimization/optimized-README/community-issue-en.md b/translation-projects/README-optimization/optimized-README/community-issue-en.md new file mode 100644 index 0000000..c943f3e --- /dev/null +++ b/translation-projects/README-optimization/optimized-README/community-issue-en.md @@ -0,0 +1,24 @@ +# openEuler Issues Repository + +English | [简体中文](./README_cn.md) + +### Introduction + +As openEuler includes many projects, and it is not always easy for you to identify the project to submit issues. + +This repository is to accept such issues. + + +### Use Scenarios + +When you find an issue but you are not sure which repository to submit to, please feel free to submit it here. + +If you find the issue opened in this repository should be traced or fixed in another repository. Please open another issue in that repository, comment on the new one and ask to close here. + +### Maintainers + + - freesky-edward + +### Warning + +Please **don't** open any Pull Requests in this repository. diff --git a/translation-projects/README-optimization/optimized-README/compass-ci-en.md b/translation-projects/README-optimization/optimized-README/compass-ci-en.md new file mode 100644 index 0000000..208acbb --- /dev/null +++ b/translation-projects/README-optimization/optimized-README/compass-ci-en.md @@ -0,0 +1,213 @@ +## Compass-CI + +### Introduction + +Compass-CI is an open-source software platform supporting continuous integration. It provides developers with the test, login, assistant fault demarcation, and historical data-based analysis services for upstream open-source software (from Github, Gitee, GitLab, and other hosting platforms). Compass-CI performs automatic tests (including the build tests and the use case tests included in software packages) based on the open-source software PR to build an open and complete test system. + +### Features + +**Test Service** + +Compass-CI monitors Git repositories of much open-source software. Once a code update is detected, Compass-CI automatically triggers the [automated test](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/features/test-service/test-oss-project.zh.md). Developers can also [manually submit for tests](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/job/submit/submit-job.zh.md). + +**Logging In to the Commissioning Environment** + +Using SSH to [log in to the test environment for commissioning](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/features/login-to-testbox/log-in-machine-debug.md) + +**Test Result Analysis** + +Analyze and compare historical test results through the [Web](https://compass-ci.openeuler.org/) interface. + +**Test Result Reproduction** + +All deterministic parameters for test running are recorded in the job.yaml file. Submit the job.yaml file again to run the same test in the same software and hardware environments. + +**Error Locating** + +If a new error ID is generated, the bisect is automatically triggered to locate the commit that introduces the error ID. + +## Getting Started + +**Automated Test** + +1. Add the URL of the repository to be tested to the [upstream-repos](https://gitee.com/wu_fengguang/upstream-repos.git) repository. [Compiling test cases](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add-testcase.md) and adding the URL to the [lkp-tests](https://gitee.com/wu_fengguang/lkp-tests) repository. For details, see [this document](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/features/test-service/test-oss-project.zh.md). +2. Run the git push command to update the repository. The test is automatically triggered. +3. On the web page, click [view](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/result/browse-results.zh.md) and [compare](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/result/compare-results.zh.md) to view the test result. (web: ) + +**Automatic Test Example** + +How can I automatically test my repository on Compass-CI? + +1. Fork upstream-repos repository ([https://gitee.com/wu_fengguang/upstream-repos) ](https://gitee.com/wu_fengguang/upstream-repos %EF %BC %89) and git clone it to the localhost. + +2. Create the **b/backlight/backlight** file. The file content is as follows: + + ``` + --- + url: + - https://github.com/baskerville/backlight + ``` + +3. Add test case + + You can compile test cases and add them to the **lkp-tests** repository. + + You can also use the existing test cases in the jobs directory of the lkp-tests repository (). + + Add the **DEFAULTS** file to the directory where the backlight file is stored and add the configuration information. + + ``` + submit: + - command: testbox=vm-2p16g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 api-avx2neon.yaml + branches: + - master + - next + - command: testbox=vm-2p16g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 other-avx2neon.yaml + branches: + - branch_name_a + - branch_name_b + ``` + +4. Submit a PR to add the new file to the upstream-repos repository. + +**Manually Submitting a Test Task** + +1. [Install the compass-ci client.](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/install/install-cci-client.md) +2. [Compile test cases](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add-testcase.md) [to manually submit a test task.](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/job/submit/submit-job.zh.md) +3. On the [web page](https://compass-ci.openeuler.org/jobs), click [view](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/result/browse-results.zh.md) and [compare](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/result/compare-results.zh.md) to view the test result. + +**Example** + +1. The Compass-CI client has been installed following the procedure in [Installing‧the‧Compass-ci‧Client‧on‧the‧Local‧PC]([doc/install/install-cci-client.md · Fengguang/compass-ci - Gitee.com](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/install/install-cci-client.md) ). + +2. Submit the test in a YAML file. You need to prepare the YAML file of the test task in advance. + + You can directly use the existing test cases in the **jobs** directory of the **lkp-tests** repository (). + + The following uses **iperf.yaml** as an example: + + ```yaml + suite: iperf + category: benchmark + + runtime: 300s + + cluster: cs-localhost + + if role server: + iperf-server: + + if role client: + iperf: + protocol: + - tcp + - udp + ``` + +3. Run the submit command to submit the **iperf.yaml** test task. + + ```shell + hi8109@account-vm ~% submit iperf.yaml testbox=vm-2p8g + submit iperf.yaml, got job_id=z9.173924 + submit iperf.yaml, got job_id=z9.173925 + ``` + +**Logging in to the Test Environment** + +1. Send an email to [compass-ci-robot@qq.com](mailto:compass-ci-robot@qq.com) to [apply for an account](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/account/apply-account.md). +2. Complete the environment configuration based on the email feedback. +3. Add the sshd field to the test task and submit the corresponding task. [Log in to the test environment](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/features/login-to-testbox/log-in-machine-debug.md). + +**Example** + +The test case is **spinlock.yaml**. If a submitted test case fails to be executed, how do I log in to the executor to perform commissioning? + +```yaml +suite: spinlock +category: benchmark +nr_threads: +- 1 +spinlock: +``` + +1. To log in to the executor before running the spinlock test script, modify the YAML file as follows: + + ```yaml + suite: spinlock + category: benchmark + nr_threads: + - 1 + + ssh_pub_key: <%= File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp rescue nil %> + sshd: + runtime: 1h + sleep: + + spinlock: + ``` + + **ssh_pub_key**: Carries the local pub_key for password-free login. + + **sshd**: Indicates that the executor needs to run the lkp-tests/damon/sshd script to establish an SSHR reverse tunnel for SSH login. + + **runtime**: sleep time + + **sleep**: Placed before spinlock, indicating that the spinlock script is executed one hour after sleep. + +2. To log in to the executor after the spinlock test fails, modify the YAML file as follows: + + ```yaml + suite: spinlock + category: benchmark + nr_threads: + - 1 + spinlock: + + on_fail: + sshd: + sleep: 1h + ``` + + **on_fail**: The test case is executed after the test case fails to be executed. + +3. Run the **submit -m -c spinlock.yaml** command to submit the modified YAML file. + + After the SSHD tunnel is established, the PC automatically connects to the executor. + + ```shell + hi8109@account-vm ~% submit -m -c spinlock.yaml + submit_id=6f2d11df-2198-41e9-a0e6-6aa67f9b46e2 + submit spinlock.yaml, got job id=z9.10155176 + query=>{"job_id":["z9.10155176"]} + connect to ws://api.compass-ci.openeuler.org:20001/filter + {"level_num":2,"level":"INFO","time":"2021-09-17T17:21:03.436+0800","from":"172.17.0.1:40014","message":"access_record","status_code":200,"method":"GET","resource":"/job_initrd_tmpfs/z9.10155176/job.cgz","job_id":"z9.10155176","job_state":"download","api":"job_initrd_tmpfs","elapsed_time":0.465723,"elapsed":"465.72µs"} + + The dc-8g testbox is starting. Please wait about 30 seconds + {"level_num":2,"level":"INFO","time":"2021-09-17T17:21:08+0800","mac":"02-42-ac-11-00-03","ip":"","job_id":"z9.10155176","state":"running","testbox":"dc-8g.taishan200-2280-2s48p-256g--a67-14","status_code":200,"method":"GET","resource":"/~lkp/cgi-bin/lkp-wtmp?tbox_name=dc-8g.taishan200-2280-2s48p-256g--a67-14&tbox_state=running&mac=02-42-ac-11-00-03&ip=&job_id=z9.10155176","api":"lkp-wtmp","elapsed_time":19.024787,"elapsed":"19.02ms"} + {"level_num":2,"level":"INFO","time":"2021-09-17T17:21:12.622+0800","from":"172.17.0.1:42838","message":"access_record","status_code":200,"method":"GET","resource":"/~lkp/cgi-bin/lkp-jobfile-append-var?job_file=/lkp/scheduled/job.yaml&job_id=z9.10155176&job_state=running","job_id":"z9.10155176","api":"lkp-jobfile-append-var","elapsed_time":74.76464,"elapsed":"74.76ms","job_state":"running","job_stage":"running"} + {"level_num":2,"level":"INFO","time":"2021-09-17T17:21:12.982+0800","tbox_name":"dc-8g.taishan200-2280-2s48p-256g--a67-14","job_id":"z9.10155176","ssh_port":"21063","message":"","state":"set ssh port","status_code":200,"method":"POST","resource":"/~lkp/cgi-bin/report_ssh_info","api":"report_ssh_info","elapsed_time":0.414042,"elapsed":"414.04µs"} + ssh root@172.168.131.2 -p 21063 -o StrictHostKeyChecking=no -o LogLevel=error + root@dc-8g.compass-ci.net ~# + ``` + +## Contributing to Compass-CI + +We welcome new contributors, and we are happy to guide our contributors. Compass-CI is mainly a project developed using Ruby, and we follow the [Ruby Community Code Style](https://ruby-china.org/wiki/coding-style). If you want to participate in the community and contribute to the Compass-CI project, [this page](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/help/learning-resources.md) will provide you with more information, including all languages and tools used by Compass-CI. + +## Website + +All test results have been added to the open-source software list of the Compass-CI platform. Historical test results can be found on the [official website](https://compass-ci.openeuler.org/). + +## Joining Us + +You can join us by subscribing to our [mailing list](https://mailweb.openeuler.org/postorius/lists/compass-ci.openeuler.org/). + +Welcome to join us to improve: + +- The git bisect capability. +- The data analysis capability. +- The data result in visualization capability. + +## Learn More + +Click to [Learn More](./doc/) diff --git a/translation-projects/README-optimization/optimized-README/digest-list-tools-en-opt.md b/translation-projects/README-optimization/optimized-README/digest-list-tools-en-opt.md new file mode 100644 index 0000000..53802d7 --- /dev/null +++ b/translation-projects/README-optimization/optimized-README/digest-list-tools-en-opt.md @@ -0,0 +1,437 @@ +# digest-list-tools + +## Description + +Integrity Measurement Architecture (IMA) is software in the Linux kernel for measuring files accessed with the execve(), mmap(), and open() system calls. Measurements can be reported to a remote verifier or compared to reference values, for appraisal. + +The IMA Digest Lists extension stores in the kernel memory reference values of OS software, and adds a new entry to the measurement list only if calculated file digests are not found among those values. This new type of IMA measurement list which only contains digest lists and unknown files uses a different PCR, which can be specified in the kernel command line with the option 'ima_digest_list_pcr=#PCR'. + +The main purpose of this extension is to overcome one of the main challenges when OS files are measured: final PCR values when the OS is running cannot be predicted, as files can be accessed in a different order due to parallel execution. + +With the Digest Lists extension, this problem does not arise as only the measurement of the preloaded digest lists will be used to update the PCR. In the good case, the PCR is not further extended if file digests are found in one of the lists. In the bad case, the PCR is extended with the digest of unknown files. + +The IMA Digest Lists extension can be also used to grant access to files when the appraisal is enabled. There are two possible usages. Access can be granted if the digest of the file content is found in a digest list: this is less secure as metadata is not taken into account. Access can be granted if the digest of metadata is found in a digest list; this is more secure as the current value of extended attributes and inode attributes protected by EVM must match with those set when the digest list was created (e.g. by the vendor). + +More information about the extension can be found at the URL: + +https://github.com/euleros/linux/wiki/IMA-Digest-Lists-Extension + + + +## Software Architecture + +digest-list-tools provides a set of tools necessary to configure the IMA Digest +Lists extension: + +- gen_digest_lists: + Generates digest lists from different sources, e.g. the RPM database, an RPM package, or a directory; +- manage_digest_lists: + Manages digest lists and converts digest lists of arbitrary formats to the format supported by the kernel; +- upload_digest_lists: + Runs parsers of digest lists to upload those that are in a format not recognized by the kernel; +- verify_digest_lists: + Verifies the integrity of digest lists; +- setup_ima_digest_lists: + Generates digest lists, and optionally updates the initial ramdisk, including the digest lists just created; + +- setup_ima_digest_list_demo: + Script with a predefined workflow to create digest lists. + +Both manage_digest_lists and gen_digest_lists have a modular design: they can support additional parsers/generators. Third-party libraries should be placed in the $libdir/digestlist directory. + + + +### Lifecycle + + gen_digest_lists: + +----------------------+ + | Source (e.g. RPM DB) | (1) provide source + +----------------------+ + | + | + +------------+ +-------------+ (3) generate digest list and sign + | Generator 1| ... | Generator N | ---------------------------------| + +------------+ +-------------+ | + +-----------------------------------+ +-------------+ | + | Base library (I/O, xattr, crypto) | <---- | Signing Key | | + +-----------------------------------+ +-------------+ | + (2) provide signing key | + +------+--------------+ + | Sig | Digest list | + | | (fmt N) | + +------+--------------+ + manage_digest_lists: | + (4) parse digest list (fmt N) | + +----------+ +----------+ | + | Parser 1 | ... | Parser N | <--------------------------------| + +----------+ +----------+ | + +-----------------------------------+ | + | Compact list API (generator) | (5) convert to compact list | + +-----------------------------------+ and sign | + +-----------------------------------+ +-------------+ | + | Base library (I/O) | <---- | Signing Key | | + +-----------------------------------+ +-------------+ | + | | + | | + | | + upload_digest_lists: | | + | (6) upload digest list +------------+ + +-------------+ | Parser | + | Kernel | <--------------------------------- | exec | + +-------------+ +------------+ + + + +### Digest List Types + +Digest list types have been defined to restrict the usage of digest list data for different purposes. + +- COMPACT_KEY: + This type of digest list contains the public key used to verify the signatures of the other digest lists. + +- COMPACT_PARSER: + This type of digest list contains the digests of the parser executable and its shared libraries (including the ones that support new digest list formats). IMA will not allow userspace processes to upload converted digest lists unless they have this digest type. + +- COMPACT_FILE: + This type of digest list contains digests of regular files. + +- COMPACT_METADATA: + This type of digest list contains digests of file metadata calculated in the same way as for EVM portable signatures. + + + +### Digest List Modifiers + +Digest list modifiers are used to provide additional attributes to digest list types. + +- COMPACT_MOD_IMMUTABLE: + This modifier restricts the usage of the file if the appraisal is in enforcing mode. Files whose digest has this modifier can be opened only for read. + + + +### Digest List Directory + +All digest lists are stored by default in the /etc/ima/digest_lists directory. +The format of the file is as follows: + +<#position>-\_list-\-\ + +For example, a typical content of the digest list directory is: + +``` +/etc/ima/digest_lists/0-metadata_list-rpm-libxslt-1.1.29-4.fc27-x86_64 +/etc/ima/digest_lists/0-metadata_list-rpm-sqlite-libs-3.20.1-2.fc27-x86_64 +/etc/ima/digest_lists/0-metadata_list-rpm-xkeyboard-config-2.22-1.fc27-noarch +``` + + +## Installation +### Use Case - Measurement and Appraisal of Executable Code + +This setup procedure can be used to enable appraisal of binaries, shared libraries, and scripts with digest lists. + +#### Prerequisite for Measurement + +- check the algorithm of digests in the RPM database by executing: +``` + rpm -q systemd --queryformat "%{RPMTAG_FILEDIGESTALGO}\n" +``` + the association between ID and digest algorithms can be retrieved at: + https://tools.ietf.org/html/rfc4880#section-9.4 +- add to the kernel command line: +``` + ima_hash= +``` + +#### Prerequisite for Appraisal + +- generate a signing key and a certificate including the public key; certs/signing_key.pem in the kernel source can be used +- convert the certificate to DER format and copy it to /etc/keys: +``` + openssl x509 -in certs/signing_key.pem -out /etc/keys/x509_evm.der \ + -outform der +``` +- add an IMA signature to x509_evm.der with the private part of the same key +- remove 'root=' option from the kernel command line and add the following line to /etc/dracut.conf: +``` + kernel_cmdline+="root=" +``` +- add the following line to /etc/dracut.conf, to include the public key to verify the digest lists: +``` + install_items+="/etc/keys/x509_ima.der /etc/keys/x509_evm.der" +``` + + +#### Bootloader Configuration + +It is recommended to create the following entries and add the string below to the kernel command line: + +1) MEASUREMENT +``` + ima_digest_list_pcr=11 ima_policy="tcb|initrd" +``` + +2) APPRAISAL ENFORCE +``` + ima_digest_list_pcr=11 ima_policy="tcb|initrd|appraise_tcb|appraise_initrd" \ + ima_appraise=digest ima_appraise=enforce-evm +``` + +#### IMA Policy + +The following policy must be written to /etc/ima/ima-policy: + +``` +measure func=MMAP_CHECK mask=MAY_EXEC +measure func=BPRM_CHECK mask=MAY_EXEC +measure func=MODULE_CHECK +measure func=FIRMWARE_CHECK +measure func=POLICY_CHECK +appraise func=MODULE_CHECK appraise_type=imasig +appraise func=FIRMWARE_CHECK appraise_type=imasig +appraise func=KEXEC_KERNEL_CHECK appraise_type=imasig +appraise func=POLICY_CHECK appraise_type=imasig +appraise func=BPRM_CHECK appraise_type=imasig +appraise func=MMAP_CHECK +``` + +The imasig requirement cannot be applied to the MMAP_CHECK hook, as some processes (e.g. firewalld) map as executable files in tmpfs. + + +#### Setup + +In a system with the RPM package manager, digest lists can be generated with the command: + +``` +# gen_digest_lists -t metadata -f rpm+db -i l: -o add -p -1 -m immutable \ + -i f:compact -i F:/lib/firmware -i F:/lib/modules -d /etc/ima/digest_lists \ + -i i: -i x: -i e: +``` + +The command above selects only packaged files with execute bit set and all the files in the /lib/firmware and /lib/modules directories. It adds both IMA and EVM digests to the digest lists for all packages in the RPM database. + +Without an execution policy hardcoded in the kernel, it is necessary to create a complete digest list for systemd, as configuration files will be still measured and appraised until the custom policy is loaded by systemd itself: + +``` +# gen_digest_lists -t metadata -f rpm+db -i l: -o add -p -1 -m immutable \ + -i f:compact -i F:/lib/firmware -i F:/lib/modules -d /etc/ima/digest_lists \ + -i i: -i x: -i p:systemd +``` + +With a custom kernel, it is necessary to additionally execute: + +``` +# gen_digest_lists -t metadata -f compact -i l: -o add -p -1 -m immutable \ + -i I:/lib/modules/`uname -r` -d /etc/ima/digest_lists -i i: -i x: +``` + +Other files not known by the package manager can be also added to a digest list: +``` +# gen_digest_lists -t metadata -f unknown -i l: -o add -p -1 -m immutable \ + -i D:/etc/ima/digest_lists -i I: -d /etc/ima/digest_lists \ + -i i: -i x: -i e: +``` + +After digest lists are created, they must be signed with evmctl: + +``` +# evmctl sign -o -a sha256 --imahash --key -r \ + /etc/ima/digest_lists +``` + +Regenerate the initial ram disk and include the custom IMA policy: + +``` +# dracut -f -exattr -I /etc/ima/ima-policy +``` + +To execute the command above, that includes extended attributes in the initial ram disk, it is necessary to apply the patches available at: + +https://github.com/euleros/cpio/tree/xattr-v1 +https://github.com/euleros/dracut/tree/digest-lists + + +Digest lists will be automatically included in the initial ram disk by the new dracut module 'digestlist', part of this software. Its configuration file is in /etc/dracut.conf.d. + + +#### Boot Process + +Digest lists are loaded as early as possible during the boot process, so that digests can be found before files are accessed. The kernel reads and parses the digest lists in the /etc/ima/digest_lists directory. + + +#### Software Update + +If new RPMs are installed on the system, new digest lists must be created with the same commands introduced above. The new digest lists are not automatically loaded at boot until the initial ram disk is regenerated. A systemd service will be developed to load new digest lists without regenerating the initial ram disk. + + + +### Use Case - Immutable and Mutable Files (with HMAC Key) + +The steps described below represent only a configuration example. The list of files that should be included in the digest lists and the type (immutable or mutable) depend on user requirements. The setup process is organized into two different steps. First, the system is booted in rescue mode so that the digest of mutable files can be reliably calculated (there is no process accessing them). + +During the first step, the administrator launches the setup_ima_digest_lists_demo script to create digest lists for the system. +It might be done by the software vendor if the content of all files that will be measured/appraised is known in advance. Otherwise, the administrator becomes responsible for the initial values of the files that will be accessed by the system, by signing the digest lists. At this stage, the HMAC key is not yet available. It will be created and sealed once the digest lists are generated. + +For the second step, the administrator runs the system in the final configuration, so that the HMAC key can be unsealed, but still selects the rescue mode. During this step, the administrator launches again the setup_ima_digest_lists_demo script to add an HMAC to every file verified with the digest lists. + +#### Prerequisite for measurement: + +- add 'iversion' mount option in /etc/fstab (if the filesystem supports it) +- check the algorithm of digests in the RPM database by executing: +``` + rpm -q systemd --queryformat "%{RPMTAG_FILEDIGESTALGO}\n" +``` + the association between ID and digest algorithms can be retrieved at: + https://tools.ietf.org/html/rfc4880#section-9.4 +- add to the kernel command line: +``` + ima_hash= +``` + +#### Prerequisite for appraisal: + +- generate a signing key and a certificate including the public key; certs/signing_key.pem in the kernel source can be used +- convert the certificate to DER format and copy it to /etc/keys: +``` + openssl x509 -in certs/signing_key.pem -out /etc/keys/x509_ima.der \ + -outform der +``` +- generate EVM keys; follow instructions at https://sourceforge.net/p/linux-ima/wiki/Home/, section 'Creating trusted and EVM encrypted keys' +- remove 'root=' option from the kernel command line and add the following line to /etc/dracut.conf: +``` + kernel_cmdline+="root=" +``` +- copy the following dracut modules from the GIT repository at https://github.com/dracutdevs/dracut to /usr/lib/dracut/modules.d: +``` + 96securityfs 97masterkey 98integrity +``` +- include dracut modules in the ram disk by adding to /etc/dracut.conf: +``` + add_dracutmodules+=" securityfs masterkey integrity" +``` +- add the following lines to /etc/dracut.conf, to include the public key to verify the digest lists, and the EVM keys: +``` + install_items+="/etc/keys/x509_ima.der" + install_items+="/etc/keys/kmk-trusted.blob /etc/keys/evm-trusted.blob" +``` + (in the last line, replace kmk-trusted with kmk-user if a user key was used as masterkey) +- add the following line to /etc/dracut.conf, to include SELinux labels in the initial ram disk: +``` + install_items+="/etc/selinux/targeted/contexts/files/file_contexts" + install_items+=/etc/selinux/targeted/contexts/files/file_contexts.subs_dist" +``` + + +#### Bootloader Configuration + +It is recommended to create the following entries and add the string below to the kernel command line: + +1) SETUP +``` + systemd.unit=setup-ima-digest-lists.service +``` + +2) MEASUREMENT +``` + ima_digest_list_pcr=11 ima_policy="tcb|initrd" +``` + +3) APPRAISAL ENFORCE SETUP +``` + ima_digest_list_pcr=11 ima_policy="tcb|initrd|appraise_tcb|appraise_initrd| \ + appraise_tmpfs" ima_appraise=digest ima_appraise=enforce-evm evm=random + systemd.unit=setup-ima-digest-lists.service +``` + +4) APPRAISAL ENFORCE +``` + ima_digest_list_pcr=11 ima_policy="tcb|initrd|appraise_tcb|appraise_initrd| \ + appraise_tmpfs" ima_appraise=digest ima_appraise=enforce-evm evm=random +``` + +5) APPRAISAL PERMISSIVE +``` + ima_digest_list_pcr=11 ima_policy="tcb|initrd|appraise_tcb|appraise_initrd| \ + appraise_tmpfs" ima_appraise=digest ima_appraise=log-evm evm=random +``` + + +#### Setup - First Phase + +##### With RPM Package Manager + +digest-list-tools includes a script called setup_ima_digest_lists_demo to simplify the creation of digest lists. It will create the following digest lists: + +- digest lists from package manager +- digest list of unknown files in the initial ram disk (some are generated by dracut) +- digest list of IMA policy +- digest list of unknown files in the root filesystem so that appraisal can be enabled (important: digest of metadata will be created from the current value of extended attributes; they must be checked by the administrator before the digest list is generated and signed) + +1) Execute: + +``` +# setup_ima_digest_lists_demo initial [signing key] [X.509 certificate] +``` + +The procedure is interactive and the script asks the user to confirm/edit the list of files whose digest will be included in the digest list. + +2) Reboot + +Reboot the system to load the new digest lists during the boot process. + + +##### Without RPM Package Manager + +An alternative way to create a digest list is to directly take file digests from the filesystem without using the package manager. To do that, it is sufficient to edit setup_ima_digest_lists_demo and to comment on the line that begins with 'setup_ima_digest_lists distro'. + + +#### Setup - Second Phase + +After the first phase of the setup, /etc/ima/digest_lists contains all the digest lists necessary to boot the system with appraisal enabled and enforced. The remaining step is to add an HMAC to every file added to the digest lists. + +1) Execute: + +``` +# setup_ima_digest_lists_demo final +``` + +### Software Update + +#### Generation + +Digest lists can be generated with the gen_digest_lists tool. A description of this command can be obtained by executing the command: + +``` +$ man gen_digest_lists +``` + +### Integrity Verification + +The measurement list, after loading the digest lists, will look like: + +``` +11 ima-ng boot_aggregate +11 ima-ng /etc/keys/x509_ima.der +11 ima-ng [...]/0-parser_list-compact-manage_digest_lists +11 ima-ng [...]/0-key_list-signing_key.der +11 ima-ng [...]/1-parser_list-compact-libparser-ima.so +11 ima-ng [...]/2-parser_list-compact-libparser-rpm.so +11 ima-ng [...]/0-file_list-rpm-libxslt-1.1.29-4.fc27-x86_64 +... + +``` + +An attestation server can use the verify_digest_lists tool to verify the integrity of digest lists. For example, it can execute: + +``` +$ verify_digest_lists +``` + + +## Author +Written by Roberto Sassu, . + + + +## Copying +Copyright (C) 2018-2020 Huawei Technologies Duesseldorf GmbH. Free use of this software is granted under the terms of the GNU Public License 2.0 (GPLv2). diff --git a/translation-projects/README-optimization/optimized-README/docs-en.md b/translation-projects/README-optimization/optimized-README/docs-en.md new file mode 100644 index 0000000..be88989 --- /dev/null +++ b/translation-projects/README-optimization/optimized-README/docs-en.md @@ -0,0 +1,58 @@ +# openEuler DOCS + +English | [简体中文](./README.md) + +### Introduction + +DOCS contains all documents of the openEuler community, including the release notes, OS installation guide, administrator guide, virtualization, container, A-Tune user guide, and application development guide. + +### Searching for a Document in DOCS + +1. Open the **docs** folder. The folder contains documents in Chinese (**zh** folder) and English (**en** folder). For example, in the **en** folder, the **docs** folder contains different documents, and the **menu** folder displays the list of documents. +2. Open the **docs** folder. You can see sub-folders, each with relevant documents inside. The following table shows an example of what the sub-folders may contain: + +| Folder | Content | +| ------------------ | ---------------------------------- | +| **A-Tune** | *A-Tune User Guide* | +| **Administrator** | *Administrator Guide* | +| **ApplicationDev** | *Application Development Guide* | +| **Container** | *Container User Guide* | +| **Installation** | *Installation Guide* | +| **Quickstart** | *Quick Start* | +| **Releasenotes** | *Release Notes* | +| **SecHarden** | *Security Hardening Guide* | +| **Virtualization** | *Virtualization Application Guide* | +| **userGuide** | *openEuler Toolset User Guide* | +| **StratoVirt** | *StratoVirt User Guide* | + + +### Modifying a Document + +When the openEuler version information is updated, the documents here also need to be updated. Thank you for providing updates. Refer to [DOCS Development Guide](https://gitee.com/lss410313/docs/wikis/Home) if you have questions. + +### Checking the Relationship Between Versions and Branches +Before performing updates, you need to first make sure the branch you choose is consistent with the updated version. The DOCS contains the following four branches: + +| Branch | Description | Documentation | +| ---------------------- | ----------------------------------------------- | ------------------------------------------------------------ | +| **master** | development branch, which is the default branch | | +| **stable2-1.0\_Base** | 1.0 Base version branch | **DOCS** > **1.0 BASE** on the [openEuler community website](https://openeuler.org/) | +| **stable2-20.03\_LTS** | 20.03 LTS version branch | **DOCS** > **20.03 LTS** on the [openEuler community website](https://openeuler.org/) | +| **stable2-20.09** | 20.09 version branch | **DOCS** > **20.09** on the [openEuler community website](https://openeuler.org/) | + +### Participation +Create or reply to an issue: You can discuss with us by creating or replying to an issue. +Submit a Pull Request (PR): You can participate in SIG by submitting a PR. For details, please refer to [PR Submission Guide](https://gitee.com/openeuler/community/blob/master/zh/contributors/pull-request.md). +Make a comment: You can submit comments on issues or PRs. You can also comment on the document through **Feedback** on the [website document page](https://docs.openeuler.org/en/). +Welcome to submit PRs! + +### Member +#### Maintainers +- Rudy_Tan[@rudy_tan](https://gitee.com/rudy_tan) +- amyMaYun[@amy_mayun](https://gitee.com/amy_mayun) +- qiaominna[@qiaominna](https://gitee.com/qiaominna) + + +### Contacting Us +E-mail: doc@openeuler.org +IRC: #openeuler-doc -- Gitee