From 737fa8685c325acabeee5619f6d74e02bcd72fc0 Mon Sep 17 00:00:00 2001 From: cf-zhao Date: Wed, 12 Feb 2025 15:32:49 +0800 Subject: [PATCH] Add build script and update README. --- README-zh.md | 64 +++++++++ README.md | 90 +++++++----- build.sh | 385 +++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 507 insertions(+), 32 deletions(-) create mode 100644 README-zh.md create mode 100755 build.sh diff --git a/README-zh.md b/README-zh.md new file mode 100644 index 000000000000..a71d0fa4552c --- /dev/null +++ b/README-zh.md @@ -0,0 +1,64 @@ +## 1、项目介绍 + +欢迎来到openEuler社区的LLVM项目！本仓库是[llvm-project](https://github.com/llvm/llvm-project)下游仓库。 + +此仓库包含LLVM项目的源代码，LLVM基础设施是一个用于构建高度优化的编译器、优化器和运行时环境的工具包。 + +LLVM项目有多个组件。该项目的核心组件被称为“LLVM”，它包含处理中间表示及将其转换为目标文件所需的所有工具（包括汇编器、反汇编器、位码分析器和为位码优化器）、库和头文件。 + +类C语言使用Clang前端，该组件将C、C++、Objective-C和Objective-C++代码编译成LLVM位码，并使用LLVM将其转换为二进制目标文件。 + +其他组件包括：C++标准库（libc++）、LLD链接器等。 + +## 2、构建指导 + +首先通过git下载源码，然后通过build.sh脚本一键式构建LLVM。构建方式有`直接命令行构建`和`容器化构建`两种。 + +### 2.1、直接命令行构建 + +推荐使用openEuler操作系统进行构建，如果您使用其他操作系统，建议使用容器化构建方式。 + +首先确保系统安装了依赖软件包，可以用如下命令安装。 +``` +yum install -y gcc g++ make cmake openssl-devel python3 \ +python3-setuptools python-wheel texinfo binutils-devel libatomic +``` +然后可以通过 `./build.sh -h`查看当前工程支持的构建选项。通过命令行执行一键式构建，例如： +``` +./build.sh -r -b release -X X86 -j 8 +``` + +### 2.2、容器化构建 + +为了解决由于开发环境差异导致的构建失败和构建产物二进制差异问题，openEuler LLVM项目提供了容器化构建方法。得益于[openEuler容器镜像项目](https://gitee.com/openeuler/openeuler-docker-images)，提前制作了[llvm-build-deps容器镜像](https://gitee.com/openeuler/openeuler-docker-images/tree/master/llvm-build-deps)。开发者可以通过`build.sh`脚本的`-C`选项启用容器化构建，例如： +``` +./build.sh -C -r -b release -X X86 -j 8 // 添加了-C选项 +``` + +相关依赖： +* 开发环境需要正确安装了docker应用。 +* 用户加入了docker用户组，使得`build.sh`脚本执行docker命令时不需要再加`sudo`命令。可以通过如下命令将当前用户加入docker用户组。 +``` +sudo usermod -aG docker ${USER} +``` +注意：第一次执行容器化构建时，脚本会自动从镜像仓库拉取llvm-build-deps容器镜像。 + +## 3、贡献指导 + +1、Fork 本仓库 +2、新建 Feat_xxx 分支 +3、提交代码 +4、新建 Pull Request + +## 4、讨论与求助 + +### 4.1、上游社区 +* 加入[discourse论坛](https://discourse.llvm.org/)，提出或参与问题、RFC等交流。 + +* 社区参与者的[行为规范](https://llvm.org/docs/CodeOfConduct.html). + +### 4.1、openEuler社区Compiler SIG +几种方式： +* 订阅[Compiler SIG邮件列表](https://mailweb.openeuler.org/postorius/lists/compiler@openeuler.org/) +* 在[openEuler论坛](https://forum.openeuler.org/?locale=zh_CN)发帖讨论。 +* 微信交流群：请先添加微信名`Compiler_Assistant` diff --git a/README.md b/README.md index a9b29ecbc1a3..8665dede3cda 100644 --- a/README.md +++ b/README.md @@ -1,44 +1,70 @@ -# The LLVM Compiler Infrastructure +## 1. Project Introduction -[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/llvm/llvm-project/badge)](https://securityscorecards.dev/viewer/?uri=github.com/llvm/llvm-project) -[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/8273/badge)](https://www.bestpractices.dev/projects/8273) -[![libc++](https://github.com/llvm/llvm-project/actions/workflows/libcxx-build-and-test.yaml/badge.svg?branch=main&event=schedule)](https://github.com/llvm/llvm-project/actions/workflows/libcxx-build-and-test.yaml?query=event%3Aschedule) +Welcome to the LLVM project in the openEuler community! This warehouse is the downstream warehouse of [llvm-project](https://github.com/llvm/llvm-project). -Welcome to the LLVM project! +This repository contains the source code for LLVM, a toolkit for the construction of highly optimized compilers, optimizers, and run-time environments. -This repository contains the source code for LLVM, a toolkit for the -construction of highly optimized compilers, optimizers, and run-time -environments. +The LLVM project has multiple components. The core of the project is itself called "LLVM". This contains all of the tools, libraries, and header files needed to process intermediate representations and convert them into object files. Tools include an assembler, disassembler, bitcode analyzer, and bitcode optimizer. -The LLVM project has multiple components. The core of the project is -itself called "LLVM". This contains all of the tools, libraries, and header -files needed to process intermediate representations and convert them into -object files. Tools include an assembler, disassembler, bitcode analyzer, and -bitcode optimizer. +C-like languages use the [Clang](https://clang.llvm.org/) frontend. This component compiles C, C++, Objective-C, and Objective-C++ code into LLVM bitcode -- and from there into object files, using LLVM. -C-like languages use the [Clang](https://clang.llvm.org/) frontend. This -component compiles C, C++, Objective-C, and Objective-C++ code into LLVM bitcode --- and from there into object files, using LLVM. +Other components include: the [libc++ C++ standard library](https://libcxx.llvm.org/), the [LLD linker](https://lld.llvm.org/), and more. -Other components include: -the [libc++ C++ standard library](https://libcxx.llvm.org), -the [LLD linker](https://lld.llvm.org), and more. +## 2. Construction Guide -## Getting the Source Code and Building LLVM +You can use `git` to download the source code, and then use the `build.sh` script to build the LLVM in one-click mode. There are two build modes: `build with command line` and `build with container`. -Consult the -[Getting Started with LLVM](https://llvm.org/docs/GettingStarted.html#getting-the-source-code-and-building-llvm) -page for information on building and running LLVM. +### 2.1. Build with command line directly -For information on how to contribute to the LLVM project, please take a look at -the [Contributing to LLVM](https://llvm.org/docs/Contributing.html) guide. +You are advised to use the `openEuler` for building. If you use other operating systems, you are advised to use the containerized building mode. -## Getting in touch +Ensure that the dependency software packages are installed. You can run the following command to install the software packages: -Join the [LLVM Discourse forums](https://discourse.llvm.org/), [Discord -chat](https://discord.gg/xS7Z362), -[LLVM Office Hours](https://llvm.org/docs/GettingInvolved.html#office-hours) or -[Regular sync-ups](https://llvm.org/docs/GettingInvolved.html#online-sync-ups). +` ` ` +yum install -y gcc g++ make cmake openssl-devel python3 \ +python3-setuptools python-wheel texinfo binutils-devel libatomic +` ` ` -The LLVM project has adopted a [code of conduct](https://llvm.org/docs/CodeOfConduct.html) for -participants to all modes of communication within the project. +You can run the `./build.sh -h` command to view the build options supported by the current project. Run the following command to perform a one-click build: + +` ` ` +./build.sh -r -b release -X X86 -j 8 +` ` ` + +### 2.2 Build with container + +The openEuler LLVM project provides a containerized building mode to solve the problems of build failures and binary differences of build products caused by development environment differences. Thanks to the [openEuler container image project](https://gitee.com/openeuler/openeuler-docker-images), the [llvm-build-deps container image](https://gitee.com/openeuler/openeuler-docker-images/tree/master/llvm-build-deps) is created in advance. Developers can enable containerized builds using the `-C` option of the `build.sh` script. For example: + +` ` ` +./build.sh -C -r -b release -X X86 -j 8 // added -C option +` ` ` + +Dependency: +* The Docker application must be correctly installed in the development environment. +* The user is added to the docker user group so that the `sudo` command is not required when the `build.sh` script executes the docker command. You can run the following command to add the current user to the docker user group: + +` ` ` +sudo usermod -aG docker ${USER} +` ` ` + +Note: When you perform a containerized build for the first time, the script automatically pulls the `llvm-build-deps container image` from the image repository. + +## 3. Contribution guidance + +1. Fork This Warehouse +2. Create the Feat_xxx branch. +3. Submit the code. +4. Create a Pull Request. + +## 4. Discussion and help-seeking + +### 4.1 Upstream Community +* Join the [Discourse Forum](https://discourse.llvm.org/) . + +* [Code of Conduct](https://llvm.org/docs/CodeOfConduct.html) for Community Participants. + +### 4.1. Compiler SIG of the openEuler community +There are several ways: +* Subscribe to the [Compiler SIG mailing list](https://mailweb.openeuler.org/postorius/lists/compiler@openeuler.org/) +* Post a discussion at the [openEuler forum](https://forum.openeuler.org/?locale=zh_CN). +* WeChat communication group: Please add the WeChat name `Compiler_Assistant` first. \ No newline at end of file diff --git a/build.sh b/build.sh new file mode 100755 index 000000000000..afa030c258fb --- /dev/null +++ b/build.sh @@ -0,0 +1,385 @@ +#!/bin/bash + +# Tools to use for bootstrapping. +C_COMPILER_PATH=gcc +CXX_COMPILER_PATH=g++ + +# Initialize our own variables: +buildtype=RelWithDebInfo +backends="all" +enabled_projects="clang;lld;compiler-rt;openmp;clang-tools-extra" +embedded_toolchain="0" +split_dwarf=on +use_ccache="0" +do_install="0" +clean=0 +containerize=0 +docker=$(type -p docker) +host_arch="$(uname -m)" +unit_test="" +install="install" +install_toolchain_only="0" +verbose="" +dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )" +build_dir_name="build" +install_dir_name="install" +build_prefix="$dir/$build_dir_name" +install_prefix="$dir/$install_dir_name" + +# Use 8 threads for builds and tests by default. Use more threads if possible, +# but avoid overloading the system by using up to 50% of available cores. +threads=8 +nproc=$(type -p nproc) +if [ -x "$nproc" -a -f /proc/loadavg ]; then + loadavg=$(awk '{printf "%.0f\n", $1}' < /proc/loadavg) + let threads="($($nproc) - $loadavg) / 2" + if [ $threads -le 0 ]; then + threads=1 + fi +fi + +# Exit script on first error. +set -e + +usage() { + cat < /dev/null 2>&1 +} + +# Handle interrupts. When not running a containerized build, we have to enable +# job control (-m), and make sure to delete our own long-running child +# processes. In particular, ninja and python (llvm-lit) refuse to terminate +# when Jenkins aborts the parent process and disconnects. +set -m +handle_abort() { + local rc=$1 sig=$2 + build_cleanup + trap - EXIT SIGHUP SIGINT SIGTERM + local children="$(jobs -p)" + for cgrp in $children ; do + kill -$sig -${cgrp} > /dev/null 2>&1 + done + if [ -n "$sig" ]; then + kill -$sig 0 + else + exit $rc + fi +} + +if [ $containerize -eq 0 ]; then + trap - SIGCHLD + trap 'handle_abort $?' EXIT + trap 'handle_abort 129 HUP' SIGHUP + trap 'handle_abort 130 INT' SIGINT + trap 'handle_abort 143 TERM' SIGTERM +else + cmd=$(readlink --canonicalize-existing $0) + + # Generate passwd/group files for the container. + # lit.py depends on correct results from getpwent. + homedir=$(realpath $HOME) + passwd=$(mktemp /tmp/passwd.XXXXXX) + echo "root:x:0:0::/root:/bin/bash" > $passwd + echo "user:x:$(id -u):$(id -g)::$homedir:/bin/bash" >> $passwd + + group=$(mktemp /tmp/group.XXXXXX) + echo "root:x:0:" > $group + echo "users:x:$(id -g):" >> $group + + # Re-run myself in a openEuler container. The --cap-add option is needed + # to pacify llvm-exegesis unit tests. Make sure that the container is + # stopped if the child process is interrupted (e.g. by Jenkins). + # Note that the container ID file is created with `mktemp -u` because + # `docker run` refuses to overwrite an existing ID file. + echo "Re-launching in container." + containerid=$(mktemp -u /tmp/docker-cid.$$.XXXXXX) + docker_cleanup() { + local rc=$1 sig=$2 + docker container stop $(cat $containerid) > /dev/null 2>&1 + rm $passwd $group $containerid + build_cleanup + trap - EXIT SIGHUP SIGINT SIGTERM # avoid infinite recursion + if [ -n "$sig" ]; then + kill -$sig 0 + else + exit $rc + fi + } + trap 'docker_cleanup $?' EXIT + trap 'docker_cleanup 129 HUP' SIGHUP + trap 'docker_cleanup 130 INT' SIGINT + trap 'docker_cleanup 143 TERM' SIGTERM + + DOCKER_IMAGE="llvm-build-deps:latest" + docker_opts="--rm + --cap-add=SYS_ADMIN + --cap-add=SYS_PTRACE + --security-opt seccomp=unconfined + --user $(id -u):$(id -g) + --workdir=$(realpath $PWD) + --ulimit core=0 + --ulimit stack=-1 + --cidfile $containerid + -v $homedir:$homedir + -v $passwd:/etc/passwd + -v $group:/etc/group + -e BINUTILS_INCDIR=/usr/local/include + hub.oepkgs.net/openeuler/${DOCKER_IMAGE}" + + if [ -t 1 ]; then + $docker run -it $docker_opts ${cmd} ${containerized_opts[@]} + exit $? + else + set -x + $docker run $docker_opts ${cmd} ${containerized_opts[@]} & + wait $! || exit $? + exit 0 + fi +fi + +echo "Using $threads threads." + +CMAKE_OPTIONS="-DCMAKE_INSTALL_PREFIX=$install_prefix \ + -DCMAKE_BUILD_TYPE=$buildtype \ + -DCMAKE_C_COMPILER=$C_COMPILER_PATH \ + -DCMAKE_CXX_COMPILER=$CXX_COMPILER_PATH \ + -DLLVM_TARGETS_TO_BUILD=$backends " + +gold=$(type -p ld.gold) +if [ -z "$gold" -o ! -x "$gold" ]; then + echo "$0: no usable ld.gold" + exit 1 +fi + +# If the invocation does not force a particular binutils installation, check +# that we are using an acceptable version. +if [ -n "$BINUTILS_INCDIR" ]; then + llvm_binutils_incdir="-DLLVM_BINUTILS_INCDIR=$BINUTILS_INCDIR" +else + incdir=$(realpath --canonicalize-existing $(dirname $gold)/../include) + if [ -z "$incdir" -o ! -f "$incdir/plugin-api.h" ]; then + echo "$0: plugin-api.h not found; required to build LLVMgold.so" + exit 1 + fi + llvm_binutils_incdir="-DLLVM_BINUTILS_INCDIR=$incdir" +fi + +# Warning: the -DLLVM_ENABLE_PROJECTS option is specified with cmake +# to avoid issues with nested quotation marks +if [ $use_ccache == "1" ]; then + echo "Build using ccache" + CMAKE_OPTIONS="$CMAKE_OPTIONS \ + -DCMAKE_C_COMPILER_LAUNCHER=ccache \ + -DCMAKE_CXX_COMPILER_LAUNCHER=ccache " +fi + +if [ $embedded_toolchain == "1" ]; then + echo "Build for embedded cross tool chain" + enabled_projects="clang;lld;compiler-rt;" + CMAKE_OPTIONS="$CMAKE_OPTIONS \ + -DLLVM_BUILD_FOR_EMBEDDED=ON" +fi + +# When set LLVM_INSTALL_TOOLCHAIN_ONLY to On it removes many of the LLVM development +# and testing tools as well as component libraries from the default install target. +if [ $install_toolchain_only == "1" ]; then + echo "Only install toolchain" + CMAKE_OPTIONS="$CMAKE_OPTIONS -DLLVM_INSTALL_TOOLCHAIN_ONLY=ON" +fi + +# Build and install +if [ $clean -eq 1 -a -e "$install_prefix" ]; then + rm -rf "$install_prefix" +fi +mkdir -p "$install_prefix/bin" + +if [ $clean -eq 1 -a -e "$build_prefix" ]; then + rm -rf "$build_prefix" +fi + +mkdir -p "$build_prefix" && cd "$build_prefix" +cmake $CMAKE_OPTIONS \ + -DCOMPILER_RT_BUILD_SANITIZERS=on \ + -DLLVM_ENABLE_PROJECTS=$enabled_projects \ + -DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi;libunwind" \ + -DLLVM_USE_LINKER=gold \ + -DLLVM_LIT_ARGS="-sv -j$threads" \ + -DLLVM_USE_SPLIT_DWARF=$split_dwarf \ + -DCMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO="-Wl,--gdb-index -Wl,--compress-debug-sections=zlib" \ + -DCMAKE_EXE_LINKER_FLAGS_DEBUG="-Wl,--gdb-index -Wl,--compress-debug-sections=zlib" \ + -DBUILD_SHARED_LIBS=OFF \ + -DLLVM_ENABLE_LIBCXX=OFF \ + -DLLVM_ENABLE_ZLIB=ON \ + -DLLVM_BUILD_RUNTIME=ON \ + -DLLVM_INCLUDE_TOOLS=ON \ + -DLLVM_BUILD_TOOLS=ON \ + -DLLVM_INCLUDE_TESTS=ON \ + -DLLVM_BUILD_TESTS=ON \ + -DLLVM_INCLUDE_EXAMPLES=ON \ + -DLLVM_BUILD_EXAMPLES=OFF \ + -DCLANG_DEFAULT_PIE_ON_LINUX=ON \ + -DCLANG_ENABLE_ARCMT=ON \ + -DCLANG_ENABLE_STATIC_ANALYZER=ON \ + -DCLANG_PLUGIN_SUPPORT=ON \ + -DLLVM_DYLIB_COMPONENTS="all" \ + -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=ON \ + -DCMAKE_SKIP_RPATH=ON \ + -DLLVM_ENABLE_FFI=ON \ + -DLLVM_ENABLE_RTTI=ON \ + -DLLVM_USE_PERF=ON \ + -DLLVM_INSTALL_GTEST=ON \ + -DLLVM_INCLUDE_UTILS=ON \ + -DLLVM_INSTALL_UTILS=ON \ + -DLLVM_INCLUDE_BENCHMARKS=OFF \ + -DENABLE_LINKER_BUILD_ID=ON \ + -DLLVM_ENABLE_EH=ON \ + -DCLANG_DEFAULT_UNWINDLIB=libgcc \ + -DLIBCXX_STATICALLY_LINK_ABI_IN_STATIC_LIBRARY=ON \ + -DLIBCXX_ENABLE_ABI_LINKER_SCRIPT=ON \ + -DLIBOMP_INSTALL_ALIASES=OFF \ + $llvm_binutils_incdir \ + $verbose \ + ../llvm + +make -j$threads +if [ $do_install == "1" ]; then + make -j$threads $verbose $install +fi + +if [ -n "$unit_test" ]; then + make -j$threads $verbose check-all +fi + +cd .. + +# When building official deliverables, minimize file permissions under the +# installation directory. +if [ "$install" = "install/strip" ]; then + find $install_prefix/bin/ -type f -exec strip {} \; + find $install_prefix -type f -exec chmod a-w {} \; +fi + +# In openEuler embedded building system, it need wrap llvm-readelf +# to replace binutils-readelf. +if [ -e "$install_prefix/bin/llvm-readobj" ]; then + ln -sf llvm-readobj $install_prefix/bin/llvm-readelf +fi + +echo "$0: SUCCESS" -- Gitee