diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000000000000000000000000000000000000..ee5839968a2bf86c93283efc09d40fd050b7cfa2 --- /dev/null +++ b/LICENSE @@ -0,0 +1,127 @@ + 木兰宽松许可证, 第2版 + + 木兰宽松许可证， 第2版 + 2020年1月 http://license.coscl.org.cn/MulanPSL2 + + + 您对“软件”的复制、使用、修改及分发受木兰宽松许可证，第2版（“本许可证”）的如下条款的约束： + + 0. 定义 + + “软件”是指由“贡献”构成的许可在“本许可证”下的程序和相关文档的集合。 + + “贡献”是指由任一“贡献者”许可在“本许可证”下的受版权法保护的作品。 + + “贡献者”是指将受版权法保护的作品许可在“本许可证”下的自然人或“法人实体”。 + + “法人实体”是指提交贡献的机构及其“关联实体”。 + + “关联实体”是指，对“本许可证”下的行为方而言，控制、受控制或与其共同受控制的机构，此处的控制是指有受控方或共同受控方至少50%直接或间接的投票权、资金或其他有价证券。 + + 1. 授予版权许可 + + 每个“贡献者”根据“本许可证”授予您永久性的、全球性的、免费的、非独占的、不可撤销的版权许可，您可以复制、使用、修改、分发其“贡献”，不论修改与否。 + + 2. 授予专利许可 + + 每个“贡献者”根据“本许可证”授予您永久性的、全球性的、免费的、非独占的、不可撤销的（根据本条规定撤销除外）专利许可，供您制造、委托制造、使用、许诺销售、销售、进口其“贡献”或以其他方式转移其“贡献”。前述专利许可仅限于“贡献者”现在或将来拥有或控制的其“贡献”本身或其“贡献”与许可“贡献”时的“软件”结合而将必然会侵犯的专利权利要求，不包括对“贡献”的修改或包含“贡献”的其他结合。如果您或您的“关联实体”直接或间接地，就“软件”或其中的“贡献”对任何人发起专利侵权诉讼（包括反诉或交叉诉讼）或其他专利维权行动，指控其侵犯专利权，则“本许可证”授予您对“软件”的专利许可自您提起诉讼或发起维权行动之日终止。 + + 3. 无商标许可 + + “本许可证”不提供对“贡献者”的商品名称、商标、服务标志或产品名称的商标许可，但您为满足第4条规定的声明义务而必须使用除外。 + + 4. 分发限制 + + 您可以在任何媒介中将“软件”以源程序形式或可执行形式重新分发，不论修改与否，但您必须向接收者提供“本许可证”的副本，并保留“软件”中的版权、商标、专利及免责声明。 + + 5. 免责声明与责任限制 + + “软件”及其中的“贡献”在提供时不带任何明示或默示的担保。在任何情况下，“贡献者”或版权所有者不对任何人因使用“软件”或其中的“贡献”而引发的任何直接或间接损失承担责任，不论因何种原因导致或者基于何种法律理论，即使其曾被建议有此种损失的可能性。 + + 6. 语言 + “本许可证”以中英文双语表述，中英文版本具有同等法律效力。如果中英文版本存在任何冲突不一致，以中文版为准。 + + 条款结束 + + 如何将木兰宽松许可证，第2版，应用到您的软件 + + 如果您希望将木兰宽松许可证，第2版，应用到您的新软件，为了方便接收者查阅，建议您完成如下三步： + + 1， 请您补充如下声明中的空白，包括软件名、软件的首次发表年份以及您作为版权人的名字； + + 2， 请您在软件包的一级目录下创建以“LICENSE”为名的文件，将整个许可证文本放入该文件中； + + 3， 请将如下声明文本放入每个源文件的头部注释中。 + + Copyright (c) [Year] [name of copyright holder] + [Software Name] is licensed under Mulan PSL v2. + You can use this software according to the terms and conditions of the Mulan PSL v2. + You may obtain a copy of Mulan PSL v2 at: + http://license.coscl.org.cn/MulanPSL2 + THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. + See the Mulan PSL v2 for more details. + + + Mulan Permissive Software License，Version 2 + + Mulan Permissive Software License，Version 2 (Mulan PSL v2) + January 2020 http://license.coscl.org.cn/MulanPSL2 + + Your reproduction, use, modification and distribution of the Software shall be subject to Mulan PSL v2 (this License) with the following terms and conditions: + + 0. Definition + + Software means the program and related documents which are licensed under this License and comprise all Contribution(s). + + Contribution means the copyrightable work licensed by a particular Contributor under this License. + + Contributor means the Individual or Legal Entity who licenses its copyrightable work under this License. + + Legal Entity means the entity making a Contribution and all its Affiliates. + + Affiliates means entities that control, are controlled by, or are under common control with the acting entity under this License, ‘control’ means direct or indirect ownership of at least fifty percent (50%) of the voting power, capital or other securities of controlled or commonly controlled entity. + + 1. Grant of Copyright License + + Subject to the terms and conditions of this License, each Contributor hereby grants to you a perpetual, worldwide, royalty-free, non-exclusive, irrevocable copyright license to reproduce, use, modify, or distribute its Contribution, with modification or not. + + 2. Grant of Patent License + + Subject to the terms and conditions of this License, each Contributor hereby grants to you a perpetual, worldwide, royalty-free, non-exclusive, irrevocable (except for revocation under this Section) patent license to make, have made, use, offer for sale, sell, import or otherwise transfer its Contribution, where such patent license is only limited to the patent claims owned or controlled by such Contributor now or in future which will be necessarily infringed by its Contribution alone, or by combination of the Contribution with the Software to which the Contribution was contributed. The patent license shall not apply to any modification of the Contribution, and any other combination which includes the Contribution. If you or your Affiliates directly or indirectly institute patent litigation (including a cross claim or counterclaim in a litigation) or other patent enforcement activities against any individual or entity by alleging that the Software or any Contribution in it infringes patents, then any patent license granted to you under this License for the Software shall terminate as of the date such litigation or activity is filed or taken. + + 3. No Trademark License + + No trademark license is granted to use the trade names, trademarks, service marks, or product names of Contributor, except as required to fulfill notice requirements in Section 4. + + 4. Distribution Restriction + + You may distribute the Software in any medium with or without modification, whether in source or executable forms, provided that you provide recipients with a copy of this License and retain copyright, patent, trademark and disclaimer statements in the Software. + + 5. Disclaimer of Warranty and Limitation of Liability + + THE SOFTWARE AND CONTRIBUTION IN IT ARE PROVIDED WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL ANY CONTRIBUTOR OR COPYRIGHT HOLDER BE LIABLE TO YOU FOR ANY DAMAGES, INCLUDING, BUT NOT LIMITED TO ANY DIRECT, OR INDIRECT, SPECIAL OR CONSEQUENTIAL DAMAGES ARISING FROM YOUR USE OR INABILITY TO USE THE SOFTWARE OR THE CONTRIBUTION IN IT, NO MATTER HOW IT’S CAUSED OR BASED ON WHICH LEGAL THEORY, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. + + 6. Language + + THIS LICENSE IS WRITTEN IN BOTH CHINESE AND ENGLISH, AND THE CHINESE VERSION AND ENGLISH VERSION SHALL HAVE THE SAME LEGAL EFFECT. IN THE CASE OF DIVERGENCE BETWEEN THE CHINESE AND ENGLISH VERSIONS, THE CHINESE VERSION SHALL PREVAIL. + + END OF THE TERMS AND CONDITIONS + + How to Apply the Mulan Permissive Software License，Version 2 (Mulan PSL v2) to Your Software + + To apply the Mulan PSL v2 to your work, for easy identification by recipients, you are suggested to complete following three steps: + + i Fill in the blanks in following statement, including insert your software name, the year of the first publication of your software, and your name identified as the copyright owner; + + ii Create a file named “LICENSE” which contains the whole context of this License in the first directory of your software package; + + iii Attach the statement to the appropriate annotated syntax at the beginning of each source file. + + + Copyright (c) [Year] [name of copyright holder] + [Software Name] is licensed under Mulan PSL v2. + You can use this software according to the terms and conditions of the Mulan PSL v2. + You may obtain a copy of Mulan PSL v2 at: + http://license.coscl.org.cn/MulanPSL2 + THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. + See the Mulan PSL v2 for more details. diff --git a/README.md b/README.md index b5f27a28a57e2079b6a37e45e37c6a7dd04b7a39..85d78394f0656bfff3bfe32fdadd5f68ab01b884 100644 --- a/README.md +++ b/README.md @@ -1,37 +1,70 @@ # NestOS #### 介绍 -NestOS is an automatically updated minimal operating system developed based on openEuler +NestOS是一款在openEuler社区CloudNative sig组孵化的云底座操作系统，专注于提供最佳的容器主机，大规模下安全的运行容器化工作负载。 -#### 软件架构 -软件架构说明 +Nest即“巢穴”，作为可以包容podman（鼹鼠）、iSulad（蚂蚁）、docker（集装箱）的“巢穴”，NestOS即化身为可以搭载iSulad、docker、podman等基础平台的操作系统。 +NestOS将配置工具ignition与rpm-ostree、OCI支持、SElinux强化等技术集成在一起，采用基于双系统分区、容器技术和集群架构的设计思路，搭载iSulad、docker、podman等主流容器基础平台，克服了由于用户修改系统内容、用户服务对系统组件依赖，以及系统软件包升级时不稳定中间态等种种导致升级过程不可靠的因素，最终以一种轻量级、定制化的操作系统呈现出来。可以适应各种不同的基础设施环境，并与OKD紧密集成，针对运行Kubernetes进行了优化，使系统具备十分便捷的集群组建能力。 -#### 安装教程 +![image-20211015170943884](graph/README/image-20211015170943884.png) -1. xxxx -2. xxxx -3. xxxx +#### 快速开始 +[在虚拟化平台部署-以VMware为例](https://gitee.com/openeuler/NestOS/blob/master/docs/%E5%BF%AB%E9%80%9F%E5%BC%80%E5%A7%8B.md) -#### 使用说明 +#### NestOS应用指南 +1. [rpm-ostree使用](https://gitee.com/openeuler/NestOS/blob/master/docs/rpm-ostree%E4%BD%BF%E7%94%A8.md) -1. xxxx -2. xxxx -3. xxxx +2. [k8s+iSulad搭建](https://gitee.com/openeuler/NestOS/blob/master/docs/K8S+iSulad%E6%90%AD%E5%BB%BA.md) -#### 参与贡献 +3. [zincati自动更新](https://gitee.com/openeuler/NestOS/blob/master/docs/zincati%E8%87%AA%E5%8A%A8%E6%9B%B4%E6%96%B0%E4%BD%BF%E7%94%A8.md) -1. Fork 本仓库 -2. 新建 Feat_xxx 分支 -3. 提交代码 -4. 新建 Pull Request +#### 功能特性 +1. 容器技术 +2. rpm-ostree +3. nestos-installer +4. Zincati +5. Cincinnati +6. ignition系统初始化 +7. Afterburn -#### 特技 +详细内容请点击[功能特性详细说明](https://gitee.com/openeuler/NestOS/blob/master/docs/%E5%8A%9F%E8%83%BD%E7%89%B9%E6%80%A7%E6%8F%8F%E8%BF%B0.md) -1. 使用 Readme\_XXX.md 来支持不同的语言，例如 Readme\_en.md, Readme\_zh.md -2. Gitee 官方博客 [blog.gitee.com](https://blog.gitee.com) -3. 你可以 [https://gitee.com/explore](https://gitee.com/explore) 这个地址来了解 Gitee 上的优秀开源项目 -4. [GVP](https://gitee.com/gvp) 全称是 Gitee 最有价值开源项目，是综合评定出的优秀开源项目 -5. Gitee 官方提供的使用手册 [https://gitee.com/help](https://gitee.com/help) -6. Gitee 封面人物是一档用来展示 Gitee 会员风采的栏目 [https://gitee.com/gitee-stars/](https://gitee.com/gitee-stars/) +#### 容器性能测试 + +使用NestOS 20211009版本，横向对比 docker，podman，iSulad 容器引擎性能。测试结果如下: + +| operator(ms) | Docker | Podman | iSulad | vs Docker | vs Podman | +| :----------: | :----: | :----: | :----: | :-------: | :-------: | +| 100*creat | 1745 | 22919 | 1122 | -36% | -95% | +| 100*start | 8561 | 8133 | 1561 | -82% | -81% | +| 100*stop | 1483 | 1445 | 296 | -80% | -80% | +| 100*rm | 1691 | 5286 | 741 | -56% | -86% | + +更多详细内容请[点击](https://gitee.com/openeu/NestOS/blob/master/docs/%E6%80%A7%E8%83%BD%E5%AF%B9%E6%AF%94%E6%B5%8B%E8%AF%95.md) + +#### FAQ + +1、NestOS与Fedora CoreOS是什么关系？ + +NestOS是基于Fedora CoreOS的衍生版本，后续我们将会结合openeuler社区的各项特性，独立维护发展，也欢迎各位伙伴在issue中提出自己的需求和意见。 + +2、config文件夹是什么？ + +config是制作NestOS镜像时使用的配置文件，由于现阶段没有跟随社区版本发布，所以该文件为我们根据openeuler定制的一份配置文件。配置文件中有一些注释，以及个别软件包引入与否，均为我们本地进行测试评估需要，现开源出来供大家交流学习使用。并且配置文件中所包含的软件包源、IP地址均为本地搭建。 + +#### 主要贡献者 + +| Gitee ID | 公司 | 邮箱 | +| :-----------: | :------: | :---------------------: | +| @duyiwei7w | 麒麟软件 | duyiwei@kylinos.cn | +| @ningjinnj | 麒麟软件 | ningjin@kylinos.cn | +| @shanph | 麒麟软件 | lishanfeng@kylinos.cn | +| @yahoohey | 麒麟软件 | chenjian@kylinos.cn | +| @wangyueliang | 麒麟软件 | wangyueliang@kylinos.cn | +| @fu-shanqing | 麒麟软件 | fushanqing@kylinos.cn | +| @kylinhao | 麒麟软件 | wanghao6@kylinos.cn | +| @duguhaotian | 华为 | liuhao27@huawei.com | + +欢迎感兴趣的小伙伴加入我们 \ No newline at end of file diff --git a/config/COPYING b/config/COPYING new file mode 100644 index 0000000000000000000000000000000000000000..660c8228beb2050c8ee1ab70192f9c474ab00c97 --- /dev/null +++ b/config/COPYING @@ -0,0 +1,21 @@ +Copyright 2021 NestOS Authors. + +Permission is hereby granted, free of charge, to any person obtaining +a copy of this software and associated documentation files (the +"Software"), to deal in the Software without restriction, including +without limitation the rights to use, copy, modify, merge, publish, +distribute, sublicense, and/or sell copies of the Software, and to +permit persons to whom the Software is furnished to do so, subject to +the following conditions: + +The above copyright notice and this permission notice shall be included +in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. +IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY +CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, +TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE +SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + diff --git a/config/image-base.yaml b/config/image-base.yaml new file mode 100644 index 0000000000000000000000000000000000000000..66e023a444d06d0c248bfcbe3f6acaeabedb6aa9 --- /dev/null +++ b/config/image-base.yaml @@ -0,0 +1,28 @@ +# This file is shared by all streams. For a stream-specific change, use +# image.yaml instead. + +# Target disk size in GB. +# Make it at least 10G because we want the rootfs to be at least 8G: +# https://github.com/coreos/fedora-coreos-tracker/issues/586 +size: 10 + +extra-kargs: + # Disable SMT on systems vulnerable to MDS or any similar future issue. + - mitigations=auto,nosmt + +# Disable networking by default on firstboot. We can drop this once cosa stops +# defaulting to `ip=dhcp,dhcp6 rd.neednet=1` when it doesn't see this key. +ignition-network-kcmdline: [] + +# Optional remote by which to prefix the deployed OSTree ref +ostree-remote: openEuler + +# We want read-only /sysroot to protect from unintentional damage. +# https://github.com/ostreedev/ostree/issues/1265 +sysroot-readonly: true + +# After this, we plan to add support for the Ignition +# storage/filesystems sections. (Although one can do +# that on boot as well) + + diff --git a/config/image.yaml b/config/image.yaml new file mode 100644 index 0000000000000000000000000000000000000000..9a5e77fdd9e8f92b05430a45ba9ce7d30148d939 --- /dev/null +++ b/config/image.yaml @@ -0,0 +1,5 @@ +# This file can optionally contain configuration specific to the stream, +# similarly to manifest.yaml. Unlike image-base.yaml, which is shared by all +# streams. +include: image-base.yaml +squashfs-compression: gzip diff --git a/config/live/EFI/openEuler/grub.cfg b/config/live/EFI/openEuler/grub.cfg new file mode 100644 index 0000000000000000000000000000000000000000..f208a7ff5070c207f15326d66fe952787dadbd70 --- /dev/null +++ b/config/live/EFI/openEuler/grub.cfg @@ -0,0 +1,34 @@ +# Note this file mostly matches the grub.cfg file from within the +# efiboot.img on the NestOS Server DVD iso. Diff this file with that +# file in the future to pick up changes. +# +# One diff to note is we use linux and initrd instead of linuxefi and +# initrdefi. We do this because it works and allows us to use this same +# file on other architecutres. +# +# This file gets embedded into the efiboot.img on our NestOS ISO. +set default="1" + +function load_video { + insmod efi_gop + insmod efi_uga + insmod video_bochs + insmod video_cirrus + insmod all_video +} + +load_video +set gfxpayload=keep +insmod gzio +insmod part_gpt +insmod ext2 + +set timeout=5 +### END /etc/grub.d/00_header ### + +### BEGIN /etc/grub.d/10_linux ### +menuentry 'NestOS (Live)' --class openeuler --class gnu-linux --class gnu --class os { + linux /images/pxeboot/vmlinuz @@KERNEL-ARGS@@ ignition.firstboot ignition.platform.id=metal +################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################ COREOS_KARG_EMBED_AREA + initrd /images/pxeboot/initrd.img /images/ignition.img +} diff --git a/config/live/README-devel.md b/config/live/README-devel.md new file mode 100644 index 0000000000000000000000000000000000000000..6316954340443ad6600e834578b0146d19c37ad0 --- /dev/null +++ b/config/live/README-devel.md @@ -0,0 +1,13 @@ +These files will be copied to the target live ISO +via the CoreOS Assembler buildextend-live call. It +picks up all files in the NestOS-config/live/ +directory and copies them to the base of the ISO. + +Files currently copied are: + +- isolinux/boot.msg +- isolinux/isolinux.cfg + +Files that get copied into efiboot.img in the ISO: + +- EFI/grub.cfg \ No newline at end of file diff --git a/config/live/isolinux/boot.msg b/config/live/isolinux/boot.msg new file mode 100644 index 0000000000000000000000000000000000000000..362e9e5ebe5e755aac832f8ac64e8da51fcbaeeb --- /dev/null +++ b/config/live/isolinux/boot.msg @@ -0,0 +1,5 @@ + +splash.lss + + - Press the 0107 key to boot. + diff --git a/config/live/isolinux/isolinux.cfg b/config/live/isolinux/isolinux.cfg new file mode 100644 index 0000000000000000000000000000000000000000..06159f6ef9976a6dbaa5cf29612240f57b3d1a64 --- /dev/null +++ b/config/live/isolinux/isolinux.cfg @@ -0,0 +1,75 @@ +# Note this file mostly matches the isolinux.cfg file from the NestOS +# Server DVD iso. Diff this file with that file in the future to pick up +# changes. +serial 0 +default vesamenu.c32 +# timeout in units of 1/10s. 50 == 5 seconds +timeout 50 + +display boot.msg + +# Clear the screen when exiting the menu, instead of leaving the menu displayed. +# For vesamenu, this means the graphical background is still displayed without +# the menu itself for as long as the screen remains in graphics mode. +menu clear +menu background splash.png +menu title NestOS +menu vshift 8 +menu rows 18 +menu margin 8 +#menu hidden +menu helpmsgrow 15 +menu tabmsgrow 13 + +# Border Area +menu color border * #00000000 #00000000 none + +# Selected item +menu color sel 0 #ffffffff #00000000 none + +# Title bar +menu color title 0 #ff7ba3d0 #00000000 none + +# Press [Tab] message +menu color tabmsg 0 #ff3a6496 #00000000 none + +# Unselected menu item +menu color unsel 0 #84b8ffff #00000000 none + +# Selected hotkey +menu color hotsel 0 #84b8ffff #00000000 none + +# Unselected hotkey +menu color hotkey 0 #ffffffff #00000000 none + +# Help text +menu color help 0 #ffffffff #00000000 none + +# A scrollbar of some type? Not sure. +menu color scrollbar 0 #ffffffff #ff355594 none + +# Timeout msg +menu color timeout 0 #ffffffff #00000000 none +menu color timeout_msg 0 #ffffffff #00000000 none + +# Command prompt text +menu color cmdmark 0 #84b8ffff #00000000 none +menu color cmdline 0 #ffffffff #00000000 none + +# Do not display the actual menu unless the user presses a key. All that is displayed is a timeout message. + +menu tabmsg Press Tab for full configuration options on menu items. + +menu separator # insert an empty line +menu separator # insert an empty line + +label linux + menu label ^NestOS (Live) + menu default + kernel /images/pxeboot/vmlinuz + append initrd=/images/pxeboot/initrd.img,/images/ignition.img @@KERNEL-ARGS@@ ignition.firstboot ignition.platform.id=metal +################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################ COREOS_KARG_EMBED_AREA + +menu separator # insert an empty line + +menu end diff --git a/config/live/zipl.prm b/config/live/zipl.prm new file mode 100644 index 0000000000000000000000000000000000000000..56a2c07e8930f56d1b15b5888724ffac9f80dc41 --- /dev/null +++ b/config/live/zipl.prm @@ -0,0 +1 @@ +@@KERNEL-ARGS@@ ignition.firstboot ignition.platform.id=metal diff --git a/config/manifest.yaml b/config/manifest.yaml new file mode 100644 index 0000000000000000000000000000000000000000..98db1880656703630c6e8bb8804d1ade66af2563 --- /dev/null +++ b/config/manifest.yaml @@ -0,0 +1,15 @@ +ref: openEuler/${basearch}/nestos/stable +include: manifests/nestos.yaml + +releasever: "LTS" + +rojig: + license: MIT + name: nestos + summary: NestOS stable + +repos: + - nestos + +add-commit-metadata: + fedora-coreos.stream: stable diff --git a/config/manifests/bootable-rpm-ostree.yaml b/config/manifests/bootable-rpm-ostree.yaml new file mode 100644 index 0000000000000000000000000000000000000000..ddc39387a9fa658e81020835a52580b163df4f5f --- /dev/null +++ b/config/manifests/bootable-rpm-ostree.yaml @@ -0,0 +1,31 @@ +# This minimal base starts just from: kernel + systemd + rpm-ostree + bootloader. +# The intent of this is to inherit from this if you are doing something highly +# custom that e.g. might not involve Ignition or podman, but you do want +# rpm-ostree. +# We expect most people though using coreos-assembler to inherit from +# fedora-coreos-base.yaml. +packages: + # Kernel + systemd. Note we explicitly specify kernel-{core,modules} + # because otherwise depsolving could bring in kernel-debug. + - kernel systemd +# - kernel-devel kernel-tools kernel-headers + # kernel-core kernel-modules + # rpm-ostree + - rpm-ostree nss-altfiles + # firmware updates + - fwupd + +# bootloader +packages-aarch64: + - grub2-efi-aa64 efibootmgr shim +packages-ppc64le: + - grub2 ostree-grub2 +packages-s390x: + # On Fedora, this is provided by s390utils-core. on RHEL, this is for now + # provided by s390utils-base, but soon will be -core too. + - /usr/sbin/zipl +packages-x86_64: + - grub2 efibootmgr shim +# grub2-efi-x64-cdboot grub2-efi-x64-modules grub2-tools-efi grub2-emu grub2-emu-modules + - microcode_ctl + - grub2-efi-x64 diff --git a/config/manifests/bootupd.yaml b/config/manifests/bootupd.yaml new file mode 100644 index 0000000000000000000000000000000000000000..a107a98dee59aa556cef9c3c1a3b10df067ef47a --- /dev/null +++ b/config/manifests/bootupd.yaml @@ -0,0 +1,16 @@ +# Integration with https://github.com/coreos/bootupd +# xref https://github.com/coreos/fedora-coreos-tracker/issues/510 +packages: + - bootupd + +postprocess: + - | + #!/bin/bash + set -xeuo pipefail + ## Until we have https://github.com/coreos/rpm-ostree/pull/2275 + mkdir -p /run + # Transforms /usr/lib/ostree-boot into a bootupd-compatible update payload + /usr/bin/bootupctl backend generate-update-metadata / + chmod -R +x /usr/bin/ + chmod -R +x /usr/sbin/ + chmod -R +x /usr/libexec/ diff --git a/config/manifests/file-transfer.yaml b/config/manifests/file-transfer.yaml new file mode 100644 index 0000000000000000000000000000000000000000..aa584dd02decf7b6da94f8d07fe4ed97ba717cb6 --- /dev/null +++ b/config/manifests/file-transfer.yaml @@ -0,0 +1,6 @@ +# Moving files around and verifying them +packages: + # - git-core + - git + - gnupg2 + - rsync diff --git a/config/manifests/group b/config/manifests/group new file mode 100644 index 0000000000000000000000000000000000000000..fac113c56a4b8d4f77d3d8925b01ce8bc523b650 --- /dev/null +++ b/config/manifests/group @@ -0,0 +1,58 @@ +root:x:0: +bin:x:1: +daemon:x:2: +sys:x:3: +adm:x:4: +tty:x:5: +disk:x:6: +lp:x:7: +mem:x:8: +kmem:x:9: +wheel:x:10: +cdrom:x:11: +mail:x:12: +man:x:15: +sudo:x:16: +dialout:x:18: +floppy:x:19: +games:x:20: +tape:x:30: +video:x:39: +ftp:x:50: +lock:x:54: +audio:x:63: +nobody:x:99: +users:x:100: +utmp:x:22: +utempter:x:35: +ssh_keys:x:999: +systemd-journal:x:190: +dbus:x:81: +polkitd:x:998: +etcd:x:997: +dip:x:40: +cgred:x:996: +tss:x:59: +avahi-autoipd:x:170: +rpc:x:32: +sssd:x:993: +dockerroot:x:986: +rpcuser:x:29: +nfsnobody:x:65534: +kube:x:994: +sshd:x:74: +chrony:x:992: +tcpdump:x:72: +ceph:x:167: +input:x:995: +systemd-timesync:x:991: +systemd-network:x:990: +systemd-resolve:x:989: +systemd-bus-proxy:x:988: +cockpit-ws:x:987: + +#duyiwei +named:x:25: +dhcpd:x:177: +dnsmasq:x:980: + diff --git a/config/manifests/grub2-removals.yaml b/config/manifests/grub2-removals.yaml new file mode 100644 index 0000000000000000000000000000000000000000..f4800ddd113edbd3c8244afbeb68c55fa5d82a2e --- /dev/null +++ b/config/manifests/grub2-removals.yaml @@ -0,0 +1,8 @@ +remove-from-packages: + # The grub bits are mainly designed for desktops, and IMO haven't seen + # enough testing in concert with ostree. At some point we'll flesh out + # the full plan in https://github.com/coreos/fedora-coreos-tracker/issues/47 + - [grub2-tools, /etc/grub.d/08_fallback_counting, + /etc/grub.d/10_reset_boot_success, + /etc/grub.d/12_menu_auto_hide, + /usr/lib/systemd/.*] diff --git a/config/manifests/ignition-and-ostree.yaml b/config/manifests/ignition-and-ostree.yaml new file mode 100644 index 0000000000000000000000000000000000000000..df57ea6736821f6581cc71f95bed64971518377e --- /dev/null +++ b/config/manifests/ignition-and-ostree.yaml @@ -0,0 +1,48 @@ +# Defines the "core" of a Fedora CoreOS like system; basically (ignition, ostree) +# plus other default tweaks. Things in this file should be something we expect +# basically everyone using both Ignition and (rpm-)ostree to want. +# It may be used as an inheritance base by other projects like Fedora Silverblue or RHCOS. +# One good model is to add fedora-coreos-config as a git submodule. See: +# https://github.com/coreos/coreos-assembler/pull/639 + +# Include rpm-ostree + kernel + bootloader +include: bootable-rpm-ostree.yaml + +initramfs-args: + # make it a hard error if Ignition can't be included + - --add=ignition + +# Modern defaults we want +boot-location: modules +tmp-is-dir: true + +# Required by Ignition, and makes the system not compatible with Anaconda +machineid-compat: false + +packages: + - ignition + - dracut-network + # for encryption + - clevis clevis-dracut clevis-systemd + # - clevis-luks +remove-from-packages: + # We don't want systemd-firstboot.service. It conceptually conflicts with + # Ignition. We also inject runtime bits to disable it in systemd-firstboot.service.d/fcos-disable.conf + # to make it easier to use systemd builds from git. + - [systemd, /usr/bin/systemd-firstboot, + /usr/lib/systemd/system/systemd-firstboot.service, + /usr/lib/systemd/system/sysinit.target.wants/systemd-firstboot.service] + # We don't want auto-generated mount units. See also + # https://github.com/systemd/systemd/issues/13099 + - [systemd-udev, /usr/lib/systemd/system-generators/systemd-gpt-auto-generator] + +postprocess: + # Undo RPM scripts enabling units; we want the presets to be canonical + # https://github.com/projectatomic/rpm-ostree/issues/1803 + - | + #!/usr/bin/env bash + set -xeuo pipefail + rm -rf /etc/systemd/system/* + systemctl preset-all + rm -rf /etc/systemd/user/* + systemctl --user --global preset-all diff --git a/config/manifests/nestos-base.yaml b/config/manifests/nestos-base.yaml new file mode 100644 index 0000000000000000000000000000000000000000..121d9bfcb577ec1ad6ad5ff1e375ee0faf1806c2 --- /dev/null +++ b/config/manifests/nestos-base.yaml @@ -0,0 +1,195 @@ +# This file is most of a NestOS like system; it inherits from "core". +# Add things in this file which are somewhat "opinionated", not necessarily +# core functionality. + +include: + - ignition-and-ostree.yaml + - file-transfer.yaml + - networking-tools.yaml + - system-configuration.yaml + - user-experience.yaml + +initramfs-args: + - --no-hostonly + # We don't support root on NFS, so we don't need it in the initramfs. It also + # conflicts with /var mount support in ignition because NFS tries to mount stuff + # in /var/ and then ignition can't cleanly unmount it. For example: + # https://github.com/dracutdevs/dracut/blob/1856ae95c873a6fe855b3dccd0144f1a96b9e71c/modules.d/95nfs/nfs-start-rpc.sh#L7 + # See also discussion in https://github.com/coreos/fedora-coreos-config/pull/60 + - --omit=nfs + # Omit these since we don't use them + - --omit=lvm + - --omit=iscsi + +# Be minimal +recommends: false + +ignore-removed-users: + - root +ignore-removed-groups: + - root +etc-group-members: + - wheel + - sudo + - systemd-journal + - adm + +check-passwd: + type: "file" + filename: "passwd" +check-groups: + type: "file" + filename: "group" + +default-target: multi-user.target + +# we can drop this when it's the rpm-ostree default +rpmdb: sqlite + +# ⚠⚠⚠ ONLY TEMPORARY HACKS ALLOWED HERE; ALL ENTRIES NEED TRACKER LINKS ⚠⚠⚠ +# See also the version of this in fedora-coreos.yaml +postprocess: + - | + #!/usr/bin/env bash + /usr/sbin/mpathconf --enable + + - | + #!/usr/bin/env bash + rm -rf /etc/gshadow + echo "u polkitd - polkitd" > /usr/lib/sysusers.d/polkit.conf + echo "u chrony - chrony" > /usr/lib/sysusers.d/chrony.conf + echo "u sshd - sshd" > /usr/lib/sysusers.d/sshd.conf + echo "u rpc - rpc" > /usr/lib/sysusers.d/rpc.conf + # This will be dropped once rpm-ostree because module-aware. + # https://github.com/projectatomic/rpm-ostree/issues/1542#issuecomment-419684977 + # https://github.com/projectatomic/rpm-ostree/issues/1435 + - | + #!/usr/bin/env bash + set -xeuo pipefail + for x in /etc/yum.repos.d/*modular.repo; do + sed -i -e 's,enabled=[01],enabled=0,' ${x} + done + + # Enable SELinux booleans used by OpenShift + # https://github.com/coreos/fedora-coreos-tracker/issues/284 + - | + #!/usr/bin/env bash + set -xeuo pipefail + setsebool -P -N virt_use_samba on # RHBZ#1754825 + + # Mask dnsmasq. We include dnsmasq for host services that use the dnsmasq + # binary but intentionally mask the systemd service so users can't easily + # use it as an external dns server. We prefer they use a container for that. + # https://github.com/coreos/fedora-coreos-tracker/issues/519 + - | + #!/usr/bin/env bash + systemctl mask dnsmasq.service + # Mask systemd-repart. Ignition is responsible for partition setup on first + # boot and does not use systemd-repart currently. See also + # https://github.com/coreos/fedora-coreos-tracker/issues/570 + - | + #!/usr/bin/env bash + systemctl mask systemd-repart.service + + # Neuter systemd-resolved for now. + # https://github.com/coreos/fedora-coreos-tracker/issues/649#issuecomment-743219353 + # Note: When removing this, we likely also want to remove + # coreos-reset-stub-resolv-selinux-context.{path,service} and their presets. + - | + #!/usr/bin/env bash + set -xeuo pipefail + # Get us back to Fedora 32's nsswitch.conf settings + sed -i 's/^hosts:.*/hosts: files dns myhostname/' /etc/nsswitch.conf + mkdir -p /usr/lib/systemd/resolved.conf.d/ + cat > /usr/lib/systemd/resolved.conf.d/nestos-stub-listener.conf <<'EOF' + # https://github.com/coreos/fedora-coreos-tracker/issues/649#issuecomment-736104003 + [Resolve] + DNSStubListener=no + EOF + + # Set the fallback hostname to `localhost`. This piggybacks on the + # postprocess script above which neuters systemd-resolved, because + # currently, a fallback hostname of `localhost` + systemd-resolved breaks + # rDNS. Eventually, we should be able to drop this at the same time as we drop + # the above. See: https://bugzilla.redhat.com/show_bug.cgi?id=1892235#c25 + - | + #!/usr/bin/env bash + source /etc/os-release + if [ -z "${DEFAULT_HOSTNAME:-}" ]; then + echo 'DEFAULT_HOSTNAME=localhost' >> /usr/lib/os-release + fi + +# Packages listed here should be specific to Fedore CoreOS (as in not yet +# available in RHCOS or not desired in RHCOS). All other packages should go +# into one of the sub-manifests listed at the top. +packages: + # Container tooling + - crun + - polkit + - afterburn-dracut + - ssh-key-dir + # Containers + - systemd-container catatonit + - fuse-overlayfs slirp4netns + # name resolution for podman containers + # https://github.com/coreos/fedora-coreos-tracker/issues/519 + - dnsmasq + - iSulad + # Remote IPC for podman + - libvarlink-util + # Minimal NFS client + - nfs-utils-coreos + # Active Directory support + - adcli + # Additional firewall support; we aren't including these in RHCOS or they + # don't exist in RHEL + # WireGuard https://github.com/coreos/fedora-coreos-tracker/issues/362 + - wireguard-tools + # Storage + - btrfs-progs + - WALinuxAgent-udev + # Allow communication between sudo and SSSD + # for caching sudo rules by SSSD. + # https://github.com/coreos/fedora-coreos-tracker/issues/445 + - libsss_sudo + # SSSD; we only ship a subset of the backends + - sssd-client sssd-ad sssd-ipa sssd-krb5 sssd-ldap + # Used by admins interactively + - attr + - openssl + - lsof + # Provides terminal tools like clear, reset, tput, and tset + - ncurses + # file-transfer: note fuse-sshfs is not in RHEL + # so we can't put it in file-transfer.yaml + - fuse-sshfs + # Improved MOTD experience + - console-login-helper-messages-motdgen + # i18n + - kbd + # nvme-cli for managing nvme disks + - nvme-cli + # zram-generator (but not zram-generator-defaults) for F33 change + # https://github.com/coreos/fedora-coreos-tracker/issues/509 + - zram-generator + +# This thing is crying out to be pulled into systemd, but that hasn't happened +# yet. Also we may want to add to rpm-ostree something like arch negation; +# basically right now it doesn't exist on s390x. +# Anyways, it was requested by the Red Hat perf team for RHCOS, so we have it here. +# https://serverfault.com/questions/513807/is-there-still-a-use-for-irqbalance-on-modern-hardware +# https://access.redhat.com/solutions/41535 +packages-x86_64: + - irqbalance +packages-ppc64le: + - irqbalance + - librtas + - powerpc-utils-core + - ppc64-diag-rtas +packages-aarch64: + - irqbalance + +# See https://github.com/coreos/bootupd +arch-include: + x86_64: bootupd.yaml + aarch64: bootupd.yaml diff --git a/config/manifests/nestos.yaml b/config/manifests/nestos.yaml new file mode 100644 index 0000000000000000000000000000000000000000..17086044fec29a597a4659997875f3c28257acb6 --- /dev/null +++ b/config/manifests/nestos.yaml @@ -0,0 +1,95 @@ + + +include: nestos-base.yaml + +automatic-version-prefix: "${releasever}..dev" +mutate-os-release: "${releasever}" + +packages: + - openEuler-release-nestos + - openEuler-repos-ostree + - openEuler-repos-modular + - openEuler-repos-archive + - docker-engine + - zincati + +etc-group-members: + # Add the docker group to /etc/group + # https://github.com/coreos/fedora-coreos-tracker/issues/2 + # This will be no longer needed when systemd-sysusers has been implemented: + # https://github.com/projectatomic/rpm-ostree/issues/49 + - docker + - isulad + - podman + +# XXX: this is used by coreos-assembler for artifact naming... +rojig: + license: MIT + name: nestos + summary: NestOS base image + +# ⚠⚠⚠ ONLY TEMPORARY HACKS ALLOWED HERE; ALL ENTRIES NEED TRACKER LINKS ⚠⚠⚠ +# See also the version of this in fedora-coreos-base.yaml +postprocess: + # Disable Zincati and fedora-coreos-pinger on non-release builds + # https://github.com/coreos/fedora-coreos-tracker/issues/212 + - | + #!/usr/bin/env bash + set -xeuo pipefail + source /etc/os-release + if [[ $OSTREE_VERSION = *.dev* ]]; then + mkdir -p /etc/nestos-pinger/config.d /etc/zincati/config.d + echo -e 'reporting.enabled = false' > /etc/nestos-pinger/config.d/95-disable-on-dev.toml + echo -e 'updates.enabled = false' > /etc/zincati/config.d/95-disable-on-dev.toml + fi + # Users shouldn't be configuring `rpm-ostreed.conf` + # https://github.com/coreos/fedora-coreos-tracker/issues/271 + - | + #!/usr/bin/env bash + set -xeuo pipefail + cat > /tmp/rpm-ostreed.conf << 'EOF' + # By default, this system has its OS updates managed by + # `zincati.service`. Changes made to this file may + # conflict with the configuation of `zincati.service`. + # See https://github.com/coreos/zincati for additional + # information. + + EOF + cat /usr/etc/rpm-ostreed.conf >> /tmp/rpm-ostreed.conf + cp /tmp/rpm-ostreed.conf /usr/etc/rpm-ostreed.conf + rm /tmp/rpm-ostreed.conf + +remove-from-packages: + # Drop NetworkManager support for ifcfg files, see also corresponding + # overlay.d/14NetworkManager-plugins + - [NetworkManager, /usr/lib64/NetworkManager/.*/libnm-settings-plugin-ifcfg-rh.so] + +remove-files: + # We don't ship man(1) or info(1) + - usr/share/info + - usr/share/man + # Drop text docs too + - usr/share/doc + +# Things we don't expect to ship on the host. We currently +# have recommends: false so these could only come in via +# hard requirement, in which case the build will fail. +exclude-packages: + - python + - python2 + - python2-libs + - python3-libs + - nodejs + - dnf + - cowsay + # Let's make sure initscripts doesn't get pulled back in + # https://github.com/coreos/fedora-coreos-tracker/issues/220#issuecomment-611566254 + # For (datacenter/cloud oriented) servers, we want to see the details by default. + # https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/HSMISZ3ETWQ4ETVLWZQJ55ARZT27AAV3/ + - plymouth + +# And remove some cruft from grub2 +arch-include: + x86_64: grub2-removals.yaml + aarch64: grub2-removals.yaml + ppc64le: grub2-removals.yaml diff --git a/config/manifests/networking-tools.yaml b/config/manifests/networking-tools.yaml new file mode 100644 index 0000000000000000000000000000000000000000..ee8e4084e6bfd642db2af4aeb6b78005ef0dcabb --- /dev/null +++ b/config/manifests/networking-tools.yaml @@ -0,0 +1,22 @@ +# This defines a set of tools that are useful for configuring, debugging, +# or manipulating the network of a system. It is desired to keep this list +# generic enough to be shared downstream with RHCOS. + +packages: + # Standard tools for configuring network/hostname + - NetworkManager hostname + # Interactive Networking configuration during coreos-install + - NetworkManager-tui + # Teaming https://github.com/coreos/fedora-coreos-config/pull/289 + # and http://bugzilla.redhat.com/1758162 + - NetworkManager-team teamd + # Support for cloud quirks and dynamic config in real rootfs: + # https://github.com/coreos/fedora-coreos-tracker/issues/320 + - NetworkManager-cloud-setup + # Route manipulation and QoS + - iproute iproute-tc + # Firewall manipulation + - iptables + - nftables + # Interactive network tools for admins + - socat net-tools bind-utils diff --git a/config/manifests/passwd b/config/manifests/passwd new file mode 100644 index 0000000000000000000000000000000000000000..b05ebdbfe8f608c1bb430d287efc4888442426fa --- /dev/null +++ b/config/manifests/passwd @@ -0,0 +1,33 @@ +root:x:0:0:root:/root:/bin/bash +bin:x:1:1:bin:/bin:/sbin/nologin +daemon:x:2:2:daemon:/sbin:/sbin/nologin +adm:x:3:4:adm:/var/adm:/sbin/nologin +lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin +sync:x:5:0:sync:/sbin:/bin/sync +shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown +halt:x:7:0:halt:/sbin:/sbin/halt +mail:x:8:12:mail:/var/spool/mail:/sbin/nologin +operator:x:11:0:operator:/root:/sbin/nologin +games:x:12:100:games:/usr/games:/sbin/nologin +ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin +nobody:x:99:99:Nobody:/:/sbin/nologin +dbus:x:81:81:System message bus:/:/sbin/nologin +polkitd:x:999:998:User for polkitd:/:/sbin/nologin +etcd:x:998:997:etcd user:/var/lib/etcd:/sbin/nologin +tss:x:59:59:Account used by the trousers package to sandbox the tcsd daemon:/dev/null:/sbin/nologin +avahi-autoipd:x:170:170:Avahi IPv4LL Stack:/var/lib/avahi-autoipd:/sbin/nologin +rpc:x:32:32:Rpcbind Daemon:/var/lib/rpcbind:/sbin/nologin +sssd:x:995:993:User for sssd:/:/sbin/nologin +dockerroot:x:997:986:Docker User:/var/lib/docker:/sbin/nologin +rpcuser:x:29:29:RPC Service User:/var/lib/nfs:/sbin/nologin +nfsnobody:x:65534:65534:Anonymous NFS User:/var/lib/nfs:/sbin/nologin +kube:x:996:994:Kubernetes user:/:/sbin/nologin +sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin +chrony:x:994:992::/var/lib/chrony:/sbin/nologin +tcpdump:x:72:72::/:/sbin/nologin +ceph:x:167:167:Ceph daemons:/var/lib/ceph:/sbin/nologin +systemd-timesync:x:993:991:systemd Time Synchronization:/:/sbin/nologin +systemd-network:x:991:990:systemd Network Management:/:/sbin/nologin +systemd-resolve:x:990:989:systemd Resolver:/:/sbin/nologin +systemd-bus-proxy:x:989:988:systemd Bus Proxy:/:/sbin/nologin +cockpit-ws:x:988:987:User for cockpit-ws:/:/sbin/nologin diff --git a/config/manifests/system-configuration.yaml b/config/manifests/system-configuration.yaml new file mode 100644 index 0000000000000000000000000000000000000000..295820113b3e3adbf47279e3cd55481b4ed7e580 --- /dev/null +++ b/config/manifests/system-configuration.yaml @@ -0,0 +1,39 @@ +# These are packages that are related to configuring parts of the system. +# It is intended to be kept generic so that it may be shared downstream with +# RHCOS. + +packages: + # Configuring SSH keys, cloud provider check-in, etc + - afterburn + # NTP support + - chrony + # Installing CoreOS itself + - coreos-installer coreos-installer-bootinfra + # Storage configuration/management + ## cloud-utils-growpart - For growing root partition + - cifs-utils + - cloud-utils-growpart + - cryptsetup + - device-mapper-multipath + - e2fsprogs + - iscsi-initiator-utils + - lvm2 + - mdadm + - sg3_utils + - xfsprogs + # User configuration + - passwd + - shadow-utils + # SELinux policy + - selinux-policy-targeted + # There are things that write outside of the journal still (such as the + # classic wtmp, etc.). auditd also writes outside the journal but it has its + # own log rotation. + # Anything package layered will also tend to expect files dropped in + # /etc/logrotate.d to work. Really, this is a legacy thing, but if we don't + # have it then people's disks will slowly fill up with logs. + - logrotate + # Boost starving threads + # https://github.com/coreos/fedora-coreos-tracker/issues/753 + - stalld + - bc diff --git a/config/manifests/user-experience.yaml b/config/manifests/user-experience.yaml new file mode 100644 index 0000000000000000000000000000000000000000..6cd01ef026b05cf2dcf7c63d31bbf6f07fbded0f --- /dev/null +++ b/config/manifests/user-experience.yaml @@ -0,0 +1,35 @@ +# These packages are either widely used utilities/services or +# are targeted for improving the general CoreOS user experience. +# It is intended to be kept generic so that it may be shared downstream with +# RHCOS. + +packages: + # Basic user tools + ## jq - parsing/interacting with JSON data + - bash-completion + - wget + - coreutils + - jq + - less + - sudo + - vim-minimal + # File compression/decompression + ## bsdtar - dependency of 35coreos-live dracut module + - bsdtar + - bzip2 + - gzip + - tar + - xz + # Improved MOTD experience + - console-login-helper-messages-issuegen + - console-login-helper-messages-profile + # kdump support + # https://github.com/coreos/fedora-coreos-tracker/issues/622 + - kexec-tools + # Remote Access + - openssh-clients openssh-server + # Container tooling + - podman + - docker-runc + - skopeo + - toolbox diff --git a/config/nestos-pool.repo b/config/nestos-pool.repo new file mode 100644 index 0000000000000000000000000000000000000000..ae37c274546389f992b5e9253b2883a5e26e9bd1 --- /dev/null +++ b/config/nestos-pool.repo @@ -0,0 +1,10 @@ +[nestos] +name= extra repository - $basearch +baseurl=http://10.1.110.88/nestos/nestos_x86 +enabled=1 +#repo_gpgcheck=0 +type=rpm-md +gpgcheck=0 +#skip_if_unavailable=True + + diff --git a/config/overlay.d/05core/etc/security/pwquality.conf.d/20-disable-dict.conf b/config/overlay.d/05core/etc/security/pwquality.conf.d/20-disable-dict.conf new file mode 100644 index 0000000000000000000000000000000000000000..e946311eaddae3acc9623deade031a1d405f30c6 --- /dev/null +++ b/config/overlay.d/05core/etc/security/pwquality.conf.d/20-disable-dict.conf @@ -0,0 +1,3 @@ +# We don't ship cracklib dicts, so don't try to use them to validate +# password changes. +dictcheck = 0 diff --git a/config/overlay.d/05core/etc/sudoers.d/coreos-sudo-group b/config/overlay.d/05core/etc/sudoers.d/coreos-sudo-group new file mode 100644 index 0000000000000000000000000000000000000000..2b3669d4df8ac54124c8a7a419d98979f37e2c91 --- /dev/null +++ b/config/overlay.d/05core/etc/sudoers.d/coreos-sudo-group @@ -0,0 +1,2 @@ +# https://github.com/openshift/os/issues/96 +%sudo ALL=(ALL) NOPASSWD: ALL diff --git a/config/overlay.d/05core/usr/lib/NetworkManager/conf.d/20-client-id-from-mac.conf b/config/overlay.d/05core/usr/lib/NetworkManager/conf.d/20-client-id-from-mac.conf new file mode 100644 index 0000000000000000000000000000000000000000..320ea4a460e3d381e0151f1981cd6525ed20e274 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/NetworkManager/conf.d/20-client-id-from-mac.conf @@ -0,0 +1,2 @@ +[connection] +ipv4.dhcp-client-id=mac diff --git a/config/overlay.d/05core/usr/lib/coreos/generator-lib.sh b/config/overlay.d/05core/usr/lib/coreos/generator-lib.sh new file mode 100644 index 0000000000000000000000000000000000000000..b133e5ac67def5d08052fa0126c406b00035434a --- /dev/null +++ b/config/overlay.d/05core/usr/lib/coreos/generator-lib.sh @@ -0,0 +1,19 @@ +# File intended to be sourced by shell script generators shipped with CoreOS systems + +# Generators don't have logging right now +# https://github.com/systemd/systemd/issues/15638 +exec 1>/dev/kmsg; exec 2>&1 + +UNIT_DIR="${1:-/tmp}" + +have_karg() { + local arg="$1" + local cmdline=( $( "$initdir/etc/sysctl.d/10-dont-ratelimit-kmsg.conf" +} diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/25coreos-azure-udev/module-setup.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/25coreos-azure-udev/module-setup.sh new file mode 100644 index 0000000000000000000000000000000000000000..49ece8bea3363b3d46a9217877456eb269b387e0 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/25coreos-azure-udev/module-setup.sh @@ -0,0 +1,17 @@ +#!/bin/bash +# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*- +# ex: ts=8 sw=4 sts=4 et filetype=sh + +# We want to provide Azure udev rules as part of the initrd, so that Ignition +# is able to detect disks and act on them. +# +# If the WALinuxAgent-udev package is changed to install the udev rules as +# part of the initramfs, we should drop this module. +# + + +install() { + inst_multiple \ + /usr/lib/udev/rules.d/66-azure-storage.rules \ + /usr/lib/udev/rules.d/99-azure-product-uuid.rules +} diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-ignition/coreos-boot-edit.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-ignition/coreos-boot-edit.service new file mode 100644 index 0000000000000000000000000000000000000000..b51059f008bc1eb4723eef4e94152e928e238af9 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-ignition/coreos-boot-edit.service @@ -0,0 +1,26 @@ +# This unit will run late in the initrd process after Ignition is completed +# successfully and temporarily mount /boot read-write to make edits +# (e.g. removing firstboot networking configuration files if necessary). + +[Unit] +Description=CoreOS Boot Edit +ConditionPathExists=/usr/lib/initrd-release +OnFailure=emergency.target +OnFailureJobMode=isolate + +# Since we are mounting /boot, require the device first. This isn't strictly +# necessary since we run late, but on principle let's make clear the dependency. +Requires=dev-disk-by\x2dlabel-boot.device +After=dev-disk-by\x2dlabel-boot.device +# Start after Ignition has finished +After=ignition-files.service +# As above, this isn't strictly necessary, but on principle. +After=coreos-multipath-wait.target + +[Service] +Type=oneshot +ExecStart=/usr/sbin/coreos-boot-edit +RemainAfterExit=yes +# MountFlags=slave is so the umount of /boot is guaranteed to happen. +# /boot will only be mounted for the lifetime of the unit. +MountFlags=slave diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-ignition/coreos-boot-edit.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-ignition/coreos-boot-edit.sh new file mode 100644 index 0000000000000000000000000000000000000000..3b16813b46433afbcca237b7f3d1cdd228bee019 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-ignition/coreos-boot-edit.sh @@ -0,0 +1,35 @@ +#!/bin/bash +set -euo pipefail + +# For a description of how this is used, see `coreos-boot-edit.service`. + +cmdline=( $(/dev/kmsg; exec 2>&1 + +UNIT_DIR="${1:-/tmp}" + +cmdline=( $( "${UNIT_DIR}/coreos-ignition-setup-user.service.d/diskful.conf" </dev/null || . /lib/dracut-lib.sh + type ip_to_var &>/dev/null || . /lib/net-lib.sh + set -euo pipefail +} + +dracut_func() { + # dracut is not friendly to set -eu + set +euo pipefail + "$@"; local rc=$? + set -euo pipefail + return $rc +} + +# Determine if the generated NM connection profiles match the default +# that would be given to us if the user had provided no additional +# configuration. i.e. did the user give us any network configuration +# other than the default? We determine this by comparing the generated +# output of nm-initrd-generator with a new run of nm-initrd-generator. +# If it matches then it was the default, if not then the user provided +# something extra. +are_default_NM_configs() { + # Make two dirs for storing files to use in the comparison + mkdir -p /run/coreos-teardown-initramfs/connections-compare-{1,2} + # Make another that's just a throwaway for the initrd-data-dir + mkdir -p /run/coreos-teardown-initramfs/initrd-data-dir + # Copy over the previously generated connection(s) profiles + cp /run/NetworkManager/system-connections/* \ + /run/coreos-teardown-initramfs/connections-compare-1/ + # Do a new run with the default input + /usr/libexec/nm-initrd-generator \ + -c /run/coreos-teardown-initramfs/connections-compare-2 \ + -i /run/coreos-teardown-initramfs/initrd-data-dir -- ip=dhcp,dhcp6 + # remove unique identifiers from the files (so our diff can work) + sed -i '/^uuid=/d' /run/coreos-teardown-initramfs/connections-compare-{1,2}/* + # currently the output will differ based on whether rd.neednet=1 + # was part of the kargs. Let's ignore the single difference (wait-device-timeout) + sed -i '/^wait-device-timeout=/d' /run/coreos-teardown-initramfs/connections-compare-{1,2}/* + if diff -r -q /run/coreos-teardown-initramfs/connections-compare-{1,2}/; then + rc=0 # They are the default configs + else + rc=1 # They are not the defaults, user must have added configuration + fi + rm -rf /run/coreos-teardown-initramfs + return $rc +} + +# Propagate initramfs networking if desired. The policy here is: +# +# - If a networking configuration was provided before this point +# (most likely via Ignition) and exists in the real root then +# we do nothing and don't propagate any initramfs networking. +# - If a user did not provide any networking configuration +# then we'll propagate the initramfs networking configuration +# into the real root, but only if it's different than the NM +# defaults (trying dhcp/dhcp6 on everything). If it's just the +# defaults then we want to avoid a slight behavior diff between +# propagating configs and just booting with no configuration. See +# +# + +propagate_initramfs_networking() { + # Check for any real root config in the two locations where a user could have + # provided network configuration. On FCOS we only support keyfiles, but on RHCOS + # we support keyfiles and ifcfg + if [ -n "$(ls -A /sysroot/etc/NetworkManager/system-connections/)" -o \ + -n "$(ls -A /sysroot/etc/sysconfig/network-scripts/)" ]; then + echo "info: networking config is defined in the real root" + realrootconfig=1 + else + echo "info: no networking config is defined in the real root" + realrootconfig=0 + fi + + # Did the user tell us to force initramfs networking config + # propagation even if real root networking config exists? + # Hopefully we only need this in rare circumstances. + + forcepropagate=0 + if dracut_func getargbool 0 'coreos.force_persist_ip'; then + forcepropagate=1 + echo "info: coreos.force_persist_ip detected: will force network config propagation" + fi + + if [ $realrootconfig == 1 -a $forcepropagate == 0 ]; then + echo "info: will not attempt to propagate initramfs networking" + fi + + if [ $realrootconfig == 0 -o $forcepropagate == 1 ]; then + if [ -n "$(ls -A /run/NetworkManager/system-connections/)" ]; then + if are_default_NM_configs; then + echo "info: skipping propagation of default networking configs" + else + echo "info: propagating initramfs networking config to the real root" + cp -v /run/NetworkManager/system-connections/* /sysroot/etc/NetworkManager/system-connections/ + coreos-relabel /etc/NetworkManager/system-connections/ + fi + else + echo "info: no initramfs networking information to propagate" + fi + fi +} + +# Propagate the ip= karg hostname if desired. The policy here is: +# +# - IF a hostname was detected in ip= kargs by NetworkManager +# - AND no hostname was set via Ignition (realroot `/etc/hostname`) +# - THEN we make the hostname detected by NM apply permanently +# by writing it into `/etc/hostname` +# +propagate_initramfs_hostname() { + if [ -e '/sysroot/etc/hostname' ]; then + echo "info: hostname is defined in the real root" + echo "info: will not attempt to propagate initramfs hostname" + return 0 + fi + + # If any hostname was provided NetworkManager will write it out to + # /run/NetworkManager/initrd/hostname. See + # https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/481 + if [ -s /run/NetworkManager/initrd/hostname ]; then + hostname=$( /sysroot/etc/hostname + coreos-relabel /etc/hostname + else + echo "info: no initramfs hostname information to propagate" + fi +} + +down_interface() { + echo "info: taking down network device: $1" + # On recommendation from the NM team let's try to delete the device + # first and if that doesn't work then set it to down and flush any + # associated addresses. Deleting virtual devices (bonds, teams, bridges, + # ip-tunnels, etc) will clean up any associated kernel resources. A real + # device can't be deleted so that will fail and we'll fallback to setting + # it down and flushing addresses. + if ! ip link delete $1; then + ip link set $1 down + ip addr flush dev $1 + fi +} + +# Iterate through the interfaces in the machine and take them down. +# Note that in the futre we would like to possibly use `nmcli` networking off` +# for this. See the following two comments for details: + +down_interfaces() { + if ! [ -z "$(ls /sys/class/net)" ]; then + for f in /sys/class/net/*; do + interface=$(basename "$f") + # The `bonding_masters` entry is not a true interface and thus + # cannot be taken down. Also skip local loopback + case "$interface" in + "lo" | "bonding_masters") + continue + ;; + esac + # When we start taking down devices some other devices can + # start to disappear (for example vlan on top of interface). + # If the device we're about to take down has disappeared + # since the start of this loop then skip taking it down. + if [ ! -e "$f" ]; then + echo "info: skipping teardown of ${interface}; no longer exists." + continue + fi + down_interface $interface + done + fi +} + +main() { + # Load libraries from dracut + load_dracut_libs + + # Take down all interfaces set up in the initramfs + down_interfaces + + # Clean up all routing + echo "info: flushing all routing" + ip route flush table main + ip route flush cache + + # Hopefully our logic is sound enough that this is never needed, but + # user's can explicitly disable initramfs network/hostname propagation + # with the coreos.no_persist_ip karg. + if dracut_func getargbool 0 'coreos.no_persist_ip'; then + echo "info: coreos.no_persist_ip karg detected" + echo "info: skipping propagating initramfs settings" + else + propagate_initramfs_hostname + propagate_initramfs_networking + fi + + # Now that the configuration has been propagated (or not) + # clean it up so that no information from outside of the + # real root is passed on to NetworkManager in the real root + rm -rf /run/NetworkManager/ +} + +main diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-ignition/module-setup.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-ignition/module-setup.sh new file mode 100644 index 0000000000000000000000000000000000000000..52abd13864936295ee9b2ccc8eccc3c6da9127ee --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-ignition/module-setup.sh @@ -0,0 +1,59 @@ +#!/bin/bash +# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*- +# ex: ts=8 sw=4 sts=4 et filetype=sh + +depends() { + echo systemd network ignition coreos-live +} + +install_ignition_unit() { + local unit="$1"; shift + local target="${1:-ignition-complete.target}"; shift + local instantiated="${1:-$unit}"; shift + inst_simple "$moddir/$unit" "$systemdsystemunitdir/$unit" + # note we `|| exit 1` here so we error out if e.g. the units are missing + + systemctl -q --root="$initdir" add-requires "$target" "$instantiated" || exit 1 +} + +install() { + inst_multiple \ + basename \ + diff \ + lsblk \ + sed \ + sgdisk + + inst_simple "$moddir/coreos-diskful-generator" \ + "$systemdutildir/system-generators/coreos-diskful-generator" + + inst_script "$moddir/coreos-gpt-setup.sh" \ + "/usr/sbin/coreos-gpt-setup" + + inst_script "$moddir/coreos-ignition-setup-user.sh" \ + "/usr/sbin/coreos-ignition-setup-user" + + # For consistency tear down the network and persist multipath between the initramfs and + # real root. + inst_script "$moddir/coreos-teardown-initramfs.sh" \ + "/usr/sbin/coreos-teardown-initramfs" + install_ignition_unit coreos-teardown-initramfs.service + + # units only started when we have a boot disk + # path generated by systemd-escape --path /dev/disk/by-label/root + install_ignition_unit coreos-gpt-setup.service ignition-diskful.target + + # dracut inst_script doesn't allow overwrites and we are replacing + # the default script placed by Ignition + binpath="/usr/sbin/ignition-kargs-helper" + cp "$moddir/coreos-kargs.sh" "$initdir$binpath" + install_ignition_unit coreos-kargs-reboot.service + + inst_script "$moddir/coreos-boot-edit.sh" \ + "/usr/sbin/coreos-boot-edit" + # Only start when the system has disks since we are editing /boot. + install_ignition_unit "coreos-boot-edit.service" \ + "ignition-diskful.target" + + install_ignition_unit coreos-ignition-setup-user.service +} diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/coreos-live-clear-sssd-cache.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/coreos-live-clear-sssd-cache.service new file mode 100644 index 0000000000000000000000000000000000000000..758bb617af64226f0f263d252603dc8853adea6d --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/coreos-live-clear-sssd-cache.service @@ -0,0 +1,18 @@ +# SSSD caches passwd data from /etc in /var. If we have a persistent /var +# but not a persistent /etc, ignition-files.service can think a user +# already exists when in fact it needs to be (re-)created. Clear the +# cache to avoid this. + +[Unit] +Description=Clear SSSD NSS cache in persistent /var +DefaultDependencies=false +ConditionPathExists=/run/ostree-live +ConditionPathExists=/sysroot/var/lib/sss/mc + +After=ignition-mount.service +Before=ignition-files.service + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/usr/bin/rm -r /sysroot/var/lib/sss/mc diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/coreos-live-unmount-tmpfs-var.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/coreos-live-unmount-tmpfs-var.service new file mode 100644 index 0000000000000000000000000000000000000000..de5080f611e367687994abe3b9e0e6f6be311169 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/coreos-live-unmount-tmpfs-var.service @@ -0,0 +1,24 @@ +# If the user specified a persistent /var, ideally it'd just be mounted +# overtop of our tmpfs /var and everything would be fine. That works +# fine in the initramfs, where ignition-mount handles the mounting. +# But in the real root, the user's mount unit is ignored by systemd, +# since there's already a filesystem mounted on /var. To fix this, we +# notice that the user wants to mount /var, and unmount our tmpfs /var +# before switching roots. + +[Unit] +Description=Unmount live /var if persistent /var is configured +DefaultDependencies=false +ConditionPathExists=/run/ostree-live +ConditionPathExists=|/sysroot/etc/systemd/system/var.mount +ConditionPathExists=|/sysroot/etc/fstab +Before=initrd-switch-root.target + +# Run after Ignition mounts are unmounted, since the Ignition config +# presumably mounted overtop /sysroot/var +After=ignition-mount.service + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/usr/sbin/coreos-live-unmount-tmpfs-var diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/coreos-live-unmount-tmpfs-var.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/coreos-live-unmount-tmpfs-var.sh new file mode 100644 index 0000000000000000000000000000000000000000..9b61d89914cbfea4846fe0b31e65a81ebd3ccec3 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/coreos-live-unmount-tmpfs-var.sh @@ -0,0 +1,33 @@ +#!/bin/bash +# If the user specified a persistent /var, ideally it'd just be mounted +# overtop of our tmpfs /var and everything would be fine. That works +# fine in the initramfs, where ignition-mount handles the mounting. +# But in the real root, the user's mount unit is ignored by systemd, +# since there's already a filesystem mounted on /var. To fix this, we +# notice that the user wants to mount /var, and unmount our tmpfs /var +# before switching roots. + +set -euo pipefail + +should_unmount() { + # Did the user specify a mount unit for /var? + if [ -e /sysroot/etc/systemd/system/var.mount ]; then + return 0 + fi + + # Is there an fstab entry for /var? + if [ -e /sysroot/etc/fstab ]; then + # Uncommented entry with mountpoint on /var, without noauto in options + result=$(awk '(! /^\s*#/) && ($2 == "/var") && ($4 !~ /noauto/) {print "found"}' /sysroot/etc/fstab) + if [ -n "$result" ]; then + return 0 + fi + fi + + return 1 +} + +if should_unmount; then + echo "Unmounting /sysroot/var" + umount /sysroot/var +fi diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/coreos-liveiso-persist-osmet.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/coreos-liveiso-persist-osmet.service new file mode 100644 index 0000000000000000000000000000000000000000..95684102df110e0c98149a622341af17a614dbd0 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/coreos-liveiso-persist-osmet.service @@ -0,0 +1,15 @@ +[Unit] +Description=Persist osmet files (ISO) +DefaultDependencies=false +ConditionPathExists=/run/ostree-live +ConditionKernelCommandLine=coreos.liveiso +RequiresMountsFor=/run/media/iso +Before=initrd-switch-root.target + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/usr/bin/mkdir -p /run/coreos-installer/osmet +# bsdtar reads cpio archives, and unlike cpio(1L), knows how to seek over +# members it isn't reading +ExecStart=/usr/bin/bsdtar -x -C /run/coreos-installer/osmet -f /run/media/iso/images/pxeboot/rootfs.img *.osmet diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/coreos-livepxe-persist-osmet.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/coreos-livepxe-persist-osmet.service new file mode 100644 index 0000000000000000000000000000000000000000..17484e66a1ee6df574dbec648dbca5ba016688cd --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/coreos-livepxe-persist-osmet.service @@ -0,0 +1,14 @@ +[Unit] +Description=Persist osmet files (PXE) +DefaultDependencies=false +ConditionPathExists=/run/ostree-live +ConditionKernelCommandLine=!coreos.liveiso +# Downloads and unpacks the osmet files if not already appended +After=coreos-livepxe-rootfs.service +Before=initrd-switch-root.target + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/usr/bin/mkdir -p /run/coreos-installer/osmet +ExecStart=/usr/bin/sh -c "if ls /*.osmet &>/dev/null; then cp /*.osmet /run/coreos-installer/osmet; fi" diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/coreos-livepxe-rootfs.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/coreos-livepxe-rootfs.service new file mode 100644 index 0000000000000000000000000000000000000000..ed935ba162a7e7ea7e0f1374c4f55dc1f7df7f92 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/coreos-livepxe-rootfs.service @@ -0,0 +1,21 @@ +[Unit] +Description=Acquire live PXE rootfs image +DefaultDependencies=false +ConditionPathExists=/usr/lib/initrd-release +ConditionPathExists=/run/ostree-live +ConditionKernelCommandLine=!coreos.liveiso + +After=basic.target +# Network is enabled here +After=nm-run.service +# compat: remove when everyone is on dracut 053+ +After=dracut-initqueue.service + +# If we fail, the boot will fail. Be explicit about it. +OnFailure=emergency.target +OnFailureJobMode=isolate + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/usr/sbin/coreos-livepxe-rootfs diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/coreos-livepxe-rootfs.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/coreos-livepxe-rootfs.sh new file mode 100644 index 0000000000000000000000000000000000000000..483587efa5629c3e888ee08fe0c90544e35c6a18 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/coreos-livepxe-rootfs.sh @@ -0,0 +1,67 @@ +#!/bin/bash +# Ensure that a PXE-booted system has a valid rootfs. + +set -euo pipefail + +# Get rootfs_url karg +set +euo pipefail +. /usr/lib/dracut-lib.sh +rootfs_url=$(getarg coreos.live.rootfs_url=) +set -euo pipefail + +if [[ -f /etc/coreos-live-rootfs ]]; then + # rootfs image was injected via PXE. Verify that the initramfs and + # rootfs versions match. + initramfs_ver=$(cat /etc/coreos-live-initramfs) + rootfs_ver=$(cat /etc/coreos-live-rootfs) + if [[ $initramfs_ver != $rootfs_ver ]]; then + echo "Found initramfs version $initramfs_ver but rootfs version $rootfs_ver." >&2 + echo "Please fix your PXE configuration." >&2 + exit 1 + fi +elif [[ -n "${rootfs_url}" ]]; then + # rootfs URL was provided as karg. Fetch image, check its hash, and + # unpack it. + echo "Fetching rootfs image from ${rootfs_url}..." + if [[ ${rootfs_url} != http:* && ${rootfs_url} != https:* ]]; then + # Don't commit to supporting protocols we might not want to expose in + # the long term. + echo "coreos.live.rootfs_url= supports HTTP and HTTPS only." >&2 + echo "Please fix your PXE configuration." >&2 + exit 1 + fi + + # First, reach out to the server to verify connectivity before + # trying to download and pipe content through other programs. + # Doing this allows us to retry all errors (including transient + # "no route to host" errors during startup). Note we can't use + # curl's --retry-all-errors here because it's not in el8's curl yet. + # We retry forever, matching Ignition's semantics. + curl_common_args="--silent --show-error --insecure --location" + while ! curl --head $curl_common_args "${rootfs_url}" >/dev/null; do + echo "Couldn't establish connectivity with the server specified by coreos.live.rootfs_url=" >&2 + echo "Retrying in 5s..." >&2 + sleep 5 + done + + # We don't need to verify TLS certificates because we're checking the + # image hash. + # bsdtar can read cpio archives and we already depend on it for + # coreos-liveiso-persist-osmet.service, so use it instead of cpio. + # We shouldn't need a --retry here since we've just successfully HEADed the + # file, but let's add one just to be safe (e.g. if the connection just went + # online and flickers or something). + if ! curl $curl_common_args --retry 5 "${rootfs_url}" | \ + rdcore stream-hash /etc/coreos-live-want-rootfs | \ + bsdtar -xf - -C / ; then + echo "Couldn't fetch, verify, and unpack image specified by coreos.live.rootfs_url=" >&2 + echo "Check that the URL is correct and that the rootfs version matches the initramfs." >&2 + exit 1 + fi +else + # Nothing. Fail. + echo "No rootfs image found. Modify your PXE configuration to add the rootfs" >&2 + echo "image as a second initrd, or use the coreos.live.rootfs_url= kernel parameter" >&2 + echo "to specify an HTTP or HTTPS URL to the rootfs." >&2 + exit 1 +fi diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/is-live-image.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/is-live-image.sh new file mode 100644 index 0000000000000000000000000000000000000000..318ad0beafcea0ac9f23b4153ef0e3f6fe48357d --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/is-live-image.sh @@ -0,0 +1,6 @@ +#!/bin/sh +# Script invoked by ignition-dracut generator to detect whether this is a +# live system without a root device. We can't test for /run/ostree-live +# because it's created by a generator. +# This file is created by coreos-assembler buildextend-live. +test -f /etc/coreos-live-initramfs diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/live-generator b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/live-generator new file mode 100644 index 0000000000000000000000000000000000000000..3f137a5927534832f64e63d454c615a118b43fcf --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-live/live-generator @@ -0,0 +1,223 @@ +#!/bin/bash +# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*- +# ex: ts=8 sw=4 sts=4 et filetype=sh + +# Generators don't have logging right now +# https://github.com/systemd/systemd/issues/15638 +exec 1>/dev/kmsg; exec 2>&1 + +command -v getarg >/dev/null || . /usr/lib/dracut-lib.sh + +set -e + +UNIT_DIR="${1:-/tmp}" + +add_requires() { + local name="$1"; shift + local target="$1"; shift + local requires_dir="${UNIT_DIR}/${target}.requires" + mkdir -p "${requires_dir}" + ln -sf "../${name}" "${requires_dir}/${name}" +} + +if ! is-live-image; then + exit 0 +fi + +# Create stamp file that everything else should use to detect a live boot +> /run/ostree-live + +add_requires sysroot.mount initrd-root-fs.target +add_requires sysroot-etc.mount initrd-root-fs.target +add_requires sysroot-var.mount initrd-root-fs.target + +mkdir -p "${UNIT_DIR}/ostree-prepare-root.service.d" +cat > "${UNIT_DIR}/ostree-prepare-root.service.d/10-live.conf" <"${UNIT_DIR}/sysroot.mount" < "${initrd_rootdev_target_d}/50-root-device.conf" <"${UNIT_DIR}/run-media-iso.mount" <"${UNIT_DIR}/sysroot.mount" <"${UNIT_DIR}/sysroot-xfs-ephemeral-mkfs.service" <<'EOF' +[Unit] +DefaultDependencies=false +# Let's be sure we have basic devices, but other than that we +# can run really early. +After=systemd-tmpfiles-setup-dev.service +ConditionPathExists=/usr/lib/initrd-release +# Something seems to be causing us to rerun? +ConditionPathExists=!/run/ephemeral + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/bin/sh -c 'set -euo pipefail; mem=$$(($$(stat -f -c "%%b * %%s / 1024" /run))) && /bin/truncate -s $${mem}k /run/ephemeral.xfsloop' +ExecStart=/sbin/mkfs.xfs /run/ephemeral.xfsloop +ExecStart=/bin/mkdir /run/ephemeral +EOF +add_requires sysroot-xfs-ephemeral-mkfs.service initrd-root-fs.target + +cat >>"${UNIT_DIR}/run-ephemeral.mount" <"${UNIT_DIR}/sysroot-xfs-ephemeral-setup.service" < "${UNIT_DIR}/sysroot-etc.mount" +cat >>"${UNIT_DIR}/sysroot-etc.mount" <"${UNIT_DIR}/sysroot-var.mount" +cat >>"${UNIT_DIR}/sysroot-var.mount" <>"${UNIT_DIR}/sysroot-relabel.service" < /tmp/cmdline + mount --bind /tmp/cmdline /proc/cmdline + ;; + stop) + umount /proc/cmdline + rm /tmp/cmdline + ;; + *) + echo "Usage: $0 {start|stop}" >&2 + exit 1 + ;; +esac diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-multipath/coreos-multipath-generator b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-multipath/coreos-multipath-generator new file mode 100644 index 0000000000000000000000000000000000000000..7165620fb193a9d7f5c3509c6f54fc2c67a03d2f --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-multipath/coreos-multipath-generator @@ -0,0 +1,30 @@ +#!/bin/bash + +# Generators don't have logging right now +# https://github.com/systemd/systemd/issues/15638 +exec 1>/dev/kmsg; exec 2>&1 + +command -v getargbool >/dev/null || . /usr/lib/dracut-lib.sh + +set -e + +if is-live-image; then + exit 0 +fi + +UNIT_DIR="${1:-/tmp}" + +add_requires() { + local name="$1"; shift + local target="$1"; shift + local requires_dir="${UNIT_DIR}/${target}.requires" + mkdir -p "${requires_dir}" + ln -sf "../${name}" "${requires_dir}/${name}" +} + +if getargbool 0 rd.multipath; then + add_requires coreos-multipath-wait.target initrd.target + if ! getargbool 0 ignition.firstboot; then + add_requires coreos-multipath-trigger.service initrd.target + fi +fi diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-multipath/coreos-multipath-trigger.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-multipath/coreos-multipath-trigger.service new file mode 100644 index 0000000000000000000000000000000000000000..60ab199d2e186254e7a3dd4abcb8d05fc37f252e --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-multipath/coreos-multipath-trigger.service @@ -0,0 +1,19 @@ +# This unit is needed in the LUKS-on-multipath case on subsequent boots. When +# multipathd takes ownership of the individual paths, the by-uuid/ symlink +# which systemd-cryptsetup@.service binds to gets lost. So we retrigger udev +# here to make sure it's re-added. +# +# This is tracked at: + + +[Unit] +Description=CoreOS Trigger Multipath +DefaultDependencies=false +Requires=coreos-multipath-wait.target +After=coreos-multipath-wait.target +Before=cryptsetup-pre.target + +[Service] +Type=oneshot +ExecStart=/usr/sbin/udevadm trigger --settle --subsystem-match block +RemainAfterExit=yes diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-multipath/coreos-multipath-wait.target b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-multipath/coreos-multipath-wait.target new file mode 100644 index 0000000000000000000000000000000000000000..b003f4d94bf0170b117d1ed4459c9c21543d9c47 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-multipath/coreos-multipath-wait.target @@ -0,0 +1,17 @@ +[Unit] +Description=CoreOS Wait For Multipathed Boot +DefaultDependencies=false +Before=dracut-initqueue.service +After=dracut-cmdline.service +Requires=dev-disk-by\x2dlabel-dm\x2dmpath\x2dboot.device +After=dev-disk-by\x2dlabel-dm\x2dmpath\x2dboot.device +Requires=multipathd.service +After=multipathd.service + +# This is already enforced transitively by coreos-gpt-setup.service, but +# let's be more explicit and list it directly here too. +Before=coreos-ignition-setup-user.service + +# This is already enforced by coreos-multipath-trigger.service, though ideally +# eventually we can get rid of that one and then we *would* need this. +Before=cryptsetup-pre.target diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-multipath/coreos-propagate-multipath-conf.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-multipath/coreos-propagate-multipath-conf.service new file mode 100644 index 0000000000000000000000000000000000000000..27d1d5e7fa33d319b8bb8d3b925a65842eb12481 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-multipath/coreos-propagate-multipath-conf.service @@ -0,0 +1,21 @@ +[Unit] +Description=CoreOS Propagate Multipath Configuration +Before=initrd.target + +# we write to the rootfs, so run after it's ready +After=initrd-root-fs.target + +# That service starts initrd-cleanup.service which will race with us completing +# before we get nuked. Need to get to the bottom of it, but for now we need +# this (XXX: add link to systemd issue here). +Before=initrd-parse-etc.service + +ConditionKernelCommandLine=rd.multipath=default + +OnFailure=emergency.target +OnFailureJobMode=isolate + +[Service] +Type=oneshot +ExecStart=/usr/sbin/coreos-propagate-multipath-conf +RemainAfterExit=yes diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-multipath/coreos-propagate-multipath-conf.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-multipath/coreos-propagate-multipath-conf.sh new file mode 100644 index 0000000000000000000000000000000000000000..ebf0113737b0ed59f3aa1ba5b76a2ee6c67ff1d8 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-multipath/coreos-propagate-multipath-conf.sh @@ -0,0 +1,17 @@ +#!/bin/bash +set -euo pipefail + +# Persist automatic multipath configuration, if any. +# When booting with `rd.multipath=default`, the default multipath +# configuration is written. We need to ensure that the multipath configuration +# is persisted to the final target. + +if [ ! -f /sysroot/etc/multipath.conf ] && [ -f /etc/multipath.conf ]; then + echo "info: propagating automatic multipath configuration" + cp -v /etc/multipath.conf /sysroot/etc/ + mkdir -p /sysroot/etc/multipath/multipath.conf.d + coreos-relabel /etc/multipath.conf + coreos-relabel /etc/multipath/multipath.conf.d +else + echo "info: no initramfs automatic multipath configuration to propagate" +fi diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-multipath/module-setup.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-multipath/module-setup.sh new file mode 100644 index 0000000000000000000000000000000000000000..d467dd42573909196744c8386d0b148947bd81d9 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-multipath/module-setup.sh @@ -0,0 +1,28 @@ +#!/bin/bash +# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*- +# ex: ts=8 sw=4 sts=4 et filetype=sh + +install_ignition_unit() { + local unit=$1; shift + local target=${1:-complete} + inst_simple "$moddir/$unit" "$systemdsystemunitdir/$unit" + # note we `|| exit 1` here so we error out if e.g. the units are missing + + systemctl -q --root="$initdir" add-requires "ignition-${target}.target" "$unit" || exit 1 +} + +install() { + inst_script "$moddir/coreos-propagate-multipath-conf.sh" \ + "/usr/sbin/coreos-propagate-multipath-conf" + + install_ignition_unit coreos-propagate-multipath-conf.service subsequent + + inst_simple "$moddir/coreos-multipath-generator" \ + "$systemdutildir/system-generators/coreos-multipath-generator" + + # we don't enable these; they're enabled dynamically via the generator + inst_simple "$moddir/coreos-multipath-wait.target" \ + "$systemdsystemunitdir/coreos-multipath-wait.target" + inst_simple "$moddir/coreos-multipath-trigger.service" \ + "$systemdsystemunitdir/coreos-multipath-trigger.service" +} diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-network/50-afterburn-network-kargs-default.conf b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-network/50-afterburn-network-kargs-default.conf new file mode 100644 index 0000000000000000000000000000000000000000..44735dd6661c363d5d1c7841d2bfd110b94278af --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-network/50-afterburn-network-kargs-default.conf @@ -0,0 +1,7 @@ +# This contains the default kargs for firstboot network configuration. +# Default values can be dynamically overridden by platform-specific +# logic (e.g. injected via a back-channel). + + +[Service] +Environment=AFTERBURN_NETWORK_KARGS_DEFAULT='ip=dhcp,dhcp6' diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-network/coreos-copy-firstboot-network.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-network/coreos-copy-firstboot-network.service new file mode 100644 index 0000000000000000000000000000000000000000..af9c020a8c6ab2583eab3b8db01def071d1de352 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-network/coreos-copy-firstboot-network.service @@ -0,0 +1,61 @@ +# This unit will run early in boot and detect if the user copied +# in firstboot networking config files into the installed system +# (most likely by using `coreos-installer install --copy-network`). +# Since this unit is modifying network configuration there are some +# dependencies that we have: +# +# - Need to look for networking configuration on the /boot partition +# - i.e. after /dev/disk/by-label/boot is available +# - and after the ignition-dracut GPT generator (see below) +# - Need to run before networking is brought up. +# - This is done in nm-initrd.service [1] +# - i.e. Before=nm-initrd.service +# - Need to make sure karg networking configuration isn't applied +# - There are two ways to do this. +# - One is to run *before* the nm-config.sh [2] that runs as part of +# dracut-cmdline [3] and `ln -sf /bin/true /usr/libexec/nm-initrd-generator`. +# - i.e. Before=dracut-cmdline.service +# - Another is to run *after* nm-config.sh [2] in dracut-cmdline [3] +# and just delete all the files created by nm-initrd-generator. +# - i.e. After=dracut-cmdline.service, but Before=nm-initrd.service +# - We'll go with the second option here because the need for the /boot +# device (mentioned above) means we can't start before dracut-cmdline.service +# +# [1] https://github.com/dracutdevs/dracut/blob/master/modules.d/35network-manager/nm-initrd.service +# [2] https://github.com/dracutdevs/dracut/blob/master/modules.d/35network-manager/nm-config.sh +# [3] https://github.com/dracutdevs/dracut/blob/master/modules.d/35network-manager/module-setup.sh#L34 +# +[Unit] +Description=Copy CoreOS Firstboot Networking Config +ConditionPathExists=/usr/lib/initrd-release +DefaultDependencies=false +Before=ignition-diskful.target +Before=nm-initrd.service +# compat: remove when everyone is on dracut 054+ +Before=dracut-initqueue.service +After=dracut-cmdline.service +# Any services looking at mounts need to order after this +# because it causes device re-probing. +After=coreos-gpt-setup.service +# Since we are mounting /boot/, require the device first +Requires=dev-disk-by\x2dlabel-boot.device +After=dev-disk-by\x2dlabel-boot.device +# And since the boot device may be on multipath; optionally wait for it to +# appear via the dynamic target. +After=coreos-multipath-wait.target +# Need to run after coreos-enable-network since it may re-run the NM cmdline +# hook which will generate NM configs from the network kargs, but we want to +# have precedence. +After=coreos-enable-network.service +# We've seen races with ignition-kargs.service, which accesses /boot rw. +# Let's introduce some ordering here. Need to use `Before` because otherwise + +Before=ignition-kargs.service + +[Service] +Type=oneshot +RemainAfterExit=yes +# The MountFlags=slave is so the umount of /boot is guaranteed to happen +# /boot will only be mounted for the lifetime of the unit. +MountFlags=slave +ExecStart=/usr/sbin/coreos-copy-firstboot-network diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-network/coreos-copy-firstboot-network.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-network/coreos-copy-firstboot-network.sh new file mode 100644 index 0000000000000000000000000000000000000000..61c486c89c37810899817fea6908b070c8777332 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-network/coreos-copy-firstboot-network.sh @@ -0,0 +1,31 @@ +#!/bin/bash +set -euo pipefail + +# For a description of how this is used see coreos-copy-firstboot-network.service + +bootmnt=/mnt/boot_partition +mkdir -p ${bootmnt} +bootdev=/dev/disk/by-label/boot +firstboot_network_dir_basename="coreos-firstboot-network" +initramfs_firstboot_network_dir="${bootmnt}/${firstboot_network_dir_basename}" +initramfs_network_dir="/run/NetworkManager/system-connections/" +realroot_firstboot_network_dir="/boot/${firstboot_network_dir_basename}" + +# Mount /boot. Note that we mount /boot but we don't unmount boot because we +# are run in a systemd unit with MountFlags=slave so it is unmounted for us. +# Mount as read-only since we don't strictly need write access and we may be +# running alongside other code that also has it mounted ro +mount -o ro ${bootdev} ${bootmnt} + +if [ -n "$(ls -A ${initramfs_firstboot_network_dir} 2>/dev/null)" ]; then + # Clear out any files that may have already been generated from + # kargs by nm-initrd-generator + rm -f ${initramfs_network_dir}/* + # Copy files that were placed into boot (most likely by coreos-installer) + # to the appropriate location for NetworkManager to use the configuration. + echo "info: copying files from ${initramfs_firstboot_network_dir} to ${initramfs_network_dir}" + mkdir -p ${initramfs_network_dir} + cp -v ${initramfs_firstboot_network_dir}/* ${initramfs_network_dir}/ +else + echo "info: no files to copy from ${initramfs_firstboot_network_dir}. skipping" +fi diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-network/coreos-enable-network.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-network/coreos-enable-network.service new file mode 100644 index 0000000000000000000000000000000000000000..92c4829cd1331e57185d7a6c7143328c621c4bc6 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-network/coreos-enable-network.service @@ -0,0 +1,30 @@ +[Unit] +Description=CoreOS Enable Network +ConditionPathExists=/etc/initrd-release +DefaultDependencies=false +After=basic.target + +# Triggering conditions for cases where we need network: +# * when Ignition signals that it is required for provisioning. +# * on live systems fetching the remote rootfs in initramfs. +# * on Azure and Azure Stack Hub, for hostname fetching (metadata endpoint) and boot check-in (wireserver). +ConditionPathExists=|/run/ignition/neednet +ConditionKernelCommandLine=|coreos.live.rootfs_url +ConditionKernelCommandLine=|ignition.platform.id=azure +ConditionKernelCommandLine=|ignition.platform.id=azurestack + +# Creates /run/ignition/neednet +After=ignition-fetch-offline.service +# Needs networking +Before=ignition-fetch.service + +# See hack in coreos-enable-network, as well as coreos-copy-firstboot-network.service. +After=dracut-cmdline.service +Before=nm-initrd.service +# compat: remove when everyone is on dracut 054+ +Before=dracut-initqueue.service + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/usr/sbin/coreos-enable-network diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-network/coreos-enable-network.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-network/coreos-enable-network.sh new file mode 100644 index 0000000000000000000000000000000000000000..2bb4fcf6bb84185cc99f9e015fd2525668bcb3ed --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-network/coreos-enable-network.sh @@ -0,0 +1,27 @@ +#!/bin/bash +set -euo pipefail + +set +euo pipefail +. /usr/lib/dracut-lib.sh +set -euo pipefail + +dracut_func() { + # dracut is not friendly to set -eu + set +euo pipefail + "$@"; local rc=$? + set -euo pipefail + return $rc +} + +# If networking hasn't been requested yet, request it. +if ! dracut_func getargbool 0 'rd.neednet'; then + echo "rd.neednet=1" > /etc/cmdline.d/40-coreos-neednet.conf + + # Hack: we need to rerun the NM cmdline hook because we run after + # dracut-cmdline.service because we need udev. We should be able to move + # away from this once we run NM as a systemd unit. See also: + + set +euo pipefail + . /usr/lib/dracut/hooks/cmdline/99-nm-config.sh + set -euo pipefail +fi diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-network/module-setup.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-network/module-setup.sh new file mode 100644 index 0000000000000000000000000000000000000000..58052ee8d105c9df1a17b367b1ed0ac6d7b64c59 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/35coreos-network/module-setup.sh @@ -0,0 +1,28 @@ +install_and_enable_unit() { + unit="$1"; shift + target="$1"; shift + inst_simple "$moddir/$unit" "$systemdsystemunitdir/$unit" + # note we `|| exit 1` here so we error out if e.g. the units are missing + + systemctl -q --root="$initdir" add-requires "$target" "$unit" || exit 1 +} + +install() { + inst_simple "$moddir/coreos-enable-network.sh" \ + "/usr/sbin/coreos-enable-network" + install_and_enable_unit "coreos-enable-network.service" \ + "initrd.target" + + inst_simple "$moddir/coreos-copy-firstboot-network.sh" \ + "/usr/sbin/coreos-copy-firstboot-network" + # Only run this when ignition runs and only when the system + # has disks. ignition-diskful.target should suffice. + install_and_enable_unit "coreos-copy-firstboot-network.service" \ + "ignition-diskful.target" + + # Dropin with firstboot network configuration kargs, applied via + # Afterburn. + inst_simple "$moddir/50-afterburn-network-kargs-default.conf" \ + "/usr/lib/systemd/system/afterburn-network-kargs.service.d/50-afterburn-network-kargs-default.conf" + +} diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-conf/00-core.ign b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-conf/00-core.ign new file mode 100644 index 0000000000000000000000000000000000000000..3ddac11f3a9068f39f3f7d6b367692ad001ae727 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-conf/00-core.ign @@ -0,0 +1,19 @@ +{ + "ignition": { + "version": "3.0.0" + }, + "passwd": { + "users": [ + { + "name": "core", + "gecos": "CoreOS Admin", + "groups": [ + "adm", + "sudo", + "systemd-journal", + "wheel" + ] + } + ] + } +} diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-conf/README.md b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-conf/README.md new file mode 100644 index 0000000000000000000000000000000000000000..793e519232c8026de1468790c8ab4f4e06347156 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-conf/README.md @@ -0,0 +1 @@ +`00-core.ign` is the base config shared between FCOS and RHCOS. The configs specific to FCOS are in [50ignition-conf-fcos](../../../../../../15fcos/usr/lib/dracut/modules.d/50ignition-conf-fcos). \ No newline at end of file diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-conf/module-setup.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-conf/module-setup.sh new file mode 100644 index 0000000000000000000000000000000000000000..7e06855bf603949b456c8564077a8f5820d67866 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-conf/module-setup.sh @@ -0,0 +1,13 @@ +#!/bin/bash +# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*- +# ex: ts=8 sw=4 sts=4 et filetype=sh + +depends() { + echo ignition +} + +install() { + mkdir -p "$initdir/usr/lib/ignition/base.d" + inst "$moddir/00-core.ign" \ + "/usr/lib/ignition/base.d/00-core.ign" +} diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/coreos-check-rootfs-size b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/coreos-check-rootfs-size new file mode 100644 index 0000000000000000000000000000000000000000..d758ac489c11d90f3553e2183f3a79ff52920ac9 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/coreos-check-rootfs-size @@ -0,0 +1,36 @@ +#!/bin/bash +set -euo pipefail + +# See also ignition-ostree-check-rootfs-size.service + + +srcdev=$(findmnt -nvr -o SOURCE /sysroot | tail -n1) +size=$(lsblk --nodeps --noheadings --bytes -o SIZE "${srcdev}") + +MINIMUM_GB=8 +MINIMUM_BYTES=$((1024 * 1024 * 1024 * MINIMUM_GB)) + +MOTD_DROPIN=/etc/motd.d/60-coreos-rootfs-size.motd + +YELLOW=$(echo -e '\033[0;33m') +RESET=$(echo -e '\033[0m') + +if [ "${size}" -lt "${MINIMUM_BYTES}" ]; then + mkdir -p "/sysroot/$(dirname "${MOTD_DROPIN}")" + cat > "/sysroot/${MOTD_DROPIN}" <&2 +} + +fatal() { + err "$@" + exit 1 +} + +if [ $# -eq 0 ]; then + err "Usage: $0 [PATTERN...]" + err " e.g.: $0 /etc/passwd '/etc/group*'" +fi + +if [ ! -f /sysroot/etc/selinux/config ]; then + exit 0 +fi + +source /sysroot/etc/selinux/config + +if [ -z "${SELINUXTYPE:-}" ]; then + fatal "Couldn't find SELINUXTYPE in /sysroot/etc/selinux/config" +fi + +file_contexts="/sysroot/etc/selinux/${SELINUXTYPE}/contexts/files/file_contexts" + +prefixed_patterns=() +while [ $# -ne 0 ]; do + pattern=$1; shift + prefixed_patterns+=("/sysroot/$pattern") +done +setfiles -vFi0 -r /sysroot "$file_contexts" "${prefixed_patterns[@]}" diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/coreos-rootflags.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/coreos-rootflags.sh new file mode 100644 index 0000000000000000000000000000000000000000..1a7c0a299660ee751e6247e2669efed7408e79d2 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/coreos-rootflags.sh @@ -0,0 +1,30 @@ +#!/bin/bash +set -euo pipefail + +# see related comment block in transposefs.sh re. inspecting the config directly +ignition_cfg=/run/ignition.json +rootpath=/dev/disk/by-label/root + +query_rootfs() { + local filter=$1 + jq -re ".storage?.filesystems? // [] | + map(select(.label == \"root\" and .wipeFilesystem == true)) | + .[0] | $filter" "${ignition_cfg}" +} + +# If the rootfs was reprovisioned, then the mountOptions from the Ignition +# config has priority. +if [ -d /run/ignition-ostree-transposefs/root ]; then + if query_rootfs 'has("mountOptions")' >/dev/null; then + query_rootfs '.mountOptions | join(",")' + exit 0 + fi +fi + +eval $(blkid -o export ${rootpath}) +if [ "${TYPE}" == "xfs" ]; then + # We use prjquota on XFS by default to aid multi-tenant Kubernetes (and + # other container) clusters. See + # https://github.com/coreos/coreos-assembler/pull/303/commits/6103effbd006bb6109467830d6a3e42dd847668d + echo "prjquota" +fi diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-check-rootfs-size.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-check-rootfs-size.service new file mode 100644 index 0000000000000000000000000000000000000000..5802f49de9e42c45cbf057df0a41e9a33cdce8e6 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-check-rootfs-size.service @@ -0,0 +1,13 @@ +[Unit] +Description=Ignition OSTree: Check Root Filesystem Size +Documentation=https://docs.fedoraproject.org/en-US/fedora-coreos/storage/ +DefaultDependencies=false +ConditionKernelCommandLine=ostree +ConditionPathExists=!/run/ostree-live +After=ignition-ostree-growfs.service +After=ostree-prepare-root.service + +[Service] +Type=oneshot +ExecStart=/usr/libexec/coreos-check-rootfs-size +RemainAfterExit=yes diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-firstboot-uuid b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-firstboot-uuid new file mode 100644 index 0000000000000000000000000000000000000000..e7809535177bce5b2abafcd20510777d99bfe0ad --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-firstboot-uuid @@ -0,0 +1,67 @@ +#!/bin/bash +set -euo pipefail + +# coreos-assembler generates disk images which are installed bit-for-bit +# or booted directly in the cloud. +# Generate new UUID on firstboot; this is general best practice, but in the future +# we may use this for mounting by e.g. adding a boot= and root= kernel args. + +label=$1 + + +bootfs_uuid="96d15588-3596-4b3c-adca-a2ff7279ea63" +rootfs_uuid="910678ff-f77e-4a7d-8d53-86f2ac47a823" + +target=/dev/disk/by-label/${label} +if ! [ -b "${target}" ]; then + echo "$0: Failed to find block device ${target}" 1>&2 + exit 1 +fi + +eval $(blkid -o export ${target}) +case "${label}" in + root) orig_uuid="${rootfs_uuid}"; orig_type=xfs ;; + boot) orig_uuid="${bootfs_uuid}"; orig_type=ext4 ;; + *) echo "unexpected ${label}"; exit 1 ;; +esac + +if [ "${TYPE}" == "${orig_type}" ] && [ "${UUID}" == "${orig_uuid}" ]; then + case "${TYPE}" in + ext4) + # If the filesystem supports metadata_csum_seed then the UUID is stored + # in the superblock and there is no need to worry with an fsck. For the + # boot filesystem this FS feature wasn't supported by GRUB until recently. + # https://lists.gnu.org/archive/html/grub-devel/2021-06/msg00031.html + # Once grub is updated in all systems we care about we can standardize + # on the metadata_csum_seed and delete the `else` code block. + if tune2fs -l ${target} | grep 'metadata_csum_seed'; then + tune2fs -U random "${target}" + else + # Run an fsck since tune2fs -U requires the FS to be clean + e2fsck -fy "${target}" + # We just ran an fsck, but there is a bug where tune2fs -U will still + # complain. It will still error if the last checked timestamp (just + # set by the e2fsck above) is older than the last mount timestamp (happens + # on systems with out of date or non-functioning hardware clocks). + # Potentially fixed in future by: https://www.spinics.net/lists/linux-ext4/msg78012.html + tune2fsinfo="$(tune2fs -l ${target})" + lastmount=$(echo "$tune2fsinfo" | grep '^Last mount time:' | cut -d ':' -f 2,3,4) + lastfsck=$(echo "$tune2fsinfo" | grep '^Last checked:' | cut -d ':' -f 2,3,4) + lastmountsse=$(date --date="$lastmount" +%s) + lastfscksse=$(date --date="$lastfsck" +%s) + if (( lastfscksse < lastmountsse )); then + echo "Detected timestamp of last fsck is older than timestamp of last mount." + echo "Setting "${target}" timestamp of last fsck to same time as last mount." + tune2fs -T $(date --date="$lastmount" +%Y%m%d%H%M%S) "${target}" + fi + # Finally, we can randomize the UUID + tune2fs -U random "${target}" + fi ;; + xfs) xfs_admin -U generate "${target}" ;; + *) echo "unexpected filesystem type ${TYPE}" 1>&2; exit 1 ;; + esac + udevadm settle + echo "Regenerated UUID for ${target}" +else + echo "No changes required for ${target} TYPE=${TYPE} UUID=${UUID}" +fi diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-growfs.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-growfs.service new file mode 100644 index 0000000000000000000000000000000000000000..8704894f3b18133c23847bee66a3e3cebc6a0b6e --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-growfs.service @@ -0,0 +1,15 @@ +[Unit] +Description=Ignition OSTree: Grow root filesystem +DefaultDependencies=false +ConditionKernelCommandLine=ostree +ConditionPathExists=!/run/ostree-live +Before=initrd-root-fs.target +After=sysroot.mount ignition-ostree-mount-firstboot-sysroot.service +# This shouldn't be strictly necessary, but it's cleaner to not have OSTree muck +# around with moving mounts while we're still resizing the filesystem. +Before=ostree-prepare-root.service + +[Service] +Type=oneshot +ExecStart=/usr/sbin/ignition-ostree-growfs +RemainAfterExit=yes diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-growfs.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-growfs.sh new file mode 100644 index 0000000000000000000000000000000000000000..90f9595476a4824c15d74e57fc9e04e3b5871d6d --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-growfs.sh @@ -0,0 +1,119 @@ +#!/bin/bash +set -euo pipefail + +# This script is run by ignition-ostree-growfs.service. It grows the root +# partition, unless it determines that either the rootfs was moved or the +# partition was already resized (e.g. via Ignition). + +# If root reprovisioning was triggered, this file contains state of the root +# partition *before* ignition-disks. +saved_partstate=/run/ignition-ostree-rootfs-partstate.sh + +# We run after the rootfs is mounted at /sysroot, but before ostree-prepare-root +# moves it to /sysroot/sysroot. +path=/sysroot + +# The use of tail is to avoid errors from duplicate mounts; +# this shouldn't happen for us but we're being conservative. +src=$(findmnt -nvr -o SOURCE "$path" | tail -n1) + +if [ ! -f "${saved_partstate}" ]; then + partition=$(realpath /dev/disk/by-label/root) +else + # The rootfs was reprovisioned. Our rule in this case is: we only grow if + # the partition backing the rootfs is the same and its size didn't change + # (IOW, it was an in-place reprovisioning; e.g. LUKS or xfs -> btrfs). + source "${saved_partstate}" + if [ "${TYPE}" != "part" ]; then + # this really should never happen; but play nice + echo "$0: original rootfs blockdev not of type 'part'; not auto-growing" + exit 0 + fi + partition=$(realpath "${NAME}") + if [ "${SIZE}" != "$(lsblk --nodeps -bno SIZE "${partition}")" ]; then + echo "$0: original root partition changed size; not auto-growing" + exit 0 + fi + if ! lsblk -no MOUNTPOINT "${partition}" | grep -q '^/sysroot$'; then + echo "$0: original root partition no longer backing rootfs; not auto-growing" + exit 0 + fi +fi + +# Go through each blockdev in the hierarchy and verify we know how to grow them +lsblk -no TYPE "${partition}" | while read dev; do + case "${dev}" in + part|crypt) ;; + *) echo "error: Unsupported blockdev type ${dev}" 1>&2; exit 1 ;; + esac +done + +# Get the filesystem type before extending the partition. This matters +# because the partition, once extended, might include leftover superblocks +# from the previous contents of the disk (notably ZFS), causing blkid to +# refuse to return any filesystem type at all. +eval $(blkid -o export "${src}") +ROOTFS_TYPE=${TYPE:-} +case "${ROOTFS_TYPE}" in + xfs|ext4|btrfs) ;; + *) echo "error: Unsupported filesystem for ${path}: '${ROOTFS_TYPE}'" 1>&2; exit 1 ;; +esac + +# Now, go through the hierarchy, growing everything. Note we go one device at a +# time using --nodeps, because ordering is buggy in el8: +current_blkdev=${partition} +while true; do + eval "$(lsblk --paths --nodeps --pairs -o NAME,TYPE,PKNAME "${current_blkdev}")" + MAJMIN=$(echo $(lsblk -dno MAJ:MIN "${NAME}")) + case "${TYPE}" in + part) + eval $(udevadm info --query property --export "${current_blkdev}" | grep ^DM_ || :) + if [ -n "${DM_MPATH:-}" ]; then + # Since growpart does not understand device mapper, we have to use sfdisk. + echo ", +" | sfdisk --no-reread --no-tell-kernel --force -N "${DM_PART}" "/dev/mapper/${DM_MPATH}" + udevadm settle # Wait for udev-triggered kpartx to update mappings + else + partnum=$(cat "/sys/dev/block/${MAJMIN}/partition") + # XXX: ideally this'd be idempotent and we wouldn't `|| :` + growpart "${PKNAME}" "${partnum}" || : + fi + ;; + crypt) + # XXX: yuck... we need to expose this sanely in clevis + (. /usr/bin/clevis-luks-common-functions + eval $(udevadm info --query=property --export "${NAME}") + # lsblk doesn't print PKNAME of crypt devices with --nodeps + PKNAME=/dev/$(ls "/sys/dev/block/${MAJMIN}/slaves") + clevis_luks_unlock_device "${PKNAME}" | cryptsetup resize -d- "${DM_NAME}" + ) + ;; + # already checked + *) echo "unreachable" 1>&2; exit 1 ;; + esac + holders="/sys/dev/block/${MAJMIN}/holders" + [ -d "${holders}" ] || break + nholders="$(ls "${holders}" | wc -l)" + if [ "${nholders}" -eq 0 ]; then + break + elif [ "${nholders}" -gt 1 ]; then + # this shouldn't happen since we've checked the partition types already + echo "error: Unsupported block device with multiple children: ${NAME}" 1>&2 + exit 1 + fi + current_blkdev=/dev/$(ls "${holders}") +done + +# Wipe any filesystem signatures from the extended partition that don't +# correspond to the FS type we detected earlier. +wipefs -af -t "no${ROOTFS_TYPE}" "${src}" + +# TODO: Add XFS to https://github.com/systemd/systemd/blob/master/src/partition/growfs.c +# and use it instead. +case "${ROOTFS_TYPE}" in + xfs) xfs_growfs "${path}" ;; + ext4) resize2fs "${src}" ;; + btrfs) btrfs filesystem resize max ${path} ;; +esac + +# this is useful for tests +touch /run/ignition-ostree-growfs.stamp diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-mount-firstboot-sysroot.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-mount-firstboot-sysroot.service new file mode 100644 index 0000000000000000000000000000000000000000..3ba677d0e3f9953fb14748178a1b539eec9f6b14 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-mount-firstboot-sysroot.service @@ -0,0 +1,25 @@ +[Unit] +Description=Ignition OSTree: Mount (firstboot) /sysroot +# These dependencies should match the "other" in +# ignition-ostree-mount-subsequent-sysroot.service +DefaultDependencies=false +# If root is specified, then systemd's generator will win +ConditionKernelCommandLine=!root +ConditionKernelCommandLine=ostree +# This is redundant since we're queued on -diskful.target, but eh. +ConditionPathExists=!/run/ostree-live +# There can be only one, Highlander style +Conflicts=ignition-ostree-mount-subsequent-sysroot.service +Before=initrd-root-fs.target +After=ignition-disks.service +# Note we don't have a Requires: /dev/disk/by-label/root here like +# the -subsequent service does because ignition-disks may have +# regenerated it. +Requires=ignition-disks.service +# These have an explicit dependency on After=sysroot.mount today +Before=ostree-prepare-root.service ignition-remount-sysroot.service + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/usr/sbin/ignition-ostree-mount-sysroot diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-mount-subsequent-sysroot.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-mount-subsequent-sysroot.service new file mode 100644 index 0000000000000000000000000000000000000000..92dde886f0987c5e801452295085e53aa2d4d69a --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-mount-subsequent-sysroot.service @@ -0,0 +1,24 @@ +# Note this unit is conditionally enabled by ignition-ostree-generator +[Unit] +Description=CoreOS: Mount (subsequent) /sysroot +# These dependencies should match the "other" in +# ignition-ostree-mount-firsboot-sysroot.service +DefaultDependencies=false +# If root is specified, then systemd's generator will win +ConditionKernelCommandLine=!root +ConditionKernelCommandLine=ostree +ConditionPathExists=!/run/ostree-live +# There can be only one, Highlander style +Conflicts=ignition-ostree-mount-firstboot-sysroot.service +# And in contrast to the firstboot, we expect +# the root device to be ready. +Requires=dev-disk-by\x2dlabel-root.device +After=dev-disk-by\x2dlabel-root.device +Before=initrd-root-fs.target +# This has an explicit dependency on After=sysroot.mount today +Before=ostree-prepare-root.service + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/usr/sbin/ignition-ostree-mount-sysroot diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-mount-sysroot.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-mount-sysroot.sh new file mode 100644 index 0000000000000000000000000000000000000000..a51c4b26fec62b166f817904fab190a8c02ca217 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-mount-sysroot.sh @@ -0,0 +1,18 @@ +#!/bin/bash +set -euo pipefail + +# Note that on *new machines* this script is now only ever used on firstboot. On +# subsequent boots, systemd-fstab-generator mounts /sysroot from the +# root=UUID=... and rootflags=... kargs. + +# We may do a migration window at some point where older machines have these +# kargs injected so that we can simplify the model further. + +rootpath=/dev/disk/by-label/root +if ! [ -b "${rootpath}" ]; then + echo "ignition-ostree-mount-sysroot: Failed to find ${rootpath}" 1>&2 + exit 1 +fi + +echo "Mounting ${rootpath} ($(realpath "${rootpath}")) to /sysroot" +mount -o "$(coreos-rootflags)" "${rootpath}" /sysroot diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-mount-var.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-mount-var.service new file mode 100644 index 0000000000000000000000000000000000000000..09d6c15ab31bc4b1b19ce861085435341b65d8cf --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-mount-var.service @@ -0,0 +1,26 @@ +[Unit] +Description=Mount OSTree /var +DefaultDependencies=false +ConditionKernelCommandLine=ostree +ConditionPathExists=!/run/ostree-live + +# Make sure ExecStop= runs before we switch root +Before=initrd-switch-root.target + +# Make sure if ExecStop= fails, the boot fails +OnFailure=emergency.target +OnFailureJobMode=isolate + +# Make sure /sysroot is mounted first, since we're mounting under there +Requires=initrd-root-fs.target +After=initrd-root-fs.target + +# Need to do this before Ignition mounts any other filesystems (potentially +# shadowing our own bind mount). +Before=ignition-mount.service + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/usr/sbin/ignition-ostree-mount-var mount +ExecStop=/usr/sbin/ignition-ostree-mount-var umount diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-mount-var.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-mount-var.sh new file mode 100644 index 0000000000000000000000000000000000000000..885598e2a7db944c414d46d2003b3ea738498033 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-mount-var.sh @@ -0,0 +1,52 @@ +#!/bin/bash +set -euo pipefail + +fatal() { + echo "$@" >&2 + exit 1 +} + +if [ $# -ne 1 ] || { [[ $1 != mount ]] && [[ $1 != umount ]]; }; then + fatal "Usage: $0 " +fi + +get_ostree_arg() { + # yes, this doesn't account for spaces within args, e.g. myarg="my val", but + # it still works for our purposes + ( + IFS=$' ' + # shellcheck disable=SC2013 + for arg in $(cat /proc/cmdline); do + if [[ $arg == ostree=* ]]; then + echo "${arg#ostree=}" + fi + done + ) +} + +do_mount() { + ostree=$(get_ostree_arg) + if [ -z "${ostree}" ]; then + fatal "No ostree= kernel argument in /proc/cmdline" + fi + + deployment_path=/sysroot/${ostree} + if [ ! -L "${deployment_path}" ]; then + fatal "${deployment_path} is not a symlink" + fi + + stateroot_var_path=$(realpath "${deployment_path}/../../var") + if [ ! -d "${stateroot_var_path}" ]; then + fatal "${stateroot_var_path} is not a directory" + fi + + echo "Mounting $stateroot_var_path" + mount --bind "$stateroot_var_path" /sysroot/var +} + +do_umount() { + echo "Unmounting /sysroot/var" + umount /sysroot/var +} + +"do_$1" diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-populate-var.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-populate-var.service new file mode 100644 index 0000000000000000000000000000000000000000..d7aa622caec80f5b99c3f0839d29e2ca233a659c --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-populate-var.service @@ -0,0 +1,16 @@ +[Unit] +Description=Populate OSTree /var +DefaultDependencies=false +ConditionKernelCommandLine=|ostree +ConditionPathExists=|/run/ostree-live + +# Need to do this with all mount points active +After=ignition-mount.service + +# But *before* we start dumping files in there +Before=ignition-files.service + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/usr/sbin/ignition-ostree-populate-var diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-populate-var.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-populate-var.sh new file mode 100644 index 0000000000000000000000000000000000000000..fb5092d829e7fe51951e7c1a89ec00ee1ce0618b --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-populate-var.sh @@ -0,0 +1,45 @@ +#!/bin/bash +set -euo pipefail + +fatal() { + echo "$@" >&2 + exit 1 +} + +if [ $# -ne 0 ]; then + fatal "Usage: $0" +fi + +# See the similar code block in Anaconda, which handles this today for Atomic +# Host and Silverblue: +# https://github.com/rhinstaller/anaconda/blob/b9ea8ce4e68196b30a524c1cc5680dcdc4b89371/pyanaconda/payload/rpmostreepayload.py#L332 + +for varsubdir in lib log home roothome opt srv usrlocal mnt media; do + + # If the directory already existed, just ignore. This addresses the live + # image case with persistent `/var`; we don't want to relabel all the files + # there on each boot. + if [ -d "/sysroot/var/${varsubdir}" ]; then + continue + fi + + if [[ $varsubdir == lib ]] || [[ $varsubdir == log ]]; then + # Simply manually mkdir /var/{lib,log}; the tmpfiles.d entries otherwise + # reference users/groups which we don't have access to from here + # (though... we *could* import them from the sysroot, and have + # nss-altfiles in the initrd, but meh... let's just wait for + # systemd-sysusers which will make this way easier: + + mkdir -p /sysroot/var/${varsubdir} + else + systemd-tmpfiles --create --boot --root=/sysroot --prefix="/var/${varsubdir}" + fi + + if [[ $varsubdir == roothome ]]; then + # TODO move this to tmpfiles.d once systemd-tmpfiles handles C! with --root correctly. + + cp /sysroot/etc/skel/.bash* /sysroot/var/${varsubdir} + fi + + coreos-relabel "/var/${varsubdir}" +done diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-transposefs-detect.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-transposefs-detect.service new file mode 100644 index 0000000000000000000000000000000000000000..389dc9eedf644f449307ae88abae07cb591b41e6 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-transposefs-detect.service @@ -0,0 +1,20 @@ +[Unit] +Description=Ignition OSTree: Detect Partition Transposition +DefaultDependencies=false +After=ignition-fetch.service +Before=ignition-disks.service +Before=initrd-root-fs.target +Before=sysroot.mount +ConditionKernelCommandLine=ostree +OnFailure=emergency.target +OnFailureJobMode=isolate + +# This stage requires udevd to detect disks +Requires=systemd-udevd.service +After=systemd-udevd.service + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/usr/libexec/ignition-ostree-transposefs detect +ExecStop=/usr/libexec/ignition-ostree-transposefs cleanup diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-transposefs-restore.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-transposefs-restore.service new file mode 100644 index 0000000000000000000000000000000000000000..4eca578934571af45dbc659cce9761151ca1e72e --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-transposefs-restore.service @@ -0,0 +1,20 @@ +[Unit] +Description=Ignition OSTree: Restore Partitions +DefaultDependencies=false +After=ignition-disks.service +# Avoid racing with UUID regeneration +After=ignition-ostree-uuid-root.service +Before=ignition-ostree-growfs.service +Before=ignition-ostree-mount-firstboot-sysroot.service +OnFailure=emergency.target +OnFailureJobMode=isolate + +ConditionKernelCommandLine=ostree +ConditionPathIsDirectory=/run/ignition-ostree-transposefs + +[Service] +Type=oneshot +RemainAfterExit=yes +# So we can transiently mount sysroot +MountFlags=slave +ExecStart=/usr/libexec/ignition-ostree-transposefs restore diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-transposefs-save.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-transposefs-save.service new file mode 100644 index 0000000000000000000000000000000000000000..bc03499ecbfee558ab229f9d8f93fefa8d4727c8 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-transposefs-save.service @@ -0,0 +1,19 @@ +[Unit] +Description=Ignition OSTree: Save Partitions +DefaultDependencies=false +After=ignition-ostree-transposefs-detect.service +Before=ignition-disks.service +ConditionKernelCommandLine=ostree +ConditionPathIsDirectory=/run/ignition-ostree-transposefs +# Any services looking at mounts need to order after this +# because it causes device re-probing. +After=coreos-gpt-setup.service +OnFailure=emergency.target +OnFailureJobMode=isolate + +[Service] +Type=oneshot +RemainAfterExit=yes +# So we can transiently mount sysroot +MountFlags=slave +ExecStart=/usr/libexec/ignition-ostree-transposefs save diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-transposefs.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-transposefs.sh new file mode 100644 index 0000000000000000000000000000000000000000..18224c363c8a095c9916f8e094797e2f6f194110 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-transposefs.sh @@ -0,0 +1,244 @@ +#!/bin/bash +set -euo pipefail + +boot_sector_size=440 +esp_typeguid=c12a7328-f81f-11d2-ba4b-00a0c93ec93b +bios_typeguid=21686148-6449-6e6f-744e-656564454649 +prep_typeguid=9e1a2d38-c612-4316-aa26-8b49521e5a8b + +# This is implementation details of Ignition; in the future, we should figure +# out a way to ask Ignition directly whether there's a filesystem with label +# "root" being set up. +ignition_cfg=/run/ignition.json +root_part=/dev/disk/by-label/root +boot_part=/dev/disk/by-label/boot +esp_part=/dev/disk/by-label/EFI-SYSTEM +bios_part=/dev/disk/by-partlabel/BIOS-BOOT +prep_part=/dev/disk/by-partlabel/PowerPC-PReP-boot +saved_data=/run/ignition-ostree-transposefs +saved_root=${saved_data}/root +saved_boot=${saved_data}/boot +saved_esp=${saved_data}/esp +saved_bios=${saved_data}/bios +saved_prep=${saved_data}/prep +zram_dev=${saved_data}/zram_dev +partstate_root=/run/ignition-ostree-rootfs-partstate.sh + +# Print jq query string for wiped filesystems with label $1 +query_fslabel() { + echo ".storage?.filesystems? // [] | map(select(.label == \"$1\" and .wipeFilesystem == true))" +} + +# Print jq query string for partitions with type GUID $1 +query_parttype() { + echo ".storage?.disks? // [] | map(.partitions?) | flatten | map(select(try .typeGuid catch \"\" | ascii_downcase == \"$1\"))" +} + +# Print partition labels for partitions with type GUID $1 +get_partlabels_for_parttype() { + jq -r "$(query_parttype $1) | .[].label" "${ignition_cfg}" +} + +# Mounts device to directory, with extra logging of the src device +mount_verbose() { + local srcdev=$1; shift + local destdir=$1; shift + echo "Mounting ${srcdev} ($(realpath "$srcdev")) to $destdir" + mkdir -p "${destdir}" + mount "${srcdev}" "${destdir}" +} + +# Sometimes, for some reason the by-label symlinks aren't updated. Detect these +# cases, and explicitly `udevadm trigger`. +# See: https://bugzilla.redhat.com/show_bug.cgi?id=1908780 +udev_trigger_on_label_mismatch() { + local label=$1; shift + local expected_dev=$1; shift + local actual_dev + expected_dev=$(realpath "${expected_dev}") + actual_dev=$(realpath "/dev/disk/by-label/$label") + if [ "$actual_dev" != "$expected_dev" ]; then + echo "Expected /dev/disk/by-label/$label to point to $expected_dev, but points to $actual_dev; triggering udev" + udevadm trigger --settle "$expected_dev" + fi +} + +# Print partition offset for device node $1 +get_partition_offset() { + local devpath=$(udevadm info --query=path "$1") + cat "/sys${devpath}/start" +} + +mount_and_restore_filesystem_by_label() { + local label=$1; shift + local mountpoint=$1; shift + local saved_fs=$1; shift + local new_dev + new_dev=$(jq -r "$(query_fslabel "${label}") | .[0].device" "${ignition_cfg}") + udev_trigger_on_label_mismatch "${label}" "${new_dev}" + mount_verbose "/dev/disk/by-label/${label}" "${mountpoint}" + find "${saved_fs}" -mindepth 1 -maxdepth 1 -exec mv -t "${mountpoint}" {} \; +} + +case "${1:-}" in + detect) + # Mounts are not in a private namespace so we can mount ${saved_data} + wipes_root=$(jq "$(query_fslabel root) | length" "${ignition_cfg}") + wipes_boot=$(jq "$(query_fslabel boot) | length" "${ignition_cfg}") + creates_esp=$(jq "$(query_parttype ${esp_typeguid}) | length" "${ignition_cfg}") + creates_bios=$(jq "$(query_parttype ${bios_typeguid}) | length" "${ignition_cfg}") + creates_prep=$(jq "$(query_parttype ${prep_typeguid}) | length" "${ignition_cfg}") + if [ "${wipes_root}${wipes_boot}${creates_esp}${creates_bios}${creates_prep}" = "00000" ]; then + exit 0 + fi + echo "Detected partition replacement in fetched Ignition config: /run/ignition.json" + # verify all ESP, BIOS, and PReP partitions have non-null unique labels + unique_esp=$(jq -r "$(query_parttype ${esp_typeguid}) | [.[].label | values] | unique | length" "${ignition_cfg}") + unique_bios=$(jq -r "$(query_parttype ${bios_typeguid}) | [.[].label | values] | unique | length" "${ignition_cfg}") + unique_prep=$(jq -r "$(query_parttype ${prep_typeguid}) | [.[].label | values] | unique | length" "${ignition_cfg}") + if [ "${creates_esp}" != "${unique_esp}" -o "${creates_bios}" != "${unique_bios}" -o "${creates_prep}" != "${unique_prep}" ]; then + echo "Found duplicate or missing ESP, BIOS-BOOT, or PReP labels in config" >&2 + exit 1 + fi + mem_available=$(grep MemAvailable /proc/meminfo | awk '{print $2}') + # Just error out early if we don't even have 1G to work with. This + # commonly happens if you `cosa run` but forget to add `--memory`. That + # way you get a nicer error instead of the spew of EIO errors from `cp`. + # The amount we need is really dependent on a bunch of factors, but just + # ballpark it at 3G. + if [ "${mem_available}" -lt $((1*1024*1024)) ] && [ "${wipes_root}" != 0 ]; then + echo "Root reprovisioning requires at least 3G of RAM" >&2 + exit 1 + fi + modprobe zram num_devices=0 + read dev < /sys/class/zram-control/hot_add + # disksize is set arbitrarily large, as zram is capped by mem_limit + echo 10G > /sys/block/zram"${dev}"/disksize + # Limit zram to 90% of available RAM: we want to be greedy since the + # boot breaks anyway, but we still want to leave room for everything + # else so it hits ENOSPC and doesn't invoke the OOM killer + echo $(( mem_available * 90 / 100 ))K > /sys/block/zram"${dev}"/mem_limit + mkfs.xfs -q /dev/zram"${dev}" + mkdir "${saved_data}" + mount /dev/zram"${dev}" "${saved_data}" + # save the zram device number created for when called to cleanup + echo "${dev}" > "${zram_dev}" + + if [ "${wipes_root}" != "0" ]; then + mkdir "${saved_root}" + fi + if [ "${wipes_boot}" != "0" ]; then + mkdir "${saved_boot}" + fi + if [ "${creates_esp}" != "0" ]; then + mkdir "${saved_esp}" + fi + if [ "${creates_bios}" != "0" ]; then + mkdir "${saved_bios}" + fi + if [ "${creates_prep}" != "0" ]; then + mkdir "${saved_prep}" + fi + ;; + save) + # Mounts happen in a private mount namespace since we're not "offically" mounting + if [ -d "${saved_root}" ]; then + echo "Moving rootfs to RAM..." + mount_verbose "${root_part}" /sysroot + cp -aT /sysroot "${saved_root}" + # also store the state of the partition + lsblk "${root_part}" --nodeps --pairs -b --paths -o NAME,TYPE,SIZE > "${partstate_root}" + fi + if [ -d "${saved_boot}" ]; then + echo "Moving bootfs to RAM..." + mount_verbose "${boot_part}" /sysroot/boot + cp -aT /sysroot/boot "${saved_boot}" + fi + if [ -d "${saved_esp}" ]; then + echo "Moving EFI System Partition to RAM..." + mount_verbose "${esp_part}" /sysroot/boot/efi + cp -aT /sysroot/boot/efi "${saved_esp}" + fi + if [ -d "${saved_bios}" ]; then + echo "Moving BIOS Boot partition and boot sector to RAM..." + # save partition + cat "${bios_part}" > "${saved_bios}/partition" + # save boot sector + bios_disk=$(lsblk --noheadings --output PKNAME --paths "${bios_part}") + dd if="${bios_disk}" of="${saved_bios}/boot-sector" bs="${boot_sector_size}" count=1 status=none + # store partition start offset so we can check it later + get_partition_offset "${bios_part}" > "${saved_bios}/start" + fi + if [ -d "${saved_prep}" ]; then + echo "Moving PReP partition to RAM..." + cat "${prep_part}" > "${saved_prep}/partition" + fi + echo "zram usage:" + read dev < "${zram_dev}" + cat /sys/block/zram"${dev}"/mm_stat + ;; + restore) + # Mounts happen in a private mount namespace since we're not "offically" mounting + if [ -d "${saved_root}" ]; then + echo "Restoring rootfs from RAM..." + mount_and_restore_filesystem_by_label root /sysroot "${saved_root}" + chcon -v --reference "${saved_root}" /sysroot # the root of the fs itself + chattr +i $(ls -d /sysroot/ostree/deploy/*/deploy/*/) + fi + if [ -d "${saved_boot}" ]; then + echo "Restoring bootfs from RAM..." + mount_and_restore_filesystem_by_label boot /sysroot/boot "${saved_boot}" + chcon -v --reference "${saved_boot}" /sysroot/boot # the root of the fs itself + fi + if [ -d "${saved_esp}" ]; then + echo "Restoring EFI System Partition from RAM..." + get_partlabels_for_parttype "${esp_typeguid}" | while read label; do + # Don't use mount_and_restore_filesystem_by_label because: + # 1. We're mounting by partlabel, not FS label + # 2. We need to copy the contents to each partition, not move + # them once + # 3. We don't need the by-label symlink to be correct and + # nothing later in boot will be mounting the filesystem + mountpoint="/mnt/esp-${label}" + mount_verbose "/dev/disk/by-partlabel/${label}" "${mountpoint}" + find "${saved_esp}" -mindepth 1 -maxdepth 1 -exec cp -a {} "${mountpoint}" \; + done + fi + if [ -d "${saved_bios}" ]; then + echo "Restoring BIOS Boot partition and boot sector from RAM..." + expected_start=$(cat "${saved_bios}/start") + get_partlabels_for_parttype "${bios_typeguid}" | while read label; do + cur_part="/dev/disk/by-partlabel/${label}" + # boot sector hardcodes the partition start; ensure it matches + cur_start=$(get_partition_offset "${cur_part}") + if [ "${cur_start}" != "${expected_start}" ]; then + echo "Partition ${cur_part} starts at ${cur_start}; expected ${expected_start}" >&2 + exit 1 + fi + # copy partition contents + cat "${saved_bios}/partition" > "${cur_part}" + # copy boot sector + cur_disk=$(lsblk --noheadings --output PKNAME --paths "${cur_part}") + cat "${saved_bios}/boot-sector" > "${cur_disk}" + done + fi + if [ -d "${saved_prep}" ]; then + echo "Restoring PReP partition from RAM..." + get_partlabels_for_parttype "${prep_typeguid}" | while read label; do + cat "${saved_prep}/partition" > "/dev/disk/by-partlabel/${label}" + done + fi + ;; + cleanup) + # Mounts are not in a private namespace so we can unmount ${saved_data} + if [ -d "${saved_data}" ]; then + read dev < "${zram_dev}" + umount "${saved_data}" + rm -rf "${saved_data}" "${partstate_root}" + echo "${dev}" > /sys/class/zram-control/hot_remove + fi + ;; + *) + echo "Unsupported operation: ${1:-}" 1>&2; exit 1 + ;; +esac diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-uuid-boot.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-uuid-boot.service new file mode 100644 index 0000000000000000000000000000000000000000..cde3b1629653b83d976dbda4cb1f63edf50a21f5 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-uuid-boot.service @@ -0,0 +1,22 @@ +[Unit] +Description=Ignition OSTree: Regenerate Filesystem UUID (boot) +DefaultDependencies=false +ConditionPathExists=/usr/lib/initrd-release +ConditionKernelCommandLine=ostree +ConditionPathExists=!/run/ostree-live +# We run pretty early +Before=coreos-copy-firstboot-network.service +Before=coreos-ignition-setup-user.service +Before=ignition-fetch-offline.service +# Any services looking at mounts need to order after this +# because it causes device re-probing. +After=coreos-gpt-setup.service + +Before=systemd-fsck@dev-disk-by\x2dlabel-boot.service +Requires=dev-disk-by\x2dlabel-boot.device +After=dev-disk-by\x2dlabel-boot.device + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/usr/sbin/ignition-ostree-firstboot-uuid boot diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-uuid-root.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-uuid-root.service new file mode 100644 index 0000000000000000000000000000000000000000..7164aaf5c3bf9840a47f2aad50d1b0764bca0dab --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/ignition-ostree-uuid-root.service @@ -0,0 +1,24 @@ +[Unit] +Description=Ignition OSTree: Regenerate Filesystem UUID (root) +# These conditions match mount-firstboot-sysroot.service +DefaultDependencies=false +ConditionKernelCommandLine=ostree +ConditionPathExists=!/run/ostree-live +Before=initrd-root-fs.target +After=ignition-disks.service +# If we've reprovisioned the rootfs, then there's no need to restamp +ConditionPathExists=!/run/ignition-ostree-transposefs + +After=dev-disk-by\x2dlabel-root.device +# Avoid racing with fsck +Before=systemd-fsck@dev-disk-by\x2dlabel-root.service + +# Note we don't have a Requires: /dev/disk/by-label/root here like +# the -subsequent service does because ignition-disks may have +# regenerated it. +Before=ignition-ostree-mount-firstboot-sysroot.service + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/usr/sbin/ignition-ostree-firstboot-uuid root diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/module-setup.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/module-setup.sh new file mode 100644 index 0000000000000000000000000000000000000000..ccaf8ccb21a7f06f24cd47b9ddf967bc79259788 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/module-setup.sh @@ -0,0 +1,103 @@ +#!/bin/bash +# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*- +# ex: ts=8 sw=4 sts=4 et filetype=sh + +depends() { + echo ignition rdcore +} + +install_ignition_unit() { + local unit=$1; shift + local target=${1:-complete} + inst_simple "$moddir/$unit" "$systemdsystemunitdir/$unit" + # note we `|| exit 1` here so we error out if e.g. the units are missing + systemctl -q --root="$initdir" add-requires "ignition-${target}.target" "$unit" || exit 1 +} + +installkernel() { + # Used by ignition-ostree-transposefs + instmods -c zram +} + +install() { + inst_multiple \ + realpath \ + setfiles \ + chcon \ + systemd-sysusers \ + systemd-tmpfiles \ + sort \ + uniq + + # ignition-ostree-growfs deps + inst_multiple \ + basename \ + blkid \ + cat \ + dirname \ + findmnt \ + growpart \ + realpath \ + resize2fs \ + tail \ + tune2fs \ + touch \ + xfs_admin \ + xfs_growfs \ + wc \ + wipefs + + # growpart deps + # Mostly generated from the following command: + # $ bash --rpm-requires /usr/bin/growpart | sort | uniq | grep executable + # with a few false positives (rq, rqe, -v) and one missed (mktemp) + inst_multiple \ + awk \ + cat \ + dd \ + grep \ + mktemp \ + partx \ + rm \ + sed \ + sfdisk \ + sgdisk \ + find + + for x in mount populate; do + install_ignition_unit ignition-ostree-${x}-var.service + inst_script "$moddir/ignition-ostree-${x}-var.sh" "/usr/sbin/ignition-ostree-${x}-var" + done + + inst_simple \ + /usr/lib/udev/rules.d/90-coreos-device-mapper.rules + + inst_multiple jq chattr + inst_script "$moddir/ignition-ostree-transposefs.sh" "/usr/libexec/ignition-ostree-transposefs" + for x in detect save restore; do + install_ignition_unit ignition-ostree-transposefs-${x}.service + done + + # Disk support + install_ignition_unit ignition-ostree-mount-firstboot-sysroot.service diskful + for p in boot root; do + install_ignition_unit ignition-ostree-uuid-${p}.service diskful + done + inst_script "$moddir/ignition-ostree-firstboot-uuid" \ + "/usr/sbin/ignition-ostree-firstboot-uuid" + install_ignition_unit ignition-ostree-mount-subsequent-sysroot.service diskful-subsequent + inst_script "$moddir/ignition-ostree-mount-sysroot.sh" \ + "/usr/sbin/ignition-ostree-mount-sysroot" + inst_script "$moddir/coreos-rootflags.sh" \ + "/usr/sbin/coreos-rootflags" + + install_ignition_unit ignition-ostree-growfs.service + inst_script "$moddir/ignition-ostree-growfs.sh" \ + /usr/sbin/ignition-ostree-growfs + + install_ignition_unit ignition-ostree-check-rootfs-size.service + inst_script "$moddir/coreos-check-rootfs-size" \ + /usr/libexec/coreos-check-rootfs-size + + inst_script "$moddir/coreos-relabel" /usr/bin/coreos-relabel +} diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/50coreos-kernel/coreos-check-kernel.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/50coreos-kernel/coreos-check-kernel.service new file mode 100644 index 0000000000000000000000000000000000000000..ce8a0ac524a5ee848e2a759bfdd045e0c54a4fae --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/50coreos-kernel/coreos-check-kernel.service @@ -0,0 +1,10 @@ +[Unit] +Description=Check that initrd matches kernel +DefaultDependencies=false +Before=sysinit.target systemd-modules-load.service +ConditionPathIsDirectory=!/usr/lib/modules/%v + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/bin/false diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/50coreos-kernel/module-setup.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/50coreos-kernel/module-setup.sh new file mode 100644 index 0000000000000000000000000000000000000000..6c278a93e204f9111b3a0ae433341613cb34492e --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/50coreos-kernel/module-setup.sh @@ -0,0 +1,14 @@ +install_unit() { + unit="$1"; shift + target="$1"; shift + inst_simple "$moddir/$unit" "$systemdsystemunitdir/$unit" + # note we `|| exit 1` here so we error out if e.g. the units are missing + systemctl -q --root="$initdir" add-requires "$target" "$unit" || exit 1 +} + +install() { + inst_multiple \ + false + + install_unit "coreos-check-kernel.service" "sysinit.target" +} diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/60coreos-agetty-workaround/coreos-touch-run-agetty.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/60coreos-agetty-workaround/coreos-touch-run-agetty.service new file mode 100644 index 0000000000000000000000000000000000000000..195b392ddbe9db53aa16c74b08ad086d08afe3d4 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/60coreos-agetty-workaround/coreos-touch-run-agetty.service @@ -0,0 +1,10 @@ +# Temporary hack to work around agetty SELinux denials. +[Unit] +Description=CoreOS: Touch /run/agetty.reload +Documentation=https://bugzilla.redhat.com/show_bug.cgi?id=1932053 +DefaultDependencies=false + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/usr/bin/touch /run/agetty.reload diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/60coreos-agetty-workaround/module-setup.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/60coreos-agetty-workaround/module-setup.sh new file mode 100644 index 0000000000000000000000000000000000000000..cce3ace68ac5e29c99a2adc3be4fbd862876cff2 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/60coreos-agetty-workaround/module-setup.sh @@ -0,0 +1,19 @@ +#!/bin/bash +# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*- +# ex: ts=8 sw=4 sts=4 et filetype=sh + + +install_unit() { + local unit=$1; shift + inst_simple "$moddir/$unit" "$systemdsystemunitdir/$unit" + # note we `|| exit 1` here so we error out if e.g. the units are missing + systemctl -q --root="$initdir" add-requires initrd.target "$unit" || exit 1 +} + +install() { + inst_multiple \ + touch + + # TODO f35: check if we can drop this whole module + install_unit coreos-touch-run-agetty.service +} diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/99emergency-timeout/ignition-virtio-dump-journal.service b/config/overlay.d/05core/usr/lib/dracut/modules.d/99emergency-timeout/ignition-virtio-dump-journal.service new file mode 100644 index 0000000000000000000000000000000000000000..18a964cef98b94512ac5dd42e4990e8f0bf501fd --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/99emergency-timeout/ignition-virtio-dump-journal.service @@ -0,0 +1,15 @@ +[Unit] +Description=Dump journal to virtio port +ConditionPathExists=/etc/initrd-release +DefaultDependencies=false +ConditionVirtualization=|kvm +ConditionVirtualization=|qemu +Requires=systemd-journald.service +After=systemd-journald.service +After=basic.target + +[Service] +Type=oneshot +RemainAfterExit=yes +EnvironmentFile=/run/ignition.env +ExecStart=/usr/bin/ignition-virtio-dump-journal diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/99emergency-timeout/ignition-virtio-dump-journal.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/99emergency-timeout/ignition-virtio-dump-journal.sh new file mode 100644 index 0000000000000000000000000000000000000000..806b374e413df1d11fbca6dda9172c7be9ddf1f3 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/99emergency-timeout/ignition-virtio-dump-journal.sh @@ -0,0 +1,13 @@ +#!/bin/bash +set -euo pipefail + +port=/dev/virtio-ports/com.coreos.ignition.journal +if [ -e "${port}" ]; then + # Sync to backing filesystem before dumping what's there + journalctl --sync + journalctl -o json > "${port}" + # And this signals end of stream + echo '{}' > "${port}" +else + echo "Didn't find virtio port ${port}" +fi diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/99emergency-timeout/module-setup.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/99emergency-timeout/module-setup.sh new file mode 100644 index 0000000000000000000000000000000000000000..d8ba3c039df5ba525209932bc04f53202c000e0b --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/99emergency-timeout/module-setup.sh @@ -0,0 +1,23 @@ +#!/bin/bash +# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*- +# ex: ts=8 sw=4 sts=4 et filetype=sh + +install_unit_wants() { + local unit="$1"; shift + local target="$1"; shift + local instantiated="${1:-$unit}"; shift + inst_simple "$moddir/$unit" "$systemdsystemunitdir/$unit" + # note we `|| exit 1` here so we error out if e.g. the units are missing + systemctl -q --root="$initdir" add-wants "$target" "$instantiated" || exit 1 +} + +install() { + inst_multiple \ + cut \ + date + + inst_hook emergency 99 "${moddir}/timeout.sh" + + inst_script "$moddir/ignition-virtio-dump-journal.sh" "/usr/bin/ignition-virtio-dump-journal" + install_unit_wants ignition-virtio-dump-journal.service emergency.target +} diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/99emergency-timeout/timeout.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/99emergency-timeout/timeout.sh new file mode 100644 index 0000000000000000000000000000000000000000..85fb3d761de729d2679e269875adbafa0ed69744 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/99emergency-timeout/timeout.sh @@ -0,0 +1,99 @@ +# Before starting the emergency shell, prompt the user to press Enter. +# If they don't, reboot the system. +# +# Assumes /bin/sh is bash. + +# _wait_for_journalctl_to_stop will block until either: +# - no messages have appeared in journalctl for the past 5 seconds +# - 15 seconds have elapsed +_wait_for_journalctl_to_stop() { + local time_since_last_log=0 + + local time_started="$(date '+%s')" + local now="$(date '+%s')" + + while [ ${time_since_last_log} -lt 5 -a $((now-time_started)) -lt 15 ]; do + sleep 1 + + local last_log_timestamp="$(journalctl -e -n 1 -q -o short-unix | cut -d '.' -f 1)" + local now="$(date '+%s')" + + local time_since_last_log=$((now-last_log_timestamp)) + done +} + +_prompt_for_timeout() { + local timeout=300 + local interval=15 + + if [[ -e /.emergency-shell-confirmed ]]; then + return + fi + failed=$(systemctl --failed --no-legend --plain | cut -f 1 -d ' ') + if [ -n "${failed}" ]; then + # Something failed, suppress kernel logs so that it's more likely + # the useful bits from the journal are available. + dmesg --console-off + + # There's a couple straggler systemd messages. Wait until it's been 5 + # seconds since something was written to the journal. + _wait_for_journalctl_to_stop + + # Print Ignition logs + if echo ${failed} | grep -qFe 'ignition-'; then + cat < 0 ]]; do + local m=$(( $timeout / 60 )) + local s=$(( $timeout % 60 )) + local m_label="minutes" + if [[ $m = 1 ]]; then + m_label="minute" + fi + + if [[ $s != 0 ]]; then + echo -n -e "Press Enter for emergency shell or wait $m $m_label $s seconds for reboot. \r" + else + echo -n -e "Press Enter for emergency shell or wait $m $m_label for reboot. \r" + fi + + local anything + if read -t $interval anything; then + > /.emergency-shell-confirmed + return + fi + timeout=$(( $timeout - $interval )) + done + + echo -e "\nRebooting." + # This is not very nice, but since reboot.target likely conflicts with + # the existing goal target wrt the desired state of shutdown.target, + # there doesn't seem to be a better option. + systemctl reboot --force + exit 0 +} + +# If we're invoked from a dracut breakpoint rather than +# dracut-emergency.service, we won't have a controlling terminal and stdio +# won't be connected to it. Explicitly read/write /dev/console. +_prompt_for_timeout < /dev/console > /dev/console diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/99journal-conf/00-journal-log-forwarding.conf b/config/overlay.d/05core/usr/lib/dracut/modules.d/99journal-conf/00-journal-log-forwarding.conf new file mode 100644 index 0000000000000000000000000000000000000000..f66ea93fa98bc12542679350d4a21ed414aad7f9 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/99journal-conf/00-journal-log-forwarding.conf @@ -0,0 +1,12 @@ +[Journal] +# For now we are using kmsg for multiplexing output to +# multiple console devices during early boot. +# +# We do not want to use kmsg in the future as there may be sensitive +# ignition data that leaks to non-root users (by reading the kernel +# ring buffer using `dmesg`). In the future we will rely on kernel +# console multiplexing (link below) for this and will not use kmsg. +# + +ForwardToKMsg=yes +MaxLevelKMsg=info diff --git a/config/overlay.d/05core/usr/lib/dracut/modules.d/99journal-conf/module-setup.sh b/config/overlay.d/05core/usr/lib/dracut/modules.d/99journal-conf/module-setup.sh new file mode 100644 index 0000000000000000000000000000000000000000..e6626b28995f01467e6df940ef93f58f470f0d0c --- /dev/null +++ b/config/overlay.d/05core/usr/lib/dracut/modules.d/99journal-conf/module-setup.sh @@ -0,0 +1,12 @@ +#!/bin/bash +# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*- +# ex: ts=8 sw=4 sts=4 et filetype=sh + +depends() { + echo systemd +} + +install() { + inst_simple "$moddir/00-journal-log-forwarding.conf" \ + "/etc/systemd/journald.conf.d/00-journal-log-forwarding.conf" +} diff --git a/config/overlay.d/05core/usr/lib/sysctl.d/10-coreos-ratelimit-kmsg.conf b/config/overlay.d/05core/usr/lib/sysctl.d/10-coreos-ratelimit-kmsg.conf new file mode 100644 index 0000000000000000000000000000000000000000..e605a61c6cb5aecef2e77bd4645ed0e2cd8388bb --- /dev/null +++ b/config/overlay.d/05core/usr/lib/sysctl.d/10-coreos-ratelimit-kmsg.conf @@ -0,0 +1,3 @@ +# See also 10coreos-sysctl dracut module, which turns off ratelimiting in the +# initrd. +kernel.printk_devkmsg = ratelimit diff --git a/config/overlay.d/05core/usr/lib/systemd/journald.conf.d/10-coreos-persistent.conf b/config/overlay.d/05core/usr/lib/systemd/journald.conf.d/10-coreos-persistent.conf new file mode 100644 index 0000000000000000000000000000000000000000..7910c16785a23f5efff7c90da61dc1f781ae4819 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/systemd/journald.conf.d/10-coreos-persistent.conf @@ -0,0 +1,6 @@ +# Hardcode persistent journal by default. journald has this "auto" behaviour +# that only makes logs persistent if `/var/log/journal` exists, which it won't +# on first boot because `/var` isn't fully populated. We should be able to get +# rid of this once we move to sysusers and create the dir in the initrd. +[Journal] +Storage=persistent diff --git a/config/overlay.d/05core/usr/lib/systemd/system-generators/coreos-boot-mount-generator b/config/overlay.d/05core/usr/lib/systemd/system-generators/coreos-boot-mount-generator new file mode 100644 index 0000000000000000000000000000000000000000..c1beaebbcbc9dcce5d8fe16299479d0246146b43 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/systemd/system-generators/coreos-boot-mount-generator @@ -0,0 +1,79 @@ +#!/bin/bash +export PATH="/usr/bin:/usr/sbin:${PATH}" +set -euo pipefail + +. /usr/lib/coreos/generator-lib.sh + +# Turn out if you boot with "root=..." $UNIT_DIR is not writable. +[ -w "${UNIT_DIR}" ] || { + echo "skipping coreos-boot-mount-generator: ${UNIT_DIR} is not writable" + exit 0 +} + +# If there's already an /etc/fstab entries for /boot, then this is is a non-FCOS +# system, likely RHCOS pre-4.3 (which still used Anaconda). In that case, we +# don't want to overwrite what the systemd-fstab-generator will do. +if findmnt --fstab /boot &>/dev/null; then + exit 0 +fi + +# Don't create mount units for /boot on live systems. +# ConditionPathExists won't work here because conditions don't affect +# the dependency on the underlying device unit. +if [ -f /run/ostree-live ]; then + exit 0 +fi + +add_wants() { + local name="$1"; shift + local wants_dir="${UNIT_DIR}/local-fs.target.wants" + mkdir -p "${wants_dir}" + ln -sf "../${name}" "${wants_dir}/${name}" +} + +# Generate mount units that work with device mapper. The traditional +# device unit (dev-disk-by\x2dlabel...) does not work since it is not the +# device that systemd will fsck. This code ensures that if the label +# is backed by a device-mapper target the dev-mapper.*.device is used. +mk_mount() { + local mount_pt="${1}"; shift + local path="${1}"; shift + local options="${1}"; shift + + local devservice=$(systemd-escape -p ${path} --suffix=service) + local unit_name=$(systemd-escape -p ${mount_pt} --suffix=mount) + + cat > "${UNIT_DIR}/${unit_name}" < "${out_dir}/10-autologin.conf" < /etc/sysctl.d/20-coreos-autologin-kernel-printk.conf +# Raise console message logging level from DEBUG (7) to WARNING (4) +# so that kernel debug message don't get interspersed on the console +# that +# may frustrate a user trying to interactively do an install with +# nmtui and coreos-installer. +kernel.printk=4 +EOF +} + +write_interactive_live_motd() { + # Write motd to a tmp file and not directly to /etc/motd because + # SELinux denies write from init_t to etc_t + cat < /run/interactive-live-motd + +########################################################################### +Welcome to the NestOS live environment. This system is running completely +from memory, making it a good candidate for hardware discovery and +installing persistently to disk. Here is an example of running an install +to disk via coreos-installer: + +sudo coreos-installer install /dev/sda \\ + --ignition-file config.ign + +You may configure networking via 'sudo nmcli' or 'sudo nmtui' and have +that configuration persist into the installed system by passing the +'--copy-network' argument to 'coreos-installer install'. Please run +'coreos-installer install --help' for more information on the possible +install options. +########################################################################### + +EOF + # Create coreos-cp-interactive-live-motd.service to copy over the motd in + # place. Note this intentionally overwrites the existing motd, which is + # blank on FCOS and populated on RHCOS. + service="coreos-cp-interactive-live-motd.service" + cat < "${UNIT_DIR}/${service}" +# generated by coreos-liveiso-autologin-generator +[Unit] +Description=Copy CoreOS Interactive Live MOTD +Before=systemd-user-sessions.service +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/bin/cp -v /run/interactive-live-motd /etc/motd +EOF + mkdir -p "${UNIT_DIR}/multi-user.target.wants" + ln -sf "../${service}" "${UNIT_DIR}/multi-user.target.wants/" +} + +# Only allow automatic autologin on live systems +if [ ! -e /run/ostree-live ]; then + exit 0 +fi + +# Autologin on ISO boots but not PXE boots. The only way to tell the +# difference is a kernel argument. +if ! have_karg coreos.liveiso; then + exit 0 +fi + +# If the user supplied an Ignition config, they have the ability to enable +# autologin themselves. Don't automatically render them insecure, since +# they might be running in production and booting via e.g. IPMI. + +ign_usercfg_msg=$(journalctl -q MESSAGE_ID=57124006b5c94805b77ce473e92a8aeb IGNITION_CONFIG_TYPE=user) +if [ -n "${ign_usercfg_msg}" ]; then + exit 0 +fi + +write_dropin "getty@.service" "--noclear" +# Also autologin on serial console if someone enables that +write_dropin "serial-getty@.service" "--keep-baud 115200,38400,9600" + +# When the installer runs a lot of things happen on the system (audit +# messages from running via sudo, re-reading partition table messages, +# mounting filesystem messages, etc.). Quieting the verbosity of the +# kernel console will help us keep our sanity. +quiet_kernel_console_messages + + +# Write an motd that will let the user know about the live environment +# and what is possible. +write_interactive_live_motd diff --git a/config/overlay.d/05core/usr/lib/systemd/system-preset/40-coreos-systemd.preset b/config/overlay.d/05core/usr/lib/systemd/system-preset/40-coreos-systemd.preset new file mode 100644 index 0000000000000000000000000000000000000000..a242ebaaa1c0c69bee0257ac2af230ce9a341b8c --- /dev/null +++ b/config/overlay.d/05core/usr/lib/systemd/system-preset/40-coreos-systemd.preset @@ -0,0 +1,16 @@ +# This file contains overrides for systemd services that are +# enabled by default, but conflict with things we ship. + +# We don't have swap by default, and systemd-oomd hard requires it. +disable systemd-oomd.service + +# Disable systemd-firstboot because it conflicts with Ignition. +# In most cases this is handled via the remove-from-packages +# bits in the manifest (ignition-and-ostree.yaml), but +# we want to support overlaying builds of systemd from git. +disable systemd-firstboot.service + +# This hasn't been tested with ostree/rpm-ostree and heavily overlaps +# with the latter. Preemptively disable the service; it will hopefully +# be subpackaged though for Fedora. +disable systemd-sysext.service diff --git a/config/overlay.d/05core/usr/lib/systemd/system-preset/40-coreos.preset b/config/overlay.d/05core/usr/lib/systemd/system-preset/40-coreos.preset new file mode 100644 index 0000000000000000000000000000000000000000..15100944fa3675cf3dcb223b6b3597d9838389cf --- /dev/null +++ b/config/overlay.d/05core/usr/lib/systemd/system-preset/40-coreos.preset @@ -0,0 +1,23 @@ + + +enable console-login-helper-messages-gensnippet-os-release.service +enable console-login-helper-messages-gensnippet-ssh-keys.service + +enable coreos-update-ca-trust.service + +enable coreos-ignition-firstboot-complete.service +# Boot checkin services for cloud providers. +enable afterburn-checkin.service +enable afterburn-firstboot-checkin.service +# Target to write SSH key snippets from cloud providers. +enable afterburn-sshkeys.target +# Update agent +enable zincati.service +# Testing aid +enable coreos-liveiso-success.service +# See bootupd.yaml +enable bootupd.socket + + +enable rtas_errd.service +enable clevis-luks-askpass.path diff --git a/config/overlay.d/05core/usr/lib/systemd/system/coreos-ignition-firstboot-complete.service b/config/overlay.d/05core/usr/lib/systemd/system/coreos-ignition-firstboot-complete.service new file mode 100644 index 0000000000000000000000000000000000000000..42adf1e6b061f8ac616a3c1395362dcbbb4b9857 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/systemd/system/coreos-ignition-firstboot-complete.service @@ -0,0 +1,21 @@ +[Unit] +Description=CoreOS Mark Ignition Boot Complete +Documentation=https://docs.fedoraproject.org/en-US/fedora-coreos/ +ConditionKernelCommandLine=ignition.firstboot +ConditionPathExists=!/run/ostree-live +RequiresMountsFor=/boot + +[Service] +Type=oneshot +RemainAfterExit=yes +# The MountFlags=slave is so we remount /boot temporarily writable; +# see https://github.com/ostreedev/ostree/issues/1265 for the bigger picture. +# This option creates a new mount namespace; from the point of view of +# everything else, /boot stays readonly. We only have a transient writable mount +# for the lifetime of the unit. +MountFlags=slave +ExecStart=/usr/libexec/coreos-ignition-firstboot-complete + +[Install] +# Part of basic.target so this happens early on in firstboot +WantedBy=basic.target diff --git a/config/overlay.d/05core/usr/lib/systemd/system/coreos-liveiso-success.service b/config/overlay.d/05core/usr/lib/systemd/system/coreos-liveiso-success.service new file mode 100644 index 0000000000000000000000000000000000000000..e7dc06182225e2d016e4c5c0b6f4a6ac17538067 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/systemd/system/coreos-liveiso-success.service @@ -0,0 +1,28 @@ +# This is used by our test framework in coreos-assembler +# since for the "live ISO without Ignition" case we +# don't have an easy way to test it. +[Unit] +Description=CoreOS Live ISO virtio success +Documentation=https://github.com/coreos/fedora-coreos-config +# Only run on the Live ISO, and only if there's no Ignition config +ConditionKernelCommandLine=coreos.liveiso +ConditionPathExists=!/config.ign +ConditionVirtualization=|kvm +ConditionVirtualization=|qemu +# Start running late to help ensure that the below conditional works +After=systemd-user-sessions.service +ConditionPathExists=/dev/virtio-ports/coreos.liveiso-success + +[Service] +Type=simple +# https://stackoverflow.com/questions/44358723/systemd-unit-file-problems-with-tr +IgnoreSIGPIPE=false +# See https://cgit.freedesktop.org/systemd/systemd/plain/src/systemd/sd-messages.h for the MESSAGE_ID source. +# The logic here is that we're doing a streaming journalctl query (-f to follow) +# and the `| head` bit will cause the pipeline to wait until at least one line is +# emitted, which will happen when a user login starts. We then just write a static +# knows how to read. +ExecStart=/bin/sh -c 'journalctl -b -q -f --no-tail -o cat -u systemd-logind.service MESSAGE_ID=8d45620c1a4348dbb17410da57c60c66 | head -1; echo coreos-liveiso-success > /dev/virtio-ports/coreos.liveiso-success' + +[Install] +WantedBy=multi-user.target diff --git a/config/overlay.d/05core/usr/lib/systemd/system/coreos-update-ca-trust.service b/config/overlay.d/05core/usr/lib/systemd/system/coreos-update-ca-trust.service new file mode 100644 index 0000000000000000000000000000000000000000..d5e811f85b9224fe20de7af2fbec642e74c2cdf4 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/systemd/system/coreos-update-ca-trust.service @@ -0,0 +1,31 @@ +# This service is currently specific to Fedora CoreOS, +# but we may want to add it to the base OS in the future. +# The idea here is to allow users to just drop in CA roots +# via Ignition without having to know to run the special +# update command. +[Unit] +Description=Run update-ca-trust +ConditionFirstBoot=true +# All services which use ConditionFirstBoot=yes should use +# Before=first-boot-complete.target, which is a target that +# was introduced in https://github.com/systemd/systemd/issues/4511 +# and hasn't propagated everywhere yet. Once the target propagates +# everywhere, we can drop the systemd-machine-id-commit.service +# from the Before= line. +Before=first-boot-complete.target systemd-machine-id-commit.service +Wants=first-boot-complete.target +ConditionDirectoryNotEmpty=/etc/pki/ca-trust/source/anchors/ +# We want to run quite early, in particular before anything +# that may speak TLS to external services. In the future, +# it may make sense to do this in the initramfs too. +DefaultDependencies=no +After=local-fs.target +Requires=local-fs.target + +[Service] +ExecStart=/usr/bin/update-ca-trust extract +Type=oneshot +RemainAfterExit=yes + +[Install] +WantedBy=basic.target diff --git a/config/overlay.d/05core/usr/lib/systemd/system/emergency.service.d/coreos-sulogin-force.conf b/config/overlay.d/05core/usr/lib/systemd/system/emergency.service.d/coreos-sulogin-force.conf new file mode 100644 index 0000000000000000000000000000000000000000..7300c8593902dff6ef746caee742c7021873f8b9 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/systemd/system/emergency.service.d/coreos-sulogin-force.conf @@ -0,0 +1,7 @@ + +# https://github.com/systemd/systemd/pull/10397 +# We want things like `systemd.unit=emergency.target` and `single` on the +# kernel command line to just work even with our locked root account. +# This file is used as an override for both emergency.target and rescue.target. +[Service] +Environment=SYSTEMD_SULOGIN_FORCE=1 diff --git a/config/overlay.d/05core/usr/lib/systemd/system/rescue.service.d/coreos-sulogin-force.conf b/config/overlay.d/05core/usr/lib/systemd/system/rescue.service.d/coreos-sulogin-force.conf new file mode 100644 index 0000000000000000000000000000000000000000..7300c8593902dff6ef746caee742c7021873f8b9 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/systemd/system/rescue.service.d/coreos-sulogin-force.conf @@ -0,0 +1,7 @@ + +# https://github.com/systemd/systemd/pull/10397 +# We want things like `systemd.unit=emergency.target` and `single` on the +# kernel command line to just work even with our locked root account. +# This file is used as an override for both emergency.target and rescue.target. +[Service] +Environment=SYSTEMD_SULOGIN_FORCE=1 diff --git a/config/overlay.d/05core/usr/lib/systemd/system/systemd-firstboot.service.d/fcos-disable.conf b/config/overlay.d/05core/usr/lib/systemd/system/systemd-firstboot.service.d/fcos-disable.conf new file mode 100644 index 0000000000000000000000000000000000000000..fc7f00518b3a66b3df8a49a804502abbb86ee0ab --- /dev/null +++ b/config/overlay.d/05core/usr/lib/systemd/system/systemd-firstboot.service.d/fcos-disable.conf @@ -0,0 +1,5 @@ +# See the comment in 40-coreos-systemd.preset; we're +# keeping this even stronger disable override for now, +# but it may not really be necessary. +[Unit] +ConditionPathExists=/run/nosuchfile diff --git a/config/overlay.d/05core/usr/lib/udev/rules.d/65-gce-disk-naming.rules b/config/overlay.d/05core/usr/lib/udev/rules.d/65-gce-disk-naming.rules new file mode 100644 index 0000000000000000000000000000000000000000..e19c1c5b91040c470743915049f1fb9d75380883 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/udev/rules.d/65-gce-disk-naming.rules @@ -0,0 +1,38 @@ +# Copyright 2016 Google Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# Name the attached disks as the specified by deviceName. + +ACTION!="add|change", GOTO="gce_disk_naming_end" +SUBSYSTEM!="block", GOTO="gce_disk_naming_end" + +# SCSI naming +KERNEL=="sd*|vd*", ENV{ID_VENDOR}=="Google", IMPORT{program}="scsi_id --export --whitelisted -d $tempnode" + +# NVME naming +KERNEL=="nvme0n1*", ENV{ID_VENDOR}=="Google", ENV{ID_SERIAL_SHORT}="local-nvme-ssd-0" +KERNEL=="nvme0n2*", ENV{ID_VENDOR}=="Google", ENV{ID_SERIAL_SHORT}="local-nvme-ssd-1" +KERNEL=="nvme0n3*", ENV{ID_VENDOR}=="Google", ENV{ID_SERIAL_SHORT}="local-nvme-ssd-2" +KERNEL=="nvme0n4*", ENV{ID_VENDOR}=="Google", ENV{ID_SERIAL_SHORT}="local-nvme-ssd-3" +KERNEL=="nvme0n5*", ENV{ID_VENDOR}=="Google", ENV{ID_SERIAL_SHORT}="local-nvme-ssd-4" +KERNEL=="nvme0n6*", ENV{ID_VENDOR}=="Google", ENV{ID_SERIAL_SHORT}="local-nvme-ssd-5" +KERNEL=="nvme0n7*", ENV{ID_VENDOR}=="Google", ENV{ID_SERIAL_SHORT}="local-nvme-ssd-6" +KERNEL=="nvme0n8*", ENV{ID_VENDOR}=="Google", ENV{ID_SERIAL_SHORT}="local-nvme-ssd-7" +KERNEL=="nvme*", ENV{ID_VENDOR}=="Google", ENV{ID_SERIAL}="Google_EphemeralDisk_$env{ID_SERIAL_SHORT}" + +# Symlinks +KERNEL=="sd*|vd*|nvme*", ENV{DEVTYPE}=="disk", ENV{ID_VENDOR}=="Google", SYMLINK+="disk/by-id/google-$env{ID_SERIAL_SHORT}" +KERNEL=="sd*|vd*|nvme*", ENV{DEVTYPE}=="partition", ENV{ID_VENDOR}=="Google", SYMLINK+="disk/by-id/google-$env{ID_SERIAL_SHORT}-part%n" + +LABEL="gce_disk_naming_end" diff --git a/config/overlay.d/05core/usr/lib/udev/rules.d/68-azure-sriov-nm-unmanaged.rules b/config/overlay.d/05core/usr/lib/udev/rules.d/68-azure-sriov-nm-unmanaged.rules new file mode 100644 index 0000000000000000000000000000000000000000..59cf73bb0a424596836dd13efb4e0ea2f4d6aefd --- /dev/null +++ b/config/overlay.d/05core/usr/lib/udev/rules.d/68-azure-sriov-nm-unmanaged.rules @@ -0,0 +1,4 @@ +# Accelerated Networking on Azure exposes a new SRIOV interface to the VM. +# This interface is transparently bonded to the synthetic interface, +# so NetworkManager should just ignore any SRIOV interfaces. +SUBSYSTEM=="net", DRIVERS=="hv_pci", ACTION=="add", ENV{NM_UNMANAGED}="1" diff --git a/config/overlay.d/05core/usr/lib/udev/rules.d/90-coreos-device-mapper.rules b/config/overlay.d/05core/usr/lib/udev/rules.d/90-coreos-device-mapper.rules new file mode 100644 index 0000000000000000000000000000000000000000..385f262437206bb464771e16b899e1d543b96366 --- /dev/null +++ b/config/overlay.d/05core/usr/lib/udev/rules.d/90-coreos-device-mapper.rules @@ -0,0 +1,27 @@ +# CoreOS-specific symlinks for dm-multipath filesystem labels, +# used for `label=boot` and `label=root`. + +ACTION=="remove", GOTO="dm_label_end" +SUBSYSTEM!="block", GOTO="dm_label_end" +KERNEL!="dm-*", GOTO="dm_label_end" + +# Ensure that the device mapper target is active +ENV{DM_SUSPENDED}=="1", GOTO="dm_label_end" + +# Only act on filesystems. This should prevent layered devices +# such as Raid on Multipath devices from appearing. +ENV{ID_FS_USAGE}!="filesystem", GOTO="dm_label_end" + +# And if the filesystem doesn't have a label+uuid, we're done. +ENV{ID_FS_LABEL_ENC}!="?*", GOTO="dm_label_end" +ENV{ID_FS_UUID_ENC}!="?*", GOTO="dm_label_end" + +# Setup up Multipath labels and UUID's. Match on DM_UUID which +# is stable regardless of whether friendly names are used or not. +# 66-kpartx.rules use DM_UUID to match for linear mappings on multipath +# targets. +ENV{DM_UUID}=="*mpath*" \ + , SYMLINK+="disk/by-label/dm-mpath-$env{ID_FS_LABEL_ENC}" \ + , SYMLINK+="disk/by-uuid/dm-mpath-$env{ID_FS_UUID_ENC}" + +LABEL="dm_label_end" diff --git a/config/overlay.d/05core/usr/libexec/coreos-ignition-firstboot-complete b/config/overlay.d/05core/usr/libexec/coreos-ignition-firstboot-complete new file mode 100644 index 0000000000000000000000000000000000000000..b9d105c25b611f691709f11549580277feafff57 --- /dev/null +++ b/config/overlay.d/05core/usr/libexec/coreos-ignition-firstboot-complete @@ -0,0 +1,18 @@ +#!/bin/bash +set -euo pipefail + +mount -o remount,rw /boot + +if [[ $(uname -m) = s390x ]]; then + zipl +fi + +# We're done provisioning. Remove the whole /boot/ignition directory if present, +# which may include a baked Ignition config. See + +rm -rf /boot/ignition + +# Regarding the lack of `-f` for rm ; we should have only run if GRUB detected +# this file. Fail if we are unable to remove it, rather than risking rerunning +# Ignition at next boot. +rm /boot/ignition.firstboot diff --git a/config/overlay.d/05core/usr/share/licenses/fedora-coreos-config/LICENSE b/config/overlay.d/05core/usr/share/licenses/fedora-coreos-config/LICENSE new file mode 100644 index 0000000000000000000000000000000000000000..660c8228beb2050c8ee1ab70192f9c474ab00c97 --- /dev/null +++ b/config/overlay.d/05core/usr/share/licenses/fedora-coreos-config/LICENSE @@ -0,0 +1,21 @@ +Copyright 2021 NestOS Authors. + +Permission is hereby granted, free of charge, to any person obtaining +a copy of this software and associated documentation files (the +"Software"), to deal in the Software without restriction, including +without limitation the rights to use, copy, modify, merge, publish, +distribute, sublicense, and/or sell copies of the Software, and to +permit persons to whom the Software is furnished to do so, subject to +the following conditions: + +The above copyright notice and this permission notice shall be included +in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. +IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY +CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, +TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE +SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + diff --git a/config/overlay.d/05core/usr/share/licenses/fedora-coreos-config/README.md b/config/overlay.d/05core/usr/share/licenses/fedora-coreos-config/README.md new file mode 100644 index 0000000000000000000000000000000000000000..c966244dfd822ad33e309edd53d68b7463f63a22 --- /dev/null +++ b/config/overlay.d/05core/usr/share/licenses/fedora-coreos-config/README.md @@ -0,0 +1,16 @@ +# nestos-config + +Today most components of NestOS are built as RPMs; this +is the main exception. nest-config is "architecture-independent glue" +and the overhead of building an RPM for each change is onerous. + +It's also *the* central point of management (e.g. it contains lockfiles), so having it be +an RPM too would become circular. Instead, coreos-assembler directly consumes it. + + +From a running system, to find the source commit use: +``` +$ rpm-ostree status -b --json | jq -r '.deployments[0]."base-commit-meta"."coreos-assembler.config-gitrev"' +c8dbed9ce223bf86737c82dd763670c8a34e950f +$ +``` diff --git a/config/overlay.d/08nouveau/etc/modprobe.d/blacklist-nouveau.conf b/config/overlay.d/08nouveau/etc/modprobe.d/blacklist-nouveau.conf new file mode 100644 index 0000000000000000000000000000000000000000..2ea53bda3a0d80cba884a20e9e2a4a5814197da7 --- /dev/null +++ b/config/overlay.d/08nouveau/etc/modprobe.d/blacklist-nouveau.conf @@ -0,0 +1,2 @@ + +blacklist nouveau diff --git a/config/overlay.d/09misc/etc/sysconfig/README b/config/overlay.d/09misc/etc/sysconfig/README new file mode 100644 index 0000000000000000000000000000000000000000..4d8d9bb9304b579697fc757d5c9eaa64f0b98471 --- /dev/null +++ b/config/overlay.d/09misc/etc/sysconfig/README @@ -0,0 +1,9 @@ +This directory is a legacy of Red Hat Linux days. +Do not write new software that uses configuration +files here. Instead your software should use a regular +config file in `/etc/foo.conf`, a configuration directory +such as `/etc/foo/`. + +Where appropriate, it's also best practice to use "systemd style config" +where default config files live in `/usr/lib/foo` that can be +overridden in `/etc` and `/run`. diff --git a/config/overlay.d/09misc/usr/lib/tmpfiles.d/coreos-fix-etc-ownership.conf b/config/overlay.d/09misc/usr/lib/tmpfiles.d/coreos-fix-etc-ownership.conf new file mode 100644 index 0000000000000000000000000000000000000000..8d52ab0603f3d62d696bc83cb323dc8034b481b9 --- /dev/null +++ b/config/overlay.d/09misc/usr/lib/tmpfiles.d/coreos-fix-etc-ownership.conf @@ -0,0 +1,11 @@ + +# Fix mode (chmod g-w) for existing files on the system during boot +#z /etc/crypto-policies/state/current 644 root root +#z /etc/group 644 root root +#z /etc/group- 644 root root +#z /etc/iscsi/initiatorname.iscsi 644 root root +#z /etc/passwd 644 root root +#z /etc/passwd- 644 root root +#z /etc/selinux/config 644 root root +#z /etc/ssh/sshd_config.d/40-disable-passwords.conf 644 root root +#z /etc/systemd/dont-synthesize-nobody 644 root root diff --git a/config/overlay.d/12kdump/usr/lib/systemd/system/kdump.service.d/remount-boot.conf b/config/overlay.d/12kdump/usr/lib/systemd/system/kdump.service.d/remount-boot.conf new file mode 100644 index 0000000000000000000000000000000000000000..10d437d8e01539a89526873906ce096962401375 --- /dev/null +++ b/config/overlay.d/12kdump/usr/lib/systemd/system/kdump.service.d/remount-boot.conf @@ -0,0 +1,9 @@ + +# `/boot` is read-only, but `kdump.service` wants to +# places its generated initramfs alongside the default +# initramfs under `/boot/ostree`. +# Until `kdump` gains the ability to place its initramfs +# elsewhere, temporarily remount `/boot` read-write before +# the `kdump` initramfs is generated. +[Service] +ExecStartPre=/usr/bin/mount -o remount,rw /boot diff --git a/config/overlay.d/14NetworkManager-plugins/usr/lib/NetworkManager/conf.d/10-disable-default-plugins.conf b/config/overlay.d/14NetworkManager-plugins/usr/lib/NetworkManager/conf.d/10-disable-default-plugins.conf new file mode 100644 index 0000000000000000000000000000000000000000..3182f6702711be53476df04d635a4d21750b0394 --- /dev/null +++ b/config/overlay.d/14NetworkManager-plugins/usr/lib/NetworkManager/conf.d/10-disable-default-plugins.conf @@ -0,0 +1,9 @@ +# Stop NetworkManager from trying to load the ifcfg-rh plugin by default, +# which we don't ship. This actually disables all default plugins, of which +# ifcfg-rh is currently the only one. +# +# Note that we must do this for now because `-=` syntax doesn't work +# with compiled-in defaults. Proposed upstream fix: +# https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/491 +[main] +plugins= diff --git a/config/overlay.d/15fcos/etc/ssh/sshd_config.d/40-disable-passwords.conf b/config/overlay.d/15fcos/etc/ssh/sshd_config.d/40-disable-passwords.conf new file mode 100644 index 0000000000000000000000000000000000000000..f0faa508601208cc2034ec0c80958725baa78f6d --- /dev/null +++ b/config/overlay.d/15fcos/etc/ssh/sshd_config.d/40-disable-passwords.conf @@ -0,0 +1,4 @@ +# Disable password logins by default. +# This file must sort before 50-redhat.conf, which enables +# PasswordAuthentication. +PasswordAuthentication no diff --git a/config/overlay.d/15fcos/usr/lib/dracut/modules.d/50ignition-conf-fcos/20-aws-nm-cloud-setup.ign b/config/overlay.d/15fcos/usr/lib/dracut/modules.d/50ignition-conf-fcos/20-aws-nm-cloud-setup.ign new file mode 100644 index 0000000000000000000000000000000000000000..0d39b1686c192c34ab2a5ada5e5acceb73acebfe --- /dev/null +++ b/config/overlay.d/15fcos/usr/lib/dracut/modules.d/50ignition-conf-fcos/20-aws-nm-cloud-setup.ign @@ -0,0 +1,16 @@ +{ + "ignition": { + "version": "3.0.0" + }, + "storage": { + "files": [ + { + "path": "/etc/systemd/system/nm-cloud-setup.service.d/env-aws.conf", + "contents": { + "source": "data:,%5BService%5D%0AEnvironment%3DNM_CLOUD_SETUP_EC2%3Dyes%0A" + }, + "mode": 420 + } + ] + } +} diff --git a/config/overlay.d/15fcos/usr/lib/dracut/modules.d/50ignition-conf-fcos/20-azure-nm-cloud-setup.ign b/config/overlay.d/15fcos/usr/lib/dracut/modules.d/50ignition-conf-fcos/20-azure-nm-cloud-setup.ign new file mode 100644 index 0000000000000000000000000000000000000000..ed2a5c5ac8e17c1a85209dce72acadbcffec60c6 --- /dev/null +++ b/config/overlay.d/15fcos/usr/lib/dracut/modules.d/50ignition-conf-fcos/20-azure-nm-cloud-setup.ign @@ -0,0 +1,16 @@ +{ + "ignition": { + "version": "3.0.0" + }, + "storage": { + "files": [ + { + "path": "/etc/systemd/system/nm-cloud-setup.service.d/env-azure.conf", + "contents": { + "source": "data:,%5BService%5D%0AEnvironment%3DNM_CLOUD_SETUP_AZURE%3Dyes%0A" + }, + "mode": 420 + } + ] + } +} diff --git a/config/overlay.d/15fcos/usr/lib/dracut/modules.d/50ignition-conf-fcos/20-gcp-nm-cloud-setup.ign b/config/overlay.d/15fcos/usr/lib/dracut/modules.d/50ignition-conf-fcos/20-gcp-nm-cloud-setup.ign new file mode 100644 index 0000000000000000000000000000000000000000..22966dd36b0a1c5190b4d6430a62b4648b85541b --- /dev/null +++ b/config/overlay.d/15fcos/usr/lib/dracut/modules.d/50ignition-conf-fcos/20-gcp-nm-cloud-setup.ign @@ -0,0 +1,16 @@ +{ + "ignition": { + "version": "3.0.0" + }, + "storage": { + "files": [ + { + "path": "/etc/systemd/system/nm-cloud-setup.service.d/env-gcp.conf", + "contents": { + "source": "data:,%5BService%5D%0AEnvironment%3DNM_CLOUD_SETUP_GCP%3Dyes%0A" + }, + "mode": 420 + } + ] + } +} diff --git a/config/overlay.d/15fcos/usr/lib/dracut/modules.d/50ignition-conf-fcos/30-afterburn-sshkeys-core.ign b/config/overlay.d/15fcos/usr/lib/dracut/modules.d/50ignition-conf-fcos/30-afterburn-sshkeys-core.ign new file mode 100644 index 0000000000000000000000000000000000000000..98fc47ad332156ce52fa3907a9335660d94833a7 --- /dev/null +++ b/config/overlay.d/15fcos/usr/lib/dracut/modules.d/50ignition-conf-fcos/30-afterburn-sshkeys-core.ign @@ -0,0 +1,13 @@ +{ + "ignition": { + "version": "3.0.0" + }, + "systemd": { + "units": [ + { + "enabled": true, + "name": "afterburn-sshkeys@core.service" + } + ] + } +} diff --git a/config/overlay.d/15fcos/usr/lib/dracut/modules.d/50ignition-conf-fcos/README.md b/config/overlay.d/15fcos/usr/lib/dracut/modules.d/50ignition-conf-fcos/README.md new file mode 100644 index 0000000000000000000000000000000000000000..a9a2be9129e7e30b71842a52dd3252bebe94ae16 --- /dev/null +++ b/config/overlay.d/15fcos/usr/lib/dracut/modules.d/50ignition-conf-fcos/README.md @@ -0,0 +1 @@ +FCOS enables `afterburn-sshkeys@core.service` from `30-afterburn-sshkeys-core.ign`, allowing the user to prevent Ignition from enabling the service with a user config if the user wants to change the username. Unlike FCOS, RHCOS doesn't fetch SSH keys from cloud providers and thus doesn't need `afterburn-sshkeys@core.service`. diff --git a/config/overlay.d/15fcos/usr/lib/dracut/modules.d/50ignition-conf-fcos/module-setup.sh b/config/overlay.d/15fcos/usr/lib/dracut/modules.d/50ignition-conf-fcos/module-setup.sh new file mode 100644 index 0000000000000000000000000000000000000000..8e9f9d923dab24aaca440d03ade8165a2cdeb469 --- /dev/null +++ b/config/overlay.d/15fcos/usr/lib/dracut/modules.d/50ignition-conf-fcos/module-setup.sh @@ -0,0 +1,31 @@ +#!/bin/bash +# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*- +# ex: ts=8 sw=4 sts=4 et filetype=sh + +depends() { + echo ignition +} + +install() { + mkdir -p "$initdir/usr/lib/ignition/base.d" + mkdir -p "$initdir/usr/lib/ignition/base.platform.d" + + # Common entries + inst "$moddir/30-afterburn-sshkeys-core.ign" \ + "/usr/lib/ignition/base.d/30-afterburn-sshkeys-core.ign" + + # Platform specific: aws + mkdir -p "$initdir/usr/lib/ignition/base.platform.d/aws" + inst "$moddir/20-aws-nm-cloud-setup.ign" \ + "/usr/lib/ignition/base.platform.d/aws/20-aws-nm-cloud-setup.ign" + + # Platform specific: azure + mkdir -p "$initdir/usr/lib/ignition/base.platform.d/azure" + inst "$moddir/20-azure-nm-cloud-setup.ign" \ + "/usr/lib/ignition/base.platform.d/azure/20-azure-nm-cloud-setup.ign" + + # Platform specific: gcp + mkdir -p "$initdir/usr/lib/ignition/base.platform.d/gcp" + inst "$moddir/20-gcp-nm-cloud-setup.ign" \ + "/usr/lib/ignition/base.platform.d/gcp/20-gcp-nm-cloud-setup.ign" +} diff --git a/config/overlay.d/15fcos/usr/lib/motd.d/tracker.motd b/config/overlay.d/15fcos/usr/lib/motd.d/tracker.motd new file mode 100644 index 0000000000000000000000000000000000000000..837cc57ccfbfbbb07cfe22f4541f00d890ec44fe --- /dev/null +++ b/config/overlay.d/15fcos/usr/lib/motd.d/tracker.motd @@ -0,0 +1,3 @@ +Tracker: https://github.com/coreos/fedora-coreos-tracker +Discuss: https://discussion.fedoraproject.org/c/server/coreos/ + diff --git a/config/overlay.d/15fcos/usr/lib/systemd/system-preset/45-fcos.preset b/config/overlay.d/15fcos/usr/lib/systemd/system-preset/45-fcos.preset new file mode 100644 index 0000000000000000000000000000000000000000..eec4287d49940331f0ceae6339367ce67a127fac --- /dev/null +++ b/config/overlay.d/15fcos/usr/lib/systemd/system-preset/45-fcos.preset @@ -0,0 +1,9 @@ +# User metrics client +enable fedora-coreos-pinger.service +# Provide information if no ignition is provided +enable coreos-check-ignition-config.service +enable coreos-check-ssh-keys.service +# Check if cgroupsv1 is still being used +enable coreos-check-cgroups.service +# Clean up injected Ignition config in /boot on upgrade +enable coreos-cleanup-ignition-config.service diff --git a/config/overlay.d/15fcos/usr/lib/systemd/system/coreos-check-cgroups.service b/config/overlay.d/15fcos/usr/lib/systemd/system/coreos-check-cgroups.service new file mode 100644 index 0000000000000000000000000000000000000000..18e4b85ad7501bede7a66bba73ffff0fc6c6b5ed --- /dev/null +++ b/config/overlay.d/15fcos/usr/lib/systemd/system/coreos-check-cgroups.service @@ -0,0 +1,11 @@ +# This service is used for printing a message if +# cgroups v1 is still being used +[Unit] +Description=Check if cgroupsv1 is still being used +ConditionControlGroupController=v1 +[Service] +Type=oneshot +ExecStart=/usr/libexec/coreos-check-cgroups +RemainAfterExit=yes +[Install] +WantedBy=multi-user.target diff --git a/config/overlay.d/15fcos/usr/lib/systemd/system/coreos-check-ignition-config.service b/config/overlay.d/15fcos/usr/lib/systemd/system/coreos-check-ignition-config.service new file mode 100644 index 0000000000000000000000000000000000000000..1a91853d6bcbc071fe83b34c77899d2d95604021 --- /dev/null +++ b/config/overlay.d/15fcos/usr/lib/systemd/system/coreos-check-ignition-config.service @@ -0,0 +1,16 @@ +# This service is used for printing a message if +# no Ignition config is provided. +[Unit] +Description=Check if Ignition config is provided +# Only perform checks on the first (Ignition) boot as they are +# mostly useful only on that boot. This ensures systems started +# before Ignition/Afterburn started logging structured data don't +# get misleading messages. Also handles the case where the journal +# gets rotated and no longer has the structured log messages. +ConditionKernelCommandLine=ignition.firstboot +[Service] +Type=oneshot +ExecStart=/usr/libexec/coreos-check-ignition-config +RemainAfterExit=yes +[Install] +WantedBy=multi-user.target diff --git a/config/overlay.d/15fcos/usr/lib/systemd/system/coreos-check-ssh-keys.service b/config/overlay.d/15fcos/usr/lib/systemd/system/coreos-check-ssh-keys.service new file mode 100644 index 0000000000000000000000000000000000000000..793b26c5be6cb958057225360f7274317088f3c6 --- /dev/null +++ b/config/overlay.d/15fcos/usr/lib/systemd/system/coreos-check-ssh-keys.service @@ -0,0 +1,22 @@ +# This service is used for printing a message if no ssh keys were added +# by Ignition/Afterburn +[Unit] +Description=Check that ssh-keys are added by Afterburn/Ignition +# It allows other units to synchronize around any instance +# of `afterburn-sshkeys@` and not just the `core` user. + +After=afterburn-sshkeys.target +# Only perform checks on the first (Ignition) boot as they are +# mostly useful only on that boot. This ensures systems started +# before Ignition/Afterburn started logging structured data don't +# get misleading messages. Also handles the case where the journal +# gets rotated and no longer has the structured log messages. +ConditionKernelCommandLine=ignition.firstboot + +[Service] +Type=oneshot +ProtectHome=read-only +ExecStart=/usr/libexec/coreos-check-ssh-keys +RemainAfterExit=yes +[Install] +WantedBy=multi-user.target diff --git a/config/overlay.d/15fcos/usr/lib/systemd/system/coreos-cleanup-ignition-config.service b/config/overlay.d/15fcos/usr/lib/systemd/system/coreos-cleanup-ignition-config.service new file mode 100644 index 0000000000000000000000000000000000000000..2df7e2db1a4560e62c21f2806e4b74c24c8c37fd --- /dev/null +++ b/config/overlay.d/15fcos/usr/lib/systemd/system/coreos-cleanup-ignition-config.service @@ -0,0 +1,20 @@ +[Unit] +Description=Clean Up Injected Ignition Config in /boot +Documentation=https://github.com/coreos/fedora-coreos-tracker/issues/889 +# Newer Ignition will handle this on first boot; we only want to clean up +# leftover configs on upgrade. Disambiguate those two code paths for tests. +ConditionKernelCommandLine=!ignition.firstboot +RequiresMountsFor=/boot +ConditionPathExists=/boot/ignition +# We ship a kdump.service dropin that remounts /boot rw; avoid conflicts +Before=kdump.service + +[Service] +Type=oneshot +ExecStart=/usr/libexec/coreos-cleanup-ignition-config +RemainAfterExit=yes +# MountFlags=slave ensures the rw mount of /boot is private to the unit +MountFlags=slave + +[Install] +WantedBy=multi-user.target diff --git a/config/overlay.d/15fcos/usr/libexec/coreos-check-cgroups b/config/overlay.d/15fcos/usr/libexec/coreos-check-cgroups new file mode 100644 index 0000000000000000000000000000000000000000..8dfaf0c7df4b95a9299c4944df1d6b9da05dfc2a --- /dev/null +++ b/config/overlay.d/15fcos/usr/libexec/coreos-check-cgroups @@ -0,0 +1,24 @@ +#!/usr/bin/bash +# This script checks if the system is still using cgroups v1 +# and prints a message to the serial console. + +# Change the output color to yellow +warn=$(echo -e '\033[0;33m') +# No color +nc=$(echo -e '\033[0m') + +motd_path=/run/motd.d/30_cgroupsv1_warning.motd + +cat << EOF > "${motd_path}" +${warn} +############################################################################ +WARNING: This system is using cgroups v1. For increased reliability +it is strongly recommended to migrate this system and your workloads +to use cgroups v2. For instructions on how to adjust kernel arguments +to use cgroups v2, see: + +To disable this warning, use: +sudo systemctl disable coreos-check-cgroups.service +############################################################################ +${nc} +EOF diff --git a/config/overlay.d/15fcos/usr/libexec/coreos-check-ignition-config b/config/overlay.d/15fcos/usr/libexec/coreos-check-ignition-config new file mode 100644 index 0000000000000000000000000000000000000000..f96d743503700aec9bec32f571b64b2e7da979dc --- /dev/null +++ b/config/overlay.d/15fcos/usr/libexec/coreos-check-ignition-config @@ -0,0 +1,26 @@ +#!/usr/bin/bash +# The logic for the message_id is handled in + +# In this script, we need to capture the journald +# log with the particular message_id and query using +#`jq` utility to check if a user config is provided. + +# Change the output color to yellow +warn='\033[0;33m' +# No color +nc='\033[0m' + + +# It will track the journal messages related to an Ignition config provided +# by the user. +output=$(journalctl -o json-pretty MESSAGE_ID=57124006b5c94805b77ce473e92a8aeb | jq -s '.[] | select(.IGNITION_CONFIG_TYPE == "user")'| wc -l) + +if [[ $output -gt 0 ]];then + echo "Ignition: user-provided config was applied" > /etc/issue.d/30_ignition_config_info.issue +else + echo -e "${warn}Ignition: no config provided by user${nc}" > /etc/issue.d/30_ignition_config_info.issue +fi + +# Ask all running agetty instances to reload and update their +# displayed prompts in case this script was run before agetty. +/usr/sbin/agetty --reload diff --git a/config/overlay.d/15fcos/usr/libexec/coreos-check-ssh-keys b/config/overlay.d/15fcos/usr/libexec/coreos-check-ssh-keys new file mode 100644 index 0000000000000000000000000000000000000000..f7182f59f1474ebb1971f30f97ba3eff611fbf51 --- /dev/null +++ b/config/overlay.d/15fcos/usr/libexec/coreos-check-ssh-keys @@ -0,0 +1,50 @@ +#!/usr/bin/bash +# This script will print a message in the serial console +# if no ssh keys were added by Ignition/Afterburn. +main() { + # Change the output color to yellow + warn='\033[0;33m' + # No color + nc='\033[0m' + + + # source. It will track the authorized-ssh-keys entries in journald + # provided via Ignition. + ignitionusers=$( + journalctl -o json-pretty MESSAGE_ID=225067b87bbd4a0cb6ab151f82fa364b | \ + jq -r '.MESSAGE' | \ + xargs -I{} echo "Ignition: {}") + + + # source. It will track the authorized-ssh-keys entries in journald + # provided via Afterburn. + afterburnusers=$( + journalctl -o json-pretty MESSAGE_ID=0f7d7a502f2d433caa1323440a6b4190 | \ + jq -r '.MESSAGE' | \ + xargs -I{} echo "Afterburn: {}") + + output='' + if [ -n "$ignitionusers" ]; then + output+="$ignitionusers" + fi + if [ -n "$afterburnusers" ]; then + # add newline if needed + if [ -n "$output" ]; then + output+=$'\n' + fi + output+="$afterburnusers" + fi + + if [ -n "$output" ]; then + echo "$output" > /etc/issue.d/30_ssh_authorized_keys.issue + else + echo -e "${warn}No SSH authorized keys provided by Ignition or Afterburn${nc}" \ + > /etc/issue.d/30_ssh_authorized_keys.issue + fi + + # Ask all running agetty instances to reload and update their + # displayed prompts in case this script was run before agetty. + /usr/sbin/agetty --reload +} + +main diff --git a/config/overlay.d/15fcos/usr/libexec/coreos-cleanup-ignition-config b/config/overlay.d/15fcos/usr/libexec/coreos-cleanup-ignition-config new file mode 100644 index 0000000000000000000000000000000000000000..172fb48317b825dd5bdc7baa19cb7fb944dfb474 --- /dev/null +++ b/config/overlay.d/15fcos/usr/libexec/coreos-cleanup-ignition-config @@ -0,0 +1,9 @@ +#!/usr/bin/bash +# +# Clean up existing nodes that have a world-readable /boot/ignition/config.ign. +# Remove this after the next barrier release on all streams. + +set -euo pipefail + +mount -o remount,rw /boot +rm -rf /boot/ignition diff --git a/config/overlay.d/20platform-chrony/usr/lib/systemd/system-generators/coreos-platform-chrony b/config/overlay.d/20platform-chrony/usr/lib/systemd/system-generators/coreos-platform-chrony new file mode 100644 index 0000000000000000000000000000000000000000..d6136e8d194eec5ffdbb5c788d5532a0687eec80 --- /dev/null +++ b/config/overlay.d/20platform-chrony/usr/lib/systemd/system-generators/coreos-platform-chrony @@ -0,0 +1,90 @@ +#!/bin/bash +set -euo pipefail +# Configuring the timeserver for the platform is often handled +# by pre-baking a config into a particular image for a platform, but +# that doesn't work for us because we have a single update stream. Hence +# this generator dynamically inspects the platform and reconfigures chrony. +# + +exec 1>/dev/kmsg; exec 2>&1 + +self=$(basename $0) +confpath=/run/coreos-platform-chrony.conf + +# Yeah this isn't a completely accurate kernel argument parser but +# we don't have one shared across shell services at the moment. +platform="$(grep -Eo ' ignition.platform.id=[a-z]+' /proc/cmdline | cut -f 2 -d =)" +case "${platform}" in + azure|aws|gcp) ;; # OK, this is a platform we know how to support + *) exit 0 ;; +esac + +# Exit early if we have already been run once +if [[ -f "${confpath}" ]]; then + echo "$self: ${confpath} already exists; skipping" + exit 0 +fi + +# Exit early if chrony configuration as been changed from the image default +if ! cmp {/usr,}/etc/chrony.conf >/dev/null; then + echo "$self: /etc/chrony.conf is modified; not changing the default" + exit 0 +fi + +# If not set already (by host customization or this script), set +# PEERNTP=no so that DHCP-provided NTP servers are not added to chrony. +# By doing this we assume the better NTP server choice is the +# platform-provided link-local NTP server rather than others from DHCP. +# TODO: once https://bugzilla.redhat.com/show_bug.cgi?id=1828434 is +# resolved, this won't be required. +if [ ! -e /etc/sysconfig/network ] || ! grep -q "PEERNTP" /etc/sysconfig/network; then + cat <> /etc/sysconfig/network +# PEERNTP=no is automatically added by default when a platform-provided time +# source is available, but this behavior may be overridden through an Ignition + +PEERNTP=no +EOF +fi + +(echo "# Generated by $self - do not edit directly" + sed -e s,'^makestep,#makestep,' -e s,'^pool,#pool,' -e s,'^leapsectz,#leapsectz,' < /etc/chrony.conf +cat < "${confpath}" +case "${platform}" in + azure) + (echo '# See also https://docs.microsoft.com/en-us/azure/virtual-machines/linux/time-sync' + echo 'refclock PHC /dev/ptp0 poll 3 dpoll -2 offset 0' + echo 'leapsectz right/UTC' + ) >> "${confpath}" ;; + aws) + (echo '# See also https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-time.html' + echo 'server 169.254.169.123 prefer iburst minpoll 4 maxpoll 4' + ) >> "${confpath}" ;; + gcp) + (echo '# See also https://cloud.google.com/compute/docs/instances/managing-instances#configure-ntp' + echo '# and https://cloud.google.com/compute/docs/images/configuring-imported-images' + echo 'server metadata.google.internal prefer iburst' + ) >> "${confpath}" ;; + *) echo "should not be reached" 1>&2; exit 1 ;; +esac +# Policy doesn't allow chronyd to read run_t +chcon --reference=/etc/chrony.conf "${confpath}" + +UNIT_DIR="${1:-/tmp}" + +unitconfpath="${UNIT_DIR}/chronyd.service.d/coreos-platform-chrony.conf" +mkdir -p $(dirname "${unitconfpath}") +cat >"${unitconfpath}" << EOF +[Service] +ExecStart= +ExecStart=/usr/sbin/chronyd -f ${confpath} \$OPTIONS +EOF + +echo "$self: Updated chrony to use ${platform} configuration ${confpath}" diff --git a/config/overlay.d/README.md b/config/overlay.d/README.md new file mode 100644 index 0000000000000000000000000000000000000000..761579e36077b32584f0ce3aacc41b7cc9b558d3 --- /dev/null +++ b/config/overlay.d/README.md @@ -0,0 +1,43 @@ +05core +----- + +This overlay matches `nestos-base.yaml`; core Ignition+ostree bits. + +08nouveau +--------- + +Blacklist the nouveau driver because it causes issues with some NVidia GPUs in EC2, +and we don't have a use case for NestOS with nouveau. + +"Cannot boot an p3.2xlarge instance with RHCOS (g3.4xlarge is working)" + +09misc +------ + +* Warning about `/etc/sysconfig`. +* Temporary systemd-tpmfiles.d config to fix ownership and permissions in /etc + +14NetworkManager-plugins +------------------------ + +Disables the Red Hat Linux legacy `ifcfg` format. + +15fcos +------ + +Things that are more closely "NestOS": + +* disable password logins by default over SSH +* enable SSH keys written by Ignition and Afterburn +* branding (MOTD) +* enable services by default (NestOS-pinger) +* display warnings on the console if no ignition config was provided or no ssh + key found. + +20platform-chrony +----------------- + +Add static chrony configuration for NTP servers provided on platforms +such as `azure`, `aws`, `gcp`. The chrony config for these NTP servers +should override other chrony configuration (e.g. DHCP-provided) +configuration. diff --git "a/docs/K8S+iSulad\346\220\255\345\273\272.md" "b/docs/K8S+iSulad\346\220\255\345\273\272.md" new file mode 100644 index 0000000000000000000000000000000000000000..05d21d310639bc3c00000dea7a38549361e012ee --- /dev/null +++ "b/docs/K8S+iSulad\346\220\255\345\273\272.md" @@ -0,0 +1,339 @@ +# K8S+iSulad 搭建 + +**以下步骤在master节点和node节点均需执行**，本教程以master为例 + +## 开始之前 + +需准备如下内容 +1.NestOS-LTS-live.x86_64.iso +2.一台主机用作master，一台主机用作node + +## 组件下载 + +编辑源文件,添加k8s的阿里云源 + +``` +vi /etc/yum.repos.d/openEuler.repo +``` + +添加如下内容 + +``` +[kubernetes] +name=Kubernetes +baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/ +enabled=1 +gpgcheck=1 +repo_gpgcheck=1 +gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg +``` + +下载k8s组件以及同步系统时间所用组件 + +``` +rpm-ostree install kubelet kubeadm kubectl ntp ntpdate wget +``` + +![image-20211014203239831](graph/K8S+iSulad搭建/image-20211014203239831.png) + +重启生效 + +``` +systemctl reboot +``` + +选择正确的分支 + +![image-20211014203252726](graph/K8S+iSulad搭建/image-20211014203252726.png) + +查看软件包是否已安装 + +``` +rpm -qa | grep kube +``` + +![image-20211014203302042](graph/K8S+iSulad搭建/image-20211014203302042.png) + +## 配置环境 + +### 修改主机名，以master为例 + +``` +hostnamectl set-hostname k8s-master +sudo -i +``` + +编辑/etc/hosts + +``` +vi /etc/hosts +``` + +添加如下内容，ip为主机ip + +``` +192.168.237.133 k8s-master +192.168.237.135 k8s-node01 +``` + +### 同步系统时间 + +``` +ntpdate time.windows.com +systemctl enable ntpd +``` + +``` +关闭swap分区，防火墙,selinux +``` + +NestOS默认无swap分区，默认关闭防火墙 +关闭selinux如下 + +``` +vi /etc/sysconfig/selinux +修改为SELINUX=disabled +``` + +### 网络配置，开启相应的转发机制 + +创建配置文件 + +``` +vi /etc/sysctl.d/k8s.conf +``` + +添加如下内容 + +``` +net.bridge.bridge-nf-call-iptables=1 +net.bridge.bridge-nf-call-ip6tables=1 +net.ipv4.ip_forward=1 +``` + +使配置生效 + +``` +modprobe br_netfilter +sysctl -p /etc/sysctl.d/k8s.conf +``` + +## 配置iSula + +查看k8s需要的系统镜像，需注意pause的版本号 + +``` +kubeadm config images list +``` + +![image-20211014203312883](graph/K8S+iSulad搭建/image-20211014203312883.png) + +修改daemon配置文件 + +``` +vi /etc/isulad/daemon.json +``` + +``` +##关于添加项的解释说明## +registry-mirrors 设置为"docker.io" +insecure-registries 设置为"rnd-dockerhub.huawei.com" +pod-sandbox-image 设置为"registry.aliyuncs.com/google_containers/pause:3.5"(使用阿 +里云,pause版本可在上一步查看) +network-plugin 设置为"cni"。 +cni-bin-dir 设置为"/opt/cni/bin"; +cni-conf-dir 设置为"/etc/cni/net.d" +``` + +修改后的完整文件如下 + +``` +{"group": "isula", +"default-runtime": "lcr", +"graph": "/var/lib/isulad", +"state": "/var/run/isulad", +"engine": "lcr", +"log-level": "ERROR", +"pidfile": "/var/run/isulad.pid", +"log-opts": { +"log-file-mode": "0600", +"log-path": "/var/lib/isulad", +"max-file": "1", +"max-size": "30KB" +}, +"log-driver": "stdout", +"container-log": { +"driver": "json-file" +}, +"hook-spec": "/etc/default/isulad/hooks/default.json", +"start-timeout": "2m", +"storage-driver": "overlay2", +"storage-opts": [ +"overlay2.override_kernel_check=true" +], +"registry-mirrors": [ +"docker.io" +], +"insecure-registries": [ +"rnd-dockerhub.huawei.com" +], +"pod-sandbox-image": "registry.aliyuncs.com/google_containers/pause:3.5", +"native.umask": "secure", +"network-plugin": "cni", +"cni-bin-dir": "/opt/cni/bin", +"cni-conf-dir": "/etc/cni/net.d", +"image-layer-check": false, +"use-decrypted-key": true, +"insecure-skip-verify-enforce": false +} +``` + +启动相关服务 + +``` +systemctl restart isulad +systemctl enable isulad +systemctl enable kubelet +``` + +**以上为master，node节点均需执行的操作。** + +## master节点初始化 + +**该部分仅master节点执行。** +初始化，在这一步会拉取镜像，需等待一小段时间。也可在该步骤之前手动拉取镜像。 + +``` +kubeadm init --kubernetes-version=1.22.2 --apiserver-advertise- +address=192.168.237.133 --cri-socket=/var/run/isulad.sock --image-repository +registry.aliyuncs.com/google_containers --service-cidr=10.10.0.0/16 --pod- +network-cidr=10.122.0.0/16 +``` + +``` +##关于初始化参数的解释说明## +kubernetes-version 为当前安装的版本 +apiserver-advertise-address 为master节点ip +cri-socket 指定引擎为isulad +image-repository 指定镜像源为阿里云,可省去修改tag的步骤 +service-cidr 指定service分配的ip段 +pod-network-cidr 指定pod分配的ip段 +``` + +初始化成功后可看到如下界面: + +![image-20211014203323220](graph/K8S+iSulad搭建/image-20211014203323220.png) + +复制最后两行内容方便后续node节点加入使用 + +``` +kubeadm join 192.168.237.133:6443 --token j7kufw.yl1gte0v9qgxjzjw --discovery- +token-ca-cert-hash +sha256:73d337f5edd79dd4db997d98d329bd98020b712f8d7833c33a85d8fe44d0a4f5 --cri- +socket=/var/run/isulad.sock +``` + +**注意**:添加--cri-socket=/var/run/isulad.sock以使用isula为容器引擎 +查看下载好的镜像 + +``` +isula images +``` + +![image-20211014203329239](graph/K8S+iSulad搭建/image-20211014203329239.png) + +按照初始化成功所提示，配置集群 + +``` +mkdir -p $HOME/.kube +cp -i /etc/kubernetes/admin.conf $HOME/.kube/config +chown $(id -u):$(id -g) $HOME/.kube/config +export KUBECONFIG=/etc/kubernetes/admin.conf +source /etc/profile +``` + +查看健康状态 + +``` +kubectl get cs +``` + +![image-20211014203335591](graph/K8S+iSulad搭建/image-20211014203335591.png) + +可以看到controller-manager,scheduler状态为unhealthy，解决方法如下: +编辑相关配置文件 + +``` +vi /etc/kubernetes/manifests/kube-controller-manager.yaml +``` + +``` +注释如下内容: +--port=0 +修改hostpath: +将所有/usr/libexec/kubernetes/kubelet-plugins/volume/exec 修改为/opt/libexec/... +``` + +``` +vi /etc/kubernetes/manifests/kube-scheduler.yaml +``` + +``` +注释如下内容: +--port=0 +``` + +等待片刻后，再次查看健康状态 + +![image-20211014203341778](graph/K8S+iSulad搭建/image-20211014203341778.png) + +## 配置网络插件 + +仅需要在master节点配置网络插件，但是要在**所有节点**提前拉取镜像,拉取镜像指令如下。 + +``` +isula pull calico/node:v3.19.3 +isula pull calico/cni:v3.19.3 +isula pull calico/kube-controllers:v3.19.3 +isula pull calico/pod2daemon-flexvol:v3.19.3 +``` + +**以下步骤仅在master节点执行** +获取配置文件 + +``` +wget https://docs.projectcalico.org/v3.19/manifests/calico.yaml +``` + +编辑calico.yaml 修改所有/usr/libexec/... 为 /opt/libexec/... +然后执行如下命令完成calico的安装: + +``` +kubectl apply -f calico.yaml +``` + +通过kubectl get pod -n kube-system查看calico是否安装成功 +通过kubectl get pod -n kube-system查看是否所有pod状态都为running + +![image-20211014203349296](graph/K8S+iSulad搭建/image-20211014203349296.png) + +## node节点加入集群 + +在node节点执行如下指令，将node节点加入集群 + +``` +kubeadm join 192.168.237.133:6443 --token j7kufw.yl1gte0v9qgxjzjw --discovery- +token-ca-cert-hash +sha256:73d337f5edd79dd4db997d98d329bd98020b712f8d7833c33a85d8fe44d0a4f5 --cri- +socket=/var/run/isulad.sock +``` + +通过kubectl get node 查看master，node节点状态是否为ready + +![image-20211014203355103](graph/K8S+iSulad搭建/image-20211014203355103.png) + +再次查看pod + +![image-20211014203400852](graph/K8S+iSulad搭建/image-20211014203400852.png) + +至此，k8s部署成功。 \ No newline at end of file diff --git "a/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203239831.png" "b/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203239831.png" new file mode 100644 index 0000000000000000000000000000000000000000..de10fbbd08ca213a300c3da60cccd9641366ffeb Binary files /dev/null and "b/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203239831.png" differ diff --git "a/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203252726.png" "b/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203252726.png" new file mode 100644 index 0000000000000000000000000000000000000000..2087aa455a3146df474d9f0e2bba63b774ed3119 Binary files /dev/null and "b/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203252726.png" differ diff --git "a/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203302042.png" "b/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203302042.png" new file mode 100644 index 0000000000000000000000000000000000000000..39fcfa715c2a33cb357a06fd0b68c70e2ae2af34 Binary files /dev/null and "b/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203302042.png" differ diff --git "a/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203312883.png" "b/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203312883.png" new file mode 100644 index 0000000000000000000000000000000000000000..7161e18aef15332219d9104205d99d7f6a0cce8c Binary files /dev/null and "b/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203312883.png" differ diff --git "a/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203323220.png" "b/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203323220.png" new file mode 100644 index 0000000000000000000000000000000000000000..00d241873bfc478613d759e63ac2cc54ff38aeff Binary files /dev/null and "b/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203323220.png" differ diff --git "a/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203329239.png" "b/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203329239.png" new file mode 100644 index 0000000000000000000000000000000000000000..be7a62462e3932e223380ec2220527a496908161 Binary files /dev/null and "b/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203329239.png" differ diff --git "a/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203335591.png" "b/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203335591.png" new file mode 100644 index 0000000000000000000000000000000000000000..f254b9c4bd7aed91ea76dcbd3c4620923b9178c3 Binary files /dev/null and "b/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203335591.png" differ diff --git "a/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203341778.png" "b/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203341778.png" new file mode 100644 index 0000000000000000000000000000000000000000..0c78083792128e5535eff78cf82aa04c58d65a00 Binary files /dev/null and "b/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203341778.png" differ diff --git "a/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203349296.png" "b/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203349296.png" new file mode 100644 index 0000000000000000000000000000000000000000..4efc2f3dff77f19f649fd947d6f33614af1f8b09 Binary files /dev/null and "b/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203349296.png" differ diff --git "a/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203355103.png" "b/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203355103.png" new file mode 100644 index 0000000000000000000000000000000000000000..e72f884f675bd4aeb8cd011863c39de4242009f0 Binary files /dev/null and "b/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203355103.png" differ diff --git "a/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203400852.png" "b/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203400852.png" new file mode 100644 index 0000000000000000000000000000000000000000..d51f743558a3925421a4e68dfa28b1644714694a Binary files /dev/null and "b/docs/graph/K8S+iSulad\346\220\255\345\273\272/image-20211014203400852.png" differ diff --git "a/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014201905155.png" "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014201905155.png" new file mode 100644 index 0000000000000000000000000000000000000000..049680c86b7542848cbd82a3d041f2fbcdb06cdd Binary files /dev/null and "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014201905155.png" differ diff --git "a/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014201914711.png" "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014201914711.png" new file mode 100644 index 0000000000000000000000000000000000000000..2087aa455a3146df474d9f0e2bba63b774ed3119 Binary files /dev/null and "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014201914711.png" differ diff --git "a/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014201922069.png" "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014201922069.png" new file mode 100644 index 0000000000000000000000000000000000000000..fb2694e2fbaf75fc7f51b895d5f12d5a63d3c8a2 Binary files /dev/null and "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014201922069.png" differ diff --git "a/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014201929746.png" "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014201929746.png" new file mode 100644 index 0000000000000000000000000000000000000000..c7096f237b558237b72c32e496f828365c150c15 Binary files /dev/null and "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014201929746.png" differ diff --git "a/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014201940141.png" "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014201940141.png" new file mode 100644 index 0000000000000000000000000000000000000000..0d8bff0e0fc1466af8d7a04596b2b9df715cdb1b Binary files /dev/null and "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014201940141.png" differ diff --git "a/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014201948988.png" "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014201948988.png" new file mode 100644 index 0000000000000000000000000000000000000000..2ef804f1d19210ea1f11ee51775f2392d86145d3 Binary files /dev/null and "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014201948988.png" differ diff --git "a/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014201956536.png" "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014201956536.png" new file mode 100644 index 0000000000000000000000000000000000000000..3638e7d822e84c8fcb8a86d686df5621703f6278 Binary files /dev/null and "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014201956536.png" differ diff --git "a/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014202004110.png" "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014202004110.png" new file mode 100644 index 0000000000000000000000000000000000000000..36d90d98233c1a2b7871dd4f95cb917cbcd983f4 Binary files /dev/null and "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014202004110.png" differ diff --git "a/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014202014370.png" "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014202014370.png" new file mode 100644 index 0000000000000000000000000000000000000000..a49383418b2a015fe539c1396737cc9ca229e4e9 Binary files /dev/null and "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014202014370.png" differ diff --git "a/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014202023177.png" "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014202023177.png" new file mode 100644 index 0000000000000000000000000000000000000000..8bc43f6cd5664465f79ef4a575dca36cf7aa4647 Binary files /dev/null and "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014202023177.png" differ diff --git "a/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014202030442.png" "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014202030442.png" new file mode 100644 index 0000000000000000000000000000000000000000..70f683e22eb816fe9cbe21aac233cd67d62bc15c Binary files /dev/null and "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014202030442.png" differ diff --git "a/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014202037703.png" "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014202037703.png" new file mode 100644 index 0000000000000000000000000000000000000000..1548fbff8ef772221a0d05a0ea96c51f26e286fe Binary files /dev/null and "b/docs/graph/rpm-ostree\344\275\277\347\224\250/image-20211014202037703.png" differ diff --git "a/docs/graph/zincati\350\207\252\345\212\250\346\233\264\346\226\260\344\275\277\347\224\250/0880c4c80710ab88d007.png" "b/docs/graph/zincati\350\207\252\345\212\250\346\233\264\346\226\260\344\275\277\347\224\250/0880c4c80710ab88d007.png" new file mode 100644 index 0000000000000000000000000000000000000000..4bb0d97a15e7e55aad6148099c74bc6bb0c21daa Binary files /dev/null and "b/docs/graph/zincati\350\207\252\345\212\250\346\233\264\346\226\260\344\275\277\347\224\250/0880c4c80710ab88d007.png" differ diff --git "a/docs/graph/zincati\350\207\252\345\212\250\346\233\264\346\226\260\344\275\277\347\224\250/0880c4c80710abc840-1634214176877.png" "b/docs/graph/zincati\350\207\252\345\212\250\346\233\264\346\226\260\344\275\277\347\224\250/0880c4c80710abc840-1634214176877.png" new file mode 100644 index 0000000000000000000000000000000000000000..38862cb13b7d0ddbd97d8f854da48767c016d1dc Binary files /dev/null and "b/docs/graph/zincati\350\207\252\345\212\250\346\233\264\346\226\260\344\275\277\347\224\250/0880c4c80710abc840-1634214176877.png" differ diff --git "a/docs/graph/zincati\350\207\252\345\212\250\346\233\264\346\226\260\344\275\277\347\224\250/0880c4c80710abc840-1634214176878.png" "b/docs/graph/zincati\350\207\252\345\212\250\346\233\264\346\226\260\344\275\277\347\224\250/0880c4c80710abc840-1634214176878.png" new file mode 100644 index 0000000000000000000000000000000000000000..38862cb13b7d0ddbd97d8f854da48767c016d1dc Binary files /dev/null and "b/docs/graph/zincati\350\207\252\345\212\250\346\233\264\346\226\260\344\275\277\347\224\250/0880c4c80710abc840-1634214176878.png" differ diff --git "a/docs/graph/zincati\350\207\252\345\212\250\346\233\264\346\226\260\344\275\277\347\224\250/0880c4c80710abc840.png" "b/docs/graph/zincati\350\207\252\345\212\250\346\233\264\346\226\260\344\275\277\347\224\250/0880c4c80710abc840.png" new file mode 100644 index 0000000000000000000000000000000000000000..38862cb13b7d0ddbd97d8f854da48767c016d1dc Binary files /dev/null and "b/docs/graph/zincati\350\207\252\345\212\250\346\233\264\346\226\260\344\275\277\347\224\250/0880c4c80710abc840.png" differ diff --git "a/docs/graph/zincati\350\207\252\345\212\250\346\233\264\346\226\260\344\275\277\347\224\250/0880c4c80710abca0b.png" "b/docs/graph/zincati\350\207\252\345\212\250\346\233\264\346\226\260\344\275\277\347\224\250/0880c4c80710abca0b.png" new file mode 100644 index 0000000000000000000000000000000000000000..515ea3f673c70f01bb962fa6743ebc6ba215259a Binary files /dev/null and "b/docs/graph/zincati\350\207\252\345\212\250\346\233\264\346\226\260\344\275\277\347\224\250/0880c4c80710abca0b.png" differ diff --git "a/docs/graph/\345\277\253\351\200\237\345\274\200\345\247\213/image-20211014200951942.png" "b/docs/graph/\345\277\253\351\200\237\345\274\200\345\247\213/image-20211014200951942.png" new file mode 100644 index 0000000000000000000000000000000000000000..1bf0a86838c6b268e0215f4c4183b96f37681d26 Binary files /dev/null and "b/docs/graph/\345\277\253\351\200\237\345\274\200\345\247\213/image-20211014200951942.png" differ diff --git "a/docs/graph/\345\277\253\351\200\237\345\274\200\345\247\213/image-20211014201036415.png" "b/docs/graph/\345\277\253\351\200\237\345\274\200\345\247\213/image-20211014201036415.png" new file mode 100644 index 0000000000000000000000000000000000000000..3e3f429fad3046b3a20a2a7996e0302d44aa1de9 Binary files /dev/null and "b/docs/graph/\345\277\253\351\200\237\345\274\200\345\247\213/image-20211014201036415.png" differ diff --git "a/docs/graph/\345\277\253\351\200\237\345\274\200\345\247\213/image-20211014201046509.png" "b/docs/graph/\345\277\253\351\200\237\345\274\200\345\247\213/image-20211014201046509.png" new file mode 100644 index 0000000000000000000000000000000000000000..959851e3a119440b6b9be6bd1e845d80d0845428 Binary files /dev/null and "b/docs/graph/\345\277\253\351\200\237\345\274\200\345\247\213/image-20211014201046509.png" differ diff --git "a/docs/rpm-ostree\344\275\277\347\224\250.md" "b/docs/rpm-ostree\344\275\277\347\224\250.md" new file mode 100644 index 0000000000000000000000000000000000000000..0f0587f6c373381e8d3f60c03ca1fa9566a1fa62 --- /dev/null +++ "b/docs/rpm-ostree\344\275\277\347\224\250.md" @@ -0,0 +1,122 @@ +# rpm-ostree使用 + +## rpm-ostree安装软件包 + +安装wget + +``` +rpm-ostree install wget +``` + +![image-20211014201905155](graph/rpm-ostree使用/image-20211014201905155.png) + +重启系统,可在启动时通过键盘上下按键选择rpm包安装完成后或安装前的系统状态，其中【ostree:0】为安装之后的版本。 + +``` +systemctl reboot +``` + +![image-20211014201914711](graph/rpm-ostree使用/image-20211014201914711.png) + +查看wget是否安装成功 + +``` +rpm -qa | grep wget +``` + +![image-20211014201922069](graph/rpm-ostree使用/image-20211014201922069.png) + +## rpm-ostree 手动更新升级 NestOS + +在NestOS中执行命令可查看当前rpm-ostree状态，可看到当前版本为LTS.20210927.dev.0 + +``` +rpm-ostree status +``` + +![image-20211014201929746](graph/rpm-ostree使用/image-20211014201929746.png) + +执行检查命令查看是否有升级可用，发现存在LTS.20210928.dev.0版本 + +``` +rpm-ostree upgrade --check +``` + +![image-20211014201940141](graph/rpm-ostree使用/image-20211014201940141.png) + +预览版本的差异 + +``` +rpm-ostree upgrade --preview +``` + +![image-20211014201948988](graph/rpm-ostree使用/image-20211014201948988.png) + +可以看到，在0928的最新版本中，我们将wget包做了引入。 +下载最新的ostree和RPM数据，不需要进行部署 + +``` +rpm-ostree upgrade --download-only +``` + +![image-20211014201956536](graph/rpm-ostree使用/image-20211014201956536.png) + +重启NestOS，重启后可看到系统的新旧版本两个状态,选择最新版本的分支进入 + +``` +rpm-ostree upgrade --reboot +``` + +## 比较NestOS版本差别 + +检查状态，确认此时ostree有两个版本,分别为LTS.20210927.dev.0和LTS.20210928.dev.0 + +``` +rpm-ostree status +``` + +![image-20211014202004110](graph/rpm-ostree使用/image-20211014202004110.png) + +根据commit号比较2个ostree的差别 + +``` +rpm-ostree db diff 55eed9bfc5ec fe2408e34148 +``` + +![image-20211014202014370](graph/rpm-ostree使用/image-20211014202014370.png) + +## 系统回滚 + +当一个系统更新完成，之前的NestOS部署仍然在磁盘上，如果更新导致了系统出现问题，可以使用之前的部署回滚系统。 + +### 临时回滚 + +要临时回滚到之前的OS部署，在系统启动过程中按住shift键，当引导加载菜单出现时，在菜单中选择相关的分支。 + +### 永久回滚 + +要永久回滚到之前的操作系统部署，登录到目标节点,运行rpm-ostree rollback，此操作将使用之前的系统部署作为默认部署，并重新启动到其中。 +执行命令，回滚到前面更新前的系统。 + +``` +rpm-ostree rollback +``` + +![image-20211014202023177](graph/rpm-ostree使用/image-20211014202023177.png) + +重启后失效。 + +## 切换版本 + +在上一步将NestOS回滚到了LTS.20210927.dev.0版本，可以通过命令切换当前 NestOS 使用的rpm-ostree版本，将LTS.20210927.dev.0切换为LTS.20210928.dev.0版本。 + +``` +rpm-ostree deploy -r LTS.20210928.dev.0 +``` + +![image-20211014202030442](graph/rpm-ostree使用/image-20211014202030442.png) + +重启后确认目前NestOS已经使用的是LTS.20210928.dev.0版本的ostree了。 + +![image-20211014202037703](graph/rpm-ostree使用/image-20211014202037703.png) + diff --git "a/docs/zincati\350\207\252\345\212\250\346\233\264\346\226\260\344\275\277\347\224\250.md" "b/docs/zincati\350\207\252\345\212\250\346\233\264\346\226\260\344\275\277\347\224\250.md" new file mode 100644 index 0000000000000000000000000000000000000000..871c90a135f194e25a44a0b0b173d7baf7b9979e --- /dev/null +++ "b/docs/zincati\350\207\252\345\212\250\346\233\264\346\226\260\344\275\277\347\224\250.md" @@ -0,0 +1,41 @@ +# zincati自动更新使用 + +zincati负责NestOS的自动更新，zincati通过cincincati提供的后端来检查当前是否有可更新版本，若检测到有可能新版本，会通过rpm-ostree进行下载。 + +目前系统默认关闭zincati自动更新服务，可通过修改配置文件设置为开机自动启动自动更新服务。 + +``` +vi /etc/zincati/config.d/95-disable-on-dev.toml +``` + +将updates.enabled设置为true +同时增加配置文件，修改cincincati后端地址 + +``` +vi /etc/zincati/config.d/update-cincincati.toml +``` + +添加如下内容 + +``` +[cincinnati] +base_url="http://10.1.110.88:8080" +``` + +重新启动zincati服务 + +``` +systemctl restart zincati.service +``` + +当有新版本时，zincati会自动检测到可更新版本，此时查看rpm-ostree状态，可以看到状态是“busy”，说明系统正在升级中。 + +![蓝信图片_0880c4c80710ab88d007](graph/zincati自动更新使用/0880c4c80710ab88d007.png) + +一段时间后NestOS将自动重启，此时再次登录NestOS，可以再次确认rpm-ostree的状态，其中状态转为"idle"，而且当前版本已经是“20211013”，这说明rpm-ostree版本已经升级了。 + +![0880c4c80710abc840](graph/zincati自动更新使用/0880c4c80710abc840-1634214176877.png) + +查看zincati服务的日志，确认升级的过程和重启系统的日志。另外日志显示的"auto-updates logic enabled"也说明更新是自动的。 + +![0880c4c80710abca0b](graph/zincati自动更新使用/0880c4c80710abca0b.png) \ No newline at end of file diff --git "a/docs/\345\212\237\350\203\275\347\211\271\346\200\247\346\217\217\350\277\260.md" "b/docs/\345\212\237\350\203\275\347\211\271\346\200\247\346\217\217\350\277\260.md" new file mode 100644 index 0000000000000000000000000000000000000000..a9121e98f8a9e30de658be5cf5065d066cf8d7c4 --- /dev/null +++ "b/docs/\345\212\237\350\203\275\347\211\271\346\200\247\346\217\217\350\277\260.md" @@ -0,0 +1,103 @@ +# 功能特性描述 + +## 容器技术 + +NestOS通过容器化 (containerized) 的运算环境向应用程序提供运算资源，应用程序之间共享系统内核和资源，但是彼此之间又互不可见。这意味着应用程序将不会再被直接安装到操作系统中，而是通过 Docker 运行在容器中。大大降低了操作系统、应用程序及运行环境之间的耦合度。相对于传统的应用程序部署部署方式而言，在NestOS 集群中部署应用程序更加灵活便捷，应用程序运行环境之间的干扰更少，而且操作系统自身的维护也更加容易。 + +## rpm-ostree + +### 系统更新 + +rpm-ostree是一种镜像/包混合系统，可以看成是rpm和ostree的合体。一方面它提供了基于rpm的软件包安装管理方式，另一方面它提供了基于ostree的操作系统更新升级。rpm-ostree将这两种操作都视为对操作系统的更新，每次对系统的更新都像rpm-ostree在提交“Transaction-事务”，从而确保更新全部成功或全部失败，允许在更新系统后回滚到更新前的状态。 + +rpm-ostree在更新操作系统的时候会有2个bootable区域，分别为更新前和更新后的，对系统的更新升级只有在重启操作系统后才生效。如果软件安装或升级出现问题，通过rpm-ostree回滚会使nestos系统返回到先前的状态。我们可以查看nestos的“/ostree/”和“/boot/”目录，它们是ostree Repository环境并且可以观察到boot使用哪个ostree。 + +### 文件系统 + +在rpm-ostree的文件系统布局中，只有/etc和/var是唯一可写的目录，/var中的任何数据不会被触及，而是在升级过程中共享。在系统升级的过程中采用新的默认值/etc，并将更改添加到顶部。这意味着升级将会接收/etc中新的默认文件，这是一个非常关键的特性。 + +Ostree旨在可以并⾏安装多个独⽴操作系统的版本，ostree依赖于⼀个新的ostree ⽬录，该目录实际上可以并⾏安装在现有的操作系统或者是占据物理/root目录的发⾏版本中。每台客⼾机和每组部署上都存储在 /ostree/deploy/STATEROOT/CHECKSUM 上，而且还有⼀个ostree存储库存储在 /ostree/repo 中。每个部署主要由⼀组指向存储库的硬链接组成，这意味着每个版本都进⾏了重复数据的删除并且升级过程中只消耗了与新⽂件成⽐例的磁盘空间加上⼀些恒定的开销。 + +Ostree模型强调的是OS只读内容保存在 /usr 中，它附带了⽤于创建Linux只读安装以防⽌⽆意损坏的代码，对于给定的操作系统，每个部署之间都有⼀个 /var 共享的可供读写的⽬录。Ostree核⼼代码不触及该⽬录的内容，如何管理和升级状态取决于每个操作系统中的代码。 + +### 系统扩展 + +出于安全性和可维护性的考虑，nestos让基础镜像尽可能保持小巧和精简。但是在某些情况下，需要向基本操作系统本⾝添加软件，例如驱动软件，VPN等等，因为它们⽐较难容器化。这些包拓展了操作系统的功能，为此，rpm-ostree将这些包视为拓展，而不是仅仅在用户运⾏时提供。也就是说，目前nestos对于实际安装哪些包没有限制，默认情况下，软件包是从openEuler仓库下载的。 + +要对软件包进⾏分层，需要重新编写⼀个systemd单元来执⾏rpm-ostree命令安装所需要的包，所做的更改应⽤于新部署，重新启动才能⽣效。 + +## nestos-installer + +nestos-installer是⼀个帮助安装Nestos的程序，它可以执⾏以下操作： + +（1）安装操作系统到⼀个⽬标磁盘，可使⽤Ignition和⾸次引导内核参数对其进⾏对其进⾏⾃定义(nestos-installer install) + +（2）下载并验证各种云平台、虚拟化或者裸机平台的操作系统映像(nestos-installer download) + +（3）列出可供下载的nestos镜像(nestos-installer list-stream) + +（4）在ISO中嵌⼊⼀个Ignition配置，以⾃定义地从中启动操作系统(nestos-installer iso ignition) + +（5）将Ignition配置包装在initd映像中，该映像可以被加⼊到PXE initramfs中以⾃定义从中启动的操作系统(nestos-installer pxe ignition) + +## zincati + +Zincati是nestos⾃动更新的代理，它作为Cincinnati和rpm-ostree的客户端，负责⾃动更新/重启机器。Zincati有如下特点： + +（1）⽀持⾃动更新代理，⽀持分阶段推出 + +（2）通过toml配置文件支持运行时自定义，用户自定义配置文件可覆盖默认配置 + +（3）多种更新策略 + +（4）通过维护窗口每周在特定时间范围内进行更新的策略 + +（5）收集和导出本地运行的zincati内部指标，可提供给Prometheus以减轻跨大量节点的监控任务 + +（6）具有可配置优先级的⽇志记录 + +（7）通过Cincinnati协议⽀持复杂的更新图 + +（8）通过外部锁管理器⽀持集群范围的重启编排 + +## 系统初始化（Ignition） + +Ignition 是一个与分发无关的配置实用程序，不仅用于安装，还读取配置文件（JSON 格式）并根据该配置nestos系统。可配置的组件包括存储和文件系统、systemd单元和用户。 + +Ignition仅在系统第一次引导期间运行一次（在initramfs中）。因为 Ignition 在启动过程的早期运行，所以它可以在用户空间开始启动之前重新分区磁盘、格式化文件系统、创建用户和写入文件。 因此，systemd 服务在 systemd 启动时已经写入磁盘，从而加快了启动速度。 + +（1）Ignition 仅在第一次启动时运行 + +Ignition 旨在用作配置工具，而不是配置管理工具。 Ignition 鼓励不可变的基础设施，其中机器修改要求用户丢弃旧节点并重新配置机器。 + +（2）Ignition不是在任何情况下都可以完成配置 + +Ignition 执行它需要的操作，使系统与 Ignition 配置中描述的状态相匹配。 如果由于任何原因 Ignition 无法提供配置要求的确切机器，Ignition 会阻止机器成功启动。例如，如果用户想要获取托管在 https://example.com/foo.conf 的文档并将其写入磁盘，如果无法解析给定的 URL，Ignition 将阻止机器启动。 + +（3）Ignition只是声明性配置 + +Ignition配置只描述了系统的状态，没有列出 Ignition 应该采取的一系列步骤。 + +Ignition 配置不允许用户提供任意逻辑（包括 Ignition 运行的脚本）。用户只需描述哪些文件系统必须存在、哪些文件必须创建、哪些用户必须存在等等。任何进一步的定制都必须使用由 Ignition 创建的 systemd 服务。 + +（4）Ignition配置不应手写 + +Ignition 配置被设计为人类可读但难以编写，是为了阻止用户尝试手动编写配置。可以使用Butane或类似工具生成或转化生成Ignition 配置。 + +## Afterburn + +Afterburn是类似于云平台⼀样的⼀次性代理，可以⽤于与特定的提供者的元数据端点进⾏交互，通常和Ignition结合使⽤。 + +Afterburn包含了很多可以在实例⽣命周期中不同时间段运⾏的模块。下⾯的服务是根据特定的平台可能在第⼀次启动时在initramfs中运⾏的： + +（1）设置本地主机名 + +（2）加⼊⽹络命令⾏参数 + +以下的功能是在⼀定条件下，作为systemd服务单元在⼀些平台上才能使⽤的： + +（1）为本地系统用户安装公共SSH密钥 + +（2）从实例元数据中检索属性 + +（3）给提供者登记以便报道成功的启动或实例供应 \ No newline at end of file diff --git "a/docs/\345\277\253\351\200\237\345\274\200\345\247\213.md" "b/docs/\345\277\253\351\200\237\345\274\200\345\247\213.md" new file mode 100644 index 0000000000000000000000000000000000000000..9b327d79bc26cfbfb840b1f05adc40fc750ad205 --- /dev/null +++ "b/docs/\345\277\253\351\200\237\345\274\200\345\247\213.md" @@ -0,0 +1,155 @@ +# 快速开始 + +## 在 VMware 上部署 NestOS + +本指南展示了如何在VMware虚拟机管理程序上配置最新的 NestOS。 + +目前NestOS仅支持x86_64架构。 + +### 开始之前 + +​ 在开始部署 NestOS 之前,需要做如下准备工作: + +- 下载 NestOS ISO +- 准备 config.bu 文件 +- 配置 butane 工具(Linux环境/win10环境) +- 安装有VMware的宿主机 + +### 初步安装与启动 + +![image-20211014200951942](graph/快速开始/image-20211014200951942.png) + +#### 启动 NestOS + +NestOS 初次启动如下图所示 + +![image-20211014201036415](graph/快速开始/image-20211014201036415.png) + +初次启动 NestOS ，ignition 尚未安装，可根据系统提示使用 coreos-installer 组件进行 ignition的安 +装。 + +![image-20211014201046509](graph/快速开始/image-20211014201046509.png) + +### 配置 ignition 文件 + +#### 获取 Butane + +可以通过 Butane 将 bu 文件转化为 igniton 文件。ignition 配置文件被设计为可读但难以编写，是为了 +阻止用户尝试手动编写配置。 +Butane 提供了多种环境的支持，可以在 linux/windows 宿主机中或容器环境中进行配置。 + +``` +docker pull quay.io/coreos/butane:release +``` + +#### 生成登录密码 + +在宿主机执行如下命令，并输入你的密码。 + +``` +# openssl passwd -1 -salt yoursalt +Password: +$1$yoursalt$1QskegeyhtMG2tdh0ldQN0 +``` + +#### 生成ssh-key + +在宿主机执行如下命令，获取公钥和私钥以供后续 ssh 登录。 + +``` +# ssh-keygen -N '' -f ./id_rsa +Generating public/private rsa key pair. +Your identification has been saved in ./id_rsa +Your public key has been saved in ./id_rsa.pub +The key fingerprint is: +SHA256:4fFpDDyGHOYEd2fPaprKvvqst3T1xBQuk3mbdon+0Xs root@host-12-0-0-141 +``` + +``` +The key's randomart image is: ++---[RSA 3072]----+ +| ..= . o . | +| * = o * . | +| + B = * | +| o B O + . | +| S O B o | +| * = . . | +| . +o . . | +| +.o . .E | +| o*Oo ... | ++----[SHA256]-----+ +``` + +可以在当前目录查看id_rsa.pub公钥 + +``` +# cat id_rsa.pub +ssh-rsa +AAAAB3NzaC1yc2EAAAADAQABAAABgQDjf+I9QQZ+3vNWomqxpHkZq7ONHcEYBzs4C9RZahmLYVPBf/3y +HF5wTtfl5CBviERUnLGFn8c4Ua9MNWcJL6zE01xXtDZ2db7vaPwP3Qbo1lKJg1BVw6u+5bMKCJxEnN9+ +aOiX3A2XpkUVxhCoGlei3j78oLRU3ucCLn6m7wVE+P53tNQ5364xWqbAsDXdze4xnNZjlzH9JvjJ5IJY +WjwrD7UUkfI8qDj5ub9Gz+nSenaaSboWADsKe4JTLoU2Gz5fPLCj+uuNFpZAUc/GCe47He5UO6IbHjDI +bxqhzYZQTXdwIgKIM1PL19IkAAY07gU53b4gDSDj7SZYB+jjtgG8VoFF4m7nCJgRDeUKGTNT5fsLPKAZ +tmBvy9Mg5qkK/LisEzjUwPPh1NEb8bgN251wPXmPMjQ1aMzD8t9blq40KEyod2Eg05nW2q5/90ICNQBa +r9AkQrQ/3j8WsejvqseWIi1kq68pqvtcBJkCMiIfzIoUgCgcolw3fZprDhgfau8= root@host-12-0- +0-141 +``` + +#### 编写bu文件 + +进行最简单的初始配置，如需更多详细的配置，参考后面的 ignition 详解。 +如下为最简单的 config.bu 文件 + +``` +variant: NestOS +version: 1.1.0 +passwd: + users: + - name: core + password_hash: "$1$yoursalt$1QskegeyhtMG2tdh0ldQN0" + ssh_authorized_keys: + - "ssh-rsa + AAAAB3NzaC1yc2EAAAADAQABAAABgQDjf+I9QQZ+3vNWomqxpHkZq7ONHcEYBzs4C9RZahmLYVPBf/3y + HF5wTtfl5CBviERUnLGFn8c4Ua9MNWcJL6zE01xXtDZ2db7vaPwP3Qbo1lKJg1BVw6u+5bMKCJxEnN9+ + aOiX3A2XpkUVxhCoGlei3j78oLRU3ucCLn6m7wVE+P53tNQ5364xWqbAsDXdze4xnNZjlzH9JvjJ5IJY + WjwrD7UUkfI8qDj5ub9Gz+nSenaaSboWADsKe4JTLoU2Gz5fPLCj+uuNFpZAUc/GCe47He5UO6IbHjDI + bxqhzYZQTXdwIgKIM1PL19IkAAY07gU53b4gDSDj7SZYB+jjtgG8VoFF4m7nCJgRDeUKGTNT5fsLPKAZ + tmBvy9Mg5qkK/LisEzjUwPPh1NEb8bgN251wPXmPMjQ1aMzD8t9blq40KEyod2Eg05nW2q5/90ICNQBa + r9AkQrQ/3j8WsejvqseWIi1kq68pqvtcBJkCMiIfzIoUgCgcolw3fZprDhgfau8= root@host-12-0- + 0-141" +``` + +#### 生成ignition文件 + +将 config.bu 通过 Butane 工具转换为 config.ign 文件，如下为在容器环境下进行转换。 + +``` +# docker run --interactive --rm quay.io/coreos/butane:release \ +--pretty --strict < your_config.bu > transpiled_config.ign +``` + +也可在其他环境下进行转换，Butane提供了多种转换的方式,可在如下地址查看。 +https://github.com/coreos/butane + +### 安装 NestOS + +将宿主机生成的的config.ign文件通过scp拷贝到前面初步启动的 NestOS 中，该OS目前运行在内存中, +并没有安装到硬盘。 + +``` +sudo -i +scp root@10.1.110.88:/root/config.ign /root #从10.1.110.88拷贝,请自行更换你的生成ign文件的机器的ip +``` + +根据系统所给提示，执行如下指令完成安装。 + +``` +coreos-installer install /dev/sda --ignition-file config.ign +``` + +安装完成后重启 NestOS 。 + +``` +systemctl reboot +``` + diff --git "a/docs/\346\200\247\350\203\275\345\257\271\346\257\224\346\265\213\350\257\225.md" "b/docs/\346\200\247\350\203\275\345\257\271\346\257\224\346\265\213\350\257\225.md" new file mode 100644 index 0000000000000000000000000000000000000000..65a7bc0a6fde45f7b822c83b3556c32047bdfed7 --- /dev/null +++ "b/docs/\346\200\247\350\203\275\345\257\271\346\257\224\346\265\213\350\257\225.md" @@ -0,0 +1,28 @@ +# 性能对比测试 + +使用NestOS 20211009版本,横向对比 docker,podman,iSulad 容器引擎性能。测试结果如下。 +x86 machine : + +| Configuration | Information | +| :-----------: | :----------: | +| OS | NestOS | +| Kernel | linux 5.10.0 | +| CPU | 8 cores | +| Memory | 16 GB | + +软件版本： + +| Name | Version | +| ------ | ------------------------------------------------------------ | +| iSulad | Version 2.0.8, commit 9aa57ef27d3719729097f75c65125519497b8b66 | +| docker | Version: 18.09.0, Git commit: 1c709d9 | +| podman | Version 0.10.1 | + +使用NestOS 20211009版本，横向对比 docker，podman，iSulad 容器引擎性能。测试结果如下: + +| operator(ms) | Docker | Podman | iSulad | vs Docker | vs Podman | +| :----------: | :----: | :----: | :----: | :-------: | :-------: | +| 100*creat | 1745 | 22919 | 1122 | -36% | -95% | +| 100*start | 8561 | 8133 | 1561 | -82% | -81% | +| 100*stop | 1483 | 1445 | 296 | -80% | -80% | +| 100*rm | 1691 | 5286 | 741 | -56% | -86% | \ No newline at end of file diff --git a/graph/README/image-20211015170943884.png b/graph/README/image-20211015170943884.png new file mode 100644 index 0000000000000000000000000000000000000000..48ec43323740338a864e7d0f171520dae2547a96 Binary files /dev/null and b/graph/README/image-20211015170943884.png differ