diff --git a/documentation/brief-introduction-to-edge-computing.md b/documentation/brief-introduction-to-edge-computing.md new file mode 100644 index 0000000000000000000000000000000000000000..e45cfe658aebe20521f8980e68bdfd9bceddb0f1 --- /dev/null +++ b/documentation/brief-introduction-to-edge-computing.md @@ -0,0 +1,52 @@ +# Brief Introduction to Edge Computing + +Edge computing is a new computing mode that works at the network edge. It processes data through downstream cloud services and upstream device applications. Edge computing allocates computing and storage resources to edge nodes, which are closer to users. The nodes analyze data from nearby users and upload only important information such as computing results to the cloud, greatly releasing server pressure. + +The edge nodes have lots of real-time and complete data to be processed. As a data entry, edge computing can better adapt to frequent data interaction and deal with real-time, deterministic, and diversified data. Therefore, edge computing can solve problems like **high bandwidth loads** and **network latency**, and reduce **data management costs**. + +## Composition of Edge Computing + +Edge computing consists of three parts: + +- Endpoint: collects data, directs data to edge nodes or cloud centers, and is responsible for environment awareness. + +- Edge node: processes data in real time and makes decisions quickly. However, it is usually integrated on the endpoint. + +- Cloud computing center: Theoretically, computing tasks should be distributed on edge nodes. However, due to factors such as costs and solutions, the cloud computing center is still the main place for global, non-real-time, and long-period big data processing and analysis tasks, and stores a large amount of data. + +Data storage plays an important role in actual applications. Limited by performance, endpoints cannot store a large amount of data. It can only upload data to the cloud center nearby (involving remote transmission, data compression, network speed control, and on-demand triggering), while results of edge computing are permanently stored by the cloud computing center. + +Due to tight budget, it is hard to deploy workloads on the cloud, edge, and device at the same time or in a timely manner. Take smart vehicles as an example: + +1. Sensors on a vehicle collect data and send the data to the TBOX or domain controller (VDCM/BDCM) through the CAN bus and SOME/IP protocol. + +2. The TBOX on the vehicle or the MCU in the domain controller receives and calculates the data, and sends it to the MPU. The MPU runs Linux or QNX and performs feature calculation based on *model-driven on-board edge computing*, the on-board signal storage stores the original and the aggregated signals. The signals are transmitted to the HMI and the cloud computing center of the host factory through the gateway. + +3. Environment in the cloud computing center of the host factory serves either for production or algorithm development. In the production environment, the IoV platform collects uploaded signals and the battery monitoring platform obtains battery data in real time. The signals and data are input to the big data computing cluster for batch computing and real-time warning analysis on the cloud. The results are stored in the data lake and data warehouse. + +4. The algorithm development environment uses an independent test data pool to copy signals from the data lake. The developed cloud algorithm uses the latest data for exploration and analysis, and the successfully developed model is sent to the vehicle for deployment. + +Edge computing ensures data to be processed in a timely and secure manner, even without network connection. + +## Use of openEuler in Edge Computing + +Integrated by openEuler 21.09 Edge, the edge-cloud synergy framework KubeEdge can perform unified management and provisioning of edge-cloud applications. Take the edge computing platform empowered by Kunpeng and openEuler as an example. By analyzing and comparing license plate images and vehicle information read by RSUs on edge computing nodes and reporting exceptions in real time to toll stations, openEuler **helps remove provincial toll gates on highways**. By cooperation with State Grid Corporation of China, data extraction, transfer, aggregation, analysis, and application are also achieved, creating a flexible network, computing, and storage environment. By collecting, analyzing, and automatically processing operating data, it improves O&M efficiency, reduces operating costs, and carries **monitoring applications for edge substations in the power system**. + +## Building an Edge-Cloud Synergy Cluster + +To learn more, see the process of [Building a Cloud-Edge Collaborative Cluster Using KubeEdge and iSulad on openEuler – CSDN Blog](https://gitee.com/link?target=https%3A%2F%2Fblog.csdn.net%2Fweixin_41033724%2Farticle%2Fdetails%2F121312708). + +And see the official document for deployment: [Deployment Guide on KubeEdge (openeuler.org)](https://gitee.com/link?target=https%3A%2F%2Fdocs.openeuler.org%2Fzh%2Fdocs%2F21.09%2Fdocs%2FKubeEdge%2FKubeEdge%E9%83%A8%E7%BD%B2%E6%8C%87%E5%8D%97.html). + +## Conclusion + +Edge computing is one of 10 strategic technologies in the future. With the implementation of applications such as smart city, autonomous driving, and industrial Internet, massive amounts of data will be generated at the edge. IDC predicts that the amount of data generated each year in China in 2025 will reach 48.6 ZB. Centralized cloud computing cannot meet the requirements of frequent data interactions in terms of bandwidth loads, network latency, and data management costs. The value of edge computing is highlighted. + +As a technology emerging in recent years, edge computing is gradually entering our lives. openEuler keeps pace with the era. With the benefit of combination with hardware, openEuler has been establishing itself in edge computing. **The monitoring applications of edge substations in the power system** and **the cancellation of provincial toll stations on national highways** make these technologies closer to us. + +## References + +- [Edge Computing: Development and Challenges (Zhihu.com)](read://https_zhuanlan.zhihu.com/?url=https%3A%2F%2Fzhuanlan.zhihu.com%2Fp%2F432773278) +- [Building a Cloud-Edge Collaborative Cluster Using KubeEdge and iSulad on openEuler – CSDN Blog](https://gitee.com/link?target=https%3A%2F%2Fblog.csdn.net%2Fweixin_41033724%2Farticle%2Fdetails%2F121312708) +- [Kunpeng + OpenEuler Software and Hardware Collaborative Edge Computing Platform – Edge Computing Consortium (ecconsortium.org)](http://www.ecconsortium.org/Lists/show/id/515.html) +- [openEuler-White Paper-2109.pdf](https://www.openeuler.org/whitepaper/openEuler-whitepaper-2109.pdf) \ No newline at end of file diff --git a/documentation/introduction-to-the-openEuler-network-subsystem.md b/documentation/introduction-to-the-openEuler-network-subsystem.md new file mode 100644 index 0000000000000000000000000000000000000000..68d5e90ac5e6dfeac443971baa63aa587556f4ec --- /dev/null +++ b/documentation/introduction-to-the-openEuler-network-subsystem.md @@ -0,0 +1,117 @@ +# Introduction to the openEuler Network Subsystem + +Interconnection between computers is key to the computer network for ensuring information transmission and sharing. As bottom-layer software, the OS plays an important role in communication. + +> The openEuler network subsystem is responsible for network I/Os. It exchanges data with network devices (such as routers and switches). + +This blog briefly introduces the network subsystem of openEuler by explaining the network model and the innovative technologies of the openEuler kernel in network communication. In this way, it may provide you with a basic understanding of the network subsystem of openEuler. + +## Network Model + +The OSI seven-layer reference model is a standard system formulated by the ISO for interconnection between computers or communication systems. It specifies a series of abstract term concepts and protocols. However, in reality, the TCP/IP five-layer model has gained an advantage by quickly occupying the market (the first one takes all). TCP/IP consists of only five layers: application layer, transport layer, network layer, link layer, and physical layer, greatly reducing the workload of hardware vendors. The presentation layer and session layer in the OSI model are replaced with the application layer in the TCP/IP model. + +The computer network is used for end-to-end communication. On the data plane, almost each layer directly communicates with the same layer on the peer end, regardless of details of the bottom layer. The following describes the main functions of the five-layer model. + +- Physical layer: transmits data packets physically, such as Wi-Fi and Ethernet protocols. +- Data link layer: transmits data between endpoints. +- Network layer: forwards data packets and encodes hosts. +- Transport layer: transmits data between nodes. +- Application layer: various application layer protocols, such as HTTP and FTP. + +Generally, the physical layer and application layer are not involved in the OS. Therefore, pay attention only to the following in most cases: + +1. Data protocol parsing at the data link layer: After a data packet is received, it is transmitted to the network layer. The destination determines the subsequent operations. + +2. Protocol parsing at the network layer: Analyze the destination. If the destination is the local host, the data is uploaded. Otherwise, the data is returned to the data link layer for transmission. + +3. Data sending at the transport layer: The transport layer delivers data packets in sequence based on the given data packet format. + +## Linux Subsystem + +As a general-purpose OS, Linux is required to be compatible with most hardware. Therefore in practice, the gaps between physical devices are narrowed by logical abstraction and parameter adjustment. The following takes `net_device`, the most important structure of Linux network devices, as an example. + +``` +struct net_device { + char name[IFNAMSIZ]; + struct netdev_name_node *name_node; + struct dev_ifalias __rcu *ifalias; + /* + * I/O specific fields + * FIXME: Merge these and struct ifmap into one + */ + unsigned long mem_end; + unsigned long mem_start; + unsigned long base_addr; + int irq; + + /* + * Some hardware also needs these fields (state,dev_list, + * napi_list,unreg_list,close_list) but they are not + * part of the usual set specified in Space.c. + */ + + unsigned long state; + + struct list_head dev_list; + struct list_head napi_list; + struct list_head unreg_list; + struct list_head close_list; + struct list_head ptype_all; + struct list_head ptype_specific; + + struct { + struct list_head upper; + struct list_head lower; + } adj_list; + + netdev_features_t features; + netdev_features_t hw_features; + netdev_features_t wanted_features; + netdev_features_t vlan_features; + netdev_features_t hw_enc_features; + netdev_features_t mpls_features; + netdev_features_t gso_partial_features; + + int ifindex; + int group; + …… +``` + +For details about the code, see [https://github.com/openeuler-mirror/native-turbo-kernel/blob/62f8a4e02acf2a3367942f70f127def4138dd213/include/linux/netdevice.h#:~:text=struct%20net_device%20%7B,int%09%09%09group%3B](https://github.com/openeuler-mirror/native-turbo-kernel/blob/62f8a4e02acf2a3367942f70f127def4138dd213/include/linux/netdevice.h#:~:text=struct net_device {,int group%3B). As the main implementation of the data link layer, the network device driver is responsible for two major tasks: + +- Receiving data packets and transmitting them to the network layer +- Sending data packets to other ends + +As an abstraction of network devices, this structure contains all information about the network device driver. However, in reality, additional definition is required. In this case, you only need to define a structure to contain the `net_device` structure as a member. Before using this function, you need to register the device by calling register_netdev() and define the data structure. + +Here we can see that the Linux protocol stack calls some functions to implement general processing at the data link layer and routing at the IP layer. + +> Remember that the layers of the OSI model and TCP/IP model are logical, which makes it easy to understand and define the boundaries and responsibilities of software design. + +## XDP + +The openEuler 21.09 White Paper mentions a kernel innovation, that is, support for XDP. + +> XDP is a powerful new networking feature that enables high-performance programmable access to networking packets before they enter the networking stack, improving network performance. It works in various scenarios such as anti-DDoS, firewall, and network QoS. + +XDP advantages: programmability and kernel collaboration. + +**Programmability**: In the trend of intelligent network hardware, programmability can be applied to multiple scenarios. + +**Kernel collaboration**: XDP does not completely bypass the kernel. It can work with the kernel when necessary to facilitate unified network management and deployment. + +XDP defines a limited execution environment and runs in an eBPF instruction VM. It processes user-defined data packets (including redirection ones) at the position where the packets can be processed the earliest after the NIC receives them. + +For example, when the server is handling DDoS attacks, XDP can directly process the IP addresses in the blocklist in the kernel, which greatly improves the CPU utilization. + +## References + +[openEuler Network Subsystem: Introduction to the Architecture of the openEuler Network Subsystem (gitee.com)](https://gitee.com/MrRlu/openeuler_network_subsystem#结语) + +[Linux Kernel Network (1) – Preliminary Study on Kernel Network](https://gitee.com/link?target=https%3A%2F%2Fzhuanlan.zhihu.com%2Fp%2F363718587) + +[openEuler-whitepaper-2109.pdf](https://gitee.com/link?target=https%3A%2F%2Fwww.openeuler.org%2Fwhitepaper%2FopenEuler-whitepaper-2109.pdf) + +[Using eXpress Data Path (XDP) to Defend Against DDoS Attacks](https://gitee.com/link?target=https%3A%2F%2Fblog.csdn.net%2Fdog250%2Farticle%2Fdetails%2F77993218) + + \ No newline at end of file