# ovirt-engine **Repository Path**: ovirt-engine-dev-training/ovirt-engine ## Basic Information - **Project Name**: ovirt-engine - **Description**: No description available - **Primary Language**: Java - **License**: Apache-2.0 - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2022-04-25 - **Last Updated**: 2024-05-29 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README = oVirt Engine - Open Virtualization Manager Welcome to the oVirt Engine - Open Virtualization Manager source repository. This repository is hosted on [gerrit.ovirt.org:ovirt-engine](https://gerrit.ovirt.org/#/admin/projects/ovirt-engine) and a **backup** of it is hosted on [GitHub:ovirt-engine](https://github.com/oVirt/ovirt-engine) == How to contribute === Submitting patches Patches are welcome! Please submit patches to [gerrit.ovirt.org:ovirt-engine](https://gerrit.ovirt.org/#/admin/projects/ovirt-engine). If you are not familiar with the review process for Gerrit patches you can read about [Working with oVirt Gerrit](https://ovirt.org/develop/dev-process/working-with-gerrit.html) on the [oVirt](https://ovirt.org/) website. **NOTE**: We might not notice pull requests that you create on Github, because we only use Github for backups. === Found a bug or documentation issue? To submit a bug or suggest an enhancement for oVirt Engine please use [oVirt Bugzilla for ovirt-engine product](https://bugzilla.redhat.com/enter_bug.cgi?product=ovirt-engine). If you find a documentation issue on the oVirt website please navigate and click "Report an issue on GitHub" in the page footer. == Still need help? If you have any other questions, please join [oVirt Users forum / mailing list](https://lists.ovirt.org/admin/lists/users.ovirt.org/) and ask there. == Developer mode installation === Preparations ==== Prerequisites Install the following system components: - java-11-openjdk-devel - mime-types or mailcap - unzip - openssl - bind-utils - postgresql-server >= 12.0 - postgresql >= 12.0 - postgresql-contrib >= 12.0 - python3-dateutil / dateutil - python3-cryptography / cryptography - python3-m2crypto / m2crypto - python3-psycopg2 / psycopg - python3-jinja2 / Jinja2 - python3-libxml2 / libxml2[python] - python3-daemon - python3-otopi >= 1.1.0 - python3-ovirt-setup-lib - maven >= 3.5 - ansible >= 2.9.0 - ansible-runner-service-dev >= 1.0.6 - ovirt-ansible-roles >= 1.2.0 - ovirt-imageio-daemon >= 2.0.6 - ovirt-engine-metrics (optional) - python3-ovirt-engine-sdk4 (optional) - ansible-lint / python3-ansible-lint (optional) - python-flake8 / pyflakes (optional) - python-pep8 / pep8 (optional) - python-docker-py (optional) - python2-isort (optional) - python3-distro ==== Note on Java versions The project is built and run using java 11. 4.3 branches and earlier are excluded. ==== Prepare your dev environment for java 11 - Use `alternatives` command to configure `java` and `javac` to version 11: ```console $ sudo alternatives --config java There are 4 programs which provide 'java'. Selection Command ----------------------------------------------- 1 java-latest-openjdk.x86_64 (/usr/lib/jvm/java-12-openjdk-12.0.1.12-1.rolling.fc30.x86_64/bin/java) 2 java (/opt/jdk-9/bin/java) 3 java-11-openjdk.x86_64 (/usr/lib/jvm/java-11-openjdk-11.0.3.7-5.fc30.x86_64/bin/java) *+ 4 java-1.8.0-openjdk.x86_64 (/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.fc30.x86_64/jre/bin/java) Enter to keep the current selection[+], or type selection number: 3 $ sudo alternatives --config javac There are 2 programs which provide 'javac'. Selection Command ----------------------------------------------- *+ 1 java-1.8.0-openjdk.x86_64 (/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.fc30.x86_64/bin/javac) 2 java-11-openjdk.x86_64 (/usr/lib/jvm/java-11-openjdk-11.0.3.7-5.fc30.x86_64/bin/javac) Enter to keep the current selection[+], or type selection number: 2 ``` - Maven >= 3.5 is required, download and extract if not installed using distribution package management, and add to PATH. Fedora 30 ```console $ sudo dnf module install maven ``` - verify your `mvn` version: ```console $ mvn -v | head -1 Apache Maven 3.5.4 (Red Hat 3.5.4-5) ``` - export `JAVA_HOME` if `mvn` is not executing using java-11: ```console #put this in your ~/.bashrc $ export JAVA_HOME=/lib/jvm/java-11 $ mvn -v | grep "Java version: " Java version: 11.0.4, vendor: Oracle Corporation, runtime: /usr/lib/jvm/java-11-openjdk-11.0.4.11-0.fc30.x86_64 ``` WildFly 15 is required along with ovirt-engine-wildfly-overlay. Preferred way is to install following packages: - ovirt-engine-wildfly - ovirt-engine-wildfly-overlay Both packages can be installed from ovirt-master-snapshot-static repository: [ovirt-master-snapshot-static] name=Latest oVirt master additional nightly snapshot baseurl=http://resources.ovirt.org/pub/ovirt-master-snapshot-static/rpm/@DIST@$releasever/ enabled=1 skip_if_unavailable=1 gpgcheck=0 Please replace `@DIST@` with `fc` for Fedora or `el` for Centos/RHEL. Alternatively, repository list can be updated using the following command: $ sudo yum install -y http://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm OVN/OVS is an optional dependency. If you want to use it, check the requirements in the ovirt-engine.spec.in file for a list of packages. Otherwise, you should reply 'No' when asked about it by engine-setup. For Fedora prerequisites installation, following command can be applied: $ sudo yum install -y $(cat automation/check-patch.packages) Install additional packages for Fedora: $ sudo yum install -y \ ansible \ ansible-runner-service-dev \ bind-utils \ libxml2-python \ m2crypto \ mailcap \ openssl \ ovirt-ansible-roles \ ovirt-engine-metrics \ ovirt-engine-wildfly \ ovirt-engine-wildfly-overlay \ ovirt-setup-lib \ python2-daemon \ python2-dateutil \ python2-jinja2 \ python-ovirt-engine-sdk4 \ unzip Optional packages for Fedora: $ sudo yum install -y \ python2-docker-py \ python2-flake8 \ python2-isort \ python2-pep8 ==== System settings Build locales requires at least 10240 file descriptors, create the following file, replace with user that is used for building, and logout/login: ./etc/security/limits.d/10-nofile.conf ---- hard nofile 10240 # soft nofile 10240 # optional, to apply automatically ---- If soft limit was not set, before building, apply new limit using: $ ulimit -n 10240 Development environment by default uses ports 8080 (HTTP), 8443 (HTTPS), 8787 (java debug), and 54323 (ovirt-imageio-proxy) so make sure they are accessible from the outside. For example: firewall-cmd --add-port=8080/tcp --permanent firewall-cmd --add-port=8443/tcp --permanent firewall-cmd --add-port=8787/tcp --permanent firewall-cmd --add-port=54323/tcp --permanent If you also want to connect to the database from the outside: firewall-cmd --add-port=5432/tcp --permanent Finally, apply changes using: firewall-cmd --reload If compiling in a virtual machine, javac might experience difficulties on guests with dynamically growing RAM so it's better to have VM's starting allocation and maximum allocation set to the same value. ==== PostgreSQL accessibility Initialize PostgreSQL configuration files: $ sudo postgresql-setup --initdb --unit postgresql # fedora Configure PostgreSQL to accept user and password: Locate `pg_hba.conf` within your distribution, common locations are: - `/var/lib/pgsql/data/pg_hba.conf` - `/etc/postgresql-*/pg_hba.conf` - `/etc/postgresql/*/main/pg_hba.conf` Within `pg_hba.conf` set method to `password` for `127.0.0.1/32` and `::1/128` for IPv4 and IPv6 local connections correspondingly. If you want to make postgres accessible from the outside, change `127.0.0.1/32` to `0.0.0.0/0` and `::1/128` to `::/0`. Tune PostgreSQL configuration: Locate `postgresql.conf` within your distribution, common locations are: - `/var/lib/pgsql/data` - `/etc/postgresql*` Within `postgresql.conf` make sure following values are set: max_connections = 150 work_mem = 8MB autovacuum_max_workers = 6 autovacuum_vacuum_scale_factor = 0.01 autovacuum_analyze_scale_factor = 0.075 maintenance_work_mem = 64MB If you want to connect from the outside, set also: listen_addresses = '*' Enable and start (`systemctl enable postgresql --now`). ==== Database creation Create database for ovirt-engine, usually the following sequence should work to create a user named `engine` that owns database named `engine`: # su - postgres -c "psql -d template1" template1=# create user engine password 'engine'; template1=# drop database engine; template1=# create database engine owner engine template template0 encoding 'UTF8' lc_collate 'en_US.UTF-8' lc_ctype 'en_US.UTF-8'; template1=# \q Enable uuid-ossp extension for the database: # su - postgres -c "psql -d engine" engine=# CREATE EXTENSION "uuid-ossp"; engine=# \q ==== Ansible Runner Service configration Since oVirt 4.4 the engine is integrated with Ansible Runner Service. To properly integrate the development environment with the Ansible Runner Service, you need to edit `/etc/ansible-runner-service/config.yaml` file as follows: --- playbooks_root_dir: '$PREFIX/share/ovirt-engine/ansible-runner-service-project' ssh_private_key: '$PREFIX/etc/pki/ovirt-engine/keys/engine_id_rsa' port: 50001 target_user: root Please replace `$PREFIX` with the real path of your development environment, which you've specified during the compilation of the engine, for example: playbooks_root_dir: '/home/user/ovirt-engine/share/ovirt-engine/ansible-runner-service-project' ssh_private_key: '/home/user/ovirt-engine/etc/pki/ovirt-engine/keys/engine_id_rsa' After edditing the file, make sure you've restarted the Ansible Runner Service service: # systemctl restart ansible-runner-service # systemctl enable ansible-runner-service === Development ==== Environment Development environment is supported only under non-root account. Do not run this sequence as root. Each instance of application must be installed at different `PREFIX` and use its own database. Throughout this document application is installed using `PREFIX="${PREFIX}"` and engine database and user, these should be changed if a new instance is required. Do not mix different versions of product with same `PREFIX/database`. From this point on, the `"${PREFIX}"` will be used to mark the prefix in which you selected to install the development environment. ==== Build To build and install ovirt-engine at your home folder under ovirt-engine directory execute the following command: $ make clean install-dev PREFIX="${PREFIX}" NOTE: `${PREFIX}` should be replaced with the location in which you intend to install the environment. NOTE: Add SKIP_CHECKS=1 to disable tests. ===== Build targets all:: Build project. clean:: Clean project. all-dev:: Build project for development. install-dev:: Install a development environment at PREFIX. dist:: Create source tarball out of git repository. maven:: Force execution of maven. generated-files:: Create file from templates (.in files). + When creating new templates, generated files will be automatically appears in .gitignore, updated .gitignore should be part of commiting new templates. ===== Build customization The following `Makefile` environment variables are available for build customization: PREFIX:: Installation root directory. Default is `/usr/local`. BUILD_GWT:: Build GWT. Default is `1`. BUILD_ALL_USER_AGENTS:: Build GWT applications for all supported browsers. Default is `0`. BUILD_LOCALES:: Build GWT applications for all supported locales. default is `0`. BUILD_DEV:: Add extra development flags. Usually this should not be used directly, as the all-dev sets this. Default is `0`. BUILD_UT:: Perform unit tests during build. Default is `0`. BUILD_JAVA_OPTS_MAVEN:: Maven JVM options. Can be defined as environment variable. Default is empty. BUILD_JAVA_OPTS_GWT:: GWT compiler and dev mode JVM options. Can be defined as environment variable. default is empty. NOTE: Note that `BUILD_JAVA_OPTS_GWT` overrides `BUILD_JAVA_OPTS_MAVEN` when building GWT applications (`BUILD_JAVA_OPTS_MAVEN` settings still apply, unless overridden). DEV_BUILD_GWT_DRAFT:: Build "draft" version of GWT applications without optimizations. This is useful when profiling compiled applications in web browser. Default value is `0`. + Following changes are applied for draft builds: - Prevent code and CSS obfuscation. - Reduce the level of code optimizations. + On local development environment, using GWT Super Dev Mode (see below) is preferred, as it automatically disables all optimizations and allows you to recompile the GWT application on the fly. + DEV_BUILD_GWT_SUPER_DEV_MODE:: Allows debugging GWT applications via Super Dev Mode, using web browser's JavaScript development tooling. Default value is `1`. + Do a local Engine development build as you normally would. Then, start the Super Dev Mode code server as following: $ make gwt-debug DEV_BUILD_GWT_SUPER_DEV_MODE=1 In your browser, open http://127.0.0.1:9876/ and save the "Dev Mode On" bookmark. Next, visit the GWT application URL (as served from Engine) and click "Dev Mode On". This allows you to recompile and reload the GWT application, reflecting any changes you've made in the UI code. DEV_EXTRA_BUILD_FLAGS:: Any maven build flags required for building. + For example, if your machine is low on memory, limit maximum simultaneous GWT permutation worker threads: + DEV_EXTRA_BUILD_FLAGS="-Dgwt.compiler.localWorkers=1" DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS:: Any maven build flags required for building GWT applications. + By default, GWT applications are built for Firefox only. To build for additional browsers, provide comma-separated list of user agents, see `frontend/webadmin/modules/pom.xml` for full list. + For example, to build for Firefox and Chrome: + DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS="-Dgwt.userAgent=gecko1_8,safari" + To build for all supported browsers, use `BUILD_ALL_USER_AGENTS=1`. + For example, to build only the English and Japanese locale: + DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS="-Dgwt.locale=en_US,ja_JP" + To build for all supported locales, use `BUILD_LOCALES=1`. + For example to build engine without obfuscated Javascript code: + DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS="-Dgwt.style=pretty" + + To build engine without obfuscated CSS styles: + DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS="-Dgwt.cssResourceStyle=pretty" + DEV_REBUILD:: Disable if only packaging components were modified. Default is `1`. PY_VERSION:: Python defaults to python3 if available, use PY_VERSION=2 in order to override. + This options affects various services and several features written in python. NOTE: `engine-setup` which runs otopi, uses different customized variable `OTOPI_PYTHON` WILDFLY_OVERLAY_MODULES:: Change location of WildFly overlay modules. If you want to disable WildFly overlay configuration completely, please set to empty string. Default is `/usr/share/ovirt-engine-wildfly-overlay/modules`. === Setup To setup the product use the following command: $ "${PREFIX}/bin/engine-setup" NOTE: otopi, and therefore engine-setup, now defaults to python3 except el7, use: + `$ OTOPI_PYTHON=/usr/bin/python2 "${PREFIX}/bin/engine-setup"` + to override. During engine setup, a certificate has to be issued and you will be asked for a hostname. If you want to upload and download images from administration portal, it has to be the name by which your machine is accessible from the outside. === JBoss If you want to use different WildFly/EAP installation, specify it at `--jboss-home=` parameter of setup. === Environment OVIRT_ENGINE_JAVA_HOME:: Select a specific Java home. OVIRT_ENGINE_JAVA_HOME_FORCE:: Set to non zero to bypass Java compatibility check. === Refresh If there are no significant changes, such as file structure or database schema, there is no need to run the setup again, `make install-dev ` will overwrite files as required, run `engine-setup` to refresh database schema. Do remember to restart the engine service. If there is a significant change, safest path is to stop service, remove `${PREFIX}` directory, build and setup. The `${PREFIX}/bin/engine-cleanup` tool is also available to cleanup the environment, it is useful for application changes, less for packaging changes. === Service administration Most utilities and services are operational, including PKI, host deploy. To start/stop the engine service use: $ "${PREFIX}/share/ovirt-engine/services/ovirt-engine/ovirt-engine.py" start While the service is running, this command will not exit. Press -C to stop service. Access using HTTP or HTTPS: - http://:8080 - https://:8443 === Remote debug By default, debug address is `127.0.0.1:8787`. If you want to make engine accessible to the remote debugger, after running engine-setup edit the following file: ${PREFIX}/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf: ENGINE_DEBUG_ADDRESS=0.0.0.0:8787 === Running instance management (JMX) ovirt-engine service supports jmx as management interface. Actually, this is the standard jboss jmx interface, while authentication can be done using any engine user with SuperUser role. Access is permitted only from the local host. Access JMX shell using provide OPTIONAL_COMMAND for non interactive usage: $ "${JBOSS_HOME}/bin/jboss-cli.sh" \ --connect \ --timeout=30000 \ --controller=localhost:8706 \ --user=admin@internal \ --commands="OPTIONAL_COMMA_SEPARATED_COMMANDS" Useful commands: Modify log level:: + /subsystem=logging/logger=org.ovirt.engine.core.bll:write-attribute(name=level,value=DEBUG) Create a new log category:: + /subsystem=logging/logger=org.ovirt.engine:add Get the engine data-source statistics:: + ls /subsystem=datasources/data-source=ENGINEDataSource/statistics=jdbc/ Get threading info:: + ls /core-service=platform-mbean/type=threading/ By default JMX access is available only to localhost, to open JMX to world, add `${PREFIX}/etc/ovirt-engine/engine.conf.d/20-setup-jmx-debug.conf` with: ENGINE_JMX_INTERFACE=public === DAO tests Create empty database for DAO tests refer to <>. Provided user is `engine`, password is `engine` and database is `engine_dao_tests`. $ PGPASSWORD=engine \ ./packaging/dbscripts/schema.sh \ -c apply -u engine -d engine_dao_tests Run build as: $ make maven BUILD_GWT=0 BUILD_UT=1 EXTRA_BUILD_FLAGS="-P enable-dao-tests \ -D engine.db.username=engine \ -D engine.db.password=engine \ -D engine.db.url=jdbc:postgresql://localhost/engine_dao_tests" === VM console After the environment is setup and installed, some adjustments are required. Copy `vmconsole-host` configuration: $ sudo cp -p "${PREFIX}/share/ovirt-engine/conf/ovirt-vmconsole-proxy.conf \ /etc/ovirt-vmconsole/ovirt-vmconsole-proxy/conf.d/50-ovirt-vmconsole-proxy.conf If selinux is enabled on your machine, set type on vmconsole helper using: $ sudo chcon --type=bin_t "${PREFIX}/libexec/ovirt-vmconsole-proxy-helper/ovirt-vmconsole-list.py" === ovirt-imageio After setup, you need to run ovirt-imageio manually if you want to upload and download images via the administration portal. To run ovirt-imageio, run the following command: $ ovirt-imageio --conf-dir $PREFIX/etc/ovirt-imageio This assumes you have installed `ovirt-imageio-daemon` and you have run `engine-setup`. In development mode, ovirt-imageio logs to stderr using DEBUG level. If you would like to log to a file create a log directory: $ mkdir $PREFIX/var/log/ovirt-imageio And install a drop-in configuration file to override engine developement setup: $ cat $PREFIX/etc/ovirt-imageio/conf.d/99-local.conf [handlers] keys = logfile [logger_root] handlers = logfile [handler_logfile] args = ('/home/username/ovirt-engine/log/ovirt-imageio/daemon.log',) === RPM packaging $ make dist $ rpmbuild -ts @tarball@ # yum-builddep @srpm@ # rpmbuild -tb @tarball@ The following spec file variables are available for package customization: ovirt_build_quick:: Quick build, best for syntax checks. Default is `0`. ovirt_build_minimal:: Build minimal Firefox only package. Default is `0`. ovirt_build_user_agent:: When using quick or minimal build, build only for this user agent. Default is `gecko1_8` (Firefox). To build for Chrome use `safari`. ovirt_build_gwt:: Build GWT components. Default is `1`. ovirt_build_all_user_agents:: Build GWT components for all supported browsers. Default is `1`. ovirt_build_locales:: Build GWT components for all supported locales. Default is `1`. Examples: Build minimal rpm package for Firefox:: $ rpmbuild -D"ovirt_build_minimal 1" -tb @tarball@ Build minimal rpm package for Chrome or Safari:: $ rpmbuild -D"ovirt_build_minimal 1" -D"ovirt_build_user_agent safari" -tb @tarball@