diff --git a/ohos/docs/04_development/FlutterEnginePreload.md b/ohos/docs/04_development/FlutterEnginePreload.md new file mode 100644 index 0000000000000000000000000000000000000000..66b43676daafa85f2fdcbc67604a1c1b7da98b6a --- /dev/null +++ b/ohos/docs/04_development/FlutterEnginePreload.md @@ -0,0 +1,133 @@ +# FlutterEnginePreload User Guide + +`FlutterEnginePreload` aims to accelerate the rendering of hybrid applications combining native and Flutter, optimizing user experience. The core idea is to move part of the execution logic to the moment of user interaction (e.g., button press) instead of executing it after the event is triggered (e.g., button release). + +## Prerequisites + +The application needs a mechanism to navigate from a native page to a Flutter page. For instance, when a user clicks a button to navigate, preloading can be used to speed up Flutter page rendering. + +--- + +## Steps to Use + +In the native `arkts` page, first import the following utility class: + + import { FlutterEnginePreload } from '@ohos/flutter_ohos'; + +### External Interfaces Provided + +`FlutterEnginePreload` provides two core interfaces: + +1. **`preloadEngine`** +2. **`predrawEngine`** + +--- + +## Interface Description + +### 1. `preloadEngine` + +The parameters of `preloadEngine` are as follows: + +- **`Context`**: Pass the current page's `context`. +- **`Params`**: Pass the corresponding NAPI parameters, which are mostly consistent with `FlutterEntry` and `FlutterAbility`. Detailed settings can refer to `FlutterAbilityLaunchConfigs`, including but not limited to: + - **`dart_entrypoint`**: Sets the entry point function name (default is `main`). + - **`route`**: Sets the default initial route (default is `/`). + - **`cached_engine_id`**: Sets the unique ID of the cached `FlutterEngine` you want to reuse. If this parameter is passed, the pre-created engine can be reused (the engine must have been previously added to the `FlutterEngineCache` for management). + - **`viewport_metrics`**: Developers can set `viewport_metrics` during preloading, especially when the Flutter page to be rendered is not fullscreen. You need to set `viewportMetrics.physicalWidth` and `viewportMetrics.physicalHeight` as the width and height for pre-rendering; otherwise, it defaults to fullscreen pre-rendering. +- **`nextViewId`**: Identifies the `FlutterView` ID to be pre-rendered. You can pass `null`, which will automatically retrieve the next ID by calling `FlutterManager.getInstance().getNextFlutterViewId()`. + - **Note**: When pre-rendering multiple pages simultaneously, different `nextViewId` values must be passed. You can use `getNextFlutterViewId(offset)` to pass an offset and get the `(offset+1)`th `viewId`. + +--- + +### 2. `predrawEngine` + +The `predrawEngine` interface is similar to `preloadEngine`, except that its first parameter directly passes a `FlutterEngine` instance. Therefore, the pre-created engine does not need to be maintained by `FlutterEngineCache`. + +The usage and configuration of the other parameters are the same as `preloadEngine`. + +--- + +## Implementation Examples + +### Example 1: Simple Case + +Below is the typical implementation logic for preloading a Flutter page through a button: + +```arkts +build() { + Button(this.title) + .onClick(() => { + try { + router.pushUrl({ url: 'pages/Flutter', params: { route: this.route } }); + } catch (err) { + Log.d(TAG, `跳转页面${this.route} error === ${JSON.stringify(err)}`); + } + }) + .onTouch((touch: TouchEvent) => { + if (touch.type == TouchType.Down) { + let params: Record = {}; + params['route'] = this.route; + FlutterEnginePreload.preloadEngine(getContext(this), params); + } + }); +} +``` + +For a complete demo, refer to: [Flutter_it_preload](https://gitee.com/geng_fei/flutter_it/tree/flutter_page-multi_engine-predraw/) + +**Preloading Logic Location**: `pages/index.ets` + +--- + +### Example 2: Complex Case + +Below is a more complex scenario where a cached `FlutterEngine` is used, the target page contains multiple Flutter pages, and dimensions and `FlutterViewId` offsets need to be set: + +```arkts +.onTouch((touch: TouchEvent) => { + if (touch.type == TouchType.Down) { + let params: Record = {}; + let viewports: ViewportMetrics = new ViewportMetrics(); + // 多引擎时,对传入的 view ID 手动添加偏移 + let nextViewId = FlutterManager.getInstance().getNextFlutterViewId(); + let nextViewId2 = FlutterManager.getInstance().getNextFlutterViewId(1); + viewports.physicalWidth = display.getDefaultDisplaySync().width; + viewports.physicalHeight = display.getDefaultDisplaySync().height / 2; + params[FlutterAbilityLaunchConfigs.PRELOAD_VIEWPORT_METRICS_KEY] = viewports; + + FlutterEnginePreload.predrawEngine( + FlutterEngineCache.getInstance().get('DoubleFlutterTop'), + params, + nextViewId + ); + + FlutterEnginePreload.predrawEngine( + FlutterEngineCache.getInstance().get('DoubleFlutterBottom'), + params, + nextViewId2 + ); + } +}); +``` + +For a complete demo, refer to: [Multiple_flutters_predraw](https://gitee.com/geng_fei/multiple_flutters/tree/predraw/) + +**Preloading Logic Location**: `pages/MainPage.ets` + +--- + +## Verification of Results + +After correctly adding `preload` in the project, the rendering speed of page navigation can be significantly optimized. To verify the effect before and after optimization, you can: + +1. **Slow-motion recording**: Observe the speed of Flutter page loading during navigation. +2. **Capture application traces**: Analyze the changes in page loading time. +3. **DevTools**: During mixed application development and debugging, the native application is opened by default, and DevTools will not establish a connection. When preload is triggered (e.g., touch event), even if no actual navigation occurs, the connection will be automatically established. + +--- + +## Notes + +1. When setting `viewport_metrics`, ensure that the passed width and height match the actual rendering area. +2. If multiple pages are to be pre-rendered simultaneously, ensure that `nextViewId` values are not repeated. \ No newline at end of file diff --git a/ohos/docs/04_development/Import-Custom-Rendered-Content-into-Flutter.md b/ohos/docs/04_development/Import-Custom-Rendered-Content-into-Flutter.md new file mode 100644 index 0000000000000000000000000000000000000000..1d7d2172c2676dd9d9e78ddfe340cd45681ff8ac --- /dev/null +++ b/ohos/docs/04_development/Import-Custom-Rendered-Content-into-Flutter.md @@ -0,0 +1,145 @@ +# How to Import Self-Rendered Content into Flutter + +## Background Introduction + +We support importing externally self-rendered content (whether drawn using `NativeDrawing` and other HarmonyOS CAPI or via OpenGL-related interfaces) into Flutter's `Texture` widget through external textures for display. Below are the specific implementation steps. + +--- + +## Implementation Steps + +### 1. Dart-Side Operations + +#### 1.1 Create a Texture Widget + +- Create a Texture widget and bind the `textureId`. + +#### 1.2 Register the Texture + +- Create a `MethodChannel`: + + final MethodChannel _channel = const MethodChannel('PictureChannel'); + +- Register the texture in the `initState` function: + + _textureId = await _channel.invokeMethod("registerStarTexture"); + +#### 1.3 Unregister the Texture + +- Unregister the texture in the `dispose` function: + + _channel.invokeMethod('unregisterTexture', {'textureId': _textureId}); + +--- + +### 2. HarmonyOS-Side Operations + +#### 2.1 Import the Utility Class + +You need to import the following utility class: + + import { TextureRegistry, SurfaceTextureEntry } from '@ohos/flutter_ohos/src/main/ets/view/TextureRegistry'; + +#### 2.2 Provided Implementation Methods + +We provide two methods for importing self-rendered content into Flutter textures. + +--- + +## Method 1: Draw to a Buffer via `getNativeWindowId` + +Obtain the engine-level `OHNativeWindow` pointer corresponding to the `Texture` by using `getNativeWindowId`, and then draw self-rendered content to the buffer associated with the `window`. + +### ArkTS-Side Implementation Steps + +1. Obtain the relevant information when registering the texture: + +```arkts +this.textureId = this.textureRegistry!.getTextureId(); +this.surfaceTextureEntry = this.textureRegistry!.registerTexture(this.textureId); +this.windowId = this.surfaceTextureEntry.getNativeWindowId(); +drawNapi.drawPatternWindow(this.windowId, 300, 300); // Needs to be implemented in the cpp layer, receiving the window pointer +``` + +2. Call the `cpp` layer function to draw self-rendered content to the `window`: + + drawNapi.drawPatternWindow(this.windowId, 300, 300); // Needs to be implemented in the cpp layer + +### CPP-Side Implementation Steps + +1. Use `OH_NativeWindow_NativeWindowHandleOpt` to set the `window`'s `USAGE`, width, height, and other information. +2. Call `OH_NativeWindow_NativeWindowRequestBuffer` to get the buffer of the `window`. +3. Use `OH_NativeWindow_GetBufferHandleFromNative` to get the `bufferHandle`. +4. Use `OH_NativeWindow_NativeWindowFlushBuffer` to put the produced buffer into the buffer queue for consumption. + +--- + +## Method 2: Use `setExternalNativeImage` + +Directly pass the `NativeImage` pointer used for content production to the Flutter engine layer. The engine layer uses the `NativeImage` and registers a frame-available callback using `OH_NativeImage_SetOnFrameAvailableListener` for content consumption. + +### ArkTS-Side Implementation Steps + +1. Obtain the relevant information when registering the texture: + +```arkts +this.textureId = this.textureRegistry!.getTextureId(); +this.surfaceTextureEntry = this.textureRegistry!.registerTexture(this.textureId); +drawNapi.drawPatternWindow(1000, 1000); +this.nativeImageId = drawNapi.getNativeImage(); // Get the NativeImage used for content production +this.textureRegistry!.setExternalNativeImage(this.textureId, this.nativeImageId); // Pass the NativeImage to the engine layer +``` + +### CPP-Side Implementation Steps + +Here are the steps for a simple drawing implementation based on OpenGL: + +1. Use `OH_NativeImage_Create` to create a `NativeImage`. +2. Use `OH_NativeImage_AcquireNativeWindow` to get the `window` associated with the `NativeImage`. +3. Use `OH_NativeWindow_NativeWindowHandleOpt` to set `USAGE` and width/height. +4. Convert `NativeWindow` to the type `EGLNativeWindowType`. +5. Call the following OpenGL-related commands to complete the rendering: + - `eglCreateWindowSurface` + - `eglCreateContext` + - `eglMakeCurrent` + - OpenGL drawing commands + - `eglSwapBuffers` + +--- + +## Texture Unregistration + +No matter which method is used above, the following function needs to be called to unregister the texture: + + this.textureRegistry!.unregisterTexture(textureId); + +--- + +## Reference Implementations + +### Reference for Method 1 + +[flutter_it_window](https://gitee.com/geng_fei/flutter_it/pulls/2) + +### Reference for Method 2 + +[flutter_it_image](https://gitee.com/geng_fei/flutter_it/tree/flutter_page-multi_engine-predraw) + +--- + +## Code Location Description + +1. **Dart-Side Code** + The relevant code is located in `lib/testpicture/PicturePage.dart`. The texture component added at the bottom is used to import self-rendered content. + +2. **ArkTS-Side Code** + The relevant code is located in `ohos/entry/src/main/ets/pictureplugin/PicturePlugin.ets`, which includes functions for registering and unregistering textures. + +3. **Native Rendering Code** + The relevant code is located in `ohos/entry/src/main/cpp`. + +--- + +## Notes + +1. When using Method 2, developers need to manage the lifecycle of `NativeImage` by themselves and ensure that the object is not destroyed before `unregister`. \ No newline at end of file diff --git "a/ohos/docs/04_development/\345\246\202\344\275\225\344\275\277\347\224\250FlutterEnginePreload.md" "b/ohos/docs/04_development/\345\246\202\344\275\225\344\275\277\347\224\250FlutterEnginePreload.md" new file mode 100644 index 0000000000000000000000000000000000000000..7dae8600d12a688a3356792e14983156380694f2 --- /dev/null +++ "b/ohos/docs/04_development/\345\246\202\344\275\225\344\275\277\347\224\250FlutterEnginePreload.md" @@ -0,0 +1,133 @@ +# FlutterEnginePreload 使用指南 + +`FlutterEnginePreload` 旨在加速原生与 Flutter 混编应用的跳转渲染,优化用户体验。其基本思路是将部分执行逻辑提前到用户交互事件(如按钮按下)发生时,而不是在触发事件(如按钮抬起)后执行。 + +## 使用前提 + +应用需要通过某种方式从原生页面跳转到 Flutter 页面。例如,当用户点击一个按钮跳转页面时,可以通过预加载的方式加速 Flutter 页面渲染。 + +--- + +## 使用步骤 + +在原生 `arkts` 页面中,需要先引用以下工具类: + +`import { FlutterEnginePreload } from '@ohos/flutter_ohos';` + +### 提供的外部接口 + +`FlutterEnginePreload` 提供两个核心接口: + +1. **`preloadEngine`** +2. **`predrawEngine`** + +--- + +## 接口说明 + +### 1. `preloadEngine` + +`preloadEngine` 的参数说明如下: + +- **`Context`**:传入当前页面的 `context` 即可。 +- **`Params`**:传递对应的 NAPI 参数,基本与 `FlutterEntry` 和 `FlutterAbility` 一致。详细设置可参考 `FlutterAbilityLaunchConfigs`,包括但不限于以下参数: + - **`dart_entrypoint`**:设置入口点函数名称(默认为 `main`)。 + - **`route`**:设置默认初始路由(默认为 `/`)。 + - **`cached_engine_id`**:设置想要复用的缓存 `FlutterEngine` 的唯一标识 ID。传入此参数后,可复用预创建的引擎(需之前已将该引擎放入 `FlutterEngineCache` 缓存管理中)。 + - **`viewport_metrics`**:开发者在预加载时,可提前设置 `viewport_metrics`,特别是当待渲染的 Flutter 页面并非全屏时。需要提前设置 `viewportMetrics.physicalWidth` 和 `viewportMetrics.physicalHeight` 作为预渲染的宽高,否则默认按全屏预渲染。 +- **`nextViewId`**:标识想要预渲染的 FlutterView 的 ID。可以传 `null`,此时会通过调用 `FlutterManager.getInstance().getNextFlutterViewId()` 自动获取下一个 ID。 + - **注意**:当希望同时预渲染多个页面时,需要传入不同的 `nextViewId`。可通过 `getNextFlutterViewId(offset)` 传入偏移量获取接下来的第 `offset+1` 个 `viewId`。 + +--- + +### 2. `predrawEngine` + +`predrawEngine` 的接口与 `preloadEngine` 类似,不同之处是其第一个参数直接传递一个 `FlutterEngine` 实例。因此,预创建的引擎可以不用 `FlutterEngineCache` 维护。 + +其余参数的使用和配置与 `preloadEngine` 一致。 + +--- + +## 实现示例 + +### 示例 1:简单情况 + +以下为典型实现逻辑,通过按钮预加载 Flutter 页面: + +```arkts +build() { + Button(this.title) + .onClick(() => { + try { + router.pushUrl({ url: 'pages/Flutter', params: { route: this.route } }); + } catch (err) { + Log.d(TAG, `跳转页面${this.route} error === ${JSON.stringify(err)}`); + } + }) + .onTouch((touch: TouchEvent) => { + if (touch.type == TouchType.Down) { + let params: Record = {}; + params['route'] = this.route; + FlutterEnginePreload.preloadEngine(getContext(this), params); + } + }); +} +``` + +完整 Demo 可参考:[Flutter_it_preload](https://gitee.com/geng_fei/flutter_it/tree/flutter_page-multi_engine-predraw/) + +**预加载逻辑位置**:`pages/index.ets` + +--- + +### 示例 2:复杂情况 + +以下为更复杂的场景,使用缓存的 `FlutterEngine`,且目标页面包含多个 Flutter 页面,并需要设置尺寸以及 `FlutterViewId` 偏移量: + +```arkts +.onTouch((touch: TouchEvent) => { + if (touch.type == TouchType.Down) { + let params: Record = {}; + let viewports: ViewportMetrics = new ViewportMetrics(); + // 多引擎时,对传入的 view ID 手动添加偏移 + let nextViewId = FlutterManager.getInstance().getNextFlutterViewId(); + let nextViewId2 = FlutterManager.getInstance().getNextFlutterViewId(1); + viewports.physicalWidth = display.getDefaultDisplaySync().width; + viewports.physicalHeight = display.getDefaultDisplaySync().height / 2; + params[FlutterAbilityLaunchConfigs.PRELOAD_VIEWPORT_METRICS_KEY] = viewports; + + FlutterEnginePreload.predrawEngine( + FlutterEngineCache.getInstance().get('DoubleFlutterTop'), + params, + nextViewId + ); + + FlutterEnginePreload.predrawEngine( + FlutterEngineCache.getInstance().get('DoubleFlutterBottom'), + params, + nextViewId2 + ); + } +}); +``` + +完整 Demo 可参考:[Multiple_flutters_predraw](https://gitee.com/geng_fei/multiple_flutters/tree/predraw/) + +**预加载逻辑位置**:`pages/MainPage.ets` + +--- + +## 效果验证 + +在项目中正确加入 `preload` 后,可显著优化页面跳转的渲染速度。优化前后对比,可通过以下方式验证效果: + +1. **慢镜头拍摄**:观察跳转过程中 Flutter 页面加载的速度。 +2. **抓取应用 Trace**:分析页面加载时间的变化。 +3. **devtools**: 混编应用开发调测时,默认打开原生应用,devtools并不会建立连接,当触发preload时(例如touch事件),即使并未发生真实跳转也会自动建立连接。 + +--- + +## 注意事项 + +1. 当设置 `viewport_metrics` 时,确保传入的宽高与实际渲染区域一致。 +2. 如果需要同时预渲染多个页面,请确保 `nextViewId` 不重复。 diff --git "a/ohos/docs/04_development/\345\246\202\344\275\225\345\257\274\345\205\245\350\207\252\346\270\262\346\237\223\345\206\205\345\256\271.md" "b/ohos/docs/04_development/\345\246\202\344\275\225\345\257\274\345\205\245\350\207\252\346\270\262\346\237\223\345\206\205\345\256\271.md" new file mode 100644 index 0000000000000000000000000000000000000000..3ac605468a0e3bc09fe559c81051981c2af61b5f --- /dev/null +++ "b/ohos/docs/04_development/\345\246\202\344\275\225\345\257\274\345\205\245\350\207\252\346\270\262\346\237\223\345\206\205\345\256\271.md" @@ -0,0 +1,145 @@ +# 如何导入自渲染内容到 Flutter + +## 背景介绍 + +我们支持将外部自渲染内容(无论是通过 `NativeDrawing` 等鸿蒙 CAPI 绘制,还是通过 OpenGL 相关接口绘制)通过外接纹理的方式导入 Flutter 的 `Texture` 组件进行展示。以下是具体实现方法。 + +--- + +## 实现步骤 + +### 1. Dart 侧操作 + +#### 1.1 创建Texture组件 + +- 创建Texture组件,绑定textureId + +#### 1.2 注册纹理 + +- 创建 `MethodChannel`: + + final MethodChannel _channel = const MethodChannel('PictureChannel'); + +- 在 `initState` 函数中注册纹理: + + _textureId = await _channel.invokeMethod("registerStarTexture"); + +#### 1.3 注销纹理 + +- 在 `dispose` 函数中注销纹理: + + _channel.invokeMethod('unregisterTexture', {'textureId': _textureId}); + +--- + +### 2. 鸿蒙侧操作 + +#### 2.1 引入工具类 + +需要引入以下工具类: + +import { TextureRegistry, SurfaceTextureEntry } from '@ohos/flutter_ohos/src/main/ets/view/TextureRegistry'; + +#### 2.2 提供的实现方式 + +我们提供了两种将自渲染内容导入 Flutter 纹理的实现方式。 + +--- + +## 实现方式一:通过 `getNativeWindowId` 绘制到 Buffer + +通过 `getNativeWindowId` 获取引擎层 `Texture` 对应的 `OHNativeWindow` 指针,然后将自渲染内容绘制到该 `window` 对应的 Buffer 上。 + +### ArkTS 侧实现步骤 + +1. 注册纹理时获取相关信息: + +```arkts + this.textureId = this.textureRegistry!.getTextureId(); + this.surfaceTextureEntry = this.textureRegistry!.registerTexture(this.textureId); + this.windowId = this.surfaceTextureEntry.getNativeWindowId(); + drawNapi.drawPatternWindow(this.windowId, 300, 300);// 需要在cpp层实现draw函数,接收window指针 +``` + +2. 调用 `cpp` 层函数,将自渲染内容绘制到该 `window`: + + drawNapi.drawPatternWindow(this.windowId, 300, 300); // 需要在 cpp 层实现此函数 + +### CPP 侧实现步骤 + +1. 使用 `OH_NativeWindow_NativeWindowHandleOpt` 设置 `window` 的 `USAGE`、宽高等信息。 +2. 调用 `OH_NativeWindow_NativeWindowRequestBuffer` 获取 `window` 的 Buffer。 +3. 使用 `OH_NativeWindow_GetBufferHandleFromNative` 获取 `bufferHandle`。 +4. 使用 `OH_NativeWindow_NativeWindowFlushBuffer` 将生产好的 Buffer 放到 Buffer 队列中供消费。 + +--- + +## 实现方式二:通过 `setExternalNativeImage` + +直接将用于内容生产的 `NativeImage` 指针传递给 Flutter 引擎层,由引擎层利用该 `NativeImage` 通过 `OH_NativeImage_SetOnFrameAvailableListener` 注册帧可用回调进行内容消费。 + +### ArkTS 侧实现步骤 + +1. 注册纹理时获取相关信息: + +```arkts + this.textureId = this.textureRegistry!.getTextureId(); + this.surfaceTextureEntry = this.textureRegistry!.registerTexture(this.textureId); + drawNapi.drawPatternWindow(1000, 1000); + this.nativeImageId = drawNapi.getNativeImage();// 获取用于内容生产的NativeImage + this.textureRegistry!.setExternalNativeImage(this.textureId, this.nativeImageId);// 将NativeImage传入引擎层 +``` + +### CPP 侧实现步骤 + +以下为基于 OpenGL 的简单绘制实现步骤: + +1. 使用 `OH_NativeImage_Create` 创建 `NativeImage`。 +2. 使用 `OH_NativeImage_AcquireNativeWindow` 获取 `NativeImage` 关联的 `window`。 +3. 使用 `OH_NativeWindow_NativeWindowHandleOpt` 设置 `USAGE` 和宽高。 +4. 将 `NativeWindow` 转换为 `EGLNativeWindowType` 类型。 +5. 调用以下 OpenGL 相关指令完成绘制: + - `eglCreateWindowSurface` + - `eglCreateContext` + - `eglMakeCurrent` + - `gl` 绘制相关指令 + - `eglSwapBuffers` + +--- + +## 纹理注销 + +无论使用上述哪种方式,都需要调用以下函数进行纹理注销: + +this.textureRegistry!.unregisterTexture(textureId); + +--- + +## 参考实现 + +### 实现方式一参考 + +[flutter_it_window](https://gitee.com/geng_fei/flutter_it/pulls/2) + +### 实现方式二参考 + +[flutter_it_image](https://gitee.com/geng_fei/flutter_it/tree/flutter_page-multi_engine-predraw) + +--- + +## 代码位置说明 + +1. **Dart 侧代码** + 相关代码位于 `lib/testpicture/PicturePage.dart`,最下方添加的纹理组件用于自渲染内容导入。 + +2. **ArkTS 侧代码** + 相关代码位于 `ohos/entry/src/main/ets/pictureplugin/PicturePlugin.ets`,包含了纹理的注册和注销函数。 + +3. **原生绘制代码** + 相关代码位于 `ohos/entry/src/main/cpp`。 + +--- + +## 注意事项 + +1. 开发者在使用方式二时,需要自行管理 `NativeImage` 的生命周期,并确保在 `unregister` 前对象不会销毁。 \ No newline at end of file