From c7428103394f6a7eb639e8bed440af6a4112e52a Mon Sep 17 00:00:00 2001 From: MTChannn Date: Fri, 14 Feb 2025 14:53:51 +0800 Subject: [PATCH] =?UTF-8?q?=E6=B7=BB=E5=8A=A0Flutter=E5=A4=96=E6=8E=A5?= =?UTF-8?q?=E7=BA=B9=E7=90=86=E7=AC=AC=E4=B8=80=E5=B8=A7=E8=83=8C=E6=99=AF?= =?UTF-8?q?=E8=89=B2=E8=87=AA=E5=AE=9A=E4=B9=89=E6=96=87=E6=A1=A3=E8=AF=B4?= =?UTF-8?q?=E6=98=8E?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: MTChannn --- ...11\350\207\252\345\256\232\344\271\211.md" | 146 ++++++++++++++++++ ...-(background)-customization-for-flutter.md | 146 ++++++++++++++++++ 2 files changed, 292 insertions(+) create mode 100644 "ohos/docs/04_development/Flutter OHOS\345\244\226\346\216\245\347\272\271\347\220\206\347\254\254\344\270\200\345\270\247\357\274\210\350\203\214\346\231\257\357\274\211\350\207\252\345\256\232\344\271\211.md" create mode 100644 ohos/docs/04_development/external-texture-first-frame-(background)-customization-for-flutter.md diff --git "a/ohos/docs/04_development/Flutter OHOS\345\244\226\346\216\245\347\272\271\347\220\206\347\254\254\344\270\200\345\270\247\357\274\210\350\203\214\346\231\257\357\274\211\350\207\252\345\256\232\344\271\211.md" "b/ohos/docs/04_development/Flutter OHOS\345\244\226\346\216\245\347\272\271\347\220\206\347\254\254\344\270\200\345\270\247\357\274\210\350\203\214\346\231\257\357\274\211\350\207\252\345\256\232\344\271\211.md" new file mode 100644 index 00000000..d1c1edc6 --- /dev/null +++ "b/ohos/docs/04_development/Flutter OHOS\345\244\226\346\216\245\347\272\271\347\220\206\347\254\254\344\270\200\345\270\247\357\274\210\350\203\214\346\231\257\357\274\211\350\207\252\345\256\232\344\271\211.md" @@ -0,0 +1,146 @@ + +# Flutter OHOS外接纹理第一帧（背景）自定义 + +## 概述 +在这个 Flutter 插件中，`flutterRenderer.setTextureBackGroundColor` 和 `flutterRenderer.setTextureBackGroundPixelMap` 方法用于设置视频播放器纹理的背景颜色和纹理内容。背景颜色和纹理图像可以在视频加载或播放之前设置，确保用户在等待视频渲染时看到合适的背景。 + +## `flutterRenderer` 类型 +`flutterRenderer` 是 `TextureRegistry` 类型的实例，它是通过 `FlutterState` 类获取的。`FlutterState` 类包含两个重要成员： +- **`binaryMessenger`** (`BinaryMessenger`): 用于与 Flutter 引擎之间进行消息传递。 +- **`textureRegistry`** (`TextureRegistry`): 用于管理纹理和注册新纹理。 + +### `FlutterState` 类 +```typescript +export class FlutterState { + private binaryMessenger: BinaryMessenger; + private textureRegistry: TextureRegistry; + + constructor(binaryMessenger: BinaryMessenger, textureRegistry: TextureRegistry) { + this.binaryMessenger = binaryMessenger; + this.textureRegistry = textureRegistry; + } + + getBinaryMessenger(): BinaryMessenger { + return this.binaryMessenger; + } + + getTextureRegistry(): TextureRegistry { + return this.textureRegistry; + } +} +``` + +通过 `FlutterState.getTextureRegistry()` 方法，你可以访问 `flutterRenderer`，它是 `TextureRegistry` 的一个实例，并且可以使用它来管理和注册视频播放器的纹理。 + +## 方法签名 + +### `flutterRenderer.setTextureBackGroundColor` +```dart +flutterRenderer.setTextureBackGroundColor(int textureId, int color) +``` + +- **`textureId`** (`int`): 纹理的 ID，用于标识要设置背景颜色的纹理。 +- **`color`** (`int`): 背景颜色的值，采用 32 位十六进制格式 `0xAABBGGRR`。 + +### `flutterRenderer.setTextureBackGroundPixelMap` +```dart +flutterRenderer.setTextureBackGroundPixelMap(int textureId, image.PixelMap pixelMap) +``` + +- **`textureId`** (`int`): 纹理的 ID，用于标识要设置图像纹理的纹理。 +- **`pixelMap`** (`image.PixelMap`): 包含图像数据的 `PixelMap` 对象，用作纹理的背景图像。 + +## 主要功能 +1. **设置背景颜色** + `setTextureBackGroundColor` 方法用于设置纹理的背景颜色。在视频加载之前，它为纹理提供一个预设颜色，通常用于显示纯色背景。 + +2. **设置背景图像** + `setTextureBackGroundPixelMap` 方法则允许你将一帧视频图像或任何图像设置为纹理的背景，通常在视频文件加载时，用第一帧图像填充纹理。 + +## 示例代码 + +### 设置背景颜色为红色 +```dart +flutterRenderer.setTextureBackGroundColor(textureId, 0xFF0000FF); +``` + +在此示例中，背景颜色被设置为红色 (`0xFF0000FF`)，这意味着： +- `FF` 是 Alpha 值，表示完全不透明。 +- `00` 是绿色分量，表示没有绿色。 +- `00` 是蓝色分量，表示没有蓝色。 +- `FF` 是红色分量，表示最大红色。 + +### 设置背景图像为视频的第一帧 +```dart +let avImageGenerator = await media.createAVImageGenerator(); +let pixelMap = await avImageGenerator.fetchFrameByTime(0, media.AVImageQueryOptions.AV_IMAGE_QUERY_NEXT_SYNC, { + width: -1, + height: -1 +}); +flutterRenderer.setTextureBackGroundPixelMap(textureId, pixelMap); +``` + +这段代码中： +1. 使用 `AVImageGenerator` 获取视频的第一帧图像。 +2. 使用 `flutterRenderer.setTextureBackGroundPixelMap` 方法将这帧图像设置为纹理的背景图像。 + +### 设置纹理的背景颜色和背景图像 +在创建视频播放器时，可以同时设置纹理的背景颜色和背景图像，确保视频加载之前有一个合适的显示效果。 + +```dart +flutterRenderer.setTextureBackGroundColor(textureId, 0xFF0000FF); // 红色背景 +flutterRenderer.setTextureBackGroundPixelMap(textureId, pixelMap); // 第一帧图像 +``` + +## 常见用例 + +1. **视频加载前的占位背景** + 在视频播放器初始化时，使用背景颜色或背景图像填充纹理，以确保用户在视频加载期间不会看到空白界面。 + +2. **显示加载状态** + 在等待视频加载和渲染时，可以使用预设的背景颜色或图像表示加载状态，提升用户体验。 + +3. **自定义视频背景** + 可以通过背景图像或颜色定制播放器的外观，增强视觉效果。 + +## 代码使用示例 + +在三方库[video_player_ohos](https://gitee.com/openharmony-sig/flutter_packages/tree/master/packages/video_player/video_player_ohos)中的[`VideoPlayerApiImpl.ets`](https://gitee.com/openharmony-sig/flutter_packages/blob/master/packages/video_player/video_player_ohos/ohos/src/main/ets/components/videoplayer/VideoPlayerApiImpl.ets)中，`create` 方法中创建了 `AVImageGenerator` 和 `AVMetadataExtractor` 来获取视频的元数据和图像数据。如果视频资源来自资产或本地文件，则第一帧会作为背景设置。 + +```dart +if (asset != null) { + let avMetaExtractor = await media.createAVMetadataExtractor(); + avMetaExtractor.fdSrc = await this.getContext().resourceManager.getRawFd("flutter_assets/" + asset); + let mateData = await avMetaExtractor.fetchMetadata(); + if (mateData.hasVideo == CommonConstants.YES) { + let avImageGenerator = await media.createAVImageGenerator(); + avImageGenerator.fdSrc = await this.getContext().resourceManager.getRawFd("flutter_assets/" + asset); + let pixelMap = await avImageGenerator.fetchFrameByTime(0, media.AVImageQueryOptions.AV_IMAGE_QUERY_NEXT_SYNC, { + width: -1, + height: -1 + }); + flutterRenderer.setTextureBackGroundPixelMap(textureId, pixelMap); + avImageGenerator.release(); + } + avMetaExtractor.release(); +} +``` + +在这里，`flutterRenderer.setTextureBackGroundPixelMap(textureId, pixelMap)` 会将从视频资源中提取的第一帧图像设置为纹理的背景。 + +## 注意事项 +- **纹理 ID**: 确保在调用这些方法之前已经为纹理分配了有效的 `textureId`。 +- **内存管理**: 使用 `setTextureBackGroundPixelMap` 设置图像后，确保适当管理内存，避免内存泄漏。 +- **视频资源**: 确保资源路径（如资产路径或本地路径）正确，确保能够成功加载视频文件并提取图像。 + +## 总结 +`flutterRenderer.setTextureBackGroundColor` 和 `flutterRenderer.setTextureBackGroundPixelMap` 方法是 Flutter 视频播放器插件中非常实用的功能，它们允许开发者在视频加载期间自定义纹理的显示效果。通过设置背景颜色或图像，开发者能够提升用户体验，确保在视频内容渲染之前显示合适的界面。 + +## 功能需求背景和困难 +目前 Flutter 外接纹理默认第一帧固定为白色背景，但对于视频等内容，默认白色背景不符合需求。具体来说，有些业务需要白色背景显示，但其他业务可能需要黑色或者其它颜色的背景显示。因此，固定为白色背景的实现无法满足所有业务场景，尤其是视频类业务。 + +## 预期 +为了支持更多场景，应允许通过 Texture 容器的背景色控制或者提供一个设置背景色的接口。这样，开发者可以根据需求动态设置纹理的背景颜色。 + +## iOS 和安卓对标方案 +目前，iOS 和安卓默认第一帧的背景色为黑色。不过，考虑到先前的更改是将第一帧的背景色从黑色改为白色，提供一个用户可自定义背景颜色的能力显得更加合理。这样不仅保留了对旧版的兼容性，也为业务需求提供了更多的灵活性。 diff --git a/ohos/docs/04_development/external-texture-first-frame-(background)-customization-for-flutter.md b/ohos/docs/04_development/external-texture-first-frame-(background)-customization-for-flutter.md new file mode 100644 index 00000000..6290c27b --- /dev/null +++ b/ohos/docs/04_development/external-texture-first-frame-(background)-customization-for-flutter.md @@ -0,0 +1,146 @@ + +# External Texture First Frame (Background) Customization for Flutter + +## Overview +In this Flutter plugin, the `flutterRenderer.setTextureBackGroundColor` and `flutterRenderer.setTextureBackGroundPixelMap` methods are used to set the background color and content of the video player texture. The background color and texture image can be set before the video is loaded or played, ensuring that the user sees an appropriate background while waiting for the video to render. + +## `flutterRenderer` Type +`flutterRenderer` is an instance of `TextureRegistry`, which is obtained through the `FlutterState` class. The `FlutterState` class contains two important members: +- **`binaryMessenger`** (`BinaryMessenger`): Used for message passing between Flutter and the engine. +- **`textureRegistry`** (`TextureRegistry`): Used to manage textures and register new textures. + +### `FlutterState` Class +```typescript +export class FlutterState { + private binaryMessenger: BinaryMessenger; + private textureRegistry: TextureRegistry; + + constructor(binaryMessenger: BinaryMessenger, textureRegistry: TextureRegistry) { + this.binaryMessenger = binaryMessenger; + this.textureRegistry = textureRegistry; + } + + getBinaryMessenger(): BinaryMessenger { + return this.binaryMessenger; + } + + getTextureRegistry(): TextureRegistry { + return this.textureRegistry; + } +} +``` + +Through the `FlutterState.getTextureRegistry()` method, you can access `flutterRenderer`, which is an instance of `TextureRegistry`, and it can be used to manage and register video player textures. + +## Method Signatures + +### `flutterRenderer.setTextureBackGroundColor` +```dart +flutterRenderer.setTextureBackGroundColor(int textureId, int color) +``` + +- **`textureId`** (`int`): The ID of the texture used to identify which texture to set the background color for. +- **`color`** (`int`): The background color value, using the 32-bit hexadecimal format `0xAABBGGRR`. + +### `flutterRenderer.setTextureBackGroundPixelMap` +```dart +flutterRenderer.setTextureBackGroundPixelMap(int textureId, image.PixelMap pixelMap) +``` + +- **`textureId`** (`int`): The ID of the texture used to identify which texture to set the image for. +- **`pixelMap`** (`image.PixelMap`): A `PixelMap` object containing image data to be used as the background image for the texture. + +## Key Features +1. **Set Background Color** + The `setTextureBackGroundColor` method is used to set the background color of the texture. Before the video is loaded, it provides a preset color for the texture, typically used for displaying a solid color background. + +2. **Set Background Image** + The `setTextureBackGroundPixelMap` method allows you to set a video frame image or any image as the background of the texture, typically filling the texture with the first frame image when the video file is loaded. + +## Example Code + +### Set Background Color to Red +```dart +flutterRenderer.setTextureBackGroundColor(textureId, 0xFF0000FF); +``` + +In this example, the background color is set to red (`0xFF0000FF`), meaning: +- `FF` is the Alpha value, indicating fully opaque. +- `00` is the green component, indicating no green. +- `00` is the blue component, indicating no blue. +- `FF` is the red component, indicating maximum red. + +### Set Background Image to the First Frame of the Video +```dart +let avImageGenerator = await media.createAVImageGenerator(); +let pixelMap = await avImageGenerator.fetchFrameByTime(0, media.AVImageQueryOptions.AV_IMAGE_QUERY_NEXT_SYNC, { + width: -1, + height: -1 +}); +flutterRenderer.setTextureBackGroundPixelMap(textureId, pixelMap); +``` + +In this code: +1. Use `AVImageGenerator` to get the first frame image of the video. +2. Use `flutterRenderer.setTextureBackGroundPixelMap` to set this frame image as the texture background. + +### Set Both Background Color and Image for the Texture +When creating the video player, you can set both the background color and the background image for the texture to ensure an appropriate display before the video loads. + +```dart +flutterRenderer.setTextureBackGroundColor(textureId, 0xFF0000FF); // Red background +flutterRenderer.setTextureBackGroundPixelMap(textureId, pixelMap); // First frame image +``` + +## Common Use Cases + +1. **Placeholder Background Before Video Loads** + Use the background color or image to fill the texture while the video player initializes, ensuring that users do not see a blank screen during video loading. + +2. **Displaying Loading Status** + While waiting for the video to load and render, you can use a preset background color or image to indicate the loading state, enhancing the user experience. + +3. **Custom Video Background** + You can customize the appearance of the player by using background images or colors, enhancing the visual effect. + +## Code Usage Example + +In the third-party library [video_player_ohos](https://gitee.com/openharmony-sig/flutter_packages/tree/master/packages/video_player/video_player_ohos), in the [`VideoPlayerApiImpl.ets`](https://gitee.com/openharmony-sig/flutter_packages/blob/master/packages/video_player/video_player_ohos/ohos/src/main/ets/components/videoplayer/VideoPlayerApiImpl.ets) class, the `create` method uses `AVImageGenerator` and `AVMetadataExtractor` to fetch video metadata and image data. If the video resource is from assets or a ... + +```dart +if (asset != null) { + let avMetaExtractor = await media.createAVMetadataExtractor(); + avMetaExtractor.fdSrc = await this.getContext().resourceManager.getRawFd("flutter_assets/" + asset); + let mateData = await avMetaExtractor.fetchMetadata(); + if (mateData.hasVideo == CommonConstants.YES) { + let avImageGenerator = await media.createAVImageGenerator(); + avImageGenerator.fdSrc = await this.getContext().resourceManager.getRawFd("flutter_assets/" + asset); + let pixelMap = await avImageGenerator.fetchFrameByTime(0, media.AVImageQueryOptions.AV_IMAGE_QUERY_NEXT_SYNC, { + width: -1, + height: -1 + }); + flutterRenderer.setTextureBackGroundPixelMap(textureId, pixelMap); + avImageGenerator.release(); + } + avMetaExtractor.release(); +} +``` + +Here, `flutterRenderer.setTextureBackGroundPixelMap(textureId, pixelMap)` sets the first frame image extracted from the video resource as the texture background. + +## Notes +- **Texture ID**: Ensure that the texture has a valid `textureId` before calling these methods. +- **Memory Management**: After setting the image with `setTextureBackGroundPixelMap`, ensure proper memory management to avoid memory leaks. +- **Video Resources**: Ensure that the resource paths (e.g., asset or local path) are correct and the video file can be successfully loaded and extracted. + +## Summary +The `flutterRenderer.setTextureBackGroundColor` and `flutterRenderer.setTextureBackGroundPixelMap` methods are very useful in the Flutter video player plugin. They allow developers to customize the display effect of the texture during video loading, ensuring a proper interface before the video content is rendered. + +## Functional Requirements Background and Challenges +Currently, Flutter external textures default to a white background for the first frame. However, for video and other content, the default white background does not meet the needs. Specifically, some business scenarios require a white background, but others may need black or different colored backgrounds. Therefore, the fixed white background implementation cannot meet all business scenarios, especially for video-based applications. + +## Expected Outcome +To support more scenarios, it is expected to control the background color of the texture container or provide an interface to set the background color. This allows developers to dynamically set the texture's background color based on their needs. + +## iOS and Android Benchmarking +Currently, iOS and Android default to a black background for the first frame. However, considering the previous change from black to white, it is more reasonable to provide users with the ability to customize the background color. This not only preserves compatibility with older versions but also offers more flexibility for business requirements. -- Gitee