# flutter_opengl_texture2

**Repository Path**: scenario-samples/flutter_opengl_texture2

## Basic Information

- **Project Name**: flutter_opengl_texture2
- **Description**: 【鸿蒙 Harmony Next 示例 代码】通过AVCodec进行视频解码并通过NativeImage+opengl实现二次处理，最终渲染至Flutter页面上
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 1
- **Forks**: 0
- **Created**: 2025-06-13
- **Last Updated**: 2026-02-28

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Flutter通过NativeImage二次OpenGL处理并渲染

## 介绍：
 直播场景下，需要对视频进行解码，并通过opengl进行二次处理，最终渲染到屏幕上；并且此demo正对Flutter三方框架场景，通过外接纹理方式渲染到Flutter Texture上，不使用PlatformView包裹XComponent的方式。


## 实现思路
1. 使用AVCodec Kit对视频进行解码，输出到NativeImage关联的NativeWindow上
   ```
   nativeImage_ = OH_NativeImage_Create(-1, GL_TEXTURE_EXTERNAL_OES);
   nativeImageWindow_ = OH_NativeImage_AcquireNativeWindow(nativeImage_);
   ret = OH_NativeImage_GetSurfaceId(nativeImage_, &nativeImageSurfaceId_);
   int code = SET_BUFFER_GEOMETRY;
   int32_t width = 350;
   int32_t height = 550;
   int32_t ret = OH_NativeWindow_NativeWindowHandleOpt(nativeImageWindow_, code, width, height);
   OH_VideoDecoder_SetSurface(decoder_, nativeImageWindow_)
   ```
2. 使用NativeImage绑定opengl外接纹理，实现二次处理
   ```
   glGenTextures(1, &nativeImageTexId_);
   OH_NativeImage_AttachContext(nativeImage_, nativeImageTexId_);
   renderContext_->MakeCurrent(eglSurface_);
   ret = OH_NativeImage_GetTransformMatrix(nativeImage_, matrix);
   
   shaderProgram_->Use();
   shaderProgram_->SetInt("texture", 0);
   glActiveTexture(GL_TEXTURE0);
   glBindTexture(GL_TEXTURE_EXTERNAL_OES, nativeImageTexId_);
   shaderProgram_->SetMatrix4v("matTransform", matrix, 16, false);
   glBindVertexArray(vertexArrayObject_);
   glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_INT, Detail::indices);
   renderContext_->SwapBuffers(eglSurface_);
   ```
3. 应用启动时，进行Flutter侧与HarmonyOS侧的外接纹理绑定
   ```
   this.textureId = this.textureRegistry!.getTextureId();
   this.surfaceId = this.textureRegistry!.registerTexture(this.textureId)!.getSurfaceId();
   ```
   生成SurfaceId给到C++侧绑定eglSurface
   ```
   OHNativeWindow* nativeWindow;
   OH_NativeWindow_CreateNativeWindowFromSurfaceId(surfaceId, &nativeWindow);
   PluginRender::GetInstance()->renderThread_->UpdateNativeWindow(nativeWindow, 350, 550);
   renderContext_->CreateEglSurface(static_cast<EGLNativeWindowType>(nativeWindow_))
   ```


## 效果预览：

<img src="./gif/flutter_opengl_texture.gif" width="200">

## 工程目录：

```
flutter_opengl_texture
    |---ohos/entry/src/main
    |   |---cpp/
    |       |---capbilities/                       // opengl渲染相关
    |           |---audio_decoder.cpp              // 音频解码
    |           |---demuxer.cpp                    // 媒体解封装器
    |           |---video_decoder.h                // 视频解码
    |       |---common/                             
    |           |---sample_callback.cpp            // 采样回调函数  
    |       |---render/
    |           |---egl_render_context.cpp         // opengl相关信息与上下文  
    |           |---plugin_render.cpp              // 插件入库，业务逻辑
    |           |---render_thread.cpp              // 渲染线程，对解码数据进行二次处理并渲染
    |           |---shader_program.cpp             // 着色器
    |       |---sample/player/
    |           |---player.cpp                     // 解码播放器        
    |       |---types/libnativerender
    |           |---index.d.ts                     // C++侧映射接口导出                
    |   |---ets/
    |       |---entryability/                       
    |           |---TextureExamplePlugin.ets        // 注册Flutter插件和MethodChannel
    |       |---pages/
    |           |---Index.ets                       // 加载FlutterPage
    |---lib/    
        |---main.dart                               // Flutter首页 Texture组件
```

## 约束与限制：
- IDE: DevEco Studio 5.0.5 Release
- SDK: API Version 17 Release及以上版本

## 参考文档
- [音视频编解码](https://developer.huawei.com/consumer/cn/doc/harmonyos-guides/audio-video-codec)
- [NativeImage开发指导 (C/C++)](https://developer.huawei.com/consumer/cn/doc/harmonyos-guides/native-image-guidelines)
- [NativeVSync开发指导 (C/C++)](https://developer.huawei.com/consumer/cn/doc/harmonyos-guides/native-vsync-guidelines)
- [Flutter OHOS外接纹理适配](https://gitcode.com/openharmony-tpc/flutter_samples/blob/master/ohos/docs/04_development/Flutter%20OHOS%E5%A4%96%E6%8E%A5%E7%BA%B9%E7%90%86%E9%80%82%E9%85%8D%E7%AE%80%E4%BB%8B.md)


## 一份简单的问卷反馈

亲爱的Harmony Next开发者，您好！<br>
为了协助您高效开发，提高鸿蒙场景化示例的质量，希望您在浏览或使用后抽空填写一份简单的问卷，我们将会收集您的宝贵意见进行优化:heart:

[:arrow_right: **点击此处填写问卷** ](https://wj.qq.com/s2/19042938/95ab/)

<div style="text-align: left;">