feat(flame_3d): initial implementation of 3D support (#3012)

Co-authored-by: Lukas Klingsbo <[email protected]>

packages/flame_3d/.metadata

@@ -0,0 +1,10 @@
+# This file tracks properties of this Flutter project.
+# Used by Flutter tool to assess capabilities and perform upgrades etc.
+#
+# This file should be version controlled and should not be manually edited.
+
+version:
+  revision: "1b197762c51e993cb77d7fafe9729ef2506e2bf7"
+  channel: "beta"
+
+project_type: package

packages/flame_3d/CHANGELOG.md

@@ -0,0 +1,3 @@
+## 0.1.0-dev.1
+
+- Initial experimental release of `flame_3d`.

packages/flame_3d/CONTRIBUTING.md

@@ -0,0 +1,48 @@
+# Contribution Guidelines
+
+Read the main [Flame Contribution Guidelines](https://github.com/flame-engine/flame/blob/main/CONTRIBUTING.md)
+first and then come back to this one.
+
+## How To Contribute
+
+
+### Environment Setup
+
+First follow the steps described in the main [Flame Contribution Guidelines](https://github.com/flame-engine/flame/blob/main/CONTRIBUTING.md#environment-setup)
+
+After you have followed those steps you have to setup Flutter to use the specific build that this
+package is built against:
+
+```sh
+cd $(dirname $(which flutter)) && git checkout 8a5509ea6a277d48c15e5965163b08bd4ad4816a -q && echo "Engine commit: $(cat internal/engine.version)" && cd - >/dev/null
+```
+
+This will check out the GIT repo of your Flutter installation to the specific commit that we require
+and also gets us t he the commit SHA of the Flutter Engine that you need to use in setting up the 
+Flutter GPU. For that you can follow the steps described in the 
+[Flutter Wiki](https://github.com/flutter/flutter/wiki/Flutter-GPU#try-out-flutter-gpu). 
+
+Once you have cloned the Flutter engine you can add the `flutter_gpu` as an override dependency 
+to the `pubspec_overrides.yaml` file in the `flame_3d` directory and it's example:
+
+```yaml
+dependency_overrides:
+  ... # Melos related overrides
+  flutter_gpu:
+    path: <path_to_the_cloned_flutter_engine_directory>/lib/gpu
+```
+
+After all of that you should run `flutter pub get` one more time to ensure all dependencies are 
+set up correctly.
+
+
+### Shader changes
+
+If you have added/changed/removed any of the shaders in the `shaders` directory make sure to run the
+build script for shaders:
+
+```sh
+dart bin/build_shaders.dart
+```
+
+This is currently a manual process until Flutter provides bundling support.

packages/flame_3d/LICENSE

@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) 2024 Blue Fire
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+

packages/flame_3d/README.md

@@ -0,0 +1,139 @@
+<!-- markdownlint-disable MD013 -->
+<p align="center">
+  <a href="https://flame-engine.org">
+    <img alt="flame" width="200px" src="https://user-images.githubusercontent.com/6718144/101553774-3bc7b000-39ad-11eb-8a6a-de2daa31bd64.png">
+  </a>
+</p>
+
+<p align="center">
+Adds 3D support for <a href="https://github.com/flame-engine/flame">Flame</a> using the <a href="https://github.com/flutter/flutter/wiki/Flutter-GPU">Flutter GPU</a>.
+</p>
+
+<p align="center">
+  <a title="Pub" href="https://pub.dev/packages/flame_3d" ><img src="https://img.shields.io/pub/v/flame_3d.svg?style=popout" /></a>
+  <a title="Test" href="https://github.com/flame-engine/flame/actions?query=workflow%3Acicd+branch%3Amain"><img src="https://github.com/flame-engine/flame/workflows/cicd/badge.svg?branch=main&event=push"/></a>
+  <a title="Discord" href="https://discord.gg/pxrBmy4"><img src="https://img.shields.io/discord/509714518008528896.svg"/></a>
+  <a title="Melos" href="https://github.com/invertase/melos"><img src="https://img.shields.io/badge/maintained%20with-melos-f700ff.svg"/></a>
+</p>
+
+---
+<!-- markdownlint-enable MD013 -->
+
+<!-- markdownlint-disable-next-line MD002 -->
+# flame_3d
+
+This package provides an experimental implementation of 3D support for Flame. The main focus is to 
+explore the potential capabilities of 3D for Flame while providing a familiar API to existing Flame
+developers.
+
+Supported platforms:
+
+| Platform | Supported |
+| -------- | --------- |
+| Android  | ❌        |
+| iOS      | ❌        |
+| macOS    | ✅        |
+| Windows  | ❌        |
+| Linux    | ❌        |
+| Web      | ❌        |
+
+## Prologue
+
+**STOP**, we know you are hyped up and want to start coding some funky 3D stuff but we first have to
+set your expectations and clarify some things. So turn down your music, put away the coffee and make
+some tea instead because you have to do some reading first!
+
+This package provides 3D support for Flame but it depends on the still experimental 
+[Flutter GPU](https://github.com/flutter/flutter/wiki/Flutter-GPU) which in turn depends on 
+Impeller. The Flutter GPU is currently not shipped with Flutter so this package wont work without 
+following the prerequisites steps.
+
+Because we depend on Flutter GPU this package is also highly experimental. Our long term goal is to
+eventually deprecate this package and integrate it into the core `flame` package, for more 
+information on this see the [Roadmap](https://github.com/flame-engine/flame/blob/main/packages/flame_3d/ROADMAP.md).
+
+This package does not guarantee that it will follow correct [semver](https://semver.org/) versioning
+rules nor does it assure that it's APIs wont break. Be ready to constantly have to refactor your 
+code if you are planning on using this package in a semi-production environment, which we do not
+recommend. 
+
+Documentation and tests might be lacking for quite a while because of the potential constant changes
+of the API. Where possible we will try to provide in-code documentation and code examples to help
+developers but our main goal for now is to enable the usage of 3D rendering within a Flame 
+ecosystem.
+
+
+## Prerequisites
+
+Before you can get started with using this package a few steps have to happen first. Step one is 
+switching to a specific commit on the Flutter tooling. Because this package is still experimental 
+some of the features it requires are still being worked on from the Flutter side.
+
+So to make sure you are using the same build that we use while developing you have to manually 
+checkout a specific Flutter build. Thankfully we were able to simplify that process into a 
+one-liner:
+
+```sh
+cd $(dirname $(which flutter)) && && git fetch && git checkout bcdd1b2c481bca0647beff690238efaae68ca5ac -q && echo "Engine commit: $(cat internal/engine.version)" && cd - >/dev/null
+```
+
+This will check out the GIT repo of your Flutter installation to the specific commit that we require
+and also return the commit SHA of the Flutter Engine that it was build with. We need for step two.
+
+Step two is setting up the Flutter GPU. You can follow the steps described in the [Flutter Wiki](https://github.com/flutter/flutter/wiki/Flutter-GPU#try-out-flutter-gpu). 
+The engine commit that you should use is the one we got in step one.
+
+Once you have cloned the Flutter engine you can add the `flutter_gpu` as an override dependency 
+to your `pubspec.yaml` or in a `pubspec_overrides.yaml` file:
+
+```yaml
+dependency_overrides:
+  flutter_gpu:
+    path: <path_to_the_cloned_flutter_engine_directory>/lib/gpu
+```
+
+Step three would be to enable impeller for the macOS platform, add the following to the 
+`Info.plist` in your `macos/` directory:
+
+```xml
+<dict>
+    ...
+	<key>FLTEnableImpeller</key>
+	<true/>
+</dict>
+```
+
+Now everything is set up you can start doing some 3D magic! You can check out the
+[example](https://github.com/flame-engine/flame/tree/main/packages/flame_3d/example) to see how you
+can set up a simple 3D environment using Flame.
+
+
+## Building shaders
+
+You can write your own shaders and use them on Materials. Currently Flutter does not do the bundling
+of shaders for us so this package provides a simple dart script. Create your fragment and vertex 
+shader in a `shaders` directory, make sure the file names are identical. Like so:
+
+- `my_custom_shader`.frag
+- `my_custom_shader`.vert
+
+You can then run `dart pub run flame_3d:build_shaders` to bundle the shaders. They will 
+automatically be placed in `assets/shaders`. 
+
+You can check out the
+[default shaders](https://github.com/flame-engine/flame/tree/main/packages/flame_3d/shaders) if you
+want to have some examples.
+
+
+## Contributing
+
+Have you found a bug or have a suggestion of how to enhance the 3D APIs? Open an issue and we will
+take a look at it as soon as possible.
+
+Do you want to contribute with a PR? PRs are always welcome, just make sure to create it from the
+correct branch (main) and follow the [checklist](.github/pull_request_template.md) which will
+appear when you open the PR.
+
+For bigger changes, or if in doubt, make sure to talk about your contribution to the team. Either
+via an issue, GitHub discussion, or reach out to the team using the 
+[Discord server](https://discord.gg/pxrBmy4).

packages/flame_3d/ROADMAP.md

@@ -0,0 +1,50 @@
+# Roadmap
+
+In the interest of transparency, we provide a high-level detail of the roadmap for adding 3D 
+support to Flame. We hope this roadmap will help others in making plans and priorities based on the
+work we are doing and potentially contribute back to the project itself.
+
+The goal of the package can be split up into two sections, the primary goal is to provide an API for
+Flame developers so they can create 3D environments without having to learn new Flame concepts. This
+means the package will tie into the existing [FCS](https://docs.flame-engine.org/latest/flame/components.html#component)
+and provide the tools needed, like a [`CameraComponent`](https://docs.flame-engine.org/latest/flame/camera_component.html), 
+`World` and similar components. 
+
+In a perfect world this API does not depend or even know about the Flutter GPU, which brings us 
+to our secondary goal: to abstract the Flutter GPU into an API that is user-friendly for 3D 
+development. That includes simplifying things like creating render targets, setting up the color 
+and depth textures and configuring depth stencils. But it also includes higher level APIs like 
+geometric shapes, texture/material rendering and creating Meshes that can use those shapes and 
+materials.
+
+## Goals
+
+### Abstracting the Flutter GPU into a user-friendly API for 3D
+
+- [x] Abstract the GPU setup into a class that represents the graphics device
+  - [ ] Setup binding logic for meshes, geometry and materials.
+- [ ] Provide a `Mesh` API
+  - [x] Provide `Surface`s that can hold geometric shapes.
+  - [x] Provide a `Material` API
+    - [x] Define a `Texture` API to be used with the `Material` API
+      - [x] Support images as textures
+      - [x] Support single color textures
+      - [x] Support generated textures
+    - [x] Provide a standard `Material`
+  - [ ] Support custom shaders
+    - [ ] Add a more dev friendly way to set uniforms
+  - [x] Support multiple `Material`s by defining surfaces on a mesh.
+
+
+### Providing a familiar API for developers to use 3D with Flame
+
+- [x] Use the existing `CameraComponent` API for 3D rendering
+  - [x] Provide a custom `World`
+  - [x] Support existing and custom viewports
+  - [ ] Support existing and custom viewfinders
+- [x] Create a new core component for 3D rendering (`Component3D`)
+  - [x] Implement a `Transform3D` for 3D transformations
+    - [x] Implement a notifying `Vector3` and `Quaternion` for 3D positioning and rotation
+  - [ ] Add support for gesture event callbacks
+- [x] Create a component that can show meshes (`MeshComponent`)
+  - [x] Ensure materials can be set outside of construction (in the `onLoad` for instance)

packages/flame_3d/analysis_options.yaml

@@ -0,0 +1 @@
+include: package:flame_lint/analysis_options_with_dcm.yaml

packages/flame_3d/bin/build_shaders.dart

@@ -0,0 +1,62 @@
+import 'dart:convert';
+import 'dart:io';
+
+/// Bundle a shader (<name>.frag & <name>.vert) into a single shader bundle and
+/// store it in the assets directory.
+///
+/// This script is just a temporary way to bundle shaders. In the long run
+/// Flutter might support auto-bundling themselves but until then we have to
+/// do it manually.
+///
+/// Note: this script should be run from the root of the package:
+/// packages/flame_3d
+void main() async {
+  final root = Directory.current;
+
+  final assets = Directory.fromUri(root.uri.resolve('assets/shaders'));
+  // Delete all the bundled shaders so we can replace them with new ones.
+  if (assets.existsSync()) {
+    assets.deleteSync(recursive: true);
+  }
+  // Create if not exists.
+  assets.createSync(recursive: true);
+
+  // Directory where our unbundled shaders are stored.
+  final shaders = Directory.fromUri(root.uri.resolve('shaders'));
+  if (!shaders.existsSync()) {
+    return stderr.writeln('Missing shader directory');
+  }
+
+  // Get a list of unique shader names. Each shader should have a .frag and
+  // .vert with the same basename to be considered a bundle.
+  final uniqueShaders = shaders
+      .listSync()
+      .whereType<File>()
+      .map((f) => f.path.split('/').last.split('.').first)
+      .toSet();
+
+  for (final name in uniqueShaders) {
+    final bundle = {
+      'TextureFragment': {
+        'type': 'fragment',
+        'file': '${root.path}/shaders/$name.frag',
+      },
+      'TextureVertex': {
+        'type': 'vertex',
+        'file': '${root.path}/shaders/$name.vert',
+      },
+    };
+
+    final result = await Process.run(impellerC, [
+      '--sl=${assets.path}/$name.shaderbundle',
+      '--shader-bundle=${jsonEncode(bundle)}',
+    ]);
+
+    if (result.exitCode != 0) {
+      return stderr.writeln(result.stderr);
+    }
+  }
+}
+
+final impellerC =
+    '${Platform.environment['FLUTTER_HOME']}/bin/cache/artifacts/engine/darwin-x64/impellerc';

packages/flame_3d/example/.metadata

@@ -0,0 +1,45 @@
+# This file tracks properties of this Flutter project.
+# Used by Flutter tool to assess capabilities and perform upgrades etc.
+#
+# This file should be version controlled and should not be manually edited.
+
+version:
+  revision: "1b197762c51e993cb77d7fafe9729ef2506e2bf7"
+  channel: "beta"
+
+project_type: app
+
+# Tracks metadata for the flutter migrate command
+migration:
+  platforms:
+    - platform: root
+      create_revision: 1b197762c51e993cb77d7fafe9729ef2506e2bf7
+      base_revision: 1b197762c51e993cb77d7fafe9729ef2506e2bf7
+    - platform: android
+      create_revision: 1b197762c51e993cb77d7fafe9729ef2506e2bf7
+      base_revision: 1b197762c51e993cb77d7fafe9729ef2506e2bf7
+    - platform: ios
+      create_revision: 1b197762c51e993cb77d7fafe9729ef2506e2bf7
+      base_revision: 1b197762c51e993cb77d7fafe9729ef2506e2bf7
+    - platform: linux
+      create_revision: 1b197762c51e993cb77d7fafe9729ef2506e2bf7
+      base_revision: 1b197762c51e993cb77d7fafe9729ef2506e2bf7
+    - platform: macos
+      create_revision: 1b197762c51e993cb77d7fafe9729ef2506e2bf7
+      base_revision: 1b197762c51e993cb77d7fafe9729ef2506e2bf7
+    - platform: web
+      create_revision: 1b197762c51e993cb77d7fafe9729ef2506e2bf7
+      base_revision: 1b197762c51e993cb77d7fafe9729ef2506e2bf7
+    - platform: windows
+      create_revision: 1b197762c51e993cb77d7fafe9729ef2506e2bf7
+      base_revision: 1b197762c51e993cb77d7fafe9729ef2506e2bf7
+
+  # User provided section
+
+  # List of Local paths (relative to this file) that should be
+  # ignored by the migrate tool.
+  #
+  # Files that are not part of the templates will be ignored by default.
+  unmanaged_files:
+    - 'lib/main.dart'
+    - 'ios/Runner.xcodeproj/project.pbxproj'

packages/flame_3d/example/README.md

@@ -0,0 +1,3 @@
+# flame_3d example
+
+An example for using the `flame_3d` package.

packages/flame_3d/example/analysis_options.yaml

@@ -0,0 +1 @@
+include: package:flame_lint/analysis_options_with_dcm.yaml