Move vulkan codegen generators #1851

MarkY-LunarG · 2024-10-30T18:24:59Z

These changes moves the current Vulkan and base generators into a better layout that can share code in a more understandable way.

Instead of:

   base_generators/
     base* files
   dx12_generators/
     dx12*  files
   vulkan_generators/
     vulkan* files

After this change:

   dx12_generators/
     dx12*  files
   khronos_generators/
      khronos_base_* files     <-  base* files
      vulkan_generators/
           vulkan* files

Also removed some duplication in the files and created utility functions to remove some additional Vulkan-specific items in the lower levels.

This is a big step towards building the appropriate structure for sharing files with OpenXR.

ci-tester-lunarg · 2024-10-30T18:25:02Z

CI gfxreconstruct build queued with queue ID 291417.

ci-tester-lunarg · 2024-10-30T18:30:00Z

CI gfxreconstruct build # 5243 running.

ci-tester-lunarg · 2024-10-30T19:04:09Z

CI gfxreconstruct build # 5243 passed.

jzulauf-lunarg

These look good. Probably can squash the "Rename Base..."

framework/generated/generated_vulkan_struct_handle_mappers.cpp

framework/generated/khronos_generators/khronos_base_generator.py

framework/generated/khronos_generators/base_generators/base_decoder_body_generator.py

framework/generated/khronos_generators/khronos_base_generator.py

...rk/generated/khronos_generators/base_generators/base_struct_handle_mappers_body_generator.py

ci-tester-lunarg · 2024-11-11T18:22:38Z

CI gfxreconstruct build queued with queue ID 300213.

MarkY-LunarG · 2024-11-11T18:23:04Z

@locke-lunarg or @davidd-lunarg , can one of you review this because it's codegen?

ci-tester-lunarg · 2024-11-11T18:25:01Z

CI gfxreconstruct build # 5335 running.

ci-tester-lunarg · 2024-11-11T19:01:30Z

CI gfxreconstruct build # 5335 passed.

ci-tester-lunarg · 2024-11-11T19:02:53Z

CI gfxreconstruct build queued with queue ID 300271.

ci-tester-lunarg · 2024-11-11T19:05:01Z

CI gfxreconstruct build # 5336 running.

ci-tester-lunarg · 2024-11-11T19:42:15Z

CI gfxreconstruct build # 5336 passed.

Move the Vulkan generator scripts under a khronos_generators folder to prepare it for for sharing with other khronos APIs.

DX12 was still relying on one file in the base_generators folder. Make a copy in the DX12 folder, and remove all references to the Vulkan and base_generators folders in the generate_dx12.py script.

Move the reformat_code.py into the base of khronos_generators.

Move common items from the vulkan_generators/base_generator.py to the khronos_generators/khronos_base_generator.py

Move more common type parsing into the khronos_base_generator from the vulkan/base_generator.

base_generator code still has vulkan-specific items in it. This starts the process of pulling those out.

Remove more vulkan-specific items in the base_generator code path.

Modify the handling of 'pNext' extended struct variables in the base_generators to be more Khronos-generic instead of Vulkan-specific.

First phase moving of base_generator files to have a khronos identifier

Since the khronos_base files were referring back to the parent "base_generator" (which in this case was Vulkan) it still resulted in a tight coupling. Move the "write" method into the khronos_base_generator script to solve this.

Consolidate the structure type name generation into one location shared with all future Khronos APIs.

Put variable naming in a common location

Removed local variables for use in decoder body scripts and created a "all_cmd_params" that isn't cleared after every feature update.

Added raising an exception on Python methods that should be implemented by the Khronos API for the functionality to properly work.

ci-tester-lunarg · 2024-11-21T17:37:34Z

CI gfxreconstruct build queued with queue ID 307679.

ci-tester-lunarg · 2024-11-21T17:40:01Z

CI gfxreconstruct build # 5366 running.

ci-tester-lunarg · 2024-11-21T18:14:25Z

CI gfxreconstruct build # 5366 passed.

locke-lunarg · 2024-11-25T18:42:58Z

framework/generated/khronos_generators/khronos_base_decoder_body_generator.py

@@ -213,7 +192,7 @@ def make_decode_invocation(self, value, preamble, main_body, epilogue):
                    is_class and value.pointer_count > 1
                ):
                    if type_name in self.base_header_structs.keys():
-                        base_type_name = self.gen_child_var_name(value.base_type)
+                        base_type_name = self.makeSimpleVarName(value.base_type)


I wonder that should we unify the naming rule to pothole_case(make_simple_var_name)? or is it better to use dromedaryCase here? Plus, there are many other dromedaryCase functions. Should they be pothole_case? or dromedaryCase?

@bradgrantham @MarkY-LunarG @davidd-lunarg -- is there a standard all new code (including new functions for this refactor) should adhere to? Right now it's a mix.

I don't have a preference but we state PEP-0008 in CONTRIBUTING.md. PEP-8 appears to prefer pothole/snake case for functions and variables. Am I reading it right? https://peps.python.org/pep-0008/#function-and-variable-names

I was following the Khronos standard of naming which uses lower-camel-case for function names and lower-snake-case for variables. I honestly missed the whole PEP-0008 statement. I'm ok with reverting it back.

For example, beginFile, endFile, etc.

There are some functions with a specified naming from from the reg.py calls. We can PEP-0008 everything else in the new code.

Having function and variable naming match is somewhat odd until one considers how lambda heavy idiomatic/pythonic coding is.

I'll review that new def's match the standard.

locke-lunarg · 2024-11-25T19:05:46Z

framework/generated/khronos_generators/khronos_base_generator.py


    Adds options used by FrameworkGenerator objects during C++ language
    code generation.

    Additional members
      blacklists - Path to JSON file listing apicalls and structs to ignore.
      platform_types - Path to JSON file listing platform (WIN32, X11, etc.)
-        specific types that are defined outside of the Vulkan header.
+        specific types that are defined outside of the OpenXR header.


Should it be Khronos header?

locke-lunarg · 2024-11-25T20:53:38Z

I think we should rename BaseGenerator to VulkanBaseGenerator since only Vulkan uses it, and BaseGenerator inherites from KhronosBaseGenerator. It confused me to feel that BaseGenerator was the base of KhronosBaseGenerator.

MarkY-LunarG requested review from davidlunarg, locke-lunarg, bradgrantham-lunarg and davidd-lunarg October 30, 2024 18:24

MarkY-LunarG requested a review from jzulauf-lunarg October 30, 2024 18:56

jzulauf-lunarg approved these changes Nov 8, 2024

View reviewed changes

MarkY-LunarG removed request for davidlunarg and bradgrantham-lunarg November 11, 2024 18:21

MarkY-LunarG force-pushed the marky-move-vulkan-codegen-generators branch from 1c1c5cd to 9809efb Compare November 11, 2024 19:02

MarkY-LunarG mentioned this pull request Nov 12, 2024

Move vulkan codegen generators 2 #1874

Open

MarkY-LunarG requested review from panos-lunarg and removed request for davidd-lunarg November 21, 2024 17:30

MarkY-LunarG added 8 commits November 21, 2024 12:37

codegen vulkan: Move vulkan_generator scripts

5e238b4

Move the Vulkan generator scripts under a khronos_generators folder to prepare it for for sharing with other khronos APIs.

codegen dx12: Copy reformat_code.py

d0d2fad

DX12 was still relying on one file in the base_generators folder. Make a copy in the DX12 folder, and remove all references to the Vulkan and base_generators folders in the generate_dx12.py script.

codegen: Move base_generators under khronos_generators

7a11e55

codegen: Move common file into khronos_generator

60a2ed4

Move the reformat_code.py into the base of khronos_generators.

codegen: Move common items from vulkan to khronos

d7608d1

Move common items from the vulkan_generators/base_generator.py to the khronos_generators/khronos_base_generator.py

codegen dx12: Remove duplicate item

931ca07

codegen: Move more from vulkan to khronos

f425d15

Move more common type parsing into the khronos_base_generator from the vulkan/base_generator.

codgen: Automatically determine dispatchable handles

3c0c4f0

MarkY-LunarG added 16 commits November 21, 2024 12:37

codegen: Modify base_generators to remove Vulkan

7c635b2

base_generator code still has vulkan-specific items in it. This starts the process of pulling those out.

codegen: Remove more vulkan in base_generators

f983c7d

Remove more vulkan-specific items in the base_generator code path.

codegen: Modidfy extended struct handling

7cfe191

Modify the handling of 'pNext' extended struct variables in the base_generators to be more Khronos-generic instead of Vulkan-specific.

codegen: Move base_generator scripts up to khronos

ab912be

codegen: Start renaming base files

7e5c081

First phase moving of base_generator files to have a khronos identifier

codegen: Rename base_replay_consumer_body

e16046a

codegen: Rename BaseDecoderBody to match filename

b26951c

codegen: Updated BaseStructDecoderBody

04d34e8

codegen: Rename BaseStructDecodersHeader

317354b

codgen: Rename BaseStructHandleMappersBody

753c2af

codegen: Rename BaseStructHandleMappersHeader

5b96981

codegen: Update khronos base to have a write

b1bdac3

Since the khronos_base files were referring back to the parent "base_generator" (which in this case was Vulkan) it still resulted in a tight coupling. Move the "write" method into the khronos_base_generator script to solve this.

codgen: Consolidate structure_type name gen

eda0b01

Consolidate the structure type name generation into one location shared with all future Khronos APIs.

codegen: Consolidate var naming

5c7655e

Put variable naming in a common location

codgen: Remove local variables

e95a66d

Removed local variables for use in decoder body scripts and created a "all_cmd_params" that isn't cleared after every feature update.

codegen: Add exceptions for unimplemented overrides

e28c3a3

Added raising an exception on Python methods that should be implemented by the Khronos API for the functionality to properly work.

MarkY-LunarG force-pushed the marky-move-vulkan-codegen-generators branch from 9809efb to e28c3a3 Compare November 21, 2024 17:37

locke-lunarg approved these changes Nov 25, 2024

View reviewed changes

locke-lunarg reviewed Nov 25, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move vulkan codegen generators #1851

Move vulkan codegen generators #1851

MarkY-LunarG commented Oct 30, 2024 •

edited

Loading

ci-tester-lunarg commented Oct 30, 2024

ci-tester-lunarg commented Oct 30, 2024

ci-tester-lunarg commented Oct 30, 2024

jzulauf-lunarg left a comment

ci-tester-lunarg commented Nov 11, 2024

MarkY-LunarG commented Nov 11, 2024

ci-tester-lunarg commented Nov 11, 2024

ci-tester-lunarg commented Nov 11, 2024

ci-tester-lunarg commented Nov 11, 2024

ci-tester-lunarg commented Nov 11, 2024

ci-tester-lunarg commented Nov 11, 2024

ci-tester-lunarg commented Nov 21, 2024

ci-tester-lunarg commented Nov 21, 2024

ci-tester-lunarg commented Nov 21, 2024

locke-lunarg Nov 25, 2024 •

edited

Loading

jzulauf-lunarg Nov 25, 2024 •

edited

Loading

bradgrantham-lunarg Nov 26, 2024

MarkY-LunarG Nov 26, 2024

MarkY-LunarG Nov 26, 2024

jzulauf-lunarg Nov 26, 2024

locke-lunarg Nov 25, 2024

locke-lunarg commented Nov 25, 2024 •

edited

Loading

Move vulkan codegen generators #1851

Are you sure you want to change the base?

Move vulkan codegen generators #1851

Conversation

MarkY-LunarG commented Oct 30, 2024 • edited Loading

ci-tester-lunarg commented Oct 30, 2024

ci-tester-lunarg commented Oct 30, 2024

ci-tester-lunarg commented Oct 30, 2024

jzulauf-lunarg left a comment

Choose a reason for hiding this comment

ci-tester-lunarg commented Nov 11, 2024

MarkY-LunarG commented Nov 11, 2024

ci-tester-lunarg commented Nov 11, 2024

ci-tester-lunarg commented Nov 11, 2024

ci-tester-lunarg commented Nov 11, 2024

ci-tester-lunarg commented Nov 11, 2024

ci-tester-lunarg commented Nov 11, 2024

ci-tester-lunarg commented Nov 21, 2024

ci-tester-lunarg commented Nov 21, 2024

ci-tester-lunarg commented Nov 21, 2024

locke-lunarg Nov 25, 2024 • edited Loading

Choose a reason for hiding this comment

jzulauf-lunarg Nov 25, 2024 • edited Loading

Choose a reason for hiding this comment

bradgrantham-lunarg Nov 26, 2024

Choose a reason for hiding this comment

MarkY-LunarG Nov 26, 2024

Choose a reason for hiding this comment

MarkY-LunarG Nov 26, 2024

Choose a reason for hiding this comment

jzulauf-lunarg Nov 26, 2024

Choose a reason for hiding this comment

locke-lunarg Nov 25, 2024

Choose a reason for hiding this comment

locke-lunarg commented Nov 25, 2024 • edited Loading

MarkY-LunarG commented Oct 30, 2024 •

edited

Loading

locke-lunarg Nov 25, 2024 •

edited

Loading

jzulauf-lunarg Nov 25, 2024 •

edited

Loading

locke-lunarg commented Nov 25, 2024 •

edited

Loading