RFC: What's next for premake-next?

[starkos]

TL;DR: I need to make a call on how to proceed with premake-next: push ahead on a separate branch for a version 6.x, or backport the changes to the current 5.x version?

If you've been following along, you're probably aware that I've been working on something called premake-next, doing my best to address what I feel are the biggest shortcomings and development bottlenecks. If this is news to you, here's the original README I wrote for the endeavor, and a list of major changes made so far. You can also see examples of what it looks like in practice over in the community updates here and here.

It's turning out well. I've confirmed that it will work as advertised and solve a whole class of configuration issues that have plagued the project. Now I need to make a call on how to proceed: push ahead with the new code on a separate branch for a version 6.x, or backport the changes to the current 5.x version.

I feel pretty strongly that the best path forward is…

Push ahead as 6.x

The path I'd prefer to take is:

1. Create a new 6.x branch off of main, drop in the current premake-next revision, and continue development along that branch. I could also keep the code in a separate premake6 repository but I feel like this project is fragmented enough as it is. Open to feedback on that.
2. Tag 5.x as stable—warts, flaws, and all, and continue to support it as we have been
3. When the time comes, branch off a 5.x support branch and merge 6.x to main

The pros to this approach:

• People invested in v5 can continue using it without having to deal with new, potentially breaking changes in each new release and migrate when they're ready
• We can tag Premake5 as stable and get it into package repositories
• So much of the current code needs to be touched regardless. Even if I wasn't swapping out the entire configuration & query system the Gmake and Xcode exporters are a mess, and I feel strongly that the project needs to move to the new system. Many of the underlying APIs need to be cleaned up and their symbols standardized in order to support long-term module support.
• I can move more quickly and break things for the better

The cons:

• Premake5 will continue to be a glorious hot mess and a testament to the compounding powers of technical debt
• Rewrites are bad, and it will be work to capture all of the improvements to v5
• Having two separate development branches could confuse potential contributors
• Module developers will need to support two major revisions

The alternative is…

Backport to 5.x

The pros to this approach:

• Improvements could start landing and be usable sooner
• It avoids rewriting (except not really, see above)
• The versioning story is simpler

The cons are all the reasons everyone throws around to justify a rewrite:

• I'll need to provide migration paths and fallbacks for all of the new changes to give people time to adjust to the new code, which will take extra effort and time. Or I'll have to regularly break the code. Probably both.
• Modules and overrides will definitely break, often and repeatedly. There just isn't going to be any way around that. Large functions are going to be broken up for overloading (see the Visual Studio exporter for an example), symbols are going to be standardized.

Rewrites are bad

I know first hand: rewriting is bad. I'm trying to minimize it where I can, by copying in code over as-is, getting it working in place, and then refactoring to fit it into the new project. Now that the foundational stuff is there I should be able to do this more. Whatever approach gets taken, I recognize this.

So…what's your take?

[AllergicMushroom]

Definitely the first path for me. I don't like the idea of recycling old code and being between a messy v5 and a recently born v6. This way also seems easier for you and contributors.

Finally, and this is totally personal, I'd prefer a separate v6 that you're proud of instead of a fusion between v5 and v6 that you juge as "satisfying".

Either way, I'm waiting for it :)

[redorav]

For me, probably the main concern is whether it can become another premake4 vs premake5 kind of thing. I already find it to be a shame that GENie spawned off into its own thing and both projects, being essentially the same, don't collaborate. Personally, I can rewrite generation scripts once premake6 is stable but I wonder about big companies or established codebases. Will they stick with premake5 and therefore any good improvements that would land into this project be lost?

Regardless, I feel quite excited for premake6

[starkos]

I hear you, and I agree. If we take the "branch of v6" approach—and it sounds like consensus is going that way—v5 will remain "the" version of Premake. I'll continue to work the PRs and make v5 releases. I will not be making any official releases of v6 until it is up to a beta level of features and stability. When things get that far, both versions will live in this repository (right?), and there will be a single website publishing both releases.

Also: the community is much more involved with v5 than they were with v4. People used to throw stuff over the wall and wait for me to fix it; development is more collaborative and proactive now. (Would have been happy to work with the GENie folks too if they had bothered to reach out, at least right up to the point where they stripped my name & copyright from all the code they lifted.)

[nickclark2016]

As a user, I'd prefer the first path, as it won't break my code. However, I'd be more than happy to test on premake6 on some of my projects once that becomes stable enough for a 6.x branch, assuming that's the route that you decide to take.

[samsinsane]

I'm a bit late to the party here, but I agree with everyone else here, the first path is the best option.

Regarding concerns about user adoption of v6, I think having a migration guide once it's stable will help greatly. On top of this, having scripts or a v5->v6 action could be helpful too.

[starkos]

On paper, I think I should be able to put together a compatibility module to at least handle the project APIs (the stuff listed in the v5 _premake_init.lua). Probably won't be much I can do for overloads or third-party modules but I will continue making an effort to document the differences as best I can. Thanks for the feedback!

[samsinsane]

Yeah, overloads and modules have always been problematic so I don't think that's really an issue.

[nickclark2016]

I want to follow up on this @starkos. I've got a bit of free time and would be interested in helping out with v6. What can I do to help you out right now (if anything)?

[starkos]

Well, let's see…if you haven't already, get up and running on the v6 branch here. I'm currently slapping together a v6 version of the documentation website to cover the big changes while they are still fresh in my mind, hope to have that done enough shortly. After that I was thinking to bootstrap Makefile and Xcode exporters and try to get them up to the same level as what I've already done for Visual Studio. Anything there sound interesting?

[nickclark2016]

I can handle makefiles if you'd like. From a sanity standpoint, I assume we do not want to bring over the "gmake" vs "gmake2" notation. We just have "gmake" again?

[Jarod42]

I can update https://github.com/Jarod42/premake-sample-projects to include v6 (and possibly migration scripts when available :-) ).

(I can help on Codelite/CodeBlocks generators).

[starkos]

I would love to be able to divide and conquer these exporters, but make sure you know what you're signing up for! 😆

For Makefiles (and probably CodeLite/Code::Blocks too), we'll have to…

• wade through the differences between the two exporters. My understanding is that gmake2 was started by Blizzard to improve/enable support for custom rules but I don't know where it was left off.
• The code will have to be reworked to follow the same coding patterns as the Visual Studio exporter to allow modules to extend.
• The structure of the makefiles we generate will have to be rethought to support the abilities of the new query system. In v5, given a script like...

workspace 'MyWorkspace'
   configurations { 'Debug', 'Release' }
   defines 'WKS'

   project 'MyProject'
      files { '**.h', '**.cpp' }
      defines 'PRJ'

      filter { 'configurations:Debug' }
         defines 'PRJ_DEBUG'

       filter { 'file:SomeFile.cpp', 'configurations:Debug' }
         defines 'FILE_DEBUG'

...the v5 Gmake exporter would see a "Debug" configuration object with the defines { 'WKS', 'PRJ', 'PRJ_DEBUG' }. Values were always inherited from workspace to project to configuration and there wasn't anything the exporter could do about it, which is why there is so much duplication in our currently generated makefiles. We now have the ability to ask the state system to only show the additions to the containing scope, which means we should be able to eliminate that duplication. So in v6, we could query a project level configuration which inherits from the workspace ({ 'WKS', 'PRJ' }), a "Debug" configuration which does not inherit ({ 'PRJ_DEBUG' }), and also a file-specific "Debug" configuration with no inheritance ({ 'FILE_DEBUG' }). Need to figure out how to make use of that capability.

I think the v5 makefiles support limited per-file switches, but I'm fairly confident it doesn't support per-configuration file level switches (the last filter block in the example). That needs to be added, and I haven't had a chance yet to consider how that might be expressed in a makefile.

But wait, there's more! Exporters now have full control over their own state queries, which means they can queries for multiple platforms (Windows, Linux, etc.), which means we could potentially export a single makefile capable of building across multiple platforms. Again, I haven't dug into how that might be expressed in a makefile, and what additions to the scripting APIs might be needed.

Oh, and we've got to sort out the whole language+toolset thing. And custom build rules.

Still in? 😆

IMHO probably the best first step would be to handcraft makefile(s) to address the stuff mentioned here, in particular per-file-and-configuration build switches, and a proof-of-concept of multiplatform makefiles, if that's feasible. Those would be super useful as references for building out the new exporter, and would make the rest of the work more "fill in the blank", if that makes sense.

A personal life goal is get v6 to generate its own Bootstrap.mak.

Lots of stuff there, feel free to lob questions at me.