Improve Objective-C support

[LunaTheFoxgirl]

This pull-request aims to improve the Objective-C support of LDC. This PR additionally changes the Objective-C generation system to only support the Objective-C 2.0 ABI, as very little documentation of older ABIs have survived.

A good chunk of this code is more or less taken from the OpenD implementation, but I'm already working on cleaning parts up as I go. I am looking into how feasible it'll be to make extern(Objective-C) classes export neccesary ABI for Objective-C code to be able to call it as well, which mainly involves metadata generation.

TODO List:

• Classes
  • ClassRO Tables
  • Class List Tables
  • Swift Stub Class Tables (@swift keyword)
  • Dynamic Class Casting
• Methods
  • Method List Tables
  • Selector Reference Tables
• Protocols
  • Protocol List Table
  • Dynamic Protocol Casting
• Instance Variables
  • Generate instance variables and offsets

[LunaTheFoxgirl]

Question: in this implementation I wish to add another UDA to ldc.attributes called @swift.

Idea there is to implement support for binding Swift interop class stubs as well, which have unique behaviour that would require this new attribute.

Essentially Swift may declare Objective-C classes in the __objc_stubs section with an isa field pointing to address 0x1. Followed by a function pointer that would need to be invoked to instantiate the class definition, via the objc_loadClassRef helper function.

This would allow interop with Swift code using bridging headers and wouldn't be too much extra work.

Is it okay if I add that to this PR?

[thewilsonator]

Is it okay if I add that to this PR?

Yes.

[adamdruppe]

I am looking into how feasible it'll be to make extern(Objective-C) classes export neccesary ABI for Objective-C code to be able to call it as well, which mainly involves metadata generation.

this already works in opend so if you copy the impl it should work here too; that's how the final iphone demo from my blog worked, as well as simpledisplay's cocoa implementation, and to triple check i just did one more test:

@interface Thing : NSObject
   - (void) test
@end 

int main() {
  [[[Thing alloc] init] test];
}

and then the D

extern(Objective-C)
class Thing : NSObject {
  void test() @selector("test") {
        printf("here\n");
   }
}

and all worked.

of course, possible i missed some details that won't emerge until you get into more features, but still a lot already works in the opend world.

[adamdruppe]

BTW something i think would be kinda cool is a kind of import objective-c

dstep can do a good amount of that already, and the objective c parts look easier to parse and interface with than regular import c

so im p sure it doable with a week or two of work

[LunaTheFoxgirl]

BTW something i think would be kinda cool is a kind of import objective-c

dstep can do a good amount of that already, and the objective c parts look easier to parse and interface with than regular import c

so im p sure it doable with a week or two of work

Yeah, I'm thinking of making an Objective-C code generator once I get the compiler support up and running.
Got protocols working now, might implement categories as well. Next up is codegen for swift stub classes.

[thewilsonator]

(once this is good to go) Could you please open a PR to DMD with all of the frontend/druntime changes? (Just stub out anything needs anything extra to work)

[LunaTheFoxgirl]

(once this is good to go) Could you please open a PR to DMD with all of the frontend/druntime changes? (Just stub out anything needs anything extra to work)

Sure, will do. However I think I'll reduce the scope of this PR slightly to just be the core objective-c and Swift stub class support. Given that some existing Apple platforms don't align neatly with the code I was referencing I'm doing a pretty big refactor of the objective-c data struct generation to be compatible with all of the Apple sub architectures supported.

[LunaTheFoxgirl]

cc @adamdruppe once I finish this PR you may want to back integrate it into OpenD's LDC fork. Your current implementation is broken on quite a few platforms and in some cases may crash the Xcode linker

[adamdruppe]

plz write up some notes as you go, as it might also apply back to dmd's separate implementation (and the abi spec on the website) as well. depends on if it was just me bugging it up or if the original was wrong too.

much of it came from reverse engineering the output on a x86 Mac so no surprise they'd be some differences (one we all found early is the arm objc_msgSend is slightly different on arm), then i just sucked at ldc code so prolly a lot of impl bugs too lol.

[LunaTheFoxgirl]

plz write up some notes as you go, as it might also apply back to dmd's separate implementation (and the abi spec on the website) as well. depends on if it was just me bugging it up or if the original was wrong too.

much of it came from reverse engineering the output on a x86 Mac so no surprise they'd be some differences (one we all found early is the arm objc_msgSend is slightly different on arm), then i just sucked at ldc code so prolly a lot of impl bugs too lol.

Yeah my recommendation there is to look at the actual objective-c runtime source. The data structures relevant are listed there https://github.com/apple-oss-distributions/objc4 as well as the Swift interop documentation.

I'm adding the finishing touches to this PR tonight, and will mark it as ready to merge then.

[kinke]

Nice work, thanks for tackling this! Just a superficial review from my side; I have 0 interest in Apple specifics and don't wanna get into its details myself.

[LunaTheFoxgirl]

Nice work, thanks for tackling this! Just a superficial review from my side; I have 0 interest in Apple specifics and don't wanna get into its details myself.

Yeah, I kind of have to get into the specifics myself due to my business needing it. Initially just used cursed compile time shenanigans to get things working but I ended up realising that in some ways to get my product onto the App Store for iPad and a second product to iPhones I need to have proper support in LDC.

I plan to make further PRs in the future to improve the state of objective-c (and Swift) support to LDC and possibly DMD.

[kinke]

There are at least 2 Objective-C tests from upstream that are still disabled for LDC: compilable/objc_{class,interface_final_19654}.d. I guess just an oversight? (I've just grepped the dmd-testsuite CI log for objc.)

[LunaTheFoxgirl]

There are at least 2 Objective-C tests from upstream that are still disabled for LDC

yup, just an oversight on my part, there we go.

[LunaTheFoxgirl]

@kinke okay, those tests have been added and they pass now, also added selector auto-gen for property functions like specified in the DIP.

That being said, this selector gen should be implemented upstream properly down the line. Will look into making a patch to DMD later, then phasing out my little hack there.

[jacob-carlborg]

also added selector auto-gen for property functions like specified in the DIP.

That's not implemented in upstream IIRC.

Try to upstream as much as possible when implementing new features.

[LunaTheFoxgirl]

I will, currently doing some refactoring to make the code easier to follow.
After this is merged, I'll look into making a PR for upstream.

That being said, DMD is practically useless for apple device development given that it's arm support is not completed, and I have limited time to spend on this. I really need to get back to work on my products.

[LunaTheFoxgirl]

@adamdruppe I've refactored the code to be less messy, hopefully it's easier to follow now?

[adamdruppe]

I'll reread the whole thing when I get another chunk of time free, idk if it'll be today or tomorrow, but I'll let you know when I do.

My biggest concern is just if it works and that last test I did with it passing was a good sign!

[LunaTheFoxgirl]

@kinke anything left to solve? All the tests succeed (minus the ones that are either broken for unrelated reasons and the ones where you've run out of compute credits)

[LunaTheFoxgirl]

Also it seems my refactor somehow fixed all the random asserts that was happening.

[LunaTheFoxgirl]

I think that's everything?
Sorry if I'm a bit pushy, I've been working unholy hours on this the past few days. I'm very exhausted ^^;