List of frameworks to translate

[silvanshade]

This is a list of frameworks I've been working through in the translation PRs.

This is basically just what shows up on the Apple website under system frameworks, technologies page, the SDK directories, etc.

Some of these are obsolete and probably not worth bothering with. Several have issues that will prevent translation until more work is done on the translator (e.g., the core foundation stuff). Some give errors related to seemingly not finding the header files (not sure why, but I've added notes about those below).

Some are only available in Swift and I guess this will only increase in the future. It would be nice if we could figure out a way to still include bindings for those but I suppose it might require building out some new tooling?

• Accelerate
• Accessibility
• Accounts (deprecated)
• AddressBook + AddressBookUI.h
• ActivityKit (swift)
• AdServices
• AdSupport
• AGL (deprecated)
• AppClip
• AppIntents (swift)
• AppKit
• AppleArchive
• AppleCryptoKit (swift)
• AppleScriptKit
• AppleScriptObjC
• ApplicationServices (needs Carbon stuff)
• AppTrackingTransparency (doesn't find header?)
• ARKit (needs iOS?)
• AssetsLibrary (deprecated, needs iOS)
• AudioDriverKit
• AudioToolbox
• AudioUnit
• AudioVideoBridging
• AuthenticationServices
• AutomaticAssessmentConfiguration
• Automator
• AVFAudio
• AVFKit
• AVFoundation
• AVRouting
• BackgroundAssets
• BackgroundTasks
• BlockStorageDeviceDriverKit
• BusinessChat
• CalendarStore (deprecated?)
• CallKit
• Carbon
• CareKit
• CarKey (requires special entitlement)
• CarPlay (catalyst / iOS)
• CFNetwork
• CHIP
• ClassKit
• ClockKit (catalyst)
• CloudKit
• Cocoa
• Collaboration (needs CoreServices and Security)
• ColorSync
• Combine (swift)
• Compression
• Contacts
• ContactsUI
• CoreAudio
• CoreAudioKit
• CoreAudioTypes
• CoreBluetooth
• CoreData
• CoreDisplay
• CoreFoundation
• CoreGraphics
• CoreHaptics
• CoreImage
• CoreLocation
• CoreLocationUI
• CoreMedia
• CoreMediaIO
• CoreMIDI
• CoreMIDIServer
• CoreML
• CoreMotion
• CoreNFC
• CoreServices
• CoreSpotlight
• CoreTelephony
• CoreText
• CoreTransferrable
• CoreVideo
• CoreWLAN
• CreateML (swift)
• CreateMLComponents (swift)
• CryptoTokenKit
• DeviceActivity
• DarwinNotify
• DataDetection
• DeveloperToolsSupport (swift)
• DeviceActivity
• DeviceCheck
• DeviceDiscoveryExtension
• DeviceDiscoveryUI
• DeviceManagement
• DirectoryService
• DiskArbitration (doesn't find header?) (needs CoreFoundation?)
• DiskRecording (deprecated?)
• DiskRecordingUI (deprecated?)
• dispatch
• Distributed (swift)
• dnssd
• DocC
• DriverKit
• DVDPlayback (deprecated?)
• Endpoint Security
• EventKit
• EventKitUI
• ExceptionHandling
• ExecutionPolicy (doesn't find header?)
• ExposureNotification
• ExtensionFoundation (swift)
• ExtensionKit
• ExternalAccessory
• FamilyControls (swift)
• FileProvider
• FileProviderUI
• FinderSync (doesn't find header?)
• ForceFeedback (doesn't find header?)
• FWAUserLib
• GameController
• GameKit
• GameplayKit (needs SIMD typedefs)
• GLKit
• GLUT
• GroupActivities (swift)
• GSS
• HealthKit
• HIDDriverKit
• HomeKit
• Hypervisor
• iAd
• ICADevices
• IdentityLookup
• ImageCaptureCore
• ImageIO
• IMServicePlugIn
• InputMethodKit
• InstallerJS
• InstallerPlugins
• InstantMessage
• Intents
• IntentsUI
• IOBluetooth
• IOBluetoothUI
• IOKit
• IOSurface
• IOUSBHost
• iTunesLibrary
• JavaNativeFoundation
• JavaRuntimeSupport
• JavaScriptCore
• Kerberos
• Kernel
• KernelManagement
• LatentSemanticMapping
• LDAP
• LinkPresentation
• LocalAuthentication
• LocalAuthenticationEmbeddedUI
• MailKit
• ManagedSettings (swift)
• ManagedSettingsUI (swift)
• MapKit
• Matter (requires raising recursion limit to higher than 1024; never saw it finish compiling)
• MatterSupport
• MediaAccessibility
• MediaLibrary
• MediaPlayer
• MediaSetup
• MediaToolbox
• Messages
• MessageUI
• Metal
• MetalFX
• MetalKit
• MetalPerformanceShaders
• MetalPerformanceShadersGraph
• MetricKit
• MLCompute
• ModelIO (needs SIMD typedefs)
• MultipeerConnectivity
• MusicKit (swift)
• NaturalLanguage
• NearbyInteraction
• NetFS
• Network
• NetworkingDriverKit
• NetworkExtension
• NewsstandKit
• NotificationCenter
• ObjectiveCRuntime
• OpenAL
• OpenCL
• OpenDirectory
• OpenGL
• OpenGLES
• os
• OSAKit
• OSLog
• PackageDescription (swift)
• ParavirtualizedGraphics
• PassKit
• PCIDriverKit
• PCSC
• PDFKit
• PencilKit
• PHASE (needs AVFoundation and CoreMedia)
• PhotoKit
• PlaygroundBluetooth (swift)
• PlaygroundSupport (swift)
• PreferencePanes
• ProximityReader (swift)
• PushKit
• PushToTalk
• Quartz
• QuartzCore
• QuickLook
• QuickLookThumbnailing
• QuickLookUI
• RealityKit
• RegexBuilder (swift)
• ReplayKit
• ResearchKit
• RoomPlan (swift)
• SafariServices
• SafetyKit
• SceneKit (needs ModelIO and CoreGraphics)
• ScreenCaptureKit
• ScreenSaver
• ScreenTime
• ScriptingBridge
• SCSIControllerDriverKit
• SCSIPeripheralsDriverKit
• Security
• SecurityFoundation
• SecurityInterface
• SensorKit
• SerialDriverKit
• ServiceManagement
• ShazamKit
• simd
• SiriCloudMedia
• SiriKit
• Social
• SoundAnalysis
• Spatial
• Speech
• SpriteKit (needs SIMD typedefs)
• StoreKit
• StoreKitTest
• SyncServices
• SwiftCharts
• SwiftUI
• System
• SystemConfiguration
• SystemExtensions
• TabularData
• Tcl
• ThreadNetwork
• TVML
• TVMLKit
• TVServices
• TVUIKit
• TWAIN
• UIKit
• UniformTypeIdentifiers
• USBDriverKit
• USBSerialDriverKit
• UserNotifications
• UserNotificationsUI
• VideoDecodeAcceleration
• VideoSubscriberAccount
• VideoToolbox
• Virtualization
• Vision
• VisionKit
• vmnet
• WatchConnectivity
• WatchKit
• WeatherKit (swift)
• WebKit
• WidgetKit (swift)
• XcodeKit
• xcselect
• XCTest
• XPC

[madsmtm]

Some are only available in Swift and I guess this will only increase in the future. It would be nice if we could figure out a way to still include bindings for those but I suppose it might require building out some new tooling?

Indeed, that's basically doing the entirety of objc2, just in a new language (people have already begun in swift-rs, swift-bridge and swift-bindgen), and that will, well, take a lot of work.

In any case, I suspect it will be years before Apple releases a Swift-only frameworks that an application developer actually needs to use (SwiftUI is still very much in its early stages). So let's tackle that when the need arises!

[silvanshade]

In any case, I suspect it will be years before Apple releases a Swift-only frameworks that an application developer actually needs to use (SwiftUI is still very much in its early stages). So let's tackle that when the need arises!

I agree that it probably won't become mandatory for quite some time. Still, it's a little disappointing that some of the newer technologies will be unavailable.

I wonder how feasible it would be to write some sort of objective-c bridging modules for those few cases and then run the translator against that. It wouldn't be a fully automatic process but maybe worth experimenting with?

Another possibility would be to more seriously consider the approach you described in #345. I only looked at it briefly so far, but it seems that if we were to build against that implementation, we could take advantage of the clang module importer (for objective-C frameworks), on the one hand, but also be able to re-use most of the same infrastructure for importing swift modules, right? As I understand it, most of the work we would need to do would be to implement a new code generator that maps the swift module AST (whether it is synthesized from objective-C or directly from native swift) to our rust equivalent.

[madsmtm]

I wonder how feasible it would be to write some sort of objective-c bridging modules for those few cases and then run the translator against that. It wouldn't be a fully automatic process but maybe worth experimenting with?

You mean modules that use Swift's @objc attributes? + some generated headers? Or is there some mechanism for using Swift from inside Objective-C that I'm not aware of?

Another possibility would be to more seriously consider the approach you described in #345. I only looked at it briefly so far, but it seems that if we were to build against that implementation, we could take advantage of the clang module importer (for objective-C frameworks), on the one hand

I am seriously considering that approach, it's honestly the only sustainable option, header-translator is a hack that has to go one day.

be able to re-use most of the same infrastructure for importing swift modules, right?

Probably, and you do make a good point about it being a good starting point for it!

As I understand it, most of the work we would need to do would be to implement a new code generator that maps the swift module AST (whether it is synthesized from objective-C or directly from native swift) to our rust equivalent.

I honestly don't think that part will be too difficult, if we get the above-mentioned setup for it.

Rather, I worry that getting the ABI and calling convention right (which is still not done for objc2) will take a lot of time, or perhaps more aptly, mental bandwidth (there's a lot to learn!), especially if we need to handle generics.

---

Anyhow, sorry for my dismissiveness, I think I'm just focused on getting objc2 v0.3 done (I have been working on it for more than 18 months now, would be nice to actually finally release something useful into the hands of users) - this is also why I'm punting on #345, since it will take time that I would rather be focusing elsewhere.

So it's not that I don't want to work on/support Swift, it's just that I can neither allocate the time nor energy for it, nor promise that I will be able to, sorry about that :/

[silvanshade]

You mean modules that use Swift's @objc attributes? + some generated headers? Or is there some mechanism for using Swift from inside Objective-C that I'm not aware of?

Yeah, probably something like that. I'm not entirely certain how it would work out since I've barely used Swift, but it at least seems plausible that something like that could be done, at least on a smaller scale, and with enough effort. But that's something to worry about later I guess.

Anyhow, sorry for my dismissiveness, I think I'm just focused on getting objc2 v0.3 done (I have been working on it for more than 18 months now, would be nice to actually finally release something useful into the hands of users) - this is also why I'm punting on #345, since it will take time that I would rather be focusing elsewhere.

So it's not that I don't want to work on/support Swift, it's just that I can neither allocate the time nor energy for it, nor promise that I will be able to, sorry about that :/

Yeah, that's perfectly reasonable. I'm mostly just thinking out loud here as far as what it would take to get closer to a complete translation of these frameworks.

I agree that in the near term it's more important to focus on ironing out some of the current issues and working toward getting the next release out.

[silvanshade]

@madsmtm is it currently possible to run the translator for different platforms (e.g., iOS, tvOS, watchOS)? Some of these frameworks are only available on certain platforms and it would be nice to include them.

Also, what about Catalyst frameworks? It seems the header translator doesn't find these which I suspect is probably because they are in a different location. Looks like they are inSystem/iOSSupport/System/Library/Frameworks versus System/Library/Frameworks.

Maybe some command line options could be added to control this?

[silvanshade]

I also wonder if we may need to think of how to structure the modules differently to account for platform differences. For example, some of these frameworks are available for standard macOS development but also Catalyst. I'm not sure to what extent the header definitions are different but they are separate frameworks so that might be an issue.

[madsmtm]

This is the real reason why header-translator is not just a fork of bindgen: It is because I was hoping we'd be able to take the list of Stmt's generated by the different platforms, and then combine them with cfg statements, such that we could expose a really nice docs.rs output with working "only available on target_os = macos" notes.

Not sure about catalyst yet.

[lachlansneff]

Hi, I'm interested in using the Vision api here. Could y'all point me to some resources on how I could get that ball rolling?

[madsmtm]

The documentation for doing so is quite lacking, but the first steps for you would probably be taking a look at header-translator's readme, and then add the library to translation-config.toml and framework-includes.h.

Maybe you can also take some inspiration from the framework addition PRs that @silvanshade has contributed in the past.

Feel free to ask further questions, I'm also available on Matrix/Element as @madsmtm:matrix.org btw.

[bes]

UiKit support would be a dream come true ❤️

[marcpabst]

Hey, I'm interested in adding support for ARKit. Are there any problems I'd need to anticipate, given it's iOS-only?

[madsmtm]

Definitely problems, you can't do iOS-only frameworks yet, though I'm inching closer to it. Tentative timeline would be UIKit / support for other iOS frameworks around the start of March.

[marcpabst]

Oh, that would be nice! I'm currently using a wacky apple-sys/objc implementation which feels a bit brittle and takes ages to compile.

[roniemartinez]

I hope there is some progress with CoreServices (Launch Services), too.

At the moment, I am using icrate's NSWorkspace to check and set the default browser, however this API is not available in older versions of MacOS. For now, my alternative is to add and use the core_foundation crate for any version less than 12.

• setDefaultApplicationAtURL:toOpenURLsWithScheme:completionHandler (12+): https://developer.apple.com/documentation/appkit/nsworkspace/3753005-setdefaultapplicationaturl?language=objc
• Deprecated LSSetDefaultHandlerForURLScheme (<12): https://developer.apple.com/documentation/coreservices/1447760-lssetdefaulthandlerforurlscheme?language=objc

A large chunk of MacOS users are still on <12 (if the statistics are correct) so hopefully this can be added up in the queue.

[madsmtm]

Update on UIKit: I've put up #600, which I expect will be a bunch of trouble to get fully compiling, but mostly a bunch of tedious trouble, not a bunch of impossible-this-is-not-how-the-world-works trouble ;)

CC @simlay @jpedrick

[jinleili]

Apple provides a new framework called CompositorServices for immersive apps in visionOS. It would be great if we could have interface bindings for CompositorServices.

[madsmtm]

Apple provides a new framework called CompositorServices for immersive apps in visionOS. It would be great if we could have interface bindings for CompositorServices.

Yeah, that'd indeed be nice. A bit difficult because it's only available on visionOS, so we'd have to do even further hacks to compile on that platform as well, but eh ;)

I won't have the time to do it myself for at least a few weeks, but you can try if you want? You'd probably have to edit around here to compile for visionOS, and then add the framework-crates/objc2-compositor-services/translation-config.toml to make header-translator compile it.

[jinleili]

I can give it a try this weekend.

[ijsnow]

CoreServices would be great! I'd like to use Dictionary Services.

[Brooooooklyn]

CoreAudio please!

[madsmtm]

Update: I managed to get a bare-bones CoreFoundation set up in #676, and have been slowly accumulating bindings to other frameworks like CoreGraphics, CoreText, CoreVideo, IOSurface etc. since then.

I won't guarantee anything in terms of CoreServices and CoreAudio yet though (especially because a lot of work is needed to actually make them nice to use), so in the meantime I'll link existing crates for these:

• core-services
• coreaudio-rs

Not great if you're also using objc2, but there are ways to work around that.