PactSwift repository history purge July 2024

[surpher]

TL;DR

Warning

• PactSwift project's history will be rewritten and old commits and releases up to latest will be purged.
• Purge will happen on July 1st, 2024 or any time during the week starting with.
• Size of PactSwift and dependent repositories will be reduced.
• Forks will break when attempting to sync with original repository.

Context

Due to the nature of this project and various attempts of sharing incredibly oversized libpact_ffi binaries this repository has grown in size to a level where it is no longer maintainable. Using binaries built from Rust code allows us to keep the same implementation of Pact specification across different platforms.

Relying on binaries however requires us to keep each of the releases and attempts to test changes where binaries were only slightly modified committed into the repository making the repository unnecessarily big.

PactSwift also aims to support both x86_64 and arm64 platforms for simulators and physical devices (with caveats). libpact_ffi binaries need to be compiled from rust code for each of the platforms and targets. That means each of the binaries must contain runtimes so that PactSwift users do not need to install and configure rust on their development and CI machines, and can run on a physical device. Unfortunately, this means that binaries end up being "fat binaries" containing code for their specific architectures.

Example sizes of libpact_ffi binaries:

Platform	Architecture	Target	Running on	Size
iOS	arm64	iOS Simulator	💻 Apple Silicon mac	~75.2 MB
iOS	arm64	iOS	📱 iPhone, iPad	~75.2 MB
darwin	arm64	macOS app	🖥️ Apple Silicon mac	~76 MB
iOS	x86_64	iOS Simulator	💻 Intel mac	~74.5 MB
darwin	x86_64	macOS app	🖥️ Intel mac	~77.1 MB
				~378MB

With the above being configured in a way that PactSwift users only need to worry about #import PactSwift, that means all those MBs need committed into the repository and they are pulled onto developers' machines. Imagine the size of all the versions of those binaries being committed into the repository.

The big size of PactSwift along with the size of PactSwiftMockServer dependency can be detrimental to developer experience, development machines, CI agents, network bandwidth allowance, etc. It blew up to multiple GBs!

Unfortunately Swift Package Manager clones the entire repository instead of just doing a shallow clone. This means that all the machines in the world cloning this repository (including CI agents) need to pull GBs worth of data every time they trigger resolve dependencies command.

PactSwift used to actively support Carthage as well. Using Carthage suffers from the same issue of cloning the entire repository. Additionally, Carthage seems to be dropping transient dependencies and so the Carthage support was gradually ignored.

This is a non-sponsored Open Source project and there is no intention to pay for GitHub LFS nor bandwidth needed to transfer all this data most of which is useless.

Tip

More information about what libpact_ffi is can be found here.

Alternatives

Building binary dependencies on developers' machines

It would be possible to reduce the size of this project by pushing the responsibility for sourcing libpact_ffi binaries to users of this framework.

This could be either by:

• Manually downloading and seting up a specific release archive from pact-reference project
• Building the binaries for needed targets on developer's machines.

Both of these have been tried and tested and proven to itroduce a barrier to adoption.

XCFrameworks

XCFrameworks were introduced to PactSwift its dependencies as they have proven to strip the binaries down to the size below 100MB in some cases. Yet they still wrapped at least 3 making each new release containing changes to libpact_ffi blow up by another 300MB or so.

Despite that, XCFrameworks remain the most appropriate way to share binary targets.

Example sizes of libpact_ffi binaries in XCFramework:

Platform	Target	Size	Note
iOS Simulator	arm64, x86_64	~74 MB	fat binary
iOS device	arm64	~38.9 MB
darwin	arm64, x86_64	~81.5 MB	fat binary
		~197.7 MB

These sizes are much more palatable being ~52% of their original size, yet still extremely big to keep them all in same repo or shared as a non-compressed dependency!

Decision

Caution

1. ⚠️ This project's history will be purged! ⚠️
2. History will be purged on July 1st, 2024 (or the week starting with)!
3. Before the purge, an archive of this repository will be created.
4. One new release will be created that will include warning messages about purging history, with ability to supress it.
5. Only the latest release will remain available on a protected branch named compatibility/vX.Y.Z - containing the code > supporting Objective-C and Linux.
6. PactSwiftMockServer repository will only be responsible to maintain the wrapper code around libpact_ffi and generate > rust binaries.
7. PactSwiftMockServer will be converted into XCFramework.
8. XCFramework will be split into two, each will support either iOS or macOS target, but not both.
9. XCFramework will be hosted in a separate repository and PactSwift will be set to point at a ZIP archive (see more).
10. Next version of PactSwfit will shift its main focus to support modern swift concurrency.

Consequences

• Forks of PactSwift repository will experience issues trying to sync with original repository.
• Projects set up to use any other version than the latest release will break.
• Projects set up to use a specific commit hash or branch will break.
• XCFrameworks can be distributed through SPM, Carthage and CocoaPods - support for more package managers may be re-introduced.

Resources

• Task: Find a way to reduce PactSwiftMockServer repo size #117
• Is PactSwift still very active? #109 (comment)
• https://developer.apple.com/documentation/xcode/distributing-binary-frameworks-as-swift-packages
• https://forums.swift.org/t/shallow-git-clone-for-ci-purposes/25032
• SwiftPM requires downloading entire git repository, which can be much larger than necessary swiftlang/swift-package-manager#6062
• [SR-3313] Swift package manager should clone with depth 1 by default swiftlang/swift-package-manager#5159
• Question: Allow shallow copy Carthage/Carthage#1067
• https://medium.com/@mihail_salari/a-deep-dive-into-xcframeworks-creation-universality-and-distribution-c3cdb08545c3

[surpher]

UPDATE:

Due to lack of time, this action is pushed out by 2 weeks.