LibreSSL XCFramework
30/09/20 21:21
Time for a little behind-the-scenes:
Many frameworks used for iOS apps, including the database clients I make, depend on OpenSSL. LibreSSL is fork of OpenSSL that remains API compatible (meaning you can use LibreSSL in place of OpenSSL), but has a lot taken out and tries to keep what is left in more secure. For many of those frameworks, LibreSSL is both sufficient and preferable. But whichever you choose, bringing them into an app built on Apple's ecosystems can be hard. There are a bunch of scripts out there, some ancient, some well maintained, and today I want to talk about the script that I've been putting my efforts into: build-xcframeworks/libressl
The script was born out of Annunziato Tocci's workflowproducts/libpq-ios. This script builds libpq, which depends on libressl. The problem I was having at the time I came across it was that I could not get it to build for macOS Catalyst. I was very fortunate that Annunziato was generous with his time, and interested in making it work too. With his insights into what was needed for the compiles, together we made it build for more and more architectures. A big thank you also to everyone on the Apple Developer Forums who helped with my questions.
This is where the story usually ends for the scripts, and then the iOS project could make multiple targets and lipo together the different architectures needed. (lipo is a command line tool for working on universal files, that I tend to use as a verb) And while I'd already gone down this route for other libraries, there are many pains involved in that, since you cannot lipo together builds for the iOS simulator, macOS and/or macOS Catalyst. From lipos perspective, they're all x86_64. So having them in different combinations becomes very burdensome with multiple targets and multiple libraries to keep in sync.
Now, the title gives it all away, but at WWDC 2019 Apple introduced XCFrameworks. The purpose of it is to make one library definition that suits all targets and architectures. Now, hold on for a sec, what's the difference between a target and an architecture? Well, the three x86_64 libraries above were all the same architecture, but different targets: iOS simulator, macOS and macOS Catalyst. On the other hand, iOS for a long while was one target but different architectures: armv6, armv7, armv7s and arm64. These days, I'm sorry to say, but I only care about arm64 and arm64e. More on the last one later. But in short, compiling libressl as an XCFramework would make it way less complicated to include it in my apps. After all, my apps currently target iPhone, iPad, the simulators, and macOS Catalyst. So I can take away the head-ache of three different architectures to manage.
Now, there's one part of XCFrameworks I did not know coming into this. You see, I was under the impression that it would take a library compiled for different architectures and different targets and nicely put it together. This seemed true for Intel only, but add Apple Silicon, and it became obvious that this was not true. It only supports taking in different targets. This means that before handing the build binaries to xcodebuild to make the xcframework, we needed to lipo the libraries of the same target but different architectures together. With them in place, we could make the xcframework, and you have the finished build of our repo, you can find the binaries here.
Of course there is one more catch: the remaining relevant architectures (for now) are arm64, arm64e, x86_64 and x86_64h. If you are running macOS 11 with Xcode installed and run
file /Applications/Xcode.app/Contents/MacOS/Xcode
you will see
/Applications/Xcode.app/Contents/MacOS/Xcode (for architecture x86_64): Mach-O 64-bit executable x86_64
/Applications/Xcode.app/Contents/MacOS/Xcode (for architecture arm64): Mach-O 64-bit executable arm64
At first glance that looks fine. But, the x86_64h was introduced with Haswell, and Haswell CPUs debuted on the Macs in 2013. arm64e was introduced on the A12 chip that runs the Apple Silicon DTKs, and it's safe to assume the next Apple Silicon Macs won't be a step back. So really, I expected to focus on arm64e and x86_64h, but the Xcode binary suggests otherwise. Having asked around, it seems we should keep focussing on x86_64 and arm64. Why not do both? Because, when trying to add them together in the same XCFramework I'd often get A library with the identifier "" already exists . So having to choose between the two variants, I was encouraged to follow Apple's lead.
So there you have it, LibreSSL built as an XCFramework, matching version 3.0.2, 3.1.4 and 3.2.1, can now easily be built from the script (or downloaded as binary releases), supporting arm64 on iOS, and x86_64 and arm64 on the iOS Simulator, macOS and macOS Catalyst. For my part, I'll use it for my apps MySQL Commander for macOS, iPad and iPhone, MSSQL Commander for macOS, iPad and iPhone, PostgreSQL Commander for macOS (coming soon), iPad and iPhone, and Graph Gopher. It's already simplified my work there a lot. I'll also use it to contribute to other open source projects that depend on LibreSSL, such as the aforementioned libpq and libssh2. I hope others will also use it for other libraries, helping to bring them over to the platforms. I wish that it would be be more easy to include an XCFramework dependency when building libraries configured with configure, but that's a story for another day. Like I mentioned at the top, LibreSSL is a dependency for or can replace the OpenSSL dependency for many open and closed source projects, so I hope our efforts will be of good use to you and in your projects, either directly or transiently through the dependencies you'll include.
Many frameworks used for iOS apps, including the database clients I make, depend on OpenSSL. LibreSSL is fork of OpenSSL that remains API compatible (meaning you can use LibreSSL in place of OpenSSL), but has a lot taken out and tries to keep what is left in more secure. For many of those frameworks, LibreSSL is both sufficient and preferable. But whichever you choose, bringing them into an app built on Apple's ecosystems can be hard. There are a bunch of scripts out there, some ancient, some well maintained, and today I want to talk about the script that I've been putting my efforts into: build-xcframeworks/libressl
The script was born out of Annunziato Tocci's workflowproducts/libpq-ios. This script builds libpq, which depends on libressl. The problem I was having at the time I came across it was that I could not get it to build for macOS Catalyst. I was very fortunate that Annunziato was generous with his time, and interested in making it work too. With his insights into what was needed for the compiles, together we made it build for more and more architectures. A big thank you also to everyone on the Apple Developer Forums who helped with my questions.
This is where the story usually ends for the scripts, and then the iOS project could make multiple targets and lipo together the different architectures needed. (lipo is a command line tool for working on universal files, that I tend to use as a verb) And while I'd already gone down this route for other libraries, there are many pains involved in that, since you cannot lipo together builds for the iOS simulator, macOS and/or macOS Catalyst. From lipos perspective, they're all x86_64. So having them in different combinations becomes very burdensome with multiple targets and multiple libraries to keep in sync.
Now, the title gives it all away, but at WWDC 2019 Apple introduced XCFrameworks. The purpose of it is to make one library definition that suits all targets and architectures. Now, hold on for a sec, what's the difference between a target and an architecture? Well, the three x86_64 libraries above were all the same architecture, but different targets: iOS simulator, macOS and macOS Catalyst. On the other hand, iOS for a long while was one target but different architectures: armv6, armv7, armv7s and arm64. These days, I'm sorry to say, but I only care about arm64 and arm64e. More on the last one later. But in short, compiling libressl as an XCFramework would make it way less complicated to include it in my apps. After all, my apps currently target iPhone, iPad, the simulators, and macOS Catalyst. So I can take away the head-ache of three different architectures to manage.
Now, there's one part of XCFrameworks I did not know coming into this. You see, I was under the impression that it would take a library compiled for different architectures and different targets and nicely put it together. This seemed true for Intel only, but add Apple Silicon, and it became obvious that this was not true. It only supports taking in different targets. This means that before handing the build binaries to xcodebuild to make the xcframework, we needed to lipo the libraries of the same target but different architectures together. With them in place, we could make the xcframework, and you have the finished build of our repo, you can find the binaries here.
Of course there is one more catch: the remaining relevant architectures (for now) are arm64, arm64e, x86_64 and x86_64h. If you are running macOS 11 with Xcode installed and run
file /Applications/Xcode.app/Contents/MacOS/Xcode
you will see
/Applications/Xcode.app/Contents/MacOS/Xcode (for architecture x86_64): Mach-O 64-bit executable x86_64
/Applications/Xcode.app/Contents/MacOS/Xcode (for architecture arm64): Mach-O 64-bit executable arm64
At first glance that looks fine. But, the x86_64h was introduced with Haswell, and Haswell CPUs debuted on the Macs in 2013. arm64e was introduced on the A12 chip that runs the Apple Silicon DTKs, and it's safe to assume the next Apple Silicon Macs won't be a step back. So really, I expected to focus on arm64e and x86_64h, but the Xcode binary suggests otherwise. Having asked around, it seems we should keep focussing on x86_64 and arm64. Why not do both? Because, when trying to add them together in the same XCFramework I'd often get A library with the identifier "
So there you have it, LibreSSL built as an XCFramework, matching version 3.0.2, 3.1.4 and 3.2.1, can now easily be built from the script (or downloaded as binary releases), supporting arm64 on iOS, and x86_64 and arm64 on the iOS Simulator, macOS and macOS Catalyst. For my part, I'll use it for my apps MySQL Commander for macOS, iPad and iPhone, MSSQL Commander for macOS, iPad and iPhone, PostgreSQL Commander for macOS (coming soon), iPad and iPhone, and Graph Gopher. It's already simplified my work there a lot. I'll also use it to contribute to other open source projects that depend on LibreSSL, such as the aforementioned libpq and libssh2. I hope others will also use it for other libraries, helping to bring them over to the platforms. I wish that it would be be more easy to include an XCFramework dependency when building libraries configured with configure, but that's a story for another day. Like I mentioned at the top, LibreSSL is a dependency for or can replace the OpenSSL dependency for many open and closed source projects, so I hope our efforts will be of good use to you and in your projects, either directly or transiently through the dependencies you'll include.