LibreSSL XCFramework
30/09/20 21:21
Time for a little behind-the-scenes:
Many frameworks used for iOS apps, including the database clients I make, depend on OpenSSL. LibreSSL is fork of OpenSSL that remains API compatible (meaning you can use LibreSSL in place of OpenSSL), but has a lot taken out and tries to keep what is left in more secure. For many of those frameworks, LibreSSL is both sufficient and preferable. But whichever you choose, bringing them into an app built on Apple's ecosystems can be hard. There are a bunch of scripts out there, some ancient, some well maintained, and today I want to talk about the script that I've been putting my efforts into: build-xcframeworks/libressl
The script was born out of Annunziato Tocci's workflowproducts/libpq-ios. This script builds libpq, which depends on libressl. The problem I was having at the time I came across it was that I could not get it to build for macOS Catalyst. I was very fortunate that Annunziato was generous with his time, and interested in making it work too. With his insights into what was needed for the compiles, together we made it build for more and more architectures. A big thank you also to everyone on the Apple Developer Forums who helped with my questions.
This is where the story usually ends for the scripts, and then the iOS project could make multiple targets and lipo together the different architectures needed. (lipo is a command line tool for working on universal files, that I tend to use as a verb) And while I'd already gone down this route for other libraries, there are many pains involved in that, since you cannot lipo together builds for the iOS simulator, macOS and/or macOS Catalyst. From lipos perspective, they're all x86_64. So having them in different combinations becomes very burdensome with multiple targets and multiple libraries to keep in sync.
Now, the title gives it all away, but at WWDC 2019 Apple introduced XCFrameworks. The purpose of it is to make one library definition that suits all targets and architectures. Now, hold on for a sec, what's the difference between a target and an architecture? Well, the three x86_64 libraries above were all the same architecture, but different targets: iOS simulator, macOS and macOS Catalyst. On the other hand, iOS for a long while was one target but different architectures: armv6, armv7, armv7s and arm64. These days, I'm sorry to say, but I only care about arm64 and arm64e. More on the last one later. But in short, compiling libressl as an XCFramework would make it way less complicated to include it in my apps. After all, my apps currently target iPhone, iPad, the simulators, and macOS Catalyst. So I can take away the head-ache of three different architectures to manage.
Now, there's one part of XCFrameworks I did not know coming into this. You see, I was under the impression that it would take a library compiled for different architectures and different targets and nicely put it together. This seemed true for Intel only, but add Apple Silicon, and it became obvious that this was not true. It only supports taking in different targets. This means that before handing the build binaries to xcodebuild to make the xcframework, we needed to lipo the libraries of the same target but different architectures together. With them in place, we could make the xcframework, and you have the finished build of our repo, you can find the binaries here.
Of course there is one more catch: the remaining relevant architectures (for now) are arm64, arm64e, x86_64 and x86_64h. If you are running macOS 11 with Xcode installed and run
file /Applications/Xcode.app/Contents/MacOS/Xcode
you will see
/Applications/Xcode.app/Contents/MacOS/Xcode (for architecture x86_64): Mach-O 64-bit executable x86_64
/Applications/Xcode.app/Contents/MacOS/Xcode (for architecture arm64): Mach-O 64-bit executable arm64
At first glance that looks fine. But, the x86_64h was introduced with Haswell, and Haswell CPUs debuted on the Macs in 2013. arm64e was introduced on the A12 chip that runs the Apple Silicon DTKs, and it's safe to assume the next Apple Silicon Macs won't be a step back. So really, I expected to focus on arm64e and x86_64h, but the Xcode binary suggests otherwise. Having asked around, it seems we should keep focussing on x86_64 and arm64. Why not do both? Because, when trying to add them together in the same XCFramework I'd often get A library with the identifier "" already exists . So having to choose between the two variants, I was encouraged to follow Apple's lead.
So there you have it, LibreSSL built as an XCFramework, matching version 3.0.2, 3.1.4 and 3.2.1, can now easily be built from the script (or downloaded as binary releases), supporting arm64 on iOS, and x86_64 and arm64 on the iOS Simulator, macOS and macOS Catalyst. For my part, I'll use it for my apps MySQL Commander for macOS, iPad and iPhone, MSSQL Commander for macOS, iPad and iPhone, PostgreSQL Commander for macOS (coming soon), iPad and iPhone, and Graph Gopher. It's already simplified my work there a lot. I'll also use it to contribute to other open source projects that depend on LibreSSL, such as the aforementioned libpq and libssh2. I hope others will also use it for other libraries, helping to bring them over to the platforms. I wish that it would be be more easy to include an XCFramework dependency when building libraries configured with configure, but that's a story for another day. Like I mentioned at the top, LibreSSL is a dependency for or can replace the OpenSSL dependency for many open and closed source projects, so I hope our efforts will be of good use to you and in your projects, either directly or transiently through the dependencies you'll include.
Many frameworks used for iOS apps, including the database clients I make, depend on OpenSSL. LibreSSL is fork of OpenSSL that remains API compatible (meaning you can use LibreSSL in place of OpenSSL), but has a lot taken out and tries to keep what is left in more secure. For many of those frameworks, LibreSSL is both sufficient and preferable. But whichever you choose, bringing them into an app built on Apple's ecosystems can be hard. There are a bunch of scripts out there, some ancient, some well maintained, and today I want to talk about the script that I've been putting my efforts into: build-xcframeworks/libressl
The script was born out of Annunziato Tocci's workflowproducts/libpq-ios. This script builds libpq, which depends on libressl. The problem I was having at the time I came across it was that I could not get it to build for macOS Catalyst. I was very fortunate that Annunziato was generous with his time, and interested in making it work too. With his insights into what was needed for the compiles, together we made it build for more and more architectures. A big thank you also to everyone on the Apple Developer Forums who helped with my questions.
This is where the story usually ends for the scripts, and then the iOS project could make multiple targets and lipo together the different architectures needed. (lipo is a command line tool for working on universal files, that I tend to use as a verb) And while I'd already gone down this route for other libraries, there are many pains involved in that, since you cannot lipo together builds for the iOS simulator, macOS and/or macOS Catalyst. From lipos perspective, they're all x86_64. So having them in different combinations becomes very burdensome with multiple targets and multiple libraries to keep in sync.
Now, the title gives it all away, but at WWDC 2019 Apple introduced XCFrameworks. The purpose of it is to make one library definition that suits all targets and architectures. Now, hold on for a sec, what's the difference between a target and an architecture? Well, the three x86_64 libraries above were all the same architecture, but different targets: iOS simulator, macOS and macOS Catalyst. On the other hand, iOS for a long while was one target but different architectures: armv6, armv7, armv7s and arm64. These days, I'm sorry to say, but I only care about arm64 and arm64e. More on the last one later. But in short, compiling libressl as an XCFramework would make it way less complicated to include it in my apps. After all, my apps currently target iPhone, iPad, the simulators, and macOS Catalyst. So I can take away the head-ache of three different architectures to manage.
Now, there's one part of XCFrameworks I did not know coming into this. You see, I was under the impression that it would take a library compiled for different architectures and different targets and nicely put it together. This seemed true for Intel only, but add Apple Silicon, and it became obvious that this was not true. It only supports taking in different targets. This means that before handing the build binaries to xcodebuild to make the xcframework, we needed to lipo the libraries of the same target but different architectures together. With them in place, we could make the xcframework, and you have the finished build of our repo, you can find the binaries here.
Of course there is one more catch: the remaining relevant architectures (for now) are arm64, arm64e, x86_64 and x86_64h. If you are running macOS 11 with Xcode installed and run
file /Applications/Xcode.app/Contents/MacOS/Xcode
you will see
/Applications/Xcode.app/Contents/MacOS/Xcode (for architecture x86_64): Mach-O 64-bit executable x86_64
/Applications/Xcode.app/Contents/MacOS/Xcode (for architecture arm64): Mach-O 64-bit executable arm64
At first glance that looks fine. But, the x86_64h was introduced with Haswell, and Haswell CPUs debuted on the Macs in 2013. arm64e was introduced on the A12 chip that runs the Apple Silicon DTKs, and it's safe to assume the next Apple Silicon Macs won't be a step back. So really, I expected to focus on arm64e and x86_64h, but the Xcode binary suggests otherwise. Having asked around, it seems we should keep focussing on x86_64 and arm64. Why not do both? Because, when trying to add them together in the same XCFramework I'd often get A library with the identifier "
So there you have it, LibreSSL built as an XCFramework, matching version 3.0.2, 3.1.4 and 3.2.1, can now easily be built from the script (or downloaded as binary releases), supporting arm64 on iOS, and x86_64 and arm64 on the iOS Simulator, macOS and macOS Catalyst. For my part, I'll use it for my apps MySQL Commander for macOS, iPad and iPhone, MSSQL Commander for macOS, iPad and iPhone, PostgreSQL Commander for macOS (coming soon), iPad and iPhone, and Graph Gopher. It's already simplified my work there a lot. I'll also use it to contribute to other open source projects that depend on LibreSSL, such as the aforementioned libpq and libssh2. I hope others will also use it for other libraries, helping to bring them over to the platforms. I wish that it would be be more easy to include an XCFramework dependency when building libraries configured with configure, but that's a story for another day. Like I mentioned at the top, LibreSSL is a dependency for or can replace the OpenSSL dependency for many open and closed source projects, so I hope our efforts will be of good use to you and in your projects, either directly or transiently through the dependencies you'll include.
Creating a connection to MySQL
07/09/20 16:53
Let's create a regular connection to your MySQL database. For this, your MySQL database needs to be reachable via the internet, and you need to have its hostname, port-number, username, password and database name.
When I write hostname here, that usually looks like a string with dots between it, like "mydatabase.mydomain.com". Other times, you might have an IP address, like "192.168.0.10", for instance when connecting to a server on your local network. In this document I'll use the word hostname to refer to either.
When you launch MySQL Commander for iPad, tap the button called "Create new Connection"
Figure 1: The app launched, with the "Create new Connection" in the bottom left
Figure 2: The "Add Connection" screen.
Then let's choose a Title, by tapping the right-hand side of the Title row. Unless you have an external keyboard attached, the iPad keyboard should appear. If you have an external keyboard attached, you can see the cursor blinking on the far right. You can now type the desired connection title.
After that, repeat for the rows "Hostname or IP address", "Database", "Username" and "Password". If your port-number is not the default 3306, edit it to the correct value too. If you have an external keyboard, you can use the tab key to jump between the fields. Otherwise, just tap the row you want to edit.
We're not going to enable an SSH tunnel for now, so leave it disabled.
Now that you've entered all the information, test the connection by tapping the button "Test Connection". A dialog "Verifying connection" will appear, probably very briefly. If it stays up for long, that could indicate either a poor internet connection, or that either the hostname or port is invalid. In either case, you'll get to know.
Figure 3: An example connection. Encircled you find the "Test Connection" button.
Assuming that you've entered the correct data, the dialog disappears, and that means everything validated correctly. You can save the connection by tapping the confirm button in the top right corner. When you tap confirm, MySQL Commander will briefly validate the connection. Then you will see the main screen again, now populated with your new MySQL connection. Tap it, and you'll be connected to it, and ready to explore it and do whatever you need done to your database.
When I write hostname here, that usually looks like a string with dots between it, like "mydatabase.mydomain.com". Other times, you might have an IP address, like "192.168.0.10", for instance when connecting to a server on your local network. In this document I'll use the word hostname to refer to either.
When you launch MySQL Commander for iPad, tap the button called "Create new Connection"
Figure 1: The app launched, with the "Create new Connection" in the bottom left
Figure 2: The "Add Connection" screen.
Then let's choose a Title, by tapping the right-hand side of the Title row. Unless you have an external keyboard attached, the iPad keyboard should appear. If you have an external keyboard attached, you can see the cursor blinking on the far right. You can now type the desired connection title.
After that, repeat for the rows "Hostname or IP address", "Database", "Username" and "Password". If your port-number is not the default 3306, edit it to the correct value too. If you have an external keyboard, you can use the tab key to jump between the fields. Otherwise, just tap the row you want to edit.
We're not going to enable an SSH tunnel for now, so leave it disabled.
Now that you've entered all the information, test the connection by tapping the button "Test Connection". A dialog "Verifying connection" will appear, probably very briefly. If it stays up for long, that could indicate either a poor internet connection, or that either the hostname or port is invalid. In either case, you'll get to know.
Figure 3: An example connection. Encircled you find the "Test Connection" button.
Assuming that you've entered the correct data, the dialog disappears, and that means everything validated correctly. You can save the connection by tapping the confirm button in the top right corner. When you tap confirm, MySQL Commander will briefly validate the connection. Then you will see the main screen again, now populated with your new MySQL connection. Tap it, and you'll be connected to it, and ready to explore it and do whatever you need done to your database.
Vapor with Theo - Getting Started
16/07/20 17:53
Theo, the Neo4j Swift driver, works well with Vapor 4.
For future reference: This post was written with Theo 5.2.0, Vapor 4.15.2 and Swift 5.2.4
Let's have a look at how to use it together, starting with the crudest example:
In the code above, we start with the theoClient function that in it's completion block provides a Bolt client. Bolt is the binary protocol used to talk to a Neo4j instance. I've configured it to connect to the instance on my machine at port 7687, the default Bolt port. I provide a username and password, and ask it to connect via SSL, a configuration you may know from the Neo4j browser as bolt+s.
For this example, I've written a router with only one endpoint and implemented it straight away. When a request is made, we make a promise to Vapor that we'll return a string. Then we ask for a Theo client, and define a new node we want to insert. The node contains a single label: "New", and a single property "createdTime", with the current time as a TimeInterval (a.k.a. Double).
We ask the client to create it, and if it is successful, we fulfil our promise saying "Newly created node!". If, however, there is a problem, we complete with a failure and report the error.
For future reference: This post was written with Theo 5.2.0, Vapor 4.15.2 and Swift 5.2.4
Let's have a look at how to use it together, starting with the crudest example:
In the code above, we start with the theoClient function that in it's completion block provides a Bolt client. Bolt is the binary protocol used to talk to a Neo4j instance. I've configured it to connect to the instance on my machine at port 7687, the default Bolt port. I provide a username and password, and ask it to connect via SSL, a configuration you may know from the Neo4j browser as bolt+s.
For this example, I've written a router with only one endpoint and implemented it straight away. When a request is made, we make a promise to Vapor that we'll return a string. Then we ask for a Theo client, and define a new node we want to insert. The node contains a single label: "New", and a single property "createdTime", with the current time as a TimeInterval (a.k.a. Double).
We ask the client to create it, and if it is successful, we fulfil our promise saying "Newly created node!". If, however, there is a problem, we complete with a failure and report the error.