Repository navigation
Support Apple app store bitcode #35968
Description
Bitcode is the future of Apple's app distribution and we don't support it now. Doing so is tricky because Apple choses whatever rev of LLVM they want and upgrade it per their whims. We can't couple our LLVM to theirs because we have our own needs and can't be forced to upgrade whenever Apple decides.
Here are some options:
- We ship an entirely separate toolchain for targetting Apple bitcode. This is quite ugly.
- We change rustc to dynamically load LLVM, create a rust-llvm package and an optional rust-llvm-apple package, and have rustc load the apple LLVM to deal with their bitcode.
- We create a rustc_llvm_apple crate and just link two entire LLVMs into rustc at all times. This probably would not fly
- We create a build of rustc called rustc-apple and ship it in an optional rustc-apple package. It is exactly like rustc except instead of linking to rustc_llvm it links to rustc_llvm_apple. When rustc gets a request to emit bitcode it just defers completely to the rustc-apple binary. I actually fancy this solution quite a bit.
- We could investigate a bitcode sanitizer to translate our bitcode to theirs. I think the likelihood of success here is low because maintenance. I believe RenderScript does this.
I think the LLVM dynamic loading and defer-to-alternate-rustc solutions are most promising.
cc https://users.rust-lang.org/t/ios-rust-integration/6928/4
cc @bluejekyll
Activity
nagisa commentedon Aug 24, 2016
Dynamically loading LLVM is certainly not an option. The LLVM API is less stable than its bitcode and, if we build against LLVM x.y, then having LLVM x.z instead of x.y will almost certainly break rustc. Just link to the system LLVM statically (like
--llvm-rootalready does).Just link the system LLVM (statically) for apple distributions. In my experience it already works kinda well.
Okay… so, the thing is that LLVM bitcode format is kinda stable between versions of LLVM. Does apple do something unusual to make that not true, or is the bitcode format used not LLVM altogether (i.e. has apple specific stuff)?
I feel like I heard all the way back in 2013-ish that using LLVM bitcode as distribution format is stupid. I certainly agree; does distributing native binary libraries not work anymore for iOS?
nagisa commentedon Aug 24, 2016
I’m also very interested in what would happen when Apple upgrades to LLVM version with incompatible bitcode format internally and people still compile stuff with old compilers.
bluejekyll commentedon Aug 24, 2016
I experimented briefly with this, i.e. output bitcode during cargo builds, then tried to create a static library with the bitcode included. When I tried to llvm-link against the bitcode library, I got an incompatible LLVM version from the bitcode.
I don't have an easy test to reproduce this, but my guess is there is just a dumb version check that denies linking between different LLVM versions? Or, I just did something entirely wrong. I will try to come up with a test case when I have some time to look at this again.
sanxiyn commentedon Aug 24, 2016
It would be useful to have exact error message from llvm-link.
nagisa commentedon Aug 24, 2016
@bluejekyll LLVM has a number of bitcode files in its test directory. Tests against these bitcode files are run continuously (llvm-dis-3.8/opt-3.8 understands a 3 year old bitcode file from 3.2 just fine, for example), so it must be some apple stuff.
ricky26 commentedon Aug 24, 2016
This is somewhat easier said than done - as far as I'm aware, the only blessed apple-llvm version for App Store uploads is the one shipped with the current Xcode. This also potentially means maintaining the LLVM bindings for two LLVM versions (not necessarily two neighbouring minor versions, either). I don't think it's valid to just use an old version of apple-llvm.
I think they avoid this by only allowing the latest Xcode to submit apps (and I'm pretty sure it's burned into the output image which LLVM version you used).
IIRC, the bitcode is packaged for each architecture, since cross-architecture bitcode isn't generated by clang (I think that's an anti-goal of clang and bitcode in general). It's stored in a section for each object file so at the very least it's duplicated. That could be part of why someone mentioned that bitcode might not be the best way of going about this.
I feel a bit like all of the recommended solutions are somewhat icky. The least messy way I can think of is to allow different targets to overload the codegen behaviour and have the Apple codegen path be in a dynamic crate. (Which would just be the regular codegen path compiled against apple-llvm.)
Since this bug mentions the App Store, is it worth talking about the exception handling story here? (i.e.
panic=abortis strictly required at the moment.)bluejekyll commentedon Aug 24, 2016
@nagisa thanks for letting me know this. It gives me hope that there might still be a solution here, and that I was probably doing something wrong.
nagisa commentedon Aug 24, 2016
@ricky26 good points.
Isn’t Xcode LLVM on apple the same thing as system LLVM? I meant the Xcode LLVM then. We would have to make sure xcode is always the most recent version when shipping rustc trains.
Of course the way apple does things rules out us producing a valid apple bitcode with an old version of rustc and essentially forces us throw all the benefits our stability story provides out the window, and I don’t see any way this could be fixed.
We already maintain¹ support for LLVM versions 3.7 through 3.9 (and potentially trunk). As long as Xcode’s LLVM is not some ancient version, I think we’re good in that regard. If Xcode LLVM is really some ancient/custom/etc version, then I don’t see us being able to support this feature at all, then. Especially since we have no option of sending patches to that LLVM in order to add the features we need. I also wouldn’t want to lock rustc into supporting 3.7 forever in case Apple decided to not update Xcode LLVM until 2038.
¹: however, if rustc was built against LLVM x.y, it must be linked to LLVM x.y exactly.
92 remaining items
Narniab commentedon Sep 13, 2020
-Cembed-bitcode=no does not work for ios target. there are still bitcode section in .a file
how to build without bitcode?
fzyzcjy commentedon Oct 13, 2021
Hi, after years, is there any updates? Thanks! In other words, is there anyone who successfully make an app with Rust to deploy to apple store?
extrawurst commentedon Oct 13, 2021
We have an app using rust In the stores. No Bitcode required unless you want to support Apple Watch so far
fzyzcjy commentedon Oct 13, 2021
@extrawurst Thank you very much! So if I understand correctly: I just need to disable bitcode, and everything would be fine?
thombles commentedon Oct 13, 2021
If you do want the bitcode for some reason or another, the scripts we maintain at $work continue to function for building iOS apps with bitcode and submitting to the app store, up to Xcode 13. More broadly though I'm not aware of any changes regarding Apple's forks of LLVMs and the difficulty of making this work with mainline Rust development.
fzyzcjy commentedon Oct 13, 2021
@thombles Thank you!
saks commentedon Mar 4, 2022
I've encountered an issue that looks similar:
I'm pretty sure that bitcode is disabled, since it works if I use rust 1.55 (before upgrade to LLVM 12) and xcode project is configured to disable bitcode.
I'd appreciate if anyone can suggest how to debug this further. Thanks!
bernaferrari commentedon Jun 7, 2022
Not anymore 😛
https://developer.apple.com/documentation/xcode-release-notes/xcode-14-release-notes
jzxchiang1 commentedon Jun 18, 2022
Does that mean we no longer need to use a specialized toolchain like https://github.com/getditto/rust-bitcode, and can instead safely disable bitcode in our XCode app?
bernaferrari commentedon Jun 18, 2022
Yes.
workingjubilee commentedon Jul 6, 2022
@thombles makes a good point here:
I'd also take "Xcode 15 is announced", on the assumption that if anything goes awry we'll hear about it by then, but if anyone wants to reap this issue before then, they're welcome to it.
jfro commentedon Jul 12, 2022
fwiw we currently ship macOS (non-store) & iOS (TestFlight & Store) with a Rust library driving most of our core functionality. We've never used bitcode since we didn't target any of the previously bitcode requiring platforms.
We'll be shipping an update in the near future that'll also be built with 14 I believe (think they're accepting 14 builds now...)
thombles commentedon Sep 22, 2022
No surprises really, but I've confirmed that the standard Rust toolchain is working fine all the way to App Store validation using Xcode 14. I've added a notice to
rust-bitcodeto advise users it's no longer needed. Dare we... close this? :)nagisa commentedon Nov 4, 2022
Closing as per the comment above.