Check tier 3 targets in CI

[jyn514]

NOTE: this issue does not affect the documented target tier policy of the Rust project. We reserve the right to break tier 3 targets at any time, according to the guidelines at https://doc.rust-lang.org/nightly/rustc/target-tier-policy.html#tier-3-target-policy.

Currently, maintaining std for tier 3 targets is very painful for t-libs: std::sys is highly platform dependent, and we have no tier 2 targets that use sys::unsupported so it's hard to tell if it even compiles. To reduce the maintenance burden, I propose we start running x check --target foo for each tier 3 target, to make sure it compiles.

Note that this does not involve distributing any release artifacts; users wanting to target tier 3 platforms will still need to build rustc from source or use -Zbuild-std, but it's more likely the standard library will at least compile, even if we make no guarantees about whether it will run correctly.

We have a few options about how to test this:

1. Run this on PR CI only, which acts as a janky "allow-fail" action.
2. Run this in bors merges, which requires it to pass for the PR to merge. To allow the target to fail to compile, we'd manually disable it in the src/ci test runner file.
3. Add an allow-fail action which runs on bors merges, which doesn't block the merge but still lets us know when the target fails to build. I'm a little hesitant about this because I think it's rare people will actually look at the CI for the master branch, but maybe it's good enough.

I suggest that we wait to decide until the PR fixing these targets, since that will let us know the extent of the current breakage, and gives us a rough estimate of how much effort it will be to keep these compiling (option 2).

I'm very curious to hear @rust-lang/libs's opinion on this, particularly on the testing strategy and whether they think this will increase or decrease the maintenance burden.

cc https://rust-lang.zulipchat.com/#narrow/stream/242791-t-infra/topic/.60x.20check.60.20for.20tier.203.20targets

---

Mentoring instructions:

1. Add a new CI job which runs x check --stage 1 for each tier 3 platform.

   rust/src/doc/rustc/src/platform-support.md

   Lines 211 to 322 in 4a2de63

   	target | std | host | notes
   	-------|:---:|:----:|-------
   	`aarch64-apple-ios-macabi` | ? | | Apple Catalyst on ARM64
   	`aarch64-apple-tvos` | * | | ARM64 tvOS
   	[`aarch64-apple-watchos-sim`](platform-support/apple-watchos.md) | ✓ | | ARM64 Apple WatchOS Simulator
   	[`aarch64-kmc-solid_asp3`](platform-support/kmc-solid.md) | ✓ | | ARM64 SOLID with TOPPERS/ASP3
   	[`aarch64-nintendo-switch-freestanding`](platform-support/aarch64-nintendo-switch-freestanding.md) | * | | ARM64 Nintendo Switch, Horizon
   	[`aarch64-pc-windows-gnullvm`](platform-support/pc-windows-gnullvm.md) | ✓ | ✓ |
   	[`aarch64-unknown-nto-qnx710`](platform-support/nto-qnx.md) | ? | | ARM64 QNX Neutrino 7.1 RTOS |
   	`aarch64-unknown-freebsd` | ✓ | ✓ | ARM64 FreeBSD
   	`aarch64-unknown-hermit` | ✓ | | ARM64 HermitCore
   	`aarch64-unknown-linux-gnu_ilp32` | ✓ | ✓ | ARM64 Linux (ILP32 ABI)
   	`aarch64-unknown-netbsd` | ✓ | ✓ |
   	[`aarch64-unknown-openbsd`](platform-support/openbsd.md) | ✓ | ✓ | ARM64 OpenBSD
   	`aarch64-unknown-redox` | ? | | ARM64 Redox OS
   	`aarch64-uwp-windows-msvc` | ? | |
   	`aarch64-wrs-vxworks` | ? | |
   	`aarch64_be-unknown-linux-gnu_ilp32` | ✓ | ✓ | ARM64 Linux (big-endian, ILP32 ABI)
   	`aarch64_be-unknown-linux-gnu` | ✓ | ✓ | ARM64 Linux (big-endian)
   	[`arm64_32-apple-watchos`](platform-support/apple-watchos.md) | ✓ | | ARM Apple WatchOS 64-bit with 32-bit pointers
   	[`armeb-unknown-linux-gnueabi`](platform-support/armeb-unknown-linux-gnueabi.md) | ✓ | ? | ARM BE8 the default ARM big-endian architecture since [ARMv6](https://developer.arm.com/documentation/101754/0616/armlink-Reference/armlink-Command-line-Options/--be8?lang=en).
   	`armv4t-none-eabi` | * | | ARMv4T A32
   	`armv4t-unknown-linux-gnueabi` | ? | |
   	[`armv5te-none-eabi`](platform-support/armv5te-none-eabi.md) | * | | ARMv5TE A32
   	`armv5te-unknown-linux-uclibceabi` | ? | | ARMv5TE Linux with uClibc
   	`armv6-unknown-freebsd` | ✓ | ✓ | ARMv6 FreeBSD
   	`armv6-unknown-netbsd-eabihf` | ? | |
   	[`armv6k-nintendo-3ds`](platform-support/armv6k-nintendo-3ds.md) | ? | | ARMv6K Nintendo 3DS, Horizon (Requires devkitARM toolchain)
   	`armv7-apple-ios` | ✓ | | ARMv7 iOS, Cortex-a8
   	[`armv7-sony-vita-newlibeabihf`](platform-support/armv7-sony-vita-newlibeabihf.md) | ? | | ARM Cortex-A9 Sony PlayStation Vita (requires VITASDK toolchain)
   	[`armv7-unknown-linux-uclibceabi`](platform-support/armv7-unknown-linux-uclibceabi.md) | ✓ | ✓ | ARMv7 Linux with uClibc, softfloat
   	[`armv7-unknown-linux-uclibceabihf`](platform-support/armv7-unknown-linux-uclibceabihf.md) | ✓ | ? | ARMv7 Linux with uClibc, hardfloat
   	`armv7-unknown-freebsd` | ✓ | ✓ | ARMv7 FreeBSD
   	`armv7-unknown-netbsd-eabihf` | ✓ | ✓ |
   	`armv7-wrs-vxworks-eabihf` | ? | |
   	[`armv7a-kmc-solid_asp3-eabi`](platform-support/kmc-solid.md) | ✓ | | ARM SOLID with TOPPERS/ASP3
   	[`armv7a-kmc-solid_asp3-eabihf`](platform-support/kmc-solid.md) | ✓ | | ARM SOLID with TOPPERS/ASP3, hardfloat
   	`armv7a-none-eabihf` | * | | ARM Cortex-A, hardfloat
   	[`armv7k-apple-watchos`](platform-support/apple-watchos.md) | ✓ | | ARM Apple WatchOS
   	`armv7s-apple-ios` | ✓ | |
   	`avr-unknown-gnu-atmega328` | * | | AVR. Requires `-Z build-std=core`
   	`bpfeb-unknown-none` | * | | BPF (big endian)
   	`bpfel-unknown-none` | * | | BPF (little endian)
   	`hexagon-unknown-linux-musl` | ? | |
   	`i386-apple-ios` | ✓ | | 32-bit x86 iOS
   	`i686-apple-darwin` | ✓ | ✓ | 32-bit macOS (10.7+, Lion+)
   	`i686-pc-windows-msvc` | * | | 32-bit Windows XP support
   	`i686-unknown-haiku` | ✓ | ✓ | 32-bit Haiku
   	`i686-unknown-netbsd` | ✓ | ✓ | NetBSD/i386 with SSE2
   	[`i686-unknown-openbsd`](platform-support/openbsd.md) | ✓ | ✓ | 32-bit OpenBSD
   	`i686-uwp-windows-gnu` | ? | |
   	`i686-uwp-windows-msvc` | ? | |
   	`i686-wrs-vxworks` | ? | |
   	[`m68k-unknown-linux-gnu`](platform-support/m68k-unknown-linux-gnu.md) | ? | | Motorola 680x0 Linux
   	`mips-unknown-linux-uclibc` | ✓ | | MIPS Linux with uClibc
   	[`mips64-openwrt-linux-musl`](platform-support/mips64-openwrt-linux-musl.md) | ? | | MIPS64 for OpenWrt Linux MUSL
   	`mipsel-sony-psp` | * | | MIPS (LE) Sony PlayStation Portable (PSP)
   	[`mipsel-sony-psx`](platform-support/mipsel-sony-psx.md) | * | | MIPS (LE) Sony PlayStation 1 (PSX)
   	`mipsel-unknown-linux-uclibc` | ✓ | | MIPS (LE) Linux with uClibc
   	`mipsel-unknown-none` | * | | Bare MIPS (LE) softfloat
   	`mipsisa32r6-unknown-linux-gnu` | ? | |
   	`mipsisa32r6el-unknown-linux-gnu` | ? | |
   	`mipsisa64r6-unknown-linux-gnuabi64` | ? | |
   	`mipsisa64r6el-unknown-linux-gnuabi64` | ? | |
   	`msp430-none-elf` | * | | 16-bit MSP430 microcontrollers
   	`powerpc-unknown-linux-gnuspe` | ✓ | | PowerPC SPE Linux
   	`powerpc-unknown-linux-musl` | ? | |
   	`powerpc-unknown-netbsd` | ✓ | ✓ |
   	`powerpc-unknown-openbsd` | ? | |
   	`powerpc-wrs-vxworks-spe` | ? | |
   	`powerpc-wrs-vxworks` | ? | |
   	`powerpc64-unknown-freebsd` | ✓ | ✓ | PPC64 FreeBSD (ELFv1 and ELFv2)
   	`powerpc64le-unknown-freebsd` | | | PPC64LE FreeBSD
   	`powerpc-unknown-freebsd` | | | PowerPC FreeBSD
   	`powerpc64-unknown-linux-musl` | ? | |
   	`powerpc64-wrs-vxworks` | ? | |
   	`powerpc64le-unknown-linux-musl` | ? | |
   	[`powerpc64-unknown-openbsd`](platform-support/openbsd.md) | ✓ | ✓ | OpenBSD/powerpc64
   	`powerpc64-ibm-aix` | ? | | 64-bit AIX (7.2 and newer)
   	`riscv32gc-unknown-linux-gnu` | | | RISC-V Linux (kernel 5.4, glibc 2.33)
   	`riscv32gc-unknown-linux-musl` | | | RISC-V Linux (kernel 5.4, musl + RISCV32 support patches)
   	`riscv32im-unknown-none-elf` | * | | Bare RISC-V (RV32IM ISA)
   	[`riscv32imac-unknown-xous-elf`](platform-support/riscv32imac-unknown-xous-elf.md) | ? | | RISC-V Xous (RV32IMAC ISA)
   	`riscv32imc-esp-espidf` | ✓ | | RISC-V ESP-IDF
   	`riscv64gc-unknown-freebsd` | | | RISC-V FreeBSD
   	`riscv64gc-unknown-linux-musl` | | | RISC-V Linux (kernel 4.20, musl 1.2.0)
   	[`riscv64gc-unknown-openbsd`](platform-support/openbsd.md) | ✓ | ✓ | OpenBSD/riscv64
   	`s390x-unknown-linux-musl` | | | S390x Linux (kernel 3.2, MUSL)
   	`sparc-unknown-linux-gnu` | ✓ | | 32-bit SPARC Linux
   	`sparc64-unknown-netbsd` | ✓ | ✓ | NetBSD/sparc64
   	[`sparc64-unknown-openbsd`](platform-support/openbsd.md) | ✓ | ✓ | OpenBSD/sparc64
   	`thumbv4t-none-eabi` | * | | ARMv4T T32
   	[`thumbv5te-none-eabi`](platform-support/armv5te-none-eabi.md) | * | | ARMv5TE T32
   	`thumbv7a-pc-windows-msvc` | ? | |
   	`thumbv7a-uwp-windows-msvc` | ✓ | |
   	`thumbv7neon-unknown-linux-musleabihf` | ? | | Thumb2-mode ARMv7a Linux with NEON, MUSL
   	[`wasm64-unknown-unknown`](platform-support/wasm64-unknown-unknown.md) | ? | | WebAssembly
   	`x86_64-apple-ios-macabi` | ✓ | | Apple Catalyst on x86_64
   	`x86_64-apple-tvos` | * | | x86 64-bit tvOS
   	[`x86_64-apple-watchos-sim`](platform-support/apple-watchos.md) | ✓ | | x86 64-bit Apple WatchOS simulator
   	[`x86_64-pc-nto-qnx710`](platform-support/nto-qnx.md) | ? | | x86 64-bit QNX Neutrino 7.1 RTOS |
   	[`x86_64-pc-windows-gnullvm`](platform-support/pc-windows-gnullvm.md) | ✓ | ✓ |
   	`x86_64-pc-windows-msvc` | * | | 64-bit Windows XP support
   	`x86_64-sun-solaris` | ? | | Deprecated target for 64-bit Solaris 10/11, illumos
   	`x86_64-unknown-dragonfly` | ✓ | ✓ | 64-bit DragonFlyBSD
   	`x86_64-unknown-haiku` | ✓ | ✓ | 64-bit Haiku
   	`x86_64-unknown-hermit` | ✓ | | HermitCore
   	`x86_64-unknown-l4re-uclibc` | ? | |
   	[`x86_64-unknown-openbsd`](platform-support/openbsd.md) | ✓ | ✓ | 64-bit OpenBSD
   	`x86_64-uwp-windows-gnu` | ✓ | |
   	`x86_64-uwp-windows-msvc` | ✓ | |
   	`x86_64-wrs-vxworks` | ? | |

   should be up to date; feel free to post here or on zulip if there are targets that are supported by the compiler but not in that document.
   1a. maybe we should add a tool to keep those in sync?
2. Put up a PR so the CI job runs. Fix compile errors as appropriate. Some targets may not compile on all hosts (e.g. iOS will likely need a macOS host, any MSVC target will need an MSVC host); either find a way to add support or add more runners so all targets are covered. Try to avoid spending too much time on any single target; if it takes too long it's ok to skip it for now. If it ends up being an ongoing issue to support a specific target we may want to consider asking one of the target maintainers (and if they're unresponsive, or there are no maintainers, making a t-compiler MCP to remove it).

cc @joboet, I think you said you were working on making it easier to cross-check targets from any host.

[the8472]

Add an allow-fail action which runs on bors merges, which doesn't block the merge but still lets us know when the target fails to build. I'm a little hesitant about this because I think it's rare people will actually look at the CI for the master branch, but maybe it's good enough.

Maybe that could be added to the bors comment that happens when tests finish?

[cuviper]

Non-blocking feedback comments will need to propagate back to PRs that were rolled up too. That will need some state tracking, so it doesn't keep nagging if a tier-3 target stays broken for a while.

[jyn514]

If we're going to do state tracking then IMO we should go all the way to the existing toolstate infrastructure, so we're not reinventing it from scratch.

[m-ou-se]

This was discussed two weeks ago in the library team meeting. There was a 3:2 split between

a. "just add the tier 3 checks as regular CI checks, and document that manually disabling tier 3 checks is fine", and
b. "use something like the existing toolstate tracking (that doesn't block CI)".

(See Zulip.)

I voted for the second option, but maybe we should just try the first option to see how that works out. Maybe the kind of breakage that happens for those targets will be simple and small enough to not be an issue. (And if it does become an issue, we can change it in the future.)

The first option does probably become quite annoying if we don't have a good way to run those checks manually though. So I think we need local ./x.py check would "Just Work™" when cross compiling for all tier 3 targets, and/or there needs to be a way to ask bors to run those checks, or those checks should always run on PRs directly (or at least those that touch the standard library).

[thomcc]

The first option does probably become quite annoying if we don't have a good way to run those checks manually though.

How many (if any) targets currently supriously fail to check the parts of the stdlib they support¹ if you deinit the llvm submodule? I think doing so fixes all of the failures I knew of (because we no longer try to build the compiler-rt code in compiler-builtins), but there might be some weird ones I don't know about.

Obviously even if the answer is "all of them", that's not a good long term solution (it's annoying and non-obvious to have to deinit it).

Footnotes

1. E.g. obviously no_std-only targets should only check core (or core+alloc if they support alloc). ↩

[jyn514]

The first option does probably become quite annoying if we don't have a good way to run those checks manually though. So I think we need local ./x.py check would "Just Work™" when cross compiling for all tier 3 targets, and/or there needs to be a way to ask bors to run those checks, or those checks should always run on PRs directly (or at least those that touch the standard library).

this should be the case once #102579 is merged

[Noratrieb]

I think this issue needs data on when breakage actually happens, and I happen to have all the data regarding tier 3 breakage within the last two years. See the second section on https://does-it-build.noratrieb.dev. It also includes detailed build logs if you click on a build.