[prototype] Two-pass R8 JNI name rewriting before ILLink and ILC

[simonrozsival]

Context

Advances #12535.

.NET for Android currently prevents R8 name obfuscation because managed bindings embed Java/JNI class, method, field, and descriptor names. If R8 renames the Java side while those managed values remain unchanged, direct JNI lookup and native registration fail.

This draft prototypes a build-time solution with no runtime lookup table: R8 first establishes the Java names, managed assemblies are rewritten to those names before ILLink or NativeAOT ILC, and final R8 shrinking reuses the same mapping.

Mono.Cecil is intentionally not used. Managed PE rewriting is implemented with System.Reflection.Metadata/System.Reflection.Metadata.Ecma335.

How the design evolved

The initial investigation considered generating short names ourselves and supplying them to R8, or carrying an original-to-obfuscated lookup table into the app. A runtime table would support already-compiled bindings but would add packaged data, startup work, and lookup overhead to JNI operations. Letting R8 remain the naming authority also handles its actual collision and optimization behavior.

The first working prototype ran after final R8 and rewrote assemblies immediately before packaging. It proved the PE rewriter and produced a runnable MAUI app with a 2.40% signed-APK reduction, but it was CoreCLR-specific and had to repair compressed-assembly size metadata after rewriting.

The design in this PR now follows the backend-neutral pipeline discussed in #12535:

1. Generate the pre-trim trimmable typemap, Java callable wrappers, manifest, and registration sources.
2. Compile the complete Java input set for a naming pass.
3. Run R8 with minification enabled but tree shaking and optimization disabled, producing a seed mapping.txt.
4. Rewrite managed inputs into a staging directory using that mapping.
5. Feed the rewritten assemblies to ILLink or NativeAOT ILC.
6. Generate final managed-reachability ProGuard rules using original Java names plus allowobfuscation.
7. Run final R8 over the original Java inputs with normal shrinking/optimization and -applymapping.
8. Package the resulting DEX and optimized managed output normally.

Moving the rewrite before ILLink/ILC removes the old post-R8 compressed-assembly workaround and lets typemap generation and both managed backends consume the same obfuscated JNI names.

Implementation

Naming-only R8 pass

R8 now supports a seed-mapping mode that:

• enables minification;
• disables tree shaking and optimization;
• writes a mapping file;
• accepts the same built-in, AAPT, user, and library naming constraints when available;
• pins generated ACWs whose names are still referenced by manifests/resources;
• can remove the global -dontobfuscate rule when JNI rewriting is enabled.

The naming pass uses a dedicated Javac invocation. Reusing _CompileJava caused a target-cycle through RID-specific _ResolveAssemblies, so the prototype compiles the generated Java sources and application JAR inputs without entering that graph.

Managed assembly staging

RewriteJniNamesForR8 can rewrite an item set into a destination directory while preserving MSBuild metadata. It uses content-aware output writes so unchanged staged files retain timestamps.

For CoreCLR, staged assemblies replace both ManagedAssemblyToLink and the corresponding ResolvedFileToPublish entries before _RunILLink. Replacing both is required because ILLink removes publish inputs by item identity; retaining the originals causes duplicate publish outputs.

For NativeAOT, staged assemblies replace ManagedBinary, IlcCompileInput, and IlcReference before the ILC response file is written. Multi-RID inner builds are serialized while this prototype is enabled so they share one seed mapping safely.

Mapping-aware final keep rules

GenerateProguardConfiguration receives the seed mapping and reverses rewritten Java class/member names back to their original names. It emits -keep,allowobfuscation and -keepclassmembers,allowobfuscation, allowing final R8 to retain managed-reachable members without overriding -applymapping.

Constructor handling accepts both managed .ctor and rewritten <init> spellings and emits valid <init>(...); syntax.

R8Mapping now provides O(1) reverse class lookup and descriptor-aware reverse method lookup for overloaded methods.

NativeAOT's prebuilt host is one deliberate exception to full remapping: it looks up mono.android.Runtime and its static fields by fixed JNI names from native code. Those literals cannot be rewritten in managed IL, so the small runtime bridge class and its members remain preserved while application and binding symbols are still obfuscated.

PE rewrite coverage

The existing rewriter handles:

• RegisterAttribute type/member names;
• JNI signature attributes;
• JniPeerMembers identifiers;
• RegisterNatives data;
• direct JNI class/member lookup strings;
• JNI descriptors, including embedded parameter and return types;
• null-terminated UTF-8 values stored in FieldRVA data;
• owner-specific shared strings;
• arbitrary replacement lengths through complete PE reconstruction;
• metadata token, resource, exception-region, PDB identity, and FieldRVA preservation;
• byte-identical no-op assemblies.

Rewritten strong-named assemblies are currently left delay-signed because the build step does not have a signing key. That remains productization work.

Validation

CoreCLR

A Release dotnet new maui --sample-content app was built with:

• net11.0-android;
• android-arm64;
• CoreCLR;
• trimmable typemap;
• R8 shrinking/optimization;
• the two-pass JNI rewrite enabled.

The signed APK built successfully, installed on an API 35 arm64 emulator, cold-launched its generated activity, rendered the sample-content UI, remained alive, and produced no JNI, managed, Java, or native fatal errors.

Seed/final mapping comparison found no surviving ordinary class-name disagreements. The remaining class differences are R8 removed/synthetic optimization artifacts; managed-reachable keep rules reuse the seed names.

Focused host tests pass: 45/45 across R8Tests, R8MappingTests, and RewriteJniNamesForR8Tests.

Measured CoreCLR results:

Metric	Baseline	Two-pass	Delta
Signed APK	15,681,455 bytes	15,394,735 bytes	-286,720 bytes (-1.83%)
Clean/rebuild wall time	63.37 s	64.41 s	+1.04 s
Settled no-op build	4.26 s	4.02 s	effectively unchanged

The first no-op build after a rebuild can still rerun expensive work (38.15 s observed); complete target Inputs/Outputs remain follow-up work.

NativeAOT

NativeAOT was validated end-to-end with the same MAUI sample, android-arm64, Release, trimmable typemap, R8, and PublishAot=true. The local Android workload was tested with the available matching preview-7 .NET SDK/ILCompiler/Android NativeAOT framework pack because the checkout's RC compiler packs were not available on the configured feed.

Both baseline and two-pass APKs built from clean obj/bin trees, installed on an API 35 arm64 emulator, cold-launched, and remained alive after 12 seconds. The two-pass process remained alive as PID 3625 with no fatal, JNI lookup, NoSuchMethodError, NoSuchFieldError, or native loader errors.

The first two-pass runtime attempt found a real NativeAOT-specific issue: R8 renamed mono.android.Runtime.mono_android_GCUserPeer, while the prebuilt native host still requested that literal field name. Preserving the runtime bridge class fixed the failure; the clean rebuilt APK then launched successfully.

Metric	Baseline	Two-pass	Delta
Signed APK	19,490,367 bytes	19,138,111 bytes	-352,256 bytes (-1.81%)
Clean build wall time	54.40 s	85.73 s	+31.33 s (+57.6%)

Binary-log task timings (elapsed task time, not additive wall-clock time because MSBuild work can be nested or overlap):

Stage	Baseline	Two-pass	Delta
Typemap generation	2.893 s	3.292 s	+0.399 s
Seed Java compilation	—	1.508 s	+1.508 s
Naming-only R8	—	16.964 s	+16.964 s
Managed assembly rewriting	—	2.390 s	+2.390 s
ILC	28.300 s	31.856 s	+3.556 s
Final Java compilation	1.451 s	1.781 s	+0.330 s
Final R8	7.457 s	10.550 s	+3.093 s

The naming pass is the dominant added cost. This prototype favors maximum compression and correctness over Release build speed, as intended.

Managed IL and typemap size analysis

The size investigation exposed and fixed a correctness gap in the original prototype. FieldRVA typemap strings were already rewritten, but assembly-level TypeMapAttribute<T> keys and JavaPeerAliasesAttribute arrays were not. NativeAOT consumes that metadata to construct its type map, so leaving the original Java names there was not merely a missed size optimization: it could make the NativeAOT type map disagree with R8-renamed Java classes. The rewriter now recognizes generic attribute constructors encoded through a TypeSpecification, rewrites indexed keys such as java/util/Collection[0] while preserving their suffix, and rewrites peer-alias arrays.

A direct production-task measurement covered all 86 generated typemap assemblies and 13,031 real TypeMapAttribute<T> entries, with a representative short-name mapping for all 12,855 distinct Java classes:

Typemap assembly metric	Before	Rewritten	Delta
Raw file size	10,798,592 bytes	9,781,760 bytes	-1,016,832 bytes (-9.42%)
Gzip-equivalent size	1,624,571 bytes	1,517,125 bytes	-107,446 bytes (-6.61%)
Metadata size	—	—	-967,604 bytes
.text virtual size	—	—	-1,018,520 bytes

_Mono.Android.TypeMap.dll alone became 514,048 bytes smaller. The representative names were generally longer than aggressive R8 names, so this demonstrates a measurable, conservative IL/metadata reduction rather than forecasting an exact final APK delta.

Ordinary binding assemblies do not show a comparable PE-size reduction. Their JNI #US heap shrank by 108 bytes in the measured set, while reconstructed custom-attribute/blob layout grew slightly. An apparent ~758 KB reduction was almost entirely the rewriter removing Authenticode certificate data; after excluding certificates and PE file-alignment effects, those PEs grew by roughly 2 KB overall. That certificate removal must not be credited to shorter JNI strings.

NativeAOT does not package these assemblies as IL: ILC compiles them into the application native library. The previously measured 1.81% NativeAOT APK reduction came almost entirely from classes.dex (-636,720 bytes uncompressed, -355,248 bytes compressed); the application native library grew by 416 bytes uncompressed. A fresh end-to-end APK measurement with the corrected typemap attributes is still pending because the prototype clean-build seed-Javac graph does not yet populate the correctly classified AAR/JAR inputs early enough. The focused R8/JNI suite, including a structurally faithful generic typemap fixture, passes 66/66.

Rejected optimization direction: We considered replacing the naming-only R8 pass with a managed deterministic name allocator that would emit mapping.txt directly. Although collision avoidance and user-rule handling are tractable, correct member naming also requires reproducing R8 knowledge about inheritance, interface dispatch, library overrides, and other minifier constraints. That would duplicate a substantial and correctness-critical part of R8. The prototype therefore keeps R8 as the naming authority; this idea is not planned for prototyping beyond documenting the tradeoff.

Current limitations / follow-up work

• Make early AAPT/resource-derived naming constraints reliable on a completely clean build without re-entering _ResolveAssemblies.
• Add full target-level Inputs/Outputs around staged rewriting; task outputs already use content-aware writes.
• Define the production strong-name re-signing strategy.
• Add multi-ABI and clean/incremental integration coverage.
• Decide the servicing/GA opt-in surface; the prototype remains behind $(_AndroidEnableR8JniNameRewriting).
• ACWs referenced from manifests/resources remain pinned until those surfaces are rewritten or receive deterministic generated short names.