ao_coreaudio: unregister hotplug listener when init fails

[SeanLF]

ao_coreaudio's init() registers the hotplug listener with ao as clientData, then returns CONTROL_ERROR on any later failure without unregistering it. ao_uninit() skips driver->uninit() for a failed init but talloc_free()s the ao regardless, so CoreAudio keeps two listeners pointing at freed memory. mpv falls back to the next driver, so the process stays alive and the next device change enters hotplug_cb() on the freed ao.

ao_coreaudio_exclusive.c already undoes its own listener at the equivalent error label; this does the same.

Common on macOS 26/27, where init_audiounit() fails with -50 for some channel layouts and mpv falls back to ao_avfoundation:

[ao/coreaudio] unable to set the input channel layout on the audio unit ([206][255][255][255]/-50)
[ao] Failed to initialize audio driver 'coreaudio'
AO: [avfoundation] 44100Hz mono 1ch float

Trigger a device change after that and, on an ASan build:

ERROR: AddressSanitizer: heap-use-after-free
READ of size 8 in hotplug_cb
freed by thread T5 here:
    free / ta_free / ao_uninit

Clean with the patch; coreaudio still initialises and plays normally where it did before. Reported as #18274.

Reproduction without audio hardware

Create and destroy an aggregate device to produce a kAudioHardwarePropertyDevices change — no driver install or root needed:

CFMutableDictionaryRef d = CFDictionaryCreateMutable(kCFAllocatorDefault, 0,
    &kCFTypeDictionaryKeyCallBacks, &kCFTypeDictionaryValueCallBacks);
CFDictionarySetValue(d, CFSTR(kAudioAggregateDeviceNameKey), CFSTR("repro"));
CFDictionarySetValue(d, CFSTR(kAudioAggregateDeviceUIDKey), CFSTR("io.mpv.repro"));
AudioObjectID dev;
AudioHardwareCreateAggregateDevice(d, &dev);
AudioHardwareDestroyAggregateDevice(dev);

The device must not be private — a private aggregate device is visible only to the creating process and delivers no cross-process notification.

Play a file with a mono audio track (which makes coreaudio init fail), then run that. On a release build the crash is intermittent rather than reliable: mp_msg_level() dereferences log unconditionally, so it only faults when the freed block happens to be reused and zeroed, which is the null dereference in mp_msg_va from #18274. Under ASan it reproduces every time.

Tested on macOS 27.0 (26A5416b), Apple M4 Pro.

Adjacent issues I left alone

• With --coreaudio-change-physical-format, a failure after init_physical_format() leaves the device on the changed format — p->original_asbd is only restored in uninit(). Same structural gap, one field over.
• p->audio_unit is never set to NULL after AudioComponentInstanceDispose(), so hotplug_cb's if (p->audio_unit) guard can call AudioUnitGetProperty on a disposed instance. Returns an error rather than crashing on macOS 27.
• I checked whether a dispatched callback could still run during unregister_hotplug_cb(). Holding a listener inside a 1500 ms sleep, AudioObjectRemovePropertyListener() called from another thread blocked 1504-1505 ms across three runs and no callback fired after it returned, so I found no race to guard against here. Single-OS observation, not a documented guarantee.

---

Not blocking this PR, just flagging it since it is the underlying cause: the rule that a failed init() never gets uninit() is only implicit in ao_uninit() and ao_init(), and internal.h's comment on .init does not mention it. ao_alsa, ao_pulse, ao_wasapi and ao_coreaudio_exclusive all clean up after themselves; ao_coreaudio was the one that did not. Would you want a note on .init in internal.h, or uninit() called for failed inits too? Happy to send either as a separate PR.

Assisted with Claude Code

[CounterPillow]

why duplicate unregister_hotplug_cb here but not in init?

[kasper93]

Can we instead register callbacks after successful init, just move it down in the init() function?

[CounterPillow]

Okay, judging by these interactions you literally do not understand the code you're submitting, and now we're prompting Claude through you as a proxy. Cool

[SeanLF]

@kasper93 Fair call-out on the process. I applied the fix you requested (register after init)