SDL3: Mixer changes, what to do?

[Starbuck5]

As many of you may know, SDL3_mixer features substantial changes over SDL2_mixer. See https://wiki.libsdl.org/SDL3_mixer/README-migration. It has yet to be formally released, and also relies on an unreleased SDL version.

I've spent quite a bit of time this weekend going through the API and writing a stub file that exposes all SDL3_mixer functionality the way I would do it with a fresh port. I also had to do the same for SDL3's audio module, because SDL3_mixer relies on input/output with audio types like formats, specs, and devices, to various degrees.

mixer2.pyi

from collections.abc import Callable
from typing import Type, TypedDict, TypeVar

import _audio as audio
from pygame.typing import FileLike
from typing_extensions import Buffer

def init() -> None: ...
def quit() -> None: ...
def get_sdl_mixer_version(linked: bool = True) -> tuple[int, int, int]: ...
def ms_to_frames(sample_rate: int, ms: int) -> int: ...
def frames_to_ms(sample_rate: int, frames: int) -> int: ...
def get_decoders() -> list[str]: ...

T = TypeVar("T")
track_stopped_callback = Callable[[T, Track], None]
track_mix_callback = Callable[[T, Track, audio.AudioSpec, Buffer], None]
group_mix_callback = Callable[[T, Group, audio.AudioSpec, Buffer], None]
post_mix_callback = Callable[[T, Mixer, audio.AudioSpec, Buffer], None]

class Mixer:
    def __init__(
        self,
        device: audio.AudioDevice = audio.DEFAULT_PLAYBACK_DEVICE,
        spec: audio.AudioSpec | None = None,
    ) -> None: ...
    @property
    def gain(self) -> float: ...
    @gain.setter
    def gain(self, value: float): ...
    def play_tag(
        self,
        tag: str,
        loops: int = 0,
        max_ms: int = -1,
        start_ms: int = 0,
        loop_start_ms: int = 0,
        fadein_ms: int = 0,
        append_silence_ms: int = 0,
    ) -> None: ...
    def stop_tag(self, tag: str, fade_out_ms: int = 0) -> None: ...
    def pause_tag(self, tag: str) -> None: ...
    def resume_tag(self, tag: str) -> None: ...
    def set_tag_gain(self, tag: str, gain: float) -> None: ...
    def play_audio(self, audio: Audio) -> None: ...
    def stop_all_tracks(self, fade_out_ms: int = 0) -> None: ...
    def pause_all_tracks(self) -> None: ...
    def resume_all_tracks(self) -> None: ...
    @property
    def format(self) -> audio.AudioSpec: ...
    def set_post_mix_callback(
        self, callback: post_mix_callback | None, userdata: T
    ) -> None: ...

class MemoryMixer(Mixer):
    def __init__(self, spec: audio.AudioSpec) -> None: ...
    def generate(self, buffer: Buffer, buflen: int) -> None: ...

class Audio:
    def __init__(
        self,
        file: FileLike,
        predecode: bool = False,
        preferred_mixer: Mixer | None = None,
    ) -> None: ...
    @classmethod
    def from_raw(
        cls, buffer: Buffer, spec: audio.AudioSpec, preferred_mixer: Mixer | None = None
    ) -> Audio: ...
    @classmethod
    def from_sine_wave(
        hz: int,
        amplitude: float,
        preferred_mixer: Mixer | None = None,
    ) -> Audio: ...
    @property
    def duration_frames(self) -> int | None: ...
    @property
    def duration_ms(self) -> int | None: ...
    @property
    def duration_infinite(self) -> bool: ...
    @property
    def format(self) -> audio.AudioSpec: ...
    def ms_to_frames(self, ms: int) -> int: ...
    def frames_to_ms(self, frames: int) -> int: ...
    def get_metadata() -> _AudioMetadataDict: ...

class Group:
    def __init__(self, mixer: Mixer) -> None: ...
    @property
    def mixer(self) -> Mixer: ...
    def set_post_mix_callback(
        self, callback: group_mix_callback | None, userdata: Type[T]
    ) -> None: ...

class Track:
    def __init__(self, mixer: Mixer) -> None: ...
    def set_audio(self, audio: Audio | None) -> None: ...
    def get_audio(self) -> Audio | None: ...
    def set_audiostream(self, audiostream: audio.AudioStream | None) -> None: ...
    def get_audiostream(self) -> audio.AudioStream | None: ...
    def set_filestream(self, file: FileLike) -> None: ...
    def play(
        self,
        loops: int = 0,
        max_frame: int = -1,
        max_ms: int = -1,
        start_frame: int = 0,
        start_ms: int = 0,
        loop_start_frame: int = 0,
        loop_start_ms: int = 0,
        fadein_frames: int = 0,
        fadein_ms: int = 0,
        append_silence_frames: int = 0,
        append_silence_ms: int = 0,
    ) -> None: ...
    @property
    def mixer(self) -> Mixer: ...
    def add_tag(self, tag: str) -> None: ...
    def remove_tag(self, tag: str) -> None: ...
    def set_group(self, group: Group | None) -> None: ...
    def set_playback_position(self, frames: int) -> None: ...
    def get_playback_position(self) -> int: ...
    def get_remaining_frames(self) -> int | None: ...
    def ms_to_frames(self, ms: int) -> int: ...
    def frames_to_ms(self, ms: int) -> int: ...
    def stop(self, fade_out_frames: int = 0) -> None: ...
    def pause(self) -> None: ...
    def resume(self) -> None: ...
    @property
    def playing(self) -> bool: ...
    @property
    def paused(self) -> bool: ...
    @property
    def looping(self) -> bool: ...
    @property
    def gain(self) -> float: ...
    @gain.setter
    def gain(self, value: float): ...
    @property
    def frequency_ratio(self) -> float: ...
    @frequency_ratio.setter
    def frequency_ratio(self, value: float): ...
    def set_output_channel_map(self, channel_map: list[int] | None) -> None: ...
    def set_stereo(self, left_gain: float, right_gain: float) -> None: ...
    def set_3d_position(self, position: tuple[float, float, float]) -> None: ...
    def get_3d_position(self) -> tuple[float, float, float]: ...
    def set_stopped_callback(
        self, callback: track_stopped_callback | None, userdata: T
    ) -> None: ...
    def set_raw_callback(
        self, callback: track_mix_callback | None, userdata: T
    ) -> None: ...

class AudioDecoder:
    def __init__(self, file: FileLike) -> None: ...
    @property
    def format(self) -> audio.AudioSpec: ...
    def decode(buffer: Buffer, spec: audio.AudioSpec) -> int: ...

class _AudioMetadataDict(TypedDict):
    title: str | None
    artist: str | None
    album: str | None
    copyright: str | None
    track: int | None
    total_tracks: int | None

audio.pyi

from collections.abc import Callable
from typing import TypeVar

from pygame.typing import FileLike
from typing_extensions import Buffer

def init() -> None: ...
def quit() -> None: ...
def get_init() -> bool: ...
def get_current_driver() -> str: ...
def get_drivers() -> list[str]: ...
def get_playback_devices() -> list[AudioDevice]: ...
def get_recording_devices() -> list[AudioDevice]: ...
def mix_audio(dst: Buffer, src: Buffer, format: AudioFormat, volume: float) -> None: ...
def load_wav(file: FileLike) -> tuple[AudioSpec, bytes]: ...
def convert_samples(
    src_spec: AudioSpec, src_data: Buffer, dst_spec: AudioSpec
) -> bytes: ...

DEFAULT_PLAYBACK_DEVICE: AudioDevice
DEFAULT_RECORDING_DEVICE: AudioDevice

T = TypeVar("T")
stream_callback = Callable[[T, AudioStream, int, int], None]
post_mix_callback = Callable[[T, AudioStream, Buffer], None]
iteration_callback = Callable[[T, AudioDevice, bool], None]

class AudioFormat:
    @property
    def bitsize(self) -> int: ...
    @property
    def bytesize(self) -> int: ...
    @property
    def is_float(self) -> bool: ...
    @property
    def is_int(self) -> bool: ...
    @property
    def is_big_endian(self) -> bool: ...
    @property
    def is_little_endian(self) -> bool: ...
    @property
    def is_signed(self) -> bool: ...
    @property
    def is_unsigned(self) -> bool: ...
    @property
    def name(self) -> str: ...
    @property
    def silence_value(self) -> bytes: ...
    def __index__(self) -> int: ...
    def __repr__(self) -> str: ...

U8: AudioFormat
S8: AudioFormat
S16LE: AudioFormat
S16BE: AudioFormat
S32LE: AudioFormat
S32BE: AudioFormat
F32LE: AudioFormat
F32BE: AudioFormat
S16: AudioFormat
S32: AudioFormat
F32: AudioFormat

class AudioSpec:
    def __init__(self, format: AudioFormat, channels: int, frequency: int) -> None: ...
    @property
    def format(self) -> AudioFormat: ...
    @property
    def channels(self) -> int: ...
    @property
    def frequency(self) -> int: ...
    @property
    def framesize(self) -> int: ...
    def __repr__(self) -> str: ...

class AudioDevice:
    def open(self, spec: AudioSpec | None = None) -> LogicalAudioDevice: ...
    def open_stream(
        self,
        spec: AudioSpec | None,
        callback: stream_callback | None,
        userdata: T | None,
    ) -> AudioStream: ...
    @property
    def is_playback(self) -> bool: ...
    @property
    def name(self) -> str: ...
    # Need something for https://wiki.libsdl.org/SDL3/SDL_GetAudioDeviceFormat
    @property
    def channel_map(self) -> list[int] | None: ...

class LogicalAudioDevice(AudioDevice):
    def pause(self) -> None: ...
    def resume(self) -> None: ...
    @property
    def paused(self) -> bool: ...
    @property
    def gain(self) -> float: ...
    @gain.setter
    def gain(self, value: float) -> None: ...
    def set_iteration_callbacks(
        self,
        start: iteration_callback | None,
        end: iteration_callback | None,
        userdata: T,
    ) -> None: ...
    def set_post_mix_callback(
        self, callback: post_mix_callback | None, userdata: T
    ) -> None: ...

class AudioStream:
    def __init__(self, src_spec: AudioSpec, dst_spec: AudioSpec) -> None: ...
    def bind(self, device: LogicalAudioDevice) -> None: ...
    def unbind(self) -> None: ...
    def clear(self) -> None: ...
    def flush(self) -> None: ...
    @property
    def num_available_bytes(self) -> int: ...
    @property
    def num_queued_bytes(self) -> int: ...
    def get_data(self, size: int) -> bytes: ...
    def put_data(self, data: Buffer) -> None: ...
    def pause_device(self) -> None: ...
    def resume_device(self) -> None: ...
    @property
    def device_paused(self) -> bool: ...
    @property
    def device(self) -> LogicalAudioDevice | None: ...
    @property
    def src_spec(self) -> AudioSpec: ...
    @src_spec.setter
    def src_spec(self, value: AudioSpec) -> None: ...
    @property
    def dst_spec(self) -> AudioSpec: ...
    @dst_spec.setter
    def dst_spec(self, value: AudioSpec) -> None: ...
    @property
    def gain(self) -> float: ...
    @gain.setter
    def gain(self, value: float) -> None: ...
    @property
    def frequency_ratio(self) -> float: ...
    @frequency_ratio.setter
    def frequency_ratio(self, value: float) -> None: ...
    def set_input_channel_map(self, channel_map: list[int] | None) -> None: ...
    def get_input_channel_map(self) -> list[int] | None: ...
    def set_output_channel_map(self, channel_map: list[int] | None) -> None: ...
    def get_output_channel_map(self) -> list[int] | None: ...
    def lock(self) -> None: ...
    def unlock(self) -> None: ...
    def set_get_callback(
        self, callback: stream_callback | None, userdata: T
    ) -> None: ...
    def set_put_callback(
        self, callback: stream_callback | None, userdata: T
    ) -> None: ...
    def __repr__(self) -> str: ...

I also have these stubs on a branch: 0c12dba

EDIT 2025-12-09 : replaced stubs with newer versions.

Simple code examples I believe would work:

# Broadcast audio to all speakers
mixers: list[pygame.Mixer] = []

for device in pygame.audio.get_playback_devices():
    mixers.append(pygame.Mixer(device))

output_audio = pygame.Audio("test.mp3")

for mixer in mixers:
    mixer.play_audio(output_audio)

# Loop mic input into speaker output
microphone_stream = pygame.audio.DEFAULT_RECORDING_DEVICE.open_stream()

speakers = pygame.mixer.Mixer()
speakers_track = pygame.mixer.Track(speakers)
speakers_track.set_audiostream(microphone_stream)
speakers_track.play()

[Starbuck5]

To summarize some of the changes, the explicit "music" doesn't exist now, such that any Track object could be streaming from a file. Channels don't exist now, but Tracks do... The doc I linked does a way better job of summarizing.

Initially, my plan was to do a fresh port of SDL3_mixer and then put in a compatibility layer in Python. This would allow us to move forward with an interface closer to what SDL natively has, with a best effort compatibility layer for folks to not be too inconvenienced. The docs would have these two parts of the mixer module separated into different pages, even if at runtime it's all in the same namespace. So if you can imagine, the top bar would have "mixer", which would link to mixer2.html, and then mixer2.html would contain a link to mixer.html as the "legacy API." (This would also save old URLs)

But now that I've written it all out maybe it wouldn't be too bad to just natively support something pretty similar to what we've got now. Like when you initialize the module it could just create a mixer for the default speakers, and an array of preallocated Tracks. And then if you squint at them the Audio and Track objects are similar to the existing Sound and Channel objects. We could make it so there's a play() method on Audio that finds a preallocated track on the default mixer. And then we could smooth out the terminology differences like gain vs volume, unpause vs resume, etc. And we'd still be able to expand the functionality, just without abandoning our existing paradigm quite as much. I think mixer.music would still be a Python compat layer though.

[Starbuck5]

Hoping to get some other opinions on this, I know it's a lot of information to sort through!

I think right now I'm back to leaning with fresh-port-full-compat-layer.

[OlaKenji]

I don’t fully understand all the details, but the current music/mixer setup feels quite limited. Any changes that could improve or expand it would be a welcome improvement.

[LeWolfYT]

What happens doesn’t matter too much to me other than wanting AudioStream implemented. I think keeping compatibility would be fine as long as some of the new methods are added.

[Starbuck5]

I've been working on this off and on for a while just put out the first PR on this topic: #3657.

It implements most of the audio module for SDL3.