sound

sound#

Description#

Plugin to interact with a sound device.

Among the other features, enabling this plugin allows you to stream audio over HTTP from the device where the application is running, if platypush.backend.http.HttpBackend is enabled.

Simply open http://<host>:<backend-port>/sound/stream.[mp3|wav|acc|ogg]?token=<your-token> to access live recording from the audio device.

It can also be used as a general-purpose audio player and synthesizer, supporting both local and remote audio resources, as well as a MIDI-like interface through the play() command.

Configuration#

sound:
  # [Optional]
  # Index or name of the default input device. Use
  # `SoundPlugin.status <https://docs.platypush.tech/platypush/plugins/sound.html#platypush.plugins.sound.SoundPlugin.status>`_ to get the available devices. Default: system
  # default
  # input_device:   # type=Union[int, str, NoneType]

  # [Optional]
  # Index or name of the default output device.
  # Use `SoundPlugin.status <https://docs.platypush.tech/platypush/plugins/sound.html#platypush.plugins.sound.SoundPlugin.status>`_ to get the available devices. Default:
  # system default
  # output_device:   # type=Union[int, str, NoneType]

  # [Optional]
  # Blocksize to be applied to the input device.
  # Try to increase this value if you get input overflow errors while
  # recording. Default: 1024
  # input_blocksize: 1024  # type=int

  # [Optional]
  # Blocksize to be applied to the output device.
  # Try to increase this value if you get output underflow errors while
  # playing. Default: 1024
  # output_blocksize: 1024  # type=int

  # [Optional]
  # When running in synth mode, this is the maximum
  # number of generated audio frames that will be queued before the
  # audio device processes them (default: 100).
  # queue_size: 10  # type=Optional[int]

  # [Optional]
  # Path of the ``ffmpeg`` binary (default: search for
  # the ``ffmpeg`` in the ``PATH``).
  # ffmpeg_bin: ffmpeg  # type=str

  # [Optional]
  # How often the `RunnablePlugin.loop <https://docs.platypush.tech/platypush/plugins/.html#platypush.plugins.RunnablePlugin.loop>`_ function should be
  # executed (default: 15 seconds). *NOTE*: For back-compatibility
  # reasons, the `poll_seconds` argument is also supported, but it's
  # deprecated.
  # poll_interval: 15  # type=Optional[float]

  # [Optional]
  # How long we should wait for any running
  # threads/processes to stop before exiting (default: 5 seconds).
  # stop_timeout: 5  # type=Optional[float]

  # [Optional]
  # If set to True then the plugin will not monitor
  # for new events. This is useful if you want to run a plugin in
  # stateless mode and only leverage its actions, without triggering any
  # events. Defaults to False.
  # disable_monitor: False  # type=bool

Dependencies#

pip

pip install sounddevice numpy

Alpine

apk add ffmpeg portaudio-dev py3-numpy

Debian

apt install ffmpeg portaudio19-dev python3-numpy

Fedora

yum install ffmpeg python-numpy portaudio-devel

Arch Linux

pacman -S ffmpeg python-numpy python-sounddevice portaudio

Triggered events#

Actions#

Module reference#

class platypush.plugins.sound.SoundPlugin(input_device: int | str | None = None, output_device: int | str | None = None, input_blocksize: int = 1024, output_blocksize: int = 1024, queue_size: int | None = 10, ffmpeg_bin: str = 'ffmpeg', **kwargs)[source]#

Bases: RunnablePlugin

Plugin to interact with a sound device.

Among the other features, enabling this plugin allows you to stream audio over HTTP from the device where the application is running, if platypush.backend.http.HttpBackend is enabled.

Simply open http://<host>:<backend-port>/sound/stream.[mp3|wav|acc|ogg]?token=<your-token> to access live recording from the audio device.

It can also be used as a general-purpose audio player and synthesizer, supporting both local and remote audio resources, as well as a MIDI-like interface through the play() command.

__init__(input_device: int | str | None = None, output_device: int | str | None = None, input_blocksize: int = 1024, output_blocksize: int = 1024, queue_size: int | None = 10, ffmpeg_bin: str = 'ffmpeg', **kwargs)[source]#
Parameters:

  • input_device – Index or name of the default input device. Use status() to get the available devices. Default: system default

  • output_device – Index or name of the default output device. Use status() to get the available devices. Default: system default

  • input_blocksize – Blocksize to be applied to the input device. Try to increase this value if you get input overflow errors while recording. Default: 1024

  • output_blocksize – Blocksize to be applied to the output device. Try to increase this value if you get output underflow errors while playing. Default: 1024

  • queue_size – When running in synth mode, this is the maximum number of generated audio frames that will be queued before the audio device processes them (default: 100).

  • ffmpeg_bin – Path of the ffmpeg binary (default: search for the ffmpeg in the PATH).

main()[source]#

Implementation of the main loop of the plugin.

pause_playback(device: int | str | None = None, streams: Iterable[str | int] | None = None)[source]#
Parameters:

  • device – Only stop the streams on the specified device, by name or index (default: all).

  • streams – Streams to stop by index or name (default: all).

pause_recording(device: int | str | None = None)[source]#

Toggle the recording pause state on the selected device (default: default input device), if it is running.

If paused, the recording will be resumed. If running, it will be paused. Otherwise, no action will be taken.

play(resource: str | None = None, file: str | None = None, sound: dict | Iterable[dict] | None = None, device: int | str | None = None, duration: float | None = None, blocksize: int | None = None, sample_rate: int | None = None, channels: int = 2, volume: float = 100, dtype: str | None = None, format: str | None = None, stream_name: str | None = None, stream_index: int | None = None, join: bool = False)[source]#

Plays an audio file/URL (any audio format supported by ffmpeg works) or a synthetic sound.

Parameters:

  • resource – Audio resource to be played. It can be a local file or a URL.

  • file – Deprecated. Use resource instead.

  • sound –

    Sound to play. Specify this if you want to play synthetic sounds. You can also create polyphonic sounds by just calling play multiple times. Frequencies can be specified either by midi_note - either as strings (e.g. A4) or integers (e.g. 69) - or by frequency (e.g. 440 for 440 Hz). You can also specify a list of sounds here if you want to apply multiple harmonics on a base sound. Some examples:

    {
    "frequency": 440,  # 440 Hz
    "volume":    100,  # Maximum volume
    "duration":  1.0   # 1 second or until stop_playback
}

    [
    {
        "midi_note": "A4", # A @ 440 Hz
        "volume":    100,  # Maximum volume
        "duration":  1.0   # 1 second or until stop_playback
    },
    {
        "midi_note": "E5", # Play the harmonic one fifth up
        "volume":    25,   # 1/4 of the volume
        "duration":  1.0   # 1 second or until stop_playback
        "phase":     3.14  # ~180 degrees phase
        # Make it a triangular wave (default: sin).
        # Supported types: "sin", "triang", "square",
        # "sawtooth"
        "shape:      "triang"
    }
]

    [
    {
        "midi_note": "C4", # C4 MIDI note
        "duration":  0.5   # 0.5 seconds or until stop_playback
    },
    {
        "midi_note": "G5", # G5 MIDI note
        "duration":  0.5,  # 0.5 seconds or until stop_playback
        "delay":     0.5   # Start this note 0.5 seconds
                           # after playback has started
    }
]

  • device – Output device (default: default configured device or system default audio output if not configured)

  • duration – Playback duration, in seconds. Default: None - play until the end of the audio source or until stop_playback().

  • blocksize – Audio block size (default: configured output_blocksize or 2048)

  • sample_rate – Audio sample rate. Default: audio file sample rate if in file mode, 44100 Hz if in synth mode

  • channels – Number of audio channels. Default: number of channels in the audio file in file mode, 1 if in synth mode

  • volume – Playback volume, between 0 and 100. Default: 100.

  • dtype – Data type for the audio samples, if playing raw PCM audio frames. Supported types: ‘float64’, ‘float32’, ‘int32’, ‘int16’, ‘int8’, ‘uint8’.

  • format – Output audio format, if you want to convert the audio to another format before playing it. The list of available formats can be retrieved through the ffmpeg -formats command. Default: None

  • stream_index – If specified, play to an already active stream index (you can get them through query_streams()). Default: creates a new audio stream through PortAudio.

  • stream_name – Name of the stream to play to. If set, the sound will be played to the specified stream name, or a stream with that name will be created. If not set, and stream_index is not set either, then a new stream will be created on the next available index and named platypush-stream-<index>.

  • join – If True, then the method will block until the playback is completed. Default: False.

query_streams()[source]#

Deprecated alias for status().

record(device: int | str | None = None, output_device: int | str | None = None, fifo: str | None = None, outfile: str | None = None, duration: float | None = None, sample_rate: int | None = None, dtype: str = 'int16', blocksize: int | None = None, latency: float | str = 'high', channels: int = 1, volume: float = 100, redis_queue: str | None = None, format: str = 'wav', stream: bool = True, stream_name: str | None = None, play_audio: bool = False)[source]#

Return audio data from an audio source

Parameters:

  • device – Input device (default: default configured device or system default audio input if not configured)

  • output_device – Audio output device if play_audio=True (audio pass-through) (default: default configured device or system default audio output if not configured)

  • fifo – Path of a FIFO that will be used to exchange audio frames with other consumers.

  • outfile – If specified, the audio data will be persisted on the specified audio file too.

  • duration – Recording duration in seconds (default: record until stop event)

  • sample_rate – Recording sample rate (default: device default rate)

  • dtype – Data type for the audio samples. Supported types: ‘float64’, ‘float32’, ‘int32’, ‘int16’, ‘int8’, ‘uint8’. Default: int16.

  • blocksize – Audio block size (default: configured input_blocksize or 2048)

  • play_audio – If True, then the recorded audio will be played in real-time on the selected output_device (default: False).

  • latency – Device latency in seconds (default: the device’s default high latency)

  • channels – Number of channels (default: 1)

  • volume – Recording volume, between 0 and 100. Default: 100.

  • redis_queue – If set, the audio chunks will also be published to this Redis channel, so other consumers can process them downstream.

  • format – Audio format. Supported: wav, mp3, ogg, aac, flac. Default: wav.

  • stream – If True (default), then the audio will be streamed to an HTTP endpoint too (default: /sound/stream<.format>).

  • stream_name – Custom name for the output stream.

recordplay(*args, **kwargs)[source]#

Deprecated alias for record().

set_volume(volume: float, device: int | str | None = None, streams: Iterable[str | int] | None = None)[source]#

Set the audio input/output volume.

Parameters:

  • volume – New volume, between 0 and 100.

  • device – Set the volume only on the specified device (default: all).

  • streams – Set the volume only on the specified list of stream indices/names (default: all).

start()#

Start the plugin.

status() → List[dict][source]#
Returns:

The current status of the audio devices and streams.

Example:

[
  {
    "streams": [
      {
        "device": 3,
        "direction": "output",
        "outfile": "/dev/null",
        "infile": "/mnt/hd/media/music/audio.mp3",
        "ffmpeg_bin": "ffmpeg",
        "channels": 2,
        "sample_rate": 44100,
        "dtype": "int16",
        "streaming": false,
        "duration": null,
        "blocksize": 1024,
        "latency": "high",
        "redis_queue": "platypush-stream-AudioResourcePlayer-3",
        "audio_pass_through": false,
        "state": "PAUSED",
        "started_time": "2023-06-19T11:57:05.882329",
        "stream_index": 1,
        "stream_name": "platypush:audio:output:1"
      }
    ],
    "index": 3,
    "name": "default",
    "hostapi": 0,
    "max_input_channels": 32,
    "max_output_channels": 32,
    "default_samplerate": 44100,
    "default_low_input_latency": 0.008707482993197279,
    "default_low_output_latency": 0.008707482993197279,
    "default_high_input_latency": 0.034829931972789115,
    "default_high_output_latency": 0.034829931972789115
  }
]
stop()#

Stop the plugin.

stop_playback(device: int | str | None = None, streams: Iterable[str | int] | None = None)[source]#
Parameters:

  • device – Only stop the streams on the specified device, by name or index (default: all).

  • streams – Streams to stop by index or name (default: all).

stop_recording(device: int | str | None = None, timeout: float | None = 2)[source]#

Stop the current recording process on the selected device (default: default input device), if it is running.

stream_recording(*args, **kwargs)[source]#

Deprecated alias for record().

wait_stop(timeout=None)#

Wait until a stop event is received.