Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/strands/experimental/bidi/agent/agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
continuous responses including audio output.

Key capabilities:

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mkdocs requires a whitespace before a bulleted list to display it correctly. Otherwise everything is presented on a single line.

- Persistent conversation connections with concurrent processing
- Real-time audio input/output streaming
- Automatic interruption detection and tool execution
Expand Down Expand Up @@ -233,6 +234,7 @@ async def send(self, input_data: BidiAgentInput | dict[str, Any]) -> None:

Args:
input_data: Can be:

- str: Text message from user
- BidiInputEvent: TypedEvent
- dict: Event dictionary (will be reconstructed to TypedEvent)
Expand Down
2 changes: 2 additions & 0 deletions src/strands/experimental/bidi/io/audio.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ def get(self, byte_count: int | None = None) -> bytes:

Args:
byte_count: Number of bytes to get from buffer.

- If the number of bytes specified is not available, the return is padded with silence.
- If the number of bytes is not specified, get the first chunk put in the buffer.

Expand Down Expand Up @@ -274,6 +275,7 @@ def __init__(self, **config: Any) -> None:

Args:
**config: Optional device configuration:

- input_buffer_size (int): Maximum input buffer size (default: None)
- input_device_index (int): Specific input device (default: None = system default)
- input_frames_per_buffer (int): Input buffer size (default: 512)
Expand Down
1 change: 1 addition & 0 deletions src/strands/experimental/bidi/io/text.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ def __init__(self, **config: Any) -> None:

Args:
**config: Optional I/O configurations.

- input_prompt (str): Input prompt to display on screen (default: blank)
"""
self._config = config
Expand Down
4 changes: 4 additions & 0 deletions src/strands/experimental/bidi/models/bidi_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
text, and tool interactions.

Features:

- Persistent connection management with connect/close lifecycle
- Real-time bidirectional communication (send and receive simultaneously)
- Provider-agnostic event normalization
Expand Down Expand Up @@ -96,16 +97,19 @@ async def send(

Args:
content: The content to send. Must be one of:

- BidiTextInputEvent: Text message from the user
- BidiAudioInputEvent: Audio data for speech input
- BidiImageInputEvent: Image data for visual understanding
- ToolResultEvent: Result from a tool execution

Example:
```
await model.send(BidiTextInputEvent(text="Hello", role="user"))
await model.send(BidiAudioInputEvent(audio=bytes, format="pcm", sample_rate=16000, channels=1))
await model.send(BidiImageInputEvent(image=bytes, mime_type="image/jpeg", encoding="raw"))
await model.send(ToolResultEvent(tool_result))
```
"""
...

Expand Down
2 changes: 2 additions & 0 deletions src/strands/experimental/bidi/models/gemini_live.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
official Google GenAI SDK for simplified and robust WebSocket communication.

Key improvements over custom WebSocket implementation:

- Uses official google-genai SDK with native Live API support
- Simplified session management with client.aio.live.connect()
- Built-in tool integration and event handling
Expand Down Expand Up @@ -221,6 +222,7 @@ def _convert_gemini_live_event(self, message: LiveServerMessage) -> list[BidiOut
"""Convert Gemini Live API events to provider-agnostic format.

Handles different types of content:

- inputTranscription: User's speech transcribed to text
- outputTranscription: Model's audio transcribed to text
- modelTurn text: Text response from the model
Expand Down
1 change: 1 addition & 0 deletions src/strands/experimental/bidi/models/novasonic.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
InvokeModelWithBidirectionalStream protocol.

Nova Sonic specifics:

- Hierarchical event sequences: connectionStart → promptStart → content streaming
- Base64-encoded audio format with hex encoding
- Tool execution with content containers and identifier tracking
Expand Down
4 changes: 2 additions & 2 deletions src/strands/experimental/bidi/models/openai.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,8 @@
"""Max timeout before closing connection.

OpenAI documents a 60 minute limit on realtime sessions
(https://platform.openai.com/docs/guides/realtime-conversations#session-lifecycle-events). However, OpenAI does not
emit any warnings when approaching the limit. As a workaround, we configure a max timeout client side to gracefully
([docs](https://platform.openai.com/docs/guides/realtime-conversations#session-lifecycle-events)). However, OpenAI does
not emit any warnings when approaching the limit. As a workaround, we configure a max timeout client side to gracefully
handle the connection closure. We set the max to 50 minutes to provide enough buffer before hitting the real limit.
"""
OPENAI_REALTIME_URL = "wss://api.openai.com/v1/realtime"
Expand Down
6 changes: 5 additions & 1 deletion src/strands/experimental/bidi/types/events.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
capabilities with real-time audio and persistent connection support.

Key features:

- Audio input/output events with standardized formats
- Interruption detection and handling
- Connection lifecycle management
Expand All @@ -12,6 +13,7 @@
- JSON-serializable events (audio/images stored as base64 strings)

Audio format normalization:

- Supports PCM, WAV, Opus, and MP3 formats
- Standardizes sample rates (16kHz, 24kHz, 48kHz)
- Normalizes channel configurations (mono/stereo)
Expand All @@ -29,6 +31,7 @@

AudioChannel = Literal[1, 2]
"""Number of audio channels.

- Mono: 1
- Stereo: 2
"""
Expand Down Expand Up @@ -362,7 +365,6 @@ class BidiInterruptionEvent(TypedEvent):

Parameters:
reason: Why the interruption occurred.
response_id: ID of the response that was interrupted (may be None).
"""

def __init__(self, reason: Literal["user_speech", "error"]):
Expand Down Expand Up @@ -592,6 +594,7 @@ def details(self) -> dict[str, Any] | None:
# BidiInputEvent in send() methods for sending tool results back to the model.

BidiInputEvent = BidiTextInputEvent | BidiAudioInputEvent | BidiImageInputEvent
"""Union of different bidi input event types."""

BidiOutputEvent = (
BidiConnectionStartEvent
Expand All @@ -606,3 +609,4 @@ def details(self) -> dict[str, Any] | None:
| BidiErrorEvent
| ToolUseStreamEvent
)
"""Union of different bidi output event types."""
5 changes: 1 addition & 4 deletions src/strands/experimental/hooks/__init__.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,4 @@
"""Experimental hook functionality that has not yet reached stability.

BidiAgent hooks are also available here to avoid circular imports.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not because of a circular import but because our session manager code references the bidi hook events. Regardless, this is an implementation detail and does not need to be stated in public docs.

"""
"""Experimental hook functionality that has not yet reached stability."""

from .events import (
AfterModelInvocationEvent,
Expand Down
11 changes: 4 additions & 7 deletions src/strands/experimental/hooks/events.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
"""Experimental hook events emitted as part of invoking Agents.
"""Experimental hook events emitted as part of invoking Agents and BidiAgents.

This module defines the events that are emitted as Agents run through the lifecycle of a request.

BidiAgent hook events are also defined here to avoid circular imports.
This module defines the events that are emitted as Agents and BidiAgents run through the lifecycle of a request.
"""

import warnings
Expand All @@ -19,8 +17,8 @@
from ..bidi.models import BidiModelTimeoutError

warnings.warn(
"These events have been moved to production with updated names. Use BeforeModelCallEvent, "
"AfterModelCallEvent, BeforeToolCallEvent, and AfterToolCallEvent from strands.hooks instead.",
"BeforeModelCallEvent, AfterModelCallEvent, BeforeToolCallEvent, and AfterToolCallEvent are no longer experimental."
"Import from strands.hooks instead.",
DeprecationWarning,
stacklevel=2,
)
Expand All @@ -32,7 +30,6 @@


# BidiAgent Hook Events
# These are defined here to avoid circular imports with the bidi package


@dataclass
Expand Down
8 changes: 4 additions & 4 deletions tests/strands/experimental/bidi/models/test_openai.py
Original file line number Diff line number Diff line change
Expand Up @@ -816,7 +816,7 @@ async def test_tool_result_single_text_content(mock_websockets_connect, api_key)
async def test_tool_result_single_json_content(mock_websockets_connect, api_key):
"""Test tool result with single JSON content block."""
_, mock_ws = mock_websockets_connect
model = BidiOpenAIRealtimeModel(api_key=api_key)
model = BidiOpenAIRealtimeModel(client_config={"api_key": api_key})
await model.start()

tool_result: ToolResult = {
Expand Down Expand Up @@ -846,7 +846,7 @@ async def test_tool_result_single_json_content(mock_websockets_connect, api_key)
async def test_tool_result_multiple_content_blocks(mock_websockets_connect, api_key):
"""Test tool result with multiple content blocks (text and json)."""
_, mock_ws = mock_websockets_connect
model = BidiOpenAIRealtimeModel(api_key=api_key)
model = BidiOpenAIRealtimeModel(client_config={"api_key": api_key})
await model.start()

tool_result: ToolResult = {
Expand Down Expand Up @@ -884,7 +884,7 @@ async def test_tool_result_multiple_content_blocks(mock_websockets_connect, api_
async def test_tool_result_image_content_raises_error(mock_websockets_connect, api_key):
"""Test that tool result with image content raises ValueError."""
_, mock_ws = mock_websockets_connect
model = BidiOpenAIRealtimeModel(api_key=api_key)
model = BidiOpenAIRealtimeModel(client_config={"api_key": api_key})
await model.start()

tool_result: ToolResult = {
Expand All @@ -903,7 +903,7 @@ async def test_tool_result_image_content_raises_error(mock_websockets_connect, a
async def test_tool_result_document_content_raises_error(mock_websockets_connect, api_key):
"""Test that tool result with document content raises ValueError."""
_, mock_ws = mock_websockets_connect
model = BidiOpenAIRealtimeModel(api_key=api_key)
model = BidiOpenAIRealtimeModel(client_config={"api_key": api_key})
await model.start()

tool_result: ToolResult = {
Expand Down
2 changes: 1 addition & 1 deletion tests/strands/experimental/hooks/test_hook_aliases.py
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ def test_deprecation_warning_on_import(captured_warnings):

assert len(captured_warnings) == 1
assert issubclass(captured_warnings[0].category, DeprecationWarning)
assert "moved to production with updated names" in str(captured_warnings[0].message)
assert "are no longer experimental" in str(captured_warnings[0].message)


def test_deprecation_warning_on_import_only_for_experimental(captured_warnings):
Expand Down
Loading