Pipecat-ai

Added

- Allow specifying a `DeepgramSTTService` url which allows using on-prem
Deepgram.

- Added new `FastAPIWebsocketTransport`. This is a new websocket transport that
can be integrated with FastAPI websockets.

- Added new `TwilioFrameSerializer`. This is a new serializer that knows how to
serialize and deserialize audio frames from Twilio.

- Added Daily transport event: `on_dialout_answered`. See
https://reference-python.daily.co/api_reference.html#daily.EventHandler

- Added new `AzureSTTService`. This allows you to use Azure Speech-To-Text.

Performance

- Convert `BaseOutputTransport` and `BaseOutputTransport` to fully use asyncio
and remove the use of threads.

Other

- Added `twilio-chatbot`. This is an example that shows how to integrate Twilio
phone numbers with a Pipecat bot.

- Updated `07f-interruptible-azure.py` to use `AzureLLMService`,
`AzureSTTService` and `AzureTTSService`.

Performance

- Break long audio frames into 20ms chunks instead of 10ms.

Added

- Added `report_only_initial_ttfb` to `PipelineParams`. This will make it so
only the initial TTFB metrics after the user stops talking are reported.

- Added `OpenPipeLLMService`. This service will let you run OpenAI through
OpenPipe's SDK.

- Allow specifying frame processors' name through a new `name` constructor
argument.

- Added `DeepgramSTTService`. This service has an ongoing websocket
connection. To handle this, it subclasses `AIService` instead of
`STTService`. The output of this service will be pushed from the same task,
except system frames like `StartFrame`, `CancelFrame` or
`StartInterruptionFrame`.

Changed

- `FrameSerializer.deserialize()` can now return `None` in case it is not
possible to desearialize the given data.

- `daily_rest.DailyRoomProperties` now allows extra unknown parameters.

Fixed

- Fixed an issue where `DailyRoomProperties.exp` always had the same old
timestamp unless set by the user.

- Fixed a couple of issues with `WebsocketServerTransport`. It needed to use
`push_audio_frame()` and also VAD was not working properly.

- Fixed an issue that would cause LLM aggregator to fail with small
`VADParams.stop_secs` values.

- Fixed an issue where `BaseOutputTransport` would send longer audio frames
preventing interruptions.

Other

- Added new `07h-interruptible-openpipe.py` example. This example shows how to
use OpenPipe to run OpenAI LLMs and get the logs stored in OpenPipe.

- Added new `dialin-chatbot` example. This examples shows how to call the bot
using a phone number.

Added

- Added a new `FunctionFilter`. This filter will let you filter frames based on
a given function, except system messages which should never be filtered.

- Added `FrameProcessor.can_generate_metrics()` method to indicate if a
processor can generate metrics. In the future this might get an extra argument
to ask for a specific type of metric.

- Added `BasePipeline`. All pipeline classes should be based on this class. All
subclasses should implement a `processors_with_metrics()` method that returns
a list of all `FrameProcessor`s in the pipeline that can generate metrics.

- Added `enable_metrics` to `PipelineParams`.

- Added `MetricsFrame`. The `MetricsFrame` will report different metrics in the
system. Right now, it can report TTFB (Time To First Byte) values for
different services, that is the time spent between the arrival of a `Frame` to
the processor/service until the first `DataFrame` is pushed downstream. If
metrics are enabled an intial `MetricsFrame` with all the services in the
pipeline will be sent.

- Added TTFB metrics and debug logging for TTS services.

Changed

- Moved `ParallelTask` to `pipecat.pipeline.parallel_task`.

Fixed

- Fixed PlayHT TTS service to work properly async.

Fixed

- Fixed an issue with `SileroVADAnalyzer` that would cause memory to keep
growing indefinitely.

Added

- Added `DailyTransport.participants()` and `DailyTransport.participant_counts()`.