Griptape

Added

- `Workflow.input_tasks` and `Workflow.output_tasks` to access the input and output tasks of a Workflow.
- Ability to pass nested list of `Tasks` to `Structure.tasks` allowing for more complex declarative Structure definitions.
- `TavilyWebSearchDriver` to integrate Tavily's web search capabilities.
- `ExaWebSearchDriver` to integrate Exa's web search capabilities.
- `Workflow.outputs` to access the outputs of a Workflow.
- `BaseFileLoader` for Loaders that load from a path.
- `BaseLoader.fetch()` method for fetching data from a source.
- `BaseLoader.parse()` method for parsing fetched data.
- `BaseFileManager.encoding` to specify the encoding when loading and saving files.
- `BaseWebScraperDriver.extract_page()` method for extracting data from an already scraped web page.
- `TextLoaderRetrievalRagModule.chunker` for specifying the chunking strategy.
- `file_utils.get_mime_type` utility for getting the MIME type of a file.
- `BaseRulesetDriver` for loading a `Ruleset` from an external source.
- `LocalRulesetDriver` for loading a `Ruleset` from a local `.json` file.
- `GriptapeCloudRulesetDriver` for loading a `Ruleset` resource from Griptape Cloud.
- Parameter `alias` on `GriptapeCloudConversationMemoryDriver` for fetching a Thread by alias.
- Basic support for OpenAi Structured Output via `OpenAiChatPromptDriver.response_format` parameter.
- Ability to pass callable to `activity.schema` for dynamic schema generation.

Changed

- **BREAKING**: Renamed parameters on several classes to `client`:
- `bedrock_client` on `AmazonBedrockCohereEmbeddingDriver`.
- `bedrock_client` on `AmazonBedrockCohereEmbeddingDriver`.
- `bedrock_client` on `AmazonBedrockTitanEmbeddingDriver`.
- `bedrock_client` on `AmazonBedrockImageGenerationDriver`.
- `bedrock_client` on `AmazonBedrockImageQueryDriver`.
- `bedrock_client` on `AmazonBedrockPromptDriver`.
- `sagemaker_client` on `AmazonSageMakerJumpstartEmbeddingDriver`.
- `sagemaker_client` on `AmazonSageMakerJumpstartPromptDriver`.
- `sqs_client` on `AmazonSqsEventListenerDriver`.
- `iotdata_client` on `AwsIotCoreEventListenerDriver`.
- `s3_client` on `AmazonS3FileManagerDriver`.
- `s3_client` on `AwsS3Tool`.
- `iam_client` on `AwsIamTool`.
- `pusher_client` on `PusherEventListenerDriver`.
- `mq` on `MarqoVectorStoreDriver`.
- `model_client` on `GooglePromptDriver`.
- `model_client` on `GoogleTokenizer`.
- **BREAKING**: Renamed parameter `pipe` on `HuggingFacePipelinePromptDriver` to `pipeline`.
- **BREAKING**: Removed `BaseFileManager.default_loader` and `BaseFileManager.loaders`.
- **BREAKING**: Loaders no longer chunk data, use a Chunker to chunk the data.
- **BREAKING**: Removed `fileutils.load_file` and `fileutils.load_files`.
- **BREAKING**: Removed `loaders-dataframe` and `loaders-audio` extras as they are no longer needed.
- **BREKING**: `TextLoader`, `PdfLoader`, `ImageLoader`, and `AudioLoader` now take a `str | PathLike` instead of `bytes`. Passing `bytes` is still supported but deprecated.
- **BREAKING**: Removed `DataframeLoader`.
- **BREAKING**: Update `pypdf` dependency to `^5.0.1`.
- **BREAKING**: Update `redis` dependency to `^5.1.0`.
- **BREAKING**: Remove `torch` extra from `transformers` dependency. This must be installed separately.
- **BREAKING**: Split `BaseExtractionEngine.extract` into `extract_text` and `extract_artifacts` for consistency with `BaseSummaryEngine`.
- **BREAKING**: `BaseExtractionEngine` no longer catches exceptions and returns `ErrorArtifact`s.
- **BREAKING**: `JsonExtractionEngine.template_schema` is now required.
- **BREAKING**: `CsvExtractionEngine.column_names` is now required.
- **BREAKING**: Renamed`RuleMixin.all_rulesets` to `RuleMixin.rulesets`.
- **BREAKING**: Renamed `GriptapeCloudKnowledgeBaseVectorStoreDriver` to `GriptapeCloudVectorStoreDriver`.
- **BREAKING**: `OpenAiChatPromptDriver.response_format` is now a `dict` instead of a `str`.
- `MarkdownifyWebScraperDriver.DEFAULT_EXCLUDE_TAGS` now includes media/blob-like HTML tags
- `StructureRunTask` now inherits from `PromptTask`.
- Several places where API clients are initialized are now lazy loaded.
- `BaseVectorStoreDriver.upsert_text_artifacts` now returns a list or dictionary of upserted vector ids.
- `LocalFileManagerDriver.workdir` is now optional.
- `filetype` is now a core dependency.
- `FileManagerTool` now uses `filetype` for more accurate file type detection.
- `BaseFileLoader.load_file()` will now either return a `TextArtifact` or a `BlobArtifact` depending on whether `BaseFileManager.encoding` is set.
- `Structure.output`'s type is now `BaseArtifact` and raises an exception if the output is `None`.
- `JsonExtractionEngine.extract_artifacts` now returns a `ListArtifact[JsonArtifact]`.
- `CsvExtractionEngine.extract_artifacts` now returns a `ListArtifact[CsvRowArtifact]`.
- Remove `manifest.yml` requirements for custom tool creation.

Fixed

- Anthropic native Tool calling.
- Empty `ActionsSubtask.thought` being logged.
- `RuleMixin` no longer prevents setting `rulesets` _and_ `rules` at the same time.
- `PromptTask` will merge in its Structure's Rulesets and Rules.
- `PromptTask` not checking whether Structure was set before building Prompt Stack.
- `BaseTask.full_context` context being empty when not connected to a Structure.
- Tool calling when using `OpenAiChatPromptDriver` with Groq.

Added

- `BaseArtifact.to_bytes()` method to convert an Artifact's value to bytes.
- `BlobArtifact.base64` property for converting a `BlobArtifact`'s value to a base64 string.
- `CsvLoader`/`SqlLoader`/`DataframeLoader` `formatter_fn` field for customizing how SQL results are formatted into `TextArtifact`s.
- `AzureOpenAiTextToSpeechDriver`.
- `JsonSchemaRule` for instructing the LLM to output a JSON object that conforms to a schema.
- Ability to use Event Listeners as Context Managers for temporarily setting the Event Bus listeners.
- Ability to use Drivers Configs as Context Managers for temporarily setting the default Drivers.
- Generic type support to `ListArtifact`.
- Iteration support to `ListArtifact`.

Changed

- **BREAKING**: Removed `CsvRowArtifact`. Use `TextArtifact` instead.
- **BREAKING**: Removed `DataframeLoader`.
- **BREAKING**: Removed `MediaArtifact`, use `ImageArtifact` or `AudioArtifact` instead.
- **BREAKING**: `CsvLoader` and `SqlLoader` now return `ListArtifact[TextArtifact]`.
- **BREAKING**: Removed `ImageArtifact.media_type`.
- **BREAKING**: Removed `AudioArtifact.media_type`.
- **BREAKING**: Removed `BlobArtifact.dir_name`.
- **BREAKING**: Moved `ImageArtifact.prompt` and `ImageArtifact.model` into `ImageArtifact.meta`.
- **BREAKING**: `ImageArtifact.format` is now required.
- **BREAKING**: Removed the `__all__` declaration from the `griptape.mixins` module.
- Updated `JsonArtifact` value converter to properly handle more types.
- `AudioArtifact` now subclasses `BlobArtifact` instead of `MediaArtifact`.
- `ImageArtifact` now subclasses `BlobArtifact` instead of `MediaArtifact`.
- Removed `__add__` method from `BaseArtifact`, implemented it where necessary.

Fixed

- Crash when passing "empty" Artifacts or no Artifacts to `CohereRerankDriver`.

**Note**: This release includes breaking changes. Please refer to the [Migration Guide](./MIGRATION.md030x-to-031x) for details.

Added

- Parameter `meta: dict` on `BaseEvent`.
- `AzureOpenAiTextToSpeechDriver`.
- Ability to use Event Listeners as Context Managers for temporarily setting the Event Bus listeners.
- `JsonSchemaRule` for instructing the LLM to output a JSON object that conforms to a schema.
- Ability to use Drivers Configs as Context Managers for temporarily setting the default Drivers.

Changed

- **BREAKING**: Drivers, Loaders, and Engines now raise exceptions rather than returning `ErrorArtifacts`.
- **BREAKING**: Parameter `driver` on `BaseConversationMemory` renamed to `conversation_memory_driver`.
- **BREAKING**: `BaseConversationMemory.add_to_prompt_stack` now takes a `prompt_driver` parameter.
- **BREAKING**: `BaseConversationMemoryDriver.load` now returns `tuple[list[Run], dict]`. This represents the runs and metadata.
- **BREAKING**: `BaseConversationMemoryDriver.store` now takes `runs: list[Run]` and `metadata: dict` as input.
- **BREAKING**: Parameter `file_path` on `LocalConversationMemoryDriver` renamed to `persist_file` and is now type `Optional[str]`.
- **BREAKING**: Removed the `__all__` declaration from the `griptape.mixins` module.
- `Defaults.drivers_config.conversation_memory_driver` now defaults to `LocalConversationMemoryDriver` instead of `None`.
- `CsvRowArtifact.to_text()` now includes the header.

Fixed

- Parsing streaming response with some OpenAI compatible services.
- Issue in `PromptSummaryEngine` if there are no artifacts during recursive summarization.
- Issue in `GooglePromptDriver` using Tools with no schema.
- Missing `maxTokens` inference parameter in `AmazonBedrockPromptDriver`.
- Incorrect model in `OpenAiDriverConfig`'s `text_to_speech_driver`.
- Crash when using `CohereRerankDriver` with `CsvRowArtifact`s.
- Crash when passing "empty" Artifacts or no Artifacts to `CohereRerankDriver`.

Fixed

- Ensure thread safety when publishing events by adding a thread lock to batch operations in `BaseEventListenerDriver`.
- `FileManagerTool` failing to save Artifacts created by `ExtractionTool` with a `CsvExtractionEngine`.

Fixed

- `CsvExtractionEngine` not using provided `Ruleset`s.
- Docs examples for Extraction Engines not properly passing in schemas.

Added

- `AstraDbVectorStoreDriver` to support DataStax Astra DB as a vector store.
- Ability to set custom schema properties on Tool Activities via `extra_schema_properties`.
- Parameter `structure` to `BaseTask`.
- Method `try_find_task` to `Structure`.
- `TranslateQueryRagModule` `RagEngine` module for translating input queries.
- Global event bus, `griptape.events.EventBus`, for publishing and subscribing to events.
- Global object, `griptape.configs.Defaults`, for setting default values throughout the framework.
- Unique name generation for all `RagEngine` modules.
- `ExtractionTool` for having the LLM extract structured data from text.
- `PromptSummaryTool` for having the LLM summarize text.
- `QueryTool` for having the LLM query text.
- Support for bitshift composition in `BaseTask` for adding parent/child tasks.
- `JsonArtifact` for handling de/seralization of values.
- `Chat.logger_level` for setting what the `Chat` utility sets the logger level to.
- `FuturesExecutorMixin` to DRY up and optimize concurrent code across multiple classes.
- `utils.execute_futures_list_dict` for executing a dict of lists of futures.
- `GriptapeCloudConversationMemoryDriver` to store conversation history in Griptape Cloud.
- `griptape.utils.decorators.lazy_property` for creating lazy properties.

Changed

- **BREAKING**: Removed all uses of `EventPublisherMixin` in favor of `EventBus`.
- **BREAKING**: Removed `EventPublisherMixin`.
- **BREAKING**: Removed `Pipeline.prompt_driver` and `Workflow.prompt_driver`. Set this via `griptape.configs.Defaults.drivers.prompt_driver` instead. `Agent.prompt_driver` has not been removed.
- **BREAKING**: Removed `Pipeline.stream` and `Workflow.stream`. Set this via `griptape.configs.Defaults.drivers.prompt_driver.stream` instead. `Agent.stream` has not been removed.
- **BREAKING**: Removed `Structure.embedding_driver`, set this via `griptape.configs.Defaults.drivers.embedding_driver` instead.
- **BREAKING**: Removed `Structure.custom_logger` and `Structure.logger_level`, set these via `logging.getLogger(griptape.configs.Defaults.logger_name)` instead.
- **BREAKING**: Removed `BaseStructureConfig.merge_config`.
- **BREAKING**: Renamed `StructureConfig` to `DriversConfig`, moved to `griptape.configs.drivers` and renamed fields accordingly.
- **BREAKING**: `RagContext.output` was changed to `RagContext.outputs` to support multiple outputs. All relevant RAG modules were adjusted accordingly.
- **BREAKING**: Removed before and after response modules from `ResponseRagStage`.
- **BREAKING**: Moved ruleset and metadata ingestion from standalone modules to `PromptResponseRagModule`.
- **BREAKING**: Dropped `Client` from all Tool names for better naming consistency.
- **BREAKING**: Dropped `_client` suffix from all Tool packages.
- **BREAKING**: Added `Tool` suffix to all Tool names for better naming consistency.
- **BREAKING**: Removed `TextArtifactStorage.query` and `TextArtifactStorage.summarize`.
- **BREAKING**: Removed `TextArtifactStorage.rag_engine`, and `TextArtifactStorage.retrieval_rag_module_name`.
- **BREAKING**: Removed `TextArtifactStorage.summary_engine`, `TextArtifactStorage.csv_extraction_engine`, and `TextArtifactStorage.json_extraction_engine`.
- **BREAKING**: Removed `TaskMemory.summarize_namespace` and `TaskMemory.query_namespace`.
- **BREAKING**: Removed `Structure.rag_engine`.
- **BREAKING**: Split `JsonExtractionEngine.template_generator` into `JsonExtractionEngine.system_template_generator` and `JsonExtractionEngine.user_template_generator`.
- **BREAKING**: Split `CsvExtractionEngine.template_generator` into `CsvExtractionEngine.system_template_generator` and `CsvExtractionEngine.user_template_generator`.
- **BREAKING**: Changed `JsonExtractionEngine.template_schema` from a `run` argument to a class attribute.
- **BREAKING**: Changed `CsvExtractionEngine.column_names` from a `run` argument to a class attribute.
- **BREAKING**: Removed `JsonExtractionTask`, and `CsvExtractionTask` use `ExtractionTask` instead.
- **BREAKING**: Removed `TaskMemoryClient`, use `QueryClient`, `ExtractionTool`, or `PromptSummaryTool` instead.
- **BREAKING**: `BaseTask.add_parent/child` now take a `BaseTask` instead of `str | BaseTask`.
- Engines that previously required Drivers now pull from `griptape.configs.Defaults.drivers_config` by default.
- `BaseTask.add_parent/child` will now call `self.structure.add_task` if possible.
- `BaseTask.add_parent/child` now returns `self`, allowing for chaining.
- `Chat` now sets the `griptape` logger level to `logging.ERROR`, suppressing all logs except for errors.

Fixed

- `JsonExtractionEngine` failing to parse json when the LLM outputs more than just the json.
- Exception when adding `ErrorArtifact`'s to the Prompt Stack.
- Concurrency bug in `BaseVectorStoreDriver.upsert_text_artifacts`.
- Schema issues with Tools that use lists.
- Issue with native Tool calling and streaming with `GooglePromptDriver`.
- Description not being used properly in `StructureRunTool`.