Features
Management User Interface (UI)
The Management UI enables FoundationaLLM (FLLM) administrators to configure Agents without having to directly call the Management API. With this release, the Management UI has been enhanced to:
- Support all aspects of creating and configuring Knowledge Management Public Agents in the Management Portal. This includes:
- Selecting existing Content Source and Indexing Profiles
- Selecting the Chunk and Overlap size settings
- Configuring user-agent interactions, such as saving conversations and enabling Gatekeeper functionality
- Allowing the user to create the System Prompt to influence the tone and functionality of the Agent.
> [!NOTE]
> If you have not created vectorization profiles required by the agent, you will need to run the Postman collection found here first: [Directly calling the Vectorization API | FoundationaLLM](https://docs.foundationallm.ai/development/calling-apis/directly-calling-vectorization-api.html?q=postman).
A future release of the Management UI will support the creation of vectorization profiles.
For more information on these changes, please review the document: [Management UI | FoundationaLLM](https://docs.foundationallm.ai/setup-guides/management-ui/management-ui.html)
Vectorization Profiles
The FLLM Vectorization pipelines require Content Source, Text Partitioning, Text Embedding, and Indexing profiles. With
this release, you can now:
- Create all profile types without restarting the containers
- Use the Management API to create, read, update, and delete all profile types
For more information on these changes, please review the following
documents:
[Managing vectorization profiles | FoundationaLLM](https://docs.foundationallm.ai/setup-guides/vectorization/vectorization-profiles.html)
[Directly calling the Vectorization API | FoundationaLLM](https://docs.foundationallm.ai/development/calling-apis/directly-calling-vectorization-api.html?q=postman).
Knowledge Management Agents (LangChain)
The Knowledge Management Agent is responsible for providing users with a customized copilot experience based on its configuration. With this release:
- Create Knowledge Management agents that enable Retrieval Augmented Generation (RAG) workflows targeting vectorized content generated by the Vectorization pipelines.
- Use the Management API to create, read, update, and delete Knowledge Management Agents
- LangChain can be used for Knowledge Management Agents
For more information on these changes, please review the document: [Knowledge management agent | FoundationaLLM](https://docs.foundationallm.ai/setup-guides/agents/knowledge-management-agent.html)
Internal Context Agents (LangChain)
The Internal Context Agent provides a pass-through mechanism that sends the user prompt directly to the large language model (LLM) without any additional processing or context. With this release:
- The Internal Context Agent now has its own resource type.
- You can use the Management API to create, read, update, and delete Internal Context agents where no system prompt or content source is required. This gives users the ability to create their own fully constructed prompt and send it directly to OpenAI.
For more information on these changes, please review the document: [Internal context agent | FoundationaLLM](https://docs.foundationallm.ai/setup-guides/agents/internal-context-agent.html)
Direct Completion Requests (No need for sessions)
- You can now send a request to an Agent without first creating a chat session.
- Session-less completion is part of the Core API and uses Entra Auth
For more information on these changes, please review the document: [Core API | FoundationaLLM](https://docs.foundationallm.ai/setup-guides/exposed-apis/core-api.html?q=Session-less)
> [!NOTE]
> Management Client and API App Registrations are a requirement to utilize this functionality.
General Improvements
Management API
Extended configuration validation
- Validation is now in place to ensure that required App Configuration settings, Key Vault secrets, and environment variables are in place before starting the Management API. Nonconforming configuration parameters will be written to the logs indicating what is missing, and the Management API will not start.
Improved data validation
- Added more thorough validation and reporting of data validation issues when creating Agents and Vectorization profiles.
- Optional fields are no longer required for validation.
For more information on these changes, please review the document: [Azure App Configuration values | FoundationaLLM](https://docs.foundationallm.ai/deployment/app-configuration-values.html)
Generalized Events Platform
- The new Events Platform was added to propagate changes across supported services. You will no longer need to restart the services when configuration changes are made or new resources are created.
- There are two (2) different types of Events: `Storage` for handling Azure Blob Storage events and `Vectorization` Events for handling Vectorization pipeline requests.
- The following services can now take advantage of the Generalized Events platform:
- Core API supports Storage Events
- Agent Factory API supports Storage Events
- Vectorization API supports Storage and Vectorization Events
- Vectorization Worker supports Storage and Vectorization Events
- Management API supports Storage and Vectorization Events
Open Telemetry
- Telemetry influences how logs are collected from services. It was identified that Python services were not functioning well with the App Insights SDK, so all Python and .NET services are now using Open Telemetry. All logs are still written to Azure Application Insights for simplified analysis.
> [!NOTE]
> The Gatekeeper Integration API is not utilizing Open Telemetry with this release.
Performance and Stability Improvements
- Added new management endpoints on each of the Python APIs to allow resource providers and App Configuration values to be refreshed. Those endpoints can be called from the Management UI to perform that action.
Maintenance & Bug Fixes
- Credentials for API Management calls to API backends are now explicitly set.
- Updated the Bicep Template so that the App Gateway uses OWASP 3.2.
- Disabled Key Vault setting: `Allow trusted Microsoft services to bypass this firewall`.
- Updated script to generate a host file for standard AKS.
- Updated the versions for major Python libraries: FastAPI, LangChain, OpenAI, and gunicorn.
- Updated all resource providers to return "404 Not Found" instead of "500 Internal Error" when resources are not found.
- The Management API required the `object_id` field when creating new resources. This requirement has been removed.
- The Vectorization pipelines can process PowerPoint (`.pptx`) files.