Foundationallm

Improvements

Fixes downstream package dependency issue for the use of MS Presidio in the Gatekeeper Integration API.

Fixes logic for asynchronous vectorization processing while improving performance.

Fixes bring your own OpenAI deployment pipeline.

Fixes permission issue for the Gatekeeper API having access to Azure Content Safety.

Gateway API

The Gateway API is a load balancing and resiliency solution for embeddings. It sits in front of Azure OpenAI, serving vectorization embedding requests with the correct model and automatically handling rate limits.

- Vectorization Text Embedding Profiles can be configured to use `GatewayTextEmbedding`, complementing the existing `SemanticKernelTextEmbedding`
- Vectorization with the Gateway API only supports asynchronous requests

Agent RBAC

Agent-level RBAC enables FoundationaLLM administrators to manage access to individual agents, protecting organizations from data exfiltration. When a user creates an agent through the Management API, they will automatically be granted Owner access.

Vectorization Request Management Through the Management API

Users can submit and trigger Vectorization requests through the Management API, rather than the separate Vectorization API, improving consistency across the platform. Creating and triggering Vectorization requests are handled as two separate HTTP requests.

Citations Available in the Chat UI

Knowledge Management agents without Inline Contexts will include citations, indicating the document from the vector store used to answer the user's request.

Agent to Agent Conversations

Through the Semantic Kernel API, FoundationaLLM enables robust agent-to-agent interactions. Users can develop complex, multi-agent workflows that perform well across a variety of tasks.

End to end Testing architecture

With the release of 0.7.0, FoundationaLLM has established an elaborate architecture for E2E testing

Improvements

- User portal session linking and loading improvements
- Documentation updates for ACA and AKS deployments
- Added fix to ensure API keys are unique
- Some restructuring of folders and file movement
- Added support for prompt injection detection
- Added support for authorizing multiple resources in a single request
- Vectorization pipeline execution and state management improvements
- Added the ability for invocation of external orchestration services
- Added the ability to create OneLake synchronous and asynchronous vectorization
- Added support for GPT-3.5 1106 and GPT-4o

Changes to the 0.6.0 release

This document outlines the changes made to the FoundationaLLM project in the 0.6.0 release.

Zero trust - removing dependencies on API keys

The following components are now added to the list of Entra ID managed identity-based authentication support:
- Azure CosmosDB service
- Azure OpenAI in LangChain
- AzureAIDirect orchestrator
- AzureOpenAIDirect orchestrator

Citations

Citations (which means Explainability) is to be able to justify the responses returned by the agent and identify the source of where it based the response on. This release include the API portion of this feature and in next releases we will include the UI portion of this feature.

Features

`AzureAIDirect` orchestrator

Allows pointing agents directly (no orchestrators involved) to any LLM deployed in an Azure Machine Learning workspace (e.g., Llama-2 or Mistral models).

`AzureOpenAIDirect` orchestrator

Allows pointing agents directly (no orchestrators involved) to any LLM deployed in an Azure OpenAI deployment.

Override LLM parameters in completion requests

A new section is available in the completion requests that allows direct overrides of LLM parameters (e.g., `top_p`, `temperature`, and `logprobs` for GPT).

RBAC Roles

RBAC roles (`Reader`, `Contributor` and `User Access Administrator`) are now activated on the Management API, Core API, and Agent Factory API.

Vectorization

- Improved validation of vectorization requests (rejecting immediately requests for file types that are not supported).
- Stop vectorization request processing after N failed attempts at any given step.
- Dynamic pace of processing in vectorization worker.
- Add custom metadata to a vectorization request.

Zero trust - removing dependencies on API keys

The following components have now Entra ID managed identity-based authentication support:

- Vectorization content sources
- Resource providers
- Azure AI Search
- Authorization store and API
- Azure AI Content Safety

The following components are getting Entra ID managed identity-based authentication support in the next release:

- Azure CosmosDB service
- Azure OpenAI in LangChain
- `AzureAIDirect` orchestrator
- `AzureOpenAIDirect` orchestrator

Management Portal & API Updates

Data Sources

- Data Sources consolidate Vectorization Content Source Profiles, Text Partitioning Profiles, and Text Embedding Profiles
- Users simply need to create a Data Source and select a target Azure AI Search Index to run end-to-end Vectorization from the Management Portal
- Content Source Profiles, Text Partitioning Profiles, and Text Embedding Profiles will remain available for more advanced use cases

Configuration Management

- Management Portal automatically configures Azure App Configuration keys and Azure Key Vault secrets for new Data Sources
- Management API enables management of all Azure App Configuration keys and Azure Key Vault secrets

API Changes

- Agents
- Core API
- Session-less Completion: Removal of `X-AGENT-HINT` header & passing agent name in the JSON body
- Vectorization path casing

Features

Fixes the issue with the prompt prefix not being added to the context for the Internal Context agent.

Features

Fixes support for the vectorization of PPTX files.