Openlayer

No entries match these filters. Clear filters

1. changelog Oct 28, 2026 · issue 123

   Openlayer adds governance lifecycle stages, per-step test evaluation, EU data residency, Gateway routing/allow-block lists, new API endpoints, and multiple project dashboards.

   • Adds evaluationScope: step config field for per-step evaluation of tests on each trace step.
   • Adds openlayer tests delete CLI command — archives by default, --force permanently deletes.
   • Extends openlayer init with Copilot CLI and Gemini CLI support, custom CA bundles, a 'debug with an agent' option, and .openlayerignore for smaller uploads.
   • Adds Gateway GET /v1/models (per key) and OpenAI-compatible POST /v1/embeddings API endpoints.
   • Adds governance registry and framework endpoints to the public API and SDKs, writable with a workspace API key.

   +26 more show less

   • Adds a new batch-delete endpoint for asynchronous, batched, idempotent row deletes.
   • Adds EU data region — choose United States or European Union at sign-in/sign-up; account and data stay in the selected region.
   • Adds project lifecycle stages — define stages, assign reviewers per stage, and receive warnings when an earlier stage's requirements are no longer met.
   • Adds audit-ready framework export — export a compliance report for any framework.
   • Adds custom intake and risk-assessment forms with AI-assisted completion.
   • Adds redesigned framework activation — pick projects explicitly, add criteria for future projects, preview tests before creation, and set custom framework rules during creation.
   • New detectionNew integrationAdds Custom columns — a named JSONPathJSONPathA query language for extracting values from JSON documents, modeled after XPath for XML. Cyber tools use it to pinpoint specific fields in structured data without parsing the whole document manually. into structured outputsstructured outputsForcing a model's response to match a given format or JSON schema, so software can parse it reliably instead of hoping the text is well formed., defined once per project and usable in tests, filters, and tables across every data source and development commit.
   • Adds Jev-as-a-judge test that scores every row in the window and explains its score on demand.
   • Adds support for upper-only or lower-only automatic thresholds.
   • Adds AI-powered insights controls — workspace and project admins can enable/disable insights, override the judge model, and lock settings.
   • Adds ability to duplicate a test or set of tests into other projects, and export test configs as JSON from the results page.
   • Adds workspace-wide project tags with names and colors, and test tags settable from a pushed tests.json.
   • Adds multiple dashboards per project — create from the default layout or blank, rename, duplicate, delete, set visibility, and switch via breadcrumb or ⌘K.
   • Adds mode-aware usage, cost, and judge-cost widgets on the project dashboard; token consumption on the workspace dashboard; and provider grouping in cost-by-model.
   • Adds LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. cost table — view and edit token costs with workspace-level overrides.
   • Adds Observations view — inspect and compare steps across traces.
   • Adds row comments with threaded replies, @mentions, and a Feedback filter on the rows table.
   • Adds custom metric run logs and errors surfaced in the UI.
   • Adds ability to recover monitoring data after a project is deleted.
   • Adds API key expiry dates, automatic rotation, and full API key management.
   • Updated supported platformAdds Gateway routing by deployment, group, user, or key with narrowest-match precedence.
   • Adds Gateway model access allow and block lists, scoped to key, user, group, or deployment.
   • Adds Gateway All traffic observability destination that publishes every request.
   • Adds Gateway high availability — safe on multiple replicas with health endpoints and graceful shutdown; admin changes and usage limits apply within seconds.
   • Adds development-mode test result-over-time graph; development-mode tests can be paused.
   • Adds test result filters and open-row state persisted in the URL, so reloads and shared links restore them.

   31 details in Everything

   • New compliance framework NYDFS Circular Letter No. 7 (2024) built-in framework
   • New detection Bias and fairness tests for protected group disparities
   • New detection Prompt injection tests with multilingual support
   • New detection CPNI and telecom guardrail with PII recognizers
   • New integration Databricks AI/BI Genie discovery and monitoring integration
   • New integration Palantir Foundry AIP Agents and AIP Logic integration
   • New integration Claude Code and Claude Desktop OpenTelemetry telemetry
   • New integration AI usage-log data sources: OpenAI Compliance Logs, Claude Compliance API, Microsoft 365 Copilot
   • New integration CrowdStrike Falcon Guardian integration for AI coding agents
   • New integration Microsoft Agent Framework tracing integration
   • Updated integration Google ADK tracing updated for agent handoffs and google-adk 2.x
   • New integration Remote MCP connector with OAuth2 and permission levels
   • New supported platform AWS ECS Fargate reference deployments with Terraform
   └──▷ BREAKING ON UPGRADE

   • !Gateway Teams are renamed to Groups — any configuration, API calls, or documentation referencing Teams in the Gateway admin UI must be updated to Groups.
2. docs update Oct 21, 2026 · issue 115

   Openlayer gateway documents error formats, check ordering, 503 causes, and X-Request-ID tracing across all endpoints.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

   └──▷ TRY IT
   Attach a correlation ID to every API call so your operator can pinpoint the exact request behind a 422 guardrail block or 429 limit error in gateway logs.
   copy

   $ curl https://your-gateway.example.com/v1/responses \
     -H "Authorization: Bearer sk-olga-..." \
     -H "X-Request-ID: checkout-7f3a2c" \
     -H "Content-Type: application/json" \
     -d '{"model": "gpt-4o-mini", "input": "Hello!"}'

   • Adds X-Request-ID request header support so operators can trace exact gateway requests in logs and as the Openlayer trace ID.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠
   • Documents error format per endpoint: /v1/messages returns Anthropic shape; /v1/responses and /v1/embeddings return OpenAI shape; /v1/models returns shape based on path (/anthropic/v1/models or /openai/v1/models) or SDK headers on bare /v1/models.
   • Adds error type mapping from HTTP status code: invalid_request_error (400/422), authentication_error (401), permission_error (403), not_found_error (404), rate_limit_error (429), api_error (5xx).
   • Documents 503 Retry-After headers per message prefix: gateway at capacity and no upstream capacity for carry 1 s; guardrail loading and database unavailable carry 5 s; provider config errors carry none.
   • Documents /v1/images/generations error body as {"detail": ...} with guardrail blocks carrying type, stage, and reason fields rather than the OpenAI shape.

   +2 more show less

   • Documents gateway check order (API key → model access → usage limits → input guardrails → routing/provider → output guardrails) determining which error a caller sees when multiple checks could fail.
   • Documents that 422 guardrail blocks on the way out still count tokens toward usage limits.

   7 details in Everything

3. Governance lifecycle stages, bias and fairness tests, and multiple project dashboards seen Oct 21, 2026 · issue 115

   Openlayer adds governance lifecycle stages, bias/fairness tests, 8 new integrations, Gateway routing, and a public governance API.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

   ⤢ ⤢ ⤢ ⤢
   └──▷ TRY IT
   Delete a specific test permanently from the CLI, bypassing the default archive behavior.
   copy

   $ openlayer tests delete <test-id> --force

   Enable per-step evaluation so tests run against each trace step independently, not just the final output.
   tests.json copy

   {
     "evaluationScope": "step"
   }

   • Adds evaluationScope: step config field for per-step evaluation — run tests on each individual trace step.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠
   • Adds openlayer tests delete CLI command — archives tests by default; --force flag permanently deletes them.
   • Extends openlayer init to support Copilot CLI and Gemini CLI, custom CA bundles, a 'debug with an agent' option, and .openlayerignore for smaller uploads.
   • Adds governance registry and framework endpoints to the public API and SDKs, writable with a workspace API key.
   • Adds API key expiry dates, automatic rotation, and full API key management via the API.

   +5 more show less

   • Adds bias and fairness tests — four test types check protected groups and their intersections for outcome, score, performance, and metric disparities, with an Analysis tab for charts, drill-down, and CSV export.
   • Adds prompt injectionprompt injectionAn attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. tests — choose the detector and a sensitivity level (multilingual included), see why a detection fired, and pick the judge model.
   • Adds Freshness and Column freshness tests — alert when a data source stops receiving data or a datetime column gets too old, with manual or automatic thresholds.
   • Adds Custom columns — a named JSONPathJSONPathA query language for extracting values from JSON documents, modeled after XPath for XML. Cyber tools use it to pinpoint specific fields in structured data without parsing the whole document manually. into structured outputsstructured outputsForcing a model's response to match a given format or JSON schema, so software can parse it reliably instead of hoping the text is well formed., defined once per project and usable in tests, filters, and tables across every data source and development commit.
   • Adds Jev-as-a-judge test that scores every row in the window and explains its score on demand.
   • Adds evaluationScope: step config field for per-step evaluation — run tests on each individual trace step.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠
   • Adds openlayer tests delete CLI command — archives tests by default; --force flag permanently deletes them.
   • Adds governance registry and framework endpoints to the public API and SDKs, writable with a workspace API key.
   • Adds API key expiry dates, automatic rotation, and full API key management via the API.
   • Adds bias and fairness tests — four test types check protected groups and their intersections for outcome, score, performance, and metric disparities, with an Analysis tab for charts, drill-down, and CSV export.

   +4 more show less

   • Adds prompt injectionprompt injectionAn attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. tests — choose the detector and a sensitivity level (multilingual included), see why a detection fired, and pick the judge model.
   • Adds Freshness and Column freshness tests — alert when a data source stops receiving data or a datetime column gets too old, with manual or automatic thresholds.
   • Adds Custom columns — a named JSONPathJSONPathA query language for extracting values from JSON documents, modeled after XPath for XML. Cyber tools use it to pinpoint specific fields in structured data without parsing the whole document manually. into structured outputsstructured outputsForcing a model's response to match a given format or JSON schema, so software can parse it reliably instead of hoping the text is well formed., defined once per project and usable in tests, filters, and tables across every data source and development commit.
   • Adds Jev-as-a-judge test that scores every row in the window and explains its score on demand.

   1 more in Everything

4. docs update Oct 10, 2026 · issue 100

   Openlayer: Adds Export test configs from the table header overflow menu to export every test at once.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

   └──▷ HOW TO FIND IT
   Replicate a working test suite across projects when onboarding a new LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. task — use bulk duplicate to push all tests at once and review compatibility warnings before confirming.
   📍In the Manage Tests table, select all rows, open the header overflow menu, and choose Duplicate selected to project. Select one or more target projects (incompatible task types appear greyed out), then confirm — copies begin running immediately.
   Export the full test config from one project to back it up or import it elsewhere.
   📍In the Manage Tests table header overflow menu, choose Export test configs to download all test definitions in the table.
   • Copies carry over thresholds, cadence, and data settings; data-source selections and tags are not transferred, and column-specific settings are copied as-is but skipped at run time if target columns are missing.

   1 detail in Everything

5. docs update Oct 9, 2026 · issue 098

   Openlayer adds bulk and single-test duplication across projects, with compatibility filtering and pre-copy editing.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

   └──▷ HOW TO FIND IT
   Copy a configured test — thresholds, cadence, and data settings intact — into a different project in your workspace without re-building it from scratch.
   📍In a project's Tests view, open the overflow menu on any test and choose Duplicate to project, select the target project, then use Edit before creating to adjust any settings before confirming.
   • Adds Duplicate selected (bulk) action to copy selected tests within the same project.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠
   • Adds Duplicate selected to project (bulk) and Duplicate to project (single-test overflow) actions to copy tests to one or more other projects in the workspace, with incompatible project targets shown as unavailable and partially compatible targets receiving only supported tests.
   • Adds Edit before creating option when duplicating a single test to one project, letting you adjust settings before the copy is made.

   3 details in Everything

6. docs update Oct 6, 2026 · issue 096
   • New integration Claude Code integration
   • New integration Microsoft 365 Copilot integration
   • New integration Palantir Foundry integration
   • New integration Claude Compliance integration
7. docs update Oct 6, 2026 · issue 096

   Openlayer adds governance/compliance REST APIsREST APIsA convention for building HTTP-based interfaces where resources are addressed by URL and manipulated with standard verbs like GET, POST, and DELETE, giving cyber tools a predictable way to integrate with external services., Jev-as-a-judge test, and new integrations for Claude, Microsoft 365 Copilot, and Palantir Foundry.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

   └──▷ TRY IT
   Pull a compliance audit-ready archive for a governance framework to hand off to auditors without manual exports.
   copy

   $ curl -X GET 'https://api.openlayer.com/api-reference/rest/governance/export-framework' \
     -H 'Authorization: Bearer <your-api-key>' \
     --output framework-audit.zip

   Rotate a compromised API key in an automated pipeline without touching the Openlayer UI.
   copy

   $ curl -X POST 'https://api.openlayer.com/api-reference/rest/api-keys/rotate-api-key' \
     -H 'Authorization: Bearer <your-api-key>' \
     -H 'Content-Type: application/json'

   Check whether a background governance or ingestion job has completed before proceeding in a CI pipeline.
   copy

   $ curl -X GET 'https://api.openlayer.com/api-reference/rest/background-tasks/retrieve-background-task' \
     -H 'Authorization: Bearer <your-api-key>'

   • Adds GET /api-reference/rest/workspaces/retrieve-workspace endpoint to retrieve workspace details.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠
   • Adds Palantir Foundry integration to monitor AIP Agents and AIP Logic functions in Openlayer.

   2 details in Everything

8. docs update Oct 4, 2026 · issue 094

   Openlayer adds workspace-level AI feature configuration with per-project overrides, model selection, and cost tracking⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

   └──▷ HOW TO FIND IT
   Set up the Intake assistant for your workspace so it can draft project details and risk scores using your own model provider key.
   📍In Workspace Settings, go to AI Features › Intake assistant, select your model, then navigate to Environment and add your model provider API key.
   Lock a workspace-wide AI feature so individual projects cannot toggle it off, while still permitting per-project model overrides.
   📍In Workspace Settings, go to AI Features › [feature], enable the lock that 'stops projects from turning the feature on or off'.
   • Adds workspace-level AI feature configuration with per-project override support, covering Intake assistant, Trace summaries, and other AI-powered features.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠
   • Adds Intake assistant to draft project details, intake answers, and risk scores during project registration or from the project's Lifecycle page.
   • Adds a per-feature enable/disable lock that stops individual projects from turning a feature on or off, while still allowing per-project model overrides.
   • Adds model selection per AI feature, allowing each feature to use a different configured model.
   • Adds AI spend tracking broken down by feature — including Intake assistant — over a configurable time range, estimated using token prices from LLM costs.

   +1 more show less

   • Requires adding the selected model provider's API key under the workspace or project Environment settings for Intake assistant; Openlayer does not supply its own provider keys.

   6 details in Everything

9. docs update Oct 4, 2026 · issue 094

   Openlayer adds governance lifecycle stages, AI-assisted intake drafting, and a workspace-level review approval queue.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

   └──▷ HOW TO FIND IT
   Enable the AI intake assistant so teams can auto-draft risk scores and intake answers when registering a new AI project.
   📍1. Go to Settings → AI features and enable Intake assistant, then select your model. 2. Go to Settings → Environment and add your model provider API key. 3. In Register your project, complete the optional context step and select AI Assistant to generate draft answers. 4. Review each suggestion and accept only the fields you want before saving.
   Configure workspace lifecycle stages with ordered reviewers so each stage requires sequential sign-off before a project advances.
   📍1. Go to Settings › Governance › Lifecycle stages. 2. Select the Openlayer template or start from scratch. 3. For each stage, choose Ordered approval order and add reviewers in the desired sequence. 4. Set per-stage requirements (e.g. owner assigned, passing tests, minimum record count) from the requirements catalog.
   Handle a blocked project by reviewing risk exceptions and sending it back to an earlier stage for remediation.
   📍1. Open the project page and select Risk exceptions to review risks blocking the current lifecycle requirement. 2. If rework is needed, click Send back in the approval view and choose an earlier stage. 3. Monitor progress via the Review queue in the workspace sidebar under Pending other approvers.
   • Supports PDF, DOCX, MD, and TXT documents up to 25 MB as context for AI-assisted intake drafting (browser extracts text; up to ~60,000 characters used; file is not stored by Openlayer).
   • Adds workspace-level lifecycle stages configured under Settings › Governance › Lifecycle stages, with an Openlayer-provided four-stage template (Requirements, development, pipeline, and production stages) or a blank starting point.
   • Adds two approval-order modes per stage — Concurrent (all reviewers approve simultaneously) and Ordered (reviewers notified and approve in sequence).
   • Adds a Review queue in the workspace sidebar to manage pending lifecycle approvals, with a count badge and separate Needs your review and Pending other approvers views.
   • Adds Risk exceptions view on the project page to review exceptions associated with risks that block a lifecycle requirement.

   +4 more show less

   • Sends Lifecycle review requested and Lifecycle stage changed notifications on stage submission and transition; reminders are rate-limited to one per hour by default.
   • Supports per-project stage overrides to customize requirements or approvers for any stage without allowing advancement past unmet requirements.
   • Adds Send back action to return a project to an earlier stage for additional review.
   • Adds Retire option in the project options menu (⋯) to remove a project from its active lifecycle, with a Reopen action to restore it.

   9 details in Everything

10. docs update Oct 4, 2026 · issue 094

    Openlayer adds API key expiry, rotation with grace periods, and full programmatic key management via REST APIREST APIA web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services..⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

    ⤢ ⤢ ⤢
    └──▷ TRY IT
    Automate zero-downtime key rotation from a scheduled job or secret manager hook, giving applications 24 hours to pick up the new secret before the old one expires.
    copy

    $ curl --request POST \
      --url https://api.openlayer.com/v1/workspaces/<workspace-id>/api-keys/<api-key-id>/rotate \
      --header 'Authorization: Bearer <token>' \
      --data '{"gracePeriodHours": 24}'

    • Adds expiresAt field to API key creation and rotation API calls, with UI options of 7, 30, 60, or 90 days, or 1 year; keys default to never expiring.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠
    • Adds gracePeriodHours parameter to the POST /v1/workspaces/<workspace-id>/api-keys/<api-key-id>/rotate endpoint, allowing the previous secret to remain valid for up to 7 days after rotation to enable zero-downtime rollover.
    • Adds secret field in rotate API responses, returning the new secret exactly once.
    • Adds status and expiresAt fields to List API keys and Retrieve API key endpoints; secrets are never returned in listing or retrieval calls.
    • Adds Update an API key endpoint to rename existing keys.

    +4 more show less

    • Adds Delete an API key endpoint that immediately invalidates all secrets for the key.
    • Expired keys return 401 Unauthorized on every request and cannot be renewed or rotated — a new key must be created.
    • Read-only credentials (such as MCPMCPModel Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. client connections) can list and retrieve keys but cannot create, change, rotate, or delete them.
    • Keys created or rotated using an expiring API key cannot outlive the caller's key; omitting expiresAt inherits the caller's expiry, and a later expiry or null is rejected.

    9 details in Everything

11. docs update Oct 4, 2026 · issue 094

    Openlayer adds three new integrations: Claude Code trace collection via managed org settings, Microsoft 365 Copilot prompt/response import via Microsoft GraphMicrosoft GraphMicrosoft's unified REST API for accessing data and services across Microsoft 365, Azure AD, and related cloud products, giving cyber tools a single authenticated endpoint to query users, emails, files, and sign-in logs., and Claude Compliance session ingestion.

    • New integrationAdds Claude Code integration to send Claude Code traces to Openlayer via managed organizational settings.
    • New integrationAdds Microsoft 365 Copilot integration to import first-party prompts and responses from Microsoft Graph.
    • New integrationAdds Claude Compliance integration to ingest claude.ai chats and Cowork / Claude Code sessions into Openlayer.

    3 details in Everything

12. docs update Oct 4, 2026 · issue 094

    Openlayer: Adds GET /api-reference/rest/workspaces/retrieve-workspace endpoint to retrieve workspace details.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

    • Adds API key management endpoints: List API keys, Create API key, Retrieve API key, Update API key (rename), Delete API key, and Rotate API key (replace the secret).
    • Adds List invites endpoint to list invites in a workspace.
    • Adds test management endpoints: Create test, List tests, and Update tests for a project.
    • Adds Create an upload url (presigned URL) and Retrieve a download url (short-lived) endpoints for file storage operations.
    • Adds Retrieve project commit endpoint to fetch a specific project commit.

    +4 more show less

    • Adds a full governance REST APIREST APIA web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. surface: list/create/retrieve/update/export frameworks, list per-project compliance stats, list framework documents with sections and rules, list section and subsection rules, and full CRUDCRUDThe four basic database operations — Create, Read, Update, Delete — used as shorthand for any system that manages persistent data records, giving cyber tools a standard model for interacting with stored information. for rules and rule results including evidence attachment and workspace-level compliance statistics.
    • Adds Retrieve a background task endpoint to poll a task's status and outputs.
    • Adds Jev-as-a-judge test type to the test catalog for threshold-based LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. evaluation.
    • Adds project lifecycle configuration to define workspace lifecycle stages and guide projects through requirements and review.

    9 details in Everything

13. docs update Sep 26, 2026 · issue 086

    Openlayer adds a full Governance REST APIREST APIA web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services., prompt-injectionprompt-injectionAn attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. detection, EU data residency, and integrations for Azure AI Speech, Google Gemini, and OpenAI Compliance Logs.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

    └──▷ TRY IT
    Pull a workspace-level compliance roll-up to see how many governance rules are passing, failing, pending, or due for renewal — useful for a daily compliance dashboard.
    copy

    $ curl -X GET 'https://api.openlayer.com/governance/rule-stats' \
      -H 'Authorization: Bearer <token>' \
      -H 'Accept: application/json'

    Export a governance framework as an audit-ready zip archive — attach it directly to a compliance review or regulator submission.
    copy

    $ curl -X POST 'https://api.openlayer.com/governance/frameworks/<framework-id>/export' \
      -H 'Authorization: Bearer <token>' \
      -H 'Accept: application/zip' \
      -o framework-audit.zip

    Attach evidence to a rule result to satisfy an evidence rule programmatically — for example, after an automated test run completes.
    copy

    $ curl -X POST 'https://api.openlayer.com/governance/rule-results/<rule-result-id>/evidence' \
      -H 'Authorization: Bearer <token>' \
      -H 'Content-Type: application/json' \
      -d '{"description": "Automated test suite passed on 2025-07-01", "url": "https://ci.example.com/runs/42"}'

    • Adds GET /governance/frameworks/{id}/rules, GET /governance/frameworks/{id}/projects, and GET /governance/frameworks/{id}/rule-stats to list a framework's rules, projects, and per-project compliance roll-ups.
    • Adds GET /governance/rules, POST /governance/rules, GET /governance/rules/{id}, PATCH /governance/rules/{id}, and DELETE /governance/rules/{id} REST endpoints for full CRUDCRUDThe four basic database operations — Create, Read, Update, Delete — used as shorthand for any system that manages persistent data records, giving cyber tools a standard model for interacting with stored information. on governance rules.
    • Adds GET /governance/rule-results, GET /governance/rule-results/{id}, PATCH /governance/rule-results/{id}, GET /governance/rule-results/{id}/evidence, and POST /governance/rule-results/{id}/evidence to track and satisfy evidence requirements against rules.
    • Adds support for managing and exporting test configurations from the Results page.
    • Adds governance intake and risk forms, letting workspace admins define intake questions and risk dimensions shared across all projects.

    +1 more show less

    • Adds a project lifecycle view to review intake and risk information and advance projects through each governance lifecycle stage.

    6 details in Everything

14. docs update Sep 25, 2026 · issue 085

    Openlayer adds Azure AI Speech and OpenAI Compliance integrations for speech and enterprise ChatGPT observability.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

    • Adds Azure AI Speech integration to observe speech-to-text, translation, and text-to-speech workloads with Openlayer.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠
    • Adds OpenAI Compliance integration to ingest ChatGPT Enterprise and Codex activity into Openlayer.

    2 details in Everything

15. docs update Sep 25, 2026 · issue 085

    Openlayer: Adds GET /governance/frameworks to list governance frameworks in a workspace.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

    • Adds POST /governance/frameworks to create a custom governance framework in a workspace.
    • Adds GET /governance/frameworks/{id} to retrieve a governance framework by its id.
    • Adds PATCH /governance/frameworks/{id} to update a governance framework.
    • Adds GET /governance/frameworks/{id}/export to export a framework's evidence and progress as an audit-ready zip archive.
    • Adds GET /governance/frameworks/{id}/rules to list the rules that belong to a framework.

    +23 more show less

    • Adds GET /governance/frameworks/{id}/projects to list the projects a framework applies to.
    • Adds GET /governance/frameworks/{id}/documents to list the documents attached to a framework.
    • Adds GET /governance/frameworks/{id}/documents/{docId} to retrieve a framework document including its sections, subsections, and mapped rules.
    • Adds GET /governance/documents/{docId}/sections/{sectionId}/rules to list the rules mapped to a section of a framework document.
    • Adds GET /governance/documents/{docId}/subsections/{subsectionId}/rules to list the rules mapped to a subsection of a framework document.
    • Adds GET /governance/rules to list all governance rules in a workspace.
    • Adds POST /governance/rules to create a governance rule in a workspace.
    • Adds GET /governance/rules/{id} to retrieve a governance rule by its id, including the frameworks it belongs to and its tags.
    • Adds PATCH /governance/rules/{id} to update a governance rule (only fields sent are changed).
    • Adds DELETE /governance/rules/{id} to delete a governance rule and its rule results.
    • Adds GET /governance/rule-results to list rule results across a workspace.
    • Adds GET /governance/rule-results/{id} to retrieve a rule result by its id.
    • Adds PATCH /governance/rule-results/{id} to update a rule result (only fields sent are changed).
    • Adds GET /governance/rule-results/{id}/evidence to list the evidence attached to a rule result.
    • Adds POST /governance/rule-results/{id}/evidence to attach evidence to a rule result, satisfying an evidence rule.
    • Adds GET /governance/rule-stats to retrieve a workspace-level compliance roll-up (counts of passing, failing, pending, and renewal-due rule results).
    • Adds GET /governance/rule-tags to list all rule tags in a workspace.
    • Adds GET /development/retrieve-download-url to exchange a storageUri for a short-lived presigned download URL.
    • Adds GET /background-tasks/{id} to retrieve a background task's status, progress, and results.
    • Adds has-prompt-injection test to detect prompt injectionprompt injectionAn attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. attempts, with optional LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. judge explanations.
    • Supports choosing United States or European Union data residency regions at login or sign-up.
    • Adds Openlayer Agent Skills to teach AI coding assistants (Claude Code, Cursor, GitHub Copilot) how to integrate a codebase with Openlayer.
    • Adds Google Gemini integration to evaluate Gemini models and use them as LLM judges with Openlayer.

    28 details in Everything

16. docs update Sep 23, 2026 · issue 083

    Openlayer adds a Lifecycle page with stage advancement, intake overview, risk scorecard, and lifecycle history for governance-mode projects.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

    └──▷ HOW TO FIND IT
    Advance a high-risk AI project through its governance stages and audit who moved it and when.
    📍In the project sidebar, select Lifecycle. Check the Lifecycle stage section for 2/4 criteria met. Once all required criteria are satisfied, use the next-stage control to advance. Then select View history to open Lifecycle history and confirm the Stage, Date, and Actor of each transition.
    • Adds a Lifecycle page (reachable from the project sidebar or command menu) that consolidates lifecycle stage, intake information, and risk scorecard for governance-mode projects.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠
    • Adds a Lifecycle stage section with a segmented progress bar, a stage counter (e.g. Stage 2 of 5), per-stage criteria with Expand/Collapse controls, and progress reporting such as 2/4 criteria met.
    • Adds Send back for re-review control on the Lifecycle page to return a project for another review, restricted to workspace admins and project access-group admins.
    • Adds View history on the Lifecycle page to open a Lifecycle history modal listing each stage transition with Stage, Transition, Date, and Actor columns.
    • Adds a Legacy lifecycle value banner prompting migration when a project carries a lifecycle value from an earlier workflow.

    +3 more show less

    • Adds an Intake overview tab on the Lifecycle page that renders one card per section of the workspace intake form, with a Search intake information… field, and supports a user-arranged two-column card layout saved by section ID.
    • Adds a Risk tab on the Lifecycle page that opens a Risk scorecard driven by the workspace's risk template, with categories, dimensions, and descriptions pulled from that template.
    • Adds a Project details card shown before workspace-defined intake sections during project registration, surfacing name, description, owner, and tags as fixed project properties.

    8 details in Everything

17. docs update Sep 23, 2026 · issue 083

    Openlayer adds workspace-level governance with intake forms and risk templates for AI project registration.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

    └──▷ HOW TO FIND IT
    Set up a workspace intake form and risk template so every new AI project is assessed consistently at registration time.
    📍In Openlayer, go to Settings › Governance, open the Intake tab, then choose Start from Openlayer template or Start from scratch. Organize questions into sections and save. Switch to the Risk tab to define risk categories and dimensions, then save. Project teams will now see these forms on their project's Lifecycle page.
    • Adds a Settings › Governance area with Intake and Risk tabs where workspace admins can define intake forms (organized into sections) and risk templates (categories and dimensions) used across all projects.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠
    • Intake forms and risk assessments are answered per-project from the project's Lifecycle page during registration or afterward.
    • Workspace admins can start from the Openlayer template or from scratch when building intake forms and risk dimensions; workspace members can view but not edit these definitions.
    • Removing an intake field or risk dimension after projects have responded does not erase recorded data — historical answers and scores are retained.

    4 details in Everything

18. docs update Sep 23, 2026 · issue 083

    Openlayer: New Export a framework endpoint (/api-reference/rest/governance/export-framework) generates an audit-ready zip archive of a framework's evidence and progress.

    └──▷ TRY IT
    Check workspace-wide compliance health — how many rules are passing, failing, pending, or due for renewal — as part of a scheduled reporting job.
    copy

    $ curl -X GET 'https://api.openlayer.com/api-reference/rest/governance/retrieve-rule-stats' \
      -H 'Authorization: Bearer <your_api_key>'

    • New Attach evidence to a rule result endpoint (/api-reference/rest/governance/create-rule-result-evidence) lets you satisfy evidence rules programmatically.
    • New List a framework's per-project compliance endpoint (/api-reference/rest/governance/list-project-rule-stats) surfaces a compliance roll-up per project for a given framework.
    • Adds EU data region support — choose United States or European Union when signing up or logging in to Openlayer.
    • Adds intake and risk forms to define intake questions and risk dimensions shared across a workspace.
    • Adds project lifecycle governance view to review intake and risk, then advance projects through each governance stage.

    +1 more show less

    • Adds test export capability from the Results page for managing and exporting test configurations.

    6 details in Everything

19. announced Sep 23, 2026 · issue 093

    Openlayer launches jevals, an open-source Python library for typed agent evals and guardrails, plus Jev as a native judge inside Openlayer.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

    └──▷ USE IT
    Run security and behavior checks on a live agent trace to catch indirect injection and scope violations before acting on the result.
    python copy

    from jevals import evaluate
    from jevals.agent import ToolChoice, Grounded, StayedInScope
    from jevals.security import IndirectInjection
    
    result = evaluate(
        {"messages": messages, "tools": tools},
        [
            ToolChoice(),
            Grounded(),
            StayedInScope(),
            IndirectInjection(),
        ],
    )
    print(result.table())

    Install jevals to start writing typed agent evals and guardrails in a new project.
    copy

    $ pip install jevals

    • Adds pip install jevals open-source Python library for agent evals and runtime guardrails built around typed decision models (Jev-style), replacing freeform LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code.-judge text generation with structured choices, scores, and probabilities.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠
    • Adds evaluate() function from jevals accepting agent messages and tool schemas, returning typed results via .table().
    • Adds 37 prebuilt evaluations in jevals covering tool selection, groundedness, task completion, scope adherence, prompt injectionprompt injectionAn attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools., and sensitive information — importable as ToolChoice, Grounded, StayedInScope, IndirectInjection, and more from modules jevals.agent and jevals.security.
    • Supports custom evaluations written in Python or YAML alongside the prebuilt checks.
    • Adds gate mechanism in jevals that applies a policy to evaluation results and routes agent actions to proceed, block, or require human review — enabling the same eval definition to run in both development testing and production guardrails.

    +2 more show less

    • Supports hosted Jev, local Kev and Laya backends, and conventional LLM backends, with evaluation definitions remaining stable across backend switches.
    • Adds Jev as a native judge directly inside Openlayer, alongside existing classifier evaluation tooling (F1 scores, confusion matrices, drift, explainability).

    7 details in Everything

20. docs update Sep 22, 2026 · issue 082

    Openlayer adds EU data residency with a dedicated eu.openlayer.com host alongside the existing US region.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

    └──▷ HOW TO FIND IT
    Access Openlayer under EU data residency — navigate directly to the EU host so your account data stays in the European Union.
    📍In the browser, go to eu.openlayer.com. On the login or signup form, select the Data region field and choose European Union (EU data region), then complete authentication.
    • Adds a European Union data region accessible at eu.openlayer.com, separate from the existing United States region at app.openlayer.com.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠
    • Adds a Data region selector on the login and signup forms, letting users choose between US data region and EU data region before authenticating.

    2 details in Everything

21. docs update Sep 22, 2026 · issue 082

    Openlayer adds EU data region support, letting users choose between United States and European Union at login.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

    ⤢ ⤢
    └──▷ HOW TO FIND IT
    Select the EU data region at account creation so your data stays within European Union infrastructure.
    📍On the Openlayer login or sign-up page, locate the Data region field and select European Union before completing sign-up. Your account and all session data will be scoped to the EU region.
22. docs update Sep 22, 2026 · issue 082

    Openlayer adds bulk test config export from the Results page, downloading definitions as a ready-to-use tests.json array.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

    └──▷ HOW TO FIND IT
    Export all selected test definitions at once via the Results page bulk action, then review the downloaded file.
    📍Results page › select tests › overflow menu › Export selected test configs
    • Adds Export test config from a single test's row overflow menu (⋯) on the Results page to download its definition as a tests.json array.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠
    • Adds Export selected test configs bulk action on the Results page to download multiple test definitions as a single tests.json array, also copying the JSON to the clipboard.
    • Exported tests.json array can be checked into a repository and used with openlayer push to keep tests in sync — distinct from the openlayer tests command output, which wraps definitions in _meta and items.
    • Adds Export test config from a single test's row overflow menu (⋯) on the Results page to download its definition as a tests.json array.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠
    • Adds Export selected test configs bulk action on the Results page to download multiple test definitions as a single tests.json array, also copying the JSON to the clipboard.

    1 more in Everything

23. docs update Sep 22, 2026 · issue 082

    Openlayer adds OpenAI Compliance integration to ingest ChatGPT Enterprise and Codex activity.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

24. docs update Sep 22, 2026 · issue 082

    Openlayer adds a full Governance API, prompt-injectionprompt-injectionAn attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. detection, data residency choice, OpenAI Compliance Logs ingestion, and Google Gemini integration.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

    └──▷ TRY IT
    Export a governance framework as an audit-ready zip archive after all evidence has been attached to its rule results.
    copy

    $ curl -X GET 'https://api.openlayer.com/governance/frameworks/{id}/export' \
      -H 'Authorization: Bearer <token>' \
      --output framework-audit.zip

    Attach evidence to a rule result to satisfy a compliance requirement programmatically.
    copy

    $ curl -X POST 'https://api.openlayer.com/governance/rule-results/{ruleResultId}/evidence' \
      -H 'Authorization: Bearer <token>' \
      -H 'Content-Type: application/json' \
      -d '{"name": "Bias evaluation report", "url": "https://storage.example.com/report.pdf"}'

    Check workspace-wide compliance health — how many rules are passing, failing, pending, or due for renewal — in a single call.
    copy

    $ curl -X GET 'https://api.openlayer.com/governance/rule-stats' \
      -H 'Authorization: Bearer <token>'

    • Adds GET /governance/frameworks (List frameworks), POST /governance/frameworks (Create a framework), GET /governance/frameworks/{id} (Retrieve a framework), PATCH /governance/frameworks/{id} (Update a framework), and GET /governance/frameworks/{id}/export (Export a framework) REST endpoints to programmatically manage governance frameworks and download audit-ready zip archives.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠
    • Adds GET /governance/frameworks/{id}/rules (List a framework's rules), GET /governance/frameworks/{id}/projects (List a framework's projects), GET /governance/frameworks/{id}/project-rule-stats (List a framework's per-project compliance), GET /governance/frameworks/{id}/documents (List a framework's documents), and GET /governance/frameworks/{id}/documents/{docId} (Retrieve a framework document) REST endpoints for deep inspection of framework structure and compliance roll-ups.
    • Adds GET /governance/frameworks/{id}/documents/{docId}/sections/{sectionId}/rules (List a section's rules) and GET /governance/frameworks/{id}/documents/{docId}/sections/{sectionId}/subsections/{subsectionId}/rules (List a subsection's rules) REST endpoints to enumerate rules mapped to specific document sections.
    • Adds GET /governance/rules (List rules), POST /governance/rules (Create a rule), GET /governance/rules/{id} (Retrieve a rule), PATCH /governance/rules/{id} (Update a rule), and DELETE /governance/rules/{id} (Delete a rule) REST endpoints to fully manage individual governance rules.
    • Adds GET /governance/rule-results (List rule results), GET /governance/rule-results/{id} (Retrieve a rule result), PATCH /governance/rule-results/{id} (Update a rule result), GET /governance/rule-results/{id}/evidence (List a rule result's evidence), and POST /governance/rule-results/{id}/evidence (Attach evidence to a rule result) REST endpoints to track and satisfy evidence requirements.

    +8 more show less

    • Adds GET /governance/rule-stats (Retrieve workspace compliance statistics) REST endpoint returning a roll-up of how many rules are passing, failing, pending, or due for renewal across a workspace.
    • Adds GET /governance/rule-tags (List rule tags) REST endpoint for enumerating tags across workspace governance rules.
    • Adds GET /development/retrieve-download-url REST endpoint to exchange a storageUri for a short-lived presigned download URL.
    • Adds GET /background-tasks/{id} (Retrieve a background task) REST endpoint to poll status, progress, and results of async operations.
    • Adds a Has prompt injection test (catalog entry has-prompt-injection-count) that detects prompt injection attempts and supports optional LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. judge explanations.
    • Adds a Google Gemini integration to evaluate Google Gemini models and use them as LLM judges.
    • Adds Openlayer Agent Skills to teach AI coding assistants (Claude Code, Cursor, GitHub Copilot) how to integrate a codebase with Openlayer.
    • Adds a Governance API guide covering how to create frameworks and rules, attach evidence, and pull compliance status and audit-ready exports.

    13 details in Everything

25. docs update Sep 14, 2026 · issue 074

    Openlayer adds audit-ready evidence export from compliance framework pages, with SHA-256SHA-256A cryptographic hash function that turns data into a fixed 256-bit fingerprint. It is used to check integrity; hashing does not encrypt the original data. checksums and scoped project exports.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

    ⤢ ⤢
    └──▷ HOW TO FIND IT
    Package all compliance evidence for an auditor across every project in a framework.
    📍In the Openlayer UI, go to the workspace framework page, open the options menu (⋯) next to the framework, and select Export report. The download includes uploaded evidence files, a markdown control-status report, and CSV manifests with SHA-256 checksums. Ensure your role has the Export framework evidence permission.
    Scope an audit evidence package to a single project when only one project is under review.
    📍In the Openlayer UI, navigate to the specific project, open its framework page, open the options menu (⋯), and select Export report. The resulting zip is scoped to that project only.
    • Adds Export report action (via the ⋯ options menu) on workspace and project framework pages to download a zip containing uploaded evidence files, a markdown progress/control-status report, and CSV manifests with SHA-256 checksums — requires the Export framework evidence permission.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠
    • Supports scoping the export to a single project by triggering Export report from the project's framework page instead of the workspace framework page.
    • Adds Export report action (via the ⋯ options menu) on workspace and project framework pages to download a zip containing uploaded evidence files, a markdown progress/control-status report, and CSV manifests with SHA-256 checksums — requires the Export framework evidence permission.⁠︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄⁠

    1 more in Everything

26. docs update Sep 5, 2026 · issue 065

    Openlayer's Amazon Bedrock integration now traces embeddingembeddingLists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG. invocations for Titan and Cohere models

    ⤢ ⤢
27. docs update Sep 5, 2026 · issue 065

    Openlayer MCPMCPModel Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. connector adds OAuth 2.1OAuth 2.1A consolidated draft of the OAuth authorization framework that merges OAuth 2.0 with its most important security BCP updates, dropping unsafe flows so tools can delegate user access without handling credentials directly. resource-server mode, configurable via environment variables, with tools for pipelines, traces, and governance.

    ⤢
    • Adds OPENLAYER_MCP_OAUTH_ISSUER environment variable to set the Authorization Server base URL, used as token issuer and advertised sign-in endpoint.
    • Adds OPENLAYER_MCP_PUBLIC_URL environment variable to set the connector's public base URL; MCP is served at <OPENLAYER_MCP_PUBLIC_URL>/mcp.
    • Adds OPENLAYER_MCP_OAUTH_JWKS_URI environment variable to optionally override the jwks_uri advertised by the Authorization Server.
    • Adds OPENLAYER_MCP_AUTH_MODE environment variable to control caller authentication; defaults to resource-server, with oidc and none as narrower alternatives.
    • Adds MCP tools to list, create, and update inference pipelines, check data ingestion status, and re-evaluate monitoring tests over a time range.

    +4 more show less

    • Adds MCP tools to fetch and filter production traces by inference ID and patch rows to backfill ground-truth labels for scored tests.
    • Adds MCP tools to investigate sessions and users, including session- and user-level aggregations, AI summaries, and sessions behind failing tests.
    • Adds MCP tools to list, create, update, and delete tests (integrity, consistency, performance, and LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code.-as-a-judge), trigger evaluation, and drill into failed rows with per-row scores and explanations.
    • Adds MCP tools to review and edit governance posture: frameworks, documents, rules, rule tags, rule results, and evidence attached to results.

    9 details in Everything

28. docs update Sep 5, 2026 · issue 065

    Openlayer adds configurable data retention periods for monitoring data, with options from 7 days to indefinite.

    └──▷ HOW TO FIND IT
    Set a 90-day rolling retention window so monitoring data is automatically purged nightly and does not accumulate indefinitely.
    📍1. In your project, go to Project settings → General. 2. Scroll to the Data retention section. 3. Under Retention period, open the selector (placeholder: Select retention period) and choose 90 days. 4. Confirm the toast 'Data retention updated — Monitoring data older than 90 days will be permanently deleted.'
29. docs update Sep 5, 2026 · issue 065

    Openlayer adds project-scoped Secret and Plain environment variables with workspace inheritance and LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code.-as-a-judge connection testing.

    ⤢
    └──▷ HOW TO FIND IT
    Store an OpenAI API key as a secret at the project level so it is encrypted at rest and never visible to teammates, then verify the connection to the judge model.
    📍1. Open your project and go to Settings › Environment. 2. Under Secret variables, click Add variable, enter the key name (e.g. OPENAI_API_KEY) and its value, then save. 3. Navigate to Settings › LLM-as-a-judge, select OpenAI / gpt-4.1-mini-2025-04-14 in the Default LLM picker, then click Test connection to confirm Openlayer can reach the model.
    Store a non-sensitive endpoint or region as a Plain variable so teammates can look it up without needing admin access.
    📍1. Open your project and go to Settings › Environment. 2. Under Plain variables, click Add variable, enter the key (e.g. AWS_REGION) and value (e.g. us-east-1), then save. 3. Confirm the value is visible to workspace members on the same page.
    • Adds Secret and Plain variable kinds under Settings > Environment: secrets are encrypted at rest and value-masked to workspace members; plain variables are readable and suited for client IDs, regions, and endpoints.
    • Adds project-level Settings > Environment page for variables scoped to a single project, with the same Secret variables and Plain variables sections as the workspace page.
    • Workspace-level variables are automatically inherited at the project level, shown with an INHERITED FROM WORKSPACE badge, and can be overridden by setting a different value at the project level.
    • Adds Settings > LLM-as-a-judge page with Default LLM judge, Default LLM model picker, and Test connection button to verify that Openlayer can reach the selected provider model.

    4 details in Everything

30. docs update Sep 5, 2026 · issue 065

    Openlayer adds opt-in default-deny project visibility via Hide projects without access group workspace setting

    ⤢ ⤢
    └──▷ HOW TO FIND IT
    Enable default-deny project visibility before creating access groups so non-admins never see unassigned projects
    📍1. Go to Workspace settings → Security → Access control 2. Enable Hide projects without access group 3. Create your access groups and add users to them before creating new projects
    • Adds Hide projects without access group under Workspace settings → Security → Access control to make projects invisible to non-admins unless they are explicitly added via an access group, inverting the default open-visibility posture

    1 detail in Everything

31. docs update Sep 5, 2026 · issue 065

    SAML SSOSAML SSOAn authentication protocol that lets a user log in once with an identity provider and access multiple services without re-entering credentials, by passing signed XML assertions between parties. directory sync gains a No access group mapping and ongoing reconciliation that corrects role drift and removes stale members.

    └──▷ HOW TO FIND IT
    Suppress workspace access for a company-wide IdPIdPA service that authenticates users and supplies identity information to other applications. It is the sign-in authority behind many single sign-on systems. group that directory sync must see, without removing it from the sync.
    📍In the SAML SSO configuration screen, under group mappings, map your company-wide group (e.g. Everyone) to the No access role. Confirm that at least one other group is still mapped to Admin. The mapped group appears in the No access section, separate from groups with real roles.
    • Adds No access as a mappable role in SAMLSAMLA standard for passing signed identity assertions between a sign-in provider and an application. It lets a user sign in through their organization's identity provider. SSOSSOSingle sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. group configuration, letting you include IdP-pushed groups (such as a company-wide directory group) without granting any workspace permissions or surfacing the workspace in members' switchers.
    • Reconciliation events are recorded in the workspace audit trail for review.
    • Adds No access as a mappable role in SAMLSAMLA standard for passing signed identity assertions between a sign-in provider and an application. It lets a user sign in through their organization's identity provider. SSOSSOSingle sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. group configuration, letting you include IdP-pushed groups (such as a company-wide directory group) without granting any workspace permissions or surfacing the workspace in members' switchers.

    1 more in Everything

32. docs update Sep 5, 2026 · issue 065

    Openlayer adds a No access role for IdPIdPA service that authenticates users and supplies identity information to other applications. It is the sign-in authority behind many single sign-on systems. directory sync group mapping, letting synced groups exist without granting workspace permissions.

33. docs update Sep 5, 2026 · issue 065

    Openlayer adds workspace security controls: domain restrictions, project visibility, connected apps management, and audit trail export

    ⤢ ⤢
    └──▷ HOW TO FIND IT
    Lock down workspace sprawl so every employee with a company email joins the single governed workspace instead of creating their own.
    📍1. Go to Workspace settings → Security → Domains 2. Add and verify your organization's email domain 3. Enable Restrict workspace creation by domain
    Respond to an auditor request by exporting a time-scoped CSV of all security-related workspace actions.
    📍1. Go to Workspace settings → Security → Compliance & audit 2. Set the desired time range 3. Click Export — the CSV is generated in the background and downloaded when ready
    • Adds Restrict workspace creation by domain toggle under Workspace settings → Security to prevent users with a verified organization domain from creating new workspaces, routing them into the existing governed workspace instead.
    • Adds Hide projects without access group control under Access control to limit non-admins to only projects they have been explicitly added to via an access group (off by default).
    • Adds Connected apps panel listing authorized applications (such as the Openlayer MCPMCPModel Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. connector) with full or read-only access, with per-connection revocation to immediately cut off access.
    • Adds audit trail export under Compliance & audit that produces a CSV of security-related workspace events, optionally filtered by time range, available to workspace admins.

    4 details in Everything

34. docs update Sep 5, 2026 · issue 065

    Openlayer gateway gains workspace connections, traffic-routing destinations, and multimodalmultimodalAble to work with more than one type of input or output, such as text, images and audio. A multimodal model can combine information across those formats. media attachment support in traces.

    ⤢ ⤢ ⤢
    └──▷ HOW TO FIND IT
    Mirror all gateway traffic into one shared data source while routing a specific group to its own, using All traffic and a group destination together.
    📍In the portal, go to Settings › Connect a workspace, enter your Openlayer API key and leave Base URL blank for the public API. Then add two destinations: one with target All traffic pointing to your audit data source, and one with target set to your specific Group pointing to that team's project data source.
    Temporarily pause trace delivery to a destination without losing its data source mapping, for example during a data source migration.
    📍In the portal, open the destination you want to pause and enable the mute toggle. The destination retains its workspace, project, and data source configuration and can be re-enabled when ready.
    • Adds workspace connection management under Settings, where you supply an Openlayer API key and optional Base URL for on-premises installs; the gateway verifies, encrypts, and stores the credential and reflects back project and member counts.
    • Supports connecting more than one workspace so a single gateway can publish traces into separate Openlayer workspaces simultaneously.
    • Adds destinations that route traces by target — Group, User, All traffic, or Unclaimed — to a chosen workspace, project, and data source, with fan-out semantics so multiple matching destinations each receive a copy.
    • Introduces All traffic destination type that mirrors every trace into one data source regardless of other matching destinations.
    • Introduces Unclaimed fallback destination that receives only traces no more specific destination claimed.

    +4 more show less

    • Adds mute toggle on destinations to pause trace delivery without losing the data source configuration.
    • Supports creating a project or data source directly from the destination picker without leaving the portal.
    • Carries images and documents sent to or returned by a model into traces as attachments uploaded to Openlayer storage, avoiding inline base64 and keeping large-image traces readable.
    • Provides two deployment-wide attachment options: whether to upload attachment bytes at all, and whether to upload media supplied as a URL (attachment uploads on by default, URL uploads off by default).

    9 details in Everything

35. docs update Sep 5, 2026 · issue 065

    Openlayer Gateway adds model access rules to allow or block specific models per API key, user, group, or deployment

    └──▷ TRY IT
    Verify that a blocked model is refused and not surfaced — confirm a 403 permission_error and that GET /v1/models omits the blocked model for the key.
    copy

    $ curl -s -o /dev/null -w '%{http_code}' https://<gateway>/v1/chat/completions \
      -H 'Authorization: Bearer <api-key>' \
      -H 'Content-Type: application/json' \
      -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "hello"}]}'
    # Expect: 403
    
    curl -s https://<gateway>/v1/models \
      -H 'Authorization: Bearer <api-key>'
    # Expect: gpt-4o absent from the list

    Navigate to the Gateway console to create an allow-only rule that restricts a specific API key to a single model family using a prefix wildcard.
    📍1. In the Gateway console, go to Policies › Model access. 2. Click Create a rule. 3. Set Who it's for to the target API key. 4. Set the rule type to Allow only. 5. Enter the model name pattern, e.g. claude-opus-* (prefix wildcard). 6. Save — the key can now call only models matching that prefix; all others return 403.
    • Adds model access rules under Policies > Model access to allow or block specific model names (e.g. gpt-4o, claude-sonnet-4-5) per API key, user, group, or all traffic — before routing occurs.
    • Supports prefix wildcard matching (e.g. claude-opus-*) so rules continue to cover a model family when vendors append new date suffixes to names.
    • Refused requests return 403 mapped to permission_error in the caller's dialect, and GET /v1/models is filtered by the same policy so blocked models never appear in a client's model picker.

    3 details in Everything

36. docs update Sep 5, 2026 · issue 065

    Openlayer Gateway gains stackable guardrails for PIIPIIPersonally Identifiable Information — data that can identify a specific individual, such as names, email addresses, or SSNs. Cyber tools flag or redact it to limit exposure and meet privacy regulations., prompt injectionprompt injectionAn attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools., and content safety on LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. traffic

    ⤢ ⤢
    └──▷ HOW TO FIND IT
    Set up a PII guardrail that redacts email addresses and phone numbers in LLM responses before they reach end users, while blocking credit card numbers outright.
    📍In the Openlayer console, go to Policies › add a guardrail, then: set target to the relevant API key or group; choose PII as the type; set stage to Output; under Redacted entities add Email and Phone number; under Blocked entities add Credit card number; raise the confidence threshold to reduce false positives; save.
    Add a prompt injection guardrail scoped to a user group to block jailbreak attempts on input traffic only, without affecting system prompts.
    📍In the Openlayer console, go to Policies › add a guardrail, then: set target to the relevant group; choose Prompt injection as the type; set stage to Input only; set the confidence threshold to your desired sensitivity; save.
    Enable content safety scanning on both input and output for a specific API key, monitoring only Threat and Identity hate categories to minimise false positives.
    📍In the Openlayer console, go to Policies › add a guardrail, then: set target to the relevant API key; choose Content safety as the type; set stage to Both; deselect all categories except Threat and Identity hate; raise the per-category threshold as needed; save.
    • Adds PII guardrail type with Blocked entities (reject on match) and Redacted entities (replace matched spans with placeholders) lists, covering universal types like emails, phone numbers, and credit card numbers as well as country-specific identifiers, with a configurable confidence threshold to reduce false positives.
    • Adds prompt injection guardrail type that scores user turns only (skipping system prompts and assistant/tool turns) and always blocks, with a configurable confidence threshold.
    • Adds content safety guardrail type scoring text across six categories — Toxic, Severe toxic, Obscene, Threat, Insult, and Identity hate — with a per-category threshold probability and block-or-allow outcome (no redaction path).
    • Guardrails can be scoped to a group, a user, or a single API key (including keys with No group assigned), and multiple matching guardrails stack so a key can be screened simultaneously by its own guardrail and its group's.
    • Guardrail stage can be set to Output (scan responses only) or Both (scan input and output); prompt injection offers Input only.

    +1 more show less

    • Guardrails table shows each guardrail's stage, violation action, violation count over the last 7 days, and last-triggered time — storing only metadata, never the underlying text.

    6 details in Everything

37. docs update Sep 5, 2026 · issue 065

    Custom metric parameters now support enum to restrict inputs to a dropdown of fixed values in the test creation modal.

    ⤢ ⤢ ⤢ ⤢
    └──▷ USE IT
    Restrict a comparison parameter to exact or fuzzy so testers pick from a dropdown instead of typing a free-form value.
    json copy

    {
      "comparison": {
        "enum": ["exact", "fuzzy"]
      }
    }

38. docs update Sep 5, 2026 · issue 065

    Openlayer adds bulk test management — run, pause, duplicate, tag, archive, and delete multiple tests at once from the Results page.

    └──▷ HOW TO FIND IT
    Bulk-pause noisy tests during an incident so they stop computing results until you re-enable them.
    📍On the Results page, check the boxes next to all affected tests, then open the overflow menu and choose Pause selected.
    • Adds bulk actions to the Results page overflow menu: Run selected, Pause selected, Duplicate selected, Tag selected, Archive selected, Unarchive selected, and Delete selected for managing multiple tests at once.
    • Adds Archive selected bulk action that hides tests while preserving result history, reversible from the platform.
    • Adds openlayer tests delete CLI command to remove or archive tests outside the Results page UI.

    3 details in Everything

39. docs update Sep 5, 2026 · issue 065

    Openlayer adds automatic thresholds, filter value suggestions, and test pausing to test configuration

    ⤢
    └──▷ HOW TO FIND IT
    Pause all tests in a project while replaying historical data so results are not polluted during the replay window.
    📍In the tests list, open the overflow menu on a test row and select Pause. While paused, a banner reads 'Tests are paused. You can still create tests, but they will not compute results.' Select Unpause in the banner when ready to resume.
    Fail a cost metric test only when costs rise — not when they drop — by using a one-sided automatic threshold.
    📍On the test threshold tab, select Auto, then choose Upper only so the test fails only when the observed metric exceeds the predicted upper bound.
    Limit LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code.-as-a-judge scoring to a random 20% sample of rows, capped at 50 rows, to control cost on large datasets.
    📍In Judge settings, set Sample size (%) to 20, Data limit (rows) to 50, and Sampling method to Random. To apply the same defaults project-wide, go to Settings → LLM-as-a-judge and set the same values.
    • Adds an Auto threshold tab to tests so Openlayer learns the historical metric distribution and fails a test when the observed value falls outside predicted bounds, with Both bounds, Upper only, or Lower only directionality controls.
    • Adds Sample size (%), Data limit (rows), and Sampling method controls under Judge settings for LLM-as-a-judge tests, configurable per-test or project-wide under Settings → LLM-as-a-judge.
    • Adds Evaluation windows settings in monitoring mode — Cadence (Scheduled or Manual), Evaluation delay, Include historical data in each run, and Anchor evaluation windows toggles.
    • Adds filter value suggestions in the column filter picker, showing values Openlayer has already seen, each with a match count, scoped to equality, membership, contains, or ordering comparisons; high-cardinality columns show a Type to search values prompt instead.
    • Adds the ability to pause and unpause tests via the overflow menu (Pause / Unpause) so tests stop computing results during data replays or upstream fixes.
    • Adds an Auto threshold tab to tests so Openlayer learns the historical metric distribution and fails a test when the observed value falls outside predicted bounds, with Both bounds, Upper only, or Lower only directionality controls.
    • Adds Evaluation windows settings in monitoring mode — Cadence (Scheduled or Manual), Evaluation delay, Include historical data in each run, and Anchor evaluation windows toggles.
    • Adds filter value suggestions in the column filter picker, showing values Openlayer has already seen, each with a match count, scoped to equality, membership, contains, or ordering comparisons; high-cardinality columns show a Type to search values prompt instead.
    • Adds the ability to pause and unpause tests via the overflow menu (Pause / Unpause) so tests stop computing results during data replays or upstream fixes.

    1 more in Everything

40. docs update Sep 5, 2026 · issue 065

    Openlayer adds data export from records and test-result tables in CSV, XLSX, and JSON formats with score filtering

    └──▷ TRY IT
    Download a monitoring pipeline directly from the terminal without opening the UI.
    copy

    $ openlayer export

    Export records with test scores attached via the platform UI.
    📍1. Go to Records (or a test-result table) in monitoring or development. 2. Open the table options overflow menu and select Export data. 3. In the modal, choose a format: CSV, XLSX, or JSON. 4. Under Scores, select All scores or Scores for {testName}. 5. Select Export data — the download starts automatically when the file is ready.
    • Adds an Export data control to the Records table and test-result tables in both monitoring and development, supporting CSV, XLSX, and JSON output formats.
    • Adds score filtering in the export modal — choose No scores, All scores, or Scores for {testName} to control which test scores are attached to the exported file.
    • Adds openlayer export CLI command to download a monitoring pipeline from the command line.
    • Export jobs are tracked as activity states Exporting data, Exported data, or Failed to export data, giving visibility into export progress.
    • Export access is gated by an Export data permission — Viewer role cannot export.

    5 details in Everything

41. docs update Sep 5, 2026 · issue 065

    Openlayer adds AI-powered trace, session, and test summaries with workspace- and project-level controls and semantic searchsemantic searchSearching by meaning rather than requiring the same words. It commonly compares vector representations to find related content, even when the wording differs.

    └──▷ HOW TO FIND IT
    Automatically summarize test results as they finish computing so your team gets an immediate failure-mode digest without manual review.
    📍In the workspace, go to Settings → AI features → Test summaries, enable the feature, and toggle on Generate automatically.
    Lock AI summary settings workspace-wide so projects cannot override them, while still letting individual projects swap the underlying model.
    📍In the workspace, go to Settings → AI features, open the desired summary section (Trace summaries, Session summaries, or Test summaries), enable Locked for all projects, and set the workspace-level Model.
    • Adds AI summary button in the trace details view to summarize, analyze, and get up to speed with a trace without reading line-by-line spans.
    • Adds Session summaries feature that summarizes user sessions, configurable under Settings → AI features.
    • Adds Test summaries feature that summarizes test result sets, with an optional Generate automatically control to summarize results as they finish computing, configurable under Settings → AI features.
    • Adds Trace summaries feature for individual traces, configurable under Settings → AI features at workspace or project level.
    • Adds workspace-level Locked for all projects control per AI summary feature, preventing projects from enabling, disabling, or changing automatic generation (per-project model overrides still apply).

    +3 more show less

    • Adds AI usage section under Workspace settings → AI features showing spend and volume broken down by feature over a selectable time range, estimated using token prices from Workspace settings → LLM costs.
    • Adds semantic search for goal subpopulations that matches by meaning instead of exact text, requiring a ClickHouse-backed pipeline; the embedding modelembedding modelLists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG. is set per deployment via the EMBEDDING_MODEL environment variable.
    • Enables per-project override of workspace AI summary defaults via Settings → AI features within each project, mirroring workspace controls for Trace summaries, Session summaries, and Test summaries.

    8 details in Everything

    └──▷ BREAKING ON UPGRADE

    • !Changing the EMBEDDING_MODEL value requires re-embeddingembeddingLists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG. all stored data because existing vectors are not comparable across models.
42. docs update Sep 5, 2026 · issue 065

    Openlayer adds four new integrations: Google Conversational Search, Portkey, Palantir Foundry AIP Agents, and Databricks AI/BI Genie.

    • Adds tracing for Google Cloud Discovery Engine conversational search calls via the Openlayer Ruby SDK.
    • Adds tracing for Portkey completions with Openlayer.
    • Adds monitoring support for Palantir Foundry AIP Chatbots with Openlayer.
    • Adds discovery and conversation monitoring for Databricks AI/BI Genie spaces in Openlayer.

    4 details in Everything

43. docs update Sep 5, 2026 · issue 065

    Openlayer adds gateway guardrails, PIIPIIPersonally Identifiable Information — data that can identify a specific individual, such as names, email addresses, or SSNs. Cyber tools flag or redact it to limit exposure and meet privacy regulations. redaction, A2A protocolA2A protocolAn open protocol from Google for communication between AI agents, letting them discover each other's capabilities and exchange tasks without custom integration code. tracing, bulk delete API, and integrations with Portkey, Databricks, and Palantir.

    └──▷ HOW TO FIND IT
    Enforce a guardrail on every gateway request to block prompt injectionprompt injectionAn attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. attempts before they reach your model.
    📍In the Openlayer console, go to Gateway › Guardrails, enable Stop prompt injection, and apply the policy to the target key, user, or group.
    Export test-result records as JSON for offline analysis or compliance archiving.
    📍In the Openlayer console, open a project's test-result table, click Export data, and select JSON (CSV and XLSX are also available).
    • Adds Guardrails gateway feature to block or redact PII, stop prompt injection, and screen for unsafe content on requests and responses.
    • Adds Redact PII capability to strip personally identifiable information from trace data before it is stored.
    • Adds bulk-delete REST endpoint to delete multiple records by their IDs from a data source (formerly inference pipeline).
    • Adds Set usage limits gateway feature to cap spend, requests, or tokens per key, user, or group, enforced on every request.
    • Adds Assign groups gateway feature to group API keys so they share usage limits and guardrails.

    +16 more show less

    • Adds Send invites gateway feature to let people create their own API key from a link, pre-bound to a group and usage limits.
    • Adds Model access gateway control to allow or block models per key, user, group, or the whole deployment.
    • Adds Data retention project setting to configure how long a project keeps monitoring data, with nightly cleanup of data older than the retention period.
    • Adds Update existing traces capability to update or delete traces previously published to the Openlayer platform.
    • Adds Export data capability to download records from the Records table or a test-result table as CSV, XLSX, or JSON.
    • Adds A2AA2AAgent2Agent, an open protocol started by Google for AI agents from different vendors to discover each other, delegate tasks and exchange results. It complements MCP, which connects an agent to tools. Protocol integration to trace multi-agent systems communicating over the A2A protocol as one merged trace or as linked per-project traces.
    • Adds Portkey integration to trace and monitor Portkey completions with Openlayer.
    • Adds Databricks AI/BI Genie integration to discover Genie spaces and monitor their conversations in Openlayer.
    • Adds Palantir Foundry AIP Agents integration to connect Palantir Foundry AIP Chatbots for trace ingestion, evaluation, and monitoring.
    • Adds Project lifecycle governance feature to advance a project through Intake, Development, Staging, Production, and Retired stages.
    • Adds AI-generated summaries for sessions, traces, and test results.
    • Adds Workspace dashboard to scan recently viewed items, portfolio health, project status, compliance, and cost across a workspace.
    • Adds Project dashboard to track a project's metrics, tests, cost, and activity.
    • Adds In-app notifications center to read workspace and project events and configure which app events are received.
    • Adds Track usage gateway view to see what each key, user, and group is consuming and what it costs.
    • Adds bulk test management on the Results page to run, pause, duplicate, tag, archive, or delete multiple tests at once.

    21 details in Everything

44. docs update Aug 19, 2026 · issue 050

    Openlayer MCPMCPModel Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. server launches with remote and local connectors, OAuth, and purpose-built tools for LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. pipeline management.

    ⤢
    └──▷ HOW TO FIND IT
    Add the remote MCP connector in any MCP client to get instant access to Openlayer tools without installing anything locally.
    📍https://mcp.openlayer.com/mcp
    Point the local server at a self-hosted Openlayer instance instead of Openlayer Cloud.
    copy

    $ OPENLAYER_API_KEY=YOUR_OPENLAYER_API_KEY OPENLAYER_BASE_URL=https://openlayer.example.com uvx openlayer-mcp

    • Adds a remote MCP connector at https://mcp.openlayer.com/mcp using Streamable HTTPStreamable HTTPAn MCP transport protocol that lets a client send JSON-RPC requests over HTTP POST while the server streams responses back using Server-Sent Events, replacing the older stdio and SSE transports for remote tool communication. transport with OAuth sign-in — no local install or API key required.
    • Adds a local MCP server via uvx openlayer-mcp using stdio transport, authenticated with the OPENLAYER_API_KEY environment variable.
    • Adds OPENLAYER_BASE_URL environment variable to the local server to point at a self-hosted Openlayer instance instead of Openlayer Cloud.
    • Exposes push_commit and call_openlayer_api tools exclusively on the local connector — push_commit reads a local directory, call_openlayer_api provides freeform HTTP API access.
    • Exposes create_presigned_upload_url and finalize_commit on the remote connector as the upload path for clients with their own file/shell access (e.g. Claude Code), using a storage_uri returned from the presigned upload.

    +6 more show less

    • Adds list_workspaces tool to identify which workspace a given MCP connection is bound to, supporting multiple simultaneous workspace connections.
    • Supports self-hosted deployments serving the remote MCP connector at <OPENLAYER_URL>/mcp, authenticating against the on-premise OAuth2OAuth2An open authorization framework (RFC 6749) that lets a service grant limited access to a user's resources on another service without exposing credentials, giving cyber tools a standard way to authenticate against third-party APIs. Authorization Server with no external identity provideridentity providerA service that authenticates users and supplies identity information to other applications. It is the sign-in authority behind many single sign-on systems..
    • Tools are annotated as read-only or destructive, enabling MCP clients to auto-approve reads and prompt confirmation for writes.
    • Supports listing, creating, and inspecting projects, commits, data sources (inference pipelines), production traces, sessions, and per-user activity with filters.
    • Supports listing and creating tests — integrity, consistency, performance, and LLM-as-a-judge — plus triggering evaluation, reading results, and drilling into failed rows.
    • Connected apps (OAuth sessions) are reviewable and revocable per workspace under Settings → Security → Connected apps.

    11 details in Everything

45. docs update Aug 19, 2026 · issue 050

    Openlayer CLI gains a tests command to export, archive, and delete project tests (goals)

46. updated seen Aug 12, 2026 · issue 043

    Openlayer adds row-level test result API, project deletion API, MCPMCPModel Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. custom functions, and metric bundle downloads

    ⤢ ⤢ ⤢ ⤢
    • New API endpoint to pull individual rows from specific test results, enabling programmatic drill-down into evaluation data.
    • New API endpoint to delete projects, available via the TypeScript SDK.
    • MCP integration now supports custom functions callable from Openlayer.
    • Custom metrics bundles are now viewable and downloadable directly from the UI.

    4 details in Everything

    └──▷ BREAKING ON UPGRADE

    • !Openlayer system tags are renamed to categories; user-defined labels remain tags.
47. changelog Aug 11, 2026 · issue 042

    Openlayer adds remote MCPMCPModel Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. connector, Agent Skills, semantic searchsemantic searchSearching by meaning rather than requiring the same words. It commonly compares vector representations to find related content, even when the wording differs., webhooks for test results, and 25+ new capabilities in its August 2026 release.

    ⤢ ⤢ ⤢ ⤢
    └──▷ TRY IT
    Bootstrap a new project from scratch — tracing, links, and monitoring configured in one shot — without manual setup steps.
    copy

    $ openlayer init

    • Adds openlayer init — one guided command takes a fresh directory to a pushable, linked, monitored project with tracing instrumentation.
    • Adds non-interactive CLI mode — link, login, and resource creation run headless, enabling use from AI coding agents and CI pipelines.
    • Expands openlayer export to support CSV and XLSX formats (previously JSON only) and adds the ability to include scores in exports.
    • Adds webhooks for test results — subscribe a webhook endpoint to test-suite outcomes and get notified when tests pass or fail.
    • Adds multimodalmultimodalAble to work with more than one type of input or output, such as text, images and audio. A multimodal model can combine information across those formats. support to the /responses and /messages Gateway endpoints — media sent and received, not just text.

    +29 more show less

    • Adds a new PATCH endpoint to update a project's name and description via API, picked up by the Python, Go, Java, and Ruby SDKs and the MCP server.
    • Expands workspace API key surface — keys now work for updating projects, cost-override endpoints, test retrieval/update/deletion, and seven previously session-only read routes.
    • Adds sorting and filtering to more list endpoints — workspace rules, project members, and user sessions; rules tables also gain server-side group filtering.
    • Adds a remote, OAuth-protected MCP connector — the MCP server runs hosted instead of local-only, usable in Claude Code and claude.ai, with in-chat workspace switching.
    • Adds governance and lifecycle tools via MCP — frameworks, rules, and rule results are reachable through MCP, growing the total toolset from 44 to 53 tools.
    • Adds Openlayer Agent Skills — packaged skills that teach AI coding assistants to integrate a codebase with Openlayer (tracing, offline evals, tests, guardrails, CI gating); ships as a Claude Code plugin, Cursor plugin, or via the skills CLI.
    • Adds dedicated Gemini and Vertex tracing in Python and TypeScript for the unified Gemini SDK including Vertex, where previously only the legacy SDK was auto-instrumented.
    • Adds Claude Agent SDK tracing for TypeScript — drop-in import auto-traces agent runs.
    • Adds native IBM watsonx Orchestrate integration, modeled on the existing Agentforce and Copilot Studio integrations.
    • Adds semantic (model-based) routing — routing rules can score each prompt in-process and route it to a strong or weak target by threshold, alongside static first-match rules.
    • Adds per-user usage limits and guardrails in the Gateway — limits and guardrails can target an individual user, including per-user caps within a team.
    • Adds a content safety guardrail type for Gateway requests and responses.
    • Adds a missing-guardrails governance rule — a new integrity test that flags traces missing an expected guardrail.
    • Adds US National Provider Identifier (NPI) as a native PIIPIIPersonally Identifiable Information — data that can identify a specific individual, such as names, email addresses, or SSNs. Cyber tools flag or redact it to limit exposure and meet privacy regulations. check — detectable in Contains-PII tests and redactable in traces.
    • Adds per-project data retention — set a rolling retention window (7/30/90/180/365 days or indefinite); monitoring data older than the window is deleted.
    • Adds the ability to pause and resume individual monitoring tests without disabling the whole project.
    • Adds bulk session deletion — multi-select sessions and delete with a full cascade to their traces, scores, and embeddingsembeddingsLists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG..
    • Adds workspace-level dashboards — at-a-glance cost, usage, and project health across the workspace with drill-in widgets.
    • Adds AI-generated session, trace, and test-result summaries — session and trace detail views open with an AI summary; test results include a summary of dominant failure modes with drill-down to exact rows.
    • Adds semantic search filters — embeddingembeddingLists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG.-based subpopulation filtering for tests, beyond exact and keyword matching.
    • Adds an official Helm chart for the Gateway for Kubernetes deployments.
    • Adds a deployment admin area — operators get background-job queue health, metrics, history, and license/feature status, gated by a dedicated deployment-admin role.
    • Adds granular token usage and cost tracking — cached, reasoning, and audio token categories are priced and stored; LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code.-as-judge evaluator runs are cost-tracked per test result.
    • Adds Gateway spend visibility — per-model breakdown on the usage endpoint, a per-user spend aggregation endpoint, projected savings when creating a limit, key rotation, and provider secrets settable from the admin UI.
    • Adds browser-based CLI login — sign in with a device-code browser flow instead of pasting an API key.
    • Adds suggested filter values — filter fields suggest likely values as you type, served from the backend and respecting other active filters.
    • Adds bulk test actions — duplicate several tests at once and schedule runs for several tests at once; view, download, and delete uploaded custom metric files.
    • Adds on-prem deployment options — CSICSIAn ambiguous acronym spanning Container Storage Interface, Crime Scene Investigation, and other expansions — without more context about which domain this tool operates in, a reliable definition cannot be given. Secrets Store support, external/managed ClickHouse, native Oracle Object Storage as a storage backend, and a multi-arch Gateway image for ARMARMA family of RISC processor architectures designed by Arm Holdings and licensed to chip makers worldwide; tools target it to run natively on mobile, embedded, and increasingly server and desktop hardware. hosts.
    • Adds a free public LLM cost explorer tool at llm-costs.openlayer.com for browsing and comparing LLM pricing across providers, including per-token-category pricing (cached, reasoning, audio).

    34 details in Everything

48. docs update Aug 11, 2026 · issue 042

    Openlayer Skills package conditions AI coding agents (Claude Code, Cursor, Codex, Copilot, Gemini CLI) to follow Openlayer's current best practices end to end.

    └──▷ TRY IT
    Install the Openlayer skill into all detected coding agents on your machine in one command.
    copy

    $ npx skills add openlayer-ai/openlayer-skills --skill "openlayer"

    Authenticate your agent to Openlayer, or point it at a self-hosted instance.
    copy

    $ export OPENLAYER_API_KEY=<your-api-key>
    export OPENLAYER_BASE_URL=<your-instance-url>  # self-hosted only

    • Adds npx skills add openlayer-ai/openlayer-skills --skill "openlayer" via the skills CLI to install the Openlayer agent skill into Codex, GitHub Copilot, Gemini CLI, Windsurf, opencode, and dozens of other coding agents in one step.
    • Supports manual installation for any agent that reads a skills directory by symlinking or copying the skills/openlayer folder from github.com/openlayer-ai/openlayer-skills.
    • Skill covers tracing/monitoring via the Python or TypeScript SDK, offline evals with openlayer.json + tests.json, test and guardrail creation from the catalog, and CI/CD build gating on eval regressions.
    • Integrates the Openlayer MCPMCPModel Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. server so agents can query the workspace and search Openlayer documentation directly.
    • Supports both LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code./agent applications and traditional ML models (scikit-learn, XGBoost, and similar).

    5 details in Everything

49. docs update Aug 11, 2026 · issue 042

    Openlayer MCPMCPModel Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. server now available as a hosted, OAuth-protected remote connector at https://mcp.openlayer.com — no local install required.

    ⤢
    └──▷ TRY IT
    Register the Openlayer remote MCP server in Claude Code so the assistant can access your Openlayer workspace without managing an API key.
    copy

    $ claude mcp add --transport http openlayer https://mcp.openlayer.com

    Add the Openlayer MCP server to a VS Code project for use with GitHub Copilot or other MCP-aware extensions.
    json copy

    {
      "servers": {
        "openlayer": {
          "type": "http",
          "url": "https://mcp.openlayer.com"
        }
      }
    }

    Bridge the remote connector through mcp-remote for clients (e.g. Windsurf, Zed) that lack native remote-MCP + OAuth support.
    json copy

    {
      "command": "npx",
      "args": ["-y", "mcp-remote", "https://mcp.openlayer.com"]
    }

    • Adds a hosted, OAuth-protected remote MCP connector at https://mcp.openlayer.com — no API key or local install needed; sign-in is handled via OAuth against your Openlayer account.
    • Supports workspace switching directly from the chat interface when a user belongs to multiple workspaces.
    • Integrates with Claude desktop/claude.ai via Settings → Connectors → Add custom connector using https://mcp.openlayer.com.
    • Integrates with Claude Code via claude mcp add --transport http openlayer https://mcp.openlayer.com.
    • Integrates with Cursor via .cursor/mcp.json with "url": "https://mcp.openlayer.com".

    +3 more show less

    • Integrates with VS Code via the MCP: Add Server command palette action (HTTP type) or .vscode/mcp.json with "type": "http" and "url": "https://mcp.openlayer.com".
    • Integrates with Codex via codex mcp add openlayer --url https://mcp.openlayer.com.
    • Supports clients without native remote-MCP + OAuth via mcp-remote bridge: npx -y mcp-remote https://mcp.openlayer.com.

    8 details in Everything

50. docs update Aug 11, 2026 · issue 042

    Openlayer adds semantic searchsemantic searchSearching by meaning rather than requiring the same words. It commonly compares vector representations to find related content, even when the wording differs. filters for test subpopulations, matching rows by meaning rather than exact wording.

    └──▷ HOW TO FIND IT
    Narrow a test to rows about billing issues even when users phrase the problem differently (charged twice, invoice looks wrong).
    📍In your project, go to Tests and create or edit a test. Under subpopulation filters, select the free-text column (e.g. user message), set the match type to semantic, and enter the phrase billing problems. The test will evaluate only rows semantically similar to that phrase.
    • Adds a semantic match type in test subpopulation filters, using embeddingsembeddingsLists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG. to match rows by topic or intent rather than exact or keyword wording — useful for free-text columns like user messages, model outputs, and transcripts.
    • Semantic filters compose with existing metadata conditions (date ranges, user IDs, tags) to narrow the exact data slice a test runs on.

    2 details in Everything

51. docs update Aug 11, 2026 · issue 042

    Openlayer adds AI-generated summaries for sessions, traces, and test results to surface failure patterns without raw log reading.

    • New AI summaries on the session detail view explain what the user was trying to do, what the agent did, and where it struggled — accessible via Projects > Sessions > open a session.
    • New AI summaries on the trace detail view replace line-by-line span reading — accessible by opening any trace from the Traces view.
    • New AI summaries on test results surface dominant failure modes and link each mode to the exact matching rows — accessible by opening a test result above the row-level results.
    • Summaries are generated on demand and cached so that reopening the same session or result is instant.

    4 details in Everything

52. docs update Aug 11, 2026 · issue 042

    Openlayer adds support for monitoring IBM watsonx Orchestrate agents

53. docs update Aug 11, 2026 · issue 042

    Openlayer adds AI-generated summaries, semantic searchsemantic searchSearching by meaning rather than requiring the same words. It commonly compares vector representations to find related content, even when the wording differs. filters, agent skills for coding assistants, and IBM Watson Orchestrate integration.

    • Adds AI-generated summaries for sessions, traces, and test results so practitioners see a plain-language overview without reconstructing events span by span.
    • Adds semantic search filters for building test subpopulations using embeddingembeddingLists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG.-based matching instead of exact or keyword matching.
    • Adds Openlayer Agent Skills to teach AI coding assistants (Claude Code, Cursor) how to integrate a codebase with Openlayer.
    • Adds IBM watsonx Orchestrate integration for agent tracing, evaluation, and monitoring.

    4 details in Everything

54. updated seen Aug 11, 2026 · issue 042

    Openlayer adds AI summaries, semantic searchsemantic searchSearching by meaning rather than requiring the same words. It commonly compares vector representations to find related content, even when the wording differs., remote MCPMCPModel Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface., webhooks, per-user Gateway limits, and a Helm chart for the Gateway in its August 2026 release.

    ⤢ ⤢ ⤢ ⤢
    └──▷ TRY IT
    Bootstrap a new project with tracing fully wired in one guided step, suitable for kicking off a new LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. app or CI environment.
    copy

    $ openlayer init

    • Adds openlayer init CLI command to guide a fresh directory to a linked, monitored project with tracing instrumentation in one step.
    • Adds non-interactive (headless) CLI mode — link, login, and resource creation now accept no user prompts, enabling use from AI coding agents and CI pipelines.
    • Expands openlayer export to support CSV and XLSX output formats (previously JSON only) and adds a flag to include scores in the export.
    • Adds openlayer.init() automatic instrumentation for Python — one call patches every installed supported LLM SDK (OpenAI, Anthropic, Mistral, Groq, Gemini, LiteLLM, Portkey, Google ADKADKGoogle's Agent Development Kit, an open-source framework for building and orchestrating AI agents; cyber tools use it to compose multi-agent workflows with built-in tool-calling and state management., Azure); opt out with auto_instrument=False.
    • Introduces webhooks for test results — subscribe a webhook endpoint to test-suite outcomes and receive notifications when tests pass or fail.

    +39 more show less

    • Adds /responses and /messages Gateway endpoints with multimodalmultimodalAble to work with more than one type of input or output, such as text, images and audio. A multimodal model can combine information across those formats. support, accepting and returning media as well as text.
    • Expands API-key surface so workspace API keys now cover updating projects, cost-override endpoints, test retrieval/update/deletion, and seven previously session-only read routes.
    • Adds a new PATCH endpoint to update a project's name and description, surfaced in the Python, Go, Java, and Ruby SDKs and the MCP server.
    • Adds sorting and filtering to more list endpoints: workspace rules, project members, and user sessions, plus server-side group filtering on rules tables.
    • Adds a per-user spend aggregation endpoint and a per-model breakdown on the usage endpoint for Gateway spend visibility.
    • Adds Gemini and Vertex tracing in Python and TypeScript via a dedicated tracer for the unified Gemini SDK, including Vertex AI (previously only the legacy SDK was auto-instrumented).
    • Adds Claude Agent SDK tracing for TypeScript — drop-in import auto-traces agent runs.
    • Adds Claude Agent SDK tracing for Python — instruments the full agent loop, capturing tool calls, MCP tools, subagent nesting, session continuity, cost, and tokens.
    • Adds a remote, OAuth-protected MCP connector — the MCP server runs hosted (not local-only), usable in Claude Code and claude.ai, with in-chat workspace switching.
    • Expands MCP tooling to 53 tools (up from 44), adding governance/lifecycle access (frameworks, rules, rule results) and 12 new session, trace, and test-lifecycle tools.
    • Introduces Openlayer Agent Skills — packaged skills for AI coding assistants (Claude Code plugin, Cursor plugin, or skills CLI) covering tracing, offline evals, tests, guardrails, and CI gating for both LLM and traditional/tabular ML workflows.
    • Adds native IBM watsonx Orchestrate integration, modeled on the existing Agentforce and Copilot Studio integrations.
    • Adds semantic (model-based) routing — routing rules can score each prompt in-process and send it to a strong or weak target by threshold, alongside existing static first-match rules.
    • Adds per-project data retention with a configurable rolling window (7, 30, 90, 180, or 365 days, or indefinite); monitoring data older than the window is deleted.
    • Adds per-user usage limits and guardrails in the Gateway, including per-user caps within a team.
    • Adds a content safety guardrail type for Gateway requests and responses.
    • Adds a missing-guardrails governance rule — a new integrity test that flags traces missing an expected guardrail.
    • Adds US National Provider Identifier (NPI) as a native PIIPIIPersonally Identifiable Information — data that can identify a specific individual, such as names, email addresses, or SSNs. Cyber tools flag or redact it to limit exposure and meet privacy regulations. check, detectable in Contains-PII tests and redactable in traces.
    • Adds project lifecycle stages and evidence documents — attach a lifecycle stage and evidence to a project for governance tracking.
    • Adds rule deactivation reasons — supply a reason when deactivating a rule; it is displayed on the rule page.
    • Adds workspace-level dashboards with at-a-glance cost, usage, and project health across the workspace, including drill-in widgets.
    • Adds AI-generated summaries to session, trace, and test-result detail views, with drill-down from failure modes to the exact rows behind each pattern.
    • Adds semantic (embeddingembeddingLists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG.-based) search filters for subpopulation filtering in tests, beyond exact and keyword matching.
    • Adds pause and resume for individual monitoring tests without disabling the whole project.
    • Adds bulk session deletion — multi-select sessions and delete with full cascade to their traces, scores, and embeddingsembeddingsLists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG..
    • Adds a Helm chart for the Gateway for Kubernetes deployments.
    • Adds a deployment admin area for on-prem operators with background-job queue health, metrics, history, and license/feature status, gated by a dedicated deployment-admin role.
    • Adds browser-based CLI login via device-code browser flow instead of requiring an API key to be pasted.
    • Adds bulk test actions — duplicate or schedule runs for several tests at once, and view, download, or delete uploaded custom metric files.
    • Adds a timeline (Gantt-style) view for traces showing every span's start time, duration, and nesting side by side.
    • Adds Gateway API key invite links — admins can generate links that let users self-provision their own Gateway API keys.
    • Adds compliance test visibility to results pages — shows which tests are required for compliance, in context.
    • Introduces a free public LLM cost explorer at llm-costs.openlayer.com for browsing and comparing LLM pricing across providers, including per-token-category pricing (cached, reasoning, audio).
    • Adds granular token usage tracking — cached, reasoning, and audio token categories are now priced and stored, with per-API-key cost visible in the UI; LLM-as-judge evaluator runs are cost-tracked per test result.
    • Adds LangChain ToolMessage.artifact capture (including document lists, promoted to trace-level context) in the Python and TypeScript SDKs.
    • Unifies trace configuration around init() in the SDK — settings merge incrementally instead of resetting on each call (configure() is deprecated in favor of init()).
    • Adds on-prem support for CSICSIAn ambiguous acronym spanning Container Storage Interface, Crime Scene Investigation, and other expansions — without more context about which domain this tool operates in, a reliable definition cannot be given. Secrets Store, external/managed ClickHouse, native Oracle Object Storage as a storage backend, and a multi-arch Gateway image for ARMARMA family of RISC processor architectures designed by Arm Holdings and licensed to chip makers worldwide; tools target it to run natively on mobile, embedded, and increasingly server and desktop hardware. hosts.
    • Adds the ability to rename data sources directly from the data source row menu.
    • Adds row data download in development mode.

    44 details in Everything

    └──▷ BREAKING ON UPGRADE

    • !configure() is deprecated in favor of init() for trace configuration in the SDK; settings now merge incrementally instead of resetting on each call.
55. AI summaries, semantic search filters, and the remote MCP connector Aug 8, 2026 · release notes

    Openlayer vAI adds semantic searchsemantic searchSearching by meaning rather than requiring the same words. It commonly compares vector representations to find related content, even when the wording differs. filters, remote MCPMCPModel Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. connector, AI summaries, and openlayer init CLI.

    └──▷ USE IT
    Bootstrap a new project with tracing instrumentation using the new CLI command
    shell copy

    openlayer init --guided

    Update a project's name and description via the expanded API surface
    shell copy

    curl -X PATCH https://api.openlayer.com/api/projects/<project_id> \
      -H 'Authorization: Bearer <WORKSPACE_API_KEY>' \
      -H 'Content-Type: application/json' \
      -d '{
        "name": "New Project Name",
        "description": "Updated description for vAI release"
      }'

    Export test results to CSV and XLSX formats including scores using the new CLI capabilities
    shell copy

    openlayer export --format csv,xlsx --include-scores > results.csv

    • Adds semantic search filters for embeddingembeddingLists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG.-based subpopulation filtering of tests
    • Introduces the remote, OAuth-protected MCP connector with in-chat workspace switching
    • Opens sessions, traces, and test results with AI summaries of dominant failure modes
    • Adds openlayer init guided CLI command to bootstrap a monitored project
    • Enables non-interactive CLI mode for link, login, and resource creation via headless execution

    +15 more show less

    • Supports native IBM watsonx Orchestrate integration through Agent Skills
    • Adds /responses and /messages endpoints support for media (multimodalmultimodalAble to work with more than one type of input or output, such as text, images and audio. A multimodal model can combine information across those formats. Gateway)
    • Introduces content safety guardrail type for Gateway requests and responses
    • Allows setting per-user usage limits and guardrails in the Gateway
    • Provides bulk session deletion with cascade to traces, scores, and embeddingsembeddingsLists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG.
    • Enables pause and resume of individual monitoring tests without disabling the project
    • Adds per-project data retention windows (7/30/90/180/365 days or indefinite)
    • Exposes a new PATCH endpoint to update a project's name and description via API
    • Expands workspace API keys to cover project updates, cost-override endpoints, and test retrieval/deletion
    • Adds server-side group filtering for workspace rules, project members, and user sessions list endpoints
    • Supports CSV and XLSX export formats in openlayer export, including scores
    • Reveals granular token usage by category (cached, reasoning, audio) per API-key in the UI
    • Adds US National Provider Identifier as a native PIIPIIPersonally Identifiable Information — data that can identify a specific individual, such as names, email addresses, or SSNs. Cyber tools flag or redact it to limit exposure and meet privacy regulations. check detectable in Contains-PII tests
    • Introduces Missing-guardrails governance rule to flag traces lacking expected guardrails
    • Deploys an official Helm chart for Gateway Kubernetes deployments (On-Prem)

    20 details in Everything

56. updated seen Aug 6, 2026 · issue 037

    June 2026: Gantt-style trace timeline view makes slow spans and unexpected branches immediately visible.

    ⤢ ⤢ ⤢ ⤢
    • New Gantt-style trace timeline view lines up every span by start time, duration, and nesting to surface slow steps and unexpected branches at a glance.

    1 detail in Everything

57. docs update seen Aug 1, 2026 · issue 032

    Openlayer adds a guided CLI init command and multimodalmultimodalAble to work with more than one type of input or output, such as text, images and audio. A multimodal model can combine information across those formats. trace support for images, audio, and files.

    • New init CLI command sets up Openlayer in a project through a single guided flow.
    • Supports attaching images, audio, and files to traces for multimodal LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. monitoring.

    2 details in Everything

58. docs update seen Jul 24, 2026 · issue 024

    Openlayer CLI gains data-sources, profile, projects, tests, and update commands; adds tabular ML prediction publishing.

    └──▷ TRY IT
    Export all test definitions for a project to review or version-control your monitoring goals.
    copy

    $ openlayer tests

    Switch between workspaces or CI service accounts by managing named authentication profiles.
    copy

    $ openlayer profile

    • New data-sources CLI command to manage Openlayer data sources directly from the terminal.
    • New profile CLI command to manage authentication profiles for the Openlayer CLI.
    • New projects CLI command to manage Openlayer projects from the CLI.
    • New tests CLI command to export test definitions (goals) for an Openlayer project.
    • New update CLI command to self-update the Openlayer CLI to the latest version.

    +1 more show less

    • Supports publishing traditional/tabular ML model predictions to Openlayer for monitoring, streamed in real-time or uploaded in batches.

    6 details in Everything

59. docs update seen Jul 24, 2026 · docs issue 004

    Openlayer CLI gains data-sources, profile, projects, tests, and update commands; adds tabular ML prediction publishing.

    └──▷ TRY IT
    Export all test definitions for a project to review or version-control your monitoring goals.
    copy

    $ openlayer tests

    Switch between workspaces or CI service accounts by managing named authentication profiles.
    copy

    $ openlayer profile

    • New data-sources CLI command to manage Openlayer data sources directly from the terminal.
    • New profile CLI command to manage authentication profiles for the Openlayer CLI.
    • New projects CLI command to manage Openlayer projects from the CLI.
    • New tests CLI command to export test definitions (goals) for an Openlayer project.
    • New update CLI command to self-update the Openlayer CLI to the latest version.

    +1 more show less

    • Supports publishing traditional/tabular ML model predictions to Openlayer for monitoring, streamed in real-time or uploaded in batches.

    6 details in Everything

60. Batch test re-runs and expanded integrations seen Jul 13, 2026 · issue 014

    Openlayer adds batch test re-runs, Ruby SDK, Gemini/Copilot Studio integrations, and new drift detection.

    ⤢
    └──▷ TRY IT
    Retrieve test results programmatically to pipe into downstream alerting or reporting pipelines.
    copy

    $ curl -X GET 'https://api.openlayer.com/v1/projects/<project-id>/test-results' \
      -H 'Authorization: Bearer <api-key>'

    • Adds batch and manual test re-runs so results can be recalculated retroactively when new data is added.
    • Supports automatic threshold configuration for tests built on custom metrics.
    • Introduces custom time range selection for filtering test execution history.
    • New primary key uniqueness constraint test type available on the platform.
    • Adds Ruby SDK for instrumenting Openlayer in Ruby applications.

    +11 more show less

    • New Strands Agents SDK integration for tracing agentic workflows.
    • New Microsoft Copilot Studio integration.
    • New Google Gemini API integration.
    • New API endpoints for retrieving test results and managing projects.
    • Adds Characteristic Stability Index (CSICSIAn ambiguous acronym spanning Container Storage Interface, Crime Scene Investigation, and other expansions — without more context about which domain this tool operates in, a reliable definition cannot be given.) as a drift detection method.
    • Adds step detection support in the LangChain integration.
    • Adds model output capture in the Google ADKADKGoogle's Agent Development Kit, an open-source framework for building and orchestrating AI agents; cyber tools use it to compose multi-agent workflows with built-in tool-calling and state management. integration.
    • Smoother Snowflake connection flow now supports views.
    • Simplified Databricks onboarding flow.
    • Redesigned monitoring test results graph for clearer visibility.
    • Clearer trace visualizations in the UI.

    16 details in Everything

61. Complete design system overhaul, Snowflake Integration seen Jul 13, 2026 · issue 014

    Openlayer adds Snowflake integration, new security role, prompt injectionprompt injectionAn attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. tests, and expanded LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. tracing for Bedrock, OpenAI Agents, Pydantic AI, and LangGraph.

    ⤢
    └──▷ HOW TO FIND IT
    Connect a Snowflake table directly to a project to run evals against live warehouse data without manual exports.
    📍In the console, open your project, go to Data Sources › Connect, select Snowflake, and choose the target table.
    Enable a prompt injection test on an LLM data source to automatically detect adversarial inputs in production traffic.
    📍In the console, open your project's Tests tab, click Add Test, and select Prompt Injection from the Evals library.
    • New UI with full light and dark mode support, updated design system, and improved navigation hierarchy.
    • Adds priority levels for tests and more data visualizations throughout the product.
    • SDK support for tracing Bedrock models.
    • SDK support for tracing OpenAI Agents.
    • SDK support for tracing Pydantic AI systems.

    +12 more show less

    • SDK support for tracing LangGraph systems.
    • New Member restricted security role that can perform member actions without viewing data source data.
    • Direct Snowflake table connections to projects via native integration.
    • Project, dataset, and table dropdowns when connecting BigQueryBigQueryA serverless, cloud-hosted analytical database from Google that runs SQL queries across massive datasets without managing infrastructure, giving cyber tools scalable storage and fast querying over large volumes of log or event data. tables.
    • Supports hosting Openlayer on subpaths in on-premises deployments.
    • Allows users to override LLM costs with custom cost values.
    • Includes standard deviation score in LLM-as-a-judge and Ragas test results.
    • New prompt injection test to detect adversarial attacks on LLM systems.
    • Per-row scores now visible for metrics like semantic similarity and exact match in data tables.
    • API endpoints that return record counts and last record date for data sources are now faster.
    • Faster data source creation via API.
    • Supports streaming data with numpy arrays in the API request body.

    17 details in Everything

    └──▷ BREAKING ON UPGRADE

    • !Inference pipelines are renamed to data sources throughout the platform
62. Custom metrics, rotating API keys, and new models for direct-to-API calls seen Jul 13, 2026 · issue 014

    Openlayer adds custom metrics via openlayer.json, rotating API keys, and 11 new models for direct-to-API evaluation calls.

    ⤢
    └──▷ USE IT
    Define a custom domain-specific metric in openlayer.json so it appears as a project-wide metric and a runnable test on all your data.
    json copy

    {
      "metrics": [
        {
          "name": "toxicity_score",
          "description": "Custom toxicity scorer for our domain",
          "type": "custom",
          "script": "metrics/toxicity.py"
        }
      ]
    }

    Rotate API keys by creating a new key before revoking the old one — navigate to your account API settings to manage multiple keys.
    📍In the Openlayer console, go to Account Settings › API Keys, click Create new key, copy the new key into your services, then delete the old key.
    • Supports custom metrics uploaded via openlayer.json, usable as project-wide metrics or as runnable eval tests alongside standard metrics.
    • Enables creation, renaming, and deletion of multiple personal API keys to support API key rotation workflows.
    • Adds ability to specify desired metrics directly in openlayer.json for API-driven evaluation runs.
    • Adds 11 new models for direct-to-API calls: GPT-4o, GPT-4 Turbo, Claude 3.5 Sonnet, Claude 3 Haiku, Claude 3 Opus, Claude 3 Sonnet, Command R, Command R Plus, Gemini 1.0 Flash, Gemini 1.5 Flash, and Gemini 1.5 Pro.
    • Adds git repository and organization linking in git settings pages for tighter version-control integration.

    +1 more show less

    • New UI button to view the processing status of a commit during project loading.

    6 details in Everything

63. Improved quality control over your LLM’s responses with annotations and human feedback seen Jul 13, 2026 · issue 014

    Openlayer adds request annotations, human feedback ratings, data export, and Git integration to LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. quality control.

    ⤢
    └──▷ HOW TO FIND IT
    Review a filtered set of low-quality responses offline by exporting only the flagged requests cohort from your workspace.
    📍In the console, go to your workspace › Requests, apply the desired filters (e.g., thumbs-down ratings), then click Export to download the filtered cohort as a file.
    Connect a project to a Git repository and configure the target branch and root directory for version-controlled evaluation pipelines.
    📍In the console, go to your Project › Settings › Git Repository, enter the repository URL, set the branch and root directory, then save.
    • Adds per-request annotations with custom values, including step-level annotation for individual trace steps when tracing is enabled.
    • Adds thumbs up/thumbs down human feedback ratings on every request to quickly surface good and bad model responses.
    • Enables data export from the UI, including filtered cohorts of request data.
    • Introduces updated navigation with breadcrumbs and the ability to copy project name and inference pipeline ID.
    • Adds project-level Git repository settings, including editable branch and root directory configuration.

    +4 more show less

    • Adds ability to add and edit ground truths on individual requests.
    • Adds filtering of test modal data by the selected evaluation window.
    • Adds more project templates.
    • Improves the OpenAI SDK integration.

    9 details in Everything

64. Improved test diagnosis page, SAML SSO, design refreshes, + more seen Jul 13, 2026 · issue 014

    Openlayer adds SAML SSOSAML SSOAn authentication protocol that lets a user log in once with an identity provider and access multiple services without re-entering credentials, by passing signed XML assertions between parties., Azure OpenAI evaluator, Ragas metrics, and an overhauled test diagnosis page.

    ⤢
    └──▷ HOW TO FIND IT
    Enforce org-level SSOSSOSingle sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. for your team's Openlayer workspace to add a layer of authentication security.
    📍In the console, go to Settings › Collaboration › SAML SSO and configure your identity provider credentials.
    • Adds SAMLSAMLA standard for passing signed identity assertions between a sign-in provider and an application. It lets a user sign in through their organization's identity provider. SSO support via any major identity provideridentity providerA service that authenticates users and supplies identity information to other applications. It is the sign-in authority behind many single sign-on systems., enabling teams to enforce org-level authentication for workspace access.
    • Supports Azure OpenAI as an LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. evaluator integration.
    • Adds additional Ragas metrics — faithfulness and answer correctness — for evaluation.
    • New list view on the test diagnosis page surfaces error messages for skipped and errored tests, with filtering by result type and a status-breakdown overview.
    • Test diagnosis page sections are now collapsible for a streamlined review experience.

    +4 more show less

    • Updated cost table for OpenAI models in the Observability view.
    • Notifications for new inference pipelines now include the pipeline name.
    • More informative tooltips on test cards.
    • Groq guide now available in documentation.

    9 details in Everything

65. Introducing Openlayer Governance seen Jul 13, 2026 · issue 014

    Openlayer adds Governance mode, A/B testing, new data backends, CLI data sources, and expanded tracing support.

    ⤢
    └──▷ HOW TO FIND IT
    Enable Governance mode to define and track enforcement rules across your workspace's AI projects.
    📍In the Openlayer console, open your workspace, navigate to Governance, select New Framework, define rules (e.g. PII leakage test, monitoring required, risk level), assign owners, and track compliance across projects.
    • New Governance mode lets teams define and enforce custom policy frameworks (PIIPIIPersonally Identifiable Information — data that can identify a specific individual, such as names, email addresses, or SSNs. Cyber tools flag or redact it to limit exposure and meet privacy regulations. leakage, prompt injectionprompt injectionAn attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools., monitoring requirements, ownership, risk level) across all projects and workspaces, with rule owners, comment threads, and evidence uploads.
    • New workspace overview and project inventory gives a birds-eye view of project activity with categorization and labeling across the workspace.
    • Adds support for Databricks, Redshift, and Postgres as third-party data backends in Monitoring mode.
    • Enables A/B testing across two commits (Development mode) or two data sources (Monitoring mode) to compare performance across metrics.
    • Supports configuring directory sync for SAMLSAMLA standard for passing signed identity assertions between a sign-in provider and an application. It lets a user sign in through their organization's identity provider./SSOSSOSingle sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management.-enabled workspaces via the UI.

    +12 more show less

    • Adds custom onboarding flow for users purchasing Openlayer through the AWS Marketplace.
    • New API endpoint to delete custom metrics.
    • Adds Python tracing support for OpenAI systems using the .parse() method or Responses API.
    • Adds Python tracing support for systems built with Google ADKADKGoogle's Agent Development Kit, an open-source framework for building and orchestrating AI agents; cyber tools use it to compose multi-agent workflows with built-in tool-calling and state management..
    • Allows configuring trace settings (timeouts, retries) in Python via defined settings.
    • CLI now supports creating data sources directly.
    • CLI now supports deleting custom metrics directly.
    • On-premises deployments can now leverage internal LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. gateways.
    • New UI flow for accepting Assistant-generated tests, allowing users to select a subset, refine suggestions, or retry requests.
    • Enables filtering columns in the Data page by clicking on a column header.
    • Improved third-party data backend experience (Snowflake, Databricks) with faster multi-table connections, schema inference, and remote test execution for large data volumes.
    • Enhanced navigation header for faster movement between projects and modes.

    17 details in Everything

66. Openlayer Gateway, customizable dashboards, and Salesforce Agentforce GA seen Jul 13, 2026 · issue 014

    Openlayer adds a managed AI gateway, customizable dashboards, Salesforce Agentforce GA, Helm on-prem, new guardrails, governance frameworks, and audit trails.

    ⤢ ⤢ ⤢
    └──▷ HOW TO FIND IT
    Route all LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. calls through the Openlayer Gateway to get centralized API key management, rate limiting, and end-to-end observability without changing your model provider code.
    📍In the console, go to Gateway › Admin Portal, create a gateway API key, assign rate limits and team, then point your LLM client's base URL to the Openlayer Gateway endpoint instead of the provider directly.
    Delete stale monitoring records programmatically to keep your monitoring dataset clean.
    copy

    $ curl -X DELETE https://api.openlayer.com/v1/monitoring/records \
      -H 'Authorization: Bearer <your-api-key>' \
      -H 'Content-Type: application/json' \
      -d '{"recordIds": ["<record-id-1>", "<record-id-2>"]}'

    Deploy Openlayer on-prem to your Kubernetes cluster using the new Helm chart from Docker Hub OCIOCIOpen Container Initiative standard, maintained by the Linux Foundation, defining the image format and runtime spec that container tools share so images built once run anywhere OCI-compliant..
    copy

    $ helm install openlayer oci://registry-1.docker.io/openlayer/openlayer \
      --set storage.type=s3 \
      --set database.url=<your-db-url> \
      --set replicaCount=3 \
      --set image.pullSecrets[0]=<pull-secret> \
      --set node.heapSize=4096

    • New Openlayer Gateway: centralized managed control layer for all LLM calls, with admin portal, API key management, per-key rate limits and freeze, and team-based organization.
    • Customizable project home pages: pin metric widgets, view project notifications inline, and navigate alerts from a new sidebar notification center.
    • Salesforce Agentforce integration reaches GA, with configurable login domain (production, sandbox, or My Domain) and updated Builder schema capturing every intermediate step.
    • On-prem Helm chart published to Docker Hub OCI for Kubernetes deployments, with config for storage, database, replicas, image pull secrets, and Node heap.
    • Built-in governance frameworks available out of the box: EU AI ActEU AI ActEU regulation that classifies AI systems by risk level and imposes compliance obligations—documentation, transparency, human oversight—on developers and deployers, shaping how AI-based cyber tools are built and sold in Europe., TRAIGA, E23 MRM, and the Openlayer Governance Framework.

    +30 more show less

    • Full workspace audit trails with activity log export.
    • Workspace-wide PIIPIIPersonally Identifiable Information — data that can identify a specific individual, such as names, email addresses, or SSNs. Cyber tools flag or redact it to limit exposure and meet privacy regulations. redaction on trace processing, with per-project entity-type configuration.
    • New guardrails: Llama Prompt Guard for prompt injectionprompt injectionAn attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools., English and Portuguese toxicity (with chunking and category filtering), and Brazilian PII recognizers (CPF, CNPJ, CEP, phone) with checksum validation.
    • New tests: Bias, Contains PHIPHIProtected Health Information: individually identifiable health data regulated under the US HIPAA law, whose strict handling requirements drive cyber tools to audit, mask, or restrict access to it., and Unauthorized Tool Call.
    • Universal search across projects, sessions, users, and tests from a single search bar.
    • Dialogflow integration added.
    • Azure Content Understanding tracer added to SDKs.
    • PydanticAI OTelOTelAn open standard and set of tools for collecting traces, metrics and logs from applications. It lets services export observability data without tying instrumentation to one monitoring vendor. GenAI attribute parsing added to SDKs.
    • Supports tracing the OpenAI Responses API from the TypeScript SDK.
    • openlayer-ts SDK now accepts .csv datasets.
    • Trace decorator gains a promote parameter to surface nested step outputs to the parent span.
    • Session-level aggregated metrics: cost, latency, and token counts rolled up per session.
    • LLM cost configuration: view and edit per-model pricing used for cost calculations.
    • View, edit, and download custom-metric parameter bundles from the UI.
    • LLM evaluator settings are now configurable for Ragas metrics.
    • Monitoring-data limit notifications added.
    • Test-connection button for LLM configuration.
    • BigQueryBigQueryA serverless, cloud-hosted analytical database from Google that runs SQL queries across massive datasets without managing infrastructure, giving cyber tools scalable storage and fast querying over large volumes of log or event data. data sources now supported without a timestamp column.
    • Browser-based openlayer login via device code flow added to the CLI.
    • New API endpoint to delete records in monitoring mode.
    • New API endpoint to retrieve sessions linked to a test result.
    • dateUpdated filter added to the List Tests API.
    • Set workspace admins via environment variables on on-prem deployment.
    • License usage and version reporting from on-prem deployments to Openlayer.
    • Automatic threshold mode for custom metrics.
    • Batch actions (multi-select and delete) on rows, rules, and projects tables.
    • Filtering and sorting on sessions, users, data sources, projects, rules, and framework progress tables.
    • Tag multiple tests at once.
    • Domain verification added to workspace settings.
    • Workspace setting to restrict workspace creation to deployment admins.

    35 details in Everything

67. Pausing tests, checks for duplicate keys, and new integrations seen Jul 13, 2026 · issue 014

    Openlayer adds Google ADKADKGoogle's Agent Development Kit, an open-source framework for building and orchestrating AI agents; cyber tools use it to compose multi-agent workflows with built-in tool-calling and state management. integration, Snowflake remote execution, pausing tests, duplicate-key checks, and cloud marketplace availability.

    ⤢
    • Adds column distribution graphs for LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. projects on the platform.
    • New SDK integration with Google Agent Development Kit (ADK) for evaluating multi-agent systems.
    • Enables pausing test execution in Monitoring mode.
    • New CLI command to export tests for a project.
    • New test to check for duplicate unique and primary keys.

    +8 more show less

    • Supports connecting to Snowflake views and running tests via remote execution.
    • Enables custom SQL tests that join multiple tables.
    • Supports specifying custom TLS certificates as environment variables to access external services.
    • Adds Azure managed identity support for storage connections (On-Prem).
    • Openlayer now available on AWS, Azure, and Google Cloud marketplaces.
    • Allows defining specific columns to check for nulls in Null Rows tests.
    • Assigns static IPs to Openlayer servers to simplify allowlisting.
    • Renders higher decimal precision for test result values in the UI.

    13 details in Everything

68. Project-level secrets, tracing LLM requests with OpenTelemetry seen Jul 13, 2026 · issue 014

    Openlayer adds project-level secrets, OpenTelemetryOpenTelemetryAn open standard and set of tools for collecting traces, metrics and logs from applications. It lets services export observability data without tying instrumentation to one monitoring vendor. tracing, Azure OpenAI for evals, and new platform APIs.

    ⤢
    └──▷ HOW TO FIND IT
    Store a sensitive API key once at the project level so all evals and tests can reference it without hardcoding credentials.
    📍In the Openlayer console, go to your Project › Settings › Secrets and add a new secret (e.g., OPENAI_API_KEY). Reference it by name in your test configurations.
    Set a deployment-wide default LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code.-as-a-judge API key for all on-prem workspaces so teams don't need to configure keys individually.
    📍In the Openlayer on-prem admin console, go to Deployment Settings › Default API Keys and set the key for LLM-as-a-judge.
    • Adds project-level secrets storage in the Platform so API keys and auth tokens can be securely referenced across tests without hardcoding.
    • Enables setting default LLM-as-a-judge API keys for an entire on-premise deployment, eliminating per-workspace/project key configuration.
    • Adds SDK support for tracing LLM requests with OpenTelemetry.
    • Supports Azure OpenAI deployments as the backend for LLM-as-a-judge evaluations.
    • Adds gpt-4o-mini as an available LLM evaluator model.

    +6 more show less

    • Adds JSON schema validation test type to Evals.
    • New API endpoints for workspace creation/retrieval, API key creation, and member invitations.
    • New /versions/{id} endpoint on the public API for retrieving a specific commit/version by ID.
    • New SDK endpoint to retrieve a commit by ID.
    • Adds search, sort, and filter for inference pipelines in the Platform UI and via the API.
    • Adds default test cases and metrics to LLM project templates.

    11 details in Everything

69. SAML Directory Sync, new LLM-as-a-judge models, and website refresh seen Jul 13, 2026 · issue 014

    Openlayer adds SAMLSAMLA standard for passing signed identity assertions between a sign-in provider and an application. It lets a user sign in through their organization's identity provider. directory sync, Cohere/Vertex AI LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code.-as-a-judge models, new RAGRAGRetrieval-augmented generation: looking up relevant documents and adding them to the prompt before the model answers. It grounds responses in private or current data without retraining. metrics, Viewer role, and expanded APIs.

    ⤢
    └──▷ TRY IT
    Include project and workspace details in a single inference pipeline GET call, avoiding extra round-trips.
    copy

    $ curl -X GET 'https://api.openlayer.com/v1/inference-pipelines/<id>?expand=project,workspace' \
      -H 'Authorization: Bearer <token>'

    Check how many tests are skipped or failing on a project version without fetching full test details.
    copy

    $ curl -X GET 'https://api.openlayer.com/v1/projects/<project-id>/versions/<version-id>' \
      -H 'Authorization: Bearer <token>'

    Enable SAML directory sync so workspace membership mirrors your identity provideridentity providerA service that authenticates users and supplies identity information to other applications. It is the sign-in authority behind many single sign-on systems.'s security groups.
    📍In the Openlayer console, go to Workspace Settings › Security › SAML SSO and enable Directory Sync to map your IdP groups to Openlayer workspaces.
    • Supports SAML SSOSAML SSOAn authentication protocol that lets a user log in once with an identity provider and access multiple services without re-entering credentials, by passing signed XML assertions between parties. directory sync so Openlayer workspaces can be governed by existing organizational security groups.
    • Adds Cohere and Vertex AI models as judge options for LLM-as-a-judge evaluation tests.
    • New Faithfulness and Answer Correctness metrics available for RAG system evaluations.
    • New Viewer workspace role with read-only access — no write, update, or delete permissions on resources.
    • Adds expand parameter to inference pipeline GET endpoints, embeddingembeddingLists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG. project and workspace data directly in the response body.

    +8 more show less

    • Adds skipped and failing test counts to project version and inference pipeline API objects.
    • Lower latency for the data stream endpoint.
    • Supports async data uploads and faster batch uploads via pyarrow in the Python SDK.
    • Enables pushing commits to the platform via the Python SDK.
    • New tabular view of test results in test modals.
    • Adds pie graph for test results on the project home screen.
    • Slack notifications for pipeline creation now include the pipeline name.
    • Adds Java code samples to documentation.

    13 details in Everything

70. Stronger security, multimodal tracing, and expanded governance controls seen Jul 13, 2026 · issue 014

    Openlayer adds MFAMFAMulti-factor authentication, requiring proof from different categories, such as a password and a hardware key, to sign in. It reduces reliance on a password alone., multimodalmultimodalAble to work with more than one type of input or output, such as text, images and audio. A multimodal model can combine information across those formats. traces, directory sync, MCPMCPModel Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. integration, and expanded API/CLI controls for AI governance.

    ⤢
    └──▷ TRY IT
    Force a metrics run in CI to ensure all metrics execute even when no changes are detected.
    copy

    $ openlayer metrics run -force

    • Adds platform-wide multi-factor authenticationmulti-factor authenticationMulti-factor authentication, requiring proof from different categories, such as a password and a hardware key, to sign in. It reduces reliance on a password alone. (MFA) for enterprise security.
    • Supports deployment-wide directory sync with project-level access group support, including a NoAccess group level for finer-grained access control.
    • Renders images, audio, and PDFs inline in trace steps and data tables for richer multimodal observability.
    • Surfaces rule-result summaries on Governance and Frameworks pages to streamline audit readiness and compliance reporting.
    • New Langflow integration for connecting Langflow-based AI workflows.

    +17 more show less

    • Adds MCP support: custom functions are now callable from Openlayer.
    • Adds OpenAI Responses API support in the TypeScript SDK.
    • New API endpoint to pull rows from specific test results.
    • New API endpoint to delete projects.
    • Adds API key authentication support for test result endpoints.
    • Custom metrics bundle is now viewable and downloadable from the UI.
    • New On-Prem Diagnostics page for testing connections and sending test emails.
    • Enables data download in development mode.
    • Real-time system health now reflected on the Status page.
    • CLI openlayer push wait mode is now CI-mode compatible.
    • CLI openlayer metrics run gains a -force flag and now prints which metrics are being run.
    • Project owners now have admin-level access for all project-level actions.
    • Session and user IDs in SDKs now accept a broader range of characters.
    • LangChain tracer now supports VertexAI API Keys.
    • ConversationalSearchService traces now include document chunks and nested step hierarchy.
    • Renames Openlayer system tags to categories; user-defined labels remain tags.
    • Detected PIIPIIPersonally Identifiable Information — data that can identify a specific individual, such as names, email addresses, or SSNs. Cyber tools flag or redact it to limit exposure and meet privacy regulations. types now render as tags in test results.

    22 details in Everything

    └──▷ BREAKING ON UPGRADE

    • !Openlayer system tags are renamed to categories; user-defined labels remain tags, which may break any workflows, scripts, or integrations that reference the old tags name for system tags
71. Support for tracking users and sessions, test tags, and more seen Jul 13, 2026 · issue 014

    Openlayer adds user/session tracking, test tags, LiteLLM support, PIIPIIPersonally Identifiable Information — data that can identify a specific individual, such as names, email addresses, or SSNs. Cyber tools flag or redact it to limit exposure and meet privacy regulations. redaction guardrails, and multi-agent LangGraph workflows.

    ⤢
    └──▷ HOW TO FIND IT
    Redact PII automatically from traces in production before they reach Openlayer.
    📍In the Openlayer SDK, enable the guardrail system for tracing by configuring the guardrails option to redact PII on your tracer initialization.
    Control which AWS Signature version Openlayer uses in an on-prem deployment without changing application code.
    copy

    $ export AWS_SIGNATURE_VERSION=v4

    Set up user and session tracking to view full trace history per user in the Openlayer app.
    📍In the Openlayer app, go to the Data section and enable user and session tracking to associate traces with individual users and sessions.
    • Adds user and session tracking in the Data section, enabling full session and record history per user for behavioral analysis.
    • Adds predefined and custom tags on tests to browse and organize tests by outcome, domain (e.g., safety, data drift), or governance framework (e.g., EU AI ActEU AI ActEU regulation that classifies AI systems by risk level and imposes compliance obligations—documentation, transparency, human oversight—on developers and deployers, shaping how AI-based cyber tools are built and sold in Europe.).
    • New API support for user-defined tags on tests and filtering tests by those tags.
    • New API support for batch LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. calls.
    • New API support for user sessions to track traces across interactions.

    +8 more show less

    • New LiteLLM integration via SDKs for tracing LiteLLM-powered calls.
    • New guardrail system in SDKs for Openlayer tracing to redact PII from traces.
    • On-Prem: AWS Signature version is now configurable via the AWS_SIGNATURE_VERSION environment variable.
    • New API filter on version goals by whether they use validation or training datasets.
    • On-Prem: Openlayer now supports the base Redis image for versions 8.2.1+.
    • On-Prem: Adds custom schema support for Postgres tables connected to Openlayer.
    • SDK support for Python 3.13.
    • New comprehensive extraction and multi-agent LangGraph workflow support in SDKs.

    13 details in Everything

72. Test bundles, new tests, support for new Python runtimes seen Jul 13, 2026 · issue 014

    Openlayer adds test bundles (EU AI ActEU AI ActEU regulation that classifies AI systems by risk level and imposes compliance obligations—documentation, transparency, human oversight—on developers and deployers, shaping how AI-based cyber tools are built and sold in Europe., OWASPOWASPA nonprofit foundation that publishes open standards, guides, and ranked vulnerability lists—most famously the OWASP Top 10—giving cyber tools a vendor-neutral benchmark to test or report against., agentic), new LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. tests, and expanded Python runtime support.

    ⤢
    └──▷ HOW TO FIND IT
    Apply a pre-built EU AI Act test bundle to catch compliance gaps across your LLM pipeline before deploying.
    📍In the Openlayer app, go to Tests › Test Bundles, select the EU AI Act bundle, and click Apply to your project.
    Filter your test results to surface only high-priority failing tests during an incident triage.
    📍In the Openlayer app, go to Tests, then use the Filter controls to set Priority = High and Status = Failing.
    • Introduces test bundles grouping related tests by policy or use case — pre-built sets for EU AI Act, OWASP, agentic workflows, and data quality.
    • Adds new tests to the library: groundedness for LLMsLLMsA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code., toxicity score, prompt injectionprompt injectionAn attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. detection, and competitor-recommendation checks.
    • Supports new Python runtimes for CLI development mode.
    • Adds filtering of tests by priority, status, name, and other attributes in the platform UI.
    • Updates OpenTelemetryOpenTelemetryAn open standard and set of tools for collecting traces, metrics and logs from applications. It lets services export observability data without tying instrumentation to one monitoring vendor. GenAI semantic conventions attribute handling in the API.

    +2 more show less

    • Enhances secret management interface in the platform.
    • Adds LangChain callback handler and Bedrock Agents support via TypeScript SDK trace refactoring.

    7 details in Everything

73. The Openlayer MCP server, Automatic thresholds, BigQuery Integration and Anomaly Detection, Project-level access groups seen Jul 13, 2026 · issue 014

    Openlayer adds MCPMCPModel Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. server, automatic thresholds, BigQueryBigQueryA serverless, cloud-hosted analytical database from Google that runs SQL queries across massive datasets without managing infrastructure, giving cyber tools scalable storage and fast querying over large volumes of log or event data. integration, anomaly detection, and project-level access groups.

    ⤢
    └──▷ HOW TO FIND IT
    Let Openlayer automatically determine acceptable thresholds for a new test instead of guessing manually — ideal when you lack historical baselines.
    📍In the Openlayer app, go to your project › Tests › Create new test. In the threshold configuration step, select automatic instead of entering a manual value. Openlayer will derive thresholds from historical data patterns for that metric.
    Connect a BigQuery table to a project for ongoing, automated data quality monitoring without manual data uploads.
    📍In the Openlayer app, go to your project › Integrations › BigQuery and follow the prompts to link a BigQuery table. Once connected, Openlayer will continuously pull data and run data quality tests against it.
    Run Openlayer tests and explore results without leaving your IDE by using the MCP server.
    📍In your IDE's MCP configuration, add the Openlayer MCP server as a provider. Once connected, invoke Openlayer tests directly from the IDE workflow to get evaluation results inline during development.
    • New Openlayer MCP server enables use of Openlayer tests directly inside IDE workflows.
    • Supports automatic, data-driven thresholds for tests that adapt over time based on historical patterns — covering cost, data quality, and GPT eval scores.
    • Adds anomaly detection to flag test results that deviate from predicted norms.
    • New column statistic test supports mean, median, min, max, std, sum, count, and variance.
    • New raw SQL query test for custom evaluation logic.

    +21 more show less

    • Adds anomalous column detection test.
    • New BigQuery integration enables directly connecting a project to BigQuery tables for continuous data quality monitoring.
    • Supports creating tests programmatically via the API.
    • Adds async OpenAI tracer to the SDK.
    • SDK convenience function to copy tests from one project to another.
    • SDK push function gains an option to wait for commit completion.
    • Adds support for Gemini 2.0 models in LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code.-as-a-judge tests.
    • New priority property for tests (critical, high, medium, low).
    • Enables running monitoring mode tests against the entire history of data rather than moving windows.
    • Supports immediately executing tests in monitoring mode.
    • Parses OpenTelemetryOpenTelemetryAn open standard and set of tools for collecting traces, metrics and logs from applications. It lets services export observability data without tying instrumentation to one monitoring vendor. traces from Semantic Kernel and Spring AI.
    • Adds root cause analysis and segment distribution graphs to test diagnostic pages.
    • Daily feature distribution graphs for tabular data projects.
    • Project-level permissions through access groups.
    • On-premise deployment guides for OpenShift and AWS EKSEKSAmazon's managed Kubernetes service, run by AWS, that handles control-plane provisioning so teams can deploy containerized workloads without operating the underlying cluster infrastructure themselves..
    • Option to include or exclude inference pipelines when creating project-level tests.
    • Adds record count and last record received date to inference pipeline views.
    • Surfaces OpenAI refusals to users in LLM-as-a-judge tests.
    • Adds notifications when batch data uploads fail.
    • Test failures no longer cause a commit's status to fail.
    • Adds OpenLIT integration notebook to the SDK.

    26 details in Everything

74. Trace timeline, project lifecycles, and Claude Agent SDK tracing seen Jul 13, 2026 · issue 014

    Openlayer adds Gantt trace timelines, Claude Agent SDK tracing, auto-instrumentation, and Gateway guardrails enforcement.

    ⤢ ⤢
    └──▷ USE IT
    Bootstrap full automatic LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. SDK instrumentation for all supported providers in one call, then run your app — no per-SDK setup needed.
    python copy

    import openlayer
    openlayer.init(api_key="<your-api-key>", project_name="<project-name>")
    # All installed supported SDKs (OpenAI, Anthropic, Mistral, Groq, Gemini, LiteLLM, Portkey, Google ADK, Azure) are now patched automatically.

    Opt out of automatic instrumentation when you want manual control over which SDKs are traced.
    python copy

    import openlayer
    openlayer.init(api_key="<your-api-key>", project_name="<project-name>", auto_instrument=False)

    Inspect the full agent loop — tool calls, MCPMCPModel Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. tools, subagent nesting, and token costs — by enabling Claude Agent SDK tracing in Python.
    python copy

    import openlayer
    import anthropic
    
    openlayer.init(api_key="<your-api-key>", project_name="<project-name>")
    # The Claude Agent SDK is now instrumented: tool calls, MCP tools, subagents,
    # session continuity, and cost/tokens will appear in Openlayer traces.

    • New Gantt-style timeline view for traces — every span's start time, duration, and nesting shown side by side, making slow steps and unexpected branches immediately visible.
    • Adds project lifecycle stages and evidence document attachments for governance tracking, living alongside tests and results on each project.
    • First-party tracing for the Anthropic Claude Agent SDK in Python — captures tool calls, MCP tools, subagent nesting, session continuity, and cost and token counts per run.
    • New automatic SDK instrumentation via openlayer.init() — one call patches every installed supported LLM SDK (OpenAI, Anthropic, Mistral, Groq, Gemini, LiteLLM, Portkey, Google ADKADKGoogle's Agent Development Kit, an open-source framework for building and orchestrating AI agents; cyber tools use it to compose multi-agent workflows with built-in tool-calling and state management., Azure); opt out with auto_instrument=False.
    • Gateway can now enforce guardrails policies inline on LLM requests routing through it.

    +9 more show less

    • Admins can generate Gateway API key invite links so users can self-provision their own Gateway API keys.
    • Adds rule deactivation reasons — supply a reason when deactivating a rule; it is displayed on the rule page.
    • Results pages now surface which tests are required for compliance, shown in context.
    • Data sources can now be renamed directly from their row menu in the UI.
    • Row data can now be downloaded directly from the development-mode view.
    • LangChain callback now captures ToolMessage.artifact (including document lists, promoted to trace-level context) instead of dropping it.
    • Trace configuration is now unified around init() — settings merge incrementally instead of resetting on each call; configure() is deprecated in favor of init().
    • New openlayer tests export CLI subcommand support for projects that use tags.
    • Dialogflow integration now promotes relevant metadata to the row level and allows editing the agent display name.

    14 details in Everything

    └──▷ BREAKING ON UPGRADE

    • !The configure() function is deprecated in favor of init(); trace configuration is now unified around openlayer.init(), with settings merging incrementally instead of resetting on each call.
    • !The Copilot Studio Logic App and Batch API integration paths are now deprecated in favor of the pull-based native data source.
75. Trace timeline, project lifecycles, and Claude Agent SDK tracing Jun 4, 2026 · release notes

    New Gantt trace timeline view, Claude Agent SDK tracing, automatic init() instrumentation, Gateway guardrail enforcement, and invite link key provisioning

    • Adds openlayer.init() auto-instrumentation function to patch OpenAI, Anthropic, Mistral, Groq, Gemini, LiteLLM, Portkey, Google ADKADKGoogle's Agent Development Kit, an open-source framework for building and orchestrating AI agents; cyber tools use it to compose multi-agent workflows with built-in tool-calling and state management., and Azure SDKs automatically
    • Adds Gantt-style trace timeline view showing span start time, duration, and nesting for debugging slow steps or unexpected branches
    • Adds project lifecycle tracking with evidence document attachment for governance process visibility
    • Adds Gateway inline guardrails enforcement on LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. requests routing through the service
    • Adds Gateway admin API key invite links allowing users to self-provision their own Gateway keys

    5 details in Everything

76. Openlayer Gateway, customizable dashboards, and Salesforce Agentforce GA May 15, 2026 · release notes

    Openlayer Spring 2026 adds Gateway, Helm chart, Salesforce integration, new guardrails/tests, customizable dashboards, and API filters.

    • Adds openlayer Gateway, a managed control layer for LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. calls with an admin portal for per-key rate limits, freeze, and team-based organization
    • Adds on-prem Helm chart published to Docker Hub OCIOCIOpen Container Initiative standard, maintained by the Linux Foundation, defining the image format and runtime spec that container tools share so images built once run anywhere OCI-compliant. for Kubernetes deployments with config for storage, database, replicas, image pull secrets, and Node heap
    • Adds Salesforce Agentforce integration (GA) allowing end-to-end tracing with configurable login domain (production, sandbox, or My Domain) and updated Builder schema
    • Adds Llama Prompt Guard prompt injectionprompt injectionAn attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. guardrail, English/Portuguese toxicity guardrails, and Brazilian PIIPIIPersonally Identifiable Information — data that can identify a specific individual, such as names, email addresses, or SSNs. Cyber tools flag or redact it to limit exposure and meet privacy regulations. recognizers (CPF, CNPJ, CEP, phone)
    • Adds new platform tests: Bias, Contains PHI, and Unauthorized tool call

    +14 more show less

    • Adds workspace-level and per-project PII redaction in traces with configurable entity types
    • Adds universal search across projects, sessions, users, and tests from one search bar
    • Adds customizable project dashboards with pin-able widgets, inline notifications, and a sidebar notification center
    • Adds Delete records in monitoring mode via API
    • Adds Retrieve sessions linked to a test result via API
    • Adds dateUpdated filter on the List Tests API
    • Adds BigQueryBigQueryA serverless, cloud-hosted analytical database from Google that runs SQL queries across massive datasets without managing infrastructure, giving cyber tools scalable storage and fast querying over large volumes of log or event data. data source support for datasets without a timestamp column
    • Adds workspace settings to restrict workspace creation to deployment admins and prevent users from given domains from creating workspaces
    • Adds domain verification on the workspace settings page
    • Adds built-in governance frameworks: EU AI ActEU AI ActEU regulation that classifies AI systems by risk level and imposes compliance obligations—documentation, transparency, human oversight—on developers and deployers, shaping how AI-based cyber tools are built and sold in Europe., TRAIGA, E23 MRM, and Openlayer Governance Framework
    • Adds audit trails with full workspace activity log and export
    • Adds session-level aggregated metrics for cost, latency, and token counts
    • Adds LLM cost configuration to view and edit per-model pricing for cost calculations
    • Adds ability to view, edit, and download custom-metric parameter bundles from the UI

    19 details in Everything

77. Stronger security, multimodal tracing, and expanded governance controls Mar 4, 2026 · release notes

    Openlayer adds MFAMFAMulti-factor authentication, requiring proof from different categories, such as a password and a hardware key, to sign in. It reduces reliance on a password alone., multimodalmultimodalAble to work with more than one type of input or output, such as text, images and audio. A multimodal model can combine information across those formats. trace rendering, API project deletion, and custom function MCPMCPModel Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. integration.

    • Adds platform-wide multi-factor authenticationmulti-factor authenticationMulti-factor authentication, requiring proof from different categories, such as a password and a hardware key, to sign in. It reduces reliance on a password alone. (MFA) for enterprise security
    • Images and audio now render inline in trace steps and data tables via UI/UX updates
    • PDF preview is available in traces and data tables through the UI
    • Enables deployment-wide directory sync with project-level access group support for granular IAMIAMIdentity and Access Management — a framework of policies and services for controlling who can authenticate to systems and what resources they are authorized to use, letting cyber tools enforce least-privilege and audit access.
    • Governance and frameworks pages surface rule-result summaries for audit readiness

    +8 more show less

    • Data download capability is now available in development mode
    • New diagnostics page allows testing connections and sending test emails on-prem
    • Adds Langflow integration for workflow interoperability
    • TypeScript SDK gains OpenAI Responses API support
    • Pulls rows from specific test results via a new API endpoint
    • Adds the ability to delete projects via the API
    • Custom metrics bundles are viewable and downloadable directly from the UI
    • Supports MCP custom functions callable from Openlayer for extended agent capabilities

    13 details in Everything

78. Batch test re-runs and expanded integrations Jan 28, 2026 · release notes

    Openlayer adds batch test re-runs, new Ruby/Strands/Gemini/Copilot integrations, and CSICSIAn ambiguous acronym spanning Container Storage Interface, Crime Scene Investigation, and other expansions — without more context about which domain this tool operates in, a reliable definition cannot be given. drift detection

    • Adds Batch test re-runs to recalculate results when new data is added retroactively
    • Adds Openlayer Ruby SDK support for programmatic interaction
    • Adds Strands Agents SDK integration
    • Adds Microsoft Copilot Studio integration
    • Adds Google Gemini API integration

    +5 more show less

    • Adds Custom time range selection for test execution
    • Adds Manual test re-runs capability
    • Adds New primary key uniqueness constraint test type
    • Adds API endpoints for retrieving test results and managing projects
    • Adds Characteristic Stability Index (CSI) as a drift detection method

    10 details in Everything

79. Pausing tests, checks for duplicate keys, and new integrations Dec 23, 2025 · release notes

    Openlayer adds Google ADKADKGoogle's Agent Development Kit, an open-source framework for building and orchestrating AI agents; cyber tools use it to compose multi-agent workflows with built-in tool-calling and state management. integration, pausable tests in Monitoring mode, duplicate key checks, CLI export, Snowflake/remote support, and marketplace availability.

    • Adds openlayer test export CLI command to export tests for a project
    • Adds ability to pause test execution in Monitoring mode via the platform UI
    • Adds new integration with Google Agent Development Kit (ADK) for evaluating multi-agent systems
    • Adds new platform test type to check for duplicate unique and primary keys
    • Adds support connecting to Snowflake views and running tests via remote execution

    +5 more show less

    • Adds ability to run custom SQL tests joining multiple tables
    • Adds OPENLAYER_CERTS environment variables (or equivalent config) to specify custom certs for external services
    • Adds Azure managed identity support for storage connections in On-Prem deployments
    • Makes Openlayer available on AWS, Azure, and Google Cloud marketplaces
    • Adds ability to assign static IPs to Openlayer servers for allowlisting

    10 details in Everything

80. Introducing Openlayer Governance Nov 24, 2025 · release notes

    Openlayer Governance introduces policy enforcement, new data sources, and CLI/API capabilities for custom metrics.

    • Introduces Governance mode to define and enforce policies across projects in a workspace
    • Adds a new workspace overview and project inventory for tracking activity
    • Supports Databricks, Redshift, and Postgres as 3rd-party data backends in Monitoring mode
    • Enables A/B testing across two commits in Development mode or data sources in Monitoring mode
    • Allows configuring directory sync in workspaces with SAMLSAMLA standard for passing signed identity assertions between a sign-in provider and an application. It lets a user sign in through their organization's identity provider./SSOSSOSingle sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. enabled via the UI

    +6 more show less

    • Adds DELETE capability for custom metrics via the API
    • Adds Python SDK support for tracing OpenAI systems using .parse() method or Responses API
    • Adds Python SDK support for tracing systems built with Google ADKADKGoogle's Agent Development Kit, an open-source framework for building and orchestrating AI agents; cyber tools use it to compose multi-agent workflows with built-in tool-calling and state management.
    • Allows configuring trace settings like timeouts and retries in Python SDK
    • Enables creating data sources directly from the CLI
    • Enables deleting custom metrics directly from the CLI

    11 details in Everything

81. Support for tracking users and sessions, test tags, and more Oct 22, 2025 · release notes

    Openlayer adds user/session tracking, test tags, LiteLLM guardrails, and AWS env var support

    • AWS_SIGNATURE_VERSION environment variable allows setting the AWS signature version for On-Prem deployments.
    • Adds user_sessions support in the API to track traces per user session.
    • Adds user-defined tags on tests with filtering capabilities via the API.
    • Adds batch LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. call support to the API.
    • Adds LiteLLM integration for SDK tracing.

    5 details in Everything

82. Test bundles, new tests, support for new Python runtimes Sep 20, 2025 · release notes

    Openlayer introduces test bundles, new LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. evaluation tests, and Python runtime support in CLI.

    • Introduces test bundles to group tests for use cases like the EU AI ActEU AI ActEU regulation that classifies AI systems by risk level and imposes compliance obligations—documentation, transparency, human oversight—on developers and deployers, shaping how AI-based cyber tools are built and sold in Europe., OWASPOWASPA nonprofit foundation that publishes open standards, guides, and ranked vulnerability lists—most famously the OWASP Top 10—giving cyber tools a vendor-neutral benchmark to test or report against., agentic workflows, and data quality.
    • Adds new test types to the library: groundedness for LLMsLLMsA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code., toxicity score, prompt injectionprompt injectionAn attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. checks, and competitor recommendation checks.
    • Adds support for new Python runtimes in the CLI development mode.

    3 details in Everything

83. Complete design system overhaul, Snowflake Integration Aug 21, 2025 · release notes

    New Snowflake integration, Bedrock/LangGraph tracing support, dark mode UI, and member-restricted role.

    • Add directly connect Snowflake tables to projects integration
    • Add SDK support for tracing Bedrock models, OpenAI Agents, Pydantic AI systems, and LangGraph systems
    • Add new Member restricted role that can perform member actions without viewing data source data
    • Allow hosting Openlayer on subpaths in on-prem deployments
    • Allow users to override LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. costs with custom costs via the platform

    +2 more show less

    • Include standard deviation score in LLM-as-a-judge and Ragas test results
    • Add new prompt injectionprompt injectionAn attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. test to detect adversarial attacks on LLM systems

    7 details in Everything

84. The Openlayer MCP server, Automatic thresholds, BigQuery Integration and Anomaly Detection, Project-level access groups Jun 14, 2025 · release notes

    Openlayer adds MCPMCPModel Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. server, automatic test thresholds, BigQueryBigQueryA serverless, cloud-hosted analytical database from Google that runs SQL queries across massive datasets without managing infrastructure, giving cyber tools scalable storage and fast querying over large volumes of log or event data. integration, anomaly detection, and project-level access groups.

    • Releases the Openlayer MCP server for IDE workflows
    • Adds automatic threshold option for tests in the app UI
    • Introduces anomaly detection to flag test results deviating from automatic thresholds
    • Supports creating tests via API
    • Adds direct BigQuery integration for continuous data quality monitoring

    5 details in Everything

85. Project-level secrets, tracing LLM requests with OpenTelemetry Mar 15, 2025 · release notes

    Add project-level secrets management, default API keys for on-prem LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code.-as-judge, and OpenTelemetryOpenTelemetryAn open standard and set of tools for collecting traces, metrics and logs from applications. It lets services export observability data without tying instrumentation to one monitoring vendor. tracing support

    • Support project-level secrets stored and referenced directly from the Platform UI/API

    1 detail in Everything

86. SAML Directory Sync, new LLM-as-a-judge models, and website refresh Jan 16, 2025 · release notes

    Openlayer adds SAMLSAMLA standard for passing signed identity assertions between a sign-in provider and an application. It lets a user sign in through their organization's identity provider. directory sync, Cohere/Vertex AI LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code.-as-judge models, and expand API query param.

    • Directory sync with SAML
    • Use Cohere, Vertex AI models as options for LLM-as-a-judge metrics
    • Add expand to inference pipeline GETs so projects and workspaces are included in the response body
    • New Viewer role in workspaces that doesn’t have write, update or delete permissions on resources

    4 details in Everything

87. Improved test diagnosis page, SAML SSO, design refreshes, + more Oct 31, 2024 · release notes

    Openlayer adds SAML SSOSAML SSOAn authentication protocol that lets a user log in once with an identity provider and access multiple services without re-entering credentials, by passing signed XML assertions between parties. support and a test diagnosis page with error visibility and filtering

    • Enables SAMLSAMLA standard for passing signed identity assertions between a sign-in provider and an application. It lets a user sign in through their organization's identity provider. SSOSSOSingle sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. authentication for workspaces via major identity providersidentity providersA service that authenticates users and supplies identity information to other applications. It is the sign-in authority behind many single sign-on systems..
    • Adds List view on the test diagnosis page to scan skipped/errored tests and filter by type.
    • Shows an overview at the top of the test diagnosis page with total results per status and recent issues.
    • Makes sections in the test diagnosis page collapsible.
    • Supports Azure OpenAI as an LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. evaluator.

    +3 more show less

    • Adds Ragas metrics (faithfulness, answer correctness) to Evals.
    • Updates cost table for OpenAI models in Observability.
    • Lists pipeline names in notifications for new inference pipelines.

    8 details in Everything

88. Custom metrics, rotating API keys, and new models for direct-to-API calls Jul 1, 2024 · release notes

    Openlayer introduces custom metric upload via config, API key rotation management, and expanded model support for direct API calls.

    • openlayer.json now accepts metrics configuration to upload and use custom metrics as project-wide evaluations or individual tests
    • API keys can be created, renamed, and deleted directly via the platform to enable rotation

    2 details in Everything

89. Improved quality control over your LLM’s responses with annotations and human feedback May 31, 2024 · release notes

    Openlayer adds request annotations, thumbs-up/down ratings, ground truth editing, and filtered test modal data.

    • Adds the ability to annotate any request with custom values for granular quality control
    • Adds thumbs up / thumbs down rating UI to requests for scanning good and bad responses
    • Adds ability to add ground truths to requests and edit existing ground truths
    • Filters data in individual test modals by selected evaluation window

    4 details in Everything

90. Simple, dev-focused workflow for AI evals May 3, 2024 · release notes

    Openlayer adds GitHub integration for continuous evals via CLI and REST APIREST APIA web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. with grouped-by-column test support

    • Adds GitHub integration that commits to Openlayer on every GitHub commit to trigger continuous evaluation
    • Adds ability to customize the evaluation workflow using the CLI and REST API
    • Adds template repositories for common AI eval use cases with cloning support in the Openlayer UI
    • Adds None value support in the token column via API
    • Adds new test type allowing results to be grouped by a specific column

    5 details in Everything

91. Trace every step of your requests Mar 28, 2024 · release notes

    Openlayer adds request tracing for debugging and Azure OpenAI integration support

    • Adds the ability to trace every step of a request in the UI, including latency and function inputs & outputs
    • Adds support for using Azure OpenAI models
    • Expands request streaming capabilities to include support for function callingfunction callingA model's ability to answer with a structured request to run a named function or API, which the application executes and returns. It is how agents take actions.

    3 details in Everything

92. More tests around latency metrics Feb 15, 2024 · release notes

    Openlayer v1.0 adds granular latency percentiles (min, median, p90, p95, p99) and subpopulation filters for data integrity/consistency tests.

    • Adds Min Latency, Median Latency, 90th Percentile Latency, 95th Percentile Latency, and 99th Percentile Latency test types in the Performance page
    • Enables subpopulation filters for Data Integrity or Data Consistency tests to target specific data clusters
    • Adds project name search capability in the project overview UI

    3 details in Everything

93. Go deep on test result history and add multiple criteria to GPT evaluation tests Feb 1, 2024 · release notes

    Openlayer adds multi-criteria GPT evaluation tests and deep test result history visualization

    • Adds ability to add multiple criteria to a single GPT evaluation test, allowing unique thresholds for each criterion in one go
    • Enables selecting specific date ranges and scrubbing through graphs on individual test pages to view requests from specific time periods

    2 details in Everything

94. Cost-per-request, new tests, subpopulation support for data tests, and more precise row filtering Jan 17, 2024 · release notes

    Openlayer adds cost-per-request monitoring, subpopulation filtering for data tests, and new Column average/contains string test types.

    • Adds Column average test to ensure column averages fall within a specified range
    • Adds Column contains string test where values in one column must contain strings from another column
    • Adds cost-related tests: Max cost, Mean cost, and Total cost per evaluation window
    • Shows request costs on the Requests page with cost graph support
    • Enables subpopulation filtering for integrity and consistency data tests using row-level metrics like conciseness

    +1 more show less

    • Returns to the timeline page to visualize commit performance against goals over time

    6 details in Everything

95. Log multi-turn interactions, sort and filter production requests, and token usage and latency graphs Dec 21, 2023 · release notes

    Adds multi-turn chat history logging, token/latency graphs, Gemini model support, and request sorting/filtering

    • Logs full multi-turn chat history for production requests in monitoring mode
    • Displays token usage and latency graphs over time for all requests
    • Supports Google's Gemini model for evaluation and comparison
    • Adds row-by-row explanations for tests using GPT evaluation
    • Allows sorting and filtering production requests by timestamp, token usage, or latency

    5 details in Everything

96. GPT evaluation, Great Expectations, real-time streaming, TypeScript support, and new docs Nov 30, 2023 · release notes

    Openlayer introduces GPT evaluation, Great Expectations integration, real-time streaming API, and TypeScript SDK

    • Adds built-in GPT evaluation tests for model outputs using descriptive criteria via LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. grading
    • Adds support for creating and running tests from Great Expectations (GX) data integrity checks
    • Adds new /stream endpoint for real-time data ingestion instead of batch uploads
    • Releases V1 of the Openlayer TypeScript client library for logging requests
    • Adds a new Requests page in the UI to view real-time model request streams and metadata like token count and latency

    5 details in Everything

    └──▷ BREAKING ON UPGRADE

    • !The term goals has been renamed to tests across the product, documentation, and website; references to the old term are no longer valid.
97. Enhanced onboarding, redesigned navigation, and new goals Nov 17, 2023 · release notes

    Openlayer adds new consistency/integrity goals and arbitrary column names in thresholds.

    • Adds size-ratio goal to enforce required size ratios between datasets
    • Adds character_length goal to enforce character limits on text-based columns

    2 details in Everything

98. Evals for LLMs, real-time monitoring, Slack notifications and so much more! Oct 24, 2023 · release notes

    Openlayer adds LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. evals in dev/monitoring modes, Slack notifications, and workspace secrets

    • Adds LLMs in development mode for experimenting with prompts and models via UI prompt playground connecting to OpenAI, Anthropic and Cohere.
    • Adds LLMs in monitoring mode to evaluate production responses using the same tests from development while measuring token usage, latency, drift and data volume.
    • Adds Slack integration to receive notifications for goal status updates, team activity like comments, and workspace updates.

    3 details in Everything

99. Regression projects, toasts, and artifact retrieval Jun 29, 2023 · release notes

    Openlayer adds tabular regression evaluation with MSE/MAE metrics and commit artifact downloads.

    • Adds support for evaluating tabular regression models in projects
    • Enables downloading models and datasets (commit artifacts) via the platform UI
    • Allows deleting commits through the platform interface

    3 details in Everything

100. Sign in with Google, sample projects, mentions and more! May 30, 2023 · release notes

     Openlayer v230530 adds Google SSOSSOSingle sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management., sample projects, user mentions, shell model uploads, ROC AUC metric, and workspace browsing.

     • Enabled SSO (single sign-onsingle sign-onSingle sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management.) with Google for authentication
     • Added sample projects to all workspaces
     • Added support for mentioning users, goals, and commits in goal comments and descriptions using @ and # symbols
     • Added the ability to upload shell models without a model binary for explainability and fairness analysis
     • Added ROC AUC to available project metrics

     +1 more show less

     • Added an overview page to browse and navigate to projects

     6 details in Everything