No entries match these filters. Clear filters
Lakebase now available in three additional regions (2026-11-20)
Nov 20, 2026
·
issue 146
MCP tools in Genie Agents are generally available (2026-11-20)
Nov 20, 2026
·
issue 146
Databricks adds generally available MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. tool support to Genie Agents, letting authors attach external MCP servers from Unity Gateway︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Attach external MCP servers registered in Unity Gateway to Genie Agents, in addition to Databricks-provided connectors.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Agent mode APIs now support MCP tools. Not checked yet
Attach external MCP servers registered in Unity Gateway to Genie Agents, in addition to Databricks-provided connectors.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 more in Everything
The Databricks GitHub app will request permission to create and manage pull requests (2026-11-20)
Nov 20, 2026
·
issue 146
Databricks adds pull-request creation and management via the Databricks GitHub app for Genie Code and Genie ZeroOps︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Genie Code and Genie ZeroOps can now create and manage pull requests on your behalf once the Databricks GitHub app is granted Read and write access to pull requests — your existing branch protection rules and required reviews still apply.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Warehouse-level statement timeouts are now generally available (2026-11-20)
Nov 20, 2026
·
issue 146
Databricks adds generally available warehouse-level statement timeouts, configurable via the statement_timeout field across the SQL warehouses API, SDKs, Terraform, and Dec︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ USE IT
Cap long-running queries on a shared warehouse so runaway jobs cannot monopolize compute — set statement_timeout in your Terraform warehouse resource.
main.tf
copy
resource "databricks_sql_warehouse" "shared" {
name = "shared-warehouse"
statement_timeout = 3600
}
Apply a statement timeout directly via the SQL warehouses API to an existing warehouse without changing IaC IaC Infrastructure as Code: the practice of defining and provisioning compute infrastructure through machine-readable config files rather than manual processes, giving cyber tools repeatable, version-controlled environment setup. .
copy
$ curl -X PATCH https://<workspace-host>/api/2.0/sql/warehouses/<warehouse-id> \
-H 'Authorization: Bearer <token>' \
-H 'Content-Type: application/json' \
-d '{"statement_timeout": 3600}'
Set per-warehouse query time limits using the statement_timeout field in the SQL warehouses API, Databricks SDKs, Terraform, or Declarative Automation Bundles to automatically cancel runaway statements.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
OAuth authentication for webhook destinations will soon be available by default for workspaces with the compliance security profile enabled (2026-11-20)
Nov 20, 2026
·
issue 145
Databricks adds OAuth 2.0 OAuth 2.0 An open authorization framework (IETF RFC 6749) that lets a service grant limited API access to a third party without sharing credentials, giving cyber tools a standard way to authenticate against external platforms. client-credential authentication for webhook destinations in compliance security profile workspaces︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Databricks-to-Databricks Delta Sharing across regulatory domains is now generally available (2026-11-19)
Nov 19, 2026
·
issue 146
Databricks adds generally available Databricks-to-Databricks Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. across regulatory domains, including commercial-to-GovCloud and Azure China︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Manage Lakewatch with the Databricks CLI (Beta) (2026-11-19)
Nov 19, 2026
·
issue 146
Databricks adds a lakewatch command group to the CLI for managing datasources, parsers, and detection rules︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Lakebase now supports Postgres 19 (2026-11-19)
Nov 19, 2026
·
issue 145
LTAP Direct Writes for synced tables is now generally available (2026-11-18)
Nov 18, 2026
·
issue 146
Databricks adds LTAP Direct Writes for synced tables, writing loads directly to Lakebase storage to speed up initial loads and full refreshes︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
When creating a synced table with a large initial data load, select LTAP Direct Writes so the ingestion goes straight to Lakebase storage and does not slow down running application queries.
📍 In the Databricks UI, go to Lakebase › Synced Tables › Create Synced Table, then under Sync method select LTAP Direct Writes before saving.
OAuth M2M authentication for the Salesforce, ServiceNow, and Jira connectors (2026-11-18)
Nov 18, 2026
·
issue 145
ai_forecast version 2 is now generally available (2026-11-18)
Nov 18, 2026
·
issue 145
Databricks adds holiday support, external covariates, and non-negative forecasts to the generally available ai_forecast version 2 in Databricks SQL︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Configurable retention for system tables is generally available (2026-11-18)
Nov 18, 2026
·
issue 144
Databricks adds configurable retention for system tables, with account-wide periods from 30 to 3,650 days now generally available︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Set an account-wide retention period from 30 to 3,650 days for supported system tables, with the first 395 days free and storage charges applying beyond that threshold.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 more in Everything
Configure materialization TTLs for individual OpenSharing recipients (2026-11-18)
Nov 18, 2026
·
issue 144
Databricks adds per-recipient materialization TTL TTL Time to live, a limit on how long data remains valid or how far a packet can travel. Caches use a time limit; IP packets use a hop limit. overrides for open sharing, superseding the metastore default︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Set a materialization time-to-live time-to-live Time to live, a limit on how long data remains valid or how far a packet can travel. Caches use a time limit; IP packets use a hop limit. (TTL) per open sharing recipient, overriding the metastore-level default.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 more in Everything
Evaluate streaming responses incrementally with service policies (Beta) (2026-11-18)
Nov 18, 2026
·
issue 144
Databricks adds incremental streaming evaluation for model output policies, with an option to buffer the full response before release︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ BREAKING ON UPGRADE
! Existing model output policies now default to streaming evaluation; previously buffered responses are no longer held until completion unless Full response evaluation is explicitly selected.
Service policies protect embedding inputs (Beta) (2026-11-18)
Nov 18, 2026
·
issue 144
Databricks adds service policy guardrails for single and batch embedding embedding Lists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG. inputs to mask sensitive data or deny unsafe requests before the model runs︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Smart Routing now supports external models for coding agents (Beta) (2026-11-18)
Nov 18, 2026
·
issue 143
Databricks Unity Gateway adds external model routing for coding-agent subagents in Smart Routing Beta ︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ USE IT
Add an external model source for coding-agent subagents so Smart Routing can dispatch to it alongside built-in system.ai models.
yaml
copy
models:
sources:
- system.ai
- <your-external-model-provider-service>
Genie Code CLI is now generally available (2026-11-18)
Nov 18, 2026
·
issue 142
Databricks adds general availability of the Genie Code CLI, sharing one budget and free monthly usage with the workspace.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
JDBC Unity Catalog connections are now available by default for workspaces with the compliance security profile enabled (2026-11-17)
Nov 17, 2026
·
issue 144
Databricks adds JDBC JDBC A Java standard API for connecting applications to relational databases, maintained as part of the Java SE platform, letting tools query any compliant database through a single consistent interface. Unity Catalog connections, with SSL SSL A cryptographic protocol for encrypting network connections, now deprecated in favor of TLS but whose name persists colloquially. Cyber tools reference it when handling HTTPS or other encrypted-channel inspection and termination. certificate support, as a default capability for compliance security profile workspaces.
Lakebase streaming sink is generally available (2026-11-17)
Nov 17, 2026
·
issue 144
Databricks adds general availability of the Lakebase streaming sink, with merge strategies, additive schema evolution, and external PostgreSQL write support︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Write to an external PostgreSQL database through a Unity Catalog connection using the Lakebase streaming sink Public preview .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Use merge strategies and additive schema evolution via mergeSchema in the Lakebase streaming sink on Databricks Runtime 19 and above. Not checked yet The Lakebase streaming sink now supports Trigger.AvailableNow() on serverless jobs and notebooks. Not checked yet
Write to an external PostgreSQL database through a Unity Catalog connection using the Lakebase streaming sink Public preview .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Use merge strategies and additive schema evolution via mergeSchema in the Lakebase streaming sink on Databricks Runtime 19 and above. Not checked yet
1 more in Everything
OpenAI GPT-6.2 Astra and Sol are now available on Unity Gateway (2026-11-17)
Nov 17, 2026
·
issue 144
Analyze agent usage and find wasted tokens with Unity Gateway Insights (Beta) (2026-11-17)
Nov 17, 2026
·
issue 143
Databricks adds Unity Gateway Insights Beta to report on agent usage patterns and token waste across sessions︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Generate Unity Gateway Insights reports to see what users invoke agents for and where tokens are wasted, built from a sample of sessions in the unified trace table.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
TIMERANGE_NOW() on the Lakewatch query page uses the selected time range (2026-11-16)
Nov 16, 2026
·
issue 144
Databricks adds historical-data testing to Lakewatch detection rules via updated TIMERANGE_NOW() behaviour︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Replay a past time window in Lakewatch to verify that a detection rule fires on historical data rather than live events.
📍 In the Lakewatch query page, select a historical time range, then write your detection predicate using TIMERANGE_NOW() as the window boundary — it will resolve to the end of your selected range, not the current clock time.
Test a detection rule's predicates against historical data in Lakewatch: TIMERANGE_NOW() now returns the end of the selected time range instead of the current time.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 more in Everything
Databricks Runtime 19: November 16, 2026 (2026-11-16)
Nov 16, 2026
·
issue 142
Databricks Runtime 19 adds real-time watermark propagation by default for new streaming queries in real-time mode︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ USE IT
Preserve the previous watermark behavior for a new real-time streaming query that must not evict state sooner than before.
env
copy
spark.databricks.streaming.realTimeMode.realTimeWatermark.enabled = false
└──▷ BREAKING ON UPGRADE
! New streaming queries in real-time mode now use real-time watermark propagation by default; state is evicted sooner than before. Queries restarted from an existing checkpoint keep their original behavior, but new queries will behave differently. Set spark.databricks.streaming.realTimeMode.realTimeWatermark.enabled to false before starting a new query to restore the previous behavior.
Governed tag propagation supports classic compute (Beta) (2026-11-16)
Nov 16, 2026
·
issue 142
Databricks adds governed tag propagation support for classic compute running Databricks Runtime 19 or above︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
IP functions will soon be available by default for workspaces with the compliance security profile enabled (2026-11-16)
Nov 16, 2026
·
issue 142
Databricks adds IP functions for working with IP addresses and CIDR CIDR A notation for a block of IP addresses, such as 10.0.0.0/24. The number after the slash says how many leading address bits identify the network. blocks in compliance security profile workspaces︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Share Unity Catalog agent skills with OpenSharing and Databricks Marketplace (Beta) (2026-11-16)
Nov 16, 2026
·
issue 142
Databricks adds support for sharing Unity Catalog agent skills via OpenSharing and Databricks Marketplace Beta ︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
PostgreSQL integrated CDC pipeline is now in Beta (2026-11-15)
Nov 15, 2026
·
issue 142
Databricks adds a single integrated CDC pipeline for PostgreSQL, requiring no separate ingestion gateway, now in Beta.
Efficient column updates (Beta) (2026-11-13)
Nov 13, 2026
·
issue 143
Databricks adds Efficient Column Updates Beta to add and backfill columns without rewriting unchanged column data︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Workspace admin enabling Efficient Column Updates so teams can add and backfill Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. columns without full table rewrites
📍 In the Databricks workspace, go to Settings › Previews and enable Efficient Column Updates .
Files API 3.0 will soon be generally available (2026-11-13)
Nov 13, 2026
·
issue 143
Databricks Files API 3.0 adds multipart uploads for files larger than 5 GB and conditional operations on Unity Catalog volumes︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Upload files larger than 5 GB to Unity Catalog volumes using multipart uploads in Files API 3.0.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Guard uploads, downloads, and deletions against unintended changes using conditional operations that check the current file version before the operation in Files API 3.0. Not checked yet
2 more in Everything
Genie Migrate can convert T-SQL to Lakeflow pipelines (Beta) (2026-11-13)
Nov 13, 2026
·
issue 143
Genie Migrate adds Beta T-SQL to Lakeflow pipeline conversion; workspace admins enable it via Migration Projects Experimental Features on the Previews page.
External access to views through the Iceberg REST catalog is in Beta (2026-11-13)
Nov 13, 2026
·
issue 142
Databricks adds Beta support for querying Unity Catalog views from external Iceberg engines via the Iceberg REST catalog Iceberg REST catalog A standard HTTP API specification for managing Apache Iceberg table metadata, letting any compatible engine or cyber tool discover, read, and write Iceberg tables without vendor-specific connectors. .
Submit support cases from your workspace with the Get support form (2026-11-13)
Nov 13, 2026
·
issue 142
Databricks adds an in-workspace Get support form that pre-fills issue details, attaches diagnostics, and suggests an AI-generated solution before submitting a case︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
When you hit an error in your workspace and want to open a support case without leaving Databricks or manually gathering diagnostics
📍 In the workspace, navigate to Help › Get support, fill in the issue details (the form auto-attaches diagnostic files), review the AI-generated solution suggestion, then click Submit.
Importing Power BI and Tableau files with Genie Code will soon be available by default for workspaces with the compliance security profile enabled (2026-11-13)
Nov 13, 2026
·
issue 139
Databricks adds Power BI and Tableau file import via Genie Code for compliance security profile workspaces︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Enable Genie Code BI file import early for a compliance security profile workspace before the December 15, 2026 default rollout.
📍 In the workspace settings, go to Previews and enable Import BI files using Genie Code .
AI Function task support in Declarative Automation Bundles (Beta) (2026-11-13)
Nov 13, 2026
·
issue 138
Databricks adds AI Function task support to Declarative Automation Bundles Beta ︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Databricks now supports AI Function tasks in Declarative Automation Bundles, enabling AI-driven steps to be defined and orchestrated as bundle resources.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Genie Agents enhancements (2026-11-12)
Nov 12, 2026
·
issue 143
Databricks adds a dedicated Connectors section to Genie Agent settings for adding connectors Beta ︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Add a connector to a Genie Agent using the new dedicated Connectors section in agent settings.
📍 In the Genie Agent settings, go to Configure settings › Connectors, then select Add connector to attach a data connector to the agent.
Budgets now support daily thresholds in addition to monthly thresholds (2026-11-12)
Nov 12, 2026
·
issue 142
Databricks adds daily per-user budget thresholds alongside monthly thresholds for Unity Gateway budgets︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Flows are securable objects in Unity Catalog (Beta) (2026-11-12)
Nov 12, 2026
·
issue 139
Databricks adds Unity Catalog fine-grained permissions for pipeline flows, with five new grantable privileges on individual flows︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ TRY IT
Restrict a service principal to only triggering a specific pipeline flow refresh without granting broader pipeline access.
copy
$ GRANT REFRESH FLOW ON FLOW my_catalog.my_schema.my_flow TO `service-principal@example.com`;
Postgres 18.7, 17.12, and 16.16 now available (2026-11-12)
Nov 12, 2026
·
issue 139
ABAC DENY policies support AI securables and attachment to individual objects (Beta) (2026-11-12)
Nov 12, 2026
·
issue 138
Databricks adds DENY policy support for AI securables and direct attachment to individual objects in beta︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Attach DENY policies directly to individual tables, volumes, functions, models, secrets, model services, model provider services, MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. services, agent services, and skills using the ON clause.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Apply DENY policies to model services, model provider services, MCP services, agent services, and skills using the FOR clause. Not checked yet
Attach DENY policies directly to individual tables, volumes, functions, models, secrets, model services, model provider services, MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. services, agent services, and skills using the ON clause.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 more in Everything
Artifacts and slides in Genie One (Beta) (2026-11-12)
Nov 12, 2026
·
issue 138
Databricks adds Artifacts in Genie One Beta — AI-built slide decks and visual pages you can present, edit, and share from chat︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Enable the Artifacts Beta so your workspace users can ask Genie One to build and share slide decks from chat.
📍 1. Sign in as a workspace admin.
2. Go to Settings › Previews.
3. Find Artifacts and turn it on.
Enable the Artifacts preview by turning on the Artifacts preview setting from the Previews page in workspace admin settings.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Databricks Genie One now builds Artifacts — slide decks and custom visual pages from your data — that persist in the workspace, can be edited or updated via chat, and store a data snapshot so viewers see the data without needing access to source tables. Not checked yet
2 details in Everything
Connectors section in Genie Agent settings (Beta) (2026-11-12)
Nov 12, 2026
·
issue 138
Databricks adds a dedicated Connectors section to Genie Agent settings Beta ︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Add a connector to a Genie Agent when you want it to query external data sources alongside your Databricks data.
📍 In the Databricks UI, open your Genie Agent, go to Configure settings, locate the new Connectors section, and select Add connector.
Dashboard enhancements and bug fixes (2026-11-12)
Nov 12, 2026
·
issue 138
Databricks AI/BI Dashboards add pivot table sorting, associative filter control, and undo/redo for dashboard variables︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Viewers can now sort pivot tables by row and column.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Authors can now style pivot table headers separately for each row, column, and value field. Not checked yet Pivot table values can now adjust their height to wrap text. Not checked yet Authors can now resize the divider between columns and values in a pivot table. Not checked yet Authors can now turn off associative filtering between dashboard filters. Not checked yet
+3 more
show less
Actions on dashboard variables now support undo and redo. Not checked yet Genie Code can now write custom calculations that reference other custom calculations in the same dataset. Not checked yet Ask Genie now appears only on widgets that Genie can use as context, no longer on images, filters, or older text boxes. Not checked yet
Viewers can now sort pivot tables by row and column.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Authors can now style pivot table headers separately for each row, column, and value field. Not checked yet Authors can now turn off associative filtering between dashboard filters. Not checked yet Actions on dashboard variables now support undo and redo. Not checked yet
4 more in Everything
Genie One desktop app is generally available on macOS (2026-11-12)
Nov 12, 2026
·
issue 138
Databricks adds the Genie One desktop app as generally available on macOS, with window navigation, automatic updates, and a dock badge for unread activity︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Google Gemini 4 Hydrogen is now available on Unity Gateway (2026-11-12)
Nov 12, 2026
·
issue 138
Links open in the Genie mobile app (2026-11-12)
Nov 12, 2026
·
issue 138
Databricks Genie now opens chat and dashboard links directly in the Genie mobile app on iOS and Android︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Links to Genie One chats and published dashboards now open directly in the Genie mobile app on iOS and Android instead of in a browser.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
OAuth on-behalf-of for service principals is generally available (2026-11-12)
Nov 12, 2026
·
issue 138
Databricks adds OAuth on-behalf-of (OBO) OAuth on-behalf-of (OBO) A Microsoft OAuth 2.0 extension flow where a service exchanges a token it received for a new token scoped to a downstream API, letting cyber tools propagate delegated user identity through multi-tier call chains. for service principals as a generally available default, removing the need to create an OAuth secret︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Use the standard user-to-machine (U2M) OAuth flow to act as a service principal you manage without creating an OAuth secret, now that OAuth on-behalf-of (OBO) for service principals is on by default — requires the Service Principal Manager role on the service principal.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Scheduled tasks can email other users in Genie One (2026-11-12)
Nov 12, 2026
·
issue 138
Databricks adds the ability to send scheduled Genie One task emails to other users in your organization︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Share scheduled Genie One task results with teammates automatically, without requiring them to set up their own tasks.
📍 In Genie One, open a scheduled task, then under Send email to , add other users in your organization as recipients.
Use Genie One as your Chrome new tab page (Beta) (2026-11-12)
Nov 12, 2026
·
issue 138
Databricks adds a Chrome extension that replaces the new tab page with the Genie One home page Beta ︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Replace every Chrome new tab with the Genie One home page so your team can query data or open dashboards instantly, without navigating to your Databricks workspace.
📍 1. Install the Genie One Chrome extension from the Chrome Web Store.
2. When prompted, sign in with your existing Databricks credentials.
3. Select the one workspace you want the extension to connect to.
4. Open a new tab — Genie One loads automatically, ready for questions, search, or dashboards.
Connect GitHub Copilot CLI and GitHub Copilot in Visual Studio Code with Unity Gateway (2026-11-11)
Nov 11, 2026
·
issue 138
Databricks adds GitHub Copilot in Visual Studio Code to Unity Gateway agent configuration via ug configure --agents copilot︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ TRY IT
Set up GitHub Copilot in VS Code and Copilot CLI together through Unity Gateway, specifying models and tracing in one pass.
copy
$ ug configure --agents copilot
Bring your own ChatGPT subscription to Codex with Unity Gateway (2026-11-11)
Nov 11, 2026
·
issue 137
Databricks adds support for bringing your own ChatGPT subscription to Codex via Unity Gateway︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Connect ChatGPT to Genie One with the Databricks Genie plugin (2026-11-11)
Nov 11, 2026
·
issue 137
Databricks adds a Genie plugin that connects ChatGPT to the Genie One MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. for natural-language data queries.
Connect Claude to Genie One with the Databricks Genie connector (2026-11-11)
Nov 11, 2026
·
issue 137
Databricks adds a Genie connector that links Claude to Genie One MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. , letting users ask natural-language questions about their data from Claude.
Connect Grok Bot/Cursor to Genie One with the Databricks Genie plugin (2026-11-11)
Nov 11, 2026
·
issue 137
Databricks adds a Genie plugin for Grok Bot and Cursor, connecting them to the Genie One MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. so users can query their data from either tool.
Feature Views are now available by default for workspaces with the compliance security profile enabled (2026-11-11)
Nov 11, 2026
·
issue 137
The Genie One desktop app for Windows is in Beta (2026-11-11)
Nov 11, 2026
·
issue 137
Databricks adds Windows support for the Genie One desktop app in Beta, letting users chat with data without a browser︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Write custom service policies in Python (Beta) (2026-11-11)
Nov 11, 2026
·
issue 137
Databricks adds support for writing custom service policies as Python functions in Unity Catalog Beta ︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Write custom service policies as Python functions in Unity Catalog to express rules that SQL cannot handle, such as regular expressions, Python packages, or calls to an external guardrail through a service credential.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Pages are generally available (2026-11-11)
Nov 11, 2026
·
issue 136
Databricks adds Pages — governed business-concept definitions — to general availability, with a new permission and REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. for managing them︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Publish and manage governed definitions of business concepts, organized by domain on the Discover page, now that Pages are generally available.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Control who can publish and manage Pages in a domain with the new MANAGE PAGES permission. Not checked yet Manage Pages programmatically via the REST API. Not checked yet
3 details in Everything
Databricks OLTP adds read replica support to offload Data API read requests from primary compute︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Databricks adds AI Function tasks as a new task type in Lakeflow Jobs︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Databricks Designer adds a Multi-Row Formula operator for window calculations and file export to Google Drive and SharePoint︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Compute a per-customer running total and row-over-row change in a visual workflow without writing SQL.
📍 In the Designer canvas, add a Multi-Row Formula operator after your source, then:
1. Click Add formula › Running aggregate . Set New column name to customer_running_total, Function to Sum , Of column to amount . Add order_date (Ascending) as Sort and customer_id as Grouping.
2. Click Add formula › Difference from previous row . Set New column name to amount_change, Of column to amount , Rows back to 1. Same Sort and Grouping.
3. Click Apply .
Export workflow results to a SharePoint folder for sharing with stakeholders who don't have Databricks access.
📍 In the Designer canvas, connect your final operator's outgoing port to a Destination operator. Set Destination to Microsoft 365 , enable the SharePoint connector and click Connect . Paste your SharePoint folder link into Folder URL , enter a File name , and choose Excel as the format. Click Apply .
Write workflow output files to Google Drive or SharePoint by setting Destination to Google Drive or Microsoft 365 , entering a Folder URL , and signing in via the connector.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Databricks Designer now adds a Multi-Row Formula operator that computes new columns from neighboring rows, supporting running aggregates, row number and rank, lag/lead values, row-over-row differences, moving averages, exponential smoothing, and custom Spark SQL window expressions via OVER clauses. Not checked yet Control the row window in Multi-Row Formula with shared Sort and Grouping fields — sort columns set row order, grouping columns restart calculations per distinct group. Not checked yet Use Running aggregate formulas (Sum, Average, Minimum, Maximum, Count) in Multi-Row Formula to build cumulative values from the first row of a group through the current row. Not checked yet Use Row number / rank mode in Multi-Row Formula to number or rank rows with configurable Start at values and output as integer or text, supporting sequential, gap-rank, and dense-rank modes. Not checked yet
+5 more
show less
Use Value from another row in Multi-Row Formula to read a column's value from a lag or lead row, with Rows away and If no such row controls for null, zero, or current-row fallback. Not checked yet Use Difference from previous row in Multi-Row Formula to subtract a prior row's value from the current row, controlled by the Rows back field. Not checked yet Use Moving average in Multi-Row Formula to average a configurable Window (rows) of rows including the current row, with partial windows at group boundaries. Not checked yet Use Exponential smoothing in Multi-Row Formula to compute an exponentially weighted moving average with a configurable Smoothing factor (α) between 0 and 1 (default 0.3). Not checked yet Use Custom expression in Multi-Row Formula to evaluate any Spark SQL expression with its own OVER clause for window functions not covered by the other formula types. Not checked yet
10 details in Everything
Databricks adds Genie Clean Room Beta , letting a collaborator query shared data in natural language without accessing the underlying data or proprietary IP︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Databricks Genie adds user-aware workspace instructions and an expandable visualization side panel︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ USE IT
Route Finance and Sales users to different metrics in Genie chat responses without changing their data permissions.
.genie_workspace_instructions.md
copy
# User-aware behavior
- Load the user's info to get their group memberships.
- If the user belongs to the Finance group, include gross margin and operating expenses in financial summaries.
- If the user belongs to the Sales group, include bookings and pipeline in performance summaries.
Customize chat behavior per user group by adding group-specific logic to .genie_workspace_instructions.md, including a Load the user's info instruction before group-specific rules.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Open any visualization in a side panel at larger size by clicking the expand control in the lower-right corner of the visualization card. Not checked yet
2 details in Everything
Databricks adds Tableau and Sigma as planned external knowledge sources and enables MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. connectors by default for compliance-profile workspaces︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Databricks-provided MCP connectors in Agent (Google Drive, Gmail, Google Calendar, Microsoft 365, Atlassian, Slack, and GitHub) will be enabled by default starting December 2026 for workspaces with the compliance security profile enabled.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Databricks plans to support Tableau and Sigma as external knowledge sources. Not checked yet
2 details in Everything
└──▷ BREAKING ON UPGRADE
!
Beginning January 2027, AI Functions that previously required DBR 15.4 LTS or above (except `ai_similarity`) now require DBR 17.3 LTS or above; calls on runtimes below 17.3 return an error.
Our best guess
Databricks Unity Gateway adds Moonshot AI Kimi K3.1 Pro, GitHub Action skill publishing, desktop AI agent support, and AI Functions budget tracking︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Publish Git-managed skills to Unity Gateway using a GitHub Action Beta : map folders to skills and trigger updates on merges to your publishing branch.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Configure Claude desktop and ChatGPT desktop work agents with governed models, MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. servers, and skills through Unity Gateway; subscription relay is available in Claude desktop. Not checked yet Unity Gateway now supports Moonshot AI Kimi K3.1 Pro (with vision, reasoning, and agentic tool use tool use A model's ability to answer with a structured request to run a named function or API, which the application executes and returns. It is how agents take actions. ) on pay-per-token endpoints in us-east-1. Not checked yet Unity Gateway budgets now include spend from AI Functions such as ai_parse_document, ai_extract, and ai_classify across batch and real-time execution paths. Not checked yet
Publish Git-managed skills to Unity Gateway using a GitHub Action Beta : map folders to skills and trigger updates on merges to your publishing branch.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Configure Claude desktop and ChatGPT desktop work agents with governed models, MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. servers, and skills through Unity Gateway; subscription relay is available in Claude desktop. Not checked yet Unity Gateway now supports Moonshot AI Kimi K3.1 Pro (with vision, reasoning, and agentic tool use tool use A model's ability to answer with a structured request to run a named function or API, which the application executes and returns. It is how agents take actions. ) on pay-per-token endpoints in us-east-1. Not checked yet
1 more in Everything
└──▷ BREAKING ON UPGRADE
! Databricks Runtime 18.2 and 18.2 for Machine Learning have reached End of Support.
└──▷ BREAKING ON UPGRADE
! Databricks Runtime 18.2 and 18.2 for Machine Learning have reached End of Support.
Reading masked fields on dedicated access mode compute now requires Runtime 15.4 LTS or above and serverless compute enabled; older runtimes return a permission error.
└──▷ BREAKING ON UPGRADE
!
Reading masked fields from compute that uses dedicated access mode requires Runtime 15.4 LTS or above and serverless compute enabled on the workspace — queries on older runtimes fail with 'Principal does not have USE SCHEMA on Schema system.__internal_functions'.
Our best guess
Databricks adds anomaly-detection GA, ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. on shared views, serverless log delivery to Unity Catalog volumes, access request GA, group-aware chat instructions, Unity Gateway agent support, and a large Lakeflow Designer October release with new operators, AI capabilities, checkpointing, and sharing features.
New supported platform Anomaly detection for data quality monitoring is now generally available; results land in the system.data_quality_monitoring.table_results system table. Not checked yet Deliver serverless job application logs to a Unity Catalog volume for retention and analysis Beta . Not checked yet Apply ABAC row filters and column masks to views, materialized views, and streaming tables shared through OpenSharing Beta . Not checked yet Request for access is now generally available; users can request privileges on Unity Catalog objects and track status in Catalog Explorer. Not checked yet Workspace admins can tailor Databricks Assistant chat behavior by group via workspace instructions for chat. Not checked yet
+3 more
show less
Configure Claude desktop and ChatGPT desktop work agents with governed models, MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. servers, and skills via Unity Gateway. Not checked yet Lakeflow Designer October release adds the ai_decide AI Function operator Beta , an HTML operator, shareable full-screen previews, checkpoint support (On or Auto ), Run up to this operator , new Prepare actions (Parse JSON , Rename columns , Multi column formula ), personal Google and Microsoft account connectors, Auto layout group , operator favorites, Sort by Modified , canvas operator @ references in AI prompts, and Reset preview overrides . Not checked yet Databricks Genie app in Microsoft Teams now shows chart visualizations as images in replies Public preview . Not checked yet
8 details in Everything
Databricks: Run Claude Code and Codex in SSH tunnel sessions, with model requests routed through Unity Gateway using your SSH credentials (CLI 1.23.0+).
└──▷ TRY IT
Grant broad secrets discoverability in a catalog without exposing secret values or allowing code references.
copy
$ GRANT BROWSE ON CATALOG <catalog_name> TO <principal>;
Trigger a Lakeflow Job when one or more upstream jobs complete, including across workspaces attached to the same metastore Beta . Not checked yet New Okta connector in Lakeflow Connect Beta ingests identity and access data — users, groups, group memberships, app integrations, devices, policies, and administrator roles — using SSWS API token authentication with incremental ingestion. Not checked yet Add AI Function tasks to jobs Beta to run ai_parse_document, ai_extract, or ai_classify on volume files or table rows without writing SQL, with incremental processing per run. Not checked yet Approve or deny Genie tool actions directly from Slack and Microsoft Teams approval prompts without leaving the chat Public preview . Not checked yet Allow traffic from Power BI, Tableau Cloud, dbt Cloud, and the Genie apps for Slack and Microsoft Teams in context-based ingress policies. Not checked yet
+1 more
show less
Pull Power BI semantic model metadata into Genie as external knowledge sources Beta , enabled by an account admin from the account console. Not checked yet
Trigger a Lakeflow Job when one or more upstream jobs complete, including across workspaces attached to the same metastore Beta . Not checked yet New Okta connector in Lakeflow Connect Beta ingests identity and access data — users, groups, group memberships, app integrations, devices, policies, and administrator roles — using SSWS API token authentication with incremental ingestion. Not checked yet Add AI Function tasks to jobs Beta to run ai_parse_document, ai_extract, or ai_classify on volume files or table rows without writing SQL, with incremental processing per run. Not checked yet Approve or deny Genie tool actions directly from Slack and Microsoft Teams approval prompts without leaving the chat Public preview . Not checked yet Pull Power BI semantic model metadata into Genie as external knowledge sources Beta , enabled by an account admin from the account console. Not checked yet
1 more in Everything
└──▷ BREAKING ON UPGRADE
!
AI Functions in SQL queries will no longer be supported on Databricks Runtime versions below 17.3 LTS starting January 2027; `ai_similarity` and the AI Functions REST API are unaffected.
Our best guess
Databricks Runtime 17.3 LTS now defaults Structured Streaming state store to RocksDB with changelog checkpointing enabled︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Databricks Runtime 19 adds WITH HISTORY and WITHOUT HISTORY options to ALTER SHARE ADD/ALTER VIEW︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ TRY IT
Explicitly exclude history when sharing a view in a Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. share, so recipients see only the current snapshot.
copy
$ ALTER SHARE my_share ADD VIEW my_catalog.my_schema.my_view WITHOUT HISTORY;
Databricks Asset Bundles adds workspace.sync_mode with immutable, in-place, and overwrite options to control how bundle files reach deployed resources︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ USE IT
Prevent non-admin users from modifying deployed production job files by using an immutable snapshot deployment.
databricks.yml
copy
sync_mode: immutable
Force a fresh immutable snapshot after a snapshot was modified outside the bundle, unblocking the next deployment.
copy
$ databricks bundle deploy --force
Control how bundle files are made available to deployed resources with the new workspace.sync_mode config key, accepting values overwrite, in-place, or immutable; defaults to in-place for development deployments from the workspace and overwrite otherwise (available from version 1.23.0).︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Lock deployed assets against modification by non-admin users by setting workspace.sync_mode: immutable, which uploads bundle files and artifacts as a single immutable snapshot; recover from an out-of-band snapshot change by running databricks bundle deploy --force. Not checked yet Reference workspace source files directly without uploading by setting workspace.sync_mode: in-place, replacing the deprecated presets.source_linked_deployment: true setting. Not checked yet Define named feature flags for a bundle using the new features mapping, where each key is the name of a feature (available from version 1.22.0). Not checked yet
4 details in Everything
└──▷ BREAKING ON UPGRADE
!
`presets.source_linked_deployment` is deprecated; set `workspace.sync_mode: in-place` instead of `source_linked_deployment: true`, or `workspace.sync_mode: overwrite` instead of `source_linked_deployment: false`.
Our best guess
Databricks adds UNDROP VOLUME and SHOW VOLUMES DROPPED commands to restore and list dropped volumes︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ TRY IT
Recover a volume accidentally dropped, then confirm it is no longer listed as dropped.
copy
$ SHOW VOLUMES DROPPED;
UNDROP VOLUME <volume_name>;
Restore a dropped volume with the new UNDROP VOLUME command.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet List dropped volumes with the new SHOW VOLUMES DROPPED command. Not checked yet
2 details in Everything
Databricks Genie now supports Volumes, letting users ask questions about unstructured files alongside structured data Public preview ︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Databricks: Databricks now supports Wikis Beta on tables, letting teams document a table's meaning, intended use, and sources inside Unity Catalog.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Databricks adds an AI Function task Beta for jobs, letting you schedule AI Functions without writing SQL and process only new or changed inputs each run.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ BREAKING ON UPGRADE
!
AI Functions will require Databricks Runtime 17.3 LTS or above (an upcoming change from the current 15.4 LTS minimum).
Our best guess
Databricks adds Secrets in Unity Catalog, enabling secret creation, access grants, and reads in notebooks, jobs, and UDFs︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Databricks now supports Secrets in Unity Catalog, letting you create secrets, grant access, and read them in notebooks, jobs, and UDFs.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Allow ChatGPT and Claude as partner platforms in context-based ingress policies (2026-11-10)
Nov 10, 2026
·
issue 137
Databricks adds ChatGPT and Claude as partner platforms in context-based ingress policies︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Text and binary inputs for protobuf requests to custom model serving endpoints (Public Preview) (2026-11-10)
Nov 10, 2026
·
issue 137
Databricks adds text and binary data support (images, audio, documents) in protobuf protobuf A language-neutral binary serialization format created by Google that encodes structured data more compactly and faster than JSON or XML, making it attractive for high-throughput inter-process or network communication. requests to custom model serving endpoints as KServe v2 KServe v2 An open inference protocol (maintained by the KServe community) that standardises how clients send prediction requests to ML model servers, letting cyber tools swap backends without changing their inference code. BYTES tensors BYTES tensors Tensors whose elements are raw byte values rather than numeric types, used in ML frameworks to handle arbitrary binary data or variable-length strings without numeric interpretation. ︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
The Genie One Chrome extension is in Beta (2026-11-10)
Nov 10, 2026
·
issue 137
Databricks adds the Genie One Chrome extension in Beta, making every new Chrome tab a workspace entry point for queries, search, and dashboards︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Set Genie One as your Chrome new tab page so every tab opens directly into your Databricks workspace for instant querying and dashboard access.
📍 1. Go to the Chrome Web Store and search for Genie One .
2. Click Add to Chrome to install the extension.
3. Open a new Chrome tab — it now loads Genie One in your Databricks workspace.
Install the Genie One Chrome extension from the Chrome Web Store to open your Databricks workspace in every new Chrome tab, giving one-click access to asking questions, searching across tools, and opening dashboards.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
External access to Unity Catalog managed Delta tables will soon be available by default for workspaces with the compliance security profile enabled (2026-11-10)
Nov 10, 2026
·
issue 136
Databricks adds default external access to Unity Catalog managed Delta tables for compliance security profile workspaces︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Anthropic Claude Fable 5.2 is now available on Unity Gateway (2026-11-09)
Nov 9, 2026
·
issue 136
Databricks adds Anthropic Claude Fable 5.2 to Unity Gateway, accessible via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. ︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Use cleaned AI answers in Lakeflow Designer (Public Preview) (2026-11-09)
Nov 9, 2026
·
issue 136
Databricks adds AI Function output cleaning in Lakeflow Designer, surfacing answer values, error messages, and raw responses in separate columns Public preview ︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
When building a pipeline in Lakeflow Designer, use the Clean step on an AI Function output to split results into separate columns before passing them to downstream transformations.
📍 In Lakeflow Designer, select an AI Function transformation node, then choose Clean to split the output into answer, error message, and raw response columns for downstream use.
Databricks Genie app for Slack is now generally available (2026-11-09)
Nov 9, 2026
·
issue 135
Databricks Genie app in Microsoft Teams is now generally available (2026-11-09)
Nov 9, 2026
·
issue 135
Genie Ontology is now generally available (2026-11-09)
Nov 9, 2026
·
issue 135
Databricks adds Genie Ontology as a generally available context layer for AI agents across Unity Catalog assets︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Genie Search is now generally available (2026-11-09)
Nov 9, 2026
·
issue 135
Databricks adds Genie Search (GA), indexing org knowledge across connected sources with permission-aware results inside Genie One.
Databricks adds Genie Search, a generally available knowledge-indexing capability that connects multiple sources so Genie One can search all indexed content in one place. Not checked yet Genie Search indexes each document's permissions alongside its content, so users see only documents they can already access in the source system. Not checked yet New integration Genie One now includes an org-directory feature powered by Genie Search that lets users find teammates, browse the org chart, and view profiles sourced from Workday. Not checked yet Genie One's Discover view now supports searching Databricks assets and content from connected sources in a single search experience via Genie Search. Not checked yet
4 details in Everything
Provision Claude desktop with MDM using Unity Gateway (2026-11-09)
Nov 9, 2026
·
issue 135
Databricks Unity Gateway adds ug configure --global to provision Claude desktop app machine-wide via MDM MDM Mobile device management, centrally configuring and managing enrolled devices. Administrators use it to apply settings, distribute applications and enforce device security policies. on macOS︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ TRY IT
Deploy a governed Claude desktop configuration to all managed Macs in your org without requiring users to configure it themselves.
copy
$ ug configure --global
Shift dates in Lakeflow Jobs parameters with date arithmetic (Beta) (2026-11-09)
Nov 9, 2026
·
issue 135
Databricks adds date arithmetic to Lakeflow Jobs dynamic value references, letting parameters shift dates by a duration before formatting︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ USE IT
Pass yesterday's date to a task parameter — useful for incremental ETL jobs that should process the previous day's data.
text
copy
{{(job.start_time - 1.day).iso_date}}
New jobs compute defaults to Auto access mode (2026-11-09)
Nov 9, 2026
·
issue 134
Databricks now defaults new jobs compute to Auto access mode, switching to Standard unless ML runtime, GPU GPU A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. , or Runtime < 14.3 is selected︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ BREAKING ON UPGRADE
!
New jobs compute created in the UI defaults to Auto access mode instead of Dedicated; workspaces with the compliance security profile enabled are affected in December 2026. Existing jobs are not changed, and a cluster policy enforcing a different access mode still takes precedence.
Our best guess
Recover dropped Unity Catalog volumes with UNDROP VOLUME (2026-11-07)
Nov 7, 2026
·
issue 135
Databricks adds UNDROP VOLUME WITH ID to recover dropped Unity Catalog volumes within 7 days︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ TRY IT
Restore an accidentally dropped Unity Catalog volume, preserving its original privileges and tags, within the 7-day recovery window.
copy
$ SHOW VOLUMES DROPPED;
-- Note the volume_id from the output, e.g. 'abc123'
UNDROP VOLUME WITH ID 'abc123';
Distinguish Lakeflow Designer Python previews from full runs (Public Preview) (2026-11-06)
Nov 6, 2026
·
issue 136
Databricks adds config["is_preview"] to Lakeflow Designer Python operators so code can skip side effects during pipeline previews︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ USE IT
Guard irreversible side effects (writes, API calls) so they are skipped when a designer is previewing the operator rather than running the full pipeline.
python
copy
if not config['is_preview']:
write_to_external_system(df)
Load Unity Catalog tables into PyTorch on AI Runtime (Beta) (2026-11-06)
Nov 6, 2026
·
issue 135
Databricks adds UCDataset to stream Unity Catalog tables directly into PyTorch training loops on AI Runtime, without Spark or a SQL warehouse︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Load Unity Catalog tables directly into PyTorch training loops on AI Runtime using UCDataset , with no Spark or SQL warehouse required.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Apply filter expressions and deletion vectors at read time with UCDataset , so only matching rows are streamed. Not checked yet UCDataset now splits matching rows across GPUs GPUs A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. and data loader workers automatically. Not checked yet
3 details in Everything
Install drivers for JDBC Unity Catalog connections from Maven (2026-11-06)
Nov 6, 2026
·
issue 133
Databricks adds Maven coordinate installation for JDBC JDBC A Java standard API for connecting applications to relational databases, maintained as part of the Java SE platform, letting tools query any compliant database through a single consistent interface. drivers in Unity Catalog connections︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Unity Catalog wikis are in Beta (2026-11-06)
Nov 6, 2026
·
issue 133
Databricks adds Unity Catalog wikis in Beta for collaborative table documentation in Catalog Explorer︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Enable the Unity Catalog wikis beta for your account so data stewards can begin documenting tables.
📍 In the Databricks Account Console, go to Settings › Previews, find Unity Catalog wikis , and enable it.
Databricks now supports wikis on Unity Catalog tables, letting data owners and stewards draft and publish documentation — manually or with Genie Code — while viewers can comment on, suggest edits to, and react to published wikis in Catalog Explorer.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Account admins can control access to Unity Catalog wikis from the Previews page. Not checked yet
2 details in Everything
Metric view listings are generally available in Databricks Marketplace (2026-11-05)
Nov 5, 2026
·
issue 133
Databricks Marketplace adds general availability for metric view listings, letting providers publish governed business metrics to Databricks recipients︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Allow partner platforms in context-based ingress policies (2026-11-05)
Nov 5, 2026
·
issue 132
Databricks adds Power BI, Tableau Cloud, dbt Cloud, and Genie apps for Slack and Microsoft Teams as allowable sources in context-based ingress policies︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Dashboard enhancements and bug fixes (2026-11-05)
Nov 5, 2026
·
issue 132
Databricks AI/BI dashboards add variable reordering, field display names, and Unity Catalog relationship import via Genie Code︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
When a field variable exposes raw column names, set readable labels so business users see Total revenue instead of Sum of sales without changing the underlying query.
📍 In the dashboard editor, open a field variable, select an option, and enter a display name in the Display name field. To bulk-update names, open Genie Code and type: 'Set the display name for the Revenue option to Total revenue'.
When authoring a dashboard over Unity Catalog metric views that already declare relationships, pull those relationships in automatically rather than redefining them by hand.
📍 In the dashboard editor, open Genie Code and type: 'Import relationships from Unity Catalog metric views'. Genie Code will add the declared relationships as dashboard relationships.
Reorder dashboard variables directly in the dashboard editor without workarounds.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Set a display name for each option in a field variable — for example, Total revenue instead of Sum of sales — and ask Genie Code to set, rename, or clear those display names while preserving ones you have already set. Not checked yet Import relationships declared on Unity Catalog metric views into a dashboard as dashboard relationships using Genie Code for dashboard authoring Beta . Not checked yet Waterfall chart labels now stay visible when they are close to the top or bottom of the axis. Not checked yet
Reorder dashboard variables directly in the dashboard editor without workarounds.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Set a display name for each option in a field variable — for example, Total revenue instead of Sum of sales — and ask Genie Code to set, rename, or clear those display names while preserving ones you have already set. Not checked yet Import relationships declared on Unity Catalog metric views into a dashboard as dashboard relationships using Genie Code for dashboard authoring Beta . Not checked yet
1 more in Everything
External knowledge sources are in Beta (2026-11-05)
Nov 5, 2026
·
issue 132
Databricks adds external knowledge sources Beta , enabling account admins to allow Genie to use Power BI measure definitions when answering questions.
New integration Enable Power BI semantic model metadata ingestion into Genie by turning on External knowledge from the account console Previews page, so Genie can use Power BI measure definitions when answering questions. Not checked yet
1 detail in Everything
Genie Agents enhancements (2026-11-05)
Nov 5, 2026
·
issue 132
Databricks Genie Agents now surface per-message OAuth identity claims in the Monitor tab and restrict answers to sources each user can access︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Track which app end user asked each question when an OAuth custom identity claim is in use — open the Monitor tab and add the identity column.
📍 In the Genie Agent UI, go to the Monitor tab, open the column picker, and enable the Custom claim column to see each message's OAuth custom identity claim alongside the message.
Genie Agents now answer using only the volumes or MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. connections a user can access, so partial-access users are no longer blocked from using an agent.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet The Monitor tab now shows each message's OAuth custom identity claim — enable the Custom claim column from the column picker to identify which end user sent each message. Not checked yet Benchmark evaluations now run questions in parallel, finishing sooner. Not checked yet
Genie Agents now answer using only the volumes or MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. connections a user can access, so partial-access users are no longer blocked from using an agent.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet The Monitor tab now shows each message's OAuth custom identity claim — enable the Custom claim column from the column picker to identify which end user sent each message. Not checked yet
1 more in Everything
Genie One enhancements (2026-11-05)
Nov 5, 2026
·
issue 132
Databricks Genie One adds OpenTelemetry OpenTelemetry An open standard and set of tools for collecting traces, metrics and logs from applications. It lets services export observability data without tying instrumentation to one monitoring vendor. trace export to the Unity Gateway unified trace table and an expanded visualization side panel︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Export Genie Code and Genie One conversation turns to the Unity Gateway unified trace table as OpenTelemetry spans — a workspace admin enables tracing per product from the Previews page; by default only the trace table owner can query the traces.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Open a visualization from a Genie One response in a side panel to interact with it at a larger size next to the conversation. Not checked yet
2 details in Everything
More external knowledge sources (2026-11-05)
Nov 5, 2026
·
issue 132
Run Claude Code and Codex in the SSH tunnel (2026-11-05)
Nov 5, 2026
·
issue 132
Databricks adds Claude Code and Codex support in SSH tunnel sessions, routing model requests through Unity Gateway︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Moonshot AI Kimi K3.1 Pro is now available on Unity Gateway (2026-11-04)
Nov 4, 2026
·
issue 132
Publish Unity Gateway Skills with a GitHub Action in Beta (2026-11-04)
Nov 4, 2026
·
issue 132
Databricks adds a GitHub Action Beta to publish Git-managed Unity Gateway skills to Unity Catalog on branch merge︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
AI Function tasks for jobs (Beta) (2026-11-04)
Nov 4, 2026
·
issue 131
Databricks adds AI Function tasks Beta for jobs, enabling no-SQL ai_parse_document, ai_extract, and ai_classify on volumes or tables with incremental runs.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Add an AI Function task to a job to classify rows in a table without writing SQL, writing results to a Unity Catalog table for downstream tasks.
📍 In the Jobs UI, open your job and click Add task , then select AI Function as the task type, choose ai_classify as the function, point it at your input table, and specify a Unity Catalog output table.
Run ai_parse_document, ai_extract, or ai_classify on files in a volume or rows of a table as a job task, with incremental processing of new or changed inputs and results written to a Unity Catalog table.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Control Lakebase sink upserts with merge strategies (2026-11-03)
Nov 3, 2026
·
issue 130
Databricks adds merge strategies for Lakebase sink upserts in Structured Streaming, controlling column-level update behavior on Databricks Runtime 19+︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Databricks now supports merge strategies when writing to Lakebase from Structured Streaming, letting you keep the first or last value, ignore nulls, or select values by an ordering column during upserts — requires Databricks Runtime 19 or above.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Okta connector is available in Beta (2026-11-03)
Nov 3, 2026
·
issue 130
Trigger Lakeflow Jobs when upstream jobs complete (Beta) (2026-11-03)
Nov 3, 2026
·
issue 130
Databricks adds cross-workspace upstream job completion triggers for Lakeflow Jobs in Beta︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Trigger a downstream Lakeflow Job in another workspace as soon as an upstream job finishes, without manual scheduling.
📍 In the Databricks UI, open the downstream job, go to Triggers, select Add trigger , choose Job completion , and specify the upstream job (including cross-workspace jobs attached to the same Unity Catalog metastore) as a dependency.
Configure a Lakeflow Job to run automatically when one or more upstream jobs complete, including jobs in other workspaces sharing the same Unity Catalog metastore, by setting dependencies on the downstream job.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Zerobus Ingest into default storage is now generally available (2026-11-02)
Nov 2, 2026
·
issue 130
Databricks adds general availability of Zerobus Ingest into default storage tables in Lakeflow Connect︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Databricks Runtime 19: November 2, 2026 (2026-11-02)
Nov 2, 2026
·
issue 129
Databricks Runtime 19 adds WITH HISTORY and WITHOUT HISTORY options to ALTER SHARE ADD/ALTER VIEW︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ TRY IT
Add a view to a Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. share while exposing its full history to recipients.
copy
$ ALTER SHARE my_share ADD VIEW my_catalog.my_schema.my_view WITH HISTORY;
Genie Agents enhancements (2026-11-02)
Nov 2, 2026
·
issue 129
Databricks Genie Agent now analyzes unstructured files in Unity Catalog volumes and answers with source-page citations︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Enable Genie Agent file analysis for Unity Catalog volumes so your workspace can query PDFs and slide decks with cited answers.
📍 In the Databricks workspace, go to Settings › Previews and enable 'Analyze files in volumes with a Genie Agent'.
Higher limits and BROWSE support for Unity Catalog secrets (2026-11-02)
Nov 2, 2026
·
issue 129
Databricks adds BROWSE privilege support for Unity Catalog secrets and raises per-schema and per-metastore secret limits︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Discover Unity Catalog secrets and their metadata without read access to values or code references by granting BROWSE on a catalog, now extended to cover Unity Catalog secrets.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Store up to 1,000 secrets per schema and 10,000 per metastore in Unity Catalog, up from 100 and 1,000, for both Databricks-managed secrets and secrets backed by AWS Secrets Manager. Not checked yet
Discover Unity Catalog secrets and their metadata without read access to values or code references by granting BROWSE on a catalog, now extended to cover Unity Catalog secrets.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 more in Everything
Managed warehouse for MLflow (Beta) (2026-11-02)
Nov 2, 2026
·
issue 127
Databricks adds managed warehouse for MLflow traces so users can view Unity Catalog traces without selecting a SQL warehouse Beta ︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Workspace admins enabling managed warehouse so analysts can view MLflow traces without configuring a SQL warehouse
📍 In the Databricks console, go to Settings › Previews and enable Managed Warehouse for MLflow .
AI Functions usage is now included in Unity Gateway budgets (2026-11-01)
Nov 1, 2026
·
issue 130
Databricks now includes AI Functions spend in Unity Gateway budgets for batch and real-time execution paths︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Monitor data freshness in Lakeflow Connect database connector pipelines (Public Preview) (2026-10-30)
Oct 30, 2026
·
issue 130
Databricks adds a Data freshness column and event-log fields to Lakeflow Connect CDC pipelines, with alerting support, in Public Preview.
New supported platform A new Data freshness column in the Lakeflow Connect pipeline tables list shows time since the latest ingested source commit for each CDC table. Not checked yet Data freshness and end-to-end lag are queryable from the ingestion pipeline's event log, with alert creation supported. Not checked yet
2 details in Everything
Claude Code auto mode for Anthropic and non-Anthropic models (2026-10-30)
Oct 30, 2026
·
issue 126
Databricks adds Claude Code auto mode support for Anthropic and non-Anthropic models to reduce routine permission prompts︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Moonshot AI Kimi K2.7 is retired (2026-10-30)
Oct 30, 2026
·
issue 126
└──▷ BREAKING ON UPGRADE
!
Moonshot AI Kimi K2.7 is retired as of October 30, 2026; workloads using it must migrate to Kimi K3 or another supported model.
Our best guess
Relationships between metric views (Beta) (2026-10-30)
Oct 30, 2026
·
issue 126
Databricks adds relationship constraints between Unity Catalog metric views so AI assistants and BI tools choose the right join path automatically︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Define how metric views join each other — including cardinality and join condition — using ALTER VIEW ...︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ ADD CONSTRAINT ... RELATIONSHIP, so AI assistants and BI tools automatically select the correct join path; requires Databricks Runtime 19 or a SQL warehouse. Not checked yet
1 detail in Everything
Thinking Machine Labs Inkling is retired (2026-10-30)
Oct 30, 2026
·
issue 126
└──▷ BREAKING ON UPGRADE
!
Thinking Machine Labs Inkling is retired as of October 30, 2026; workloads using it must migrate to GLM 5.3 or Kimi K3.
Our best guess
Track Unity Gateway skill usage with audit logs and the usage dashboard (2026-10-30)
Oct 30, 2026
·
issue 126
Databricks adds audit logging and a usage dashboard tab for Unity Gateway Skill activity︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Audit which users are loading or fetching Unity Gateway Skills by filtering your audit log stream for the aiGatewaySkills service.
📍 In the Databricks console, go to Settings › Audit Logs and filter by service name aiGatewaySkills to see skill list, load, and file fetch events by user.
Monitor download and load activity across all Unity Gateway Skills for capacity or compliance review.
📍 In the Databricks console, go to the Usage dashboard and select the Skills tab to view downloads, loads, and activity broken down by user.
Track Unity Gateway Skill activity — list, load, and file fetch events — through audit logs recorded under the aiGatewaySkills service.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet View skill downloads, loads, and per-user activity in the new Skills tab of the usage dashboard. Not checked yet
2 details in Everything
Microsoft Graph Logs connector is available in Beta (2026-10-30)
Oct 30, 2026
·
issue 125
Native XML file format support is now generally available (2026-10-29)
Oct 29, 2026
·
issue 126
Databricks adds generally available native XML file format support for batch and streaming ingestion via Auto Loader, read_files, COPY INTO, and SQL functions︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ USE IT
Extract structured fields from an XML column already stored in a Delta table using SQL.
sql
copy
SELECT from_xml(xml_col, schema_of_xml('<record><id>1</id><name>foo</name></record>')) AS parsed
FROM my_table;
Ingest, query, and parse XML files in batch or streaming workloads using Auto Loader , read_files, and COPY INTO.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Parse XML data in SQL using the from_xml and schema_of_xml functions. Not checked yet
2 details in Everything
Real-time mode in Lakeflow pipelines is now generally available (2026-10-29)
Oct 29, 2026
·
issue 126
Databricks adds general availability of real-time mode in Lakeflow pipelines for sub-5ms end-to-end latency on classic and serverless compute︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Dashboard relationships are now generally available (2026-10-29)
Oct 29, 2026
·
issue 125
Databricks adds generally available dashboard relationships for building multi-fact data models scoped to AI/BI dashboards︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Genie Agents enhancements and bug fixes (2026-10-29)
Oct 29, 2026
·
issue 125
Databricks Genie Agents adds guided agent creation and volume file links in embedded agents︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Databricks Genie Agents adds a guided creation experience that walks through building an effective Genie Agent.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Databricks Genie Agents now retrieves files faster when answering questions Beta . Not checked yet Content search details — such as the number of files indexed — are now hidden from users with CAN VIEW or CAN RUN permission on a Genie Agent Beta . Not checked yet Links to volume files in an embedded Genie Agent now open the file directly instead of redirecting users to the workspace. Not checked yet
Databricks Genie Agents adds a guided creation experience that walks through building an effective Genie Agent.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
3 more in Everything
Xero connector is available in Beta (2026-10-29)
Oct 29, 2026
·
issue 125
Multiple real-time flows per serverless pipeline and source stage adaptive repartitioning are in Beta (2026-10-28)
Oct 28, 2026
·
issue 125
Databricks adds multiple real-time flows per serverless pipeline and source stage adaptive repartitioning in Beta︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Run multiple real-time mode flows in one serverless pipeline without flows competing for task slots, available on the preview channel in Beta.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Source stage adaptive repartitioning now automatically adjusts the number of tasks reading from Kafka or Kinesis sources to match traffic, removing the need to manually set maxPartitions or shardsPerTask. Not checked yet
2 details in Everything
View personal and shared spend limits with the ug CLI (2026-10-28)
Oct 28, 2026
·
issue 125
Databricks adds spend-limit visibility to the ug CLI, showing personal and shared limits sorted by highest usage percentage︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ TRY IT
See every personal and shared spend limit in your account to audit which budgets are close to exhaustion.
View up to five personal and shared spend limits sorted by highest percentage used with ug usage, or see all applicable limits with ug usage --all.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Serverless GPU for clean room notebooks is now generally available (2026-10-28)
Oct 28, 2026
·
issue 124
Databricks adds serverless GPU GPU A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. compute for clean room notebooks, enabling model inference and training over shared data︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Serverless GPU in clean rooms will soon be on by default for accounts where it's turned off (2026-10-28)
Oct 28, 2026
·
issue 124
Databricks adds serverless GPU GPU A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. support in clean rooms, now available via account-level preview toggle︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Turn on serverless GPU for clean rooms early, before the automatic November 2026 rollout, so your team can run notebooks on GPU compute within a clean room now.
📍 In the account console, go to Previews and enable Clean Rooms AI Runtime .
Enable serverless GPU in clean rooms before the November 26, 2026 default rollout by turning on Clean Rooms AI Runtime from the Previews page in the account console.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Unity Gateway Usage Dashboard v1.0 (2026-10-27)
Oct 27, 2026
·
issue 124
Databricks adds the Unity Gateway Usage Dashboard v1.0 with coding-agent adoption tracking, model and user spend analysis, and tag-based spend attribution︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Navigate to the Unity Gateway Usage Dashboard to review coding-agent adoption and session spend broken down by tag, model, and user.
📍 In the Databricks console, go to Unity Gateway › Usage Dashboard to explore coding-agent adoption, harness usage, estimated session spend, and tag-based spend attribution.
Databricks now offers the Unity Gateway Usage Dashboard v1.0 for tracking coding-agent adoption, harness usage, and estimated session spend.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet The dashboard now supports tag-based spend attribution and expanded model and user spend analysis. Not checked yet
2 details in Everything
AI Functions Unity Catalog permissions is now generally available (2026-10-27)
Oct 27, 2026
·
issue 123
Databricks AI Functions Unity Catalog permissions reach GA, letting metastore admins control access to task-specific AI Functions via the system.ai schema.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Default OAuth apps for external AI agents are now generally available (2026-10-27)
Oct 27, 2026
·
issue 123
External AI agents can now authenticate to Databricks MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. servers via default OAuth client IDs, eliminating manual OAuth app setup by account admins.
New integration Enables ChatGPT, Codex, Claude, Claude Code, and Cursor to connect to Databricks MCP servers (including the Genie One MCP server) using default OAuth client IDs, with no OAuth app creation required from an account admin. Not checked yet
1 detail in Everything
Editing files in Unity Catalog volumes is now generally available (2026-10-27)
Oct 27, 2026
·
issue 122
Edit files in Unity Catalog volumes directly in the Databricks workspace file editor — no download/upload required.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Edit a config or data file in a Unity Catalog volume directly without leaving the workspace.
📍 In the workspace, open Catalog Explorer, navigate to your Unity Catalog volume, select a file, and choose Edit to modify it in the workspace file editor — no download or re-upload needed.
Enables in-place editing of files stored in Unity Catalog volumes via the workspace file editor, accessible from Catalog Explorer or the schema browser in a notebook or SQL editor.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
OpenAI GPT-6.1 Terra is now available on Unity Gateway (2026-10-27)
Oct 27, 2026
·
issue 122
OpenAI GPT-6.1 Terra is now available on Unity Gateway via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Account-level security features are now available by default at GA (2026-10-26)
Oct 26, 2026
·
issue 123
Databricks account-level security features (network, encryption, compliance) now auto-available at GA — no manual preview opt-in required.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Account-level security features covering network connectivity (private connectivity), network security (ingress/egress controls), encryption, and enhanced security and compliance are now automatically available to all accounts upon General Availability, removing the requirement for account admins to manually enable them on the Previews page.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Improved Lakeflow Performance Observability is now generally available (2026-10-26)
Oct 26, 2026
·
issue 122
Lakeflow Jobs gains GA job-level query performance metrics and an Optimize button that opens Genie Code for query improvement.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Review query performance bottlenecks and open Genie Code for AI-assisted query tuning after a serverless job run.
📍 In the Lakeflow Jobs UI, open a completed job run, select a supported task, view the query performance metrics and data volume metrics panel, then click Optimize to open Genie Code with the performance insights loaded.
Improved Lakeflow Performance Observability will soon be available by default for workspaces with the compliance security profile enabled (2026-10-26)
Oct 26, 2026
·
issue 122
Databricks: Adds job-level query performance metrics and insights to Lakeflow Jobs for supported tasks, enabled by default in compliance security profile workspaces.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Review job-level data volume and query performance metrics for a serverless Lakeflow job to identify slow queries before they affect SLAs.
📍 In the Lakeflow Jobs UI, open a job run, navigate to the Query Performance tab to view metrics and insights, then click the Optimize button to open Genie Code for AI-assisted query improvements.
Supports query profiling for serverless compute, SQL Warehouses, and Lakeflow Pipelines. Not checked yet
1 detail in Everything
QuickBooks connector is available in Beta (2026-10-26)
Oct 26, 2026
·
issue 122
PayPal connector is available in Beta (2026-10-24)
Oct 24, 2026
·
issue 122
Files API 3.0 for Unity Catalog volumes is now in Beta (2026-10-23)
Oct 23, 2026
·
issue 119
Files API 3.0 Beta adds multipart uploads for files larger than 5 GB and conditional operations to Unity Catalog volumes.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Adds multipart upload support in Files API 3.0, enabling uploads of files larger than 5 GB to Unity Catalog volumes.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Adds conditional operations in Files API 3.0, allowing upload, download, and delete of files in Unity Catalog volumes based on their current version to prevent unintended changes. Not checked yet
2 details in Everything
Google Analytics Aggregated connector is available in Beta (2026-10-23)
Oct 23, 2026
·
issue 119
Lakeflow Connect adds a managed Google Analytics Aggregated connector Beta for ingesting GA4 report data into Databricks.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Enable the Google Analytics Aggregated connector Beta for your workspace before configuring ingestion pipelines.
📍 In the Databricks workspace, go to Settings › Previews, find the Google Analytics Aggregated connector, and toggle it on.
Adds a managed Google Analytics Aggregated connector to Lakeflow Connect Beta , supporting incremental ingestion of Google Analytics 4 report data — user activity, acquisition, engagement, monetization, demographics, and tech reports.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Supports custom report definitions with user-defined dimensions, metrics, and aggregation level. Not checked yet Supports both service account and OAuth user-to-machine (U2M) authentication for the Google Analytics Aggregated connector. Not checked yet Workspace admins can enable or disable the Google Analytics Aggregated connector from the Previews page. Not checked yet
4 details in Everything
Box Audit Logs managed ingestion connector (Beta) (2026-10-23)
Oct 23, 2026
·
issue 117
DataFrame caching on serverless compute is now in Public Preview (2026-10-23)
Oct 23, 2026
·
issue 117
DataFrame caching via df.cache() and df.persist() is now in Public Preview on serverless compute, enabled by default for most workspaces.
New supported platform Enables df.cache() and df.persist() on serverless compute to reuse expensive DataFrame results within a session. Not checked yet Workspace admins can disable the preview from the Previews page. Not checked yet
2 details in Everything
Dashboard enhancements (2026-10-22)
Oct 22, 2026
·
issue 119
AI/BI Dashboards gain private bookmarks, bulk import of up to 100 Power BI/Tableau files, layered combo charts, and custom waterfall totals.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Migrate a large Power BI or Tableau workspace to Databricks AI/BI dashboards all at once rather than one file at a time.
📍 In the Databricks UI, open Genie Code, select Import multiple files in bulk , and upload up to 100 Power BI (.pbix) or Tableau (.twbx/.twb) files to convert them to AI/BI dashboards and metric views.
Let analysts return quickly to a frequently used filter state without affecting other users' dashboard views.
📍 While viewing a dashboard, apply the desired filters, open the bookmarks menu, and select Save a private bookmark — the saved filter combination is visible only to your account.
Adds bulk import via Genie Code to bring up to 100 Power BI or Tableau files at a time into AI/BI dashboards and metric views.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Adds private bookmarks so dashboard viewers can save and return to filter combinations visible only to themselves. Not checked yet Adds layered combo charts, letting authors build combo charts from separate chart layers. Not checked yet Adds customizable totals on waterfall charts. Not checked yet Adds full-name tooltips when hovering over a field or measure name in the field picker. Not checked yet
Adds bulk import via Genie Code to bring up to 100 Power BI or Tableau files at a time into AI/BI dashboards and metric views.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Adds private bookmarks so dashboard viewers can save and return to filter combinations visible only to themselves. Not checked yet Adds layered combo charts, letting authors build combo charts from separate chart layers. Not checked yet
2 more in Everything
Genie One enhancements (2026-10-22)
Oct 22, 2026
·
issue 119
Genie One adds a conversation outline for navigation, skill-file uploads, and mobile email prefill prefill The first phase of inference, where the model reads the whole prompt in parallel before producing its first token. It drives time-to-first-token; decode is the token-by-token phase after it. .
Adds Upload skills button to upload existing skill files and folders directly to your skills folder in the workspace Public preview . Not checked yet Adds conversation outline along the left edge of Genie One conversations to jump between messages and hover-preview responses. Not checked yet The Genie mobile app now remembers and prefills the last signed-in email address at next sign-in Public preview . Not checked yet
3 details in Everything
Serverless job environments from pyproject.toml (Beta) (2026-10-22)
Oct 22, 2026
·
issue 119
Serverless job tasks can now read dependencies from a pyproject.toml file to configure their environment.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Enable Project-Scoped Shared Compute so serverless job tasks can pick up dependencies from a pyproject.toml file.
📍 In the Databricks workspace, go to Settings › Previews and enable Project-Scoped Shared Compute .
Serverless notebook and Python script tasks support pyproject.toml for configuring job environments, configurable via the Jobs UI, Jobs API, or a bundle.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Workspace admins can enable Project-Scoped Shared Compute on the Previews page to use this feature. Not checked yet
2 details in Everything
Shared Serverless for Asset Bundles (Beta) (2026-10-22)
Oct 22, 2026
·
issue 119
Databricks Asset Bundles can now share serverless compute via a bundle-root pyproject.toml environment definition Beta .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Activate Project-Scoped Shared Compute so your workspace can use Shared Serverless for Asset Bundles.
📍 1. In your Databricks workspace, go to Settings › Previews.
2. Locate Project-Scoped Shared Compute and toggle it on.
Microsoft Entra ID connector is available in Beta (2026-10-22)
Oct 22, 2026
·
issue 117
Genie Code tasks in Lakeflow Jobs are available by default for workspaces with the compliance security profile enabled starting in November 2026 (2026-10-21)
Oct 21, 2026
·
issue 117
Databricks: Genie Code tasks now run by default in Lakeflow Jobs on compliance-profile workspaces, autonomously as job steps with links to their chats.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Genie Code tasks in Lakeflow Jobs are now generally available (2026-10-21)
Oct 21, 2026
·
issue 117
Genie Code tasks in Lakeflow Jobs reach GA, enabling autonomous prompt execution as part of automated job pipelines.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Adds Genie Code task type to Lakeflow Jobs, which runs a prompt autonomously, reads upstream task outputs, calls tools as needed, and returns a link to the resulting Genie Code chat.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Lakeflow pipelines system tables are now generally available (GA) (2026-10-21)
Oct 21, 2026
·
issue 117
Lakeflow pipelines and pipeline_update_timeline system tables reach GA, giving full account-level pipeline audit data.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Promotes the lakeflow system tables pipelines and pipeline_update_timeline to generally available, recording pipeline definitions and update history across your account.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Screen MCP tool calls and results for jailbreaks (Beta) (2026-10-21)
Oct 21, 2026
·
issue 116
Unity Gateway gains jailbreak and prompt injection prompt injection An attack where text the model reads, such as a web page, file or tool result, carries instructions that override the user's. It is the main security risk for agents with tools. screening for MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. tool calls and results Beta .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Asana connector is available in Beta (2026-10-20)
Oct 20, 2026
·
issue 115
Oracle Eloqua connector is available in Beta (2026-10-20)
Oct 20, 2026
·
issue 115
Databricks: Adds a managed Oracle Eloqua connector to Lakeflow Connect Beta for ingesting Eloqua data into Databricks.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Enable the Oracle Eloqua connector Beta for your workspace before building an ingestion pipeline.
📍 In the Databricks console, go to Settings › Previews, find Oracle Eloqua connector , and toggle it on.
Supports OAuth user-to-machine (U2M) authentication, incremental ingestion, UI-based pipeline authoring, Declarative Automation Bundles, and API-based pipeline authoring for the Oracle Eloqua connector. Not checked yet Workspace admins can enable or disable the Oracle Eloqua connector from the Previews page. Not checked yet
2 details in Everything
SurveyMonkey connector is available in Beta (2026-10-20)
Oct 20, 2026
·
issue 115
DeepSeek V4.1 Flash (1017) is now available on Unity Gateway (2026-10-19)
Oct 19, 2026
·
issue 115
Filter data per end user with OAuth custom identity claims (2026-10-19)
Oct 19, 2026
·
issue 115
Databricks lets you embed custom identity claims in OAuth tokens to filter rows per end user via dynamic views or ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. policies.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Lakebase Insights moves to per-project settings, with on-demand runs (Beta) (2026-10-19)
Oct 19, 2026
·
issue 115
Lakebase Insights moves to per-project Settings with on-demand runs, replacing the old observability configuration workflow.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Trigger an immediate, unscheduled Insights analysis on a specific project without waiting for a scheduled run.
📍 In the Databricks UI, open the project, go to Settings › Insights, and click Run now .
Enable Insights for a workspace so telemetry begins flowing into system.lakebase for all subsequent project analyses.
📍 In the Databricks UI, go to the workspace admin panel, open Previews, and enable Lakebase Insights .
Enables Lakebase Insights configuration directly in each project's Settings for users with CAN MANAGE, removing the need to create a separate observability configuration.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Supports on-demand Insights runs at any time, with or without a scheduled analysis frequency set in Settings . Not checked yet Reads telemetry from system.lakebase system tables to diagnose issues and recommend next steps. Not checked yet Enabling the Lakebase Insights preview also auto-enables Lakebase Observability in System Tables if not already active; workspace admins can enable the observability preview independently to capture telemetry without Insights. Not checked yet
Enables Lakebase Insights configuration directly in each project's Settings for users with CAN MANAGE, removing the need to create a separate observability configuration.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Supports on-demand Insights runs at any time, with or without a scheduled analysis frequency set in Settings . Not checked yet
2 more in Everything
└──▷ BREAKING ON UPGRADE
! Jobs scheduled through an observability configuration have stopped and their schedules do not carry over; those configurations no longer write telemetry to their Unity Catalog destinations, so dashboards retain historical data but receive no new data. To resume scheduled insights, enable Insights and set an analysis frequency in each project's Settings .
Delegate access to Genie budgets (2026-10-16)
Oct 16, 2026
·
issue 115
Databricks Genie budgets now support ACLs ACLs An access control list, a set of rules allowing or denying operations for particular users, services or network addresses. It decides who can use a resource. , letting admins delegate Budget Manager or Budget Viewer roles to users, groups, or service principals.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Delegate budget access with access control lists (ACLs) (2026-10-16)
Oct 16, 2026
·
issue 115
└──▷ HOW TO FIND IT
Delegate read-only budget visibility to a service principal so automated reporting jobs can access cost data without admin rights.
📍 In the Databricks console, go to your Account › Budgets, select a budget, open Permissions, and assign the Budget Viewer role to the desired user, group, or service principal.
File upload in Genie One and Genie Agents is now generally available (2026-10-16)
Oct 16, 2026
·
issue 115
File upload in Genie One and Genie Agents is now GA, enabling natural-language analysis of uploaded files alongside workspace data.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
File upload in Genie One and Genie Agents will soon be available by default for compliance security profile workspaces with supported compliance standards (2026-10-16)
Oct 16, 2026
·
issue 115
File upload in Genie One and Genie Agents — upload files to a conversation and analyze them in natural language alongside your other data.
Updated compliance framework Enables file upload in Genie One and Genie Agents by default for workspaces with the compliance security profile enabled and a supporting compliance standard, allowing natural-language analysis of uploaded files alongside other data. Not checked yet
1 detail in Everything
TypeSafe AI Jev 1.14.0 is now available on Unity Gateway (2026-10-16)
Oct 16, 2026
·
issue 111
READ METADATA privilege support for compliance security profile standards (2026-10-16)
Oct 16, 2026
·
issue 110
The READ METADATA privilege is now enabled by default for workspaces with the compliance security profile active.
Consumer access to query Unity Gateway services is generally available (2026-10-15)
Oct 15, 2026
·
issue 111
Consumer access to Unity Gateway services is now GA — no account-admin preview toggle required to query shared model and MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. services.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Account admins can set budgets and rate limits, or disable direct model access, for consumer access users to control increased model traffic to external providers.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Dashboard enhancements and bug fixes (2026-10-15)
Oct 15, 2026
·
issue 111
AI/BI dashboards add Genie-powered insight cards, pivot table drill-down, and theme font inheritance for rich text widgets.
Adds Dashboard Insights Beta : Genie analyzes charts on a published dashboard page and surfaces trends and outliers as insight cards for viewers. Not checked yet Adds pivot table drill-down, letting viewers traverse row and column hierarchies one level at a time. Not checked yet Rich text widget titles and descriptions now inherit the dashboard theme's font styles. Not checked yet
3 details in Everything
Genie Agents enhancements (2026-10-15)
Oct 15, 2026
·
issue 111
Genie Agent mode gains Flash Beta for speed-first responses and upgraded models for long-context tasks.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Adds Flash Beta option to Agent mode, prioritizing response speed over depth for simpler, everyday tasks.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Upgrades Agent mode underlying models for better long-context handling and faster responses. Not checked yet
Adds Flash Beta option to Agent mode, prioritizing response speed over depth for simpler, everyday tasks.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 more in Everything
Genie One enhancements (2026-10-15)
Oct 15, 2026
·
issue 111
Genie One gains collaborative editing, group-membership context, and conversation recall for faster repeated investigations.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Enables collaborative document editing in Genie One — users with CAN EDIT permission can now edit shared documents.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Genie One now returns a user's group memberships alongside user info, letting workspace instructions and table descriptions tailor responses per group. Not checked yet Genie One can now reuse context from past investigations to answer similar questions faster. Not checked yet
Enables collaborative document editing in Genie One — users with CAN EDIT permission can now edit shared documents.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Genie One can now reuse context from past investigations to answer similar questions faster. Not checked yet
1 more in Everything
Lakebase Search supports Chinese, Japanese, and Korean full-text search (2026-10-15)
Oct 15, 2026
·
issue 111
lakebase_tokenizer 0.2.0 adds CJK full-text search via tokenizer_chinese, tokenizer_japanese, and tokenizer_korean built-in configurations.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ TRY IT
Upgrade an existing lakebase_tokenizer installation to 0.2.0 to gain CJK support without rebuilding existing configurations or indexes.
copy
$ ALTER EXTENSION lakebase_tokenizer UPDATE;
Adds the tokenizer_morph template to lakebase_tokenizer 0.2.0, which uses morphological analysis with bundled language dictionaries to segment CJK text — supporting compound-word modes (whole, split, or both), Japanese particle/function-word filtering and base-form conversion, and Korean particle/ending filtering.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet CJK tokenizer output is compatible with GIN indexes, the lakebase_text index, and the lakebase_bm25 index for BM25 BM25 The classic keyword-ranking formula behind traditional search engines. Hybrid search combines it with embedding similarity so exact terms and meaning both count. ranking, so existing index infrastructure works unchanged. Not checked yet
Adds the tokenizer_morph template to lakebase_tokenizer 0.2.0, which uses morphological analysis with bundled language dictionaries to segment CJK text — supporting compound-word modes (whole, split, or both), Japanese particle/function-word filtering and base-form conversion, and Korean particle/ending filtering.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 more in Everything
MySQL integrated CDC pipeline is now generally available (2026-10-15)
Oct 15, 2026
·
issue 111
Replicate data in and out of Lakebase with Postgres logical replication (2026-10-15)
Oct 15, 2026
·
issue 111
Lakebase gains native Postgres logical replication, enabling live inbound migration with sync-until-cutover and outbound change publishing to external Postgres databases.
New integration Supports native Postgres logical replication to stream changes into Lakebase, enabling live migration from an existing Postgres database with continuous sync until cutover. Not checked yet Supports publishing changes from Lakebase to external Postgres databases via logical replication. Not checked yet
2 details in Everything
The agentic code converter is generally available for SQL conversion (2026-10-15)
Oct 15, 2026
·
issue 111
Databricks agentic code converter is GA for SQL, converting T-SQL, Snowflake, Redshift, Oracle, BigQuery BigQuery A serverless, cloud-hosted analytical database from Google that runs SQL queries across massive datasets without managing infrastructure, giving cyber tools scalable storage and fast querying over large volumes of log or event data. , and Teradata dialects into Databricks SQL.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
ADBC driver for Power BI is now generally available (2026-10-15)
Oct 15, 2026
·
issue 110
ADBC ADBC Arrow Database Connectivity, an Apache standard API for querying databases that returns results directly as Apache Arrow columnar buffers, eliminating per-row copying overhead and making bulk data transfer fast for analytics pipelines. driver for the Databricks connector in Power BI is now generally available and set as the default for new connections.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Databricks adds a Data Science Agent that builds entire notebooks — EDA, forecasting, ML — from a single chat prompt.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Workspace-level package repositories for Databricks Apps are generally available (2026-10-14)
Oct 14, 2026
·
issue 106
Databricks Apps gains GA workspace-level default Python and npm package repository configuration with per-app overrides.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Workspace admins can configure default Python and npm package repositories for Databricks Apps, with app-level overrides still supported.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
External writes to managed Delta tables are now generally available (2026-10-13)
Oct 13, 2026
·
issue 111
Databricks now generally supports external Apache Spark clients writing to Unity Catalog managed Delta tables.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
AI Runtime will soon be available by default for workspaces with the compliance security profile enabled (2026-10-13)
Oct 13, 2026
·
issue 109
AI Runtime gains support for compliance security profile workspaces on AWS and Azure, enabling serverless GPU GPU A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. model training and fine-tuning without infrastructure provisioning.
Databricks AI Runtime SDK replaces serverless_gpu (2026-10-13)
Oct 13, 2026
·
issue 109
Databricks AI Runtime SDK (databricks-sdk-air) replaces serverless_gpu in GPU GPU A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. environment v6, bringing new databricks.air namespace.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Introduces the databricks-sdk-air package with @distributed and ray_init APIs and databricks.air.data utilities under the databricks.air namespace, replacing the deprecated serverless_gpu package starting in GPU environment version 6.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Renames gpus and gpu_type arguments to num_accelerators and accelerator_type; the old names still work in databricks.air but emit a FutureWarning and will be removed in version 7. Not checked yet
Introduces the databricks-sdk-air package with @distributed and ray_init APIs and databricks.air.data utilities under the databricks.air namespace, replacing the deprecated serverless_gpu package starting in GPU environment version 6.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 more in Everything
└──▷ BREAKING ON UPGRADE
! GPU environment version 7 will drop support for the gpus and gpu_type argument names entirely; callers must migrate to num_accelerators and accelerator_type before upgrading past version 6.! Version 6 is the last GPU environment version that includes the serverless_gpu package; workloads on version 7 or later must migrate to databricks-sdk-air.
External access to views through the Unity REST API is generally available (2026-10-13)
Oct 13, 2026
·
issue 109
Unity REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. now supports external engine queries against Unity Catalog views over Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. tables (GA).︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Google Gemini 4 Argon is now available on Unity Gateway (2026-10-13)
Oct 13, 2026
·
issue 108
On-demand state repartitioning is now generally available (2026-10-13)
Oct 13, 2026
·
issue 108
Databricks GA: resize shuffle partitions in stateful Structured Streaming or Lakeflow pipelines without losing checkpoint state.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Context-based ingress policies for inbound private connectivity are now generally available (2026-10-12)
Oct 12, 2026
·
issue 108
Context-based ingress policies for inbound Private Link Private Link An AWS networking feature that exposes a service over a private endpoint inside a VPC, so traffic never traverses the public internet, giving cyber tools a way to integrate with cloud services without opening public routes. reach GA, letting admins combine VPC VPC A logically isolated virtual network inside a public cloud provider's infrastructure, letting tools deploy resources with private addressing, controlled routing, and firewall rules without managing physical hardware. endpoint rules with identity and request-type conditions.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Enable context-based ingress policies for a Databricks account where the preview has not yet been turned on.
📍 In the account console, go to Previews and enable Context-Based Ingress: Workspace Private Access Policies , then configure your policy under Configure private access policy.
Enables the Context-Based Ingress: Workspace Private Access Policies feature via the account console Previews page for accounts where it is not yet active.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Agent memory pipelines are in Beta (2026-10-12)
Oct 12, 2026
·
issue 106
Databricks launches Agent memory pipelines in Beta, letting agents distill and recall facts across conversations.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Adds Agent memory pipelines Beta , which connect a session store to a memory store and use an LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. to distill durable facts, preferences, and decisions from stored conversations into memory entries recalled in later sessions.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Supports running a memory pipeline on demand or on an approximately 24-hour schedule. Not checked yet
2 details in Everything
Feature Views are now generally available (2026-10-12)
Oct 12, 2026
·
issue 106
Databricks Feature Views reach GA for batch, streaming, and request-time data.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Feature Views will soon be available by default for workspaces with the compliance security profile enabled (2026-10-12)
Oct 12, 2026
·
issue 106
Databricks Feature Views reach GA and will auto-enable for compliance security profile workspaces in mid-November 2026.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Feature Views are now generally available, enabling declarative feature definition and computation from batch or streaming data for model training and online serving.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Governed tag propagation is in Beta (2026-10-12)
Oct 12, 2026
·
issue 106
Unity Catalog adds Beta tag propagation — governed tags auto-apply to derived tables and views via lineage.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Parameters for Lakeflow pipelines are now generally available (2026-10-12)
Oct 12, 2026
·
issue 106
Lakeflow pipeline parameters are now GA, letting a single pipeline target different catalogs, schemas, or tables via key-value overrides.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Unity Gateway Skills API and developer tools are available in Beta (2026-10-12)
Oct 12, 2026
·
issue 106
Unity Gateway Skills REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. launches in Beta with full CRUD CRUD The four basic database operations — Create, Read, Update, Delete — used as shorthand for any system that manages persistent data records, giving cyber tools a standard model for interacting with stored information. plus finalize, available across all Databricks SDKs and CLI.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Adds the Unity Gateway Skills REST API in Beta, supporting create, read, update, list, delete, and finalize operations.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Exposes Skills API support across Databricks CLI, Python SDK, Go SDK, Go Modular SDK, Java SDK, and JavaScript SDK. Not checked yet
2 details in Everything
Databricks AI Functions adds ai_decide for evaluating questions against text or structured data with probability/choice/score output.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Adds ai_decide SQL function and REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. to evaluate questions against text or structured data and return probabilities, choices, or scores.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
JDBC Unity Catalog connections will soon be available by default for workspaces with the compliance security profile enabled (2026-10-09)
Oct 9, 2026
·
issue 106
JDBC JDBC A Java standard API for connecting applications to relational databases, maintained as part of the Java SE platform, letting tools query any compliant database through a single consistent interface. Unity Catalog connections with SSL SSL A cryptographic protocol for encrypting network connections, now deprecated in favor of TLS but whose name persists colloquially. Cyber tools reference it when handling HTTPS or other encrypted-channel inspection and termination. certificate support become available by default for compliance security profile workspaces.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Unity Gateway supports Amazon Bedrock Mantle (2026-10-09)
Oct 9, 2026
·
issue 106
Code execution in Genie One (Public Preview) (2026-10-09)
Oct 9, 2026
·
issue 100
Genie One gains secure sandbox code execution for advanced analyses beyond SQL Public preview .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Use OpenJev as the evaluator for LLM-as-a-judge service policies (Beta) (2026-10-09)
Oct 9, 2026
·
issue 100
Databricks adds OpenJev (Qwen3.5 4B) as a faster, token-free evaluator option for LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. -as-a-judge service policies Beta .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Use request tags in Unity Gateway service policies (Beta) (2026-10-09)
Oct 9, 2026
·
issue 100
Unity Gateway service policies can now enforce caller-supplied request tags before model requests or MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. tool calls proceed.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Google Workspace connector is available in Beta (2026-10-09)
Oct 9, 2026
·
issue 097
Databricks Genie agents now support attaching MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. connectors to access external tools and data sources Beta .
New integration Attach MCP connectors to a Genie agent to enable access to external tools and sources such as Google Drive and Slack Beta . Not checked yet
1 detail in Everything
DeepSeek V4.1 Flash supports priority pay-per-token and reserved provisioned throughput (2026-10-07)
Oct 7, 2026
·
issue 097
DeepSeek V4.1 Flash now available with priority pay-per-token and reserved provisioned throughput on Databricks.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Adds priority pay-per-token serving for DeepSeek V4.1 Flash to reduce latency for latency-sensitive traffic.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Adds reserved provisioned throughput for DeepSeek V4.1 Flash for dedicated, predictable capacity. Not checked yet
2 details in Everything
CREATE OR REPLACE VIEW will soon preserve grants, comments and tags by default (2026-10-06)
Oct 6, 2026
·
issue 097
CREATE OR REPLACE VIEW will preserve grants, tags, and comments in-place by default starting December 2, 2026.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ TRY IT
Explicitly drop grants, tags, and comments when replacing a view — necessary after December 2, 2026 when the default changes to in-place preservation.
copy
$ CREATE OR REPLACE VIEW main.analytics.my_view NOCOPY * AS SELECT id, name FROM main.raw.source_table;
Adds NOCOPY * modifier to CREATE OR REPLACE VIEW <catalog>.<schema>.<view> NOCOPY * to explicitly drop grants, tags, and comments when replacing a view (governed tags are still retained).︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Raises a CANNOT_DROP_TAGGED_COLUMN error when a governed tag on a view column is not removed before dropping that column via CREATE OR REPLACE VIEW, protecting dependent ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. policies. Not checked yet
Adds NOCOPY * modifier to CREATE OR REPLACE VIEW <catalog>.<schema>.<view> NOCOPY * to explicitly drop grants, tags, and comments when replacing a view (governed tags are still retained).︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 more in Everything
└──▷ BREAKING ON UPGRADE
! Starting December 2, 2026, CREATE OR REPLACE VIEW no longer drops and re-creates the view — it updates in-place, preserving grants, tags, and comments. Workflows that relied on CREATE OR REPLACE VIEW to reset grants or tags will need to use CREATE OR REPLACE VIEW <catalog>.<schema>.<view> NOCOPY * explicitly, or remove governed tags from columns before dropping them.
OAuth custom identity claims for service principal tokens will soon be available (2026-10-05)
Oct 5, 2026
·
issue 097
Databricks adds OAuth custom identity claims for service principal tokens, enabling per-user row filtering through a single service principal.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ TRY IT
Request a service principal token with a custom identity claim so downstream SQL queries can filter rows to only that end user's data.
copy
$ curl -X POST https://<workspace-host>/oidc/v1/token \
-H 'Content-Type: application/x-www-form-urlencoded' \
-d 'grant_type=client_credentials' \
-d 'client_id=<sp-client-id>' \
-d 'client_secret=<sp-client-secret>' \
-d 'scope=all-apis' \
-d 'custom_claim=alice@example.com'
Create a dynamic view that isolates each end user's rows by comparing a column to the custom identity claim on the token — write the filter to match no rows when the claim is NULL.
sql
copy
CREATE OR REPLACE VIEW user_orders_view AS
SELECT * FROM orders
WHERE user_email = current_oauth_custom_identity_claim()
AND current_oauth_custom_identity_claim() IS NOT NULL;
Adds custom_claim parameter to OAuth token requests using client_credentialsclient_credentials An OAuth 2.0 grant type where a service authenticates directly with an authorization server using its own ID and secret, with no user involved, to obtain an access token for machine-to-machine API calls. or urn:ietf:params:oauth:grant-type:token-exchange grant types, letting applications attach a custom identity claim to service principal tokens.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Adds current_oauth_custom_identity_claim() SQL function to read the custom identity claim value at query time, returning NULL when no claim is present. Not checked yet The Genie Conversation API, the SQL Statement Execution API, and the Databricks JDBC JDBC A Java standard API for connecting applications to relational databases, maintained as part of the Java SE platform, letting tools query any compliant database through a single consistent interface. Driver honor the custom_claim, enabling per-user data isolation through a single service principal via dynamic view row filters. Not checked yet
Adds custom_claim parameter to OAuth token requests using client_credentialsclient_credentials An OAuth 2.0 grant type where a service authenticates directly with an authorization server using its own ID and secret, with no user involved, to obtain an access token for machine-to-machine API calls. or urn:ietf:params:oauth:grant-type:token-exchange grant types, letting applications attach a custom identity claim to service principal tokens.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Adds current_oauth_custom_identity_claim() SQL function to read the custom identity claim value at query time, returning NULL when no claim is present. Not checked yet
1 more in Everything
The remote_query function will soon be available by default for workspaces with the compliance security profile enabled (2026-10-05)
Oct 5, 2026
·
issue 097
Databricks enables remote_query table-valued function by default for compliance security profile workspaces in November 2026.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
ai_parse_document now extracts chart data as JSON (2026-10-05)
Oct 5, 2026
·
issue 097
ai_parse_document now extracts chart values, series, legends, and axes as structured JSON for search and agent use.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Run Lakeflow pipelines as a group (2026-10-05)
Oct 5, 2026
· release notes
Databricks allows setting a Lakeflow pipeline's run-as identity to an account group.
Adds the ability to set a pipeline's run-as identity to an account group Not checked yet Pipeline updates now execute with the permissions of the configured group Not checked yet
2 details in Everything
Databricks AI Functions gains ai_decide — a new SQL function that evaluates questions against text or structured data and returns probabilities, choices, or scores.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Customer-managed keys for query history are now generally available (2026-10-02)
Oct 2, 2026
·
issue 096
Databricks GA: customer-managed keys now decrypt query history fields statement_text and error_message in the system catalog.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
JDBC Unity Catalog connections are now generally available (2026-10-02)
Oct 2, 2026
·
issue 093
JDBC JDBC A Java standard API for connecting applications to relational databases, maintained as part of the Java SE platform, letting tools query any compliant database through a single consistent interface. Unity Catalog connections reach GA on serverless compute, Databricks Runtime 19+, and SQL warehouses 2025.40+︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Customer-managed keys for query history are now generally available
Oct 2, 2026
· release notes
Customer-managed keys for query history are now generally available
Adds customer-managed key support for decrypting statement_text and error_message in the query history system table. Not checked yet
1 detail in Everything
OpenJev (Qwen3.5 4B) is now available on Unity Gateway
Oct 2, 2026
· release notes
OpenJev (Qwen3.5 4B) pay-per-token model available on Unity Gateway
Adds OpenJev (Qwen3.5 4B) pay-per-token model service accessible via the TypeSafe System One API Not checked yet Enables evaluation of yes/no, choice, and scoring questions with structured answers and probabilities Not checked yet
2 details in Everything
Cross-engine attribute-based access controls (ABAC) is now generally available (2026-10-01)
Oct 1, 2026
·
issue 093
Cross-engine ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. is now GA: external engines can read Unity Catalog tables with row filters and column masks enforced by Databricks serverless compute.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Dashboard enhancements and bug fixes (2026-10-01)
Oct 1, 2026
·
issue 092
Databricks AI/BI dashboards gain SQL dataset materialization Beta , multi-table data source selection, and Genie Code image widget authoring.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Extends dashboard materialization Beta to cover SQL datasets in addition to local metric views.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Allows selecting multiple tables simultaneously in the Add data source picker when building a dashboard data model. Not checked yet Promotes numeric window measures to general availability in the low-code metric view UI. Not checked yet Embedded dashboards can now show, collapse, or hide global filters by default. Not checked yet Scheduled emails now preserve attachments that fit within the size limit instead of dropping all attachments when the combined image and PDF exceed it. Not checked yet
+3 more
show less
Genie Code can use images stored in your workspace as image widgets, not only uploaded files. Not checked yet Genie Code can create dashboard materializations on request Beta . Not checked yet Dashboard canvas stays visible while Genie Code builds datasets step by step. Not checked yet
Extends dashboard materialization Beta to cover SQL datasets in addition to local metric views.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Allows selecting multiple tables simultaneously in the Add data source picker when building a dashboard data model. Not checked yet Genie Code can create dashboard materializations on request Beta . Not checked yet
5 more in Everything
Genie One enhancements (2026-10-01)
Oct 1, 2026
·
issue 092
Genie One gains .txt file uploads, streaming responses, chat download, and personalized sample questions.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Download a Genie One chat session to share or archive a conversation with your team.
📍 In Genie One, open the sidebar and select Manage chats , then choose the chat you want and select Download .
Supports uploading plaintext .txt files to a Genie One conversation, in addition to previously supported file types.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Streams Genie One responses in real time as they are generated, so answers appear progressively rather than all at once. Not checked yet Enables downloading a Genie One chat via the Manage chats sidebar. Not checked yet Displays personalized sample questions beneath the search bar on the Genie One homepage. Not checked yet
4 details in Everything
LTAP Direct Writes is generally available (2026-10-01)
Oct 1, 2026
·
issue 092
Databricks Lakebase: LTAP Direct Writes is GA, accelerating initial loads and full refreshes by bypassing the compute endpoint.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Opt in a new synced table to LTAP Direct Writes to avoid large initial loads competing with live queries on your compute endpoint.
📍 When creating a synced table in the Lakebase UI: go to Synchronization settings and select LTAP Direct Writes . This option is available for Lakebase projects running Postgres 16, 17, or 18.
Schedule automatic data refresh in the Databricks Excel Add-in (Public Preview) (2026-10-01)
Oct 1, 2026
·
issue 092
Databricks Excel Add-in gains scheduled automatic data refresh Public preview with interval and calendar options.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Cross-engine attribute-based access controls (ABAC) is now generally available
Oct 1, 2026
· release notes
Databricks cross-engine ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. GA: external engines can now read Unity Catalog tables with enforced row filters and column masks.
Control spreadsheet writeback with a workspace admin setting (2026-09-30)
Sep 30, 2026
·
issue 092
Workspace admins can now gate Excel and Google Sheets writeback via the Allow spreadsheet writeback security setting.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ BREAKING ON UPGRADE
! Excel and Google Sheets writeback now requires a workspace admin to explicitly enable the Allow spreadsheet writeback security setting — existing writeback workflows will stop working until the setting is enabled.
Salesforce Data 360 file sharing will soon be available by default for workspaces with the compliance security profile enabled (2026-09-30)
Sep 30, 2026
·
issue 092
Salesforce Data 360 file sharing connector becomes available by default for compliance security profile workspaces, enabling federated queries on Databricks compute.
Query tags for SQL warehouses are now generally available (2026-09-30)
Sep 30, 2026
·
issue 091
Query tags for SQL warehouses are now GA, enabling custom key-value cost attribution visible in system.query.history and the UI.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Supported Postgres extensions updated to new versions (2026-09-30)
Sep 30, 2026
·
issue 091
Existing extension installations retain their current version until manually upgraded via ALTER EXTENSION.
Table update triggers on OpenSharing objects and system tables are now generally available (2026-09-30)
Sep 30, 2026
·
issue 091
Lakeflow Jobs table update triggers now GA for OpenSharing objects and system tables on Databricks.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Databricks JDBC Driver (Simba) 2.8.4 is available (2026-09-29)
Sep 29, 2026
·
issue 091
Databricks JDBC JDBC A Java standard API for connecting applications to relational databases, maintained as part of the Java SE platform, letting tools query any compliant database through a single consistent interface. Driver (Simba) 2.8.4 adds Java 25 support.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
ABAC on views is in Beta (2026-09-29)
Sep 29, 2026
·
issue 090
Databricks ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. row filters and column masks now extend to views in Unity Catalog (Beta, DBR 19+).︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Enable the ABAC on Views beta so your team can apply existing row filters and column masks to views in Unity Catalog.
📍 In the account console, go to Previews and enable ABAC on Views .
Extends ABAC row filters and column masks to views (in addition to tables), enabling Unity Catalog policies to control access to sensitive data exposed through views.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Enables the ABAC on Views preview via the account console's Previews page (requires Databricks Runtime 19 or above). Not checked yet
Extends ABAC row filters and column masks to views (in addition to tables), enabling Unity Catalog policies to control access to sensitive data exposed through views.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 more in Everything
Agent Bricks CLI is in Beta (2026-09-29)
Sep 29, 2026
·
issue 090
Databricks launches Agent Bricks CLI (databricks-agentbricks) in Beta for building and deploying custom code agents.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Adds databricks-agentbricks CLI for scaffolding agent projects from bundled framework templates, running them locally, and deploying to the Databricks agent runtime with managed memory, sessions, tools, and MLflow tracing.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
OpenAI GPT-6.1 Sol is now available on Unity Gateway (2026-09-29)
Sep 29, 2026
·
issue 090
OpenAI GPT-6.1 Sol is now available on Databricks Unity Gateway via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Horizontal scaling for Databricks Apps is now generally available (2026-09-29)
Sep 29, 2026
·
issue 088
Databricks Apps now supports horizontal scaling GA: multiple instances behind one URL with zero-downtime deployments and session affinity.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Genie One memory is now generally available (2026-09-29)
Sep 29, 2026
· release notes
Genie One memory GA enables persistent context retention in Databricks
└──▷ HOW TO FIND IT
Use when you want the AI to apply your specific reporting style without repeating it every time.
📍 In the chat, ask: 'Remember that I prefer tables with bold headers and no trailing zeros', then confirm by accepting the suggestion.
Adds Genie One memory capability to retain preferences and reporting conventions across conversations Not checked yet Allows users to ask Genie One to remember facts or accept its suggestions during a chat session Not checked yet
2 details in Everything
Horizontal scaling for Databricks Apps is now generally available â
Sep 29, 2026
· release notes
Horizontal scaling for Databricks Apps is now generally available.
Enables running a Databricks app across multiple instances behind a single app URL for higher availability and concurrency. Not checked yet
1 detail in Everything
Databricks JDBC Driver (Simba) 2.8.4 is available â
Sep 29, 2026
· release notes
Databricks JDBC JDBC A Java standard API for connecting applications to relational databases, maintained as part of the Java SE platform, letting tools query any compliant database through a single consistent interface. Driver (Simba) 2.8.4 adds Java 25 support.
OpenAI GPT-6.1 Sol is now available on Unity Gateway â
Sep 29, 2026
· release notes
OpenAI GPT-6.1 Sol now accessible via Unity Gateway Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. .
Lakebase Search adds a configurable text tokenizer for keyword search (2026-09-28)
Sep 28, 2026
·
issue 090
Lakebase Search gains lakebase_tokenizer, a configurable full-text tokenizer with stopwords, synonyms, stemming, and Unicode normalization.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Adds the lakebase_tokenizer PostgreSQL extension to control how Lakebase Search tokenizes text, with configurable normalization (lowercase, Unicode, accent-stripping), stopword and synonym lists managed in SQL, and a native English stemmer with possessive-stripping.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Produces standard tsvector output, making lakebase_tokenizer compatible with to_tsvector, GIN indexes, and the lakebase_bm25 index for BM25 BM25 The classic keyword-ranking formula behind traditional search engines. Hybrid search combines it with embedding similarity so exact terms and meaning both count. ranking. Not checked yet
2 details in Everything
1Password Event Logs connector (Beta) (2026-09-28)
Sep 28, 2026
·
issue 089
Lakeflow Connect gains a 1Password Event Logs connector Beta ; workspace admins control access via the Previews page.
New integration Adds a 1Password Event Logs connector in Lakeflow Connect Beta , ingesting audit events, item usages, and sign-in attempts from the 1Password Events API into Databricks using an Events Reporting bearer token bearer token A credential that grants access to whoever presents it, without a separate proof of possession. Protecting the token matters because a stolen copy can be used by someone else. . Not checked yet Workspace admins can enable or disable the 1Password Event Logs connector from the Previews page. Not checked yet
2 details in Everything
CrowdStrike Falcon Event Stream connector (Beta) (2026-09-28)
Sep 28, 2026
·
issue 089
Workspace admins control Beta access via the Previews page; authentication requires a CrowdStrike API client with Event streams Read permission using OAuth 2.0 OAuth 2.0 An open authorization framework (IETF RFC 6749) that lets a service grant limited API access to a third party without sharing credentials, giving cyber tools a standard way to authenticate against external platforms. client credentials.
New integration Workspace admins can enable or disable the CrowdStrike Falcon Event Stream connector via the Previews page. Not checked yet
1 detail in Everything
Microsoft 365 Unified Audit Logs connector is available in Beta (2026-09-28)
Sep 28, 2026
·
issue 089
Databricks Lakeflow Connect adds a managed Microsoft 365 Unified Audit Logs connector Beta using OAuth machine-to-machine auth with incremental ingestion.
Workday Activity Logging connector (Beta) (2026-09-28)
Sep 28, 2026
·
issue 089
Databricks Lakeflow Connect introduces a Beta Workday Activity Logging connector that incrementally ingests Workday user activity log events using OAuth refresh tokens refresh tokens A credential used to obtain a new access token without signing in again. It is sent to the authorization service rather than used directly to call the protected API. .
Okta System Logs connector is available in Beta â
Sep 28, 2026
· release notes
Okta System Logs connector is available in Beta for Lakeflow Connect
Supports SSWS API token authentication for the new Okta connector. Not checked yet
1 detail in Everything
Microsoft 365 Unified Audit Logs connector is available in Beta â
Sep 28, 2026
· release notes
Databricks Lakeflow Connect now supports Beta ingestion of Microsoft 365 audit logs via a new connector using OAuth machine-to-machine auth and incremental ingestion.
Akamai WAF WAF A reverse proxy that inspects HTTP/HTTPS traffic and blocks requests matching known attack patterns such as SQL injection or XSS, sitting between clients and a web application to filter malicious input before it reaches the server. connector in Beta allows ingesting WAF events into Lakeflow Connect using EdgeGrid credentials.
Adds Akamai WAF connector Beta to Lakeflow Connect for workspace admins. Not checked yet Enables ingestion of web application firewall security events from Akamai into Databricks via the new connector. Not checked yet
2 details in Everything
1Password Event Logs connector (Beta) â
Sep 28, 2026
· release notes
New beta 1Password Event Logs connector ingests audit events and sign-in attempts into Databricks Lakeflow Connect
xAI Grok 4.7 is now available on Unity Gateway â
Sep 28, 2026
· release notes
xAI Grok 4.7 is now available on Unity Gateway via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves.
Workday Activity Logging connector (Beta) â
Sep 28, 2026
· release notes
Databricks Lakeflow Connect adds Beta Workday Activity Logging connector for incremental ingestion of user activity events via OAuth refresh tokens refresh tokens A credential used to obtain a new access token without signing in again. It is sent to the authorization service rather than used directly to call the protected API.
Workspace admins can enable or disable access to the Beta connector from the Previews page in the Databricks UI. Not checked yet
1 detail in Everything
CrowdStrike Falcon Event Stream connector (Beta) â
Sep 28, 2026
· release notes
Databricks adds Beta CrowdStrike Falcon Event Stream connector via Lakeflow Connect
Ingests Falcon Event Stream events from CrowdStrike Falcon into Databricks using OAuth 2.0 OAuth 2.0 An open authorization framework (IETF RFC 6749) that lets a service grant limited API access to a third party without sharing credentials, giving cyber tools a standard way to authenticate against external platforms. client credentials. Not checked yet Workspace admins can control access to the new Beta connector from the Previews page. Not checked yet
2 details in Everything
Usage tracking adds session metadata and token details (Public Preview) (2026-09-27)
Sep 27, 2026
·
issue 089
system.ai_gateway.usage gains coding-agent session metadata, invocation source, prompt-cache token fields, and search counts.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Adds token-detail fields to system.ai_gateway.usage capturing tokens written to the 5-minute and 1-hour prompt caches, plus file and web search counts.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Usage tracking adds session metadata and token details (Public Preview) â
Sep 27, 2026
· release notes
System tables gain session metadata, token cache details, and search counts in Public Preview.
system.ai_gateway.usage table now includes coding-agent session metadata and invocation source Not checked yet Token-detail fields capture tokens written to the 5-minute and 1-hour prompt caches Not checked yet New file and web search counts are included in usage tracking Not checked yet
3 details in Everything
Databricks: Adds ai_detect_anomalies SQL function to identify anomalous values in historical time series data and return expected values, prediction bounds, and anomaly scores.
Databricks adds automatic sensitive data classification and tagging within Unity Catalog.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Automatically classify and tag sensitive data in Unity Catalog with the new Data Classification capability.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Genie Code can now build intelligent document processing pipelines (2026-09-25)
Sep 25, 2026
·
issue 086
Genie Code can now orchestrate end-to-end intelligent document processing pipelines for PDFs, contracts, and invoices.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Tags for dashboards, notebooks, Databricks apps, and Genie Agents are now generally available (2026-09-25)
Sep 25, 2026
·
issue 085
Databricks GA: apply key-value tags to dashboards, notebooks, apps, and Genie Agents for workspace asset organization.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
The Genie One MCP server is now generally available (2026-09-25)
Sep 25, 2026
·
issue 084
Genie One MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. server hits GA as system.ai.genie_one_mcp in Unity Gateway, exposing Genie to any MCP client with Unity Catalog permissions enforced.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ BREAKING ON UPGRADE
! The Beta endpoint https://<workspace-hostname>/api/2.0/mcp/genie is deprecated and will be sunset on October 31, 2026; all workloads must be moved to the system.ai.genie_one_mcp MCP Service before that date.
Tags for dashboards, notebooks, Databricks apps , and Genie Agents are now generally available â
Sep 25, 2026
· release notes
Tags for dashboards, notebooks, Databricks apps and Genie Agents are now generally available
Apply tags to organize and categorize workspace assets (dashboards, notebooks, Databricks apps, and Genie Agents) Not checked yet Supports both governed and ungoverned key-value tag formats Not checked yet
2 details in Everything
Deleting a Unity Catalog pipeline now retains tables by default â
Sep 25, 2026
· release notes
Databricks Unity Catalog pipelines now retain tables/materialized views by default on delete.
The Pipelines DELETE API retains materialized views, streaming tables, and views by default instead of deleting them. Not checked yet Setting the cascade field to true in the API request allows deleting a pipeline along with its associated tables. Not checked yet
2 details in Everything
The Genie One MCP server is now generally available â
Sep 25, 2026
· release notes
Genie One MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. server generally available via system.ai.genie_one_mcp in Unity Gateway
The previous Beta endpoint /api/2.0/mcp/genie is deprecated and will be removed on October 31, 2026. Not checked yet
1 detail in Everything
└──▷ BREAKING ON UPGRADE
! The endpoint /api/2.0/mcp/genie is deprecated and will stop working after October 31, 2026.
Genie Code can now build intelligent document processing pipelines â
Sep 25, 2026
· release notes
Databricks Genie Code now builds end-to-end document processing pipelines for unstructured data.
Databricks AI Functions gains ai_predict_class and ai_predict_value for in-SQL classification and regression prediction.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Adds ai_predict_class SQL function to train a classification model and predict class labels for rows where the target column is NULL.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Adds ai_predict_value SQL function to train a regression model and predict numeric values for rows where the target column is NULL. Not checked yet
2 details in Everything
ai_search can search Lakebase synced tables (Beta) (2026-09-24)
Sep 24, 2026
·
issue 089
ai_search now supports Lakebase synced tables via a new lakebase_search knowledge source, combinable with vector_search in one call Beta .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Dashboard enhancements and bug fixes (2026-09-24)
Sep 24, 2026
·
issue 085
Databricks AI/BI dashboards gain column pinning, natural-language filtering, LOD expressions LOD expressions A formula language used in Tableau for computing aggregations at a specified level of detail, independent of the view's granularity, giving cyber tools precise control over how metrics are scoped and grouped. , and per-widget filter overrides.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Prevent a KPI widget from being affected by page-level date or region filters so it always shows the full baseline metric.
📍 In the dashboard editor, select the widget you want to isolate, open its settings panel, and enable Ignore filters so the widget displays unfiltered data while all other widgets on the page continue to respond to dashboard filters.
Adds column pinning and formatting, search, a column explorer, natural-language filtering, and a selection details panel to the dataset results table and metric view preview grid.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Adds include-style level of detail (LOD) expressions to the metric view editor. Not checked yet Enables configuring a dashboard widget to ignore selected dashboard filters while those filters continue to apply to the rest of the page. Not checked yet
Adds column pinning and formatting, search, a column explorer, natural-language filtering, and a selection details panel to the dataset results table and metric view preview grid.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Enables configuring a dashboard widget to ignore selected dashboard filters while those filters continue to apply to the rest of the page. Not checked yet
1 more in Everything
Genie Agents enhancements (2026-09-24)
Sep 24, 2026
·
issue 085
Genie Agents gains column picker for monitoring table and page-level citations for volume file answers.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Narrow the monitoring table to only the columns relevant to your team's triage workflow.
📍 In the Genie Agent, go to Monitor tab › agent monitoring table › column picker, then select or deselect the columns you want to display.
Adds a column picker to the agent monitoring table on the Monitor tab, letting you choose which columns are displayed.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Adds page-level citations Beta so that when a Genie Agent answers using content from an attached volume file, citations point to the specific page of the source file. Not checked yet
2 details in Everything
Genie One enhancements (2026-09-24)
Sep 24, 2026
·
issue 085
Genie One adds chat pinning and MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. write actions in the mobile app Beta .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Adds the ability to pin a chat to keep it at the top of the Genie One sidebar.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Adds MCP write actions support in the Genie One mobile app Beta , enabling write operations from connected external tools. Not checked yet
2 details in Everything
Sharing metric views with OpenSharing is now generally available (2026-09-24)
Sep 24, 2026
·
issue 085
Databricks GA: share metric views via OpenSharing and query them with MEASURE(...) on serverless compute.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Sharing metric views with OpenSharing is now generally available â
Sep 24, 2026
· release notes
OpenSharing now supports sharing and querying metric views on serverless compute via MEASURE(...)
ai_search can search Lakebase synced tables (Beta) â
Sep 24, 2026
· release notes
Adds lakebase_search knowledge source to ai_search for querying Lakebase synced tables
Adds lakebase_search as a new knowledge source type in the ai_search function. Not checked yet Allows combining lakebase_search sources with vector_search sources in a single ai_search call. Not checked yet
2 details in Everything
Time travel queries now enforce ABAC policies (Beta) (2026-09-23)
Sep 23, 2026
·
issue 085
Databricks Runtime 19 adds ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. row filter and column mask enforcement for Unity Catalog time travel queries Beta .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Build governed data and AI apps without code (Beta) (2026-09-23)
Sep 23, 2026
·
issue 084
Databricks adds no-code governed AI app building via Genie App Builder, App Spaces, and Serverless Micro Apps Beta .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Adds Genie App Builder to author governed data and AI apps in natural language without writing code Beta .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Adds App Spaces to enforce governance policies across apps built with Genie App Builder Beta . Not checked yet Adds Serverless Micro Apps that scale to zero when idle Beta . Not checked yet
3 details in Everything
Scala UDFs can access Unity Catalog secrets (2026-09-23)
Sep 23, 2026
·
issue 084
Scala UDFs can now read Unity Catalog secrets via a SECRETS clause, with caller or owner permissions depending on UDF scope.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ TRY IT
Define a Unity Catalog Scala UDF that reads a secret without exposing it to callers — the SECRETS clause grants owner-permission access so callers need no direct secret privilege.
copy
$ CREATE OR REPLACE FUNCTION my_catalog.my_schema.fetch_api_key()
RETURNS STRING
LANGUAGE SCALA
SECRETS ('my_scope' = 'my_key')
ENVIRONMENT_VERSION 6
AS $$
dbutils.secrets.get(scope = "my_scope", key = "my_key")
$$;
Unity Catalog Scala UDFs declare required secrets in a SECRETS clause and run under the function owner's permissions, so callers need no direct secret access.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Requires environment_version set to 6 or above for Unity Catalog Scala UDFs (and session-scoped UDFs on serverless compute) to access secrets; on classic compute, requires Databricks Runtime 19 or above with standard access mode. Not checked yet Limits secret-accessing Unity Catalog Scala UDFs to serverless SQL warehouses only among SQL warehouse types; dedicated access mode compute does not support direct invocation. Not checked yet
3 details in Everything
Scalar and Batch Unity Catalog Python UDFs are now generally available (2026-09-23)
Sep 23, 2026
·
issue 084
Unity Catalog Python UDFs reach GA with named handlers, dependency installs, and credential access — five-UDF-per-query limit removed.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Supports named handlers for row-at-a-time scalar UDFs in Unity Catalog Python UDFs.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Removes the five-UDF-call-per-query limit for Unity Catalog Python UDFs. Not checked yet Enables installing custom dependencies in Unity Catalog Python UDFs. Not checked yet Enables access to Unity Catalog service credentials and secrets from within Python UDF functions. Not checked yet Sets environment_version to 6 or above in the UDF definition to unlock scalar UDF capabilities on serverless compute and other newly GA features. Not checked yet
+1 more
show less
Aligns scalar TIMESTAMP inputs with PySpark behavior for Unity Catalog Python UDFs. Not checked yet
Supports named handlers for row-at-a-time scalar UDFs in Unity Catalog Python UDFs.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Sets environment_version to 6 or above in the UDF definition to unlock scalar UDF capabilities on serverless compute and other newly GA features. Not checked yet
4 more in Everything
Build governed data and AI apps without code (Beta) â
Sep 23, 2026
· release notes
Databricks introduces Genie App Builder Beta for building governed data and AI apps via natural language prompts.
Introduces Genie App Builder in Beta, allowing users to build apps from natural-language prompts within a governance-enforced App Space. Not checked yet
1 detail in Everything
Scala UDFs can access Unity Catalog secrets â
Sep 23, 2026
· release notes
Scala UDFs can now access Unity Catalog secrets via a SECRETS clause.
Requires environment_version = 6 or above on serverless compute and Databricks Runtime 19+ with standard access mode on classic compute for secret access in Scala UDFs. Not checked yet Extends Unity Catalog Scala UDF secret access support to only serverless SQL warehouses; dedicated access mode does not support direct invocation. Not checked yet
2 details in Everything
└──▷ BREAKING ON UPGRADE
! Dedicated access mode compute no longer supports direct invocation of Unity Catalog Scala UDFs that access secrets.
Databricks adds automatic sensitive data classification and tagging in Unity Catalog.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Centrally configure coding agents (2026-09-22)
Sep 22, 2026
·
issue 084
Databricks adds central configuration for coding agents, covering models, MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. servers, skills, routing, and MDM MDM Mobile device management, centrally configuring and managing enrolled devices. Administrators use it to apply settings, distribute applications and enforce device security policies. rollout.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Connect coding agents to Unity Gateway with the ug CLI (2026-09-22)
Sep 22, 2026
·
issue 084
New ug CLI connects coding agents (Claude Code, Codex, Pi) to Unity Gateway with one command.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Unified tracing now includes coding agent activity (2026-09-22)
Sep 22, 2026
·
issue 084
Unity Gateway unified tracing now captures coding-agent tool and skill activity alongside model requests.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Review coding-agent tool and skill usage alongside model requests in the Govern traces UI.
📍 In the Databricks console, go to Govern › Traces to view unified traces that now include coding-agent activity.
Adds coding-agent activity (tools and skills used) to Unity Gateway unified tracing, viewable in Govern > Traces or via Genie.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Invite OpenSharing recipients by email (Beta) (2026-09-22)
Sep 22, 2026
·
issue 083
Databricks OpenSharing now supports inviting recipients by email with a secure activation link Beta .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Adds email-based invitation for OpenSharing recipients, sending a secure activation link instead of requiring manual exchange of a sharing identifier or credential file Beta .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Lakebase backup schedule API (Beta) (2026-09-22)
Sep 22, 2026
·
issue 083
Databricks adds a Beta API to programmatically manage Lakebase backup schedules with configurable cadence and retention.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Adds a Beta snapshot schedule API for Lakebase that lets you read and set a root branch's automated snapshot cadences (daily, weekly, or monthly) and per-cadence retention periods.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
OpenAI GPT-6 Luna is now available on Unity Gateway (2026-09-22)
Sep 22, 2026
·
issue 083
OpenAI GPT-6 Luna is now accessible via Databricks Unity Gateway through Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. pay-per-token.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Shopify connector (Beta) (2026-09-22)
Sep 22, 2026
·
issue 083
Databricks Lakeflow Connect adds a Shopify connector Beta for ingesting e-commerce data via OAuth.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Centrally configure coding agents â
Sep 22, 2026
· release notes
Databricks introduces centralized configuration for coding agents via MDM MDM Mobile device management, centrally configuring and managing enrolled devices. Administrators use it to apply settings, distribute applications and enforce device security policies.
Adds MDM rollout capability for coding agent settings including models, MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. servers, skills, default harness, and Smart Routing. Not checked yet
1 detail in Everything
OpenAI GPT-6 Luna is now available on Unity Gateway â
Sep 22, 2026
· release notes
OpenAI GPT-6 Luna is now available on Unity Gateway Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves.
Unified tracing now includes coding agent activity â
Sep 22, 2026
· release notes
Databricks adds coding agent activity to unified traces in Unity Gateway, visible via Govern > Traces or Genie.
Anthropic Claude Opus 5.5 is now available on Unity Gateway â
Sep 22, 2026
· release notes
Anthropic Claude Opus 5.5 added to Unity Gateway via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves.
Invite OpenSharing recipients by email (Beta) â
Sep 22, 2026
· release notes
Databricks adds beta email invitations for OpenSharing recipients
Adds OpenSharing by email to invite recipients via a secure activation link instead of exchanging identifiers or credential files. Not checked yet
1 detail in Everything
Connect coding agents to Unity Gateway with the ug CLI â
Sep 22, 2026
· release notes
New ug CLI for connecting coding agents to Unity Gateway
Introduces the ug CLI to connect Claude Code, Codex, Pi, and other coding agents to Unity Gateway with a single command. Not checked yet ug handles authentication and configuration for supported coding agents automatically. Not checked yet ug adds approved MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. servers and skills to the connected coding agent environment. Not checked yet ug displays spend information directly in the terminal during operation. Not checked yet
4 details in Everything
Databricks adds Beta Shopify connector in Lakeflow Connect for ingesting e-commerce data via OAuth.
Back model provider service credentials with a Unity Catalog secret (2026-09-21)
Sep 21, 2026
·
issue 083
Databricks lets you back model provider API keys with customer-owned Unity Catalog secrets, enabling zero-downtime key rotation.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Backs a model provider service credential with a customer-owned Unity Catalog secret so that rotating the secret takes effect at query time with no service change.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Databricks Runtime 19: September 21, 2026 (2026-09-21)
Sep 21, 2026
·
issue 082
Databricks Runtime 19 adds truncate, trim_array functions and session-scoped Python UDF worker env vars via spark.pythonWorkerEnv.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ USE IT
Inject a custom environment variable (e.g. a secret path) into every Python UDF worker in a session without restarting the cluster.
python
copy
spark.conf.set("spark.pythonWorkerEnv.MY_SECRET_PATH", "/dbfs/secrets/my_secret.txt")
Truncate a monetary amount to 2 decimal places without rounding, useful for audit-safe financial calculations.
copy
$ SELECT truncate(price, 2) AS truncated_price FROM transactions;
Drop the last N elements from an array column to trim trailing nulls or sentinel values before aggregation.
copy
$ SELECT trim_array(tags, 2) AS trimmed_tags FROM events;
Adds spark.pythonWorkerEnv.<variable-name> configuration prefix to set session-scoped environment variables for Python UDF scalar worker processes.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Adds truncate function to SQL, Scala DataFrame, and Python DataFrame APIs to truncate a number to a specified scale without rounding. Not checked yet Adds trim_array function to SQL, Scala DataFrame, and Python DataFrame APIs to remove elements from the end of an array (positive count) or the beginning (negative count). Not checked yet Enables auto-optimized shuffle v2 by default for workloads that do not set a fixed shuffle partition count. Not checked yet
Adds spark.pythonWorkerEnv.<variable-name> configuration prefix to set session-scoped environment variables for Python UDF scalar worker processes.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Adds truncate function to SQL, Scala DataFrame, and Python DataFrame APIs to truncate a number to a specified scale without rounding. Not checked yet Adds trim_array function to SQL, Scala DataFrame, and Python DataFrame APIs to remove elements from the end of an array (positive count) or the beginning (negative count). Not checked yet
1 more in Everything
The query history system table is now generally available (2026-09-21)
Sep 21, 2026
·
issue 082
The system.query.history system table is now generally available for account-wide query telemetry.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Back model provider service credentials with a Unity Catalog secret â
Sep 21, 2026
· release notes
Databricks now supports backing model provider service credentials with Unity Catalog secrets for automatic rotation at query time.
Adds the ability to back a model provider service's API key with a customer-owned Unity Catalog secret. Not checked yet
1 detail in Everything
The query history system table is now generally available â
Sep 21, 2026
· release notes
The system.query.history system table is now generally available in Databricks
Adds the system.query.history system table as a GA feature to access account-wide query execution status, duration metrics, and query source for SQL warehouses and serverless compute. Not checked yet
1 detail in Everything
Configure provider discounts for external model spend (2026-09-19)
Sep 19, 2026
·
issue 083
Unity Gateway lets you configure a price multiplier on model provider services to reflect negotiated rates for external model spend.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Databricks: Automatically classify and tag sensitive data in Unity Catalog with new Data Classification capability.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Databricks Apps telemetry is now generally available (2026-09-18)
Sep 18, 2026
·
issue 079
Databricks Apps telemetry now GA: traces, logs, and metrics auto-persisted to Unity Catalog via OpenTelemetry OpenTelemetry An open standard and set of tools for collecting traces, metrics and logs from applications. It lets services export observability data without tying instrumentation to one monitoring vendor. .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Supports custom instrumentation via the OpenTelemetry SDK alongside automatic system logs and usage event capture once app telemetry is enabled.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Lakebase Search is generally available (2026-09-18)
Sep 18, 2026
·
issue 079
Lakebase Search GA adds lakebase_vector and lakebase_text Postgres extensions for scalable vector and BM25 BM25 The classic keyword-ranking formula behind traditional search engines. Hybrid search combines it with embedding similarity so exact terms and meaning both count. full-text search.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ TRY IT
Enable Lakebase Search in your project, then install both extensions to run hybrid semantic + keyword search against a single Postgres database.
copy
$ -- After enabling Lakebase Search in project settings:
CREATE EXTENSION lakebase_vector;
CREATE EXTENSION lakebase_text;
-- Create a vector index (lakebase_ann) on an embeddings column
CREATE INDEX ON documents USING lakebase_ann (embedding);
-- Create a BM25 full-text index (lakebase_bm25) on a text column
CREATE INDEX ON documents USING lakebase_bm25 (body_tsvector);
Enable Lakebase Search through the project settings UI before installing the extensions.
📍 In your Lakebase project, go to Project Settings › Lakebase Search and enable the feature, then install lakebase_vector and/or lakebase_text via SQL.
Adds the lakebase_vector Postgres extension with the lakebase_ann index type for approximate nearest-neighbor vector search, drop-in compatible with pgvector, scaling to over 1 billion vectors on a single index with index builds up to 50-100x faster than HNSW HNSW Hierarchical Navigable Small World, a graph index that finds approximate nearest neighbours among millions of vectors in milliseconds. Most vector databases use it by default. .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Adds the lakebase_text Postgres extension with the lakebase_bm25 index type for BM25 full-text search using standard tsvector types and operators, adding BM25 ranking and top-K pushdown not available in PostgreSQL's native GIN index. Not checked yet Supports hybrid search hybrid search Combining keyword search with semantic or vector search. The results use both exact term matches and similarity in meaning. by combining lakebase_vector and lakebase_text to return semantic and exact-term matches in a single ranked list within one Lakebase database. Not checked yet Delivers storage-backed indexes that are available immediately after scale-to-zero with no warmup required, enabling search in serverless Lakebase projects. Not checked yet
4 details in Everything
Databricks Apps is now on by default for workspaces with the compliance security profile enabled â
Sep 18, 2026
· release notes
Databricks Apps enabled by default for workspaces with compliance security profile
Base environments are now supported on classic compute (Beta) (2026-09-17)
Sep 17, 2026
·
issue 079
Databricks base environments for Python dependency management now supported on classic compute Beta .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Supports Databricks-provided and custom workspace-level base environments for managing Python dependencies on classic compute clusters.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Unity Gateway support for compliance security profile standards (2026-09-17)
Sep 17, 2026
·
issue 079
Unity Gateway now available by default for workspaces with the compliance security profile enabled, covering all Databricks compliance standards.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Dashboard enhancements and bug fixes (2026-09-17)
Sep 17, 2026
·
issue 078
Databricks AI/BI Dashboards gains metric references in widget titles, pivot-table styling, per-facet axis scaling, and Genie Code SQL parameter support.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Surface a live KPI directly in a widget title so viewers always see the current filtered total without reading the chart itself.
📍 In a visualization widget's title or description field, type @ to open the field reference picker, select the desired field and aggregation function (e.g. Sum), then confirm to insert the live metric reference.
Prevent misleading axis comparisons on a faceted chart where subcharts have very different value ranges.
📍 Open the faceted chart's axis settings, locate Axis range , and set it to Per facet so each subchart scales to its own data.
Adds the ability to convert a SQL dataset into a local metric view directly from the dataset's context menu, with Genie Code handling the conversion.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Genie Code for dashboard authoring now understands SQL dataset parameters when building and editing dashboards. Not checked yet Enables inserting live, formatted field references (sum, average, etc.) into visualization widget titles and descriptions by typing @, with support for changing the field, aggregation function, and number or date formatting. Not checked yet Adds a configurable default expand level for pivot table row and column hierarchy when the table first loads. Not checked yet Adds Per facet axis range scaling on faceted charts, allowing each subchart to scale independently instead of sharing one range. Not checked yet
+3 more
show less
Adds color palette previews to the preset theme picker before a theme is selected. Not checked yet Enables reordering of options in a field variable. Not checked yet Adds independent styling of pivot table totals headers and values, including conditional styling support. Not checked yet
Adds the ability to convert a SQL dataset into a local metric view directly from the dataset's context menu, with Genie Code handling the conversion.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Enables inserting live, formatted field references (sum, average, etc.) into visualization widget titles and descriptions by typing @, with support for changing the field, aggregation function, and number or date formatting. Not checked yet Adds Per facet axis range scaling on faceted charts, allowing each subchart to scale independently instead of sharing one range. Not checked yet
5 more in Everything
Genie Agents enhancements and bug fixes (2026-09-17)
Sep 17, 2026
·
issue 078
Genie Agents gains 100-page PDF uploads, six-month activity summaries, and source-scoped data access.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Raises the PDF upload limit for Genie One and Genie Agents from 20 pages to 100 pages.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Extends the Monitor tab so authors can summarize up to six months of Genie Agent activity, up from an unspecified shorter window. Not checked yet Restricts a Genie Agent to only the data assets explicitly attached as sources, preventing it from drawing on unattached data objects. Not checked yet
Raises the PDF upload limit for Genie One and Genie Agents from 20 pages to 100 pages.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
2 more in Everything
└──▷ BREAKING ON UPGRADE
! A Genie Agent now uses only data assets explicitly attached as sources; any previously implicit or ambient data access will no longer work.
Metastore-level ABAC policies are in Beta (2026-09-17)
Sep 17, 2026
·
issue 078
Unity Catalog gains metastore-level ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. policies in Beta, letting one row filter, column mask, GRANT, or DENY rule span every catalog.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Skills in Genie One support multiple files and folders (2026-09-17)
Sep 17, 2026
·
issue 078
Genie One skills now support multiple files and sub-folders, up to 3 levels deep and 50 files per skill.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Unity Gateway support for compliance security profile standards â
Sep 17, 2026
· release notes
Unity Gateway now supports all Databricks compliance security profile standards by default.
Metastore-level ABAC policies are in Beta â
Sep 17, 2026
· release notes
Metastore-level ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. policies are now in Beta within Unity Catalog, allowing governance teams to enforce data protection rules organization-wide.
Adds ability to attach ABAC policies (row filter, column mask, GRANT, or DENY) at the metastore level Not checked yet
1 detail in Everything
VARIANT cast extends to ARRAY and MAP columns in ABAC column masks (2026-09-16)
Sep 16, 2026
·
issue 082
VARIANT column masks in ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. now cover ARRAY and MAP types alongside STRUCT, requiring Databricks Runtime 19+.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Account-only user access to Genie One (Beta) (2026-09-16)
Sep 16, 2026
·
issue 079
Genie One Beta now lets account-only users access account-level AI-BI without workspace assignment or consumer access grants.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Unity Gateway API and developer tools are generally available (2026-09-16)
Sep 16, 2026
·
issue 077
Unity Gateway API for model, model-provider, and MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. services reaches GA with full CRUD CRUD The four basic database operations — Create, Read, Update, Delete — used as shorthand for any system that manages persistent data records, giving cyber tools a standard model for interacting with stored information. across Terraform Terraform An open-source infrastructure-as-code tool by HashiCorp that provisions cloud and on-premises resources through declarative configuration files, letting cyber tools automate and version-control their deployment environments. , CLI, and six SDKs.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Unity Gateway API is now generally available with full create, read, update, list, and delete support for managing model services, model provider services, and MCP services.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Supports Unity Gateway management via Terraform provider 1.132.0 and later. Not checked yet Supports Unity Gateway management via Databricks CLI v1.17.0 and later, including Declarative Automation Bundles support Beta . Not checked yet Supports Unity Gateway management via Python SDK databricks-sdk 0.136.0 and later. Not checked yet Supports Unity Gateway management via Go SDK databricks-sdk-go v0.178.0 and later. Not checked yet
+2 more
show less
Supports Unity Gateway management via Java SDK databricks-sdk-java 0.153.0 and later. Not checked yet Supports Unity Gateway management via JavaScript SDK @databricks/sdk-aigateway 0.19.0 and later. Not checked yet
7 details in Everything
Managed agent memory and sessions are in Beta â
Sep 16, 2026
· release notes
Databricks adds Beta support for managed agent memory and sessions with semantic search semantic search Searching by meaning rather than requiring the same words. It commonly compares vector representations to find related content, even when the wording differs. .
Adds Managed agent memory as a fully managed, Lakebase-backed store providing durable long-term memory with semantic search for agents built on any framework Not checked yet Adds Managed agent sessions as a fully managed, Lakebase-backed store to persist conversation history across turns for agents built on any framework Not checked yet
2 details in Everything
Account-only user access to Genie One (Beta) â
Sep 16, 2026
· release notes
Genie One now supports account-only users accessing Genie One without workspace or consumer access assignment.
Enables account-only users in a registered identity provider identity provider A service that authenticates users and supplies identity information to other applications. It is the sign-in authority behind many single sign-on systems. to access Genie One without being assigned to a workspace or granted consumer access Not checked yet Automatically enrolls eligible account-only users when the feature is enabled Not checked yet
2 details in Everything
Unity Gateway API and developer tools are generally available â
Sep 16, 2026
· release notes
Unity Gateway API GA adds CRUD CRUD The four basic database operations — Create, Read, Update, Delete — used as shorthand for any system that manages persistent data records, giving cyber tools a standard model for interacting with stored information. for model services, providers, and MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. servers across SDKs, CLI, and Terraform.
Adds unity-gateway resource support in Databricks Terraform provider v1.132.0+ for managing model services Not checked yet Adds databricks sdk ai gateway subcommands in Databricks CLI v1.17.0+ for creating and managing model services Not checked yet Adds databricks_sdk.ai.gateway module in Python SDK 0.136.0+ for programmatic management of model services Not checked yet Adds sdk.NewAIGatewayClient method in Go SDK v0.178.0+ for Unity Gateway API access Not checked yet Adds DatabricksAIGatewayClient class in Java SDK 0.153.0+ for managing model providers and MCP servers Not checked yet
5 details in Everything
Databricks adds ai_transcribe function to transcribe audio files to text with timestamps and speaker labeling.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Automatic change data feed will soon be available in all supported regions (2026-09-15)
Sep 15, 2026
·
issue 077
Databricks Automatic CDF computes row-level changes at query time without per-table enablement, rolling out through October 2026.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Introduces Automatic Change Data Feed (Auto CDF), which computes row-level changes at query time using row tracking — eliminating the need to enable change data feed change data feed A stream of records describing every insert, update, and delete that occurs in a database, captured in real time so downstream systems can react to changes without polling. on each individual Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. or Apache Iceberg Apache Iceberg An open table format for large analytic datasets, maintained by the Apache Software Foundation, that gives cyber tools reliable schema evolution, time-travel queries, and atomic commits on object storage or HDFS. v3 table.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Requires Databricks Runtime 19 or above with row tracking enabled on the table. Not checked yet
2 details in Everything
ABAC on views is in Beta (2026-09-15)
Sep 15, 2026
·
issue 076
Databricks ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. row filters and column masks now extend to views in Unity Catalog (Beta, DBR 19+).︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Enable the ABAC on Views beta so that existing ABAC policies automatically apply to views, not just tables.
📍 In the account console, go to Previews and enable ABAC on Views .
Extends ABAC row filters and column masks to views (in addition to tables), enabling Unity Catalog policies to control access to sensitive data exposed through views.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Enables the ABAC on Views preview via the account console's Previews page. Not checked yet
2 details in Everything
Data Classification now scans Unity Catalog views (Beta) (2026-09-15)
Sep 15, 2026
·
issue 076
Databricks Data Classification now scans Unity Catalog views Beta in addition to tables.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Sharing foreign Delta tables with OpenSharing is now generally available (2026-09-15)
Sep 15, 2026
·
issue 076
Databricks OpenSharing now GA for sharing foreign Delta Delta The difference between two versions of a file or data stream; tools use deltas to transmit or store only what changed, reducing bandwidth and storage overhead. tables from OneLake OneLake Microsoft's unified data lake built into Microsoft Fabric that gives every workspace a single, tenant-wide storage layer, letting cyber tools read and write data without managing separate storage accounts. , Hive metastore Hive metastore A central metadata repository, part of the Apache Hive project, that stores schema and location information for distributed data sets so query engines can discover and read structured data without hardcoding paths. , and AWS Glue without data copying.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
The READ METADATA privilege will soon be available by default for workspaces with the compliance security profile enabled (2026-09-15)
Sep 15, 2026
·
issue 076
READ METADATA privilege becomes available by default for compliance security profile workspaces, exposing grants, row filters, column masks, and ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. policies.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Lakebase snapshots API (Beta) (2026-09-15)
Sep 15, 2026
·
issue 075
Databricks adds a Beta snapshots API for Lakebase, enabling programmatic create, get, list, delete, and branch-restore of point-in-time snapshots.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Sharing foreign Delta tables with OpenSharing is now generally available â
Sep 15, 2026
· release notes
OpenSharing GA allows sharing foreign Delta tables from OneLake OneLake Microsoft's unified data lake built into Microsoft Fabric that gives every workspace a single, tenant-wide storage layer, letting cyber tools read and write data without managing separate storage accounts. , Hive, or AWS Glue without copying data.
Data Classification now scans Unity Catalog views (Beta) â
Sep 15, 2026
· release notes
Data Classification Beta scans Unity Catalog views for sensitive columns
Window measures on a numeric index column in metric views (2026-09-14)
Sep 14, 2026
·
issue 078
Databricks metric views gain window measures on numeric index columns for non-calendar periods like fiscal weeks and 4-4-5.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
JAR tasks on serverless compute will soon be available for workspaces with the compliance security profile enabled (2026-09-14)
Sep 14, 2026
·
issue 077
JAR tasks on serverless compute coming to compliance security profile workspaces in early October 2026︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
The counter_diff SQL window function is now available (2026-09-14)
Sep 14, 2026
·
issue 077
Databricks adds counter_diff SQL window function for time-series counter analysis with reset handling.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Genie visualizations in Slack replies (Public Preview) (2026-09-14)
Sep 14, 2026
·
issue 075
Databricks Genie now renders chart visualizations as images directly in Slack replies Public preview .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Genie visualizations in Slack replies (Public Preview) â
Sep 14, 2026
· release notes
Databricks Genie visualizations now appear as images in Slack replies
Genie chart visualizations are rendered as images in Slack reply messages Not checked yet
1 detail in Everything
Session restore for serverless jobs is in Public Preview (2026-09-11)
Sep 11, 2026
·
issue 075
Databricks adds session restore for serverless jobs, letting you load a failed run's Python variables and Spark session into a notebook for live debugging.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
DeepSeek V4.1 Flash is now available on Unity Gateway (2026-09-11)
Sep 11, 2026
·
issue 072
DeepSeek V4.1 Flash now available on Databricks Unity Gateway with text and image input support.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Sharing foreign Iceberg tables with OpenSharing is now generally available (2026-09-11)
Sep 11, 2026
·
issue 072
Databricks OpenSharing now GA for foreign Iceberg Iceberg An open table format for large analytic datasets, maintained by the Apache Software Foundation, that gives cyber tools reliable schema evolution, time-travel queries, and atomic commits on object storage. tables federated via Lakehouse Federation Lakehouse Federation A data architecture pattern that lets a single query engine read from multiple external data sources—databases, data lakes, warehouses—without moving or copying the data, reducing pipeline complexity for tools that need broad data access. , readable by external Iceberg clients.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Sharing foreign schemas and tables with OpenSharing is now generally available (2026-09-11)
Sep 11, 2026
·
issue 072
Databricks OpenSharing now supports sharing foreign schemas and tables federated via Lakehouse Federation Lakehouse Federation A data architecture pattern that lets a single query engine read from multiple external data sources—databases, data lakes, warehouses—without moving or copying the data, reducing pipeline complexity for tools that need broad data access. without copying data.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Enables recipients to read data in shared foreign tables and foreign schemas via OpenSharing.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Standard performance mode is now available for one-time runs in the Jobs API (2026-09-11)
Sep 11, 2026
·
issue 072
Jobs API gains performance_target: STANDARD for one-time runs, also supported in Airflow's DatabricksSubmitRunOperator.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ TRY IT
Submit a one-time Databricks run with standard performance mode to control cost vs. speed trade-offs at submission time.
copy
$ curl -X POST https://<databricks-instance>/api/2.1/jobs/runs/submit \
-H 'Authorization: Bearer <token>' \
-H 'Content-Type: application/json' \
-d '{"performance_target": "STANDARD", "run_name": "my-one-time-run", "tasks": [{"task_key": "main", "existing_cluster_id": "<cluster-id>", "notebook_task": {"notebook_path": "/path/to/notebook"}}]}'
Adds performance_target field set to STANDARD in the Jobs API for one-time runs, enabling standard performance mode alongside the existing options.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Supports performance_target: STANDARD in Apache Airflow Apache Airflow An open-source workflow orchestration platform maintained by the Apache Software Foundation that lets developers schedule, monitor, and chain tasks as directed acyclic graphs, making it useful for automating data pipelines. 's DatabricksSubmitRunOperator for one-time Databricks job submissions. Not checked yet
Adds performance_target field set to STANDARD in the Jobs API for one-time runs, enabling standard performance mode alongside the existing options.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 more in Everything
Session restore for serverless jobs is in Public Preview â
Sep 11, 2026
· release notes
New session restore feature for serverless jobs.
Sharing foreign Iceberg tables with OpenSharing is now generally available â
Sep 11, 2026
· release notes
Databricks now generally available for sharing and reading foreign Iceberg tables via OpenSharing with Lakehouse Federation Lakehouse Federation A data architecture pattern that lets a single query engine read from multiple external data sources—databases, data lakes, warehouses—without moving or copying the data, reducing pipeline complexity for tools that need broad data access. .
Standard performance mode is now available for one-time runs in the Jobs API â
Sep 11, 2026
· release notes
One-time Jobs API runs now support STANDARD performance_target for cost optimization
DatabricksSubmitRunOperator (Airflow) now supports the STANDARD performance mode. Not checked yet
1 detail in Everything
DeepSeek V4.1 Flash is now available on Unity Gateway â
Sep 11, 2026
· release notes
DeepSeek V4.1 Flash model now available on Unity Gateway via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. .
Managed tables in Lakeflow pipelines (Beta) (2026-09-10)
Sep 10, 2026
·
issue 076
Lakeflow pipelines now support managed tables and append flows with checkpoint migration in Beta.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Adds support for creating managed tables in Lakeflow pipelines and writing to them via one or more append flows using append_flow Beta .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Adds import_checkpoint on append_flow to migrate existing Structured Streaming workloads into a Lakeflow pipeline without reprocessing the source. Not checked yet Adds create_table and CREATE TABLE ... FLOW (pipelines) syntax for defining managed tables and flows within Lakeflow pipelines. Not checked yet
3 details in Everything
Databricks-provided MCP connectors for Genie One and Genie Code are generally available (2026-09-10)
Sep 10, 2026
·
issue 072
Databricks Genie gains MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. connectors for Google Drive, Gmail, Google Calendar, Microsoft 365, Atlassian, Slack, and GitHub — governed through Unity Catalog and Unity Gateway︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Adds Databricks-provided MCP connectors for Google Drive, Gmail, Google Calendar, Microsoft 365, Atlassian, Slack, and GitHub, enabling Genie One and Genie Code to search and act on external company tools while answering questions.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Authenticates each user individually via OAuth so tokens are never shared across users. Not checked yet
2 details in Everything
Dashboard enhancements (2026-09-10)
Sep 10, 2026
·
issue 071
Databricks dashboards gain relationship-scoped filters, theme import/export, path maps, LOD expressions LOD expressions A formula language used in Tableau for computing aggregations at a specified level of detail, independent of the view's granularity, giving cyber tools precise control over how metrics are scoped and grouped. , and richer Ask Genie context.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Adds Include -style level of detail (LOD) expressions in YAML to the metric view editor.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Adds Line shape control to line and area charts, connecting data points with linear, smooth, or stepped lines. Not checked yet Adds Apply to setting on the pivot table color scale, letting you paint the color gradient on cell text as well as the background. Not checked yet Supports exporting a dashboard theme to a JSON file and importing a theme JSON file into another dashboard for reuse and sharing. Not checked yet Supports more parameterized calculations when importing Tableau reports via the /importBI Genie Code command. Not checked yet
+7 more
show less
Adds relationship-scoped filters Public preview : on dashboards with relationships, each widget now responds only to filters whose dataset is related to it. Not checked yet Lets you choose which SQL warehouse runs materialization when exporting a dashboard dataset as a materialized view in Unity Catalog. Not checked yet Scopes Ask Genie suggested starter questions to the current dashboard page and widget on published dashboards. Not checked yet Generates Key Insights per dashboard page, with a New insights badge and live-generating indicator when new insights are available. Not checked yet Shows the selected chart as context in the Ask Genie input bar on published dashboards, enabling questions about a specific chart. Not checked yet Adds path map support for connecting ordered point sequences into a line from separate longitude/latitude columns or a single point geometry column, enabling GPS-track-style trajectories. Not checked yet Genie Code now prefers governed metric views over ad-hoc SQL datasets and reuses an existing metric view when building a dashboard. Not checked yet
Adds Include -style level of detail (LOD) expressions in YAML to the metric view editor.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Adds Line shape control to line and area charts, connecting data points with linear, smooth, or stepped lines. Not checked yet Supports exporting a dashboard theme to a JSON file and importing a theme JSON file into another dashboard for reuse and sharing. Not checked yet Adds relationship-scoped filters Public preview : on dashboards with relationships, each widget now responds only to filters whose dataset is related to it. Not checked yet Adds path map support for connecting ordered point sequences into a line from separate longitude/latitude columns or a single point geometry column, enabling GPS-track-style trajectories. Not checked yet
7 more in Everything
Databricks-managed MCP connectors for Genie One are generally available (2026-09-10)
Sep 10, 2026
·
issue 071
Genie One adds GA Databricks-managed MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. connectors for Google Drive, Gmail, Microsoft 365, Atlassian, and Slack via Unity Catalog.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Governs all MCP connector tool calls through Unity Catalog and Unity Gateway, enforcing built-in policies, approval requirements, and logging usage and audit activity.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Requires per-user individual authentication with OAuth tokens that are never shared across users. Not checked yet
2 details in Everything
Genie Agents enhancements (2026-09-10)
Sep 10, 2026
·
issue 071
Databricks Genie Agents now support up to 50 data sources and 200,000 stored conversations per agent.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Raises the per-agent data source limit to 50 tables, views, or metric views, up from 30.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Raises the per-agent conversation storage limit to 200,000 conversations, up from 10,000. Not checked yet
Raises the per-agent data source limit to 50 tables, views, or metric views, up from 30.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 more in Everything
Genie One enhancements (2026-09-10)
Sep 10, 2026
·
issue 071
Genie One gains code skills on by default, backend steering, PowerPoint uploads, and memory confirmation prompts.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Steer a long-running Genie One response mid-flight to refocus the answer without cancelling and restarting.
📍 While Genie One is generating a response, type a steering message (e.g. Focus only on Q3 data ) in the input box and send it — Genie One will incorporate the guidance into the answer still being generated.
Review and approve a memory suggestion so Genie One retains a key fact for future conversations.
📍 After Genie One surfaces a memory prompt (e.g. 'Remember: your fiscal year starts in February — save this?'), choose Approve, modify the text, or Reject before Genie One stores it.
Enables Genie Code skills in Genie One by default, with per-skill toggle controls on the customization page.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Adds backend steering to Genie One, letting you send steering messages while a response is in progress to guide the output without waiting for it to finish. Not checked yet Supports uploading .pptx (PowerPoint) files directly into a Genie One conversation. Not checked yet Adds memory confirmation prompts Beta to Genie One, where the assistant detects facts worth remembering, surfaces a suggestion, and requires explicit approval, modification, or rejection before saving. Not checked yet
4 details in Everything
Maintenance windows for continuous pipelines (2026-09-10)
Sep 10, 2026
·
issue 071
Lakeflow continuous pipelines gain maintenance windows to control when platform restarts occur.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Adds maintenance window scheduling for continuous Lakeflow pipelines, letting you specify a day of the week, start hour, and time zone so Databricks applies runtime upgrades and serverless infrastructure restarts only within that one-hour window — configurable via the Jobs UI, the Pipelines UI, or the Jobs REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
The Databricks Excel Add-in is now generally available (2026-09-10)
Sep 10, 2026
·
issue 071
Databricks Excel Add-in is now GA, letting users query, import, and write back Unity Catalog data directly from Excel.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Adds the Databricks Excel Add-in (GA) to connect Microsoft Excel to a Databricks workspace for browsing and importing Unity Catalog tables and views, running SQL queries, building pivot tables, using custom functions, and writing data back to Unity Catalog.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet Adds Live query mode when building pivot tables in the Excel Add-in, refreshing results in real time as fields are updated. Not checked yet
2 details in Everything
Managed tables in Lakeflow pipelines (Beta) â
Sep 10, 2026
· release notes
Managed tables support in Lakeflow pipelines allows writing to new targets via append flows and import_checkpoint.
Enables creating managed tables in Lakeflow pipelines with one or more append_flow steps. Not checked yet Adds import_checkpoint capability on an append_flow to migrate existing Structured Streaming workloads without reprocessing the source. Not checked yet
2 details in Everything
The Databricks Excel Add-in is now generally available â
Sep 10, 2026
· release notes
Databricks Excel Add-in GA: browse Unity Catalog tables, run SQL, and write back.
Generally available the Databricks Excel Add-in for connecting Microsoft Excel to a Databricks workspace Not checked yet Enable Live query in pivot tables to see results as fields are updated Not checked yet
2 details in Everything
Maintenance windows for continuous pipelines â
Sep 10, 2026
· release notes
New maintenance window capability for continuous Lakeflow pipelines via UI and REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. .
Allows setting a one-hour maintenance window (day, hour, time zone) for continuous pipelines to control restart timing for platform updates like runtime versions. Not checked yet Adds configuration options for maintenance windows accessible in the Jobs UI, Pipelines UI, and the Jobs REST API. Not checked yet
2 details in Everything
Configurable retention for system tables is in Beta (2026-09-09)
Sep 9, 2026
·
issue 075
Databricks lets account admins set system table retention from 30 to 3,650 days Beta .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Adds account-wide configurable data retention period (30–3,650 days) for supported system tables, settable by account admins.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Anaplan connector (Beta) (2026-09-09)
Sep 9, 2026
·
issue 071
Databricks Lakeflow Connect gains a new Anaplan connector Beta for ingesting audit trail events and user account records.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Workspace admins can enable or disable the Anaplan connector through the Previews page.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄ Not checked yet
1 detail in Everything
Anysphere Organization connector (Beta) (2026-09-09)
Sep 9, 2026
·
issue 071
Lakeflow Connect adds Anysphere (Cursor) Organization connector in Beta to ingest members and groups as daily snapshots.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
└──▷ HOW TO FIND IT
Enable the Anysphere Organization connector for your workspace so it can begin ingesting Cursor org members and groups daily.
📍 In the Databricks workspace, go to Settings › Previews, locate the Anysphere Organization connector, and toggle it on.
Celigo connector (Beta) (2026-09-09)
Sep 9, 2026
·
issue 071
Lakeflow Connect adds a Beta Celigo connector to ingest integrator.io audit log events into Databricks.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
External secrets in Unity Catalog (Beta) (2026-09-09)
Sep 9, 2026
·
issue 071
Unity Catalog now surfaces AWS Secrets Manager secrets as read-only, governable objects via External Secrets Beta .︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Horizontal scaling for Databricks Apps will soon be available for workspaces with the compliance security profile enabled (2026-09-09)
Sep 9, 2026
·
issue 070
Horizontal scaling for Databricks Apps comes to compliance security profile workspaces, enabling multi-instance apps behind a single URL.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Genie Code adds conversation controls â
Sep 9, 2026
· release notes
Genie Code adds /btw for side questions and chat branching/editing capabilities
Adds branch and edit capabilities to Genie Code conversations Not checked yet Adds /btw command for quick side questions without thread impact Not checked yet
2 details in Everything
Anysphere Organization connector (Beta) â
Sep 9, 2026
· release notes
Lakeflow Connect introduces a new Beta connector for Anysphere (Cursor) Orgs, ingesting members and groups via their Admin API Admin API A dedicated HTTP interface that exposes management and configuration operations separately from the data plane, letting cyber tools automate provisioning, policy changes, or monitoring without touching the main user-facing service. key as daily snapshots.
Databricks adds Beta Celigo connector for Lakeflow Connect to ingest audit log events.
Enables access control via the Previews page for workspace admins Not checked yet
1 detail in Everything
Configurable retention for system tables is in Beta â
Sep 9, 2026
· release notes
Beta: Account admins can set configurable retention (30-3650 days) for supported system tables.
New beta Anaplan connector in Lakeflow Connect ingests audit and user records.
Introduces authentication via an Anaplan CA certificate paired with its private key for the new connector Not checked yet
1 detail in Everything
ABAC DENY policies are in Beta (2026-09-08)
Sep 8, 2026
·
issue 069
Unity Catalog gains ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. DENY policies in Beta, letting you block MANAGE ACCESS CONTROL even for object owners.
Incremental ingestion for Salesforce formula fields in Lakeflow Connect will soon be generally available (2026-09-08)
Sep 8, 2026
·
issue 069
Lakeflow Connect shifts Salesforce formula field ingestion from full snapshots to incremental by default, cutting cost and pipeline run time.
└──▷ BREAKING ON UPGRADE
! When incremental Salesforce formula field ingestion reaches GA, full snapshots will no longer be the default for Lakeflow Connect pipeline runs — existing pipelines that relied on the full-snapshot default will change behavior on upgrade.
Lakebase now supports PCI-DSS and HITRUST in all AWS regions (2026-09-08)
Sep 8, 2026
·
issue 069
Lakebase PCI-DSS PCI-DSS The Payment Card Industry Data Security Standard, security requirements for protecting payment account data. It covers organizations that store, process or transmit that data and relevant service providers. and HITRUST HITRUST A US-based certification framework for healthcare data security, maintained by the HITRUST Alliance, that maps controls from HIPAA, NIST, and ISO into a single assessable standard tools can target to prove compliance. compliance now available in all AWS regions (previously us-east-1 only)
time_bucket SQL function aligns timestamps to fixed-width time buckets (2026-09-08)
Sep 8, 2026
·
issue 069
Databricks adds time_bucket SQL function for aligning timestamps to arbitrary fixed-width intervals.
Adds time_bucket SQL function that returns the start of a fixed-width interval (e.g. 15-minute or 3-month buckets) aligned to a caller-specified origin, for cases where date_trunc calendar units are too coarse or misaligned. Not checked yet
1 detail in Everything
time_bucket SQL function aligns timestamps to fixed-width time buckets â
Sep 8, 2026
· release notes
Databricks adds time_bucket SQL function to align timestamps to fixed-width intervals.
Adds time_bucket SQL function that returns the start of a fixed-width interval for a timestamp. Not checked yet
1 detail in Everything
See which chunks support each ai_search answer (Beta) â
Sep 8, 2026
· release notes
Adds generate_citations option to ai_search function to show supporting chunks
Databricks introduces ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. DENY policies in Unity Catalog to explicitly block MANAGE ACCESS CONTROL privileges.
Zerobus Ingest into default storage is now in Public Preview (2026-09-07)
Sep 7, 2026
·
issue 069
Databricks Lakeflow Connect: Zerobus Ingest into default storage tables now in Public Preview.
Zerobus Ingest into default storage is now in Public Preview â
Sep 7, 2026
· release notes
Zerobus Ingest into default storage tables enters Public Preview
OpenAI GPT-6 Astra is now available on Unity Gateway (2026-09-04)
Sep 4, 2026
·
issue 067
OpenAI GPT-6 Astra now available on Databricks Unity Gateway via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. pay-per-token.
Data Classification will soon scan Unity Catalog views (2026-09-03)
Sep 3, 2026
·
issue 070
Data Classification extends sensitive-data scanning to Unity Catalog views Beta , using the same detection engine as tables.︀︄︇︀︅︉︀︅︀︀︃︀︀︅︅︀︅︊︀︄︆︀︃︄
Dashboard enhancements and bug fixes (2026-09-03)
Sep 3, 2026
·
issue 067
Databricks dashboards gain embedded filter APIs, cross-dataset filtering, Slack/Teams CSV attachments, and color-by-name mappings.
└──▷ HOW TO FIND IT
Deliver a scheduled dashboard snapshot to a Teams channel with a CSV data attachment so recipients can slice the raw numbers without opening Databricks.
📍 In the dashboard, open Subscriptions › Add subscription, set the destination to your Microsoft Teams channel, and under Data attachments select CSV. Save and schedule the subscription.
Enable cross-dataset filtering so that a region filter on a sales dataset automatically narrows widgets built on the related returns dataset.
📍 In the dashboard editor, open Data › Relationships, click Add relationship , select the two datasets and the shared key field, then save. Filters applied to either dataset will now propagate to the other.
Adds embedded dashboard filter APIs to set and retrieve filter values programmatically in embedded dashboards. Not checked yet Adds support for ranked table calculations when importing reports via the /importBI Genie Code command from Power BI and Tableau. Not checked yet Adds Data attachments to dashboard subscriptions delivered to Slack and Microsoft Teams, supporting CSV, TSV TSV A plain-text format storing tabular data with fields separated by tab characters, making it easy for cyber tools to parse and exchange structured records without a CSV parser's quoting complexity. , or Excel format. Not checked yet Adds cross-dataset filtering through relationships Public preview : filters, selections, and parameters applied to one dataset automatically carry across to widgets built on related datasets. Not checked yet Adds color mappings in a dashboard's theme settings, letting you assign a specific color to a named value used consistently across all visualizations (color by field and color by series). Not checked yet
+6 more
show less
Adds rich text formatting support for dynamic data references in text widgets, widget titles, and descriptions. Not checked yet Adds multiple-field support to dashboard variables, letting viewers add, remove, and swap multiple fields on a visualization at once instead of one at a time. Not checked yet Dashboard variable options now show their source dataset as a subtitle. Not checked yet Downloads from a metric view now use the display names defined for its fields. Not checked yet Scrollbars now remain always visible on scrollable faceted charts. Not checked yet Genie Code now automatically places new widgets in available layout space when adding them to a dashboard. Not checked yet
Adds embedded dashboard filter APIs to set and retrieve filter values programmatically in embedded dashboards. Not checked yet Adds Data attachments to dashboard subscriptions delivered to Slack and Microsoft Teams, supporting CSV, TSV TSV A plain-text format storing tabular data with fields separated by tab characters, making it easy for cyber tools to parse and exchange structured records without a CSV parser's quoting complexity. , or Excel format. Not checked yet Adds cross-dataset filtering through relationships Public preview : filters, selections, and parameters applied to one dataset automatically carry across to widgets built on related datasets. Not checked yet Adds color mappings in a dashboard's theme settings, letting you assign a specific color to a named value used consistently across all visualizations (color by field and color by series). Not checked yet
7 more in Everything
Environment version 6 is now available (2026-09-03)
Sep 3, 2026
·
issue 067
Databricks adds environment version 6 as a selectable base environment for serverless and standard classic compute.
Genie Agents enhancements (2026-09-03)
Sep 3, 2026
·
issue 067
Genie Agents now surfaces SQL function descriptions when a function is attached as a tool.
Displays the function description when a SQL function is attached as a Genie Agents tool, giving users immediate context on what each function does. Not checked yet Upgrades the large language models large language models A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. used in Genie Agents chat mode. Not checked yet
2 more in Everything
Genie One enhancements (2026-09-03)
Sep 3, 2026
·
issue 067
Genie mobile app gains push notifications for scheduled tasks on iOS; embedded dashboards hide unsupported Ask Genie button.
Enables push notifications for scheduled tasks to iOS devices in the Genie mobile app Public preview . Not checked yet Hides the Ask Genie button in embedded dashboards when the feature is not supported. Not checked yet
Enables push notifications for scheduled tasks to iOS devices in the Genie mobile app Public preview . Not checked yet
1 more in Everything
Share metric views with OpenSharing (Beta) (2026-09-03)
Sep 3, 2026
·
issue 067
Databricks adds OpenSharing support for metric views, enabling cross-metastore and cross-account sharing Beta .
Enables sharing metric views with Databricks users in other metastores or accounts using OpenSharing Beta . Not checked yet
1 detail in Everything
Web search in Genie One (Beta) (2026-09-03)
Sep 3, 2026
·
issue 067
Genie One Beta gains public web search to answer questions requiring current information, with cited source links.
Environment version 6 is now available â
Sep 3, 2026
· release notes
Databricks Environment version 6 is now available as a base environment option for serverless and standard classic compute.
Share metric views with OpenSharing (Beta) â
Sep 3, 2026
· release notes
Adds ability to share metric views with users in other metastores or accounts via OpenSharing Beta .
AI Search supports struct and map columns and filter-only queries (2026-09-02)
Sep 2, 2026
·
issue 067
Databricks AI Search gains struct/map column indexing and filter-only queries without vector or keyword retrieval.
Enables indexing and filtering of struct and map columns in AI Search indexes. Not checked yet Supports filter-only queries in AI Search that return matching rows without requiring vector or keyword retrieval. Not checked yet
Supports filter-only queries in AI Search that return matching rows without requiring vector or keyword retrieval. Not checked yet
1 more in Everything
Git Folder Serverless is in Beta (2026-09-02)
Sep 2, 2026
·
issue 067
Databricks Git Folder Serverless Beta shares serverless compute and a pyproject.toml-managed environment across notebooks in a Git folder.
Google Gemini 3.8 Flash now available as a Databricks-hosted model (2026-09-02)
Sep 2, 2026
·
issue 067
Databricks Model Serving adds Google Gemini 3.8 Flash as a Databricks-hosted model via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. .
Google Gemini 3.8 Flash now available as a Databricks-hosted model â
Sep 2, 2026
· release notes
Google Gemini 3.8 Flash available as a Databricks-hosted model via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves.
Adds support for the Google Gemini 3.8 Flash model in Model Serving Not checked yet
1 detail in Everything
Databricks adds Git Folder Serverless in beta for sharing serverless compute and pyproject.toml environments across notebooks.
AI Search supports struct and map columns and filter-only queries â
Sep 2, 2026
· release notes
AI Search now supports indexing/filtering struct and map columns and filter-only queries
Adds support for indexing and filtering struct and map columns in AI Search indexes Not checked yet Enables filter-only queries that return rows matching a filter without performing vector or keyword retrieval Not checked yet
2 details in Everything
Automatic change data feed is now generally available (2026-09-01)
Sep 1, 2026
·
issue 068
Auto CDF reaches GA, delivering row-level change tracking at query time with ~15% faster MERGE/UPDATE and no per-table setup.
Adds Automatic change data feed change data feed A stream of records describing every insert, update, and delete that occurs in a database, captured in real time so downstream systems can react to changes without polling. (Auto CDF), which computes row-level changes at query time using row tracking — no manual ENABLE CHANGE DATA FEED required on each Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. or Apache Iceberg Apache Iceberg An open table format for large analytic datasets, maintained by the Apache Software Foundation, that gives cyber tools reliable schema evolution, time-travel queries, and atomic commits on object storage or HDFS. v3 table. Not checked yet Auto CDF makes MERGE and UPDATE operations approximately 15% faster on tables queried for changes by eliminating write-time overhead. Not checked yet Auto CDF works with batch queries, Structured Streaming, and Delta Lake Sharing when using Databricks Runtime 19 LTS or above with row tracking enabled. Not checked yet
Adds Automatic change data feed change data feed A stream of records describing every insert, update, and delete that occurs in a database, captured in real time so downstream systems can react to changes without polling. (Auto CDF), which computes row-level changes at query time using row tracking — no manual ENABLE CHANGE DATA FEED required on each Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. or Apache Iceberg Apache Iceberg An open table format for large analytic datasets, maintained by the Apache Software Foundation, that gives cyber tools reliable schema evolution, time-travel queries, and atomic commits on object storage or HDFS. v3 table. Not checked yet Auto CDF works with batch queries, Structured Streaming, and Delta Lake Sharing when using Databricks Runtime 19 LTS or above with row tracking enabled. Not checked yet
1 more in Everything
Anthropic Claude Fable 5.1 now available as a Databricks-hosted model (2026-09-01)
Sep 1, 2026
·
issue 067
Databricks Unity Gateway adds Anthropic Claude Fable 5.1 as a hosted model accessible via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. .
Apache Arrow support for Zerobus Ingest is now generally available (2026-09-01)
Sep 1, 2026
·
issue 067
Apache Arrow Flight support for Zerobus Ingest in Lakeflow Connect is now generally available.
Genie Code scheduled tasks are now generally available (2026-09-01)
Sep 1, 2026
·
issue 067
Genie Code scheduled tasks are now GA, enabling recurring AI-driven analysis to run unattended as background jobs.
└──▷ HOW TO FIND IT
Set up a recurring analysis task when you want Genie Code to automatically run a prompt (e.g. a weekly data quality report) without manual intervention.
📍 In the Genie Code chat, type your prompt (e.g. 'Summarize data quality issues in the sales table') and then go to Schedules › Create schedule to set the recurrence. Each run produces a new continuable Genie chat with the results.
Enables scheduling Genie Code to run a prompt on a recurring schedule, producing a continuable Genie chat with results for each run — accessible via chat prompt or the Schedules menu. Not checked yet
1 detail in Everything
Genie Code scheduled tasks will soon be available by default for workspaces with the compliance security profile enabled (2026-09-01)
Sep 1, 2026
·
issue 067
Databricks Genie Code gains scheduled task support for compliance security profile workspaces in October 2026.
Enables Genie Code scheduled tasks for workspaces with the compliance security profile, allowing prompts to run on a recurring schedule as unattended background tasks that produce a continuable Genie chat with results. Not checked yet
1 more in Everything
Genie One and Genie Agents can now use models served through OpenAI on Databricks (2026-09-01)
Sep 1, 2026
·
issue 067
Genie One and Genie Agents now support models served through OpenAI on Databricks when partner-powered AI is enabled.
Genie One web search is in Beta (2026-09-01)
Sep 1, 2026
·
issue 067
Genie One gains public web search Beta to answer questions using current external sources with citations.
Enables Genie One to search the public web for current information — such as recent release notes, third-party documentation, or news — and cite external sources as links in its responses. Not checked yet
1 detail in Everything
Anthropic Claude Fable 5.1 now available as a Databricks-hosted model â
Sep 1, 2026
· release notes
Anthropic Claude Fable 5.1 is now available as a Databricks-hosted model via Unity Gateway.
Apache Arrow support for Zerobus Ingest is now generally available â
Sep 1, 2026
· release notes
Apache Arrow support for Zerobus Ingest in Lakeflow Connect is now GA
Genie Code scheduled tasks are now generally available â
Sep 1, 2026
· release notes
Genie Code scheduled tasks now GA, enabling recurring analysis and unattended background work via chat or the Schedules menu.
Genie Code scheduled tasks are now generally available, allowing users to schedule prompts to run on a recurring basis and produce continuable Genie chats as results. Not checked yet
1 detail in Everything
Genie One enters Beta with public web search and source citation capabilities.
Enables Genie One to retrieve current information from the public web for answers involving recent release notes, documentation, or news Not checked yet Adds automatic citation of external sources as links in Genie One's responses Not checked yet
2 details in Everything
Automatic change data feed is now generally available â
Sep 1, 2026
· release notes
Databricks GA release introduces Automatic Change Data Feed (Auto CDF) for Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. and Iceberg tables with performance improvements.
Adds automatic row-level change tracking at query time without manual table configuration Not checked yet Improves MERGE and UPDATE operation speed by approximately 15% on tables queried for changes Not checked yet
2 details in Everything
ABAC GRANT policies are generally available for models and services тАЛ
Aug 31, 2026
· release notes
ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. GRANT policies are now GA for models and services (MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. support included)
Adds ABAC GRANT policies to dynamically grant Unity Catalog privileges on models, model services, and MCP services based on governing tags Not checked yet Expands ABAC policy scope to include model context protocol (MCP) services Not checked yet
2 details in Everything
Restrict service principal OAuth secrets to specific API scopes тАЛ
Aug 29, 2026
· release notes
Restrict service principal OAuth secrets to specific API scopes
Adds ai_transcribe() SQL function Beta to transcribe audio into timestamped, speaker-labeled text segments.
Introduces ai_transcribe() SQL function Beta for audio transcription Not checked yet ai_transcribe() returns a VARIANT output with time-stamped segments and optional speaker labels Not checked yet
2 details in Everything
Zhipu AI GLM 5.3 now available as a Databricks-hosted model тАЛ
Aug 29, 2026
· release notes
Databricks adds Zhipu AI GLM 5.3 as a hosted model via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. .
Monitor and cap external provider costs тАЛ
Aug 28, 2026
· release notes
Unity Gateway allows tracking and capping spend on external AI providers like Bedrock.
Beta release of Unity Catalog Skills allowing governance and sharing of agent skills via SKILL.md specification.
Publishes agent skills as first-class catalog.schema.skill securables in Unity Catalog Not checked yet Enables coding agents to load skills live over MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. or download published skills Not checked yet
2 details in Everything
Genie Code is now available as a Lakeflow Jobs task (Beta) тАЛ
Aug 27, 2026
· release notes
Genie Code is now available as a Lakeflow Jobs task Beta
Atlassian Audit Logs connector (Beta) тАЛ
Aug 27, 2026
· release notes
New Atlassian Audit Logs connector in Lakeflow Connect Beta for ingesting organization events.
Adds the Atlassian Audit Logs connector to Lakeflow Connect Beta to ingest user, security, and configuration changes from Atlassian into Databricks. Not checked yet Exposes the new connector feature on the workspace admin Previews page for enabling access. Not checked yet
2 details in Everything
Lakeflow Designer August Release тАЛ
Aug 27, 2026
· release notes
Lakeflow Designer August release adds Unique operator, dynamic Select, Python preview mode, and visual canvas diff.
Adds the Unique operator to remove duplicate rows, optionally keying on a subset of columns and ordering the rows Not checked yet Expands the Aggregate operator with COUNT DISTINCT function support Not checked yet Adds dynamic column selection to the Select operator by data type, name pattern, or formula Not checked yet Adds config["is_preview"] check capability in the Python operator for distinct preview vs. run code paths Not checked yet Enables streaming output from the Python operator into the output pane during execution Not checked yet
+8 more
show less
Allows disabling individual operators or entire groups to exclude them from runs Not checked yet Adds operator lineage highlighting on the canvas when an operator is selected Not checked yet Adds a Run all button to execute every operator in the visual data prep at once Not checked yet Shows exact row count beneath each operator by running it in Max mode Not checked yet Adds Apply to canvas capability to transform results-table changes (renaming, filtering, inserting) back into operators Not checked yet Adds Load more functionality for filter values in the Filter operator Not checked yet Adds @ mention menu in expression editors to search columns and built-in SQL functions Not checked yet Shows a visual diff on the canvas in version history highlighting added, removed, or changed operators Not checked yet
13 details in Everything
AI Functions REST API is now generally available тАЛ
Aug 27, 2026
· release notes
AI Functions REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. GA allows calling ai_parse_document, ai_extract, and ai_classify programmatically.
Share foreign Delta tables with OpenSharing (Public Preview) тАЛ
Aug 27, 2026
· release notes
Adds OpenSharing capability for sharing foreign Delta tables from OneLake OneLake Microsoft's unified data lake built into Microsoft Fabric that gives every workspace a single, tenant-wide storage layer, letting cyber tools read and write data without managing separate storage accounts. , Hive, and AWS Glue in public preview.
Genie Code for AI Runtime (Public Preview) тАЛ
Aug 27, 2026
· release notes
Databricks introduces Genie Code for AI Runtime in public preview to generate distributed training code and debug GPU GPU A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. failures.
Zerobus Ingest into streaming tables is now generally available тАЛ
Aug 26, 2026
· release notes
Zerobus Ingest into streaming tables is now GA in Lakeflow Connect, enabling writes to streaming tables with standard Delta table mechanics.
Query text is now masked by default in system tables and APIs тАЛ
Aug 26, 2026
· release notes
Default masking of query text in system tables (system.query.history, system.access.audit) and APIs
statement_text in system.query.history returns <REDACTED> instead of the raw SQL by default. Not checked yet query_text in the Query History API and List Queries API returns <REDACTED>. Not checked yet Keys in request_params within system.access.audit containing SQL definitions are now omitted. Not checked yet
3 details in Everything
└──▷ BREAKING ON UPGRADE
! statement_text in system.query.history no longer returns raw SQL; it now returns <REDACTED>.! query_text in the Query History API and List Queries API no longer returns raw SQL; it now returns <REDACTED>.! Sensitive keys in request_params within system.access.audit are now omitted from the output.
Anysphere Audit Logs connector (Beta) тАЛ
Aug 25, 2026
· release notes
New beta Anysphere Audit Logs connector in Lakeflow Connect for Cursor admin APIs
ai_prep_search supports plain text and markdown input (Beta) тАЛ
Aug 25, 2026
· release notes
New ai_prep_search SQL function accepts plain text and markdown for RAG RAG Retrieval-augmented generation: looking up relevant documents and adding them to the prompt before the model answers. It grounds responses in private or current data without retraining. indexing.
The INSERT , UPDATE , and DELETE privileges are in Beta тАЛ
Aug 24, 2026
· release notes
Unity Catalog introduces INSERT, UPDATE, DELETE privileges in Beta
Adds INSERT, UPDATE, and DELETE privileges as least-privilege alternatives to MODIFY in Unity Catalog for controlled write access without schema changes. Not checked yet Requires Databricks Runtime 18.1 or above to utilize the new granular DML privileges. Not checked yet
2 details in Everything
Beta Verkada connector added to Lakeflow Connect for ingesting org audit logs and users.
Connect to the Databricks Genie app in Microsoft Teams with your workspace URL (Public Preview) тАЛ
Aug 24, 2026
· release notes
Databricks Genie app in Microsoft Teams now accepts workspace URL for connection
Default Python package repositories are GA for Lakeflow pipelines and classic compute тАЛ
Aug 21, 2026
· release notes
Default Python package repositories are now GA for Lakeflow pipelines and classic compute
Monitor all AI activity with the unified trace table (Beta) тАЛ
Aug 21, 2026
· release notes
Beta unified trace table captures all Unity Gateway AI requests/responses in OpenTelemetry OpenTelemetry An open standard and set of tools for collecting traces, metrics and logs from applications. It lets services export observability data without tying instrumentation to one monitoring vendor. format within Unity Catalog.
Context attributes in ABAC column mask and row filter policies (Beta) тАЛ
Aug 21, 2026
· release notes
Databricks adds context attributes to ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. column mask and row filter policies in Beta.
Adds request.client_id and request.is_on_behalf_of as context attributes for Unity Catalog ABAC conditions Not checked yet Supports has_context_attribute and has_context_attribute_value functions in the WHEN clause of column mask policies Beta Not checked yet Supports has_context_attribute and has_context_attribute_value functions in the WHEN clause of row filter policies Beta Not checked yet
3 details in Everything
Salesforce connector mTLS authentication is now generally available тАЛ
Aug 20, 2026
· release notes
Salesforce connector mTLS mTLS Mutual TLS, an encrypted connection where both client and server present certificates to prove their identity. It helps services verify who they are communicating with. authentication is now generally available in Lakeflow Connect
Write data back to Databricks from Google Sheets тАЛ
Aug 20, 2026
· release notes
Databricks Connector for Google Sheets adds write-back to Unity Catalog tables
Enables writing data from Google Sheets directly back to a Databricks Unity Catalog table Not checked yet
1 detail in Everything
Performance mode now applies to materialized view and streaming table refreshes from dbt tasks тАЛ
Aug 20, 2026
· release notes
dbt job-level performance_mode now propagates to materialized view and streaming table refreshes
└──▷ BREAKING ON UPGRADE
! Standard mode increases refresh startup latency from under a minute to four to six minutes for affected dbt refreshes.
Role-based access control (RBAC) is generally available тАЛ
Aug 19, 2026
· release notes
RBAC RBAC Role-based access control, where permissions are assigned to roles such as reader or administrator and users receive those roles. It makes access easier to manage than individual permission lists. allows users to assume roles, applying only that role's permissions rather than accumulated ones, preventing data mixing across use cases.
ai_extract precision mode is now generally available тАЛ
Aug 19, 2026
· release notes
Databricks ai_extract precision mode is now GA for advanced extraction workloads
The ai_extract function's precision mode is generally available for multi-page documents, schemas with 50+ fields, and complex conditional logic. Not checked yet
1 detail in Everything
Choose an effort level in Genie Code тАЛ
Aug 19, 2026
· release notes
Genie Code now allows users to choose between Auto and Low effort levels to optimize for cost or speed.
Adds Auto (default) and Low effort levels for Genie Code to balance response quality and cost Not checked yet
1 detail in Everything
Identity attributes in ABAC column mask policies (Beta) тАЛ
Aug 17, 2026
· release notes
Adds has_identity_attribute_value and has_identity_attribute_tag_match functions to ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. column mask policy conditions in Unity Catalog
Adds support for the has_identity_attribute_value function in the WHEN clause of Unity Catalog ABAC column mask policies to target users by synced identity attributes like department or country Not checked yet Adds support for the has_identity_attribute_tag_match function in the WHEN clause of Unity Catalog ABAC column mask policies to target users by synced identity attribute tags Not checked yet
2 details in Everything
Session restore for serverless jobs is now in Beta тАЛ
Aug 17, 2026
· release notes
Serverless jobs now save state snapshots for session restore in Beta
Adds automatic state snapshot capture for serverless jobs on failure, runs >= 30 minutes, or cancellation, enabling session restoration to interactive notebooks. Not checked yet
1 detail in Everything
DeepSeek V4 Pro (0813) is now available on Unity Gateway тАЛ
Aug 14, 2026
· release notes
DeepSeek V4 Pro (0813) added to Unity Gateway for agentic coding and complex reasoning.
Adds DeepSeek V4 Pro (0813) model support to Unity Gateway Not checked yet
1 detail in Everything
Beta Workiva connector added to Lakeflow Connect to ingest audit activity, users, and roles.
Sharing managed Iceberg tables is now generally available тАЛ
Aug 14, 2026
· release notes
Databricks now supports sharing managed Iceberg tables via Databricks-to-Databricks, open sharing, and to external clients.
Adds the ability for providers to share managed Iceberg tables in Databricks-to-Databricks sharing workflows Not checked yet Extends open sharing capabilities to include managed Iceberg tables Not checked yet
2 details in Everything
Cross-workspace access for serverless (Beta) тАЛ
Aug 13, 2026
· release notes
Beta release enabling serverless ingress/egress control between Databricks workspaces for Account admins.
cross-workspace access for serverless workloads allows Account admins to define source workspaces permitted for ingress and destination workspaces permitted for egress. Not checked yet
1 detail in Everything
Share data behind a firewall with SecureConnect is generally available тАЛ
Aug 13, 2026
· release notes
SecureConnect for OpenSharing is generally available, allowing data sharing behind firewalls without recipient allowlisting.
Expands SecureConnect networking billing to include charges for recipients and additional access scenarios. Not checked yet
1 detail in Everything
SQL alert task in Lakeflow Jobs is now available by default for workspaces with the compliance security profile enabled тАЛ
Aug 13, 2026
· release notes
SQL alert task in Lakeflow Jobs available by default for compliance workspaces
Enables sql_alert task in Lakeflow Jobs for evaluating Databricks SQL alerts within job pipelines, returning evaluation state as a task output. Not checked yet
1 detail in Everything
Tags page in Governance Hub (Beta) тАЛ
Aug 13, 2026
· release notes
Beta Tags page in Governance Hub provides centralized view of tag usage and recommendations
Beta feature: Databricks stores nine IdP IdP A service that authenticates users and supplies identity information to other applications. It is the sign-in authority behind many single sign-on systems. -sourced attributes on account users for Unity Catalog ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. policies
Account admins can now provision nine IdP-sourced attributes (e.g., title, department, cost center) to account users via automatic identity management or account SCIM SCIM A standard for creating, updating and removing user accounts and groups across applications. It helps an identity provider keep access in sync when people join, change roles or leave. 2.1. Not checked yet Unity Catalog ABAC column-masking policies can now reference these stored identity attributes for access control decisions. Not checked yet The Identity Attribute Control List allows governance over which specific attributes are permitted to be written from the identity provider identity provider A service that authenticates users and supplies identity information to other applications. It is the sign-in authority behind many single sign-on systems. . Not checked yet
3 details in Everything
Block or redact sensitive data in requests and responses тАЛ
Aug 13, 2026
· release notes
Databricks adds Sensitive Data Detection guardrail Beta to block or redact PII PII Personally Identifiable Information — data that can identify a specific individual, such as names, email addresses, or SSNs. Cyber tools flag or redact it to limit exposure and meet privacy regulations. in LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. interactions
Adds Sensitive Data Detection guardrail Beta to detect and block or redact structured sensitive data in requests/responses using regex and checksums. Not checked yet Enables Enhanced Unity Gateway preview on the account console to activate the new guardrail capabilities. Not checked yet
2 details in Everything
Beta release of the Amplitude connector for Lakeflow Connect to ingest product analytics data.
Custom trace views in the MLflow trace explorer are in Beta тАЛ
Aug 12, 2026
· release notes
Beta release of custom trace views in the MLflow trace explorer using Genie.
Glean connector in Lakeflow Connect (Beta) тАЛ
Aug 12, 2026
· release notes
Beta Glean connector added to Lakeflow Connect for ingesting company-wide usage insights and go links.
Define business concepts with Pages (Beta) тАЛ
Aug 12, 2026
· release notes
Introduces Pages Beta in Unity Catalog Semantics to define governed business terms and KPIs
Adds Pages as a new UI feature within Unity Catalog Semantics for defining business concepts like acronyms and KPIs. Not checked yet
1 detail in Everything
Google Search Console connector is available in Beta тАЛ
Aug 12, 2026
· release notes
Databricks Lakeflow Connect adds a managed Google Search Console connector in Beta.
Supports daily and hourly analytics dimensions (date, country, device, page, query), verified sites, and sitemaps via the new connector. Not checked yet
1 detail in Everything
Unity Catalog ABAC GRANT policies support more securable types in Beta тАЛ
Aug 11, 2026
· release notes
Unity Catalog ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. GRANT policies now support model services, MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. services, and agent securable types in Beta
ABAC GRANT policies now support model provider services as a securable type. Not checked yet ABAC GRANT policies now support Model Context Protocol Model Context Protocol Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. (MCP) services as a securable type. Not checked yet
2 details in Everything
REPLACE USING flows for standalone streaming tables (Beta) тАЛ
Aug 11, 2026
· release notes
Adds REPLACE USING flow for standalone streaming tables in Beta, allowing partial snapshot key-based updates without CDC feeds.
Adds REPLACE USING flow capability for standalone streaming tables Beta to replace rows matching key columns while preserving others. Not checked yet Introduces SEQUENCE BY column support in REPLACE USING flows to order updates and resolve out-of-order arrivals by highest sequence win. Not checked yet
2 details in Everything
ABAC GRANT policies now cover Unity Gateway resource types (Beta) тАЛ
Aug 11, 2026
· release notes
ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. GRANT policies now support model services, MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. and agent services via Unity Gateway tags
Databricks Unity Catalog READ METADATA privilege is now GA
Privilege grants row filters and column masks, and ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. policies Not checked yet
1 detail in Everything
Beta availability of the managed Notion connector in Lakeflow Connect to ingest pages, databases, blocks, comments, and users.
Ingests pages, data sources, databases, blocks, comments, and users from Notion into Databricks Not checked yet
1 detail in Everything
Alert system tables are in Public Preview тАЛ
Aug 10, 2026
· release notes
Databricks adds Public Preview system.alert schema with alerts and alert_evaluation_history tables for audit and analysis.
The system.alert.alerts table records configuration of every alert Not checked yet The system.alert.alert_evaluation_history table records one row per alert evaluation Not checked yet
2 details in Everything
DeepSeek V4 Flash (0731) is now available on Unity Gateway тАЛ
Aug 10, 2026
· release notes
DeepSeek V4 Flash (0731) is now available on Unity Gateway for reasoning and coding.
Databricks adds beta ai_search() SQL function to query AI Search indexes directly in queries.
Adds the ai_search SQL function to retrieve and synthesize answers from configured AI Search indexes. Not checked yet
1 detail in Everything
Tag automations Beta lets you automatically assign or remove governed tags on Unity Catalog assets via natural language (Genie) or manual rules.
Introduces Tag automations in Beta to automatically maintain tag accuracy on Unity Catalog tables and volumes Not checked yet Adds the ability to define automation rules using natural language via Genie or manually by specifying scope, conditions, and action Not checked yet Includes a dry run phase that identifies matching assets without applying tags when an automation is saved Not checked yet
3 details in Everything
Change data feed on materialized views (Beta) тАЛ
Aug 7, 2026
· release notes
New beta capability to read change data feeds directly from materialized views for auditing and replication.
Adds the ability to read a change data feed change data feed A stream of records describing every insert, update, and delete that occurs in a database, captured in real time so downstream systems can react to changes without polling. from a materialized view in Databricks SQL or Lakeflow pipelines using Databricks Runtime 18 LTS or above Not checked yet
1 detail in Everything
Unity Gateway API: Manage resources with SDKs, the CLI, and Terraform тАЛ
Aug 6, 2026
· release notes
Unity Gateway resources are now manageable via SDKs, CLI, and Terraform for full CRUD CRUD The four basic database operations — Create, Read, Update, Delete — used as shorthand for any system that manages persistent data records, giving cyber tools a standard model for interacting with stored information. operations.
Adds databricks-sdk (Python) version 0.125.0+ support for Unity Gateway resource management Not checked yet Adds databricks-sdk-go (Go) version v0.170.0+ support for Unity Gateway resource management Not checked yet Adds databricks-sdk-java (Java) version 0.143.0+ support for Unity Gateway resource management Not checked yet Adds databricks-cli version 1.11.0+ commands for managing Unity Gateway resources programmatically Not checked yet Adds Terraform provider version 1.124.0+ support for Unity Gateway endpoints, model providers, and MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. servers Not checked yet
5 details in Everything
Databricks-managed MCP connectors are now integrated with Unity Gateway (Beta) тАЛ
Aug 6, 2026
· release notes
Databricks-managed MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. connectors for Genie One and Code migrate to Unity Gateway Beta for centralized governance.
Genie Code web search enters Beta, allowing current information lookups when enabled in Previews.
Enables Genie Code to search the public web for current information answers Not checked yet Requires manual activation from the Previews page to use web search Not checked yet
2 details in Everything
Moonshot AI Kimi K3 is now available on Unity Gateway тАЛ
Aug 6, 2026
· release notes
Moonshot AI Kimi K3 model added to Unity Gateway with native vision support for long-horizon coding and reasoning.
Adds Moonshot AI Kimi K3 model availability on Unity Gateway for long-horizon coding, knowledge work, and reasoning tasks Not checked yet Introduces native vision capability to the Kimi K3 model served via Unity Gateway Not checked yet
2 details in Everything
Search and replace text across files тАЛ
Aug 5, 2026
· release notes
Adds search and replace text capability across files in Git folders and Declarative Automation Bundles via the Search pane.
Genie One is now available in Microsoft Excel тАЛ
Aug 5, 2026
· release notes
Genie One is now available in the Databricks Excel Add-in, allowing users to query governed Databricks data using natural language and import results as native rows and columns.
REPLACE USING flows for pipelines (Beta) тАЛ
Aug 5, 2026
· release notes
Databricks adds Beta REPLACE USING flows for pipelines to handle partial snapshots without explicit CDC.
Introduces REPLACE USING flow in Lakeflow pipelines Beta to replace rows matching key columns from partial snapshots Not checked yet
1 detail in Everything
Genie One is now available in Google Sheets тАЛ
Aug 5, 2026
· release notes
Genie One now available in the Databricks Connector for Google Sheets
Schema-level HTTP connections in Unity Catalog are now generally available тАЛ
Aug 4, 2026
· release notes
Schema-level HTTP connections are generally available in Unity Catalog
Adds HTTP connection capability scoped to a specific schema within Unity Catalog Not checked yet
1 detail in Everything
Full page Genie Code is now generally available тАЛ
Aug 4, 2026
· release notes
Databricks introduces generally available Full Page Genie Code for parallel chats and integrated file editing.
Beta release of the managed PagerDuty connector in Lakeflow Connect for ingesting incident and audit data.
SharePoint connectors (managed and standard) reach GA in Lakeflow Connect for ingestion.
Adds managed SharePoint connector for fully managed ingestion pipelines via Lakeflow Connect Not checked yet Adds standard SharePoint connector allowing custom pipelines built with Spark and SQL APIs via Lakeflow Connect Not checked yet
2 details in Everything
Sharing Delta tables with deletion vectors and column mapping is now generally available тАЛ
Aug 4, 2026
· release notes
Delta sharing with deletion vectors and column mapping is GA
Enables sharing Delta tables with deletion vectors enabled in Databricks Not checked yet Enables sharing Delta tables with column mapping enabled in Databricks Not checked yet
2 details in Everything
Sharing Delta tables with Iceberg reads enabled is now generally available тАЛ
Aug 4, 2026
· release notes
Databricks makes it generally available to share Delta tables while allowing receivers to use Iceberg reads.
NetSuite connector reaches GA in Lakeflow Connect for ingesting ERP data into Databricks.
The managed NetSuite connector in Lakeflow Connect is now generally available. Not checked yet
1 detail in Everything
Format AI/BI dashboard widget titles and descriptions with rich text тАЛ
Aug 4, 2026
· release notes
Databricks now supports rich text formatting for AI/BI dashboard widget titles and descriptions
Google Drive connectors for Lakeflow Connect reach general availability for managed and standard ingestion.
Adds GA managed connector in Lakeflow Connect for fully managed Google Drive ingestion pipelines Not checked yet Adds GA standard connector in Lakeflow Connect for custom pipelines using Spark and SQL APIs Not checked yet
2 details in Everything
Automatic cost attribution for materialized views and streaming tables in Databricks SQL тАЛ
Aug 3, 2026
· release notes
Databricks SQL materialized views and streaming tables now inherit SQL warehouse custom tags for cost attribution
Refresh materialized views and streaming tables without exempting the pipeline owner from ABAC policies тАЛ
Aug 3, 2026
· release notes
Refresh materialized views and streaming tables without exempting pipeline owner from ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. policies
Serverless compute access control is now generally available тАЛ
Aug 3, 2026
· release notes
Serverless compute access control is now generally available, allowing admins to govern access via Default Interactive and Automated Compute objects.
Adds general availability of serverless compute access control with Default Interactive Compute and Default Automated Compute objects for governing notebook/job access Not checked yet Sets default Can Use permission on both new compute objects for all workspace users Not checked yet
2 details in Everything
Environment variables for serverless jobs are in Beta тАЛ
Aug 3, 2026
· release notes
Serverless Lakeflow Jobs tasks can now receive environment variables.
Allows defining named environment variable entries at the job level. Not checked yet
1 detail in Everything
Filter updates for the Databricks Excel Add-in тАЛ
Aug 3, 2026
· release notes
Databricks Excel Add-in gains cascading filters and case-insensitive matching via ILIKE while removing legacy operators.
Enables cascading filters in the Databricks Excel Add-in to filter imported data dynamically Not checked yet
1 detail in Everything
Secrets in Unity Catalog is now GA, allowing secrets as securable objects governed by three-level namespace and catalog privileges.
Databricks editor adds spaces to organize work by folder or project
Packaged clean rooms are now generally available
Jul 31, 2026
· release notes
Databricks packaged clean rooms are now generally available
New beta OpenAI connector ingests organization admin data into Lakeflow Connect
Web terminal on serverless GPU compute ( AI Runtime ) is in Public Preview
Jul 31, 2026
· release notes
Public preview of web terminal on serverless GPU GPU A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. compute (AI Runtime)
Use Unity Gateway with your own Claude enterprise subscription
Jul 30, 2026
· release notes
Enable Unity Gateway to route Claude Code via existing Anthropic subscriptions for governance and cost control.
Enables routing Claude Code requests through Unity Gateway using an existing Claude Pro, Max, Enterprise, or Team subscription Not checked yet
1 detail in Everything
Python UDTFs in Unity Catalog are now generally available
Jul 28, 2026
· release notes
Python UDTFs in Unity Catalog GA on Databricks Runtime 18 with serverless, classic, and SQL warehouse support
General availability of Python user-defined table functions (UDTFs) in Unity Catalog on Databricks Runtime 18 Not checked yet Ability to invoke registered Python UDTFs in the FROM clause of SQL statements to return multiple rows and columns Not checked yet Execution support for Unity Catalog Python UDTFs on serverless compute, classic compute with standard access mode, and SQL warehouses Not checked yet
3 details in Everything
Anthropic Claude Opus 5 now available as a Databricks-hosted model
Jul 24, 2026
· release notes
Databricks Model Serving now supports Anthropic Claude Opus 5 as a Databricks-hosted model.
Adds Claude-Opus-5 as a Databricks-hosted model in Model Serving, accessible via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. with pay-per-token pricing. Not checked yet Enables querying reasoning capabilities through the hosted Claude Opus 5 instance. Not checked yet Enables querying vision capabilities through the hosted Claude Opus 5 instance. Not checked yet
3 details in Everything
Git CLI commands in Git folders are now in Public Preview
Jul 24, 2026
· release notes
Git CLI functionality is now available in Public Preview for eligible workspaces.
Adds git stash, interactive rebase, and submodule support to workspace Git folders. Not checked yet
1 detail in Everything
Lakeflow Designer Summer Release adds Prepare, Enter Data, and Select operators; enhances Join, Filter, Output, and Source operators; improves UI navigation.
Adds Prepare operator for common data-cleansing tasks including text case changes, data type conversions, value replacement, and formulas Not checked yet Adds Enter Data operator with a spreadsheet-style editor to manually create tabular data Not checked yet Adds Select operator to choose columns to keep, rename columns, and reorder them in the output Not checked yet Adds Visualization operator to create and view visualizations directly on the canvas Not checked yet Expands Output operator to support inserting or appending data in Unity Catalog tables, publishing results as materialized views, and writing CSV, Excel, or JSON files to Unity Catalog volumes Not checked yet
+13 more
show less
Expands Join operator with three default outputs (unmatched left rows, matched rows, unmatched right rows) for easier workflow branching Not checked yet Adds Match case option to the Join operator, which now matches string keys case-insensitively by default Not checked yet Expands Filter operator with separate outputs for matching and non-matching rows to branch data based on filter conditions Not checked yet Improves date filtering in the Filter operator's picker to support navigation across years as well as months Not checked yet Adds ai_query, ai_parse_document, ai_forecast, and ai_prep_search AI functions to the AI Function operator Not checked yet Expands Source operator to target an entire Unity Catalog volume or folder path instead of individual files Not checked yet Allows using Unity Catalog metric views as source nodes in Lakeflow Designer Not checked yet Adds inline rich-text editing to the Note operator for headings, text styling, links, images, and tables directly on the canvas Not checked yet Adds a searchable table of contents panel in the left sidebar to find and navigate to any operator Not checked yet Adds command palette accessible via Cmd+Shift+P (macOS) or Ctrl+Shift+P (Windows) to quickly access editor actions Not checked yet Adds a dedicated Visual data prep listing page in the sidebar with a getting-started video banner Not checked yet Enables choosing the serverless environment used for both interactive runs and scheduled jobs when working with visual data prep files Not checked yet Supports deploying and running visual data prep files with Declarative Automation Bundles by configuring them as notebook tasks and referencing notebook_task.notebook_path Not checked yet
18 details in Everything
Governed tags system table is in Beta
Jul 24, 2026
· release notes
Databricks beta release: new system.tags.governed_tags system table for governed tag management
C++ SDK for the Zerobus Ingest connector is now available in Beta
Jul 23, 2026
· release notes
New beta C++ SDK for Zerobus Ingest connector supports high-throughput gRPC gRPC An open-source remote procedure call framework from Google that uses HTTP/2 and Protocol Buffers to let services call each other's functions across a network, giving tools low-latency, strongly typed inter-process communication. ingestion into Unity Catalog.
Adds C++ SDK for the Zerobus Ingest connector in Lakeflow Connect Not checked yet
1 detail in Everything
Genie Code scheduled tasks are in Beta
Jul 23, 2026
· release notes
New Beta capability for automating Genie Code sessions on a schedule.
Schedules prompt execution for unattended background tasks. Not checked yet
1 detail in Everything
Role-based access control (RBAC) is in Public Preview
Jul 22, 2026
· release notes
Databricks RBAC RBAC Role-based access control, where permissions are assigned to roles such as reader or administrator and users receive those roles. It makes access easier to manage than individual permission lists. enters Public Preview, allowing users to assume specific roles for session-limited permissions.
Role-based access control Role-based access control Role-based access control, where permissions are assigned to roles such as reader or administrator and users receive those roles. It makes access easier to manage than individual permission lists. (RBAC) is now in Public Preview Not checked yet
1 detail in Everything
Databricks-hosted sample datasets available in the samples catalog
Jul 22, 2026
· release notes
Databricks-hosted sample datasets are now available in the Unity Catalog samples catalog.
Variant and Variant shredding are now generally available
Jul 22, 2026
· release notes
Variant and Variant shredding are generally available in Databricks for high-performance semi-structured data handling.
VARIANT columns in new tables now support automatic Variant shredding for optimized query performance Not checked yet
1 detail in Everything
Override the REPLACE WHERE predicate for a single refresh
Jul 22, 2026
· release notes
Databricks adds WHERE clause to REFRESH STREAMING TABLE to override REPLACE WHERE for single refreshes
Priority pay-per-token for Foundation Model APIs
Jul 22, 2026
· release notes
Databricks adds Priority pay-per-token mode for Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. to ensure consistent latency under load
Enables Priority pay-per-token (priority mode) as a payment option for Foundation Model APIs, now generally available for partner models and in preview for open source models. Not checked yet Provides more consistent availability for latency-sensitive, real-time applications under load via the new priority mode. Not checked yet
2 details in Everything
Email notifications for expiring service principal tokens are now generally available
Jul 22, 2026
· release notes
Email notifications for expiring service principal tokens are now generally available
Databricks automatically sends email notifications to workspace admins when a service principal token is set to expire within seven days Not checked yet
1 detail in Everything
Transactions are now generally available
Jul 22, 2026
· release notes
Databricks makes transactions on Unity Catalog managed Delta tables GA using BEGIN ATOMIC or BEGIN TRANSACTION syntax
Transactions that write to Unity Catalog managed Delta tables are now generally available. Not checked yet Group multiple SQL statements across one or more tables into a single atomic transaction using BEGIN ATOMIC ... END; syntax. Not checked yet Support grouping multiple SQL statements across one or more tables into a single atomic transaction using BEGIN TRANSACTION; ... COMMIT; syntax. Not checked yet
3 details in Everything
Automated setup for S3 external locations
Jul 22, 2026
· release notes
Adds automated S3 external location setup via IAM IAM Identity and Access Management — a framework of policies and services for controlling who can authenticate to systems and what resources they are authorized to use, letting cyber tools enforce least-privilege and audit access. temporary delegation for provisioning roles, credentials and locations
Enables automated setup for S3 external locations using AWS IAM AWS IAM AWS Identity and Access Management is Amazon's service for controlling who and what can call AWS APIs, by attaching policies that allow or deny actions on resources. Cyber tools use it to scope credentials to least privilege. temporary delegation to configure the external location and provision all required resources including the IAM role, storage credential, and external location. Not checked yet Provides built-in approval workflows for users who need AWS admin authorization during S3 external location provisioning. Not checked yet
2 details in Everything
Google Gemini 3.6 Flash now available as a Databricks-hosted model
Jul 21, 2026
· release notes
Google Gemini 3.6 Flash now available as a Databricks-hosted model via Model Serving
Google Gemini 3.5 Flash Lite now available as a Databricks-hosted model
Jul 21, 2026
· release notes
Google Gemini 3.5 Flash Lite added as a Databricks-hosted model via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. .
Mention Genie Code in notebook comments
Jul 20, 2026
· release notes
Databricks Adds ability to mention Genie Code in notebook comments for inline AI assistance
Enables @ mention of Genie Code directly inside notebook comment threads to request AI help without leaving the thread Not checked yet
1 detail in Everything
Anthropic managed ingestion connector (Beta)
Jul 17, 2026
· release notes
Beta release of the Anthropic managed ingestion connector in Lakeflow Connect for ingesting audit and chat data.
Adds Beta support for ingesting Compliance API data from Anthropic into Databricks via Lakeflow Connect Not checked yet
1 detail in Everything
Databricks Genie app in Microsoft Teams is now in Public Preview
Jul 17, 2026
· release notes
Databricks Genie app in Microsoft Teams is now in Public Preview.
Connect and govern any external model provider
Jul 17, 2026
· release notes
Databricks adds Unity Gateway for unified governance and observability of any external model provider
Enables connecting and governing any external AI model provider (e.g., OpenAI, Anthropic) via Unity Gateway Not checked yet Adds standard GRANT and REVOKE permissions (EXECUTE to query, MANAGE to update) for external models within Unity Catalog Not checked yet Enables default usage tracking with token counts, latency, and cost attribution per user, team, or app Not checked yet Supports attaching inference tables to log full request and response payloads to governed Delta tables Not checked yet
4 details in Everything
Scala and Java UDFs in Unity Catalog are now generally available
Jul 16, 2026
· release notes
Databricks GA of Scala and Java UDFs in Unity Catalog for governance across notebooks, jobs, and SQL warehouses.
Adds general availability for registering Scala and Java user-defined functions (UDFs) in Unity Catalog Not checked yet Enables reuse of existing JVM JVM The Java Virtual Machine, a runtime maintained by Oracle (and implemented by others) that executes Java bytecode on any OS, giving cyber tools a portable, garbage-collected environment with a large ecosystem of libraries. logic within SQL warehouses, notebooks, and jobs with Unity Catalog governance Not checked yet Provides better performance than Python UDFs on all compute tiers, including serverless Not checked yet
3 details in Everything
Databricks adds Beta agentic code converter for migrating legacy SQL dialects to ANSI SQL.
ai_extract and ai_classify are now available by default for workspaces with the compliance security profile enabled
Jul 15, 2026
· release notes
ai_extract and ai_classify SQL AI functions are now enabled by default for compliance workspaces
Thinking Machine Labs Inkling now available as a Databricks-hosted model in Public Preview
Jul 15, 2026
· release notes
Databricks Model Serving adds Thinking Machine Labs Inkling as a hosted MoE MoE A model design that splits its layers into many expert sub-networks and routes each token to only a few. The model holds many parameters but spends the compute of a much smaller one per token. model.
Inkling supports mixed text/image inputs, 1 million token context window context window The maximum amount of text, counted in tokens, a model can take in and produce in one request. Everything the model knows about the task has to fit inside it. , and is optimized for coding, reasoning, and agentic tool-use. Not checked yet
1 detail in Everything
Data Classification is now available by default for additional compliance standards
Jul 15, 2026
· release notes
Data Classification enabled by default for C5, TISAX TISAX A vehicle-industry information-security assessment standard, maintained by the ENX Association, that lets automotive suppliers prove their data-handling practices to OEMs without each OEM running its own separate audit. , and IRAP compliance profiles
Custom URLs for your Databricks account (Public Preview)
Jul 15, 2026
· release notes
Public Preview: Custom URLs for Databricks account entry point
Allowlist partner platform IPs in context-based ingress control (Beta)
Jul 14, 2026
· release notes
Context-based ingress control now allows allowing partner platform IPs via Partner Platforms network source.
View warehouse activity details (General Availability)
Jul 13, 2026
· release notes
SQL warehouse monitoring UI gains Activity details toggle with color-coded annotations and filtered query history
Activity details toggle in SQL warehouse monitoring UI now generally available for color-coded annotations on Running clusters chart Not checked yet Hover over bars to see metadata or click fetching activity to filter the query history table Not checked yet
2 details in Everything
Secrets in Unity Catalog (Public Preview)
Jul 10, 2026
· release notes
Databricks releases Public Preview for storing secrets as objects in Unity Catalog, enabling unified governance.
Protobuf tensor input for custom model serving endpoints (Public Preview)
Jul 9, 2026
· release notes
Databricks adds Protobuf tensor input for custom model serving endpoints in Public Preview.
Allows sending Content-Type: application/x-protobuf requests with a serialized KServe v2 KServe v2 An open inference protocol (maintained by the KServe community) that standardises how clients send prediction requests to ML model servers, letting cyber tools swap backends without changing their inference code. ModelInferRequest to custom model serving endpoints. Not checked yet
1 detail in Everything
Beta release of the Strac connector for Lakeflow Connect allows ingesting data-security audit events.
OpenAI GPT-5.6 Sol, GPT-5.6 Terra, and GPT-5.6 Luna now available as Databricks-hosted models
Jul 9, 2026
· release notes
Databricks Model Serving adds OpenAI GPT-5.6 Sol, Terra, and Luna as Databricks-hosted models accessible via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves.
Databricks SQL alerts are now available by default for workspaces with the compliance security profile enabled
Jul 9, 2026
· release notes
SQL alerts enabled by default for workspaces with compliance security profile
Zhipu AI GLM 5.2 now available as a Databricks-hosted model
Jul 8, 2026
· release notes
Databricks adds GLM 5.2 to Model Serving
REPLACE WHERE flows are now generally available
Jul 7, 2026
· release notes
REPLACE WHERE flows for pipelines and streaming tables are now generally available
Excel file format support is now available by default
Jul 7, 2026
· release notes
Databricks now supports ingesting, parsing, and querying .xls, .xlsx .xlsx The file format used by Microsoft Excel for spreadsheets, based on the Open XML standard. Cyber tools use it to ingest or export structured tabular data in a format analysts already work with. , and .xlsm files by default without external libraries.
Adds built-in support for .xls, .xlsx, and .xlsm file formats to all workspaces Not checked yet Enables direct ingest and query of Excel files without external libraries or manual conversions Not checked yet
2 details in Everything
Private Link for account-level resources (Beta)
Jul 6, 2026
· release notes
Databricks adds beta Private Link Private Link An AWS networking feature that exposes a service over a private endpoint inside a VPC, so traffic never traverses the public internet, giving cyber tools a way to integrate with cloud services without opening public routes. support for account-level resources like the console
Okta support for automatic identity management is now generally available
Jul 3, 2026
· release notes
Databricks now supports Okta as an identity provider identity provider A service that authenticates users and supplies identity information to other applications. It is the sign-in authority behind many single sign-on systems. for automatic user and group synchronization.
Okta integration allows syncing users and groups without SCIM SCIM A standard for creating, updating and removing user accounts and groups across applications. It helps an identity provider keep access in sync when people join, change roles or leave. . Not checked yet
1 detail in Everything
Google Gemini 3.1 Flash Image and Gemini 3 Pro Image now available as Databricks-hosted models
Jul 1, 2026
· release notes
Databricks now hosts Google Gemini 3.1 Flash Image and Gemini 3 Pro Image models
Model Serving adds support for Google Gemini 3.1 Flash Image as a Databricks-hosted model via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. Not checked yet Model Serving adds support for Google Gemini 3 Pro Image as a Databricks-hosted model via Foundation Model APIs Not checked yet
2 details in Everything
Databricks introduces Beta Veeva Vault connector in Lakeflow Connect for ingesting standard and custom objects.
Adds Veeva Vault connector Beta to Lakeflow Connect for workspace admins to enable via the Previews page Not checked yet Enables ingestion of standard objects, custom objects, system tables, and audit trail tables from Veeva Vault using OAuth 2.0 OAuth 2.0 An open authorization framework (IETF RFC 6749) that lets a service grant limited API access to a third party without sharing credentials, giving cyber tools a standard way to authenticate against external platforms. M2M authentication via Microsoft Entra ID Entra ID Microsoft's cloud-based identity and access management service, formerly Azure Active Directory, used by cyber tools to authenticate users and enforce access policies in Microsoft 365 and Azure environments. Not checked yet
2 details in Everything
Anthropic Claude Sonnet 5 now available as a Databricks-hosted model
Jun 30, 2026
· release notes
Databricks now hosts Anthropic Claude Sonnet 5, accessible via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. for agentic workflows.
Adds Claude Sonnet 5 as a Databricks-hosted model in Model Serving Not checked yet Enables access to Claude Sonnet 5 through Foundation Model APIs Not checked yet
2 details in Everything
Databricks introduces Lakehouse Real-Time Beta , a serverless SQL warehouse type for low-latency, high-concurrency sub-second queries against Unity Catalog.
Run a subset of job tasks with their dependencies
Jun 30, 2026
· release notes
Adds + modifier to job task keys to run upstream/downstream dependencies via UI, API, CLI, SDKs or Bundles
Adds +task_key+ modifier to run a task with both upstream and downstream dependencies Not checked yet Extends partial run capability to UI, REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. , Databricks CLI, SDKs, and Declarative Automation Bundles Not checked yet
2 details in Everything
SQL alert task in Lakeflow Jobs is now generally available
Jun 29, 2026
· release notes
Lakeflow Jobs gains generally available SQL alert task to evaluate alerts and branch downstream logic.
Adds a new SQL alert task in Lakeflow Jobs that evaluates a Databricks SQL alert and returns its state as a task output value for branching. Not checked yet
1 detail in Everything
Custom Docker images for AI Runtime CLI workloads (Beta)
Jun 29, 2026
· release notes
Databricks Air now supports custom Docker containers for AI Runtime CLI workloads.
Adds support for bringing your own Docker container image to air (AI Runtime CLI) workloads. Not checked yet
1 detail in Everything
Parquet v2 for Delta Lake tables is generally available
Jun 29, 2026
· release notes
Databricks Runtime 18.1+ now supports Parquet v2 for Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. tables to improve query performance.
Adds delta.parquet.format.version table property to enable Parquet v2 features such as advanced encodings and INT64 timestamps. Not checked yet
1 detail in Everything
Govern LLMs in Unity Catalog with model services (Beta)
Jun 29, 2026
· release notes
Beta release adds model services in Unity Catalog to govern and share Databricks-hosted LLMs LLMs A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. across workspaces.
Adds model services as a new Unity Catalog securable for governing Databricks-hosted LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. endpoints Not checked yet Adds system.ai schema containing ready-to-use model services for foundation models Not checked yet Enables creation of custom model services via the Unity Gateway UI, Catalog Explorer, or Unity Catalog REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. Not checked yet
3 details in Everything
Microsoft Dynamics 365 connector (GA)
Jun 26, 2026
· release notes
Microsoft Dynamics 365 connector in Lakeflow Connect reaches general availability for CRM and ERP data ingestion.
Metric view parameters are now in Public Preview
Jun 26, 2026
· release notes
Databricks Metric Views support parameters for dynamic query-time binding.
Adds parameter support in metric views to bind values at query time, allowing multiple query variants from a single definition. Not checked yet
1 detail in Everything
Data Classification is now available by default for some workspaces with the compliance security profile enabled
Jun 23, 2026
· release notes
Data Classification is now available by default for workspaces with the compliance security profile enabled
Block specific internet destinations in network policies (Public Preview)
Jun 23, 2026
· release notes
New capability to block specific internet destinations in serverless network policies via REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. .
Change the owner of standalone pipeline streaming tables and materialized views
Jun 23, 2026
· release notes
New ALTER STREAMING TABLE and ALTER MATERIALIZED VIEW ... SET OWNER TO capability in Databricks SQL
Adds ALTER STREAMING TABLE <table_name> SET OWNER TO <user_or_role> syntax to Databricks SQL for changing ownership of standalone pipeline streaming tables Not checked yet Adds ALTER MATERIALIZED VIEW <view_name> SET OWNER TO <user_or_role> syntax in Databricks SQL to change ownership of materialized views backed by standalone pipelines Not checked yet
2 details in Everything
Standalone pipeline materialized views support expectations
Jun 23, 2026
· release notes
Standalone pipeline materialized views now support data quality expectations via the CONSTRAINT clause
Databricks adds managed agent memory in Beta with Unity Catalog storage and REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. access.
Adds Managed agent memory feature allowing AI agents to store long-term memories across conversations. Not checked yet
1 detail in Everything
Databricks connector for Power Platform is generally available
Jun 23, 2026
· release notes
Databricks connector for Microsoft Power Platform reaches general availability.
Square managed ingestion connector (Beta)
Jun 22, 2026
· release notes
Beta release of the Square managed ingestion connector for Lakeflow Connect.
Wiz Audit Logs managed ingestion connector (Beta)
Jun 22, 2026
· release notes
Databricks adds beta Wiz Audit Logs managed ingestion connector via Lakeflow Connect
Genie Code system table now counts only user-submitted interactions
Jun 22, 2026
· release notes
Genie Code system table system.access.assistant_events now counts only user-submitted interactions, excluding background/automated requests.
The Discover page is now in Public Preview
Jun 22, 2026
· release notes
The Discover page is now in Public Preview for browsing Unity Catalog assets.
Netskope Logs managed ingestion connector (Beta)
Jun 20, 2026
· release notes
Databricks adds Beta Netskope Logs managed ingestion connector via Lakeflow Connect
Incremental ingestion for Workday Reports (Beta)
Jun 18, 2026
· release notes
Databricks Lakeflow Connect Workday connector adds beta incremental ingestion via configurable cursor columns
Adds cursor_column configuration option to the Lakeflow Connect Workday Reports connector in Beta for selective row ingestion based on last pipeline run Not checked yet Enables performance and cost reduction by ingesting only rows changed since the previous run when a cursor column is set Not checked yet
2 details in Everything
Salesforce Marketing Cloud managed ingestion connector (Beta)
Jun 18, 2026
· release notes
Adds Beta Salesforce Marketing Cloud managed ingestion connector for campaigns and engagement data.
Lakehouse Federation Lakehouse Federation A data architecture pattern that lets a single query engine read from multiple external data sources—databases, data lakes, warehouses—without moving or copying the data, reducing pipeline complexity for tools that need broad data access. syncs table/column comments from external sources to Unity Catalog foreign tables in Beta
Omnigent coding agent meta-harness is now in Beta
Jun 17, 2026
· release notes
Databricks adds Omnigent, a beta coding agent meta-harness with shared sessions and context policies.
Adds Omnigent beta: a web UI for managing coding agents (e.g. Claude Code, Codex) with persistent shareable sessions, team collaboration, mobile access, contextual policies, and unified deployment infrastructure. Not checked yet
1 detail in Everything
Lakeflow Designer is now generally available
Jun 16, 2026
· release notes
Lakeflow Designer GA launches a visual drag-and-drop canvas for preparing data with no-code workflows governed by Unity Catalog.
Introduces Lakeflow Designer , a visual, no-code experience on a drag-and-drop canvas to prepare and transform data with built-in operators or natural language. Not checked yet
1 detail in Everything
Install third-party apps from Databricks Marketplace
Jun 16, 2026
· release notes
Install third-party data and AI applications from Databricks Marketplace in Unity Catalog-governed workspaces
Allows discovery, installation, and execution of third-party data and AI apps from the Databricks Marketplace within a Unity Catalog-governed workspace Not checked yet Introduces a dedicated Apps category filter in the Open Marketplace UI to locate available third-party applications Not checked yet
2 details in Everything
Databricks-to-Databricks Delta Sharing across regulatory domains is now available (Public Preview)
Jun 15, 2026
· release notes
Public Preview of Databricks-to-Databricks Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. across regulatory domains (commercial to AWS GovCloud/DoD, Azure China).
OIDC is now supported for sharing to Iceberg clients
Jun 15, 2026
· release notes
Databricks adds OIDC OIDC OpenID Connect, a sign-in protocol built on OAuth 2.0. An application uses identity information from a trusted provider instead of maintaining a separate password for each user. support for sharing OpenSharing to Iceberg clients like Snowflake and OSS OSS Open-source software: code whose licence permits anyone to inspect, modify, and redistribute it. Cyber tools build on it to reuse auditable libraries without licensing fees. Spark
Adds OpenID Connect (OIDC) federation support for reading OpenSharing shares via Iceberg REST Catalog clients Not checked yet
1 detail in Everything
AUTO CDC flows for streaming tables in Databricks SQL are generally available
Jun 15, 2026
· release notes
Databricks SQL now supports AUTO CDC flows for streaming tables in CREATE STREAMING TABLE.
AUTO CDC clause added to CREATE STREAMING TABLE in Databricks SQL for processing change data capture records. Not checked yet Automated handling of out-of-order records as SCD type 1 or SCD type 2 without requiring complex MERGE INTO logic. Not checked yet
2 details in Everything
Configure the recovery period for dropped managed tables (Public Preview)
Jun 15, 2026
· release notes
New configurable recovery period (0-30 days) for dropped managed tables in Unity Catalog
Adds the ability to configure the recovery period for dropped managed tables in Unity Catalog at the catalog or schema level. Not checked yet Introduces a new configuration option to set the recovery duration to 0 hours (immediate purge) or between 7–30 days. Not checked yet
2 details in Everything
Use Universal Commits to purchase Marketplace partner products
Jun 15, 2026
· release notes
Enables purchasing eligible Databricks Marketplace partner products using up to 10% of Universal Commits.
Share data behind a firewall with SecureConnect (Public Preview)
Jun 15, 2026
· release notes
New SecureConnect feature allows sharing data from behind firewalls without allowlisting recipient networks
Adds SecureConnect capability to OpenSharing, enabling data sharing from cloud storage behind a firewall or private endpoint without adding recipient networks to an allowlist. Not checked yet Extends SecureConnect support to clean rooms. Not checked yet
2 details in Everything
Databricks Runtime 19 for Machine Learning
Jun 15, 2026
· release notes
Databricks Runtime 19 ML updates core libraries including CUDA CUDA NVIDIA's platform and programming interface for running general computation on its GPUs. Nearly every AI training and inference stack is built on it first. 13.0, flash_attn, and adds GPU GPU A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. -native Scala JARs for XGBoost.
Adds flash_attn version 2.8.3 to the default Python environment on GPU clusters Not checked yet Updates CUDA support to version 13.0 and includes cuDNN 9.23.0.39 for GPU clusters Not checked yet Adds GPU-specific XGBoost4j Scala JARs (xgboost4j-gpu_2.13 and xgboost4j-spark-gpu_2.13) to GPU cluster classpaths Not checked yet
3 details in Everything
└──▷ BREAKING ON UPGRADE
! The TensorFlow/Keras stack (tensorflow, keras, tf_keras) is removed; workloads must install these via cluster library or init script! Approximately 40 ML runtime packages are removed in this version compared to Databricks Runtime 18 ML
ai_query function is now generally available for querying AI models from SQL or Python.
Databricks Runtime 19 introduces Auto CDF, new SQL functions (truncate, trim_array, parse_sql), and Python worker env vars
spark.pythonWorkerEnv.<variable-name> config prefix allows setting session-scoped environment variables for Python UDF workers Not checked yet Automatic change data feed change data feed A stream of records describing every insert, update, and delete that occurs in a database, captured in real time so downstream systems can react to changes without polling. (Auto CDF) enables row-level change tracking at query time without manual per-table configuration Not checked yet New truncate(number, scale) function truncates numbers to a specified scale without rounding in SQL and DataFrame APIs Not checked yet New trim_array(array, count) function removes elements from array ends based on sign of count in SQL and DataFrame APIs Not checked yet New variant_strip_nulls(value) function recursively removes variant null fields in SQL and DataFrame APIs Not checked yet
+9 more
show less
New parse_sql(sql_statement) function returns structured JSON metadata for parsed SQL statements Not checked yet New collect_union aggregate function returns distinct union of array elements within groups in SQL and DataFrame APIs Not checked yet spark.databricks.execution.pythonUDF.arrow.enabled config enables Arrow-optimized Python UDFs by default; set to false to restore previous serialization behavior Not checked yet INSERT INTO ... REPLACE ON <columns> and INSERT INTO ... REPLACE USING <subquery> syntax supports upsert-style writes in SQL Not checked yet withSchemaEvolution() method in DataFrame writer API enables automatic schema evolution for compatible tables Not checked yet Dataset.zipWithIndex added to Scala API for pairing elements with index positions Not checked yet DataFrameGroupBy.cov and SeriesGroupBy.describe added to pandas API on Spark for covariance and descriptive statistics Not checked yet Plan download links for query plans in SVG, DOT, and TXT formats now available in Spark UI execution page Not checked yet TopN pushdown for fine-grained access control on dedicated compute pushes ORDER BY ... LIMIT operations to remote cluster Not checked yet
14 details in Everything
└──▷ BREAKING ON UPGRADE
! Clusters that set JNAME=zulu17-ca-amd64 must remove that environment variable because JDK 17 is removed in Databricks Runtime 19; only JDK 21 is supported! Approximately 90 standard runtime Python packages are removed (including plotly, seaborn, openai, langchain-core, huggingface_hub, psycopg2, pyodbc); workloads must install these explicitly via cluster library or init script! Setting pipelines.pipelineId table property manually results in an error as it is now reserved; remove explicit assignments from table DDL before upgrading! In standard access mode, restricted Spark configurations (including spark.driver.extraJavaOptions, spark.executor.extraJavaOptions, spark.jars, spark.files, spark.executorEnv.*, spark.kubernetes.*) can no longer be set; cluster creation or editing fails with an error
Use OAuth to authenticate Databricks connections in Power Platform
Jun 15, 2026
· release notes
Databricks adds OAuth authentication for Power Platform connections
Managed disaster recovery is now in Public Preview
Jun 12, 2026
· release notes
Databricks introduces Managed Disaster Recovery in Public Preview, replicating Unity Catalog metadata and workspace assets to a secondary region.
Auto time-to-live for automatic row deletion is generally available
Jun 12, 2026
· release notes
Auto-TTL TTL Time to live, a limit on how long data remains valid or how far a packet can travel. Caches use a time limit; IP packets use a hop limit. for automatic row deletion is now generally available for Unity Catalog managed Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. , Apache Iceberg Apache Iceberg An open table format for large analytic datasets, maintained by the Apache Software Foundation, that gives cyber tools reliable schema evolution, time-travel queries, and atomic commits on object storage or HDFS. , and streaming tables.
Converting a partitioned table to liquid clustering is generally available
Jun 12, 2026
· release notes
General availability of ALTER TABLE ... REPLACE PARTITIONED BY WITH CLUSTER BY for converting Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. tables to liquid clustering in Databricks Runtime 18.1+
Faster inference tables for custom model serving endpoints
Jun 12, 2026
· release notes
Adds configurable sampling and telemetry ingestion for custom model serving inference tables via AI Gateway UI.
Mission Critical add-on is now in Public Preview
Jun 12, 2026
· release notes
Databricks introduces the Mission Critical workspace add-on in Public Preview, bundling managed disaster recovery and Enhanced Security and Compliance (ESC) capabilities. Access is gated and enabled per workspace via the new Add-ons tab in the account console.
Email notifications for expiring service principal tokens (Beta)
Jun 12, 2026
· release notes
Databricks Beta adds automatic email alerts for service principal tokens expiring within seven days.
ABAC GRANT policies for models (Beta)
Jun 11, 2026
· release notes
Beta release of ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. GRANT policies to dynamically grant EXECUTE privilege on Unity Catalog models via governed tags
Adds ABAC GRANT policy support for the EXECUTE privilege on models attached at catalog or schema level. Not checked yet Extends attribute-based access control attribute-based access control Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. to system.ai foundation models and customer-registered MLflow models. Not checked yet
2 details in Everything
Agent Bricks Supervisor Agent now supports Unity Catalog volumes as subagent tools
Jun 11, 2026
· release notes
Supervisor Agent now supports Unity Catalog volumes as subagent tools
ai_extract is now generally available
Jun 11, 2026
· release notes
ai_extract function GA for schema-driven structured data extraction from text
GA release of the ai_extract function which extracts structured data from text and documents based on a user-provided schema. Not checked yet Supports nested objects and arrays within the extracted schema. Not checked yet
2 details in Everything
Endpoint telemetry for custom model serving endpoints is generally available
Jun 11, 2026
· release notes
Databricks adds generally available endpoint telemetry for custom model serving endpoints, persisting OTel OTel An open standard and set of tools for collecting traces, metrics and logs from applications. It lets services export observability data without tying instrumentation to one monitoring vendor. logs, traces, and metrics to Unity Catalog.
ai_classify is now generally available
Jun 11, 2026
· release notes
Databricks ai_classify function is GA with support for custom labels, descriptions, global instructions, and multi-label classification.
ai_classify function general availability allows classifying text according to custom labels, supporting label descriptions, global instructions, and multi-label classification. Not checked yet No-code UI for building, validating, and iterating on classification agents via the Classification menu. Not checked yet
2 details in Everything
Genie Code now uses models served through OpenAI on Databricks
Jun 11, 2026
· release notes
Genie Code now supports models served through OpenAI on Databricks alongside Azure and Anthropic
Schedule deferred policy enforcement for all-purpose compute
Jun 10, 2026
· release notes
Adds configurable deferred (on-next-restart) and immediate (restart-and-enforce) compute policy enforcement with cancel options.
Enforce on next restart schedules a configuration update to apply the next time the compute terminates or restarts, avoiding immediate interruption of running workloads Not checked yet Restart and enforce applies the configuration update right away Not checked yet Workspace admins can cancel a scheduled enforcement for a single compute resource or for an entire policy Not checked yet
3 details in Everything
Databricks Lakehouse Replay Beta automatically replays serverless workloads against upcoming Databricks Runtime releases to detect regressions before production.
Databricks Runtime 18 LTS adds SQL scripting, parameter markers, IDENTIFIER clause, vector math aggregates, and geospatial functions
CREATE OR REPLACE TEMP TABLE now supported for creating or replacing temporary tables in a single statement Not checked yet DESCRIBE FLOW command is now available to describe flow objects Not checked yet DESCRIBE HISTORY write option flags (isDynamicPartitionOverwrite, canOverwriteSchema, etc.) now appear in operationParameters for WRITE and REPLACE TABLE operations Not checked yet SET METADATA ON COLUMN and UNSET METADATA ON COLUMN SQL commands are now available for managing semantic metadata on Unity Catalog table columns Not checked yet SQL scripting feature is now generally available, supporting CURSOR variables for iterating over query result sets row by row Not checked yet
+21 more
show less
INSERT INTO ... BY NAME now supports the REPLACE WHERE clause Not checked yet Named (:param) and unnamed (?) parameter markers can now be used virtually anywhere a literal value of the appropriate type can be used, including DDL statements Not checked yet The IDENTIFIER clause reach has been expanded to nearly everywhere an identifier is permitted, allowing strings to be cast to SQL object names Not checked yet max_by and min_by aggregate functions now accept an optional third argument limit (up to 100,000) to return top-K or bottom-K values Not checked yet New vector math aggregate functions are available: vector_avg, vector_sum, vector_norm, and vector_normalize for dense vector columns Not checked yet Approximate distinct count using Datasketches Theta Sketch is supported via new theta_sketch_agg and related functions Not checked yet Approximate quantile computation using KLL Sketches is supported via new kll_sketch_agg_bigint and related functions Not checked yet New geospatial functions st_azimuth, st_boundary, st_closestpoint, and st_geogfromewkt are now available Not checked yet Apache Spark now supports native GeometryType and GeographyType data types in the PySpark and Java APIs Not checked yet Approximate top-K most frequent values can be returned using the new approx_top_k aggregate function Not checked yet Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. tables using pre-signed URL or cloud token sharing modes now support multi-statement transactions Not checked yet parse_timestamp SQL function parses timestamp strings using multiple patterns with Photon engine support for improved performance Not checked yet array_agg and collect_list aggregate functions now support IGNORE NULLS and RESPECT NULLS clauses Not checked yet PIVOT clause now supports aliases to rename pivot column expressions in the output Not checked yet Trigger.AvailableNow in Python streaming readers is now available, enabling one-time batch processing of all available data Not checked yet The state repartition API for TransformWithState streaming operators is now available in PySpark Not checked yet DataFrame.applyInPandas now accepts a function with an Iterator[pandas.DataFrame] signature to reduce overhead when processing large groups Not checked yet Spark Session.emptyDataFrame is now available as a convenience API for creating an empty DataFrame Not checked yet TIMESTAMP WITH LOCAL TIME ZONE is now supported as a SQL syntax alias for the TIMESTAMP_LTZ type Not checked yet The to_json function now accepts a sortKeys option to produce JSON output with keys in sorted order Not checked yet spark.catalog.* API methods now have feature parity with their equivalent DDL commands for catalog, schema, and table operations Not checked yet
26 details in Everything
Databricks Genie app in Microsoft Teams (Beta)
Jun 10, 2026
· release notes
Databricks Genie app in Microsoft Teams Beta allows users to ask data questions within Teams.
Adds the Databricks Genie app to Microsoft Teams for beta testing Not checked yet Enables chat with Genie or a pinned Genie Agent via direct message or @mention Not checked yet
2 details in Everything
Automatic identity management for Okta (Public Preview)
Jun 9, 2026
· release notes
New public preview feature allows syncing Okta users and groups into Databricks without SCIM SCIM A standard for creating, updating and removing user accounts and groups across applications. It helps an identity provider keep access in sync when people join, change roles or leave. provisioning.
Adds support for automatic identity management with Okta as the source of record Not checked yet Enables searching for Okta users and groups in identity federated workspaces Not checked yet
2 details in Everything
Databricks Genie app for Slack (Public Preview)
Jun 9, 2026
· release notes
Databricks Genie app for Slack enters Public Preview, enabling data queries via DM or channel mentions.
Adds steps integration for the Databricks Genie app in Slack Public preview , allowing users to get answers by sending a direct message or mentioning @Genie. Not checked yet Enables pinning specific Genie Agents to channels or group DMs, where mentions route to the pinned agent. Not checked yet
2 details in Everything
The update_flow API for Spark Declarative Pipelines on Lakeflow is generally available
Jun 9, 2026
· release notes
@dp.update_flow decorator now GA for Spark Declarative Pipelines on Lakeflow, enabling update-output mode
Anthropic Claude Fable 5 now available as a Databricks-hosted model
Jun 9, 2026
· release notes
Anthropic Claude Fable 5 added to Databricks Model Serving
External lineage is now generally available
Jun 8, 2026
· release notes
External lineage generally available in Unity Catalog; Lakeflow Connect now auto-records source lineage.
Adds External Lineage capability to Unity Catalog, allowing registration of non-Databricks assets like Salesforce, MySQL, Tableau, or Power BI into a single graph Not checked yet Enables automatic source lineage recording from source tables to destination tables in Unity Catalog for Lakeflow Connect managed ingestion pipelines Not checked yet
2 details in Everything
Inbound Private Link for performance-intensive services is now generally available
Jun 8, 2026
· release notes
Inbound Private Link Private Link An AWS networking feature that exposes a service over a private endpoint inside a VPC, so traffic never traverses the public internet, giving cyber tools a way to integrate with cloud services without opening public routes. for Lakebase Autoscaling and Zerobus Ingest is GA.
Provides a configuration guide for setting up inbound PrivateLink for these services Not checked yet
1 detail in Everything
Monday.com managed ingestion connector (Beta)
Jun 8, 2026
· release notes
Databricks adds a Beta Monday.com managed ingestion connector to Lakeflow Connect for ingesting work management data.
Adds Lakeflow Connect integration with Monday.com, enabling ingestion of boards, users, teams, and activity logs. Not checked yet
1 detail in Everything
Zoho Books managed ingestion connector (Beta)
Jun 8, 2026
· release notes
Databricks adds Beta Zoho Books managed ingestion connector to Lakeflow Connect for accounting data.
AI Search supports selected text columns, sorting, and aggregations (Beta)
Jun 8, 2026
· release notes
AI Search beta adds query-time column selection, sorting, and aggregations
Adds selected_text_columns to limit keyword matching in full-text and hybrid queries Not checked yet Adds result sorting by indexed columns in search queries Not checked yet Adds aggregate counts for values and ranges in search results Not checked yet
3 details in Everything
AWS Graviton instances are now supported with the compliance security profile
Jun 5, 2026
· release notes
Databricks now supports AWS Graviton instances for compliance security profiles in commercial and GovCloud regions.
Managed SharePoint connector adds structured file ingestion and file metadata support (Beta)
Jun 5, 2026
· release notes
Managed SharePoint connector now supports structured ingestion (CSV/JSON/XML/Excel/Parquet/Avro/ORC), metadata, filters, and schema controls in Lakeflow Connect
Adds structured file ingestion for CSV, JSON, XML, Excel, Parquet, Avro, and ORC formats via the unified managed file source API in the Managed SharePoint connector (Lakeflow Connect). Not checked yet Adds file metadata ingestion capability to the Managed SharePoint connector. Not checked yet
2 details in Everything
External data access for pipeline streaming tables and materialized views (Public Preview)
Jun 5, 2026
· release notes
External Delta and Iceberg clients can now read pipeline streaming tables and materialized views via Unity Catalog and Iceberg REST APIs REST APIs A convention for building HTTP-based interfaces where resources are addressed by URL and manipulated with standard verbs like GET, POST, and DELETE, giving cyber tools a predictable way to integrate with external services. .
Write data from Microsoft Excel back to Databricks
Jun 4, 2026
· release notes
New Databricks Excel Add-in feature to write data from Microsoft Excel directly to Unity Catalog tables
Genie moves to a pay-as-you-go model with 150 free DBUs/month, while new budget controls allow admins to manage costs.
Agent Bricks Supervisor Agent now supports web search as a built-in tool
Jun 3, 2026
· release notes
Databricks Supervisor Agent now includes built-in web search capability for answering questions with current public web info.
Table update triggers on Delta Sharing and system tables (Beta)
Jun 3, 2026
· release notes
Databricks adds Lakeflow Job triggers for Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. updates and system table changes in Beta
Adds Lakeflow Jobs trigger support for data shared via Delta Sharing (Databricks-to-Databricks only) and system tables. Not checked yet Enables triggering jobs when a provider updates a shared table or view, or when new operational data lands in a system table. Not checked yet
2 details in Everything
Lakeflow Connect now supports RabbitMQ classic queues in beta to facilitate message streaming into Unity Catalog.
Column popularity is now available in the Catalog Explorer
Jun 1, 2026
· release notes
Catalog Explorer UI now shows column popularity metrics in the table Overview tab.
Query-based connectors for Lakeflow Connect are now generally available
May 29, 2026
· release notes
Databricks makes Lakeflow Connect query-based connectors GA for Oracle, Teradata, SQL Server, MySQL, MariaDB, PostgreSQL, and Lakehouse Federation Lakehouse Federation A data architecture pattern that lets a single query engine read from multiple external data sources—databases, data lakes, warehouses—without moving or copying the data, reducing pipeline complexity for tools that need broad data access. sources.
Adds Lakeflow Connect query-based connectors as a general capability for ingesting data from databases using cursor columns without CDC or an ingestion gateway. Not checked yet Adds support for Oracle, Teradata, SQL Server, MySQL, MariaDB, and PostgreSQL as source systems for Lakeflow Connect query-based connectors. Not checked yet Adds support for all Lakehouse Federation data sources using foreign catalog ingestion in Lakeflow Connect query-based connectors. Not checked yet
3 details in Everything
Lakeflow Designer updates for May 29, 2026
May 29, 2026
· release notes
Lakeflow Designer adds bidirectional AI descriptions, N-way Combine, custom SQL joins, and multi-modal multi-modal Able to work with more than one type of input or output, such as text, images and audio. A multimodal model can combine information across those formats. outputs.
Adds bidirectional AI-generated descriptions: editing an operator's description reconfigures the operator Not checked yet Adds N-way Combine support to the Combine operator for any number of input tables Not checked yet Adds custom join conditions via UI arrow click allowing manual SQL entry or edited AI description Not checked yet Enables multi-modal output rendering (plots, HTML, images) directly in the preview pane Not checked yet Makes the output panel configurable with resizable input/output views Not checked yet
+2 more
show less
Adds inline parameter examples for SQL and Python operators Not checked yet Displays a one-line Genie Code change summary above the input box Not checked yet
7 details in Everything
Databricks Apps horizontal scaling (Beta)
May 29, 2026
· release notes
Databricks Apps Beta adds horizontal scaling with session affinity and opt-out of pre-installed Python libraries.
Enables running a Databricks app across multiple instances behind a single URL for higher availability and zero-downtime deployments Not checked yet Adds session_affinity routing to keep per-user data on the same instance Not checked yet Allows opting out of pre-installed Python libraries to run on a clean base OS image when enabling horizontal scaling Not checked yet
3 details in Everything
Anthropic Claude Opus 4.8 now available as a Databricks-hosted model
May 28, 2026
· release notes
Anthropic Claude Opus 4.8 available as a Databricks-hosted model
Cross-engine attribute-based access controls (Beta)
May 28, 2026
· release notes
Cross-engine ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. allows external engines to read Unity Catalog managed tables with server-side row and column security
Google Drive managed ingestion connector (Beta)
May 27, 2026
· release notes
Databricks adds Beta Google Drive managed ingestion connector in Lakeflow Connect for structured and unstructured file ingest
Real-time mode in Spark Declarative Pipelines on Lakeflow , and the update_flow API are now available (Public Preview)
May 27, 2026
· release notes
Databricks Lakeflow now supports real-time mode via the new update_flow decorator and API in Public Preview.
Adds the update_flow decorator for Spark Declarative Pipelines on Lakeflow to enable real-time mode with end-to-end latency as low as five milliseconds Not checked yet Enables stateful aggregations without requiring a watermark in the new real-time mode Not checked yet
2 details in Everything
Refresh policies and EXPLAIN CREATE MATERIALIZED VIEW for materialized views are now generally available
May 27, 2026
· release notes
Refresh policies and EXPLAIN CREATE MATERIALIZED VIEW are GA for materialized views in Databricks
EXPLAIN CREATE MATERIALIZED VIEW allows checking incrementalization capability before defining a view. Not checked yet Refresh policies control when Databricks performs incremental or full refresh for materialized views. Not checked yet
2 details in Everything
Workspace admin setting for serverless notebook execution timeout
May 27, 2026
· release notes
Workspace admins can now set a default execution timeout for serverless notebooks via admin settings.
Automatic identity management with Microsoft Entra ID is now GA
May 26, 2026
· release notes
Databricks enables automatic syncing of Microsoft Entra ID Entra ID Microsoft's cloud-based identity and access management service, formerly Azure Active Directory, used by cyber tools to authenticate users and enforce access policies in Microsoft 365 and Azure environments. identities to workspaces without SCIM SCIM A standard for creating, updating and removing user accounts and groups across applications. It helps an identity provider keep access in sync when people join, change roles or leave. .
Genie Code Agent mode is now available by default for workspaces with the compliance security profile enabled
May 25, 2026
· release notes
Genie Code Agent mode is now default for compliance workspaces
Account access denylist for automatic identity management (Public Preview)
May 22, 2026
· release notes
Databricks introduces an account access denylist for automatic identity management in public preview on AWS and GCP.
Account admins can now add users, groups, or service principals to a denylist to block their access. Not checked yet Denylist membership is transitive — denying a group also denies all of its members. Not checked yet
2 details in Everything
Databricks CLI reaches GA with OS-native secure storage for U2M tokens
The databricks CLI is now generally available (GA). Not checked yet Updates the Databricks CLI to store user-to-machine (U2M) tokens in OS-native secure storage. Not checked yet
2 details in Everything
Instance events and instance pools system tables are now available (Public Preview)
May 21, 2026
· release notes
Public Preview: system.compute.instance_events and system.compute.instance_pools system tables for tracking compute state and history.
Adds system.compute.instance_events system table to track state transitions of classic compute instances Not checked yet Adds system.compute.instance_pools system table to access the full history of instance pool configurations across your account Not checked yet
2 details in Everything
Unity Catalog managed Apache Iceberg tables, foreign Apache Iceberg tables, and Apache Iceberg v3 features are generally available
May 21, 2026
· release notes
Unity Catalog now generally available for managed and foreign Iceberg tables, plus Iceberg v3 features like deletion vectors.
Supports creating and managing Apache Iceberg Apache Iceberg An open table format for large analytic datasets, maintained by the Apache Software Foundation, that gives cyber tools reliable schema evolution, time-travel queries, and atomic commits on object storage or HDFS. tables within Unity Catalog using Databricks Runtime, Databricks SQL, or external Iceberg-compatible engines Not checked yet Enables reading Iceberg tables managed by external catalogs (AWS Glue, Hive metastore Hive metastore A central metadata repository, part of the Apache Hive project, that stores schema and location information for distributed data sets so query engines can discover and read structured data without hardcoding paths. , Snowflake Horizon Catalog) through Lakehouse Federation Lakehouse Federation A data architecture pattern that lets a single query engine read from multiple external data sources—databases, data lakes, warehouses—without moving or copying the data, reducing pipeline complexity for tools that need broad data access. Not checked yet Adds support for Iceberg v3 features including deletion vectors, the VARIANT data type, and row lineage on managed Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. , managed Iceberg, and foreign Iceberg tables Not checked yet
3 details in Everything
Workspace sidebar navigation updates
May 21, 2026
· release notes
Workspace sidebar reorganizes sections into Analytics and renames SQL section to Lakehouse.
Groups Dashboards, Genie Agents, and Alerts under a new Analytics section in the workspace sidebar. Not checked yet Renames the existing SQL section to Lakehouse , containing SQL Editor, Queries, Query History, and SQL Warehouses. Not checked yet Updates the app switcher label from Lakehouse to Analytics and AI . Not checked yet
3 details in Everything
Authenticate as a service principal using OAuth U2M (Beta)
May 20, 2026
· release notes
Beta support for authenticating as a service principal via OAuth U2M OAuth U2M OAuth flow in which a human user delegates scoped access to a service on their behalf, letting cyber tools act with the user's permissions without handling raw credentials. flow
Adds OAuth U2M flow to authenticate as a service principal's identity instead of the user account. Not checked yet
1 detail in Everything
Serve custom LLMs with Custom Model Serving (Beta)
May 20, 2026
· release notes
Databricks adds Custom Model Serving Beta to serve custom and fine-tuned LLMs LLMs A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. using vLLM.
Enables serving of multimodal multimodal Able to work with more than one type of input or output, such as text, images and audio. A multimodal model can combine information across those formats. models via the new Custom Model Serving capability Not checked yet Supports PEFT PEFT Adapting a pretrained model by training only a small subset of its parameters or small added modules. Methods such as LoRA reduce the memory and storage needed for fine-tuning. recipes for model fine-tuning through Custom Model Serving Beta Not checked yet
2 details in Everything
Vector Search storage-optimized endpoints are GA
May 20, 2026
· release notes
Vector Search storage-optimized endpoints are generally available, offering higher capacity and significantly faster indexing.
Storage-optimized endpoints support over one billion vectors at dimension 768. Not checked yet Indexing is 10-20x faster than standard endpoints. Not checked yet Pricing is optimized for large vector counts. Not checked yet
3 details in Everything
Google Gemini 3.5 Flash now available as a Databricks-hosted model
May 19, 2026
· release notes
Google Gemini 3.5 Flash added as a Databricks-hosted model accessible via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves.
Adds Google Gemini 3.5 Flash to Model Serving as a Databricks-hosted model. Not checked yet Enables access to Google Gemini 3.5 Flash through Foundation Model APIs. Not checked yet
2 details in Everything
Databricks Container Services for standard compute is now available (Beta)
May 19, 2026
· release notes
Beta: Databricks Container Services for standard compute allows custom Docker images on Databricks Runtime 18.3+
Enables specification of a custom Docker image when creating standard compute environments on Databricks Runtime 18.3 or later Not checked yet
1 detail in Everything
Customer-managed keys now support MLflow managed evaluation features
May 19, 2026
· release notes
Customer-managed keys (CMK) now support MLflow managed evaluation features including production monitoring, the Review App, and evaluation datasets.
Enables production monitoring of AI apps with MLflow 3 scorers in workspaces encrypted with customer-managed keys. Not checked yet
1 detail in Everything
Databricks Add-in for Excel support for all compliance security profile standards
May 19, 2026
· release notes
Databricks Add-in for Excel now supports all compliance security profile standards.
Lakeflow Designer now enabled by default with AI operator search, configurable sample size, Git integration, and user-defined operators.
└──▷ HOW TO FIND IT
Export your current visual data prep flow for version control or sharing.
📍 File › Export
Enables Lakeflow Designer by default for all free edition, premium, and enterprise tier workspaces Not checked yet Adds AI operator search to the operator panel, suggesting operators based on intent (e.g., typing average by month surfaces Aggregate operator ) Not checked yet Adds Configurable sample size allowing users to set the number of sample rows for each operator, with limits remembered per operator Not checked yet Enables exporting, cloning, and importing visual data prep files via File > Export, File > Clone, and storage in Git folders Not checked yet Introduces User-defined operators Public preview supporting uc-udf, uc-udtf, and python-run-function types to extend Lakeflow Designer with custom business logic Not checked yet
5 details in Everything
Agent Bricks Supervisor Agent now supports nesting supervisor agents as subagent tools
May 19, 2026
· release notes
Databricks Agent Bricks Supervisor Agent now supports nesting supervisor agents as subagent tools.
Adds the ability to nest another Supervisor Agent as a subagent tool for nested multi-agent orchestration. Not checked yet
1 detail in Everything
Unified runs list is generally available
May 18, 2026
· release notes
Databricks releases unified runs list with real-time filtering by status, time, user, run ID, error codes, and trend visualization.
Adds a unified Runs list view to see all job and pipeline runs in one place. Not checked yet Provides a visualization and summary showing the current top five error codes for trend spotting. Not checked yet
2 details in Everything
Agent Bricks Supervisor Agent now supports vector search indexes as subagent tools
May 15, 2026
· release notes
Supervisor Agent now supports vector search indexes as subagent tools.
Genie Code chat threads can now be shared with users, groups, or service principals.
Agent Bricks Supervisor Agent now supports Unity Catalog tables as subagent tools
May 15, 2026
· release notes
Supervisor Agent now supports Unity Catalog tables as subagent tools
View run history for scheduled refreshes in Google Sheets
May 14, 2026
· release notes
Enhanced Google Sheets integration now exposes granular metadata for scheduled data refresh operations.
SQL alert task in Lakeflow Jobs is now in Public Preview
May 14, 2026
· release notes
SQL alert task in Lakeflow Jobs is now in Public Preview
Databricks SQL alerts are now generally available
May 14, 2026
· release notes
Databricks SQL alerts are now generally available for monitoring KPIs and data quality via scheduled queries.
Configure an additional recipient for workspace operational emails
May 14, 2026
· release notes
Adds ability to configure an additional recipient for workspace operational emails in Databricks.
Workspace admins can now add an additional email recipient to receive operational emails about a workspace. Not checked yet
1 detail in Everything
HubSpot connector in Lakeflow Connect is now GA for ingesting Marketing Hub data.
HIPAA support for the Databricks Add-in for Excel
May 11, 2026
· release notes
HIPAA HIPAA A US federal law enacted in 1996 that sets privacy and security requirements for protected health information, giving cyber tools a compliance baseline to enforce or audit against. compliance support added to Databricks Add-in for Excel in Public Preview
Databricks Add-in for Excel now supports workspaces with HIPAA compliance across all supported regions. Not checked yet
1 detail in Everything
App telemetry for Databricks Apps is now in Public Preview
May 11, 2026
· release notes
Public Preview of App Telemetry for Databricks Apps allowing traces, logs, and metrics to be persisted to Unity Catalog via OpenTelemetry OpenTelemetry An open standard and set of tools for collecting traces, metrics and logs from applications. It lets services export observability data without tying instrumentation to one monitoring vendor. .
Adds the ability to collect traces, logs, and metrics from Databricks Apps using the OpenTelemetry protocol. Not checked yet Enables persistence of collected telemetry data into Unity Catalog tables. Not checked yet
2 details in Everything
Catalog commits are now generally available
May 8, 2026
· release notes
Databricks makes catalog commits GA for Unity Catalog managed tables, enabling multi-statement transactions and broadening product support.
Makes catalog commits generally available for Unity Catalog managed Delta tables to expand interoperability and governance capabilities. Not checked yet Enables new features such as multi-statement, multi-table transactions when catalog commits are enabled on a Unity Catalog managed Delta table. Not checked yet
2 details in Everything
Faster package installs with %uv pip in serverless notebooks
May 7, 2026
· release notes
Adds %uv pip magic command for faster Python package installs in serverless notebooks
Standalone pipelines on serverless general compute (Beta)
May 7, 2026
· release notes
Beta allows creating standalone pipelines (SQL/Python) from notebooks on serverless general compute
Standalone pipelines (renamed from pipelines on Databricks SQL), including materialized views and streaming tables, can now be created from a notebook running on serverless general compute Not checked yet
1 detail in Everything
Graviton instance types are available for the compliance security profile
May 7, 2026
· release notes
Graviton (Arm64) instance types are now available for Databricks workspaces using the compliance security profile with FedRAMP Moderate FedRAMP Moderate A US government authorization tier requiring cloud services to meet a defined set of NIST 800-53 controls before federal agencies can use them, giving cyber tools a pre-vetted compliance baseline for government contracts. , IRAP, or CCCS Medium standards.
Databricks adds Automatic Identity Management for syncing users/groups from Microsoft Entra ID Entra ID Microsoft's cloud-based identity and access management service, formerly Azure Active Directory, used by cyber tools to authenticate users and enforce access policies in Microsoft 365 and Azure environments. and Okta
Introduces Automatic identity management feature to sync users and groups from external identity providers identity providers A service that authenticates users and supplies identity information to other applications. It is the sign-in authority behind many single sign-on systems. Not checked yet Adds Public Preview support for Microsoft Entra ID, including service principals and nested groups Not checked yet Adds Private Preview support for Okta integration Not checked yet
3 details in Everything
Alibaba Cloud Qwen3.5 122B A10B now available as a Databricks-hosted model in Public Preview
May 5, 2026
· release notes
Alibaba Cloud Qwen3.5 122B A10B model now available via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. in Public Preview
Smartsheet managed ingestion connector (Beta)
May 5, 2026
· release notes
Databricks introduces a Beta Smartsheet connector in Lakeflow Connect, enabling users to ingest data from Smartsheet sheets directly into their Databricks environment.
Outlook managed ingestion connector (Beta)
May 5, 2026
· release notes
Beta release of the Outlook managed ingestion connector in Lakeflow Connect for Databricks
Agent Bricks Supervisor Agent now supports custom MCP servers and custom agents on Databricks Apps
Apr 30, 2026
· release notes
Databricks Supervisor Agent now allows custom MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. servers and custom agents as subagents.
MLflow trace storage in Unity Catalog is now in Public Preview
Apr 30, 2026
· release notes
MLflow trace storage in Unity Catalog enters Public Preview with OpenTelemetry OpenTelemetry An open standard and set of tools for collecting traces, metrics and logs from applications. It lets services export observability data without tying instrumentation to one monitoring vendor. format support
Disabling tasks in Lakeflow Jobs is now GA
Apr 30, 2026
· release notes
Lakeflow Jobs GA support for disabling tasks at runtime without removal
GA: Disable a task in a Lakeflow job to skip it at run time while keeping configuration and history. Not checked yet
1 detail in Everything
Attribute-based access control (ABAC) is now generally available
Apr 28, 2026
· release notes
Databricks ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. GA adds session-identity-based row filters and column masks on Unity Catalog securables
ABAC is now generally available, allowing governed tags to dynamically enforce row filter and column mask policies on tables, materialized views, and streaming tables Not checked yet Row filters and column masks applied via ABAC are now evaluated using the session user's identity rather than the view or function owner's identity Not checked yet
2 details in Everything
└──▷ BREAKING ON UPGRADE
! ABAC policies attached to securables now use the session user's identity for evaluation instead of the view or function owner's identity.
Cloud token access for open Delta Sharing sharing (GA)
Apr 28, 2026
· release notes
GA of cloud token access for open Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. allows direct cloud storage reads via directory-based access mode for eligible tables.
Add SET TAG/UNSET TAG support for Unity Catalog functions on Databricks Runtime 18.1+
SET TAG and UNSET TAG commands now apply to Unity Catalog securable object type function Not checked yet
1 detail in Everything
Spark Declarative Pipelines on Lakeflow updates are retained for 60 days
Apr 28, 2026
· release notes
Spark Declarative Pipelines update history retention extended from 30 to 60 days in UI and REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services.
5X-Large SQL warehouse size (Public Preview)
Apr 28, 2026
· release notes
5X-Large SQL warehouse size (512 workers) now available in Public Preview.
Adds 5X-Large cluster size option for serverless and pro SQL warehouses, providing 512 workers Not checked yet
1 detail in Everything
Connect Databricks to on-premises resources using an SSH reverse tunnel
Apr 28, 2026
· release notes
Adds SSH reverse tunnel support for Databricks classic and serverless compute to connect to on-premises resources.
Recently Resolved Incidents section in Data Quality Monitoring (Beta)
Apr 27, 2026
· release notes
Data Quality Monitoring dashboard adds Recently Resolved Incidents section to track self-healing tables.
Manage Supervisor Agent programmatically with the Databricks SDK (Beta)
Apr 24, 2026
· release notes
Databricks SDK Beta now supports creating and managing Supervisor Agents programmatically.
Adds the ability to create and manage Supervisor Agent instances and their tools via the Databricks SDK for Python Not checked yet
1 detail in Everything
Lakeflow Designer is now in Public Preview
Apr 22, 2026
· release notes
Lakeflow Designer enters public preview, adding a visual drag-and-drop canvas for building data transformation workflows.
Language selector for AI-generated comments in Catalog Explorer
Apr 22, 2026
· release notes
Adds a language selector in Catalog Explorer UI to set the default language for AI-generated comments
CPU_MEDIUM and CPU_LARGE workload types for Model Serving (Beta)
Apr 21, 2026
· release notes
Adds CPU_MEDIUM and CPU_LARGE workload types for Model Serving to increase memory per worker.
Enables CPU_MEDIUM workload type for Model Serving Beta to trade concurrency for more memory per worker Not checked yet Enables CPU_LARGE workload type for Model Serving Beta to trade concurrency for more memory per worker Not checked yet
2 details in Everything
Altering a catalog or schema's managed storage location is now GA
Apr 17, 2026
· release notes
Altering a catalog or schema's managed storage location is now generally available.
Customer-managed keys now support model serving
Apr 16, 2026
· release notes
Databricks adds customer-managed key support for model serving container images and artifacts.
ai_parse_document is now generally available
Apr 16, 2026
· release notes
ai_parse_document function is generally available for parsing structured content from unstructured documents.
Anthropic Claude Opus 4.7 now available as a Databricks-hosted model
Apr 16, 2026
· release notes
Mosaic AI Model Serving now supports Anthropic Claude Opus 4.7 as a Databricks-hosted model with API access
Adds Anthropic Claude Opus 4.7 to Mosaic AI Model Serving as a new Databricks-hosted foundation model Not checked yet Enables pay-per-token access via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. for the new model Not checked yet Supports reasoning and vision query capabilities through the Foundation Model APIs Not checked yet
3 details in Everything
Query-based connectors for Lakeflow Connect (Public Preview)
Apr 15, 2026
· release notes
Lakeflow Connect Public Preview adds query-based connectors for Oracle, Teradata, SQL Server, MySQL, MariaDB, PostgreSQL and Lakehouse Federation Lakehouse Federation A data architecture pattern that lets a single query engine read from multiple external data sources—databases, data lakes, warehouses—without moving or copying the data, reducing pipeline complexity for tools that need broad data access. sources.
Adds query-based connectors in Lakeflow Connect Public preview to ingest data by querying source directly via a cursor column without CDC or ingestion gateway Not checked yet Supports Oracle, Teradata, SQL Server, MySQL, MariaDB, PostgreSQL, and all Lakehouse Federation data sources using foreign catalog ingestion Not checked yet
2 details in Everything
AI Gateway now governs MCP servers (Beta)
Apr 15, 2026
· release notes
AI Gateway Beta now governs MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. servers with access control, usage monitoring, and audit capabilities.
Expands AI Gateway's scope to include MCP servers alongside LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. endpoints and coding agents Not checked yet
1 detail in Everything
Updated AWS tag character restrictions
Apr 15, 2026
· release notes
AWS tag character restrictions tightened for Databricks custom usage tags
Enforces that Databricks custom usage tag keys and values cannot contain spaces or forward slashes (/) Not checked yet Prohibits tag keys consisting solely of a single period (.), two periods (..), or the string _index Not checked yet
2 details in Everything
└──▷ BREAKING ON UPGRADE
! Tag keys and values can no longer contain spaces or forward slashes (/)! Tag keys consisting only of '.', '..', or _index are now invalid
Create and write to Unity Catalog tables from external Delta clients (Beta)
Apr 14, 2026
· release notes
Beta: Unity Catalog REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. now supports create and write access for external Delta clients.
The Unity REST API now accepts create and write requests from external Delta clients such as Apache Spark. Not checked yet Full read, write, and create operations are supported for Unity Catalog external Delta tables. Not checked yet Beta support for creating and writing to Unity Catalog managed Delta tables requires enabling catalog-managed commits. Not checked yet
3 details in Everything
SQL reference documentation for managing governed tags
Apr 13, 2026
· release notes
Databricks adds CREATE GOVERNED TAG and other new SQL statements for managing governed tags
└──▷ TRY IT
List all currently defined governed tags in the workspace.
copy
$ SHOW GOVERNED TAGS
Adds CREATE GOVERNED TAG SQL statement to define new governed tags. Not checked yet Adds ALTER GOVERNED TAG SQL statement to modify existing governed tags. Not checked yet Adds DROP GOVERNED TAG SQL statement to remove governed tags. Not checked yet Adds DESCRIBE GOVERNED TAG SQL statement to view tag details. Not checked yet Adds SHOW GOVERNED TAGS SQL statement to list all governed tags. Not checked yet
5 details in Everything
New integration enabling Lovable-hosted apps to query Databricks lakehouse data via REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. and OAuth.
Adds support for connecting Lovable-hosted apps to Databricks workspaces using the REST API and OAuth machine-to-machine authentication. Not checked yet
1 detail in Everything
Column-level SAP governance tags sync into Unity Catalog
Apr 13, 2026
· release notes
Column-level SAP governance tags from BDC sync into Unity Catalog for ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. .
Supports syncing fieldSemantics, isPotentiallyPersonal, and isPotentiallySensitive tags from SAP Business Data Cloud to Unity Catalog columns. Not checked yet Enables using synced SAP governance tags in Unity Catalog attribute-based access control attribute-based access control Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. (ABAC) policies. Not checked yet
2 details in Everything
New cascade field for deleting Unity Catalog pipelines (Beta)
Apr 9, 2026
· release notes
Adds cascade field to the Unity Catalog Pipelines DELETE API to control dependent object removal
Pipelines DELETE API now accepts a cascade query parameter; default is true, setting it to false retains associated materialized views, streaming tables, and views as inactive after pipeline deletion. Not checked yet
1 detail in Everything
Share foreign Iceberg tables with Delta Sharing (Public Preview)
Apr 9, 2026
· release notes
Databricks enables sharing of foreign Iceberg tables via Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. in public preview
Allows providers to add foreign Iceberg tables federated from external catalogs into a Delta Sharing share for read-only recipient access Not checked yet
1 detail in Everything
Databricks introduces ai_prep_search Beta SQL function to create search-ready chunks from parsed documents.
Adds the ai_prep_search SQL function Beta for generating vector-search-optimized chunks from ai_parse_document output Not checked yet
1 detail in Everything
Scoped personal access tokens are now generally available
Apr 8, 2026
· release notes
Databricks GA release for scoped personal access tokens personal access tokens A personal access token, a credential used by scripts and integrations instead of a user's password. Its permissions and expiry depend on the service issuing it. allows locking down token permissions to specific API operations.
Git-backed app deployments are now generally available
Apr 7, 2026
· release notes
Git-backed app deployments are now GA: deploy Databricks apps directly from Git repos via new workspace-level configuration.
Enables direct deployment of Databricks apps from Git repositories by configuring a Git reference and source code path Not checked yet Adds ability to enforce Git-only deployments across a Databricks workspace Not checked yet
2 details in Everything
Workspace skills for Genie Code are now available
Apr 6, 2026
· release notes
Workspace admins can now create and share agent skills for Genie Code across the workspace.
SAP semantic metadata syncs automatically into Unity Catalog
Apr 6, 2026
· release notes
Automatic sync of SAP BDC semantic metadata (comments, keys, tags) into Unity Catalog for ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. policies
Databricks Marketplace listing creation wizard
Apr 6, 2026
· release notes
Databricks Marketplace provider console adds a step-by-step wizard for creating listings.
The Marketplace provider console now features a step-by-step listing creation wizard that guides you through setting up general information, data assets, attributes, and details on separate pages. Not checked yet
1 detail in Everything
Governed tags are generally available
Apr 2, 2026
· release notes
Governed tags are generally available for Unity Catalog and workspace objects.
Customer-managed keys for Unity Catalog are generally available
Apr 1, 2026
· release notes
Customer-managed keys (CMK) for Unity Catalog are generally available.
Adds support for using cloud KMS KMS A managed service that creates, stores, rotates and controls access to encryption keys, usually backed by HSMs. Applications ask it to encrypt or decrypt rather than holding keys themselves. keys to encrypt data in Unity Catalog catalogs backed by default storage. Not checked yet
1 detail in Everything
Databricks Documentation site table of contents tabs
Mar 31, 2026
· release notes
Databricks docs site now offers tabbed navigation for usage guides, getting started, dev tools, partners, reference, resources, and release notes.
Agent Bricks Knowledge Assistant and Supervisor Agent are now available in additional regions
Mar 31, 2026
· release notes
Agent Bricks Knowledge Assistant and Supervisor Agent now available in ap-south-1 (Mumbai) AWS region
ai_classify and ai_extract now support v2 signatures (Public Preview)
Mar 27, 2026
· release notes
ai_classify and ai_extract AI Functions support v2 signatures
ai_classify function supports new v2 signature with improved quality and performance. Not checked yet ai_extract function supports new v2 signature with greater flexibility. Not checked yet New native composability between ai_classify, ai_extract, and ai_parse_document for end-to-end document processing workflows. Not checked yet
3 details in Everything
Classification and Information Extraction are in Public Preview
Mar 27, 2026
· release notes
New public preview UI for document classification and information extraction using ai_classify and ai_extract v2 signatures.
Adds a visual UI for classifying documents into predefined categories with label configuration and scale run capabilities Not checked yet Adds a visual UI for extracting structured data from documents, tables, PDFs, and images using defined extraction schemas Not checked yet
2 details in Everything
Auto Loader supports automatic type widening (Public Preview)
Mar 25, 2026
· release notes
Databricks Auto Loader adds addNewColumnsWithTypeWidening schema evolution mode to automatically widen data types during ingestion.
Adds the addNewColumnsWithTypeWidening schema evolution mode to Auto Loader, enabling automatic widening of data types (e.g., int to long, float to double) without requiring a data rewrite. Not checked yet Extends automatic type widening support in Auto Loader to all file formats with schema evolution capabilities, including text formats (JSON, CSV, XML) and binary formats (Avro, Parquet). Not checked yet
2 details in Everything
Databricks Runtime 16.4 LTS — Auto Loader supports automatic type widening (Public Preview)
Mar 25, 2026
· release notes
Auto Loader adds addNewColumnsWithTypeWidening schema evolution mode for automatic type widening Public preview
Adds addNewColumnsWithTypeWidening schema evolution mode to Auto Loader, automatically widening data types like int to long or float to double without rewriting data. Not checked yet Enables automatic type widening for text formats (JSON, CSV, XML) and binary formats (Avro, Parquet) in Auto Loader schema evolution. Not checked yet
2 details in Everything
New fields in lineage system table entity metadata
Mar 25, 2026
· release notes
Databricks adds new metadata fields to lineage system tables.
Supervisor Agent is now available by default for Enhanced Security and Compliance customers
Mar 25, 2026
· release notes
Supervisor Agent now available by default for Enhanced Security and Compliance customers on specific compliance profiles
Dynamic parameters in the new SQL editor
Mar 24, 2026
· release notes
New SQL editor adds Dynamic dropdown widget for named parameter markers
Databricks AI Runtime single-node GPU GPU A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. tasks enter Public Preview
Adds AI Runtime for single-node deep learning workloads with GPU support to Databricks serverless compute. Not checked yet
1 detail in Everything
Create alerts for anomaly detection in the Data Quality Monitoring UI (Beta)
Mar 19, 2026
· release notes
New capability to create and view alerts for anomaly detection directly in the Data Quality Monitoring UI Beta
Databricks JDBC driver 2.8.0 is available
Mar 18, 2026
· release notes
Databricks JDBC JDBC A Java standard API for connecting applications to relational databases, maintained as part of the Java SE platform, letting tools query any compliant database through a single consistent interface. driver 2.8.0 sets EnableTelemetry to true by default and removes Log4j dependency.
Databricks usage dashboard is now generally available
Mar 18, 2026
· release notes
The Databricks usage dashboard is now GA, allowing admins to import pre-built dashboards.
Search in metric views now returns correct results across paginated results and supports filtering by display name.
Health indicators for tables in Catalog Explorer (Public Preview)
Mar 13, 2026
· release notes
Public Preview introduces health indicators for tables in Catalog Explorer, displaying freshness and completeness status via anomaly detection.
5X-Large serverless and pro SQL warehouse size is now available in Beta.
Databricks Assistant is now Genie Code
Mar 11, 2026
· release notes
Databricks Assistant rebranded as Genie Code with GA agent mode for multi-step data tasks.
Genie Code Agent mode is generally available for data science, data engineering, and dashboard authoring. Not checked yet
1 detail in Everything
Databricks SQL pipelines support notifications and performance mode (Beta)
Mar 11, 2026
· release notes
Beta: Databricks SQL pipelines add failure notifications and serverless performance mode for materialized views and streaming tables.
Databricks SQL now supports failure notifications for materialized views and streaming tables defined and scheduled within the service. Not checked yet
1 detail in Everything
Declarative Feature Engineering APIs (Beta)
Mar 9, 2026
· release notes
Declarative feature engineering APIs Beta for defining and materializing time-windowed aggregation features in AWS us-east-1 and us-west-2
Adds Feature Views endpoint to define features in Unity Catalog declaratively Not checked yet Adds Materialize Feature Views endpoint to materialize batch features to Delta tables or Lakebase Not checked yet Enables serving features via online model serving using Unity Catalog models Not checked yet
3 details in Everything
Inbound Private Link for performance-intensive services (Public Preview)
Mar 9, 2026
· release notes
Databricks adds Inbound Private Link Private Link An AWS networking feature that exposes a service over a private endpoint inside a VPC, so traffic never traverses the public internet, giving cyber tools a way to integrate with cloud services without opening public routes. for Zerobus Ingest and Lakebase Autoscaling in Public Preview
Adds Inbound Private Link support for performance-intensive services including Zerobus Ingest and Lakebase Autoscaling. Not checked yet Opens configuration path to set up Inbound PrivateLink specifically for these performance-intensive services. Not checked yet
2 details in Everything
BI compatibility mode for metric views in Power BI (Beta)
Mar 9, 2026
· release notes
Beta: BI compatibility mode for Databricks metric views in Power BI rewrites SQL aggregations.
New Databricks Apps resources allow adding other apps and Unity Catalog tables for communication and data access.
Adds capability to add Databricks app resources to a Databricks app for app-to-app communication. Not checked yet Adds capability to add Unity Catalog table resources to a Databricks app for governed structured data access. Not checked yet
2 details in Everything
Endpoint telemetry for custom model serving endpoints (Beta)
Mar 6, 2026
· release notes
Beta adds endpoint telemetry for custom model serving endpoints into Unity Catalog Delta tables via OpenTelemetry OpenTelemetry An open standard and set of tools for collecting traces, metrics and logs from applications. It lets services export observability data without tying instrumentation to one monitoring vendor. .
OpenAI GPT-5.4 now available as a Databricks-hosted model
Mar 5, 2026
· release notes
OpenAI GPT-5.4 now available as a Databricks-hosted model via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. .
Tree view of the workspace browser side panel (Public Preview)
Mar 5, 2026
· release notes
Databricks adds a tree view to the workspace browser side panel for hierarchical navigation.
Supervisor Agent is now available in additional regions
Mar 3, 2026
· release notes
Supervisor Agent expands to new regions; consult docs for the full list of supported locations.
Knowledge Assistant is now available by default for Enhanced Security and Compliance customers
Mar 3, 2026
· release notes
Knowledge Assistant available by default for Enhanced Security and Compliance customers with HIPAA HIPAA A US federal law enacted in 1996 that sets privacy and security requirements for protected health information, giving cyber tools a compliance baseline to enforce or audit against. controls
Managed OAuth flows for external MCP servers
Mar 2, 2026
· release notes
Adds managed OAuth flows for external MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. servers to eliminate manual credential management.
Supports managed OAuth for select external MCP servers (Glean MCP, GitHub MCP, Google Drive API, SharePoint API). Not checked yet Allows connecting these providers without registering a separate OAuth application or managing client credentials. Not checked yet
2 details in Everything
Connect to Databricks from Microsoft Excel (Public Preview)
Mar 2, 2026
· release notes
Databricks Excel Add-in enters Public Preview, enabling Excel-to-workspace data connection, import, and SQL execution
Adds the Databricks Excel Add-in to Public Preview for connecting Microsoft Excel to a Databricks workspace Not checked yet Enables importing data directly from Databricks tables or Metric Views into Excel Not checked yet Allows executing SQL queries against Databricks resources directly within the Excel environment Not checked yet
3 details in Everything
Notebook tagging public preview allows users to categorize and manage notebooks with governed tags for certification and deprecation.
Schedule data refreshes in Google Sheets (Public Preview)
Mar 2, 2026
· release notes
Adds scheduled automatic data refreshes (hourly/daily/weekly) for the Databricks Connector for Google Sheets in Public Preview.
Customer-managed keys for Unity Catalog (Public Preview)
Mar 2, 2026
· release notes
Databricks adds public preview support for customer-managed keys (CMK) to protect data in Unity Catalog catalogs
Enables protection of data in Unity Catalog catalogs with your own encryption keys using customer-managed keys (CMK). Not checked yet
1 detail in Everything
Python unit testing in the workspace
Feb 26, 2026
· release notes
Integrated Python unit testing tools added to Databricks workspace with sidebar and panel UI.
Adds the tests sidebar in the workspace for discovering pytest-based tests Not checked yet Enables inline execution glyphs within the notebook editor to run tests directly Not checked yet Introduces the testing results bottom panel tab to view test outcomes and debug failures Not checked yet
3 details in Everything
OpenAI GPT-5.3 Codex now available as a Databricks-hosted model
Feb 25, 2026
· release notes
Databricks adds GPT-5.3 Codex to Mosaic AI Model Serving via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. .
Adds OpenAI GPT-5.3 Codex as a Databricks-hosted model in Mosaic AI Model Serving. Not checked yet Enables access to the new model via Foundation Model APIs with pay-per-token pricing. Not checked yet
2 details in Everything
Serverless outbound IPs available through public JSON endpoint (Public Preview)
Feb 25, 2026
· release notes
Databricks adds public JSON endpoint to retrieve serverless outbound IP addresses
Exposes a public JSON endpoint for retrieving serverless outbound IP addresses Not checked yet Replaces the previous stable IPs that were shared with customers in Public Preview Not checked yet
2 details in Everything
ADBC driver is now the default driver for new Power BI connections
Feb 20, 2026
· release notes
Power BI Desktop and Service now default to the Arrow Database Connectivity (ADBC ADBC Arrow Database Connectivity, an Apache standard API for querying databases that returns results directly as Apache Arrow columnar buffers, eliminating per-row copying overhead and making bulk data transfer fast for analytics pipelines. ) driver for new Databricks connections.
New Power BI connections automatically use the ADBC driver instead of ODBC ODBC A standard API, maintained by Microsoft, for connecting applications to relational databases regardless of vendor. Cyber tools use it to query databases without writing engine-specific code. Not checked yet Ability to switch back to ODBC drivers for new connections if required Not checked yet
2 details in Everything
Google Gemini 3.1 Pro Preview now available as a Databricks-hosted model
Feb 19, 2026
· release notes
Databricks adds Google Gemini 3.1 Pro Preview as a Databricks-hosted model via Mosaic AI Model Serving
Scoped personal access tokens (Beta)
Feb 18, 2026
· release notes
Adds scoped personal access tokens personal access tokens A personal access token, a credential used by scripts and integrations instead of a user's password. Its permissions and expiry depend on the service issuing it. to limit API permissions
Enables selection of a token type when creating personal access tokens access tokens A credential that authorizes a client to use a service or API. It usually has limited permissions and a lifetime; anyone holding it may be able to use those permissions. to apply API scopes Not checked yet
1 detail in Everything
Databricks adds Discover page with Domains, custom curation, and AI recommendations
Adds the Discover page as a centralized interface for searching, browsing, and previewing data assets governed by Unity Catalog. Not checked yet Adds Domains for grouping data assets by functional area (e.g., Marketing, Finance) to improve discovery and stewardship. Not checked yet Adds Custom curation capabilities allowing curators to customize and highlight specific assets within their organization or domain. Not checked yet Adds AI-powered recommendations to surface popular and valuable datasets for curation at scale. Not checked yet Enables Unified discovery of tables, dashboards, Genie Agents, and other assets in a single interface. Not checked yet
5 details in Everything
Anthropic Claude Sonnet 4.6 now available as a Databricks-hosted model
Feb 17, 2026
· release notes
Mosaic AI Model Serving now supports Anthropic Claude Sonnet 4.6 as a Databricks-hosted model via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves.
Qwen3-Embedding-0.6B now available in Public Preview as a Databricks-hosted model
Feb 17, 2026
· release notes
Mosaic AI Model Serving adds a new Databricks-hosted embedding model embedding model Lists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG. available in Public Preview.
Adds Qwen3-Embedding-0.6B to Mosaic AI Model Serving as a Databricks-hosted model accessible via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. pay-per-token Not checked yet
1 detail in Everything
Release notes for Databricks AI Gateway Beta describe a new enterprise control plane for governing LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. endpoints and coding agents.
Provides a rich UI for AI Gateway management and configuration. Not checked yet Adds improved observability capabilities for monitored LLM endpoints and coding agents. Not checked yet Expands API coverage for programmatic interaction with AI Gateway governance features. Not checked yet
3 details in Everything
Deploy Databricks apps from Git repositories (Beta)
Feb 6, 2026
· release notes
Beta feature allows deploying Databricks apps directly from Git repositories without manual uploads.
Enables direct deployment of Databricks apps from Git repositories for any branch, tag, or commit Not checked yet
1 detail in Everything
Query tags for SQL warehouses (Public Preview)
Feb 6, 2026
· release notes
Adds QUERY_TAGS SQL statement and system.query.history support for cost attribution
SET QUERY_TAGS statement available to assign custom key-value tags to SQL workloads Not checked yet Query tags now appear in the system.query.history table and on the Query History UI page Not checked yet
2 details in Everything
Applying filters, masks, tags, and comments to pipeline-created datasets is now GA
Feb 5, 2026
· release notes
GA: Apply filters, masks, tags, and comments to pipeline-created datasets via SQL or Lakeflow UI.
Anthropic Claude Opus 4.6 now available as a Databricks-hosted model
Feb 5, 2026
· release notes
Databricks Mosaic AI Model Serving now supports Anthropic Claude Opus 4.6 as a Databricks-hosted model.
Default SQL warehouse settings (General Availability)
Feb 5, 2026
· release notes
Default SQL warehouse settings GA: workspace and user-level defaults for SQL authoring surfaces.
Workspace administrators can now set a default SQL warehouse automatically selected in SQL editor, AI/BI dashboards, Genie Agents, Alerts, and Catalog Explorer Not checked yet Individual users can override the workspace default by setting their own user-level default warehouse Not checked yet
2 details in Everything
Connect Databricks Assistant to MCP servers
Feb 4, 2026
· release notes
Databricks Assistant in Agent mode now connects to external tools and data sources via the Model Context Protocol Model Context Protocol Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. (MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. )
Adds Agent mode support for Databricks Assistant to connect to any MCP servers added to your workspace Not checked yet
1 detail in Everything
Select tables and create pivot tables in Google Sheets
Feb 3, 2026
· release notes
New integration allowing direct import of Databricks Catalog Explorer tables as pivot tables in Google Sheets via the Databricks Connector.
Regional model hosting for Genie Agents in Japan and Korea
Feb 2, 2026
· release notes
Genie Agents now support regional model hosting in ap-northeast-1 (Tokyo) and ap-northeast-2 (Seoul).
Genie Agents use models hosted in ap-northeast-1 (Tokyo) for workspaces in that region, eliminating cross-Geo processing. Not checked yet Genie Agents use models hosted in ap-northeast-2 (Seoul) for workspaces in that region, eliminating cross-Geo processing. Not checked yet
2 details in Everything
Databricks adds Beta Zendesk Support connector to ingest ticket and community data.
Data Quality Monitoring Anomaly Detection public preview
Feb 2, 2026
· release notes
Databricks Data Quality Monitoring Anomaly Detection public preview now enables schema-level anomaly learning and consolidated UI/system table view.
New tagging capability allows users to categorize apps, though search support is currently unavailable.
App tags cannot be used in search queries at this time. Not checked yet
1 detail in Everything
Lakeflow jobs system tables are now generally available (GA)
Jan 30, 2026
· release notes
Lakeflow jobs system tables (jobs, job_tasks, etc.) are now GA
Makes jobs, job_tasks, job_run_timeline, and job_task_run_timeline system tables generally available for querying account-wide job insights Not checked yet Adds the Jobs system table reference documentation for these new GA tables Not checked yet
2 details in Everything
Users can now paste images directly from their local file system into Databricks notebook Markdown cells.
Serverless workspaces are now generally available
Jan 29, 2026
· release notes
Databricks serverless workspaces are now generally available for new deployments.
└──▷ HOW TO FIND IT
Deploy a new serverless workspace to get started with the managed experience.
📍 See Create a serverless workspace
OpenAI GPT-5.2 Codex now available as a Databricks-hosted model
Jan 29, 2026
· release notes
OpenAI GPT-5.2 Codex available as a Databricks-hosted model via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves.
Compute sizing for Databricks Apps is now generally available
Jan 28, 2026
· release notes
Databricks Apps compute sizing is now generally available.
Knowledge Assistant is now available in more regions
Jan 27, 2026
· release notes
Databricks Knowledge Assistant is now available in us-east-2, ca-central-1, eu-central-1, eu-west-1, ap-southeast-1, and ap-southeast-2.
Managed MCP servers is now Public Preview
Jan 27, 2026
· release notes
Managed MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. servers enter Public Preview, allowing AI agents to connect Databricks resources with external APIs.
Adds Managed MCP servers in Public Preview for agent connectivity Not checked yet
1 detail in Everything
Store and query MLflow traces in Unity Catalog (Beta)
Jan 26, 2026
· release notes
Databricks adds beta support for storing and querying MLflow traces as OpenTelemetry OpenTelemetry An open standard and set of tools for collecting traces, metrics and logs from applications. It lets services export observability data without tying instrumentation to one monitoring vendor. events in Unity Catalog Delta tables
Databricks adds a beta Google Drive connector via Lakeflow Connect for file ingestion.
You can now use custom base environments for Python, Python Wheels, and notebook tasks in serverless jobs
Jan 20, 2026
· release notes
Serverless jobs now support custom base environments for Python and notebook tasks via YAML.
Ingest Salesforce formula fields incrementally (Beta)
Jan 13, 2026
· release notes
Lakeflow Connect now supports incremental ingestion of Salesforce formula fields in Beta.
Enables incremental formula field ingestion for Salesforce sources to improve performance and reduce costs Not checked yet
1 detail in Everything
Knowledge Assistant is now generally available
Jan 13, 2026
· release notes
Knowledge Assistant is now generally available in select US regions for workspaces without Enhanced Security and Compliance features.
OpenAI GPT-5.1 Codex Max and Codex Mini now available as Databricks-hosted models
Jan 12, 2026
· release notes
Databricks Model Serving now hosts OpenAI GPT-5.1 Codex Max and Mini for code tasks
Adds support for OpenAI GPT-5.1 Codex Max as a Databricks-hosted model in Mosaic AI Model Serving. Not checked yet Adds support for OpenAI GPT-5.1 Codex Mini as a Databricks-hosted model in Mosaic AI Model Serving. Not checked yet Enables access to the new Codex models via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. pay-per-token pricing. Not checked yet
3 details in Everything
Automatic email notifications for expiring personal access tokens (GA)
Jan 6, 2026
· release notes
Databricks GA adds automatic email notifications for expiring personal access tokens personal access tokens A personal access token, a credential used by scripts and integrations instead of a user's password. Its permissions and expiry depend on the service issuing it.
Create skills for Databricks Assistant
Jan 6, 2026
· release notes
Databricks Assistant in Agent mode now supports creating user-defined skills for domain-specific tasks.
Adds User skills to Databricks Assistant in Agent mode that follow the open Agent Skills standard and are automatically loaded when relevant. Not checked yet
1 detail in Everything
Databricks Assistant on the documentation site
Dec 29, 2025
· release notes
Databricks Assistant (Genie) is now available on the documentation site for Q&A and discovery.
Databricks Assistant Agent mode is now in Public Preview
Dec 23, 2025
· release notes
Databricks Assistant Agent mode public preview adds automated code generation, execution, and error fixing.
Enables Databricks Assistant Agent mode by default for most customers to automate multi-step workflows from a single prompt Not checked yet Adds automatic sampling of data and cell outputs to improve agent result accuracy Not checked yet Allows the agent to choose between Azure OpenAI or Anthropic endpoints hosted within the Databricks security perimeter Not checked yet
3 details in Everything
Single-use refresh tokens for OAuth applications
Dec 22, 2025
· release notes
Databricks adds single-use refresh tokens refresh tokens A credential used to obtain a new access token without signing in again. It is sent to the authorization service rather than used directly to call the protected API. for OAuth apps
Anthropic Claude Haiku 4.5 now available as a Databricks-hosted model
Dec 19, 2025
· release notes
Databricks Mosaic AI Model Serving now supports Anthropic Claude Haiku 4.5 via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. .
Update request parameters for Delta Sharing recipient audit log events
Dec 19, 2025
· release notes
Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. recipient audit logs now include catalog_name and renamed share_name parameters.
Adds catalog_name request parameter to deltaSharingProxy* audit log events for Delta Sharing recipients. Not checked yet Renames the share field to share_name in deltaSharingProxy* audit log events. Not checked yet
2 details in Everything
└──▷ BREAKING ON UPGRADE
! The share field in deltaSharingProxy* audit log events is renamed to share_name.
MySQL connector in Lakeflow Connect (Public Preview)
Dec 18, 2025
· release notes
New public preview MySQL connector in Lakeflow Connect enables incremental ingestion from Amazon RDS, Aurora, Azure DB for MySQL, Cloud SQL, and EC2 EC2 Amazon's cloud virtual-machine service, letting tools run on-demand servers without managing physical hardware. .
Adds a fully-managed MySQL connector in Lakeflow Connect Public preview to enable incremental data ingestion from MySQL databases Not checked yet Adds support for five source platforms: Amazon RDS for MySQL, Amazon Aurora MySQL, Azure Database for MySQL, Google Cloud SQL for MySQL, and MySQL on EC2 Not checked yet
2 details in Everything
View latest scheduled notebook job results
Dec 18, 2025
· release notes
Notebooks can now display and update with the latest scheduled run results directly in UI.
Adds the ability to show the latest scheduled notebook run directly within your notebook and notebook dashboards Not checked yet Enables updating a notebook with the latest run results Not checked yet
2 details in Everything
Context based ingress control is now in Public Preview
Dec 17, 2025
· release notes
Databricks introduces Context-based ingress control for workspace access and Gemini 3 Flash as a hosted model.
Adds context-based ingress control in public preview to govern workspace access via identity, network source, and request type policies Not checked yet Adds Gemini 3 Flash as a Databricks-hosted model with multimodal multimodal Able to work with more than one type of input or output, such as text, images and audio. A multimodal model can combine information across those formats. capabilities Not checked yet
2 details in Everything
New resource types for Databricks Apps
Dec 17, 2025
· release notes
Databricks Apps now supports MLflow experiments, vector search indexes, UDFs, and Unity Catalog connections as resources.
Adds mlflow_experiment resource type for managing MLflow experiments within a Databricks App Not checked yet Adds vector_search_index resource type for managing vector search indexes within a Databricks App Not checked yet Adds udf resource type for managing user-defined functions within a Databricks App Not checked yet Adds unity_catalog_connection resource type for managing Unity Catalog connections within a Databricks App Not checked yet
4 details in Everything
PostgreSQL connector in Lakeflow Connect (Public Preview)
Dec 16, 2025
· release notes
Databricks adds Public Preview for fully-managed PostgreSQL connector in Lakeflow Connect
Adds PostgreSQL connector to Lakeflow Connect for incremental data ingestion from Amazon RDS PostgreSQL, Amazon Aurora PostgreSQL, Azure Database for PostgreSQL, Google Cloud SQL for PostgreSQL, and on-premises sources Not checked yet
1 detail in Everything
OpenAI GPT-5.2 now available as a Databricks-hosted model
Dec 11, 2025
· release notes
Databricks Mosaic AI Model Serving adds OpenAI GPT-5.2 via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves.
Mosaic AI Model Serving now supports OpenAI GPT-5.2 as a Databricks-hosted model Not checked yet Access the new model via Foundation Model APIs using pay-per-token pricing Not checked yet
2 details in Everything
Disable legacy features settings are now GA
Dec 11, 2025
· release notes
Two new admin settings are generally available to help migrate accounts and workspaces to Unity Catalog by disabling legacy features.
Discover files in Auto Loader efficiently using file events
Dec 10, 2025
· release notes
Auto Loader with file events reaches GA, enabling efficient file discovery for Unity Catalog.
file_events flag enables Auto Loader to discover files with notification efficiency while retaining directory listing setup simplicity Not checked yet Recommended configuration method for using Auto Loader with Unity Catalog Not checked yet
2 details in Everything
Change owner for materialized views or streaming tables defined in Databricks SQL
Dec 10, 2025
· release notes
Databricks SQL adds owner change for materialized views and streaming tables via Catalog Explorer
Adds CHANGE OWNER capability for materialized views defined in Databricks SQL through Catalog Explorer. Not checked yet Adds CHANGE OWNER capability for streaming tables defined in Databricks SQL through Catalog Explorer. Not checked yet
2 details in Everything
New beta connector for Jira data ingestion.
Ingests Jira issues, comments, and attachments metadata. Not checked yet
1 detail in Everything
Customizable SharePoint connector (Beta)
Dec 10, 2025
· release notes
Databricks adds a customizable SharePoint connector for flexible file ingestion with schema control.
Adds the customizable SharePoint connector to enable structured, semi-structured, and unstructured file ingestion into Delta tables. Not checked yet
1 detail in Everything
New NetSuite connector allows programmatic data ingestion via API, CLI, or notebook.
Adds the NetSuite connector in Public Preview to ingest data from NetSuite2.com. Not checked yet Enables configuration of the NetSuite connection programmatically using the Databricks API. Not checked yet
2 details in Everything
New columns in Lakeflow system tables (Public Preview)
Dec 9, 2025
· release notes
New columns added to Lakeflow system tables for enhanced job monitoring in public preview
Adds trigger, trigger_type, run_as_user_name, creator_user_name, paused, timeout_seconds, health_rules, deployment, and create_time columns to the jobs table. Not checked yet Adds timeout_seconds and health_rules columns to the job_tasks table. Not checked yet Adds source_task_run_id, root_task_run_id, compute, termination_type, setup_duration_seconds, queue_duration_seconds, run_duration_seconds, cleanup_duration_seconds, and execution_duration_seconds columns to the job_run_timeline table. Not checked yet Adds compute, termination_type, task_parameters, setup_duration_seconds, cleanup_duration_seconds, and execution_duration_seconds columns to the job_task_run_timeline table. Not checked yet Adds create_time column to the pipelines table. Not checked yet
5 details in Everything
ForEachBatch for Spark Declarative Pipelines on Lakeflow is available (Public Preview)
Dec 9, 2025
· release notes
Spark Declarative Pipelines on Lakeflow gains ForEachBatch sink for micro-batch processing
Vector Search reranker is now generally available
Dec 8, 2025
· release notes
Vector Search reranker reranker A model that re-scores a first batch of search results against the query and reorders them, more accurately than the fast first-stage search. reaches GA to improve retrieval quality.
Adds general availability for the Vector Search reranker capability. Not checked yet
1 detail in Everything
Expanded regional availability for C5 and TISAX compliance
Dec 8, 2025
· release notes
Databricks expands C5 and TISAX TISAX A vehicle-industry information-security assessment standard, maintained by the ENX Association, that lets automotive suppliers prove their data-handling practices to OEMs without each OEM running its own separate audit. compliance standards availability to all regions and serverless compute
New token expiration policy for open Delta Sharing
Dec 8, 2025
· release notes
New policy limiting Open Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. recipient token expiration to one year
Open sharing recipient tokens are now capped at a maximum expiration of one year from creation. Not checked yet Tokens with no expiration or longer than one year can no longer be created. Not checked yet
2 details in Everything
└──▷ BREAKING ON UPGRADE
! Existing open sharing recipient tokens issued before December 8, 2025, with expiration dates after December 8, 2026, or with no expiration date, automatically expire on December 8, 2026.
Built-in Excel file format support (Beta)
Dec 2, 2025
· release notes
Databricks adds built-in support for reading Excel files via Spark DataFrames
Spark Declarative Pipelines on Lakeflow stream progress metrics are generally available
Nov 25, 2025
· release notes
Spark Declarative Pipelines on Lakeflow now allows users to query event logs for stream progress metrics.
Anthropic Claude Opus 4.5 now available as a Databricks-hosted model
Nov 24, 2025
· release notes
Databricks adds Anthropic Claude Opus 4.5 support in Mosaic AI Model Serving
Automate classic workspace creation with AWS IAM temporary delegation
Nov 21, 2025
· release notes
Databricks adds automated classic workspace creation via AWS IAM AWS IAM AWS Identity and Access Management is Amazon's service for controlling who and what can call AWS APIs, by attaching policies that allow or deny actions on resources. Cyber tools use it to scope credentials to least privilege. temporary delegation
Real-time collaboration in notebook cells, files, and SQL editor
Nov 21, 2025
· release notes
Real-time collaboration for simultaneous editing in notebook cells, files, and the SQL editor
Configure full refresh behavior for managed database connectors
Nov 21, 2025
· release notes
Allows configuring automatic full refresh schedules for managed database connectors like SQL Server.
Adds the ability to schedule full refresh snapshots and enable automatic recovery from unsupported schema changes for managed database connectors. Not checked yet
1 detail in Everything
Automatic email notifications for expiring personal access tokens (Public Preview)
Nov 20, 2025
· release notes
Automatic email notifications for expiring personal access tokens personal access tokens A personal access token, a credential used by scripts and integrations instead of a user's password. Its permissions and expiry depend on the service issuing it. Public preview
Google Gemini 3 Pro Preview now available as a Databricks-hosted model
Nov 19, 2025
· release notes
Databricks adds Google Gemini 3 Pro Preview as a hosted model in Mosaic AI Model Serving
ABAC-secured assets shared using Delta Sharing is in Public Preview
Nov 17, 2025
· release notes
Public Preview of applying ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. policies to tables and schemas in Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. .
└──▷ HOW TO FIND IT
Apply an ABAC policy to a specific schema before sharing it via Delta Sharing.
📍 Add tables and schemas secured by ABAC policies to a share
Consume the shared data and apply your own ABAC constraints.
📍 Read ABAC-secured data and apply ABAC policies
Jobs can now be triggered on source view update
Nov 17, 2025
· release notes
Table update triggers for Databricks Jobs now support views as a source.
Adds view-based sourcing to table update job triggers. Not checked yet
1 detail in Everything
Delta Sharing views automatically permitted in serverless network policies
Nov 17, 2025
· release notes
Serverless network policies now automatically permit Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. views across storage locations.
OpenAI GPT-5.1 now available as a Databricks-hosted model
Nov 14, 2025
· release notes
OpenAI GPT-5.1 now available as a Databricks-hosted model via Mosaic AI Model Serving
Enables foundation model APIs with pay-per-token pricing for the new model Not checked yet Supports querying reasoning capabilities of the new model Not checked yet Supports querying vision capabilities of the new model Not checked yet Enables batch inference workloads for the new model using AI Functions Not checked yet
4 details in Everything
Improved full refresh flow for managed database connectors
Nov 14, 2025
· release notes
Managed database connectors delay table refresh until after snapshot to reduce downtime
Connect Genie Agents to Microsoft Copilot Studio (Public Preview)
Nov 14, 2025
· release notes
Connect Databricks Genie Agents to Microsoft Copilot Studio via Managed MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. Servers
Convert foreign tables to Unity Catalog managed or external tables (Public Preview)
Nov 13, 2025
· release notes
New ALTER TABLE SET MANAGED and SET EXTERNAL commands convert foreign tables to Unity Catalog
Adds ALTER TABLE SET MANAGED with MOVE or COPY options to convert foreign tables to Unity Catalog managed tables Not checked yet Adds ALTER TABLE SET EXTERNAL with optional DRY RUN validation to convert foreign tables to Unity Catalog external tables Not checked yet Adds ALTER CATALOG DROP CONNECTION to convert entire foreign catalogs to standard Unity Catalog catalogs Not checked yet
3 details in Everything
Lakeflow Connect now supports SFTP SFTP SSH File Transfer Protocol, a file transfer standard built on top of SSH that provides encrypted upload, download, and remote file management; cyber tools use it to move files or exfiltrate data over an authenticated, encrypted channel. as an ingest source in Public Preview
JDBC Unity Catalog connection in Beta
Nov 13, 2025
· release notes
Adds Beta JDBC JDBC A Java standard API for connecting applications to relational databases, maintained as part of the Java SE platform, letting tools query any compliant database through a single consistent interface. Unity Catalog connections for governed external database access.
Enables JDBC Unity Catalog connection to read/write to external databases via Spark Data Source API or Databricks Remote Query SQL API Not checked yet Makes JDBC connection reusable across any Unity Catalog compute type: serverless, standard clusters, dedicated clusters, and Databricks SQL Not checked yet
2 details in Everything
New MCP Servers tab in the workspace to manage installed servers and browse Databricks Marketplace.
MCP Servers tab provides a curated list of MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. servers available to install from Databricks Marketplace Not checked yet MCP Servers tab displays your managed and external MCP servers Not checked yet
2 details in Everything
List MCP servers in Databricks Marketplace (Public Preview)
Nov 6, 2025
· release notes
Databricks Marketplace adds support for listing and installing Model Context Protocol Model Context Protocol Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. (MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. ) servers in public preview
Serverless notebook tasks can now use jobs environment
Nov 5, 2025
· release notes
Serverless notebook tasks now support selecting a specific jobs environment instead of auto-inheriting the notebook's.
Adds capability to select a specific job environment for serverless notebook tasks Not checked yet Changes default behavior: serverless notebook tasks no longer automatically inherit the notebook's environment Not checked yet
2 details in Everything
Updated tag search syntax in workspace search
Nov 5, 2025
· release notes
Updated workspace search syntax for tables and views by tag
Retains tag:<tag_key> syntax for searching by tag key alone Not checked yet
1 detail in Everything
Function calling and structured output now available using the Alibaba Cloud Qwen3-Next Instruct model (Beta)
Nov 4, 2025
· release notes
Mosaic AI Model Serving adds support for function calling function calling A model's ability to answer with a structured request to run a named function or API, which the application executes and returns. It is how agents take actions. and structured output structured output Forcing a model's response to match a given format or JSON schema, so software can parse it reliably instead of hoping the text is well formed. with Alibaba Cloud Qwen3-Next Instruct Beta
Adds function calling capability to Mosaic AI Model Serving when using the Qwen3-Next Instruct model Not checked yet Adds structured output capability to Mosaic AI Model Serving when using the Qwen3-Next Instruct model Not checked yet
2 details in Everything
Audit logs for request-for-access events are now emitted
Nov 3, 2025
· release notes
Databricks now emits request-for-access destination actions to audit logs Public preview
Adds request for access destination action events to Databricks audit logs Not checked yet
1 detail in Everything
Attribute-based access control (ABAC) now in Public Preview
Nov 3, 2025
· release notes
Databricks ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. moves to account-level enforcement with automatic column mask type casting
Adds account-level enforcement for Attribute-based access control Attribute-based access control Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. (ABAC) policies across all workspaces attached to an account's metastore Not checked yet Enables automatic type casting for column mask functions defined in ABAC policies to match target column data types Not checked yet
2 details in Everything
└──▷ BREAKING ON UPGRADE
! Existing column mask functions might fail if their return types are incompatible with the target column's type due to new automatic type casting behavior
Improved notebook debugger experience
Oct 31, 2025
· release notes
Databricks notebook debugger adds multi-file debugging capability
Enables breakpoints and stepping into functions across multiple workspace files in the Python notebook interactive debugger Not checked yet
1 detail in Everything
Salesforce ingestion connector supports history tracking (SCD type 2)
Oct 31, 2025
· release notes
Lakeflow Connect Salesforce connector now supports SCD Type 2 history tracking for managed sources
Adds %skip magic command in notebooks to skip individual cells during execution.
Feedback model deprecated for AI agents
Oct 29, 2025
· release notes
Databricks deprecates the experimental feedback model for AI agents; users must migrate to MLflow 3.
Deprecates the experimental feedback model for newly deployed AI agents starting November 1, 2025 Not checked yet Enables use of the log_feedback API in MLflow 3 to collect assessments on agent traces Not checked yet
2 details in Everything
└──▷ BREAKING ON UPGRADE
! Newly deployed agents after November 1, 2025 will not include the experimental feedback model
Supervisor Agent now supports Unity Catalog functions and external MCP servers
Oct 29, 2025
· release notes
Supervisor Agent now supports Unity Catalog functions and external MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. servers as tools
Adds support for Unity Catalog functions as tools in Supervisor Agent Not checked yet Adds support for external MCP (Model Context Protocol Model Context Protocol Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. ) servers as tools in Supervisor Agent Not checked yet
2 details in Everything
New compute policy form is now generally available
Oct 29, 2025
· release notes
Databricks new compute policy form is GA, allowing admin configuration via dropdowns and UI elements without writing policy syntax.
New compute policy form is generally available for configuring policy definitions using dropdown menus and other UI elements. Not checked yet
1 detail in Everything
OpenAI GPT 5 models are now generally available in Mosaic AI Model Serving
Oct 28, 2025
· release notes
Mosaic AI Model Serving now supports GPT-5, GPT-5 mini, and GPT-5 nano models.
Adds GPT-5 model support to Mosaic AI Model Serving Not checked yet Adds GPT-5 mini model support to Mosaic AI Model Serving Not checked yet Adds GPT-5 nano model support to Mosaic AI Model Serving Not checked yet
3 details in Everything
Gemini 2.5 Pro and Flash models now supported on pay-per-token endpoints
Oct 28, 2025
· release notes
Databricks extends its Model Serving capabilities to include Google's Gemini 2.5 Pro and Flash models for pay-per-token usage.
Adds Gemini 2.5 Pro and Gemini 2.5 Flash to supported model list in Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. pay-per-token endpoint Not checked yet
1 detail in Everything
Zerobus Ingest connector in Lakeflow Connect (Public Preview)
Oct 23, 2025
· release notes
Databricks adds Zerobus Ingest connector in Lakeflow Connect Public preview for record-by-record gRPC gRPC An open-source remote procedure call framework from Google that uses HTTP/2 and Protocol Buffers to let services call each other's functions across a network, giving tools low-latency, strongly typed inter-process communication. ingestion into Delta tables.
Databricks Runtime 17.3 LTS adds remote_query TVF, SQL EXECUTE IMMEDIATE with constants, and JDK 21 preview.
Adds the remote_query table-valued function to fetch data from remote SQL and NoSQL databases using Unity Catalog connections Not checked yet Adds support for constant expressions as arguments in EXECUTE IMMEDIATE SQL statements Not checked yet Adds JDK 21 support for classic compute clusters Public preview Not checked yet Adds metadata column in DESCRIBE QUERY and DESCRIBE TABLE output for semantic metadata propagation on metric views Not checked yet Enables appending to tables with fine-grained access control on dedicated compute (general availability) Not checked yet
+3 more
show less
Adds st_dump, st_numinteriorrings, and st_interiorringn functions for polygon interior ring operations Not checked yet Adds support for LIMIT ALL in recursive common table expressions (CTEs) to remove row count restrictions Not checked yet Changes the default for cloudFiles.useIncrementalListing from auto to false for Auto Loader Not checked yet
8 details in Everything
└──▷ BREAKING ON UPGRADE
! The input_file_name function is no longer supported; use _metadata.file_name instead! Auto Loader's cloudFiles.useIncrementalListing default changed from auto to false; set explicitly to auto to preserve previous behavior
Alibaba Cloud Qwen3-Next Instruct now available in Beta
Oct 22, 2025
· release notes
Databricks extends Mosaic AI Model Serving with a new generative model option for Beta users in two specific regions.
Databricks adds Compatibility Mode public preview for syncing Unity Catalog tables to external Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. and Iceberg clients
Generates a read-only version of Unity Catalog managed tables, streaming tables, or materialized views automatically synced with the original. Not checked yet Enables external Delta Lake and Iceberg clients like Amazon Athena, Snowflake, and Microsoft Fabric Microsoft Fabric A Microsoft cloud analytics platform that unifies data engineering, warehousing, and business intelligence under one SaaS service, giving cyber tools a single place to ingest, store, and query large volumes of telemetry. to read Databricks data. Not checked yet
2 details in Everything
Zstd is now the default compression for new managed tables
Oct 21, 2025
· release notes
Databricks Runtime 16.0+ uses Zstd compression by default for new managed tables instead of Snappy.
Databricks introduces public preview of the unified runs list to monitor job and pipeline executions together.
Adds unified runs list Public preview to view both job and pipeline runs in a single interface Not checked yet
1 detail in Everything
Dashboard and Genie Agents tagging (Public Preview)
Oct 16, 2025
· release notes
This release introduces tagging capabilities for Dashboards and Genie Agents in Databricks, allowing users to organize resources and trigger automated processes such as warehouse assignment based on tag values like Work in progress or Certified .
Declarative Automation Bundles in the workspace is GA
Oct 16, 2025
· release notes
This release marks the general availability of Declarative Automation Bundles, allowing users to collaboratively manage bundle updates within the Databricks workspace interface.
Job backfills allow you to trigger job runs to backfill data from the past
Mosaic AI Model Serving adds multimodal multimodal Able to work with more than one type of input or output, such as text, images and audio. A multimodal model can combine information across those formats. input support for Databricks-hosted foundation models via paid APIs and AI Functions.
Adds multimodal input support to Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. (pay-per-token and provisioned throughput) for hosted foundation models Not checked yet Enables multimodal inputs in both real-time inference and batch inference workloads via AI Functions Not checked yet
2 details in Everything
Databricks releases SQL MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. server in beta, allowing AI agents to execute queries against Unity Catalog via a new API endpoint.
Adds the https://<workspace-hostname>/api/2.0/mcp/sql endpoint for the SQL managed MCP server Not checked yet
1 detail in Everything
The billable usage table now records the performance mode of serverless jobs and pipelines
Oct 9, 2025
· release notes
Billing logs now include the product_features.performance_target column for serverless jobs and pipelines.
The Data Science Agent can now also use models served through Anthropic on Databricks
Oct 8, 2025
· release notes
The Data Science Agent now supports models served through Anthropic on Databricks when partner-powered AI features are enabled.
Enables the Data Science Agent to use Anthropic models hosted within the Databricks security perimeter. Not checked yet
1 detail in Everything
Certification status system tag is in Public Preview
Oct 6, 2025
· release notes
Databricks adds system.certification_status governed tag for catalogs, schemas, tables, and more
Adds the system.certification_status governed tag to indicate if data assets are certified or deprecated Not checked yet
1 detail in Everything
Mosaic AI Model Serving now supports OpenAI GPT 5 models (preview)
Oct 6, 2025
· release notes
Mosaic AI Model Serving adds public preview support for OpenAI GPT-5, GPT-5 mini, and GPT-5 nano models.
Anthropic Claude Sonnet 4 is available for batch inference in US regions
Oct 3, 2025
· release notes
Mosaic AI Model Serving adds support for Anthropic Claude Sonnet 4 in batch inference workflows.
Serverless notebook session auto-restoration, 100MB IPYNB support, and cell execution minimap added.
Adds a Cell execution minimap in the right margin for visual navigation Not checked yet Enables automatic restoration of Python variables and Spark state when reconnecting to serverless notebooks Not checked yet Supports importing and exporting IPYNB notebooks up to 100 MB with revision snapshot autosaving, manual saving, and cloning Not checked yet Adds a choice to include or exclude cell outputs when cloning and exporting notebooks Not checked yet
4 details in Everything
Anthropic Claude Sonnet 4.5 now available as a Databricks-hosted model
Oct 3, 2025
· release notes
Mosaic AI Model Serving adds Anthropic Claude Sonnet 4.5 as a Databricks-hosted model with Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. support
Convert to Unity Catalog managed table from external table
Oct 2, 2025
· release notes
ALTER TABLE ... SET MANAGED command is generally available to convert external tables to Unity Catalog managed tables.
Git email identity configuration for Git folders
Oct 1, 2025
· release notes
Databricks Git folders now allow configuring a separate email identity for commits via GIT_AUTHOR_EMAIL and GIT_COMMITTER_EMAIL environment variables
Adds support for specifying a custom Git provider email address in Databricks Git folders that is used as both the author and committer identity for commits Not checked yet
1 detail in Everything
New permissions for the Databricks GitHub App
Oct 1, 2025
· release notes
Databricks GitHub App requests updated permissions to link commits via email
New email permission requested by the Databricks GitHub App to retrieve and save primary account email to Linked Git credential Not checked yet
1 detail in Everything
Pipeline update timeline table is now available (Public Preview)
Sep 30, 2025
· release notes
Databricks adds system.lakeflow.pipeline_update_timeline table in Public Preview for pipeline update history.
Enables fine-grained analysis of pipeline update triggers, results, and compute usage via the new system table. Not checked yet
1 detail in Everything
Lakeflow Pipelines Editor is now in public preview
Sep 29, 2025
· release notes
Lakeflow Pipelines Editor enters public preview, changing pipeline source format to Python and SQL files
The Lakeflow Pipelines Editor is now in public preview, allowing users to view pipelines as a set of files in the pipeline asset browser. Not checked yet The default source code format for pipelines changes from notebooks to Python and SQL code files. Not checked yet
2 details in Everything
└──▷ BREAKING ON UPGRADE
! The default source code format for pipelines is changed from notebooks to Python and SQL code files.
Anthropic Claude Opus 4.1 now available as a Databricks-hosted foundation model
Sep 29, 2025
· release notes
Mosaic AI Model Serving now supports Anthropic Claude Opus 4.1 via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves.
Mosaic AI Agent Framework supports automatic authentication passthrough for Lakebase resources
Sep 23, 2025
· release notes
Mosaic AI Agent Framework adds automatic authentication passthrough for Lakebase resources (MLflow 3.3.2+)
Requires MLflow version 3.3.2 or above to enable the new passthrough capability Not checked yet
1 detail in Everything
Explore table data using an LLM (Public Preview)
Sep 22, 2025
· release notes
Catalog Explorer gains natural language querying for sample data.
Compliance support for Mosaic AI Vector Search standard endpoints
Sep 18, 2025
· release notes
Mosaic AI Vector Search standard endpoints now support FedRAMP Moderate FedRAMP Moderate A US government authorization tier requiring cloud services to meet a defined set of NIST 800-53 controls before federal agencies can use them, giving cyber tools a pre-vetted compliance baseline for government contracts. and IRAP in compliant workspaces.
Genie One provides a non-technical entry point for business users to view AI/BI dashboards and ask data questions.
Generates natural language responses for data queries within the new interface. Not checked yet
1 detail in Everything
Delta Sharing on Lakehouse Federation is in Beta
Sep 16, 2025
· release notes
Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. on Lakehouse Federation Lakehouse Federation A data architecture pattern that lets a single query engine read from multiple external data sources—databases, data lakes, warehouses—without moving or copying the data, reducing pipeline complexity for tools that need broad data access. enters Beta for sharing foreign schemas and tables
Discover files in Auto Loader efficiently using file events without enrollment (Public Preview)
Sep 16, 2025
· release notes
Public Preview of cloudFiles.useManagedFileEvents in Auto Loader for efficient file discovery.
Mount Delta shares to an existing shared catalog
Sep 12, 2025
· release notes
Mount existing Delta Shares into an existing shared catalog instead of creating new catalogs.
Eliminates the need to create a new catalog for each new share Not checked yet
1 detail in Everything
The SAP Business Data Cloud (BDC) Connector for Databricks is generally available
Sep 10, 2025
· release notes
The SAP Business Data Cloud (BDC) Connector for Databricks is generally available.
Adds the SAP BDC Connector to enable secure, zero-copy data sharing between SAP BDC and Unity Catalog-enabled Databricks workspaces. Not checked yet
1 detail in Everything
Google Analytics Raw Data connector GA
Sep 10, 2025
· release notes
Google Analytics 4 Raw Data connector GA in Lakeflow Connect
Spark Declarative Pipelines on Lakeflow now supports stream progress metrics in Public Preview
Sep 10, 2025
· release notes
Lakeflow pipelines can now expose stream progress metrics via event log queries.
Python custom data sources can be used with Spark Declarative Pipelines on Lakeflow
Sep 10, 2025
· release notes
New capability to use Python custom data sources and sinks in Spark Declarative Pipelines on Lakeflow
Databricks Apps support for Genie resources
Sep 8, 2025
· release notes
Databricks Apps now supports adding a Genie Space as an app resource to enable natural language querying.
Databricks connector in Microsoft Power Platform is in Public Preview
Sep 5, 2025
· release notes
Databricks connector now in public preview for Power Apps, Power Automate, and Copilot Studio
Serverless GPU compute now supports H100 accelerators (Beta)
Sep 5, 2025
· release notes
Serverless GPU GPU A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. compute adds H100 accelerator support for single-node workflows.
Serverless GPU compute now supports on-demand H100 accelerators Not checked yet
1 detail in Everything
MLflow metadata is now available in system tables (Public Preview)
Sep 5, 2025
· release notes
MLflow metadata is now available in system tables for workspace-wide querying Public preview
Makes MLflow tracking metadata accessible via Databricks system tables, enabling central visibility and lakehouse-based analysis across the workspace Not checked yet
1 detail in Everything
Databricks Online Feature Stores (Public Preview)
Sep 5, 2025
· release notes
Databricks Online Feature Stores public preview adds Lakebase-backed low-latency feature access with Unity Catalog and MLflow integrations
Adds Databricks Online Feature Stores (powered by Lakebase) for highly-scalable, low-latency feature data access consistent with offline feature tables. Not checked yet Provides native integration with Unity Catalog , MLflow , and Mosaic AI Model Serving to allow productionized endpoints, agents, and rule engines to automatically access features. Not checked yet
2 details in Everything
Databricks Assistant Agent mode: Data Science Agent is in Beta
Sep 3, 2025
· release notes
Databricks Assistant Agent mode (Data Science Agent) enters Beta, enabling multi-step workflow orchestration from a single prompt.
Adds Agent mode to Databricks Assistant for orchestrating multi-step workflows from a single prompt Not checked yet Introduces the Data Science Agent , a custom agent capable of planning solutions, retrieving assets, running code, and fixing errors automatically for tasks like EDA, forecasting, and machine learning Not checked yet
2 details in Everything
AI agents: Authorize on-behalf-of-user Public Preview
Sep 2, 2025
· release notes
On-behalf-of-user authorization for AI agents deployed to Model Serving endpoints Public preview
SQL Server connector supports SCD type 2
Sep 1, 2025
· release notes
Lakeflow Connect SQL Server connector adds support for SCD type 2 history tracking
New Beta capability allows Databricks agents to integrate with external Model Context Protocol Model Context Protocol Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. (MCP MCP Model Context Protocol, a standard for connecting AI applications to external tools and data sources. A compatible server exposes capabilities an assistant can use through a common interface. ) servers for broader tool access.
Migrate Spark Declarative Pipelines on Lakeflow from legacy publishing mode is now GA
Aug 28, 2025
· release notes
Migrate Spark Declarative Pipelines on Lakeflow from legacy publishing mode is now GA
Track and navigate notebook runs with the new cell execution minimap
Aug 28, 2025
· release notes
Databricks adds a cell execution minimap to track notebook run states in the right margin
Adds a cell execution minimap in the right margin of the notebook UI to display execution states (skipped, queued, running, success, error) and allow direct navigation by clicking. Not checked yet Enables hovering over the minimap to view details for each cell's execution state. Not checked yet
2 details in Everything
New table property to control Delta table compression
Aug 26, 2025
· release notes
Adds delta.parquet.compression.codec table property to control Delta table compression
INSERT REPLACE USING and INSERT REPLACE ON commands are now GA in Databricks Runtime 17.2.
Databricks Runtime 17.2 and Databricks Runtime 17.2 ML are in Beta
Aug 26, 2025
· release notes
Databricks Runtime 17.2 Beta adds SQL/API enhancements and new migration options.
SQL and API enhancements are available in the Databricks Runtime 17.2 Beta release Not checked yet New migration options have been added to Databricks Runtime 17.2 Not checked yet
2 details in Everything
Databricks adds governed tags in Public Preview for enforcing consistent tagging on data assets
Adds Governed tags feature allowing admins to define allowed keys/values and control user/group assignment across catalogs, schemas, and tables Not checked yet
1 detail in Everything
OAuth token federation now GA allows API access via IdP IdP A service that authenticates users and supplies identity information to other applications. It is the sign-in authority behind many single sign-on systems. tokens with UI and API config
OAuth token federation is now generally available for secure access to Databricks APIs using identity provider identity provider A service that authenticates users and supplies identity information to other applications. It is the sign-in authority behind many single sign-on systems. (IdP) tokens. Not checked yet Configuration of federation policies is supported directly in the Databricks UI, the Databricks CLI, and the REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. . Not checked yet
2 details in Everything
Microsoft SQL Server connector becomes generally available in Lakeflow Connect
Data type mapping update for Lakebase synced tables
Aug 25, 2025
· release notes
Synced tables now map source TIMESTAMP to TIMESTAMP WITH TIMEZONE for new tables.
Budget policy support for Lakebase database instances and synced tables (Public Preview)
Aug 25, 2025
· release notes
Public Preview: Tag Lakebase database instances and synced tables with budget policies for billing attribution
Adds budget policy tagging support for Lakebase database instances in Public Preview Not checked yet Adds budget policy tagging support for synced tables in Public Preview Not checked yet
2 details in Everything
Automatic liquid clustering is now available for Spark Declarative Pipelines on Lakeflow
Aug 25, 2025
· release notes
Adds CLUSTER BY AUTO to Spark Declarative Pipelines for automatic liquid clustering.
Exposes new or updated SQL definitions in create_streaming_table, table, CREATE MATERIALIZED VIEW (pipelines), and CREATE STREAMING TABLE (pipelines) that support the new clustering mode. Not checked yet
1 detail in Everything
Salesforce Data Cloud File Sharing connector GA
Aug 22, 2025
· release notes
Salesforce Data Cloud File Sharing connector reaches GA for Lakehouse Federation Lakehouse Federation A data architecture pattern that lets a single query engine read from multiple external data sources—databases, data lakes, warehouses—without moving or copying the data, reducing pipeline complexity for tools that need broad data access. .
Enhanced autocomplete for complex data types in notebooks
Aug 22, 2025
· release notes
Enhanced notebook autocomplete for complex data types (structs, maps, arrays) and CTE columns in SQL cells
Adds enhanced autocomplete suggestions for structs, maps, and arrays in SQL notebook cells Not checked yet Enables column recommendations from underlying table structure when autocompleting CTEs defined with SELECT * Not checked yet
2 details in Everything
Declarative Automation Bundles support for Lakebase database resources
Aug 21, 2025
· release notes
Declarative Automation Bundles now support Lakebase database catalogs, instances, and synced tables
Adds database_catalog resource type to Declarative Automation Bundles Not checked yet Adds database_instance resource type to Declarative Automation Bundles Not checked yet Adds synced_database_table resource type to Declarative Automation Bundles Not checked yet
3 details in Everything
Databricks Assistant integrated with compute
Aug 21, 2025
· release notes
Databricks Assistant integrated with compute for chat-based resource creation and policy queries.
Adds Databricks Assistant chat panel to compute pages to help create new compute resources, pools, and policies Not checked yet Adds ability to ask the Assistant questions about existing compute resources via chat Not checked yet
2 details in Everything
Set the run-as user for Spark Declarative Pipelines on Lakeflow
Aug 18, 2025
· release notes
Adds run_as_user to Spark Declarative Pipelines for service principal identity management.
Databricks-hosted models for Assistant are now GA
Aug 14, 2025
· release notes
Databricks Assistant now uses hosted models on all cloud platforms at GA
OpenAI GPT OSS models now supported on provisioned throughput
Aug 13, 2025
· release notes
Databricks adds support for OpenAI GPT OSS OSS Open-source software: code whose licence permits anyone to inspect, modify, and redistribute it. Cyber tools build on it to reuse auditable libraries without licensing fees. 120B and 20B models on provisioned throughput Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. .
OpenAI GPT OSS models now support function calling
Aug 13, 2025
· release notes
OpenAI GPT OSS OSS Open-source software: code whose licence permits anyone to inspect, modify, and redistribute it. Cyber tools build on it to reuse auditable libraries without licensing fees. models (120B, 20B) now support function and tool calling tool calling A model's ability to answer with a structured request to run a named function or API, which the application executes and returns. It is how agents take actions.
Column masks now retained when replacing a table
Aug 12, 2025
· release notes
Column masks retained on CREATE OR REPLACE TABLE and DataFrame save/replace operations
CREATE OR REPLACE TABLE retains existing column-level security policies when a column name matches, preventing accidental removal of masks. Not checked yet DataFrame APIs saveAsTable, replace, and createOrReplace retain existing column masks on replacement without requiring explicit mask specification. Not checked yet
2 details in Everything
Single-node compute on standard access mode is now GA
Aug 12, 2025
· release notes
Databricks GA: single-node compute with standard access mode for small jobs
General availability of single-node compute resources with standard access mode. Not checked yet Support for full user isolation when multiple users share a single-node compute resource. Not checked yet
2 details in Everything
Databricks Assistant: User and workspace instructions now available
Aug 12, 2025
· release notes
Databricks Assistant now supports custom user and workspace-wide instructions to tailor AI responses.
Adds custom user instructions for Databricks Assistant to guide preferences and coding conventions Not checked yet Adds workspace-wide instructions for admins that take priority over user instructions Not checked yet
2 details in Everything
Lakebase Public Preview enabled by default
Aug 11, 2025
· release notes
Lakebase: Managed Postgres OLTP Database preview is now enabled by default
Jobs in continuous mode can now have task-level retries for failed tasks
Aug 11, 2025
· release notes
Databricks Jobs in continuous mode now support task-level retries for failed tasks.
Adds the ability to configure individual task retries within continuous jobs. Not checked yet
1 detail in Everything
Access requests in Unity Catalog (Public Preview)
Aug 11, 2025
· release notes
Databricks adds public preview of self-service access request destinations in Unity Catalog
Enables configurable access request destinations (emails, Slack, or Microsoft Teams channels) on Unity Catalog securable objects to automate notification routing for discoverability-driven requests. Not checked yet Adds support for default email destinations so requests automatically route to the catalog or object owner’s email when no specific destination is configured. Not checked yet
2 details in Everything
Serverless workspaces are now Public Preview
Aug 11, 2025
· release notes
Serverless workspaces enter public preview with pre-configured compute and storage
Path credential vending (Public Preview)
Aug 8, 2025
· release notes
Adds path credential vending to Unity Catalog for short-lived external location credentials.
Create external Delta tables from third-party clients (Public Preview)
Aug 8, 2025
· release notes
Public preview allows creating Unity Catalog external Delta tables from third-party clients like Apache Spark
Use OpenAI GPT OSS models for batch inference and structured outputs
Aug 7, 2025
· release notes
Databricks adds batch inference and structured outputs structured outputs Forcing a model's response to match a given format or JSON schema, so software can parse it reliably instead of hoping the text is well formed. for OpenAI GPT OSS OSS Open-source software: code whose licence permits anyone to inspect, modify, and redistribute it. Cyber tools build on it to reuse auditable libraries without licensing fees. 120B and 20B models via AI Functions
Adds support for ai_query() batch inference with OpenAI GPT OSS 120B and GPT OSS 20B models Not checked yet Enables structured outputs specification for inference workloads using OpenAI GPT OSS 120B and GPT OSS 20B Not checked yet
2 details in Everything
Serverless compute is now available in ap-southeast-3
Aug 6, 2025
· release notes
Serverless compute for notebooks and workflows is now available in ap-southeast-3
OpenAI GPT OSS models now available on Mosaic AI Model Serving
Aug 5, 2025
· release notes
Mosaic AI Model Serving adds support for OpenAI GPT OSS OSS Open-source software: code whose licence permits anyone to inspect, modify, and redistribute it. Cyber tools build on it to reuse auditable libraries without licensing fees. 120B and 20B models.
Power BI Databricks connector supports M2M OAuth
Aug 5, 2025
· release notes
Power BI Databricks connector now supports M2M OAuth via Client credentials auth option.
Adds support for .databricks.us/.mil domains, OAuth token exchange OAuth token exchange An OAuth 2.0 extension (RFC 8693) that lets a service swap one token for another with different scopes, audience, or type, so cyber tools can delegate access across service boundaries without re-authenticating the user. , VOID type, and Windows Server 2025.
Added support for Databricks domains cloud.databricks.us and cloud.databricks.mil Not checked yet Enabled OAuth Token exchange Token exchange An OAuth 2.0 protocol extension (RFC 8693) that lets a service trade one token for another with different scopes, audience, or type, enabling delegation and impersonation across service boundaries without re-authenticating the user. for IDPs different from the host Not checked yet Added CRL CRL A Certificate Revocation List is a signed file, published by a certificate authority, listing certificates it has revoked before expiry so that clients can reject compromised or mis-issued credentials without contacting the CA live. (Certificate Revocation Lists) cache support on Windows when UseSystemTruststore is enabled Not checked yet Added VOID type column support, so that VOID columns are now correctly listed in SQLGetColumns Not checked yet Added support for Windows Server 2025 Not checked yet
5 details in Everything
└──▷ BREAKING ON UPGRADE
! The connector no longer supports Databricks Runtime version 10.4 LTS.! Ubuntu 20.04 is no longer supported.
PrivateLink connections from serverless compute to resources in your VPC through an NLB is generally available
Aug 4, 2025
· release notes
Serverless compute can now connect to VPC VPC A logically isolated virtual network inside a public cloud provider's infrastructure, letting tools deploy resources with private addressing, controlled routing, and firewall rules without managing physical hardware. resources via AWS PrivateLink AWS PrivateLink An AWS networking feature that exposes a service over a private VPC endpoint rather than the public internet, letting tools move data between accounts or to AWS services without traffic ever leaving Amazon's backbone. through an NLB.
Edit mode in Assistant does multi-cell code refactoring and more
Aug 1, 2025
· release notes
Databricks Assistant edit mode enables multi-cell code refactoring.
Adds edit mode to Databricks Assistant for reasoning and editing across multiple notebook cells. Not checked yet
1 detail in Everything
Databricks Terraform provider supports Lakebase resources
Jul 31, 2025
· release notes
Databricks Terraform provider adds support for Lakebase resources
Synced tables are now metered and billed
Jul 31, 2025
· release notes
Synced tables are now metered and billed via system.billing.usage
Delta Sharing supports sharing tables and schemas secured by ABAC policies (Beta)
Jul 31, 2025
· release notes
Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. now supports including tables/schemas secured by ABAC ABAC Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. policies in shares Beta .
Adds ability for providers to include tables and schemas protected by attribute-based access control attribute-based access control Attribute-based access control, where rules evaluate facts about the user, resource and request. Access can depend on conditions such as department, data sensitivity or device status. (ABAC) in a Delta share. Not checked yet
1 detail in Everything
New compute policy form (Public Preview)
Jul 31, 2025
· release notes
Databricks introduces a new UI-based compute policy form with dropdowns and separated sections for Advanced options, tagging, and permissions.
Adds New definition dropdown menus to the compute policy form, allowing configuration of rules without referencing raw policy syntax Not checked yet
1 detail in Everything
Sharing streaming tables and materialized views is GA
Jul 30, 2025
· release notes
Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. GA for streaming tables and materialized views reduces limitations for providers and recipients.
Generally available Delta Sharing support for streaming tables Not checked yet Generally available Delta Sharing support for materialized views Not checked yet
2 details in Everything
Jobs & Pipelines list now includes Databricks SQL pipelines
Jul 29, 2025
· release notes
The Jobs & Pipelines list view now includes pipelines created with Databricks SQL , specifically materialized views and streaming tables.
Disable DBFS root and mounts is in Public Preview
Jul 28, 2025
· release notes
New capability to disable DBFS root and mount access in existing workspaces is now available in public preview.
Adds the ability to disable access to the Databricks Filesystem (DBFS) root and mounts for existing workspaces. Not checked yet
1 detail in Everything
Improvements to the notebook editing experience
Jul 25, 2025
· release notes
Databricks adds split view, native find-and-replace, and tab group icons for notebooks
Adds split view to edit notebooks side by side Not checked yet Pressing Cmd + F (Mac) or Ctrl + F (Windows) opens the native Databricks find-and-replace tool in notebooks Not checked yet Adds new icons on the top left of the editor to quickly switch between tab groups based on authoring contexts Not checked yet
3 details in Everything
Expanded region availability for ai_parse_document
Jul 25, 2025
· release notes
Expanded region availability for ai_parse_document function
ai_parse_document is now available in us-west-2, us-east-1, us-east-2, ap-northeast-1, ap-northeast-2, ap-south-1, ap-southeast-1, ap-southeast-2, ca-central-1, eu-central-1, eu-west-1, eu-west-2, sa-east-1. Not checked yet
1 detail in Everything
New columns available in query history system table
Jul 24, 2025
· release notes
Databricks adds cache_origin_statement_id, query_parameters, written_rows, and written_files columns to the query history system table
query_history system table now exposes cache_origin_statement_id for tracing cached query origins Not checked yet query_history system table now includes query_parameters struct showing named and positional parameters Not checked yet query_history system table adds written_rows and written_files columns to track persistent storage writes Not checked yet
3 details in Everything
Moving tables between Spark Declarative Pipelines on Lakeflow pipelines is now GA
Jul 24, 2025
· release notes
Unity Catalog ETL pipelines on Lakeflow now support moving tables between Spark Declarative Pipelines (GA)
Selectively replace data with INSERT REPLACE ON (Public Preview)
Jul 23, 2025
· release notes
Databricks 17.1 adds INSERT REPLACE ON SQL command for selective table data replacement
Dynamic partition overwrite with INSERT REPLACE USING (Public Preview)
Jul 23, 2025
· release notes
Databricks adds INSERT REPLACE USING SQL command for dynamic partition overwrite in Runtime 16.3.
Real-time mode in Structured Streaming (Public Preview)
Jul 22, 2025
· release notes
Databricks adds real-time trigger type for Structured Streaming in public preview.
Adds realtime as a new trigger type to Structured Streaming for sub-second latency processing. Not checked yet
1 detail in Everything
Databricks introduces Supervisor Agent in Beta to orchestrate Genie Agents and Knowledge Assistant endpoints.
Adds Supervisor Agent beta capability for designing multi-agent AI systems Not checked yet Enables integration of Genie Agents (formerly Genie Spaces) with Knowledge Assistant agent endpoints Not checked yet Connects the system to the Review App for natural-language feedback from subject matter experts Not checked yet
3 details in Everything
Explore table data using an LLM (Beta)
Jul 16, 2025
· release notes
Adds Catalog Explorer's Ask a question prompt to generate SQL from natural language using an LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code.
Enables asking natural language questions about sample data in Catalog Explorer, with the Assistant generating SQL based on metadata and usage patterns Not checked yet Adds query validation capability before execution against underlying tables Not checked yet
2 details in Everything
Simplified compute form enabled by default
Jul 15, 2025
· release notes
Databricks enables simplified compute creation form by default for all-purpose and jobs compute in UI
Google Gemma 3 12B now available on Mosaic AI Model Serving
Jul 15, 2025
· release notes
Google Gemma 3 12B added to Mosaic AI Model Serving with API and AI Functions support.
Adds Google Gemma 3 12B as a Databricks-hosted foundation model on Mosaic AI Model Serving Not checked yet Enables Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. pay-per-token for Gemma 3 12B Not checked yet Enables Foundation Model APIs provisioned throughput for Gemma 3 12B Not checked yet Supports AI Functions (real time and batch inference) with Gemma 3 12B Not checked yet
4 details in Everything
The CAN VIEW permission on SQL warehouses is now generally available
Jul 15, 2025
· release notes
CAN VIEW permission on SQL warehouses is generally available
Databricks connector for Power BI now supports the ADBC driver (Public Preview)
Jul 11, 2025
· release notes
Databricks connector for Power BI now supports the ADBC ADBC Arrow Database Connectivity, an Apache standard API for querying databases that returns results directly as Apache Arrow columnar buffers, eliminating per-row copying overhead and making bulk data transfer fast for analytics pipelines. driver alongside ODBC ODBC A standard API, maintained by Microsoft, for connecting applications to relational databases regardless of vendor. Cyber tools use it to query databases without writing engine-specific code. Public preview
Databricks now supports tracking alert changes via Git folders, enabling version control and explicit resume of cloned alerts.
Enables placement of alerts in Databricks Git folders for change management Not checked yet Cloned alerts start with paused schedules requiring explicit resumption by users Not checked yet
2 details in Everything
Expanded feature support for FIPS-Compliant environments
Jul 11, 2025
· release notes
Databricks adds FIPS FIPS A set of US federal standards published by NIST that define approved cryptographic algorithms and modules; cyber tools use it to meet compliance requirements in government and regulated environments. -compliant support for Model Serving pay-per-token, batch inference, and Databricks-hosted models.
Adds pay-per-token endpoint support in FIPS-compliant environments for foundation model API Not checked yet Enables batch_inference capability in FIPS-compliant environments using Databricks-hosted models Not checked yet Extends compliance security profile to include FedRAMP Moderate FedRAMP Moderate A US government authorization tier requiring cloud services to meet a defined set of NIST 800-53 controls before federal agencies can use them, giving cyber tools a pre-vetted compliance baseline for government contracts. , IRAP, and Canada Protected B for Model Serving Not checked yet
3 details in Everything
Databricks Apps support for Lakebase resources
Jul 8, 2025
· release notes
Databricks Apps now supports adding Lakebase database instances as app resources.
Allows access to Lakebase resources as environment variables from app code Not checked yet
1 detail in Everything
Parent tasks ( Run job and For each ) now have a separate limit
Jul 4, 2025
· release notes
Run job and For each parent tasks now have a separate concurrency limit
Run job and For each parent tasks now use a separate limit for the number of simultaneously running child processes, independent of the overall task limit. Not checked yet Parent tasks no longer count against the overall concurrent task limit. Not checked yet
2 details in Everything
Git folders now supports multiple Git credentials per user (Public Preview)
Jul 2, 2025
· release notes
Git folders in Databricks now allows users to configure and manage multiple Git credentials through the workspace UI, supporting one or multiple Git providers.
New Databricks JDBC Driver is open source
Jun 26, 2025
· release notes
Databricks releases its JDBC JDBC A Java standard API for connecting applications to relational databases, maintained as part of the Java SE platform, letting tools query any compliant database through a single consistent interface. driver source code publicly under Apache 2.0 for community contributions
File arrival triggers with file events is now generally available
Jun 25, 2025
· release notes
File arrival triggers with file events GA, raising limits to 1k/triggers per workspace and removing the 10k files/trigger cap
file arrival triggers with file events now generally available for external locations (S3, Azure storage, Google Cloud Storage). Not checked yet Raises the trigger limit to 1,000 per workspace. Not checked yet Removes the previous 10,000 files per trigger limit. Not checked yet
3 details in Everything
Parse documents using ai_parse_document (Beta)
Jun 25, 2025
· release notes
Adds ai_parse_document() beta function to extract structured content from unstructured documents using Databricks Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. .
OIDC federation for Databricks-to-open Delta Sharing is generally available
Jun 24, 2025
· release notes
OIDC OIDC OpenID Connect, a sign-in protocol built on OAuth 2.0. An application uses identity information from a trusted provider instead of maintaining a separate password for each user. federation for Databricks-to-open Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. is generally available
Custom rate limits for model serving endpoints
Jun 23, 2025
· release notes
Databricks adds custom rate limits for model serving endpoints via AI Gateway
Sharing managed Iceberg tables in Delta Sharing is in Public Preview
Jun 23, 2025
· release notes
Databricks now supports sharing managed Iceberg tables via Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. in both Databricks-to-Databricks and open sharing modes during Public Preview.
Migrate Spark Declarative Pipelines on Lakeflow pipelines from legacy publishing mode
Jun 23, 2025
· release notes
Enables migrating Spark Declarative Pipelines on Lakeflow from legacy single catalog/schema to multi-catalog schema publishing mode
Expanded feature support for FIPS-Compliant environments
Jun 20, 2025
· release notes
Databricks expands FIPS FIPS A set of US federal standards published by NIST that define approved cryptographic algorithms and modules; cyber tools use it to meet compliance requirements in government and regulated environments. -Compliant environment support to include Model Serving, Mosaic AI Gateway, and external models
Adds FIPS-compliant support for Model Serving CPU CPU The main general-purpose processor in a computer. It runs application instructions and coordinates work with memory and other hardware. and GPU GPU A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. workloads Not checked yet Adds FIPS-compliant support for Foundation Model API provisioned throughput endpoints (excluding LLaMA 4) Not checked yet
2 details in Everything
AI documentation for Unity Catalog tables and columns now uses the same model as Databricks Assistant
Jun 18, 2025
· release notes
AI documentation for Unity Catalog tables/columns now uses the Databricks-hosted model.
Jobs & Pipelines in the left navigation menu
Jun 18, 2025
· release notes
Jobs & Pipelines unified in left navigation, replacing separate Pipelines and Workflows items
Microsoft SharePoint connector (Beta)
Jun 11, 2025
· release notes
Databricks adds a Microsoft SharePoint connector Beta in Lakeflow Connect for data ingestion.
Managed Apache Iceberg tables are in Public Preview
Jun 11, 2025
· release notes
Managed Apache Iceberg Apache Iceberg An open table format for large analytic datasets, maintained by the Apache Software Foundation, that gives cyber tools reliable schema evolution, time-travel queries, and atomic commits on object storage or HDFS. tables enter public preview with Iceberg REST Catalog API integration.
Enables reading and writing to managed Apache Iceberg tables from Databricks or external engines via the Iceberg REST Catalog API Not checked yet Automatically applies liquid clustering for predictive optimization in managed Apache Iceberg tables Not checked yet
2 details in Everything
Lakebase, a managed PostgreSQL OLTP database, is in Public Preview
Jun 11, 2025
· release notes
Databricks introduces Lakebase as a new managed PostgreSQL OLTP service in public preview, allowing users to run transactional workloads integrated with the platform.
AI Gateway is now generally available
Jun 11, 2025
· release notes
Mosaic AI Gateway is now generally available with governance, monitoring, and traffic routing capabilities.
Mosaic AI Gateway GA provides centralized governance for model serving endpoints. Not checked yet Adds permission and rate limiting controls to manage access and volume. Not checked yet Enables payload logging via inference tables to audit data sent to model APIs. Not checked yet Implements usage tracking using system tables to monitor operational costs. Not checked yet Supports traffic routing to load balance across multiple models. Not checked yet
+1 more
show less
Introduces fallback mechanisms for external models to minimize production outages. Not checked yet
6 details in Everything
Convert to Unity Catalog managed table from external table is in Public Preview
Jun 11, 2025
· release notes
ALTER TABLE ... SET MANAGED now in Public Preview for converting external tables to Unity Catalog managed tables
Foreign Apache Iceberg tables are in Public Preview
Jun 11, 2025
· release notes
Foreign Apache Iceberg Apache Iceberg An open table format for large analytic datasets, maintained by the Apache Software Foundation, that gives cyber tools reliable schema evolution, time-travel queries, and atomic commits on object storage or HDFS. tables are in Public Preview via Lakehouse Federation Lakehouse Federation A data architecture pattern that lets a single query engine read from multiple external data sources—databases, data lakes, warehouses—without moving or copying the data, reducing pipeline complexity for tools that need broad data access.
Reads Iceberg tables managed by foreign catalogs (HMS, Glue, Snowflake Horizon Catalog) using Lakehouse Federation Not checked yet Applies Unity Catalog advanced features (fine-grained access controls, lineage, and auditing) to these foreign Iceberg tables Not checked yet
2 details in Everything
Databricks v2025.06 introduces AUTO CDC APIs (AUTO CDC INTO, create_auto_cdc_flow) replacing APPLY CHANGES for Lakeflow Spark Declarative Pipelines.
Adds AUTO CDC INTO SQL API to create flows supporting change data feeds in Spark Declarative Pipelines on Lakeflow Not checked yet Adds Python APIs create_auto_cdc_flow and create_auto_cdc_from_snapshot_flow for automated change data feed change data feed A stream of records describing every insert, update, and delete that occurs in a database, captured in real time so downstream systems can react to changes without polling. management Not checked yet
2 details in Everything
Serverless performance targets is now GA
Jun 10, 2025
· release notes
Serverless performance targets GA: enables Performance optimized setting for jobs and pipelines.
Databricks Tag Policies in Beta allows admins to enforce consistent tagging across catalogs, schemas, and tables at the account level.
Tag policies is now in Beta, enabling admins to define allowed tag keys/values and control which users/groups can assign them across all workspaces. Not checked yet
1 detail in Everything
Configure PrivateLink connections from serverless compute to S3 Storage buckets (Public Preview)
Jun 9, 2025
· release notes
Databricks serverless compute can now use AWS PrivateLink AWS PrivateLink An AWS networking feature that exposes a service over a private VPC endpoint rather than the public internet, letting tools move data between accounts or to AWS services without traffic ever leaving Amazon's backbone. to access S3 buckets privately.
Cross-platform view sharing is now GA
Jun 9, 2025
· release notes
Databricks 2025-06: Cross-platform view sharing via Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. GA, OpenSharing costs, and materialization history system table.
Enables cross-platform view sharing via Delta Sharing for general availability. Not checked yet Introduces OpenSharing cost tracking and configuration for data access and billing methods when sharing views. Not checked yet Adds a new system table to track shared materialized data history. Not checked yet
3 details in Everything
Manage network policies for serverless egress control (Generally Available)
Jun 9, 2025
· release notes
Databricks GA: Manage network policies for serverless egress control to allowlist domains and monitor denials.
Enables configuration and enforcement of outbound network policies for serverless compute resources including SQL warehouses and model serving endpoints. Not checked yet Adds ability to allowlist specific domains and storage accounts in restricted mode. Not checked yet Supports a dry-run mode to monitor policy impact before enforcement without blocking traffic. Not checked yet Allows viewing and analyzing denial logs in Unity Catalog for auditing and troubleshooting outbound egress. Not checked yet
4 details in Everything
PrivateLink connections from serverless compute to resources in your VPC through an NLB is in Public Preview
Jun 6, 2025
· release notes
Databricks adds Public Preview of PrivateLink PrivateLink An AWS networking feature that exposes a service over a private endpoint inside a VPC, so traffic never traverses the public internet, giving cyber tools a way to integrate with cloud services without opening public routes. connections from serverless compute to VPC VPC A logically isolated virtual network inside a public cloud provider's infrastructure, letting tools deploy resources with private addressing, controlled routing, and firewall rules without managing physical hardware. resources via NLB.
Enables AWS PrivateLink AWS PrivateLink An AWS networking feature that exposes a service over a private VPC endpoint rather than the public internet, letting tools move data between accounts or to AWS services without traffic ever leaving Amazon's backbone. connectivity from serverless compute to VPC resources through a network load balancer (NLB). Not checked yet
1 detail in Everything
Databricks Apps support for compliance security profile standards
Jun 6, 2025
· release notes
Databricks Apps now supports HIPAA HIPAA A US federal law enacted in 1996 that sets privacy and security requirements for protected health information, giving cyber tools a compliance baseline to enforce or audit against. , PCI-DSS PCI-DSS The Payment Card Industry Data Security Standard, security requirements for protecting payment account data. It covers organizations that store, process or transmit that data and relevant service providers. , and FedRAMP Moderate FedRAMP Moderate A US government authorization tier requiring cloud services to meet a defined set of NIST 800-53 controls before federal agencies can use them, giving cyber tools a pre-vetted compliance baseline for government contracts. compliance security profile standards
Serverless GPU GPU A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. compute is now in Beta for single and multi-node deep learning workloads.
Enables Serverless GPU compute within the Serverless compute offering for custom deep learning training and fine-tuning. Not checked yet
1 detail in Everything
New region: AWS Asia Pacific (Jakarta) ( ap-southeast-3 )
Jun 5, 2025
· release notes
Databricks now supports AWS Asia Pacific (Jakarta) region for new workspace deployment
New consumer entitlement is generally available
Jun 5, 2025
· release notes
Databricks introduces new consumer entitlement for limited read-only workspace access.
Workspace admins can grant consumer access as an entitlement to users, service principals, and groups. Not checked yet
1 detail in Everything
Salesforce Data Cloud File Sharing connector (Public Preview)
Jun 4, 2025
· release notes
Adds Salesforce Data Cloud File Sharing connector Public preview for zero-copy file federation.
Mosaic AI Vector Search storage-optimized endpoints are Public Preview
Jun 3, 2025
· release notes
Mosaic AI Vector Search introduces storage-optimized endpoints with over one billion vector capacity and faster indexing in Public Preview.
Adds storage-optimized endpoint option to Mosaic AI Vector Search for large-scale vector storage. Not checked yet Enables indexing performance gains of 10-20x with the new storage-optimized endpoints. Not checked yet
2 details in Everything
Unity Catalog HTTP connections support OAuth User-to-Machine Per User credentials (Public Preview)
Jun 2, 2025
· release notes
Unity Catalog HTTP connections now support per-user OAuth credentials for external service access.
Adds per-user OAuth credential flow to Unity Catalog HTTP connections Not checked yet Replaces shared single-credential OAuth with individual user identities Not checked yet
2 details in Everything
Improved UI for managing notebook dashboards
May 30, 2025
· release notes
Improved UI for managing notebook dashboards
Adds quick navigation between a notebook and its associated dashboards via the top right corner. Not checked yet
1 detail in Everything
SQL editor gains result filtering for visuals, query generation from results, column expansion hover, and custom formatting settings
└──▷ HOW TO FIND IT
Access custom SQL formatting settings via the menu path.
📍 View › Developer Settings › SQL Format
Adds View > Developer Settings > SQL Format menu path to access custom SQL formatting settings in the new SQL editor and notebook editor. Not checked yet Enables filters applied to result tables to also affect visualizations for interactive exploration. Not checked yet Adds ability to create a new query from a filtered results table or visualization in a SQL notebook. Not checked yet Adds hover-over * feature in a SELECT * query to expand the columns in the queried table. Not checked yet
4 details in Everything
Knowledge Assistant for domain-specialized chatbots is in Beta
May 29, 2025
· release notes
Databricks Knowledge Assistant enters Beta for domain-specialized, feedback-driven chatbots
└──▷ HOW TO FIND IT
Create a domain-specialized Q&A chatbot and improve its accuracy through SME feedback loops.
📍 See Use Knowledge Assistant to create a high-quality chatbot over your documents
Knowledge Assistant Beta enables creation of question-and-answer chatbots over documents with quality improvement via natural language feedback. Not checked yet
1 detail in Everything
Workspaces system table now available (Public Preview)
May 28, 2025
· release notes
Databricks Workspaces system table (system.access.workspaces_latest) now available in public preview for monitoring active workspaces.
Adds system.access.workspaces_latest system table to monitor the latest state of all active workspaces in your account Not checked yet Enables joining system.access.workspaces_latest with other system tables to analyze reliability, performance, and cost across workspaces Not checked yet
2 details in Everything
Claude Sonnet 4 and Claude Opus models now available on Mosaic AI Model Serving
May 27, 2025
· release notes
Mosaic AI Model Serving expands its supported model portfolio by adding new foundation models.
Adds Claude Sonnet 4 model availability in Mosaic AI Model Serving for Databricks-hosted foundation models. Not checked yet Adds Claude Opus 4 model availability in Mosaic AI Model Serving for Databricks-hosted foundation models. Not checked yet
2 details in Everything
FedRAMP Moderate support in additional AWS regions
May 23, 2025
· release notes
Databricks adds FedRAMP Moderate FedRAMP Moderate A US government authorization tier requiring cloud services to meet a defined set of NIST 800-53 controls before federal agencies can use them, giving cyber tools a pre-vetted compliance baseline for government contracts. support in AWS us-east-2 and us-west-1 regions
Databricks SQL introduces a consolidated Beta interface for managing alerts, combining query setup, conditions, schedules, and notifications.
Adds a new consolidated UI for Databricks SQL alerts in Beta, unifying query setup, condition definition, scheduling, and notification destinations into a single interface. Not checked yet
1 detail in Everything
Account admins in pay-as-you-go accounts can manage payment methods and view credits via the updated Subscription and billing tab.
Adds the ability to add and update payment methods (credit cards and AWS Marketplace accounts) in the Subscription and billing tab. Not checked yet Enables switching between payment methods at any time from the Subscription and billing tab. Not checked yet Exposes active and expired credits, including free trial credits, in the Subscription and billing tab. Not checked yet
3 details in Everything
Pipelines system table is now available (Public Preview)
May 20, 2025
· release notes
New system.lakeflow.pipelines system table Public preview tracks pipeline history as SCD2.
Declarative Automation Bundles in the workspace (Public Preview)
May 19, 2025
· release notes
Declarative Automation Bundles in the workspace Public preview
You can now collaborate with multiple parties with Databricks Clean Rooms
May 14, 2025
· release notes
Databricks Clean Rooms adds multi-party collaboration support with new notebook approval workflows.
Adds support for up to 10 collaborators in Databricks Clean Rooms for complex, multi-party data projects Not checked yet Introduces a new notebook approval workflow allowing designated runners and requiring explicit approval before execution Not checked yet Adds auto-approval options for trusted partners within the notebook approval workflow Not checked yet Provides difference views for easy review and auditing of changes in Databricks Clean Rooms Not checked yet
4 details in Everything
Serverless compute is now available in us-west-1
May 14, 2025
· release notes
Serverless compute is now available in us-west-1 for SQL warehouses, notebooks, jobs, and DLT.
Enables serverless compute for SQL warehouses in the us-west-1 region Not checked yet Enables serverless compute for notebooks, jobs, and DLT in the us-west-1 region Not checked yet
2 details in Everything
Model Serving CPU and GPU workloads now support compliance security profile standards
May 13, 2025
· release notes
Databricks Model Serving workloads now support HIPAA HIPAA A US federal law enacted in 1996 that sets privacy and security requirements for protected health information, giving cyber tools a compliance baseline to enforce or audit against. , PCI-DSS PCI-DSS The Payment Card Industry Data Security Standard, security requirements for protecting payment account data. It covers organizations that store, process or transmit that data and relevant service providers. , and UK Cyber Essentials Cyber Essentials A UK government-backed certification scheme that defines a baseline set of technical controls organizations must implement to defend against common cyber threats, giving cyber tools a compliance benchmark to assess or report against. Plus via compliance security profiles
Adds HIPAA to the list of supported compliance standards for Model Serving CPU CPU The main general-purpose processor in a computer. It runs application instructions and coordinates work with memory and other hardware. and GPU GPU A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. workloads. Not checked yet Adds PCI-DSS to the list of supported compliance standards for Model Serving CPU and GPU workloads. Not checked yet Adds UK Cyber Essentials Plus to the list of supported compliance standards for Model Serving CPU and GPU workloads. Not checked yet
3 details in Everything
Databricks multi-factor authentication (MFA) is now generally available
May 13, 2025
· release notes
Databricks allows account admins to require or recommend MFA MFA Multi-factor authentication, requiring proof from different categories, such as a password and a hardware key, to sign in. It reduces reliance on a password alone. for all users at login
Databricks Apps (Generally Available)
May 13, 2025
· release notes
Databricks Apps reaches GA, enabling interactive full-stack applications in Databricks workspaces.
Provisioned throughput workloads now support compliance security profile standards
May 13, 2025
· release notes
Databricks Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. provisioned throughput now supports HIPAA HIPAA A US federal law enacted in 1996 that sets privacy and security requirements for protected health information, giving cyber tools a compliance baseline to enforce or audit against. , PCI-DSS PCI-DSS The Payment Card Industry Data Security Standard, security requirements for protecting payment account data. It covers organizations that store, process or transmit that data and relevant service providers. , and UK Cyber Essentials Cyber Essentials A UK government-backed certification scheme that defines a baseline set of technical controls organizations must implement to defend against common cyber threats, giving cyber tools a compliance benchmark to assess or report against. Plus compliance profiles
Adds HIPAA to the list of compliance security profile standards available for Foundation Model APIs on provisioned throughput workloads Not checked yet Adds PCI-DSS to the list of compliance security profile standards available for Foundation Model APIs on provisioned throughput workloads Not checked yet Adds UK Cyber Essentials Plus to the list of compliance security profile standards available for Foundation Model APIs on provisioned throughput workloads Not checked yet
3 details in Everything
Databricks JDBC JDBC A Java standard API for connecting applications to relational databases, maintained as part of the Java SE platform, letting tools query any compliant database through a single consistent interface. 2.7.3 adds Azure Managed Identity OAuth, token exchange token exchange An OAuth 2.0 protocol extension (RFC 8693) that lets a service trade one token for another with different scopes, audience, or type, enabling delegation and impersonation across service boundaries without re-authenticating the user. , VOID/Variant/TIMESTAMP_NTZ types, and cloud.databricks.us/mil support.
Added Auth_Flow=3 property for Azure Managed Identity OAuth 2.0 OAuth 2.0 An open authorization framework (IETF RFC 6749) that lets a service grant limited API access to a third party without sharing credentials, giving cyber tools a standard way to authenticate against external platforms. authentication Not checked yet Added OAuth Token Exchange for IDPs different than host to exchange access tokens access tokens A credential that authorizes a client to use a service or API. It usually has limited permissions and a lifetime; anyone holding it may be able to use those permissions. for a Databricks access token access token A credential that authorizes a client to use a service or API. It usually has limited permissions and a lifetime; anyone holding it may be able to use those permissions. Not checked yet Added VOID, Variant , and TIMESTAMP_NTZ data type support in getColumns() and getTypeInfo() APIs Not checked yet The driver now lists columns with unknown or unsupported types and maps them to SQL VARCHAR in the getColumns() metadata API Not checked yet Added support for cloud.databricks.us and cloud.databricks.mil domains when connecting via OAuth (AuthMech=11) Not checked yet
5 details in Everything
Updated lineage system tables schema
May 11, 2025
· release notes
Databricks lineage system tables updated with new primary key and metadata columns for May 2025
system.access.column_lineage and system.access.table_lineage now include a new record_id column as the primary key for each lineage record. Not checked yet entity_metadata column is added to system.access.column_lineage and system.access.table_lineage, replacing the deprecated entity_type, entity_run_id, and entity_id columns. Not checked yet event_id column is added to log the identifier for the lineage event across multiple related rows. Not checked yet
3 details in Everything
└──▷ BREAKING ON UPGRADE
! The entity_type, entity_run_id, and entity_id columns in system.access.column_lineage and system.access.table_lineage are deprecated and replaced by entity_metadata.
Databricks Runtime 16.4 LTS adds cloudFiles.cleanSource to Auto Loader, Python DataSourceReader.pushFilters API, and listagg/string_agg SQL functions.
└──▷ USE IT
Use the new IDENTIFIER clause to dynamically specify a catalog name in DBSQL
sql
copy
CREATE CATALOG IDENTIFIER(:param)
Adds the cloudFiles.cleanSource Auto Loader option to instruct the stream to move or delete processed files from the source directory Not checked yet Adds pushFilters method to the DataSourceReader Python interface to allow filter pushdown for batch reads Not checked yet Supports MERGE INTO on tables with fine-grained access control on dedicated compute, now generally available (GA) Not checked yet Adds support for using IDENTIFIER clause in DBSQL catalog operations such as CREATE CATALOG, DROP CATALOG, COMMENT ON CATALOG, and ALTER CATALOG Not checked yet Adds new SQL aggregate functions listagg and string_agg to aggregate STRING and BINARY values within a group Not checked yet
5 details in Everything
└──▷ BREAKING ON UPGRADE
! Files with a .gzip extension are now read as gzip-compressed files; previously uncompressed data in .gzip files will fail with java.io.IOException: incorrect header check unless the extension is renamed or decompression is skipped using io.compression.codecs=org.apache.hadoop.io.compress.PassthroughCodec! Data source table reads now respect options set for all plans when cached; previously only the first cached plan's options were used, so queries changing options like delimiter on a cached table may produce different results! Redaction rule moved from analyzer to optimizer; DataFrames saved with valid SECRET SQL functions will no longer contain redacted values in the target table
Automatically provision users (JIT) GA
May 9, 2025
· release notes
Databricks GA: Automatically provision users via Just-in-Time (JIT JIT A runtime compilation strategy where code is translated from an intermediate form to native machine instructions at execution time, giving tools the speed of compiled code without sacrificing dynamic flexibility. ) provisioning on first SSO SSO Single sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. login
Enables just-in-time (JIT) provisioning to automatically create new user accounts during first-time authentication for Single Sign-On (SSO) users. Not checked yet
1 detail in Everything
Configure Python syntax highlighting in Databricks notebooks
May 8, 2025
· release notes
Databricks adds pyproject.toml-based configuration for Python syntax highlighting and linters
Adds pyproject.toml support in notebook ancestor paths or home folder to configure Python syntax highlighting Not checked yet Enables configuration of ruff, pylint, pyright, and flake8 linters via the new config file Not checked yet Allows disabling Databricks-specific linting rules through the same pyproject.toml configuration Not checked yet
3 details in Everything
Jobs and pipelines now share a single, unified view (Public Preview)
May 7, 2025
· release notes
Databricks adds a unified view to display jobs, ETL, and ingestion pipelines in one list Public preview .
Llama 4 Maverick is now supported on provisioned throughput workloads (Public Preview)
May 5, 2025
· release notes
Llama 4 Maverick support added to Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. on provisioned throughput workloads Public preview
The jobs system tables (Public Preview) are enabled by default
May 1, 2025
· release notes
The system.lakeflow schema containing jobs system tables is now enabled by default in all Unity Catalog workspaces
system.lakeflow system tables are now available by default for querying job metadata and state Not checked yet
1 detail in Everything
Stats collection is now automated by predictive optimization
Apr 30, 2025
· release notes
Predictive optimization now handles stats collection automatically for Unity Catalog managed tables.
Google Analytics raw data connector (Public Preview)
Apr 25, 2025
· release notes
Adds Google Analytics raw data connector via Lakeflow Connect and Google BigQuery BigQuery A serverless, cloud-hosted analytical database from Google that runs SQL queries across massive datasets without managing infrastructure, giving cyber tools scalable storage and fast querying over large volumes of log or event data.
Claude 3.7 Sonnet now available in EU regions
Apr 25, 2025
· release notes
Claude 3.7 Sonnow available in EU regions for Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. and AI Functions
Enables Claude 3.7 Sonnet access in EU regions for pay-per-token or AI Functions workloads Not checked yet
1 detail in Everything
Share streaming tables and materialized views to all recipients using Delta Sharing (Public Preview)
Apr 23, 2025
· release notes
Databricks adds capability to share streaming tables and materialized views via Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. Public preview
Use vector search indexes to prototype tool-calling AI agents in AI Playground
Apr 23, 2025
· release notes
Adds vector search index as a direct tool in AI Playground agents without custom Unity Catalog functions
AI Gateway now supports custom model serving endpoints
Apr 21, 2025
· release notes
Mosaic AI Gateway now supports custom model serving endpoints with governance features.
Adds support for custom model serving endpoints in Mosaic AI Gateway Not checked yet Enables permission and rate limiting on custom model serving endpoints Not checked yet Enables payload logging to inference tables for custom model serving endpoints Not checked yet Enables usage tracking via system tables for custom model serving endpoints Not checked yet
4 details in Everything
Strict enforcement of row-level security and column masking policies in Delta Sharing
Apr 21, 2025
· release notes
Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. strictly enforces row-level security and column masking policies on shared assets regardless of application order.
Ensures that row filters and column masks applied to tables underlying a Delta Sharing data asset are consistently enforced on recipients, even if the policies were added after the asset was shared. Not checked yet
1 detail in Everything
Run code in Databricks Assistant on serverless compute
Apr 18, 2025
· release notes
Databricks Assistant now runs generated code on serverless compute
AWS PrivateLink now supported in us-west-1 region
Apr 18, 2025
· release notes
AWS PrivateLink AWS PrivateLink An AWS networking feature that exposes a service over a private VPC endpoint rather than the public internet, letting tools move data between accounts or to AWS services without traffic ever leaving Amazon's backbone. now supported in us-west-1 region
Adds us-west-1 to the list of regions where AWS PrivateLink PrivateLink An AWS networking feature that exposes a service over a private endpoint inside a VPC, so traffic never traverses the public internet, giving cyber tools a way to integrate with cloud services without opening public routes. is available for Databricks workspaces. Not checked yet
1 detail in Everything
Custom LLM for custom AI text agents is in Beta
Apr 17, 2025
· release notes
Databricks Beta adds Custom LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. support for creating specialized AI text agents
Adds Custom LLM in Beta to create specialized AI agents for text tasks like summarization and classification Not checked yet Enables content generation and text transformation capabilities through the new Custom LLM feature Not checked yet
2 details in Everything
Access UDF context information using TaskContext
Apr 14, 2025
· release notes
Access user identity and cluster tags in Unity Catalog Python UDFs via the TaskContext API.
Databricks Assistant provides doc citations and follow-up suggestions
Apr 3, 2025
· release notes
Databricks Assistant adds doc citations and follow-up suggestions to responses
Assistant now includes citations from documentation in its answers Not checked yet Assistant now provides follow-up suggestion prompts for next steps Not checked yet
2 details in Everything
Databricks multi-factor authentication (MFA) is now available (Public Preview)
Apr 1, 2025
· release notes
Databricks introduces native MFA MFA Multi-factor authentication, requiring proof from different categories, such as a password and a hardware key, to sign in. It reduces reliance on a password alone. for non-SSO SSO Single sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. accounts in public preview
Enables Account admins to enable multi-factor authentication multi-factor authentication Multi-factor authentication, requiring proof from different categories, such as a password and a hardware key, to sign in. It reduces reliance on a password alone. (MFA) directly within Databricks for users without SSO. Not checked yet Allows configuration of MFA policy as either recommend or require for user login flows. Not checked yet
2 details in Everything
The compute system tables are now generally available (GA)
Apr 1, 2025
· release notes
Databricks compute system tables (clusters, node_types, node_timeline) reach GA
clusters compute system table is now GA for all-purpose compute, jobs compute, and DLT pipelines insights. Not checked yet node_types compute system table is now GA for all-purpose compute, jobs compute, and DLT pipelines insights. Not checked yet node_timeline compute system table is now GA for all-purpose compute, jobs compute, and DLT pipelines insights. Not checked yet
3 details in Everything
The BROWSE privilege is GA for Unity Catalog objects.
BROWSE privilege is now generally available, allowing users to view metadata without read access. Not checked yet Supports viewing object metadata in Catalog Explorer , the schema browser, search results, the lineage graph, information_schema, and the REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. . Not checked yet
2 details in Everything
Disable your legacy Hive metastore (Public Preview)
Mar 31, 2025
· release notes
Public Preview allows disabling legacy Hive metastore Hive metastore A central metadata repository, part of the Apache Hive project, that stores schema and location information for distributed data sets so query engines can discover and read structured data without hardcoding paths. direct access to enforce Unity Catalog governance
Adds the ability to disable direct access to the legacy Hive metastore in a Databricks workspace as a Public Preview feature. Not checked yet Enforces that all data access and governance for the workspace flows through Unity Catalog once the legacy metastore is disabled. Not checked yet
2 details in Everything
Claude Sonnet 3.7 now supported on Mosaic AI Model Serving
Mar 26, 2025
· release notes
Mosaic AI Model Serving now supports Anthropic's Claude Sonnet 3.7 model
Additional audit logging for Databricks Apps events (Public Preview)
Mar 25, 2025
· release notes
Databricks Apps audit logging now captures createApp, installTemplateApp, updateApp, startApp, stopApp, deployApp, and deleteApp events in Public Preview.
Group clusters are generally available (GA)
Mar 25, 2025
· release notes
Databricks Group Clusters GA: assign all-purpose and jobs compute resources to groups via Dedicated access mode
Databricks Information Extraction enters Beta for transforming unlabeled text into structured tables
New CAN VIEW permission available for SQL warehouses (Public Preview)
Mar 25, 2025
· release notes
Adds CAN VIEW permission for SQL warehouses in public preview to allow monitoring without query execution.
Adds the CAN VIEW permission type for SQL warehouse access control lists, enabling users to monitor query history and profiles without executing queries. Not checked yet
1 detail in Everything
Databricks adds a column explorer for tabular notebook outputs with search, pin, and rearrange capabilities
AI Gateway now supports pay-per-token endpoints
Mar 24, 2025
· release notes
Mosaic AI Gateway adds governance and monitoring features for pay-per-token endpoints.
Enables permission and rate limiting on Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. pay-per-token endpoints Not checked yet Adds payload logging via inference tables to monitor data sent to model APIs Not checked yet Adds usage tracking via system tables to monitor operational usage and costs Not checked yet
3 details in Everything
Databricks Connector for Google Sheets (Public Preview)
Mar 20, 2025
· release notes
Databricks Connector for Google Sheets enters Public Preview
Introduces Databricks Connector for Google Sheets to allow connecting from Google Sheets to a Databricks workspace Not checked yet
1 detail in Everything
Delta Sharing supports renaming tables
Mar 19, 2025
· release notes
Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. now supports renaming shared tables with automatic reconciliation for recipients.
AutoML supports clusters in dedicated access mode assigned to groups
Mar 17, 2025
· release notes
AutoML now supports clusters in dedicated access mode assigned to groups
OIDC federation for Databricks-to-open Delta Sharing (Public Preview)
Mar 17, 2025
· release notes
OIDC OIDC OpenID Connect, a sign-in protocol built on OAuth 2.0. An application uses identity information from a trusted provider instead of maintaining a separate password for each user. federation for Databricks-to-open Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. allows recipients to use their own IdP IdP A service that authenticates users and supplies identity information to other applications. It is the sign-in authority behind many single sign-on systems.
Hive metastore federation is generally available
Mar 14, 2025
· release notes
Databricks GA adds Hive metastore Hive metastore A central metadata repository, part of the Apache Hive project, that stores schema and location information for distributed data sets so query engines can discover and read structured data without hardcoding paths. federation with support for HMS 3.1, Postgres/SQL Server metadata stores, row/column masking, Delta shallow clones, and cross-workspace DBFS
Adds Hive metastore 3.1 as a supported version for Hive metastore federation Not checked yet Adds Postgres and SQL Server as supported metadata stores for Hive metastore federation (in addition to MySQL) Not checked yet Enables applying row filters and column-level masks to Hive tables accessed via Hive metastore federation Not checked yet Adds preview support for reading Delta shallow clones defined in the Hive metastore Not checked yet Adds private preview support for reading DBFS-mount-backed tables across workspaces Not checked yet
5 details in Everything
Download as Excel is now supported in notebooks connected to serverless compute
Mar 13, 2025
· release notes
Adds Download as Excel for notebook tabular results on serverless compute
Compute logs can now be delivered to volumes (Public Preview)
Mar 13, 2025
· release notes
Databricks adds ability to deliver compute resource logs to Unity Catalog volumes
Adds support for delivering driver, worker, and event logs from compute resources to a Unity Catalog volume. Not checked yet
1 detail in Everything
Bind storage credentials, service credentials, and external locations to specific workspaces (GA)
Mar 12, 2025
· release notes
Databricks GA allows binding storage credentials, service credentials, and external locations to specific workspaces.
Genie Conversation API is now in Public Preview
Mar 11, 2025
· release notes
Genie Conversations API enters Public Preview, enabling question-asking and SQL generation via REST.
Genie Conversation API is now in Public Preview, allowing users to ask questions, retrieve generated SQL and query results, or get details associated with a Genie Space using the Databricks REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. . Not checked yet
1 detail in Everything
New open source Databricks JDBC Driver is GA
Mar 11, 2025
· release notes
Databricks JDBC JDBC A Java standard API for connecting applications to relational databases, maintained as part of the Java SE platform, letting tools query any compliant database through a single consistent interface. Driver v3 is generally available and will soon be open source
GA release of the new Databricks JDBC Driver version 3 via Maven Repository Not checked yet
1 detail in Everything
Perform batch inference using AI Functions (Public Preview)
Mar 10, 2025
· release notes
Databricks Mosaic AI now supports batch inference via AI Functions Public preview , enabling deployment of pipelines and transformation of unstructured data.
External models now supports OpenAI-compatible LLMs
Mar 10, 2025
· release notes
External models in Mosaic AI Model Serving now supports arbitrary OpenAI-compatible custom providers.
Adds the ability to use custom provider options for external models that are compatible with the OpenAI API but not explicitly listed. Not checked yet Expands model provider coverage in Mosaic AI Model Serving to include alternate providers behind custom proxies. Not checked yet
2 details in Everything
AI Gateway now supports fallbacks for external models
Mar 4, 2025
· release notes
AI Gateway now supports fallback configurations for external models.
Databricks ODBC ODBC A standard API, maintained by Microsoft, for connecting applications to relational databases regardless of vendor. Cyber tools use it to query databases without writing engine-specific code. 2.9.1 adds idle statement timeout and token passthrough renewal
Adds driver-side timeout option for idle statements to prevent hanging during result fetching. Not checked yet Adds token renewal support for the token passthrough OAuth flow. Not checked yet
2 details in Everything
Databricks Assistant provides table insights and lineage
Feb 27, 2025
· release notes
Databricks Assistant now provides table lineage and insights via Genie Code in Unity Catalog workspaces.
Delta Sharing from open sharing providers to Databricks using OAuth
Feb 26, 2025
· release notes
Databricks adds OAuth Client Credentials OAuth Client Credentials An OAuth 2.0 grant type where a service authenticates directly with an authorization server using its own ID and secret, receiving a token to call APIs without any user involved. support for Open-to-Databricks Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. imports
Import open Delta Sharing shares directly into Databricks using the Import provider dialog
Feb 26, 2025
· release notes
New Import provider dialog to import Open Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. shares into Unity Catalog
MLflow Tracing is GA on Databricks, enabling tracking of inputs, outputs, and metadata for model/agent requests to debug bugs and compare performance.
MLflow Tracing is generally available for tracking inputs, outputs, and metadata associated with each step of a model or agent request. Not checked yet
1 detail in Everything
Event time ordering for initial snapshot stream processing is GA
Feb 25, 2025
· release notes
Databricks makes withEventTimeOrder GA for Structured Streaming Delta sources.
Adds the withEventTimeOrder option to Structured Streaming queries reading from Delta tables, enabling event-time ordering during initial snapshot processing. Not checked yet
1 detail in Everything
Streaming reads on Unity Catalog views is GA
Feb 25, 2025
· release notes
Streaming reads on Unity Catalog views is GA
OAuth secret lifetime now configurable
Feb 24, 2025
· release notes
OAuth secrets for service principals now have a configurable maximum lifetime of up to two years.
Model serving billing records are now logged at five-minute intervals
Feb 18, 2025
· release notes
Model serving billing records now logged at five-minute intervals to improve cost observability
Improved model serving billing aggregation frequency from hourly to five-minute intervals. Not checked yet Improved provisioned throughput usage billing aggregation frequency from hourly to five-minute intervals. Not checked yet
2 details in Everything
Support of Scala streaming foreach , foreachBatch , and flatMapGroupsWithState on standard access mode compute (formerly shared access mode)
Feb 7, 2025
· release notes
Scala streaming methods foreach, foreachBatch, and flatMapGroupsWithState now supported on standard access mode compute.
DataStreamWriter.foreach is available for Scala streaming on Databricks Runtime 16.1+ in standard access mode compute Not checked yet DataStreamWriter.foreachBatch is available for Scala streaming on Databricks Runtime 16.2+ in standard access mode compute Not checked yet KeyValueGroupedDataset.flatMapGroupsWithState is available for Scala streaming on Databricks Runtime 16.2+ in standard access mode compute Not checked yet
3 details in Everything
Unity Catalog -governed access to external cloud services using service credentials is now GA
Feb 7, 2025
· release notes
Unity Catalog GA: Service credentials for external cloud access now support Scala SDKs
service_credentials for Unity Catalog-governed access to external cloud services is now generally available (GA) Not checked yet Scala SDKs are now supported alongside Python SDKs for service credential authentication Not checked yet
2 details in Everything
Download as Excel is now supported in notebooks connected to SQL warehouses
Feb 6, 2025
· release notes
Notebooks connected to SQL warehouses can now export tabular cell results as Excel files
Adds Download as Excel support for notebook cells with tabular data connected to SQL warehouses Not checked yet
1 detail in Everything
DLT now supports publishing to tables in multiple schemas and catalogs
Feb 5, 2025
· release notes
DLT pipelines default to multi-catalog/schema publishing, removing mandatory LIVE virtual schema syntax
New pipelines now support creating and updating materialized views and streaming tables in multiple catalogs and schemas by default. Not checked yet The pipeline configuration now requires specifying a target schema which becomes the default for the pipeline. Not checked yet The LIVE virtual schema requirement has been removed, allowing use of standard SQL syntax without it. Not checked yet
3 details in Everything
Write data from pipelines to external services with DLT sinks (Public Preview)
Feb 5, 2025
· release notes
DLT sinks enter Public Preview, enabling writes from pipelines to Kafka, Event Hubs, Unity Catalog tables, and Hive metastore Hive metastore A central metadata repository, part of the Apache Hive project, that stores schema and location information for distributed data sets so query engines can discover and read structured data without hardcoding paths. tables
Adds DLT sink API in Public Preview for writing transformed pipeline data to external targets Not checked yet Supports Azure Event Hubs as a sink target via the new DLT sink capability Not checked yet Supports Apache Kafka as a sink target via the new DLT sink capability Not checked yet Supports Unity Catalog managed tables as a sink target via the new DLT sink capability Not checked yet Supports Hive metastore managed tables as a sink target via the new DLT sink capability Not checked yet
5 details in Everything
Databricks Runtime 14.3 LTS+ GA for querying Structured Streaming state data and metadata
Predictive optimization can now be enabled at the catalog or schema level
Jan 31, 2025
· release notes
Databricks allows predictive optimization to be enabled at the catalog or schema level without account-level enablement.
Adds the ability to enable predictive optimization independently at the catalog or schema level. Not checked yet
1 detail in Everything
Databricks on AWS GovCloud DoD is now GA
Jan 30, 2025
· release notes
Databricks on AWS GovCloud DoD reaches GA for DoD agencies with FedRAMP High FedRAMP High A US federal authorization tier for cloud services, managed by GSA, certifying that a provider meets the strictest NIST 800-53 controls required to handle the most sensitive unclassified government data. and IL5 requirements.
Makes Databricks on AWS GovCloud DoD generally available. Not checked yet
1 detail in Everything
Filtering full datasets for large tables is now supported
Jan 30, 2025
· release notes
Adds option to apply filters to full datasets in large tables instead of truncated data.
Supports applying filters to the entire dataset for large tables (output >2MB or >10,000 rows). Not checked yet
1 detail in Everything
Connect AI agent tools to external services (Public Preview)
Jan 29, 2025
· release notes
Public preview of connecting AI agent tools to external services via HTTP requests
AI agent tools can now connect to external applications like Slack, Google Calendar, or any service with an API using HTTP requests Not checked yet
1 detail in Everything
Databricks Clean Rooms GA introduces management APIs, single-metastore self-collaboration, Azure output tables, HIPAA HIPAA A US federal law enacted in 1996 that sets privacy and security requirements for protected health information, giving cyber tools a compliance baseline to enforce or audit against. profiles, and federated sharing preview.
Adds Management APIs to automate Clean Rooms setup, orchestration, and monitoring Not checked yet Adds Self-Collaboration feature to create Clean Rooms in a single metastore for testing Not checked yet Extends Output Tables support to central Clean Rooms hosted on Azure (previously AWS only) Not checked yet Adds HIPAA compliance security profile option when creating a Clean Room Not checked yet Introduces Federated Sharing Preview to collaborate across clouds and data platforms via query federation Not checked yet
5 details in Everything
Comments now support email notifications and @ mentions
Jan 25, 2025
· release notes
Databricks code comments now support @ mentions and email notifications.
Adds the ability to mention users directly in comments by typing @ followed by their username. Not checked yet Enables email notifications for users when they are mentioned in comments. Not checked yet
2 details in Everything
New keyboard shortcuts and UI settings to adjust editor font size in Databricks notebooks and SQL editors
Adds Alt + , Alt - (Windows/Linux) and Opt + , Opt - (macOS) shortcuts to quickly adjust font size in notebook, file, and SQL editors. Not checked yet Adds a Developer > Editor font size setting in the UI to manually control the editor font size. Not checked yet
2 details in Everything
OAuth token federation is now available in Public Preview
Jan 24, 2025
· release notes
Public preview of Databricks OAuth token federation for account admins
Import workspace files with drag-and-drop
Jan 24, 2025
· release notes
Databricks adds drag-and-drop import for workspace files and folders across the UI.
Adds drag-and-drop import capability for files and folders in the primary file browser page Not checked yet Adds drag-and-drop import support in the workspace file browser side panel available in notebook, query, and file editors Not checked yet
2 details in Everything
Custom Python AI agents now support AI Gateway and streaming output
Jan 24, 2025
· release notes
Mosaic AI Agent Framework adds streaming output and automatic AI Gateway inference tables for custom Python agents.
Adds streaming output support for deployed custom Python agents to improve time to first token time to first token Time to first token, the wait between submitting a request and receiving the first generated token. It includes delays such as queuing and processing the prompt. Not checked yet Enables AI Gateway inference tables automatically for custom Python agents to provide enhanced logging metadata Not checked yet
2 details in Everything
Notebooks are now supported as workspace files
Jan 23, 2025
· release notes
Databricks supports notebooks as workspace files, enabling programmatic creation, reading, and deletion via the filesystem API.
Adds notebook type to workspace files, allowing programmatic write, read, and delete operations on .ipynb files via the standard workspace filesystem API. Not checked yet
1 detail in Everything
Notebook output improvements for Databricks notebooks include column filtering, copy-as formats, and cell download naming.
Adds Is one of column filtering in notebook result tables via the column menu Not checked yet Supports copying result tables as CSV, TSV TSV A plain-text format storing tabular data with fields separated by tab characters, making it easy for cyber tools to parse and exchange structured records without a CSV parser's quoting complexity. , or Markdown via right-click context menu Not checked yet Changes default download name for cell results to match the notebook filename Not checked yet
3 details in Everything
Failed tasks in continuous jobs are now automatically retried
Jan 22, 2025
· release notes
Continuous jobs now auto-retry failed tasks with exponential backoff
Failed task runs in continuous jobs are automatically retried with exponentially increasing delay until the maximum retry limit is reached Not checked yet
1 detail in Everything
Databricks JDBC JDBC A Java standard API for connecting applications to relational databases, maintained as part of the Java SE platform, letting tools query any compliant database through a single consistent interface. driver 2.7.1 adds OAuth port overrides, refresh tokens refresh tokens A credential used to obtain a new access token without signing in again. It is sent to the authorization service rather than used directly to call the protected API. , and system trust store support.
Adds OAuthEnabledIPAddressRanges property to override default OAuth callback ports for restricted networks Not checked yet Adds Auth_RefreshToken property to enable automatic token refresh Not checked yet Adds UseSystemTrustStore property (value 1) to verify connections using the system's trusted store certificates Not checked yet Adds UseServerSSLConfigsForOAuthEndPoint property to share SSL SSL A cryptographic protocol for encrypting network connections, now deprecated in favor of TLS but whose name persists colloquially. Cyber tools reference it when handling HTTPS or other encrypted-channel inspection and termination. configuration for the OAuth endpoint Not checked yet Changes default authentication to disable BASIC auth; requires allowBasicAuthentication=1 to re-enable Not checked yet
5 details in Everything
└──▷ BREAKING ON UPGRADE
! BASIC authentication is now disabled by default; clients must set allowBasicAuthentication=1 to use it
databricks-agents SDK 0.14.0 release: Custom evaluation metrics
Jan 14, 2025
· release notes
Databricks agents SDK 0.14.0 adds custom evaluation metrics and databricks-agent model type support.
Adds support for the ChatAgent and ChatModel classes from the mlflow.evaluate model_type='databricks-agent' harness Not checked yet
1 detail in Everything
AI Gateway now supports provisioned throughput (Public Preview)
Jan 10, 2025
· release notes
Mosaic AI Gateway Public Preview adds provisioned throughput for model serving endpoints with governance features.
Adds provisioned throughput support to Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. on model serving endpoints Public preview . Not checked yet Enables permission and rate limiting governance on endpoints using provisioned throughput. Not checked yet Enables payload logging via inference tables for audit on endpoints using provisioned throughput. Not checked yet Enables usage tracking via system tables for operational monitoring on endpoints using provisioned throughput. Not checked yet Enables AI Guardrails for request/response filtering on endpoints using provisioned throughput. Not checked yet
+1 more
show less
Enables traffic routing capabilities on endpoints using provisioned throughput. Not checked yet
6 details in Everything
The default format for new notebooks is now IPYNB (Jupyter) format
Dec 20, 2024
· release notes
Databricks now defaults new notebooks to IPYNB (Jupyter) format instead of Source files
New default notebook format for workspaces is now IPYNB (.ipynb) Not checked yet
1 detail in Everything
databricks-agents SDK 0.13.0 release
Dec 18, 2024
· release notes
databricks-agents SDK 0.13.0 allows specifying Databricks CLI profile and MLflow registry URI for deployment, adds secret-based auth to mlflow.evaluate()
Allows agents.deploy() and other APIs to honor the current active Databricks CLI profile and MLflow model registry URI by respecting DATABRICKS_CONFIG_PROFILE and MLFLOW_REGISTRY_URI environment variables. Not checked yet Adds secret-based authentication support for clients in mlflow.evaluate() . Not checked yet
2 details in Everything
External groups are now labeled and immutable
Dec 18, 2024
· release notes
External groups now labeled External and immutable by default in account console
External groups created via SCIM SCIM A standard for creating, updating and removing user accounts and groups across applications. It helps an identity provider keep access in sync when people join, change roles or leave. are explicitly labeled as External Not checked yet External groups can no longer be updated from the Databricks account console or workspace admin settings page by default Not checked yet Adds Immutable external groups toggle in the account console preview page to allow updates to external group membership Not checked yet
3 details in Everything
└──▷ BREAKING ON UPGRADE
! External groups can no longer be updated from the Databricks account console or workspace admin settings page by default
Specify response formats for ai_query
Dec 17, 2024
· release notes
Adds responseFormat parameter to ai_query for structured model outputs
Adds responseFormat field to ai_query requests to specify the desired output structure for queried models Not checked yet
1 detail in Everything
Assign compute resources to groups (Public Preview)
Dec 17, 2024
· release notes
Assign all-purpose compute to groups via new Dedicated access mode
vector_search now supports vector embedding inputs
Dec 17, 2024
· release notes
vector_search function now accepts query_vector parameter for direct vector embedding embedding Lists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG. inputs
vector_search AI Function supports query_text or query_vector parameters to search vector indexes Not checked yet
1 detail in Everything
Test clean rooms with collaborators within the same metastore
Dec 17, 2024
· release notes
Test clean rooms with collaborators within the same metastore
Unity Catalog MANAGE privilege (Public Preview)
Dec 14, 2024
· release notes
Databricks adds MANAGE privilege for Unity Catalog securable objects in Public Preview
Meta Llama 3.3 is now available for provisioned throughput workloads
Dec 13, 2024
· release notes
Meta Llama 3.3 is now available for Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. on provisioned throughput workloads
View streaming workload metrics for your DLT pipeline updates (Public Preview)
Dec 12, 2024
· release notes
New DLT UI view for streaming workload metrics (backlog seconds, bytes, records, files) in public preview.
View streaming workload metrics for your job runs (Public Preview)
Dec 12, 2024
· release notes
Databricks Jobs UI now displays streaming workload backlog metrics (seconds, bytes, records, files) for Spark Structured Streaming sources in Public Preview.
Unified login is now enabled on accounts without single sign-on
Dec 12, 2024
· release notes
Unified login is now automatically enabled for accounts without SSO SSO Single sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. . It remains disabled only if the account had existing SSO configurations.
Unity Catalog can federate to Hive metastore s and AWS Glue
Dec 11, 2024
· release notes
Unity Catalog now supports federating to Hive metastores and AWS Glue for data governance.
Adds the ability to use Unity Catalog to access and govern data registered in externally-managed Hive metastores Not checked yet Adds support for federating Unity Catalog to AWS Glue Not checked yet Extends Unity Catalog federation to include legacy internal Databricks Hive metastores Not checked yet
3 details in Everything
Meta Llama 3.3 70B Instruct is now available on Model Serving
Dec 11, 2024
· release notes
Mosaic AI Model Serving adds Meta Llama 3.3 70B Instruct with Function Calling support
Adds Meta-Llama-3.3-70B-Instruct to Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. pay-per-token endpoints, replacing the deprecated Meta-Llama-3-70B-Instruct model. Not checked yet
1 detail in Everything
└──▷ BREAKING ON UPGRADE
! The Meta-Llama-3.1-70B-Instruct model replaces Meta-Llama-3-70B-Instruct in Foundation Model APIs pay-per-token endpoints starting December 11, 2024.
Remove metastore-level storage to enforce catalog-level storage isolation
Dec 11, 2024
· release notes
Databricks adds the ability to remove the metastore-level storage root, enabling strict catalog-level isolation for managed tables and volumes.
Enforces data storage isolation at the catalog or schema level by allowing removal of the shared metastore storage root. Not checked yet
1 detail in Everything
Network access events system table is now available (Public Preview)
Dec 11, 2024
· release notes
Databricks adds Network Access Events system table in Public Preview to log denied internet access
Manage serverless outbound network connections with serverless egress control
Dec 11, 2024
· release notes
Serverless egress control allows restricting outbound network connections for serverless workloads.
Adds serverless egress control to restrict outbound access for specified internet destinations. Not checked yet
1 detail in Everything
Maximum personal access token lifetime now 730 days (two years)
Dec 5, 2024
· release notes
Default maximum lifetime for Databricks personal access tokens personal access tokens A personal access token, a credential used by scripts and integrations instead of a user's password. Its permissions and expiry depend on the service issuing it. increased to 730 days
Improve Databricks-to-Databricks Delta Sharing table read performance with history sharing (Public Preview)
Dec 5, 2024
· release notes
Databricks adds Public Preview of Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. History Sharing for faster cross-workspace table reads.
Enables history sharing to improve Databricks-to-Databricks Delta Sharing table read performance Public preview Not checked yet
1 detail in Everything
Mosaic AI Model Training - serverless forecasting (Public Preview)
Dec 5, 2024
· release notes
Databricks introduces Mosaic AI Model Training serverless forecasting in public preview.
Introduces managed serverless compute for AutoML forecasting tasks Not checked yet Adds Unity Catalog support to the forecasting workflow Not checked yet Enables access to deep learning algorithms within the forecasting experience Not checked yet Provides an upgraded interface for Mosaic AI Model Training - forecasting Not checked yet
4 details in Everything
Additional metadata for the table tooltip in notebooks
Nov 25, 2024
· release notes
Notebook cell tooltips for Unity Catalog tables now show owner, last updated, size, description, and a link to Catalog Explorer
Databricks Assistant can use Databricks-hosted models
Nov 22, 2024
· release notes
Databricks Assistant enters Public Preview with support for Databricks-hosted models.
Databricks Assistant uses Databricks-hosted models Public preview instead of partner-hosted models. Not checked yet Models are managed and hosted on the same infrastructure that powers Databricks Model Serving. Not checked yet
2 details in Everything
Emergency access for workspaces without unified login
Nov 21, 2024
· release notes
Adds emergency workspace login with Databricks credentials + MFA MFA Multi-factor authentication, requiring proof from different categories, such as a password and a hardware key, to sign in. It reduces reliance on a password alone. when unified login is disabled.
Warehouses system table is now available (Public Preview)
Nov 15, 2024
· release notes
Public preview of system.compute.warehouses system table for SQL warehouse configuration snapshots
Clean Rooms public preview is ungated, with new features
Nov 13, 2024
· release notes
Databricks Clean Rooms public preview ungated on Azure and AWS with new task types, output tables, and serverless egress control.
Clean room notebook task types now let you run Databricks notebooks in a clean room as part of a workflow, supporting task values and output tables Not checked yet Output tables are temporary read-only tables generated by a notebook run and shared to the notebook runner's Unity Catalog metastore Not checked yet Serverless egress control lets you restrict outbound access from central clean rooms to specified internet destinations Not checked yet Clean Rooms public preview is now open to all customers on Azure and AWS Not checked yet
4 details in Everything
Foundation Model APIs pay-per-token available in all Model Serving supported regions
Nov 7, 2024
· release notes
Databricks expands pay-per-token billing for Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. to all supported Mosaic AI Model Serving regions.
Enable cost attribution of fine-grained access control on dedicated compute (formerly single user compute)
Oct 31, 2024
· release notes
Custom tags on dedicated compute are now propagated to serverless queries and recorded in system.billing.usage
Custom tags assigned to a dedicated compute instance are now automatically propagated to any query that is forwarded to serverless compute for data filtering, enabling accurate cost attribution. Not checked yet The cost attribution data from these propagated queries is now recorded in the system.billing.usage table. Not checked yet
2 details in Everything
Enable a metastore to be assigned automatically to new workspaces
Oct 28, 2024
· release notes
New account-level setting to auto-assign a Unity Catalog metastore to newly created workspaces.
Adds the ability for account admins to configure automatic assignment of a Unity Catalog metastore when creating new workspaces in the same region. Not checked yet
1 detail in Everything
Configure an encryption algorithm on an external location
Oct 28, 2024
· release notes
Allows configuring server-side encryption algorithms for AWS S3 external locations in Unity Catalog.
Adds support to specify the SSE SSE Server-Sent Events, a browser API and HTTP-based protocol where a server pushes a stream of text events to a client over a single long-lived connection, giving tools a lightweight alternative to WebSockets for one-way real-time updates. algorithm (SSE-S3 or SSE-KMS KMS A managed service that creates, stores, rotates and controls access to encryption keys, usually backed by HSMs. Applications ask it to encrypt or decrypt rather than holding keys themselves. ) when configuring an external location for AWS S3 Not checked yet Enables access to S3 buckets that require server-side encryption via new external location configuration Not checked yet
2 details in Everything
Structured outputs supported on Mosaic AI Model Serving (Public Preview)
Oct 16, 2024
· release notes
Adds support for specifying JSON schemas in Mosaic AI Model Serving chat model responses.
Adds JSON schema specification capability to Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. in Mosaic AI Model Serving. Not checked yet
1 detail in Everything
AI Functions now available in EU regions
Oct 15, 2024
· release notes
AI Functions powered by Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. are now available in EU regions (eu-west-1 and eu-central-1).
Attribute serverless usage with serverless budget policies (Public Preview)
Oct 15, 2024
· release notes
Public Preview of serverless budget policies to enforce custom tags for billing attribution.
Adds serverless budget policies allowing workspace admins to assign custom tag enforcement to users, groups, or service principals. Not checked yet
1 detail in Everything
Serverless compute automatically enabled on eligible accounts
Oct 15, 2024
· release notes
Serverless compute automatically enabled by default for notebooks, jobs, and DLT on eligible Premium+ accounts
The billable usage system table now attributes networking costs
Oct 11, 2024
· release notes
The billable usage system table has been updated to include new fields for monitoring networking costs associated with serverless compute connections.
Databricks Assistant Autocomplete is now generally available
Oct 10, 2024
· release notes
Databricks Assistant Autocomplete GA provides intelligent inline code suggestions for Python and SQL.
Compliance security profile with serverless compute support
Oct 3, 2024
· release notes
Databricks compliance security profile now supports serverless compute for SQL warehouses, notebooks, workflows, and DLT pipelines.
Serverless compute is now available in the eu-west-2 region
Oct 2, 2024
· release notes
Serverless compute for notebooks, workflows, and DLT is now available in eu-west-2.
External models now supports instance profiles for Amazon Bedrock
Sep 30, 2024
· release notes
Databricks external models now support instance profiles for Amazon Bedrock access
Meta Llama 3.2 3B and 1B models are supported in Foundation Model Fine-tuning
Sep 25, 2024
· release notes
Databricks Foundation Model Fine-tuning adds support for Meta Llama 3.2 3B and 1B models.
Emergency access no longer requires single sign-on
Sep 25, 2024
· release notes
Databricks emergency access can now be configured without SSO SSO Single sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. , supporting up to 20 users with FIDO FIDO An open authentication standard maintained by the FIDO Alliance that replaces passwords with public-key cryptography tied to a hardware authenticator or biometric, giving cyber tools phishing-resistant login without shared secrets. 2 MFA MFA Multi-factor authentication, requiring proof from different categories, such as a password and a hardware key, to sign in. It reduces reliance on a password alone. .
Emergency access for account admins no longer requires Single Sign-On (SSO) configuration. Not checked yet
1 detail in Everything
Meta Llama 3.2 3B and 1B models are supported in Model Serving
Sep 25, 2024
· release notes
Meta Llama 3.2 (3B and 1B) models added to Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. in Databricks Model Serving.
vector_search() now available to all customers (Public Preview)
Sep 25, 2024
· release notes
vector_search() function is now generally available in public preview for querying Mosaic AI Vector Search indexes via SQL
vector_search() is now available to all customers in supported regions without requiring an access request Not checked yet
1 detail in Everything
Prototype and export tool-calling AI agents in AI Playground
Sep 24, 2024
· release notes
Prototype and export tool-calling tool-calling A model's ability to answer with a structured request to run a named function or API, which the application executes and returns. It is how agents take actions. AI agents in the Databricks AI Playground. Users can now provide tools as Unity Catalog functions to interact directly with the agent within the playground environment. Additionally, there is support for exporting the AI agent into notebooks, enabling further iteration, quality evaluation, and deployment processes.
Publish to Power BI is now generally available
Sep 24, 2024
· release notes
The Publish to Power BI feature, which was in private preview, has reached general availability. Users can now natively create semantic models from Databricks tables or schemas and publish them directly to the Power BI service without intermediate steps.
General availability of the direct-to-Power BI publishing capability. Not checked yet
1 detail in Everything
GTE v1.5 (English) embedding models are now supported in Foundation Model API provisioned throughput
Sep 13, 2024
· release notes
Foundation Model API provisioned throughput now supports GTE v1.5 (English) embedding models embedding models Lists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG.
Adds support for gte-base-en-v1.5 and gte-large-en-v1.5 in Foundation Model API provisioned throughput Not checked yet
1 detail in Everything
Databricks Assistant Quick Fix debugs code inline
Sep 12, 2024
· release notes
Databricks Assistant Quick Fix now recommends single-line fixes inline for runtime errors
Adds Assistant Quick Fix inline single-line fix recommendations when code execution returns an error Not checked yet
1 detail in Everything
Mosaic AI Gateway enters Public Preview with centralized governance, rate limiting, logging, guardrails, and traffic routing for model endpoints.
AI Gateway is now Public Preview as a centralized service for managing AI model usage Not checked yet Adds permission and rate limiting controls to manage access to model endpoints Not checked yet Adds payload logging via inference tables to monitor data sent to model APIs Not checked yet Adds usage tracking via system tables to monitor endpoint operational usage and costs Not checked yet Adds AI Guardrails to prevent unwanted or unsafe data in requests and responses Not checked yet
+1 more
show less
Adds traffic routing capabilities to minimize production outages during and after deployment Not checked yet
6 details in Everything
Extended AI-generated comments support
Sep 6, 2024
· release notes
Extended AI-generated comments now support Unity Catalog catalogs, schemas, functions, models, and volumes via Catalog Explorer.
Inline assistant added to Catalog Explorer for editing comments Not checked yet
1 detail in Everything
Databricks extension for Visual Studio Code is GA
Sep 4, 2024
· release notes
Databricks extension for VS Code reaches general availability
Databricks IDE extension is generally available for Visual Studio Code, enabling connection to remote Databricks workspaces. Not checked yet Supports defining and deploying Declarative Automation Bundles directly from the VSCode IDE. Not checked yet Enables debugging notebooks and running them as jobs within the VSCode IDE. Not checked yet Allows running files on clusters or as jobs via the extension. Not checked yet Provides local-to-workspace code synchronization capabilities. Not checked yet
5 details in Everything
Foundation Model APIs pay-per-token available in eu-central-1 and eu-west-1
Sep 3, 2024
· release notes
Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. pay-per-token billing now available in eu-central-1 and eu-west-1.
Pay-per-token pricing for Foundation Model APIs becomes available in the eu-central-1 region. Not checked yet Pay-per-token pricing for Foundation Model APIs becomes available in the eu-west-1 region. Not checked yet
2 details in Everything
Function calling is now supported on Foundation Model APIs provisioned throughput
Sep 3, 2024
· release notes
Adds OpenAI-compatible function calling function calling A model's ability to answer with a structured request to run a named function or API, which the application executes and returns. It is how agents take actions. support to Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. on provisioned throughput.
System tables are now generally available
Sep 3, 2024
· release notes
Databricks system tables are GA with new system.billing.usage and system.billing.list_price tables
System tables platform is now generally available Not checked yet New system.billing.usage table is now available for querying billing data Not checked yet New system.billing.list_price table is now available for querying list price information Not checked yet
3 details in Everything
Monitor Unity Catalog object usage against quotas using the new Resource Quotas APIs
Aug 30, 2024
· release notes
New Resource Quotas APIs enable monitoring Unity Catalog object usage against limits
AskSupport replaced by Databricks assistant
Aug 29, 2024
· release notes
Databricks Assistant replaces AskSupport for workspace-based technical support.
Replaces the Slack-based AskSupport channel with the in-workspace Databricks Assistant. Not checked yet
1 detail in Everything
Lakeflow system tables are extend with additional columns
Aug 23, 2024
· release notes
Lakeflow system tables gain new columns for job details and run context
Adds description column to the system.lakeflow.jobs table Not checked yet Adds run_type, run_name, compute_ids, termination_code, and job_parameters columns to the system.lakeflow.job_run_timeline table Not checked yet Adds job_run_id, parent_run_id, and termination_code columns to the system.lakeflow.job_task_run_timeline table Not checked yet
3 details in Everything
Python code formatting error highlights
Aug 23, 2024
· release notes
Databricks August release adds inline Python formatting error highlights in notebooks and file editors.
Highlights Python formatting errors (e.g., unexpected indentation, long line length) in notebooks and file editors Not checked yet
1 detail in Everything
Delta Sharing : More Delta Lake features now supported by the Python and Power BI connectors
Aug 21, 2024
· release notes
Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. Python (1.1.0+) and Power BI (v2.132.908.0+) connectors now support Column mapping name mode, deletion vectors, and Uniform format
Delta Sharing adds support for TimestampNTZ
Aug 21, 2024
· release notes
Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. adds TimestampNTZ support for Databricks Runtime 14.1+ and Spark 3.3+
The Databricks Jobs For each task is GA
Aug 21, 2024
· release notes
Databricks For each task is generally available for looping other tasks with different parameters.
Databricks Runtime 15.4 LTS for Machine Learning
Aug 19, 2024
· release notes
Databricks Runtime 15.4 LTS ML adds sample weights for classification in AutoML and supports Graviton instance types.
AutoML now supports sample weights for classification, letting you adjust the importance of each class during classification model training Not checked yet
1 detail in Everything
└──▷ BREAKING ON UPGRADE
! Petastorm is deprecated and will not be pre-installed in releases after 15.4 LTS ML! spark-tensorflow-distributor is deprecated and will not be pre-installed in releases after 15.4 LTS ML
Notebook autocomplete now prioritizes suggestions based on individual Unity Catalog metadata and usage.
Adds personalized suggestion rankings for notebook autocomplete using Unity Catalog metadata. Not checked yet
1 detail in Everything
Databricks Runtime 15.4 LTS adds UTF-8 validation functions, try_url_decode, and UniForm Iceberg enablement.
is_valid_utf8, make_valid_utf8, validate_utf8, and try_validate_utf8 functions added for UTF-8 string validation Not checked yet try_url_decode function added to decode URL-encoded strings, returning NULL on invalid format Not checked yet ALTER TABLE ... DROP FEATURE checkConstraints allows dropping the check constraints table feature from a Delta table Not checked yet Support for enabling UniForm Iceberg on existing tables via ALTER TABLE without rewriting data files Not checked yet Optional use of unenforced foreign key constraints in the optimizer by specifying the RELY keyword in FOREIGN KEY definitions Not checked yet
+1 more
show less
Dedicated compute supports fine-grained access control and materialized views in public preview Not checked yet
6 details in Everything
└──▷ BREAKING ON UPGRADE
! Calling any Python UDF, UDAF, or UDTF that uses a VARIANT type as an argument or return value throws an exception! spark.sql.legacy.jdbc.useNullCalendar is now set to true by default! Views now adapt to schema changes in the underlying query; this replaces the previous default of BINDING mode! Undocumented ! syntax as a synonym for NOT outside boolean expressions is no longer allowed! Specifying column types, NOT NULL constraints, or DEFAULT in CREATE VIEW with named columns is no longer allowed
Symlinks are now supported in workspace files
Aug 18, 2024
· release notes
Databricks workspace files now support symlinks in web terminal and notebook cells.
Adds tooltip display for symlink targets showing underlying path and copy option Not checked yet
1 detail in Everything
Configure your workspace's default access mode for jobs compute
Aug 16, 2024
· release notes
Workspace admins can now define a default access mode for jobs compute. This setting applies to any compute resources created without an explicit mode, simplifying configuration management.
New slash commands for Databricks Assistant
Aug 14, 2024
· release notes
Databricks Assistant adds /findTables, /findQueries, /prettify, /rename, and /settings slash commands for notebook shortcuts
└──▷ HOW TO FIND IT
Standardize code formatting within a notebook cell directly through the Assistant command line.
📍 /prettify
Adds /findTables slash command to search for relevant tables via Unity Catalog metadata Not checked yet Adds /findQueries slash command to search for relevant queries via Unity Catalog metadata Not checked yet Adds /prettify slash command to format code in the notebook Not checked yet Adds /rename slash command to suggest updated names for notebook cells and other elements Not checked yet Adds /settings slash command to adjust notebook settings directly from Assistant Not checked yet
5 details in Everything
Databricks JDBC JDBC A Java standard API for connecting applications to relational databases, maintained as part of the Java SE platform, letting tools query any compliant database through a single consistent interface. driver 2.6.40 adds OIDC OIDC OpenID Connect, a sign-in protocol built on OAuth 2.0. An application uses identity information from a trusted provider instead of maintaining a separate password for each user. discovery, refresh token refresh token A credential used to obtain a new access token without signing in again. It is sent to the authorization service rather than used directly to call the protected API. support, and OAuth redirect port configuration.
ProxyIgnoreList property is now supported when UseProxy is set to 1 Not checked yet Adds OAuth2RedirectUrlPort property to configure the OAuth redirect port for unified default OAuth options Not checked yet Updated Arrow support. The driver now uses Apache Arrow version 14.0.2 Not checked yet
3 details in Everything
Serverless SQL warehouses with the compliance security profile is now GA
Aug 13, 2024
· release notes
Serverless SQL warehouses with the compliance security profile are now GA, making them the default for eligible workspaces.
Specify columns to sync for Mosaic AI Vector Search
Aug 12, 2024
· release notes
Adds the ability to specify a subset of columns to sync for Mosaic AI Vector Search.
Allows specifying a subset of columns (besides the primary key and embedding embedding Lists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG. column) to be included in a Mosaic AI Vector Search index Not checked yet
1 detail in Everything
Compute policy enforcement now available
Aug 8, 2024
· release notes
Compute policy enforcement allows admins to update compute resources to the latest policy version via UI or Cluster Policies API.
Format columns in notebook and query results tables
Aug 6, 2024
· release notes
Adds Currency, Percentage, URL column formats and decimal place control in notebook and query results tables.
Supports Currency, Percentage, and URL formatting for columns in notebook and query result tables Not checked yet Allows setting the number of decimal places for formatted columns Not checked yet
2 details in Everything
Row filters and column masks are now GA, with improvements
Aug 6, 2024
· release notes
Row filters and column masks are GA on Databricks Runtime 12.2+ with new support for constant expressions, materialized views, streaming tables, MERGE, and table sampling.
Support for constant expressions in policy parameters (strings, numeric, intervals, booleans, nulls) Not checked yet Public Preview support for applying row filters and column masks to materialized views and streaming tables Not checked yet Support for MERGE statements on tables with applied filters and masks Not checked yet
3 details in Everything
Canadian Centre for Cybersecurity (CCCS) Medium (Protected B) compliance controls
Aug 2, 2024
· release notes
Databricks adds Canadian Centre for Cybersecurity (CCCS) Medium (Protected B) compliance controls for workspace configuration.
Lakehouse Federation is generally available (GA)
Aug 1, 2024
· release notes
Lakehouse Federation Lakehouse Federation A data architecture pattern that lets a single query engine read from multiple external data sources—databases, data lakes, warehouses—without moving or copying the data, reducing pipeline complexity for tools that need broad data access. GA adds MySQL, PostgreSQL, Redshift, Snowflake, MSSQL, Synapse, and Databricks connectors with SSO SSO Single sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. and pushdown improvements.
Adds Lakehouse Federation connectors for MySQL, PostgreSQL, Amazon Redshift, Snowflake, Microsoft SQL Server, Azure Synapse, and Databricks as generally available. Not checked yet Adds single sign-on single sign-on Single sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. (SSO) authentication support in the Snowflake and Microsoft SQL Server Lakehouse Federation connectors. Not checked yet
2 details in Everything
Create a new workspace with enhanced security and compliance features
Aug 1, 2024
· release notes
Enables configuring compliance security profile, standards, and enhanced monitoring at workspace creation
Adds the ability to enable the compliance security profile during workspace deployment Not checked yet Adds the ability to add compliance standards when creating a new workspace Not checked yet Adds the ability to enable enhanced security monitoring during workspace creation Not checked yet
3 details in Everything
Command palette available in notebooks
Jul 31, 2024
· release notes
Databricks notebooks now include a command palette accessible via Cmd+Shift+P / Ctrl+Shift+P
Adds a command palette to Databricks notebooks for quick access to frequently used actions Not checked yet
1 detail in Everything
Embed and drag & drop images in notebooks
Jul 31, 2024
· release notes
Adds support for embedding embedding Lists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG. and drag-and-drop of images directly in notebook markdown cells
Supports drag-and-drop of images from desktop into markdown cells to automatically upload and display them Not checked yet Enables embedding images directly within markdown cells Not checked yet
2 details in Everything
Sharing schemas using Delta Sharing is now GA
Jul 30, 2024
· release notes
Sharing schemas via Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. is now GA
General availability of sharing entire schemas using Delta Sharing Not checked yet
1 detail in Everything
Support for Cloudflare R2 storage is GA
Jul 30, 2024
· release notes
Databricks GA adds support for Cloudflare R2 as cloud storage for Unity Catalog and Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. .
Cloudflare R2 is now generally available for use as cloud storage for data registered in Unity Catalog Not checked yet Support for Cloudflare R2 extends to all Databricks data and AI assets previously supported in AWS S3, Azure Data Lake Storage, and Google Cloud Storage Not checked yet
2 details in Everything
Updates to Databricks Git folders authentication and sharing behaviors
Jul 29, 2024
· release notes
New Databricks Git folders sharing links, persisted dialog state, and enhanced diff highlighting
Adds Git folder sharing to generate URL links for pre-filled Add Git folder dialogs among workspace users Not checked yet Persists Git folder dialog state in the browser URL to allow reopening or sharing with identical information displayed Not checked yet Adds darker red and green highlighting in the Git folder diff view for replaced text and multiple lines of changes Not checked yet
3 details in Everything
Cluster library installation timeout
Jul 29, 2024
· release notes
Clusters now enforce a 2-hour timeout for library installation, marking long-running installs as failed.
Library installation on clusters now has a hard timeout of 2 hours. Installations exceeding this duration are marked as failed. Not checked yet
1 detail in Everything
Model sharing using Delta Sharing is now generally available
Jul 26, 2024
· release notes
Databricks now offers GA Model Sharing via Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. for AI models in Unity Catalog.
Databricks allows account admins to enable cross-Geo processing for Designated Services.
Single sign-on (SSO) is supported in Lakehouse Federation for SQL Server
Jul 25, 2024
· release notes
Lakehouse Federation Lakehouse Federation A data architecture pattern that lets a single query engine read from multiple external data sources—databases, data lakes, warehouses—without moving or copying the data, reducing pipeline complexity for tools that need broad data access. adds SSO SSO Single sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. support for SQL Server connections in Unity Catalog
New open-source Databricks JDBC JDBC A Java standard API for connecting applications to relational databases, maintained as part of the Java SE platform, letting tools query any compliant database through a single consistent interface. driver enters public preview with OAuth, Cloud Fetch, and Unity Catalog volume ingestion support.
Adds OAuth authentication support to the JDBC driver Not checked yet Adds Cloud Fetch feature for query results Not checked yet Adds Unity Catalog volume ingestion capability Not checked yet
3 details in Everything
Node timeline system table is now available (Public Preview)
Jul 23, 2024
· release notes
New node_timeline table in Databricks system.compute schema for minute-by-minute compute utilization metrics
Dedicated compute (formerly single user) supports fine-grained access control, materialized views , and streaming tables
Jul 23, 2024
· release notes
Databricks Dedicated compute now supports fine-grained access control with materialized views and streaming tables via Databricks Runtime 15.4 LTS.
Adds Databricks Runtime 15.4 LTS support for fine-grained access control on dedicated compute resources, routing specific queries to serverless compute for data filtering. Not checked yet
1 detail in Everything
Scala is GA on Unity Catalog standard compute
Jul 23, 2024
· release notes
Scala is generally available on Unity Catalog standard compute in Databricks Runtime 15.4 LTS
Adds Scala support on Unity Catalog standard compute (formerly shared access mode) starting with Databricks Runtime 15.4 LTS. Not checked yet Enables scalar user-defined functions (UDFs) in Scala on this compute tier. Not checked yet
2 details in Everything
Meta Llama 3.1 is now supported in Model Serving
Jul 23, 2024
· release notes
Databricks Mosaic AI adds support for Meta Llama 3.1 models (8B, 70B, 405B) in Model Serving.
Adds Meta-Llama-3.1-405B-Instruct and Meta-Llama-3.1-70B-Instruct to Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. pay-per-token endpoints Not checked yet Enables production usage of the full Llama 3.1 suite (8B, 70B, 405B) via provisioned throughput in the US region Not checked yet
2 details in Everything
└──▷ BREAKING ON UPGRADE
! Starting July 23, 2024, Meta-Llama-3.1-70B-Instruct replaces support for Meta-Llama-3-70B-Instruct in Foundation Model APIs pay-per-token endpoints.
Unity Catalog will soon drop support for storage credentials that use non-self-assuming IAM roles
Jul 22, 2024
· release notes
Unity Catalog enforcing self-assuming IAM IAM Identity and Access Management — a framework of policies and services for controlling who can authenticate to systems and what resources they are authorized to use, letting cyber tools enforce least-privilege and audit access. roles for all storage credentials by Jan 20, 2025
Enforces that AWS IAM AWS IAM AWS Identity and Access Management is Amazon's service for controlling who and what can call AWS APIs, by attaching policies that allow or deny actions on resources. Cyber tools use it to scope credentials to least privilege. roles for storage credentials must be self-assuming, ceasing work of non-compliant credentials. Not checked yet Requires new storage credentials to use self-assuming IAM roles starting September 20, 2024. Not checked yet
2 details in Everything
└──▷ BREAKING ON UPGRADE
! Storage credentials using non-self-assuming AWS IAM roles will cease to work on January 20, 2025.
Notebooks: toggle more visible cell titles
Jul 18, 2024
· release notes
Adds Show promoted cell titles setting to Notebook developer settings for visibility.
Delta Sharing lets you share tables that use liquid clustering
Jul 16, 2024
· release notes
Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. supports sharing tables enabled for liquid clustering.
Query history system table is now available (Public Preview)
Jul 16, 2024
· release notes
Databricks adds query history system table in Public Preview for logging SQL warehouse queries.
Adds a new query system schema containing detailed logs of each query run on a SQL warehouse Not checked yet
1 detail in Everything
Databricks Connect for Python now supports serverless compute
Jul 15, 2024
· release notes
Databricks Connect for Python now supports connecting to serverless compute in Public Preview.
Adds serverless compute support to Databricks Connect for Python, enabling connections via the new configuration options. Not checked yet
1 detail in Everything
Serverless compute for workflows is GA
Jul 15, 2024
· release notes
Serverless compute for Databricks workflows is now GA, allowing jobs to run without infrastructure configuration.
Serverless compute for workflows is now generally available Not checked yet Run Lakeflow Jobs with serverless compute for workflows Not checked yet
2 details in Everything
Partition metadata logging for Unity Catalog external tables
Jul 15, 2024
· release notes
Databricks Runtime 13.3 LTS adds partition metadata logging for external tables.
Serverless compute for notebooks is GA
Jul 15, 2024
· release notes
Serverless compute for notebooks is now generally available, enabling immediate run of Python or SQL code with on-demand scalable compute.
Filter data outputs using natural language prompts
Jul 11, 2024
· release notes
Databricks adds natural language filtering for data outputs via the Assistant.
Forecast time series data using ai_forecast()
Jul 11, 2024
· release notes
Databricks SQL introduces ai_forecast() for time series extrapolation
Databricks Assistant system table now available (Public Preview)
Jul 10, 2024
· release notes
Databricks Assistant events now logged to system table system.access.assistant_events
Foundation Model Fine-tuning available to all us-west-2 customers (Public Preview)
Jul 10, 2024
· release notes
Foundation Model Fine-tuning now generally available in us-west-2 via Mosaic AI Model Training.
End of life for Databricks-managed passwords
Jul 10, 2024
· release notes
End of support for Databricks-managed passwords (basic auth)
└──▷ BREAKING ON UPGRADE
! Users attempting to authenticate with Databricks-managed passwords to the UI or APIs will be rejected; a unique code is sent via email instead if SSO SSO Single sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. is not configured.
Lakehouse Federation supports Salesforce Data Cloud (Public Preview)
Jul 10, 2024
· release notes
Lakehouse Federation Lakehouse Federation A data architecture pattern that lets a single query engine read from multiple external data sources—databases, data lakes, warehouses—without moving or copying the data, reducing pipeline complexity for tools that need broad data access. now supports Salesforce Data Cloud in public preview
Adds support for running federated queries against data managed by Salesforce Data Cloud Not checked yet
1 detail in Everything
Sign-in with one-time passcodes and external accounts
Jul 10, 2024
· release notes
Databricks expands authentication methods to support one-time passcodes and external identity providers identity providers A service that authenticates users and supplies identity information to other applications. It is the sign-in authority behind many single sign-on systems. .
Enables users to authenticate via one-time passcodes or external accounts such as Google and Microsoft. Not checked yet
1 detail in Everything
SQL File task support for files with multi-statement SQL queries is GA
Jul 10, 2024
· release notes
SQL File task now supports multi-statement SQL queries in a single file.
SQL File task can now execute multiple SQL statements from a single uploaded file. Not checked yet
1 detail in Everything
Catalog Explorer adds Quick access section and moves Compute, Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. , etc. to gear icon menu.
Adds a Quick access section in Catalog Explorer to display recent Unity Catalog objects, favorite Unity Catalog objects, and catalogs and schemas for browsing. Not checked yet
1 detail in Everything
Serve multiple external models from a single model serving endpoint
Jun 27, 2024
· release notes
Mosaic AI Model Serving allows multiple external models on a single endpoint
Databricks Assistant is now Generally Available with code generation, SQL editing, and visualization capabilities.
GA release of Databricks Assistant, unlocking code generation, debugging, optimization, and explanation for users Not checked yet New ability to create visualizations from data directly within the Assistant interface Not checked yet
2 details in Everything
Databricks Assistant now gives titles to threads
Jun 27, 2024
· release notes
Databricks Assistant now auto-titles threads based on prompt context.
Adds automatic thread titling in Databricks Assistant for easier navigation. Not checked yet
1 detail in Everything
New improved catalog creation UI with stepwise workflow
Adds a new stepwise catalog creation UI that separates required from optional configurations and supports standard, shared, and foreign catalogs. Not checked yet
1 detail in Everything
Notebooks Assistant automatically diagnoses errors
Jun 16, 2024
· release notes
Notebooks Assistant now automatically diagnoses code errors in notebooks using AI.
Adds automatic error diagnosis for notebook code via the Databricks Assistant Not checked yet
1 detail in Everything
Create budgets to monitor account spending (Public Preview)
Jun 13, 2024
· release notes
Databricks adds account-level budget creation with workspace and tag filters in Public Preview.
Account admins can create budgets to track spending for a Databricks account Not checked yet Budgets support customized filters to track spending based on workspace and custom tags Not checked yet
2 details in Everything
Importable cost management dashboard added to account console (Public Preview)
Jun 13, 2024
· release notes
Account admins can now import customizable cost management dashboards into Unity Catalog workspaces.
Volume sharing using Delta Sharing is now generally available
Jun 13, 2024
· release notes
Volumes can now be shared via Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. as GA across Unity Catalog metastores
Unified login is now generally available
Jun 12, 2024
· release notes
Unified login GA allows managing one SSO SSO Single sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. config for the entire account, requiring all users (admin and regular) to sign in via SSO.
Unified login allows you to manage one SSO configuration for your Databricks account and workspaces. Not checked yet
1 detail in Everything
Mosaic AI Agent Framework (Public Preview)
Jun 12, 2024
· release notes
Mosaic AI Agent Framework enters public preview with RAG RAG Retrieval-augmented generation: looking up relevant documents and adding them to the prompt before the model answers. It grounds responses in private or current data without retraining. application support
Introduces Mosaic AI Agent Framework for building production-quality RAG applications Not checked yet Enables creation and logging of agents and chains Not checked yet Provides parameterization capabilities for agents and chains to facilitate rapid experimentation and iteration Not checked yet Adds built-in metrics for comparing runs across document retrieval, response accuracy, cost, and latency Not checked yet Supports custom LLM LLM A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. judges for evaluating retrieval and response accuracy Not checked yet
+4 more
show less
Allows deployment of AI applications with automated request and response logging Not checked yet Includes a built-in review application to collect user feedback Not checked yet Offers Agent Evaluation capabilities to assess application quality, latency, and cost Not checked yet Integrates MLflow Tracing for analyzing and comparing traces across code execution Not checked yet
9 details in Everything
Query a vector search index using vector_search()
Jun 10, 2024
· release notes
Adds the vector_search() SQL function to query Mosaic AI Vector Search indexes.
Databricks adds Public Preview for function calling function calling A model's ability to answer with a structured request to run a named function or API, which the application executes and returns. It is how agents take actions. with DBRX Instruct and Meta-Llama-3-70B-Instruct models
Function calling is now available in Public Preview for Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. pay-per-token models. Not checked yet Function calling support added for the DBRX Instruct model. Not checked yet Function calling support added for the Meta-Llama-3-70B-Instruct model. Not checked yet
3 details in Everything
Column mapping is generally available in Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. on Databricks
GTE is now supported in Model Serving
Jun 7, 2024
· release notes
Databricks Mosaic AI Model Serving now supports General Text Embedding Embedding Lists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG. (GTE) models via Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. .
Mosaic AI Vector Search now supports hybrid search
Jun 7, 2024
· release notes
Mosaic AI Vector Search introduces hybrid keyword-similarity search combining vector embeddings vector embeddings Lists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG. with traditional keyword matching.
Adds hybrid keyword-similarity search to Mosaic AI Vector Search, enabling combined vector-based embedding embedding Lists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG. and traditional keyword-based retrieval for more relevant results. Not checked yet
1 detail in Everything
Databricks Assistant: Threads & queries experience enabled by default
Jun 6, 2024
· release notes
Databricks Assistant now tracks query threads and history by default in the editor.
Route optimized serving endpoints enforce network restrictions
Jun 6, 2024
· release notes
Databricks expands security controls for Route Optimized Serving Endpoints by allowing enforcement of network restrictions including IP ACLs ACLs An access control list, a set of rules allowing or denying operations for particular users, services or network addresses. It decides who can use a resource. and PrivateLink PrivateLink An AWS networking feature that exposes a service over a private endpoint inside a VPC, so traffic never traverses the public internet, giving cyber tools a way to integrate with cloud services without opening public routes. .
Route optimized serving endpoints now enforce network restrictions such as IP access control lists or PrivateLink. Not checked yet
1 detail in Everything
Quickly access Catalog Explorer tables from notebooks
Jun 5, 2024
· release notes
The Databricks service adds a UI interaction feature in notebooks allowing users to quickly link from table names directly to Catalog Explorer for detailed metadata inspection.
Databricks adds live preview and toolbar for Markdown cells in notebooks
Adds a toolbar with buttons for common Markdown elements (headers, lists, links) to notebook cell editors Not checked yet
1 detail in Everything
The new Databricks Notebook UI is now generally available
Jun 4, 2024
· release notes
The new Databricks Notebooks UI is now generally available
Adds a modern UX with an updated Notebook interface for streamlined coding and organization Not checked yet
1 detail in Everything
Databricks ODBC ODBC A standard API, maintained by Microsoft, for connecting applications to relational databases regardless of vendor. Cyber tools use it to query databases without writing engine-specific code. driver 2.8.2 adds extended SQL commands for Unity Catalog volumes and updates OAuth config defaults.
Adds support for extended SQL PUT, GET, and REMOVE commands to ingest and manage large datasets in Unity Catalog volumes Not checked yet Updates default OAuth configuration values (authorization endpoint, client ID, user-to-machine scope, machine-to-machine scope) for compliance Not checked yet Enables the driver to accept a private signed CA if added to the Windows Trusted Root CA certificate store Not checked yet
3 details in Everything
Serverless firewall configuration now supports more compute types
May 31, 2024
· release notes
Serverless firewall configuration now supports more compute types
Adds support for configuring firewalls via Network connectivity configurations (NCCs) on serverless jobs and notebooks. Not checked yet Extends NCC-based firewall support to DLT pipelines. Not checked yet Extends NCC-based firewall support to model serving CPU CPU The main general-purpose processor in a computer. It runs application instructions and coordinates work with memory and other hardware. endpoints. Not checked yet
3 details in Everything
New dashboard helps Databricks Marketplace providers monitor listing usage
May 28, 2024
· release notes
The Provider Analytics Dashboard uses system table data to give Marketplace providers visibility into listing performance.
Monitors listing views, requests, and installs via a new dashboard powered by Marketplace system tables. Not checked yet
1 detail in Everything
Improved search and filtering in notebook and SQL editor results tables
May 28, 2024
· release notes
Databricks adds search and dropdown filtering to notebook/SQL results tables
Bulk move and delete workspace objects from the workspace browser
May 24, 2024
· release notes
Bulk move and delete workspace objects via the workspace browser UI
Adds multi-select and drag-and-drop support for moving or deleting multiple workspace objects in the workspace browser. Not checked yet
1 detail in Everything
View system-generated federated queries in Query Profile
May 24, 2024
· release notes
Lakehouse Federation Lakehouse Federation A data architecture pattern that lets a single query engine read from multiple external data sources—databases, data lakes, warehouses—without moving or copying the data, reducing pipeline complexity for tools that need broad data access. adds ability to view system-generated federated queries and metrics in the Query Profile graph view.
New dbt-databricks connector 1.8.0 adopts decoupled dbt architecture
May 23, 2024
· release notes
dbt-databricks connector 1.8.0 adopts decoupled architecture, adding databricks_tags support and auto-refresh for materialized views
Adopts the decoupled dbt architecture with a shared abstraction layer between the adapter and dbt-core. Not checked yet Supports Unity Catalog securable object tags via the new databricks_tags model configuration key. Not checked yet Adds the ability to schedule automatic refreshes for materialized views and streaming tables. Not checked yet
3 details in Everything
Unity Catalog objects are available in recents and favorites
May 23, 2024
· release notes
Adds Unity Catalog object recents and favorites to UI navigation
Adds support for viewing recently accessed Unity Catalog objects in the recents list Not checked yet
1 detail in Everything
New compliance and security settings APIs (Public Preview)
May 23, 2024
· release notes
Databricks introduces Public Preview APIs for Compliance Security Profile, Enhanced Security Monitoring, and Automatic Cluster Update settings.
Adds API endpoints to manage the compliance_security_profile configuration for workspaces Not checked yet Adds API endpoints to manage the enhanced_security_monitoring configuration for workspaces Not checked yet Adds API endpoints to manage the automatic_cluster_update settings for workspaces Not checked yet
3 details in Everything
Bind storage credentials and external locations to specific workspaces (Public Preview)
May 22, 2024
· release notes
Bind storage credentials and external locations to specific workspaces Public preview
Add workspace-scoped bindings for storage credentials and external locations to prevent cross-workspace access Not checked yet
1 detail in Everything
New Tableau connector for Delta Sharing
May 22, 2024
· release notes
Databricks introduces a new Tableau connector for Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. to simplify data access in Tableau Desktop.
Pre-trained models in Unity Catalog (Public Preview)
May 21, 2024
· release notes
Databricks adds pre-trained AI models to Unity Catalog for inference workflows.
The compliance security profile now supports AWS Graviton instance types
May 21, 2024
· release notes
Databricks compliance security profile now supports AWS Graviton instance types
New Git folder sharing workflow with pre-populated clone dialogs
Adds a banner alert when sharing a Git folder that copies a link to create a new Git folder Not checked yet Adds a Create Git folder button on alert banners for folders created by other users, pre-populating values from the source Not checked yet
2 details in Everything
Unified Login now supported with AWS PrivateLink (Private Preview)
May 15, 2024
· release notes
Unified Login (SSO SSO Single sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. ) now supported with AWS PrivateLink AWS PrivateLink An AWS networking feature that exposes a service over a private VPC endpoint rather than the public internet, letting tools move data between accounts or to AWS services without traffic ever leaving Amazon's backbone. connectivity for workspaces.
Adds support for Unified Login when connecting via AWS PrivateLink PrivateLink An AWS networking feature that exposes a service over a private endpoint inside a VPC, so traffic never traverses the public internet, giving cyber tools a way to integrate with cloud services without opening public routes. Not checked yet
1 detail in Everything
Foundation Model Training (Public Preview)
May 13, 2024
· release notes
Databricks Platform adds Foundation Model Training in public preview.
Unified Login now supported with AWS PrivateLink (Public Preview)
May 10, 2024
· release notes
Databricks adds Unified Login support with AWS PrivateLink AWS PrivateLink An AWS networking feature that exposes a service over a private VPC endpoint rather than the public internet, letting tools move data between accounts or to AWS services without traffic ever leaving Amazon's backbone. in Public Preview
Unified Login is now supported with private connectivity between users and their Databricks workspaces. Not checked yet
1 detail in Everything
Allow users to copy data to the clipboard from results table
May 9, 2024
· release notes
A new configuration option allows administrators to manage clipboard copying permissions for results tables across multiple Databricks interfaces.
Attribute tag values for Unity Catalog objects can now be 1000 characters long (Public Preview)
May 8, 2024
· release notes
Databricks Unity Catalog now supports attribute tag values up to 1000 characters.
New capabilities for Mosaic AI Vector Search
May 8, 2024
· release notes
Mosaic AI Vector Search adds PrivateLink PrivateLink An AWS networking feature that exposes a service over a private endpoint inside a VPC, so traffic never traverses the public internet, giving cyber tools a way to integrate with cloud services without opening public routes. /IP access lists, CMK endpoints, saved embeddings embeddings Lists of numbers a model produces to represent the meaning of text, images or code, so similar items sit close together. They power semantic search and RAG. to Delta table
Enables Customer Managed Keys (CMK) for new Vector Search endpoints created on or after May 8, 2024 Not checked yet Allows saving generated embeddings directly to a Delta table via the Create an AI Search index workflow Not checked yet
2 details in Everything
Databricks JDBC JDBC A Java standard API for connecting applications to relational databases, maintained as part of the Java SE platform, letting tools query any compliant database through a single consistent interface. driver 2.6.38 adds native query support with new properties and JWT JWT An open standard (RFC 7519) for encoding claims as a signed, optionally encrypted JSON object passed between parties. Cyber tools use it to verify identity and permissions without querying a central session store. auth
Adds SPARK_CLI_SERVICE_PROTOCOL_V8 support for native parameterized queries (limit 256 parameters) Not checked yet Adds UseNativeQuery=1 property to enable data ingestion via Unity Catalog volumes Not checked yet Adds query_id retrieval on IHadoopStatement via the new QueryProfile interface Not checked yet Adds EnableAsyncModeForMetadataOperation=1 property for async metadata Thrift calls on SPARK_CLI_SERVICE_PROTOCOL_V9 servers Not checked yet Adds UseJWTAssertion=1 property to support JWT assertion OAuth with client credentials Not checked yet
5 details in Everything
Notebooks now detect and auto-complete column names for Spark Connect DataFrames
May 1, 2024
· release notes
Notebooks auto-detect and autocomplete column names for Spark Connect DataFrames.
Cancel pending serving endpoint updates in Model Serving
Apr 29, 2024
· release notes
Serving UI gains ability to cancel pending serving endpoint updates.
Databricks Assistant: Threads & history
Apr 29, 2024
· release notes
New feature allowing users to track query threads and history within editor contexts via opt-in.
Adds opt-in query thread tracking and session history within Databricks Assistant editor contexts Not checked yet
1 detail in Everything
Data lineage now captures reads on tables with column masks and row-level security
Apr 25, 2024
· release notes
Data lineage now captures reads on tables with column masks and row-level security
New columns added to the billable usage system table (Public Preview)
Apr 24, 2024
· release notes
New columns added to system.billing.usage for identifying product and feature usage.
Meta Llama 3 is supported in Model Serving for AWS
Apr 24, 2024
· release notes
Databricks Mosaic AI Model Serving on AWS now supports the Meta Llama 3 (70B Instruct) foundation model.
Mosaic AI Model Serving in pay-per-token serving endpoint regions now supports the Meta-Llama-3-70-Instruct dense model architecture. Not checked yet
1 detail in Everything
Delta Sharing supports tables that use column mapping (Public Preview)
Apr 23, 2024
· release notes
Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. now supports sharing tables that use column mapping in Public Preview
Get serving endpoint schemas (Public Preview)
Apr 22, 2024
· release notes
Databricks introduces a new capability to programmatically retrieve the schema definitions for model and feature serving endpoints via API.
Jobs created through the UI are now queued by default
Apr 18, 2024
· release notes
Databricks Jobs UI default behavior change for concurrency handling.
Configuring access to resources from serving endpoints is GA
Apr 17, 2024
· release notes
GA configuring environment variables on serving endpoints to access external resources
env configuration field allows setting environment variables on model and feature serving endpoints Not checked yet
1 detail in Everything
Lakehouse Federation supports foreign tables with case-sensitive identifiers
Apr 15, 2024
· release notes
Lakehouse Federation Lakehouse Federation A data architecture pattern that lets a single query engine read from multiple external data sources—databases, data lakes, warehouses—without moving or copying the data, reducing pipeline complexity for tools that need broad data access. now supports case-sensitive identifiers for foreign tables in MySQL, SQL Server, BigQuery BigQuery A serverless, cloud-hosted analytical database from Google that runs SQL queries across massive datasets without managing infrastructure, giving cyber tools scalable storage and fast querying over large volumes of log or event data. , Snowflake, and Postgres.
Compute cloning now clones any libraries installed on the original compute
Apr 11, 2024
· release notes
Compute cloning now automatically clones installed libraries.
Adds automatic library cloning during compute clone operations. Not checked yet
1 detail in Everything
Provisioned throughput in Foundation Model APIs is GA and HIPAA compliant
Mar 27, 2024
· release notes
Databricks Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. now offers GA provisioned throughput with HIPAA HIPAA A US federal law enacted in 1996 that sets privacy and security requirements for protected health information, giving cyber tools a compliance baseline to enforce or audit against. compliance
General availability of provisioned throughput model serving for Foundation Model APIs. Not checked yet HIPAA compliance status applied to Foundation Model APIs provisioned throughput workloads. Not checked yet
2 details in Everything
MLflow now enforces quota limits for experiments and runs
Mar 27, 2024
· release notes
MLflow enforces quota limits on parameters, tags, metric steps, and experiment run counts.
Enforces a quota limit on the total number of parameters for all runs. Not checked yet Enforces a quota limit on the total number of tags for all runs. Not checked yet Enforces a quota limit on the total number of metric steps for all runs. Not checked yet Enforces a quota limit on the total number of runs for all experiments. Not checked yet
4 details in Everything
The Jobs UI is updated to better manage jobs deployed by Declarative Automation Bundles
Mar 26, 2024
· release notes
Declarative Automation Bundle jobs are now read-only by default in the Jobs UI to prevent drift.
Accessing resources from serving endpoints using instance profiles is GA
Mar 25, 2024
· release notes
Instance profiles now GA for model serving endpoints to access AWS resources
Google Cloud Vertex AI supported as model provider for external models
Mar 25, 2024
· release notes
Mosaic AI Model Serving adds support for Google Cloud Vertex AI as an external model provider.
Self-service sign-up for private exchange providers in Marketplace
Mar 22, 2024
· release notes
Databricks Marketplace adds self-service sign-up workflow for private exchange providers.
Databricks ODBC ODBC A standard API, maintained by Microsoft, for connecting applications to relational databases regardless of vendor. Cyber tools use it to query databases without writing engine-specific code. driver 2.8.0 adds JWT JWT An open standard (RFC 7519) for encoding claims as a signed, optionally encrypted JSON object passed between parties. Cyber tools use it to verify identity and permissions without querying a central session store. OAuth, token renew, and new dependency versions.
Adds support for JWT assertion as the client credentials for OAuth Not checked yet Adds token renew support for token passthrough authentication Not checked yet Updates third-party libraries: Arrow to 15.0.0 on Windows, libcURL to 8.6.0, and Zlib to 1.3.1 Not checked yet
3 details in Everything
Delegate the ability to view an object's metadata in Unity Catalog (Public Preview)
Mar 15, 2024
· release notes
Adds BROWSE privilege in Unity Catalog Public Preview for metadata discovery without read access
Introduces the BROWSE privilege to grant permission to view an object's metadata without requiring read access to the data itself Not checked yet Allows granting BROWSE on a catalog or external location, with inheritance to all current and future objects within that catalog Not checked yet Enables metadata viewing via Catalog Explorer, schema browser, search results, lineage graph, information_schema, and the REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. for users holding BROWSE without USE CATALOG or USE SCHEMA Not checked yet
3 details in Everything
SQL warehouses for notebooks are now generally available
Adds support for using SQL warehouses within the notebook authoring environment. Not checked yet
1 detail in Everything
Databricks Feature Serving is generally available for external model access.
Databricks Feature Serving reaches general availability. Not checked yet
1 detail in Everything
Restrict changing a job owner and the run as setting
Feb 29, 2024
· release notes
New Account-level configuration option to tighten control over workspace administrator privileges regarding job ownership and execution context.
Restrict creating a personal access token for a service principal
Feb 29, 2024
· release notes
Adds account-level enforcement requiring Service Principal User role to create PATs PATs A personal access token, a credential used by scripts and integrations instead of a user's password. Its permissions and expiry depend on the service issuing it. for service principals.
Account admins can now enforce that workspace admins must have the Service Principal User role on a service principal in order to create a personal access token personal access token A personal access token, a credential used by scripts and integrations instead of a user's password. Its permissions and expiry depend on the service issuing it. on behalf of it. Not checked yet
1 detail in Everything
Support for cloud storage firewall from serverless compute (Public Preview)
Feb 23, 2024
· release notes
Adds cloud storage firewall support for Databricks serverless compute
Account admins can enforce that serverless compute resources use stable IP CIDR CIDR A notation for a block of IP addresses, such as 10.0.0.0/24. The number after the slash says how many leading address bits identify the network. blocks to connect to resources Not checked yet
1 detail in Everything
Unity Catalog volumes are now generally available for managing non-tabular data access in Databricks Runtime 14.3 LTS+.
Use AI Functions to invoke an AI model from Foundation Model APIs
Feb 22, 2024
· release notes
Introduces AI Functions to invoke Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. for tasks like sentiment analysis.
File arrival triggers in Databricks Jobs is GA
Feb 7, 2024
· release notes
File arrival triggers in Databricks Jobs is GA for Unity Catalog volumes and external locations
Databricks Git server proxy is now generally available, allowing users to securely connect Databricks Repos to on-premises Git servers.
Delta Sharing supports tables that use deletion vectors (Public Preview)
Feb 1, 2024
· release notes
Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. Public Preview adds support for sharing tables that use deletion vectors.
Databricks Runtime 14.3 LTS adds native XML support, Cloudflare R2 storage, EXECUTE IMMEDIATE SQL, and standard access mode enhancements
Adds Native XML file format support in Public Preview for ingestion, querying, and parsing of XML data without external jars. Not checked yet Adds Cloudflare R2 storage support Public preview as a cloud storage option for Unity Catalog and Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. . Not checked yet Adds new SQL function EXECUTE IMMEDIATE to support parameterized queries in SQL. Not checked yet Enables querying Structured Streaming state data and metadata for stateful streaming queries. Not checked yet Allows recomputing data skipping statistics for Delta tables after changing columns used for data skipping. Not checked yet
+5 more
show less
Enables Microsoft Entra ID for Kafka authentication on standard access mode clusters via OAuth. Not checked yet Adds support for file and partition pruning with DynamicFilePruning and DynamicPartitionPruning for the EqualNullSafe operator in JOINs. Not checked yet Enables Cloud Fetch by default in AWS workspaces with bucket versioning enabled. Not checked yet Allows declaring temporary variables in a SQL Session using session-scoped variables. Not checked yet Removes support for hive.aux.jars.path and hive.server2.global.init.file.location (or HIVE_CONF_DIR) configurations for hive-thriftserver connections. Not checked yet
10 details in Everything
└──▷ BREAKING ON UPGRADE
! The hive.aux.jars.path property is no longer supported for hive-thriftserver connections.! The hive.server2.global.init.file.location property or HIVE_CONF_DIR environment variable is no longer supported for hive-thriftserver connections.
New UI capability to create Unity Catalog managed tables from files stored in volumes.
Native XML file format support (Public Preview)
Jan 31, 2024
· release notes
Databricks adds native XML file format support in public preview, enabling ingestion, querying, and parsing of XML data.
Adds from_xml SQL expression for parsing XML data Not checked yet Enables reading and writing XML files natively without external jars Not checked yet Supports automatic schema inference and evolution for XML sources Not checked yet
3 details in Everything
Share AI models using Databricks Marketplace (Public Preview)
Jan 31, 2024
· release notes
New capability to share AI models registered in Unity Catalog via Databricks Marketplace Public preview
Enables sharing of AI models registered in Unity Catalog through the Marketplace interface Not checked yet
1 detail in Everything
Monitor GPU model serving workloads using inference tables
Jan 29, 2024
· release notes
Adds inference tables to monitor GPU GPU A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. model serving workloads.
Access controls lists can be enabled on upgraded workspaces
Jan 24, 2024
· release notes
Databricks enables manual activation of access control lists for upgraded Standard-to-Premium workspaces
Enables workspace admins to manually enable Workspace access control on upgraded Standard-to-Premium workspaces. Not checked yet Enables workspace admins to manually enable Cluster, pool, jobs access control on upgraded Standard-to-Premium workspaces. Not checked yet Enables workspace admins to manually enable Cluster visibility control on upgraded Standard-to-Premium workspaces. Not checked yet Enables workspace admins to manually enable Jobs visibility control setting on upgraded Standard-to-Premium workspaces. Not checked yet Enables workspace admins to manually enable Workspace visibility control setting on upgraded Standard-to-Premium workspaces. Not checked yet
5 details in Everything
Marketplace listing events system table now available (Public Preview)
Jan 24, 2024
· release notes
Public Preview of system.marketplace.listing_funnel_events system table for monitoring marketplace listing actions
Share AI models using Delta Sharing (Public Preview)
Jan 19, 2024
· release notes
Databricks workspaces enable sharing AI models via Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. .
Warehouse events system table is now available (Public Preview)
Jan 18, 2024
· release notes
A new system table captures SQL warehouse lifecycle events for monitoring and management purposes.
UI experience for OAuth app registration
Jan 16, 2024
· release notes
Adds UI experience to enable, disable, and modify custom and published OAuth applications.
Enables managing custom OAuth applications directly through the Databricks UI, including enabling and disabling capabilities. Not checked yet
1 detail in Everything
Reuse subnets across workspaces for customer-managed VPCs
Jan 16, 2024
· release notes
Enables sharing subnets across multiple Databricks workspaces for customer-managed VPCs VPCs Virtual Private Clouds — isolated, software-defined network environments inside a public cloud provider such as AWS, Azure, or GCP — give cyber tools a controlled network boundary with configurable routing, subnets, and access rules. .
Adds the ability to share subnets for a customer-managed VPC VPC A logically isolated virtual network inside a public cloud provider's infrastructure, letting tools deploy resources with private addressing, controlled routing, and firewall rules without managing physical hardware. between multiple Databricks workspaces Not checked yet Requires updated VPC configuration via Configure a customer-managed VPC Not checked yet
2 details in Everything
Databricks ODBC ODBC A standard API, maintained by Microsoft, for connecting applications to relational databases regardless of vendor. Cyber tools use it to query databases without writing engine-specific code. driver 2.7.7 adds Azure Managed Identity auth and token cache disable option.
Adds azure.managed_identity authentication method to enable Azure Managed Identity login Not checked yet Adds the ability to disable the token cache for browser-based OAuth flows via configuration Not checked yet Adds support for asynchronous metadata operations requiring server protocol SPARK_CLI_SERVICE_PROTOCOL_V9 or above Not checked yet Adds support for Parameterized Query in Native Query Mode requiring server protocol SPARK_CLI_SERVICE_PROTOCOL_V8 or above Not checked yet
4 details in Everything
Share dynamic views using Delta Sharing (Public Preview)
Dec 21, 2023
· release notes
Databricks allows sharing dynamic views via Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. to restrict data access based on recipient properties in Databricks-to-Databricks flows.
Share volumes using Delta Sharing (Public Preview)
Dec 20, 2023
· release notes
Databricks adds public preview for sharing Unity Catalog Volumes via Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. across workspaces and clouds.
Enables sharing Unity Catalog volumes between Databricks workspaces on different Unity Catalog metastores using Delta Sharing in Public Preview Not checked yet Provides native integration with Unity Catalog to manage, govern, audit, and track usage of shared non-tabular data assets stored in Volumes Not checked yet
2 details in Everything
External models support in Model Serving is in Public Preview
Dec 11, 2023
· release notes
Mosaic AI Model Serving now allows users to add endpoints for accessing models hosted outside of Databricks.
New notebook cell results rendering available (Public Preview)
Dec 8, 2023
· release notes
New public preview notebook cell result rendering with search and copy capabilities.
Enables new tabular output rendering in notebooks that supports searching results and copying subsets to the clipboard Not checked yet
1 detail in Everything
Databricks Online Tables is Public Preview
Dec 8, 2023
· release notes
Databricks Online Tables enter public preview as serverless tables with low-latency access for model serving and RAG RAG Retrieval-augmented generation: looking up relevant documents and adding them to the prompt before the model answers. It grounds responses in private or current data without retraining. workloads.
Public Preview of Databricks Online Tables : fully serverless tables that auto-scale with request load for low latency and high throughput. Not checked yet Online Tables are optimized for use with Mosaic AI Model Serving , Feature & Function Serving , and retrieval-augmented generation retrieval-augmented generation Retrieval-augmented generation: looking up relevant documents and adding them to the prompt before the model answers. It grounds responses in private or current data without retraining. (RAG) applications. Not checked yet
2 details in Everything
Databricks JDBC JDBC A Java standard API for connecting applications to relational databases, maintained as part of the Java SE platform, letting tools query any compliant database through a single consistent interface. driver 2.6.36 adds native OAuth U2M OAuth U2M OAuth flow in which a human user delegates scoped access to a service on their behalf, letting cyber tools act with the user's permissions without handling raw credentials. and M2M support
Adds oauthInteractiveFlow for native browser-based user-to-machine authentication Not checked yet Adds oauthServiceAccountClientCredentials to allow setting client ID and secret for machine-to-machine auth Not checked yet
2 details in Everything
Support for referencing workspace files from init scripts
Dec 6, 2023
· release notes
Databricks now supports referencing other workspace files (libraries, configs) from init scripts.
Adds ability to reference workspace files like libraries, configuration files, or shell scripts from init scripts stored with workspace files Not checked yet
1 detail in Everything
Feature & Function Serving is Public Preview
Dec 5, 2023
· release notes
Databricks Feature & Function Serving enters public preview for external model inference.
Feature & Function Serving endpoints are now available in public preview to expose Databricks data to external models and applications. Not checked yet Endpoints provide automatic scaling for real-time traffic, offering high availability and low latency. Not checked yet
2 details in Everything
Foundation Model APIs is Public Preview
Dec 4, 2023
· release notes
Databricks adds Public Preview for Foundation Model APIs Foundation Model APIs Hosted interfaces to large pretrained AI models—such as those from OpenAI, Anthropic, or Google—that let cyber tools query reasoning or language capabilities without training or hosting a model themselves. via dedicated model serving endpoints.
Adds access to state-of-the-art open source models through a dedicated Foundation Model APIs endpoint available in your Databricks Workspace Not checked yet Introduces pay-per-token pricing for invoking these new AI model APIs Not checked yet
2 details in Everything
Databricks introduces a new unified admin settings UI consolidating SQL settings with improved navigation.
Updates the workspace admin settings UI to incorporate previously separate SQL settings Not checked yet Adds easy navigation between workspace settings and user settings in the admin interface Not checked yet
2 details in Everything
Mosaic AI Vector Search is Public Preview
Nov 30, 2023
· release notes
Mosaic AI Vector Search is now in public preview. It provides a serverless similarity search engine for storing vector representations and metadata in indexes.
Access controls lists can no longer be disabled
Nov 29, 2023
· release notes
Workspace and object access controls are now permanently enabled on Premium+ workspaces.
Access control settings (Workspace, Cluster/pool/jobs, Cluster visibility, Jobs visibility, Workspace visibility) can no longer be disabled from the workspace admin settings page or the Workspace Conf API for Premium plan or above workspaces. Not checked yet
1 detail in Everything
└──▷ BREAKING ON UPGRADE
! It is no longer possible to disable Workspace access control, Cluster/pool/jobs access control, Cluster visibility control, Jobs visibility control, or Workspace visibility control via the workspace admin settings page or the Workspace Conf API on Premium+ workspaces.
New behaviors and actions in Catalog Explorer for volumes
Nov 22, 2023
· release notes
New Create volume action and Delete/Download file actions in Catalog Explorer for Unity Catalog volumes.
Adds the ability to create a new volume in the target schema directly from the Upload to volume UI if none exists Not checked yet Adds Delete file action to Catalog Explorer for files in volumes Not checked yet Adds Download file action to Catalog Explorer for files in volumes Not checked yet
3 details in Everything
Databricks SQL Connector for Python version 3.0.0
Nov 17, 2023
· release notes
Databricks SQL Connector for Python v3.0 adds native parameterized queries and SQLAlchemy >=2.0 support
Adds support for native parameterized SQL queries (requires Databricks Runtime 14.2+) Not checked yet Completely rewrites the SQLAlchemy dialect to support SQLAlchemy >= 2.0 Not checked yet Enables Cloud Fetch by default via use_cloud_fetch config on databricks.sql.client Not checked yet Adds Numeric type scale and precision support (e.g. Numeric(10,2) ) Not checked yet Adds support for new data types: TIME, TIMESTAMP, TIMESTAMP_NTZ, TINYINT Not checked yet
+2 more
show less
Adds reading/writing of PrimaryKeyConstraint and ForeignKeyConstraint via SQLAlchemy dialect Not checked yet Adds support for paging through results using LIMIT and OFFSET Not checked yet
7 details in Everything
└──▷ BREAKING ON UPGRADE
! Removes support for Python 3.7! Drops support for SQLAlchemy 1.x in the dialect
Delegate the ability to create a storage credential in Unity Catalog
Nov 15, 2023
· release notes
Databricks introduces CREATE STORAGE CREDENTIAL privilege to delegate storage credential creation in Unity Catalog.
Consume data products in Databricks Marketplace using external platforms
Nov 10, 2023
· release notes
Databricks Marketplace now allows consumption of tabular data via Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. connectors for non-Unity Catalog workspaces
Enables consumers without Unity Catalog-enabled Databricks workspaces to access data products in Databricks Marketplace using Delta Sharing open sharing connectors. Not checked yet Extends Marketplace data access to external platforms including Microsoft Power BI, Microsoft Excel, pandas, Apache Spark, and non-Unity Catalog Databricks workspaces. Not checked yet
2 details in Everything
OAuth machine-to-machine (M2M) and user-to-machine (U2M) authentication is GA for Databricks.
Automatic enablement of Unity Catalog for new workspaces
Nov 8, 2023
· release notes
Automatic enablement of Unity Catalog for new workspaces.
Lakehouse Federation adds support for Google BigQuery
Nov 3, 2023
· release notes
Lakehouse Federation Lakehouse Federation A data architecture pattern that lets a single query engine read from multiple external data sources—databases, data lakes, warehouses—without moving or copying the data, reducing pipeline complexity for tools that need broad data access. now supports the BigQuery BigQuery A serverless, cloud-hosted analytical database from Google that runs SQL queries across massive datasets without managing infrastructure, giving cyber tools scalable storage and fast querying over large volumes of log or event data. connector type for federated queries.
Published partner OAuth applications are easier to use by default
Oct 31, 2023
· release notes
Published partner OAuth apps enabled by default, removing need for per-account activation commands
Published partner OAuth applications are now enabled by default for all Databricks accounts, allowing seamless sign-on from partner BI tools without personal access tokens personal access tokens A personal access token, a credential used by scripts and integrations instead of a user's password. Its permissions and expiry depend on the service issuing it. Not checked yet
1 detail in Everything
Configure parameters on a Databricks job that can be referenced by all job tasks
Oct 30, 2023
· release notes
Databricks adds job-level parameters and dynamic value references for passing context between tasks.
Adds the ability to configure parameters on a Databricks job that are automatically passed to all job tasks accepting key-value pairs Not checked yet Adds support for an expanded set of dynamic value references to pass context and state between job tasks Not checked yet
2 details in Everything
View the YAML source for a Databricks job
Oct 30, 2023
· release notes
Adds View YAML/JSON option to Databricks job details page for exporting job source.
Predictive optimization (Public Preview)
Oct 24, 2023
· release notes
Databricks introduces predictive optimization to auto-manage Delta table maintenance.
Predictive optimization removes the need for manual maintenance operations. Not checked yet
1 detail in Everything
Compute system tables are now available (Public Preview)
Oct 23, 2023
· release notes
Databricks adds Public Preview of system.compute schema with clusters and node_types tables for compute monitoring
system.compute.clusters table provides full history of all-purpose and jobs cluster configurations over time. Not checked yet system.compute.node_types table captures currently available node types with basic hardware information. Not checked yet
2 details in Everything
Feature Engineering in Unity Catalog is GA
Oct 19, 2023
· release notes
Feature Engineering in Unity Catalog is GA; any Delta table with a primary key can now serve as a feature store.
Databricks GA on-demand feature computation for inference-time inputs and complex graphs
Enables computation of ML features at inference time using inputs only available then (e.g., user location) Not checked yet Allows training code to define arbitrary graphs of feature lookups and computations executed during training and inference Not checked yet Governs feature computation logic, models, and data via Unity Catalog Not checked yet
3 details in Everything
AI-generated table comments (Public Preview)
Oct 18, 2023
· release notes
Databricks introduces Public Preview for AI-generated table and column comments in Unity Catalog via Catalog Explorer.
Adds AI-generated comments to tables and columns in Catalog Explorer, powered by LLMs LLMs A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. analyzing metadata like schema and column names. Not checked yet Enables viewing, editing, and adding AI-generated comments for any managed object in Unity Catalog during Public Preview. Not checked yet
2 details in Everything
Libraries now supported in compute policies (Public Preview)
Oct 17, 2023
· release notes
Databricks adds library installation via compute policies Public preview
Workspace admins can add libraries to compute policies, enabling automatic installation on associated compute resources. Not checked yet Compute-scoped libraries installed via policy are now locked from user install/uninstall actions, and previous manual installations are uninstalled upon policy assignment. Not checked yet
2 details in Everything
└──▷ BREAKING ON UPGRADE
! Previous manually installed libraries on compute associated with a new library policy are uninstalled when the policy is applied.
Models in Unity Catalog is GA with centralized access control and lineage
General availability of Models in Unity Catalog for centralized model lifecycle management. Not checked yet Centralized access control and auditing capabilities for ML models in Unity Catalog. Not checked yet Model lineage tracking across workspaces in Unity Catalog. Not checked yet Enhanced MLOps deployment workflows via Unity Catalog. Not checked yet
4 details in Everything
Partner Connect supports Monte Carlo
Oct 16, 2023
· release notes
Partner Connect supports connecting Databricks workspaces to Monte Carlo
Adds Monte Carlo as a supported destination in Partner Connect for workspace integration Not checked yet
1 detail in Everything
New Databricks clusters now enable Photon by default for compatible runtimes and node types.
Sets Photon as the default runtime_engine for new all-purpose and job clusters created via the UI when using a compatible cluster policy. Not checked yet Automatically enables Photon for new clusters where Databricks Runtime supports it, the node_type is supported, and runtime_engine is not explicitly set to STANDARD. Not checked yet
2 details in Everything
IP access lists for the account console is GA
Oct 11, 2023
· release notes
IP access lists for Databricks Account Console reach GA, allowing IP range-based access control.
IP access lists for the account console are now generally available to control access by IP address ranges. Not checked yet
1 detail in Everything
Use workspace-catalog binding to give read-only access to a catalog
Oct 4, 2023
· release notes
Databricks adds read-only workspace-catalog bindings and a new API endpoint to replace the deprecated workspace-bindings path.
Allows workspace-catalog binding configurations to enforce read-only access for specific workspaces Not checked yet Introduces /api/2.1/unity-catalog/bindings/ as the new API endpoint for managing these bindings Not checked yet
2 details in Everything
└──▷ BREAKING ON UPGRADE
! The /api/2.1/unity-catalog/workspace-bindings/ API endpoint is deprecated and replaced by /api/2.1/unity-catalog/bindings/
Databricks SDK for Python updated to version 0.10.0 (Beta)
Prerelease
Oct 3, 2023
· release notes
Databricks SDK for Python 0.10.0 adds a new service, 10 dataclasses, and 6 fields alongside 7 breaking changes.
Adds the Jobs service to interact with Databricks Jobs programmatically. Not checked yet Introduces 10 new dataclasses for typed access to API resources. Not checked yet Adds 6 new fields across existing models. Not checked yet
3 details in Everything
Databricks extension for Visual Studio Code updated to version 1.1.4
Oct 2, 2023
· release notes
Databricks extension for VS Code v1.1.4 adds custom workspace URL support
Lakehouse Federation available on single user clusters
Sep 29, 2023
· release notes
Lakehouse Federation Lakehouse Federation A data architecture pattern that lets a single query engine read from multiple external data sources—databases, data lakes, warehouses—without moving or copying the data, reducing pipeline complexity for tools that need broad data access. is now available on single user clusters with Databricks Runtime 13.1 and above.
GPU model serving optimized for LLMs in Public Preview
Sep 28, 2023
· release notes
Databricks adds public preview for GPU GPU A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. -accelerated PyTorch and TensorFlow model serving, optimized for LLMs LLMs A large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. .
Prevent runs of a job from being skipped when you reach concurrency limits
Sep 27, 2023
· release notes
Databricks adds queueing option for jobs to prevent skips at concurrency limits
Adds the queueing job configuration option to keep runs from being skipped when concurrency limits are reached Not checked yet
1 detail in Everything
Databricks CLI updated to version 0.206.0 (Public Preview)
Sep 27, 2023
· release notes
Databricks CLI 0.206.0 adds pipeline cluster overrides in Declarative Automation Bundles and improves bundle init UX.
Adds target overrides for pipeline clusters within Declarative Automation Bundles Not checked yet Improves the bundle template initialization experience Not checked yet
2 details in Everything
Databricks Terraform provider updated to version 1.27.0
Sep 27, 2023
· release notes
Databricks Terraform provider v1.27.0 adds owner and force_destroy args to databricks_metastore_data_access.
Adds the owner attribute to the databricks_metastore_data_access resource to specify the metastore owner. Not checked yet Adds the force_destroy argument to the databricks_metastore_data_access resource to allow deleting the associated AWS IAM AWS IAM AWS Identity and Access Management is Amazon's service for controlling who and what can call AWS APIs, by attaching policies that allow or deny actions on resources. Cyber tools use it to scope credentials to least privilege. role even if it still has attached policies. Not checked yet
2 details in Everything
Databricks SDK for Go updated to version 0.21.0 (Beta)
Prerelease
Sep 27, 2023
· release notes
Databricks SDK for Go 0.21.0 adds over a dozen new APIs and fields with breaking API changes.
└──▷ BREAKING ON UPGRADE
! Introduces over a dozen breaking API changes that will require updates to existing Go code using the SDK
Inference tables for model serving endpoints is Public Preview
Sep 26, 2023
· release notes
Public Preview of Inference tables allows model serving endpoints to log request/response data to Unity Catalog Delta tables for monitoring and debugging.
Databricks ODBC ODBC A standard API, maintained by Microsoft, for connecting applications to relational databases regardless of vendor. Cyber tools use it to query databases without writing engine-specific code. driver 2.7.5 adds OAuth 2.0 OAuth 2.0 An open authorization framework (IETF RFC 6749) that lets a service grant limited API access to a third party without sharing credentials, giving cyber tools a standard way to authenticate against external platforms. client auth, OIDC OIDC OpenID Connect, a sign-in protocol built on OAuth 2.0. An application uses identity information from a trusted provider instead of maintaining a separate password for each user. discovery, and customer-provided key encryption.
Adds OAuth 2.0 client authentication enhancement for connecting to Databricks workspaces Not checked yet Adds support for automatic OIDC endpoint discovery to simplify connection configuration Not checked yet Enables server-side encryption using customer-provided keys (BYOK BYOK Bring Your Own Key — a cloud deployment model where the customer supplies and controls the encryption keys rather than the provider, so sensitive data stays under the customer's cryptographic authority. ) for data at rest in transit Not checked yet
3 details in Everything
Running jobs as a service principal is GA
Sep 21, 2023
· release notes
General Availability of running Databricks jobs as a service principal identity
run_as setting for job runs now supports service principals for production use Not checked yet
1 detail in Everything
Configure Tableau and Power BI OAuth with SAML SSO
Sep 20, 2023
· release notes
Adds support for configuring Tableau and Power BI OAuth login using SAML SSO SAML SSO An authentication protocol that lets a user log in once with an identity provider and access multiple services without re-entering credentials, by passing signed XML assertions between parties. instead of only OIDC OIDC OpenID Connect, a sign-in protocol built on OAuth 2.0. An application uses identity information from a trusted provider instead of maintaining a separate password for each user. .
Enables SAML SAML A standard for passing signed identity assertions between a sign-in provider and an application. It lets a user sign in through their organization's identity provider. -based SSO SSO Single sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. configuration for Tableau OAuth login Not checked yet Enables SAML-based SSO configuration for Power BI OAuth login Not checked yet
2 details in Everything
Databricks Connect for Databricks Runtime 14.0
Sep 20, 2023
· release notes
Databricks Connect for Runtime 14.0 adds full Structured Streaming support including foreachBatch.
Supports long running queries that reattach to the query after interruption Not checked yet
1 detail in Everything
Databricks CLI updated to version 0.205.1 (Public Preview)
Sep 20, 2023
· release notes
Databricks CLI 0.205.1 introduces target overrides for job tasks in Declarative Automation Bundles, enhancing automation capabilities.
Databricks SDK for Go updated to version 0.20.0 (Beta)
Prerelease
Sep 20, 2023
· release notes
Databricks SDK for Go v0.20.0 adds over a dozen APIs and six fields
Adds half-a-dozen new fields to existing structures in the SDK. Not checked yet
1 detail in Everything
└──▷ BREAKING ON UPGRADE
! The release notes state that a few APIs have been removed from version 0.20.0; consult the changelog for specific names.
Databricks SDK for Python updated to version 0.9.0 (Beta)
Prerelease
Sep 20, 2023
· release notes
Databricks SDK for Python 0.9.0 Beta adds ~20 dataclasses and ~10 fields with method and field changes.
Adds approximately 20 new dataclasses to the Databricks SDK for Python Not checked yet Adds approximately 10 new fields to existing dataclasses Not checked yet Changes methods and fields in the SDK Not checked yet
3 details in Everything
Databricks Connect V2 is Public Preview for Scala
Sep 19, 2023
· release notes
Databricks Connect V2 Public Preview for Scala added alongside Python query cancellation support
Databricks Connect for Databricks Runtime 13.3 LTS now supports Scala in Public Preview, including DataFrames, Delta, Structured Streaming, and more. Not checked yet query cancellation is now supported for Python . Not checked yet DatabricksSession class can now be used in Databricks notebooks and jobs. Not checked yet
3 details in Everything
Databricks model serving now supports GPU GPU A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. compute for PyTorch and TensorFlow models logged via mlflow.pyfunc, mlflow.pytorch, mlflow.tensorflow, and mlflow.transformers.
On-demand feature computation now available in Unity Catalog (Public Preview)
Sep 14, 2023
· release notes
Unity Catalog Public preview adds on-demand machine learning feature computation at inference time
Databricks CLI updated to version 0.205.0 (Public Preview)
Sep 13, 2023
· release notes
Databricks CLI 0.205.0 adds initial support for Declarative Automation Bundles in Public Preview.
Declarative Automation Bundles (Public Preview)
Sep 12, 2023
· release notes
Databricks introduces Declarative Automation Bundles in public preview to define data, analytics, and ML projects via source files for CI/CD integration.
Declarative Automation Bundles allow end-to-end data, analytics, and ML projects to be expressed as a collection of source files. Not checked yet Supports applying source control, code review, testing, and CI/CD best practices to data engineering workflows. Not checked yet
2 details in Everything
Databricks extension for Visual Studio Code updated to version 1.1.2
Sep 12, 2023
· release notes
Adds experimental support for %run and # MAGIC in Databricks notebooks
Filter sensitive data with row filters and column masks (Public Preview)
Sep 11, 2023
· release notes
Databricks adds Public Preview for row filters and column masks implemented as SQL UDFs to filter sensitive data.
Adds support for row filters and column masks in tables, allowing users to restrict access to sensitive data by implementing these controls as SQL user-defined functions (UDFs). Not checked yet
1 detail in Everything
Pricing system table is now available (Public Preview)
Sep 8, 2023
· release notes
Databricks adds system.billing.list_prices system table to track SKU costs in public preview
Adds system.billing.list_prices system table for monitoring SKU cost and pricing changes over time Not checked yet
1 detail in Everything
System tables now include Marketplace schema (Public Preview)
Sep 8, 2023
· release notes
Databricks System Tables add Marketplace schema for querying listing access events
System tables now include a marketplace schema that stores Marketplace provider data. Not checked yet Adds the marketplace.listing_access_events table to retrieve consumer info for actions taken on listings. Not checked yet
2 details in Everything
Databricks CLI updated to version 0.204.0 (Public Preview)
Sep 6, 2023
· release notes
Databricks CLI v0.204.0 groups permission commands and updates existing ones
Groups previously scattered permission-related commands into dedicated subcommands. Not checked yet Updates behavior and flags across several existing Databricks CLI commands. Not checked yet
2 details in Everything
Databricks SDK for Python updated to version 0.8.0 (Beta)
Prerelease
Sep 4, 2023
· release notes
Databricks SDK Python v0.8.0 Beta: renames permissions APIs, adds fields/dataclasses, introduces changes/removals.
Renames the existing permissions APIs for clarity and consistency in the SDK client methods. Not checked yet
1 detail in Everything
└──▷ BREAKING ON UPGRADE
! The permissions API methods have been renamed; calling them by their previous names will raise an AttributeError.
Databricks SDK for Go updated to version 0.19.0 (Beta)
Prerelease
Sep 4, 2023
· release notes
Databricks SDK Go 0.19.0 adds 8 APIs and renames permissions APIs
Adds 8 new APIs to the Databricks SDK for Go client library Not checked yet Adds 5 new fields to existing SDK structs Not checked yet
2 details in Everything
GitHub apps integration in Repos is GA
Sep 1, 2023
· release notes
GitHub apps integration in Databricks Repos is now generally available.
Databricks SDK for Python updated to version 0.7.0 (Beta)
Prerelease
Aug 29, 2023
· release notes
Databricks SDK for Python v0.7.0 introduces files API breaking changes and new dataclasses
Standard access mode (formerly shared access mode) clusters can now connect to public networks outside of Databricks VPC
Aug 25, 2023
· release notes
Standard access mode clusters can now connect to public networks outside of Databricks VPC VPC A logically isolated virtual network inside a public cloud provider's infrastructure, letting tools deploy resources with private addressing, controlled routing, and firewall rules without managing physical hardware. on ports other than 80/443
Go to definition for functions and variables in Python notebooks
Aug 24, 2023
· release notes
Adds Go to definition for functions and variables in Python notebooks
Enables automatic navigation to the definition of a variable or function defined within an attached Python notebook in Databricks Runtime 12.2 LTS and above Not checked yet
1 detail in Everything
Volumes support for init scripts and JARs is in Public Preview
Aug 23, 2023
· release notes
Databricks introduces Public Preview support for storing init scripts and JARs in Unity Catalog volumes on Databricks Runtime 13.3+ with assigned or shared access.
Allowlist for init scripts, JARs, and Maven coordinates on Unity Catalog shared clusters is in Public Preview
Aug 23, 2023
· release notes
Databricks now offers a public preview of the Unity Catalog allowlist to restrict init scripts, JARs, and Maven coordinates on shared access mode clusters with Databricks Runtime 13.3+.
Databricks adds tag support for Unity Catalog securable objects.
Easier Databricks Repos .ipynb file output commits
Aug 21, 2023
· release notes
Databricks Repos adds commit_output config file and UI notifications for controlling .ipynb notebook output commits.
commit_output config file: Repos can now create this config file to control notebook output commit behavior. Not checked yet UI Notifications: The UI now displays status notifications when .ipynb output commits are processed. Not checked yet File menu control: You can use the File menu in Databricks Repos to toggle whether notebook outputs are included in commits. Not checked yet
3 details in Everything
Databricks SDK for Python updated to version 0.6.0 (Beta)
Prerelease
Aug 18, 2023
· release notes
Databricks SDK for Python v0.6.0 Beta adds ~20 dataclasses and ~10 new APIs
Adds approximately 20 new dataclasses to the Databricks SDK for Python Not checked yet Adds approximately 10 new APIs to the Databricks SDK for Python Not checked yet
2 details in Everything
Databricks GA brings unified schema browser with condensed tree and object count filtering.
Adds a unified schema browser across notebooks, SQL Editor, and Catalog Explorer Not checked yet Displays Unity Catalog object counts at the schema level when filtering Not checked yet
2 details in Everything
Programmatic write support for workspace files
Aug 11, 2023
· release notes
Databricks adds programmatic write capabilities to workspace files for Databricks Runtime 11.2+.
Clusters running Databricks Runtime 11.2 and above now support programmatically creating, updating, and deleting files and directories within the workspace. Not checked yet
1 detail in Everything
Databricks SDK for Python updated to version 0.5.0 (Beta)
Prerelease
Aug 11, 2023
· release notes
Databricks SDK for Python 0.5.0 Beta updates API signatures and adds fields/dataclasses
Changes the signatures of a half-dozen APIs in Databricks SDK for Python version 0.5.0 Beta Not checked yet Adds a dozen fields to the SDK models Not checked yet Removes a half-dozen fields from the SDK models Not checked yet
3 details in Everything
└──▷ BREAKING ON UPGRADE
! The signatures of a half-dozen APIs in Databricks SDK for Python version 0.5.0 Beta have changed, which may break existing code that relies on the previous API structure
Access resources from serving endpoints with instance profiles (Public Preview)
Aug 10, 2023
· release notes
Add an instance profile to a model serving endpoint to access AWS resources.
Databricks SDK for Go updated to version 0.15.0 (Beta)
Prerelease
Aug 7, 2023
· release notes
Databricks SDK for Go 0.15.0 Beta refactors permissions-related APIs.
refactors permissions-related APIs to make them more understandable and easier to use Not checked yet Updated to version 0.15.0 Not checked yet
2 details in Everything
Databricks SDK for Python updated to version 0.4.0 (Beta)
Prerelease
Aug 7, 2023
· release notes
Databricks SDK for Python v0.4.0 Beta introduces a refactor of its permissions-related APIs, aimed at improving developer experience and code clarity.
Feature Engineering in Unity Catalog is Public Preview
Aug 3, 2023
· release notes
Unity Catalog becomes a feature store in public preview, supporting Delta tables with primary keys.
Unity Catalog serves as a feature store for model training or inference Not checked yet
1 detail in Everything
Share schemas using Delta Sharing (Public Preview)
Aug 2, 2023
· release notes
Share entire schemas via Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. in Public Preview
Enables SQL-based schema sharing requiring a SQL warehouse or Databricks Runtime 13.2+ cluster Not checked yet Enables Catalog Explorer UI-based schema sharing with no compute requirements Not checked yet
2 details in Everything
Run tasks conditionally in your Databricks jobs
Aug 2, 2023
· release notes
Databricks Jobs adds Run if condition to configure task execution based on dependency status.
Lakehouse Monitoring is Public Preview
Aug 1, 2023
· release notes
Databricks Lakehouse Monitoring enters public preview with dashboard generation and alerting capabilities
Adds Lakehouse Monitoring feature in public preview to integrate data and model quality monitoring into the Databricks platform Not checked yet Enables automatic summary dashboard generation for visualizing data quality over time Not checked yet Allows setup of alerts based on monitored data metrics Not checked yet
3 details in Everything
Compliance security profile works with serverless SQL warehouses in some regions (Public Preview)
Aug 1, 2023
· release notes
Databricks Compliance Security Profile now supports serverless SQL warehouses in us-east-1 Public Preview.
New Git operations are generally available: Merge branches, rebase and pull with conflict resolution
Aug 1, 2023
· release notes
Databricks Git operations (merge, rebase, pull with conflict resolution) are GA in Databricks Repos.
Enables merge and rebase of branches directly in Databricks Repos Not checked yet Enables pull from a remote Git repo with conflict resolution in Databricks Repos Not checked yet
2 details in Everything
Group manager permission is in Public Preview
Jul 31, 2023
· release notes
Public Preview adds group manager permission allowing users to manage group membership and delegate permissions
Workspace admins can now create account groups
Jul 31, 2023
· release notes
Workspace admins can now create account groups from their identity federated workspaces
Databricks SDK for Python updated to version 0.3.0 (Beta)
Prerelease
Jul 27, 2023
· release notes
Databricks SDK for Python 0.3.0 adds Account Network Policy service support and nested query parameter handling.
Adds AccountNetworkPolicy service client to the SDK for managing account-level network policies Not checked yet
1 detail in Everything
Run another job as a task in a Databricks job
Jul 25, 2023
· release notes
Databricks adds a Run Job task type to decompose workflows into component jobs.
Lakehouse Federation lets you run queries against external database providers (Public Preview)
Jul 21, 2023
· release notes
Lakehouse Federation Lakehouse Federation A data architecture pattern that lets a single query engine read from multiple external data sources—databases, data lakes, warehouses—without moving or copying the data, reducing pipeline complexity for tools that need broad data access. Public Preview allows running queries against external database providers via Unity Catalog.
Introduces Lakehouse Federation, a query federation platform enabling ad hoc reporting and exploratory analysis against multiple external data sources without migrating data to Unity Catalog. Not checked yet
1 detail in Everything
Repos can now be moved to trash for recovery within 30 days.
Databricks CLI updated to version 0.201.0 (Public Preview)
Jul 18, 2023
· release notes
Databricks CLI v0.201.0 enhances usability with new profile tab completion and a refined authentication login flow.
Adds tab completion for referencing Databricks authentication configuration profiles Not checked yet Improves the auth login experience Not checked yet
2 details in Everything
Databricks Assistant is in Public Preview
Jul 18, 2023
· release notes
Databricks Assistant is now in public preview, offering an AI companion for code and query assistance.
AI-based pair-programmer for notebooks, queries, and files Not checked yet
1 detail in Everything
Deactivate users and service principals from your account
Jul 13, 2023
· release notes
Databricks adds ability to deactivate users and service principals without removing permissions.
Adds the capability to deactivate a user or service principal from a Databricks account Not checked yet Deactivated identities can no longer log in to the Databricks account or identity federated workspaces Not checked yet Deactivation preserves all existing permissions and workspace objects for the impacted identity Not checked yet
3 details in Everything
Unity Catalog volumes are in Public Preview
Jul 12, 2023
· release notes
Unity Catalog volumes enter Public Preview for managing cloud object storage access and governance on Databricks Runtime 13.2+
Introduces Volumes in Unity Catalog as a new resource type for accessing, storing, governing, and organizing files in cloud object storage Not checked yet Enables volume management capabilities starting with Databricks Runtime 13.2 Not checked yet
2 details in Everything
Functions now displayed in Catalog Explorer (Public Preview)
Jul 10, 2023
· release notes
Catalog Explorer now supports viewing and managing permissions for SQL and Python UDFs in Public Preview.
Adds functions visibility to Catalog Explorer for SQL and Python UDFs, allowing users to view function details and manage permissions via a new UI interface. Not checked yet Enables permission management capabilities directly within the Catalog Explorer UI for newly displayed user-defined functions. Not checked yet
2 details in Everything
Databricks extension for Visual Studio Code updated to version 1.1.0
Jul 10, 2023
· release notes
Databricks VS Code extension 1.1.0 enables Databricks Connect by default and adds experimental Jupyter notebook support.
Databricks Terraform provider updated to version 1.21.0
Jul 7, 2023
· release notes
Databricks Terraform provider v1.21.0 adds dashboard subscriptions and account-level data sources
Adds subscriptions attribute to databricks_dashboard resource for dashboard subscription management Not checked yet Adds subscriptions support in alert SQL tasks within the databricks_job resource Not checked yet Defines generic Databricks data utilities for defining workspace and account-level data sources Not checked yet
3 details in Everything
Databricks Terraform provider updated to version 1.20.0
Jun 29, 2023
· release notes
Databricks Terraform provider 1.20.0 adds Unity Catalog binding resource and improved account-level user/group export
Adds databricks_catalog_workspace_binding resource to manage Unity Catalog workspace bindings Not checked yet Improves the export of users and groups at the account level Not checked yet
2 details in Everything
Databricks SDK for Python is now available in Beta for automating accounts, workspaces, and resources.
Databricks SDK for Go updated to version 0.12.0 (Beta)
Prerelease
Jun 28, 2023
· release notes
Databricks SDK for Go 0.12.0 adds job-level parameters and task conditional execution support.
Adds job-level-parameters support to enable passing dynamic values at runtime Not checked yet Adds job-task-conditional-execution capability allowing tasks to depend on previous task states Not checked yet
2 details in Everything
Databricks CLI updated to version 0.200.1 (Public Preview)
Jun 28, 2023
· release notes
Databricks CLI 0.200.1 adds local file default and provider detection for fs and repos commands
Local files are now the default for fs ls and fs cp commands, removing the need to specify remote URIs for local operations Not checked yet repos create command gains automatic cloud provider detection during workspace registration Not checked yet
2 details in Everything
Databricks SDK for Go Beta releases, enabling automation of accounts and workspaces via Go.
Access audit log, billable usage, and lineage system tables (Public Preview)
Jun 28, 2023
· release notes
Databricks adds public preview of system tables for account-wide audit, usage, and lineage data.
Databricks Marketplace now GA with private exchanges and USE MARKETPLACE ASSETS privilege
Introduces the USE MARKETPLACE ASSETS privilege to simplify consumer permission management for Marketplace access. Not checked yet Adds support for private exchanges within Databricks Marketplace, allowing organizations to restrict data product distribution. Not checked yet
2 details in Everything
Databricks extension for Visual Studio Code (General Availability)
Jun 26, 2023
· release notes
Databricks VS Code extension reaches general availability with IDE-integrated lakehouse connectivity and debugging.
The Databricks IDE extension is now generally available for Visual Studio Code Not checked yet Enables connecting to the lakehouse without leaving the IDE Not checked yet Provides interactive debugging capabilities within the IDE Not checked yet Supports running Jupyter notebooks directly in VS Code Not checked yet Integrates code navigation, refactoring, completion, and error identification Not checked yet
5 details in Everything
See a visual overview of completed job runs in the Databricks Jobs UI
Jun 22, 2023
· release notes
Databricks Jobs UI now offers a visual overview of completed job runs with filtering and common error lists.
Enhanced Databricks Jobs UI provides a visual overview of completed job runs, filterable by run status and time, listing the most common job errors (default time filter: previous 48 hours) Not checked yet
1 detail in Everything
Unified Login (SSO SSO Single sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. ) enabled by default for new Databricks accounts created after June 21, 2023
└──▷ BREAKING ON UPGRADE
! New accounts no longer support separate workspace-level SSO management; all workspaces and users must sign in using the account-level SSO configuration.
Unified login for accounts created before June 21, 2023 (Public Preview)
Jun 21, 2023
· release notes
Databricks adds Public Preview of unified login allowing a single SSO SSO Single sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. configuration across account and workspaces for legacy accounts.
Unifies account-level SSO configuration to apply to workspaces via unified login in Public Preview for accounts created before June 21, 2023 Not checked yet
1 detail in Everything
Full-page workspace browser includes Repos
Jun 14, 2023
· release notes
Full-page workspace browser now supports browsing Databricks Repos alongside workspace content
Adds Repos to the full-page workspace browser, allowing users to browse repository content alongside workspace assets via the sidebar. Not checked yet Adds a contextual browser for navigating Repos directly from within a notebook view. Not checked yet
2 details in Everything
Databricks Terraform provider updated to version 1.19.0
Jun 14, 2023
· release notes
Databricks Terraform provider 1.19.0 adds run_as fields to jobs, force_destroy to external locations, and cluster policy attributes
Adds run_as and run_as_user_name arguments to the databricks_job resource to control job execution identity. Not checked yet Adds force_destroy argument to databricks_external_location resource for destructive deletion of underlying storage. Not checked yet Expands the schema for databricks_cluster_policy by adding several new attributes. Not checked yet
3 details in Everything
Improved pagination of results from List all jobs and List job runs API requests
Jun 14, 2023
· release notes
New page_token parameter for Jobs API list endpoints, replacing offset/limit
Adds page_token parameter to GET /api/2.1/jobs/list endpoint Not checked yet Adds page_token parameter to GET /api/2.1/jobs/runs/list endpoint Not checked yet
2 details in Everything
└──▷ BREAKING ON UPGRADE
! offset and limit parameters are deprecated on GET /api/2.1/jobs/list and GET /api/2.1/jobs/runs/list, requiring migration to page_token
Run jobs as a service principal (Public Preview)
Jun 13, 2023
· release notes
Databricks Public Preview allows running jobs via a service principal identity to prevent failures from user churn.
Adds the ability for users and groups to be granted permissions to use a service principal for job runs instead of the job owner's identity Not checked yet Releases support for configuring the Run as user for job runs to specify the service principal identity Not checked yet
2 details in Everything
New service principal UI provides better management experience
Jun 13, 2023
· release notes
New service principal UI for adding principals, configuring entitlements, and managing permissions.
Adds a new service principal UI for managing service principals in your workspace Not checked yet Enables configuration of service principal entitlements via the new UI Not checked yet Allows management of service principal permissions via the new UI Not checked yet
3 details in Everything
Databricks Marketplace: Private exchanges are now available
Jun 7, 2023
· release notes
Databricks Marketplace introduces private exchanges for sharing data products with invited consumers only.
Private exchanges enable data providers to share data products with a select group of invited consumers. Not checked yet Private listings appear on the invited consumers' Marketplace home page, but do not appear in the public marketplace. Not checked yet
2 details in Everything
Databricks Marketplace: Consumers can uninstall data products using the UI
Jun 5, 2023
· release notes
Databricks Marketplace adds UI capability for consumers to uninstall data products via the My requests interface.
Databricks Connect V2 is GA for Python
Jun 2, 2023
· release notes
Databricks Connect V2 GA for Python with Structured Streaming, Delta, Pandas, and OAuth support
Adds general availability of Databricks Connect V2 for Python starting with Databricks Runtime 13.1 Not checked yet Enables Structured Streaming capabilities in Python via Databricks Connect Not checked yet Supports all Delta features through the Python connector Not checked yet Adds Pandas on Spark support for Python workloads Not checked yet Introduces experimental OAuth authentication via the included Databricks SDK for Python Not checked yet
5 details in Everything
Databricks Terraform provider updated to version 1.18.0
Jun 2, 2023
· release notes
Databricks Terraform provider 1.18.0 adds databricks_volume resource and databricks_model_serving permissions support.
View data from notebooks, SQL editor, and Catalog Explorer with the unified schema browser (Public Preview)
Jun 1, 2023
· release notes
New unified schema browser in Public Preview for viewing Unity Catalog data from notebooks, SQL editor, and Catalog Explorer.
Adds unified schema browser to view all data in the Unity Catalog metastore directly within notebooks and the SQL editor. Not checked yet Enables filtering the unified schema browser view using For you (active data and favorites) or by selecting specific catalog entries. Not checked yet
2 details in Everything
Databricks Marketplace adds eight new data provider listings including Tego Cyber threat intel
Adds Tego Cyber as a Databricks Marketplace provider for cybersecurity and threat intelligence data. Not checked yet Adds Vaisala as a provider for weather and environment data. Not checked yet Adds IQVIA as a provider for healthcare and life sciences data. Not checked yet Adds Morningstar as a provider for investment offering data and research insights. Not checked yet Adds Sucali as a provider for real estate performance metrics for hedge funds and investors. Not checked yet
+3 more
show less
Adds Veritas Data Research as a provider for a comprehensive index of US mortality data. Not checked yet Adds Collectors Data Store as a provider for sports and trading card data. Not checked yet Adds Ribbon Health as a provider for healthcare provider data. Not checked yet
8 details in Everything
Run Databricks notebooks on SQL warehouses (Public Preview)
May 31, 2023
· release notes
Public Preview allows running Databricks notebooks on SQL warehouses for optimized SQL analytics.
Create or modify table UI supports Avro, Parquet, and text file uploads
May 29, 2023
· release notes
UI update allowing direct file uploads of Avro, Parquet, and text formats.
The Create or modify table UI now supports uploading .avro files. Not checked yet The Create or modify table UI now supports uploading .parquet files. Not checked yet The Create or modify table UI now supports uploading .txt files. Not checked yet
3 details in Everything
Use the add data UI to load data using Unity Catalog external locations (Public Preview)
May 29, 2023
· release notes
Add Data UI now loads from Unity Catalog external locations Public preview
Prevent Enter key from accepting autocomplete suggestion
May 26, 2023
· release notes
Adds a UI option to prevent Enter from accepting autocomplete in the SQL editor
Adds Disable enter to accept autocomplete toggle via the SQL editor kebab menu Not checked yet
1 detail in Everything
Databricks Terraform provider updated to version 1.17.0
May 22, 2023
· release notes
Databricks Terraform provider v1.17.0 adds cluster policy impact info and drops 32-bit binaries
Adds documentation on the impact of using a cluster policy in the databricks_cluster resource to help users understand constraints before applying infrastructure changes Not checked yet Removes support for releasing 32-bit binaries, meaning the provider will no longer produce or support 32-bit distribution artifacts Not checked yet
2 details in Everything
Upload data UI supports new column data types
May 19, 2023
· release notes
Upload data UI now supports STRUCT, ARRAY, and DECIMAL(P,S) column data types
Configure your workspace to use IMDS v2 (GA)
May 18, 2023
· release notes
Databricks GA allows enforcing IMDSv2 for workspace metadata access via a new admin setting.
enforce_imds_v2 workspace admin setting is now generally available to enforce the use of AWS IMDS version 2. Not checked yet Workspaces created after October 1, 2022 enable enforce_imds_v2 by default. Not checked yet
2 details in Everything
M7g and R7g Graviton3 instances are now supported on Databricks
May 17, 2023
· release notes
M7g and R7g Graviton3 instances now supported on Databricks clusters.
Adds M7g and R7g Graviton3 instance types to cluster creation configuration. Not checked yet
1 detail in Everything
All users can connect to Fivetran using Partner Connect (Public Preview)
May 17, 2023
· release notes
All users can now connect to Fivetran via Partner Connect in public preview, no longer restricted to workspace admins.
Adds support for all workspace users to connect to Fivetran using Partner Connect Public preview Not checked yet Removes the requirement for the workspace admin user role to establish Fivetran connections via Partner Connect Not checked yet
2 details in Everything
Bind Unity Catalog catalogs to specific workspaces
May 17, 2023
· release notes
New workspace-catalog binding feature limits Unity Catalog access to specific workspaces.
Adds the ability to bind Unity Catalog catalogs to specific workspaces for isolation. Not checked yet
1 detail in Everything
Authentication using OAuth tokens for service principals (Public Preview)
May 17, 2023
· release notes
New capability to authenticate Databricks REST APIs REST APIs A convention for building HTTP-based interfaces where resources are addressed by URL and manipulated with standard verbs like GET, POST, and DELETE, giving cyber tools a predictable way to integrate with external services. via OAuth tokens for service principals.
Schedule automatic cluster update (Public Preview)
May 16, 2023
· release notes
Databricks adds automatic cluster update scheduling in public preview for Enterprise workspaces
Adds automatic_cluster_update configuration allowing admins to schedule compute restarts for latest images and security updates Not checked yet
1 detail in Everything
Databricks JDBC JDBC A Java standard API for connecting applications to relational databases, maintained as part of the Java SE platform, letting tools query any compliant database through a single consistent interface. driver 2.6.33 adds Primary Key and Foreign Key APIs for Unity Catalog enabled servers.
Adds PrimaryKeys and ForeignKeys support in the JDBC driver when connecting to a server that has Unity Catalog enabled. Not checked yet
1 detail in Everything
Databricks Terraform provider updated to version 1.16.0
May 12, 2023
· release notes
Databricks Terraform provider v1.16 introduces workspace file management and job notification configuration via new IaC IaC Infrastructure as Code: the practice of defining and provisioning compute infrastructure through machine-readable config files rather than manual processes, giving cyber tools repeatable, version-controlled environment setup. resources.
Adds databricks_workspace_file resource for managing workspace files Not checked yet Adds notification_settings block to the databricks_job resource Not checked yet
2 details in Everything
Compliance security profile now supports more EC2 instance types
May 11, 2023
· release notes
Databricks Compliance Security Profile now supports additional AWS Nitro EC2 EC2 Amazon's cloud virtual-machine service, letting tools run on-demand servers without managing physical hardware. instance types
Databricks Terraform provider updated to version 1.15.0
May 5, 2023
· release notes
Databricks Terraform provider v1.15.0 adds databricks_sql_table resource and workspace init script support
databricks_sql_table: New experimental resource to manage SQL table definitions in Databricks via Terraform Not checked yet
1 detail in Everything
Account nicknames now available in the account console
May 5, 2023
· release notes
Account admins can now set nicknames for their Databricks accounts via the account console.
Share notebooks using Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage.
Adds Databricks-to-Databricks notebook sharing via Delta Sharing for cross-metastore and cross-account collaboration Not checked yet
1 detail in Everything
Databricks introduces unified navigation in public preview to reduce clicks and context switches.
Adds unified navigation experience to the Workspace UI Not checked yet Reduces clicks and context switches across product areas via the new unified navigation layout Not checked yet
2 details in Everything
AWS fleet instance types now available
May 1, 2023
· release notes
Databricks now supports AWS fleet instance types for clusters and pools to optimize capacity.
Adds support for fleet instance types when creating a cluster or pool, mapping to multiple comparable AWS instance types for best spot/on-demand availability. Not checked yet
1 detail in Everything
Allows configuring the Python formatter in Databricks Repos via a Black specification.
Cluster-scoped init scripts can now be stored in workspace files
Apr 24, 2023
· release notes
Cluster-scoped init scripts can now be stored in workspace files.
Control access to the account console by IP address ranges (Public Preview)
Apr 17, 2023
· release notes
Databricks adds IP access lists and audit events to control account console access
Workspaces with security profile or ESM include audit logs rows for system and monitor logs
Apr 14, 2023
· release notes
Workspaces with security profile or ESM now include audit logs for system and monitor logs.
Adds system log rows to the workspace audit log for environments using the compliance security profile or enhanced security monitoring. Not checked yet
1 detail in Everything
New cluster metrics UI for Databricks Runtime 13.0+ clusters replaces Ganglia with live and historical views
Adds a new cluster metrics UI accessible from clusters running Databricks Runtime 13.0 or above, replacing the previous Ganglia monitoring tool. Not checked yet
1 detail in Everything
Databricks Terraform provider updated to version 1.14.3
Apr 14, 2023
· release notes
Databricks Terraform provider v1.14.3 adds exporter support for sql_alert and notifications for pipelines
Adds exporter support for databricks_sql_alert to generate Terraform configuration for existing SQL alerts. Not checked yet Adds notification support in databricks_pipeline to configure alert triggers for pipeline events. Not checked yet
2 details in Everything
Audit log entries for changed admin settings for workspaces and accounts
Apr 14, 2023
· release notes
Databricks adds audit log entries for workspace and account admin setting changes via new deleteSetting and setSetting events.
Adds deleteSetting event to account-level audit logs for changed account admin settings Not checked yet Adds setSetting event to account-level audit logs for changed account admin settings Not checked yet Adds deleteSetting event to workspace-level audit logs for changed workspace admin settings Not checked yet Adds setSetting event to workspace-level audit logs for changed workspace admin settings Not checked yet
4 details in Everything
Create or modify table from file upload page supports JSON file uploads
Apr 3, 2023
· release notes
Table creation UI now accepts JSON files for direct data ingestion.
Adds JSON file upload support to the 'Create or modify table from file upload' page Not checked yet
1 detail in Everything
Databricks Terraform provider updated to version 1.14.0
Mar 31, 2023
· release notes
Databricks Terraform provider v1.14.0 adds run_if conditions and file arrival triggers to databricks_job.
Adds run_if condition support to the databricks_job resource Not checked yet Adds file arrival trigger support to the databricks_job resource Not checked yet
2 details in Everything
Distributed training with TorchDistributor
Mar 27, 2023
· release notes
Databricks adds distributed PyTorch training via TorchDistributor on Databricks Runtime 13.0 ML
Databricks file editor now supports direct execution of Python, SQL, R, and Scala files
Databricks SQL Serverless is GA
General availability of Databricks SQL Serverless in supported regions for instant compute and cost optimization. Not checked yet
1 detail in Everything
Changes for workspace settings for serverless SQL warehouses
Mar 22, 2023
· release notes
Serverless SQL warehouses are enabled by default for accounts with serverless compute; old toggle removed
Workspace-level serverless SQL warehouse setting is now enabled by default for accounts with serverless compute Not checked yet Removal of the workspace-level UI toggle to enable or disable serverless SQL warehouses Not checked yet
2 details in Everything
└──▷ BREAKING ON UPGRADE
! The workspace-level setting to enable or disable serverless SQL warehouses was removed from the workspace settings UI.
.ipynb (Jupyter) notebook support in Repos (preview)
Mar 22, 2023
· release notes
Databricks Repos adds preview support for .ipynb Jupyter notebooks with commit output control and diff views
Adds Allow committing .ipynb notebook output admin setting to control whether notebook outputs are persisted during commits Not checked yet Enables cloning, editing, and pushing .ipynb files directly within Databricks Repos Not checked yet Introduces Code diff and Raw diff views for inspecting cell-level or JSON metadata changes in .ipynb notebooks Not checked yet
3 details in Everything
Databricks Terraform provider updated to version 1.12.0
Mar 9, 2023
· release notes
Adds databricks_model_serving resource to manage model serving endpoints
Adds the databricks_model_serving resource for managing ML model serving deployments. Not checked yet Deprecates the schedule block within the databricks_sql_query resource. Not checked yet
2 details in Everything
Databricks introduces Model Serving (formerly Serverless Real-Time Inference) as a generally available service for low-latency model deployment and automatic scaling.
Model Serving is now generally available as a highly available, low-latency service for deploying machine learning models. Not checked yet
1 detail in Everything
View frequent queries and users of a table using the Insights tab
Mar 6, 2023
· release notes
Insights tab added to Catalog Explorer in Unity Catalog to view frequent queries and users of a table.
SAML single sign-on (SSO) in the account console is generally available
Feb 28, 2023
· release notes
SAML SAML A standard for passing signed identity assertions between a sign-in provider and an application. It lets a user sign in through their organization's identity provider. and OIDC OIDC OpenID Connect, a sign-in protocol built on OAuth 2.0. An application uses identity information from a trusted provider instead of maintaining a separate password for each user. account console SSO SSO Single sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. is generally available
Enables SAML single sign-on single sign-on Single sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. (SSO) in the account console for all account admins and users Not checked yet
1 detail in Everything
└──▷ BREAKING ON UPGRADE
! All account admins and users are now required to use SSO to log in to the account console (only the account owner can use username/password)
Ray on Databricks public preview allows running Ray applications via the Ray on Spark API with Databricks Runtime 12.0+
Adds Ray on Spark API to create Ray clusters and run Ray applications starting with Databricks Runtime 12.0 Not checked yet
1 detail in Everything
Databricks Jobs now supports running continuous jobs
Feb 22, 2023
· release notes
Continuous trigger type added to Databricks Jobs for always-on job runs
Adds continuous trigger type to enable always-active job runs Not checked yet
1 detail in Everything
Improvements to the MLflow experiment UI
Feb 17, 2023
· release notes
MLflow experiment UI adds configurable chart view, revamped parallel coordinates for tuning, and enhanced table search/filtering.
Adds a configurable chart view in the MLflow experiment UI to visualize model performance insights. Not checked yet Introduces a revamped parallel coordinates experience for hyperparameter tuning within the experiment UI. Not checked yet Enhances the streamlined table view with improvements for search and filtering of training runs. Not checked yet
3 details in Everything
Databricks extension for Visual Studio Code (Public Preview)
Feb 14, 2023
· release notes
Databricks VS Code extension Public preview enables remote cluster execution with IDE authoring
Adds the Databricks extension for Visual Studio Code to connect clusters and run code remotely from within the IDE. Not checked yet Enables source control, modularized code, refactoring, and unit testing workflows directly in Visual Studio Code. Not checked yet
2 details in Everything
Serverless Real-Time Inference Public Preview now available to all customers
Feb 9, 2023
· release notes
Serverless Real-Time Inference moves to public preview for all customers.
Databricks Terraform provider updated to version 1.9.2
Feb 3, 2023
· release notes
Databricks Terraform provider v1.9.2 adds node type selector and file library type.
Adds databricks_node_type selector to allow explicit control over cluster node types in pipeline configurations Not checked yet Adds new file library type for databricks_pipeline to support direct file-based dependency management Not checked yet
2 details in Everything
Audit logs include entries for OAuth SSO authentication to the account console (Public Preview)
Jan 31, 2023
· release notes
Databricks audit logs now capture OAuth SSO SSO Single sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. authentication events for the account console.
Easier creation and editing of Databricks jobs in the UI
Jan 30, 2023
· release notes
Databricks introduces a unified job view in the UI to simplify creation and editing of job tasks.
Adds a unified view in the Jobs UI that displays the full job details while adding or modifying tasks Not checked yet
1 detail in Everything
REST API Reference is now available for browsing API documentation
Jan 27, 2023
· release notes
Databricks releases a new REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. Reference documentation experience in beta.
Databricks Terraform provider updated to version 1.9.1
Jan 25, 2023
· release notes
Adds max_clusters_per_user to cluster policies, cluster_mount_infos to clusters, and parent attributes to SQL objects in v1.9.1
Adds max_clusters_per_user attribute to databricks_cluster_policy resource for per-user cluster limits Not checked yet Adds cluster_mount_infos argument to databricks_cluster resource to configure mount information directly on the compute instance Not checked yet Adds parent attribute to databricks_sql_query and databricks_sql_dashboard resources to manage logical grouping in the workspace Not checked yet
3 details in Everything
Account users can update email preferences in the account console
Jan 23, 2023
· release notes
Account users can now update instructional product and feature email preferences in the account console.
Databricks Marketplace supports volume sharing
Jan 18, 2023
· release notes
Databricks Marketplace now shares volumes as non-tabular data products in Unity Catalog-enabled workspaces.
Databricks Terraform provider updated to version 1.9.0
Jan 17, 2023
· release notes
Databricks Terraform provider 1.9.0 adds databricks_mws_credentials data source
Databricks Terraform provider updated to version 1.8.0
Jan 10, 2023
· release notes
Databricks Terraform provider 1.8.0 adds new resources for instance pools, Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. providers, and permission assignments
Adds databricks_instance_pool resource to manage compute instance pools declaratively Not checked yet Adds databricks_provider resource specifically for configuring Delta Sharing integrations Not checked yet Adds databricks_permission_assignment resource to handle granular permission assignments Not checked yet
3 details in Everything
Databricks SQL Driver for Go is Generally Available
Dec 21, 2022
· release notes
Databricks SQL Driver for Go v1.0.0 GA adds context cancellation, logging, and NewConnector
dbsql.NewConnector() added to initialize database connections Not checked yet Client-side logging allows setting logger verbosity and query/connection identifiers Not checked yet maxRows omission now defaults to 100,000 with direct results enabled Not checked yet Context cancellation and timeout support cancels queries on the server when context is cancelled or timed out Not checked yet Support for setting initial namespace and session parameters including timezone Not checked yet
5 details in Everything
Prevent concurrent workspace updates
Dec 15, 2022
· release notes
Databricks now rejects concurrent account console or API workspace updates to prevent data corruption.
Enables admins to avoid conflicting workspace updates and data corruption errors by waiting for the current operation to finish before retrying. Not checked yet
1 detail in Everything
Databricks Terraform provider updated to version 1.7.0
Dec 14, 2022
· release notes
Databricks Terraform provider v1.7.0 adds databricks_cluster_policy data source and exports for service principal and SQL global config resources.
Adds databricks_cluster_policy data source to look up existing cluster policies Not checked yet Enables exporting of the databricks_service_principal resource state Not checked yet Enables exporting of the databricks_sql_global_config resource state Not checked yet
3 details in Everything
Enhanced notifications for your Databricks jobs (Public Preview)
Dec 7, 2022
· release notes
Databricks jobs support webhook and Slack notification channels in public preview
Adds webhook notification support for Databricks job run events Not checked yet Adds native Slack notification support for Databricks job run events Not checked yet
2 details in Everything
Use SQL to specify schema- and catalog-level storage locations for Unity Catalog managed tables
Dec 6, 2022
· release notes
Adds MANAGED_LOCATION to CREATE CATALOG and CREATE SCHEMA for Unity Catalog storage control
Capturing lineage data with Unity Catalog is now generally available
Dec 6, 2022
· release notes
Unity Catalog now generally available for capturing and viewing lineage data
Makes Unity Catalog lineage capture GA, enabling users to view data lineage via the Lineage in Unity Catalog feature. Not checked yet
1 detail in Everything
Billable usage graphs can now aggregate by individual tags
Dec 6, 2022
· release notes
Billable usage graphs now aggregate by individual tags, with tags included in CSV reports.
billable usage graphs in the account console can now aggregate usage by individual tags Not checked yet billable usage CSV reports include default and custom tags Not checked yet
2 details in Everything
Databricks ODBC ODBC A standard API, maintained by Microsoft, for connecting applications to relational databases regardless of vendor. Cyber tools use it to query databases without writing engine-specific code. 2.6.29 adds Unity catalog primary/foreign key support for schema discovery
Improved serverless SQL warehouse support for customer-managed keys
Dec 1, 2022
· release notes
Serverless SQL warehouses now support customer-managed keys for managed services and root S3 bucket
Upload data UI can now be disabled via admin settings
Nov 17, 2022
· release notes
Admins can now disable the Upload Data UI for all personas including legacy DBFS.
Adds an admin setting to disable the new Upload data UI and the legacy DBFS file upload UI Not checked yet Extends the upload data UI disablement to Data Science & Engineering , Databricks Mosaic AI , and Databricks SQL personas Not checked yet
2 details in Everything
Work with large repositories with Sparse Checkout
Nov 14, 2022
· release notes
Sparse Checkout support in Repos lets you clone a subset of directories from large remote repositories
Adds Sparse Checkout capability to Repos for cloning subsets of remote repositories. Not checked yet Allows working with monorepos or repositories exceeding standard size limits by fetching only specified directory paths. Not checked yet
2 details in Everything
Databricks Terraform provider updated to version 1.6.5
Nov 11, 2022
· release notes
Databricks Terraform provider 1.6.5 adds query_plan to databricks_sql_visualization and name filter to databricks_job data source.
Adds query_plan parameter to the databricks_sql_visualization resource to control execution plan visualization. Not checked yet
1 detail in Everything
Create or modify table from file upload page now supports overwrite
Nov 8, 2022
· release notes
File upload page now supports overwriting existing managed Delta tables
Create or modify table from file upload page now supports multiple files
Nov 8, 2022
· release notes
The 'Create or modify table from file upload' page now supports loading up to 10 files simultaneously.
Connect to Fivetran from the add data UI
Nov 3, 2022
· release notes
Adds Fivetran connection option in the add data UI via Partner Connect.
Enables connecting to Fivetran directly from the add data UI using the same steps as the Partner Connect portal. Not checked yet
1 detail in Everything
Databricks Terraform provider updated to version 1.6.2
Nov 2, 2022
· release notes
Databricks Terraform provider 1.6.2 adds runtime_engine to clusters and auto-detects AWS CodeCommit in repos.
Adds runtime_engine attribute to the databricks_cluster resource Not checked yet Adds auto-detection of AWS CodeCommit URLs for the databricks_repo resource Not checked yet
2 details in Everything
Search for tables in Unity Catalog is GA
Nov 1, 2022
· release notes
New GA feature enabling search for tables, comments, columns, and column comments within Unity Catalog via the Databricks workspace UI.
Databricks SQL Driver for Node.js is Generally Available
Oct 27, 2022
· release notes
Databricks SQL Driver for Node.js GA (v1.0.0) with direct results and async support
The DBSQLClient constructor now accepts an optional logger parameter for client-side logging. Not checked yet The runAsync option is now available for all operations that support it, enabling asynchronous execution. Not checked yet Direct results are enabled by default for all supported IOperation methods and are controlled via the maxRows parameter (default 100,000; set to null to disable). Not checked yet
3 details in Everything
Deploy models for streaming inference with DLT notebooks
Oct 27, 2022
· release notes
Databricks introduces automatic notebook generation for deploying models to Delta Live Tables (DLT) for streaming inference.
Adds capability to automatically generate a Databricks notebook configured for streaming inference via DLT when deploying a model. Not checked yet
1 detail in Everything
Enforce user isolation cluster types on a workspace
Oct 24, 2022
· release notes
New workspace setting Enforce User Isolation prevents creation of shared clusters.
Improvements to AWS PrivateLink support for updating workspaces
Oct 17, 2022
· release notes
Enable AWS PrivateLink AWS PrivateLink An AWS networking feature that exposes a service over a private VPC endpoint rather than the public internet, letting tools move data between accounts or to AWS services without traffic ever leaving Amazon's backbone. workspace updates via admin console or Account API
Adds ability to upgrade a workspace's PrivateLink PrivateLink An AWS networking feature that exposes a service over a private endpoint inside a VPC, so traffic never traverses the public internet, giving cyber tools a way to integrate with cloud services without opening public routes. support for front-end, back-end, or both types of connectivity via the admin console or Account API Not checked yet Adds ability to add a private access settings object via the admin console UI or Account API Not checked yet Adds ability to add or update a workspace's registered VPC VPC A logically isolated virtual network inside a public cloud provider's infrastructure, letting tools deploy resources with private addressing, controlled routing, and firewall rules without managing physical hardware. endpoints by creating and updating a network configuration object via the admin console UI or Account API Not checked yet
3 details in Everything
Update a failed workspace with Databricks-managed VPC to use a customer-managed VPC
Oct 17, 2022
· release notes
Now supports converting failed Databricks workspaces with Databricks-managed VPCs VPCs Virtual Private Clouds — isolated, software-defined network environments inside a public cloud provider such as AWS, Azure, or GCP — give cyber tools a controlled network boundary with configurable routing, subnets, and access rules. to customer-managed VPCs.
Allows conversion of a workspace in the failed state from Databricks-managed VPC VPC A logically isolated virtual network inside a public cloud provider's infrastructure, letting tools deploy resources with private addressing, controlled routing, and firewall rules without managing physical hardware. to a customer-managed VPC Not checked yet
1 detail in Everything
AWS PrivateLink support is now generally available
Oct 17, 2022
· release notes
AWS PrivateLink AWS PrivateLink An AWS networking feature that exposes a service over a private VPC endpoint rather than the public internet, letting tools move data between accounts or to AWS services without traffic ever leaving Amazon's backbone. for Databricks is generally available, enabling private connectivity from VPCs VPCs Virtual Private Clouds — isolated, software-defined network environments inside a public cloud provider such as AWS, Azure, or GCP — give cyber tools a controlled network boundary with configurable routing, subnets, and access rules. and on-premises networks.
Personal Compute cluster policy is available by default to all users
Oct 11, 2022
· release notes
Personal Compute cluster policy is now available by default to all users
Add data UI provides a central UI for loading data to Databricks
Oct 11, 2022
· release notes
Add data UI provides a central UI for loading data to Databricks
Partner Connect supports connecting to Hevo Data
Oct 11, 2022
· release notes
Databricks Partner Connect now supports new integrations with Hevo Data
Adds Hevo Data as a supported integration target in Partner Connect Not checked yet
1 detail in Everything
SQL persona integrated with new search experience
Oct 6, 2022
· release notes
Databricks adds global search for queries, dashboards, and alerts directly in the SQL persona
Serverless SQL warehouses are available in regions eu-central-1 and us-east-2
Oct 4, 2022
· release notes
Serverless SQL warehouses now available in eu-central-1 and us-east-2
Protect and control access to some types of encrypted data with customer-managed keys (GA)
Sep 23, 2022
· release notes
Customer-managed keys (CMK) for encryption in Databricks managed services and workspace storage reach GA.
Orchestrate Databricks SQL tasks in your Databricks workflows (Public Preview)
Sep 12, 2022
· release notes
Databricks adds sql task type to workflows for public preview, enabling SQL query execution, dashboard refreshes, and alert refreshes.
sql task type in Databricks Workflows allows running SQL queries, refreshing SQL dashboards, and refreshing SQL alerts. Not checked yet
1 detail in Everything
Capture and view lineage data with Unity Catalog (Public Preview)
Sep 9, 2022
· release notes
New Unity Catalog public preview adds near-real-time lineage capture and viewing for queries across notebooks, jobs, and dashboards.
Adds near real-time lineage visualization in Catalog Explorer for queries running in a Databricks cluster or SQL warehouse Not checked yet Adds lineage retrieval via the Databricks REST API Not checked yet
2 details in Everything
Search for tables using Catalog Explorer (Public Preview)
Sep 8, 2022
· release notes
Adds table and column search to Catalog Explorer in Unity Catalog
Enables searching for tables by name, comment, column name, and column comment in Catalog Explorer Not checked yet
1 detail in Everything
New data trasformation card on workspace landing pages
Sep 8, 2022
· release notes
New Transform data card on Data Science & Engineering and SQL workspace landing pages
Support for AWS Graviton instances is GA
Sep 1, 2022
· release notes
Databricks GA support for AWS Graviton2-based Amazon EC2 EC2 Amazon's cloud virtual-machine service, letting tools run on-demand servers without managing physical hardware. instances
Databricks ODBC ODBC A standard API, maintained by Microsoft, for connecting applications to relational databases regardless of vendor. Cyber tools use it to query databases without writing engine-specific code. driver 2.6.26 adds asynchronous query cancellation on HTTP connections.
Adds the ability to asynchronously cancel queries on HTTP connections upon API request Not checked yet
1 detail in Everything
Databricks Feature Store client now available on PyPI
Aug 26, 2022
· release notes
Databricks Feature Store client now available on PyPI for local IDE support and unit testing.
└──▷ USE IT
Install the client locally in a notebook cell for IDE autocompletion and unit testing support.
shell
copy
%pip install databricks-feature-store
Adds the databricks-feature-store package to PyPI, enabling installation via %pip install databricks-feature-store. Not checked yet
1 detail in Everything
Users can be members of multiple Databricks accounts
Aug 25, 2022
· release notes
Users can now be members of multiple Databricks accounts
A user can now be a member of more than one Databricks account Not checked yet
1 detail in Everything
Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. is GA with managed Databricks-to-Databricks sharing, new UI/SQL/REST APIs REST APIs A convention for building HTTP-based interfaces where resources are addressed by URL and manipulated with standard verbs like GET, POST, and DELETE, giving cyber tools a predictable way to integrate with external services. , IP/region restrictions, and delegation.
Adds Databricks Runtime 11.1 as the first release supporting generally available Delta Sharing Not checked yet Enables fully managed Databricks-to-Databricks Delta Sharing without exchanging tokens when sharing data within the same account Not checked yet Adds a new UI to create and manage providers, recipients, and shares Not checked yet Provides SQL and REST APIs for creating and managing providers, recipients, and shares alongside full CLI and Terraform support Not checked yet Supports sharing incremental versions or querying changes via Change Data Feeds Not checked yet
+3 more
show less
Introduces IP access lists and region restrictions to restrict recipient access to downloading credential files or querying data Not checked yet Enables default data sharing within the same Databricks account using Delta Sharing Not checked yet Allows delegation of Delta Sharing management to non-admin users to enforce separation of duties Not checked yet
8 details in Everything
Databricks GA: account-level identity federation for users, service principals, and groups
Partner Connect supports connecting to Stardog
Aug 24, 2022
· release notes
Partner Connect now supports connecting Databricks workspaces to Stardog.
Serverless Real-Time Inference in Public Preview
Aug 16, 2022
· release notes
Serverless Real-Time Inference now in public preview
Adds Serverless Real-Time Inference to process MLflow models as REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. endpoints using managed compute. Not checked yet Workspace admins can enable the Serverless Real-Time Inference feature for their workspace. Not checked yet
2 details in Everything
Serverless SQL warehouses improvements
Aug 15, 2022
· release notes
Serverless SQL warehouses now use private worker nodes and direct S3 gateway endpoints for same-region access.
Serverless SQL warehouse worker nodes are private, removing public IP addresses from compute instances Not checked yet Serverless SQL warehouses in the same AWS region as the workspace now use AWS gateway endpoints for direct Amazon S3 access Not checked yet
2 details in Everything
Share VPC endpoints among Databricks accounts
Aug 15, 2022
· release notes
Databricks now allows workspaces in different accounts to reuse the same customer-managed AWS VPC VPC A logically isolated virtual network inside a public cloud provider's infrastructure, letting tools deploy resources with private addressing, controlled routing, and firewall rules without managing physical hardware. endpoints, simplifying network architecture.
Notification upon notebook completion
Aug 15, 2022
· release notes
Adds browser notifications that appear when a running notebook completes while the user navigates away.
Notification upon notebook completion now appears in your browser when a running notebook finishes after you navigate away. Not checked yet
1 detail in Everything
AWS PrivateLink private access level ANY is deprecated
Aug 15, 2022
· release notes
AWS PrivateLink AWS PrivateLink An AWS networking feature that exposes a service over a private VPC endpoint rather than the public internet, letting tools move data between accounts or to AWS services without traffic ever leaving Amazon's backbone. ANY private access level deprecated; new default is ACCOUNT
The Account API can now be used to update private access settings fields without changing the private access level. Not checked yet
1 detail in Everything
└──▷ BREAKING ON UPGRADE
! You can no longer create a new private access settings with the ANY private access level.! You can no longer update an existing private access settings to change it to the ANY private access level.
Workspace search gains GA status with enhanced capabilities including searching within notebook content and filtering results by object type.
Improved workspace search is now generally available Not checked yet
1 detail in Everything
Improvements to AWS PrivateLink connectivity
Aug 9, 2022
· release notes
Databricks now allows users to enable PrivateLink PrivateLink An AWS networking feature that exposes a service over a private endpoint inside a VPC, so traffic never traverses the public internet, giving cyber tools a way to integrate with cloud services without opening public routes. and configure private DNS directly via the account console during workspace creation.
Enables self-service configuration of AWS PrivateLink AWS PrivateLink An AWS networking feature that exposes a service over a private VPC endpoint rather than the public internet, letting tools move data between accounts or to AWS services without traffic ever leaving Amazon's backbone. connectivity through the Databricks account console without requiring support contact. Not checked yet Adds the ability to enable private DNS for new AWS VPC VPC A logically isolated virtual network inside a public cloud provider's infrastructure, letting tools deploy resources with private addressing, controlled routing, and firewall rules without managing physical hardware. endpoints during the workspace creation workflow. Not checked yet
2 details in Everything
Compliance controls FedRAMP Moderate, PCI-DSS, and HIPAA (GA)
Aug 4, 2022
· release notes
FedRAMP Moderate FedRAMP Moderate A US government authorization tier requiring cloud services to meet a defined set of NIST 800-53 controls before federal agencies can use them, giving cyber tools a pre-vetted compliance baseline for government contracts. , PCI-DSS PCI-DSS The Payment Card Industry Data Security Standard, security requirements for protecting payment account data. It covers organizations that store, process or transmit that data and relevant service providers. , and HIPAA HIPAA A US federal law enacted in 1996 that sets privacy and security requirements for protected health information, giving cyber tools a compliance baseline to enforce or audit against. compliance controls are now generally available for Databricks.
Add security controls with the compliance security profile (GA)
Aug 4, 2022
· release notes
GA release of the compliance security profile for enhanced security controls.
Job execution now waits for cluster libraries to finish installing
Aug 1, 2022
· release notes
Updates to job startup behavior regarding library installation timing.
Photon native vectorized query engine is now generally available in Databricks Runtime 11.1
Databricks SQL Serverless supports instance profiles whose name does not match its associated role
Jul 14, 2022
· release notes
Databricks SQL Serverless now supports explicit IAM IAM Identity and Access Management — a framework of policies and services for controlling who can authenticate to systems and what resources they are authorized to use, letting cyber tools enforce least-privilege and audit access. role ARNs in instance profiles via API and UI
Adds support for editing instance profiles to add or change an affiliated IAM role ARN, allowing the profile name and role name to differ. Not checked yet Expands Instance Profile API to allow specifying an explicit IAM role ARN when creating an instance profile. Not checked yet
2 details in Everything
Verbose audit logs now record when Databricks SQL queries are run
Jul 14, 2022
· release notes
Verbose audit logs now capture Databricks SQL query submission and completion events
Adds verbose audit logs workspace configuration setting to record when Databricks SQL queries are run. Not checked yet Audit logs now generate separate records for query submission and query completion (including cancellation). Not checked yet
2 details in Everything
Databricks ODBC ODBC A standard API, maintained by Microsoft, for connecting applications to relational databases regardless of vendor. Cyber tools use it to query databases without writing engine-specific code. driver 2.6.25 adds configuration options to mitigate straggling result file downloads
Databricks Runtime 11.1 (Beta)
Prerelease
Jul 8, 2022
· release notes
Databricks Runtime 11.1 Beta releases are now available.
ALTER TABLE permission changes for Unity Catalog
Jun 30, 2022
· release notes
Unity Catalog now allows MODIFY permission to update table metadata via ALTER TABLE without OWNERSHIP
Reduces ALTER TABLE privilege requirement from OWNERSHIP to MODIFY for metadata operations (comments, properties, columns) in Unity Catalog Not checked yet Requires OWNERSHIP only for changing owner, granting permissions, renaming tables, or modifying view definitions Not checked yet
2 details in Everything
Databricks ODBC ODBC A standard API, maintained by Microsoft, for connecting applications to relational databases regardless of vendor. Cyber tools use it to query databases without writing engine-specific code. driver 2.6.24 adds CTAS query translation, SQL_ATTR_QUERY_TIMEOUT override, and OpenSSL update.
Adds support to configure query translation to CTAS syntax Not checked yet Allows users to override SQL_ATTR_QUERY_TIMEOUT in the connector Not checked yet
2 details in Everything
Databricks Terraform provider is now GA
Jun 22, 2022
· release notes
Databricks Terraform provider reaches general availability with Unity Catalog support
Databricks Terraform provider is now generally available for production use. Not checked yet Adds support for deploying Unity Catalog governance features via the Terraform provider. Not checked yet
2 details in Everything
Enhanced security monitoring (Public Preview)
Jun 21, 2022
· release notes
Enhanced security monitoring Public preview on Databricks E2 adds hardened disk image and additional logging agents.
HIPAA compliance controls for E2 (Public Preview)
Jun 21, 2022
· release notes
Public preview of HIPAA HIPAA A US federal law enacted in 1996 that sets privacy and security requirements for protected health information, giving cyber tools a compliance baseline to enforce or audit against. compliance controls on Databricks E2
Audit logs can now record when a notebook command is run
Jun 14, 2022
· release notes
Audit logs now record notebook command execution via the verbose audit logs workspace configuration
Communicate between tasks in your Databricks jobs with task values
Jun 13, 2022
· release notes
Databricks adds taskValues subutility to pass outputs between tasks in a job run
Adds the dbutils.jobs.taskValues subutility (task values) for communicating values between tasks in a Databricks job Not checked yet Enables passing the output of a machine learning model to downstream tasks within the same job run Not checked yet
2 details in Everything
Enable account switching in the Databricks UI
Jun 8, 2022
· release notes
Databricks UI adds an account switcher for users belonging to multiple accounts.
Adds ability to switch between Databricks accounts via the UI for users with multi-account access Not checked yet
1 detail in Everything
Copy and paste notebook cells between tabs and windows
Jun 6, 2022
· release notes
Allows copying notebook cells between different browser tabs or windows.
Enables copy and paste of cells across distinct browser tabs or windows. Not checked yet
1 detail in Everything
Additional data type support for Databricks Feature Store automatic feature lookup
Jun 6, 2022
· release notes
Databricks Feature Store auto-lookup supports DecimalType, ArrayType, MapType
Upgrade wizard makes it easier to copy databases and multiple tables to Unity Catalog (Public Preview)
May 13, 2022
· release notes
New upgrade wizard in Public Preview copies databases and tables from Hive Metastore to Unity Catalog.
Adds an upgrade wizard to copy complete schemas (databases) and multiple tables from the default Hive metastore Hive metastore A central metadata repository, part of the Apache Hive project, that stores schema and location information for distributed data sets so query engines can discover and read structured data without hardcoding paths. to the Unity Catalog metastore. Not checked yet The upgrade wizard streamlines permission assignment for the upgraded schemas and tables during migration. Not checked yet
2 details in Everything
Power BI Desktop system-wide HTTP proxy support
May 11, 2022
· release notes
Power BI connector adds automatic system-wide HTTP proxy detection for Databricks connections in Power BI Desktop.
Streamline billing and account management by signing up for Databricks using AWS Marketplace
May 10, 2022
· release notes
Enable AWS Marketplace signup for Databricks to streamline billing via Purchaser role and consolidated charges.
Enables signing up for Databricks directly through AWS Marketplace. Not checked yet Allows users with the Purchaser role to manage subscriptions via AWS. Not checked yet Consolidates Databricks charges on the AWS Billing & Cloud Management dashboard. Not checked yet
3 details in Everything
Authenticate to the account console using SAML 2.0 (Public Preview)
May 3, 2022
· release notes
SAML 2.0 SAML 2.0 A standard for passing signed identity assertions between a sign-in provider and an application. It lets a user sign in through their organization's identity provider. is now an available SSO SSO Single sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. protocol for the Databricks account console.
Feature Store now supports publishing features to AWS DynamoDB
Apr 7, 2022
· release notes
Feature Store releases capability to integrate with DynamoDB.
Databricks Feature Store is generally available
Mar 22, 2022
· release notes
Databricks Feature Store GA enables cross-org feature sharing and consistent code for training/inference.
Databricks Feature Store is now generally available for feature sharing and discovery across organizations. Not checked yet
1 detail in Everything
Test DLT preview functionality with the new channel setting (Public Preview)
Mar 14, 2022
· release notes
Adds channel setting to DLT pipelines to test Public Preview functionality
Adds channel configuration option to Delta Live Tables (DLT) pipelines for testing upcoming features in public preview. Not checked yet Enables switching pipeline behavior between the current stable release and the preview channel via the channel setting. Not checked yet
2 details in Everything
Improved error handling for DLT Python functions (Public Preview)
Mar 14, 2022
· release notes
DLT enhances reliability for Python functions by adding validation logic to table property definitions.
Adds type checking to prevent non-string values in table_properties. Not checked yet
1 detail in Everything
Unity Catalog is available in Public Preview
Mar 10, 2022
· release notes
Databricks adds Unity Catalog as a new preview feature for centralized data governance and security.
Easier scheduling for your DLT pipelines (Public Preview)
Mar 2, 2022
· release notes
DLT pipeline scheduling can now be created directly from the DLT UI in public preview
Enables creating scheduled Databricks jobs for triggered pipelines directly from the DLT UI instead of requiring the separate Jobs UI. Not checked yet
1 detail in Everything
Easily browse the history of your DLT pipeline updates (Public Preview)
Mar 2, 2022
· release notes
DLT pipeline update history now browsable in the UI via the Update history drop-down
Customer-managed VPCs are now available in ap-northeast-2
Feb 7, 2022
· release notes
Customer-managed VPCs VPCs Virtual Private Clouds — isolated, software-defined network environments inside a public cloud provider such as AWS, Azure, or GCP — give cyber tools a controlled network boundary with configurable routing, subnets, and access rules. are now available in AWS ap-northeast-2 region.
Databricks hosted MLflow models can now look up features from online stores
Feb 3, 2022
· release notes
Databricks hosted MLflow models can now automatically lookup features from online stores at serving time.
Databricks Runtime 10.3 and 10.3 ML are GA; 10.3 Photon is Public Preview
Feb 2, 2022
· release notes
Databricks Runtime 10.3 GA and 10.3 ML GA are now available; 10.3 Photon enters public preview.
Databricks Runtime 10.3 is generally available Not checked yet Databricks Runtime 10.3 ML is generally available Not checked yet
2 details in Everything
MLflow Model Registry Webhooks on Databricks (Public Preview)
Jan 31, 2022
· release notes
Databricks releases public preview of MLflow Model Registry Webhooks to enable event-driven automation in machine learning workflows.
Adds support for configuring webhooks to listen for MLflow Model Registry events on Databricks Not checked yet
1 detail in Everything
Add or rotate the customer-managed key for managed services on a running workspace
Jan 25, 2022
· release notes
Enables adding or rotating customer-managed keys (CMK) for managed services on running Databricks workspaces.
Adds the ability to add a customer-managed key (CMK) for notebook source, secrets, and Databricks SQL query history on a running workspace. Not checked yet Adds the ability to rotate (update) the customer-managed key for managed services on a running workspace without recreating the workspace. Not checked yet
2 details in Everything
Real-time view of currently running and recently completed job runs in the Databricks Jobs UI.
Improved cluster management for jobs that orchestrate multiple tasks
Jan 24, 2022
· release notes
Databricks adds cluster reuse for multi-task jobs to reduce resource usage
Enables reusing job clusters across multiple tasks within a single job run Not checked yet
1 detail in Everything
Databricks Connector for Tableau 2021.4
Dec 23, 2021
· release notes
Databricks Connector for Tableau 2021.4 adds Unity Catalog three-level namespace navigation
DLT UI adds JSON specification editor and data quality metric tooltips.
Adds direct JSON editing in the Create Pipeline dialog via a JSON button to allow users to edit the pipeline specification directly. Not checked yet
1 detail in Everything
Syntax highlighting and autocomplete for SQL commands in Python cells
Dec 6, 2021
· release notes
Adds syntax highlighting and autocomplete for SQL commands (e.g., spark.sql) in Databricks Python cells
Rename, delete, and change permissions for MLflow experiments from experiment page (Public Preview)
Dec 6, 2021
· release notes
MLflow experiments can now be renamed, deleted, and have permissions changed directly from the experiment page.
Adds ability to rename an MLflow experiment directly from its experiment page Not checked yet Adds ability to delete an MLflow experiment directly from its experiment page Not checked yet Adds ability to change permissions for an MLflow experiment directly from its experiment page Not checked yet
3 details in Everything
New data profiles in notebooks: tabular and graphic summaries of your data (Public Preview)
Dec 6, 2021
· release notes
Version 3.60 introduces public preview data profiling capabilities, allowing users to view interactive summaries of dataframes and tables via a new UI tab or programmatic utility.
Interactive visualization of data profiles directly within the notebook environment. Not checked yet
1 detail in Everything
Databricks Partner Connect reaches general availability for discovering and connecting tools to the lakehouse
Partner Connect is now generally available Not checked yet
1 detail in Everything
Delta Sharing Connector for Power BI
Nov 15, 2021
· release notes
Databricks releases the Power BI Delta Sharing Delta Sharing An open protocol, maintained by the Linux Foundation, for sharing live datasets across organizations without copying them, letting cyber tools query remote data in place over standard REST and cloud storage. connector to discover and visualize shared datasets via the open protocol.
Databricks ODBC ODBC A standard API, maintained by Microsoft, for connecting applications to relational databases regardless of vendor. Cyber tools use it to query databases without writing engine-specific code. driver 2.6.19 now propagates SQLState errors to clients
Propagates SQLState error conditions from Databricks to the client Not checked yet
1 detail in Everything
Limit the set of VPC endpoints your workspace can use for AWS PrivateLink connections (Public Preview)
Oct 19, 2021
· release notes
Databricks adds private_access_level config and Account API to limit AWS PrivateLink AWS PrivateLink An AWS networking feature that exposes a service over a private VPC endpoint rather than the public internet, letting tools move data between accounts or to AWS services without traffic ever leaving Amazon's backbone. VPC VPC A logically isolated virtual network inside a public cloud provider's infrastructure, letting tools deploy resources with private addressing, controlled routing, and firewall rules without managing physical hardware. endpoints for workspaces.
Adds the private_access_level configuration field to limit AWS PrivateLink PrivateLink An AWS networking feature that exposes a service over a private endpoint inside a VPC, so traffic never traverses the public internet, giving cyber tools a way to integrate with cloud services without opening public routes. VPC endpoints for a workspace Not checked yet Introduces an Account API operation to update an existing private access settings object for workspaces Not checked yet
2 details in Everything
Databricks Jobs Orchestration GA with API v2.1 for multi-task workflows
Jobs orchestration is now generally available, enabling definition and execution of jobs with multiple tasks. Not checked yet Jobs API version 2.1 is released, adding full support for orchestrating multiple tasks within Databricks jobs. Not checked yet
2 details in Everything
Adds Unity Catalog three-level namespace support, query cancellation, and native passthrough for Databricks SQL/Runtime 8.3+
Adds support for navigation through three-level namespaces in the Unity Catalog Not checked yet Ensures that query execution can be cancelled Not checked yet Enables native query passthrough for reduced latency on Databricks SQL and Databricks Runtime 8.3 and above Not checked yet
3 details in Everything
Repos now supports arbitrary file types
Oct 7, 2021
· release notes
Files in Repos now supports arbitrary file types beyond notebooks, enabling sync of data and config files.
Adds sync support for any file type to Files in Repos (e.g. .py, .csv, .json, .yaml), not just notebooks Not checked yet Enables viewing and editing text-encoded files directly within the Databricks Repo UI Not checked yet
2 details in Everything
Databricks ODBC ODBC A standard API, maintained by Microsoft, for connecting applications to relational databases regardless of vendor. Cyber tools use it to query databases without writing engine-specific code. driver 2.6.18 adds three-namespace Unity Catalog support
Adds three-level namespace support (catalog.schema.table) for Unity Catalog integration Not checked yet
1 detail in Everything
Customer control of workspace login by Databricks staff
Sep 24, 2021
· release notes
New account-level setting to control Databricks staff login access to workspaces
Adds account-level controls for workspace access by Databricks personnel Not checked yet
1 detail in Everything
Adds Cloud Fetch parallel data loading and latency-reducing roundtrip removal to the JDBC JDBC A Java standard API for connecting applications to relational databases, maintained as part of the Java SE platform, letting tools query any compliant database through a single consistent interface. driver.
Adds support for Cloud Fetch , a mechanism for fetching data in parallel through cloud storage to improve performance with BI tools Not checked yet Reduces client-to-Databricks Runtime latency by removing unnecessary roundtrips Not checked yet
2 details in Everything
Databricks now available in region ap-southeast-1
Sep 15, 2021
· release notes
Databricks service is now available in the AWS ap-southeast-1 region.
Streamlined management of settings for Databricks jobs
Sep 8, 2021
· release notes
New Job details side panel in Databricks jobs UI for managing common settings.
Databricks SQL Public Preview available in all workspaces
Sep 7, 2021
· release notes
Databricks SQL public preview available in all workspaces
Databricks Repos generally available with Git integration and API support for GitHub, Bitbucket, GitLab, and Azure DevOps.
Repos generally available: create or clone existing Git repositories directly within Databricks to work with notebooks in a version-controlled environment. Not checked yet Integration with external Git providers: natively supports GitHub, Bitbucket, GitLab, and Azure DevOps for repository synchronization. Not checked yet Access to Repos API: enables programmatic management of repositories and CI/CD workflow integration via dedicated APIs. Not checked yet
3 details in Everything
Serverless SQL provides instant compute, minimal management, and cost optimization for SQL queries (Public Preview)
Aug 30, 2021
· release notes
Serverless SQL warehouses preview adds instant compute to Databricks SQL with a new admin activation step
Adds serverless SQL endpoints to the platform, moving compute resources from the AWS account to the Databricks account for lower latency and cost optimization. Not checked yet
1 detail in Everything
Low-latency delivery of audit logs is generally available
Aug 17, 2021
· release notes
Low-latency audit logs GA with Account API self-service config for Premium+ plans.
Adds low-latency audit log delivery capability for accounts on the Premium plan or above, typically delivering logs within less than 15 minutes of an auditable event. Not checked yet
1 detail in Everything
Log delivery APIs now report delivery status
Aug 4, 2021
· release notes
AWS log delivery APIs now report log_delivery_status with last attempt time and failure details.
Configure Databricks for SSO with Microsoft Entra ID in your Azure tenant
Aug 3, 2021
· release notes
New feature allowing Databricks workspaces to integrate with Microsoft Entra ID Entra ID Microsoft's cloud-based identity and access management service, formerly Azure Active Directory, used by cyber tools to authenticate users and enforce access policies in Microsoft 365 and Azure environments. for SSO SSO Single sign-on, using one identity provider to sign in to multiple applications. It reduces separate passwords and centralizes account and access management. in Azure.
Reorderable and resizable widgets in notebooks
Aug 2, 2021
· release notes
Adds reorderable and resizable Databricks notebook widgets via UI
Enables users to change the order and sizes of Databricks widgets in a notebook by clicking in the widgets bar Not checked yet
1 detail in Everything
Bulk import and export notebooks in a folder as source files
Aug 2, 2021
· release notes
New capability to bulk import/export workspace folders as source ZIP archives for CI/CD.
Adds the ability to export all notebooks in a workspace folder as a single ZIP archive of source files (excluding command results). Not checked yet Enables importing a notebook source ZIP archive into another workspace or folder, recreating the original notebooks. Not checked yet
2 details in Everything
Feature freshness information available in Databricks Feature Store UI (Public Preview)
Aug 2, 2021
· release notes
Databricks Feature Store UI now displays feature freshness metadata in Public Preview
Adds last update timestamp display for feature tables in the Databricks Feature Store UI Not checked yet
1 detail in Everything
Autocomplete in SQL notebooks now uses all-caps for SQL keywords
Aug 2, 2021
· release notes
SQL notebook autocomplete now outputs SQL keywords in all-caps.
Autocomplete output for SQL keywords in notebooks is now capitalized (e.g., SELECT instead of select). Not checked yet
1 detail in Everything
Quickly define pipeline settings when you create a new DLT pipeline
Aug 2, 2021
· release notes
Adds inline pipeline settings definition in the Create Pipeline UI dialog for DLT
Allows defining pipeline settings directly in the Create Pipeline dialog in the DLT user-interface Not checked yet
1 detail in Everything
Databricks now allows displaying up to 10,000 result rows in notebooks, raising the previous limit of 1,000.
Increases the maximum number of result rows displayed in a notebook from 1,000 to 10,000. Not checked yet
1 detail in Everything
Manage MLflow experiment permissions with the Databricks REST API
Aug 2, 2021
· release notes
Databricks adds ability to manage MLflow experiment permissions via the Permissions REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. .
Adds Permissions API endpoints for managing read, edit, and manage access on MLflow experiments Not checked yet
1 detail in Everything
Databricks web interface is localized in Portuguese and French (Public Preview)
Jul 29, 2021
· release notes
Databricks adds French and Portuguese localization to the web interface Public preview .
API changes for updating and replacing IP address lists
Jul 6, 2021
· release notes
Databricks API adds PATCH to update IP list values and PUT to replace contents
Adds PATCH support for /ip-access-lists/<list-id> to programmatically update individual values in an existing IP address list Not checked yet Leverages existing PUT on /ip-access-lists/<list-id> to programmatically replace the contents of an existing IP address list while maintaining the original internal ID Not checked yet
2 details in Everything
Improved access to results in the MLflow runs table
Jul 6, 2021
· release notes
MLflow runs table now shows run age instead of start time and reorders columns.
Metrics columns are ordered before Parameters columns to identify best runs faster Not checked yet Start Time column displays run age (hover for actual time) to reduce column width Not checked yet
2 details in Everything
Databricks ODBC ODBC A standard API, maintained by Microsoft, for connecting applications to relational databases regardless of vendor. Cyber tools use it to query databases without writing engine-specific code. driver 2.6.17 adds Cloud Fetch support for parallel data retrieval via cloud storage.
Adds Cloud Fetch mechanism to fetch data in parallel through cloud storage, improving performance for BI tools by reducing unnecessary client-to-Databricks Runtime roundtrips. Not checked yet
1 detail in Everything
Use an API to download usage data directly
Jun 16, 2021
· release notes
Databricks 3.48 enables direct API-based retrieval of billable usage logs in CSV, eliminating the previous requirement for S3 bucket delivery setup.
Photon runtimes now support i3.xlarge instances (Public Preview)
Jun 14, 2021
· release notes
Photon runtimes now support i3.xlarge instances in Public Preview
Adds i3.xlarge instance type support for clusters running Photon images Not checked yet
1 detail in Everything
Databricks Mosaic AI: a data-native and collaborative solution for the full ML lifecycle
May 27, 2021
· release notes
Adds Machine Learning persona in UI sidebar with dashboard, Experiments page, AutoML public preview, and Feature Store public preview.
Adds Machine Learning persona selectable in the Databricks UI sidebar to access purpose-built ML environment Not checked yet Adds new Dashboard page with convenient resources, recents, and getting started links for ML users Not checked yet Adds new Experiments page that centralizes experiment discovery and management Not checked yet Adds AutoML in Public Preview to automatically generate ML models from data Not checked yet Adds Feature Store in Public Preview to catalog ML features, with data-lineage–based feature search Not checked yet
5 details in Everything
Create and manage ETL pipelines using DLT (Public Preview)
May 26, 2021
· release notes
Public preview of Databricks Live Tables (DLT) for declarative ETL pipeline creation and management
Adds Public Preview support for Databricks Live Tables (DLT), a cloud service to create and manage ETL pipelines Not checked yet Enables a declarative interface to build data processing pipelines with test-driven development and uniform error handling policies Not checked yet Allows monitoring of pipelines, visualization of dependencies, and management across different environments Not checked yet Automates deployment of pipelines supporting upgrade, rollback, and incremental reprocessing capabilities Not checked yet
4 details in Everything
Manage where notebook results are stored (Public Preview)
May 18, 2021
· release notes
New capability to configure notebook result storage location via Configure notebook result storage location .
Enables storing interactive notebook results in the root S3 storage bucket instead of Databricks-managed storage. Not checked yet
1 detail in Everything
Increased limit for the number of terminated all-purpose clusters
May 18, 2021
· release notes
Databricks increases the limit for terminated all-purpose clusters from 120 to 150 in v3.46.
Increased the limit for the number of terminated all-purpose clusters allowed in a workspace to 150 (previously 120). Not checked yet Increased the limit on the number of terminated all-purpose clusters returned by the Clusters API request to 150. Not checked yet
2 details in Everything
Changes to the Account API for customer-managed keys
May 10, 2021
· release notes
Account API adds use_cases property for key configurations and renames customer_managed_key_id in workspace configurations
Adds the use_cases required property to account API key configurations to indicate the purpose of the key Not checked yet Renames the customer_managed_key_id field to managed_services_customer_managed_key_id in workspace configurations Not checked yet
2 details in Everything
└──▷ BREAKING ON UPGRADE
! The customer_managed_key_id field is renamed to managed_services_customer_managed_key_id in workspace configurations; API calls must use the new name.! The use_cases property becomes a required field for key configurations.
Customer-managed keys for workspace storage (Public Preview)
May 10, 2021
· release notes
Databricks adds public preview for customer-managed keys (CMK) to encrypt workspace S3 buckets and optionally cluster node EBS EBS Amazon's block-storage service for EC2 instances, providing persistent network-attached volumes that survive instance restarts and can be snapshotted, giving cyber tools durable, scalable disk without managing physical drives. volumes.
Adds support for configuring a customer-managed workspace storage key in new or existing E2 workspaces Public preview . Not checked yet Enables using the same CMK to optionally encrypt workspace cluster node EBS volumes. Not checked yet
2 details in Everything
AWS PrivateLink for Databricks workspaces (Public Preview)
Apr 20, 2021
· release notes
Databricks releases AWS PrivateLink AWS PrivateLink An AWS networking feature that exposes a service over a private VPC endpoint rather than the public internet, letting tools move data between accounts or to AWS services without traffic ever leaving Amazon's backbone. in public preview for E2 workspaces, enabling private VPC VPC A logically isolated virtual network inside a public cloud provider's infrastructure, letting tools deploy resources with private addressing, controlled routing, and firewall rules without managing physical hardware. endpoints for frontend and backend connectivity.
New free trial and pay-as-you-go customers are now on the E2 version of the platform
Apr 12, 2021
· release notes
Free trial and pay-as-you-go accounts now provisioned on E2 platform with account console and management APIs.
Free trial sign-up no longer requires credit card information up front. Not checked yet New user-friendly account console available for free trial customers to set up subscriptions, create workspaces, add account admins, manage billing, and view usage. Not checked yet Pay-as-you-go customers now gain access to E2 platform benefits including multiple workspaces per account, enterprise-grade security, and account management APIs. Not checked yet
3 details in Everything
Databricks Repos let you use Git repositories to integrate Databricks with CI/CD systems
Mar 16, 2021
· release notes
Databricks adds Git integration for repos via GitHub, Bitbucket, and GitLab.
Added modification_time to the DBFS REST API get-status and list responses
Mar 2, 2021
· release notes
DBFS REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. now returns modification_time in get-status and list responses
New Databricks Power BI connector (GA)
Feb 26, 2021
· release notes
New Databricks Power BI connector reaches GA with optimized ODBC ODBC A standard API, maintained by Microsoft, for connecting applications to relational databases regardless of vendor. Cyber tools use it to query databases without writing engine-specific code. driver and integrated configuration dialog
Adds a general availability Power BI connector for Azure Databricks, replacing the public preview released in September 2020. Not checked yet Introduces a simple configuration dialog within Power BI to set up connections with just a couple of clicks. Not checked yet Leverages a new Databricks ODBC driver to provide faster imports and optimized metadata calls for improved performance. Not checked yet Ensures that data access from Power BI respects existing Databricks table access control policies. Not checked yet
4 details in Everything
Add and manage account admins using the SCIM API (E2 accounts, Public Preview)
Feb 25, 2021
· release notes
E2 accounts can now add and manage account admins via the SCIM SCIM A standard for creating, updating and removing user accounts and groups across applications. It helps an identity provider keep access in sync when people join, change roles or leave. API Public preview
Adds SCIM API support for adding multiple account administrators on E2 accounts Not checked yet
1 detail in Everything
Billable usage and audit log S3 bucket policy and object ACL changes
Jan 14, 2021
· release notes
Databricks adds cross-account S3 support for audit/billable logs via new BucketOwnerFullControl ACL ACL An access control list, a set of rules allowing or denying operations for particular users, services or network addresses. It decides who can use a resource. template
Adds automatic usage and audit log delivery to an S3 bucket in a different AWS account than the IAM IAM Identity and Access Management — a framework of policies and services for controlling who can authenticate to systems and what resources they are authorized to use, letting cyber tools enforce least-privilege and audit access. role's account Not checked yet Provides a template to create the S3 bucket policy required for cross-account log delivery Not checked yet Supports BucketOwnerFullControl Canned ACL for usage/audit logs delivered to S3 Not checked yet
3 details in Everything
E2 platform comes to the Asia Pacific region
Jan 4, 2021
· release notes
E2 platform becomes available in AWS ap-southeast-2, ap-south-1, and ap-northeast-1 regions.
Enables creation of multiple workspaces per account in the AWS Asia Pacific region Not checked yet Adds support for the ap-southeast-2 (Sydney) region Not checked yet Adds support for the ap-south-1 (Mumbai) region Not checked yet Adds support for the ap-northeast-1 (Tokyo) region Not checked yet
4 details in Everything
Existing Databricks accounts migrate to E2 platform today
Dec 16, 2020
· release notes
Migration brings new security features and multi-workspace management capabilities via the Account API.
New global init script framework is GA
Dec 14, 2020
· release notes
Databricks GA of new global init script framework with enhanced security and management via API and UI.
Init scripts now require admin permissions for creation, viewing, and deletion. Not checked yet Launch failures for init scripts are now logged. Not checked yet Execution order for multiple init scripts can be configured. Not checked yet Init scripts can access cluster-related environment variables. Not checked yet
4 details in Everything
Jobs API now supports updating existing jobs
Dec 14, 2020
· release notes
Databricks Jobs API gains Update endpoint for partial job configuration changes
Adds Update (update() ) API endpoint to modify a subset of fields for existing jobs without overwriting all settings Not checked yet
1 detail in Everything
New account console enables customers on the E2 platform to create and manage multiple workspaces (Public Preview)
Dec 8, 2020
· release notes
The E2 platform introduces a new account console in public preview for managing multiple workspaces and viewing organization spend.
Use your own keys to secure notebooks (Public Preview)
Dec 1, 2020
· release notes
Public Preview allows Databricks E2/workspaces to encrypt notebooks and secrets using customer-managed keys
Enables customer-managed keys for notebooks to specify your own encryption key for notebook and secret data in the control plane Not checked yet
1 detail in Everything
Filter experiment runs based on whether a registered model is associated
Dec 1, 2020
· release notes
Adds ability to filter experiment runs by association with registered model versions
Adds filter capability for experiment runs based on whether they created a model version Not checked yet
1 detail in Everything
Introduces Databricks SQL public preview for ad-hoc querying and dashboarding on the lakehouse.
Adds Databricks SQL public preview environment for running ad-hoc queries and creating dashboards. Not checked yet Adds SQL warehouses to provide reliability, quality, scale, security, and performance for analytics workloads. Not checked yet Enables direct integration with Tableau and Microsoft Power BI to query lakehouse data. Not checked yet
3 details in Everything
Web terminal available on Databricks Free Edition
Nov 5, 2020
· release notes
Web terminal enabled by default on Databricks Free Edition
New Databricks Power BI connector available in the online Power BI service (Public Preview)
Oct 28, 2020
· release notes
Databricks Power BI connector now available in the online Power BI service Public preview
Self-service, low-latency audit log configuration (Public Preview)
Oct 14, 2020
· release notes
Databricks adds Account API for self-service, low-latency audit log delivery to S3.
Adds the ability to configure Databricks audit logs via the Account API for Premium+ accounts. Not checked yet Enables a single shared audit log delivery configuration across all workspaces in an account. Not checked yet Delivers auditable events with low latency, typically within 15 minutes. Not checked yet
3 details in Everything
New Databricks Power BI connector (Public Preview)
Sep 22, 2020
· release notes
Databricks integrates with Power BI Desktop via a new public preview connector.
New JDBC and ODBC drivers bring faster and lower latency BI
Sep 15, 2020
· release notes
New Databricks JDBC JDBC A Java standard API for connecting applications to relational databases, maintained as part of the Java SE platform, letting tools query any compliant database through a single consistent interface. /ODBC ODBC A standard API, maintained by Microsoft, for connecting applications to relational databases regardless of vendor. Cyber tools use it to query databases without writing engine-specific code. drivers support Microsoft Entra ID Entra ID Microsoft's cloud-based identity and access management service, formerly Azure Active Directory, used by cyber tools to authenticate users and enforce access policies in Microsoft 365 and Azure environments. OAuth2 OAuth2 An open authorization framework (RFC 6749) that lets a service grant limited access to a user's resources on another service without exposing credentials, giving cyber tools a standard way to authenticate against third-party APIs. , HTTP proxies, and Arrow serialization for lower latency.
Adds Microsoft Entra ID OAuth2 access token access token A credential that authorizes a client to use a service or API. It usually has limited permissions and a lifetime; anyone holding it may be able to use those permissions. authentication for JDBC/ODBC connections Not checked yet Adds support for connections using HTTP proxy Not checked yet Improves result transfer speed via Apache Arrow serialization Not checked yet
3 details in Everything
MLflow Model Registry supports sharing of models across workspaces
Sep 9, 2020
· release notes
MLflow Model Registry now supports sharing models across Databricks workspaces.
Adds the ability to register, track, and load MLflow models across multiple Databricks workspaces. Not checked yet
1 detail in Everything
E2 architecture—now GA—provides better security, scalability, and management tools
Sep 1, 2020
· release notes
Databricks E2 architecture general availability on AWS with stronger security controls and API-driven management.
Makes the new E2 architecture generally available for production use on AWS. Not checked yet
1 detail in Everything
Token Management API is GA and admins can use the Admin Console to grant and revoke user access to tokens
Sep 1, 2020
· release notes
Databricks Token Management API is GA with Admin Console controls for user token grants and revocations
Token Management API reaches general availability for monitoring and revoking users' personal access tokens personal access tokens A personal access token, a credential used by scripts and integrations instead of a user's password. Its permissions and expiry depend on the service issuing it. Not checked yet
1 detail in Everything
└──▷ BREAKING ON UPGRADE
! The Token Management API parameter created_by is renamed to created_by_id! A new required or available parameter created_by_username appears in the Token Management API response or input
Message size limits for Shiny apps increased
Sep 1, 2020
· release notes
Databricks increases Shiny app maximum application size from 10 MB to 20 MB.
Maximum Shiny app package size raised to 20 MB (previously 10 MB) Not checked yet
1 detail in Everything
Databricks Runtime 7.2 Genomics GA introduces optimized numpy to Java array conversion for Glow-based genomic analysis.
Adds significant performance improvement for converting literal numpy 1D and 2D float-typed ndarrays to Java arrays. Not checked yet
1 detail in Everything
Databricks Runtime 7.2 GA adds Auto Loader directory listing mode, cloud resource management API, rate limiting, and option validation.
Adds cloudFiles.listingMode to readStream for S3 directory polling. Not checked yet Exposes Scala APIs for Auto Loader's notification services (listing and tearing down). Not checked yet Supports cloudFiles.maxBytesPerTrigger option in Auto Loader streams. Not checked yet Enables cloudFiles.validateOptions=false in Auto Loader to skip input validation. Not checked yet
4 details in Everything
Adds new file upload dialog for small tabular data files to Databricks notebooks (v3.25).
Databricks IP Access List API general availability with renamed list_type values
The IP Access List API is now generally available to restrict workspace access to approved IP addresses. Not checked yet
1 detail in Everything
└──▷ BREAKING ON UPGRADE
! The list_type values for the IP Access List API are renamed: WHITELIST becomes ALLOW and BLACKLIST becomes BLOCK.
SCIM API filter and sort improvements
Aug 4, 2020
· release notes
The SCIM SCIM A standard for creating, updating and removing user accounts and groups across applications. It helps an identity provider keep access in sync when people join, change roles or leave. API v2 in Databricks 3.25 gains client-side filtering on the active attribute and explicit sorting controls.
Admins can now filter user lists by the active status. Not checked yet
1 detail in Everything
New, more secure global init script framework (Public Preview)
Aug 4, 2020
· release notes
Databricks v3.25 introduces a new global init script framework with public preview status and admin API support.
Adds the ability to set execution order for multiple global init scripts in the cluster configuration. Not checked yet Adds Global Init Scripts REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. endpoints for creating and managing init scripts programmatically. Not checked yet Adds visibility into launch failures for global init scripts via new logging mechanisms. Not checked yet
3 details in Everything
Databricks version 3.25 introduces a new browser-based shell interface for clusters, simplifying remote access by removing the need for SSH key management.
Web terminal allows multiple users to attach to a single cluster simultaneously without configuration overhead. Not checked yet
1 detail in Everything
Databricks Runtime 7.1 ML GA enables pip/conda magics by default and updates core ML libraries.
pip and conda magic commands are now enabled by default in Databricks Runtime 7.1 for Machine Learning Not checked yet Updates to mlflow (1.8.0 -> 1.9.1), pytorch (1.5.0 -> 1.5.1), torchvision (0.6.0 -> 0.6.1), horovod (0.19.1 -> 0.19.5), and pillow (7.0.0 -> 7.1.0) Not checked yet Adds spark-tensorflow-distributor version 0.1.0 to the runtime Not checked yet
3 details in Everything
Databricks Runtime 7.1 Genomics GA introduces LOCO transformation, GloWGR output reshaping, and unpaired RNASeq alignments.
LOCO transformation for genomics workflows Not checked yet GloWGR output reshaping function Not checked yet
2 details in Everything
Databricks Runtime 7.1 GA adds BigQuery BigQuery A serverless, cloud-hosted analytical database from Google that runs SQL queries across massive datasets without managing infrastructure, giving cyber tools scalable storage and fast querying over large volumes of log or event data. connector, %pip commands, Koalas, and Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. metadata/PERFORMANCE enhancements.
Adds %pip magic command for managing Python libraries in notebook sessions Not checked yet Adds Google BigQuery connector integration Not checked yet Includes Koalas DataFrame library Not checked yet
3 details in Everything
Customer-managed VPC deployments (Public Preview) can now use regional VPC endpoints
Jul 8, 2020
· release notes
Customer-managed VPC VPC A logically isolated virtual network inside a public cloud provider's infrastructure, letting tools deploy resources with private addressing, controlled routing, and firewall rules without managing physical hardware. deployments in Public Preview can now leverage regional AWS VPC endpoints for enhanced security and cost efficiency.
Billable usage logs delivered to your own S3 bucket (Public Preview)
Jun 30, 2020
· release notes
Public preview of S3 delivery for billable usage logs provides programmatic access to cluster DBU data for account owners.
Databricks Runtime 7.0 ML GA adds notebook-scoped conda/pip environments, RStudio Server Open Source v1.2, and new ML libraries
Adds notebook-scoped Python libraries and custom environments managed by conda and pip commands. Not checked yet Integrates RStudio Server Open Source v1.2 into the runtime. Not checked yet Includes updates for major Python packages including tensorflow, tensorboard, pytorch, xgboost, sparkdl, and hyperopt. Not checked yet Adds support for newly installed Python packages lightgbm, nltk, petastorm, and plotly. Not checked yet
4 details in Everything
Databricks Runtime 7.0 GA, powered by Apache Spark 3.0
Jun 18, 2020
· release notes
Databricks Runtime 7.0 adds Scala 2.12 support, Structured Streaming web UI, ANSI SQL compatibility, and Join hints.
Supports Scala 2.12 as the language version Not checked yet Adds a Structured Streaming web UI for monitoring streaming jobs Not checked yet Enables better compatibility with ANSI SQL standards in Spark SQL Not checked yet Adds support for Join hints to guide query optimization Not checked yet
4 details in Everything
Deploy Databricks workspaces in your own VPC (Public Preview)
Jun 1, 2020
· release notes
Public preview allows deploying workspaces in a customer-managed VPC VPC A logically isolated virtual network inside a public cloud provider's infrastructure, letting tools deploy resources with private addressing, controlled routing, and firewall rules without managing physical hardware. via the Account API.
Supports creating Databricks workspaces in a customer-managed VPC on the E2 platform using Multi-workspace API (Account API) with VPC ID, security group ID, and subnet IDs. Not checked yet
1 detail in Everything
Secure cluster connectivity with no open ports on your VPCs and no public IP addresses on Databricks workers (Public Preview)
Jun 1, 2020
· release notes
Public Preview: Secure Cluster Connectivity enables outbound-only TLS 1.2 connections, removing the need for open inbound ports or VPC VPC A logically isolated virtual network inside a public cloud provider's infrastructure, letting tools deploy resources with private addressing, controlled routing, and firewall rules without managing physical hardware. peering.
Adds Secure cluster connectivity Public preview allowing compute plane VMs in customer-managed VPCs VPCs Virtual Private Clouds — isolated, software-defined network environments inside a public cloud provider such as AWS, Azure, or GCP — give cyber tools a controlled network boundary with configurable routing, subnets, and access rules. to initiate outbound TLS 1.2 connections to the Databricks control plane without opening inbound ports. Not checked yet Eliminates the need for VPC peering by using a new network architecture for connectivity between the Databricks SaaS SaaS Software delivered over the internet and billed by subscription, hosted and operated by the vendor so the buyer skips installation and infrastructure management. control plane and AWS VPC compute plane. Not checked yet
2 details in Everything
Restrict access to Databricks using IP access lists (Public Preview)
Jun 1, 2020
· release notes
New public preview capability allowing admins to restrict workspace access via IP allow/block lists through a new API.
Databricks Runtime 6.6 for Genomics GA
May 26, 2020
· release notes
Databricks Runtime 6.6 for Genomics GA adds GFF3 reading, custom references, per-sample timeouts, BAM export, and manifest blobs.
Adds support for reading GFF3 format files directly in the runtime Not checked yet Enables configuration of a custom reference genome for alignment workflows Not checked yet Introduces per-sample pipeline timeouts to control execution duration per sample Not checked yet Provides a new BAM export option to write output in BAM format Not checked yet Supports handling of manifest blobs for metadata or large file indexing Not checked yet
5 details in Everything
Databricks Runtime 6.6 GA adds automatic schema evolution for merge, improved performance for matched-only merge operations, and CONVERT TO DELTA via table identifiers.
Adds merge clause support for automatic schema evolution to upsert data with changing schemas Not checked yet Improves performance of merge operations containing only matched clauses (update and delete actions without insert) Not checked yet Enables conversion of Parquet tables in the Hive metastore Hive metastore A central metadata repository, part of the Apache Hive project, that stores schema and location information for distributed data sets so query engines can discover and read structured data without hardcoding paths. to Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. using the CONVERT TO DELTA command with table identifiers Not checked yet
3 details in Everything
Databricks Runtime 7.0 ML (Beta)
Prerelease
May 21, 2020
· release notes
Databricks Runtime 7.0 ML Beta adds notebook-scoped library management via conda/pip and includes lightgbm and plotly packages.
Adds notebook-scoped Python libraries and custom environments managed by conda and pip commands Not checked yet Newly added Python packages lightgbm, nltk, petastorm, and plotly Not checked yet
2 details in Everything
Databricks Runtime 6.6 for Genomics (Beta)
May 7, 2020
· release notes
Databricks Runtime 6.6 for Genomics Beta adds GFF3 reader, custom reference genome support, per-sample timeouts, BAM export, and manifest blobs.
Adds GFF3 file format reading capability to the Databricks genomics pipeline. Not checked yet Enables loading of custom reference genome files for variant calling or alignment instead of relying on default references. Not checked yet Introduces per-sample pipeline timeouts to control execution limits on a per-sample basis. Not checked yet Adds a BAM export option to write processed genomic data back into BAM format. Not checked yet Supports reading manifest blobs as input for batch job orchestration. Not checked yet
5 details in Everything
Databricks Runtime 6.6 (Beta)
Prerelease
May 7, 2020
· release notes
Databricks Runtime 6.6 Beta adds automatic schema evolution for merge, improved matched-only merge performance, and CONVERT TO DELTA for Hive Parquet tables.
merge operation now supports automatic schema evolution to upsert changing data schemas Not checked yet Improved performance for merge operations containing only MATCHED clauses (update/delete actions) Not checked yet Added CONVERT TO DELTA SQL syntax to convert Hive metastore Hive metastore A central metadata repository, part of the Apache Hive project, that stores schema and location information for distributed data sets so query engines can discover and read structured data without hardcoding paths. Parquet tables to Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. Not checked yet
3 details in Everything
Databricks Runtime 6.5 adds streaming rate-limiting and Delta operation metrics in history.
Adds maxRecordsPerBatch to control the maximum number of records processed per micro-batch in Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. streaming Not checked yet Exposes write, update, and delete operation metrics in Delta table history for all changes Not checked yet
2 details in Everything
Databricks Runtime 6.4 for Genomics GA
Feb 26, 2020
· release notes
Databricks Runtime 6.4 for Genomics GA adds configurable pipeline stages and Python/Scala APIs for Glow functions.
DNASeq Pipeline users can selectively disable read alignment, variant calling, or variant annotation stages. Not checked yet Users can perform single-end read alignment in DNASeq Pipeline. Not checked yet Glow includes Python and Scala APIs for DataFrame operations, previously available only via SQL expressions. Not checked yet
3 details in Everything
Databricks Runtime 6.4 GA adds Auto Loader public preview, COPY INTO idempotent retries, table history metrics, and inline Matplotlib figures
Adds Auto Loader Public preview for incremental processing of new data files arriving on cloud blob stores during ETL, replacing inefficient file-based structured streaming. Not checked yet Adds COPY INTO SQL command Public preview to load data into Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. with idempotent retries without requiring Apache Spark DataFrame APIs. Not checked yet Shows operation metrics for all writes, updates, and deletes on a Delta table directly in the table history view. Not checked yet Enables inline Matplotlib figures by default in Databricks notebooks Public preview . Not checked yet
4 details in Everything
New data ingestion network adds partner integrations with Delta Lake (Public Preview)
Feb 24, 2020
· release notes
New Partner Integrations gallery adds data source connectors via Fivetran, Qlik, Infoworks, StreamSets, and Syncsort to Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. Public preview
Adds the Partner Integrations gallery accessible from the Databricks workspace Not checked yet Enables ingestion from hundreds of data sources through partners Fivetran, Qlik, Infoworks, StreamSets, and Syncsort into Delta Lake Not checked yet
2 details in Everything
All cluster and pool tags now propagate to usage reports
Jan 30, 2020
· release notes
Cluster and pool tags now propagate to usage reports with a new tags column in the CSV export
New tags column added to the usage download CSV report showing all custom and default cluster tags plus pool tags. Not checked yet Pool tags (custom and default) now propagate to the cluster level for DBU usage capture. Not checked yet
2 details in Everything
Databricks Runtime 6.3 for Genomics GA
Jan 22, 2020
· release notes
Databricks Runtime 6.3 for Genomics GA adds Delta tables support, auto VCF VCF VMware Cloud Foundation, a VMware (now Broadcom) platform that bundles compute, storage, and networking virtualisation into one stack, giving cyber tools a unified API surface across private-cloud infrastructure. annotation parsing, improved variant splitter, and faster regression functions.
Supports Delta tables as input to the joint genotyping pipeline Not checked yet Adds automatic annotation parsing when reading VCF files Not checked yet Introduces an improved multiallelic variant splitter Not checked yet
3 details in Everything
Databricks Runtime 6.3 GA introduces manifest file support and concurrency/performance improvements.
Adds support for processing engines using manifest files to read Delta tables Not checked yet Improves concurrency for all Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. operations Not checked yet Improves support for file compaction Not checked yet Improves performance for insert-only merge Not checked yet
4 details in Everything
Databricks Runtime 6.2 for Genomics GA
Dec 3, 2019
· release notes
Databricks Runtime 6.2 for Genomics GA introduces Firth logistic regression, Hail 0.26.0, and LOFTEE integration.
Azure Databricks SCIM provisioning connector available in the app gallery
Dec 3, 2019
· release notes
Microsoft Entra ID Entra ID Microsoft's cloud-based identity and access management service, formerly Azure Active Directory, used by cyber tools to authenticate users and enforce access policies in Microsoft 365 and Azure environments. (formerly Azure Active Directory Active Directory Microsoft's directory service for Windows networks that centralises authentication and authorisation; cyber tools target or integrate with it because it controls identity and access across most enterprise environments. ) app gallery now lists the Azure Databricks SCIM SCIM A standard for creating, updating and removing user accounts and groups across applications. It helps an identity provider keep access in sync when people join, change roles or leave. Provisioning Connector for user and group provisioning setup.
Databricks Connect now supports Databricks Runtime 6.1
Dec 3, 2019
· release notes
Databricks Connect now supports Databricks Runtime 6.1
Databricks Runtime 6.2 Beta
Prerelease
Nov 15, 2019
· release notes
Databricks Runtime 6.2 Beta introduces insert-only Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. merges and multi-region read support for the Redshift connector.
Adds insert-only merge optimization to Delta Lake, enabling efficient CDC workloads by only writing new rows. Not checked yet Adds multi-region support for reads in the Redshift connector. Not checked yet
2 details in Everything
Secrets referenced by Spark configuration properties and environment variables (Public Preview)
Nov 7, 2019
· release notes
Databricks Runtime 6.1+ now supports referencing secrets directly in Spark configuration properties and environment variables.
Adds the ability to reference a secret in a Spark configuration property or environment variable Not checked yet
1 detail in Everything
Databricks Runtime 6.1 for Machine Learning GA
Oct 22, 2019
· release notes
Databricks Runtime 6.1 GA adds GPU GPU A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. cluster support and upgrades TensorFlow, PyTorch, Torchvision, and MLflow libraries.
Upgrades MLflow to version 1.3.0 Not checked yet
1 detail in Everything
Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. APIs in Public Preview and Dynamic File Pruning enabled by default in Databricks Runtime 6.1.
Adds Python APIs for Delta tables Public preview Not checked yet Enables Dynamic File Pruning (DFP) by default Not checked yet Moves GPU GPU A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. support to Databricks Runtime ML, restricting Runtime 6.1 to CPU CPU The main general-purpose processor in a computer. It runs application instructions and coordinates work with memory and other hardware. clusters Not checked yet
3 details in Everything
└──▷ BREAKING ON UPGRADE
! Databricks Runtime 6.1 supports only CPU clusters; GPU clusters require Databricks Runtime ML.
Databricks Runtime 6.0 ML GA adds Spark-based access to MLflow experiment data and a Java client, expanding tooling for machine learning workflows.
Adds a new Spark data source for MLflow experiments to provide a standard API to load experiment run data Not checked yet Adds MLflow Java Client library support Not checked yet
2 details in Everything
Databricks Runtime 6.0 adds Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. DML/vacuum/history APIs, Python 3.7, and multi-plot notebook support.
Adds new Scala and Java APIs for Delta Lake DML commands, plus vacuum and history utility commands Not checked yet Enhances the DBFS FUSE FUSE A Linux kernel interface that lets userspace programs implement a filesystem without kernel code, maintained as part of the kernel project; tools use it to mount virtual or synthetic filesystems without requiring root or kernel modules. client with faster and more reliable reads/writes during model training Not checked yet Supports multiple matplotlib plots per notebook cell Not checked yet Updates the Python runtime to version 3.7 and refreshes core libraries (numpy, pandas, matplotlib) Not checked yet Ends support for Python 2 Not checked yet
5 details in Everything
└──▷ BREAKING ON UPGRADE
! Python 2 clusters are no longer supported in this release
Account usage reports now show usage by user name
Oct 1, 2019
· release notes
Account usage reports on the Console now display usage by user name instead of just ID.
Launch pool-backed automated clusters that use Databricks Light (Public Preview)
Oct 1, 2019
· release notes
Databricks Light added as a runtime option for pool-backed automated job clusters Public preview
Beta support for m5a and r5a instances
Oct 1, 2019
· release notes
Databricks adds beta support for m5a and r5a EC2 EC2 Amazon's cloud virtual-machine service, letting tools run on-demand servers without managing physical hardware. instances
Databricks Runtime 6.0 Beta
Prerelease
Sep 12, 2019
· release notes
Databricks Runtime 6.0 Beta adds Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. APIs/vacuum/history, DBFS FUSE FUSE A Linux kernel interface that lets userspace programs implement a filesystem without kernel code, maintained as part of the kernel project; tools use it to mount virtual or synthetic filesystems without requiring root or kernel modules. v2, and multiple matplotlib plots per cell.
Adds new Scala and Java APIs for Delta Lake DML, plus vacuum and history utility commands Not checked yet Enhances DBFS FUSE client to version 2 for faster and more reliable reads/writes during model training Not checked yet Supports multiple matplotlib plots per notebook cell instead of one Not checked yet Updates Python base to version 3.7 (and associated libraries) Not checked yet
4 details in Everything
└──▷ BREAKING ON UPGRADE
! Python 2 support is removed; existing Python 2 code will fail on this runtime.
Custom cluster tags and pool tags play better together
Aug 6, 2019
· release notes
Adds cluster-level custom tags for pool-backed clusters in Databricks Runtime 2.103
Enables configuring custom_tags on pool-backed clusters, applying both inherited pool tags and cluster-specific tags. Not checked yet
1 detail in Everything
Preload the Databricks Runtime version on pool idle instances
Aug 6, 2019
· release notes
Adds Preloaded Spark Version UI field to speed up cluster launches from pools.
Adds Preloaded Spark Version selection in the Pool UI to preload Databricks Runtime on idle instances. Not checked yet
1 detail in Everything
MLflow 1.1 brings several UI and API improvements
Aug 6, 2019
· release notes
MLflow 1.1 adds paginated runs UI, parallel coordinates plots, tag management UI/API, and MLflowContext API.
Runs overview UI now paginates results beyond 100 runs with a Load more button Not checked yet Compare runs UI gains a parallel coordinates plot for n-dimensional parameter/metric analysis Not checked yet Run overview UI allows adding and editing tags; experiment search view displays tags Not checked yet Adds MLflowContext API for creating and logging runs similar to Python API Not checked yet Adds DeleteTag API endpoint for removing tags from MLflow runs Not checked yet
5 details in Everything
Databricks Runtime 5.5 for Machine Learning
Jul 15, 2019
· release notes
Databricks Runtime 5.5 ML adds MLflow 1.0 and upgrades TensorFlow, PyTorch, scikit-learn
Adds MLflow 1.0 Python package for machine learning experiment tracking and model management. Not checked yet Upgrades TensorFlow from 1.12.0 to 1.13.1, enabling new features and bug fixes in the core library. Not checked yet Upgrades PyTorch from 0.4.1 to 1.1.0, providing updated deep learning capabilities. Not checked yet Upgrades scikit-learn from 0.19.1 to 0.20.3, bringing new estimators and performance improvements. Not checked yet
4 details in Everything
Databricks Runtime 5.5 adds Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. on Databricks Auto Optimize GA, Presto/Athena support, Glue Catalog metastore, and R Secrets API.
Delta Lake on Databricks Auto Optimize is now Generally Available Not checked yet Adds Presto and Athena support for Delta tables on AWS S3 Public preview Not checked yet Supports Glue Catalog as a Databricks metastore Not checked yet Adds improved binary file data source and scalar iterator pandas UDF for faster model inference pipelines Public preview Not checked yet
4 details in Everything
Account usage chart updated to display usage grouped by workload type
Jun 26, 2019
· release notes
Databricks account usage chart now groups DBU consumption by workload type with backfilled historical data.
Updates the account usage graph to display DBUs grouped by workload type (product SKU): data analytics, data engineering, and data engineering light. Not checked yet Backfills the usage graph with prior month data. Not checked yet
2 details in Everything
└──▷ BREAKING ON UPGRADE
! The quantity column in the downloadable usage CSV report (via Download itemized usage ) is renamed to dbus starting July 1.
Databricks Runtime 5.4 with Conda (Beta)
Jun 3, 2019
· release notes
Databricks Runtime 5.4 with Conda introduces two selectable root Conda environments for cluster creation.
Databricks Runtime 5.4 adds Auto Optimize (Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. ), Databricks Connect IDE support, and AWS Glue metastore.
Adds autoOptimize option in Delta Lake Public Preview for automatic table optimization Not checked yet Enables use of favorite IDEs via Databricks Connect notebook server integration Not checked yet Provides AWS Glue metastore integration in Public Preview Not checked yet
3 details in Everything
Databricks Runtime 5.4 for Machine Learning
Jun 3, 2019
· release notes
Databricks Runtime 5.4 ML adds MLflow integration, SparkTrials, and Horovod log visibility
Adds MLlib integration with MLflow for experiment tracking and model registry in public preview Not checked yet Includes Hyperopt with the new SparkTrials class pre-installed for distributed hyperparameter tuning in public preview Not checked yet Makes HorovodRunner output sent from Horovod to the Spark driver node visible in notebook cells Not checked yet
3 details in Everything
MLflow 1.0 adds CLI subcommands, batch logging API, SQL search, HDFS backend, and ONNX ONNX An open file format for trained models, so a model built in one framework can run in another runtime or on other hardware. ONNX Runtime is the common engine that executes it. flavor.
Adds mlflow artifacts, models, db, and server CLI subcommands for dedicated management of each component Not checked yet Adds runs/log-batch REST API REST API A web API design style where clients interact with resources over HTTP using standard methods like GET, POST, and DELETE, giving cyber tools a predictable, language-agnostic way to integrate with external services. endpoint (and Python, R, Java methods) to log multiple metrics, params, and tags in a single request Not checked yet Adds support for x coordinates in the Tracking API and UI visualization components to plot metrics against provided x-coordinate values Not checked yet Adds HDFS as an artifact store backend option Not checked yet Adds experimental ONNX model flavor support Not checked yet
5 details in Everything
Databricks Runtime 5.3 GA releases Delta time travel and MySQL replication preview alongside library upgrades.
Databricks Runtime 5.3 ML GA adds Conda package management, PyArrow BinaryType in pandas UDFs, and private preview MLflow integration for pyspark tuning algorithms
Adds Conda package management for Python libraries in Databricks Runtime ML clusters Not checked yet Enables BinaryType in Arrow-based conversion via pandas UDF through PyArrow upgrade Not checked yet Provides private preview of MLlib integration with MLflow for automatic run logging in CrossValidator and TrainValidationSplit Not checked yet
3 details in Everything
Databricks Light generally available as a new runtime option for JAR, Python, and spark-submit jobs.
Adds searchable input for cluster selection when attaching notebooks
Enables searching for a cluster by typing in the cluster selector when attaching a notebook Not checked yet
1 detail in Everything
Notebooks automatically have associated MLflow experiment
Mar 19, 2019
· release notes
Every workspace notebook now automatically has an associated MLflow experiment.
Manage groups from the Admin Console
Mar 19, 2019
· release notes
Databricks v2.93 adds a Groups tab in the Admin Console for creating and managing groups
Adds the Groups tab to the Admin Console, allowing admins to create new groups. Not checked yet Enables adding users and subgroups to a group directly from the Admin Console UI. Not checked yet Allows granting cluster creation permissions to group members via the Admin Console. Not checked yet Supports adding instance profiles to group members through the Admin Console interface. Not checked yet Provides functionality to manage parent-child relationships between groups in the Admin Console. Not checked yet
5 details in Everything
Managed MLflow on Databricks Public Preview
Mar 5, 2019
· release notes
Databricks introduces public preview of Managed MLflow with integrated tracking UI and job execution.
Adds hosted MLflow Tracking Server for collaborative experiment tracking within Databricks Workspaces Not checked yet Adds integrated experiment UI to organize results and capture notebook revisions automatically Not checked yet Adds ability to run MLflow projects as Databricks jobs remotely or directly from notebooks Not checked yet
3 details in Everything
New interactive charts offer rich client-side interactions
Mar 3, 2019
· release notes
New interactive chart types with custom plot options (Y-axis range, markers, log scale) and a built-in toolbar for client-side interactions in Version 3.14.
Replaces bar chart and line chart implementations with two new interactive chart types Not checked yet Adds Y-axis range setting to the line chart Not checked yet Adds show/hide markers option to the line chart Not checked yet Adds log scale application to the Y-axis for the line chart Not checked yet Introduces a built-in toolbar supporting rich client-side interactions for both new chart types Not checked yet
5 details in Everything
Azure Data Lake Storage connector is generally available
Feb 15, 2019
· release notes
Azure Data Lake Storage connector is now generally available for Databricks.
Databricks Runtime 5.2 adds Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. support and Spark 2.4.0
Adds Delta Lake transactional storage layer to Databricks Runtime Not checked yet Upgrades Apache Spark engine to version 2.4.0 Not checked yet Integrates Structured Streaming feature improvements Not checked yet
3 details in Everything
Databricks Runtime 5.2 for Machine Learning (Beta) release
Jan 24, 2019
· release notes
Databricks Runtime 5.2 ML adds Horovod multi-GPU GPU A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. support and Pregel API to GraphFrames
Adds HorovodRunner built-in support for mapping training processes to GPUs GPUs A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. on multi-GPU instance types Not checked yet Updates GraphFrames to support the Pregel API (Python) with performance optimizations Not checked yet Makes HorovodRunner.run() return the return value from the first training process Not checked yet
3 details in Everything
Databricks Runtime 5.1 for Machine Learning (Beta) release
Dec 18, 2018
· release notes
Databricks Runtime 5.1 ML Beta adds PyTorch support for deep learning
Databricks Runtime 5.1 adds Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. support alongside Spark 2.4.0 and updated libraries.
Adds support for the new Delta Lake data lakehouse layer on top of Apache Spark. Not checked yet
1 detail in Everything
Custom Spark heap memory settings enabled
Dec 4, 2018
· release notes
Version 2.85 enables custom spark.executor.memory and spark.driver.memory settings
Adds the ability to set spark.executor.memory for Spark executors Not checked yet Adds the ability to set spark.driver.memory for the Spark driver Not checked yet
2 details in Everything
Databricks Runtime 5.0 for Machine Learning (Beta) release
Nov 19, 2018
· release notes
Databricks Runtime 5.0 ML adds HorovodRunner for distributed training and Conda support
Adds Conda support for package management in the runtime environment Not checked yet Adds MLeap integration for model serialization Not checked yet Adds GraphFrames integration for graph processing Not checked yet
3 details in Everything
Databricks Runtime 5.0 adds Delta Lake Delta Lake An open-source storage layer maintained by the Linux Foundation that adds ACID transactions and versioned history to data lakes, letting tools reliably read and write large-scale tabular data without corruption. support and idle execution context eviction.
Adds Delta Lake support as a new data management capability Not checked yet Introduces automatic eviction of idle execution contexts once the cluster reaches the maximum limit of 145 Not checked yet
2 details in Everything
This release adds a preview feature enabling SCIM SCIM A standard for creating, updating and removing user accounts and groups across applications. It helps an identity provider keep access in sync when people join, change roles or leave. -based user provisioning and deprovisioning with OneLogin for Databricks.
SCIM provisioning using Okta and Microsoft Entra ID (Preview)
Sep 26, 2018
· release notes
Databricks adds preview support for SCIM SCIM A standard for creating, updating and removing user accounts and groups across applications. It helps an identity provider keep access in sync when people join, change roles or leave. user/group provisioning with Okta and Microsoft Entra ID Entra ID Microsoft's cloud-based identity and access management service, formerly Azure Active Directory, used by cyber tools to authenticate users and enforce access policies in Microsoft 365 and Azure environments.
Enables SCIM provisioning of users and groups from Okta and Microsoft Entra ID Not checked yet
1 detail in Everything
New environment variables in init scripts
Sep 4, 2018
· release notes
Databricks 2.79 adds DB_DRIVER_IP, DB_INSTANCE_TYPE, DB_PYTHON_VERSION, and DB_IS_JOB_CLUSTER environment variables to cluster-scoped init scripts
DB_DRIVER_IP is now available in cluster-scoped init scripts, providing the driver node IP address. Not checked yet DB_INSTANCE_TYPE is now available in cluster-scoped init scripts, indicating the Spark node instance type. Not checked yet DB_PYTHON_VERSION is now available in cluster-scoped init scripts, showing the Python version used on the cluster. Not checked yet DB_IS_JOB_CLUSTER is now available in cluster-scoped init scripts, indicating whether the cluster was created for a job. Not checked yet
4 details in Everything
EBS leaked volumes logging and deletion
Sep 4, 2018
· release notes
Databricks v2.79 adds logging and default deletion of leaked EBS EBS Amazon's block-storage service for EC2 instances, providing persistent network-attached volumes that survive instance restarts and can be snapshotted, giving cyber tools durable, scalable disk without managing physical drives. volumes for terminated clusters
Enables locating and deleting unattached EBS volumes left by premature instance termination via the KnowledgeBase article EBS Leaked Volumes . Not checked yet
1 detail in Everything
Databricks Workspace sidebar redesign for v2.79 (Aug 28 - Sep 4, 2018)
Adds horizontal scroll arrows and two-swimlane limit to the workspace file navigation Not checked yet Makes the default view on clicking Workspace or Home buttons show your own user file view Not checked yet Enables pinning the sidebar open to keep the main pane unobstructed Not checked yet
3 details in Everything
Libraries API supports Python wheel files
Aug 7, 2018
· release notes
Libraries API now supports installing Python wheel files with dependency handling.
Databricks introduces HorovodEstimator for distributed multi-GPU GPU A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. deep learning on Spark DataFrames.
Databricks Runtime 4.1 for Machine Learning (Beta)
May 18, 2018
· release notes
Databricks Runtime 4.1 ML adds distributed TensorFlow training via Horovod and pre-installed ML libraries.
Adds support for distributed TensorFlow training using Horovod in Databricks Runtime ML Beta Not checked yet Includes pre-installed popular machine learning libraries such as TensorFlow, Keras, and XGBoost Not checked yet
2 details in Everything
This release introduces an autostart capability for Databricks clusters. If a cluster is terminated, subsequent JDBC JDBC A Java standard API for connecting applications to relational databases, maintained as part of the Java SE platform, letting tools query any compliant database through a single consistent interface. /ODBC ODBC A standard API, maintained by Microsoft, for connecting applications to relational databases regardless of vendor. Cyber tools use it to query databases without writing engine-specific code. commands or assigned jobs will automatically trigger a restart. This enables uninterrupted scheduled execution without manual user action.
New AWS EC2 EC2 Amazon's cloud virtual-machine service, letting tools run on-demand servers without managing physical hardware. P3 instance types available for Databricks GPU GPU A graphics processing unit, a processor that performs many calculations in parallel. AI workloads use GPUs to speed up model training and inference. clusters.
Adds support for AWS EC2 P3 instance types on Databricks clusters Not checked yet Decreases the cost of clusters running on P2 instances Not checked yet
2 details in Everything
Databricks enhances the Spark experience in version 2.70 by introducing interactive error tips for troubleshooting.
Databricks now provides tips to help you interpret and troubleshoot many of the errors you might see when you run Spark commands. Not checked yet
1 detail in Everything
Clusters API: added UPSIZE_COMPLETED event type
May 1, 2018
· release notes
Databricks Clusters API adds UPSIZE_COMPLETED event type to signal node addition completion.
New customers can now host their DBFS Workspace root in their own AWS S3 bucket, moving storage off Databricks-managed infrastructure.
Adds support for hosting DBFS (Workspace) root in a user-provided AWS S3 bucket for new customers Not checked yet
1 detail in Everything
New line chart supports time-series data
Mar 6, 2018
· release notes
New line chart with time-series support replaces deprecated legacy option
Adds a new line chart that fully supports time-series data Not checked yet
1 detail in Everything
Databricks adds ML Model Export to save Spark models and pipelines for external use.
Introduces ML Model Export functionality allowing export of models and full ML pipelines from Apache Spark for scoring on other platforms Not checked yet Releases MLeap ML model export capability targeting low-latency, lightweight ML-powered applications Not checked yet
2 details in Everything
Databricks CLI 0.5.0 enables direct interaction with the Libraries API and simplifies managing credentials for multiple Databricks deployments through config-based profiles.
Apache Airflow 1.9.0 includes Databricks integration
Jan 2, 2018
· release notes
Databricks integration moves from preview to public in Airflow 1.9.0.
Databricks tasks are now publicly available for triggering Databricks runs in Airflow DAGs. Not checked yet
1 detail in Everything