Databricks Unveils Predictive AI Functions in Beta

Databricks has introduced a suite of five native SQL capabilities called Predictive AI Functions, currently available in beta. The update features an enhanced version of ai_forecast alongside four entirely new tools: ai_top_drivers, ai_detect_anomalies, ai_predict_value, and ai_predict_class. Designed to bridge the gap between business analysts and data science infrastructure, these SQL-based functions run advanced predictive workloads—such as time-series forecasting, regression, classification, and driver analysis—directly within dashboards, SQL environments, and the natural-language interface Genie.

The upgraded ai_forecast tool utilizes a transformer-based time series foundation model that comes pretrained on diverse temporal data patterns. This foundation architecture automatically captures trends and seasonality without requiring custom model tuning. In addition to basic projections, the function incorporates built-in holiday calendars to account for localized demand fluctuations and allows practitioners to supply external covariates like pricing shifts or marketing budgets. Databricks tested the upgraded forecasting model using the fev-bench benchmark, measuring task win rate via scaled quantile loss against traditional cloud data warehouse offerings to demonstrate lower probabilistic error rates.

The remaining tools target ad-hoc analytical tasks across massive datasets. Analysts can run ai_detect_anomalies over temporal tables to highlight unexpected data points, or deploy ai_predict_value and ai_predict_class to handle continuous regression and categorical predictions across millions of rows. Meanwhile, ai_top_drivers breaks down which business segments contributed most to metric changes, as demonstrated on the NYC taxi trips dataset to isolate monthly revenue shifts. While all functions operate natively in SQL and dashboards, ai_top_drivers is already integrated into Databricks' Genie interface for plain-language querying, with the remaining four functions slated to arrive on Genie shortly.

By centralizing and managing these underlying machine learning models on the platform, Databricks eliminates the need for data teams to build custom pipelines or configure dedicated machine learning infrastructure. For practitioners, this shift compresses time-to-insight for routine forecasting and diagnostic queries, allowing business units to execute complex statistical analyses directly inside their existing reporting pipelines.