Databricks Blog - Software

The key to production AI agents: Evaluations | Databricks Blog

Discover why generic metrics often fail in enterprise environments, what effective evaluation systems require and how to create continuous optimization that builds user trust.

Databricks on Databricks: Scaling Database Reliability | Databricks Blog

Databricks engineers share how they scaled database reliability using big data analytics tools like Databricks, shifting from reactive monitoring to proactive scoring to enforce best practices!

Customizing Databricks Assistant with Instructions | Databricks Blog

Persist your preferences in Databricks Assistant with Instructions—teach it once, and it remembers your code style, libraries, and domain context.

High Performance Ratelimiting at Databricks | Databricks Blog

An Engineering Blog Post that will deep dive into how Databricks revamped its Ratelimiting Architecture.

Automatic Identity Management for Entra ID is now Generally Available in Azure Databricks | Databricks Blog

Automatic Iden

The New Investment Regime Is Here. Are You Ready for It? | Databricks Blog

Discover how structural shifts and thematic forces are reshaping investing, and learn why modern, integrated data platforms can give investors a lasting edge.

How Observability in Lakeflow Helps You Build Reliable Data Pipelines | Databricks Blog

Observability capabilities in Lakeflow help monitor, troubleshoot, and optimize workflows at scale, for more reliable, healthy, and fresh data pipelines.

AI as a conduit for management cybernetics | Databricks Blog

How AI can support management cybernetics to increase accountability, do variety engineering and improve decision making at all levels of an organization.

How Kythera Labs, a Databricks Built-On Partner, saves $2M+/year using Delta Sharing | Databricks Blog

See how Kythera Labs uses Delta Sharing to cut storage 80%, save $4M, and share 300M patient records securely for faster healthcare insights.

From Search to Sale: How AI Is Redefining Customer Engagement and Loyalty in Retail | Databricks Blog

AI transforms retail by powering real-time personalization, smarter search, and scalable service—driving loyalty and revenue across the journey.

Now GA: Share Materialized Views and Streaming Tables with Delta Sharing | Databricks Blog

Share real-time streaming tables and materialized views and reduce costs - now in GA

Introducing Databricks Assistant Data Science Agent | Databricks Blog

Introducing the Data Science Agent for Databricks Assistant, an autonomous partner to accelerate data science in notebooks and the SQL Editor.

Architecting a High-Concurrency, Low-Latency Data Warehouse on Databricks That Scales | Databricks Blog

Discover how the Databricks Data Intelligence Platform enables scalable, high-concurrency, low-latency data warehousing for rapid analytics, BI reporting, AI use cases and all other data needs.

How to use Lakebase as a transactional data layer for Databricks Apps | Databricks Blog

Int

The New Foundation for Enterprise AI | Databricks Blog

Databricks + SAP unify enterprise data, breaking down silos to power AI, streamline governance, cut costs, and accelerate business impact.

Cracking Complex Contracts with GenAI on Azure Databricks | Databricks Blog

Working with Advancing Analytics and Databricks, a leading life sciences organization implemented an innovative Generative AI solution that has transformed their contract analysis process, delivering remarkable efficiency gains and business insights. Learn how they did it.

A Unified Approach to Data Exfiltration Protection on Databricks | Databricks Blog

Comprehensive framework and proven strategies for mitigating unauthorized data movement on Databricks.

Delivering Agentic BI: How to Unify Infrastructure, Data and Semantics | Databricks Blog

Agentic BI requires data intelligence, when AI understands the company’s data and its unique business concepts to truly unlock self-sufficiency and turbocharge productivity.

From Booking to Bon Voyage: How AI Is Redefining the Travel & Hospitality Experience | Databricks Blog

Databricks helps travel brands deliver personalized, real-time guest experiences, boosting loyalty, revenue, and operational efficiency.

Tecton is Joining Databricks to Power Real-Time Data for Personalized AI Agents | Databricks Blog

Tecton is joining Databricks to deliver fast, reliable real-time data for AI agents.

Introducing Real-Time Mode in Apache Spark™ Structured Streaming | Databricks Blog

Real-time mode in Spark Structured Streaming delivers millisecond latency for fraud detection, recommendations, and more - now in Public Preview on Databricks.

Introducing SQL Stored Procedures in Databricks | Databricks Blog

Mana

Reranking in Mosaic AI Vector Search for Faster, Smarter Retrieval in RAG Agents | Databricks Blog

Boost RAG agent quality with reranking—deliver more relevant answers in less time with a single parameter in Mosaic AI Vector Search.

Reverse ETL with Lakebase: Activate your lakehouse data for operational analytics | Databricks Blog

Introduction: Analy

Securing the Future: How Databricks Powers Financial Services | Databricks Blog

Discover how leading banks and financial institutions use the Databricks Data Intelligence Platform to strengthen cybersecurity, simplify compliance, and future-proof operations. Download the ebook to unlock actionable insights, best practices, and real-world outcomes.

Processing Millions of Events from Thousands of Aircraft with One Declarative Pipeline | Databricks Blog

Ingest live aircraft data at scale using custom PySpark data sources. Transform complex streaming data engineering into simple SQL with Databricks Lakeflow Declarative Pipelines.

How AI is Transforming the Way Retailers Connect With Consumers | Databricks Blog

Discover how top retailers like adidas, Pilot, and PRADA use Databricks to personalize experiences and optimize operations.

Announcing Public Preview of Request for Access in Unity Catalog | Databricks Blog

This blog post will cover the new Request for Access feature and how admins can get started by setting up access request destinations for their datasets!

Optimizing Materialized Views Recomputes | Databricks Blog

Optimize Materialized View recomputes in Databricks. Learn to detect and fix full recomputes, improve incremental refreshes, and reduce costs with best practices.

Teaching AI to Fix Your Code: My Summer Improving Quick Fix at Databricks | Databricks Blog

How I helped build a smarter code assistant using best-of-k sampling and reward models—plus what I learned about AI, systems, and myself along the way.

7x Faster Medical Image Ingestion with Python Data Source API | Databricks Blog

It has always been a challenge to ingest compressed files at scale. Zipfiles are infamously “unsplittable”; however, we will show this simply isn’t true. Ingest your archive of zipfiles at scale now with a user-defined Python Data Source.

Build with GPT-5 on Databricks with AI Gateway | Databricks Blog

Get started with GPT-5 and gpt-oss on Databricks today. Govern usage, manage costs, and monitor quality with Mosaic AI Gateway.

Agent Learning from Human Feedback (ALHF): A Databricks Knowledge Assistant Case Study | Databricks Blog

Discover how Agent Learning from Human Feedback (ALHF) powers Databricks Agent Bricks, enabling AI agents like Knowledge Assistant to adapt to expert expectations with minimal natural language feedback—boosting response quality and enterprise readiness.

Personalizing Players' Experiences with Recommendation Systems | Databricks Blog

See how in-game recommenders boost player expression, save time, and guide progression, creating value for developers beyond monetization tactics.

Introducing OpenAI’s New Open Models on Databricks | Databricks Blog

Databricks partners with OpenAI to launch gpt-oss 20B & 120B, high-performance open models for reasoning, tool use, and enterprise AI. Customize, fine-tune, and securely deploy AI agents on your data.

Announcing Databricks Vancouver Development Center | Databricks Blog

Databricks launches its Vancouver R&D hub to drive innovation in AI/BI, real-time analytics, and next-gen enterprise data applications.

What’s New in AI/BI - July 2025 Roundup | Databricks Blog

The Power of RLVR: Training a Leading SQL Reasoning Model on Databricks | Databricks Blog

Discover how Databricks used RLVR to train a state-of-the-art SQL reasoning model—beating proprietary LLMs on BIRD using only open models and public data.

Continuous Environmental Monitoring Using the New transformWithState API | Databricks Blog

Learn how to build operational monitoring pipelines using Spark 4.0’s new transformWithState API with advanced state and TTL support.

From Chaos to Control: A Cost Maturity Journey with Databricks | Databricks Blog

Control Databricks costs! Learn strategies for cost maturity, architectural considerations, and technical solutions. Optimize spending and performance.