Blog
Our senior engineers build data platforms, AI agents and long-running systems alongside our clients' own teams, and they write up what happened when those systems met production, including what broke and what we'd do the same way again. If an AI pilot of yours never got past the demo, or a data project is waiting on headcount, you'll find the same problems here and how our engineers dealt with them.
Short takes on what Databricks just shipped and what we read into it, published on the Field Guide.
Files API 3.0 in Beta adds multipart uploads and ETag conditional writes to Unity Catalog volumes. What we would use first, and the network work.
Read on the Field GuideAsync processing for Python transformWithState is GA on Databricks Runtime 19. When we would convert a stateful processor, and what to check first.
Read on the Field GuideDatabricks serverless job tasks can now read their environment from pyproject.toml, in Beta. Why we would treat uv.lock as part of the job.
Read on the Field Guide117 posts
Lenders ask whether AI agent controls come first or after the build. How AutoFabric puts risk, compliance and team ownership into each action on Databricks.
Most lenders' fraud checks and examiner answers run on last night's loan data. How Zerobus Ingest gets it there in seconds, and where Kafka still fits.
WebMCP doesn't replace MCP. Using our own techfabric.com MCP server, we show which agent tasks belong on a server and which belong in an open web page.
Security reviews stall when nobody can say who reads which table, file or model. How Unity Catalog put all three under one set of grants at Location Services.
An agent that works for ninety seconds in a demo meets rate limits, crashes, deploys and overnight waits in production. How Temporal and Databricks handle each.
CustomerLake builds Customer 360 profiles and audiences on data already in Databricks. What's in Beta, and the workloads we'd move to it first.
A backdated endorsement can erase what an insurer's data showed before it. We built a Databricks pipeline that keeps both answers for audits and claims.
Claude Sonnet and Opus 5.5 on Databricks return HTTP 400 for a response_format request. Why we moved Fabric's schemas into the prompt, and what that costs.
A coding agent reported 33 of 41 test tasks successful while the three packages it changed never ran. The checks that would have caught it.
A test can catch a broken data rule and the release still goes out green. We reproduced that false success and built a runner that stops it.
When a process stops halfway, the cost shows up weeks later. How we keep business processes and AI agents finishing their work with Temporal and Databricks.
When each engineer's coding agent reads its own copy of the team's rules, the agents drift apart. How we keep one reviewed set of rules for every agent.
A retried Databricks job can double-count a report without failing anything. The backend habits that prevent it, and how our test kit fails jobs on purpose.
How to tell whether a model trained on your own data beats the one in production, what Databricks AI Runtime changes, and when training is worth it.
AI agents need access to loan systems, with limits and a record of all they do. How we built that into AutoFabric, our auto refinance platform.
Engineers who solve the problems rarely have time to write them up. We built an MCP server so they publish from their coding agent, under the site's checks.
A data quality suite can pass every day and still let a duplicate order into the revenue report. We fed our rules twelve defects to find which got through.
Five client reports as five embedded dashboards meant a new iframe and embed token on every switch. Why we merged them into one AI/BI dashboard, and the cost.
ElevenLabs' Architect drafts and tests fixes for a voice agent. What our own ElevenLabs test calls taught us to check before trusting one.
A failed Databricks quality check doesn't stop bad data reaching reports on its own. We designed a release gate that does, and tested it on six cases.
An agent's memory is a write path into production. How we gate memory writes in Fabric, and how we'd carry that to Databricks agent memory.
Cross-tenant customer-managed keys let a customer hold the key to their data in a provider's Azure Postgres app. What saying yes costs, and how we'd build it.
Azure's built-in PostgreSQL backups stop at 35 days and outlive a deleted server by five days. Which production servers we'd give a Backup vault, and the cost.
Azure Databricks' private network gateway lets serverless reach private databases through one subnet. What it changes for a migration, and what we'd test first.
Gemini 4 Argon's price doubles on December 1. We price a model by what a finished agent task costs, and in our own replay the cheapest model finished none.
Enterprise customers worry about AI agents going rogue. The limits we put on our own agents, and how Databricks and Temporal would carry them to clients.
An agent that only answers when prompted leaves new events waiting. How we built AutoFabric's agents to act on events safely, and what MCP Events changes.
Agent 365 with API Management lets a company decide in one place which tools its AI agents may use. What it covers, set against our own MCP server.
We tried reading our own Databricks tables from DuckDB and none were readable. What that means for sharing lakehouse data with partners and AI agents.
Meta's Muse Wonder says it checks in before acting on anything important. What that would and wouldn't change for the approval gates in agents we build.
Responsive, the SlateDB team, is joining Databricks to work on Lakebase. The two Lakebase limits that shape what we build, and which the news could move.
Coding agents' token bills keep growing. What OpenCode 2.0 changes, and the config we'd commit to route routine work to cheaper open-weight models.
Microsoft announced Business Central mirroring of any table into Fabric, with no export to maintain. Where it fits a Databricks build, and where it stops.
AI agents need search and reading that are cheap and don't break. Why ours switched to Cloudflare's web search and web fetch, and what it fixed.
An AI agent that reads links from email or news feeds can be pointed at private addresses. How we closed the main path once, in the framework our agents run on.
Our 81 city pages held through Google's September 2026 spam update while our agent notes fell out. What kept them indexed, and the checks AI drafts pass.
We built a sandbox by hand for the AI agent that repairs our website. Microsoft Execution Containers declare that boundary as a policy. What we'd move onto it.
Approving a claim or routing a ticket is a call software repeats all day. Why Decision-1 checks our posts first, and how it compares with Jev, Clef and Kev.
Alibaba Cloud's SkillFS controls which skills an AI agent can load. What our own writing agent's two-day stall taught us about the same problem.
BC29 lets one Business Central item carry several excise duties. What it changes for the item records, period close and reporting in the systems we build.
Funke, announced by Databricks on 8 October, parses HL7v2 into Unity Catalog intact. What it changes for clinical pipelines, and what it leaves alone.
A client's embedded Databricks reports sent one query to their app database five times per filter click. How we cut that to once, and what the rule costs.
Replit Agent now deploys apps inside Databricks with a Lakebase database behind each. What we'd decide before anyone prompts an internal tool into existence.
Supermemory remembers each person an AI agent talks to. Where we'd use it beside the reviewed memory our own agents keep, and the defaults we'd change.
Databricks bought Row Zero, a spreadsheet that reads the lakehouse live. What it changes for month-end exports out of Business Central, and what hasn't shipped.
Software that approves claims or routes tickets needs every call cheap and repeatable. Where Clef-flash, Clef and Clef-omni each fit, and what each one costs.
When an app's database and the lakehouse disagree, so do the reports. What we measured building on Databricks Lakebase, and which databases we'd move first.
BC29 keeps extensions published from VS Code installed when a sandbox updates. What that changes for UAT on the Business Central systems we extend.
Each Business Central index slows every write to its table, in every company. BC29 lets AL code switch indexes per company. How we would use it, and where not.
A shared test database lets one change break another and holds up releases. How we give each test run its own Lakebase copy, and how we'd do it for AI agents.
When a refinance file stalls on one missing detail, an AI phone call can get it in minutes. How we built calls a lender can prove were allowed, now in a demo.
How we move Synapse and Snowflake estates onto Databricks with Lakebridge and Airlift. One manufacturing migration took six weeks instead of six months.
GPT-6's three models and its effort setting decide most of what an AI workload costs. How we'd route work across Astra, Sol and Luna, and run it on Databricks.
Meta's ads MCP server is in Databricks Marketplace. What it lets an agent do, how Unity Gateway and Meta rules fence it, and where it does not fit.
Business Central agents could only read a few standard tables without custom development. Version 29.0 lets them query the rest. How it works, where it stops.
Choosing a managed Postgres for apps beside your data platform? Lakebase against Snowflake Postgres on branching, scale to zero and governance. Lakebase wins.
Customers asking for a SOC 2 report? Preparing for SOC 2 Type II, we found the controls were already running and proving them was the work.
An AI agent can show records a person isn't cleared to see with every row filter working. Four FHIR scenarios on Databricks, with the performance tradeoffs.
Lenders kick back refinance packages over a mismatched odometer or an expired payoff. Where decision models like Jev and Clef catch it, logged on Databricks.
Three ways to report on Dynamics 365 Business Central, what each one is good for, the limits Microsoft documents, and how to tell which one you need.
The EU moved its high-risk AI deadline to December 2027. What NIST AI RMF, ISO 42001 and the EU AI Act ask, mapped to controls you can run on Databricks.
An Apache-2.0 .NET kit that resolves the customer tenant before a request reaches Databricks SQL, Jobs, AI/BI, Genie or Model Serving.
One ticket, one platform, and the cheapest first step. Moving a B2B product's client reporting onto Databricks, starting with Lakehouse Federation.
Microsoft's extended support for Dynamics NAV 2018 ends on 12 January 2028. What moving to Business Central involves, and why the warehouse floor sets the size.
Prove It, by TechFabric engineer Ivan Vydrin, won in the Creativity & Originality track of Databricks' global Genie-Powered App Challenge, judged by Databricks.
Microsoft and Databricks shipped the interop this year. For teams already on Azure the question is which platform holds the tables and which one reads them.
Snowflake and Databricks now read and write each other's Iceberg tables. If a vendor is still selling you a migration on lock-in, they are selling you 2023.
Export to Data Lake stops permanently on 30 November 2026. Microsoft names two replacements for Dynamics 365 teams. There is a third answer.
Staging has ten thousand rows, production has forty million, and every migration is a guess. What Lakebase branching does to a team's calendar.
Databricks' smart routing sends routine coding-agent work to cheaper models and keeps the agents' traces in Unity Catalog. What it saves, and what to check.
Every job green, nothing failing, and the numbers quietly wrong because a source stopped sending. Freshness, volume and schema checks on system tables.
Genie fails when two teams mean different things by the same word. Genie Accuracy hardens the semantic layer. TechFabric Experiments keeps the suite running.
Warehouse migrations slip because the exclusions were never written down. The Migration Readiness Sprint inventories the estate before anyone picks a date.
A Databricks cost dashboard shows spend. A two-week Health Check finds the spend that bought nothing, from all-purpose clusters to orphaned schedules.
Lakehouse pilots skip grants and then cannot be handed to security. The Lakehouse Launchpad puts Unity Catalog first, with TechFabric Airlift underneath.
A team shipped an agent in nine days and argued for two months about whether its answers were right. Write down what a good answer is first.
Moving QA off ad-hoc notebooks and onto version-controlled test automation with Databricks Repos. Repo structure, modular tests, and how a team works in one.
Writing PyTest data quality checks inside Databricks notebooks. Bronze-to-silver validation, edge cases, fixtures and error handling, with examples.
AI governance fails when it lives in documents instead of infrastructure. Compliance comes from systems that enforce the rules while the model is running.
Enterprise software fails when IT mandates tools people do not want. Adoption comes from pull, and the same rule now applies to agents.
Companies spend millions on enterprise AI that employees barely touch while shadow AI spreads. The problem is the tools, not the people ignoring them.
AI changed build versus buy. Now it is changing delivery. Small teams of senior engineers embedded in your operations, shipping in weeks rather than months.
Agile answered expensive execution, not planning. As AI collapses build time, deep upfront specification plus fast execution ships in weeks.
The engineering patterns that keep an autonomous system reliable in production. Guardrails, observability, circuit breakers, and where each one belongs.
Faster models and larger context windows make a custom AI build viable in weeks, and data sovereignty exposes vendor limits. When to buy, build, or do both.
When onboarding smaller customers pulls engineers off the roadmap, build a self-serve way in and leave the core platform alone. How to find where the cost sits.
Cloudflare, AWS and Azure all went down in late 2025. What their postmortems show, and the restore drills that decide how long your own outage lasts.
DASF stands for Databricks AI Security Framework. Version 3.0 maps 97 AI risks to 73 controls. What it covers, and how to enforce those controls at runtime.
Who can see a row, who changed it, and is an agent held to a person's grants? A Unity Catalog Health Check finds where coverage stops halfway.
What Databricks AI/BI is: dashboards plus Genie, which answers plain-English questions with SQL. What changed in 2026, accuracy, and cost.
How data platforms went from warehouses to lakehouses to data intelligence, and the four platform decisions that are hard to undo later.
Warehouse migrations slip on scope, not conversion. Two weeks on TechFabric Airlift inventories the estate, agrees exclusions and hands you a wave plan.
Horizontal AI versus vertical AI. Why businesses need a shared AI layer across their systems, what goes in it, and when a single-app copilot still wins.
ASP.NET Core has very extensible logging interface. I provides an ILogger interface along with few default implementations that can be used to log data.
Azure AI Foundry is now Microsoft Foundry. What it is, how it relates to Azure, and the day-one decisions a CTO makes on projects, data location and cost.
Catch bugs before production by running tests automatically in Azure DevOps. Unit tests on CI, Selenium on release, Test Plans on demand, with the YAML.
Service Fabric is Microsoft's microservices orchestrator. Service discovery, fault tolerance and containers, set up for production rather than a demo.
Service Fabric's built-in reverse proxy exposes every microservice in the cluster or none. How Traefik lets you choose which services the world can reach.
Some tenants want a cheap shared setup, others an isolated one. How we keep LLM costs, vendor swaps and per-tenant deployments manageable with Temporal.
An AI assistant that calls several tools per question can fail halfway and lose the answer. How we made a RAG chatbot durable with Temporal, with the code.
How we designed Fiber Copilot to stand up chatbots and copilots on your own data, in your own cloud, with the parser, vector store and model you choose.
Off-the-shelf or custom CRM? Buy when your sales process is standard. Build when the workflow is your product. Most firms do best buying and extending.
The Dynamics 365 quote process state by state, with the product catalog it needs. Templates and PDFs, approvals in Power Automate, and quote to order.
What Databricks is, what changed between 2024 and 2026, how the bill works and how to read it, and the governance and lock-in questions to settle first.
Small automation requests swamping your IT team? How Power Automate takes them off engineers, and what to check first, from licences to AI Builder credits.
The AI worth building usually needs two departments' data. How Amazon and KLM joined theirs to cut waste, and how to find the same opportunity in your company.
How to make one test suite check every release in each environment, browser and screen size your customers use, from GitHub Actions, without rewrites.
Why companies migrate to Dynamics 365 CRM in 2026, from on-premises Dynamics CRM, Salesforce and ERP-only estates. The dates, the prices and the route.
Twelve things to get right when hiring Databricks specialists in-house, from people who have built the teams and know what they screen for.
Retries, idempotency and state across services used to take more time than the business logic. A senior developer on what changed after adopting Temporal.
A prompt can follow every rule and still get a confident, wrong answer from generative AI. One real prompt taken apart, fixed and tested before users see it.
Logistics robots are the largest category of service robot. What they need from your ERP and warehouse system, what Dynamics 365 offers, and what to fix first.
Weighing Temporal for workflows that must survive failures in production? What our engineers learned modelling ticket purchases, returns and delayed flights.
Cypress component testing catches a broken React form in a test that runs in about a second, long before QA. A working setup on Vite, and when to use it.
Durable execution for teams whose order, payment or onboarding flows break halfway. How Temporal records every step and survives crashes, and where it stops.
How organizations improve data visibility. Report off the database, add a read replica or build a warehouse, then use lineage to see how data changed.
Why an app's interface slows the team down as it grows, and how a design system keeps one change one edit, with tokens, shared components and CI checks.
