Writing
Part 7: Building AI Observability and Adoption Programs
Buying AI tools is easy. Changing how an engineering organization works is the part that actually takes effort, and it doesn't happen because someone sends an enthusiastic all-hands email. Adoption has to be run like a product: instrumented, supported, taught, iterated, and connected to outcomes someone can point at.
See the gradient
The first thing the data shows you — this picks up directly from the traces in Part 3 — is how wide the gap is between shallow and deep usage. Some engineers use AI as a slightly better search box. Others have quietly redesigned entire workflows around it. License dashboards make these two groups look identical. The adoption program exists to see the difference, and then to help more teams climb the gradient instead of hoping they figure it out independently.
The deep users are the most valuable resource in the whole program, and most organizations waste them. They've already discovered, through trial and error on company time, which patterns actually work inside this company — with these codebases, these policies, these tools. The program's job is to find them and turn their local hacks into supported platform capabilities, instead of leaving the same discoveries to be made independently by forty teams.
Run it like reliability work
The operating cadence that works looks a lot like reliability engineering. Review the traces. Find the bottlenecks — missing tools, missing context, permission blockers, policy nobody has clarified. Fix the platform. Publish the pattern. Repeat. Code labs and internal demos work much better when they teach workflows that already succeeded down the hall, rather than vendor demos that succeeded in a video.
What to measure follows from that cadence. For each workflow, where it sits on the spectrum from manual to assisted to staged-agent to automatic. Whether quality moved: reverts, defects, review burden, incident rate. Whether cycle time moved: time to first draft, to review, to merge, to launch. And what's blocking the next step. Notice the absence of "total tokens consumed" — token counts measure enthusiasm, not value, and they're embarrassingly easy to inflate.
One caution from watching this go wrong: the goal is not to force everyone to use AI, and mandate-flavored programs tend to generate exactly the shallow usage their dashboards then proudly report. The goal is to find where AI genuinely changes the production function and make that change repeatable. Some workflows won't benefit yet. Saying so out loud buys credibility for the ones that do.
AI adoption isn't a launch announcement. It's an operating program, and it compounds for the organizations willing to learn from their own best users.
This is Part 7, the final part of the AI-for-Work series. There's also a companion essay on the mindset underneath it all: When "Move Fast" Meets "Move Safely".