by Braden Kelley and Art Inteligencia
Every systems migration I’ve ever seen gets sold internally with some version of the same reassurance: “nothing will change for the customer.” I understand why people say it — it’s meant to calm nerves and keep the project moving. It’s also almost never entirely true, and the gap between that promise and reality is exactly where a lot of quiet, expensive customer experience damage happens.
Migrations are measured by the wrong success criteria
A CRM cutover, a new billing platform, a support tooling replacement — these get judged by IT and project management on a specific set of criteria: did the data migrate correctly, is uptime where it should be, did the go-live happen on schedule. Those are the right questions for a systems team to ask. They’re the wrong questions, or at least an incomplete set, for understanding whether the customer experience survived the transition intact. A migration can hit every technical success metric on the project plan and still quietly degrade the actual experience a customer has, because nobody on the technical side was specifically measuring for that.
The silent regression problem
The most expensive migration failures I’ve seen aren’t the dramatic ones — the outage, the data loss, the system that won’t come back up. Those get noticed immediately and fixed fast, precisely because they’re loud. The expensive ones are silent regressions: a new billing platform that technically works but changes the invoice format customers had built internal processes around, so now their AP department has to redo reconciliation manually every month. A new CRM that migrates the data correctly but changes how support reps see a customer’s history, so reps start asking questions customers have already answered before, over and over, without anyone flagging it as a problem because each individual instance looks like a minor inconvenience rather than a pattern.
None of that shows up in a migration status report. All of it shows up, eventually, in retention numbers that took a hit for reasons nobody traced back to the systems change that happened two quarters earlier.
Why you need a real baseline before you touch anything
You can’t know what changed for customers after a migration if you don’t have an honest picture of what their experience actually looked like before it — not the technical specs of the old system, but the lived experience of using it. That means validated personas, a current journey map across the specific touchpoints the migration will touch, and a firsthand walkthrough of those touchpoints as they exist today. This is the step migrations skip most often, because the old system is about to be replaced anyway and it feels like wasted effort to study something on its way out. It isn’t wasted — it’s the only way to tell the difference between “the migration caused this” and “this was already broken and we just never noticed.”
What to actually check after cutover
Once the new system is live, the audit isn’t finished — it shifts to verification. This means walking the same touchpoints again, deliberately, rather than assuming success because no one’s complained yet. Complaints are a lagging indicator; most customers adapt to a worse experience quietly before they ever formally report it. It also means comparing the same data points — response times, resolution rates, whatever mattered in the baseline — against the pre-migration numbers specifically, not just against general benchmarks, since the question that matters is whether this specific change made things better, worse, or invisible.
The teams that get this right treat it as one audit, not two
The most effective version of this isn’t a rushed pre-migration check plus a separate, disconnected post-migration review commissioned only if something goes wrong. It’s a single audit designed from the start to bracket the migration — same personas, same touchpoints, same evaluation criteria, measured before and after, so the comparison is actually apples to apples. That structure is also what makes the business case afterward credible: “here’s exactly what improved, what regressed, and what stayed flat” is a far stronger position than a vague sense that the migration “went fine.”
Where to start
If you have a systems migration coming up and want a credible before-picture while there’s still time to establish one, a Customer Experience Audit scoped to bracket the migration is exactly the right tool — and if you’re trying to build the case for why that baseline is worth the investment before the project timeline locks in, the CX ROI Calculator is a fast way to put a number on what a silent regression could actually cost you.
Image Credits: Pixabay
Content Authenticity Statement: The topic area, key elements to focus on, etc. were decisions made by Braden Kelley, with a little help from Claude to clean up the article.
Sign up here to get Human-Centered Change & Innovation Weekly delivered to your inbox every week.