GUEST POST from Arlen Meyers
It’s pothole season so be careful when you are driving. If you hit a pothole, it might throw your front end out of alignment
When your front end is out of alignment, it means the angles of your wheels are no longer set to the manufacturer’s specifications. This affects how the wheels contact the road, which can change your vehicle’s handling, tire wear, and fuel efficiency.
Similarly, it’s AI season so be sure it’s aligned too.
“We must eliminate programs and activities that do not fully align with our strategy and priorities, even those we have had for many years. Administrative savings alone will not be enough,” said the email signed by Dr. Michael Brown, president of the academic medical center, CFO Niyum Gandhi and COO Dr. Ron Walls, the Boston Business Journal reported.
Artificial intelligence (AI) alignment is the process of encoding human values and goals into AI models to make them as helpful, safe and reliable as possible.
In sick care, the sextuple aims are quality, cost, equitable access, patient experience, sick care professional experience, and productivity. Most of it now has just improved shareholder value.
Society is increasingly reliant on AI technologies to help make decisions. But this growing reliance comes with risk: AI models can produce biased, harmful and inaccurate outputs that are not aligned with their creators’ goals and original intent for the system.
Alignment works to reduce these side effects, helping ensure AI systems behave as expected and in line with human values and goals. For example, if you ask a generative AI chatbot how to build a weapon, it can respond with instructions or it can refuse to disclose dangerous information. The model’s response depends on how its creators aligned it.
AI front end misalignment can cost you, patients and stakeholders.
Misalignment in medicine usually isn’t dramatic. It happens when a system optimizes for something that is not quite what clinicians and patients care about, and the gap shows up as quiet, systematic harm. Like bankruptcy, it happens gradually and all of a sudden.
The proxy problem. The best-known case is a widely used risk algorithm studied by Obermeyer and colleagues in 2019. It predicted future healthcare costs as a stand-in for health needs. Because Black patients historically had less money spent on them at the same level of illness, the model rated them as healthier than equally sick white patients and routed fewer of them into extra-care programs. The model did exactly what it was told. The objective was the problem.
Performing well in testing but not in practice. Models trained on one hospital’s data often degrade at another because of different patient populations, equipment, or documentation habits. An external evaluation of Epic’s sepsis prediction model found it missed many cases and generated heavy alert volume, which contributes to alert fatigue. Dermatology and imaging tools trained mostly on lighter skin or a narrow set of scanners can underperform for everyone else.
Human-AI interaction failures. Clinicians can over-trust a confident output (automation bias) or learn to ignore a noisy one. Over time, reliance on AI can also erode the skills needed to catch its errors. When something goes wrong, accountability among the developer, the hospital, and the clinician is often unclear.
Generative AI specific risks. Language models can fabricate details in a clinical summary, invent citations, agree with a patient’s or clinician’s mistaken premise (sycophancy), or give plausible but unsafe advice. Documentation tools can also drift toward whatever maximizes billing or throughput rather than accuracy.
Feedback loops. If a model shapes who gets screened or treated, future training data reflects the model’s own decisions, which can lock in and amplify existing disparities.
What helps. Choosing outcome measures that reflect actual patient benefit, validating locally before deployment, monitoring performance continuously by subgroup, keeping a meaningful human in the loop, being transparent about limitations, and having clear regulatory and liability pathways. For entrepreneurs, this is increasingly a commercial issue as well as an ethical one, since buyers and regulators are asking for this evidence.
What is the ROI on AI? When was the last time it reduced your sick care insurance premiums, the cost of the last prescription your filled or your out-of-pocket costs incurred when you went to the urgent care center with symptoms of the flu?
AI defensive medicine is making things worse. The care these systems recommended cost substantially more than physicians’ documented care, chiefly because they suggested testing at visits where the physician had ordered none. LLM outputs are suggestions, not plans of care; final decisions should remain with clinicians.
Artificial intelligence has produced “zero financial payoff” to U.S. companies outside of the tech sector, argues Apollo Chief Economist Torsten Slok. Slok claims that while revenue per worker and profit margins have risen for the “Magnificent Seven,” both have stagnated across the S&P 493 — with no clear indication of AI-driven productivity growth. “Mainstream corporate America is pulling back and starting to view token spending not as a source of growth but as a cost center that weighs on margins,” Slok said.
As noted in a recent WSJ article, perhaps we are experiencing a kind of Solow Paradox 2.0, with the digital age more around us than ever except in the productivity statistics. There are several reasons for this lag. First of all, we’re in the early deployment years of major recent innovations, including cloud computing, IoT, big data and analytics, robotics, and AI and machine learning. Notes McKinsey: “The challenge of adoption in the current digital wave may be even harder because of the broad range of uses of digital that not only help improve current processes but fundamentally transform business models and operations.”
How to make doctors more productive | LinkedIn
Dr Wachter reminds us: as if on cue, his post last week on AI alignment in healthcare just received some empirical meat to put on its bones. His argument, in brief, was this: the critical alignment question for healthcare AI won’t be how to prevent rogue AI from causing harm. It will be about striking the right balance between cost and quality in AI-enabled decision support and, increasingly, agentic systems.
In other words, How to Survive Being the Human in the Poop: A Field Guide.
Two common C-suite behaviors can stall AI initiatives: claiming sole control over decisions or disengaging from those seen as outside one’s expertise. Because AI crosses functional boundaries, executives must learn to share authority, resources, and accountability rather than rely on the domain-specific ownership that helped them advance. CEOs can foster this collaborative approach by: defining the business problem before assigning ownership, building cross-functional relationships before they’re needed, treating early AI plans as experiments, rewarding jointly owned outcomes, and establishing rules for resolving disagreements.
Compassionate capitalism should guide your way to stay in alignment.
Slow down, drive carefully and don’t ignore the dashboard warning lights.
Image Credit: Pixabay
Sign up here to get Human-Centered Change & Innovation Weekly delivered to your inbox every week.