Tag Archives: Design Sprints

The Surprising Power of Business Experiments

Leave a reply

The Surprising Power of Business ExperimentsInterview with Stefan H. Thomke

I had the opportunity recently to interview fellow author Stefan H. Thomke, the William Barclay Harding Professor of Business Administration at Harvard Business School to talk with him about his new book Experimentation Works: The Surprising Power of Business Experiments, to explore the important role that experimentation plays in business and innovation.

1. Why is there a business experimentation imperative?

My book Experimentation Works is about how to continuously innovate through business experiments. Innovation is important because it drives profitable growth and creates shareholder value. But here is the dilemma: despite being awash in information coming from every direction, today’s managers operate in an uncertain world where they lack the right data to inform strategic and tactical decisions. Consequently, for better or worse, our actions tend to rely on experience, intuition, and beliefs. But this all too often doesn’t work. And all too often, we discover that ideas that are truly innovative go against our experience and assumptions, or the conventional wisdom. Whether it’s improving customer experiences, trying out new business models, or developing new products and services, even the most experienced managers are often wrong, whether they like it or not. The book introduces you to many of those people and their situations—and how business experiments raised their innovation game dramatically.

2. What makes a good business experiment, and what are some of the keys to successful experiment design?

In an ideal experiment, testers separate an independent variable (the presumed cause) from a dependent variable (the observed effect) while holding all other potential causes constant. They then manipulate the former to study changes in the latter. The manipulation, followed by careful observation and analysis, yields insight into the relationships between cause and effect, which ideally can be applied and tested in other settings. To obtain that kind of learning—and ensure that each experiment contains the right elements and yields better decisions—companies should ask themselves seven important questions: (1) Does the experiment have a testable hypothesis? (2) Have stakeholders made a commitment to abide by the results? (3) Is the experiment doable? (4) How can we ensure reliable results? (5) Do we understand cause and effect? (6) Have we gotten the most value out of the experiment? And finally, (7) Are experiments really driving our decisions? Although some of the questions seem obvious, many companies conduct tests without fully addressing them.

Here is a complete list of elements that you may find useful:

Hypothesis

  • Is the hypothesis rooted in observations, insights, or data?
  • Does the experiment focus on a testable management action under consideration?
  • Does it have measurable variables, and can it be shown to be false?
  • What do people hope to learn from the experiments?

Buy-in

  • What specific changes would be made on the basis of the results?
  • How will the organization ensure that the results aren’t ignored?
  • How does the experiment fit into the organization’s overall learning agenda and strategic priorities?

Feasibility

  • Does the experiment have a testable prediction?
  • What is the required sample size? Note: The sample size will depend on the expected effect (for example, a 5 percent increase in sales).
  • Can the organization feasibly conduct the experiment at the test locations for the required duration?

Reliability

  • What measures will be used to account for systemic bias, whether it’s conscious or unconscious?
  • Do the characteristics of the control group match those of the test group?
  • Can the experiment be conducted in either “blind” or “double-blind” fashion?
  • Have any remaining biases been eliminated through statistical analyses or other techniques?
  • Would others conducting the same test obtain similar results?

Causality

  • Did we capture all variables that might influence our metrics?
  • Can we link specific interventions to the observed effect?
  • What is the strength of the evidence? Correlations are merely suggestive of causality.
  • Are we comfortable taking action without evidence of causality?

Value

  • Has the organization considered a targeted rollout—that is, one that takes into account a proposed initiative’s effect on different customers, markets, and segments—to concentrate investments in areas when the potential payback is the highest?
  • Has the organization implemented only the components of an initiative with the highest return on investment?
  • Does the organization have a better understanding of what variables are causing what effects?

Decisions

  • Do we acknowledge that not every business decisions can or should be resolved by experiments? But everything that can be tested should be tested.
  • Are we using experimental evidence to add transparency to our decision-making process?

Experimentation Works3. Is there anything special about running online experiments?

In an A/B test, the experimenter sets up two experiences: the control (“A”) is usually the current system—considered the champion—and the treatment (“B”) is some modification that attempts to improve something—the challenger. Users are randomly assigned to the experiences, and key metrics are computed and compared. (A/B/C or A/B/n tests and multivariate tests, in contrast, assess more than one treatment or modifications of different variables at the same time.) Online, the modification could be a new feature, a change to the user interface (such as a new layout), a back-end change (such as an improvement to an algorithm that, say, recommends books at Amazon), or a different business model (such as an offer of free shipping). Whatever aspect of customer experiences companies care most about—be it sales, repeat usage, click-through rates, or time users spend on a site—they can use online A/B tests to learn how to optimize it. Any company that has at least a few thousand daily active users can conduct these tests. The ability to access large customer samples, to automatically collect huge amounts of data about user interactions on websites and apps, and to run concurrent experiments gives companies an unprecedented opportunity to evaluate many ideas quickly, with great precision, and at a negligible cost per additional experiment. Organizations can iterate rapidly, win fast, or fail fast and pivot. Indeed, product development itself is being transformed: all aspects of software—including user interfaces, security applications, and back-end changes—can now be subjected to A/B tests (technically, this is referred to as full stack experimentation).

4. What are some of the keys to building a culture of large-scale experimentation?

Shared behaviors, beliefs, and values (aka culture) are often an obstacle to running more experiments in companies. For every online experiment that succeeds, nearly 10 don’t—and in the eyes of many organizations that emphasize efficiency, predictability, and “winning,” those failures are wasteful. To successfully innovate, companies need to make experimentation an integral part of everyday life—even when budgets are tight. That means creating an environment in which employees’ curiosity is nurtured, data trumps opinion, anyone (not just people in R&D) can conduct or commission a test, all experiments are done ethically, and managers embrace a new model of leadership. More specifially, companies have addressed some of these obstacles in the following ways:

They Cultivate Curiosity

Everyone in the organization, from the leadership on down, needs to value surprises, despite the difficulty of assigning a dollar figure to them and the impossibility of predicting when and how often they’ll occur. When firms adopt this mindset, curiosity will prevail and people will see failures not as costly mistakes but as opportunities for learning. Many organizations are also too conservative about the nature and amount of experimentation. Overemphasizing the importance of successful experiments may inadvertently encourage employees to focus on familiar solutions or those that they already know will work and avoid testing ideas that they fear might fail.

They Insist That Data Trump Opinions

The empirical results of experiments must prevail when they clash with strong opinions, no matter whose opinions they are. But this is rare among most firms for an understandable reason: human nature. We tend to happily accept “good” results that confirm our biases but challenge and thoroughly investigate “bad” results that go against our assumptions. The remedy is to implement the changes experiments validate with few exceptions. Getting executives in the top ranks to abide by this rule is especially difficult. But it’s vital that they do: Nothing stalls innovation faster than a so-called HiPPO—highest-paid person’s opinion. Note that I’m not saying that all management decisions can or should be based on experiments. Some things are very difficult, if not impossible, to conduct tests on—for example, strategic calls on whether to acquire a company. But if everything that can be tested online is tested, experiments can become instrumental to management decisions and fuel healthy debates.

They Embrace a Different Leadership Model

If most decisions are made through experiments, what’s left for managers to do, beyond developing the company’s strategic direction and tackling big decisions such as which acquisitions to make? There are at least three things:
Set a grand challenge that can be broken into testable hypotheses and key performance metrics. Employees need to see how their experiments support an overall strategic goal.

Put in place systems, resources, and organizational designs that allow for large-scale experimentation. Scientifically testing nearly every idea requires infrastructure: instrumentation, data pipelines, and data scientists. Several third-party tools and services make it easy to try experiments, but to scale things up, senior leaders must tightly integrate the testing capability into company processes.

Be a role model. Leaders have to live by the same rules as everyone else and subject their own ideas to tests. Bosses ought to display intellectual humility and be unafraid to admit, “I don’t know…” They should heed the advice of Francis Bacon, the forefather of the scientific method: “If a man will begin with certainties, he shall end in doubts; but if he will be content to begin with doubts, he shall end in certainties.”

Continue reading the article on InnovationManagement.se


Accelerate your change and transformation success

Subscribe to Human-Centered Change & Innovation WeeklySign up here to get Human-Centered Change & Innovation Weekly delivered to your inbox every week.

Design Sprints for Culture

Leave a reply

Rapidly Prototyping Your Work Environment

Design Sprints for Culture

GUEST POST from Chateau G Pato
LAST UPDATED: January 12, 2026 at 11:53AM

We often talk about Design Sprints in the context of products, features, or services. Teams huddle for five days, brainstorm, prototype, and test an idea with real users. It’s a powerful methodology for de-risking innovation and accelerating learning. But what if we applied this same rapid prototyping mindset to something even more fundamental to organizational success: our culture?

As a human-centered change architect, I believe that our work environment, our internal processes, and the very fabric of how we collaborate are all “products” that can and should be continuously designed, prototyped, and refined. Just as customer experience needs constant auditing, employee experience requires intentional, iterative design. The ‘Design Sprint for Culture’ is precisely this – a concentrated effort to identify a cultural challenge, brainstorm potential solutions, build a prototype of a new behavior or process, and test its efficacy in a short, focused burst.

Think about the common cultural pain points: siloed departments, ineffective meetings, lack of psychological safety, or disengaged hybrid teams. These aren’t abstract problems; they manifest as concrete frustrations in daily work. A Design Sprint for Culture allows us to treat these challenges not as intractable issues, but as design problems. It moves us from endless debates about “what’s wrong” to actionable experiments in “what could be better.”

Why Prototype Culture?

The traditional approach to cultural change is often slow, top-down, and prone to resistance. Large-scale initiatives, year-long training programs, or mandated values statements rarely achieve the desired impact because they lack immediate feedback loops and rarely involve those most affected by the change. Culture, after all, is the sum of shared habits and behaviors. To change culture, we must change habits, and to change habits, we must prototype new behaviors.

A cultural sprint offers:

  • Rapid Learning: Instead of waiting months to see if a new policy works, you can test a small behavioral shift in a week.
  • Employee Empowerment: By involving employees directly in the design and prototyping of cultural solutions, you foster ownership and reduce resistance.
  • De-risking Change: You don’t have to bet the farm on a massive cultural overhaul. Small, tested interventions are less disruptive and more likely to succeed.
  • Tangible Outcomes: The output isn’t a strategy document, but a tangible artifact – a new meeting agenda, a communication protocol, a team ritual – that can be immediately experienced.

“Innovation isn’t just about inventing new products; it’s about inventing better ways for humans to work together to create value. Our internal culture is the ultimate product of our collective efforts, and it deserves the same rigorous design thinking as our external offerings.” –- Braden Kelley

The Cultural Sprint Framework (Adapted)

While the exact steps can be tailored, a Cultural Design Sprint generally follows a similar five-day structure to a traditional sprint:

  1. Understand & Define (Day 1): Identify a specific cultural challenge. Frame it as a problem statement. Map out current behaviors and their impact.
  2. Diverge & Ideate (Day 2): Brainstorm a wide range of solutions. Think outside the box: what new behaviors, rituals, or processes could address the defined problem?
  3. Decide & Storyboard (Day 3): Select the most promising ideas. Storyboard how the new cultural behavior/process would work step-by-step.
  4. Prototype (Day 4): Create a tangible, low-fidelity prototype of the new cultural element. This could be a new meeting structure, a communication template, a defined decision-making process, or a micro-learning module.
  5. Test & Reflect (Day 5): Implement the prototype with a small, representative group (e.g., one team, a few individuals). Gather immediate feedback. What worked? What didn’t? What did we learn?

Case Studies in Cultural Prototyping

Case Study 1: Re-energizing Hybrid Meetings

A global software company was struggling with disengaged hybrid meetings. Remote participants felt ignored, and in-office attendees found themselves distracted. Endless debates about technology solutions went nowhere. A small cross-functional team, including remote and in-office employees, convened for a 3-day Cultural Design Sprint.

They defined the problem as: “How might we make hybrid meetings equally engaging and productive for all participants?” They prototyped a new “Hybrid Meeting Protocol” which included:

  • Dedicated “Remote Ambassador” role for each meeting, responsible for monitoring chat and ensuring remote voices were heard.
  • A “5-Minute Focus” warm-up activity to align everyone before diving into content.
  • Mandatory use of a digital whiteboard for all brainstorming, regardless of location.

This protocol was tested with three pilot teams for a week. The immediate feedback was overwhelmingly positive. Remote employees reported feeling significantly more included, and overall meeting effectiveness improved by 25% (as measured by a quick post-meeting survey). The prototype was then refined and rolled out incrementally across the organization, rather than as a top-down mandate.

Case Study 2: Cultivating Psychological Safety in a Design Team

A fast-paced agency’s design team was experiencing a drop in innovative ideas. Post-mortems revealed that junior designers felt intimidated to share early concepts due to fear of criticism from senior members. A one-week Cultural Design Sprint focused on improving psychological safety.

Their challenge: “How might we create a feedback environment where designers at all levels feel safe to share unfinished work?” The team prototyped a “WIP (Work In Progress) Review” ritual:

  • A designated “Safe Space” meeting for early concepts, with strict rules: “No solutions, just questions” and “Focus on the idea, not the person.”
  • A visual “Vulnerability Scale” where designers could indicate how raw their work was, setting expectations.
  • Anonymous feedback submission for certain stages.

The prototype was tested for two weeks. The design team observed a 40% increase in early-stage concept sharing. Junior designers reported feeling more comfortable and valued. The success led to integrating elements of the WIP Review into other team interactions, fostering a more open and collaborative critique culture.

Conclusion: The Future is Designed, Not Dictated

The challenges facing modern organizations are complex, and traditional approaches to cultural change are often too slow and too rigid. By embracing the principles of Design Sprints for Culture, we empower our people to become co-creators of their work environment. We move from abstract conversations about values to concrete experiments in behavior. We build cultures that are resilient, adaptable, and genuinely human-centered – because they are designed by humans, for humans. It’s time to stop talking about culture and start prototyping it.

Frequently Asked Questions (FAQ)

Q: What is a Design Sprint for Culture?

A: It’s a focused, short-term (typically 3-5 day) workshop where a team identifies a specific cultural challenge, brainstorms solutions, prototypes a new behavior or process, and tests it with a small group of employees.

Q: How is it different from traditional cultural change initiatives?

A: Unlike traditional, top-down, and slow initiatives, a cultural sprint is rapid, iterative, and bottoms-up. It prioritizes hands-on prototyping and immediate feedback from employees, de-risking change and fostering ownership.

Q: What kind of cultural challenges can a sprint address?

A: It can address a wide range of issues, such as improving meeting effectiveness, fostering psychological safety, enhancing cross-functional collaboration, defining hybrid work norms, or re-energizing team rituals. The key is to define a specific, actionable problem.

Extra Extra: Because innovation is all about change, Braden Kelley’s human-centered change methodology and tools are the best way to plan and execute the changes necessary to support your innovation and transformation efforts — all while literally getting everyone all on the same page for change. Find out more about the methodology and tools, including the book Charting Change by following the link. Be sure and download the TEN FREE TOOLS while you’re here.

Image credits: Unsplash

Subscribe to Human-Centered Change & Innovation WeeklySign up here to get Human-Centered Change & Innovation Weekly delivered to your inbox every week.