Escaping the build trap — Mellisa Perri

44 min readNov 17, 2020

Key takeaways

I loved the clarity of thought from Melissa Perri, the frameworks and systems to process product management roles, and awareness to avoid common yet prevalent anti-patterns.

I highly recommend this book for PMs and leaders in Tech and you can buy it here.

My key takeaways from the book were

Help define PM roles with the right responsibilities
Guidelines to empower PMs with a strategy that promotes good decisions making
Understand the process of determining what products to build through experimentation and optimization
Guidelines to create right org policies, culture, and rewards to enable product management to thrive

Does this sound like your story?

Marquetly (an imaginary company) fast-growing, hired 100’s of people in a short time, mostly developers. With the realization of the need to have a PM org, people with no prior experience in product management were moved to that role. Suddenly a product org of 20 reporting to a VP product. Symptoms of lack of roadmap, deeper backlogs, and lack of experienced hand in product management. An external coach was brought in to help coach the PMs with frameworks building products.

While the storm settled down, the CEO wanted to ship more features and was frustrated about prioritization. That was the tip of the iceberg.

Beneath the surface, there were 20 major projects with one PM and few developers each, and commits were made for launches. There was pressure to ship 20 major things with constrained bandwidth, growth started declining, and the pressure or raising next round mounted, resulting in getting it out of the door culture. PMs stopped talking to customers, but shipped loads of features, and everyone was happy with the output of an execution machine.

Slowly thinking and creation bugs started creeping in, adoption got poorer. Everyone in the org had a different output metric in mind while the CEO wanted the revenue metric to grow, indicating poor alignment.

Though there were a bunch of smart people and tonnes of features got shipped, there were too many things prioritized (everything being a P0 priority), with different teams trying to optimize for their output leading to the build trap. In build trap ecosystems, developers get rewarded for writing tons of functional code, designers for fine-tuning interactions and creating pixel-perfect designs, product managers for writing long specification documents, or, in an Agile world, creating extensive backlogs and getting things out of the door.

Product Management Role

Chapter 1. The value exchange system

Real value realization for a customer is when their needs and wants are fulfilled and in exchange, they pay for it thereby helping business to realize the value as money, data, knowledge, capital.

Feature launches, great product specs, extensive backlogs, process, systems, seamless execution all are output they don’t generate the intended value/outcomes.

The manifestation of output-driven cultures manifest in poor adoption, orphaning of features, and suboptimal approaches like building feature parity with the competitor who has abysmal adoption of a feature due to lack of realized value for the customers.

Customers are influenced by communities, families, friends, technology, and their needs are continuously changing and evolving. Businesses have little control over the levers of change, and they need to learn the changes and act in a continuum. Without understanding them and hearing them out on how a feature is solving their problems well and fulling their needs and wants in a continuum, the value will remain unrealized.

Few questions that will keep everyone honest in value exchange systems are

How do you know what was shipped was successful?
Did it really add value to the customers and fulfill their needs and wants?
How many of you went back and iterated on the last thing you shipped?

Chapter 2. Project vs. Product vs. Services

Many companies confuse these two as one and dilute the role of the product, a common anti-pattern to watch and avoid.

Products are vehicles of value. They deliver value repeatedly to customers and users, without requiring to build something new every time.

Services use human labor to deliver value to the user. A project has a discrete scope of work with a timebound but specific output. In a product, shipping is just the first step, and it needs to be nurtured and grown to maturity.

Chapter 3. The product-led organization

Product-led organizations create harmony between business, market, tech, and customers. They optimize for their business outcomes, align product strategy to these goals, and then prioritize the most effective projects that will help develop products into sustainable drivers of growth. Other options that have worked for companies are Visionary led (Apple), Technology Led (Google), Sales led (Oracle).

Chapter 4. What we know and What we don’t

Product development is full of uncertainty. When beginning a project, it’s important to separate the facts from the things that we need to learn.

Product management is the domain of recognizing and investigating the known unknowns and of reducing the universe around the unknown unknowns.

You need to separate these items as facts and to label those that you are unsure about as our known unknowns. Known unknowns are clarified enough that you know which question to ask. They are assumptions that you want to test, data points that you can investigate, or problems that you can identify and explore. You use discovery methods and Experimentation to clarify these, turn them into facts, and build to satisfy those facts. Unknown knowns are those moments when you say, “I feel like this is the right thing to do.” Although we should all listen to our intuition, you should also be cautious.

Chapter 5. The role of a product manager

Primarily role of a product manager is to optimize the value exchange system by synthesizing multiple pieces of data, including user analytics, customer feedback, market research, and stakeholder opinions, and then determining in which direction the team should move. They keep the team focused on the why — why are we building this product, and what outcome will it produce?

Chapter 6. Bad product manager archetypes

The reality is that only a few career paths available to learn product management through formal education and it is a lot more experiential. Most of the experiential largely fall into writing specs, aligning teams, gathering requests, and taking features to live and fixing bugs. To understand what product manager role is not, here are the archetypes of not so great product managers.

The Mini-CEO

They are set up for failures as they don’t have authority over people, teams, decisions and they largely play an influencing role instead.

Title typically manifests in arrogant behavior dictating from on high and in general hated by developers and designers.

2. The Waiter

They are order takers and they gather requirements from stakeholders. Their thought process lacks vision and independent decision making and is reactive. They typically struggle with prioritization and tend to end up in the trap of the product death cycle by asking customers to come up with their own solutions. They focus more on when than why and rarely push back on ASKS.

3. The former project manager

Project managers and product manager roles are very different. Former are wired to think when and later are wired to think why. Companies muddy these roles into one, often resulting in losing track of a strategic mindset that understands customer, business, market organization.

Chapter 7. A great product manager

The real role of a product manager is to work with a team to create the right product that balances meeting business needs by solving user problems. They recognize team members' strengths and persuade them to see and achieve a common goal. They figure out what to build thorough customer research, expert information, market research, business direction, experiments, and data analysis. They are comfortable to navigate through an unknown universe by learning and staying grounded.

Lone wolfs often don’t make great PMs as they tend to solution in a vacuum resulting in dissonance with designers and developers.

Product management is about looking at the entire system, the requirements the feature components, the value propositions, ecosystem compatibility, the user experience, underlying business model, the pricing, the integrations, and solving elegantly for revenue without impacting the check metrics.

Good vs Great

Great PMs are tech-literate to make trade-off decisions and not necessarily tech fluent and they are market aware and not necessarily traditional domain experts.

Great PMs empathize with customers, talk to them, understand their frustrations and pain, seek the truth, work backward, and design success metrics that alleviate their pain and improve the funnel as a by-product. They design several alternatives as solutions and experiments to arrive at the most optimal one.

Great PMs anchor problems with a why, why to do? why now? what is the desired result? how does success look like? what can go wrong? how do we mitigate risks? They arbitrate bias, organization pressures, and personal choices by keeping users as a focal point. They seek the truth on

How do we determine value?
How do we measure the success of our products in the market?
How do we make sure we are building the right thing?
How do we price optimally?
How do we position the GTM?
What makes sense to build versus buy?
How can we integrate and create the distribution strategy to enter new markets?

Good product managers

Prioritize works towards clear outcome-oriented goals
Define and discover customer and business value and marry them to achieve desired value for both
Define processes needed to reduce uncertainty about product success in the market

Product managers in senior roles concentrate on vision and strategy for the teams based on market research, understanding company goals, strategy, and current state of the product. The rest of the team focus on the validation of the strategy and build/experiment solutions around the strategy.

Chapter 8. The product manager career path

Most of the work done by PMs can be bucketed as

Tactical

Building features and getting them out of the door
Breaking down features
Coordinating with Tech and Designers
Crunching data to determine what to do next

Strategic

Positioning the product
Winning the market
Meeting OKRs
Carving the path to future

Operational

Creation of road map that connects current to future state
Aligning teams
Working with the development team
Measuring data

PM career path in successful product organizations have the right mix of Tactical, Strategic, and Operational responsibilities and it is not one size that fits all.

APM

It is an individual contributor role
Typically people out of school or a career switch start here
They need a formal mentoring pair for them to learn and grow
Willingness to learn and affinity to a domain are key considerations for hiring them

It is an individual contributor role
They work closely with design and engineering teams and help build the right solutions
They strategize how to feature vision and fitment to the over product.
They tactically ensure smooth execution
Day to day work is more toward operational aspects than strategic aspects and around the delivery of features in the road map addressing short term needs
They communicate and align teams on the performance of features
100% operational is a common trap that should be carefully avoided

SPM

It is an individual contributor role (similar to an architect role in Tech)
Compared to a PM role, there is a higher order of problem complexity and broader scope
They are entrepreneurial and strike a good balance between strategic and operational efforts
They are a great fit for leading and building 0–1 product

DOP

They have people responsibilities
Their core responsibility is to create strategic alignment and drive operational efficiency
They are responsible for creating the strategic roadmap over a 12-month horizon

VPP

They have people responsibilities
They oversee strategy and operations for a product line
They are expected to connect, convert and cascade company goals and visions to product goals
They have the ability to zoom-in and zoom-out between strategic and day and day operations
Their role is predominately a strategic role
They are capable of leading and driving both 0–1 and >1 product journies

CPO

They typically oversee the entire product portfolio
They bring strategic alignment across products to the big picture
They have the ability to interface at board level
They are great in building alignment across functions
They are resilient and truth-seeking

Chapter 9. Organizing Your Teams

Creating and structuring PM organizations around value streams will help achieve value with a single line of accountability for these values and not fell helpless with matrix responsibilities to get anything done.
Creating an anchor who will pull together a holistic vision for each value stream.
Identifying and chartering for tactical, strategic and operational needs-based on the right PM role (APM, PM, SPM…)
Balancing senior and junior talent for balanced outcomes across Strategic, Operational and Tactical initiatives.

Product Management Strategy

A good strategy is a decision framework that connects the vision and economic outcomes of the company back to the product portfolio, individual product initiatives, and the solution options for the teams. It helps to win by executing with focus on the core mission and making hard calls to remove effort on things that are not aligned with the overall strategy.

Netflix Strategy

Chapter 10. What is strategy?

A strategy is a deployable decision-making framework, enabling action to achieve desired outcomes, constrained by current capabilities, coherently aligned to the existing context.

A good strategy isn’t a detailed plan. It’s a framework that helps you make decisions. Too often, people think of their product strategy as a document made up of a stakeholder’s wish list of features and detailed information on how those wishes should be accomplished. It focuses on visions and higher-level goals and sustains the organization for years.

Chapter 11. Strategic gaps

When a strategy is approached as a plan it results in gaps between outcomes, plans, and actions and creates friction within an organization.

The Knowledge Gap

The Knowledge Gap is the difference between what management would like to know and what the company actually knows. Organizations try to fill this gap by providing and demanding more detailed information.

A deluge of information isn’t always that helpful for upper management. You need to focus on communicating and asking for just enough information to make a decision. The upper management should be limiting its direction to defining and communicating the strategic intent, or the goals of the business. The strategic intents are guiding principles that point the team toward the outcomes the businesses want to achieve.

The Alignment Gap

The Alignment Gap is the difference between what people do and what management wants them to do, which is to achieve the business goals. Organizations try to fill this gap by providing more detailed instruction; whereas, instead, they should allow each level within the company to define how it will achieve the intent of the next level up.

At one company, I walked around asking all of the product managers on the hundred or so teams why they were working on their current project. I then asked their leaders the same question. I got two different answers from these two different levels. They could not connect the activities of the teams back to the outcomes of the companies because the leadership had passed down feature requests rather than expected outcomes and goals.

Instead of sending down mandates, organizations should, instead, turn to align every level of the company around the why and should give the next layer down the opportunity to figure out the how and report back. When done this way, product management is successful. When leadership is not aligned at the top, the issues trickle all the way down to the teams.

The Effects Gap

The Effects Gap is the difference between what we expect our actions to achieve and what actually happens. When organizations do not see the results they want, they try to fill this gap by putting more controls in place. However, that is the worst thing you can do in this scenario. Giving individuals and teams the freedom to adjust their actions so that they are in line with their goals is what will truly allow them to achieve results.

Autonomous teams

While autonomy is what allows organizations to scale, autonomy with a lack of initiatives and results creates dissonance.

“My manager keeps handing me solutions. Every time I try to suggest something different, I’m shut down. When we went Agile, we were told that our Scrum teams were supposed to be autonomous.” This is definitely not autonomy.

“Our product managers won’t step up and own the product. I have to keep prescribing things for them, but it’s because they don’t take initiative.” It’s an interesting dichotomy but a common one I’ve seen in companies stuck in the build trap. These are all symptoms of not having a good strategy framework that enables action. When teams are not aligned with a clear direction and goals, they cannot effectively make decisions. If they dare to try, much of the time, the leader steps in and says, “No, that’s not right.”

Chapter 12. Creating a Good Strategic Framework

A good company strategy should be made up of two parts: the operational framework, or how to keep the day-to-day activities of a company moving; and the strategic framework, or how the company realizes the vision through product and service development in the market. Many companies confuse these two frameworks and treat them as one and the same. Although both are important, getting the strategic framework right is essential for developing great products and services.

At Marquetly, the leadership team was prioritizing the work itself, based on what it thought was right to build rather than on feedback from customers. It was reacting to the customers that screamed the loudest instead of evaluating whether those requests matched the strategic objectives. The morale of the company was low, and, because of that, employees were not producing.

Tying budgeting, strategy, and product development to this artificial yearly time cycle only create a lack of focus and follow-through. Instead, companies should be continuously evaluating where they are and where they need to take action, and then fund those decisions.

Think of the major pieces of work you do that are actually bets. Henrik Kniberg, a former consultant at Spotify, explains that this is how Spotify thinks. The company operates using something called DIBBs, which stands for Data, Insights, Beliefs, and Bets. The first three things, data, insight, and beliefs, inform a piece of work called a bet. The concept of thinking of initiatives as bets is powerful because it sets up a different type of expectation.

Strategies are interconnecting stories told throughout the organization that explain the objective and outcomes, tailored to a specific time frame. We call this act of communicating and aligning those narratives strategy deployment.

There are many examples of strategy deployment across organizations. OKRs are a type of strategy deployment used by Google. Hoshin Kanri is a strategy deployment method used by Toyota. Even the military uses strategy deployment with mission command. All of these are based on the same premise - setting the direction for each level of an organization so they can act.

While choosing the right framework is important for the organization, but understanding what makes a good strategy framework is even more important. In most product organizations, there should be four major levels of strategy deployment.

Vision
Strategic intent
Product initiatives
Options

Strategy deployment is about setting the right level of goals and objectives throughout the organization to narrow the playing field so that teams can act. So, while executives might be looking at a five-year strategy, middle management is thinking in smaller strategies - yearly or quarterly - bounding the teams in a direction that allows them to make decisions on a monthly or weekly basis.

The continuous improvement framework practiced at Toyota, called the Improvement Kata, helped it determine its strategies. The Kata teaches people in the company how to strategically tackle problems to reach goals. Mike Rother documented how the process works in his book.

The Four Steps to the Improvement Kata” from Toyota Kata Practice Guide, by Mike Rother

As part of Plan Do Check Act (PDCA) cycles - in step 4, teams go through and systematically identify obstacles standing in the way of reaching the target condition, their next goal, plan out how to tackle it, and then experiment in order to see whether the plan worked. They then reflect on the progress (check) and act accordingly in the next round.

Chapter 13 Company-Level Vision and Strategic Intents

Vision

Company leaders need to spend time communicating their vision, explaining their choices, and painting an image of what is to come. This doesn’t mean that you need to get super-detailed about how this all manifests. It just means that you must tell a story. When that story is told, you can remind everyone through the simple vision statement.

One of the biggest issues I hear from executives is that they do not have the data they need to make decisions. People ask them to create a vision, but they do not continuously surface information in a way that helps inform the strategic decisions that enable the organization to achieve the vision. The teams should be out there, analyzing, testing, and learning and then communicating what they discover back to their peers and their management

Mission vs Vision

A good mission explains why the company exists. A vision, on the other hand, explains where the company is going based on that purpose. I find that the best thing a company can do is to combine both the mission and the vision into one statement to provide the value proposition of the company — what the company does, why it does it, and how it wins doing that.teams.

Examples of Vision

At Bank of America, we are guided by a common purpose to help make financial lives better by connecting clients and communities to the resources they need to be successful. — BANK OF AMERICA

Becoming the best global entertainment distribution service, licensing entertainment content around the world, creating markets that are accessible to lm makers, and helping content creators around the world to nd a global audience. — NETFLIX

Strategic intents

Strategic intents communicate the company’s current areas of focus that help realize the vision. Strategic intents usually take a while to reach, on the magnitude of one to several years.

Strategic intents are always aligned to the current state of the business. When determining what these intents are, the C-Suite of the company should ask, “What is the most important thing we can do to reach our vision, based on where we are now?” There should not be a laundry list of desires or goals — just a few key things that need to happen to make a big leap forward. Keeping the list of strategic intents small helps with the focus for everyone.

With 5,000 people, they shipped only one feature per quarter because everyone was wildly distracted and working on too many things. One intent is usually good for a small company, and three are plenty for a large organization. Yes, three. I know that sounds like very few goals for an organization of thousands of people, but that is key. This is also where the level and time frame matter. Strategic intents should be at a high level and business-focused. They are about entering new markets, creating new revenue streams, or doubling down in certain areas.

To understand how to set strategic intents, we had to first understand what business value really means. Joshua Arnold, a business and product consultant and expert on the cost of delay, uses a great model for thinking about business value,1 as shown below

Chapter 14 Product vision and portfolio

Product Vision

The product vision communicates why you are building something and what the value proposition is for the customer. Amazon does this particularly well by creating what they call Press Release documents for every product vision. These short (typically a page or two) notices describe the problem the user is facing and how the solution enables the user to solve that problem.

The product vision emerges from experimentation around solving problems for users. After you validate that the solution is the right one, you can grow it into a scalable, maintainable product. But you need to be careful not to make the product vision too specific. It cannot describe every little feature but should include more of the main capabilities it enables for the user. If you are too prescriptive, it could stifle the way you grow the product and what you might add to it later.

Product Portfolio

The VP of product usually is the one who owns the product vision, but they might not be the first one to set it. As I said, products emerge out of experimentation, so usually, a smaller team is responsible for determining what that product looks like. As the product becomes more robust, you build a team around it to grow it. But the VP of the product should make sure everyone is aligned to this holistic vision. In companies with one product, the product initiatives describe the major user problems that the company is prioritizing. They need to be aligned to both the product initiative and the strategic intents. The VP of product works with the product managers below them to determine which are the right problems to solve to achieve bot.

The chief product officer (CPO) is responsible for setting the direction and overseeing the product portfolio. Having a philosophy for how your products or services help your company reach that vision in the near term or long term is key. To get there, the CPO answers these questions for their team:

How do all of our products work as a system to provide value to our customers?
What unique value does each of the product lines offer that makes this a compelling system?
What overall values and guidelines should we consider when deciding on new product solutions?

Product Management Process

The best solutions are linked to real problems that users want to be solved. Product managers use a process to identify which of those problems the team can solve to further the business and achieve the strategy. Product managers can rely on the Product Kata to help them develop the right experimental mindset to fall in love with the problem rather than the solution. They continue iterating until they reach the outcome.

Chapter 15 The Product Kata

To understand the direction, you are looking at either the vision, strategic intent, or product initiative, depending on which level you are starting on. The current state is related to where you stand in relation to your vision. It also reflects the current state of the outcomes, including the current measurement of those outcomes.

Option goals are our next level of goals from the team. These are the outcomes you need to achieve in order to make progress toward your initiative or intent. Then you conduct your product process to experiment around systematically tackling problems to reach your goal.

Goal setting

What is the goal?
Where are we now in relation to that goal?
What is the biggest problem or obstacle standing in the way of me reaching that goal?
How do I try to solve that problem?
What do I expect to happen (hypothesis)?
What actually happened, and what did we learn?

Don’t spend your time overdesigning and creating unique, innovative solutions for things that are not core to your value proposition. If someone has already solved that problem with a best practice, learn from that, implement their solutions, gather data to determine if it’s successful in your situation, and then iterate. Reserve your time and energy for the things that will make or break your value proposition.

The best thing you can do, at this point, is to kill the bad ideas! The fewer features, the better. That is how you reduce the complexity of products. Otherwise, you can quickly run into feature fatigue from customers. Remember, it’s about quality, not quantity. The tasks of focusing on the Product Kata and identifying which phase you are in and what tools are available there are key to successful product management.

Chapter 16. Understanding the Direction and Setting Success Metrics

Product metrics tell you how healthy your product is, and, ultimately, your business, given that a healthy product contributes to the overall health of the business. They are the lifeblood of every product manager. Keeping a pulse on your product is crucial for knowing when you should act and where. This is how we set direction. But it’s easy to become stuck measuring the wrong things.

Frequently, teams turn to measure what we call vanity metrics. This concept, introduced in Lean Startup, is about goals that look shiny and impressive because they always get bigger. People are excited to share how many users are on their product, how many daily page views they have, or how many logins their system has. Although these numbers may make you look great to investors

You can easily turn a vanity metric into an actionable metric by adding a time component to it. Do you have more users this month than last? What did you do differently? Think carefully about how to add context and meaning to your facts and figures. Consider the meaning behind the metrics and how they help inform your decisions and understanding. In addition to vanity metrics, I often see product teams measuring output-oriented metrics, such as the number of features shipped, story points complete, or user stories worked on. Although these are good productivity metrics, they are not product metrics. They cannot tie the results of product development back to the business.

There are many product frameworks available to help you think through the appropriate product goals. My two favorites are Pirate Metrics and HEART metrics.

Pirate Metrics

Pirate Metrics were created by Dave McClure, founder of 500 Startups, to talk about the life cycle of users through your product. Think of it as a funnel: users finding your product is acquisition; users having a great first experience is activation; keeping users returning to your product is called retention; users recommending others because they love your product is referral; and, finally, users paying for your product because they see value in

The HEART Framework

HEART metrics measure happiness, engagement, adoption, retention, and task success. These are usually used to talk about a specific product or feature. Here, adoption is similar to activation in Pirate Metrics because you are talking about someone using the product for the first time. Retention is the same as in Pirate Metrics.

With HEART, you add in other metrics to talk about how the user interacts with the product. Happiness is a measure of how satisfied the user is with the product. Engagement is a measure of how often users interact with the product. Task success measures how easy it is for a user to accomplish what they were meant to with the product.

Retention is a lagging indicator, which is impossible to act on immediately. It will be months before you have solid data to show that people stayed with you. That is why we also need to measure leading indicators like activation, happiness, and engagement. Leading indicators tell us whether we’re on our way to achieving those lagging indicators like retention.

Chapter 17. Problem Exploration

Understanding the Problem

Product managers are often spoken about as the “voice of the customer,” yet too many of us are not getting out and talking to customers as much as we should. Why? Because it involves talking to (gasp) people. It takes a lot of effort to line up the interviews, and sometimes that can seem more daunting than staying inside and jumping right into A/B testing or sifting through data. Although data analysis is important, it can’t tell the entire story. So it’s essential that we all go talk to actual humans to get to the heart of their problems.

User research, observations, surveys, and customer feedback are all tools that you can harness to better explore the problem from a user standpoint.

Problem-based user research is generative research, meaning that its purpose is to find the problem you want to solve. It involves going to the source of the customer’s problem and understanding the context around it.

This is what Marquetly did. The team went to the customers, conducted observations, and then asked questions. “What is the biggest problem standing in the way of you finishing your course? What’s the pain?” When conducting problem-based research, you are trying to identify the pain point and the root cause of the problem. When you understand the context around a customer’s problem, you can form a better solution to solve it. Without that, you are just guessing.

It’s easy to fall into the trap of solving problems before you find their root causes. We’re all prone to problem solve, even if we don’t know what the problem is. Our brains love thinking in terms of solutions. However, this can be risky for business. If you don’t have an underlying understanding of the problem, you can never deliberately create the right solution. The only way you can end up there is by luck. I’m not saying that this process is easy, but it’s a more efficient, effective, and successful way. One easy slip when you’re in this mode is pretending that the problem is the lack of a feature.

Nobody wants to hear that their baby is ugly. The way around this is to not get too attached. Kill the bad ideas before they take up too much time and energy from the teams and before you get hooked on them. Instead, fall in love with the problem you are solving.

Chapter 18. Solution Exploration

Companies often confuse the building to learn and building to earn. Experimentation is all about building to learn. It allows you to understand your customers better and to prove whether there is value in solving a problem. Experiments should not be designed to last for a long time. By nature, they are meant to prove whether a hypothesis is true or false, and, in software, we want to do this as quickly as possible.

The most important piece of the MVP is the learning and it should be the minimum amount of effort to learn. This keeps us anchored on outcomes rather than outputs.

There are many ways to experiment to learn. Concierge, Wizard of Oz, and concept testing are three examples of solution experiments. With any experiment, it is important to think of how you will end it — to “close the loop.

Setting expectations on experiments with your customers is key to keeping them happy and to mitigating the risk of a failed experiment. Explain to them why you are testing, when, and how the experiment will end, and what you plan to do next. Communication is key to a successful experimentation process.

Concierge experiments

Concierge experiments deliver the end result to your client manually, but they do not look like the final solution at all. Your customers will understand that you’re doing it manually and that it’s not automated. It’s one of my favorite experiment types because it doesn’t involve coding and it’s quick to get started. Because you get to work with your customers closely, there is a ton of feedback flowing through and there are tight learning loops. Concierge experimenting can be a very powerful tool. The thing to note with this method is that it does not scale, given that it’s labor-intensive.

Wizard of Oz

The method I suggest for reaching a broader audience for feedback is called the Wizard of Oz. The idea behind the Wizard of Oz is that, unlike the concierge experiment, it looks and feels like a real, finished product. Customers don’t know that, on the backend, it’s all manual. Someone is pulling the strings — just like the Wizard of Oz. This is a great technique when you are looking for feedback at scale.

Wizard of Oz can also be combined with techniques such as A/B testing. In A/B testing, you split a portion of your traffic to a new solution idea to see whether it moves a metric compared to the current state of the site. You can use this outside of Wizard of Oz, as well, to test new designs or messages on your website. But you need to be careful about when you use A/B testing. You wouldn’t want to use A/B testing in two instances: if you were still very unsure about your solution direction or if you did not have enough traffic to those pages for the results to have statistical significance. If the latter, you could use techniques like concept testing to get feedback.

Concept Testing

Concept testing is another solution experiment that focuses more on high-touch interaction with the customer. In this case, you try to demonstrate or show concepts to the user to gauge their feedback. These can vary in execution, from landing pages and low fidelity wireframes to higher-fidelity prototypes or videos of how the service might play out. The idea here is to pitch the solution idea in the fastest, lightest way possible to convey the message.

It’s important to note that this type of experiment tends to be more generative than evaluative. Just like problem research, generative solution experiments help you to gain more awareness around what a user desires in a solution. When you show the concept to users, you are asking them to put themselves into the scenario in which they are experiencing the problem, and you are asking them questions about how the solution would or would not solve their problem.

If you want to make it evaluative, to firmly test a hypothesis, you need a definitive pass-or-fail criterion when interviewing a customer about the concept. This can be what I call an ask - something you would need from the user, either in the form of a commitment, monetary value, time, or some other investment that shows they are interested. Landing pages almost always pitch the idea and contain an ask in the form of entering an email address.

Learning reduces risk. The goal of solution exploration is to get faster feedback. If we take too long to get feedback, we not only waste money but we also waste time. The opportunity cost of building the wrong thing is too high. Every industry and product has unknowns - getting creative about how you answer these unknowns is key.

Experimentation on internal products is often missed out. It is equally important to make the product for internal users efficient to do their job well. At Marquetly, By experimenting and making internal products better, internal users were happier, and we reduced the churn of the employees in this position who had previously felt handicapped to do their job well. These people were also able to get more done, and the operating costs for our business went down as a result of not having to keep hiring at an incredible pace. Internal tools are often neglected, but they still matter to the company. They need to be treated the same way as any other product.

Solutioning is a journey by itself and you need to understand the direction, diagnose the problem, learn more about it, and then learn what the right solution is. After you have experimented to prove value, you can concentrate on building your first version and optimizing it.

Chapter 19. Building and Optimizing Your Solution

Evolving the Product Vision

Story mapping and North Star documents are two ways to help teams find alignment around the vision.

A North Star document explains the product in a way that can be visualized by the entire team and company. This includes the problem it is solving, the proposed solution, the solution factors that matter for success, and the outcomes the product will result in. North Stars are great for providing context to a wide audience. They should be evolved over time, as you learn more about your product. It’s important to note that this is not an action plan — it does not include how the team will be building the product. That is where story mapping comes in.

Story mapping helps teams break down their work and align around goals. Its purpose is to help the team communicate about their work and what needs to get done to deliver value.

Prioritizing Work

Prioritization, as I mentioned earlier, is a top issue for most product managers. There are many frameworks out there that will help you prioritize, like benefits mapping, Kano models, and others, but my favorite is Cost of Delay. If you understand the desired outcomes from a strategic perspective, you can use the Cost of Delay to help you determine what to ship sooner.

Cost of Delay combines urgency and value so that you can measure impact and prioritize what you should be doing first. When you think of building and releasing that first version of the product, you need to consider the trade-offs between the amount of value you can capture with the scope of the release and the time it takes to get it out the door. It’s an optimization problem. You want to reduce scope enough so that you can capture the maximum value in the right time. If you wait too long because you over scoped the release, you lose the money you could have been making. Worse, a competitor could swoop in and steal your market. Then you’d have a higher threshold to entry, and your product will need to be light years better than the competitions. On the flip side, you don’t want to release something that is terrible and provides minimal benefit to the user in order to get it out early. Then you could lose early adopters, and it’s difficult to win someone back after they’ve had a terrible experience.

Cost of Delay can help end many debates about what should and should not be prioritized first. If you want to learn more, head to Black Swan Farming and read about how to utilize the concept in your company.

The Real Definition of Done

We are done developing or iterating on a feature only when it has reached its goals. Often, teams ship a first version of the feature and then move on to the next, without measuring the outcomes for the user. As Jeff Gothelf, the author of Sense & Respond, once said, “Version 2 is the biggest lie in software development.” This mentality always leads to a build trap.

The Product-Led Organization

The product-led organization is characterized by a culture that understands and organizes around outcomes over outputs, including a company cadence that revolves around evaluating its strategy in accordance with meeting outcomes. In product-led organizations, people are rewarded for learning and achieving goals. Management encourages product teams to get close to their customers, and product management is seen as a critical function that furthers the business.

To truly get out of the build trap, you need to become a product-led organization, both in mentality and practice

Chapter 20. Outcome-Focused Communication

The senior leadership team understands the outcomes. They get what it means to see results, and you could see the benefits in aligning the strategy through the organization back to them. Companies who get stuck in the build trap are not patient enough to see outcomes emerge.

If there is one main reason I have seen companies fail to make a transition, it’s the lack of leadership buy-in to move to an outcome-oriented company. Lead- ers will say that they want to achieve results, but, at the end of the day, they still measure success by features shipped

Cadences and Communication

Great communication, in the form of working agreements, meeting cadences, and roadmaps, can solve many of the alignment problems in the company.

Visibility in organizations is absolutely key. The more leaders can understand where teams are, the more they will step back and let the teams execute. The more you try to hide your progress, the wider that knowledge gap becomes. Leaders will demand more information and will crack down on team members the freedom to explore. If you keep things transparent, you will have more freedom to become autonomous.

Many companies fall back into bad habits because they have not figured out how to consistently communicate progress across the company in the form of outcomes and when leaders do not see progress toward goals, they quickly resort to their old ways to control.

Most companies have a few core meetings to evaluate progress and to make strategic decisions from a product level.

Quarterly business reviews

During quarterly business review meetings, the senior leadership team, made up of the executives and the highest level of the organization, should be discussing progress toward the strategic intents and outcomes of a financial nature. This includes reviewing revenue for the quarter, the churn of customers, and costs associated with development or operations.

The chief product officer (CPO) and their VPs of the product are responsible for communicating how the outcomes of product initiatives have furthered strategic intents as Karen did at Marquetly. New strategic intents can be introduced in this meeting, as older ones are coming to completion. It is not a place to prioritize new product initiatives or to go into detail on them. That is what the product initiative review is for.

Product initiative reviews

The product initiative review is another quarterly meeting that can be staggered with the quarterly business review on off months. This meeting is for the product development side of the house — CPO, CTO, design leaders, the VPs of product, and the product managers.

Here we review the progress of the options against the product initiatives and adjust our strategy accordingly. This is the place for product managers to talk about the results of preliminary experimentation, research, or first releases, as they relate to overall goals. New product initiatives can be introduced in this meeting for feedback and buy-in, along with funding from the product development leadership group. Product teams can ask for more funding to build the first version or to optimize an existing solution.

Release reviews

Release reviews provide the opportunity for teams to show off the hard work they have done and to talk about success metrics. These should happen monthly, before features go out, to showcase what is in the pipeline to be released. During this meeting, we should be communicating only what we know is going to ship — not experiments or research being conducted. Although not necessary, most executives like to attend this meeting to see what is being shipped out to customers. This can also be a place for teams to communicate their roadmaps internally so that marketing,

Roadmaps

Instead of thinking of roadmaps as a Gantt chart, you should view them as an explanation of strategy and the current stage of your product. This combines the strategic goals with the themes of work and the emerging product deliverables from it.

Usually, roadmaps consist of a few key parts:

The theme
Hypothesis
Goals and success metrics
Stage of development
Any important milestones

Typical stages of development are

Experiment

This phase is to understand the problem and to determine whether it’s worth solving. Teams in this phase are conducting problem exploration and solution exploration activities. No production code is being created.

Alpha

This phase is to determine whether the solution is desirable to the customers. This is a minimum feature set or a robust solution experiment, but built-in production code and live for a small set of users. These users understand that they are getting early access to a feature that might change or be killed if it is not solving their problems.

Beta

This phase is to determine whether the solution is scalable, from a technical standpoint. Although not always needed, this is a good phase to reduce risk. This release is available to more customers than the Alpha phase but is still only a smaller subset of the entire population since we are still testing. At this point, we’ve proven that the solution is desirable to customers, so it is unlikely that this feature will be killed unless it is not technically stable.

Generally Available (GA)

This phase means that the solution is widely available to all of our clients

Product Operations

When companies consist of just a handful of product teams, it is fairly easy to keep track of what’s going on. Leaders can walk over to the product managers to learn about progress on goals. Processes are usually determined at the team level. Coordination is not a large concern. But, as product teams scale to more than a few teams, keeping track of progress, goals, and processes becomes a challenge.

Product operations are a very small team (two people) to help streamline operations and reporting. They oversaw the cadences of strategy, found an analytics partner to set up tracking, and collected and organized the progress toward goals into reports for executives. This allows the product people to focus on what they were good at, while product operations helped them to make informed decisions, by surfacing up those reports.

In larger organizations, the product operations team reports to the CPO but it needs an experienced leader, usually at the VP level, to oversee it. This team is in charge of streamlining all operational and process work that product teams need to be successful. This includes:

Create automated and streamlined ways to collect data on progress toward goals and outcomes across teams.
Report on goals, outcomes, roadmaps, progress, capacity, and costs across the product organization, translating these activities into financial implications for the company executives.
Set up and maintain a product analytics platform to report on product engagement metrics across the organization.
Standardize product processes that go across teams, such as strategy cadences, experimentation tracking and feedback, documentation on product features, collecting data, setting goals, creating and maintaining roadmaps, and sales enablement.
Organize and run critical product meetings for strategy creation, strategy deployment, and releases.
Conduct any coaching or training for the product teams.

The point of this team is not to dictate how the members of a team work together to build the product but, instead, to create the criteria for inputs and outputs of the work. they are not creating the product roadmap for the teams. They are creating a system and template for teams to input their goals, themes, progress, and details that can then be shared around the organization. They are not dictating whether a team can talk to users. They are creating systems that help teams figure out which users to target for their experimentation.

The product operations team should be made up of a combination of project managers and product people. It’s good to allocate a few developers to this team, as well, so they can integrate with third parties, if needed, or build custom tools to fit a specific purpose.

Chapter 21. Rewards and Incentives

Rewards and incentives are motivators for the employees of every company. The biggest issue I see with companies trying to transition to becoming product-led is what they don’t evaluate their current reward structures to make sure they incentivize the right behavior.

Companies should be rewarding people for moving the business forward — achieving outcomes, learning about your users, and finding the right business opportunities. At the end of the day, the rest is just vanity metrics.

Chapter 22. Safety and Learning

In addition to reward structures that prevent people from innovating, the culture of the organization plays a big part. You might not be judging your teams for success based only on outputs, but they may still not be willing to try new things. Why? There may not be enough safety in the organization to fail and learn.

To really push boundaries, teams are going to have to try some perceivably wild stuff. It might not be the solution you originally thought of and the teams might not have all the answers at the beginning, but if they are not allowed to explore these weird paths, they will never push the status quo. The status quo is safe. The status quo keeps you from innovating.

Many companies fail slowly. They release products and never measure whether those products do anything. They just let them sit there, collecting dust in a sea of endless features, never knowing whether they are producing value. This is the more dangerous and costly way to fail. Taking 10 years to fail, slowly burning through cash and never getting anywhere, is more problematic than allowing for smaller failures along the way.

If you adopt a great product mindset and you give people the freedom to fail, what you’re doing is allowing them to fail quickly, quietly, and at a lower cost because they’re testing things early. That’s the type of failure you want to encourage. That’s the type of failure from which we can recover. Ideally, product managers should risk mitigators who say, “Hey, I think the most costly thing we can do is build this product without knowing it’s the right product to build. How do I test it and ensure that this is actually what we want? How do I become more confident that we’re on the right path before I invest money in this?” Leaders who give people the room to do that see the best results and avoid the build trap.

If you are a product manager, think about how you can change your message to your boss and begin to gain trust by working this way. If you are a manager, be open to the possibility that new ways of working are also beneficial and be ready to help your product manager establish boundaries, rather than saying no to them. And, finally, if you are an executive, think about how you can create safe spaces for people to learn. There are many different ways to create boundaries. One way I frequently recommend is to segment your user base into populations for Alpha and Beta testing like I mentioned in the communication chapter. That means, instead of launching the product to everyone, start with a small representative population, learn from them, and then expand to more people as you feel more confident.

Chapter 23. Budgeting

One of the factors that lead to the output-over-outcome mindset in organizations is the way that they do budgeting.

Product-led companies invest in and budget for work based on their portfolio distribution and the stage of their work. This means allocating the appropriate funds across product lines for things that are known knowns and ready to be built, and it means setting aside money to invest in discovering new opportunities that will propel your business model forward. They then allocate more and more funds to grow the opportunities as they become validated.

Chapter 24. Customer Centricity

Many of the top companies today, such as Amazon, Net ix, Zappos, Dollar Shave Club, and Disney, have gotten where they are by focusing on the customer. You can see this attitude manifest in the way that executives talk about and treat their customers.

One of the most famous Je Bezos quotes about how Amazon succeeds is, “The most important single thing is to focus obsessively on the customer. Our goal is to be earth’s most customer-centric company.” This approach really defines everything that Amazon does, and it pays off.

The core of what it means to be customer-centric is to put yourself into your customers’ shoes and ask, “What would make my customers happy and move our business forward?

Chapter 25. Marquetly: The Product-Led Company

It took a few more years for Marquetly to completely escape the build trap. Many of its people had been working with an output mentality for a very long time. These people didn’t initially believe in this new way of working, but, as full results began trickling in over the next few years, it was difficult to argue with the numbers.

Marquetly was able to achieve its strategic intents, growing its revenue in the enterprise and individual markets, which resulted in it being acquired for a very large sum of money by a larger educational company. The company continued to prioritize its strategy on a rolling basis until it reached its goals. The artificial time bounds of yearly budgeting and strategy creation disappeared. Instead, the company took an investment-minded approach, budgeting each year for growth strategies, while funding initiatives that the product teams validated through experimentation and research.

Marquetly ended up killing a lot of its ideas early on. That allowed it to focus on what really mattered to achieve its goal. Marquetly was successful because it had a leader who understood that change started with him. The CEO, Chris knew that, if he did not adopt the outcome-oriented mindset, the customer-centricity, and the comfort with uncertainty, no one else in his organization would.

He surrounded himself with smart product leaders, like CPO Jen, and then trusted his team to achieve the outcomes. They brought in more senior product people to train the junior product people. Christa and her team became an early success story, shared through the organization, and recounted to new hires so that they would understand they could think outside the box.

Chapter 26. My career learnings as a PM

I learned that my role was not that of the big idea generator but that of the bad idea terminator.
I needed to learn to be humble and to gain the support and buy-in of my team in order to make great products.
Experimenting with my team taught me the power of data.
Data beats any opinion every time.
As I moved on to more senior roles, I learned that having a good strategic framework could make or break a company.
That if you did not judge people for success by outcomes, you would never achieve those outcomes.
I watched a few companies crumble under the weight of a bad strategic framework.
Becoming a consultant taught me about the power of personalities in an organization. People will get in the way of a good product every time. Even if it is the best idea for the company, if it doesn’t meet the personal agendas of senior stakeholders, it can be squashed. To mitigate that risk, you need to deeply understand what motivates people and to know how you can address their personal motivations by introducing information and data that wins them over

Chapter 27. Six Questions to Determine Whether a Company Is Product-Led

Who came up with the last feature or product idea you built?

If I ask a product manager this question, I hope to see a look of confusion on his or her face. “What do you mean who came up with it? Well, our team did. Right? That’s how it normally works.” This kind of response is a sign of a healthy product management organization, in which management sets the goals and the team is given room to figure out how to reach them. The product manager should be leading the charge to discover user problems and to solve them. This doesn’t mean that an important initiative or solution idea can’t come from management every once in a while, but that should be the exception, not the rule. It’s a huge red flag when a team not only can’t take ownership of what it is building but can’t even tell me why it is building it. This means that the originator of the idea never connected the why to the what.

2. What was the last product you decided to kill?

Another sign of an unhealthy product management culture is the inability to kill a product or idea that will not help a company reach its goals. If you hear, “We never really kill anything,” it often means that there’s a pretty big problem. Typically, this happens for one of the following reasons:

The organization already committed the idea to customers. Often, someone from marketing has promised a client that a particular feature is in the works and then the company feels committed to fully follow through with it. It doesn’t matter whether the client actually requested it or whether it achieves any of the organization’s desired goals.

Budgeting can’t budge. In some large organizations, in which the budget is set at the beginning of the year, the team must spend all of it, or else it will not receive an equally large budget the following year. This concept is baffling, but it happens.

No pushback to management. Again, a lack of testing and questioning potential features signifies a lack of empowerment in a team. If a team doesn’t feel like it is safe to say to management, “Hey, that thing we tested, well, it doesn’t work and we don’t think it’s worth the money to build it,” the chances of a successful environment for product management are slim.

3. When’s the last time you talked with your customers?

What I dread hearing is, “Oh, well, management doesn’t really let us talk to customers. They’re worried about us annoying them too much.” Without a healthy dialogue between a company and its customers, there is no way to truly learn about what the customers want or need. An organization set up for success not only allows product managers to talk to customers, but it also encourages them to do so and recognizes this process as a huge part of the job.

In fact, your interviewee should be probing you for clues that you are comfortable talking to customers and that you aren’t planning to spend all your time safely indoors writing user stories.

4. What is your goal?

This is the first question I ask any product manager during an interview process.

If the product manager cannot articulate a clear goal, it’s a sign of poor product management at the organizational level. If the product manager does have a goal but it is more output centric than outcome-focused, this also signifies an unhealthy product team. An output-centric team measures success in terms of meeting product shipment deadlines. It pays little attention to what these products are actually doing for its business.

The purpose of a product manager is to create value for the business by creating value for the customer. If the product manager does not understand the vision of the company, how are they supposed to figure out how to get there? Goals should be outcome-oriented, actionable, and clearly communicated throughout the organization.

5. What are you currently working on?

Truly successful product manager talks more passionately about the problems the product development team is solving than the solutions they are shipping. This is one of the biggest signs of success, for me, and it goes hand-in-hand with the question of goals. When I ask product managers this question, I want to hear about what big problems they are tackling for the user and the business. Of course, they will talk about the solution, as well, but more in the context of what it will do to help solve their problems. If this mindset is encouraged throughout the organization, you can hear it echoed at all levels.

6. What are your product managers like?

As product managers, we want to work in an organization where the role is respected and well regarded. I’ve seen many organizations where the product management function was not well-respected. There were two causes: product managers were either seen as too strong, or they were seen as too weak.

In the first instance, product managers were seen as dictators who threw out requirements to the team rather than involving them in their decision-making process. The teams grew resentful and felt they were treated as resources rather than as colleagues. A good product manager knows that getting buy-in from the whole team is crucial. The product manager is not the only person who should be coming up with the ideas but should instead be harnessing their team’s full capacity. A sign of a healthy product team is hearing development and UX people say, “I love my product manager. She has a clear direction, communicates well, and helps keep us stay focused on the goals and problems.”

In the second instance, product managers are seen as weak in the organization because they are beaten down by stakeholders2 and management. When product managers are seen as project managers, they hold no decision-making power. Stakeholders and management use them to just usher their own ideas through. Product managers don’t feel like they can say no because of the potential for strong backlash.

The dream organization for product people is one that sees product managers as leaders who help shape the direction of the company and the services they provide to their customers. They are respected as partners in steering the ship forward. These six questions can help you to ensure that the company you are in—or want to join—will support and encourage you to do everything you can to succeed.