Dan Olsen

[Amy Jo Kim] Welcome, Dan, to the Getting2Alpha Podcast.

[Dan Olsen] Thanks. It’s a pleasure to be here talking with you.

Briefly just tell us, what is your current role and focus professionally?

I’m a consultant, coach and speaker in product management, UX design and lean startup. I’m usually an interim VP of product for my clients. I’m very hands-on and I usually work on-site with the teams. I also give workshops and coach teams and I am a speaker at events like you are. We always run into each other at a lot of events. I also started, two and a half years ago, a meetup just to share, bring in great speakers, and share ideas called “The Lean Product & Lean UX Meetup” in Palo Alto. We have a monthly event so those are the things that keep me busy most of the time.

Awesome. Give us a whirlwind tour of your background. How did you first get started in design and tech, and how did you decide what to pursue along the way?

The story really starts back when I was 10. My parents gave me a computer so I kind of grew up with computers as the age of computing became more popular. I’ve been very comfortable with digital technology from an early age. After college, I actually worked on submarine design which was a lot of fun and I realized I loved designing and building systems that had a lot of complexity in them and after doing that I decided I wanted to go to business school to build on my tech background, and so that’s what brought me out to the Valley 20 years ago. When I was at business school, I realized there’s this thing called product management where you get to basically do that and define products. I asked everybody where’s the best place to learn product management and back then everybody said Intuit and so I interviewed and got a job at Intuit and I worked there for 5 years and it was great.

It was like a post MBA in product management, marketing, UX design, product development. After working at Intuit, my plan was to work at startups and take what I’d learned and apply them at startups. I helped a friend with a bootstrap startup in Madrid, Spain which was a lot of fun, out of his apartment. I’m actually half Spanish so that was a lot of fun. Then I worked as head of Product Management at Friendster back in the day, which was exciting. After doing a few startups. I also started my own company which was a lot of fun. I did that for 4 years. After that, I basically got into helping companies as an interim VP of product and being a consultant.

Wow. What was the most surprising thing you learned from working at Friendster?

Two things, one of them is in my book actually, is I created this hierarchy of web user needs, which is like Maslow’s hierarchy, but for web users because at Friendster unfortunately, we had issues with the site being up and with the pages loading quickly enough and so a lot of people at the company would spend time talking about, “We should have this feature. We should do this biz dev deal.” I kept trying to reiterate like, “Hey, those things don’t matter if the product’s not up.” It’s funny because I don’t know if you’ve been checking out Pokemon Go at all but in the early days of Pokemon Go, they were having up-time issues too and it just gave me flashbacks to Friendster. That was one thing I learned. The other big I learned is when it comes to social products, a lot of times people talk about network effects and tech products, and social products have an even stronger network effect than typical digital products do. A typical products basically have like Metcalfe’s law, but for social products there’s the things called Reed’s law.

What’s interesting is the more people you have, the more valuable your network becomes. Even stronger than other technologies. The biggest, most important feature for any social product is not actually any functionality, it’s how many users are on it basically. It’s like an unspoken feature. It doesn’t matter how good your feature set is if you don’t have enough users when it’s a social product. Those are 2 of the main things that I learned there.

Those are awesome things to learn. You are the author of a well-known book called “The Lean Product Playbook.” It’s a great book. We’ll link that in the show notes. I have 3 questions about the book. Let me ask them and then we’ll go through them one by one. The first is, what is it that first prompted you to write it? What was that impetus? Then also, how did writing it deepen your understanding of product development? Because it must’ve. Then the third question I have is in truly in product fashion, how did you decide what to include and what to leave out of the book?

Those are great questions. What prompted me to write is basically I have invested heavily in my career in learning about product management and UX design, and trying new things an iterating, and actually being a consultant. One of the interesting things I stumbled in in being a product consultant is it accelerated my learning because I was working with multiple startups instead of just being at one company and seeing instance. I saw multiple ones. I basically from my career and from my MBA and from my work experience and from my consulting experience, started just crafting together theories and processes and frameworks that became helpful. Just iterated those over time and improve those over time. I also started speaking awhile back and putting those ideas on slides and talking about them with people. I would iterate them based on the questions I would get from the audience. Entrepreneurs and product mangers and designers would ask me questions and the next time I gave the talk, I would add some slides that would address that.

The book was just basically like a natural evolution of speaking and consulting where I just came up with these ideas and frameworks for how to think about building great products and achieving product/market fit. That’s why I started my Meetup as well. I’m just a big believer in sharing best practices and people sharing examples and advice and comparing notes like you and I are talking about. When you have a group setting, that’s one of the most great ways you can learn is from learning from other people.

Right. In a curated group setting especially.

Yeah. In a curated group setting. Exactly.

The art of that … Curation is something that, I have to say, I think you’re really good at.

Oh, thanks. I appreciate it.

How did you decide what to leave in and what to leave out?

Basically I said about the thing … It’s funny because someone we were just talking about was basically saying, “Wow.” I had some people read my book. “Wow. You tried,” what they said. They got it. They’re like, “You tried in a single book to explain how to do product management end-to-end.” That’s basically what I tried to do. It’s 335 pages so it’s a little on the longer side, but I wanted … Part of the thing is the core part of the book is what I called “The Lean Product Process.” It’s a 6 step framework that you can follow to go from an idea to going down the path of iterating towards product/market fit. I had to convey the whole process so it’s a chapter for each of those steps. There’s also some introductory material about problem space versus solution space that’s really foundational and important. That’s in there. UX design is critical. I think it’s just so important. You can have the best product strategy and the best idea for features and what to put in your MVP, but if you fall short in UV design, it doesn’t matter.

I have a whole chapter dedicated to UX design in there. That’s the core of the book is that how to iterate. From my talks, I knew end-to-end case studies are valuable for people. They hear you talk about the 6 steps and they nod their heads and they get it, but when you show them an end-to-end example they really get it. There’s a whole chapter on it and set an example. At that point, and I’m a big advocate to evaluate as much as you can before you start coding, I wanted to not leave people hanging there but then talk about okay, how do you actually build it once you validated the product, your MVP, how do you actually build it? There’s a whole chapter on agile development that covers both Scrum and Kanban which are the 2 most popular ways of doing it. As a product manager, it’s important to know how those are supposed to work so you can effectively with developers. Then the final 2 chapters are just basically once you’ve launched a product or you have a product at site, you can use metrics to improve it over time.

There’s 2 chapters again with the case studies. That’s where I drew the boundary. I’m like, “I want a comprehensive soup to nuts. If you have an idea, how do you validate it with customers? How do you build it and launch it? How do you optimize it with metrics?” The things that I left out were basically things more related to team, like team structure. There’s a little bit in there but like what are the skills that you need and how do the different functional groups, PM, design, and dev, how do they work together effectively? I didn’t have time to get into that too much. The other adjacent topic I didn’t have time to get into it was just marketing. How do you message your product? How do you market it? How do you talk about it effectively? Those were just things that in the grand scheme of things just stuck out. Didn’t make sense to be in the book. I have some talks on both of those topics. I just wanted to be that comprehensive single resource that covered everything end-to-end when you’re building a new product.

How did your understanding of product management change as a result of writing that book?

The cool thing was, like I mentioned, I’ve been developing these ideas and iterating them and improving them over time on how to approach product management and building great products and frameworks. What it did is it gave me time to take a step back and reflect and really refine and polish and iterate them further. It’s so funny. If you probably go back and look at my old slides, there are earlier cruder iterations but it wasn’t as polished. For example in the book, I created a framework called “The Product/Market Fit Pyramid.” It’s a 5 level pyramid that is a way to think about product/market fit. That came out in my talks right before I started writing the book. It was this molecule. There’s a molecule out there like a 3 person molecule. Customer-problem-solution molecule. That’s what it was before. It really iterated to this pyramid. It gave me a chance to reflect and come up with the best way to visualize it and convey it. That was really cool.

Then also I knew that I’d worked with a lot of different types of MVPs and it’s funny because you see all these debates online about what is an MVP and what’s not an MVP like flame wars. There’s a little blog post that says, “A landing page is not an MVP.” Then there were responses, “A landing page is an MVP.” In my mind I’ve seen a lot of teams get tripped up on that. I created a framework that categorizes all the possible MVPs so that both of those people can be right in a sense. It’s like, “That’s a product MVP versus a marketing MVP. This is a qualitative MVP versus quantitative MVP.” Let me take a step back and synthesize things and create these higher level frameworks that are really powerful.

It’s interesting because that really intersects with the research on high performing individuals in teams. At least I know from high performing individuals that having the right mental model often is one of the key differences between being able to really knock it out of the park.

I agree. I agree. It’s funny. About a couple of weeks ago, someone had a Tweet. It’s like, “Hey. Product management.” Someone has shared a collection of frameworks. This is just great. You have to know which one applies when. I used to play poker and they ask you, “Okay. Here’s a hand. What should you do?” The answer is almost always, “It depends,” and then you get into like, “If this, then that. If this, then that.” Same thing with these frameworks and mental models. I think mental models are very important and huge. If you have the right mental model, can make viewing a problem or situation so much clear then if you don’t.

The reality is your model and my model and the lean startup model and what Marty Cagan does and it’s all very similar.

Yeah.

It’s all different lenses of viewing the same fundamental process which is iterative experimental design and development.

Yeah. One of the funny things along those lines, I was an electrical engineering major. I pride myself on being relatively scientific. You see people increasingly talk about the scientific applied for product development. I think they’re definitely some parallels. It falls a little short I think because people are people. It’s not like these immutable laws of nature or anything like that, but it’s more statistical and probabilistic about how do people respond to what you’re putting out there. When you see thought leaders sharing similar ideas, you can’t help but feel like there’s some fundamental underlying truth that’s being uncovered. I remembered this happened earlier in my career when I was at Intuit, I developed a framework that I covered that’s key in the book about how to really make sure what you’re building is going to add customer value called importance versus satisfaction. You basically try to assess the customer need that this feature of the product’s going to address. How important is that? It’s importance around importance. Then how satisfied are people with the current alternatives that are out there. You want to build things that have high important needs with low satisfaction.

I developed this on my own as a product manager at Intuit trying to figure out how to prioritize all of the different feature ideas we had. Then I remember picking up Anthony Alwick’s book called “What Customers Want.” I was randomly at the Stanford Business School library and his book was on the shelf. It was promoted there. I grabbed it and I’m like, “Oh, yeah. He has his own importance versus satisfaction framework. This is amazing.” You know what I mean? I’m like, “There must be something to this importance-satisfaction.” Then the Kano model. I studied, what I didn’t mention also, between undergrad and going to business school, at night I did a Master’s in industrial engineering where I learned about lean manufacturing. That’s where I learned about the Kano model which focuses a lot … It doesn’t focus on importance but it focuses on satisfaction. You see people focusing on these concepts. That was another thing that I did in the book. It helped me really take that importance versus satisfaction framework and actually make it quantitative and more rigorous, but you’re right. There’s a more commonality to what everybody’s saying.

It’s like, “Hey. Articulate your assumptions or your hypotheses. Figure out the fastest way to test them. Avoid investing too much before you validate it. Figuring out what the highest risks are. Iterating. Being customer centric. Getting out of the building.” Those are all common concepts that you hear again and again.

Yup. Then there is specialization and differentiation that’s happening within that space. It’s funny. It reminds me of when I was in college and I took comparative religions class. I went, “Oh my God. They’re all talking about the same thing.”

Yup.

Yet, they fight.

That’s funny. Yeah.

You talk to a lot of different product creators. You give talks. You give workshops. You work deeply with teams. What are some of the most common mistakes that you see product creators, particularly first time product creators, making in the early stages when they’re testing their ideas and bringing their ideas to life?

One mistake I see actually even that, one of the most common mistakes I see people jumping into solution space too soon before they really clarify or understand the problem space. Example after example of startups where it’s like a founding team, they come up with an idea. They start designing and building the product right way and then they launch it. Then they realize that it doesn’t resonate with people. Hopefully they realize, “Wow. We skipped a step. We didn’t get clear on who’s this for. What it’s going to do for them? How’s it going to do that in a way that’s better or different than from what’s out there?” That’s a big thing. That’s why the first chapter in the book after the introduction is just problem space versus solution space. I’m excited. You hear people talk more about problem space and solution space more and more at conferences and in talks. That’s the first one is, “Hey. Before you jump into solution space, one, be aware when you’re jumping into solution space. You’re transitioning into it. Two, before you do, try to get really clear on the problem space.” That’s one.

The other one when they’re testing what I found is, it’s … I understand it’s a pragmatic problem. Sometimes people will follow the right exercises. Come up with some wireframes or prototypes to get feedback, but then they don’t talk with the right people. They either talk to whoever’s available. It’s like, “Hey. Let me just grab whoever.” One, is the mistake people make is friends and family. Even if they don’t do friends and family, let’s talk to whoever is available. Like I’m at a co-working space right now. I can just go randomly grab people in the hallway and say, “Hey. What do you think of this product? Or my wireframe?” That wouldn’t be good because the whole point is it’s built for a specific target customer. Right? That’s why in my product/market fit pyramid the bottom layer is the target customer because everything depends on them. That’s it. People talk to whoever is available which I understand. Sometimes it’s just hard to find the right people. The other is not being rigorous enough about defining the target market.

I’ll talk to the people and be like, “Yeah. We’re focused on … We’re actually …” I was a judge at the D school for a competition. I was basically instead of focusing on what the product design was, I was like, “Who’s this for and what does it do for them?” I remember asking one group of college students, “Who’s this for,” and they’re was, “Millennials.” I didn’t say anything to them but in my head I’m like, “Oh my gosh. That’s a huge group. That’s definitely not segmented enough or targeted enough.” It’s like millennials that work, millennials that don’t work, women, men, what needs do they have, busy millennials. As I got to know their product, I could start to think of other facets or segments we can add on to millennials. It happens again and again whether it’s defining your product or your feature or your problem space or your customers. It’s like peeling an onion and most people just stay at the superficial high level and really, really good products and good product teams, keep peeling layers and getting more and more insight and more detail and are able to refine that.

That’s awesome. When you’re working with teams deeply, bringing these projects to life and working with them on problem management, how do you approach early testing and iteration?

I’m a big, big believer in test before you code because it’s just human nature. One is it’s a more efficient use of research. Secondly, it’s just human nature that once you start coding and building something, it’s going to basically … People are going to beholden to it and want to adhere to it because they invested in it. Even if they don’t emotionally, there’s just a practical standpoint of, “Okay. We built the code this way and the objects this way. Now we need to pivot 90 degrees. Let’s just try to reuse this.” In a way it adds constraint and baggage and tech debt in your solution space before you’ve gotten clear on your problem space. I like to basically work with teams to explicitly articulate hypotheses about who’s this for, what cluster of needs is it going to solve? Let us score those on importance and satisfaction. The other thing that’s cool about the Kano model for people who are familiar with it is it basically classifies needs or features into 3 categories: must haves, Performance benefits where more is better, and delighters.

Speaking of mental models, it’s a great lens to think through as your talking about features you’re building. Categorizing in the 1 of those 3 categories. Then basically brainstorming a lot of different ideas. The toughest part with the most uncertainty is getting your MVP right. It’s like, “Great. We did all that rigorous thinking in the problem space. We have clear hypotheses who it’s for. We have clear hypotheses about what needs it’s going to meet for the customers. We have clear hypotheses about how it’s going to do that in a way that’s better. Now we need to figure out what’s that feature set that we’re going to test on our MVP.” That’s where you cannot put enough in and then it won’t test well. What most people do is put too much in. The funny thing is MVP was designed to precisely prevent you from putting too much in. Right? It’s like the opposite of a team of developers go into a cave for 18 months head down and then emerge and launch their product and then realize that nobody wants it. The whole point is to build the least amount that you can. Test it.

That can be tough. I always tie it back to the value proposition. The other thing is sometimes people in their MVP, they’ll put all the must haves but they won’t have any of their differentiators like their delighters or their features that are going to outperform. That’s a mistake that I see people make. Probably one of the most popular figures in the book is the MVP slide that basically uses a modified version of Aarron Walter’s pyramid that talks about a product. At the bottom of the pyramid is functional. It needs to have a certain level of functionality. It needs to have a certain level of reliability. It needs to have a certain amount of usability and a certain amount of delight basically. It’s funny. Someone just emailed me, he’s like, “I love your DURF framework.” Like the D-U-R-F. It’s called the DURF framework for short. The other mistake I see people do is they will use MVP as an excuse to launch a buggy product that has a bad UX. They just say, “It’s just an MVP.” When you test that, it doesn’t test well.

The bottom line is yeah, you don’t want to build the whole set of functionality but the part that you do build it has to be usable enough and reliable enough. I’m not trying to have my cake and eat it too and say, “You’re going to have some MVP that’s like pixel perfect and bulletproof.” That’s what we do basically. Once we get clear on what our MVP feature set is then we get clear on wireframes to bring these experiences to life. We stay in low fidelity on purpose to iterate until we’re happy. One we’re happy with that, we take it to high fidelity, working with the visual designer. Once we’re happy with the visual designs, the mock ups, then we’ll basically throw them into a tool like Envision. I love Envision and make it a clickable or a tappable experience. Then we will recruit target users based on that persona definition that we had. I’ll moderate the users through those high fidelity clickable prototypes. You get a lot of great feedback when you do it that way and then you iterate.

Once you’re getting to the point where you’re getting a positive response from people, no major negatives and less negatives as you … I recommend doing these in waves of 5 to 8 people at a time and then pattern matching and addressing the top issues and then doing it again. Once you do that with high fidelity clickable or tappable prototypes, you can feel pretty confident going into development after that that you’re going to be building something that people want.

Looking back at the projects that light you up the most, what do you feel is your superpower as a product creator? What’s your sweet spot?

One of the superpowers is basically being comfortable and an expert in the problem space. In many ways, the core of my book is how to think about the problem space and do so rigorously and explicitly. Asking a lot of why questions of like who’s this for, what needs do they have that aren’t met, how do we articulate that? It’s messy in the problem space. We’re at the toughest part of a product. I just actually met with a startup team an hour ago that’s spun out of another startup. They’re like, “Yeah. That startup was great because we’re working in the payroll space and all the customers knew what they wanted. They knew it was a very mature market category. It was very clear how to improve upon what was there. Now as a new spin up we’re starting with a blank slate, blue sky. We don’t know what to do.” Basically in those environments creating structure out of nothing, putting a strong person idea out there that we can then build and iterate on and basically creating frameworks of how to think about the problem space and the competitive landscape, that’s a superpower of mine.

Then I think just rigorous ruthless prioritization. I will make people rank order a list and then we’ll decide okay, what’s most important and we’ll use that to basically make our product decisions for MVP and also for our future sprints. We get really clear on what’s important. The other thing that’s interesting is helping people with MVPs. It’s this funny thing. I have so many entrepreneurs that are coming to me and said, “Dan, I read your book. Been to your talk. Love what you have to say. Understand that MVP can’t be too big. Here’s why my MVP needs to have X, Y and Z in it.” Then I’ll sit down and tell them, “Okay. Tell me more about it.” Invariably, I’ll be able to find a way like when did you think about this? What about trimming this? What about this? There’s always 80-20. The 80-20 rule is always at play. When it’s your baby, when it’s your product, they can be really hard. It’s almost easier for someone else to challenge your MVP and come up with ways to make it even smaller scope. That’s something I often help people out with too.

Scope down.

Yes.

That’s actually a game designer’s credo. That’s something I learned in game design is scope down and find the fun early.

Yeah.

That can really help focus your MVP too. Not so much find the fun, but what’s the core loop because you can get really lost in the onboarding.

Definitely.

You could go nuts with the onboarding.

Yeah.

If you don’t have a core loop, you got a leaky bucket.

Yeah. That’s the other thing I do. In the metrics chapters, I build on Dave McClure’s Startup Metrics for Pirates framework and basically talked about that too. It’s so funny. I just gave a talk at the Traction Conference in Vancouver about growth and so many people will try to pour gasoline on the fire. Pour water into a leaky bucket without realizing they have a leaky bucket. I think again you hear people use the leaky bucket metaphor and it’s like okay, you need to make sure your product/market fit is good enough and your conversion rate is good enough before you go spend all this money that you’d otherwise would waste, right, on acquisition. In the book I focus a lot. One of the insights I had to in writing the book is everyone’s trying to achieve product/market fit. There’s actually a quantitative metric that is a direct measure of product/market fit. In the book I talk about the difference between Oprah techniques which are more qualitative, which are really important like in the problem space when you’re trying to customer needs, you have to use Oprah techniques.

You can’t survey your way out of that. There’s no way. I see people trying to use surveys way too early and that’s why we’re talking about the problem of survey’s article. I see people misusing them all the time. In fact in my book, I strongly advise against using it by NPS, net promoter score, and Sean Ellis’ survey are probably the 2 that I wouldn’t recommend using. Unless you have someone who’s really good at surveys. The quantitative side I call the Spock methodologies, and basically if you want to have a quantitative measure of product/market fit, it’s basically your retention rate over time. You pick a certain time. In the book, I talk about retention curves and cohorts. The retention curves flatten out over time. They can go to 0. If you have no product/market fit, they’re going to 0. If you have a decent level of product/market fit, they’ll taper out at around 10%. After 90 days, 10% of the people that signed up are still using it. Some other product might have 20%. Some other product might have 30%.

Whatever that terminal percentage is, that is the quantitative measure of your product/market fit. Basically retention is what you should be focused on first. It’s like when we’re saying find the fun. Make sure you’re creating value that people are sticking around and are retaining. Once they are, then you can focus on acquisition so you’re not sending people to a leaky bucket.

Yup. There’s actually a lot of that information out there about how to drive retention, that’s basically based on Skinner box operant conditioning techniques, but that’s perhaps a topic for another day. I would love to have that. Yeah. Totally. That’s awesome. Bouncing off what you just said, whose work are your paying attention to these days? What trends are you following? What are you seeing that’s new and stimulating to you?

What’s interesting is I mainly try to stay abreast of how people are able to create, again without coding, create high fidelity, high enough interactivity prototypes of their ideas. Once you’ve done this great thinking in the problem space, you have to build a prototype to test it with people. I’m just amazed that there’s constantly new tools coming out. It’s interesting because a lot of times for my clients, I’ll interview and recruit designers and I always them what tools they use. You can date designers from what tools they use. Yeah. Some of them are like, “Yeah. I use Omnigraffle.” That was the cutting edge tool years ago. I’m not saying it doesn’t serve its purpose for certain things but as a wireframing tool that was the efficient frontier. Balsamic came along which made it really easier to make it be clickable. They deliberately make it low fidelity. I think that was an improvement in tools. You’ve got tools like Envision now. You’ve got mobile optimized tools like Flinto and Marvel. There’s just more and more of these. Adobe just gave out with their tool. It’s an exciting time.

It’s never been easier to create a representative prototype that’s high fidelity, high enough interactivity to test with people. I look at the tools and then I also look at the methodologies and processes that people are following to do that. We were just talking before we got on the interview about the design sprints process. That’s very exciting. A lot of people are very excited about that it’s a methodical way to get feedback. Reading the typical product blogs and websites and hashtags on Twitter and seeing what success, what companies are doing. It’s a really exciting time because also everyone’s sharing what they’re doing. People are more and more transparent of sharing their wireframes or clickable mock ups of what the test results were. More and more case studies you see. The other cool thing is increasingly you see designers creating videos. Instead of just showing these static wireframes, they’re creating a little video of them clicking or tapping through. You can experience the user experience. A higher fidelity way of seeing it as a passive observer.

Cool. What’s coming up for you on the horizon that’s exciting?

I continue to work with companies. I really enjoy that. On my book front “The Lean Product Playbook,” I was really excited because the audible version recently came out. That was the top customer request that I have from product/market fit standpoint. I can totally relate as someone who’s very busy, I have kids. I’m working a lot. I increasingly rely on audio books to read books basically. I was really excited that came out. I also was excited. It’s got translated in a few languages so I think the Chinese version is coming out this fall. There’s also going to be a version for India and Turkey coming out. Aside from that, just continue to run the Meetup. I’ve got monthly meetup in Palo Alto if anyone’s interested in checking it out. It’s Meetup.com/lean-product. We have monthly speakers. Amy Jo Kim was a speaker. She rocked it. Everyone loved your talk on how to apply game thinking to improve their product. Then I’m giving a workshop. I try to do public workshops. I haven’t done one in awhile so I’m going to be doing one on Friday, September 16th in Palo Alto. The URL for that is ProductPlaybook.eventbrite.com.

Basically it’ll be a half day workshop where we’ll dive into a lot of the stuff that I was covering at a high level here in this interview.

That’s exciting. We’ll definitely give you a live link to that in the show notes. Dan, thank you so much for joining and sharing your tips and insights and stories. It’s fascinating.

Thanks Amy Jo. It’s always great to talk with you. It was a pleasure.