Cindy Alvarez

[Amy Jo Kim] Welcome, Cindy, to the Getting2Alpha Podcast.

[Cindy Alvarez] Thanks, it’s great to be here.

I’m really excited to dig in and learn more about your background and how you see the world. For those who aren’t familiar with you, give us a whirlwind tour of your background. How did you first get started in design and tech and what were the key pivot points for you along the way that led you to where you are now?

Sure, I started out as someone who might be one of the least likely to be in tech. In fact, I joked that in high school … I remember looking through those college books, dating me perhaps, that we had the physical books that you could check out from the library with the 100 best colleges. I remember looking and thinking I didn’t want to apply anywhere that had a computer science requirement because I just really hated computers, so that was an inauspicious start.

What happened is I went to college, I went to Harvard. I wanted to study psychology. I’ve always been really interested in people and how they behave. I had every intention of being a professor. As it turns out I think no one is more glad than me that I ended up taking some experimental classes and getting some bad grades that knocked me out of grad school consideration because I think I would have been a miserable academic. Anyway, in college I quickly realized that I was going to need a computer to write papers and I got it. It was the most expensive thing I’d ever bought. I realized that it would just be idiocy to own this incredibly expensive thing and not know anything about it and so I started playing with it.

It was fascinating. I just became incredibly fascinated with the computer and how it worked and taking it apart and messing with programs. Those people who are old school math geeks will recognize that … I discovered ResEdit and it was the most fascinating thing ever. Then with my newfound knowledge, I got an on-campus job in tech support. I think that was the moment when I realized … I was working in tech support in the psychology department and I was working with these Nobel Prize-winning professors, these incredibly brilliant people who could not figure out where their document went or how to get it to print.

Oh, my god.

It wasn’t their fault. It was inexplicable. You’d get these error messages that were meaningless or they would save somewhere and then they had no search capability. There was no visualization. It was this realization that people’s behaviors are very situational and technology was this incredible thing with all this potential, but it was really cutting off access to a lot of people because it was so inhuman and so I wanted to make it more human.

Wow, that’s your awakening.

Yeah.

Then what happened?

I worked tech support, I started making web pages for people. Again dating me the graphical browser, I was using a pre-1.0 version of Mosaic and then Netscape came out. People started making websites and I was one of them. That was another thing of interacting with first student groups and then small businesses who liked the idea of a website, but it was this intensely difficult process with lots of steps. You need lots of arcane knowledge to actually create them.

From there I was hooked. After college I worked as a network admin for a year, but I’d always really liked design, so I started doing design for startups. I started doing visual design and then interaction design because I really wanted to get more into the story. Then even with interaction design I found myself sometimes designing stories that I didn’t really think ought to be written and so I started doing more product management. Somewhere along the way I stumbled into this opportunity to instead of running product for a startup to build a research culture at this rapidly growing startup Yammer. I joined there and then about a year later we got acquired by Microsoft. Since then I have been working at Microsoft, which is something that I also would never have predicted for myself.

Wow, that’s amazing. Along the way you wrote a book called Lean Customer Development that is very well-known and is teaching a lot of people really valuable skills. What prompted you to write that book and share your knowledge in that form?

I’ve been in startups my entire career until Microsoft and I think even before there was really a defined role for it I saw the need to talk to customers, find out what their problems were, find out what we could sell them basically. One of those early startups was a company called Yodlee which made financial applications. When you’re selling to banks and credit card companies, their sales cycle is incredibly slow. You might get a meeting with someone every 12 to 18 months. If you don’t have something they want to buy you don’t get to talk to them again for 12 to 18 months which is a long time in the life of a startup.

I started figuring out how could we make sure that we were presenting something compelling to the Bank of Americas and the Citibanks of the world, all of whom felt rightfully that they knew their stuff and they didn’t necessarily want to hear from this pitily startup. One of the things that I did was to go out and find customers of our customers and talk to them directly which it’s not that hard to find a bunch of Bank of America customers. We could walk into a sales meeting and pitch a product and when they said we don’t really know if our customers want this I could say actually they do because I talked to them and here’s everything that they said about it. Here are the pain points they have currently and how we can address it with this product.

It really changed the tenor of sales calls and I realized the power in that. That’s something that I brought with me into other startups and then it happens that one of the next ones was Kissmetrics and one of our advisors was this guy named Eric Ries who was writing a blog post about this thing that he called “The Lean Startup”. He wrote a book, he started a movement that’s gotten crazy from there. One of the things that … One of the tenets of that is this promise of customer development, so basically the notion that you should build your customer base before or along with your product, which is really entirely backwards from the way most people have been doing it.

Then what happened?

I talked about this a lot. I found myself talking to startups a lot, repeating this a lot and realizing that this was stuff that people didn’t know. That of all of the Lean Startup tenets, customer development was the one that was hardest for people to adopt because it goes most at odds with what startup founders are good at, which is building and selling and creating the reality distortion field. I realized hey, this is a really important thing that people need to learn how do I do a neutral interview, how do I find out what people’s problems are, how do I direct someone away from the person who says that they want the faster horse and get them to realize that maybe they could try out a car. Incidentally, there’s no evidence that Henry Ford actually said that, but it’s been cited a lot.

It’s inspiring, so …

Exactly, and so I wrote a book. Yeah, which was fun, but incredibly long and time-consuming.

Is that something you’re planning to do again in the near future or not so much?

I think writing a book is like having a baby. You need a few years of amnesia to kick in, so you forget how bad it was and then maybe you’ll do it again, so maybe.

When you were writing your book on lean customer development how did you decide what to focus on, what to include, and what to leave out?

I did what anyone should do in customer development, I talked to my potential customers. I wrote an outline and I brought it to people and said is this what you want to know? What else do you want to know? What are the problems you’re having when you’re trying to figure out what to build? Of course, along the way I’ve been mentoring startups, I’m still a mentor to startups, I’m an advisor for the Alchemist Accelerator. I had this constant influx of people saying what do I do here, what do I do when this customer says this. It really … The outline wrote itself, the book I wrote and that was hard. The questions that people had were the questions that actual future customers of the book were asking me. Every time someone would ask something I would think ooh, I need to cover that.

As I went I started sending out chapters to people and saying read this, do you believe this, what else do you want to know, what am I missing? I got really valuable feedback from a lot of startup founders who would reach to me and say, okay, I buy this part, but I don’t know how to do it. Or I don’t understand how you went from this part at the beginning of the chapter to this part at the end and so I really did a lot of work with my future customers.

That’s awesome. I love that. You’ve had a lot of exposure to lots of startups, lots of entrepreneurs, and you’re in an interesting vantage point to see patterns. What are some of the most common mistakes you see first-time entrepreneurs making when they’re in the early stages of designing and testing their ideas?

Sure, so one of the most common mistakes is not getting alignment in the room that you’re in. Startup founders always laugh, they’re like oh, it’s just me and my co-founder, we’re a five-person team, we’re aligned. You’re not. I have never seen a team who could do a sticky note exercise and all write down the same things. One of the things I say is what is your hypothesis, what is the problem that you believe you’re solving for customers, and what value are they going to get by using your product? All of you write it down on a piece of paper. Don’t look at each other’s and then go around and read them.

I have always seen co-founders who sit across the table from each other 14 hours a day, oh, that’s not what I thought. That’s by far the most common mistake. When you’re not all pointing in the same direction you’re not necessarily going to get to a finish line or you’re going to get to one finish line, but no one else agrees. That’s one of them. The other one is just the simplest, falling in love with your own creation, so we want to build. I’m a creative person, I like to design things, I like to cook, I like to write, I like to just jump in. Once you do that, once you have done something you feel attached to it and so you’re really defensive of any evidence that it might not be the right solution. I really push people not to think about solutions first, to force themselves to ask questions and to really identify the assumptions that they have and say we know this is an assumption and try and actively go out and disprove it.

Think like a scientist.

Exactly.

That’s my catchphrase and I think what’s been awesome about seeing the ripples of the pebble in the pond that The Lean Startup did is so many people understanding this fundamental, experimental approach which has customer discovery in it. It’s actually learn, measure, build, not so much build, measure, learn. I completely subscribe to that. I have my own way of teaching it based on long-term engagement. It’s so interesting because as a scientist by background, it’s the experimental process. Apply it to product and you can’t be all like I’m going to publish this in science about it, you have to be really lean and mean about it.

It’s also what people actually learn in design school because I think that what Eric Ries discovered was not something that everybody didn’t know, it’s something that tech-centric founders often don’t know. I learned how to do this by working with a trained designer on a number of projects and he always started with exactly what you and I do in customer discovery with me and what we were doing because that’s how they were taught. In a sense it’s bringing design thinking, not the rigid, here’s the way you run a design sprint, here’s exactly what you do design thinking, but designer training type of thinking to start up idea development.

Absolutely. I joke that my book is a book about user research. That’s essentially what it is. There’s very little in it that any trained user research professional would know. I think the difference is that, unfortunately, historically research has been siloed off in a lot of organizations. It’s an R&D org, it’s someone who does usability testing, it’s someone who … maybe a contractor because a company doesn’t see the worth in hiring a full-time researcher. Often it comes from a more academic tradition which is less compatible with the startup lean, mean experimental. What you’ve seen is a lot of companies where research is viewed as a nice-to-have.

When I talk about customer development the reason I really liked that as a phrase is because it ties it back to being part of the decision making process. I think that’s one of the things that I brought into Yammer and I’ve been trying to evangelize to other parts of Microsoft is that you have these people who know how to do this. The problem is that they’re not sitting elbow-to-elbow with the decision makers. The really awesome insights that they’re coming up with are not necessarily making it into the product.

Better yet, they’re not necessarily allowing someone to say hey, this thing on the roadmap I don’t think we should build this. I think this is a waste of our time and not where we should be focusing attention. That lack is what really sees a company’s advantage being dulled. If there’s anything that I hope has happened through lean customer development it’s making more people, the people in the seat of decision making power realize that hey, this is really important. I need to do this and also if I have people who do this better than me I need to listen to them.

That is such an excellent and articulate statement of what needs to happen. Then there’s all these little roadblocks and frictions in reality. One of the things that a lot of my clients struggle with and I know I’ve struggled with is translating the results of research, the results of customer development, the results of interviews, the results of all of that stuff into design-ready form, into a form that the product team can digest and take quick action on. What are some of the methods that you’ve tried that work really well, or maybe that you’ve tried that didn’t work, or maybe that worked in one situation and not another from making that bridge because that’s one of the things that I’ve developed a lot of tools for and I bet you have, too.

Sure, so I’ll start with failure because people always like to hear those. Earlier on in my time at Yammer as doing research for other projects, mostly projects that were immediate, we were during short-term research, usability testing, et cetera. In talking to people something kept coming up which is people would say … We were talking to people who used Yammer, who liked it. People would say I don’t post, I don’t feel comfortable with that, I’m afraid to post. That to be honest was the classic feedback that no one wanted to hear.

When I started reporting that back I got a ton of pushback, I got flack for being negative, I got a lot of that’s not even a solvable problem, so let’s ignore it and maybe talk to the wrong people and all the classic things that you hear when you have research feedback that no one wants to hear. Just bringing that to the forefront was not helpful at all. What I actually needed to do was couch it in terms that people were more comfortable with which in our case was quantitative data.

One of the researchers on my team worked with one of our data analysts and we started saying what do we have numerically that proves that we have a bunch of people who are afraid to post and better yet that there might be some way that we could change their environment to change that. Literally just started hunting through the data and this is where researchers who ask good questions are the best thing because this woman just kept asking questions. When do people behave differently? Where are they posting? Where are people changing their behaviors?

We found some things that really should be commonsense, but so little actually is, that people posted much more in private groups. That the smaller the group was the higher the percentage of people who posted and so now we started seeing some patterns. It’s not that people aren’t going to use this, it’s that we’re asking them to do the equivalent of get up on stage and do a dance and most people don’t want to do that. If we give them more the equivalent of the closed karaoke room with just five of your friends then people are going to get up and sing. It really had to be find the pattern, prove it in a way that people will feel comfortable accepting that and then start looking for solutions.

I think that’s really been the biggest thing when I talk to other research teams at Microsoft is you might not be a product manager, but you need to think enough like a product manager that you can come up with plausible solutions. They don’t necessarily need to be the solutions we will adopt, but people need something to react to. You might be completely wrong, but if you give someone a wrong solution they’re better able to say no, no, no, we’d never do that, we’d do this instead. Otherwise you’d never get that we’d do this instead.

Got it. When you find patterns in the data, whether the data’s coming from interviews or the data’s coming from usage, are there ways you’ve learned to present it to the team and to stakeholders that are particularly effective?

Yeah, for one qualitative information will never be quantitative and you don’t do any good by dressing it up in those clothes because people will pick holes in it. I never say everyone says this. We don’t say 87% of people, that’s putting math clothes on qualitative research and it doesn’t hold up, but storytelling, here’s a pattern we’ve been seeing. We’ve talked to a variety of people in a variety of industries and they all have this problem and here’s how we know they have this problem. Here’s what we’ve observed, here’s how we’ve tried to disprove that maybe they actually have a different problem. The fundamental issue is they have this problem and we thought about what solutions could we possibly do and here are a couple of solutions that might address that problem. It’s a story, it’s open-ended, it’s suggesting something actionable, but it’s also not dictating because no one likes that either.

Have you ever tried using a job story format to express that exact thing succinctly?

I feel we do a lot of oral storytelling here, so I love the Jobs-to-be-Done framework. Just the nature of Yammer’s culture is such that what has worked for us has been a lot of storytelling. We used to believe this, then we talked to people, now we know this, here’s what we should do next.

How do you integrate play-testing and prototyping into your development process?

That’s a really interesting thing because I love the notion of prototypes and I find that Yammer and actually any really personal data-rich product there’s a very limited amount of information you can get out of prototype testing. Yammer’s one of the things … A Yammer network, the entire value of it is what’s in it. It’s the content that’s put in by your network, by your company’s employees, by the people you work with and the projects you work on. You can show someone a prototype with dummy data and it will be meaningless to them. You cannot draw any conclusions about will they use this thing or not. You can pick up on things like is this copy choice terrible, is it confusing. You can pick up on blanket usability errors like there’s 20 places to click and I don’t know where, but you won’t actually capture that value.

I found similar things at Kissmetrics, we did web analytics and again you can show people a funnel, but when it’s not their data you don’t really know what they feel about it. When I worked at Yodlee, this financial data, you could show someone, you could walk someone through a prototype of a funds transfer application and they’d say it was great. Then you’d build the thing and once they were actually trying to transfer their own money then you discovered all the problems with it because the level of attention that people pay is just different when it’s their own thing. We do prototypes to explore ideas, to uncover low level usability issues and to assess our copy and to really test our own notion of storyboarding. For usage the only thing we found that works is actually building an MVP and putting it in front of people.

Yep, and that’s where companies who have a nice clean API, so you can build a quick prototype against that data really win. That’s an issue that I definitely run into quite a bit in my work with startups. It’s not something that everybody wants to build early on, but a lot of people forget to build their API.

Yes.

You’ve had this fascinating range of experiences that go through tech support and UX design and product management and deep usability in customer development knowledge and hands on expertise. What do you see in your future? Where are you headed here?

Sure, so I think the most interesting thing I see is the potential of scaling this out. As I mentioned I’m at Microsoft, not where I would have envisioned myself, but it’s a really fascinating time at the company because we’ve had a recent leadership change. There’s really a big cultural change undergoing of wait, we need to listen to customers, we need to be more experimental, we need to learn. In some regards we missed the boat on a couple of things, the Internet for one and then mobile. There’s some things where we’re behind. I think introducing that humility has been really interesting. I see it as I work at a place right now that has over 100,000 incredibly smart people most of whom don’t know how to do what I do. If I can find a way to scale some of what I do to that immense organization I just think that’s incredibly exciting. What could we do if we actually talk to our customers.

Wow, I have some ideas on that, we should talk and you, I know, are exploring those conversations. That would be amazing to be able to really scale it. I got to tell you it’s not just you who has that problem. How do you actually scale that special combination of knowledge and technique that makes change, makes transformation?

Yeah, so much of it is not just the skills. I do a workshop for companies on customer development. It’s your pretty standard here’s why you should talk to customers, here’s the kinds of things you should be asking. We’ll do a mock interview, we’ll figure out what we learned from it. That’s your entry level. I recently started doing a second one which is the, okay, you started doing customer development, now here’s all the problems that you’re running into. I know you are and here’s how to overcome them. It’s been funny because I started out and there was this collective moment where everyone’s like oh, we thought it was just us and I was like yeah, I’m sorry. You can know how to do an interview and do a really brilliant set of interviews and still have a hard time turning that into product because there’s a whole lot of other stuff that happens.

Like we talked about earlier, how do you deliver what you learned in a way that people will accept it? How do you say, okay, what are we going to do today? How do you break down some big finding into what do we do this week and next week? How do we know that we’re still on track? How do you deal with people who say, frankly, I don’t want to change? I was good at doing my job the old way. Now you’re asking me to do something that’s less certain and it’s chaotic and I don’t have 18-month roadmaps. I’m really uncomfortable and I’m going to try and subtly undermine you. That’s what a lot of companies and not just big enterprises, I see this in startups, too, are dealing with.

Yep, and there’s a whole army of people stepping up to help them. It’s the ripples in the pond of Lean Startup and also Agile, people that … There’s been Agile coaches for awhile and many of them are very knowledgeable about Lean Startup and their specialties and people that have a lot of organizational change experience. That’s being transformed by all this customer development first effort. That leads into a question I have. How do you think of your superpower? What kind of projects light you up the most? What’s that nugget of just energy that’s your superpower?

Oh, let’s see. I love learning what people are having problems with and trying to figure out what can we do about this? I’m super tactical. Strategically, what’s my one-year, five-year plan? I don’t know, but put me in front of a … Here’s people who have a problem. They know they have a problem, they don’t know what to do about it. How do we figure out what the problem is, figure out what has worked in the past, figure out what their limits are and get them over it and actually make progress in that day by day. I guess it is a very coaching relationship, but I love building things. I think fundamentally what I get really excited about is seeing people who are faced with a problem and helping them overcome it.

I love that. It goes back to your creation story in tech support. It loops right back to that because that’s what you were doing, what you just said.

Yeah, how people behave is fascinating and you just take that and you apply it to this problem of how do we form habits that we want to maintain? How do we have a healthy relationship with technology? How do we use technology to be better humans keeping in mind that we are still humans and we sabotage ourselves all the time.

We do and then we get back up. What are you seeing that’s new and exciting to you in the tech and design landscape these days? What trends of any kinds are you following? Where are you paying attention?

One of the things I think is interesting is that I feel tech is becoming smaller and smaller and more and more personal. What I mean by that is the difference between whether you want to use this app or that app or you want to adopt this behavior or that behavior is incredibly small. I’ve seen people abandon apps because the pull-to-refresh was just a bit too slow. Or someone doesn’t like searching because the results are just a little too cramped together.

I think now that technology is something that’s everywhere, it’s in everything we do, we are allowed to be incredibly picky. That’s for good and bad because sometimes we have the perfect app and it actually … It aligns with our mental model so well that it makes us incredibly powerful and incredibly able to do new things. It also makes technology really hard to design because the things that make people uncomfortable or turn them off something are very small and they seem ridiculous.

I was at a customer site a couple weeks ago and someone was complaining about the way Yammer’s notifications worked. It was a timing thing and she was describing in this great detail about how she would be using Yammer. She’d reply to a post that mentioned her and then five seconds later she’d get a push notification on her phone. She would look at it and it would be the same post and then she would get irritated. She’s going on and on and after three minutes she stopped and she’s like oh, my gosh, I feel like such a diva complaining about this. I was like, but no, it’s annoying you to the point where it’s actually hindering your productivity.

That’s important for us to know, but it would be incredibly hard. We’d never suss that out from quantitative data. We probably wouldn’t suss it out from usability testing. You actually need to have this heart-to-heart conversation across the table looking at each other face-to-face for me to realize that this weird tiny little thing is actually a huge friction point. I think that’s really interesting that our pickiness level is going way up, but I think it also means that the potential to have the perfect match with some technology is going up.

Where do those cross, those two trends?

I think it’s again understanding what people are trying to do and what are their constraints, what makes them happy, what motivates them. If you can home in on those things you can find a solution that is really, really perfect.

Is there anybody out there whose work you’re following or articles you’re reading that’s inspiring to you, that is dealing with these issues?

I’ve read a lot of Nir Eyal and Dan Ariely and irrational thinking, Jane McGonigal and “SuperBetter”. I think those are things that I’ve been thinking about a lot is how can we trick ourselves or nudge ourselves or encourage ourselves to be better than we are. Using that as a framework I’ve really been trying to look at what are the things that work for me and why do they work and thinking about how can we better at that in general, how can we recognize … When is nudging too much nudging? When does it get into preaching?

There was a great article recently about NextDoor, the neighborhood social network. They found that people were posting basically racial profiling posts. Like there’s an African American man walking around my neighborhood looking suspicious. That’s not a crime, that’s you seeing someone of color and assuming they’re suspicious. They were horrified by this and so they’ve been revamping their crime and safety section, so that if you actually post something that mentions an ethnicity, but you don’t mention any criminal activity they’ll ask you for more details. What were they doing that was suspicious? Can you provide more information about them? It’s not saying you’re a bad, racist person because no one would react well to that. We shut down when people criticize us directly.

It’s just making you think was that guy actually suspicious? Was he looking in windows or trying to open car doors? Or was it just that you didn’t recognize him and you have a preconceived idea about people of color and maybe I don’t need to post this after all. I think things like that are fascinating. It’s looking at a pattern, it’s thinking about how do we deliver the data in a way that people will respond to and it’s creating a better environment. It may not be actually doing anything to people’s implicit biases. It’s making a more humane environment for the other people using that technology, which is a complete 180 from Twitter who’s basically we’re not going to do anything about harassment on Twitter because our numbers are going down and that just makes Twitter worse and worse.

Small, highly effective nudges.

Yeah, I think there’s tremendous power in that.

Wow, so where’s your focus these days? What’s coming up for you? What topics are you really passionate about?

Oh, in addition to getting people to be their better selves I find … One of the things and I feel we’re a little behind on the think pieces, but I’ve been playing a lot of Pokémon Go and I really like that right now. I think there’s something really interesting about the stories that came out just after it was released about all the people who were going out and walking more because they had this really fun little nudge to go do it. I think there’s things about the game play of Pokémon Go that can definitely be improved which is to say there’s not much game play.

It is a motivating factor and I find that I use that, I use Fitbit which has seven-day rolling leader boards of who’s walked the most. I use a couple of other apps that will nudge me to get a little bit more exercise and that’s a healthy thing. I think that one app on its own isn’t motivating all the time, but I rotate between them. I always have something, some ridiculous external factor that’s making me go out and walk another half mile or something. I think I’m actually fairly active anyway, I run.

I think there’s this just great visceral joy of using your body that I think technology has for a long time had the risk of trimming down. We can get so much from our smartphones, why get off the couch, why go anywhere? You can just tap with your finger instead of wandering around. I think that more than anything the augmented reality notion of Pokémon Go being fun, I think what if we could do that to make people more encouraged to reconnect with that visceral joy of using your body. You think if Pokémon Go can get you to walk an extra mile that’s pretty cool.

You think we have VR on the horizon and sure, we could use it to make some really cool first-person shooters, but you also think of how can we use that to make people feel they can do more physically than they actually can. What if you have someone who’s totally out of shape and they go to the gym and they’re walking on the treadmill and they’re out of breath and they’re just like this is so disheartening. You can use VR to make that person feel they were doing something amazing. You could make jumping on a trampoline feel like you were careening off a giant cliff. That’s super interesting and it’s not my background to actually build that stuff, but I really hope that someone is working on this.

That’s awesome. Cindy, thank you so much for sharing your time and insights and visions. It’s incredibly inspiring to talk to you.

Thank you, always good to talk to you as well, Amy.