Dollars to Donuts: Gregg Bernstein of MailChimp
Welcome to the debut episode of Dollars to Donuts. Today’s guest is Gregg Bernstein, who manages customer research at MailChimp. We discuss how MailChimp uses research to uncover new product opportunities, how the right research artifacts can best provide value to different internal audiences and how humility is an essential soft skill for successful researchers.
We research people and we try to better understand what would make them happy, what would fit into their lifestyle, what would fit into their day-to-day. We don’t go out with an idea for a product in mind. – Gregg Bernstein
Show Links
- Gregg Bernstein
- Follow Gregg on Twitter
- MailChimp
- Evernote
- Laurissa Wolfram-Hvass
- Stephanie Troeth
- Aarron Walter
- MailChimp Snap
- Square
- Web Form Design: Filling in the Blanks by Luke Wroblewski
- Jenn Downs
- Urie Bronfenbrenner – Ecological Systems Theory
- The Big Web Show
- Jeffery Zeldman
- useronboard.com
- MailChimp UX Newsletter
Follow Dollars to Donuts on Twitter (and Stitcher) and show us some love by leaving a review on iTunes.
Transcript
Steve Portigal: Thanks Gregg for being part of this. Let’s start by having you describe a little bit about your company. What does MailChimp do? What do you do there?
Gregg Bernstein: I work at MailChimp and we primarily sponsor podcasts, but there’s also a business model behind it.
We make software to make and send email newsletters. That’s the primary business right there. What I do at MailChimp is manage the customer research team. That’s another way of saying “the qualitative researchers.”
Steve: When you put the word “qualitative,” it makes me think there are other people that are involved with other types of data. Is that correct?
Gregg: We work hand-in-hand with a quantitative research team, a group of data scientists. Together, we’re like Reese’s Pieces of research. We have the chocolaty goodness and the peanut butter, the quantitative and the qualitative. Together, we get a better picture of who our customers are, and what they’re trying to do with our software.
Steve: I think the Reese’s Pieces qual/quant metaphor is something that should go viral. I hope it does.
Gregg: I’m sure it’s unoriginal. It was probably said by somebody much smarter and much more original. If you want to give me credit, that’s fine.
Steve: I’m going to attribute it to you, because I think “Reese’s Pieces” is a nice evolution. It’s not even the peanut butter cup. It’s all the way to the snackable form.
Gregg: There you go. All right, I’ll take it.
Steve: Okay. Good. What does your role entail then, in terms of leading this group?
Gregg: My role is to align the company’s business interests and the interests of our developers and designers with the information that we have on hand and the information that we need to collect.
If I could see what projects are coming up, I can speak to my team. We could figure out what secondary research data we have from previous surveys, previous interviews, from the data that we get from web traffic.
Then, I can suggest other avenues of exploration. Maybe we need to do some interviews. Maybe we need to do these surveys. It’s just a lot of managing what streams of data we have already and what do we need to augment that.
Steve: I really love one of the things you just said, which is that you’re working proactively. You’re looking to see what’s coming in terms of design and development and you’re making recommendations as to how you can inform that. Is that right?
Gregg: That is right. Typically, we have a road map for the year, or even beyond the year, of not products we specifically want to build but just ideas. For instance, what does e-commerce look like for a MailChimp customer? What we can do is we can try not to reinvent the wheel; we can look at what we’ve already done or what our customers who are e-commerce customers are already doing.
We collect our secondary data. We try to figure out what holes there are in that secondary data and then we augment it by doing a survey or going to visit customers in their workspaces, in their offices or in their homes. We try to put it all together and say, “Here’s what we already knew. Here’s what we just found out.”
Then, we can translate that into, not quite feature recommendations, but ideas about what our customers are attempting, what their day looks like, what other software they use and we try to give our designers, developers and our marketing team a fuller sense of who our customers are, what they’re trying to do and why they would come to MailChimp to do something.
Steve: The road map you’re referring to, where is that coming from?
Gregg: What’s interesting about working at MailChimp is we say that ideas can come from anywhere. Of course, ideas come from the top down. We have a CEO, he’s a very sharp guy and he points us in a direction, but we can also get ideas just from support tickets that come in.
We will see a constant complaint and we will wonder, “Is there something here that we can make better?” Or “Are we hearing about another piece of software with which we should integrate?” The difference between MailChimp and other places is that we have the agency then to say, “This is a valid idea. Let’s research it. Let’s see if there’s something to this.”
Then, we can act upon it. We can present the data to the developers and designers, and we can turn that into a either a refinement to our existing software or it can become a totally new product.
Steve: As researchers, you have autonomy – maybe not total autonomy, but some level of autonomy – in terms of where can you use the tools of research to provide value to other parts of MailChimp and ultimately to the end users you’re serving.
Gregg: As researchers, we do have the autonomy to say, “You know, I think we have some secondary data sources that could answer a question or that could point us in a direction.”
Or we could say, “You know, there’s something on our road map that we’re not quite sure about. We really need to go out into the field and study this further. Why don’t we send out a survey, and then why don’t we go visit these five cities where we see this type of activity? Let’s interview 20 people, and then we’ll have a better sense of what we’re looking at and what we need to do.”
The methodology is really up to us. The direction is there. We know not what needs to be built, but we know what we’re trying to understand. Then, as researchers, we have the autonomy to shape the research.
Then, going further, in terms of how we present that information, we do a lot of different types of sharing. Sometimes, a simple email with a few bullet points is all it takes to share what we know.
Sometimes, though, we need to travel into the field with a videographer and create pretty much a short film that we can share with everybody in the company, so that they can better understand what we’re seeing and see the space in which somebody might use MailChimp.
One thing that I have learned is that there is no substitute for seeing somebody’s office or where they work. It’s not just because you get a sense of what kind of machinery they’re using, what kind of software they’re using, but is it loud or is it quiet? Are they on a really old Dell or a brand new Mac?
There’s all this contextual data that is going to impact how our designers design. If somebody is using an old monitor, very subtle color palettes aren’t going to fly. They won’t be able to see the differences. Somebody who is using a really old Dell or even Gateway computer is not going to have the processor speed to do certain things that we might have in mind.
We’ll show a video and it kind of opens everyone’s eyes to, “Oh, not every customer is a rocket scientist.” That sounded terrible. “Not every customer has the newest and greatest equipment and software. Some people are getting by with old equipment, because their business hasn’t upgraded yet.”
We’ll do customer videos. We’ll create posters. Maybe somebody could just learn by osmosis, by walking by a poster of some different MailChimp personas. Not only are the researchers designing the studies, but they’re also shaping how we present it to get the best output and get the best impact around the company.
Steve: It seems to me that the researchers really developing the methodology of sharing is not done as much as developing a methodology of let’s just call it “gathering,” research methodologies versus sharing methodologies.
You guys sound like you’re very mindful of building a palette and expanding it, and making choices and trying new things, in order to have the impact that you want to have.
Gregg: Yeah. That’s true. We realize that a Google Doc is valuable as an archival piece of information, as a historical document, but not everyone is going to learn from reading a Google Doc. Not everyone has the time to scroll through and read everything.
We try to use as many methodologies of sharing as possible. One thing that we’ve tried to do is make everyone a researcher. We’ve done that by creating a business Evernote account, where every interview transcript, every article that we stumble across, every research artifact is in Evernote, so that anybody in the company who has a question can at least start somewhere and see, “What do we already know about this?”
For some people, that’s all they need. That’s a great way for them to learn what’s being researched at MailChimp. Other people need videos, so we try to create video artifacts from our customer interviews.
We’ll get together on a Friday morning. We’ll serve coffee and donuts and we’ll show three customer videos, each of them about 12 minutes long. Then, we’ll have a conversation about what we saw.
If we do a usability test, we’ll bring in some nachos. We’ll get the developers, the designers, the QA team involved, and we’ll show footage of the usability test. For some people, they need to see it. They don’t want to see the actual usability test. They want to see the highlights.
We try to give a variety of avenues to learn what the research team is learning. If we stick to just one, that’s going to exclude a lot of people. We try to cover our bases.
Some people like to read, some people like to watch a video, some people want to have a conversation. We want to cater to anybody who’s interested in learning about what we’re doing.
Steve: There’s a really important, big idea in what you’re saying. It reminds me of when James Cameron did “Avatar.” There was some story that went around. Maybe you know the details more than I do. But I think they created 70 final versions of it to go in a lot of different kinds of environments, depending on what the audio part was going to be, what the video part was going to be. They took the same set of data basically and rendered it all these different ways for these different audiences. That’s maybe more technical. You are talking more contextual You think about someone making a movie. They just make the movie, and they ship it. You realize what it takes to be state of the art. This is where my metaphor for you guys is, that you are doing something very state of the art. That means that you’re trying to render your data or your findings in many, many different forms in order to engage as many different people inside your organization.
Gregg: That’s a good way to put it. While we don’t quite have the budget of James Cameron, we do have agency to create these artifacts, to hire videographer, to provide our findings in whatever means we think is going to be most effective.
The makeup of our team is pretty indicative of that. One of my colleagues is Dr. Laurissa Wolfram Hvass. She has a PhD in Rhetoric. She studied how to communicate with people. She’s constantly thinking about what’s the best method to deliver this information.
Another one of my colleagues is Steph Troeth. She has managed an agency. She’s thinking, “What’s the best way to explain the business side of what we’re seeing to stakeholders around the company?” It’s a real multidisciplinary team, thinking about multidisciplinary delivery methods, if you want to put it that way.
Steve: That’s fantastic. My eyes are opened. The questions we might hear from each other in this field are “What’s the best way to do X?” Anyone with experience would say, “Oh, there’s multiple ways and it depends” but what I’m getting from you is a richer version of that, including telling the same story multiple ways and making that a competency.
Gregg: Sure. One of the things that we’re trying to do is create three artifacts for every project. That is a quick email that has just some bullet points; a link to the raw data, which might be a spreadsheet; and then a link to maybe a richer report.
At the very bare minimum, somebody who just wants to read the highlights can see the bullet points, somebody who is maybe a quantitative researcher wants to dig into the numbers can create some pivot tables in our spreadsheet, and somebody who really needs to get the extra in depth detail can read our full report.
That’s an addition to any other artifacts, like the video or poster or chart, whatever it is that we deliver. We try to at least have three different deliveries: short, long and spreadsheet style, I guess you could put it.
Steve: Are there signs you’re able to see internally that either data or information frameworks, stuff that you’re coming up with and putting out there in these different ways, do you see it take root?
Gregg: Just earlier this year we were studying e-commerce companies, as I mentioned earlier. We typically ask, “What software do you use?” or “What apps do you use?” Among these small-business owners and these small retail shops, we kept hearing about Instagram, which everybody uses Instagram.
It got us thinking, “What is the connection between MailChimp and Instagram and small business or MailChimp and Instagram and e-commerce?” We realized that a lot of these shopkeepers might get inventory.
Maybe they take a picture of it with Instagram. Maybe they just want to send a quick email right there, while they’re unpacking their inventory that’s just come in off the truck. They want to say, “Hey, you three customers. I think this is a product that you might be interested in.” Our Director of UX, Aarron Walter walked over to our mobile designers.
He said, “I have this idea. I keep seeing that our retail customers are using Instagram. Maybe there’s a quick way where you just launch an app, select a photo from Instagram library and send it to a few customers on your mailing list.” The mobile designers took that idea. They built a prototype.
We brought in some of our customers. We tested it out. They gave us some feedback, and then we shipped a product that was just based on going out and interviewing customers, connecting the dots between their behaviors, their contexts and their app usage.
We launched a product that makes it easy to send a campaign to specific customers on your mailing list, using photos either from your phone library or from Instagram. That took place in a matter of about eight months that was research in the field, collaboration between our teams and shipping a product. It was pretty rewarding to see how research influenced that.
Steve: That’s fantastic, and that’s research…were you looking for that kind of opportunity in the research?
Gregg: No, we weren’t. It was really just research into, “Who are these customers? What are they doing? What systems do they use? What is their day-to-day like?” Sending an email campaign is not trivial.
We make it quick. We make it easy, but it’s still time out of your day. Everybody’s got their phones out. Everybody’s Instagramming, so why not just play off of that and why not just create an app that fits into that mindset of quick sharing?
Not only quick sharing but quick collection, taking a quick picture, and then sharing it with a few select people while it’s on your mind, instead of making it something you have to add to your checklist of things to do as you run your business.
Steve: That’s great. What do you think the conditions are inside your organization where that was able to happen? Because the counter-narrative often is that groups like yours are finding things that are exciting and that are actionable, as opposed to just interesting from a researcher’s point of view, but there’s things that get in the way of anybody buying-in and taking action.
Your story just seems like everyone was pointed in the same direction. I wonder if you can reflect on your own culture and see what makes that happen.
Gregg: I truly believe that ideas can come from anywhere here. We are empowered to listen to our customers. We don’t go out researching products. We research people and we try to better understand what would make them happy, what would fit into their lifestyle, what would fit into their day-to-day. We don’t go out with an idea for a product in mind.
I think that subtle shift in just being interested in our customers empowers us to come up with ideas, and we can quickly validate. “No, we can build this and it will fit into how they run their business. It will fit into the time of day when they are on their devices or when they’re working.”
If that’s your operating principle, “Can we make our customers happy, and does this fit into their lifestyle?” it make it real easy to get buy-in on projects and products we want to build. Snap came about because we could see there was a real use-case for this.
Steve: Snap is the name of the product that you guys launched?
Gregg: Correct. MailChimp Snap. Time and again, when we speak to customers and we hear how they work and just what their day-to-day is like, it just makes it real easy to paint a picture, tell a story about our customers and figure out what we can do to make their lives a little easier and make use of our software – be it MailChimp, be it Snap –make that a little easier for them.
Steve: I love how you said, “We’re studying people, not products.” I’m paraphrasing it, but you put it really in a nice way. You mentioned this too, with describing their monitors and people’s setups and so on, that this issues of context is obviously really important to how you’re able to get the kind of information that provides these ideas.
Gregg: To me, I live and die with context. I go into an interview. The questions are important, but I pick up on body language. I’m also looking at voice inflection, equipment and the office culture. I think all of that is as important as the data that you collect and the answers to the interview questions.
Right now, you and I are speaking over an audio Skype connection. Not seeing your face is driving me crazy, because I just live and die on the context. It’s really important to me in how we research. I need to see the office. I need to see the equipment. I need to pick up on the office culture.
Something that I like to do is we’ll ask, “Tell me about a typical day.” Somebody might say, “Well, I got up this morning, and I got some coffee on the way to the office.” That right there is an opportunity to get some context. I could say, “Well, how did you pay for your coffee?” “I used a credit card.”
“Do you remember what kind of system they were using when you paid for your coffee?” “Oh, I didn’t really pay attention” or “Oh, they used Square.” Well, mentioning Square tells me a ton about this customer that I didn’t know already.
I know that they are pretty tech-savvy because, one, they know the difference between a cash register and a point-of-sale system. They are technically the same, but I guess a modern point-of-sale system. Two, they know the difference between Square and another point-of-sale system.
I can create a persona around this person based on what they do but also how savvy they are around technology. Those little bits of context are everything to how we research here. We spend a lot of time traveling, going out in the field and talking to customers.
Not just because we want to know them, but we want to know more about them what their day is like, what their life is like. All the things that we can’t get sitting here in our office.
Steve: That’s a really lovely case beyond empathy. There’s a lot of motherhood and apple pie kind of reasons why we should know people as real people. Not that those aren’t true, but you’re connecting the dots I think in a lovely way that says that that information helps us do our job.
Gregg: I think that’s a good way to put it. Last year, I was doing an interview with a customer, and they were a music promotion company. While we were in the middle of the interview, one of the bands they represent came by. The office for this promotions company happen to be next door to a coffee shop.
This band dropped by and they said, “You know, I think we’ll set up and do an acoustic show next door.” That context of seeing, “OK, it’s very normal for a band to drop by. That’s part of their day-to-day,” but seeing that maybe there’s an opportunity to get the word out about this. Maybe there’s a MailChimp angle.
A band drops by all the time. How can they quickly spread the word with people nearby? “Hey, they’re playing a show next door. Come on by.” There’s got to be some way to translate what we’re seeing in the context of a band dropping by. How do we translate that into use of our app?
I’m not saying we’re always looking for the business opportunity, but we’re wondering is there a way to make our app easy for this situation, because this is their day-to-day? Maybe there’s a way to make our product fit into it a little better.
Again, with MailChimp Snap, now that’s very easy to take a picture and send it out to the people nearby who are able to come and drive to the coffee shop and see that concert on a moment’s notice.
Steve: Were you in the field for that particular experience?
Gregg: I was in the field to interview that customer because they seemed interesting, but not because of that particular fact.
Steve: Let me ask you a nerdy researcher-to-researcher question, because you’ve told that story in hindsight now. As you said, not everything is about the opportunities but you were able to see that one.
If you go back to when you’re there and this amazing thing happens in the field, where suddenly context really pays off because here’s an experience you wouldn’t have thought to ask for, let alone consider that it could even happen.
At that moment, when you’re just doing the researcher thing of shifting gears to accommodate this new experience, are you having the thought about opportunities that you’ve just described then?
Gregg: No, I’m not thinking about opportunities right then and there, but I am thinking, “This is really important, and there’s something to this.” It’s more of putting a star in my notes or making it really bold like, “Band drops in.” If you look at my Google Doc from that day, it probably says, “Band dropped by!”
Then, in hindsight, as I see that we’re developing Snap, I can point to this as validation or a use-case. Noting it, being aware of it and then saving it, filing it away for later is important.
Often when we debrief after an interview, me and whoever I did the interview with, I’ll look back at those bolded notes or the things I’ve starred. I’ll just say, “There’s something to this.”
That’s code for “I don’t know what this means and I don’t know if there’s anything that we’ll build based on this or if this has any significance, but in the moment this meant something. Let’s remember this.”
Steve: Let me switch gears a little bit. Can you just talk a little about your own history with MailChimp and the history of the team, and where you’ve come from to get to this point that you’ve been describing for us?
Gregg: Sure. I’ll take you way back to my story.
Steve: Yes, please.
Gregg: That sounded totally egotistical. Let’s make this about me!
Steve: I think we should have some music come in, some way-back music.
Gregg: Can we do that?
Steve: Sure.
Gregg: [laughs] Back when I was an undergrad, I reached my credit limit where I needed to pick a major. I couldn’t take any more classes without declaring a major. Of all the majors that were available to me, advertising just seemed fine. I wasn’t passionate about it, but I figured I’d get to learn a little bit about research and writing. I’d get to be a little creative.
It wasn’t ’til I was most of the way done with that that I discovered graphic design. I stayed in school a little longer and took as many graphic design classes as I could as an unofficial minor.
I was able to work out a deal where I could take some classes in the art department, because otherwise if I wanted to actually study graphic design I would have had to stay in school for about three more years, which wasn’t really something I wanted to do.
I had a lot of friends who were in punk rock bands and who ran labels. I’d look at the albums and I’d think, “I can do this. This is what I want to do. I want to design albums.” After I got out of school, I started doing it. I guess it’s that fake-it-’til-you make-it thing, where I said, “I’m a graphic designer and I design albums.”
I did that for about 10 years. I’d pigeonholed myself into punk rock and indie rock. It was great until I was doing the same thing, day-in day-out, working with the same type of clients. I burned myself out pretty bad. I knew I liked the idea of design, but I was tired of doing it. At that time, a friend of mine was a professor at Georgia State University.
He said, “You know, we need somebody to teach design and typography here.” That’s sounded fantastic to me, because it was still design and it was still design thinking but I didn’t actually have to do it myself. I started teaching, and I loved it immediately. I thought, “This is what I want to do for the rest of my life.”
Then, I found out that, if I wanted to do it for the rest of my life, I needed to go back to school and get my MFA in design. That’s what I did. I went to the Savannah College of Art and Design and started getting my MFA in design. It was during that time that I discovered user experience. I came across Luke Wroblewski’s book about Web form design.
It changed my life because for the first time design made sense. It was just about aesthetics. It was about logic, research, testing and putting things into sequence. It was design and it was problem-solving, but it wasn’t about aesthetics.
I finished my MFA. My thesis project was on Apple’s Terms of Service and just how, for a company that’s so devoted to user experience and aesthetics, the Terms of Service looked like it was designed by a lawyers or lawyers.
I redesigned it as a user-experience design project and made them usable and readable. I studied HCI, and I studied accessibility to make sure the color palette was not going to be unreadable or unviewable.
I studied Luke’s recommendations on forms. I read every book I could about user experience design in this thesis project, and it got noticed by Aarron Walter here at MailChimp. We met for coffee.
He said, “You know, the project you did, your thesis on redesigning Apple’s Terms of Service from a user-experience perspective, why don’t you come to MailChimp and do that kind of work for us?”
I realized I could either start teaching then or I could teach anytime, but MailChimp was a once in a lifetime opportunity to do this work that was really exciting to me. That’s how I ended up at MailChimp.
One of my first projects here at MailChimp was to answer the simple question of, “Who is a typical MailChimp customer?” For this project, I worked with my colleague in the UK. Her name is Steph Troeth. Together, we set about defining who is a typical MailChimp customer.
We started by interviewing people around the company, people who’ve been here for years. We asked them, “Who do you think uses MailChimp and why?” Then we started looking at the data, looking at industries that were signing up for our products. We were looking at, “Who pays for our service versus who is free? What integrations do they use?”
We painted this picture of, “Here’s who we think uses MailChimp and here is the actual MailChimp customer.” We came up with five distinct personas, which were our imaginary MailChimp customer I guess our idealized version and then four other ones. We created these personas, and we turned them into posters that we hung up around the office.
That was really the turning point, where research became a real thing at MailChimp. It wasn’t just something that was done as a project. It was, “We have a research team, and we’re bringing value to the company.” That project gave us agency where now, when somebody had a question about anything related to MailChimp, then they could come to us.
They could say, “Can you study this the way you studied who our customers are?” That’s really how MailChimp took shape. It’s not to say research wasn’t done here previously. My colleague, Jenn Downs, who had gone on to work for a start-up, she really did a lot to revolutionize mobile testing to study how people are using MailChimp on a mobile device.
She did a lot of usability testing, but in terms of real qualitative research and ethnographic-style research, that all started about three years ago, starting with that personas project that Steph and I did. [Note: Gregg would like to clarify that he misspoke here in a way that diminished Jenn’s accomplishments; during this time, Jenn Downs was leading impactful research projects at Mailchimp, including rigorous studies of mobile email behaviors and in-person studies of small business practices.] Since then our team has grown. There’s now five of us of different skill sets. We’re all generalists, but we each have our go-to strengths.
We have somebody who’s an analyst who is looking at spreadsheets. She’s taking her qualitative data but studying it from a quantitative lens. We’ll code our data. She’ll analyze it, and she’ll generate an analysis of it. We have people who are good at sharing the research, people who are good at doing usability tests, people who are good at interviews.
We all have our strengths, and they all work really well together to…I’m sorry. I’m losing a thread here, but all of our strengths combine to help our colleagues in different departments do their work with more information and with a better understanding of who our customers are.
Steve: That’s a great history, and I like how your personal history informs your story of the team’s history.
Gregg: Thank you. I got a little long winded there, but I hope it helps. [laughs]
Steve: It’s awesome. We love detail, and we love stories. What do you look for? I don’t know if it’s any one thing or any set of things, but when you’re talking to people about joining this team, which has grown, what are some of the things that you’re paying attention to?
Gregg: I’m looking for humility, first and foremost. I don’t want somebody who thinks they have all the answers. I don’t want somebody who can’t collaborate. We look for strong collaborators, strong storytellers and a lot of humility, because if you’re not humble you’re not going to be able to make yourself subservient to the information or subservient to the interview subject.
You have to go in with the mindset that the customer knows more than you do and that your colleagues know more than you do.
I guess my operating philosophy is that everything is important until proven otherwise, and I extend that to the people I work with. Everybody’s an expert, and nobody’s proven me otherwise here. Everybody here has strengths, and they all have skills. I can learn from everybody I work with.
I look for that when I’m looking for team members, all the people I can learn from and people who are willing to learn from me and willing to learn from everybody else they encounter, be it here or out in the field. My voice just cracked really bad, so be it here or out in the field.
Steve: Given your culture of ideas come from anywhere or can come from anywhere, I’m still curious from an organizational/structural/org-chart point of view, where does your group sit within the whole MailChimp structure?
Gregg: The research team, we work very closely with our data science team. The data science team feeds data to everybody – designers, developers, marketing, support, knowledge base. All the information is being gathered by us. Everybody under the data science umbrella and anybody in the company has access to us and to what we know.
We wait for them to ask or we give it them proactively, but we are very accessible to everyone in the company. We try to share as much as we can with as many people as we can.
Steve: In hearing you describe that, I don’t know if it’s a hub and spoke or some kind of other model throughout MailChimp, how you’re engaging, who your work is being shared with. Maybe it sounds like that’s even more important or more defining for the work that you’re doing than, say, something like more of a hierarchical reporting structure.
Gregg: We’re very flat in how we work. Certainly, there are team leads and project leads. Once a project or a research finding is captured, we share it with the people who need to know it.
Let’s say there’s a developer who’s building something for an Android device. If I happen to have done an interview and the person I was interviewing was using an Android device in an interesting way, I will seek out that developer and say, “Hey, I saw something interesting. I just wanted to pass this along.”
There are formal channels of communication where reports or a video will go to the person who needs to see it first, but nothing is really a secret. We try to share, either through conversations, through meetings, through the archives that we create.
Steve: It sounds like the size that the company is at makes that feasible? Is that true?
Gregg: We’re at a sweet spot, in terms of how big our company is where we’re growing. It makes it harder to have those one to one conversations, but we still try to maintain those open lines of communication, where if I run into somebody in the hall I can have a conversation with them.
If I need to have a meeting with an entire team, that’s also a possibility. If I need to send a report to an entire department, that can happen too. There are really no limits. Nothing is siloed off. Whoever needs the information can get the information.
Steve: I had been thinking in that great research way, where your question presupposes a framework, and you’ve really changed that for me. I had this assumption that where research sits relative to other functions can be defining, in terms of what impact they’re able to have or how they’re working.
While that may be true in some cases, in your case that’s not the defining factor. It’s really more about what is the overall collaborative culture at MailChimp? That flatness that you’re describing, that integrated shared mission, seems like who are your bosses and who are their bosses seems relatively moot for the kinds of things that we’re talking about.
Gregg: That’s a really good way to put it. It’s really just about making informed decisions. Anybody who needs to be informed, they have agency to get that information. Whether that’s over instant messenger, an email or a meeting, there’s really not a lot of walls.
Anybody can ask for it and they will get what they need and more to make better decisions about the work they do.
Steve: Do you have thoughts about where you’d like to see things go, with how research is being done at MailChimp? I’m not asking so much as what’s on your growth road map, but just big-picture ideas or anything that’s tickling you?
Gregg: I really love the idea of presenting the full picture, and the full picture is that magical combination of quantitative and qualitative data. Our team is really great about going out and speaking to people and getting the context, but the stories need to be backed up with secondary data.
Lately, we’ve been taking a more holistic approach to how we work. We do an audit of every available piece of secondary data on a given topic. We find the gaps in our knowledge. We will then go out and send a survey, or we’ll set up interviews. We’ll fill the holes in our knowledge, and then we’ll come up with a novel way to share it.
It’s taken us a few years to get to the point where we now know this is how it should work. What I’m excited about is, every time I think we’ve figured out the best practice for us, we figure out a new way of doing things. Last year, it was putting everything into Evernote.
This year it’s taking what’s in Evernote, taking the data that our quantitative data scientists have, and marrying that into a fuller picture. I don’t have an idea what research will look like at MailChimp in a year, but I’m excited that every year we seem to get a little smarter, a little more efficient and a little bit more informed about how we should do our projects.
Steve: I want to ask not so much a MailChimp-focused question, but thinking about this thing that you do that supports other things that companies do, if that’s a fair enough framework.
Gregg: Sure.
Steve: What do you think research does? What’s the role? What’s the benefit of this kind of thing?
Gregg: I think it really depends on the company’s philosophy. I mean if you’re researching a product, you’re going to only find really information about your product. I think the real value of research is to go in with an open mind and that is, what can you learn from your customers independent of this product and this idea you have?
Then, after the fact, you go in and you see, “Does your product fit into what you just observed?” When my team goes in for an interview, we don’t really go in with a list of questions.
We just go in with a few bullet points that we want to make sure we somehow, some way, cover during the conversation as organically as possible because we don’t want to ask questions that lead us just to a validation of an assumption that we have. We don’t want to create this situation where we’re only seeing what we want to see.
We try to go in I guess it’s a grounded theory where whatever happens happens, and it becomes important because it’s happening in the interview, not because we go in looking for it. Does that make sense?
Steve: Did you say “grounded theory”?
Gregg: I did.
Steve: Can you explain what that is?
Gregg: Probably not very eloquently, but the idea is you let the data dictate what’s important. You don’t put parameters on what you’re looking for. For instance, one of the ways we collect information from customers is through email. They can give us feedback about our app.
This isn’t tech support, and it’s not a survey so much as just, “Send us an email about something you have to say about our app.” That is just a data dump. People will give us ideas or they’ll talk about pain points that they have, which aren’t really technical issues. It’s just how our app could make their lives a little easier if we changed one thing.
That is a perfect example of grounded theory, because we’re not looking for anything in particular. We’re not asking about anything in particular, but over time trends will emerge from those emails we get. We’ll see, “Oh, this is something we need to fix,” “This is something we need to build,” or “This is an integration that we need to connect.”
Steve: You talked about subservience to the information being an individual’s mindset, but now it scales up I think, as you were talking about that. Subservience to the information is grounded theory, I think.
Gregg: Yeah, that’s correct. Of course, we go in knowing that we’re still there on behalf of MailChimp. We’re coming from technology. We’re coming from software.
I’m less interested in specific questions about my business or software than I am about, “Who are the people I’m speaking to? What are they doing? What motivates them? Are they intrinsically motivated? If they are, that means they are going to seek out better products. “Are they using MailChimp because, their boss makes them?” Well, that changes the context of who this person is and why they are using our software.
I go in with blinders on. That’s the opposite of what I meant to say. [laughs]
Steve: What is the metaphor for…
Gregg: I go in with open eyes. I go in with clear eyes, full heart, can’t lose. I’m sorry, that’s probably “Friday Night Lights.”
Steve: The cultural references are getting thick here. Even though blinders are meant to focus you, there is something I like your imagery because there is something about a device that you attach that directs where you look.
If we could do a conceptual product design here for researchers it would be eye stretchers or something? Something that forces you to have that open eyes.
Gregg: Yeah. Take in everything. Don’t just look in one place, look around the whole room. There is a psychologist that I have become a fan of, I’m not sure that he is even alive anymore, his name is Urie Bronfenbrenner. I don’t know if you’re familiar with him. He developed this ecological model of human development. His premise was, to understand people, you have to understand not just the person, but all the little interactions that person has. The people that they speak to at work, people that they speak to at home and the people that they speak to at church, temple or what have you. Then, you also have to look at the system of government under which they live. Who’s the president? How’s the economy doing? There’s all these things that don’t affect them directly, but it still impacts their lives.
You also have to look at, what are the norms for the culture in which they live? A customer in North America has a different way of looking at the world than a customer in India or Australia. Then, you have to look at where they are in time. The person using MailChimp today versus the person who was using MailChimp three years ago, it might be very different.
This idea that you look at the entire ecology, that’s what guides me as a researcher. I don’t want to talk to this person. I also want to understand where this person fits into their office, where this person fits into their neighborhood. What kind of person is this person I’m interviewing? I really want to get that full picture. I need the context, and that helps me make a better decision about what’s important, what’s important for our developers and designers to know.
Steve: Gregg, this is fantastic stuff. I want to just do a couple of wrap up questions.
Gregg: Sure, anything.
Steve: Then, I’m going to do the researcher wrap up questions, which is to ask you if there is something we should’ve talked about, that we didn’t, that you want to make sure that people hear?
Gregg: There’s one thing that I think we needed to talk about, that is on Jeffery Zeldman’s show, his big web show. He did an interview with Jason Fried last year. They called it “Two Jews Talking About Web Design.” I’m disappointed that you didn’t start this interview off by saying “This is two Jews talking about customer research.”
Steve: We could always cut the tape and go back.
[laughter]
Steve: Cut the tape, it’s not really tape, but…
Gregg: The other elephant in the room, you and I spoke a few months ago, I gave you a really great title for this podcast, which you laughed at. I want to research and better understand why “The Steve Re Portigal” didn’t make the cut.
Steve: Maybe that could be a special episode title. I haven’t figured out if there’s going to be episode titles. I thought episode titles would just be the name of the person, but maybe we could have, if we were Colbert, we would layer the titles. It would be “Dollars to Donuts, the Steve Re Portigal: Two Jews Talk About Research.”
Gregg: I think that’s fantastic.
Steve: Do we want to leave any of this in?
Gregg: It couldn’t hurt. [laughs]
Steve: We may have to.
Gregg: No, I can’t think of anything that you didn’t ask about. I think we covered how we work and the philosophy of MailChimp research. Is there anything that you feel like we didn’t cover?
Steve: I think this is great. You’ve opened up my mind a lot. I was going to ask if you had any questions for me, but they were more accusations. If it’s going to be two Jews talking about anything, I guess we are going to get some accusations.
Gregg: I’m sure we will. I do want to say…
Steve: No, you just accused me of a couple of things. I don’t mean from others. I mean from each other.
Gregg: Isn’t it natural, though, that other people would accuse us of something? As Jews, we’re worriers, we have to worry about these things. Your book was influential in how we thought about ethnographic research.
We call it ethnographic research, but if you ask an academic, of course, what we’re doing is not ethnographic. It’s just as Kelly Goto would say, the deep-hang out or the customer interview, but your book was really a great resource for us. Thank you for that.
Steve: Thank you.
Gregg: I don’t know if you want me to plug somebody else, but there’s another person who’s doing…it’s research but it’s also UI design. I really like the work that Samuel Hulick is doing with his useronboard.com website. All he does is study onboarding from a design perspective. Onboarding into signing up for using an app, and every week he’s tackling a different one.
I highly recommend it. I’ve already learned a lot, just in how to think about a product and how to think about where people might get lost as they’re trying to use your app. It’s opened my mind into things we need to study, things that need to become formal research projects here at MailChimp. I just wanted to plug that…I don’t know if there’s a way to fit that in. Maybe we shouldn’t. [laughs]
Steve: Yeah, we’ll get that for people. That’s great. All right, thank you very much, Gregg. I appreciate your time, and all your detailed stories and your big ideas. There’s a lot to learn from, so thanks for taking the time to talk with us.
Gregg: Steve, it was a real honor to speak to you. I’ve admired your work for a while, so this was a career highlight. Thank you so much.
Steve: [laughs] Thank you.
Gregg: You’re welcome.