Portigal | 17. Tomer Sharon of Goldman Sachs

Podcast: Download

In this episode of Dollars to Donuts, I talk with Tomer Sharon, the Head of User Research and Metrics at Goldman Sachs. We talk about how to assess potential hires for user research positions, infrastructure for capturing and searching a body of data, and developing a practice inside a willing, yet large, organization.

Some parts of kind of pure research are creative. Probably the biggest one is translating a set of questions that a team has into okay, what are we going to do to get answers? If it was that easy to come up with an answer to that, then anybody could do that well. That’s not the case. A lot of people are having a lot of trouble with that part. So, I think that’s a creative part. You’re not going to see a beautiful painting coming out of that, but it is creative. – Tomer Sharon

Show Links

Follow Dollars to Donuts on Twitter and help other listeners find the podcast by leaving a review on iTunes.

Transcript

Steve Portigal: Greetings, humans! Thanks for listening to Dollars to Donuts, the podcast where I talk to people who lead user research in their organization.
Over the past while I’ve been putting together a household emergency kit. It’s primarily shopping exercise, and I’ve ordered a hand crank and solar powered radio, a replacement for matches, latex gloves, bandages, and air filter masks (which we made use of during a period of dangerously poor air quality recently). The last step was getting some food that will last – cans of soup and stew, crackers, single-serve breakfast cereals. There’s something satisfying about acquiring a bunch of stuff and storing it away, somewhat organized. And that led to a stray thought that I noticed – “Oh, I can’t wait to use all this great stuff!” And then I realized how crazy that sounded. I don’t want to use it! I don’t want there to be some emergency that is bad enough that I’m drinking the emergency water stored in the garage and eating canned stew, also stored in the garage. I mean, yes, we’ll eat or donate the food before it expires and replace it, but it’s a whole set of preparations that I hope I’ll not use, which leaves me with the hope for no shopping gratification, kind of a confusing way to feel.

But it did remind of the recent workshop I led with researchers in Sydney, Australia. We looked at a lot of the user research war stories I’ve been collecting, like those published in Doorbells, Danger, and Dead Batteries, and we pulled out lessons and best practices. There was – as there always is – a lot of discussion about safety and preparation. It seemed to me that people who worked in organizations with established safety cultures already had a strong baseline of safety procedures for user research and fieldwork, like texting a contact before and after a session, not going out alone, and so on. Their work cultures were strong on processes, especially for safety, so thinking about this for research was obvious. But not everyone works in that kind of environment, and plenty of researchers work for themselves, without that corporate structure to support them in creating best practices for safety. Anyway it led to a lot of discussion beyond just safety about running through possibilities ahead of time so that when any situation comes up it’s not a surprise and there’s at least a starting point already established about how to respond. I think this is a great idea, but I think we have to acknowledge the limitations – you can’t plan for every possible situation, there are always going to be things that come up that you probably haven’t ever considered. I think that some planning for the unexpected will help you to adapt in the moment to surprises, but that’s different than the false comfort of assuming you have every contingency planned for.

I hope I never have to make use of our large cache of sterile latex gloves, but maybe just having acquired them I’m in a slightly better situation for some other unexpected situation?

You can help me continue to produce this podcast for you. I run my own small business, and you can hire me! I lead in-depth user research that helps drive product and strategy decisions, that helps drive internal change. I provide guidance to teams while they are learning about their customers, and I deliver training that teaches people how to be better at user research and analysis. You can buy either of my two books – the classic Interviewing Users and Doorbells, Danger, and Dead Batteries, a book of stories from other researchers about the kinds of things that happen when they go out into the field. You can also review this podcast on iTunes and review the books on Amazon. With your support, I can keep doing this podcast for you.

All right! Time for my interview with Tomer. Tomer Sharon is the Head of User Research and Metrics at Goldman Sachs. He’s worked at Google and WeWork, and written two books – It’s Our Research, and Validating Product Ideas Through Lean User Research.

Thanks for being on the podcast. It’s great to have you here – virtually here in audio space that we’re all sharing. Why don’t you start off – just do a little introduction. Who are you? Where do you work? What are you doing?

Tomer Sharon: Okay. Thank you for having me, first. My name is Tomer Sharon. I am currently Head of User Research and Metrics at Goldman Sachs. I do have a second day job. I’m also heading a design group for a product called PWM (Private Wealth Management). Yeah, this is where I’m at in the past – well, almost a year.

Steve: Alright. So, what is Goldman Sachs?

Tomer: Goldman Sachs is, I would say, an investment bank. Probably one of the more important banks in the world. Big corporate. Definitely not one you would associated with design and research, at least not that type of research. But they are changing and they’re celebrating 150 years this year and they’re moving towards what’s called outside digital transformation and that includes learning more from their audiences and investing a lot more in design.

Steve: What’s the relationship between the existence of your role and this larger shift that’s going on?

Tomer: I think there’s a strong relationship. They have been realizing that they can’t just be living in their own box and they have to open up and try and understand audiences that they’re engaged with already and new audiences. I’ll give an example. Goldman has a commercial bank that’s called Marcus. It’s been around for a couple of years, but still these are consumers that Goldman is now trying to attract. So, it’s definitely not the kind of typical audience that they’re used to. So, they understand that they need to open up, learn from them, and design for them and with them, and that is a shift that has been happening in the past few years. And my role – I didn’t replace anyone. I’m the first one. – is a part of that shift.

Steve: Is there any sort of one point or one incident, or key moment, I guess, which sort of marks that transition in a company like Goldman to say yeah, we’ve been doing it this way, we need to do it this way.

Tomer: I think it happened – I don’t know if there was one event, but I think it happened in the past maybe two years. The user experience team there was very small and then suddenly they decided that it’s time to invest more in that. And from zero it went to several dozens, many dozens, within a year. And the more people do their work, show their work, share their work, and their work is very successful, the more teams and leaders talk about that and then it’s like a cycle that feeds itself and then it grows and grows. So, that’s been happening a lot in the past few years.

Steve: And do you have a perspective on – this awareness that you’re describing at Goldman seems to align with what I’ve heard and observed in financial services in general. That it’s an industry that maybe wasn’t seen as paying attention to consumers, users, design, and over the past number of years.

Tomer: Yeah. I will admit, this is my first financial services job. So, I’m not really familiar with that world other than Goldman. But, so I hear. I don’t really know from first experience.

Steve: But I’m inferring – correct me if I’m wrong – I’m inferring that that also is not a conversation that you’re having inside Goldman?

Tomer: No. It’s very, I would say, kind of practical and tactical. We’re not talking about the concept of having me and people like me there. We’re just focusing on doing the work that we know how to do, we’ve been doing for many years, and just bringing that insight and understanding of that world to an organization that wasn’t aware of that previously.

Steve: That world is the process and tools of learning about people?

Tomer: Um, yeah. I would say process, tools, people. They’re used to hiring different people. I saw that in Google at the time. I saw that at WeWork at the time where even formally you don’t have in the – I don’t know HR systems – you don’t have names for roles for what we do, for who we are. I can’t remember the names, but we were engineers or UI engineers, or things like that. Until you get recognized, and I experienced that at Google, and then you do have a job family for design and for research and so on. It’s a process.

Steve: So, at Goldman you’re trying to then like – that’s part of the…

Tomer: Well, I’m not – we’re too, especially in research, we’re too few people to start having a job family in the HR systems, but we’ll get there. What I’m trying to do is first a lot of evangelism. A lot of conversations with people to plant seeds in their minds that they might need somebody like that. That they might engage in a project, a one-off project, and learn more about it. And with several groups and divisions it works already. So, we have people there and they start doing their work.

Steve: It’s interesting that you’re sort of highlighting evangelism vs. like conducting research.

Tomer: Right. Right. Sometimes it’s integrated. Sometimes I’m asking people to – some kind of a leap of faith. And then we actually, we do the research and then the work talks for itself. I don’t need to evangelize anymore. They get it. They understand. They want more of it and from there the doors are open. I prefer it this way. Even in the past, I prefer to kind of do the work show then kind of wave my hands and talk about it. I found it always be more meaningful to people and more persuading.

Steve: I’ve heard this. I don’t know if this is what’s playing out for you in these situations, but that you help someone take a leap of faith. You do this work. The results speak for themselves for them. Do those results – or how can you help those results spread elsewhere in the organization?

Tomer: Um, I use that example when I talk with others and I invite people to connect. I mean in the same company you can hear from them. You can meet them. I sometimes facilitate those meetings. And then they hear from others. I know it will take time, that it’s not something that happens over a day or a week or a month. Sometimes a year. But these conversations happen, whether I’m aware of them, or not. Whether I facilitate them or not. I’m confident that the more work we do the more people we’ll have, and we’ll be more impactful.

Steve: How does that work across – I don’t know if they’re called business units. Goldman as you said speaks to different kinds of users, different kinds of customers, with its products and services.

Tomer: So, what I do – so, as soon as we have researchers joining the group, we assign them to a – let’s call it a business unit, or a product team. And then they’re theirs 100% of the time. I only support them with infrastructure, career growth and things like that. And then they do the work with the team. I do a lot of legwork kind of before that happens to make sure that they have a team that wants them, that needs them, and that knows what to expect. So, it’s not the first time that they hear about that research thing with the appearance of the person. So, that’s how I intend to kind of continue doing rather than hire to – like work in an agency model. I have a pool of people that kind of come and go kind of based on projects.

Steve: Okay, so, just to reiterate, you talk about help someone to take a leap of faith, and I’m maybe changing your words a little bit. And a next step from that is to hire someone and put them in that team.

Tomer: Yes.

Steve: And then they do the work and then the work, as you said, the work speaks for itself, but it’s the work that you’ve staffed someone into that team?

Tomer: Yeah. I’ll just add to that. In many cases it’s not really a leap of faith because there are people who are – I call myself an outsider – who didn’t grow up in Goldman, that know about this field, know about research, know about people like us. So, they’re very open to having them. So, that’s much easier.

Steve: So, you said you’re not looking to do an agency model.

Tomer: No.

Steve: From your face, I think there’s a point of view.

Tomer: Yes, definitely.

Steve: What’s the point of view behind that?

Tomer: The point of view is that I want people to feel they belong; the researchers feel they belong to a team. And I feel that if I just send them off to short term projects they can’t grow with the team. They can’t have any history with the product. They’re not familiar. They’re really like consultants that come and go. I want them to feel a part of the team, understand all the history, sit with them. I don’t look for them to sit next to me. I want them to sit with their teams. And then going to be a part of that team and then be more impactful. I strongly believe that this is the way to go.

Steve: Yeah.

Tomer: And I’ve seen that happen before with my own eyes. Not WeWork, because we didn’t do that at WeWork, but definitely at Google where we experienced, at some point, a decentralization. So, there was one UX group and they were decentralized into the different business units. Not much has changed because people were assigned to teams already, but it just felt like the right thing to do, to have people sit with their teams, work with the same team, same product, for long periods of time.

Steve: So what implications, if any, does that have for what kind of skillset someone that you’re hiring needs to have?

Tomer: Yeah. That’s a good one. So, I’m looking for more experienced people. More senior people. There will be a time where we will hire more junior people and that’s not the time when we’re just starting. And I had the same thing at WeWork when we were building a group from scratch. You need experienced people. So, I’m looking for people who are eager and passionate about building something from scratch, taking a team that maybe doesn’t know anything about user research and try and kind of build that relationship and build those results. We talked about that earlier. With that team and grow with them, grow the practice with them. And maybe if it’s extremely successful grow a team there at some point.

Steve: There’s always a thing with questions and answers, at least for me. I ask a question, I have an idea what answer I’m going to get. And it’s interesting when the answer is different. I know it doesn’t always cleave this way, but there’s sort of the soft skills and the hard skills thing. I imagined I was going to get a hard skills answer because I was thinking about some of the pragmatic constraints of this, but your answer emphasized more, I think, softer skills. Growth, advocacy, leadership from a research point of view.

Tomer: I think it kind of almost goes without saying that once you’re – I don’t know, maybe I’m wrong. But once you’re – I always like to – I tell that to my people when we hire. We look for resumes that scream researcher. So, if your resume screams researcher, then that’s covered. I’m more interested in the soft skills.

Steve: Yeah. Can I push on that just a little bit more?

Tomer: Sure.

Steve: And I think – my bias here, or the perspective I come from is the consultant. So, I come in and out. So, I know what the limitations of that are and so just from where I sit, I see the challenges of what you’re describing because – I mean there’s a life cycle of when a team is developing – there’s probably a smarter way to say that. But something is being built and it goes through stages over weeks/months/years and so the research needs change and so the way the researcher has to respond to that, or lead that, or support that changes. So, that’s kind of the – I’m just revealing what I was probing on here, what I thought the answer was going to be. When you say screams researcher in a resume does that sort of imply that the ability to – are you inferring from that resume the ability to support that team and all these different areas of its evolution?

Tomer: Yeah. A resume that screams researcher is a resume that you see all the methods, you see the flexibility in kind of not sticking to one method or one approach. Doing that in multiple cultures, multiple companies. Have a trajectory of growth. So, you’re doing kind of from less meaningful things to more meaningful things, and so on. So, that to me screams researcher. Not, you know, you look at the job titles and you ask yourself why am I even reading this resume. And then you read a cover letter that says something along the lines of “I’ve always wanted to be.” Maybe at some point, but not now.

Steve: Yes. There is an archetype of a person I think that identifies strongly with researchers and sort of what they’re about, but their experience doesn’t match that.

Tomer: And I’m smiling because there’s always an exception. I remember one at WeWork where – and I was talking – speaking the same way – only senior people. I’m not looking for more junior people. And then I hired two junior people because there’s always an exception. Because you see somebody who makes me ignore everything I just said and hire them.

Steve: What’s an example of something that overrides that?

Tomer: I don’t even know how to explain it. It’s just – sometimes it’s the spark in the eyes that you see immediately with people and you see that they are going to be like a sponge. That’s a good metaphor. And you just know it’s going to work.

Steve: And you said passion early on.

Tomer: Yeah, yeah, yeah.

Steve: And I think evidence of passion is past performance. But that’s not the only evidence of passion if you’re seeing that with people that you’re meeting.

Tomer: Right, right. Although in most cases I’m very reluctant to hire based on passion if you don’t have experience. Because you can have a lot of passion and motivation, but you know nothing. It doesn’t mean you’re going to be bad, but it means a lot of people are going to need to support. So, I think kind of my approach is that I’ll try and do zero to very little of that because we’re just starting. We’re just a few people. We don’t really have the time that we need to support our teams. We don’t have the time to support another person. Not now. When we grow, yeah, but that’s definitely – you know, if we were 10/20 senior people on the team I would say we have to have more junior people because we can’t have a team of only senior people. But right now, we’re not even close to that.

Steve: Can you say the size that you are now?

Tomer: Yeah, sure. We are four. In a couple of weeks, we’ll be five. Worldwide, both here in New York and in London.

Steve: And is there like a roadmap for number of people to get to that 10 or 20?

Tomer: Yeah. I don’t want to mention numbers, but I think we’re on a path of growth.

Steve: Okay. So, you talk about sort of the time commitment required to support people at different levels. I want to go back and clarify one thing. It was almost an aside. You said that you don’t want to do the agency model. You want people with these teams, but you’re trying to provide infrastructure. And then you used a phrase like career growth. So, given the size you’re at now, acknowledging that there’s bandwidth challenges here, so what does infrastructure look like? What does career growth look like now?

Tomer: Um, infrastructure. So, that’s easy. So, there’s a lot of – when I look around me there’s a lot of motivation to do research, but there are needs in terms of – and gaps in terms of tools, knowledge, guidance, process. So, research was happening before there were researchers, but it was kind of very kind of based on sporadic motivation from different people who are really trying to do the best they could do. So, we need to support them with services that are out there, industry standard services. So, we need to sign agreements and get those licenses on board, different vendors.

Steve: What’s an example?

Tomer: UserTesting, UserZoom. These would be two, but there are more. And, um – let’s see what else. We are kind of creating process for, okay what happens if you want to do research, but you don’t have a researcher and you probably won’t have a researcher in the next few years? But you still – you acknowledge the fact that you need research now. So, um – so we have some – we have established some kind of a way to ask for that and then sign up for office hours or something like that, and then get some help from us. We will advise you and help you kind of get going without the researcher. What else? We started working with OKRs. So, if that’s – to me that’s a part of kind of infrastructure. Or the cool kids now call it research ops. So, research ops.

Steve: So, explain what OKRs are and explain what research ops is.

Tomer: OKRs is Objectives and Key Results. It’s a goal setting approach. There are many. This is just the one that I’m used to, familiar with. And that’s just a way to set goals and see how you’re doing. So, we do that as a research group as well. Research ops – I would take the easy route and say this is everything that helps research happen without the actual research.

Steve: Are the OKRs an example of research ops? That’s what you were saying?

Tomer: So, the tools that I mentioned – the OKRs, hiring, knowledge management.

Steve: All of the infrastructure?

Tomer: Yes. If somebody wants to do – I don’t know, whatever – a usability test, and they don’t know how. So, you want them to have the tool. We want them to have a knowledge base that they can access and see okay, what do I need to do to run the usability test? And we want them to have guidance and support from a person, from a researcher who knows what they’re doing. And I would say all of that is research ops. For researchers, a big part of research ops is participant recruitment. So, finding people to learn from. That’s a big part. And something that I’m truly passionate about is kind of insight repositories. Building some kind of a repository that we can pull from later on. I would add to that any infrastructure involving measuring the user experience. So, building that. I would also include that under research ops.

Steve: So, you said you’re passionate about insight repository?

Tomer: Yeah.

Steve: Do I have to ask an actual question?

Tomer: What do you want to know?

Steve: What are you trying to get to? At Goldman, what are you working towards with that?

Tomer: It’s the same as everywhere. I, for many years, I realized I was kind of bothered by how wasteful research is. That even I felt that I’m “learning the same things over and over again.” I know that other people did research, in the same company, did research that I’m about to do and even if I get their insights I’m going to do that again. And I know it’s really, really messy and hard to retain all the knowledge that you gather from research. The second I had an opportunity to do something about it, I did. That was at WeWork. And we built a system that we called Polaris for that, to solve these problems. We identified the – it’s going to sound funny, but we identified that the main problem, the main root cause for these problems are – or is, the research report, that was what I called back then the atomic unit of a research insight and we changed that unit into – that’s why the metaphor breaks – a smaller atom, that we called a nugget, a research nugget. And that’s what we stored in this repository. So, a nugget was a combination of an observation, evidence and tags.

Steve: What was the last one?

Tomer: Tags. And then due to these tags you can then search through this database and find answers to questions that you didn’t design studies around. So, it happened many times at WeWork where people came to us and said, “what do we know about…?” or, “can we do a study about blah blah?” And we said let’s try Polaris first and realized that we have all the answers without even needing to do more research. This will only happen – you have to change your ways a little bit, not just work with a system like that – this will only happen if you do kind of continuous research – continuous and open-ended research. Back then at WeWork we did exit interviews. So, every WeWork member, customer that decided to leave, we pinged them and talked with them, interviewed them, and asked them kind of very open-ended questions such as why are you leaving? What worked well at WeWork? What didn’t work so well? If you had 15 minutes with the CEO, what would you tell him? Things like that. And then that allowed us to have answers to many, many questions because these research participants, these exiting members decided what they wanted to talk about. If they wanted to talk about – I don’t know, whatever – the price that was too high for them, or the coffee that was too great for them to leave to another place, or whatever it is that they chose, went into the system and then we could pull it out later on and then see okay we heard – and combining with – combining that with a user experience measurement system would lead you to a situation where you can say okay, I saw that satisfaction with coffee in our WeWork buildings in the Netherlands has gone down in the past month. Here is a play list of three Dutch members bitch about coffee from the past month. And then you have the what happened – the numbers. You have the why it happened from these videos. And if you’re going to “serve that” to the person that buys or decides how to brew coffee in the Netherlands then that’s half way through to the solution. So, we imagine something like that at Goldman as well.

Steve: So, it only works, you said, if you have kind of ongoing open-ended research?

Tomer: Yeah.

Steve: Why is that?

Tomer: Because if you always – so, the other type of research I had to give it a name. So, I would call it dedicated research. Dedicated research is research that you do, and you know what research questions you have beforehand, and you answer those questions. And then you can create nuggets and it’s all good. But then you’ll only have answers to those questions. When you do open-ended you have answers to questions you never imagined that you might have, or may have in the future.

Steve: What if my research question is what are the highs and lows of a WeWork member?

Tomer: So, if you do that once you’ll get a snapshot of that point in time. But if you do that continuously, all the time – and at WeWork we, at some point, interviewed -all the UX team members did that on a regular frequency. Then you have thousands and thousands of data points.

Steve: Okay. So, there’s sort of a scale here – scope or scale. I don’t even know which one it is, but there’s an amount of data that covers a breadth of topics and that is also refreshed.

Tomer: Yes.

Steve: Okay. So, what’s – I would hate if anyone asked me this question, but I’m going to ask you. What’s an insight? Because you were kind of saying a nugget is this sort of stuff, but you brought up insight. So, what’s an insight.

Tomer: I actually have a definition for that. I’m thinking about that these days. An insight to me is a deep understanding of the situation. So, you – I’m trying to think of an example. I’ll go to WeWork again. So, imagine a researcher that walks in a WeWork building, looking around, and then sees WeWork has shared open spaces, but also private offices. Okay. So, let’s say they notice in private offices that a lot of them have printers that they’ve brought in and that looks odd because WeWork offers printing. So, we have printers and we offer that as a part of your WeWork membership and if you’re going to go over you pay for more. It’s a nice stream of revenue for WeWork and when they counted it – let’s say the count it and saw that half of the private offices brought in their own printers. They go to a second building and a third building and they count, and they see the same. Half of the members that have private offices brought in their own printer. So, let’s say they stop here, go back to the office and add an insight, nugget, whatever we call it, to the system saying half of WeWork members in those buildings brought in their own printer. That to me is not a deep understanding of the situation. It’s very interesting. It may be indicative of something that’s going on that we’re not aware of, but that’s not enough. We have to understand why? So, I would encourage that researcher to knock on the doors and ask why? And then we may hear things like, “oh, you know you have a 15-page manual on how to install printers and I’m not going to waste my time on that.” Or, “you have to log in each time you go to a printer and that’s taking more time.” And so on and so forth. “We do steal your paper, so we enjoy that.” That’s what I mean by deeper understanding, so we can better understand the situation, know to answer why that is happening, have some evidence and then I would say that’s an insight. That’s a deep understanding of the situation. And to me the system that we built is going to enforce that, so you cannot submit nuggets or insights without that why part? Just facts are not enough. We don’t need facts. We need facts plus why they are what they are.

Steve: Right. There’s a behavior and what’s the reason for that behavior. I struggle when I hear people talk about insights because sometimes they talk about why a single person is doing something, as opposed to sort of why users of a system are experiencing something. Like, in your scenario, you knock on somebody’s door and say what’s up with your own printer? “It’s too hard to install.” And you come back and say – like there’s a difference between we understand why this person was doing it and then sort of the generalized conclusion. I don’t know, I’m putting my own language into your framework. It’s probably not working, but…

Tomer: That’s alright. What we would do with Polaris is gather those individual insights. So, each one would be a nugget. And then if you have 100 of these, the only difference would be the video, the person in front of the camera explaining why they brought in a printer, or whatever it is. And then you can create a playlist and show it to IT, or whoever decided that we’re going to go with this system for printing, and have them decide what they’re going to do about it.

Steve: Okay, so let me push on that a little bit. Because when you talk to say 5 people or something, about this behavior, bringing in your own printers, people are going to give related, but sort of seemingly individualized explanations. And there’s an act of interpretation, analysis and synthesis – words that we often use – to sort of say well let’s look at all of those. Like what’s the overarching reason? Talk about deep understanding. It’s not that we loose track of the fact that the installation manual is messy. There are not enough plugs. There are a lot of sort of reasons, but the larger issue is something like complexity, or not seen as adding to my business or something. There’s a higher order thing. To me, that’s what the insight is. I think you were talking about doing that.

Tomer: Yeah.

Steve: The words are slippery here when we’re talking about insights and nuggets and sort of explanations.

Tomer: Nugget is more the kind of the technical way we called it. But, I agree. And the way it happened in Polaris is through those playlists. So, we encourage people, both in – you know people who belong to the UX team, and ones that are not, to collect these nuggets into playlists and then prove a point, or do this analysis, get to an insight and share it. So, it depends on what you find and what you collect there, but let’s say you search for printers and then you get 73 results. You sift through them and you see that 15 are not really related to what you’re trying to communicate here. The rest is too many, so you’ll pick maybe 7 that the videos are really good, like “good participants” that eloquently explain the point and then you can add your analysis in writing and describe that higher insight, or deeper insight.

Steve: Right. So, Polaris kind of sets you up to make that interpretation.

Tomer: Yeah, yeah.

Steve: It gives you some structure or some way to quickly…

Tomer: Yes, yes.

Steve: Does Polaris capture that thing it facilitates you to make?

Tomer: What do you mean?

Steve: So, create this playlist. You watch the playlist and you come up with sort of new articulation. The biggest issue around printers for us is that we are doing this, and they are thinking that. Right? That’s a new piece of knowledge that’s created by reviewing what Polaris gives you.

Tomer: Yeah. Polaris was not smart. It’s just a tool. It would do whatever – it allows you to do whatever you want to do with it. So, it allows you to add text to do that. So, if that’s a yes then it allows you to do that. Polaris, in its kind of essence, is very simple. Just a tool that facilitates the kind of storage of those nuggets and creation of those playlists.

Steve: Yeah. So, that really helps me understand sort of what you’re aiming at when you talk about insight repository. It’s to me something you can re-query to come up with new conclusions.

Tomer: Yes. Exactly.

Steve: As opposed to sort of here’s all the things that we’ve concluded.

Tomer: Yes. Exactly. Exactly.

Steve: Okay. Wow. Here’s the GIF of the mind blowing up a little bit.

Tomer: You want more?

Steve: If you have more, yes.

Tomer: Now it’s just ideas and that’s not something I’m kind of working towards in Goldman, but I’m thinking – so imagine every company that should have a Polaris has it. That’s also a waste because then every company has its own repository and then I’m sure there’s a lot of overlap. So, maybe in the future there should be a kind of open…

Steve: Panopticon.

Tomer: And open Polaris, or whatever we call it, that anybody can contribute to and anybody can pull from. I’m not doing anything about that.

Steve: A friend of mine – so, this is like a third hand quote that I’m sure I’m misquoting, but a friend of mine told me this and he was quoting the head of knowledge management at NASA. This guy says, “the best knowledge management tool is lunch.”

Tomer: I can see where – I can see why that was said. Yet, I wholeheartedly reject the idea.

Steve: Yeah

Tomer: I mean, that’s not scalable. That’s not what if I went to the bathroom when that happened? What if I started working in NASA the day after that important insight was shared? I can understand the kind of anecdotal part of it, how it’s useful. But to me there has to be something – I don’t know what to call it – more solid.

Steve: Yeah. Analogously, you started off our conversation by describing the lunching that you’re doing at Goldman, connecting people, meeting people. That’s different I think than transmitting nuggets, but you are using sort of lunch, and I mean very vaguely lunch, time with people, talking to them as a way to – I don’t know, as a way to do what? What’s the difference between sort of the NASA lunch thing and what you’re doing with socializing and connecting people?

Tomer: I think what I’m trying to do is kind of socialize a discipline. And if I understand it correctly, the NASA lunch is to socialize an insight maybe. So, yeah, I don’t think we’re there yet in terms of socializing insights.

Steve: And who knows what the context of that quote, which has been quoted, which I’m requoting. It may be exactly coherent with what you’re talking about.

Tomer: Maybe

Steve: I want to loop back to something that you talked about as part of this infrastructure. You said there’s groups that don’t have a researcher and may never have a researcher. So, what are sort of tools, knowledgebase, that can help them do things? I feel like that’s – there’s a thing in our field about who does research? And I’m not even sure what the label for that is?

Tomer: Job security.

Steve: Yeah. Is that what it is?

Tomer: Should we let them do research?

Steve: Right.

Tomer: Yeah. It makes me laugh. Of course, we should let them. As if we’re the authority. But, yeah, of course. I mean why would anyone not be allowed to do research? Because they didn’t go to school? I don’t think so. If somebody wants to do it, to me that’s huge. So, we should give them everything we can to let them do it. Are they going to be doing a bad job? Maybe. But to me bad research is better than no research. It’s a first step and we are good about the tools, the socializing what we do, socializing best practices, things will get better. Yeah, there will probably be crap added in the first few times. Maybe the first 20 times. But if they really want to and they have this passion, then why kill it by saying that it’s our job, or something like that. So, yes, I’m all for “letting them” do research. Definitely.

Steve: I mean I think you highlighted exactly – so there’s – I think job security is a fear, but I think bad research is also a fear, as you said.

Tomer: Yeah. I’m okay with that.

Steve: And you said bad research is better than no research.

Tomer: To, me, yeah. 100%. Yes

Steve: I like how definitive you are.

Tomer: I’m…

Steve: Because that’s a hot topic, I think. I’ve heard people go back and forth on it.

Tomer: I know. I heard that too. I’m definitely on that side.

Steve: I also will say you’re describing ways to limit or mitigate bad research.

Tomer: Help make it better.

Steve: Yeah.

Tomer: Yeah, yeah. I mean first they need to know about it. What happens – there are so many people who develop, let’s say software, at Goldman. They’re not even aware of our existence. It’s not that they think about it and say, “uh, no, I’m not going to do it.” They don’t even know that this is happening, that we are there. So, I think there’s a long way to go. We need to kind of be more popular and be more known and then provide all the tools, help, guidance, knowledge that we can. Knowing that we can’t support everybody. It’s not going to happen. It happened even at Google that had, at the time 100s, and today probably a lot more researchers. There are teams that build stuff. Why not allow them to do research to help them?

Steve: You made the comment that research was happening before there were researchers.

Tomer: Yeah.

Steve: In the history of the world that’s also true, right.

Tomer: Yeah. True. We can’t stop that or control that.

Steve: So, maybe just a slight shift. We’re sort of talking about who’s allowed to do what, or what are researchers and what do we do? You mentioned early on that you also take on an additional role. Do you want to give some context to that? And what does that mean for you?

Tomer: Yeah, I also lead a design group for a product, Private Wealth Management. It allows very rich people to manage their money. It’s a part of a service that Goldman offers. And the digital aspect of it is not the primary aspect of it. It’s just kind of a supporting role. It’s mostly based on a relationship between an advisor, a Goldman Sachs advisor, and a client. And there’s the usual suspect of website apps and so on. I’m now leading a group of people that design that. And I mean design with the expanded way we define it. So, it’s not just designers. It’s also researchers and data people and a writer and prototyping and so on.

Steve: Are there differences between managing researchers – we talked before about people coming in with resumes that scream researchers – vs. all the different kinds of functions you’re working with on that team?

Tomer: Honestly, no. I don’t think so. I would be the first to admit it. I’m personally not the best designer. Not in the world definitely and not that I could be. But, I think there are things that are similar no matter what kind of group you’re leading. It’s good that you know something about what the group is doing, but I think it’s mostly about empowering the right people, giving them what they need, releasing them of things that are just stupid, that they don’t need to do, and they don’t need to be involved in, and focusing them on what they are passionate about. This doesn’t have any direct relationship with design or research or whatever it is.

Steve: Yeah. There’s an email list that I think you and I are both on that’s about design and user research. And there was a thread, or maybe people were having a conference call. I can’t even remember how it manifested, but the topic was researchers managing designers, which seems like it’s a newer thing. If you look historically, like research was sort of the accessory or adjacency to design, so design teams kind of managed researchers. But as research has grown there’s other people in the situation like yours where their label for themselves would lean more towards researcher, but they’re managing designers. So, it’s interesting that you sort of don’t see a difference because I feel like the thrust of this group needing to talk was hey, there’s something different here and so how are we going to deal with it?

Tomer: I know I have kind of my internal bias toward research. So, I’m probably more kind of attentive to mostly when that is not happening. Maybe than a person who would be a designer that manages designers. But I’m just guessing. I don’t know. I know that I’m definitely – I care about research and I notice and say something when it doesn’t happen. I don’t know – does it have to be a designer – designers need to know if that’s their thing.

Steve: I think I might switch gears entirely here.

Tomer: Go for it.

Steve: I’d love to just go way back, like as far back as you want to go and maybe give the story of things you did in your life to kind of get here, whether those are work or school or other things?

Tomer: That got me here?

Steve: Yeah. What’s your sort of background, or your narrative arc, if you will?

Tomer: So, I’ll tell you something from a long time ago that probably really was the tipping point, that I wasn’t even aware of that that was the tipping point at the time. I mean it’s not a secret I’m originally from Israel and I relocated, it was 12 years ago. And while in Israel I served in the Army. I signed for a career in the Army. So, the whole shebang. I was going to be a career officer for the long term. But then when I was 24, long story short, I was a paraglider. And I took a course and I got injured badly and I was out of the Army for a year. I was at home recovering. And that was a bad thing. Bad injury, but that opened my eyes. And during that year I came back to the Army and said I want to cancel the whole thing; I don’t want to stay; I want out. I will do my next job because we planned for one more job, but that’s it. I’m out. And that’s what happened. I think without that accident I would probably – I’d probably be retired by now, but I would be a career officer and not what I am today. And kind of looking back, I’m happy that that’s what happened. I would say that’s the biggest thing that affected what I’m doing today.

Steve: So, opening your eyes was realizing that you didn’t want to go on the path that you were on. Was there any hints for you of what path you did want to pursue?

Tomer: I knew it was creative. I was in a wheelchair for four months and then crutches and then learn how to walk again. And I made my way, once I was able to kind of get up, to a local artist that gave kind of very open-ended lessons in his basement, like a couple of blocks from my house at the time. So, I started painting and tried all kinds of ways to paint. And I know – I didn’t know to say that I will be an artist and honestly, I wasn’t really good at it. But I knew it would be something creative. I didn’t know exactly what.

Steve: If you look at your work today, does it match that?

Tomer: Um, not 100% overlap, but I feel some of it is, yeah.

Steve: I mean I have wrestled, mostly privately, with just the idea is research a creative field? Or are we creative?

Tomer: Some of it is.

Steve: I found myself in a collaboration with people that I think more traditionally fit that job description and I kind of had my hair blown back, just on sort of the speed and breadth of making stuff. It was definitely intimidating.

Tomer: So, I would call myself a researcher, but still I was heavily involved in shaping Polaris. That’s a product. It’s not research work. I’m now involved in creating a system that – I also lead a small team of engineers that build a system to measure the user experience. So, that is definitely more creative than maybe research. But some parts of kind of pure research are creative. To me probably the biggest one is translating one or a set of questions that a team has into okay, what are we going to do to get answers. That’s not always – if it was that easy to come up with an answer to that, then anybody could do that well. That’s not the case. A lot of people are having a lot of trouble with that part. So, I think that’s a creative part. You’re not going to see a beautiful painting coming out of that, but it is creative.

Steve: Right. I think, for me, creating the new story out of a bunch of experiences or nuggets, or whatever you’re pulling from…

Tomer: Realizing, getting to an insight. Yeah.

Steve: So, what did you do after the art class? What do you end up doing?

Tomer: I applied – I applied to what we call today, probably a visual communications program. Got accepted to one of the best ones, if not the best one in Israel at the time, and last minute decided that it’s not for me. And then I learned – I studied copywriting. So, I’m a certified copywriter, in Hebrew though. And then I took my first job – or, I worked. I worked for 3 years as – I didn’t really know what I was going to do, so I did something that I knew how to do and that was I worked in a very small consultancy for military oriented industries. So, I did that for – I was a project manager. I did that for I think 3 years. That was the time where I kind of learned all these things and really set my mind that anything army related is not for me. And then my first real job in that direction was I was – funny how names were at that time. I was an internet copywriter, which we would probably call today a content strategist, for a website that I would compare it to Monster or Indeed, or something like that today. And there I got exposed to – probably at the time it was Jakob Nielsen and people in that area. I started reading more. I did some, again what we would call today, product management work there. And then I asked them to switch to what we would call today a researcher. They said, “no.” And I was like, okay, and I looked for a company who would take me. And there was one company that took me as – again, it wasn’t researcher. It was called a usability something. And that’s it. That’s how it started. They were very brave, I should admit, because I didn’t know much.

Steve: But they took you as a researcher?

Tomer: Yeah.

Steve: So, that was sort of your first time with the title.

Tomer: Yeah. You want an even funnier story. The only person who actually knew what it was was the CEO. He was the one who interviewed me. And then he hired me and then a month later he decided to leave. So, the only person who really knew what I was doing left about a month in. But that went well. That’s how it started.

Steve: What was the point at which you came to the U.S.?

Tomer: Um, so, I had a couple more jobs and I realized very quickly that, at the time at least, there weren’t enough, or even at all, opportunities to grow into managing a group or a team of people who do that. I was always the only person in the company that did that. And I realized, or I got to the conclusion that the only place for me to work in a company that had a lot of people of my tribe would be here in the states. And I also realized the army there is not like the army here. I didn’t have any academic degree. I also realized that I needed the right degree because these companies that I was thinking about would not even read my resume. My resume didn’t scream researcher. So, I went to school. I continued working and went to school. I completed my Bachelor’s and then applied to Bentley University here and then moved. And as I was studying there I – during school there, in Massachusetts, I contacted Google and things rolled from there.

Steve: We talked about WeWork a little bit. Can you – like what was the – what was your role at Google and maybe what was your role at WeWork?

Tomer: Google, I was a user researcher, a senior user researcher. First in advertising. They’ve changed all the names by now, but at the time it was DoubleClick for Publishers. That was the product I was involved in. Ironically, I was the only researcher there in that group of hundreds of people. And after, I think, 2 ½ years I switched transferred to Search and in Search, I was first Voice Search and then we called it, at the time, Core Search – the bag of 10 blue links and how they developed from that point to what you see today as all the visual aspects of the result and so on. So, that started back then, vertical by vertical – TV, movies, music. I was doing a lot of research into search results for sports. I know a lot more than I should about all kinds of sports, cricket and so on. Yeah.

Steve: And then what was the role you took at WeWork?

Tomer: WeWork, I was head of user experience. So, started a group from scratch. The goal there, and this is kind of following several conversations with the CEO and co-founder that hired me. At the time – again, I’m sure things have changed since. But at the time WeWork had three big groups internally that kind of built or created the three aspects of the WeWork product. They were called digital, physical and community. And Adam, the CEO, felt, and he was very right, that while each group is doing a great job, sometimes if you look – if you’re going to think or look from how things are from the perspective of the customer, the member, there are gaps between those groups that we’re not even aware of. And the goal was to identify those gaps, to me that translates to research, and then solve them – solve the problems there. I’m thinking of an example. Think about conference rooms. So, conference rooms is something that WeWork offers. Members pay for it. So, somebody physically designed the conference room. An architect decided on the size and location. An interior designer decided on the mood and what will be in the room. Somebody from IT picked and AV system for that room. Somebody in digital developed a system to book this room. Somebody in community designed a policy of how to use this room. And community team members in the buildings enforce this policy. Everything is good. Everything is working well, but then situations happen. Such as members come to a meeting, they book the room and then another member is squatting the room and refusing to go out. Even if they walk in they realize that I’m a startup, I booked a room for a meeting with a potential investor and then I see a room that is designed as a music room with bean bags and no projector, clearly inappropriate for my meeting. Or, that person who’s squatting, I’m going to the community manager, but they are dealing with a leak of water from the ceiling on another member’s head. They’re all very nice, but they can’t solve my problem right now. So, this is what I’m talking about when I said the gap between one of those groups. So, we’re trying to identify those gaps – because in many cases we didn’t even know about this – and try and solve them. That was the premise back then.

Steve: You’ve written books. You give a lot of talks.

Tomer: Less now.

Steve: You have a good sort of history of creating material. You’ve interviewed a lot of people. People listening, what would you send them to, to buy, read, watch?

Tomer: Well, our publisher would not be a publisher – would be happy if I say that they should buy my book, yours too. And that would be – the name of the book is Validating Product Ideas Through Lean User Research. That’s a book for – I would only mention that because my first book is for researchers, and a lot of researchers do listen, so maybe I’ll mention both. But the second book, the one I mentioned, is for – it’s a step by step guide into answering different research questions that people have. Each chapter is a research question, step by step, on how to answer it through research that anybody can do quickly. The first book is called It’s Our Research. It’s to solve a problem that, at least at the time – I now have different thoughts about that – but, at least at the time a lot of researchers had, maybe today as well, and that is a problem that a lot of people don’t want to do research because they feel they have the answers already, or they have good intuition. And also, once they agree to do research in some cases they don’t want to act on it.

Steve: Wait, act on?

Tomer: On their research results.

Steve: What they’ve learned? Yes

Tomer: So, that’s a book that’s supposed to help with that problem. And I’m kind of having different thoughts now because my answer in that book was – and that’s why I called it It’s Our Research – make them feel that it’s their research as much as you feel it’s yours and then they would want it and they would do something about it. That was my point. Today, I’m having kind of different thoughts about how to get to a point where research is wanted and acted on. And I will try it at some point. We just need to grow a little bit at Goldman. But, my thoughts are when you – and I posted something about that recently – when you plug your own charger into the wall, do you really care how electricity gets there? What’s happening in the power plant? Why it’s working? How is it efficient? Is not efficient, and so on. You don’t really care. You want your phone charged. My thoughts about research is why wouldn’t it be the same. People have questions. Research provides answers. Yes, a lot is going on to get to those answers, but if you have a question, why do we need to bother you with all the details? Just do the thing, trust us to do the thing, and we’ll give you an answer to your question. I’m going to try that at some point. I’m thinking – it’s not political, but building a wall between stakeholders and researchers. That wall could be Slack or something like that through which stakeholders ask questions and researchers provide answers. If we have an answer immediately, if we have a system like Polaris or something like that, we can provide an answer. If we don’t, we will just ask a few questions, follow-up questions, and then do the research and get the answer. Just thoughts. I haven’t tried it yet.

Steve: Which makes me think of the research ops piece a little bit where – like building up participant recruitment infrastructure…

Tomer: Yeah.

Steve: …is an interesting one because back in the old days where we had to do everything ourselves, you’re learning about your problem by figuring how to – by recruiting. You also learn about your problem by dealing with your stakeholder and seeing what – it’s that art piece vs. this kind of process infrastructure piece and it’s interesting to think about like what is lost and what is gained? Or how is changed when you create infrastructure that – like if you’re a researcher and you’re completely decoupled from participant recruiting, that may change how you deal with people that you meet, or how you deal with framing the problem. So, for everything that we build up a process, that’s efficiency, that kind of is a query system, how does that change what we do? And who is coming to this field?

Tomer: Yeah.

Steve: These are not necessarily my own thoughts, but just things I’m hearing from people as well.

Tomer: One other thing that I would send people to is a series of Medium posts that I’ve published in the past year, maybe less, about measuring user experience. A lot of people like to talk about metrics these days. I took the HEART framework from Google and then we have a post per letter about happiness, engagement, adoption, retention and task success. And for each one, what it is, what’s important to measure, why, how, mistakes and what actions you can take from each one? So, this is something that I’m interested in these days, measurements. And I’m trying to figure out the “H” part, the happiness part, specifically. There are a ton of challenges with that. How to measure satisfaction and happiness. I’m also posting – kind of tracking my – not tracking my own life but paying more attention to when I’m exposed to requests to rate satisfaction and happiness and I share them with people, with my thoughts about them.

Steve: Okay. Great. Anything else that we should talk about in this conversation?

Tomer: I said I am speaking kind of publicly a lot less, but I do do that from time to time. I’ll be speaking in two conferences, the Face of Finance in April in New York and in London, User Research London in June, in London.

Steve: Alright. Well, thanks for taking the time to chat and sharing all the information and stories and everything. I really appreciate it.

Tomer: That was fun. Thank you.

Steve: Thanks. And so concludes another episode of Dollars to Donuts. Follow the podcast on Twitter, and subscribe to the podcast at portigal.com/podcast, or iTunes, or Spotify, or Stitcher, or anyplace you get your podcasts. Also online at portigal.com/podcast is the transcript and links for this episode (and of course all the previous episodes). At Amazon and rosenfeldmedia.com you can buy Tomer’s books and my books. Our rocking theme music was written and performed by Bruce Todd.

17. Tomer Sharon of Goldman Sachs

Show Links

Transcript

@DollarsToDonuts

Episodes

Praise for Dollars to Donuts