Dollars to Donuts: Nancy Frishberg of Financial Engines

Dollars to Donuts logo
My guest today is Nancy Frishberg, the manager of user research at Financial Engines. We discuss recruiting participants in an enterprise setting (where users are customers of your customers), finding the generative in the evaluative and how to think about collaborative workspace as entirely separate from reporting structure.

Everybody in the organization should be interested in user research and care about what our users think and have opinions based in fact, and just not see their own experience as one facet of many people’s experiences. – Nancy Frishberg

Show Links

Follow Dollars to Donuts on Twitter (and Stitcher) and show us some love by leaving a review on iTunes.


Steve Portigal: Thanks, Nancy for joining us. I appreciate you taking the time to talk to us.

Nancy Frishberg: I’m delighted to be with you Steve. It’s always good to have a conversation with you.

Steve: Oh. Thanks. You too. Let’s just start with kind of introductions. Tell me about your organization and your role there.

Nancy: I am working currently at a company called Financial Engines. Financial Engines is….We have to be the largest something. We are the largest registered investment adviser in the United States. That means that we work with organizations that you’re probably familiar with like Fidelity, Vanguard and others in that same category who we call providers. I think they call them that too.

Also with the employers – very big employers – a bunch of the Fortune 500 and 1,000 Group to hold and to manage the investment funds that ordinary employees are putting away in their 401ks. We manage 401ks for people.

My role here is manager of user research. I’m the first such manager that they have had or even full time user researcher that they’ve had.

About two years ago, the company looked around and said, “You know, we’ve been spending a lot of time talking to the employers and making sure that they’re happy. They understand that we are sharing the responsibility with them of keeping people’s 401ks safe and well-invested. What we want to do is make a better customer experience for the end user, for the employees.”

There’s a big push on. We’ve grown from one person to about 15 in the user experience group and I’m part of that big growth spurt.

Steve: You’re the first person that’s dedicated to research. Have there been others as part of that growth spurt?

Nancy: I have two contractors working for me now. Depending on how the planning goes for 2015, I may be hiring some people also or we may be rearranging our group. I’ve got a junior user researcher working with me and also a recruiter full time each.

Steve: Having a full time recruiter sounds like a luxury. I don’t know. Is it?

Nancy: I don’t think we could do this job without it, because when we want to get acquainted with our actual end users there’s a bit of a dance that we do with the employers.

We can’t just call up anybody who’s in our database. We have to make sure that the employers are on board and which group of employers.

Plus, we are restricted from recruiting people from specific employers, while there are other kinds of communications that are taking priority like enroll in our programs.

Bringing awareness to the programs is more important than recruiting to the user research effort in the big scheme of the business, even if it means there’s a pause in recruiting from certain employers for a month or so.

Anyway, there are lots of reasons why it’s helpful to have a full time recruiter. I was working with somebody who was trying to do it part time, and do a bunch of other project management. It was too much for her. We agreed having a full time recruiter would be the right answer.

Steve: How does your full time recruiter work internally to track all those different efforts going on elsewhere?

Nancy: We have a kind of coordination. The Financial Engines headquarters is here in Sunnyvale, in the Bay Area, San Francisco Valley. We have two other locations. We have the Boston office, which is where all the employer and the provider relationships get managed out of.

We have the call center, which is full of people who are licensed, financial advisers. That sits in Phoenix. There’s a lot of coordination that goes on amongst three different offices.

We are in the midst of bringing Salesforce to live to help us, so that if Boston says Acme is in their campaign period from now until the 15th December, you can’t talk to them.

We’re talking now in November, so that would be a month. You can’t talk to them for user research purposes just now. What we’re hoping in Salesforce, is we’ll be able to put a little tick in there. They won’t even come up as eligible to be invited.

What we’re doing, we’re inviting people to the user research council. Once they’re invited, we know that they’re in the pull of eligible people to participate in our studies, and then we try to pre-qualify.

We need people this we week who are 35 to 45. Or we need people who have savings in the range of 25K to 150K, or over 150 but less than a million dollars, or whatever it is.

We can do that kind of filtering with Salesforce. So far we’ve been doing it in haphazard ways, while we’ve been waiting for Salesforce to get implemented.

Steve: But through Salesforce you would have all of things that are blocking you now such as other campaigns or other factors. Those are all represented in Salesforce.

Nancy: Those will be represented in Salesforce. They are being implemented right now, you know this month, a bunch of them. They have been, some of the things that I am not even aware of have been implemented all the way through the summer until now, but the stuff that is kind of more visible to me is getting implemented this month.

Steve: I can see a difficult situation where you guys would really have to scramble to get the information to find out where you are allowed to play or not, but it sounds like the system that is being developed, technology is that these different groups have to work together and that no one wants to step on anyone’s toes, and you guys are included in the plan of how that is going to be implemented.

Nancy: I think that is a fair assessment. I should say that when I came in here as the user research manager, it was not just my director of user experience who knew what he wanted and knew what a good research program might look like, it was also the CEO who understood what customer experience ought to be and how we hadn’t yet built out all the aspects of it that we wanted to.

I have got really terrific management support for the program, and I spend a bunch of time, I think selling into the organization initially like about 4 to 6 months explaining why I was putting in place various policies, and how I would communicate and getting some advice from other people about you know you don’t want to do this, you do want to do this, stay in touch with these folks and you know I have made a few missteps and they’ve left me out of some things.

But for example, even today somebody, a perspective participant that we have been trying to schedule got our email from Financial Engines. Not related to the research program and set reply and said, “okay” to that email, and when shall we schedule our survey by which she meant our session.

Luckily the three people on her account looked at each other and wrote me a note and said, ” Is this one of yours,” and I wrote them, I was happy to write back and say, ” Thank you, yes she is and here is what happened, you know we had a scheduling problem before that email went out and am sure she just saw Financial Engines and she thought you guys were me as well,” you know. It’s all one big organization, so it’s starting to act like that.

Steve: That’s a success where, they know where to turn to and not just act siloed.

Nancy: Exactly, and for example like one of the things that we’ve learned is when we send out one of these randomly selected invitations to 5000 or 10000 people, you know the 8 million people or 9 million people who are eligible for our services so am taking very small samples if you think of it I that terms.

But when I send out one of those you know invitations to a select group, then I also send notification to Phoenix, and the adviser center and their managers, to the investments relations person, to all the admins in all the different buildings in case somebody calls in and gets that phone number and says, ”Now what is thing supposed to be?”

They can each say this is a legitimate invitation, our user research program is building you know more people in to the people who give feedback and yes please respond if you have the time.

Steve: I mean you are describing now sort of a set of best practices for reaching out to customers that are also going to be interacting with the brand or the product through other channels. Are these things that you have kind of invented or built yourself?

Nancy: Yeah, but I think I have been in enough different organizations where I’ve osmosed what is the best practice. I don’t think I completely invented it out of whole cloth. I have been involved in getting feedback from people from even before there was such a term as usability.

Steve: You’ve seen organizationally what the hiccups can be.

Nancy: Exactly, and I have worked in very large organizations I mean there were smoother and less smooth interactions among widely different departments. Now luckily this is an, I feel very privileged because this is an organization that’s not that big we only have 500 employees roughly now. Although we are serving 8 million people or 9 million people.

As more of those people become aware of our services, we’re probably going to have to grow even more because people want to talk to an adviser. Yes, I think that best practices have grown out of my experiences and other places. I just didn’t sit down and spend three days in the dark and think about it.

Steve: It seems like something that… if it hasn’t been written up and shared in the community that analyzing, you should do it…lots of people address the same kind of things you’re talking about. I think there’s some great knowledge that needs to be repeated and shared because this is a very common challenge.

Nancy: Yes. Thank you. I think it is too. I’ve been trying to figure out both what I can say how much detail I can give around it. All the things we’re talking about today are very general and that’s fine. I’d like to develop a little more and make sure that I’m not speaking out of turn.

Steve: Sure.

Nancy: We are a regulated industry. Every employee is considered to be in the position of adviser, whether or not, we actually are licensed. Our audience are end user subscribers, don’t know the difference between me and somebody who really is an adviser.

Steve: Going back, you talked about the organization the amount of support and enthusiasm for the kind of work that you’re doing. Was that already there? Or is that something that you have? Have you nurtured that? Yourself?

Nancy: I sure hope so. [laughs] I sure hope I have. I think that it was there in the sense that people could say the words. We need them have closer connections with our customers. We really need to collect feedback regularly and then everybody went through agile training. It was not just lip service but all the stuff about agile says you’ve got to do a demo every week, or every two weeks, or whatever your sprint length is.

You and I know demo doesn’t just mean demo. It means show something that you can get feedback about. I have demonstrated several ways of what kinds of feedback you can get or how you can get people to talk to you authentically and not necessarily tell you what you want to hear. I always invite people to give us candid feedback. Be brutally honest. It’s okay. Now, is the right time and certainly now before we release something that doesn’t please you and doesn’t do the work you wanted to do.

Steve: That word feedback makes me wonder about the kinds of things that you’re learning and where they fit in the development process. There is a kind of continuing between generative and evaluative. What’s the mix of work that you’re doing?

Nancy: Yes. I think that we do a lot of evaluative work but it’s also…the piece of it that’s generative is, we say, how can we get people to engage with our site more and feel better about it. It doesn’t necessarily mean that they’re spending a lot more time with us but just that they know to come back that there will be something new or that we are responding to the kinds of interactions they’re having with the site.

What do people want to know? They want to know, “Will I have enough money for retirement? How much is enough? Am I on track? What am I going to do if the market takes a big downturn like it did in 2008?”

Then, there’s a whole bunch of other questions that they have, but those are like the two or three big ones. We say, “Well, here’s a view on what you might want to know.” Then, we try to figure out how can we help you understand how the funds would grow under average market conditions or how would that be different under a projected poor market condition.

Can we visualize it? Which of the several visualizations that we think might be meaningful actually resonates with people like ordinary humans who aren’t spending all the time thinking about investments. I think some of those are generative.

Of course, we’re also looking at our current products and how they can get improved. Of course, as we move from that generative stuff closer to something that can get coded, it’s not just a wire frame or a nice Axure prototype that we’re being more evaluative I would guess.

Steve: Over the course of a given quarter…

Nancy: Yeah.

Steve: I don’t even know if the framework of generative and evaluative is fair to you. Can you characterize over some period of time what the mixes in terms of different types of questions you are asking or questions you are trying to answer?

Nancy: Let me give an example that I can actually talk about. I hope this is still true at the point when the webcast gets heard or the podcast gets heard.

If you go to the front page of, we have a social security tool. If you are 55 or you can pretend or you know somebody who is 55 or above because we really only have a seven year window of prediction, and 62 is when people first become eligible for Social Security benefits.

You can plug in some numbers, your age, spouse’s age, if you are our customer already then we know enough about you to say what your current salary is and a few other things like that. You can plug in these things and then we will say to you what your expected benefit might be at age 62, at age 66, at age 70, or anything between there.

Then, if you decide that you want to share more information with us like accounts that we aren’t managing or we don’t know about, you might have had a previous job, you might have rolled something over to an IRA, or you might have a pension from a previous job, that would also help us to predict your income for the coming period after you stop working at your main job.

I will say about that tool. We spent the first quarter I was here playing around with two or three competing different designs which I think was responsive to the generative question.

How should we represent the timeline? Do we need to use years or should we use your age? Which is going to be more meaningful to you? How can we show that the whole thing doesn’t end just because the horizontal space on the screen ends? What’s the right way to show that we expect you to live beyond 77 or whatever fits on that horizontal space?

Those were a lot of experiments we did. Some of them we did with paper prototyping and some of them we did in OmniGraffle kind of PDF prototypes. We had for a while OmniGraffle competing with Axure. One was slightly more interactive, but one was slightly more developed.

It was very interesting to watch how the folks on our team and this was mostly the design part of the team, were playing with each other and trying to express the same ideas in different ways. Then, after what I call a quarter of the design sprints, we were going along at two week intervals doing testing during each of those periods.

Then we joined forces with a technology team with a group of developers who could build out some of our ideas, and who also could make new suggestions and think about how the things that we had thought were going to work well, or would work with the existing platform that we already had going. I think that may give a little color there.

Steve: This is a product within Financial Engines that didn’t exist that you guys were putting out?

Nancy: Correct. This is a free product. That is to say, we’re interested in people who aren’t yet our customers. We don’t have a very robust retail offering at this point, but we have some. You can sign up as a retail person and get a managed account with us.

Steve: That would be me?

Nancy: That would be you, right. Somebody like you, who’s in business for yourself, rather than somebody who works for Acme, the giant corporation.

Steve: At the outset, you were describing how the creation of the user research role was about talking to these end users. There was already a lot of interaction with the providers, the Acmes.

Nancy: Acmes are the sponsors. Providers are the Fidelitys and the Vanguards.

Steve: Try me again, then. If I’m an employee, I work at a big HMO, and my HMO is a customer of Financial Engines, what’s the label for them?

Nancy: They’re the sponsor.

Steve: They’re the sponsor, okay.

Nancy: They’re the sponsor of the platform that we offer.

Steve: They might have a contract with Fidelity.

Nancy: They will have a contract with only one, either Fidelity, Vanguard, Xerox, Mercer…there are about a dozen or 15 companies in the United States that offer those services.

Steve: They are using your products to provide their services and more to the sponsor.

Nancy: We have two different configurations of how we do it. This may be more detail than you care about. We have what’s called a provider partner relationship with some of them. This, actually, is how we work with Vanguard. That means that they use our engine, the calculator, the algorithm, the secret sauce of Financial Engines, they use it, but they provide their own adviser center. They don’t go to our Phoenix center. They don’t send their employees to the Phoenix center.

On the other hand, Fidelity and several of the other ones, Hewitt, Mercer, Xerox, I can’t remember if there’s another, they work directly with us. We have the contract to be the adviser center, as well as to provide the online advice product, which is the do it yourself product, as well as what’s called professional management, where we do it for you.

Steve: That’s not too much…

Nancy: Those are the…go ahead.

Steve: I was going to say, that’s not too much detail. We’re talking research. It’s researchers talking, so we like detail.

I guess that leads to my question, which is you’re brought in because of an identified need to focus on the end user. There was already relationships and interactions with these institutions, across these different business models for the relationships, but I’m wondering if your work encompasses interacting with those institutions.

Nancy: I have had some conversations with the employers. I would love to have more. I’m eager to get in front of the employers. I think if they understand what the user research program is about, and as I’ve said to our internal relationship managers, the salespeople and relationship people who work with the employers, I say every one of these companies has a user research program about their own customers.

I can name people for you from several of these companies whom I’ve met or trained in different techniques, interacted with their conferences. If we need any support from inside Acme, I know Acme’s user research people, or I’m only one step away, probably. They haven’t taken me up on that too much yet. For example, I am learning that the employer, for example, is reluctant to have us send too many different kinds of email to their employees.

They want the employees to be doing the work that they hired them for, and not to be spending all their time worrying about the various vendors who are providing benefits. We’re just one of several vendors that provides benefits. Of course, we’re the most important one, because that’s where we are. It’s the center of the universe. [laughs]

What I learn, then, is the reason they want to see a copy of the letter that I like to send out to invite people to participate in our studies is they want to send it on to their IT department to say, when somebody says, “What is this? Is this junk? Is this spam? Is this phishing?” The IT leader and the HR person, who’s our contact, can both say, together in one voice, “No, no, this is a real program. This is coming from Financial Engines. This is going to make your 401k management better.”

Before, I was worried that they wanted to wordsmith or restrict me in some way. As I’ve been talking to them, I realized, no, they’re my partner. I need them to understand what I’m doing so that they can advocate for me on the inside.

Steve: It’s analogous to what you said earlier, you’re presenting a consistent voice, and it’s clear where things are coming from.

Nancy: It’s also fun…from the inside people, it’s okay. I can use them all in our studies. They are all professionally managed, eligible to be professionally managed by Financial Engines. All of us have the same thing, but you know what? Our employees are no smarter about investments than the rest if the employees of other companies.

Some are very savvy, and some say, “Do it for me. I don’t understand it. I have other things that I like to spend time on, so go ahead and do it for me.” We’ve got the whole range of people who work here. Not everybody is a savvy investor here.

Steve: It sounds like you’re heavily focused on the end user, in terms of where you’re learning from that’s going to inform design. Sort of the relationships that Financial Engines have with these different kinds of organizations. That’s not what user research is supporting.

Nancy: Correct. The business-to-business stuff sometimes needs surveys done, or the relationship managers make a bunch of phone calls to the 6 or 10 people that they coordinate with, bring back that feedback. But I haven’t been involved in a lot of that feedback with the institutions, correct.

Steve: I want to talk a little about your team. We started by talking about the person who’s focused on recruiting. You said that “fingers crossed,” there’s a possibility to grow your team. In general, imagine yourself talking to candidates. I’m wondering what kind of things you look for, that say this person’s a good researcher, or someone that I want on my team.

Nancy: I had an open req for a little while this past fall, and then we had a re-org, and I lost my req, of course. This is the story of corporate America. I had a great time talking to six or eight people who were interested in the role, and some of the attractive things that I found among those people that I was talking to are…what I was looking for something, was somebody who, at baseline, understands qualitative research and has some experience with it. I didn’t want to have to explain what that meant.

Luckily these days, there are undergraduate and graduate programs, unlike when I was a kid, where people get this. They don’t have to learn it on the street like I did. The thing that I was really looking for was people who were not like me. People whose additional specialties were in other areas, so that we would have more techniques that we felt confident in and more perspectives, and so on.

I found a couple of people who sounded like they were great storytellers, and could make that story come alive. I found one or two people who had a lot of experience with mobile, which is something I don’t have enough experience with, although I’m getting better. That’s what I was looking for. The baseline and then some differentiator.

Steve: That’s a good way to put it. We use those words, junior and senior, when we’re hiring. Do you have a vocabulary or sense of what that continuum is?

Nancy: Junior is somebody who’s got the book learning, and may not have had a lot of practical experience. The practical experience includes two things. It includes a facility in handling the end user interactions. How do you conduct a session? What’s worrisome, what’s not worrisome? What’s the garden variety mess up you don’t have to get excited about, and what’s a big deal? The other thing about the junior is, if it’s somebody who comes pretty close to right out of school or a training program with…

I haven’t had a chance to interview anybody who’s come out the General Assembly program, so I don’t know whether I’d think of them as great or not. This is somebody who may not have a lot of corporate experience, like an academic. They might not know what to pay attention to, and what the coded communication inside corporate America means.

We had a meeting earlier today, in the company that seemed last-minute. It showed up on my calendar early this morning. I thought, “All-hands meeting. Now I can’t have my 1:00 meeting with the guy that I was coordinating schedules with. We’ll have to put that one off until tomorrow.”

What’s an “all-hands” meeting? Why would there be one today, there’s one tomorrow? What big announcement was it? Somebody who’s going to be properly aroused. I don’t mean flustered, but, “This is important if they put an all-hands meeting, even for just a half hour, on the schedule the same day. We ought to show up for that.” Somebody who knows that’s important, that helps.

That’s also, like, how you do business in a larger organization than just the people you can count on your hand?

Steve: This makes me think, how do you explain what you’ve learned in research in a way that’s relevant to different kinds of stakeholders, who have different concerns, different orientations, and different vocabularies?

Nancy: I don’t even know if I have a good example for that one, but in a single day I talk to a lot of people in consumer product marketing. I talk to relationship manager, I talk to the technology people, I talk to HR, I talk to the brand group, I talk to the corporate communications people, I had a quick conversation with the CEO right before that all-hands meeting. I can and should be able to talk to anybody and get them aroused in the proper way about our efforts.

Steve: You described a bunch of different functions, but where in the organization does your group…where are you guys?

Nancy: User research was part of user experience until this recent re-org that I mentioned and currently we are in the same macro-organization but we now have marketing divided into two groups, one is the institutional and, I think they call it institutional and distributional services or something like that, that’s the sponsors and the providers, the Acmes and the Fidelitys and Vanguards.

The other side is called consumer and product marketing so I’m part of that larger group and I’ve now been joined up with the consumer marketing and we are about to fill the role of director of customer insights and so user research as conceived today is going to report to customer insights.

Steve: That sound like, we were talking about code words in corporate America. That sounds like what we might label market research versus design research, is that right?

Nancy: Correct, right, so I think what my group is doing is, guiding, I don’t want to say we’re doing all the work, but we’re guiding the user and product research and the director is going to be guiding and perhaps conducting or hiring out for more of the market and customer research, although customer in this case has double meaning, right?

Steve: Where you are in the reporting structure, that’s not the same as who you collaborate with. I liked your stories before about working with designers. Is that going to continue as the structure evolves?

Nancy: Oh yeah, no. I still sit with the designers. Just because our organizational structure is one way, doesn’t mean we can’t have the landscape of the physical structure be a different way, so we sit among the designers. We have a very open seating area. This particular assignment and the one immediately before it which was also for another financial institution, are the first times in my career, which is long as you may know where I have not had an office, a physical office. Pretty bizarre.

Steve: What do you have?

Nancy: I have a table, I have two screens, I have a laptop, I have a headset, I have all the corporate tools and we have a lot of conference rooms, so if I want to have a private conversation like I’m having with you now, I reserve a conference room but otherwise I stand at my desk or I sit at my desk.

We now have standing desks which is very fun. I stand at my desk at my cushioned mat thank goodness, so my knees don’t give out, but I do like the standing part and people come by or I go wandering the halls and have quick conversations with a bunch of them, and we communicate via Team Track which is our tool for tracking everything from, “I need legal signature on this contract,” to, “The light’s out in the ladies room.”

Steve: Is that the whole organization that is like that, is it called open office?

Nancy: No, you mean what the physical layout is?

Steve: Yes.

Nancy: Yes, we have an open office plan I guess, or cubicles or whatever. I sit in a kind of round area where there’s three of us on one side. Three of us make one part of the arc and then three people make the other part of the arc. That’s the inside of the UX bullpen and then on the outside of the arc there are people sitting also, kind of interlaced or back to back arcs of three or four desks.

Steve: We’ll put a graphic of that in the show notes, no we won’t.

Nancy: I can give you one of those! [laughs] For me, the jury is out on whether this is a great way to work or not. I haven’t figured how to work with the noise level as effectively as I should, or could maybe, as other people do. I also don’t keep my headphones on all the time with music going, so I’m sort of in between.

Steve: It’s interesting, we started talking about the relationships and collaborations across the different functions and then we got into the physical layout. They seem acoustic otherwise. It seems very compatible. You’re talking about moving around and talking to everyone, and integrating with different functions.

You have one kind of collaborative structure, and you have one reporting structure. It sounds like that openness in general is part of how you’re approaching this work.

Nancy: I think that’s true. Also, I like to say of myself, and you may have heard me say this before, “I’m a lumper, not a splitter,” which is an anthropological term.

It talks to paleology, how people think of whether it’s multiple species or a single species, and I’m always lumping. I’m happy to talk to everybody, and I don’t see any barriers.

Everybody in the organization should be interested in user research and care about what our users think and have opinions based in fact, and just not see their own experience as one facet of many people’s experiences.

Steve: It sounds like everything you do is towards supporting that. It’s a cultural element. It’s kind of a shared belief, a shared value.

Nancy: Yeah. I don’t think I’ve gotten there all the way yet. I think there’s plenty more to do. I don’t think we are doing all we can do to report out results from different projects for example, so there can be more shared learning’s.

Nobody likes the purple arrow. I made that one up. Nobody likes the purple arrow in our tests. Then you should be able to tell that to another group, who have already started incorporating the purple arrow in their stuff to say, “Are you finding any differences?” “Oh, we haven’t even asked about that. We assumed that it was good.” You need for everybody to pay attention to those results.

Steve: In an ideal world, how would you get the purple arrow information out to the rest of the org?

Nancy: There’s two or three ways. One way is the designers share it among themselves. In so far as it’s just a purple arrow, or a layout issue, or a color, or a font, or a order of presentation, information architecture kind of thing. The designers can go a long way to making that happen.

However, sometimes you need to understand what the marketing people are trying to accomplish, and what the meaning behind all of this is. “Why are we even incorporating a purple arrow. Purple isn’t our brand color.”

What you realize the purple arrow represents is this is the way that people know they’re still on our site. It’s the thing that is somehow connected to our brand for them.

Of all the vendors that my company does business with that I have access to, the gym, the healthcare organization, the retirement funds, the long term care insurance company. Whatever they all are, right?

Because big companies offer a lot of different kinds of benefits. “Oh, when I see the purple arrow, then I know that’s the retirement one.”

Actually, we have such a thing that we’re trying to decide, “Must we keep it, it’s so familiar? May we transform it, and have a new familiar?” How hard will that be to keep the brand loyalty if we change that symbol that people are familiar with?

Steve: I’m sorry. You said there was a couple. Did we get one, or did we get both of them?

Nancy: A couple for which? Tell me again. Oh, how do we communicate? That was one, yes. Another is now that we sort of have a handle on how to do recruiting, and how to make studies happen on a regular pace, I’m working on the next phase, which is, “How do we share results widely? How do we share raw data widely?”

Just to pick on brand again, there are brand relevant results in many, many of the user studies that we do, even 15 minute knit ones, or half an hour conversation about something. You think has nothing to do with brand. People bring up things that to me ring, “Brand has to hear this.”

I want to find a mechanism where I can say, “This one has a brand thing in it, and it’s about 17 minutes in. This recording brand ought to listen to, especially at minute 17.”

I have a tool that I am getting approved by all of the powers that be. We’re going to do a trial on it pretty soon, which will allow us to upload video, upload audio. That means WebEx, and go to meeting sessions. As well as anything live in person that we do, plus any documents that we have, which might be design documents, or white papers, or external surveys that are done, and other kinds of industry reports, and then organize them in the way this tool calls it. They call them “channels.”

I could have a brand channel in there. That brand can put stuff in there, but I can also point to things from other, Project A, B, and C, and just take a clip up of Project A at around minute 17, because that is relevant to brand, and put that clip in the brand channel.

Then let people have a chance to, if they have a question like, “Well, what is the meaning of the purple arrow?” It’s not a purple arrow, but that kind of thing.

Then they would have a whole bunch of examples where people talk about it, and be able to reflect on, “Is this fatal? Are we stuck with the purple arrow forever? Shall we see if we can take all of the meaning that’s accrued there, and transfer it to another symbol or phrase?” Or whatever the right carrier for it is.

It wouldn’t be me evaluating all of that stuff. It would be making all of that available to other people to evaluate, because I don’t pretend I understand all of the business requirements and constraints, and desires.

Steve: That raises an issue for me that conventional wisdom says that you are at risk for sharing raw data, and this is sort of the cliché. It’s us and them that researchers have. That they, who aren’t us, will take a single data point and make a decision about it.

One person at minute 17 talking about brand is either worthless, because it’s rejected. It’s just what one person said, or it’s magnified in importance, because it’s a story that can be held onto and retold.

It sounds like culturally you guys are coming from a different place. I wonder if you have thoughts about those disaster scenarios that I am describing?

Nancy: I worry a little bit about that. I also have great hopes in the wisdom of the crowd, and the fact that many perspectives… This is too many cooks spoil the broth versus many hands make light work, right? You know, the dueling efforts, or something, right? I’m going to come down on the side of, “Many hands make light work.”

I think that many perspectives on whatever the brand element is, or whatever the component is. Is this the best way to express, “Buy our thing now?” I think that having the folks who know what sponsors think and what they value, and having the folks who are very familiar with email marketing techniques and successes.

You know, all of the different aspects of how we reach people, and engage people, and enroll people, and sustain them as they go along with their saving programs. They borrow against their 401k. They repay their 401k.

There are all those different things that we’re going to benefit from having a lot of different eyes and voices in there.

Steve: If it were me, because what I hear you doing anyway is fostering a conversation about how to think about users, how to think about real people. That these artifacts, which they distort because they’re edited, but they also reveal because they’re edited.

Maybe this is you creating a different literacy around this very different kind of data. Especially, for a financial company, this must be very different kind of data.

Nancy: The people who have been with a company for a very long time, and we have a whole other research group that’s called “Investment Research.”

It’s very funny when we have recruiting on our “Careers” page. If I’m recruiting for a researcher, and my colleague in Investment Research is recruiting for a researcher, really the job descriptions are so different.

I’m trying to see if I can get the HR Department to start calling us each by slightly different names, put a noun in or an adjective ahead of that researcher.

Yes, I would hope that there’s a lot more literacy around it. We had a little reading group for awhile, where the folks in Investment Research had a lot of questions. I’ve been trying to introduce them to some of the UX literature. It’s so wonderful, because you get to see the light bulbs go off all over again.

They ask questions about, “Well, why is this, this way on the front page, or on this third page of the experience? This doesn’t make any sense,” and then they click on something.

I say, “This is a good example of Fitt’s Law. Let me explain Fitt’s Law to you.” Then there’s all these subtleties of, “How do you arrange things on a page that are governed by principles that these folks have never even thought about?”

They appreciate more that it’s not just choosing a nice color and typing the words out in the correct spelling.

Steve: I also like in your story that you are blurring research and design. You’re talking about things that, where something goes in a page, that’s a design decision not a research decision. I’m feeling silly a little bit, but it’s all part of what are working on.

Nancy: Absolutely. Sure, the research that we’re doing is listening to real…we provide a context of the conversation between our company and our real consumer end users about what they want to do with their retirement funds.

Do you trust somebody else to manage your funds for you? Do you feel capable to manage your funds for yourself? How are you making that decision? How are you and your spouse making that decision, if you’re married? Then, do you have a clear sense of what your choices and your options are?

Just in the Social Security tool example, if you are closing in on social security benefits, you go to the social security administration office, if you are lucky enough to have one nearby or you call them up, they will not give you any advice.

They will answer your questions as thoroughly as they can, but they will not offer anything if you haven’t asked about it. That’s some of the things that people don’t think to ask about is, “Well, I’m saying I want to retire. I want to start taking my benefits at this age, and then what about my spouse? When should he or she take benefits?”

The social security administration will not have that conversation of “How to optimize that.” They won’t explain to you what spousal benefits are unless you know the term and you can ask, “How do my spousal benefits differ depending on what age I claim or my spouse claims.”

This is a complicated issue, and one of our guys from the investment side sat down and did a rough calculation, and said there’s probably at least 8,000 ways for a married couple to take social security benefits as a couple. It has to do with who starts when and what.

If you think of it as month by month each because the social security benefit payout increases a little bit each month, right until you get to a maximum age, it’s even more if you think of it as month by month. Anyway, there is a lot of choices there, not everybody has all those choices available to them, but even just an ordinary married couple would have a whole bunch.

That’s the kind of thing we hope that we can do better but then social security will because we’ll talk about a couple. The tricky part there is they have to trust us enough and understand what fiduciary responsibility is on our side.

I think that’s something that we haven’t communicated well and is not well understood also. That is Financial Engines as a whole is a fiduciary. We are bound by law to tell you what’s good for you and not what’s good for us. We don’t sell any particular product. We don’t have a fund that we push on people because we don’t make any money off of it.

We make money off the employer, buying the whole platform and a very small – we call a basis points, like it’s tens and hundreds of a percent of your value of your fund for the management of it, which is different from getting a commission.

Steve: Help me link it back. I think you raised this issue of trust being this important thing to present and to instill. Before we were talking about internally how research and design or at least you were talking about them kind of altogether. What’s the link between the process and the kind of experience you’re trying to create for people?

Nancy: Good, thank you for bringing me back. The more I can help the rest of the people who might weigh in on what the new products will be or how we will recast our existing products can help them understand what users say they want, how users behave when they get confronted with the things we offer them, and how they opt in or opt out of things.

The better we can clarify what our product offerings are, both in the verbal sense and then also in the way the interactions are designed.

I think that having a conversation of the whole because I don’t think the designers fully understand all the business issues involved in what we could offer people and what we can’t offer people. I think having everybody involved in those discussions is going to be the best.

Steve: That’s great. Maybe just one larger question, as we look to wrap up.

Nancy: How can we get larger than that? [laughs]

Steve: My question is about you. What are the things about you that maybe methodological or anything in your background and the experience that you’ve had that makes you great at what you do?

Nancy: Thank you. I loved that. Let’s see…I think…I come up from a background where I studied linguistics. That was my area of focus as an undergraduate, and as a graduate student, and now I have a PhD, so all I know is linguistics. I was always interested in what we used to call applied linguistics.

That is, not necessarily the theory of language, but to take the ideas about the theory of language and see how it plays out in the real world. How do children learn language? How do we learn a second or a third language?

My language of specialty is American Sign Language, so that’s all about deafness and disability and what’s normal. For me, deaf is very normal. There are deaf people who have other disabilities. For me being deaf, and I should think about what we want to do about this particular podcast to make it accessible for a deaf audience, I come from a standpoint of inclusion in that regard.

My exposure to deaf folks and my experience as a sign language interpreter, and a trainer, and evaluator of sign language interpreters was always about inclusion. How do we make a world where everybody can be part of whatever they want to be? I think that’s the kind of set of attitudes I bring here also.

Steve: That’s lovely, and there will be a transcript.

Nancy: Good.

Steve: People that are reading the transcript right now already know that, because they’re reading it.

Nancy: [laughs] Right, right.

Steve: What didn’t we talk about yet that you’d want to make sure we cover?

Nancy: Good question. I think we talked about the User Research Council. That is, my internal name for the panel of people who are willing to give us feedback and who we pay a pittance for it. We have a small thank you gift, as do many organizations, and we are opening that panel to people who are not currently our customers and who are just interested in giving feedback and especially people who are closing in on retirement.

We want to make sure we provide that great planning and transition support as people get to age, whatever the right age for you or them is 50, 55, 60, 65, 70. I had a great conversation with a guy a couple months ago who was working as a sales associate in a well known home improvement store.

He said, “I’m just doing this to have a social life, to have, to get out of the house. My wife is handling all the financial planning, she’s that’s what her job is. It’s great, I’m happy to talk you and give you feedback on your tools, but um you know, I don’t need to work because I’m 75,” or whatever he was. “I just, I don’t need to work for the money. I need to work for the, the human interaction.” That was so much fun to realize, that he’s our customer too, right?

Steve: It’s helpful to cast a wide net and get different kinds of people into your sample?

Nancy: Absolutely, absolutely. We want to be able serve people that are in jobs like cashier and shipping clerk, as well as people who are in the executive office, and everywhere in between.

Steve: People listening to this or reading this transcript, where can they get on the panel?

Nancy: On the Financial Engines home page we have a link to Contact, as many people do. On the bottom of the list of contacts, as I’m speaking to you today, I hope that’s where it is at the point when people are listening, is a link to the User Research Council. We’d be delighted to have any of your listeners, their parents, neighbors, relatives, and friends join up in our panel.

Steve: That’s great. Nancy, do you have any questions for me?

Nancy: I always have questions for you Steve, but I don’t think they’ll fit in this space [laughs].

Steve: We’ll take that offline then. Thanks.

Nancy: Okay.

Steve: Thanks very much. It’s been a really enlightening and enjoyable conversation. It sounds like you’re doing great stuff. I really appreciate you sharing it with us and everybody listening.

Nancy: Thanks for inviting me to be part of your podcast series.


About Steve