CHALLENGE:
Decisions made based on preconceived ideas had repeatedly led to ineffective user interface designs
SOLUTION:
My work conducting user interviews and developing data-driven personas uncovered an audience that hadn’t been clearly recognized and led to critical insights about other audiences
RESULT:
We were able to make dramatic short-term UI improvements while making the case for a broader shift in approach over the long-term
Case Study:
Eaton University plays a critical professional development, employee orientation, and compliance role for a globally distributed manufacturing company with over 80,000 employees. All employees are required to interact with Eaton University on an annual basis to complete an ethics training course. Despite this annual touch-point, many employees remained unaware of the breadth and depth of resources available through the university. Departments routinely purchased materials, such as newspaper and magazine subscriptions, that could be procured freely or with dramatic price reductions via the library.
I was brought in as a consultant to guide the user-experience design, particularly focusing on user research, information architecture and usability testing.
Confronted with issues of both lack of awareness among employees and associated low utilization, the Eaton University team was tasked with redesigning the portal to Eaton University to make it more welcoming and user-friendly. I was brought in as a consultant to guide the user-experience design, particularly focusing on user research, information architecture and usability testing.
The University includes a Learning Management System (LMS) that supports face to face, remote instructor led, and asynchronous online courses. In addition to the LMS, the university has a library with an extensive collection of online databases and materials. Finally, the university offers a variety of career development and learning resources that have been developed internally with both web and SharePoint components.
Frustration with the LMS in terms of speed, functionality and reliability were an open-secret within the organization. However, prior to being brought in for this engagement the executive team leading Eaton University (and championing this project) had already made the decision that this project would not include making any substantive changes to the LMS. The reasoning given for this decision was that such changes would be a) too expensive and b) were not important enough/truly necessary.
When we began our work the Eaton University team didn’t really have any formal definitions around their user base. Following our kick off meeting they defined the following audiences:
- Learning Professionals: A person in the business who doesn’t work for Eaton Univ and has a job description that includes responsibilities related to training
- Course Owners: People who own things that live in the LMS, they may not have it in their job resp (their manager might).
- Functional/Section Leaders: Senior leaders responsible for a named function or specific business sector
- Regional Hub Managers: Country staff responsible for training and development needs for that country only
- New Employees: New to company or new to role (as with promotion)
- Knowledge Workers: corporate employees, engineers, sales force, factory floor managers
I was not surprised to find that it was much easier for them to define and describe the people who worked most closely with Eaton University as producers and managers of content than the actual end users, but I pointed this out to them and suggested that it would be important for us to really probe into the needs and nature of end users in our research work. The team agreed with this and we proceeded by dividing the audiences into two broad categories—producers and consumers. We interviewed 1-2 representatives from each producer group and then targeted 3-5 representatives from the consumer side.
The subject and resource needs of these employees varied quite a bit—but fundamental aspects of the user experience cut across audiences and suggested a radically different way of understanding users.
As I began the interviews my “gut” feeling was that job role and subject area would be extremely important in understanding the needs of users in this system. I campaigned for additional user interviews to be sure that I could speak to several people from engineering, sales and factories because I found it hard to believe that there would be a great deal of overlap in the needs and nature of these employees.
As is often the case with user research, my original idea was partially correct—but also very much off base!
As is often the case with user research, my original idea was partially correct—but also very much off base! The subject and resource needs of these employees varied quite a bit—but fundamental aspects of the user experience cut across audiences and suggested a radically different way of understanding users.
I conducted 27 interviews with Eaton employees in the US, India, China, Europe and Brazil over a two week period. Together with my intern, a Master’s Candidate from Kent State University, we transcribed the notes and distilled the data from the interviews into a set of 391 discrete data points.
I conducted 27 interviews with Eaton employees in the US, India, China, Europe and Brazil over a two week period. Together with my intern, a Master’s Candidate from Kent State University, we transcribed the notes and distilled the data from the interviews into a set of 391 discrete data points. In the process of analyzing the data a set of profiles that would become the personas for Eaton University began to emerge.
Key take-aways from the research were as follows:
- Frustration with the LMS was severe. Users had difficulty finding required courses, much less courses that might be of interest to them. Some users would complete a required course only to have the system freeze after they clicked the “submit” button and the class would not register as complete. The Eaton University team was horrified to learn that because the course was required and users were unclear or unable to get assistance some would take the course more than once as their only source of remedy.
- LMS frustration, paired with mandatory courses (annual for many employees but some employees in the sales force and functions had multiple compliance related courses), had created a mind-set among many employees that Eaton University was to be avoided until and unless there was no alternative. This stance could not have been further from the perspective the team wished to engender.
- There was a great deal of similarity in patterns of use across audiences; engineers didn’t look very different from sales and marketing or supply chain management when it came to how they interacted with the system. Desired features and functionality were startlingly similar.
In developing my report and the personas I concluded that I had two tasks. The first was to provide the team with the insights needed to move forward with the interface redesign project they were immediately tasked with, but the second was to launch a convincing case for prioritizing the previously rejected upgrades to the LMS.
Analysis and extrapolation from the data led us to propose the following personas for Eaton University. (Refer to PDF for more detail: EnhancedLearnerPersonas2-23-2012)
Consumers:
- Non-elective users: people whose interactions with Eaton University are defined by and limited to taking mandatory courses. Probably the largest consumer group.
- Guided users: those participating in a structured program of professional development where courses are prescribed for them. Includes new employee orientation.
- Self-directed users: continuous learners who are receptive to new learning opportunities. They often read and or/pursue professional development activities on their own. They may not be familiar with the library or have made heavy use of courses; they are often unaware of the full suite of resources available to them.
- Power users: comfortable interacting with wide variety of Eaton University platforms, generally experience success, are undaunted by challenges and resourceful about seeking out assistance.
Producers:
- Learning professionals: those whose primary job responsibilities are tied to training (primary) and those whose job responsibilities include a significant training component (secondary).
- Course owners: Learning professionals with a unique set of needs and challenges related to interacting with Eaton University in general and the LMS in particular.
The personas were very well received by the team and a series of posters and hand-outs were created to disseminate throughout the Eaton University organization.
The personas were very well received by the team and a series of posters and hand-outs were created to disseminate throughout the Eaton University organization. There was general agreement that for the first time the team had a way of understanding their users that could guide them in design decisions moving forward.
As is always the case with user research, there was a wealth of fascinating information, much of it not directly germane to the problem at hand and yet potentially valuable if it could make it into the right hands. We worked with the team to maximize the value of the research by getting these “nuggets” disseminated effectively.
Moving into the user interface design portion of the engagement, our priorities were as follows:
- Make life easier for non-elective users. Reduce suffering in the hopes of moving some of these users onto a path toward becoming a self-directed, guided or power users.
- Enrich the experience for self-directed users. Look for ways to amplify their understanding of the landscape. As these folks were often managers, help them help non-elective users.
- Avoid alienating power-users. With the broader project goals and limited resources we made a conscious decision not to design for power-users, as they were already the best-served and most successful audience (as well as the smallest).
We designed a new information architecture, created a working prototype, and conducted remote usability testing sessions with 18 participants in the US, Europe and India via video-conferencing. The design underwent a round of iteration following usability testing and then went into development. (Functional specifications were developed in concert with design and testing.)
Before we could even begin advocating for a change in the LMS, the Eaton University executive director attended our user research findings session and noted that the data suggested that the issues with the LMS were more problematic than had been understood. We learned that an effort to upgrade and redesign the LMS had been begun and then mothballed due to resource constraints. He agreed to consider this in budget preparations for the coming year.
As an outside consultant I do not always get to see my designs all the way through to fruition, and as this was an intranet I did not have an opportunity to interact with the finished product. The client reports that this interface has been well received and they continue to use the personas for guidance.
