The 7th Annual Usability & User Experience conference took place at Bentley College on Wednesday, May 28, 2008.
Presentation Abstracts and Biographies
Web Apps: The Collision of Design and Business
Presenter: Jared Spool
Summary: Who has the best practices for web app design and what do they do to make world-class applications? Jared will share the latest research in the techniques and methodologies that can make a huge difference.
Thousands of sites have the same web app: the e-commerce checkout process. They all do essentially the same thing: exchange the site’s products for the customer’s money, simultaneously arranging for delivery and payment. Yet, every single one is different and despite their prevalence, frequently hard to use and frustrating. Why is this?
Creating easy-to-use and delightful applications, such as checkout, while meeting the needs of the organization is one of the most difficult challenges designers face. While many sites struggle, some design teams have learned how to do this especially well.
In this presentation, Jared will share UIE’s latest research on designing successful web-based applications. He’ll describe the techniques and methodologies used by the best designers and the common traps they’ve learned to avoid.
Bio: If youve ever seen Jared speak about usability, you know that hes probably the most effective, knowledgeable communicator on the subject today. What you probably dont know is that he has guided the research agenda and built User Interface Engineering into the largest research organization of its kind in the world. Hes been working in the field of usability and design since 1978, before the term “usability” was ever associated with computers.
Jared spends his time working with the research teams at the company, helps clients understand how to solve their design problems, explains to reporters and industry analysts what the current state of design is all about, and is a top-rated speaker at more than 20 conferences every year. He is also the conference chair and keynote speaker at the annual User Interface Conference, is on the faculty of the Tufts University Gordon Institute, and manages to squeeze in a fair amount of writing time.
Remote User Testing: Global Testing Without Moderator Bias
Presenter: Ania Rodriguez
In times where marketing professionals and usability professionals share the mantra of making a web-site easy to use, there is always a questions as to how your work complement each other so that each of you are successful in meeting the needs of users. Typically, marketing professionals use surveys to understand the needs of users. Then they may complement surveys with focus groups or vice versa. On the other spectrum, usability professionals are work closely with users using usability testing to gather user input. Usability professionals including designers, information architects, and researchers rely on validating their designs with real end-users. However, often times budget is limited and at most usability professionals have to run walkthroughs to gather feedback. At times, usability professionals are afforded the opportunity to conduct research using a usability lab, whether remote or in-person. In any event, both remote or in-person usability studies require a moderator and hence can be time-intensive to gather a large sample of feedback. Hence, often times decision makers will on average limit the scope of usability testing to about 12-15 users, and use other feedback mechanisms to get a larger representation of their user base.
A new methodology that is task-based and requires no moderation has surfaced in the recent years that collects both attitudinal and behavior data from real users or representative users. This technology, often coined remote unmoderated usability testing, marries both survey data to usability task based data including clickstream. These technologies allow marketing and usability professionals to gain insights about who their users are, why they are visiting their site (intent) and what frustrations, if any, do they visit while on your site. The value of such technologies allows decision makers to rank order the more critical items and make informed decisions about their site.
The differences between this new technology and methodology versus an in-lab or simple survey is simple:
Quality of data
Time and expense
Focus of talk
This presentation will discuss:
What are moderated and unmoderated usability tests?
What goes into a remote unmoderated test script?
Comparisons between moderated and unmoderated tests?
Why unmoderated usability tests may help you get better quality of data, reach a larger geography, while keeping the relative time and expense to run such a large sample lower?
Bio: Ms. Rodriguez is a usability professional for over 10 years with a list of current clients including eBay, Intuit, Cox, AT&T, Dell, Citizens, Liberty Mutual, and GEICO. She joined Keynote Systems as Senior Manager of their Customer Experience Research practice last year. Previously, Ania was part of IBM Global Services Usability Engineering National Practice, where she led large user experience client engagements through UI design, development, and usability testing for 8 years. Before that she was a human factors engineer with Pratt & Whitney. Ania has spoken a various large symposiums and online seminar series including more recently for the American Marketing Association (AMA) Online Seminar Series on the topic of Web 2.0 and e-commerce. She holds professional affiliations with AMA, UPA and HFES where she is the Internet Technical Group Chair. Ania holds a B.S. from Carnegie Mellon and her M.S. from University of Miami in Industrial Engineering.
Tips and Tricks for Measuring the User Experience
Presenters: Bill Albert and Tom Tullis
Summary: This presentation shares some of the tips and tricks we have learned on how to measure the user experience. We will review some of the most effective metrics, such as:
Performance metrics (i.e., task success and time-on-task)
Self-reported metrics (i.e., SUS)
Issues-based metrics (i.e., issue prioritization)
Physiological metrics (i.e., eye tracking)
Combined metrics (i.e., scorecards)
We will also discuss some of the common techniques for collecting and analyzing usability data. For example, we will show how to easily calculate confidence intervals for binomial distributions such as task success. The presentation will conclude with a brief discussion of review some of the tools we use to collect and analyze usability data. We will demonstrate a few free calculators that are indispensable for any usability professional. The focus will be on providing highly practical information to the audience that they can use within their organizations. We will discuss ways to quickly capture and analyze usability data without a negative impact to the budget or timeline. This presentation will be highly interactive. The audience will be encouraged to ask questions throughout the presentation. In addition, we will give several demonstrations of various online calculators and ways to analyze usability data using Microsoft Excel.
Bios: William (Bill) Albert is currently a director of User Experience at Fidelity Investments. He has published more than 20 papers and has presented his research at many professional and academic conferences. Albert has been awarded prestigious fellowships through the University of California and the Japanese government for his research in human factors and spatial cognition. He received B.A. and M.A. degrees from the University of Washington and a Ph.D. from Boston University.
Thomas S.(Tom) Tullis is senior vice president of User Experience at Fidelity Investments. Tullis received a B.A. from Rice University, an M.A. in experimental psychology from New Mexico State University, and a Ph.D. in engineering psychology from Rice University. During his 30 years of experience in humancomputer interface studies, he has published more than 50 papers in numerous technical journals and has been an invited speaker at national and international conferences. Tom also holds eight U.S. patents and is an Adjunct Professor at Bentley College. Tullis and Albert have collaborated on the first-ever book on usability metrics, entitled: Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. The book is published by Morgan Kauffman, an imprint of Elsevier.
Multivariate Testing for Optimal Website Usability
Presenter: Eric J. Hansen
Summary: The most usable web sites are those that successfully accomplish their key goals, whether they are bringing users to a check out page, filling out registration forms, or increasing time spent on the site. While SEM and natural traffic bringing in more and more visitors, the site must then keep that momentum going and ultimately convert the visitor to a customer. By using tools that easily test variations of factors such as landing pages, content,image placement and registration forms, web site design and features can best be optimized to best meet the organization’s conversion goals.
Eric Hansen, founder and president of website conversion optimization technology provider SiteSpect, will provide details on how to optimize the entire conversion process-from landing page to checkout. In this highly interactive presentation, particular emphasis will be placed on describing practical optimization strategies that can increase the usability of web sites–including segmentation, design, targeting, multivariate test design, and analysis of results. The audience will see how some web sites are experiencing conversion rate increases from 25% to 250%, and learn a repeatable process that is both practical and effective. This presentation will provide details about the results achieved by specific companies-which designs work best, which ad or headlines create the greatest impact. Audience members will:
Discover how to easily increase conversion rates for their websites
Determine the best test campaigns to quickly get results that impact their bottom line
Learn practical tips to create segments, design tests, and analyze results
Bio: Eric Hansen is the president of SiteSpect, Inc., and the architect of the firms patent-pending solution for optimizing web sites through non-intrusive A/B and multivariate testing. Prior to SiteSpect, Hansen founded Worldmachine Technologies, a consulting firm focused on the design and development of complex web sites.
Visualization Techniques and Tools
Presenter: Michael Ledoux
Summary: I will be presenting an array of tools and techniques (simulated in the presentation) for deriving meaning from large amounts of data. The current graphical tool sets available from many software products fall short in helping decision makers clearly see the issues in the data. But there are tools & techniques that can help users better visualize answers.
Bio: Michael Ledoux is a Senior User Experience designer at EnerNOC (a leading demand response and energy management solutions company). His experience spans 27 years in software development, usability engineering (information architecture, usability testing, and user-experience design). Hehas worked for Varian Semiconductor, BMC Software, Harte-Hanks Data Technologies and IBM corporation. He holds a M.S. degree from Bentley in Human Factors and Information Design and a B.S. in Business Administration from Northeastern University (where he also played varsity ice-hockey). His hobbies include playing acoustic blues guitar, snow-boarding and mountain climbing.
Moderating Usability Tests: 10 Golden Rules
Presenters: Beth Loring & Joe Dumas
Summary: From many conversations with other usability practitioners, we know that our training as moderators was typical: a colleague let us watch a few sessions and then watched us struggle for a few sessions. There was no formal training, no set of professionally accepted guidelines, and no book on the topic. While it was never stated explicitly, we learned that moderating skills were considered part of the “art” of testing, while many of the other skills needed for testing had been standardized. As we considered this situation and talked about our ideas on moderating, we realized that we were in agreement on at least 90% of our practices. Consequently, in 2004 we created a set of 10 “golden rules” for moderators, and developed a tutorial for the annual UPA meeting on how to moderate. We gave that tutorial for three years, refining our ideas and how to present them. We also created a number of videos that illustrated some good and bad practices described in the rules.
At this session, we will present the rules and invite discussion from the audience about them: Are they the right 10 rules? Do they capture the practice of experienced moderators? Are their additional rules we have missed? We will also show sections from videos that illustrate some of the good and bad practices, as well as sections from an expert panel discussion of the videos.
Audience Involvement: The purpose of the session is to discuss our proposed rules with the audience and get their feedback. In addition, we will ask the audience to view the videos, note the good and bad practices they see, and then discuss their reactions.
Bios: Beth Loring, Director of the Bentley Design and Usability Center, has over twenty years experience in product design and usability. She has expertise in user requirements gathering, UI design, ergonomics, and evaluation methods. She has designed and evaluated a wide array of consumer products, web sites, desktop software, business applications, and medical devices. Beth has also been a faculty member in Bentleys Information Design Certificate program since 2001.Before joining Bentley in 2003, Beth was a Principal Research Scientist at the American Institutes for Research, and before that she spent five years as the human factors team leader at IDEO Product Development in Boston. Beth holds an M.S. in Engineering Design from Tufts University and is a Certified Human Factors Professional. She has over twenty publications and is coauthor of Moderating Usability tests: Principles for Interacting with Participants (Morgan Kaufmann Publishers, 2008) with Joe Dumas. Beth is past chair of both the New England Chapter of the Human Factors and Ergonomics Society and Boston CHI.
Joe Dumas is a recognized expert in usability evaluation with over twenty-five years of experience. He has moderated or observed others moderate thousands of usability testing sessions and taught numerous students and usability professionals how to moderate. He is currently a usability consultant for Oracle Corporation. He was a senior human factors specialist at Bentley Colleges Design and Usability Center and taught graduate courses in the colleges graduate program in Human Factors in Information Design. Joe holds a Ph.D. in cognitive psychology from the State University of New York at Buffalo. He is the author of and numerous articles written for researchers and practitioners, as well as three books: Designing User Interfaces for Software, A Practical Guide to Usability Testing with Ginny Redish, and Moderating Usability tests: Principles for Interacting with Participants with Beth Loring.
Persona Development and Validation — What Do You Need to Know?
Presenter: Jen McGinn
Summary: The value of personas is often debated — specifically, when and how should they be used? In this talk, we’ll cover that quickly and then move onto practical tips and tools for developing personas. For example, how can you get stakeholder buy-in? How can you develop personas that resonate with your developers? Where can you find pictures? How much detail should you include in your personas? How do you know when you have enough, too many, or too few personas? What are the different persona development approaches, and when can or should you use them? And what about validating the personas? Do you need to do it? And what are the different ways to validate your personas? How long will it take, and how much will it cost?
In this 45-minute talk, I’ll answer those questions, and give you the most valuable resources literature, including the new findings, from CHI ’08. I’ll also briefly cover a new technique that a colleague and I developed for creating data-driven personas, which is self-validating, relatively inexpensive, and produces statistically significant results–qualities rarely achieved in persona development.
While we won’t have time for a lab, this presentation will be informal and interactive–bring your own questions, problems, and opinions, and you’ll leave with answers, solutions, and possibly a new outlook on how you approach persona development the next time.
Bio: Jen McGinn is a Senior User Experience Engineer in the Software Experience Design (xDesign) group at Sun Microsystems, in Burlington, MA. She has written branded interaction guidelines for installation, developed personas for Sun Learning Services and Sun Labs, and simplified the user experience with the Java Enterprise System. Jen has worked for Sun for more than 12 years in user experience, technical training, and technical writing. Before joining Sun, Jen led other lives as a software developer and system administration consultant. She holds a BS in Information Systems from UMBC and an MS in Human Factors in Information Design from Bentley.
Starting Up a Usability Program
Presenter: Deb Biggar
Summary: How do you get a usability program started in a large organization? How do you mix in the contributions of the development team, QA, product developers, executives, marketing folks and art directors to prove that usability testing can be helpful?
Come hear how one UX professional got things started in a large organization. Deb Biggar works an independent contractor for RedPlum.com. Since launching in January 2008, this site has had over 800,000 site visitors. This site undergoes monthly usability studies and iterative design improvements based on the usability study findings that began as a result of Deb starting up their usability program.
Bio: Deb Biggar is an independent user experience consultant. She has been helping to design UIs for the web since 2000, and fell in love with the user experience field while a graduate student at Bentley College. After graduating from Bentley in 2006, she had a baby (actually, the baby came 2 weeks before graduation), and founded BiggarNet, LLC. She couldn’t be happier with her current adventure of getting a usability program started at Valassis on their very first consumer facing web site, RedPlum.com. RedPlum.com has committed to investing in user research studies through the 3rd quarter of 2008.
Beyond Card Sorting: Research Methods for Organizing Content Rich Web Sites Run Amok
Presenter: Michael Hawley
Summary: Content management systems are powerful tools for web site teams. They allow site designers to define an overall information hierarchy and page templates that maintain a level of consistency for sites with large amounts of information. However, content management systems also make it easy for a distributed network of content providers to add content to a site. In a very short time, a web site can outgrow the page templates and information architecture that was originally designed.
Our company has had a number of customers approach us with this very problem. They originally designed their site with a certain organization and as content needs changed, the original hierarchy of content no longer made sense. As we review the various user-centered design techniques that we can employ to re-design the site, invariably customers will become attached to the method of card sorting. In particular they gravitate toward online card sorting as they feel the quantitative nature of the exercise will give them a greater degree of confidence in the results. It is easy to see why online card sorting is such an appealing method-it is fairly inexpensive, does not take a significant amount of time, and delivers specific results on relationships between content that is based on accepted statistical methods. Historically we have used online card sorting and several variations of the method as the primary activity in information architecture re-design projects. However, based on our experience with online card sorting, we now find ourselves warning our customers that online card sorting will not give them the quite the answers they are looking for. While we may still include an online sort as component of a project, it is only a small component of a larger research process.
In this presentation, I will discuss the limitations we have seen with card sorting and their impact on our results. I will also discuss a test sort research study we conducted to help us better understand the nuances of the method. During this portion of the presentation, I will solicit audience participation. The audience will not actually do a card sort. Rather, I will show the cards we used in our study, then ask the audience to privately note what organizational strategy they would use for the cards if they were the participant in a study. Lastly, I will compare the organizational strategies of the audience with the variety of organizational strategies we saw in our study and show how they align or dont align with the card study results. By asking the audience to participate, I hope to demonstrate nuances of open card sorting and develop an understanding of its strengths and weaknesses.
In addition to the limitations, I will also discuss several variations on the method or alternative methods to card sorting our company has tried to develop a customer-centric information hierarchy on large-scale sites. For each method, I will give an overview of the approach, a sample from our work, and the lessons we learned.
Bio: As the leader of Mad*Pows UX team, Michael Hawley leverages 15 years experience in the software industry and expertise in user experience research and usability design to deliver value to clients such as Fidelity, Autodesk and Monster. Prior to Mad*Pow, Hawley served as Usability Project Manager for Staples, where he led customer-facing initiatives including e-commerce and Web sites, marketing communications, and print materials. Prior to Staples, Michael worked at Bentley College Design and Usability Center where he was responsible for planning, executing, and analyzing user experience evaluations for corporate clients in healthcare, financial services, academia and other industries.
Michael holds his M.S. in Human Factors in Information Design from Bentley College McCallum Graduate School of Business, and a B.A. from the University of Michigan. He continues to explore trends within the UX discipline as a published columnist, with articles appearing in publications such as UXMatters, iMedia, TMCNet, and CPWire.
Design Happens or Why Lipstick Doesnt Belong on Pigs
Presenter: Deborah Falck
Summary: (PowerPoint presentation with examples of successful and not so successful visual design in user interfaces.)
Visual design has lost its distinction in the software product development process and has been blurred with other professions. This creates three problems:
The job of visual design is dispersed and the visual design needs of the interface are neglected.
Visual designers and/or clients approach the work from a marketing/print design perspective
Visual designers are over-tasked
Computer interfaces are ubiquitous in todays workplace. Almost all aspects of work today involve computers and computer interfaces. As efficient as computers are in processing and presenting information, the proliferation of computer interfaces consumes a users limited amount of available time and users are almost always in information overload. If a user cannot read, understand, interact effectively with an application, that application will be problematic and costly in any number of ways, no matter how valuable the application.
How did we get into this predicament?
Before the proliferation of personal computers, graphic designers needed to not only be trained in numerous design skills such as color theory, page layout and typography, but they also possessed extensive technical skills required for the production of their wares. Specifying copy for the typesetter, producing camera-ready artwork, finding ones way around a PMS color specification book and signing off at a press run, were all part of the job.
However, personal computers have made it possible for anyone with a small amount of computer savvy to produce desktop publishing and create websites. This has also blurred the lines between the domains of developers, interaction and UI designers and visual designers. Many clients are sometimes blissfully ignorant or worse as to the differences in skill sets, and results.
What is good visual design and what are some of the elements that make it effective?
Bio: Deborah Falck is the principal of Design Presence, a consulting firm specializing in visual design. Solutions based on a deep appreciation of users needs, implementation of fundamental design principals and emphasis on teamwork are its guiding ideals. By applying these principles to her work over many years she brought forward visual design expertise into the development of information technology products. The focus of visual design based on user needs, is one of the most important ingredients of a user’s experience with products and technology.
Ms. Falck has solved complex design problems for numerous clients including legal and medical software, telephony products, search engines and received numerous design awards and patents for her work. She has studied with pioneers in the field of design, including Muriel Cooper, Milton Glaser, Ati Gropius Johansen and William Porter. She received a BFA from Boston University College of Fine Arts and graduate training at University of Michigan in Human Factors.
The User Interaction Cycle
Presenter: Ben Dubrovsky
Summary: The User Interaction Cycle is a model for thinking about HCI that easily leads to:
A framework for organizing usability methodologies and tools
A proposed set of “primitives” of user behavior
A way to consider testing the primitive components of a design in addition to testing the completed design
A checklist of steps to ensure solidly usable designs
Consideration of other issues related to usability
The Cycle was developed by considering how any user may interact with any computer-based system. An interaction by a user can be described as many, many iterations of a simple loop:
Figure out what to do next
See if it worked
The first element in the Cycle is characterized by modeling the work process. The second element involves a user engaging the UI to provide input. The third element involves assessing results.
Further analysis leads to more detail in the cycle, and a list of usability tools, techniques and concepts associated with each step:
Physical Work Model next step
Time Motion Studies
Software Work Model next step(s)
Set expectations as to what should happen
High-level Paper Mockups
Cognitive mapping (map what you want to do to the available interface)
Blocking, chunking, grouping
Millers magic number 7 +/- 2
Plan for Physical Action, or Target Acquisition
Physical Actions, or Mechanics
Isolate and Identify Feedback
Process or Interpret Feedback
Compare Feedback to Original Expectations; Iterate.
Gulf of Execution
I will discuss how I have applied this model to designing usability tests and how the Cycle relates to issues of accessibility and the issues of expert and naive users.
I would like to use this presentation to both present my thinking to the community, and to gather feedback from the community as to its applicability and extensibility. I anticipate a good discussion.
For a more detailed view of the User Interaction Cycle, please visit: http://www.dubrovsky.org/portfolio/writings/details/?type=u&id=1
Bio: Ben Dubrovsky has been designing user interfaces, user experiences and information architectures for over 15 years, and has applied usability and user-centered design principles to web, kiosk, and other interactive communications. He has designed, produced, and programmed interactive projects for such clients as Volkswagen, Ford Motor Co. (MIMC Award Winner), and Motorola (4 Cindy and 1 New York Film Festival Awards).
Ben is currently employed at Iron Mountain Digital in Southborough, MA, and has previously worked at Harvard Business School, co-founded and ran ReadyAbout Interactive, a new media design and consulting company in Boston, and spent a year teaching in the Savannah College of Art & Designs Interactive Media program.
Bens formal training is in human factors engineering and computer science from Tufts University (BSE), computer science from Harvard University (MS), and continued study at the MIT Media Lab.
Next Generation Interaction Paradigms: Useful or Novelty?
Presenter: Amy Cueva
I will present 3 demos of next generation interaction paradigms, and then facilitate the audience in a discussion about whether new approaches present more problems than they solve or are entirely worth the learning curve they introduce.
Autodesk Green Building Demo Multi-Touch Interface
The flash demo/mock software experience created for Autodesk was meant to feature the capabilities of new technologies to assist architects in creating greener buildings. Since it would be presented using a multi-touch UI, a newer technology, and would be presented in front of thousands of people, the wow factor was important. The users would be very few and number and could be trained, so interpretation of the interface was not a big issue. However the multi touch nature of the interface posed usability considerations to be solved for.
Aetna Smart Source Interactive Information Map
The health information search engine experience created for Aetna was meant to guide an Aetna member towards getting information about any health conditions they may have or be concerned about. An interactive information map executed in flash was used to give the user a strong indication of the breadth and depth of information available for a given condition. Clicking on the map would change the search results displayed. In usability testing we found that users wanted to be able to understand how large the map was and view its entire contents, while also having the ability to zoom into the map and see the search results along side, such that the results were not pushed below the fold.
Timberland Creative Services Portfolio Visual Search Results
The interactive portfolio experience created for Timberland was meant to feature the work of their internal creative agency in an engaging manner. A pageless design approach was used to provide an interface where the user is able to filter by various criteria. The corresponding results are displayed as thumbnails, the relevant pieces coming to the foreground, and non-relevant pieces fading into the background. The piece would be launched from the intranet as well as used for internal presentations, so the interface needed to be easily understandable while also creating an stunning showcase for the agencys best work.
Conversation around guidelines/war story sharing:
Do new interface approaches present more problems than they solve?
When should a new interface approach be used? Not be used?
Has there ever been a time when a team was keen on exploring something a new approach and user research or testing reined them in?
Have you ever designed an interface where it made perfect sense for the piece to LOOK AWESOME but not actually WORK WELL?
Has exploring a new interface approach ever brought small details to the interface, that wouldnt have been thought of otherwise?
What are your favorite new approaches that have caught on and become standard?
When have you seen new approaches hit a roadblock with the technology that was chosen to implement?
Bio: Amy Cueva is Co-Founder and Chief Experience Officer at Mad*Pow, a user centered design studio in Portsmouth, NH. She has built Mad*Pow’s user centered design methodology as the vehicle to synergize business goals, user needs, and technology requirements. She partners with clients like Google, Aetna, Fidelity, and Monster to create strong cross-channel digital strategies, first class user experiences, and streamlined internal processes. She is the secretary and one of the charter members of the NH UPA, and is co-presenting with Chauncey Wilson at National UPA 2008, A Portfolio of Brainstorming Techniques: From the Nominal Group Technique to the Lotus Blossom Approach.
So You’re Thinking About Consulting? Think About This…
Presenter: Lynn Cherny
Summary: One year ago, I quit my job and started consulting full-time, after 10 years of industrial wage slavery. I was financially successful in this year, but made a lot of mistakes. I managed to fall into bad headhunter relationships, make mistakes in my accounting that required a 101 class to fix, became thoroughly confused about whether to be incorporated or not, and generally made a lot of newbie mistakes with a handful of clients ranging from garage startups to established software firms. Other local consultants gave me advice and I learned from my mistakes. I can tell you how I did it and what I could have done better; and how it compares to what other local consultants say.
I will cover:
Your use of the internet to advertize yourself (search engine optimization, job sites, Linked In, blogs, etc.)
Branding (logo, name, etc.)
What to charge (the many factors and equations; plus: “they’re charging WHAT and someone is really paying it??”)
Headhunters and job offer pressures
Basic accounting and expenses to track
… And other things I learned the very, very hard way, like the portable office equipment it might be nice to own because the client site is a cave with rocks to sit on.
You’ll get a handout with the Top 10 Most Important Consulting Considerations in case you too want to do this!
Bio: Lynn Cherny has a Ph.D. from Stanford that she hasn’t used in years, except for some statistical skills. She has 12 years of experience working at and/or managing interface design at companies including TiVo, Excite, Adobe, The MathWorks, and AT&T Labs. Her current consulting identity is Ghostweather Research & Design, LLC. She can be reached at email@example.com.
Critical Conversations about UCD: Challenging Assumptions, Fondly Held Beliefs, and Common Practices
Presenters: Chauncey Wilson, Tom Spine, and James McElroy
Summary: In this panel, three experienced HCI professionals with backgrounds in product design and usability will reflect on the evolution (or in some cases the de-evolution) of the field. Well look at commonly held assumptions about our field and question their validity.
The HCI literature is filled with articles advocating user-centered design (or close cousins like usability engineering or user experience design). There are more jobs for user-centered design (UCD) practitioners than ever, and new books on UCD are filling our bookshelves. While things seem relatively rosy for UCD, we feel that it is time to test some of our assumptions and reflect on the question “Are we headed in the right direction?”
Some of the topics the panel will address include:
Usability Methods are Failures. Products that tend to get iconic status like the iPhone® are often the result of a process that diverges from common UCD in many ways. Experts cant show much true evidence that much ballyhooed methods like contextual inquiry, think-aloud usability testing, and affinity diagramming yield great products. While there are some focused studies that do indeed show some benefits of UCD, most of us would be hard pressed to show where and how UCD actually affected product success. Can UCD methods really achieve breakthrough success? If not, perhaps usability is a failure, and perhaps our field methods are a failure. Perhaps great design breakthroughs only come from talented designers or small design teams naturally gifted, creative design forces.
The Problems with Problems. There seems to be a lot of faith in our UCD methods and processes, but even experts cannot agree on what a “problem” is because problems are so contextual. How can we have credibility when we struggle to define “usability problem”?
Splintering of the Profession. Those of us who have been in the field for several decades see splintering into more and more specialized (and insular) societies, conferences and languages. We keep coming up with more and more titles to describe what we do, and rediscovering concepts that have been around instead of innovating beyond them. We are splintering into sub-groups who debate the meaning of “design” or “usability” endlessly instead of working together to build a shared understanding of the extensive body of knowledge the profession has amassed over the past several decades and come up with a way to clearly explain our value to outsiders. Information architecture (IA) for example, deals with the organization of functions and content which is exactly what usability engineers did in the 1980s, except that we just considered it an important part of our job. In our own area of Boston, the number of professional meetings taxes our calendars.
Statistical Knowledge is Critical. Many UCD practitioners assume that small numbers of participants provide results that are indicative of the larger population. This stems from early research showing that a large percentage of “usability problems” are revealed with samples of 5 to 8 people. While small numbers of participants may reveal major problems of very isolated, small portions of a system, most systems today are nothing like those used in the early research. As Jeff Sauro highlights in his work on confidence intervals, having 5 people succeed on a task does not mean that the success rate for the population of users will also be 100% (it could range between 60 and 100%). There is often an assumption in our field that statistics is not all that important, but in fact, some solid grounding in sampling and research principles is critical for understanding our data and the results of our analyses. Randomness, for example, is an underlying and powerful principle that is often ignored in our interpretation and reporting of data.
Quick and Dirty Can Be Very Dirty. In the early 1990s, the field seemed to shift to quick and dirty, discount, and guerilla methods that stressed how minimally trained people could conduct tests of complex products with little training. Videotaping and video analysis became a taboo and conferences were filled with topics like how to test on zero budget. We have probably gone too far in the direction of discount evaluation and should focus on a mix of discount and robust methods.
Each of the panelists will reflect on aspects of UCD like those listed above that:
Limit our ability to deliver innovative products
Show how our assumptions can be faulty
Hurt our credibility
Lead to erroneous conclusions
Audience Interaction: The panelists will spend about eight minutes each discussing topics similar to those listed above and will then invite the audience to point out their own areas of concern or challenge the panelists about their positions.
Bios: Chauncey Wilson is a Senior User Researcher at Autodesk in Waltham and instructor in the HFID graduate program at Bentley College. He has more than 25 years in the field. Chauncey has presented at CHI, UPA, HFES, and STC conferences and has co-authored chapters in the 1997 Handbook of HCI, and Cost-Justifying Usability, Second Edition: An Update for the Internet Age, Second Edition. Chauncey wrote “The Well-Tempered Practitioner” column for the ACM CHI publication Interactions during 2006 and 2007, collaborated on the UPA Code of Conduct, and is editor of the Methods sections of the UPA Body of Knowledge (BoK). In addition to his usability and design work, Chauncey is also a serious amateur chef, gadgeteer, and photographer.
Tom Spine is a Product Design Manager at Autodesk, Inc., in Manchester, NH. In the more than two decades since Tom received his Masters degree from Virginia Tech, he has been an individual contributor and manager in usability and product design groups at Digital, Dun & Bradstreet Software, Sun Microsystems, Symantec, IBM/Lotus, and Autodesk. His work has spanned products spaces as diverse as programmer productivity, system management, online collaboration, and computer-aided design. Tom claims he very well may have more professional experience in the creation of corporate and industry style guides than anyone else on the planet.
James McElroy is a Product Design Manager at Autodesk in Waltham. He has 12 years of experience building teams and processes focused on improving the user experience provided by a diverse range of products (including consumer electronics, websites, and complex desktop software). Prior to joining Autodesk, James founded and directed the User Research group at Monster.com, where he built and managed a state-of-the art usability studio. James has also worked at EMC and Bose (where he received a patent for his work on an innovative handheld touchscreen controller). James is active in the UX community and has been a popular speaker at a variety of national and local conferences and was one of the first graduates of the Bentley College Human Factors in Information Design masters degree program.
Why Usability Should Never Come First and the Importance of Front-End Design
Presenters: David Rondeau and Traci Lepore
Summary: Any good design process starts at the beginning. And that means understanding first whether what you are going to design is useful, not whether it is usable. To do this data is required to generate design thinking. In this presentation we will discuss how that data can be gathered and how it is used to drive design, using examples of data and design from an exciting project developing a mobile sports application driven by the needs and values of the data, ensuring its usefulness. Knowing time and resources are always an issue; also discussed will be different levels of data gathering that can be done given time and resource constraints. Once you have a design concept that is useful, it is then important to validate with users. We will talk about what that means and how it is still not testing usability, but rather the validation of the concepts that are iterated and developed into refined UI. We will also show examples of different levels of fidelity in prototypes as you move from concept validation to testing that more refined UI. Once the concept and UI are validated, this is when the usability becomes important. To end we will discuss the difference between testing the concept for usefulness and testing the design for usability and how that data differs and impacts the design.
Bios: David Rondeau, Design Chair: David has over 12 years of design experience that spans disciplines (graphic, visual interface, user interaction, and user experience) as well as media (print, multimedia, software, and web). Since 2001, he has served as the Design Chair at InContext, overseeing designs produced at the company, in addition to being the Lead Interface Designer on many projects.
He has gathered and analyzed customer data to create design solutions across a wide variety of projects, including: enterprise portals, websites, collaboration environments for distributed teams, financial reporting tools, information environments, web publishing workflow support, video editing tools, and new approaches to online learning. He also developed designs for products created at InContext: CDTools a Windows-based application that supports the Contextual Design process and mSports a mobile phone application for real-time information about sports games.
Prior to joining InContext, David was a Senior Multimedia Designer at SilverPlatter Education, Inc., where he designed and developed award-winning products for medical education. He holds a B.F.A. in Graphic Design from the University of Lowell, MA.
Traci Lepore, Principal Interaction Designer: Traci has seven years experience as a professional interaction designer including over 6 years experience at InContext. She has acted as the designer on several projects including software redesign supporting a review workflow in a manufacturing environment, mobile application product development, as well as web site redesigns that include information architecture restructuring. Currently she is working as a Design Expert helping to oversee the projects. Traci has participated in all phases of the Contextual Design process from collecting and analyzing customer data through product design.
Prior to joining InContext, Traci worked as a designer in Marketing and Advertising in the Boston area. She holds a B.S. in Communications Media from Fitchburg State College and is currently working on a M.A. in Theater Education at Emerson College.
Prototyping A to V: Comparing Tools & Methods
Presenters: Kirsten Robinson, Rich Beauregard, Paul Turner
Summary: The panelists will explore a range of prototyping tools and methods, including paper, drawing tools such as Visio, and new tools for creating interactive prototypes, such as Axure. Well discuss issues of fidelity, collaboration, interactivity, learning curves, sharing prototypes with clients, users and colleagues, and more. We will share tips and take questions from the audience. This panel is appropriate for those new to prototyping, or experienced prototypers who want to explore new tools and methods.
Rich Beauregard will discuss an innovative way to collaboratively work with product owners to iteratively produce wireframes, from quick cut-and-paste exercises to fully realized wireframes in just a few working sessions. He has used this methodology successfully and would like to share his experiences and learnings with the greater user experience community.
Using Axure RP Pro 4.6, Paul Turner will demonstrate how to build interactive mid-fidelity prototypes that include working drop-down boxes, list boxes, radio buttons and dynamic panels, which are updated independently without refreshing an entire web page. He will also cover how to add widgets to your design and how to build conditional logic through property sheets. Paul will re-create part of a graduate student project that was originally developed using MS PowerPoint.
Kirsten Robinson will cover drawing tools — the middle ground between paper and Axure. Products such as Visio and PowerPoint can be used to prototype user interfaces with a range of fidelities. Some interactivity is possible to illustrate the users path through a web site or application. Prototypes are relatively quick and easy to create and iterate, and can be shared remotely with clients, colleagues, and users.
We look forward to an interactive session with audience members sharing their own experiences and asking lots of questions.
Bios: Rich Beauregard is a User Experience professional (BFA and MSHFID) with over 13 years of information architecture, usability and art direction experience at leading interactive agencies and financial services companies. He is currently working as a Design Lead/Information Architect at Fidelity Investments.
Kirsten Robinson is a Principal User Experience Consultant with Dynamic Diagrams, an information design consultancy. She is responsible for several aspects of user experience design including research, requirements analysis, interaction design, information architecture, and usability evaluation. Kirsten holds a MS in Human Factors from Bentley and a BA in Cognitive Science from Brown University.
Paul Turner is an MS in Human Factors in Information Design candidate at Bentley College who expects to graduate in June 2008. Paul is designing a banking GUI that is used to open personal and business accounts. He is also a lyricist for classical, folk, and jazz songs.
Innovative Interfaces – Transforming Data into Insight
Presenter: Eva Kaniasty
Summary: The information explosion that accompanied the world wide web led to exponential increases in the amount of information available to the average human, but until recently, the tools available to users to make sense of this information remained limited and crude. As anybody who has used the web to do research can attest, finding the right information is a lot harder than finding a lot of information. Recent advances in web technologies have made it possible to design interfaces that respond in real time to changes in data and user input, leading to an explosion of innovative interfaces that allow users to directly manipulate and visualize data in new ways. In this talk, we will deconstruct some cutting-edge interfaces which support users in a wide range of tasks, from organizing and exploring information, to making predictions, learning and decision-making. We’ll touch upon how these interfaces take advantage of users’ perceptual and cognitive strengths, and provide support for their weaknesses. We’ll consider the usability of various approaches, and the design patterns behind them. The interfaces we’ll consider include dynamic visualizations that incorporate advanced filtering options, new approaches to imposing structure on freeform information such as user reviews, and even humble graphs packed with aha! moments. What these interfaces have in common is their ability to provide customized information on demand, and this time around users get to customize more than their color schemes.
The audience is invited to bring their own examples of innovative interfaces that grabbed their attention.
Bio: Eva Kaniasty received her Master’s in Human Factors in Information Design from Bentley College, and currently works as an Information Architect at SimpleTuition, a startup in Newton, MA. She holds a BA from Boston University in Psychology and a Certificate in Client-Server Programming and Web Development from Clark University. Her past work experience includes user research at the Bentley Design and Usability Center, as well as interaction design and web programming in the non-profit, healthcare, software, and financial sectors.
Establishing and Evolving a UX Team in Your Company
Presenter: Mary Weinstein
Summary: What does it take to develop or enhance your companys websites and improve user experience? A skilled and knowledgeable team may not be enough.
Whats working or not working in your current team and approach? How does your team need to evolve or become involved to continue bringing value to your companys short-term and long-term strategies? Youll have the opportunity to learn from others experiences and share some of your own in this interactive session. Here are some of the topics we plan to cover:
Identifying the gaps in your current team’s skill sets and experience
Developing a plan to evolve your team now and in the future
Promoting the need and benefits to your senior leadership
Turning challenges and obstacles into opportunities
Hiring and/or training the right internal or external associates to fill the void
Providing the right tools for your User Experience (UX) team to be successful
Aligning a User Centered Design (UCD) process within your companys existing process
Demonstrating the benefits project by project
Bio: Mary Weinstein has been leading and implementing major web design and development initiatives for more than 12 years. She started out integrating the Internet into her marketing communications strategies for high-tech clients like IBM and as a National Marketing Manager for Minolta. After a few years, she was so taken by this medium, that she moved to Boston to manage website development and web marketing programs for clients like Dell and GM.
Six years ago, Mary moved into financial services at a large well-known insurance company. As an eBusiness Manager, she leveraged her experience with UCD processes to take her team to the next level. Recently, she was given the opportunity to lead initiatives to improve some of the complex business processes at her company.
Mary graduated from the Stern School of Business at New York University with a B.S. in Marketing and International Business. She is an active member of UPA and BPM Institute. She has completed courses in Usability, Advanced Interface Design and Web 2.0 and is currently pursuing certification in Business Process Management.
Winning the Case for Usability
Presenter: Tonya Price
Summary: This hands-on, participatory session will cover techniques for how to build the case for usability in Web site design by taking a poorly designed Web site and analyzing effective techniques for getting non-usability professionals to understand the benefits of good usability design. The VP of Marketing’s niece has designed a web site that is terrible from a usability viewpoint, but the marketing team loves what they see. How do you build the case for improving the site when the marketing people think it is just what they need? The presenter shows the site, reading from the VP’s email to the CEO recommending the site be adopted. Then the audience will be invited to divide into teams for the purpose of analyzing the site. The teams will take 10 minutes to make a list identifying the usability problems with the home page and five minutes to analyze a sub-page. The team(s) who correctly identifies the most issues comes forward. They will be the marketing team for the next part of the exercise. The rest of the audience teams will then take 10 minutes to formulate strategies for convincing the marketing team to allow them to improve the usability of the site. The marketing team will rate each presentation according to which team convinced them to change their designs. After a discussion on strategies for how to build the case for usability, the audience is invited to share their stories of techniques used to get poor usability decisions reversed. To conclude the audience helps develop their list for the ten most effective techniques for convincing non-usability professionals to adopt good usability practices.
Bio: Tonya Price holds an MBA from Cornell University and is a certified PMP who has spent the past twelve years managing Web design teams. She founded StrategicIdeas.com in 1996, an early Web design firm that emphasized the importance of usability, accounting for firm’s rapid growth until it was acquired by UltraNet Communications. Tonya was the Director of Web Services at UltraNet until the telecommunications company acquired by RCN, where she was later promoted to Director of Web Services. Currently Tonya is the Director of Web Operations for Worcester Polytechnic Institute where she is managing the redesign of an 88,000 page Web site. Tonya has given presentations on Web design to the New England Business Network International Regional Meeting, the MassCUE Boston meetings, Babson College, Framingham State College, the Women’s Success Network and to numerous small chapter meetings of various business organizations.
Getting close to Our Users… a Multi-method Study
Presenter: Bryn Dews
Summary: MITRE ITs User-Centered Design Team (we have several groups doing Human Factors work at MITRE) has begun a series of annual studies to better understand how our employees work with the various corporate IT tools. Our first time was purely a learning experience (2006) which helped our second study (2007) be more successful. We adopted a multi-method approach to provide a pyramid of data with a broad enough base to sell our results to management. The bottom layer of our pyramid was a short (20-Question) survey. We invited a random 20% sample of employees to respond, garnering a 42% response rate (n=538). Our middle layer: 36 one-hour interviews with employees. Interviews were a combination of closed questions, open questions, simple tasks (show me how you would ), and a catalogue of applications about which we asked 3 basic questions (How often do you use it? How do you get to it? How did you first hear about it?). The top of the pyramid was six longer observations of employees working at their computers. Observations were mostly silent (saved clarifying questions to the end) and general (just watch what they do and how they do it), but captured a specific set of data across all observations that was relevant to our topic areas.
Our process included:
IT stakeholder interviews (to narrow study focus)
Structure study parts
Present data to stakeholders and larger IT community
Ill present our process and the lessons learned.
4 hour observations are tough sell
Try more, shorter sessions next year
Smaller window = less data
Easier to schedule
Multiple facilitators make it harder to pick up on trends across all sessions.
Try more frequent team updates
Review session notes as theyre written
Analyzing qualitative data in crunch always daunting!
Affinity Diagram worked well for analyzing qualitative data. But
Multiple people reading notes, adding to diagram led to duplication.
Streamlined process by reading and highlighting same printouts.
Consistent session number labels would make it easier to backtrack.
Good handwriting helps.
Overall well worth effort in terms of data collected and outreach to MITRE community!
My presentations always include some audience surveys and will include inviting audience members to comment on their own similar experiences with the various methods Ill talk about.
Bio: Bryn Dews is the Team Lead for the User-Centered Design group inside of MITRE’s IT division. Starting in 2000, she was the proponent for bringing Human Factors engineers inside IT to be part of application development teams. She was rewarded with a team of four Human Factors Engineers in May 2003 and the team has grown both in size and accomplishments ever since.
Combining Personas with On-site Surveys and Web Analytics to Drive Business Strategy and Design
Presenter: Tim Harter
Summary: Attendees at this session will:
Learn how you can use on-site surveys and web analytics to quantify and validate Personas.
Determine whether these methods are valid for approximating actual user behavior.
Hear a case study of how these methods were used to drive strategy for a large e-commerce website.
The audience will have time for questions and discussion about their experience with personas, whether the methods discussed in the presentation are the best way to address the business questions being proposed, and any alternative methods they have tried or think would be valuable.
Bio: Tim Harter has worked in usability for 5 years and is currently responsible for managing and executing usability activities for Staples North American Delivery business.
Going Beyond Your Mandate: A Case for Building Business Cases
Presenter: Amy Cueva
Summary: This presentation will hit the below points, cite case studies, and provide time for Q&A/war stories to be shared at the end.
The Business Value of Experience Design
Well designed experiences can lift the organization to the next level, often impacting the bottom line, and thrown together experiences can have certain negative effects on the organization.
There are times when experience design gets exposure with and is valued by higher ups within the organization. They understand the possibilities, want to push the envelope, and hold people accountable to a certain level of quality.
In other cases, experience design is not understood or valued within the organization. The value may not be clear, or amongst the companys deep list of priorities, user experience is not a focus.
Linking Business Objectives with Experience Design
Through the user centered design process, experience designers develop a keen sense of what the audience is looking for, what drives them, what will bring them satisfaction, and what will cause them to return.
If that knowledge is brought to bear against what the organization is trying to achieve, the implications to the experience become clear:
Conversion: Delivering enough persuasive information to assist customers in making their decision.
Lead Generation: Integrating a “carrot” for the prospect while also setting appropriate expectations.
Adoption/Utilization: Making the capabilities, value and output of the interaction immediately evident.
Engagement: Finding that one compelling feature that will get them coming back for more and telling their friends.
Satisfaction Retention: Making their primary goals seamless, and linked to their overarching workflow.
Awareness/Volume: Moving from dealing with incoming requests to a destination site and towards outreach via a coordinated digital communications strategy.
Channel Migration: Enhance the awareness around and ease of use for the self service channel.
Internal Resource Optimization: Understand real workflows, integrate systems improve tools.
Telling the Story
In some situations, we are brought in during a research, strategy and planning period, where we can make recommendations which will craft the experience roadmap for the organization. In other situations we are engaged for tactical deliveries against an existing roadmap.
In either situation, the initiatives on the roadmap can be prioritized based upon impact to the user, relative impact to the business, and cost associated with design and implementation. Usually with some investigation, business goals can be tied to something measurable and in some cases a dollar value can be estimated.
A simple business case for an enhancement, or larger initiative may include the following details:
What is it?
What value does it create for the user?
What business goal does it address?
How will we know that it has been successful?
What is the approximate corresponding business value?
What is the approximate cost to implement?
After this information is put together, it becomes very clear where the organization should focus their design and development efforts.
Bio: Amy Cueva is Co-Founder and Chief Experience Officer at Mad*Pow, a user centered design studio in Portsmouth, NH. She has built Mad*Pow’s user centered design methodology as the vehicle to synergize business goals, user needs, and technology requirements. She partners with clients like Google, Aetna, Fidelity, and Monster to create strong cross-channel digital strategies, first class user experiences, and streamlined internal processes. She is the secretary and one of the charter members of the NH UPA, and is co-presenting with Chauncey Wilson at National UPA 2008, A Portfolio of Brainstorming Techniques: From the Nominal Group Technique to the Lotus Blossom Approach.
Physical and Online Card Sorts: A Practical Overview and Case Study
Presenters: Bob Thomas, Katelyn Thompson
Summary: Card sorts are an effective usability method for driving the information architecture of websites and applications. “Card sorting is excellent for situations where you want the users mental model to drive the information architecture of the product” (Courage & Baxter, 2005, pp. 415-416). “A card sorting exercise can help you understand the match between the end users mental model and the information designs conceptual model” (Arnowitz et al, 2007).
We present the results of an open, physical card sort for a corporate intranet, in which employees of the corporation participated. We recommend a new navigational hierarchy and organization, based on the results of the card sort. We will also present the results of the card sort using two popular card sort analysis tools, including:
Donna Maurers card sort analysis spreadsheet
EZCalc and USort, freely available card sort applications originally developed but discontinued by IBM
The presentation discusses the relative merits of open vs. closed card sorts, as well as physical vs. online card sorts. It also gives a practical overview of the most popular online card sorting applications, such as:
The presentation revolves around a case study that details the navigational hierarchy and organization of Insite, the intranet web site of Measured Progress, a not-for-profit educational assessment company. After an initial overview of intranet design and business objectives, the presentation focuses on the pros and cons of broad-and-shallow versus narrow-and-deep navigational categories in creating the primary and secondary levels of an information architecture. It also details how users access information on corporate intranets. The presentation analyzes Insites current navigational hierarchy and employees use of Insite, presents the results of an open card sort in which employees participated, and recommends a new navigational hierarchy and organization, with the aim of improving the efficiency, effectiveness, and satisfaction of Measured Progress employees. We will show dendrograph results and wireframes of new designs.
The presentation will conclude with questions from the audience.
Bios: Bob Thomas is a Research Associate in the Design and Usability Center. He has played critical roles in testing and evaluation projects for clients such as Partners Healthcare, Siemens, the New York State Department of Health, Historic New England, XaR, and a manufacturer of insulin pumps. Before joining the DUC, Bob was the Director of Product Management for Bitstream Inc. in Cambridge, Massachusetts, managing typeface design and font technology products. He has extensive industry experience in usability, product management, consulting, technical writing, and graphic design. Bob is currently pursuing an MS in Human Factors in Information Design at Bentley College, and is expected to graduate in May 2008.
Katelyn is a Research Associate in the Design and Usability Center. She has experience with usability testing, field research, information architecture, web design, and media arts. While working at the DUC Katelyn has conducted usability testing, field research, and expert reviews for clients such as Planned Parenthood and Lifeline. Before joining the DUC, Katelyn was a contractor at MassMutual Financial Group where she performed usability tests, card-sorting exercises, and field research. Katelyn holds a BS in Information Design and Corporate Communications and Liberal Studies from Bentley College. She is currently pursuing a MS in Human Factors in Information Design at Bentley College.
8 Simple Rules (for Designing/Developing a Really Great Product That Your Customers Will Love and Want to Actually Use)
Presenter: Kenny Kutney
Summary: All too often, software projects and products fail, even when they have great UIs. Some of the reasons for that are UI-related. This presentation will highlight some of the most egregious of those issues, and help you think about software design a bit more holistically. Topics covered include customer focus, requirements triage, the role of UI in a software product, avoiding the stuff engineers love to do, the dark side of style guides, standardization and art. I’ll share some of my personal experiences and “lessons learned”, and show some fun examples, both good and bad.This is about a 30-minute presentation targeted primarily for non-usability software development professionals, ideally upper-level managers. For us UX folks, it’s a lot of motherhood and apple pie (with a few generous chunks of controversy on top), but, if you’re new to the UX world, or you’re managing a non-UX group that has need of a UX skill set, then this is for you.
Bio: Kenny Kutney is a New Hampshire-based independent interaction and UI designer and strategist. He has 20 years experience designing award-winning software products, web applications and websites for startups to large corporations, including Akoura Biometrics, Digital Equipment Corporation, Concord Communications and NuMega Technologies.
Peer Coaching: a Simple & Effective Way for User Experience Teams to Help Each Other
Presenter: Amy Kidd
Summary: Recently, our user experience group has grown at a fast pace. In less than 5 years, we moved from a smaller team all reporting to one manager to a group of 35, with 5 distinct subteams. Along with this growth, the need for a larger support structure for employees has emerged. In addition to support and coaching from managers, we needed to provide a structure that enabled team members to support one another.
Our management team identified a number of challenges we wanted to address. These included:
Management bandwidth was limited due to the increase in staff, hiring efforts, and many other initiatives. We needed a way to help individuals receive coaching beyond the amount the management team was able to provide.
We needed to leverage the few “old timers” in the group to help new people make the transition to our company culture.
We wanted to encourage more long term partnerships to help individuals learn from each other.
We wanted a way to leverage our senior team members to provide coaching on UCD methods and techniques to junior employees.
We wanted to encourage greater usage of informal, one-on-one design reviews to improve our deliverables.
We wanted more cross-subteam interaction to build relationships within the group and to help facilitate consistent design standards across our products.
We wanted a way to better leverage our teams wide variety of skills and experience as a resource for individuals within the team.
The solution to this is a peer coaching program in which teammembers are paired with partners for six month cycles. This program has the following goals:
Use our skills to help each other
Use one-on-one design reviews to improve our work
Build relationships with other Usability folks
Everyone learns a little about projects outside their area
Weve been using peer coaching for over a year and the program has proven to be a successful solution to the concerns we were trying to address.
This presentation will explain the challenges we faced, the peer coaching program designed and implemented, and the results obtained. The closing of the presentation will include information and tools the help other user experience groups create similar programs.
Goals for the Session
Attendees at this session will:
Learn about one companys experiences introducing a peer coaching program
Understand the benefits of using a peer coaching program to facilitate user experience professionals supporting each other
Understand the kinds of problem a peer coaching program may address
Leave with a plan and templates for starting their own peer coaching program
Handouts or Other Session Material
One page summary of the material
One page plan for a peer coaching program
Template documents and checklists to use in a peer coaching program
Bio: Amy Kidd is a manager in Usability at The MathWorks, Inc. Amy has provided user-centered design support to software development teams at The MathWorks for over 7 years. She also maintains and delivers a User-Centered Design Course, which introduces engineers, quality engineers, and documentation writers to user-centered design and teaches basic skills in gathering requirements and designing user interfaces.
Previously, Amy worked in Development Training and was responsible for helping subject matter experts design and deliver training courses on technical topics and processes. Before joining The MathWorks, she was a Consultant with PricewaterhouseCoopers.
Amy is currently pursuing an MS in Human Factors in Information Design at Bentley College. As an undergraduate, she attended University of Virginia and studied Math, Cognitive Science, and Computer Science.
How IBM Maximized User Input in the Design of Lotus Notes 8
Presenter: Elizabeth M. Comstock
Summary: Customer input is essential when designing and developing a usable and enjoyable user interface. The Lotus Notes User Studies Team embarked on a 2½-year mission to improve the user experience of Lotus Notes 8. We used variations on at least a dozen usability methods, from standard methods such as laboratory usability tests to more custom methods, such as spec reviews with customers, public blogs, and testing multiple participants at customer sites with our fully functional traveling test environment. In all, we included usability and design input from several thousand customers.
We would like to share the successes and challenges weve faced while working on Lotus Notes, a mission-critical collaboration product with approximately 120 million users worldwide. During this presentation we will describe:
How we began the process and came together as a team
The methods we used, highlighting those that worked best
The importance of the close relationship we shared with the User Interface Designers
Tracking our feedback in our own “Usability Scorecard Database”
While discussing the methods weve used, we will invite the audience members to share their experiences, whether the methods weve described have been successful for them, and whether theyve employed alternate methods that we might try!
Bio: Elizabeth M. Comstock, PhD, leads the Lotus Notes User Studies Team for IBM Software. Other members of the team and co-authors of this presentation are Sheri Branco and Deborah Maurer.
Prior to joining IBM in 2004, Betsy worked on the usability of collaborative hardware and software products at Polycom, PictureTel, and Digital Equipment Corporation.
The Convergence of Usability and Web Analytics
Presenter: Dan Berlin, Bob Thomas
Summary: Organizations today use web analytics tools to track hits and page views but a hit is only a line in a web log. What do you do with that data?
Web analytics is no longer just about hits and page view traffic. Mechanisms are available that allow a practitioner to get a fully-rounded view of the user via their navigation behavior. In the realm of usability, we tend to concentrate on qualitative information, which serves us well. However, quantitative data can be more salient to clients. Web analytics provides this quantitative data and offers a perfect complement to usability studies. Demographics are only part of the equation. As Joseph Carrabis, Principal of Next Stage Evolution, states, “I know who you are within two seconds, but those are the cheap seats. That doesnt interest me anymore.” Companies want to determine how the online behavior of their website visitors aligns with their business goals and strategic direction. Or as Judah Phillips, Director of Web Analytics at Reed Business, says, “Web analytics refers to the collection, measurement, reporting, and analysis around qualitative and quantitative information related to the behavior of an online audience.”
This presentation will inform usability practitioners about the pros and cons of incorporating web analytics into their work. It will discuss different approaches to web analytics, including:
“Integration of website activity… with the offline business”
“Navigation behavior of the visitor”
“Psychological engagement of the visitor”
“Physiological measurements of the visitor”
“Return on investment (ROI), as measured by key performance indicators (KPI)”
- Paul Legutko, Vice President of Analytics, Semphonics
To track demographics and behavior, companies use web analytics software to create data models so that they can analyze those demographics and behaviors, and then inspect web log data to see how past behaviors differ from current behavior. They want to identify what theyve done differently to encourage visitors to travel through their site, and what current and unique visitors they are attracting.
The presentation will include video clips from a round table discussion with practitioners in the field of web analytics, including:
Joseph Carrabis, Principal, Next Stage Evolution
Jack Carroll, Principal, VOC Partners
Paul Legutko, Vice President of Analytics, Semphonics
Judah Phillips, Director of Web Analytics, Reed Business Information
We will also be discussing web analytics methods, companies, and solutions that are available to supplement traditional usability work.
The presentation will conclude with questions from the audience.
Bios: Dan Berlin is a full-time MBA+MSHFID student at Bentley College. Originally from Long Island, Dan came to Massachusetts to attend Brandeis University where he earned a B.A. in Psychology. Before attending Bentley College, Dan worked for seven years at SeaChange International, a video server company in Acton, MA. After completing his studies at Bentley, Dan plans on working for a few years, and then going back to school for a PhD.
Bob Thomas is a Research Associate in the Design and Usability Center. He has played critical roles in testing and evaluation projects for clients such as Partners Healthcare, Siemens, the New York State Department of Health, Historic New England, XaR, and a manufacturer of insulin pumps. Before joining the DUC, Bob was the Director of Product Management for Bitstream Inc. in Cambridge, Massachusetts, managing typeface design and font technology products. He has extensive industry experience in usability, product management, consulting, technical writing, and graphic design. Bob is currently pursuing an MS in Human Factors in Information Design at Bentley College, and is expected to graduate in May 2008.
Widening Usabilitys Reach: Extending Usability Methods to Marketing and Design
Presenter: Ashley Annis
Summary: This talk will present a case study of usability specialists testing a consumer website for usability, marketing (branding and messaging), and graphic design. We present four methods, one tool, and sample deliverables for adapting traditional usability methods for purposes other than traditional user experience. The methods include a questionnaire, a discussion guide, and two methods known as “clicky” and “sticky.” The tool in question is Uzantos MindCanvas. Deliverables include a test plan and a results document. We share lessons learned and questions that remain unanswered. Attendees will participate via hands-on demonstrations of methods and tools. For example, we will demonstrate one of our methods/tools by asking volunteers to interact with one of the tools we used during face-to-face usability sessions with users.
The case study will show that the methods we helped design and implement were successful: they met project goals, facilitated communication among different team members, and resulted in findings deliverables that were easily transferable beyond the bounds of the usability effort. The findings documents could be used by different stakeholders, to serve their purposes; for example: the business sponsors, who needed to justify the project to upper level management; marketing, who needed to report market segments and justify marketing efforts; and information architects and designers, who needed confirmation of their work and/or recommendations for improving upon it, etc. We examined the clients demands and interviewed stakeholders in order to delineate purposes for each demand. Findings from the stakeholder interviews showed that the client had the following questions:
Ensure that the site is usable. The client wanted to know: Can users find key content and features? Can they use those features with ease? Do they like the content and features?
Investigate whether the level of company branding is appropriate. The client wanted to know: How strongly or lightly should the site be branded?
Confirm that messaging and various copy best serves users interests while meeting the marketings objectives. The client wanted to know: How well is messaging working? How do users respond to snippets of copy and special offers?
Validate the content, prominence, and placement of imagery. The client wanted to know: What should be the images emphasize: families? Diverse populations? Should the site feature more images, or fewer?
Using the methods we employed, we were able to address the clients demands: The project was a success in part because the deliverables and methods were easily adapted by outside market researchers, and the findings deliverables were easy for the client to extend for reporting progress to higher level executives. Although we originally perceived the effort as extending beyond traditional notions of usabilityindeed, the client demands *do* slip into what have been traditionally perceived as marketing and graphic designwe found by the end of the project that by extending our usability approaches we were best positioned to answer questions that neither the market researchers nor the graphic designers were able to answer. Put another way, we found that other professionals extend what usability practitioners are able to accomplish with methods traditional to usability.
Bio: Ashley Annis, Ph.D., is the Director of User Experience at Bridgeline Software. She is responsible for all user experience and usability aspects of the web applications delivered out of Bridgelines New England office. Ashley is a usability specialist with over 10 years of experience that spans multiple industries: Prior to joining Bridgeline Software, she provided usability services for pharmaceutical and life sciences industries, higher education, financial services, and high tech. Across these industries, Ashley has led engagements at multinational corporations as well as at entrepreneurial start-ups, non-profits and not-for-profits, professional organizations and associations, and at large colleges and universities.
Ashley holds a Ph.D. from Rensselaer Polytechnic Institute and is currently Information Director and serving on the Board for the Association of Computing Machinery’s Special Interest Group on the Design of Communication (SIGDOC). She is also a member of various user experience organizations, including the ACM’s SIGCHI. Ashley has taught user experience courses to undergraduates and graduate students and has given over a dozen talks at national and international conferences on topics related to usability and user experience design.
Metaverses and User Experience
Presenter: Stephen Denning
Summary: In recent years, we have seen the rise in popularity of metaverses, such as Second Life. These metaverses allow users to explore a rich 3D environment through their personalized “avatar”. Second Life, in particular, is a virtual world that aims to be 3D version of the internet. It has become a favorite talking point with big businesses – many of whom now have a presence there, launching new hotels, testing cars and opening virtual stores and offices. Despite some slow down after the huge initial hype, it looks like there is more to come. According to research house Gartner 80% of internet users will visit sites like Second Life by 2011.
The biggest challenge with leveraging technology such as this, is separating the hype from the useful benefit. Through my work with PA Consulting Group (a global management and technology consultancy), I have been looking at how companies can effectively use channels such as Second Life for reaching new customers, “virtualizing” existing business, testing prototypes and carrying out focus groups. This gives us a potential new channel for performing some of the user-centric design work that we do.
In this presentation I would plan to discuss the background to Second Life, how I have seen it being used effectively (using examples from PA Consulting Group), and how we could potentially utilize it in our work. The session would be partly presentation and partly hands-on with the opportunity to check out some existing projects in Second Life.
Bio: Stephen Denning is a Research Associate at the Bentley College Design and Usability Center. He has ten years experience in IT consulting and software delivery, working as a Principal Consultant for a leading global management and technology consultancy firm based in the UK. In this role he worked to deliver innovative solutions to complex problems for a wide range of clients in both the private and public sector.
Stephen’s experience in usability and user-centric design has been applied to the successful delivery of many client projects and he is now building on that expertise by pursuing a MS in Human Factors in Information Design at Bentley. Stephen holds an Honors degree in Computer Science from Heriot-Watt University in Edinburgh, Scotland, and is a member of the British Computer Society.
Online Consumer Health
Presenter: Lisa Neal
Summary: With consumer-directed care, patients are being asked to play a greater role in their health care. Moreover, those with chronic diseases often get better counsel from other sufferers than they do from physicians. This session will cover the most effective ways to design and evaluate health web sites and online health communities.
Changes in the health care system and the pervasiveness of the Internet have led to an increased use of the Internet by healthcare consumers. 80% of people in the US who use the Internet are using it for health searches. Health web sites and online health communities provide a means for patients and their families to learn about an illness and seek support. The importance of online consumer health is evidenced by the popularity of sites such as WebMD and RevolutionHealth. Consumer health sites have a significant impact on the quality of life of their users who often turn to them before seeking medical help. Health web sites and online health communities raise difficult design challenges. These challenges include wide variability of participant’s medical expertise, health literacy, and technology literacy. A major risk is the potential consequences when poor advice is taken or when professional treatment is not sought.
By participating in this interactive discussion you will learn:
How health web sites and online health communities are being used by patients and their loved ones
How the nature of the disease or illness impacts site design
How innovative Web 2.0 technologies can enhance participation
The disconnect that exists between patients and providers and what can be done about it
Bio: Lisa Neal is an Adjunct Clinical Professor at Tufts University School of Medicine, where she teaches a course on Online Health Communities. She has designed and evaluated online communities for clients including Michael J Fox Foundation, Environment and Health Group, and the NIH. Lisa is Editor-in-Chief of eLearn Magazine. She holds a Ph.D. in Computer Science from Harvard University. Lisa’s blog on health and education is at http://lisaneal.com.
Streamlined Product Evaluation
Presenter: Catherine Weeks
Summary: This session will focus on a method of performing effective evaluations of a product when given minimal resources. These resources constraints may include some combination of staffing, scheduling, and monetary restrictions. Usability practitioners will find that by using this method, they will be able to provide feedback to the product team in terms of usability, product effectiveness, and potential customer satisfaction. The presenter will illustrate how this evaluation method was used in assessing two business line products. Analysis of the two case studies will help to illustrate the pros and cons of the evaluation method.
This presentation will include a group discussion of how individuals define “Heuristic Review”, “Expert Review”, and “Product Walkthrough”, and how the groups definition of these concepts compares to how they are implemented in this evaluation method.
The session will end with time for Q&A.
Bio: Catherine is an employee of SeaChange International in Acton,Massachusetts. As a User Experience Engineer at SeaChange, she has worked on a variety of business line products as well as concepts designs for consumer products. Her responsibilities have ranged from requirements definitions to usability testing across different technology platforms. In her personal time,Catherine enjoys digital photography and freelance web design.
Moving up the Usability Food Chain: A Cross-Disciplinary Holistic Viewpoint
Presenter: Alex Conn, Ph.D.
Summary: What role does usability play in a rapidly changing technology marketplace where new products seem to appear daily and companies pop up and disappear almost as often? The Web has transformed the way we interact, the way we transact, and even the way we think. Concepts such as disruptive innovations, the “flattened world,” and value propositions provide only partial insights into what is happening in this confusing arena.
When companies purchase or build solutions, they are generally addressing underlying problems or going after a potential opportunities. These often-complex architectures aim to transform the business by providing added value to their customers. While traditional customer value propositions have focused on functions, features, and cost, we believe another dimension is equally important: the fit between the solution and the users “way of doing things.”
Businesses customers and internal users must interact not only with user interfaces, but also with team cultures, organizational structures, and policy designs that can completely overshadow the areas that we traditionally consider to be “usability.” In this presentation we will consider the drive toward consumer focused companies and how the success of everything from mass customization to web commerce is often characterized by the match between the interfaces (products, services, web site, advertisements) and an increasingly fragmented user communitys way of doing things. How do we recognize and formalize these ways that user groups do things so that new capabilities provide understandable enhancements rather than surprising and unnecessary changes to the interaction paradigm? Building on Kim Vicentes Human-Tech ladder, we explore many examples where a broad cross-disciplinary view of usability can provide key insights about the success of products, services, and the interactions necessary to obtain them.
We will use slides with an interactive discussion, employing live web examples, if possible.
Bio: Dr. Alex Conn has worked for many years on technology architectures in which usability has been a core design consideration. At Digital Equipment, he pioneered work on security architecture and policy, internationalization, and solution architecture. At the 1995 national CHI conference he presented his Time Affordances paper. His current solution architecture methodology work explores how stakeholder usage architecture is key to the success of any solution. He presented his experiences in designing Blackberry interfaces at a startup at the 2005 UPA Mini-conference. As Principal at APConnsulting, LLC, Dr. Conn develops and teaches corporate courses on Solution Architecture. He is working on a book that helps the solution buyer navigate the complicated marketplace of products, services, and the interaction environments they provide. His Ph.D. in Computer Engineering is from University of California, Berkeley before teaching at the University of Maryland and recently in at the University of Massachusetts, Boston honors department.