Page layout, content and navigation are not always designed at the same time. It is often necessary to work through at least part of these processes separately. As a result, it may not be possible to test layouts with realistic content until a relatively late stage in the design process, meaning that usability problems relating to the layout may not be found at the appropriate time.

Various solutions exist for this problem. One is the possibility of testing early prototype layouts containing ‘greeked’ text – that is, the ‘lorem imsum’ placeholder text commonly used for layout design [1]. A method for testing the recognisability of page elements was discussed in Neilsen’s Alertbox back in 1998 [2], though the concept originated with Thomas S. Tullis [3].

Technique

Testing will require several users – around six is helpful without being excessively time-consuming. Ensure that they have not seen or discussed the layouts before the test! First, create a list of elements that should be visible upon the layout. Nielsen provides a list of nine standard elements that are likely to present on all intranet pages – but in your particular case you may wish to alter this list a little to encompass all of the types of element present on this template.

Give each test user a copy of each page – in random sequence, to eliminate any systematic error that might result from carrying the experience with the first page through to the second. Ask the test user to draw labelled blocks around the parts of the page that correspond to the elements you have identified. Depending on circumstances, you may find that encouraging the user to ‘think aloud’ may provide useful information, but be careful not to ‘lead’ the user to a preferred solution.

Finally, ask the user to give a simple mark out of ten for ‘appeal’. This is not a very scientific measure, but is nonetheless of interest since this allows you to contrast the user’s subjective measure of preference against the data that you have gathered (the number of elements correctly identified). Nielsen points out that the less usable page is often given a higher average mark by the user.

Scoring The Test

With the information provided, draw a simple table:

This provides you with a basic score. You will probably also find your notes from think-aloud sessions to be very useful in identifying the causes of common misunderstandings and recommending potential solutions.

When Should Page Template Evaluation Be Carried Out?

This technique can be applied on example designs, so there is no need to create a prototype Web site; interface ideas can be mocked up using graphics software. These mockups can be tested before any actual development takes place. For this reason, the template testing approach can be helpful when commissioning layout template or graphical design work. Most projects will benefit from a user-centred design process, an approach that focuses on supporting every stage of the development process with user-centred activities, so consider building approaches like this one into your development plans where possible.

Conclusions

If a developing design is tested frequently, most usability problems can be found and solved at an early stage. The testing of prototype page layouts is a simple and cheap technique that can help to tease out problems with page layout and visual elements. Testing early and often can save money by finding these problems when they are still cheap and simple to solve.

It is useful to make use of various methods of usability testing during an iterative design and development cycle, since the various techniques often reveal different sets of usability problems – testing a greeked page template allows us to separate the usability of the layout itself and the usability of the content that will be placed within this content [2]. It is also important to evaluate issues such as content, navigation mechanisms and page functionality, by means such as heuristic evaluation and the cognitive walkthrough – see QA Focus documents on these subjects [4] [5]. Note that greeked template testing does look at several usability heuristics: Aesthetic & minimalist design and Consistency and standards are important factors in creating a layout that scores highly on this test.

Finally, running tests like this one can help you gain a detailed understanding of user reactions to the interface that you are designing or developing.

References

1. Lorem Ipsum Generator,
   <http://lorem-ipsum.perbang.dk/>
2. Testing Greeked Page Templates, Jakob Nielsen,
   <http://www.useit.com/alertbox/980517.html>
3. A method for evaluating Web page design concepts, T.S. Tullis. In ACM Conference on Computer-Human Interaction CHI 98 Summary (Los Angeles, CA, 18-23 April 1998), pp. 323-324.
4. Introduction To Cognitive Walkthroughs, QA Focus briefing document no. 87,
   <http://www.ukoln.ac.uk/qa-focus/documents/briefings/briefing-87/>
5. Heuristic Evaluation, QA Focus briefing document no. 89,
   <http://www.ukoln.ac.uk/qa-focus/documents/briefings/briefing-89/>

Further Information

• Getting Started With User Centred Design, UsabilityNet,
  <http://www.usabilitynet.org/tools/gettingstarted.htm>

Although, as described in [1] AJAX can enhance the usability of Web-based services developers need to be aware of various usability problems which may be encountered.

Challenges For AJAX Developers

Concept of State

Once a Web page has been downloaded it has traditionally remained static. AJAX uses dynamic Web page updates which means that state transition (the move from one page view to another) is more complex, as separate elements may update asynchronously. AJAX applications frequently do not store application state information; this breaks the ‘back’ button functionality of the browser. Many Web users use the back button as their primary means of navigation and struggle to control the system without it.

AJAX requires developers to explicitly support this functionality in their software, or use a framework that supports it natively. Various solutions to this problem have been proposed or implemented, such as the use of invisible IFRAME elements that invoke changes which populate the history originally used by the browser’s back button.

A related issue is that as AJAX allows asynchronous data exchange with the server, it is difficult for users to bookmark a particular state of the application. Solutions to this problem are appearing. Some developers use the URL anchor or fragment identifier to keep track of state and therefore allow users to return to the application in a given state.

The asynchronous nature of AJAX can also confound search engines which traditionally record only a page’s static content. Since these usually disregard JavaScript entirely, an alternative access must be provided if it is desirable for a Web page to be indexed.

User Expectations

There are certain expectations of how Web-based information will be displayed and processed. Without explicit visual clues to the contrary, users are unlikely to realise that the content of a page is being modified dynamically. AJAX applications often do not offer visual clues if, for example, a change is being made to the page or content is being preloaded. The usual clues (such as the loading icon) are not always available. Solving this requires designers to explicitly support this functionality, using traditional user interface conventions wherever possible or alternative clues where necessary.

Response Time

AJAX has the potential to reduce traffic between the browser and the server as information can be sent or requested as and when required. However, this ability can easily be misused, such as by polling the server for updates excessively frequently. Since data transfer is asynchronous, a lack of bandwidth need not be perceivable to the user; however, ensuring this is the case requires smart preloading of data.

Design Issues

AJAX provides techniques that previously were available only by using DHTML or a technology like Flash. There is a concern that, as with previous technologies, designers have access to a plethora of techniques that bring unfamiliar usability or accessibility problems. Gratuitous animation, pop ups, blinking text and other distractions all have accessibility implications and stop the user from fully focussing on the task at hand.

Accessibility

Most methods of AJAX implementation rely heavily on features only present in desktop graphical browsers and not in text-only readers. Developers using AJAX technologies in Web applications will find attempting to adhere to WAI accessibility guidelines a challenge. They will need to make sure that alternate options for users on other platforms, or with older browsers and slow Internet connections, are available.

Conclusions

The concerns surrounding adoption of AJAX are not unfamiliar. Like Flash, the technologies comprising AJAX may be used in many different ways; some are more prone to usability or accessibility issues than others. The establishment of standard frameworks, and the increasing standardisation of the technologies behind AJAX, is likely to improve the situation for the Web developer.

In the meantime, the key for developers is to remember is that despite the availability of new approaches, good design remains essential and Jacob Nielson’s Ten Usability Heuristics [2] should be kept in mind. AJAX applications need to be tested to deal with the idiosyncrasies of different browsers, platforms and usability issues and applications should degrade gracefully for those users who do not have JavaScript enabled.

Note that as the use of AJAX increases and more programming libraries become available, many of the issues will be resolved. In parallel it is likely that over time browsers will standardise and incorporate better support for new technologies.

References

1. An Introduction To AJAX, Cultural Heritage briefing document no. 43, UKOLN,
   <http://www.ukoln.ac.uk/cultural-heritage/documents/briefing-43/>
2. Ten Usability Heuristics), Useit.com,
   <http://www.useit.com/papers/heuristic/heuristic_list.html>

When designing a Web site or program, the obvious question to ask at once is, “who are my audience?” It seems natural to design with users in mind, and just as natural to wish to build a product that is satisfactory to all one’s users – however, experience shows that it is difficult to design something that appeals to everybody [1]. Instead, it is useful to start with a few sample profiles of users, typical examples of the audience to whom the design should appeal, and design to their needs. Not only is it easier for the designer, but the result is usually more appealing to the user community.

Researching A User Persona

The first step in developing a user persona is to learn a little about your users; qualitative research techniques like one-to-one interviews are a good place to start. It’s best to talk to several types of users; don’t just focus on the single demographic you’re expecting to appeal to, but consider other groups as well. Focusing on one demographic to the exclusion of others may mean that others do not feel comfortable with the resulting design, perhaps feeling alienated or confused. The expected result of each interview is a list of behaviour, experience and skills. After a few interviews, you should see some trends emerging; once you feel confident with those, it’s time to stop interviewing and start to build personas.

Developing A User Persona

Once you have an idea of each type of persona, write down the details for each one. It may help to write a sort of biography, including the following information:

• Vital statistics: name, age, gender and personality details (shy, timid, outgoing?)
• Interests and hobbies
• Experience and education
• Motivation

You can even find a photograph or sketch that you feel fits the personality and add it to the persona’s description.

Why User Personas?

The intent behind a user persona is to create a shared vocabulary for yourself and your team when discussing design questions and decisions. User personas provide easy-to-remember shorthand for user types and behaviour, and can be used to refer to some complex issues in a simple and generally understood way. Sharing them between management and development teams, perhaps even with funders, also provides a useful avenue for effective communication of technical subjects. Furthermore, it is much easier to design for a persona with whom one can empathise than for a brief, dry description of user demographics.

It is good practice, when making design decisions, to consider each user persona’s likely reaction to the result of the decision. Which option would each user persona prefer?

User personas can also feed in to discount usability testing methods such as the cognitive walkthrough, saving time and increasing the effectiveness of the approach.

Finally, the research required to create a user persona is an important first step in beginning a user-centred design process, an approach that focuses on supporting every stage of the development process with user-centred activities, which is strongly recommended in designing for a diverse user group.

Conclusions

User personas are a useful resource with which to begin a design process, which allow the designers to gain understanding of their users’ expectations and needs in a cheap and simple manner, and can be useful when conducting discount usability testing methods. Additionally, they make helpful conversational tools when discussing design decisions.

References

1. The Inmates are Running the Asylum, Alan Cooper, ISBN: 0672316498

Further Information

• Ten Usability Heuristics, Jakob Nielsen,
  <http://www.useit.com/papers/heuristic/heuristic_list.html>
• Getting Started With User Centred Design, UsabilityNet,
  <http://www.usabilitynet.org/tools/gettingstarted.htm>

Heuristic evaluation is a method of user testing, which enables a product to be assessed in order to identify usability problems – that is, places where the product is not easy to use. It is a discount (“quick and dirty”) method, which means that it is cheap and requires relatively little expertise.

What’s Involved In Heuristic Evaluation?

In this technique, a number of evaluators are first introduced to the heuristics, then given some tasks to complete and invited to report the problems – where the system fails to comply with the heuristics – either verbally or in some form of written report or checklist. Unlike many forms of usability testing, the evaluators do not have to be representative of the system’s expected users (although they can be!), nor do the evaluators have to be experts, as the heuristics can be read and understood in a few minutes. Just three to five evaluators are needed to find the majority of usability problems, so the technique is quite efficient and inexpensive.

The problems found in heuristic evaluation essentially represent subjective opinions about the system. Evaluators will frequently disagree (there are no absolute right or wrong answers) but these opinions are useful input to be considered in interface design.

What Heuristics Should I Use?

There are several sets of possible heuristics available on the Web and elsewhere. This reflects the fact that they are “rules of thumb”, designed to pick out as many flaws as possible, and various sets of usability evaluators have found different formalisations to be most useful for their needs, e.g. [1]. Probably the most commonly used is Nielsen’s set of ten usability heuristics [2] given below with a sample question after each one:

• Visibility of system status: Does the system give timely & appropriate feedback?
• Match between system and the real world: Is it speaking the users’ language?
• User control and freedom: How hard is it to undo unwanted actions?
• Consistency and standards: Does it follow conventions and expectations?
• Error prevention: Are potential errors recognised before becoming a problem?
• Recognition rather than recall: Does the system rely on the users’ memory?
• Aesthetic & minimalist design: Are dialogs cluttered with information?
• Help users recognise, diagnose & recover from errors: Are error messages useful?
• Help and documentation: Is there online help? Is it useful?

An excellent resource to help you choose a set of heuristics is the Interactive Heuristic Evaluation Toolkit [3] which offers heuristics tailored to your expected user group, type of device, and class of application.

When Should Heuristic Evaluation Be Carried Out?

As heuristic evaluation is simple and cheap, it is possible to use it to quickly test the usability of a web site at any stage in its development. Waiting until a fully functional prototype Web site exists is not necessary; interface ideas can be sketched out onto paper or mocked up using graphics software or Flash. These mockups can be tested before any actual development takes place.

Most projects will benefit from a user-centred design process, an approach that focuses on supporting every stage of the development process with user-centred activities. It is advisable to test early and often, in order to ensure that potential problems with a design are caught early enough that they can be solved cheaply. However, even web sites that are already active can benefit from usability testing, since many such problems are easily solved, but some problems are difficult or expensive to solve at a late stage.

Conclusions

If a developing design is tested frequently, most usability problems can be found and solved at an early stage. Heuristic evaluation is a simple and cheap technique that finds the majority of usability problems. An existing Web site or application will often benefit from usability testing, but testing early and often provides the best results. Finally, it is useful to alternate use of heuristic evaluation with use of other methods of usability testing, such as user testing, since the two techniques often reveal different sets of usability problems.

References

1. Heuristic Evaluation – A System Checklist, Deniese Pierotti, Xerox Corp.
   <http://www.stcsig.org/usability/topics/articles/he-checklist.html>
2. Heuristic Evaluation, Jakob Nielsen,
   <http://www.useit.com/papers/heuristic/>
3. Interactive Heuristic Evaluation Toolkit,
   <http://www.id-book.com/catherb/>

Further Information

• Ten Usability Heuristics, Jakob Nielsen,
  <http://www.useit.com/papers/heuristic/heuristic_list.html>
• Getting Started With User Centred Design, UsabilityNet,
  <http://www.usabilitynet.org/tools/gettingstarted.htm>

A key issue in usability is that of understanding users, and a key part of user-centred design is that of describing the tasks that the users expect to be able to accomplish using the software you design [1]. Because of the origins of usability as a discipline, a lot of the terminology used when discussing this issue comes from fields such as task analysis. This briefing paper defines some of these terms and explains the relationship between usability and task analysis.

What Is Task Analysis?

Within the usability and human-computer interaction communities, the term is generally used to describe study of the way people perform tasks – that is, the way in which a task is currently performed in real-life situations. Task analysis does not describe the optimal or ideal procedure for solving a problem. It simply describes the way in which the problem is currently solved.

Gathering Data For Task Analysis

Since the intent of task analysis is description of an existing system, the ideal starting point is data gathered from direct observation. In some cases, this is carried out in a controlled situation such as a usability laboratory. In others, it is more appropriate to carry out the observation “in the field” – in a real-life context. These may yield very different results!

Observational data can be gathered on the basis of set exercises, combined with the “think-aloud” technique, in which subjects are asked to describe their actions and their reasoning as they work through the exercise. Alternatively, observations can be taken by simply observing subjects in the workplace as they go through a usual day’s activities. The advantage of this latter method is principally that the observer influences events as little as possible, but the corresponding disadvantage is that the observations are likely to take longer to conclude.

Unfortunately, there are significant drawbacks of direct observation, principally cost and time constraints. For this reason, task analysis is sometimes carried out using secondary sources such as manuals and guidebooks. This, too, has drawbacks – such sources often provide an idealised or unrealistic description of the task.

A third possibility is conducting interviews – experts, themselves very familiar with a task, can easily answer questions about that task. While this can be a useful way of solving unanswered questions quickly, experts are not always capable of precisely explaining their own actions as they can be too familiar with the problem domain, meaning that they are not aware on a conscious level of the steps involved in the task.

Analysing Observations

There are several methods of analysing observational data, such as knowledge-based analysis, procedural [2] or hierarchical task analysis, goal decomposition (the separation of each goal, or step, into its component elements) and entity-relationship based analysis. Data can also be visualised by charting or display as a network. Some methods are better suited to certain types of task – e.g. highly parallel tasks are difficult to describe using hierarchical task analysis (HTA). On the other hand, this method is easy for non-experts to learn and use. Each answers a slightly different question – for example, HTA describes the knowledge and abilities required to complete a task, while procedural task analysis describes the steps required to complete a task.

A simple procedural task analysis is completed as follows:

1. Choose the appropriate procedure to complete the task that is being analysed.
2. Determine and write down each step in that procedure; break down each step as far as possible.
3. Complete every step of the procedure.
4. Check that the procedure gave the correct result.

These steps can be charted as a flowchart for a clear and easy to read visual representation.

Conclusions

Task analysis provides a helpful toolkit for understanding everyday processes and for describing how human beings solve problems. It is not appropriate to perform detailed task analysis in every situation, due to cost and complexity concerns. However, the results of a task analysis can usefully inform design or pinpoint usability problems, particularly differences between the system designer’s assumptions and the users’ “mental models” – ways of looking at – the task to be performed.

References

1. Task Analysis and Human-Computer Interaction, Crystal & Ellington,
   <http://www.ils.unc.edu/~acrystal/AMCIS04_crystal_ellington_final.pdf>
2. Procedural Task Analysis,
   <http://classweb.gmu.edu/ndabbagh/Resources/Resources2/procedural_analysis.htm>

The cognitive walkthrough is a method of discount (“quick and dirty”) usability testing requiring several expert evaluators. A set of appropriate or characteristic tasks to be completed is compiled. The evaluators then “walk” through each task, noting down problems or difficulties as they go.

Since cognitive walkthroughs are often applied very early in development, the evaluators will often be working with mockups of interfaces such as paper prototypes and role-playing the part of a typical user. This is made much simpler if user personas, detailed descriptions of fictitious users, have been developed, because these simplify the role-playing element of cognitive walkthrough. These are often developed at the beginning of a user-centred design process, because designers often find it much easier to design to the needs of a specific user.

Evaluators are typically experts such as usability specialists, but the same basic technique can also be applied successfully in many different situations.

The Method

Once you have a relatively detailed prototype, paper or otherwise, you are ready to try a cognitive walkthrough.

Start off by listing the tasks that you expect users to be able to perform using your Web site or program. To do this, think about the possible uses of the site; perhaps you are expecting users to be able to book rooms or organise tours, or find out what events your organisation is running in the next month, or find opening times and contact details for your organisation. Write down each of these tasks.

Secondly, separate these tasks into two parts: the user’s purpose (their intention) and the goals that they must achieve in order to complete this. Take the example of organising a tour; the user begins with the purpose of finding out what tours are available. In order to achieve this, they look for a link on your Web site leading to a Web page detailing possible tours. Having chosen a tour, they gain a new purpose – organising a tour date – and a new set of goals, such as finding a Web page that lets them book a tour date and filling it out appropriately.

Separating tasks into tiny steps in this way is known as decomposition, and it is mostly helpful because it allows you to see exactly where and when the interface fails to work with the user’s expectations. It is important to do this in advance, because otherwise you find yourself evaluating your own trial-and-error exploration of the interface! Following these steps “wearing the users’ shoes” by trying out each step on a prototype version of the interface shows you where the user might reach an impasse or a roadblock and have to retrace his or her steps to get back on track. As a result, you will gain a good idea of places where the interface could be made simpler or organised in a more appropriate manner.

To help this process, a Walkthrough Evaluation Sheet is filled in for each step taken. An example is shown below [1]:

1. Will the users be trying to produce whatever effect the action has?
2. Will users see the control (button, menu, switch, etc.) for the action?
3. Once users find the control, will they recognize that it produces the effect they want?
4. After the action is taken, will users understand the feedback they get, so they can go on to the next action with confidence?

Advantages and Disadvantages

Cognitive walkthroughs are often very good at identifying certain classes of problems with a Web site, especially showing how easy or difficult a system is to learn or explore effectively – how difficult it will be to start using that system without reading the documentation, and how many false moves will be made in the meantime.

The downside is principally that on larger or more complex tasks they can sometimes be time-consuming to perform, so the technique is often used in some altered form. For example, instead of filling out an evaluation sheet at each step, the evaluation can be recorded on video [2]; the evaluator can then verbally explain the actions at each step.

Conclusions

‘Cognitive walkthroughs are helpful in picking out interface problems at an early stage, and works particularly well together with a user-centred design approach and the development of user personas. However, the approach can sometimes be time-consuming, and since reorganising the interface is often expensive and difficult at later stages in development, the cognitive walkthrough is usually applied early in development.

References

1. Evaluating the design without users, from Task-Centered User Interface Design,
   <http://hcibib.org/tcuid/chap-4.html>
2. The Cognitive Jogthrough,
   <http://portal.acm.org/citation.cfm?id=142869>

Usability refers to a quality attribute that assesses how easy user interfaces are to use. The term is also used to refer to a number of techniques and methods for improving usability during the various stages of design and development.

What Does Usability Include?

Usability can be separated into several components [1] such as:

Learnability:

How easy it is to get to grips with an unfamiliar interface?

Efficiency:

How quickly an experienced user can perform a given task?

Memorability:

Once familiar with an interface, is it easily forgettable?

Errors:

How easy is it to make mistakes/recover from mistakes?

Satisfaction:

Is the design enjoyable to use?

These characteristics are all useful metrics, although the importance of each one depends on the expected uses of the interface in question. In some circumstances, such as software designed for a telephone switchboard operator, the time it takes for a skilled user to complete a task is rather more important than learnability or satisfaction. For an occasional web user, a web site’s designers may wish to focus principally on providing a site that is learnable, supports the user, and is enjoyable to use. Designing a usable site therefore requires a designer to learn about the needs of the site’s intended users, and to test that their design meets the criteria mentioned above.

Why Does Usability Matter?

More attention is paid to accessibility than to usability in legislation, perhaps because accessibility is perceived as a clearly defined set of guidelines, whilst usability itself is a large and rather nebulous set of ideas and techniques. However, a Web site can easily pass accessibility certification, and yet have low usability; accessibility is to usability what legible handwriting is to authorship. Interfaces with low usability are often frustrating, causing mistakes to be made, time to be wasted, and perhaps impede the user from successfully reaching their intended goal at all. Web sites with low usability will not attract or retain a large audience, since if a site is perceived as too difficult to use, visitors will simply prefer to take their business elsewhere.

Usability Testing

User testing is traditionally an expensive and complicated business. Fortunately, modern discount (‘quick and dirty’) methods have changed this, so that it is now possible to quickly test the usability of a web site at any stage in its development. This process, of designing with the user in mind at all times, is known as user-centred design. At the earliest stages, an interface may be tested using paper prototypes or simple mockups of the design. It is advisable to test early and often, to ensure that potential problems with a design are caught early enough to solve cheaply and easily. However, completed Web sites also benefit from usability testing, since many such problems are easily solved.

User testing can be as simple as asking a group of users, chosen as representative of the expected user demographic, to perform several representative tasks using the Web site. This often reveals domain-specific problems, such as vocabulary or language that is not commonly used by that group of users. Sometimes user testing can be difficult or expensive, so discount techniques such as heuristic evaluation [2], where evaluators compare the interface with a list of recommended rules of thumb, may be used. Other discount techniques include cognitive walkthrough in which an evaluator role-plays the part of a user trying to complete a task. These techniques may be applied to functional interfaces, to paper prototypes, or other mockups of the interface.

A common method to help designers is the development of user personas, written profiles of fictitious individuals who are designed to be representative of the site’s intended users. These individuals’ requirements are then used to inform the design process and to guide the design process.

Conclusions

Considering the usability of a web site not only helps users, but also tends to improve the popularity of the site in general. Visitors are likely to get a better impression from usable sites. Quick and simple techniques such as heuristic evaluation can be used to find usability problems; frequent testing of a developing design is ideal, since problems can be found and solved early on. Several methods of usability testing can be used to expose different types of usability problems.

References And Further Information

1. Usability 101: Introduction to Usability, J. Nielsen,
   <http://hcibib.org/tcuid/chap-4.html>
2. Heuristic Evaluation, J. Nielsen,
   <http://portal.acm.org/citation.cfm?id=142869>

• Ten Usability Heuristics, J. Nielsen,
  <http://www.useit.com/papers/heuristic/heuristic_list.html>

<!–

• Usability Evaluation with the Cognitive Walkthrough, Rieman et al,
  <http://www.bui.fh-hamburg.de/pers/ursula.schulz/eulerev/cogwalk1.htm>

–>

• Personas: Matching a design to the users’ goals,
  <http://www.uie.com/articles/personas/>