Netscape 6.1/Mozilla 0.9.4 Usability Summary Report

August 13, 2001

A usability evaluation was conducted with the Netscape 6.1 client on August 8 & 9, 2001. Issues that are relevant to Mozilla, as well as the commercial client, are summarized here.

A total of six users participated, in the evaluation. User experience ranged from intermediate to advance. Overall, participants were able to effectively use the products once they had located them. It was difficult in some cases for the people to find the various products with the menus or from icons that are provided in the interface. Also, there was a significant amount of confusion regarding the difference between the client based products and the web based counter parts that are provided as part of the Netscape website.

The main issues uncovered during testing are summarized below.

Issue

Recommendation

Ambiguous terms used in some of the products.

Re-evaluate the terms used for top level menus and icons.

Confusion regarding the difference between client based and web-based products.

TBD

Difficulty locating various parts of the client (i.e. address book).

Re-evaluate the menu system and icon locations for products.

Inability to close the sidebar

Provide an easy way to close the sidebar.

Problematical Tasks

Read email and reply to messages

Users 1, 2, 3, and 6 all had to be prompted to access the client email program. These users all chose to access Webmail when asked to read email messages. After accessing Webmail, users were prompted to look for a way to check the other email address that they had been given. None of these four users were able to access the client-based email without help from the test facilitator.
User 4 began each new task by closing the active program and starting the new tasks from the program group. This user started the mail client from the program group.
User 5 was the only person in the evaluation that understood the difference between client-based mail and Webmail.

Add sender to address book

User 1 was very confused about the difference between the collected address book and the personal address book. The user attempted to copy and address from one address book to the other but never succeeded in doing so.
User 2 was unable to locate the address book and had to be prompted to the address book.
Users were mostly successful with this task, however only one user in test clicked on the address in email message to add it to the address book. The other people wrote down the email address and made a new entry in the address book

Use imported book mark

All of the users were able to quickly access the list of bookmarks. However users 2 and 6 noticed the personal bookmarks folder was empty and assumed that no bookmarks were imported from the previous version of Netscape.

Add address to address book from card

All of the users were able to successfully access the address book with some difficulty finding it in the menus or via pull down menus. However, 4 of 6 users edited an existing entry rather than creating a new entry. This was due to some confusion about how to initiate a new address book entry.

Key Findings

Each usability concern is given a severity rating of 1 to 5 weighted by the issueâs expected frequency of occurrence and impact to the customerâs work.

5Ê- Cosmetic or detail defects that are irritating to customers. These errors can accumulate and give the product and ãunfinishedä or ãbuggyä impression to customers.
4 - Defect causes a distraction to the customer but no real loss in the ability to get needed tasks accomplished.
3 - Defect causes customer to change work strategy to get around problems with the interface. Work goals can still be accomplished with some effort.
2 - Defect is a severe inhibitor to accomplishing work goals. In some cases, customers cannot successfully complete the task.
1 - Defect is critical and results in system or data loss. Customer is unable to achieve needed work goals.

Client Address Book vs. Web Address Book

Issue	Description	Severity	Recommendation
Finding the client address book.	Users in the evaluation had a very difficult time finding the client-based address book.Ê The looked first to the tool bar and then to the sidebar for the address book.Ê Most of the users never noticed the icons and the bottom left of the screen that accessed the address book	2	Add a button for address book to the toolbar of the mail client. Consider having a sidebar tab for address book.
Confusion between the address books	Several of the participants accessed both the client address book and the web address book during the test.Ê These users believed that these were the same address book and would contain the same information.	3	TBD
Collected addresses	Users did not know what the term ãcollected addressesä meant.Ê This was particular problematic for the less experienced users.	4	Consider alternate naming for this address book.
Moving addresses	The less experienced users had no idea how to move an address from one address book to the other.Ê These users tended to edit an address and find a way to move it.	4	Provide a drop down in address edit mode to select the address book where the card will be located.

Toolbars

Issue	Description	Severity	Recommendation
Tasks menu item	The tasks menu item is ambiguous and not well understood by users.Ê They tend to translate this menu item in a literal sense (as tasks in a to do list).	2	Consider renaming this menu item to a term that is better understood by the users.
Icons	The more advanced users in the test wanted a way to get rid of the icons on the toolbar and have only text.Ê These users tried the arrows but this resulted in the bar closing completely.	5	Provide a way to display only text in the toolbar.
AB - New Card	The term ãnew cardä was a bit confusing the people in the test.	5	Consider changing the name of this button ãNew Addressä or another term.

Sidebar

Issue	Description	Severity	Recommendation
Usage	5 of 6 users in the test used the sidebar to add buddies and to use Instant Messenger.Ê 1 users in the test used the search sidebar tab when searching.Ê 1 user in the evaluation used the sidebar to read the news.	Info
Closing the sidebar	Several users wanted to close the sidebar during the test.Ê However, none of the users were able to close it.Ê One user attempted to drag the sidebar closed but it kept popping back open.Ê The rest of the users expected to find an X button of similar control.

Misc. Issues

Issue	Description	Severity	Recommendation
Hot Keys	The advanced users in the evaluation looked for hotkeys to perform various repetitive tasks during the test.Ê However, in most cases they were not able to find the hotkeys.	3
Right Mouse Menus	Often times, there were no right mouse menus when users attempted to right click on various items.Ê This was most apparent in the bookmark list.Ê Users expected to be able to delete bookmarks directly from the list using a right mouse menu.	3	Provide right mouse menu support consistently throughout the client.
Online Help	Several users accessed online help during the test.Ê However. Once in the help system there is no way to search the contents.	3	Provide a search engine for the help contents.
Searching	Most of the users in the evaluation went to an outside search engine when asked to search the internet.Ê Users mainly went to Google and Yahoo to perform the searches.	Info
Feedback when adding a bookmark	Users expected to have some sort of feedback when adding a bookmark.	4	Add some feedback when adding a bookmark.
Application icons	Only one user discovered the application icons located at the bottom left had side of the screen.	Info

Task Scenarios

You have just heard the Netscape 6.1 is available for download. Download Netscape 6.1 and install it on your computer.
Read all of your email messages.
Go to the email message with the subject ãWhen are you free for lunchä and add the sender to your address book.
You would like to catch up on the latest news, go to the website www.cnn.com and read the news.
Add the following people to your Buddy List: JonQAnnoyer, JTGeyerman and HFU1.
You have several people that you would to send an email to and let them know what you have been doing. Send an email to Fred Jones, Mary Sommers, and Jeff Williams and tell them what you have been doing of late. (Hint: All of these people are in you address book).
You need to update an address in your address book. Change John Smithâs email address to HFU1@aol.com in your address book.
See if JTGeyerman is currently online. If he is, send him an Instant Message.
You like to visit the web site Amazon.com to shop for stuff. You recall that you added Amazon to your list of bookmarks before you installed Netscape 6.1. See if Amazon.com is still in your list of bookmarks.
You are thinking about taking a trip to the Grand Canyon. Search for a web site with information about the Grand Canyon. When you have found a web site that you like, add it to your bookmarks.
Your were given a business card the day and wanted tokeep track of the information on the card. Add the information on the business card to your address book.
You no longer visit the web site ESPN.com. Remove this site from you list of bookmarks.

Issue	Recommendation
Ambiguous terms used in some of the products.	Re-evaluate the terms used for top level menus and icons.
Confusion regarding the difference between client based and web-based products.	TBD
Difficulty locating various parts of the client (i.e. address book).	Re-evaluate the menu system and icon locations for products.
Inability to close the sidebar	Provide an easy way to close the sidebar.