Usability Test of a Materials Management Application: Test Report
For this test I created list of Usabilty Issues with over 70 recommendations for improving the application. This excerpt from the report first lists higher level problems with the conceptual design. About 75% of the issues found in this test were fixed in the product.

1. Complexity & Too Many Choices
   Overall, the system is extremely complex. The many layers of choices and parameters make it difficult to find the characteristics you want and to remember what choices you have already made.
   Recommendations:
   1. Reduce the number of choices available to the user. This could be done by simply not allowing some choices. For example: limit searches in the Characteristics Tree to only the ten most frequently used characteristics. To support more detailed search, provide a "More choices" button or an "Advanced search" mode that presents all of the available choices.
   2. Another strategy is to collapse the trees down into fewer meaningful units. For example: the four types of Density (Relative, Absolute, etc.) in the current Characteristics Tree could be under one item. The user could then select the type of Density to search on by using radio buttons.
   3. Make sure that the choices offered correspond to the way the user is searching. For example: when you click "Material contains Component", you get a version of the Characteristics Tree, when all you really need is a list of Component materials.
   4. To further simplify, put the Language choice under the Extras menu.
2. Search Behavior
   Users should not be able to search one more than one of these fields at once: the parameters are often mutually exclusive. For example: if I type in a Material ID, then I type something in the Name field that does not correspond to the Material ID that I entered, what am I really searching on? How do these four fields interact?
   Recommendations: (See Figures, page 1)
   
   • Put the Name field at the top since it is the most frequently used.
   • Provide an F4 help button for the Material field.
   • Use radio buttons to make the exclusive choices more clear. If I am searching on a Material ID number, I should not be allowed to also search for a Name. I should only be able to search by Material ID, by Name or by Material.
3. Terminology in the Entry Screen
   The similar terms Name/Identifier/Material/Material cause confusion.
   Recommendations:
   • Change "Material" to "Material Number" or "Material code" or "Material ID"
   • Put the "Number" field - which is now under the Name group - under some kind of Options pop-up.
   • Use a more precise term than "Name". Is it Material Name, Material Name, or what?
   • Simplify the matrix of choices under the name group. Put only the most important controls at the top level.
4. Navigation
   Many actions cause you to go into an entirely separate window. You never know where "homebase" is. e.g.: After you create a Hit List, and then you want to modify a parameter for one of the items in the list, you have to go to another window.
   Recommendations:
   
   • Many of these controls can just be in pop-up dialogs instead.
   • This application should have four primary windows: Material Search, Material Hit List, Reports Search, Report Tree or List. Everything else should just be pop-up dialogs.
5. Old Parameters (leftovers)
   There is no way to easily determine the status of search parameters. Left over parameters from previous search often led to failures on subsequent searches. This can result in critical errors, e.g.: say you looked for all Material with Methanol as Content. Later, you look for all explosive Material, but the Methanol parameter is still active. You get a list of explosive Material that is much shorter than it should be and miss many items. (see User 3 videotape, 00:56:00 - 01:30:00)
   Recommendation:
   
   • Post a check dialog to ask the user: Do you want to use the already existing parameters?
   • For the New Input button, call it New Search or something more obvious not just New Input.
   • When the user has already created a Hit List and then wants to do a further search, just post a pop-up with the search controls from within the Hit List window, rather than sending him back to the original Entry Screen.

Entry Screen

1. To simplify the appearance of this screen, make the additional Specifications buttons into push buttons with labels. (See Figures, page 1) If this is not possible then:
   
   • Change the labels so that the parameter names (Header Data, Characteristics, Components, etc.) are to the left so they are easier to see.
   • Put a colon after each.
   • Move the pop-up buttons to the left so they are closer to the labels.
2. Users thought that the Hit List icon and the new Input button were all the same button.
   
3. The experienced user pointed out that everything on this screen is Criterion for Material choice. Perhaps that would be the title of the window, allowing us to use more descriptive names for the groups of controls in the window. I suggest calling this window Material: Search
   
4. Move all of the input fields to the left so they are closer to the labels. Align within groups not across groups.
   

1. In the input pop-ups (e.g. Density) it is very confusing to have to click the Refresh button. It is not clear why this button is there or what the icon means. We should refresh the dialog automatically when a user closes one of the F4 pop-ups.
   
2. There are two methods for selecting the Precision (greater than, equal, etc.). This should just be one field, with one method (see Figures, page 2).
   
3. The F4 pop-ups for many values are very confusing (e.g. Density: Value). Even our expert user did not understand.
   Recommendation: Just eliminate them and put these simple choices right in the dialogs themselves, or use a simpler approach as in the Figures, page 2.
4. In the input pop-ups (e.g. Density) the Precision and Value or Temp, etc. fields that relate to each other should be side by side rather one above the other. See Figure page 2.
   
5. It is not obvious how to input ranges of numbers or multiple conditions (300 < x < 500). When you try to type these in, the system is unforgiving, e.g.: if you put in a blank or forget a blank it will not accept the input. Recommendation: Allow users to type in great/less than conditions. Make Range an explicit choice in the F4 popup.
   
6. When you type in invalid values you get confusing messages like "not numeric". We need to give clear explanations of what types of input are allowable in a given field.
   Recommendation:
   • When the user enters and invalid value, give a more information in the message pop-up including examples of correct entries:
     The value you entered is not valid. Please enter a value following these examples:
     Single value: 10
     Range: 10 - 20
     Any of several values 10; 20; 30
     
   • Also, the text input needs to be much more forgiving, e.g. accept a blank even if it is "not allowed".
     
7. In the input pop-ups (e.g. Density) the fields are too wide. Make the fields only as wide as the maximum allowable input.
8. Several of the input pop-ups have scroll bars that are not needed. These should be removed and the dialogs should be made shorter, top to bottom.
   
9. When you choose multiple inputs - e.g. red, brown, blue under Colors - then you close the input pop-up, you see only one choice with its cryptic code. You have to close the dialog and then go right back in, or refresh the dialog to see that what you wanted has actually happened.
   Recommendation: refresh the dialog automatically when an F4 pop-up is closed.
10. Currently, when you select a Precision, you get a cryptic code back. You should just see the >, ca., = or whatever.
    
11. It is not clear what the symbols under precision mean (=:, =, etc.)
    
12. Whenever there is an input that has units (e.g. g/cm3), if the user can not change the units put them to the right of the input field as notation text, not in the fields themselves. (See Figures, page 2)