CS6021 - Foundations of Interactive Media – Cooperative Evaluation Project InFocus - ‘Promixa 9260’ LCD Projector Cooperative Evaluation Project Evaluation Team: Daniel Scully - 0419842 Paul Kennedy - 0420107 Ronan Tighe - 0044709 Masters Interactive Media University of Limerick 2004 1
Table of Contents LCD Projector................................................................................................................1 Report Overview............................................................................................................3 Why is evaluation so important? ...........................................................................4 The Proxima 9260..........................................................................................................
Bibliography ................................................................................................................56 Appendix......................................................................................................................57 A note on the set-up of the Proxima projector in the Interactive Media lab........57 Report Overview This report will evaluate the Proxima 9260 projector that is situated in the Interactive Media lab.
and what problems we encounter in the pilot study. The evaluations we then be discussed followed by an analysis and recommendations of how we feel the projector could be improved. First it is important to understand why the evaluation is so imperative to the success of a product. Why is evaluation so important? The evaluation phase of the design process is one of the most important parts but is still overlooked by many developers.
The Proxima 9260 Proxima (now called AskProxima after a merger with Ask) is a brand of the InFocus Corporation. InFocus was founded in 1982 and has been a consent innovator in the field of digital projection. The Proxima 9260 was released in early 2001. The Proxima 9260 is a stationary projector that is attached to the ceiling of our lab about four metres away from the projector screen. The long and short zoom lenses mean that it can accommodate small rooms like our lab as well.
InFocus conducted extensive research that led them to realise that “easy set-up, management and usability are of primary importance to buyers and users of projection technology”. (InFocus 2002) They believe that their projectors are consumer centric and are trying to “drive digital projection adoption mainstream”. It is clear from InFocus’s press releases that they are trying to enter the home entertainment market.
Preparing The Evaluation The goal of this study is to conduct a cooperative evaluation of the InFocus Proxima 1690 Projector in terms of its design, usability and menu functions. The Proxima is designed to be compatible with both computer and video inputs either as a portable of fixed projector that can be used in the workplace or at home. This is a picture of the projector shown here.
• Using the integrated menu functions (shown below) The target test group was selected in order to represent the typical users that would operate the projector in a classroom or presentation environment. The tasks selected are intended to test the projectors ability to meet the needs of these typical users in a classroom or presentation environment. We wanted to evaluate the features a typical user would most commonly use day to day, rather than attempting to evaluate every feature and option.
User Two Date 14/12/04 Sex Male Age 15 - 18 Occupation Undergraduate Previous experience with the 9620 No Previous experience with any other Yes projector User Three Date 14/12/04 Sex Female Age 18 - 25 Occupation Undergraduate Previous experience with the 9620 No Previous experience with any other No projector User Four Date 14/12/04 Sex Male Age 18 - 25 Occupation Undergraduate Previous experience with the 9620 No Previous experience with any other Yes pro
User Five Date 14/12/04 Sex Male Age 18 - 25 Occupation Undergraduate Previous experience with the 9620 No Previous experience with any other No projector User Six Date 14/12/04 Sex Male Age 18 - 25 Occupation Undergraduate Previous experience with the 9620 No Previous experience with any other No projector User Seven Date 15/12/04 Sex Male Age 18 - 25 Occupation Postgraduate Previous experience with the 9620 No Previous experience with any other Yes project
User Eight Date 15/12/04 Sex Male Age 25 - 35 Occupation Postgraduate Previous experience with the 9620 No Previous experience with any other No projector User Nine Date 17/12/04 Sex Male Age 18 - 25 Occupation Other Previous experience with the 9620 No Previous experience with any other No projector 11
Focus of Evaluation The first area of study is the remote control for the projector. This artefact is very important feature of the projector operation especially as the model that is the focus of this evaluation is fixed to the ceiling. Most of the same functions, which appear on the remote control, are also available on the actual projector casing - except for the mouse control and laser pointer features. However, you would need to be very tall indeed in order to reach them.
The final section of the evaluation is concerned with identifying the individual users for the evaluation. In this respect, we created a short questionnaire, which asked for a little background about the users and their familiarity with projectors. In addition, the users were also debriefed after the tasks were completed the test was over. The evaluation team chatted with the user about the test in general, asking questions like which events were the most noteworthy for the user.
and questionnaire, the space itself, all of the equipment, and the feelings expressed during the evaluation. It is important for the user to feel comfortable while doing the evaluation. Each individual task is planned out from proposal to completion, so that the users can carry out the tasks in a smooth flowing sequence. This makes the entire experience less stressful for the user.
was used to take close up images of the projector, remote control and the integrated menu. The tasks were laid out in such a way as to relax the user initially with familiar tasks – such as ‘turn on the projector and use the laser pointer’. These simple tasks made the pilot user more comfortable and at ease with the evaluation before reaching the more complicated, less familiar tasks such as ‘change the language on the interface’.
The Interactive Media lab environment was less than ideal for the purpose of a “Think Aloud” evaluation. The reason being that the lab is constantly in use by up to twenty people at once. The rest of our colleagues did their best to accommodate us during the testing, but it was a distraction to our users, as they felt scrutinised not only by the evaluation team and its recording equipment but also by the sheer number of people in the room.
In addition, we provided an introduction on the next page preceding the task list, which we asked the users to please read first. It appeared like this: - “PLEASE READ FIRST: Hello, and thank you for coming along to take part in this cooperative evaluation. The tasks, which you will be asked to perform, are listed below. Please take as long as you need to complete each task.
The users were not given any time to familiarise themselves with the projector, remote control, menu or the operators manual prior to beginning the evaluation. The evaluation team felt that this might detract from the honesty of the users initial experience during the evaluation itself. During the evaluation the users were encouraged to utilise either the Help Index or PDF Operators Manual, which were available to them if they wished to solve a problem.
choice and the mental processes they go through when attempting to solve a problem. Task Selection This evaluation is interested in finding out how easy it is for the user to mentally map what they wish to achieve to the functionality of the Proxima remote control and menu interface. This study hopes to assess how intuitive the system is for the user, and how easy it is for the user to read from the interface the solution for a particular task.
prompted the user to explore the functionality of both the remote control and the integrated menu. The users were each given 11 individual goals or tasks to achieve using whatever method they felt appropriate.
4. It is necessary to set up the projector for use by a French delegation later tonight. Please attempt to change the interface language to French. 5. The French delegation has just called and cancelled their session for tonight; please reset the interface language back to English. 6. Please switch between the ‘Video’ and ‘Computer Input’ channels. 7. Because of the low light in the room, it is necessary that you adjust the brightness control of the projector. 8.
The user was given the remote control and told to begin. One of the evaluation team used a video camera to capture to actions of the recruit, focusing on either the integrated menu or the way the user utilised the remote control. It was important that the voice of the recruit was captured on video as the team utilised the ‘thinking aloud’ method.
also allowed users to gauge how they felt about the evaluation and the projector. There were a couple of recruits who also used the space at the end of this sheet to volunteer comments such as, what they did not like about the projector. The questionnaire is shown below: Thank you for completing the task list; the evaluation team would appreciate it if you would take a further moment to complete a short questionnaire on the next page.
_______________________________________________________ Overall how hard did you find the tasks to complete? Very Hard: Hard: Easy: Very Easy: ___ ___ ___ ___ Did you need to use the manual provided: Yes: No: ___ ___ If yes, did you find it useful? Yes: No: ___ ___ Would you feel confident in using this projector again? Yes: No: ___ ___ Would you consider buying this projector? Yes: No: ___ ___ Once this questionnaire had been completed the users were thanked for participating and asked if there w
The Evaluations User One Task One: Please turn on the projector. This user took time to locate IR sensor located at the rear of the projector, she also seemed unsure as to whether or not the projector is active during the projector bulbs warm up period. Task Two: Once the projector has been started up, utilise the integrated laser pointer on the remote control, please use it to point at something on the screen.
projected on the screen. Please utilise the Visual Zoom feature to zoom in and out with the picture. After initial failed attempts to operate the zoom button on the remote control, the user switches the projector from video to computer setting. Once in computer mode the zoom feature of the remote control is easily achieved. Task Seven: Please switch between the ‘Computer Input’ and ‘Video’ channels. This user had familiarised herself to this function through selecting this function earlier in the session.
Summary: This user expressed that she felt that the tasks set where hard to complete. This could be due to her lack of previous experience with such devices. Following our session despite the difficulties encountered she expressed feeling more confident about using a similar projector in the future. Remote and GUI interface despite it’s flaws is picked up within a relatively short period of time, this is due more to user interaction than design feature. User Two Task One: Please turn on the projector.
Task Five: The French delegation has just cancelled their session for tonight; please reset the language to English. User easily completes the task, this time settling on the button left of the thumb wheel to affect action. Task Six: It is necessary to draw particular attention to an item projected on the screen. Please utilise the Visual Zoom feature to zoom in and out with the picture. User easily operates remotes zoom control. Task Seven: Please switch between the ‘Computer Input’ and ‘Video’ channels.
Summary This user was one of the youngest surveyed in our group aged seventeen. He expressed that he felt the tasks laid out for him where very easy, he had also that he had previous experience with similar projectors. Overall this user quickly found his way around the remote control and on screen GUI. User Three Task One: Please turn on the projector. This User was confused as to where to point the remote control, i.e. at the IR sensor located to the rear of the projector.
The clearly marked zoom function on the remote makes this task easy for the user Task Seven: Please switch between the ‘Computer Input’ and ‘Video’ channels. This user opts to switch using the remote control button Task Eight: Because of the low light in the room, it necessary that you adjust the brightest control of the projector. GUI navigation yields no problems, the small up and down arrows pose problems once again, user takes time to settle on correct control adjuster.
User Four Task One: Please turn on the projector. Task completed without any problems. Task Two: Once the projector has been started up, utilise the integrated laser pointer on the remote control, please use it to point at something on the screen. Task completed without any problems. Task Three: The volume on the projector needs to be adjusted, please access volume control and raise volume. Task completed without any problems.
Task Nine: Please select the ‘widescreen’ in order to view a DVD in it’s preferred format. Similar concerns as the last task are expressed by the user Task Ten: Please ‘Freeze’ a frame of film on the projector for a moment and then unfreeze it. User easily locates and uses ‘Freeze’ button on remote Task Eleven: That completes this segment of the evaluation; please reset the projector by turning it off now and then switching it back on. Task completed without any problems.
Task completed without any problems. Task Five: The French delegation has just cancelled their session for tonight; please reset the language to English. Task completed without any problems. Task Six: It is necessary to draw particular attention to an item projected on the screen. Please utilise the Visual Zoom feature to zoom in and out with the picture. This user takes time to locate the zoom button on the remote. Task Seven: Please switch between the ‘Computer Input’ and ‘Video’ channels.
User Six Task One: Please turn on the projector. User points remote at side to projector, then locates the IR sensor at the back of the projector. Task Two: Once the projector has been started up, utilise the integrated laser pointer on the remote control, please use it to point at something on the screen. Task completed without any problems. Task Three: The volume on the projector needs to be adjusted, please access volume control and raise volume. Volume control poses no problems for the user.
Task Nine: Please select the ‘widescreen’ in order to view a DVD in it’s preferred format. Task completed without any problems. Task Ten: Please ‘Freeze’ a frame of film on the projector for a moment and then unfreeze it. Task completed without any problems. Task Eleven: That completes this segment of the evaluation; please reset the projector by turning it off now and then switching it back on. Task completed without any problems.
Task Five: The French delegation has just cancelled their session for tonight; please reset the language to English. Task completed without any problems. Task Six: It is necessary to draw particular attention to an item projected on the screen. Please utilise the Visual Zoom feature to zoom in and out with the picture. Task completed without any problems. Task Seven: Please switch between the ‘Computer Input’ and ‘Video’ channels. Task completed without any problems.
the remote IR sensor is located at the rear of the projector. Task Two: Once the projector has been started up, utilise the integrated laser pointer on the remote control, please use it to point at something on the screen. Task completed without any problems. Task Three: The volume on the projector needs to be adjusted, please access volume control and raise volume. Task completed without any problems. Task Four: It is necessary to set up the projector for use by a French delegation later tonight.
Task Eight: Because of the low light in the room, it necessary that you adjust the brightest control of the projector. The thumb stick navigation once again proves problematic for the user. Task Nine: Please select the ‘widescreen’ in order to view a DVD in it’s preferred format. User checks remote and then returns to the GUI interface to complete task. Task Ten: Please ‘Freeze’ a frame of film on the projector for a moment and then unfreeze it. Task completed without any problems.
Task Four: It is necessary to set up the projector for use by a French delegation later tonight. Please attempt to change the interface language to French. Checks remote for language option, then settles on the menu button, GUI navigation is at first confused, user sees goal to far right of GUI but cannot reach it, expresses perceived delay in remote function, this is due to user location Task Five: The French delegation has just cancelled their session for tonight; please reset the language to English.
Summary User found tasks easy.
Analysis of Evaluation Once all the evaluations were carried out the group sat down and studied the recording of the tapes. There are a number of principles that can be used to evaluate a product; we have chosen to evaluate the projector by studying its affordances and constraints. By do the analysis in this way we can see the overall positive and negative aspects of the remote control and on-screen menus instead of the aspects that were involved in each task we asked the users to carry out.
• Cognitive Affordance • Physical Affordance • Sensory Affordance • Functional Affordance It is worthwhile to look at the remote control and on-screen menu with relation to each of these affordance. Cognitive Affordance Cognitive affordance is a feature of design that helps the user know how to do something. Good cognitive affordance on the remote control is the clearly labelled button for zoom, focus and volume.
interfaces warning against using just graphics he also warns against just using one word to describe what in the menu (Norman 2003). This is also something that appears in the projectors on-screen menu as can be seen in picture Menu Two. One of the most difficult tasks for the users in our evaluation was changing the brightness of the projector this was because of the poor cognitive affordance of the menu.
of the main areas in which the projector failed to accommodate users. Even when the users stood behind the projectors they had problems getting a response due to the narrow angle at which the remote sensors picked up a signal from the remote. Although the users of our evaluation knew which button to press to alter the volume it took many of them a number of attempts at pressing the volume button to succeed. The pointer button that moves the on-screen cursor is also difficult to control.
Good sensory affordance on the remote controller is the yellow colour of the laser button. This helps the user locate the button quickly and easily. The on-screen menu has poor sensory affordance as it took users to much time to locate where the cursor was on the screen the first time they accessed the menu. Laser Button Functional Affordance Functional affordance is the design feature that helps the user accomplish the tasks attended.
Physical Constraints Physical constraints limit possible operations. The fact the projector is attached to the ceiling limits the user to using the remote control and not the control panel on the projector. The size of the human hand limits the size of the remote control. The design of the buttons apply some physical constraints the zoom, focus and volume buttons can only be press in a vertical direction.
button but used their logical when asked to use it by pressing the button that said ‘laser’ on it.
Further Analysis and Recommendations The tasks set for our group of users where specifically aimed at the projectors remote control and on screen graphic user interface, this was primarily due to the projector being mounted out of the reach of most users. They where aimed towards achieving what would be simple every day interactions involving the projectors remote control and graphic user interface, ranging from volume control to language and display settings.
and control. It compromised of numerous left to right drop down boxes, once again the GUI design reflects the remotes design in that it is not clearly labelled and many of the icons used are none descript giving no indication as to what there function actually is until they have been selected. Once selected the lists value appears, i.e. the word screen appears on the GUI’s ‘screen’ menu only after it has been navigated to.
particular operational style was adopted. However the apparent and obvious lack of a user-friendly interface has led to changes in our prototype; aimed towards effecting major improvement towards easier navigation and control of the onscreen GUI. Changes will be implemented on the remote itself as well as the projected GUI. Possible Solutions The first change we would make to the overall design of the projector would be to address the shortcomings of the IR sensor located on the projector.
Our amendment would locate a 360 degree IR sensor, similar to those used in motion detection systems, these would not be recessed but bulb shaped and located on the front, rear and top of the projector or the bit that faces the floor once mounted on the ceiling. Thus providing the user full range of movement with the remote control.
The second problem we would address would be the layout of the remote control itself. When one looks at control system on the projector itself it appears quite intuitive and well laid out. A similar approach to the remote would have greatly cut down user confusion. Below is a diagram of the interface on the actual projector and the remote control itself.
The main issue our users had with the remote interface was the lack of clear indication as to the thumb sticks function. It is similar in many ways to TV but the dials main attribute is that it was designed to allow users to operate it as a mouse wheel. This means that the shape and location of the dial are adequate however the fact it is poorly labelled led to confusion among our users.
recharging cradle could also be incorporated to ensure optimum performance under such a power consuming condition. The last area in need of amendment was the GUI itself, complaints about ranged from it being to small to too bulky and difficult to navigate. Given the available screen size we would firstly make the interface much bigger, also giving each menu a clear label as opposed to nondescript icons. So the original GUI looked like the diagram below.
Conclusion We undertook the cooperative evaluation of the Proxima projector because of observations we had made of its operation throughout the semester. It was plain to see that they were problems between users and the projection system. We hoped that by doing the evaluation we would identify the design problems which users experience while using the projector. We feel that we have evaluated the projector in a fair and professional manner.
References Hartson, R. (2003). "Cognitive, physical, sensory and functional affordances in interaction design." Behaviour and information technology September 2003. InFocus (2002). "Press Release." McGrenere, J. a. W., H. (2000). "Affordances: Clarifying and Evolving a Concept." Norman, D., Ed. (1988). The Design of Everyday Things, MIT Press. Norman, D. (2000). "Usability On The Web Isn't A Luxury." Information Week On - Line. Norman, D. (2003). Affordances and Design. Bibliography Monk, A., Wright, P.
Appendix A note on the set-up of the Proxima projector in the Interactive Media lab The actual set-up and wiring of the projector in the lab was not a feature of this evaluation. If we had asked the users to perform setup tasks such as check connections between projector, power, audio, video and computer – we would never have finished the evaluation. To be quite blunt, the set-up is a complete mess.
A little effort has been made to label each connection, but it still appears as a crazy mess to anyone who may be seeking to fix a problem with one of the cables. If these cables were neatly bundled together, securely tacked to the floor or underneath the desk – they would be less likely to be disconnected inadvertently by a passer by or someone just sitting at the desk. It appears like an accident waiting to happen. Another constantly reoccurring problem is that of the power supply.
technical damage to the systems connected. Furthermore, of great importance in the day-to-day use of projectors is that they have a controlled power off. This allows an integral fan to switch on and cool the projector lamp. This is vital to preserving the lifetime off the projector bulbs, which are very expensive to replace. A carpenter and a technician working together for a couple of hours could solve these problems.
