If you remember, a few months back I was talking about instrumenting Tuxpaint and conducting a study with it. Well, we managed to conduct the study at Science world, Vancouver. I am going to present the result of the study broken into multiple mails.

About 111 children (66 girls, 45 boys) from age 3 to 12 participated in our study. We designed the study so that children could walk in, paint any picture they wanted using Tuxpaint, print it in color and take it with them. A small sticker containing the Tuxpaint mascot and the URL tuxpaint.org was also given to the children.

Overall the children liked Tuxpaint. When asked if they would like to use this at home, a number of them replied in the affirmative. At that point, I gave them the sticker. I told the parents that they could download the program for free from the URL in the sticker. Several parents were pleasantly surprised on hearing that.

I am going to list several rough-edges of Tuxpaint that I observed was making the children's experience less pleasant. I am reserving the most important one that I observed for a separate mail. Its going to be long. So when you are digging deep into it, don't let your mind fool you into thinking that I am bitching about Tuxpaint. I love Tuxpaint and appreciate all the effort that has gone into it. Consider this as my effort into making Tuxpaint a little bit better. If any of these problems are given a shot at, I would be more than happy to evaluate with kids as to whether the solution works or not.
  1. Busy icon: When the 'New' button is clicked, it takes about 5 to 10 seconds to show the gallery of templates (most of the time in code is I guess spent in searching for items and constructing this gallery). Most of the children and parents were confused during this time. The behaviour ranged from a.not noticing that some process was going on and trying to click, b.thinking that something has gone wrong or c. That the software had stopped working. These behaviour could be avoided by improving on two factors: a. Improving the visibility of the busy icon – making it bigger, animated and/or switch to the standard hour glass icon (as the watch icon I guess is less popular and hence harder to arrive at the meaning). b. Part of the confusion might result from the fact that when a user clicks 'New' he expects a 'New' canvas. Bringing up a dialog to choose from a gallery of installed templates is probably not the expected behaviour. I think that for children who have used other computer software, this behaviour might also contradict with what they had observed in other software. One solution might be to create a separate button 'Fill in gallery' or something to that effect to invoke the gallery of templates and have the 'New' button just create a new canvas immediately. Another option is to show the dialog immediately after clicking the 'New' button but to fill in with the snapshot icons of templates inside the dialog one by one (sort of like how the file explorers generate thumbnails when switched from list view to thumbnail view). This in addition to cutting down the waiting time might also give enough time for children's attention to peruse each thumbnail as it is generated.

  2. Colors icon: Though it is a non-clickable text label it has the visual affordance of a clickable toolbar icon. This ends up confusing some users. Could make it much more like a label and less like a button. 

  3. Magic tools like Fill and Kaleidoscope should be made more prominent – maybe promoted to primary toolbar. As far as Fill is concerned I am sure that it could be placed in the primary toolbar as it I observed it to be one of the most used functions. Most children started the session by changing the colour of the canvas/paper – most often by painting the canvas meticulously by hand (using the paint function) as they hadn't discovered the Fill function yet – a cumbersome process. And the process of filling in a part of the painting with a colour by hand is also a painstaking process involves lots overspilling/boundary-crossing and hence repeated use of eraser and paint. This could be made a lot easier by using the inbuilt Fill function. We should make this function a lot more visible than it currently is. I would recommend placing it next to the 'Eraser' icon or the 'New' icon. I would also recommend activating the fill function automatically once the user has chosen a template from the new dialog that consists of line drawings that the user is supposed to colour in.

  4. The various tools in Magic can be categorized into those that are standalone functions (i.e they can create visible artifacts on their own) and those that are modifier functions (i.e they need an existing image so that they can act upon it and produce a visible artifact – on their own they are useless). The Magic functions can be further categorized based on that some of them can be applied on the entire canvas by toggling the fullscreen button, while the other functions don't have a fullscreen-mode.
    The difference between these 4 different categories is not very clear in the interface and they are also not organized based on these categories – making these distinctions not easily discoverable & usable. e.g most children meticulously applied the effect over the entire canvas even though the function had a fullscreen mode which if toggled would get the job done in a single-click. Several children were also confused as to why the modifier magic function is not painting anything on their canvas (which is empty).

  5. Help system: I could observe only a couple of people who were reading the help. It seemed as if they were selectively blind to the region. Some parents even suggested that having a help system would be nice, when asked about how they would like to see the program improve. When pointed to the fact that the help was right there, they seemed surprised and said in a defensive tone that it is better to have it under the pointer popping out from the icons where it would be more noticeable. I think the user's obliviousness/blindness to the help system is caused by two factors. One: that it is purely text which younger children can't/won't make use of and Two: the help text is displayed at the bottom of the screen when the user clicks on a function icon. But this is also exactly the time when the tool bar of colourful icons on the right hand side of the screen also changes to reflect the selected function. Hence there is lesser possibility for the user's attention to be drawn on to the plain text at the bottom of the screen.

    Tackling the first problem – creating a help system for the children who can't yet read – is a problem that deserves further probing. I am not sure how to solve if effectively. But regarding the second problem – one that of attention – I think it can be solved by triggering a change of help text to occur when the pointer hovers on the icons. I remember children pondering a few seconds before clicking on an icon, possibly try to decipher the meaning of the icon. Changing the help text at this moment would bring more attention to it (as it would be the only changing element in the interface at that point) and it might also help the users who can read to make the choice by lowering the cost of getting the information. For the younger users, maybe a sound reflecting the function's usage could be sounded on hover. But I imagine that this could become annoying really quickly (e.g 'Eraser' being sounded for the nth time when the child hovers on it for the nth time). We could reduce that by increasing the time elapsed between the point in time when the on-hover event occurs and the point when the sound is triggered – proportionally with the number of times the user has clicked on the icon with a cut-off. This would mean that eventually you stop hearing the sounds.

  6. The proximity of the colour picker to the scroll buttons was a problem as number of children accidentally fired the colour picker when they intended to scroll. The problem was magnified as the colour picker is a modal dialog box. This leaves several users confused for moments before they get rid of the dialog.

  7. The greyed out symbol drawn at the top-right corner of the dialog boxes seemed to confuse users who would try to click it (possibly as a way to get rid of the dialog, a close button) but infact it was not clickable.

Consistency/mental-models:

  1. Resizing: should be made universal.. like Stamps also with brushes, magic, paint, eraser. There were several children who expected the resizing to be universal once they experienced it in some functions and asked me how to resize something else.

  2. Some stamps' colour can be changed but most cant. Should be some indication to denote this difference.

  3. No undo for some functions such as Magic – flip page.

  4. Reset tools to default (paint) on New page. A child had trouble because eraser was the active tool when he clicked New page and it took the 4 year old some time to figure out why the software is not responding (painting). Similarly reset selected colours on New page. One child had the same selected as the colour of the New page and spent some time wondering why none of the strokes appear on the screen. There was also interesting incidences when a child after selecting a black paper was wondering why none of the paint strokes was producing a visible effect (default black paint).

  5. Selecting a template using New page should be achievable using single-click rather than a two click operation. Younger children often wonder why after selecting a template, nothing happens. The tool bar icons and most of the functions are single-click activated. I guess they expect the template selection to be single-click too. Most of them do not click on a template icon as part of exploring the various templates – if that is a concern. Even then, we could wait for n seconds from the point that the child had selected a template by clicking on its icon to the point of opening it. This would give sufficient time to the child to change his selection if necessary. Such a mechanism would be helpful especially for the younger children. 

  6. Font selection is confusing for most children (age 3-10) with the current onscreen visual representation: various fonts being representing by an icon of 'A' rendered in the font that it represents. Several children were wondering where the other Alphabets are .. 'searching for B'.. “Where is B”. This was especially true for children under the age of 10 whom I guess are not very familiar with typing on the keyboard and hence had to resort to visually clicking the alphabets onscreen. A number of children wanted to type something on their picture – usually their name after they had finished painting. Very young children (3-6) have conceptual difficulty of understanding fonts. But even then the current visual representation could be improved their access to alphabets instead of representing a myriad of fonts that only the older children might care about. As far as I remember, none of the children, young or old, used different fonts for the text they typed out. I think that in a painting program, one might not expect/care for a vast array of different fonts that may or may not look radically different from one another. Simple straightforward entry of short phrases is all that is required by most users and the current interface is confusing and cumbersome for the requirement. 

  7. The visual design of the icons and several metaphors used in the software could be designed better to reflect the mental-models of the children. e.g almost all children couldn't understand the representation of the system busy state by the 'watch icon' brought on after clicking 'New'. The younger children were confused with the colour picker dialog (with 'back' being the only text on it). For a adult who has used colour-pickers before, there seems to be nothing there to complain. But for children it was not straightforward. They fumbled around not knowing what to do, not clear on what is clickable on the dialog and finally clicking 'back' button as it seemed to be the only thing clickable. A dialog that looked like a palette or something the children are used to in the real world (a swatch of colours in a painting box ..etc) could help reduce this confusion.

    One more observation on the color dialog: making that tiny + sign on the left that dismisses the dialog (that is barely noticeable and surprisingly no child used it) into a more standard cross sign on the right with improved visibility (why not make this dismissal mechanism consistent in all dialogs).

  8. Disable Open when there are no files to open similar to the Save button which is currently disabled when there is nothing to save.

    Safe defaults:

  1. File saving/over-writing: The “Save your picture first?” dialog with two options “Yes, save it!”, “No, don't bother saving!” occurs in two contexts: a. After clicking 'New' button and selecting a template without having saved previous work b. After clicking 'Open' button and selecting an existing picture to open. A similar dialog appears with some additional dialog text when you click 'Quit' without having saved your changes. All most all children of age 3 to 6 and some from age 7 to 10 a.cannot read and comprehend the text, b.understand the underlying file system abstractions to make a decision and chose an option. Many children in the higher age group 7 to 12 also lack the patience to read the text completely. This is especially true in the context of 'New' button. This results in them choosing an option randomly without understanding the consequences and losing all their work. This should be avoided as it contribute towards negative affect. Tuxpaint has options such as autosave and always save as new or always overwrite ..etc which should be enabled by default atleast for younger users (3-6). This would enable the children to have a more positive experience. I imagine that they would learn the default behaviour of the software (autosave, auto overwrite/ auto newsave) soon enough and adapt their behaviour accordingly in due course. We could also extend the Undo operation to encompass the file opening / over-writing behaviour so as to ensure that a child's work is never lost. But most users of the lower group are not familiar with the concept / functionality of the undo button. Hence my recommendation would be to default to options that minimizes instances of dialog boxes especially since the cost associated with defaulting to such options are mostly storage related, which should be of a lesser concern in this digital age of ever expanding storage in personal computers.

  2. Mouse control: For the majority of the younger children, one of the most frustrating part of the experience was controlling the mouse. They would often lose control and make an off-course trailing mark. They would then start erasing the mistake and would most often end up erasing more than they intended. Frustrated, they would continue to erase the entire picture and start-over. This would continue over and over again until they either got it right or chose to ignore the deviations (or in few cases, learn successfully to control the mouse). A number of parents of these younger children also mentioned 'mouse control' as a significant difficulty that their child faces when using a computer. I think it would enhance the experience of these younger kids if we could default to safer mouse controls (lower speed, threshold ..etc) if the user is in this age group (3-6).


There were only a couple of children who were very advanced users for their age group (photoshop, photobooth by age 10-12) who were not much impressed by the program or its features. None of them had any significant complaints about the program except when the sticky-click was ON. The number of users who lost interest and left abruptly was minimal. There were more children who were nudged by their parents to end their session quickly so that they can move on to the next activity (a program/show that started on a specific time ..).  


--
Mohan Raj
Master's student - Human computer Interaction
University of British Columbia, Canada