In this mail, I present what I consider to be the most important observation to come out of the study.

I am going to explain the problem and also discuss some potential design solutions. Some mock-ups of the designs can be found in the document.

I am planning on implementing one or more of these solutions into Tuxpaint within a month and then testing it out with children to see if they work as intended by Feb. I would appreciate any help in this regard with implementing the solutions (I have basic knowledge in C and none in SDL).

Based on my prior observation of children interacting with tuxpaint in particular and computers in general, I knew that one of the major hurdles that prevented children from operating the software on their own without requiring adult help was dialog boxes. I had observed that children would be stumped when a dialog box appeared not knowing how to deal with it. After staring at it for some time they would turn towards me with a confused look. Some children would assume that something had gone wrong and would be concerned – even though the dialog box might be a direct result of one of their actions and might be waiting for their input to proceed. But I had assumed that most of this difficulty with dialog boxes would be attributable to the fact that none of the children I had observed prior to the study had English as their first language. Hence what happened at the study took me by surprise.

Going into the study, I had not anticipated dialog boxes as a potential point of observation. Over the course of the observation, I started noticing a pattern – the dumbfound quizzical faces of younger children (3-6) when they were faced with dialog boxes. As far as I remember none of them were able to make a choice by themselves – especially with dialog boxes that had multiple choices to choose from. I imagine that a dialog box with a single OK button colored green is easier to dismiss, especially after the first time you do it and observe that you are still able to continue with your work. The older children (7-12) exhibited less confusion in dealing with dialog boxes. This was especially true for children of age 10-12. I suspect that most of this might have to do with their prior computer experience. But even among the children who seemed to have no problem with dialog boxes, I am not sure how many of them understood and made the choices consciously. For there was instances when the child would approach me after losing his/her work because he/she had made a choice without knowing what it is that they were choosing. A solution might be to have all operations within a application be undo-able – including file opening/saving/over-writing ..etc. But I guess this is practically not possible.

Based on the demonstrated behavior of the children during the observational study and post-study data analysis and from literature in developmental psychology we identified three groups that shared common behavioral patterns. These groups could potentially benefit from different targeted designs.

Group1: Ages 3 to 6, pre-readers. 
Most of the children in this group have not yet learned to read. In our observations, we found that they also had the least control over mouse and hence co-operated the most with their parents. We found that many of them could not even make the causal link between the dialogs and their actions which triggered them. The result was that when a dialog appeared children got confused and at times even startled. Almost all of them required help from adults to dismiss a dialog. Most of them were probably facing dialogs for the first time and hence seemed to be wondering why something had appeared in the center of the screen blocking their view and preventing them from using the program. 

Group2: Ages 7 to 9, beginning readers, explorers. 
This group turned out to be the most curious and performed the most exploration. Their curiosity might explain the fact that this group explored the most number and type of functions and hence encountered the largest share of dialog boxes. This group operated largely without the help of adults. Though children in this age group can read at a preliminary level, many of them we observed seemed to have conceptual difficulty in understanding abstractions represented by the information in dialogs such as file systems. They typically had almost no problem with making the causal link unlike Group1. Though most of them seemed to be not entirely comfortable with having to deal with dialogs, many seemed to have learned that clicking on a choice, usually arbitrarily, dismisses the dialog. But this behavior of randomly making a choice resulted in loss of work in some cases, for example where the children chose to overwrite or not to save their work, without understanding the implications of their choice.
This is the group that could be most benefited from the redesigns as described below.

Group3: Ages 10 to 12, unruffled know-it-alls.
This group seemed to have the least problem with dialogs. They are also most likely to have the most exposure to technology and hence, I think, might have learnt to handle dialogs and get comfortable with them. But researchers has observed that children in general tend to treat those designs that they perceive as being designed for younger kids unfavorably. Hence it remains to be seen how Group3 will react to the redesigns targeted at to the other groups.

I am thinking about having a startup screen which displays existing users and where the child enter his/her name, gender and age to create a new user. This information could then be used to customize the interface according to their age.

For example, the children from this Group1, I think are better off without dialogs. If we find that the child using the program is from this age group, I strongly recommend that the program should rely on safe defaults instead of popping a dialog. I found that most dialogs could be prevented by passing in appropriate command line options. So I think we should make use of those options for this group at least.
We could also turn on/off functions depending upon the age input.

Problems with Dialog boxes
The problems that the children had with dialog boxes can be summarized as follows:
1. Causality (How did it appear all of a sudden?): Not being able to make the link between their action and the resultant dialog.
2. Purpose (Why is it here?): Not being able to understand that the software is trying to have a conversation with them i.e. providing them some information and asking them to make a choice.
3. Hindrance (Why is it stuck?): The modal nature of a dialog box confuses many children as to why they cannot continue interacting with the software. 
4. Communication (What is it saying?): Not being able to understand the content of what is being communicated. This essentially boils down to not being able to read the text on the dialog and/or not being able to understand the represented abstractions.
5. Consequence (What should I do now?): Not understanding the full implications that their choices would have and having to deal with the negative consequences of the choices made in haste.
6. Patience (Whatever…): Not wanting to spend time reading and understanding what the software is trying to communicate to them.

Children in Group1 were affected for the most part by problems 1, 2, 3, 4; Group2 with problem 5 and to some extent problems 4, 6; and Group 3 with problem 6. For Group2 and especially Group3 children, plain text seems to fail as an effective information communication medium in delivering information in both help and dialogs even though they possess the capabilities to read text. 

Potential design solutions (please refer the mock-ups)
1. Improving Causality: By redesigning the dialog box to resemble a callout and animating its appearance to originate from the element that triggered it could improve the affordance of causality and purpose. (Figure 1 in mock-up)
2. Minimizing Hindrance: Making dialogs non-modal and gradually disappearing after a certain number of seconds will make them less of a hindrance and make especially Group1 children more comfortable with the notion of dialogs until they are ready to deal with them. (Fig 2)
3. Improving Communication: Increasing the information bandwidth by replacing or augmenting dialog text with symbols and diagrams wherever possible to make the information more accessible to readers who cannot or would not read. The appearance of words can also be altered to convey more information. We have to remember however that Group1 children are not yet completely familiar with the concept of symbols, signs and their representative meanings. (Fig 4.1, 4.2)
4. Illustrating Consequences: The Consequences of the various choices can be made more evident by dynamically revealing pictograms, symbols representing the consequences of a choice when the child hovers over the corresponding choice. (Fig 4.1, 4.2)
5. Safer arbitrary choices: We can reduce negative consequences of choices made in haste by:  (Fig 6)
a. Having a persistent choice in all dialogs, in a consistent position with a consistent look and feel, that dismisses the dialog without making any change. The choice could be labeled ‘Take me back’ or ‘Do Nothing’.
b. Activating various choices in a dialog one by one in the ascending order of disruption. For example, in a dialog box asking the child whether to overwrite or save as a new file, the order of activation of buttons would be:
i. ‘Take me back’
ii. ‘Save as a new file’
iii. ‘Save over previous file’
c. Revealing the dialog text gradually in parts, with the order of appearance of a word depending upon the importance of the word in helping the child to make a decision. 
This should lower the decision time and minimize resulting negative consequences in situations where the child makes a random choice in haste. Over time, we believe that this model offers a comfortable learning environment to the child, one which they can grow with as they become more capable and/or comfortable at processing all the presented information. Care should be taken to not annoy children who know what they are doing (are making conscious choices) by providing them with mechanisms to bypass this system.
I think that this solution will be particularly effective given the impatience I noticed among kids. Most of them simply don't want to read any text - help or dialog. Most of them just skim the text before making a decision in a hurry. Researchers have also found that people read less and skim more, the more experience they gain with technology. I think this solution hence will benefit the children the most. So I guess this will work well for Group2 and esp Group3 children.

One could imagine that for children of Groups 1 and 2 who cannot yet read by themselves, reading out the text and the choices on a dialog box could serve as a solution. But we have decided not to pursue audio as a design solution owing to its several limitations:
1. Completeness: Audio only solves the problem of Communication and to a certain extent that of Purpose. It leaves open other problems. It affords poor skimmability of information and hence could worsen the problem of Patience especially for experienced children. Further, conveying multiple choices unambiguously using audio modality becomes challenging and hence could worsen the problem of Consequence.
2. Scalability: Audio messages take relatively more time and effort to produce even in a single language and thus would take substantially more effort to translate into other languages. Hence it would fail as a scalable universal design solution.

Alternate solutions? Thoughts ?

I am excited about the possibility of making Tuxpaint even better and improving the experience for lots of children.

Mohan Raj
Master's student - Human computer Interaction
University of British Columbia, Canada