Computer Interfaces:

Inquiry and Implications

 

 

 

 

 

 

 

 

 

 

 

 

By

 

Warren Buckleitner and Noel Estabrook

 

 

 

CEP 916 Dr. McCloed  Dr. Byers
Michigan State University

 

Draft 2

May 1994

 

 

 

INTRODUCTION

 

When Johann Gutenberg (1397?-1468) and his colleagues launched modern day mass printing in Germany, they facilitated the availability of printed materials for the masses, along with a new kind of problem.  The common people could not read the materials (Leuherman 1981). Today, printing technology has evolved so that the dominance of print as a medium of communication and instruction has elevated the issue of literacy into one of the major concerns of modern day education. Mass produced textbooks, for example, have become an integral part of today's mass education. 

 

Some say the impact of microcomputer technology on civilization is second only to Gutenberg's development of moveable type.  While this may be the case, the impact of computers on education has been slow and is still not fully realized. Unprecedented advances in microelectronics have reduced the size and cost, while increasing the power of, computing technology, and have made microcomputers affordable in classroom and home environments.  Like the printing press, the new technology  seems to be ready for the masses before the masses are ready for the technology.

 

Confusing menus, command lines and directions that may mean nothing to the novice computer user seem to be common.  It often seems as if those programming software imagine a group of people much like themselves making use of their technology.  In the past ten years, the number of computer interfaces has multiplied. For instance, in 1981, just 18% of schools reported as having computers. By 1991, 98% had computers.  Yesterday's ringing cash register today employs touch-pad input such as those seen in every  McDonald's restaurant. Industrial presses are computer controlled, and in the cockpits of modern jets, digital terminals have replaced analog dials. As more and more humans learn and work via computers, interface issues become more critical. Software designers who understand the mental models of those who will use their products are more likely to be successful. 

 

In education, this problem can be even worse.  It has been reported that 30% of all teachers are estimated to have had less than 8 hours of training in the use of computers and software (Dickinson, 1992).  Furthermore, the time needed for proper training in the use of technology is quite often simply not there. 

 

It seems clear then, that there is a need for software and technology that is "transparent".  That is, programs which are easily understood by those not familiar with "bytes", "RAM" and "SCSI drives".  Software designs must allow for individual differences and experimentation. We feel that the easier and more accessible technology is, the more it will be utilized by those in education, including teachers, administrators, counselors and students.

 

It is the aim of this paper to explore this issue from two perspectives: the end user and the existing body of software.  In Part 1, we will examine the computer technology from the user's perspective.  To do this, we have exposed two computer novices, an adult and a child to the same software program, a Macintosh text-graphics package called Easy Color Paint  (MECC 1992) and carefully recorded their reactions to different components of the computer interface[1].  While both subjects had limited computer experience before, this was the first time with this particular package. From this, we were able to make some generalizations about strengths and weaknesses of this computer interfaces and the kinds of design features that facilitate smooth use of computer technology. 

 

In Part 2, we attempt to learn more about whether the needs of the user are met by the kinds of currently available software to  gauge the "state of the industry."  These two perspectives: what is desired and what currently exists make up the theme of this paper and hopefully will provide some insight to  improve the area of computer interfaces of the future. 

 

 

How People Interact With Computers: A Look at the Literature

We first attempt to examine some of the processes involved in learning the use of a novel software package. We argue that the growing importance of microcomputers as knowledge systems as well as creative tools necessitates research on cognitive processes related to the learning of new software. While much of the instruction in the use of software is provided in formal settings such as schools and colleges, such traditional sources of training may not be adequate to meet the challenges of rapid innovations and changes in software (Shrager & Klahr, 1983).

 

As a result of this, it seems that the instruction provided in formal settings needs to be built upon as technology is upgraded and changed.  Consequently, users of computer technology find that they have frequent occasions where they engage in self-instruction, often with the help of manuals, on-line help, user groups, friends and vendor's toll-free numbers.

 

We will first examine how two individuals with low to moderate computer experience engage a new piece of software and construct meaning out of their interaction with it in an environment supported by on-line help, manuals, and an expert. We believe this type of research is an important contribution to the study of user compatibility.

           

The majority of the literature related to computers and cognition is related to instruction in subject areas or concepts related to subject areas (Papert, 1980; Rubin, 1983; Suppes, 1966; Burton & Brown, 1982). There is also a sizable body of literature which critically examines the epistemological assumptions of various forms of computer mediated instruction (Sardello, 1983; Winograd & Flores, 1986).

 

In developing this project we were primarily interested in literature associated with the problems in learning to use new computer software. In concentrating on this, we sought to draw a distinction between the meta-cognitive aspects of how people learn, and the actual learning of the software's content. We were encouraged to find that some research has been conducted along the lines of our interest--that is, how people learn to use certain computer applications. However, this research was very limited. The following is an outline of some of this work.

 

Wright (1983) notes that people working with a wide variety of technologies have been generally found to assume that "there are quicker ways of finding out [information] than by reading the documentation." She finds that people prefer to "play with the instrument itself", or to ask "someone who knows" if the task proves to be entirely unfamiliar (p. 12). She also reports that studies of the use of manuals by novices indicate that the majority prefer to leaf through a document as opposed to using contents lists or index. In addition she finds that "many of the people who turn to a manual will be looking for answers to questions about what to do, rather than seeking an understanding of why they should do it" (p. 13).

 

Wright argues that readers of manuals actively organize "past experience" and bring this experience to bear by generating "expectancies"(p. 13). She cautions that readers sometimes draw "inferences from the text which are not intended" (p. 13) by the authors of the manuals and urges more research to explore the problems which arise "when the reader, having carried out what was believed to be procedure, finds that something unexpected happens" (p. 15).

 

Moll (1987) considers the contribution of psychologically oriented research to software design. He indicates a growing trend of doing research which focuses on the construction of "mental models".  Moll's paper cites over twenty-five researchers who have published work on various aspects of mental models. The term "mental models" in computer related work has sometimes been explained as "operative images" (Moll, p. 404). Mental models have also been viewed as "psychological structures  which regulate the working activity" through "predictions and expectations" (p. 404-5). They are viewed as "incomplete, inconsistent, unstable in time, oversimplified, and often rife with superstition" (p. 405). In essence, the mental model can be thought of as some  already developed "schema" which a learner brings to a learning situation.

 

"In general, they need not be technically accurate (and usually are not) but they must be functional" (p. 405). Computer related skills may be represented as  "problem space" models (Douglas & Moran, 1983). From this point of view, the "acting person", or learner, "must have a certain knowledge from which he may derive predictions regarding his analysis, valuation, and problem solving. A computer user needs knowledge of the possible states of the system, an understanding of the goals which the system can help him attain, and the respective necessary operations" (p. 405).  As they work towards their goals, users

 

derive hypothesis based on their mental models with respect to the solution of their  problems..or unsuccessfully attempt operations... without having analyzed the conditions of their current situation (p. 405).

 

Waern (1987) argues that an important attribute of computer systems  is their capacity to facilitate "learners' exploratory creation of [their] own models" (p. 273). She suggests five mental models which may come into play in using a computer system. These are (1) models used for planning; (2) models used for observation and interpretation;  (3) models use for problem solving; (4) models used for communication and (5) models for creativity. The following is a brief outline of these models.

 

 

Mental Models Used for Planning

 

If participants are asked to think aloud, their plans as well as actions can be noted. Waern suggests that we note the type of planning performed. Statements which indicate planning may include "The easiest way must be to...So let me start..." (Waern, p. 287). These statements refer to "procedures to perform the task."

 

 

Mental Models Used for Observation and Interpretation

 

Waern argues that "there are two important ingredients in learning: observing the situation and the outcomes from actions, and interpreting the effects of actions" (p. 281).  Mental models will determine what is seen and not seen. A novice user may overlook certain messages if he/she can not fit them into a mental model. And "when something unexpected happens the model will determine how the user interprets the results" (p. 282). In such circumstances, the user may give up or work towards a new model which  "can embrace" the unexpected event. So "without a mental model, important events may not be noticed, and if noticed, their importance may not be  appreciated" (p. 282).  Waern gives the following example of an of a research subject whose task is to reorganize text:

 

"Interesting that they move. Here, they are very close, the columns, and then you find that they move to the right." (p. 288)

 

This is an example of a learner making an observation without attempting interpretation. Waern's study finds that, generally, learners do not have models for observation. In many cases her subjects were content to note, "Hmmm... is that why they move? ...Well, I'll figure that out later" (p. 288).  In many instances "subjects are so eager to continue their work that they do not stop for reflecting" (p. 288). As a result, interpretations are not always performed. This observation is confirmed by Mantel, Haskell (1983).

 

 

Mental Models Used for Problem Solving

 

Models for problem solving are associated with a task performance. The subject is engaged in searching for a method to perform a task. There is therefore a goal (the accomplished task) and an initial situation (the current situation as perceived by the user). According to Shrager and Klahr (1983), the learner actively constructs and tests hypotheses about how the program works. A learner may learn by designing "play" activities (Shrager and Klahr, p. 227). This means setting up a "complex action goal" and then trying to attain the goal "by use of existing knowledge." 

 

Waern's subjects had difficulties developing an adequate model. Development of a model was often hindered by forgetfulness of previous observations,  for example

 

"What did I do last time, when everything jumped so easily? I took away a word, and everything moved one step to the left. I'll move the cursor in any case. What did I do? I moved the row so easily." (p. 289).

 

Mental Models Used for Communication

 

Models may also be used in communicating concepts. After undergoing the learning experience, Waern (1987) asked her subjects to describe  their experience of using the software in terms of advice to a novice.  One subject referred to an experience with a word processor by saying, "it is important to remember that it really works like a typewriter" (p. 290).  Another subject said,

 

"It's different from a typewriter because you can't just type over on a typewriter; you have to rub out first. And you can move lines sideways, and you can make empty lines and move lines downwards, and all kinds of splendid things you can't do on a typewriter." (p. 290).

 

Mental Models Used for Creativity

 

Waern views creativity as emerging out of generalizations and conflicts. Creativity through generalization occurs when "ideas which are relevant in a particular context are applied to another context" (Waern, p. 285). Conceptual conflict arises if a model does not result in an anticipated outcome. Such conflict  triggers conceptual activity which may result in breakthroughs in the development of efficient ways of handling tasks.

 

Waern gives useful examples of conceptual conflict. A novice subject working on a piece of software is reported as saying, "It's quite awkward this--I get furious. Now it jumps out again--it doesn't do as it should!" (p. 290). Waern found that, under pressure of time, learners generally failed to develop creative models.

 

We found this literature useful in conceptualizing our study of learners in computer contexts involving specific software. The broad range of approaches cited helped us in developing an understanding of learning new software in an environment supported by manuals, an expert and on-line help. On the basis of these readings we were able to generate some questions to guide our research.

 

Our study sought to examine the manner in which three learners approached an unfamiliar piece of software. Given the widespread use of computer technology, from pre-school to college levels, we were interested in exploring how learners drawn from the cross-section of the educational spectrum (and therefore across ages) would engage the task of learning a new piece of software. We decided to vary the educational level of the three participants and in so doing obtain an age variation as well. The participants worked with a common piece of software on similar equipment.

 

In considering the outcomes of the study, we were interested in the learning strategies used by each of our participants. We also wanted to compare the learning strategies adopted by the three learners in the hope of finding areas which might prove to be worthy of further examination in the development of computer interfacing. Our search of the literature had failed to discover research on comparative differences between computer users of different ages and different educational levels. Other researchers have focused on specific educational levels, e.g.. Papert (1980), who looked at elementary school children.

 

Drawing on the literature outlined above, we defined a learning strategy as made up of two components: 1) the use of mental models in learning new software and 2) the use of resources such as manuals, on-line HELP and experts. We then went on to formulate two research questions for our study. These are:

 

-What uses of mental models can be inferred from the observation of three non-expert learners engaged in learning a novel text/graphics software;

 

-What resources are used by the informants to support their learning?

 

A Case Study: Behaviors of Two First-Time Software Users

We begin by looking at the ways in which two participants of varying age use the same software package.  The first is a pre-school learner as referred to as Alex (Case A),  and the second is a  college learner referred to as Sandra (Case B).  The only criterea for their selection was that they (1) were willing to take the time to participate in our study, and (2) they had no prior experience with our software.  Our rationale for selecting two learners of different ages was to explore (compare and contrast) interface issues from two distinct developmental perspectives.

In the course of this study, we administered a pre- and post-observation questionnaire (Appendix A).  This was done for several reasons, which are listed below.

Regarding the learners before the observations, we wanted to know:

1.  If the expertise of the participants suitable for our research (neither novice nor expert). 

2.  How the participants viewed computers and technology (hostile, friendly, comfortable, etc.).

3.  How the participants view both themselves as learners and learning.

4. How the participants think they will learn the particular software presented to them.

It should be noted that these questions (as is also true of the post-observation questionnaire) were not formally asked of the pre-school learner for reasons of comprehension.  However,  the pre-schooler's attitudes and proficiencies were still discernible.

The post-observation questionnaire was designed to concentrate more specifically on the software package used.  As a result, our focus was narrowed to look at areas 2-4 above and any possible changes, with more attention paid to #4.  As a result, we were primarily interested in knowing:

1. As with the pre-observation questionnaire, how  the participants viewed computers and technology (hostile, friendly, comfortable, etc.) after their experience.

2.  How the participants view both themselves as learners and learning after having learned using this software.

3.  How the participants' actual learning compares with how they thought they would learn.  Also, how do they now view effective learning in this type of context?

Case A

"Alex" is a 4.9 year old Caucasian male student attending a preschool program in Ypsilanti,  Michigan.   He was recruited after discussions with the pre-school teachers about children who were likely to include computer use in their activities and who would be cooperative.  We initially identified one child based on these general criteria.

 

However, after our preliminary observation, we realized that an extended study of Alex in conditions where he was isolated from his peers was not feasible.  After the first observation, Alex was not willing to cooperate in the milieu which we had chosen.  We therefore changed our approach and decided to observe Alex as part of a larger group of pre-school children working on the same software.  This re conceptualization proved to be fruitful and proved to be an important learning experience for us.

 

Alex is comfortable using computers in general; more specifically, he demonstrated familiarity with the mouse operation and other basic computer functions such as saving and printing work and accessing other software programs. Besides his experience using computers almost daily at the pre-school, Alex also has a computer at home which is used primarily by his parents.  While he has had experience with other software packages, including another pre-school-level paint package (KidPix by Broderbund Software) , this was his first experience with the software used in this project,  Easy Color Paint program. 

 

We attempted to administer a formal pre- and post-observation questionnaire. However, we immediately found it necessary to modify our use of these tools to take into account the age and comprehension level of our subject. Through more informal discussions with Alex over the course of the interviews, we were able to assess that he perceived himself as being competent in the use of computers.

 

Case B

"Sandra" is a 19-year-old female Hispanic undergraduate student from the East Lansing area.  She was recruited from an office where one of the researchers works and seemed interested in participating when asked.

Sandra told us that she used a computer every day at work for word processing tasks.  She informed us that she had only used one art-based program in the past (Print Shop), but on an extremely limited basis.  She did know how to use a mouse, but also stated that she had practically no experience with either the type of computer or software that we were going to observe her using.  In her self-report, she told us that she felt she was a 4 (on a scale of 1 to 10) in regards to her proficiency in using computers.

 

Sandra's view of computers and technology seemed quite positive.  She informed us that she usually liked to use computers.  She did indicate that when she didn't understand a particular task, she didn't especially like using them.   When asked if she would use a computer more if she could, she said, "yes, definitely".  She also felt that a computer could be used to help learning, "if you know how [to use one".

 

As a learner, Sandra seemed to see herself in a rather positive light.  She informed us that she definitely liked to learn new things.  When asked how she knew she had learned something, she replied, "[when] you are able to do it, [when] you remember how you do it".  She indicated to us that she felt she learned best by practicing a task and doing it herself.  She stated that resorting to resources such as manuals would be a last resort.

 

When asked how she would learn aspects of this particular piece of software, she indicated that she would learn it by "going through it".  She thought she would learn it best by just doing tasks repeatedly and would use extra sources of help only if all else failed.

 

Her self-­assessment seems confirmed by her answers to other questions on the pre-observation questionnaire.  It was revealed that she uses computers on a regular basis at her part-time job with the university.  She indicated they were used primarily for word processing and spreadsheet applications.  She also indicated she had very little experience with graphics-based software.

 

The Research Setting

The main part of this research involved the observation of the participants learning the software over several sessions.  The software we chose was Easy Color Paint (ECP) 2.0 by MECC (1991) which was installed on a Macintosh LC in each case.  This is a full feature drawing program designed for use by children or adults.  It includes a range of basic art tools, e.g., brush shapes, shading, lines, geometric shapes, and cutting and pasting.  The program employs standard Macintosh pull-down menus.  Text can easily be added to a picture in a variety of fonts (type styles), sizes and colors.  The program includes 24 pictures to color and on-line help.  Products can also be saved or printed. 

By the third observation, we wanted the participants to have learned how to use some or all (depending upon age) of the following functions:

-Drawing line(s) of differing thicknesses.

-Clearing a screen of all drawing/ writing.

-Fill in background/foreground colors.

-Use the on-line help function.

-Save screens of work to a storage device.

-Change colors and create different patterns.

-Undo mistakes.

-Erase

-"Drag" an object to another part of the screen.

-Magnifying a portion of the screen in order to fill in fine details.

-Type text of differing sizes on the screen.

-Draw different types of "frames" around objects.

-Create different geometric shapes.

-Fill in spaces using "gradients" (colors that go from dark to light and vice versa).

In addition, the participants were given several different sources of aid in case they encountered problems (again, these varied depending upon the ages of the learners).  There was an on-line help function, by which the learner could find answers to problems within the program itself, a user's manual (not used with the pre-school learner), and an expert in the use of the software (represented by the each of the respective researchers).

 

The Rationale

We used a case study approach for this particular project.  As Glenda Bissex says,  "case studies...enable us to see individuals as individuals" (p. 10).  In addition, there are some comparisons that can be drawn even among a few cases.  Bissex continues, "when several individuals are compared, common traits as well as differences become apparent" (p. 10).  She also states that this method may perhaps "mean nothing to a scientist" (p. 10),  but can in fact have implications in the study of the humanities.

 

According to Moll (1987) the most commonly used techniques in the study of "conscious cognition" in computer contexts employ interviews, questionnaires, "thinking aloud" and video-taping. The disadvantage of "thinking aloud" is  with "subjects who are less practiced in formulating their thoughts" and who may "verbalize a small amount of their thought processes" (p. 407). Other participants may verbalize that they have carried out actions which they have not in effect carried out (p. 407). As a result, Moll argues for the use of more than one method for collecting data. In our study, we monitored the participant's verbalizations as well as their actions using audio and video tape.  Our study followed five basic steps:

 

1. Pre-observation evaluation

2. Observation #1

3. Observation #2

4. Observation #3

5. Post-observation evaluation

 

Each observation lasted between 45 minutes and 1 hour.

 

In the first observation, the participants were allowed to explore the software in any manner they wished.  They could ask any questions or consult any resources they wished.  This session was designed to allow them to explore the software.  They were told that they would eventually need to reproduce some functions of the software and to experiment with this in mind.

 

The participants were given some specific instructions during the second observation (which was done 7 to 10 days after the first observation).  In the initial stages of this  session, the participants were encouraged to re-familiarize themselves with the software and do any further exploration they wished.  They were then given various specific tasks to do.  Following is a sample[2]:

"Draw a straight line splitting the screen in half.""Draw a diamond."

"Write 'hello' in big yellow letters." Also during the second observation, the participants were informed that they would be required to fulfill specific tasks as well as drawing a "free-form" picture during the last session.  By the end of the second observation, the participants would have used all of the tools necessary to complete the third observation.

 

The third observation was designed to see how many of the tools were understood and used.  The participants were given specific tasks similar to the above in order to complete a pre-formed picture. They were then encouraged to draw any picture they wished using at least five of the previously learned tools (this task was not required of the pre-school learner). 

 

During these observations, careful note was taken of the exact steps, tools and procedures the participants took.  Also,  anecdotal evidence (comments, questions, etc.) was noted.

Results

Due to the different natures of the two learners studied, the results will be reported in two  different sections.  Again, due to the dissimilar ages of the learners, data was gathered differently in each case.  One can imagine the difficulty in asking a 4-year-old to tell "how proficient [they] are at using computers"!

 

As a result, the data gathered on Alex (et al) does not include some of the quantitative data reported from the observations of the two older learners.  This is due in part to issues such as time on task and attention span. In this section, we feel it is best to treat each group as a distinct case.

 

 

Case A

 

Since part of the design of Easy Color Paint employed text, such as printed pull-down menus and on-line help, these had limited meaning to Alex, since he had yet to develop the level of reading ability used in this program.

 

He would attempt to sound out and pronounce the command words before selecting them, but often would not comprehend their meaning. This was illustrated by his repeated use of the word "soiled" for "solid" from the "Fatbits" menu.  He would point at the lines and say "these lines are soiled now after selecting the "Solid" option.

 

Despite this, Alex did demonstrate that he knew that text had function, for example, "print" and "save."  He was also able to sound out some words, and was able to write his name.  In general, Alex did not grasp the meaning of all the text used in Easy Color Paint.  He was, however, able to discover the appropriate functions of many of the individual menu items.

 

Our initial attempts at asking Alex to tell us about his learning or thinking processes were unsuccessful.  For example, when we asked him "how do you know when you've learned something?", he responded by describing an art activity he had worked on the previous day. If we asked him to give his rationale for selecting a particular tool or color, his response was "because".    We found that it was more practical to measure his level of knowledge by having him teach us, or better yet, teach another child.  It was only by talking about activities that were of immediate interest to Alex that we could talk with any degree of detail.

 

Talking about an abstract concept such as learning is difficult for a pre-schooler. The pre-schoolers we observed would either indirectly answer our learning-related question, or refer to something more meaningful; generally a current event or concrete object.  The following are several examples:

Researcher: How did you learn how to use computers?

Alex: I have a computer at home

R. How did you learn to use that one?

A. I just did

R. Do you think it helps you get smarter?

A. Yes

R. How?

A. It just does.

R. Do you know how to read

A. Nods his head yes

R. Did the computer help you learn to read?

A. Nods "yes."

R. How?

A. Well I can make letters.

Another child was asked about learning:

R. What are you learning when you do this?

C. I like it when it does this (demonstrating and hopping with excitement).

R. How do you know if you've learned something?

C. Because I know it.

 

A different child:

R. Show me what you can do with this program.

C. Do you know how you can make big dots? I can make big dots -- you just have to keep pressing on this (mouse button). Watch... see? (demonstrating). (The children seem proud to show off their new abilities/discovery).

R. When you use this computer with the mouse, what are you learning about?

C. I just learn things to do.

R. Do you think you learn something when you use the computer?

C. I have a cold. I'm almost better.

One consistent technique that we discovered among all the pre-school learners we observed was the use of exploration. Alex in particular demonstrated a strong tendency to  find the extreme limits of the software.

 

Once he would find a feature, (e.g., filling a screen completely with the color repeatedly), Alex would typically test it's extreme boundary. One of his favorite discoveries occurred in our last observation after he had typed his name on the screen. He then discovered the point size feature by accident when the letter became very small. He quickly returned to the same menu, this time picking the largest number on the very end of the list. This made the letters of his name very large (nearly an inch tall). He was very pleased with the result.

 

Over an 8 minute period in the second observation, we counted over 50 different actions (mouse clicks, selection from tool bars, etc.).  He had discovered every sub-menu by going right down the menu bar.  At one point he went  in and out of three layers of menus into some advanced features.  Over this period his picture changed 19 times.

 

It seemed that the pre-schoolers rarely asked an "expert" adult for help.  Even when they did ask, it was after every other option had been exhausted.  The students would often turn off the computer and start from scratch instead of asking for assistance. This was in spite of the fact that  we told the children that we were available. 

 

Even when a child was confused or stuck and help was offered, the request was ignored. Only one child during the three hours of observation, Brandy, deliberately asked us for help; once when she was trying to change colors and again a short time later when trying to draw.   She was the only child to actually say, "I need help."

 

We also noticed extensive use of creativity in these children's learning of ECP.  This was expressed in a variety of pretending behaviors inspired by the accessible features of ECP. In our first observation, Alex discovered the fill option and painted the screen completely black.  He then found the eraser.  When we asked him to explain what he was doing, he said "I'm painting the screen blank."  His creations were fun to watch develop. 

 

- "Hey...that's no fair -- it's not a hexagon" (when adding a shape).

- "I'm looking for different shapes.  That's a square.  That's a nonagon."

- "Now it's a green island -- I'm going to paint a square -- it's a blanket.  Now I have a black sea."

- "That'd be a pretty picture.  Look at my ...square; hey, now it's a ball.  Now it's a blanket-- Now it's a ball again [as he rounds the edges with the paint tool] for Santa.  I want to print."

In order to see how the pre-schoolers learned, we found an effective technique was to observe some of the learners teach others.  This gave us insight into how the children learned themselves.

 

During one session, Alex helped Brandy with ECP for a period of about 3 minutes. This provided an interesting chance for us to see Alex in the role of a teacher helping a novice acquire a new ability.  It was in this context that we gained the most insight of Alex's view of learning.  He was not verbal in this role. He put his hand over hers and guided the mouse, along with short explanations "go to this side, and click on this.... do you see?"  He would end up completing the task himself, showing the girl some of the possibilities.

 

Brandy, a three-year-old girl who was working with ECP for the first time was asked directly to tell us about how she learned to use the mouse, she replied "by watching him and him" pointing to Alex and another boy standing nearby.

 

In order to get some feeling for his ability to demonstrate his skills in a direct way,  our group had originally planned on using a structured built-in tutorial task. However, in light of our newly found knowledge about pre-school learners, we realized that this would have little meaning to Alex.

 

Instead, we asked him to draw  a picture of a man.  After more than 15 minutes of experimentation with other features, we still didn't have our "man."  Finally, we called him over and asked him one more time: "can you draw a man with the red crayon?" (The "red crayon" referred to here is actually the "pencil" tool used with "red" picked from the color palette). In less than 30 seconds, using the "red crayon", he quickly sketched a stick figure with  eyes, mouth, and a cartoon-style speech balloon containing the word "ga" (like a baby would say). 

He then typed his name on his own, the numerals 1-10, and adjusted the size of the font.  We then asked him to add a green Christmas tree.  He did so with no problem. Alex left no doubt that he had mastered at least the basic functions of ECP and could demonstrate this knowledge in a non-spontaneous manner.

 

 

Case B

Observations

 

In learning this software, Sandra had several options when considering what actions to take.  In using the program, she could access the on-screen menu, which requires no manipulation outside of simply clicking on the "tool", or the pull-down menu, which contains more complex functions and procedures.  In the first session, her use of on-screen tools dominated. 

 

As requested tasks became more sophisticated, and as Sandra discovered the need for further exploration of the program, the use of the pull-down menus (even for use of the simpler tools) became more prevalent.   In addition to these avenues of exploration, there were three other sources of help that the participant could go to if she was unsure of how to proceed.  There was a user's manual, an on-line help menu and an expert readily available at all three sessions. A summary of her use of these various strategies appears below (Table 1).

 

The above data show that only 11% of the observed interactions with the software were involved with consulting formal help resources in the first session.  In the second observation, during which she was given specific tasks for the first time, this increased to 20%.  During the third observation, in which she was given a mixture of more specific instructions and self-directed activity, she resorted to the formal resources of help 25% of the time. This also shows that the respective percentages with regards to "experimental" methods (i.e. picking tools and using them without accessing any external help) are 89% for the first observation, 80% for the second, and 75% for the third.

    

REPORT OF ACTIVITIES BY TIME

 

 

EXPLORATION

MANUAL

EXPERT

ON-LINE HELP

Session 1

89%

1%

9%

1%

Session 2

80%

1%

16%

3%

Session 3

75%

3%

19%

3%

 

                                                                                                                        TABLE 1

 

First Session

 

In the first observation, Sandra was asked to learn to do "as many things as possible".  Her first actions were to test as many of the on-screen tools as possible.  She appeared to readily grasp the function of the "pencil" and "magnifying" (which magnifies a portion of the screen) tools, as well as the "hand" (which moves the whole screen) and several tools which allow for the drawing of different geometric shapes.  One exception seemed to be in regards to the "magnifying" tool.  In a subsequent section, she appeared to have either forgotten how to use the tool, or else showed that she had not really understood the tool's use in the first place.

 

The most interesting event occurred when she attempted to "clear" the screen early in the session.  She signaled her intention of doing this by saying, "I am going to erase the screen now".  She first attempted to do this by choosing the "eraser" tool from the on-screen menu and began erasing the screen.  When the expert asked her if she thought there was a faster way of erasing the screen, she agreed that there should be.

 

She eventually went to the on-line help for assistance but could not locate the correct menu to assist her.  After this, she returned to the "eraser" and tried the previous method again.  Upon coming to the conclusion that this wouldn't work, she consulted the owner's manual and looked up "Erase" in the index. 

 

She did find a method of clearing the screen which required several steps but accomplished what she wanted to do.  In the process, she picked a pull-down menu which has a "Clear Screen" option clearly displayed but appeared not to pay attention to it.  She muttered "I want to erase the whole thing" several times during this process.

 

Later on in the session, she once again wanted to clear the entire screen.  Instead of choosing to use her previous strategy, she went directly to the pull-down menu and chose the "Clear Screen" option.  She appeared surprised but made no comments indicating that she realized what had happened (such as, "oh, that's how you do it").

 

She also began learning how to fill in colors using the "bucket" tool.  Although she succeeded in getting the tool to work correctly on several occasions, it is still clear that she did not really understand how it works, as toward the end of the session she stated, "I'm still trying to figure out how this works".  She also attempted to use the "text" tool (which appears as an "A" in the on-screen menu).  After several tries, she did successfully use the tool.

 

Second Session

 

During the second session, she said she was going to "goof around" for a while.  She first went to the on-screen menu and tried to use familiar tools ("hand", "pencil", "geometric shapes").  After this initial experimentation, she once again attempted to clear the screen and failed to do so.  She eventually went to a pull-down menu and chose the "undo" command (which simply cancels the last operation performed).   In doing so, she had to view (and actually high-light in the process) the "Clear Screen" option.  When asked why she chose "undo" instead of "Clear Screen", she said she didn't "want to lose the whole screen".  After several more tries, she did try the "Clear Screen" option.

 

It was at this time that we began giving her more specific tasks.  She was asked to make several very thin lines.  She used the "pencil" tool to draw them, but the settings were set so that a rather thick line was drawn.  She randomly picked pull-down menus in order to make the line thinner, and eventually asked the expert how to do it.

 

She was also asked to move an object on the screen.  She immediately went to the "hand" option to move it.  When asked if she thought there might be another way to do it, she indicated she was "not sure". When asked to explore the possibility, she tried all options (including on-line help and the manual), but finally gave up on the attempt.  During this time, she indicated that "on-line doesn't really help".

 

She showed that she had not retained knowledge of the "text" and "bucket" tools, either.  She was able to use them after some experimentation, however. It was during the latter part of the session where she began correctly using the "Clear Screen" command.  Not only did she begin going to it when she wanted to clear the screen, but she started expressing that she wanted to "clear the screen" instead of "erasing" it.

 

The last task she was asked to do was  a "gradient fill".  This was a very difficult task requiring several steps using both on-screen and pull-down menus.  She discovered one of the proper steps, but had the expert take her through the entire process.

 

Third Session

 

This session consisted of two parts.  Sandra was first asked to do specific tasks to construct a particular picture.  She was then allowed to create a picture of her own during the second part.

 

She used many of the previous tools mentioned without hesitation ("pencil", "bucket", "magnifying", "text").  However, she still seemed uncertain as to exactly how the "bucket" tool worked.  When asked why she had to use the "bucket" three different  times in order to fill in the entire area, she responded, "maybe that's just how the pictures are".

 

She also had to move an object using the "lasso" tool instead of the "hand" tool.  After trying to use this tool for a few moments, she went right to the manual and was successful in using the "lasso".  When asked to do another "gradient fill", she was still unsure and had to be led through the process by the expert.  Sandra once again showed mastery of the "Clear Screen" concept, as she used it often and without hesitation.  She also referred to "clearing" as the opposed to "erasing" the screen.

 

When asked to create her own picture, she used the simple on-screen tools that she had first learned ("line", "pencil", "eraser").  She did use the "lasso" tool again, this time without hesitation or problems.  The one problem she did have was with the "bucket" tool.  She was wary of using it for fear of filling in the whole screen with the chosen color.  Her fears were justified as she did succeed in filling in the entire screen.  After failing to discover what the problem was, she chose to "undo" the previous "bucket" command and announced, "there, I guess I'm done".

 

Post-Observation Questionnaire

As noted earlier, we wanted to find out several things from the post-evaluation interview.  The first was to see if there was any change in how Sandra viewed technology.  When asked if she thought this experience had changed how she felt about computers, she answered with an emphatic "no!"  She also indicated that she couldn't see any use for her using this or similar software in the future.

 

Next, we wanted to see how she viewed herself as a learner after this experience.  She indicated that she felt she did learn something from this software.  This consisted of mostly concrete operations as shown by her statement that she had become familiarized with "tools, functions and patterns".   When asked how she knew she had learned something, she re-stated her previous definition that she was able to recall it and she "didn't have to look things up as much". 

 

When she was asked if she viewed learning any differently after her experience, she initially stated that she hadn't.  However, before the next question was asked, she said, "well, yes, I do".  She then went on to explain that she had never realized the importance of practice and repetition involved in learning a task before.  She noted the importance of doing a task "over and over".

 

The last thing we wanted to know was how Sandra felt the  learning process had actually compared with her pre-observation expectations.  She stated that she had gone about learning the software "pretty much" as she had thought she would, but felt she had consulted the expert more than she would have expected.  She also stated that she felt she would probably go about learning similar software in a similar manner in the future.  When questioned on how she might teach someone else this same software, she said she would "have them get on the software...and have them do it themselves".

 

Conclusions from Part 1

In evaluating our results, there were several questions that we wanted to answer.  First, taking the mental model approach suggested by writers such as Moll, Waern and Wright, we wished to learn what mental models, or pre-existing mental constructs, our learners had in approaching an unfamiliar piece of software.  Secondly, we wanted to know what resources the learners would use in exploring and expanding these models.  Finally, we wished to see if any similarities or differences could be seen between the learners in the three chosen age groups.

 

Faced with a small sample size and the impossibility of truly comparing the learners to one another, we feel that it would be more productive to instead compare the learners to one another against the background of what we know about mental models and how learners think based on some of the literature.  For instance, do we see in the results patterns which are consistent with some of the ideas put forth in the previous literature that we have looked at?  Do the two learners using this particular software evidence behavior that might reflect some of those which are suggested by our study of mental model theories? 

 

Thus, we do think it will be useful to examine how particular learning constructs evidence themselves in these learners' actions and to also see how these same models materialize as actions on their part.  In so doing, we can compare the two learners in how they relate to learning processes.  As we will discuss later, for instance, we do think it is useful to talk about the fact that all three learners used a great deal of exploration in their learning.

 

All  researchers reported a strong inclination on the part of the learners to explore and "play" with the program.  In fact, several of the researchers reported difficulty  in getting the learner to talk out loud about their learning during the sessions due to the fact that they were so engrossed in exploring the software.  While it certainly can't be denied that this could be partly due to both the nature of the software and the nature of our particular learners, it is also quite possible that some of this tendency can be attributed to how learners approach software in general.

 

Moll proposed the idea that learners must have a certain knowledge from which to derive predictions.  As an extension of that, we would expect that this "knowledge base" would expand and change over time.  The amount of expertise reached by the three learners over a short period of time indicates that knowledge which allowed functional operation was indeed occurring in the learners.  It was also apparent that there were certain expectations about computers that the learners brought to this software/computer context.  Both Alex and Sandra stated on different occasions; "there has to be a [faster/better] way to do this".

 

Perhaps the best example of this process comes from Sandra and her attempts at clearing the screen.  Her initial attempts at performing this task seemed directly tied to the fact that she "knew" that one takes writing off of a paper by "erasing" it.  The persistence with which she initially insisted on attempting to "erase" the screen adequately shows that this "prior knowledge" was in action.  As she became more familiar with the software, however, both her language and her approach to the task changed, as shown by her frequent references to "clearing" the screen by the latter stages of the observations.

 

We, along with Moll, suggested that computer systems allow for a great deal of  exploratory creativity by learners in examining and testing their own models.  Again, this appeared to be true with our learners.  The amount of time spent in creation using different functions paired with the total amount of time in experimentation versus help-referencing show that the learners were in fact using some of the creative capacities of the particular system to which they were exposed (see Appendix B for examples).  

 

In her attempts to move an object, Sandra accomplished the task using a tool normally utilized in a different way; and the pre-school children were observed to be especially proud of new discoveries.

 

Observation and interpretation also appeared to be an important function of mental models to Moll.  These functions help the learner to determine what to pay attention to, what to ignore, and how to fit new information into what they already know.

 

We saw many of these proclivities in the observations of our learners.  For instance, it was interesting to note what apparent inconsistencies were ignored by the learners.  Alex seems to present an excellent example of this.  When attempting to make a solid line, he picked the appropriate word from the menu.  However, he read the word as "soiled", as opposed to "solid".  When he pointed at the screen and showed satisfaction at what he had accomplished, it was  clear that he had fulfilled a desired function.  However, he ignored the fact that he was referring to this line as "soiled". 

 

This instance points to what Moll referred to as the tendency to accomplish a task rather than always attending to the cognitive aspects of a task.  Alex had accomplished his goal and was satisfied with the outcome; he did not appear bothered by the fact that his language did not match his actions.

 

Another aspect of this ability by  learners to overlook inconsistencies is the tendency of learners to give up upon failure of incorporating a certain model into their thinking.  While this seemed to be present in some cases, we found it to not always hold true.  Sandra evidenced this, as noted, during the end of the last observation.  After making several attempts at filling in a space and failing, she chose to "undo" her command and declare, "I guess I'm done". Having not understood the operation of a function, she chose to instead to simply not make an attempt.

 

We also noticed that the development of models was possibly hindered by forgetfulness in some cases, as was the case in some of Waern's participants.  For instance, although Sandra used the "Clear Screen" command successfully by the end of the first session, she had forgotten how to use it by the beginning of the second session 7 days later.  It again took her several minutes to use it successfully and it was only during the last observation that she used the command successfully throughout only during the entire session.

 

Alex appeared to be an exception to this, however.  When he was on-task, he seemed to perform functions readily and retain the knowledge in future sessions.  This could be attributed in part to the fact that he had used an art-based program in the past, but it is impossible to know this for sure without further research.

 

Alex and Sandra both seemed to view the experience in more practical terms by communicating what they could do as opposed to what it was like to learn.   When asked how he knew he had learned something, Alex responded, "well, I can make letters".  Similarly, Sandra told us that she knew she had learned some of the capabilities of ECP because she could "do things" and she "knew how to do it".  This is not to say they had not constructed any models (although they may have not), merely that they had not communicated them.

 

The creativity involved in using this software was evident, the conceptual conflict Waern mentioned as leading to the creativity was not as readily apparent but was present, nonetheless.  Sandra was able to clear the screen initially via non-standard means.  Despite the fact that they did not fully conceptualize the task, they were still able to complete it to their satisfaction. 

 

In examining this data, then, we did make note of some similarities as well as differences.  It was apparent that all three learners preferred exploration as opposed to consultation of outside resources.  The percentage of time spent in exploratory-type activities was overwhelming, as all three learners used such strategies extensively during all sessions.

 

Surprising to us was the fact that the youngest learner in our study were the most likely to not seek help from external sources.  The act of actually shutting off the computer and starting over as opposed to seeking help was not approximated in either of the older learners.  Although the older learners also failed to use outside help extensively, the reticence to do seek this help was found to be much stronger in the younger learners. Since we tend to see adults as able to guide and help children learn, it is sometimes astonishing to see that such help is often not welcome!

 

Part 2: A Look At Other Software In Light of Part 1 Observations

Based on the data collected from our observations of the two learners, can we make some assumptions about the design of software?  Our observational data illustrates that there are similarities in interaction styles common to both learners regardless of age or other individual variables. If the software interface matches an individual learning style, then the learner is more likely have a successful experience.  When the two don't match, the results can be frustrating; the learner might give up or select some less challenging activity to pursue in order to achieve success. Some might even avoid computer-related activities altogether.

We now look at a sample of 41 children's software programs currently on the market for MS-DOS and Macintosh computers.  This age level presents a particular challenge for software designers because of the developmental levels of the children and their concrete, egocentric  modalities of thought.

Attributes Of Computer Users: Some Inferences From Our Observations

 

Based on both the mental model theory proposed by Moll and our observations of two learners from varying developmental levels, , we attempt to make some deductions about first-time computer users as they work to understand a software package:

 

1.  First-time computer users need to have a base level of technical skills. For example, both of our users struggled when manipulating the mouse to line the arrow up with a program icon. In most programs designed over the past year, mouse operation is a prerequisite skill. If a first time user has never used a mouse or has not yet mastered spatial abilities, his or her use of the software will be limited until a basic "point and click" aptitude is mastered.  Pull-down menus, use of shift, control and function keys, or the necessity of typing a word can make a program more complex to a user who may not yet have these skills.

2. There must be a minimum dependence on written instructions which are at an appropriate reading level. If the reading ability required to operate a program is higher than the program's intended audience, problems can occur. In our observation of Alex, for example, none of the help utilities were used because he did not have the ability to read. In our examination of the software, we will measure the skills required, or Minimum User Competency (MUC) and the menu design (MD) for each of the 41 programs. These scales attempt to take into consideration the level or reading ability used in the program, as well as the possibility that a user's manual may need to be used.

3. Software users are active, not passive.  Both Alex and Sandra rarely paused while using the program.  They like to experiment with all the keys in order to determine the capabilities and functions of the software. If they enter into an activity, they want to know that they can also exit the application easily. We will consider the ability of each package to support this kind of active experimentation.

4. First time users need clear, understandable feedback. Response to the keystroke must be quick to insure a feeling of control, and visual and auditory feedback techniques that are implemented must be of meaning to the intended audience (in this case young children). Our checklist will take these issues into consideration.

Unlocking the potential of the computer for first time users depends on the design of the interface. A piece of software with a well-designed, ergonomic interface will be more likely to lead to success while one that is less transparent may lead to frustration and failure. We will look more closely at the interface designs of 41 programs by way of four checklists (Buckleitner, 1992) which are described below. For the following four categories, we have supplied (1) a description of the scale; (2) the scale itself; (3) the results of how the software we examined performed on that scale and  (4) an interpretation of those results.

These checklists are a part of a software evaluation system used on 480 other children's programs. They attempt to provide a workable framework for each separate variable along one dimension. Scoring was done by one researcher.  There was no inter-rater reliability in the scoring process.

 

Minimum user competency scale (MUC)

 

(1) Description

We first attempted to measure the "minimum user competency" (MUC) required to use the software, that is, the physical skills required to operate the controls. To do this, we first considered the kinds of skills required by various kinds of software interfaces and arranged them from easy to difficult. The easier the program to use, the higher the associated numeral.  This allows us to rank the software in ordered categories. 

 

The reviewers instructions are: "When considering the portions of the program designed or intended for the child's use, rate the method in which the child's answers are entered into the computer (Numbers indicate the method's point value)".  (The higher the number, the easier it is to use).

 

(2) Scale

 

Easy/Less complicated

 

9___Touch screen or use voice

8___Touch any key or one key

7___Move the cursor to a visual representation (icon) using mouse input

6___Use only 1 to 4 keys on the keyboard consistently e.g., spacebar and  RETURN

5___Move the cursor with the arrow keys

4___Find and press one of the number keys and/or RETURN

3___Find and press one of the letter keys and/or RETURN

2___Type a Word/RETURN

1___Press more than one key at a time e.g., CONTROL-C

Hard/More complicated

3) Results

Of the 41 programs reviewed, the Mean score on the above scale was a 6.6. The ECP (Easy Color Paint) score was 7.

(4) Interpretation

The higher the score on this scale, the less complicated the initial mechanical skills to operate a program. The analysis of the 41 new programs yielded an average score of 6.6 with a Standard Deviation of 1.53 on this scale. This tells us that most of the software (almost 70%) scored between 5 and 8. We can conclude from this data that the mouse is becoming the standard interface for children. It was interesting to observe that young learners (as young as three) were able to master basic mouse skills as fast or faster than adult learners. The touch screen, which is easier to use by young children has not been widely accepted, most likely because of the widespread acceptance of the mouse and the expense of the touch screen devices ($200 to $300).

MENU DESIGN SCALE (MD)

(1)  Description

A menu is defined as a point in the program where decisions are listed. In order to make a decision, the user must interact in some way with a program menu. For example, a menu which makes use of well-illustrated pictures or icons and is not dependent on text would score better than a menu made up of printed words only. Consideration of these issues is of particular importance if the software package is marketed toward young users such as Alex. Like the previous scale, different types of menus were first listed. Next, they were rank ordered according to difficulty, relative to each other.  This process allowed us to rank-order each program's menus relative to other programs.

(2) Scale                     

Easy/Less Complicated

6___Picture or talking menu, touch screen to make the selection

5___Picture or talking menu, move cursor to selection with mouse

4___Picture or talking menu, move cursor with arrow keys/RETURN to make a selection.

3___Picture or talking menu, press specific key to make choice (e.g., a number key)

2___Simple written menu with less than six choices, using large letters.                                                                                                                                                                                       1___Written menu with more than six choices and small letters

Hard/More complicated

(3)     Results

Of the 41 programs reviewed, the Mean score was 3.95. ECP score was 5.

(4)      Interpretation

The mean of nearly 4 on this scale is a good indicator that children's software is somewhat easy to use, but is still not ideal.  Point and click picture menus like those used with ECP are becoming standard for IBM compatible and Macintosh computers.

 

EASE OF USE

 

(1)       Description

 

How easy is the program to use by a first time user?  We measure the software using this scale which attempts to capture some of the key issues first time users might face. To attempt to measure this software attribute, we  used a collection of 3 category Likert scale checklists on factors identified as being related to "ease of use."  Numerical values were created by adding the values created by each scale.  A check under the "always" column was worth 1 point. S.E. (some extent) = 1/2 point, and "never" = 0. NA marks were not considered in this analyses.

 

(2)      Scale

 

                                                                              Always     S.E.            Never       NA

-Can the user do the program the first time

 without help                                                      ____          ____          ____          ____

-Can the user do the program after the first

 few times without help                                   ____          ____             ____                                                          ____

-Does the user have control over

       - time allowed for problems                     ____          ____          ____          ____

       - rate of display                                          ____          ____          ____          ____

       - order of display                                        ____          ____          ____          ____

       - exiting at any time                                  ____          ____          ____          ____

-Do the written instructions

            - provide technical instructions when

     needed                                                   ____          ____          ____          ____

  - provide strategies for extending

             concepts                                                 ____          ____          ____          ____

            - are well organized                              ____          ____          ____          ____

            TOTAL                                                       ____          ____          ____          ____

(3)      Results

Of the 41 programs reviewed, the Mean score was 7.36 (SD = 1.30). ECP score was 9.5.

(4)      Interpretation

9 is the highest score possible, and 0 is the lowest.  The mean of 7.36 gives us a feeling for how "friendly" the software is to a first time user.  Of 41 titles we chose to review, the interfaces appear to be generally quite "friendly".  However, one can still see that a significant amount of the software we reviewed had scores below 7, so again, there is still much room for improvement.

FEEDBACK

 

(1) Description

 

The type of feedback a program gives a user may be an important influence on matching a learning style.  In our study, the learner needed frequent feedback that he or she could understand. At certain points, messages sent by the software were confusing, for example, when Alex was attempting to change brushes and nothing happened. To attempt to measure this software attribute, we  used a collection of 3 category likert scale checklists on factors identified as being related to "feedback."  Numerical values were created by adding the values created by each scale.  A check under the "always" column was worth 1 point. S.E. (some extent) = 1/2 point, and "never" = 0. NA marks were not considered in this analyses.

 

(2) Scale

                                                                             Always     S.E.            Never       NA

 

-The child is aware of when he/she makes

  an incorrect response                                    ____          ____          ____          ____

-The child is aware of when he/she makes

  a correct response                                          ____          ____          ____          ____

-Feedback responses are varied                     ____          ____          ____          ____

-Feedback is directly correlated

   to keystroke                                                     ____          ____          ____          ____

-Feedback is appropriate because it

          -is non threatening                                ____          ____          ____          ____

          -reinforces content                                ____          ____          ____          ____

          -is understood by the user                     ____          ____          ____          ____

-Feedback effectively makes use of sound and

  graphic capacities of the computer            ____          ____          ____          ____

-A record of the child's work

            - can be stored                                       ____          ____          ____          ____

            - can be printed                                     ____          ____          ____          ____

            - is informative                                     ____          ____          ____          ____

             TOTAL                                                      ____          ____          ____          ____

(3)      Results

Of the 41 pieces of software reviewed, the Mean score was 8.93 on a scale of 0 to 12. ECP score was 9.

(4)      Interpretation

The scores above indicate that this group of software is for the most part successful in being responsive to the user.  It is interesting to note that some of this responsivity is due in part to the increasing speed of central processing units (CPU's) of the newer computers. They allow better graphics and faster feedback.

Conclusion

 

In looking back and applying our research to software design, we feel that it is  important to point out what we found to be some of the tendencies in our learners, and by so doing, look at software design in a different light.  By  looking at some of our learners' inclinations can we hopefully give some insight into how software design of the future can be more effective.

 

Both learners relied on exploration more than anything else in learning to use this  software.  This seemed in keeping with what the learners had predicted they would do.  This, as noted earlier, could not be determined with the pre-school learners, as they had a difficult time expressing how they thought they would learn. Sandra, for instance, indicated she learned best by practicing and doing.

 

Both of these learners' actual practice was consistent with these statements, as they were observed to primarily use techniques of exploration (usually 80% or more of the time) while referencing other sources of information only infrequently.  This pattern was true of the pre-schoolers also, although we could not quantify this as well at this age because of the high social context of the observations.

 

There is no way of knowing, given the limited scope of this project, if the  participants do in fact learn "by doing and exploring" in other environments.

In regards to the referencing of additional help, the adult subject exhibited a consistent pattern; i.e. she explored first, and utilized the expert only occasionally while only infrequently making use of on-line help or the manual. The pre-schooler had limited reading ability so manual and on-line help was not used. 

Since some computer manuals are cryptic and confusing at times, many people  will ask someone for help first because it offers more immediate results.  As a result, both cases exhibited a standard hierarchy of method:  exploratory methods were most frequently used, typically followed by utilization of the expert, with the  on-line help and manual being referred to as a last resort.

The analysis of the data of this report gives us an interesting look at some of the specific issues dealing with the usability of certain software with a particular group of users. We can say with some accuracy that while the interface designs are generally acceptable and usable for the intended audience, there is still a substantial number of software packages now on the market that score low on our scales. This means that it is possible to invest in a software title that may not perform well due to a design that does not consider the mental models typically employed by young children.  This kind of specific design information may be useful to software designers.

The limitations of the scales used to gather the quantitative information should also be noted.  To be effective, they should be standardized and subject to more rigorous quantitative design standards. As they are, the best we can hope for is they give us a general comparison of 41 software titles. Making something easy to use can be the most complex job possible, yet that is apparently the task of the educational software designer.

What then, does all of this mean?  There seem to be, in our learners at least, some general behaviors that should be applicable to software development.  For instance, given the reticence of our learners to access both on-line help and reference manuals, transparent software would most likely reap the largest benefits to both learners and software developers alike.

 

If our learners were to prove to represent learners in general, in fact, it would seem that extensive development of manuals (both on-line and written) may in fact be unnecessary.  This in no way suggests that learners won't access help and won't need it.  Rather, we are suggesting that perhaps it would be more beneficial for software designers to either simplify and re-structure help sources, or do further research on what types of help may in fact go under-utilized with a vision towards scaling back certain types of help altogether.

 

The second characteristic of our learners that seems to hold importance for software designers is the tendency to think in an "every-day English" schema.  In other words, it may be quite common for an experienced computer user to immediately look for the "Clear Screen" menu.  However, this may not be as true with the average user of technology.  The average (or novice) learner may be more likely to think in terms of "erasing" and "exploring a museum".  It may be reasonable to suggest that the developers of technology take this into account when designing user interfaces.

 

A last aspect that we found striking in our subjects was the amount of creativity  they employed.   They found a way to complete the given tasks, even if those methods were not those that the software engineers designed to be most efficient or expedient.  Thus, it may be important for the industry to be aware of the importance of building creativity into a system.  For instance, would it be possible to allow several different ways to perform a function?  Even if this required more program memory or more choices for the user to pick from, might not it also possibly allow for more exploration and utilization on the part of the learner?

 

These are just some of the issues that user-interface engineers will need to face in the coming years.  It is certain that, in time, the computer literacy of the average user will increase.  But it is equally likely that the number and sophistication of users will increase at a faster rate if the technology of the future is more transparent and user-friendly.

 

The implications of this for education seem clear.  Teachers lack time to learn new aspects of their profession, children are already being asked to gain as much knowledge as ever before.  Will the technology they use in the future make them work even harder to attain mastery, or will the ease with which they can adapt to this new paradigm increase?  This is a question that technology must answer in the years to come and one upon which we hope this paper has shed some new light.

 

 

References

 

Anderson et al, (1981). In defense of a comprehensive view of computer  literacy--A reply to Luehrmann.  Mathematics Teacher, December.

 

Bissex, G. (1987). Why Case Studies?  In G. Bissex & R. Bullock (Eds.) Seeing for Ourselves: Case-Study Research by Teachers of Writing. Portsmouth: Heinemann, 7-19.

 

Buckleitner, W. (1992) High/Scope Buyer's Guide to Children's Software. High/Scope Press, Ypsilanti, MI.

 

Burton, R. (1982). An investigation of computer coaching for informal learning activities. In D. Sleeman & J. S. Brown (Eds.) Intelligent Tutoring Systems. New York: Academic Press.

 

Chapman, R. S. et al (1983) Microcomputer testing and teaching of verb meaning: What's easy and what's hard. In A. Cherry (Ed.)  Classroom Computers and Cognitive Science.  New York: Academic Press.

 

Dickinson, D. (1992). Multiple technologies for Multiple Intelligences.  Electronic School. Sept., (A8-A12).

 

Douglas, S. & Moran, T. (1983). Learning text editor semantics by analogy. Human Factors in Computing Systems SIGCHI Bulletin, 207-211.

 

Green, T. R. (1983). Learning Big and Little Programming Languages. In A. Cherry (Ed.) Classroom Computers and Cognitive Science New York: Academic Press.

 

Luehrmann, A. (1981) Computer Literacy--What should it be? Mathematics Teacher, December.

 

Magers, C. (1983). An Experimental Evaluation of On-line HELP for Non-Programmers. Human Factors in Computing Systems, SIGCHI Bulletin, 277-281.

 

Mantel, M., & Haskell, N. (1983). Autobiography of a first-time discretionary microcomputer user. Human Factors in Computing Systems, SIGCHI Bulletin, 286-290.

 

Moll, T. (1987). On Methods of Analysis of Mental Models and the Evaluation of Interactive  Computer Systems. In M. Frese, E. Ulich, W. Dzida (Eds.), Psychological issues of human-computer interaction in the workplace. New York: Elsevier Science Publishers.

 

Papert, S. (1980). Mindstorms: Children, Computers and Powerful Ideas. New York: Basic Books.

 

Richards, J (1987).  Rx for Editor in Chief.  In G. Bissex & R. Bullock (Eds.), Seeing for ourselves: Case-Study Research by Teachers of Writing. Portsmouth: Heinemann.

 

Rubin, A. (1983). The Computer Confronts Language Arts: Cans and ShouLds for Education. In A. Cherry (Ed.), Classroom Computers and Cognitive Science , New York: Academic Press.

Sardello, R.(1985). The Technological Threat to Education. In D. Sloan (Ed.),  The Computer in Education: A Critical Perspective. New York: Teachers' College Press.

 

Shrager, J. & Klahr, D. (1983). Learning in an instructionless environment: Observation and analysis.  Human Factors in Computing Systems, SIGCHI Bulletin, 226-229.

 

Suppess, P. (1966) The uses of Computers in Classrooms.  Scientific American, 215, # 3.

 

Waern, Y. (1987). Mental models in learning computerized tasks.  In M. Frese, E. Ulich, W. Dzida (Eds.), Psychological issues of human-computer interaction in the workplace. New York: Elsevier Science Publishers. 

 

Winograd, T. & Flores, F. (1986). Understanding Computers and Cognition: A New Foundation for Design. Norwood, New Jersey: Ablex Publishing Corp.

 

Wright, P. (1983). Manual Dexterity: A user oriented approach to creating computer documentation. Human Factors in Computing Systems SIGCHI Bulletin, 11-17.

 

 

 

 

 

 

 

Appendix A: Questionnaires

 

Pre- Observation Questionnaire

 

1.         Participant's name:

            Participant's Age:

            Participant's Sex:

            Participant's Race:

2.         Do you normally use computers much (computer, Nintendo)?

3.         How often do you use one?  Once a month...week....daily?

4.         What do you usually use one for (or why do you use it)?

            Games, Art, Writing, etc., probe.

5.         Have you ever used a mouse with a computer?

6.         Have you ever used an art-based computer program in the past?

            If yes, which one?

7.         How proficient are you at using computers?

8.         Do you like to use computers?  Why or why not?

9.         Would you use a computer more if you could?

10.       Do you like learning new things?

11.       How do you know when you have learned something?

12.       Do you think a computer can help you learn?  How?

13.       What method(s) help you learn things best?  Watching others, using a             computer, reading a book, having someone explain?

14.       What types of hardware/software do you already know how to use?

15.       How do you think you might go about learning this software?

 

 

Post-Observation Questionnaire

 

1.         Did you learn anything?

2.         How do you know?  What signals or indicates this?

3.         Did you do what you thought you would to learn this software?

            If you did something different, what was it, and why do you think you did             it?

4.         Would you use this software again if given the chance?  Without being asked?

            Why or why not?

5.         Did you feel you had a systematic way of learning this before you started?

6.         What is your role in the learning process?  What part does the computer             play? Other resources?

7.         If you had to learn this (or similar) software again, would you go about it             the same way?  If not, what would you do differently?

8.         Has this experience changed your view of learning?  How?

9.         If you were asked to teach this software most effectively to someone else,             how

            would you do it?

10.       Has this experience changed how you feel about computers?  In what             way?

11.       Did you learn anything from this software?  What?  How do you know?

 

 

Appendix B: Software Data

Title                Date      Price      Concept Area     MUC  MD EAS  FEED

 

A-Mazing Mouse     1987       $45.00       Spatial Relations   5     2    7   8

 

Barbie Design Studio 1992        $29.00       Creative products   3     4   6.5   11

Build-A-Scene      1992        $75.00       Spatial         9     na   8   6.7

Dr. Peet's Pict. Writ1992     $45.00      Reading Process  7     5    8     9

Dragon Tales       1991           na      Reading            7     5    7.5  11

Explore-A-Story +             1992        $85.00         Reading process     7    2     8.5   9

EZ  Play           1992       $45.00      Utility           8    5    8.5  10.5

Fairy Tale Factory 1992       $15.00      Creative products   7    2    8   10.5

Interactive Storytime

Vol. 1 (CD)                   1991        $44.99         Reading process     7    5    5      8

JPuzzle 1.1                   1992        $3.00         Spatial       7    3    6.5  9

Just Grandma and Me  1992        $59.95        Reading       7    6    8.5  7

Key Words: First

Keyboarding Skills    1992    $35.00      Keyboarding        6    6    6     8

Kid Desk              1992    $49.95      Utility            9    6   9   12

Kid Works 2           1992    $59.95      Writing            7    7   7.5   11

Lexia I, II, III      1992    $250.00     Reading skills     7     5   7   10.5

Little People Farm

Creativity Kit        1992    $45.00      Creative products  7    5   8.5        8.5

Little People Main

Street Cr. Kit        1992    $45.00      Creative products   7     5   8.5  7.5

Mario Teaches Typing 1992     $30.00      Typing             7    7   8     11

Marvin the Moose      1991    $45.00      Reading process    7    3   7     8

MetroGnome's Music 1992      $35.00      Music             7    5   8.5   9

Millie's Math House           1992        $65.00         Math       8      6   9   11.5

Operation: Math Min   1987    $30.00      Spatial            6    6.5       10 

Pip                   1991    $35.00      Reading process    7    na   8.5  10

Podd                  1990    $45.00      Reading process    2    na   8     9

Point to Pictures     1991    $75.00      Special Ed         9    6    7     6

Reader Rabbit'

Ready for Letters      1992   $45.00      Reading skills     7    5    9   8.5

Reading Adventures

of Oz               1992      $35.00      Reading            7    5    8   8.5

SnapDragen1.0          1992   $49.95      Matching           7    5    8   10

Snoopy's Game Club    1992    $49.95      Puzzles            7    5    7   6.5

Stickybear Reading

Room                1992       $55.00      Reading skills     7    5   7   10.5

Stop, Look and Listen!1992    $69.95      Reading skills     6    2   5.5    6

Storybook Theatre   1992       $95.00      Writing process    7    5   7.5 11.5

Storybook Weaver    1992       $65.00      Writing process    7    5   8   10.5

SuperPrint for

the Macintosh       1992       $90.00      Utility            7    5   6.5  8

The Gingerbread Man1992       $35.00      Reading process    7    5    4    5

Time & Money Adv

of L. Dragen        1989       $45.00      Clocks & Money     4    2 7     7.5

Tommy the Turtle    1992       $29.00      Telling time       3    2  3     5.5

Woolly's Garden     1992       $65.00      Plants             6    4  8.5  10.5

Zug's Dinosaur World 1992     $35.00      Dinosaurs          7    5  8.5   9.5

Zug's Race Through

Space                1992        $35.00       Solar System          7   5  6.5   8.5

 

AVERAGE MUC:           6.56               STANDARD DEV(MUC):             1.53

AVERAGE MD:                                   3.95                   STANDARD DEV(MD):    1.68

AVERAGE EASE OF USE:                    7.37                   STANDARD DEV(EOU)    1.30

AVERAGE FEEDBACK  8.93               STANDARD DEV(FEEDBACK)    1.78

 


Much of the research presented in Part 1 of this paper was gathered as part of a larger project.  Many thanks go to those who helped gather and analyze the data for this portion of our report: Kedmon Hungwe, Jim Bunnell, Colleen Ludorf and Tom Spotts.

 

 

[2] Many of the formalized tasks originally intended for this study were not performed by the pre-school subject(s) due to their cognitive level.  However, there was still much emphasis on exploration and learning appropriate to their particular stage of development.