|
Updated 7/30/2007 by Warren Buckleitner
Tell a child that there are a billion stars in the sky,
and she’ll believe you. Tell her the paint on that park
bench is wet, and she’ll have to touch it to find out. (Adapted from an anonymous quotation).
DOWNLOADS
Software Evaluation 101 some slides from a 2006 presentation.
1/2 page Serious Games Testers Club form (for quick game reviews, used by the SGTC members at Mediatech).
See also: How We Rate Interactive Media: About the Ratings and CTR's Software Evaluation Instrument.
Please note the Creative Commons license at the bottom of this page. We think the creative commons concept works especially well for reviews and evaluation rubrics. In short, the license we've selected lets you use the ideas on this page as long as you don't alter them and you provide accurate attribution.
A COMPLEX MEDIUM
Choices abound when it comes to children’s interactive media products.
There are simulations, open-ended creativity tools,
structured activities, puzzles, tutorials and reference
titles; made by hundreds of publishers with a varying degree of knowledge about child development and programming techniques.
In addition, there are dozens of hardware options (we count over 20 as of 2007), including handheld game systems, TV toys, interactive DVDs, computers, and toy-based systems. All of this brings a great deal of potential "mush" to the job of trying to assign ratings to any title.
CTR'S "Least Worst" EVALUATION PROCESS
As said above, assigning quantitative ratings to any published product is at best an inexact science. Opportunities for bias, either theoretical or from some culturally derived source, creep in from everywhere. Confounds can include your own (or your testers) hardware familiarity, the age/developmental level of the testers, the state of the code; to name a few. An evaluation done at 2:00 AM is not likely to be as accurate as one done at the start of a day. A review must consider the claims made by an interactive media product, and then attempt to gauge the
program’s success at meeting the claims.
A GENERIC CHECKLIST FOR ONE DEFINITION OF QUALITY
Here's the form we use in our reviewer training. Each reviewer tests ten titles, and the scores are compared to a trained reviewer. The goal is to get inter-rater reliability, so that the scores are statistically significant.
The form we've used boils down 50 factors into five categories, as follows: ease of use, educational value, entertainment value, design features and overall value.
In the printed version of the form, the "Not Applicable" option can be used at will by the rater, which can create a high degree of varience in the rating. A video game that is not designed to be educational can hardly be accurately rated at a teaching tool. Likewise, the entertainment value of a math drill is not likely to be very high. That's why this is a generic instrument. It should be seen as a starting point in the review process.
Following are the criteria we use in evaluating
each program:
I. Ease of Use (Can my child use it with minimal
help?)
___Skills needed to operate the program are in range of the
child
___Children can use the program independently after the
first use
___Accessing key menus is straightforward
___Reading ability is not prerequisite to using the
program
___Graphics make sense to the intended user
___Printing routines are simple
___It is easy to get in or out of any activity at any
point
___Getting to the first menu is quick and easy
___Controls are responsive to the touch
___Written materials are helpful
___Instructions can be reviewed on the screen, if
necessary
___Children know if they make a mistake
___Icons are large and easy to select with a moving
cursor
___Installation procedure is straightforward and easy to
do
II. Childproof (Is it designed with
"child-reality" in mind?)
___Survives the "pound on the keyboard" test
___Offers a quick, clear, obvious response to a child’s
action
___The child has control over the rate of display
___The child has control over exiting at any time
___The child has control over the order of the display
___Title screen sequence is brief or can be bypassed
___When a child holds a key down, only one input is sent to
the computer
___Files not intended for children are safe
___Children know when they’ve made a mistake
___This program would operate smoothly in a home or
classroom setting
III. Educational (What can my child learn from
this program?)
___Offers a good presentation of one or more content
areas
___Graphics do not detract from the program’s
educational intentions
___Feedback employs meaningful graphic and sound
capabilities
___Speech is used
___The presentation is novel with each use
___Good challenge range (this program will grow with the
child)
___Feedback reinforces content (embedded reinforcements are
used)
___Program elements match direct experiences
___Content is free from gender bias
___Content is free from ethnic bias
___A child’s ideas can be incorporated into the
program
___The program comes with strategies to extend the
learning
___There is a sufficient amount of content
IV. Entertaining (Is this program fun to use?)
___The program is enjoyable to use
___Graphics are meaningful and enjoyed by children
___This program is appealing to a wide audience
___Children return to this program time after time
___Random generation techniques are employed in the
design
___Speech and sounds are meaningful to children
___Challenge is fluid, or a child can select own level
___The program is responsive to a child’s actions
___The theme of the program is meaningful to children
V. Design Features (How "smart" is this
program?)
___The program has speech capacity
___Has printing capacity
___Keeps records of child’s work
___"Branches" automatically: challenge level is fluid
___A child’s ideas can be incorporated into the
program
___Sound can be toggled or adjusted
___Feedback is customized in some way to the individual
child
___Program keeps a history of the child’s use over a
period of time
___Teacher/parent options are easy to find and use
VI. Value (How much does it cost vs. what it does?
Is it worth it?)
Considering the factors rated above, and the average retail
price of software, rate this program’s relative
value.
Poor...................................Good
1 2 3 4 5 6 7 8 9 10
Copyright 1983-2007 Children's Technology Review
All Rights Reserved 
|
