Doctoral Thesis and Published Papers
Doctoral Thesis
Title
An investigation into the psychometric properties of a computer based interactive arithmetic test: Estimates of reliability and validity and a comparison of cultural bias in its results with that found in the results of a written arithmetic test.
Author
Jonathan Hippisley.
Abstract
The purposes of the research described in this thesis were twofold. First, to establish whether data gathered by a specially developed computer-based interactive arithmetic test (CBIAT) could be used to create a reliable and valid measure of the elementary mathematical abilities of Australian Aboriginal and non-Aboriginal children. Second, to establish whether the data from the CBIAT showed less cultural bias against the Aboriginal children than data gathered from a conventional written arithmetic test.
A context for the research is provided by a review of the literature, which describes evidence of cultural bias in conventional attainment tests for as long as they have been used, during the current century. The literature review describes how interactive testing procedures, based on the ideas of the Russian psychologist, Lev Vygotsky, have been developed as a means of overcoming the bias observed in the data from conventional tests. It is explained that although research indicates that data from such procedures show less cultural bias than that from conventional tests, they are expensive to administer, involving personal interaction between examiners and examinees, difficult to standardise, and not widely used.
The research comprised four studies. The purpose of the First Study was to demonstrate that the CBIAT was user friendly and to show that children not only were happy to use it, but also persisted in using it for long enough to gather the required assessment data. The purpose of the Second Study was to demonstrate that the data gathered by the CBIAT, and in particular the scoring rate (defined as the number of items answered correctly per minute or cpm), conformed with the Rasch model of objective measurement. The purpose of the Third Study was to assess the reliability and validity of data gathered by the CBIAT from a large sample of primary school students.
The Fourth Study, representing the culmination of the research, had two essential purposes. The first was to confirm that the high levels of reliability and the positive validity found in the third study, whose participants were predominantly non-Aboriginal children, persisted when the CBIAT was applied to Aboriginal children. The second purpose was to demonstrate that the results from the CBIAT showed less cultural bias against the Aboriginal children than a conventional written arithmetic test. The results of the study were positive: the reliability and validity of the CBIAT were confirmed, and the results were shown to be less culturally biased than those from a written test.
The thesis concludes that the CBIAT, which is very inexpensive to administer and simple to replicate, especially compared with personally administered interactive tests, represents a practicable alternative to conventional tests for the assessment of the numeracy skills of Australian Aboriginals, who have persistently performed below the rest of the population in such tests (Stanley and Hansen, 1998). It is recommended that further research be carried out to determine whether the findings of the current research might be applied in the assessment of numeracy of other educationally disadvantaged minority groups.
Availability
The full thesis is available for viewing at the University of Western Australia.
Published Papers
Journal of Computer Assisted Learning v15, pp 41-47
Title
Student opinions on an interactive arithmetic test.
Authors
Jonathan Hippisley and Stephen Houghton
Abstract
The aim of this study was to assess student attitudes towards an interactive arithmetic test. A sample of 45 primary school children from Years 1 and 2 in two Western Australian primary schools participated in the study. After using the test for a period of six weeks students were invited to detail what they liked and disliked about the test and how they might improve it. Analysis of this written feedback revealed a generally positive attitude towards the test. There were just a few negative responses, mainly pertaining to the slow speed of the program on the computers used in the study.
Availability
British Journal of Education Technology v29, n4, pp303-320
Title
The reliability and validity of an interactive arithmetic test.
Authors
Jonathan Hippisley and Graham Douglas
Abstract
A sample of 331 primary school children from seven primary schools, from Years 3 through 7, participated in this study to determine the reliability of a computer resident interactive arithmetic test, using both the method of a single sitting and that of parallel forms. Reliability coefficients up to .96 were calculated for scoring rates by the single sitting methods and up to .87 by the method parallel forms. The study also attempted to determine the validity of the interactive test by comparing it with the arithmetic subtests of the Key Math Test (KMT, Connolly et al, 1971). Positive, but weak, correlations up to .65 were recorded, reflecting differences in the medium of administration and assessment methodology of the two tests.
Availability
Education Research & Perspectives v25, n2, pp59-67
Title
Looking at data from an interactive arithmetic test from the perspective of a probabilistic model.
Author
Jonathan Hippisley
Abstract
Since the publication of Probabilistic Models (Rasch 1960) a growing number of researchers have been advocating the use of probabilistic or stochastic models in the analysis of data from psychometric tests. The focus of interest of such researchers is usually static data from untimed static tests. The analysis of data from speeded tests usually follows the traditional deterministic methodology. This paper will refer to the chapters in Probabilistic Models which deal with the rate at which events occur; it will examine whether similar analysis might be applied to the scoring rate from an interactive arithmetic test (IT); and it will follow the reasoning of Rasch in testing whether actual data from the IT fits the model designed by him for the analysis of timed oral reading rates.
Availability
WAIER Forum 1997 Session B.5
Title
Boolean versus continuous variables as tools of measurement
Author
Jonathan Hippisley
Abstract
A Boolean variable, such as the write/wrong response to an item in a school test, represents a very crude method of measurement by the standards of natural science. This paper will consider from first principles the implications of combining a Boolean variable such as a write/wrong response to a question in an arithmetic test with a continuous variable such as the time taken to produce a correct response. The paper will also compare raw score distributions from an interactive arithmetic test with scoring rate distributions, and discuss implications for the relative reliability of the two metrics.
Availability
Educational Research, Vol. 47, No. 2, June 2005, pp. 205-215
Title
A cross-cultural comparison of numeracy skills using a written and an interactive arithmetic test
Authors
Jonathan Hippisley, Graham Douglas and Stephen Houghton
Abstract
This study applied two arithmetic tests, one written and one one computer-based interactive, to samples of primary school children from two populations, one suburban non-Aboriginal and one rural Aboriginal. The results from the written test were significantly (p50.001) better for the non- Aboriginal children than for the Aboriginal children. This was not the case with the results from the computer-based interactive test. The study used Rasch-based methodology to reduce the results from the two tests to a common scale, to ascertain whether the Aboriginal children performed better (in relation to the non-Aboriginal children) in the computer-based than in the written test. The study found that this was the case, and concluded that the results from the computer-based test exhibited less cultural bias against the Aboriginal children than the written test.