On Friday I attended the Ignite the Spark Conference at UNSW in the hope of learning some new stuff about assessment. Associate Professor David Blaazer, the Deputy Dean for UNSW Arts and Social Sciences, opened up the conference after a reflective Welcome to Country delivered by a representative of the Dharawal people, and this led into an address from DoE Director of Secondary Education Lila Mularczyk, who spoke enthusiastically about the conference's role in highlighting the need to tie our assessment practice to current policy.
Assessment is a funny beast though.
All teachers do it so it's not KLA specific, and the need for Professional Development around assessment can sometimes get relegated elsewhere due to its non-denominational nature. With this in mind, it's always interesting to see who shows up to these kinds of conferences. Suffice too say - there were a lot of teachers here and the presenters represented many different sectors of the education system - from primary to secondary, private to public, and including counsellors, psychologists, NESA representatives, DoE corporate, and other agencies who help care for our kids.
The keynote address (which this blog focuses on) was delivered by Dr. Jae Yup Jared Jung, and focused on his research relating to assessment in gifted education, which represented a systematic review he has published in the Journal of Advanced Academics (28, 163-203). NESA considers 'differentiation' to include adjusting and modifying assessment activities in order to cater for both individual students and larger groups, and the idea of assessing students to locate giftedness is accordingly mandated. It should be noted, however, that there is no real standardisation for catering to the gifted in NSW and this is why Dr Jung's research is so incredibly relevant.
 |
| Dr. Jung demonstrates what a non-verbal ability test looks like. |
The Research
In his search of research literature from 2005 to 2016, Dr. Jung essayed each of the major academic databases and the reference lists of retrieved articles found in the top research journals. By doing this, he found the following:
- 148 articles in total that dealt with the topic of assessment in gifted education.
- A clear majority within this of 128 articles that deal with identification of gifted students.
- Only 14 articles that covered evaluations of gifted programs.
- And just 8 articles that deal with learning growth.
The majority of the presentation went on to cover the ways that gifted students are identified (since this is where the majority of research focuses). Dr Jung identifies two major categories of common assessment: Objective (or Traditional) assessments, which are quantitative and result in a number used to identify ability, and Subjective (or Non-Traditional), which involve qualitative judgements made by teachers, psychologists, counsellors, parents, etc.
Objective Assessments
1. IQ Tests
These involve problem solving, utilising logic, and demonstrating recognition of patterns or relationships. Examples of this kind of testing includes the Wechsler Intelligence Scale for Children (WISC-V), the Stanford Binet 5 test, the Kaufmann Brief Intelligence Test, the Cognitive Abilities Test (CoGAT), the Australian Council for Educational Research General Abilities Test (AGAT), Raven's Progressive Matrices (RPM), and the Naglieri Nonverbal Ability Test II (NNAT-II).
Benefits: The WISC-V is designed to measure five different areas (verbal comprehension, fluid reasoning, visual spatial ability, working memory, and processing speed) and therefore allows for the recording of strengths and weaknesses in said areas. The Kaufmann Brief Intelligence Test is an abbreviated IQ test that can be given by teachers rather than psychologists, and the CoGAT and AGAT can be administered in groups rather than individually (which is practical for schools). The Raven's and Naglieri methods are nonverbal ability tests that assess students through the use of visual-spatial stimuli and can be useful because they reduce the reliance on a student's prior educational history or ability to speak English as a first language (or at all).
Concerns: The WISC-V and Stanford Binet 5 tests must be administered by appropriate accredited professionals (psychologists) and are designed for 1:1 testing. These can therefore be quite expensive; $1000-2000 per individual student.
 |
| Quote: "Aw, c'mon! Tell me which Poplar Tree is the tallest!!!" The Window Effect refers to the idea that if we assess whether students can achieve to a certain point then we can't see what they could do beyond this point. |
2. Above Level Tests
Achievement tests designed for older students in higher grades, with 'higher ceilings' (see cartoon above). These tests are difficult for the majority of testees to 'top out', and assist in the search for talented individuals whose abilities may go beyond the scope of most assessment tasks. Examples include American-style examinations such as the SATs, ACTs, and EXPLORE tests.
Benefits: Allows for the identification of outliers: the small percentage of people whose ability does not fit on the regular scale of assessment. This style of testing is also designed for specific age groups, so it allows for more targeted assessing of students.
Concerns: Results can be disheartening for the majority of students who undertake the test, so this data usually needs to be kept confidential and for teacher purposes only.
3. Computer Adaptive Tests
Online testing instruments that adapt questions to suit student ability, and scale each question based on prior performance in the same test so that the student is constantly working within their zone of proximal development. Abbreviated as 'CATs'.
Benefits: Can be a very efficient way to gather data as the computer generates reports automatically. It also differentiates the test for each individual student, which means no two tests are the same.
Concerns: CATs can be calibrated too closely to a 'pass-fail' dichotomy that doesn't allow for a diverse data set. These tests can also only measure as far as the student goes - when a student or the test finishes we may still not know what the student is truly capable of.
Subjective Assessments
1. Nominations
Having 'gifted' students nominated for recognition allows for us to consider the unique perspectives of various groups familiar with gifted students. One such group are parents, who are a subjective source of information but can also provide lots of information that we don't or can't have access to otherwise, such as milestones like the moment the child first spoke or formed a sentence.
Benefits: Nominations allow for assessment of different types of giftedness, and can also be a useful way of incorporating the input of a student's individual teachers (therefore making them part of the consultative process, which is useful and supportive in the formation of any kind of school-based gifted program).
Concerns: This process relies on human judgement and therefore lacks the psychometric rigour of I.Q. testing and similar methodologies.
2. Performance-Based Assessment
These are assessments that look more like subject-specific assessment tasks. This style of assessment could be: asking students to act in a prescribed way to create a product or a response, such as a written composition, or a teacher observing students in problem-based learning activities.
Benefits: The familiarity of this kind of testing will appeal to teachers already used to gathering data in this way. This method is also fairly open-ended and can elicit a wide range of results.
Concerns: There may be a lack of clarity in how it should be scored as each teacher will mark to the criteria in their own particular way. It can also be more time-consuming than objective methods of assessment as each individual student requires an individualised evaluation of their work.
3. Dynamic Assessment
Of all the things Dr. Jung spoke about in his keynote, I found his discussion of Dynamic Assessment the most interesting. I probably found this the most interesting thing about the whole conference and I can see myself looking forward into it in the immediate future.
Dynamic Assessment is a process that involves interaction between the teacher and the student, with a focus on how the student responds to educational interventions. It's not about the student's response to the task but about measuring how a student improves or reacts to a task. There are two main kinds:
- The first is the 'Sandwich Format', in which a student is pretested, given an intervention, and then post-tested. In the post-test the level of improvement in the student is measured in comparison tot he pretest.
- The second kind is the 'Cake Format', in which a student is given a task with a series of items and assistance is provided only when the student encounters difficulties. The amount of instances, or types of intervention, are then measured to show how much growth the student exhibited, or how much/little help they needed in independently adapting to the demands of the task.
Benefits: This process deliberately acknowledges the inequality of educational opportunity among students - as each student comes to us with their own unique context this means that their giftedness may have been either nurtured in the past or completely ignored. The result is that there isn't an even playing field when it comes to trying to 'see' the natural level of giftedness in a student. Using Dynamic Assessment minimises the role of performance in previous assessment tasks as this form of assessment focuses on observing the student's capacity in adapting to difficulty.
Concerns: A high degree of standardisation is needed when training teachers to administer this process of assessment. As such it can be time-consuming, resource and staff-intensive, and involves a lot of effort.
 |
| Dr. Jung |
Further Observations of Dr. Jung
1. Multiple Criteria Identification
With so many different methods of assessment available to identify gifted students the consensus seems to be that multiple criteria should be used together. This allows for the minimisation of bias and multiple opportunities for students from diverse backgrounds to demonstrate giftedness. So, with this in mind, Dr. Jung asserts that we should use both objective and subjective methods. Some other things to take into account:
- We need to ensure that we're using instruments appropriate to our context - this means look at current student numbers, demographics of the region, etc.
- Identification needs to take place as early as possible and there should also be training provided for the teachers identifying the students.
- If using multiple criteria, the data should be collected and analysed concurrently so no students get left behind.
2. Disadvantaged Groups in Gifted Programs
Dr. Jung also expressed several concerns in relation to the under-representation of students from disadvantaged backgrounds. He posited that this happens due to several factors:
- Inappropriate/unfamiliar content in assessment tasks.
- Pre-defined conceptions of giftedness could be inherent in assessment tasks.
- Pre-conceptions of teachers could lead to unconscious bias.
- Socio-economic factors.
- Lack of a mechanism for universal assessment of gifted students in NSW.
This last point is a big issue with giftedness in general. Not all schools test for giftedness and the ones that do may do so in very different ways, with specific ideas of what giftedness means. Many schools have 'self-select', 'opportunity', or 'academic-focus' classes but the criteria used to create these classes often have little to do with natural giftedness and more to do with prior performance.
Nevertheless, as we move forward into the 21st century, I have strong hopes that NSW will inch its way closer to some kind of standardised mandate to identify and better support those who may be hiding in plain sight.
Dr. Jung's research can be found here, but you will need access to academic journals in order to read it in full.
No comments:
Post a Comment