Ignite the Spark 2018 Conference: Ways of Assessing Giftedness

On Friday I attended the Ignite the Spark Conference at UNSW in the hope of learning some new stuff about assessment. Associate Professor David Blaazer, the Deputy Dean for UNSW Arts and Social Sciences, opened up the conference after a reflective Welcome to Country delivered by a representative of the Dharawal people, and this led into an address from DoE Director of Secondary Education Lila Mularczyk, who spoke enthusiastically about the conference's role in highlighting the need to tie our assessment practice to current policy. 

Assessment is a funny beast though.

All teachers do it so it's not KLA specific, and the need for Professional Development around assessment can sometimes get relegated elsewhere due to its non-denominational nature. With this in mind, it's always interesting to see who shows up to these kinds of conferences. Suffice too say - there were a lot of teachers here and the presenters represented many different sectors of the education system - from primary to secondary, private to public, and including counsellors, psychologists, NESA representatives, DoE corporate, and other agencies who help care for our kids.  

The keynote address (which this blog focuses on) was delivered by Dr. Jae Yup Jared Jung, and focused on his research relating to assessment in gifted education, which represented a systematic review he has published in the Journal of Advanced Academics (28, 163-203). NESA considers 'differentiation' to include adjusting and modifying assessment activities in order to cater for both individual students and larger groups, and the idea of assessing students to locate giftedness is accordingly mandated. It should be noted, however, that there is no real standardisation for catering to the gifted in NSW and this is why Dr Jung's research is so incredibly relevant.

Dr. Jung demonstrates what a non-verbal ability test looks like.

The Research

In his search of research literature from 2005 to 2016, Dr. Jung essayed each of the major academic databases and the reference lists of retrieved articles found in the top research journals. By doing this, he found the following:

• 148 articles in total that dealt with the topic of assessment in gifted education.
• A clear majority within this of 128 articles that deal with identification of gifted students.
• Only 14 articles that covered evaluations of gifted programs.
• And just 8 articles that deal with learning growth. 

The majority of the presentation went on to cover the ways that gifted students are identified (since this is where the majority of research focuses). Dr Jung identifies two major categories of common assessment: Objective (or Traditional) assessments, which are quantitative and result in a number used to identify ability, and Subjective (or Non-Traditional), which involve qualitative judgements made by teachers, psychologists, counsellors, parents, etc. 

Objective Assessments

1. IQ Tests

These involve problem solving, utilising logic, and demonstrating recognition of patterns or relationships. Examples of this kind of testing includes the Wechsler Intelligence Scale for Children (WISC-V), the Stanford Binet 5 test, the Kaufmann Brief Intelligence Test, the Cognitive Abilities Test (CoGAT), the Australian Council for Educational Research General Abilities Test (AGAT), Raven's Progressive Matrices (RPM), and the Naglieri Nonverbal Ability Test II (NNAT-II).   

Benefits: The WISC-V is designed to measure five different areas (verbal comprehension, fluid reasoning, visual spatial ability, working memory, and processing speed) and therefore allows for the recording of strengths and weaknesses in said areas. The Kaufmann Brief Intelligence Test is an abbreviated IQ test that can be given by teachers rather than psychologists, and the CoGAT and AGAT can be administered in groups rather than individually (which is practical for schools). The Raven's and Naglieri methods are nonverbal ability tests that assess students through the use of visual-spatial stimuli and can be useful because they reduce the reliance on a student's prior educational history or ability to speak English as a first language (or at all).

Concerns: The WISC-V and Stanford Binet 5 tests must be administered by appropriate accredited professionals (psychologists) and are designed for 1:1 testing. These can therefore be quite expensive; $1000-2000 per individual student.

Quote: "Aw, c'mon! Tell me which Poplar Tree is the tallest!!!" The Window Effect refers to the idea that if we assess whether students can achieve to a certain point then we can't see what they could do beyond this point.

2. Above Level Tests

Achievement tests designed for older students in higher grades, with 'higher ceilings' (see cartoon above). These tests are difficult for the majority of testees to 'top out', and assist in the search for talented individuals whose abilities may go beyond the scope of most assessment tasks. Examples include American-style examinations such as the SATs, ACTs, and EXPLORE tests. 

Benefits: Allows for the identification of outliers: the small percentage of people whose ability does not fit on the regular scale of assessment. This style of testing is also designed for specific age groups, so it allows for more targeted assessing of students.

Concerns: Results can be disheartening for the majority of students who undertake the test, so this data usually needs to be kept confidential and for teacher purposes only.

3. Computer Adaptive Tests

Online testing instruments that adapt questions to suit student ability, and scale each question based on prior performance in the same test so that the student is constantly working within their zone of proximal development. Abbreviated as 'CATs'.

Benefits: Can be a very efficient way to gather data as the computer generates reports automatically. It also differentiates the test for each individual student, which means no two tests are the same.

Concerns: CATs can be calibrated too closely to a 'pass-fail' dichotomy that doesn't allow for a diverse data set. These tests can also only measure as far as the student goes - when a student or the test finishes we may still not know what the student is truly capable of.

Subjective Assessments

1. Nominations

Having 'gifted' students nominated for recognition allows for us to consider the unique perspectives of various groups familiar with gifted students. One such group are parents, who are a subjective source of information but can also provide lots of information that we don't or can't have access to otherwise, such as milestones like the moment the child first spoke or formed a sentence.

Benefits: Nominations allow for assessment of different types of giftedness, and can also be a useful way of incorporating the input of a student's individual teachers (therefore making them part of the consultative process, which is useful and supportive in the formation of any kind of school-based gifted program).

Concerns: This process relies on human judgement and therefore lacks the psychometric rigour of I.Q. testing and similar methodologies.

2. Performance-Based Assessment

These are assessments that look more like subject-specific assessment tasks. This style of assessment could be: asking students to act in a prescribed way to create a product or a response, such as a written composition, or a teacher observing students in problem-based learning activities.

Benefits: The familiarity of this kind of testing will appeal to teachers already used to gathering data in this way. This method is also fairly open-ended and can elicit a wide range of results.

Concerns: There may be a lack of clarity in how it should be scored as each teacher will mark to the criteria in their own particular way. It can also be more time-consuming than objective methods of assessment as each individual student requires an individualised evaluation of their work.

3. Dynamic Assessment

Of all the things Dr. Jung spoke about in his keynote, I found his discussion of Dynamic Assessment the most interesting. I probably found this the most interesting thing about the whole conference and I can see myself looking forward into it in the immediate future.

Dynamic Assessment is a process that involves interaction between the teacher and the student, with a focus on how the student responds to educational interventions. It's not about the student's response to the task but about measuring how a student improves or reacts to a task. There are two main kinds: 

• The first is the 'Sandwich Format', in which a student is pretested, given an intervention, and then post-tested. In the post-test the level of improvement in the student is measured in comparison tot he pretest. 
• The second kind is the 'Cake Format', in which a student is given a task with a series of items and assistance is provided only when the student encounters difficulties. The amount of instances, or types of intervention, are then measured to show how much growth the student exhibited, or how much/little help they needed in independently adapting to the demands of the task.

Benefits: This process deliberately acknowledges the inequality of educational opportunity among students - as each student comes to us with their own unique context this means that their giftedness may have been either nurtured in the past or completely ignored. The result is that there isn't an even playing field when it comes to trying to 'see' the natural level of giftedness in a student. Using Dynamic Assessment minimises the role of performance in previous assessment tasks as this form of assessment focuses on observing the student's capacity in adapting to difficulty.

Concerns: A high degree of standardisation is needed when training teachers to administer this process of assessment. As such it can be time-consuming, resource and staff-intensive, and involves a lot of effort.

Dr. Jung

Further Observations of Dr. Jung

1. Multiple Criteria Identification

With so many different methods of assessment available to identify gifted students the consensus seems to be that multiple criteria should be used together. This allows for the minimisation of bias and multiple opportunities for students from diverse backgrounds to demonstrate giftedness. So, with this in mind, Dr. Jung asserts that we should use both objective and subjective methods. Some other things to take into account:

• We need to ensure that we're using instruments appropriate to our context - this means look at current student numbers, demographics of the region, etc.
• Identification needs to take place as early as possible and there should also be training provided for the teachers identifying the students.
• If using multiple criteria, the data should be collected and analysed concurrently so no students get left behind.

2. Disadvantaged Groups in Gifted Programs

Dr. Jung also expressed several concerns in relation to the under-representation of students from disadvantaged backgrounds. He posited that this happens due to several factors:

• Inappropriate/unfamiliar content in assessment tasks.
• Pre-defined conceptions of giftedness could be inherent in assessment tasks.
• Pre-conceptions of teachers could lead to unconscious bias.
• Socio-economic factors.
• Lack of a mechanism for universal assessment of gifted students in NSW.

This last point is a big issue with giftedness in general. Not all schools test for giftedness and the ones that do may do so in very different ways, with specific ideas of what giftedness means. Many schools have 'self-select', 'opportunity', or 'academic-focus' classes but the criteria used to create these classes often have little to do with natural giftedness and more to do with prior performance.

Nevertheless, as we move forward into the 21st century, I have strong hopes that NSW will inch its way closer to some kind of standardised mandate to identify and better support those who may be hiding in plain sight.

Dr. Jung's research can be found here, but you will need access to academic journals in order to read it in full.

Paperman and Assessment FOR Learning

One of my favourite things about the new Australian curriculum is the clarification of assessment into the categories 'as', 'of', and 'for'. It takes the summative/formative dichotomy that extra step further and puts the emphasis back onto the process of learning rather than just the end point.

I know, right, I'm an exciting guy. 

For each of my classes I try to build in some informal diagnostic assessment (assessment for learning) that lets me know where each of my students are at. There are a few reasons for doing this:

1. Know Your Students. It helps me get to know my students - if I can't see how they respond to certain kinds of activities then I won't be of much help to them as the topic progresses.    
2. Structured Teaching. As an individual, I respond well to structure, so having a process that puts students through a standardised task appeals to me.
3. Teaching and Learning Cycle. Pre-test, diagnostic task, or assessment for learning - whatever you want to call it - this stage of the Teaching and Learning Cycle allows me to collect data that can be used as a starting point for each term. It also informs my reporting; writing reports is always easier when I have a bunch of figures in front of me that I can line up against my personal impression of each student. The more figures I have, the closer I'll get to accurately representing their true ability.
4. Differentiation. The push at most schools right now is for differentiated curriculum, something that has grown as a response to the mixed ability classroom and as a way to support teachers in catering for the widest possible array of learners. A great way to establish loose streams of 'Adjusted', 'Core' and 'Extension' students within the classroom is to use a diagnostic test at the start of a unit. 
5. Student Response. Presenting work as an informal assessment seems to make the kids sit up a little straighter in class. What I mean is, students take this work a little more seriously than most student-centered work I put down in front of them. In addition, I always try to make it clear why they're doing each informal assessment by linking it to skills that they will need for upcoming formal assessment tasks.

An effective assessment for learning should connect to what I'm looking to assess towards the end of the same unit. What I plan to assess should inform every part of the unit (backwards mapping yo!), so this is always my starting point. In this case the plan is to have students write an exposition piece in which they persuade their audience. The argument they have to make is: why Tim Winton's Blueback should win the 'Novel of the Year' award.

I actually jumped into this blog post with a specific activity and resource in mind, so now that I've gotten all the context out of the way I can do what I came here to do: talk about Paperman.

Paperman is a fantastic and dialogue-free award-winning animated film by Disney. I use it as a diagnostic text that can help me quickly check on the students' analytical and persuasive writing skills before we do Blueback for the rest of the term.

First, we watch the film. As there's no dialogue and it only goes for five minutes, I think you'll be hard-pressed to find many kids who can't get into it. While watching, the students can start collecting some ideas about their impressions of the film by filling in this sheet: Thinking About Paperman. Have them watch the film at least twice and model at least one idea on the board so they can see the sort of things they can write. You can even got a step further and have the whole class contribute opinions and make a big class example on the board so students can fill their sheets up nice and full.

The next step is, in the next lesson, have your students sit the diagnostic test. This should roughly take half a lesson. Give your students as long as they want though, if a Year 7 student has it in them to spend the whole lesson writing their responses up, then give them that time. This is their chance to show what they're capable of. I even let some students take the sheet home and complete it there if they're that way inclined - I know I can't always focus in a noisy room of about 30 people, so if a student wants to do well by completing the task in a quieter environment then who am I to stand in their way?

Here's the diagnostic test: "Here I am!" It breaks down as follows.

Section 1 is a series of True or False questions that test general comprehension. It's important to allow a section that doesn't require much in the way of writing because if we want to assess comprehension then we shouldn't let a student's inability to write get in the way of that. Writing and comprehending are two completely separate skills. Sometimes we have to make sure that we're assessing one thing at a time. Plus it never hurts to let students succeed in English without having to be great writers, especially at a Year 7 level when we want to encourage engagement with English as much as possible and set them up for a long-term journey.

Section 2 tests vocabulary through the use of a 'best fit' cloze passage. The word bank includes more than one correct answer for each missing word, however, students are asked to pick the best word to complete each statement. This lets the teacher see how far the students' vocabulary extends.

Section 3 is where the actual mechanics of writing are looked at. This means articulating ideas, showing an ability to elaborate on ideas, structuring writing into paragraphs, writing for an audience, and showing an ability to evaluate. Very few students will be able to achieve full marks in this section, and the criteria has been intentionally 'amped' up so that we can avoid the students 'topping out'*. It needs to be difficult because if we're not looking for that tiny percentage of students who can do these really sophisticated and unexpected things, then we're never going to find those students in the first place.

Here's the marking criteria/answers: "Hello there!"

One more thing before I finish up; if you do decide to use this diagnostic assessment and collect some data it's worth keeping in mind that this information is just for you, the teacher, rather than for the students and their parents. I don't hand the papers back with the results because I don't think it's particularly useful for the students. It's not the sort of assessment that students have a chance to study for so it wouldn't be fair to give marks back to them. Plus it could be potentially demoralising to give students access to marks accrued in a test that has been deliberately 'stretched out' to accommodate the largest possible range of student ability. If any students ask for their marks (and very few do, because I don't use the word 'assessment' when I talk to them about this task) then I would simply tell them that the information was just for me and that the marks don't count towards their reports.

*By 'topping out' I mean that I don't want every student meeting the criteria too easily and getting full marks. If this happened then it would mean that I'm not extending my measurements far enough to see where their abilities end. This is another good reason why the students shouldn't be given their marks back: only the most gifted will achieve 'A-range' results.

Extending Stage 4 Students in English

Blueback, a novel that I highly recommend for Year 7 students - it's both easy to read and possesses depth of language/ideas.

In addition to my role as classroom teacher, I also work at my school as the Gifted and Talented Co-ordinator. This has predominantly meant the implementation of a class-sized pull-out program for identified gifted students (more on this at a future date), however, before I got to that point I piloted an experimental phase in which individual students with potential giftedness were targeted. At the time (2 years ago), I was teaching a mixed-ability Year 7 English class. There were two students in particular in this class who showed classic signs of advanced ability and I wanted to give them a meaningful opportunity to use and stretch this ability. 

It's worth noting that these were the sort of students who would not initially present as gifted upon first impression but, after extensive diagnostic testing, I felt that I had confirmed that their potential had gone unrecognised. Before I could create a larger program that would cater to students such as these and, indeed, develop a system of testing that would establish levels of giftedness amongst their entire cohort, it would be necessary to pilot a smaller program of increased support that would prove the validity of specifically catering for gifted students. 

So now we get to the real point of this blog entry, extending these two Year 7 students with activities based on the current topic that was running during class-time. In line with the requisite academic research, I decided that the work had to be:

• Relevant to what was being studied. 
• Not a simple case of 'extra' work.
• The kind of work that would challenge these students enough to allow them to devote the same amount of quality time that any other student would spend on core activities back in the mixed classroom. 

Anything less would risk student disengagement, the same sort of thing that led to these kids 'flying under the radar' in the first place.   

The topic in question back in 2014 was a novel study of Tim Winton's beautiful ecological story Blueback. The resource I've attached below was designed to be used by tutors who had been assigned to the targeted students, but, really, it can be used in any context.

In Stage 4, annotating texts is probably something that most students would not be completely comfortable with. I usually introduce it to Year 9 students, which put me in mind of using it with gifted Year 7s as it would be a step-up for them. The style of annotation featured in the resource makes use of a few higher order skills:

• The annotating of techniques is a form of attribute-listing that requires students to taxonomise information in their heads.
• The second part of the activity, in which students identify their own techniques, contains elements of backwards-engineering. 
• Explicitly teaching the skill of annotating equips gifted students with a method that they can use to design their own systems for note-taking.

I've resisted explaining the actual mechanics of the resource in any great detail because it's all on the sheet below. If you have any specific activities that you like to use to extend junior students, please feel free to drop me a comment below - I'm always on the look-out for more ideas!

Resource: Extending Stage 4 Students in English