The Beginning of a Journey to Assessment and Data Literacy for Teachers

Jim Tognolini asserts that assessment is a central activity in all education and that its main function is to improve learning. He stresses that professional judgement is essential in this process and that such judgement is owned by teachers. Jim outlines how the Centre for Educational Measurement and Assessment (University of Sydney) used the findings of their research to develop professional learning modules for teachers on Assessment Literacy and Data Literacy. . .

Introduction

Whenever I talk about assessment to people, I like to make it very clear how I define assessment. This is because I think it is critical in assessment at all levels to have a shared understanding of what it is we are talking about. Too often, particularly in assessment, measurement and evaluation, we have discussions with colleagues using terms with the assumption we all have the same shared meaning. Generally, we do not.

Assessment

Assessment is a central activity in education and necessary for understanding student progress and development. It involves making judgments, based on evidence, about progress in the acquisition of the knowledge and skills required by the syllabuses and the school curriculum. It is through assessment that we understand learning is occurring, hence assessing is an essential component of classroom activity.

The definition that Gordon Stanley and I arrived at, after experimenting with different definitions over many years, is “Assessment involves professional judgement based upon an image formed by the collection of information about student performance.” (Tognolini & Stanley, 2007). As teachers, we are primary agents in assessment and we are doing it formally and informally every day. It is interesting to reflect on the relationship between the classroom assessment and the formal assessment (e.g. HSC, NAPLAN, NAP). The validity and the credibility of the formal assessments are largely determined by classroom assessments. If, for example, the results for the HSC were totally “different” from those that the teachers expected (i.e. the students who they expected to do well, based on the classroom assessments, did a lot worse; and those who were expected to do poorly did a lot better across the system) then it would call into doubt the results in the formal assessments. Going back to the definition of assessment above, assessment involves professional judgement and that professional judgement is owned on a day-to-day basis by teachers. However, the interest in assessment goes beyond the classroom. There are many other players who have a stake in assessment and in understanding what it means.

The process of assessment is generally used, at classroom level, for the purpose of monitoring performance and for understanding the learning that has taken place so the next step in the learning process can be undertaken. Thus, from an educational perspective, the main function of assessment is to improve learning. Whenever information is collected about student performance there should be an intention by the teacher to provide feedback to the students in a way that will make clear where the students are in their learning and what they have to do to progress.

When the learning goals have been achieved, the purpose of assessment is to provide evidence of that achievement. Assessment provides feedback to the student, the teacher and the education system. Feedback is an essential requirement for learning (Hattie & Timperley, 2007).

As indicated in the definition, central to the way that teachers assess is the idea of building up an image of what it is students know and can do. It is this image in a standards-referencing system that is used by teachers to build evidence to “track” and report student progress along a developmental continuum (commonly referred to as progression).

The assessment and evidence generation, and interpretation, role that teachers have is critical to the success of the teaching and learning. If the assessments are not really assessing the appropriate outcome and/or are technically unsound, then the image of what it is that students know and can do is flawed and this is not fair to the students.

Purpose of the paper

The purpose of this paper is to show how, starting with Standard 5 of the Australian Professional Standards for Teachers (AITSL, 2014), we identified the specific elaborations of assessment knowledge and understanding that teachers (at the Graduate and Proficient career levels respectively), need to be able to demonstrate in order to provide evidence that they have met the standard at the particular career level. Using the elaborations that teachers and assessment experts described as essential, we built two professional development programs: one is called Assessment Literacy for School Teachers and the other Data Literacy for School Teachers.

This paper describes how the Centre for Educational Measurement and Assessment (CEMA) at the University of Sydney, working in conjunction with the NSW Education Standards Authority (NESA) took each of the foci for Standard 5, identified the specific elaborations of assessment knowledge and understanding that teachers ( at the Graduate and Proficient career levels respectively), need to be able to demonstrate in order to provide evidence that they have met the standard at the particular career level.

Assessment and the Australian Professional Standards for Teachers

Assessment is so important to the practice of teaching that it is reflected in the Australian Professional Standards for Teachers (AITSL, 2014) https://www.aitsl.edu.au/teach/standards as a separate standard (Assess, provide feedback and report on student learning) along with its 5 foci (5.1 Assess student learning; 5.2 Provide feedback to students on their learning; 5.3 Make consistent and comparable judgements; 5.4 Interpret student data; and, 5.5 Report on student achievement).

Elaborations of Standard 5 of the Australian Professional Standards for Teachers

Starting with the foci, we delineated each of the focus areas for Standard 5 into elaborations. An elaboration is a description of the knowledge and understanding the teachers need in order to be able to demonstrate the standard.

Table 1 gives an example of this process.

TABLE 1. Knowledge and understanding needed to be able to demonstrate the graduate and proficient standards described in the Australian Professional Standards for Teachers

5.1 Assess student learning:

(click on table to view)

It can be seen from Table 1 that in Column 3, (headed “Proficient Standard Descriptor”) the Proficient standard for focus 5.1 “Assess student learning” is presented. In column 4, (headed “Elaborations of knowledge and understanding”) are a couple of the potential assessment knowledge and understanding elaborations that teachers might need to be able to demonstrate the standard descriptor presented in column 3.

TABLE 2 Distribution of elaborations by graduate and proficient career levels of Standard 5 of the Australian Professional Standards for Teachers

(click on table to view)

It can be seen from Table 2 that there were 46 elaborations of the 5 foci associated with Standard 5 at the Proficient teacher level.

Each of the elaborations was then placed in a questionnaire and presented to a small sample of a) primary teachers; b) secondary teachers; and, c) assessment experts identified by the Deans of Education at Universities.

Respondents were asked to rate each of the elaborations as “Essential”, “Desirable” or “Not Necessary” to demonstrate the standard aligned with the elaboration. Each question was framed as follows “To what extent do proficient teachers (or teachers at the Proficient career level) have to know “THE ELABORATION IS STATED” to demonstrate the specified standard.

For example, “To what extent do Proficient teachers have to know how to deliver feedback that includes suggestions for improvement e.g. skill development, content knowledge, foundational subject knowledge or skills, test technique, student health, wellbeing and motivation” to demonstrate PROFICIENT STANDARD 5.2?

After completing the questionnaires, respondents were invited to attend focus group sessions to discuss the outcomes from their responses to the questionnaire. The overall outcome from the questionnaire, and the associated focus groups, was a comprehensive pool of content that could be tailored to assessment professional development courses to provide support to teachers responsible for assessing and making evidence-based decisions on assessment data.

The content split naturally into two groups, some we called Assessment Literacy and the others we called Data Literacy.

Assessment Literacy and Data Literacy

There are numerous examples of definitions of what is meant by assessment literacy in the literature. For example, Stiggins (1991a; 1991b) defined teacher assessment literacy as knowledge about educational assessment and the skills required to apply that knowledge to measure student learning. Fullan and Watson (2000) defined it as the capacity of teachers, either on their own, or in collaboration with other teachers, to analyse and correctly comprehend student work and performance data, and, at the same time, be able to design classroom and school plans to change circumstance as needed to improve results.

We suggest, from an analysis of the elaborations that emerge from the Australian Professional Teacher Standards, that it refers to the knowledge and understanding required to be able to carry out good quality assessment practice. This includes the understanding of assessment terminology; the development and use of assessment methods and techniques; and an understanding of what is required to achieve high standards in assessment.

In essence, this means assessment literate teachers know how to write and select high-quality assessments (including Higher Order Thinking Skill (HOTS) items); know how to integrate results obtained from assessments with improvements in learning (Formative Assessment and Assessment for Learning); know how to communicate accurately about student learning; and can demonstrate data literacy.

Data literacy, on the other hand, as defined by Mandinach and Honey (2008) is “… the ability to understand and use data effectively to inform decisions. It is comprised of a specific skill set and knowledge base that enables teachers to transform data into information and ultimately to actionable knowledge (evidence)”. An analysis of our elaborations suggests that data literate teachers know how to identify, collect, organise, analyse, summarise and prioritise data. They also know how to develop hypotheses; identify problems; interpret data; convert data into evidence; and, determine, plan, implement, and monitor courses of action. These are the steps that enable teachers to take data, and turn it into evidence which can then be used to inform decision making and teaching in the classroom.

The Importance of Assessment Literacy and Data Literacy for Teachers

At the national and state levels, data are informing major educational, accountability and investment decisions. At the school level, school assessment data, (including data from large-scale testing programs such as the National Assessment Program – Literacy and Numeracy (NAPLAN), and the Higher School Certificate (HSC), underpin policy and accountability processes between school administrators, principals, teachers, students and school communities.

Arguably more important, is the use of assessment and data literacy to inform teaching, and to support learning in the classroom. The education community, including NESA, has identified an urgent need to develop teachers’ professional practice in these areas over the coming five years. In response to this need, The Centre (CEMA) has entered into a partnership with NESA. The main aim of this partnership is to provide professional learning to teachers at all levels within schools.

Learning modules have been designed for primary and secondary teachers in Assessment Literacy and these will soon be complemented by a suite of modules on Data Literacy, which is currently being designed. The Centre is planning to develop further modules for teachers in middle and senior leadership roles.

The professional learning modules in Assessment Literacy support teachers’ engagement with, and understanding of, assessment and the use of data to inform teaching and learning.

The modules in Assessment Literacy are endorsed by NESA at Proficient Teacher level, meaning that all teachers may learn more about aspects of assessment while, at the same time, achieving their hours for maintenance of accreditation. Highly Accomplished and Lead Teachers may complete these modules as ‘Teacher identified’ PD. They are competitively priced. Furthermore, the modules are offered fully online and are, therefore, available on demand and can be completed at the teacher’s own pace.

The University has assessed the professional learning modules for articulation to award programs, particularly the Master of Education programs, of which there are nine streams, including Digital Technologies, Special and Inclusive Education, Educational Management and Leadership. Information on the Master units may be found on CEMA’s website.

References:

Australian Institute for Teaching and School Leadership (AITSL), (2014). Australian Professional Standards for Teachers.https://www.aitsl.edu.au/teach/standards

Fullan, M., & Watson, N. (2000). School-Based Management: Reconceptualizing To Improve Learning Outcomes. School Effectiveness and School Improvement, 11, 453-473.https://www-tandfonline-com.ezproxy2.library.usyd.edu.au/doi/abs/10.1076/sesi.11.4.453.3561

Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81-112. http://journals.sagepub.com.ezproxy1.library.usyd.edu.au/doi/full/10.3102/003465430298487

Mandinach, E. B., Honey, M., Light, D., & Brunner, C. (2008). A conceptual framework for data-driven decision making. In E. B. Mandinach & M. Honey (Eds.), Data-driven school improvement: Linking data and learning (pp. 13-31). New York, NY: Teachers College Press

Stiggins, R. J. (1991a). Assessment Literacy. Phi Delta Kappan, 72, 534-539 https://sydney.alma.exlibrisgroup.com/discovery/openurl?institution=61USYD_INST&vid=61USYD_INST:sydney&ctx_enc=info:ofi%2Fenc:UTF-8&rft.volume=72&ctx_tim=2020-08-03T22:58:00Z&rfr_id=info:sid%2Fgale:&rft.jtitle=Phi%20Delta%20Kappan&rft.aufirst=Richard&rft.genre=article&rft.issue=7&rft.pages=534&rft.aulast=Stiggins&svc_val_fmt=info:ofi%2Ffmt:mtx:sch_svc&req_dat=info:sid%2Fgale&url_ver=Z39.88-2004&rft.date=1991-03&rft.spage=534&rft.au=Stiggins,%20Richard%20J.&rft.atitle=Assessment%20literacy&rft.issn=00317217

Stiggins, R. J. (1991b). Relevant Classroom Assessment Training for Teachers. Educational Measurement: Issues and Practice, 10(1), 7-12. https://onlinelibrary-wiley-com.ezproxy2.library.usyd.edu.au/doi/pdfdirect/10.1111/j.1745-3992.1991.tb00171.x

Tognolini, J. & Stanley, G. (2007). Standards-Based Assessment: A tool and means to the development of human capital and capacity building in education. Australian Journal of Education, 51(2), 129-145. https://journals-sagepub-com.ezproxy1.library.usyd.edu.au/doi/abs/10.1177/000494410705100203#articleCitationDownloadContainer

Professor Jim Tognolini is Director of The Centre for Educational Measurement and Assessment (CEMA) which is situated within the University of Sydney School of Education and Social Work. The work of the Centre is focused on the broad areas of teaching, research, consulting and professional learning for teachers.

The Centre is currently providing consultancy support to a number of schools. These projects include developing a methodology for measuring creativity; measuring 21st Century Skills; developing school-wide practice in formative assessment. We have a number of experts in the field: most notably, Professor Jim Tognolini, who in addition to conducting research offers practical and school-focused support.

The professional learning modules are available now at the following site: https://sydney.nicheit.com.au/education_social_work/web/workshop

During registration, teachers are required to use their individual teacher email address and create a 9-character password to access the selected module in Canvas. Payment by credit card is required to complete registration and be granted access to the course.