Key Ideas in Validity & Reliability for Teachers

Assessment and Measurement in Teaching: Professor Patty LeBlanc
Part 1: http://www.youtube.com/watch?v=IF-oeuidRuU
Part 2: http://www.youtube.com/watch?v=C3Zc8g9BwKg 

          After watching the videos, I have better understanding about validity and reliability. The following is the summary and important points that I got from the videos. 

          According to Dr. Patty LeBlanc, the two key questions in assessment are validity and reliability.

(i) Validity: Does this test measure what it supposes to measure?

(ii) Reliability: Does this test consistently measure what it supposes to measure?

      Validity is a more important concept for classroom-based test and in education measurement (standardized test). It concerns whether or not a test measure what it claims to measure. "Does this test measure what was taught and learn?" There are 3 basic ways to determine validity on a test, which are (i) content validity (Does the test measure what was taught?), (ii) construct validity (Does the test measure the characteristics/quality/construct that is designed to measure?) and (iii) criterion/predictive validity. 

          On the other hand, reliability deals with consistency measurement. E.g. It involves giving same tests over and over again to different individuals OR a test given multiple times to the same individual --> then take measurement --> average --> determine consistency of measurement. 

How to determine the reliability of a test?

Take multiple measures of the test to determine consistency and mathematically express consistency is the number between 0-1. The higher the number (score), the greater the reliability/consistency.

Factors that influence reliability

(i) The number of subjects (people) that are tested. 

The higher the number, the more accurate the reliability score would be. 

(ii) The number of items on a test. One essay is not enough to measure everything that are covered in a course. Generally, 30 items are recommended for assessment of knowledge or skills. 

 * Relationship between Validity and Reliability * 

If a test is VALID, it will be RELIABLE!

A test may be RELIABLE but can still NOT be VALID.

The picture below can clearly illustrates the relationship. 

           (A)                                            (B)            

Accurate = VALID; Precise = RELIABLE

Picture A: Precise/Consistent, Not Accurate (Reliable, Not Valid)

Picture B: Accurate, Precise (Valid, Reliable)

NOTE: The darts are consistent for both pictures. Meaning that

if the darts are CONSISTENT, they can be ACCURATE or NOT ACCURATE. 

However, if the darts are ACCURATE, they will be CONSISTENT. 

Therefore, if a test is VALID, it will be RELIABLE.

If a test is RELIABLE, it can be VALID or NOT VALID.

Different Learners, Different Abilities, Different Learning Styles --> SAME TEST

"For a fair selection, everybody has to take the same exam: please climb that tree."

          Is it still considered fair if you ask a fish or an elephant to climb a tree (which is impossible for them) and include monkey as one of the participants? The result is obvious, only the monkey is able to climb the tree. So, what about the rest of the animals? Failed the test??? What if you ask the animals to swim? The result will be different! Every animals have their own abilities. The ability to climb a tree does not reflect their actual ability. 

"Everybody is a genius, but if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid."

-Albert Einstein

    Same to human beings also. Different students have different abilities and learning styles. So, in my opinion, it's indeed unfair to judge everyone's abilities on a particular exam. However, due to our educational system, every students must sit for a standardized exam. Actually,  if a student perform badly in a test, it doesn't mean that he or she is "not good". Meanwhile, the one performing well in the same test is not always good in everything. This is because some people can do well in exam conditions whereas others can present their knowledge better orally. This is why I think it's not fair to assess or measure students' performance using a particular test only.

Thought on Youtube Video Watched (What was Significant to Me in the Assessment Used)

Comprehensive Assessment - An Overview

READ

REMEMBER

REGURGITATE

     While watching the video, I came across the three terms above and of course, we are too familiar with these terms. This was what we as 90's kids did during our primary and secondary educations. We sat for innumerable quizzes, exams and tests. We read, remembered and regurgitated the facts and throw them out during tests. Sometimes, we didn't even have the time to digest the new information. After tests, we forgot everything. So seriously, there is nothing much that we can learn from memorising factual information. We couldn't apply them in real world situation! What teachers expect from students is not memorising skills but other skills or abilities such as communication and leadership skills and also ability to collaborate with each other. 

     As what I have seen from the video, the implementation of the alternative assessment (performance-based assessment) in education involves students to develop performances, where they will create products and the teacher will access and evaluate their performances based on certain scoring guide. In my point of view, this is indeed a good approach for teachers to test students' abilities and knowledge. Through projects, teacher can access students' understanding on particular subjects and their capability in applying certain concepts. Moreover, performance assessment emphasises in depth learning and less focuses on drills. This is what we need in 21st century education. Traditionally, teachers could not test leadership skills, problem solving skills and etc in students through black and white paper. However, these vital these skills could be assessed through performance-based practice in classroom, where students will do their project independently, with teacher as facilitator and perform whatever they know. 

          

     One of the part of video that caught my attention is when Professor Linda Darling-Hammond from Stanford University School of Education mentions "the time is teaching and learning". Actually, by conducting performance-based assessment, it will consume a lot of time and energy. Nevertheless, while conducting the assessment in real life, students are actually learning and teachers can give immediate feedback on what to do to meet students' needs. 

Feedback on Anderson's Article: Three Things I liked about the Article

          Firstly, I liked the article by Anderson because she reminds me that there is a need to shift from traditional assessment to alternative assessment practices in education nowadays. I agree with her point that traditional lecturing causes students to build castle in the air or not paying attention in the classroom by doing something else. On the other hand, the shift from traditional to alternative assessment is crucial so as to facilitate active language learning. This is because traditionally, evaluation of student learning is based on objective questions, which inhibits active learning and high thinking skills of students. Significantly, in this high-paced world and modern education, obviously, objective questions are not adequate and relevant to measure students' performance.  

          Other than that, I liked this article as it highlights the differences between traditional and alternative assessment. The clear and detailed explanations by Anderson make the philosophical beliefs and theoretical assumptions of the assessments more comprehensible and interesting. Besides that, the comparison of both the assessments is summarised in a diagram, which is much simpler and comprehensive. 

        Another thing that I found interesting about this article is the issues related to rubrics, under the effects of alternative assessment paradigm (constructivist). Anderson has mentioned that in constructivist classroom, students are required to establish rubrics and develop criteria. However, it would be time consuming. Therefore, I think that it would be challenging for teachers to decide whether they want their students to gain knowledge or to learn how to learn.