Negative scoring (10/21/97)

There have been heated complaints on the comment board, which reflect students' ill-feeling toward negative scoring scheme. I include below several comments taken from the comment board, illustrating some points and counterpoints involved. My special thanks to Marcus' insightful comments. My general outlook is the following.

The grading scheme used by the Homework Service of Professor Moore and his colaborators is similar to those used in the Standardized Test, such as SAT, GRE, etc. They all involve negative scoring.

I have since looked into an alternative computer algorithm for the multiple choices which does not involve negative scoring. Let me use a simple example to explain how it works. Suppopse, the exam has a maximum of 100points. There are 10 questions. Each question has 10 multiple choices with one choice being the correct answer. It assigns

• 10 points to the correct choice
• 0 point to anyone of the incorrect choices
• 1 point for a blank answer.

The last case is an essential part of the computer algorithm, so that a student will not bother with doing random guesses. Here the computer has already credited a student with an average score for a random guess. There are two important points here.

• It is intrinsic to a multiple-choice-type of question, that there should be a distinction between "no-answer" and the "wrong answer". "wrong answer" is not as good as "no-answer".
• Compared to the negative-scoring algorithm, for our present example, the average raw score of the class is now shifted upward by 10 points. Since the scaled-score distribution is a measure of relative standing of students in the class, it should not be effected by this overall shifting. Thus I expect that the distribution based on non-negative scoring should be similar to that based on the negative-scorring scheme.

A more important issue is on the "no-partial-credit policy" associated with the the computerized exam. We are working hard toward giving fair computerized exams. Our guideline is that we put more emphasis in testing student's understanding of physics, logical reasoning and analytical skills. We put less emphasis on problems which involve many steps.

Several concrete steps we have taken come to mind are:

• Our use of IQ questions help to steer students in the direction of doing qualitative and analytical reasoning.
• We have switched over to textual multiple choice dominant format
• In the past two exams, most of the questions involve very few steps.

Certainly we welcome students' input, especially suggestions which will help us to develop better computerized exam problems. Also, I refer you to Marcus' comments below, and to my comments of 10/16/97 on the related issues.

Lastly, please take a look at my suggestion of 10/19/97 on "An exercise which may help you to improve your standing in this class". Do continue to use the comment board to air your concerns. Thank you. --= C. Chiu

• Comment board text: Friday, October 17, 1997 at 11:02:41 (CDT)
• Comment board text: October 17, 1997 at 11:47:45 (CDT)
• Comment board text: Friday, October 17, 1997 at 20:42:33 (CDT)
• Marcus's comments on negative scoring

Hello Dr Chiu, I am a student in your phy303k class and i have a question to ask you. isnt it true that when we get an answer wrong on the test, we not only lose points for that but we also get negative marking for guessing.

i am sorry if i sound rude, but i dont find that logical, honestly speaking. the reason is that when we have 8-10 choices to pick from, it is highly improbable that anyone would guess. it would be close to impossible to guess the right answer. if there were only 3-5 choices then it would be reasonable, but 10 choices and guessing. i cant even imagine that. dr chiu, i think we should be penalized for guessing, not for answering wrong. if our math is wrong or we have made a silly mistake in any way, we should lose points for that. but penalizing us for getting a wrong answer is merely encouraging us to guess, as we will be penalized for guessing anyways, whether we guess or not.

i worked out all the problems on the test, however my math was not completely accurate. i got killed because of that. my grade is so low that i feel ashamed to be in your class. dr chiu, please dont punish us for doing incorrect math. punish us only for guessing. in my opinion, if that matters please change it so that if we have 5 or less choices, only then should we lose credit for guessing. not otherwise. thank you an honest student.

Comment board text: October 17, 1997 at 11:47:45 (CDT)

I totally agree with the honest student. What kind of policy is assigning negative points to wrong answers? I, along with many others, think it is totally ridiculous. Maybe I'm missing the point, is there some kind of reasoning behind this policy? As if the tests aren't hard enough without losing additional points for attempting questions that you are unsure of. Have you ever heard of an educated guess?

In addition, I don't think it's fair to expect us to do five hours of homework a week, along with about 10-15 hours of studying for the test. We should not have homework due the same week we have a test in your class. The work load is just too much. I don't mean to sound like I'm whining, and I understand this is a fast-paced class, but it's just too much to ask. We do have other classes that require just as much if not more studying than Physics.

Please also try to cover homework concepts before Friday. The homework is due Saturday, and you don't even cover everything until the day before it is due. The homework is also rarely representative of what we cover in class. Some of the questions are outright crazy. Please take this into consideration..

Comment board text: Friday, October 17, 1997 at 20:42:33 (CDT)

Dr.Chiu: I do not want to discourage you in any way nor do I have anything against you. I actually think that you are one of the best professors around, not only in the physics department, but perhaps thoughout the whole UT campus, and I have tons of respect for you. But my problem today is not just my own personly problem, I represent the countless Physics students who encounter this same problem, and I hope by expressing this problem in great detail, something can be done about it.

My concern today involves the Physics quiz. I think it is totally unfair that there are negative points assign to the test. I can understand the purpose of the of the negative points so that students will not guess, but has the Physics department ever put any consideration to those student who actually work out the problem and miss it by a slight margin.

As hard as the quiz is alreally sometimes there are trick problems on the quiz. Why can't the Phyics department give some points out to problems that have a tiny mistake in them. Instead we are given negative points. As bad as it is that we did not receive some credit on the problems, there are negative points given. I am just so angry by this. This is perhaps the only department that not only does not give partial credit to problems, but assigns negative points, I mean what is that? Dr.Chiu, I do not mind studying, but to know that just a small mistake not only does not earn you partial credit, but also cost you points to be taken off, that Dr.Chiu is like a stab in the heart. It discourages even the best students at UT.

Dr. Chiu I hope you understand point, and you have to be in our shoes to understand what this is really like. I hope as great and understanding person that you are, I hope you can help. disappointed

Marcus's comments on negative scoring:

The negative scoring system was invented by statisticians to differentiate between ``educated'' guessing and "random" guessing. It is not used as a purely evil measure to punish innocent students, which seems to be the impression of some comment-writers. It works by assigning a small negative score to an incorrect answer, finely tuned so that if you guess randomly, you will end up with a small negative score on the average, but if you manage to eliminate around half of the choices you end up with a small positive score on the average. This is called a "zero-sum game". Thus the negative scoring rewards educated guessing on the average and penalizes completely random guessing on the average. If you don't like thinking statistically (which is the only way to think when discussing guesswork) then you just have to make sure all your answers are correct.

Comment from Oct 17, 11:47:45:

" . . totally ridiculous..As if the tests aren't hard enough without losing additional points for attempting questions that you are unsure of."

• The negative scoring only affects the grading of the quiz, relative between students. The difficulty of the test is the same with or without negative scoring.

"Have you ever heard of an educated guess?"

• Yes, as you can see we have. In fact, physicist generally value highly the art of "guesstimates", the ability to make useful quick estimates, or as in this case, to rule out impossible answers, if any. Because of our respect for this ability, the "zero-sum game" gives the educated guesser a slight advantage against the random guesser, a provision which, as you also remark, few other departments care to do.

Comment from Oct 17, 20:42:33:

"totally unfair...I can understand the purpose of the of the negative points so that students will not guess"

• Of course, this is a misunderstanding of both the purpose and the actual result of the system. It is not intented to discourage students from guessing, but instead is acutally designed to reward sensible guessing. See explanation above.

"not only does not give partial credit to problems, but assigns negative points"

• This addresses a completely different question, that of computerized grading in general. (The two are totally independent, since one could certainly use the "zero-sum game" without computerized grading).

My two cents for computerized grading go as follows:

There are two aspects to computerized grading: a) the fact that the questions are multiple-choice and b) the fact that it is graded by a computer. Most people actually mean to criticize point a) when they criticize computer grading.

When I came here I thought computerized multiple-choice grading was only for little kids and Americans. Now I realize that any multiple-choice test can, when designed carefully, reach the same level of testing effectiveness as textual answers.

To see why, consider what actually happens when you write a complete answer to a quiz question and hand it in to a grader. I know many grad students who work as graders, and they all use the same principle: write down some things which, if included, will score points. This is exactly what the computer will do, if the questions are broken down well enough, only the computer does the same job without errors. (Barring any scanning mistakes).

For higher-level classes with fewer students, where the questions involve several pages of calculations, computerized grading will not do, but for introductory classes as this, it fulfils its mission every day.

Regards,

Marcus