Monday 28 April 2008

Academic rebellion - being critical


Being critical is something that you will need to scoop the top marks. No news there. But how do you go about it? Of course, I can't tell you how to be critical of specific papers but I can tell you about a mindset that's useful as well as a systematic method of attacking a paper.

When you read papers and books by academics you know you can trust what they are saying, right? Wrong. What you are doing here is handing over your money before you have had a chance to try out the goods.

Conformity is seared into us from our schooling because rebellion gets punished. This carries over into University. Academics are seen as unwavering sources of truth and students should yield to what they profess. Such deference is appropriate if we are talking about manners: they know more than you do. That much buys respect.

But we are not talking about manners; we are talking about finding the truth (that sounds very grand, doesn't it?). The assumption that a piece of writing is automatically correct or worthy just because it has been written by someone with a few extra letters after their name is the enemy of critical thinking.

Instead of holding the author in very high regard and instead of expecting what you are about to read to be true, flip the situation on its head. Hold the author in very low regard and let her prove herself. You need to leave open the possibility that, despite all their training, and their fancy office, and complicated lectures, academics might be wrong.

If you want to be critical take on the attitude of the curmudgeonly fish in the pic at the top: enter all academic waters equipped with incredulity and contempt for the author and their argument and you will become a sharper critic for it (if you need help, imagine they have done you a "terrible wrong"). Start them in the negative and let them buy their way into the positive with their argument. Uncritical people read to be told; critical people read to be persuaded.

Generally speaking, this philosophy should allow you to pick holes in the way people write. If what they have said doesn't convince you of their point, don't buy it. Sentences should build up points, which should overflow into arguments. If you don't feel this happening you have a case to make against them.

Think about a paper like the great 'Honda Cog' ad (or Guinness' not-so-cheap knock-off): if everything 'just works' then you have yourself a brilliant paper. If, however, something along the argument of the paper breaks down then the whole trick falls apart preventing solid conclusion, then you can bring it to the marker's attention.



Taking a modular approach to criticism is useful and makes attacking a big paper much easier. In the introduction, does the author set up all the logical links for the study's motivation? Or is there something shaky on which the rest of the paper is predicated?

Is the method the correct one? What about confounds that might have been forgotten? Sometimes something as subtle as having a blind open or closed in a room can make all the difference (e.g. Li., P., & Gleitman, L. (2002). Turning the tables: language and spatial reasoning. Cognition, 83(3), 265-294).

Have the statistics been carried out properly? Have they used ANOVA but violated its assumptions (e.g. normal distribution). In the discussion, have these stats been interpreted correctly? Do the data really support the conclusions? Have they made a causal link where it is only possible to make a correlational one? Is there perhaps an alternative, more parsimonious explanation; if they report the sound of hooves and conclude zebras, are horses not a more likely answer? One actionable thing to do is write down how the argument is set up in numbered form, taking each proposition by itself. This helps you to see the weak assumptions made by re-authoring the paper yourself.

So today's takeaway nugget: be incredulous, contemptuous and modular and be a better critic for it.

No comments: