Many Grumble readers will have come across the mini-PAT. For any that have not, it is a peer assessment tool . There is a big industry now in these things. They go by various names: multi-source assessment, 360 degree feedback, peer review and peer rating are all examples of the same sort of appraisal technique. And, in case you're wondering, SPRAT is the Sheffield Peer Review Assessment Tool. In Sheffield they gave up making knives some decades ago. They now have a big computer, loads of staff and do mini-ePATs. And they are thriving. This is big business. Sheffield is now the premier provider of workplace assessment across the UK. So they must be doing a good job. Or so you would think.
If you ask those that should know about the evidence base for these mini-PATs you will get a blunt reply. They will look down their noses at you and there will be the implication that you must be pretty ignorant on the topic to have needed to ask the question. The curt put-down will mention validity, reliability and the like. You will have to wait until a Bank Holiday weekend to find out what the evidence really is. And then you will get a shock. Because if you look up the rather sparse number of publications on the topic you will find statements such as this:
It is no secret that mini-PAT lacks sufficient field evaluation and has not gone through any stringent criteria that are required for the validation of an assessment tool.
No evidence of reliability has been published to date.........
Furthermore, more generally:
Non specific feedback does little to effect a change in performance.
In a third of cases feedback results in decreased performance.
An improved outcome is by no means the rule.
It is then that you will understand why Dr Grumble has been given such short shrift when he has queried these dubious newfangled initiatives. There is no firm evidence to support the mini-PAT. The emperor really has no clothes. Yet every new doctor in the land is being subjected to trial by mini-ePAT. And the results will be stored on a Big Brother computer for the rest of time. A whole industry in Sheffield is dedicated to this stuff. All based on smoke and mirrors. Does anybody know where Dr Grumble can buy a decent knife?
Aza Abdulla A critical analysis of mini peer assessment tool (mini-PAT)