Yes, Computer Scientists Are Hypercritical
Jeannette M. Wing
October 6, 2011
Are computer scientists hypercritical? Are we more critical than scientists and engineers in other disciplines? Some numbers from the National Science Foundation support the claim that we are hypercritical.
I have some data from the UK (dated 2007) which shows a similar trend. EPSRC is ithe UK NSF equivalent. I have made it available at http://www.cs.ucl.ac.uk/staff/A.Finkelstein/review.pdf
Perhaps the problem is not hypercritical CISE reviewers but grade inflation in the rest of the fields. What distribution of scores are NSF proposals supposed to get? Is CISE following instructions on the distribution? The fact that the mean difference between awards and non-awards is larger in CISE implies that CISE is making better use of the scoring system. Perhaps it is the physicists and mathematicians who need to fix their "everything is wonderful" scoring system.
Either that or NSF needs to adjust scores to percentiles within each panel, to remove scorer calibration errors.
I agree that we are usually more critical than others. But, as some have pointed out, it is necessary to identify which fields pull the average up and which pull it down. Also, it is necessary to know if the differences in the grades imply in differences on the rates of proposals awarded/denied. If there is no difference for this comparison, then there is no problem on being hypercritical or not.
Paulista State University, Brazil
This article simply ROCKS ! That was a great read for me. keep it up with all the good work..
I think Yannis makes some good points. I also have the impression that overall, CS is a less nasty field than many others, although perhaps more critical. If you want to see nasty, take a look at the humanities.
A factor not mentioned so far is that relative to other fields, resources in CS have historically been plentiful. We get new buildings funded by industry billionaires, we have access to relatively broad and generous government sources, and we have access to more non-government funding sources than many fields. Our academic salaries are higher than most, and any time we are dissatisfied with academia, there are plentiful industry jobs.
So we can afford to be critical.
As scientists, we should also acknowledge that these numbers don't actually tell us whether we're more critical or not. It's possible that they show that CS grant proposals are truly worse than those in other fields. It's also possible that CS proposals are better, and the criticalness is even worse than the numbers show.
Ultimately the apportionment of resources, awards, etc comes down to a social and policy decision about the value of CS as a field. I'm perfectly comfortable asserting that CS provides more benefit to society than other fields, and therefore is deserving of more resources.