Computers: 20 years on

Just added 32GB of memory to my home desktop, bringing the total to 40GB.  This is 300 times more memory than my top-end work desktop in 2000. And it's twice the size of that machine's hard drive.

Define 'Highly Cited'

It might be this:

LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324.

This paper introduced convolutional (weight-sharing) networks - now popularly known as Deep Neural Networks - and showed they could be used in real-world problems. Cited 24,100 times, according to  Google Scholar (2020-01-29) - over 1,000 citations per year on average.

Oh and - psychologists take note - published in conference proceedings.

Not a one off. How about this conference paper. It's by Simonman & Zisserman, it's a development of the LeCun paper, it was published in 2009, and has averaged 5,500 citations per year.

h = 23

My Google scholar h-index just hit 23, about 8 months since the last rise. Steady progress, I guess...

Donation to GNU Octave

Donated $10 to GNU Octave, a free software replacement for MATLAB. A key resource to liberate scientific code from proprietary software.

Science is not advertising...or shouldn't be

“Six stone lighter now. I have more energy and feel healthier than I have for a long time” 

- Shirley Hardy, Atkins diet

 
Back in November 2018, a few of my colleagues read a recently-published article in Psychonomic Bulletin & Review. The article concerned the evidence for dissociable learning processes in comparative and cognitive psychology. We had all previously critiqued, in print, some part of the evidence presented. We had no particular reason to assume that the authors would agree with our critiques --- and that's fine, it's all part of the continuing debate and dialogue of science. What was perturbing was that the review had largely been written as if no such critiques existed.

In our response, (now accepted by PB&R) we coined the term testimonial review for this type of article. The term refers to a well-known technique in advertising where one promotes a product by highlighting cases that put your product in a good light. Of course, you can't scientifically evidence a claim simply by reporting the data that supports it. One has to consider both the evidence for, and against. You weigh the evidence and come to a conclusion. Good science involves showing your working, so one would expect this process of weighing evidence to be part of any scientific review paper. We call this a balanced review.

 

Testimonial reviews are not good science. They are potentially misleading, and may result in others basing their own work around the incorrect assumption that a particular issue is resolved. Science isn't advertising ... or, at least, it shouldn't be.

 

h = 22

My Google Scholar h-index reaches 22, about 10 months after it reached 21. Steady progress, I guess :-)

Seven ways to fix the replication crisis

I gave a talk yesterday that was an opinionated survey of seven causes of the replication crisis in psychology, and seven actions we could all take today to avoid it in future. All the slides are on github. In brief:

1. Publication bias
Publication bias comes in part from null results being meaningless with  traditional statistics. Use Bayes Factors instead, they can provide evidence for the null, and are easy to do in R.

2. Small sample size
Most of us do not collect enough data in our experiments. Use a power calculation to work out an appropriate sample size. This is easy to do in R.

3. Misunderstanding statistics
No-one in psychology really understands p values. Also, a p value between .04 and .05 is strangely common in psychology, yet p-values in this range provide only very weak evidence. Use Bayes Factors instead

4. Low reproducibility
If you run a different experiment to me, and do different analysis, is it that surprising you get a different answer? Ensure your work is reproducible by publishing your raw data, analysis scripts, stimuli, and experiment code.

5. ‘p’ hacking
Common practices in flexible analysis, like testing for significance after every 10 participants, and stopping when it's significant, can lead to false positive rates of about 60%. Pre-register your next big study, so you don't fool yourself.

6. Poor project management
Most psychologists do not have adequate private archiving and recording within their own labs. Use a version control system (e.g. github) to improve project management in your lab.

7. Publication norms
Pressure to publish lots of papers leads to lots of poor outputs, rather than a few good ones. Publish fewer, better papers. If you are a manager, focus hiring, promotion, and appraisal less on volume and more on quality.

CC-BY-SA 4.0