Now blogging on personal website

 All future blog posts will be at andywills.info

Post-Brexit bullsh*t

 A week ago, I bought a second-hand cassette deck on eBay, from a seller in Germany, for €59 (about £51). Given the model, this was a pretty good price. Shipping was quoted as €22 (£19) - OK. Then, the post-Brexit shit started to hit the fan.

First, eBay charged VAT at 20% on the purchase, as apparently they now have do for items bought from the EU for under £135. So, that's another £14.

Then the seller got in touch to say the courier had increased their prices as a result of the UK leaving the EU. So, that was another £11. 

Then, today, the courier got in contact to say they wouldn't deliver the parcel until I paid another £10 in import VAT and a £12 clearance fee. So, another £22.

All told, Brexit cost me an additional £47 on a item that cost £51. 

I did not vote for this. Those of you who did, welcome to the new world, I hope you feel it was all worth it.

h = 25

 My Google Scholar h-index just ticked over to 25; five months since the last uptick. Minorly faster steady progress ;-)

The Five Years of the Linux Desktop

 I've been using Linux exclusively - desktop, laptop, and server - for about five years now. It's been a really great experience, and it's been fun to see the world move in the same direction. Microsoft bought github, introduced WSL, and "loves Linux" now, apparently. Well, better than the "Linux is a cancer" MS view of old. We now teach R on a web server rather than SPSS on Windows, and test using OpenSesame on JATOS, rather than E-prime on Windows. I made some new friends at the local LUG and Tech Jam. It's been quite a ride. Looking forward to the next five years.

h = 24

 

Nine months after reaching 23, my Google Scholar h-index is now 24. The steady progress of this index continues.

Computers: 20 years on

Just added 32GB of memory to my home desktop, bringing the total to 40GB.  This is 300 times more memory than my top-end work desktop in 2000. And it's twice the size of that machine's hard drive.

Define 'Highly Cited'

It might be this:

LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324.

This paper introduced convolutional (weight-sharing) networks - now popularly known as Deep Neural Networks - and showed they could be used in real-world problems. Cited 24,100 times, according to  Google Scholar (2020-01-29) - over 1,000 citations per year on average.

Oh and - psychologists take note - published in conference proceedings.

Not a one off. How about this conference paper. It's by Simonman & Zisserman, it's a development of the LeCun paper, it was published in 2009, and has averaged 5,500 citations per year.

h = 23

My Google scholar h-index just hit 23, about 8 months since the last rise. Steady progress, I guess...

Donation to GNU Octave

Donated $10 to GNU Octave, a free software replacement for MATLAB. A key resource to liberate scientific code from proprietary software.

Science is not advertising...or shouldn't be

“Six stone lighter now. I have more energy and feel healthier than I have for a long time” 

- Shirley Hardy, Atkins diet

 
Back in November 2018, a few of my colleagues read a recently-published article in Psychonomic Bulletin & Review. The article concerned the evidence for dissociable learning processes in comparative and cognitive psychology. We had all previously critiqued, in print, some part of the evidence presented. We had no particular reason to assume that the authors would agree with our critiques --- and that's fine, it's all part of the continuing debate and dialogue of science. What was perturbing was that the review had largely been written as if no such critiques existed.

In our response, (now accepted by PB&R) we coined the term testimonial review for this type of article. The term refers to a well-known technique in advertising where one promotes a product by highlighting cases that put your product in a good light. Of course, you can't scientifically evidence a claim simply by reporting the data that supports it. One has to consider both the evidence for, and against. You weigh the evidence and come to a conclusion. Good science involves showing your working, so one would expect this process of weighing evidence to be part of any scientific review paper. We call this a balanced review.

 

Testimonial reviews are not good science. They are potentially misleading, and may result in others basing their own work around the incorrect assumption that a particular issue is resolved. Science isn't advertising ... or, at least, it shouldn't be.

 

h = 22

My Google Scholar h-index reaches 22, about 10 months after it reached 21. Steady progress, I guess :-)

Seven ways to fix the replication crisis

I gave a talk yesterday that was an opinionated survey of seven causes of the replication crisis in psychology, and seven actions we could all take today to avoid it in future. All the slides are on github. In brief:

1. Publication bias
Publication bias comes in part from null results being meaningless with  traditional statistics. Use Bayes Factors instead, they can provide evidence for the null, and are easy to do in R.

2. Small sample size
Most of us do not collect enough data in our experiments. Use a power calculation to work out an appropriate sample size. This is easy to do in R.

3. Misunderstanding statistics
No-one in psychology really understands p values. Also, a p value between .04 and .05 is strangely common in psychology, yet p-values in this range provide only very weak evidence. Use Bayes Factors instead

4. Low reproducibility
If you run a different experiment to me, and do different analysis, is it that surprising you get a different answer? Ensure your work is reproducible by publishing your raw data, analysis scripts, stimuli, and experiment code.

5. ‘p’ hacking
Common practices in flexible analysis, like testing for significance after every 10 participants, and stopping when it's significant, can lead to false positive rates of about 60%. Pre-register your next big study, so you don't fool yourself.

6. Poor project management
Most psychologists do not have adequate private archiving and recording within their own labs. Use a version control system (e.g. github) to improve project management in your lab.

7. Publication norms
Pressure to publish lots of papers leads to lots of poor outputs, rather than a few good ones. Publish fewer, better papers. If you are a manager, focus hiring, promotion, and appraisal less on volume and more on quality.

CC-BY-SA 4.0

PsychoPy on Raspberry Pi

The problem
 
My department is fortunate to have several multi-seat testing rooms for psychological research. The downside is the computers inside them are slightly ageing. They're several-year-old desktop machines with integrated graphics that were originally Windows 7, but have been converted to Windows 10.

Since that conversion, Psychopy, a great open-source experiment generator has been experiencing intermittent issues with graphics-related freezes. It's not all machines, and not all the time, but sporadically they hang for 4-5 seconds before updating the screen. This is bad news for some experiments. Psychopy does not officially support integrated graphics, so our attempts to get this resolved with the developers have so far met with limited success.

Some solutions I didn't go for

1. Upgrade the machines

A £30 discrete graphics card would probably do the trick, but with the number of machines we have across the department, that's still quite a cost overall.

2.  Boot to Linux

We've never been able to replicate this hanging issue on any Linux machine, so it seems Windows specific. Unfortunately, booting from USB is disabled on these machines.


3. Use Linux laptops

Our lab is probably only testing six people at any one time, so we could buy a set of laptops for this purpose and just move them into the testing rooms when we test. This would work, but is potentially a bit expensive (perhaps £2000).

The solution I'm now trying:

4. Use Raspberry Pis

Raspberry Pis are cheap, and the testing rooms already have monitors, keyboards and mice in them (connected to the desktop machines). So, total cost per seat is £51.75. That's for a Raspberry Pi 3, case, power supply, 8GB SD card, official power supply, and HDMI to DVI cable.

The Psychopy programs I've tested so far on this setup work fine.

Please stop using this graph to argue your topic is popular

It seems to have become quite common to use this sort of "number of publications" graph to argue for the importance of one's own research area:

 

The graph shows that the number of papers including a particular search term in their title, abstract, or keywords, has risen dramatically over the last few decades. In the example above, the search term in this case is "meditation OR mindfulness", following an analysis reported by Van Dam et al. (2017).

These were just some data I had easily to hand - there's no intention to imply here that mindfulness research is particularly prone to this kind of analysis.

One problem with this kind of analysis is that the number of scientific publications per year is also increasing for most disciplines. It's fairly easy to add this information to such a graph. For example, let's plot the number of papers including the word "psychology" in their title, abstract, or keywords, on the same axes:

This puts a slightly different perspective on things, and provides little support for Van Dam's claim that: "Over the past two decades, writings on mindfulness and meditation practices have saturated the public news media and scientific literature". The literature seems very far from saturated on this particular topic (depending on what 'saturated' is taken to mean - clearly not saturated in the same way that e.g. the US market for refrigerators is saturated).

If we express the number of "mindfulness OR meditation" papers as a percentage of the number of "psychology" papers, we get the following:

This graph gives us a different perspective to the one offered by Van Dam. The 'saturation' of mindfulness research in psychology was around a fairly stable low level of 1-2% from 1975 to 2000. It rose to a peak of about 6%  in 2012, but has been declining since.

Of course, there would be other, probably better ways, of calculating the 'market share' of a concept in the scientific literature than the method used here. The main point here is simply to demonstrate that raw counts are a very poor measure.

pu067

PsychoPy on Linux

We've been using PsychoPy in our lab for a few years now, developing mainly on Linux, but often collecting data on Windows (shared testing facilities). On the basis of this experience, I recommend installing PsychoPy on Linux thus:

sudo apt install psychopy

This gives you quite an out of date version (1.85.3, released Aug 2017, for Ubuntu 18.04, as of Dec 2018), but trying to build it from source, or with pip, is just a huge time sink, often fails, and can kinda mess up your system in the process.  

This leads to a further recommendation. For archive purposes, archive your .py file as well as your .psyexp file, as more recent versions of Builder produce .psyexp files that are not backwards compatible!  

In fact, for portability, seriously consider treating PsychoPy mainly as a Python library, and develop experiments entirely as .py files, i.e. avoid Builder entirely. Builder is a good teaching tool, but anyone who can write R scripts can also learn how to write experiments directly in Python.

Renewed my FSFE donation

I renewed my Free Software Foundation Europe  supporter donation (60 EUR annually).

Donating to EFF

Donated $65 to the Electronic Frontier Foundation.

Donating to TOR

Just donated $75 to the TOR project.

Plymouth University to open Brain Imaging Centre in 2020

Many years in the planning, yesterday we bought a 3T Siemens scanner to put in our new Brain Imaging Centre building (picture above). Due to open for business in Spring 2020.

Doom (2016) on Linux

Valve recently announced Steam Play, a system that allows previously Windows-only games to be played on Linux. It's in beta at the moment, but I thought I'd give it a go with Doom (2016). After a bit of a faff, it seems to work pretty well. The faff was that, on initially running, you just get a black screen. A search through the forums revealed that ALT+Enter brings it into a window on the desktop (rather than fullscreen), and it then works. From there, you can press ESC to bring up the game options, and set it back to fullscreen. Played for about an hour - works pretty well!

Of course, Doom 2016  not open source ... but it's still interesting how good Linux is getting as a games platform.