Physics World article context #3 – the problem with rating scales3 min read

This is the third of a series of blog posts on the context behind my Physics World article: A physicist’s experience of the mental-health system. There is a lot of backstory. So during the editing process, I sent documents to the editor to help explain some of the views I express in the article. I have decided to add them as blog posts.

It was during my rTMS treatment that I first encountered the rating scales. And, had I been completely honest, I would have said that I could not answer the questions. That I could not give a rating at all. Especially as every scale asked for an average over the last 2 weeks. I have thought of an analogy that I hope might get across the reason why I felt it was pretty much impossible to give a rating.

Say you have lived in Southern California USA for your entire life. You are used to warm, dry and sunny weather year-round. In the previous two weeks however, the weather has been very unCalifornian. On the first day it was foggy and a bit chilly out, the next few days were pretty warm for this time of year (winter) temps going up to 24 Celsius, with a lot of Sun. On the weekend though it rained persistently. Last week however started extraordinarily with snow, possibly the first time you had seen it in your life. The rest of the week then stayed pretty chilly, five or six degrees Celsius, before returning to warm and cloudy.

Then someone asked you a series of questions: In the last two weeks,

How bad was the rain on a scale of 0-4.

How warm or cold it was you thought it was, again from 0-4. (cold is 0, warm is 4)

How bad was the snow from 0-4.

How cloudy was it from 0-4 etc.

How could you answer the questions? Do I only rate it for the day it snowed? Or is it for all the days? It was cloudy when it snowed, so does that count? It was both cold and hot. If I put 2 that would represent midteen temperatures which it wasn’t at all during the 2 weeks.

So, doing your best, you score everything at their extremes, either zero or four, because it was the worst weather you had seen in your life.

But the day after you answer these questions, you move north to Seattle on the Northwestern US coast. The following two weeks have temperatures below zero Celsius, snow all day every day, and a storm blows through. Which is normal for the region.

Then someone asks you the same questions: in the last two weeks,

How bad was the rain on a scale of 0-4.

How warm or cold it was you thought it was from 0-4. (cold is 0, warm is 4)

How bad was the snow from 0-4. Etc.

You have already maxed out the rating system two weeks ago. And in the future the weather is going to stay somewhat around what happened in the last two weeks. What do you do now? Stay with maxed out scores? or readjust? What do you readjust to? When do you readjust? You haven’t experienced the weather yet over a significant period of time to know what is ‘bad’ weather.

I went to a support group meeting with other patients who were receiving treatment there. The one thing we were unanimous on was the incredibly poor quality of the ratings. None of us knew what rating to answer for each question. I think I was the only scientist there too. It was obvious to us (who had mental illnesses) that the ratings were not very good at all.

Mental illnesses are complex and somewhat unpredictable diseases like the weather. Symptoms fluctuate, come and go, and every day seems different than the last – even if there is a general theme. It was what mine was like anyway (when really bad). Actually, In the above analogy I don’t make clear that a lot of the times I couldn’t even make the distinction between symptoms. They were fluid, not discrete.