×

AI in Research: A New Era, But Not The End of Human Insight

Written by Tamsin Maries, Research Manager

I have recently re-joined the world of research following a couple of years away recovering from Long COVID. I’ve discovered that not only have I really missed it and that everyone is a lot younger, but that everything really has moved on. There are new methodologies to learn about, new topics and trends that clients want to explore, and new and more efficient ways of getting things done.

An AI digital illustration of a female market researcher using AI. She is looking at a screen with a range of simple icons such as looking glasses, bar charts and gender icons around the outside of the image.

Which brings me to Artificial Intelligence (AI). Two years ago in the research world, AI was occasionally whispered and wondered about, still a far-away concept, hard to grasp. Clients included one or two questions about it in their surveys, trying to gauge early opinion and engagement. People half-joked about machines taking over their jobs, or even the world. ChatGPT had not yet been released. And so, for me, it was quite an adjustment to wrap my head around the fact that AI is literally here, ubiquitous, free, and easily accessible to most of us at the click of a button. ChatGPT now has over 200 million weekly users. Talk about a mental and cultural shift.

One of my first tasks at M·E·L Research was to lead on the analysis of a survey that involved 2,000 open text responses, many individually long enough to fill an A4 page. These were sometimes deeply personal, written by a particular set of participants experiencing turbulent and challenging times, all eager to ensure their voice was heard. Many had clearly put a huge amount of thought into what they wanted to say, and as I started to read the first few responses, I realised I was faced with that familiar researcher responsibility – doing justice to the thoughts and feelings of others.

Open text questions have sometimes and at some agencies been a knotty issue in research. A half-way house between quantitative closed-response questions and qualitative research. Done well, they can provide richness of detail behind the “why” of human opinion and behaviour. But too often, open text data ends up underutilised due to time and budget constraints, and open questions can even end up left out of surveys completely.

I have seen other agencies actively discourage clients from including them in surveys due to the time and expense of cleaning, coding and analysis. I’ve watched skilled coders read through and manually code every single response – time-consuming and labour-intensive, even at the speed they can work at. I’ve seen word clouds created with the data, which whilst speedy and pretty, certainly don’t do justice to the richness and complexity of the data. And so, confronted with my 2,000 verbatims, I wondered if AI had a role to play here.

I started out with some trepidation. I fleetingly wondered, ‘what if it is better than me?’

Mainly I was intrigued and keen to get started. It took me a while to be able to write effective commands, much like learning to refine search engine queries years ago (showing my age now). I had to resist the urge to say “thank you” every time AI completed a task for me – I mean what is the etiquette?! I lost count of the number of iterations I ran. I was initially overwhelmingly impressed. It definitely saved me some time, even despite my beginner status. I very quickly had sight of the key themes and an idea of prevalence.

But then I hit a point where it stopped being useful to me, it only got me so far. The code themes were very generic. I discovered that AI can code concrete terms beautifully but struggles with nuance and context-specific language. And I just could not get to a point where I was happy with the accuracy of the coding. I did not feel that the iterated refinements accurately reflected the voices and opinions of those so keen to be heard. And I still had more than 40% of my verbatims coded as “Other”.

So back I went to manual, human coding. It was time-intensive and a bit of a slog. But I got there, finally happy with accuracy, depth of analysis, and down to just 1% “Other”. Happy that I could now truly reflect what my respondents wanted to say, happy that I could give them a voice and the power to change their situation, and happy that I could give my client the answers they needed to take action.

So, what did I learn from my first AI experience? I learned about AI capabilities and how to use it. I learned a lot about the research topic. If AI had done all the coding for me, I would not have understood the data, the respondents, the context, as well as I do now. I would not have understood my client as well as I do now. Whilst AI offers a glimpse of potential solutions, I learned that it cannot replace a human researcher just yet. Thankfully, there is still very much a place for a curious human, to conduct the kind of analysis that requires deep thought and time to make connections. But boy did it take a long time. The tension between speed and depth of analysis remains, and the search for the perfect open text analysis solution continues. But at least I can enjoy feeling a little more up to date!

Tamsin is a white woman with dark shoulder length hair and is wearing a blue and white striped t-shirt.


Get in touch

    Latest News

    Cookie Overview

    This website uses cookies so that we can provide you with the best user experience possible.

    Cookies are stored in your browser and perform functions such as recognising when you return to our website and helping our team to understand which sections of the website you find most interesting and useful. They help us improve our service to you and others.