I use AI to correct sentences I write in English. I’ve also used it to build my own English-learning app. I like choosing what to practice and fitting the study around my goals.
That made me wonder whether I could learn more this way than by taking lessons at a language school or with a private tutor.
My expectation was fairly simple. AI could help me practice exactly what I need, whenever I want. The difficult part might be keeping it up. Lessons give you appointments and someone who expects you to show up. I feel reasonably comfortable maintaining my own routine, so the freedom of studying independently appeals to me.
But enjoying that freedom doesn’t tell me how much I’m learning.
We used this question to try Fair Research, an experimental skill I developed to help an AI assistant investigate a question and give other credible explanations room to be considered. In my Medium essay, I explored the idea behind it. Here, we followed what happened when we tried it on a question I care about.
A question without my expected answer
Our first attempt happened in a conversation where the assistant already knew my expectation and had seen some sources. To see how it would approach the question without those details, we started a new conversation with this request:
Usa $fair-research in modalità Standard per indagare questa domanda:
negli adulti di età superiore ai 30 anni, studiare autonomamente con un
assistente AI migliora l’apprendimento di una lingua diversa dalla propria
lingua madre rispetto alle lezioni in una scuola di lingua o con un tutor privato?In English: for adults over 30, does studying independently with an AI assistant help them learn a language better than school lessons or private tutoring? By another language, I mean one other than their native language.
We didn’t tell it what answer we expected or which sources to use. We also didn’t repeat the research instructions already contained in the skill.
The assistant started by deciding what would count as learning. If AI corrects a sentence for me, can I write a new one correctly without its help? Can I still do it later, and use what I’ve learned in a real conversation?
It also looked for fair comparisons: learners starting at a similar level and spending a similar amount of time studying. AI used on its own needed to be distinguished from AI used alongside a teacher.
The assistant made these choices before searching, without asking us to define every detail. Because it explained them, we could see whether it had understood the question. The skill also allows it to suggest a different question when the original is too broad or unclear to answer usefully.
Useful evidence, with an awkward gap
The search found studies suggesting that AI can help people learn. The difficulty was knowing how much those findings told us about our comparison.
For example, Lyu, Lai and Guo brought together 31 studies and found benefits for learning. Only ten used generative AI, the kind of technology used by assistants such as ChatGPT. The rest used chatbots whose responses followed predefined rules. Many participants were university students. These tools and learners weren’t necessarily representative of an adult over 30 studying independently today.
Guan, Li and Gu looked specifically at generative AI used outside the classroom. Their summary reported improvements in English and in learners’ ability to manage their own learning. It didn’t find a clear benefit for motivation. That was relevant to my concern about keeping up the practice, but it didn’t show that a teacher was necessary to stay motivated. We could check the published summary, while access problems prevented us from fully reviewing the studies behind it.
Comparisons with people were interesting too. Kang studied 61 college learners practicing with an AI avatar or a native speaker. The published summary reported that lower-level learners benefited more from AI, while higher-level learners benefited more from the person. That suggests starting ability can matter. It still doesn’t tell us whether AI beats a professional tutor for someone over 30.
In another study, a robot-based system did better than human tutors on some speaking measures. But the groups practiced different exercises, and the researchers hadn’t collected their ages. The robot’s advantage in speaking fluently appeared in one test, but wasn’t clearly established in two others. Those details matter when deciding what the result actually supports.
We also found a ChatGPT workshop involving 24 people aged 60 and above. It examined how they used the tool and what they thought of it. Finding older learners didn’t solve the problem: the study hadn’t compared how much they learned with AI versus lessons.
After the assistant finished, we went back to check its sources. Some papers were available in full; others could only be checked through summaries or sections available in search results. We also found that Kang’s study was included in Lyu’s review. Counting both as separate evidence would count the same work twice.
What the research could conclude
The answer was much like the one from our first attempt: the studies we found didn’t establish that studying independently with AI works better than lessons for adults over 30.
We couldn’t settle either comparison, with a school or with a private tutor. That leaves the question open. It doesn’t mean the methods work equally well, or that AI is worse.
The assistant checked whether it could say more by including younger learners, older chatbot tools and results measured soon after practice. That gave us more reasons to consider AI for particular activities. When it focused strictly on our age group and kind of study, there was much less to draw on. Neither approach gave us enough to choose a general winner.
It also considered more than my original explanation. How much someone practices, which exercises they do, their starting level and the help they receive could all affect what they learn. Personalization and consistency might be part of the answer, without explaining all of it.
Introducing my expectation afterward
We saved that answer, then told the assistant what I had expected: AI might help me study in a way that suits me, while lessons might help other people maintain a routine.
The assistant thought that was plausible, but pointed out that a good tutor can tailor the study too. The research hadn’t shown that people improved because AI personalized their practice, or how much having regular appointments helped learners over 30 keep going.
It made the same distinction about my own experience. Being comfortable with AI and able to study independently might help me use it well. It doesn’t prove that I learn more than I would with lessons.
Its suggestion was something I could actually try: after working through a correction, write a new sentence without help, return to the same difficulty later, and notice whether it becomes easier to use in conversation.
The reply took my preference seriously while leaving it open to testing. That’s useful to observe, although one reply can’t tell us how consistently the assistant would behave with other questions.
What the skill contributes, and what still needs checking
A skill is a reusable set of instructions saved in a SKILL.md file. Fair Research guides the assistant’s work with the sources it can access. It doesn’t provide papers or a research database.
It asks the assistant to clarify the question, decide how to judge the answers, and search for credible explanations beyond the first one offered. Then it checks what each source actually supports. A result can challenge an explanation, or simply show that it applies to fewer situations than we thought. Two explanations can also both be true.
The skill asks for checks like the ones we made here: are two sources repeating the same study? Would the answer change if we gave more importance to results that last? Could another reasonable reading of a study lead us somewhere else?
Light, Standard and Deep modes offer different levels of detail. We used Standard. It asks the assistant to explain which evidence supports each answer, then check whether the conclusion changes when it makes different reasonable choices. In our research, that meant asking what happened if we included younger learners or gave more importance to learning that lasted over time.
The assistant can also use numerical ratings to make these choices easier to examine. For example, it might rate how closely a study answers the question on a scale from one to four, explaining the rating. It can then try another reasonable rating, or give that criterion more importance, and see whether the conclusion changes. We didn’t use numerical ratings in this test. They’re optional, and a rating of three out of four doesn’t mean a claim has a 75% chance of being true. Deep mode asks for more extensive research; it doesn’t automatically meet the requirements of a systematic review.
The new conversation made several useful choices without a detailed prompt, but its answer was incomplete. It didn’t identify which sources it could only partly read, treated school lessons and private tutoring together, and didn’t explain why it stopped searching. In our later review, we added access notes for each source and considered the two types of lessons separately. These details came from reviewing the answer after the conversation had finished. The assistant hadn’t supplied them on its own, and its reason for ending the search still wasn’t recorded.
Both attempts reached a similar conclusion. They also shared some evidence: for example, both cited the review by Guan and colleagues. Their agreement could partly reflect reading the same research. We haven’t checked whether using the skill produces better results than researching without it, or asked an independent expert to assess the two answers. Fair Research remains experimental.
You can download the skill and find installation instructions in the Fair Research repository on GitHub. Then try it with a simple request like the one above, replacing the question with your own. For setup instructions, see the official guides to skills in Codex and skills in Claude. Our tests here used Codex; we haven’t tested Fair Research in Claude.
I still like the freedom of studying with AI. Now there’s a concrete question to keep alongside that preference: what can I do on my own after the help is gone?
Have an awesome journey,
Francesco