Seven papers at ACL2025 set out to provide some initial answers to this tough question.
View Our ResearchThere is a lot of excitement about Large Reasoning Models like OpenAI's o1 and Deepseek's R1 — but that excitement has fostered some detailed analysis and critique that seems to cast doubt on the extent to which language models can reason, and indeed whether there is any actual reasoning going on at all.
When reasoning is expressed in language – which everyone agrees is what language models might be doing – the structures that result are arguments. As we start to explore the 'R' in LRMs, we can turn to an area of philosophy, known as argumentation theory, to provide some starting points. As those philosophical and linguistic starting points lead to AI engineering, the result is argument technology.
The Centre for Argument Technology, founded over two decades ago, has been developing AI argumentation systems in domains ranging from broadcast media to geopolitics and from intelligence analysis to healthcare that have gone on to support hundreds of thousands of users around the world.
At ACL2025 the Centre has leveraged its unique datasets and software stacks to explore issues of reasoning in language models and has seven papers focusing on different aspects.
Reasoning in modern AI models is a lynchpin challenge for 2025, and we’re looking forward to chatting about the directions and burning questions in the area at ACL in Vienna.
Uses large- and very-large-scale argumentation structures to model complex reasoning processes in natural language text.
Uses the domain of logic puzzles to quantify the limits of reasoning in language models and distinguish between memorization and reasoning.
Harnesses large natural language datasets of argumentation with masking to assess LLM performance on natural reasoning tasks.
Explores complex argumentative reasoning patterns in natural dialogue and their implications for conversational AI systems.
Provides an open, extensible set of tools and libraries for processing argument in natural language. Live system demonstration.
Workshop presentation on re-framing LLMs as tools to exercise our critical thinking skills rather than replacing them.
Workshop presentation on practical challenges and solutions in deploying argument mining systems in real-world applications.
