🧠 Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers

When you think about boosting small language models (SLMs), fine-tuning or upgrading to superior models might be the first thing that comes to mind. But what if you could make these models better problem-solvers without any of that? That’s exactly what Zhenting Qi, Mingyuan Ma, Jiahang Xu, Li Lyna Zhang, Fan Yang, and Mao Yang explored in their paper, "MUTUAL REASONING MAKES SMALLER LLMS STRONGER PROBLEM-SOLVERS."

📌 The Core Idea

The goal? Simple—enhance the reasoning capabilities of SLMs. The key innovation here is a self-play mutual reasoning approach called rStar. This method breaks down reasoning into a mutual generation-discrimination process. It might sound complex, but the elegance lies in its simplicity.

🛠️ How It Works

The process involves two main steps: generation and discrimination.

1. Generation: The target SLM uses Monte Carlo Tree Search (MCTS) to generate high-quality reasoning trajectories. MCTS is supercharged with a set of human-like reasoning actions—like proposing one-step thoughts, remaining thought steps, sub-questions, re-answering sub-questions, and rephrasing questions. It’s like simulating human thought processes in problem-solving.

2. Discrimination: Once these reasoning trajectories are generated, a second SLM steps in as the discriminator. This model verifies each trajectory generated by the target SLM. Think of it as getting a second opinion to ensure the solution is solid. The mutually agreed trajectories are more likely to be correct.

💡 Why It Stands Out

• Diverse Human-like Reasoning: The use of diverse reasoning actions in MCTS helps generate high-quality solutions.

• Mutual Consistency: By having a second SLM provide feedback, the accuracy of the generated solutions is enhanced.

• No Fine-Tuning Needed: Achieves impressive results without relying on fine-tuning or superior models, making it more accessible and practical.

📊 Impressive Results

The researchers put rStar to the test across five SLMs (Phi3-mini, LLaMA2-7B, Mistral-7B, LLaMA3-8B, and LLaMA3-8B-Instruct) and five diverse reasoning tasks (GSM8K, GSM-Hard, MATH, SVAMP, and StrategyQA). The results? rStar significantly boosted accuracy—like taking GSM8K accuracy from 12.51% to 63.91% for LLaMA2-7B and from 36.46% to 81.88% for Mistral-7B. It even outperformed state-of-the-art baselines across various tasks.

👍 Advantages and 👎 Limitations

Advantages:

• State-of-the-Art Performance: Delivers top-notch accuracy without additional model training.

• Accessible: Works without needing to upgrade to superior models.

Limitations:

• Inference Costs: The method may increase inference costs due to the need for multiple rollouts in MCTS.

• Discriminator Effectiveness: The effectiveness of the discriminator model can vary depending on its capabilities.

🔍 Final Thoughts

rStar is a novel approach that significantly enhances the reasoning capabilities of SLMs during inference. By leveraging a rich set of human-like reasoning actions and mutual consistency for effective solution generation and verification, it achieves state-of-the-art performance across various tasks and models—without the need for fine-tuning or superior models. A powerful tool for improving SLMs, rStar shows what’s possible with innovative thinking.

🚀 Explore the Paper: Interested in pushing the boundaries of what small language models can achieve? This paper is a must-read.

Subscribe for more insights like this!