Large language models often plan solutions correctly yet stumble within individual reasoning steps, especially when guided by coarse, problem-level demonstrations. This paper introduces a probe- and-guide in-context learning framework that aligns guidance to the step granularity: the model first issues a brief probe for the next step, then retrieves and conditions on closely matched example steps from a cu- rated repository to execute the step with higher fidelity. The approach reduces irrelevant-example noise, improves single-step correctness with- out additional training, and slots into standard inference pipelines and search-based decoders, enhancing both candidate generation and verifi- cation. Evaluations across diverse mathematical reasoning settings show consistent gains over zero-shot and few-shot baselines, and the method composes naturally with tree-search strategies to further improve solu- tion quality while controlling token cost. The design is model-agnostic, training-free, and centers on LLM inference workflows, making it practi- cal for deployments that demand reliable, fine-grained reasoning.
Publish Year: 2026
Download PDF