Abstract: Fine-tuning Retrieval-Augmented Generation (RAG) chatbots is challenging due to the many interdependent parameters affecting performance. RAGBOT CLI is a terminal-based Python tool built atop the LangChain framework that enables systematic experimentation with RAG configurations and automated evaluation using both quantitative metrics (BLEU, ROUGE-L, semantic similarity) and qualitative ones (contextual relevance, response relevance, factual fidelity). Unlike existing frameworks, RAGBOT CLI offers a modular, project-oriented architecture and supports hybrid evaluation strategies, making it suitable for academic and professional use. This paper describes the architecture, functionalities, and practical applications, showcasing its potential impact on the development and evaluation of RAG-based chatbots.
