Konstantinos Sidiropoulos

In the rapidly evolving landscape of AI, separating genuine advancements from the noise is crucial for both industrial practitioners and academic researchers. The non-deterministic nature of LLM-driven applications amplifies this challenge, leading to difficulty in applying common software testing practices and thereby quantifying application performance.

This session aims to offer insight into mitigating these issues by delving into RAGAS (Real-World Application and Generalization Assessment Score), a novel framework designed to quantitatively measure the performance of LLM-driven applications. Drawing from well-established practices in the software testing world, it demonstrates how RAGAS can guide the development and deployment of LLM technologies with tangible benefits. A short demo is also included as a practical demonstration of how RAGAS's ability to evaluate LLM effectiveness can be translated into real-world impact.