estudy 한국어/English

LLM/Agent Quantitative Evaluation System

1,042 단어·3 분·원문(.md)

"How can we prove whether an Agent's answer quality has improved or deteriorated when applying a new prompt or replacing a search index?"

Unlike traditional software, simple unit tests in the form of assert expected == actual cannot validate the non-deterministic text output of LLMs, where words change slightly each time.

If humans cannot manually read and score thousands of answers, how should quality changes be measured with automated metrics?

To solve the above problem, the platform team is building an LLM Evaluation Pipeline. It's an automated system that scores model responses from various angles.

  • LLM as a Judge: This technique uses a model with strong reasoning capabilities as a scorer to evaluate how well the text generated by the target Agent meets predefined scoring criteria.
  • Faithfulness: A metric that measures whether the generated answer is based solely on the content of documents retrieved from RAG and if there are any hallucinations.
  • Answer Relevance: A metric that measures whether the generated answer accurately aligns with the user's original question intent.
  • Context Precision: A metric that measures whether the search module effectively retrieves the necessary relevant documents for answering the question in the top rankings.

Problem Definition #

The problem existed where traditional test automation was impossible due to the LLM's characteristic of varying responses each time,

and there was a limitation where deployment speed was delayed due to reliance on subjective and manual human evaluation whenever prompts or RAG pipelines were modified.

For example, consider a scenario where Team B replaces the embedding model in their Vector DB to improve search quality. They then face the problem of not being able to provide quantified metrics on how much customer response accuracy has actually improved, or how much hallucination (referencing irrelevant documents) has increased, thus preventing them from deciding on production deployment.

Solution #

  • Introduction of an evaluation pipeline based on the RAGAS framework: This clearly separates answer quality not just into good/bad, but into search quality (Context Precision, Recall) and generation quality (Faithfulness, Answer Relevance), allowing us to track where performance degradation occurs (Retriever or Generator).
  • CI/CD integrated evaluation: The platform team provides infrastructure that, when internal teams provide only evaluation datasets (questions, correct contexts, ideal answers), automatically runs thousands of evaluations during pipeline builds in Jenkins or GitHub Actions, reporting the quality scores of the modified agent to a dashboard.

Detailed Operating Principles and Structure #

This is the logical flow for the platform to automatically evaluate other teams' Agents and generate metrics.

graph TD
    Dataset[(Evaluation Dataset\nQuestions, Ideal Answers)] --> Tester[Platform Evaluation Engine]
    
    subgraph "Target RAG Agent"
        Tester -->|1. Input Test Query| Agent[Internal Service Agent]
        Agent -->|2. Retrieved Documents (Contexts)| Tester
        Agent -->|3. Generated Answer (Answer)| Tester
    end
    
    subgraph "LLM-as-a-Judge (Evaluator)"
        Tester -->|4. Combine Evaluation Prompts| Judge[GPT-4 Evaluator]
        Judge -->|5. Scoring (Calculate Faithfulness)| Metric1[Faithfulness: 0.95]
        Judge -->|5. Scoring (Calculate Relevance)| Metric2[Relevance: 0.88]
    end
    
    Metric1 --> Dashboard[Platform Dashboard\nQuality Report]
    Metric2 --> Dashboard
  1. Test Data Injection: The evaluation engine sends pre-prepared queries to the target agent.
  2. Response and Context Injection: Collect both the original documents referenced by the Agent to generate the answer and the final generated text.
  3. LLM Evaluator Call: The engine combines the question, retrieved documents, and generated answer, passes them to the evaluation LLM, and requests scores for each metric.
  4. Score Aggregation: The numbers 0.0 ~ 1.0 returned by the evaluation LLM are averaged and used as the final quality metric.

Example #

To understand the principle without a separate library, it's about how an LLM can be made to evaluate whether an answer hallucinates, scoring it 1 point (failure) or 5 points (success).

import json
import openai

def evaluate_faithfulness(question: str, context: str, answer: str) -> dict:
    """Scores whether the answer is based solely on the given context, using an LLM as a Judge."""
    
    evaluator_prompt = f"""
    You are a strict evaluator. Evaluate whether the given [Answer] is based solely on the content of the [Reference Document].
    If it fabricated content not in the document (Hallucination), score 1 point. If it answered accurately using only the document's content, score 5 points.
    
    The result must be returned strictly in JSON format. Example: {{"score": 5, "reason": "Matches the content of the document"}}
    
    [Question]: {question}
    [Reference Document]: {context}
    [Answer]: {answer}
    """
    
    client = openai.OpenAI(api_key="sk-...")
    response = client.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": evaluator_prompt}],
        response_format={ "type": "json_object" } # JSON 응답 강제
    )
    
    return json.loads(response.choices[0].message.content)

# Principle test
# print(evaluate_faithfulness("How many vacation days?", "Regular employee vacation is 15 days.", "It's 15 days.")) -> score: 5
# print(evaluate_faithfulness("How many vacation days?", "Regular employee vacation is 15 days.", "It's 20 days.")) -> score: 1

Looking at the Evaluation API (using RAGAS),

this is the evaluation pipeline script provided by the platform team to internal developers.

Other teams can obtain a quantified quality report through the validated RAGAS algorithm simply by passing their specifically constructed dataset (e.g., HuggingFace Dataset format) to this function.

from datasets import Dataset
from ragas import evaluate
from ragas.metrics import (
    faithfulness,
    answer_relevancy,
    context_precision
)
from langchain_openai import ChatOpenAI

def run_platform_evaluation(test_data_dict: dict):
    """
    Performs RAGAS-based quantitative evaluation using test data provided by internal product teams.
    """
    
    # 1. Convert input data to HuggingFace Dataset format required by RAGAS
    # Required keys: question, answer, contexts(List), ground_truth(ideal answer)
    dataset = Dataset.from_dict(test_data_dict)
    
    # 2. Configure high-performance LLM for evaluation (platform internal Gateway recommended)
    # Even if serving with a fast and cheap model, evaluation must always be performed with the smartest model (e.g., GPT-4) for accuracy.
    evaluator_llm = ChatOpenAI(model_name="gpt-4", temperature=0.0)
    
    # 3. Execute RAGAS evaluation (supports asynchronous parallel processing)
    print("Platform evaluation engine running... Starting metrics calculation.")
    result = evaluate(
        dataset=dataset,
        metrics=[
            faithfulness,      # Is the answer based on the context? (Hallucination measurement)
            answer_relevancy,  # Is the question and answer relevant?
            context_precision  # Are the retrieved documents useful? (Search engine performance)
        ],
        llm=evaluator_llm
    )
    
    # 4. Output results and save to dashboard (e.g., convert to pandas DataFrame)
    print("\n[Evaluation Complete] Final Quality Metrics Report:")
    print(result)
    
    return result.to_pandas()

# --- Internal Team Usage Example ---
# data = {
#     "question": ["How much are the welfare points for 2026?"],
#     "answer": ["It's 1 million won."],
#     "contexts": [["Welfare points for all employees in 2026 have been increased to 1 million won."]],
#     "ground_truth": ["1 million won"]
# }
# report_df = run_platform_evaluation(data)
# save_to_dashboard(report_df)
AI/llmagent/evaluation.md