Tag: LLM공부
All the articles with the tag "LLM공부".
-
Eval Study #4 — HITL Agent Regression Test, Passed but for the Wrong Reason
Following the previous post (agent tool selection · multi-step eval), this is a regression test for an HITL agent. This time, instead of a (question · expected tool) test set, I wrote unit-test style functions covering 3 axes of pitfalls (entering the breakpoint for dangerous tools / responding on every turn in multi-turn conversations / not misclassifying safe tools), with sys.exit(1) as the exit code on regression. The first run passed 3/3 — but that wasn't the right answer. The "call only after confirming clear intent" prompt that had been an issue before was still lingering, and the LLM was passing the test via its own self-defense mechanism. When I provoked it with a delete case, our HITL didn't trigger at all — only the LLM's own confirmation remained. Removing the prompt → the failure shifted to the tool simply not being called at all. Eventually, minimizing the prompt to "call the tool that matches the task the user requested" made it stop exactly at the delete_user tool (HITL working, exit code 0). Passing an eval by itself isn't a safety signal — verifying why it passed is the real safety. Pausing the LLM study series here for now, to resume after wrapping up backend studies.
-
Eval Study #3 — Agent Eval, the Limits of Single-Step Scoring, and Multi-Step Grading
Expanding from RAG Eval to agent Eval. Agents have 6 axes to judge (tool selection, argument extraction, multi-step trajectory, termination judgment, safety guards, final answer quality), so separating axes—like unit tests before integration tests—makes it faster to pinpoint causes. Today I covered just 3: tool selection, multi-step, and trap regression. I fed in MCP server tool metadata (name, description, schema) and measured against a (question, expected tool, expected args, level) test set → tool selection accuracy 11/13 = 84.6%. Two failure cases were interesting — (1) a single-step eval falsely flagged a case that was actually correct as a multi-step trajectory, showing the limits of single-step scoring, and (2) an ambiguous tool description caused "electricity bill" to wrongly pick consumption, while "how much money is it?" got it right — patching the system prompt instead of fixing the description risks overfitting. I then switched to multi-step eval (did it call all necessary tools, in order, using prior results, with correct termination judgment) → 4/4 = 100%, and the case that failed under single-step scoring now passed.
-
RAG Embedding Comparison — Measuring recall@k on My Blog Data (OpenAI vs bge-m3)
After establishing the ["feeling-based benchmarking → numeric benchmarking" principle](/en/posts/quant-study-00-pandas) in my quant retrospective, I actually quantified an embedding model comparison this time. I indexed 441 chunks from my blog posts with OpenAI text-embedding-3-small and bge-m3 respectively, then measured recall@3 with a test set of 20 question-answer source pairs. Overall: OpenAI 80% vs bge-m3 90%. bge-m3 hit 100% on hard-difficulty questions — the decisive factor was connecting to the source text by meaning even when words didn't overlap. On easy questions, the misses turned out to be caused by typos (cladue, underscores) — a twist showing the grading criteria itself was wrong.
-
Eval Study #2 — The Pitfalls of Similarity-Based Evaluation, 5 Principles of Test Set Design, and a Misunderstanding About the Term 'Regression Test'
My second session studying Eval. I ran similarity-based evaluation (embedding cosine similarity) myself and got an unexpected result — both OpenAI's text-embedding-3-small and bge-m3 gave the highest score to the 'wrong answer.' This is because similarity captures topical/expressive closeness, not content correctness. Just a difference in markdown formatting can swing similarity scores significantly. In practice, combining similarity with LLM-as-Judge is the standard approach. Also covers 5 principles of test set design, plus a terminology correction: it's not a 'regression test,' it's improvement validation (A/B).