Artificial Intelligence · LLMs
How would you answer an interview scenario involving LLM evaluation?
For an interview scenario involving LLM evaluation, I would first clarify the business goal, scale, constraints, and the failure or quality attribute the interviewer wants to explore. LLM evaluation measures quality across dimensions such as correctness, relevance, completeness, safety, latency, and cost. I would then explain the main alternatives and tradeoffs, identify likely failure modes, and describe how I would validate the solution through testing, observability, security controls, and recovery or rollback planning.