Artificial Intelligence · RAG
How would you answer an interview scenario involving RAG evaluation?
For an interview scenario involving RAG evaluation, I would first clarify the business goal, scale, constraints, and the failure or quality attribute the interviewer wants to explore. RAG evaluation measures retrieval quality and generation quality separately as well as end-to-end usefulness. I would then explain the main alternatives and tradeoffs, identify likely failure modes, and describe how I would validate the solution through testing, observability, security controls, and recovery or rollback planning.