DeepEval is an open-source LLM evaluation framework, built and maintained by Confident AI, for testing and benchmarking LLM applications. It provides 50+ research-backed metrics (including LLM-as-a-judge, RAG, agent, multi-turn, voice, safety, and multimodal metrics) and runs Pytest-style assertions locally or in CI/CD pipelines. It targets AI engineers, data scientists, QAs, and PMs evaluating agents, chatbots, RAG pipelines, and MCP systems, and integrates with frameworks and model providers such as LangChain, OpenAI, Anthropic, and Gemini.