# Evaliphy - End-to-End AI Testing Framework

Evaliphy simplifies end-to-end AI testing. It treats AI systems as a black box so teams can validate behavior with clear assertions, CI workflows, and human-readable reports without ML-heavy setup.

## Key Features
- **TypeScript-native**: Eval files sit in your repo alongside other tests.
- **CI/CD Ready**: Runs in CI like any other test suite.
- **Real API Testing**: Makes HTTP calls to your actual running service.
- **Built-in Judges**: Includes assertions for Faithfulness, Relevance, and Groundedness.
- **Playwright-like Mental Model**: Assertion style similar to Jest, Vitest, or Playwright.
- **No Vendor Lock-In**: Open source and provider-agnostic.

## Quick Start
```bash
npm install -g evaliphy
evaliphy init my-project
```

## Core Concepts
- **Assertions**: Use `expect(query, context, response).toBeFaithful()` to validate AI outputs.
- **HTTP Client**: Configurable client to interact with your AI API.
- **LLM-as-a-Judge**: Uses LLMs to evaluate the quality of responses based on retrieved context.

## Documentation Sections
- **Introduction**: Overview and motivation.
- **Quick Start**: Getting up and running.
- **How Evaliphy Works**: Architecture and workflow.
- **Assertions API**: Detailed guide on available assertions.
- **Configuration**: How to customize Evaliphy for your project.
- **Reporting**: Understanding evaluation results.

## Comparison with Other Tools
Evaliphy focuses on production AI testing in CI/CD with simple assertions and readable reports, while many alternatives are optimized for research workflows, benchmarking, or prompt experimentation.