Scenario-based testing & workflow execution framework.
Probitas is a comprehensive framework for scenario-based testing and workflow execution, providing:
- Type-safe scenario builder - Fluent API for defining test workflows with automatic type inference
- Powerful CLI - Command-line interface for running, listing, and managing scenarios
- Rich client ecosystem - Pre-configured clients for HTTP, databases, message queues, and more
- Flexible execution - Run scenarios in parallel or sequentially with configurable concurrency
- Smart assertions - Client-specific expectation utilities for comprehensive testing
curl -fsSL https://raw.githubusercontent.com/probitas-test/probitas/main/install.sh | bashOptions via environment variables:
# Install specific version curl -fsSL https://raw.githubusercontent.com/probitas-test/probitas/main/install.sh | PROBITAS_VERSION=0.7.1 bash # Install to custom directory curl -fsSL https://raw.githubusercontent.com/probitas-test/probitas/main/install.sh | PROBITAS_INSTALL_DIR=/usr/local/bin bash# Add the tap brew tap probitas-test/tap # Install probitas brew install probitas # Upgrade to latest version brew upgrade probitasSee probitas-test/homebrew-tap for more details.
# Run without installing nix run github:probitas-test/probitas # Install into your profile nix profile install github:probitas-test/probitasOr add to your project's flake.nix:
{ inputs = { nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable"; probitas-flake.url = "github:probitas-test/probitas"; flake-utils.url = "github:numtide/flake-utils"; }; outputs = { nixpkgs, probitas-flake, flake-utils, ... }: flake-utils.lib.eachDefaultSystem (system: let pkgs = import nixpkgs { inherit system; overlays = [ probitas-flake.overlays.default ]; }; in { devShells.default = pkgs.mkShell { packages = with pkgs; [ probitas ]; }; }); }Then enter the development shell:
nix develop probitas runThe recommended way to use Probitas in GitHub Actions:
- uses: probitas-test/setup-probitas@v1 with: version: latest # or specific version like '0.7.1' - name: Run scenarios run: probitas runSee probitas-test/setup-probitas for full documentation.
For projects using Nix flakes with nixbuild/nix-quick-install-action:
- uses: nixbuild/nix-quick-install-action@v34 - name: Run scenarios run: nix run github:probitas-test/probitas -- runOr within a Nix development shell:
- uses: nixbuild/nix-quick-install-action@v34 - name: Run scenarios run: nix develop -c probitas runCreate probitas/hello.probitas.ts:
import { client, expect, scenario, Skip } from "jsr:@probitas/probitas@^0"; const apiUrl = Deno.env.get("API_URL"); export default scenario("User API Test", { tags: ["api", "user"] }) .step("Check API availability", () => { if (!apiUrl) { throw new Skip("API_URL not configured"); } }) .resource("http", () => client.http.createHttpClient({ url: apiUrl! })) .step("Create user", async (ctx) => { const response = await ctx.resources.http.post("/users", { body: { name: "Alice", email: "alice@example.com" }, }); expect(response).toBeOk().toHaveStatus(201); return response.json as { id: string }; }) .step("Verify user exists", async (ctx) => { const { id } = ctx.previous!; const response = await ctx.resources.http.get(`/users/${id}`); expect(response).toBeOk().toHaveJsonProperty("name", "Alice"); }) .build();Note
Enable Deno LSP in your editor for the best development experience. With Deno LSP enabled, you'll get:
- Auto-completion for Probitas APIs (
scenario,expect,client, etc.) - Type checking and inline error detection
- Jump to definition for functions and types
- Hover documentation for all APIs
See Deno's editor setup guide for instructions on enabling Deno in VS Code, Vim, Neovim, and other editors.
# Run all scenarios probitas run # Run scenarios with specific tag probitas run -s tag:api # Re-run only failed scenarios from previous run probitas run --failed # Run with JSON output probitas run --reporter json # List available scenarios probitas list # Initialize configuration file probitas initA scenario is a sequence of steps that execute in order. Each step can:
- Return a value that's passed to the next step via
ctx.previous - Access all previous results via
ctx.results - Share state via
ctx.store - Use resources defined at the scenario level
- Have custom timeout and retry configuration
Resources are lifecycle-managed objects with automatic cleanup (Disposable pattern):
import { client, expect, scenario } from "jsr:@probitas/probitas@^0"; export default scenario("Database Query Test", { tags: ["db"] }) .resource("db", () => client.sql.postgres.createPostgresClient({ url: Deno.env.get("DATABASE_URL")!, })) .step("Insert test data", async (ctx) => { const result = await ctx.resources.db.query( "INSERT INTO users (name) VALUES ($1) RETURNING id", ["TestUser"], ); expect(result).toBeOk().toHaveRowCount(1); return { userId: result.rows![0].id }; }) .step("Query inserted user", async (ctx) => { const { userId } = ctx.previous!; const result = await ctx.resources.db.query( "SELECT * FROM users WHERE id = $1", [userId], ); expect(result) .toBeOk() .toHaveRowCount(1) .toHaveRowsMatching({ name: "TestUser" }); }) .build(); // Resource is automatically disposed (connection closed) after scenarioFor side effects that need cleanup without creating a reusable client:
import { client, expect, scenario } from "jsr:@probitas/probitas@^0"; export default scenario("Redis Cache Test", { tags: ["redis", "cache"] }) .resource("redis", () => client.redis.createRedisClient({ url: Deno.env.get("REDIS_URL")!, })) .setup(async (ctx) => { // Create test keys before scenario await ctx.resources.redis.set("test:counter", "0"); // Return cleanup function return async () => { await ctx.resources.redis.del(["test:counter"]); }; }) .step("Increment counter", async (ctx) => { const result = await ctx.resources.redis.incr("test:counter"); expect(result).toBeOk().toHaveValue(1); }) .step("Verify counter value", async (ctx) => { const result = await ctx.resources.redis.get("test:counter"); expect(result).toBeOk().toHaveValue("1"); }) .build();Organize scenarios with tags for filtering:
import { client, expect, scenario } from "jsr:@probitas/probitas@^0"; export default scenario("Login Flow", { tags: ["auth", "critical", "e2e"] }) .resource( "http", () => client.http.createHttpClient({ url: Deno.env.get("API_URL")! }), ) .step("Login with valid credentials", async (ctx) => { const response = await ctx.resources.http.post("/auth/login", { body: { email: "test@example.com", password: "secret" }, }); expect(response).toBeOk().toHaveStatus(200); return response.json as { token: string }; }) .step("Access protected resource", async (ctx) => { const { token } = ctx.previous!; const response = await ctx.resources.http.get("/profile", { headers: { Authorization: `Bearer ${token}` }, }); expect(response).toBeOk().toHaveJsonProperty("email", "test@example.com"); }) .build();Run with tag filters:
probitas run -s tag:auth probitas run -s "tag:critical,tag:auth" # AND logic probitas run -s "!tag:slow" # NOT logicCreate a probitas.json file in your project root:
{ "includes": ["probitas/**/*.probitas.ts"], "excludes": ["**/*.skip.probitas.ts"], "reporter": "list", "maxConcurrency": 4, "maxFailures": 0, "timeout": "30s", "selectors": ["!tag:wip"] }Configuration options:
includes- Glob patterns for scenario files to include (default:["**/*.probitas.ts"])excludes- Glob patterns for scenario files to exclude (default:[])reporter- Output format:listorjson(default:list)maxConcurrency- Maximum number of scenarios to run in parallel (default: number of CPU cores)maxFailures- Stop execution after N failures, 0 for unlimited (default:0)timeout- Default timeout for scenarios (e.g.,30s,5m,1h)selectors- Default selectors to filter scenarios (e.g.,["tag:api"],["!tag:slow"])
Generate a default configuration:
probitas initProbitas provides pre-configured clients for common services:
- HTTP - REST API testing (
client.http)- OIDC Authentication - OpenID Connect authentication for HTTP clients (
client.http.oidc)
- OIDC Authentication - OpenID Connect authentication for HTTP clients (
- GraphQL - GraphQL query testing (
client.graphql) - ConnectRPC - RPC service testing (
client.connectrpc) - gRPC - gRPC service testing (
client.grpc) - SQL Databases - PostgreSQL, MySQL, SQLite, DuckDB (
client.sql.*) - Redis - Redis operations (
client.redis) - MongoDB - MongoDB operations (
client.mongodb) - Deno KV - Deno's key-value store (
client.denoKv) - RabbitMQ - Message queue operations (
client.rabbitmq) - AWS SQS - SQS queue operations (
client.sqs)
Each client comes with specialized assertion utilities via the expect() function.
The HTTP client supports OpenID Connect (OIDC) authentication for testing APIs that require OAuth 2.0/OIDC authentication:
import { client, expect, scenario } from "jsr:@probitas/probitas@^0"; export default scenario("OIDC Authentication Test", { tags: ["auth", "oidc"] }) .resource( "http", async () => await client.http.oidc.createOidcHttpClient({ url: Deno.env.get("API_URL")!, oidc: { issuer: Deno.env.get("OIDC_ISSUER")!, clientId: "my-client-id", }, }), ) .step("Login with OIDC", async (ctx) => { const result = await ctx.resources.http.login({ type: "authorization_code", username: "user@example.com", password: "password", }); expect(result.ok).toBe(true); }) .step("Access protected resource", async (ctx) => { // Authorization header is automatically added const response = await ctx.resources.http.get("/protected"); expect(response).toBeOk().toHaveStatus(200); }) .build();The OIDC client supports:
- Automatic token discovery - Discovers OIDC endpoints from issuer URL
- Manual configuration - Or configure
authUrlandtokenUrldirectly - Authorization Code Grant - For user authentication flows
- Automatic token management - Handles token storage and Authorization header injection
Probitas provides client-specific expectation functions:
import { client, expect, scenario } from "jsr:@probitas/probitas@^0"; export default scenario("E-Commerce Order Flow", { tags: ["e2e", "order"] }) .resource( "http", () => client.http.createHttpClient({ url: Deno.env.get("API_URL")! }), ) .resource("db", () => client.sql.postgres.createPostgresClient({ url: Deno.env.get("DATABASE_URL")!, })) .step("Create order via API", async (ctx) => { const response = await ctx.resources.http.post("/orders", { body: { items: [{ productId: "prod-1", quantity: 2 }] }, }); // HTTP-specific assertions expect(response) .toBeOk() .toHaveStatus(201) .toHaveHeadersProperty("content-type", /application\/json/); return response.json as { orderId: string }; }) .step("Verify order in database", async (ctx) => { const { orderId } = ctx.previous!; const result = await ctx.resources.db.query( "SELECT * FROM orders WHERE id = $1", [orderId], ); // SQL-specific assertions expect(result) .toBeOk() .toHaveRowCount(1) .toHaveRowsMatching({ status: "pending" }); }) .step("Validate order total", async (ctx) => { const { orderId } = ctx.results[0] as { orderId: string }; const response = await ctx.resources.http.get(`/orders/${orderId}`); const order = response.json as { total: number }; // Generic value assertions (chainable) expect(order.total).toBeGreaterThan(0).toBeLessThan(10000); }) .build();# Run scenarios probitas run [options] # List scenarios without running probitas list [options] # Initialize configuration file probitas init # Cache scenario dependencies probitas cache [options] # Format scenario files probitas fmt # Lint scenario files probitas lint # Type check scenario files probitas check # Show version probitas --version # Show help probitas --helpChoose output format based on your needs:
list- Detailed human-readable output (default)json- Machine-readable JSON for CI/CD integration
probitas run --reporter jsonProbitas provides a Claude Code plugin to enhance scenario development with AI assistance:
# Add the plugin marketplace /plugin marketplace add probitas-test/claude-plugins # Install the Probitas plugin /plugin install probitas@probitas-testOr add to your project's .claude/settings.json:
{ "plugins": { "marketplaces": ["probitas-test/claude-plugins"], "installed": ["probitas@probitas-test"] }, "enabledPlugins": { "probitas@probitas-test": true } }The plugin provides:
- Scenario scaffolding - Generate scenario templates with common patterns
- Assertion suggestions - Context-aware recommendations for expect calls
- Client integration - Auto-complete for client configurations
- Error diagnostics - AI-powered debugging for failing scenarios
See probitas-test/claude-plugins for more details.
For AI assistants and language models, comprehensive documentation is available in machine-readable format:
- Documentation website: probitas-test.github.io
- LLM-optimized: llms.txt - Structured documentation following the llms.txt standard
- Markdown format: All documentation provides
index.mdfiles for easy access and parsing
For detailed API reference, use deno doc to view type signatures and documentation directly from the package:
# View expect API documentation deno doc jsr:@probitas/probitas/expect # View client API documentation deno doc jsr:@probitas/probitas/client # View main package documentation deno doc jsr:@probitas/probitasThis provides the most accurate and up-to-date type information directly from the source code.
The combination of narrative documentation (index.md/llms.txt) and API reference (deno doc) enables AI assistants to provide accurate, comprehensive guidance when working with Probitas.
Interested in contributing or maintaining Probitas? See CONTRIBUTING.md for:
- Development environment setup
- Release process and workflow
- Architecture overview
- Related repositories
Probitas is greatly inspired by runn, an excellent scenario-based testing tool. Many core concepts in Probitas, such as:
- Accessing previous step results via
ctx.previous - Integration of database clients and other service helpers beyond traditional e2e testing scope
- Scenario-based workflow execution model
are influenced by runn's thoughtful design. We express our deep gratitude and respect to the runn project and its maintainers for pioneering this approach to scenario-based testing.
See LICENSE file for details.