From upload to verdict in seven steps
No agents, no SDKs, no test scripts to write. Bring the collection you already have — BugVilla tests every endpoint as every role that uses your API.
Seven steps, three of them yours
You create the project, upload the collection, and confirm credentials. BugVilla parses, detects, generates, runs, and reports.
Create a project
Name it and you're done. Every project is provisioned with four testing surfaces — API is live today; UI, application workflow and VAPT are on the roadmap. Everything below happens on the API surface.
Upload a collection
Postman Collection v2.1, OpenAPI 3.x or Swagger 2.0 — JSON or YAML — or raw cURL. The parser extracts method, URL, path and query params, headers, body schema, auth type and examples for every endpoint. Zero annotations required.
BugVilla detects personas
Before you type a credential, the detector reads the parsed collection and works out what to ask for: the login endpoint — scored on its path and whether its body is password-shaped, because a path alone can't separate /auth/login from /auth/me — the credential fields, where the token sits in the response, and role hints from folder names and tags. Nothing is saved until you confirm.
Confirm credentials
Review the suggested personas — Admin, Member, an unauthenticated stranger. Choose an auth mode per persona: login fetches a fresh token each run, static takes a pasted token, API key or cookie, none deliberately sends nothing. Hit verify and BugVilla logs each persona in to prove the credentials work before a single test exists.
Generate suites
One click builds 7 test categories: functional happy paths, contract checks against the JSON schema (AJV), negative cases, boundary and fuzz payloads, security probes, performance baselines — plus the collection-wide authorization matrix. For the 42-endpoint collection above, that's hundreds of cases — none written by hand.
Run as every persona
Runs are async with live progress. Each persona logs in once per run — not once per case, which rate limiters would read as an attack. Extracted values like tokens and created ids chain into later requests automatically. A persona that fails to authenticate doesn't abort the run; its cases are reported as errored, so you know exactly which role is misconfigured.
Read the report
One dataset, four formats: PDF and HTML for people, CSV and JSON for machines — a finding can't appear in one and not another. Authorization results split into exposures (reached something it shouldn't) and wrongly blocked (a broken grant). The HTML report is fully self-contained: email it, open it offline. Then squash what it found.
The access matrix, explained
Every protected endpoint, replayed as every persona, asserted against an expected outcome. Here's what run #1284 produced.
Two failures, two different problems. A Member deleting an invoice is an exposure — the vulnerability. An Admin blocked from a role change is wrongly blocked — a broken grant. The report never mixes them up. Note the ✓ where a persona was correctly denied: a rejection that should happen is a pass, not a failure.
Deny is strict
A deny expectation fails on any 2xx — the persona got in, and that's the vulnerability. It also fails on any 5xx: the request reached handler code before the permission check, so the guard wasn't the first thing standing.
Allow is forgiving
An allow expectation fails only on 401 or 403. A 404 or 422 usually means bad test data, not bad permissions — failing on those would bury real authorization findings under setup noise.
What happens when the API changes
The first run is the easy part. BugVilla is built for the second upload, and the fiftieth.
Upload v2. Keep everything.
A new version never overwrites the old one — it joins a lineage, and every version keeps its own endpoints, suites and runs.
- Matched by METHOD /path. Internal ids don't survive a re-export; method plus path does.
- Fingerprint-diffed. The hash covers only what tests depend on — params, body schema, response schemas, auth. A docs-only re-export shows zero changes.
- Breaking changes flagged. Removed endpoint, newly required param, changed type, dropped success status, altered auth — anything that can invalidate a passing test.
- Contract diff on demand. Compare any two versions, not just adjacent ones.
- No persona re-setup. Personas belong to the project, not the version. v2 runs with the credentials you already confirmed.
After the v2 run, the run-to-run diff sorts every case into three buckets:
3 fixed 2 broken 1 still failing
Questions engineers actually ask
Does a failed persona login abort the run?
No. The run continues, and every case belonging to that persona is reported as errored with the auth failure attached. You learn exactly which role is misconfigured, instead of getting a run that never started.
How does request chaining work?
Values extracted from responses — tokens, created resource ids — are stored in run-scoped variables and substituted into later requests. Credentials are never shared between personas, but run-scoped variables are: an invoice id created as Admin is exactly what a Member needs to probe with.
Why do the categories run in that exact order?
Functional → contract → authorization → negative → boundary → security → performance. Functional runs first so the ids it creates exist for everything after it. Performance runs last, against a warm service, so latency baselines aren't polluted by cold starts.
Do I set up personas again for every new version?
No. Personas belong to the project, not the collection version. Upload v2 and it's immediately runnable with the personas — and credentials — you already confirmed for v1.
Watch your first matrix fill in
Free to start. Upload the collection you already have — the first matrix lands in under an hour, not a sprint.