# Quality Lab — full-stack QA showcase Quality Lab is a deliberately testable example website and a small, portfolio-ready test suite. It covers real browser behavior, API contracts, accessibility, performance, and security, while producing reports that are useful to both engineers and non-technical reviewers. The included site is not a toy static page: it has an Express API, authentication, public filtering, protected task mutations, validation, responsive UI states, rate limiting, and security headers. All data is intentionally in memory and resets when the server restarts. ## Coverage and outputs | Layer | Primary tooling | What it checks | Report | |---|---|---|---| | Functional | Playwright | navigation, filters, login/logout, task CRUD, form validation, desktop/mobile | Playwright HTML, JUnit, JSON, trace/video on failure | | API | Playwright + pytest | auth, schemas, CRUD, status/error contracts, latency | Playwright HTML + self-contained pytest HTML/JUnit | | Accessibility | axe-core + Playwright | WCAG A/AA rules, landmarks, skip link, keyboard/dialog focus | Playwright HTML with violation details | | Performance | Playwright + Lighthouse | timing/resource budgets, performance, best practices, SEO | Playwright + Lighthouse HTML/JSON | | Security | Playwright + Helmet + npm audit + optional ZAP | headers, authorization, logout, input limits, rate limiting, dependencies, passive DAST | Playwright, dependency HTML/JSON, ZAP HTML/JSON | | AI-assisted QA | OpenAI Responses API + Playwright | blind multimodal bug decision, visual/usability review, and evidence-based failure triage | Structured JSON + portfolio-style HTML | After a full run, open `reports/index.html`. It is a polished landing page linking every generated report. ## Prerequisites - Node.js 24 LTS or newer - Python 3.11 or newer - Docker only if running the optional ZAP scan - An OpenAI API key only if running live AI-assisted reviews - A desktop display, or `xvfb`, for the visible headed demo ## Install ```bash npm install npx playwright install --with-deps chromium firefox npm run setup:python ``` Start the example application at : ```bash npm start ``` Demo credentials are prefilled in the UI: ```text qa@example.com Portfolio123! ``` ## Run the suite ```bash # Every required layer; continues through failures so every report is produced npm run test:full # Focused layers npm run test:functional npm run test:api npm run test:a11y npm run test:performance:browser npm run test:lighthouse npm run test:security npm run test:python npm run test:ai:unit ``` The Python runner automatically prefers `.venv` when it contains the required packages. If the Python layer reports missing dependencies, run `npm run setup:python` once; no manual virtual-environment activation is required. Open Playwright’s native report with `npm run report`. Open the cross-tool hub by serving the repository or opening `reports/index.html` directly. ## AI-assisted visual review and failure triage The optional AI layer complements exact assertions in two places: - **Visual review** captures desktop and Pixel 7 full-page screenshots, then asks a vision-capable model for an explicit bug verdict plus evidence-visible usability, responsive-design, content, and semantic-accessibility observations. - **Failure triage** reads only failed Playwright specs, bounded error/source excerpts, and up to three failure screenshots. It classifies likely product, test, environment, or timing causes and recommends a deterministic follow-up. Both workflows use the OpenAI Responses API with strict JSON-schema output and `store: false`. GPT-5.6 Luna is the cost-sensitive default and can be overridden with `OPENAI_MODEL`. OpenAI documents that current models accept image input and that Structured Outputs enforce a supplied schema: [model overview](https://developers.openai.com/api/docs/models) and [Responses API](https://developers.openai.com/api/reference/java/resources/beta/subresources/responses). Generate the complete portfolio report without a key or API charge: ```bash npm run test:ai:demo ``` Run the same fixture-based demonstration as a visible presentation—with scrolling evidence capture, section highlights, screenshot flashes, a decision screen, automatic report reveal, and retained video: ```bash npm run test:ai:demo:headed ``` The fixture showcase demonstrates the workflow but does not claim to perform fresh reasoning. For a genuinely live blind review, put the key in the gitignored `.env` file and run: ```bash npm run test:ai:showcase # visible capture → live model decision → report tour npm run test:ai:showcase:xvfb # same workflow on Linux without a desktop display ``` The live prompt is not given a seeded defect, expected answer, or application source code. It receives clean screenshots, neutral semantic/layout evidence, console failures, and failed requests. It must choose `no-bug-observed`, `possible-bug`, or `probable-bug`, explain the strongest supplied evidence and uncertainty, and suggest deterministic follow-ups. That is real model judgment, but still advisory rather than proof of a defect. For a live review, copy `.env.example` to `.env`, add `OPENAI_API_KEY`, and run: ```bash npm run test:ai:visual # capture and review the current UI npm run test:ai:triage # triage failures in reports/playwright/results.json npm run test:ai # unit contracts + both live stages + report hub ``` Open `reports/ai/index.html`, or use the AI QA card in `reports/index.html`. AI findings are advisory by default. `AI_FAIL_ON=high`, `medium`, `low`, or `info` can make visual findings fail a command, but should only be enabled after measuring repeatability on representative runs. Visible AI runs retain WebM video under `test-results/ai-showcase/`. Adjust `AI_SHOWCASE_STEP_MS` and `AI_SHOWCASE_HOLD_MS` in `.env` to pace a recording, or set `AI_SHOWCASE_VIDEO=off` to disable built-in capture. Do not submit production credentials, personal data, or unreviewed proprietary artifacts. The triage parser excludes the Playwright configuration and process environment, redacts common token patterns, caps artifact size, and only accepts screenshot paths inside `reports/` or `test-results/`. The optional passive dynamic scan pulls the maintained OWASP ZAP container: ```bash npm run scan:zap:local npm run report:index ``` ZAP warnings are reported but do not fail the baseline command; actual failures still return a failing exit code. This keeps the example informative without teaching teams to silently ignore high-confidence findings. ## Visible portfolio workflow `npm run demo` launches Chromium headed, slows each Playwright action, adds on-screen step cues/highlights, and records video. It walks through the coverage cards, API-backed filtering, login, authenticated creation, and contact submission. ```bash npm run demo # visible browser on a desktop npm run demo:xvfb # visible virtual display on Linux npm run demo:record # headless capture when no display is available ``` The video is retained under `test-results/demo/`; its exact path is linked from `reports/demo/index.html`. For a clean portfolio recording, capture the headed browser window with OBS or the operating system’s recorder. The built-in video is a reliable backup artifact and does not overwrite the main suite report. ## CI integration [`.github/workflows/quality-suite.yml`](.github/workflows/quality-suite.yml) is ready for GitHub Actions. It: 1. installs pinned JavaScript/Python dependencies and Playwright browsers; 2. runs browser, API, axe, Lighthouse, pytest, and dependency quality gates; 3. uploads reports, traces, screenshots, and videos even when a gate fails; and 4. runs an independent OWASP ZAP baseline job and uploads its report. On a failed deterministic run, CI performs advisory AI triage when the `OPENAI_API_KEY` repository secret exists. A manually dispatched workflow can also enable **Run the optional multimodal AI visual review**. Forked pull requests without the secret simply skip AI calls. The workflow runs on pushes to `main`, pull requests, and manual dispatch. Adjust branches or split layers into a larger matrix as the project grows. ## Project map ```text app/ Express API and testable frontend tests/ Playwright suites by quality layer python_tests/ Independent pytest contract suite scripts/ Lighthouse, audit, orchestration, report hub scripts/ai/ bounded OpenAI client, capture, triage, and AI report ai_tests/ offline contracts, request-shape, and redaction tests fixtures/ai/ clearly labeled no-cost portfolio demonstration data reports/ Generated human-readable artifacts (ignored) .github/workflows/ CI pipeline playwright.config.js browsers, video/trace, reporters, local server ``` ## Deliberate scope This is a compact example, not a substitute for production threat modeling, authenticated ZAP policies, load testing in an isolated environment, or manual accessibility review. The structure is designed so those layers can be added without changing the core workflows.