KwikQA
Sign in Get started

Releases

Changelog

These are the releases of the KwikQA CLI on npm, and npx kwikqa --version shows which one you run.

0.4.1

Added

  • Shared page-object and fixture repairs in Selenium Java and pytest suites are validated across the tests they affect, end to end. A suite kept in a subdirectory of the repository is validated from its own build root, and a pytest module that names fixtures in literal usefixtures marks has those fixtures followed.

Changed

  • Diagnosis of a Selenium Java run also searches recent changes for the expected and received values of TestNG and JUnit assertions.
  • A page object's own locators are kept in the evidence, and recent changes are searched for the text they select.
  • When many tests fail through one shared page object, the request carries each source excerpt once and keeps the page object's methods and locators within its size limit.

Fixed

  • A repair to shared test support is no longer refused because a module that none of its affected tests reach has imports that KwikQA can't bound or parse, such as build tooling with dynamic imports.
  • Java sources in a Maven or Gradle suite two levels below the workspace root are found, so their failures reach the evidence.
  • Change evidence no longer dates commits against a run whose report doesn't record its start, such as a Maven Surefire report. The search takes the last commits instead.

0.4.0

Added

  • kwikqa check checks test changes that KwikQA didn't write, such as an agent's edits or Playwright's healer's. It applies the same gates as KwikQA's own repairs: scope and policy, ownership from an earlier analyze of the failing run, and validation. It asks before running tests (--yes approves), exits 0 only when every gate passes, and sends nothing to KwikQA. In CI, it publishes its verdict where kwikqa ci publishes its summary.
  • Selenium Java suites (Maven or Gradle, with TestNG or JUnit 5), pytest suites (with Selenium or Playwright), and Cucumber-JVM suites (Maven or Gradle) are diagnosed, repaired, and validated with the project's own Maven, Gradle, or pytest. The diagnosis reads their page objects, base classes, step definitions, and pytest fixtures. Maven and Gradle validation is unavailable on Windows. KwikQA never installs or downloads these tools.
  • Bring-your-own-key endpoints: KWIKQA_ENDPOINT chooses where a BYOK plan's requests go: openrouter (the default), anthropic, openai, azure-openai, bedrock, or github-models. Credentials come from the environment, so a CI job sets them from its own secrets. For a direct endpoint, its model key (such as ANTHROPIC_API_KEY) is removed from the environment of the test runs KwikQA starts.
  • More than one fix pull request per CI run: ci.fix.maxPullRequestsPerRun (1 to 5, default 1) repairs up to that many failure groups and opens a draft pull request for each one that validates. The plan sets the most: 5 on Enterprise, 1 on Pro and the team plans.
  • Shard-aware deduplication on Enterprise plans: each job of a sharded run asks the service which of its failure groups another job of the same run already diagnoses, and skips those. Set KWIKQA_SHARD where a job's own ID isn't unique per shard.

Changed

  • Repairs to shared test support, such as a page object, fixture, helper, or config file, are validated over every test that uses them instead of being refused. Before the patch is applied, those tests run once on the current code, after you approve. Afterward, each must pass again: three times in a row in Playwright, Cypress, and WebdriverIO runs, and once in Maven, Gradle, and pytest runs.
  • fix and check refuse expect(...).toPass() as a retry. An exact expected value may be replaced by a pattern, such as a regular expression or expect.any, only with cited evidence that the value varies.
  • CI use follows the plan. On Hobby, kwikqa ci analyzes CI runs locally and says why. Pro and BYOK run hosted analysis in CI on one repository of your own. Organization CI, ci.fix beyond that repository, and Slack and Teams alerts need a team plan.
  • Each request in CI that counts toward your plan also reports where the job runs: the CI system, the repository's host, owner, and name, its visibility when known, and the pull request author's login. The service keeps the repository and the author only as keyed hashes.
  • The plans are Hobby (the free plan), Pro, BYOK, Teams, BYOK Teams, Enterprise, and BYOK Enterprise. Until the paid plans open, every account uses Hobby. Prices and limits are in the Plans section of the README.

0.3.1

Fixed

  • A fix pull request's title names the kind of failure when its group has no symptom category, such as "a failure on a locator" or "a failed network request". It counts the failing tests when there are several.

0.3.0

Release candidates 0.3.0-rc.1 and 0.3.0-rc.2 were published to npm's next tag first. The notes below cover all three.

Added

  • Fix pull requests in CI. With a committed ci.fix policy ({"ci": {"fix": {"pullRequests": "draft"}}} in kwikqa.config.json), kwikqa fix in GitHub Actions, Azure Pipelines, or GitLab CI/CD repairs a failure on the runner, validates the repair, and opens a draft pull request labeled kwikqa. On GitLab it is a draft merge request. A person reviews and merges it. GitHub needs "Allow GitHub Actions to create and approve pull requests" turned on. Jenkins still needs a terminal for fix.
  • One fix per failure group. While the group's fix pull request is open, a later run of the same failure asks for no repair and updates one note on that pull request.
  • For a fork's pull request, fix is refused before any request is sent, and the job summary says what a maintainer can run.

0.2.5

Added

  • Jenkins: kwikqa ci prints its summary in the build's console log, with a link to the build, and comments on a GitHub or GitLab pull request through that host's token. A rebuild updates the comment. Bitbucket gets no comment yet.

Fixed

  • A refused GitLab note names the permissions a fine-grained GITLAB_TOKEN needs (Work Item read, create, and update on the project), as well as the api scope.

0.2.4

Added

  • GitLab CI/CD: kwikqa ci prints its summary in the job log and posts one merge request note, which a re-run updates. It needs KWIKQA_TOKEN and a GITLAB_TOKEN of your own, stored as masked variables, because CI_JOB_TOKEN can't write notes. A branch pipeline gets a note only when its branch has exactly one open merge request.

Changed

  • When a CI run finds a repairable failure, the hint says to run kwikqa analyze and then kwikqa fix on your machine, because fix needs a terminal.

Fixed

  • On Azure Pipelines, a refused pull request comment names the identity that Azure DevOps refused, which is the identity to give the "Contribute to pull requests" permission.

0.2.3

Added

  • Azure Pipelines: kwikqa ci publishes its summary as a build summary and, on an Azure Repos pull request, as one pull request thread that a re-run updates. The thread is created closed, so it never holds up a pull request under a comment-resolution policy. The build service needs "Contribute to pull requests" on the repository.

0.2.2

Fixed

  • A Slack message is sent as one rich-text block, so each of its lines stays a line when it is copied.

0.2.1

Fixed

  • A hosted analysis no longer fails when a status check's response is cut short. The check is retried.
  • The branch line of a Slack or Teams message no longer repeats the repository, which the headline names.

0.2.0

Added

  • Failure report: kwikqa analyze --html or kwikqa report writes kwikqa-report.html, a self-contained page with each failure group and its owner, the diagnosis and the evidence it cites, flaky tests, and every request sent. It is free on every plan and needs no sign-in.
  • More test reports: analyze reads JUnit XML from any tool (Vitest, Jest, Mocha, pytest, Playwright's JUnit reporter, Maven Surefire and Failsafe, and Gradle), TestNG's XML, Allure results, CTRF reports, and Cucumber JSON. It diagnoses their failures. These runs are not repaired or validated, and fix says so before it sends anything.
  • kwikqa ci analyzes a failed CI run and publishes a summary. On GitHub Actions the summary is the job summary plus one pull request comment, which a re-run updates. Without a KwikQA token it analyzes locally. On other CI systems it writes the report and posts to chat.
  • kwikqa ci posts to Slack and Microsoft Teams through incoming webhook URLs stored as CI secrets. It posts only when tests fail, unless --notify always is given.

0.1.1

Added

  • Tests in a monorepo package can be repaired and validated. Run kwikqa from the package folder that holds the tests. KwikQA finds that package's Playwright, Cypress, or WebdriverIO in its own node_modules or in one above it, up to the repository root. It never runs a copy from outside the repository.

Changed

  • The commands KwikQA suggests use the package manager that ran it: npx, pnpm exec, yarn, or bunx.

Fixed

  • A read from the KwikQA service that loses its connection is tried once more. Writes are never retried, and a failed request names its network cause.

0.1.0

Added

  • kwikqa analyze diagnoses failed Playwright Test, Cypress, and WebdriverIO runs. Each cause cites the report fields, trace events, or source lines that support it, and KwikQA says so when the evidence isn't enough. Each group of failures gets its own diagnosis request.
  • kwikqa analyze --capture re-runs the failing tests to record full evidence: Playwright traces, and for Cypress and WebdriverIO, each failed attempt's DOM, command log, console, and network. With Playwright 1.59 or later and Playwright CLI installed, it also probes locator failures.
  • kwikqa analyze --report reads Playwright's JSON report, KwikQA's Cypress and WebdriverIO reporters, mochawesome, and JUnit reports.
  • kwikqa fix proposes a repair to test code only: specs, page objects, fixtures, and helpers that kwikqa.config.json allows. It shows the diff and applies it only after you approve.
  • kwikqa validate runs the repaired test three times in a row with retries off, and a failure rolls the patch back. KwikQA runs your project's own tools and never installs packages or browsers.
  • Repairs refuse to weaken a test: no skipped or focused tests, fixed waits, retries, forced Cypress actions, or removed or swapped assertions.
  • Recent code changes that mention the failure are sent with its evidence, so a deliberate change can be told apart from a broken app. --no-changes leaves them out.
  • Hosted analysis through the KwikQA service. Sign in with kwikqa login (GitHub). Hosted requests go only to endpoints with a zero-data-retention policy. While the paid plans are coming soon, every account uses the free plan.
  • Bring your own key: the BYOK plans, when they open, send requests straight to OpenRouter with your OPENROUTER_API_KEY, so the evidence never passes through KwikQA's servers.
  • Team insights: on the team plans, when they open, the owner can turn on sharing of the names, files, and projects of failing and flaky tests, and of each fix's outcome. Evidence, source, diffs, and error messages are never shared.
  • Account and setup commands: init, login, logout, status, upgrade, and billing. analyze --yes runs unattended in CI with a CI token in KWIKQA_TOKEN.