All posts

Automate Release Smoke Checks in Broma

Automate Release Smoke Checks in Broma

Release smoke checks are where browser automation proves useful before a bad release reaches real users. A human opens the staging or release environment, logs in, follows the critical path, checks a confirmation, and takes a screenshot. That sequence repeats for every release. The work is small, but the risk of skipping it is large.

Broma's model fits this task well. The workflow runs on the user's own computer, uses a browser profile with the right cookies and extensions, and keeps credentials out of prompts. You describe the check, build the steps, run it, and review the log.

Why Release Smoke Checks Deserve Automation

Release smoke checks are repetitive. They are also high-signal. A single missed confirmation can hide a broken checkout flow, a failed migration, or a bad configuration.

Manual smoke checks work, but they are slow and inconsistent. Automation gives you repeatable paths, timestamps, run history, screenshots, and clear pass or fail logs.

A useful smoke check is not a full regression suite. It is a tripwire. It should answer one question: did the release break the critical path we just changed? That makes it easy to scope.

The point is not to replace manual testing. The point is to make the first check repeatable and comparable across releases.

Pick the Smallest Release-Blocking Path

Do not automate the whole product. Pick one critical path that would break first if a release introduced a regression.

  • Checkout success message.
  • Admin dashboard loads after login.
  • Report export downloads and opens.
  • Search returns expected results.
  • Webhook settings page renders.

For this guide, use the checkout success message.

Build the Smoke Flow Step by Step

Start With the Manual Checklist

Write down the steps you would perform manually.

  1. Open the release environment.
  2. Log in with the test account.
  3. Open the checkout page.
  4. Complete a test order or trigger the critical action.
  5. Wait for the success element.
  6. Read the confirmation text.
  7. Take a screenshot if failed or as evidence.

Create the Workflow in Broma

Create a new workflow and add steps.

  • Add a browser profile for the release environment.
  • Add an open-page step with the checkout URL.
  • Add a wait step for the success selector.
  • Add a read-text step.
  • Add a conditional branch based on the text.
  • Add a screenshot step.
  • Add a log result step.

Keep the workflow small. One URL, one critical action, one assertion, one screenshot.

Example selector and JavaScript expression:

await page.waitForSelector('#checkout-success', { timeout: 10000 });

const successText = await page.$eval(
  '#checkout-success',
  el => el.textContent
);

The branch checks whether the text includes the expected phrase.

if (!successText?.includes('Order confirmed')) {
  // go to failed branch
}

Describe the Task to the AI Agent

Prompt Broma in plain language:

"Open the staging checkout page for the June release. Wait for the order confirmation element. Read the heading. If it includes Order confirmed, take a screenshot and mark the smoke check passed. If not, take a screenshot and mark it failed."

Review the generated flow. Confirm the wait exists, the selector is scoped, the branch is correct, and the screenshot is attached to the failure path. The AI may produce a flow that clicks too early or uses a broad selector. Correct it before running.

Ask the AI to wait for a text or selector, not just click. Specify success and failure branches. Ask it to use stable identifiers when available.

Run the Check and Read the Logs

Run the workflow. Check the run log for pass or fail. A pass should include the screenshot and the read text. A fail should stop at the branch and capture evidence.

If the check fails because the page was slow, increase the wait timeout. If it fails because the selector changed, update the selector. If it fails because the text changed, adjust the expected phrase.

A good run log should capture the workflow name, environment, start time, steps visited, final status, screenshot path, and read text. Review it before sharing results. If the check passes, keep the screenshot and log. If it fails, open the screenshot and the page state before changing anything.

Run the check after deploy. Do not confuse it with a full regression suite.

Fix Flows That Break After a Release

Page changes are normal. Release smoke flows should be easy to repair.

  • Prefer stable selectors such as IDs or data attributes.
  • Use text-based checks when IDs change often.
  • Add waits for critical elements.
  • Keep failure screenshots small and focused.

Short JavaScript expression for reading a release banner:

const banner = await page.$eval('#release-banner', el => el.textContent);

When a selector breaks, open the page, inspect the DOM, and find a stable identifier. Replace the selector in the wait and read steps. Re-run. If the text changed, update the expected phrase.

If the flow is flaky, increase the timeout, add an explicit wait, or use a text check instead of a CSS selector.

Turn It Into Team Reuse

Save the workflow as a release smoke template. For future releases, copy it, change the URL, and adjust expected text. This keeps the team from rebuilding the same check.

Start small, keep the assertion narrow, and let the run log become your release record. Over time, the smoke check can grow into a lightweight release gate without becoming a full test suite.