OCR and Image Checks for Browser Automation in Broma
Why OCR and Image Checks Belong in Browser Automation
Selectors work well when text lives in HTML. But many web pages render critical information inside images, badges, canvas elements, or styled graphics that a standard "read text" step cannot see. In those cases, optical character recognition (OCR) and image checks let your Broma workflow confirm what a human would see on screen.
Because Broma is a Windows desktop app and workflows run on your own computer, these checks happen locally. You configure steps directly in the app and, when helpful, enter short JavaScript expressions to clean up extracted values.
This guide covers when to use OCR versus selectors, how to set up OCR and image-check steps, real-world examples for public pages and dashboards, and how to keep your runs reliable.
Choose the Right Step for the Job
Use these rules to decide which step to add:
- Read text with a selector: Use this first when the target text is in normal HTML. It is faster and more precise than OCR.
- Read text with OCR: Use this when text appears inside an image, a product badge, a status icon, or any non-selectable visual element.
- Image check: Use this to verify visual state, such as whether a green checkmark appeared, a button turned gray, or a screenshot matches an expected reference.
Do not use OCR as a default for everything. It is slower than selector-based reading and can misread low-contrast or stylized fonts. Combine it with a selector step when possible: use a selector to narrow to the right region, then OCR only that area.
Step-by-Step: Read Text with OCR
Here is how to add an OCR step in Broma.
- Open the target page in your Broma browser profile.
- Add a Read text (OCR) step to your workflow.
- Choose the region: full page, visible viewport, or a specific element bounds.
- Set the language to match the page text.
- Run the step once and inspect the extracted text in the run log.
- Save the output to a workflow variable, for example
ocr_stock_status. - Use that variable in a later step, such as a condition or a log entry.
Real-world example: public product stock badge
An e-commerce seller monitors a public supplier page to see whether a popular item is back in stock. The page shows an "In stock" badge, but the text is inside an image, so a selector returns nothing.
The workflow:
- Open the product URL.
- Wait for the badge element to appear.
- Add an OCR step scoped to that badge region.
- Extract text and save it to
badge_text. - Add a JavaScript expression step to normalize the result:
const raw = "{{badge_text}}";
const cleaned = raw.trim().toLowerCase();
cleaned.includes("in stock") ? "available" : "unavailable";
- Branch: if the result is "available," log the finding or send a notification; otherwise, wait and retry on the next scheduled run.
Because Broma has no public API or code package, that JavaScript lives only as a short expression inside the app step field.
Step-by-Step: Add an Image Check
Image checks compare a live screenshot against a reference image you capture in the app.
- Add a Take screenshot step and save the image as your reference. Name it something clear, such as
dashboard-green-status.png. - Add an Image check step.
- Select the current screenshot or region to compare.
- Choose the reference image from your workflow assets.
- Set a tolerance percentage. Lower tolerance means stricter matching; higher tolerance ignores minor compression or font rendering differences.
- Branch on the result: match or no match.
Real-world example: smoke check after a release
A QA tester runs a smoke check on a public staging dashboard after a release. The expected state is a green "All systems operational" banner.
The workflow:
- Open the dashboard URL.
- Wait for the banner element.
- Take a screenshot of the banner area.
- Run an image check against the stored reference
banner-green-reference.png. - If the images match, mark the smoke check as passed.
- If they do not match, take another screenshot, save it with a timestamp, and stop the flow for manual review.
Update the reference image only after an intentional, approved UI change. Do not overwrite it simply because a run failed; investigate first.
Use Short JavaScript Expressions to Clean OCR Output
OCR often returns extra spaces, line breaks, or nearby labels. Broma lets you enter short JavaScript expressions in supported step fields to transform the text before your workflow uses it.
Examples:
- Trim and collapse whitespace:
"{{ocr_text}}".replace(/\s+/g, " ").trim();
- Extract a number from a string such as "Stock: 14 units":
const m = "{{ocr_text}}".match(/\d+/);
m ? m[0] : "0";
- Convert "Yes"/"No" OCR output to a boolean:
"{{ocr_text}}".trim().toLowerCase() === "yes";
Keep expressions short. If you need complex logic, break it into multiple steps rather than writing a long script inside one field. Remember, Broma does not provide an SDK, CLI, or code package; all logic is configured through the app interface plus these small expressions.
Keep Runs Polite and Within Site Terms
Automation on public websites should be slow, respectful, and limited to public data.
- Add a wait or delay between page loads so you do not hammer a server.
- Check pages that are publicly accessible without bypassing logins or paywalls.
- Do not collect personal data, credentials, or content behind authentication unless it is your own account on your own machine.
- Review the site's terms before setting up recurring checks, and adjust frequency if the site asks for slower access.
Since workflows run locally on your Windows computer, you control the pace and the data that stays on your machine.
Troubleshoot Common OCR and Image Issues
OCR returns blank or wrong text
- The region may be too large or include ads. Narrow it to the exact element.
- Text may be too small. Increase the zoom or choose a higher-resolution screenshot if the app allows.
- Low contrast or stylized fonts can confuse OCR. Try selecting only the text line, not surrounding graphics.
Image check fails unexpectedly
- A minor layout shift, font change, or browser zoom level can break pixel comparison. Increase tolerance slightly.
- Dynamic content such as timestamps or rotating banners will never match a static reference. Crop those areas out before comparing.
- If the UI changed on purpose, capture a new reference image and update the workflow.
Reusing OCR and Image Flows Across a Team
Once a flow is stable, export it from Broma and import it on another Windows machine where a teammate has Broma installed. The exported package includes steps, selectors, OCR regions, and reference images. Teammates can adjust the target URL or schedule without rebuilding the OCR logic from scratch. Because Broma runs locally, each teammate's run uses their own browser profile, cookies, and login state.
Conclusion
OCR and image checks extend browser automation beyond HTML selectors. Use OCR when text is trapped inside images or badges, and use image checks when you need to verify visual state rather than text content. Pair them with short JavaScript expressions to clean results, run at a polite pace on public pages, and keep reference images updated only after intentional changes. With these steps, your Broma workflows can confirm what matters on the screen without writing a full automation script.
