[{"data":1,"prerenderedAt":18},["ShallowReactive",2],{"blog-post-en-polite-public-data-collection-broma":3},{"slug":4,"lang":5,"title":6,"summary":7,"date":8,"tags":9,"tag_slugs":13,"thumbnail_url":14,"translations":15,"body":16,"asset_base":17},"polite-public-data-collection-broma","en","Polite Public Data Collection with Broma","Learn how to build a compliant, rate‑limited web scraping workflow in Broma that respects site terms and delivers reliable data.","2026-11-05",[10,11,12],"automation","compliance","broma",[10,11,12],"https:\u002F\u002Fblog-api.bromaapp.com\u002Fapi\u002Fblog\u002Fposts\u002Fpolite-public-data-collection-broma\u002Fthumbnail.svg?lang=en",[5],"## Why polite public data collection matters\n\nWhen you need to gather publicly available information — price lists, product reviews, public statistics — doing it manually is time‑consuming and error‑prone. Automating the process with Broma frees you from repetitive clicks, but you must stay within the website’s terms of service and avoid overwhelming the server. A polite approach protects your access and keeps the data flow sustainable.\n\n## Understanding site terms and ethical scraping\n\nBefore you open Broma, read the site’s **Terms of Service** or **robots.txt**. Look for clauses that prohibit automated scraping, required request rates, or prohibited data uses. If the site allows scraping but asks for a delay between requests, honor that limit. This ensures you won’t be blocked and maintains a good relationship with the site owner.\n\n## Setting up a Broma workflow for data collection\n\n### Step 1: Define the target page and data points\n\n1. Open Broma and create a new workflow.\n2. In the **Start** action, enter the URL of the page you want to scrape.\n3. List the specific elements you need (e.g., product title, price, rating). Writing them down helps you stay focused during selector creation.\n\n### Step 2: Choose robust selectors\n\nBroma lets you pick a selector that survives minor layout changes. Use a combination of **CSS class** and **text content** when possible.\n\n- **Primary selector**: `.product-item .price`\n- **Fallback selector**: `span[data-price]`\n\nIf the class name changes, the fallback will still capture the price.\n\n### Step 3: Add polite delays and rate limiting\n\nTo respect the site’s rate limits:\n\n1. After each **Click** or **Read Text** action, add a **Wait** step.\n2. Set the wait time to a value that matches the site’s guidance (e.g., 2‑5 seconds).\n3. For high‑traffic sites, consider a **Loop** that iterates over a list of product URLs, inserting a longer wait (5‑10 s) between each iteration.\n\n### Step 4: Use AI assistance to draft the workflow (optional)\n\nYou can describe the task in plain language to an AI agent (your own ChatGPT, Claude, or any provider you subscribe to). The agent can output a step‑by‑step list that you paste into Broma’s **Import** feature. Review the generated steps, adjust the wait times, and verify the selectors.\n\n## Real‑world example: monitoring product reviews on a retail site\n\nImagine you need the average rating for a set of products on `example‑store.com`.\n\n1. **Start** → URL: `https:\u002F\u002Fexample-store.com\u002Fcategory\u002Fshoes`.\n2. **Loop** → Selector: `.product-card` (each card contains a product link).\n3. Inside the loop:\n   - **Click** on the product link (selector: `a.product-title`).\n   - **Wait** 3 seconds.\n   - **Read Text** → Selector: `.product-rating` (e.g., `4.5`).\n   - **Read Text** → Selector: `.product-price` (e.g., `$49.99`).\n   - **Wait** 4 seconds.\n4. **End Loop**.\n\nThe workflow now collects rating and price for every shoe listed on the category page.\n\n### Code block: short JavaScript expression for price extraction\n\nIf you prefer a custom expression, you can use Broma’s **Run JavaScript** action:\n\n```javascript\n\u002F\u002F Extract price, remove currency symbol, convert to number\nconst priceElem = document.querySelector('.price-class');\nconst priceText = priceElem ? priceElem.innerText : '';\nconst numericPrice = priceText.replace(\u002F[^0-9.]\u002Fg, '') || 0;\nreturn numericPrice;\n```\n\n## Running and monitoring the workflow\n\n### Scheduling batch runs\n\nBroma’s built‑in scheduler lets you run the workflow at a specific time or on a recurring basis (e.g., every morning at 08:00).\n\n1. Open the **Scheduler** tab.\n2. Choose **Recurring** → **Daily** → set the hour and minute.\n3. Select the workflow you created.\n4. Save. The app will launch the browser, execute the steps, and close the session automatically.\n\n### Reviewing logs and handling failures\n\nAfter each run, Broma writes a **Run Log** that shows:\n- Start\u002Fend timestamps\n- Each action with success\u002Ffailure status\n- Any error messages (e.g., element not found)\n\nTo troubleshoot:\n- Open the log, locate the failed step, and verify the selector.\n- If the page layout changed, update the selector or add a more flexible XPath.\n- Re‑run the workflow in a **Debug** mode to see live highlights.\n\n## Maintaining the workflow when pages change\n\nWebsites evolve; a selector that worked last week may break tomorrow. Follow these habits:\n\n- **Use multiple selectors** (primary + fallback) as shown earlier.\n- **Add a Wait for Element** before reading text, ensuring the element is present.\n- **Run a quick test** after any major site update; Broma’s UI highlights missing elements.\n- **Version your workflow**: duplicate the workflow before making big changes, so you can revert if needed.\n\n## Reusing the workflow across multiple accounts (if needed)\n\nIf you need to scrape data from several seller accounts on the same site, keep each account in a separate **Browser Profile** within Broma. Each profile stores its own cookies and login state, preventing cross‑account interference. Create a new workflow for each profile or parameterize the login step.\n\n## Troubleshooting common issues\n\n- **Element not found**: Check the selector in the browser’s dev tools. Ensure the class name isn’t generated dynamically (use text or attribute instead).\n- **Rate‑limit errors**: Increase the wait time or add a random jitter (`wait 3-6 seconds`).\n- **Session expiration**: Re‑login at the start of the workflow using the **Login** action, or use a profile that already contains a valid session.\n- **Unexpected page layout**: Add a **Scroll** action to bring the target element into view before reading.\n\n## Summary & next steps\n\nA polite public data collection workflow in Broma combines three core ideas:\n\n1. **Respect the site’s terms** – read policies, honor rate limits, and avoid prohibited actions.\n2. **Build resilient selectors** – mix class, attribute, and text cues, and add fallback options.\n3. **Automate with scheduled runs** – use Broma’s scheduler, monitor logs, and adjust waits as the site evolves.\n\nStart by defining a simple scrape (e.g., product titles on a catalog page). Apply the steps above, test, and iterate. As you gain confidence, expand to multi‑page loops, CSV exports, or integrate the data into your analytics dashboard. With Broma’s no‑code interface, you can keep the process reliable without writing a single line of code.\n","https:\u002F\u002Fblog-api.bromaapp.com\u002Fapi\u002Fblog\u002Fposts\u002Fpolite-public-data-collection-broma\u002Fassets",1794729622017]