I built a "polite scraper" Chrome extension instead of a server-side scraper. Here's why.
The author built a Chrome extension called SlotOwl that monitors government appointment portals for available slots, choosing to run the scraper within the user's browser instead of on a server. This 'polite scraper' approach enhances security, avoids IP bans, reduces server costs, and leverages the user to solve captchas manually. The trade-off is that users must keep their browser open for the monitoring to work.
- ▪The scraper runs in the user's browser tab, not on a server.
- ▪User credentials never leave the browser; only slot availability alerts are sent.
- ▪Polling occurs client-side, reducing server costs and avoiding automated captcha-solving.
- ▪Each user appears as a single human user, reducing detection and IP bans.
- ▪The extension uses Firebase for alert distribution, costing under $50/month at 1,000 users.
Opening excerpt (first ~120 words) tap to expand
try { if(localStorage) { let currentUser = localStorage.getItem('current_user'); if (currentUser) { currentUser = JSON.parse(currentUser); if (currentUser.id === 3904702) { document.getElementById('article-show-container').classList.add('current-user-is-article-author'); } } } } catch (e) { console.error(e); } Nik G Posted on May 1 I built a "polite scraper" Chrome extension instead of a server-side scraper. Here's why. #chrome #webdev #buildinpublic Six weeks ago I started building SlotOwl — a Chrome extension that watches government appointment portals (visa, immigration, passport, Global Entry) and notifies you the moment a slot opens. This week I shipped it.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at DEV.to (Top).
Discussion
0 commentsMore from DEV.to (Top)
