YRM

Labs

YRM Property Data Scraper

Serverless web scraping system for real estate data extraction with automated monitoring, email notifications, and data export. Built with Appwrite Functions, EdgeDB, and WebScraperEngine™.

Extraction Engine: Serverless DOM parsing with CSS selector targeting, regex pattern matching, and multi-page navigation capabilities.
Infrastructure: Appwrite Functions with 128MB memory allocation, intelligent proxy rotation, and rate limiting to avoid detection.
Data Processing: Automated data normalization, incremental storage with change detection, and webhooks for real-time notifications.
API Integration: RESTful endpoints for data retrieval, XLSX/JSON export options, and email delivery via SendGrid.
TypeScript
Next.js
Appwrite
EdgeDB
SendGrid
XLSX
WebScraperEngine
System activeLast system update: 12/13/2025

Technical Processing Pipeline

Initial Connection

1. Request routing through proxy network

2. User-agent randomization

3. Connection with anti-detection headers

Serverless Function via Appwrite
Data Extraction

1. DOM parsing with Cheerio

2. CSS selector-based element targeting

3. Recursive scraping for nested data

WebScraperEngine™ Core System
Data Processing

1. Property data normalization

2. Image URL extraction and cleaning

3. EdgeDB storage with unique constraints

EdgeDB + TypeScript
Notification System

1. Change detection algorithm

2. HTML email template generation

3. Batch delivery via SendGrid API

Next.js API + SendGrid

Processing Console

Unlimited

Target URLs

10,000+

Records/Hour

60s

Refresh Rate

Real-time

Notifications

Get Property Updates via Email

Subscribe to receive the latest property listings directly to your inbox.

Automated Property Monitoring

Start an automated workflow that checks for new properties every 60 seconds and notifies all subscribers.

Once started, the workflow will run in the background until the server restarts.

Scraper Configuration