Empire Flippers API Sync Challenge

Ruby on Rails app that ingests listings from the Empire Flippers Marketplace API, stores them in PostgreSQL, and synchronizes currently For Sale listings into HubSpot deals on a daily schedule.

What This Repo Does

• Pulls listing data from the public Empire Flippers Listings API.
• Persists listings locally with idempotent upserts (no duplicate listing_number rows).
• Syncs only For Sale listings to HubSpot deals.
• Prevents duplicate HubSpot deals by reusing stored HubSpot deal IDs and searching by deterministic deal name.
• Runs the full workflow daily using sidekiq + sidekiq-scheduler.

Major Features

• Daily orchestration job
  • DailyMarketplaceSyncJob runs:
    1. EmpireFlippers::ListingsImporter
    2. HubspotSync::DealsSyncService
• Idempotent listing storage
  • Local uniqueness key: listings.listing_number
  • Uses bulk upsert for safe reruns.
• HubSpot sync with duplicate prevention
  • Sync scope: Listing.for_sale
  • Required mapping:
    • Deal Name -> Listing #{listing_number}
    • Amount -> listing_price
    • Close Date -> now + 30 days
    • Deal Description -> summary
  • Duplicate controls:
    • Reuse existing hubspot_deal_id if still valid
    • Else search by dealname
    • Else create

Project Components

• app/services/empire_flippers/client.rb
  • API client for Empire Flippers listings endpoint.
• app/services/empire_flippers/listings_importer.rb
  • Paginates and upserts listing records.
• app/services/hubspot_sync/client.rb
  • HubSpot request wrapper.
• app/services/hubspot_sync/deals_sync_service.rb
  • Sync logic for For Sale listings and dedupe behavior.
• app/jobs/daily_marketplace_sync_job.rb
  • Daily import + sync orchestration.
• config/sidekiq.yml
  • Sidekiq queues and scheduler cron config.
• lib/tasks/empire_flippers.rake
  • Manual tasks for import, HubSpot sync, and full daily run.

Local Setup

Prerequisites

• Ruby 3.3+
• PostgreSQL
• Redis
• Bundler

1) Install dependencies

bundle install

2) Configure environment variables

Copy .env.example to .env and set values:

cp .env.example .env

Required:

• HUBSPOT_ACCESS_TOKEN (PAT or OAuth access token with deals scopes)
• EMPIRE_FLIPPERS_BASE_URL
• EMPIRE_FLIPPERS_LISTINGS_PATH
• HUBSPOT_DEAL_CLOSE_DAYS
• REDIS_URL
• DAILY_MARKETPLACE_SYNC_CRON

3) Create and migrate database

bin/rails db:create db:migrate

4) Start Redis

brew services start redis

Running the App Locally

Run test suite

bundle exec rspec

Import listings (Empire Flippers -> Postgres)

bin/rails empire_flippers:import_listings

Sync deals (Postgres For Sale listings -> HubSpot)

bin/rails empire_flippers:sync_hubspot_deals

Run full workflow once (same as daily job flow)

bin/rails empire_flippers:run_daily_sync

Start scheduler worker (daily automation)

bundle exec sidekiq -C config/sidekiq.yml

Data Flow

1. Fetch paginated listing payloads from Empire Flippers.
2. Upsert into listings table keyed by listing_number.
3. Select only listing_status = "For Sale".
4. Build HubSpot deal properties from listing fields.
5. Update existing HubSpot deal or create if not found.
6. Persist hubspot_deal_id locally for future idempotent syncs.

Verification and Metrics

Local metrics check

bin/rails runner "puts({total: Listing.count, for_sale: Listing.for_sale.count, linked_to_hubspot: Listing.for_sale.where.not(hubspot_deal_id: nil).count}.to_json)"

Duplicate safety checks

bin/rails runner "puts({duplicate_listing_numbers: Listing.group(:listing_number).having('COUNT(*) > 1').count.size, duplicate_hubspot_ids: Listing.where.not(hubspot_deal_id: nil).group(:hubspot_deal_id).having('COUNT(*) > 1').count.size}.to_json)"

Scheduler config check

bin/rails runner "cfg = YAML.load(ERB.new(File.read(Rails.root.join('config/sidekiq.yml'))).result); puts cfg[:scheduler][:schedule].keys"

Testing Coverage

RSpec includes:

• Listing model validations and scopes
• Empire Flippers importer:
  • idempotent reruns
  • updates on changed payload
  • pagination handling
  • invalid payload skip behavior
• HubSpot sync:
  • create vs update behavior
  • duplicate prevention strategy
  • For Sale filtering
• Daily orchestration job call order

Notes

• This project is designed for local/demo execution and challenge validation.
• Daily execution requires Sidekiq + Redis running continuously.
• Keep secrets only in .env (never commit tokens).