Sync cadences for the internal and external product faces
Overview
There are many facets of the Gem platform that have set sync intervals. This is usually done to load balance data syncs for the different ATS integrations to ensure we don’t have syncs get stuck in a large queue all at one time.
ATS sync cadences
Write task-based sections for each separate task that a reader may need to complete while using this feature.
1) Rippling user management sync
- Cadence: every 10 minutes
- Code:
server/run_rippling_user_management_sync_worker.py(@scheduler.scheduled_job("interval", minutes=10))
2) Email metadata sync (full inbox metadata processing)
- Cadence: every 4 minutes
- Code:
server/run_email_sync_scheduled_jobs.py(@scheduler.scheduled_job("interval", minutes=4))
3) Resync failed notes (email → Greenhouse notes retry)
- Cadence: daily at 4am America/Los_Angeles
- Code:
server/run_email_sync_scheduled_jobs.py(@scheduler.scheduled_job("cron", hour=4, timezone="America/Los_Angeles"))
4) Gmail metadata push cursor processing (near-real-time cursor partition scheduling)
- Cadence: every 10 seconds (two scheduled loops)
- Code:
server/run_gmail_metadata_sync_worker.py @scheduler.scheduled_job("interval", seconds=10)forsync_push_cursors()@scheduler.scheduled_job("interval", seconds=10)forreport_number_of_jobs_scheduled()
5) Google Calendar event sync
- Cadence: every 3 hours
- Code:
server/run_google_calendar_sync_worker.py(@scheduler.scheduled_job("cron", hour="*/3"))
6) Google Calendar webhook renewal
- Cadence: every 1 hour
- Code:
server/run_google_calendar_sync_worker.py(@scheduler.scheduled_job("cron", hour="*/1"))
7) Google Calendar webhook message processing (SQS -> processing)
- Cadence: every 10 seconds
- Code:
server/run_google_calendar_sync_worker.py(@scheduler.scheduled_job("interval", seconds=10)forprocess_webhooks())
8) Salesforce sync worker (data refresh)
- Cadence: daily at 22:40 America/Los_Angeles
- Code:
server/run_salesforce_sync_worker.py(AsyncIOScheduleradd_job(... "cron", hour=22, minute=40, timezone="America/Los_Angeles")) - Also defined:
SYNC_INTERVAL_SECONDS = 86_400(24 hours) in the same file, used for staleness/should-sync logic.
9) Salesforce seat usage sync to Salesforce
- Cadence: daily at 22:50 America/Los_Angeles
- Code:
server/run_salesforce_sync_worker.py(add_job(... hour=22, minute=50, timezone="America/Los_Angeles"))
10) SuccessFactors deep sync worker
- Established frequencies (used to decide when to enqueue work):
- Full sync frequency: every 7 days (
FULL_SYNC_FREQUENCY = timedelta(days=7)) - Incremental frequency: every 24 hours (
INCREMENTAL_SYNC_FREQUENCY = timedelta(hours=24)) - Scheduler tick cadence:
- Checks/schedules full syncs: every 10 minutes (
@scheduler.scheduled_job("cron", minute="*/10")) - Checks/schedules incremental syncs: every 10 minutes (
@scheduler.scheduled_job("cron", minute="*/10")) - Resume syncs: every 1 minute (
@scheduler.scheduled_job("cron", minute="*/1", next_run_time=datetime.now())) - Code:
server/run_success_factors_deep_sync_worker.py
11) Greenhouse incremental sync worker (non-webhook incremental paths)
- “Fast incremental sync” scheduling tick: every 5 minutes (
@scheduler.scheduled_job("interval", minutes=5, executor="frequent")) - “Nightly incremental sync” scheduling tick: every 15 minutes (
@scheduler.scheduled_job("interval", minutes=15)) - Nightly sync re-run threshold: 6 hours (
NIGHTLY_SYNC_FREQUENCY_IN_SECONDS = 6 * 60 * 60) - Code:
server/run_greenhouse_worker.py
12) Greenhouse full sync worker
- Create weekly full sync version for specific team(s): scheduled every 5 minutes during PT evening/night hours (
hour="20-23,0-5", minute="*/5", timezone="America/Los_Angeles") and additionally guarded by “last full sync >= 7 days” - Start new full sync threads: every 1 minute (
@scheduler.scheduled_job(trigger="interval", minutes=1, ...)) - Code:
server/run_greenhouse_full_sync_worker.py
13) Greenhouse webhook processing worker (process incoming GH webhooks stored in DB)
- Cadence: every 20 seconds (
@scheduler.scheduled_job("interval", seconds=20)) - Code:
server/run_greenhouse_webhook_worker.py
14) Greenhouse → Gem Scheduling webhook proxy worker (proxy selected GH webhooks to scheduling)
- Cadence: every 10 seconds (
@scheduler.scheduled_job("interval", seconds=10)) - Also sets a “freshness window” concept: only proxy webhooks from last 30 minutes (
WEBHOOK_FRESHNESS_SECONDS = 30 * 60) - Code:
server/run_gem_scheduling_webhook_proxy_worker.py
15) Taleo deep sync scheduler
- Nightly job creation: daily at hour 4 (server timezone) (
@scheduler.scheduled_job("cron", hour=4)) - Pending version polling/promoting: every 5 minutes (
@scheduler.scheduled_job("interval", minutes=5)) - Code:
server/run_taleo_sync_scheduler.py - Related polling constants (internal scheduler loop timings): every 5 minutes (
DELTA_SYNC_POLL_TALEO_INTERVAL = 5 * 60,SCHEDULER_POLL_JOBS_INTERVAL = 5 * 60) inserver/ats_integrations/taleo/sync/task_scheduler.py
16) OATS nightly migration worker (incremental/nightly migrations)
- Nightly scheduling: daily at 9pm US/Pacific (
@scheduler.scheduled_job("cron", hour=21, timezone="US/Pacific")) - Continue interrupted migrations: every hour (
@scheduler.scheduled_job("cron", hour="*", timezone="US/Pacific")) - Code:
server/run_oats_nightly_migration_worker.py
17) OATS ↔ JobTarget sync worker
- OATS events → JobTarget: every 10 seconds (
@scheduler.scheduled_job("cron", second="*/10", ...)) - Resource resync (team/user info refresh to JT): daily at midnight America/Los_Angeles (
@scheduler.scheduled_job("cron", hour=0, timezone="America/Los_Angeles")) - Code:
server/run_job_target_sync_worker.py
18) Workday sync cadence constants (used by Workday ATS sync logic)
- Cadences:
SYNC_INTERVAL = timedelta(hours=24)(daily)SYNC_INTERVAL_PARTIAL_ENABLED = timedelta(days=7)(weekly)SYNC_INTERVAL_PARTIAL_ENABLED_ONE_OFF = timedelta(days=14)(biweekly)- Code:
server/ats_integrations/workday/constants.py - Additional related constant:
PARTIAL_SYNC_FREQUENCY = timedelta(hours=24)inserver/workday_sync_utils.py
19) In-product / client-side polling or cache invalidation tied to sync propagation (Gem Scheduling UI)
- Query invalidation delays: 1s, 5s, 10s, 15s, 30s (explicitly “based on sync timings”)
- Code:
client/components/GemScheduling/Data/CrossCacheEcho/syncApolloCache.ts - Events cache refetch:
- Refetch interval: 10 seconds (
REFETCH_INTERVAL_MS = 10 * 1000) - Cache expiry: 5 minutes
- Code:
client/components/GemScheduling/Data/Events/useEventsWithCache.ts
20) Chrome extension: Indeed messages sync loop
- Cadence: controlled by a server-provided value
indeedIntegrationSettings.indeedMessageSyncInterval(the loop usessetTimeouteach cycle rather than a fixed constant) - Code:
extension/background.ts(functionindeedMessagesSync, usessetTimeout(... indeedIntegrationSettings.indeedMessageSyncInterval))