Improve account sync performance, handle concurrent market data syncing (#2236)

* PlaidConnectable concern

* Remove bad abstraction

* Put sync implementations in own concerns

* Sync strategies

* Move sync orchestration to Sync class

* Clean up sync class, add state machine

* Basic market data sync cron

* Fix price sync

* Improve sync window column names, add timestamps

* 30 day syncs by default

* Clean up market data methods

* Report high duplicate sync counts to Sentry

* Add sync states throughout app

* account tab session

* Persistent account tab selections

* Remove manual sleep

* Add migration to clear stale syncs on self hosted apps

* Tweak sync states

* Sync completion event broadcasts

* Fix timezones in tests

* Cleanup

* More cleanup

* Plaid item UI broadcasts for sync

* Fix account ID namespace conflict

* Sync broadcasters

* Smoother account sync refreshes

* Remove test sync delay

app/models/concerns/syncable.rb

@@ -6,24 +6,24 @@ module Syncable
   end
 
   def syncing?
-    syncs.where(status: [ :syncing, :pending ]).any?
+    raise NotImplementedError, "Subclasses must implement the syncing? method"
   end
 
-  def sync_later(start_date: nil, parent_sync: nil)
-    new_sync = syncs.create!(start_date: start_date, parent: parent_sync)
+  def sync_later(parent_sync: nil, window_start_date: nil, window_end_date: nil)
+    new_sync = syncs.create!(parent: parent_sync, window_start_date: window_start_date, window_end_date: window_end_date)
     SyncJob.perform_later(new_sync)
   end
 
-  def sync(start_date: nil)
-    syncs.create!(start_date: start_date).perform
+  def perform_sync(sync)
+    syncer.perform_sync(sync)
   end
 
-  def sync_data(sync, start_date: nil)
-    raise NotImplementedError, "Subclasses must implement the `sync_data` method"
+  def perform_post_sync
+    syncer.perform_post_sync
   end
 
-  def post_sync(sync)
-    # no-op, syncable can optionally provide implementation
+  def broadcast_sync_complete
+    sync_broadcaster.broadcast
   end
 
   def sync_error
@@ -31,7 +31,7 @@ module Syncable
   end
 
   def last_synced_at
-    latest_sync&.last_ran_at
+    latest_sync&.completed_at
   end
 
   def last_sync_created_at
@@ -40,6 +40,14 @@ module Syncable
 
   private
     def latest_sync
-      syncs.order(created_at: :desc).first
+      syncs.ordered.first
+    end
+
+    def syncer
+      self.class::Syncer.new(self)
+    end
+
+    def sync_broadcaster
+      self.class::SyncCompleteEvent.new(self)
     end
 end