Improve account sync performance, handle concurrent market data syncing (#2236)

* PlaidConnectable concern

* Remove bad abstraction

* Put sync implementations in own concerns

* Sync strategies

* Move sync orchestration to Sync class

* Clean up sync class, add state machine

* Basic market data sync cron

* Fix price sync

* Improve sync window column names, add timestamps

* 30 day syncs by default

* Clean up market data methods

* Report high duplicate sync counts to Sentry

* Add sync states throughout app

* account tab session

* Persistent account tab selections

* Remove manual sleep

* Add migration to clear stale syncs on self hosted apps

* Tweak sync states

* Sync completion event broadcasts

* Fix timezones in tests

* Cleanup

* More cleanup

* Plaid item UI broadcasts for sync

* Fix account ID namespace conflict

* Sync broadcasters

* Smoother account sync refreshes

* Remove test sync delay

app/models/account.rb

@@ -61,6 +61,18 @@ class Account < ApplicationRecord
     end
   end
 
+  def syncing?
+    self_syncing = syncs.incomplete.any?
+
+    # Since Plaid Items sync as a "group", if the item is syncing, even if the account
+    # sync hasn't yet started (i.e. we're still fetching the Plaid data), show it as syncing in UI.
+    if linked?
+      plaid_account&.plaid_item&.syncing? || self_syncing
+    else
+      self_syncing
+    end
+  end
+
   def institution_domain
     url_string = plaid_account&.plaid_item&.institution_url
     return nil unless url_string.present?
@@ -81,21 +93,6 @@ class Account < ApplicationRecord
     DestroyJob.perform_later(self)
   end
 
-  def sync_data(sync, start_date: nil)
-    Rails.logger.info("Processing balances (#{linked? ? 'reverse' : 'forward'})")
-    sync_balances
-  end
-
-  def post_sync(sync)
-    family.remove_syncing_notice!
-
-    accountable.post_sync(sync)
-
-    unless sync.child?
-      family.auto_match_transfers!
-    end
-  end
-
   def current_holdings
     holdings.where(currency: currency, date: holdings.maximum(:date)).order(amount: :desc)
   end
@@ -172,10 +169,4 @@ class Account < ApplicationRecord
   def long_subtype_label
     accountable_class.long_subtype_label_for(subtype) || accountable_class.display_name
   end
-
-  private
-    def sync_balances
-      strategy = linked? ? :reverse : :forward
-      Balance::Syncer.new(self, strategy: strategy).sync_balances
-    end
 end