Improve account sync performance, handle concurrent market data syncing (#2236)

* PlaidConnectable concern

* Remove bad abstraction

* Put sync implementations in own concerns

* Sync strategies

* Move sync orchestration to Sync class

* Clean up sync class, add state machine

* Basic market data sync cron

* Fix price sync

* Improve sync window column names, add timestamps

* 30 day syncs by default

* Clean up market data methods

* Report high duplicate sync counts to Sentry

* Add sync states throughout app

* account tab session

* Persistent account tab selections

* Remove manual sleep

* Add migration to clear stale syncs on self hosted apps

* Tweak sync states

* Sync completion event broadcasts

* Fix timezones in tests

* Cleanup

* More cleanup

* Plaid item UI broadcasts for sync

* Fix account ID namespace conflict

* Sync broadcasters

* Smoother account sync refreshes

* Remove test sync delay

app/models/account/sync_complete_event.rb

@@ -0,0 +1,54 @@
+class Account::SyncCompleteEvent
+  attr_reader :account
+
+  def initialize(account)
+    @account = account
+  end
+
+  def broadcast
+    # Replace account row in accounts list
+    account.broadcast_replace_to(
+      account.family,
+      target: "account_#{account.id}",
+      partial: "accounts/account",
+      locals: { account: account }
+    )
+
+    # Replace the groups this account belongs to in the sidebar
+    account_group_ids.each do |id|
+      account.broadcast_replace_to(
+        account.family,
+        target: id,
+        partial: "accounts/accountable_group",
+        locals: { account_group: account_group, open: true }
+      )
+    end
+
+    # If this is a manual, unlinked account (i.e. not part of a Plaid Item),
+    # trigger the family sync complete broadcast so net worth graph is updated
+    unless account.linked?
+      account.family.broadcast_sync_complete
+    end
+
+    # Refresh entire account page (only applies if currently viewing this account)
+    account.broadcast_refresh
+  end
+
+  private
+    # The sidebar will show the account in both its classification tab and the "all" tab,
+    # so we need to broadcast to both.
+    def account_group_ids
+      id = account_group.id
+      [ id, "#{account_group.classification}_#{id}" ]
+    end
+
+    def account_group
+      family_balance_sheet.account_groups.find do |group|
+        group.accounts.any? { |a| a.id == account.id }
+      end
+    end
+
+    def family_balance_sheet
+      account.family.balance_sheet
+    end
+end

app/models/account/syncer.rb

@@ -0,0 +1,22 @@
+class Account::Syncer
+  attr_reader :account
+
+  def initialize(account)
+    @account = account
+  end
+
+  def perform_sync(sync)
+    Rails.logger.info("Processing balances (#{account.linked? ? 'reverse' : 'forward'})")
+    sync_balances
+  end
+
+  def perform_post_sync
+    account.family.auto_match_transfers!
+  end
+
+  private
+    def sync_balances
+      strategy = account.linked? ? :reverse : :forward
+      Balance::Syncer.new(account, strategy: strategy).sync_balances
+    end
+end