1
0
Fork 0
mirror of https://github.com/maybe-finance/maybe.git synced 2025-08-04 13:05:19 +02:00

Improve account sync performance, handle concurrent market data syncing (#2236)

* PlaidConnectable concern

* Remove bad abstraction

* Put sync implementations in own concerns

* Sync strategies

* Move sync orchestration to Sync class

* Clean up sync class, add state machine

* Basic market data sync cron

* Fix price sync

* Improve sync window column names, add timestamps

* 30 day syncs by default

* Clean up market data methods

* Report high duplicate sync counts to Sentry

* Add sync states throughout app

* account tab session

* Persistent account tab selections

* Remove manual sleep

* Add migration to clear stale syncs on self hosted apps

* Tweak sync states

* Sync completion event broadcasts

* Fix timezones in tests

* Cleanup

* More cleanup

* Plaid item UI broadcasts for sync

* Fix account ID namespace conflict

* Sync broadcasters

* Smoother account sync refreshes

* Remove test sync delay
This commit is contained in:
Zach Gollwitzer 2025-05-15 10:19:56 -04:00 committed by GitHub
parent 9793cc74f9
commit 10dd9e061a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
97 changed files with 1837 additions and 949 deletions

View file

@ -28,44 +28,6 @@ module Security::Provided
end
end
def sync_provider_prices(start_date:, end_date: Date.current)
unless has_prices?
Rails.logger.warn("Security id=#{id} ticker=#{ticker} is not known by provider, skipping price sync")
return 0
end
unless provider.present?
Rails.logger.warn("No security provider configured, cannot sync prices for id=#{id} ticker=#{ticker}")
return 0
end
response = provider.fetch_security_prices(self, start_date: start_date, end_date: end_date)
unless response.success?
Rails.logger.error("Provider error for sync_provider_prices with id=#{id} ticker=#{ticker}: #{response.error}")
return 0
end
fetched_prices = response.data.map do |price|
{
security_id: price.security.id,
date: price.date,
price: price.price,
currency: price.currency
}
end
valid_prices = fetched_prices.reject do |price|
is_invalid = price[:date].nil? || price[:price].nil? || price[:currency].nil?
if is_invalid
Rails.logger.warn("Invalid price data for security_id=#{id}: Missing required fields in price record: #{price.inspect}")
end
is_invalid
end
Security::Price.upsert_all(valid_prices, unique_by: %i[security_id date currency])
end
def find_or_fetch_price(date: Date.current, cache: true)
price = prices.find_by(date: date)