Docker/Maybe
Docker/Maybe
1
0
Fork 0
mirror of https://github.com/maybe-finance/maybe.git synced 2025-08-02 20:15:22 +02:00
Code Issues Releases Activity Actions

Plaid sync domain improvements (#2267)
Some checks are pending
Publish Docker image / ci (push) Waiting to run
Details
Publish Docker image / Build docker image (push) Blocked by required conditions
Details

Browse source 
Breaks our Plaid sync process out into more manageable classes. Notably, this moves the sync process to a distinct, 2-step flow:

1. Import stage - we first make API calls and import Plaid data to "mirror" tables
2. Processing stage - read the raw data, apply business rules, build internal domain models and sync balances

This provides several benefits:

- Plaid syncs can now be "replayed" without fetching API data again
- Mirror tables provide better audit and debugging capabilities
- Eliminates the "all or nothing" sync behavior that is currently in place, which is brittle
This commit is contained in:
Zach Gollwitzer 2025-05-23 18:58:22 -04:00 • committed by GitHub
parent 5c82af0e8c
commit 03a146222d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
72 changed files with 3763 additions and 706 deletions
Download patch file Download diff file Expand all files Collapse all files

24
db/migrate/20250523131455_add_raw_payloads_to_plaid_accounts.rb Normal file
View file

@ -0,0 +1,24 @@
class AddRawPayloadsToPlaidAccounts < ActiveRecord::Migration[7.2]
def change
add_column :plaid_items, :raw_payload, :jsonb, default: {}
add_column :plaid_items, :raw_institution_payload, :jsonb, default: {}
change_column_null :plaid_items, :plaid_id, false
add_index :plaid_items, :plaid_id, unique: true
add_column :plaid_accounts, :raw_payload, :jsonb, default: {}
add_column :plaid_accounts, :raw_transactions_payload, :jsonb, default: {}
add_column :plaid_accounts, :raw_investments_payload, :jsonb, default: {}
add_column :plaid_accounts, :raw_liabilities_payload, :jsonb, default: {}
change_column_null :plaid_accounts, :plaid_id, false
change_column_null :plaid_accounts, :plaid_type, false
change_column_null :plaid_accounts, :currency, false
change_column_null :plaid_accounts, :name, false
add_index :plaid_accounts, :plaid_id, unique: true
# No longer need to store on transaction model because it is stored in raw_transactions_payload
remove_column :transactions, :plaid_category, :string
remove_column :transactions, :plaid_category_detailed, :string
end
end