Breaks our Plaid sync process out into more manageable classes. Notably, this moves the sync process to a distinct, 2-step flow:
1. Import stage - we first make API calls and import Plaid data to "mirror" tables
2. Processing stage - read the raw data, apply business rules, build internal domain models and sync balances
This provides several benefits:
- Plaid syncs can now be "replayed" without fetching API data again
- Mirror tables provide better audit and debugging capabilities
- Eliminates the "all or nothing" sync behavior that is currently in place, which is brittle
* Setup health check
* Security health checker cron
* Use resolver throughout codebase
* Use resolver for trade builder
* Add security health checks to schedule
* Handle no provider
* Lint fixes
* Update Security model to include country_code in the data definition.
* Modify Provider::SecurityConcept to define country_code for security.
* Enhance Provider::Synth to extract country_code from security data.
* Update Security::Provided to include country_code when creating security instances.
* Adjust Security::SynthComboboxOption to add country_code as an attribute.
* Revise combobox_security partial to conditionally display country flag and code.
* Save work
* Subscriptions and trials domain
* Store family ID on customer
* Remove indirection of stripe calls
* Test simplifications
* Update brakeman
* Fix stripe tests in CI
* Update billing page to show subscription details
* Remove legacy columns
* Complete billing settings page
* Fix hardcoded plan name
* Handle subscriptions for self hosting mode
* Lint fixes
* Domain model sketch
* Scaffold out rules domain
* Migrations
* Remove existing data enrichment for clean slate
* Sketch out business logic and basic tests
* Simplify rule scope building and action executions
* Get generator working again
* Basic implementation + tests
* Remove manual merchant management (rules will replace)
* Revert "Remove manual merchant management (rules will replace)"
This reverts commit 83dcbd9ff0.
* Family and Provider merchants model
* Fix brakeman warnings
* Fix notification loader
* Update notification position
* Add Rule action and condition registries
* Rule form with compound conditions and tests
* Split out notification types, add CTA type
* Rules form builder and Stimulus controller
* Clean up rule registry domain
* Clean up rules stimulus controller
* CTA message for rule when user changes transaction category
* Fix tests
* Lint updates
* Centralize notifications in Notifiable concern
* Implement category rule prompts with auto backoff and option to disable
* Fix layout bug caused by merge conflict
* Initialize rule with correct action for category CTA
* Add rule deletions, get rules working
* Complete dynamic rule form, split Stimulus controllers by resource
* Fix failing tests
* Change test password to avoid chromium conflicts
* Update integration tests
* Centralize all test password references
* Add re-apply rule action
* Rule confirm modal
* Run migrations
* Trigger rule notification after inline category updates
* Clean up rule styles
* Basic attribute locking for rules
* Apply attribute locks on user edits
* Log data enrichments, only apply rules to unlocked attributes
* Fix merge errors
* Additional merge conflict fixes
* Form UI improvements, ignore attribute locks on manual rule application
* Batch AI auto-categorization of transactions
* Auto merchant detection, ai enrichment in batches
* Fix Plaid merchant assignments
* Plaid category matching
* Cleanup 1
* Test cleanup
* Remove stale route
* Fix desktop chat UI issues
* Fix mobile nav styling issues
* AI sidebar
* Add chat and message models with associations
* Implement AI chat functionality with sidebar and messaging system
- Add chat and messages controllers
- Create chat and message views
- Implement chat-related routes
- Add message broadcasting and user interactions
- Update application layout to support chat sidebar
- Enhance user model with initials method
* Refactor AI sidebar with enhanced chat menu and interactions
- Update sidebar layout with dynamic width and improved responsiveness
- Add new chat menu Stimulus controller for toggling between chat and chat list views
- Improve chat list display with recent chats and empty state
- Extract AI avatar to a partial for reusability
- Enhance message display and interaction styling
- Add more contextual buttons and interaction hints
* Improve chat scroll behavior and message styling
- Refactor chat scroll functionality with Stimulus controller
- Optimize message scrolling in chat views
- Update message styling for better visual hierarchy
- Enhance chat container layout with flex and auto-scroll
- Simplify message rendering across different chat views
* Extract AI avatar to a shared partial for consistent styling
- Refactor AI avatar rendering across chat views
- Replace hardcoded avatar markup with a reusable partial
- Simplify avatar display in chats and messages views
* Update sidebar controller to handle right panel width dynamically
- Add conditional width class for right sidebar panel
- Ensure consistent sidebar toggle behavior for both left and right panels
- Use specific width class for right panel (w-[375px])
* Refactor chat form and AI greeting with flexible partials
- Extract message form to a reusable partial with dynamic context support
- Create flexible AI greeting partial for consistent welcome messages
- Simplify chat and sidebar views by leveraging new partials
- Add support for different form scenarios (chat, new chat, sidebar)
- Improve code modularity and reduce duplication
* Add chat clearing functionality with dynamic menu options
- Implement clear chat action in ChatsController
- Add clear chat route to support clearing messages
- Update AI sidebar with dropdown menu for chat actions
- Preserve system message when clearing chat
- Enhance chat interaction with new menu options
* Add frontmatter to project structure documentation
- Create initial frontmatter for structure.mdc file
- Include description and configuration options
- Prepare for potential dynamic documentation rendering
* Update general project rules with additional guidelines
- Add rule for using `Current.family` instead of `current_family`
- Include new guidelines for testing, API routes, and solution approach
- Expand project-specific rules for more consistent development practices
* Add OpenAI gem and AI-friendly data representations
- Add `ruby-openai` gem for AI integration
- Implement `to_ai_readable_hash` methods in BalanceSheet and IncomeStatement
- Include Promptable module in both models
- Add savings rate calculation method in IncomeStatement
- Prepare financial models for AI-powered insights and interactions
* Enhance AI Financial Assistant with Advanced Querying and Debugging Capabilities
- Implement comprehensive AI financial query system with function-based interactions
- Add detailed debug logging for AI responses and function calls
- Extend BalanceSheet and IncomeStatement models with AI-friendly methods
- Create robust error handling and fallback mechanisms for AI queries
- Update chat and message views to support debug mode and enhanced rendering
- Add AI query routes and initial test coverage for financial assistant
* Refactor AI sidebar and chat layout with improved structure and comments
- Remove inline AI chat from application layout
- Enhance AI sidebar with more semantic HTML structure
- Add descriptive comments to clarify different sections of chat view
- Improve flex layout and scrolling behavior in chat messages container
- Optimize message rendering with more explicit class names and structure
* Add Markdown rendering support for AI chat messages
- Implement `markdown` helper method in ApplicationHelper using Redcarpet
- Update message view to render AI messages with Markdown formatting
- Add comprehensive Markdown rendering options (tables, code blocks, links)
- Enhance AI Financial Assistant prompt to encourage Markdown usage
- Remove commented Markdown CSS in Tailwind application stylesheet
* Missing comma
* Enhance AI response processing with chat history context
* Improve AI debug logging with payload size limits and internal message flag
* Enhance AI chat interaction with improved thinking indicator and scrolling behavior
* Add AI consent and enable/disable functionality for AI chat
* Upgrade Biome and refactor JavaScript template literals
- Update @biomejs/biome to latest version with caret (^) notation
- Refactor AI query and chat controllers to use template literals
- Standardize npm scripts formatting in package.json
* Add beta testing usage note to AI consent modal
* Update test fixtures and configurations for AI chat functionality
- Add family association to chat fixtures and tests
- Set consistent password digest for test users
- Enable AI for test users
- Add OpenAI access token for test environment
- Update chat and user model tests to include family context
* Simplify data model and get tests passing
* Remove structure.mdc from version control
* Integrate AI chat styles into existing prose pattern
* Match Figma design spec, implement Turbo frames and actions for chats controller
* AI rules refresh
* Consolidate Stimulus controllers, thinking state, controllers, and views
* Naming, domain alignment
* Reset migrations
* Improve data model to support tool calls and message types
* Tool calling tests and fixtures
* Tool call implementation and test
* Get assistant test working again
* Test updates
* Process tool calls within provider
* Chat UI back to working state again
* Remove stale code
* Tests passing
* Update openai class naming to avoid conflicts
* Reconfigure test env
* Rebuild gemfile
* Fix naming conflicts for ChatResponse
* Message styles
* Use OpenAI conversation state management
* Assistant function base implementation
* Add back thinking messages, clean up error handling for chat
* Fix sync error when security price has bad data from provider
* Add balance sheet function to assistant
* Add better function calling error visibility
* Add income statement function
* Simplify and clean up "thinking" interactions with Turbo frames
* Remove stale data definitions from functions
* Ensure VCR fixtures working with latest code
* basic stream implementation
* Get streaming working
* Make AI sidebar wider when left sidebar is collapsed
* Get tests working with streaming responses
* Centralize provider error handling
* Provider data boundaries
---------
Co-authored-by: Josh Pigford <josh@joshpigford.com>
* Enhance Plaid connection management with re-authentication and error handling
- Add support for Plaid item status tracking (good/requires_update)
- Implement re-authentication flow for Plaid connections
- Handle connection errors and provide user-friendly reconnection options
- Update Plaid link token generation to support item updates
- Add localization for new connection management states
* Remove redundant 'reconnect' localization for Plaid items
* Enhance security information retrieval and handling
- Add support for operating MIC codes in security info fetching
- Update security uniqueness validation to handle unknown securities
- Improve security creation and update logic in Plaid investment sync
- Update combobox and view components to handle operating MIC codes
- Add unknown flag for securities with incomplete information
* Update schema.rb
* Refactor the need for mic codes
* Don't fetch prices unless a security has the necessary mic code
* Deduplication
* Lint
* Update Securities and Plaid Investment Sync
- Modify PlaidInvestmentSync to return plaid_security for USD cash
- Add non-null constraint to Securities ticker column
- Update Securities fixture to use exchange_operating_mic instead of exchange_mic
---------
Signed-off-by: Josh Pigford <josh@joshpigford.com>
* feat: Add institution details to Plaid items
- Fetch and store institution URL, ID, and primary color for Plaid items
- Update PlaidItem model to retrieve and save institution metadata
- Add new method in Plaid provider to get institution details
- Update account logo view to use institution domain for logo generation
* Add institution domain method to Account model
- Extract institution domain logic from view to Account model
- Simplify logo view by using new institution_domain method
- Improve code reusability and separation of concerns
* Initial pass at Plaid EU
* Add EU support to Plaid Items
* Lint
* Temp fix for rubocop isseus
* Merge cleanup
* Pass in region and get tests passing
* Use absolute path for translation
---------
Signed-off-by: Josh Pigford <josh@joshpigford.com>
* Add data enrichment
* Make data enrichment optional for self-hosters
* Add categories to data enrichment
* Only update category and merchant if nil
* Fix name overrides
* Lint fixes
* Basic plaid data model and linking
* Remove institutions, add plaid items
* Improve schema and Plaid provider
* Add webhook verification sketch
* Webhook verification
* Item accounts and balances sync setup
* Provide test encryption keys
* Fix test
* Only provide encryption keys in prod
* Try defining keys in test env
* Consolidate account sync logic
* Add back plaid account initialization
* Plaid transaction sync
* Sync UI overhaul for Plaid
* Add liability and investment syncing
* Handle investment webhooks and process current day holdings
* Remove logs
* Remove "all" period select for performance
* fix amount calc
* Remove todo comment
* Coming soon for investment historical data
* Document Plaid configuration
* Listen for holding updates
* Initial pass at Synth-based ticker selection
* Update _tickers.turbo_stream.erb
* Functional combobox display
* A few cleanup steps
* Linter
* Prevent long strings
* Another step towards functional combobox
* Deprecated files
* Custom Combobox implementation
* Lint
* Test suite fixes
* Lint
* Make direct use of mic codes
* Update splits
* Update trades_test.rb
* First pass at security price reference
* Data cleanup
* Synth security fetching does better with a mic_code
* Update test suite
😭
* Update schema.rb
* Update generator.rb
* Rough draft of issue system
* Simplify design
* Remove stale files from merge conflicts
* STI for issues
* Cleanup
* Improve Synth api key flow
* Stub api key for test
