1
0
Fork 0
mirror of https://github.com/maybe-finance/maybe.git synced 2025-08-08 06:55:21 +02:00
Commit graph

40 commits

Author SHA1 Message Date
Zach Gollwitzer
0329a5f211
Data exports (#2517)
* Import / export UI

* Data exports

* Lint fixes, brakeman update
2025-07-24 10:50:05 -04:00
Zach Gollwitzer
1aae00f586
perf(transactions): add kind to Transaction model and remove expensive Transfer joins in aggregations (#2388)
* add kind to transaction model

* Basic transfer creator

* Fix method naming conflict

* Creator form pattern

* Remove stale methods

* Tweak migration

* Remove BaseQuery, write entire query in each class for clarity

* Query optimizations

* Remove unused exchange rate query lines

* Remove temporary cache-warming strategy

* Fix test

* Update transaction search

* Decouple transactions endpoint from IncomeStatement

* Clean up transactions controller

* Update cursor rules

* Cleanup comments, logic in search

* Fix totals logic on transactions view

* Fix pagination

* Optimize search totals query

* Default to last 30 days on transactions page if no filters

* Decouple transactions list from transfer details

* Revert transfer route

* Migration reset

* Bundle update

* Fix matching logic, tests

* Remove unused code
2025-06-20 13:31:58 -04:00
Zach Gollwitzer
854a21993a Handle ITEM_NOT_FOUND errors on Plaid deletions 2025-05-25 11:52:29 -04:00
Zach Gollwitzer
e4ee06c9f6
Security resolver and health checker (#2281)
* Setup health check

* Security health checker cron

* Use resolver throughout codebase

* Use resolver for trade builder

* Add security health checks to schedule

* Handle no provider

* Lint fixes
2025-05-22 12:43:24 -04:00
Zach Gollwitzer
f82f77466a Fix Sentry context for security details exception 2025-05-18 10:52:35 -04:00
Zach Gollwitzer
9f13b5bb83
Handle stale syncs (#2257)
Some checks are pending
Publish Docker image / ci (push) Waiting to run
Publish Docker image / Build docker image (push) Blocked by required conditions
* Handle stale syncs

* Use `visible` sync logic in sidebar groups
2025-05-17 18:28:21 -04:00
Zach Gollwitzer
10f255a9a9
Clarify backend data pipeline naming concepts (importers, processors, materializers, calculators, and syncers) (#2255)
Some checks are pending
Publish Docker image / ci (push) Waiting to run
Publish Docker image / Build docker image (push) Blocked by required conditions
* Rename MarketDataSyncer to MarketDataImporter

* Materializers

* Importers

* More reference replacements
2025-05-17 16:37:16 -04:00
Zach Gollwitzer
6dc1d22672
Market data sync refinements (#2252)
* Exchange rate syncer implementation

* Security price syncer

* Fix issues with provider API

* Add back prod schedule

* Add back price and exchange rate syncs to account syncs

* Remove unused stock_exchanges table
2025-05-16 14:17:56 -04:00
Zach Gollwitzer
10dd9e061a
Improve account sync performance, handle concurrent market data syncing (#2236)
* PlaidConnectable concern

* Remove bad abstraction

* Put sync implementations in own concerns

* Sync strategies

* Move sync orchestration to Sync class

* Clean up sync class, add state machine

* Basic market data sync cron

* Fix price sync

* Improve sync window column names, add timestamps

* 30 day syncs by default

* Clean up market data methods

* Report high duplicate sync counts to Sentry

* Add sync states throughout app

* account tab session

* Persistent account tab selections

* Remove manual sleep

* Add migration to clear stale syncs on self hosted apps

* Tweak sync states

* Sync completion event broadcasts

* Fix timezones in tests

* Cleanup

* More cleanup

* Plaid item UI broadcasts for sync

* Fix account ID namespace conflict

* Sync broadcasters

* Smoother account sync refreshes

* Remove test sync delay
2025-05-15 10:19:56 -04:00
Zach Gollwitzer
a51c4d2cba
New onboarding, trials, Stripe integration (#2185)
Some checks are pending
Publish Docker image / ci (push) Waiting to run
Publish Docker image / Build docker image (push) Blocked by required conditions
* New onboarding, trials, Stripe integration

* Fix tests

* Lint fixes

* Fix subscription endpoints
2025-05-01 16:47:14 -04:00
Zach Gollwitzer
297a695d0f
Transaction rules engine V1 (#1900)
* Domain model sketch

* Scaffold out rules domain

* Migrations

* Remove existing data enrichment for clean slate

* Sketch out business logic and basic tests

* Simplify rule scope building and action executions

* Get generator working again

* Basic implementation + tests

* Remove manual merchant management (rules will replace)

* Revert "Remove manual merchant management (rules will replace)"

This reverts commit 83dcbd9ff0.

* Family and Provider merchants model

* Fix brakeman warnings

* Fix notification loader

* Update notification position

* Add Rule action and condition registries

* Rule form with compound conditions and tests

* Split out notification types, add CTA type

* Rules form builder and Stimulus controller

* Clean up rule registry domain

* Clean up rules stimulus controller

* CTA message for rule when user changes transaction category

* Fix tests

* Lint updates

* Centralize notifications in Notifiable concern

* Implement category rule prompts with auto backoff and option to disable

* Fix layout bug caused by merge conflict

* Initialize rule with correct action for category CTA

* Add rule deletions, get rules working

* Complete dynamic rule form, split Stimulus controllers by resource

* Fix failing tests

* Change test password to avoid chromium conflicts

* Update integration tests

* Centralize all test password references

* Add re-apply rule action

* Rule confirm modal

* Run migrations

* Trigger rule notification after inline category updates

* Clean up rule styles

* Basic attribute locking for rules

* Apply attribute locks on user edits

* Log data enrichments, only apply rules to unlocked attributes

* Fix merge errors

* Additional merge conflict fixes

* Form UI improvements, ignore attribute locks on manual rule application

* Batch AI auto-categorization of transactions

* Auto merchant detection, ai enrichment in batches

* Fix Plaid merchant assignments

* Plaid category matching

* Cleanup 1

* Test cleanup

* Remove stale route

* Fix desktop chat UI issues

* Fix mobile nav styling issues
2025-04-18 11:39:58 -04:00
Zach Gollwitzer
8648f11413
Sync hierarchy updates (#2087)
* Add parent sync orchestration

* Pass sync object to dependents
2025-04-11 12:13:46 -04:00
Zach Gollwitzer
1061aacb0f Set AI queue 2025-03-28 13:27:50 -04:00
Zach Gollwitzer
2f6b11c18f
Personal finance AI (v1) (#2022)
* AI sidebar

* Add chat and message models with associations

* Implement AI chat functionality with sidebar and messaging system

- Add chat and messages controllers
- Create chat and message views
- Implement chat-related routes
- Add message broadcasting and user interactions
- Update application layout to support chat sidebar
- Enhance user model with initials method

* Refactor AI sidebar with enhanced chat menu and interactions

- Update sidebar layout with dynamic width and improved responsiveness
- Add new chat menu Stimulus controller for toggling between chat and chat list views
- Improve chat list display with recent chats and empty state
- Extract AI avatar to a partial for reusability
- Enhance message display and interaction styling
- Add more contextual buttons and interaction hints

* Improve chat scroll behavior and message styling

- Refactor chat scroll functionality with Stimulus controller
- Optimize message scrolling in chat views
- Update message styling for better visual hierarchy
- Enhance chat container layout with flex and auto-scroll
- Simplify message rendering across different chat views

* Extract AI avatar to a shared partial for consistent styling

- Refactor AI avatar rendering across chat views
- Replace hardcoded avatar markup with a reusable partial
- Simplify avatar display in chats and messages views

* Update sidebar controller to handle right panel width dynamically

- Add conditional width class for right sidebar panel
- Ensure consistent sidebar toggle behavior for both left and right panels
- Use specific width class for right panel (w-[375px])

* Refactor chat form and AI greeting with flexible partials

- Extract message form to a reusable partial with dynamic context support
- Create flexible AI greeting partial for consistent welcome messages
- Simplify chat and sidebar views by leveraging new partials
- Add support for different form scenarios (chat, new chat, sidebar)
- Improve code modularity and reduce duplication

* Add chat clearing functionality with dynamic menu options

- Implement clear chat action in ChatsController
- Add clear chat route to support clearing messages
- Update AI sidebar with dropdown menu for chat actions
- Preserve system message when clearing chat
- Enhance chat interaction with new menu options

* Add frontmatter to project structure documentation

- Create initial frontmatter for structure.mdc file
- Include description and configuration options
- Prepare for potential dynamic documentation rendering

* Update general project rules with additional guidelines

- Add rule for using `Current.family` instead of `current_family`
- Include new guidelines for testing, API routes, and solution approach
- Expand project-specific rules for more consistent development practices

* Add OpenAI gem and AI-friendly data representations

- Add `ruby-openai` gem for AI integration
- Implement `to_ai_readable_hash` methods in BalanceSheet and IncomeStatement
- Include Promptable module in both models
- Add savings rate calculation method in IncomeStatement
- Prepare financial models for AI-powered insights and interactions

* Enhance AI Financial Assistant with Advanced Querying and Debugging Capabilities

- Implement comprehensive AI financial query system with function-based interactions
- Add detailed debug logging for AI responses and function calls
- Extend BalanceSheet and IncomeStatement models with AI-friendly methods
- Create robust error handling and fallback mechanisms for AI queries
- Update chat and message views to support debug mode and enhanced rendering
- Add AI query routes and initial test coverage for financial assistant

* Refactor AI sidebar and chat layout with improved structure and comments

- Remove inline AI chat from application layout
- Enhance AI sidebar with more semantic HTML structure
- Add descriptive comments to clarify different sections of chat view
- Improve flex layout and scrolling behavior in chat messages container
- Optimize message rendering with more explicit class names and structure

* Add Markdown rendering support for AI chat messages

- Implement `markdown` helper method in ApplicationHelper using Redcarpet
- Update message view to render AI messages with Markdown formatting
- Add comprehensive Markdown rendering options (tables, code blocks, links)
- Enhance AI Financial Assistant prompt to encourage Markdown usage
- Remove commented Markdown CSS in Tailwind application stylesheet

* Missing comma

* Enhance AI response processing with chat history context

* Improve AI debug logging with payload size limits and internal message flag

* Enhance AI chat interaction with improved thinking indicator and scrolling behavior

* Add AI consent and enable/disable functionality for AI chat

* Upgrade Biome and refactor JavaScript template literals

- Update @biomejs/biome to latest version with caret (^) notation
- Refactor AI query and chat controllers to use template literals
- Standardize npm scripts formatting in package.json

* Add beta testing usage note to AI consent modal

* Update test fixtures and configurations for AI chat functionality

- Add family association to chat fixtures and tests
- Set consistent password digest for test users
- Enable AI for test users
- Add OpenAI access token for test environment
- Update chat and user model tests to include family context

* Simplify data model and get tests passing

* Remove structure.mdc from version control

* Integrate AI chat styles into existing prose pattern

* Match Figma design spec, implement Turbo frames and actions for chats controller

* AI rules refresh

* Consolidate Stimulus controllers, thinking state, controllers, and views

* Naming, domain alignment

* Reset migrations

* Improve data model to support tool calls and message types

* Tool calling tests and fixtures

* Tool call implementation and test

* Get assistant test working again

* Test updates

* Process tool calls within provider

* Chat UI back to working state again

* Remove stale code

* Tests passing

* Update openai class naming to avoid conflicts

* Reconfigure test env

* Rebuild gemfile

* Fix naming conflicts for ChatResponse

* Message styles

* Use OpenAI conversation state management

* Assistant function base implementation

* Add back thinking messages, clean up error handling for chat

* Fix sync error when security price has bad data from provider

* Add balance sheet function to assistant

* Add better function calling error visibility

* Add income statement function

* Simplify and clean up "thinking" interactions with Turbo frames

* Remove stale data definitions from functions

* Ensure VCR fixtures working with latest code

* basic stream implementation

* Get streaming working

* Make AI sidebar wider when left sidebar is collapsed

* Get tests working with streaming responses

* Centralize provider error handling

* Provider data boundaries

---------

Co-authored-by: Josh Pigford <josh@joshpigford.com>
2025-03-28 13:08:22 -04:00
Zach Gollwitzer
19cc63c8f4
Use Redis for ActiveJob and ActionCable (#2004)
* Use Redis for ActiveJob and ActionCable

* Fix alwaysApply setting

* Update queue names and weights

* Tweak weights

* Update job queues

* Update docker setup guide

* Remove deprecated upgrade columns from users table

* Refactor Redis configuration for Sidekiq and caching in production environment

* Add Sidekiq Sentry monitoring

* queue naming fix

* Clean up schema
2025-03-19 12:36:16 -04:00
Zach Gollwitzer
f65b93a352
Data provider simplification, tests, and documentation (#1997)
* Ignore env.test from source control

* Simplification of providers interface

* Synth tests

* Update money to use new find rates method

* Remove unused issues code

* Additional issue feature removals

* Update price data fetching and tests

* Update documentation for providers

* Security test fixes

* Fix self host test

* Update synth usage data access

* Remove AI pr schema changes
2025-03-17 11:54:53 -04:00
Zach Gollwitzer
fa0248056d
Show UI warning to user when they need provider data but have not setup Synth yet (#1926)
* Simplify provider concerns

* Update tests

* Add UI warning for missing Synth key if family requires external data
2025-02-28 11:35:10 -05:00
Tony Vincent
8208722247
Feat: Data "reset" button (#1913)
* feat: Allow admins to delete family data

* feat: Allow self-hosting users to delete cached data

* Remove system tests
2025-02-28 07:49:12 -05:00
Josh Pigford
68d7cb5de6
Enhance security information retrieval and handling (#1826)
* Enhance security information retrieval and handling

- Add support for operating MIC codes in security info fetching
- Update security uniqueness validation to handle unknown securities
- Improve security creation and update logic in Plaid investment sync
- Update combobox and view components to handle operating MIC codes
- Add unknown flag for securities with incomplete information

* Update schema.rb

* Refactor the need for mic codes

* Don't fetch prices unless a security has the necessary mic code

* Deduplication

* Lint

* Update Securities and Plaid Investment Sync

- Modify PlaidInvestmentSync to return plaid_security for USD cash
- Add non-null constraint to Securities ticker column
- Update Securities fixture to use exchange_operating_mic instead of exchange_mic

---------

Signed-off-by: Josh Pigford <josh@joshpigford.com>
2025-02-11 10:40:30 -06:00
Zach Gollwitzer
536c82f2aa
Feature: Add the ability to "revert" a CSV import (#1814)
Some checks failed
Publish Docker image / ci (push) Has been cancelled
Publish Docker image / Build docker image (push) Has been cancelled
* Allow reverting imports

* Fix tests

* Add currency column to all imports

* Don't auto-enrich demo account
2025-02-07 15:36:05 -05:00
Josh Pigford
b84a33c09d
Refactor transaction enrichment to support batch processing (#1803)
* Refactor transaction enrichment to support batch processing

- Add method to enrich transactions in batches
- Implement job scheduling for unenriched transactions
- Improve logging and error handling for transaction enrichment

* Re-enable enrichment

* Fix transaction enrichment query to use correct table references

- Update queries to explicitly join and reference account_entries and account_transactions tables
- Remove unnecessary name presence check before enrichment
- Improve query precision for unenriched transaction selection

* Optimize transaction enrichment query joins

- Refactor database joins to use explicit table references
- Improve query performance for unenriched transaction selection
- Ensure correct table aliasing in enrichment methods

* Remove deprecated data enrichment job and method

- Delete EnrichDataJob as it's no longer used
- Remove `enrich_data_later` method from Account model
- Update Account::Syncer to directly call `enrich_data` instead of scheduling a job
2025-02-05 10:34:28 -06:00
Zach Gollwitzer
3140835f28
Adjust queues to prioritize account syncs (#1682) 2025-01-24 13:39:08 -05:00
Zach Gollwitzer
fe199f2357
Add account data enrichment (#1532)
Some checks failed
Publish Docker image / ci (push) Has been cancelled
Publish Docker image / Build docker image (push) Has been cancelled
* Add data enrichment

* Make data enrichment optional for self-hosters

* Add categories to data enrichment

* Only update category and merchant if nil

* Fix name overrides

* Lint fixes
2024-12-13 17:22:27 -05:00
Zach Gollwitzer
cbba2ba675
Basic Plaid Integration (#1433)
Some checks are pending
Publish Docker image / ci (push) Waiting to run
Publish Docker image / Build docker image (push) Blocked by required conditions
* Basic plaid data model and linking

* Remove institutions, add plaid items

* Improve schema and Plaid provider

* Add webhook verification sketch

* Webhook verification

* Item accounts and balances sync setup

* Provide test encryption keys

* Fix test

* Only provide encryption keys in prod

* Try defining keys in test env

* Consolidate account sync logic

* Add back plaid account initialization

* Plaid transaction sync

* Sync UI overhaul for Plaid

* Add liability and investment syncing

* Handle investment webhooks and process current day holdings

* Remove logs

* Remove "all" period select for performance

* fix amount calc

* Remove todo comment

* Coming soon for investment historical data

* Document Plaid configuration

* Listen for holding updates
2024-11-15 13:49:37 -05:00
Josh Pigford
31d9d926f7 Fix for certain securities returning incorrect prices
Some checks are pending
Publish Docker image / ci (push) Waiting to run
Publish Docker image / Build docker image (push) Blocked by required conditions
2024-11-07 11:52:16 -06:00
Josh Pigford
e7f09e6f71
Groundwork for security info (#1396)
Some checks are pending
Publish Docker image / ci (push) Waiting to run
Publish Docker image / Build docker image (push) Blocked by required conditions
* Groundwork for security info

* Lint
2024-10-30 18:08:19 -04:00
Josh Pigford
cd91e66618
Initial pass at Synth-based ticker selection (#1392)
* Initial pass at Synth-based ticker selection

* Update _tickers.turbo_stream.erb

* Functional combobox display

* A few cleanup steps

* Linter

* Prevent long strings

* Another step towards functional combobox

* Deprecated files

* Custom Combobox implementation

* Lint

* Test suite fixes

* Lint

* Make direct use of mic codes

* Update splits

* Update trades_test.rb
2024-10-30 09:23:44 -04:00
Josh Pigford
aa3342b0dc
Stock imports (#1363)
* Initial pass

* Marketstack data provider

* Marketstack data provider

* Refactor a bit
2024-10-24 16:36:50 -05:00
Zach Gollwitzer
c6bdf49f10
Account::Sync model and test fixture simplifications (#968)
* Add sync model

* Fresh fixtures for sync tests

* Sync tests overhaul

* Fix entry tests

* Complete remaining model test updates

* Update system tests

* Update demo data task

* Add system tests back to PR checks

* More simplifications, add empty family to fixtures for easier testing
2024-07-10 11:22:59 -04:00
Zach Gollwitzer
45ae4a9737
CSV Transaction Imports (#708)
Introduces a basic CSV import module for bulk-importing account transactions.

Changes include:

- User can load a CSV
- User can configure the column mappings for a CSV
- Imported CSV shows invalid cells
- User can clean up their data directly in the UI
- User can see a preview of the import rows and confirm import
- Layout refactor + Import nav stepper
- System test stability improvements
2024-05-17 09:09:32 -04:00
Josh Brown
19ee773d9b
Add ability to delete Maybe account (#698)
* Build out user deactivation and purging workflows

* Add i18n translations for user deletion

* Add tests for user deletion

* Fix lint issue
2024-04-30 11:40:31 -04:00
Jakub Kottnauer
ad4de99f1a
Add partial account sync support (#653)
* Add partial account sync support

* Fix indentation

* Use historical balance as base when doing partial sync

* Simplify controller crud tests

* Fix linter errors
2024-04-24 15:55:52 -04:00
Zach Gollwitzer
5aca2ff9b6
Add zero-config self hosting on Render (#612)
* v1 of backend implementation for self hosting

* Add docs

* Add upgrades controller

* Add global helpers for self hosting mode

* Add self host settings controller

* Conditionally show self hosting settings

* Environment and config updates

* Complete upgrade prompting flow

* Update config for forked repo

* Move configuration of github provider within class

* Add upgrades cron

* Update deploy button

* Update guides

* Fix render deployer

* Typo

* Enable auto upgrades

* Fix cron

* Make upgrade modes more clear and consistent

* Trigger new available version

* Fix logic for displaying upgrade prompts

* Finish implementation

* Fix regression

* Trigger new version

* Add i18n translations

* trigger new version

* reduce caching time for testing

* Decrease cache for testing

* trigger upgrade

* trigger upgrade

* Only trigger deploy once

* trigger upgrade

* If target is commit, always upgrade if any upgrade is available

* trigger upgrade

* trigger upgrade

* Test release

* Change back to maybe repo for defaults

* Fix lint errors

* Clearer naming

* Fix relative link

* Add abs path

* Relative link

* Update docs
2024-04-13 09:28:45 -04:00
Zach Gollwitzer
110855d077
Multi-Currency Part 2 (#543)
* Support all currencies, handle outside DB

* Remove currencies from seed

* Fix account balance namespace

* Set default currency on authentication

* Cache currency instances

* Implement multi-currency syncs with tests

* Series fallback, passing tests

* Fix conflicts

* Make value group concrete class that works with currency values

* Fix migration conflict

* Update tests to expect multi-currency results

* Update account list to use group method

* Namespace updates

* Fetch unknown exchange rates from API

* Fix date range bug

* Ensure demo data works without external API

* Enforce cascades only at DB level
2024-03-21 13:39:10 -04:00
Josh Pigford
4843cf22c6
Use Synth for exchange rates (#514)
* Switch currency seeding over to Synth

* Switch all exchange rates over to Synth
2024-03-04 10:26:20 -06:00
Zach Gollwitzer
dbf575c02a
Basic Account Balance Sync Algorithm (#501)
* Sketch out sync interface

* Add basic account sync algorithm

* Update logic for final balance in series

* Remove start_date concept

* Clean up tests

* Improve clarity of test

* Update app/models/account.rb

Co-authored-by: Rob Zolkos <rob@zolkos.com>
Signed-off-by: Zach Gollwitzer <zach.gollwitzer@gmail.com>

* Update app/models/transaction.rb

Co-authored-by: Rob Zolkos <rob@zolkos.com>
Signed-off-by: Zach Gollwitzer <zach.gollwitzer@gmail.com>

* Update app/models/valuation.rb

Co-authored-by: Rob Zolkos <rob@zolkos.com>
Signed-off-by: Zach Gollwitzer <zach.gollwitzer@gmail.com>

* Re-organize code, simplify job interface

* Consolidate balance calculations

* More cleanup

---------

Signed-off-by: Zach Gollwitzer <zach.gollwitzer@gmail.com>
Co-authored-by: Rob Zolkos <rob@zolkos.com>
2024-02-29 08:32:52 -05:00
Zach Gollwitzer
7d48c01833
Prepare fixture data for account sync tests (#493)
* Rename account balance field for clarity

`original_balance` and `original_currency` may infer that these values are "original" to the account.  In reality, they represent the "current" balance and currency on the account.

* Prepare fixture data for account sync testing

* Update to new field

* Fix conflicts

* Remove local schema change
2024-02-27 12:43:49 -05:00
Zach Gollwitzer
7e324f1b53
Scaffold out Account Syncing (#474)
* Add trends, time series, seed data

* Remove test data

* Replace old view values with helpers

* Fix tooltip bugs in D3 chart

* Fix tests

* Fix smoke test

* Add CRUD actions for valuations

* Scaffold out inline editing with Turbo

* Refactor series logic

* Scaffold out basic sync process for accounts

* Fix tests
2024-02-22 11:35:06 -05:00
Josh Pigford
aa351ae616
Multi-currency support (#425)
* Initial foundational pass at multi-currency

* Default format currency

* More work on currency and exchanging

* Re-build currencies on change

* Currency import/setup

* Background job overhaul + cheaper OXR plan support

* Lint fixes

* Test fixes

* Multi-currency setup instructions

* Allow decimals in the balance field

* Spacing fix for form

---------

Signed-off-by: Josh Pigford <josh@joshpigford.com>
2024-02-10 16:18:56 -06:00
Josh Pigford
99de24ac70 Initial commit 2024-02-02 09:05:04 -06:00