Personal finance AI (v1) (#2022)

* AI sidebar

* Add chat and message models with associations

* Implement AI chat functionality with sidebar and messaging system

- Add chat and messages controllers
- Create chat and message views
- Implement chat-related routes
- Add message broadcasting and user interactions
- Update application layout to support chat sidebar
- Enhance user model with initials method

* Refactor AI sidebar with enhanced chat menu and interactions

- Update sidebar layout with dynamic width and improved responsiveness
- Add new chat menu Stimulus controller for toggling between chat and chat list views
- Improve chat list display with recent chats and empty state
- Extract AI avatar to a partial for reusability
- Enhance message display and interaction styling
- Add more contextual buttons and interaction hints

* Improve chat scroll behavior and message styling

- Refactor chat scroll functionality with Stimulus controller
- Optimize message scrolling in chat views
- Update message styling for better visual hierarchy
- Enhance chat container layout with flex and auto-scroll
- Simplify message rendering across different chat views

* Extract AI avatar to a shared partial for consistent styling

- Refactor AI avatar rendering across chat views
- Replace hardcoded avatar markup with a reusable partial
- Simplify avatar display in chats and messages views

* Update sidebar controller to handle right panel width dynamically

- Add conditional width class for right sidebar panel
- Ensure consistent sidebar toggle behavior for both left and right panels
- Use specific width class for right panel (w-[375px])

* Refactor chat form and AI greeting with flexible partials

- Extract message form to a reusable partial with dynamic context support
- Create flexible AI greeting partial for consistent welcome messages
- Simplify chat and sidebar views by leveraging new partials
- Add support for different form scenarios (chat, new chat, sidebar)
- Improve code modularity and reduce duplication

* Add chat clearing functionality with dynamic menu options

- Implement clear chat action in ChatsController
- Add clear chat route to support clearing messages
- Update AI sidebar with dropdown menu for chat actions
- Preserve system message when clearing chat
- Enhance chat interaction with new menu options

* Add frontmatter to project structure documentation

- Create initial frontmatter for structure.mdc file
- Include description and configuration options
- Prepare for potential dynamic documentation rendering

* Update general project rules with additional guidelines

- Add rule for using `Current.family` instead of `current_family`
- Include new guidelines for testing, API routes, and solution approach
- Expand project-specific rules for more consistent development practices

* Add OpenAI gem and AI-friendly data representations

- Add `ruby-openai` gem for AI integration
- Implement `to_ai_readable_hash` methods in BalanceSheet and IncomeStatement
- Include Promptable module in both models
- Add savings rate calculation method in IncomeStatement
- Prepare financial models for AI-powered insights and interactions

* Enhance AI Financial Assistant with Advanced Querying and Debugging Capabilities

- Implement comprehensive AI financial query system with function-based interactions
- Add detailed debug logging for AI responses and function calls
- Extend BalanceSheet and IncomeStatement models with AI-friendly methods
- Create robust error handling and fallback mechanisms for AI queries
- Update chat and message views to support debug mode and enhanced rendering
- Add AI query routes and initial test coverage for financial assistant

* Refactor AI sidebar and chat layout with improved structure and comments

- Remove inline AI chat from application layout
- Enhance AI sidebar with more semantic HTML structure
- Add descriptive comments to clarify different sections of chat view
- Improve flex layout and scrolling behavior in chat messages container
- Optimize message rendering with more explicit class names and structure

* Add Markdown rendering support for AI chat messages

- Implement `markdown` helper method in ApplicationHelper using Redcarpet
- Update message view to render AI messages with Markdown formatting
- Add comprehensive Markdown rendering options (tables, code blocks, links)
- Enhance AI Financial Assistant prompt to encourage Markdown usage
- Remove commented Markdown CSS in Tailwind application stylesheet

* Missing comma

* Enhance AI response processing with chat history context

* Improve AI debug logging with payload size limits and internal message flag

* Enhance AI chat interaction with improved thinking indicator and scrolling behavior

* Add AI consent and enable/disable functionality for AI chat

* Upgrade Biome and refactor JavaScript template literals

- Update @biomejs/biome to latest version with caret (^) notation
- Refactor AI query and chat controllers to use template literals
- Standardize npm scripts formatting in package.json

* Add beta testing usage note to AI consent modal

* Update test fixtures and configurations for AI chat functionality

- Add family association to chat fixtures and tests
- Set consistent password digest for test users
- Enable AI for test users
- Add OpenAI access token for test environment
- Update chat and user model tests to include family context

* Simplify data model and get tests passing

* Remove structure.mdc from version control

* Integrate AI chat styles into existing prose pattern

* Match Figma design spec, implement Turbo frames and actions for chats controller

* AI rules refresh

* Consolidate Stimulus controllers, thinking state, controllers, and views

* Naming, domain alignment

* Reset migrations

* Improve data model to support tool calls and message types

* Tool calling tests and fixtures

* Tool call implementation and test

* Get assistant test working again

* Test updates

* Process tool calls within provider

* Chat UI back to working state again

* Remove stale code

* Tests passing

* Update openai class naming to avoid conflicts

* Reconfigure test env

* Rebuild gemfile

* Fix naming conflicts for ChatResponse

* Message styles

* Use OpenAI conversation state management

* Assistant function base implementation

* Add back thinking messages, clean up error handling for chat

* Fix sync error when security price has bad data from provider

* Add balance sheet function to assistant

* Add better function calling error visibility

* Add income statement function

* Simplify and clean up "thinking" interactions with Turbo frames

* Remove stale data definitions from functions

* Ensure VCR fixtures working with latest code

* basic stream implementation

* Get streaming working

* Make AI sidebar wider when left sidebar is collapsed

* Get tests working with streaming responses

* Centralize provider error handling

* Provider data boundaries

---------

Co-authored-by: Josh Pigford <josh@joshpigford.com>

test/controllers/chats_controller_test.rb

@@ -0,0 +1,52 @@
+require "test_helper"
+
+class ChatsControllerTest < ActionDispatch::IntegrationTest
+  setup do
+    @user = users(:family_admin)
+    @family = families(:dylan_family)
+    sign_in @user
+  end
+
+  test "cannot create a chat if AI is disabled" do
+    @user.update!(ai_enabled: false)
+    post chats_url, params: { chat: { content: "Hello", ai_model: "gpt-4o" } }
+    assert_response :forbidden
+  end
+
+  test "gets index" do
+    get chats_url
+    assert_response :success
+  end
+
+  test "creates chat" do
+    assert_difference("Chat.count") do
+      post chats_url, params: { chat: { content: "Hello", ai_model: "gpt-4o" } }
+    end
+
+    assert_redirected_to chat_path(Chat.order(created_at: :desc).first, thinking: true)
+  end
+
+  test "shows chat" do
+    get chat_url(chats(:one))
+    assert_response :success
+  end
+
+  test "destroys chat" do
+    assert_difference("Chat.count", -1) do
+      delete chat_url(chats(:one))
+    end
+
+    assert_redirected_to chats_url
+  end
+
+  test "should not allow access to other user's chats" do
+    other_user = users(:family_member)
+    other_chat = Chat.create!(user: other_user, title: "Other User's Chat")
+
+    get chat_url(other_chat)
+    assert_response :not_found
+
+    delete chat_url(other_chat)
+    assert_response :not_found
+  end
+end

test/controllers/messages_controller_test.rb

@@ -0,0 +1,22 @@
+require "test_helper"
+
+class MessagesControllerTest < ActionDispatch::IntegrationTest
+  setup do
+    sign_in @user = users(:family_admin)
+    @chat = @user.chats.first
+  end
+
+  test "can create a message" do
+    post chat_messages_url(@chat), params: { message: { content: "Hello", ai_model: "gpt-4o" } }
+
+    assert_redirected_to chat_path(@chat, thinking: true)
+  end
+
+  test "cannot create a message if AI is disabled" do
+    @user.update!(ai_enabled: false)
+
+    post chat_messages_url(@chat), params: { message: { content: "Hello", ai_model: "gpt-4o" } }
+
+    assert_response :forbidden
+  end
+end

test/controllers/settings/hostings_controller_test.rb

@@ -8,7 +8,7 @@ class Settings::HostingsControllerTest < ActionDispatch::IntegrationTest
     sign_in users(:family_admin)
 
     @provider = mock
-    Providers.stubs(:synth).returns(@provider)
+    Provider::Registry.stubs(:get_provider).with(:synth).returns(@provider)
     @usage_response = provider_success_response(
       OpenStruct.new(
         used: 10,
@@ -20,12 +20,12 @@ class Settings::HostingsControllerTest < ActionDispatch::IntegrationTest
   end
 
   test "cannot edit when self hosting is disabled" do
-    assert_raises(RuntimeError, "Settings not available on non-self-hosted instance") do
+    with_env_overrides SELF_HOSTED: "false" do
       get settings_hosting_url
-    end
+      assert_response :forbidden
 
-    assert_raises(RuntimeError, "Settings not available on non-self-hosted instance") do
       patch settings_hosting_url, params: { setting: { require_invite_for_signup: true } }
+      assert_response :forbidden
     end
   end
 
@@ -40,8 +40,6 @@ class Settings::HostingsControllerTest < ActionDispatch::IntegrationTest
 
   test "can update settings when self hosting is enabled" do
     with_self_hosting do
-      assert_nil Setting.synth_api_key
-
       patch settings_hosting_url, params: { setting: { synth_api_key: "1234567890" } }
 
       assert_equal "1234567890", Setting.synth_api_key