Transaction rules engine V1 (#1900)

* Domain model sketch

* Scaffold out rules domain

* Migrations

* Remove existing data enrichment for clean slate

* Sketch out business logic and basic tests

* Simplify rule scope building and action executions

* Get generator working again

* Basic implementation + tests

* Remove manual merchant management (rules will replace)

* Revert "Remove manual merchant management (rules will replace)"

This reverts commit 83dcbd9ff0.

* Family and Provider merchants model

* Fix brakeman warnings

* Fix notification loader

* Update notification position

* Add Rule action and condition registries

* Rule form with compound conditions and tests

* Split out notification types, add CTA type

* Rules form builder and Stimulus controller

* Clean up rule registry domain

* Clean up rules stimulus controller

* CTA message for rule when user changes transaction category

* Fix tests

* Lint updates

* Centralize notifications in Notifiable concern

* Implement category rule prompts with auto backoff and option to disable

* Fix layout bug caused by merge conflict

* Initialize rule with correct action for category CTA

* Add rule deletions, get rules working

* Complete dynamic rule form, split Stimulus controllers by resource

* Fix failing tests

* Change test password to avoid chromium conflicts

* Update integration tests

* Centralize all test password references

* Add re-apply rule action

* Rule confirm modal

* Run migrations

* Trigger rule notification after inline category updates

* Clean up rule styles

* Basic attribute locking for rules

* Apply attribute locks on user edits

* Log data enrichments, only apply rules to unlocked attributes

* Fix merge errors

* Additional merge conflict fixes

* Form UI improvements, ignore attribute locks on manual rule application

* Batch AI auto-categorization of transactions

* Auto merchant detection, ai enrichment in batches

* Fix Plaid merchant assignments

* Plaid category matching

* Cleanup 1

* Test cleanup

* Remove stale route

* Fix desktop chat UI issues

* Fix mobile nav styling issues

app/controllers/accounts_controller.rb

@@ -31,7 +31,7 @@ class AccountsController < ApplicationController
       family.sync_later
     end
 
-    redirect_to accounts_path
+    redirect_back_or_to accounts_path
   end
 
   private

app/controllers/application_controller.rb

@@ -1,5 +1,5 @@
 class ApplicationController < ActionController::Base
-  include Onboardable, Localize, AutoSync, Authentication, Invitable, SelfHostable, StoreLocation, Impersonatable, Breadcrumbable, FeatureGuardable
+  include Onboardable, Localize, AutoSync, Authentication, Invitable, SelfHostable, StoreLocation, Impersonatable, Breadcrumbable, FeatureGuardable, Notifiable
   include Pagy::Backend
 
   helper_method :require_upgrade?, :subscription_pending?

app/controllers/categories_controller.rb

@@ -56,8 +56,13 @@ class CategoriesController < ApplicationController
     redirect_back_or_to categories_path, notice: t(".success")
   end
 
+  def destroy_all
+    Current.family.categories.destroy_all
+    redirect_back_or_to categories_path, notice: "All categories deleted"
+  end
+
   def bootstrap
-    Current.family.categories.bootstrap_defaults
+    Current.family.categories.bootstrap!
 
     redirect_back_or_to categories_path, notice: t(".success")
   end

app/controllers/concerns/accountable_resource.rb

@@ -37,11 +37,15 @@ module AccountableResource
 
   def create
     @account = Current.family.accounts.create_and_sync(account_params.except(:return_to))
+    @account.lock_saved_attributes!
+
     redirect_to account_params[:return_to].presence || @account, notice: t("accounts.create.success", type: accountable_type.name.underscore.humanize)
   end
 
   def update
     @account.update_with_sync!(account_params.except(:return_to))
+    @account.lock_saved_attributes!
+
     redirect_back_or_to @account, notice: t("accounts.update.success", type: accountable_type.name.underscore.humanize)
   end
 

app/controllers/concerns/notifiable.rb

@@ -0,0 +1,58 @@
+module Notifiable
+  extend ActiveSupport::Concern
+
+  included do
+    helper_method :render_flash_notifications
+    helper_method :flash_notification_stream_items
+  end
+
+  private
+    def render_flash_notifications
+      notifications = flash.flat_map { |type, data| resolve_notifications(type, data) }.compact
+
+      view_context.safe_join(
+        notifications.map { |notification| view_context.render(**notification) }
+      )
+    end
+
+    def flash_notification_stream_items
+      items = flash.flat_map do |type, data|
+        notifications = resolve_notifications(type, data)
+
+        if type == "cta"
+          notifications.map { |notification| turbo_stream.replace("cta", **notification) }
+        else
+          notifications.map { |notification| turbo_stream.append("notification-tray", **notification) }
+        end
+      end.compact
+
+      # If rendering flash notifications via stream, we mark them as used to avoid
+      # them being rendered again on the next page load
+      flash.clear
+
+      items
+    end
+
+    def resolve_cta(cta)
+      case cta[:type]
+      when "category_rule"
+        { partial: "rules/category_rule_cta", locals: { cta: } }
+      end
+    end
+
+    def resolve_notifications(type, data)
+      case type
+      when "alert"
+        [ { partial: "shared/notifications/alert", locals: { message: data } } ]
+      when "cta"
+        [ resolve_cta(data) ]
+      when "loading"
+        [ { partial: "shared/notifications/loading", locals: { message: data } } ]
+      when "notice"
+        messages = Array(data)
+        messages.map { |message| { partial: "shared/notifications/notice", locals: { message: message } } }
+      else
+        []
+      end
+    end
+end

app/controllers/family_merchants_controller.rb

@@ -0,0 +1,54 @@
+class FamilyMerchantsController < ApplicationController
+  before_action :set_merchant, only: %i[edit update destroy]
+
+  def index
+    @breadcrumbs = [ [ "Home", root_path ], [ "Merchants", nil ] ]
+
+    @merchants = Current.family.merchants.alphabetically
+
+    render layout: "settings"
+  end
+
+  def new
+    @merchant = FamilyMerchant.new(family: Current.family)
+  end
+
+  def create
+    @merchant = FamilyMerchant.new(merchant_params.merge(family: Current.family))
+
+    if @merchant.save
+      respond_to do |format|
+        format.html { redirect_to family_merchants_path, notice: t(".success") }
+        format.turbo_stream { render turbo_stream: turbo_stream.action(:redirect, family_merchants_path) }
+      end
+    else
+      render :new, status: :unprocessable_entity
+    end
+  end
+
+  def edit
+  end
+
+  def update
+    @merchant.update!(merchant_params)
+    respond_to do |format|
+      format.html { redirect_to family_merchants_path, notice: t(".success") }
+      format.turbo_stream { render turbo_stream: turbo_stream.action(:redirect, family_merchants_path) }
+    end
+  end
+
+  def destroy
+    @merchant.destroy!
+    redirect_to family_merchants_path, notice: t(".success")
+  end
+
+  private
+
+    def set_merchant
+      @merchant = Current.family.merchants.find(params[:id])
+    end
+
+    def merchant_params
+      params.require(:family_merchant).permit(:name, :color)
+    end
+end

app/controllers/merchants_controller.rb

@@ -1,46 +0,0 @@
-class MerchantsController < ApplicationController
-  before_action :set_merchant, only: %i[edit update destroy]
-
-  def index
-    @merchants = Current.family.merchants.alphabetically
-
-    render layout: "settings"
-  end
-
-  def new
-    @merchant = Merchant.new
-  end
-
-  def create
-    @merchant = Current.family.merchants.new(merchant_params)
-
-    if @merchant.save
-      redirect_to merchants_path, notice: t(".success")
-    else
-      redirect_to merchants_path, alert: t(".error", error: @merchant.errors.full_messages.to_sentence)
-    end
-  end
-
-  def edit
-  end
-
-  def update
-    @merchant.update!(merchant_params)
-    redirect_to merchants_path, notice: t(".success")
-  end
-
-  def destroy
-    @merchant.destroy!
-    redirect_to merchants_path, notice: t(".success")
-  end
-
-  private
-
-    def set_merchant
-      @merchant = Current.family.merchants.find(params[:id])
-    end
-
-    def merchant_params
-      params.require(:merchant).permit(:name, :color)
-    end
-end

app/controllers/rules_controller.rb

@@ -0,0 +1,77 @@
+class RulesController < ApplicationController
+  include StreamExtensions
+
+  before_action :set_rule, only: [  :edit, :update, :destroy, :apply, :confirm ]
+
+  def index
+    @rules = Current.family.rules.order(created_at: :desc)
+    render layout: "settings"
+  end
+
+  def new
+    @rule = Current.family.rules.build(
+      resource_type: params[:resource_type] || "transaction",
+    )
+  end
+
+  def create
+    @rule = Current.family.rules.build(rule_params)
+
+    if @rule.save
+      redirect_to confirm_rule_path(@rule)
+    else
+      render :new, status: :unprocessable_entity
+    end
+  end
+
+  def apply
+    @rule.update!(active: true)
+    @rule.apply_later(ignore_attribute_locks: true)
+    redirect_back_or_to rules_path, notice: "#{@rule.resource_type.humanize} rule activated"
+  end
+
+  def confirm
+  end
+
+  def edit
+  end
+
+  def update
+    if @rule.update(rule_params)
+      respond_to do |format|
+        format.html { redirect_back_or_to rules_path, notice: "Rule updated" }
+        format.turbo_stream { stream_redirect_back_or_to rules_path, notice: "Rule updated" }
+      end
+    else
+      render :edit, status: :unprocessable_entity
+    end
+  end
+
+  def destroy
+    @rule.destroy
+    redirect_to rules_path, notice: "Rule deleted"
+  end
+
+  def destroy_all
+    Current.family.rules.destroy_all
+    redirect_to rules_path, notice: "All rules deleted"
+  end
+
+  private
+    def set_rule
+      @rule = Current.family.rules.find(params[:id])
+    end
+
+    def rule_params
+      params.require(:rule).permit(
+        :resource_type, :effective_date, :active,
+        conditions_attributes: [
+          :id, :condition_type, :operator, :value, :_destroy,
+          sub_conditions_attributes: [ :id, :condition_type, :operator, :value, :_destroy ]
+        ],
+        actions_attributes: [
+          :id, :action_type, :value, :_destroy
+        ]
+      )
+    end
+end

app/controllers/trades_controller.rb

@@ -48,7 +48,7 @@ class TradesController < ApplicationController
 
     def entry_params
       params.require(:entry).permit(
-        :name, :enriched_name, :date, :amount, :currency, :excluded, :notes, :nature,
+        :name, :date, :amount, :currency, :excluded, :notes, :nature,
         entryable_attributes: [ :id, :qty, :price ]
       )
     end

app/controllers/transaction_categories_controller.rb

@@ -1,16 +1,34 @@
 class TransactionCategoriesController < ApplicationController
+  include ActionView::RecordIdentifier
+
   def update
     @entry = Current.family.entries.transactions.find(params[:transaction_id])
     @entry.update!(entry_params)
 
+    transaction = @entry.transaction
+
+    if needs_rule_notification?(transaction)
+      flash[:cta] = {
+        type: "category_rule",
+        category_id: transaction.category_id,
+        category_name: transaction.category.name
+      }
+    end
+
+    transaction.lock_saved_attributes!
+    @entry.lock_saved_attributes!
+
     respond_to do |format|
       format.html { redirect_back_or_to transaction_path(@entry) }
       format.turbo_stream do
-        render turbo_stream: turbo_stream.replace(
-          "category_menu_transaction_#{@entry.transaction_id}",
-          partial: "categories/menu",
-          locals: { transaction: @entry.transaction }
-        )
+        render turbo_stream: [
+          turbo_stream.replace(
+            dom_id(transaction, :category_menu),
+            partial: "categories/menu",
+            locals: { transaction: transaction }
+          ),
+          *flash_notification_stream_items
+        ]
       end
     end
   end
@@ -19,4 +37,16 @@ class TransactionCategoriesController < ApplicationController
     def entry_params
       params.require(:entry).permit(:entryable_type, entryable_attributes: [ :id, :category_id ])
     end
+
+    def needs_rule_notification?(transaction)
+      return false if Current.user.rule_prompts_disabled
+
+      if Current.user.rule_prompt_dismissed_at.present?
+        time_since_last_rule_prompt = Time.current - Current.user.rule_prompt_dismissed_at
+        return false if time_since_last_rule_prompt < 1.day
+      end
+
+      transaction.saved_change_to_category_id? &&
+      transaction.eligible_for_category_rule?
+    end
 end

app/controllers/transactions_controller.rb

@@ -54,6 +54,8 @@ class TransactionsController < ApplicationController
 
     if @entry.save
       @entry.sync_account_later
+      @entry.lock_saved_attributes!
+      @entry.transaction.lock!(:tag_ids) if @entry.transaction.tags.any?
 
       flash[:notice] = "Transaction created"
 
@@ -68,7 +70,19 @@ class TransactionsController < ApplicationController
 
   def update
     if @entry.update(entry_params)
+      transaction = @entry.transaction
+
+      if needs_rule_notification?(transaction)
+        flash[:cta] = {
+          type: "category_rule",
+          category_id: transaction.category_id,
+          category_name: transaction.category.name
+        }
+      end
+
       @entry.sync_account_later
+      @entry.lock_saved_attributes!
+      @entry.transaction.lock!(:tag_ids) if @entry.transaction.tags.any?
 
       respond_to do |format|
         format.html { redirect_back_or_to account_path(@entry.account), notice: "Transaction updated" }
@@ -79,7 +93,8 @@ class TransactionsController < ApplicationController
               partial: "transactions/header",
               locals: { entry: @entry }
             ),
-            turbo_stream.replace(@entry)
+            turbo_stream.replace(@entry),
+            *flash_notification_stream_items
           ]
         end
       end
@@ -89,9 +104,21 @@ class TransactionsController < ApplicationController
   end
 
   private
+    def needs_rule_notification?(transaction)
+      return false if Current.user.rule_prompts_disabled
+
+      if Current.user.rule_prompt_dismissed_at.present?
+        time_since_last_rule_prompt = Time.current - Current.user.rule_prompt_dismissed_at
+        return false if time_since_last_rule_prompt < 1.day
+      end
+
+      transaction.saved_change_to_category_id? &&
+      transaction.eligible_for_category_rule?
+    end
+
     def entry_params
       entry_params = params.require(:entry).permit(
-        :name, :enriched_name, :date, :amount, :currency, :excluded, :notes, :nature, :entryable_type,
+        :name, :date, :amount, :currency, :excluded, :notes, :nature, :entryable_type,
         entryable_attributes: [ :id, :category_id, :merchant_id, { tag_ids: [] } ]
       )
 

app/controllers/users_controller.rb

@@ -49,6 +49,11 @@ class UsersController < ApplicationController
     end
   end
 
+  def rule_prompt_settings
+    @user.update!(rule_prompt_settings_params)
+    redirect_back_or_to settings_profile_path
+  end
+
   private
     def handle_redirect(notice)
       case user_params[:redirect_to]
@@ -72,10 +77,14 @@ class UsersController < ApplicationController
       user_params[:email].present? && user_params[:email] != @user.email
     end
 
+    def rule_prompt_settings_params
+      params.require(:user).permit(:rule_prompt_dismissed_at, :rule_prompts_disabled)
+    end
+
     def user_params
       params.require(:user).permit(
         :first_name, :last_name, :email, :profile_image, :redirect_to, :delete_profile_image, :onboarded_at, :show_sidebar, :default_period, :show_ai_sidebar, :ai_enabled, :theme,
-        family_attributes: [ :name, :currency, :country, :locale, :date_format, :timezone, :id, :data_enrichment_enabled ]
+        family_attributes: [ :name, :currency, :country, :locale, :date_format, :timezone, :id ]
       )
     end
 

app/controllers/valuations_controller.rb

@@ -44,6 +44,6 @@ class ValuationsController < ApplicationController
   private
     def entry_params
       params.require(:entry)
-            .permit(:name, :enriched_name, :date, :amount, :currency, :notes)
+            .permit(:name, :date, :amount, :currency, :notes)
     end
 end