WIP: Introduce have_reported_error matcher by skatkov · Pull Request #2849 · rspec/rspec-rails

skatkov · 2025-06-04T22:53:33Z

rspec-rails is missing support for Rails ErrorReporter, this was introduced to rails in v7 and has been evolving ever since. With my client, we have moved to using ErrorReporter as a unified error reporting interface, so we can easily move from one error tracking software to another with minimal code changes. And we had a need to test this interface with rspec, so we implemented our own matcher to handle this.

I'm suggesting our internal implementation as is. This is probably not suitable as is for this gem, but I'd like to open discussion with this starting point.

Example usage

@example Checking for any error
  expect { Rails.error.report(StandardError.new) }.to have_reported_error

@example Checking for specific error class
  expect { Rails.error.report(MyError.new) }.to have_reported_error(MyError)

@example Checking for specific error instance with message
  expect { Rails.error.report(MyError.new("message")) }.to have_reported_error(MyError.new("message"))

@example Checking error attributes
  expect { Rails.error.report(StandardError.new, context: "test") }.to have_reported_error.with_context(section: "test")

@example Checking error message patterns
  expect { Rails.error.report(StandardError.new("test message")) }.to have_reported_error(/test/)

@example Negation
  expect { "safe code" }.not_to have_reported_error

TODO

Outline of things that we want to do before marking this as completed.

Use does_not_match? for negated matcher.
Use a more strict values_match?(@attributes, actual) for matcher
Let's do matching_reports method.
WIP: Introduce have_reported_error matcher #2849 (comment)

pirj

Thank you!

JonRowe · 2025-06-25T17:14:51Z

+  Scenario: Using in controller specs
+    Given a file named "spec/controllers/users_controller_spec.rb" with:
+      """ruby
+      require "rails_helper"
+
+      RSpec.describe UsersController, type: :controller do
+        describe "POST #create" do
+          it "reports validation errors" do
+            expect {
+              post :create, params: { user: { email: "invalid" } }
+            }.to have_reported_error(ValidationError)
+          end
+        end
+      end
+      """
+    When I run `rspec spec/controllers/users_controller_spec.rb`
+    Then the examples should all pass
+
+  Scenario: Using in request specs
+    Given a file named "spec/requests/users_spec.rb" with:
+      """ruby
+      require "rails_helper"
+
+      RSpec.describe "Users", type: :request do
+        describe "POST /users" do
+          it "reports processing errors" do
+            expect {
+              post "/users", params: { user: { name: "Test" } }
+            }.to have_reported_error.with(context: "user_creation")
+          end
+        end
+      end
+      """
+    When I run `rspec spec/requests/users_spec.rb`
+    Then the examples should all pass
+


This isn't a controller / request specific matcher so these feel out of place, why these, why not every aspect of Rails... I'd just cut them.

JonRowe · 2025-06-25T17:25:27Z

👋 I think this would be a great addition, sorry for the size of the review its been on my todo list for a while, its mostly just grammar / wording tweaks plus a few "fit" things.

The one change I do want to see though is dropping instance matching, I don't think it makes sense over just providing class / message / using with.

skatkov · 2025-06-27T11:45:47Z

@JonRowe thanks for review.

The one change I do want to see though is dropping instance matching, I don't think it makes sense over just providing class / message / using with.

Which of these you would rather go with?
.have_reported_error(StandardError, /wrong attributes/).with(param1: 'test')

or

.have_reported_error(StandardError).with(message:/wrong attributes/, params1: 'test1')

??

I assume, that first option is prefered to keep similarity with raised_error matcher.
https://rspec.info/features/3-13/rspec-expectations/built-in-matchers/raise-error/

JonRowe · 2025-06-27T11:59:45Z

Given that you already have with, I think:

.have_reported_error(StandardError).with(/wrong attributes/, params1: 'test1')

But if you insisted on message: in with I could be persuaded

pirj · 2025-06-27T14:31:21Z

It makes sense to separate matching of args passed to report and exception’s own args, including the message. Both can receive args, right? It would open a way for ambiguity and confusion.
raise_error accepts have_attributes as an argument matcher. It supports with_message. Wdyt if we reserve with for additional args passed to report, but keep it separate from with_message?

skatkov · 2025-06-29T18:44:41Z

It makes sense to separate matching of args passed to report and exception’s own args, including the message. Both can receive args, right? It would open a way for ambiguity and confusion.

The only argument that the exception class accepts is "message". We can of course subclass it and support way more than that, but I'm not sure if we should support this in a matcher?

I have a feeling, that having a matcher interface similar to raise_error would make it simpler for people to memorize it. I don't mind changing .with to .have_attributes as well.

JonRowe · 2025-06-30T08:02:50Z

I hadn't clocked that the extra attributes were in addition to the error, I think matching the raise_error interface in terms of report_error(ErrorClass, message_or_regexp) makes sense with the existing with becoming something clearer, with_metadata? What does Rails call the extra data?

skatkov · 2025-06-30T09:58:19Z

Here is a source code for Rails.error.report.

https://github.com/rails/rails/blob/36601bbb02bc3570f5609db2127c77afca575d6c/activesupport/lib/active_support/error_reporter.rb#L233C23-L233C112

These attributes in rails case have a name "context". so with_context might be more appropriate.

pirj · 2025-07-11T18:55:38Z

+        end
+
+        def failure_message
+          if !@error_subscriber.events.empty? && !@attributes.empty?


Cosmetic: any? or present??

pirj · 2025-07-11T18:57:19Z

+
+        def failure_message
+          if !@error_subscriber.events.empty? && !@attributes.empty?
+            event_context = @error_subscriber.events.last.attributes[:context]


Won’t it be confusing to use last when several errors were reported?

pirj · 2025-07-11T19:07:30Z

+              return "Expected error message to be '#{@expected_message}', but got: #{reported_errors}"
+            end
+          else
+            if @expected_error && !actual_error.is_a?(@expected_error)


actual_error should be is_a? expected_class, no? Just by looking at the implementation. Or what is the case when they won’t match?

pirj · 2025-07-12T09:56:44Z

+
+        private
+
+        def error_matches_expectation?


Is the last condition necessary? We already check if it’s empty? before calling this method.

This particular method checks that the error matches one that we expect, not just that the error occurred.

Here we check if @reports.empty? and return.

Right after the check, we call error_matches_expectation?.

The last condition check @reports.count.positive? here is redundant.

pirj

A fee minor things and simplifications and it looks good to go.

Let’s leave out matching multiple with chains, severity qualifiers, and all extras for later.

pirj · 2025-07-12T13:18:59Z

+      #
+      # @param expected_error_or_message [Class, String, Regexp, nil] the expected error class, message string, or message pattern
+      # @param expected_message [String, Regexp, nil] the expected error message to match
+      def have_reported_error(expected_error_or_message = UndefinedValue, expected_message = nil)


We have default attributes both here and in the initializer. Worth leaving just here?

There are still defaults both here and there:

def initialize(expected_error_or_message = UndefinedValue, expected_message = nil)

Let's remove there.

pirj · 2025-07-12T13:27:05Z

+        end
+
+        def self.process_with_context
+          Rails.error.report(ArgumentError.new("Invalid input"), context: { context: "user_processing", severity: :error })


Nested context here, too. Rename inner to topic: or :section?

Commit grammar improvements Co-authored-by: Jon Rowe <mail@jonrowe.co.uk>

Co-authored-by: Phil Pirozhkov <pirj@users.noreply.github.com>

skatkov force-pushed the have-reported-error branch from 8b837a8 to d2d0e51 Compare June 7, 2025 20:44

pirj reviewed Jun 7, 2025

View reviewed changes

pirj reviewed Jun 21, 2025

View reviewed changes