-
-
Notifications
You must be signed in to change notification settings - Fork 1.4k
In-memory order updater #5872
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
In-memory order updater #5872
Conversation
27f19a8
to
340032c
Compare
d22210b
to
d244646
Compare
Just making a note that we are waiting on Alistair to rebase #6026 against this. @AlistairNorman could you just make sure that gets done at some point before the 20th? That way we can have it for our next session. |
a3bb1fc
to
06e3a2a
Compare
9fd5c8d
to
27e4988
Compare
4a0f623
to
90f5420
Compare
While working on the in-memory updater in solidusio#5872, we found the need to change how item totals were being calculated, so that we could mark adjustments for destruction without actually destroying them, while still keeping tax adjustments intact. This change is completely backwards-compatible with the current OrderUpdater, so to reduce the scope of our PR, we wanted to make this change separately. Since the OrderUpdater is already very large, this helps reduce its responsibilities and makes it easier to test this behaviour. We don't see it as necessary to make this a configurable class, but this change leaves that option open in the future. Co-authored-by: Adam Mueller <[email protected]> Co-authored-by: Alistair Norman <[email protected]> Co-authored-by: Chris Todorov <[email protected]> Co-authored-by: Harmony Bouvier <[email protected]> Co-authored-by: Sofia Besenski <[email protected]> Co-authored-by: benjamin wil <[email protected]>
While working on the in-memory updater in solidusio#5872, we found the need to change how item totals were being calculated, so that we could mark adjustments for destruction without actually destroying them, while still keeping tax adjustments intact. This change is completely backwards-compatible with the current OrderUpdater, so to reduce the scope of our PR, we wanted to make this change separately. Since the OrderUpdater is already very large, this helps reduce its responsibilities and makes it easier to test this behaviour. We don't see it as necessary to make this a configurable class, but this change leaves that option open in the future. Co-authored-by: Adam Mueller <[email protected]> Co-authored-by: Alistair Norman <[email protected]> Co-authored-by: Chris Todorov <[email protected]> Co-authored-by: Harmony Bouvier <[email protected]> Co-authored-by: Sofia Besenski <[email protected]> Co-authored-by: benjamin wil <[email protected]>
While working on the in-memory updater in solidusio#5872, we found the need to change how item totals were being calculated, so that we could mark adjustments for destruction without actually destroying them, while still keeping tax adjustments intact. This change is completely backwards-compatible with the current OrderUpdater, so to reduce the scope of our PR, we wanted to make this change separately. Since the OrderUpdater is already very large, this helps reduce its responsibilities and makes it easier to test this behaviour. We don't see it as necessary to make this a configurable class, but this change leaves that option open in the future. Co-authored-by: Adam Mueller <[email protected]> Co-authored-by: Alistair Norman <[email protected]> Co-authored-by: Chris Todorov <[email protected]> Co-authored-by: Harmony Bouvier <[email protected]> Co-authored-by: Sofia Besenski <[email protected]> Co-authored-by: benjamin wil <[email protected]>
While working on the in-memory updater in solidusio#5872, we found the need to change how item totals were being calculated, so that we could mark adjustments for destruction without actually destroying them, while still keeping tax adjustments intact. This change is completely backwards-compatible with the current OrderUpdater, so to reduce the scope of our PR, we wanted to make this change separately. Since the OrderUpdater is already very large, this helps reduce its responsibilities and makes it easier to test this behaviour. We don't see it as necessary to make this a configurable class, but this change leaves that option open in the future. Co-authored-by: Adam Mueller <[email protected]> Co-authored-by: Alistair Norman <[email protected]> Co-authored-by: Chris Todorov <[email protected]> Co-authored-by: Harmony Bouvier <[email protected]> Co-authored-by: Sofia Besenski <[email protected]> Co-authored-by: benjamin wil <[email protected]>
9ecbd13
to
92fc679
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #5872 +/- ##
==========================================
+ Coverage 89.35% 92.71% +3.35%
==========================================
Files 961 399 -562
Lines 20195 8101 -12094
==========================================
- Hits 18046 7511 -10535
+ Misses 2149 590 -1559 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
92fc679
to
e7faef3
Compare
f3d92a4
to
a5ed6c6
Compare
d0322b2
to
07add2a
Compare
d22dbf5
to
770e527
Compare
770e527
to
c8e434e
Compare
7bf4adb
to
2d497bd
Compare
67186f3
to
defde7e
Compare
5423a0e
to
62209cf
Compare
2b60e3b
to
083fafe
Compare
This puts all the update and recalculate methods together. Co-Authored-By: Adam Mueller <[email protected]> Co-Authored-By: benjamin wil <[email protected]> Co-Authored-By: Andrew Stewart <[email protected]> Co-Authored-By: Harmony Bouvier <[email protected]> Co-Authored-By: Jared Norman <[email protected]> Co-Authored-By: Kendra Riga <[email protected]> Co-Authored-By: Sofia Besenski <[email protected]> Co-Authored-By: Chris Todorov <[email protected]> Co-Authored-By: Tom Van Manen <[email protected]> Co-Authored-By: Noah Silvera <[email protected]>
Co-authored-by: Adam Mueller <[email protected]> Co-authored-by: Andrew Stewart <[email protected]> Co-authored-by: Jared Norman <[email protected]> Co-authored-by: Noah Silvera <[email protected]> Co-authored-by: benjamin wil <[email protected]> Co-authored-by: Alistair Norman <[email protected]>
Includes a small refactor to the internal recalculate method to simplify the code while maintaining the existing logic around only persisting when the values have changed. We'll use this persist flag to eventually only save changes to the DB when requested. Allowing us to use this adjuster to update the order in-memory. Co-authored-by: An Stewart <[email protected]> Co-authored-by: Harmony Evangelina <[email protected]> Co-authored-by: Kendra Riga <[email protected]> Co-authored-by: Nick Van Doorn <[email protected]> Co-authored-by: Sofia Besenski <[email protected]> Co-authored-by: benjamin wil <[email protected]>
Similar to the previous change. We're now passing the persist flag down to all promotion order adjusters. This does not implement the logic within the individual adjuster classes to skip persistance when required, only ensures the flag is pass down from our in-memory order updater. Co-Authored-By: Adam Mueller <[email protected]> Co-Authored-By: Andrew Stewart <[email protected]> Co-Authored-By: Harmony Evangelina <[email protected]> Co-Authored-By: Jared Norman <[email protected]> Co-Authored-By: Kendra Riga <[email protected]> Co-Authored-By: Senem Soy <[email protected]> Co-Authored-By: benjamin wil <[email protected]>
Previously this would update the eligible column. We now only assign the value and then save if persist is true. Co-Authored-By: Adam Mueller <[email protected]> Co-Authored-By: Andrew Stewart <[email protected]> Co-Authored-By: Harmony Evangelina <[email protected]> Co-Authored-By: Jared Norman <[email protected]> Co-Authored-By: Kendra Riga <[email protected]> Co-Authored-By: benjamin wil <[email protected]> Co-Authored-By: Chris Todorov <[email protected]> Co-Authored-By: Nick Van Doorn <[email protected]> Co-Authored-By: Sofia Besenski <[email protected]>
We were missing the whole path of doing order level adjustments, e.g. The CreateDiscountedItem benefit.
This is in service of the in-memory order updater. Co-authored-by: Adam Mueller <[email protected]> Co-authored-by: Alistair Norman <[email protected]> Co-authored-by: Andrew Stewart <[email protected]> Co-authored-by: benjamin wil<[email protected]> Co-authored-by: Chris Todorov <[email protected]> Co-authored-by: Harmony Bouvier <[email protected]> Co-authored-by: Jared Norman <[email protected]> Co-authored-by: Nick Van Doorn <[email protected]> Co-authored-by: Noah Silvera <[email protected]> Co-authored-by: Senem Soy <[email protected]> Co-authored-by: Sofia Besenski <[email protected]>
Instead of destroying. See code comments and spec changes. This is similar to how we handled this in the legacy promotion system. Co-authored-by: Adam Mueller <[email protected]> Co-authored-by: Alistair Norman <[email protected]> Co-authored-by: An Stewart <[email protected]> Co-authored-by: benjamin wil <[email protected]> Co-authored-by: Harmony Evangelina <[email protected]> Co-authored-by: Jared Norman <[email protected]> Co-authored-by: Nick Van Doorn <[email protected]> Co-authored-by: Senem Soy <[email protected]> Co-authored-by: Sofia Besenski <[email protected]>
Integration level test using the InMemoryOrderUpdater to ensure we are not persisting changes during the promotion recalculations. Co-authored-by: Chris Todorov <[email protected]> Co-authored-by: benjamin wil <[email protected]> Co-authored-by: An Stewart <[email protected]> Co-authored-by: Jared Norman <[email protected]>
Integration level test using the InMemoryOrderUpdater to ensure we are not persisting changes during the promotion recalculations. Co-authored-by: Chris Todorov <[email protected]> Co-authored-by: benjamin wil <[email protected]> Co-authored-by: An Stewart <[email protected]> Co-authored-by: Jared Norman <[email protected]>
We want to protect against manipulative database queries in the legacy promotion system We also want to ensure that objects in memory have their attributes changed correctly but not persisted Co-authored-by: Jared Norman <[email protected]> Co-authored-by: Kendra Riga <[email protected]> Co-authored-by: Adam Mueller <[email protected]> Co-authored-by: Alistair Norman <[email protected]> Co-authored-by: Senem Soy <[email protected]> Co-authored-by: Sofia Besenski <[email protected]> Co-authored-by: Noah Silvera <[email protected]>
In an associated change we made the `CreateDiscountedItem` benefit create in-memory line items. That causes an issue during the recalculation of the item totals. Co-authored-by: benjamin wil <[email protected]> Co-authored-by: Kendra Riga <[email protected]> Co-authored-by: Noah Silvera <[email protected]> Co-authored-by: Jared Norman <[email protected]>
This will be used by the in-memory order updater to log manipulative queries when the persist flag is set to false. This will inform application developers whether their application is complying with the in-memory order updater's "no writes" philosophy. Previously the `db-query-matchers` gem we use to check for manipulative queries was only used in tests. However, we now use it in application code so the dependency must be moved to the gemspec. Co-authored-by: Alistair Norman <[email protected]> Co-authored-by: Sofia Besenski <[email protected]> Co-authored-by: Adam Mueller <[email protected]> Co-authored-by: Chris Todorov <[email protected]> Co-authored-by: benjamin wil <[email protected]>
Enabling manipulative query logging will show the number of manipulative queries on each call to Spree::InMemoryOrderUpdater#recalculate. Co-authored-by: Chris Todorov <[email protected]> Co-authored-by: Adam Mueller <[email protected]> Co-authored-by: Alistair Norman <[email protected]> Co-authored-by: Senem Soy <[email protected]> Co-authored-by: benjamin wil <[email protected]>
This method will be used in the in memory order updater. We only want to persist to the database once we perform the action, not when we are computing the amount.
When computing item totals after a line item benefit is removed, we need to ensure we aren't using the marked for destruction items. We also needed to make a change to the legacy promotion order updater patch because it is incorrectly being used in our tests even when using the new promotion system. Co-authored-by: Jared Norman <[email protected]> Co-authored-by: Noah Silvera <[email protected]>
We want to make sure we don't persist anything when computing amounts becuase that will cause issues with the in-memory order updater.
Our legacy promotion order updater patch did not have updated logic to ensure it ignored adjustments that were marked for desctruction. Co-authored-by: Jared Norman <[email protected]> Co-authored-by: Noah Silvera <[email protected]>
Co-authored-by: benjamin wil <[email protected]> Co-authored-by: Chris Todorov <[email protected]>
Instead of calling update_columns, we should just use assign_attributes everywhere and rely on our order persistance bubbling down to the line item when appropriate. This avoids issues with the in-memory order updater when attempting to update values on newly created promotional items that won't get persisted until later in the recalculation flow. Co-authored-by: Jared Norman <[email protected]> Co-authored-by: Noah Silvera <[email protected]>
Sets up some shared examples so that we can run the full promotion integration suite against both the existing order updater and the in-memory order updater. Co-authored-by: Jared Norman <[email protected]> Co-authored-by: Noah Silvera <[email protected]>
Now that we are going to be using parts of this gem in our monitoring class, we need it to be required everywhere. This change also ensures the configuration is shared across the Solidus gems, which previously was not the case. Co-authored-by: benjamin wil <[email protected]> Co-authored-by: Sofia Besenski <[email protected]>
We no longer want to have individual methods use a persist: flag, instead we can save at the end if needed. Co-authored-by: Chris Todorov <[email protected]> Co-authored-by: Senem Soy <[email protected]> Co-authored-by: benjamin wil <[email protected]>
We no longer want to have individual methods use a persist: flag, instead we can save at the end if needed.
2d124da
to
6fc0879
Compare
We don't want our in-memory order updater to have to persist shipment state. Because we no longer rely on `Shipment#update_state` which uses `update_column`, we are now logging any additional `Shipment#state` changes when the order is saved. The changes to the specs reflect this change in behaviour. Co-authored-by: Jared Norman <[email protected]> Co-authored-by: Noah Silvera <[email protected]> Co-authored-by: Alistair Norman <[email protected]>
Due to upstream changes after a rebase, these tests started failing because the `order` being set up (`Spree::Order.create`) did not have proper shipments or inventory units created for it. The new tests use factories to more realistically set up orders, shipments, and inventory units in various states to simulate order updater functionality. Co-authored-by: Alistair Norman <[email protected]> Co-authored-by: Kendra Riga <[email protected]> Co-authored-by: Chris Todorov <[email protected]>
Switch to in-memory order updater and log queries to separate file. This is a temporary change to allow us to produce a log of the manipulative queries triggered by running the test suite on CI. Co-authored-by: Benjamin Willems <[email protected]> Co-authored-by: Sofia Besenski <[email protected]>
6fc0879
to
ae58cce
Compare
We want state change logging for shipment and payment state changes. We got to the point of a failing spec - more jobs (and not the ones we want) are being enqueued than we want currently. I left the empty context in for 'logs a state change for the shipment' so it doesn't get forgotten about, but we didn't quite start on it yet.
Summary
This pull request proposes a replacement to the default
Spree::OrderUpdater
that has new and improved functionality:There may be some beneficial side-effects that come out of this new order updater implementation:
We don't expect this to be the default order updater implementation in the next minor version of Solidus, but we would like to propose it as the default for the next major version of Solidus.
Note: The commits on this pull request have a long list of co-authors, as the Super Good team is approaching this as a collaborative mob programming exercise.
Milestones
For this order updater, we intend to achieve the following during updates:
We appreciate that there is a lot of complexity to achieving these goals (dealing with active promotions, for example).
Notes
Spree::OrderUpdater
makes on a typical recalculate.Todo
recalculate_item_total
when line item totals changeInMemoryOrderUpdater
to not marked for destructionOrderUpdater
to not marked for destructionupdate_taxes
Write theInMemoryOrderAdjuster
(also, should we rename this toInMemoryOrderPromotionAdjuster
)LineItem#set_required_attributes
inafter_initialize
instead ofbefore_validation
. We think this may be a breaking change that requiresfurther investigation. (We investigated and don't believe it is a breaking change.)
ensure the above
promotions.order_adjuster_class
Spree::LineItem#set_required_attributes
compute_amount
. We know the create quantity adjustments does. This action persists when compute_amount is called.persist: false
Checklist
Check out our PR guidelines for more details.
The following are mandatory for all PRs:
The following are not always needed: