-
Notifications
You must be signed in to change notification settings - Fork 8
Update Quick Start docs #296
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
jamesnaftel
wants to merge
9
commits into
main
Choose a base branch
from
james/ch4210
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
6053b6c
First draft of ruby instructions
jamesnaftel da78f3d
link from main README.md to Ruby
jamesnaftel 4f8a1a2
updated based on Ross' comments and added observation mode
jamesnaftel 0fd4892
add link to pypi docs
jamesnaftel c67d762
Added Christina's PR feedback.
jamesnaftel 6320aee
fixed formatting and added support email
jamesnaftel 4542623
Update docs/yourbase-ruby.md
jamesnaftel 0174eac
updated based on PR comments
jamesnaftel 10dd863
clarity and naming improvements
jamesnaftel File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| # YourBase Ruby Test Selection | ||
|
|
||
| Tests are important. For large monoliths, they're also a major source of drag on velocity. | ||
|
|
||
| YourBase is a tool that traces your tests to determine which files they depend on. It uses this information to determine which tests must run, because they are new or have changed dependencies. Tests without changed dependencies are skipped. | ||
|
|
||
| YourBase works with Ruby versions >= 2.3 and RSpec 3+. | ||
|
|
||
| ## Getting Started | ||
| 1. Add `gem 'yourbase-rspec', '~> x.x.x',` to your Gemfile. If you have a `:test` group, add it there. | ||
| 2. `bundle install` from the command line. | ||
| 3. If you are not in a Rails project, you will also need to `require 'yourbase-rspec` your spec_helper (or at the top of the spec file you want to run). | ||
|
|
||
| ## First run | ||
|
|
||
|
|
||
| Run your tests with the same command you typically use. You should see a rocket ship 🚀 at the beginning the RSpec test section. The rocket ship signifies that the YourBase Test Selection is properly loaded and tracing your tests. | ||
|
|
||
| > Note: If you are using Spring and do not see the rocket ship, please run `spring stop` before running your tests. | ||
|
|
||
| The first time you run your tests with `yourbase-rspec` will take the typical amount of time as it records tracing data to map dependencies (a "cold build"). If you run the same test again without changing any code, you should see everything skipped! Subsequent runs will only run examples that are new or depend on changed files. | ||
|
|
||
| After the run finishes, running again will skip all tests. Modifying a dependency will run only tests whose code paths touched the changed code. | ||
|
|
||
| You're YourBased! 🚀 | ||
|
|
||
| ## RSpec Output | ||
|
|
||
| YourBase adds to the RSpec output to give you information while about if examples are being run or skipped, and why. | ||
|
|
||
| The default RSpec output from the `ProgressFormatter` (`.....*..... F........`) will print a `.` in yellow (or the color you have set for `:pending`) when an example is skipped. | ||
|
|
||
| The `DocumentationFormatter` will add the reason an example group is being run in green (or the color your have set for `:success`), and the reason an example group is being skipped in yellow (or the color you have set for `:pending`). | ||
|
|
||
| The summary line will show how show how many examples were skipped! 🚀 | ||
| ```plain | ||
| 1 examples, 0 failures, 1 skipped with YourBase🚀 | ||
| ``` | ||
|
|
||
| ## Parallelization and Sharding | ||
| The `yourbase-rspec` gem supports your workflows for both parallelization (running tests in more than one process at a time on the same machine) and sharding (running tests on more than one machine). Dependency histories are keyed off of the code state (git hash), and all tracing data derived from an identical code state is grouped for future use. | ||
|
|
||
| The environment variables `YOURBASE_ACTIVE_COHORT` and `YOURBASE_COHORT COUNT` control which tests **might** run. Tests that are in the active cohort will be checked against dependency changes, and _tests that are not in the active cohort will be automatically skipped_. | ||
|
|
||
| YourBase cohorts are assigned based on consistent hashing of the example group name AND the number of cohorts. An example group that is in cohort `1` will always be in cohort `1` unless either the `YOURBASE_COHORT_COUNT` OR example group name are changed. | ||
|
|
||
| The `YOURBASE_ACTIVE_COHORT` is 1-indexed (it starts at 1, not 0). If you are sharding with YourBase cohorts, and you set `YOURBASE_COHORT_COUNT=2`, then one of your shard should have `YOURBASE_ACTIVE_COHORT=1` and the other should have `YOURBASE_ACTIVE_COHORT=2`. | ||
|
|
||
| Unless the value of `YOURBASE_COHORT_COUNT` is set and is greater than 1, cohorts are turned off. | ||
|
|
||
| ## Observation Mode | ||
| The yourbase-rspec gem includes an "observation mode" which allows you to test drive the gem without actually skipping any tests. | ||
|
|
||
| In “observation mode” all [command-line specified] examples will be run, but `yourbase-rspec` will monitor if our test selection would have skipped any examples that ultimately failed. At the end, it will print out the names of any example group that would have been incorrectly skipped, or it will confirm that none were. | ||
|
|
||
| To access observation mode, set `YOURBASE_OBSERVATION_MODE=true` in the environment, and run your specs. The documentation formatter isn’t required, but it will print the reasons why YourBase would select to run or skip a given example group. | ||
|
|
||
| `$ YOURBASE_OBSERVATION_MODE=true bundle exec rspec --format documentation` | ||
|
|
||
| Instead of a single rocketship, you’ll see the following at the top of the rspec output for observation mode: | ||
| `:rocket: YourBase test selection is in observation mode. All example groups will be run. :rocket:` | ||
|
|
||
| And then at the bottom, below the RSpec summary, you should see this: | ||
| `🚀 YourBase observation mode: all "skipped" example groups passed successfully! 🚀` | ||
|
|
||
| If you instead see: `🚀 YourBase observation mode: some "skipped" example groups contained failures 🛸` followed by one or more example group names, it means that yourbase-spec would have skipped at least one test that failed when it was actually run. We hope you'll never see this, and we hope that you'll email us if you do, at: <support@yourbase.io> | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should have a section about Spring here, because it's the most common reason they won't see the rocketship.
Right now, they need to run
spring stopfrom the command line to turn it off, but it would be better if we could figure out if manually requiringyourbase-rspecat the top of the spec_helper.rb gets around the incompatibility, so that we could tell them to do that instead of having to stop Spring. Stopping Spring is definitely fine in the CI but is a little kludgier for local dev where Spring does actual useful things, including shorten the spin-up time for "loading the rails environment" for specs.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok. Have this in my notes but wanted to try it out first. Will add it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think for now it is sufficient to say that if you do not see the rocketship but ARE running Spring, that it should be turned off to accelerate the tests.