Skip to content

E2E-TESTS.md

Latest commit

 

History

History
296 lines (190 loc) · 10.5 KB

E2E-TESTS.md

File metadata and controls

296 lines (190 loc) · 10.5 KB
Raw

WooCommerce PayPal Payments Plugin Tests

Depends on @inpsyde/playwright-utils package.

Detailed information about current test project can be found in docs.

Table of Content

Local installation

  1. Clone repository locally:

    git clone https://github.com/woocommerce/woocommerce-paypal-payments.git

Project configuration (Devs)

  1. Create and configure .env file:

  2. Set general variables following these steps.

  3. Set PayPal API keys and test credentials. See .env.example. The .env content with actual test users' credentials is stored in 1Password.

Project configuration (QA team)

  1. Create and configure .env file:

    1.1 Set general variables following these steps.

    1.2 Set PayPal API keys and test credentials. See .env.example. For Google Pay transaction tests, make sure to also set GOOGLE_PAY_EMAIL and GOOGLE_PAY_PASSWORD. The .env content with actual test users' credentials is stored in 1Password.

  2. Configure playwright.config.ts of the project following these steps.

  3. Configure reporting to Xray in Jira following these steps.

  4. To avoid conflicts make sure any other payment plugins are deleted.

  5. Additional website and WooCommerce configuration is done automatically via setup-woocommerce dependency project (see /tests/_setup/woocommerce.setup.ts).

Installation of playwright-utils for local development

  1. Add yarn to devDependencies in package.json:

    "devDependencies": {
	"yarn": "^1.22.21",

  2. Execute steps for local development described here.

    General one-line tsc-watch command is:

    npm run e2e:setup:tests && cd ./playwright-utils && yarn devLocal

  3. After development is finished cleanup package.json and regenerate the package-lock.json.

Running tests

  • npm run tests:all -- Runs all the tests
  • npm run tests:critical -- Runs all the critical tests
  • npm run tests:onboarding -- Runs all the onboarding tests

Additional options to run tests from command line

  • Add scripts to package.json of the project (eligible for Windows, not tested on other OS):

     "scripts": {
 	"test:smoke":  "npx playwright test --grep \"@Smoke\"",
 	"test:critical": "npx playwright test --grep \"@Critical\"",
 	"test:ui": "npx playwright test --grep \"UI\"",
 	"test:functional": "npx playwright test --grep \"Functional\"",
 	"test:all": "npm run test:ui & npm run test:functional"
 },

    Run script with the following command:

     npm run test:critical

  • Run several tests by test ID (on Windows, Powershell):

     npx playwright test --grep --% "PCP-123^|PCP-124^|PCP-125"

    It may be required additionally to specify the project (if tests relate to more then one project):

     npx playwright test --project "project-name" --grep --% "PCP-123^|PCP-124^|PCP-125"

Autotest Execution workflow

  1. Create Test Execution ticket in Jira, named after the tested plugin version, for example "Test Execution for v2.3.4-rc1, PHP8.1".

    Note: For autotest execution there's no need to manually add tests cases to the execution - the executed tests will be imported automatically after execution.

  2. Link release ticket (via tests: PCP-234) to test execution ticket.

  3. Set Test Execution ticket status In progress.

  4. In .env file add/update test execution ticket key (XRAY_TEST_EXEC_KEY='PCP-234').

  5. Download tested plugin .zip package (usually attached to release ticket) and add it to /tests/qa/resources/files. You may need to remove version number from the file name (expected name: woocommerce-paypal-payments.zip).

  6. Optional: delete previous version of tested plugin from the website if you don't execute plugin foundation tests.

  7. Start autotest execution from command line for the defined scope of tests (see this section). You should see Test execution Jira key: PCP-234 in the terminal.

  8. When finished test results will be imported to the specified test execution ticket in Jira.

  9. Analyze failed tests (if any). Restart execution for failed tests, possibly in debug mode (for Windows):

    npx playwright test --grep --% "PCP-123^|PCP-124^|PCP-125" --debug

    Note: command for restarting failed/skipped tests is posted to the terminal after the execution.

  10. Report bugs (if any) and attach them to the test-runs of failed tests (Click "Create defect" or "Add defect" on test execution screen).

  11. If needed fix failing tests in a new branch, create a PR and assign it for review.

  12. Set Test execution ticket status to Done.

Coding standards

Before committing changes run following command:

npm run e2e:lint:js:fix

Reset Kinsta env

Note: the staging env on Kinsta should be created and the script to reset env provided by devops (if not - create a ticket on SDO board).

Find SSH data in Kinsta dashboard for your tested env. Replace data in the following one-line command and run it in the terminal to reset the env:

ssh <your-ssh-username>@<your-ssh-host> -p <your-ssh-port> '${HOME}/bin/reset-wp.sh --wp-version=6.9 --wp-type=single && exit'

Automated env setup scripts for migration

Preconditions

Local usage of automated env setup scripts assumes that the following steps are fulfilled:

  1. Clone PayPal repo to your local machine:

    git clone https://github.com/woocommerce/woocommerce-paypal-payments.git

    Note: temporary, for migration testing change branch to dev/qa/migration-tests: git checkout dev/qa/migration-tests.

  2. Copy following packages into /tests/qa/resources/files:

    • Configured PayPal plugin package (e.g. v3.4.1) named as woocommerce-paypal-payments.zip

    • Optional: Paypal plugin version to upgrade/downgrade to (e.g. v3.0.0 or v4.0.0) as woocommerce-paypal-payments-update.zip.

    • WooCommerce Subscriptions package named as woocommerce-subscriptions.zip

  3. Install Node dependencies and Playwright:

    npm run e2e:setup:tests

  4. In the root create a .env file and copy-paste content from PCP .env vault of 1Password replacing all the data for your test env. Alternatively use .env.example.

  5. Run the scripts described below.

Script naming convention

Scripts follow a naming pattern:

Prefix Meaning Example
e2e:env:reset Reset only env npm run e2e:env:reset
e2e:env:reset:wc Reset only env + WooCommerce npm run e2e:env:reset:wc
e2e:env:reset:pcp:* Reset env + WooCommerce + PCP setup (single run) npm run e2e:env:reset:pcp:usa
e2e:env:setup:* PCP/config setup only, no reset npm run e2e:env:setup:pcp:usa

Reset env and WooCommerce setup

  • Resets the env
  • Configures website permalinks (%postname%)
  • Installs plugins and themes:
    • WooCommerce
    • Storefront theme
    • Additional plugins (Disable Nonce, WC Subscriptions, etc.)
  • Configures WooCommerce for default country (USA):
    • API keys
    • Country/currency: USA/USD
    • Taxes: included, 10% rate
    • Shipping: flat rate/10 USD and free
    • Emails: disabled
  • Creates test entities:
    • Classic cart and checkout
    • Tested products
    • Coupons
    • Registered US customer
# Reset env only
npm run env:reset

# Reset env, WooCommerce
npm run env:reset:wc

WooCommerce setup only (no SSH reset):

npm run e2e:env:setup:wc

Setup PCP for specific country

  • Installs PCP plugin (woocommerce-paypal-payments.zip).
  • Connects merchant from specified country.

For USA — PCP + connected US merchant + ACDC enabled (PayPal and other PMs disabled).

With reset Without reset
npm run e2e:env:reset:pcp:usa npm run e2e:env:setup:pcp:usa

For Germany & PUI (disabled by default)

With reset Without reset
npm run e2e:env:reset:pcp:germany npm run e2e:env:setup:pcp:germany

For Mexico & OXXO (disabled by default)

With reset Without reset
npm run e2e:env:reset:pcp:mexico npm run e2e:env:setup:pcp:mexico

Upgrade PCP (without env reload)

Installs woocommerce-paypal-payments-update.zip over the existing PCP installation.

npm run e2e:env:setup:pcp:update

Switch checkout layout (without env reload)

Enable classic cart/checkout

npm run e2e:env:setup:checkout:classic

Enable block cart/checkout

npm run e2e:env:setup:checkout:block

Switch tax configuration (without env reload)

Tax included in prices

npm run e2e:env:setup:tax:inc

Tax excluded from prices

npm run e2e:env:setup:tax:exc