Skip to content

Testing.md

Latest commit

 

History

History
85 lines (54 loc) · 5.12 KB

Testing.md

File metadata and controls

85 lines (54 loc) · 5.12 KB
Raw

Testing

Philosophy

It's useful to read Kent Dodd’s thinking

Why?

  1. our language/ecosystem has outstanding static analysis. eslint and ts (strictly used!) catch a lot of the problems that UT were historically used for
  2. a lot of our code is sitting on top of big libraries and/or a salesforce org’s APIs
    1. The amount of stubbing to do command-level unit tests makes it kinda silly
    2. stubbed tests won’t catch all the real-world stuff that can go wrong with connections, orgs, and networking. If we live where customers live, we’ll be annoyed and want to handle these

So we have really strong types and eslint rules. You can add local, custom eslint rules to prevent common mistakes (AIs are good at these) so devs get squiggles instead of waiting for test run failures (shift left!)

unit tests

be pragmatic with these. If your unit (function, preferably) has some complex logic and not a lot of dependencies, it's probably a good place to write a unit test. Pure functions are ideal for testing. if you see a LOT of stubs, that's probably not a good unit test

Use TS to make impossible states impossible (the compiler flags errors!) instead of writing unit tests to handle "does my function check for null handling" ?

This repo uses jest for testing and it's probably the best place to start. Docs

coverage

We don't compute test coverage or send it anywhere, nor is there a requirement to do so.

Use ex: npm run test -w packages/salesforcedx-vscode-soql -- --coverage to see it for any package

End to End Testing

e2e is even more crucial in the extensions because much of the extensions API is "run-time only." You could write unit tests to check vscode notifications, but you're mocking all of that and there's no way to assert that you're doing it correctly. It's more useful to have an e2e environment running real vscode and asserting it does what you expect.

These run locally and in Github Actions using Playwright.

playwright

See .claude/skills/playwright-e2e/SKILL.md for guidelines on writing, running, and debugging Playwright tests.

pros

  • fast
  • allows parallel execution
  • records videos
  • works on the web (with vscode-test-web) so a single test runs everywhere
  • nice debug mode (step through the test steps)
  • desktop tests can inspect on-disk telemetry artifacts (O11y spans + AppInsights events) for integration testing

cons

  • you might spend some time screwing around getting the selectors right (recommendation: have your LLM log html snapshots so it can iterate based on the contents)
  • our test using this are much newer, and the code exists only within the extensions, so you'll be copy-pasting a bit if you want to reuse that. On the roadmap to make that more shared

Use Playwright for any new e2e tests.

VSCode DOM

one of the most annoying parts about e2e testing is the "dynamic" nature of the vscode DOM. For example, you might think that everything in the orgBrowser tree is a div. It's actually lazy-loaded, such that anything above or below the visible viewport is removed from the dom, and as you scroll up or down, things are attached/detached

So you go to assert the number of elements, or search the DOM and don't get what you'd expect

When the VSCode UI changes, you might have to update your e2e tests. And you might have to only run tests on newer versions (you still support older versions but don't test against them)

Playwright selector gotchas

  • quick input: use activeQuickInputTextField, activeQuickInputWidget, openCommandPalette, executeCommandWithCommandPalette from @salesforce/playwright-vscode-ext — 1.116+ often fails toBeVisible() on .quick-input-widget while open; follow package attached waits + force fill/click, not visible-widget filtering
  • quick-pick rows: waitForQuickInputFirstOption(page) (ARIA options + Monaco list rows; attached-state waits)
  • accept first quick-pick option: selectFirstQuickInputOption(page, { confirmCommitted?, commitTimeout? }) — unifies flaky .click() / Enter / evaluate click variants; commits via DOM evaluate click with optional Enter fallback when confirmCommitted predicate doesn't pass within commitTimeout
  • quick input text: fill input.input; avoid keyboard.type into ambiguous focus
  • editor content via keyboard: call disableMonacoAutoClosing(page) before page.keyboard.type() to prevent bracket/quote duplication; avoids clipboard (shared global resource + parallel worker races)
  • Problems view assertions: clear Filter Problems input before counting diagnostics
  • expectProblemsCount* helpers now clear Problems filter before assertion; use helpers over ad-hoc row counts

See Also