cicd pipeline for testing code changes #274
Description
The runner needs a cicd pipeline for testing new pushes/PRs against coding standards, with a container for database builds that mimic a hosting environment, and automation that triggers prepare.php and test.php scripts to return any errors and other output- preferably with a Hosting Team reporting bot relaying data.
Tech specs
Keeping in line with Core standards - this should be built out using GitHub workflows/ actions runner image and Docker containerization for database testing, with phpcs, wpcs coding standard tests and automated triggering of runner scripts. Ideally a Hosting Team reporting bot would post results on PRs. The process of implementing this can be done iteratively.
First Runner code testing can be put in place, then automation triggering environment tests to simulate what happens in a hosting environment, and lastly a bot to report back any errors in PR comments. Then rinse and repeat for the reporter.
Which repos?
PHPunit Test Runner - to start, and as the highest priority
PHPunit Test Reporter - next priority although there is not much development happening here.
Examples:
https://github.com/WordPress/wordpress-develop
Pipeline References:
Docker-compose.yml
Use Cases
Runner Issue #212 : Code here switches to using short array syntax ( $var = [] vs. $var = array() ). The former is eprohibited in wp coding standards. Tests would catch that and prompt a fix.
Runner issue #271: Incorrect syntax was not caught by humans and resulted in a parse error on host test environments