Skip to content

Add raw data regression tests.#354

Merged
aaronweeden merged 2 commits intoubccr:xdmod11.0from
aaronweeden:add-raw-data-regression-tests
Dec 21, 2023
Merged

Add raw data regression tests.#354
aaronweeden merged 2 commits intoubccr:xdmod11.0from
aaronweeden:add-raw-data-regression-tests

Conversation

@aaronweeden
Copy link
Copy Markdown
Contributor

@aaronweeden aaronweeden commented Oct 13, 2023

Description

This PR adds regression tests of the warehouse/raw-data REST endpoint for the SUPREMM realm.

This PR is based on #343. The diff between them can be viewed here.

Tests performed

In a Docker container running tools-ext-01.ccr.xdmod.org/xdmod-job_performance-10.5.0:rockylinux8-0.1 with the XDMOD_TEST_MODE environment variable set to fresh_install, and then in another Docker container with the same image with it set to upgrade:

  1. Run the following commands:
    export COMPOSER_ALLOW_SUPERUSER=1
    export XDMOD_REALMS='jobs,storage,cloud,supremm,jobefficiency'
    export XDMOD_IS_CORE=yes
    export XDMOD_INSTALL_DIR=/xdmod
    export XDMOD_SRC_DIR=./../xdmod
    git clone https://github.com/aaronweeden/xdmod-supremm -b php-error-log /xdmod-supremm
    cd /xdmod-supremm
    dnf module -y reset nodejs
    dnf module -y install nodejs:16
    git clone https://github.com/ubccr/xdmod $XDMOD_SRC_DIR
    ln -s `pwd` $XDMOD_SRC_DIR/open_xdmod/modules/supremm
    ./tests/ci/scripts/install_mongo.sh
    composer install --no-progress
    composer config -g cache-files-ttl 315360000
    composer install -d $XDMOD_SRC_DIR --no-progress
    mkdir -p ~/phpunit
    mkdir -p /tmp/screenshots
    export XDMOD_SOURCE_DIR=/xdmod
    export SHIPPABLE_BUILD_DIR=/xdmod-supremm
    pushd $XDMOD_SRC_DIR
    ~/bin/buildrpm xdmod supremm
    popd
    ./tests/integration/scripts/bootstrap.sh
    ./tests/integration/scripts/validate.sh
    composer install -d $XDMOD_SRC_DIR --no-progress
    composer install --no-progress
    ./tests/ci/scripts/qa-test-setup.sh
    export XDMOD_INSTALL_DIR=/usr/share/xdmod
    ./tests/ci/scripts/post-install-test.sh
    sed -i 's/\$phpunit /$phpunit --debug /' ./tests/component/runtests.sh
    ./tests/component/runtests.sh > /component-before.txt
    composer install -d $XDMOD_SRC_DIR --no-progress
    sed -i 's/\$phpunit /$phpunit --debug /' ./tests/integration/runtests.sh
    ./tests/integration/runtests.sh > /integration-before.txt
    sed -i 's/\$phpunit /$phpunit --debug /' ./tests/regression/runtests.sh
    ./tests/regression/runtests.sh > /regression-before.txt
    ./tests/artifacts/update-artifacts.sh
    cd tests/unit/
    ../../vendor/bin/phpunit --debug . > /unit-before.txt
    cd /xdmod-supremm
    git clone https://github.com/aaronweeden/xdmod-supremm -b add-raw-data-regression-tests /xdmod-supremm-new
    unalias rm
    rm -r /xdmod-supremm/tests
    cp -r /xdmod-supremm{-new,}/tests
    sed -i 's/\$phpunit /$phpunit --debug /' ./tests/component/runtests.sh
    ./tests/component/runtests.sh > /component-after.txt
    composer install -d $XDMOD_SRC_DIR --no-progress
    sed -i 's/\$phpunit /$phpunit --debug /' ./tests/integration/runtests.sh
    ./tests/integration/runtests.sh > /integration-after.txt
    sed -i 's/\$phpunit /$phpunit --debug /' ./tests/regression/runtests.sh
    ./tests/regression/runtests.sh > /regression-after.txt
    ./tests/artifacts/update-artifacts.sh
    cd tests/unit/
    ../../vendor/bin/phpunit --debug . > /unit-after.txt
    cd /xdmod-supremm
    
  2. Compare /unit-{before,after}.txt and make sure the only difference is the amount of time the tests ran.
  3. Compare /component-{before,after}.txt and make sure the only difference is the amount of time the tests ran.
  4. Compare /regression-{before,after}.txt and confirm two new tests were run for WarehouseRawDataSupremmTest::testGetRawData, and make sure otherwise the only differences are randomized parameters, amount of time the tests ran, and possibly transient "other notices" related to small differences in column values.
  5. Compare /integration-{before,after}.txt and make sure the only difference is the amount of time the tests ran.
  6. Make sure this command runs successfully: REG_TEST_ALL=1 /xdmod-supremm/tests/regression/runtests.sh

I also made sure CircleCI uploads the new test results file /root/phpunit/xdmod-supremm-regression-warehouse.xml.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project as found in the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@aaronweeden aaronweeden added the qa label Oct 13, 2023
@aaronweeden aaronweeden added this to the 11.0.0 milestone Oct 13, 2023
@aaronweeden aaronweeden force-pushed the add-raw-data-regression-tests branch 3 times, most recently from cdaa69b to 7f9e27a Compare December 4, 2023 16:35
@aaronweeden aaronweeden force-pushed the add-raw-data-regression-tests branch 9 times, most recently from 89076c4 to d362f78 Compare December 7, 2023 18:44
@aaronweeden aaronweeden marked this pull request as ready for review December 7, 2023 19:59
@aaronweeden aaronweeden force-pushed the add-raw-data-regression-tests branch 2 times, most recently from 9e9421a to 936864e Compare December 12, 2023 19:31
@aaronweeden aaronweeden force-pushed the add-raw-data-regression-tests branch 6 times, most recently from d692ba9 to e28930a Compare December 15, 2023 20:24
@aaronweeden aaronweeden force-pushed the add-raw-data-regression-tests branch from e28930a to 7880f4a Compare December 18, 2023 14:13
@aaronweeden aaronweeden merged commit 3f00d49 into ubccr:xdmod11.0 Dec 21, 2023
@aaronweeden aaronweeden deleted the add-raw-data-regression-tests branch December 21, 2023 19:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants