Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/flake8.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ jobs:
runs-on: ubuntu-24.04

steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6

- name: Set up Python 3.12
uses: actions/setup-python@v5
uses: actions/setup-python@v6
with:
python-version: 3.12

Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ jobs:
runs-on: ubuntu-24.04

steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6

- uses: actions/setup-python@v5
- uses: actions/setup-python@v6
with:
python-version: 3.12

Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ jobs:
os: [ windows-latest, macos-latest, ubuntu-24.04 ] # macos-12 is not playing well with Tk
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
- uses: actions/checkout@v6
- uses: actions/setup-python@v6
with:
python-version: 3.12
- name: Install Pip Dependencies
Expand Down
18 changes: 9 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# EnergyPlus Regressions

[![Documentation Status](https://readthedocs.org/projects/energyplusregressiontool/badge/?version=latest)](https://energyplusregressiontool.readthedocs.io/en/latest/?badge=latest)
[![Run Tests](https://github.com/NREL/EnergyPlusRegressionTool/actions/workflows/test.yml/badge.svg)](https://github.com/NREL/EnergyPlusRegressionTool/actions/workflows/test.yml)
[![PyPIRelease](https://github.com/NREL/EnergyPlusRegressionTool/actions/workflows/release.yml/badge.svg)](https://github.com/NREL/EnergyPlusRegressionTool/actions/workflows/release.yml)
[![Flake8](https://github.com/NREL/EnergyPlusRegressionTool/actions/workflows/flake8.yml/badge.svg)](https://github.com/NREL/EnergyPlusRegressionTool/actions/workflows/flake8.yml)
[![Coverage Status](https://coveralls.io/repos/github/NREL/EnergyPlusRegressionTool/badge.svg?branch=master)](https://coveralls.io/github/NREL/EnergyPlusRegressionTool?branch=master)
[![Run Tests](https://github.com/NatLabRockies/EnergyPlusRegressionTool/actions/workflows/test.yml/badge.svg)](https://github.com/NatLabRockies/EnergyPlusRegressionTool/actions/workflows/test.yml)
[![PyPIRelease](https://github.com/NatLabRockies/EnergyPlusRegressionTool/actions/workflows/release.yml/badge.svg)](https://github.com/NatLabRockies/EnergyPlusRegressionTool/actions/workflows/release.yml)
[![Flake8](https://github.com/NatLabRockies/EnergyPlusRegressionTool/actions/workflows/flake8.yml/badge.svg)](https://github.com/NatLabRockies/EnergyPlusRegressionTool/actions/workflows/flake8.yml)
[![Coverage Status](https://coveralls.io/repos/github/NatLabRockies/EnergyPlusRegressionTool/badge.svg)](https://coveralls.io/github/NatLabRockies/EnergyPlusRegressionTool)

## Overview

Expand All @@ -25,16 +25,16 @@ This tool works on all three major platforms: Windows, Mac, and Ubuntu (LTS).
GitHub Actions test on multiple platforms, and it is regularly used on all three as well.

To install the tool, simply `pip` install it into your Python environment (either system or virtual environment)
The project page on PyPi is: https://pypi.org/project/energyplus-regressions/.
The project page on PyPi is: https://pypi.org/project/energyplus-regressions/.

- Download using Pip (`pip install energyplus-regressions`).
- Once installed into the Python install, there will be a binary available to run: `energyplus_regression_runner`.
- Once installed into the Python install, there will be a binary available to run: `energyplus_regression_runner`.

## Development

For setting up a development environment to do _work_ on this tool, the steps are pretty minimal:
- Install Python, if needed
- Clone this repository (`git clone https://github.com/NREL/EnergyPlusRegressionTool`)
- Clone this repository (`git clone https://github.com/NatLabRockies/EnergyPlusRegressionTool`)
- Install dependencies (`pip3 install -r requirements.txt`)

## Documentation
Expand All @@ -45,5 +45,5 @@ This documentation is written using RST with Sphinx, and published on [ReadTheDo
## Testing

Exhaustive unit tests have been added to the "underneath the hood" code, like the functions that calculate diffs and run builds.
The unit tests are run on [Github Actions](https://github.com/NREL/EnergyPlusRegressionTool/actions).
The unit tests are run on [Github Actions](https://github.com/NatLabRockies/EnergyPlusRegressionTool/actions).
The GUI code is not unit tested, but tested routinely on all platforms.
14 changes: 7 additions & 7 deletions docs/intro.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,12 @@ that are included in EnergyPlus development.
History/Credits
---------------

This program was based on a test suite used at the National Renewable
Energy Laboratory (NREL) that was developed by Kyle Benne and Jason
Turner, and consisted of both ruby and python scripts, including a job
manager to allow multiple threads to run concurrently. Although this
script was at the heart of the inspiration for creating this current
tool, very little of the original code exists.
This program was based on a test suite used at the National Laboratory
of the Rockies (NLR) (formerly NREL) that was developed by Kyle Benne
and JasonTurner, and consisted of both ruby and python scripts,
including a jobmanager to allow multiple threads to run concurrently.
Although this script was at the heart of the inspiration for creating
this current tool, very little of the original code exists.

The mathematical comparisons performed on the EnergyPlus output files
(MathDiff and TableDiff) were developed originally by Santosh Philip and
Expand All @@ -40,4 +40,4 @@ Known issues
------------

Known issues for this tool are found on the
`issue list <https://github.com/NREL/EnergyPlusRegressionTool/issues>`_.
`issue list <https://github.com/NatLabRockies/EnergyPlusRegressionTool/issues>`_.
2 changes: 1 addition & 1 deletion energyplus_regressions/__init__.py
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
NAME = 'energyplus_regressions'
VERSION = '2.1.6'
VERSION = '2.1.7'
2 changes: 1 addition & 1 deletion energyplus_regressions/diffs/ci_compare_script.py
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ def main_function(file_name, base_dir, mod_dir, base_sha, mod_sha, _make_public,
elif entry.table_diffs.small_diff_count > 0:
has_small_diffs = True
print_message("Table small diffs.")
if entry.table_diffs.string_diff_count > 1: # There's always one...the time stamp
if entry.table_diffs.string_diff_count > 0:
has_diffs = True
print_message("Table string diffs.")

Expand Down
35 changes: 19 additions & 16 deletions energyplus_regressions/diffs/table_diff.py
Original file line number Diff line number Diff line change
Expand Up @@ -127,11 +127,11 @@ def thresh_abs_rel_diff(abs_thresh: float, rel_thresh: float, x: str, y: str) ->


def prev_sib(entity):
"""get soup.previousSibling ignoring stripping out all blank spaces"""
"""get soup.previous_sibling ignoring stripping out all blank spaces"""
prevs = entity
i = 0
while i == 0:
prevs = prevs.previousSibling
prevs = prevs.previous_sibling
if isinstance(prevs, NavigableString):
utxt = prevs.strip()
if utxt == '':
Expand Down Expand Up @@ -228,7 +228,7 @@ def match_search_rows_to_base_rows(base_rows, search_rows):
def hdict2soup(soup, heading, num, hdict, tdict, horder):
"""Create soup table (including anchor and heading) from header dictionary and error dictionary"""
# Append table anchor
atag = Tag(soup, name='a', attrs=[('name', '%s%s' % ('tablehead', num,))])
atag = Tag(soup, name='a', attrs={'name': '%s%s' % ('tablehead', num,)})
soup.body.append(atag)

# Append table heading
Expand All @@ -237,7 +237,7 @@ def hdict2soup(soup, heading, num, hdict, tdict, horder):
soup.body.append(htag)

# Append table
tabletag = Tag(soup, name='table', attrs=[('border', '1')])
tabletag = Tag(soup, name='table', attrs={'border': '1'})
soup.body.append(tabletag)

# Append column headings
Expand Down Expand Up @@ -280,7 +280,7 @@ def hdict2soup(soup, heading, num, hdict, tdict, horder):
tabletag.append(trtag)
for h in horder:
if h not in hdict:
tdtag = Tag(soup, name='td', attrs=[("class", "big")])
tdtag = Tag(soup, name='td', attrs={"class": "big"})
tdtag.append('ColumnHeadingDifference')
elif h == 'Subcategory':
# Some tables such as the Source Energy End Use Components
Expand All @@ -297,7 +297,7 @@ def hdict2soup(soup, heading, num, hdict, tdict, horder):
val = hdict[h][i]
if isinstance(val, tuple) and len(val) == 2:
diff, which = val
tdtag = Tag(soup, name='td', attrs=[('class', which)])
tdtag = Tag(soup, name='td', attrs={'class': which})
try:
tdtag.append(str(diff))
except Exception: # pragma: no cover
Expand All @@ -312,7 +312,7 @@ def hdict2soup(soup, heading, num, hdict, tdict, horder):
tdtag.append(str(val))
else:
(diff, which) = hdict[h][i]
tdtag = Tag(soup, name='td', attrs=[('class', which)])
tdtag = Tag(soup, name='td', attrs={'class': which})
try:
tdtag.append(str(diff))
except Exception: # pragma: no cover
Expand Down Expand Up @@ -384,33 +384,36 @@ def make_err_table_row(err_soup, tabletag, uheading, count_of_tables, abs_diff_f

if small_diff > 0 or big_diff > 0 or string_diff > 0:
file_name = os.path.basename(abs_diff_file)
atag = Tag(err_soup, name='a', attrs=[('href', '%s#tablehead%s' % (file_name, count_of_tables))])
atag = Tag(err_soup, name='a', attrs={'href': '%s#tablehead%s' % (file_name, count_of_tables)})
atag.append('abs file')
tdtag_abs_link.append(atag)

file_name = os.path.basename(rel_diff_file)
atag = Tag(err_soup, name='a', attrs=[('href', '%s#tablehead%s' % (file_name, count_of_tables))])
atag = Tag(err_soup, name='a', attrs={'href': '%s#tablehead%s' % (file_name, count_of_tables)})
atag.append('rel file')
tdtag_rel_link.append(atag)

tdtag_big_diff = Tag(err_soup, name='td', attrs=[('class', 'big')] if big_diff > 0 else [])
tdtag_big_diff = Tag(err_soup, name='td', attrs={'class': 'big'} if big_diff > 0 else None)
trtag.append(tdtag_big_diff)
tdtag_big_diff.append(str(big_diff))

tdtag_small_diff = Tag(err_soup, name='td', attrs=[('class', 'small')] if small_diff > 0 else [])
tdtag_small_diff = Tag(err_soup, name='td', attrs={'class': 'small'} if small_diff > 0 else None)
trtag.append(tdtag_small_diff)
tdtag_small_diff.append(str(small_diff))

tdtag_equal = Tag(err_soup, name='td', attrs=[])
tdtag_equal = Tag(err_soup, name='td')
trtag.append(tdtag_equal)
tdtag_equal.append(str(equal))

tdtag_string_diff = Tag(err_soup, name='td', attrs=[('class', 'stringdiff')] if string_diff > 0 else [])
tdtag_string_diff = Tag(err_soup, name='td', attrs={'class': 'stringdiff'} if string_diff > 0 else None)
trtag.append(tdtag_string_diff)
tdtag_string_diff.append(str(string_diff))

tdtag_table_size_error = Tag(err_soup, name='td', attrs=[
('class', 'table_size_error')] if size_error > 0 or not_in_1 > 0 or not_in_2 > 0 else [])
tdtag_table_size_error = Tag(
err_soup,
name='td',
attrs={'class': 'table_size_error'} if size_error > 0 or not_in_1 > 0 or not_in_2 > 0 else None
)
trtag.append(tdtag_table_size_error)
tdtag_table_size_error.append(
'size mismatch' if size_error > 0 else 'not in 1' if not_in_1 > 0 else 'not in 2' if not_in_2 > 0 else '')
Expand Down Expand Up @@ -458,7 +461,7 @@ def table_diff(
features='html.parser')

# Make error table
tabletag = Tag(err_soup, name='table', attrs=[('border', '1')])
tabletag = Tag(err_soup, name='table', attrs={'border': '1'})
err_soup.body.append(tabletag)

# Make error table headings
Expand Down
6 changes: 3 additions & 3 deletions energyplus_regressions/structures.py
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ def __init__(self, args_from_table_diff):
self.big_diff_count = args_from_table_diff[2]
self.small_diff_count = args_from_table_diff[3]
self.equal_count = args_from_table_diff[4]
self.string_diff_count = args_from_table_diff[5] # There's always one...the time stamp
self.string_diff_count = args_from_table_diff[5]
self.size_err_count = args_from_table_diff[6]
self.not_in_1_count = args_from_table_diff[7]
self.not_in_2_count = args_from_table_diff[8]
Expand All @@ -159,7 +159,7 @@ def to_dict(self):
response['big_diff_count'] = self.big_diff_count
response['small_diff_count'] = self.small_diff_count
response['equal_count'] = self.equal_count
response['string_diff_count'] = self.string_diff_count # There's always one...the time stamp
response['string_diff_count'] = self.string_diff_count
response['size_err_count'] = self.size_err_count
response['not_in_1_count'] = self.not_in_1_count
response['not_in_2_count'] = self.not_in_2_count
Expand Down Expand Up @@ -464,7 +464,7 @@ def add_test_entry(self, this_entry):
self.big_table_diffs.add_to_data(this_entry.basename, "table")
elif this_entry.table_diffs.small_diff_count > 0:
self.small_table_diffs.add_to_data(this_entry.basename, "table")
if this_entry.table_diffs.string_diff_count > 1: # There's always one...the time stamp
if this_entry.table_diffs.string_diff_count > 0:
self.big_table_diffs.add_to_data(this_entry.basename, "table")

# check the textual diffs
Expand Down
27 changes: 27 additions & 0 deletions energyplus_regressions/tests/diffs/test_ci_compare_script.py
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,33 @@ def test_main_function(self):
test_mode=True
)
self.assertIn('Table small diffs', out.getvalue().strip())

def test_main_function_reports_single_table_string_diff(self):
end_string = 'EnergyPlus Completed Successfully-- 0 Warning; 0 Severe Errors; Elapsed Time=00hr 00min 3.06sec'
self._write_files_to_both_folders('eplusout.end', end_string, end_string)
shutil.copy(
os.path.join(self.tbl_resource_dir, 'eplustbl_has_string_diff_base.htm'),
os.path.join(self.temp_base_dir, 'eplustbl.htm')
)
shutil.copy(
os.path.join(self.tbl_resource_dir, 'eplustbl_has_string_diff_mod.htm'),
os.path.join(self.temp_mod_dir, 'eplustbl.htm')
)
with captured_output() as (out, err):
main_function(
file_name='HVACTemplate-5ZoneFanCoil',
base_dir=self.temp_base_dir,
mod_dir=self.temp_mod_dir,
base_sha='base123',
mod_sha='mod456',
_make_public=True,
device_id='some_device_id',
test_mode=True
)
output = out.getvalue().strip()
self.assertIn('Table string diffs', output)
self.assertNotIn('Table big diffs', output)
self.assertNotIn('Table small diffs', output)
# now test one where every single file has big diffs
shutil.copy(
os.path.join(self.csv_resource_dir, 'eplusout.csv'),
Expand Down
18 changes: 18 additions & 0 deletions energyplus_regressions/tests/test_structures.py
Original file line number Diff line number Diff line change
Expand Up @@ -258,3 +258,21 @@ def test_to_json_object_response(self):
self.assertIn('runs', obj)
self.assertIn('diffs', obj)
self.assertIn('results_by_file', obj)

def test_to_json_summary_reports_single_table_string_diff_as_big_table(self):
c = CompletedStructure(
Path('/a/source/dir'), Path('/a/build/dir'),
Path('/b/source/dir'), Path('/b/build/dir'),
Path('/r/dir1'), Path('/r/dir2'),
datetime.now()
)
t = TestEntry('filename', 'weather')
t.add_summary_result(EndErrSummary(EndErrSummary.STATUS_SUCCESS, 1, EndErrSummary.STATUS_SUCCESS, 1))
t.add_table_differences(TableDifferences(['', 1, 0, 0, 2, 1, 0, 0, 0]))
c.add_test_entry(t)

obj = c.to_json_summary()

self.assertEqual(['filename'], obj['diffs']['big_table'])
self.assertEqual([], obj['diffs']['small_table'])
self.assertEqual(1, obj['results_by_file'][0]['table_diffs']['string_diff_count'])
4 changes: 2 additions & 2 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# the core dependencies actually needed
pypubsub
beautifulsoup4==4.12.3
beautifulsoup4==4.14.3

# for running tests
coverage
Expand All @@ -19,4 +19,4 @@ docutils # pinning this because it breaks at 0.18
wheel

# for polishing up the Pip install
PLAN-Tools>=0.5
PLAN-Tools>=0.7
9 changes: 4 additions & 5 deletions setup.py
Original file line number Diff line number Diff line change
@@ -1,16 +1,15 @@
import codecs
import os
from platform import system
from setuptools import setup

from energyplus_regressions import NAME, VERSION

this_dir = os.path.abspath(os.path.dirname(__file__))
with codecs.open(os.path.join(this_dir, 'README.md'), encoding='utf-8') as i_file:
with open(os.path.join(this_dir, 'README.md'), encoding='utf-8') as i_file:
long_description = i_file.read()


install_requires = ['PyPubSub', 'beautifulsoup4==4.12.3', 'PLAN-Tools>=0.5']
install_requires = ['PyPubSub', 'beautifulsoup4==4.14.3', 'PLAN-Tools>=0.7']
if system() == 'Windows':
install_requires.append('pypiwin32')

Expand All @@ -22,9 +21,9 @@
package_data={
'energyplus_regressions': ['diffs/math_diff.config', 'icons/icon.png', 'icons/icon.ico', 'icons/icon.icns']
},
url='https://github.com/NREL/EnergyPlusRegressionTool',
url='https://github.com/NatLabRockies/EnergyPlusRegressionTool',
license='ModifiedBSD',
author='Edwin Lee, for NREL, for United States Department of Energy',
author='Edwin Lee, for NLR, for United States Department of Energy',
description='A Python 3 library for evaluating regressions between EnergyPlus builds.',
long_description=long_description,
long_description_content_type='text/markdown',
Expand Down