Skip to content

Warn instead of throwing exception if raw data stream is closed unexpectedly.#89

Open
aaronweeden wants to merge 3 commits intoubccr:v1.x.yfrom
aaronweeden:error-to-warning
Open

Warn instead of throwing exception if raw data stream is closed unexpectedly.#89
aaronweeden wants to merge 3 commits intoubccr:v1.x.yfrom
aaronweeden:error-to-warning

Conversation

@aaronweeden
Copy link

@aaronweeden aaronweeden commented Aug 6, 2025

Description

When calling get_raw_data(), if the stream is closed before all rows are sent, a RuntimeError is raised, and the partial results are thrown away. This PR changes the behavior to instead log a warning so that partial results can be kept without needing to fetch all the data again.

This PR also adds code to catch requests.exceptions.ChunkedEncodingError which is another error that can occur when a connection is closed early.

Tests performed

With the changes from this PR installed, I tested running the following and confirming it causes the warning:

with dw:
    print(dw.get_raw_data(
        duration=('2025-01-01', '2025-05-01'),
        realm='SUPREMM',
    ))

and changed the end date to 2025-01-01 and confirmed it didn't cause the warning.

Type of change:

  • Refactoring / documentation update (non-breaking change which does not change functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix, feature, or removal that would cause existing functionality to change)
  • Release preparation

Checklist:

  • The milestone is set correctly on the pull request
  • The appropriate labels have been added to the pull request
  • Updates have been made to the xdmod-notebooks repository as necessary, and the notebooks all run successfully

@aaronweeden aaronweeden added enhancement New feature or request bug Something isn't working and removed enhancement New feature or request labels Aug 6, 2025
@aaronweeden aaronweeden added this to the 1.1.1 milestone Aug 6, 2025
@aaronweeden aaronweeden marked this pull request as ready for review August 6, 2025 20:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant