Skip to content

Conversation

@MaelleBaillet5
Copy link

Summary

Add clickhouse__safe_cast macro that automatically provides default values for ClickHouse types when casting null values. This eliminates the need to specify all non-nullable columns in unit test fixtures.

Fixes #315

Checklist

Delete items not relevant to your PR:

  • Unit and integration tests covering the common scenarios were added
  • A human-readable description of the changes was provided to include in CHANGELOG
  • For significant changes, documentation in https://github.com/ClickHouse/clickhouse-docs was updated with further explanations or tutorials

@CLAassistant
Copy link

CLAassistant commented Oct 22, 2025

CLA assistant check
All committers have signed the CLA.

Copy link
Contributor

@windsurf-bot windsurf-bot bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 To request another review, post a new comment with "/windsurf-review".

-- This macro allows you to set a default value for columns based on their type.

{% macro clickhouse__safe_cast(field, dtype) %}
{%- if field == 'null' -%}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The condition field == 'null' checks if the field is literally the string 'null', not if it's a SQL NULL value. For SQL NULL values, you should use {{ field }} is null instead. The current implementation will only work if 'null' is passed as a string literal.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're correct that the condition field == 'null' checks for the literal string 'null' rather than a SQL NULL value. This is actually intentional for this macro's specific use case.
The safe_cast macro is designed specifically to solve issue #315 - handling non-nullable ClickHouse columns in dbt unit tests.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @MaelleBaillet5 ! Thanks for contributing this. This PR is looking nice and I'd like to merge it ASAP.

The only thing I'm missing now is a proper test for the unit-testing part. As you mention, this is specifically designed to solve the issue in #315, so it would be great to add a test to ensure #315 is not happening again. I think creating an additional test and adding there the unit-test listed in the "Steps to reproduce" section would be enough. Would you add it?

@MaelleBaillet5
Copy link
Author

/windsurf-review

Copy link
Contributor

@windsurf-bot windsurf-bot bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other comments (2)

💡 To request another review, post a new comment with "/windsurf-review".

Comment on lines 6 to 12
{% macro clickhouse__safe_cast(field, dtype) %}
{%- if field == 'null' -%}
CAST(defaultValueOfTypeName('{{ dtype }}') AS {{ dtype }})
{%- else -%}
CAST({{ field }} AS {{ dtype }})
{%- endif -%}
{% endmacro %}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current implementation only handles the literal string 'null', but might miss actual null values passed as variables. Consider enhancing the implementation to also check for actual null values:

Suggested change
{% macro clickhouse__safe_cast(field, dtype) %}
{%- if field == 'null' -%}
CAST(defaultValueOfTypeName('{{ dtype }}') AS {{ dtype }})
{%- else -%}
CAST({{ field }} AS {{ dtype }})
{%- endif -%}
{% endmacro %}
{% macro clickhouse__safe_cast(field, dtype) %}
{%- if field is none or field == 'null' -%}
CAST(defaultValueOfTypeName('{{ dtype }}') AS {{ dtype }})
{%- else -%}
CAST({{ field }} AS {{ dtype }})
{%- endif -%}
{% endmacro %}

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The safe_cast macro is exclusively designed for dbt unit test fixtures, where
all values are passed as string literals from YAML files. When a column is omitted
in a fixture, dbt-core's get_empty_schema_sql macro passes the literal string
'null' (not a SQL NULL value).

Comment on lines 102 to 106
def test_safe_cast_types(self, project):
"""Test that safe_cast preserves the expected data types"""
# Run the model
results = run_dbt(["run", "--select", "safe_cast_test"])
assert len(results) == 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test method should also use the shared fixture to avoid running the model again.

Suggested change
def test_safe_cast_types(self, project):
"""Test that safe_cast preserves the expected data types"""
# Run the model
results = run_dbt(["run", "--select", "safe_cast_test"])
assert len(results) == 1
def test_safe_cast_types(self, project, run_model):
"""Test that safe_cast preserves the expected data types"""
# Get column types from ClickHouse

@MaelleBaillet5
Copy link
Author

  • I modified the macro because I had a bug with a DataTime(“Paris/Europe”) type and added this case to the tests.
  • I added the requested unit test, but is it in the right folder?
  • I saw that the CI is not passing. I don't have this problem locally, but I tried to adapt it. We'll see if it passes in the CI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Null unit test inputs raise error for non-nullable columns

3 participants