Skip to content

Add support for importing Python helper functions into Jinja2 templates - #399

Merged
thomaspatzke merged 8 commits into
mainfrom
copilot/add-import-support-python-scripts
Nov 2, 2025
Merged

Add support for importing Python helper functions into Jinja2 templates#399
thomaspatzke merged 8 commits into
mainfrom
copilot/add-import-support-python-scripts

Conversation

Copilot AI commented Oct 26, 2025

Copy link
Copy Markdown
Contributor
  • Explore repository structure and understand the codebase
  • Run existing tests to verify current state
  • Design solution for importing helper Python scripts into Jinja2 templates
  • Implement changes to TemplateBase to support custom functions via vars parameter
  • Add comprehensive tests for the new functionality
  • Update docstrings for QueryTemplateTransformation and TemplateFinalizer
  • Verify all tests pass (29 tests passing)
  • Run full test suite
  • Request code review
  • Address review feedback:
    • Removed vendor-specific Splunk RBA test
    • Removed redundant risk mapping test
    • Deleted splunk_rba_helpers.py test file
    • Simplified template_vars.py by removing format_price and get_risk_mapping functions
    • Updated all tests to use parse_json for simpler test cases

Summary

Successfully implemented support for importing Python helper scripts into Jinja2 templates. The implementation:

  • Adds a new vars parameter to TemplateBase
  • Loads functions/variables from a vars dictionary in the Python file
  • Makes them available as Jinja2 template globals
  • Works with both QueryTemplateTransformation and TemplateFinalizer
  • Supports YAML pipeline configurations
  • Includes comprehensive error handling
  • Fully tested with simplified, vendor-neutral examples
Original prompt

This section details on the original issue you should resolve

<issue_title>Add support for importing helper Python scripts into post processing</issue_title>
<issue_description>Jinja is a limited template language and I'm quickly approaching the edges of what is possible or what is reasonable code to write.

I am currently writing a template to support Risk Based Alerting in Splunk ES8 and I am assigning risk_object and risk_category based on the logsource of the rule. I would also like to be able to access these mappings in a linter to validate that the rule I am transforming has the correct fields present at PR time so my hope is to have a json file similar to logsources.json that i can reference.

Unfortunately using a json file in Jinja2 to make a scalable solution here is possible but ugly. Jinja only wants to accept imported files as string objects and doesn't have native functionality for parsing things like a json object... so I do some ugly splitting and parsing that would otherwise be trivial in Python.

If I had access to some python files from the jinja template I could do something like this example from SO:

@app.context_processor
def utility_processor():
    def format_price(amount, currency=u'€'):
        return u'{0:.2f}{1}'.format(amount, currency)
    return dict(format_price=format_price)

The above when used like so:

{{ format_price(0.33) }}

Which outputs the input price with the currency symbol

So in my case I could use this to parse the JSON and return the exact fields I want into the jinja template if I were able to define the python file I want to include in the postprocessing template.
</issue_description>

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 3 commits October 26, 2025 23:52
Co-authored-by: thomaspatzke <1845601+thomaspatzke@users.noreply.github.com>
Co-authored-by: thomaspatzke <1845601+thomaspatzke@users.noreply.github.com>
Co-authored-by: thomaspatzke <1845601+thomaspatzke@users.noreply.github.com>
Copilot AI changed the title [WIP] Add support for importing helper Python scripts in post processing Add support for importing Python helper functions into Jinja2 templates Oct 27, 2025
Copilot AI requested a review from thomaspatzke October 27, 2025 00:00
Comment thread tests/test_finalization_tranformations.py Outdated
Comment thread tests/test_postprocessing_transformations.py Outdated
Co-authored-by: thomaspatzke <1845601+thomaspatzke@users.noreply.github.com>
Copilot AI requested a review from thomaspatzke October 29, 2025 23:15
@thomaspatzke
thomaspatzke marked this pull request as ready for review October 29, 2025 23:19
Comment thread tests/files/template_vars.py Outdated
Comment thread tests/files/template_vars.py Outdated
…mapping

Co-authored-by: thomaspatzke <1845601+thomaspatzke@users.noreply.github.com>
@thomaspatzke
thomaspatzke merged commit c9b56ae into main Nov 2, 2025
20 checks passed
@thomaspatzke
thomaspatzke deleted the copilot/add-import-support-python-scripts branch November 2, 2025 00:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for importing helper Python scripts into post processing

2 participants