Skip to content

Conversation

@mashcroft3
Copy link
Contributor

Summary

This PR introduces a bug fix for plugin database lookups, a performance and reliability refactor for Python package enumeration, and an enhancement to the pytest configuration.


Changes

  • pytest.ini:

    • Added norecursedirs to the [pytest] configuration. This prevents the test collector from scanning virtual environments (*/venv/*, */site-packages/*) and temporary directories (test/tmp), improving performance and preventing the execution of third-party tests.
  • transformerlab/db/db.py:

    • BUG FIX: The save_plugin function was modified to query for plugins using select(Plugin).where(Plugin.name == name) instead of session.get(Plugin, name).
    • Reason: The previous implementation incorrectly assumed the name field was the primary key. This change corrects the lookup logic to query against the unique name column, resolving an issue where existing plugins could not be found and updated.
  • transformerlab/routers/serverinfo.py:

    • REFACTOR: The get_python_library_versions endpoint was updated to use importlib.metadata to enumerate installed packages.
    • Reason: This removes the need for a subprocess call to pip, which improves endpoint performance and security. The previous pip method is retained as a fallback for increased resilience.

@codecov
Copy link

codecov bot commented Sep 16, 2025

Codecov Report

❌ Patch coverage is 46.15385% with 14 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
transformerlab/routers/serverinfo.py 45.45% 10 Missing and 2 partials ⚠️
transformerlab/db/db.py 50.00% 1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

Copy link
Contributor

@deep1401 deep1401 Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work with the correct python executable if we have multiple of them locally?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes this enumerates the distributions visible to the interpreter running the server (the same as sys.executable)!

Copy link
Contributor

@deep1401 deep1401 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All changes work for me but I think the plugin one might be unnecessary as that function doesn't get used

return sqlalchemy_to_dict(plugin) if plugin else None


async def save_plugin(name: str, type: str):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a leftover function, we dont use this one anymore or the associated parent function, I think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it gets used in plugin_download but maybe that endpoint is no longer used? mainly there was a test associated with this function that was failing locally

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The plugin_download function is deprecated now. Do we also get rid of the test then?

return r


@router.get("/python_libraries")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried this change and it worked for me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants