fix: deletion fixes, codebase detection perf improvements and misc refactoring - #675
Conversation
…factoring - Capture comprehensive pre-deletion graph state snapshot - Validate API deletion stats against actual pre-deletion state - Ensure complete repository deletion by checking for any remaining nodes/relationships - Remove unused `rel_count_by_type` function - Simplify node count checks for integration tests
|
Connected to Huly®: UNOPL-812 |
|
Claude finished @JayGhiya's task —— View job PR Review: Deletion Fixes and Performance ImprovementsI've completed my comprehensive review of this substantial refactoring PR. Here's my detailed analysis: Review Progress
✅ Strengths and Improvements1. Enhanced Deletion Logic (
|
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
PR Code Suggestions ✨ |
Adds a step to install the ripgrep tool in the Python build workflow. Ripgrep is a fast and efficient file search tool that can be useful for various development tasks.
fix: Install ripgrep in Python build workflow
|
Claude finished @JayGhiya's task —— View job Claude finished @JayGhiya's task —— View job PR Review: Deletion Fixes and Performance ImprovementsI've completed my comprehensive review of this substantial refactoring PR. Here's my detailed analysis: Review Progress
✅ Strengths and Improvements1. Enhanced Deletion Logic (
|
CI Feedback 🧐A test triggered by this PR failed. Here is an AI-generated analysis of the failure:
|
…factor fix: deletion fixes, codebase detection perf improvements and misc refactoring
User description
rel_count_by_typefunctionPR Type
Bug fix, Enhancement, Tests
Description
• Performance improvements: Replaced
AsyncDetectorWrapperandPythonCodebaseDetectorwith newPythonRipgrepDetectorfor faster codebase detection using ripgrep• Enhanced deletion validation: Added comprehensive pre-deletion state capture and validation of API deletion statistics against actual graph state
• Graph deletion reliability: Fixed deletion logic to use prefix matching instead of relationship traversal, preventing issues with missing relationships
• Code refactoring: Consolidated base model imports to use
unoplat_code_confluence_commonspackage, removing duplicate model definitions• Simplified package detection: Replaced weight-based package manager detection with ordered evaluation (first match wins) approach
• Bug fixes: Fixed node creation return values, package discovery infinite loops, and various import path issues
• Test improvements: Enhanced integration tests with better deletion verification and updated test data to reflect structural changes
• Configuration updates: Added
contains_absencefield support and updated dependency versionsDiagram Walkthrough
File Walkthrough
6 files
main.py
Replace codebase detector with ripgrep-based implementationunoplat-code-confluence-ingestion/code-confluence-flow-bridge/src/code_confluence_flow_bridge/main.py
• Replaced
AsyncDetectorWrapperandPythonCodebaseDetectorwithPythonRipgrepDetectorfor improved performance• Added initialization
of shared
PythonRipgrepDetectorinstance in application lifespan•
Simplified SSE event generation by removing queue-based progress
tracking
• Updated codebase detection endpoint to use new detector
directly
python_ripgrep_detector.py
Add new ripgrep-based Python codebase detectorunoplat-code-confluence-ingestion/code-confluence-flow-bridge/src/code_confluence_flow_bridge/parser/package_manager/detectors/python_ripgrep_detector.py
• New fast Python package manager detector using ripgrep for file
discovery
• Implements async detection with ordered evaluation (first
match wins)
• Maintains same interface as
PythonCodebaseDetectorfordrop-in replacement
• Uses breadth-first processing to prevent nested
directory conflicts
graph_assertions.py
Enhance graph deletion validation and remove unused functionsunoplat-code-confluence-ingestion/code-confluence-flow-bridge/tests/utils/graph_assertions.py
• Removed unused
rel_count_by_typefunction• Added comprehensive
pre-deletion state capture functionality
• Added validation of API
deletion statistics against actual graph state
• Enhanced repository
deletion verification with better error reporting
ripgrep_utils.py
Add ripgrep utilities for fast file discovery and searchingunoplat-code-confluence-ingestion/code-confluence-flow-bridge/src/code_confluence_flow_bridge/parser/package_manager/detectors/ripgrep_utils.py
• New utility module providing async wrappers for ripgrep subprocess
calls
• Implements fast file discovery using glob patterns
• Provides
content searching and Python package root detection
• All functions
use asyncio.subprocess for non-blocking I/O
ordered_detection.py
Add ordered package manager detection without weightsunoplat-code-confluence-ingestion/code-confluence-flow-bridge/src/code_confluence_flow_bridge/parser/package_manager/detectors/ordered_detection.py
• New module implementing simple ordered detection logic
• Evaluates
package managers in sequence with first match wins approach
• Supports
file-based and glob-based signatures with content matching
• Removes
complex weight calculations and tie-breaking logic
settings.py
Add contains_absence field to Signature configuration modelunoplat-code-confluence-ingestion/code-confluence-flow-bridge/src/code_confluence_flow_bridge/models/configuration/settings.py
• Added
contains_absencefield toSignaturemodel for specifyingsubstrings that must NOT appear in files
11 files
test_delete_repository.py
Improve deletion test validation with comprehensive state checkingunoplat-code-confluence-ingestion/code-confluence-flow-bridge/tests/integration/test_delete_repository.py
• Added pre-deletion state snapshot capture for validation
• Enhanced
deletion verification with API statistics validation
• Replaced manual
verification with comprehensive helper functions
• Added sanity checks
to ensure content exists before deletion
test_generic_codebase_parser.py
Update imports and test expectations for codebase root packageunoplat-code-confluence-ingestion/code-confluence-flow-bridge/tests/parser/test_generic_codebase_parser.py
• Updated import to use
StructuralSignaturefromunoplat_code_confluence_commons.base_models• Removed
rolefield fromProgrammingLanguageMetadatainitialization• Added support for root
directory package detection (7th package)
• Updated test expectations
to handle codebase root package
test_framework_detection_structural_signature.py
Update test imports and file paths for commons migrationunoplat-code-confluence-ingestion/code-confluence-flow-bridge/tests/parser/test_framework_detection_structural_signature.py
• Updated import to use
StructuralSignaturefrom commons package•
Changed test file path to point to commons package location for
structural_signature.pytest_tree_sitter_structural_signature.py
Update test imports and line number expectationsunoplat-code-confluence-ingestion/code-confluence-flow-bridge/tests/parser/test_tree_sitter_structural_signature.py
• Consolidated imports to use
StructuralSignatureandFunctionInfofrom commons package
• Adjusted line number expectations in tests due
to import changes
test_framework_definitions_ingestion.py
Update framework model imports to use commons packageunoplat-code-confluence-ingestion/code-confluence-flow-bridge/tests/integration/test_framework_definitions_ingestion.py
• Updated import to use framework models from commons package
test_poetry_strategy.py
Remove role field from metadata initializationunoplat-code-confluence-ingestion/code-confluence-flow-bridge/tests/parser/package_manager/test_poetry_strategy.py
• Removed
rolefield fromProgrammingLanguageMetadatainitializationtest_uv_strategy.py
Remove role field from metadata initializationunoplat-code-confluence-ingestion/code-confluence-flow-bridge/tests/parser/package_manager/test_uv_strategy.py
• Removed
rolefield fromProgrammingLanguageMetadatainitializationtest_pip_strategy.py
Remove role field from metadata initializationunoplat-code-confluence-ingestion/code-confluence-flow-bridge/tests/parser/package_manager/test_pip_strategy.py
• Removed
rolefield fromProgrammingLanguageMetadatainitializationtest_main_py_structural_signature.json
Update test data to reflect main.py structural changesunoplat-code-confluence-ingestion/code-confluence-flow-bridge/tests/parser/test_main_py_structural_signature.json
• Updated line numbers and function signatures throughout the JSON
test data
• Reflects changes in the main.py file structure due to
import reorganization
• Updated logging format strings and function
call patterns
test_self_extraction_tree_sitter_structural_signature.json
Update test data line numbers for structural signature changesunoplat-code-confluence-ingestion/code-confluence-flow-bridge/tests/parser/test_self_extraction_tree_sitter_structural_signature.json
• Updated line numbers throughout the JSON test data to reflect import
changes
• Adjusted global variable positions and class method line
numbers
test_instance_variable_edge_cases.json
Reorder global variables in test dataunoplat-code-confluence-ingestion/code-confluence-flow-bridge/tests/parser/test_instance_variable_edge_cases.json
• Reordered global variables in the test data
• Changed order of
GLOBAL_CONFIG,ANOTHER_GLOBAL, andapp_instanceentries4 files
code_confluence_graph_deletion.py
Improve graph deletion reliability with prefix matchingunoplat-code-confluence-ingestion/code-confluence-flow-bridge/src/code_confluence_flow_bridge/processor/db/graph_db/code_confluence_graph_deletion.py
• Updated package and file retrieval to use prefix matching instead of
relationship traversal
• Simplified deletion logic by removing
recursive package processing
• Enhanced debugging with more detailed
logging throughout deletion workflow
• Fixed potential issues with
missing relationships during deletion
generic_codebase_parser.py
Fix node creation return value and package discovery logicunoplat-code-confluence-ingestion/code-confluence-flow-bridge/src/code_confluence_flow_bridge/parser/generic_codebase_parser.py
• Fixed node creation to return empty string instead of None when no
record found
• Added safety check to prevent infinite loop in package
discovery
• Improved package hierarchy traversal logic
test_start_ingestion.py
Fix import path for database moduleunoplat-code-confluence-ingestion/code-confluence-flow-bridge/tests/integration/test_start_ingestion.py
• Fixed import path to use proper
src.prefix for database moduleenvironment_utils.py
Fix import path for settings moduleunoplat-code-confluence-ingestion/code-confluence-flow-bridge/src/code_confluence_flow_bridge/utility/environment_utils.py
• Fixed import path to use proper
src.prefix for settings module9 files
__init__.py
Refactor base model imports to use commons packageunoplat-code-confluence-ingestion/code-confluence-flow-bridge/src/code_confluence_flow_bridge/models/code_confluence_parsing_models/init.py
• Moved base model imports (
ClassInfo,FunctionInfo,StructuralSignature,VariableInfo) tounoplat_code_confluence_commons.base_models• Consolidated imports
from commons package
python_framework_detection_service.py
Refactor imports to use commons package and fix import pathsunoplat-code-confluence-ingestion/code-confluence-flow-bridge/src/code_confluence_flow_bridge/engine/python/python_framework_detection_service.py
• Updated imports to use base models from commons package
• Fixed
import paths to use proper
src.prefix for local modulescode_confluence_graph_ingestion.py
Update framework import to use commons packageunoplat-code-confluence-ingestion/code-confluence-flow-bridge/src/code_confluence_flow_bridge/processor/db/graph_db/code_confluence_graph_ingestion.py
• Updated import to use
Frameworkfrom commons package instead oflocal postgres models
• Fixed import path for database session
simplified_python_detector.py
Refactor imports to use commons package base modelsunoplat-code-confluence-ingestion/code-confluence-flow-bridge/src/code_confluence_flow_bridge/engine/python/simplified_python_detector.py
• Consolidated imports to use base models from commons package
•
Removed individual model imports in favor of commons package imports
tree_sitter_structural_signature.py
Refactor imports to use commons package base modelsunoplat-code-confluence-ingestion/code-confluence-flow-bridge/src/code_confluence_flow_bridge/parser/tree_sitter_structural_signature.py
• Consolidated imports to use base models from commons package
•
Removed individual model imports in favor of commons package
framework_query_service.py
Refactor framework query imports to use commons packageunoplat-code-confluence-ingestion/code-confluence-flow-bridge/src/code_confluence_flow_bridge/processor/db/postgres/framework_query_service.py
• Updated imports to use framework-related models from commons package
• Moved
FeatureSpec,FeatureAbsolutePath, andFrameworkFeatureimportsto commons
unoplat_file.py
Refactor imports to use commons package modelsunoplat-code-confluence-ingestion/code-confluence-flow-bridge/src/code_confluence_flow_bridge/models/code_confluence_parsing_models/unoplat_file.py
• Updated imports to use
StructuralSignatureandDetectionfromcommons package
framework_detection_service.py
Refactor imports to use commons package modelsunoplat-code-confluence-ingestion/code-confluence-flow-bridge/src/code_confluence_flow_bridge/engine/framework_detection_service.py
• Updated imports to use
StructuralSignatureandDetectionfromcommons package
framework_loader.py
Refactor framework loader imports to use commons packageunoplat-code-confluence-ingestion/code-confluence-flow-bridge/src/code_confluence_flow_bridge/processor/db/postgres/framework_loader.py
• Updated imports to use framework models from commons package
• Fixed
import path for settings module
2 files
sse_response.py
Add type ignore annotation and formatting fixesunoplat-code-confluence-ingestion/code-confluence-flow-bridge/src/code_confluence_flow_bridge/parser/package_manager/detectors/sse_response.py
• Added type ignore comment for
initmethod• Minor formatting
adjustments to method signature
sync_db_cleanup.py
Add missing newline at end of fileunoplat-code-confluence-ingestion/code-confluence-flow-bridge/tests/utils/sync_db_cleanup.py
• Added missing newline at end of file
2 files
rules.yaml
Rewrite package manager detection rules with simplified configurationunoplat-code-confluence-ingestion/code-confluence-flow-bridge/src/code_confluence_flow_bridge/parser/package_manager/detectors/rules.yaml
• Complete rewrite of package manager detection rules with simplified
Python-only configuration
• Added
contains_absencefield support forexcluding patterns
• Removed weight-based evaluation in favor of
ordered evaluation (first match wins)
• Simplified schema with
homogeneous mapping structure
pyproject.toml
Update dependencies and pytest configuration, remove build systemunoplat-code-confluence-ingestion/code-confluence-flow-bridge/pyproject.toml
• Removed build system configuration section (
build-system)• Updated
unoplat-code-confluence-commonsdependency from version v0.20.0 tov0.22.0
• Added
pythonpath = ["src"]configuration to pytest toolsettings
1 files
Dockerfile
Add ripgrep package to Docker container stagesunoplat-code-confluence-ingestion/code-confluence-flow-bridge/Dockerfile
• Added
ripgreppackage installation to both build and runtime stages• Enhanced container capabilities with additional search tooling
10 files
This is part 1 of 2 in a stack made with GitButler: