Skip to content

feat(engine-models): Add pydantic models for custom grammar detection engine - #662

Merged
JayGhiya merged 7 commits into
mainfrom
update-code-confluence-commons
Jul 25, 2025
Merged

feat(engine-models): Add pydantic models for custom grammar detection engine#662
JayGhiya merged 7 commits into
mainfrom
update-code-confluence-commons

Conversation

@JayGhiya

@JayGhiya JayGhiya commented Jul 22, 2025

Copy link
Copy Markdown
Member

User description

This commit introduces a set of Pydantic models for the custom grammar
detection engine. The key changes are:

  • Defines enums for TargetLevel, LocatorStrategy, and Concept to represent the various concepts used in the refactored engine.
  • Introduces the ConstructQueryConfig model to represent the language-specific configuration for constructing the concept queries.
  • Defines the FeatureSpec model to represent the strongly-typed feature specification from the schema.
  • Adds the Detection and DetectionResult models to represent the results of feature detection in source code.
  • Introduces specialized detection information models for AnnotationLikeInfo, CallExpressionInfo, and InheritanceInfo.

These models provide a structured and type-safe way to work with the custom grammar detection engine, improving maintainability and robustness of the codebase.


PR Type

Enhancement


Description

  • Add comprehensive Pydantic models for custom grammar detection engine

  • Introduce structural signature models for source code analysis

  • Add SQLModel framework metadata models with PostgreSQL support

  • Migrate build system from Poetry to uv/hatch


Diagram Walkthrough

flowchart LR
  A["Engine Models"] --> B["Detection Engine"]
  C["Structural Models"] --> D["Code Analysis"]
  E["Framework Models"] --> F["PostgreSQL Storage"]
  G["Build System"] --> H["uv/hatch Migration"]
Loading

File Walkthrough

Relevant files

@huly-for-github

Copy link
Copy Markdown

Connected to Huly®: UNOPL-799

@qodo-code-review

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Duplicate File

This file appears to be a duplicate of the structural signature models that are already defined in the base_models/structural_signature.py file. The models have slight differences which could lead to confusion and maintenance issues.

"""Pydantic models for file structural signatures."""

from typing import List, Optional

from pydantic import BaseModel, Field


class ImportInfo(BaseModel):
    """Information about import statements in a file."""
    start_line: int = Field(..., description="Starting line number of import block")
    end_line: int = Field(..., description="Ending line number of import block")
    imports: List[str] = Field(..., description="Array of import statement strings")


class VariableInfo(BaseModel):
    """Information about a global variable declaration."""
    start_line: int = Field(..., description="Starting line number of variable declaration")
    end_line: int = Field(..., description="Ending line number of variable declaration")
    signature: str = Field(..., description="Complete variable declaration line(s)")


class FunctionInfo(BaseModel):
    """Information about a function or method."""
    start_line: int = Field(..., description="Line number where the function starts")
    end_line: int = Field(..., description="Line number where the function ends")
    signature: str = Field(..., description="Complete function declaration including decorators, def/async def, parameters, return type")
    docstring: Optional[str] = Field(None, description="Function docstring")


class ClassInfo(BaseModel):
    """Information about a class definition."""
    start_line: int = Field(..., description="Line number where the class starts")
    end_line: int = Field(..., description="Line number where the class ends")
    signature: str = Field(..., description="Complete class declaration including decorators and inheritance")
    docstring: Optional[str] = Field(None, description="Class docstring")
    class_variables: List[VariableInfo] = Field(default_factory=list, description="Class-level variables")
    methods: List[FunctionInfo] = Field(default_factory=list, description="Class methods")
    nested_classes: List['ClassInfo'] = Field(default_factory=list, description="Nested class definitions")


class StructuralSignature(BaseModel):
    """
    Structural signature of a source code file.

    This model captures the high-level structure and outline of a source file,
    including module-level constructs, imports, functions, classes, and their
    positions within the file.
    """
    module_docstring: Optional[str] = Field(None, description="Module-level docstring")
    imports: Optional[ImportInfo] = Field(default=None, description="Import statements block")
    global_variables: List[VariableInfo] = Field(default_factory=list, description="Module-level variables")
    functions: List[FunctionInfo] = Field(default_factory=list, description="Module-level functions")
    classes: List[ClassInfo] = Field(default_factory=list, description="Class definitions")

    # Additional metadata
    total_lines: Optional[int] = Field(None, description="Total number of lines in the file")
    has_main_block: bool = Field(False, description="Whether the file has if __name__ == '__main__' block")
    encoding: Optional[str] = Field("utf-8", description="File encoding")


# Update forward references for nested classes
ClassInfo.model_rebuild()
Model Inconsistency

The StructuralSignature model is missing the imports field that exists in the duplicate file, and ClassInfo uses 'vars' instead of 'class_variables'. These inconsistencies between the two versions could cause integration issues.

    vars: List[VariableInfo] = Field(default_factory=list, description="Class and instance variables")
    methods: List[FunctionInfo] = Field(default_factory=list, description="Class methods")
    nested_classes: List['ClassInfo'] = Field(default_factory=list, description="Nested class declarations")


class StructuralSignature(BaseModel):
    """
    Structural signature of a source code file.

    This model captures the high-level structure and outline of a source file,
    including module-level constructs, functions, classes, and their
    positions within the file.
    """
    module_docstring: Optional[str] = Field(None, description="Module-level docstring")
    global_variables: List[VariableInfo] = Field(default_factory=list, description="Module-level variables")
    functions: List[FunctionInfo] = Field(default_factory=list, description="Module-level functions")
    classes: List[ClassInfo] = Field(default_factory=list, description="Class definitions")
Hardcoded Path

The src configuration contains a hardcoded absolute path specific to a user's local machine, which will not work in other environments and should be made relative or removed.

src = ["/Users/jayghiya/Documents/unoplat/unoplat-codebase-understanding/unoplat-code-confluence-commons"]
line-length = 88

@qodo-code-review

qodo-code-review Bot commented Jul 22, 2025

Copy link
Copy Markdown

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
Possible issue
Fix conflicting field name

The field name vars conflicts with Python's built-in vars() function and differs
from the original class_variables field name. This could cause naming conflicts
and breaks compatibility with existing code.

unoplat-code-confluence-commons/unoplat_code_confluence_commons/base_models/structural_signature.py [35]

-vars: List[VariableInfo] = Field(default_factory=list, description="Class and instance variables")
+class_variables: List[VariableInfo] = Field(default_factory=list, description="Class-level variables")
  • Apply / Chat
Suggestion importance[1-10]: 7

__

Why: The suggestion correctly identifies that the field name vars shadows a Python built-in function, which is poor practice and could lead to confusion or bugs.

Medium
  • Update

JayGhiya added 2 commits July 22, 2025 18:35
… engine

This commit introduces a set of Pydantic models for the custom grammar detection engine. The key changes are:

- Defines enums for `TargetLevel`, `LocatorStrategy`, and `Concept` to represent the various concepts used in the refactored engine.
- Introduces the `ConstructQueryConfig` model to represent the language-specific configuration for constructing the concept queries.
- Defines the `FeatureSpec` model to represent the strongly-typed feature specification from the schema.
- Adds the `Detection` and `DetectionResult` models to represent the results of feature detection in source code.
- Introduces specialized detection information models for `AnnotationLikeInfo`, `CallExpressionInfo`, and `InheritanceInfo`.

These models provide a structured and type-safe way to work with the custom grammar detection engine, improving maintainability and robustness of the codebase.
This change introduces a new method `sync_frameworks_for_codebase` that
handles the synchronization of framework nodes based on package
dependencies. The key changes are:

- The framework sync logic is moved outside of the main package metadata
  ingestion transaction to prevent "got Future attached to a different
  loop" errors in CI environments.
- PostgreSQL queries and Neo4j operations are kept in separate contexts
  to avoid event loop conflicts.
- The method first checks for known frameworks in Postgres, then creates
  the necessary framework nodes and relationships in Neo4j.
- The version of the `code-confluence-flow-bridge` package is bumped to
  0.41.0 to reflect this change.
@JayGhiya
JayGhiya force-pushed the update-code-confluence-commons branch from 86aedef to 070f25e Compare July 22, 2025 13:17
@JayGhiya

Copy link
Copy Markdown
Member Author

ok this has been a lesson. i had given access to agent to run commands of act (Tool we use to run github action locally) it added bind and removed all uncommitted changes of our experiment with pydantic ai agent all files gone revoking access for that command

@JayGhiya
JayGhiya merged commit c31e710 into main Jul 25, 2025
2 checks passed
JayGhiya added a commit that referenced this pull request Apr 8, 2026
feat(engine-models): Add pydantic models for custom grammar detection engine
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant