Auto-detect src paths for execution environments with autoSearchPaths enabled

[heejaechang]

Description

Fixes an issue where python.analysis.autoSearchPaths only auto-detected and added <project-root>/src to extraPaths, but did not apply the same logic to execution environment roots. When a project has multiple execution environments with their own src directories (e.g., env1/src, env2/src), these were not being automatically added to the search paths.

This change extends the auto-detection behavior to also check for and add <executionEnvironment.root>/src directories when:

• autoSearchPaths is enabled
• Neither the top-level config nor the execution environment explicitly specifies extraPaths
• The src directory exists and is not a package (no __init__.py)

This mirrors the existing behavior for project-root src directories and helps maintain consistent import resolution across multi-environment projects.

How you figured out what to do

The investigation focused on two key areas:

1. configOptions.ts: Located the existing auto-detection logic in initializeFromJson() that handles project-root src paths. The logic checks for autoSearchPaths, verifies the src directory exists, and ensures it's not a package before adding it.
2. setupExecutionEnvironments(): Extended this method to apply similar auto-detection logic to each execution environment's root directory.

The implementation needed to respect the suppression mechanism where explicitly setting extraPaths to an empty array (either at config or environment level) prevents auto-detection.

Implementation

Key changes:

1. Refactored auto-detection logic into a reusable static method _tryGetAutoDetectedSrcExtraPath() that:

   • Takes a root URI and checks for a src subdirectory
   • Returns the path only if src exists and is not a package
   • Returns undefined otherwise

2. Extended setupExecutionEnvironments() to:

   • Accept fs (FileSystem) and autoSearchPaths parameters
   • Track whether extraPaths was specified at the config level
   • For each execution environment with a root, attempt auto-detection if neither config-level nor env-level extraPaths were specified
   • Deduplicate to avoid adding the same path twice

3. Updated call sites in service.ts and testState.ts to pass the required parameters

Testing

Added comprehensive test coverage in config.test.ts with the following scenarios:

1. AutoSearchPathsOnWithExecutionEnvironmentRoots: Verifies auto-detection adds src paths for multiple execution environments
2. AutoSearchPathsOnWithExecutionEnvironmentRootsConfigExtraPathsEmptySuppresses: Ensures empty extraPaths at config level suppresses auto-detection
3. AutoSearchPathsOnWithExecutionEnvironmentRootsEnvExtraPathsEmptySuppresses: Ensures empty extraPaths at environment level suppresses auto-detection
4. AutoSearchPathsOnWithExecutionEnvironmentRootsSrcIsPkgSuppresses: Verifies src directories containing __init__.py are not added
5. AutoSearchPathsOnWithExecutionEnvironmentRootDotDedupes: Ensures no duplicates when execution environment root is "." (project root)

Test fixtures include sample projects with various configurations to cover all edge cases.

Addresses

Addresses microsoft/pylance-release#7378

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

sympy (https://github.com/sympy/sympy)
-   .../projects/sympy/sympy/solvers/ode/nonhomogeneous.py:467:28 - error: Argument of type "Unknown | None" cannot be assigned to parameter "expr" of type "Expr" in function "make_args"
+   .../projects/sympy/sympy/solvers/ode/nonhomogeneous.py:467:28 - error: Argument of type "Expr | Unknown | None" cannot be assigned to parameter "expr" of type "Expr" in function "make_args"
-     Type "Unknown | None" is not assignable to type "Expr"
+     Type "Expr | Unknown | None" is not assignable to type "Expr"
+     Attribute "pop" is unknown (reportAttributeAccessIssue)
+   .../projects/sympy/sympy/solvers/ode/ode.py:610:22 - error: Cannot access attribute "pop" for class "tuple[()]"
+     Attribute "pop" is unknown (reportAttributeAccessIssue)
+   .../projects/sympy/sympy/solvers/ode/ode.py:610:22 - error: Cannot access attribute "pop" for class "tuple[str, ...]"
+   .../projects/sympy/sympy/solvers/ode/ode.py:1516:38 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:1517:38 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:1527:9 - error: No overloads for "update" match the provided arguments (reportCallIssue)
+   .../projects/sympy/sympy/solvers/ode/ode.py:1527:12 - error: "update" is not a known attribute of "None" (reportOptionalMemberAccess)
+   .../projects/sympy/sympy/solvers/ode/ode.py:1527:19 - error: Argument of type "Unknown | dict[Unknown, Unknown] | None" cannot be assigned to parameter "m" of type "Iterable[tuple[str, Unknown]]" in function "update"
+     Type "Unknown | dict[Unknown, Unknown] | None" is not assignable to type "Iterable[tuple[str, Unknown]]"
+       "None" is incompatible with protocol "Iterable[tuple[str, Unknown]]"
+         "__iter__" is not present (reportArgumentType)
+   .../projects/sympy/sympy/solvers/ode/ode.py:1534:43 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:1534:64 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:1540:9 - error: No overloads for "update" match the provided arguments (reportCallIssue)
+   .../projects/sympy/sympy/solvers/ode/ode.py:1540:12 - error: "update" is not a known attribute of "None" (reportOptionalMemberAccess)
+   .../projects/sympy/sympy/solvers/ode/ode.py:1540:19 - error: Argument of type "Unknown | dict[Unknown, Unknown] | None" cannot be assigned to parameter "m" of type "Iterable[tuple[str, Unknown]]" in function "update"
+     Type "Unknown | dict[Unknown, Unknown] | None" is not assignable to type "Iterable[tuple[str, Unknown]]"
+       "None" is incompatible with protocol "Iterable[tuple[str, Unknown]]"
+         "__iter__" is not present (reportArgumentType)
+   .../projects/sympy/sympy/solvers/ode/ode.py:1547:43 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:1547:74 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:1553:9 - error: No overloads for "update" match the provided arguments (reportCallIssue)
+   .../projects/sympy/sympy/solvers/ode/ode.py:1553:12 - error: "update" is not a known attribute of "None" (reportOptionalMemberAccess)
+   .../projects/sympy/sympy/solvers/ode/ode.py:1553:19 - error: Argument of type "Unknown | dict[Unknown, Unknown] | None" cannot be assigned to parameter "m" of type "Iterable[tuple[str, Unknown]]" in function "update"
+     Type "Unknown | dict[Unknown, Unknown] | None" is not assignable to type "Iterable[tuple[str, Unknown]]"
+       "None" is incompatible with protocol "Iterable[tuple[str, Unknown]]"
+         "__iter__" is not present (reportArgumentType)
+   .../projects/sympy/sympy/solvers/ode/ode.py:1560:49 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:1560:70 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
-   .../projects/sympy/sympy/solvers/ode/ode.py:1691:36 - error: Cannot access attribute "lhs" for class "Expr"
-     Attribute "lhs" is unknown (reportAttributeAccessIssue)
-   .../projects/sympy/sympy/solvers/ode/ode.py:1691:55 - error: Cannot access attribute "rhs" for class "Expr"
-     Attribute "rhs" is unknown (reportAttributeAccessIssue)
-   .../projects/sympy/sympy/solvers/ode/ode.py:1692:36 - error: Cannot access attribute "lhs" for class "Expr"
-     Attribute "lhs" is unknown (reportAttributeAccessIssue)
-   .../projects/sympy/sympy/solvers/ode/ode.py:1693:17 - error: No overloads for "__setitem__" match the provided arguments (reportCallIssue)
-   .../projects/sympy/sympy/solvers/ode/ode.py:1693:17 - error: Argument of type "Equality | BooleanFalse | BooleanTrue | Unknown | Expr" cannot be assigned to parameter "value" of type "Equality | BooleanFalse | BooleanTrue" in function "__setitem__"
-     Type "Equality | BooleanFalse | BooleanTrue | Unknown | Expr" is not assignable to type "Equality | BooleanFalse | BooleanTrue"
-       Type "Expr" is not assignable to type "Equality | BooleanFalse | BooleanTrue"
-         "Expr" is not assignable to "Equality"
-         "Expr" is not assignable to "BooleanFalse"
-         "Expr" is not assignable to "BooleanTrue" (reportArgumentType)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3380:7 - error: "update" is not a known attribute of "None" (reportOptionalMemberAccess)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3380:38 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3381:7 - error: "update" is not a known attribute of "None" (reportOptionalMemberAccess)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3381:38 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3382:14 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3383:14 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3384:14 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3396:50 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3397:50 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3398:50 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3438:5 - error: No overloads for "update" match the provided arguments (reportCallIssue)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3438:7 - error: "update" is not a known attribute of "None" (reportOptionalMemberAccess)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3438:14 - error: Argument of type "Unknown | dict[Unknown, Unknown] | None" cannot be assigned to parameter "m" of type "Iterable[tuple[str, Unknown]]" in function "update"
+     Type "Unknown | dict[Unknown, Unknown] | None" is not assignable to type "Iterable[tuple[str, Unknown]]"
+       "None" is incompatible with protocol "Iterable[tuple[str, Unknown]]"
+         "__iter__" is not present (reportArgumentType)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3439:18 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3439:43 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3440:9 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3440:16 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3440:24 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3440:31 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3441:9 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3441:15 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3441:23 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3441:30 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3442:7 - error: "update" is not a known attribute of "None" (reportOptionalMemberAccess)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3442:46 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3442:54 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3443:9 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3443:19 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3443:29 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
+   .../projects/sympy/sympy/solvers/ode/ode.py:3444:10 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)

... (truncated 98 lines) ...

[erictraut]

This change doesn't make sense to me. The autoSearchPaths mechanism was added to support pylance scenarios for users with no interest in type checking and a low threshold for adding custom settings. Execution environments are a power user feature for users who are using type checking and want to highly customize their configuration. If someone has a src subdirectory in a monorepo, they should specify that as the execution environment root, not its parent directory.

[heejaechang]

alright, I will close the issue then. thank you!