Fix task planner matching plans to wrong tasks (issue #3953)

[devin-ai-integration]

Summary

Fixes issue #3953 where the task planner was attaching plans to the wrong tasks when the LLM returned them out of order.

The Problem: The original implementation used zip() to blindly pair tasks with plans, assuming they were in the same order. However, LLMs sometimes return plans in unexpected order (e.g., starting with "Task Number 21" instead of "Task Number 1"), causing plan mismatches.

The Solution:

• Extract task numbers from each plan using regex pattern matching on the task field
• Build a mapping of task number → plan
• Apply plans to tasks based on correct task number matches
• Add warning logs when task numbers can't be extracted or plans are missing

Changes:

• Modified Crew._handle_crew_planning() to use task number extraction instead of positional matching
• Added comprehensive test that reproduces the bug with plans in wrong order and verifies the fix

Review & Testing Checklist for Human

• Test with real LLM planning: Run a crew with planning=True and 5-10 tasks to verify the regex pattern r"Task Number (\d+)" correctly matches actual LLM outputs from the planning agent
• Verify warning logs: Check that warning messages are visible and actionable when plans don't match (test by manually creating a malformed plan response)
• Edge case: Different task number formats: Consider if LLMs might format task numbers differently (e.g., "task number" lowercase, "Task Fix tiny typo #1", "Task 1:", etc.) and whether the regex should be more flexible
• Large crew test: Test with a crew that has 20+ tasks (matching the original issue report mentioning "Task Number 21") to ensure the fix works at scale

Test Plan

1. Create a crew with 5-10 tasks and planning=True
2. Run crew.kickoff() and verify each task receives the correct plan in its description
3. Check logs for any warnings about missing or mismatched plans
4. Optionally: Add a print statement in _handle_crew_planning() to log the plan_map and verify task numbers are extracted correctly

Notes

• The fix assumes task numbers in plans follow the format "Task Number X" (case-insensitive) as generated by CrewPlanner._create_tasks_summary()
• If a plan's task number can't be extracted, it's logged as a warning but execution continues
• The test uses mocked LLM responses, so real-world testing is important to validate the regex pattern

Link to Devin run: https://app.devin.ai/sessions/c96705b2147b411682a3b3d6339b4a13
Requested by: João ([email protected])

---

Note

Maps LLM-generated plans to the correct tasks by extracting task numbers (regex) instead of relying on order; adds warnings and a unit test for out-of-order plans.

• Core (planning):
  • Replace positional zip() pairing with task-number mapping in Crew._handle_crew_planning().
    • Extract task numbers via regex r"Task Number (\d+)" from each plan’s task field and build plan_map.
    • Append the matched plan to the corresponding task description (1-indexed).
    • Log warnings when the task number can't be extracted or when no plan exists for a task.
• Tests:
  • Add test_crew_planning_with_mismatched_task_order verifying correct plan-task matching when the planner returns plans out of order and ensuring wrong plans aren’t attached to wrong tasks.

^{Written by Cursor Bugbot for commit 9311689. This will update automatically on new commits. Configure here.}

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring