Skip to content

Commit 5af83ea

Browse files
Reproduce executor-side session detachment via scoped session in test
Review asked for the concrete code path that detaches the TaskInstances rather than simulating it with session.expunge(). The mechanism is the executor opening and closing a scoped session while adopting: create_session() returns the scheduler's own thread-scoped session and closes it on exit, expunging everything the orphan query loaded. KubernetesExecutor's completed-pod adoption did exactly this from #66400 until #67850, and released providers up to 10.17.x still ship it (the combination Airflow 3.2.2 constraints pin). The mock now performs that exact operation, so the test exercises the production mechanism end-to-end instead of hand-detaching instances.
1 parent b1ee077 commit 5af83ea

1 file changed

Lines changed: 14 additions & 6 deletions

File tree

airflow-core/tests/unit/jobs/test_scheduler_job.py

Lines changed: 14 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3659,10 +3659,14 @@ def refuse_adoption(tis):
36593659
def test_adopt_or_reset_orphaned_tasks_handles_detached_tis(self, dag_maker, session, mock_executor):
36603660
"""
36613661
Reset and adoption must not depend on the session state of the TaskInstances the
3662-
executor hands back: the reset path copies the full row into TaskInstanceHistory and
3663-
the adopt path reads last_heartbeat_at / dag_run.conf, none of which are loaded by the
3664-
orphan query — on a detached instance those reads raise DetachedInstanceError and
3665-
crash the scheduler loop.
3662+
executor hands back. An executor can open-and-close a scoped session while adopting
3663+
(KubernetesExecutor's completed-pod adoption did exactly that until #67850, and
3664+
released providers <= 10.17.x still do); closing the shared thread-scoped session
3665+
detaches every TaskInstance the orphan query loaded. The reset path then copies the
3666+
full row into TaskInstanceHistory and the adopt path reads last_heartbeat_at /
3667+
dag_run.conf — reads of unloaded attributes that raise DetachedInstanceError on a
3668+
detached instance and crash the scheduler loop; where attributes happen to be
3669+
loaded, the writes are silently lost instead.
36663670
"""
36673671
from airflow.models.taskinstancehistory import TaskInstanceHistory
36683672

@@ -3689,8 +3693,12 @@ def test_adopt_or_reset_orphaned_tasks_handles_detached_tis(self, dag_maker, ses
36893693
session.expunge_all()
36903694

36913695
def adopt_second_reset_first(tis):
3692-
for ti in tis:
3693-
session.expunge(ti)
3696+
# What KubernetesExecutor's completed-pod adoption does mid-adopt in released
3697+
# providers (<= 10.17.x; fixed on its side by #67850): create_session() returns
3698+
# the scheduler's own thread-scoped session, and closing it detaches every
3699+
# TaskInstance the orphan query loaded.
3700+
with create_session():
3701+
pass
36943702
return [ti for ti in tis if ti.task_id == "op1"]
36953703

36963704
mock_executor.try_adopt_task_instances.side_effect = adopt_second_reset_first

0 commit comments

Comments
 (0)