Skip to content

fix(kubernetes): do not fail task when xcom sidecar kill fails after successful XCom read - #71372

Open
waterWang wants to merge 2 commits into
apache:mainfrom
waterWang:fix-xcom-kill-no-fail-71369
Open

fix(kubernetes): do not fail task when xcom sidecar kill fails after successful XCom read#71372
waterWang wants to merge 2 commits into
apache:mainfrom
waterWang:fix-xcom-kill-no-fail-71369

Conversation

@waterWang

Copy link
Copy Markdown

Issue

Closes #71369

Problem

When KubernetesPodOperator is used with do_xcom_push=True, the XCom sidecar container starts and the XCom value is read successfully from /airflow/xcom/return.json. However, the subsequent extract_xcom_kill step attempts to terminate the sidecar via kill -2 over kubectl exec, which fails with Permission denied on certain container runtimes (e.g., k3s/containerd).

Because extract_xcom_kill is called in the finally block of extract_xcom, the PodCommandException propagates and discards the already-retrieved XCom value, causing the task to be marked as Failed despite the payload being successfully read.

Fix

Wrap extract_xcom_kill in an inner try/except in the finally block. Since the XCom value has already been successfully retrieved, a failure to clean up the sidecar is a non-fatal issue — the sidecar will be cleaned up when the pod is deleted (per on_finish_action). The exception is logged as a warning instead of failing the task.

Test

Added test_extract_xcom_kill_failure_does_not_fail_task which verifies that when extract_xcom_kill raises PodCommandException, the extract_xcom method still returns the successfully-read XCom value.

@boring-cyborg boring-cyborg Bot added area:providers provider:cncf-kubernetes Kubernetes (k8s) provider related issues labels Aug 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:cncf-kubernetes Kubernetes (k8s) provider related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

KubernetesPodOperator: XCom sidecar cleanup fails with "Permission denied" on kill, causing task failure after successful XCom read

1 participant