Skip to content

Sandbox “no space left on device” — 6,543 failed workflow runs and a full hour of failures #42286

Description

@pavlov-victor

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • 【中文用户 & Non English User】请使用英语提交,否则会被关闭 :)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

Cloud version

Cloud or Self Hosted

Cloud

Steps to reproduce

  1. Run a published Dify Cloud workflow containing Python Code nodes through POST
    https://api.dify.ai/v1/workflows/run.
  2. During normal shadow traffic on September 14, 2026, between 12:43 and 14:01 UTC, executions
    repeatedly failed with:

Got error code: -500.
Got error msg: open /var/sandbox/sandbox-python/tmp/.py:
no space left on device

  1. Our export contains 6,543 failures with this error. During 13:00–14:00 UTC, all 4,953 workflow
    runs failed.
  2. Successful processing resumed after approximately 14:01 UTC.

We cannot reproduce the storage exhaustion on demand because the sandbox infrastructure is managed by
Dify Cloud. We can provide affected run IDs and sanitized logs for investigation.

✔️ Expected Behavior

Python Code nodes should execute reliably without failures caused by exhausted temporary storage.

Dify Cloud should maintain sufficient sandbox capacity and automatically clean up temporary execution
files. Infrastructure failures should not cause a prolonged outage across otherwise valid workflow
requests.

❌ Actual Behavior

No response

Activity

  1. pavlov-victor commented on Sep 14, 2026

    @pavlov-victor
    Author

    langgenius/dify-sandbox#290 my pull request to solve this problem

  2. fatelei commented on Sep 15, 2026

    @fatelei
    Contributor

    it has been fixed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    🐞 bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions