Skip to content

Fix ckb export EEXIST failures by using per-range subdirectories - #121

Draft
Xcodes-chain with Copilot wants to merge 2 commits into
mainfrom
copilot/fix-export-command-test-issues
Draft

Fix ckb export EEXIST failures by using per-range subdirectories#121
Xcodes-chain with Copilot wants to merge 2 commits into
mainfrom
copilot/fix-export-command-test-issues

Conversation

Copilot AI commented Mar 10, 2026

Copy link
Copy Markdown
Contributor

ckb export refuses to write into a non-empty directory (EEXIST, errno 17). All export tests shared a single export/ directory, causing cascading failures across 8 of 11 tests once the first export populated it.

Changes

  • Per-range subdirectory: _export_blocks(from, to) now targets export/{from}-{to}/ instead of the shared export/ dir — each call gets a clean directory
  • Filename normalization fallback: after export, if ckb-{from}-{to}.jsonl isn't found, glob-discovers the single .jsonl in the range dir and renames it to the expected name — handles any ckb export naming variations
  • Explicit error handling: raises FileNotFoundError or RuntimeError when the export produces zero or multiple .jsonl files, instead of silently returning a nonexistent path
  • Housekeeping: removed unused time import; fixed implicit f-string concatenation in test_10
# Before — all ranges share one dir, second export hits EEXIST
run_command(f"./ckb export --target {self.export_dir} --from {from_block} --to {to_block}")

# After — each range gets its own clean subdirectory
range_dir = os.path.join(self.export_dir, f"{from_block}-{to_block}")
run_command(f"mkdir -p {range_dir}")
run_command(f"./ckb export --target {range_dir} --from {from_block} --to {to_block}")
Original prompt

Problem

The CI integration tests in test_cases/ckb_command/test_export_import.py are failing with 8 out of 11 tests broken. The failing job logs: https://github.com/cryptape/ckb-py-integration-test/actions/runs/22847094649

Root Cause

The ckb export command refuses to export when the target directory already contains files, throwing:

Export error: Os { code: 17, kind: AlreadyExists, message: "File exists" }

The _export_blocks() helper method exports all ranges to the same shared export/ directory. When test_01 exports blocks 1-50, it creates a file in export/. When test_02 tries to export blocks 1-10 to the same directory, ckb export sees the existing file and fails with EEXIST. This cascades to all subsequent tests.

Two failure modes observed

  1. Export tests (test_01, test_02, test_03): The export command fails or the expected output filename ckb-{from}-{to}.jsonl doesn't match what ckb export actually generates, resulting in FileNotFoundError / AssertionError: File not found.

  2. Import tests (test_05, test_07, test_08, test_09, test_10): These call _export_blocks(1, 50) again. Since the expected file doesn't exist (wrong name or previous failure), the cache check (if os.path.exists(expected)) fails, and it tries to re-export into the already-populated directory, triggering EEXIST.

Required Fix

Modify the _export_blocks() method in test_cases/ckb_command/test_export_import.py to use a separate subdirectory per block range for each export operation. This avoids the EEXIST conflict because each ckb export call gets its own clean target directory.

Current code (lines 60-69):

def _export_blocks(self, from_block, to_block):
    """Export a range from the source node, return the output file path."""
    expected = os.path.join(self.export_dir, f"ckb-{from_block}-{to_block}.jsonl")
    if os.path.exists(expected):
        return expected
    run_command(
        f"cd {self.node.ckb_dir} && ./ckb export "
        f"--target {self.export_dir} --from {from_block} --to {to_block}"
    )
    return expected

Should be changed to use a per-range subdirectory:

def _export_blocks(self, from_block, to_block):
    """Export a range from the source node, return the output file path."""
    range_dir = os.path.join(self.export_dir, f"{from_block}-{to_block}")
    expected = os.path.join(range_dir, f"ckb-{from_block}-{to_block}.jsonl")
    if os.path.exists(expected):
        return expected
    # Use a dedicated subdirectory per range to avoid EEXIST from ckb export
    run_command(f"mkdir -p {range_dir}")
    run_command(
        f"cd {self.node.ckb_dir} && ./ckb export "
        f"--target {range_dir} --from {from_block} --to {to_block}"
    )
    return expected

Additionally, if ckb export generates a file with a different naming convention than ckb-{from}-{to}.jsonl, the method should discover the actual output file. Add a fallback using glob to find the exported .jsonl file in the range directory and rename/return it:

import glob

def _export_blocks(self, from_block, to_block):
    """Export a range from the source node, return the output file path."""
    range_dir = os.path.join(self.export_dir, f"{from_block}-{to_block}")
    expected = os.path.join(range_dir, f"ckb-{from_block}-{to_block}.jsonl")
    if os.path.exists(expected):
        return expected
    run_command(f"mkdir -p {range_dir}")
    run_command(
        f"cd {self.node.ckb_dir} && ./ckb export "
        f"--target {range_dir} --from {from_block} --to {to_block}"
    )
    # If expected file not found, locate the actual exported file and rename it
    if not os.path.exists(expected):
        files = glob.glob(os.path.join(range_dir, "*.jsonl"))
        if len(files) == 1:
            os.rename(files[0], expected)
    return expected

Only the file test_cases/ckb_command/test_export_import.py needs to be changed. The _export_blocks method is the only place that needs modification.

The following is the prior conversation context from the user's chat exploration (may be truncated):

User: 帮我分析一下这个报错,解决一下这个问题

Assistant: ## 分析报告

问题根因

这次 CI 失败的 核心原因 是:ckb export 命令在导出时,目标目录(export/)中已经存在同名文件,导致报错:

Export error: Os { code: 17, kind: AlreadyExists, message: "File exists" }

这个错误是 Linux 系统错误码 EEXIST (17),意味着 ckb export 不支持覆盖已有文件,而测试代码中多个测试方法尝试导出相同 block 范围到同一个目录。


失败分为两类

第 1 类:前 3 个 export 测试(test_01、test_02、test_03)

这些测试调用 _export_blocks() 进行导出,但导出命令本身静默失败了(exit code 非 0 抛出异常被忽略了,或者第一次就已经失败)。导出文件没有被成功创建,后续断言找不到文件:

FAILED test_01 - AssertionError: File not found: .../export/ckb-1-50.jsonl
FAILED test_02 - FileNotFoundError: .../export/ckb-1-10.jsonl
FAILED test_03 - FileNotFoundError: .../export/ckb-10-20.jsonl

关键线索_export_blocks() 方法里有一个短路逻辑:

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

*This pull request was created from Copilot chat.*
>

<!-- START COPILOT CODING AGENT TIPS -->
---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more [Copilot coding agent tips](https://gh.io/copilot-coding-agent-tips) in the docs.

…errors

Co-authored-by: 15168316096 <45781590+15168316096@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix failing CI integration tests for ckb export command Fix ckb export EEXIST failures by using per-range subdirectories Mar 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants