Skip to content

Base chunk size on number of dask processes #119

Description

@martin-springer

Is your feature request related to a problem? Please describe.
To improve computational efficiency, the chunk size should be somewhat correlated with the number of dask processes.

Describe the solution you'd like
We need to re-chunk some of the dask arrays during an analysis. Currently, this is done based on arbitrary chunk sizes.

Describe alternatives you've considered
Obtain the number of processes from dask cluster and design chunk size based on that.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions