Blockwise documentation#936
Conversation
Correct grammatical errors and improve clarity in the documentation regarding the BlockwiseCoreg class.
Updated the BlockwiseCoreg class description and steps for clarity. Improved multiprocessing configuration example.
| @@ -580,9 +580,7 @@ coregistration methods and for translation only. Rotation shifts will be soon im | |||
| Please set `only_translation=True` when initializing the coregistration method ICP(), LZD() or CPD(). | |||
There was a problem hiding this comment.
xdem.coreg.ICP, xdem.coreg.LZD or xdem.coreg.CPD
| A {class}`~xdem.coreg.BlockwiseCoreg` splits a coregistration across different spatial blocks of an elevation dataset, running that | ||
| method independently in each block. | ||
| By default, these blocks are loaded sequentially to reduce the memory used during a coregistration. | ||
| A {class}`~xdem.coreg.BlockwiseCoreg` splits DEMs into grids, then a coregistration method runs independently in each block. Afterwards, an interpolation method is used to obtain the overall offset. The advantages of this tool are that memory consumption is reduced and local errors are better detected. |
There was a problem hiding this comment.
why do you delete the information that the blocks are loaded sequentially to reduce the memory used ? I think this part is really important
| A {class}`~xdem.coreg.BlockwiseCoreg` splits a coregistration across different spatial blocks of an elevation dataset, running that | ||
| method independently in each block. | ||
| By default, these blocks are loaded sequentially to reduce the memory used during a coregistration. | ||
| A {class}`~xdem.coreg.BlockwiseCoreg` splits DEMs into grids, then a coregistration method runs independently in each block. Afterwards, an interpolation method is used to obtain the overall offset. The advantages of this tool are that memory consumption is reduced and local errors are better detected. |
There was a problem hiding this comment.
not clear that it is the same coregistration method for each block
| These two parameters do **not** need to be the same size. | ||
| ``` | ||
|
|
||
| In this example, processing is performed sequentially. However, it is possible to enable multiprocessing on {class}`~xdem.coreg.BlockwiseCoreg`, the configuration is described in the advanced example. |
There was a problem hiding this comment.
add a link to this example
| blockwise.fit(ref_dem, tba_dem_shifted) | ||
| ``` | ||
|
|
||
| To give you a clearer picture, here is how {func}`~xdem.coreg.BlockwiseCoreg.fit`, divides a DEM with a block size of 500. |
| # Create a configuration without multiprocessing cluster (tasks will be processed sequentially) | ||
| mp_config = MultiprocConfig(chunk_size=500, outfile="aligned_dem.tif", cluster=None) | ||
| blockwise = xdem.coreg.BlockwiseCoreg(xdem.coreg.NuthKaab(), mp_config=mp_config) | ||
| # BlockwiseCoreg is also available without mp_config but with parent_path parameters. |
There was a problem hiding this comment.
this comment " BlockwiseCoreg is also available without mp_config but with parent_path parameters." is not clear for me
| A {class}`~xdem.coreg.BlockwiseCoreg` splits a coregistration across different spatial blocks of an elevation dataset, running that | ||
| method independently in each block. | ||
| By default, these blocks are loaded sequentially to reduce the memory used during a coregistration. | ||
| A {class}`~xdem.coreg.BlockwiseCoreg` splits DEMs into grids, then a coregistration method runs independently in each block. Afterwards, an interpolation method is used to obtain the overall offset. The advantages of this tool are that memory consumption is reduced and local errors are better detected. |
There was a problem hiding this comment.
"Interpolation method": Maybe "interpolation" or "aggregation" method: We want to support things like the mean over all blocks (what many people do), not only the more advanced 2D fitting done in the current Blockwise.
Final statement: The primary objective is to better correct identify and local errors. Additionally, it makes scalable execution (which includes parallel processing and out-of-memory considerations) straightforward, but we do intend to support scalable execution also directly in normal coregistration (pretty close now), so it's not the main objective here.
Don't worry too much about it, I can also update this later on when I add the Scalability documentation in xDEM 😉
| blockwise.fit(ref_dem, tba_dem_shifted) | ||
| ``` | ||
|
|
||
| To give you a clearer picture, here is how {func}`~xdem.coreg.BlockwiseCoreg.fit`, divides a DEM with a block size of 500. |
There was a problem hiding this comment.
Stylistic, but it's not common practice to use active voice ('we', "you") in the documentation, except for big notes in landing pages or so ("If you want to convert your GDAL pipeline, this is what you do..."), but not in feature pages of documentation.
|
Hello, Thank you for your feedback ! I have corrected the formatting issues. Regarding the blockwise description paragraph, I have revised it to first explain local errors detection, and then detail the sequential process, as well as the possibility of use larger datasets. I'm still not sure how I approached interpolation and aggregation. |
rhugonnet
left a comment
rhugonnet
left a comment
There was a problem hiding this comment.
Thanks! Perfect, no need to polish the aggregation/interpolation too much, as this will be adjusted in the PRs that will add the aggregagation!
3 participants
Resolves #868
The aim of this PR is to rewrite the blockwise documentation, both in the coregistration section and in the advanced example.
@belletva , @adehecq , @rhugonnet here are the links to the read the docs:
For the coregistration section:
https://xdem-ould-a.readthedocs.io/en/latest/coregistration.html#dividing-coregistration-in-blocks
For the example:
https://xdem-ould-a.readthedocs.io/en/latest/advanced_examples/plot_blockwise_coreg.html