PFT/landuse/urban tools: availability, documentation, training #2206
Replies: 20 comments 15 replies
-
|
@lawrencepj1 shared with us by email the latest pdf of his documentation. I'm open to uploading here if you all think it makes sense. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @slevis-lmwg yes please upload and make available as a work in progress |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
Meeting 2023/11/16 @lawrencepj1 @ekluzek @mvdebolskiy @slevis-lmwg Peter points out:
So Peter proposes standardizing steps 2 and 3 initially. Peter is organizing step 2 on the cgd machines here: Peter is organizing step 3 on cheyenne and needs to move stuff to /campaign before cheyenne goes away. This is a sample namelist used by the C-program (each directory here has a diff. namelist): For /glade/p data we decided (Erik, Peter, and Sam L will meet Dec 6 to do the following):
|
Beta Was this translation helpful? Give feedback.
-
|
@lawrencepj1 can you add the path to where READMEs and shell-scripts are for step 2? |
Beta Was this translation helpful? Give feedback.
-
|
@mvdebolskiy I think you mean the path that I listed under step 2 above. The README is there and the scripts are in /tools. Do you mean something else? |
Beta Was this translation helpful? Give feedback.
-
|
Yeah, step two. I thought it was also on thesis. nvm |
Beta Was this translation helpful? Give feedback.
-
|
Updates: |
Beta Was this translation helpful? Give feedback.
-
|
@mvdebolskiy thank you for tracking today's conversation with these notes. |
Beta Was this translation helpful? Give feedback.
-
|
README from Land Use Data Prep CTSM52 Land Use Data Prep Workflow This is step two in the bigger picture of generating CTSM raw PFT/CFT data.
Use tools/runcreateluh2modisearthstatextraptreerawfiles CTSM52_MODISCurrent CTSM52_POTVEGMax 2005
Generate the Descriptor files in rawdata: Generate the Descriptor files in surfdata:
Generate the Descriptor files in rawdata: Generate the Descriptor files in surfdata:
In surfdata: |
Beta Was this translation helpful? Give feedback.
-
|
Moving tools from Peter to ESCOMP:
Peter will present on CESM version for CCIS with datasets created with the previous version of mksurf tools.
Newer versions of landuse tools are not on github. (for scenarios). |
Beta Was this translation helpful? Give feedback.
-
|
@mvdebolskiy offered to make a list of TODO checkboxes here in the discussion. Possibly at the next meeting we can decide to start a project board. |
Beta Was this translation helpful? Give feedback.
-
|
Notes from 17th May 2024:
|
Beta Was this translation helpful? Give feedback.
-
|
Notes 18th September 2024
|
Beta Was this translation helpful? Give feedback.
-
|
October Agenda: |
Beta Was this translation helpful? Give feedback.
-
|
Descriptor files tool have been moved from |
Beta Was this translation helpful? Give feedback.
-
|
Discussion on CMIP7 LULCC #2851 |
Beta Was this translation helpful? Give feedback.
-
|
Updated README file with all tools using namelist files rather than command line arguments. For @slevis-lmwg @ekluzek @mvdebolskiy CTSM52 Land Data Prep Workflow This is step two in the bigger picture of generating CTSM raw PFT/CFT data. Historical Descriptor File Generation
tools/runcreateworkclm5extrapallvcfpftrawdata worknamelist/CTSM52_MODISCURRENTEXTRAPVCFPFT_generateallrawdata.txt Check Extrapolation netcdf files in worksurfdata for PFT composition (TREE_EXTRAP,HERB_EXTRAP,GRASS_EXTRAP) tools/runcreateworkclm5surfdata worknamelist/CTSM52_MODISCURRENTEXTRAPTREEVCFPFT_generatesurfdata.txt
Generate the Descriptor files in rawdata: Generate the Descriptor files in surfdata:
Generate the Descriptor files in rawdata: Generate the Descriptor files in surfdata:
tools/runcreateluh2descpftlaisurfdata luh2descnamelist/CTSM52_LUH2HISTDESCPFTLAIS_generatedata.txt Future SSP Time Series Descriptor File Generation - Assumes that Historical Descriptor Files have been generated.
|
Beta Was this translation helpful? Give feedback.
-
|
Meet with @slevis-lmwg @lawrencepj1 and @wwieder. Mostly confirming the CMIP7 process for ctsm5.4 and CESM3.0 release.
Non-CMIP7 work will be outside of the standard workflow. On a branch or in a separate project. Reforestation, or bioenergy work for example. |
Beta Was this translation helpful? Give feedback.
-
|
Meeting with @lawrencepj1 @ekluzek 2025/2/28 When the new version becomes visible in /project/ccis/lawrence (today it's not), Peter will put it on github. Today's process... On a cgd machine Peter made a demo directory with an older version because the new is not visible right now: cd /project/tss/lawrence/ctsm52landusedataprepdemo/ctsm52landusedata025
# In the future we will obtain the above with git clone of the dataprep tool. Then...
cd bin
make
# Next follow the instructions in the README (this part is step 2).
# Now copy the .nc outputs to /glade and work on derecho (this part is step 3).
# The user would git clone the landuse data tool now.
cd /glade/campaign/cesm/development/lmwg/landuse_source_tools
# Notice here /MksrfDataTools is the old mksurfdata_map refurbished to work on derecho
cd CTSM52LandUseDataTool/bin
make
cd ../../../landuse_source_data/CTSM53RawData/globalctsm53histMKSRFDeg025_demo
/glade/campaign/cesm/development/lmwg/landuse_source_tools/CTSM52LandUseDataTool/bin/ctsm52landusedatatool globalctsm52histTRENDY2023namelist.txt # <-- the namelist is the only file you need to update, then you may want to run with qcmd --
# When cloning, the user will get a demo namelist.txt file and demo look-up tables
# Next is step 4, i.e. the mksurfdata_esmf (or mksurfdata_map) step to generate fsurdat/landuse filesNext meeting March 28 to do all this from github and to discuss CMIP 7 (see invite). |
Beta Was this translation helpful? Give feedback.
-
|
@lawrencepj1 will add new tools to github, for biofuels and reforestation/deforestation. We will put them in ESCOMP, but not fully support them like CTSM. We will meet with @olyson to talk about dynamic urban datasets. Will meet in August for this. We discussed it and want to make sure we keep this capability as Keith continues to phase into retirement. Dynamic lakes -- we thought about it, but don't have a plan for it. SSP datasets for CMIP7 will be coming out as a ctsm6.1/cesm3.1 @lawrencepj1 will create CMIP6 SSP scenario data extensions to CMIP7 historical harmonized with LUH3 and year 2024. This will be available for people to use. And a way to show that it will work for the SSP version update. So a non-supported option that people can use by swapping in those datasets for the CMIP6 SSP datasets that WILL be in the release. TRENDY -- we will verify with @adrifoster that we will create datasets at hcru_hcru resolution. @lawrencepj1 is creating the rawdata that @slevis-lmwg will use in mksurfdata_esmf for the datasets needed. This needs to be 1700, and 1700-2024 transient. |
Beta Was this translation helpful? Give feedback.
-
|
The one issue for the release is #3300 to make sure it's in the CTSM checkout. |
Beta Was this translation helpful? Give feedback.
-
|
Land Use and Urban Data Tools Meeting. 08/06/2025 Peter, Sam L., and Erik Land Use Data Tools: Peter has migrated the CLM5 Land Data Tools from CGD /project/ccis to: The tools have individual projects that progress through the workflow:
Each data directory has a set of tools:
Current projects include cmip6, smoothed, ccis, trendy2025. Keith is very interested in joining the next meeting to discuss the inclusion of urban tools which have been developed with CLM5 through CTSM 5.4. Today also made progress in #3300. |
Beta Was this translation helpful? Give feedback.
-
|
@mvdebolskiy we were just meeting, and we wondered the status of the conversion of the tech note from word to RST? I know you worked on this a long time ago and maybe didn't finish. And I know you are also busy with the NorESM release. But, we just wanted to know where you were at with it, so we can plan on what to do next. If you converted some of it, and developed a method to do the conversion we could take up doing that. |
Beta Was this translation helpful? Give feedback.
-
|
Nice! You are ahead of us -- I just sent you an email about getting advice from you on some of this. There we want to do a separate User's Guide type thing to document how to use the tools, that we'll start in markdown from the get-go for it. Yes, markdown seems easier to work with. |
Beta Was this translation helpful? Give feedback.
-
|
@ekluzek I've tried Markdown and rst with pandoc a year ago, the only issue is going through the reformated document and fixing some bullet points,tables. The two other things to be done is to extract 125 images from the .docx and deal with references (unless you want them to be plain-text). |
Beta Was this translation helpful? Give feedback.
-
|
Ahh, right, references. Markdown can't handle that, so RST works. Let's bring in what there is now and make issues to finish it off. |
Beta Was this translation helpful? Give feedback.
-
|
In the CTSM SE meeting this morning, we decided to (1) put whatever Mat has now into the CTSM-LUH-LandUseDataPrepTools repo, (2) add that repo as a submodule to CTSM even though it's not fully done, (3) |
Beta Was this translation helpful? Give feedback.
-
|
Just noting that Myst markdown has good support for references. It was used for the SEA ISS Conference Proceeding Papers. |
Beta Was this translation helpful? Give feedback.
-
|
We had a shorter meeting today. @olyson is out today. So will meet next week. We talked about the documentation. And had the idea of the User's Guide being a Jupyter notebook. This might be an easy way to get it started, and also be useful for users to demonstrate some of the code. The downside is that Notebooks need to stay up to date with the code, and need conda environments and such to stay working. So there is work in maintaining Notebooks and making sure users can use them on any platform (container to handle this?). Or we could just say -- it's only supported on NSF-NCAR JupyterHub? Could we just use the containers that @briandobbins made for the CESM Tutorial for this? Brian also supported the Tutorials for CTSM and tower sites. Also notes that Myst markdown is good for papers and documents. It was used in the SEA ISS conference for the conference proceedings. It has good support for references and figures in formal papers. |
Beta Was this translation helpful? Give feedback.
-
|
Meeting 2025/9/10 with @olyson @lawrencepj1 @wwieder @ekluzek @slevis-lmwg Urban tools status CLM5 CTSM5.3 CTSM5.4 |
Beta Was this translation helpful? Give feedback.
-
|
I remembered to ask Keith about the lake data, too: |
Beta Was this translation helpful? Give feedback.
-
|
2025/11/5 meeting with @lawrencepj1 @olyson
|
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
TODOs as of Feb 2024.
mksrurfdata_esmfto explain what is expected to be in the mksruf_ files for landuse.@lawrencepj1 @mvdebolskiy @ekluzek
Beta Was this translation helpful? Give feedback.
All reactions