Soil Probe Data to go with Fluxes

[wilsonsj100]

@wilsonsj100 will find previous code cleaning up old Steven's probe data

@zoeread will add this to the macrophyte fluxes and then make sure that probe data with the smart chamber is being brought in with the fluxes

[zoeread]

@wilsonsj100 I added the HydraGo probe data to the 2022-2025 macrophyte fluxes. I checked in the code if we're missing any HydraGo data, and generated a list of flux IDs that we don't have data for. Since there were so many fluxes missing HydraGo data, I exported the list to a csv file in each year.
2025: https://github.com/COMPASS-DOE/synoptic-discrete-data/blob/8c67cd744e206ee744cb8d5a1c0df42ab94fba82/Macrophyte_Fluxes/Synoptic_CB/2025/fluxes_missing_hydrago_data_2025.csv
2024: https://github.com/COMPASS-DOE/synoptic-discrete-data/blob/8c67cd744e206ee744cb8d5a1c0df42ab94fba82/Macrophyte_Fluxes/Synoptic_CB/2024/fluxes_missing_hydrago_data_2024.csv
2023: https://github.com/COMPASS-DOE/synoptic-discrete-data/blob/8c67cd744e206ee744cb8d5a1c0df42ab94fba82/Macrophyte_Fluxes/Synoptic_CB/2023/fluxes_missing_hydrago_data_2023.csv
2022: https://github.com/COMPASS-DOE/synoptic-discrete-data/blob/8c67cd744e206ee744cb8d5a1c0df42ab94fba82/Macrophyte_Fluxes/Synoptic_CB/2022/fluxes_missing_hydrago_data_2022.csv

[wilsonsj100]

Can we do a few spot checks and see if these data were collected on a few data sheets? I know there are sometimes when we don't get these measurements, but I feel like this appears to be a lot. I would definitely expect a lot in the first year or two, but after that I feel like we usually did pretty well.

We can pull TEROS data for the zone during the time of the fluxes and put it into another column so that we can use that too.

[wilsonsj100]

The reason I want to check is to see if these are naming issues.

[zoeread]

I went back and spot checked a few of these. A few of the datasheets say the water was too high for the HydraGo, but most seem to say the HydraGo was used although we don't have data.

• For 2024, I checked the missing data for GCW_WC_1_202405. I'm a bit confused because the datasheets say GWI was sampled on 5/23 and 5/16, but on the 5/23 GWI flux datasheet it looks like there are some GCW IDs so I assume that's actually GCW and that's what was input for the fluxes but not there is no data for HydraGo.
• Checked MSM 202408 and fluxes were taken, no HydraGo data though it was checked off on datasheet
• SWH 202409 flux datasheet says "no HydraGo, water too high" but no notes for all other sites missing data for 202409.
• SWH 202509 WC water was too high to do fluxes or HydraGo, then came back later in September and only did fluxes? Or maybe Evan still has the HydraGo data? B/c he had forgotten to upload the flux data for that extra day as well.
• GCW 202509 - fluxes done and HydraGo checked off on data sheet, but no HydraGo data.
• GWI and MSM 202507 checked off on datasheet that the soil probe was used, but no HydraGo data.

It would be nice if the HydraGo connected to a tablet or something, just because trying to wrangle data from people's phones could be causing issues. But I'm not sure if it just doesn't work on the tablet?

[zoeread]

We can pull TEROS data for the zone during the time of the fluxes and put it into another column so that we can use that too.

I can add in TEROS data, is that the 10cm soil data from the L2 datasets?

[wilsonsj100]

Yes, I would average the closest timestamp TEROS 10cm data from L2 (there are multiple per zone) and put the data in new columns that have TEROS or something in the column ID.

Let's see if we can check in with Evan about whether or not he had that data from some of these dates from this past here. I think that some of the days it was taken should be in there. Are you basing the data present or absent based on the files we have or?

[zoeread]

Okay, that makes sense. I can send Evan the list of sample IDs I don't have HydraGo data for in 2025. These are samples that I have flux data for, but couldn't find the HydraGo data for on the S Drive.

[wilsonsj100]

Okay great - I will start working on compiling the emails of data files I have and then we can also chat with the team Friday. Hopefully we can get the HydraGO working with the tablet for future trips :)

[zoeread]

I added the TEROS data to the flux data as well, although we don't have TEROS data for GWI, as well as SWH 2024 and there are gaps in a lot of the other sensor data.

[wilsonsj100]

Thank you for adding that! I wonder why we don't have any TEROS data for GWI - like at all? for any of the years?

[zoeread]

It's just not in the L2 dataset, maybe it's somewhere else?

[wilsonsj100]

It looks like for some reason those folders just didn't download when I downloaded the dataset... why? I have no idea. Let me download them and add them to the L2 folder

[wilsonsj100]

Okay I put the GWI folders in the v2-1 folder on the S-drive - sorry about that... no idea how it happened