We know that the string packets do not have a timestamp, but all the other packets have it.
However, I ran into a few files that do have timestamp data but the timestamp do not make sense, like having day=0. These files are decimated by the instrument so probably some records were changed in a way different from the original files (the original pre-decimation files do not have this problem).
I wonder for these packets if we can set them to NaT to allow set_groups functions to work correctly.
Pinging @imranmaj for comments.
Error message
Example: one of the error messages:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Input In [8], in <cell line: 1>()
----> 1 ed_avgd = ep.open_raw("/Volumes/MURI_4TB/OER_glider_ADCP/ADCP_on_CTD/8302021C1_avgd.ad2cp", sonar_model="AD2CP")
File ~/code_git/echopype/echopype/convert/api.py:435, in open_raw(raw_file, sonar_model, xml_path, convert_params, storage_options)
431 parser = SONAR_MODELS[sonar_model]["parser"](
432 file_chk, params=params, storage_options=storage_options
433 )
434 parser.parse_raw()
--> 435 setgrouper = SONAR_MODELS[sonar_model]["set_groups"](
436 parser,
437 input_file=file_chk,
438 output_path=None,
439 sonar_model=sonar_model,
440 params=_set_convert_params(convert_params),
441 )
443 # Setup tree dictionary
444 tree_dict = {}
File ~/code_git/echopype/echopype/convert/set_groups_ad2cp.py:35, in SetGroupsAd2cp.__init__(self, *args, **kwargs)
33 super().__init__(*args, **kwargs)
34 self.pulse_compressed = self.parser_obj.get_pulse_compressed()
---> 35 self._make_time_coords()
36 with resources.open_text(convert, "ad2cp_fields.yaml") as f:
37 self.field_attrs: Dict[str, Dict[str, Dict[str, str]]] = yaml.safe_load(f)
File ~/code_git/echopype/echopype/convert/set_groups_ad2cp.py:52, in SetGroupsAd2cp._make_time_coords(self)
50 if not packet.has_timestamp():
51 continue
---> 52 timestamps.append(packet.timestamp)
53 i = len(timestamps) - 1
54 if packet.is_average():
File ~/code_git/echopype/echopype/convert/parse_ad2cp.py:328, in Ad2cpDataPacket.timestamp(self)
326 seconds = self.data["seconds"]
327 microsec100 = self.data["microsec100"]
--> 328 return np.datetime64(
329 f"{year:04}-{month:02}-{day:02}T{hour:02}:{minute:02}:{seconds:02}.{microsec100:04}"
330 )
ValueError: Day out of range in datetime string "1910-01-00T00:00:00.0000"
Example data
https://drive.google.com/drive/folders/1SUrARFdtHt81JqmqD6CDn-Ph8PpCm_bK?usp=sharing
- 8282021A1_avgd.ad2cp has problem
- 8302021C1_avgd.ad2cp has problem
- 9142021A1_avgd.ad2cp does not have problem
We know that the string packets do not have a timestamp, but all the other packets have it.
However, I ran into a few files that do have timestamp data but the timestamp do not make sense, like having day=0. These files are decimated by the instrument so probably some records were changed in a way different from the original files (the original pre-decimation files do not have this problem).
I wonder for these packets if we can set them to NaT to allow set_groups functions to work correctly.
Pinging @imranmaj for comments.
Error message
Example: one of the error messages:
Example data
https://drive.google.com/drive/folders/1SUrARFdtHt81JqmqD6CDn-Ph8PpCm_bK?usp=sharing