Skip to content

load_magic_dl1_data_files should filter by date #333

Description

@HealthyPear

This function is only used by lst1_magic_event_coincidence and at the moment it loads all MAGIC calibrated files from an input directory using as regex f"{input_dir}/dl1_*.h5"

then it proceeds to go through the files with a for cycle, which would be OK if the required operation scope was the cycle itself, but instead it does data_list.append(df_events) which means that it will store in memory all the events of all the list of files

since MAGIC date = LST-1 + 1 day, we know which dates from the MAGIC input list to consider, so it does not make sense to load them all (potentially months)

Unfrotunately it seems that the name of the DL1 files from MAGIC have only the run number, so I suggest to:

  • (my recommendation) either modify the upstream script to save those filenames keeping the date and to allow this function to accept a keyword argument filter_date to add to the regex f"{input_dir}/dl1_{date}_*.h5"
  • use the informations from the database to filter only the MAGIC runs effectively taken on the same day of the LST run currently in use

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions