Add the logcheckreport.py script

[wilkmar]

The script generates a report with all checks that depend on log messages.

The example 'short' report:

Number of 'input' checks: 60
Number of scenario files with 'input' checks: 54
Number of scenario files analyzed: 173

The example 'full' report:

===================================================
Files with 'input' checks (log files):
- hotsos/defs/scenarios/lxd/lxcfs_deadlock.yaml:
    - fuse_out_of_threads_error
- hotsos/defs/scenarios/lxd/bugs/lp1807628.yaml:
    - lxcfs_segfault
- hotsos/defs/scenarios/rabbitmq/bugs/lp1943937.yaml:
    - has_1943937
- hotsos/defs/scenarios/storage/ceph/ceph-osd/osd_flapping.yaml:
    - osd_flapping
- hotsos/defs/scenarios/storage/ceph/ceph-osd/osd_slow_ops.yaml:
    - osd_slow_ops
- hotsos/defs/scenarios/storage/ceph/ceph-osd/pg_overdose.yaml:
    - ceph_osd_withhold_creation
- hotsos/defs/scenarios/storage/ceph/ceph-osd/osd_latency.yaml:
    - osd_disk_latency
- hotsos/defs/scenarios/storage/ceph/ceph-mon/mon_elections_flapping.yaml:
    - ceph_log_has_election_calls
- hotsos/defs/scenarios/storage/ceph/ceph-mon/osd_flapping.yaml:
    - osd_flapping
- hotsos/defs/scenarios/storage/ceph/ceph-mon/recent_laggy_pgs.yaml:
    - recent_laggy_pgs
- hotsos/defs/scenarios/storage/ceph/ceph-mon/osd_slow_ops.yaml:
    - osd_slow_ops
- hotsos/defs/scenarios/storage/ceph/ceph-mon/osd_slow_heartbeats.yaml:
    - osd_slow_heartbeats
- hotsos/defs/scenarios/storage/ceph/ceph-mgr/autoscaler_overlap_roots.yaml:
    - overlapping_roots
- hotsos/defs/scenarios/storage/nfs/nfs-detect-mount-name-resolution-failure.yaml:
    - has_nfs_name_resolution_failures
- hotsos/defs/scenarios/openstack/keystone/bugs/lp1896125.yaml:
    - has_maxconn_reached_error
- hotsos/defs/scenarios/openstack/neutron/oslo_privsep_errors.yaml:
    - l3_agent_privsep_errors
    - ovs_agent_privsep_errors
- hotsos/defs/scenarios/openstack/neutron/bugs/lp1929832.yaml:
    - has_1929832
- hotsos/defs/scenarios/openstack/neutron/bugs/lp1996594.yaml:
    - has_1996594
- hotsos/defs/scenarios/openstack/neutron/bugs/lp1883089.yaml:
    - has_1883089
- hotsos/defs/scenarios/openstack/neutron/bugs/lp1965297.yaml:
    - has_1965297
- hotsos/defs/scenarios/openstack/neutron/bugs/lp1979089.yaml:
    - has_1979089
- hotsos/defs/scenarios/openstack/neutron/bugs/lp1948466.yaml:
    - has_1948466
- hotsos/defs/scenarios/openstack/neutron/bugs/lp2017748.yaml:
    - has_2017748
- hotsos/defs/scenarios/openstack/neutron/bugs/lp1907686.yaml:
    - has_1907686
- hotsos/defs/scenarios/openstack/neutron/bugs/lp1928031.yaml:
    - has_1928031
- hotsos/defs/scenarios/openstack/neutron/bugs/lp1896506.yaml:
    - has_1896506
- hotsos/defs/scenarios/openstack/oslo_messaging/lp1934937.yaml:
    - journalctl_has_1934937
- hotsos/defs/scenarios/openstack/nova/bugs/lp1860743.yaml:
    - has_1860743
- hotsos/defs/scenarios/openstack/nova/bugs/lp2012284.yaml:
    - has_2012284
- hotsos/defs/scenarios/openstack/nova/bugs/lp1972028.yaml:
    - logs
- hotsos/defs/scenarios/openstack/nova/bugs/lp1761062.yaml:
    - has_1761062
- hotsos/defs/scenarios/openstack/nova/bugs/lp1967956.yaml:
    - logs
- hotsos/defs/scenarios/openstack/nova/bugs/lp1904580.yaml:
    - has_1904580
- hotsos/defs/scenarios/openstack/nova/bugs/lp1888395.yaml:
    - has_1888395_pt1
- hotsos/defs/scenarios/openstack/nova/bugs/lp1944619.yaml:
    - has_1944619
- hotsos/defs/scenarios/openstack/octavia/excessive_failovers.yaml:
    - auto_failovers_day_limit
    - auto_failovers_hour_limit
    - over_mtu_drops_ohm0_kernlog
- hotsos/defs/scenarios/openstack/octavia/bugs/sb1896125.yaml:
    - has_2008099_logs
- hotsos/defs/scenarios/openstack/octavia/bugs/lp2029857.yaml:
    - has_2029857
- hotsos/defs/scenarios/mysql/bugs/lp372017.yaml:
    - free_blocks_difficult
- hotsos/defs/scenarios/mysql/bugs/lp2039444.yaml:
    - has_2039444
- hotsos/defs/scenarios/juju/bugs/lp1852502.yaml:
    - has_lp1852502
- hotsos/defs/scenarios/juju/bugs/lp1996230.yaml:
    - has_lp1996230
- hotsos/defs/scenarios/juju/bugs/lp1948906.yaml:
    - has_lp1948906
- hotsos/defs/scenarios/juju/bugs/lp1983140.yaml:
    - has_lp1983140
- hotsos/defs/scenarios/juju/bugs/lp1910958.yaml:
    - has_lp1910958
- hotsos/defs/scenarios/juju/bugs/lp1895040.yaml:
    - has_unknown_relation_errors
- hotsos/defs/scenarios/juju/bugs/lp1983506.yaml:
    - has_lp1983506
- hotsos/defs/scenarios/openvswitch/service_restarts.yaml:
    - ovs_frequent_restarts
- hotsos/defs/scenarios/openvswitch/ovn/bfd_flapping.yaml:
    - vswitchd_to_ovn_controller_inactivity_timeouts
    - chassis_gw_port_reassignments
- hotsos/defs/scenarios/openvswitch/ovn/ovn_elections.yaml:
    - sb_ovn_elections
    - nb_ovn_elections
- hotsos/defs/scenarios/kernel/qla2xxx.yaml:
    - qla2xxx_skipping_scsi_scan_host
- hotsos/defs/scenarios/kernel/disk_failure.yaml:
    - disk_failure
    - any_io_error
- hotsos/defs/scenarios/kernel/network/misc.yaml:
    - has_nf_conntrack_full
- hotsos/defs/scenarios/landscape/bugs/lp2081056.yaml:
    - has_2081056
===================================================
Number of 'input' checks: 60
Number of scenario files with 'input' checks: 54
Number of scenario files analyzed: 173

[pponnuvel]

Why would we need this? Why do we care if x number of scenarios depended on log messages?

[wilkmar]

Why would we need this? Why do we care if x number of scenarios depended on log messages?
@pponnuvel there is an ongoing activity towards making hotsos checks (or at least some of them) available in COS. One of the ideas is to convert hotsos check that rely on log messages to the Loki alerting rules. As a prereq, I had to understand how many of those log-based checks we have, Therefore I proposed this script.

[pponnuvel]

Why would we need this? Why do we care if x number of scenarios depended on log messages?
@pponnuvel there is an ongoing activity towards making hotsos checks (or at least some of them) available in COS. One of the ideas is to convert hotsos check that rely on log messages to the Loki alerting rules. As a prereq, I had to understand how many of those log-based checks we have, Therefore I proposed this script.

Ah, I see. Thanks!

[dosaboy]

@wilkmar im happy to get this landed if we can improve a couple of things; firstly you need to add this to tox.ini pyfiles e.g.

diff --git a/tox.ini b/tox.ini
index b4ac51c76..5e7d7146b 100644
--- a/tox.ini
+++ b/tox.ini
@@ -31,7 +31,8 @@ passenv =
 pyfiles =
     {toxinidir}/hotsos \
     {[testenv]unit_tests} \
-    {toxinidir}/tools/validation
+    {toxinidir}/tools/validation \
+    {toxinidir}/tools/misc
 setenv = 
     HOTSOS_ROOT={toxinidir}/hotsos
     PYTHONHASHSEED=0

it will raise quite a lot of linting errors that need to be fixed.

Secondly, we now have three areas of code that are discovering scenario files but in different ways and I would like to explore how we can unify them.

• https://github.com/canonical/hotsos/blob/main/tools/validation/hotyvalidate.py
• https://github.com/canonical/hotsos/blob/main/hotsos/core/ycheck/scenarios.py
• https://github.com/canonical/hotsos/blob/main/tests/unit/utils.py#L45