diff --git a/.github/workflows/litevm.yml b/.github/workflows/litevm.yml
index 15be1162..8d642dce 100644
--- a/.github/workflows/litevm.yml
+++ b/.github/workflows/litevm.yml
@@ -19,7 +19,7 @@ jobs:
- uses: actions/setup-python@v4
name: Set up Python
with:
- python-version: '3.x'
+ python-version: '3.12'
- name: Install pre-commit
run: pip install pre-commit
- name: Run pre-commit hooks
@@ -62,7 +62,7 @@ jobs:
- name: Build and install drgn with CTF support
run: |
cd ..
- git clone https://github.com/brenns10/drgn -b ctf_0.0.32
+ git clone https://github.com/brenns10/drgn -b ctf_0.0.33
cd drgn
../drgn-tools/venv/bin/pip install .
- name: Run tests
diff --git a/README.md b/README.md
index 0dbb23a4..3eab8c2c 100644
--- a/README.md
+++ b/README.md
@@ -1,77 +1,92 @@
# drgn-tools
-drgn-tools is a library of helpers for use with [drgn][drgn]. It contains
-helpers with a slightly reduced scope than what drgn itself can contain.
-
-The main target for these helpers is the Oracle UEK kernel. Helpers may contain
-code that assumes a UEK configuration and UEK architectures. This makes a lower
-bar for acceptance than drgn, where helpers should be as configuration and
-architecture agnostic as possible. In general, helpers should be contributed to
-drgn itself, unless there is an Oracle UEK-specific reason to keep them here.
-
-In addition to the helper functions, drgn-tools contains some helper utilities
-that we find useful:
-
-1. The `DRGN` script which can automatically download, extract, and load
- debuginfo for UEK kernels.
-2. The `corelens` tool, which contains a library of modules that can extract
- information from a live kernel or vmcore, and write it to a sosreport-style
- directory for later analysis.
-
-Please note, each drgn-tools version is only supported with a corresponding
-version of Drgn.
-
-See the [documentation][doc] for more information on how to use these tools and
-how to contribute to them.
+drgn-tools is Oracle's set of helpers and tools based on [drgn][drgn], including
+Corelens, our tool for generating summary reports of a vmcore or live system.
+
+The main target for drgn-tools is the Oracle UEK kernel. Thus, it may assume
+kernels with UEK configurations and architectures. This is different from drgn,
+which aims to support a much broader set of configurations and architectures.
+Where possible, we aim to contribute to drgn upstream first.
+
+In addition to helper functions, drgn-tools contains some tools that we find
+useful:
+
+1. The `drgn_tools.cli` script which provides a CLI similar to drgn's, with our
+ helpers imported.
+2. The `corelens` tool (`drgn_tools.corelens`), which contains a library of
+ modules that can extract information from a live kernel or vmcore, and write
+ it to a sosreport-style directory for later analysis. You can learn more
+ about Corelens in [this article][blog-corelens].
+
+Drgn-tools also contains logic that can help automatically fetch & extract
+Oracle Linux kernel debuginfo, as well as configure and load Compact Type Format
+(CTF) for supported UEK kernels. This makes using tools like Corelens and the
+CLI seamless on OL, frequently not even requiring installation of debuginfo
+packages. You can learn more about CTF in [this article][blog-ctf].
## Getting Started
-Requires Python 3.6 or later, and an Linux system (preferably Oracle Linux 8 or
-later). For this guide, we'll assume you have a core dump (vmcore).
+Oracle Linux users should consult the Oracle Linux Documentation section
+entitled "Installing drgn-tools" for the most up-to-date instructions: [OL 8][],
+[OL 9][], [OL 10][]. To summarize, enable the Add-ons channel, and then install
+the `drgn-tools` package. For example:
-1. Install Drgn, if you haven't already: `pip install drgn`. Alternatively, `yum
- install drgn`, or use your system's package manager, if appropriate.
-2. Clone the repository: `git clone
- https://github.com/oracle-samples/drgn-tools`
+``` sh
+dnf config-manager --enable ol10_addons
+dnf install drgn-tools
+```
-That's it! See below for ways to use drgn-tools.
+For other users, or those interested in running from source: Drgn-tools requires
+Python 3.6 or later, and drgn 0.0.32 or later. These can be installed from your
+OS package manager preferably, or via pip, uv, etc. Once installed, you can
+clone `drgn-tools` with git and start running against your kernel.
## Documentation
-You can find documentation for the helpers, as well as contributing guide and
-guide to using our tools, [here](https://oracle-samples.github.io/drgn-tools/).
+Documentation for usage of drgn & drgn-tools can be found in the Oracle Linux
+Documentation section entitled ["Debugging the Kernel with Drgn and
+Corelens"][ol10doc].
-## Examples
+You can find some automatically generated documentation of the helpers, as well
+as contributing guide and guide to using our tools,
+[here](https://oracle-samples.github.io/drgn-tools/). Please note that this site
+is not always fully up-to-date, and content here reflects internal
+implementation details not supported by Oracle.
-One of the benefits of using drgn-tools, in addition to the added UEK-specific
-helpers, is the ability to fetch debuginfo directly from the Oracle debuginfo
-Yum server. To enable this, you should put the following contents in
-`~/.config/drgn_tools.ini`:
-
-``` ini
-[debuginfo]
-fetchers = OracleLinuxYumFetcher
-```
+## Examples
-With that, you can use the drgn-tools CLI with:
+Use Corelens to generate a report of the running kernel. Output is written to a
+directory named "output":
``` sh
-python -m drgn_tools.cli VMCORE
+corelens /proc/kcore -a -o output
```
-To run it against the running kernel, use:
+The above command uses the Corelens binary installed to the system. If you have
+cloned the drgn-tools source code, you could instead run this command from the
+git repository root:
``` sh
-python -m drgn_tools.cli /proc/kcore
+python -m drgn_tools.corelens /proc/kcore -a -o output
```
-Use the drgn-tools Corelens system (which outputs a range of information from
-several diagnostic systems):
+One of the benefits of using drgn-tools, in addition to the added UEK-specific
+helpers, is the ability to fetch debuginfo directly from the Oracle debuginfo
+Yum server. Corelens and the drgn-tools CLI enable this automatically, but when
+drgn-tools is installed, you can use it with drgn too:
``` sh
-python -m drgn_tools.corelens VMCORE
+# Use CTF (on Oracle Linux with UEK)
+drgn -c /proc/kcore --try-symbols-by=ctf
+
+# Download and extract debuginfo RPMs for Oracle Linux
+drgn -c /proc/kcore --try-symbols-by=ol-download
```
+Downloaded debuginfo is stored in a directory `~/vmlinux_repo` by default. You
+can configure this and many other aspects of debuginfo finding and loading in
+`/etc/drgn_tools.ini` or `~/.config/drgn_tools.ini`.
+
## Help
If you're having trouble using drgn-tools or its helpers, please create a Github
@@ -96,10 +111,17 @@ vulnerability disclosure process.
## License
-Copyright (c) 2023 Oracle and/or its affiliates.
+Copyright (c) 2023-2025 Oracle and/or its affiliates.
Released under the Universal Permissive License v1.0 as shown at
.
[drgn]: https://drgn.readthedocs.io
[doc]: https://oracle-samples.github.io/drgn-tools/
+[OL 8]: https://docs.oracle.com/en/operating-systems/oracle-linux/8/drgn/installing_drgn_tools.html
+[OL 9]: https://docs.oracle.com/en/operating-systems/oracle-linux/9/drgn/installing_drgn_tools.html
+[OL 10]: https://docs.oracle.com/en/operating-systems/oracle-linux/10/drgn/installing_drgn_tools.html
+[drgn plugin]:https://drgn.readthedocs.io/en/latest/advanced_usage.html#writing-plugins
+[blog-corelens]: https://blogs.oracle.com/linux/corelens-a-microscope-for-your-vmcores
+[blog-ctf]: https://blogs.oracle.com/linux/introducing-ctf-support-in-drgn-for-oracle-linux
+[ol10doc]: https://docs.oracle.com/en/operating-systems/oracle-linux/10/drgn/about_drgn_and_corelens.html
diff --git a/buildrpm/python-drgn-tools.spec b/buildrpm/python-drgn-tools.spec
index 7ac28cfc..4b299973 100644
--- a/buildrpm/python-drgn-tools.spec
+++ b/buildrpm/python-drgn-tools.spec
@@ -20,7 +20,7 @@
Name: python-drgn-tools
-Version: 2.1.0
+Version: 2.2.0
Release: 1%{?dist}
Summary: Helper scripts for drgn, containing the corelens utility
@@ -59,7 +59,7 @@ a running kernel image (via /proc/kcore).}
# The drgn dependency can be fulfilled by drgn with, or without, CTF support.
# However, drgn-tools is tied to specific drgn releases.
%global drgn_min 0.0.32
-%global drgn_max 0.0.33
+%global drgn_max 0.0.34
%package -n drgn-tools
Summary: %{summary}
@@ -135,6 +135,27 @@ rm %{buildroot}/usr/bin/DRGN
%endif
%changelog
+* Tue Nov 04 2025 Stephen Brennan - 2.2.0-1
+- Rework drgn-tools debuginfo loading to be based on drgn's Module API (Stephen Brennan)
+- Create "oracle" drgn plugin which encapsulates drgn-tools debuginfo logic (Stephen Brennan)
+- New corelens module: "pstack" for printing userspace stack traces (Stephen Brennan)
+- Corelens module: equivalent to oled memstate [Orabug: 37357348] (Yassine Larhrissi)
+- Corelens fails accessing rds_ib_devices [Orabug: 37502613] (Stephen Brennan)
+- Print ioeventfd, iobus, vmstat and vcpustat information in kvm corelens module. [Orabug: 37713468] (Siddhi Katage)
+- crash in corelens rds module [Orabug: 38225228] (Stephen Brennan)
+- test_dump_page_cache_pages_pinning_cgroups produces too much output [Orabug: 37974100] (Stephen Brennan)
+- Test failure for module_build_id() in Linux 6.14 [Orabug: 37973187] (Stephen Brennan)
+- False negatives in module debuginfo detection [Orabug: 37894875] (Stephen Brennan)
+- Make md helper not crash with uninitialized percpu refcount [Orabug: 37968889] (Junxiao Bi)
+- UEK8, drgn-tools-2.1.0-1.el9.noarch : Error with corelens binary when run with /proc/kcore or vmcore [Orabug: 37894852] (Stephen Brennan)
+- drgn-tools-2.1.0-1.el9: python traceback when ctrl-c done with corelens command [Orabug: 37894865] (Stephen Brennan)
+- Mountinfo fails on a (nearly) empty struct mount [Orabug: 37911508] (Stephen Brennan)
+- lockup: detect the blocker for process hang in RCU grace period [Orabug: 37899681] (Richard Li)
+- Add vectorinfo module to drgn_tools [Orabug: 38383772] (Srivathsa Dara)
+- corelens: dump panic bt [Orabug: 38074929] (Richard Li)
+- Enhance rds helper to extract rdma resources and RDS QP state [Orabug: 38221449] (Anand Khoje)
+- Add sosreport module for collecting corelens reports (Anil Palakunnathu Kunnengeri)
+
* Thu Apr 17 2025 Stephen Brennan - 2.1.0-1
- Add helper and module for unsubmitted pending work (Imran Khan)
- Add -V option to display version, and include the version in corelens reports (Stephen Brennan) [Orabug: 37503503]
diff --git a/drgn_tools.ini b/drgn_tools.ini
index b618ebcb..ebec6eaa 100644
--- a/drgn_tools.ini
+++ b/drgn_tools.ini
@@ -10,6 +10,18 @@
# These configurations affect the behavior of the Oracle plugin and its debug
# info finders. They specify paths and how the finders behave. However, they do
# not control *which* finders get enabled or when.
+#
+# Paths here will be formatted using Python's str.format() function, with the
+# following string keys available:
+#
+# - bits: 32 or 64 depending on the architecture of the kernel
+# - rpm: the name of the debuginfo RPM for this kernel version
+# - ol_version, olver: the "major version" of Oracle Linux for that kernel
+# - arch: the architecture string from the kernel release string
+# - uname: the kernel release string
+# - vmcore_path: the original path of the core dump
+#
+# Further string keys can be defined using "expansions", see that section below.
# Controls the path where drgn-tools searches for DWARF debuginfo, and where it
# would extract the files from a debuginfo RPM. This path does not need to
@@ -71,3 +83,40 @@
# ol-local-rpm, and ol-download, as well as drgn's "standard" finder.
#
# disable_dwarf = false
+
+[extractions]
+
+# Extractions are an uncommonly used feature, which add flexibility for
+# constructing paths. The string configurations above, such as vmlinux_repo,
+# rpm_path_format, and urls, each can specify an important path for finding
+# debuginfo. They are able to use Python formatting strings, using pre-defined
+# fields related to the kernel version and vmcore path.
+#
+# However, in some cases the path can't be determined just from the kernel
+# version or vmcore_path. For example, one requirement might be storing the
+# debuginfo in a subdirectory adjacent to the vmcore. For such a requirement,
+# you could use an expansion to achieve it.
+#
+# Expansions define a new field, and they must be of the form:
+#
+# new_field = old_field:default_value:regex
+#
+# Where:
+# old_field: the name of the field you're extracting text from
+# default_value: the value to use for the new_field if old_field doesn't
+# exist or the regex does not match
+# regex: a regular expression pattern. Group number 1 from this regex will
+# be used for the new_field value.
+#
+# For example, the following extracts the directory containing the vmcore to a
+# new field named vmcore_dir. If for any reason the extraction could not work,
+# we fall back to "/var/tmp/" for safety:
+#
+# [extractions]
+# vmcore_dir = vmcore_path:/var/tmp/:^(.*/)[^/]+$
+#
+# With this, you can specify that debuginfo should be extracted adjacent to the
+# vmcore, or even in the same directory:
+#
+# vmlinux_repo = {vmcore_dir}/debuginfo
+# vmlinux_repo = {vmcore_dir}
diff --git a/drgn_tools/block.py b/drgn_tools/block.py
index c986b056..b0bc3c9f 100644
--- a/drgn_tools/block.py
+++ b/drgn_tools/block.py
@@ -20,6 +20,7 @@
from drgn.helpers.linux.device import MAJOR
from drgn.helpers.linux.device import MINOR
from drgn.helpers.linux.list import list_for_each_entry
+from drgn.helpers.linux.timekeeping import ktime_get_coarse_ns
from drgn.helpers.linux.xarray import xa_for_each
from drgn_tools.bitops import for_each_bit_set
@@ -337,7 +338,7 @@ def rq_pending_time_ns(rq: Object) -> int:
if has_member(rq, "start_time"):
return (prog["jiffies"] - rq.start_time).value_() * 1000000
elif has_member(rq, "start_time_ns"):
- base = prog["tk_core"].timekeeper.tkr_mono.base
+ base = ktime_get_coarse_ns(prog)
delta = base - rq.start_time_ns
return delta.value_() if base > rq.start_time_ns else 0
else:
diff --git a/drgn_tools/cli.py b/drgn_tools/cli.py
index d4c7c3e0..7ba472d0 100644
--- a/drgn_tools/cli.py
+++ b/drgn_tools/cli.py
@@ -110,6 +110,7 @@ def main() -> None:
prog = Program()
prog.cache["drgn_tools.debuginfo.options"] = opts
+ prog.cache["drgn_tools.debuginfo.vmcore_path"] = args.vmcore
try:
prog.set_core_dump(args.vmcore)
except PermissionError:
@@ -144,8 +145,10 @@ def main() -> None:
if prog.cache.get("using_ctf"):
db_kind = "CTF"
+ db_file = prog.cache.get("ctf_file", "(unknown)")
else:
- db_kind = f"DWARF: {prog.main_module().debug_file_path}"
+ db_kind = "DWARF"
+ db_file = prog.main_module().debug_file_path
def banner_func(banner: str) -> str:
header = version_header()
@@ -154,7 +157,7 @@ def banner_func(banner: str) -> str:
imports = "\n"
for mod_name, names in CLI_HELPERS.items():
imports += f">>> from {mod_name} import {', '.join(names)}\n"
- db_info = f"Using {db_kind}"
+ db_info = f"Using {db_kind}: {db_file}"
db_info += "\n" + str(get_module_load_summary(prog))
return (
header
diff --git a/drgn_tools/corelens.py b/drgn_tools/corelens.py
index c5e00d75..eb134879 100644
--- a/drgn_tools/corelens.py
+++ b/drgn_tools/corelens.py
@@ -36,6 +36,7 @@
from drgn_tools.logging import FilterMissingDebugSymbolsMessages
from drgn_tools.module import get_module_load_summary
from drgn_tools.util import redirect_stdout
+from drgn_tools.util import redirectable
log = logging.getLogger("corelens")
@@ -360,6 +361,7 @@ def _load_prog_and_debuginfo(args: argparse.Namespace) -> Program:
prog = Program()
prog.cache["drgn_tools.debuginfo.options"] = opts
+ prog.cache["drgn_tools.debuginfo.vmcore_path"] = args.vmcore
try:
prog.set_core_dump(args.vmcore)
except PermissionError:
@@ -729,8 +731,13 @@ def _do_main() -> None:
load_time = time.time() - start_time
log.info("%s", _version_string())
- kind = "CTF" if prog.cache.get("using_ctf") else "DWARF"
- log.info("Loaded %s debuginfo in in %.03fs", kind, load_time)
+ if prog.cache.get("using_ctf"):
+ kind = "CTF"
+ db_file = prog.cache.get("ctf_file", "(unknown)")
+ else:
+ kind = "DWARF"
+ db_file = prog.main_module().debug_file_path
+ log.info("Loaded %s debuginfo (%s) in in %.03fs", kind, db_file, load_time)
log.debug(
"Enabled debuginfo finders: %r", prog.enabled_debug_info_finders()
)
@@ -765,7 +772,8 @@ def main() -> None:
pass
-def run(prog: Program, cl_cmd: str) -> None:
+@redirectable
+def run(prog: Program, cl_cmd: str, outfile: Optional[str] = None) -> None:
"""
Run a single corelens command
@@ -775,6 +783,7 @@ def run(prog: Program, cl_cmd: str) -> None:
against ``prog``.
:param cl_cmd: command string to execute
+ :param outfile: filename to redirect output to (see @redirectable)
"""
cmd = shlex.split(cl_cmd)
module_name, args = cmd[0], cmd[1:]
@@ -800,8 +809,8 @@ def make_runner(prog: Program) -> Callable[[str], None]:
argument. This is intended for interactive environments.
"""
- def cl(cl_cmd: str) -> None:
- return run(prog, cl_cmd)
+ def cl(cl_cmd: str, outfile: Optional[str] = None) -> None:
+ return run(prog, cl_cmd, outfile=outfile)
return cl
diff --git a/drgn_tools/debuginfo.py b/drgn_tools/debuginfo.py
index 3a339bda..a08ad0bb 100644
--- a/drgn_tools/debuginfo.py
+++ b/drgn_tools/debuginfo.py
@@ -20,6 +20,7 @@
import re
import shutil
import subprocess
+import sys
import tempfile
from pathlib import Path
from typing import Dict
@@ -263,15 +264,37 @@ def is_vmlinux(module: Module) -> bool:
)
-def find_debug_info_vmlinux_repo(repo_dir: Path, modules: List[Module]):
+def find_debug_info_vmlinux_repo(
+ repo_dir: Path, modules: List[Module], extracted: Set[str]
+):
+ """
+ Helper function for loading debuginfo out of a standard vmlinux_repo dir
+
+ Of particular importance is the "extracted" parameter. This keeps track of
+ all modules for which there is an existing file in the vmlinux_repo, which
+ we have tried to load. If drgn failed to load it, there's likely a build ID
+ mismatch, which means that it would be useless to re-download or re-extract
+ the file from the RPM.
+
+ :repo_dir: the full path to the directory for this kernel version
+ :modules: the list of drgn modules we need debuginfo for
+ :extracted: set of module names which is updated with each file we encounter
+ """
for module in modules:
if not module.wants_debug_file():
continue
if is_vmlinux(module):
module.try_file(repo_dir / "vmlinux")
elif module_is_in_tree(module):
- filename = f"{module.name.replace('-', '_')}.ko.debug"
- module.try_file(repo_dir / filename)
+ file = repo_dir / f"{module.name.replace('-', '_')}.ko.debug"
+ if file.exists():
+ module.try_file(file)
+ extracted.add(module.name)
+ if module.wants_debug_file():
+ log.warning(
+ "module %s has a debuginfo file but drgn rejected it -- likely a build ID mismatch",
+ module.name,
+ )
class DebugInfoOptionsExt:
@@ -293,6 +316,9 @@ class DebugInfoOptionsExt:
ctf_file: Optional[str]
dwarf_dir: Optional[str]
+ # A list of field extractions that will be included into the path expansions
+ extractions: List[Tuple[str, str, str, "re.Pattern"]]
+
def __init__(
self,
repo_paths: List[str],
@@ -305,6 +331,7 @@ def __init__(
disable_dwarf: bool = False,
ctf_file: Optional[str] = None,
dwarf_dir: Optional[str] = None,
+ extractions: Optional[List[Tuple[str, str, str, "re.Pattern"]]] = None,
) -> None:
self.repo_paths = repo_paths
self.local_path = local_path
@@ -318,6 +345,7 @@ def __init__(
self.ctf_file = ctf_file
self.dwarf_dir = dwarf_dir
+ self.extractions = extractions or []
def _get_host_ol() -> Optional[int]:
@@ -384,6 +412,7 @@ class OracleDebuginfo:
version: KernelVersion
extracted: Set[str]
cached_rpm: Optional[Path]
+ _fmtparams: Optional[Dict[str, str]]
def __init__(self, opts: DebugInfoOptionsExt, prog: Program):
self.opts = opts
@@ -392,6 +421,50 @@ def __init__(self, opts: DebugInfoOptionsExt, prog: Program):
self.version = KernelVersion.parse(uname)
self.extracted = set()
self.cached_rpm = None
+ self.warned_mismatch = False
+ self._fmtparams = None
+
+ @property
+ def fmtparams(self) -> Dict[str, str]:
+ if self._fmtparams is None:
+ self._fmtparams = self.version.format_params()
+ # TODO: Drgn 0.0.33 introduces core_dump_path, which may contain the
+ # file path of the core dump if it was available to drgn. For prior
+ # drgn versions, we can retrieve it from a custom cache, which we
+ # set in corelens and the drgn-tools cli. The drgn CLI does not set
+ # this cache entry, however, so this won't work in all cases.
+ vmcore_path = getattr(
+ self.prog,
+ "core_dump_path",
+ self.prog.cache.get("drgn_tools.debuginfo.vmcore_path"),
+ )
+ if vmcore_path:
+ self._fmtparams["vmcore_path"] = os.path.abspath(vmcore_path)
+
+ # The "extractions" feature allows specifying some format parameter
+ # to get created based on a value extracted from another. This is
+ # useful for, e.g., determining the location of vmlinux_repo based
+ # on the path of the vmcore or some part of its version.
+ for extracted, from_field, default, expr in self.opts.extractions:
+ self._fmtparams[extracted] = default
+ if from_field in self._fmtparams:
+ value = self._fmtparams[from_field]
+ m = expr.match(value)
+ if m:
+ self._fmtparams[extracted] = m.group(1)
+ else:
+ log.warning(
+ "error: for field %s %r, expr didn't match: %r",
+ from_field,
+ value,
+ expr.pattern,
+ )
+ else:
+ log.warning(
+ "error: expansion exists for field %s but the field isn't present",
+ from_field,
+ )
+ return self._fmtparams
def ol_vmlinux_repo_finder(self, modules: List[Module]) -> None:
# We would like to run this unconditionally regardless of whether any of
@@ -411,12 +484,43 @@ def ol_vmlinux_repo_finder(self, modules: List[Module]) -> None:
)
modules[0].build_id = None
- fmtparams = self.version.format_params()
for repo_format in self.opts.repo_paths:
- repo_dir = Path(repo_format.format(**fmtparams))
+ repo_dir = Path(repo_format.format(**self.fmtparams))
if repo_dir.is_dir():
log.debug("ol-vmlinux-repo: loading from %s", repo_dir)
- find_debug_info_vmlinux_repo(repo_dir, modules)
+ find_debug_info_vmlinux_repo(repo_dir, modules, self.extracted)
+
+ def check_installed_debuginfo(self, modules: List[Module]) -> bool:
+ """
+ Check whether we should skip loading modules from other finders because
+ we have an installed debuginfo RPM.
+
+ If we get to the point where there are apparent in-tree modules which do
+ not have debuginfo, then normally our finders would download or extract
+ debuginfo for them. But if the debuginfo RPM is installed, then drgn has
+ certainly already tried to load all the in-tree modules from the
+ installed debuginfo. Thus, there's likely a build ID mismatch issue, and
+ there's no point in our downloading and/or extracting the debuginfo again.
+
+ Print a warning in this case, and return True to signal to the download
+ & extraction finders that they should not bother to continue.
+ """
+ if not modules:
+ return False
+ vmlinux_path = (
+ f"/usr/lib/debug/lib/modules/{self.version.original}/vmlinux"
+ )
+ if not os.path.exists(vmlinux_path):
+ return False
+ if not self.warned_mismatch:
+ modnames = ", ".join(m.name for m in modules)
+ log.warning(
+ "debuginfo RPM is installed, yet the following in-tree modules"
+ " failed to load (ksplice cold-patch?): %s",
+ modnames,
+ )
+ self.warned_mismatch = True
+ return True
def ol_local_rpm_finder(self, modules: List[Module]) -> None:
# The local RPM finder must extract to a directory: the vmlinux repo. We
@@ -427,12 +531,11 @@ def ol_local_rpm_finder(self, modules: List[Module]) -> None:
if not self.opts.repo_paths:
log.debug("ol-local-rpm: no vmlinux repo to extract to, exiting")
return
- fmtparams = self.version.format_params()
- dest_dir = Path(self.opts.repo_paths[-1].format(**fmtparams))
+ dest_dir = Path(self.opts.repo_paths[-1].format(**self.fmtparams))
if self.cached_rpm and self.cached_rpm.exists():
source_rpm = self.cached_rpm
else:
- source_rpm = Path(self.opts.local_path.format(**fmtparams))
+ source_rpm = Path(self.opts.local_path.format(**self.fmtparams))
if not source_rpm.exists():
log.debug("ol-local-rpm: local RPM is missing: %s", source_rpm)
@@ -452,6 +555,8 @@ def ol_local_rpm_finder(self, modules: List[Module]) -> None:
"ol-local-rpm: no vmlinux/in-tree modules need debug info, exiting"
)
return
+ if self.check_installed_debuginfo(mods_needing_debuginfo):
+ return
modnames = [m.name for m in mods_needing_debuginfo]
extract_rpm(
@@ -461,8 +566,9 @@ def ol_local_rpm_finder(self, modules: List[Module]) -> None:
permissions=0o777,
caller="ol-local-rpm: ",
)
- find_debug_info_vmlinux_repo(dest_dir, mods_needing_debuginfo)
- self.extracted.update(modnames)
+ find_debug_info_vmlinux_repo(
+ dest_dir, mods_needing_debuginfo, self.extracted
+ )
def _delete_cached_rpm(self):
if self.cached_rpm and self.cached_rpm.exists():
@@ -478,9 +584,8 @@ def ol_download_finder(self, modules: List[Module]) -> None:
log.debug("ol-download: no vmlinux repo to extract to, exiting")
return
- fmtparams = self.version.format_params()
- out_dir = Path(self.opts.repo_paths[-1].format(**fmtparams))
- dest_rpm = Path(self.opts.local_path.format(**fmtparams))
+ out_dir = Path(self.opts.repo_paths[-1].format(**self.fmtparams))
+ dest_rpm = Path(self.opts.local_path.format(**self.fmtparams))
# Normally, ol-local-rpm is enabled whenever ol-download is. But it's
# possible for that not to be the case. In that case, ensure that a
@@ -506,8 +611,10 @@ def ol_download_finder(self, modules: List[Module]) -> None:
"ol-download: no vmlinux/in-tree modules need debug info, exiting"
)
return
+ if self.check_installed_debuginfo(mods_needing_debuginfo):
+ return
- urls = [url_fmt.format(**fmtparams) for url_fmt in self.opts.urls]
+ urls = [url_fmt.format(**self.fmtparams) for url_fmt in self.opts.urls]
tmp = tempfile.NamedTemporaryFile(
suffix=".rpm", mode="wb", delete=False
)
@@ -553,8 +660,9 @@ def ol_download_finder(self, modules: List[Module]) -> None:
modnames = [m.name for m in mods_needing_debuginfo]
out_dir.mkdir(parents=True, exist_ok=True)
extract_rpm(path, out_dir, modnames, caller="ol-download: ")
- find_debug_info_vmlinux_repo(out_dir, mods_needing_debuginfo)
- self.extracted.update(modnames)
+ find_debug_info_vmlinux_repo(
+ out_dir, mods_needing_debuginfo, self.extracted
+ )
def ctf_finder(self, modules: List["Module"]):
ctf_loaded = self.prog.cache.get("using_ctf", False)
@@ -570,10 +678,9 @@ def ctf_finder(self, modules: List["Module"]):
# Internal systems may have a `vmlinux.ctfa` file in the normal vmlinux
# repo path.
if self.opts.repo_paths:
- fmtparams = self.version.format_params()
ctf_paths.append(
os.path.join(
- self.opts.repo_paths[-1].format(**fmtparams),
+ self.opts.repo_paths[-1].format(**self.fmtparams),
"vmlinux.ctfa",
)
)
@@ -588,6 +695,7 @@ def ctf_finder(self, modules: List["Module"]):
):
load_ctf(self.prog, path)
self.prog.cache["using_ctf"] = True
+ self.prog.cache["ctf_file"] = path
ctf_loaded = True
module.debug_file_status = ModuleFileStatus.DONT_NEED
log.info("ctf: loaded %s", path)
@@ -643,6 +751,17 @@ def get_debuginfo_config() -> DebugInfoOptionsExt:
else:
url_list = ["https://oss.oracle.com/ol{olver}/debuginfo/{rpm}"]
+ # Extractions might be used to help determine format parameters for the
+ # paths & URLs above
+ extractions = []
+ if config.has_section("extractions"):
+ for extracted_field, value in config["extractions"].items():
+ from_field, default, expression = value.split(":")
+ default = os.path.expanduser(default)
+ extractions.append(
+ (extracted_field, from_field, default, re.compile(expression))
+ )
+
def getbool(name, default):
truthy = ("t", "true", "1", "y", "yes")
strval = config.get("debuginfo", name, fallback=default).lower()
@@ -664,6 +783,7 @@ def getbool(name, default):
enable_extract=enable_extract,
enable_ctf=enable_ctf,
disable_dwarf=disable_dwarf,
+ extractions=extractions,
)
@@ -740,13 +860,14 @@ def extract_rpm(
permissions: Optional[int] = None,
caller: Optional[str] = None,
) -> Dict[str, Path]:
- log.info(
- "%sextracting %d debuginfo modules (%s) from %s...",
- caller or "",
- len(modules),
- ", ".join(f"{s}" for s in modules[:3])
- + ("..." if len(modules) > 3 else ""),
- source_rpm,
+ print(
+ "{}extracting {} debuginfo modules ({}) from {}...".format(
+ caller or "",
+ len(modules),
+ ", ".join(modules[:3]) + ("..." if len(modules) > 3 else ""),
+ source_rpm,
+ ),
+ file=sys.stderr,
)
if not dest_dir.exists():
# Rather than use .mkdir(exist_ok=True), we do the test explicitly here,
diff --git a/drgn_tools/kvm.py b/drgn_tools/kvm.py
index 79bde2d2..9c3ed0f5 100644
--- a/drgn_tools/kvm.py
+++ b/drgn_tools/kvm.py
@@ -473,6 +473,8 @@ class KvmUtil(CorelensModule):
"""
name = "kvm"
+ skip_unless_have_kmods = ["kvm"]
+ debuginfo_kmods = ["kvm-intel", "kvm-amd"]
default_args = [
[
diff --git a/drgn_tools/meminfo.py b/drgn_tools/meminfo.py
index 5c764944..a5462c22 100644
--- a/drgn_tools/meminfo.py
+++ b/drgn_tools/meminfo.py
@@ -512,7 +512,10 @@ def get_all_meminfo(prog: Program) -> Dict[str, int]:
# Since 194df9f66db8d ("mm: remove NR_BOUNCE zone stat") in v6.16, NR_BOUNCE
# is removed from stats and set to zero.
stats["Bounce"] = global_stats.get("NR_BOUNCE", 0)
- stats["WritebackTmp"] = global_stats["NR_WRITEBACK_TEMP"]
+ # Since commit 8356a5a3b078c ("mm, vmstat: remove the NR_WRITEBACK_TEMP
+ # node_stat_item counter") in 6.17, this element is removed and hardcoded
+ # zero.
+ stats["WritebackTmp"] = global_stats.get("NR_WRITEBACK_TEMP", 0)
stats["CommitLimit"] = get_vm_commit_limit(prog)
# ``vm_committed_as`` is a percpu counter object. It has percpu
diff --git a/drgn_tools/memstate.py b/drgn_tools/memstate.py
index a3e1ff53..f8202915 100644
--- a/drgn_tools/memstate.py
+++ b/drgn_tools/memstate.py
@@ -18,6 +18,7 @@
from drgn.helpers.linux.pid import for_each_task
from drgn.helpers.linux.slab import for_each_slab_cache
from drgn.helpers.linux.slab import get_slab_cache_aliases
+from drgn.helpers.linux.timekeeping import ktime_get_real_seconds
from drgn_tools.buddyinfo import get_per_zone_buddyinfo
from drgn_tools.corelens import CorelensModule
@@ -128,11 +129,7 @@ def print_pretty_numastat_kb(str_msg: str, numa_arr_kb: List) -> None:
def get_time(prog: Program) -> str:
- try:
- timekeeper = prog["shadow_timekeeper"]
- except KeyError:
- timekeeper = prog["tk_core"].shadow_timekeeper
- return time.ctime(timekeeper.xtime_sec)
+ return time.ctime(ktime_get_real_seconds(prog).value_())
def memstate_header(prog: Program, print_header: bool = True) -> None:
diff --git a/drgn_tools/module.py b/drgn_tools/module.py
index 136d37d7..9258c4ed 100644
--- a/drgn_tools/module.py
+++ b/drgn_tools/module.py
@@ -6,6 +6,7 @@
from typing import NamedTuple
from typing import Optional
from typing import Tuple
+from typing import TYPE_CHECKING
from drgn import cast
from drgn import FaultError
@@ -18,6 +19,9 @@
from drgn_tools.taint import Taint
+if TYPE_CHECKING:
+ from drgn_tools.debuginfo import OracleDebuginfo
+
__all__ = (
"ParamInfo",
@@ -26,28 +30,18 @@
"get_module_load_summary",
"module_exports",
"module_is_in_tree",
- "module_is_ksplice_cold_patch",
"module_params",
)
-def module_is_ksplice_cold_patch(module: RelocatableModule) -> bool:
- # Normally, ksplice modules are live patches, which are loaded into the
- # kernel and patch the already loaded code. For patched kernel modules,
- # ksplices may also contain a "cold-patched" module which is a new copy of
- # the module with the updated code, avoiding the need to live-patch if the
- # module is not yet loaded. The downside is that these are new build
- # artifacts with different build IDs. The packaged debuginfo does not apply
- # to them, and drgn rightly rejects them.
- return "__tripwire_table" in module.section_addresses
-
-
def module_is_in_tree(module: Module) -> bool:
+ # Note that this will return True for Ksplice "cold patch" modules. We
+ # cannot reliably detect them, but they do not match the shipped debuginfo,
+ # and drgn rightly rejects their debuginfo.
return (
module.prog.flags & ProgramFlags.IS_LINUX_KERNEL
and isinstance(module, RelocatableModule)
and not (module.object.taints & (1 << Taint.OOT_MODULE))
- and not module_is_ksplice_cold_patch(module)
)
@@ -273,9 +267,9 @@ class ModuleLoadSummary(NamedTuple):
total_mods: int
ksplice_mods: List[RelocatableModule]
- ksplice_cold_patch_mods: List[RelocatableModule]
other_oot: List[RelocatableModule]
loaded_mods: List[RelocatableModule]
+ mismatched_mods: List[RelocatableModule]
missing_mods: List[RelocatableModule]
def __str__(self) -> str:
@@ -283,17 +277,17 @@ def __str__(self) -> str:
details = []
if self.ksplice_mods:
details.append(f"{len(self.ksplice_mods)} are ksplices")
- if self.ksplice_cold_patch_mods:
- details.append(
- f"{len(self.ksplice_cold_patch_mods)} are cold-patched "
- "ksplice modules"
- )
if self.other_oot:
details.append(
f"{len(self.other_oot)} are other out-of-tree modules"
)
if self.loaded_mods:
details.append(f"{len(self.loaded_mods)} have debuginfo")
+ if self.mismatched_mods:
+ details.append(
+ f"{len(self.mismatched_mods)} had debuginfo files drgn "
+ "could not load (ksplice cold-patch?)"
+ )
# it's nice to have confirmation, regardless of whether it is 0
details.append(f"{len(self.missing_mods)} are missing debuginfo")
return text + ", ".join(details)
@@ -316,17 +310,20 @@ def add(mods: List[RelocatableModule], kind: str) -> None:
add(self.loaded_mods, "in-tree with debuginfo")
add(self.missing_mods, "in-tree, but missing debuginfo")
+ add(
+ self.mismatched_mods,
+ "in-tree, mismatched debuginfo (ksplice cold-patch?)",
+ )
add(self.ksplice_mods, "ksplice patches")
- add(self.ksplice_cold_patch_mods, "ksplice cold-patched modules")
add(self.other_oot, "other out-of-tree modules")
return "\n".join(lines)
def all_mods(self) -> List[RelocatableModule]:
return (
self.ksplice_mods
- + self.ksplice_cold_patch_mods
+ self.other_oot
+ self.loaded_mods
+ + self.mismatched_mods
+ self.missing_mods
)
@@ -337,31 +334,39 @@ def get_module_load_summary(prog: Program) -> ModuleLoadSummary:
"""
total_mods = 0
ksplice_mods = []
- ksplice_cold_patch_mods = []
other_oot = []
loaded_mods = []
+ mismatched_mods = []
missing_mods = []
using_ctf = prog.cache.get("using_ctf")
+ dbinfo: Optional["OracleDebuginfo"] = prog.cache.get(
+ "drgn_tools.debuginfo"
+ )
+ extracted = set()
+ if dbinfo is not None:
+ extracted = dbinfo.extracted
for mod in prog.modules():
if not isinstance(mod, RelocatableModule):
continue
total_mods += 1
if module_is_in_tree(mod):
- if mod.wants_debug_file() and not using_ctf:
+ if not mod.wants_debug_file():
+ loaded_mods.append(mod)
+ elif using_ctf:
missing_mods.append(mod)
+ elif mod.name in extracted:
+ mismatched_mods.append(mod)
else:
- loaded_mods.append(mod)
+ missing_mods.append(mod)
elif mod.name.startswith("ksplice"):
ksplice_mods.append(mod)
- elif module_is_ksplice_cold_patch(mod):
- ksplice_cold_patch_mods.append(mod)
else:
other_oot.append(mod)
return ModuleLoadSummary(
total_mods,
ksplice_mods,
- ksplice_cold_patch_mods,
other_oot,
loaded_mods,
+ mismatched_mods,
missing_mods,
)
diff --git a/drgn_tools/numastat.py b/drgn_tools/numastat.py
index 50762280..82cfda27 100644
--- a/drgn_tools/numastat.py
+++ b/drgn_tools/numastat.py
@@ -151,7 +151,10 @@ def get_per_node_meminfo(prog: Program, node: Object) -> Dict[str, int]:
if "NR_UNSTABLE_NFS" in node_zone_stats:
mm_stats["NFS_Unstable"] = node_zone_stats["NR_UNSTABLE_NFS"]
mm_stats["Bounce"] = bounce_pages
- mm_stats["WritebackTmp"] = node_zone_stats["NR_WRITEBACK_TEMP"]
+ # Since commit 8356a5a3b078c ("mm, vmstat: remove the NR_WRITEBACK_TEMP
+ # node_stat_item counter") in 6.17, this element is removed and hardcoded
+ # zero.
+ mm_stats["WritebackTmp"] = node_zone_stats.get("NR_WRITEBACK_TEMP", 0)
# Collect transparent hugepage meminfo.
unit = mm_consts["TRANS_HPAGE_UNIT"]
diff --git a/drgn_tools/rds.py b/drgn_tools/rds.py
index 10691ade..00a2259b 100644
--- a/drgn_tools/rds.py
+++ b/drgn_tools/rds.py
@@ -33,11 +33,13 @@
from drgn.helpers.linux.list import list_for_each
from drgn.helpers.linux.list import list_for_each_entry
from drgn.helpers.linux.pid import find_task
+from drgn.helpers.linux.timekeeping import ktime_get_real_seconds
from drgn_tools.corelens import CorelensModule
from drgn_tools.module import ensure_debuginfo
from drgn_tools.table import print_table
from drgn_tools.table import Table
+from drgn_tools.util import redirectable
# Golbal variables and definitions #
@@ -159,7 +161,7 @@ def get_connection_uptime(conn: Object) -> timedelta:
:returns: Conn up time as a string
"""
prog = conn.prog_
- curr_time = prog["tk_core"].timekeeper.xtime_sec
+ curr_time = ktime_get_real_seconds(prog)
conn_restart_time = conn.c_path.cp_reconnect_start
time_since = curr_time - conn_restart_time
return timedelta(seconds=int(time_since))
@@ -509,19 +511,16 @@ def ensure_mlx_core_ib_debuginfo(prog: drgn.Program, dev_name: str) -> bool:
# RDS Corelens module functions #
+@redirectable
def rds_dev_info(
prog: drgn.Program,
ret: bool = False,
- outfile: Optional[str] = None,
- report: bool = False,
) -> Optional[List[Object]]:
"""
Print the IB device info
:param prog: drgn program
:param ret: If true the function returns the ``struct rds_ib_device`` list and None if the arg is false
- :param outfile: A file to write the output to.
- :param report: Open the file in append mode. Used to generate a report of all the functions in the rds module.
:returns: A List of ``struct rds_ib_device`` or None
"""
@@ -565,7 +564,7 @@ def rds_dev_info(
[index, rds_ib_device, ib_device, dev_name, node_name, ip_str]
)
- print_table(info, outfile, report)
+ print_table(info)
if ret:
return rds_ib_dev_list
@@ -573,15 +572,11 @@ def rds_dev_info(
return None
-def rdma_resource_usage(
- prog: Program, outfile: Optional[str] = None, report: bool = False
-) -> None:
+@redirectable
+def rdma_resource_usage(prog: Program) -> None:
"""
- Print RDMA restrack resource usage counts for ALL mlx5_* devices, similar to 'rdma res show'
-
- :param prog: drgn.Program
- :param outfile: A file to write the output to.
- :param report: Whether to open file in append mode for report.
+ Print RDMA restrack resource usage counts for ALL mlx5_* devices, similar to
+ 'rdma res show'
"""
dev_kset = prog["devices_kset"]
data = [["Index", "Device", "PD", "CQ", "QP", "CM_ID", "MR", "CTX", "SRQ"]]
@@ -627,22 +622,16 @@ def fmt(val):
except Exception:
continue
- print_table(data, outfile, report)
+ print_table(data)
-def rds_stats(
- prog: drgn.Program,
- fields: Optional[str] = None,
- outfile: Optional[str] = None,
- report: bool = False,
-) -> None:
+@redirectable
+def rds_stats(prog: drgn.Program, fields: Optional[str] = None) -> None:
"""
Print the RDS stats and counters.
:param prog: drgn program
:param fields: List of comma separated fields to print. It also supports substring matching for the fields provided. Ex: 'conn_reset, ib_tasklet_call, send, ...'
- :param outfile: A file to write the output to.
- :param report: Open the file in append mode. Used to generate a report of all the functions in the rds module.
:returns: None
"""
msg = ensure_debuginfo(prog, ["rds"])
@@ -659,9 +648,10 @@ def rds_stats(
rds_stats.extend(rds_get_stats(prog, "rds_stats", fields_list))
rds_stats.extend(rds_get_stats(prog, "rds_ib_stats", fields_list))
- print_table(rds_stats, outfile, report)
+ print_table(rds_stats)
+@redirectable
def rds_conn_info(
prog: drgn.Program,
laddr: Optional[str] = None,
@@ -669,8 +659,6 @@ def rds_conn_info(
tos: Optional[str] = None,
state: Optional[str] = None,
ret: bool = False,
- outfile: Optional[str] = None,
- report: bool = False,
) -> Optional[List[Object]]:
"""
Display all RDS connections
@@ -681,8 +669,6 @@ def rds_conn_info(
:param tos: comma separated string list of TOS. Ex: '0, 3, ...'
:param state: comma separated string list of conn states. Ex 'RDS_CONN_UP, CONNECTING, ...'
:param ret: If true the function returns the ``struct rds_ib_connection`` list and None if the arg is false
- :param outfile: A file to write the output to.
- :param report: Open the file in append mode. Used to generate a report of all the functions in the rds module.
:returns: A List of ``struct rds_ib_connection`` that match the filters provided or None
"""
msg = ensure_debuginfo(prog, ["rds"])
@@ -768,7 +754,7 @@ def rds_conn_info(
]
)
- print_table(conn_list, outfile, report)
+ print_table(conn_list)
if ret:
return ib_conn_list
@@ -863,6 +849,7 @@ def rds_ib_conn_ring_info(
print_table(ring_info)
+@redirectable
def rds_info_verbose(
prog: drgn.Program,
laddr: Optional[str] = None,
@@ -870,8 +857,6 @@ def rds_info_verbose(
tos: Optional[str] = None,
fields: Optional[str] = None,
ret: bool = False,
- outfile: Optional[str] = None,
- report: bool = False,
) -> Optional[List[Object]]:
"""
Print the rds conn stats similar to rds-info -Iv
@@ -882,8 +867,6 @@ def rds_info_verbose(
:param tos: comma separated string list of TOS. Ex: '0, 3, ...'
:param fields: List of comma separated fields to display. It also supports substring matching for the fields provided. Ex: 'Recv_alloc_ctr, Cache Allocs, Tx, ...'
:param ret: If true the function returns the ``struct rds_ib_connection`` list and None if the arg is false
- :param outfile: A file to write the output to.
- :param report: Open the file in append mode. Used to generate a report of all the functions in the rds module.
:returns: A List of ``struct rds_ib_connection`` that match the filters provided or None
"""
msg = ensure_debuginfo(prog, ["rds"])
@@ -1101,7 +1084,7 @@ def rds_info_verbose(
for col in conn_info:
del col[index]
- print_table(conn_info, outfile, report)
+ print_table(conn_info)
if ret:
return ics
@@ -1109,11 +1092,8 @@ def rds_info_verbose(
return None
-def rds_conn_cq_eq_info(
- prog: drgn.Program,
- outfile: Optional[str] = None,
- report: bool = False,
-) -> None:
+@redirectable
+def rds_conn_cq_eq_info(prog: drgn.Program) -> None:
"""
Display CQ and EQ info per RDS IB connection in a table format
"""
@@ -1135,8 +1115,6 @@ def rds_conn_cq_eq_info(
"RCQ_EQNo",
"RCQ_EQ_ptr",
],
- outfile=outfile,
- report=report,
)
for dev in for_each_rds_ib_device(prog):
@@ -1183,19 +1161,16 @@ def rds_conn_cq_eq_info(
table.write()
+@redirectable
def rds_sock_info(
prog: drgn.Program,
ret: bool = False,
- outfile: Optional[str] = None,
- report: bool = False,
) -> Optional[List[Object]]:
"""
Print the rds socket info similar to rds-tools -k
:param prog: drgn program
:param ret: If true the function returns the ``struct rds_sock`` list and None if the arg is false
- :param outfile: A file to write the output to.
- :param report: Open the file in append mode. Used to generate a report of all the functions in the rds module.
:returns: A List of ``struct rds_sock`` or None
"""
msg = ensure_debuginfo(prog, ["rds"])
@@ -1268,7 +1243,7 @@ def rds_sock_info(
comm,
]
)
- print_table(fields, outfile, report)
+ print_table(fields)
if ret:
return sock_list
@@ -1276,6 +1251,7 @@ def rds_sock_info(
return None
+@redirectable
def rds_print_recv_msg_queue(
prog: drgn.Program,
laddr: Optional[str] = None,
@@ -1284,8 +1260,6 @@ def rds_print_recv_msg_queue(
lport: Optional[str] = None,
rport: Optional[str] = None,
ret: Optional[bool] = False,
- outfile: Optional[str] = None,
- report: bool = False,
) -> None:
"""
Print the rds recv msg queue similar rds-info -r
@@ -1296,8 +1270,6 @@ def rds_print_recv_msg_queue(
:param tos: comma separated string list of TOS. Ex: '0, 3, ...'
:param lport: comma separated string list of lport. Ex: 2259, 36554, ...'
:param rport: comma separated string list of rport. Ex: 2259, 36554, ...'
- :param outfile: A file to write the output to.
- :param report: Open the file in append mode. Used to generate a report of all the functions in the rds module.
:returns: None
"""
@@ -1314,8 +1286,6 @@ def rds_print_recv_msg_queue(
"Seq",
"Bytes",
],
- outfile=outfile,
- report=report,
)
if laddr:
@@ -1373,6 +1343,7 @@ def rds_print_recv_msg_queue(
return None
+@redirectable
def rds_print_send_retrans_msg_queue(
prog: drgn.Program,
queue: str,
@@ -1382,8 +1353,6 @@ def rds_print_send_retrans_msg_queue(
lport: Optional[str] = None,
rport: Optional[str] = None,
ret: Optional[bool] = False,
- outfile: Optional[str] = None,
- report: bool = False,
) -> None:
"""
Print the rds send or retransmit msg queue similar rds-info -st
@@ -1395,8 +1364,6 @@ def rds_print_send_retrans_msg_queue(
:param tos: comma separated string list of TOS. Ex: '0, 3, ...'
:param lport: comma separated string list of lport. Ex: 2259, 36554, ...'
:param rport: comma separated string list of rport. Ex: 2259, 36554, ...'
- :param outfile: A file to write the output to.
- :param report: Open the file in append mode. Used to generate a report of all the functions in the rds module.
:returns: None
"""
@@ -1412,8 +1379,6 @@ def rds_print_send_retrans_msg_queue(
"Seq",
"Bytes",
],
- outfile=outfile,
- report=report,
)
if laddr:
@@ -1484,6 +1449,7 @@ def rds_print_send_retrans_msg_queue(
return None
+@redirectable
def rds_print_msg_queue(
prog: drgn.Program,
queue: str = "All",
@@ -1493,8 +1459,6 @@ def rds_print_msg_queue(
lport: Optional[str] = None,
rport: Optional[str] = None,
ret: Optional[bool] = False,
- outfile: Optional[str] = None,
- report: bool = False,
) -> None:
"""
Print the rds msg queue similar rds-info -srt
@@ -1506,8 +1470,6 @@ def rds_print_msg_queue(
:param tos: comma separated string list of TOS. Ex: '0, 3, ...'
:param lport: comma separated string list of lport. Ex: 2259, 36554, ...'
:param rport: comma separated string list of rport. Ex: 2259, 36554, ...'
- :param outfile: A file to write the output to.
- :param report: Open the file in append mode. Used to generate a report of all the functions in the rds module.
:returns: None
"""
@@ -1519,14 +1481,21 @@ def rds_print_msg_queue(
queue = queue.lower()
if queue not in ("all", "send", "snd", "retrans", "re", "recv", "rcv"):
print(
- f"Unknown queue type '{queue}'. Expected: all, send, retrans, or recv"
+ f"Unknown queue type '{queue}'. Expected: all, send, retrans, or recv",
)
return
if queue in ("send", "snd", "all"):
rds_print_send_retrans_msg_queue(
- prog, "send", laddr, raddr, tos, lport, rport, ret, outfile, report
+ prog,
+ "send",
+ laddr,
+ raddr,
+ tos,
+ lport,
+ rport,
+ ret,
)
- print("\n")
+ print()
if queue in ("retrans", "re", "all"):
rds_print_send_retrans_msg_queue(
prog,
@@ -1537,13 +1506,17 @@ def rds_print_msg_queue(
lport,
rport,
ret,
- outfile,
- report,
)
- print("\n")
+ print()
if queue in ("recv", "rcv", "all"):
rds_print_recv_msg_queue(
- prog, laddr, raddr, tos, lport, rport, ret, outfile, report
+ prog,
+ laddr,
+ raddr,
+ tos,
+ lport,
+ rport,
+ ret,
)
@@ -1579,8 +1552,6 @@ def print_mr_list_head_info(
"length",
"pd",
],
- outfile=None,
- report=False,
)
for list_ptr in list_for_each(list_head.address_of_()):
@@ -1658,13 +1629,13 @@ def rds_get_mr_list_info(
)
-def report(prog: drgn.Program, outfile: Optional[str] = None) -> None:
+@redirectable
+def report(prog: drgn.Program) -> None:
"""
Generate a report of RDS related data.
This functions runs all the functions in the module and saves the results to the output file provided.
:param prog: drgn.Program
- :param outfile: A file to write the output to.
:returns: None
"""
msg = ensure_debuginfo(prog, ["rds"])
@@ -1672,14 +1643,14 @@ def report(prog: drgn.Program, outfile: Optional[str] = None) -> None:
print(msg)
return None
- rds_dev_info(prog, outfile=outfile, report=False)
- rdma_resource_usage(prog, outfile=outfile, report=False)
- rds_sock_info(prog, outfile=outfile, report=True)
- rds_conn_info(prog, outfile=outfile, report=True)
- rds_info_verbose(prog, outfile=outfile, report=True)
- rds_conn_cq_eq_info(prog, outfile, report=True)
- rds_stats(prog, outfile=outfile, report=True)
- rds_print_msg_queue(prog, queue="All", outfile=outfile, report=True)
+ # rds_dev_info(prog)
+ # rdma_resource_usage(prog)
+ rds_sock_info(prog)
+ rds_conn_info(prog)
+ rds_info_verbose(prog)
+ rds_conn_cq_eq_info(prog)
+ rds_stats(prog)
+ rds_print_msg_queue(prog, queue="All")
class Rds(CorelensModule):
diff --git a/drgn_tools/sys.py b/drgn_tools/sys.py
index de447dc3..c8e0e172 100644
--- a/drgn_tools/sys.py
+++ b/drgn_tools/sys.py
@@ -15,6 +15,8 @@
from drgn.helpers.linux import for_each_task
from drgn.helpers.linux.sched import loadavg
from drgn.helpers.linux.sched import task_state_to_char
+from drgn.helpers.linux.timekeeping import ktime_get_boottime_seconds
+from drgn.helpers.linux.timekeeping import ktime_get_real_seconds
from drgn_tools.corelens import CorelensModule
from drgn_tools.cpuinfo import aarch64_get_cpu_info
@@ -81,14 +83,10 @@ def get_sysinfo(prog: Program) -> Dict[str, Any]:
uts = prog["init_uts_ns"]
else:
raise Exception("error: could not find utsname information")
- try:
- timekeeper = prog["shadow_timekeeper"]
- except KeyError:
- # 20c7b582e88b8 ("timekeeping: Move shadow_timekeeper into tk_core")
- # Starting in v6.13
- timekeeper = prog["tk_core"].shadow_timekeeper
- date = time.ctime(timekeeper.xtime_sec)
- uptime = str(datetime.timedelta(seconds=int(timekeeper.ktime_sec)))
+ date = time.ctime(ktime_get_real_seconds(prog).value_())
+ uptime = str(
+ datetime.timedelta(seconds=ktime_get_boottime_seconds(prog).value_())
+ )
jiffies = int(prog["jiffies"])
nodename = uts.name.nodename.string_().decode("utf-8")
release = uts.name.release.string_().decode("utf-8")
diff --git a/drgn_tools/table.py b/drgn_tools/table.py
index d9070f61..162c0836 100644
--- a/drgn_tools/table.py
+++ b/drgn_tools/table.py
@@ -1,6 +1,5 @@
# Copyright (c) 2023, Oracle and/or its affiliates.
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/
-import sys
from typing import Any
from typing import Dict
from typing import Iterable
@@ -8,11 +7,7 @@
from typing import Optional
-def print_table(
- fields: List[List[Any]],
- outfile: Optional[str] = None,
- report: bool = False,
-) -> None:
+def print_table(fields: List[List[Any]]) -> None:
"""
Print a given nested list as table, given that the first list is the column headers.
@@ -26,25 +21,14 @@ def print_table(
for col in range(len(fields[0])):
col_widths.append(max(len(str(row[col])) for row in fields))
- out = sys.stdout
- if outfile and report:
- out = open(outfile, "a")
- print("\n", file=out)
- elif outfile:
- out = open(outfile, "w")
-
for entry in fields:
print(
"".join(
str(val).ljust(col_width + 2)
for (val, col_width) in zip(entry, col_widths)
).rstrip(),
- file=out,
)
- if outfile:
- out.close()
-
class Table:
"""
@@ -75,16 +59,9 @@ class Table:
:class:`FixedTable`.
:param header: a list of column specifiers, see above for details
- :param outfile: optional output file name (default is stdout)
- :param report: when true, outfile is opened in append mode
"""
- def __init__(
- self,
- header: List[str],
- outfile: Optional[str] = None,
- report: bool = False,
- ):
+ def __init__(self, header: List[str]):
# Name of each header
self.header = []
# Function (str, int) -> str to justify each column entry
@@ -106,13 +83,6 @@ def __init__(
self.formats.append(fmt)
self.widths = [len(h) for h in header]
self.rows: List[List[str]] = []
- self.out = sys.stdout
- self.close_output = bool(outfile)
- if outfile and report:
- self.out = open(outfile, "a")
- self.out.write("\n\n")
- elif outfile:
- self.out = open(outfile, "w")
def _build_row(
self, fields: Iterable[Any], update_widths: bool = True
@@ -144,9 +114,9 @@ def _row_str(self, row: List[str]) -> str:
def write(self) -> None:
"""Print the table to the output file"""
- print(self._row_str(self.header), file=self.out)
+ print(self._row_str(self.header))
for row in self.rows:
- print(self._row_str(row), file=self.out)
+ print(self._row_str(row))
class FixedTable(Table):
@@ -190,7 +160,7 @@ def __init__(
if widths:
self.widths = widths
self.print_immediately = True
- print(self._row_str(self.header), file=self.out)
+ print(self._row_str(self.header))
def add_row(self, fields: Iterable[Any]) -> None:
"""Add a row to the table (it is immediately printed)"""
@@ -200,48 +170,29 @@ def add_row(self, fields: Iterable[Any]) -> None:
if not self.print_immediately:
self.print_immediately = True
row = self._build_row(fields, update_widths=True)
- print(self._row_str(self.header), file=self.out)
+ print(self._row_str(self.header))
else:
row = self._build_row(fields, update_widths=False)
- print(self._row_str(row), file=self.out)
+ print(self._row_str(row))
def write(self) -> None:
"""Signals that no more rows will be added."""
if not self.print_immediately:
# The header was never printed. Do it now, since we are expected to
# print a blank table.
- print(self._row_str(self.header), file=self.out)
+ print(self._row_str(self.header))
-def print_dictionary(
- dictionary: Dict[str, Any],
- outfile: Optional[str] = None,
- report: bool = False,
-) -> None:
+def print_dictionary(dictionary: Dict[str, Any]) -> None:
"""
Align and print the data
:param dictionary: dictionary to print
- :param outfile: A file to write the output to.
- :param report: Open the file in append mode.
:returns: None
"""
lcol_length = 10
for title in dictionary:
lcol_length = max(len(title), lcol_length)
- out = sys.stdout
- if outfile and report:
- out = open(outfile, "a")
- print("\n", file=out)
- elif outfile:
- out = open(outfile, "w")
-
for title in dictionary:
- print(
- f"{title.ljust(lcol_length)}: {dictionary[title]}",
- file=out,
- )
-
- if outfile:
- out.close()
+ print(f"{title.ljust(lcol_length)}: {dictionary[title]}")
diff --git a/drgn_tools/util.py b/drgn_tools/util.py
index dbdd6fbf..64e171a1 100644
--- a/drgn_tools/util.py
+++ b/drgn_tools/util.py
@@ -1,6 +1,7 @@
# Copyright (c) 2023, Oracle and/or its affiliates.
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/
import argparse
+import contextlib
import logging
import re
import sys
@@ -8,6 +9,7 @@
import typing as t
from contextlib import contextmanager
from enum import IntEnum
+from functools import wraps
from urllib.error import HTTPError
from urllib.request import Request
from urllib.request import urlopen
@@ -567,3 +569,40 @@ def __call__(
for element in value.split(","):
result.append(self.element_type(element))
setattr(namespace, self.dest, result)
+
+
+F = t.TypeVar("F", bound=t.Callable[..., t.Any])
+
+
+def redirectable(f: F) -> F:
+ """
+ A decorator which allows any function to be redirected to stdout
+
+ Any function wrapped with this decorator can be called with an optional
+ string parameter "outfile", which specifies a filename that can be used to
+ redirect stdout. By default, the filename is opened in write mode
+ (truncating any former contents). But an explicit mode ""
+ """
+
+ @wraps(f)
+ def inner(*args, **kwargs):
+ outfile = kwargs.pop("outfile", None)
+ mode = "w"
+ # :w or :a can be explicitly specified at the end of the filename
+ if outfile and outfile[-2:] == ":a":
+ outfile = outfile[:-2]
+ mode = "a"
+ elif outfile and outfile[-2:] == ":w":
+ mode = "w"
+ outfile = outfile[:-2]
+ with contextlib.ExitStack() as es:
+ # Optionally redirect stdout
+ if outfile:
+ es.enter_context(
+ contextlib.redirect_stdout(
+ es.enter_context(open(outfile, mode))
+ )
+ )
+ return f(*args, **kwargs)
+
+ return t.cast(F, inner)
diff --git a/setup.py b/setup.py
index c353790d..8e9b7e00 100644
--- a/setup.py
+++ b/setup.py
@@ -9,7 +9,7 @@
long_description = "drgn helper script repository"
-RELEASE_VERSION = "2.1.0"
+RELEASE_VERSION = "2.2.0"
PACKAGES = ["drgn_tools"]
@@ -88,7 +88,7 @@ def get_version():
description="drgn helper script repository",
long_description=long_description,
install_requires=[
- "drgn>=0.0.32,<0.0.33",
+ "drgn>=0.0.32,<0.0.34",
],
url="https://github.com/oracle-samples/drgn-tools",
author="Oracle Linux Sustaining Engineering Team",
@@ -105,5 +105,6 @@ def get_version():
"DRGN=drgn_tools.cli:main",
"corelens=drgn_tools.corelens:main",
],
+ "drgn.plugins": ["oracle=drgn_tools.debuginfo"],
},
)
diff --git a/tests/conftest.py b/tests/conftest.py
index 34eca901..9b69f932 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -172,8 +172,6 @@ def pytest_configure(config):
f"vmcore {vmcore}" if vmcore else f"live {os.uname().release}",
debuginfo_kind,
)
- if CTF:
- print("TESTING WITH CTF")
global PROG
PROG = drgn.Program()
@@ -188,6 +186,17 @@ def pytest_configure(config):
global KVER
KVER = KernelVersion.parse(PROG["UTS_RELEASE"].string_().decode())
+
+ print(
+ "BEGIN {} {} TEST: {} (Python {}.{}, drgn {})".format(
+ "CTF" if CTF else "DWARF",
+ "VMCORE" if VMCORE else "LIVE",
+ f"{vmcore} {KVER.original}" if VMCORE else KVER.original,
+ sys.version_info[0],
+ sys.version_info[1],
+ getattr(drgn, "__version__", "unknown"),
+ )
+ )
if CTF:
try:
from drgn.helpers.linux.ctf import load_ctf
diff --git a/tests/test_pstack.py b/tests/test_pstack.py
index 85f84b91..03afd2ca 100644
--- a/tests/test_pstack.py
+++ b/tests/test_pstack.py
@@ -3,6 +3,7 @@
import argparse
import gzip
import sys
+import time
from subprocess import PIPE
from subprocess import Popen
@@ -34,6 +35,8 @@ def sleeping_proc():
while b"ready" not in data:
data.extend(proc.stdout.read(1))
try:
+ # Give it some time to settle.
+ time.sleep(0.1)
yield proc
finally:
proc.terminate()
diff --git a/tests/test_vectorinfo.py b/tests/test_vectorinfo.py
index ba21c5c0..df745ec9 100644
--- a/tests/test_vectorinfo.py
+++ b/tests/test_vectorinfo.py
@@ -1,8 +1,14 @@
# Copyright (c) 2025, Oracle and/or its affiliates.
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/
+import pytest
+
from drgn_tools import vectorinfo
-def test_vectorinfo(prog):
+def test_vectorinfo(prog, kver):
+ if kver.arch != "x86_64":
+ pytest.skip("Only x86_64 is supported")
+ if kver.uek_version is not None and kver.uek_version < 6:
+ pytest.skip("UEK6 or later is required")
vectorinfo.print_vector_matrix(prog)
vectorinfo.print_vectors(prog, True)