Framework - Extension/ directory¶
This directory contains C source files to implement Python bindings to
internal subroutines of crash
and GDB
.
In addition, its Makefile is used to build the binary distribution of
PyKdump - a single self-sufficient file mpykdump.so that can be
loaded as an extension in the crash
environment.
Directory Structure¶
configure - configuration script to generate extra Makefiles
epython.c - initialization and invoking programs
functions.c - generic functions and bindings to crash internals
gdbspec.c - bindings to GDB internals
Makefile - main Makefile
makestdlib.py - create and package a subset of Python Standard Library
minpylib-3.N.lst - list of files from Python 3.N Standard Library to include
pyconf.py - getting info about Python used for builds
pykdump.h - main header
pyparsing.py - a third-party Python module to implement parsers
Setup.local-3.N - used when you build Python from sources
testmod/ - a test DLKM, used for framework testing
writeREADME.py - generates README with the contents of mpykdump.so
Dependency on Python and Crash Versions¶
Usually there is no need to change anything when a new minor version
of crash
or Python
is released. But when there is a new
major release of either crash
or Python
, it is quite
possible that the C-module will need some changes.
Dependency on crash
Version¶
There are four types of changes in crash
per se that might need changes
in the C-module. In addition, we depend on the GDB
emebedded in crash
.
Generating Makefiles during Configuration¶
Before running the make
command to build the extension, you need to
run the configure
script to generate Makefiles.
In particular, we need to get the crash
version (to be used in reports
and checks), GDB
version, and target. The crash
version used to be
specified directly in its Makefile until recently:
VERSION=7.2.8
but after crash
project migration to Github, it is not there anymore:
VERSION=
As a result, the pyconf.py
script that was used to extract
version/target from this Makefile needed to be modified, so now we rely
on build_data.c:
char *build_command = "crash";
char *build_data = "Tue Sep 1 08:12:55 EDT 2020 by uid=1000(alexs) on zbook";
char *build_target = "X86_64";
char *build_version = "7.2.8";
char *compiler_version = "gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0";
But it is always possible that in the future it will change again.
Internal subroutines/variables/macros¶
The C-module depends on several internal crash
subroutines, variables,
and macros, providing wrappers (usually called Python bindings) for
them so that these subroutines can be accessed from Python. Examples of such
subroutines/macros:
symbol_exists(symbol)
MEMBER_SIZE(name, member)
Some constants:
KVADDR/UVADDR/PHYSADDR
PAGESIZE/PAGE_CACHE_SHIFT
When a new version of crash
is released, it is possible that some
of these subroutines change or become unavailable (this has happened at least once).
An example: recent versions of crash-7 have a set_error() subroutine that can be used to redirect/suppress internal crash/GDB errors and messages.
This subroutine is used in the PyKdump C-module. As a result, if you try to load mpykdump.so with crash as shipped on RHEL7, you will see:
extend: /usr/local/lib/mpykdump64.so: undefined symbol: set_error
To workaround this, we can use weak symbols (a feature of GCC), like this:
// Weak symbols as needed for compatibility with older versions of crash
extern FILE * set_error(char *target) __attribute__ ((weak));
<snip>
// Python bindings to crash internal subroutine set_error()
// You provide a single argument - a string with target name, and
// subroutine returns old target name
static PyObject *
py_crash_set_error(PyObject *self, PyObject *pyargs) {
char *target;
PyObject *rc;
// If set_error() is unavailable, do nothing and return None
if (!set_error) {
Py_INCREF(Py_None);
return Py_None;
}
The logic is as follows: if 'set_error' cannot be resolved when loading the .so, it will be NULL. After that, we check whether it is NULL and if yes, return None without doing anything. In more complex cases, we might use our own subroutine instead, or print a warning.
Signal Handlers and Executing crash
Commands¶
crash
has its own signal handlers for several signals,
e.g. SIGINT. When we execute Python code, Python has its own signal
handlers. To make everything work properly, we need to save/restore
signal handlers when executing Python.
PyKdump provides several commands to execute a crash
built-in command and
return the result as a string. The logic is rather complicated:
We need to feed a string as a command line for
crash
to execute, modify file descriptors to retrieve output, and after command completion do some cleanup.While executing the
crash
builtin, we need to use its own signal handler, and after that install the Python signal handler again.
Dependency on GDB¶
crash
is built on top of GDB
, and to access symbolic
information (such as struct/union definitions) we need to execute
internal GDB
subroutines. New major releases of crash
are
usually rebased on a newer major GDB
version.
As a result, some enumeration definitions (used by GDB
) can change
(this has happened twice), subroutine signatures can change, and GDB
cleanup/error processing can change.
So if PyKdump built on top of a new major crash
does not work
properly, be ready to look not only in crash
sources but GDB
sources as well (provided with patches in the crash
tarfile).
Dependency on Python Version¶
When there is a new major release of Python, three things might need changing:
The contents of Setup.local used to build Python from sources
The list of Python Standard Library subroutines to be included
The way to initialize the Python environment and execute Python code
Setup.local and minpylib-3.N.lst¶
These two files are often updated together. If you decide to include another module from the Python Standard Library, this often (but not always) requires linking statically another C-module included in the Python sources distribution. This has nothing to do with a Python major version change.
But it is not unusual that a new major release of Python rearranges the library, so that you will need to change the contents of minpylib-3.N.lst to make things work. This happened e.g. while migrating from Python-3.6 to Python-3.7.
At this time, the official PyKdump releases are built using Python 3.9. Some code for Python 3.10 enablement has been included, but this is not yet complete.
Python Initialization and Code Execution¶
The Python environment is initialized only once, while loading the extension. Initialization subroutines - part of the Python C-API - are regularly improved and older ones are sometimes obsoleted. This means that we might need to modify the logic of the _init_python() subroutine (defined in epython.c).
There are two sources of PyKdump Python code to execute:
from real files (either user-developed programs or local GIT repo)
from the ZIP-file part of the binary mpykdump.so module
Once again, new major releases of Python regularly improve the existing C-API subroutines, while some old ones are being obsoleted.