Assuming you have a core dump generated by a containerized Agent (with
the c_core_dump
Agent option enabled), load the core dump with GDB with the provided Dockerfile.
The image built from the Dockerfile includes:
- the debug symbols of the Agent (datadog-agent-dbg), for the version you're debugging
- the GDB scripts for Go and Python (only tested with Python 3)
The image is based on the datadog/agent
image for the version of the Agent you're debugging, so that the debug symbols of the system libraries are at the correct version.
- Build the image, passing as build argument the Agent version that generated the core dump. Example:
docker build -t gdb-agent:7.18.0 --build-arg AGENT_VERSION=7.18.0 .
For Agent 7.26.0 or earlier, if the Debian snapshot repos definitions in the Agent image do not actually reflect the debian repo snapshots
that were used at the time of the original Agent image build, the optional build arg DEBIAN_REPO_SNAPSHOT_DATE
must be set to the snapshot's date.
Example: for 7.25.1
, which was built on 2021-01-26
, you have to specify --build-arg DEBIAN_REPO_SNAPSHOT_DATE=20210126T000000Z
.
Optional build args which configure how to source the datadog-agent-dbg
package:
APT_REPO
: APT repo (default:apt.datadoghq.com
)APT_DISTRIBUTION
: APT distro (default:stable
)
Example: for 7.18.0-rc.2
use --build-arg APT_REPO=apt.datad0g.com --build-arg APT_DISTRIBUTION=beta
.
- Navigate the core dump file in a gdb prompt:
docker run -it -v /path/to/core_dump_file:/root/core gdb-agent:7.18.0 /root/core
To use go-delve instead of gdb:
docker run -it -v /path/to/core_dump_file:/root/core --entrypoint /root/go/bin/dlv \
gdb-agent:7.18.0 core /opt/datadog-agent/bin/agent/agent /root/core
You now have access to a regular gdb prompt, with the Python GDB extension. Example of commands:
info threads # list threads
thread <ID> # switch to thread
bt # show stacktrace for current thread
py-bt # show python stacktrace (if selected thread is a python thread and has python frames)
See https://devguide.python.org/gdb/#gdb-7-and-later for more info on the Python-specific commands.
- add cpython sources