quality-info

This is a repository for a variety of scripts to collect information about the quality of debuginfo generated by compilers. The tooling works with RPM (https://rpm.org/) and spack (https://spack.io/) packages. The bin directory holds the scripts used to collect the information. The data directory holds the collected data.

#Setup

The following list of command tries to make the set up of the tooling as easy as possible. However, be aware that lots of diskspace is needed to store the dozens of variant compiles.

sudo dnf install git -y
git clone https://github.com/wcohen/quality_info.git
export PATH=~/quality_info/bin/:/usr/lib64/openmpi/bin:$PATH
sudo ./quality_info/bin/spack_rpms
./quality_info/bin/spack_prep
. spack/share/spack/setup-env.sh

Building spack package variants

Below is an example of how to build the various variants of the spack mfem package (https://spack.readthedocs.io/en/latest/package_list.html#mfem) with a shared library:

. spack/share/spack/setup-env.sh
cd spack
gen_spack_variants mfem+shared

Analyze spack variant builds

Once the all the variants of the spack package are built there are some commands to analyze the various binaries.

The following command will provide very simple comma separated data:

$ analyze_spack_variants  mfem+shared /lib/libmfem.so.4.1

An example line is like the one below. The version of fedora (fc32), the compiler (gcc), the optimization level (-O2), Link Time Optimization (lto), no split dwarf (no split between the commas, on x86_64. The next number is the number of unique breakpoints in the source code, 16914. Note that for inlined functions and LTO the same line could have multiple breakpoints. This analysis is not counting those duplicates. The last entry is a sum of the variables available at each of those breakpoints (100965).

fc32,gcc,o2,lto,d4,,x86_64,16914,100965




$ analyze_spack_abi mfem+shared /lib/libmfem.so.4.1

The analyze_spack_abi command above will create a file libmfem.so.4.1_spack_abi_file_abidiff.txt. This will be output from abidiff showin the differences between the baseline gcc version compiled with -O2 and default -g and the other variant build. Before each abidiff output there is a line starting with a '#' indicating the variant being compared to the baseline.

$ analyze_spack_lines mfem+shared /lib/libmfem.so.4.1

The analyze_spack_lines command above will create a file libmfem.so.4.1_spack_lines_file.txt. This analysis generates lists of every filename:lineno:column information that has executable code associated with it. It compares each of those lists with the baseline gcc variant compiled with -O2 and -g and reports any differences.

Build RPM variants

Below is an example building variants of the binutils RPM that can be used for analysis:

dnf download --source binutils
rpm -Uvh ./binutils*src.rpm
cd ~/rpmbuild/SPECS
gen_rpm_variants binutils.spec

Analyze RPM variants

The commands for rpm versions are fairly similar to the spack versions.

analyze_rpm_lines binutils.spec /usr/bin/ld

The command above will do the same comparison as analyze_spack_lines but for the ld command in the bintuils RPM package. The commands should be similar to the spack versions substituting "rpm" for "spack" in the commands