Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

report the apt cache timestamp #182

Merged
merged 1 commit into from
Oct 14, 2023

Conversation

anarcat
Copy link
Contributor

@anarcat anarcat commented Oct 13, 2023

We use the pkgcache.bin modification time as a heuristic, but there might be a better way to do this.

We also silently ignore errors when pulling that file to avoid broken systems from breaking the other metrics. I believe this will lead to a null metric which is probably what we want anyway.

A possible improvement to this would be to individually inspect the InRelease files and report the mirror's timestamps as well, but that's more involved. It's also not a priority compared to this fix, because we want the update timestamp if we remove the apt update run (in #181).

Closes: #180

apt_info.py Outdated Show resolved Hide resolved
Copy link
Contributor

@SuperQ SuperQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor naming nit, otherwise, LGTM.

We use the `pkgcache.bin` modification time as a heuristic, but there
might be a better way to do this.

We also silently ignore errors when pulling that file to avoid broken
systems from breaking the other metrics. I believe this will lead to a
null metric which is probably what we want anyway.

A possible improvement to this would be to individually inspect the
`InRelease` files and report the mirror's timestamps as well, but
that's more involved. It's also not a priority compared to this fix,
because we want the update timestamp if we remove the `apt update`
run (in prometheus-community#181).

Closes: prometheus-community#180

Signed-off-by: Antoine Beaupré <[email protected]>
Co-authored-by: Ben Kochie <[email protected]>
@anarcat anarcat force-pushed the apt-cache-timestamp branch from 2ba296a to dc6568e Compare October 14, 2023 02:32
@anarcat
Copy link
Contributor Author

anarcat commented Oct 14, 2023

merged your suggestion in the commit, along with a credit in commitlog, should be ready to merge now.

@anarcat anarcat requested a review from SuperQ October 14, 2023 02:33
@SuperQ SuperQ merged commit b579cf5 into prometheus-community:master Oct 14, 2023
3 checks passed
@julian-klode
Copy link

This has all a bit too fast for me to actually give any feedback (I'm afk all week except this morning more or less), but pkgcache.bin will also get updated whenever you install or upgrade or remove a package. srcpkgcache.bin is rebuilt whenever the sources.list changes, you ran clean, or the lists actually changed.

The best approach arguably is to check /var/lib/apt/periodic/update-success-stamp to see when the last succesful update was - but it is only set by the periodic script, so that only makes sense if the option is set (querying that with apt_pkg.init_config(); apt_pkg.config["APT::Periodic::Update-Package-Lists"] is nicest if you want to use python3-apt, but could do apt-config get too).

The best proxy otherwise is just the mtime of the /var/lib/apt/lists directory as we always rename() files from the partial directory into there, so it should always be updated as far as I understand directory modification times. The modification times of the files meanwhile are the modification times on the server and hence not meaningful.

This is a common issue that other tools also have and APT itself will need to deal with, I think we're going to end up adding actual stamp files to the apt code itself so you can always tell when the last update was and whether there were errors. Maybe just dump a (machine-readable) update.log in there, once we have machine-readable error codes.

@anarcat anarcat deleted the apt-cache-timestamp branch October 17, 2023 14:23
@anarcat
Copy link
Contributor Author

anarcat commented Oct 17, 2023

great feedback @julian-klode - i've opened #183 to tweak this properly.

anarcat added a commit to anarcat/node-exporter-textfile-collector-scripts that referenced this pull request Oct 17, 2023
As reported in prometheus-community#182, `pkgcache.bin` gets updated on more than just
`apt update`. In particular, it gets modified when a package is
installed, upgraded or removed.

So let's fallback on a better heuristic, which is the `lists`
directory, which gets updated when new mirror lists gets moved into
place. This does run the risk of staying silently unchanged if there's
no change on mirrors but (a) that's rather infrequent and (b) we might
actually want to warn on that anyway.
anarcat added a commit to anarcat/node-exporter-textfile-collector-scripts that referenced this pull request Oct 17, 2023
As reported in prometheus-community#182, `pkgcache.bin` gets updated on more than just
`apt update`. In particular, it gets modified when a package is
installed, upgraded or removed.

So let's fallback on a better heuristic, which is the `APT::Periodic`
timestamp. This should give us a much more precise and deliberate
status.

We also fallback to the `lists` directory, which gets updated when new
mirror lists gets moved into place. This does run the risk of staying
silently unchanged if there's no change on mirrors but (a) that's
rather infrequent and (b) we might actually want to warn on that
anyway.

Signed-off-by: Antoine Beaupré <[email protected]>
anarcat added a commit to anarcat/node-exporter-textfile-collector-scripts that referenced this pull request Oct 17, 2023
As reported in prometheus-community#182, `pkgcache.bin` gets updated on more than just
`apt update`. In particular, it gets modified when a package is
installed, upgraded or removed.

So let's fallback on a better heuristic, which is the `APT::Periodic`
timestamp. This should give us a much more precise and deliberate
status.

We also fallback to the `lists` directory, which gets updated when new
mirror lists gets moved into place. This does run the risk of staying
silently unchanged if there's no change on mirrors but (a) that's
rather infrequent and (b) we might actually want to warn on that
anyway.

Signed-off-by: Antoine Beaupré <[email protected]>
Closes: prometheus-community#183
anarcat added a commit to anarcat/node-exporter-textfile-collector-scripts that referenced this pull request Oct 17, 2023
As reported in prometheus-community#182, `pkgcache.bin` gets updated on more than just
`apt update`. In particular, it gets modified when a package is
installed, upgraded or removed.

So let's fallback on a better heuristic, which is the `APT::Periodic`
timestamp. This should give us a much more precise and deliberate
status.

We also fallback to the `lists` directory, which gets updated when new
mirror lists gets moved into place. This does run the risk of staying
silently unchanged if there's no change on mirrors but (a) that's
rather infrequent and (b) we might actually want to warn on that
anyway.

Signed-off-by: Antoine Beaupré <[email protected]>
Closes: prometheus-community#183
anarcat added a commit to anarcat/node-exporter-textfile-collector-scripts that referenced this pull request Oct 17, 2023
As reported in prometheus-community#182, `pkgcache.bin` gets updated on more than just
`apt update`. In particular, it gets modified when a package is
installed, upgraded or removed.

So let's fallback on a better heuristic, which is the `APT::Periodic`
timestamp. This should give us a much more precise and deliberate
status.

We also fallback to the `lists` directory, which gets updated when new
mirror lists gets moved into place. This does run the risk of staying
silently unchanged if there's no change on mirrors but (a) that's
rather infrequent and (b) we might actually want to warn on that
anyway.

Signed-off-by: Antoine Beaupré <[email protected]>
Closes: prometheus-community#183
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

apt info should report cache age
3 participants