Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FTL is stuck in a crash loop #2140

Open
rvankraaij opened this issue Dec 22, 2024 · 5 comments · Fixed by #2141
Open

FTL is stuck in a crash loop #2140

rvankraaij opened this issue Dec 22, 2024 · 5 comments · Fixed by #2141
Labels
Bug in development Bug Bugfix in progress Pi-hole v6.0

Comments

@rvankraaij
Copy link

FTL enters a crash loop and fails to start. This is the log output:

2024-12-22 14:22:32.051 CET [4061M] INFO: ########## FTL started on raspberrypi! ##########
2024-12-22 14:22:32.052 CET [4061M] INFO: FTL branch: development
2024-12-22 14:22:32.053 CET [4061M] INFO: FTL version: vDev-c04fa58
2024-12-22 14:22:32.053 CET [4061M] INFO: FTL commit: c04fa58
2024-12-22 14:22:32.053 CET [4061M] INFO: FTL date: 2024-12-22 08:39:41 +0100
2024-12-22 14:22:32.053 CET [4061M] INFO: FTL user: pihole
2024-12-22 14:22:32.053 CET [4061M] INFO: Compiled for linux/arm/v6 (compiled on CI) using cc (Alpine 14.2.0) 14.2.0
2024-12-22 14:22:32.075 CET [4061M] INFO: Wrote config file:
2024-12-22 14:22:32.076 CET [4061M] INFO: - 150 total entries
2024-12-22 14:22:32.076 CET [4061M] INFO: - 134 entries are default
2024-12-22 14:22:32.076 CET [4061M] INFO: - 16 entries are modified
2024-12-22 14:22:32.076 CET [4061M] INFO: - 0 entries are forced through environment
2024-12-22 14:22:32.086 CET [4061M] INFO: Parsed config file /etc/pihole/pihole.toml successfully
2024-12-22 14:22:32.109 CET [4061M] INFO: PID of FTL process: 4061
2024-12-22 14:22:32.113 CET [4061M] INFO: listening on 0.0.0.0 port 53
2024-12-22 14:22:32.113 CET [4061M] INFO: listening on :: port 53
2024-12-22 14:22:32.116 CET [4061M] INFO: PID of FTL process: 4061
2024-12-22 14:22:32.122 CET [4061M] INFO: Database version is 21
2024-12-22 14:22:32.124 CET [4061M] INFO: Database successfully initialized
2024-12-22 14:22:36.535 CET [4061M] INFO: Imported 31889 queries from the on-disk database (it has 3652521 rows)
2024-12-22 14:22:36.536 CET [4061M] INFO: Parsing queries in database
2024-12-22 14:22:36.922 CET [4061M] INFO: 10000 queries parsed...
2024-12-22 14:22:37.281 CET [4061M] INFO: 20000 queries parsed...
2024-12-22 14:22:37.636 CET [4061M] INFO: 30000 queries parsed...
2024-12-22 14:22:37.707 CET [4061M] INFO: Imported 31889 queries from the long-term database
2024-12-22 14:22:37.707 CET [4061M] INFO: -> Total DNS queries: 31889
2024-12-22 14:22:37.707 CET [4061M] INFO: -> Cached DNS queries: 28640
2024-12-22 14:22:37.707 CET [4061M] INFO: -> Forwarded DNS queries: 2015
2024-12-22 14:22:37.707 CET [4061M] INFO: -> Blocked DNS queries: 782
2024-12-22 14:22:37.707 CET [4061M] INFO: -> Unknown DNS queries: 0
2024-12-22 14:22:37.707 CET [4061M] INFO: -> Unique domains: 776
2024-12-22 14:22:37.707 CET [4061M] INFO: -> Unique clients: 17
2024-12-22 14:22:37.708 CET [4061M] INFO: -> DNS cache records: 92
2024-12-22 14:22:37.708 CET [4061M] INFO: -> Known forward destinations: 1
2024-12-22 14:22:38.853 CET [4061M] INFO: FTL is running as user pihole (UID 999)
2024-12-22 14:22:38.855 CET [4061M] INFO: Reading certificate from /etc/pihole/tls.pem ...
2024-12-22 14:22:38.855 CET [4061M] INFO: Using SSL/TLS certificate file /etc/pihole/tls.pem
2024-12-22 14:22:38.859 CET [4061M] INFO: Restored 1 API session from the database
2024-12-22 14:22:38.871 CET [4061M] INFO: Blocking status is enabled
2024-12-22 14:22:39.191 CET [4061/T4063] INFO: Compiled 1 allow and 0 deny regex for 17 clients in 229.4 msec
2024-12-22 14:22:42.959 CET [4061/T4062] INFO: Received 8/8 valid NTP replies from pool.ntp.org
2024-12-22 14:22:42.959 CET [4061/T4062] INFO: Time offset: 3.869193e-02 ms (excluded 1 outliers)
2024-12-22 14:22:42.959 CET [4061/T4062] INFO: Round-trip delay: 9.669440e+00 ms (excluded 1 outliers)
2024-12-22 14:22:43.000 CET [4061/T4069] INFO: NTP server listening on 0.0.0.0:123 (IPv4)
2024-12-22 14:22:43.000 CET [4061/T4070] INFO: NTP server listening on :::123 (IPv6)
2024-12-22 14:22:55.928 CET [4061M] INFO: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
2024-12-22 14:22:55.929 CET [4061M] INFO: ----------------------------> FTL crashed! <----------------------------
2024-12-22 14:22:55.929 CET [4061M] INFO: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
2024-12-22 14:22:55.929 CET [4061M] INFO: Please report a bug at https://github.com/pi-hole/FTL/issues
2024-12-22 14:22:55.929 CET [4061M] INFO: and include in your report already the following details:
2024-12-22 14:22:55.929 CET [4061M] INFO: FTL has been running for 23 seconds
2024-12-22 14:22:55.929 CET [4061M] INFO: FTL branch: development
2024-12-22 14:22:55.929 CET [4061M] INFO: FTL version: vDev-c04fa58
2024-12-22 14:22:55.930 CET [4061M] INFO: FTL commit: c04fa58
2024-12-22 14:22:55.930 CET [4061M] INFO: FTL date: 2024-12-22 08:39:41 +0100
2024-12-22 14:22:55.930 CET [4061M] INFO: FTL user: started as pihole, ended as pihole
2024-12-22 14:22:55.930 CET [4061M] INFO: Compiled for linux/arm/v6 (compiled on CI) using cc (Alpine 14.2.0) 14.2.0
2024-12-22 14:22:55.930 CET [4061M] INFO: Process details: MID: 4061
2024-12-22 14:22:55.930 CET [4061M] INFO: PID: 4061
2024-12-22 14:22:55.930 CET [4061M] INFO: TID: 4061
2024-12-22 14:22:55.930 CET [4061M] INFO: Name: pihole-FTL
2024-12-22 14:22:55.931 CET [4061M] INFO: Received signal: Segmentation fault
2024-12-22 14:22:55.931 CET [4061M] INFO: at address: 0x30
2024-12-22 14:22:55.931 CET [4061M] INFO: with code: SEGV_MAPERR (Address not mapped to object)
2024-12-22 14:22:55.931 CET [4061M] INFO: !!! INFO: pihole-FTL has not been compiled with glibc/backtrace support, not generating one !!!
2024-12-22 14:22:55.931 CET [4061M] INFO: ------ Listing content of directory /dev/shm ------
2024-12-22 14:22:55.931 CET [4061M] INFO: File Mode User:Group Size Filename
2024-12-22 14:22:55.931 CET [4061M] INFO: rwxrwxrwx root:root 360 .
2024-12-22 14:22:55.931 CET [4061M] INFO: rwxr-xr-x root:root 4K ..
2024-12-22 14:22:55.932 CET [4061M] INFO: rw------- pihole:pihole 56 FTL-lock
2024-12-22 14:22:55.932 CET [4061M] INFO: rw------- pihole:pihole 328 FTL-counters
2024-12-22 14:22:55.933 CET [4061M] INFO: rw------- pihole:pihole 136 FTL-settings
2024-12-22 14:22:55.933 CET [4061M] INFO: rw------- pihole:pihole 82K FTL-strings
2024-12-22 14:22:55.933 CET [4061M] INFO: rw------- pihole:pihole 29K FTL-domains
2024-12-22 14:22:55.934 CET [4061M] INFO: rw------- pihole:pihole 340K FTL-clients
2024-12-22 14:22:55.934 CET [4061M] INFO: rw------- pihole:pihole 12K FTL-upstreams
2024-12-22 14:22:55.935 CET [4061M] INFO: rw------- pihole:pihole 2M FTL-queries
2024-12-22 14:22:55.935 CET [4061M] INFO: rw------- pihole:pihole 8K FTL-overTime
2024-12-22 14:22:55.935 CET [4061M] INFO: rw------- pihole:pihole 20K FTL-dns-cache
2024-12-22 14:22:55.936 CET [4061M] INFO: rw------- pihole:pihole 4K FTL-per-client-regex
2024-12-22 14:22:55.936 CET [4061M] INFO: rw------- pihole:pihole 560K FTL-fifo-log
2024-12-22 14:22:55.936 CET [4061M] INFO: rw------- pihole:pihole 4K FTL-clients-lookup
2024-12-22 14:22:55.937 CET [4061M] INFO: rw------- pihole:pihole 8K FTL-domains-lookup
2024-12-22 14:22:55.937 CET [4061M] INFO: rw------- pihole:pihole 4K FTL-dns-cache-lookup
2024-12-22 14:22:55.937 CET [4061M] INFO: rw------- pihole:pihole 786K FTL-recycler
2024-12-22 14:22:55.937 CET [4061M] INFO: ---------------------------------------------------
2024-12-22 14:22:55.937 CET [4061M] INFO: Please also include some lines from above the !!!!!!!!! header.
2024-12-22 14:22:55.938 CET [4061M] INFO: Thank you for helping us to improve our FTL engine!
2024-12-22 14:22:55.938 CET [4061M] INFO: Waiting for threads to join
2024-12-22 14:22:55.938 CET [4061M] INFO: Thread database (0) is idle, terminating it.
2024-12-22 14:22:55.938 CET [4061M] INFO: Thread housekeeper (1) is idle, terminating it.
2024-12-22 14:22:55.938 CET [4061M] INFO: Thread dns-client (2) is idle, terminating it.
2024-12-22 14:22:55.938 CET [4061M] INFO: Thread timer (3) is idle, terminating it.
2024-12-22 14:22:55.938 CET [4061M] INFO: Thread ntp-client (4) is idle, terminating it.
2024-12-22 14:22:55.938 CET [4061M] INFO: All threads joined
2024-12-22 14:22:55.944 CET [4061M] INFO: Stored 1 API session in the database
2024-12-22 14:22:56.981 CET [4061M] INFO: ########## FTL terminated after 24s (code 1)! ##########
@rvankraaij rvankraaij changed the title FTL is stuck in a crash loop after the recent update FTL is stuck in a crash loop Dec 22, 2024
@DL6ER
Copy link
Member

DL6ER commented Dec 22, 2024

Sorry to hear you are having issues. Could you attach the debugger to get a backtrace of the crash location for us? Detailed instructions can be found here: https://deploy-preview-338--pihole-docs.netlify.app/ftldns/gdb/

We're here in case anything is unclear. Thanks for your help!

@rvankraaij
Copy link
Author

rvankraaij commented Dec 22, 2024 •
edited
Loading

This is the debugger output:

Thread 1 "pihole-FTL" received signal SIGSEGV, Segmentation fault.
return_reply (now=<optimized out>, forward=forward@entry=0x76bae8c0, header=header@entry=0x76d205e0, n=n@entry=601,
    status=<optimized out>, status@entry=524288) at /app/src/dnsmasq/forward.c:1465
1465    /app/src/dnsmasq/forward.c: No such file or directory.
(gdb) backtrace
#0  return_reply (now=<optimized out>, forward=forward@entry=0x76bae8c0, header=header@entry=0x76d205e0, n=n@entry=601,
    status=<optimized out>, status@entry=524288) at /app/src/dnsmasq/forward.c:1465
#1  0x000f4cdc in reply_query (fd=<optimized out>, now=<optimized out>) at /app/src/dnsmasq/forward.c:1307
#2  0x000e0f50 in check_dns_listeners (now=<optimized out>) at /app/src/dnsmasq/dnsmasq.c:1866
#3  0x000e3718 in main_dnsmasq (argc=<optimized out>, argv=<optimized out>) at /app/src/dnsmasq/dnsmasq.c:1280
#4  0x00010430 in main (argc=<optimized out>, argv=0x7ec91df4) at /app/src/main.c:123

@DL6ER
Copy link
Member

DL6ER commented Dec 22, 2024

Thank you. This seems to be a regression from the recently merged dnsmasq update in #2136.

Please try if

sudo pihole checkout ftl fix/dnsmasq_multi_replies

fixes your issue.

@rvankraaij
Copy link
Author

Switching to the fix/dnsmasq_multi_replies branch resolved the issue. Thank you for the quick response!

@DL6ER DL6ER mentioned this issue Dec 22, 2024
Merged
5 tasks
@DL6ER DL6ER linked a pull request Dec 22, 2024 that will close this issue
Fix crash on multiple replies #2141
Merged
5 tasks
@DL6ER
Copy link
Member

DL6ER commented Dec 22, 2024

The fix has been merged into development, the binaries should be ready in a few minutes. Please go back to development using

sudo pihole checkout ftl development

to receive further updates in the future. Thanks for reporting, helping to resolve and confirming the fix! 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug in development Bug Bugfix in progress Pi-hole v6.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix crash on multiple replies pi-hole/FTL
2 participants
@rvankraaij @DL6ER