Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

invalid next size #7

Open
MickMouse opened this issue Dec 9, 2019 · 39 comments
Open

invalid next size #7

MickMouse opened this issue Dec 9, 2019 · 39 comments

Comments

@MickMouse
Copy link

Hi,

Grab this latest git and compile on latest Debian 10.2.
When try to test pre:

[14:02:47] [L] site pre TEST-UNKNOWN mp3
[14:02:47] [L] 200- malloc(): invalid next size (unsorted)
[14:02:47] [L] 200- Aborted
[14:02:47] [L] 200 Command Successful.

@glftpd
Copy link
Owner

glftpd commented Dec 9, 2019

please compile it with debug info and try to either get core dump or attach debugger to the crash

(debug info can be enabled by editing Makefile.config and changing -O2 to -g, you can attach debugger by adding some Sleep() into the main code and attaching gdb to the pid while it sleeps)

@MickMouse
Copy link
Author

Hi

See the following in dmesg:
[Tue Dec 10 13:45:24 2019] foo-pre[4218]: segfault at 0 ip 000055d731d32fd4 sp 00007fff49842a30 error 4 in foo-pre[55d731d2f000+8000]
[Tue Dec 10 13:45:24 2019] Code: 48 33 14 25 28 00 00 00 75 05 48 83 c4 18 c3 e8 b2 c1 ff ff 66 90 48 83 ec 18 64 48 8b 04 25 28 00 00 00 48 89 44 24 08 31 c0 <48> 83 3f 00 0f 95 c0 48 8b 54 24 08 64 48 33 14 25 28 00 00 00 75
[Tue Dec 10 13:45:35 2019] foo-pre[4221]: segfault at 0 ip 0000560869760fd4 sp 00007ffc594e1a30 error 4 in foo-pre[56086975d000+8000]
[Tue Dec 10 13:45:35 2019] Code: 48 33 14 25 28 00 00 00 75 05 48 83 c4 18 c3 e8 b2 c1 ff ff 66 90 48 83 ec 18 64 48 8b 04 25 28 00 00 00 48 89 44 24 08 31 c0 <48> 83 3f 00 0f 95 c0 48 8b 54 24 08 64 48 33 14 25 28 00 00 00 75
[Tue Dec 10 13:46:57 2019] foo-pre[4310]: segfault at 0 ip 00005619ad1d0d04 sp 00007ffe4d595b10 error 4 in foo-pre[5619ad1cc000+9000]
[Tue Dec 10 13:46:57 2019] Code: 48 33 14 25 28 00 00 00 75 05 48 83 c4 18 c3 e8 62 b4 ff ff 66 90 48 83 ec 18 64 48 8b 04 25 28 00 00 00 48 89 44 24 08 31 c0 <48> 83 3f 00 0f 95 c0 48 8b 54 24 08 64 48 33 14 25 28 00 00 00 75
[Tue Dec 10 13:47:58 2019] foo-pre[4611]: segfault at 0 ip 000056215bea1fd4 sp 00007fff9555e990 error 4 in foo-pre[56215be9e000+8000]
[Tue Dec 10 13:47:58 2019] Code: 48 33 14 25 28 00 00 00 75 05 48 83 c4 18 c3 e8 b2 c1 ff ff 66 90 48 83 ec 18 64 48 8b 04 25 28 00 00 00 48 89 44 24 08 31 c0 <48> 83 3f 00 0f 95 c0 48 8b 54 24 08 64 48 33 14 25 28 00 00 00 75

Also current the line inside Makefile.config is not with O2 but O0, what else can i do for you to grab you more details?

@MickMouse
Copy link
Author

Extra information:

It appears when you just site pre a empty dir nothing goes wrong:
[13:58:18] [R] site pre TEST1-UNKNOWN mp3
[13:58:18] [R] 200- /.__ ________ /
[13:58:18] [R] 200- \ / _ | / __ /__/
_ /
[13:58:18] [R] 200- __ / / // / \ / / ( original f00-pre )
[13:58:18] [R] 200- /
/ /_
\ / ________________________
[13:58:18] [R] 200- /
/ (
/ /
[13:58:18] [R] 200- / /
[13:58:18] [R] 200- * Looks like this is going to be a UNKNOWN pre..
[13:58:18] [R] 200- * Destination for pre will be the mp3 section..
[13:58:18] [R] 200- * Totals of this pre for announce: 0F 0.0B
[13:58:18] [R] 200- * Moving files to destination dir..
[13:58:18] [R] 200- -- From: /site/groups/UNKNOWN/TEST1-UNKNOWN
[13:58:18] [R] 200- -- To: /site/incoming/mp3/2019-12-10/TEST1-UNKNOWN Done
[13:58:18] [R] 200- * Updating userfiles ..
[13:58:18] [R] 200- ________________________________________________
[13:58:18] [R] 200- foo-pre-v3.3 (c) tanesha /
_______________
[13:58:18] [R] 200-
[13:58:18] [R] 200 Command Successful.

As soon the directory contains any files then it gives the malloc(): invalid next size (unsorted)
Also here some logging from compile added to this message :)
foo-pre_compilelog.txt

@silv3rr
Copy link
Contributor

silv3rr commented Dec 11, 2019

you might want to try https://github.com/silv3rr/foo-tools

@MickMouse
Copy link
Author

Hi silv3rr,

Compile goes perfect but shows the same behaviour :(
[10:58:45] [L] 200- malloc(): invalid next size (unsorted)
[10:58:45] [L] 200- Aborted
[10:58:45] [L] 200 Command Successful.

@MickMouse
Copy link
Author

Hi

I've added sleep(60); inside the code off: int pre(char *section, char *dest, char *src, char *rel, char *group, char *argv[]) { within glftpd site pre dir section, the proccess fall into sleep.

After that i attach gdb to the process:
gdb -p 9326
GNU gdb (Debian 8.2.1-2+b3) 8.2.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/.
Find the GDB manual and other documentation resources online at:
http://www.gnu.org/software/gdb/documentation/.

For help, type "help".
Type "apropos word" to search for commands related to "word".
Attaching to process 9326
Reading symbols from /jail/glftpd/bin/foo-pre...done.
Reading symbols from /lib/x86_64-linux-gnu/libdl.so.2...Reading symbols from /usr/lib/debug/.build-id/d3/583c742dd47aaa860c5ae0c0c5bdbcd2d54f61.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...Reading symbols from /usr/lib/debug/.build-id/18/b9a9a8c523e5cfe5b5d946d605d09242f09798.debug...done.
done.
Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from /usr/lib/debug/.build-id/f2/5dfd7b95be4ba386fd71080accae8c0732b711.debug...done.
done.
0x00007f8c56c0c6f4 in __GI___nanosleep (requested_time=requested_time@entry=0x7ffc0a674330, remaining=remaining@entry=0x7ffc0a674330) at ../sysdeps/unix/sysv/linux/nanosleep.c:28
28 ../sysdeps/unix/sysv/linux/nanosleep.c: No such file or directory.
(gdb)
And then? What todo to get more information for you guys?

@MickMouse
Copy link
Author

I found out with next i can show more information:

Attached the output of gdb before it reaches a SIGABRT
foo-pre_gdblog.txt

@glftpd
Copy link
Owner

glftpd commented Dec 13, 2019

thanks nice progress

after the SIGABRT do a "bt full" to get info where it happened

also you can use "step" to go into the function where it crashed (userfile_update) instead of using next

@MickMouse
Copy link
Author

MickMouse commented Dec 13, 2019

Hi

See the 2 files added it shows a more detailed overview what happens, i'll try to do some more with using step :)

foo-pre_gdblog_#1.txt
foo-pre_gdblog_#2.txt

@glftpd
Copy link
Owner

glftpd commented Dec 15, 2019

thanks ill have a look

@glftpd
Copy link
Owner

glftpd commented Dec 16, 2019

I dont see an error directly around the point of crash just yet but the error indicates a memory corruption which could have happened earlier. Could you compile with these flags which might give us the error when it happens ?
-g -fsanitize=address -fsanitize=undefined -D_FORTIFY_SOURCE=2 -fstack-protector-all

@MickMouse
Copy link
Author

MickMouse commented Dec 17, 2019

Hi

I've changed Makefile.config to have the line CC = gcc -g -fsanitize=address -fsanitize=undefined -D_FORTIFY_SOURCE=2 -fstack-protector-all which give a shitload off error :) but foo-pre does compile but as soon you run it: site pre the following error show within glftpd (after a few libs have been copyed to /lib dir inside glftpd)

[10:50:19] [L] site pre
[10:50:19] [L] 200- ==7365==WARNING: reading executable name failed with errno 2, some stack frames may not be symbolized
[10:50:19] [L] 200- ==7365==WARNING: reading executable name failed with errno 2, some stack frames may not be symbolized
[10:50:19] [L] 200- ==7365==AddressSanitizer CHECK failed: ../../../../src/libsanitizer/sanitizer_common/sanitizer_procmaps_common.cc:75 "((data_.proc_self_maps.len)) > ((0))" (0x0, 0x0)
[10:50:19] [L] 200-
[10:50:19] [L] 200-
[10:50:19] [L] 200 Command Successful.

Here the small log within ../pre/
/jail/sources/foo-tools-master_glftpd/src/pre# make
gcc -g -fsanitize=address -fsanitize=undefined -D_FORTIFY_SOURCE=2 -fstack-protector-all -c foo-pre.c -I..
foo-pre.c: In function ‘filelist_find_by_dir’:
foo-pre.c:614:4: warning: implicit declaration of function ‘touch_dir’; did you mean ‘fchdir’? [-Wimplicit-function-declaration]
touch_dir(buf);
^~~~~~~~~
fchdir
foo-pre.c:626:4: warning: implicit declaration of function ‘touch_file’; did you mean ‘flockfile’? [-Wimplicit-function-declaration]
touch_file(buf);
^~~~~~~~~~
flockfile
foo-pre.c: In function ‘pre_do_modules’:
foo-pre.c:978:3: warning: ‘return’ with no value, in function returning non-void
return;
^~~~~~
foo-pre.c:969:5: note: declared here
int pre_do_modules(filelist_t *files, char *path, char *argv[], struct subdir_list *subdirs) {
^~~~~~~~~~~~~~
foo-pre.c: In function ‘pre’:
foo-pre.c:1061:25: warning: implicit declaration of function ‘get_mp3_genre’ [-Wimplicit-function-declaration]
sprintf(gtmp, "%s", get_mp3_genre(buf));
^~~~~~~~~~~~~
gcc -g -fsanitize=address -fsanitize=undefined -D_FORTIFY_SOURCE=2 -fstack-protector-all -c gl_userfile.c -I..
make -C mp3genre
make[1]: Entering directory '/jail/sources/foo-tools-master_glftpd/src/pre/mp3genre'
gcc -g -fsanitize=address -fsanitize=undefined -D_FORTIFY_SOURCE=2 -fstack-protector-all -c mp3genre.c -o mp3genre.o
mp3genre.c: In function ‘get_mp3_genre’:
mp3genre.c:30:8: warning: implicit declaration of function ‘strcmp’ [-Wimplicit-function-declaration]
if (!strcmp((const char *)"TAG",(const char *)mp3_fbuf)) {
^~~~~~
make[1]: Leaving directory '/jail/sources/foo-tools-master_glftpd/src/pre/mp3genre'
gcc -g -fsanitize=address -fsanitize=undefined -D_FORTIFY_SOURCE=2 -fstack-protector-all -o foo-pre foo-pre.o ../collection/strlist.o ../lib/gllogs.o ../lib/pwdfile.o ../util/linefilereader.o ../collection/hashtable.o ../lib/stringtokenizer.o ./gl_userfile.o ./mp3genre/mp3genre.o -ldl
config.log

@glftpd
Copy link
Owner

glftpd commented Dec 20, 2019

ok but what happens when you add the sleep and use the gdb like before ? those compilation warnings just mean the code is crap but everyone knew that :)

@MickMouse
Copy link
Author

Hi,

placed the sleep inside: int main(int argc, char *argv[]) {
glftpd show the same error while running site pre

[11:27:59] [L] site pre
[11:27:59] [L] 200- ==3968==WARNING: reading executable name failed with errno 2, some stack frames may not be symbolized
[11:27:59] [L] 200- ==3968==WARNING: reading executable name failed with errno 2, some stack frames may not be symbolized
[11:27:59] [L] 200- ==3968==AddressSanitizer CHECK failed: ../../../../src/libsanitizer/sanitizer_common/sanitizer_procmaps_common.cc:75 "((data_.proc_self_maps.len)) > ((0))" (0x0, 0x0)
[11:27:59] [L] 200-
[11:27:59] [L] 200-
[11:27:59] [L] 200 Command Successful.

@MickMouse
Copy link
Author

MickMouse commented Dec 24, 2019

Running the ./foo-pre from command line and attached gdb:
gdb -p 5724
GNU gdb (Debian 8.2.1-2+b3) 8.2.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/.
Find the GDB manual and other documentation resources online at:
http://www.gnu.org/software/gdb/documentation/.

For help, type "help".
Type "apropos word" to search for commands related to "word".
Attaching to process 5724
Reading symbols from /jail/glftpd/bin/foo-pre...done.
Reading symbols from /lib/x86_64-linux-gnu/libasan.so.5...Reading symbols from /usr/lib/debug/.build-id/3c/f2e4b5261216f9a156ed5dc2953d8b6f98987d.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libdl.so.2...Reading symbols from /usr/lib/debug/.build-id/d3/583c742dd47aaa860c5ae0c0c5bdbcd2d54f61.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libubsan.so.1...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...Reading symbols from /usr/lib/debug/.build-id/18/b9a9a8c523e5cfe5b5d946d605d09242f09798.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/librt.so.1...Reading symbols from /usr/lib/debug/.build-id/5d/cf98ad684962be494af28a1051793fd39e4ebc.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libpthread.so.0...Reading symbols from /usr/lib/debug/.build-id/e9/1114987a0147bd050addbd591eb8994b29f4b3.debug...done.
done.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Reading symbols from /lib/x86_64-linux-gnu/libstdc++.so.6...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libm.so.6...Reading symbols from /usr/lib/debug/.build-id/88/5dda4b4a5cea600e7b5b98c1ad86996c8d2299.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libgcc_s.so.1...(no debugging symbols found)...done.
Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from /usr/lib/debug/.build-id/f2/5dfd7b95be4ba386fd71080accae8c0732b711.debug...done.
done.
0x00007f69dcd9e6f4 in __GI___nanosleep (requested_time=0x7fff60a0edf0, remaining=0x7fff60a0edf0) at ../sysdeps/unix/sysv/linux/nanosleep.c:28
28 ../sysdeps/unix/sysv/linux/nanosleep.c: No such file or directory.
(gdb) next
__sleep (seconds=0) at ../sysdeps/posix/sleep.c:62
62 ../sysdeps/posix/sleep.c: No such file or directory.
(gdb) next
64 in ../sysdeps/posix/sleep.c
(gdb) next
main (argc=1, argv=0x7fff60a0ef38) at foo-pre.c:1447
1447 pre_init();
(gdb) next
[Inferior 1 (process 5724) exited with code 01]
(gdb) next
The program is not being run.
(gdb)

Did not find environment for glftpd, please run only from within glftpd as 'site' command!
==5724==LeakSanitizer has encountered a fatal error.
==5724==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1
==5724==HINT: LeakSanitizer does not work under ptrace (strace, gdb, etc)

@MickMouse
Copy link
Author

I've added the sleep inside this code: int pre_handler(int argc, char *argv[]) {
Attached gdb:

gdb -p 10291
GNU gdb (Debian 8.2.1-2+b3) 8.2.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/.
Find the GDB manual and other documentation resources online at:
http://www.gnu.org/software/gdb/documentation/.

For help, type "help".
Type "apropos word" to search for commands related to "word".
Attaching to process 10291
Reading symbols from /jail/glftpd/bin/foo-pre...done.
Reading symbols from /lib/x86_64-linux-gnu/libdl.so.2...Reading symbols from /usr/lib/debug/.build-id/d3/583c742dd47aaa860c5ae0c0c5bdbcd2d54f61.debug...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...Reading symbols from /usr/lib/debug/.build-id/18/b9a9a8c523e5cfe5b5d946d605d09242f09798.debug...done.
done.
Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from /usr/lib/debug/.build-id/f2/5dfd7b95be4ba386fd71080accae8c0732b711.debug...done.
done.
0x00007fa02bb236f4 in __GI___nanosleep (requested_time=requested_time@entry=0x7ffccdd12b30, remaining=remaining@entry=0x7ffccdd12b30) at ../sysdeps/unix/sysv/linux/nanosleep.c:28
28 ../sysdeps/unix/sysv/linux/nanosleep.c: No such file or directory.
(gdb) next
__sleep (seconds=0) at ../sysdeps/posix/sleep.c:62
62 ../sysdeps/posix/sleep.c: No such file or directory.
(gdb) next
64 in ../sysdeps/posix/sleep.c
(gdb) next
pre_handler (argc=3, argv=0x7ffccdd13998) at foo-pre.c:1291
1291 env = get_context();
(gdb) next
1292 cfg = get_config();
(gdb) next
1294 pre_log("START", "%s %s %s", ht_get(env, PROPERTY_USER), argv[1], argv[2]);
(gdb) next
1297 if (tmp = ht_get(cfg, PROPERTY_ETCDIR))
(gdb) next
1300 if (tmp = ht_get(cfg, PROPERTY_TEXT_HEAD))
(gdb) next
1301 printf(tmp);
(gdb) next
1303 groups = user_find_groups(ht_get(env, PROPERTY_USER));
(gdb) next
1305 if (!groups)
(gdb) next
1308 tmp = ht_get(cfg, PROPERTY_ADDMP3GENRE);
(gdb) next
1310 if (argc < 2) {
(gdb) next
1318 if (tmp)
(gdb) next
1319 addmp3genre = atoi(tmp);
(gdb) next
1324 if (strchr(argv[1], '/'))
(gdb) next
1327 char *sourcebis = getcwd(NULL, 0);
(gdb) next
1330 group = group_find_by_dir(groups, sourcebis);
(gdb) next
1332 if (!group) {
(gdb) next
1340 pre_log("GROUP", "%s %s", sourcebis, group);
(gdb) next
1342 printf(" * Looks like this is going to be a %s pre..\n", group);
(gdb) next
1343 ht_put(env, PROPERTY_GROUP, group);
(gdb) next
1346 dest_section = section_find_by_name(group, argc > 2 ? argv[2] : 0);
(gdb) next
1348 if (!dest_section) {
(gdb) next
1354 printf(" * Destination for pre will be the %s section..\n", dest_section);
(gdb) next
1355 ht_put(env, "section", dest_section);
(gdb) next
1357 destpath = section_expand_path(dest_section);
(gdb) next
1358 ht_put(env, "RESOLVEDDESTINATION", destpath);
(gdb) next
1360 strcpy(source, sourcebis);
(gdb) next
1361 strcat(source, "/");
(gdb) next
1362 strcat(source, argv[1]);
(gdb) next
1365 if ((stat(source, &st) == -1) || !S_ISDIR(st.st_mode)) {
(gdb) next
1372 touch_dir(source);
(gdb) next
1375 sprintf(destination, "%s/%s", destpath, argv[1]);
(gdb) next
1377 rc = stat(destination, &st);
(gdb) next
1380 if ((rc == 0) && (argc > 3) && (!strcasecmp(argv[3], "force"))) {
(gdb) next
1392 if (stat(destination, &st) == -1)
(gdb) next
1393 pre(dest_section, destination, source, argv[1], group, argv);
(gdb) next

Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb)

@MickMouse
Copy link
Author

Hmm,

Ok, i've removed sleep(60); and edited:
below: // give credits to original uploaders

from:
printf(" * Updating userfiles ..\n");
strcpy(ubuf, "");
for (tcred = credits; tcred; tcred = tcred->next) {
tmp = userfile_update(tcred);
if (tmp)
strcat(ubuf, tmp);
}

setuid(olduid);

if (strlen(ubuf) > 2)
	ubuf[strlen(ubuf) - 2] = 0;

to:
/*
printf(" * Updating userfiles ..\n");
strcpy(ubuf, "");
for (tcred = credits; tcred; tcred = tcred->next) {
tmp = userfile_update(tcred);
if (tmp)
strcat(ubuf, tmp);
}

setuid(olduid);

if (strlen(ubuf) > 2)
	ubuf[strlen(ubuf) - 2] = 0;
*/

this to disable the updateing of userfile at all, foo-pre runs properly then and no mistakes inside the logs. releasedir moved to destination and output as follows:

[12:21:23] [L] site pre TEST4-GUEST mp3
[12:21:23] [L] 200- /.__ ________ /
[12:21:23] [L] 200- \ / _ | / __ /__/
_ /
[12:21:23] [L] 200- __ / / // / \ / / ( original f00-pre )
[12:21:23] [L] 200- /
/ /_
\ / ________________________
[12:21:23] [L] 200- /
/ (
/ /
[12:21:23] [L] 200- / /
[12:21:23] [L] 200- * Looks like this is going to be a GUEST pre..
[12:21:23] [L] 200- * Destination for pre will be the mp3 section..
[12:21:23] [L] 200- * Totals of this pre for announce: 11F 139.4Mb
[12:21:23] [L] 200- * Moving files to destination dir..
[12:21:23] [L] 200- -- From: /site/groups/GUEST/TEST4-GUEST
[12:21:23] [L] 200- -- To: /site/incoming/mp3/2019-12-24/TEST4-GUEST Done
[12:21:23] [L] 200- ________________________________________________
[12:21:23] [L] 200- foo-pre-v3.3 (c) tanesha /
_______________
[12:21:23] [L] 200-
[12:21:23] [L] 200 Command Successful.

@silv3rr
Copy link
Contributor

silv3rr commented Dec 24, 2019

not sure if related but you could try setting addmp3genretogllog=1 in pre.cfg
or even better test an older/original version of the src without my mp3 genre crap added
(e.g. 86263b4)

also, make sure chown user/group set in pre.cfg group exist in gl's passwd/group file
and try setting group.GUEST.ratio=3 in pre.cfg

@MickMouse
Copy link
Author

Hi,
A new try with debian 9 pre goes but with the following error

[14:19:25] [R] 200- *** Error in `/bin/foo-pre': free(): invalid next size (fast): 0x00005631cad48b90 ***
[14:19:25] [R] 200- Aborted
[14:19:25] [R] 200- /____.__ ________ _____/
[14:19:25] [R] 200- \ / _ | / __ /__/
_ /
[14:19:25] [R] 200- __ / / // / \ / / ( original f00-pre )
[14:19:25] [R] 200- /
/ /_
\ / ________________________
[14:19:25] [R] 200- /
/ (
/ /

[14:19:25] [R] 200- / /
[14:19:25] [R] 200- * Looks like this is going to be a GUEST pre..
[14:19:25] [R] 200- * Destination for pre will be the xxx section..
[14:19:25] [R] 200- * Totals of this pre for announce: 22F 1015.8Mb
[14:19:25] [R] 200- * Moving files to destination dir..
[14:19:25] [R] 200- -- From: /site/groups/GUEST/Release-GUEST
[14:19:25] [R] 200- -- To: /site/incoming/xxx/Release-GUEST Done
[14:19:25] [R] 200- * Updating userfiles ..
[14:19:25] [R] 200- -- USER: 24 file/s, 1015.8 Mb (s:0), Credits: 0.0 Mb, Ratio: 0 (s:0)
[14:19:25] [R] 200 Command Successful.

Will add debug for it.

@MickMouse
Copy link
Author

Copyed libgcc_s.so.1 to the lib dir inside glftpd:

Following stuff is shown now: (Debian 9)
File try01.txt attached is the logging where the sleep command is added to the first main proc: int main(int argc, char *argv[]) in foo-pre.c

At this point there has been no logging inside glftpd.log at all.
Logging inside foo-pre.log is normal as expected.

try01.txt

@MickMouse
Copy link
Author

On try02 you'll see the output not with gdb but with strace.

try02.txt

@MickMouse
Copy link
Author

MickMouse commented Jan 10, 2020

Ok gimmie some input if possible, i've been adding some code to disable updating userfile as show earlier above, the next size error is gone.

So i have been checking inside:
/*

  • Takes care of updating userfile, by help of gl_userfile module.
    */
    char * userfile_update(creditlist_t *l) {

From there i've narrowd down how far the code normaly excecude until the error show inside the client. Added is the code off userfile_update try03.txt

Code: /* edited to see if the error next size is gone */ inside the file is where i gues errors are because if that code is executed we face the errors from try01.txt

try03.txt

@glftpd
Copy link
Owner

glftpd commented Jan 10, 2020

as its a memory corruption issue that happens first but is detected later in the code it would be hard to figure out by commenting out code this way. I would highly recommend either using tool like asan (i showed the flags before) or valgrind to find the exact cause or find some way to replicate the issue so i can fix it myself.

@MickMouse
Copy link
Author

Hook me up a bit how i could use asan or valgrind. The flags you showed before make the excecutable not being able to ran inside like glftpd when running site pre. Don't you have the issue on Debian 10 or 9.

@MickMouse
Copy link
Author

valgrind --leak-check=full --show-leak-kinds=all ./foo-pre
==1731== Memcheck, a memory error detector
==1731== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==1731== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info
==1731== Command: ./foo-pre
==1731==
Did not find environment for glftpd, please run only from within glftpd as 'site' command!
==1731==
==1731== HEAP SUMMARY:
==1731== in use at exit: 16 bytes in 1 blocks
==1731== total heap usage: 3 allocs, 2 frees, 1,592 bytes allocated
==1731==
==1731== 16 bytes in 1 blocks are still reachable in loss record 1 of 1
==1731== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==1731== by 0x109A49: get_config (foo-pre.c:61)
==1731== by 0x10CDA4: pre_init (foo-pre.c:1367)
==1731== by 0x10CE7D: main (foo-pre.c:1392)
==1731==
==1731== LEAK SUMMARY:
==1731== definitely lost: 0 bytes in 0 blocks
==1731== indirectly lost: 0 bytes in 0 blocks
==1731== possibly lost: 0 bytes in 0 blocks
==1731== still reachable: 16 bytes in 1 blocks
==1731== suppressed: 0 bytes in 0 blocks
==1731==
==1731== For counts of detected and suppressed errors, rerun with: -v
==1731== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

This show when just run valgrind on the build foo-pre excutable.

@glftpd
Copy link
Owner

glftpd commented Jan 10, 2020

Remember that you need to run the full command that crashes under valgrind. foo-pre runs chrooted so you might need valgrind under the chroot too. Another option is to run the inetd under valgrind and have it follow child processes. Neither is simple. Also you want to detect memory corruption and not leaks so you have to put that into the command line too. Using asan or something like that might still be easier option but hard to say.

@MickMouse
Copy link
Author

I’ll add it to the chroot and run from there. -tool=memcheck -v any other you’d like to see?

@MickMouse
Copy link
Author

I've been playing and thinking more, if i change the following 2 lines from pre.cfg errors are gone.

countable=.none
creditable=
.none

All sort of dirs i've tryed and it pred without errors, but as soon the dir contains any files with *.none extension the error is back. So i really think it's something todo with that. What you think?? i'm not a pro in all this :)

@MickMouse
Copy link
Author

If the error shows up, the userfile USER.pre-tmp still exist, and rename(ufnew, userfile); from gl_userfile_add_stats never be reached it seems. Just the file is half written:

USER Added by glftpd
GENERAL 0,0 -1 0 0
LOGINS 10 0 -1 -1
TIMEFRAME 0 0
FLAGS 1347ABCDEFGHI
TAGLINE No Tagline Set
DIR /
ADDED XXXXXX glftpd
EXPIRES 0
CREDITS 289284 0 0 0 0 0 0 0
RATIO 0 -1 -1 -1 -1 -1 -1 -1
ALLUP 549 61146914 5709 1368 26736080 1525 13 212849 13 0 0 0 17926 131282104 18240 0 0 0 4458 407089604 7775 0 0 0

@MickMouse
Copy link
Author

Compared to the original line in the userfile:

ALLUP 530 61055486 5707 1410 26925160 1567 13 212849 13 0 0 0 17926 131282104 18240 0 0 0 4458 407089604 7775 0 0 0

After the ALLUP it stops and leave the file.

@MickMouse
Copy link
Author

disable:
// call gl_userfile_* to get the userfile updated.
//rc = gl_userfile_add_stats(userfile, l->files, addkb, spend, (long)ratio * addkb, gl_stat_section, gl_credit_section);

and:
[14:47:16] [R] site pre VRVR xxx
[14:47:16] [R] 200- /.__ ________ /
[14:47:16] [R] 200- \ / _ | / __ /__/
_ /
[14:47:16] [R] 200- __ / / // / \ / / ( original f00-pre )
[14:47:16] [R] 200- /
/ /_
\ / ________________________
[14:47:16] [R] 200- /
/ (
/ /
[14:47:16] [R] 200- / /
[14:47:16] [R] 200- * Looks like this is going to be a GUEST pre..
[14:47:16] [R] 200- * Destination for pre will be the xxx section..
[14:47:16] [R] 200- * Totals of this pre for announce: 17F 93618304B
[14:47:16] [R] 200- * Moving files to destination dir..
[14:47:16] [R] 200- -- From: /site/groups/GUEST/VRVR
[14:47:16] [R] 200- -- To: /site/incoming/xxx/VRVR Done
[14:47:16] [R] 200- * Updating userfiles ..
[14:47:16] [R] 200- -- USER: 19 file/s, 89.3 Mb (s:0), Credits: 267.9 Mb, Ratio: 3 (s:0)
[14:47:16] [R] 200- ________________________________________________
[14:47:16] [R] 200- foo-pre-v3.3 (c) tanesha /
_______________
[14:47:16] [R] 200-
[14:47:16] [R] 200 Command Successful.

@MickMouse
Copy link
Author

Hi

Ive found out change your userfile
ALLUP
ALLDN
WKUP
WKDN
DAYUP
DAYDN
MONTHUP
MONTHDN

change all to 0 and it works
[15:49:58] [R] site pre RKRE xxx
[15:49:58] [R] 200- /.__ ________ /
[15:49:58] [R] 200- \ / _ | / __ /__/
_ /
[15:49:58] [R] 200- __ / / // / \ / / ( original f00-pre )
[15:49:58] [R] 200- /
/ /_
\ / ________________________
[15:49:58] [R] 200- /
/ (
/ /
[15:49:58] [R] 200- / /
[15:49:58] [R] 200- * Looks like this is going to be a GUEST pre..
[15:49:58] [R] 200- * Destination for pre will be the xxx section..
[15:49:58] [R] 200- * Totals of this pre for announce: 17F 93618304B
[15:49:58] [R] 200- * Moving files to destination dir..
[15:49:58] [R] 200- -- From: /site/groups/GUEST/RKRE
[15:49:58] [R] 200- -- To: /site/incoming/xxx/RKRE Done
[15:49:58] [R] 200- * Updating userfiles ..
[15:49:58] [R] 200- -- USER: 19 file/s, 89.3 Mb (s:0), Credits: 0.0 Mb, Ratio: 0 (s:0)
[15:49:58] [R] 200- ________________________________________________
[15:49:58] [R] 200- foo-pre-v3.3 (c) tanesha /
_______________
[15:49:58] [R] 200-
[15:49:58] [R] 200 Command Successful.

ALLUP 19 91428 2 21 94540 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
ALLDN 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
WKUP 19 91428 2 21 94540 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
WKDN 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
DAYUP 19 91428 2 21 94540 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
DAYDN 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
MONTHUP 19 91428 2 21 94540 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
MONTHDN 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

@MickMouse
Copy link
Author

ALLUP 530 61055486 5707 1410 26925160 1567 13 212849 13 0 0 0 17926 131282104 18240 0 0 0 4458 407089604 7775 0 0 0
ALLDN 84 362221 163 6 80376 6 1626 90807634 1927 8 4199936 103 360 7506723 446 19 5020 19 74718 4394178361 92619 0 0 0
WKUP 0 0 0 1142 17558084 1173 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
WKDN 0 0 0 0 0 0 0 0 0 1 1024000 12 0 0 0 0 0 0 0 0 0 0 0 0
DAYUP 0 0 0 1142 17558084 1173 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
DAYDN 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
MONTHUP 0 0 0 1142 17558084 1173 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
MONTHDN 0 0 0 0 0 0 0 0 0 1 1024000 12 0 0 0 0 0 0 162 7529736 170 0 0 0

this was in the USER file before. what to change to have the old file with stats working?

@silv3rr
Copy link
Contributor

silv3rr commented Jan 16, 2020

From a quick glance looks like it could be related to stat_sections

Adding ALLUP, WKUP etc stats (and creds) are handled in foo-tools/src/pre/gl_userfile.c by _gl_userfile_add_stats
Like you noticed it opens and writes to userfile.pre-tmp then should close and rename file after adding the stats/cred lines. However in your case no line gets added (rc is not 0).

You could try testing only using the default stat section in gl and pre conf

I'll try to have further look if time permits

@MickMouse
Copy link
Author

MickMouse commented Jan 16, 2020

hi!

tested with just having 1 stat line in glftpd.conf:
stat_section default * yes

here the result:
[11:10:14] [R] site pre TRTR xxx
[11:10:14] [R] 200- /.__ ________ /
[11:10:14] [R] 200- \ / _ | / __ /__/
_ /
[11:10:14] [R] 200- __ / / // / \ / / ( original f00-pre )
[11:10:14] [R] 200- /
/ /_
\ / ________________________
[11:10:14] [R] 200- /
/ (
/ /
[11:10:14] [R] 200- / /
[11:10:14] [R] 200- * Looks like this is going to be a GUEST pre..
[11:10:14] [R] 200- * Destination for pre will be the xxx section..
[11:10:14] [R] 200- * Totals of this pre for announce: 17F 89.3Mb
[11:10:14] [R] 200- * Moving files to destination dir..
[11:10:14] [R] 200- -- From: /site/groups/GUEST/TRTR
[11:10:14] [R] 200- -- To: /site/incoming/xxx/TRTR Done
[11:10:14] [R] 200- * Updating userfiles ..
[11:10:14] [R] 200- -- USER: 19 file/s, 89.3 Mb (s:0), Credits: 0.0 Mb, Ratio: 0 (s:0)
[11:10:14] [R] 200- ________________________________________________
[11:10:14] [R] 200- foo-pre-v3.3 (c) tanesha /
_______________
[11:10:14] [R] 200-
[11:10:14] [R] 200 Command Successful.

At this moment i have been able to reproduce this error on different debian 9 and 10 systems

@MickMouse
Copy link
Author

Hi,

Found out the if the glftpd.conf have more stats_section which also have multiple [:and:] in the lines aswell it fails like:
stat_section line /site/incoming/section/[:and:]/site/incoming/section2/[:and:]/site/incoming/section3/* no
stat_section line2 /site/incoming/section4/[:and:]/site/incoming/section5/[:and:]/site/incoming/section6/* no

disable one of the 2 and it works.

@silv3rr
Copy link
Contributor

silv3rr commented Jan 24, 2020

stat_section line /site/incoming/section/[:and:]/site/incoming/section2/[:and:]/site/incoming/section3/* no
stat_section line2 /site/incoming/section4/[:and:]/site/incoming/section5/[:and:]/site/incoming/section6/* no
disable one of the 2 and it works.

You should always leave the default first in gl conf:
stat_section DEFAULT * yes

Also make sure your stat_sections match pre.cfg sections (gl_stat_section=<number>)

foopre probably messes up trying to find stat_sec while adding stats to userfile. Ill see if i can add a better check.

@SoniCFL
Copy link

SoniCFL commented Jan 24, 2020

Hi!

If you use different stat_section with own credits then yes, but his problem looks more like a low buffer to add stats. I would suggest to increase the buffer inside gl_userfile.c are your stats for certain section bigger then > 0123456789?

Just make sure you have the first stat_section DEFAULT * yes/no the yes or no is for if credits are to be counted seperately if you not add it the default is no

I’ll upload my foo-tools and you could try that aswell.

@SoniCFL
Copy link

SoniCFL commented Mar 22, 2020

MickMouse, is your issue fixed or still experience problems?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants