SIGILL after the last record is read

[DNikolaevAtRocket]

Hello,

Thanks for the nice lib and the examples!

There's a nasty issue with the Fread function I hoped you could help to resolve.

I run the code from the example: https://github.com/ibmruntimes/go-recordio/blob/main/v2/example-recordio/testpds.go to read some data (from a sequential dataset which I don't think matters).
From time to time, it crashes with SIGILL on the last Fread call when there's no data left to read.
The call is supposed to return 0 to exit the reading loop (just in case, extra checks with fh.Feof() nor fh.Ferror() don't help to prevent the call).

I happens sporadically but often enough to become a show stopper.

Golang version: go1.23.0 zos/s390x
z/OS version: v3.1

SIGILL: illegal instruction
PC=0x303beb26 m=0 sigcode=15
instruction bytes @ 0x303beb1e
 0xe3 0xf0 0xf1 0x60 0x0 0x17 0xd 0xef 0x94 0xef 0xc3 0xcc 0xe3 0x10 0x49 0x78 0x0 0x16 0xb9 0x2 0x0 0x11 0x47 0x80 0x81 0x58 0xe3 0xb0 0x81 0xb6 0x0 0x4 0x47 0xf0 0x81 0x5c 0xb9 0x82 0x0 0xbb 0x91 0x80 0xc3 0xcc 0x47 0xe0 0x81 0xa2 0xd5 0x3 0x49 0x20 0x81 0xce 0x47 0xb0 0x81 0xa2 0x94 0x7f 0xc3 0xcc 0x96 0x10

pc 0x303beb26
303beb16: 0017e3f0 f0180017 e3f0f160 00170def ...........`.... ..T00...T01-....
303beb26: 94efc3cc e3104978 0016b902 00114780 ......Ix......G. m.C.T...........
303beb36: 8158e3b0 81b60004 47f0815c b98200bb .X......G..\.... a.T.a....0a*.b..
303beb46: 9180c3cc 47e081a2 d5034920 81ce47b0 ....G.....I ..G. j.C..\asN...a...
303beb56: 81a2947f c3cc9610 c3cc4110 4984e310 ..........A.I... asm"C.o.C....dT.
303beb66: 49280024 41104928 e3f00010 0017e3f0 I(.$A.I(........ ........T0....T0
303beb76: f2200017 e3f0f018 0017e3f0 f1140017 . .............. 2...T00...T01...
303beb86: 0def94ef c3ccb904 003beb4f 48000004 .........;.OH... ..m.C......|....

r0 0x57

r1 0x50082ebc68
50082ebc58: 010d0000 10800002 00000000 00000000 ................ ................
50082ebc68: 00000050 082ebc98 00000050 082ebca0 ...P.......P.... ...&...q...&....
50082ebc78: 00000050 082ebcb0 00000050 082ebcb8 ...P.......P.... ...&.......&....
50082ebc88: 00000050 082ebcbc 00000050 082ebcc0 ...P.......P.... ...&.......&...{
50082ebc98: 00000002 00000014 00000050 082ebca8 ...........P.... ...........&...y
50082ebca8: 00000000 00000000 00000000 00000000 ................ ................
50082ebcb8: 00000000 00000000 00000000 00000000 ................ ................
50082ebcc8: 00000000 00000000 00000000 00000000 ................ ................

r2 0x0

r3 0x30bdeef0
30bdeee0: 00000000 30752370 00000000 30751f60 ....0u#p....0u.` ...............-
30bdeef0: 0001018f 49c3c5c5 00000000 00000000 ....I........... .....CEE........
30bdef00: 00000000 30bb6c00 00000000 30741250 ....0.l.....0t.P ......%........&
30bdef10: 00000000 304a49b8 00000000 304a4870 ....0JI.....0JHp ................
30bdef20: 00000000 00000000 00000000 30714d58 ............0qMX ..............(.
30bdef30: 00000000 30bddb70 00000000 308d3cf8 ....0..p....0.<. .....].........8
30bdef40: 00000000 308476f0 00000000 30425030 ....0.v.....0BP0 .....d.0......&.
30bdef50: 00000000 30536668 00000000 30425068 ....0Sfh....0BPh ..............&.

r4 0x50082eb340
50082eb330: 00000000 00000000 00000000 00000000 ................ ................
50082eb340: 00000000 00000000 00000000 00000000 ................ ................
50082eb350: 00000000 00000000 00000000 00000000 ................ ................
50082eb360: 00000000 00000000 00000000 00000000 ................ ................
50082eb370: 00000000 00000000 00000000 00000000 ................ ................
50082eb380: 00000000 00000000 00000000 00000000 ................ ................
50082eb390: 00000000 00000000 00000000 00000000 ................ ................
50082eb3a0: 00000000 00000000 00000000 00000000 ................ ................

r5 0x50082ed150
50082ed140: 00000000 00000000 00000000 00000000 ................ ................
50082ed150: d7d7e2c4 e60002a0 fff64bd0 00000000 ..........K..... PPSDW....6.}....
50082ed160: 00000006 00400429 00000000 fd7fffff .....@.)........ ..... ......."..
50082ed170: 84800000 2c000000 00000000 00000000 ....,........... d...............
50082ed180: 30744cc0 00000000 00000073 082ee2b8 0tL........s.... ..<{..........S.
50082ed190: 0871c730 082ee324 082ed820 307449d8 .q.0...$... 0tI. ..G...T...Q....Q
50082ed1a0: 082ee328 082ee558 00000004 2fdfb2a8 ...(...X..../... ..T...V........y
50082ed1b0: 082ee81f 4e600050 00108398 082ef0e0 ....N`.P........ ..Y.+-.&..cq..0\

r6 0x303beb26
303beb16: 0017e3f0 f0180017 e3f0f160 00170def ...........`.... ..T00...T01-....
303beb26: 94efc3cc e3104978 0016b902 00114780 ......Ix......G. m.C.T...........
303beb36: 8158e3b0 81b60004 47f0815c b98200bb .X......G..\.... a.T.a....0a*.b..
303beb46: 9180c3cc 47e081a2 d5034920 81ce47b0 ....G.....I ..G. j.C..\asN...a...
303beb56: 81a2947f c3cc9610 c3cc4110 4984e310 ..........A.I... asm"C.o.C....dT.
303beb66: 49280024 41104928 e3f00010 0017e3f0 I(.$A.I(........ ........T0....T0
303beb76: f2200017 e3f0f018 0017e3f0 f1140017 . .............. 2...T00...T01...
303beb86: 0def94ef c3ccb904 003beb4f 48000004 .........;.OH... ..m.C......|....

r7 0x303beb88
303beb78: 0017e3f0 f0180017 e3f0f114 00170def ................ ..T00...T01.....
303beb88: 94efc3cc b904003b eb4f4800 000407f7 .......;.OH..... m.C......|.....7
303beb98: 00000000 303bea9a ffffffff ffffffff ....0;.......... ................
303beba8: 12345678 d7d7e2c4 00000019 00000002 .4Vx............ ....PPSD........
303bebb8: 00000014 02ce0fff 00000020 80000001 ........... .... ................
303bebc8: 00020906 000001c0 00077c7c e9c5d9d9 ..........||.... .......{..@@ZERR
303bebd8: c1000000 03000004 003dc3ac 00000000 .........=...... A.........C.....
303bebe8: 00000018 fffffdec 51000000 f2f0f2f3 ........Q....... ............2023

r8 0x303be9ea
303be9da: 46600024 eb134880 0024a74b fe600d80 F`.$..H..$.K.`.. .-........x..-..
303be9ea: 41d04890 b9040091 e3c004b8 0017e3c0 A.H............. .}.....jT{....T{
303be9fa: c0580004 e3c0c008 0004b904 0059d207 .X...........Y.. {...T{{.......K.
303bea0a: 49685028 d7034920 4920b982 0022e3a0 IhP(..I I ...".. ..&.P......b..T.
303bea1a: 81be0017 4160813c 4170819e e31051f0 ....A`.<Ap....Q. a....-a...a.T..0
303bea2a: 0004b920 00164780 8052b920 00174770 ... ..G..R. ..Gp ................
303bea3a: 80b6d207 c36881ae d5035000 81c24770 .....h....P...Gp ..K.C.a.N.&.aB..
303bea4a: 80b0e310 51f00004 b9200016 47808078 ....Q.... ..G..x ..T..0..........

r9 0x50082ed150
50082ed140: 00000000 00000000 00000000 00000000 ................ ................
50082ed150: d7d7e2c4 e60002a0 fff64bd0 00000000 ..........K..... PPSDW....6.}....
50082ed160: 00000006 00400429 00000000 fd7fffff .....@.)........ ..... ......."..
50082ed170: 84800000 2c000000 00000000 00000000 ....,........... d...............
50082ed180: 30744cc0 00000000 00000073 082ee2b8 0tL........s.... ..<{..........S.
50082ed190: 0871c730 082ee324 082ed820 307449d8 .q.0...$... 0tI. ..G...T...Q....Q
50082ed1a0: 082ee328 082ee558 00000004 2fdfb2a8 ...(...X..../... ..T...V........y
50082ed1b0: 082ee81f 4e600050 00108398 082ef0e0 ....N`.P........ ..Y.+-.&..cq..0\

r10 0x12345678
12345668: ******** ******** ******** ******** ................ ................
12345678: ******** ******** ******** ******** ................ ................
12345688: ******** ******** ******** ******** ................ ................
12345698: ******** ******** ******** ******** ................ ................
123456a8: ******** ******** ******** ******** ................ ................
123456b8: ******** ******** ******** ******** ................ ................
123456c8: ******** ******** ******** ******** ................ ................
123456d8: ******** ******** ******** ******** ................ ................

r11 0x50082ed150
50082ed140: 00000000 00000000 00000000 00000000 ................ ................
50082ed150: d7d7e2c4 e60002a0 fff64bd0 00000000 ..........K..... PPSDW....6.}....
50082ed160: 00000006 00400429 00000000 fd7fffff .....@.)........ ..... ......."..
50082ed170: 84800000 2c000000 00000000 00000000 ....,........... d...............
50082ed180: 30744cc0 00000000 00000073 082ee2b8 0tL........s.... ..<{..........S.
50082ed190: 0871c730 082ee324 082ed820 307449d8 .q.0...$... 0tI. ..G...T...Q....Q
50082ed1a0: 082ee328 082ee558 00000004 2fdfb2a8 ...(...X..../... ..T...V........y
50082ed1b0: 082ee81f 4e600050 00108398 082ef0e0 ....N`.P........ ..Y.+-.&..cq..0\

r12 0x5000108398
5000108388: 00000000 00000000 00000000 00000000 ................ ................
5000108398: 00000000 00000000 00000000 00000000 ................ ................
50001083a8: 00000000 00000000 00000000 00000000 ................ ................
50001083b8: 00000000 00000000 00000000 00000000 ................ ................
50001083c8: 00000000 00000000 00000000 00000000 ................ ................
50001083d8: 00000000 00000000 00000000 00000000 ................ ................
50001083e8: 00000000 00000000 00000000 00000000 ................ ................
50001083f8: 00000000 00000000 00000000 00000000 ................ ................

r13 0x50082ebbd0
50082ebbc0: 00000050 0010a970 00000050 0010a978 ...P...p...P...x ...&..z....&..z.
50082ebbd0: 00000000 30209f58 00000000 00000006 ....0 .X........ ................
50082ebbe0: 00000050 082ebfb0 00000050 082ec5e0 ...P.......P.... ...&.......&..E\
50082ebbf0: 00000050 00108398 00000000 3094b118 ...P........0... ...&..cq.....m..
50082ebc00: 00000050 00108398 00000000 3094d018 ...P........0... ...&..cq.....m}.
50082ebc10: 00000000 00000000 00000000 00000000 ................ ................
50082ebc20: 00000000 00000000 00000000 00000000 ................ ................
50082ebc30: 00000000 00000000 00000000 00000000 ................ ................

r14 0x303beb26
303beb16: 0017e3f0 f0180017 e3f0f160 00170def ...........`.... ..T00...T01-....
303beb26: 94efc3cc e3104978 0016b902 00114780 ......Ix......G. m.C.T...........
303beb36: 8158e3b0 81b60004 47f0815c b98200bb .X......G..\.... a.T.a....0a*.b..
303beb46: 9180c3cc 47e081a2 d5034920 81ce47b0 ....G.....I ..G. j.C..\asN...a...
303beb56: 81a2947f c3cc9610 c3cc4110 4984e310 ..........A.I... asm"C.o.C....dT.
303beb66: 49280024 41104928 e3f00010 0017e3f0 I(.$A.I(........ ........T0....T0
303beb76: f2200017 e3f0f018 0017e3f0 f1140017 . .............. 2...T00...T01...
303beb86: 0def94ef c3ccb904 003beb4f 48000004 .........;.OH... ..m.C......|....

r15 0x1300
12f0: ******** ******** ******** ******** ................ ................
1300: ******** ******** ******** ******** ................ ................
1310: ******** ******** ******** ******** ................ ................
1320: ******** ******** ******** ******** ................ ................
1330: ******** ******** ******** ******** ................ ................
1340: ******** ******** ******** ******** ................ ................
1350: ******** ******** ******** ******** ................ ................
1360: ******** ******** ******** ******** ................ ................

link 0x303beb26
303beb16: 0017e3f0 f0180017 e3f0f160 00170def ...........`.... ..T00...T01-....
303beb26: 94efc3cc e3104978 0016b902 00114780 ......Ix......G. m.C.T...........
303beb36: 8158e3b0 81b60004 47f0815c b98200bb .X......G..\.... a.T.a....0a*.b..
303beb46: 9180c3cc 47e081a2 d5034920 81ce47b0 ....G.....I ..G. j.C..\asN...a...
303beb56: 81a2947f c3cc9610 c3cc4110 4984e310 ..........A.I... asm"C.o.C....dT.
303beb66: 49280024 41104928 e3f00010 0017e3f0 I(.$A.I(........ ........T0....T0
303beb76: f2200017 e3f0f018 0017e3f0 f1140017 . .............. 2...T00...T01...
303beb86: 0def94ef c3ccb904 003beb4f 48000004 .........;.OH... ..m.C......|....
ar0  0x0        ar1  0x0
ar2  0x0        ar3  0x0
ar4  0x0        ar5  0x0
ar6  0x0        ar7  0x0
ar8  0x0        ar9  0x0
ar10 0x0        ar11 0x300a4bfa
ar12 0x0        ar13 0x0
ar14 0x0        ar15 0x0
psalaa 0x7f69edc0
lca64 0x5000100000
savstack 0x50082ef9f8
lestack 0xc0003f7060
c0003f6fe0: 00000000 00000000 00000000 00000000 ................ ................
c0003f6ff0: 00000000 00000000 00000000 00000000 ................ ................
c0003f7000: 00000000 00000000 00000000 00000000 ................ ................
c0003f7010: 00000000 00000000 00000000 00000000 ................ ................
c0003f7020: 00000000 00000000 00000000 00000000 ................ ................
c0003f7030: 00000000 00000000 00000000 00000000 ................ ................
c0003f7040: 00000000 00000000 00000000 00000000 ................ ................
c0003f7050: 00000000 00000000 00000000 00000000 ................ ................
c0003f7060: 000000c0 003f6a60 00000000 30bddc40 .....?j`....0..@ ...{...-.....].
c0003f7070: 00000000 308dacd8 00000000 306dbafa ....0.......0m.. .......Q....._..
c0003f7080: 00000000 30bd47c0 00000000 7f69edc0 ....0.G......i.. .....].{...."..{
c0003f7090: 00000000 0000004f 00000050 0010bab8 .......O...P.... .......|...&....
c0003f70a0: 000000c0 004078be 00000000 00000000 .....@x......... ...{. ..........
c0003f70b0: 00000050 0851e320 00000000 00000000 ...P.Q. ........ ...&..T.........
c0003f70c0: 00000050 00108398 00000050 0010ba98 ...P.......P.... ...&..cq...&...q
c0003f70d0: 00000000 00000000 00000000 00000000 ................ ................
**potential caller
c000407a80: 00000000 00000000 00000000 00000000 ................ ................

goroutine 1 gp=0xc0000041c0 m=0 mp=0x301db2f8 [running]:
runtime: g 1 gp=0xc0000041c0: unknown pc 0x303beb26
stack: frame={sp:0x1300, fp:0x0} stack=[0xc00049c000,0xc0004ac000)

runtime: g 1 gp=0xc0000041c0: unknown pc 0x303beb26
stack: frame={sp:0x1300, fp:0x0} stack=[0xc00049c000,0xc0004ac000)


goroutine 2 gp=0xc000004700 m=nil [force gc (idle)]:
runtime.gopark(0x301253f0, 0x301da6a8, 0x11, 0xa, 0x1)
        <go-install-path>/go/latest/src/runtime/proc.go:432 +0x148 fp=0xc000466fb0 sp=0xc000466f98 pc=0x3009c910
runtime.goparkunlock(...)
        <go-install-path>/go/latest/src/runtime/proc.go:438
runtime.forcegchelper()
        <go-install-path>/go/latest/src/runtime/proc.go:345 +0xd2 fp=0xc000466fd8 sp=0xc000466fb0 pc=0x3005ef3a
runtime.goexit({})
        <go-install-path>/go/latest/src/runtime/asm_zos_s390x.s:800 +0x2 fp=0xc000466fd8 sp=0xc000466fd8 pc=0x300a41ca
created by runtime.init.6 in goroutine 1
        <go-install-path>/go/latest/src/runtime/proc.go:333 +0x30

goroutine 3 gp=0xc000004c40 m=nil [GC sweep wait]:
runtime.gopark(0x301253f0, 0x301da8b8, 0xc, 0x9, 0x1)
        <go-install-path>/go/latest/src/runtime/proc.go:432 +0x148 fp=0xc000467790 sp=0xc000467778 pc=0x3009c910
runtime.goparkunlock(...)
        <go-install-path>/go/latest/src/runtime/proc.go:438
runtime.bgsweep(0xc000470000)
        <go-install-path>/go/latest/src/runtime/mgcsweep.go:277 +0xaa fp=0xc0004677c8 sp=0xc000467790 pc=0x300450e2
runtime.gcenable.gowrap1()
        <go-install-path>/go/latest/src/runtime/mgc.go:203 +0x5e fp=0xc0004677d8 sp=0xc0004677c8 pc=0x30036c56
runtime.goexit({})
        <go-install-path>/go/latest/src/runtime/asm_zos_s390x.s:800 +0x2 fp=0xc0004677d8 sp=0xc0004677d8 pc=0x300a41ca
created by runtime.gcenable in goroutine 1
        <go-install-path>/go/latest/src/runtime/mgc.go:203 +0xa8

goroutine 4 gp=0xc000004e00 m=nil [GC scavenge wait]:
runtime.gopark(0x301253f0, 0x301daa78, 0xd, 0xa, 0x2)
        <go-install-path>/go/latest/src/runtime/proc.go:432 +0x148 fp=0xc000467f80 sp=0xc000467f68 pc=0x3009c910
runtime.goparkunlock(...)
        <go-install-path>/go/latest/src/runtime/proc.go:438
runtime.(*scavengerState).park(0x301daa78)
        <go-install-path>/go/latest/src/runtime/mgcscavenge.go:425 +0x72 fp=0xc000467fa8 sp=0xc000467f80 pc=0x3004234a
runtime.bgscavenge(0xc000470000)
        <go-install-path>/go/latest/src/runtime/mgcscavenge.go:658 +0x5a fp=0xc000467fc8 sp=0xc000467fa8 pc=0x300429f2
runtime.gcenable.gowrap2()
        <go-install-path>/go/latest/src/runtime/mgc.go:204 +0x5e fp=0xc000467fd8 sp=0xc000467fc8 pc=0x30036bd6
runtime.goexit({})
        <go-install-path>/go/latest/src/runtime/asm_zos_s390x.s:800 +0x2 fp=0xc000467fd8 sp=0xc000467fd8 pc=0x300a41ca
created by runtime.gcenable in goroutine 1
        <go-install-path>/go/latest/src/runtime/mgc.go:204 +0x10e

goroutine 5 gp=0xc000005340 m=nil [finalizer wait]:
runtime.gopark(0x30125168, 0x30209218, 0x10, 0xa, 0x1)
        <go-install-path>/go/latest/src/runtime/proc.go:432 +0x148 fp=0xc000466718 sp=0xc000466700 pc=0x3009c910
runtime.runfinq()
        <go-install-path>/go/latest/src/runtime/mfinal.go:193 +0x13a fp=0xc0004667d8 sp=0xc000466718 pc=0x30035782
runtime.goexit({})
        <go-install-path>/go/latest/src/runtime/asm_zos_s390x.s:800 +0x2 fp=0xc0004667d8 sp=0xc0004667d8 pc=0x300a41ca
created by runtime.createfing in goroutine 1
        <go-install-path>/go/latest/src/runtime/mfinal.go:163 +0x64

SIGKILL

[ccw-1]

When C/IO is used on data sets, it can generate SIGIOERR, I have updated the example to ignore this signal, error will be returned instead.

[DNikolaevAtRocket]

@ccw-1,

Thanks for looking into this.
However, it does not fix the issue.
It's still sporadic, but a SIGILL is guarantied for about 20 runs in my environment.
I assume It somehow depends on the current storage layout, and sometimes it starts occurring every second run.

Can you please re-open it?

[ccw-1]

@DNikolaevAtRocket do you have a small test case that I can repeat? for example
./testpds -t "rb,type=record" -i "//'DATASET(MEMBER)'"

For the data set that fails for you, what are the characteristics? (e.g. record size, format etc)

[DNikolaevAtRocket]

@ccw-1,

The default args should do.
I use the command to re-create it:
seq 1000 | xargs -I{} ./testpds (using both seq and xargs from zopen)

I've tested it with different types of datasets, and apparently it doesn't matter.
E.g. it occurs when reading SYS1.MACLIB(EXCP) exactly the same way it does with huge VBS datasets holding SMF data.

Also, I tried it on different LPAR's, and probability of the issue varies significantly, though it's never zero in any of my environments.

If you have any ideas regarding the directions to dig into, I'd be glad to give it a try.

[ccw-1]

@DNikolaevAtRocket
I tested with seq 100000 | xargs -I{} ./testpds without any failure.

This is built with 1.23.4, I wonder if you can try this and see if you can reproduce the problem.
testpds