jwt.Parse Crashes Under Load in Custom Tyk Plugin

[gurpartap2020]

Hi Team,

I am using the jwt.Parse function to validate tokens in a custom Tyk plugin. Under high load, the code frequently crashes, causing the gateway to fail. This issue occurs when multiple requests are sent concurrently, and the plugin tries to validate tokens. It seems related to unhandled panics or resource management problems.

Thanks
GP
log-error.txt

[oxisto]

We cannot really help you without you sharing a code example. I am not familiar with this framework. Maybe this issue is more likely to be handled by them?

[gurpartap2020]

Thanks oxisto for replay

func checkToken(tokenString string, key *rsa.PublicKey) bool {
authHeader := strings.TrimPrefix(tokenString, "Bearer ")
var token, err = jwt.Parse(authHeader, func(token *jwt.Token) (interface{}, error) {
if _, ok := token.Method.(*jwt.SigningMethodRSA); !ok {
return nil, fmt.Errorf("unexpected signing method: %s", "alg")
}
return key, nil
})

if err != nil {
	if errors.Is(err, jwt.ErrTokenExpired) {
		logger.Error("Token is expired")
		tok = "Token is expired"
	} else {
		logger.Error("Failed to parse JWT: ", err)
		tok = "Failed to parse JWT"
	}
	return false
}
if !token.Valid {
	logger.Error("Token is invalid")
	tok = "Token is invalid"
	return false
}
return true

}

[mfridman]

If you have panic recovery and logging, do you have a stack trace showing which line is causing a panic?

Do you have any profiling or basic metrics, such as memory usage, goroutine counts, CPU utilization, etc.?

I'm afraid this is a bit of a goose chase without additional information. The most we can do (and probably should at some point) is benchmark and optimize a parts of the codebase.

[oxisto]

If you have panic recovery and logging, do you have a stack trace showing which line is causing a panic?

Do you have any profiling or basic metrics, such as memory usage, goroutine counts, CPU utilization, etc.?

I'm afraid this is a bit of a goose chase without additional information. The most we can do (and probably should at some point) is benchmark and optimize a parts of the codebase.

The crash is in the attached log file in the first post

fatal error: unknown caller pc
runtime stack:
runtime.throw({0x7c5e292f6b93?, 0x7c5e29b7ee00?})
	runtime/panic.go:1023 +0x5e fp=0x7c5e4e7fbad0 sp=0x7c5e4e7fbaa0 pc=0x7c5e2892557e
runtime.(*unwinder).next(0x7c5e4e7fbb80)
	runtime/traceback.go:469 +0x24c fp=0x7c5e4e7fbb48 sp=0x7c5e4e7fbad0 pc=0x7c5e2894b28c
runtime.copystack(0xc000cafdc0, 0x7c5eeb7aa6ed?)
	runtime/stack.go:930 +0x2fb fp=0x7c5e4e7fbc40 sp=0x7c5e4e7fbb48 pc=0x7c5e2893fb7b
runtime.shrinkstack(0xc000cafdc0)
	runtime/stack.go:1214 +0x117 fp=0x7c5e4e7fbc60 sp=0x7c5e4e7fbc40 pc=0x7c5e28940a97
runtime.newstack()
	runtime/stack.go:1062 +0x39b fp=0x7c5e4e7fbe10 sp=0x7c5e4e7fbc60 pc=0x7c5e2894005b
runtime.morestack()
	runtime/asm_amd64.s:616 +0x77 fp=0x7c5e4e7fbe18 sp=0x7c5e4e7fbe10 pc=0x7c5e289552f7
goroutine 425 gp=0xc000cafdc0 m=15 mp=0xc001300008 [running]:
encoding/json.(*decodeState).scanWhile(0xc004d62240, 0x9)
	encoding/json/decode.go:292 +0xd9 fp=0xc0053296f0 sp=0xc0053296e8 pc=0x7c5e28b236b9
encoding/json.(*decodeState).object(0xc004d62240, {0x49706e0?, 0xc000bce2c8?, 0x7c5ee0f57958?})
	encoding/json/decode.go:662 +0x765 fp=0xc0053299a8 sp=0xc0053296f0 pc=0x7c5e28b24f05
encoding/json.(*decodeState).value(0xc004d62240, {0x49706e0?, 0xc000bce2c8?, 0x0?})
	encoding/json/decode.go:374 +0x3e fp=0xc005329a18 sp=0xc0053299a8 pc=0x7c5e28b2389e
encoding/json.(*decodeState).unmarshal(0xc004d62240, {0x49706e0?, 0xc000bce2c8?})
	encoding/json/decode.go:181 +0x11f fp=0xc005329a90 sp=0xc005329a18 pc=0x7c5e28b22fff
encoding/json.Unmarshal({0xc0046ead80, 0x234, 0x234}, {0x49706e0, 0xc000bce2c8})
	encoding/json/decode.go:108 +0x105 fp=0xc005329ac8 sp=0xc005329a90 pc=0x7c5e28b22be5
github.com/golang-jwt/jwt/v5.(*Parser).ParseUnverified(0xc004d07aa0, {0xc004d64007, 0x4c0}, {0x54d44b8, 0xc004d07ad0})
	github.com/golang-jwt/jwt/[email protected]/parser.go:169 +0x358 fp=0xc005329b70 sp=0xc005329ac8 pc=0x7c5e292d6938
github.com/golang-jwt/jwt/v5.(*Parser).ParseWithClaims(0xc004d07aa0, {0xc004d64007?, 0xc004717400?}, {0x54d44b8, 0xc004d07ad0}, 0xc005329cd0)
	github.com/golang-jwt/jwt/[email protected]/parser.go:56 +0x47 fp=0xc005329c30 sp=0xc005329b70 pc=0x7c5e292d5fe7
github.com/golang-jwt/jwt/v5.(*Parser).Parse(...)
	github.com/golang-jwt/jwt/[email protected]/parser.go:45
github.com/golang-jwt/jwt/v5.Parse({0xc004d64007, 0x4c0}, 0xc002a45cd0, {0x0, 0x0, 0x45403c0?})
	github.com/golang-jwt/jwt/[email protected]/parser.go:226 +0xd2 fp=0xc005329c80 sp=0xc005329c30 pc=0x7c5e292d6ef2

For me this looks like a bug in Go / encoding.json

[mfridman]

Ah gotcha, I didn't open the attached .txt file so missed this 😅