Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Nacos http error, code=403,msg=Invalid signature #12934

Open
abcdocker opened this issue Dec 9, 2024 · 4 comments
Open

Nacos http error, code=403,msg=Invalid signature #12934

abcdocker opened this issue Dec 9, 2024 · 4 comments
Labels
status/need feedback

Comments

@abcdocker
Copy link

abcdocker commented Dec 9, 2024 •
edited
Loading

  • client 2.2.0
  • nacos server 2.3.0
  • nacos cluster 3

Problem description: In the case where all configurations are normal, the client connection to Nacos prompts a user authentication error of 403. The username and password configurations are normal, and there is no user permission issue. After restarting the client multiple times, it occasionally connects. Currently, the troubleshooting approach is a bit confusing. Please provide some troubleshooting ideas

2024-12-05 17:19:06 [background-preinit]  INFO  org.hibernate.validator.internal.util.Version -HV000001: Hibernate Validator 6.2.5.Final
2024-12-05 17:19:06 [main]  INFO  com.alibaba.nacos.client.env.SearchableProperties -properties search order:PROPERTIES->JVM->ENV->DEFAULT_SETTING
2024-12-05 17:19:07 [main]  INFO  com.alibaba.nacos.plugin.auth.spi.client.ClientAuthPluginManager -[ClientAuthPluginManager] Load ClientAuthService com.alibaba.nacos.client.auth.impl.NacosClientAuthServiceImpl success.
2024-12-05 17:19:07 [main]  INFO  com.alibaba.nacos.plugin.auth.spi.client.ClientAuthPluginManager -[ClientAuthPluginManager] Load ClientAuthService com.alibaba.nacos.client.auth.ram.RamClientAuthServiceImpl success.
2024-12-05 17:19:09 [main]  ERROR com.alibaba.cloud.nacos.client.NacosPropertySourceBuilder -get data from Nacos error,dataId:app 
com.alibaba.nacos.api.exception.NacosException: http error, code=403,msg=Invalid signature,dataId=app,group=KK_YY,tenant=ABC
	at com.alibaba.nacos.client.config.impl.ClientWorker$ConfigRpcTransportClient.queryConfig(ClientWorker.java:987)
	at com.alibaba.nacos.client.config.impl.ClientWorker.getServerConfig(ClientWorker.java:404)
	at com.alibaba.nacos.client.config.NacosConfigService.getConfigInner(NacosConfigService.java:184)
	at com.alibaba.nacos.client.config.NacosConfigService.getConfig(NacosConfigService.java:96)
	at com.alibaba.cloud.nacos.client.NacosPropertySourceBuilder.loadNacosData(NacosPropertySourceBuilder.java:85)
	at com.alibaba.cloud.nacos.client.NacosPropertySourceBuilder.build(NacosPropertySourceBuilder.java:73)
	at com.alibaba.cloud.nacos.client.NacosPropertySourceLocator.loadNacosPropertySource(NacosPropertySourceLocator.java:199)
	at com.alibaba.cloud.nacos.client.NacosPropertySourceLocator.loadNacosDataIfPresent(NacosPropertySourceLocator.java:186)
	at com.alibaba.cloud.nacos.client.NacosPropertySourceLocator.loadApplicationConfiguration(NacosPropertySourceLocator.java:141)
	at com.alibaba.cloud.nacos.client.NacosPropertySourceLocator.locate(NacosPropertySourceLocator.java:103)
	at org.springframework.cloud.bootstrap.config.PropertySourceLocator.locateCollection(PropertySourceLocator.java:51)
	at org.springframework.cloud.bootstrap.config.PropertySourceLocator.locateCollection(PropertySourceLocator.java:47)
	at org.springframework.cloud.bootstrap.config.PropertySourceBootstrapConfiguration.initialize(PropertySourceBootstrapConfiguration.java:95)
	at org.springframework.boot.SpringApplication.applyInitializers(SpringApplication.java:618)
	at org.springframework.boot.SpringApplication.prepareContext(SpringApplication.java:385)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:306)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1317)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306)
	at com.abc.mmmm.appmmmmAppApplication.main(appmmmmAppApplication.java:14)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)
2024-12-05 17:19:09 [main]  ERROR com.alibaba.cloud.nacos.client.NacosPropertySourceBuilder -get data from Nacos error,dataId:app-mmmm-app.yml 
com.alibaba.nacos.api.exception.NacosException: http error, code=403,msg=Invalid signature,dataId=app.yml,group=KK_YY,tenant=abc
	at com.alibaba.nacos.client.config.impl.ClientWorker$ConfigRpcTransportClient.queryConfig(ClientWorker.java:987)
	at com.alibaba.nacos.client.config.impl.ClientWorker.getServerConfig(ClientWorker.java:404)
	at com.alibaba.nacos.client.config.NacosConfigService.getConfigInner(NacosConfigService.java:184)
	at com.alibaba.nacos.client.config.NacosConfigService.getConfig(NacosConfigService.java:96)
	at com.alibaba.cloud.nacos.client.NacosPropertySourceBuilder.loadNacosData(NacosPropertySourceBuilder.java:85)
	at com.alibaba.cloud.nacos.client.NacosPropertySourceBuilder.build(NacosPropertySourceBuilder.java:73)
	at com.alibaba.cloud.nacos.client.NacosPropertySourceLocator.loadNacosPropertySource(NacosPropertySourceLocator.java:199)
	at com.alibaba.cloud.nacos.client.NacosPropertySourceLocator.loadNacosDataIfPresent(NacosPropertySourceLocator.java:186)
	at com.alibaba.cloud.nacos.client.NacosPropertySourceLocator.loadApplicationConfiguration(NacosPropertySourceLocator.java:144)
	at com.alibaba.cloud.nacos.client.NacosPropertySourceLocator.locate(NacosPropertySourceLocator.java:103)
	at org.springframework.cloud.bootstrap.config.PropertySourceLocator.locateCollection(PropertySourceLocator.java:51)
	at org.springframework.cloud.bootstrap.config.PropertySourceLocator.locateCollection(PropertySourceLocator.java:47)
	at org.springframework.cloud.bootstrap.config.PropertySourceBootstrapConfiguration.initialize(PropertySourceBootstrapConfiguration.java:95)
	at org.springframework.boot.SpringApplication.applyInitializers(SpringApplication.java:618)
	at org.springframework.boot.SpringApplication.prepareContext(SpringApplication.java:385)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:306)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1317)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306)
	at com.abc.mmmm.appmmmmAppApplication.main(appmmmmAppApplication.java:14)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)

bootstrap.yml

spring:
  cloud:
    nacos:
      config:
        server-addr: 10.1.1.20:8848,10.1.1.21:8848,10.1.1.22:8848
        username: nacos
        password: abc2024!
        namespace: abc
        group: KK_YY
        file-extension: yml
      discovery:
        server-addr: 10.1.1.20:8848,10.1.1.21:8848,10.1.1.22:8848
        username: nacos
        password: abc2024!
        namespace: abc
        group: KK_YY

  application:
    name: app
  profiles:
    active: prod

The configuration used is nacos administrator privileges, and occasional connection issues may occur. The port policy is set to 8848

Server log

##################### access_log #######################
10.11.14.81 - - [05/Dec/2024:17:17:05 +0800] "GET /nacos/v1/cs/configs?accessToken=eyJhbGciOiJIUzM4NCJ9.eyJzdWIiOiJuYWNvcyIsImV4cCI6MTczMzQwODE0MH0.TuoE99RMz6fZm0TmcII3w-sUc6YVHSm7MhobwRV74h3r_JbKZaaBNhu1009tNuur&show=all&dataId=app.yml&group=KK_YY&tenant=abc&namespaceId=abc HTTP/1.1" 200 - 3 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36 Edg/129.0.0.0 -
10.11.14.81 - - [05/Dec/2024:17:17:05 +0800] "POST /nacos/v1/cs/configs?accessToken=eyJhbGciOiJIUzM4NCJ9.eyJzdWIiOiJuYWNvcyIsImV4cCI6MTczMzQwODE0MH0.TuoE99RMz6fZm0TmcII3w-sUc6YVHSm7MhobwRV74h3r_JbKZaaBNhu1009tNuur HTTP/1.1" 200 14 7 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36 Edg/129.0.0.0 -
10.11.14.81 - - [05/Dec/2024:17:17:05 +0800] "GET /nacos/v1/cs/configs?dataId=&group=&appName=&config_tags=&pageNo=1&pageSize=10&tenant=abc&search=blur&accessToken=eyJhbGciOiJIUzM4NCJ9.eyJzdWIiOiJuYWNvcyIsImV4cCI6MTczMzQwODE0MH0.TuoE99RMz6fZm0TmcII3w-sUc6YVHSm7MhobwRV74h3r_JbKZaaBNhu1009tNuur&username=nacos HTTP/1.1" 200 15153 2 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36 Edg/129.0.0.0 -

################## alipay-jraft.log ###############

2024-12-05 17:03:25,236 INFO Truncated prefix logs in data path: /opt/nacos/data/protocol/raft/naming_instance_metadata/log from log index 6 to 6, cost 0 ms.

2024-12-05 17:04:16,303 INFO Truncated prefix logs in data path: /opt/nacos/data/protocol/raft/naming_service_metadata/log from log index 6 to 6, cost 0 ms.

2024-12-05 17:04:29,588 INFO Truncated prefix logs in data path: /opt/nacos/data/protocol/raft/naming_persistent_service/log from log index 6 to 6, cost 0 ms.

2024-12-05 17:27:57,862 INFO Truncated prefix logs in data path: /opt/nacos/data/protocol/raft/naming_persistent_service_v2/log from log index 6 to 6, cost 0 ms.

2024-12-05 17:33:25,236 INFO Truncated prefix logs in data path: /opt/nacos/data/protocol/raft/naming_instance_metadata/log from log index 6 to 6, cost 0 ms.

2024-12-05 17:34:16,303 INFO Truncated prefix logs in data path: /opt/nacos/data/protocol/raft/naming_service_metadata/log from log index 6 to 6, cost 1 ms.

2024-12-05 17:34:29,587 INFO Truncated prefix logs in data path: /opt/nacos/data/protocol/raft/naming_persistent_service/log from log index 6 to 6, cost 0 ms.

2024-12-05 17:57:57,862 INFO Truncated prefix logs in data path: /opt/nacos/data/protocol/raft/naming_persistent_service_v2/log from log index 6 to 6, cost 0 ms.

2024-12-05 18:03:25,237 INFO Truncated prefix logs in data path: /opt/nacos/data/protocol/raft/naming_instance_metadata/log from log index 6 to 6, cost 0 ms.

2024-12-05 18:04:16,303 INFO Truncated prefix logs in data path: /opt/nacos/data/protocol/raft/naming_service_metadata/log from log index 6 to 6, cost 1 ms.

############# config-client-request.log #####################

2024-12-05 17:16:52,843|1|200|10.11.5.18|listen|3|true|||unknown
2024-12-05 17:16:59,353|0|200|10.11.5.21|listen|3|true|||unknown
2024-12-05 17:17:04,064|0|200|10.11.5.22|listen|3|true|||unknown
2024-12-05 17:17:05,486|6|true|10.11.14.81|publish|app.yml|KK_YY|abc|54f6338b5a80e07f7cbc640d9463d6a8|null
2024-12-05 17:17:16,025|0|200|10.11.5.17|listen|4|true|||unknown
2024-12-05 17:17:19,544|1|200|10.11.5.22|listen|6|true|||unknown
2024-12-05 17:17:22,614|1|200|10.11.5.21|listen|4|true|||unknown

################# config-dump.log #######################

2024-12-05 17:17:05,486 INFO [dump] add formal task. groupKey=app.yml+KK_YY+abc

2024-12-05 17:17:05,562 INFO [dump] process formal task. groupKey=app.yml+KK_YY+abc

2024-12-05 17:17:05,566 INFO [dump] md5 changed, save to disk cache ,groupKey=app.yml+KK_YY+abc, newMd5=54f6338b5a80e07f7cbc640d9463d6a8,oldMd5=

2024-12-05 17:17:05,566 INFO [dump] md5 changed, update md5 and timestamp in jvm cache ,groupKey=app.yml+KK_YY+abc, newMd5=54f6338b5a80e07f7cbc640d9463d6a8,oldMd5=,lastModifiedTs=1733390225000

2024-12-05 17:17:11,211 WARN [dump-change-ignore] timestamp is outdated,groupKey=app.yml+KK_YY+abc

######################### config-fatal.log ###########################

2024-12-05 17:21:26,243 WARN aggr dataId whitelist is blank.

2024-12-05 17:21:26,243 WARN switch config is blank.

2024-12-05 17:21:41,245 WARN aggr dataId whitelist is blank.

2024-12-05 17:21:41,245 WARN switch config is blank.

2024-12-05 17:21:56,247 WARN aggr dataId whitelist is blank.

2024-12-05 17:21:56,247 WARN switch config is blank.

2024-12-05 17:22:11,249 WARN aggr dataId whitelist is blank.

2024-12-05 17:22:11,249 WARN switch config is blank.

2024-12-05 17:22:26,251 WARN aggr dataId whitelist is blank.

################ config-memory.log ############

2024-12-05 17:18:32,474 INFO groupCount = 17, subscriberClientCount = 0, subscriberCount = 0

2024-12-05 17:18:32,477 INFO [long-pulling] client count 0

2024-12-05 17:18:42,474 INFO toNotifyTaskSize = 0

2024-12-05 17:18:42,474 INFO toClientNotifyTaskSize = 0

2024-12-05 17:18:42,474 INFO groupCount = 17, subscriberClientCount = 0, subscriberCount = 0

2024-12-05 17:18:42,477 INFO [long-pulling] client count 0

2024-12-05 17:18:52,475 INFO toNotifyTaskSize = 0

2024-12-05 17:18:52,475 INFO toClientNotifyTaskSize = 0

2024-12-05 17:18:52,475 INFO groupCount = 17, subscriberClientCount = 0, subscriberCount = 0

2024-12-05 17:18:52,477 INFO [long-pulling] client count 0

2024-12-05 17:19:02,475 INFO toNotifyTaskSize = 0

############# config-pull-check.log ###############

app.yml+KK_YY+abc|10.11.5.16|54f6338b5a80e07f7cbc640d9463d6a8|2024-12-05 17:38:10

############### config-server.log##############

2024-12-05 17:22:26,251 INFO Check changed configs finished,cost:0,set next start time to 2024-12-05 17:22:26.249

2024-12-05 17:22:41,251 INFO DumpChange start ,from time 2024-12-05 17:22:26.249,current time 2024-12-05 17:22:41.251

2024-12-05 17:22:41,251 INFO Start to check delete configs from  time 2024-12-05 17:22:26.249

2024-12-05 17:22:41,251 INFO Check delete configs from  time 2024-12-05 17:22:26.249

2024-12-05 17:22:41,253 INFO Check delete configs finished,cost:2

2024-12-05 17:22:41,253 INFO Check changeConfig start

2024-12-05 17:22:41,253 INFO Check changed configs from  time 2024-12-05 17:22:26.249,lastMaxId=0

2024-12-05 17:22:41,253 WARN clientIpWhiteList is blank.close whitelist.

################ core-auth.log ###############

2024-12-05 17:27:34,501 DEBUG auth permission: Permission{resource='Resource{namespaceId='abc', group='KK_YY', name='', type='naming', properties={requestClass=ServiceListRequest, action=r}}', action='r'}, nacosUser: NacosUser{token='eyJhbGciOiJIUzM4NCJ9.eyJzdWIiOiJuYWNvcyIsImV4cCI6MTczMzQwNTk3MH0._XXknspX_rRx8-KFhehMBla-0ScYac5Ki2ESHPD-WOowHoN5ytBadiDZA1fX3DDM', globalAdmin=false}

2024-12-05 17:27:34,501 DEBUG auth permission: Permission{resource='Resource{namespaceId='abc', group='KK_YY', name='', type='naming', properties={requestClass=ServiceListRequest, action=r}}', action='r'}, nacosUser: NacosUser{token='eyJhbGciOiJIUzM4NCJ9.eyJzdWIiOiJuYWNvcyIsImV4cCI6MTczMzQwNTk3MH0._XXknspX_rRx8-KFhehMBla-0ScYac5Ki2ESHPD-WOowHoN5ytBadiDZA1fX3DDM', globalAdmin=false}

2024-12-05 17:27:35,361 DEBUG auth start, request: ServiceListRequest

2024-12-05 17:27:35,361 DEBUG auth start, request: ServiceListRequest

##################### nacos.log #####################


2024-12-05 17:25:02,012 INFO Long connection metrics detail ,Total count =40, sdkCount=38,clusterCount=2

2024-12-05 17:25:02,012 INFO Out dated connection ,size=0

2024-12-05 17:25:02,012 INFO Connection check task end

2024-12-05 17:25:04,640 INFO ConnectionMetrics, totalCount = 40, detail = {long_connection=40, long_polling=0}

2024-12-05 17:25:05,012 INFO Connection check task start

2024-12-05 17:25:05,012 INFO Long connection metrics detail ,Total count =40, sdkCount=38,clusterCount=2

2024-12-05 17:25:05,012 INFO Out dated connection ,size=0

2024-12-05 17:25:05,012 INFO Connection check task end
nacos-cluster.log
2024-12-05 17:18:13,111 INFO [serverlist] membercount=3

2024-12-05 17:19:03,219 INFO [serverlist] membercount=3

2024-12-05 17:19:53,333 INFO [serverlist] membercount=3

2024-12-05 17:20:43,448 INFO [serverlist] membercount=3

2024-12-05 17:21:33,560 INFO [serverlist] membercount=3

2024-12-05 17:22:23,671 INFO [serverlist] membercount=3

2024-12-05 17:23:13,784 INFO [serverlist] membercount=3

2024-12-05 17:24:03,897 INFO [serverlist] membercount=3

2024-12-05 17:24:54,016 INFO [serverlist] membercount=3

2024-12-05 17:25:44,132 INFO [serverlist] membercount=3

################# naming-distro.log ###############

2024-12-05 17:23:06,900 INFO PERFORMANCE:|10|28|33|-1|-1|0|0

2024-12-05 17:23:06,900 INFO Task worker status: 
naming_0%8, pending tasks: 0
naming_1%8, pending tasks: 0
naming_2%8, pending tasks: 0
naming_3%8, pending tasks: 0
naming_4%8, pending tasks: 0
naming_5%8, pending tasks: 0
naming_6%8, pending tasks: 0
naming_7%8, pending tasks: 0

##################### naming-push.log##################

2024-12-05 17:44:54,800 INFO [PUSH] Task merge for Service{namespace='abc', group='KK_YY', name='abc-mmmm-app', ephemeral=true, revision=1}

2024-12-05 17:44:55,200 INFO [PUSH-SUCC] 1ms, all delay time 552ms for subscriber 10.1.1.1, Service{namespace='abc', group='KK_YY', name='abc-mmmm-app', ephemeral=true, revision=1}, originalSize=1, DataSize=1
@YunWZ
Copy link
Contributor

YunWZ commented Dec 11, 2024

Well， u can check whether the nacos.core.auth.plugin.nacos.token.secret.key property value of each nacos-server instance is consistent

@KomachiSion
Copy link
Collaborator

What's more， make sure your jdk version support HMAC-SHA

@abcdocker
Copy link
Author

abcdocker commented Dec 12, 2024 •
edited
Loading

@KomachiSion @YunWZ

Thank you for your reply. Below are my test results

Here are my JDK version and HMAC-SHA protocol tests

nacos.core.auth.plugin.nacos.token.secret.key Same cluster configuration

JDK Version

[root@dev-app nacos]# /usr/local/jdk1.8.0_102/bin/java -version
java version "1.8.0_102"
Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)



[root@app nacos]#
[root@app nacos]# /usr/local/jdk1.8.0_102/bin/javac Main.java
[root@app nacos]# ls
config  Main.class  Main.java  naming

JDK HMAC-SHA test

[root@app nacos]# /usr/local/jdk1.8.0_102/bin/java Main
原文：测试加密
密钥：123456
密文13a4fab21f1d1fa871b5db6cfbea6996508ae4fe899ae84c1eefa59e8e4229de

teest file

public class Main {
    public static void main(String[] args) {
        String data = "测试加密";
        String secret = "123456";
        try {
            // Create HMAC-SHA256 key from the given secret
            SecretKeySpec secretKeySpec = new SecretKeySpec(secret.getBytes(), "HmacSHA256");

            // Get an instance of Mac object implementing HMAC-SHA256
            Mac mac = Mac.getInstance("HmacSHA256");
            mac.init(secretKeySpec);

            // Calculate the HMAC value
            byte[] hmacBytes = mac.doFinal(data.getBytes());

            // Convert result into a hexadecimal string
            StringBuilder sb = new StringBuilder(hmacBytes.length * 2);
            for (byte b : hmacBytes) {
                sb.append(String.format("%02x", b));
            }
            System.out.println("原文：" + data);
            System.out.println("密钥：" + secret);
            System.out.println("密文" + sb);
        } catch (Exception e) {
            throw new RuntimeException("Failed to calculate HMAC-SHA256", e);
        }
    }
}

Nacos nacos.core.auth.plugin.nacos.token.secret.key (Same cluster configuration)

nacos.core.auth.plugin.nacos.token.secret.key=SecretKey032112378901234567890123456789012345678901234567890123456111

The nacos server itself has multiple JDK versions, and I have loaded the corresponding version in the log by importing the version through the exporter JAVA_HOME variable in the startup script

JAVA_HOME=/usr/local/jdk1.8.0_102

@KomachiSion
Copy link
Collaborator

If jdk support HmacSHA256 well.

The wrong message only means your token generate with other token.secret.key and request to an diff token.secret.key server.

Please check whether do these:

  1. change the token.secret.key for nacos-server
  2. diff token.secret.key between nacos-server nodes in one cluster.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
status/need feedback
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@YunWZ @abcdocker @KomachiSion