Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: clean up configurations #3331

Open
wants to merge 32 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
4238440
feat: Add `OPERATOR` tier
victor-yanev Dec 9, 2024
937ad01
fix: build error
victor-yanev Dec 9, 2024
385d86c
fix: remove duplicated fetch of remaining budget
victor-yanev Dec 9, 2024
09d99eb
chore: improve readability
victor-yanev Dec 9, 2024
41fd3a5
fix: make sure that operator address is always up-to-date with the cl…
victor-yanev Dec 10, 2024
0344758
chore: formatting
victor-yanev Dec 10, 2024
4f8d625
chore: formatting
victor-yanev Dec 10, 2024
0307f32
chore: fix tests
victor-yanev Dec 10, 2024
44823cd
fix: do not ignore error when EVM address is not provided to `addExpe…
victor-yanev Dec 10, 2024
0f4a4e7
chore: revert unnecessary change
victor-yanev Dec 10, 2024
006de0f
chore: revert unnecessary change
victor-yanev Dec 10, 2024
4dd94b6
chore: reduce code duplication in metricService.spec.ts
victor-yanev Dec 10, 2024
1f4faf2
chore: reduce code duplication in metricService.spec.ts
victor-yanev Dec 10, 2024
ab5e636
fix: error in charts:install workflow
victor-yanev Dec 10, 2024
d30cec3
fix: hbarLimiter.spec.ts
victor-yanev Dec 10, 2024
f71eaf6
fix: remove unnecessary rest in hbarLimiter.spec.ts
victor-yanev Dec 10, 2024
83cf336
refactor: clean up configurations
victor-yanev Dec 12, 2024
8fafaa6
refactor: clean up configurations
victor-yanev Dec 12, 2024
96eb67f
refactor: fix acceptance-workflow.yml
victor-yanev Dec 12, 2024
7764413
chore: prepend 0x to operator address
victor-yanev Dec 16, 2024
db14ad6
test: extend assertions in hbarLimitService.spec.ts
victor-yanev Dec 16, 2024
bbd9c01
Merge branch '3099-Add-operator-tier' into clean-up-configuration-spa…
victor-yanev Dec 17, 2024
222ca25
fix: acceptance tests
victor-yanev Dec 17, 2024
39bd04b
fix: acceptance tests
victor-yanev Dec 17, 2024
dadab2f
Merge branch 'main' into clean-up-configuration-spaghetti
victor-yanev Dec 17, 2024
c00bb51
fix: build error after rebase from `main`
victor-yanev Dec 17, 2024
11d453e
fix: acceptance-workflow.yml
victor-yanev Dec 17, 2024
dd99a31
test: add tests for `Utils.getOperator`
victor-yanev Dec 18, 2024
c75d56c
test: extend tests in hbarLimitService.spec.ts with cases where `OPER…
victor-yanev Dec 18, 2024
529bad8
fix: utils.spec.ts
victor-yanev Dec 18, 2024
8b3d4bf
fix: unit tests
victor-yanev Dec 19, 2024
21225b9
fix: acceptance-public.yml
victor-yanev Dec 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .github/workflows/acceptance-public.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@ on:
- testnet
- previewnet

secrets:
operator_key: ${{ inputs.operator_key }}

jobs:
release-tests:
name: Release Tests
Expand All @@ -29,8 +32,9 @@ jobs:
envfile: ${{ inputs.network }}Acceptance.env
operator_id: ${{ inputs.operator_id }}
maxAttempts: 1
env:
OPERATOR_KEY_MAIN: ${{ secrets.operator_key }}
secrets:
operator_key: ${{ inputs.operator_key }}
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

publish_results:
Expand Down
15 changes: 5 additions & 10 deletions .github/workflows/acceptance-workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,27 +31,23 @@ on:
type: number
default: 3
secrets:
operator_key:
description: 'The ED25519, ECDSA, or DER encoded private key of the operator'
required: false
CODECOV_TOKEN:
description: 'Codecov upload token'
required: true


env:
OPERATOR_ID_MAIN: ${{ inputs.operator_id }}

jobs:
acceptance-workflow:
runs-on: smart-contracts-linux-large
timeout-minutes: 50
permissions:
contents: read
checks: write
env:
OPERATOR_ID_MAIN: ${{ inputs.operator_id }}
steps:
- name: Set env variables
mishomihov00 marked this conversation as resolved.
Show resolved Hide resolved
run: |
if [ -n "${{ inputs.operator_id }}" ]; then
echo "OPERATOR_ID_MAIN=${{ inputs.operator_id }}" >> $GITHUB_ENV
fi
- name: Harden Runner
uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0
with:
Expand Down Expand Up @@ -104,7 +100,6 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_PR_NUMBER: ${{ github.event.number }}
GITHUB_REPOSITORY: ${{ github.repository }}
OPERATOR_KEY_MAIN: ${{ secrets.operator_key }}

- name: Upload Heap Snapshots
if: ${{ !cancelled() }}
Expand Down
1 change: 0 additions & 1 deletion charts/hedera-json-rpc-relay/environments/minikube.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,3 @@ config:
rolling_restart:
enabled: true
schedule: "0 0 * * *"

2 changes: 1 addition & 1 deletion charts/hedera-json-rpc-relay/templates/secret.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@ stringData:
OPERATOR_ID_MAIN: {{ .Values.config.OPERATOR_ID_MAIN | quote }}
OPERATOR_KEY_MAIN: {{ .Values.config.OPERATOR_KEY_MAIN | quote }}
OPERATOR_ID_ETH_SENDRAWTRANSACTION: {{ .Values.config.OPERATOR_ID_ETH_SENDRAWTRANSACTION | default (printf "%q" "") }}
OPERATOR_KEY_ETH_SENDRAWTRANSACTION: {{ .Values.config.OPERATOR_KEY_ETH_SENDRAWTRANSACTION | default (printf "%q" "") }}
OPERATOR_KEY_ETH_SENDRAWTRANSACTION: {{ .Values.config.OPERATOR_KEY_ETH_SENDRAWTRANSACTION | default (printf "%q" "") }}
8 changes: 4 additions & 4 deletions packages/config-service/src/services/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
import dotenv from 'dotenv';
import findConfig from 'find-config';
import pino from 'pino';

import { LoggerService } from './loggerService';
import { ValidationService } from './validationService';

Expand Down Expand Up @@ -61,13 +62,12 @@ export class ConfigService {
private constructor() {
const configPath = findConfig(ConfigService.envFileName);

if (!configPath) {
if (configPath) {
dotenv.config({ path: configPath });
} else {
logger.warn('No .env file is found. The relay cannot operate without valid .env.');
}

// @ts-ignore
dotenv.config({ path: configPath });

// validate mandatory fields
ValidationService.startUp(process.env);

Expand Down
10 changes: 5 additions & 5 deletions packages/relay/src/lib/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -156,17 +156,17 @@ export default {
},

// @ts-ignore
HBAR_RATE_LIMIT_DURATION: parseInt(ConfigService.get('HBAR_RATE_LIMIT_DURATION') || '86400000'), // 1 day
HBAR_RATE_LIMIT_DURATION: parseInt(ConfigService.get('HBAR_RATE_LIMIT_DURATION')),
// @ts-ignore
// The logical OR operator || returns the first truthy value and 0 is falsy.
// The nullish coalescing operator ?? falls back to the default value when the left-hand operand is null or undefined, not when it's 0 or any other falsy value.
HBAR_RATE_LIMIT_TOTAL: BigNumber(ConfigService.get('HBAR_RATE_LIMIT_TINYBAR') ?? '800000000000'), // 8000 HBARs
HBAR_RATE_LIMIT_TOTAL: BigNumber(ConfigService.get('HBAR_RATE_LIMIT_TINYBAR')),
// @ts-ignore
HBAR_RATE_LIMIT_BASIC: BigNumber(ConfigService.get('HBAR_RATE_LIMIT_BASIC') || '1120000000'), // 11.2 HBARs
HBAR_RATE_LIMIT_BASIC: BigNumber(ConfigService.get('HBAR_RATE_LIMIT_BASIC')),
// @ts-ignore
HBAR_RATE_LIMIT_EXTENDED: BigNumber(ConfigService.get('HBAR_RATE_LIMIT_EXTENDED') || '3200000000'), // 32 HBARs
HBAR_RATE_LIMIT_EXTENDED: BigNumber(ConfigService.get('HBAR_RATE_LIMIT_EXTENDED')),
// @ts-ignore
HBAR_RATE_LIMIT_PRIVILEGED: BigNumber(ConfigService.get('HBAR_RATE_LIMIT_PRIVILEGED') || '8000000000'), // 80 HBARs
HBAR_RATE_LIMIT_PRIVILEGED: BigNumber(ConfigService.get('HBAR_RATE_LIMIT_PRIVILEGED')),
// @ts-ignore
GAS_PRICE_TINY_BAR_BUFFER: parseInt(ConfigService.get('GAS_PRICE_TINY_BAR_BUFFER') || '10000000000'),
WEB_SOCKET_PORT: ConfigService.get('WEB_SOCKET_PORT') || 8546,
Expand Down
5 changes: 0 additions & 5 deletions packages/relay/src/lib/relay.ts
Original file line number Diff line number Diff line change
Expand Up @@ -134,9 +134,7 @@ export class RelayImpl implements Relay {
const hederaNetwork: string = (ConfigService.get('HEDERA_NETWORK') || '{}').toLowerCase();
const configuredChainId = ConfigService.get('CHAIN_ID') || constants.CHAIN_IDS[hederaNetwork] || '298';
const chainId = prepend0x(Number(configuredChainId).toString(16));

const duration = constants.HBAR_RATE_LIMIT_DURATION;
const total = constants.HBAR_RATE_LIMIT_TOTAL;

this.eventEmitter = new EventEmitter();
const reservedKeys = HbarSpendingPlanConfigService.getPreconfiguredSpendingPlanKeys(logger);
Expand Down Expand Up @@ -166,9 +164,6 @@ export class RelayImpl implements Relay {
const hapiService = new HAPIService(logger, register, this.cacheService, this.eventEmitter, hbarLimitService);

this.clientMain = hapiService.getMainClientInstance();
if (this.clientMain.operatorAccountId) {
hbarLimitService.setOperatorAddress(this.clientMain.operatorAccountId.toSolidityAddress());
}

this.web3Impl = new Web3Impl(this.clientMain);
this.netImpl = new NetImpl(this.clientMain);
Expand Down
48 changes: 6 additions & 42 deletions packages/relay/src/lib/services/hapiService/hapiService.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,8 @@
*/

import { ConfigService } from '@hashgraph/json-rpc-config-service/dist/services';
import { AccountId, Client, PrivateKey } from '@hashgraph/sdk';
import dotenv from 'dotenv';
import { Client } from '@hashgraph/sdk';
import EventEmitter from 'events';
import findConfig from 'find-config';
import fs from 'fs';
import { Logger } from 'pino';
import { Counter, Registry } from 'prom-client';

Expand Down Expand Up @@ -194,18 +191,10 @@ export default class HAPIService {
eventEmitter: EventEmitter,
hbarLimitService: HbarLimitService,
) {
dotenv.config({ path: findConfig('.env') || '' });
if (fs.existsSync(findConfig('.env') || '')) {
this.config = dotenv.parse(fs.readFileSync(findConfig('.env') || ''));
} else {
this.config = {};
}

this.logger = logger;

this.hbarLimitService = hbarLimitService;
this.eventEmitter = eventEmitter;
this.hederaNetwork = (ConfigService.get('HEDERA_NETWORK') || this.config.HEDERA_NETWORK || '{}').toLowerCase();
this.hederaNetwork = ((ConfigService.get('HEDERA_NETWORK') as string) || '{}').toLowerCase();
this.clientMain = this.initClient(logger, this.hederaNetwork);

this.cacheService = cacheService;
Expand Down Expand Up @@ -290,9 +279,6 @@ export default class HAPIService {
this.clientMain = this.initClient(this.logger, this.hederaNetwork);
this.client = this.initSDKClient(this.logger);
this.resetCounters();
if (this.clientMain.operatorAccountId) {
this.hbarLimitService.setOperatorAddress(this.clientMain.operatorAccountId.toSolidityAddress());
}
}

/**
Expand Down Expand Up @@ -328,38 +314,16 @@ export default class HAPIService {
* @returns Client
*/
private initClient(logger: Logger, hederaNetwork: string, type: string | null = null): Client {
let client: Client, privateKey: PrivateKey;
let client: Client;
if (hederaNetwork in constants.CHAIN_IDS) {
client = Client.forName(hederaNetwork);
} else {
client = Client.forNetwork(JSON.parse(hederaNetwork));
}

if (type === 'eth_sendRawTransaction') {
if (
ConfigService.get('OPERATOR_ID_ETH_SENDRAWTRANSACTION') &&
ConfigService.get('OPERATOR_KEY_ETH_SENDRAWTRANSACTION')
) {
// @ts-ignore
privateKey = Utils.createPrivateKeyBasedOnFormat(ConfigService.get('OPERATOR_KEY_ETH_SENDRAWTRANSACTION'));
client = client.setOperator(
// @ts-ignore
AccountId.fromString(ConfigService.get('OPERATOR_ID_ETH_SENDRAWTRANSACTION')),
privateKey,
);
} else {
logger.warn(`Invalid 'ETH_SENDRAWTRANSACTION' env variables provided`);
}
} else {
const operatorId: string = ConfigService.get('OPERATOR_ID_MAIN') || this.config.OPERATOR_ID_MAIN || '';
const operatorKey: string = ConfigService.get('OPERATOR_KEY_MAIN') || this.config.OPERATOR_KEY_MAIN || '';

if (operatorId && operatorKey) {
privateKey = Utils.createPrivateKeyBasedOnFormat(operatorKey);
client = client.setOperator(AccountId.fromString(operatorId.trim()), privateKey);
} else {
logger.warn(`Invalid 'OPERATOR' env variables provided`);
}
const operator = Utils.getOperator(logger, type);
if (operator) {
client.setOperator(operator.accountId, operator.privateKey);
}

// @ts-ignore
Expand Down
31 changes: 11 additions & 20 deletions packages/relay/src/lib/services/hbarLimitService/index.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
/*
/*-
*
* Hedera JSON RPC Relay
*
* Copyright (C) 2022-2024 Hedera Hashgraph, LLC
* Copyright (C) 2024 Hedera Hashgraph, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand All @@ -18,7 +18,7 @@
*
*/

import { ConfigService } from '@hashgraph/json-rpc-config-service/dist/services';
import { zeroAddress } from '@ethereumjs/util';
import { AccountId, Hbar } from '@hashgraph/sdk';
import { Logger } from 'pino';
import { Counter, Gauge, Registry } from 'prom-client';
Expand Down Expand Up @@ -87,7 +87,7 @@ export class HbarLimitService implements IHbarLimitService {
* The operator address for the rate limiter.
* @private
*/
private operatorAddress?: string;
private readonly operatorAddress: string;

constructor(
private readonly hbarSpendingPlanRepository: HbarSpendingPlanRepository,
Expand All @@ -99,12 +99,11 @@ export class HbarLimitService implements IHbarLimitService {
) {
this.reset = this.getResetTimestamp();

const operatorId = ConfigService.get('OPERATOR_ID_MAIN');
const operatorKey = ConfigService.get('OPERATOR_KEY_MAIN');
if (operatorId) {
this.setOperatorAddress(AccountId.fromString(operatorId as string).toSolidityAddress());
} else if (operatorKey) {
this.setOperatorAddress(Utils.createPrivateKeyBasedOnFormat(operatorKey as string).publicKey.toEvmAddress());
const operator = Utils.getOperator(logger);
if (operator) {
this.operatorAddress = prepend0x(AccountId.fromString(operator.accountId.toString()).toSolidityAddress());
} else {
this.operatorAddress = zeroAddress();
}
quiet-node marked this conversation as resolved.
Show resolved Hide resolved

const totalBudget = HbarLimitService.TIER_LIMITS[SubscriptionTier.OPERATOR];
Expand Down Expand Up @@ -172,14 +171,6 @@ export class HbarLimitService implements IHbarLimitService {
return this.isHBarRateLimiterEnabled;
}

/**
* Sets the operator address for the rate limiter. Used for tracking operator expenses.
* @param {string} operatorAddress - The EVM address of the operator.
*/
setOperatorAddress(operatorAddress: string) {
this.operatorAddress = prepend0x(operatorAddress);
}

/**
* Resets the {@link HbarSpendingPlan#amountSpent} field for all existing plans.
* @param {RequestDetails} requestDetails - The request details used for logging and tracking.
Expand Down Expand Up @@ -555,11 +546,11 @@ export class HbarLimitService implements IHbarLimitService {
* @private
*/
private async getOperatorSpendingPlan(requestDetails: RequestDetails): Promise<IDetailedHbarSpendingPlan> {
let operatorPlan = await this.getSpendingPlan(this.operatorAddress!, requestDetails);
let operatorPlan = await this.getSpendingPlan(this.operatorAddress, requestDetails);
if (!operatorPlan) {
this.logger.trace(`${requestDetails.formattedRequestId} Creating operator spending plan...`);
operatorPlan = await this.createSpendingPlanForAddress(
this.operatorAddress!,
this.operatorAddress,
requestDetails,
SubscriptionTier.OPERATOR,
);
Expand Down
33 changes: 32 additions & 1 deletion packages/relay/src/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,11 @@
*/

import { ConfigService } from '@hashgraph/json-rpc-config-service/dist/services';
import { PrivateKey } from '@hashgraph/sdk';
import { AccountId, PrivateKey } from '@hashgraph/sdk';
import { Operator } from '@hashgraph/sdk/lib/client/Client';
import crypto from 'crypto';
import createHash from 'keccak';
import { Logger } from 'pino';

import { hexToASCII, prepend0x, strip0x } from './formatters';
import constants from './lib/constants';
Expand Down Expand Up @@ -136,4 +138,33 @@ export class Utils {
public static computeTransactionHash(transactionBuffer: Buffer): string {
return prepend0x(createHash('keccak256').update(transactionBuffer).digest('hex'));
}

/**
* Gets operator credentials based on the provided type.
* @param {Logger} logger - The logger instance
* @param {string | null} type - The type of operator (e.g. 'eth_sendRawTransaction')
* @returns {Operator | null} The operator credentials or null if not found
*/
public static getOperator(logger: Logger, type: string | null = null): Operator | null {
let operatorId: string;
let operatorKey: string;

if (type === 'eth_sendRawTransaction') {
operatorId = ConfigService.get('OPERATOR_ID_ETH_SENDRAWTRANSACTION') as string;
operatorKey = ConfigService.get('OPERATOR_KEY_ETH_SENDRAWTRANSACTION') as string;
} else {
operatorId = ConfigService.get('OPERATOR_ID_MAIN') as string;
operatorKey = ConfigService.get('OPERATOR_KEY_MAIN') as string;
}

if (!operatorId || !operatorKey) {
logger.warn(`Invalid operatorId or operatorKey for ${type ?? 'main'} client.`);
return null;
}

return {
privateKey: Utils.createPrivateKeyBasedOnFormat(operatorKey),
accountId: AccountId.fromString(operatorId.trim()),
};
}
}
Loading
Loading