Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support persist instance info collected by CloudInstanceInformationProcessor during auto-reg #2622

Merged
merged 4 commits into from
Sep 21, 2023

Conversation

zpinto
Copy link
Contributor

@zpinto zpinto commented Sep 20, 2023

Add support to persist all instance information collected by CloudInstanceInformationProcessor in CloudInstanceInformation object. Add ability for CloudInstanceInformationProcessor to produce full DOMAIN field instead of appending _instanceName unless last character in CloudInstanceInformation.CloudInstanceField.FAULT_DOMAIN is '='.

Description

Typically the information collected by the CloudInstanceInformationProcessor is used for constructing the DOMAIN field in InstanceConfig. There are some usecases where a customer wants to override the DOMAIN field from what was originally generated from auto-registration. When this happens, all the collected topology information is lost. In order to prevent losing this data, we will persist it in a new field called INSTANCE_INFO_MAP.

Possible usecase steps:

  1. node comes online for first time using auto-reg and defaultInstanceConfig that has HELIX_ENABLED = false
  2. All topology information is collected with CloudInstanceInformationProcessor and returned as CloudInstanceInformation
  3. All information in CloudInstanceInformation is persisted into INSTANCE_INFO_MAP
  4. DOMAIN is generated from CloudInstanceInformation.CloudInstanceField.FAULT_DOMAIN field
  5. node has joined the cluster in disabled state
  6. customers external hook can now run to override the DOMAIN field with whatever they want and set HELIX_ENABLED = true

Tests

  • TestInstanceConfig updated to test new INSTANCE_INFO_MAP field
  • TestParticipantAutoJoin updated to verify that INSTANCE_INFO_MAP is populated and DOMAIN is constructed without appending _instanceName in case that CloudInstanceInformationProcessor constructs the full domain(signified when there is not = at the end of CloudInstanceInformation.CloudInstanceField.FAULT_DOMAIN)
➜  helix git:(support_persist_instance_info) mvn test -o -Dtest=TestInstanceConfig -pl=helix-core 
...
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.helix.model.TestInstanceConfig
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.012 s - in org.apache.helix.model.TestInstanceConfig
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] 
[INFO] --- jacoco:0.8.6:report (generate-code-coverage-report) @ helix-core ---
[INFO] Loading execution data file /Users/zapinto/Documents/git/zpinto/helix/helix-core/target/jacoco.exec
[INFO] Analyzed bundle 'Apache Helix :: Core' with 947 classes
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  9.512 s
[INFO] Finished at: 2023-09-19T18:05:33-07:00
[INFO] ------------------------------------------------------------------------


➜  helix git:(support_persist_instance_info) mvn test -o -Dtest=TestInstanceAutoJoin -pl=helix-core
...
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.helix.integration.paticipant.TestInstanceAutoJoin
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 34.07 s - in org.apache.helix.integration.paticipant.TestInstanceAutoJoin
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] 
[INFO] --- jacoco:0.8.6:report (generate-code-coverage-report) @ helix-core ---
[INFO] Loading execution data file /Users/zapinto/Documents/git/zpinto/helix/helix-core/target/jacoco.exec
[INFO] Analyzed bundle 'Apache Helix :: Core' with 947 classes
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  43.920 s
[INFO] Finished at: 2023-09-19T18:04:52-07:00
[INFO] ------------------------------------------------------------------------

Changes that Break Backward Compatibility (Optional)

NA

Commits

  • My commits all reference appropriate Apache Helix GitHub issues in their subject lines. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters (not including Jira issue reference)
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

Code Quality

  • My diff has been formatted using helix-style.xml
    (helix-style-intellij.xml if IntelliJ IDE is used)

…tanceInformationProcessor in CloudInstanceInformation object. Add ability for CloudInstanceInformationProcessor to produce full DOMAIN field instead of appending _instanceName unless last character in CloudInstanceInformation.CloudInstanceField.FAULT_DOMAIN is '='.
@zpinto zpinto force-pushed the support_persist_instance_info branch from a2ee5a5 to a205a46 Compare September 20, 2023 16:47
… ConfigStringUtil to check if FAULT_DOMAIN is complete or needs to have _instanceName added instead of hardcoding '='.
@zpinto
Copy link
Contributor Author

zpinto commented Sep 20, 2023

This PR is ready to be merged.

Final Commit Message:
Add support to persist all instance information collected by CloudInstanceInformationProcessor in CloudInstanceInformation object. Add ability for CloudInstanceInformationProcessor to produce full DOMAIN field instead of appending _instanceName unless last character in CloudInstanceInformation.CloudInstanceField.FAULT_DOMAIN is '='.

@xyuanlu xyuanlu merged commit 045deb6 into apache:master Sep 21, 2023
3 checks passed
himanshukandwal pushed a commit to himanshukandwal/helix that referenced this pull request Sep 27, 2023
…ocessor during auto-reg (apache#2622)

Add support to persist all instance information collected by CloudInstanceInformationProcessor in CloudInstanceInformation object. Add ability for CloudInstanceInformationProcessor to produce full DOMAIN field instead of appending _instanceName unless last character in CloudInstanceInformation.CloudInstanceField.FAULT_DOMAIN is '='.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants