Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[e2e] Support to provide e2e testing configuration #752

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

eromanova
Copy link
Member

@eromanova eromanova commented Dec 10, 2024

This PR introduces the possibility to set e2e testing configuration that makes e2e testing more flexible:

  1. It's allowed to skip some providers
  2. The template can be overwritten
  3. It's allowed to skip hosted cluster deployment
  4. It's allowed to deploy >1 cluster with the same provider with different templates
  5. It's allowed to specify the upgrade flag to mark that the cluster upgrade should be tested

Config format

Config is a map where keys are the supported provider names: aws, azure, vsphere and values are the array of cluster testing configurations

<providerName0>:
- standalone:
    template: <templateName0>
    upgrade: false // if true, tests will trigger the standalone cluster upgrade
  hosted:
    template: <templateName1>
    upgrade: false // if true, tests will trigger the hosted cluster upgrade
    upgradeTemplate: <upgradeTemplate>
- standalone:
    template: <templateName1>
    upgrade: true
    upgradeTemplate: <upgradeTemplate>
<providerName1>:
...
    

If the config is unset, it'll be populated with defaults. If the config is not empty but some of the providers are missing, its deployment will be skipped. If the provider is defined but the provider config is empty, this config will be populated with default (see examples section).

The cluster name will have a postfix with an array index to ensure uniqueness.

Config Examples

Run tests with the default configuration (config is empty)

{}

Run tests on the AWS provider only with the default configuration

aws: []

Deploy AWS managed cluster with aws-standalone-cp-0-0-3 template

aws:
- standalone:
    template: aws-standalone-cp-0-0-3

Deploy Azure managed cluster with azure-standalone-cp-0-0-3 template and hosted managed cluster with azure-hosted-cp-0-0-3

azure:
- standalone:
    template: azure-standalone-cp-0-0-3
  hosted:
    template: azure-hosted-cp-0-0-3

Deploy 2 Vsphere managed clusters, one single cluster with the vsphere-standalone-cp-0-0-2 template and another with the vsphere-standalone-cp-0-0-3 template and the hosted cluster on it.

vsphere:
- standalone:
    template: vsphere-standalone-cp-0-0-2
- standalone:
    template: vsphere-standalone-cp-0-0-3
  hosted:
    template: vsphere-hosted-cp-0-0-3

Deploy Azure managed cluster with the azure-standalone-cp-0-0-2 template and hosted managed cluster with the azure-hosted-cp-0-0-2 then upgrade clusters to the newer templates (upgrade template for the standalone cluster will be selected automatically).

azure:
- standalone:
    template: azure-standalone-cp-0-0-2
    upgrade: true
  hosted:
    template: azure-hosted-cp-0-0-2
    upgrade: true
    upgradeTemplate: azure-hosted-cp-0-0-3

Code updates

  1. Added Config structure and all related functions
  2. Reworked the e2e testing flow to support the deployment of more than 1 pair of standalone and hosted clusters
  3. Support to skip tests if the provider is not present in the config or if the hosted configuration is not present.
  4. Rename MANAGED_CLUSTER_NAME -> MANAGED_CLUSTER_PREFIX in the github build actions
  5. Rework logs collector to support collecting logs from multiple clusters
  6. Some code refactoring
  7. Added wait for clusters' deletion to ensure it's actually deleted

Out of scope

  1. Upgrade testing implementation (only added the support to configure it)

Closes #641

@eromanova eromanova force-pushed the e2e-testing-config branch 2 times, most recently from 4fbeac5 to b85512a Compare December 12, 2024 13:24
@eromanova eromanova marked this pull request as ready for review December 12, 2024 14:38
@eromanova eromanova force-pushed the e2e-testing-config branch 3 times, most recently from affbf82 to d6d0ddc Compare December 12, 2024 17:40
@eromanova eromanova self-assigned this Dec 13, 2024
@eromanova eromanova requested a review from zerospiel as a code owner December 24, 2024 12:30
@eromanova eromanova added the test e2e Runs the entire provider E2E test suite, controller E2E tests are always ran label Dec 24, 2024
@eromanova eromanova marked this pull request as draft December 24, 2024 12:47
@eromanova eromanova removed the test e2e Runs the entire provider E2E test suite, controller E2E tests are always ran label Dec 25, 2024
@eromanova eromanova marked this pull request as ready for review December 25, 2024 12:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

Support to provide e2e testing configuration
1 participant