feat: deprecate and remove fields for connector

[arkbriar]

What's changed and what's your intention?

PLEASE DO NOT LEAVE THIS EMPTY !!!

Please explain IN DETAIL what the changes are in this PR and why they are needed:

• As title. The change removes the fields for connector support.

On upgrade:

• Once the new CRDs applied, the fields of RW objects in k8s will be removed automatically.
• The existing connector service, statefulset and pods will still be there but won't be touched anymore.
• StatefulSet / Pods / Service will be deleted automatically on deletion of the RW object.

On rollback:

• All the fields will be back but the values will be reset. That means the connector Pods will be deleted on rollback if not manually deleted after upgrade.

Checklist

• I have written the necessary docs and comments
• I have added necessary unit tests and integration tests

Refer to a related PR or issue link (optional)

fix #543

[wjf3121]

What will the behavior be for existing RW CR still with connector nodes once we rolled out this change to our production env?

[arkbriar]

What will the behavior be for existing RW CR still with connector nodes once we rolled out this change to our production env?

Will be like the following

On upgrade:

• Once the new CRDs applied, the fields of RW objects in k8s will be removed automatically.
• The existing connector service, statefulset and pods will still be there but won't be touched anymore.
• StatefulSet / Pods / Service will be deleted automatically on deletion of the RW object.

[codecov]

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (7da9f57) 61.85% compared to head (b11b623) 63.00%.

Files	Patch %	Lines
pkg/manager/risingwave_controller_manager_impl.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #585      +/-   ##
==========================================
+ Coverage   61.85%   63.00%   +1.15%     
==========================================
  Files          28       28              
  Lines        4868     4701     -167     
==========================================
- Hits         3011     2962      -49     
+ Misses       1808     1690     -118     
  Partials       49       49              

Flag	Coverage Δ
unittests	63.00% <0.00%> (+1.15%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[wjf3121]

What will the behavior be for existing RW CR still with connector nodes once we rolled out this change to our production env?

Will be like the following

On upgrade:

• Once the new CRDs applied, the fields of RW objects in k8s will be removed automatically.
• The existing connector service, statefulset and pods will still be there but won't be touched anymore.
• StatefulSet / Pods / Service will be deleted automatically on deletion of the RW object.

Thanks, I missed the PR description. I have 2 follow-up question:

1. Shall we remove the usage of Connector node in control plane code first? (We still have some usage like this)
2. Do we need to clean up all the existing connector node to avoid waste of resources? Once this change is rolled out, we have no control to these connector node resources.

[arkbriar]

What will the behavior be for existing RW CR still with connector nodes once we rolled out this change to our production env?

Will be like the following

On upgrade:

• Once the new CRDs applied, the fields of RW objects in k8s will be removed automatically.
• The existing connector service, statefulset and pods will still be there but won't be touched anymore.
• StatefulSet / Pods / Service will be deleted automatically on deletion of the RW object.

Thanks, I missed the PR description. I have 2 follow-up question:

1. Shall we remove the usage of Connector node in control plane code first? (We still have some usage like this)
2. Do we need to clean up all the existing connector node to avoid waste of resources? Once this change is rolled out, we have no control to these connector node resources.

1. In most cases, it's okay to leave them as is. The deserialization will ignore the missing fields if they are tagged with omitempty. The PR could be merged first and we can update the control plane accordingly.

However, there's a field doesn't have the tag. This will result in deserialization failure and break the cloud agent.

	// Running status of connector.
	Connector ComponentReplicasStatus `json:"connector"`

I will keep it in this PR and remove it later.

2. Good point! We should.

[arkbriar]

Reverted the deletion of .status.componentReplicas.connector. Also add tag omitempty to all the internal fields so that it will be safe to remove the field.

[wjf3121]

LGTM!

Shall we create an issue to track the clean up of connector nodes (I assume it's orthogonal to the deprecation and can be done in parallel)?

[arkbriar]

Shall we create an issue to track the clean up of connector nodes (I assume it's orthogonal to the deprecation and can be done in parallel)?

Yes, sounds great!

Tracked here.