Skip to content

Commit 60bb01b

Browse files
kron4egkron4eg
authored
beta: managed control plane on hetzner (#3895)
* Adjust the API to managed CP proposal * Add HetznerControlPlane API * Update API references and remove KubeletConfig from NodeSettingsSpec * Adjust NodeSet fields to be required in all API versions * Extend HetznerLoadBalancer with location, networkName, publicIP, and labels - Add new fields to HetznerLoadBalancer: Location, NetworkName, PublicIP, and Labels - Update API docs and proposals to reflect new fields and their defaults - Update conversion and deepcopy methods to handle new fields - Make Name and Type fields optional with sensible defaults * Remove NetworkName field from HetznerLoadBalancer - Removed the NetworkName field from HetznerLoadBalancer structs in all API versions and related conversion functions. - Updated API documentation to reflect the removal. - Added validation to require networkID when controlPlane is specified in HetznerSpec. * Change NodeSettingsSpec.Annotations from []string to map[string]string * Provisioner package * Update hcloud-go to v2.30.0 * Create hetzner control plane * Update github.com/hetznercloud/hcloud-go/v2 * Fix linter warnings Signed-off-by: Artiom Diomin <artiom@kubermatic.com> * read terraform outputs directly from state file when available Signed-off-by: Artiom Diomin <artiom@kubermatic.com> * improve controlPlane validation error message and add missing test cases Signed-off-by: Artiom Diomin <artiom@kubermatic.com> * Wrap provisioner errors Signed-off-by: Artiom Diomin <artiom@kubermatic.com> * go fix Signed-off-by: Artiom Diomin <artiom@kubermatic.com> * Create hetzner Machines and use them as Host for control plane Signed-off-by: Artiom Diomin <artiom@kubermatic.com> * chore: bump golangci-lint to v2.11 Signed-off-by: Artiom Diomin <artiom@kubermatic.com> * Refactor control-plane creation code * move cloud resource creation into the tasks * ask for confirmation * simplify in general Signed-off-by: Artiom Diomin <artiom@kubermatic.com> * Divide WithEnsureControlPlane and WithFindControlPlane Signed-off-by: Artiom Diomin <artiom@kubermatic.com> * provisioner: prefer IPv4 addresses and fallback to IPv6 Select machine external/internal addresses by IP family when building provisioner output. Prefer IPv4 when available and use IPv6 as fallback if no IPv4 address exists. Signed-off-by: Artiom Diomin <artiom@kubermatic.com> * Also remove implicit `IsLeader` When building host configs from machines don't assigne IsLeader, it will be assigned at later stages of the tasks flow. Signed-off-by: Artiom Diomin <artiom@kubermatic.com> * Add Hetzner MC documentation Signed-off-by: Artiom Diomin <artiom@kubermatic.com> * Populate control-plane data for config dump via tasks Signed-off-by: Artiom Diomin <artiom@kubermatic.com> * Increase the resources for build/test containers Signed-off-by: Artiom Diomin <artiom@kubermatic.com> --------- Signed-off-by: Artiom Diomin <artiom@kubermatic.com>
1 parent cadcf3f commit 60bb01b

51 files changed

Lines changed: 4035 additions & 318 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/golangci-lint.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,4 +22,4 @@ jobs:
2222
- name: golangci-lint
2323
uses: golangci/golangci-lint-action@v9
2424
with:
25-
version: v2.10
25+
version: v2.11

‎.prow/verify.yaml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -90,10 +90,10 @@ presubmits:
9090
- build
9191
resources:
9292
requests:
93-
memory: 1Gi
93+
memory: 2Gi
9494
cpu: 1
9595
limits:
96-
memory: 2Gi
96+
memory: 3Gi
9797
cpu: 2
9898
- name: pull-kubeone-test
9999
always_run: true
@@ -110,10 +110,10 @@ presubmits:
110110
- test
111111
resources:
112112
requests:
113-
memory: 1Gi
113+
memory: 2Gi
114114
cpu: 1
115115
limits:
116-
memory: 2Gi
116+
memory: 3Gi
117117
cpu: 2
118118
- name: pull-kubeone-build-image
119119
always_run: true

‎.wwhrd.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,3 +22,4 @@ exceptions:
2222
- github.com/hashicorp/go-version # MPL-2.0
2323
- github.com/hashicorp/golang-lru # MPL-2.0
2424
- github.com/hashicorp/golang-lru/simplelru # MPL-2.0
25+
- github.com/hashicorp/go-retryablehttp # MPL-2.0

‎docs/api_reference/v1beta2.en.md‎

Lines changed: 92 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
+++
22
title = "v1beta2 API Reference"
3-
date = 2026-04-03T12:52:19+02:00
3+
date = 2026-04-03T16:57:31+03:00
44
weight = 11
55
+++
66
## v1beta2
@@ -42,6 +42,8 @@ weight = 11
4242
* [HelmAuth](#helmauth)
4343
* [HelmRelease](#helmrelease)
4444
* [HelmValues](#helmvalues)
45+
* [HetznerControlPlane](#hetznercontrolplane)
46+
* [HetznerLoadBalancer](#hetznerloadbalancer)
4547
* [HetznerSpec](#hetznerspec)
4648
* [HostConfig](#hostconfig)
4749
* [IPTables](#iptables)
@@ -54,19 +56,23 @@ weight = 11
5456
* [MachineControllerConfig](#machinecontrollerconfig)
5557
* [MetricsServer](#metricsserver)
5658
* [NodeLocalDNS](#nodelocaldns)
59+
* [NodeSet](#nodeset)
60+
* [NodeSettingsSpec](#nodesettingsspec)
5761
* [NoneSpec](#nonespec)
5862
* [NutanixSpec](#nutanixspec)
5963
* [OpenIDConnect](#openidconnect)
6064
* [OpenIDConnectConfig](#openidconnectconfig)
6165
* [OpenstackSpec](#openstackspec)
6266
* [OperatingSystemManagerConfig](#operatingsystemmanagerconfig)
67+
* [OperatingSystemSpec](#operatingsystemspec)
6368
* [PodNodeSelector](#podnodeselector)
6469
* [PodNodeSelectorConfig](#podnodeselectorconfig)
6570
* [PodSecurityPolicy](#podsecuritypolicy)
6671
* [ProviderSpec](#providerspec)
6772
* [ProviderStaticNetworkConfig](#providerstaticnetworkconfig)
6873
* [ProxyConfig](#proxyconfig)
6974
* [RegistryConfiguration](#registryconfiguration)
75+
* [SSHSpec](#sshspec)
7076
* [StaticAuditLog](#staticauditlog)
7177
* [StaticAuditLogConfig](#staticauditlogconfig)
7278
* [StaticWorkersConfig](#staticworkersconfig)
@@ -339,6 +345,7 @@ ControlPlaneConfig defines control plane nodes
339345
| Field | Description | Scheme | Required |
340346
| ----- | ----------- | ------ | -------- |
341347
| hosts | Hosts array of all control plane hosts. | [][HostConfig](#hostconfig) | true |
348+
| nodeSets | | [][NodeSet](#nodeset) | true |
342349

343350
[Back to Group](#v1beta2)
344351

@@ -529,13 +536,38 @@ HelmValues configure inputs to `helm upgrade --install` command analog.
529536

530537
[Back to Group](#v1beta2)
531538

539+
### HetznerControlPlane
540+
541+
HetznerControlPlane control plane config on Hetzner
542+
543+
| Field | Description | Scheme | Required |
544+
| ----- | ----------- | ------ | -------- |
545+
| loadBalancer | LoadBalancer config of a loadbalancer | [HetznerLoadBalancer](#hetznerloadbalancer) | true |
546+
547+
[Back to Group](#v1beta2)
548+
549+
### HetznerLoadBalancer
550+
551+
HetznerLoadBalancer loadbalancer definition to create for kubeapi-server endpoint
552+
553+
| Field | Description | Scheme | Required |
554+
| ----- | ----------- | ------ | -------- |
555+
| name | Name of the loadbalancer to create. Default: \"<CLUSTER_NAME>-kubeapi\" | string | false |
556+
| type | Type of the loadbalancer to create. Default: \"lb11\" | string | false |
557+
| location | Location of the loadbalancer to create. Default: \"nbg1\" | string | false |
558+
| publicIP | PublicIP indicates whether the loadbalancer should have a public IP assigned. Default: true | *bool | false |
559+
| labels | Labels to be applied to the loadbalancer | map[string]string | false |
560+
561+
[Back to Group](#v1beta2)
562+
532563
### HetznerSpec
533564

534565
HetznerSpec defines the Hetzner cloud provider
535566

536567
| Field | Description | Scheme | Required |
537568
| ----- | ----------- | ------ | -------- |
538569
| networkID | NetworkID | string | false |
570+
| controlPlane | ControlPlane configures | *[HetznerControlPlane](#hetznercontrolplane) | false |
539571

540572
[Back to Group](#v1beta2)
541573

@@ -707,6 +739,35 @@ MetricsServer feature flag
707739

708740
[Back to Group](#v1beta2)
709741

742+
### NodeSet
743+
744+
745+
746+
| Field | Description | Scheme | Required |
747+
| ----- | ----------- | ------ | -------- |
748+
| name | | string | true |
749+
| replicas | | int | true |
750+
| generation | | int | false |
751+
| nodeSettings | | [NodeSettingsSpec](#nodesettingsspec) | false |
752+
| operatingSystem | | OperatingSystemName | true |
753+
| operatingSystemSpec | | [OperatingSystemSpec](#operatingsystemspec) | false |
754+
| ssh | | [SSHSpec](#sshspec) | true |
755+
| cloudProviderSpec | | [json.RawMessage](https://golang.org/pkg/encoding/json/#RawMessage) | true |
756+
757+
[Back to Group](#v1beta2)
758+
759+
### NodeSettingsSpec
760+
761+
762+
763+
| Field | Description | Scheme | Required |
764+
| ----- | ----------- | ------ | -------- |
765+
| labels | | map[string]string | false |
766+
| annotations | | map[string]string | false |
767+
| taints | | [][corev1.Taint](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.25/#taint-v1-core) | false |
768+
769+
[Back to Group](#v1beta2)
770+
710771
### NoneSpec
711772

712773
NoneSpec defines a none provider
@@ -774,6 +835,16 @@ OperatingSystemManagerConfig configures kubermatic operating-system-manager depl
774835

775836
[Back to Group](#v1beta2)
776837

838+
### OperatingSystemSpec
839+
840+
841+
842+
| Field | Description | Scheme | Required |
843+
| ----- | ----------- | ------ | -------- |
844+
| distUpgradeOnBoot | | bool | false |
845+
846+
[Back to Group](#v1beta2)
847+
777848
### PodNodeSelector
778849

779850
PodNodeSelector feature flag
@@ -865,6 +936,26 @@ KubeOne and kubeadm are pulled from an image registry
865936

866937
[Back to Group](#v1beta2)
867938

939+
### SSHSpec
940+
941+
942+
943+
| Field | Description | Scheme | Required |
944+
| ----- | ----------- | ------ | -------- |
945+
| publicKeys | | []string | false |
946+
| port | | int | false |
947+
| username | | string | false |
948+
| privateKeyFile | | string | false |
949+
| certFile | | string | false |
950+
| hostPublicKey | | []byte | false |
951+
| agentSocket | | string | false |
952+
| bastion | | string | false |
953+
| bastionPort | | int | false |
954+
| bastionUser | | string | false |
955+
| bastionHostPublicKey | | []byte | false |
956+
957+
[Back to Group](#v1beta2)
958+
868959
### StaticAuditLog
869960

870961
StaticAuditLog feature flag

‎docs/api_reference/v1beta3.en.md‎

Lines changed: 92 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
+++
22
title = "v1beta3 API Reference"
3-
date = 2026-04-03T12:52:19+02:00
3+
date = 2026-04-03T16:57:31+03:00
44
weight = 11
55
+++
66
## v1beta3
@@ -42,6 +42,8 @@ weight = 11
4242
* [HelmAuth](#helmauth)
4343
* [HelmRelease](#helmrelease)
4444
* [HelmValues](#helmvalues)
45+
* [HetznerControlPlane](#hetznercontrolplane)
46+
* [HetznerLoadBalancer](#hetznerloadbalancer)
4547
* [HetznerSpec](#hetznerspec)
4648
* [HostConfig](#hostconfig)
4749
* [IPTables](#iptables)
@@ -54,18 +56,22 @@ weight = 11
5456
* [MachineControllerConfig](#machinecontrollerconfig)
5557
* [MetricsServer](#metricsserver)
5658
* [NodeLocalDNS](#nodelocaldns)
59+
* [NodeSet](#nodeset)
60+
* [NodeSettingsSpec](#nodesettingsspec)
5761
* [NoneSpec](#nonespec)
5862
* [NutanixSpec](#nutanixspec)
5963
* [OpenIDConnect](#openidconnect)
6064
* [OpenIDConnectConfig](#openidconnectconfig)
6165
* [OpenstackSpec](#openstackspec)
6266
* [OperatingSystemManagerConfig](#operatingsystemmanagerconfig)
67+
* [OperatingSystemSpec](#operatingsystemspec)
6368
* [PodNodeSelector](#podnodeselector)
6469
* [PodNodeSelectorConfig](#podnodeselectorconfig)
6570
* [ProviderSpec](#providerspec)
6671
* [ProviderStaticNetworkConfig](#providerstaticnetworkconfig)
6772
* [ProxyConfig](#proxyconfig)
6873
* [RegistryConfiguration](#registryconfiguration)
74+
* [SSHSpec](#sshspec)
6975
* [StaticAuditLog](#staticauditlog)
7076
* [StaticAuditLogConfig](#staticauditlogconfig)
7177
* [StaticWorkersConfig](#staticworkersconfig)
@@ -341,6 +347,7 @@ ControlPlaneConfig defines control plane nodes
341347
| Field | Description | Scheme | Required |
342348
| ----- | ----------- | ------ | -------- |
343349
| hosts | Hosts array of all control plane hosts. | [][HostConfig](#hostconfig) | true |
350+
| nodeSets | | [][NodeSet](#nodeset) | true |
344351

345352
[Back to Group](#v1beta3)
346353

@@ -531,13 +538,38 @@ HelmValues configure inputs to `helm upgrade --install` command analog.
531538

532539
[Back to Group](#v1beta3)
533540

541+
### HetznerControlPlane
542+
543+
HetznerControlPlane control plane config on Hetzner
544+
545+
| Field | Description | Scheme | Required |
546+
| ----- | ----------- | ------ | -------- |
547+
| loadBalancer | LoadBalancer config of a loadbalancer | [HetznerLoadBalancer](#hetznerloadbalancer) | true |
548+
549+
[Back to Group](#v1beta3)
550+
551+
### HetznerLoadBalancer
552+
553+
HetznerLoadBalancer loadbalancer definition to create for kubeapi-server endpoint
554+
555+
| Field | Description | Scheme | Required |
556+
| ----- | ----------- | ------ | -------- |
557+
| name | Name of the loadbalancer to create. Default: \"<CLUSTER_NAME>-kubeapi\" | string | false |
558+
| type | Type of the loadbalancer to create. Default: \"lb11\" | string | false |
559+
| location | Location of the loadbalancer to create. Default: \"nbg1\" | string | false |
560+
| publicIP | PublicIP indicates whether the loadbalancer should have a public IP assigned. Default: true | *bool | false |
561+
| labels | Labels to be applied to the loadbalancer | map[string]string | false |
562+
563+
[Back to Group](#v1beta3)
564+
534565
### HetznerSpec
535566

536567
HetznerSpec defines the Hetzner cloud provider
537568

538569
| Field | Description | Scheme | Required |
539570
| ----- | ----------- | ------ | -------- |
540571
| networkID | NetworkID | string | false |
572+
| controlPlane | ControlPlane configures | *[HetznerControlPlane](#hetznercontrolplane) | false |
541573

542574
[Back to Group](#v1beta3)
543575

@@ -709,6 +741,35 @@ MetricsServer feature flag
709741

710742
[Back to Group](#v1beta3)
711743

744+
### NodeSet
745+
746+
747+
748+
| Field | Description | Scheme | Required |
749+
| ----- | ----------- | ------ | -------- |
750+
| name | | string | true |
751+
| replicas | | int | true |
752+
| generation | | int | false |
753+
| nodeSettings | | [NodeSettingsSpec](#nodesettingsspec) | false |
754+
| operatingSystem | | OperatingSystemName | true |
755+
| operatingSystemSpec | | [OperatingSystemSpec](#operatingsystemspec) | false |
756+
| ssh | | [SSHSpec](#sshspec) | true |
757+
| cloudProviderSpec | | [json.RawMessage](https://golang.org/pkg/encoding/json/#RawMessage) | true |
758+
759+
[Back to Group](#v1beta3)
760+
761+
### NodeSettingsSpec
762+
763+
764+
765+
| Field | Description | Scheme | Required |
766+
| ----- | ----------- | ------ | -------- |
767+
| labels | | map[string]string | false |
768+
| annotations | | map[string]string | false |
769+
| taints | | [][corev1.Taint](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.25/#taint-v1-core) | false |
770+
771+
[Back to Group](#v1beta3)
772+
712773
### NoneSpec
713774

714775
NoneSpec defines a none provider
@@ -776,6 +837,16 @@ OperatingSystemManagerConfig configures kubermatic operating-system-manager depl
776837

777838
[Back to Group](#v1beta3)
778839

840+
### OperatingSystemSpec
841+
842+
843+
844+
| Field | Description | Scheme | Required |
845+
| ----- | ----------- | ------ | -------- |
846+
| distUpgradeOnBoot | | bool | false |
847+
848+
[Back to Group](#v1beta3)
849+
779850
### PodNodeSelector
780851

781852
PodNodeSelector feature flag
@@ -854,6 +925,26 @@ KubeOne and kubeadm are pulled from an image registry
854925

855926
[Back to Group](#v1beta3)
856927

928+
### SSHSpec
929+
930+
931+
932+
| Field | Description | Scheme | Required |
933+
| ----- | ----------- | ------ | -------- |
934+
| publicKeys | | []string | false |
935+
| port | | int | false |
936+
| username | | string | false |
937+
| privateKeyFile | | string | false |
938+
| certFile | | string | false |
939+
| hostPublicKey | | []byte | false |
940+
| agentSocket | | string | false |
941+
| bastion | | string | false |
942+
| bastionPort | | int | false |
943+
| bastionUser | | string | false |
944+
| bastionHostPublicKey | | []byte | false |
945+
946+
[Back to Group](#v1beta3)
947+
857948
### StaticAuditLog
858949

859950
StaticAuditLog feature flag

0 commit comments

Comments
 (0)