Skip to content

Add TLS adherance feature gate #2680

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
richardsonnick wants to merge 1 commit into
base: master
Choose a base branch
from
+825 −0
Open

Add TLS adherance feature gate #2680

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 88 additions & 0 deletions config/v1/tests/apiservers.config.openshift.io/TLSAdherence.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
apiVersion: apiextensions.k8s.io/v1
name: "APIServer"
crdName: apiservers.config.openshift.io
featureGates:
- TLSAdherence
tests:
onCreate:
- name: Should be able to create with tlsAdherence set to LegacyExternalAPIServerComponentsOnly
initial: |
apiVersion: config.openshift.io/v1
kind: APIServer
spec:
tlsAdherence: LegacyExternalAPIServerComponentsOnly
expected: |
apiVersion: config.openshift.io/v1
kind: APIServer
spec:
audit:
profile: Default
tlsAdherence: LegacyExternalAPIServerComponentsOnly
- name: Should be able to create with tlsAdherence set to StrictAllComponents
initial: |
apiVersion: config.openshift.io/v1
kind: APIServer
spec:
tlsAdherence: StrictAllComponents
expected: |
apiVersion: config.openshift.io/v1
kind: APIServer
spec:
audit:
profile: Default
tlsAdherence: StrictAllComponents
- name: Should reject invalid tlsAdherence value
initial: |
apiVersion: config.openshift.io/v1
kind: APIServer
spec:
tlsAdherence: InvalidValue
expectedError: 'spec.tlsAdherence: Unsupported value: "InvalidValue": supported values: "LegacyExternalAPIServerComponentsOnly", "StrictAllComponents"'
onUpdate:
- name: Should be able to update tlsAdherence from LegacyExternalAPIServerComponentsOnly to StrictAllComponents
initial: |
apiVersion: config.openshift.io/v1
kind: APIServer
spec:
tlsAdherence: LegacyExternalAPIServerComponentsOnly
updated: |
apiVersion: config.openshift.io/v1
kind: APIServer
spec:
tlsAdherence: StrictAllComponents
expected: |
apiVersion: config.openshift.io/v1
kind: APIServer
spec:
audit:
profile: Default
tlsAdherence: StrictAllComponents
- name: Should be able to update tlsAdherence from StrictAllComponents to LegacyExternalAPIServerComponentsOnly
initial: |
apiVersion: config.openshift.io/v1
kind: APIServer
spec:
tlsAdherence: StrictAllComponents
updated: |
apiVersion: config.openshift.io/v1
kind: APIServer
spec:
tlsAdherence: LegacyExternalAPIServerComponentsOnly
expected: |
apiVersion: config.openshift.io/v1
kind: APIServer
spec:
audit:
profile: Default
tlsAdherence: LegacyExternalAPIServerComponentsOnly
- name: Should not allow removing tlsAdherence once set
initial: |
apiVersion: config.openshift.io/v1
kind: APIServer
spec:
tlsAdherence: StrictAllComponents
updated: |
apiVersion: config.openshift.io/v1
kind: APIServer
spec: {}
expectedError: "tlsAdherence may not be removed once set"
58 changes: 58 additions & 0 deletions config/v1/types_apiserver.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ type APIServer struct {
Status APIServerStatus `json:"status"`
}

// +openshift:validation:FeatureGateAwareXValidation:featureGate=TLSAdherence,rule="has(oldSelf.tlsAdherence) ? has(self.tlsAdherence) : true",message="tlsAdherence may not be removed once set"
type APIServerSpec struct {
// servingCert is the TLS cert info for serving secure traffic. If not specified, operator managed certificates
// will be used for serving secure traffic.
Expand Down Expand Up @@ -62,6 +63,37 @@ type APIServerSpec struct {
// The current default is the Intermediate profile.
// +optional
TLSSecurityProfile *TLSSecurityProfile `json:"tlsSecurityProfile,omitempty"`
// tlsAdherence controls which components in the cluster adhere to the TLS security profile
// configured on this APIServer resource.
//
// Valid values are "LegacyExternalAPIServerComponentsOnly" and "StrictAllComponents".
//
// When set to "LegacyExternalAPIServerComponentsOnly", only the externally exposed
// API server components (kube-apiserver, openshift-apiserver, oauth-apiserver) honor the configured
// TLS profile. Other components continue to use their individual TLS configurations.
//
// When set to "StrictAllComponents", all components must honor the configured TLS profile
// unless they have a component-specific TLS configuration that overrides it.
// This mode is recommended for security-conscious deployments and is required for
// certain compliance frameworks.
//
// Note: Some components such as Kubelet and IngressController have their own dedicated TLS
// configuration mechanisms via KubeletConfig and IngressController CRs respectively. When these
// component-specific TLS configurations are set, they take precedence over the cluster-wide
// tlsSecurityProfile. When not set, these components fall back to the cluster-wide default.
//
// Components that encounter an unknown value for tlsAdherence should treat it as "StrictAllComponents"
// and log a warning to ensure forward compatibility while defaulting to the more secure behavior.
//
// This field is optional.
// When omitted, this means the user has no opinion and the platform is left to choose reasonable defaults.
// These defaults are subject to change over time.
// The current default is LegacyExternalAPIServerComponentsOnly.
//
// Once set, this field may be changed to a different value, but may not be removed.
// +openshift:enable:FeatureGate=TLSAdherence
// +optional
Copy link
Member

@joelanford joelanford Feb 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to configure validation on this field, and set the default.

// +kubebuilder:validation:items:Enum:=LegacyExternalAPIServerComponentsOnly;StrictAllComponents
// +kubebuilder:default:=LegacyExternalAPIServerComponentsOnly

Copy link
Member

@joelanford joelanford Feb 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I see you are defining the Enum validation on the type, which is fine. Let's also include the +kubebuilder:default marker there as well.

Copy link
Member

@joelanford joelanford Feb 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, reading the tlsAdherence description again, it seems like you are wanting it to be possible for an empty string to be allowed to be persisted for this field in etcd, which would allow us in theory to change the default semantic later in the future from legacy to strict without a change in the actual tlsAdherence value.

In practice, I think the problem with that idea is that each component essentially has their own implementation of this API and we'd have to do a really good job coordinating a semantic switch from legacy to strict.

What do folks think about:

  1. Actually writing LegacyExternalAPIServerComponentsOnly to etcd by default.
  2. Telling implementers they specifically only need to look for StrictAllComponents and will not have to worry about the possibility of "" with changing semantics later.
  3. When want to set StrictAllComponents as the new default, we make sure newly installed clusters get that, and we potentially block upgrades of clusters that still use LegacyExternalAPIServerComponentsOnly? Or maybe there's some other way to encourage users to switch to StrictAllComponents?

Copy link
Contributor

@everettraven everettraven Feb 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For configuration APIs we generally try to avoid setting the default via +kubebuilder:default marker so that we can change the default behavior of the "no-opinion" mechanism.

In this case, I don't think we will be doing that but I still wouldn't use the +kubebuilder:default marker because then we won't be able to distinguish whether or not the user intentionally set the field or if we defaulted it on their behalf and you technically get locked into that default being a contractual default.

Copy link
Member

@joelanford joelanford Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so that we can change the default behavior of the "no-opinion" mechanism.

Thanks, I was hoping you'd have some good advice here. It is the intent to eventually make all components honor the cluster-wide default, so I supposed we'll just have to deal with trying to coordinate a semantic change across a bunch of components at once when the time comes.

To facilitate that, I'd suggest a library-go function somehow like this:

// in configv1
const TLSAdherenceNoOpinion = ""
func ShouldAllComponentsAdhere(tlsAdherence configv1.TLSAdherence) bool {
    if tlsAdherence == configv1.TLSAdherenceNoOpinion || tlsAdherence == configv1.LegacyExternalAPIServerComponentsOnly {
        return false
    }
    return true
}

And then when we want to change the semantic, we'd update that function to:

func ShouldAllComponentsAdhere(tlsAdherence configv1.TLSAdherence) bool {
    if tlsAdherence == configv1.LegacyExternalAPIServerComponentsOnly {
        return false
    }
    return true
}

Copy link
Contributor

@everettraven everettraven Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That aligns with what I envisioned.

Can we make sure to update the EP with a mention of this kind of check/library that implementors are expected to use?

Copy link
Author

@richardsonnick richardsonnick Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a mention of this helper function.
Also created a draft diff in library-go to add this function: openshift/library-go#2114

Will need to land this first to get the TLSAdherencePolicy type into library-go

TLSAdherence TLSAdherencePolicy `json:"tlsAdherence,omitempty"`
Copy link
Contributor

@JoelSpeed JoelSpeed Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a recent call we discussed the fact that we might need different TLSSecurityProfile structs for APIServer vs Ingress. If we had separate structs, would this field still be a sibling of the TLSSecurityProfile, or would it be a member of the TLSSecurityProfile?

Copy link
Contributor

@everettraven everettraven Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that this adherence field is only applicable for the APIServer type because it will be the central configuration type.

Ingress/Kubelet specific configurations would be overrides to the central configuration and Ingress and Kubelet would be expected to strictly adhere to those.

@joelanford @richardsonnick please correct me if I'm wrong here.

Copy link
Author

@richardsonnick richardsonnick Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is my understanding as well. The ingress/kubelet configurations are overrides to the (central authoritative) apiserver type

Copy link
Contributor

@JoelSpeed JoelSpeed Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand that relationship, that wasn't actually the key part of my question

If we had separate structs, would this field still be a sibling of the TLSSecurityProfile, or would it be a member of the TLSSecurityProfile?

Copy link
Contributor

@everettraven everettraven Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, still a sibling.

// audit specifies the settings for audit configuration to be applied to all OpenShift-provided
// API servers in the cluster.
// +optional
Expand Down Expand Up @@ -237,6 +269,32 @@ const (
type APIServerStatus struct {
}

// TLSAdherencePolicy defines which components adhere to the TLS security profile.
// Implementors should use the ShouldAllComponentsAdhere helper function from library-go
// rather than checking these values directly.
// +kubebuilder:validation:Enum=LegacyExternalAPIServerComponentsOnly;StrictAllComponents
type TLSAdherencePolicy string

const (
// TLSAdherencePolicyNoOpinion represents an empty/unset value for tlsAdherence.
// This value cannot be explicitly set and is only present when the field is omitted.
// When the field is omitted, the cluster defaults to LegacyExternalAPIServerComponentsOnly
// behavior. Components should treat this the same as LegacyExternalAPIServerComponentsOnly.
TLSAdherencePolicyNoOpinion TLSAdherencePolicy = ""

// TLSAdherencePolicyLegacyExternalAPIServerComponentsOnly means only the externally exposed
// API server components (kube-apiserver, openshift-apiserver, oauth-apiserver) honor
// the configured TLS profile. Other components continue to use their individual TLS
// configurations.
TLSAdherencePolicyLegacyExternalAPIServerComponentsOnly TLSAdherencePolicy = "LegacyExternalAPIServerComponentsOnly"

// TLSAdherencePolicyStrictAllComponents means all components must honor the configured TLS
// profile unless they have a component-specific TLS configuration that overrides it.
// This mode is recommended for security-conscious deployments and is required
// for certain compliance frameworks.
TLSAdherencePolicyStrictAllComponents TLSAdherencePolicy = "StrictAllComponents"
)

// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

// Compatibility level 1: Stable within a major release for a minimum of 12 months or 3 minor releases (whichever is longer).
Expand Down
37 changes: 37 additions & 0 deletions ...zz_generated.crd-manifests/0000_10_config-operator_01_apiservers-CustomNoUpgrade.crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -292,6 +292,40 @@ spec:
type: array
x-kubernetes-list-type: atomic
type: object
tlsAdherence:
description: |-
tlsAdherence controls which components in the cluster adhere to the TLS security profile
configured on this APIServer resource.

Valid values are "LegacyExternalAPIServerComponentsOnly" and "StrictAllComponents".

When set to "LegacyExternalAPIServerComponentsOnly", only the externally exposed
API server components (kube-apiserver, openshift-apiserver, oauth-apiserver) honor the configured
TLS profile. Other components continue to use their individual TLS configurations.

When set to "StrictAllComponents", all components must honor the configured TLS profile
unless they have a component-specific TLS configuration that overrides it.
This mode is recommended for security-conscious deployments and is required for
certain compliance frameworks.

Note: Some components such as Kubelet and IngressController have their own dedicated TLS
configuration mechanisms via KubeletConfig and IngressController CRs respectively. When these
component-specific TLS configurations are set, they take precedence over the cluster-wide
tlsSecurityProfile. When not set, these components fall back to the cluster-wide default.

Components that encounter an unknown value for tlsAdherence should treat it as "StrictAllComponents"
and log a warning to ensure forward compatibility while defaulting to the more secure behavior.

This field is optional.
When omitted, this means the user has no opinion and the platform is left to choose reasonable defaults.
These defaults are subject to change over time.
The current default is LegacyExternalAPIServerComponentsOnly.

Once set, this field may be changed to a different value, but may not be removed.
enum:
- LegacyExternalAPIServerComponentsOnly
- StrictAllComponents
type: string
tlsSecurityProfile:
description: |-
tlsSecurityProfile specifies settings for TLS connections for externally exposed servers.
Expand Down Expand Up @@ -439,6 +473,9 @@ spec:
type: string
type: object
type: object
x-kubernetes-validations:
- message: tlsAdherence may not be removed once set
rule: 'has(oldSelf.tlsAdherence) ? has(self.tlsAdherence) : true'
status:
description: status holds observed values from the cluster. They may not
be overridden.
Expand Down
37 changes: 37 additions & 0 deletions ...enerated.crd-manifests/0000_10_config-operator_01_apiservers-DevPreviewNoUpgrade.crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -292,6 +292,40 @@ spec:
type: array
x-kubernetes-list-type: atomic
type: object
tlsAdherence:
description: |-
tlsAdherence controls which components in the cluster adhere to the TLS security profile
configured on this APIServer resource.
Valid values are "LegacyExternalAPIServerComponentsOnly" and "StrictAllComponents".
When set to "LegacyExternalAPIServerComponentsOnly", only the externally exposed
API server components (kube-apiserver, openshift-apiserver, oauth-apiserver) honor the configured
TLS profile. Other components continue to use their individual TLS configurations.
When set to "StrictAllComponents", all components must honor the configured TLS profile
unless they have a component-specific TLS configuration that overrides it.
This mode is recommended for security-conscious deployments and is required for
certain compliance frameworks.
Note: Some components such as Kubelet and IngressController have their own dedicated TLS
configuration mechanisms via KubeletConfig and IngressController CRs respectively. When these
component-specific TLS configurations are set, they take precedence over the cluster-wide
tlsSecurityProfile. When not set, these components fall back to the cluster-wide default.
Components that encounter an unknown value for tlsAdherence should treat it as "StrictAllComponents"
and log a warning to ensure forward compatibility while defaulting to the more secure behavior.
This field is optional.
When omitted, this means the user has no opinion and the platform is left to choose reasonable defaults.
These defaults are subject to change over time.
The current default is LegacyExternalAPIServerComponentsOnly.
Once set, this field may be changed to a different value, but may not be removed.
enum:
- LegacyExternalAPIServerComponentsOnly
- StrictAllComponents
type: string
tlsSecurityProfile:
description: |-
tlsSecurityProfile specifies settings for TLS connections for externally exposed servers.
Expand Down Expand Up @@ -439,6 +473,9 @@ spec:
type: string
type: object
type: object
x-kubernetes-validations:
- message: tlsAdherence may not be removed once set
rule: 'has(oldSelf.tlsAdherence) ? has(self.tlsAdherence) : true'
status:
description: status holds observed values from the cluster. They may not
be overridden.
Expand Down
37 changes: 37 additions & 0 deletions ...nerated.crd-manifests/0000_10_config-operator_01_apiservers-TechPreviewNoUpgrade.crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -224,6 +224,40 @@ spec:
type: array
x-kubernetes-list-type: atomic
type: object
tlsAdherence:
description: |-
tlsAdherence controls which components in the cluster adhere to the TLS security profile
configured on this APIServer resource.
Valid values are "LegacyExternalAPIServerComponentsOnly" and "StrictAllComponents".
When set to "LegacyExternalAPIServerComponentsOnly", only the externally exposed
API server components (kube-apiserver, openshift-apiserver, oauth-apiserver) honor the configured
TLS profile. Other components continue to use their individual TLS configurations.
When set to "StrictAllComponents", all components must honor the configured TLS profile
unless they have a component-specific TLS configuration that overrides it.
This mode is recommended for security-conscious deployments and is required for
certain compliance frameworks.
Note: Some components such as Kubelet and IngressController have their own dedicated TLS
configuration mechanisms via KubeletConfig and IngressController CRs respectively. When these
component-specific TLS configurations are set, they take precedence over the cluster-wide
tlsSecurityProfile. When not set, these components fall back to the cluster-wide default.
Components that encounter an unknown value for tlsAdherence should treat it as "StrictAllComponents"
and log a warning to ensure forward compatibility while defaulting to the more secure behavior.
This field is optional.
When omitted, this means the user has no opinion and the platform is left to choose reasonable defaults.
These defaults are subject to change over time.
The current default is LegacyExternalAPIServerComponentsOnly.
Once set, this field may be changed to a different value, but may not be removed.
enum:
- LegacyExternalAPIServerComponentsOnly
- StrictAllComponents
type: string
tlsSecurityProfile:
description: |-
tlsSecurityProfile specifies settings for TLS connections for externally exposed servers.
Expand Down Expand Up @@ -371,6 +405,9 @@ spec:
type: string
type: object
type: object
x-kubernetes-validations:
- message: tlsAdherence may not be removed once set
rule: 'has(oldSelf.tlsAdherence) ? has(self.tlsAdherence) : true'
status:
description: status holds observed values from the cluster. They may not
be overridden.
Expand Down
1 change: 1 addition & 0 deletions config/v1/zz_generated.featuregated-crd-manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ apiservers.config.openshift.io:
FeatureGates:
- KMSEncryption
- KMSEncryptionProvider
- TLSAdherence
FilenameOperatorName: config-operator
FilenameOperatorOrdering: "01"
FilenameRunLevel: "0000_10"
Expand Down
Loading