containeroo.ch on contineroo.ch

Account

Mon, 01 Jan 0001 00:00:00 +0000

The API specification can be viewed here.

In order to use cloudflare-operator, you need to create an account resource. This resource contains the credentials for the Cloudflare API.

Learn more about the account resource in the getting started guide.

Core Concepts

Mon, 01 Jan 0001 00:00:00 +0000

Architecture

agent-forge-operator is a demand-driven bridge between HyperShift Agent NodePools and vSphere inventory. It does not scale NodePools directly and it does not replace the hosted cluster autoscaler. It reacts to demand that HyperShift and CAPI already expressed through AgentMachine resources.

The reconciliation flow is:

The hosted cluster autoscaler changes HyperShift NodePool demand.
HyperShift and CAPI render AgentMachine and Machine resources.
Waiting AgentMachine resources report Ready=False with Reason=NoSuitableAgents.
VsphereAgentPool reconciliation plans required capacity from AgentMachine demand, existing Agents, owned VMs, and InfraEnv state.
A VsphereAgent request creates one vSphere VM.
The VM boots the InfraEnv discovery ISO and appears as an Assisted Installer Agent.
The operator prepares the Agent so the Agent CAPI provider can bind it to a Machine.

Demand

The operator watches CAPI AgentMachine objects for one HyperShift NodePool. It creates capacity only when an AgentMachine reports Ready=False with Reason=NoSuitableAgents.

Core Concepts

Mon, 01 Jan 0001 00:00:00 +0000

Architecture

cloudflare-operator is designed to serve as the single source of truth for Cloudflare DNS records.
It relies on the Kubernetes API to store the desired state of DNS records using Custom Resource Definitions (CRDs).

DNS records

Cloudflare DNS records are specified using a CRD (dnsrecords.cloudflare-operator.io).
These records can be created manually, through a GitOps workflow, or automatically generated from Kubernetes Ingress resources and supported Gateway API routes when the feature flag is enabled.

Create DNS records from Ingress

Mon, 01 Jan 0001 00:00:00 +0000

cloudflare-operator can create DNS records from Ingress resources. This guide shows how to configure the controller to automatically create DNS records for your Ingress resources. The same annotations also apply to supported Gateway API route objects when the feature flag --enable-gateway-api is enabled.

Ingress annotations

One of the following annotations is required: cloudflare-operator.io/content or cloudflare-operator.io/ip-ref

Ingress objects that do not have one of these annotations will be ignored by cloudflare-operator.

Monitoring with Prometheus

Mon, 01 Jan 0001 00:00:00 +0000

Prerequisites

Prometheus Operator or kube-prometheus-stack.
agent-forge-operator installed through the Helm chart.

Enable metrics resources

Create a values.yaml file:

---
metrics:
  serviceMonitor:
    enabled: true
  prometheusRule:
    enabled: true

Install or upgrade the Helm chart by following the installation guide.

Available metrics

The controller exposes these application metrics:

Metric	Meaning
`agent_forge_vsphere_vm_operations_total`	VM create/delete attempts by operation and result.
`agent_forge_iso_operations_total`	ISO cache ensure/delete attempts by operation and result.
`agent_forge_pool_capacity`	Per-pool capacity gauges for desired, waiting, available, pending, and planned-create counts.

You can also use kube-state-metrics to alert on VsphereAgentPool and VsphereAgent conditions once those custom resources are configured for state metrics collection.

VsphereAgentPool

Mon, 01 Jan 0001 00:00:00 +0000

The API specification can be viewed here.

VsphereAgentPool is a namespace-scoped bridge between one HyperShift Agent NodePool and vSphere VM inventory.

HyperShift and CAPI remain authoritative. The operator reacts to AgentMachine objects that report Ready=False with Reason=NoSuitableAgents by ensuring enough matching Assisted Installer Agent objects can exist.

Example

apiVersion: agent-forge.containeroo.ch/v1alpha1
kind: VsphereAgentPool
metadata:
  name: demo-worker
  namespace: demo
spec:
  hostedClusterRef:
    name: demo
  nodePoolRef:
    name: demo-worker
  infraEnvRef:
    name: demo
  controlPlaneNamespace: demo-demo
  cleanupPolicy: Delete
  vsphere:
    credentialsSecretRef:
      name: vsphere-credentials
    datacenter: dc1
    datastoreCluster: workload-datastore-cluster
    isoDatastore: iso-datastore
    resourcePool: cluster/Resources
    folder: demo
    network: VM Network
  template:
    namePrefix: demo-worker
    numCPUs: 4
    memoryMiB: 16384
    diskGiB: 100
  agent:
    role: worker
    approve: true
    labels:
      hypershift.openshift.io/nodepool-role: worker
      customer: example
  iso:
    checkInterval: 10m
    retainVersions: 2
    pathPrefix: agent-forge/demo/demo-worker

Required spec fields

Field	Description
`spec.hostedClusterRef.name`	HostedCluster name in the same namespace as this resource.
`spec.nodePoolRef.name`	HyperShift NodePool name in the same namespace as this resource.
`spec.infraEnvRef.name`	Assisted Installer InfraEnv name in the same namespace.
`spec.controlPlaneNamespace`	Hosted control plane namespace containing CAPI `AgentMachine` and `Machine` objects.
`spec.vsphere.credentialsSecretRef.name`	Secret containing vSphere `server`, `username`, and `password`.
`spec.vsphere.datastoreCluster`	Datastore cluster for VM disks.
`spec.vsphere.isoDatastore`	Datastore for cached discovery ISOs.
`spec.vsphere.resourcePool`	Resource pool path, for example `cluster/Resources`.
`spec.vsphere.network`	vSphere network attached to the VM NIC.
`spec.agent.labels`	Labels required on discovered Agents. These should match the NodePool Agent selector.

Important optional spec fields

Field	Description
`spec.vsphere.credentialsSecretRef.namespace`	Secret namespace. Defaults to the resource namespace.
`spec.vsphere.datacenter`	vSphere datacenter name. Defaults to `dc1`.
`spec.vsphere.folder`	VM folder path. Defaults logically to the hosted cluster name when empty.
`spec.vsphere.vmTags`	Optional vSphere tag IDs to attach to created VMs.
`spec.vsphere.guestID`	Guest OS identifier. Defaults to `rhel9_64Guest`.
`spec.vsphere.scsiType`	SCSI controller type. Defaults to `pvscsi`.
`spec.vsphere.firmware`	VM firmware, `efi` or `bios`. Defaults to `efi`.
`spec.vsphere.networkAdapterType`	NIC adapter type. Defaults to `vmxnet3`.
`spec.template.namePrefix`	Prefix for operator-created VM names.
`spec.template.numCPUs`	VM vCPU count. Defaults to `4`.
`spec.template.memoryMiB`	VM memory in MiB. Defaults to `16384`.
`spec.template.diskGiB`	Primary disk size in GiB. Defaults to `100`.
`spec.agent.role`	Value for `hypershift.openshift.io/nodepool-role`. Defaults to `worker`.
`spec.agent.approve`	When true, patch matching Agents with `spec.approved=true`. Defaults to `true`.
`spec.iso.checkInterval`	How often to download and hash the InfraEnv ISO. Defaults to `10m`.
`spec.iso.retainVersions`	Number of content-addressed ISO objects to keep. Defaults to `2`.
`spec.iso.pathPrefix`	Datastore directory for cached ISO objects.
`spec.cleanupPolicy`	`Delete` removes stale owned VMs and unbound Agents. `Retain` leaves them in place. Defaults to `Delete`.

Status

VsphereAgentPool.status reports observed AgentMachine demand, matching Agent capacity, owned VM identities, active ISO cache state, planned actions, and conditions.

Create DNS records from Gateway API routes

Mon, 01 Jan 0001 00:00:00 +0000

cloudflare-operator can create DNS records from Gateway API route resources. This is an opt-in feature controlled by --enable-gateway-api.

Supported route kinds:

HTTPRoute
TLSRoute
GRPCRoute

TCPRoute and UDPRoute are not supported directly because Gateway API does not define route-level hostnames for them. If you need DNS for TCP or UDP traffic, create a DNSRecord resource manually for the hostname that should resolve to your gateway.

Prerequisites

Install the Gateway API CRDs (for example gateway.networking.k8s.io/v1 from the upstream install manifests).
Start cloudflare-operator with the flag --enable-gateway-api.

When installed with Helm, enable Gateway API reconciliation with:

Get Started

Mon, 01 Jan 0001 00:00:00 +0000

This tutorial walks through a first VsphereAgentPool and active VM reconciliation.

Before you begin

The following prerequisites are required:

A Kubernetes or OpenShift management cluster with agent-forge-operator installed.
HyperShift and Assisted Installer resources.
A hosted cluster that uses the Agent platform.
A NodePool for the hosted cluster.
CAPI AgentMachine and Machine objects rendered for that NodePool in the hosted control plane namespace.
An InfraEnv in the hosted cluster namespace with status.isoDownloadURL populated.
vSphere credentials that can upload the discovery ISO, create VMs, power on VMs, and destroy operator-owned VMs.

The examples use these names:

Get Started

Mon, 01 Jan 0001 00:00:00 +0000

This tutorial shows you how to get started with using cloudflare-operator and create a sample DNS record.

Before you begin

The following prerequisites are required to complete this tutorial:

A Kubernetes cluster with cloudflare-operator installed (follow the installation guide)
A Cloudflare account

Create Cloudflare API token

The token can be created by following this guide.

The following permissions are required:

Zone:Zone:Read
Zone:DNS:Edit

Configure the following Zone resources:

Include:All zones

or, if you want to limit the zones to which the token has access:

Monitoring with Prometheus

Mon, 01 Jan 0001 00:00:00 +0000

Prerequisites

The easiest way to deploy all the necessary components is to use kube-prometheus-stack.

Enable metrics

In order to enable metrics and automatically deploy the required resources, you need to reconfigure the Helm chart.

Create a values.yaml file with the following content:

---
metrics:
  podMonitor:
    enabled: true
  prometheusRule:
    enabled: true

Now you can install / upgrade the Helm chart by following the installation guide.

Install cloudflare-operator Grafana dashboard

# Download Grafana dashboard
wget https://raw.githubusercontent.com/containeroo/cloudflare-operator/master/config/manifests/grafana/dashboards/overview.json -O /tmp/grafana-dashboard-cloudflare-operator.json

# Create the configmap
kubectl create configmap grafana-dashboard-cloudflare-operator --from-file=/tmp/grafana-dashboard-cloudflare-operator.json

# Add label so Grafana can fetch dashboard
kubectl label configmap grafana-dashboard-cloudflare-operator grafana_dashboard="1"

Available metrics

For each cloudflare-operator.io kind, the controller exposes a gauge metric to track the status condition.

Troubleshooting

Mon, 01 Jan 0001 00:00:00 +0000

AgentMachine demand is not observed

Check AgentMachines in the hosted control plane namespace:

kubectl -n demo-demo get agentmachines.capi-provider.agent-install.openshift.io -o yaml
kubectl -n demo get vsphereagentpool demo-worker \
  -o jsonpath='{.status.conditions[?(@.type=="AgentMachineDemandFound")]}{"\n"}'

Confirm that the AgentMachine has the expected hypershift.openshift.io/nodePool annotation and reports Ready=False with Reason=NoSuitableAgents.

InfraEnv ISO is unavailable

kubectl -n demo get infraenv demo -o yaml
kubectl -n demo get vsphereagentpool demo-worker \
  -o jsonpath='{.status.conditions[?(@.type=="InfraEnvAvailable")]}{"\n"}'

The operator cannot create bootable discovery VMs until status.isoDownloadURL is populated on the InfraEnv.

VsphereAgent

Mon, 01 Jan 0001 00:00:00 +0000

The API specification can be viewed here.

VsphereAgent represents one vSphere VM requested to satisfy AgentMachine demand.

Most users do not create VsphereAgent resources manually. The AgentMachine controller creates one VsphereAgent for each waiting AgentMachine when the selected VsphereAgentPool needs additional capacity.

Example

apiVersion: agent-forge.containeroo.ch/v1alpha1
kind: VsphereAgent
metadata:
  name: demo-worker-abc12
  namespace: demo
spec:
  poolRef:
    name: demo-worker

Spec

Field	Description
`spec.poolRef.name`	Name of the `VsphereAgentPool` whose configuration is used to create and manage the VM.

Status

Field	Description
`status.observedGeneration`	Latest resource generation reconciled by the controller.
`status.vm.name`	vSphere VM name.
`status.vm.biosUUID`	VM BIOS UUID when known.
`status.vm.macAddress`	Primary NIC MAC address when known.
`status.vm.agentRef`	Discovered Assisted Installer Agent matched to this VM.
`status.vm.machineRef`	CAPI Machine paired with this VM when bound.
`status.vm.phase`	VM lifecycle phase such as `Provisioning`, `Available`, `Bound`, `Released`, or `Orphaned`.
`status.conditions`	Readiness and provider errors.

When a VsphereAgent is deleted, its finalizer deletes the paired vSphere VM unless the owning VsphereAgentPool uses spec.cleanupPolicy: Retain.

Zone

Mon, 01 Jan 0001 00:00:00 +0000

The API specification can be viewed here.

Zone resources represent Cloudflare zones and are used to tell cloudflare-operator which zones to manage.

Learn more about the zone resource in the getting started guide.

If you have more than one Account resource, set spec.accountRef.name on each Zone so the operator knows which Cloudflare credentials to use. When only one Account exists in the cluster, accountRef can be omitted.

Prevent Certain DNS Records From Being Deleted When Zone Prune Is Enabled

Prior to v1.7.0, cloudflare-operator had a fixed set of DNS records that were always ignored when Zone prune was enabled. These included records required for ACME certificate validation and Cloudflare-specific TXT records. Users could not modify this behavior.

Cleanup Policy

Mon, 01 Jan 0001 00:00:00 +0000

spec.cleanupPolicy controls whether agent-forge-operator deletes external inventory when demand disappears or when a pool is deleted.

Delete

Delete is the default policy.

spec:
  cleanupPolicy: Delete

With Delete, the operator removes stale owned vSphere VMs and unbound Agents when they are no longer needed. Scale-down is conservative: the controller first observes a paired CAPI Machine entering deletion, waits until that Machine has disappeared, and only then deletes the paired VsphereAgent, VM, and stale Agent.

DNSRecord

Mon, 01 Jan 0001 00:00:00 +0000

The API specification can be viewed here.

DNSRecord resources represent DNS records in the Cloudflare account.

For each DNS record, you need to create a DNSRecord resource.

cloudflare-operator will create the DNS record in the Cloudflare account and keep it in sync with the DNSRecord resource.

As described in the Ingress and Gateway API guides, cloudflare-operator can create DNSRecords from Ingress and supported Gateway API route resources.

DNSRecords are reconciled through the Account selected by their matching Zone. If the Zone does not set spec.accountRef.name, a DNSRecord can set spec.accountRef.name directly. If both the Zone and DNSRecord set an Account, they must reference the same Account.

Dynamic DNS with IP objects

Mon, 01 Jan 0001 00:00:00 +0000

As described in the core concept, IP objects allow you to use cloudflare-operator as a dynamic DNS controller.

In this guide, we will configure an IP object to fetch the public IP address from the internet and use it as the target content for a DNS record.

Simple example

Create an IP object with the following content:

---
apiVersion: cloudflare-operator.io/v1
kind: IP
metadata:
  name: external-v4
spec:
  ipSources:
    - requestMethod: GET
      url: https://ifconfig.me/ip
    - requestMethod: GET
      url: https://ipecho.net/plain
    - requestMethod: GET
      url: https://myip.is/ip/
    - requestMethod: GET
      url: https://checkip.amazonaws.com
    - requestMethod: GET
      url: https://api.ipify.org
  type: dynamic
  interval: 5m0s

The IP object will fetch the public IP address from the internet using one of the specified URLs.
If the request fails, the next URL will be tried. If all URLs fail, the last known IP will be used and the ready condition will be set to false.

Installation

Mon, 01 Jan 0001 00:00:00 +0000

This guide walks you through installing agent-forge-operator.

Prerequisites

Install Helm 3.
Kubernetes or OpenShift cluster.
HyperShift and Assisted Installer resources for Agent platform clusters.

Install agent-forge-operator

Helm repository

Add the containeroo Helm chart repository:

helm repo add containeroo https://charts.containeroo.ch

Update the Helm chart repository:

helm repo update

Custom Resource Definitions

The Helm chart includes the VsphereAgent and VsphereAgentPool CRDs in the chart crds/ directory. Helm installs CRDs on first install, but does not upgrade or delete CRDs during normal chart upgrades.

Installation

Mon, 01 Jan 0001 00:00:00 +0000

This guide walks you through installing cloudflare-operator.

Prerequisites

Install Helm 3
Kubernetes cluster

Install cloudflare-operator

Helm repository

Add the cloudflare-operator Helm chart repository:

helm repo add containeroo https://charts.containeroo.ch

Update the Helm chart repository:

helm repo update

Custom Resource Definitions

As per the Helm best practices, cloudflare-operator Helm chart doesn’t ship with CRDs.

To install the latest CRDs, run the following command:

kubectl apply -f https://github.com/containeroo/cloudflare-operator/releases/latest/download/crds.yaml

If you want to install a specific version of CRDs, run the following command:

Zone Migration

Mon, 01 Jan 0001 00:00:00 +0000

Starting with version v1.4.0, DNS record management has been redesigned to support externally managed records. This update requires users to explicitly specify which zones the operator should manage, providing greater control and preventing unintended modifications.

For users already running cloudflare-operator prior to v1.4.0, existing functionality remains unchanged, except that DNS records are no longer automatically pruned. To restore the previous behavior, set the spec.prune field to true in the Zone object.

Fetch IP from a Kubernetes object

Mon, 01 Jan 0001 00:00:00 +0000

If you have a Kubernetes object that contains an IP address, you can use cloudflare-operator to fetch the IP address from the object and use it as the target content for a DNS record.

This can be useful if you are using Istio or want to fetch the IP from a Kubernetes service.

This guide will show you how to configure an IP object to fetch the IP address from a Kubernetes service.

IP

Mon, 01 Jan 0001 00:00:00 +0000

The API specification can be viewed here.

As described in the core concept, IP objects can be used to follow the “don’t repeat yourself” (DRY) principle.

They also allow you to use the same IP address in multiple DNS records and let you use cloudflare-operator as a dynamic DNS service.

Learn more about the usage of IP objects in the dynamic DNS guide.

API Reference

Mon, 01 Jan 0001 00:00:00 +0000

Packages:

agent-forge.containeroo.ch/v1alpha1

agent-forge.containeroo.ch/v1alpha1

Package v1alpha1 contains API Schema definitions for the agent-forge.containeroo.ch v1alpha1 API group.

Resource Types:

AgentBindingSpec

(Appears on: VsphereAgentPoolSpec)

AgentBindingSpec describes how a discovered Assisted Installer Agent should be made consumable by the Hypershift Agent NodePool.

Field Description

role
string

Role is the Hypershift NodePool role label value to apply to discovered Agents. For worker pools this is normally “worker”.

labels
map[string]string

Labels are required on a discovered Agent before the Agent CAPI provider can bind it to an AgentMachine. These should match the NodePool spec.platform.agent.agentLabelSelector labels.

API Reference

Mon, 01 Jan 0001 00:00:00 +0000

Packages:

cloudflare-operator.io/v1

cloudflare-operator.io/v1

Package v1 contains API Schema definitions for the source v1 API group

Resource Types:

Account

Account is the Schema for the accounts API

Field Description

metadata
Kubernetes meta/v1.ObjectMeta Refer to the Kubernetes API documentation for the fields of the metadata field.

spec
AccountSpec

apiToken
AccountSpecApiToken

Cloudflare API token

interval
Kubernetes meta/v1.Duration

(Optional)

Interval to check account status

managedZones
[]string

(Optional)

List of zone names that should be managed by cloudflare-operator Deprecated and will be removed in a future release

Migration to cloudflare-operator

Mon, 01 Jan 0001 00:00:00 +0000

cloudflare-operator is designed to be the single source of truth for DNS records. This means that all records not known to cloudflare-operator will be deleted.

In order to prevent the deletion of the entire DNS zone after you have installed cloudflare-operator, you need to migrate your DNS records to cloudflare-operator.

For that we have created a migration tool that generates DNSRecord objects for all DNS records in your DNS zone.

To export your DNS records, follow this guide.

Traefik 2.2 + Docker: Global entrypoint configuration

Wed, 25 Mar 2020 00:00:00 +0000

Image Source

Introduction

With Traefik 2.2 it is now easier then ever to globally configure your entrypoints.

We will show you how you can define a global redirect to https and how to set a default certResolver. So you don’t have to set the https redirect in each docker-compose file.

Prerequisites

In order to follow along, you’ll need to read the advanced Traefik guide and the wildcard guide first!

Update Traefik Configuration

Change the entrypoints in the basic Traefik configuration file (/opt/containers/traefik/data/traefik.yml) as follows:

Traefik 2.0: Route external services through Traefik

Thu, 02 Jan 2020 00:00:00 +0000

Image Source

Introduction

In this tutorial we will show you how you can route non-Docker services through Traefik.

Let’s suppose you want to access your Pi-hole admin console (http://192.168.0.10:80/admin) by browsing to pihole.example.com.

Prerequisites

You have read our other articles:

and you use this Traefik configuration.

Make sure you configure in the providers section of your /opt/traefik/data/traefik.yml an external configuration file /config.yml.

Traefik 2.0: Paranoid about mounting /var/run/docker.sock?

Tue, 12 Nov 2019 00:00:00 +0000

Image Source

The Problem

If you have followed our previous guides, you mount the Docker Socket ( /var/run/docker.sock) into the Traefik container. If someone gets access into the Traefik container, they can gain full access to your host machine. This makes our paranoia level increase slightly…

The Solution

We found a nice little container (Socket-Proxy) which “filters” all requests to the Docker API. We can allow only get requests to the Docker API and restrict it to /containers/*.

Traefik 2.0: Wildcard Let's Encrypt Certificates

Tue, 12 Nov 2019 00:00:00 +0000

Image Source

Introduction

In this tutorial we will setup Traefik to obtain wildcard certificates from Let’s Encrypt. This requires DNS challenge to be setup. Usually Traefik obtains a certificate for every subdomain. We can simplify this process by telling Traefik to use a wildcard (*.example.com) certificate instead.

Prerequisites

Registered Domain
Authoritative DNS Servers from one of these providers (you may need to change your DNS servers of your domain to one of the provider in the list)

In this tutorial we will use Cloudflare as our DNS servers for our domain.

Using Pi-hole to route your services internally

Tue, 12 Nov 2019 00:00:00 +0000

Image Source

The Problem

If you have followed our previous guides, chances are that you have a domain, some DNS records pointing to your public IP, port forwarding enabled and a Docker server running some services.

Most likely your domain resolves to your public IP from you internal network as well. This causes a problem: All the traffic between your devices (e.g. your phone) to your server (physically in the same location) gets routed trough the internet, which means you have to utilize your upload and download bandwidth at the same time (e.g. while streaming from Plex), which not only causes a slower connection but also adds an unnecessary high latency.

Traefik 2.0 + Docker: A Simple Step by Step Guide

Tue, 24 Sep 2019 00:00:00 +0000

Image Source

Introduction

In this tutorial we will go trough the following things:

Setup and configure Traefik in a Docker container
Let’s Encrypt setup for automatic HTTPS certificates
Deploy a simple service (Portainer) and expose it to the internet

You will find all the required configuration files in our Git repository.

Prerequisites

In order to follow along, you need these things:

Docker (obviously)
Docker Compose
A domain
Ports 80 and 443 forwarded to your Docker host

Setup and configure Traefik with Let’s Encrypt

Let’s get started by setting up Traefik.

Traefik 2.0 + Docker: An Advanced Guide

Tue, 24 Sep 2019 00:00:00 +0000

Image Source

Introduction

This tutorial is the second part of this article. We will go trough the following configurations:

Add a file provider to traefik.yml
Create a config file for a central configuration for storing middlewares config.yml.
Configure a middleware chain

You will find all the required configuration files in our Git repository.

Prerequisites

In order to follow along, you’ll need to read this post!

Update Traefik configuration

To setup a reusable middleware add an additional provider in the Traefik configuration file traefik.yml (/opt/containers/traefik/traefik.yml) (lines 15 and 16).

Search Results

Mon, 01 Jan 0001 00:00:00 +0000