Kubernetes has several ways to run workloads across your clusters, and one of those specialized tools is the Kubernetes DaemonSet. A Daemon set makes sure that a specific pod will run on every nod in your cluster. This makes it an ideal situation for running background system-level services like logging, monitoring, or networking agents.
In this guide, we will walk through Kubernetes DaemonSet, how it works, when to use it, and how to create and manage one with real-world examples.
What Is a DaemonSet in Kubernetes?
A Kubernetes DaemonSet is a type of a controller that ensures that a specific pod runs on every node in the cluster. Unlike a Deployment, in which pods only run on selected nodes based on scaling or scheduling, a DaemonSet guarantees that each node runs exactly one instance of the pod (unless configured in another manner).
DaemonSets are perfect for running background services such as log collectors, monitoring agents, and network daemons that need to be present on all nodes.
How DaemonSets Work in a Kubernetes Cluster
When you create a Kubernetes DaemonSet, it will automatically schedule a pod from that set onto every node in the cluster. If a new node is added, Kubernetes automatically adds the DaemonSet pod to it. If you remove a node, the corresponding pod is also deleted.
DaemonSets work closely with the Kubelet, which makes sure that the specified pod runs on the intended nodes. You can also configure the DaemonSet to run pods on certain nodes.
Get exclusive access to all things tech-savvy, and be the first to receive
the latest updates directly in your inbox.
Related Articles: Understanding Kubernetes Workloads: Types, Use Cases & Best Practices
Common Use Cases for DaemonSets
DaemonSets are the ideal solution for tasks that need to run consistently across the specific nodes in a Kubernetes cluster. Common examples include:
- Log collection: Running Fluentd, Filebeat, or Logstash to collect and ship logs from all nodes.
- Monitoring agents: Tools like Prometheus Node Exporter or Datadog agents to collect system metrics.
- Security scanners: Kubernetes DaemonSets can deploy the tools that can help check for vulnerabilities.
- Storage drivers: Some storage plugins can run as Kubernetes DaemonSets to interact with node-level storage.
- Custom node services: Any service that needs direct node-level access or visibility.
DaemonSet vs Deployment vs StatefulSet
| Feature | DaemonSet | Deployment | StatefulSet |
| Pod Distribution | One per node (or selected nodes) | Distributed based on replicas | One per replica with stable identity |
| Use Case | System-level agents, monitoring | Stateless apps, web services | Stateful apps like databases |
| Pod Identity | Not persistent | Not persistent | Persistent with unique identity and storage |
| Volume Behavior | Usually not persistent | Usually not persistent | Supports persistent volumes per pod |
| Update Strategy | Rolling updates (can be configured) | Rolling updates | Ordered, controlled updates |
How to Create a DaemonSet in Kubernetes
Creating a DaemonSet involves defining a YAML file that includes a DaemonSet kind. Main features of a DaemonSet in Kubernetes includes:
- selector: Matches the pods controlled by the DaemonSet
- template: Describes the pod to deploy
- tolerations and nodeSelector: (optional) Control where the pods run
You can apply a DaemonSet YAML using:
kubectl apply -f my-daemonset.yaml
Once applied, Kubernetes will schedule the defined pod on every applicable node.
Kubernetes DaemonSet YAML Example
Here’s a simple YAML example that deploys a Fluentd logging agent on every node:
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: fluentd
namespace: kube-system
spec:
selector:
matchLabels:
app: fluentd
template:
metadata: labels:
app: fluentd
spec:
containers:
– name: fluentd
image: fluent/fluentd:latest
resources:
limits:
memory: 200Mi
requests:
cpu: 100m
memory: 200Mi
volumeMounts:
– name: varlog
mountPath: /var/log
volumes:
– name: varlog
hostPath: path: /var/log
This DaemonSet ensures the fluentd container runs on every node and collects logs from
/var/log.
Kubernetes DaemonSet YAML Example
Here is a basic YAML configuration for a DaemonSet that runs on a custom log collector on the every node:
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: log-agent
labels:
app: log-agent
spec:
selector:
matchLabels:
app: log-agent
template:
metadata:
labels:
app: log-agent
spec:
containers:
– name: log-agent
image: yourregistry/log-agent:latest
volumeMounts:
– name: logs
mountPath: /var/log
volumes:
– name: logs
hostPath:
path: /var/log
This configuration ensures that a log-agent pod runs on every node, with access to the /var/log directory to collect or forward system logs.
Updating and Deleting a Kubernetes DaemonSet
To update and delete a Kubernetes DaemonSet, follow the following steps:
Updating a DaemonSet
DaemonSets can be updated with the rolling update strategy if defined in the update strategy section of the YAML:
To update:
- Modify the YAML file (e.g., update the container image).
- Apply the changes:
kubectl apply -f daemonset.yaml
To check update progress:
kubectl rollout status daemonset/log-agent
Deleting a DaemonSet
To delete a DaemonSet and remove all its pods from the nodes:
kubectl delete daemonset log-agent
The pods managed by the DaemonSet will also be deleted from every node automatically.
Scheduling and Node Selectors in DaemonSets
By default, a DaemonSet sets pods on every node, but you can control where to deploy the pods using:
- nodeSelector: Match pods to nodes with specific labels
- affinity: More advanced rules for scheduling
- tolerations: Allow pods to run on tainted nodes
- hostNetwork: true: Useful for networking agents that need access to node-level interfaces
Example: Scheduling on nodes labeled monitoring=true:
spec:
template:
spec:
nodeSelector:
monitoring: “true”
Example: Using a toleration for tainted nodes:
tolerations:
– key: “node-role.kubernetes.io/master”
operator: “Exists”
effect: “NoSchedule”
These options are useful for targeting the Kubernetes DaemonSets to different node types, such as master nodes, GPU nodes, or custom roles.
Best Practices for Using DaemonSets
To make the most out of Kubernetes DaemonSets, use the following best practices:
- Be very careful when you are selecting the nodes and tolerations, and only target the ones that really need it.
- Since Kubernetes DaemonSets are running everywhere, ensure that the pods are resource-efficient.
- Configure the Kubernetes DaemonSet to use RollingUpdate so updates are done smoothly across the cluster.
- Set resource limits and requests to avoid overwhelming nodes.
- Track the changes to your Kubernetes DaemonSet configurations for better collaboration and rollback if needed.
- Place the system level DaemonSets in an appropriate namespaces (e.g., kube-system) for better organization.
Conclusion – Leveraging Kubernetes DaemonSets
Kubernetes DaemonSets are one of the most powerful tools for running system-level pods across the clusters. By learning, understanding, and implementing best practices, you can easily leverage their power!
What is the difference between a DaemonSet and a Deployment in Kubernetes?
A Deployment manages a scalable set of identical pods, distributing them based on desired replicas. A DaemonSet, on the other hand, ensures exactly one pod runs on each target node.
Can I update or delete a DaemonSet?
Yes. You can update a DaemonSet by changing its YAML definition and applying the changes. Deleting it removes all the pods it created from the cluster.
How do I schedule a DaemonSet on specific nodes only?
You can use nodeSelector, nodeAffinity, or tolerations in the pod spec to limit which nodes receive the DaemonSet pods based on labels or taints.
