Try Kubernetes

Get Started

Ready to get your hands dirty? Build a simple Kubernetes cluster that runs "Hello World" for Node.js.

Documentation

Learn how to use Kubernetes with the use of walkthroughs, samples, and reference documentation. You can even help contribute to the docs!

Community

If you need help, you can connect with other Kubernetes users and the Kubernetes authors, attend community events, and watch video presentations from around the web.

Blog

Read the latest news for Kubernetes and the containers space in general, and get technical how-tos hot off the presses.

Interested in hacking on the core Kubernetes code base?

View On Github

Explore the community

Tasks

Example Task Template

Extend kubectl with plugins

Manage HugePages

Schedule GPUs

Install Tools

Install Minikube

Install and Set Up kubectl

Manage Memory, CPU, and API Resources

Access Clusters Using the Kubernetes API

Access Services Running on Clusters

Advertise Extended Resources for a Node

Autoscale the DNS Service in a Cluster

Change the Reclaim Policy of a PersistentVolume

Change the default StorageClass

Cluster Management

Configure Default CPU Requests and Limits for a Namespace

Configure Default Memory Requests and Limits for a Namespace

Configure Memory and CPU Quotas for a Namespace

Configure Minimum and Maximum CPU Constraints for a Namespace

Configure Minimum and Maximum Memory Constraints for a Namespace

Configure Multiple Schedulers

Configure Out Of Resource Handling

Configure Quotas for API Objects

Configure a Pod Quota for a Namespace

Control CPU Management Policies on the Node

Customizing DNS Service

Debugging DNS Resolution

Declare Network Policy

Developing Cloud Controller Manager

Encrypting Secret Data at Rest

Guaranteed Scheduling For Critical Add-On Pods

IP Masquerade Agent User Guide

Kubernetes Cloud Controller Manager

Limit Storage Consumption

Namespaces Walkthrough

Operating etcd clusters for Kubernetes

Persistent Volume Claim Protection

Reconfigure a Node's Kubelet in a Live Cluster

Reserve Compute Resources for System Daemons

Romana for NetworkPolicy

Safely Drain a Node while Respecting Application SLOs

Securing a Cluster

Set Kubelet parameters via a config file

Set up High-Availability Kubernetes Masters

Set up a High-Availablity Etcd Cluster With Kubeadm

Share a Cluster with Namespaces

Static Pods

Storage Object in Use Protection

Use Calico for NetworkPolicy

Use Cilium for NetworkPolicy

Use Kube-router for NetworkPolicy

Using CoreDNS for Service Discovery

Using Sysctls in a Kubernetes Cluster

Using a KMS provider for data encryption

Weave Net for NetworkPolicy

Upgrading or downgrading Kubernetes

Cluster Management Guide for Version 1.6

Upgrading kubeadm HA clusters from 1.9.x to 1.9.y

Upgrading kubeadm clusters from 1.6 to 1.7

Upgrading kubeadm clusters from 1.7 to 1.8

Upgrading/downgrading kubeadm clusters between v1.8 to v1.9

Configure Pods and Containers

Assign CPU Resources to Containers and Pods

Assign Extended Resources to a Container

Assign Memory Resources to Containers and Pods

Assign Pods to Nodes

Attach Handlers to Container Lifecycle Events

Configure Liveness and Readiness Probes

Configure Pod Initialization

Configure Quality of Service for Pods

Configure Service Accounts for Pods

Configure a Pod to Use a ConfigMap

Configure a Pod to Use a PersistentVolume for Storage

Configure a Pod to Use a Projected Volume for Storage

Configure a Pod to Use a Volume for Storage

Configure a Security Context for a Pod or Container

Pull an Image from a Private Registry

Share Process Namespace between Containers in a Pod

Inject Data Into Applications

Define Environment Variables for a Container

Define a Command and Arguments for a Container

Distribute Credentials Securely Using Secrets

Expose Pod Information to Containers Through Environment Variables

Expose Pod Information to Containers Through Files

Inject Information into Pods Using a PodPreset

Run Applications

Delete a StatefulSet

Force Delete StatefulSet Pods

Horizontal Pod Autoscaler

Horizontal Pod Autoscaler Walkthrough

Perform Rolling Update Using a Replication Controller

Run a Replicated Stateful Application

Run a Single-Instance Stateful Application

Run a Stateless Application Using a Deployment

Scale a StatefulSet

Specifying a Disruption Budget for your Application

Update API Objects in Place Using kubectl patch

Run Jobs

Parallel Processing using Expansions

Running automated tasks with cron jobs

Access Applications in a Cluster

Accessing Clusters

Communicate Between Containers in the Same Pod Using a Shared Volume

Configure Access to Multiple Clusters

Configure Your Cloud Provider's Firewalls

Connect a Front End to a Back End Using a Service

Create an External Load Balancer

List All Container Images Running in a Cluster

Provide Load-Balanced Access to an Application in a Cluster

Use Port Forwarding to Access Applications in a Cluster

Use a Service to Access an Application in a Cluster

Web UI (Dashboard)

Monitor, Log, and Debug

Application Introspection and Debugging

Auditing

Core metrics pipeline

Debug Init Containers

Debug Pods and Replication Controllers

Debug Services

Debug a StatefulSet

Determine the Reason for Pod Failure

Developing and debugging services locally

Events in Stackdriver

Get a Shell to a Running Container

Logging Using Elasticsearch and Kibana

Logging Using Stackdriver

Monitor Node Health

Tools for Monitoring Compute, Storage, and Network Resources

Troubleshoot Applications

Troubleshoot Clusters

Troubleshooting

Extend Kubernetes

Configure the aggregation layer

Extend the Kubernetes API with CustomResourceDefinitions

Extend the Kubernetes API with ThirdPartyResources

Migrate a ThirdPartyResource to CustomResourceDefinition

Setup an extension API server

Use an HTTP Proxy to Access the Kubernetes API

TLS

Certificate Rotation

Manage TLS Certificates in a Cluster

Federation - Run an App on Multiple Clusters

Cross-cluster Service Discovery using Federated Services

Set up Cluster Federation with Kubefed

Set up CoreDNS as DNS provider for Cluster Federation

Set up placement policies in Federation

Manage Cluster Daemons

Perform a Rolling Update on a DaemonSet

Performing a Rollback on a DaemonSet

Extend Kubernetes

Install Service Catalog using Helm

Install Service Catalog using SC

Federation - Run an App on Multiple Clusters

Federated Horizontal Pod Autoscalers (HPA)

Federated Ingress

Federated Jobs

Federated Namespaces

Federated ReplicaSets

Federated Secrets

Edit This Page

Storage Object in Use Protection

FEATURE STATE: Kubernetes v1.10 beta

Persistent volume claims (PVCs) that are in active use by a pod and persistent volumes (PVs) that are bound to PVCs can be protected from pre-mature removal.

Before you begin
Storage Object in Use Protection feature used for PVC Protection
Storage Object in Use Protection feature used for PV Protection

Before you begin

The Storage Object in Use Protection feature is enabled in a version of Kubernetes in which it is supported.

Storage Object in Use Protection feature used for PVC Protection

The example below uses a GCE PD StorageClass, however, similar steps can be performed for any volume type.

Create a StorageClass for convenient storage provisioning:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: slow
provisioner: kubernetes.io/gce-pd
parameters:
  type: pd-standard

Verification scenarios follow below.

Scenario 1: The PVC is not in active use by a pod

Create a PVC:

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: slzc
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: slow
  resources:
    requests:
      storage: 3.7Gi

Check that the PVC has the finalizer kubernetes.io/pvc-protection set:

kubectl describe pvc slzc
Name:          slzc
Namespace:     default
StorageClass:  slow
Status:        Bound
Volume:        pvc-bee8c30a-d6a3-11e7-9af0-42010a800002
Labels:        <none>
Annotations:   pv.kubernetes.io/bind-completed=yes
               pv.kubernetes.io/bound-by-controller=yes
               volume.beta.kubernetes.io/storage-provisioner=kubernetes.io/gce-pd
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      4Gi
Access Modes:  RWO
Events:
  Type    Reason                 Age   From                         Message
  ----    ------                 ----  ----                         -------
  Normal  ProvisioningSucceeded  2m    persistentvolume-controller  Successfully provisioned volume pvc-bee8c30a-d6a3-11e7-9af0-42010a800002 using kubernetes.io/gce-pd

Delete the PVC and check that the PVC (not in active use by a pod) was removed successfully.

Scenario 2: The PVC is in active use by a pod

Again, create the same PVC.
Create a pod that uses the PVC:

kind: Pod
apiVersion: v1
metadata:
  name: app1
spec:
  containers:
  - name: test-pod
    image: k8s.gcr.io/busybox:1.24
    command:
      - "/bin/sh"
    args:
      - "-c"
      - "date > /mnt/app1.txt; sleep 60 && exit 0 || exit 1"
    volumeMounts:
      - name: path-pvc
        mountPath: "/mnt"
  restartPolicy: "Never"
  volumes:
    - name: path-pvc
      persistentVolumeClaim:
        claimName: slzc

Wait until the pod status is Running, i.e. the PVC becomes in active use.
Delete the PVC that is now in active use by a pod and verify that the PVC is not removed but its status is Terminating:

Name:          slzc
Namespace:     default
StorageClass:  slow
Status:        Terminating (since Fri, 01 Dec 2017 14:47:55 +0000)
Volume:        pvc-803a1f4d-d6a6-11e7-9af0-42010a800002
Labels:        <none>
Annotations:   pv.kubernetes.io/bind-completed=yes
               pv.kubernetes.io/bound-by-controller=yes
               volume.beta.kubernetes.io/storage-provisioner=kubernetes.io/gce-pd
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      4Gi
Access Modes:  RWO
Events:
  Type    Reason                 Age   From                         Message
  ----    ------                 ----  ----                         -------
  Normal  ProvisioningSucceeded  52s   persistentvolume-controller  Successfully provisioned volume pvc-803a1f4d-d6a6-11e7-9af0-42010a800002 using kubernetes.io/gce-pd

Wait until the pod status is Terminated (either delete the pod or wait until it finishes). Afterwards, check that the PVC is removed.

Scenario 3: A pod starts using a PVC that is in Terminating state

Again, create the same PVC.
Create a first pod that uses the PVC:

kind: Pod
apiVersion: v1
metadata:
  name: app1
spec:
  containers:
  - name: test-pod
    image: k8s.gcr.io/busybox:1.24
    command:
      - "/bin/sh"
    args:
      - "-c"
      - "date > /mnt/app1.txt; sleep 600 && exit 0 || exit 1"
    volumeMounts:
      - name: path-pvc
        mountPath: "/mnt"
  restartPolicy: "Never"
  volumes:
    - name: path-pvc
      persistentVolumeClaim:
        claimName: slzc

Wait until the pod status is Running, i.e. the PVC becomes in active use.
Delete the PVC that is now in active use by a pod and verify that the PVC is not removed but its status is Terminating:

Name:          slzc
Namespace:     default
StorageClass:  slow
Status:        Terminating (since Fri, 01 Dec 2017 14:47:55 +0000)
Volume:        pvc-803a1f4d-d6a6-11e7-9af0-42010a800002
Labels:        <none>
Annotations:   pv.kubernetes.io/bind-completed=yes
               pv.kubernetes.io/bound-by-controller=yes
               volume.beta.kubernetes.io/storage-provisioner=kubernetes.io/gce-pd
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      4Gi
Access Modes:  RWO
Events:
  Type    Reason                 Age   From                         Message
  ----    ------                 ----  ----                         -------
  Normal  ProvisioningSucceeded  52s   persistentvolume-controller  Successfully provisioned volume pvc-803a1f4d-d6a6-11e7-9af0-42010a800002 using kubernetes.io/gce-pd

Create a second pod that uses the same PVC:

kind: Pod
apiVersion: v1
metadata:
  name: app2
spec:
  containers:
  - name: test-pod
    image: gcr.io/google_containers/busybox:1.24
    command:
      - "/bin/sh"
    args:
      - "-c"
      - "date > /mnt/app1.txt; sleep 600 && exit 0 || exit 1"
    volumeMounts:
      - name: path-pvc
        mountPath: "/mnt"
  restartPolicy: "Never"
  volumes:
    - name: path-pvc
      persistentVolumeClaim:
        claimName: slzc

Verify that the scheduling of the second pod fails with the below warning:

Warning  FailedScheduling  18s (x4 over 21s)  default-scheduler  persistentvolumeclaim "slzc" is being deleted

Wait until the pod status of both pods is Terminated or Completed (either delete the pods or wait until they finish). Afterwards, check that the PVC is removed.

Storage Object in Use Protection feature used for PV Protection

The example below uses a HostPath PV.

Verification scenarios follow below.

Scenario 1: The PV is not bound to a PVC

Create a PV:

kind: PersistentVolume
apiVersion: v1
metadata:
  name: task-pv-volume
  labels:
    type: local
spec:
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Delete
  storageClassName: standard
  hostPath:
    path: "/tmp/data"

Check that the PV has the finalizer kubernetes.io/pv-protection set:

Name:            task-pv-volume
Labels:          type=local
Annotations:     pv.kubernetes.io/bound-by-controller=yes
Finalizers:      [kubernetes.io/pv-protection]
StorageClass:    standard
Status:          Terminating (lasts 1m)
Claim:           default/task-pv-claim
Reclaim Policy:  Delete
Access Modes:    RWO
Capacity:        1Gi
Message:         
Source:
    Type:          HostPath (bare host directory volume)
    Path:          /tmp/data
    HostPathType:  
Events:            <none>

Delete the PV and check that the PV (not bound to a PVC) is removed successfully.

Scenario 2: The PV is bound to a PVC

Again, create the same PV.
Create a PVC

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: task-pv-claim
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

Wait until the PV and PVC are bound to each other.
Delete the PV and verify that the PV is not removed but its status is Terminating:

NAME             CAPACITY     ACCESS MODES   RECLAIM POLICY   STATUS        CLAIM                   STORAGECLASS   REASON    AGE
task-pv-volume   1Gi          RWO            Delete           Terminating   default/task-pv-claim   standard                 59s

Delete the PVC and verify that the PV is removed too.

kubectl delete pvc task-pv-claim
persistentvolumeclaim "task-pv-claim" deleted
$ kubectl get pvc
No resources found.
$ kubectl get pv
No resources found.

Create an Issue Edit this Page