Skip to content

Liveness Probe with an invalid command doesn't trigger container restarts and ContainersReady remains True  #106682

Open
@dprotaso

Description

@dprotaso

What happened?

Create a deployment with an invalid liveness probe and you'll see events like

Liveness probe errored: rpc error: code = Unknown desc = failed to exec in container: failed to start exec "184850d48d723924f62e815706ee39ce8ff8b1d36abbb78552cfe634ccbc1276": OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "invalid": executable file not found in $PATH: unknown

But the container did not restart and ContainersReady status is True

What did you expect to happen?

ContainersReady is False
Container is restarted

How can we reproduce it (as minimally and precisely as possible)?

Create a deployment with an invalid liveness probe

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test
  namespace: default
spec:
  progressDeadlineSeconds: 10
  replicas: 1
  selector:
    matchLabels:
      app: test
  template:
    metadata:
      labels:
        app: test
    spec:
        containers:
        - name: liveness
          image: k8s.gcr.io/busybox
          args:
          - /bin/sh
          - -c
          - touch /tmp/healthy; sleep 10; rm -rf /tmp/healthy; sleep 600
          livenessProbe:
            exec:
              command:
                - "invalid"
            initialDelaySeconds: 5
            periodSeconds: 5

Anything else we need to know?

No response

Kubernetes version

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.1", GitCommit:"632ed300f2c34f6d6d15ca4cef3d3c7073412212", GitTreeState:"clean", BuildDate:"2021-08-19T15:45:37Z", GoVersion:"go1.16.7", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.1", GitCommit:"5e58841cce77d4bc13713ad2b91fa0d961e69192", GitTreeState:"clean", BuildDate:"2021-05-21T23:01:33Z", GoVersion:"go1.16.4", Compiler:"gc", Platform:"linux/amd64"}

Cloud provider

Kind

OS version

# On Linux:
$ cat /etc/os-release
# paste output here
$ uname -a
# paste output here

# On Windows:
C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture
# paste output here

Install tools

Container runtime (CRI) and and version (if applicable)

Related plugins (CNI, CSI, ...) and versions (if applicable)

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bug.sig/nodeCategorizes an issue or PR as relevant to SIG Node.triage/acceptedIndicates an issue or PR is ready to be actively worked on.

    Type

    No type

    Projects

    Status

    Triaged

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions