The following configuration options document all features available for writing policy files with PowerfulSeal.
General configuration of this policy.
Configure how exit is configured
Affects what PowerfulSeal does when errors occur. The default report logs an error message, updates metrics and carries on. fail-fast fails and exits immediately on first exit (useful when running as a job).
Configure how the scenarios are run
The Seal waits a randomized amount of time before executing the scenarios. This sets the MAX number of seconds that it will wait.
Value must be greater or equal to 0
The Seal waits a randomized amount of time before executing the scenarios. This sets the MIN number of seconds that it will wait.
Value must be greater or equal to 0
If set, it will exit after the given number of scenario runs. By default, it continues forever.
Value must be greater or equal to 1
Affects how the scenarios are executed. The default round-robin iterates over the every scenario in the order they were defined. random picks a random scenario from the available pool every time.
An array of scenarios, each describing a complete chaos experiment.
A scenario contains all the steps necessary to prepare, implement and validate a chaos engineering experiment.
A longer description, helping to understand what the scenario is doing when reading the yaml file. Optional.
A brief, descriptive name of the scenario. Will be used to identify the scenario in logs.
Must be at least 3 characters long
Must be at most 80 characters long
The sequence of events to prepare, validate, execute and analyse the chaos engineering experiment.
Issue an HTTP request and validate the response.
Body of the HTTP request.
Expected status code.
Value must be greater or equal to 1 and lesser or equal to 599
Number of requests to make.
Value must be greater or equal to 1
Delay in ms between retries.
Value must be greater or equal to 1
Endpoint to add to the url. For example /healthz. Works for both url and service.
Give any additional headers to the request.
If you'd like to skip the SSL validation. For example for self-signed certs.
HTTP method to use.
Proxy to use with the requests. If not set no proxy will be used. NOTE, that the probe ignores (and overrides) HTTP{S}_PROXY envvars.
Number of retries before failing the probe. Ignored, if there are no errors on the probe.
Value must be greater or equal to 1
A target of a static URL.
The fully qualified url to issue the request to. For example: http://example.com.
Target the given Kubernetes service in a given namespace. It will use directly the IP of the given service, so make sure it resolves.
Name of the kubernetes service.
Namespace of the kubernetes service.
Port number to hit. Independent of what the service defines.
Value must be greater or equal to 1 and lesser or equal to 65535
Protocol to use for the probe.
Request timeout in ms.
Value must be greater or equal to 1
Allows to execute a kubectl apply or kubectl delete command as part of the scenario.
When set to true, all the kubectl apply commands will be kubectl deleteed at the end of the scenario.
The path to the binary of kubectl.
Free-form, kubectl-compatible payload, which will be passed to kubectl as is.
Proxy to use with the kubectl command. If not set no proxy will be used. NOTE, that the probe ignores (and overrides) HTTP{S}_PROXY envvars.
Match, filter and action on pods in your kubernetes cluster. Matchers create the initial set. That set is de-duplicated and passed on to the filters. Filters can exclude items. Whatever is passed through the filters will then be actioned with every action in the action array. See below for the details of the configuration available.
An array of actions to be applied to each pod from the set.
Kill a pod.
Value must be greater or equal to 0 and lesser or equal to 1
Wait some seconds.
Value must be greater or equal to 0
Check that all pods are in the desired state, fail otherwise.
Status, as returned by kubernetes (Running, Terminating, etc).
Count the pods and fail if they don't match the desired number.
Stops the host on which the pods are running.
When set to true, the node will be restarted at the end of the scenario.
An array of filters to only select certain pods from the initial set.
Select pods by property values.
Set to true to negate the match (logical NOT)
Only pass the filter during specific time of the day and week.
Describes a time of the day, in the local timezone.
Value must be greater or equal to 0 and lesser or equal to 23
Value must be greater or equal to 0 and lesser or equal to 59
Value must be greater or equal to 0 and lesser or equal to 59
Describes a time of the day, in the local timezone.
Same definition as scenarios_items_steps_items_oneOf_i2_podAction_filters_items_oneOf_i1_dayTime_endTimeTake a random sample. Either a specific size (up to N), or a ratio (0.5 == 50%) of the initial size.
Value must be greater or equal to 0 and lesser or equal to 1
Value must be greater or equal to 1
Only pass the filter with a desired probability.
Value must be greater or equal to 0 and lesser or equal to 1
An array of match criteria to select a set of pods. Will be deduplicated.
Pick all pods for a deployment in a namespace.
Pick all pods for a deployment in a namespace.
Same definition as scenarios_items_steps_items_oneOf_i5_clone_source_oneOf_i0Pick all pods matching the particular set of labels in a namespace.
An object of retry criteria to rerun set actions
Retry the given action a set amount of times with a sleep in between
Value must be greater or equal to 0
Value must be greater or equal to 0
Retry the given action for a set amount of time with a sleep in between
Value must be greater or equal to 0
Value must be greater or equal to 0
Match, filter and action on nodes in your kubernetes cluster. It can integrate with your cloud provider and take nodes up and down to simulate VMs going down. If you're running in SSH mode, it can also execute various commands on hosts. And much more! See below for mode details.
An array of actions, which will be executed on each node.
Start a node.
Stop a node.
When set to true, the node will be restarted at the end of the scenario.
Executes arbitrary command on a particular node. Use with caution.
An array of filters, which will be applied in the defined order.
Select nodes by property values.
Set to true to negate the match (logical NOT)
Only pass the filter during specific time of the day and week.
Same definition as scenarios_items_steps_items_oneOf_i2_podAction_filters_items_oneOf_i1Take a random sample. Either a specific size (up to N), or a ratio (0.5 == 50%) of the initial size.
Same definition as scenarios_items_steps_items_oneOf_i2_podAction_filters_items_oneOf_i2Only pass the filter with a desired probability.
Same definition as scenarios_items_steps_items_oneOf_i2_podAction_filters_items_oneOf_i3An array of match criteria to select a set of nodes. Will be deduplicated.
Select nodes by property values.
Same definition as scenarios_items_steps_items_oneOf_i3_nodeAction_filters_items_oneOf_i0An object of retry criteria to rerun set actions
Retry the given action a set amount of times with a sleep in between
Same definition as scenarios_items_steps_items_oneOf_i2_podAction_retries_oneOf_i0Retry the given action for a set amount of time with a sleep in between
Same definition as scenarios_items_steps_items_oneOf_i2_podAction_retries_oneOf_i1Allows to wait a number of seconds.
Value must be greater or equal to 0
Clone an existing deployment and modify the clone to inject failure.
Labels to add/change for the clone. On top of all these, another one "chaos=true" will be added. If none are specified, they will be copied from the source.
Target the given Kubernetes service in a given namespace.
Name of the kubernetes service.
Namespace of the kubernetes service.
Specify Kubernetes labels
The label key
The label value
Mutations to apply to the clone. For example, you can add extra latency to the clone or modify its envvas.
Adds an init container that executes the desired tc command. Optionally, you can specify the image holding tc.
The arguments to execute as an array. For example - "qdisc add dev eth0 root netem delay 200ms" to add 200ms delay on egress. https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell
The command to execute as an array. Entrypoint in docker. https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell
The image to use.
The numeric UID of user to run as in the image. Root (UID 0) is not the safest option.
Value must be greater or equal to 0
Adds a sidecar container, and proxies all incoming ports of the cloned containers through itself. You can also specify any extra proxies.
The image to use to run iptables commands. iptables command needs to be in the path.
The image to use to deploy toxiproxy. It needs to start toxiproxy server by default and provide toxiproxy-cli in the path.
Extra proxies to be opened up by ToxiProxy.
The interface and port to listen on. For example 0.0.0.1:2222 to listen on port 2222 on all interfaces.
Arbitrary name. Needs to be unique.
The server to proxy to, along with the port. For example github.com:80
Toxics to apply. These directly in the format specified by toxiproxy. Learn more at https://github.com/shopify/toxiproxy
The name of the target proxy. If you'd like to target any of the automatically created proxies, just specify the port number. If you're like to target any of the manually created proxies, just use their name directly.
Specify the attributes of the toxic to apply. See the available ones at https://github.com/Shopify/toxiproxy#toxics
The name of the toxic's attribute.
The value of the toxic's attribute.
Specify the type of toxic to apply. See the available ones at https://github.com/Shopify/toxiproxy#toxics
The path to the toxiproxy cli in the toxiproxy image.
The numeric UID of user to run as in the image. Root (UID 0) is not the safest option.
Value must be greater or equal to 0
Target the given Kubernetes service in a given namespace.
The environment variable name
The environment variable value
Value must be greater or equal to 1
The object to clone. It must exist on the cluster. Its specs will be used for the clone.
Pick all pods for a deployment in a namespace.