k8sgpt
is a tool for scanning your Kubernetes clusters, diagnosing, and triaging issues in simple English.
It has SRE experience codified into its analyzers and helps to pull out the most relevant information to enrich it with AI.
Out of the box integration with OpenAI, Azure, Cohere, Amazon Bedrock, Google Gemini and local models.
$ brew install k8sgpt
or
brew tap k8sgpt-ai/k8sgpt
brew install k8sgpt
32 bit:
sudo rpm -ivh https://github.com/k8sgpt-ai/k8sgpt/releases/download/v0.3.41/k8sgpt_386.rpm
64 bit:
sudo rpm -ivh https://github.com/k8sgpt-ai/k8sgpt/releases/download/v0.3.41/k8sgpt_amd64.rpm
32 bit:
curl -LO https://github.com/k8sgpt-ai/k8sgpt/releases/download/v0.3.41/k8sgpt_386.deb
sudo dpkg -i k8sgpt_386.deb
64 bit:
curl -LO https://github.com/k8sgpt-ai/k8sgpt/releases/download/v0.3.41/k8sgpt_amd64.deb
sudo dpkg -i k8sgpt_amd64.deb
32 bit:
wget https://github.com/k8sgpt-ai/k8sgpt/releases/download/v0.3.41/k8sgpt_386.apk
apk add --allow-untrusted k8sgpt_386.apk
64 bit:
wget https://github.com/k8sgpt-ai/k8sgpt/releases/download/v0.3.41/k8sgpt_amd64.apk
apk add --allow-untrusted k8sgpt_amd64.apk
==> Installing k8sgpt from k8sgpt-ai/k8sgpt Error: The following formula cannot be installed from a bottle and must be
built from the source. k8sgpt Install Clang or run brew install gcc.
If you install gcc as suggested, the problem will persist. Therefore, you need to install the build-essential package.
sudo apt-get update
sudo apt-get install build-essential
To install within a Kubernetes cluster please use our k8sgpt-operator
with installation instructions available here
This mode of operation is ideal for continuous monitoring of your cluster and can integrate with your existing monitoring such as Prometheus and Alertmanager.
k8sgpt generate
to open a browser link to generate itk8sgpt auth add
to set it in k8sgpt.
--password
flag.k8sgpt filters
to manage the active filters used by the analyzer. By default, all filters are executed during analysis.k8sgpt analyze
to run a scan.k8sgpt analyze --explain
to get a more detailed explanation of the issues.k8sgpt analyze --with-doc
(with or without the explain flag) to get the official documentation from Kubernetes.K8sGPT uses analyzers to triage and diagnose issues in your cluster. It has a set of analyzers that are built in, but you will be able to write your own analyzers.
Run a scan with the default analyzers
k8sgpt generate
k8sgpt auth add
k8sgpt analyze --explain
k8sgpt analyze --explain --with-doc
Filter on resource
k8sgpt analyze --explain --filter=Service
Filter by namespace
k8sgpt analyze --explain --filter=Pod --namespace=default
Output to JSON
k8sgpt analyze --explain --filter=Service --output=json
Anonymize during explain
k8sgpt analyze --explain --filter=Service --output=json --anonymize
List filters
k8sgpt filters list
Add default filters
k8sgpt filters add [filter(s)]
k8sgpt filters add Service
k8sgpt filters add Ingress,Pod
Remove default filters
k8sgpt filters remove [filter(s)]
k8sgpt filters remove Service
k8sgpt filters remove Ingress,Pod
List configured backends
k8sgpt auth list
Update configured backends
k8sgpt auth update $MY_BACKEND1,$MY_BACKEND2..
Remove configured backends
k8sgpt auth remove -b $MY_BACKEND1,$MY_BACKEND2..
List integrations
k8sgpt integrations list
Activate integrations
k8sgpt integrations activate [integration(s)]
Use integration
k8sgpt analyze --filter=[integration(s)]
Deactivate integrations
k8sgpt integrations deactivate [integration(s)]
Serve mode
k8sgpt serve
Analysis with serve mode
grpcurl -plaintext -d '{"namespace": "k8sgpt", "explain" : "true"}' localhost:8080 schema.v1.ServerAnalyzerService/Analyze
{
"status": "OK"
}
Analysis with custom headers
k8sgpt analyze --explain --custom-headers CustomHeaderKey:CustomHeaderValue
K8sGPT uses the chosen LLM, generative AI provider when you want to explain the analysis results using --explain flag e.g. k8sgpt analyze --explain
. You can use --backend
flag to specify a configured provider (it's openai
by default).
You can list available providers using k8sgpt auth list
:
Default:
> openai
Active:
Unused:
> openai
> localai
> ollama
> azureopenai
> cohere
> amazonbedrock
> amazonsagemaker
> google
> huggingface
> noopai
> googlevertexai
> ibmwatsonxai
For detailed documentation on how to configure and use each provider see here.
To set a new default provider
k8sgpt auth default -p azureopenai
Default provider set to azureopenai
With this option, the data is anonymized before being sent to the AI Backend. During the analysis execution, k8sgpt
retrieves sensitive data (Kubernetes object names, labels, etc.). This data is masked when sent to the AI backend and replaced by a key that can be used to de-anonymize the data when the solution is returned to the user.
Error: HorizontalPodAutoscaler uses StatefulSet/fake-deployment as ScaleTargetRef which does not exist.
Error: HorizontalPodAutoscaler uses StatefulSet/tGLcCRcHa1Ce5Rs as ScaleTargetRef which does not exist.
The Kubernetes system is trying to scale a StatefulSet named tGLcCRcHa1Ce5Rs using the HorizontalPodAutoscaler, but it cannot find the StatefulSet. The solution is to verify that the StatefulSet name is spelled correctly and exists in the same namespace as the HorizontalPodAutoscaler.
The Kubernetes system is trying to scale a StatefulSet named fake-deployment using the HorizontalPodAutoscaler, but it cannot find the StatefulSet. The solution is to verify that the StatefulSet name is spelled correctly and exists in the same namespace as the HorizontalPodAutoscaler.
Note: Anonymization does not currently apply to events.
Anonymization does not currently apply to events.
In a few analysers like Pod, we feed to the AI backend the event messages which are not known beforehand thus we are not masking them for the time being.
The following is the list of analysers in which data is being masked:-
The following is the list of analysers in which data is not being masked:-
*Note:
k8gpt will not mask the above analysers because they do not send any identifying information except Events analyser.
Masking for Events analyzer is scheduled in the near future as seen in this issue. Further research has to be made to understand the patterns and be able to mask the sensitive parts of an event like pod name, namespace etc.
The following is the list of fields which are not being masked:-
*Note:
k8sgpt
stores config data in the $XDG_CONFIG_HOME/k8sgpt/k8sgpt.yaml
file. The data is stored in plain text, including your OpenAI key.
Config file locations:
OS | Path |
---|---|
MacOS | ~/Library/Application Support/k8sgpt/k8sgpt.yaml |
Linux | ~/.config/k8sgpt/k8sgpt.yaml |
Windows | %LOCALAPPDATA%/k8sgpt/k8sgpt.yaml |
Adding a remote cache
AWS_ACCESS_KEY_ID
and AWS_SECRET_ACCESS_KEY
are required as environmental variables.k8sgpt cache add s3 --region <aws region> --bucket <name>
k8sgpt cache add s3 --bucket <name> --endpoint <http://localhost:9000>
k8sgpt cache add s3 --bucket <name> --endpoint <https://localhost:9000> --insecure
k8sgpt cache add azure --storageacc <storage account name> --container <container name>
GOOGLE_APPLICATION_CREDENTIALS
are required as environmental variables. k8sgpt cache add gcs --region <gcp region> --bucket <name> --projectid <project id>
Listing cache items
k8sgpt cache list
Purging an object from the cache Note: purging an object using this command will delete upstream files, so it requires appropriate permissions.
k8sgpt cache purge $OBJECT_NAME
Removing the remote cache Note: this will not delete the upstream S3 bucket or Azure storage container
k8sgpt cache remove
There may be scenarios where you wish to write your own analyzer in a language of your choice. K8sGPT now supports the ability to do so by abiding by the schema and serving the analyzer for consumption. To do so, define the analyzer within the K8sGPT configuration and it will add it into the scanning process. In addition to this you will need to enable the following flag on analysis:
k8sgpt analyze --custom-analysis
Here is an example local host analyzer in Rust
When this is run on localhost:8080
the K8sGPT config can pick it up with the following additions:
custom_analyzers:
- name: host-analyzer
connection:
url: localhost
port: 8080
This now gives the ability to pass through hostOS information ( from this analyzer example ) to K8sGPT to use as context with normal analysis.
See the docs on how to write a custom analyzer
Listing custom analyzers configured
k8sgpt custom-analyzer list
Adding custom analyzer without install
k8sgpt custom-analyzer add --name my-custom-analyzer --port 8085
Removing custom analyzer
k8sgpt custom-analyzer remove --names "my-custom-analyzer,my-custom-analyzer-2"
Find our official documentation available here
Please read our contributing guide.
Find us on Slack