Skip to content

Commit 296a11d

Browse files
authored
Add memory info to docs. (#178)
* Add memory info to docs. Signed-off-by: Hannah Troisi <htroisi@pixielabs.ai> * tweak Signed-off-by: Hannah Troisi <htroisi@pixielabs.ai> * Address nserrino's comments. Signed-off-by: Hannah Troisi <htroisi@pixielabs.ai> * Add a page Signed-off-by: Hannah Troisi <htroisi@pixielabs.ai> * tweak Signed-off-by: Hannah Troisi <htroisi@pixielabs.ai> * Address nserrino's comments. Signed-off-by: Hannah Troisi <htroisi@pixielabs.ai> * address nserrino's comment Signed-off-by: Hannah Troisi <htroisi@pixielabs.ai>
1 parent be080f3 commit 296a11d

11 files changed

Lines changed: 208 additions & 32 deletions

File tree

content/en/01-about-pixie/05-faq.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -68,11 +68,11 @@ Yes. Pixie Community Cloud is a hosted version of Pixie. Pixie stores all data o
6868

6969
Pixie stores the data it collects in-memory on the nodes in your cluster; no data is sent to a centralized backend outside of the cluster. This is true for both self-hosted Pixie and [Pixie Community Cloud](#general-does-pixie-offer-a-hosted-cloud-offering).
7070

71-
Pixie has a [1GiB memory requirement](/installing-pixie/requirements/#memory) per node. After installing Pixie, it is normal to see a temporary increase in memory usage of the `vizier-pem` pods as they begin to fill their data tables.
71+
Pixie has a [1GiB memory requirement](/installing-pixie/requirements/#memory) per node. After installing Pixie, it is normal to see a temporary increase in memory usage of the `vizier-pem` pods as they begin to fill their [data tables](/reference/datatables/).
7272

7373
### How much data does Pixie store?
7474

75-
Retention time depends on the level of traffic in your cluster, but will generally be on the order of hours. Each data table in Pixie (e.g. `http_events`) has its own maximum size. These tables collect data until the maximum size is reached, and then begin dropping the oldest data.
75+
Retention time depends on the level of traffic in your cluster, but will generally be on the order of hours. Each [data table](/reference/datatables/) in Pixie (e.g. `http_events`) has its own maximum size. These tables collect data until the maximum size is reached, and then begin dropping the oldest data.
7676

7777
Pixie's [Plugin System](/reference/plugins/plugin-system) integrates with other tools to for long-term retention. Other third-party observability tools (such as [New Relic’s integration](https://newrelic.com/platform/kubernetes-pixie)) may also provide this capability.
7878

content/en/01-about-pixie/06-troubleshooting.md

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ This page describes how to troubleshoot Pixie. We frequently answer questions on
2222
- [Why can’t I see data?](#troubleshooting-operation-why-can't-i-see-data)
2323
- [Why can’t I see application profiles / flamegraphs for my pod / node?](#troubleshooting-operation-why-can't-i-see-application-profiles-flamegraphs-for-my-pod-node)
2424
- [Why is the vizier-pem pod’s memory increasing?](#troubleshooting-operation-why-is-the-vizier-pem-pod's-memory-increasing)
25+
- [Why is Pixie being OOMKilled?](#troubleshooting-operation-why-is-pixie-being-oomkilled)
2526
- [Troubleshooting tracepoint scripts.](#troubleshooting-operation-troubleshooting-pixie-tracepoint-scripts)
2627

2728
## Troubleshooting Deployment
@@ -113,7 +114,13 @@ Pixie's [continuous profiler](/tutorials/pixie-101/profiler/) currently supports
113114

114115
This is expected behavior. Pixie stores the data it collects in-memory on the nodes in your cluster; data is not sent to any centralized backend cloud outside of the cluster. So what you are observing is simply the data that it is collecting.
115116

116-
Pixie has a minimum 1GiB memory requirement per node. The default deployment is 2GiB of memory. This limit can be configured with the `--pem_memory_limit` flag when deploying Pixie. Using a value less than 1GiB is not currently recommended.
117+
When Pixie is first deployed, you can expect to see an increase in memory utilization over time. This is because the temporary data store will go from storing 0B of data to reaching its cap of 1.25G of data (in the default deployment). Once this cap is hit, old data is expired and the memory utilization shouldn’t increase anymore.
118+
119+
Pixie has a minimum 1GiB memory requirement per node. The default deployment is 2GiB of memory. To learn how to configure Pixie's memory usage, see the [Tuning Memory Usage](/reference/admin/tuning-mem-usage/) page.
120+
121+
### Why is Pixie being OOMKilled?
122+
123+
This happens when Pixie is using more memory than the limit. Try [increasing the limit](/reference/admin/deploy-options/#configuring-pixie's-memory-usage-setting-the-memory-limit), or [reducing the amount of data stored](/reference/admin/deploy-options/#configuring-pixie's-memory-usage-setting-the-data-table-storage-memory-limit) in the table store. The amount of memory needed for Pixie depends on the application, but a minimum of [1Gi](/installing-pixie/requirements/#memory) is required. However, for most clusters we recommend 2Gi. For more information, see the [Tuning Memory Usage](/reference/admin/tuning-mem-usage/) page.
117124

118125
### Troubleshooting Pixie tracepoint scripts
119126

content/en/02-installing-pixie/01-requirements.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -45,11 +45,13 @@ For local development, we recommend using Minikube with a VM driver (`kvm2` on L
4545

4646
## Memory
4747

48-
Memory requirements for your cluster nodes are as follows:
48+
Pixie requires the following memory per node:
4949

5050
| Minimum | Notes |
5151
| :-------- | :--------------------------------------------------------------------- |
52-
| 1GiB | To accommodate application pods, 4GiB+ total per node is recommended. |
52+
| 1GiB | To accommodate application pods, we recommend using no more than 25% of the nodes' total memory for Pixie. |
53+
54+
Pixie deploys its PEMs as a DaemonSet on your cluster in order to collect and store telemetry data. The default memory limit is 2Gi per PEM. The lowest recommended value is 1Gi per PEM. For more information on how to configure Pixie's memory usage, see the [Tuning Memory Usage](/reference/admin/tuning-mem-usage/) page.
5355

5456
## CPU
5557

content/en/05-reference/01-admin/04-deploy-options.md

Lines changed: 120 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -7,52 +7,150 @@ redirect_from:
77
- /admin/deploy-options/
88
---
99

10-
## Labeling Pixie's Resources
10+
Pixie offers the following deploy options:
1111

12-
When deploying Pixie using the CLI, you have the option of adding one or more custom [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) to the K8s objects deployed by Pixie.
12+
- [Annotate Pixie's resources](#annotating-pixie's-resources)
13+
- [Label Pixie's resources](#labeling-pixie's-resources)
14+
- [Provide a custom cluster name](#providing-a-custom-cluster-name)
15+
- [Configure Pixie's memory usage](#configuring-pixie's-memory-usage)
16+
- [Set the data access mode](#setting-the-data-access-mode)
1317

14-
``` bash
15-
# Deploy the Pixie Platform in your K8s cluster and give the deployed objects label(s)
16-
px deploy --labels=key1=value1,key2=value2
18+
To see the full set of deploy options, install the [Pixie CLI](/installing-pixie/install-schemes/cli/) and run `px deploy --help`.
19+
20+
## Annotating Pixie's resources
21+
22+
When deploying Pixie, you have the option of adding one or more custom [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) to the Kubernetes objects deployed by Pixie.
23+
24+
To annotate Pixie's resources when deploying with the [Pixie CLI](/installing-pixie/install-schemes/cli/), use the `--annotations` flag:
25+
26+
```bash
27+
px deploy --annotations=key1=value1,key2=value2
28+
```
1729

18-
# Example deployment with labeling:
19-
px deploy --labels=application=pixie
30+
To annotate Pixie's resources when deploying with [Helm](/installing-pixie/install-schemes/helm/), use the `annotations` field:
31+
32+
```bash
33+
helm install pixie pixie-operator/pixie-operator-chart --set deployKey=<deploy-key-goes-here> --namespace pl --create-namespace --set annotations=key1=value1,key2=value2
2034
```
2135

22-
Certain labels are reserved for internal use by Pixie. The following are reserved label strings that may _**not be**_ used:
36+
## Labeling Pixie's resources
37+
38+
When deploying Pixie, you have the option of adding one or more custom [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) to the Kubernetes objects deployed by Pixie. Certain labels are reserved for internal use by Pixie. The following are reserved label strings that may _**not be**_ used:
2339

2440
- `"vizier-bootstrap"`
2541
- `“component"`
2642
- `“vizier-updater-dep"`
2743
- `“app"`
2844

29-
## PEM Memory Usage
45+
To label Pixie's resources when deploying with the [Pixie CLI](/installing-pixie/install-schemes/cli/), use the `--labels` flag:
3046

31-
Pixie deploys its PEMs as a DaemonSet on your cluster in order to collect and store telemetry data. The default memory limit is 2Gi per PEM. The lowest recommended value is [1Gi](/installing-pixie/requirements/#memory) per PEM. You have the option to configure this value via the `--pem_memory_limit` flag in the CLI/YAML (`--set pemMemoryLimit=<limit>` in Helm).
47+
``` bash
48+
px deploy --labels=key1=value1,key2=value2
49+
```
3250

33-
## Data Access
51+
To label Pixie's resources when deploying with [Helm](/installing-pixie/install-schemes/helm/), use the `labels` field:
3452

35-
You can configure Pixie's data access mode in order to control what data can be displayed when executing a script.
36-
Pixie offers two modes:
53+
```bash
54+
helm install pixie pixie-operator/pixie-operator-chart --set deployKey=<deploy-key-goes-here> --namespace pl --create-namespace --set labels=key1=value1,key2=value2
55+
```
3756

38-
1. `Full` (default): The user has full data access, and Pixie does not redact any collected data from the user during script execution.
39-
2. `Restricted`: The user has restricted data access. Pixie will redact all columns that may potentially contain sensitive data (e.g. request/response bodies and headers). All rows in the column will be redacted, regardless of whether they do or do not actually contain PII. Metadata about these columns, such as length, will still be queryable.
57+
## Providing a custom cluster name
4058

41-
### CLI/YAML Install
59+
When deploying Pixie, you have the option of providing a custom cluster name. If you do not provide a cluster name, the name will be taken from the current kubeconfig.
4260

43-
You may specify the desired data access mode using the `--data_access` flag. If no mode is specified, the deploy assumes `Full` as default.
61+
To provide a custom cluster name when deploying with the [Pixie CLI](/installing-pixie/install-schemes/cli/), use the `--cluster_name` flag:
4462

63+
```bash
64+
# Deploy Pixie in your K8s cluster and use the provided name for the cluster.
65+
px deploy --cluster_name=<CLUSTER_NAME>
4566
```
46-
px deploy --data_access=(Full|Restricted)
67+
68+
To provide a custom cluster name when deploying with [Helm](/installing-pixie/install-schemes/helm/), use the `clusterName` field:
69+
70+
```bash
71+
helm install pixie pixie-operator/pixie-operator-chart --set deployKey=<deploy-key-goes-here> --set clusterName=<CLUSTER_NAME> --namespace pl --create-namespace
4772
```
4873

49-
### Helm Install
74+
## Configuring Pixie's memory usage
75+
76+
When deploying Pixie, you have three options for configuring PEM memory usage:
77+
78+
- [Set PEM pod memory limit](#configuring-pixie's-memory-usage-setting-the-memory-limit)
79+
- [Set PEM pod memory request](#configuring-pixie's-memory-usage-setting-the-memory-request)
80+
- [Set data table storage limit](#configuring-pixie's-memory-usage-setting-the-data-table-storage-memory-limit)
81+
82+
For more information about these options, please refer to the [Tuning Memory Usage](/reference/admin/tuning-mem-usage/) page.
83+
84+
### Setting the memory limit
85+
86+
The default memory limit is 2Gi per PEM. The lowest recommended value is [1Gi](/installing-pixie/requirements/#memory) per PEM. 1Gi is not a suitable limit for a cluster with high throughput, but it is suitable for a small cluster with limited resources.
5087

51-
You may specify the desired data access mode using the `dataAccess` field.
88+
To set Pixie's memory limit when deploying with the [Pixie CLI](/installing-pixie/install-schemes/cli/), use the `--pem_memory_limit` flag:
5289

90+
``` bash
91+
px deploy --pem_memory_limit=1Gi
92+
```
93+
94+
To set Pixie's memory limit when deploying with [Helm](/installing-pixie/install-schemes/helm/), use the `pemMemoryLimit` field:
95+
96+
```bash
97+
helm install pixie pixie-operator/pixie-operator-chart --set deployKey=<deploy-key-goes-here> --namespace pl --create-namespace --set pemMemoryLimit=1Gi
5398
```
54-
helm install pixie pixie-operator/pixie-operator-chart --set deployKey=<deploy-key-goes-here> --set clusterName=<cluster-name> --namespace pl --create-namespace
55-
--dataAccess=(Full|Restricted)
99+
100+
### Setting the memory request
101+
102+
By default, the PEM's memory request will be the same as the limit. When deploying Pixie, you can specify a different memory [request](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limits) than limit. This flag is useful for clusters in which the PEM pods are pending because there is not enough memory for them to be scheduled on the node.
103+
104+
<Alert variant="outlined" severity="info">Please note that PEM memory request must be less than or equal to the PEM memory limit.</Alert>
105+
106+
To set Pixie's memory request when deploying with the [Pixie CLI](/installing-pixie/install-schemes/cli/), use the `--pem_memory_request` flag:
107+
108+
``` bash
109+
px deploy --pem_memory_request=1Gi
110+
```
111+
112+
To set Pixie's memory request when deploying with [Helm](/installing-pixie/install-schemes/helm/), use the `pemMemoryRequest` field:
113+
114+
```bash
115+
helm install pixie pixie-operator/pixie-operator-chart --set deployKey=<deploy-key-goes-here> --namespace pl --create-namespace --set pemMemoryRequest=1Gi
116+
```
117+
118+
### Setting the data table storage memory limit
119+
120+
This is an advanced option that most developers shouldn't need. For more discussion see the [Tuning Memory Usage](/reference/admin/tuning-mem-usage/) page.
121+
122+
To set Pixie's data store memory limit when deploying with the [Pixie CLI](/installing-pixie/install-schemes/cli/), use the `--pem_flags` flag:
123+
124+
``` bash
125+
px deploy --pem_flags="PL_TABLE_STORE_DATA_LIMIT_MB=1000"
126+
```
127+
128+
To set Pixie's data store memory limit when deploying with [Helm](/installing-pixie/install-schemes/helm/), use the `dataCollectorParams.customPEMFlags.PL_TABLE_STORE_DATA_LIMIT_MB` field:
129+
130+
```bash
131+
helm install pixie pixie-operator/pixie-operator-chart --set deployKey=<deploy-key-goes-here> --namespace pl --create-namespace --set dataCollectorParams.customPEMFlags.PL_TABLE_STORE_DATA_LIMIT_MB=750
132+
```
133+
134+
## Setting the data access mode
135+
136+
When deploying Pixie, you have the option of setting the data access mode. This mode controls what data can be displayed when executing a [PxL script](/using-pixie) to query Pixie platform telemetry data. Pixie offers two data access modes:
137+
138+
- `Full` (default): The user has full data access, and Pixie does not redact any collected data from the user during script execution.
139+
140+
- `Restricted`: The user has restricted data access. Pixie will redact all columns that may potentially contain sensitive data (e.g. request/response bodies and headers). All rows in the column will be redacted, regardless of whether they do or do not actually contain PII. Metadata about these columns, such as length, will still be queryable.
141+
142+
If no data access mode is specified when deploying Pixie, the deploy assumes `Full` as default.
143+
144+
To set the data access mode when deploying with the [Pixie CLI](/installing-pixie/install-schemes/cli/), use the `--data_access` flag:
145+
146+
```bash
147+
px deploy --data_access=(Full|Restricted)
148+
```
149+
150+
To set the data access mode when deploying with [Helm](/installing-pixie/install-schemes/helm/), use the `dataAccess` field:
151+
152+
```bash
153+
helm install pixie pixie-operator/pixie-operator-chart --set deployKey=<deploy-key-goes-here> --namespace pl --create-namespace --set dataAccess=(Full|Restricted)
56154
```
57155

58156
You may also directly update the `dataAccess` field in your `values.yaml` file.

0 commit comments

Comments
 (0)