Portworx Cheat Sheet

Options can be replaced with short aliases as follows

cluster = c
inspect = i
token = t

Print the cluster provision status

$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl cluster provision-status
ansible@k8smaster1:~$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl cluster provision-status
Defaulted container "portworx" out of: portworx, csi-node-driver-registrar
NODE ID         IP    HOSTNAME  NODE STATUS POOL            POOL STATUS IO_PRIORITY SIZE  AVAILABLE USED  PROVISIONED ZONE  REGION  RACK
da51a997-cdb9-4d37-8013-65e1c0170df1  192.168.0.23  k8sworker3  Up    0 ( d50ee01b-bc4d-40cc-8957-a9594126bf96 )  Online    HIGH    200 GiB 188 GiB   12 GiB  0 B   default default default
da51a997-cdb9-4d37-8013-65e1c0170df1  192.168.0.23  k8sworker3  Up    1 ( 3bcc57d8-f991-4b4f-b1e4-e3ea0c928fe0 )  Online    HIGH    50 GiB  44 GiB    6.0 GiB 0 B   default default default
da51a997-cdb9-4d37-8013-65e1c0170df1  192.168.0.23  k8sworker3  Up    2 ( 9c7a5e27-99c1-4a2c-9589-7508556d92ae )  Online    HIGH    500 GiB 488 GiB   12 GiB  0 B   default default default
65246566-4b8c-457b-967b-13e41207a23e  192.168.0.22  k8sworker2  Up    0 ( 4b0ccdfe-c30d-425c-a40b-a8aa8aa02396 )  Online    HIGH    200 GiB 188 GiB   12 GiB  0 B   default default default
65246566-4b8c-457b-967b-13e41207a23e  192.168.0.22  k8sworker2  Up    1 ( 9fe5f06b-9b03-4d92-9c30-347b895f9924 )  Online    HIGH    50 GiB  44 GiB    6.0 GiB 0 B   default default default
65246566-4b8c-457b-967b-13e41207a23e  192.168.0.22  k8sworker2  Up    2 ( ccb42e0e-2516-494d-b166-cf4d445f8bfa )  Online    HIGH    500 GiB 488 GiB   12 GiB  0 B   default default default
d189d695-f6aa-46fb-aeba-9ecad3846394  192.168.0.21  k8sworker1  Up    0 ( 24e01ec9-6859-469b-9f36-cd45bcc01da8 )  Online    HIGH    200 GiB 188 GiB   12 GiB  0 B   default default default
d189d695-f6aa-46fb-aeba-9ecad3846394  192.168.0.21  k8sworker1  Up    1 ( f4ffaad1-0ab4-4837-a7c0-64c8b7a7735f )  Online    HIGH    50 GiB  44 GiB    6.0 GiB 0 B   default default default
d189d695-f6aa-46fb-aeba-9ecad3846394  192.168.0.21  k8sworker1  Up    2 ( 70c3de72-7054-4512-8cca-98c8f5e6641d )  Online    HIGH    500 GiB 488 GiB   12 GiB  0 B   default default default
ansible@k8smaster1:~$
$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl status
ansible@k8smaster1:~$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl status
Defaulted container "portworx" out of: portworx, csi-node-driver-registrar
Status: PX is operational
Telemetry: Healthy
Metering: Disabled or Unhealthy
License: Trial (expires in 31 days)
Node ID: da51a997-cdb9-4d37-8013-65e1c0170df1
  IP: 192.168.0.23 
  Local Storage Pool: 3 pools
  POOL  IO_PRIORITY RAID_LEVEL  USABLE  USED  STATUS  ZONE  REGION
  0 HIGH    raid0   200 GiB 12 GiB  Online  default default
  1 HIGH    raid0   50 GiB  6.0 GiB Online  default default
  2 HIGH    raid0   500 GiB 12 GiB  Online  default default
  Local Storage Devices: 3 devices
  Device  Path    Media Type    Size    Last-Scan
  0:1 /dev/sdb  STORAGE_MEDIUM_MAGNETIC 200 GiB   30 Aug 23 13:45 UTC
  1:1 /dev/sdc  STORAGE_MEDIUM_MAGNETIC 50 GiB    30 Aug 23 13:45 UTC
  2:1 /dev/sdd  STORAGE_MEDIUM_MAGNETIC 500 GiB   30 Aug 23 13:45 UTC
  * Internal kvdb on this node is sharing this storage device /dev/sdd  to store its data.
  total   - 750 GiB
  Cache Devices:
   * No cache devices
Cluster Summary
  Cluster ID: px-cluster-6da515d8-8e73-4196-b743-bb005efc355b
  Cluster UUID: 81a15216-d17a-417c-ab79-b252605815e0
  Scheduler: kubernetes
  Total Nodes: 3 node(s) with storage (3 online)
  IP    ID          SchedulerNodeName Auth    StorageNode Used  Capacity  Status  StorageStatus Version   Kernel      OS
  192.168.0.23  da51a997-cdb9-4d37-8013-65e1c0170df1  k8sworker3    Disabled  Yes   30 GiB  750 GiB   Online  Up (This node)  3.0.0.0-038005f 5.15.0-79-generic Ubuntu 22.04.3 LTS
  192.168.0.21  d189d695-f6aa-46fb-aeba-9ecad3846394  k8sworker1    Disabled  Yes   30 GiB  750 GiB   Online  Up    3.0.0.0-038005f 5.15.0-79-generic Ubuntu 22.04.3 LTS
  192.168.0.22  65246566-4b8c-457b-967b-13e41207a23e  k8sworker2    Disabled  Yes   30 GiB  750 GiB   Online  Up    3.0.0.0-038005f 5.15.0-79-generic Ubuntu 22.04.3 LTS
  Warnings: 
     WARNING: Internal Kvdb is not using dedicated drive on nodes [192.168.0.23]. This configuration is not recommended for production clusters.
Global Storage Pool
  Total Used      :  91 GiB
  Total Capacity  :  2.2 TiB
ansible@k8smaster1:~$

To list the cluster nodes.

$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl cluster list
ansible@k8smaster1:~$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl cluster list
Defaulted container "portworx" out of: portworx, csi-node-driver-registrar
Cluster ID: px-cluster-6da515d8-8e73-4196-b743-bb005efc355b
Cluster UUID: 81a15216-d17a-417c-ab79-b252605815e0
Status: OK

Nodes in the cluster:
ID					SCHEDULER_NODE_NAME	DATA IP		CPU		MEM TOTAL	MEM FREE	CONTAINERS	VERSION			Kernel			OS			STATUS
da51a997-cdb9-4d37-8013-65e1c0170df1	k8sworker3		192.168.0.23	3.784693	8.3 GB		7.0 GB		N/A		3.0.0.0-038005f		5.15.0-79-generic	Ubuntu 22.04.3 LTS	Online
d189d695-f6aa-46fb-aeba-9ecad3846394	k8sworker1		192.168.0.21	2.709568	8.3 GB		7.3 GB		N/A		3.0.0.0-038005f		5.15.0-79-generic	Ubuntu 22.04.3 LTS	Online
65246566-4b8c-457b-967b-13e41207a23e	k8sworker2		192.168.0.22	3.709949	8.3 GB		7.1 GB		N/A		3.0.0.0-038005f		5.15.0-79-generic	Ubuntu 22.04.3 LTS	Online

ansible@k8smaster1:~$ 

To Inspect a single node using node ID.

$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl cluster inspect 65246566-4b8c-457b-967b-13e41207a23e
ansible@k8smaster1:~$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl cluster inspect 65246566-4b8c-457b-967b-13e41207a23e
Defaulted container "portworx" out of: portworx, csi-node-driver-registrar
ID            		:  65246566-4b8c-457b-967b-13e41207a23e
Scheduler Node Name	:  k8sworker2
Mgmt IP       		:  192.168.0.22
Data IP       		:  192.168.0.22
CPU           		:  4.861693210393965
Mem Total     		:  8324665344
Mem Used      		:  1202769920
Volumes 		:

Status  		:  Online

ansible@k8smaster1:~$

To list and reset the token

ansible@k8smaster1:~$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl cluster token show
Defaulted container "portworx" out of: portworx, csi-node-driver-registrar
Token is 3395859c0b19094da309e880468daa52e774068763899b6896fca95658657afa0768c453e9027370e88f05911690bc2a74e25c3d844076763e45355b69f52abd
ansible@k8smaster1:~$
$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl cluster token reset

List and update cluster wide options

ansible@k8smaster1:~$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl cluster options list
Defaulted container "portworx" out of: portworx, csi-node-driver-registrar
Default RPC timeout (minutes)                           : 5
Auto decommission timeout (minutes)                     : 20
Replica move timeout (minutes)                          : 1440
Replica move timestamp records threshold                : 134217728
Internal Snapshot Interval (minutes)                    : 30
Re-add timeout (minutes)                                : 1440
Resync repl-add                                         : off
Domain policy                                           : eventual
Metro DR Domain protection                              : on
Cloudsnap optimized Restores                            : off
Cloudsnap Catalog                                       : off
Cloudsnap Abort Timeout (minutes)                       : 10
Cloudsnap Error Retry Limit                             : 3
Cloudsnap minimum Periodic schedule Interval (minutes)  : 15
Cloudsnap maximum threads                               : 16
Cloudsnap network interface                             : 
Cloudsnap Using Metadata Enabled                        : on
Cloudsnap Metadata Upload Bytes limit                   : 10240 MiB
Cloudsnap Metadata Upload Percent limit                 : 15 %
Cloudsnaps Cleanup Failed Delay                         : 0
Cloudsnap full backup frequency                         : 7
Disable Provisioning Labels                             : 
License expiry check (days)                             : 7
License expiry check interval                           : 6h0m5s
PxHttpProxy                                             : 
Temporary Kvdb loss support                             : on
Unique Blocks size of volumes check interval            : 12h0m0s
The periodicity of dumping node stats to disk in seconds : 120
Total number of node stats dump files to maintain at any given time : 10
Maximum concurrent API invocations                      : 20
Number of sharedv4 threads                              : 128
Sharedv4 (NFS) mount timeout in seconds                 : 130
Runtime options                                         : 
ScheduleSnapshots Detached Volume Action                : optimized
Cache flush                                             : disabled
Cache flush interval in seconds                         : 30
Lttng last command executed                             : 
Lttng disk usage setting (GB)                           : 0
Lttng loglevel setting                                  : 
SkinnySnap                                              : off
SkinnySnap replication factor                           : 1
Cloudsnap cluster network usage limit                   : Not configured
RelaxedReclaim                                          : off
RelaxedReclaim maximum pending                          : 256
Volume expiration minutes                               : 0s
Auto Fstrim                                             : off
Max Fstrim IO rate                                      : 32 MiB
Min Fstrim IO rate                                      : 1 MiB
Max replicas per Node                                   : 4294967295
Pool Cache Configuration                                : off
Incremental Journal Uploads Disabled                    : false
Incremental Journal Uploads interval (minutes)          : 30
Snapshot creation timeout in Minutes                    : 20
Kvdb defrag frequency in hours                          : 336
Kvdb min DB size to trigger a defrag (MiB)              : 100
Default IO Profile                                      : auto
Detect RO volumes to bounce interval (seconds)          : 30
Type of RO volumes that bounce pods                     : fada
FlashArray iSCSI Allowed IFaces                         : 
ansible@k8smaster1:~$ 

Number of days to alert before license expires, Set zero to disable the alerts. Default alert value is 7.

$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl cluster options update --license-expiry-check 10

Maximum number of replicas support per node, default value is 1024

$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl cluster options update --max-replicas-per-node 5

Default IO-Profile for volumes, accepted values are none and auto. Default value none.

When we set auto, causes repl 2 and repl 3 volumes to get db_remote IO profile optimization.

$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl cluster options update --default-io-profile none
$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl cluster options update --default-io-profile auto

More info at

https://docs.portworx.com/portworx-enterprise/concepts/io-profiles

For all available options.

$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl cluster options update --help

Pair this cluster with another Portworx cluster

$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl cluster pair list

Enable DR mode for the cluster pair

$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl cluster pair create --dr-mode

Licence management

List the license

$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl license list

Add a license from a file.

$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl license add license-file.bin

Creating, listing and deleting schedule policy

$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl sched-policy create --daily 18:00 daily_at_6_pm
$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl sched-policy create --weekly friday@18:30 weekend_snapshot
$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl sched-policy delete weekend_snapshot
$ kubectl exec -n portworx px-cluster-6da515d8-8e73-4196-b743-bb005efc355b-dgn9t -- /opt/pwx/bin/pxctl sched-policy list

Manage Portworx Volumes