Enable Top Clients/Files Metrics in Harvest

[rahulguptajss]

Top Clients collection is disabled by default. To enable Top Clients Tracking in Harvest, follow these steps. Note that Top Clients support is available only through the RestPerf Collector in ONTAP version 9.13.1 and later. The collection is enabled in the RestPerf Collector Volume template via a plugin VolumeTopClients, as detailed below.

Note: Available from Harvest 24.11.

Note: Top File Metrics are available in Harvest from 24.11.1 via same plugin. Refer here for more details.

NABox4

⚠️ Note:
Prerequisite: ssh to NABox via admin
Create relevant directories if they don't exist. You need to run these commands in sudo bash

1. Navigate to the restperf directory.

cd /etc/nabox/harvest/user/restperf

Create custom.yaml:

vi custom.yaml

Update the file with the following content:

objects:
  Volume: custom_volume.yaml

2. Create a custom_volume.yaml file in the /etc/nabox/harvest/user/restperf/9.12.0 directory. Update custom_volume.yaml as below.

name:                     Volume
query:                    api/cluster/counter/tables/volume
object:                   volume

counters:
  - ^^uuid
  - ^name                  => volume
  - ^node.name             => node
  - ^parent_aggregate      => aggr
  - ^svm.name              => svm
  - average_latency        => avg_latency
  - bytes_read             => read_data
  - bytes_written          => write_data
  - nfs.access_latency     => nfs_access_latency
  - nfs.access_ops         => nfs_access_ops
  - nfs.getattr_latency    => nfs_getattr_latency
  - nfs.getattr_ops        => nfs_getattr_ops
  - nfs.lookup_latency     => nfs_lookup_latency
  - nfs.lookup_ops         => nfs_lookup_ops
  - nfs.other_latency      => nfs_other_latency
  - nfs.other_ops          => nfs_other_ops
  - nfs.punch_hole_latency => nfs_punch_hole_latency
  - nfs.punch_hole_ops     => nfs_punch_hole_ops
  - nfs.read_latency       => nfs_read_latency
  - nfs.read_ops           => nfs_read_ops
  - nfs.setattr_latency    => nfs_setattr_latency
  - nfs.setattr_ops        => nfs_setattr_ops
  - nfs.total_ops          => nfs_total_ops
  - nfs.write_latency      => nfs_write_latency
  - nfs.write_ops          => nfs_write_ops
  - other_latency
  - read_latency
  - total_ops
  - total_other_ops        => other_ops
  - total_read_ops         => read_ops
  - total_write_ops        => write_ops
  - write_latency

plugins:
  - MetricAgent:
      compute_metric:
        - total_data ADD bytes_read bytes_written
  - Aggregator:
      # plugin will create summary/average for each object
      # any names after the object names will be treated as label names that will be added to instances
      - node
  - Volume:
      include_constituents: false

  - VolumeTopClients:
      # The maximum number of volumes to consider for top client metrics.
      # The actual maximum value is capped at 50, even if a higher number is specified.
      # When enabled, this plugin will collect read/write operations and throughput metrics for the top clients of each volume.
      - schedule:
          - data: 1h  # This value should be a multiple of the poll duration. By default, Harvest will check once an hour to see how many volumes have activity_tracking.state set to on.
      - max_volumes: 5


export_options:
  instance_keys:
    - aggr
    - node
    - style
    - svm
    - volume

3. Restart harvest

rpm/deb/native/container

Note:

Below steps are applicable for rpm/deb/native/container installation. In this example, let's assume that Harvest is installed in /opt/harvest

1. Navigate to the rest directory:

cd /opt/harvest/conf/restperf

Create custom.yaml:

vi custom.yaml

Update the file with the following content:

objects:
  Volume: custom_volume.yaml

2. Create a custom_volume.yaml file in the /opt/harvest/conf/restperf/9.12.0/ directory. Update custom_volume.yaml as below.

name:                     Volume
query:                    api/cluster/counter/tables/volume
object:                   volume

counters:
  - ^^uuid
  - ^name                  => volume
  - ^node.name             => node
  - ^parent_aggregate      => aggr
  - ^svm.name              => svm
  - average_latency        => avg_latency
  - bytes_read             => read_data
  - bytes_written          => write_data
  - nfs.access_latency     => nfs_access_latency
  - nfs.access_ops         => nfs_access_ops
  - nfs.getattr_latency    => nfs_getattr_latency
  - nfs.getattr_ops        => nfs_getattr_ops
  - nfs.lookup_latency     => nfs_lookup_latency
  - nfs.lookup_ops         => nfs_lookup_ops
  - nfs.other_latency      => nfs_other_latency
  - nfs.other_ops          => nfs_other_ops
  - nfs.punch_hole_latency => nfs_punch_hole_latency
  - nfs.punch_hole_ops     => nfs_punch_hole_ops
  - nfs.read_latency       => nfs_read_latency
  - nfs.read_ops           => nfs_read_ops
  - nfs.setattr_latency    => nfs_setattr_latency
  - nfs.setattr_ops        => nfs_setattr_ops
  - nfs.total_ops          => nfs_total_ops
  - nfs.write_latency      => nfs_write_latency
  - nfs.write_ops          => nfs_write_ops
  - other_latency
  - read_latency
  - total_ops
  - total_other_ops        => other_ops
  - total_read_ops         => read_ops
  - total_write_ops        => write_ops
  - write_latency

plugins:
  - MetricAgent:
      compute_metric:
        - total_data ADD bytes_read bytes_written
  - Aggregator:
      # plugin will create summary/average for each object
      # any names after the object names will be treated as label names that will be added to instances
      - node
  - Volume:
      include_constituents: false

  - VolumeTopClients:
      # The maximum number of volumes to consider for top client metrics.
      # The actual maximum value is capped at 50, even if a higher number is specified.
      # When enabled, this plugin will collect read/write operations and throughput metrics for the top clients of each volume.
      - schedule:
          - data: 1h  # This value should be a multiple of the poll duration. By default, Harvest will check once an hour to see how many volumes have activity_tracking.state set to on.
      - max_volumes: 5


export_options:
  instance_keys:
    - aggr
    - node
    - style
    - svm
    - volume

5. Restart harvest

In the volume dashboard under the clients panel, you should see something similar to the following:

[mamoep]

How are the volumes chosen? Let's say I have 20 volumes and set a limit of 5. Which volumes will be reported?

[rahulguptajss]

For example, for read IOPS, we will select the top 5 volumes based on descending order of read IOPS in each performance poll. The same approach applies to write IOPS, as well as read and write throughput.

[WayneShen2]

I have enable it, but I can not see any data. Have any configure must enable from Filer end (ex: Activity Tracking)

[rahulguptajss]

I have enable it, but I can not see any data. Have any configure must enable from Filer end (ex: Activity Tracking)

Yes activity tracking must be enabled for relevant volumes.

[WayneShen2]

Thanks for your confirmed.

[wooyoungAhn]

I can not see any data..

time=2024-11-21T05:33:51.153Z level=ERROR source=collector.go:483 msg="" Poller=CVO collector=RestPerf:Volume error="failed to fetchAll href=api/storage/volumes/*/top-metrics/clients..., hrefLength=168 err=error making request StatusCode: 400, Message: Volume wildcard queries are not supported for activity tracking reports., Code: 124518417, API: /api/storage/volumes/*/top-metrics/clients?fields=client_ip%2Ciops.write%2Csvm%2Cvolume.name&max_records=500&return_records=true&svm=svm_CVO1&top_metric=iops.write&volume=vol2" plugin=VolumeTopClients

[rahulguptajss]

Could you share ONTAP version?

[rahulguptajss]

I have updated the instructions to specify that ONTAP version 9.13.1 or higher is required for TopClients to function. This is because the wildcard queries used in TopClients are only supported starting from version 9.13.1 onwards.