Curious - Is there a plan to add "qtree show" metrics?

[jmg011]

Hi! Currently we see tree quotas from "qtree report" in qtree* metrics. I'm curious if there is a plan to add other qtree metrics from "qtree show" command output.

Is it something I can build with templates without having to update any plugins for qtree?

[cgrinds]

hi @jmg011 it will probably require plugin changes because these objects often require correlating multiple ZAPI/REST calls since a single API doesn't exist with all the information.

But first, let's talk details. When I run qtree show I see

umeng-aff300-05-06::> qtree show
Vserver    Volume        Qtree        Style        Oplocks   Status
---------- ------------- ------------ ------------ --------- --------
astra_devops_vs1
           astra_devops_vs1_root
                         ""           mixed        enable    normal
astra_devops_vs1
           temp1         ""           mixed        enable    normal
astra_devops_vs1
           trident_pvc_05941e9a_4b6c_430e_9f16_a322f9c8dc54
                         ""           unix         enable    normal

Can you outline which metrics from this output you want included?

[jmg011]

@cgrinds Thanks for a super quick response on this.

Below metadata, all fields as labels and status as a metric would be helpful if included in the plugin. That way, we have all the metrics for qtrees.

vserver
volume
qtree
qtree-path
security-style
user
group
unix-permissions
status
export-policy
is-export-policy-inherited
qos-policy-group

[cgrinds]

makes sense, we'll take a look

[Hardikl]

@jmg011

I have evaluated the above given field list (from qtree show cli).

Out of the given field list,
ZAPI don't have group, user, qtree-path fields in qtree-list-iter response, whereas
REST have all the fields via public(/api/storage/qtrees) and private(/api/private/cli/qtree) apis.

---

Collection via REST way

Steps to monitor all the mentioned fields via REST are as below.
Please note, you'll need to enable Rest Collector for the poller in harvest configuration and disable(commented) qtree object in ZAPI here.

1. Create a new file custom.yaml here with below contents

objects:
  Qtree:                       custom_qtree.yaml
  

2. Create a new file custom_qtree.yaml here

3. update custom_qtree.yaml template as below.

name:                       Qtree
query:                      api/storage/qtrees
object:                     qtree

counters:
  - ^^svm.name            => svm
  - ^^volume.name         => volume
  - ^^name                => qtree
  - ^export_policy.name   => export_policy
  - ^security_style       => security_style
  - ^unix_permissions     => unix_permissions
  - ^qos_policy.name      => qos_policy_group
  - id                    => id
  - filter:
      - name=!""


endpoints:
  - query: api/private/cli/qtree
    counters:
      - ^^vserver         => svm
      - ^^volume          => volume
      - ^^qtree           => qtree
      - ^status           => status
      - ^oplock_mode      => oplocks
      - ^qtree_path       => qtree_path
      - ^user             => user_name
      - ^group            => group_name
      - ^is_export_policy_inherited => is_export_policy_inherited
      - filter:
          - qtree=!""
plugins:
  - Qtree
  - LabelAgent:
      replace:
        - oplocks oplocks `enable` `enabled`

export_options:
  instance_keys:
    - qtree
    - volume
    - svm
  instance_labels:
    - export_policy
    - security_style
    - status
    - oplocks
    - qtree_path
    - user_id
    - user_name
    - group_id
    - group_name
    - unix_permissions
    - qos_policy_group
    - is_export_policy_inherited

4. Restart harvest.

5. Below qtree metrics should be available in Prometheus.

6. Regarding status as metric, This can be doable by this way example

- LabelAgent:
      value_to_num:
        - new_status status normal normal `0`

Just for reference, status value could be only "normal" or "readonly".

Note: In Harvest upgrade, REST collector would be disabled by default. You would need to enable again after upgrade to collect the qtree metrics via REST.

---

Collection via ZAPI way

Steps to monitor available fields via ZAPI are as below.

1. Create a new file custom.yaml here with below contents

objects:
  Qtree:                       custom_qtree.yaml
  

2. Create a new file custom_qtree.yaml here

3. update custom_qtree.yaml template as below.

counters:
  qtree-info:
    - ^mode                        => unix_permissions
    - ^is-export-policy-inherited  => is_export_policy_inherited
    - ^qos-policy-group            => qos_policy_group

export_options:
  instance_labels:
    - unix_permissions
    - is_export_policy_inherited
    - qos_policy_group
   

4. Restart harvest.

5. Below qtree metrics should be available in prometheus.

6. Regarding status as metric, This can be doable by this way example

- LabelAgent:
      value_to_num:
        - new_status status normal normal `0`

Just for reference, status value could be only "normal" or "readonly".

[jmg011]

@Hardikl - This is perfect. Let me try this in my dev env and get back if I notice any issues. We are still using ZAPI for all collections, so adding REST needs to be tested. I will test it as I have already started testing REST API because we need to move to it at some point.

Do you know when ZAPI support is ending?

Are there any known issues for ZAPI between different ONTAP versions?

[ruanruijuan]

@jmg011 please refer to https://mysupport.netapp.com/info/communications/ECMLP2880232.html.
This bulletin announces that NetApp ONTAPI (ZAPI) will reach end of availability (EOA) in January 2023. For customers using ONTAPI to automate ONTAP data storage management tasks, ONTAP 9.12.1 software—which is expected to release in the fourth quarter (Q4) of calendar year 2022—will be the final version to support ONTAPI.

ZAPIs are mostly backward compatible which very few difference between ONTAP versions. New ZAPIs might be added in newer versions of ONTAP, but older ZAPIs continue to work. Between ZAPI and REST, there are very known issues. It's not 100% 1:1 mapping, but we should have taken care of the migration. Hope this helps.

[jmg011]

@ruanruijuan Thank you for the information. Because ZAPI and REST templates are not 1:1 mapped, I would have to check every single metrics and labels for ZAPI to REST migration. I have created many more custom collections for business requirements that need to be updated as well. Since we have many automations consuming the data and entire monitoring solution is built on top of it, we need those compatibility checks. Is there any place where those known issues/differences are published between ZAPI & REST?

When zapi support ends, will this Harvest team do 1:1 mapping for ZAPI to REST for new Harvest versions? Or this is for business users to test? What is your suggestion?

[ruanruijuan]

Harvest has done the ZAPI to REST migration and it should be a seamless transition from the customer point of view (we are not changing grafana which consumes the same counter names). We just haven't officially announced the cutover yet since ONTAP cutover is post 9.12.1.

For the custom metrics/counters that you add yourself, on the same bulletin page above, there is a link to ONTAPI to REST API mapping information.. You can use that mapping file to find the corresponding REST end point and counters. It also mentioned a process of ONTAPI to ONTAP REST API gaps form.. These are for the configuration related counters. For performance metrics, ONTAP hasn't officially published a document yet and we will share once that is available (together with a process of how to fill the gaps if mapping is not found). If you have problem finding the right RESTAPI mapping, feel free to drop a note in our Github.

[jmg011]

@ruanruijuan - Thank you for a quick response on this. I got the info I need to plan/prep for the migration and will wait for official Harvest announcement for the cutover. My intention is to keep the metrics and labels for perf/config counters as is to maintain integrity of heavily customized/ground up dashboards and core automations consuming the metrics.

I will get back if I get stuck and need any information. Thank you for your time and useful details.

[ruanruijuan]

You are very welcome. While the REST data collectors (both configuration and performance) are already working in Harvest, feel free to give a try and let us know your feedback!