Collecting Share information from Harvest #2327

s-kuchi · 2023-08-23T19:18:32Z

s-kuchi
Aug 23, 2023

Is there a way to collect share properties data from Harvest. I don't see a yml for this. I am wondering if I am missing something.

Can someone point me to it?

Currently on harvest version 23.02.0-1 (commit 627311a)

Essentially the outputs of the following commands

vserver cifs share show
vserver cifs share properties show

Answered by rahulguptajss

Sep 7, 2023

I see. It appears we overlooked setting share-name as a key. Could you please test the following template?

name:               CIFSShare
query:              cifs-share-get-iter
object:             cifs_share

counters:
  cifs-share:
    - ^^vserver                            => svm
    - ^^share-name                         => name
    - ^volume                              => volume
    - ^path                                => path
    - share-properties:
        - ^cifs-share-properties           => properties
    - symlink-properties:
        - ^cifs-share-symlink-properties   => symlink_properties

collect_only_labels: true

export_options:
  instance_keys:
    - svm
    - name
  ins…

View full answer

cgrinds · 2023-08-23T19:36:16Z

cgrinds
Aug 23, 2023
Maintainer

hi @s-kuchi we added CIFS (SMB) templates and an SMB dashboard in 23.05.
Can you upgrade to 23.08 and let us know if that has what you need?

0 replies

s-kuchi · 2023-08-23T19:40:35Z

s-kuchi
Aug 23, 2023
Author

Hi @cgrinds : If it already exists in 23.05, then can you direct me to where they are located?

Regarding upgrade to 23.08: I have some client_timeout values set on the yaml files in the /conf templates in 23.05, would i need to set the timeouts again after the upgrade?

0 replies

rahulguptajss · 2023-08-24T07:42:21Z

rahulguptajss
Aug 24, 2023
Collaborator

@s-kuchi The templates related to CIFS in Harvest are as follows:

CIFS Session Template
SMB2 Template
CIFS Vserver Template

Unfortunately, none of these templates cater to your specific requirement as Harvest does not currently collect this data. However, We can create a new template using the ZAPI cifs-share-get-iter or the REST endpoint /protocols/cifs/shares to gather the information you need.

In response to your question about client_timeout, it's important to note that if you've modified any templates provided with Harvest, these changes will be overwritten during upgrade. To avoid this, you may want to consider extending the templates instead. Please refer to the Zapi and Rest documentation for guidance on how to extend templates effectively.

2 replies

s-kuchi Sep 5, 2023
Author

@rahulguptajss @cgrinds : I upgraded Harvest. Does the following template work?

[root@pduulpprmpol01 harvest]# more /opt/harvest/conf/zapi/cdot/9.8.0/cifs-share-get-iter.yaml 
name:               CIFSShareGet
query:              cifs-share-get-iter
object:             cifs_shareget

counters:
  cifs-shareget:
    - ^^vserver                => svm
    - ^share-name              => name
    - ^volume                  => volume
    - ^path                    => path

export_options:
  instance_keys:
    - svm
    - name
    - volume
    - path
  instance_labels:
    - svm
    - name
    - volume
    - path

rahulguptajss Sep 5, 2023
Collaborator

@s-kuchi See if below works

name:               CIFSShare
query:              cifs-share-get-iter
object:             cifs_share

counters:
  cifs-share:
    - ^^vserver                            => svm
    - ^share-name                          => name
    - ^volume                              => volume
    - ^path                                => path
    - share-properties:
        - ^cifs-share-properties           => properties
    - symlink-properties:
        - ^cifs-share-symlink-properties   => symlink_properties

collect_only_labels: true

export_options:
  instance_keys:
    - svm
    - name
  instance_labels:
    - path
    - properties
    - symlink_properties
    - volume

Gives output as below

s-kuchi · 2023-09-05T17:25:03Z

s-kuchi
Sep 5, 2023
Author

Hi @rahulguptajss . I created the template and updated the default.yml, but I couldn't find it in Prometheus. Am I missing something?

[root@pduulpprmpol01 9.8.0]# pwd
/opt/harvest/conf/zapi/cdot/9.8.0
[root@pduulpprmpol01 9.8.0]# more cifs_share.yaml 
name:               CIFSShare
query:              cifs-share-get-iter
object:             cifs_share

counters:
  cifs-share:
    - ^^vserver                            => svm
    - ^share-name                          => name
    - ^volume                              => volume
    - ^path                                => path
    - share-properties:
        - ^cifs-share-properties           => properties
    - symlink-properties:
        - ^cifs-share-symlink-properties   => symlink_properties

collect_only_labels: true

export_options:
  instance_keys:
    - svm
    - name
  instance_labels:
    - path
    - properties
    - symlink_properties
    - volume

[root@pduulpprmpol01 zapi]# more default.yaml 

collector:          Zapi

# Order here matters!
schedule:
  - data: 3m

objects:
  Aggregate:                   aggr.yaml
  AggregateEfficiency:         aggr_efficiency.yaml
  CIFSSession:                 cifs_session.yaml
  CIFSShare:                   cifs_share.yaml

1 reply

rahulguptajss Sep 6, 2023
Collaborator

@s-kuchi Could you provide the output of the following command?. Replace POLLERNAME as applicable from harvest config.

bin/poller  --poller POLLERNAME --loglevel=2 --collectors Zapi --objects CIFSShare

s-kuchi · 2023-09-06T13:45:10Z

s-kuchi
Sep 6, 2023
Author

Hi @rahulguptajss . Here is the output. I ran the yaml file through a linter, and it looks good. Let me know if anything else is missing.

[root@pduulpprmpol01 harvest]# bin/poller --poller wdc_netapp_a_fas --loglevel=2 --collectors Zapi --objects CIFSShare
2023-09-06T09:31:07-04:00 INF poller/poller.go:194 > Init Poller=wdc_netapp_a_fas configPath=./harvest.yml logLevel=info version="harvest version 23.08.0-1 (commit 874a1f87) (build date 2023-08-21T08:36:53-0400) linux/amd64\n"
2023-09-06T09:31:07-04:00 INF poller/poller.go:231 > started in foreground Poller=wdc_netapp_a_fas pid=1223242
2023-09-06T09:31:07-04:00 INF collector/helpers.go:139 > best-fit template Poller=wdc_netapp_a_fas collector=Zapi:CIFSShare path=conf/zapi/cdot/9.8.0/cifs_share.yaml v=9.7.0
2023-09-06T09:31:07-04:00 WRN poller/poller.go:665 > init collector-object error="unable to import template=[conf/zapi/cdot/9.8.0/cifs_share.yaml] yaml: line 12: found character that cannot start any token" Poller=wdc_netapp_a_fas collector=Zapi object=CIFSShare
2023-09-06T09:31:07-04:00 WRN poller/poller.go:298 > no collectors initialized, stopping Poller=wdc_netapp_a_fas
2023-09-06T09:31:07-04:00 INF poller/poller.go:523 > cleaning up and stopping [pid=1223242] Poller=wdc_netapp_a_fas

1 reply

rahulguptajss Sep 6, 2023
Collaborator

@s-kuchi It appears that the presence of tabs in your template might be causing the error, as indicated in the logs (init collector-object error="unable to import template=[conf/zapi/cdot/9.8.0/cifs_share.yaml] yaml: line 12: found character that cannot start any token"). Could you try using the following template instead?

name:               CIFSShare
query:              cifs-share-get-iter
object:             cifs_share

counters:
  cifs-share:
    - ^^vserver                            => svm
    - ^share-name                          => name
    - ^volume                              => volume
    - ^path                                => path
    - share-properties:
        - ^cifs-share-properties           => properties
    - symlink-properties:
        - ^cifs-share-symlink-properties   => symlink_properties

collect_only_labels: true

export_options:
  instance_keys:
    - svm
    - name
  instance_labels:
    - path
    - properties
    - symlink_properties
    - volume

s-kuchi · 2023-09-06T15:52:20Z

s-kuchi
Sep 6, 2023
Author

@rahulguptajss : I still get some error messages but I see the output in Prometheus. Here is the log output

[root@pduulpprmpol01 harvest]# bin/poller --poller wdc_netapp_a_fas --loglevel=2 --collectors Zapi --objects CIFSShare
2023-09-06T11:48:33-04:00 INF poller/poller.go:194 > Init Poller=wdc_netapp_a_fas configPath=./harvest.yml logLevel=info version="harvest version 23.08.0-1 (commit 874a1f87) (build date 2023-08-21T08:36:53-0400) linux/amd64\n"
2023-09-06T11:48:33-04:00 INF poller/poller.go:231 > started in foreground Poller=wdc_netapp_a_fas pid=1235542
2023-09-06T11:48:33-04:00 INF collector/helpers.go:139 > best-fit template Poller=wdc_netapp_a_fas collector=Zapi:CIFSShare path=conf/zapi/cdot/9.8.0/cifs_share.yaml v=9.7.0
2023-09-06T11:48:33-04:00 ERR prometheus/prometheus.go:161 > Issue while reading prometheus port Poller=wdc_netapp_a_fas exporter=prometheus1
2023-09-06T11:48:33-04:00 ERR poller/poller.go:863 > Unable to init exporter error="missing parameter => port" Poller=wdc_netapp_a_fas name=prometheus1
2023-09-06T11:48:33-04:00 WRN poller/poller.go:692 > exporter (prometheus1) requested by (Zapi:CIFSShare) not available Poller=wdc_netapp_a_fas
2023-09-06T11:48:33-04:00 WRN poller/poller.go:307 > no exporters initialized, continuing without exporters Poller=wdc_netapp_a_fas
2023-09-06T11:48:33-04:00 INF poller/poller.go:349 > Autosupport scheduled. Poller=wdc_netapp_a_fas asupSchedule=24h
2023-09-06T11:48:33-04:00 INF poller/poller.go:358 > poller start-up complete Poller=wdc_netapp_a_fas
2023-09-06T11:48:33-04:00 INF poller/poller.go:506 > updated status, up collectors: 1 (of 1), up exporters: 0 (of 0) Poller=wdc_netapp_a_fas
2023-09-06T11:48:33-04:00 INF collector/collector.go:483 > Collected Poller=wdc_netapp_a_fas apiMs=52 calcMs=0 collector=Zapi:CIFSShare instances=9 metrics=576 parseMs=4 pluginMs=0

6 replies

rahulguptajss Sep 6, 2023
Collaborator

That's great. You can ignore this error as We were trying to run collector in debug mode. You can run harvest normally.

s-kuchi Sep 6, 2023
Author

Awesome. Thank you for your help! @rahulguptajss . Much appreciated.

s-kuchi Sep 6, 2023
Author

Hi @rahulguptajss : Spoken too soon. Looks like the output is missing data, i am not sure where and why it is missing. For example, The netapp has 6 entries, but Prometheus output shows only one

wdc_netapp_video_fas::> cifs share show -fields vserver, share-name , path
vserver              share-name path
-------------------- ---------- ----
wdc_video_cifs_fgsvm admin$     /
wdc_video_cifs_fgsvm c$         /
wdc_video_cifs_fgsvm ipc$       /
wdc_video_cifs_fgsvm WDC_Video_Ingest /wdc_video_cifs_n01_fg01
wdc_video_cifs_fgsvm WDC_Video_MAM /wdc_video_cifs_n02_fg02
wdc_video_cifs_fgsvm WDC_Video_Production /wdc_video_cifs_n01_fg03

rahulguptajss Sep 7, 2023
Collaborator

I see. It appears we overlooked setting share-name as a key. Could you please test the following template?

name:               CIFSShare
query:              cifs-share-get-iter
object:             cifs_share

counters:
  cifs-share:
    - ^^vserver                            => svm
    - ^^share-name                         => name
    - ^volume                              => volume
    - ^path                                => path
    - share-properties:
        - ^cifs-share-properties           => properties
    - symlink-properties:
        - ^cifs-share-symlink-properties   => symlink_properties

collect_only_labels: true

export_options:
  instance_keys:
    - svm
    - name
  instance_labels:
    - path
    - properties
    - symlink_properties
    - volume

Answer selected by s-kuchi

s-kuchi Sep 7, 2023
Author

@rahulguptajss : Thank you. That worked.

rahulguptajss Sep 7, 2023
Collaborator

@s-kuchi Thanks for the update.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Collecting Share information from Harvest #2327

{{title}}

Replies: 6 comments 10 replies

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Collecting Share information from Harvest #2327

s-kuchi Aug 23, 2023

Replies: 6 comments · 10 replies

cgrinds Aug 23, 2023 Maintainer

s-kuchi Aug 23, 2023 Author

rahulguptajss Aug 24, 2023 Collaborator

s-kuchi Sep 5, 2023 Author

rahulguptajss Sep 5, 2023 Collaborator

s-kuchi Sep 5, 2023 Author

rahulguptajss Sep 6, 2023 Collaborator

s-kuchi Sep 6, 2023 Author

rahulguptajss Sep 6, 2023 Collaborator

s-kuchi Sep 6, 2023 Author

rahulguptajss Sep 6, 2023 Collaborator

s-kuchi Sep 6, 2023 Author

s-kuchi Sep 6, 2023 Author

rahulguptajss Sep 7, 2023 Collaborator

s-kuchi Sep 7, 2023 Author

rahulguptajss Sep 7, 2023 Collaborator

s-kuchi
Aug 23, 2023

Replies: 6 comments 10 replies

cgrinds
Aug 23, 2023
Maintainer

s-kuchi
Aug 23, 2023
Author

rahulguptajss
Aug 24, 2023
Collaborator

s-kuchi Sep 5, 2023
Author

rahulguptajss Sep 5, 2023
Collaborator

s-kuchi
Sep 5, 2023
Author

rahulguptajss Sep 6, 2023
Collaborator

s-kuchi
Sep 6, 2023
Author

rahulguptajss Sep 6, 2023
Collaborator

s-kuchi
Sep 6, 2023
Author

rahulguptajss Sep 6, 2023
Collaborator

s-kuchi Sep 6, 2023
Author

s-kuchi Sep 6, 2023
Author

rahulguptajss Sep 7, 2023
Collaborator

s-kuchi Sep 7, 2023
Author

rahulguptajss Sep 7, 2023
Collaborator