Snapmirror READ throughput not showing up

[wally007]

Hello,

i'm trying to monitor Snapmirror throughput between 2 clusters that are in SVM-DR relationship.

I noticed that when i query destination cluster/node as such:
sum(volume_write_data{cluster=~"$Cluster"}) by(cluster)
i get the snapmirror write throughput just fine.

However when i query source cluster/node:
sum(volume_read_data{cluster=~"$Cluster"}) by(cluster)
i dont see the corresponding read throughput.

Is that expected ? If so, how would one monitor READ throughput of the Snapmirror SVM-DR ?
Thank you

Kind regards

PS>

(i noticed that the network throughput does show TX (sent) traffic on the source and RX (received) on the destination)
sum (lif_sent_data{cluster=~"$Cluster",port=~"a.+"}) by(cluster)

The reason why i ask this question is that i saw 20Gb/s network traffic going out from the source node but there was no activity shown in the data throughput, which confused me until i realized its Snapmirror TX traffic.

[wally007]

Here is the panel that show what i'm talking about:
At 12:10am , Snapmirror SVM-DR kicks in and the data gets moved from TROY (the source- read) to AGORA (destination - write)

Query:

sum(volume_write_data{cluster=~"$Cluster"}) by(cluster)
sum(volume_read_data{cluster=~"$Cluster"}) by(cluster) *(-1)

As you can see - only WRITE throughput shows up.

[wally007]

hello,

sorry for the bump but i'm still trying to figure out how to gather the read throughput and if its possible at all.

Can anyone please help me ?

Thank you

[cgrinds]

hi @wally007 we'll take a look - are you also using the SnapMirror object in your conf/zapi/default.yaml. If so, is the total-transfer-bytes counter helpful?

[wally007]

Hi - thank you for your response.

Yes, I'm using Snapmirror in zapi ( i'm using default config except Qtree, 7mode and LUN - all of which we arent using)

I've looked at total-tranfer-bytes and it is helpful as well and i was planning to use it once i create dashboard/panel for Snapmirror stats.

However i'm running into inconsistency in performance stats.
It is confusing to see at 12am almost 20Gbit write operation on node "AGORA" (and also see it in the network bandwidth) and not able account for it. It doesnt show in the volume read stats, it doesnt show in NFS read stats (on node "TROY") - basically it looks like the cluster went haywire.

Basically it is a copy job from A to B but the transfer stats only show up on B as write. It looks like read never happend :-)

[ruanruijuan]

@wally007 We contacted ONTAP snapmirror team and was told that SnapMirror reads aren't tracked in volume counters. So the current behavior is expected. He recommended to get read counters is repl_snapdiff->num_blks_read (for LRSE transfers). But we don't think this counter is available via ZAPI.

[wally007]

thanks for following up.

I'm not too thrilled by the response from ONTAP team. Whats the reason for including WRITE counters but NOT include READ counters ?
That's like building one way highway.

I also noticed "snapmirror_last_transfer_size" is reporting incorrect values in SVM-DR relationship if 2 or more volumes are being transferred.

It seems to report only the last volume size transferred - not the whole SVM-DR transfer size.
I checked ONTAP CLI and its same there - so i'm guessing its yet another "feature" of ONTAP 👎

(opening tickets with support seem to follow the same pattern here - whenever we hit the bug we are just told this is how it is and bye)