Skip to content

Latest commit

 

History

History
104 lines (81 loc) · 3.39 KB

TopologyAPI.md

File metadata and controls

104 lines (81 loc) · 3.39 KB
Raw

SHARP Topology API

© Copyright 2021, NVIDIA CORPORATION & AFFILIATES, Inc. All Rights Reserved.

See file LICENSE for terms.

Changelog

Date Description
02 May, 2021 Initial version
18 May, 2021 Updated with new configuration required by SHARP 2.4.6 (explicit parameter which enables "Topology API"

Prerequisites

Component Version
"Server" side UFM Appliance gen 2.5 4.5.1.1 or later / UFM SW 6.7.X or later
"Client" side MOFED 5.3 or later / HPCX 2.8 or later

Configuration

Enabling SHARP in UFM Appliance

ib sharp enable
ib sm sharp enable
ib sharp smx-protocol ucx
ib sharp topology-api enable

General information

sharp_cmd - Utility to send and receive control messages to/from Aggregation Manager It supports the following commands: topology - Creates topology file that spans requested HCA GUIDs

Usage: sharp_cmd <command> [OPTIONS]
Example:
  sharp_cmd topology --ib-dev mlx5_0:1 --guids_file ../guids_file --topology_file ../topology_file --timeout 2

Options:
-d, --ib-dev
                IB device and port (device:port) to bind to
                If not specified, use first active IB device
--topology_file
                Topology file - path to output file of topology command
                If not specified, print to standard output
--guids_file
                HCA port GUIDs file for topology command
-t, --timeout
                Reply timeout [sec]
                0 - no timeout, wait forever
                x - timeout duration in seconds
                default value: 0

Example

This example describes how to run sharp_cmd installed using Mellanox HPCX.

  1. Create port GUIDs file. All GUIDs must be in same reservation

For example:

$ cat /tmp/guids_file
0x2c9000000000a
0x2c9000000000e
0x2c90000000012
0x2c90000000016
0x2c9000000001a
0x2c9000000001e
  1. Run command:
$HPCX_SHARP_DIR/bin/sharp_cmd topology --ib-dev mlx5_0:1 --guids_file /tmp/guids_file --topology_file /tmp/topology_file
       --ib-dev mlx5_0:1 --> Choose IB device that is connected to IB sunbet with SHARP Aggregation Manager.
       --topology_file /tmp/topology_file --> Path to topology output file.
  1. Check output file:
$ cat /tmp/topology_file
# Topology file generated by sharp_cmd
# Package: sharp-rc
# Version: 2.3.0
# Build Date: Nov  3 2020
# Last commit: 31cedfd

SwitchName=ibsw1 Switches=ibsw2,ibsw3
SwitchName=ibsw2 Switches=ibsw1,ibsw4,ibsw5
SwitchName=ibsw3 Switches=ibsw1,ibsw6,ibsw7
SwitchName=ibsw4 Nodes=0x2c9000000000e
SwitchName=ibsw5 Nodes=0x2c9000000001a,0x2c9000000001e
SwitchName=ibsw6 Nodes=0x2c9000000000a
SwitchName=ibsw7 Nodes=0x2c90000000012,0x2c90000000016

Notes

  • If SHARP Aggregation Manager runs in reservation mode ib sharp allocation enable, "Topology API" can query topology information only for current reservation