Manual Configuration

This page is the reference for the mango-ctl nvmf commands that build a Mango BoostX™ NRT NVMe-oF target step by step. Use it when you need a layout the policy file cannot express, or when you are diagnosing a problem and want to perform one step at a time.

For normal operation, prefer Automatic Configuration — it runs exactly these commands for you at boot.

warning

While mango-jbof is running it re-applies its own policy and will revert a configuration you build here. Before configuring by hand, either stop the service with mango-ctl nvmf autoconfig --disable, or build your configuration and then pin it with mango-ctl nvmf autoconfig --commit-state -f.

info

The nvmf commands run on the card, not on a host server.

Every nvmf subcommand accepts -g/--device-group to select the device group, which defaults to 0. A single-card appliance only has group 0, so the option is omitted throughout this page.

Service Lifecycle

The target daemon must be running before any other nvmf command will work. Until it is, every subcommand exits with code 3.

(target) ~# mango-ctl nvmf start

Option	Default	Description
-t, --trtype	RDMA	Transport type. NRT hardware supports RDMA only
-m, --max-queues	32	Maximum I/O queues per subsystem. Also capped by the hardware and by the SSD count
-c, --cpu-affinity	0	CPU core that runs the target
-o, --timeout	10	Socket close timeout, in milliseconds

-m is the value to adjust when you export many subsystems — see I/O Queue Budget.

To stop the target:

(target) ~# mango-ctl nvmf stop

info

stop is refused with exit code 6 while initiators are still connected. Disconnect them first, or force the teardown with -f/--force. Either way the stop can take up to 90 seconds; the command reports progress while it works, so let it finish rather than interrupting it.

Attaching NVMe Devices

Attaching an SSD moves it from the kernel NVMe driver to the target daemon and gives it a controller name (Nvme0, Nvme1, …) used by the later commands.

(target) ~# mango-ctl nvmf attach_nvme --all

Attach specific devices

(target) ~# mango-ctl nvmf attach_nvme <bdf> [<bdf>...] [-b <name>] [-i <queues>]

• --all attaches every NVMe SSD except the OS boot device. Passing --all together with explicit BDFs is refused with exit code 2.
• Controller names are assigned automatically. -b names a controller explicitly and therefore requires exactly one target device.
• -i sets the number of I/O queues requested per device.

To release a device back to the kernel:

(target) ~# mango-ctl nvmf detach_nvme -c Nvme0
(target) ~# mango-ctl nvmf detach_nvme --all

info

detach_nvme only requires -f/--force when the controller's namespace is still exported by a subsystem; a device that is merely attached detaches without it. An unknown controller name reports No such attached NVMe controller: <name> and exits with code 4.

To look up the controller name registered for a device:

(target) ~# mango-ctl nvmf get_nvme_info <bdf>
(target) ~# mango-ctl nvmf get_nvme_info -b Nvme0

Subsystems

(target) ~# mango-ctl nvmf create_subsys -n <nqn> [-a <target-ip> [-p 4420]] [--allow_any BOOLEAN]
(target) ~# mango-ctl nvmf create_subsys --num-subsys <count> [-a <target-ip> [-p 4420]] [--allow_any BOOLEAN]

• Omitting -n assigns an automatic NQN of the form nqn.2022-02.io.mangoboost:subsys<N>.
• --num-subsys creates that many subsystems in one call, using the automatic NQNs, and defaults to 1. It cannot be combined with -n; doing so is refused with exit code 2 and the message Do not combine --num-subsys with explicit NQNs (-n).
• Passing -a registers a listener at the same time, so a separate add_listener is not needed. The port defaults to 4420.
• --allow_any defaults to false, which admits only the host NQNs registered with add_host. Pass --allow_any true to accept any initiator.

(target) ~# mango-ctl nvmf delete_subsys -n <nqn>
(target) ~# mango-ctl nvmf delete_subsys --all

--all deletes every subsystem except the discovery subsystem. Without -f, a subsystem that still holds namespaces, hosts, or listeners is listed and nothing is deleted.

Namespaces, Hosts, and Listeners

(target) ~# mango-ctl nvmf add_ns          -n <nqn> -c <name> [<name>...] [-a <target-ip>]
(target) ~# mango-ctl nvmf remove_ns       -n <nqn> --nsid <id>
(target) ~# mango-ctl nvmf add_host        -n <nqn> --hostnqn <hostnqn>
(target) ~# mango-ctl nvmf remove_host     -n <nqn> --hostnqn <hostnqn>
(target) ~# mango-ctl nvmf add_listener    -n <nqn> -a <target-ip> [-p 4420]
(target) ~# mango-ctl nvmf remove_listener -n <nqn> -a <target-ip> [-p 4420]

• add_ns accepts several controller names after a single -c, and prints the namespace ID assigned to each. On RDMA, -a may be omitted and the address is determined automatically.

  (target) ~# mango-ctl nvmf add_ns -n <nqn> -c Nvme0 Nvme1
  Added namespace 1 (Nvme0n1) to subsystem <nqn>
  Added namespace 2 (Nvme1n1) to subsystem <nqn>

• A host NQN must be between 11 and 223 characters, the range allowed by the NVMe specification. A value outside it is rejected with exit code 2 and a message naming the limit that was crossed.

• A subsystem without a listener cannot be reached by any initiator, even though it exists. Always confirm the listener with nvmf show.

Successful operations report what changed:

Added namespace 1 (Nvme0n1) to subsystem nqn.2022-02.io.mangoboost:subsys0
Added listener 200.1.1.100:4420 to subsystem nqn.2022-02.io.mangoboost:subsys0
Added host nqn.2014-08.org.nvmexpress:uuid:a734913e-15f4-4bcb-9c1b-1ef805909638 to subsystem nqn.2022-02.io.mangoboost:subsys0
Removed namespace 1 from subsystem nqn.2022-02.io.mangoboost:subsys0

An NQN that does not exist is reported the same way by every subcommand, with exit code 4:

Subsystem not found: nqn.2022-02.io.mangoboost:subsys9

Complete Example

The sequence below exports three SSDs as three subsystems with one namespace each, which is the same layout the automatic configuration produces with max_devices_per_subsystem: 1. Replace <target-ip> with the address configured on the appliance interface.

(target) ~# mango-ctl nvmf start -t RDMA
(target) ~# mango-ctl nvmf attach_nvme --all
(target) ~# mango-ctl nvmf create_subsys -n nqn.2022-02.io.mangoboost:subsys0 -a <target-ip> --allow_any true
(target) ~# mango-ctl nvmf create_subsys -n nqn.2022-02.io.mangoboost:subsys1 -a <target-ip> --allow_any true
(target) ~# mango-ctl nvmf create_subsys -n nqn.2022-02.io.mangoboost:subsys2 -a <target-ip> --allow_any true
(target) ~# mango-ctl nvmf add_ns -n nqn.2022-02.io.mangoboost:subsys0 -c Nvme0
(target) ~# mango-ctl nvmf add_ns -n nqn.2022-02.io.mangoboost:subsys1 -c Nvme1
(target) ~# mango-ctl nvmf add_ns -n nqn.2022-02.io.mangoboost:subsys2 -c Nvme2
(target) ~# mango-ctl nvmf show

attach_nvme --all assigns the controller names in PCI address order, so verify them with nvmf show before assuming which SSD became Nvme0.

Tearing Down

Undo the configuration in the reverse order:

(target) ~# mango-ctl nvmf delete_subsys --all -f
(target) ~# mango-ctl nvmf detach_nvme --all -f
(target) ~# mango-ctl nvmf stop -f

Exit Codes

Every nvmf subcommand reports its result through the exit code.

Code	Meaning
0	success
1	the operation ran and failed
2	missing, invalid or conflicting arguments
3	the NVMe-oF service is not running
4	subsystem, controller or device does not exist
5	refused: destructive, re-run with -f/--force
6	refused: connections or resources still in use

An argument error takes precedence over the service state: a malformed command returns 2 even when the service is down, which lets a script tell a bug in itself apart from a problem with the appliance. --help always returns 0.

info

When scripting the appliance, branch on the exit code rather than on the text of the output. Where a script needs the configuration itself, use the machine-readable form: nvmf show --json or nvmf list_subsys --json.

These codes apply to the nvmf subcommands. Other command groups such as dev and mgmt are shared with other MangoBoost products and keep their own conventions.