UPS/NUT (Network UPS Tool)

UPS/NUT (Network UPS Tool)#

NUT#

NUT (Network UPS Tool) integrates a series of convenient operation and notification tools for UPS.

NUT mainly consists of three core services:

nut-driver: This service is responsible for communicating with the UPS through specific drivers.
nut-server: This service communicates with the UPS using nut-driver and publishes the UPS status through network services.
nut-monitor (nut-client): This service connects to nut-server and responds based on the UPS status.

Note: nut-client is actually a systemd symlink file, essentially still nut-monitor.

             ┌─────────────┐                  ┌────────────┐
       ┌──── │ nut-monitor │ ───────────────► │ nut-server │
       │     └─────────────┘                  └────────────┘
       │
       │                                            │
       │                                            │
       ▼                                            ▼
 ┌─────────────┐                              ┌────────────┐
 │  upssched   │                              │ nut-driver │
 └─────────────┘                              └────────────┘

        │                                           │
        │                                           │
        │                                           │
        ▼                                           ▼
┌────────────────┐                            ┌─────────────┐
│  user scripts  │                            │     UPS     │
└────────────────┘                            └─────────────┘

Install NUT#

Install directly from the pve console

apt install -y nut

Connect to the UPS with the command:

# nut-scanner -U
SNMP library not found. SNMP search disabled.
Neon library not found. XML search disabled.
IPMI library not found. IPMI search disabled.
Scanning USB bus.
[nutdev1]
	driver = "blazer_usb"
	port = "auto"
	vendorid = "0665"
	productid = "5161"
	product = "USB to Serial"
	vendor = "INNO TECH"
	bus = "001"

/bin/upsc nutdev1@localhost
Init SSL without certificate database
battery.charge: 100
battery.voltage: 13.50
battery.voltage.high: 13.00
battery.voltage.low: 10.40
battery.voltage.nominal: 12.0
...

Query the status of the UPS.

NUT Server Configuration#

After NUT is installed, configuration files will be automatically generated in the /etc/nut directory. The subsequent configuration work mainly involves adjusting various configuration files in this directory.

nut.conf#

This configuration mainly defines the operation mode of NUT, with only one configuration field MODE=xxxx. The optional values and their meanings are as follows:

none: NUT is not configured or uses an external system to start the NUT service, which can be understood as "doing nothing."
standalone: Standalone mode, generally used when there is only one UPS and it is responsible for the local system (not providing network services).
netserver: Similar to standalone mode, it will start the driver, upsd, and upsmon services. The difference is that it can provide network services, allowing nut-monitor on other machines to connect to the NUT server over the network.
netclient: Client-only mode, only starts nut-monitor, used to connect to remote NUT services.

Here, I am using the SANTAK TG-BOX 850 connected to my pve via USB, and since pve has NUT installed, it acts as the server, while others can query the server's status.

# cat /etc/nut/nut.conf
MODE=netserver

ups.conf#

This configuration is used to define how nut-driver connects to the physical UPS. The configuration format of this file is as follows:

# Note that if you want Synology or QNAP to connect to the UPS over the network,
# the name of the UPS has special requirements (Synology must be named ups, QNAP has not been tested and may be called qnapups);
[ups]
	driver = usbhid-ups
	port = auto
	desc = "TG-BOX 850"

upsd.conf#

This configuration file is used to control the network services of nut-server, such as listening ports, maximum connections, certificate configurations, etc. Since I am using it on an internal network, I only need to configure network listening, and the other parameters can remain default:

LISTEN 0.0.0.0 3493

upsd.users#

The upsd.users configuration file is used to define the usernames and passwords for connecting to the nut-server over the network. A sample configuration is as follows:

[monuser]
    password = secret
    upsmon master

About Synology and QNAP users: If you expect these two NAS devices to connect directly to the NUT server, it can be confirmed that Synology needs to ensure that there is a user named monuser with the password secret; for QNAP, I have not verified it, but the results found online indicate that there needs to be a user named admin with the password 123456.

NUT Monitoring Configuration#

Compared to the basic configuration, the core processing of NUT lies in how to properly configure monitoring; NUT supports two main types of monitoring strategies:

1. Directly configured by upsmon.conf, any events are handled by user-specified scripts, without timers or other advanced features.
1. In upsmon.conf, configure the execution script as upssched, where any events are first handled by upssched. With upssched, some advanced features can be implemented, such as selectively triggering shutdowns.

upsmon.conf#

This configuration is mainly used to configure how nut-monitor monitors the UPS, while defining what actions to take when certain events occur with the UPS. Below is a detailed explanation of the core configuration.

MONITOR Directive#

The MONITOR directive is used to define the connection address of the UPS to be monitored, with the following format:

MONITOR <system> <powervalue> <username> <password> ("master"|"slave")

<system>: The connection address of the nut-server, formatted as "UPS name" + "@" + "nut-server address", for example, myups@192.168.1.2.
<powervalue>: The number of UPS units, in most cases you only have one UPS power supply, so just write 1.
<username>/<password>: The username and password defined in upsd.users.
master/slave: master indicates that this system will shut down last, allowing the slave systems to shut down first; slave indicates that this system will shut down immediately.

Here is my configuration example:

MONITOR ups@localhost 1 monuser secret master

SHUTDOWNCMD#

The SHUTDOWNCMD directive is used to define the shutdown command when the UPS power is insufficient or needs to shut down actively. It is recommended to provide the full path.

SHUTDOWNCMD "/usr/sbin/poweroff"

NOTIFYCMD#

The NOTIFYCMD directive is a very important directive, this directive is used to configure the command executed by nut-monitor when specific events occur (such as power outages, UPS in low power, etc.). In simple terms, NOTIFYCMD defines the specific command to be executed, and you can directly configure a script you wrote here, which will be called whenever an event occurs with the UPS.

The NOTIFYCMD directive can be configured in two main ways: one is to configure it as your own script, the script needs to have executable permissions, and within the script, you can use the NOTIFY environment variable to obtain the event type and handle it yourself. This method is somewhat "simple and crude," and the degree of customization entirely depends on how you write your script. The specific variables you can use should be tested by yourself, as the variable names may differ between versions.

Another way is to configure NOTIFYCMD to execute the built-in upssched command; upssched is a scheduler provided by NUT with specific strategies; in short, it abstracts commonly used functions, and upssched has its own separate configuration, allowing for advanced scheduling strategies such as "if the power is restored within 180 seconds, do not shut down."

Here, I choose to use the second method,

NOTIFYCMD /usr/sbin/upssched

NOTIFYFLAG#

NOTIFYFLAG is also a key configuration that needs to be used in conjunction with NOTIFYCMD; the NOTIFYFLAG directive is responsible for specifying a series of UPS events that should trigger what kind of operation. The directive format is as follows:

NOTIFYFLAG <notify type> <flag>[+<flag>][+<flag>] ...

Where <notify type> indicates the event type, and the optional types are as follows:

ONLINE: UPS online, triggered when power is restored.
ONBATT: UPS is powered by battery, triggered when power is interrupted.
LOWBATT: Triggered when UPS is low on battery.
FSD: UPS is being shut down (Forced Shutdown).
COMMOK: Triggered when a successful connection is established with the nut-server.
COMMBAD: Triggered when the connection to the nut-server fails (connection lost).
SHUTDOWN: Triggered when the UPS issues a shutdown command.
REPLBATT: Triggered when the UPS needs a battery replacement.
NOCOMM: Triggered when unable to connect to the UPS (UPS not ready).

For the flag, there are usually four types, and multiple combinations are connected with a plus sign (+):

SYSLOG: Only print to syslog.
WALL: Pop up messages on the terminal (/bin/wall).
EXEC: Call the command specified by NOTIFYCMD, passing relevant events.
IGNORE: Do nothing, ignore the event.

If NOTIFYCMD uses a custom script, please configure the events that need to be handled by the script according to actual needs; if NOTIFYCMD is configured to use upssched, you can configure all events as EXEC, and then filter them in upssched.

For example, if you only want the script configured by NOTIFYCMD to handle power interruption and restoration events while also printing to syslog, you can configure it like this:

NOTIFYFLAG ONLINE SYSLOG+EXEC
NOTIFYFLAG ONBATT SYSLOG+EXEC

Since I configured NOTIFYCMD as upssched, I will pass all events to upssched:

NOTIFYFLAG ONLINE SYSLOG+EXEC
NOTIFYFLAG ONBATT SYSLOG+EXEC
NOTIFYFLAG LOWBATT SYSLOG+EXEC
NOTIFYFLAG FSD SYSLOG+EXEC
NOTIFYFLAG COMMOK SYSLOG+EXEC
NOTIFYFLAG COMMBAD SYSLOG+EXEC
NOTIFYFLAG SHUTDOWN SYSLOG+EXEC
NOTIFYFLAG REPLBATT SYSLOG+EXEC
NOTIFYFLAG NOPARENT SYSLOG+EXEC

Other Configurations#

In addition to the core configurations above, other configurations can generally remain default unless there are special circumstances; specific configurations can refer to the official documentation: UPSMON.CONF(5).

upssched.conf#

The prerequisite for using this configuration is that the NOTIFYCMD in upsmon.conf points to upssched, and NOTIFYFLAG is configured with EXEC for relevant events; this configuration mainly serves to use some built-in advanced syntax of upssched to control how specific events are handled. The upssched.conf configuration mainly consists of two parts: header option configuration and tail rule configuration.

4.2.1. Option Configuration#

The header option configuration contains only three configurations:

CMDSCRIPT: This configuration should be on the first line (recommended this way, actually it can be before the AT directive), this directive is used to define the event handling script; the script is generally written by the user, upssched will pass the specified parameters to this script and execute it based on the rules.
PIPEFN: The pipe file for inter-process communication, needs to be placed before the AT directive.
LOCKFN: The mutex lock file, used to prevent upsmon from scheduling multiple files simultaneously, needs to be placed before the AT directive.

Here is a sample script for the CMDSCRIPT configuration:

#!/usr/bin/env bash

set -ex

exec >> /var/log/upssched-cmd.log 2>&1

# Function mainly responsible for shutting down the system
function xpoweroff(){
    logger -t upssched-cmd 'Preparing to shut down XPEnology...'
    logger -t upssched-cmd 'Preparing to shut down TProxy Gateway...'
    logger -t upssched-cmd 'All necessary systems have been successfully shut down...'
}

# Determine the event triggered by upssched
case $1 in
    onbattwarn)
        logger -t upssched-cmd 'UPS has switched to battery power, preparing to safely shut down the system...'
        xpoweroff
        ;;
    ups-back-on-line)
        logger -t upssched-cmd 'Power has been restored...'
        ;;
    lowbatt)
        logger -t upssched-cmd 'UPS battery is low, shutting down the system immediately...'
        xpoweroff
        ;;
    *)
        logger -t upssched-cmd "Unrecognized command: $1"
        ;;
esac

One thing to note is that the script configured for CMDSCRIPT runs as the nut user by default, so you need to handle the permissions for the nut user properly.

For PIPEFN and LOCKFN, the official recommendation is to create a separate directory called upssched and place the files in this directory; however, some systems, such as Ubuntu Server 22.04, have /run/nut/ as tmpfs, which will lose the directory upon reboot, leading to permission issues; therefore, it is recommended to configure it directly without creating a separate directory:

CMDSCRIPT /opt/scripts/upssched-cmd.sh
PIPEFN /run/nut/upssched.pipe
LOCKFN /run/nut/upssched.lock

4.2.2. Rule Configuration#

The benefit of using upssched is that it has a built-in rule engine, allowing us to configure complex rules through simple syntax; the rule syntax of upssched is as follows:

AT notifytype upsname command

Rules start with AT, notifytype is used to specify the event type to be monitored; upsname specifies the UPS name, which can be written as * if there is only one or if you do not want to differentiate; the command part is used to specify the action to be executed, which can be of three types:

START-TIMER: Start a timer.
CANCEL-TIMER: Cancel a timer.
EXECUTE: Execute immediately.

This part may seem complex but is actually quite simple. For example, the following rule implements: When the power goes out (UPS is powered by battery), start a timer named onbattwarn, which will execute the script defined in CMDSCRIPT after 180 seconds and pass onbattwarn as the first parameter to the script; at the same time, if the power is restored within the 180 seconds of the timer (temporary flicker), cancel the script execution.

AT ONBATT * START-TIMER onbattwarn 180
AT ONLINE * CANCEL-TIMER onbattwarn

Of course, some events need to be executed immediately, for example: Immediately send a notification when the power goes out.

# Immediately execute the script specified by `CMDSCRIPT`, passing `onbattnoti` as a parameter to the script
AT ONBATT * EXECUTE onbattnoti

NUT Server Configuration Files#

The complete configuration is:

nut.conf

MODE=netserver

ups.conf

# The name is initially compatible with Synology, so it is called UPS
[ups]
    driver = "usbhid-ups"
    port = "auto"

upsd.conf

LISTEN 0.0.0.0 3493

upsd.users

[monuser]
    password = secret
    upsmon master
[qnapups]
    password = 123456
    upsmon master

upsmon.conf

MONITOR ups@localhost 1 monuser secret master
MINSUPPLIES 1
SHUTDOWNCMD "/usr/sbin/poweroff"
NOTIFYCMD /usr/sbin/upssched
POLLFREQ 5
POLLFREQALERT 5
HOSTSYNC 30
DEADTIME 15
POWERDOWNFLAG /etc/killpower
NOTIFYFLAG ONLINE SYSLOG+EXEC
NOTIFYFLAG ONBATT SYSLOG+EXEC
NOTIFYFLAG LOWBATT SYSLOG+EXEC
NOTIFYFLAG FSD SYSLOG+EXEC
NOTIFYFLAG COMMOK SYSLOG+EXEC
NOTIFYFLAG COMMBAD SYSLOG+EXEC
NOTIFYFLAG SHUTDOWN SYSLOG+EXEC
NOTIFYFLAG REPLBATT SYSLOG+EXEC
NOTIFYFLAG NOPARENT SYSLOG+EXEC
RBWARNTIME 43200
NOCOMMWARNTIME 300
FINALDELAY 5

upssched.conf

CMDSCRIPT /opt/scripts/upssched-cmd.sh
PIPEFN /run/nut/upssched.pipe
LOCKFN /run/nut/upssched.lock
AT ONBATT * START-TIMER onbattwarn 180
AT ONLINE * CANCEL-TIMER onbattwarn
AT ONLINE * EXECUTE ups-back-on-line
AT LOWBATT * EXECUTE lowbatt

upssched-cmd.sh

#!/usr/bin/env bash

set -ex

exec >> /var/log/upssched-cmd.log 2>&1

echo "Executing ID: $(id)"

SSH_CMD='ssh -o StrictHostKeyChecking=no'

case $1 in
    onbattwarn)
        logger -t upssched-cmd 'UPS has switched to battery power, preparing to safely shut down the system...'
        ;;
    ups-back-on-line)
        logger -t upssched-cmd 'Power has been restored...'
        ;;
    lowbatt)
        logger -t upssched-cmd 'UPS battery is low, shutting down the system immediately...'
        ;;
    *)
        logger -t upssched-cmd "Unrecognized command: $1"
        ;;
esac

Restart NUT Server#

systemctl restart nut-server.service
systemctl restart nut-driver.service
systemctl restart nut-monitor.service

NUT Client Configuration#

Configure the nut Client#

Edit the /etc/nut/nut.conf file and set MODE to netclient mode.

MODE=netclient

Configure upsmon#

Edit the /etc/nut/upsmon.conf file and add the following configuration items:

MONITOR ups_name@ups_ip_address 1 username password slave
MINSUPPLIES 1
SHUTDOWNCMD "/usr/sbin/poweroff"
NOTIFYCMD /usr/sbin/upssched
POLLFREQ 5
POLLFREQALERT 5
HOSTSYNC 30
DEADTIME 15
POWERDOWNFLAG /etc/killpower
NOTIFYFLAG ONLINE SYSLOG+EXEC
NOTIFYFLAG ONBATT SYSLOG+EXEC
NOTIFYFLAG LOWBATT SYSLOG+EXEC
NOTIFYFLAG FSD SYSLOG+EXEC
NOTIFYFLAG COMMOK SYSLOG+EXEC
NOTIFYFLAG COMMBAD SYSLOG+EXEC
NOTIFYFLAG SHUTDOWN SYSLOG+EXEC
NOTIFYFLAG REPLBATT SYSLOG+EXEC
NOTIFYFLAG NOPARENT SYSLOG+EXEC
RBWARNTIME 43200
NOCOMMWARNTIME 300
FINALDELAY 5
RUN_AS_USER root

Note the RUN_AS_USER; in the upsmon.conf file, it usually defines the user under which upsmon runs. In some systems, upsmon runs as a normal user by default, which may prevent the shutdown command from being executed, so it needs to be set to root.

Configure upssched#

The wall command is used in the system to broadcast messages to all logged-in users. The default configuration of upsmon will use the wall command to send notification messages to all users.

The wall tool needs to be installed; the installation process for Gentoo is:

echo "sys-apps/util-linux tty-helpers" >> /etc/portage/package.use
sudo emerge -v sys-apps/util-linux
# Verify if installed
which wall

The upssched tool allows you to execute scripts under specific conditions. Edit the /etc/nut/upssched.conf file and add the following content:

CMDSCRIPT /etc/nut/upssched-cmd.sh
PIPEFN /run/nut/upssched.pipe
LOCKFN /run/nut/upssched.lock
AT ONBATT * START-TIMER onbattwarn 600
AT ONLINE * CANCEL-TIMER onbattwarn
AT ONLINE * EXECUTE ups-back-on-line
AT LOWBATT * EXECUTE lowbatt

This will send the onbattwarn event to the script after 600 seconds of UPS power, then execute the commands defined in the script.

The content of /etc/nut/upssched-cmd.sh is:

#!/usr/bin/env bash

set -ex

exec >> /var/log/upssched-cmd.log 2>&1

echo "Executing ID: $(id)"

function xpoweroff(){
        /sbin/shutdown -h +0 "UPS has been on battery for 1 minute, shutting down."
}

case $1 in
    onbattwarn)
        logger -t upssched-cmd 'UPS has switched to battery power, preparing to safely shut down the system...'
        xpoweroff
        ;;
    ups-back-on-line)
        logger -t upssched-cmd 'Power has been restored...'
        ;;
    lowbatt)
        logger -t upssched-cmd 'UPS battery is low, shutting down the system immediately...'
        xpoweroff
        ;;
    *)
        logger -t upssched-cmd "Unrecognized command: $1"
        ;;
esac

Grant executable permissions:

sudo chmod +x /etc/nut/upssched-cmd.sh

Restart NUT Client#

systemctl enable nut-monitor.service
systemctl restart nut-monitor.service
systemctl status nut-monitor.service

Verify Configuration#

You can test the UPS status with the following command:

upsc <ups_name>@IP

intent的blog