Deploying Netdata to a New Server¶

This covers the full Netdata setup for a new server in the fleet: install, email notification config, n8n webhook integration, and Netdata Cloud claim. Applies to Ubuntu/Debian servers.

1. Install Prerequisites¶

Install jq before anything else. It is required by the custom_sender() function in health_alarm_notify.conf to build the JSON payload sent to the n8n webhook. If jq is missing, the webhook will fire with an empty body and n8n alert emails will have no information in them.

apt install -y jq

Verify:

jq --version

2. Install Netdata¶

Use the official kickstart script:

wget -O /tmp/netdata-install.sh https://get.netdata.cloud/kickstart.sh
sh /tmp/netdata-install.sh --non-interactive --stable-channel --disable-telemetry

Verify it's running:

systemctl is-active netdata
curl -s http://localhost:19999/api/v1/info | python3 -c "import sys,json; d=json.load(sys.stdin); print('Netdata', d['version'])"

3. Configure Email Notifications¶

On an EXISTING host, this cp wipes the n8n enriched-alert integration — corrected 2026-07-20

Every fleet host's live /etc/netdata/health_alarm_notify.conf carries the custom_sender() function that ships enriched alerts to the n8n webhook (verified 2026-07-20: custom_sender=3, n8n=1 fleet-wide). Copying the stock file over it (the command below) deletes that integration — and Step 6 then tells you to copy it back from another host, so following this doc top-to-bottom silently breaks then restores alerting. This is a new-server runbook; on a host that already has notifications configured, skip this cp entirely.

New host: copy from a host that already has the integration (per Step 6), not the stock template — then set the three values. Or cp the stock file only if /etc/netdata/health_alarm_notify.conf does not already exist:

# New host only — refuse to clobber an existing configured file:
cp -n /usr/lib/netdata/conf.d/health_alarm_notify.conf /etc/netdata/health_alarm_notify.conf

Edit /etc/netdata/health_alarm_notify.conf:

EMAIL_SENDER="netdata@majorshouse.com"
SEND_EMAIL="YES"
DEFAULT_RECIPIENT_EMAIL="marcus@majorshouse.com"

Or apply with sed in one shot:

sed -i 's/^#\?EMAIL_SENDER=.*/EMAIL_SENDER="netdata@majorshouse.com"/' /etc/netdata/health_alarm_notify.conf
sed -i 's/^#\?SEND_EMAIL=.*/SEND_EMAIL="YES"/' /etc/netdata/health_alarm_notify.conf
sed -i 's/^#\?DEFAULT_RECIPIENT_EMAIL=.*/DEFAULT_RECIPIENT_EMAIL="marcus@majorshouse.com"/' /etc/netdata/health_alarm_notify.conf

Restart and test:

systemctl restart netdata
/usr/libexec/netdata/plugins.d/alarm-notify.sh test 2>&1 | grep -E '(OK|FAILED|email)'

You should see three # OK lines (WARNING → CRITICAL → CLEAR test cycle) and confirmation that email was sent to marcus@majorshouse.com.

Delivery via local Postfix

Email is relayed through the server's local Postfix instance. Ensure Postfix is installed and /usr/sbin/sendmail resolves.

4. Configure n8n Webhook Notifications¶

Copy the health_alarm_notify.conf from an existing server (e.g. majormail) which contains the custom_sender() function. This sends enriched JSON payloads to the n8n webhook at https://n8n.majorshouse.com/webhook/netdata-alert.

jq required

The custom_sender() function uses jq to build the JSON payload. If jq is not installed, payload will be empty, curl will send Content-Length: 0, and n8n will produce alert emails with Host: unknown, blank alert/value fields, and Status: UNKNOWN. Always install jq first (Step 1).

After deploying the config, run a test to confirm the webhook fires correctly:

systemctl restart netdata
/usr/libexec/netdata/plugins.d/alarm-notify.sh test 2>&1 | grep -E '(custom|n8n|OK|FAILED)'

Verify in n8n that the latest execution shows a non-empty body with hostname, alarm, and status fields populated.

5. Claim to Netdata Cloud¶

Get the claim command from Netdata Cloud → Space Settings → Nodes → Add Nodes. It will look like:

wget -O /tmp/netdata-kickstart.sh https://get.netdata.cloud/kickstart.sh
sh /tmp/netdata-kickstart.sh --stable-channel \
  --claim-token <token> \
  --claim-rooms <room-id> \
  --claim-url https://app.netdata.cloud

Verify the claim was accepted:

cat /var/lib/netdata/cloud.d/claimed_id

A UUID will be present if claimed successfully. The node should appear in Netdata Cloud within ~60 seconds.

6. Verify Alerts¶

Check that no unexpected alerts are active after setup:

curl -s 'http://localhost:19999/api/v1/alarms?active' | python3 -c "
import sys, json
d = json.load(sys.stdin)
active = [v for v in d.get('alarms', {}).values() if v.get('status') not in ('CLEAR', 'UNINITIALIZED', 'UNDEFINED')]
print(f'{len(active)} active alert(s)')
for v in active:
    print(f'  [{v[\"status\"]}] {v[\"name\"]} on {v[\"chart\"]}')
"

Fleet-wide Alert Check¶

To audit all servers at once (requires Tailscale SSH access):

for host in majorlab majorhome majormail majordiscord majortoot majorlinux tttpod dca teelia; do
  echo "=== $host ==="
  ssh root@$host "curl -s 'http://localhost:19999/api/v1/alarms?active' | python3 -c \
    \"import sys,json; d=json.load(sys.stdin); active=[v for v in d.get('alarms',{}).values() if v.get('status') not in ('CLEAR','UNINITIALIZED','UNDEFINED')]; print(str(len(active))+' active')\""
done

Fleet-wide jq Audit¶

To check that all servers with custom_sender have jq installed:

for host in majorlab majorhome majormail majordiscord majortoot majorlinux tttpod dca teelia; do
  echo -n "=== $host: "
  ssh -o ConnectTimeout=5 root@$host \
    'has_cs=$(grep -l "custom_sender\|n8n.majorshouse.com" /etc/netdata/health_alarm_notify.conf 2>/dev/null | wc -l); has_jq=$(which jq 2>/dev/null && echo yes || echo NO); echo "custom_sender=$has_cs jq=$has_jq"'
done

Any server showing custom_sender=1 jq=NO needs apt install -y jq immediately.