Clustering

Clustering allows multiple CompleteFTP servers to work together as a unified file transfer platform, providing high availability and load distribution. A cluster consists of a primary server that holds the master configuration and one or more secondary servers that replicate that configuration. When changes are made on the primary, they are automatically synchronized to all secondary servers over an encrypted connection.

This chapter covers:

• Clustering concepts - Primary/secondary architecture and configuration replication
• Prerequisites - Requirements for setting up a cluster
• Server management - Adding, viewing, modifying, and removing servers
• Site-to-server mapping - Controlling which servers serve which sites
• Cluster maintenance - Upgrading, monitoring, and troubleshooting

All cluster operations use the completeftp server command group.

Note: Clustering requires the Enterprise edition of CompleteFTP.

Clustering Concepts

Primary and Secondary Servers

A CompleteFTP cluster uses a primary/secondary architecture:

• Primary server - Holds the master configuration. All configuration changes must be made on the primary server. The primary is responsible for pushing configuration updates to secondary servers.
• Secondary servers - Receive configuration from the primary via encrypted synchronization. Secondary servers serve client connections independently but do not accept configuration changes directly.

Configuration Replication

When a configuration change is made on the primary server, it is automatically replicated to all secondary servers in the cluster. This includes:

• User accounts and credentials
• Site and folder configurations
• Group memberships
• Triggers and notifications
• License information

Important: File data is not replicated between servers. Each server maintains its own file storage. Use external mechanisms (e.g., shared storage, rsync) if file replication is required.

How Clients Connect

Each server in the cluster listens on its own network interfaces and accepts client connections independently. A load balancer or DNS-based routing is typically placed in front of the cluster to distribute client connections across servers.

Prerequisites

Before setting up a cluster, ensure the following requirements are met:

Viewing Cluster Information

List Servers in the Cluster

# List all servers in the cluster
completeftp server list

By default, a standalone server shows a single entry: the local server (e.g., "Server 1").

Show Server Details

# Show all properties of a server
completeftp server show "Server 1"

Adding a Server to the Cluster

Basic Syntax

completeftp server add <localUserName> <localPassword> <remoteAddress> <remotePort> <remoteUserName> <remotePassword>

Parameters

Adding a Secondary Server

# Add a remote server to the cluster
completeftp server add admin localPass123 192.168.1.20 14983 admin remotePass456

# Add a server using a hostname
completeftp server add admin localPass123 ftp-secondary.example.com 14983 admin remotePass456

After adding, verify the server appears in the cluster:

# Confirm the new server is listed
completeftp server list

Important: The server add command must be run on the primary server. The remote server's existing configuration will be replaced by the primary's configuration during the initial synchronization.

Modifying Server Settings

# Modify a server's properties
completeftp server set "Server 2" property=value

# Disable a server (stops it from receiving sync updates)
completeftp server set "Server 2" enabled=false

# Re-enable a server
completeftp server set "Server 2" enabled=true

Site-to-Server Mapping

In a multi-site Enterprise deployment, you can control which servers handle which sites. This is configured through the site's serverMapping property.

Viewing Server Mapping

# Show the server mapping for a site
completeftp site show "My Site" serverMapping

The server mapping output shows each server and its assignment:

serverMapping:

- name: Server 1
  enabled: true
  ftpInterfaces:
  sftpInterfaces:
  httpInterfaces:

Configuring Server Mapping

Server mapping is managed through the site configuration. Each server entry in the mapping has the following properties:

When interface fields are left blank, the server uses all available interfaces for that protocol. Specifying interfaces allows fine-grained control over which network addresses each server uses for each protocol on a per-site basis.

Removing a Server from the Cluster

# Remove a server from the cluster
completeftp server remove "Server 2"

Important: Removing a server disconnects it from the cluster. The removed server retains its last synchronized configuration but will no longer receive updates from the primary.

Cluster Security Considerations

Encrypted Synchronization

All configuration synchronization between cluster members is transmitted over an encrypted connection. The admin port (14983) uses TLS to protect credentials and configuration data in transit.

Admin Credentials

• Use strong, unique admin passwords on each server
• Admin credentials are required when adding a server to the cluster
• Credentials are used for the initial handshake; subsequent synchronization uses the established secure channel
• Rotate admin passwords periodically and update cluster configurations accordingly

Network Security

# Restrict admin port access using firewall rules
# Only allow cluster members to connect on port 14983
sudo ufw allow from 192.168.1.10 to any port 14983
sudo ufw allow from 192.168.1.20 to any port 14983

Upgrading a Cluster

When upgrading CompleteFTP across a cluster, follow this procedure to minimize downtime and avoid version mismatches:

Upgrade Procedure

# 1. Verify current versions on all servers
completeftp server list

# 2. Stop the secondary server(s) first
# On each secondary server:
sudo systemctl stop completeftp

# 3. Upgrade secondary server(s)
# On each secondary server:
sudo dpkg -i completeftp-new-version.deb   # Debian/Ubuntu
# or
sudo rpm -U completeftp-new-version.rpm     # RHEL/CentOS

# 4. Start the upgraded secondary server(s)
sudo systemctl start completeftp

# 5. Stop the primary server
sudo systemctl stop completeftp

# 6. Upgrade the primary server
sudo dpkg -i completeftp-new-version.deb   # Debian/Ubuntu
# or
sudo rpm -U completeftp-new-version.rpm     # RHEL/CentOS

# 7. Start the primary server
sudo systemctl start completeftp

# 8. Verify all servers are running the same version
completeftp server list

Important: Always upgrade secondary servers before the primary. This ensures that when the primary comes back online, it can synchronize with secondaries that are already running the new version.

Monitoring Cluster Health

ClusterSyncFailed Event

CompleteFTP fires the ClusterSyncFailed event when configuration synchronization between cluster members fails. Use this event to set up alerts so that administrators are notified of sync problems immediately.

Setting Up Sync Failure Notifications

# Create a notification for cluster sync failures
completeftp notification add "ClusterSyncAlert" ClusterSyncFailed \
  admin@company.com noreply@company.com \
  "Cluster Sync Failed: %ServerName%" \
  "Cluster synchronization failed on server %ServerName% at %DateAndTime%."

# Or create a process trigger to run a script on sync failure
completeftp trigger add "ClusterSyncMonitor" ClusterSyncFailed Program \
  /usr/local/bin/alert-cluster-failure.sh '"%ServerName%" "%DateAndTime%"'

Manual Health Checks

# List all servers and verify they are enabled
completeftp server list

# Check individual server details
completeftp server show "Server 1"
completeftp server show "Server 2"

# Review server logs for sync errors
completeftp monitor set logging.level=Debug

Troubleshooting

Synchronization Failures

# Check if all servers are enabled
completeftp server list

# Verify network connectivity to the remote server
ping 192.168.1.20
nc -zv 192.168.1.20 14983

# Check that the admin port is accessible
curl -k https://192.168.1.20:14983/ 2>&1 | head -5

# Review logs for sync error details
completeftp monitor set logging.level=Debug

Server Not Joining Cluster

# Verify the remote server is running
ssh admin@192.168.1.20 "sudo systemctl status completeftp"

# Check firewall rules on both servers
sudo ufw status | grep 14983

# Ensure both servers are running the same version
# Check local version
completeftp --version

# Verify admin credentials are correct by testing locally on the remote server

Configuration Out of Sync

If a secondary server's configuration appears out of date:

# 1. Check the server is enabled in the cluster
completeftp server show "Server 2"

# 2. Temporarily disable and re-enable to force a resync
completeftp server set "Server 2" enabled=false
completeftp server set "Server 2" enabled=true

# 3. Verify sync completed
completeftp server show "Server 2"

Common Issues

Best Practices

Architecture

1. Designate a clear primary - All configuration changes should be made on the primary server only
2. Use a load balancer - Place a load balancer in front of the cluster for client connections
3. Plan for failover - Document the procedure for promoting a secondary to primary if needed
4. Keep server count manageable - More servers means more synchronization overhead

Security

1. Secure the admin port - Use firewall rules to restrict port 14983 to cluster members only
2. Use strong admin passwords - Each server should have a strong, unique admin password
3. Monitor sync events - Set up notifications for ClusterSyncFailed events
4. Encrypt network traffic - Use a VPN or private network between cluster members if they span data centers

Maintenance

1. Upgrade secondaries first - Always upgrade secondary servers before the primary
2. Test after upgrades - Verify cluster synchronization works after every upgrade
3. Keep versions matched - All servers in a cluster must run the same version
4. Schedule maintenance windows - Coordinate server restarts to minimize client impact
5. Document your cluster - Record server roles, addresses, and admin credentials securely

Monitoring

1. Automate health checks - Use triggers or external monitoring to detect sync failures
2. Review logs regularly - Check for synchronization warnings or errors
3. Monitor network latency - High latency between servers can cause sync delays
4. Track server status - Periodically verify all servers are enabled and responsive

Quick Reference

Server Management Commands

# List servers in the cluster
completeftp server list

# Show server details
completeftp server show <serverName>

# Add a server to the cluster
completeftp server add <localUserName> <localPassword> <remoteAddress> <remotePort> <remoteUserName> <remotePassword>

# Modify server settings
completeftp server set <serverName> property=value

# Remove a server from the cluster
completeftp server remove <serverName>

Server Add Parameters

localUserName    = Admin username on the local server
localPassword    = Admin password on the local server
remoteAddress    = IP or hostname of the remote server
remotePort       = Admin port on the remote server (typically 14983)
remoteUserName   = Admin username on the remote server
remotePassword   = Admin password on the remote server

Cluster Events

ClusterSyncFailed  = Fired when configuration sync between servers fails

Key Ports

14983  = Admin/cluster synchronization port (default)