Quick Start - Direct

Quick start for FoxMQ by directly invoking the binary

The FoxMQ binary can be invoked directly without any installation steps.

Step 1: Grab the Latest Release

First, grab the zip file of the latest release of FoxMQ for your target platform from our public Releases page on Github.

As of writing, binaries are available for the following platforms:

• Linux on x86-64 (AMD64)

  • GNU C Library (glibc) version 2.31 or greater is recommended.

• macOS on x86-64 (AMD64) or 64-bit ARM

  • Note: universal binary, one executable supports both architectures.

  • Latest version of macOS is recommended.

• Windows on x86-64 (AMD64)

  • Windows 10/Windows Server 2016 or later recommended.

Extract the foxmq executable (foxmq.exe on Windows) to a directory of your choosing.

Example (Linux): Fetch and Unzip FoxMQ

wget https://github.com/tashigg/foxmq/releases/download/v0.1.0/foxmq_0.1.0_linux-amd64.zip
unzip https://github.com/tashigg/foxmq/releases/download/v0.1.0/foxmq_0.1.0_linux-amd64.zip

Step 2: Generate an Address Book

For this initial release of FoxMQ, the topography of nodes in the cluster must be decided up-front. The underlying Tashi consensus algorithm supports dynamic membership, so dynamic scaling of a cluster may be added in a future release.

There is no minimum number of nodes, but 4 or more is required for high availability. Because the consensus algorithm requires a supermajority (2/3 rounded down + 1), the cluster can tolerate losing up to (but not including) 1/3 of the nodes. For example, the following table shows the failure tolerance for a few different topologies:

# of Nodes	Supermajority (2N/3 + 1)	# of Failed Nodes Tolerated
1	1	0
2	2	0
3	3	0
4	3	1
5	4	1
6	5	1
7	5	2
8	6	2
9	7	2
10	7	3
20	14	6
100	67	33

Notice how the tolerance increases after every multiple of 3 (size mod 3 = 1). Increasing the cluster size does not inherently improve failure tolerance until passing the next multiple of 3. Thus, a cluster size on the order of 3N + 1 (4, 7, 10, 13, etc.) is a good choice for a highly available configuration.

Put simply, use a cluster size 3N + 1 where N is the number of failures the cluster should tolerate.

Having more nodes failed than the number tolerated for a given configuration just means that consensus may stall; local message dispatch on a still-functional node will continue to function. If the failed nodes return to operating status, the cluster should recover.

Decide how many nodes you will want in your cluster, and then choose an IP address and UDP port pair for each (how to do so depends on the details of your deployment). These will be the addresses that the FoxMQ instances will use to talk to each other (corresponding to --cluster-addr or a socket address that connects to it).

You can then use the foxmq address-book subcommand to generate ECDSA keys for each node as well as the address book containing knowledge of the whole network. This subcommand has two options for supplying the list of addresses.

Note: FoxMQ only supports IPv4 addresses currently.

Option 1: Explicit List of Addresses

If each node will be on a different machine with a different IP address, you can specify them individually.

For example, let's say we have three different nodes on a LAN. Each one will have a different IP address, but we can use the same port on all of them for consistency; FoxMQ defaults to listening for cluster connections on port 19793. (MQTT connections default to the standard ports: 1883 for unencrypted connections, 8883 for TLS).

Example

We might have a list of addresses like so:

• 192.168.0.11

• 192.168.0.13

• 192.168.0.21

We can run the following command to generate keys and an address book:

Linux/macOS:

./foxmq address-book from-list 192.168.0.11:19793 192.168.0.13:19793 192.168.0.21:19793

Windows:

foxmq.exe address-book from-list 192.168.0.11:19793 192.168.0.13:19793 192.168.0.21:19793

Running either of the above commands will generate a foxmq.d/ subdirectory containing our key files (key_0.pem, key_1.pem, key_2.pem) and our address book file (address-book.toml).

We will use these in the next step.

Option 2: Base Address and Port Range

This option is handy if you're testing clustered mode on a single machine, or you're generating an address book for a set of Docker containers.

Instead of separate IP addresses, you can provide a single IP address and a start and end port (inclusive), and the address-book subcommand will generate a key and address book entry for each port in the range:

Example: Testing on Loopback

Say you want to run a FoxMQ cluster on the local machine. You can provide the loopback address 127.0.0.1 and then a port range for the number of instances you want to run.

Let's say we'll use three instances again; we'll specify our start port as 19793 (the ASCII string MQ as a 16-bit integer) and our end port as 19795 such that we end up with three entries:

• 127.0.0.1:19793

• 127.0.0.1:19794

• 127.0.0.1:19795

Linux/macOS:

./foxmq address-book from-range 127.0.0.1 19793 19795

Windows:

foxmq.exe address-book from-range 127.0.0.1 19793 19795

Just like Option 1, we should end up with a new foxmq.d/ directory containing our key files (key_0.pem, key_1.pem, key_2.pem) and our address book file (address-book.toml).

Step 3: Create User Credentials

For security, FoxMQ does not allow anonymous MQTT connections by default. You must either explicitly allow anonymous connections (see below), or configure at least 1 user login, or foxmq will exit with an error.

Simply run ./foxmq user add (Linux/macOS) or foxmq.exe user add (Windows) and follow the prompts to enter a username and password (the command will generate a password for you if you don't enter one).

When the command finishes, it will create foxmq.d/users.toml containing the user credentials. To add another user, simply run the command again.

Don't worry about the password; it's only stored in its hashed form and cannot be recovered.

To change the password for a user, simply delete the corresponding entry from foxmq.d/users.toml and run ./foxmq user add again.

Step 4: Start FoxMQ

Copy the foxmq.d/ directory and foxmq (or foxmq.exe) executable to each node (a distinct machine or otherwise) where you'll be running FoxMQ.

If you're just testing FoxMQ locally, you can just run all the instances in the same working directory.

For security, you should only copy the key_N.pem file appropriate to a given node, where N is the Nth index in the list or range of addresses, starting from zero. So copy only key_0.pem to the first node, key_1.pem to the second, etc.

They should all get foxmq.d/address-book.toml however.

Whether or not to copy foxmq.d/users.toml depends on whether you want to allow the same users to log in to different nodes or not. For a multi-party cluster, this may be undesirable; however, if you are using FoxMQ in clustered mode for high availability, you will likely want to have the same users file on all nodes.

Then, simply execute ./foxmq run (or foxmq.exe run) with the path to the appropriate key_N.pem file. You can either explicitly specify the path to the foxmq.d/ directory or leave it off if it exists in the current working directory.

Example

Run one of the following commands for each node, replacing N with the appropriate index for the node (0 for the first node, 1 for the second, etc).

FoxMQ will look for the foxmq.d/ subfolder in the current working directory to find the address-book.toml and users.toml.

Linux/macOS:

./foxmq run --secret-key-file=foxmq.d/key_N.pem

Windows:

foxmq.exe run --secret-key-file=foxmq.d\key_N.pem

You should now be able to connect using any MQTT v5 client on port 1883.

Optional: Allow Anonymous Logins

Add --allow-anonymous-login to above command if you wish to allow MQTT clients to connect anonymously.

Alternatively, create foxmq.d/users.toml with the following contents to allow anonymous logins by default:

[auth]
allow-anonymous-login = true

Optional: Enable Additional Logging

FoxMQ only logs errors by default.

To enable additional logging, set the environment variable RUST_LOG=foxmq=info.

Optional: Override the Listen Addresses

By default, FoxMQ opens two sockets to listen for connections:

• For connections from MQTT clients, FoxMQ listens on the standard TCP port 1883 on all interfaces (0.0.0.0:1883)

• For cluster connections from other FoxMQ brokers in the address book, FoxMQ listens on UDP port 19793 on all interfaces (0.0.0.0:19793).

You can override either of these addresses if you want to. This may be necessary if you wish to override either port due to it already being in use, or to listen on only a specific network interface.

To override the address that FoxMQ listens for MQTT connections on, pass --mqtt-addr=<address>:<port> to the foxmq run command.

To override the address that FoxMQ listens for cluster connections from other brokers on, pass --cluster-addr=<address>:<port> to the foxmq run command.

If you're running all the instances on the same machine, you'll want to remap the listen ports for every node after the first, e.g. use 1884/19794 for the second node, 1885/19795 for the third, etc.

Example

When running multiple instances locally, you'll want to override the listen addresses for the second node and beyond, so they don't collide: Linux/macOS:

./foxmq run --mqtt-addr=0.0.0.0:1884 --cluster-addr=0.0.0.0:19794 --secret-key-file=foxmq.d/key_1.pem
./foxmq run --mqtt-addr=0.0.0.0:1885 --cluster-addr=0.0.0.0:19795 --secret-key-file=foxmq.d/key_2.pem

Windows:

foxmq.exe run --mqtt-addr=0.0.0.0:1884 --cluster-addr=0.0.0.0:19794 --secret-key-file=foxmq.d\key_1.pem
foxmq.exe run --mqtt-addr=0.0.0.0:1885 --cluster-addr=0.0.0.0:19795 --secret-key-file=foxmq.d\key_2.pem

Note: Windows and Loopback/Localhost

Windows does not connect traffic on the loopback interface (127.0.0.1/localhost) to connections listening on a 0.0.0.0 address (used by FoxMQ by default to listen on all network interfaces).

If you are using a loopback address on Windows, such as that shown in Option 2 of Step 2 above, you'll want to pass127.0.0.1:<port> for --mqtt-addr and/or --cluster-addr.

Example (Windows):

foxmq.exe run --secret-key-file=dmq\key_N.pem --mqtt-addr=127.0.0.1:<port> --cluster-addr=127.0.0.1:<port>

Optional: Enable MQTT over TLS (mqtts)

Pass --mqtts to foxmq run to listen for MQTT-over-TLS (mqtts) on port 8883 (default; use --mqtts-addr to change)

FoxMQ generates self-signed TLS certificates using the given secret key by default. Most TLS client implementations will likely not accept this as-is.

Option 1: Disable Certificate Verification

You can disable certificate verification on the client which will discard the protection against man-in-the-middle (MITM) attacks but will at least use an encrypted tunnel for traffic instead of communicating in the open.

For example, with the MQTT.js client you can pass rejectUnauthorized: false as part of the options object:

const mqtt = require("mqtt");
const client = await mqtt.connectAsync("mqtts://127.0.0.1:8883", { rejectUnauthorized: false });

Option 2: Generate Explicit Pre-Signed Certificates

You can generate pre-signed certificates for each node up-front and then pass them as part of the certificate authorities (CAs) array to your client. It will then be able to verify that its connection to FoxMQ is genuine.

FoxMQ will likely gain a way to automate this as part of address-book generation in the near future, but for now we can use the utilities that come with OpenSSL. (OpenSSL is installed on most Linux distributions by default; installing it on Windows or macOS is out of scope for this guide.)

Run the following command for each node, replacing N with the 0-based index of the node (0 for the first node, 1 for the second, etc.)

openssl req -x509 -sha256 -days 3650 \
  -nodes -keyin foxmq.d/key_N.pem -out foxmq.d/key_N.crt -subj "/CN=foxmq.local" \
  -addext "subjectAltName=DNS:foxmq.local"

Then, pass --tls-cert-file=foxmq.d/key_N.crt to foxmq run.

foxmq.local is the default value FoxMQ uses for Server Name Identification (SNI). If your nodes will be available at a real domain name, you may substitute that here, and then pass the flag --server-name=<domain> to foxmq run so that FoxMQ knows to report it during the TLS handshake.

Then in your client, load the certificate for each node you wish to connect to, and pass it as a certificate authority.

For example, using MQTT.js:

const fs = require("fs").promises;
const mqtt = require("mqtt");

const cert0 = await fs.readFile("foxmq.d/key_0.crt");
const cert1 = await fs.readFile("foxmq.d/key_1.crt");
const cert2 = await fs.readFile("foxmq.d/key_2.crt");

const options = {
    ca: [cert0, cert1, cert2],
    servername: "foxmq.local"
};

const client = await mqtt.connectAsync("mqtts://127.0.0.1:8883", options);