Quick Start - Direct
Quick start for FoxMQ by directly invoking the binary
The FoxMQ binary can be invoked directly without any installation steps.
Step 1: Grab the Latest Release
First, grab the zip file of the latest release of FoxMQ for your target platform from our public Releases page on Github.
As of writing, binaries are available for the following platforms:
Linux on x86-64 (AMD64)
GNU C Library (glibc) version 2.31 or greater is recommended.
macOS on x86-64 (AMD64) or 64-bit ARM
Note: universal binary, one executable supports both architectures.
Latest version of macOS is recommended.
Windows on x86-64 (AMD64)
Windows 10/Windows Server 2016 or later recommended.
Extract the foxmq
executable (foxmq.exe
on Windows) to a directory of your choosing.
Example (Linux): Fetch and Unzip FoxMQ
Step 2: Generate an Address Book
For this initial release of FoxMQ, the topography of nodes in the cluster must be decided up-front. The underlying Tashi consensus algorithm supports dynamic membership, so dynamic scaling of a cluster may be added in a future release.
There is no minimum number of nodes, but 4 or more is required for high availability. Because the consensus algorithm requires a supermajority (2/3 rounded down + 1), the cluster can tolerate losing up to (but not including) 1/3 of the nodes. For example, the following table shows the failure tolerance for a few different topologies:
Notice how the tolerance increases after every multiple of 3 (size mod 3 = 1
). Increasing the cluster size does not inherently improve failure tolerance until passing the next multiple of 3. Thus, a cluster size on the order of 3N + 1
(4, 7, 10, 13, etc.) is a good choice for a highly available configuration.
Put simply, use a cluster size 3N + 1
where N
is the number of failures the cluster should tolerate.
Having more nodes failed than the number tolerated for a given configuration just means that consensus may stall; local message dispatch on a still-functional node will continue to function. If the failed nodes return to operating status, the cluster should recover.
Decide how many nodes you will want in your cluster, and then choose an IP address and UDP port pair for each (how to do so depends on the details of your deployment). These will be the addresses that the FoxMQ instances will use to talk to each other (corresponding to --cluster-addr
or a socket address that connects to it).
You can then use the foxmq address-book
subcommand to generate ECDSA keys for each node as well as the address book containing knowledge of the whole network. This subcommand has two options for supplying the list of addresses.
Note: FoxMQ only supports IPv4 addresses currently.
Option 1: Explicit List of Addresses
If each node will be on a different machine with a different IP address, you can specify them individually.
For example, let's say we have three different nodes on a LAN. Each one will have a different IP address, but we can use the same port on all of them for consistency; FoxMQ defaults to listening for cluster connections on port 19793
. (MQTT connections default to the standard ports: 1883
for unencrypted connections, 8883
for TLS).
Example
We might have a list of addresses like so:
192.168.0.11
192.168.0.13
192.168.0.21
We can run the following command to generate keys and an address book:
Linux/macOS:
Windows:
Running either of the above commands will generate a foxmq.d/
subdirectory containing our key files (key_0.pem
, key_1.pem
, key_2.pem
) and our address book file (address-book.toml
).
We will use these in the next step.
Option 2: Base Address and Port Range
This option is handy if you're testing clustered mode on a single machine, or you're generating an address book for a set of Docker containers.
Instead of separate IP addresses, you can provide a single IP address and a start and end port (inclusive), and the address-book
subcommand will generate a key and address book entry for each port in the range:
Example: Testing on Loopback
Say you want to run a FoxMQ cluster on the local machine. You can provide the loopback address 127.0.0.1
and then a port range for the number of instances you want to run.
Let's say we'll use three instances again; we'll specify our start port as 19793
(the ASCII string MQ
as a 16-bit integer) and our end port as 19795
such that we end up with three entries:
127.0.0.1:19793
127.0.0.1:19794
127.0.0.1:19795
Linux/macOS:
Windows:
Just like Option 1, we should end up with a new foxmq.d/
directory containing our key files (key_0.pem
, key_1.pem
, key_2.pem
) and our address book file (address-book.toml
).
Step 3: Create User Credentials
For security, FoxMQ does not allow anonymous MQTT connections by default. You must either explicitly allow anonymous connections (see below), or configure at least 1 user login, or foxmq
will exit with an error.
Simply run ./foxmq user add
(Linux/macOS) or foxmq.exe user add
(Windows) and follow the prompts to enter a username and password (the command will generate a password for you if you don't enter one).
When the command finishes, it will create foxmq.d/users.toml
containing the user credentials. To add another user, simply run the command again.
Don't worry about the password; it's only stored in its hashed form and cannot be recovered.
To change the password for a user, simply delete the corresponding entry from foxmq.d/users.toml
and run ./foxmq user add
again.
Step 4: Start FoxMQ
Copy the foxmq.d/
directory and foxmq
(or foxmq.exe
) executable to each node (a distinct machine or otherwise) where you'll be running FoxMQ.
If you're just testing FoxMQ locally, you can just run all the instances in the same working directory.
For security, you should only copy the key_N.pem
file appropriate to a given node, where N
is the Nth index in the list or range of addresses, starting from zero. So copy only key_0.pem
to the first node, key_1.pem
to the second, etc.
They should all get foxmq.d/address-book.toml
however.
Whether or not to copy foxmq.d/users.toml
depends on whether you want to allow the same users to log in to different nodes or not. For a multi-party cluster, this may be undesirable; however, if you are using FoxMQ in clustered mode for high availability, you will likely want to have the same users file on all nodes.
Then, simply execute ./foxmq run
(or foxmq.exe run
) with the path to the appropriate key_N.pem
file. You can either explicitly specify the path to the foxmq.d/
directory or leave it off if it exists in the current working directory.
Example
Run one of the following commands for each node, replacing N
with the appropriate index for the node (0
for the first node, 1
for the second, etc).
FoxMQ will look for the foxmq.d/
subfolder in the current working directory to find the address-book.toml
and users.toml
.
Linux/macOS:
Windows:
You should now be able to connect using any MQTT v5 client on port 1883.
Optional: Allow Anonymous Logins
Add --allow-anonymous-login
to above command if you wish to allow MQTT clients to connect anonymously.
Alternatively, create foxmq.d/users.toml
with the following contents to allow anonymous logins by default:
Optional: Enable Additional Logging
FoxMQ only logs errors by default.
To enable additional logging, set the environment variable RUST_LOG=foxmq=info
.
Optional: Override the Listen Addresses
By default, FoxMQ opens two sockets to listen for connections:
For connections from MQTT clients, FoxMQ listens on the standard TCP port 1883 on all interfaces (
0.0.0.0:1883
)For cluster connections from other FoxMQ brokers in the address book, FoxMQ listens on UDP port 19793 on all interfaces (
0.0.0.0:19793
).
You can override either of these addresses if you want to. This may be necessary if you wish to override either port due to it already being in use, or to listen on only a specific network interface.
To override the address that FoxMQ listens for MQTT connections on,
pass --mqtt-addr=<address>:<port>
to the foxmq run
command.
To override the address that FoxMQ listens for cluster connections from other brokers on,
pass --cluster-addr=<address>:<port>
to the foxmq run
command.
If you're running all the instances on the same machine, you'll want to remap the listen ports for every node after the first, e.g. use 1884/19794
for the second node, 1885
/19795 for the third, etc.
Example
When running multiple instances locally, you'll want to override the listen addresses for the second node and beyond, so they don't collide: Linux/macOS:
Windows:
Note: Windows and Loopback/Localhost
Windows does not connect traffic on the loopback interface (127.0.0.1
/localhost
) to connections listening on a 0.0.0.0
address (used by FoxMQ by default to listen on all network interfaces).
If you are using a loopback address on Windows, such as that shown in Option 2 of Step 2 above, you'll want to pass127.0.0.1:<port>
for --mqtt-addr
and/or --cluster-addr
.
Example (Windows):
Optional: Enable TLS (mqtts)
Pass --mqtts
to foxmq run
to listen for MQTT-over-TLS (mqtts
) on port 8883
(default; use --mqtts-addr
to change)
FoxMQ generates self-signed TLS certificates using the given secret key by default. Most TLS client implementations will likely not accept this as-is.
Option 1: Disable Certificate Verification
You can disable certificate verification on the client which will discard the protection against man-in-the-middle (MITM) attacks but will at least use an encrypted tunnel for traffic instead of communicating in the open.
For example, with the MQTT.js client you can pass rejectUnauthorized: false
as part of the options object:
Option 2: Generate Explicit Pre-Signed Certificates
You can generate pre-signed certificates for each node up-front and then pass them as part of the certificate authorities (CAs) array to your client. It will then be able to verify that its connection to FoxMQ is genuine.
FoxMQ will likely gain a way to automate this as part of address-book generation in the near future, but for now we can use the utilities that come with OpenSSL. (OpenSSL is installed on most Linux distributions by default; installing it on Windows or macOS is out of scope for this guide.)
Run the following command for each node, replacing N
with the 0-based index of the node (0
for the first node, 1
for the second, etc.)
Then, pass --tls-cert-file=foxmq.d/key_N.crt
to foxmq run
.
foxmq.local
is the default value FoxMQ uses for Server Name Identification (SNI). If your nodes will be available at a real domain name, you may substitute that here, and then pass the flag
--server-name=<domain>
to foxmq run
so that FoxMQ knows to report it during the TLS handshake.
Then in your client, load the certificate for each node you wish to connect to, and pass it as a certificate authority.
For example, using MQTT.js:
Optional: Enable WebSockets (ws)
As of release 0.2.1, FoxMQ supports MQTT over WebSockets (ws://
).
To enable MQTT over WebSockets, pass the --websockets
flag on the command line.
By default, FoxMQ listens for WebSocket connections on 0.0.0.0:8080
. To change this,
pass --websockets-addr=<address>
alongside--websockets
.
Supporting WebSockets allows FoxMQ to accept connections from MQTT clients in the browser, for example using MQTT.js:
Note that MQTT over WebSockets-over-TLS (wss://
) is not yet directly supported, which would cause problems in secure browser contexts (https://
pages, which are preferred by browsers by default for security) as insecure connections are not allowed.
However, you can still connect to FoxMQ over Websockets-over-TLS if you configure HTTPS/TLS termination through reverse-proxy software, like Nginx or a Kubernetes Ingress controller.
This works with FoxMQ because WebSockets is simply a special operating mode of the HTTP protocol, so the steps for enabling WebSockets-over-TLS are the same as enabling HTTPS.
Reverse-proxies are quite common in production deployments, as they're used for load balancing, and configuring TLS through a reverse-proxy can be more convenient than enabling it directly in the server software itself.
For example, see this tutorial for SSL with managed certificates in Google Kubernetes Engine. With the configuration shown in the tutorial, TLS certificates are issued for your application by Google's certificate authority and renewed automatically. You merely need a registered domain name for your application. This is provided at no additional cost. AWS and Azure Kubernetes Service have similar offerings.
By default, FoxMQ even answers WebSocket connections on the same port as shown in the tutorial: 8080
(TCP).
The path of the URL, if any (ws://host:port/path
) is ignored by FoxMQ in accordance with the MQTT specification. Any route besides /
(the default if not specified) will answer 404 Not Found
if a WebSockets upgrade request is not received. /
answers 200 Ok
to satisfy health checks.
Last updated