flashmq.conf (5)

By default, the flashmq daemon expects to find its configuration file at /etc/flashmq/flashmq.conf, but this can be overriden using the --config-file command-line argument; see the flashmq(1) man page for details.

Using the include_dir parameter in your config file, you can load all the *.conf files from that given directory.

To set a parameter, its name must appear on a single line, followed by one or more potentially quoted arguments.

parameter-name1 parameter-value
parameter-name2 'value with spaces'
parameter-name3 "escaped \" quote"
multi-value-param one 'two' "three" "with ' char"
      

Quoted values are the same as unquoted values when they don't need it. They are necessary for when argument values have spaces, for instance.

When setting boolean values, yes/no, true/false and on/off can all be used.

To configure the listeners, use listen blocks, defined by { and }. See EXAMPLE LISTENERS for details.

Lines beginning with the hash character (“#”) and empty lines are ignored. Thus, a line can be commented out by prepending a “#” to it.

plugin /path/to/plugin.so#

FlashMQ supports an ELF shared object (.so file) plugin interface to add functionality, authorization and authentication, because it’s hard to provide a declarative mechanism that works for everybody. See flashmq_plugin.h for the API and its documentation. It’s written in C++ for ease of passing FlashMQ internals without conversion to C, but you can basically just use a C++ compiler and program like it was C; the C++ constructs are simple.

FlashMQ will auto-detect which plugin interface you’re trying to load (Mosquitto version 2 or FlashMQ native). Keep in mind that each thread initializes the plugin, inline with multi-core programming (minimize shared data and interaction between threads). You could use static variables with thread synchronization if you really want to. And of course, any Mosquitto plugin that uses global and/or static variables instead of initializing memory in its init() method, will not be thread-safe and won’t work.

You can only have one plugin active, but you can combine it with mosquitto_password_file and mosquitto_acl_file. The password and ACL file take precedence, and on a ‘deny’, will not ask the plugin.

plugin_opt_* value#

Options passed to the plugin init() function.

plugin_serialize_init true/false#

There could be circumstances where the plugin code is mostly thread-safe, but not on initialization. Libmysqlclient for instance, needs a one-time initialization. To add to the confusion, Qt hides that away.

The plugin should preferrably be written with proper synchronization like that, but as a last resort, you can use this to synchronize initialization.

Default value: false

plugin_serialize_auth_checks true/false#

Like plugin_serialize_init, but then for all login and ACL checks.

This option may be dropped at some point, because it negates much of the multi-core design. One may as well run with only one thread then.

Default value: false

plugin_timer_period seconds#

The FlashMQ auth plugin interface has a function that is called periodically this amount of seconds. This can be used to refresh state, commit data, etc.

See flashmq_plugin.h for details.

Default value: 60

log_file /path/to/flashmq.log#

This configuration parameter sets the path to FlashMQ's log file. If you omit this option from the config file, the output will go to stdout.

log_level info/notice/warning/error/none#

Set the log level to specified level and above. That means notice will log notice, warning and error.

Use this setting over the deprecated log_debug and quiet. If you do have those directives, they override the log_level, for backwards compatability reasons.

Default value: info.

log_debug true/false#

Debug logging obviously creates a lot of log noise, so should only be done to diagnose problems.

Deprecated. Use log_level instead.

Default value: false

log_subscriptions true/false#

Default value: false

allow_unsafe_clientid_chars true/false#

If you have topics with client IDs in it, people can possibly manipulate your ACL checking by saying their client ID is 'John+foobar'. Audit your security before you allow this.

Default value: false

allow_unsafe_username_chars true/false#

If you have topics with usernames in it, people can possibly manipulate your ACL checking by saying their username is 'John+foobar'. Audit your security before you allow this.

Default value: false

max_packet_size bytes#

MQTT packets have a maximum size of about 256 MB. This memory will (temporarily) be allocated upon arrival of such packets, so there may be cause to set it lower.

This option works in conjunction with client_max_write_buffer_size to limit memory use.

Default value: 268435461

client_max_write_buffer_size bytes#

The client's write buffer is where packets are stored before the event loop has the chance to flush them out. Any time a client's connection is bad and bytes can't be flushed, this buffer fills. So, there's good reason to limit this to something sensible. A good indication is your average packet size (or better yet, a configured max_packet_size) multiplied by the amount of packets you want to be able to buffer.

Note that it's an approximate value and not a hard limit. Buffer sizes only grow by powers of two, and buffers are always allowed to grow to make place for ping packets. Additionally, upon arrival of large packets (up to max_packet_size bytes), room will be made up to twice their size. So, you may also want to reduce max_packet_size from the default.

Default value is 1048576 (1 MB)

client_initial_buffer_size bytes#

The buffers for reading and writing, also for websockets when relevant, start out with a particular size and double when they need to grow. If you know your clients send bulks of a particular size, it helps to set this to match, to avoid constant memory reallocation. The default value is set conservatively, for scenario's with millions of clients.

After buffers have grown, they are eventually reset to their original size when possible.

Also see client_max_write_buffer_size and max_packet_size.

Value must be a power of two.

Default value: 1024

mosquitto_password_file /foo/bar/mosquitto_password_file#

File with usernames and hashed+salted passwords as generated by Mosquitto's mosquitto_passwd.

Mosquitto up to version 1.6 uses the sha512 algorithm. Newer version use sha512-pbkdf2. Both are supported.

mosquitto_acl_file /foo/bar/mosquitto_acl_file#

ACL (access control lists) for users, anonymous users and patterns expandable with %u (username) and %c (clientid). Format is Mosquitto's acl_file.

allow_anonymous true/false#

This option can be overriden on a per-listener basis; see listener.allow_anonymous.

Default value: false

zero_byte_username_is_anonymous true/false#

The proper way to signal an anonymous client is by setting the 'username present' flag in the CONNECT packet to 0, which in MQTT3 also demands the absence of a password. However, there are also clients out there that set the 'username present' flag to 1 and then give an empty username. This is an undesirable situation, because it means there are two ways to identify an anonymous client.

Anonymous clients are not authenticated against a loaded plugin when allow_anonymous is true. With this option enabled, that means users with empty string as usernames also aren't.

With this option disabled, clients connecting with an empty username will be reject with 'bad username or password' as MQTT error code.

The default is to be unambigious, but this can be overridden with this option.

Default value: false

rlimit_nofile number#

The general Linux default of 1024 can be overridden. Note: systemd blocks you from setting it, so it needs to be set on the unit. The default systemd unit file sets LimitNOFILE=infinity. You may also need to set sysctl -w fs.file-max=10000000

Default value: 1000000

expire_sessions_after_seconds seconds#

Expire sessions after this time. Setting to 0 disables it and is (MQTT3) standard-compliant. But, existing sessions cause load on the server (because they cost memory and are still subscribers), so keeping sessions after any client that connects with a random ID doesn't make sense.

Default value: 1209600

quiet true/false#

Don't log LOG_INFO and LOG_NOTICE. This is useful when you have a lot of foot traffic, because otherwise the log gets filled with connect/disconnect notices.

Deprecated. Use log_level instead.

Default value: false

storage_dir /path/to/dir#

Location to store sessions, subscriptions and retained messages. Not specifying this will turn off persistence.

save_state_interval seconds#

The interval at which the state is saved, if enabled with storage_dir.

This setting is currently not applied on reload.

Default: 3623

max_qos_msg_pending_per_client

max_qos_bytes_pending_per_client#

There is a limit to how many QoS packets can be stored in a session, so you can define a maximum amount of messages and bytes. If any of these is exceeded, the packet is dropped.

Defaults:

• max_qos_msg_pending_per_client 512

• max_qos_bytes_pending_per_client 65536

max_incoming_topic_alias_value number#

Is communicated towards MQTT5 clients. It is then up to them to decide to set them or not.

Changing this setting and reloading the config only has effect on new clients, because existing clients would otherwise exceed the limit they think applies.

Default value: 65535

max_outgoing_topic_alias_value number#

FlashMQ will make this many aliases per MQTT5 client, if they ask for aliases (with the connect property TopicAliasMaximum).

Default value: 65535

thread_count number#

If you want to have a different amount of worker threads then CPUs, you can set this value. Typically you don't need to set this.

Default value: auto-detect

wills_enabled true/false#

When disabled, the server will not set last will and testament specified by connecting clients.

Default value: true

retained_messages_mode enabled/enabled_without_persistence/downgrade/drop/disconnect_with_error#

Retained messages can be a strain on the server you may not need. You can set various ways of dealing with them:

enabled. This is normal operation.

enabled_without_persistence. Like 'normal', except it won't store them to disk if storage_dir is defined.

enabled_without_retaining. This somewhat counter-intuitive sounding mode is like downgrade, except that the 'retain' flag is not removed. This allows MQTT5 subscribers that subscribe with 'retain as published' to see which messages were originally sent as retained. It's just that FlashMQ won't retain them.

downgrade. The retain flag is removed and treated like a normal publish.

drop. Messages with retain set are dropped.

disconnect_with_error. Disconnect clients who try to set them.

Default value: enabled

expire_retained_messages_after_seconds seconds#

Use this to limit the life time of retained messages. Without this, the amount of retained messages may never decrease.

Default value: 4294967296

retained_messages_delivery_limit number#

Deprecated.

retained_messages_node_limit number#

When clients place a subscription, they will get the retained messages matching that subscription. Even though traversing the retained message tree is deprioritized in favor of other traffic, it will still cause CPU load until it's done. If you have a tree with millions of nodes and clients subscribe to `#`, this is potentially unwanted. You can use this setting to limit how many nodes of the retrained tree are traversed.

Note that the topic `one/two/three` is three nodes, and each node doesn't necessarilly need to contain a message.

Default value: 4294967296

set_retained_message_defer_timeout milliseconds#

The time after which FlashMQ will fall back to (b)locking vs queued mode for setting retained messages. 0, the default, disables queued mode altogether. It's disabled by default because it can incur some extra CPU and memory overhead.

Each retained message lives in a node in a tree. The topic 'one/two/three' is three nodes. When a node in that tree does not exist yet, it needs to be created. This requires a write lock on the tree. At this point, other threads reading from or writing to the retained message tree need to wait. This can cause a compounding blocking effect, especially if many threads do it at once.

This feature is to favor server responsiveness vs the speed at which retained messages become available in the server. It is primarily useful for when you have a lot of retained messages on different/changing topics. If at first a retained message can't be set, the action to do so will be retried in the event loop, asynchronously.

This setting determines the maximum amount of time to defer setting a retained message, after which it will fall back to using locks.

Also see set_retained_message_defer_timeout_spread

Default value: 0

set_retained_message_defer_timeout_spread milliseconds#

For set_retained_message_defer_timeout, the amount of random spread between 0 and this value for the timeout. This spreads out locking over time, reducing contention.

Default value: 1000

retained_message_node_lifetime seconds#

The grace period after which a retained message node is eligible for deletion. The topic 'one/two/three' is three nodes, and if that topic had a message, it would be contained in 'three'.

FlashMQ will periodically clear out retained message nodes that have no message anymore. This is required to save memory. But, when you receive retained messages on the same topics repeatedly, it may be beneficial to keep the nodes around, to avoid the need for locks to recreate them. If you know that retained messages come and go within a certain period, it's benificial to set this value so that no unnecessary node destruction and creation takes place.

Default value: 0

subscription_node_lifetime seconds#

The grace period after which a subscription node is eligible for deletion. The subscription 'one/two/three' is three nodes.

FlashMQ will periodically clear our nodes in the subscription tree that have no entries anymore. This is required to save memory. But, when clients place the same subscriptions repeatedly, it may be beneficial to keep the nodes around, to avoid the need for locks to recreate them. If you know that certain subscription patterns come and go within a certain period, it's benificial to set this value so that no unnecessary node destruction and creation takes place.

Default value: 3600

websocket_set_real_ip_from inet4_address/inet6_address#

HTTP proxies in front of the websocket listeners can set the X-Real-IP header to identify the original connecting client. With websocket_set_real_ip_from you can mark IP networks as trusted. By default, clients are not trusted, to avoid spoofing.

You can repeat the option to allow for multiple addresses. Valid notations are 1.2.3.4, 1.2.3.4/16, 1.2.0.0/16, 2a01:1337::1, 2a01:1337::1/64, etc.

The header X-Forwarded-For is not used, because that's designed to contain a list of addresses, if applicable.

As a side note about using a proxy on your listener; you can only have an absolute max of 65535 connections to an IP+port combination (and the practical limit is lower). If you need more, you have to set up multiple listeners. This can be multiple IP addresses, or simply multiple ports.

shared_subscription_targeting round_robin/sender_hash#

When having multiple subscribers on a shared subscription (like '$share/myshare/jane/doe'), select how the messages should be distributed over the subscribers.

round_robin. Select the next subscriber for each message. There is still some amount of randomness to it because the counter for this is not thread safe. Using an atomic/mutexed counter for it would just be too slow to justify.

sender_hash. Selects a receiver deterministically based on the hash of the client ID of the sender. The selected subscriber will depend on how many subscribers there are, so if some disconnect, the distribution will change. Moreover, the selection may also change when FlashMQ cleans up empty spaces in the list of shared subscribers.

Default: round_robin

minimum_wildcard_subscription_depth number#

Defines the minimum level of the first wildcard topic filter (# and +). In a topic filter like sensors/temperature/#, that is 2. If you specify 2, a subscription to sensors/# will be denied. Remember that only MQTT 3.1.1 and newer actually notify the client of the denial in the sub-ack packet.

The reason you may want to limit it, is performance. If you have a base message load of 100,000 messages per second, each client subscribing to # causes that many permission checks per second. If you have 100 clients doing that, there will be 10 million permission checks per second.

Default: 0

wildcard_subscription_deny_mode deny_all/deny_retained_only#

For minimum_wildcard_subscription_depth, specify what you want to deny. Trying to give a client all retained messages can cause quite some load, so only denying the retained messages upon receiving a broad wildcard subscription can be useful if you have a low enough general message volume, but a high number of retained messages.

Default: deny_all

overload_mode log/close_new_clients#

Define the action to perform when the value defined with max_event_loop_drift is exceeded.

When a server is (re)started, and hundreds of thousands of clients connect, the SSL handshaking and authenticating can be so heavy that it doesn't get to clients in time. They will then reconnect and try again, and get stuck in a loop. This option is to mitigate that. With close_new_clients, new clients will be closed immediately after connecting while the server is overloaded. This will allow the worker threads to process the new clients in a controlled manner.

For really large deployments, this can be augmented with extra rate limiting in iptables, or other firewalls. A stateless method is preferred, like: iptables -I INPUT -p tcp -m multiport --dports 8883,1883 --syn -m hashlimit --hashlimit-name newmqttconns --hashlimit-above 10000/second --hashlimit-burst 15000 -j DROP

The current default is log, but that will likely change in the future.

Default: log

max_event_loop_drift milliseconds#

For overload_mode, the maximum permissible thread drift before the overload action is taken.

The drift values considered are those of the main loop, in which clients are accepted, and the median of all worker threads.

Default: 2000

include_dir /path/to/dir#

Load *.conf files from the specified directory, to merge with the main configuration file.

An error is generated when the directory is not there. This is to protect against running incorrect configurations by accident, when the dir has been renamed, for example.

Listen parameters can only be used within listen { } blocks.

port#

The default port depends on the protocol parameter and whether or not fullchain and privkey parameters are supplied:

• For unencrypted MQTT, the default port is 1883

• For encrypted MQTT, the default port is 8883

• For plain HTTP websockets, the default port is 8080

• For encrypted HTTPS websockets, the default port is 4443

protocol#

Valid values: mqtt , websockets

inet_protocol#

Valid values: ip4_ip6 , ip4 , ip6

Default: ip4_ip6

inet4_bind_address inet4address#

Default: 0.0.0.0

inet6_bind_address inet6address#

Default: ::0

fullchain /foobar/server.crt#

Specifying a chain makes the listener SSL, and also requires the privkey to be set.

privkey /foobar/server.key#

Specifying a private key makes the listener SSL, and also requires the fullchain to be set.

client_verification_ca_file /foobar/client_authority.crt#

Clients can be authenticated using X509 certificates, and the username taken from the CN (common name) field. Use this directive to specify the certificate authority you trust.

Specifying this or client_verification_ca_dir will require the listener to be TLS.

client_verification_ca_dir /foobar/dir_with_certificates#

Clients can be authenticated using X509 certificates, and the username taken from the CN (common name) field. Use this directive to specify the dir containing certificate authorities you trust.

Note that the filename requirements are dictated by OpenSSL. Use the utility openssl rehash /path/to/dir.

Specifying this or client_verification_ca_file will require the listener to be TLS.

client_verification_still_do_authn true/false#

When using X509 client authentication with client_verification_ca_file or client_verification_ca_dir, the username will not be checked with a user database or a plugin by default. Set this option to true to override that.

allow_anonymous true/false#

This allows you to override the global allow_anonymous setting on the listener level.

haproxy true/false#

Setting the listener to haproxy makes it expect the PROXY protocol and set client source address to the original client. Make sure this listener is private / firewalled, otherwise anybody can set a different source address.

Note that HAProxy's server health checks only started using the 'local' specifier as of version 2.4. This means earlier version will pretend to be a client and break the connection, causing log spam.

As a side note about using a proxy on your listener; you can only have an absolute max of 65535 connections to an IP+port combination (and the practical limit is lower). If you need more, you have to set up multiple listeners. This can be multiple IP addresses, or simply multiple ports.

See haproxy.org.

tcp_nodelay true/false#

tcp_nodelay will cause the TCP_NODELAY option to be set for the listener's socket(s), and therefore for all clients accepted on that listener.

TCP_NODELAY is a OS TCP-layer option that will cause messages written by FlashMQ to the socket to be flushed immediately, without letting Nagle's algorithm (the default) collect small outgoing TCP packets into bigger packets.

Foregoing Nagle's algorithm by setting tcp_nodelay to true may decrease latency, at the likely cost of some network efficiency.

Default: false

listen {
  protocol mqtt
  inet_protocol ip4_ip6
  inet4_bind_address 127.0.0.1
  inet6_bind_address ::1
  fullchain /foobar/server.crt
  privkey /foobar/server.key

  # default = 8883
  port 8883
}
listen {
  protocol mqtt
  fullchain /foobar/server.crt
  privkey /foobar/server.key
  client_verification_ca_file /foobar/client_authority.crt
  client_verification_still_do_authn false
}
listen {
  protocol mqtt
  inet_protocol ip4

  # default = 1883
  port 1883
}
listen {
  protocol websockets
  fullchain /foobar/server.crt
  privkey /foobar/server.key

  # default = 4443
  port 4443
}
listen {
  protocol websockets

  # default = 8080
  port 8080
}
listen {
  port 2883
  haproxy on
}

Bridges can be defined inside bridge { } blocks. A bridge is essentially just an outgoing connection to another server with loop-detection and retain flag relaying. It is not a form of clustering. Also note that one bridge is one connection, and because FlashMQ's threading model is that clients are serviced by one selected thread only, a bridge has the potential to saturate a thread, if it's heavily loaded. You could work around that by defining multiple bridges to the same server, for various topic paths. A future version of FlashMQ will likely improve upon this.

Bridges are dynamically created, removed or changed upon config reload. When a bridge configuration changes, it will disconnect and reconnect.

address address#

The DNS name, IPv4 or IPv6 address of the server you want to connect to.

port number#

The default port depends on the tls option, either 1883 or 8883.

inet_protocol ip4_ip6/ip4/ip6#

Default: ip4_ip6

tls off/on/unverified#

Set TLS mode. The value unverified means the x509 chain is not verified.

ca_file path#

File to be used for x509 certificate chain validation.

ca_dir path#

Directory containing certificates for x509 certificate chain validation.

protocol_version mqtt3.1/mqtt3.1.1/mqtt5#

Default: mqtt3.1.1

bridge_protocol_bit true/false#

An unofficial standard is to set the most significant bit of the protocol version byte to 1 to signal the connection is a bridge. This allows the other side to alter its behavior slightly. However, this is not always supported, so you can disable this if you get disconnected for reporting an invalid protocol version.

This setting has no effect when using MQTT5, because the behavior it influences is done with subscription options.

Default: true

keepalive seconds#

The time between sending ping packets to the other side.

Default: 60

clientid_prefix prefix#

The prefix of the randomly generated client ID. Client IDs cannot be explicitely set for security reasons. See Understanding clean session and clean start.

Default: fmqbridge

publish filter qos#

Messages matching this filter will be published to the other side. Examples: # or sport/tennis/#. This option can be repeated several times.

The QoS value should be seen as the QoS value of the internal subscription causing outgoing messages. Messages that are relayed have this QoS level at most.

Default: 0

subscribe filter qos#

Subscriptions for this filter is placed at the other side. Examples: # or sport/tennis/#. This option can be repeated several times.

The QoS value is like any subscription at a server. Messages received by the other end will be given this QoS level at most.

Default: 0

local_username username#

Username as seen by the local FlashMQ's plugin or ACL checks. This is not always necessary.

remote_username username#

Username sent to the remote connection.

remote_password password#

Password sent to the remote connection.

remote_clean_start true/false#

In MQTT3, this means 'clean session', meaning the remote server removes any existing session with the same ID on (re)connect, and destroys it immediately on disconnect. If you want reuseable sessions that survive disconnects, set this to false. If you also want to pick up remote sessions on FlashMQ restart, set use_saved_clientid to true.

In MQTT5, this option only influences reconnection behavior. It essentially has no effect on the first connect, because the client ID is random and will always be new (except when you set use_saved_clientid). But when set to true, any reconnects, which do use the already generated client ID, will destroy the session and in-flight messages will be lost.

Also see understanding clean session and clean start.

Default value: true

local_clean_start true/false#

In MQTT3 mode, this means 'clean session' and means the session is removed upon disconnect. If you want to reuse sessions on reconnect, set this to false. Any new start of FlashMQ will give you a new client ID so will always be a fresh session, except if you set use_saved_clientid.

In MQTT5 mode, this has no effect. If you want the session to be removed immediately on disconnect, use local_session_expiry_interval to 0.

Also see understanding clean session and clean start.

Default value: true

remote_session_expiry_interval seconds#

Is only used in MQTT5 mode and determines the amount of seconds after which the session can be removed from the remote server.

Default value: 0

local_session_expiry_interval seconds#

Determines when a local session without an active client will be removed, in both MQTT3 and MQTT5 mode. Note that in MQTT3 mode, the session is removed on disconnect when local_clean_start is true.

Default value: 0

remote_retain_available true/false#

MQTT5 allows a server to tell a client it doesn't support retained messages, or has it disabled. When using MQTT3, use this option to achieve the same.

Messages will not be relayed with 'retained as published' and the retained messages that are normally sent on matching subscription, are not sent.

Default value: true

use_saved_clientid true/false#

When you want your bridges to resume local and remote sessions after restart, set this to true and set remote_clean_start, local_clean_start, remote_session_expiry_interval and local_session_expiry_interval accordingly. It only has effect when you have set a storage_dir.

It is important to fully understand the clean session / clean start behavior and the role the client ID plays in that. The primary goal of sessions is to survive link disconnects. Configuring a fixed client ID and use that each time an MQTT client starts, is often an anti-pattern, because most clients like actual IoT devices start fresh upon restart and don't store their sessions (with in-flight packets, etc) to disk. FlashMQ does store it on disk however, so it can be used legitamately. However, you can run into unexpected situations. For instance, you will get your existing subscriptions from the session too. So, if you remove a subscribe line from your bridge configuration and restart, it will actually have no effect, because the server on the other side still has that subscription in the session.

See understanding clean session and clean start for details.

Default value: false

max_outgoing_topic_aliases amount#

If you want FlashMQ to initiate topic aliases for this bridge, set this to a non-zero value. Note that it's floored to the value the remote side gives in the CONNACK packet, so it only works if the other side permits it.

Default: 0

max_incoming_topic_aliases amount#

If you want to accept topic aliases for this bridge, set this to a non-zero value. The value is set in the CONNECT packet to inform the remote side of the wish. It's not guaranteed that the other side will actually make aliases.

Default: 0

tcp_nodelay true/false#

tcp_nodelay will cause the TCP_NODELAY option to be set for the client socket that is used to connect to the other end of the bridge.

See the documentation for the tcp_nodelay listener parameter for further elaboration.

Default: false

bridge {
    address demo.flashmq.org
    publish send/this
    subscribe receive/this
    local_username my_local_user
    remote_username my_remote_user
    remote_password my_remote_pass
    bridge_protocol_bit false
    tls on
    ca_file /path/to/ca.crt
}

Wiebe Cazemier contact@flashmq.org.

flashmq(1)

https://www.flashmq.org/

The sources for the FlashMQ manual pages are maintained in DocBook 5.2 XML files. The transformation to the multiple destination file formats is done using a bunch of XSLT 1.0 sheets, contributed to this project by Rowan van der Molen.