The overall architecture of all M-Switch products and the primary processes that comprise M-Switch are described below.
The (on disk) message queue is the centre of M-Switch. Arriving messages are written to the disk and secured, using fsync() to ensure no possibility of message loss, which is central to reliable store and forward messaging. Messages may then be checked and processed locally, before being sent on and removed from the disk. Typically, messages will remain in the queue for a very short while, often for just a few milliseconds.
Most M-Switch processes are channels. The exceptions are QMGR (see below) and three processes associated with the audit database. Channels perform many functions, including bringing messages into M-Switch, delivering or transferring messages out of M-Switch, and processing messages within M-Switch.
Queue Manager (QMGR)
QMGR is the operational core of M-Switch, communicating with channels and with management interfaces. Key QMGR tasks:
- Starting and stopping channel processes.
- Scheduling active processing and delivery of messages.
- Controlling number of connections to the SMTP Server (for M-Switch SMTP) and to the X.400 P1 server and X.400 P3 Client channel (for M-Switch X.400).
- Receiving status information from channels, on connections and message transfer status.
- Responding to information and control requests from management processes.
- Ensuring that higher priority messages are handled first.
To achieve this, QMGR makes use of three protocols:
- An internal channel communication protocol.
- AgentX to enable SNMP monitoring.
- SOM (Switch Operational Management Protocol), which is used by Isode’s management tools and provides strong authentication and data confidentiality using TLS and SASL.
QMGR is a multi-threaded event driven process that holds an internal representation of the status of all queued messages, channels and connections. By handling all of this with a single process that does not need to look at the queues on disk, very sophisticated scheduling, control and status reporting is available. QMGR is designed to efficiently manage very large message queues (hundreds of thousands of messages). More detailed information can be found on the M-Switch Queue Manager page.
SMTP & LMTP
Isode provides full support for Simple Mail Transfer Protocol (SMTP) as specified in RFC 5321 within relevant M-Switch products, conforming to the Internet host requirements for messaging (RFC 1123) including message submission (RFC 6409). SMTP support includes a number of auxiliary Internet Standard related to general capabilities and Lemonade support. A full list is given on the M-Switch SMTP conformance page.
Message Transfer by SMTP out of M-Switch is provided by the SMTP channel and Message Delivery by LMTP is provided by the LMTP channel. M-Switch may run many instances of each of these channels.
Inbound SMTP is handled by a single multi-threaded SMTP Server process (channel), so that new SMTP connections do not require new processes.
Message submission uses the SMTP protocol. The Internet Standard for Message Submission (RFC 6409) gives a special port for this (587), distinct from the SMTP port used for message transfer (25). Message submission is frequently authenticated, whereas this is unusual for message transfer. The SMTP Server can be configured to listen on both ports, or two SMTP servers can be used (one for each port).
Security for SMTP includes TLS (using START-TLS) and SASL based authentication, supporting a wide range of security options including client strong authentication using SASL EXTERNAL.
X.400 P1 is provided by the X.400 P1 Channel ("x400p1"). This may be run in two ways:
- Initiated by QMGR, to connect to remote MTAs to send and receive messages.
- Initiated by an incoming X.400 P1 connection, through the Isode OSI transport service listener ("iaed"). This will enable a remote MTA to send and receive messages.
X.400 P3 is implemented in two different channels.
- The P3 Delivery Channel ("p3") is initiated by QMGR, and is used to deliver messages to a server, and in particular to an X.400 Message Store such as M-Store X.400. This uses the "MTS Forced Access" application context.
- The P3 Client Channel is initiated by incoming X.400 P3 connections. This will be used by P3 Client applications sending and receiving messages, including applications built on the Isode X.400 Client API. This used the “MTS Access” application context. Two options are provided:
- singled threaded process initiated through the Isode OSI transport service listener ("iaed").
- A multi-threaded process. This is preferred for systems with large numbers of P3 connections.
M-Switch X.400 operates primarily at the message transport level, and so is independent of message content. Any X.400 content can be transferred, and in particular the X.400 Interpersonal Message format (P2/P22), the Military P772 format and the X.400 EDI Format (Pedi) are supported. Detailed information on X.400 feature support and conformance is provided here.
Message Submission Library
The Message Submission Library provides a standard internal service for putting messages onto the message queue, which ensures that per message functions such as authentication, routing, and processing calculations are applied consistently and correctly. Key functions provided by the submission library are:
- Sender validation.
- Recipient validation and message routing.
- Determining what internal processing is needed.
- Policy and authorization checks.
- Writing the message to disk.
M-Switch generally uses directory configuration. Table based configuration is also provided, as it is useful for some specialized deployments, particularly where a very simple static configuration is needed. The M-Switch GUI for managing directory configuration is MConsole, further information can be found on the configuration management pages for M-Switch X.400 and M-Switch SMTP. Table based configurations are managed by editing text files.
M-Switch configuration is used by all of the M-Switch processes. Logically, configuration information from the directory is used by all processes. As changes to configuration are generally infrequent, this is implemented by QMGR obtaining configuration information from the directory using LDAP, checking for updates, and sharing it with the other processes.
In order to optimize performance, the number of channels running, number of each channel type running, number of connections to a single MTA, will vary dependent on many factors, including remote system, network and local hardware performance. Manual optimization is complex, and in any event the optimal configuration is likely to vary over time. For this reason, M-Switch does a sophisticated “self test”, to measure performance and adapts configuration accordingly. This enables M-Switch to adapt its operational parameters to different hardware and network environments.
X.400 Message Routing
X.400 Message routing is configured in the directory using MConsole, based on RFC 1801 "X.400-MHS use of the X.500 Directory to support X.400-MHS Routing”. Routing is primarily done at the time the message is submitted, with directory checks also performed by the X.400 channels.
The routing tree mechanism allows for flexible routing configuration, including alternate routing to support fall-back routing and load balancing. Multiple routing trees may be used, to allow some routing configuration to be shared between MTAs, without requiring each MTA to route in exactly the same manner. Routing can be done on all X.400 address components (down to the mailbox level), including support for wild card routing, with pattern matching on address component values.
SMTP Message Routing
Routing of messages to other SMTP servers uses the domain part of the email address and is generally handled using the DNS (Domain Name System), but may also use locally configured host resolution. M-Switch also allows for connections to be configured to specific SMTP hosts using MConsole. Reasons for doing this include:
- Internal routing that overrides DNS based routing.
- Configuring security options, such as TLS for data confidentiality and SASL for peer authentication using passwords or strong authentication.
- Configuring one or more permanent SMTP connections, which will reduce message switching latency by removing the time needed to establish a connection.
M-Switch can also be set up to route based on the whole email address. This is referred to as LASER routing, after the Internet working group that developed a specification with this name. M-Switch broadly follows that specification, with a number of extensions, and options to deal with directory servers that store information in a different way.
The key concept in LASER routing is that you search in an LDAP directory for a mailbox of the form email@example.com. Routing succeeds when the search for the address returns in exactly one result for an Entry which holds suitable delivery information. Additional attributes in the entry can control with channel and host the message is then sent to. Typically, the match will use a default LMTP connection to a local M-Box server. This architecture works well for organizations with large “white pages” directories.
There are three basic ways that LASER routing may be used in conjunction with domain based routing:
- Single domain system, such as a consumer ISP or enterprise. If the domain is local, LASER routing is used, otherwise domain routing is used.
- Hosted service, handling multiple domains on one server. LASER routing is used to a single directory server. If there is a match, the message is handled according to the match. If not, domain routing is used for authorized message submission.
- Boundary switching to multiple departmental servers. Domain routing looks to identify locally configured domains. A match will identify the departmental LDAP server against which to do the LASER lookup, and the SMTP server to which matched addresses should be routed. This approach allows the boundary switch to fully validate email addresses by making use of existing LDAP servers.
M-Switch has flexible configuration that can support these setups and others. In addition to the core LASER routing, M-Switch also provides:
- An aliasing mechanism.
- A mechanism to specify alternate mailbox addresses.
- A list channel, that expands managed distribution lists.
- Sub-addressing support (joe+XXX@myorg.com).