File Transfer, XMPP & HF Radio

File Transfer & Chat

Historically, file transfer over networks was achieved by use of special file transfer protocols. In modern networks, end users rarely use these specialized protocols. When a user wishes to access a file (“pull” model), this is typically achieved by using Web (HTTP) access to the file of interest. When a user wishes to send a file to one or more users (“push” model), the file is typically attached to an email or sent using chat. A common typical chat usage of files is to share photos and other images. Although file sharing is quite distinct from basic chat, file sharing is a capability expected in a modern chat service.

Requirements for Constrained Networking & HF Radio

In modern fast networks with speeds suitable for streaming video, the performance implications of file transfer are barely worth considering. When deploying over a slow network, particularly HF radio which can have high latency and high error rate, there are some specific considerations around file transfer:

1. It is desirable that any file transfer does not unduly delay any ongoing chat.
2. Where a file is distributed to multiple recipients, it is important that only recipients interested in the file incur the overhead of the file transfer over the slow network.
3. Where a file is large, noting that the definition of large will vary according to network speed, it is important that the recipient can make a choice as to whether or not to transfer the file. In some cases, the file may be impractically large. In others, it is a choice of whether to incur overhead and delay

Point to Point XMPP File Transfer Mechanisms

XMPP has specified a number of mechanisms for transfer of files between two clients. The modern option is “XEP-0234: Jingle File Transfer”. Jingle is a generic negotiation mechanism used by XMPP, in particular to negotiate voice and video calls. In this specification it is used between two parties to negotiate a file transfer, which can be push or pull.

Jingle File Transfer can negotiate one of two classes mechanisms for transferring the file, The first is Direct transfer between the XMPP clients, using a choice of mechanisms, as illustrated above. Jingle over XMPP negotiates the file transfer, which can be either direction and the file is then transferred directly to between the clients. Issues with this approach:

1. It is often impractical for clients to use direct transfer, as there is no direct network connectivity.
2. In many environments, direct transfer raises security concerns as client activity does not get centrally audited or controlled.

The second approach is “In Band” transfer over XMPP, which is negotiated by Jingle over XMPP. This mechanism breaks the file into small pieces and transfers the pieces as a sequence of XMPP message which are transferred between the clients. Issues with this approach:

1. In band file transfer prevents intermediate checks on whole files such as file anti-virus, which may be an issue in some scenarios such as cross domain.
2. In band mixes the file data with XMPP traffic, which means that the file transfer can lead to delays of general XMPP traffic.

File Sharing & HTTP Upload

Although it is useful to share files between a pair of users, it is also desirable to share files with multiple users using a MUC room which the point to point approaches does not support.

Because of the issues with point to point options, a second mechanism is recommended for file sharing with XMPP. This approach uses the HTTP Upload mechanism specified in “XEP-0363: HTTP File Upload”. HTTP Upload allows an XMPP client to negotiate a “slot” on a server associated with the XMPP server, where it can publish a file which is assigned a URL (Web reference) that can be generally accessed. Functionally, this is like the XMPP client publishing a file on a special Web site.

The XMPP client can share this URL with XMPP recipients, who can then download the file. It is lightweight and efficient for an XMPP client to share this URL with many recipients, typically by sending the URL to a MUC room.

File Information Sharing

Swift can share a document either by file selection or by use of a local media device. On reception, the received documents or photos are shown clearly.

While the HTTP Upload URL can be shared on its own, it is more useful to share the URL with meta-information on the file, with information such as Description, Filename, and File Size. This can be done using “XEP-0385: Stateless Inline Media Sharing (SIMS)”. SIMs allows information about a file to be efficiently shared. This information can be shown to the user, as illustrated in the screenshot above. This can let a recipient decide if the file is of interest and also if the file can be realistically downloaded of the access channel that that the XMPP client has.

Operation over HF Radio

The diagram above shows communications flow using Isode products, noting that standard protocols are used throughout and other products could be used. The grey arrows show standard XMPP communication over HF Radio as described in the Isode white paper Operating XMPP over HF Radio and Constrained Networks. The products used are:

1. Swift, which is Isode’s Web XMPP Client, that connects to M-Link.
2. M-Link is Isode’s XMPP Server, which supports operation over HF Radio following “XEP-0365: Server to Server communication over STANAG 5066 ARQ”.
3. Icon-5066 is Isode’s STANAG 5066 server that provides a standard HF link layer that can multiplex applications.

It can be seen that this provides an end to end XMPP communications chain over HF Radio between a pair of Swift (XMPP) clients.

The red arrows show file transfer with the direction of arrows showing where the file flows. At the start of the transfer, the sending (right hand) Swift client will upload file/files to an area associated with the M-Link server. Swift will then share the URL of the uploaded file plus SIMS meta-data over XMPP with receiving Swift clients. In UI terms, this will appear to the sending client as simply “sending the file”. If the receiving client chooses to download the file, it will simply access the URL provided using standard HTTP (web access).

Web access over HF uses a fourth Isode product Icon-PEP which provides TCP and Web access over HF. This is described in the Isode white paper Measuring and Analysing HF-PEP for TCP communication and Web Browsing over HF. It can be seen that the file retrieval uses a completely independent channel to the XMPP traffic, sharing HF access with Icon-5066. This architecture gives two advantages:

1. The file transfer is independent of the XMPP traffic, so will not (directly) impede or delay XMPP traffic in the way that in band transfer would.
2. At the STANAG 5066 level, the file (Icon-PEP) traffic can be configured with a lower priority than the XMPP traffic, so that XMPP traffic will be sent in preference to the file transfer.

Conclusions

This white paper has shown how XMPP file transfer can be supported using HTTP Upload to provide a convenient experience for sending files to multiple recipients. Meta information shared with file reference can help recipients make sensible file download decisions. This can all be mapped onto HF Radio using Isode products in a way that decouples file transfer from the XMPP traffic.