Measuring and Analysing HF-PEP for TCP communication and Web Browsing over HF

Supporting Applications and Services over HF

This section looks at how TCP and Web fit into a general strategy for providing applications and services over HF.   STANAG 5066 is the open standard link layer for supporting applications over HF, and is the approach covered in this whitepaper.

Key Applications: Messaging and XMPP

Some mission critical applications define mappings directly onto STANAG 5066.  There are a number of options for messaging protocols, as set out in Isode whitepaper Messaging Protocols for HF Radio.  Performance of these protocols can give better than 90% link utilization as described in Isode whitepaper Measuring Performance of Messaging Protocols for HF Radio.

The approach for XMPP is described in Isode whitepaper Operating XMPP over HF Radio and Constrained Networks.  This approach has been demonstrated in UK MoD trials to work well down to 300 bps, with low latency and high reliability.

These optimized protocols obviate the requirement to operate these services using generic capabilities,  which would be significantly less efficient.

STANAG 5066 IP Client & Low Volume Applications

Most modern applications operate over IP.  STANAG 5066 defines “IP Client” services for operating an IP subnet over HF, as shown in the diagram above.   The approach is discussed in detail in the Isode whitepaper Measuring and Analysing STANAG 5066 F.12 IP Client.

The whitepaper shows that STANAG 5066 IP Client is a good approach for some low volume services such as ICMP Ping, and it works acceptably for some specialized military applications and services such as DNS Lookup.

Although STANAG 5066 IP Client follows the generic IP Architecture, the whitepaper shows that performance for TCP and bulk services in general is abysmal. Measurements from Measuring and Analysing STANAG 5066 F.12 IP Client are compared with the results in this paper using HF-PEP.

TCP and Web Services

TCP is the dominant protocol for applications running over the Internet, and it is clearly important for HF to operate a range of standard and special protocols over TCP (e.g., Database Access).

HTTP (Web) operates over TCP.  Many standard and most modern proprietary protocols operate over HTTP, rather than directly over TCP.

Support of TCP and Web applications is a key target for operation over HF and the subject of this whitepaper.

HF-PEP

HF-PEP uses a  PEP (Performance Enhancing Proxy) architecture, outlined in RFC 3135 “Performance Enhancing Proxies Intended to Mitigate Link-Related Degradations”, to remove the inefficiencies of running TCP over STANAG 5066 IP Client. The overall architecture for using HF-PEP is shown in the following diagram.

In this TCP Proxy architecture, an application is communicating over TCP, running over IP in the normal manner. The TCP connection from each application is peered with a proxy, rather than the other application. The proxies communicate using the HF-PEP protocol specified in HF-PEP: STANAG 5066 TCP Performance Enhancing Proxy Protocol (S5066-APP9).

HF-PEP operates over SLEP (SIS Layer Extension Protocol), specified in S5066-APP3. SLEP provides the Stream Services used by HF-PEP. SLEP communicates over STANAG 5066, using the local STANAG 5066 SIS (Subnet Interface Service) to connect. STANAG 5066 peers communicate over an HF network, as shown.

The Proxy can multiplex TCP connections over a single HF link, so that a single STANAG 5066 SAP can be shared by multiple TCP connections and multiple applications running over TCP.

The proxy may control TCP applications based on port including:

• Blocking certain ports, perhaps based on HF conditions and other load being handled by the proxy.  For example, to only allow one VOIP connection at any time.
• Giving priority at the proxy level to selected TCP applications, so that data from some streams is submitted ahead of other streams.
• Associating STANAG 5066 priority with each stream, so that the STANAG 5066 layer will give priority to specified streams.

A simple proxy may connect to just one peer and route all traffic to that peer. A proxy may connect to multiple peers and choose the peer proxy based on destination IP address of the destination specified in an inbound TCP connection.

Isode’s Icon-PEP product implements this architecture.

Measurements

The above architecture is used to measure performance. Components as follows:

1. HF Network simulated by Isode’s MoRaSky tool.
2. Icon-5066 (Isode STANAG 5066 product) provides STANAG 5066 service.
3. Icon-PEP (Isode pre-release product) provides F.12 IP Client and HF-PEP. It connects two IP Routers to provide an IP Subnet service over HF.
4. LH router connects to test host.
5. RH router connects to test host and Internet.

The following tests can be run from the LH Host:

1. Web Browsing to the Internet.
2. TCP Measurements using a special tool running on each of the test hosts. This tool sends a fixed amount of data and then waits for a short confirmation.

STANAG 4539 emulation was used for speeds of 9600 bps and below. STANAG 5069 with 48 kHz bandwidth was used for the 240 kbps tests.   All tests used short interleaver.  Further details provided with the tests.

TCP

TCP performance was measured with a special test tool that opens a connection and transfers a specified volume of data to a responding tool, which measures latency throughput (based on time from initial TCP request to all the data being delivered. Data volumes are measured in kilobytes (kB). Data transfers of up to 10 MBytes were made.

Size & Speed Analysis

The following tests were made over clear link for varying speeds and data sizes to provide a basic test of HF-PEP for a range of conditions.

A number of observations are made on these results:

• The results are good for all speeds and transmission volumes.
• As transmissions lengthen, utilization moves towards the link utilization achievable by raw STANAG 5066.  This is over 90% at higher speeds, and somewhat less at slower speeds, due to higher STANAG 5066 overhead due to reduced maximum C_PDU segment size.
• Operation was stable for all tests.
• TCP connect time varies somewhat due to STANAG 5066 initialization timings.  In general, this is faster at higher speeds and for smaller volumes of initial TCP data.
• Close time is quite consistent for a given speed.  It sometimes takes about double this time, which suggests that detailed timing will sometimes lead to an extra handshake.  It is not understood why 1200 bps is so much better than 9600 bps.
• For smaller transmissions, the final latency time is the same as transmission time.  This reflects that the sending TCP was able to quickly send all of the data and so all of the data had the same (initial) time stamp.  For larger transmissions, the sending TCP buffers data, and so the latency does not grow indefinitely.

The following table provides comparative information when using STANAG 5066 IP Client. A detailed analysis is provided in in Measuring and Analysing STANAG 5066 F.12 IP Client.   It can be seen that HF-PEP gives much better performance for all settings, and dramatically better for some settings.   HF-PEP is not subject to the failures and degradation seen with IP Client.

Tests with Link Errors

Error rate tests were made at 9600 bps with 100 kB of data transferred, using different Bit Error Rate (BER) values for the link.   These parameters are chosen, so that a direct comparison can be made with the measurements in Measuring and Analysing STANAG 5066 F.12 IP Client.

The BER range of 10-5 to 10-6 is a typical target for normal operation. BER 10-4 is to illustrate behavior in worse conditions.

The “STANAG 5066 Utilization” column was derived from the Icon-5066 logs for the long data transmission of each measurement. It can be seen that the TCP utilization is well aligned to the STANAG 5066 utilization. The STANAG 5066 overhead is broken into four groups:

• Protocol overhead. This increases slightly when there is data loss.
• Turnaround. This reflects slow HF switching and the need for the reverse transmission to be long enough to minimize risk of the (duplicated) Ack being lost. This is why data transmissions need to be long.
• Gaps due to data loss and completely missing D_PDUs. This is a small impact at the BER 10-4.
• Data loss due to corrupted data. This increases with error rate. BER 10-6 showed no loss in the test run.

These are seen as good results.

Web Services

Web browsing was tested by using various Web browsers to connect over the simulated HF link to Web sites on the Internet in order to test use of HF-PEP with Web.  The goal is to examine the viability of accessing the Web over HF.

Comparison of HF-PEP and STANAG 5066 IP Client

Two Web sites were examined at three speeds using the Firefox browser, to give a direct comparison to measurements made for STANAG 5066 IP Client in Measuring and Analysing STANAG 5066 F.12 IP Client.

• Example.com is a very simple web page that does not use HTTPS.
• Isode.com has a typical Web site, which is fast for normal access.

Partial loading is discussed in subsequent sections. It can be seen that HF-PEP gives dramatically better results than STANAG 5066 IP Client. It is clear that HF-PEP is the better way to support Web browsing over HF.

General Notes on Web Browsing over HF

Typical Web sites are built using many files (HTML, CSS, Javascript etc) that are retrieved. When accessing a Web page, a browser will start to retrieve files, which in turn point to other files. Files in modern websites are generally protected by use of TLS, which needs extra protocol for each retrieval. A Web site will typically have files referencing multiple domains (e.g., to load standard scripts and fonts).

Browsers will generally open multiple TCP connections to retrieve elements of a Web site in parallel, but will generally limit the number of TCP connections used.

Web browsers do not typically work by ensuring that all files of a Web page are retrieved. They will retrieve files incrementally, and then render the files that have come back as best they can. This can lead to partial loading, where some files (typically large graphics) are not loaded.

Interaction of Web Browsing with HF

The low speeds provided by HF have a significant impact on performance when accessing standard sites, as graphics and other large data blocks are commonly used.   However, the impact of “hand shaking” is often more significant due to the high latency of data over HF.  In the above measurements of example.com, which has very little data on the site, performance is limited by latency.
Some specific points are noted:

• Partial Load. This is a particular problem over HF. For the isode.com site, failure was seen on Javascript files as well as graphics. This led to a situation where the site worked, but not exactly as intended.
• Retries. Some browsers appear to retry downloads, and this can happen even when the initial retrieval later succeeds. This is problematic.
• QUIC. Some browsers and Web sites are using QUIC, which is a new protocol to access Web sites. QUIC is not suited for operation over HF using IP Client and needs to be disabled in the browser. It may make sense to standardize a QUIC proxy for operation over HF.
• Compression.  Web sites are not generally compressed, but there are mechanisms available. SLEP streams, used by HF-PEP, provide a compression option.  This was not used in these tests, and we believe it will improve performance.
• DNS. DNS is used extensively in many Web sites. DNS queries are made over UDP and mapped by Icon-PEP onto STANAG 5066 IP Client. The default DNS timeout used in the browsers tested was a bit shorter than typical retrieval times. This meant that DNS queries were sent several times and several responses come back. Delayed DNS responses also lead to ICMP traffic being sent back over HF. These effects are exacerbated when DNS queries are made during Web site retrieval, as DNS traffic is delayed by HTTP traffic. A DNS Cache on the client system is important to achieve reasonable performance. Many sites use short TTLs on DNS records, so the benefit is less than might initially be expected.

Browser Choice

The detailed way that a browser behaves has potential to  make significant difference to performance of Web browsing over HF. To optimize performance for browsing general purpose Web sites at narrowband HF speeds, it is likely that a tuned browser will be needed.

Most testing was done using Chrome and Firefox. Some tests were tried with Opera, which did not offer better performance and might be inferior.

Much testing was done with Chrome, which works reasonably well. At 9600, partial loading is observed quite a bit. There is no option to configure timeouts in Chrome, so performance has to be accepted “as is”. In practice, Chrome appeared the better option.

Firefox also works reasonably well.  It’s partial loading behavior appears slightly better than Chrome. However, it appears to retry loading in a manner which means that a lot of redundant traffic is created.   At 9600 bps, Chrome was faster than Firefox to complete a partial load of isode.com (4 minutes vs 6:30 minutes). Firefox led to HF traffic for many minutes after the partial load was completed.  In both cases, the isode.com site was quite usable, although it was not presented as designed.

Firefox provides extensive customizability of timers. Some experimentation did not lead to improvements over default behavior. There appears to be potential for tuning.
Further browser testing is desirable.

Further Measurements with HF-PEP

The Web site a-k-apart.com hosts a competition to prepare websites with less than 10 kBytes data on the initial page. This was helpful for testing at 9600 bps. Some of the sites loaded quickly and fully, but not all of them.

Tests were made on three general websites, with results below.

The initial load time was when the screen showed the core information on the page. Full load was when the browser declared load complete, after graphics and other information was loaded.

A Google search of “NATO” was made. At 9600 bps, a partial load missing some of the graphics occurred.
The Wikipedia NATO page was viewed. At 9600 bps, many of the graphics did not load.

This suggests that at higher (WBHF) speeds, that browsing many standard Web sites is viable, although slow.   At 9600 bps (top narrowband speed), some operation is still possible.

Website Recommendations

It is not anticipated that general purpose Web browsing will be the primary target of Web over HF. Consider a Mobile Unit (e.g., a ship or plane) communicating with shore systems. For many applications, “push” communication such as email will be ideal. However, Web access can complement this, enabling the Mobile Unit to have “pull” access to selected information and resources. It is anticipated that the shore systems would provide Web sites optimized for Mobile Unit access. This section provides some recommendations for such a site:

• Keep the volume of data low.
• Don’t use TLS. This eliminates hand shakes.  It also allows a Web cache to be provided on the mobile unit, which can share information between users.
• Hold all resources with the same DNS name.  This will mean that the Web client will not need to do (inefficient) DNS lookups as part of the page retrieval.
• Don’t have too many files, and avoid lots of small files.
• Structure the web page so that the core loads as quickly as possible. Users like interactivity.
• Put graphics and larger files into separately downloadable files, giving information on file size. Note that 9600 bps (top Narrowband HF speed)  is about 1 kByte/sec.  When operating at 1200 bps (mid range Narrowband HF speed), it is significantly slower.
• Structure a site into a collection of moderately sized pages.

Conclusions & Recommendations to NATO

This paper has shown the HF-PEP is an effective way to support TCP connections over HF, and that optimal use is made of the HF link using STANAG 5066 for a wide range of transmission speeds and transfer sizes in a manner resilient to HF errors.

HF-PEP provides a viable way to perform Web Browsing over HF. This can work with general purpose Web sites, but it is anticipated that special Web sites will be built for HF operation.

Recommendations to NATO:

• TCP and Web Browsing are important services for HF, and provision should be standardized.  Isode is offering SLEP and HF-PEP as the basis for NATO standards.
• IP Client is an inefficient and fragile way to offer these services.   NATO should clearly limit the recommended deployment scope of IP Client.