Measurements of S5069 and S4539 waveforms with varying interleavers over a Channel Simulator

A general principle for HF Modems is often stated that short interleavers are best for voice and longer interleavers for data. We have not been able to find substantial measurement information in the literature to back up this assertion. We have also talked with HF experts who assert that often short interleavers are better for data. It seems highly desirable to make measurements.

Longer interleavers have a number of clear downsides. They can lead to inefficient data use, where the data transmitted does not fit an exact number of data interleaver blocks. Longer interleavers also tend to lead to longer turnaround times, as described in the Isode whitepaper [Reducing Turnaround Times in STANAG 5066]. It is important to quantify the level of improvement associated with different Interleaver selection, to obtain best overall performance.

While Over The Air measurements would be ideal, a useful starting point is to make measurements using a channel simulator using the Watterson model with CCIR conditions. Systematic measurements can be made, which is not practical Over The Air.

There is an interaction between choice of frame size (STANAG 5066 C_PDU Segment Size) and performance, which is important to measure. This will enable recommendations on optimum C_PDU segment size.

We look at STANAG 4539, which is the current widely deployed modern waveform, and STANAG 5069, which is the emerging Wideband HF waveform. This enables comparison with STANAG 4539 for operation over single channel, and examination of performance on multiple channels.

In making the measurements, we determined that STANAG 5069 preamble count (M) was significant, and the effects of this are noted. A comparison of the effect of STANAG 5069 constraint length (k) is also measured.

Test Configuration & Methodology

The following diagram shows the test setup. Data is transferred from one modem to a second modem through a channel simulator. Both modems and simulators are Collins HSM 2050s, which can act both as modems and channel simulator.

HF Tool is an Isode tool that ships as part of the Icon 5066 product set. It can control the Collins modems and the Collins channel simulator. HF Tool sends a known pattern of data between the modems, controlling modem transmission and receive settings. It can determine if data is received, and if the information is correct. HF Tool also controls the channel simulator, so that type of channel (e.g., CCIR Poor) and SNR are controlled. HF Tool can run sequences of test transmissions.

For each speed/interleaver chosen, the first run is made with an SNR setting that is expected to be “mid range” for the speed. Then for subsequent runs the SNR is decremented by 1. This will be repeated until two consecutive runs lead to no signal (or data) for the entire run. Then the SNR is increased from the original level, which is repeated until three consecutive runs with zero data loss are achieved. This leads to measurements over the full range of SNR values which are useful.

The received data is then analyzed and presented graphically. For a set of frame sizes from 50 bytes to 1,000 bytes, a frame error rate is calculated. If a frame has any erroneous bits, it is treated as an error. A single bit error would cause check sum to fail. A second calculation of link utilization is done. This combines the frame error rate with the STANAG 5066 DTS overhead for a frame of that size. This allows the choice of interleaver and C_PDU segment size to be analyzed.

Basic STANAG 4539

We chose CCIR Poor as the reference channel quality to use for the tests. Tests use CCIR Poor, except where explicitly noted otherwise. The test shown below is for STANAG 4539 1200 bps.

Some notes on this diagram follow, which is an example of all the other runs.

1. The three-colour columns on the left represent runs for each interleaver size, from shortest to longest.
   • Blue is when no data is being received. Note that the top two lines are always solid blue, reflecting a completely failed transmission (except when end of SNR range is reached).
   • Green indicates when data is received correctly. Note that the bottom three lines are always solid green, reflecting loss free transmission (except when end of SNR range is reached).
   • Red indicates data errors, with length indicating number of errors.
2. For lower SNRs, note that signal is lost and then synchronized again. This is common for STANAG 4539, and is one reason why runs needed to be as long as ten minutes to get stable values.
3. The red column has dark red bars indicating Frame Error Rate (FER) for a range of frame sizes. No red indicates no frame errors. All red indicates 100% errors (no valid data).
4. The green column indicates link utilization, based on STANAG 5066 overheads for the frame size chosen. Smaller frame sizes always have few frame errors, but the overhead is higher. This column provides key analysis for choice of interleaver.

The following figure is an extract from the right-hand column showing link utilization. This is provided for many of the runs shown here, as this gives key information to analyze each run.

A quite short SNR range is relevant. If you are getting much less than 50% utilization, better throughput will be achieved by using a slower speed. For this run, SNR=9 is the point where sufficient throughput is achieved. For a range of about 3dB above this, this speed is expected to remain optimal. At higher SNRs, moving to a faster speed will generally give better performance. So the “interesting” range in this run is 9-12 dB.

It can be seen here that use of L (long) interleaver gives clearly better performance.

When the SNR is below the optimal range, shorter interleavers perform better. This is true for all of the measurements, and reflects that when errors are high, shorter interleavers are better able to slot into the gaps. However, it is desirable to optimize interleaver for the target SNR range, rather than these “out of range” results. Also, it is common that useful throughput at lower SNR is only achieved by using smaller frame sizes than would be chosen for the optimum range.

The next sections look at use of STANAG 4539 across the range of supported speeds.

STANAG 4539 at Low Speeds

This run is at 75 bps. It can be seen that Long interleaver gives dramatically better performance. At the lower speeds, the benefits of using L interleaver are very high. The lower the speed, the larger the benefit. By 1200 bps, there are still very clear benefits, but not as large.

For this speed a C_PDU segment size of 150-200 bytes seems optimal.

The throughput benefit of L interleaver at 75bps can be seen very clearly from the detail shown below. For this speed a C_PDU segment size of 100-200 bytes seems optimal.

STANAG 4539 at Top Narrowband Speeds

This run is for 8000 bps, which is one of the fastest narrow band speeds. It can be seen that for the shortest interleavers, errors occur at SNR values a long way above the target SNR range for this speed. This is termed large “SNR Spread” in this paper. The reason for this is that CCIR Poor has deep short fades, which a longer interleaver will compensate for.

This leads to a recommendation to always avoid use of US (ultra-short) and VS (very-short) interleavers.

Looking at the detail, it is clear that longer interleavers still give clear benefit, even at the fastest narrowband speeds. L (long) and VL (very-long) give relatively small benefit over M (medium).

The optimal C_PDU segment size is not particularly sensitive. Any value in the range 200-800 bytes seems reasonable.

CCIR Good

This run at 9600 bps shows how things change for a CCIR Good channel. The SNR spread does improve with longer interleaver, but this is not so large as for CCIR Poor. CCIR Good has long (10 second) fades, and the longer interleaver is not particularly effective at compensating for this.

The detailed analysis shows quite clearly that for CCIR Good, there is little throughput variation between the different interleavers.

STANAG 5069 Single Channel

The next set of measurements look at STANAG 5069 using a single 3 kHz channel, which gives a direct comparison with the STANAG 4539 measurements.

Impact of M (Preamble Count)

The following are examples of initial measurements made. The next one is 75 bps.

The following is at 2400bps.

An immediate observation is that there are lots of “blue” lines, where successful data transmission would have been expected.   The reason for this is that the 300 millisecond synchronization preamble is insufficient to gain synchronization.  STANAG 5069 is designed so that in most situations that re-synchronization during a transmission does not happen. However results suggest that STANAG 5069 is significantly better than STANAG 4539 in retaining synchronization. This does mean that, particularly for longer transmissions, it is important to gain initial synchronization.

STANAG 5069 synchronization has M 300 millisecond pre-ambles, where M is from 1 to 32. This gives a maximum preamble of 7.7 seconds. Subsequent runs were all made with M-32. For 75 bps it seems likely that 32 is a good choice of M. For higher speeds, shorter values make sense. It is clear that for optimal performance, M should be chosen based on both SNR and length of transmission.

STANAG 5069 – Lower Speeds

This is 75bps. It is clear from this run that L interleaver is the right choice.

This next is 600 bps, which is a fairly slow speed. It is clear that both S and US are poor choices. L is the best choice, but M is acceptable.

STANAG 5069 – Higher Speed

Next is a run at 4800 bps. The SNR Spread for the lower speeds is large, and it is clear that both US and S should be avoided for this reason.

The longer interleavers are also better for throughput. L should be used where possible, but M is acceptable at this higher speed.

Comparing STANAG 4539 with STANAG 5069

Using measurements made, we determined for each waveform and speed, the lowest SNR where link utilization of around 50% is achieved. This is shown in the table that follows.

For STANAG 4539, the spread is uneven, particularly at the middle speeds. 2400 bps is particularly odd. This is a waveform that is known to be sub-optimal.  Isode’s Icon-5066 is by default configured to skip 2400 and go straight from 1200 to 3200.

For STANAG 5069, the higher speeds have similar performance to STANAG 4539. 75 bps is slightly worse. The other low speeds are better and the spread of transition points is much more even. This suggests clearly that STANAG 5069 is preferable to STANAG 4539 for a 3 kHz channel.

STANAG 5069 Multiple Channel

The following measurements are all made for a 24 kHz channel, to give indication of Wideband HF performance. The first is 64,000 bps, which is close to top end. The SNR spread is large, and this suggests that US and S should be avoided.

Looking at detailed link utilization, the longer interleavers perform slightly better, but the difference is not significant.

Next at 12,800 bps, which is a mid-range speed for 24 kHz channel. At this speed, there is a clear (but not massive) improvement for longer interleavers.

This is at 600 bps, which is the slowest 24kHz channel speed. The SNR spread is lower than for the faster speeds.

At 600 bps, performance of S interleaver is significantly less and should be avoided. L is slightly better than M.

Impact of k (Constraint Length)

STANAG 5069 has two options for constraint length: k=7 and k=9. It is suggested that k=9 will give better performance. Previous measurements were made with k=9. The following measurements were made with k=7. The first two measurements are for 600 bps.

The second two measurements are for 12,800 bps.

Comparing the graphs it can be seen that the SNR spread is slightly better for k=9 and the throughput is slightly better too. This suggests clearly that k=9 should be used where possible.

Anticipated Differences for Real Network

These measurements are made based on channel simulation and real networks will have different performance characteristics.   A major difference for Skywave will be the presence of Intermediate Term Variation (ITV) reflecting changes over 30 seconds to 2 minutes timeframe. Measurements suggest that variations will be of order 10 dB.

ITV was analysed in “Optimizing applications and data links for HF radio intermediate term variations: Can you ride the wave?” presented at Nordic HF in August 2016. To get best performance a speed will be selected to optimize for the best parts of the transmission: it is better to use the good bits with a faster speed and lose more data in the poorer segments.

It is anticipated that interleaver best choice will end up with similar values to the simulation.

Notes on Choice of C_PDU Segment Size

The measurements suggest that where speed is being chosen to optimize throughput, that performance is not critically dependent on  exact choice of C_PDU segment size.  However, there is a broad trend of using larger C_PDU segment sizes for higher speeds.

At 75 bps,  a segment size in range 100-200 bytes appears optimal.  For lower narrowband speeds the standard recommendation of 200-300 bytes appears reasonable. At higher narrowband speeds, values in the range 400-600 bytes seem appropriate, increasing to 800 bytes for wideband. When operating at fixed speed, longer segment sizes may be appropriate in good conditions.

Notes on Interleaver Choice

There are a number of recommendations that can be drawn from these measurements. These are on the basis that real networks are sufficiently similar:

1. US and VS interleavers should never be used for data.
2. For very low speeds at 3 kHz,  L should always be used.
3. When transmitting large amounts of data, VL or L should generally be used, with the following caveats:
   • If data packs badly, leading to wasted space, M may be preferable.
   • For shorter transmissions, turnaround cost may be impacted by a longer interleaver. It is recommended to use the techniques described in Isode whitepaper [Reducing Turnaround Times in STANAG 5066]. If these are not used, performance may be optimized by M interleaver.

If a TDMA approach is used, fixed transmission lengths may lead to performance being optimized by using shorter interleavers. There are times when only a small amount of data needs to be transmitted. This is typically data where it is desirable to not lose data:

• Annex L token. Losing this leads to significant performance issues.
• STANAG 5066 Ack. Losing this leads to expensive retransmission.
• XMPP messages or other small messages. Typically, this is best sent quickly.

Using M/L/VL interleavers for such data is going to be wasteful. A generally sensible approach is to slow transmission rate, to reduce risk of message loss. It is also useful to repeat the critical data. An S interleaver (typically around a second) seems a minimum safe interleaver to override fades.

Analysis and Conclusions

This paper has examined interleavers and other parameters. The following recommendations are made:

• STANAG 5069 is preferable to STANAG 4539 for 3 kHz channel.
• Preamble count (M) needs to vary (large values for slow speed) and be selected based on chosen transmission speed, length of data, and criticality of data.
• Constraint k=9 is generally preferable.
• US and VS interleavers should not be used for data.
• L and VL interleavers are generally preferable, but sometimes M interleaver is best for long transmissions.
• For acks and other short transmission,  S interleaver is often best, used in conjunction with slower speed.