Transferring large attachments in the Data Exchange Layer

This article describes the data transfer of large attachments and the factors affecting their effectiveness in the Suomi.fi Data Exchange Layer. The article explains the transfer of SOAP attachments, but the same general principles apply to the transfer of REST attachments.

The article describes a basic case in the Data Exchange Layer: the information system first sends the attachments to its own security server, from which they are sent to the receiving security server and again to the receiving information system. The Data Exchange Layer’s performance in transferring large attachments has been tested using cloud services. The measurements obtained as a result of these tests are indicative and illustrate the performance profile of the transfer of attachments in the Data Exchange Layer.

Data transfer methods and testing equipment

It is recommended to use attachments for transferring large volumes of data in the Data Exchange Layer. The Data Exchange Layer supports two methods of transferring attachments since version 6.9.x:

• SOAP with Attachments
• Message Transmission Optimization Mechanism with XOP.

SOAP with Attachments (SwA)

SOAP with Attachments is the original attachment transfer method in the Data Exchange Layer. It was also supported in earlier versions of the Data Exchange Layer.

In the Data Exchange Layer, the SOAP with Attachments transfer sends each attachment as a separate multipart piece. The Data Exchange Layer does not support sending multiple attachments within a multipart piece.

SOAP with Attachments is the main method used for performance measurements related to the transfer of attached files.

Learn more about the technical specifications of SOAP with AttachmentsOpens in a new window..

Message Transmission Optimization Mechanism with XOP (MTOM)

Starting with version 6.9.x, the Data Exchange Layer also supports the Message Transmission Optimization Mechanism transfer method.

From the perspective of Security servers, MTOM does not differ significantly from the SwA transfer method, and no significant performance difference between the transfer methods was observed during testing. In other words, the findings of more accurate performance measurements made using the SwA transfer method also apply to the MTOM transfer method.

Learn more about the technical specifications of Message Transmission Optimization MechanismOpens in a new window..

Attachment transfer testing equipment

The transfer of attachments was mainly tested in an AWS environment. The m4.large models of the AWS EC2 instance were used as security servers:

• processor: 2.4 GHz Intel Xeon® E5-2676 v3 (Haswell), 2 vCPU
• memory: 8 GB
• disk: 120 GB classic/magnetic.

Outside the AWS, testing included security servers of equivalent performance.

Learn more about AWS EC2 instance typesOpens in a new window..

Transfer rate of large attachments in the Data Exchange Layer

The Data Exchange Layer transfer rate (SwA) has been tested using different attachment sizes between 1MB and 30GB. The transfer of tens of gigabytes of attachments is not typical for the message traffic on the Data Exchange Layer. Still, the duration of the transfers remains linear with the file size. In practice, this means that the data transfer time in the Data Exchange Layer increases steadily as the file size increases. In other words, there is no actual size limit for attachments transferred through the Data Exchange Layer. The only concrete limits affecting transfer are the available time and disk space.

During testing, the transfer of attachments via the Data Exchange Layer has primarily been compared to a direct SOAP transfer between information systems. The main difference in the total duration of these transfer methods is the processing of message traffic on security servers. For example, encrypting and verifying traffic causes a delay in processing the message, which increases according to the size of the message. If the performance of the security servers is designed correctly for the environment, their processing delay increases steadily as the size of the attachments increases.

The impact of delays caused by processing can be kept under control by sufficient resourcing of security servers. In this case, even large attachments can be transferred through the Data Exchange Layer relatively efficiently. In normal circumstances, the transfer of large attachments via the Data Exchange Layer with attachments of different sizes and test configurations was on average 25% slower than a direct SOAP transfer. If the processing of message traffic on security servers slowed down due to insufficient resourcing, the significance of the processing delay in the total transfer time increased and the performance difference clearly exceeded the average.

Impacts of security server resourcing on the transfer of attachments

In the configuration shown in Figure 1, the sending security server streams the request received from the information system and the attachments to the receiving security server. The receiving security server stores the data stream it receives in a disk-based cache memory. The disk cache stores all attachments that accompany the message into one temporary file. The temporary file is deleted when the contents of the file are forwarded to the receiving information system.

Picture: Figure 1. Data transfer in the Data Exchange Layer between information systems

Show larger picture

The significance of internal memory

To test the transfer of attachments, servers with 8GB of memory were used, and the memory was not found to be a significant performance constraint for message processing. The technical recommendations of the Data Exchange Layer’s security servers propose that the memory settings of the security server be dimensioned according to the load. In particularly demanding transfer conditions, it is advisable to consider upgrading memory, but based on measurements, it is not the most important factor affecting the transfer rate of large attachments.

The significance of disk space and disk speed

With disk-based caching, free disk space on the receiving security server is the key to successful attachment transfer. If the attachments that are sent with an individual message are large enough that the temporary file created by the disk cache is too large for the server disk, the sending of attached files will fail at the reception state.

The disk-based cache used by the receiving security server may also affect the overall transfer rate of attachments. Therefore, the speed of the disk memory used by the security servers must be taken into consideration when designing the data transfer capacity.

For example, when transferring a 1GB attachment on a 52Mbps network, the following results were obtained with different disk speeds:

Table 1. The role of disk speed in data transfer.

A faster disk yielded a performance improvement of about 15 percentage points with an attachment size of 1GB. On average, the change increased the Data Exchange Layer’s transfer rate for all tested file sizes by about 17 percentage points.

The importance of processor capacity

Data transfer via the Data Exchange Layer requires more processing on security servers compared to a direct SOAP transfer. An effort is made to process data as soon as it is received from the network. This means that the load on security servers when transferring large attachments increases as network transfer rate increases. In this case, the processor capacity of the security server may become one of the most important factors limiting data transfer rate between information systems.

Testing showed that when large attachments were transferred over high-speed network connections, the processor load of the security servers grew dramatically. In the extreme, the processor capacity restricted the transfer so that the processing delay’s share of the total transfer time grew dramatically and the data transfer through the Data Exchange Layer was clearly slower than a direct SOAP transfer.

When transferring a 1GB attachment, the following results were obtained:

Table 2. The role of processor load in data transfer.

The results show that a faster network causes a large enough load on the processor so that the security server’s data transfer clearly slows down compared to both a direct SOAP transfer and a transfer over the Data Exchange Layer with a slower network connection. Based on the measurements, a processor capacity inadequate for the transfer conditions is thus the most significant performance limit for the transfer of attachments in the Data Exchange Layer.

Summary

Based on the above observations, it can be concluded that resourcing security servers connected to the Data Exchange Layer to suit the operating conditions is the most important factor for transferring large amounts of data efficiently. A high-speed network and services requiring large attachments also set high requirements for the processing and disk speed of the security server. However, the size of the attachments does not in itself limit the use of the Data Exchange Layer, as there is no upper limit for the file size dependent on the Data Exchange Layer. However, due to the disk cache, the attachments to be sent must fit entirely on the disk of the receiving security server.

The transfer of attachments via the Data Exchange Layer includes a processing delay compared to a direct transfer. The delay largely affects the performance of the transfer through the Data Exchange Layer compared to a direct transfer. In particular, the processor capacity of the security server has a clear impact on the creation of transmission delay. A slow disk memory in the receiving security server may result in a slight slowdown in data transfer, while insufficient processor capacity of the security server may increase the processing delay to a factor significantly higher than the actual transfer delay in terms of total transfer time. In this case, the efficiency of attachment transfer via the Data Exchange Layer will plummet compared to a direct transfer. However, with sufficient processor capacity for the transfer conditions, the processing delay will be smaller than the transfer delay, thus maintaining a good performance of the Data Exchange Layer.