Byte transfer
Some data is too large for one RPC: the saved character of a player, a blueprint a player built, a replay. Pulse Foundation sends such bytes between the server and the client of one player with reliable RPCs, a few chunks at a time, so it works with the classic replication and with Iris and never fills the reliable buffer of the connection.
// StartupModule of a plugin, on server and clients alike
FPulseTransferChannel channel;
channel.Direction = EPulseTransferDirection::ToServer;
channel.MaxBytes = 256 * 1024;
channel.OnReceived = [](APlayerController* InPlayer, FGameplayTag InChannel, EPulseTransferDirection InWay, TArray<uint8>&& InBytes)
{
// The server got the bytes of this player (read without loading assets, see Safety)
};
Pulse::Transfer::RegisterChannel(TAG_MyChannel, MoveTemp(channel));
// A client sends
Pulse::Transfer::Send(PlayerController, TAG_MyChannel, Bytes, [WeakThis](const FPulseResult& InResult)
{
// Success once the server took every byte, else Refused, Timeout, Cancelled, NotFound or Invalid
});
How it works
- Channels: a plugin registers a channel by tag with the way it goes (
ToServer,ToClient,Both), the most bytes one transfer may have and the function that takes the bytes (with the player, the channel and the way they went). Unregister it inShutdownModule. - Send: on the server
Pulse::Transfer::Sendgoes to the client of the player controller, on a client to the server. A local player of the server (host, single player) gets the bytes at once, without RPCs: to the server on a channel that only goes there, else to the client. - Done can run at once inside
Send(a refused transfer, the local player of a host) and when the player leaves (Cancelled); a callback that uses an object captures it as a weak pointer. - Packing: the bytes are packed (
Pulse::Serialization::CompressBytes: Oodle when that is smaller, else as they are) and cut into chunks of 8 KB. - Window: at most 4 chunks of one player and way are out without an acknowledgement, all transfers together; each acknowledgement lets the next one out. At most 2 transfers per player and way run at once, one chunk each in turn, so a small one is not stuck behind a large one; more wait. Nothing ticks.
- Readiness: the server sends to a client only after the client said that it has the component, so nothing is lost when the component was just added.
- The last chunk: the receiver unpacks the bytes, acknowledges the chunk and hands the bytes to the channel. The sender gets Success.
- Stalled transfers: a timer checks while transfers run. Without progress for 30 s a waiting receiver drops the transfer (the sender gets
Pulse.Result.Failure.Refused) and a sender fails withPulse.Result.Failure.Timeout. - Component:
UPulseByteTransferComponentcarries the RPCs. The server adds it to every player controller at login while a channel is registered, else on the first send; nothing to set up.
Limits
Checked in the engine code of 5.8: a classic net connection closes when one RPC is larger than 64 KB (its partial bunch limit) or a channel has more than 512 reliable bunches out; Iris splits an RPC into at most 4096 parts of 128 bytes (512 KB) and queues 4096 reliable RPCs per object; an array in an RPC has fewer than 65535 elements. One chunk of 8 KB is about 10 bunches, the window about 40 of the 512.
The speed is set by the bandwidth of the client: the engine sends at most MaxClientRate bytes per second to a client (100000 by default, shared with everything else of the player), so 100 KB take about a second. Games that send large data raise it in their engine config.
Safety
The server takes the announced size only up to MaxBytes of the channel (plus the packing header), checks every chunk for its index and its size, unpacks with the same limit and takes at most 2 transfers per player at once. A transfer that breaks these rules is refused once (Pulse.Result.Failure.Refused) and dropped; later chunks of it get no answer.
Bytes from a client can name any asset. The server hands them to the channel inside a Pulse::Serialization::FNoAssetLoadScope: BytesToStruct and BytesToInstancedStruct then only find objects and struct types that are loaded already, and Pulse::Serialization::CanLoadAssets() tells code that resolves paths itself not to load either. Read the bytes in the receiver, not later outside the scope.
Testing
PulseTransferTest [KB] [true|false] (cheat, run as a client) sends bytes to the server on the channel Pulse.Transfer.Test (random, or a pattern that packs well with true); the server sends them back to a player that may use cheats, and the client logs the time and whether they are the same. The channel exists only where a cheat manager was created.