Serialization Protocol

Files

Serialization code is located here:

KeyValueCollection.cs

The basic approach is to place parameters into a KeyValueCollection and then serialize it into a byte array. KeyValueCollection is a thread-safe version of Dictionary<KeyType, object>.

KeyType.cs

KeyType is a struct that can take one of two forms:

• byte code
• string name.

If you define USE_BYTE_KEYS, KeyType will hold a byte code value and will be serialized as a single byte.

If USE_BYTE_KEYS is not defined, KeyType will hold a string name value and will be serialized as an ASCII string.

KeyValueReader.cs (optional)

KeyValueReader is an optional component that enables random key-value reading from a binary stream (without deserialization of the entire KeyValueCollection). Use the skip* functions to navigate through key-value pairs until the required key is found. If no skip function is provided for a type, the read function is used.

Serialization.cs

This section contains definitions of core-supported data types along with DataSerializer registrations for each of them. Serialization of the following data types is supported:

• Null: a special serializer for empty values
• Boolean, Char, Byte, SByte, Int16, UInt16, Float, Int32, UInt32, Double, Int64, UInt64, Decimal: serializers for simple types
• KeyType, Guid, TimeSpan, DateTime: basic structs
• String, CompressedString, ByteArray, CompressedByteArray: basic arrays
• Object: dynamic object, with the type discriminator serialized before the payload
• Array: typed array, with the element type discriminator serialized before the payload
• Dictionary: dynamic map (KeyValueCollection), with the type discriminator serialized before the payload of each element

Serialization_Configuration.cs

This section provides the infrastructure for registered DataSerializers. Once registered, serializers are stored in a static array[255], allowing registration for up to 255 different types. The infrastructure allows you to:

• Register a Data serializer (IDataSerializer).
• FindSerializer (by DataType).
• FindSerializer (by System.Type).
• FindSerializer (by Object).
• Check if a serializer exists for a specific System.Type using HasSerializerForType.

Serialization_DataSerializer.cs

DataType, IDataSerializer, DataSerializer types - core types of serialization system.

Serialization_Read.cs

Static extension methods of BinaryReader for reading core types.

Serialization_Skip.cs (optional)

Static extension methods of BinaryReader for skipping core types.

Serialization_Write.cs

Static extension methods of BinaryWriter for writing core types.

Implementation Notes

In general, serialization is implemented as a set of static extension methods for BinaryReader / BinaryWriter. For each primitive type, we provide a pair of methods: ReadXXX / WriteXXX, along with DataSerializer registration.

In addition to primitive types, there is a set of dynamic types: Object, Array, and Dictionary. Serialization of these types requires runtime type inspection, translation of System. Type into DataType (using FindSerializer), and storage of the type discriminator before the payload. Dynamic types utilize a table of registered serializers to translate System.Type into DataType, and to handle read/write operations using the registered serializer.

Static types

• Null - 0 bytes: do nothing

• Boolean - 1 byte: 1 - true, 0 - false

• Char - 1 byte: cast char to bytes

• Byte - 1 byte

• SByte - 1 byte

• Int16 - 2 bytes

• UInt16 - 2 bytes

• Float - 4 bytes

• Int32 - 4 bytes

• UInt32 - 4 bytes

• Double - 8 bytes

• Int64 - 8 bytes

• UInt64 - 8 bytes

• Decimal - 16 bytes

• KeyType (WriteKeyType)

  • 1 byte if USE_BYTE_KEYS defined
  • ASCII string payload (WriteStringASCII) - if USE_BYTE_KEYS is not defined
    • 1 byte - string ASCII byte array length
    • byte[] - string ASCII byte array

• Guid (WriteGuid)

  • 16 bytes (https://stackoverflow.com/a/68919259)

• TimeSpan (WriteTimeSpan)

  • 4 bytes - time interval in ticks (10000000 ticks per second)
  • 4 bytes - time interval in seconds (if USE_UNIX_TIME defined)

• DateTime (WriteDateTime)

  • 4 bytes - time in ticks elapsed since 12:00:00 midnight, January 1, 0001 (10000000 ticks per second).
  • 4 bytes - the number of seconds that have elapsed since 1970-01-01T00:00:00Z (if USE_UNIX_TIME is defined).

• String (WriteStringUTF8)

  • 4 bytes - string UTF8 byte array length
  • byte[] - string UTF8 byte array

• ByteArray (WriteByteArray)

  • 4 bytes - array length
  • byte[] - array itself

• CompressedString (WriteCompressedString)

  • compress string Unicode bytes by passing through GZipStream
  • 4 bytes - string UTF8 byte array length
  • byte[] - string UTF8 byte array

• CompressedByteArray (WriteCompressedByteArray)

  • compress array by passing through GZipStream
  • 4 bytes - array length
  • byte[] - array itself

• Object (WriteObject)

  • 1 byte - DataType code
  • value payload

• Array (WriteArray)

  • byte - element DataType code
  • int32 - array length
  • array elements
    • element payload

• Dictionary (WriteDictionary)

  • int32 - pairs count
  • dictionary pairs
    • key - KeyType payload (byte or ASCII string)
    • byte - element DataType code
    • payload - element payload