Skip to end of metadata
Go to start of metadata

Let us see what happens behind the scenes when we create a portable table with default transformers

    ITable<long, Tick> table = engine.OpenXTablePortable<long, Tick>("table");
  1. The engine will generates default dataType descriptions for long and Tick types. The descriptions practically represent the table definition
    The generated description for the key type is:


    The generated description for the record type is:

  2. Based on the descriptions the engine will build anonymous types, equivalent to the provided user types:
    The anonymous type for the keys matches the original user type:


    The anonymous type for the records is:

        typeof(Slots<string, DateTime, double, double, int,  int, string>)

    The Slots class is a generic class, internally available in the database (there are many of them):

    public class Slots<TSlot0, TSlot1, TSlot2, TSlot3, TSlot4, TSlot5, TSlot6> : ISlots
        public TSlot0 Slot0;
        public TSlot1 Slot1;
        public TSlot2 Slot2;
        public TSlot3 Slot3;
        public TSlot4 Slot4;
        public TSlot5 Slot5;
        public TSlot6 Slot6;
  3. After creation of the descriptions and creation of the anonymous types, the engine will generate two transformers – one for the keys and one for the records. The first transformer is responsible to convert all user keys to/from anonymous keys; the second transformer is responsible to convert all user records to/from anonymous records. The transformers are run-time created with .NET expressions, so there is no performance penalty. The generated code will look like:

    public class KeyTransformer : ITransformer<long, IData>
        public IData To(long value1)
            return new Data<long>(value1);
        public long From(IData value2)
            return ((Data<long>)value2).Value;
    public class RecordTransformer : ITransformer<Tick, IData>
        public IData To(Tick value1)
            var slots = new Slots<string, DateTime, double, double, int, int, string>();
            slots.Slot0 = value1.Symbol;
            slots.Slot1 = value1.Timestamp;
            slots.Slot2 = value1.Bid;
            slots.Slot3 = value1.Ask;
            slots.Slot4 = value1.BidSize;
            slots.Slot5 = value1.AskSize;
            slots.Slot6 = value1.Provider;
            var data = new Data<Slots<string, DateTime, double, double, int, int, string>>(slots);
            return data;
        public Tick From(IData value2)
            var data = (Data<Slots<string, DateTime, double, double, int, int, string>>)value2;
            Slots<string, DateTime, double, double, int, int, string> slots = data.Value;
            Tick tick = new Tick();
            tick.Symbol = slots.Slot0;
            tick.Timestamp = slots.Slot1;
            tick.Bid = slots.Slot2;
            tick.Ask = slots.Slot3;
            tick.BidSize = slots.Slot4;
            tick.AskSize = slots.Slot5;
            tick.Provider = slots.Slot6;
            return tick;

    As the code shows, the RecordTransformer converts Tick instances to Slots<string, DateTime, double, double, int, int, string> instances. Then it wraps each anonymous instance in Data<Slots<string, DateTime, double, double, int, int, string>> wrapper. Similarly, the KeyTransformer converts all long instances to long instances (i.e. there is no real transformation) and wraps each of them into Data<long> wrapper.
    As we said previously the Data<T> class in the database is a universal data wrapper:

    public interface IData
    public class Data<T> : IData
        public T Value;

    The database uses Data<T> wrappers, because all tables in STSdb 4.0 share one WaterfallTree instance. In the Data<T> values, the real T values can be from any .NET type supported by the engine. (From the engine point of view, all user keys and records from all tables are just IData instances.)

    Thus, we transform the user values to anonymous values and wrap each of them with IData instances. In the above example the real table behind our created ITable<long, Tick> table is ITable <IData, IData>, where the IData keys are actually Data<long> instances and the IData records are actually Data<Slots<string, DateTime, double, double, int, int, string>> instances.
    Let's back to the OpenXTablePortable method for a moment. When a table is being created with:

        var table = engine.OpenXTablePortable<TKey, TRecord>();

    the engine does not keep the original key and record user types (your can check the KeyType and RecordType properties - it will not keep the Tick type), Instead it stores only the key and record descriptions (the DataType instances). On the bases of them lately the engine rebuilds the anonymous types. On its turn, from the anonymous types it rebuilds the transformers code expressions. With this chain: user type -> description -> anonymous type -> transformer code, we allow the users to open the same table with completely different user types - as long as they are compatible with the stored dataType descriptions. Thus, we make the database table independent from the user types, while keeping excellent performance and ease of use.

    All these steps are happening behind the OpenXTablePortable<TKey, TRecord> method.

     The above logic, however is not true when we work with the user types directly:

        var table = engine.OpenXTable<TKey, TRecord>();

    Here the engine works directly with the TKey & TRecord types. There are no transformers - the engine will generate a concrete compare logic, a concrete persist logic etc and along with the descriptions it will store the exact TKey and TRecord full type names. And when the user opens the table later, the engine will try to restore the original types from the stored names and will regenerate the entire concrete TKey & TRecord expression trees. The direct generic OpenXTable works only with the specified user types, but provides insanitive performance.

Custom Transformers    Go to Home    Custom XTable logic

  • No labels