Virtual deployments

Deployment steps

Trac uses a ‘virtual deployment’ method which involved three steps.

IMPORT MODEL

The model code is scanned by not copied. A new metadata object is created which contains the model schema and a link to the code commit in the repository

BUILD FLOW

Models are combined into a calculation graph where the outputs of one model become the inputs to the next. Flow exist only as trac metadata and are built from the model schemas, not the model code

RUN FLOW

The model code is fetched from the repository at runtime every time a Job is launched.

Note

This short video (under 3 mins) explains the virtual deployment method.

Model schema

All trac models contain a custom function to declare their schema, which consists of:

INPUTS

The schema(s) of all the input data the model needs to run

PARAMETERS

The schema of any parameters that govern how the model runs

OUTPUTS

The schema(s) of all the data objects produced when the model runs

Note

See Modelling for how to build trac models.

Schema validation

This is an example of a schema (sub-type = table).

select file

Schema validation occurs when selecting data inputs for a job and when combining models to build a flow. In both instances, TRAC applies strict schema validation.

Key things to remember about schema validation are:

  • It occurs between schemas, not between data content

  • It considers Field name, Field Type, Business key, Categorical and Not nullable

  • The order in which field names appear is not important

  • A source schema must contain all the fields needed by the target schema

  • The source schema can contain extra fields

Note

See Data types for supported data types.