1. Local Setup

This tutorial is based on example code which can be found in the finTRAC Examples Repository.

These tutorials show how to build models using the TRAC D.A.P. runtime locally, in a context such as an IDE or the command line, without needing a running TRAC platform to hand. A model built this way needs no code changes to run on the platform - the same model class can be deployed and executed there as-is. This is possible because of the runtime’s dev mode, explained in the next tutorial.

Requirements

The TRAC D.A.P. runtime for Python has these requirements:

  • Python: 3.10 or later

  • Pandas: 1.3 up to 2.3.x (optional)

  • NumPy: 1.22 up to 2.3.x (optional, required by Pandas)

  • Polars: 1.X (optional)

3rd party libraries may impose additional constraints on supported versions of key libraries. For example, Pandas 1.5 is not available for Python 3.12 or 3.13, while NumPy 2.0 is only compatible with Pandas 2.1 and later.

Setting up a new project

If you are starting a project from scratch, it’s a good idea to follow the standard Python conventions for package naming and folder layout. If you are working on an existing project or are already familiar with the Python conventions, then you can skip this section.

For this example we will create a project folder called example-project. Typically this will be a Git repository. You will also want to create a Python virtual environment for the project. Some IDEs will be able to do this for you, or you can do it from the command line using these commands:

mkdir example-project
cd example-project
git init
python -m venv .\venv
venv\Scripts\activate

mkdir example-project
cd example-project
git init
python -m venv ./venv
. venv/bin/activate

For this tutorial we want a single Python package that we will call “tutorial”. By convention Python source code goes in a folder called either “src” or the name of your project - we will use “src”. We are going to need some config files, those should be outside the source folder. We will also need a folder for tests and a few other common project files. Here is a very standard example of what that looks like:

examples-project
├── config
│   ├── hello_world.yaml
│   └── sys_config.yaml
├── src
│   └── tutorial
│       ├── __init__.py
│       └── hello_world.py
├── test
│   └── tutorial_tests
│       ├── __init__.py
│       └── test_hello_world_model.py
├── venv
│   └── ...
├── .gitignore
├── README.txt
└── ...

Let’s quickly run through what these files are. First the src folder and the tutorial package. In this example “tutorial” is our root package, which means any import statements in our code should start with “import tutorial.” or “from tutorial.xxx import yyy”. To make the folder called “tutorial” into a Python package we have to add the special __init__.py file, initially this should be empty. We have created one module, hello_world, in the tutorial package and this is where we will add the code for our model.

It is important to note that the “src” folder is not a package, rather it is the folder where our packages live. This means that other folders and files (e.g. config, the .gitignore file and everything else) do not get muddled into the Python package tree. If you see code that says “import src.xxx” or “from src.xxx import yyy” then something has gone wrong!

The test folder contains our test code which is also arranged as a package. Notice that the package name is not the same (tutorial_tests instead of tutorial) - Python will not allow the same package to be defined in two places. Putting the test code in a separate test folder stops it getting mixed in with the code in src/, which is important when it comes to releasing code to production.

The runtime uses a few simple config files to control models during local development, so we have set up a config folder to put those in. The contents of these files is discussed later in the tutorial.

The venv/ folder is where Python puts any libraries your project uses, including the runtime library. Typically you want to ignore this folder in Git by adding it to the .gitignore file. Your IDE might do this automatically, otherwise you can create a file called .gitignore and add this line to it:

venv/**

The README.txt file is not required but it is usually a good idea to have one. You can add a brief description of the project, instructions for build and running the code etc. if you are using GitHub the contents of this file will be displayed on the home page for your repository.

Installing the runtime

The runtime package can be installed directly from PyPI:

pip install tracdap-runtime

The core runtime package does not include a dataframe library - Pandas and Polars are both optional extras, so install whichever your model needs:

pip install "tracdap-runtime[pandas]"
pip install "tracdap-runtime[polars]"

The pandas extra also pulls in NumPy automatically - there is no separate extra to install for it.

Other extras add support for particular platforms and protocols. Most models will not need them for local development:

  • pyspark - PySpark support

  • sql - SQL storage, for import and export

  • aws, gcp, azure - storage on the corresponding cloud platform

  • grpc - the runtime’s gRPC API server

If you want to target a particular version of Pandas, pin it alongside the extra:

pip install "tracdap-runtime[pandas]" "pandas == 2.1.4"

Alternatively, you can create requirements.txt in the root of your project folder and record projects requirements there.

Note

The runtime supports both Pandas 1.X and 2.X. Models written for 1.X might not work with 2.X and vice versa. Installing the pandas extra without pinning a version will pull in the latest compatible release, currently Pandas 2.X. To use Pandas 1.X in your local environment instead, pin the version explicitly:

pip install "tracdap-runtime[pandas]" "pandas == 1.5.3"

Installing other packages

The dataframe extras above are the only packages the runtime itself knows how to exchange data with, through methods like get_pandas_table(). A model is free to use any other third-party package for its own logic - for example the Using Files tutorial uses Matplotlib to render a chart as a model output. Packages like this have nothing to do with the runtime’s extras mechanism; install them the same way you would for any Python project, and record them in your own project’s requirements.txt alongside the runtime itself.

Running the tutorial examples

The examples for these tutorials are in the finTRAC Examples Repository. They are laid out in the same way as the project above, with config files in config/, source code in src/ and data files in data/. To install everything the examples need, including the packages used by the file IO and model resources examples, run this from the examples folder:

pip install -r requirements.txt

The example models import each other using the tutorial package, so the src folder needs to be on the Python path. In PyCharm, mark src as a sources root. From the command line, set the PYTHONPATH environment variable before running an example:

set PYTHONPATH=src
python src\tutorial\hello_world.py

export PYTHONPATH=src
python src/tutorial/hello_world.py

Launching models and jobs

The runtime offers three ways to launch a model or job locally:

  • launch_model() runs a single model class, with the job config supplying the parameters, inputs and outputs. This is used in most of the tutorials.

  • launch_job() runs a job config on its own. Use this for jobs that name their models in the config, such as flows, or that have no model, such as import and export jobs.

  • The command line launcher, python -m tracdap.rt.launch, runs a job config from the command line or an IDE run configuration (see Chaining Models).

launch_model() and launch_job() run in dev mode by default. In dev mode, relative config paths are resolved against the current working directory and then its parent directories, and launch_model() also searches from the directory containing the model. The command line launcher does not search: relative paths are resolved against the current working directory only, and dev mode must be turned on with --dev-mode.