Repository structure

Before you decide how to organize your automation code, it is important to consider whether or not you wish to run these automations on the Seven Bridges Platform. If you do, you have to follow certain guidelines to make sure your automation code is ready for platform execution.

When setting up a new automation project, Freyja offers a convenient 'init' sub-command that creates a Python project structure suitable for both local and platform execution. We highly recommend using this command when creating new automation projects, but you can create the required project structure also manually (not recommended).

For an automation named project_1 the project structure generated by 'freyja init' looks as follows (including hidden files):

project_1
|-- configs
|   `-- secrets.yaml      # configuration file for secret settings (optional)
|-- project1              # automation main package (required)
|   `-- __main__.py       # module containing automation main step (required)
|-- .entrypoint           # contains name of main package (i.e. "entrypoint: project1")
|-- .packignore           # files and directories to exclude when creating code package file (optional)
|-- requirements.txt      # Python dependencies (required for platform execution)
`-- README.md             # Readme file (optional)

To execute this automation locally, type python -m project1 run [args] from inside the outer project_1 directory.

To prepare a code package for execution on the Seven Bridges Platform, first change to the outer project_1 directory and then type freyja build.

The build command first infers the input-output schema from your automation main step, stores the schema in a file named '.schema' inside your project root, then gathers all files and directories inside your project root (excluding files mentioned inside .packignore), and finally compresses everything into a zip file named project_1.zip (the code package file).

The code package file can then be used as input to the 'sb automations packages create' command of the Seven Bridges Automation CLI, which uploads the code package file to the Seven Bridges Platform and creates a new code package entity. For more details, please refer to the previous section that contains a step-by-step guide on how to deploy and run an automation on the Seven Bridges Platform.

Note that for platform execution to work, all dependencies of 'project_1' must either be:

1. Packaged inside the code package file, or
2. Listed as pip-installable libraries inside 'requirements.txt'.

However, automations might have shared source code dependencies that are unpublished and do not reside inside an automation project. In cases like this, the 'freyja build' command allows you to include those dependencies during code package file creation.

Here is an example for how you can organize multiple automation projects that have shared and non-pip-installable source code dependencies:

repo_1                        # git repo with multiple automation projects
|-- project_1                 # automation project 1
    |-- configs
    |   `-- secrets.yaml      # or other config files (optional)
    |-- project1             
    |   `-- __main__.py      
    ... ...                  
    |-- .entrypoint          
    |-- .packignore         
    |-- requirements.txt
    `-- README.md
|-- project_2                 # automation project 2
    |-- configs
    |   `-- secrets.yaml      # or other config files (optional)
    |-- project2             
    |   `-- __main__.py      
    ... ...                  
    |-- .entrypoint          
    |-- .packignore         
    |-- requirements.txt
    `-- README.md
|-- shared_project_1          # shared project 1 (optional)
    |-- sharedpackage1       
    |   `-- module1.py
    ... ...                  
    |-- README.md
    `-- setup.py
 
repo_2                        # additional git repo (optional)
|-- shared_project_2          # shared project 2 (optional)
    |-- sharedpackage2       
    |   `-- module2.py
    ... ...                  
    |-- README.md
    `-- setup.py

In the above setup, we have a main git repo ('repo_1') with two automation projects ('project_1' and 'project_2') and one shared project ('shared_project_1'). In addition, we have a second git repo ('repo_2') that contains another shared project ('shared_project_2').

When you build the code package file for project_1, first change directory into 'project_1' and then use the following command:

~/project_1$ freyja build -i ../shared_project_1/sharedpackage1 -i /path/to/repo_2/shared_project_2/sharedpackage2

This will include sharedpackage1 and sharedpackage2 as dependencies inside the generated code package file.

The file structure inside the created zip file should look like this:

zip root
|-- configs
    `-- secrets.yaml         
|-- project1                  # automation main package
    `-- __main__.py           # automation main module with main step
    ... ...                  
|-- sharedpackage1            # shared package 1
|   `-- module1.py
    ... ...                  
|-- sharedpackage2            # shared package 2
|   `-- module2.py
    ... ...                  
|-- .entrypoint               # content: "entrypoint: project1"
|-- .packignore               # content: "configs/secrets.yaml" (optional)
|-- requirements.txt
`-- README.md

📘

PYTHONPATH requirement

Make sure your local PYTHONPATH includes paths to both shared_project_1 and shared_project_2.

This way, packages from both shared projects can be imported with statements 'from sharedpackage1 import ' and 'from sharedpackage2 import ', respectively, and those import statements don't need to be changed when the automation is run on the Seven Bridges Platform.

📘

Additional dependencies

All dependencies, if any, of sharedpackage1 or sharedpackage2 must be added to the requirements.txt.

Otherwise, those dependencies will not be installed when executing on the Seven Bridges Platform and you will receive an execution error.