Before you decide how to organize your automation code, it is important to consider whether or not you wish to run these automations on the Seven Bridges Platform. If you do, you have to follow certain guidelines to make sure your automation code is ready for platform execution.
When setting up a new automation project, Freyja offers a convenient 'init' sub-command that creates a Python project structure suitable for both local and platform execution. We highly recommend using this command when creating new automation projects, but you can create the required project structure also manually (not recommended).
For an automation named
project_1 the project structure generated by 'freyja init' looks as follows (including hidden files):
project_1 |-- configs | `-- secrets.yaml # configuration file for secret settings (optional) |-- project1 # automation main package (required) | `-- __main__.py # module containing automation main step (required) |-- .entrypoint # contains name of main package (i.e. "entrypoint: project1") |-- .packignore # files and directories to exclude when creating code package file (optional) |-- requirements.txt # Python dependencies (required for platform execution) `-- README.md # Readme file (optional)
To execute this automation locally, type
python -m project1 run [args] from inside the outer
To prepare a code package for execution on the Seven Bridges Platform, first change to the outer
project_1 directory and then type
The build command first infers the input-output schema from your automation main step, stores the schema in a file named '.schema' inside your project root, then gathers all files and directories inside your project root (excluding files mentioned inside
.packignore), and finally compresses everything into a zip file named
project_1.zip (the code package file).
The code package file can then be used as input to the 'sb automations packages create' command of the Seven Bridges Automation CLI, which uploads the code package file to the Seven Bridges Platform and creates a new code package entity. For more details, please refer to the previous section that contains a step-by-step guide on how to deploy and run an automation on the Seven Bridges Platform.
Note that for platform execution to work, all dependencies of 'project_1' must either be:
- Packaged inside the code package file, or
- Listed as pip-installable libraries inside 'requirements.txt'.
However, automations might have shared source code dependencies that are unpublished and do not reside inside an automation project. In cases like this, the 'freyja build' command allows you to include those dependencies during code package file creation.
Here is an example for how you can organize multiple automation projects that have shared and non-pip-installable source code dependencies:
repo_1 # git repo with multiple automation projects |-- project_1 # automation project 1 |-- configs | `-- secrets.yaml # or other config files (optional) |-- project1 | `-- __main__.py ... ... |-- .entrypoint |-- .packignore |-- requirements.txt `-- README.md |-- project_2 # automation project 2 |-- configs | `-- secrets.yaml # or other config files (optional) |-- project2 | `-- __main__.py ... ... |-- .entrypoint |-- .packignore |-- requirements.txt `-- README.md |-- shared_project_1 # shared project 1 (optional) |-- sharedpackage1 | `-- module1.py ... ... |-- README.md `-- setup.py repo_2 # additional git repo (optional) |-- shared_project_2 # shared project 2 (optional) |-- sharedpackage2 | `-- module2.py ... ... |-- README.md `-- setup.py
In the above setup, we have a main git repo ('repo_1') with two automation projects ('project_1' and 'project_2') and one shared project ('shared_project_1'). In addition, we have a second git repo ('repo_2') that contains another shared project ('shared_project_2').
When you build the code package file for project_1, first change directory into 'project_1' and then use the following command:
~/project_1$ freyja build -i ../shared_project_1/sharedpackage1 -i /path/to/repo_2/shared_project_2/sharedpackage2
This will include
sharedpackage2 as dependencies inside the generated code package file.
The file structure inside the created zip file should look like this:
zip root |-- configs `-- secrets.yaml |-- project1 # automation main package `-- __main__.py # automation main module with main step ... ... |-- sharedpackage1 # shared package 1 | `-- module1.py ... ... |-- sharedpackage2 # shared package 2 | `-- module2.py ... ... |-- .entrypoint # content: "entrypoint: project1" |-- .packignore # content: "configs/secrets.yaml" (optional) |-- requirements.txt `-- README.md
Make sure your local
PYTHONPATHincludes paths to both
This way, packages from both shared projects can be imported with statements 'from sharedpackage1 import ' and 'from sharedpackage2 import ', respectively, and those import statements don't need to be changed when the automation is run on the Seven Bridges Platform.
All dependencies, if any, of
sharedpackage2must be added to the
Otherwise, those dependencies will not be installed when executing on the Seven Bridges Platform and you will receive an execution error.
Updated over 2 years ago