Deployment options overview

This page gives quick examples of how to use the main deployment options in Meadowrun. Also see How deployment works for a look at what's happening under the covers.

We will assume you've gone through either the quickstart or one of the tutorials for a specific platform (AWS EC2, Azure VMs, GKE, or Kubernetes). In the examples, we'll use host and resources variables that we don't define here. You will need to define these variables as they were defined (as parameters) in the platform-specific tutorials.

mirror_local

mirror_local mirrors your local code and libraries. We won't go into further depth here as all of the tutorials use mirror_local by default (when no deployment is specified, mirror_local is used as the default).

git_repo

This option runs using code committed to a Git repo rather than the local code. For this example, we'll use our test repo at https://github.com/meadowdata/test_repo.

The Git repo must specify the environment/libraries to run in.

pipcondapoetry

For pip, you can run pip freeze > requirements.txt in an existing environment to produce a requirements.txt file that you commit to your git repo.

Then, you can run

meadowrun.run_function(
    lambda: sum(range(1000)) / 1000,
    host,
    resources,
    deployment=meadowrun.Deployment.git_repo(
        # URL to the repo
        https://github.com/meadowdata/test_repo
        # name of our environment file
        interpreter=meadowrun.PipRequirementsFile("requirements.txt")
)

For conda, you can run conda env export > conda_env_export.yml in an existing environment to produce a myenv.yml file that you commit to your git repo.

Conda environments are not cross-platform, so you'll need to make sure myenv.yml describes a conda environment that can be built on Linux.

Then, you can run

meadowrun.run_function(
    lambda: sum(range(1000)) / 1000,
    host,
    resources,
    deployment=meadowrun.Deployment.git_repo(
        # URL to the repo
        https://github.com/meadowdata/test_repo
        # name of our environment file
        interpreter=meadowrun.CondaEnvironmentFile("myenv.yml")
)

For poetry, you will usually already have committed the pyproject.toml and poetry.lock files to your git repo.

Then, you can run

meadowrun.run_function(
    lambda: sum(range(1000)) / 1000,
    host,
    resources,
    deployment=meadowrun.Deployment.git_repo(
        # URL to the repo
        https://github.com/meadowdata/test_repo
        # name of our environment file
        interpreter=meadowrun.PoetryProjectPath("")
)

Meadowrun will create an environment based on the file we specify and run the specified command in that environment.

In most cases, the environment must have meadowrun installed as a dependency.

For git repos that require authentication, see Use a private git repo on AWS or Use a private git repo on Azure.

container_image

In some cases, you might have all of the code and libraries you want already built in a container (for combining mirror_local or git_repo with a container image, see How deployment works).

Here's an example:

meadowrun.run_command(
    "python --version",
    host,
    resources,
    deployment=meadowrun.Deployment.container_image(
        repository="python", tag="slim-bullseye"
    ),
)

The container must have python installed and on the path. In most cases, meadowrun must be installed in the python that's on the path. (Although in this case, because we're using a simple run_command, we don't need Meadowrun installed.)

For images that require authentication, Use a private container on AWS, Use a private container on Azure, or Use a private container on Kubernetes.