Save, Push and Pull

Save, Push and Pull#

This tutorial explains how to save optimization experiments using MINTO and push them to a remote repository.

Save#

In minto, there are two methods for saving data locally: “file-based saving” and “OMMX Archive-based saving”.

File-based saving#

There are two methods for file-based saving.

experiment = minto.Experiment("sample", auto_saving=True)

As shown above, when you specify auto_saving=True, it automatically saves in file-based format each time the .log_* method is called.

Also, after the experiment is finished, you can

experiment.save("path/to/directory")

call the save method like this to perform file-based saving to the specified directory.

OMMX Archive-based saving#

experiment.save_as_ommx_archive("path/to/archive.minto.ommx")

By calling the save_as_ommx_archive method like this, you can save it as a single compressed file based on OMMX Archive.

Push and Pull to GitHub Packages#

Here we explain how to push experiments managed by MINTO to GitHub Packages using the OMMX Artifact feature, and how other users can pull these experiments. OMMX Artifact can also push experiment results to other container registries. By pushing to GitHub Packages, both code and experiment results can be managed on GitHub, making it easy to share with the team.

First, let’s conduct a numerical experiment for mathematical optimization. Here, we will use the same one as in “Quick Start”. Please see the original tutorial page for details.

import minto
import ommx_pyscipopt_adapter as scip_ad
from ommx.dataset import miplib2017

instance_name = "reblock115"
instance = miplib2017(instance_name)

timelimit_list = [0.1, 0.5, 1, 2]

experiment = minto.Experiment("scip_reblock115", auto_saving=False, verbose_logging=False)

experiment.log_global_instance(instance_name, instance)
adapter = scip_ad.OMMXPySCIPOptAdapter(instance)
scip_model = adapter.solver_input

for timelimit in timelimit_list:
    with experiment.run() as run:
        run.log_parameter("timelimit", timelimit)

        # Solve by SCIP
        scip_model.setParam("limits/time", timelimit)
        scip_model.optimize()
        solution = adapter.decode(scip_model)

        run.log_solution("scip", solution)

experiment.get_run_table()

Push to GitHub Packages#

Let’s learn how to push experiment results to GitHub Packages.

Prerequisites#

  1. You need a GitHub account and repository.

  2. If you haven’t created one yet, refer to GitHub’s documentation on repository creation.

To push to GitHub Packages, authentication using a Personal Access Token (PAT) or GITHUB_TOKEN is required in GitHub Actions. Create a PAT with write:packages permission.

For how to create a PAT, refer to GitHub’s documentation.

Using the obtained token, authenticate using one of the following two methods:

  1. Using CLI Execute ommx login to save authentication information on your local machine

    ommx login --username=<GITHUB_USERNAME> --password=<PERSONAL_ACCESS_TOKEN> https://ghcr.io

  2. Using environment variables

    • Set the following environment variables

     env:
     OMMX_BASIC_AUTH_DOMAIN: ghcr.io
     OMMX_BASIC_AUTH_USERNAME: ${{ github.actor }}
     OMMX_BASIC_AUTH_PASSWORD: ${{ github.token }}

    • For OMMX_BASIC_AUTH_PASSWORD, you need to create a GitHub Personal Access Token. To create a Personal Access Token, follow GitHub’s guide and enable the write:packages permission

Note: Push and pull to GitHub Packages uses the OMMX Artifact feature. Therefore, settings such as GitHub authentication are the same as OMMX Artifact. For details, refer to this notebook.

push#

Once authentication is set up, use the Experiment.push_github method to push to GitHub Packages. This method has the following parameters:

  • org: Organization name in GitHub

  • repo: GitHub repository name

  • name: Artifact name (default is Experiment.name)

  • tag: Tag name (default is Experiment.timestamp.strftime('%Y%m%d%H%M%S'))

# If environment variables are not set, please use the following to set them.
# Environment variables

# os.environ["OMMX_BASIC_AUTH_DOMAIN"] = 'ghcr.io'
# os.environ["OMMX_BASIC_AUTH_USERNAME"] = <GITHUB_USERNAME>
# os.environ["OMMX_BASIC_AUTH_PASSWORD"] = <PERSONAL_ACCESS_TOKEN>

Let’s try pushing to GitHub Packages using the Experiment.push_github method right away. As mentioned above, you don’t need to specify the name and tag parameters. This method returns an ommx.Artifact object, and you can check the pushed image name using ommx.Artifact.image_name.

artifact = experiment.push_github(org="Jij-Inc", repo="MINTO-Public")
artifact.image_name

Pull from GitHub Packages#

Next, let’s pull the image pushed to GitHub Packages. To do this, we use the minto.Experiment.load_from_registry method. This method supports pulling not only from GitHub Packages but also from other container registries. Therefore, you need to specify the image name directly.

exp2 = minto.Experiment.load_from_registry(
    "ghcr.io/jij-inc/minto-public/scip_reblock115:latest5"
)

Let’s look at the pulled data.

exp2.get_run_table()

Summary#

We have seen how to save data locally using minto (file-based, OMMX Archive-based) and how to push and pull to GitHub Packages.

minto makes it easy to share data with others by providing features that make data handling easier.

Contents