{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Quick Start\n",
"\n",
"This guide will walk you through the initial steps to leverage minto. In this section, we will cover four main steps from setting up minto to handling experimental data.\n",
"\n",
"- How to install minto\n",
"- Recording experimental data\n",
"- Viewing recorded experimental data in a table\n",
"- Saving and loading experimental data\n",
"\n",
"## Installation\n",
"\n",
"```bash\n",
"pip install minto\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Record Experimental Data\n",
"\n",
"With `minto` you can easily record various data generated during experiments. \n",
"In the following example, we will conduct numerical experiments on the time limit dependency of MIP solvers using PySCIPOpt, which is supported by OMMX.\n",
"`minto` natively supports OMMX Message, allowing us to smoothly perform numerical experiments through OMMX.\n",
"\n",
"Let's give it a try."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import minto\n",
"import ommx_pyscipopt_adapter as scip_ad\n",
"from ommx.dataset import miplib2017"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this tutorial, we will pick up an instance from miplib2017 as a benchmark target. We can easily obtain miplib2017 instances using ommx.dataset."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"instance_name = \"reblock115\"\n",
"instance = miplib2017(instance_name)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Using the `ommx_pyscipopt_adapter`, we convert ommx.v1.Instance to PySCIPOpt's Model and conduct experiments by varying the limits/time parameter.\n",
"\n",
"The ommx instance and solution can be saved using MINTO's `.log_*` methods. Since we're using a single instance that doesn't change throughout this numerical experiment, we store it in the `experiment` space outside the with block. \n",
"Solutions are saved within the with block (in the `run` space) since they vary for each time limit."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"timelimit_list = [0.1, 0.5, 1, 2]\n",
"\n",
"# When auto_saving is True, data is saved automatically whenever Experiment.log_* methods are called.\n",
"# Since data is saved progressively, even if an error occurs midway, the data up to that point won't be lost.\n",
"experiment = minto.Experiment(\n",
" name='scip_exp',\n",
" auto_saving=True\n",
")\n",
"\n",
"experiment.log_instance(instance_name, instance)\n",
"adapter = scip_ad.OMMXPySCIPOptAdapter(instance)\n",
"scip_model = adapter.solver_input\n",
"\n",
"for timelimit in timelimit_list:\n",
" with experiment.run():\n",
" experiment.log_parameter(\"timelimit\", timelimit)\n",
"\n",
" # Solve by SCIP\n",
" scip_model.setParam(\"limits/time\", timelimit)\n",
" scip_model.optimize()\n",
" solution = adapter.decode(scip_model)\n",
"\n",
" experiment.log_solution(\"scip\", solution)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"When converting ommx.Solution to pandas.DataFrame using the `.get_run_table` method, only the main information of the solution is displayed. If you want to access the actual solution objects, you can reference them from `experiment.dataspaces.run_datastores[run_id].solutions`."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" solution_scip \n",
" parameter \n",
" metadata \n",
" \n",
" \n",
" objective \n",
" feasible \n",
" optimality \n",
" relaxation \n",
" start \n",
" name \n",
" timelimit \n",
" run_id \n",
" elapsed_time \n",
" \n",
" \n",
" run_id \n",
" \n",
" \n",
" \n",
" \n",
" 0 \n",
" 0.000000e+00 \n",
" True \n",
" 0 \n",
" 0 \n",
" None \n",
" scip \n",
" 0.1 \n",
" 0 \n",
" 0.119688 \n",
" \n",
" \n",
" 1 \n",
" -3.027709e+07 \n",
" True \n",
" 0 \n",
" 0 \n",
" None \n",
" scip \n",
" 0.5 \n",
" 1 \n",
" 0.399354 \n",
" \n",
" \n",
" 2 \n",
" -3.027709e+07 \n",
" True \n",
" 0 \n",
" 0 \n",
" None \n",
" scip \n",
" 1.0 \n",
" 2 \n",
" 0.507941 \n",
" \n",
" \n",
" 3 \n",
" -3.027709e+07 \n",
" True \n",
" 0 \n",
" 0 \n",
" None \n",
" scip \n",
" 2.0 \n",
" 3 \n",
" 1.009734 \n",
" \n",
" \n",
"
\n",
"
"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"x = runs_table[\"parameter\", \"timelimit\"]\n",
"y = runs_table[\"solution_scip\", \"objective\"]\n",
"plt.plot(x, y, \"o-\")\n",
"plt.xlabel(\"Time limit\")\n",
"plt.ylabel(\"Objective\")\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Save and Load Experimental Data\n",
"\n",
"By setting `auto_saving` to `False` in the `Experiment` constructor, you can disable automatic saving. In this case, you can save data at any time using the `Experiment.save` method. \n",
"You can also load saved data using the `Experiment.load_from_dir` method."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"# By default, data is saved under .minto_experiments/ directory\n",
"experiment.save()"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"exp2 = minto.Experiment.load_from_dir(\n",
" \".minto_experiments/\" + experiment.experiment_name\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" solution_scip \n",
" parameter \n",
" metadata \n",
" \n",
" \n",
" objective \n",
" feasible \n",
" optimality \n",
" relaxation \n",
" start \n",
" name \n",
" timelimit \n",
" run_id \n",
" elapsed_time \n",
" \n",
" \n",
" run_id \n",
" \n",
" \n",
" \n",
" \n",
" 0 \n",
" 0.000000e+00 \n",
" True \n",
" 0 \n",
" 0 \n",
" None \n",
" scip \n",
" 0.1 \n",
" 0 \n",
" 0.119688 \n",
" \n",
" \n",
" 1 \n",
" -3.027709e+07 \n",
" True \n",
" 0 \n",
" 0 \n",
" None \n",
" scip \n",
" 0.5 \n",
" 1 \n",
" 0.399354 \n",
" \n",
" \n",
" 2 \n",
" -3.027709e+07 \n",
" True \n",
" 0 \n",
" 0 \n",
" None \n",
" scip \n",
" 1.0 \n",
" 2 \n",
" 0.507941 \n",
" \n",
" \n",
" 3 \n",
" -3.027709e+07 \n",
" True \n",
" 0 \n",
" 0 \n",
" None \n",
" scip \n",
" 2.0 \n",
" 3 \n",
" 1.009734 \n",
" \n",
" \n",
"
\n",
"
\n",
"\n",
"
\n",
" \n",
" \n",
" solution_scip \n",
" parameter \n",
" metadata \n",
" \n",
" \n",
" objective \n",
" feasible \n",
" optimality \n",
" relaxation \n",
" start \n",
" name \n",
" timelimit \n",
" run_id \n",
" elapsed_time \n",
" \n",
" \n",
" run_id \n",
" \n",
" \n",
" \n",
" \n",
" 0 \n",
" 0.000000e+00 \n",
" True \n",
" 0 \n",
" 0 \n",
" None \n",
" scip \n",
" 0.1 \n",
" 0 \n",
" 0.119688 \n",
" \n",
" \n",
" 1 \n",
" -3.027709e+07 \n",
" True \n",
" 0 \n",
" 0 \n",
" None \n",
" scip \n",
" 0.5 \n",
" 1 \n",
" 0.399354 \n",
" \n",
" \n",
" 2 \n",
" -3.027709e+07 \n",
" True \n",
" 0 \n",
" 0 \n",
" None \n",
" scip \n",
" 1.0 \n",
" 2 \n",
" 0.507941 \n",
" \n",
" \n",
" 3 \n",
" -3.027709e+07 \n",
" True \n",
" 0 \n",
" 0 \n",
" None \n",
" scip \n",
" 2.0 \n",
" 3 \n",
" 1.009734 \n",
" \n",
" \n",
"
\n",
"