comet_ml.Optimizer ¶

The Optimizer constructor.

Parameters:

• config (str | dict, default: None ) –

  Can an optimizer config id, an optimizer config dictionary, or an optimizer config filename.

• trials (int, default: None ) –

  Number of trials per parameter value set

• verbose (int, default: 1 ) –

  Level of details to show; 0 means quiet

• experiment_class (str | callable, default: 'Experiment' ) –

  Supported values are "Experiment" (the default) to use online Experiments or "OfflineExperiment" to use offline Experiments. It can also be a callable (a function or a method) that returns an instance of Experiment, OfflineExperiment, ExistingExperiment or ExistingOfflineExperiment.

See above for examples.

Optimizer.end() is called at end of experiment. Usually, you would not call this manually, as it is called directly when the experiment ends.

Optimizer.get_experiments() will iterate over all possible experiments for this sweep or search, n at a time. All experiments will have a unique set of parameter values (unless performing multiple trials per parameter set).

1
2
3
4
5
6
7
8

import comet_ml

comet_ml.login()
opt = comet_ml.Optimizer({"algorithm": "bayes", ...})

for experiment in opt.get_experiments():
    loss = fit(x, y)
    experiment.log_metric("loss", loss)

Get the id of this optimizer.

1
2
3
4
5
6

import comet_ml

comet_ml.login()
opt = comet_ml.Optimizer({"algorithm": "bayes", ...})

opt.get_id()

Optimizer.get_parameters() will iterate over all possible parameters for this sweep or search. All parameters combinations will be emitted once (unless performing multiple trials per parameter set).

1
2
3
4
5
6
7
8
9

import comet_ml

comet_ml.login()
opt = comet_ml.Optimizer({"algorithm": "bayes", ...})

for parameters in optimizer.get_parameters():
    experiment = comet_ml.start()
    loss = fit(x, y)
    experiment.log_metric("loss", loss)

Optimizer.next() will return the next experiment for this sweep or search. All experiments will have a unique set of parameter values (unless performing multiple trials per parameter set).

Normally, you would not call this directly, but use the generator Optimizer.get_experiments()

Parameters:

• kwargs (Any, default: {} ) –

  Any keyword argument will be passed to the Experiment class for creation. The API key is passed directly.

1
2
3
4
5
6
7
8
9

import comet_ml

comet_ml.login()
opt = comet_ml.Optimizer({"algorithm": "bayes", ...})

experiment = optimizer.next()
if experiment is not None:
    loss = fit(x, y)
    experiment.log_metric("loss", loss)

Optimizer.next_data() will return the next parameters in the Optimizer sweep.

Normally, you would not call this directly, but use the generator Optimizer.get_parameters()

1
2
3
4
5

import comet_ml

comet_ml.login()
opt = comet_ml.Optimizer({"algorithm": "bayes", ...})
experiment = optimizer.next_data()

Set metadata for result details.

This method replaces any previously set metadata for this optimizer instance; it does not merge with existing metadata. The metadata is stored per optimizer instance and persists for the lifetime of the optimizer object, not per experiment. Args: metadata (dict): a dictionary of result details

Get the status from the optimizer server for this optimizer.

1
2
3
4
5
6

import comet_ml

comet_ml.login()
opt = comet_ml.Optimizer({"algorithm": "bayes", ...})

opt.status()

{
    'algorithm': 'grid',
    'comboCount': 32,
    'configSpaceSize': 10,
    'endTime': None,
    'id': 'c20b90ecad694e71bdb5702778eb8ac7',
    'lastUpdateTime': None,
    'maxCombo': 0,
    'name': 'c20b90ecad694e71bdb5702778eb8ac7',
    'parameters': {
        'x': {
            'max': 20,
            'min': 0,
            'scalingType': 'uniform',
            'type': 'integer'
        }
    },
    'retryCount': 0,
    'spec': {
        'gridSize': 10,
        'maxCombo': 0,
        'metric': 'loss',
        'minSampleSize': 100,
        'randomize': False,
        'retryLimit': 20,
        'seed': 2997936454
    },
    'startTime': 1558793292216,
    'state': {
        'sequence_i': 0,
        'sequence_retry': 11,
        'sequence_trial': 0,
        'total': 32,
    },
    'status': 'running',
    'trials': 1,
    'version': '1.0.0'
}

Update an optimizer assignment status and metadata given the experiment.

Parameters:

• experiment (CometExperiment) –

  an Offline or Online Experiment instance

• status (str) –

  a status, such as "running"

• metadata (dict) –

  a dictionary of experiment info