hyperparameter-optimization

Search before asking

• I searched the issues and found no similar issues.

Ray Component

Ray Serve

What happened + What you expected to happen

See repro script, a fix would be avoid reconfiguring when there are inflight queries. Additionally, we should consider not accepting new queries when reconfigure is being called.

Reprod

Discussed in microsoft/nni#4070

[2021-08-14 10:13:41] INFO (NNIDataStore) Datastore initialization done
[2021-08-14 10:13:41] INFO (RestServer) RestServer start
[2021-08-14 10:13:41] INFO (RestServer) RestServer base port is 8080
[2021-08-14 10:13:41] I

Building the doc fails for example 40_advanced/example_single_configurations on the current development branch

Logs here

...
generating gallery for examples/40_advanced... [ 50%] example_debug_logging.py

Warning, treated as error:
/home/runner/work/auto-sklearn/auto-sklearn/examples/40_advanced/example_single_configu

Motivation

OptunaSearchCV allows to set a scoring function/string. However there is no option to tell it if the score needs to be minimized or maximized.

Description

Add an additional argument to OptunaSearchCV that tells optuna if the score should be maximized or minimized.
(like in rays TuneSearchCV where such a mode argument exists.)

When running TabularPredictor.fit(), I encounter a BrokenPipeError for some reason.
What is causing this?
Could it be due to OOM error?

Fitting model: XGBoost ...
-34.1179 = Validation root_mean_squared_error score
10.58s = Training runtime
0.03s = Validation runtime
Fitting model: NeuralNetMXNet ...
-34.2849 = Validation root_mean_squared_error score
43.63s =

Details in discussion mljar/mljar-supervised#421

I think it would be useful to have a grid search optimizer in this package. But its implementation would probably be quite different from other ones (sklearn, ...).

The requirements are:

• The grid search has to stop after n_iter instead of searching the entire search space
• The positions should not be precalculated at the beginning of the optimization (i have concerns about memory load).

If enter_data() is called with the same train_path twice in a row and the data itself hasn't changed, a new Dataset does not need to be created.

We should add a column which stores some kind of hash of the actual data. When a Dataset would be created, if the metadata and data hash are exactly the same as an existing Dataset, nothing should be added to the ModelHub database and the existing

Describe the bug
Code could be more conform to pep8 and so forth.

To Reproduce
https://app.codacy.com/gh/Neuraxio/Neuraxle/issues?&filters=W3siaWQiOiJMYW5ndWFnZSIsInZhbHVlcyI6W119LHsiaWQiOiJDYXRlZ29yeSIsInZhbHVlcyI6WyJDb2RlU3R5bGUiXX0seyJpZCI6IkxldmVsIiwidmFsdWVzIjpbXX0seyJpZCI6IlBhdHRlcm4iLCJ2YWx1ZXMiOltdfSx7ImlkIjoiQXV0aG9yIiwidmFsdWVzIjpbXX1d

Expected behavior
Less code st