automl / automl/Auto-PyTorch

Make use of data more by devising subsampling

Open
#426 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.5k
Forks
300
PR merge metrics
No merged PRs in 30d

Description

When we use a certain `memory_allocation` [^1] in `subsampling`, we reduce the number of samples until we reach the memory limit.
However, we need to come up with an appropriate value for this as when we set it too high, the training fails due to memory error while when we set it too low, we waste memory.

For now, we circumvent this issue by measuring the memory consumption when using the default config.

[^1]: The definition of the `memory_allocation` is the following:
Absolute memory in MB, e.g. 10MB is ``"memory_allocation": 10``.
The memory used by the dataset is checked after each reduction method is performed.
If the dataset fits into the allocated memory, any further methods listed in ``"methods"`` will not be performed.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.