EpistasisLab / EpistasisLab/tpot

XGBoost parameter error (colsample_bytree=1)

Open
#449 12 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Jupyter Notebook
Stars
10.1k
Forks
1.6k
PR merge metrics
No merged PRs in 30d

Description

I've seen some traffic on these issues regarding potentially getting rid of xgboost altogether due to dependency troubles, so if that is the case then this isn't relevant.

I am receiving the following error message:

Optimization Progress: 0%| | 26/10100 [00:22<2:10:08, 1.29pipeline/s][
08:35:46] c:\dev\libs\xgboost\dmlc-core\include\dmlc\./logging.h:235: [08:35:46] C:\dev\libs\xgboost
\src\tree\updater_colmaker.cc:162: Check failed: (n) > (0U) colsample_bytree=1 is too small that no
feature can be included

I know that the **colsample_bytree** parameter should be what proportion of the features you're allowing each tree to randomly sample from in order to build itself. So a colsample_bytree=1 should be telling each tree to sample from 100% of the columns/features when building a tree. (Please correct me if I'm wrong on that!)

[xgboost colsample_bytree = subsample ratio of columns when constructing each tree.](https://github.com/dmlc/xgboost/blob/master/doc/parameter.md)

This has also been [previously raised as an issue on xgboost's github repo](https://github.com/dmlc/xgboost/issues/150), but that issue was closed without really any explanation of what the user was doing wrong.

My guess is that this would be an error with what parameters are being passed into XGBoost and not necessarily an xgboost issue.

## Context of the issue

My environment:

* Windows 7 - 64-bit OS
* Python3.4 - 64-bit
* pandas version: 0.20.1
* numpy version: 1.12.1
* scipy version: 0.19.0
* tpot version: 0.7.3
* sklearn version: 0.18.1
* xgboost version: 0.6

## Process to reproduce the issue

This is my simple script to reproduce the error in my environment with random data. This error doesn't tend to occur when my `generations` and `population_size` are low (around 10-15 each). I have experienced this issue with generation/population_size as low as 32 (with this same script below). Hopefully this short script is sufficiently reproducible!

print("importing modules...")
import pandas as pd
import numpy as np
import tpot
from tpot import TPOTClassifier
from sklearn.model_selection import train_test_split
import sklearn
import scipy
import xgboost as xgb
from random import randint

# I wanted the label data to be a bit imbalanced
print("creating fake data...")
np.random.seed(1776)
df = pd.DataFrame(np.random.randn(8000,11), columns=list("ABCDEFGHIJK"))
label = np.array([randint(1,11) for mynumber in range(0, 8000)])
label[label <= 9] = 0
label[label >= 10] = 1
print(label)
df['label'] = label


# extract labels and drop them from the DataFrame
y = df['label'].values
colsToDrop = ['label']
xdf = df.drop(colsToDrop, axis=1)


x_train, x_test, y_train, y_test = train_test_split(xdf, y, train_size=0.7, test_size=0.3, random_state=1776)

# this will error out:
tpot = TPOTClassifier(generations=100, population_size=100, verbosity=2,
scoring="balanced_accuracy", cv=5, random_state=1776)
tpot.fit(x_train, y_train)

I couldn't find any prior issues that addressed this specific error I keep running into, but I apologize if I may have missed one.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.