clarite.modify.colfilter_min_cat_n

clarite.modify.colfilter_min_cat_n(data, n: int = 200, skip: Union[str, List[str], NoneType] = None, only: Union[str, List[str], NoneType] = None)

Remove binary and categorical variables which have less than <n> occurences of each unique value

Parameters:

data: pd.DataFrame

The DataFrame to be processed and returned

n: int, default 200

The minimum number of occurences of each unique value required in order for a variable not to be filtered

skip: str, list or None (default is None)

List of variables that the filter should not be applied to

only: str, list or None (default is None)

List of variables that the filter should only be applied to

Returns:

data: pd.DataFrame

The filtered DataFrame

Examples

>>> import clarite
>>> nhanes_filtered = clarite.modify.colfilter_min_cat_n(nhanes)
================================================================================
Running colfilter_min_cat_n
--------------------------------------------------------------------------------
WARNING: 36 variables need to be categorized into a type manually
Testing 362 of 362 binary variables
        Removed 248 (68.51%) tested binary variables which had a category with less than 200 values
Testing 47 of 47 categorical variables
        Removed 36 (76.60%) tested categorical variables which had a category with less than 200 values