Class distributions, multiple targets, and WKL
There are several distinct ways for a target to be multivariate. A nominal class distribution, a vector of numeric means, and several categorical model attributes require different representations and measures.
One categorical attribute with several classes¶
Instead of choosing one Iris species as positive, ask where the species distribution differs from its population distribution:
from sklearn.datasets import load_iris
import pandas as pd
import pysubgroup as sd
X, y = load_iris(return_X_y=True, as_frame=True)
data = X.assign(species=y)
target = sd.MulticlassTarget().fit(data, "species")
conditions = sd.generate_conditions(X, n_bins=3)
task = sd.SubgroupDiscoveryTask(data, target, conditions, sd.ClassDistributionL1QF(),
result_set_size=5, depth=2,
filters=[sd.MinSizeFilter(15)])
results = sd.DFS().execute(task)
sd.to_dataframe(results, data, target, ["size", "class_shares", "majority_class", "majority_share"])Let and be the subgroup and dataset class shares. Available choices include:
| Measure | Quality |
|---|---|
SplitQF | |
MulticlassChiSquaredQF | |
GiniQF | |
ClassDistributionL1QF | $(n/N)\sum_c |
MaxConfidenceQF |
The first three measures and their bounds follow the DpSubgroup report
Grosskreutz et al., 2008; the L1 measure connects to election-distribution
analysis Grosskreutz et al., 2010. MaxConfidenceQF favors purity, rather than
explicitly subtracting the dataset distribution. Missing class labels are
rejected by MulticlassTarget.
Several numeric target attributes¶
Suppose both petal length and width are outcomes of interest. Exclude both from the descriptor frame and use the sepal measurements as descriptions:
model_columns = ["petal length (cm)", "petal width (cm)"]
vector_target = sd.VectorTarget().fit(data, *model_columns)
vector_conditions = sd.generate_conditions(data[["sepal length (cm)", "sepal width (cm)"]], n_bins=4)
vector_task = sd.SubgroupDiscoveryTask(data, vector_target, vector_conditions,
sd.MeanVectorL1QF(), result_set_size=5, depth=2,
filters=[sd.MinSizeFilter(15)])
vector_results = sd.DFS().execute(vector_task)
sd.to_dataframe(vector_results, data, vector_target)MeanVectorL1QF computes
It sums absolute coordinate shifts. Units matter: a coordinate measured in thousands can dominate one measured in fractions. Choose units or preprocessing that reflect the scientific question. The measure does not directly capture changes in correlation with unchanged means; use EMM for that.
The package derives tight optimistic estimates for the L1 measures. Their exact
computation grows exponentially in the number of classes/dimensions; above
max_exact_classes or max_exact_dimensions, a looser component-wise bound is
used. These switches affect pruning, not the quality definition.
Several nominal model attributes: weighted KL divergence¶
NominalTarget stores multiple categorical attributes. The WKL quality of
Leeuwen & Knobbe (2012) compares each attribute’s marginal distribution with
its dataset distribution:
A zero subgroup probability contributes zero. A category observed in a subgroup also occurs in the containing dataset, so its dataset probability is positive. WKL is nonnegative and uses bits. Its size weight balances divergence against the amount of data represented.
Here we discretize the two petal targets, while retaining sepal descriptors:
nominal_data = data.assign(
length_band=pd.cut(data["petal length (cm)"], [-float("inf"), 2, 5, float("inf")]),
width_band=pd.cut(data["petal width (cm)"], [-float("inf"), 0.6, 1.7, float("inf")]),
)
nominal_target = sd.NominalTarget().fit(nominal_data, "length_band", "width_band")
nominal_task = sd.SubgroupDiscoveryTask(nominal_data, nominal_target, vector_conditions,
sd.WeightedKLQF(), result_set_size=5, depth=2,
filters=[sd.MinSizeFilter(15)])
nominal_results = sd.DFS().execute(nominal_task)
sd.to_dataframe(nominal_results, nominal_data, nominal_target)WeightedKLQF also accepts binary and multiclass targets. For multiple nominal
attributes it sums marginal divergences: this is not the divergence of their
full joint distribution. A pure association change can leave every marginal
unchanged and receive zero WKL. The association models in the next chapter
address relationships between attributes.
Missing nominal model values count as a category of their own. WKL currently has no optimistic estimate, so exhaustive searches cannot prune by this quality. It also supplies the distributional coding interpretation used by compression-based DSSD.
Experiment. Duplicate one nominal model attribute under a second name and fit both as targets. Why does this double WKL? What does that tell you about choosing correlated or redundant model attributes?
- Grosskreutz, H., Rueping, S., Shaabani, N., & Wrobel, S. (2008). Optimistic Estimate Pruning Strategies for Fast Exhaustive Subgroup Discovery [Techreport]. Fraunhofer IAIS. 10.24406/publica-fhg-293834
- Grosskreutz, H., Boley, M., & Krause-Traudes, M. (2010). Subgroup Discovery for Election Analysis: A Case Study in Descriptive Data Mining. Discovery Science.
- van Leeuwen, M., & Knobbe, A. (2012). Diverse Subgroup Set Discovery. Data Mining and Knowledge Discovery, 25, 208–242.