Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Class distributions, multiple targets, and WKL

There are several distinct ways for a target to be multivariate. A nominal class distribution, a vector of numeric means, and several categorical model attributes require different representations and measures.

One categorical attribute with several classes

Instead of choosing one Iris species as positive, ask where the species distribution differs from its population distribution:

Loading...

Let pG\boldsymbol p_G and pD\boldsymbol p_D be the subgroup and dataset class shares. Available choices include:

MeasureQuality
SplitQFn∑c(pG,c−pD,c)2n\sum_c(p_{G,c}-p_{D,c})^2
MulticlassChiSquaredQFn∑c(pG,c−pD,c)2/pD,cn\sum_c(p_{G,c}-p_{D,c})^2/p_{D,c}
GiniQFn/(N−n)∑c(pG,c−pD,c)2n/(N-n)\sum_c(p_{G,c}-p_{D,c})^2
ClassDistributionL1QF$(n/N)\sum_c
MaxConfidenceQFmax⁡cpG,c\max_c p_{G,c}

The first three measures and their bounds follow the DpSubgroup report Grosskreutz et al., 2008; the L1 measure connects to election-distribution analysis Grosskreutz et al., 2010. MaxConfidenceQF favors purity, rather than explicitly subtracting the dataset distribution. Missing class labels are rejected by MulticlassTarget.

Several numeric target attributes

Suppose both petal length and width are outcomes of interest. Exclude both from the descriptor frame and use the sepal measurements as descriptions:

Loading...

MeanVectorL1QF computes

qdst(G)=nN∥μG−μD∥1.q_{dst}(G)=\frac{n}{N}\left\|\boldsymbol\mu_G-\boldsymbol\mu_D\right\|_1.

It sums absolute coordinate shifts. Units matter: a coordinate measured in thousands can dominate one measured in fractions. Choose units or preprocessing that reflect the scientific question. The measure does not directly capture changes in correlation with unchanged means; use EMM for that.

The package derives tight optimistic estimates for the L1 measures. Their exact computation grows exponentially in the number of classes/dimensions; above max_exact_classes or max_exact_dimensions, a looser component-wise bound is used. These switches affect pruning, not the quality definition.

Several nominal model attributes: weighted KL divergence

NominalTarget stores multiple categorical attributes. The WKL quality of Leeuwen & Knobbe (2012) compares each attribute’s marginal distribution with its dataset distribution:

WKL(G)=nN∑j∑vPG(Mj=v)log⁡2PG(Mj=v)PD(Mj=v).WKL(G)=\frac{n}{N}\sum_j\sum_v P_G(M_j=v)\log_2\frac{P_G(M_j=v)}{P_D(M_j=v)}.

A zero subgroup probability contributes zero. A category observed in a subgroup also occurs in the containing dataset, so its dataset probability is positive. WKL is nonnegative and uses bits. Its size weight balances divergence against the amount of data represented.

Here we discretize the two petal targets, while retaining sepal descriptors:

Loading...

WeightedKLQF also accepts binary and multiclass targets. For multiple nominal attributes it sums marginal divergences: this is not the divergence of their full joint distribution. A pure association change can leave every marginal unchanged and receive zero WKL. The association models in the next chapter address relationships between attributes.

Missing nominal model values count as a category of their own. WKL currently has no optimistic estimate, so exhaustive searches cannot prune by this quality. It also supplies the distributional coding interpretation used by compression-based DSSD.

Experiment. Duplicate one nominal model attribute under a second name and fit both as targets. Why does this double WKL? What does that tell you about choosing correlated or redundant model attributes?

References
  1. Grosskreutz, H., Rueping, S., Shaabani, N., & Wrobel, S. (2008). Optimistic Estimate Pruning Strategies for Fast Exhaustive Subgroup Discovery [Techreport]. Fraunhofer IAIS. 10.24406/publica-fhg-293834
  2. Grosskreutz, H., Boley, M., & Krause-Traudes, M. (2010). Subgroup Discovery for Election Analysis: A Case Study in Descriptive Data Mining. Discovery Science.
  3. van Leeuwen, M., & Knobbe, A. (2012). Diverse Subgroup Set Discovery. Data Mining and Knowledge Discovery, 25, 208–242.