Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Data and subgroup descriptions

Before choosing an algorithm, decide what a useful description may say. An exhaustive search cannot repair an unsuitable description language. A customer identifier can isolate individual people; a measurement collected after the outcome can turn a meaningful question into a trivial one.

Descriptor and target roles

Targets and discovery tasks require a pandas DataFrame. Condition generation and encoding also accept two-dimensional arrays, whose generated names are x0, x1, and so on. Use a DataFrame when labels carry meaning.

Exclude target columns explicitly when generating descriptors. Searches also exclude the fitted target’s attributes_, but a target-derived proxy remains the analyst’s responsibility. A useful starting list contains attributes known at the relevant time and understandable to the intended reader.

Conditions and conjunctions

[age > 25 ∧ plan=plus]
Loading...

Every condition must match. Duplicate index labels are allowed: coverage is defined by row position, not by index label. Rule is immutable and hashable; subgroup discovery returns rules without a consequent. A list of discovered subgroups is not a predictive Ruleset with a default class.

OperatorMeaningConstruction
eqPresent value equals a valueCondition("plan", "eq", "plus")
nePresent value differs from a valueCondition("plan", "ne", "plus")
inPresent value belongs to a setCondition("plan", "in", ("basic", "plus"))
le, gtNumeric thresholdCondition("age", "le", 30)
intervalLeft-open, right-closed intervalCondition("age", "interval", lower=20, upper=40)
is_missingValue is missingCondition("age", "is_missing")

An unbounded interval endpoint is None. For eq, ne, and in, values must be suitable hashable category values; missingness uses its own operator.

Choosing candidate conditions manually

You do not have to use automatic condition generation. Write an ordinary Python list, specifying each permitted condition individually:

All manually specified candidates:
plan=basic
plan=plus
age ≤ 30
age > 30
age ∈ (20, 40]
age is missing

Pass this list as search_space=manual_conditions when constructing a SubgroupDiscoveryTask. Each list entry is a separate candidate predicate; the algorithm decides which candidates to combine into conjunctions. This lets you supply domain-specific thresholds or intervals instead of accepting the cuts produced by generate_conditions.

Missing values and interval boundaries

Missing descriptors match only is_missing. In particular, ne is not the Boolean complement of eq when values are missing. This prevents “unknown plan” from silently becoming “a plan other than plus.”

Loading...

The interval includes ages 30 and 40, excludes 20, and does not match the missing age. These are (20, 40] semantics. Missing target values follow target-specific rules: for example, numeric targets require finite values, while BinaryTarget treats a missing label as negative unless the selected positive value is missing.

Choosing which endpoints to include

When writing candidates manually, you can also express all four choices of open or closed endpoints. The interval operator itself always uses (lower, upper]; it has no flags that change inclusivity. Instead, choose its cut points to represent the desired set of values.

For the finite float64 ages in this example, np.nextafter(value, -np.inf) returns the immediately preceding representable number. Moving the lower cut from 20 to that predecessor includes 20; moving the upper cut from 40 to its predecessor excludes 40:

Loading...

The labels in the table name the intended intervals; internally, each condition still has the library’s (lower, upper] form with its chosen numeric cuts. No other float64 value lies between a boundary and its predecessor, so these choices express inclusion or exclusion exactly for such data. For whole-number ages, the same idea is simpler: [20, 40] can be written with lower=19, upper=40, and [20, 40) with lower=19, upper=39. Those integer cut points would not represent the same intervals for fractional ages.

Each interval is a single candidate, so it can be used directly in standard searches that allow at most one condition per feature. Missing values remain excluded from all four variants.

Generating conditions

age ∈ (-∞, 30]
age ∈ (30, 40]
age ∈ (40, ∞]
age is missing
plan=basic
plan=plus
plan is missing

Categorical and low-cardinality numeric columns generate equality conditions. Higher-cardinality numeric columns are discretized:

By default, integer cuts use positions among sorted distinct values and float cuts use quantiles. A supplied scikit-learn KBinsDiscretizer, or per_feature_discretizers, gives more control. Categorical columns exceeding max_categories are skipped; inspect the generated list rather than assuming every column contributed candidates.

Coverage for manual calculations

Positional mask: [False  True False False False]
Count: 1
Loading...

dense stores Boolean condition masks; packed stores them bit-packed. Both expose the same coverage semantics. Packed encoding is useful when you need many condition counts with reduced storage, but does not change the description language.

The standard discovery searches permit at most one condition per feature. To express a bounded age range there, supply an interval as one candidate. DSSD has a different refinement operator and can combine certain conditions on the same feature.

Experiment. Replace the interval’s upper endpoint with 30. Then set its lower endpoint to None. Predict the matching rows before running the code.