Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Specialized tasks and connections

The same condition language and search interface can support problems beyond “find an unusual label share.” Two examples are target-free pattern mining and the search for a good rule in a boosting objective.

Target-free pattern mining

With no designated outcome, FITarget stores only the dataset size. CountQF ranks descriptions by the number of rows they cover:

Loading...

Manual positive-item conditions avoid filling the result list with absence patterns if only present items matter. generate_conditions would also offer conditions for false values.

AreaQF multiplies coverage by description length:

Loading...

It can rank longer, moderately frequent descriptions ahead of shorter ones. It has no optimistic estimate in this interface and needs depth= for manual evaluation. The searches and result table helper supply depth automatically. The randomization significance helpers do not test AreaQF.

Gradient targets for rule boosting

In second-order boosting, a row contributes a gradient gig_i and Hessian hih_i of the loss at the current prediction. A candidate subgroup defines a rule whose optimal weight is

wG=−∑i∈Ggiλ+∑i∈Ghi.w_G=-\frac{\sum_{i\in G}g_i}{\lambda+\sum_{i\in G}h_i}.

The improvement of the quadratic objective, in the package’s normalization, is

q(G)=(∑i∈Ggi)22N(λ+∑i∈Ghi).q(G)=\frac{\left(\sum_{i\in G}g_i\right)^2} {2N\left(\lambda+\sum_{i\in G}h_i\right)}.

GradientBoostingQF implements this objective with the tight subset bound of Boley et al. (2021). It finds a rule useful for the current boosting step, not necessarily a subgroup with a high raw target mean.

For logistic loss at initial predicted probability one half, gi=0.5−yig_i=0.5-y_i and hi=0.25h_i=0.25:

Loading...

Both positive and negative weights can improve the objective. Gradients and Hessians must be finite, Hessians nonnegative, and regularization nonnegative. The bound uses rows ordered by gradient/Hessian ratio, so this measure stays scalar rather than using the standard count/sum batching interface.

A complete learner must update predictions, recompute derivatives, and repeat the search. This example performs one rule search only.

Discovered rules and predictive rules

A discovered Rule has no consequent. A learned predictive rule can have a consequent, and a predictive ruleset may use ordering or a default. Do not interpret an overlapping subgroup list as a ready-made classifier without a separately specified prediction procedure.

Experiment. Increase reg_lambda while keeping derivatives fixed. How do optimal weights and the preferred rule change? Relate the result to the denominator of the objective, rather than to an arbitrary score rescaling.

References
  1. Boley, M., Teshuva, S., Le Bodic, P., & Webb, G. I. (2021). Better Short than Greedy: Interpretable Models through Optimal Rule Boosting. Proceedings of the SIAM International Conference on Data Mining. https://arxiv.org/abs/2101.08380