Author: Zhilin Jin
Date: 2/14/2023
Executive Summary:
Feature interaction detection is important in almost all study fields. Correctly identifying variable interactions can help explore the relationships among features in the dataset which in return will aid to build better models containing these variables. However, many existing methods are confused feature interaction with variable importance. On the contrary, based on the formal definition of feature interaction, we built our methods leveraging the implication of the definition. In particular, expected Hessian matrix was proposed and the relationship between quadratic regression and expected Hessian matrix was established. Inspired by the proof, two methods, transformation local approach (TLA) and two-stage interaction detection (TSID), were proposed. Both methods were performed in local settings where the correlation among features were proved to be helpful under certain assumptions. Our methods were compared with other popular interaction detection methods which are currently available to be implemented. Different simulation scenarios with various levels of difficulties for feature interaction detections were set up when performing the empirical analyses. Our proposed methods performed the best when taking all the performance metrics into consideration. In addition, the proposed methods have also been extended to detect three-way interactions by leveraging multivariate adaptive regression splines (MARS). Real data analyses were conducted on the datasets of Boston Housing, Ozone Readings, Ames Iowa Housing, and Adult Diffuse Glioma which were extracted from various R packages.
