The topic of eigenvalue problems--and, more broadly, generalized eigenvalue (GEV) problems--is an important area of study that is used in a wide range of application areas (for example, structural mechanics and fluid flow). This paper, which deals with sparse GEV problems, develops a very useful approach. Additionally, it deals with three important areas arising from statistical data analysis: principal component analysis (PCA), canonical correlation analysis (CCA), and Fisher discriminant analysis (FDA). Financial trading strategies, gene expression datasets, and document translation applications are also examined.
The authors’ approach to solving the GEV problem is quite novel. Rather than using ℓ1-norm approximation to constrain cardinality and thus relax the constraint, they generate a tighter approximation associated with the negative log-likelihood of the student’s t-distribution. This allows the problem to be treated as a difference of convex functions. The problem is solved by sequences of problems that generate lower and upper bounds--a majorization-minimization method. The authors prove that the solution is globally convergent. To be clear: this does not mean convergence to a global optimum, but rather that, at any random starting point, the iterates converge to a stationary point of the difference of convex functions programming problem.
The paper is extremely thorough and well presented. The first section introduces prior work and describes the optimization problem. It then presents the sparse GEV problem, including theory, the algorithm, and convergence analysis. This is followed by experimental results for each of the application areas (PCA, CCA, and FDA). Comparisons with other methods and the selection of test problems are provided in considerable detail. The paper concludes with a discussion section and four appendices that provide details on the derivations of the method.
This excellent and thorough paper provides readers with a detailed approach to a number of important and diverse application areas.
