A very detailed and thorough description of a novel approach, this paper presents a well-motivated technical account.

The purpose of the proposed research is to define a general methodology for determining “a hierarchy of base kernels” and combining them via multiple kernel learning “to generate overall deeper kernels,” as recent advances in the area of neural networks discover the merits of deep representations. The approach consists of defining a “qualitative measure of expressiveness of a kernel” based on “trace ... and Frobenius norms of the kernel matrix,” showing “connections with the rank of the matrix,” radius of the minimum enclosing ball (MEB), and the empirical Rademacher complexity (ERC) of linear functions within the feature space. Further, the multi-kernel learning method is designed to learn the coefficients of general dot product polynomials.

The paper presents a very detailed explanation of the approach, including all formulas, theorems, and proofs, and adds very broad empirical evidence for its benefits by showing experimentation results in different contexts, comparing them with well-established baselines such as radial basis function (RBF) and homogeneous and nonhomogeneous polynomials, and demonstrating that exploiting the feature structure of the network when building the base kernels brings significant improvements in performance.

A very thorough and well-argued description, with a wealth of examples, explanations, and empirical results, and a solid basis on references and related work, this paper is an excellent read for scholars, engineers, and practitioners interested in machine learning and neural networks for research and application purposes.