In this pape, Forgione et al. propose a transfer learning methodology to adapt recurrent neural network (RNN) models for dynamic system identification to new operating conditions. The approach is premised on the concept that the dynamics of real-world systems are variable over time due to external conditions or internal changes like aging.

To address this, the authors present a strategy whereby a nominal RNN model, initially trained on a set of data, is supplemented with an additive correction term derived from fresh data representing the changed system dynamics. This correction term is learned using Jacobian feature regression (JFR) based on the model’s nominal parameters, which is expressed in a nonparametric view using Gaussian process (GP) with neural tangent kernel (NTK-GP) extended for RNNs (RNTK-GP). The paper also details efficient implementation approaches for this methodology and validates its effectiveness through numerical examples from chemical and electrical engineering domains.

The paper has many strengths:

• Innovative methodology: the paper introduces a novel approach that extends transfer learning concepts to the adaptation of RNNs in system identification, a significant contribution given the prevalence of changing system dynamics in practical scenarios.
• Practical implementation: different implementation strategies, including online and offline parameter estimation and limited-memory computation, are discussed, highlighting the versatility and practical applicability of the proposed method.
• Empirical validation: the methodology is empirically validated with numerical experiments, demonstrating its efficacy in adapting to significant system variations.
• Comprehensive comparison: the paper offers a comparative analysis with other methods such as extended Kalman filter and full model re-identification, providing a clear benchmark of the proposed methodology’s performance.
• Efficiency: the paper focuses on computational efficiency, which is critical for real-time applications. The transfer learning approach proposed requires less computational effort than re-identifying the model from scratch.

However, some possible shortcomings should also be noted:

• Complexity for practitioners: the method’s mathematical complexity could be challenging for practitioners who do not specialize in machine learning or system identification.
• Potential overfitting: while the paper discusses model adaptation, it does not explicitly address the potential for overfitting to the new data, which could limit the model’s predictive capabilities over time.
• Scalability and limitations: the tests are done on specific case studies; there’s limited discussion on the scalability of the method and its limitations in different real-world scenarios or for systems with highly complex dynamics.
• Reproducibility concerns: although the paper mentions that codes are available for result reproduction, the practicality of implementation by others is not discussed.
• Comparative analysis scope: while there is a comparison with other methods, there may be a lack of an exhaustive comparison with a broader range of available techniques that could also be used for such adaptations.

In summary, the paper presents a technically robust and potentially transformative approach for adapting RNN models to new system dynamics. However, the complexity of the methodology, potential overfitting issues, and scalability are aspects that would benefit from further exploration.