A hot topic is dealt with in this paper: new methods for video editing beyond timeline manipulation and per-frame modification. The authors propose using a framework called Videoshop, which contains a set of operations that allows users to manipulate videos by mixing two source videos into one target video. The operations work on a three-dimensional (3D) gradient field, and thus extend current two-dimensional (2D) gradient field methods. Two methods are proposed for mixing videos in gradient space: the variational method and loopy belief propagation. The paper also presents a set of possible applications, which range from video editing tasks (such as compositing) to high-definition range compression.

For practitioners in the fields of video analytics and computer vision, the paper is absolutely worth reading. The use of 3D gradient space methods is both interesting and theoretically supported. The presented use cases are relevant and compelling, and the supporting examples are impressive.

From a practical standpoint, however, the presented methods have several limitations (some of which are discussed in the paper). For example, a major disadvantage is the complexity of the method, which leads to runtime performance that is much slower than real time. Additionally, the paper does not formally evaluate the accuracy of the framework’s operations, and it does not present examples in which the proposed algorithms fail to work. The authors only evaluate their algorithm by “using a variety of examples for image/video or video/video pairs.” In addition, I would have liked to read a detailed comparison of the proposed methods to already-well-established methods. For example, how does the 3D gradient method compare in speed and accuracy to a simple frame-by-frame image composition?

I recommend reading this paper. The field is rather new, and the proposed ideas are inspiring. I also recommend that readers follow the link to supplementary data that is provided in Appendix A; the material presented there is a helpful addition to the content presented in the paper.