Image streams are often analyzed in order to monitor general activities and draw statistical conclusions about behavior. This paper proposes a method for inspecting image data by distinguishing the foreground elements from the background within a sequence of frames. Background modeling is based on a feature dictionary, where sparse features are obtained using a coding/decoding procedure to characterize local areas. The novelty and contribution of the approach reside in the fact that it works at the local patch level, providing weighted representatives. The foreground extraction is accomplished using an adaptive algorithm based on Gaussian mixture modeling, inspired by Zivkovic’s work [1]. This method suits the nature of the test imagery material, which exhibits low frame rates and lighting conditions with a specific signal-to-noise ratio. Using a deviation measure based on the percentage of foreground pixels, the approach also spots inconsistent activities (such as the one that inspired the paper’s title).
The method evaluates the auto-encoder technique and compares it with other state-of-the-art work using a performance measure based on precision and recall values. The experiments aim to detect the daily patterns of behavior within a certain time interval based on a stream of webcam images of Fifth Avenue in New York City, collected by EarthCam Network2 over about four weeks in December 2011. Among the findings is the discovery that there is less traffic on Sunday nights, possibly indicating that, “even in New York City, the city that never sleeps, people seem to have more bed time before the beginning of new work weeks.”
The material is innovative, dense, interesting, and clearly explained, except for some minor errors. For example, I was unable to locate figure 3a.
