Object recognition in color images of a complex scene based on hierarchical temporal memory in combination with the detection of significant areas

Investigator: I. Bajla

Biologically inspired hierarchical temporal memory (HTM) is one of the successful deep intelligent networks. However, its disadvantage is that it cannot be used directly to solve the detection and recognition of objects in multi-object color images of a complex scene. The basic goal of our research was to find a way to connect the HTM network with the apparatus of visual attention, which would allow the application of HTM to the task of recognizing objects in complex color images. We based the new solution of this task on our design of a parallel HTM system for three different features of objects (color, texture and shape), which we combined with an innovated computational model of visual attention, which uses maps of visual significance (so-called salient maps). Our proposed multi-stage algorithm allows to locate individual objects in the color image of complex scenes. The proposed combined system of HTM network and visual attention algorithm demonstrated better characteristics (so-called capture and success of object detection in the image) than two compared recognition approaches (cascade detectors and pattern matching). Promising results were obtained in the doctoral student’s dissertation, which was followed up during his postdoctoral stay at the Czech Technical University in Prague.

Fig. 1: Example of significance map, combined flag map and window position calculation process for HTM network application in two types of images – on partially distorted background (left column) and on completely distorted background (right column): 1 – input image of complexity 4, squares drawn the thick line represents the windows with real objects, 2 – the input image of complexity 2, the coarse squares – similarly as in Figure 1, 3,4 – the respective combined flag maps, 5,6 – the respective final maps of significance (salient maps). Asterisks indicate positions of local significance maxima, dashed squares indicate areas – sections from which image centroids (represented by triangles) are calculated. Centroids counted from the significance maps themselves are indicated by squares.

 

Foreign partner:
• Czech Institute of Informatics, Robotics, and Cybernetics, Czech Technical University in Prague

 

Related projects:
• VEGA 2/0043/13
• VEGA 2/0138/16
• European Regional Development Fund (CZ.02.1.01 / 0.0 / 0.0 / 15_0 03/0 0 0 0470)

 

Publications:

  1. ŠKOVIERA, R. – BAJLA, I. – ŠKOVIEROVÁ, J.: Object recognition in clutter color images using Hierarchical Temporal Memory combined with Salient-region detection. Neurocomputing, 2018, 307, s.172-183.  (CC, IF=3,39).
  2. ŠKOVIERA, R. – BAJLA, I. – KUČEROVÁ, J. : Object Recognition in Clutter Color Images using Hierarchical Temporal Memory combined with Salient-Region Detection. In: M.H.Hamza (ed), IASTED 2015, Proc. of the Int. Conf. on Computational Intelligence, Innsbruck, Austria, February 16 – 17, 2015, s. 245-254, (ISI-WoS).
  3. ŠKOVIERA, R.: Využitie samoorganizovaného učenia v hierarchických HTM sieťach na rozpoznávanie vizuálnych objektov v obrazových databá Dizertačná práca, 2015, FMFI UK a ÚM SAV, Bratislava, 131 s.
  4. ŠKOVIERA, R. – BAJLA, I.: Image classification based on Hierarchical Temporal Memory and color features. In: J. Maňka, M. Tyšler, V. Witkovský, Frollo, (eds), MEASUREMENT 2013, Proc. of the 9th Int. Conf. on Measurement, Smolenice, Slovakia, May 27-30, 2013, s. 63-66. Institute of Measurement Science, Slovak Academy of Sciences, Bratislava, (ISI-WoS)