A novel image retrieval based on rectangular spatial histograms of visual words


  • Dr. Zahid Mehmood University of Engineering and Technology, Taxila, Pakistan
  • Dr. Syed Muhammad Anwar University of Engineering and Technology, Taxila, Pakistan
  • Dr. Muhammad Altaf University of Engineering and Technology, Taxila, Pakistan


Bag-of-visual-words, content-based image retrieval, rectangular spatial histograms, support vector machine.


Content-based image retrieval (CBIR) provides a solution to search the images that are similar to a query image. From last few years, the bag-of-visual-words (BoVW) model gained significance and improved the performance of CBIR. In a standard BoVW model, an image is represented as an order-less histogram of visual words, by ignoring the spatial layout of the image. The spatial layout carries significant information that can enhance the image retrieval accuracy. In this paper, we present a novel method of image representation, which is based on the construction of histograms over two rectangular regions of an image. Division
of the image into two rectangular regions at the time of construction of histograms adds the spatial information to the BoVW model. The proposed image representation uses separate visual words for upper and lower rectangular regions of an image. The experimental analysis carriedout on two image datasets validates that the proposed image representation based on the division of an image into histograms of rectangles increases the performance of image retrieval.


Ali, N., K. B. Bajwa, R. Sablatnig &Z. Mehmood

(2016). Image retrieval by addition of spatial information

based on histograms of triangular regions. Computers &

Electrical Engineering, 54:539- 550.

Bosch, A., A. Zisserman &X. Munoz (2007). Image

classification using random forests and ferns. Computer

Vision, 2007. ICCV 2007. IEEE 11th International

Conference on, IEEE.

Cao, Y., C. Wang, Z. Li, L. Zhang &L. Zhang (2010).

Spatial-bag-of-features. Computer Vision and Pattern

Recognition (CVPR), 2010 IEEE Conference on, IEEE.

Cardoso, D. N. M., D. J. Muller, F. Alexandre, L. A.

P. Neves, P. M. G. Trevisani &G. A. Giraldi (2014).

Iterative technique for content-based image retrieval using

multiple SVM Ensembles." J. Clerk Maxwell, A Treatise

on Electricity and Magnetism, 2:68- 73.

Ideas, influences, and trends of the new age.

ACM Computing Surveys (CSUR),40(2):5.

Dubey, S. R., S. K. Singh &R. K. Singh (2015). Rotation

and scale invariant hybrid image descriptor and retrieval.

Computers & Electrical Engineering,46:288- 302.

Hassner, T., V. Mayzels &L. Zelnik-Manor (2012).

On sifts and their scales. Computer Vision and Pattern

Recognition (CVPR), 2012 IEEE Conference on, IEEE.

Haykin, S. &N. Network (2004). A comprehensive

foundation. Neural Networks, 2(2004).

Heikkilä, M., M. Pietikäinen &C. Schmid (2009).

Description of interest regions with local binary patterns.

Pattern recognition,42(3):425- 436.

Jhanwar, N., S. Chaudhuri, G. Seetharaman &B.

Zavidovique (2004).Content based image retrieval

using motif cooccurrence matrix. Image and Vision

Computing,22(14): 1211 -1220.

Jiji, G. W. &P. J. DuraiRaj (2015). Content-based image

retrieval techniques for the analysis of dermatological

lesions using particle swarm optimization technique.

Applied Soft Computing,30:650 -662.

Kadir, T., A. Zisserman &M. Brady (2004). An affine

invariant salient region detector. Computer Vision-ECCV

, Springer:228- 241.

Khan, R., C. Barat, D. Muselet &C. Ducottet (2012).

Spatial orientations of visual word pairs to improve bag-ofvisual-

words model. Proceedings of the British Machine

Vision Conference, BMVA Press.

Lazebnik, S., C. Schmid &J. Ponce (2006). Beyond bags of

features: Spatial pyramid matching for recognizing natural

scene categories. Computer Vision and Pattern Recognition,

IEEE Computer Society Conference on, IEEE.

Lin, C.-H., D.-C. Huang, Y.-K. Chan, K.-H. Chen &Y.-J.

Chang (2011). Fast color-spatial feature based image retrieval

methods. Expert Systems with Applications,38(9):11412- 11420.

Liu, G.-H., Z.-Y. Li, L. Zhang &Y. Xu (2011). Image

retrieval based on micro-structure descriptor. Pattern

Recognition,44(9):2123 -2133.

Liu, G.-H., L. Zhang, Y.-K. Hou, Z.-Y. Li &J.-Y. Yang

(2010). Image retrieval based on multi-texton histogram.

Pattern Recognition,43(7):2380- 2389.

Lowe, D. G. (2004).Distinctive image features from scaleinvariant

keypoints. International journal of computer


Mahmood, T., T. Nawaz, R. Ashraf, M. Shah, Z. Khan, A.

Irtaza & Z. Mehmood (2015). A survey on block based copy

move image forgery detection techniques. Emerging Technologies

(ICET), 2015 International Conference on, IEEE:1- 6.

Mahmood, T., A. Irtaza, Z. Mehmood & M. T.

Mahmood (2017). Copy–move forgery detection through

stationary wavelets and local binary pattern variance

for forensic analysis in digital images. Forensic Science

International 279:8 -21.

Mahmood, T., Z. Mehmood, M. Shah & Z. Khan

(2017). An efficient forensic technique for exposing

region duplication forgery in digital images. Applied

Intelligence:1 -11.

Mehmood, Z., S. M. Anwar, N. Ali, H. A. Habib

&M. Rashid (2016). A novel image retrieval based on

a combination of local and global histograms of visual

words. Mathematical Problems in Engineering, 2016.

Mehmood, Z., T. Mahmood &M. A. Javid (2017).

Content-based image retrieval and semantic automatic

image annotation based on the weighted average of

triangular histograms using support vector machine.

Applied Intelligence,8:1 -16.

Nosaka, R., Y. Ohkawa &K. Fukui (2011). Feature

extraction based on co-occurrence of adjacent local

binary patterns. Advances in image and video technology,

Springer: 82 -91.

Philbin, J., O. Chum, M. Isard, J. Sivic &A. Zisserman

(2007). Object retrieval with large vocabularies and

fast spatial matching. Computer Vision and Pattern

Recognition, 2007. CVPR'07. IEEE Conference on, IEEE.

Rui, Y., T. S. Huang &S.-F. Chang (1999). Image

retrieval: Current techniques, promising directions, and

open issues. Journal of visual communication and image

representation,10(1):39- 62.

Safar, M. (2009). Image approximation to efficiently support

direction queries. Kuwait J. Sci. Eng, 36(1B):147- 166.

Shawe-Taylor, J. &N. Cristianini (2004). Kernel

methods for pattern analysis, Cambridge university press.

Sivic, J. &A. Zisserman (2003). Video Google: A text

retrieval approach to object matching in videos. Computer

Vision, 2003. Proceedings. Ninth IEEE International

Conference on, IEEE.

Tian, X., L. Jiao, X. Liu &X. Zhang (2014). Feature

integration of EODH and Color-SIFT: Application to

image retrieval based on codebook. Signal Processing:

Image Communication, 29(4):530 -545.

Tomašev, N. &D. Mladenić (2015). Image hub explorer:

Evaluating representations and metrics for content-based

image retrieval and object recognition. Multimedia Tools

and Applications,74(24): 11653 -11682.

Tousch, A.-M., S. Herbin &J.-Y. Audibert (2012).

Semantic hierarchies for image annotation: A survey.

Pattern Recognition,45(1):333 -345.

Tuytelaars, T. &L. Van Gool (1999). Content-based

image retrieval based on local affinely invariant regions.

Visual Information and Information Systems, Springer.

Tuytelaars, T. &L. J. Van Gool (2000). Wide baseline

stereo matching based on local, affinely invariant regions.


Ullah, A. &B. Baharudin (2016). Pattern and semantic

analysis to improve unsupervised techniques for opinion

target identification. Kuwait Journal of Science,43(1).

Vedaldi, A. &B. Fulkerson (2010). VLFeat: An open

and portable library of computer vision algorithms.

Proceedings of the 18th ACM international conference on

Multimedia, ACM.

Vedaldi, A. &A. Zisserman (2012). Sparse kernel

approximations for efficient classification and detection.

Computer Vision and Pattern Recognition (CVPR), 2012

IEEE Conference on, IEEE.

Wang, C., B. Zhang, Z. Qin &J. Xiong (2013). Spatial

weighting for bag-of-features based image retrieval.

Integrated Uncertainty in Knowledge Modelling and

Decision Making, Springer:91 -100.

Xie, J., L. Zhang, J. You &S. Shiu (2015). Effective

texture classification by texton encoding induced statistical

features. Pattern Recognition,48(2):447 -457.

Yildizer, E., A. M. Balci, M. Hassan &R. Alhajj (2012a).

Efficient content-based image retrieval using multiple

support vector machines ensemble. Expert Systems with

Applications,39(3):2385 -2396.

Yildizer, E., A. M. Balci, T. N. Jarada &R. Alhajj

(2012b). Integrating wavelets with clustering and indexing

for effective content-based image retrieval. Knowledge-

Based Systems,31:55 -66.

Zeng, S., R. Huang, H. Wang &Z. Kang (2016).

Image retrieval using spatiograms of colors quantized by

Gaussian Mixture Models. Neurocomputing,171:673- 684.

Zhang, D., M. M. Islam &G. Lu (2012). A review

on automatic image annotation techniques. Pattern

Recognition,45(1):346- 362.