Accuracy Of Image Tagging In Flickr: A Natural Disaster Case Study

We recently received word that we had a paper accepted for the upcoming  2016 International Conference on Social Media and Society entitled “Accuracy Of User-Contributed Image Tagging In Flickr:  A Natural Disaster Case Study.” In the paper we assess the reliability of user-generated tags during a natural disaster event. Below you read the abstract of the paper.

Social media platforms have become extremely popular during the past few years, presenting an alternate, and often preferred, avenue for information dissemination within massive global communities. Such user-generated multimedia content is emerging as a critical source of information for a variety of applications, and particularly during times of crisis. In order to fully explore this potential, there is a need to better assess, and improve when possible, the accuracy of such information. This paper addresses this issue by focusing in particular on user-contributed image tagging in Flickr. We use as case study a natural disaster event (wildfire), and assess the reliability of user-generated tags. Furthermore, we compare these data to the results of a content-based annotation approach in order to assess the potential performance of an alternative, user-independent, automated approach to annotate such imagery. Our results show that Flickr user annotations can be considered quite reliable (at the level of ~50%), and that using a spatially distributed training dataset for our content-based image retrieval (CBIR) annotation process improves the performance of the content-based image labeling (to the level of ~75%).

Study methodology for comparing user-annotated to CBIR-annotated Flickr imagery
Sample of training set images (yellow) and the retrieved fire images (green).

Full reference:

Panteras, G., Lu, X., Croitoru, A., Crooks, A.T. and Stefanidis, A. (2016), Accuracy Of User-Contributed Image Tagging In Flickr:  A Natural Disaster Case Study. In Gruzd, A., Jacobson, J., Mai, P.,  Ruppert, E. and Murthy , D. (eds), Proceedings of the 7th International Conference on Social Media and Society, London, UK. (pdf)