In this paper, we present a novel system named DeepVisInterests that performs the users interests prediction task from social visual data based on a deep neural approach for the ontology construction. A comprehensive statistical study have been made to validate our DeepVisInterests system. The proposed system is based on the construction of users interests ontology using a set of deep visual features in order to learn the semantic representation for the popular topics of interests defined by Facebook. In fact, DeepVisInterests system addressed the problem of discovering the attributed interests (how the user interest can be detected from her/his provided social images in OSN) and analyzing the performance of the automatic prediction by a comparison with the self-assessed topics of interests (topics of interests provided by user in a proposed questionnaire) through our experiments applied on social images database collected from 240 Facebook users. The qualitative and the quantitative experimental study made in this paper, show that DeepVisInterests ranks top the list of recent related works with an accuracy of 0.80.
The last decades have witnessed the boom of the online social networks (OSN) with the huge amount of social textual data (e.g. comments, tags, descriptions, etc.) and social visual data (e.g. liked images, shared images, etc.). In fact, million of users named social users, visiting sites like Facebook, Pinterest, Instagram, Twitter, etc. These social networks sites principally rely on their social users to create and share data, called social data, in various types, to explain others' content with comments and to have a personal social graph based on on-line relationships. In fact, these social network sites continue to develop and social data continue to increase. In 2018, Facebook users have shared over 4 billion images with 5 billion . textual data while Flickr users have shared over 3 billion images [35]. With the development of mobile device, the delivery of images become more convenient and the most popular social networks offer the chance to share images without limits with almost 300 million images shared daily on Facebook and around 60 million images on Instagram [10].
Yet, managing and understanding the social data is still an important research challenges [42] that are useful for diverse applications such as constructing better recommendation systems and discovering social users’ lifestyle, etc. Thus, it has been well assumed that the social users are typically characterized by various features as personal attri-butes (demographic data, education, etc.), topic of interests, preferences, opinions, etc. For that, Kosinski et al. [29] and Lazzez et al. [23] confirm that the analysis of users’ social data can be used to discover a set of their personal attributes such as age, gender, politic orientation, etc., using Facebook social data. Furthermore, the social images understanding can be considered as the image classification that allow to classify and to recover the class of an input image based on its objects. Therefore, an analysis of such shared images can be applied to generate an user’ interest profile by analyzing the deep visual feature of their individual shared images and then aggregating the image-level information to predict user-level interest distribution at a fine-grained level. Evidently, as shown in figure 1, the users’ interests discovery is either done in a static method, by collecting data that hardly changes, or in a dynamic method by collecting data that frequently changes. Actually, users’ interests are presented explicitly by each user himself/ herself through likes and favourites (favourites books, movies, music, etc.) or implicitly by analyzing his/her social profile content.
Despite that, the most proposed works [29], [25] and [21] use the social textual data to analyze social networks and avoid the Fig. 1: Types of user interests visual data, due to the limitation of available social images and the lack of social images benchmarks. Our proposed system is motivated by the key observation that images has become one of the most popular types of social data through which social users convey their personal attitudes likes preferences, personal data, topics of interests, etc. However, users with diverse cultural environments, nationali-ties and ethnicities can easily communicate through a visual language to indicate their opinions, sentiments, personalities, etc [31], [48].
Due to the previous motivations, the social users’ hidden information discovery process based on these data presents an attractive aspect [43] which is illustrated in the figure 2. The included objects on each image indicate the topic of interest such as places, family, culture which can indicate the topics in-terested by the user. Yet, the trends of users interests discovery from social visual data consists on the image understanding using object recognition technology. [3]. Indeed, the proposed object-based approach contains the following key steps: (a) the topics of interests classification and (b) the users’ interest prediction. The reminder of this paper is structured as follow:
We constructed a novel social database named Smart-CityZenDB containing a set of social images gathered from 240 Facebook users accounts. This database was be annotated based on the selfassessed interests for each user in the set topics of interests defined by Facebook presented in table I.
For the users’ interests conceptualization and categorization, we constructed a users’ interests ontology based on the images’ objects extracted from a set of benchmarks databases like FoodDB [33], SportDB [18], DeepFashi-onDB [27] and a set of our constructed database for the rest of mentioned topics. For the object recognition we used various CNN architectures like LeNet [24], AlexNet [22], GoogleNet [39] and VGG19 [36].
For the users’ interest prediction, we infer the proposed ontology to result the attributed topics of interests for each user in the test database SmartCityZenDB. For that, various convolutional neural network archite
This content is AI-processed based on open access ArXiv data.
