Graph based entropy, an index of the diversity of events in their distribution to parts of a co-occurrence graph, is proposed for detecting signs of structural changes in the data that are informative in explaining latent dynamics of consumers behavior. For obtaining graph-based entropy, connected subgraphs are first obtained from the graph of co-occurrences of items in the data. Then, the distribution of items occurring in events in the data to these sub-graphs is reflected on the value of graph-based entropy. For the data on the position of sale, a change in this value is regarded as a sign of the appearance, the separation, the disappearance, or the uniting of consumers interests. These phenomena are regarded as the signs of dynamic changes in consumers behavior that may be the effects of external events and information. Experiments show that graph-based entropy outperforms baseline methods that can be used for change detection, in explaining substantial changes and their signs in consumers preference of items in supermarket stores.
Statistics and machine learning have been adopted to forecasting demands in markets [1,2]. However, changes in the market due to the effects of external events are hard to explain by learning causalities from data, because external causal events are out of data by definition. Here, we define explanation as to relate a change in the observation with causes that may not be events in the data. Let us assume that we have data on the position of sale (POS) in a supermarket as D in Eq. (1), that is the target data dealt with in this paper, where B t stands for a basket, i.e., a set of items (members of I, the set of all items in the supermarket) purchased at time t by some consumer and T is the length of the period of time in the data:
Here, suppose that the sales’ volume of coffee increases for a week beyond prediction on the POS data. The cause of this increase may be a TV program broadcasted a few days ago, about the positive effect of coffee on human’s health. Such a causality of change may be explained if a marketer focuses attention on the period of time when the external cause, i.e., the TV program about healthcare that is not in set I, occurred and if additional data about past TV programs are given. Then, the marketer can create a strategy to promote the sales of coffee by publishing a book relevant to the content of the TV program.
Change points have been detected on the changes in parameters and/in models of time series in the approach of machine learning. In Principal Component Analysis (PCA), projecting data to principal components do not only reduce computational cost, but also sharpen the sensitivity of change detection. Here, the change in the correlation, the variance, and the mean of components is detected from before to after a change [3]. Methods for detecting changes in parameters in the model capturing the structure of latent causality have been developed for both discrete [4][5][6] and continuous [7] changes, and the method for the latter is turning out to work for the former as well. The changes in the values of the parameter set , from time tt to t, are learned as [t -Δt, t] -[tt -Δt, tt] , where Δt is the width of the training time window of the data to learn [t -Δt, t] and [tt -Δt, tt] from, and t is the time step of the change. is learned to minimize the error of prediction from the reality of observable events. The precision of change detection is expected to be the better for the larger Δt that can be regarded be a part of tolerant delay, i.e., the length of time the analysis should wait for detecting a change. However, a large Δt is not reasonable from the view- point to explain the change quickly. For example, to highlight the causality above from the transient TV program to the coffee sales, Δt is better to be set to 1 month than to 1 year, so that the TV program can outstand as an essential cause in the period of length Δt.
From to the viewpoint not only to detect, but also to explain a change with linking to external knowledge, i.e., knowledge about events not included in data, there are methods to learn latent topics of interest in a sequence of words or actions without known labels corresponding to the topics. For example, consecutive time segments, each of which is relevant to a vector in the space of a limited number of latent topics that are not labeled by known labels, are obtained by the dynamic topic model (DTM [8]). By applying DTM to POS data, the changes in consumers’ interests can be detected as the boundaries between the obtained time segments corresponding to the changes in the topic vector. Topic-tracking model (TTM) has been also presented to consider the evolution of each consumer if the behavior of each consumer c is reflected on D in the form of B t,c instead of B t in Eq. ( 1) regarding each consumer as a generator of topic vectors [9]. Topic models have a potential not only to learn topics behind observed events, but also to explain changes. In contrast, the aim of this paper can be positioned as to
cope with changes, where such a transient topic, as the healthy coffee above, causes influence on the market and may disappear or get united with other topics.
Furthermore, to explain a change as an effect of an external cause, it is essential to detect a precursor that may be an evidence of the causality. In the example above, the precursor of the increase in the sales of coffee may be a novel co-occurrence of coffee with some healthy food in consumers’ purchase, because people interested in health care may be the leading users of coffee. Such a precursor should appear in a short period that is before a larger number of people start to buy coffee but is after the TV program. Thus, this paper is addressed to the problem to detect a sign of change, i.e., any evidence of the change or the precursor of the change, on the data of a short Δt and also to explain the sign with linking to external events.
Precursors to changes have been really explored in various d
This content is AI-processed based on open access ArXiv data.
