Spatial Filtering Pipeline Evaluation of Cortically Coupled Computer Vision System for Rapid Serial Visual Presentation

ORIGINAL RESEAR CH Spatial Filtering Pip eline Ev aluation of Cortically Coupled Computer Vision System for Rapid Serial Visual Presen tation Zhengw ei W ang, Graham Healy , Alan F. Smeaton and T omas E. W ard Insigh t Cen tre for Data Analytics, Dublin Cit y Universit y , Dublin 9, Ireland Email: zhengwei.wang22@mail.dcu.ie , { graham.healy, alan.smeaton, tomas.ward } @dcu.ie ABSTRA CT Rapid Serial Visual Presentation (RSVP) is a paradigm that supp orts the applica- tion of cortically coupled computer vision to rapid image search. In RSVP , images are presen ted to participants in a rapid serial sequence whic h can ev ok e Ev ent-related P otentials (ERPs) detectable in their Electroencephalogram (EEG). The contempo- rary approac h to this problem in v olv es supervised spatial filtering tec hniques whic h are applied for the purposes of enhancing the discriminativ e information in the EEG data. In this paper we make tw o primary contributions to that field: 1) W e prop ose a no v el spatial filtering method whic h w e call the Multiple Time Window LDA Beamformer (MTWLB) metho d; 2) we provide a comprehensive comparison of nine spatial filtering pip elines using three spatial filtering schemes namely , MTWLB, xD A WN, Common Spatial Pattern (CSP) and three linear classification methods Linear Discriminant Analysis (LDA), Ba yesian Linear Regression (BLR) and Logis- tic Regression (LR). Three pip elines without spatial filtering are used as baseline comparison. The Area Under Curve (AUC) is used as an ev aluation metric in this pap er. The results rev eal that MTWLB and xDA WN spatial filtering techniques enhance the classification p erformance of the pip eline but CSP do es not. The re- sults also supp ort the conclusion that LR can b e effectiv e for RSVP based BCI if discriminativ e features are a v ailable. KEYW ORDS Rapid serial visual presentation (RSVP), cortically coupled computer vision, electro encephalograph y (EEG), even t-related p otentials (ERPs), spatial filtering. 1. In tro duction There is growing in terest in using Electro encephalograph y (EEG) signals to help in searc hing images [1 – 3]. This is based on estimating image conten t by examining par- ticipan ts’ neural signals in resp onse to image presentation. The concept of Rapid Serial Visual Presentation (RSVP) can b e introduced using a familiar example, that of rapidly riffling through the pages of a b o ok in order to lo cate a needed image [4]. In RSVP , a rapid succession of target and standard (non-target) im ages are presen ted to a participan t on a display at a rate of 5 H z - 12 H z . The lo cation of target im- ages within the high-sp eed presentation is not known in adv ance by users and hence requires them to actively lo ok out for targets i.e. to attend to target images. This paradigm where users are instructed to attend to target images amongst a larger pro- p ortion of standard images is known as an o ddball paradigm and is commonly used to This work has been accepted by Brain-Computer Interfaces. DOI:10.1080/2326263X.2019.1568821. elicit Even t-related Poten tials (ERPs) suc h as the P300, a p ositive voltage deflection that typically o ccurs b et ween 300 ms - 600 ms after the app earance of a rare visual target within a sequence of frequent irrelev an t stimuli [5]. Since participants do not kno w when target images will app ear in the presentation sequence, their o ccurrence causes an atten tional-orientation resp onse that is characterized by the presence of a P300 ERP . Using mo dern signal pro cessing and machine learning techniques, RSVP can b e coupled with single-trial ERP detection to enable image search BCI applications [6,7]. Single-trial ERPs detection for an RSVP paradigm presen ts the follo wing challenges: Chal lenge 1. L ow Signal-to-noise R atio (SNR): ERP component amplitudes are often m uc h smaller than those of sp on taneous EEG components and task-related ERP comp onen ts are typically ov erwhelmed by strong ongoing EEG background activit y in single trials and so cannot b e normally visually recognized in the raw EEG trace [8]. T raditional metho ds analyze ERPs b y av eraging across several task-related trials in order to reduce or eliminate sp on taneous EEG comp onents [9]. Chal lenge 2. Curse of dimensionality: RSVP-based ERP data can hav e high di- mensionalit y spanning b oth space and time. Moreo v er, ERPs v ary greatly across par- ticipan ts and exp erimental task [9]. In order to capture the ERPs, it is necessary to c ho ose a time windo w large enough for ep o ching whic h in v olv es the time region in whic h ERPs might app ear. Moreov er the training sets a v ailable for machine learning purp oses are t ypically mo dest in size and worse, con tain relativ ely few instances of the resp onses evok ed by the infrequent (by design) target image class. Chal lenge 3. Overlapping ep o chs: The strength of the RSVP paradigm is that the rate of the stimulus sequence increases the upp er limit of p oten tial information transfer rates in BCI applications. How ev er, a relatively large time window has to b e set for ep oching in order to capture the ERPs. Therefore, there is substantial ov erlap betw een adjacen t target ep o chs and standard ep o c hs b ecause of the short Interstim ulus In terv al (ISI) used in the RSVP paradigm. In this work, we consider a pip eline combining spatial filtering and linear classifica- tion as this is the most widely used pip eline in RSVP-EEG based BCI. There are several p otential signal pre-pro cessing techniques that may increase the efficiency of detecting single-trial ERPs including time-frequency feature extraction and hierarchical discriminant comp onen t analysis (HDCA) [10,11]. How ever, spatial filtering techniques are more efficient when a full EEG cap dataset is av ailable. In this pap er, we focus on spatial filtering for signal pre-pro cessing as this is the predominant approac h used in RSVP-based BCI b ecause we are using a full EEG cap recording dataset. Spatial filtering fo cuses on enhancing task-related information contained in the EEG signal. It plays an imp ortan t role in BCI researc h b ecause it can enhance the discriminativ e information present in the EEG signal whilst reducing the ov erall data dimensionalit y . Spatial filtering has b een shown to e nhance detection accuracy with a P300 sp eller paradigm [12]. Ho w ev er, classification pip elines without spatial filtering ha v e b een proposed for single-trial ERP detection. These metho ds include widely used linear classifiers such as Linear Discriminant Analysis (LD A) [10], Logistic Regression (LR) [13] and Ba y esian Linear Regression (BLR) [14]. In v estigation of spatial filtering in RSVP-based EEG has b een explored previously in the literature [15 – 17]. What is unclear from these studies is how to determine the optimal n um b er of spatial filters i.e. this detail has been omitted in previous studies and y et this is an imp ortant consideration so is included in this inv estigation. The primary ob jective of this pap er is to explore the p erformance of pip elines that com bine differen t spatial filtering approac hes and classifiers where their resp ective hyperparameters are 2 explored through random searc h cross v alidation. A pip eline in this pap er comprises spatial filtering, feature dimensionalit y reduction and classification steps. Three spatial filtering approaches are explored in this pap er, namely xDA WN [12], multiple time windo w LDA b eamformers (MTWLB) whic h is an extension of LD A b eamformer [18] and common spatial pattern (CSP) [19]. Principal comp onen t analysis (PCA) is utilized for feature dimensionalit y reduction. Three linear classification metho ds are explored, namely LDA, BLR and LR resp ectively . There are nine pip elines in total including spatial filtering and classification com binations. Three pip elines that only apply the three classification metho ds without applying an y spatial filtering are used for comparing p erformance. This pap er should provide neurotec hnologists seeking to apply a RSVP paradigm to EEG with a comprehensive assessmen t of the comparative p erformance of b oth commonly used spatial filtering pip elines and a new metho d all assessed using a new publicly a v ailable b enchmark dataset. This pap er is organized as follo ws. Firstly , w e describ e the methodology whic h includes pip eline and p erformance ev aluation metrics used in this pap er. Secondly , we clarify the exp erimen tal RSVP-based EEG dataset used in this pap er. Finally , results and discussion are presen ted in the last t wo sections. 2. Metho dology 2.1. Pip eline Description This pap er explores nine pip elines comprising spatial filtering, feature dimensionalit y reduction and classification resp ectively along with three pip elines containing feature dimensionalit y reduction and classification as comparisons. Fig. 1 illustrates the tw o pip eline architectures under consideration in this study . With spatial filtering applied, an n channel EEG ep o c h is transformed to m source comp onen ts ( m < n ). Before the feature generation step, we apply PCA for each individual channel ∈ R L × n (pip elines without spatial filtering) and individual comp onent ∈ R L × m (pip elines with spatial filtering) on the temp oral axis for dimensionalit y reduction following the work [20], where L is the num ber of epo chs. The reason wh y w e apply PCA individually is b ecause EEG p o w er in eac h channel and eac h component is not consisten t and this step ensures that discriminativ e information is not lost. W e leav e out the PCA comp onen ts which con tain less than 1% ratio of the v ariance. The feature generation step concatenates m comp onen ts or n channel EEG to a feature vector b efore feeding to the classifier. 2.2. Sup ervise d Sp atial Filtering Before in tro ducing the sup ervised spatial filtering approaches we clarify the notations whic h will b e used. N c is the num ber of channels, N t is the num ber of time samples in an ep och, N f is the n um b er of s elected spatial filters and n is the num ber of ep o c hs. Spatial filtering creates a weigh ted com bination of each EEG c hannel input in order to enhance a particular subset of information whic h is contained in the original EEG ep och. Spatial filtering reduces the num b er of features b ecause the num ber of spatial filters selected N f is smaller than the n um b er of c hannels N c . The problem of spatial filtering is to find pro jection vectors (spatial weigh ts for each channel) w ∈ R N c × N f to pro ject X ∈ R N c × N t to a subspace, where w is calculated by different optimization 3 criterion. X sub = w T X (1) Sev eral approaches hav e b een presen ted in the literature for generating spatial fil- ters w in equation (1) in the area of BCI research. Indep endent comp onen t analysis (ICA) is a blind source separation technique which can b e used to find a linear repre- sen tation of non-Gaussian data so that the comp onents are statistically indep endent, or as indep enden t as p ossible [21]. Such a representation is capable of capturing the inheren t structure of data in many applications, and hence has application to feature extraction [2,22] and remo ving artifacts from EEG signals [23]. Sp ecifically , ICA finds a comp onen t ‘unmixing’ matrix ( w ) that, when multiplied b y the original data ( X ), yields the matrix ( X sub ) of indep endent comp onent (IC) time courses [24]. Therefore, the main purp ose of ICA is blind source separation instead of discriminating EEG in tw o exp erimen tal tasks. PCA is another statistical tec hnique that uses eigenv alue decomp osition to conv ert a set of correlated v ariables into a set of linearly uncorre- lated v ariables. PCA has b een applied to EEG signals for dimensionality reduction [25] and generating spatial filters [26]. Similar to ICA, PCA op erates without knowl- edge of stimulus t yp es hence it is an unsup ervised approach. In this work, we consider sup ervised spatial filtering metho ds that aim to enhance the difference b et w een tar- get and standard image stimuli. Three spatial filtering metho ds are considered in this pap er, namely MTWLB, xDA WN and CSP . MTWLB is an extension of the LD A b eamformer metho d which aims to maximize the signal-to-noise ratio (SNR) in each individual time window. xD A WN and CSP are based on Ra yleigh quotien ts where xD A WN maximizes the signal to signal plus noise ratio (SSNR) whereas C SP maxi- mizes the difference of the v ariance b etw een tw o classes. In the following paragraphs, w e describ e the op eration of these three spatial filters generation techniques, i.e. The LD A b eamformer with our window extensions, xDA WN and CSP . 2.2.1. LD A Be amformer The LD A b eamformer has b een successfully applied for recov ering N2 and P3 sources in an auditory experiment [18]. Considering the epo ch X ∈ R N c × N t , let column v ectors p 1 ∈ R N c × 1 and p 2 ∈ R N c × 1 b e the spatial pattern of a sp ecific comp onent in t w o differen t exp erimental conditions. W e denote the difference pattern as p := p 1 − p 2 and the co v ariance matrix Σ ∈ R N c × N c . The optimization problem of the LDA beamformer can b e stated as minimize w w T Σw s.t. w T p = 1 (2) where w is the spatial filter. Consequently , the solution to the optimization problem maximizes the signal-to-noise ratio (SNR) of the desired signal and the optimal spatial filter is giv en b y w = Σ − 1 p ( p T Σ − 1 p ) − 1 (3) The spatial pattern for the LDA b eamformer is directly estimated from the differ- ence b et w een ERP p eaks in an o ddball experiment [18]. As seen in Fig. 2, the b old red line is the ERP difference at the Pz channel and the blue line represents the peak v alue 4 timestamp of difference ERPs at the Pz channel. The different ERP v alues across all c hannels at that timestamp can then b e estimated as spatial patterns. Due to the substantial o v erlap b etw een adjacent target ep o c hs and non-target ep ochs, along with inherent v ariabilit y in ERP latencies and top ographies b etw een participan ts, w e extend the LDA b eamformer to MTWLB. The MTWLB implemen- tation can b e seen in Algorithm 1. The key idea of MTWLB is to train multiple LDA b eamformer models ov er non-o v erlapping successiv e time windo ws i.e. to train a single LD A b eamformer for each time window that is adaptive to the lo cal spatio-temp oral features c haracterizing target-related ERP activit y at that time p oin t. Algorithm 1 Implementation of MTWLB 1: Set M time windows for MTWLB; 2: Calculate Σ in eac h time window: 3: for i = 1 : M do 4: for Each difference ERP p at each timestamp in an individual time windo w do 5: w = Σ − 1 p ( p T Σ − 1 p ) − 1 6: J = w T Σw 7: end for 8: Retain w and p that minimize J 9: end for 2.2.1.1. Sp atial p attern estimation for MTWLB. In contrast with the LDA b eamformer metho d, w e estimate the spatial pattern and calculate the equation (3) separately for eac h time windo w rather than whole time series. Therefore, there are M sets of estimated spatial patterns while eac h set has 32 individual spatial patterns. The reason why we do this is the substantial ov erlap b etw een adjacent target ep o c hs and non-target ep o c hs, along with inheren t v ariability in ERP latencies and top ographies b et w een participants in RSVP-based EEG. 2.2.1.2. Covarianc e matrix estimation for MTWLB. MTWLB uses the sp ec- ified time windo w coinciding with ERP activity to estimate the co v ariance matrix instead of using the whole EEG ep o c h that is utilized by LDA b eamformer [18]. W e firstly concatenate n trial EEG ep o chs through time and trials (i.e. the concatenated data D ∈ R N c × N , where N = N t × n ) and then normalize the concatenated data through columns. The cov ariance matrix C ∈ R N c × N c can b e calculated from the nor- malized concatenated data. After using artifact rejection or other EEG pre-pro cessing metho ds, the cov ariance matrix is singular. W e used shrink age algorithms to regularize the co v ariance matrix in order to make it inv ertible [27]. As a result, MTWLB is able to generate a set of spatial filters corresp onding to ERP differences in each sp ecified time window where the num b er of time windows can b e selected using cross-v alidation. 2.2.2. xD A WN The xD A WN algorithm has been applied in BCI for ERP detection in the P300 speller paradigm [12,28] and the RSVP paradigm [15]. It considers the mo del of the recorded signals X ∈ R N t × N c as follo ws: X = DA + N (4) 5 where D ∈ R N t × N e ( N e is the n um ber of temp oral samples of the ERP response) is the T o eplitz matrix whose first column elements are set to zero except for those corresp onding to a target onset, whic h are set to one, A ∈ R N e × N c is the matrix of ERPs. Hence, DA in (4) represents the ERP resp onse corresp onding to the syn- c hronous resp onse with target stimuli. N is the on-going brain activit y , also known as EEG bac kground noise. The goal of xD A WN is to apply spatial filters w to enhance the SSNR of the ERP resp onse corresp onding to the target stimulus. The optimization problem for xD A WN can b e defined as ˆ w = arg max w T race( w T ˆ A T D T D ˆ Aw ) T race( w T X T Xw ) (5) where ˆ A is the least squares estimation of A [12]. Thus, a n um b er of spatial filters corresp onding to different evok ed resp onses can be obtained through the Rayleigh quotien t optimzation problem [29]. The n umber of spatial filters are often c hosen through cross-v alidation. 2.2.3. Common Sp atial Pattern CSP is one of the most p opular spatial filtering approac hes for motor imagery based BCIs, where the task inv olves t w o differen t states of brain activity (e.g. imagery of the mov emen t of the left or right hand) [19,30]. CSP aims to maximize the v ariance of one class and minimize the v ariance of another class. The optimization problem for CSP can also b e estimated and interpreted as Rayleigh quotient [29]. First, let X 1 ( i ) and X 0 ( i ) b e the i th ev ent lo c k ed ERP ep o c h ∈ R N c × N t and t w o co v ariance matrices Σ 1 and Σ 0 are calculated as follows (subscript “0” for standard condition and “1” for target condition) Σ a = 1 n n X i =1 X a ( i ) X T a ( i ) T race( X a ( i ) X T a ( i )) (6) The solution for CSP can b e determined through Raleigh quotients by solving a generalized eigen v alue problem { max, min } w w T Σ 1 w w T Σ 0 w (7) Similar to the previous tw o approaches, CSP is able to generate a set of spatial filters. Ho w ever, spatial filters in CSP app ear pair-by-pair b ecause CSP maximizes v ariance in one class and minimize v ariance in the other class. F rom Cecotti’s w ork, four spatial filters w ere chosen as [ w 1 , w 2 , w N c − 1 , w N c ] ( N c is the n umber of electro des) [15]. This w ork c ho oses a pair of spatial filters via cross-v alidation. A t this p oin t, w e ha ve highlighted how the three metho ds under consideration here can generate spatial filters w . All three spatial filters generated serve the same ob- jectiv e of reducing computation complexit y but the optimization target are differ- en t. MTWLB generates spatial filters based on maximizing the SNR in individual time windo ws. xD A WN, in con trast generates spatial filters based on maximizing the signal-to-signal plus noise ratio (SSNR) for the whole EEG ep o ch. Finally the metho d 6 of CSP generates spatial filters through maximizing the v ariance difference b et w een t w o classes. 2.3. F e atur e Gener ation The spatial filter w ∈ R N c × N f generated serves to transform the original EEG ep och X ∈ R N c × N t to the feature space Ψ = w T X (8) where Ψ ∈ R N f × N t . The pro jected subspace Ψ can b e represented as spatial-filtered EEG signals in volving different discriminative information corresp onding to the crite- ria used in their filter generation. PCA can then b e applied to eac h ro w in Ψ for feature reduction and generating a new set of time series which are linearly uncorrelated. In this work, principal comp onen ts whose explained v ariance ratio is greater than 1% are selected and concatenated as the feature vector whic h will b e used as input to the classification step. 2.4. Line ar Classifiers Linear classifiers are widely used for RSVP-based BCI due to their go o d p erformance, often simple implementation and lo w computational complexit y [1,2,10,11]. In this pap er, we fo cus on three widely used linear classifiers in RSVP-based BCI research, namely LD A, LLR and BLR. 2.4.1. Line ar Discriminant Analysis LD A is a sup ervised subspace learning metho d which is based on the Fisher criterion and it is equiv alen t to least squares regression (LSR) if the regression targets are set to N N 1 for samples from class 1 and N N 2 for samples from class 2 (where N is total n um b er of training samples, N 1 is the num b er of samples from class 1 and N 2 is the n um b er of samples from class 2) [31]. It aims to find an optimal linear transformation w that maps x to a subspace in which the b et w een-class scatter is maximized while the within-class scatter is minimized in that subspace. The optimization problem for LD A is to maximize the cost function as b elo w J = w T S B w w T S W w (9) where J represen ts the cost to b e minimized, S B is the b et w een-class scatter, S W is the within-class scatter, w T is the transp ose matrix to the w . Regularization is often applied in order to av oid the singular matrix problem of S W [32]. LDA enables the b est separation b et w een tw o classes on the subspace w . LDA has relatively low com- putational complexity whic h makes it suitable for online BCI systems. As mentioned earlier, classification of RSVP-based EEG data suffers from the imbalanced data set problem. In Xue’s work [33], he show ed that there is no reliable empirical evidence to supp ort that an imbalanced data set has a negativ e effect on the p erformance of LD A for generating the linear transformation vector. Consequently , LDA is suitable and has b een successfully used in RSVP-based BCI [2,10]. 7 2.4.2. Bayesian Line ar R e gr ession Ba y esian linear discriminan t analysis (BLD A), can b e seen as an extension of LDA or LSR. In BLR, regularization for parameters is used to prev en t ov erfitting caused b y high dimensional and noisy data. BLR assumes the parameter distribution and target distribution are b oth Gaussian [31]. W e introduce LSR as a starting p oin t for the description of BLR. The solution for LSR can b e stated as w = ( XX T ) − 1 Xy (10) Note that y = N N 1 for class 1 and y = − N N 2 for class 2 here (threshold can be determined b y adding a column with all one as the first column in X ). LSR do es not consider the parameter distribution in this case and it maximizes the likelihoo d. F or BLR, it considers the parameter distribution and maximizes the p osterior. Giv en the prior target distribution p ( y ) ∼ N ( µ, β − 1 ) and parameter distribution p ( w ) ∼ N (0 , α − 1 I ) (where β and α are the in v erse v ariance), BLR gives the optimized estimation for the parameter w = β ( β XX T + α I ) − 1 Xy (11) The optimization problem of BLR can b e concluded as the maxim um a p osterior (MAP) estimation [34], which lies in the assumption of an appropriate prior distri- bution of the parameter to b e estimated. Hence, the optimization dep ends on the h yp erparameters β and α . In real-w orld applications, the h yperparameters can b e tuned using cross v alidation or the maximum likelihoo d solution with an iterative algorithm [31,35]. BLR has b een shown to outp erform LDA in BCI research [14,16]. 2.4.3. L o gistic R e gr ession LR mo dels the conditional probability as a linear regression of feature inputs. The logistic mo del can b e constructed as p ( x ) = 1 1 + e − w T x + b (12) The optimization problem of an LR can b e constructed by minimizing the cost function as b elo w: J ( w , b ) = − 1 m m X i =1 [ y i log ( p ( x i )) + (1 − y i ) log (1 − p ( x i ))] + λ w T w (13) where y ∈ { 0 , 1 } , m is the sample nu mber of t wo classes and λ is the regularization parameter ( λ > 0). LR is part of a broader family of generalized linear mo dels (GLMs), where the conditional distribution of the resp onse falls in some parametric family , and the pa- rameters are set by the linear predictor. LR is the case where the resp onse (i.e. y in equation (13)) is binomial and it can giv e the prediction of the conditional probabilit y estimation. LR is easily implemented and has b een successfully applied to RSVP based BCI researc h [13,36]. 8 2.5. Evaluation The ev aluation describ ed in this work seeks to assess the relative p erformance when com bining three spatial filtering approaches with three linear clas sification metho ds, th us there are nine pip elines (spatial filtering × feature generation × classification) in total that are discussed in this paper. F or comparison, the original EEG ep o c hs without spatial filtering and only using PCA, are fed to three linear classifiers. P erformance of the differen t pip elines are ev aluated through the area under the curve (AUC) based on true p ositiv e rate (TPR) and false p ositive rate (FPR) It should b e noted that the pip eline describ ed in this pap er contains a num ber of h yp erparameters. Three spatial filtering approaches con tain a num ber of spatial filters N f as the h yperparameter. BLR contains data distribution v ariance ( β ) and parameter distribution v ariance ( α ) while LR has the regularization term ( λ ) as h yperparameters. Only LD A do es not require a h yp erparameter. T able 1 summarizes the h yp erparam- eters used in each pip eline. W e apply a random searc h [37] for 100 hyperparameter com binations in eac h pipeline and select these for ev aluation on a test set using 10-fold cross v alidation. The optimal mo del is then applied to the testing data to calculate the A UC score. 3. Data acquisition and pre-pro cessing The EEG datasets used in this w ork is from the Neurally Augmented Image Lab elling Strategies (NAILS) task as part of an op en data c hallenge carried out in 2017 [38]. EEG data from up to 9 participants in NAILS was used in this work. Data collection w as carried out with appro v al from Dublin Cit y Univ ersity’s Researc h Ethics Commit- tee (DCU REC/2016/099). EEG w as recorded along with timestamping information for image presentation (via a photo dio de and hardware trigger) to allow for precise ep oching of the EEG signals for each trial [39]. Eac h participant completed 6 different tasks (INSTR, WIND1, WIND2, UA V1, UA V2 and BIRD). F or eac h task, participan ts w ere ask ed to searc h for sp ecific target images from the presented images (i.e. an air- plane has the role of target in UA V1 and UA V2 tasks, a keyboard instrument is the target for the INSTR task, while a windfarm is the target in the WIND1 and WIND2 tasks, parrot b eing the target in the BIRD task, see Fig. 4). Eac h task w as divided in to 9 blo c ks, where eac h blo c k contained 180 images (9 targets/171 standards) th us there w ere 486 target and 9,234 standard images a v ailable for each participan t. Images were presen ted to participants at a 6 Hz presentation rate. EEG data was recorded using a 32 channel BrainVision actiCHamp at 1000 H z sampling frequency , using electro de lo cations as defined by the 10-20 system. Pre-pro cessing of some kind is generally a required step b efore an y meaningful in- terpretation or use of an y EEG data can b e realized. Pre-pro cessing typically in v olves re-referencing, filtering the signal (by applying a bandpass filter to remov e environ- men tal noise or to remov e activity in non-relev an t frequencies), ep o ching (extracting a time ep och typically surrounding the stim ulus onset) and trial/channel rejection (to remov e those containing artifacts) [9]. In this w ork, a common av erage reference (CAR) was utilized and a bandpass filter (e.g. 0.1-30 H z ) was applied to the dataset. EEG data w as then resampled at 250 H z and the analysis of a b ehavior resp onse is considered betw een 0 and 1 s after the presen tation of a stim ulus. T rial rejection based on HEOG and VEOG channels was applied for all participants. The dataset w as split in to a training/testing set of 66%/33% resp ectiv ely , b y selecting 3 blo c ks from eac h 9 searc h task to act as a withheld test set in the ev aluation. 4. Results 4.1. Imp act of Numb er of Sp atial Filters The P300 is not the only ERP that is commonly encoun tered when using an RSVP target searc h paradigm. Earlier ERPs (notably the N200) are often presen t alongside the P3 [40] and can b e useful in pro viding discriminativ e information for classification. Fig. 3 shows the discriminant ERPs for 9 participan ts in tw o different time regions and it can b e seen that b oth early time regions and later time regions generate dis- criminativ e ERP-related activity across participan ts. It can b e noted also that the sp ecific latencies and top ographies v ary across participants, hence N f ma y v ary across participan ts for capturing target-related ERP phenomena in the RSVP paradigm. F rom previous w ork [15], N f has b een set to 4 for b oth xDA WN and CSP metho ds. It is difficult to determine the optimal N f , th us w e lea v e it as a searc hing h yperparameter in our pip eline. Even selection of the num b er of spatial filters has b een stated in the area of the motor imagery BCI [41], w e hav e more hyperparameters in this work. In this case, we search for the optimal n um b er of spatial filters with other parameters together in each model, which has b een specified in the Ev aluation Section. It is worth reiterating again that this has not been explicitly rep orted up on previously in the area of RSVP . Fig. 5 sho ws an example of 10 spatial patterns and filters estimated with three spatial filtering approaches. It can b e seen that spatial patterns estimated with the same approach are differen t from each other whic h indicates target-related ERPs span broadly o ver b oth time and space in an RSVP paradigm. Hence, a search for an optimal v alue of N f is required for b est p erformance 4.2. Performanc e Evaluation This w ork ev aluated 9 differen t pip elines, comp osed of three classifiers with three spatial filtering metho ds. W e used whole original EEG ep o chs for training three clas- sifiers as measuring metrics for comparison. The AUC for each pip eline across nine participan ts are presen ted in T able 2. 4.2.1. Performanc e of sp atial filtering F or the p erformance of three spatial filtering metho ds, all three classifiers with CSP pre-pro cessing generate low er AUC score compared to those whic h do not use spatial filtering. This indicates that CSP is a wholly unsuitable spatial filtering approach for RSVP-BCI. Unlik e CSP , all three classifiers with MTWLB and xDA WN spatial filtering pre-pro cessing p erform b etter than those without spatial filtering which sho w the efficacy of MTWLB and xDA WN pre-pro cessing. This result demonstrates that it is critical to select carefully the precise spatial filtering metho d in RSVP-BCI and an ‘improp er’ spatial filtering metho d may hav e deleterious effects and degenerate p erformance to the level of not using any spatial filtering. CSP aims to maximize the EEG v ariance difference b et w een tw o classes. How ev er, the single-trial ERPs v ariance difference is v ery small in RSVP paradigm betw een t w o classes and the challenge for single-trial ERPs detection is its lo w SNR. Here w e define the ‘prop er’ spatial filtering approach for RSVP-based EEG as those metho ds 10 whic h impro v e the SNR for the EEG signals. Both MTWLB and xD A WN optimize for maximized SNR and as a result b oth p erform b etter than the inappropriately applied CSP metho d. A prop er spatial filtering metho d can not only impro v e the quality of the EEG data but also reduce the computational complexity since spatial filtering can reduce the EEG dimensionalit y . 4.2.2. Performanc e of classifier As men tioned b efore, CSP is not able to extract discriminan t features for ERPs gener- ated in an RSVP paradigm. Therefore, features generated by CSP ha v e negative effects on all three classifiers compared to the EEG data without spatial filtering. F rom re- sults generated b y LR across MTWLB and xD A WN and without spatial fitering, it can b e seen that the p erformance of LR impro v ed significantly when using non-CSP spatial filtering metho ds (i.e. 92.2% for MTWLB and 92.7% for xDA WN versus 87.5% and 88.5% without spatial filtering). This indicates that the qualit y of features has large impact on LR. F or another tw o classifiers, spatial filtering impro v es the p erfor- mance of LDA and BLR sligh tly . This indicates that LDA and BLR are more robust to the qualit y of features compared to LR. How ev er, LR sho ws go o d p erformance if go od features can b e extracted by pre-pro cessing (i.e. highest A UC score for LR with xD A WN). 5. Discussion In this pap er, we addressed tw o main issues: 1) the impact of choice of spatial filtering metho d on the p erformance of an RSVP-based BCI single trial classification task and 2) the sensitivity of different classifier’s p erformance to the feature types pro duced in a t ypical RSVP-BCI pip eline. Regarding the first issue, w e ha ve shown that the performance of our nov el MTWLB metho d and the p opular xDA WN metho d b oth improv e classifier p erformance. How- ev er, the p erformance generated by pip elines inv olving CSP is worse than those with- out using spatial fitering. This indicates that the c hoice of spatial filtering metho d is critical for single-trial detection of ERPs in an RSVP paradigm. In the literature, w e find some work whic h uses CSP for RSVP-based EEG [42,43] whic h illustrates that CSP is considered by at least some researc hers as b eing a suitable metho d for RSVP- BCI. Our results ho w ev er reveal that this is not an optimal choice. By comparing the optimization criterion of each spatial filtering approach, it should b e noted that MTWLB and xDA WN b oth use ERPs for calculating the spatial filters (i.e. ERPs difference p and estimated ERP resp onse D ˆ A are used for calculating w equation (3) and (5)). The main difference b et w een MTWLB and xD A WN is selecting the n um ber of spatial filters. In MTWLB, num b er of spatial filters is selected corresp onding to the divided individual time windo w that means eac h spatial filter maximizes SNR for ERPs in the selected time window. xDA WN uses generalized eigenv alue decomp osi- tion for whole EEG ep o c h and eigenv ectors that corresp ond to high eigenv alues will b e selected. Therefore, the num b er of spatial filters N f refers to the num b er of time windo ws in MTWLB while it refers to the num ber of eigen v ectors corresponding to the highest eigenv alues in xDA WN. F rom the classification result, the prop osed MTWLB giv es similar performance compared to xD A WN. Ho w ev er, this proposed approach can b e well-suited for generating spatial filters for those tasks that elicit ERPs in sp e- 11 cific time regions. CSP also uses generalized eigenv alue decomp osition for optimizing the spatial filter but it uses a single-trial EEG ep o c h instead of ERPs to calculate the co v ariance matrix in equation (7). Because RSVP-based EEG has low SNR, this optimization form ulation can b e effected significan tly b y lo w SNR in this case. CSP origins from the motor imagery BCI in whic h sensor motor rhythm (SMR), a p eri- o dic EEG, is elicited in the exp erimen t [44]. The optimization criterion for CSP is maximizing the difference of v ariance b et w een tw o classes which coincides with the prop ert y of SMR. In an RSVP paradigm, ERPs are elicited alongside the steady-state visual evok ed p oten tials (SSVEP) and there is a very small difference of the v ariance b et w een target and standard class es. The challenge for single-tiral ERP detection is its lo w SNR. MTWLB and xD A WN aim to impro v e the SNR and SSNR resp ectively for the reconstructed signal which ov ercomes the low SNR problem. Therefore, MTWLB and xDA WN are very suitable for single-trial ERP detection in an RSVP paradigm. W e explored further the p erformance difference b et w een MTWLB and xD A WN. W e used one-w ay ANOV A on the mean v alue of three classification metho ds applied with MTWLB (92.2%) and xDA WN (92.4%) i.e. [ F (1 , 16) = 0 . 004 , p = 0 . 95], which indi- cates the insignificant difference of the p erformance b et w een these tw o spatial filtering metho ds. This suggests that the p erformance of these t w o metho ds are similar for this dataset. While MTWLB and xDA WN p erform similarly , MTWLB still has some adv antages. First, the prop osed metho d MTWLB giv es an intuition of pro ducing the spatial filter corresp onding to the time-line. F rom the generated spatial filter w or the pro jected subspace Ψ , we can see the spatial filter or pro jected subspace change o v er time. F or example, the spatial filter and spatial pattern change with time in MTWLB from left to right in Fig. 5. It pro vides a more ph ysiological representation of the spa- tial pattern and the spatial filter changing with time whic h the conv entional spatial filtering approach is not able to represent. Second, we searched for the appropriate time window for MTWLB due to the inheren t v ariabilit y in ERP latencies b et w een participan ts in our case. How ever, MTWLB can b e effective for those cases in which the time region for the ERPs are known in adv ance. Hence, there is no need to search for the time window and the computational complexity is reduced significantly . Third, p erformance of xD A WN can b e affected b y the selected ep och length because the opti- mization of xD A WN is based on the whole ep och. On the contrary , MTWLB estimates the spatial pattern and cov ariance in the sp ecific time windo w instead of the whole EEG ep o c h. So changing ep o ch length will ha v e no effect on the p erformance of the MTWLB algorithm. Regarding the second issue of the effect of features on classifier p erformance w e ha v e sho wn the p erformance of three classifiers in different pip elines. It can b e noted that LD A and BLR outp erform LR in the CSP pip eline and the pip eline without spatial filtering. This indicates that LR is more sensitiv e to the qualit y of features compared to LDA and BLR. LR is used for mo deling the relationship b etw een indep enden t and categorical dep enden t v ariables and v ariable colinearities may hav e negative effects on estimation [45]. F rom the pipeline with prop er spatial filtering, three classifiers p erform closely to each other with MTWLB and LR outp erforming the other tw o metho ds in the pip eline when using xD A WN. This indicates that the LR classifier p erforms well with informative features as input. LD A and BLR hav e b een used more widely compared to LR in the literature since LD A performs well ev en without feature extraction and is simpler to implement [2,10,15,16]. Here we hav e sho wn that LR is able to generate very go o d p erformance when informativ e features are extracted from RSVP-based EEG. Result in this paper partly supp orts the result in [15] that spatial filtering can 12 impro v e the o v erall p erformance. In this work, we p erformed a more comprehensiv e comparison of the spatial filtering pip eline. First, we included a random searc h [37] for the set of h yp erparameters listed in the T able 1 in order to attain optimal performance. In the exp eriment, w e found that the num ber of spatial filters can ha v e critical impact on the final classification performance and it v aries with different participan ts. Instead of lea ving it as a sp ecific num ber [15], we suggest to search it as a hyperparameter in the pip eline in order to optimize the p erformance of spatial filtering. Second, the t yp e of spatial filtering is critical to the classification p erformance for differen t t yp es of EEG. F or example, CSP has b een widely used for motor imagery based BCI [30,46,47], where oscillatory EEG activit y is elicited in the experiment [48,49]. How ev er, the classification p erformance of pip eline with applying CSP spatial filtering is ev en worse than pip elines without spatial filtering, whic h supp orts the results in [15]. These results suggest that improp er use of spatial filtering in RSVP-based BCI system can ha v e negativ e impact on the classification p erformance and this conclusion can b e extended to other types of EEG activit y . On the con trary , applying the appropriate spatial filtering tec hnique (i.e. MTWLB, xDA WN for RSVP-based EEG in this w ork) results in reduced computational complexity and impro v emen t in classification p erformance. This work demonstrates that it is critical to c ho ose the appropriate type of spatial filtering for the signal pro cessing pip eline in RSVP-based BCI systems. 6. Conclusion In this work, w e present a nov el spatial filtering approach (MTWLB) for RSVP-based EEG. Our results demonstrate comparable p erformance with the leading metho d of xD A WN although the approach is significantly different. Consequen tly the metho d presen ts a different set of optimization parameters which ma y make it suitable for particular RSVP-BCI implemen tations. Even there is no statistical significance b e- t w een our prop osed metho d and xD A WN, MTWLB presents useful prop erties that lend themselv es to certain RSVP-BCI p erformance optimizations not av ailable via xD A WN. First, the metho d is more robust to EEG ep o ch length compared to the con- v en tional spatial filtering approac hes (e.g. xD A WN and CSP) b ecause its optimization relies on the time windo w instead of whole ep o chs. Second, MTWLB can b e more effec- tiv e when knowing ERPs time region in adv ance b ecause there is no need to search for the time window and the computational complexity is reduced significan tly . F urther- more this work includes a thorough ev aluation of single-trial classification pip elines with a num b er of spatial filters and classifiers in a comprehensiv e wa y using a publicly a v ailable dataset. W e hav e sho wn that the selection of spatial filtering metho d should corresp ond to the nature of ERPs elicited in the task paradigm and that naiv e applica- tion of the approach ma y not pro duce go o d performance. Finally we demonstrate that ev en though LDA and BLR are the most prev alent classification approaches used in RSVP-based BCI research, the LR metho d can b e even more effective for single-trial ERPs detection when go od quality features are made av ailable through the spatial fil- tering metho ds. In summary this pap er should help inform designers of RSVP-BCI of an appropriate spatial filtering / classifier c hoice at design time based on results with a publicly a v ailable dataset which allows comparative b enc hmarking of p erformance. 13 7. Ac kno wledgemen t This work is funded as part of the Insigh t Cen tre for Data Analytics whic h is supp orted b y Science F oundation Ireland under Grant Number SFI/12/RC/2289. 14 8. App endix (a) Pipeline including spatial filtering (b) Pipeline excluding spatial filtering Figure 1. Two pipeline arc hitectures for RSVP-based EEG discussed in this pap er. Figure 2. Spatial pattern estimation for LDA b eamformer using whole EEG ep och via training data using CAR: Particip ant 2 . 15 T able 1. Hyperparameter summary for each pip eline discussed in this pap er. Pip eline Hyp erparameter MTWLB LDA N f MTWLB BLR N f , β , α MTWLB LR N f , λ xD A WN LDA N f xD A WN BLR N f , β , α xD A WN LR N f , λ CSP LDA N f CSP BLR N f , β , α CSP LR N f , λ ALL LDA N one ALL BLR β , α ALL LR λ (a) Early time region ERP for 9 participants from left to right. (b) Later time region ERP for 9 participants from left to right. Figure 3. ERPs for each participant corresp onding to time regions of (a) 200-340 ms and (b) 370-700 ms . These are selected for presentation to emphasize the presence of discriminative ERP-related activity in these time regions across participants, namely time regions coinciding with P200, N200 and P300 ERP activit y . Figure 4. Examples of four target images used in the exp eriment. Airplane, Keyboard instrument, Wind farm and Macaw resp ectively from left to righ t. 16 (a) Spatial patterns estimated by xDA WN (b) Spatial filters estimated by xDA WN (c) Spatial patterns estimated by MTWLB (d) Spatial filters estimated by MTWLB (e) Spatial patterns estimated by CSP (f ) Spatial filters estimated by CSP Figure 5. Example of estimated spatial patterns with three spatial filtering approaches by setting N f = 10 based on 0-1 s EEG ep ochs: Particip ant 1 . 17 T able 2. A UC score (%) for different pip elines across nine participants in testing session Pip eline P articipan t Mean 1 2 3 4 5 6 7 8 9 MTWLB LDA 88.0 93.8 92.5 96.8 91.4 93.7 93.5 90.1 89.9 92.2 MTWLB BLR 88.0 93.8 92.5 96.8 92.2 93.7 93.2 90.8 93.3 92.3 MTWLB LR 88.5 93.0 89.8 97.4 91.7 94.1 93.3 91.3 90.9 92.2 Mean 88.2 93.5 91.6 97.0 91.8 93.8 93.3 90.7 91.4 92.2 xD A WN LDA 88.0 93.4 92.7 97.3 91.6 94.3 92.9 90.6 90.1 92.3 xD A WN BLR 88.5 93.4 92.8 96.6 91.6 94.3 92.8 90.6 90.1 92.3 xD A WN LR 87.4 94.1 92.9 97.2 91.9 95.3 93.8 91.3 90.7 92.7 Mean 88.0 93.6 92.8 97.0 91.7 94.6 93.2 90.8 90.3 92.4 CSP LDA 84.3 92.8 88.8 96.0 87.7 90.9 89.8 76.9 89.7 88.5 CSP BLR 84.9 92.7 88.3 96.0 87.9 90.9 89.6 77.5 89.7 88.6 CSP LR 83.8 92.7 85.7 93.1 86.6 90.3 89.5 76.8 89.0 87.5 Mean 94.3 92.7 87.6 95.0 87.4 90.7 89.6 77.1 89.5 88.2 ALL LDA 86.9 93.4 90.8 96.7 91.9 94.0 95.0 89.4 90.3 92.0 ALL BLR 88.0 93.1 91.4 96.0 91.6 93.4 93.8 89.4 90.7 91.9 ALL LR 84.4 92.4 86.0 92.3 87.6 92.1 91.5 87.3 82.8 88.5 Mean 86.4 93.0 89.4 95.0 90.4 93.2 93.4 88.7 87.9 90.8 18 References [1] Gerson AD, P arra LC, Sa jda P . Cortically coupled computer vision for rapid image search. IEEE T ransactions on Neural Systems and Rehabilitation Engi- neering. 2006 June;14(2):174–179. [2] Bigdely-Shamlo N, V ank o v A, Ramirez RR, et al. Brain activit y-based image classification from rapid serial visual presen tation. IEEE T ransactions on Neural Systems and Rehabilitation Engineering. 2008;16(5):432–441. [3] Healy G, Smeaton AF. Ey e fixation related p oten tials in a target search task. In: Engineering in Medicine and Biology So ciety , EMBC, 2011 Annual In ternational Conference of the IEEE; IEEE; 2011. p. 4203–4206. [4] Sp ence R, Witko wski M. Rapid serial visual presentation: design for cognition. Springer; 2013. [5] Polic h J. Up dating P300: an in tegrativ e theory of P3a and P3b. Clinical neuro- ph ysiology . 2007;118(10):2128–2148. [6] Lawhern V, Solon A, W ayto wich N, et al. Eegnet: a compact con volutional neural net w ork for eeg-based brain–computer interfaces. Journal of neural engineering. 2018;. [7] Lees S, Da yan N, Cecotti H, et al. A review of rapid serial visual presen tation-based brain–computer in terfaces. Journal of neural engineering. 2018;15(2):021001. [8] T eplan M, et al. F undamen tals of EEG measuremen t. Measurement science re- view. 2002;2(2):1–11. [9] Luck SJ. Even t-related p otentials. American Psychological Asso ciation; 2012. [10] Meng J, Meri ˜ no LM, Shamlo NB, et al. Characterization and robust classification of EEG signal from image RSVP even ts with independent time-frequency features. PloS one. 2012;7(9):e44464. [11] Parra LC, Christoforou C, Gerson A C, et al. Spatiotemp oral linear deco ding of brain state. IEEE Signal Pro cessing Magazine. 2008;25(1):107–115. [12] Rivet B, Souloumiac A, A ttina V, et al. xD A WN algorithm to enhance ev ok ed po- ten tials: application to brain–computer interface. IEEE T ransactions on Biomed- ical Engineering. 2009;56(8):2035–2043. [13] Huang Y, Erdogmus D, Mathan S, et al. Bo osting linear logistic regression for single trial ERP detection in rapid serial visual presentation tasks. In: Engineer- ing in Medicine and Biology So ciety , 2006. EMBS’06. 28th Annual International Conference of the IEEE; IEEE; 2006. p. 3369–3372. [14] Hoffmann U, V esin JM, Ebrahimi T, et al. An efficien t P300-based brain– computer in terface for disabled sub jects. Journal of Neuroscience metho ds. 2008; 167(1):115–125. [15] Cecotti H, Eckstein MP , Giesbrech t B. Single-trial classification of even t-related p oten tials in rapid serial visual presen tation tasks using supervised spatial filtering. IEEE transactions on neural netw orks and learning systems. 2014; 25(11):2030–2042. [16] Cecotti H, Ries AJ. Best practice for single-trial detection of even t-related p o- ten tials: Application to brain-computer interfaces. International Journal of Psy- c hoph ysiology . 2017;111:156–169. [17] W ang Z, Healy G, Smeaton AF, et al. A review of feature extraction and clas- sification algorithms for image rsvp based b ci. Signal Pro cessing and Machine Learning for Brain-Mac hine In terfaces. 2018;:243–270. [18] T reder MS, Porbadnigk AK, Av arv and FS, et al. The LDA b eamformer: Optimal 19 estimation of erp source time series using linear discriminan t analysis. NeuroIm- age. 2016;129:279–291. [19] Blankertz B, T omiok a R, Lemm S, et al. Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal pro cessing magazine. 2008;25(1):41–56. [20] Alp ert GF, Manor R, Spanier AB, et al. Spatiotemp oral representations of rapid visual target detection: a single-trial eeg classification algorithm. IEEE T ransac- tions on Biomedical Engineering. 2014;61(8):2290–2303. [21] Hyv¨ arinen A, Oja E. Indep enden t comp onent analysis: algorithms and applica- tions. Neural net w orks. 2000;13(4):411–430. [22] Makeig S, Onton J, et al. ERP features and EEG dynamics: an ICA p ersp ec- tiv e. Oxford Handb o ok of Even t-Related P oten tial Comp onen ts New Y ork, NY: Oxford. 2009;. [23] Jung TP , Mak eig S, Humphries C, et al. Remo ving electro encephalographic arti- facts b y blind source separation. Psyc hophysiology . 2000;37(2):163–178. [24] Onton J, W esterfield M, T o wnsend J, et al. Imaging h uman eeg dynamics using indep enden t comp onent analysis. Neuroscience & Biob ehavioral Reviews. 2006; 30(6):808–822. [25] Naeem M, Brunner C, Pfurtsc heller G. Dimensionalit y reduction and channel selection of motor imagery electro encephalographic data. Computational intelli- gence and neuroscience. 2009;2009. [26] Zanotelli T, San tos Filho S, Tierra-Criollo C. Optimum principal components for spatial filtering of EEG to detect imaginary mov ement b y coherence. In: En- gineering in Medicine and Biology So ciet y (EMBC), 2010 Ann ual International Conference of the IEEE; IEEE; 2010. p. 3646–3649. [27] Chen Y, Wiesel A, Eldar YC, et al. Shrink age algorithms for MMSE co v ariance estimation. IEEE T ransactions on Signal Pro cessing. 2010;58(10):5016–5029. [28] Cecotti H, Rivet B, Congedo M, et al. A robust sensor-selection metho d for P300 brain–computer in terfaces. Journal of neural engineering. 2011;8(1):016001. [29] Parlett BN. The symmetric eigenv alue problem. SIAM; 1998. [30] W ang Y, Gao S, Gao X. Common spatial pattern method for channel selelction in motor imagery based brain-computer interface. In: Engineering in Medicine and Biology So ciety , 2005. IEEE-EMBS 2005. 27th Annual International Conference of the; IEEE; 2006. p. 5392–5395. [31] Bishop CM. Pattern recognition and machine learning. Springer; 2006. [32] F riedman JH. Regularized discriminan t analysis. Journal of the American statis- tical asso ciation. 1989;84(405):165–175. [33] Xue JH, Titterington DM. Do un balanced data hav e a negativ e effect on LD A ? P attern Recognition. 2008;41(5):1558–1571. [34] Gauv ain JL, Lee CH. Maximum a p osteriori estimation for multiv ariate gaussian mixture observ ations of marko v chains. IEEE transactions on sp eec h and audio pro cessing. 1994;2(2):291–298. [35] MacKay DJ. Bay esian interpolation. Neural computation. 1992;4(3):415–447. [36] Sa jda P , Gerson AD, Philiastides MG, et al. Single-trial analysis of EEG during rapid visual discrimination: Enabling cortically-coupled computer vision. T ow ards brain-computer in terfacing. 2007;:423–44. [37] Bergstra J, Bengio Y. Random search for hyper-parameter optimization. Journal of Mac hine Learning Researc h. 2012;13(F eb):281–305. [38] Healy G, W ang Z, Currin C, et al. An eeg image-searc h dataset: A first-of-its-kind in IR/I IR. NAILS: neurally augmented image labelling strategies. In: In Pro ceed- ings of CHIR W orkshop on Challenges in Bringing Neuroscience to Research in 20 Human-Information In teraction; 11 Mar 2017, Oslo, Norw ay . [39] W ang Z, Healy G, Smeaton AF, et al. An in vestigation of triggering approaches for the rapid serial visual presentation paradigm in brain computer in terfacing. In: Signals and Systems Conference (ISSC), 2016 27th Irish; IEEE; 2016. p. 1–6. [40] O’Donnell BF, Cohen RA. The N2-P3 complex of the evok ed p oten tial and h uman p erformance. 1988;. [41] Y ang Y, Chev allier S, Wiart J, et al. Automatic selection of the num ber of spatial filters for motor-imagery b ci. In: The pro ceeding of 20th Europ ean Symp osium on Artificial Neural Net w orks, Computational Intelligence and Machine Learning (ESANN 2012); 2012. p. 109–114. [42] Y u K, Shen K, Shao S, et al. Common spatio-temporal pattern for single-trial detection of ev en t-related p oten tial in rapid serial visual presen tation triage. IEEE T ransactions on Biomedical Engineering. 2011;58(9):2513–2520. [43] Y u K, Shen K, Shao S, et al. A spatio-temp oral filtering approach to denoising of single-trial erp in rapid image triage. Journal of neuroscience metho ds. 2012; 204(2):288–295. [44] Hari R, Salmelin R. Human cortical oscillations: a neuromagnetic view through the skull. T rends in neurosciences. 1997;20(1):44–49. [45] Park H. An in tro duction to logistic regression: from basic concepts to interpreta- tion with particular atten tion to nursing domain. Journal of Korean Academ y of Nursing. 2013;43(2):154–164. [46] Ramoser H, Muller-Gerking J, Pfurtsc heller G. Optimal spatial filtering of single trial eeg during imagined hand mov emen t. IEEE transactions on rehabilitation engineering. 2000;8(4):441–446. [47] Lu H, Eng HL, Guan C, et al. Regularized common spatial pattern with aggrega- tion for eeg classification in small-sample setting. IEEE transactions on Biomed- ical Engineering. 2010;57(12):2936–2946. [48] Pfurtscheller G, Neuper C, Guger C, et al. Current trends in graz brain-computer in terface (b ci) researc h. IEEE T ransactions on Rehabilitation Engineering. 2000; 8(2):216–219. [49] Pfurtscheller G, Neup er C. Motor imagery and direct brain-computer communi- cation. Pro ceedings of the IEEE. 2001;89(7):1123–1134. 21