Parametric Order Constraints in Multinomial Processing Tree Models: An Extension of Knapp and Batchelder (2004)

Order Constrain ts 1 R unning head: Order Constrain ts P arametric Order Constrain ts in Multinomial Pro cessing T ree Mo de ls: An Extension of Knapp and Ba t cheld er (200 4 ) Karl Christoph Klauer, Henrik Singmann, a nd D a vid Kellen Alb ert-Ludwigs-Univ ersit¨ at F reiburg A uthor Not e Karl Christoph Klauer, Institut f ¨ ur Psyc hologie; Henrik Singmann, Institut f ¨ ur Psyc hologie; David Kellen, Institut f ¨ ur Psyc hologie. The first tw o authors contributed to equal exten ts to the man uscript. Correspo ndence concerning this article should b e addressed to K. C. Klauer at the Institut f ¨ ur Psyc hologie, Alb ert-Ludwigs-Univ ersit¨ at F reiburg, D-79085 F reiburg, Germany . Elec tronic mail ma y b e sen t to c hristoph.klauer@psy chologie.u ni-fr eiburg.de . A ddress information for Christoph Klauer (corresp o nding author): • E-mail: c hristoph.klauer@psy chologie.u ni-fr eiburg.de • Phone: +49 761 2032469 • P ostal address: – Ins titut f ¨ ur P syc hologie – Albert-Ludwigs-Univ ersit¨ at F reiburg – D-79085 F reiburg – German y Order Constrain ts 2 Abstract Multinomial pro cessing tree (MPT) mo dels are to o ls fo r disen tangling t he con tributions o f laten t cognitiv e pro cesses in a giv en exp erimen tal paradigm. The presen t note analyzes MPT mo dels sub ject to order constraints on subsets of its parameters. The constrain ts that we consider frequen tly arise in cases where the respo nse categories are ordered in some sense suc h as in confidence-rating data, Like rt scale data, where graded guessing tendencies or resp onse biases are created via base-rate or pay off manipulations, in the analysis o f con tingency tables with order constrain ts, a nd in man y other cases. W e sho w how to construct an MPT mo del without order constrain ts that is statistically equiv alen t to the MPT mo del with order constrain ts. This new closure result extends the mathematical analysis of the MPT class, and it offers an approach to order-restricted inference that extends the approac hes discuss ed by Knapp and Batchel der (2004 ) . The usefulnes s of the metho d is illustrated by means of an analysis of an order-constrained v ersion of the t w o-high- threshold mo del for confidence ratings. KEYW ORDS: Multinomial pro cessing tree mo dels, mathematical mo dels, categorical data, m ultinomial distribution Order Constrain ts 3 Multinomial pro cessi ng tree (MPT) mo dels are used to measure cognitiv e pro cesses in many areas of psyc hology (for reviews, see Batche lder & Riefer, 1999; Erdfelder et al., 2009). They are mo dels for categorical data. MPT mo dels are t ypically tailored to a give n exp erimen tal para digm and sp ecify how the most imp ortan t pro cesses assumed to b e in volv ed in data generation in the paradigm in teract to pro duce o bserv able resp onses. As an example consider the t w o-high-t hreshold mo del (2HTM; Sno dgrass & Corwin, 1988). The 2HTM is tailored to memory exp erimen ts in whic h old/new judgmen ts a r e requested for previously studied items in termixed with new items. In man y suc h exp erimen ts, participants are also ask ed to rate their confidence in eac h “old” or “new” judgmen t. Figure 1 sho ws a ve rsion of the mo del for a confidence rating scale with three p oints , lab eled “high”, “medium”, and “low” (Br¨ oder, Kellen, Sc h ¨ utz, & Rohrme ier, 2013 ; Klauer & Kellen , 2011). Resp onses are me diated via three laten t states, lab eled “detect o ld”, “detect new”, a nd “no detection” . P arameters D o and D n define a stim ulus-state mapping. D o is the probability of en tering the “detect old” state for an old item; D n of en tering the “detect new” state for a new item; the “no detection” state is en tered with probability 1 − D o and 1 − D n for old and new items, resp ectiv ely . The remaining parameters define state-respo nse mappings. Given one of the t w o “detect” states, the old/new judgment is in v ariably correct as regards the old v ersus new status of the test item, and the parameters s l , s m , and s h ( s l + s m + s h = 1) quan tify the probabilities of selecting, in order, the low, medium, and high confidence lev el in the o ld/new resp onse. 1 In the absence of detection, there is a guessing bias captured b y probabilit y parameter g , quan tifying the probabilit y o f guessing “old” rather than “new” . Give n that “o ld” is guessed, parameters o l , o m , and o h with o l + o m + o h = 1 parameterize the probabilities for the three confidence lev els; giv en that “new” is guessed, n l , n m , and n h parameterize these probabilities. As can b e seen, MPT mo dels assume that observ ed category coun ts arise Order Constrain ts 4 from pro cessing branche s consisting of separate conditional links or stages. Eac h branc h probability is the pro duct of its conditional link probabilities, and more than one branc h can terminate in the same observ ed category (Hu & Batch elder, 1994). In most cases, the mo dels are ev en tually represen ted as so- called binary MPT mo dels (Purdy & Batc helder, 2 0 09), b ecause man y soft w are to ols for analyzing MPT mo dels require binary MPT mo dels a s input. In a binary MPT mo del, exactly t wo links go out f rom eac h non-terminal no de. The t wo links are lab eled b y t w o parameters that sum t o one. One o f these is redundan t and is replaced by one min us the other para meter so that the remaining mo del parameters are functionally independen t, eac h suc h parameter r a nging from 0 to 1. It is straigh tforw ard to transform a non-binary MPT mo del into a statistically equiv alen t binary MPT mo del (Hu & Batche lder, 1994). T wo mo dels are statistically equiv alen t if they can predict the same sets of resp onse proba bilities. In applications, it is not uncommon that order constrain ts are predicted to hold for subsets o f the functionally indep enden t parameters of binary MPT mo dels (Baldi & Batche lder, 2003; K napp & Batche lder, 2004), a nd Knapp and Batche lder ha v e sho wn that the mo del class is closed under one or more non-ov erlapping linear orders of parametric constraints . That is, a new non-constrained binary MPT mo del can b e constructed using a differen t set of functionally indep enden t para meters that is statistically equiv a len t to the original mo del with the order constraints . Here, we consid er a differen t set of order constraints that regularly arise in applications and that a r e not co ve red b y Knapp and Batche lder (2004). The o rder constrain ts f r equen tly arise where resp o nse categories are ordered in some sense suc h as for confidence ratings or Like rt scales. They also arise where participan ts discriminate b et we en tw o or more categories of items and the probabilities of guessing the categories in “no detection” states can b e a ssumed to b e ordered; for example, b ecause base ra t es or pa yoffs systematically differ b et w een the categories. Our results also apply to the imp ortant case of order constraints on the probabilities of a Order Constrain ts 5 m ultinomial or pro duct-multi nomial distribution that is frequen tly encoun tered within and outside psyc hology (e.g., Agresti & Coull, 20 02). W e sho w tha t an MPT mo del with the order constrain ts can b e represen ted in the form o f a statistically equiv alen t non-constrained MPT mo del. This new closure prop ert y con tributes to the structural analysis of the MPT class and is immediately useful for analyzing cases in whic h the order constrain ts are to b e imp osed up on the parameters as exemplified b elo w. Considering, for example, the 2HTM for confidence ratings, a psyc hologically plausible constrain t on the parameters for the confidence lev els in the “no detection” state is that the preference f o r a giv en confidence lev el should decline from lo we st to highest confidence lev els, reflecting the respo nden t’s uncertain t y in the absence of detection: o l ≥ o m ≥ o h and n l ≥ n m ≥ n h . Con ve rsely , in “detect” states, the preference for a giv en confidence lev el should increase from lo w est to highest confidence, at least for scales with only a few confidence lev els: s l ≤ s m ≤ s h . Imp osing suc h constrain ts sharp ens the distinctions b et w een “no detection” and “detect” states b y highlighting plausible qualitativ e differences b etw ee n them. When satisfied b y the underlying probabilit y distribution, the constrain ts con tribute to making the estimation of the parameters D n and D o of the stim ulus-state mapping more precise, fo cused, and robust, and they considerably increase the mo del’s parsimon y a s elab orated on b elo w. These order constrain ts are imp osed on functionally dep enden t parameters (e.g., s l , s m , and s h ha v e to sum to 1 and are therefore not indep enden t). Hence, they are not cov ered b y Knapp and Batchel der’s (20 0 4) approac h to order constrain ts for independen t parameters. Nev ertheless, it is p ossible to express them in the language of MPT mo dels. The next section describes how to transform MPT mo dels with order constrain ts o f this kind in to equiv alen t non-constrained MPT mo dels. 2 Finally , w e illustrate the new metho d b y comparing v ersions of the 2HTM with and without order constrain ts in terms of mo del complexit y and in terms of their description of a dataset Order Constrain ts 6 b y Koen and Y onelinas (201 0). The general discussion expands on the adv an tages of the new metho d for estimation and inference with order-constrained mo dels. Order Constrain t s on Multinomial Probabilities W e consider a ba sic subtree with a ro ot, no other non-terminal no de, and tw o or more links going out from its ro ot as sho wn on the left side of Figure 2. This subtree migh t o ccur at one or more places in the tree represen tation of the complete mo del. With r ega rd to the ov erall MPT mo del, its terminal no des A 1 , . . . , A k represen t either other subtrees or observ able categories. F or example, the sub tree with three links lab eled b y parameters s l , s m , and s h in the ab o v e 2HTM o ccurs at tw o places in the pro cessing tree represen tation (see F igure 1). Our basic result describes how to represen t a linear order η 1 ≥ η 2 ≥ · · · ≥ η k on the parameters of the subtree b y means of a statistically equiv ale nt MPT mo del without order constraints on the parameters. As sho wn in Figure 2 the solution is to replace eac h o ccurrence of the subtree in question by the subtree on the righ t side in Figure 2. Replacemen t means that the tree on the right side replaces the subtree o n the left side wherev er it o ccurs in the pro cess ing- t ree represen tation. F urthermore, whatev er is app ended at the terminal no de A j of an o ccurrence of the original subtree is app ended in the replacing subtree wherev er a terminal no de lab eled A j o ccurs. As for the reparameterizations in Knapp and Batc helder (20 04) this regularly implies an increase in the size of the pro cessing tree. W e pro ve the follow ing t wo theorems: Theorem 1: F or the tree sho wn on the right side of Figure 2 and any set of non-negativ e parameters λ i , i = 1 , . . . , k , with P k i =1 λ i = 1, the probabilities of outcomes A i are ordered as P ( A 1 ) ≥ P ( A 2 ) ≥ . . . ≥ P ( A k ). Theorem 1 states that the tree indeed imp oses an order constrain t on its k outcome probabilities. Theorem 2 complemen ts this b y sho wing that any ordered set of probabilities can b e represen ted in this f orm: Theorem 2: F or an y set of ordered probabilities η i with η 1 ≥ η 2 ≥ . . . ≥ η k , Order Constrain ts 7 P i η i = 1, there exist non- negativ e v alue s λ i , i = 1 , . . . , k with P i λ i = 1 suc h that the outcome probabilities of the tree on the righ t side of Figure 2 are giv en b y P ( A i ) = η i , i = 1 , . . . , k . Pro of of Theorem 1. Note that the outcome probabilities of the tr ee on the righ t side of Figure 2 are mixtures with mixture co efficien ts λ j . The first mixture comp onen t is giv en b y the probabilit y distribution with P ( A 1 ) = 1 and P ( A j ) = 0, j > 1; the second b y P ( A 1 ) = P ( A 2 ) = 1 / 2 and P ( A j ) = 0, j > 2; the last b y P ( A j ) = 1 /k for all j . Eac h mixture comp onent satisfies the inequalities, P ( A j ) ≥ P ( A j +1 ), j = 1 , . . . , k − 1. This completes the pro of. Pro of of Theorem 2. Define mixture co efficien ts λ j as follows : λ k = k η k and λ j = j ( η j − η j +1 ) , j = 1 , . . . , k − 1 . (1) By the premises of Theorem 2 , it is immediate that λ j ≥ 0. It is furthermore easy to see via simple manipulations that P k j =1 λ j = P k j =1 η j = 1 and that η j = P k i = j λ i i . Hence, P ( A j ) = P k i = j λ i i = η j , j = k , k − 1 , . . . , 1. This completes the pro of. Extensions In this section, w e consider linear-order constrain ts on only a subset of the η i , non-o v erlapping linear-order constrain ts on the η i , more general part ia l orders, and all suc h o r ders for k = 3 and k = 4. Linear orders on subsets and m ultiple non- ov erlappi ng linear orders. Theorems 1 and 2 co ve r the case of a total order on all k parameters η i . It is straigh tforw ard to extend t he results to the case of a partial linear order on only a subset of the η i . F or example, assume k = 6 and that the constrain t η 1 ≥ η 2 ≥ η 3 is to b e imp osed. It is easy to see that this translates in to the constraint θ 1 ≥ θ 2 ≥ θ 3 in the reparameterization depicted in Figure 3, where θ 1 + θ 2 + θ 3 = 1 so that Theorems 1 and 2 apply . The reparameterization in Figure 3 a lso sho ws ho w t w o or more non-o v erlapping linear-order constrain ts can b e imp osed up on the η i with P i η i = 1. F or example, if in addition η 4 ≥ η 5 ≥ η 6 is to b e imp osed, this could b e implemen ted Order Constrain ts 8 b y imp osing the constrain t θ 4 ≥ θ 5 ≥ θ 6 , where θ 4 + θ 5 + θ 6 = 1, in terms of the parameters θ i in Figure 3. General partial orders. More general partial orders can often b e treated b y the follo wing idea. The probability distributions η = ( η 1 , . . . , η k ) that satisfy a set of linear inequalit y constrain ts form a con v ex p olytop e. Sp ecifically , they can exhaustiv ely b e represen ted as mixtures of certain fixed probabilit y distributions η 1 , η 2 , . . . , η l that w e refer to a s v ertices. F or small problems, the v ertices can b e found graphically; in complex cases, linear programming algorithms can b e used. F or example, for the linear order with η 1 ≥ η 2 ≥ . . . ≥ η k , the vertic es are giv en b y the ab o ve - men tioned k mixture comp onen ts, η 1 = (1 , 0 , . . . , 0), η 2 = (1 / 2 , 1 / 2 , 0 , . . . , 0), . . . , η k = (1 /k , . . . , 1 / k ), and the mo del parameters λ j of the non-constrained MPT mo del express ing the order constrain t are simply the mixture w eigh ts. The subtree follo wing λ j co des the probabilit y distribution sp ecified in v ertex η j P artial orders with k = 3. This immediately solv es the case of orderings among k parameters that can b e represen ted b y k v ertices. Consider, for example, k = 3, and the ordering η 1 ≤ η 2 and η 1 ≤ η 3 . This defines a con ve x p olytop e with the three v ertices η 1 = (0 , 0 , 1) , η 2 = (0 , 1 , 0) , and η 3 = (1 / 3 , 1 / 3 , 1 / 3). Hence, a non-constrained MPT mo del represen ting the order restrictions has three mixture co efficien ts λ i , i = 1 , . . . , 3, as para meters. Eac h λ i is link ed to a subtree co ding the respectiv e probabilit y distributions η i . In other cases, more than k ve rtices are required. F or example, the ordering η 1 ≥ η 2 , η 1 ≥ η 3 defines a p olytop e with four v ertices, η 1 = (1 , 0 , 0), η 2 = (1 / 2 , 1 / 2 , 0), η 3 = (1 / 2 , 0 , 1 / 2), and η 4 = (1 / 3 , 1 / 3 , 1 / 3). Using four mixture co efficien ts λ i to represen t the order-constrained MPT mo del as a non-constrained MPT mo del is p ossible, but results in an o v erparameterized mo del. This means that certain analyses and inferences av ailable fo r MPT mo dels without ov erparamete rization cannot b e done. F or example, although it is p ossible to determine the mo del’s maxim um Order Constrain ts 9 log-lik eliho o d and G 2 go o dness - of-fit statistic and to b o otstrap its distribution (e.g., Singmann & Kellen, 2013), it will not b e p ossible to determine unique parameter estimates. F or the presen t example, it can how ev er b e sho wn that represen ting the four mixture co efficien ts λ i b y tw o indep enden t parameters θ 1 and θ 2 with 0 ≤ θ i ≤ 1 suc h that λ 1 = (1 − θ 1 )(1 − θ 2 ), λ 2 = θ 1 (1 − θ 2 ), λ 3 = (1 − θ 1 ) θ 2 , and λ 4 = θ 1 θ 2 is sufficien t to span the entire p olytop e defined b y the ab ov e v ertices (see app endix). This immediately leads to a non-redundan t parameterization using tw o indep enden t parameters θ 1 and θ 2 . But an analogo us construction do es not in g eneral guarante e this in other cases. Nev ertheless, for most purp oses it is sufficien t that a non-redundan t parameterization is found that co v ers the p oin t o f maxim um lik eliho o d (as determined, for example, via estimating the o v erparameterized mo del in a first step) and its lo cal en vironmen t. This is usually not difficult to ach iev e departing fro m the v ertex represen tation. P artial orders with k = 4. T a bles 1 and 2 sho w the v ertex represen tations of the p ossible partial orders inv olvin g four outcomes A, B, C, and D as sho wn in Figure 4. The o nline supplemen tal material sk etc hes a heuristic for determining non-redundan t parameterizations of the mixture co efficien ts in these and more complex cases. Application: Order Constrain t s on t he 2HTM As already noted, the 2HTM distinguishe s “detect” states and a “no detection” state. These are laten t states, and a state-respo nse mapping is required to link the states to the observ able resp onses. Kla uer and Kellen (2 0 11) and Br¨ oder et al. (2013) prop osed relativ ely unrestricted state-resp onse mapping in whic h it is p ossible, for example, that extreme confidence ratings woul d b e preferred in a “no detection” state a nd that low confiden ce ratings would b e preferred in a “detect” state. This has prompted criticisms to the effect that the mo dels deal with rating Order Constrain ts 10 scales in an arbitrary and p ost-ho c manner (e.g., Dub e, Rotello, & P azzaglia, 20 1 3, P azziaglia, Dub e, & Rotello, 2013; see also Batc helder & Alexander, 2013). Acc ording to these critics, the mo del is ov erly complex (Dub e, Rotello, & Heit, 2011). Using the presen t results, it is p ossible to define a n order-constrained 2HTM for confidence ratings, 2HTM r , that maps the “no detection” state so that cautious lo w confidence ratings receiv e most probability mass and more extreme confidence ratings successiv ely less mass. Conv ers ely , for detect states, the preference for confidence lev els increases from low confidence to high confidence lev els in 2HTM r . Sp ecifically , 2HTM r imp oses the constrain ts that s h ≥ s m ≥ s l for the mapping from “detect” states to responses, and that o l ≥ o m ≥ o h and n l ≥ n m ≥ n o for the mappin g of the “no detection” state to r esp onses (see Fig ure 1 for the 2HTM). These constrain ts remov e most of the less plausible state-resp onse mappings that are admissible under the original 2HTM for confidence ratings. At the same time, they strongly curtail the mathematical flexibilit y of the mo del. Because these constraints can no w b e implemen ted within the MPT framew ork, w e can use standard MPT softw are to estimate, test, and analyze the mo del. F or example, w e used the R package MPTinR (Singmann & Kellen, 2013) to quan tify mo del flexibilit y of the 2HTM and the 2HTM r based on the minim um-description-length principle (Gr ¨ un w ald, 2007) that take s flexibilit y due to a mo del’s functional for m in to account, including constrain ts on flexibilit y due to inequalit y constrain ts. In the presen t context, one represen tation of the minim um-description-length principle is giv en b y the Fisher info rmation appro ximation (FIA) index (but see Hec k, Moshagen, & Erdfelder, 2014). FIA is a mo del selection index and describ es mo del flexibilit y by a p enalt y t hat is added to a lik eliho o d measure of the mo del’s (mis)fit. The mo del with the smallest index v alue is preferred. W u, Myung, a nd Batc helder (2010a, 2010b) dev elop ed metho ds to compute FIA for binary MPT mo dels that is implemen ted in MPTinR. The p enalty term for mo del flexibilit y in FIA comprises t w o additiv e terms. Order Constrain ts 11 One term dep ends up on the n um b er of parameters and the sample size similar to the p enalt y term in BIC. A second term quan tifies mo del flexibilit y due to functional form, and 2HTM a nd 2HTM r differ in the size of t his p enalt y term. Using t he ab ov e reparameterization and MPTinR, w e find the p enalties due to flexibilit y to b e -0.01 and -5.22 for 2HTM and 2HTM r , resp ectiv ely . Bec ause FIA and the normalized maxim um-lik eliho o d index o p erate on a log-likel iho o d scale, this means that the loss in go o dness of fit in terms of G 2 (Batc helder & Riefer, 1999) m ust b e larger than 10.42 = 2[ − 0 . 01 − ( − 5 . 22) ] b efore the parsimonious 2HTM r should b e abandoned in fa v or of the non-constrained 2HTM. This demonstrates that the constrain ts on flexibilit y imp osed b y the order constrain ts are substan tial. F or example, reanalyzing data from Koen and Y onelinas (201 0, pure condition), the 2HTM without order constrain ts ach iev es a G 2 statistic of 3 .3 5 and a FIA index of 8 . 2 8 (up to a n additiv e constan t). The 2HTM with order constrain ts ac hiev es a G 2 statistic of 9 .9 5 and a F IA index of 6 . 37. Th us, the loss in go o dness of fit is o utw eighed b y t he parsimon y of the constrained mo del, and the 2HTM r should b e preferred. Discussion In this note, w e extended Knapp and Batc helder’s (200 4) approac h to order constrain ts in MPT mo dels to the case o f order restrictions on non-indep enden t parameters constrained to sum to one. These restrictions frequen tly arise in cases where resp onse outcomes themselv es are ordered in some sense suc h as in confidence-rating data, Lik ert scale data (B¨ oc k enholt, 2012; Klauer & Kellen, 20 11) or where graded guessing tendencies or r esp onse biases are created via base-rate o r pa y off manipulations. Alternativ ely , the restrictions can directly arise from theoretical predictions (e.g., Rag ni, Singmann, & Steinlein, 2014). The results are useful b ecause they mak e a v ailable the growing to olb ox for statistical analyses of MPT mo del fo r the analysis of order-constrained MPT mo dels. As already exemplified, the to olb o x comprises alg o rithms and softw are for the Order Constrain ts 12 computation of FIA (Moshagen, 2010; Singmann & Kellen, 20 13; W u et al., 2 010a, 2010b), but a lso a n um b er of soft ware to ols to estimate and fit the mo dels (see Klauer, Stahl, & V oss, 2012 for a review), algorithms f or Ba y esian hierarc hical mo del extensions that capitalize on the mo del structure (Klauer, 2 0 10; Matzk e, Dolan, Batc helder, & W a g enmak ers, in press; Smith & Batc helder, 2010), algorithms and soft w are for hierarc hical laten t-class extensions of MPT mo dels in a classical inferen tial framew ork (Klauer, 2006, Stahl & Klauer, 2007 ), and algorithms for computing Ba ye s factors b et w een comp eting MPT mo dels (V andek erc khov e, Matzk e, & W agenmak ers, in press). The presen t results also apply to the imp ortan t case of order constrain ts o n the probabilities o f an observ able m ultinomial or pro duct-m ultinomial distribution, a case with many o ccurrences within and outside psyc hology — for example, in the analysis of contingen cy tables with order constrain ts (Agresti & Coull, 2002). The case is treated by Rob ertson, W right, and Dykstra (1988, chap . 5) via their elegan t isotonic regression metho d. Note, how ev er, that express ing o rder structures on observ able category probabilities in the MPT framew ork has the adv an tage of making a v ailable the ab ov e-men tioned to ols for estimation a nd inference in classical and Ba y esian framew orks. As one example, using MPTinR (Singmann & Kellen, 2013), the distribution of go o dness-of-fit statistic G 2 can b e assessed via b o otstrap metho ds under the nu ll hy p othesis that t he constrain ts are truly in fo rce, side-stepping the n umerically difficult task to ev a luate its asymptotic so-called ¯ χ 2 distribution. On the t heoretical side, these results con tribute to the study of the class o f MPT mo dels. They sho w that the class is closed under this further set of order constrain ts. This flexibilit y is surprising giv en that most o t her classes of statistical mo dels that w e are a w are of w ould not b e in v arian t under transformations as in Equation 1, nor capable of expressing distributions with and without the presen t kind of order constraints within the same class of mo dels. The abilit y of the mo del class to encompass the presen t and other kinds of meaningful order constrain ts adds to its Order Constrain ts 13 usefulnes s. Order Constrain ts 14 References Agresti, A., & Coull, B. A. (2002). The analysis of con tingency tables under inequalit y constrain ts. Journal of Statistic al Planning and In fer enc e , 107 , 45–73. doi: 10.1016/S03 7 8-3758(02 ) 0 0243-4 Baldi, P ., & Ba t cheld er, W. H. (2003). Bounds on v ariances of estimators for m ultinomial pro cess ing tree mo dels. Journal of Mathematic al Psycholo gy , 47 , 467–470. doi: 10.1016/ S0022-2496 ( 0 3)00036-1 Batc helder, W. H., & Alexander, G. E. (20 1 3). Discrete-state mo dels: Commen t on Pazzaglia, Dub e, and Rotello (2013). Psycholo gic al B ul le tin , 139 , 1204–12 1 2. doi: 10.1037/a00 33894 Batc helder, W. H., & Riefer, D. M. (1 9 99). Theoretical and empirical review of m ultinomial pro cess ing tree mo deling. Psychonomic Bul letin & R eview , 6 , 57-86. doi: 10.3758/ BF 03210812 B¨ oc ke nholt, U. (2012). Mo deling mu ltiple resp onse pro cesses in judgmen t and c hoice. Psycholo gic al Metho ds , 17 , 665–678. doi: 10.1037/ a0028111 Br¨ oder, A., Kellen, D., Sc h ¨ utz, J., & Ro hrmeier, C. (2013). V alidating a t w o-high- threshold measure men t mo del for confide nce ra ting data in recognition. Memory , 21 , 916– 9 44. doi: 10.1080/09658 211.2013.767 3 48 Dub e, C., Rotello, C. M., & Heit, E. (201 1). The b elief bias effect is aptly named: A reply to Kla uer and Kellen (2011). Psycho lo gic al R eview , 118 , 155- 163. doi: 10.1037/a00 1 9634 Dub ´ e, C., Rotello, C. M., & P azzaglia, A. M. (2013 ). The statistical accuracy and theoretical status of discrete-state MPT mo dels: Reply to Batche lder and Alexander (2013). Psycholo gic al B ul letin , 139 , 1213–1 2 20. doi: 10.1037/a00 3 4453 Erdfelder, E., Auer, T.-S., Hilbig, B. E., Assfalg, A., Moshagen, M., & Nadarevic, L. (2009). Multinomial pro cessing tree mo dels: A review of the literature. Journal of Psycholo gy , 217 , 108- 124. doi: 10.1027/00 44-3409.21 7.3.108 Order Constrain ts 15 Gr ¨ un w ald, P . (2 0 07). The minim um description length princip le . Cambridge, Mass.: MIT Press. Hec k, D . W., Moshagen, M., & Erdfelder, E. (2014). Mo del selection b y minim um description length: Low er-bo und sample sizes for the Fisher informatio n appro ximation. Journal of Mathematic al Psycholo gy , 60 , 29–34. doi: 10.1016/j.jmp.2014.0 6.002 Hu, X., & Batc helder, W. H. (1994 ). The statistical analysis of general pro cessing tree mo dels with the EM algo rithm. Psychom etrika , 5 9 , 21-47. doi: 10.1007/BF02 294263 Klauer, K. C. ( 2 006). Hierarc hical multinomial pro cessing tree mo dels: A latent-class approac h. Psychom etrika , 71 , 7-31. doi: 10.1007/ s113 36-004-1 188-3 Klauer, K. C. ( 2 010). Hierarc hical multinomial pro cessing tree mo dels: A latent-trait approac h. Psychom etrika , 75 , 70-98. doi: 10.1007/s113 3 6-009-9 1 41-0 Klauer, K. C., & Kellen, D. ( 2 010). T ow ard a complete decision mo del of item and source recognition: A discrete-state approac h. Psycho nomic Bul letin & R eview , 17 , 465-478. doi: 10.3758/PBR .17.4.465 Klauer, K. C., & Kellen, D. ( 2 011). Assess ing the b elief bias effect with R OCs: Reply to Dub e, Rotello, and Heit (2010). Psycholo gic al R eview , 1 18 , 164-173. doi: 10.1037/a00 2 0698 Klauer, K. C., Sta hl, C., & V oss, A. (2012). Multinomial mo dels and diffusion mo dels. In K. C. Klauer, A. V oss, & C. Stahl (Eds.), Co gnitive metho ds in so cial psycholo gy. Abridge d e dition (p. 331-354). New Y ork: Guilford. Knapp, B. R., & Batche lder, W. H. (200 4). Represen ting parametric order constrain ts in m ulti-trial applications of multinomi al pro cessing tree mo dels. Journal of Mathematic al Psycholo gy , 48 , 215–229. doi: 10.1016/ j.jmp.2004.03.002 K o en, J. D., & Y onelinas, A. P . (2 0 10). Memory v ariabilit y is due to the contribution of recollection and fa miliarit y , not to enco ding v ariabilit y . Journal of Exp erimental Psycholo gy. L e arning, Memory and Co gnition , 36 , 15 36-1542. doi: Order Constrain ts 16 10.1037/a00 2 0448 Matzk e, D ., Dolan, C. V., Batc helder, W. H., & W a g enmak ers, E.-J. (in press). Ba y esian estimation of m ultinomial pro cessin g tree mo dels with heterogeneit y in participan ts and items. Psychometrika . Moshagen, M. (2010). m ultiT ree: A computer progr a m for the analysis of multinomi al pro cessing tree mo dels. Behavior R ese ar c h Metho ds , 4 2 , 42–54. doi: 10.3758/BRM.42.1.42 P azzaglia, A. M., Dub e, C., & Rot ello, C. M. (2013) . A critical comparison of discrete-state and contin uous mo dels of recognition memory: Implications for recognition and b ey ond. Psycholo gic al Bul letin , 139 . doi: 10.1037 /a0033044 Purdy , B. P ., & Batc helder, W. H. (2009 ). A con text-free language for binary m ultinomial pro cess ing tree mo dels. Journal of Mathematic al Psycholo gy , 53 , 547–561. doi: 10.1016/ j.jmp.2009.07.009 Ragni, M., Singmann, H., & Steinlein, E.-M. (2014). Theory comparison for generalized quan tifiers. In P . Bello, M. Guarini, M. McShane, & B. Scassellati (Eds.), Pr o c e e dings o f the 36th annual c onfer enc e of the c o g nitive scienc e so ciety (pp. 1330–1 335). Aus tin, TX: Cognitive Science So ciet y. Rao, C. R. (1 973). Line ar statistic al i nfer enc e and its applic ations . New Y o rk: Wiley. doi: 10.1002/978 0470316436 Rob ertson, T., W right, F. T., & Dykstra, R. L. (19 8 8). Or der r estricte d statistic al infer enc e . New Y ork: Wiley. Singmann, H., & Kellen, D. (201 3). MPTinR: Analysis of multi nomial pro cessing tree mo dels with R. B ehavior R ese ar c h Metho ds , 45 , 560- 575. doi: 10.3758/s13428 - 012-025 9 -0 Smith, J. B., & Batc helder, W. H. (2010). Beta-MPT: Multinomial pro cessin g tree mo dels for addressing individual differences. Journal o f Mathematic al Psycholo gy , 54 , 167-183. doi: 10 .1016/j.jmp.2009.06.0 0 7 Sno dgrass, J. G., & Corwin, J. (1988). Pragmatics of measuring recognition memory: Order Constrain ts 17 Applications to demen tia and amnesia. Journal of Exp erimental Psycholo gy: Gener al , 117 , 34-50. doi: 10.103 7 /0096-344 5.117.1.34 Stahl, C., & K lauer, K. C. (2 007). HMMTree: A computer program for laten t- class hierarc hical multinomial pro cessing tree mo dels. Behavior R ese ar ch Metho ds , 39 , 267-273. doi: 10.3758/BF03 193157 V andek erc kho v e, J., Matzk e, D., & W a genmak ers, E.-J. (in press). Mo del comparison and the principle of parsimon y . In J. Busemey er, J. T own send, Z. J. W ang, & A. Eidels (Eds.), O xfor d ha ndb o ok of c omputational and mathematic al psycholo gy. Oxford: Oxford Univ ersit y Press. W u, H., Myung, J. I., & Ba tcheld er, W. H. (2010a ). Minim um description length mo del selection o f m ultinomial pro cessing tree mo dels. Psychonomic Bul letin & R eview , 17 , 275-286. doi: 10.375 8 /PBR.1 7 .3.275 W u, H., Myung, J. I., & Ba tcheld er, W. H. (2010b). On the minim um description length complexit y of m ultinomial pro cessing tree mo dels. Journal of Mathematic al Psycholo gy , 54 , 291 - 30 3 . doi: DOI:10.1016 /j.jmp.2010.02.001 Order Constrain ts 18 App endix W e sho w that eac h probabilit y distribution on three outcomes with η 1 ≥ η 2 and η 1 ≥ η 3 and P 3 i =1 η i = 1 can b e represen ted as a mixture with indep enden t parameters θ 1 , θ 2 , 0 ≤ θ i ≤ 1 as follo ws:        η 1 η 2 η 3        = (1 − θ 1 )(1 − θ 2 )        1 0 0        + θ 1 (1 − θ 2 )        1 / 2 1 / 2 0        +(1 − θ 1 ) θ 2        1 / 2 0 1 / 2        + θ 1 θ 2        1 / 3 1 / 3 1 / 3        . The pro of proceeds b y sho wing that this equation can b e solv ed for θ 1 and θ 2 giv en η 1 , η 2 , and η 3 . Note that η 1 − η 2 − η 3 = (1 − θ 1 )(1 − θ 2 ) − 1 3 θ 1 θ 2 . Setting ∆ = η 3 − η 2 , it follo ws that ∆ = 1 2 ( θ 2 − θ 1 ), hence θ 2 = 2∆ + θ 1 . Substituting this in the previous equation yields: 2 η 1 − 1 = η 1 − η 2 − η 3 = 1 3 (3 − 6∆ − 2 θ 1 (3 − 2∆) + 2 θ 2 1 ). Some manipulations sho w that this is equiv alent to [ θ 1 − 1 2 (3 − 2∆)] 2 = [ 1 2 (3 − 2∆)] 2 + 3( η 1 + ∆ − 1). F or this quadratic equation to b e solv able in θ 1 , it needs to b e shown that [ 1 2 (3 − 2∆)] 2 + 3( η 1 + ∆ − 1) ≥ 0 . This is equiv alen t to [ 1 2 + (1 − ∆)] 2 ≥ 3(1 − η 1 − ∆). Because η 1 + η 2 + η 3 = 1, this is equiv alen t to [ 1 2 + η 1 + 2 η 2 ] 2 ≥ 6 η 2 . Because η 1 ≥ η 2 , this is true if [ 1 2 + 3 η 2 ] 2 ≥ 6 η 2 . This is equiv alen t to (3 η 2 − 1 2 ) 2 ≥ 0. Hence, one solution of the ab o v e equation is θ 1 = 1 2 (3 − 2∆) − q [ 1 2 (3 − 2∆)] 2 − 6 η 2 . W e will sho w that 0 ≤ θ 1 ≤ 1. θ 1 ≥ 0 is to see noting that 1 2 (3 − 2∆) = 1 2 + η 1 + 2 η 2 ≥ 0. F urthermore, b ecause of this, θ 1 ≤ 1 is equiv alen t to 1 2 + η 1 + 2 η 2 − q ( 1 2 + η 1 + 2 η 2 ) 2 − 6 η 2 ≤ 1 or to η 1 + 2 η 2 − 1 2 ≤ q ( 1 2 + η 1 + 2 η 2 ) 2 − 6 η 2 . This is trivially true if the term to t he left, η 1 + 2 η 2 − 1 2 , is smaller than zero. If it is non- negative, on the other hand, this is equiv alen t to ( η 1 + 2 η 2 − 1 2 ) 2 ≤ ( 1 2 + η 1 + 2 η 2 ) 2 − 6 η 2 , whic h is equiv alent to η 1 ≥ η 2 as simple manipulations show. Because the equations are symmetrical in θ 1 and θ 2 , in terc hanging the roles of θ 1 and θ 2 and those of η 2 and η 3 sho ws that θ 2 = θ 1 + 2∆ also ranges b et w een 0 and 1. This completes the pro of. Order Constrain ts 19 F o otnotes 1 There are w ell-do cumen ted resp onse-st yle effects suc h as preferring mo derate o v er extreme resp onses or vice ve rsa as mo derated b y con textual and p ersonalit y factors (B¨ oc ke nholt, 2012 ). In the light of these effects, it is reasonable to assume that “detect” states are not necessarily alwa ys mapp ed on the highest confidence lev el. F or reasons of parsimon y and mo del iden tifiabilit y , w e assume in the presen t case that the state-respo nse mapping o f confidence ratings f o r “detect old” and “detect new” states is the same (but see Klauer & Kellen, 2010 ) . 2 The non-constrained MPT mo dels can b e transformed into equiv alent binary MPT mo dels in a second step whic h we do not describ e, b ecause it is w ell kno wn (Hu & Batc helder, 19 94). F urthermore, giv en the maxim um lik eliho o d estimates of the parameters of the binary MPT mo del and an estimate of its F isher information, maxim um like liho o d estimates of the parameters of the non- constrained MPT mo del as w ell as of the parameters of the original order-constrained MPT mo del and of their Fisher information matrices ( f or confidence interv als) can b e obtained via standard metho ds (Rao, 19 73, c hap. 6a) using the first deriv ativ es of the r esp ectiv e par a meter transformation functions that transform these mo dels’ parameters in to eac h other although there is as of yet no user-friendly softw are to accomplish t his. Order Constrain ts 20 T able 1 V ertic es fo r the Patterns in Figur e 4 V ertices P attern Category 1 2 3 4 5 6 7 8 I A 1 1/2 1/3 1/4 B 0 1/2 1/ 3 1/4 C 0 0 1/3 1 /4 D 0 0 0 1/4 I I A 1 0 0 1/ 4 B 0 1 0 1/ 4 C 0 0 1 1/4 D 0 0 0 1/4 I I I A 1 0 1/3 1/4 B 0 1 1/3 1/4 C 0 0 1/3 1 /4 D 0 0 0 1/4 IV A 1 0 1/ 2 1/4 B 0 1 1/2 1/4 C 0 0 0 1/4 D 0 0 0 1/4 V A 1 1/ 2 1/2 1/3 1/4 B 0 1/2 0 1/3 1/4 C 0 0 1/2 1 /3 1/4 D 0 0 0 0 1/4 VI A 1 1/2 1/3 1/ 3 1/4 B 0 1/2 1/ 3 1/3 1/ 4 C 0 0 1/3 0 1/4 D 0 0 0 1/3 1 /4 VI I A 1 0 1/3 1/3 1/4 B 0 1 1/3 1/3 1/4 C 0 0 1/3 0 1/4 D 0 0 0 1/3 1 /4 VI I I A 1 0 1/3 0 1/4 B 0 1 1/3 1/2 1/4 C 0 0 1/3 0 1/4 D 0 0 0 1/2 1 /4 IX A 1 1/2 1/2 1/ 3 1/3 1/4 B 0 1/2 0 1/3 1/3 1/4 C 0 0 1/2 1 /3 0 1 / 4 D 0 0 0 0 1/3 1 /4 X A 1 1/ 2 1/2 1/2 1/3 1/3 1/3 1 / 4 B 0 1/2 0 0 1/3 1/3 0 1/4 C 0 0 1/2 0 1/3 0 1/ 3 1/4 D 0 0 0 1/2 0 1/3 1/3 1/4 Order Constrain ts 21 T able 2 Non-R e d undant Par ameterizations of Mixtur e W eights for the Patterns in Figur e 4 P atterns V er- tices V VI VI I I IX X 1 (1 − θ 1 )(1 − θ 2 )(1 − θ 3 ) θ 1 θ 1 (1 − θ 1 )(1 − θ 2 )(1 − θ 3 ) (1 − θ 1 )(1 − θ 2 )(1 − θ 3 ) 2 θ 1 (1 − θ 2 )(1 − θ 3 ) (1 − θ 1 )(1 − θ 2 )(1 − θ 3 ) (1 − θ 1 )(1 − θ 2 )(1 − θ 3 ) θ 1 (1 − θ 2 )(1 − θ 3 ) θ 1 (1 − θ 2 )(1 − θ 3 ) 3 (1 − θ 1 ) θ 2 (1 − θ 3 ) (1 − θ 1 ) θ 2 (1 − θ 3 ) (1 − θ 1 ) θ 2 (1 − θ 3 ) (1 − θ 1 ) θ 2 (1 − θ 3 ) (1 − θ 1 ) θ 2 (1 − θ 3 ) 4 θ 1 θ 2 (1 − θ 3 ) (1 − θ 1 )(1 − θ 2 ) θ 3 (1 − θ 1 )(1 − θ 2 ) θ 3 θ 1 θ 2 (1 − θ 3 ) (1 − θ 1 )(1 − θ 2 ) θ 3 5 θ 3 (1 − θ 1 ) θ 2 θ 3 (1 − θ 1 ) θ 2 θ 3 (1 − θ 2 ) θ 3 θ 1 θ 2 (1 − θ 3 ) 6 θ 2 θ 3 θ 1 (1 − θ 2 ) θ 3 7 (1 − θ 1 ) θ 2 θ 3 8 θ 1 θ 2 θ 3 Note. P atterns I to IV emplo y f our mixture co efficien ts whic h are trivial to parameterize with three non-redundan t parameters θ i with 0 ≤ θ i ≤ 1, i = 1 , . . . , 3. W e b eliev e that no parameterization exists for patt ern VI I that exhaustiv ely represen ts a ll probabilit y distributions with the o rder constrain ts using three non-redundan t parameters, but did not find a pro of for the non- existence of suc h a parameterization. W e used n umerical metho ds to ascertain that the parameterizations sho wn exhaust the space of probabilit y distributions with the appropriate order constrain ts for all practical purp oses (see online supplemen tal materials). T wo−High−Threshold Model (2HTM) Old Item no detection D o g ( 1 − g ) ( 1 − D o ) s h s m s l detect old old high conf. old medium conf. old low conf . o h o m o l guess old old high conf. old medium conf. old low conf . n h n m n l guess new new high conf. new medium conf. new low conf . Ne w Item D n g ( 1 − g ) ( 1 − D n ) s h s m s l new high conf. new medium conf. new low conf . detect new o h o m o l old high conf. old medium conf. old low conf . guess old n h n m n l new high conf. new medium conf. new low conf . guess new no detection Figur e 1. The 2HTM mo del for confidence ratings. Response categories are sho wn in the middle; the tree to the left mo dels pro cessing for old items; the tree to the right for new items. Figur e 2. The tree to the righ t is a statistically equiv alen t reparameterization of the tree to the left for ordered parameters η 1 ≥ η 2 ≥ . . . ≥ η k − 1 ≥ η k . Figur e 3. The tree to the righ t is a statistically equiv alen t reparameterization of the tree to the left, if P i η i = P 3 i =1 θ i = P 6 i =4 θ i = 1. Figur e 4. Poss ible or ders with four categories A, B, C, and D. Relations implied b y transitivit y are not sho wn.