A Conversation with Nancy Flournoy

Statistic al Scienc e 2015, V ol. 30, No. 1, 133– 146 DOI: 10.1214 /14-STS495 c  Institute of Mathematical Statisti cs , 2015 A Conversation with Nancy Flourno y William F. Rosenb erge r Abstr act. Nancy Flourno y was b orn in Long Beac h, California, on Ma y 4, 1947. After graduating fr om P olytec hnic Sc ho ol in Pasadena in 1965, sh e earned a B.S. (1969) and M.S. (1971) in biostatistics from UCLA. Bet we en her bac h elors and masters d egrees, sh e work ed as a Statistician I for Re gional Medical Programs at UCLA. Afte r receiving her m aster’s degree, she sp end three ye ars at th e Southw est Lab oratory for Education Researc h and Dev elopmen t in Seal Beac h, C alifornia. Flourno y joined th e Seattle team pioneering b one marro w tr ansplan- tation in 1973. She mov ed with the transplant team into the newly formed F red Hutc hinson Cancer Researc h Cen ter in 1975 as Director of Clinical Statistics, where she sup ervised a group r esp onsible for the design and analysis of ab out 80 sim ultaneous clinical trials. T o sup p ort the Clinical Division, she sup ervised the deve lopment of an interdisci- plinary shared data soft ware system. She recru ited Leonard B. Hearne to create this database management system in 1975 (and married him in 1978). While at the Cancer C en ter, she was also at the Universit y of W ashington, where she r eceiv ed her do ctorate in biomathematics in 1982. She b ecame the fir st female director of the program in statistics at the National Science F oundation (NSF) in 1986. She receiv ed service a wards from the NSF in 1988 and the National In s titute of Statisti- cal Science in 200 6 for facilitati ng interdisciplinary researc h. Flourn o y joined the Departmen t of Ma thematics and Statist ics at American Uni- v ersit y in 1988 . She mo v ed as d epartmen t c hair to the Un iversit y of Missouri in 2002, where she b ecame Cu rators’ Distinguish ed Professor in 2012 . While at the Can cer Cen ter, Flourno y do cumented the graft-v ersus- leuk emia effect in humans and disco v ered a source of f r equen t lethal viral infections in the b one m arr o w transplan t patien ts. Later she w as influentia l in dev eloping adaptiv e exp erimen tal designs. Her n umerous honors include f ello w of the In stitute of Mathematical Stati stics (1990), the American Statistical Asso ciation (1992), the W orld Academ y of Arts and Sciences (199 2) and the American Academ y for the Adv ance- men t of Science (1993). Sh e h as receiv ed the COPSS Scott (2000) and Da vid (2007) a w ards, an d the Norwoo d (2012) a ward from the Univer- sit y of Alabama. Key wor ds and phr ases: Adaptiv e designs, clinical trials, data co ordi- nating cen ter, random w alk rules, up-and -do wn pro cedures. Wil liam F. R osenb er ger is Un iversity Pr ofessor and Chairman, Dep artment of Statist ics, Ge or ge Mason University, 4400 Un iversity D rive MS 4A7, F airfax, Vir ginia 22030-44 44, U SA e-mail: wr osenb e@gmu.e du . This is an electro nic r eprint o f the orig inal article published by the Institute of Mathematical Statistics in Statistic al Scienc e , 2 015, V ol. 30, No . 1, 133 –146 . This reprint differs from the origina l in pagination and t yp ogr aphic detail. 1 2 W. F. ROSENBER GER Fig. 1. Nancy at home in a p art of L os Angeles County that was then c al le d Poter o Heights , 1949. 1. EAR L Y LIF E Rosen b erger: T ell us a lit tle ab out y our early life. Where did y ou gro w up and what did your parent s do? Flourno y: I was b orn in Long Beac h, CA, and grew up in Los An geles Coun t y in a lemon orchard su r- rounded b y oil w ells and a flo o d plain. There w as a dairy farm nearby and w e had a donkey . My father w as a plumbing con tractor who p lu m b ed Los An- geles: restaurants, dormitories, cemeteries. He had 11 trucks go out eve ry day . My Mom w as alwa ys unhappy ab out not finish ing coll ege, so she enrolled in college when I we nt to colleg e and then d irected a presc ho ol for man y yea rs. I h av e three b rothers and one sister. I wa s s en t to Po lytec hnic Sc ho ol in P asadena as a sophomore in high sc ho ol. On the en- trance exam I had the second highest score in math in history , but I flu n k ed the English exam b ecause I didn’t kno w the words in the instructions. (So ev en then I had a on e-sided br ain!) Rosen b erger: As a young p erson, w ere you in ter- ested in mathematic s, statistics, data? What made y ou excited ab out statistics? Flourno y: High sc ho ol algebra really made m e happy; I wo uld la y on the fl o or and w ork problems for h ours. I had a new f emale ins tructor w hose hus- band had gotten a professorship across the street at Fig. 2. Nancy at her gr aduation fr om Polyte chnic Scho ol, Pasadena, 1965. Cal T ec h wh ile she just landed a high sc h o ol job; her anger came through an d I got the message that mathematics is w orth b eing passionate ab out. My lov e of statistics came as a junior at UCLA, when I to ok a course taught b y Don Ylvisak er. I ju st assumed that Don was a great teac her for all time, but he later tol d me that h e nev er had another class lik e it. F our or five stud en ts from that class w en t on to get d o ctorates in statistics. Rosen b erger: Y ou were fortunate to b e at UCLA at a time when there w ere some of the great names in biostatistics: Ab delmonen Afifi, F rank Massey , Wil Dixon, Oliv e Du n n, Virginia Clark. What professors excited y ou at UCL A? Flourno y: Afifi wa s the y oung dynamic professor and taugh t out of S c heff ´ e; all the studen ts lov ed Afifi. Dixon had a bimo dal distribution among the student s; y ou either lov ed h im or hated h im. He pu t out a thousand ideas a min ute; if y ou paid close at- ten tion, y ou would find they w ere p earls. It was a c h allenge to get what h e was sa ying as he didn ’t c h ange the tone of his v oice when he switc hed f rom one topic to another. He taught the p o we r of d ata A CONV ERSA TION WITH NANCY FLOU RNOY 3 analysis as a to ol for learnin g and a thousand lit- tle w a ys to mak e the data sing. I had a class with F rank Massey; I learned a lot, but he was quiet and not dynamic. Rosen b erger: Did y ou ha ve an y connection to the Departmen t of Statistics? Y o u men tioned Ylvisak er. What ab out P aul Ho el? Flourno y: A separate statistics departmen t did not exist at that time; it was a math department with a few statisticians. I used the Ho el, Port and Stone probabilit y b o ok w hen it was ju st a set of notes. I don’t think Ho el was the in structor though. Rosen b erger: What in terested y ou in biostatis- tics? Flourno y: Most of the statistics courses that w ere offered at UCLA were in the Departmen t of Bio- statistics. Prior to taking statistic s, I had lo v ed bio- c h emistry and w as a n utrition ma j or, leading to my ma jor in the Sc h o ol of P ublic Health (S P A). When I recog nized that with a degree in nutrition, I w ould probably only b e ab le to ru n a cafeteria in a hos- pital, I decided to get my degree in mathematics instead. I app lied rep eatedly to c hange from SP A to the College of Arts and Sciences (CAS), but m y ap- plication would get tu rned do wn. In tears, I didn’t kno w what to do. Th en th e SP A Dean asked why I w as flunkin g out, whic h d idn’t mak e sense since I alwa ys h ad gotten As and Bs. Th ey h ad lost all m y r ecords b ecause I h ad changed names when I w as previously married, and nothing follo w ed me. So that’s why they did n’t accept me at CAS. By the time th is got settled, I had enough credits to get a degree in biostatistics. 2. G RADUA TE S CHOOL Rosen b erger: What did you do after you gradu- ated? Ho w did you get to gradu ate sc h o ol? Flourno y: When I got a j ob at Regional Medi- cal Programs as a S tatistician I, one old man w ould come around and ask m e to add num b ers; I told h im he could hir e a statistical clerk for half my salary . I was told that, as a y oung w oman, m y pr esen ta- tions were not credible. So they hired a male DrPh to presen t my r ep orts in h is n ame. As a mild w a y of protesting, I put m y h air in a b un, dye d it white, and got fired. They said I was an “uppit y w oman.” A t that time, Virginia Clark wa s department chair. She said, “W e ha ve a fello ws hip, why d on’t y ou come to grad sc ho ol?” I ha ve some happ y memories of m y master’s pr ogram at UCLA: Oliv e Dunn su p ervised m y master’s thesis; Mary Ann Hill was a great teac h- ing assista nt for Dixon’s class; Carol Newto n taugh t a mean F OR TRAN p rogramming course; and Ray and Jean Mic k ey were influ en tial in my career d eci- sions. Fig. 3. Nancy with her p ar ents, Elizab eth Bli nc o e and Carr Irvine Flournoy, at her gr aduation f r om the University of Washington in 1982. 4 W. F. ROSENBER GER Rosen b erger: When yo u wo n th e David Award, y ou talk ed ab out meeting F. N. Da vid . T ell us ab out that. Flourno y: I w as in the Los An geles chapter of the ASA; around 1972, a group of us carp o oled out to UC Riverside where Da vid w as giving a talk. She had a strong p resence, standing with one leg up on a stairstep and smoking a cigar w hile she talk ed. It w as a ro omfu l of p eople, and she exuded such confidence. So I immediately started smoking cigars. I had b een used to seeing female statisticians such as Clark and Dunn b ehind a d esk and not commandin g an audience. Rosen b erger: Ho w did y ou get to Universit y of W ashington (UW)? Flourno y: After the M.S., I thought I knew ev ery- thing ab out statistics. I got a job at South w est Ed- ucation and Lab oratory f or Researc h , wh ere there w ere a lot of ed ucation psyc hologists who w ere int o exp erimenta l design. On m y second da y , th ey pre- sen ted me with computer output that h ad more than one error term; I had the go o d s ense to kee p my mouth shut. I immediately called Wil Dixon and ask ed what they were talking ab out. He rep lied, “Oh w ell, we can’t teac h you everything.” He suggested I get a b o ok by W alt F ederer. The b o ok w as out of print, but W alt got a prepr in t from In dia and sent it to m e; so I sp ent m y n ights reading F ederer’s b o ok. Later, I was tryin g to read the Journal of th e Americ an Statistic al A sso ciation to implemen t some of th e stuff I wan ted to do, and I found I couldn’t read the literature. I also wan ted to escap e th e smog of Los Angeles. So I applied to the UW, m y only application. Dic k Kronmal said there was a r esearc h assistan t p osition with the b one marro w transplan t team, whic h wa s then lo cated in the O ld Pub lic Hos- pital (recen tly Amazon) in Seattle . A t that time, there w as no statistics departmen t at UW. Th e mathematical statistics courses were taugh t in th e Departmen t of Mathematic s. I to ok the mathematica l statistics sequence from Galen Shorac k. I h ad cours es from Ron Pyk e and F ritz Sc hultz in nonparametrics. Sh ortly after F ritz left for Boeing, the remaining statistics facult y formed the Departmen t of Statistic s. In the Department of Biostatist ics, there were some female faculty: Paula Diehr and Pat W ahl. I to ok the first categorical data analysis class taugh t at UW from Norman Breslo w. He ga v e quizzes at the end of class, so I nev er paid so muc h atten tion in a course b efore. I to ok surviv al from Ross Pren tice early in the days of the Co x pro- p ortional hazards mo del. Rosen b erger: What wa s it lik e w orking with y our dissertation advisor, Llo yd Fisher? Flourno y: It work ed out w ell b ecause w e hav e sim- ilar w ork style s. Both of us had b usy consulting lives; w e would sc hedu le meetings and get our business done. 3. T HE S EA TTLE B ONE MARRO W TRANSPLANT A TION TEAM Rosen b erger: T o da y ev ery s treet corner seems to ha v e a con tract researc h organization for data co- ordinating cen ters on large clinical trials. But wh en y ou w en t to the F red Hutc hinson Cancer R esearch Cen ter, information tec hnology wa s primitiv e, suc h places did n ot exist. Y ou had to create that envi- ronment on y our o wn. What w as it lik e? Wh at w ere the c hallenges? Flourno y: That’s an interesting story . Dic k Kron- mal had inv ested a lot of effort in creating a database managemen t system without requiring a rectangular data stru cture. Up dates required ph ysically sorting the cards (remem b er all d ata records h ad to fi t int o the 80 d igits of a Hollerith punc h card—so I tend to use the w ord s “card” and “record” interc hangeably). There was a transp lan t data set in place with seven differen t kind s of cards. Kronmal h ad E. Donnal “Don” Thomas (Director of the Clinical Researc h Division at the C an cer Center) buy a computer. Th e computer weighed 50 p ounds (I could toss it in my v an and tak e it home; the cost was ab out $50,000 ), and data storage was on Phillips cassette tapes. Records could b e transmitted across th e phone wires and then in tegrated in to the database at UW. Ini- tially , there w as not muc h data (only 10 patient s) and the fir st up date took m y whole computing b u d- get for the y ear! What I did th en f or some p erio d of time w as, when it was time to d o an up d ate, pun ch cards of the wh ole database and the new dataset; I w ould physically sort and merge the cards by hand and load them in to SPSS . Th at wa s m y “dirt y laun- dry” story b ecause the laund r omat had big long ta- bles and I sorted cards while doing laundry . Kr onmal told me that, if I had an y trouble with m y new com- puter, I should call Leonard Hearne. Index sequen - tial files were brand new at that time, and Leonard used them to create an early database m anagemen t system b efore the wo rd was in the literature (see Flourno y and Hearne, 1981 , 19 90a , 199 0b ). W e used A CONV ERSA TION WITH NANCY FLOU RNOY 5 it for seve ral years unti l a commercial system came on the mark et. A t site v isits, someone would ask a question and I w ou ld p ass a note do w n to a p rogram- mer, who w ould extract the answer in 15 minutes or so. W e set the bar for oncology pr ograms. Ross Prenti ce came from the Unive rsity of W a- terlo o with a b o x of cards on th e Co x prop ortional hazards mo del; we w ere r eally early using that. Th e do ctors were smart enough to understand the limi- tations in using discriminan t analysis and they w ere thrilled to b e able to incorp orate censored surviv al data in their r egression m o dels. My work d o cument - ing graft vs. leuke mia in humans was v ery imp or- tan t (see W eiden et al., 1979 , 1981a , 1981b , 198 1c ). One hypothesis motiv ating the d ev elopmen t of b on e marro w transplantat ion was that the marro w graft w ould atta c k residu al leuke mia also. Immunologica l activit y of the graft wa s app aren t when the graft instigated an imm unological attac k on th e patien t. I mo d eled the imp act of this attac k on the r elapse rate. The protection of th e graft attac k against re- lapse greatly complicated p ost-transplan t treatmen t strategies. Bu t our findings hav e withsto o d the test of time. It w as, p erhaps, th e first ma jor applica- tion of the prop ortional hazards mo del with time- dep end en t cov ariates. Rosen b erger: When y ou think of the success of th e b one marro w tran s plan t p rogram (Don Thomas won the Nob el prize in 199 0 for dev eloping b one marro w transplan tation as a treatmen t for leuk emia), ho w m uc h did statistics and data managemen t pla y a role in that? Do y ou think statistic s and data man- agemen t will ev er get its due? Flourno y: W e had this ru diment ary set of records that could b e added onto infinitely . It started out that bacteriology wa nte d to add a card, then vi- rology , then sp ecific studies w ould add a card with their d ata. Before yo u knew it, we had an inter- disciplinary shared d atabase with assigned patien t n umb er s so all the inte grated d ata w as a v ailable. I was able to sa y “do you know what they’re do- ing in virology that’s r elated” b ecause I kn ew ev- eryb o d y’s data. It w asn’t un til man y y ears la ter that p eople started talki ng ab out ha vin g integ rated shared databases. Most w ere established for billing purp oses, not for researc h purp oses. They are d iffer- en t constructs. Hospitals would archiv e data after the b ill wa s paid but we wa nte d to ke ep it around forev er. When the program started, there w as one of ev- eryb o d y (one statistician, on e virologist, etc.), and w e w ould sit around the table and share results. It w as imp ortan t to b e influ en tial and to catc h prob- lems in data collection and qualit y con trol b efore they got big. When working with new d o ctors, there w ere h u mps y ou had to get o ver b ecause they wo uld claim that there were no qualit y con tr ol issues: their lab p eople nev er made a mistak e. A lot of n egotia- tion had to go on b efore w e could agree. Y es, we had a h uge influence. E v en randomization and blinding w as contro v ersial. If it was in th e m iddle of the night the cards might get s huffled; there w as to o m uc h ro om f or bias. W e in tro duced them to a v ery careful Fig. 4. Y ash Mittal, first female dir e ctor of the pr ob abil ity pr o gr am, and Nancy, first f em al e dir e ctor of the statistics pr o gr am, at NSF. 6 W. F. ROSENBER GER randomization regimen f or treatmen t assignmen ts, with a 24 hour on-call p erson. It will b e hard for statistics and data manage- men t to ev er get its full due b ecause the do ctors are so enamored of themselves (laughs). It’s really a strange system where the p eople with the least sci- ence bac kground usually run the science. Also, the data managemen t budget wa s alw ays the first to b e cut; y et it is v ery exp ensiv e to do a qualit y job. Rosen b erger: What h as y our r ole b een in fostering in terdisciplinary research? Flourno y: Ha vin g conducted in terdisciplinary re- searc h for more than a d ecade at the Cancer Cen ter, I knew th e p ow er that teams of inte rdisciplin ary researc h ers could br ing to b ear on imp ortan t sci- en tific questions. Coincidentally , when I w ent to the Nat ional Science F oundation (NSF) in 1986, the Division of Mathematical Sciences (DMS) had funded the Institute of Mathematic al Statistics (IMS) to write a rep ort on cr oss-d isciplinary r e- searc h . I w atc h ed the growth in their thinking as they in teracted with eac h other. A t th e time, th e discipline did not app reciate the role of applications in academic settings. I think I was able to influ - ence the IMS cross disciplinary committee on the v aluable nature of interdisciplinary work. The re- p ort of the committee had a dr amatic effect on the discipline. The rep ort prop osed establishing the Na- tional Institute of S tatistical Science. Since I w as at NSF, I wa s able to pr omote the idea of establishing a broad institute that w ould wo rk on p roblems of national imp ortance. A t the same time, I w ould receiv e prop osals f rom statisticia ns motiv ated b y applications. Be cause our budget wa s s m all, I to ok suc h p rop osals around to the relev ant disciplines th at were inv olv ed, an d was able to get some joint funding. This resulted in my getting an a w ard in 1988 f or facilitating the fund- ing of these in terdisciplinary pro jects. This also led to sp ecific DMS requests for prop osals for in terd is- ciplinary research pro jects, which are no w common throughout NSF. 4. A D APTIVE DESIGNS Rosen b erger: Ho w d id you get in terested in adap- tiv e designs? Flourno y: While at the Can cer Cen ter, the ma jor program pro ject grant had fiv e-y ear reviews. When w e prepared for the thir d one of these, w e sp ent a y ear reviewing what w e h ad done an d ho w w e w ould go forward. I n the cours e of that r etrosp ectiv e, I dev elop ed some feelings ab out th e t wo arm clinical trial. The standard ideas ab out th e t w o arm clini- cal trials came from the Peto p ap er in th e m id 70s (P eto et al., 1976 , 1977 ). But, in my exp erience, a treatmen t is a p oin t in a high dimensional space: in- v olving d rugs, r ad iation, including how muc h, ho w often; and on e learned little ab out this h igh d imen- sional space using the traditional tw o arm clinical trial. F or instance, w e sp ent fi v e yea rs comparing A to B; b ut then we go bac k to the high dimen- sional space and pic k out p oint C , and ha v e another fiv e y ears of exp erimen tation and compare A to C. Then we compare C to D, and after 15 y ears w e ha v e knowle dge of four p oints in a high dimens ional space. I b elieve d it would b e more efficien t and in- formativ e to kno w whic h direction w e should head in the high dimensional sp ace. So th at led me to think ab out adaptive designs. I recommended sev- eral to the group and the p h ysicians lik ed th e ideas, but though t they ma y b e to o radical to get funded. Another thing th at promoted m y interest was lo oking at pilot studies to d ecide what to tak e for- w ard to larger studies. Bob Tsutuk a wa was visiting the Cancer Cen ter from the Universit y of Missouri at the time. I thought h is Ba y esian ideas w ere ap- p ealing and I us ed exp ert opinion for p r ior elicita- tion (see Flourno y , 1993 ). Th e pr ior was wa y off, s o w e w ound u p with a lot of to xicities. Y ou just can’t trust the b est exp ert opinion of th e b est exp erts, and so there needed to b e some wa y to use in terim data faster to adapt and p ut m uc h less w eigh t on the prior. My later wo rk sho wed h o w random walk rules could b e constructed to do this (see Durh am and Flournoy , 1994 ; Durham, Flournoy and Rosen- b erger, 1997 ; Flourno y and Oron, 20 15 ). Rosen b erger: The first time I heard th e name Nancy Flourno y was in the con text of the 1989 ses- sion on adaptiv e d esigns at Join t Statistical Meet- ings (JSM) in W ashin gton. It turned ou t to b e one of the most cont rov ersial sessions in the history of JSM. T alk ab out that. Flourno y: My exp erience at NSF wa s that y ou don’t mak e progress w ith ou t communit y . On e p er- son alone do esn’t get muc h d on e. S o I had the idea that a JSM session on adaptiv e design wo uld bring together p eople who are in terested in adaptiv e d e- signs. I did n’t kno w a ny one per s onally . I in vited based on my impr essions of their int erests. I in- vited Don Berry , Ric hard Simon and Janis Hard- wic k. I ga ve a straightfo rward tec hnical talk on the A CONV ERSA TION WITH NANCY FLOU RNOY 7 Fig. 5. 1994 IMS Workshop on Se quential A nalysis at University of North Car olina, Chap el Hil l. Include d ar e Nancy Flournoy (se ate d se c ond fr om left), Lynne Bil lar d (imme diately b ehind Nancy), Janis Har dwi ck (to the right of Lynne), Bil l R osenb er ger (thir d r ow fr om b ack, mi dd l e), and Steve Durham (dir e ctly in fr ont of right wi ndow). topic. T h e r emaind er of the session fo cused primar- ily on criticism of the extracorp oreal mem brane oxy- genation (ECMO) trials (e.g., Barlett et al., 1985 ; O’Rourk e et al., 1989 ; W are, 1989 ). (The E CMO trial was an imp lemen tation of the rand omized p la y- the-winner r u le of W ei and Durham, 1978 , in whic h 11 babies were assigned to an exp erimenta l arm, and all survive d, while one baby assigned to the co nv en- tional arm, died. Th e historical death rate on the con v en tional arm w as 80 p ercent. ) T o m y dism a y , all the negativ e fo cus of the session wa s d irected to- w ard the adaptiv e design asp ect of the clinical trial, rather th an on the sample size and what kin d of sample size w ould b e needed for the trial to b e con- vincing. The press that w as generated b y this session set adaptiv e d esigns bac k a long time. Rosen b erger: Ho w m uc h do you th ink the failed ECMO trial inhib ited the dev elopmen t of adaptiv e designs? Flourno y: What would ha ve b een a reasonable ap- proac h? T h e original trial was unconvincing due to ha ving few patien ts, in spite of the fact that a p r ob- abilistically reasonable s topping r u le was applied. The contro ve rsy o v er the subs equen t tw o arm trial in clinical r esearc h set bac k adaptive designs wrongly . The adaptive trial was so successful that only one bab y died; is that bad? Rosen b erger: In y our 1992 AMS/IMS/SIAM con- ference on adaptiv e d esigns (Flourno y and R osen- b erger, 1995 ), you brought together some of the groundbr eak ers of ad ap tive d esigns alo ng with a n umb er of y ounger f acult y who are no w at the fore- fron t of the discipline. At the op ening session, you started b y talking ab out the need to streamline the pro cess of clinical trials, the end to ph ases and the incorp oration of dynamic in terim decisions. Y ou said that w ill revol utionize the wa y we do clinical tri- als, and that this c onference would b e an am bi- tious b eginning to that rev olution. No w , o v er t wo 8 W. F. ROSENBER GER decades later, there are 70-some sessions on adap- tiv e d esigns at the Joint Statistical Meet ings, “big- pharma” w orking groups, F o o d and Drug Adminis- tration white pap ers and guidelines, companies like ADDPLAN, and CYTEL dev oted to adaptiv e de- sign softw are and ev eryone wan ts to do adaptiv e de- signs. What took so long? Flourno y: E C MO made a steep hole to climb. W e also had to dev elop theory . It was one thing to sa y “this is a goo d idea,” and another to adequately supp ort it. S ome ideas w ere NOT go o d. Th is in- cludes a class of pro cedures that deriv e from sto chas- tic approxima tion, th at V al F edoro v coined “b est in ten tion” designs. In these designs, a target d ose is estimated (suc h as the dose ha ving a p articular p ercent toxi cit y or one that maximizes some utilit y function); then that estimate is the dose giv en to the next sub ject. Some, includin g Lai and Robbins ( 1982 ), un dersto o d early on that using this pro ce- dure without safeguards ma y resu lt in treatmen t se- quences that con v erge to the wr ong dose. But others, including myself (Li, Durham and Flourno y , 1995 ), w ere enamored of this idea a nd ignoran t of ear- lier w arnings. This approac h r emains p opular tod a y ev en as recen t pu blications are exp osing just ho w misleading it can b e (e.g., Azriel, 2012 , Oron and Hoff, 2013 ). In the 1980s, John Whitehead sp ent a y ear visiting the Cancer Center f rom the Universit y of Reading, and promoted the idea of using sequen tial stopping rules taking censoring in to accoun t. Its v alue w as so ob vious that I exp ected that by 1990 ev ery clinical trial w ould b e using these tec hniqu es. So I fo cused instead on adaptiv e allo cation. A t American Un i- v ersit y (A U), I w ork ed on theoretical problems in these areas. When I pulled my h ead out and lo ok ed around I wa s sho c k ed to see that stopping ru les in - corp orating censoring w ere not b eing used, exce pt a bit in cancer. So things that seem obvious to some can tak e a long time to en ter the m edical arena. T ak e the “3 + 3” dose escalation design as an exam- ple. It has b een sound ly discredited (Reiner, P ao- letti and O’Qu igley , 199 9 ; Lin and Sh ih, 2001 ), and y et remains a standard p ractice in oncolo gy phase I trials. Adaptiv e allocation is still in its infancy compared to sequential monitoring and stopping. No w there has deve lop ed a new b elief that simula tion is ad- equate for assessing an adaptiv e design. But rely- ing solely on sim ulation muddies the w ater b ecause there is n o glo bal view of what is driving the design. In addition, there are many pap ers in the literat ure that rep ort only av erages o ver simulatio ns without measures of v ariabilit y . When yo u consider measur es of v ariabilit y , a completely differen t picture emerges (Oron and Hoff, 2013 ). I ferv ently b eliev e in dev el- oping the theory un derlying classes of designs. F or- tunately , man y p eople are in terested in working on the theoretic al c hallenges, and there are a lot of in- teresting op en questions. Rosen b erger: M any times w h en I hear talks on adaptiv e designs I wan t to scream out “Nancy Flourno y though t of that in the 1980s.” Ho w do y ou feel ab out some of your early ideas b eing ignored? Flourno y: W ell, I’m hard ly alone in this. F or in- stance, Ch ris Jennison inv ent ed many cleve r tec h- niques for sequen tial and adaptiv e clinical trials v ery early that are sometimes “redisco v ered” with- out reference (e.g., J ennison, Johnstone and T ur n - bull, 1982 ; K ulk arni and Jenn ison, 1986 ; J ennison, 1987 ). I n my case, it amazes me that ther e are a large num b er of p eople wh o will reference a pap er from the 1980 s and ignore 30 years of m y r esearc h . F or example, the early u p and do wn pap er of Storer ( 1989 ) is often cited without reference to my later pap ers that h a ve m uc h more soph isticated cont rol of the adaptiv e pro cess. T his early pap er is used as a whippin g p ost to declare up and do wn p r o cedures inferior. An up and do wn d esign is a rand om w alk that can end an ywhere. The last state (dose) vis- ited should not b e u sed as an estimato r. But th is is done wh en the up and d o wn design is compared to other p r o cedures that deriv e from sto chastic ap- pro ximation (e.g., Zac ks, 2009 ). That b others me a lot. Rosen b erger: Ho w did y ou meet Steve Durh am? This b egan one of the great collab orations in statis- tics. T ell us ab out that. Flourno y: One of the few p ositiv e consequences of the 1989 J S M session was meeting S tev e Durham from the Univ ersity of S outh Carolina. When I w alk ed ou t the do or after the session, Steve intro- duced himself and w as v ery excited b ecause w e w ere basically working on the same mathematical prob- lems, his from an engineering motiv ation, and mine from a m edical motiv ation. W e b egan w orking to- gether righ t a w a y . He would come to W ashington, DC, to meet me, and I wen t to South Carolina. After a stint as Chair at A U, I w as on a sabbatical at the Univ ersit y of North C arolina C hap el Hill; Leonard and I b ought a house close to campus s o that we could host visitors. In p articular, Stev e Durham and A CONV ERSA TION WITH NANCY FLOU RNOY 9 I work ed together quite a lot in that house and at the Departmen t of Statistics. Several other collab- orators came do wn for extended p erio ds, includ ing y ou (W.F.R.) and tw o of my do ctoral students: Eloi Kpamagen (no w at No v a v ax) and Misrak Gezm u (no w at National Institutes of Healt h). Rosen b erger: The in tro duction of th e random w alk ru les coincided with the in tro duction of the con tin ual reassessmen t m etho d (CRM; O’Quigley , P ep e, and Fisher, 1990 ) in the Ba ye sian con text. In particular, y ou and S teve work ed out the en tire ex- act distribution theory of a class of designs, while others were relying on sim ulation. Ho w do es this rank in terms of y our cont ribu tions to statistics? Flourno y: Th e random w alk ru les are extremely practical and mathematically elegan t, so it w as a lot of fun to dev elop the theory . They are the stan- dard in many areas of science (e.g., American S o- ciet y f or T esting and Materials, 2010 ; T reutw ein, 1995 ; National Institute of Environmen tal Health Sciences, 2001 ). The k ey prop erty that we disco v ered w as how to control the allocation co v erage b y intro- ducing an appropriate bias (Durham and Flo ur n o y , 1994 ). S tev e w as alw a ys think in g in terms of engi- neering applications; I wa s alw ays thinking in the dose-resp onse con text. W e did “ reve rse engineer- ing,” in that we had a target al lo cation in min d , and w e found design parameters to facilitate this. Th e designs are nonp arametric in that allo cation do es not dep end on estimat es of mod el parameters. They are extraordinarily s imple to illustrate and ha v e ex- act distribution theory that is u n a v ailable for other, more complicat ed designs. Rosen b erger: Some hav e lu mp ed random w alk rules in the con text of generic dose escalation de- signs, suc h as the 3 + 3 d esign, that has n o op- timal prop erties. At the same time, Ba yesia n ap- proac hes, such as the CRM w ere b ecoming increas- ingly w ell-kno w n. T alk ab out the historic in terpla y among these approac h es. Flourno y: Lloyd Fisher and John O’Q u igley (fr om the Universit y of Leeds) were hired at the C an - cer Center to replace me when I left for the NSF. Llo yd and I laughed that it is not often one’s dis- sertation advisor replaces his student! John w as in i- tially resp on s ible for implementing a random wa lk rule that I had designed in a pilot stud y for a b one marro w clinical trial. He let them get a w a y with a simple dose escalat ion pro cedure, but he and Llo yd got introdu ced to the sub ject at th at time. Th ey immediately thought of doing a Bay esian alterna- tiv e, a nd it was p ublished in 1990 in Biometrics (O’Quigley , P ep e and Fisher, 1990 ); the ma jor ran- dom w alk pap er app eared in Biometrics in 1997 (Durham, Flournoy and Rosenber ger, 1997 ). Most of the Ba y esian literature w as, b y necessit y , sim ula- tion based, wh ereas Stev e an d I w ere busy obtaining a complete w ork able probabilistic th eory of the ran- dom w alk pro cedures. There are a n umb er of p hilosophical differences among th e approac hes. F edoro v w ould call the CRM a “b est in ten tion” approac h, b ecause it inv olve s pre- dicting a target dose and treating the next patien t at that dose, sequentiall y . Our approac h is estimatio n- motiv ated. Th e idea is to get allocations into a re- gion of in terest that allo ws us to efficien tly estimate the dose-resp onse curv e in that region. There is also a short-memory and long-memory distinction: allo cation probabilities for the ran - dom w alk rule con v erge exp onen tially fast to their asymptotic limits. Alternativ ely with b est inte nti on designs (which to d ate are long-memory designs), nonrepresentat iv e early allocations can cause the de- sign to con verge to the w rong dose (see, e.g. , Azriel, Mandel and Rinott, 2011 ; Oron, Azriel and Hoff, 2011 ; Azriel, 201 2 ). Su c h phenomena were observ ed early on in the conte xt of sto c hastic a pp r o xima- tion d esigns (e.g., Lai and Robbins, 1982 ; Bo zin and Zarrop, 1991 ). Adaptiv e optimal designs are promising long memory designs, but they dep end on parameter esti- mates to get started. Rand om w alk pro cedu r es that target optimal design p oin ts pro vide goo d start- up information w ith sm all sample sizes. Alterna- tiv ely , one can regularize the information matrix, a “fix” that is often called “Ba y esian d esigns” even though no p osterior distribution is ob tained. T ru e Ba yesia n estimator up dates coupled with dose al- lo cations made in some stable optimal wa y , rather than in a “best int enti on” wa y are also p romising. Rosen b erger: What is the future of adaptiv e de- signs? Do y ou think all clinical trials will even tually b e adaptiv e? Flourno y: I thin k there is a great future for adap- tiv e designs. I think exp erimen tation will alw a y s in- v olv e a series of trials; the question is ho w well one utilizes inform ation from one to th e next. Th ere is a lot of v alue in relativ ely s m all but sequen tial tr i- als (see Flourno y , 2014 ), b ecause these trials in volv e man y design features, in cluding th e grid s ize and range on w hic h yo u are op erating. The b est use of 10 W. F. ROSENBER GER one exp erimen t ma y b e to tell yo u ho w you could ha v e b etter selected design c h aracteristics; then y ou can refine the estimate of the target of interest. Some of my wo rk has b een on in ference and es- timation follo wing adaptiv e designs (e.g ., Rosen- b erger, Flournoy and Durham, 1997 ; Iv ano v a and Flourno y , 2001 ; Ma y an d Flournoy , 200 9 ; Lane, Y ao and Flourn o y , 2014 ). O ne has to b e careful doing ev erything s equ en tially b ecause some of the in terim c h anges ma y cause fin al estimates to lac k n ormal- it y . F or example, in b est-int ent ion designs, the esti- mate of a slop e parameter can m arc h off to infinit y for s ome common mo dels. Also, ev en if an adaptiv e dose-finding pro cedure h as a fixed tota l sample size, the sample sizes at eac h d ose are r andom v ariables. In u p-and-down pro cedur es, th e p rop ortion of sub- jects allo cated to eac h dose tends to a co nstant and standard asymptotic n ormalit y results. But in many other adaptiv e designs, prop ortions of sub j ects allo- cated to eac h dose tend to a random v ariable. This causes the conditional inf ormation m atrix to b e r an - dom, ev en in the limit, in which case standard cond i- tions for asymptotic normalit y fail. These are man y in teresting questions to b e explored ab ou t adaptiv e designs. 5. W OMEN IN ST A TISTICS Rosen b erger: T alk ab out the creation of P athw a ys to the F uture, its successes, and its leg acy . Flourno y: I wen t to the 1984 Annual IMS Meet- ing in Lake T aho e. A t that meeting, there w ere fiv e w omen out of ab out 200 attendees. It b ecame quite clear to me that this wa s an imp ortant place for ac a- demic statisticians to meet and fo cus on academic in terests. In antic ipation of the 1988 F ort Collins IMS meeting, whic h was separate fr om the Join t Statistical Meet ings, I decided it w ould b e great to see more w omen there. S o I b ounced id eas off Mary Ellen Bo c k (Purdu e Univ ersit y) and Lynn e Billard (Univ ersit y of Georgia). Lynne agreed to tak e the lead in organizing a wo rkshop for w omen at the u p- coming IMS meeting. Lyn ne had the b rillian t idea of having Elizabeth Scott (Univ ersit y of California, Berk eley) giv e the keynote lecture. A t that time, w e w ere debating whether there was an y gender in- equit y in academia, and we w eren’t sure. I had never exp erienced problems at UCLA or UW. How ev er, when I wen t to the NSF, Y ash Mittal (the first fe- male director of the pr obabilit y program) and I sa w that there w ere almost no female grantees, and v ery few w ere ev en applying for gran ts. The ev ening presenta tion by Scott r eally hit us v ery hard: sh e h ad tons of data and randomized studies on gender inequit y . Any questions ab out in - equities in h o w women w ere r ecruited, jud ged and v alued we re thrown out the wind o w. Scott’s wa y of handlin g this lecture w as wonderful b ecause sh e w en t through all this horribly depressing data, but she then turned aroun d and finished the lecture by telling us what we could do to pr otect our selv es. Sh e ended with tw o p ositiv e notes: that outcomes are not predetermined, and one can tak e one’s career in Fig. 6. Nancy, husb and L e onar d, and Lynne Bil lar d at their home in Chap el Hi l l, NC, 1994. A CONV ERSA TION WITH NANCY FLOU RNOY 11 Fig. 7. Nancy, I ngr am Olkin and Elizab eth Mar gosches (formerly with the Envir onmental Pr ote ction A gency) at the Cam- p anile at University of California, Berkeley, 2003. one’s o wn h ands. Lynne ran th e w orkshop for the next t wo decades, and she p resen ted Scott’s lecture with up dated data ev ery y ear. That lecture was the last lecture Scott gav e b efore sh e passed a w a y . I re- mem b er well that there w as a palpable sigh of relief from Scott—that she could tu r n o v er her cause to the next generatio n. Rosen b erger: Ho w did y ou b ecome NSF program director? What w as y our exp erience with gend er is- sues there? Flourno y: Ingram Olkin has long b een a great friend and men tor. He is the one who recommended me to the NSF for the program director p osition. I w as the first female director in the statistics program the same y ear that Y ash Mittal w as the first female probabilit y director. Some p eople had indicated to the division director their fear I wa s going to giv e all the gran t money to biostatisti cs. I convinced him that I could r epresen t the en tire statistics field. One d ay I remember answe ring the p hone and a professor on the line y elled “I said I w anted to sp eak to the director,” thin king a woman on the phone m ust b e a secretary . W e had a go o d tra v el bu d get and I wen t to as man y y oun g w omen’s lectures as I could. I w ould go up at the end of their talk and ask if they would b e in terested in applying for a grant . By the time I left NSF, the prop ortion of gran t prop osals fr om w omen w as p rop ortional to their presence in the field. A suggestion is su c h a small thing, and y et clea rly im- p ortant messages w eren’t b eing transm itted to fe- male facult y . Rosen b erger: W a s discrimination subtle or n ot so subtle when y our career was dev eloping? Flourno y: W ell, th ere w as alwa ys sexist b eha vior and man y things that were said and done are consid- ered inapprop r iate or even sexual misconduct to day . When I w ent on the j ob mark et for a fully academic p osition I found that many m en were incredulous. Some w ou ld make outrageous comment s dir ectly to me as if I were in visib le (or a man). Men in my o w n age category were often dismiss ive or oblivious to m y pr esence. Some of th e older generation was v ery helpful and sup p ortiv e (I think of Shanti Gu pta, Purdu e Unive rsity; Norman Johnson, Univ ersit y of North Carolina at Chap el Hill; Lucien LeCam, Uni- v ersit y of California, Berk eley; Ingram Olkin, Stan- ford Unive rsity; and Manny P arzen, T exas A&M Univ ersit y). The y ounger generatio n j ust th ou ght of me as another senior p erson, so they were fine. Rosen b erger: What is y our feeling ab out the role of w omen in statistics to da y? I can s ay , from my 12 W. F. ROSENBER GER p ersp ectiv e on 20 ye ars of search committees, that from a hirin g p ersp ectiv e, w e are th r illed to hav e qualified women candidates and comp ete hard to get them. And certainly p olicies on ten ure to allo w maternit y lea ve ha ve v astly improv ed o ver the years, as ha v e the comp osition of committees and senior administrators. Is there an y work left to b e don e? Flourno y: Y ou can see impr o v emen t, b u t there are still tr oubling f acts: just try to fi nd a w oman in the 2013 JSM aw ards bro c hure, for in stance. W omen are getting h ired at prop ortional rates no w , but aw ards, ten ure and adv ancement are areas wh ere there muc h is left to b e d one. See Lynne Billard’s new up date of Scott’s old d ata on the su b ject (Billard and Kafadar, 2015 ). That will depress y ou. 6. CONCLUSION Rosen b erger: Y ou talk ed a little ab out y our tr an- sition into a fully academic p osition. T h e latter part of yo ur career w as sp en t at A U and Universit y of Missouri (MU), and considerable time as depart- men t c hair, and a men tor to many div erse students. T alk ab out this. Flourno y: AU wa s a great p lace for me wh en I w en t there in 1988. I h ad left the C ancer Cen ter with a staff of 23, a b u dget of $700,000 and re- sp onsib ilities that had b ecome a bur d en wh en I b e- came con vinced of the need for more nim ble learning strategies in dose-finding clinical trials. I had eigh t do ctoral students at A U, and all but tw o of th em dev elop ed mec h anisms to con trol random walks and urn mo dels, and to p ro vide m athematical d escrip- tions of their cont rolled b eha vior. One work ed on issues of inference follo wing an adaptive design and one w ork ed on a problem in economics. I am proud that four of these studen ts are blac k and t wo are w omen. Unfortunately , a v ery destru ctiv e president came to A U, and by 2000 it w as clear th at STEM gradu- ate programs we re going to b e disman tled. AU had one of th e oldest statistics doctoral programs in the coun try and it w as sad to see it threatened b y ig- norance and arr ogance. T o remain in a department with a do ctoral program, I needed to mo ve and this led me to accept the c hair at Missouri in 2002. Wh en I stepp ed do wn as c hair in 2011, I h ad doubled th e n umb er of tenure-trac k facult y and added five teac h- ing facult y p ositions. I increased the presence of the departmen t across campus th rough join t app oin t- men ts and a targeted increase in ser v ice courses, and Fig. 8. Nancy ne ar A asgar d Pass i n the Enchantment L akes Wilderness A r e a, W ashington, wher e she was hiki ng with her husb and L e onar d and her c ol le ague L ori L. Thombs (Univer- sity of Missouri) fol lowing the 2006 Joint Statistic al Me etings in Se attle. I in cr eased th e p restige of the d epartmen t nation- ally , p ersonally pr omoting our facult y and enabling their p articipation in national and international ac- tivities. More details can b e found in a Ch apter I recen tly wrote on the history of statistics at MU (Flourno y and Galen, 2012 ). I ha v e graduated seven d o ctoral students from MU. W e work ed on adaptiv e and optimal d esigns; w e d ev elop ed new mo dels for sp ecific, c hallenging dose-resp onse problems and we ha v e illuminated the effect of h a ving dose allocations dep end on the his- tory of p r ior allo cations and resp onses. My stud ents con tin ue to br in g me a great deal of pleasure. Rosen b erger: W hat are your hobb ies an d in ter- ests? Flourno y: I lo ve hiking. I am not happy with a trip that tak es less than four da ys. A four-da y trip has tw o days out and tw o days bac k—so one is nev er v ery far from a road. After h iking for more than t wo da ys, on e m ust rely on one’s self muc h more com- pletely . It is so p eaceful. I ga v e up trying to hik e in the E ast and the Midw est United States. O ne just can’t get far enough a wa y from roads; and the moun tains aren’t h igh enough. I like trekking around A CONV ERSA TION WITH NANCY FLOU RNOY 13 tim b erline for a w eek or m ore w h ere the views are sp ectacular. I kee p going bac k to Y osemite, Kings Can y on and Sequoia National F orests. Nepal was great, to o. I try to get in one long hike eac h y ear. In the mean time, I d ance. I resumed ballet classes while at AU; it is great min d-to-b o dy exercise and w onderfu l f or strength and balance. Leonard and I enjo y E n glish country dance together. Thro w in Pi- lates and y oga and I am happy . T o survive a sev ere health chal lenge that had the do ctors stu m p ed, I gained considerable kno wledge of alternativ e metho ds and b ecame acc omplished in some. But that is another story . Rosen b erger: What’s next for Nancy Flourno y? Flourno y: W ell I ha v e a lot of ideas. I’m r eally in- terested in questions of inference f ollo wing adaptiv e designs. W e hav e some examples in tw o stage designs that maxim um lik eliho o d estimato rs are mixtur es of normals; some designs lead to estimators that are normal with random v ariances. I think our p relim- inary resu lts are generalizable, but this remains to b e sh o wn. I’m optimistic that tractable solutions to seemingly in tractable problems are at hand. REFERENCES American So ciety for T esting an d Materials (2010). Stan- dard test metho d for estimating acut e oral toxicit y in rats. American So ciety for T esting and Materials, AS TM E1163- 10. AS TM International, W est Conshohock en, P A. Azriel, D. (2012). A note on the robustness of the contin- ual reassessmen t metho d. Statist . Pr ob ab. L ett. 82 902–906. MR2910036 Azriel, D. , Mandel, M. and Rinott, Y. (2011). The treat- ment v ersus exp erimentation dilemma in d ose find in g stud- ies. J. Statist. Pl ann. I nf er enc e 141 2759–2768. MR2787743 Bar tlett, R. H . , R oloff, D. W. , Cornell, R. G. , Andrews, A. F. , Dillon, P. W. and Zwischen- berger, J. B. (1985). Ext racorporeal circulation in neona- tal respiratory failure: A prospective randomized study. Pe diatrics 76 479–487. Billard, L. and Kaf adar, K. (2015). W omen in statis- tics: Scientific con tributions v ersus rew ards. In A dvancing Women in Scienc e ( W. Pearson Jr. , L. M. Frehill and C. L. McN eel y , ed s.) Chapt er 7. Springer, New Y ork. Bozin, A. and Zarr op, M. (1991). Self-tun ing extremum op- timizer converg ence and robustness. In Pr o c. 1st Eur op e an Contr ol Conf. 91 672–677. W orld Scien tific, Singapore. Durham, S. D. and Flourno y , N. (1994). Rand om w alks for qu antil e estimation. In Statistic al De cision The ory and R elate d T opics, V (West Lafayette, IN, 1992) 467–476. Springer, New Y ork. MR1286322 Durham, S. D. , Flournoy, N. an d Rosenber ger, W. F. (1997). A random w alk rule for p hase I clinical trials. Bio- metrics 53 745–760. Flourno y, N. ( 1993). A clinical exp eriment in b one marrow transplantati on: Estimating a p ercentile point of a quan- tal resp onse curve. In Case Studies in Bayesian Statis- tics ( C. Ga tsonis , J. S. Hodges , R. E. Kass and N. D. Singpu r w alla , eds.) 324–335. Springer, New Y ork. Flourno y, N. ( 2014). A vignette of discov ery . In Past, Pr esent and F utur e of Statistic al Scienc e, in Cel ebr ation of the COPSS 50th Anniversary ( X. Lin , D. Banks , C. Gen- est , G. Molenber ghs , D. Scott and J.-L. W ang , eds.) 349–358 . Chapman & Hall/C RC, Bo ca Raton, FL. Flourno y, N. and Galen, M. (2012). H istory of the statis- tics department at the Unive rsity of Missouri. I n Hi story of Statistics Dep artments ( A. Agresti and X.-L. Meng , eds.). S pringer, N ew Y ork . Flourno y, N . and Hearne, L. B. (1981). Effects of database managemen t on the organization and adminis- tration of clinical trials. In First L awr enc e Berkeley L ab o- r atory Workshop on Statistic al Datab ase Managemen t ( H. K. T. W ong , ed.) 383–387. Lawrence Berkeley National Lab oratory , Menlo Park, CA. Flourno y, N. and Hea rne, L. B . (1990a). Quality con- trol for a shared multidisciplinary database. In Data Qual- ity C ontr ol: The ory and Pr agmatics ( G. E. Liepens and V. R. R . Uppuluri , eds.) 43–56. Dekker, N ew Y ork. Flourno y, N. and Hearne, L. B. (1990 b). Sharing scien- tific data I II : Planning and the resea rch proposal. IRB: A R eview of Hum an Subje ct R ese ar ch 12 6–9. Flourno y, N. and Or on, A. P. (2015). Up-and-down de- signs for dose-find ing. In Handb o ok of Design and Analys is of Exp eriments ( D . Bingham , A. M. Dean , M. Morris and J. Stufken , eds.) 862–8 98. Chap man & Hall/CR C, Boca Raton, FL. Flourno y, N. and R osenberger, W. F. , eds. (1995). A daptive Designs . Institute of Mathematic al Statist ics L e c- tur e Notes—Mono gr aph Series 25 . IMS, Hayw ard, CA. MR1477667 Iv anov a, A. and Flournoy, N. (2001). A birth and death urn for ternary outcomes: Sto chastic p ro cesses applied to urn m o dels. In A dvanc es in Statistic al The ory—A V olum e in Honor of The ophil os Cac oulous ( C. Charalambi des , M. V. K outras and N. Balakrishnan , eds.) 583–600. CRC Press/Chapman & Hall, Bo ca Raton, FL. Jennison, C. (1987). Efficient group sequential tests with u npredictable group sizes. Biometrika 74 155–165. MR0885928 Jennison, C. , Johnstone, I. M . and Turn bull, B . W. (1982). A symptotically optimal p rocedu res for sequential adaptive selection of the best of several normal means. In Statistic al De cision The ory and Re late d T opics, III, Vol. 2 (West Lafayette, Ind. , 1981) 55–86. Academic Press, New Y ork. MR0705308 Kulkarni, R. V. and Jennison, C. (1986 ). Optimal prop- erties of the Bec hh ofer-Kulk arni Bernoulli selection pro ce- dure. Ann. Statist. 14 298–314. MR0829570 Lai, T. L. and Robbins, H. (1982). Iterated least squares in multip erio d con trol. A dv. in Appl. Math. 3 50–73. MR0646499 Lane, A. , Y ao, P. and Flourno y, N. (2014). Information in a tw o-stage adaptive optimal design. J. Stat ist. Plann. Infer enc e 144 173–187. 14 W. F. ROSENBER GER Li, Z. , Durham, S. D. and Flourno y, N. (1995). An adaptiv e design f or maximization of a con t ingent binary resp onse. In Ad aptive Designs (South Had ley, MA, 1992) . I nstitute of Mathematic al Statistics L e ctur e Notes—Mono gr aph Series 25 179–196. I MS, Hayw ard, CA. MR1477680 Lin, Y. and Shih, W. J. (2001). Statistical prop erties of the traditional algorithm-based d esigns for phase I cancer clin- ical trial s. Biostatistics 2 203–215. Ma y, C. and Flournoy, N. (2009). Asympt otics in resp onse- adaptive designs generated by a tw o-color, randomly rein- forced urn . A nn. Statist. 37 1058–10 78. MR2502661 National Institute of Environmen tal Health Sciences (2001). The revised up -and-d own pro cedure: A test meth od for determining the acute oral toxicit y of chemicals. T echnical Rep ort 2-4501, NIEHS , W ashington, DC. O’Quigley, J. , Pepe , M. and Fisher, L. (1990). Con tinual reassessmen t method : A practical design for ph ase 1 clinical trials in cancer. Biometrics 46 33–4 8. MR1059105 O’Ro urke, P. P. , Cr one, R. K. , V a canti, J. P. , W are, J. H. , Lillehei , C. W. , P arad, R. B. and Ep- stein, M . F. (1989). Extracorporeal membrane o xygena- tion and conven tional medical therapy in n eonates with p ersisten t pulmonary hyp ertension of the newb orn: A prosp ective randomized study. Pe diatrics 84 957–963. Oro n, A. P. , Azri el, D. and Hoff, P. D. (2011). Dose- finding designs: The role of con vergence prop erties. Int. J. Biostat. 7 Art. 39, 19. MR2873999 Oro n, A. P. and Hoff, P. D. (2013). Small sample behavior of nov el phase I cancer trial desig ns. Clinic al T rials 10 63– 92 (with discussion). Peto, R. , Pike, M. C. , Arm it age, P. , Breslow , N. E. , Co x, D. R. , Ho w ard , S. V. , Mantel, N. , McPher- son, K. , Peto, J. and Smith, P. G. (1976). Design and analysis of randomized clinical trials requiring prolonged observ ation of eac h patient. I. Introdu ction and design. Br. J. Canc er 34 585–612. Peto, R. , Pike, M. C. , Arm it age, P. , Breslow , N. E. , Co x, D. R. , Ho w ard , S. V. , Mantel, N. , McPher- son, K. , Peto, J. and Smith, P. G. (1977). Design and analysis of randomized clinical trials requiring prolonged observ ation of each patient. I I. Analysis and examples. Br. J. Canc er 35 1–39. Reiner, E. , P ao letti, X. and O’Qui gley, J. (1999 ). Op- erating characteristics of the standard p hase I clinical trial design. Comput. Stat ist. D ata A nal. 30 303–315. Ro senberger, W. F. , Flourno y, N. and Durha m, S. D. (1997). A symptotic n ormalit y of maximum likelihood es- timators from m ultiparameter response-d riven designs. J. Statist. Plann. Infer enc e 60 69–76. MR1453033 Storer, B. E. (1989). Design and analysis of ph ase I clinical trials. Bi ometrics 45 925–937. MR1029610 Thall, P. F. and C ook, J. D. (2004). Dose-finding based on efficacy-toxicit y trade-offs. Biometrics 60 684–693. MR2089444 Treutwein, B. ( 1995). Minireview: Adaptive psyc hophysical proced ures. Vi sion R es. 35 2503–2522. W are, J. H. (1989). Inv estigating therapies of p otentially great b en efi t: ECMO. Statist. Sci. 4 298–340. MR1041761 Wei, L. J. and Durham, S. (1978). The ran d omized play- the-winner rule in medical trials. J. A mer. Statist. Asso c. 73 840–843. Weiden , P. L. , Flourno y, N. , Thomas, E. D. , Pren- tice, R. , Fe fer, A. , Buckner, C. D. and Storb, R . (1979). Antileuk emic effect of graft-v ersus-host disease in human recipients of allog enic-marrow graf ts. New Engl. J. Me d. 300 1068–1073. Weiden , P. L. , Flourno y, N. , Sanders, J. E. , Sulli- v an, K. M. and Th omas, E. D. (1981a). Antileuke mic effect of graft-v ersus-host diseas e con tributes to improv ed surviv al after allogeneic marrow transplantation. T r ans- plant. Pr o c. 13 248–251. Weiden , P. L. , Sulliv an, K. M. , Flournoy, N. , Storb, R. , Thomas, E. D. and the Sea ttle Bone Marro w Transplant T eam (198 1b). Antileuk emic ef- fect of chronic graft-versus-host disease. Contribution to impro ved surviv al after allogeneic marrow transplan tation. New Engl. J. Me d. 304 152 9–1533. Weiden , P. L. , Flournoy, N. , Thomas, E. D. , Fefer, A. and Storb, R. (1981c). Antitumor effect of marrow trans- plantatio n in human recipients of syngeneic or allogeneic grafts. In Gr af t-V ersus-L eukemia in Man and A nimal Mo d- els ( J. Okunewick and R. Meredi th , ed s.) 11–23. CRC Press, Bo ca Raton, FL. Zack s, S. (2009). Stage-wise A daptive Designs . Wiley , H ob o- ken, N J. MR2559830