A review of problem- and team-based methods for teaching statistics in Higher Education

A review of problem- and team-based metho ds for teac hing stati stics in Higher Educ ation Elinor Jones 1 and T om P almer 2,3 1 Departmen t of Statistical Science, Univ ersit y College London, London, UK 2 Departmen t of Mathematics and Statistics, La ncaster Univ ersit y , Lancaster, UK 3 MR C In tegrativ e Epidemiology Unit and P opulatio n Health Sciences, Bristol Medical Sc ho ol, Unive rsit y of Bristol, Bristol, UK 11th Marc h 20 21 Abstract The teac hing of statistics in higher education in the U K is still largely lecture-based. This is despite recommendations suc h as those given by the A merican Statistical Association’s GAISE rep ort that more emph asis should be placed on active l earning strategi es where stu d ents take more responsibility for their own learning. One p ossible model is th at of c ol lab- or ative le arning , where students learn in group s through carefully crafted ‘problems’, whic h has long b een s uggested as a strategy for tea c hing statis- tics. In this article, we review tw o sp ecific approaches that fall und er the collaborative learning model: problem- and team-based le arning. W e con- sider the ev idence for changing to this mo del of teac hing in statistics, as w ell as give practical suggestions on how this could b e implemented in typical statistics classes in Higher Education. 1 In tro duction Univ ersity courses in statistics hav e tr a ditionally b een g iven in the instructional style, in which a lecturer transcr ib es a set of no tes for students over a cours e of lecture s. In this pro cess studen ts are passive recipients of information. This metho d of delivery can b e sca led up to co pe with ever increasing class sizes , a crucial factor in determining which teaching metho ds could rea listically b e implemen ted, but t he quality of the resulting education is questionable. The America n Statistical Asso ciation, how ever, sp ecifically endorses a more a c- tive appro ach to teac hing with students taking responsibility for their own learn- 1 ing (Carver et a l., 2016). Going further, the idea that statistics educa tion should resemble statistics pra ctice - in terms of presenting legitimate a nd relev ant sta- tistical research questions as part o f the learning process (Rumsey, 2002), and relying on co op era tion, comm unication, and team-work (Ros eth et al., 2008) - is clearly adv antageous but do es not a alw ays happ en in higher education. Collab ora tive le arning, where students learn in g roups thr o ugh carefully cr afted ‘problems’, has long b een suggested as a strategy for tea ching statistics (Garfield, 1993; Ca rver et a l., 2016; Ros eth et a l., 2 008). Despite r ecommendations, statis- tics education in hig he r education in the UK is still la rgely lecture- based, though the tide is s lowly turning. In this article, we rev iew tw o approa ches that fall under the co lla b orative learn- ing mo del: problem- and team-bas e d lear ning (PBL and TBL). W e fo cus o n PBL and TB L for tw o reasons. Firstly , they provide a strategy for fundamen- tally changing the nature of how statistics is taug ht throughout a cour se or mo dule, rather than p ossibly o ne-off activities to promote effective learning. Secondly these learning mo dels have been used extensively in other disciplines to goo d effect, with a considerable b o dy of evidence do cumenting their adv an- tages. Though the r eview has in mind intro ductory underg raduate statistics, we ho pe that the ideas discus s ed her e may a lso b e useful to those teaching mor e adv anced cour ses. The pap er is o rganised as follows. In Section 2 we describ e bo th PBL a nd TBL, and consider the evidence for using these stra tegies - b oth sp ecifically for statistics and a lso more ge ner ally - in Section 3 . In Section 4 we consider the practicalities of using these meth o ds for the teaching of statistics and offer t ips for effective implemen ta tion. 2 Group-based enquiry-driv en teac hing meth- o ds PBL and TBL ha ve made their mark in a num b er of disciplines, including medicine and allie d health pro fessions, business, and engineering . Both ap- proaches fall under the umbrella o f ‘active learning’, lo osely defined as engag ing student s in activity , which have b een a dvocated for STEM disciplines including Mathematics (Braun et al., 20 1 7). In this case, the activity consists of student s learning throug h a sequence of car efully crafted problems in small gr oups or teams. The set-up, and therefore the na ture of how students learn, is different. In PBL, the pro blem p o sed be c omes the sour ce of learning : students b ecome indep endent seekers of information in order to pr ovide a so lution, but under the guidance of a facilitator. F or TBL, how ever, students learn first b y using reso ur ces made av ailable by the instr uctor a nd cla ss time is then dedica ted to applying this knowledge to solv e t he pro blem in tea ms. 2 In this section we re view the ‘cla ssic’ implemen tation of b oth PBL and T B L. Examples of possible v ar iations on these a re given later in Sections 3 and 4 . 2.1 Problem-based learning Problem based learning has b een used in medical s chools and law schools a s early as the 196 0s, for example b y McMa s ter Medica l Schoo l, and with increa s- ing upta ke since the 1980 s (Knight and Y or ke , 20 03; B oud and F eletti , 199 7; Sch warz et al., 2001). The tra ditional instructio na l appro a ch to medical ed- ucation, consisting of an intensiv e pre-clinical p er io d of basic s cience lectur es follow ed by a clinical teaching pr o gramme, has b een criticised for failing to eq uip do ctors with all the skills they needed and for not pro viding studen ts with the context o f how their knowledge should b e applied (Lancaster Medical Sc ho o l, 2016). In the UK the Gener al Medical Council set the require ments for how medi- cal s tuden ts should b e trained. They advocate PBL for the following reas ons (Lancaster Medical Sc ho ol, 2 016; Sav e ry, 2006): • students must hav e resp onsibility for their o w n learning, s ince lea rning is most effective when it is active; • pro blem scenario s facing students should b e co mplex, since real-world medical pr oblems ar e rarely stra ig htf orward; • lear ning should b e in tegrated from a wide ra ng e o f disciplines a nd sub ject areas; • lear ning s hould integrate co llab oratio n, since clinica l pr actice demands that do ctor s shar e information and w or k constructively with others; • students sho uld share with their w ork gr oups what they have learned and how that con tr ibutes to th e so lution of the problem; • a summary analys is of what has b een lear ned sho uld b e undertaken b e - cause refle c tio n and ev aluation a re critical; • self and p ee r assessment sho uld b e regularly under taken. PBL aims to teach students to identify problems and then to des ign a set o f ob jectives, the acco mplishment of which will lead to the developmen t of the solution (S chm idt, 1983). Medicine isn’t the only area to wide ly use PBL: la w is the other main area which has adopted this strategy at scale. Sch warz et al. (2001) p oints out that the challenges face d by a law schoo l hav e similarites and differences with thos e faced by a medical schoo l. It is not unreasona ble to a ssume tha t the same is true in statistics tra ining : many of the asp e c ts listed ab ov e apply directly to applied or pra ctical s tatistics education, with the others requiring only minor mo difications in lang ua ge. Though the fo cus here is on the teac hing and learning of applied, or pr actical, statistics, there is also evidence tha t P BL ca n b e used in teaching and lear ning the more theoretica l 3 asp ects of sc ient ific disciplines, including in mathema tics (Dahl , 2 0 18). There is certainly scop e, therefore , to do similar ly with teaching mor e theo r etical asp ects of statistics. 2.1.1 Group formation and nature of problems In contrast to tra ditio nal lecture cour ses common in Higher Education, s tudents are randomly ass igned to sma ll gr o ups (typically b etw een 6 and 10 student s, though groups are changed every few weeks) to w ork on an op en-ended pr oblem, often calle d scenarios. Within a P BL gro up ther e are sp ecific roles. Each member of the g roup is exp ected to peform eac h r ole at lea st once p er term. The r oles are as fo llows: • Chair – to mov e the group thr o ugh the stage s of P BL in a timely man- ner, to ensur e cov er age o f a topic, and to encourag e all o f the g roup to participate. • Scrib e – listens and records informa tion (often on a whiteb oa rd), writes up the agreed ob jectives, and con tributes to discussions. • Other g roup members – contribute to disc us sions, ar ticula te knowledge, ident ify streng ths and w eakness es in the group’s kno wledge. 2.1.2 F ormat of PBL ses sions The gr oups meet to discus s the problem, along with a tutor who acts as a fa c il- itator b y as king questions and prompts to guide discussion tow ar d the learning outcomes. With each new scenario the student s rota te thr ough the differ ent roles (chair p erso n, scrib e, group member), whic h gives them the chance to develop ne w skills. During the first group meeting, studen ts identif y what pa rts of their k nowledge are lacking in tackling the problem. They then set their own goals in terms of what information they requir e in order to solve the problem in hand. E a ch mem ber of the group resear ches the required information. The gro up then reconv e ne s to discuss what they learnt from their self-s tudy , and apply their new knowledge to the pr oblem in hand. The r e will t ypically b e several days betw een the fir st ses sion and the second session. 2.1.3 Assessm en t Each PBL session is ev aluated thro ugh surveys in which the student s reflect on their learning exp eriences. The facilitator guides students throug h this se lf- assessment of outcomes relative to the goals they set a t the start, to s how them the extent of t heir le a rning. Final assessment can ta ke any form, a nd need not be reliant on t he gr oup-work during PBL. 4 2.2 T eam-based learning 2.2.1 T eam formation and nature of probl ems Similarly to PBL, the fundamental idea of TBL is that students work on pro- fessionally relev ant pr oblems. That is, pro blems that are similar to what th ey might encoun ter in the workplace (Michaelsen et al., 200 4). The learning dif- fers from that of P BL, how e ver. T eams are fo r med o f b etw een five and seven student s, but are not randomly allocated. Instead, th e instruc to r car efully cre- ates groups to e nsure that they are heterogeneo us, for example, in terms of preparatio n or previous ex p er ience. Students do not c hange groups during the course. TBL is now p opular in Nurs ing and Medical scho o ls (Liu and Beaujean, 2017), a nd though in these settings the focus tends to b e on developing a pplied knowledge, ther e are examples of its use in teaching more theor etical a sp ects o f mathematics (Parappilly et a l., 2019) and ph ys ic s (P ar appilly et al., 2015). 2.2.2 F ormat of TBL sess ions Before the session Prior to the teams meeting to discuss the allo cated prob- lem, e a ch student must prepar e for the gro up work by studying the provided materials. This could be in the form of reading, w atching videos , or any o ther activity that prepares students sufficie ntly for the task ahead. During the session – readiness assurance The Readiness Assurance Pro- cess (RAP) aims to ensur e that all students hav e the pr e-requiste k nowledge of the cour s e material se lf-s tudied, in order to take part in the later problem solving exercis e. Each studen t completes a n individua l test (individual Readiness Assurance T e s t, or iRA T), designed to highlight any deficiencies in the student’s unders ta nding of the pr e-class material. These tests are typically quick-fire m ultiple choice tes ts , done using electro nic voting equipment so that they ar e marked ins tantaneously . Once completed, students re-take the test, but this time in their allo cated groups (team Readiness Ass urance T est, or tRA T). Discussion of the ques tions among team members is encourage d, so that the tRA T itself b ecomes a learning to ol where students learn from eac h o ther. Results are av aila ble immediately after the tRA T, a llowing students to as s ess their understanding, but also to contest the questions and/or answers. T eams provide written justification as to wh y they think they deserve a higher mark, with evidence, e.g. fr om course material, for the instructor to consider. If the instructor finds in fa vour of the team, additional marks are awarded. How e ver, only teams who co ntest will b e eligible for additional mar ks, even if the problem detected w as c o mmon for all g roups. This encourages students to question the material, a nd also helps with team c o hesion. 5 During the sess ion – l ecture Finally , a lecture co mpo nent directed at all groups g ives the instructor opp or tunit y to clar ify misunders tandings and com- mon conceptual e r rors which were pic ked up dur ing the iRA T and tRA T. During the ses sion – working on a problem The r emaining time is sp ent on s olving a pr oblem using the mater ial learnt. The problems a r e aimed at testing students’ deep er understanding of the course material, while the RAP pro cess tests base knowledge (Liu and Bea ujea n, 2017). Such pro blems gener- ally satisfy f our criter ia (commonly kno wn a s the “ 4S”): • the problem m us t be Significan t; • the teams hav e a Specific set of possible answers fr om which th ey cho ose one; • each team works on the Same problem; • teams m ust Simultaneously rep or t their final a nswer. Impo rtantly , the pr oblem must not b e e a sily segmented in to smaller pa rts that different team members ca n tackle: the idea is that the gro up works together on the w ho le problem. 2.2.3 Assessm en t Assessment for t he course is g enerally a combination of individual tests (iRA T scores and final exam mark), and gr oupw or k mark (tRA T scores, s cores fr om the pro blem and pe e r ev aluations). F ur ther s ummative a ssessment can take a ny format. 3 Evidence of effectiv eness Among their recommendations, the GA ISE College Report suggests that mo d- ern statistics education should teach sta tis tica l thinking as a n inv estigative pr o- cess of problem solving a nd decision making, should in teg rate real data with a context a nd purp os e, and f oster activ e learning (Carver e t al., 2016). All these attributes are fundament al to b oth pro blem- and team- ba s ed learning. The GAISE College Repo rt also calls for using technology to explore concepts and analyse data. While in medicine, for ex ample, hands -on patient-based a ctivities may not be p ossible in a classro om setting, we can incor p o rate practical data analysis into problem- or team-based learning in statistics. The general a pproach of group-ba s ed lea rning aligns with the constructivist philosophy of lear ning, where studen ts activ ely construct their own knowledge rather than passively receiv ing it (Garfield, 1993). Not only do student s learn the sub ject matter in this w ay , but they develop s ofter s kills in problem solving that captures some of the non-for mal le a rning that happ ens in the workplace (Eraut, 20 00). 6 Group-based lear ning is p osited as a largely po s itive str a tegy for teaching non- sp ecialist students. Thoug h we found no published liter ature on the effective- ness of such stra tegies in teaching statistics to sp ecia lis t students (i.e. those pursuing degr ees in the mathematical s ciences), group-based lea rning has been successfully implemented for mathema tics students in discrete ma thematics (Paterson and Sneddon, 2011). Though t here are numerous appro aches to measuring effectiv eness in tea ching, the evidence relating to gro up- based learning tend to fall into three catego ries: • p erfor mance on end- of-mo dule ass e ssments or similar; • long- term retention of information; • student enjoymen t or enga gement with the material. W e discus s the findings of other studies in implementing v ar iants of TBL or PBL in each of these ca teg ories. Ev idence of impact on staff is considered separately in Section 4. 3.1 P er for mance on end-of-mo dule assessmen t s Kalaian and K asim (20 14)’s meta-ana lysis o f the effectiveness o f group- based learning in statistics, in co mpa rison to lecture- based instruction, revealed that their effectiv eness is dep endent on the t yp e of gro up-based lea rning imple- men ted. In particula r , co o p erative o r c ollab ora tive learning (for example TBL) was fo und t o be effectiv e while no evidence of improv ed a cademic ac hievemen t was found for inquiry-based metho ds (suc h as P BL). Though the meta-analysis did not point to an over al l b enefit to using PBL in compariso n to lectur e -based instructio n, there are examples of sup erio r studen t per formance on statistics a ssessments after a P B L-type cour se rather than a lecture course (Ka rpiak, 2011). How ever, it is not cle ar whether this is gen- uinely due to b etter understanding of the cour se mater ial o r some other factors (Gijbels et al., 200 5; Karpiak, 2011). F or no n-statistics ma jor cours e s in pa rtic- ular, the use of PBL may be helpf ul b ecause it genera tes a co nstant use for the statistical metho dolo g y , and hence pr ovides students with a motiv ation to lear n (Jaki and Autin , 2009). B e tter performa nce on mo dule asses sments could also be a conseque nce of s tuden ts enga g ed in active learning as opp osed to lea rn- ing passively , r ather than the e ffect of the P BL itself, or unwittingly increa sing the amoun t of guidance from PBL tutors to studen ts esp ecially since students bene fit from guidance in v ery small groups (Bud´ e et a l., 2009). Improv ed grades on end-of-mo dule tests w as also observed for TBL for service- t ype cours es mathematics (Nanes, 2 014) and s p e cifically in statistics (Liu and Beaujean, 2017; Haidet et al., 201 4). 7 3.2 Long-term r eten t ion of kno wledge While there is a growing b o dy o f evidence to suggest v ario us group- ba sed lear n- ing metho ds improv e end-of-module assessments, far fewer s tudies ha ve lo oked at the lo ng-term impact of these stra tegies on knowledge retention. W e found no studies lo o king at long-term retention o f knowledge and skills in statistics , a nd only one in teaching medical studen ts (Emke et al., 201 6). In this study , which lo oked at shor t- and long - term retention of kno wledge and compar ed a co hort of s tuden ts taug ht via TBL with a cohort that w a s traditionally taught, there was s o me evidenc e that the TBL gr o up p erformed better on assess ment s in the short-ter m but no ev idence tha t they retained mor e knowledge long er term. 3.3 Studen t enjoym en t and engagemen t A lar ge b o dy of evidence in the litera ture po ints to g roup-bas e d le a rning as being a p os itive exp erience for students. That this is a n asp ect that rec eives most attention is not s urprising given the difficulties in compar ing understanding of course material b etw e en co ho rts. Studen ts ar e generally p o sitive abo ut TBL in mathematics (Nanes, 20 14; Kr o gstie et a l., 2018) a nd in statistics (St. Clair a nd Chihar a, 20 12). In pa rticular, some re- po rted studen ts finding mathematical ideas more accessible when t he material was taugh t as a TBL class as opp os ed to traditiona l lectures (Paterson et al., 2013). Balancing this overwhelming po sitivity are some interesting s tuden t in- sights from other studies. Naturally , not all student s will enjoy an active gr oup- learning en vironment (Haidet et al., 20 14), but more specifically a group en v i- ronment can enco urage so me s tudents to ‘coast’ in TB L ma ths cla sses, r elying o n their team-mates f or bac k-up (Paterson et al., 20 13). Other s - p erhaps weak er student s - ma y find the team e nvironment int imidating (St. Cla ir and Chihara, 2012). In the latter ca s e howev er, team- working and communication is a n essen- tial sk ill which should b e dev elo p ed alo ngside mathematical or statistical skills (Nanes, 201 4; Tin ungki, 201 5). Though mo s t o f the resear ch we found on student enga g ement was based o n TBL, a sp ects of problem-base d learning for large cohorts hav e b een consid- ered. Kleger is a nd Hurren (2011) found that PBL se s sions for a phar ma ceutical course increas ed attendance in comparis on to traditiona l lectures. This w as tri- alled with and without student a dditional ma rks for attendance. They found that offering such a reward for attendanc e did not significantly affect atten- dance rates. F or la rge statistics classes in particular, Bud´ e et al. (2009) found that more guidance from tutors/facilitators during the session r esulted in be t- ter student per ception of the course . They warn that increasing the amo unt of guidance from tutor s in a PBL setting could ina dverten tly lead students to bec ome pas sive ab out their learning a nd less mo tiv ated, though they did no t find evidence of this in their study . 8 While the evidence on balance sugges ts impro ved student engagement through the use of TBL and PBL, it is not clear whether these approaches to teaching will suit all students. Making learning inclus ive, for example to thos e with additional educational needs , ma y mean that adaptations to PBL and TBL ar e necessary though to our knowledge there are no published pap ers exploring this particular asp ect. 4 Problem- and team-based learning of statis- tics in practice Statistics is per haps an obvious candidate for group- based lear ning, rich with opp ortunities in tackling ‘real’ problems and can easily b e framed as a b eliev a ble and relev ant problem for either team- or problem-based learning strategies. It is ther efore not surpr ising that PBL for exa mple has b een used in statistics courses for o ver 20 years (Hillmer , 199 6; Boyle, 1999). The natur e of statistics means it is rather dependent on the or der that the material is intro duced. Its hig hly str uctured and s ometimes abstract nature makes teaching statis tics via gro up- based lear ning a challenge: deficiencies in understanding of basic c o ncepts may cause difficulties in under standing mo re complex pr o cedures (Bland, 2004; Bud´ e e t al., 200 9). Studen ts can’t front-load a large a mount of information so adaptations ma y be necessa ry . Nane s (2014) for example, in tea ching a cours e o n linear alg ebra v ia TBL, sugges ts increa sing the amount o f testing a nd making the real w orld problems sho rter though ca re m ust b e tak en not to ‘teach to the test’. It is not unre asonable to assume that the same issue could arise in teac hing statistics in this manner. Implemen ting group-based lear ning in statistics therefore needs careful consid- eration, a nd in some cas e s modifica tions may be necessary . In this sectio n we review the ma jor comp onents of gro up-based lear ning and give a dvice on prac- tical solutions to po tent ial issues. 4.1 The real-w orld problems Problems for gr oup-based lear ning ca n take any format, though will be differen t in nature for PBL and TBL. In TBL, s tuden ts ar e required to co mplete a set of learning activities for the sessio n - such as pre-reading , watc hing videos , or completing other tasks - when technical information ca n b e conv eyed which is relev ant in s olving the proble m. In this wa y , common pro ce dur es such as hypothesis testing and statistical modelling can b e taught. Contrast this with PBL: it is pr obably unrealistic to exp ect students to tackle data- driven problems solely throug h PBL (Bla nd, 2004). F o r example, exp ecting studen ts to come to the conclusion that a t -test is appropriate without some prior knowledge is unreasona ble, though intro ducing these co ncepts in other wa ys is p o s sible. Here we discuss alternatives to data-driven real-world problems , and their suitability for b oth TBL a nd PB L. 9 Real-world problems that don’t require directly handling data may b e easier to implemen t in class, esp ecially if computing power is not req uired. These can still pr ovide a r ich learning e x p e r ience, and indeed may enhance a student’s broad understanding of the s ub ject while alleviating some of the difficulties in obtaining rea l, relev ant, a nd well structured data f or teac hing purp ose s. Alternatives to da ta driven problems could b e of the form of a resear ch pap er or similar (Bland, 2 0 04). Asking students to read, dig est and rep or t back on findings from a resear ch paper - esp ecially if it is of dir e ct interest to the students - would broaden the sc o p e o f statistics education, taking the emphasis awa y fro m mechanical details to interpretation o f results, and also mo tiv a ting studen ts to see the p ow er of statistics in their own discipline. This s trategy in pa rticular is suitable for b oth PBL and TBL. In P B L, the tas k could be phrased around understanding the statistical met ho ds emplo yed in a pap er and why they w ere used. In TBL, student s could cr itically appr aise the use of techniques in co nt ext and sugg est a lternative wa ys o f address ing the pap er ’s res earch questions a nd/or put forward a different analysis plan f or the data collected. In the same vein, students could b e asked to provide advice on a co nsultancy basis either o n the design o f a n exp eriment or on the analy sis of previo usly- collected data. F or the latter, carrying out the data ana ly sis could b e s et as a task outside the class (in the case of PBL, for example, befor e the next group meeting), or even as pre- work in TB L b efore the next session. Results of which could then b e used a s a springbo a rd for the following workshop either in terms of discussing output or applying t he res ults to a connected problem. These a re r elatively easy ideas to implement in introductor y o r even intermedi- ate co urses in statistics. T eaching more adv anced mathematical statistics in the same v ein requir es more though t. E m bedding the topic of interest in to a real- world pro blem may require a little flexibilit y in what we think of as ‘r eal-world’ as has b een done in more adv anced mathematics courses (Nanes, 2014), though this is not a lwa ys the c a se (for exa mple, it is not difficult to think of many applications of the central limit theorem). With P BL we hav e the adv antage of a dedicated facilita tor who can help to g uide students throug h p ossibly abstra ct ideas, and in TBL the course materia l students read b efore the gr oup sessio n can provide the necess a ry theory b efore tackling the problem. In teaching more abstract co nce pts like this, teachers may need to provide students with more guidance on how to tackle the problem, esp ecially in the co ntext of TBL, fo r example in explicitly a sking for students to think a bo ut designing simulations in or der to reach a solutio n. Whatever the format o f a real- world pro blem, Garfield (19 93) emphasises that the hallmar k s o f go o d g roup activities include that all s tudents contribute to the task in ha nd, and sug g est that this could b e done by simply emphas ising this. Both PBL and TBL b enefit fro m having problems to so lve that cannot be split into smaller sub- pr oblems to b e tac kled individually . In App endices A a nd B w e provide an outline of a PB L a nd TBL sessio n, 10 resp ectively . The P BL sessio n fra mes a general q ue s tion a b o ut men tal health and requir es the students to identify the gaps in their knowledge a nd, somewhat independently , fill those gaps in orde r to complete the task. The TBL sessio n is based on the more technical ar ea of probability . Here, the pre-s ession mater ial that students a re requir ed to work o n ensure that they hav e the necessa ry basic understanding of pro bability which c an then b e a pplied to a pro blem co ncerning the sensitivity and sp ecificity of medical tes ts. 4.2 T eac hing space An y form o f gr oup-based lea rning b enefits fro m s uitable cla ssro om- like tea ch- ing s pace where students can comfor tably work in gro ups . T raditionally for PBL, this req uir es sourcing a suita ble ro o m for each group and having access to lea rning spa ces conducive to gro up work has been found to improv e ses- sion outcomes (Sc hw arz et a l., 2001; Jones, 1988). This is often to o complex to manage, esp ecially with ever increa sing class sizes in statis tics , with the only viable alternative to host sess ions in la rge lec tur e thea tres (Nicholl and Lou, 2012; Kle geris and Hurr en, 2 011; Rob er ts et al., 2 005). With some or ganisa- tion, how ever, running PBL in these space s is not ins urmountable. TBL, by its very nature, isn’t hampe red by such space c o nstraints and is de- signed to work in lec ture theatr es. Not a ll lecture theatr es a re created equal, how ever: single level lecture thea tres will ma ke it far ea sier for s tudents to inter- act within their tea m in compariso n to the usual slo ping tiered theatre. It is not surprising that Esp ey (2008) found that student attitudes tow ar d team-based learning impr ov ed with when studen ts per ceived the en viro nment they were in to b e a comfortable space in whic h t o work in their teams. Nic holl and Lou (201 2) sug gests using a classro om that is larger than you need for your group size, to create a more comfortable environmen t for students and to a llow the instructo r easy ac c e ss to ea ch g roup. O f course, computer labs may be re q uired for problems requiring a data-driven solution. Co mputer r o oms are often easier to set up for gr oups to w or k together in the s e nse that they allo w some flexibility in rea rrang ing seating eas ily to suit each tea m. It may b e b etter for g roup co he s ion if students a re not allo cated a P C each; one P C p er team go es so me wa y to ensure that the s tuden ts in a gro up interact with one another rather than eac h student ‘doing t heir own thing’. 4.3 Staff r esources T raditional PBL is staff-intensive, requir ing a tutor o r facilitator fo r each gr oup. This is unlikely to be an optio n for man y cour ses, esp ecially as classes in statis- tics are rapidly increasing in size. Though TBL may seem more pra c tica l a s it do es not require a fac ilita tor for each group, so me institutions have been suc- cessful in running PBL se s sions with o nly one fa cilitator for the entire class. Re- searchers found that r unning PBL alongside traditional lectur es in bio chemistry and physiology , without having a dedicated tutor for ea ch g roup, was succ e ssful 11 in terms of improving pro blem solving skills as w ell as studen t satisfac tio n and motiv ation (Klegeris and Hurr en, 2011). Nic holl a nd Lou (2012) suggest using on-line pla tforms s uch a s Poll E verywhere o r Twitter so that s tuden ts can send questions to t he lone instructo r, who in t urn ca n either pro ject answers for the whole class to see or initiate a class discussion. Without a tutor fo r ea ch gro up, how ever, students need to have some background knowledge of the topic under consideratio n (Nic holl and Lou, 201 2). In con tra s t, Rob erts et al. (20 05) co mpa res tra ditional PBL for undergraduate medics with a mo difica tio n wher e students tackle PBL-like tasks without a ded- icated g roup tutor. They conclude that the mo dification is a useful alternative when insufficient staff res ources are av ailable. They do find, ho wever, that stu- dent s with a dedicated facilitato r are more likely to p erceive the lear ning activity as b eing s up er ior thoug h no difference w a s detected betw een the t wo g roups in terms of ac hievement . 4.4 Creation of gr oups and studen t engagemen t T eaching in the mathema tical sciences is o ften in traditio na l lecture s with indi- vidual assignments a nd assessmen ts. T his is a t odds with the nature of mathe- matics at res e arch level: a fundamentally collab or ative endeav o ur. In sta tistics, courses with gro up-work components ha ve been co mmon for quite s ome time, as repo r ted b y Garfield (1993), Hillmer (1996), Boyle (19 99), and Jaki (2009). How ever, students’ exp erience of this wa y of working needs to b e taken into account at the start of a ny cour se using group-based lea rning. Like any new interv ent ion, it may take time for s tuden ts to get used to the idea of working in gro ups. Students may engag e more with the pro cess once they get used to it, so doing this every now and again might no t s how the rea l p otential of team-based learning. In the first instance, explaining the structur e of ea ch sessio n, making clear how groups are exp ected to w ork together, w ha t is e x p e c ted f rom studen ts, how to access help, the role o f any facilitator s, and g eneral c o de of conduct, sho uld be the first priority; this is esp ecia lly so for implementations in la rge classes (Rob e r ts et al., 200 5). In par ticular, studen ts who hav e little or no exp erience with small-group learning strategies like PBL or T B L will need more supp o r t, and a ll s ources of help need to b e highlighted. All gr oups need to feel that they understand the task in hand, feel confiden t tha t th ey can spea k to a tutor when they need guidance , and that they hav e s ufficient r e s ources. F o r the latter in particular, this may mean suitable written and/ o r videoe d material. It has bee n suggested that r e c ording any lecture co mpo nents of cour s es - whic h o ccur in b oth PBL a nd TBL - b enefits studen ts (Ja ki, 2 009; Jaki a nd Autin, 2 009). Course leaders m us t b e prepar e d for initial s tudent resistance, esp ecially if stu- dent s’ other cour ses are taught traditio nally . Some student s may see gr o up- based learning as a glorified version o f se lf-study (in which case, wh y pay for an education?) while others may worry that their mark s will be unfav ourably 12 influenced if having to rely on teammates. Resp onses to such cr iticisms and con- cerns could for example include the p edag ogical reaso ns for teaching s ta tistics in a gr o up-based lea rning en viro nmen t, or the benefit in ter ms of development of soft skills v alued by employ er s. Students worried tha t their gra des will b e un- fav oura bly influence d may b e placated if they a re rea ssured of the pro cedures in place to ensure that mark s a re allo cated fairly . Even the str o ngest students b e n- efit from g roup-bas ed learning: stro ng student s in groups that work well (e.g. where students a re inv ested in the group’s achievemen t), could benefit from thinking ab out concepts at a deep er level in o rder to explain them to weaker mem ber s of the group. There are als o opp or tunities f or students to assess ea ch other. Strategies such as group members having to asses s and provide feedback to each other can help students feel that contribution is rewarded while co asting in the group ha s negative consequences (F ree man and Mc kenzie, 2002). How gro ups a re formed can influence the success of a g roup-base d learning course: these lear ning s trategies work only when students enga ge, a nd if stu- dent s are inexp erience d in gr oup w orking then this needs to b e monitored and managed ca refully (Hansen, 2006). In their original format, both PB L a nd TBL g roups are chosen by the course leader and these groups r emain to g ether for more than one session to enco urage cohesio n and e ns ure diversity of groups. Groups that remain tog ether over a p erio d of time instead of changing on a reg u- lar bas is tend to display a mor e p ositive gro up dynamic (Sweet and Michaelsen, 2007). What is more, b etter student enga gement within g roups has b een noted when gr oups not o nly stay together but a lso work together on a r egular basis (Theobald et al., 2 017). Once gr o ups are for med, in ter na l dynamics ca n influence studen t performa nce. F actors that hav e b een found to low er student ac hievemen t include being in a group wher e one student dominates, and/or feeling uncomfortable in the gr oup; these tends to b e mor e prominent factors when the group-work in volves high- stakes as sessment (Theobald et al., 20 1 7). Strategies to facilitate p ositive gro up working metho ds and thus increas e en- gagement may b e useful, es pe c ia lly if studen ts are not used to group-ba s ed working. One appro ach that has b een sug gested to increase students’ comfort in gro ups is to e stablish g roup ‘norms ’ (Theo bald et al., 2 017). F or exa mple, groups could b e requir ed to write and submit their own contract for co de of conduct, g oals, and methods o f working, which could help in esta blis hing trust through clar ifying co mmitmen ts to each o ther (Hunsaker et al., 2 011). These contracts could also b e helpful in giving groups a wa y of dealing with dominant student s. If g roup-work le ads to a summative assessment, a co ntract could in addition allow the gr oup to negotia te mark allo cation, for exa mple in how marks are distributed betw een gro up- a nd individual- compo nent s, or ho w individual student s will b e assessed if p eer- assessment is to be used. It may b e tempting to allow students to cho o se their o wn groups fo r a num b er of reaso ns , including making students feel more comfor table (Theobald et al., 2017), po tentially dec r easing student resistance, and ease of a dministration. In 13 addition, allowing for changing tea ms in each s e ssion may also b e tempting. This is esp ecia lly so if students a re no t requir ed to attend lecture s essions mak ing steady teams difficult to mana ge, though this has been shown not to b e as effective as groups repe atedly working together (Sweet and Michaelsen, 20 0 7). Allowing studen ts to choose ma y also compromise the heterog eneity within groups. Moreover, those who do n’t hav e an immedia te friendship group in cla s s may b e severely disadv antaged when studen ts are permitted to choose their own gro up: though no published work could b e found loo king at the effects of this, the authors ’ own exp erie nce is that s tudents joining a group of students who a lready know each other may result in pro blems with gr oup attachmen t, while cr eating extra groups consisting of these studen ts ma y lead to feelings o f resentmen t. A half-way hous e is to in volve students in th e for mation o f gr oups in the sense that they decide on how the groups a re chosen even though, ultimately , the groups ar e chosen by the course lea der. F o r example, inv olving students in decisions ar ound the co mpo sition o f gr o ups: sho uld they be randomly a ssigned, how long should groups work tog ether f or, should groups be mixed in terms of achiev ement in previous cours e s, s hould gr o ups b e balanced in ter ms of a cademic background of students, should g roups b e balanced in ter ms of gender or any other characteristic? Students who feel that they hav e some say ov er how their education is managed are more likely to engage in the first place (Bovill , 2 019). 4.5 Staff r eaction There is evidence to sug gest that lecturers find the use of group- based le a rning a satisfying ex pe r ience (Jones, 1 988), a nd it is r easonable to think that this is beca use the pro ces s is a more in ter active exp er ie nc e than dida ctic teaching. Through this in teraction the course le a der may naturally find t hat they ha ve a better understanding of a student’s strengths and weaknesses, enabling them to address these issues directly . When making the transition to group- ba sed lea rning Boud and F eletti (1 997) and Sch warz et al. (2001) found that intro ducing s tuden ts and faculty members int o the new curr iculum, as opp osed to s imply starting it witho ut int ro ductory sessions, help ed in its success ful adoption. 5 Discussion The imp or tance of quality o f teaching in the UK Higher Education sector is em- phasised by the introduction of the T ea ching Ex c ellence and Student Outcomes F ramework (TEF, Office for St uden ts (2018)) to sit along side the Resear ch Ex- cellence F r amework (REF). The first TEF aw ar ds w ere assigned in 2 0 18, and were ev aluated for each Universit y as a whole. The second round of TEF aw ards, planned for 2021, will include sub ject/depar tment al sp ecific ass essment. Uni- 14 versities are encouraging teaching staff to mo dernise their tea ching, with fo c us on the T E F but also the National Student Sur vey r esults (Richardson, 201 3). There is mounting evidence that tra ditional lecture course s are not a s effective a s ‘active’-t yp e learning stra tegies in Science, T echnology , Engineer ing, and Math- ematics (STE M) sub jects (F reeman et al., 2014), and indeed s p e c ific evidence that PBL and TBL - as active learning metho ds - are effective. Bland (200 4) go es as far a s saying that not using such metho ds (PBL in this sp ecific case) for statistics a nd r esearch metho ds tr aining is detrimental to students, while Tinu ngki (2 0 15) highlights the im po rtance of c o mmun ication in lear ning ma th- ematics which is well a ddr essed in group-ba sed lear ning. Indeed, b oth P B L a nd TBL are ideally placed to meet the needs o f employers, who hav e o ften iden tified po or tea m working skills, po o r written communication s kills, and p o or o ral pre- sentation skills in gradua tes (Knight and Y or ke , 2003). This is a lso identified in the guidelines for undergraduate progra mmes in the closely allied discipline of data science, which recommend that ‘pro jects inv o lving group analysis and pr e- sentation should b e common througho ut the curriculum’ (V ea ux et al., 201 7). Though PBL or TBL are not the only metho ds for implementing this, similar group based metho ds a re b ecoming po pular in data science education, s ee for example C ¸ etink ay a-Rundel and Ellison (2020); Saltz and Hec k ma n (2016). There has b een an ex plosion of technological adv ances since Gelman a nd Nolan (2002) o utlined approa ches for teaching statistics . In the mo dern tea ching e nvi- ronment b oth teachers and students a re surr ounded by resourc e s which weren’t previously av ailable. F r om a student’s p ersp ective getting information has never bee n so easy , speeding up task s such as the resea r ch comp onent in PBL. F r om a teacher’s p ers p e ctive, numerous platforms (listed in App endix C) make learn- ing and interaction with a cla ss mor e mana geable, whilst s tuden t monito ring bec omes ever easier with tra cking via virtual lea rning environment s or auto - mated ma r king o f online quizzes . These fa c to rs co nt ribute to the s uccess o f group-ba s ed lear ning stra tegies. How ever, while for statis tics mo dules where there is an applied or prac tica l comp onent there is clear scop e to apply gr oup-based learning for the whole or at lea st par t of the module, it is not clear how, or e ven whether , suc h learning strategies ar e suitable for statistics modules whic h are of a more mathematical nature. In the first instance , the technical nature may make it difficult to create a tr uly ‘real-world’ pro blem, and this was als o no ted b y Nanes (2014). Secondly , highly mathematical mo dules generally r ely more heavily on students’ prior knowledge. F or s tudents whose prior knowledge isn’t strong, per sonality and motiv a tion is likely to play a large part in their success on the mo dule: a group-ba s ed lea rning module co uld be intimidating, or the gro up en viro nment may b e the key to success. Paterson et al. (2013) note tha t few ma thema tics lecturers use a gr oup-based learning approach to their teach ing. One po ssible reaso n for this is tha t rewrit- ing ex isting courses to en tirely group-ba sed learning modules in o ne fell swoop may not b e practical. It needn’t b e all- or-nothing, how ever. Introducing stu- 15 dent s to group-based lear ning slowly may b e b eneficia l (Bo ud and F eletti, 199 7; Sch warz et al., 2001). Elements of gr oup-based learning co uld b e wea ved through mo dules, for ex ample par ticular topics within a mo dule could b e taught in a group-ba s ed setting, or even just pa r ticular sessions. Care needs to b e taken, how ever, in ensuring that students know why you are doing this, and how it bene fits them, otherwise t he ris k is that studen ts w on’t eng age. Using group-bas e d learning needn’t mean using only PBL, or only TBL, how- ever. Some educators hav e exp er iment ed with co mbinin g parts of PBL a nd TBL to maximise the b enefits to students. F or exa mple, combining the p eer feedback (TBL) with an initial gro up discuss io n b efor e the pr e-reading assign- men ts (PBL), are p ossibly positive enhancements to any g roup learning strate- gies (Dolmans et a l., 2015). Online v ariants of PBL hav e also been tria lled successfully (de Jong et al., 2013). That mo difications to the traditional PBL and TBL metho ds ha ve been successful shows that these strategies are rip e for shaping to fit b oth the pra ctical constraints of the course, as w ell as the cours e conten t. W e hav e shown that implementation of PB L, TB L , or a v aria nt there of, is p o ssi- ble in the teaching of applied s tatistics. How e ver gr oup-based learning is imple- men ted, the emphasis on a more rounded student educatio n is clear . O f cour se, these a r e not the only activ e learning stra tegies. Which is t he most effective is the sub ject of its own debate, and students with differe nt learning styles may prefer differ ent teaching metho ds (Blo o m et a l., 1956), but this revie w shows that there is sco p e for group-based learning in statistics. Ac kno wledgemen ts TP w ould like to thank Sim on Allan and Dr Anne-Marie Hough to n from Lan- caster Universit y’s Postgraduate Certificate in Academic P ractice progr amme for helpful advice. TP was suppo rted by the Integrative E pidemiology Unit, which receives funding from the UK Medical Research Council and the Univer- sity of Bristol (MC UU 00011 /1 and MC U U 00011/3 ). The authors would like to thank t he t wo ano nymous referees f or their detailed and constructive comments whic h impro ved the pap er . This pre-pr int has been published o nline: Jo ne s E and Palmer T. A review of group-bas ed metho ds for tea ching s tatistics in higher e ducation. T ea ching Mathematics and its Applications: An International Jo urnal of the IMA. Pub- lished online 09-03-202 1, https ://doi .org/10 .1093/teamat/hrab002 . Author biographies Elinor J ones is an Asso ciate P rofessor (T ea ching) in the Depar tmen t of Statisti- cal Science at Universit y Colle g e London. She was aw a rded a PhD in Statistics 16 and P robability from The Universit y of Manchester in 20 09. She is interested in how to engage students in the lear ning o f statistics, particula rly through active learning strateg ies. T om Palmer is a Senior Lecturer in Biosta tistics in the MRC In tegrative Epi- demiology Unit, in Bristo l Medica l School. He was aw ar ded a PhD in Medical Statistics a nd Genetic Epidemiology from the University of Leices ter in 2 009. He is interested in different teaching metho ds and how to apply these to teaching statistics. Author con tact details Elinor Jone s , Department of Statistical Sc ie nc e , University College Lo ndon, Gow er Street, London, W C1E 6BT. Ema il: elino r.jones@ucl.a c.uk T om P a lmer, MR C Integrative Epidemiology Unit, Univ ers ity of B r istol, Oak- field House, O akfield Gr ove, Bristol, BS8 2BN. Email: tom.palmer @bristol.a c .uk. Some of this pap er was prepared whilst TP was employed in the Department o f Mathematics and Sta tis tics , La nca ster Universit y , Lancaster, LA1 4YF. References J. M. Bland. T ea ching s ta tistics to medical studen ts using problem-bas e d learning: the Australia n exper ience. BMC Me dic al Educ ation , 4:31, 2004. doi: 10.1 186/ 1472 - 69 20- 4- 31. B. Blo om, M. D. Engelhar t, E. J. F ur st, W. H. Hill, and D. R. Kra th wohl. T axonomy of e duc ational obje ctives: Handb o ok I: Co gnitive domain . David McKay Compa ny , New Y ork, US, 1956. D. Boud and G. F eletti. The chal lenge of pr oblem-b ase d le arning . Kog an Page, London, UK, 1 997. C. Bovill. Co-crea tion in lear ning a nd teaching: the c ase for a who le-class approach in higher education. Higher Ed uc ation , 79(6):1023–1 037, 2019. doi: 10.1 007/ s1073 4- 019- 0 0453- w. C. R. Boyle. A problem-based learning a pproach to teac hing biostatistics . Journal of Statistics Educ ation , 7(1), 1999. URL http:/ /jse. amstat.org/secure/v7n1/boyle.cfm . B. Braun, P . Bremser , A. M. Duv al, E. Lockwoo d, and D . White. What do e s active le a rning mean for mathematicians? N otic es of the AMS , 64(2): 124–1 29, 20 17. doi: 10.1 090/no ti1472. L. Bud´ e, T. I mbo s, M. W . J. v . d. Wiel, N. J. B r o ers, and M. P . F. Berger. The effect of dir ective tutor guidance in pro blem-based learning of sta tistics on students’ p e rceptions a nd ac hievemen t. Higher Educ ation , 57(1):2 3–36, 2009. doi: 10.10 07/s1 0734 - 008- 9130- 8. 17 R. Car ver, M. E verson, J . Gabros e k, N. Horton, R . Lo ck, M. Mo c ko, A. Rossma n, G. H. Rowell, P . V elleman, J . Witmer, and B. W o o d. Guidelines for Assessmen t and Instr uction in Sta tistics E ducation (GAISE) College Rep ort. T echnical rep ort, America n Statistical Ass o ciation, 201 6. URL https: //www .amstat.org/ASA/Education/Guidelines- for- A ssessment- and- Instr uction- in- Statist i c s - E d u c a t i o n - R e p o r t s . a s p x . Accessed 20 1 9-08 -12. M. C ¸ etink ay a-Rundel and V. Ellison. A fre s h lo o k at intro ductory data science. Journ al of St at ist ics Educ ation , 2 020. doi: 10.108 0/10 691898.2020.1804497. P ublished online: 14 Sep 2020. B. Dahl. What is the problem in problem-based learning in higher education mathematics. Eur op e an Journal of E ngine ering Educ ation , 43(1):112 –125, 2018. doi: 10.10 80/0 30437 97.2017.1320354. N. de Jo ng, D. M. L. V erstege n, F. E. S. T an, and S. J. O’Connor . A compariso n of classro o m and online async hro nous problem-based learning for students undertaking statistics training as par t of a public hea lth masters degree . A dvanc es in He alth Scienc es Educ ation , 1 8:245– 264, 2 013. doi: 10.1 007/ s1045 9- 012- 9 368- x. D. Dolmans, L. Mic haelsen, J. v an Merri¨ enbo er, and C. v an de r Vleuten. Should we cho ose betw e en problem-ba s ed learning and team-ba sed learning? No, combine the b est o f b oth w or lds! Me dic al T e acher , 37(4):354–3 59, 20 15. doi: 10.3 109/ 0142 1 59X.2014.948828. A. R. Emke, A. C. Butler, and D. P . L a rsen. Effects of team-ba sed learning on short-term and long-term retention of factual knowledge. Me dic al T e acher , 38(3):306 –311 , 2 0 16. doi: 10.3 109/ 01421 59X.2015.1034663. M. Er aut. Non-formal le a rning and tacit knowledge in professional w o rk. British Journal of Educ ational Psycholo gy , 70:11 3–13 6, 200 0. doi: 10.134 8/00 0709900158001. M. Es pey . Do es Space Matter? Classro om Des ig n a nd T eam-Ba s ed Learning. Applie d Ec onomic Persp e ctives and Policy , 30 (4):764– 775, 200 8. doi: 10.111 1/j.146 7- 9353.2008.00445.x. M. F reema n and J. Mckenzie. SP ARK, a confiden tial web–based template for self and peer ass essment o f student tea mw ork: b enefits of ev aluating a cross different sub jects. British Journal of Educ ational T e chnolo gy , 33(5):55 1–56 9, 2002. doi: 10.11 11/1 467- 8535.00291. S. F re eman, S. L. Eddy , M. McDonough, M. K . Smith, N. Okoroafor, H. Jor dt, and M. P . W enderoth. Active learning increa s es student p er formance in science, engineer ing, and ma thema tics. PNAS Pr o c e e dings of the National A c ademy of Scienc es of the Unite d States of A meric a , 111(23):841 0–84 15, 2014. doi: 10.10 73/pnas .1319 030111. 18 J. Gar field. T eaching statistics using small-gro up co oper ative learning. Journal of Statistics Ed u c ation , 1 (1 ), 19 93. doi: 1 0.1080 /106 9189 8.1993.11910455. A. Gelman a nd D. Nolan. T e aching Statistics: A b ag of trick s . O x ford Univ ersity Pr ess, New Y or k, US, 2 002. D. Gijb els, F. Doch y , P . V . den Bossche, and M. Segers. Effects of problem-based lear ning: A meta-analysis fr o m the angle o f as s essment. R eview of Educ ational R ese ar ch , 75(1):27–61, 2005. doi: 10.310 2/00 346543075001027. P . Haidet, K. K ubitz, and W . McCormack. Analysis of t he T ea m- Based Learning Literatur e: TBL Co mes of Age. Jour n al on E xc el lenc e in C ol le ge T e aching , 25:303–333 , 2 014. R. S. Hanse n. Benefits and problems with studen t t eams: Sugg e stions for improving team pro jects. Journal of Educ ation for Business , 82 (1):11–1 9, 2006. doi: 10.32 00/jo eb.8 2.1.11 - 19 . S. C. Hillmer. A problem-solving approa ch to teach ing business s tatistics. The Americ an Statistician , 50 :249–2 56, 1996 . doi: 1 0.230 7/26 8 4667. P . Hunsaker, C. Pav ett, and J . Hun saker. Increasing student-learning team effectiveness with team c harters. Journal of E duc ation for Business , 86(3): 127–1 39, 20 11. doi: 10.1 080/0 8832 323.2010.489588. T. Jaki. Recording le c tures as a s e r vice in a service course. Journal of Statistics Educ ation , 17(3):1–13 , 20 09. doi: 10.108 0/10 691898.2009.11889534. T. Jaki a nd M. Autin. Using a problem-bas ed approach to teac h sta tis tics to po stgradua te science students: A cas e study. MSOR Conne ctions , 9(2): 40–47 , 20 09. K. Jones . Inter active le arning: A guide for fa cilitators . Koga n Page, London, 1988. S. A. Kalaian and R. M. Kasim. A meta- analytic review of studies of the effectiveness o f small-gr oup learning metho ds on statistics achievemen t. Journal of Statistics Educ ation , 22(1), 2014 . do i: 10.108 0/10 691898.2014.11889691. C. P . Karpiak. Assessment o f Proble m- Based Learning in the Undergraduate Statistics Cours e. T e aching of Psycholo gy , 38(4):25 1 –254 , 2 011. doi: 10.117 7/00 98628311421322. A. Kleg eris and H. H urren. Impac t of problem-ba sed learning in a large classro o m setting: student p erception and problem-solving skills . A dvanc es in Physio lo gy Educ ation , 35(4):408–41 5, 2011. doi: 10.115 2/adv an.0 0046.20 11. 19 P . T. Knigh t a nd M. Y or ke. A s sessment, L e arning and Employability . Op en Univ ersity Pr ess, Maidenhea d, UK, 200 3. B. Kro g stie, K. Berntsen, and A. W r ˚ alsen. Adapting team-based learning in a mathematics course for computer engineer ing s tuden ts. Pr o c e e dings fr om the annual NOK O BIT c onfer enc e held at Svalb ar d the 18th–20th of Septemb er 2018 , 26(1), 2018. Lancaster Medical Sc ho ol. Problem-Ba sed Le arning Handb o o k Study G uide: T utor V ersio n. Lancas ter Universit y Medica l School Study Guide, 2016 . S. Liu a nd A. Beaujean. The effectiveness of team-ba sed learning o n aca demic outcomes: A meta-analysis. Scholarship of T e aching and Le arning in Psycholo gy , 3(1):1–14 , 20 17. doi: 10.1 0 37/s tl00000 75. T. Mann. Law s tuden t b ecomes sixth to ‘commit s uicide’ in Bristol this year, May 2 017. URL ht tps:/ /bit.l y/2mq2VqI . Accessed 2019-09-1 9. L. K. Mic haelsen, A . B. K night, and L. D. Fink. T e am-b ase d L e arning: A tr ansformative use of small gr oups in c ol le ge te aching . Stylus Publishing, 2004. K. M. N anes. A modified a pproach to team-based learning in linear algebra courses. Int ernational Journal o f Ma thematic al Educ ation in Scienc e and T e chnolo gy , 45(8):1208–12 19, 2014 . doi: 10.1 080/ 0020 7 39X.2014.920558. T. A. Nicholl and K . Lo u. A mo del for small-gr oup pr oblem-based le a rning in a larg e class facilitated b y one instr uctor. Americ an Journal of Pharmac eut ic al Educ ation , 7 6(6):117 , 2012 . doi: 1 0.568 8 /a jp e766 117. Office for Students. What is the TE F?, 2018. URL https: //www .officeforstudents.org.uk/advice- and- guidance/teaching/what- is- the- tef/ . Accessed 20 1 9-08 -12. M. Parappilly , L. Schmidt, and S. D. Ritter. Ready to lea rn physics: a team-based learning model f or firs t year university . Euro p e an Journal of Physics , 36(5 ):05505 2, 201 5. doi: 10.10 88/01 43- 0807/36/5/055052. M. Parappilly , R. J. W o odman, and S. R andhaw a. F eas ibilit y and effectiveness of different mo dels o f team-based le arning appro a ches in stemm-bas ed disciplines. R ese ar ch in Scienc e Educ ation , 2019. doi: 10.100 7/s11 165- 019- 09888- 8 . J. Paterson and J. Sneddo n. Co nversations ab o ut curriculum change: mathematical thinking and team-based lear ning in a discrete mathematics course. Intern at ional Journal of Mathematic al Educ ation in Scienc e and T e chnolo gy , 42(7):879–889 , 2011 . doi: 10.1 080/ 0020 7 39X.2011.613487. 20 J. Paterson, L. Sheryn, and J. Sneddon. Studen t r esp onses to tea m- based learning in t ertiary m athematics cour ses. Pr o c e e dings of 15th A nnual Confer en c e on R ese ar ch in U nder gr aduate Mathematics Educ ation , 2: 619–6 26, 20 13. J. T. E. Richardson. The national student s urvey and its impact on uk higher education. In M. Shah and C. S. Nair, editors, Enhancing student fe e db ack and impr ovement systems in tertiary e duc ation , n umber 5 in CAA Q uality Series, pag es 76–84 . Commiss ion for Aca demic Accr e ditation, UAE, A bu Dhabi, June 2013. URL http://o ro.op en.ac.uk/37730/ . ISSN 2227-496 0. C. Rob erts, M. Lawson, D. Newble, A. Self, a nd P . Chan. The in tr o duction of large class problem-based learning into an undergr aduate medical curriculum: a n ev a lua tion. Me dic al T e acher , 27(6):527–53 3, 200 5. doi: 10.108 0/01 421590500136352. C. J. Roseth, J . B. G arfield, and D. Ben-Zvi. Co llab oratio n in learning a nd teaching s tatistics. Journ al of St at ist ics Educ ation , 16(1), 2008. doi: 10.108 0/10 691898.2008.11889557. D. J. Rumsey . Statistical literacy as a g oal for in tro ductor y statistics courses. Journal of Statistics Educ ation , 10(3), 2002 . do i: 10.108 0/10 691898.2002.11910678. J. Saltz and R. Hec kma n. Big da ta science education: A case study of a pro ject-fo cused introductor y cours e. Themes in Scienc e and T e chnolo gy Educ ation , 8(2):85 –94, 2016 . URL https: //www .learntechlib.org/p/171521 . R. J. Sav ery . Overview of problem-bas e d learning: definitions and distinctions . Inter disciplinary Journal of Pr oblem-Base d L e arning , 1(1):9 – 20, 200 6. doi: 10.777 1/15 41- 5015.1002. B. Schloerke, J. Allaire , and B. Bo rges. le arnr: Inter active T u torials for R , 2018. URL https://C RAN.R - pr oject.org/package=learnr . R pack a g e version 0 .9.2.1. H. Schmidt. Problem based le a rning: ratio nale and desc ription. Me dic al Educ ation , 17(1 ):11–16 , 19 83. doi: 1 0.111 1 /j.1365 - 2923.1983.tb01086.x. P . Sch warz, S. Mennin, and G. W ebb, editors. Pr oblem-b ase d le arning: c ase studies, exp erienc e and pr actic e . Routledge, London, UK, 2001. K. St. Clair and L. Chihara. T eam-bas ed lear ning in a statistical litera cy class. Journal of Statistics Educ ation , 20(1), 2012 . do i: 10.108 0/10 691898.2012.11889633. M. Sweet and L. K . Michaelsen. How group dynamics r esearch can inform the theory and practice of p o stsecondar y small gro up learning. Educ ational Psycholo gy Revi ew , 19(1):31–47 , 20 07. doi: 1 0.100 7/s10 648- 006- 9035- y. 21 E. J. Theo bald, S. L. Eddy , D. Z. Grunspan, B. L. Wigg ins , and A . J. Cro we. Studen t p erception of group dynamics predicts individual p erfor mance: Comfort and e q uit y matter . PL oS One , 12(7 ), 2017. doi: 10.137 1/journa l.po ne.0181 336. G. M. Tinungki. The role of co o pe rative lear ning type team assisted individualization to impro ve the students’ mathematics c ommunication ability in the sub ject of pr obability theory . Journ al of Educ ation and Pr actic e , 6(32 ):27–31 , 20 15. R. D. D . V eaux, M. Agarwal, M. Av ere tt, B . S. Baumer, A. Bray , T. C. Bressoud, L. Bry ant , L. Z. Cheng, A. F r ancis, R. Gould, A. Y. Kim, M. Kr etchmar, Q. Lu, A. Moskol, D. Nolan, R. P elayo, S. Raleig h, R. J . Sethi, M. Sondja ja, N . Tiruv ilua mala, P . X. Uhlig, T. M. W ashingto n, C. L. W e sley , D. White, and P . Y e. Curriculum g uideline s for undergraduate progra ms in data science. Annual R eview of Statistics and Its Applic ation , 4 (1):15–30 , Mar . 201 7. doi: 10.114 6/annurev- statistics- 060116- 053930. A Example statistics PBL session: Studen t men- tal health – is there a crisis? A.1 Scenario Consider the scenario in which you ar e working as a statistician in the civil service in the Department for E ducation. Rec e nt ly ther e hav e b e en several case s of students committing suicide. F or example, six students co mmitted suicide in the 20 16/20 17 academic year in Bristol (Mann , 201 7). Ministers wan t to know if there really is a c r isis in ter ms of t he num b er of stu- dent s suffering from men ta l health problems. Y ou a re tasked with inv estig ating this issue. One minister has r ead so me epidemiolog ical re s earch and is curious as to whether this y ear’s cases are reflective of a truly increasing trend in mental health cases or whether this increase is an a nomalous spike in the data. Y o u a re tasked with preparing a short structured rep ort (no more than 5 pages) and presentation on this question. One iss ue to consider is that there is limited extra governmen t funding for your analysis: you a re not b e able to carry out a new study to in vestigate the issue . Therefore, you should addres s how y ou can overcome this limit ation. A.2 Indicativ e learning ob ject ives Statistical metho ds: • res e a rch how to estimate differen t meas ur es of a sso ciation; 22 • inv es tigate time series metho ds. Applied statistics: • res e a rch epidemiolo g ical concepts such as incidence and prev alence; • understa nd the differences b e tween different absolute and re la tive mea- sures of a sso ciatio n such as the risk differe nce and the risk ratio. Consider which measures migh t be more informative for public health policy mak- ers. Statistical prog ramming: • demonstra te how to access publicly av a ilable datasets a nd prepare these for analy sis using a soft ware of your choice; • show how to presen t complex longitudinal ana lyses gra phica lly . A.3 F or lecturers This ses sion is designed as a PBL exercise to b e run o ver a week. It is aimed at po stgradua te studen ts. A plan for the sessions could be as follows: • Monday: read and digest pro blem sheet with bra in stor ming sessio n to ident ify what issues they will need to rese a rch. • T uesday: Students per form res earch. • W ednesday: Ca tch up sess ion - the studen ts a ssess their pro gress in their PBL gr oups, and disc uss what extra to pic s they need to research. • Thursday: Students finish r esearch and prepar e their pres ent ation for F ri- day . • F riday: Studen ts present their findings (as a g roup) to the whole co hort of P B L gro ups . B Example statistics TBL session: conditional probabilit y and diagnostic tests B.1 Learning ob jectiv es By the e nd of this section, students should b e able to: • Expla in the difference b etw een the union of tw o (or mo re) even ts and their int ersection. • Calcula te the pro bability o f the union o f tw o (or mor e) e ven ts and the int ersection o f t wo (o r more ) ev e nts. 23 • Distinguish betw een independent and dep endent even ts. • Expla in intuitiv ely the idea b ehind conditional pr obability . • Use tables and tree diagrams to compute co nditional pro babilities. • Expla in the r ationale b ehind Ba yes’ theorem, and use it to compute con- ditional pr obabilities. • Compute probabilities in a range of settings. B.2 Before the session Studen ts w ork through a directed set of materials, which may include reading, watc hing instructional videos, quizzes, exerc ises, among o ther things. B.3 During t he session: multiple ch oice quiz Studen ts complete a short multiple choice quiz individually , answ ering a range of questio ns on probability whic h might include bo th theoretical questions such as ask ing studen ts to apply their judgement on whether t w o events ar e indep en- dent , throug h to computing probabilities. Once the t est is complete and submitted, students join their team and answ er the s ame multiple choice quiz. This time e a ch q uestion can b e discussed within the team and the gro up must decide on their joint final answer for each question for submission. Results a r e av ailable immediately after the team multiple choice quiz, and students ca n ar gue their c a se with the course leader if they think they deserve a higher mark than the o ne they received. In statistics, this may b e bec ause q uestions or the ch oice o f answers were p o or ly worded and th us caus e d confusion. As the results of the tes t are av aila ble immediately , the co urse leader can identify any c o mmon misco nceptions or error s. A very brief lecture follows to clar ify these. B.4 During t he session: working on the problem An example problem in this c ase could be the follo wing . Down ’s syndr ome is a genetic c ondition resulting in so me level of le arning dis- ability, with ar ound one in every 1,000 b abies b orn having the c ondition. Exp e c- tant mo thers c an opt to take a serum scr e ening test to assess the risk of having a b aby with Down ’s syndr ome. The test outc ome is either ‘p ositive’ or ‘ne gative’ for Down ’s syndr ome. As with the m ajority of me dic al tests, the test isn ’t 100% ac cur ate. F r om ex- tensive r ese ar ch, it is known that the t est is able t o dete ct Down ’s syndr ome, when the b aby has Down ’s syndr ome, in ab out 85% of c ases. Conversely , when 24 the b aby do esn ’t have Down ’s syn dr ome the test identifies this in ab out 96% of c ases. A pr e gn ant woman r e c eives a p ositive test for Down ’s syndr ome, and asks you for ad vic e on h ow lik ely the t est is t o b e c orr e ct. What ar e the chanc es that h er b aby ha s Down ’s syndr ome? P oss ible answ ers: ab out 85 .0 0%, ab out 2 .08%, ab out 0.10%, a bo ut 0.0 89% Correct answer : ab out 2 .08% This problem requires student s to iden tify the required probability from a text description, translate the given information in to appropria te probabilities, and manipulate the (indirectly) giv en probabilities in a non-trivial manner to com- pute the final probabilit y . The w o rk in volved means that it is v er y difficult to think of a w ay of splitting the work b etw een group mem b ers: each step ab ov e depe nds on information fro m the pr evious step. The three incor r ect a nswers ar e deliber ately given as ‘common misconceptions’: 85% is the usual pr o secutor’s falla cy , 0 .1% is the prev alence of Down’s s yndrome without accounting for the additional information fro m a p ositive tes t, and 0.089% represen ts the situation where the ca lculation of the probabilit y that a test is p ositive is incor rect (co mputed without taking the complement of the sp ecificity). It is us e ful that all a nswers hav e a ba s is in the num b ers given her e; if not then students w on’t a r rive at that particular a ns wer and so unless they are merely g uessing it is of no use. C Helpful app s, w ebsites, and tec hnologies This section lists some resources which ca n be used b y both lecturers and stu- dent s to mak e se s sions more interactive. • Genera l advice: – Collab oratio n of TBL practitioners (the T eam Based Learning Col- lab orative) whic h ma kes example t eaching material a v a ilable online ( http:/ /www. teamb asedlearning.org/ ). • Online p olls and quizzes: Studen ts in gro ups could use these reso urces to aid each other’s learning. F or ex ample, in P BL the note taker of the group could ma int ain the notes of the session in an in teractive Padlet pag e instead o f taking notes on a whiteb oar d. – Sli.do ht tps:/ /www. sli.do/ : website to crea te audience p o lls; – Kaho ot htt ps:// kahoot .it : website to cr eate and run o nline quizzes; – T urning Point : audience r e sp onse sys tem and po lling softw a re https: //www .turningtechnologie s.com/turningpoint ; 25 – Men timeter http s://w ww.me ntimeter.com : App and website to cre- ate interactiv e presentations; – W o ocla p ht tps:/ /www.w ooclap.com : c r eate and run online quizzes , real-time discus sion b oar ds suitable for classro om use; – Padlet htt ps:// en- gb.pa dlet.com/ : In ter active web pages with a wide ra nge of templates including note pinboar ds and word clouds. • R pac k ag e s: – There is a tas k view on CRAN listing R pa ck ages helpful for teaching Statistics, https: //CRA N.R- project.org/view=TeachingStatistics . – The Bay es ian task view als o has a sectio n devoted ex plicitly to teach- ing Bay esian Statistics. https: //CRA N.R- project.org/view=Bayesian – The lea r nr pack age ht tps:/ /rstu dio.github.io/learnr/ crea tes R tutoria ls and quizze s (Sc hlo erke et al., 2018). – The exams pack ag e h ttp:/ /www.r - exams.org/ allows a user to cr e - ate quizzes from an R script. The q uizzes can be exp orted in v arious formats, such as the xml fo rmat for a moo dle quiz which can be embedded into a mo o dle page. – The Shiny runtime ( https: //shi ny.rst udio.com/ ) pro duces web applications r unning R c o de. • Noteb o ok formats: The notebo ok formats are v aluable to students because they ca n con tain a mix of wr iting (using either markdown or L A T E X syntax), c o de, and the output of the co de. These do cuments ca n be worked on by a group in a collab ora tive environmen t pr oviding say RStudio serv er . – R Markdown Noteb o oks : RStudio ( h ttps:/ /rstu dio.com ) provide the . nb.ht ml for mat in which the cells a re a ctive within a n RStudio session. These files can also b e viewed in a web browser, at which po int the cells ar e no lo nger active but can still b e view e d. – Jupyter (formerly Ipython) noteb o oks, ht tps:/ /jupyt er.org/ : These noteb o oks allow use r s to distribute html do cuments in which the cells of the noteb o o k ex e cute ana ly ses if the user has the appropr iate ker- nel installed. If the kernel is not ins talled the cells ca nnot b e executed but the documents can still be vie wed in a bro w s er. • Pr esentation and document formats: T o ols for creating a ttr a ctive slides or do cuments a re us e ful for course lec- turers, but also fo r s tudents if their ta sks include submitting o r pres ent ing work. 26 – ioslides - creates html slides with in ter active conten t, e.g. gra phics. These can b e produce d from RMarkdown files . – Prezi http s://p rezi.c om - crea tes a ttractive presentations which don’t follow the tr a ditional slide f ormat. – Microsoft Swa y: An application to pro duce interactive repor ts and presentations. h ttps:/ /sway .office .com/my – L A T E X Beamer (the German for overhead pro jector). A p opular mo d- ern Beamer theme is the Metr op olis theme https: //git hub.com/matze/mthem e . – Overleaf https: //www .overleaf.com : Studen ts can work collab or a- tively on L A T E X do cuments (including reports and Beamer pr e senta- tions). 27