Laplace, Fourier, and stochastic diffusion

LAPLACE, FOURIER, AND STOCHASTIC DIF FUSION T. N. Narasimhan Dec 13, 2009 T. N. Narasimhan’s interest in mathematics stems from his use of diffusion equation to study water flow, chemical migration, and heat transport in soils, aquifers, and other geological systems. After obtaining a doctorate in engineering science from the University of California at Berkeley, he held jo int appointments at Berkeley in the departments of Materials Science and Engineeri ng , a nd Environmental Science, Policy and Management. He is affili ated with the Earth Sciences Division of Lawrence Berkeley National Laboratory. After three decades of serv ice, he retired in 2006. A B S T R A C T Stochastic diffusion equation, which attained prominence with Einstein’s work on Brownian motion at the beg inning of the twentieth ce ntury , was first formulated by Laplace a century earlier as part of his work on Central L imit Theorem. Between 1807 and 1811, Fourier’ s work on heat diffusion, and L aplace’s wor k on probability influenced and inspired e ach other. This brief period of intera ction between these two illustrious figures must be consider ed remarkable for its profound impac t on subsequent developments in mathematical phy sics, probability theory and pure analy sis. 1 INTRODUCTION Stochastic diffusion attained prominence w ith Albert Einstein’s 1905 paper [3] on random motion of colloidal particles in water. Central to Einstein’s contribution was an equa tion, analogous to Fourier ’s heat equation, with the dependent var iable being probability density, rather than temperature . But, the equation had bee n introduced a centur y earlier by Pierre Simon L aplace as part of his major wor k on probability theory , especially what we now call the Central L imit Theorem. During a brief four -y ear period, 1807-1811, L aplace’s path-breaking investigation of probability intersected with Joseph Fourier’s profound study of heat movement in solids. This fortunate intersection led to these fer tile minds mutually influencing and inspiring each other, re sulting in the creation of modern mathema tical phy sics. Although L aplace wa s a phy sicist at heart, his pioneering work on stochastic diffusion was of abstract, mathe matical nature, while Fourier’s w ork was devoted to bring ing a quantitatively observable phy sical process within the folds of rig orous mathematics. Philosophically , this dichotomy of diffusion, P a g e 1 o f 1 7 unify ing the abstrac t and the observable is noteworthy , as has been pointed out by Narasimhan [20]. Indeed, F ourier himself was struck by this fascinating consilience. I t is fitting to start with Fourier’s ow n words. I n “Preliminary Discourse” of Analy tical Theory of Heat, Four ier [7, p. 7] observed , “ W e see, 1 for example, that the same expression whose abstract properties geomete rs had considered, and which in this respect belongs to general analysis, represents as well the motion of light in the atmosphere, as it determines the laws of diffusion of heat in solid matter, and enters into all the chief problems of the theory of probability. ” Clearly , Fourier was aware tha t his theory of heat and the theory of probability had mathema tical connections. I f so, what is the “ qu’une m ême expression” that impressed Fourier? What metaphors connect the observa ble and the abstract? This paper explores these questions. Fourier’s 1807 ma sterpiece disappeare d after his death in 1830, and was discovere d more than fifty y ears later by Gaston Darboux. But, until the publication of a detailed ac count of this work in 1972 by Grattan-Guinness [8], it went largely unnoticed, and Fourier’ s 1822 book [6,7] was the principal source of information on his contribution. However, Gra ttan-Guinness’ work broug ht to light many important developments that occurred be tween 1807 and 1822 relating to ac ceptance of Four ier’s work by the leadership of French mathematics, including L aplace. Mathematical aspe cts of these developments ha ve since been addressed by historians Herivel [12], Bru [2], and Gillespie [9] . Additionally , Hald [11] gives a comprehensive ac count of Laplac e’s work on probability leading to the publication of his masterpiece Théorie analytique des probabilites [17]. The present work c omplements these contributions by focusing attention on the intera ctions between Fourier a nd Laplac e between 1807 and the publica tion of Fourier’s book in 1822. The perspec tives presented seek a comparative understanding of the beginnings of phy sical and stochastic diffusion. I n a modern sense, “stochastic diffusion” implies uncertainty associated with random proc esses. Extension of probability theory to random variables was pioneere d during the middle of the nineteenth century by Aug ustine Cournot who proposed a practical theory of random variables [2]. During a better part of the 18 century , probability was associated with mathema tics of th O n v o i t , p a r e x e m p l e , q u ’ u n e m ê m e e x p r e s s i o n , d o n t l e s g é o m è t r e s a v a i e n t c o n s i d é r é l e s p r o p r ié t é s 1 a b s t r a i t e s e t q u i , s o u s c e r a p p o r t , a p p a r t i e n t à l ’ A n a l y s e g é n é r a l e , r e p r e s é n t e a u s s i l e m o u v e m e n t d e l a l u m i è r e d a n s l ’a t m o s p h è r e , q u ’e l l e d é t e r m i n e l e s l o i s d e l a d i ff u s i o n d e l a c h a l e u r d a n s l a m a ti è r e s o l i d e , e t q u ’ e l l e e n t r e d a n s t o u t e s l e s q u e s ti o n s p r i n c i p a l e s d e l a T h é o r i e d e s p r o b a b i l i t é s [ F o u r i e r , 6 , p . x x i i i ] P a g e 2 o f 1 7 actual ga mes, theory of risks, and thought experiments involving hy pothetical urns. In a pape r presented to the Fr ench Academy on March 10, 1773, Laplac e made a bold departure, a nd applied probability to celestial mechanics for study ing causes of events. L ater, he extended probability to the problem of correcting instrumental error in phy sical observations [9]. Thus, L aplace’s work was de voted to investigation of theory of errors, rather than formal study of random variables. I t is worth noting that the phrase Central L imit Theorem came to be established only during the twentieth ce ntury , at the sugg estion of Polyà [21, 23]. 2 CLUES TO CONNECTIONS BETWE EN DIFFUSION AND P ROBABILITY Spreading of hea t in a solid is governed by combined effects of thermal conductivity and thermal capacity of a solid material, as heat is driven from a loc ation of higher to one of lower temperature. The rmal conductivity is a proportionality constant linking heat flux with temperature g radient, and thermal capa city , first defined and measured by Lavoisier a nd Laplac e [19], quantifies the relationship between magnitude of temper ature and quantity of heat stored in a body . Spreading of proba bility represents increase in uncertainty in cumulative er ror of a finite number of random variables in proportion to number of variables summed up. Fourier’s [4] parabolic equa tion for heat diffusion is, (1) where K is thermal conductivity , T, temperature, C, thermal capa city , x, distance along the abscissa, and t, time. A diffusion problem is fully defined when (1) is augmented by appropriate boundary and initial conditions. I n his 1807 monograph, Fourier de voted attention exclusively to bounded, sy mmetrical solids [rod, prism, sphere, cube, ring ]. Assuming K and C to be independent of tempera ture, and thus linearizing the differe ntial equation, he pioneered a number of novel mathematical techniques for solving it. I n stochastic diffusion, spreading is quantified by increase in varianc e of cumulative err or distribution in direct proportion to number of samples. The Central L imit Theorem is a P a g e 3 o f 1 7 1 2 n mathematical elabora tion of this fact. Let x , x , ......... x be independently and identically distributed random variables with fre quency function f(x), mean ì , and var iance ó . The Central 2 n 1 2 n L imit Theorem states that s = x + x , + ........+ x asy mptotically approaches normal distribution with mean n ì and variance n ó [11]. 2 Comparison of the two phenomena shows that time in the thermal proce ss is analogous to number of samples in error propag ation. However, the e rror propag ation problem has no feature analogous to the bounding sur face of a solid in the heat flow problem. Consequently , to understand how the “same expression which dete rmines the laws of heat diffusion in solid matter enters into all the chief problems of the theor y of probability ”, it is necessary to examine how Fourier posed a nd solved heat flow problems in infinite media. L ooking care fully at Fourier’ s Analy tical Theory of Heat [6,7] from this perspective, the connection is readily found in Article 364, which deals with transient heat flow a long an infinite line, over a seg ment of which arbitrar y initial conditions are prescribed, with zero temper ature elsewhe re. 3 LAPLACE (1809), AND F OURIER (1811) L aplace, who had actively worked on probability theory for over fifteen y ears from 1771, devoted most of his attention over the next twenty y ears to the study of planetary mechanics, culminating in the publication of Traité de mecaniqué céleste betwee n 1799 and 1805. He returned to probability thereafter, and pr ovided the first proof of the Centra l Limit Theorem in 1810 for variables with a continuous uniform distribution [11]. Leading up to this work, in 1809, L aplace [14] investigated the use of génér atrice functions to evaluate the proba bility that the sum of a given number of identically distributed random variables would take on a g iven value. I t was well known through ear lier investigations of James Bernoulli, Abra ham de Moivre and others that the required probability constituted the coefficient of a particular te rm in a power series stemming from the génér atrice function. The dif ficulty was that estimating the numerical mag nitude of the coefficient wa s mathematically very difficult when the number of r andom variables summed up became ver y large. Joseph L agrang e and L aplace devote d much of their energ ies to evaluating the required coef ficients using finite differenc e equations and associated definite integ rals. I n the subsection entitled, “On definite integrals of pa rtial differenc e equations” , L aplace [14, p. 2 235] started with the study of linear second-order partial differential equations whose solution S u r l e s I n té g r a l e s d é f i n e s d e s É q u a t i o n s à d if f é r e n c e s p a r t i e l l e s 2 P a g e 4 o f 1 7 involved two arbitrary functions. In a spec ial case, this genera l problem reduced to one whose solution involved but a single arbitrary function. This was a parabolic equation of the form, in L aplace’s notation [14, p. 238], (2) He then showed how the e stimation of the coefficient of a particula r term in a power series expansion could be transformed to finding a solution to the aforesaid equation. x,x’ L et u be a power series e xpansion in t and t’, with y being the coe fficient of (t t’ .) L et u be x. x’ x,x’ x,x ’ the génér atrice function for y , and u[[1/t - 1] - [1/t’ - 1] ] be the génér atrice function for Ä y 2 2 x,x ’ - Ä ’y , the characteristic Ä being rela tive to x and Ä ’ being relative to x’. The g oal was to solve x,x ’ for the coeff icient y , when the number of terms to be considered in the power series are larg e. To this end, L aplace used recur sive relations to arrive at the finite diffe rence equa tion, ( 3 ) I n the infinitesimal limit, this l ed to the differential equation [in L aplace’s notation], ( 4 ) Based on his ear lier work, L aplace then demonstrated that, (5) x,x ’ satisfied the partial differential e quation (4), where ö is any arbitrary function. Here, y represents the probability that the sum of x’ identically distributed random var iables takes on the P a g e 5 o f 1 7 value x. Comparing with the heat equation, probability y corresponds to temperature, the magnitude of the sum of ra ndom variables, x, corresponds to distance x, and the number of x,0 random variables, x’, corresponds to time. The coe fficient, y , represent initial conditions. The form of L aplace’s solution immediately revealed to Fourier that he could see k integral solutions to the heat equation in addition to the series solutions. Thus inspired, Fourier [5] expanded his 1807 work, and filed it with the I nstitut de France on September 28, 1811 in response to the prize competition it had set up. The significant a ddition in this expansion was Chapter XI on the linear movement and va riation of heat in a body with one infinite dimension. For the first time, Fourier a ddressed the problem of heat movement in a solid without a bounding surface. Problems of this ty pe are driven solely by initial conditions. I n particular, Fourier c onsidered an infinite line with - 4 < x <+ 4 . At time t = 0, the temperature every where along this line was zero, except over a seg ment extending on either side of x = 0. Over the seg ment, temperature distribution was an a rbitrarily prescribed function f(x). He considered sever al cases with f(x) representing different patterns of temper ature variation over the segment. The g overning differ ential equation was, (6) with initial condition f(x). To solve for u(x,t), he sought solutions in three differe nt forms, two of them involving convolution integrals with the hea t kernel, (7a) and (7b) P a g e 6 o f 1 7 (7c) Fourier we nt on to show through a series of transfor mations that (7c) y ielded solution of the form, (8) Note that , (8) has the same form as L aplace’s solution (5) to the probability problem (4). Although Fourier [5] did not refer to L aplace’s work in the Prize Essay , he acknowledg ed in Fourier [7, Art. 364] , “ This integral which contains one arbitrary function was not known when 3 we had undertaken our researche s on the theory of heat, which were transmitted to the Institute of France in the month of December, 1807: it has bee n given by M. Laplace, in a work which forms part of Volume VIII of the Mémoires de l’Éc ole Polytechnique; we apply it simply to the determination of the linear movement of heat ”. With some modifications and chang e of sy mbols these results were pr esented in Fourier [6,7] as Chapter I X, Section I . His conclusion at the end of this section was that solutions to equation (6) arrived a t through different forms [e.g . (7a), (7b), (8)] were equivalent. Following his 1809 contribution, L aplace chose to pursue proof of the Central L imit Theorem using chara cteristic functions rather than the diffe rential equation, and announc ed his result to the Academy in April 1810 [2,22] . Soon thereafter, he found that the a ppearance of Carl Friedrich Gauss’ rece ntly published work on the method of least squares had clear ly shown the connection between Central L imit Theorem and linear estimation [22]. The following y ear he gener alized a two-urn problem of Bernoulli and for mulated a model involving what is now refe rred to as Markov Chain with transition probabilities. This model led to a second order partial diff erential equation, C e t t e i n t é g r a l e , q u i c o n t i e n t u n e f o n c t i o n a r b i t r a i r e , n ’é t a i t p o i n t c o n n u e lo r s q u e n o u s a v o n s e n t r e p r i s n o s 3 r e c h e r c h e s s u r l a T h é o r i e d e l a c h a l e u r , q u i o n t é t é r e m i s e s à l ’ I n s t i t u t d e F r a n c e d a n s l e m o i s d e d é c e m b r e 1 8 0 7 ; e l l e a é t é d o n n é e p a r M . L a p l a c e , d a n s u n O u v r a g e q u i f a i t e p a r t i e d u T o m e V I I I d u J o u r n a l d e l ’ É c o l e P o l y te c h n i q u e [ ] ; n o u s n e fa i s o n s q u e l ’ a p p l i q u e r à l a d é t e r m i n a t i o n d u m o u v e m e n t l i n é a i r e d e l a c h a l e u r . [ F o u r i e r , 6 , p . 4 1 4 ] 1 P a g e 7 o f 1 7 (9) His solution to this problem involved polynomials, which would subsequently be recog nized as being proportional to Her mite polynomials [11]. The solution anticipated Fourier- Hermite series for functions defined ove r infinite domains. 4 THE EXPRESSION I n Analy tical Theory of Heat , Fourier [6,7] considered heat movement in infinite solids. He started with as satisfy ing the differ ential equation, , and showed that (10) Consequently , the aforesa id differential equation is satisfied by , (11) where á is any constant. I f we let (x - á ) /4t = q , then, 2 2 P a g e 8 o f 1 7 (12) I f, in (11), we set t = ó , then, is the probability density function for norma l 2 distribution with mean á and var iance ó . Thus, Fourier established that probability density 2 function which play s a fundamental role in probability theory , also forms part of solutions fundamental to transient heat diffusion in infinite media. 5 MOVEMENT OF LIGHT IN THE ATMOSPHERE Now we consider F ourier’s [7, p. 7] reference to, “ ... the same expression .... represents as well the motion of light in the atmosphere, ...” . Presumably, Four ier was referr ing to L aplace’s work, 4 “ Mémoire sur les mouve ments de la lumière dans les milieux diaphanes ”, re ad before the Académie in 1808, and published in 1810 [16]. A major part of this work was devoted by Laplac e to developing a theory for transmission of light in transparent media, including the atmosphere, based on the philosophy of action-at-distance. L aplace believed so strong ly in this philosophy [10, p. 93] that he extended it, by analogy with light propag ation in the atmosphere, to a discussion of the movement of heat in solids. This he did in a long “Note”. T he central concept in this approac h was that of interacting molecules of light, or analog ously , of heat. L aplace prefac ed his discussion of heat propag ation with the statement, “ By considering action-at-distance of molec ule and molecule, and extending such action to heat, we arrive, through a simple and precise way, at the true differential equations that “ . . . . . q u ’ u n e m ê m e e x p r e s s i o n , . . . . . r e p r é s e n t e a u s s i l e m o u v e m e n t d e l a l u m i è r e d a n s l ’a t m o s p h e r e , . . . ” 4 [ F o u r i e r , 6 , p . x x i i i ] P a g e 9 o f 1 7 describe heat movem ent in solid bodies and its variations on their surf ace, and thus this very important branch of physics enters in the area of Analysis” . 5 For L aplace, ac tion-at-distance in reg ard to heat was embodied in Newton’s pr inciple that the quantity of heat communicated by a body to its neighbor is proportional the difference in their temperatures. B ased on this, he then presented the par tial differential equation for the flow of heat in a solid by analogy with similar derivation for the propagation of lig ht [16, p. 293; Laplac e’s notation], (13) where the constant a is thermal c onductivity . He then observed that this equation can be gener alized to three spatial dimensions. I t is not clear if this Note was prepared in 1808 whe n the paper was prese nted before the I nstitut , or it was prepare d in 1810. Regardless, it is clear that L aplace implicitly conce ded Fourier’ s priority in presenting the par abolic equation. However , his desire seems to be one of providing a better way of deriving the equation than what had been a chieved by Fourier. This is evident in his assertion, “ However, just like mathematicians arrived at the e quations describing the movement of light in atmosphere starting from an inaccurate hypothesis, the hypothesis that the action of heat is limited to contact area can lead to the equations describing heat move ment inside and at the surface of bodies. I need to take note that M. Fourier already arrived at these equations, the real bases of which seem to be those I just presented. ” 6 E n f i n l a c o n s i d é r a t i o n d e s a c t io n s a d d i s t a n s d e m o l é c u l e à m o lé c u l e , é t e n d u e à l a c h a l e u r , c o n d u i t d 'u n e 5 m a n i è r e c l a i r e e t p r é c i s e a u x v é r i t a b l e s é q u a t i o n s d i f f é r e n t i e l l e s d u m o u v e m e n t d e l a c h a l e u r d a n s l e s c o r p s s o li d e s e t d e s e s v a r i a t i o n s à l e u r s u r f a c e , e t p a r l à c e t t e b r a n c h e t r e s im p o r t a n t d e l a P h y s i q u e r e n t r e d a n s l e d o m a i n e d e l 'A n a l y s e [ 1 6 , p . 2 9 0 ] . M a i s , d e m e m e q u e l e s g e o m e t r e s a v a i e n t é t é c o n d u i t s a u x e q u a t i o n s d u m o u v e m e n t d e l a l u m i è r e d a n s 6 l ’a t m o s p h è r e , e n p a r t a n t d ’ u n e s u p p o s i t i o n i n e x a c t e , d e m ê m e l ’ h y p o t h è s e d e l ’ a c t io n d e l a c h a l e u r l i m i t é e a u c o n t a c t p e u t c o n d u i r e a u x é q u a t i o n s d u m o u v e m e n t d e l a c h a l e u r d a n s l ’ i n t é r i e u r e t à l a s u r f a c e d e s c o r p s . J e d o i s o b s e r v e r P a g e 1 0 o f 1 7 From the foreg oing, there is little doubt that Fourier’s statement, “ .... represénte aussi le mouvement de la lumière dans l’atmosphère... ..” refer s to Laplac e [16]. Given that, it is pertinent to examine how Fourier approache d the derivation of the same he at equation. As has been described in detail by Grattan-Guinness [8], and Herivel [12], Fourier began his investiga tion of heat around 1804, starting with action-at-distance and Newton’ s principle. I n this, he followed the same line of rea soning as Biot [1] before him. However, he e ncountered difficulties in formally setting up a differ ential equation. Consequently , he abandoned action-at-distance, and introduced the continuity assumption that the state of heat at a point depends solely on the immediately preceding point. This approach essentially introduced the notion of a continuum. That this approach has withstood the test of time sug gests that L aplace’s claim that his method provides the “rea l base” of the heat equation does not carr y convinction. 6 PERSONAL INTERACTIONS As we have see n, the period 1807 - 1811 was remarkable in the history of mathematical statistics and mathematical phy sics. On the human side, this period was distinguished by an initial rivalry and subsequent rapproc hement between two intense individuals who were r evolutionizing science. What was the nature of this per sonal interaction? There is little doubt that Fourier wa s the first to formulate the parabolic equation in 1807. Although he had experimented f or three deca des with a variety of transforms (including what would later be termed a s Fourier transform) to solve diffe rence equa tions and differential equations, L aplace apparently did not recognize that the para bolic equation would help evaluate not only the mean but also the variance as w ell of the sum of a larg e number of random variable s [22]. His 1809 formulation was clearly catalyzed by Fourier’s 1807 monograph. O n his part, Fourier wa s inspired by Laplac e’s 1809 formulation of the parabolic equa tion to recog nize that heat flow in infinite domains constituted a new class of problems, and that solutions to the heat equation can also be obtaine d in the form of integrals. Clearly , physical diffusion and stoc hastic diffusion had mutually influenced eac h other at birth. q u e " M . F o u r i e r e s t d é j à p a r v e n u à c e s é q u a t i o n s , d o n t l e s v é r i t a b l e s f o n d e m e n t s m e p a r a i s s e n t ê t r e c e u x q u e j e v i e n s d e p r é s e n t e r . [ 1 6 , p . 2 9 5 ] P a g e 1 1 o f 1 7 During the per iod 1807 to 1811, when both Laplac e and Fourier were intensely addressing their respective topics, there was some tension between them. While L aplace [14], in his 1809 work, did not acknowledge F ourier’s 1807 work, F ourier [5] failed to cite Laplac e’s work in his Prize Essay . L aplace [16, p. 295] conceded that Fourier ha d already presented the heat equation, but asserted that his derivation based on action-a t-distance was more f undamental than Fourier’s derivation. The tension gradua lly gave w ay to mutual respect when Fourier spe nt nearly a year in Paris, starting from the summer of 1809. During this stay , Fourier re gularly attended meetings in L aplace’s estate at Ar cueil, the uncontested center of w orld science at that time [2,8]. Thus, although Fourier [6, p. xxiii] did not specifically mention L aplace, it is clear that he was referr ing to Laplac e in stating, “... qu’une m ème expre ssion, don’t les géométres avaient considéré les propriétés et qui, sous ce rapport, appartient à l’Analyse générale,... ”. F or his part, L aplace [18, p. 83] reproduced the hea t equation and the equation at the boundary and complimented Fourier by stating, “ ... M. Fourier was the first to present the fundamental equations (1) and (2) in the excellent paper that won the prize proposed by the I nstitute on the Theory of Heat; I shall give their demonstration in a different book.” Presumably , “j’en donnerai la démonstration dans un 7 autre livre ” refer s to Laplac e [16, p. 295]. As cited at the beg inning of this paper, Fourier w as impressed by the fact that the “sa me expression” which determines the laws of diffusion of he at also enters into all “chief problems of the theory of probability ”. What “same expression” was Fourier fascina ted about? From what we have seen, there ar e three possibilities. The first is the convolution integral, which he borrowed from L aplace’s 1809 paper. The sec ond is the expression, which is essentially the same as the probability density for nor mal distribution. The third is the parabolic equation itself, in view of L aplace’s work on propag ation of light in the atmosphere. M . F o u r i e r a d o n n é l e p r e m i e r l e s é q u a t i o n s f o n d a m e n t a l e s ( 1 ) a n d ( 2 ) d a n s l ’ e x c e l l e n t e p i è c e q u i a 7 r e m p o r t é l e p r i x p r o p o s é p a r l ’ I n s t i t u t s u r l a T h é o r i e d e l a c h a l e u r ; j ’ e n d o n n e r a i l a d é m o n s t r a t i o n d a n s u n a u t r e l i v r e [ L a p l a c e , 1 8 2 3 , p . ? ? ] P a g e 1 2 o f 1 7 7 CONCEPT OF A F UNCTION AND P HYSICAL IMPLICATIONS To L agrang e, Fourier’ s statement that an arbitrary function could be expressed as a trigonometric series was so unexpected that he opposed it strong ly . I n a recent paper, K ahane [13] presents new evidence on L agrang e’s errone ous criticism, based on a “Schriftstück” of L agrange mentioned in Bernhar d Riemann’s Habilitation dissertation. As Kahane shows, L agrange’s c riticism was an indication that the concept of a function, which w as a source of controve rsy among Jean d’Alembert, L eonhard Euler, a nd Daniel Bernoulli during the 18 century , was still evolving th around 1800. I ndeed, Fourier’ s work inspired Augustine Cauchy , Lejeune Dirichlet and Riemann to continue to refine the conc ept of a function to pave the way for Georg Cantor and other s to lay the foundations of modern theory of functions of a real var iable. L agrang e’s criticism stemmed from the fact that he took a tr igonometric series define d over 0 < x < ð /2 and showed that it led to an inconsistent result when x was set to 0. I n response, Fourier had pointed out that equations of certain ty pe cannot be used without specify ing the limits between which the va lues of the variable have to be c onsidered. But L agrang e did not relent. Fourier’s c ontribution to the theory of functions was to establish that a function is only valid over a specified domain. Against this backg round it is worthwhile to examine the phy sical implications of functions and their domains. Whereas Fourie r’s 1807 monograph e stablished that any function defined over a bounded domain could be represente d by trigonometric ser ies, Laplac e’s solution of a gener al second-order partial diff erential equation (9) in ter ms poly nomials anticipated the later development showing that an ar bitrary function over an infinite domain could be represented using Fourier- Hermite series. Thus, finite and infinite domains fall into distinct categ ories in terms of represe ntative functions. Finite and infinite domains also relate to distinct categ ories of phy sical problems. In a tra nsient sy stem, heat flow is driven by non-uniform spatial distribution of temperature at the initial time, or by external influences acting on the bounding surfaces of the sy stem, or both. In infinite, unbounded solid bodies, the initial condition is the sole cause of heat flow. The self-smoothing tendency of the system is to dissipate disturbances by itself, without any external influence. Here, the fundamental problem of interest in an infinite sy stem is the release of a cer tain amount of heat P a g e 1 3 o f 1 7 in the vicinity of a point at time zero, and to predict the spre ading (diffusion) of heat f or t > 0. This is referred to as an instantane ous source. All other problems pe rtaining to an infinite sy stem can be solved by superposition of this fundamental solution using convolution integrals. The fundamental problem is inherently symmetrical. L aplace’s stochastic diffusion problem is also an initial value problem. Given a probability density distribution for n = 0, n being analogous to time, one solves for the spreading of probability density as n becomes prog ressively larg e. I n the process, no restrictions are plac ed on the randomness of the values sampled. Time in the phy sical problem and number of samples in the stochastic problem a re unbounded and tend to infinity . Therefore , in problems involving infinite domains, the time derivative, or equivalently , the derivative with refe rence to number of samples n has to be non- zero. An equilibrium state is not theoretically definable. I t is interesting that recursive formulas and diff erence e quations play ed a very important role in many of probability problems solved by Laplace. L aplace has r emarked [11, p. 338] that if the initial distribution were known, all subsequent distributions can be calculate d with the help of the recursive for mula. This remark reinforces the view that evolution in time (or, equivalently , number of samples) is centra l to stochastic diffusion. I n contrast, boundary -value problems ar e driven by forces imposed on the boundar y by external causes. Under time-invariant bounda ry conditions, the sy stem is driven to steady state flow characte rized by vanishing time derivative. The solution satisfies Dirichlet Principle, an integ ral that has to be minimized. The unique solution is independent of any initial condition that may have existed at the beg inning. I n stochastic diffusion, spreading continues with progressively increasing number of samples as long as random sampling continues unfettered by external influence. Therefore, if any bounds on are set on the value of the sum of the r andom variables, then ra ndomness is inhibited by external causes. I f sampling is continued under bounde d conditions, sy stem progress will be influenc ed more and more by boundary conditions. Thus, problems involving infinite domains and finite domains constitute two distinct classes. Mathematically , the behavior of the latter can be descr ibed using trigonometric ser ies, while the former can be described using Her mite poly nomials. Initial-boundary value problems may be P a g e 1 4 o f 1 7 considered to be mixed problems, combining fe atures of both. Semantically , it is i nteresting to note that in Analy tic Theory of Heat, Fourier [6,7] uses the word “diffusion” in the heading of Chapter I X devoted to infinite media. This is eminently rea sonable because “diffusion” or “spreading ” can occur only in infinite domains where no external forces inhibit spreading . Fourier himself notice d this difference between pr oblems defined over finite and infinite domains when he [7, Article 343] stated , “ In the problems we previously discussed, the integral is 8 subjected to a third condition which depends on the state of the surface: for which reason the analysis is more complex, and the solution requires the employm ent of exponential terms. The form of the integral is very much more simple, when it nee d only satisfy the initial state;...” 8 CONCLUDING REMARK L aplace and Fourier w ere natural philosophers seeking to comprehend a finite wor ld subject to errors of discrete obse rvations. Their differ ence equations and r ecursive relations were use less when the number of observations wer e larg e. To overcome this difficulty , their creative intellects led them from differe nce equations to differential equations and a host of definite integrals and converg ent algebra ic series. Yet, the observational world re mains finite and discrete, and the algebra ic expressions are but idealized approximations of reality . We continue to grapple with balancing the discrete and the continuous. An intriguing question emerges: if a dig ital computer had been available to Lag range, L aplace, and Fourier to handle larg e numbers, what cour se would mathematics have taken? 9 ACKNOWLEDGMENTS I am very grate ful to Jean-Pierre Kahane for many illuminating discussions on history of mathematics in France as w ell as for deep insig hts into the theory of functions and its relation to mathematical phy sics. I thank Ghislain de Marsily , and Roger Hahn for criticisms and valuable suggestions. Norber t Schappacher’ s critical review he lped greatly in revising the paper. Over the D a n s l e s q u e s t i o n s q u e n o u s a v o n s t r a i t é e s p r é c é d e m m e n t , l ’ i n te g r a l e e s t a s s u j e t t i e à u n e t r o i s i è m e 8 c o n d i t i o n q u i d é p e n d d e l ’ é t a t d e l a s u r f a c e . C ’ e s t p o u r c e t t e r a i s o n q u e l ’ a n a l y s e e n e s t p l u s c o m p o s é e e t q u e l a s o l u t i o n e x i g e l ’ e m p l o i d e s t e r m e s e x p o n e n t i e l s . L a f o r m e d e l ’ i n t e g r a l e e s t b e a u c o u p p l u s s i m p l e l o r s q u ’ e l l e d o i t s e u l e m e n t s a t i s f a i r e à l ’ e t a t i n i t i a l , . . . . [ F o u r i e r , 6 , p . 3 8 8 ] . P a g e 1 5 o f 1 7 past decade, I have had many valuable exchanges with Stephen Stigler on the history of mathematical statistics. These exchang es have been of help in prepa ring this paper. REFERENCES [1] Biot, J. B.; Mémoire sur la chaleur , Bibliotheque Britannique , 27, 310-329, 1804. [2] Bru, B.; Poisson, the probability calculus, and public educ ation. English translation of an article published in 1981 by Glenn Shafer, Journ@l Electroniqued’Histoire des Probablités et de la Statistique , 1 (2), 25 p., November 2005. [3] Einstein, A.; Über die von der molekularkinetischen F lussigkeiten suspendierten Teilchen. Ann. Der Phys. , 17 (1905), 549- 560, 1905. [English translation in Investigations on the theory of Brownian movement by Albert Einstein, Edited with notes by R. Fürth, pp. 1-35, Methuen, L ondon, 1926] [4] Fourier, J. B. J.; Mémoire sur la propagation de la chaleur . Read before the I nstitut on December 20, 1807. Unpublished, and pre served in MS 1851 of the École Na tionale des Ponts et Chaussées, Paris. Full text with commentary by I . Grattan-Guinness [8]. [5] Fourier, J. B. J.; Théorie du mouvement de la chaleur dans les corps solides, The Prize Essay , deposited with the I nstitut on September 28, 1811. Published in 1824 as Mémoires l’Académie Royale des Sc iences de l’Institut de France , years 1819 and 1820 , 185-556, with one Plate containing 13 fig ures. [6] Fourier, J. B. J.; Théorie Analytique de la Chaleur . C. F. Didot, Paris, 563 p., 1822a. [7] Fourier, J. B. J.; Analytical Theory of Heat , 1822b. Translated with notes by A. Freeman, the Cambridge Univer sity Press, L ondon, 1878, 466 pages. [8] Grattan-Guinness, I ., in collaboration with J. R. Ravetz, ; Joseph Fourier, 1768-1830 , The M.I .T. Press, Cambridge, Mass., 516 p., 1972. [9] Gillespie, C. C., Pierre Simon Laplace 1749-1827, A life in Exact Science , Princeton University Press, Princeton New Jersey , 322 p.,1997. [10] Hahn, R.; Pierre Simon Laplace 1749-1827, A Determined Scie ntist , Harvard University Press, Cambridge, Mass., 310 p., 2005. [11] Hald, Anders.; A History of Mathematical Statistics from 1750 to 1930 . John Wiley and Sons, New York, 793 p., 1998. [12] Herivel, J.; Joseph Fourier, The Man and the Physicist , Clarendon Press, Oxford, 1975. P a g e 1 6 o f 1 7 [13] Kahane, J-P., Partial differential equations, trig onometric series, and the c oncept of a function around 1800: a brief story about Lag rang e and Fourier, Proc. Sym posia in Pure Math. , 79, 187-205, 2008. [14] L aplace, P. S.; Mémoire sur divers points d’Analy se, Journal de l’Écolé Polytechnique , Tome VI I I., 229-265, 1809, Oe uvres Completes, XI V, 178-214. [15] L aplace, P. S.; Mémoires sur les approximations des formules qui sont fonctions de très grands nombre s et sur leur application aux probabilités, Mémoires l’Acadé mie Royale des Sciences , Paris, 353-415, 1810, Oe uvres Completes, XI I , 301-353. [16] L aplace, P. S.; Mémoire sur les mouvements de la lumière da ns les milieux diaphanes, Mémoires de l’Académie de s Sciences, I Série, Tome X, 1810, Oeuvres Completes de st Laplace , Vol 12, 267-298. [17] Laplace, P. S.; Théorie analytique des probabilites . V Courcier , Paris, 1812, 1814, 506 p. [18] L aplace, P.S.; De la chaleur de la Terre et de ladiminution se la durée du jour par son refroidissement, 1823, Oeuv res Completes de Laplace , Vol 5, p. 83. [19] L avoisier, A. L . and Laplac e, P. S.; Mémoire sur la Chaleur, Mém oires de l’Académie Royale des Sciences , Paris, for the y ear 1780, 355-408. Paper read June 28, 1783. [20] Narasimhan, T. N.; On the dichotomous history of diffusion, Physics Today , 62(7), 48-53, 2009. [21] Poly à, G.; Uber den zentralen Gre nzwertsatz der Wahrscheinlihkeitsrechung und das Momentenproblem , Math. Zeit. , 8, 171-181, 1920. [22] Stigler, S. M.; The History of Statistics , Harvard University Press, 410 p., 1986. [23] Stigler, S. M.; Personal communication, 2008. T. N. Narasimhan Department of Materials Science and Eng ineering 210 Hearst Memorial Mining Building University of California Berkeley , Ca 94720-1760 Email: tnnarasimhan@L BL .gov P a g e 1 7 o f 1 7