Use of convexity in contour detection

Use of con v exit y in con tour detection Victor Ch urchill Departmen t of Mathematics, Dartmouth College Ma y 31, 2019 Abstract In this pap er, we formulate a simple algorithm that detects contours around a region of interest in an image. After an initial smo othing, the metho d is based on viewing an image as a top ographic surface and finding con vex and/or conca ve regions using simple calculus-based testing. The algorithm can achiev e multi-scale con tour detection by altering the initial smoothing. W e sho w that the metho d has promise by comparing results on sev eral images with the watershed transform performed on the gradient images. 1 In tro duction In its first application, the watershed transform was performed on the gradient mo dulus of an image in order to detect con tours around catchmen t basins, [3]. In this pap er, a simple metho d for detecting similar con tours is presen ted. In particular, contours at the b oundary of conv ex and conca ve regions in the image are detected. This is achiev ed by lo cating zero crossings of Gaussian curv ature in the image when view ed as a top ographic surface. T o identify depressions in the top ograph y , con vex regions of the surface are located. T o iden tify protrusions, concav e regions are lo cated. In the prop osed metho d these tw o region types can be view ed separately or together dep ending on the application. The pap er begins b y considering con vex regions in differentiable functions. In general, a twice differen tiable function of n v ariables f is con v ex at a point if and only if Hessian matrix is positive semidefinite at that p oin t, [5]. Similarly f is conca ve if and only if the Hessian matrix is negativ e semidefinite. This condition can b e in terpreted geometrically as the requirement that f has positive (up ward or down ward, resp ectively) Gaussian curv ature at each p oin t x . If we w ere considering b oth con vex and concav e regions, this is where the surface has p ositiv e Gaussian curv ature. F or biv ariate functions, these tests for con vexit y and concavit y are p erformed b y considering the b ehavior of the determinan t of the Hessian matrix as well the second deriv ative with resp ect to the first v ariable. In order to formulate p erform these con vexit y and conca vit y tests on a digital image, a pre-pro cessing smo othing is p erformed so that the image b etter appro ximates a differentiable function. The size of the smo othing k ernel determines the size of the features around which con tours are formed. The results sho w that this con tour detection metho d provides several adv antages ov er the wa- tershed transform. The b oundaries are alw ays closed contours for segmen tation, as it is imp ossible to mov e from from p ositive to negativ e Gaussian curv ature without crossing zero. The metho d gen- erally av oids o versegmen tation that occurs with the watershed and an y p oten tial region-merging, user-defined markers [8], or geo desic correction [9]. The other core adv antage of this technique is al- gorithmic and co de simplicity . Unlik e the watershed transform, we don’t need any notions of flo o ding, top ographic distance [7], or graph theory [10]. The results are ac hieve d using simple calculus-based testing. This enables a v ery fast computation. In our results, we compare with the watershed trans- form p erformed on the gradient modulus of the image of in terest. 1 2 Metho ds 2.1 Con tour detection via con vexit y for functions Con vexit y of smooth biv ariate functions can be determined by performing the second partial deriv ativ e test at every point in the domain. Images inherently ha ve tw o dimensions, so it mak es intuitiv e sense that the second partial deriv ative test for functions of tw o v ariables should hav e a connection with conca vity in images. Let f ( x, y ) b e a differentiable real-v alued function of t wo v ariables whose second partial deriv ativ es exist. The Hessian matrix of f is H ( x, y ) =  f xx ( x, y ) f xy ( x, y ) f y x ( x, y ) f y y ( x, y )  . (1) Define D ( x, y ) as the determinan t of H ( x, y ) D ( x, y ) := f xx ( x, y ) f y y ( x, y ) − ( f xy ( x, y )) 2 . (2) The following conditions define conv exit y for biv ariate functions. 1. If D ( x, y ) > 0 and f xx ( x, y ) > 0 then f is con vex at ( x, y ) . 2. If D ( x, y ) > 0 and f xx ( x, y ) < 0 then f is conca ve at ( x, y ) . Note that these t wo conditions are equiv alen t to H ( x, y ) b eing p ositive semidefinite and negative semidefinite, resp ectiv ely . This can also b e viewed as a condition on the Gaussian curv ature, defined b y K ( x, y ) = D ( x, y ) (1 + ( f x ( x, y )) 2 + ( f y ( x, y )) 2 ) 2 . (3) As the denominator is greater than zero, sgn ( D ) = sgn ( K ) . Hence if w e wish to lo ok at con vex and conca ve regions as a single unit, we only need to find D ( x, y ) > 0 , whic h corresp onds precisely to regions with p ositive Gaussian curv ature. Note that Gaussian curv ature is rotation-in v ariant, so it suffices to consider partial deriv atives in the cardinal directions. Example As an example of finding con vex and concav e regions in a differen tiable function of tw o v ariables whose second partial deriv atives exist, w e consider the function z ( x, y ) = 3(1 − x ) 2 e − x 2 − ( y +1) 2 − 2( x − 5 x 3 − 5 y 5 ) e − x 2 − y 2 − 1 3 e − ( x +1) 2 − y 2 . (4) The function and its top ographic surface are sho wn in Figure 1, and its con v ex and concav e regions and their b oundary con tours are shown in Figure 2. Note that the ma jority region where the function is neither con vex nor conca v e corresp onds to negative Gaussian curv ature. That is, in these areas the principal curv atures are of differing signs. 2 -6 3 -4 -2 2 0 3 1 2 2 4 y 0 1 6 x 0 8 -1 -1 -2 -2 -3 -3 Figure 1: Image and surface plot of z ( x, y ) . Figure 2: Conv ex (blue) and conca ve (red) regions of z ( x, y ) and their b oundaries. 2.2 Con tour detection via con vexit y for images Digital space is fundamentally differen t than function space. There are no infinitely small elements that are required for calculus. Hence an image needs to b e pre-pro cessed before the conv exity test can b e p erformed. Without pre-processing, the test typically generates v ery small scale closed contours whic h are not typically useful. Here, this is addressed by performing a smo othing whereb y the image is conv olved with a Gaussian k ernel defined b y k ( x, y ) = 1 2 π σ 2 e − x 2 + y 2 2 σ 2 . (5) The parameter σ determines the size of the smo othing kernel. As σ increases, increasingly large con vex and conca ve regions are typically flattened and hence not detected b y the test. Hence, as sho wn later in the results, there is an inherent opp ortunity to detect con tours at multiple scales in the same image b y p erforming boundary detection on t wo differen tly smo othed v ersions of the same image. This can b e seen in Figures 5 and 6. The algorithm for contour detection is written out fully in Algorithm 1. 3 Algorithm 1 Con tour detection via conv exity 1: Smo oth the image f using a Gaussian kern el of size σ to obtain f σ . 2: Compute D ( x, y ) at eac h pixel in f σ . 3: If ( p, q ) is a pixel such that D ( p, q ) > 0 and f σ xx ( p, q ) > 0 , lab el it with a 0 . Otherwise, 1 . 4: Consider the exterior b oundary of the region found in the previous step. Figure 3: Bubble detection in a radiographic plate [2] via w atershed (left) and conv exity (righ t). 3 Results In this section, w e compare the simple con vexit y-based contour detection algorithm with the w ater- shed transform as formed in [7] applied to gradient images. In the original pap er on watersheds, [3], one application considered w as bubble detection in a radiographic plate, [4]. This example is rep eated in Figures 3. W e also lo ok at detecting dark circular shap es in a gel electrophoresis image in Figure 4, where many more dark, and o verlapping, sp ots are presen t. Figure 5 sho ws the abilit y of the metho d to detect features of differen t scales using σ = 7 . 5 , 15 , 30 . Figure 6 combines the σ = 7 . 5 and 30 results to depict multi-scale features of the original image. Small features are sho wn as filled in regions and large scales are shown as b oundaries. Finally , Figure 7 shows a large ( 1200 × 1200 pixel) image of galaxies from which concav e regions ha ve b een detected such that individual galaxies are identified b y b ounding con tours. 4 Conclusion and F uture W ork This pap er presen ted a con tour detection algorithm that identifies con v ex and conca ve regions of an image. The results sho w sev eral adv an tages of the prop osed metho d for contour detection ov er the gradien t watershed which we summarize b elow. First, the metho d typically do es not o versegmen t if a reasonable smoothing parameter is c hosen. Ev en if extraneous con vex or conca ve regions are identified, they are alw a ys in the form of closed contours, and pruning based on size could b e performed as p ost-pro cessing. F oreground and background are muc h more clearly separated than b y the gradien t w atershed. This metho d also has the adv antage of providing multi-scale feature information in the image through the use of multiple smo othing k ernels of different sizes. The simplicity of the metho d is superior b oth heuristically and in the co de, an example of whic h is sho wn in the App endix. This simplicit y enables a v ery fast execution. A final adv antage is how easily extended this metho d is to 4 Figure 4: Contours on an electrophoresis gel image [2] via watershed (left) and conv exit y (right). higher dimensions. As mentioned in the introduction, the con vexit y of an n -dimensional function is determined b y the Hessian b eing p ositive semidefinite, or negative semidefinite for conca vity . This simply-ev aluated condition will allo w contour and region detection on 3D and 4D datasets that ha ve b een appropriately smoothed as w ell. Finally , it is the author’s hop e that this tec hnique will also aid in the problem of change detection, where a time-differenced image of the same scene can b e smo othed and areas where change has o ccurred will be highligh ted for further insp ection. App endix - MA TLAB Co de Example % load image o f type double im _ original = imre ad( 'image' ); % smooth image sigma = 10; im = imgaussfil t(im _ original,sigma ); % compute Hess ian determinant [imx,imy] = im gradientxy(im); [imxx,imxy] = imgradientxy(imx); [~,imyy] = img radientxy(imy); D = imxx. * imyy - imxy.^2; % find convex and concave regions logical = D > 0 ; % consider the exterior boundary dilate = imdila te(logical,ones(3)); boundary = and( ~logical,dilate); 5 Figure 5: Con tour detection via conv exity in an image of n uclei [6] using three different-sized smo oth- ing kernels. Righ t is a closeup of the full image on the left. References [1] Hubble sees galaxies galore. Hubble Sp ac e T elesc op e (Mar 2014). Credit: NASA, ESA, and S. Bec kwith (STScI) and the HUDF T eam. https://www.spacetelescope.org/images/heic0406a/. 6 Figure 6: Multi-scale detection via con vexit y in an image of n uclei [6] using t wo differen t-sized smo othing kernels. Righ t is a closeup of the full image on the left. Figure 7: Galaxy contour detection on an image from the Hubble Space T elescop e, [1]. Right is a closeup of the full image on the left. [2] Beucher, S. The w atershed transformation. Image Se gmentation and Mathematic al Morpholo gy - Centr e for Mathematic al Morpholo gy (May 2010). h ttps://www.cmm.mines- paristec h.fr/ beucher/wtshed.h tml. [3] Beucher, S., and Lantuejoul, C. Use of watersheds in contour detection. In Pr o c e e dings of the International W orkshop on Image Pr o c essing (1979), CCETT. [4] Beucher, S., and Meyer, F. The morphological approac h to segmentation: the w atershed transformation. Optic al Engine ering-New Y ork-Mar c el Dekker Inc orp or ate d- 34 (1992), 433–433. [5] Bo yd, S., and V andenber ghe, L. Convex optimization . Cam bridge univ ersit y press, 2004. 7 [6] Eddins, S. Cell segmentation. Steve on Image Pr o c essing and MA TLAB (Jun 2006). Credit: Image courtesy of Dr. Ramiro Massol. https://blogs.math works.com/stev e/2006/06/02/cell- segmen tation/. [7] Meyer, F. T op ographic distance and w atershed lines. Signal pr o c essing 38 , 1 (1994), 113–125. [8] Meyer, F. The watershed concept and its use in segmen tation: a brief history . arXiv pr eprint arXiv:1202.0216 (2012). [9] Najman, L., and Schmitt, M. Geo desic saliency of w atershed con tours and hierarc hical segmen tation. IEEE T r ansactions on p attern analysis and machine intel ligenc e 18 , 12 (1996), 1163–1173. [10] Vincent, L., and Soille, P. W atersheds in digital spaces: an efficien t algorithm based on immersion simulations. IEEE T r ansactions on Pattern Analysis & Machine Intel ligenc e , 6 (1991), 583–598. 8