Astro-WISE processing of wide-field images and other data

arXiv:1111.6792v1 [astro-ph.IM] 29 Nov 2011 **Volume Title** ASP Conference Series, Vol. **Volume Number** **Author** c⃝**Copyright Year** Astronomical Society of the Pacific Astro-WISE processing of wide-field images and other data Hugo Buddelmeijer1, O. Rees Williams2, John P. McFarland1, and Andrey Belikov1 1Kapteyn Astronomical Institute, Postbus 800, 9700 AV Groningen, The Netherlands 2Donald Smits Center for Information Technology, Postbus 11044, 9700 CA Groningen, The Netherlands Abstract. Astro-WISE1 (Vriend et al. 2012) is the Astronomical Wide-field Imaging System for Europe (Valentijn et al. 2007). It is a scientific information system which consists of hardware and software federated over about a dozen institutes throughout Europe. It has been developed to exploit the ever increasing avalanche of data produced by astronomical surveys and data intensive scientific experiments in general. The demo explains the architecture of the Astro-WISE information system and shows the use of Astro-WISE interfaces. Wide-field astronomical images are derived from the raw image to the final catalog according to the user’s request. The demo is based on the standard Astro-WISE guided tour, which can be accessed from the Astro- WISE website. The typical Astro-WISE data processing chain is shown, which can be used for data handling for a variety of different instruments, currently 14, including Omega- CAM, MegaCam, WFI, WFC, ACS/HST, etc2. 1. Typical Data Processing Chain Astro-WISE was originally designed to handle the large datasets from the OmegaCAM instrument such as the KiDS survey (Verdoes Kleijn et al. 2012). The backbone of Astro-WISE is set by the way it captures all science products obtained in survey oper- ations in an object-oriented data model. Figure 1 shows the main astronomical classes in the Astro-WISE environment, the basic elements of the data processing chain. Each block is a class and each instance of a class is called a Target. The class incorporates a method to derive the data of the Target from other objects, called its dependencies. The user has an ability to combine recipes in a pipeline to process the data by directly requesting the final required data product he or she is interested in. All pro- cessing parameters, along with the full data lineage, are saved in the metadata through the persistence of all objects. In the demo the user goes from a RawScienceFrame (raw frame observed by the VST and ingested in Astro-WISE) to a SourceList (the cata- 1http://www.astro-wise.org 2http://www.astro-wise.org/portal/instruments_index.shtml 1 2 Buddelmeijer, H., Williams, O. R., McFarland, J. P. and Belikov, A. N. Figure 1. A target diagram: slightly simplified object model that is a view of the dependencies of “targets” to the ocean of raw observational data of astronomical wide-field imaging. The arrows indicate the backward chaining to the raw data, not the progression through any processing pipeline. The colors provide a visual grouping of similar types of data products. log produced from reduced, regridded and coadded images)3, using the Astro-WISE infrastructure through the web services described below. An Astro-WISE node is the building element of Astro-WISE infrastructure. It con- sists of data storage element (dataserver, which stores all the files with images), meta- data database (RDBMS, which keeps metadata including links between data items), computing elements (Distributed Processing Unit) and a number of interfaces and ser- vices which allow to the user to browse and process data stored in the system (see Begeman et al. (2010) for more technical details). 2. Services and Interfaces The main language for the system is Python, but each user can develop her/his own application or use an existing application which can be wrapped into Python. Usu- ally, users develop pipelines or workflows using existing “blocks” with the help of pre- defined Python libraries and classes. The user can also change an existing data model if necessary or implement a new one. The Command Line Interface of Astro-WISE, AWE (Astro-WISE Environment), can be installed on a site without any other components of Astro-WISE (data server 3http://www.astro-wise.org/portal/howtos/man_howto_tutorial_science/man_howto_tutorial_science.shtml Astro-WISE processing of wide-field images and other data 3 and metadata database). Basically the AWE prompt is a link to a local Python installation with the Astro-WISE libraries and environments. Apart from the AWE prompt, Astro-WISE supports a range of web interfaces. This makes it possible for a user to work with data stored in Astro-WISE without the AWE prompt using a web browser only. The web interfaces are divided into two types: data browsing/exploration and data processing/qualification. The first group includes: • dbviewer4 – the metadata database interface which allows browsing and querying all attributes of all persistent objects stored in the system, • Go-WISE5 – allows querying on a limite

This content is AI-processed based on open access ArXiv data.