Difference between revisions of "Decrypthon Data Center"
(→Exploitations) |
(→Exploitations) |
||
| (16 intermediate revisions by the same user not shown) | |||
| Line 11: | Line 11: | ||
Management of generated data (results) on the Grid | Management of generated data (results) on the Grid | ||
Sharing of data and services for the scientific community | Sharing of data and services for the scientific community | ||
| − | http://decrypthon.u-strasbg.fr/birdweb/query.do | + | web server: [http://decrypthon.u-strasbg.fr/birdweb/query.do] |
| − | [[Image: | + | [[Image:ddc_architecure.jpg]] |
==Functionnalities== | ==Functionnalities== | ||
| − | The Decrypthon Date Center is be able to manage databases of various types, the table 1 details the data banks often used by biologists. These biological datasets are widely distributed over Internet and made available in different formats. The integration system must be able to cope with five strong constraints: | + | The Decrypthon Date Center is be able to manage databases of various types, the table 1 details the data banks often used by biologists. These [[Bird_Databases_List | biological datasets]] are widely distributed over Internet and made available in different formats. The integration system must be able to cope with five strong constraints: |
• Volume of the data: the public biological information consists of genome and gene sequences, protein structures and sequence alignments representing more than one terabyte. | • Volume of the data: the public biological information consists of genome and gene sequences, protein structures and sequence alignments representing more than one terabyte. | ||
| Line 25: | Line 25: | ||
• Safety and confidentiality: The system must also integrate clinical databases. For obvious questions of confidentiality and ethics, it is not always possible to duplicate such databases or to transfer sensitive data on the networks. The pertinent information thus remains under the control of the owners who allow a restricted access to the remote database. | • Safety and confidentiality: The system must also integrate clinical databases. For obvious questions of confidentiality and ethics, it is not always possible to duplicate such databases or to transfer sensitive data on the networks. The pertinent information thus remains under the control of the owners who allow a restricted access to the remote database. | ||
| − | • Interoperability capacities: In bioinformatics, many toolkits | + | • Interoperability capacities: In bioinformatics, many toolkits have been developed with different programming languages and are composed of several independent software components sharing the same data through different protocols. Thus the system must provide an easy access for these external softwares through independent methods such as http service, web service or API. |
• Query expression and treatment: the system must also provide a simple protocol allowing users, which are generally not computer scientists, to express easily their query or retrieval protocol on data banks without knowing the structure of the relational database. This protocol should be written in flat files or XML format in order to reuse it in any other data warehouse created by the same integration system. | • Query expression and treatment: the system must also provide a simple protocol allowing users, which are generally not computer scientists, to express easily their query or retrieval protocol on data banks without knowing the structure of the relational database. This protocol should be written in flat files or XML format in order to reuse it in any other data warehouse created by the same integration system. | ||
| Line 34: | Line 34: | ||
| − | In this project, BIRD is the core of centre (figure xx). BIRD uses DB2 as indexing and very large data storage unit. BIRD provides | + | In this project, [[BIRD]] is the core of centre (figure xx). BIRD uses IBM DB2 as indexing and very large data storage unit. BIRD provides query services for several high level applications. BIRD is installed with the storage node of computing grid. WebSphere Federer Server-WFS (called federation database) is used to integrate the remote clinical database hosted by Universal Mutation Database-UMD software. This federation database communicates with each data source by means of software modules called wrappers. A wrapper contains characteristics about a specific data source. It provides a DB2 relational model for the remote data and assists the federated engine in query processing by sending sub-queries to the remote data source. |
The “Registrer” module allows us to automatically publish metadata and important information of a remote database. Thanks to this module, WFS system generates the virtual relational schema according to the remote database. It will be also possible for us to search metadata of a recorded data source. | The “Registrer” module allows us to automatically publish metadata and important information of a remote database. Thanks to this module, WFS system generates the virtual relational schema according to the remote database. It will be also possible for us to search metadata of a recorded data source. | ||
| − | Figure 1 et 2 illustrate the data centre implemented by BIRD which makes integration of data of various sources and shares these data for the applications which run on the grid computing and for the biological community. The centre shares its data integrated with applications/clients and users of several manners: via an API Java, of a Web interface (figure 3) as well as language | + | Figure 1 et 2 illustrate the data centre implemented by BIRD which makes integration of data of various sources and shares these data for the applications which run on the grid computing and for the biological community. The centre shares its data integrated with applications/clients and users of several manners: via an API Java, of a Web interface (figure 3) as well as language [[BIRDQL]] via orders HTTP. |
| + | ==[[Bird_Databases_List | biological datasets]]== | ||
==Exploitations== | ==Exploitations== | ||
| Line 45: | Line 46: | ||
LGBI Laboratory | LGBI Laboratory | ||
| + | |||
-Ordali software | -Ordali software | ||
| + | |||
-Macsim | -Macsim | ||
| + | |||
-Gscope | -Gscope | ||
| + | |||
-.... | -.... | ||
... | ... | ||
| + | |||
| + | ==Contact== | ||
| + | |||
| + | TOURSEL Thierry, PhD | ||
| + | Project Manager | ||
| + | AFM Association Française contre les Myopathies | ||
| + | Direction Scientifique | ||
| + | 1 rue de l'Internationale | ||
| + | BP59 91002 Evry cedex | ||
| + | FRANCE | ||
| + | Tel (33) 1 69 13 22 23 | ||
| + | Fax (33) 1 69 13 22 22 | ||
| + | |||
| + | |||
| + | FACCI Hervé | ||
| + | Senior Consultant | ||
| + | IBM Global Business Services / Ile de France | ||
| + | Tel : #33 (0)1 49 05 69 76 (33-6976) | ||
| + | Cell : #33 (0)6 72 75 23 27 | ||
| + | hfacci@fr.ibm.com | ||
| + | |||
| + | |||
| + | POCH Olivier,PhD | ||
| + | Director of LGBI Laboratory | ||
| + | IGBMC Strasbourg | ||
| + | 1 rue Laurent Fries | ||
| + | BP 10142 | ||
| + | 67404 Illkirch CEDEX / France | ||
| + | Mail:[mailto:poch@igbmc.fr poch@igbmc.fr] | ||
| + | Tel: 0033 388653302 | ||
| + | |||
| + | |||
| + | |||
| + | NGUYEN Ngoc Hoan,PhD | ||
| + | Architect/Administrator of Decrypthon Data Center | ||
| + | IGBMC Strasbourg | ||
| + | 1 rue Laurent Fries | ||
| + | BP 10142 | ||
| + | 67404 Illkirch CEDEX / France | ||
| + | Mail:[mailto:nguyen@igbmc.fr nguyen@igbmc.fr] | ||
| + | Tel: 0033 388653302 | ||
Latest revision as of 05:55, 20 June 2008
Contents
Overview
Since 2000, thanks to the availability of the human genome and the rapid progress of biotechnologies and information technologies, numerous large biomedical datasets have been generated. Thus, modern biomedical information corresponds to a high volume of heterogeneous data that doubles in size every year (Statistics NCBI) and that covers very different data types, including patient data (from phenotypic, environmental or behavioral origins), gene data (including genome environment, gene expression status, enzymatic activity, gene product modification…) and the processes, protocols or treatments used to generate the information. In this context, systemic approaches are now being developed to analyze and compare this huge amount of information, in order to identify genes and to predict their functions in the cascade of events and networks involved for example, in the emergence of a disease. This requires the development of dynamic and powerful systems to store, assemble, integrate and process very large datasets from different sources. Recently, the Decrypthon initiative (Decrypthon), resulting from a collaboration between AFM/CNRS/IBM, has been instigated, firstly to develop a computing grid that connects hundreds of processors installed in various data-processing centres of French universities and, secondly to provide a facilitated access to the data for the scientific biological community. In the framework of the Decrypthon initiative, several biomedical projects are in progress requiring on the one hand, a strong computational capacity and on the other hand, the deployment in the grid environment of a data integration system able to manage automatically large volumes of heterogeneous data and to quickly process complex queries and versioning management (called Decrypthon Data Center).
The BIRD System was used to implementation of Décrypthon Data Center
Sharing of large scale biological data for applications (Macsims, MS2PH, Magos, Ordalie..) Running on the Décrypthon Grid. Management of generated data (results) on the Grid Sharing of data and services for the scientific community web server: [1]
Functionnalities
The Decrypthon Date Center is be able to manage databases of various types, the table 1 details the data banks often used by biologists. These biological datasets are widely distributed over Internet and made available in different formats. The integration system must be able to cope with five strong constraints:
• Volume of the data: the public biological information consists of genome and gene sequences, protein structures and sequence alignments representing more than one terabyte.
• Heterogeneity and management of the data: ranging from three-dimensional models and sequence alignments, up to images of gels and scientific articles, these data are provided in different formats. This requires the developments of specific parsers corresponding to each data type. Additionally the data generated by the scientific projects have to be stored and indexed in real time.
• Safety and confidentiality: The system must also integrate clinical databases. For obvious questions of confidentiality and ethics, it is not always possible to duplicate such databases or to transfer sensitive data on the networks. The pertinent information thus remains under the control of the owners who allow a restricted access to the remote database.
• Interoperability capacities: In bioinformatics, many toolkits have been developed with different programming languages and are composed of several independent software components sharing the same data through different protocols. Thus the system must provide an easy access for these external softwares through independent methods such as http service, web service or API.
• Query expression and treatment: the system must also provide a simple protocol allowing users, which are generally not computer scientists, to express easily their query or retrieval protocol on data banks without knowing the structure of the relational database. This protocol should be written in flat files or XML format in order to reuse it in any other data warehouse created by the same integration system.
Implementation
The data centre is integrated directly in the Decrypthon computing grid in order to efficiently share all the data necessary for the biological applications requiring a strong potential for calculation and storage. The Decrypthon data centre contains a local database of nucleotide, genomic and proteomic sequences. It provides access methods for heterogeneous and distributed data, as well as a treatment of queries, and data analysis tools.
In this project, BIRD is the core of centre (figure xx). BIRD uses IBM DB2 as indexing and very large data storage unit. BIRD provides query services for several high level applications. BIRD is installed with the storage node of computing grid. WebSphere Federer Server-WFS (called federation database) is used to integrate the remote clinical database hosted by Universal Mutation Database-UMD software. This federation database communicates with each data source by means of software modules called wrappers. A wrapper contains characteristics about a specific data source. It provides a DB2 relational model for the remote data and assists the federated engine in query processing by sending sub-queries to the remote data source.
The “Registrer” module allows us to automatically publish metadata and important information of a remote database. Thanks to this module, WFS system generates the virtual relational schema according to the remote database. It will be also possible for us to search metadata of a recorded data source.
Figure 1 et 2 illustrate the data centre implemented by BIRD which makes integration of data of various sources and shares these data for the applications which run on the grid computing and for the biological community. The centre shares its data integrated with applications/clients and users of several manners: via an API Java, of a Web interface (figure 3) as well as language BIRDQL via orders HTTP.
biological datasets
Exploitations
MS2PH & MS2PH-DB Project
MYOBASE (in project)
LGBI Laboratory
-Ordali software
-Macsim
-Gscope
-....
...
Contact
TOURSEL Thierry, PhD Project Manager AFM Association Française contre les Myopathies Direction Scientifique 1 rue de l'Internationale BP59 91002 Evry cedex FRANCE Tel (33) 1 69 13 22 23 Fax (33) 1 69 13 22 22
FACCI Hervé Senior Consultant IBM Global Business Services / Ile de France Tel : #33 (0)1 49 05 69 76 (33-6976) Cell : #33 (0)6 72 75 23 27 hfacci@fr.ibm.com
POCH Olivier,PhD Director of LGBI Laboratory IGBMC Strasbourg 1 rue Laurent Fries BP 10142 67404 Illkirch CEDEX / France Mail:poch@igbmc.fr Tel: 0033 388653302
NGUYEN Ngoc Hoan,PhD Architect/Administrator of Decrypthon Data Center IGBMC Strasbourg 1 rue Laurent Fries BP 10142 67404 Illkirch CEDEX / France Mail:nguyen@igbmc.fr Tel: 0033 388653302
