PipeWork

From Wiki

Revision as of 16:55, 26 February 2009

PipeWork is a web tool allowing the user to create and evaluate his own flowcharts to process, query, filter and display the data provided by the web server. A user friendly web interface allows to manage the PipeWorks with no need of any programming language.

 see the PipeWorkReferenceManual

How it works

A PipeWork is a network of WorkingElements connected together by pipes carrying any kind of data. A PipeWorkElement (PWE) accepts 0, 1 or more arguments as input data, processes these values to create an output value which is sent to an other PipeWorkElement.

The web server provides a set of PipeWorkElements (PWEs).

Input and output of PWEs

• Some of them have no input arguments and create an output value when they are launched. For example the PWE ImannoAnnotateGeneList outputs the list of genes already annotated. The PWE GenoretWpList outputs the list of WorkPackages.
• Others need an input value to be entered by the user. PWE Enter asks the user to enter a line of text, Text a text with newlines, SelectFromListOfGenoretWpList expects the user to select one WP within a list.
• All other PWEs need their inputs to be connected to the output of an other PWE.
• The output of only one PWE is considered as the 'ouptut' of the PipeWork.

The PWEs can be classified as : (see full list here)

• user's interactive input elements : Enter, Select, MultipleSelect, Text, Browse
• mathematical functions : Add, Multiply, Divide, Compare, Sinus, SquareRoot, etc.
• list (or array) processing : Union, Intersection, SubList, ListElement, ListToText, TextToList, ListLength, etc.
• conditional evaluation : SwitchBetween, UseOnlyParam, etc.
• display : ToHtml, ToHtmltable, AddHtmlLink
• thematic specific functions
  • GX : Experiment, Sample, SignalIntensity, ClusterContex, ClusterFromGene, ...
  • ImAnno : GnePaintGeneList, ListOfTissues, FromSieve, CorrelateTissue, ...** etc.

Evaluation of a PipeWork

The PipeWork is evaluated recursively starting with the output PWE : When the user asks the PipeWork to be evaluate, a server side program evaluates the 'output' PWE. For that it evaluates recursively all PWEs connected to the inputs of that PWE in the same order as the argument list. Recursively all inputs of all inputs PWEs are evaluated. The evaluation is done in functional way, not as data driven.

When, during an evaluation of a PipeWork, the same PWE is asked to be evaluated a second time the memorized ouptut value is used.

Why a PipeWork

One can imagine that a programmer writes a php program for any question a user might ask about his favorite list of genes. This works for some time, but what will happend if the gene list changes, when the available data are updated ... or when the question changes, or the display? And if hundreds of users ask tens of questions?

The users can learn php. Or we can invent a new 'metalanguage' allowing the biologist to run complex queries with his easy-to-write 'metaprogram'.

Most of the queries we have to run concerns lists of something to be correlated with other lists, combining set of results with the complement of other sets ... PipeWork provides an increasing set of these kind of basic processing elements. These can easily connected together by the user itself as he would do it on a blackboard trying to explain what he needs.

Each question can be grapically represented by the user, stored, retrieved, updated and reused for any other similar query.

The merge of several PipeWorks is easy to do. Simply connect the outputs of one PipeWork to the inputs of an other.