Bioinformatician's Pocket Reference !!

It is amusing how brain of bioinformaticians work! Learning a new programming language for days feels so much of fun that making 5 minute discussion with neighbours (unless under special circumstances!) in our own mother-tongue. Today every bioinformatician keeps more than few languages and core IT toolkits on their plate. It has become mandatory to be able to mould different code snippets to build our own custom workflows, and thus keeping syntax at our fingertips has become essential.

Although Google is best way to get syntax problem solved, it is not a bad idea to keep reference sheets is our smartphones or stick out some printed sheets on the back of your door, in the old fashion way!!



    1) Apache

    2) Awk/Gwak

    3) C

    4) C++

    5) Debian

    6) Git

    7) HTML

    8) Java

    9) Mathematica

    10) Matlab

    11) MySQL

    12) Perl

    13) PHP

    14) Python

    15) Screen

    16) Ubuntu

    17) UNIX

    18) Vim

These are handpicked reference sheets and you may encounter various other versions of these over Internet. If you find any version of reference sheet which is worth sharing, feel free to paste the link below.

At the end, I sincerely acknowledge the authors who put their efforts in designing these informative reference sheets and made it available for us.

Gene Ontology (GO) Enrichment Analysis in Novel Transcriptomes using BiNGO!!

A greater hurdle while dealing with differentially expressed transcripts in novel organisms is the Gene Ontology (GO) enrichment analysis and their visual interpretation.

To date there are several open-source applications available to extract GO terms corresponding to protein/nucleotide sequences (A detailed list can be accessed here, However, the best I have experienced for the De novo transcripts is InterProScan), and to perform enrichment analysis (A detailed list is here). Most of these enrichment tools work like a charm for model organisms, but only handful of them support the incorporation of custom annotations. One such tool is BiNGO (Biological Networks Gene Ontology tool), an open-source Java plug-in of Cytoscape. BiNGO can be used either on a list of genes, or interactively on subgraphs of biological networks visualized in Cytoscape. BiNGO maps the predominant functional themes of the tested gene set on the GO hierarchy.

In order to use BiNGO for novel organisms, one need to provide a custom annotation file (CAF). In principle, CAF contains the gene/transcript and GO relationship, with one relationship per line, eg.

XLOC_000001=0005515
XLOC_000001=0008270
XLOC_000001=0016491
XLOC_000002=0055114
XLOC_000003=0016491
.
.
XLOC_999999=9999999


The left value is the transcript name and right value is the GO category (without the prefix, 'GO:') obtained using InterProScan or synonymous tool.

The first line of GAF should always be:

(species=Custom_species)(type=Biological Process)(curator=GO)

You can choose to change species name from "Custom_species" to something else. Once the building of GAF (GAF.txt) is complete for all the annotated transcripts. It can be used in place of "Select organism/annotation" by choosing "Custom" option. (As shown in the figure below)

 

Additionally, one can also choose to switch to a newer ontology (obo) file downloaded from geneontology.org download page. After providing gene list of interest and choosing the appropriate options, hit the "Start BiNGO" button to start the analysis.


Cytoscape together with BiNGO offers several downstream network grooming options, which you may find useful. For more on this, visit BiNGO and Cytoscape user guides. Hope this helps in your endeavor.

Docear: For Scientific Literature Management

In the field of research we encounter numerous useful articles. This plethora of literature, thus demands a methodical management. While I was struggling to find an efficient way to put all my articles into perspective, I came across 'Docear', which is described as a free and open source academic literature suite. After experiencing the simplicity and benefits of using Docear in academic projects, I was pleasantly surprised and equally tempted to write about it.

The most appealing features of Docear, includes:
1) The simple user-interface
2) Ability to help create mind maps and link documents directly
3) Seamless integration of PDFs along with their headings and custom annotations
4) Powerful search, and
5) Reference management (including support with MS word)
6) Works with Windows, Mac and Unix

The further detailed list of features can be accessed from their official website. Additionally, the website is equipped with tutorials and snapshots, which are very straight forward. I find this open-source project promising and I am hoping to receive timely updates for it in future.

Many kudos to the Docear team for developing this truly resourceful software suite.

Creating 'Swap' ['Virtual'] Memory on Linux/Unix Operating System

Here's some help for when you have too little RAM/memory and are trying to do memory-intensive steps, like indexing the human genome reference or doing other NGS-related processing.

The way to do it is to create a 'swap file', as follows:

1) Check disk/drive usages:

2) Create the space. This step is long if you select a large amount of space. In this example, 512MB is created under root (/), given that block size (bs) is 1024 bytes:


3) Switch it on:

*This will only last until you restart the operating system, so, useful as a temporary measure. To make the swapfile permanent, do the following:

4) Open the following file:

Paste in the following:

Flat files to Databases: For better Speed, Integration and Sharing

In an ordinary dictionary, a word can be sought in two different ways:

1. Use the index and locate your word of choice, or,
2. Start with "A" and keep going, one by one until you get there.

Obviously, the first way is the smart way. But, when it comes to a real-time organised data, most of us prefer the second way by choosing to read (line by line) and write into the flat files; even when the task is repetitive. Relational Database Management System (RDBMS), such as SQL (can be MySQL, OpenSQL, SQLite, PostgreSQL etc) are well suited for such tasks, yet they are under-implemented by many of the bioinformaticians. 

The use of databases can be intimidating without the formal training of database management, but this overall picture has changed to a great extent with the advent of Object Oriented Mapping (ORM) frameworks. ORMs provide language-specific, object-oriented access to databases. It brings the database handling in the comfort zone of object oriented language of user's choice. For example, in order to access a sequence in the database, one can execute,
this will issue an SQL command at the back-end which is,

Another hectic of database handling is the server setup and maintenance issues. This can be reduced to a great extent by adopting a flexible, server-less and fully embed-able RDBMS, such as SQLite or BerkeleyDB. The rest of the operations of creating, modifying and deleting databases, tables and rows are well taken care by ORMs. The most popular ORMs include SQLObject (Python), DBIx::Class (Perl) and Hybernate (Java), which are open source and easily implementable.

In the modern era, the data is integrated from multiple sources and in complex fashions. This vast amount of information needs to be extracted in a reasonable way and channeled into the manageable and biologically meaningful outcomes in respect to medical applications. The database system offers efficient handling of the data and at the same time it delivers easy access via web applications, making it more suitable for scientific data sharing.
    

Python Modules: Expand your reach in Bioinformatics! (Part#2: Hybrid Programming)

A very classic question in bioinformatics is, which programming language is the best for a bioinformatician? Discussions like this never end with a conclusive answer. Interestingly, people find this question as a piece of cake and jump at it with whatever they have in their hands! The result is, you get a nice rainbow of choices, right from "C" to "PHP"!

Each programming language has its own perks and disadvantages. For example, "C" has an incredible speed in execution but it is equally code-intensive in writing even a simple program. Python and Perl on other hand make the same program code-lite but with a mediocre speed of execution. Apart from these performance issues, every language is blessed with a varying degree of third party modules/libraries.

Python has provided interfaces to many system calls and libraries, giving direct access to the shell of an operating system (modules like os, subprocess let you call unix commands directly from the python terminal). Python is also usable as an extension language for applications written in other languages that need easy-to-use scripting or automation interfaces. More than 15 coding projects have started to establish a platform where python can be integrated with other programming languages like C, Java, Perl, PHP, R, Fortran etc.

These hybrid platforms are either available as python modules which can easily be imported, like we do for general (numpy, maths, random etc) modules or accessible from a parent language (i.e. Jython, python implemented in Java)

A detailed list of these hybrid platforms are accessible from here.

Some fascinating platforms I couldn't resist to mention here are:

• elmer: Elmer allows developers to write code in Python and execute it in C or Tcl.
• JPype: JPype allows python programs to fully access java class libraries.
• PyPerlish: Allows the usage of perl idioms in python.
• RPy: Simple and efficient access to R from python.

It is interesting to note that every platform mentioned here was somebody's dream. Since shifting to a new language might deliver new exciting features but at the same time it takes away what you loved the most about the previous one. Following are the words from the creator of PyPerlish,

"I've used perl for several years, and been very impressed with its ease of use. When you need to do something new, chances are there is an idiom which lets you do it in a few keystrokes. I didn't want to lose that in moving to python. Somehow I wanted to get the benefits of perl's idioms with the robust scalability and maintainability of python. So the idea is to emulate perl idioms, no matter how we implement the python code under the covers."       -- Harry George

 

Python Modules: Expand your reach in Bioinformatics! (Part#1: Phyloinformatics)

Python is getting increasingly popular among bioinformaticians, not just due to its simplistic yet powerful structure but also due to the third party modules which are imparting domain specific added advantages. This series is dedicated towards compilation of such modules, specific to each domain.

In this section, the most popular python modules in phyloinformatics are introduced.
  

• E.T.E a python Environment for phylogenetic Tree Exploration

"ETE is a python programming toolkit that assists in the automated manipulation, analysis and visualization of phylogenetic and other type of trees. It provides a wide range of tree handling methods, node annotation features, programmatic access to the phylomeDB database, and automatic orthology and paralogy prediction methods. In addition, an interactive tree visualization program, as well as a highly customizable tree drawing engine, is included."    -- ETE website

ETE examples: Tree with multiple sequence alignment, Bar chart and Pie chart

ETE is very well documented and pretty easy to use. Traversing the tree in different directions (from root to leaves, and leaves to root), manipulating (adding/removing) custom features to an individual node of tree, creating graphics rich plots, integrating multiple sequence alignments, evolutionary hypothesis testing and much more can be easily achieved with this module.

• DendroPy

"DendroPy is a Python library for phylogenetic computing. It provides classes and functions for the simulation, processing, and manipulation of phylogenetic trees and character matrices, and supports the reading and writing of phylogenetic data in a range of formats, such as NEXUS, NEWICK, NeXML, Phylip, FASTA etc. Application scripts for performing some useful phylogenetic operations, such as data conversion and tree posterior distribution summarization, are also distributed and installed as part of the libary. DendroPy can thus function as a stand-alone library, a component of more complex multi-library phyloinformatic pipelines, or as a scripting “glue” that assembles and drives such pipelines."    -- DendroPy Website

Compared to ETE, DendroPy is more focused towards computational aspect of phyloinformatics, which includes simulation of birth-death process trees, population genetic trees, coalescent tress etc. DendroPy also allows calculation of general tree statistics like tree length, node age, probability under the coalescent model, tree distances etc. Unlike ETE, DendroPy also supports variety of character matrices (DNA, RNA, Proteins, any continuous/ discrete-value data), but at the same time DendroPy allows Phylogenetic Independent Contrasts (PIC) analysis (as described by Felsenstein 1985) given a tree and continuous character matrix.
CAUTION: The current release (3.2.0) do not support python 3.0

• Bio.Phylo (BioPython)

Bio.Phylo module was introduced in BioPython 1.54. This module is simplistic but covers all the necessary functionalities including, parsing/writing various tree formats, displaying trees in different color palettes, searching and traversing methods, clade/node specific information extraction/modification etc. Bio.Phylo also allows integration of third-party application like PAML for phylogenetic analysis by maximum likelihood. Likewise, BioPython wrappers are also available for PhyML, RAxML and FastTree.

All the three modules are well documented and irreplaceable given their functional disparity. There are also couple of other modules which are highly function specific and might just fit into your requirement list. These are,

• P4: a python package for phylogenetics
• For maximum likelihood and Bayesian phylogenetic analysis on molecular sequences

• Mavric: a python toolkit for phylogenetics
• Fully interactive editing of phylogenetic trees