小生这厢有礼了(BioFaceBook Personal Blog) » genome

The SILVA and “All-species Living Tree Project (LTP)” taxonomic frameworks

szypanther — Fri, 06 Dec 2013 07:45:24 +0000

SILVA (from Latin silva, forest, http://www.arb-silva.de) is a comprehensive resource for up-to-date quality-controlled databases of aligned ribosomal RNA (rRNA) gene sequences from the Bacteria, Archaea and Eukaryota domains and supplementary online services. SILVA provides a manually curated taxonomy for all three domains of life, based on representative phylogenetic trees for the small- and large-subunit rRNA genes. This article describes the improvements the SILVA taxonomy has undergone in the last 3 years. Specifically we are focusing on the curation process, the various resources used for curation and the comparison of the SILVA taxonomy with Greengenes and RDP-II taxonomies. Our comparisons not only revealed a reasonable overlap between the taxa names, but also points to significant differences in both names and numbers of taxa between the three resources.

BEReX : Biomedical Entity-Relation eXplorer

szypanther — Fri, 06 Dec 2013 02:14:20 +0000

BEReX is a new biomedical knowledge integration, search, and exploration tool. BEReX integrates eight popular databases (STRING, DrugBank, KEGG, PharmGKB, BioGRID, GO, HPRD, and MSigDB) and delineates an integrated network by combining the information available from these databases. Users search the integrated network by entering keywords and BEReX returns a sub-network matching the keywords. The resulting graph can be explored interactively. BEReX allows users to find the shortest paths between two remote nodes; find the most relevant drugs, diseases, pathways and so on, related to the current network; expand the network by particular types of entities and relations; and modify the network by removing or adding selected nodes. BEReX is implemented as a stand-alone Java application.

1. Program availability and requirements

Operating systems : Windows, Mac, Linux
Java runtime : JRE6 or higher is need to run the application

2. Installation

Install JRE(skip this step if you have JRE 6 or later, or JDK 1.6 or later)
Download BEReX (v.1.0) and unzip the package file:
1. Windows : berex-v1-windows.zip(184MB)
2. Mac : berex-v1-mac.zip(184MB)
3. Linux : berex-v1-linux.zip(184MB)
Run BEReX.bat (for Windows) or BEReX.sh.command (for Mac) or BEReX.sh (for Linux)
Please cite the following article when using BEReX.
- Jeon,M., Lee,S., Lee,K., Tan,A., Kang,J.; BEReX: Biomedical Entity-Relationship eXplorer. Bioinformatics (2013) doi: 10.1093/bioinformatics/btt598

3. Documentation

User Guide : BEReX v1.0 User Guide

4. Souce Code

Source Code : berex_sourcecode.zip

5. License

BEReX is licensed under the GNU General Public License and is 100% freely available to both commercial and academic users. See the file LICENSE.txt in the BEReX distribution package or this URL for the full text of the license: http://www.gnu.org/licenses/gpl.html

6. Contact – for bugs, comments and questions

Minji Jeon: mjjeon@korea.ac.kr
Jaewoo Kang: kangj@korea.ac.kr

Last updated on Sep 25, 2013

KEGG annotation pipeline

szypanther — Wed, 16 Oct 2013 05:10:45 +0000

KEGG Pathway Pipeline:

blastall -p blastp -d KEGG -i Haiyan.Pep.fasta -m 7 -a 10 -o Haiyan.Pep.fasta.blastp.m7 &
./tBLASTnParser.pl Haiyan.Pep.fasta.blastp.m7 Haiyan.Pep.fasta.blastp.m8
sed ‘1,1d’ Haiyan.Pep.fasta.blastp.m8 > Haiyan.Pep.fasta.blastp.m8.delhead

/home/zhouzh/lib/454-2.5/bin/runAssembly -m -cpu 16 -cdna -nobig -o Test sff/GV1NGBM02.sff

./draw_png.py -i ACYPIprot.KO.file -p /home/shenzy/KEGG/ko_org -o map_result5

step 1:
/home/soft/blast-2.2.23/bin/blastall -p blastp -d KEGG -i MBL_relation.fa -a 15 -b 30 -v 30 -m 7 -FF -o MBL_relation.fa.blastp2.m7 &

step2:
/home/shenzy/work_python_script_bak/tBLASTnParser.pl MBL_relation.fa. 2.m7 MBL_relation.fa.blastp2.m8

sed -e ‘1d’ G_seq_fkegg_Mix4.blastp.m8.result > G_seq_fkegg_Mix4.blastp.m8.result.nohead

./handle_KEGG_blast.py -i MBL_relation.fa.blastp2.m8 -j ../ko_gene -g anno_file2 -s anno_file_status2 > KO_list_file2

step3:
handle anno_file_status !!!!!!!!! not ko_list_file !! (must del BR:ko04091: …… and PATH:…..)

./draw_png2.py -i MBL2.KOFILE -p /home/shenzy/KEGG/ko_org/ -o MBLkeggMAP.result2

/home/soft/velvet_1.0.19/shuffleSequences_fastq.pl lane3_1209.read2.fq.t10l40.bowtie.file lane3_1209.read1.fq.t10l40.bowtie.file lane3_1209.t10l40.bowtie.pe12.fq
/home/soft/fastx_toolkit-0.0.13/src/fastq_to_fasta/fastq_to_fasta -n -i lane3_1209.t10l40.bowtie.pe12.fq -o lane3_1209.t10l40.bowtie.pe12.fa
cat lane3_1209.t10l40.bowtie.pe12.fa lane3_read12.fa lane4_read12.fa s_2_pe12.fasta > s_2343_pe12.fasta

blastall -p blastp -d ../KEGG -i AphisVelvet.pep -a 15 -b 30 -v 30 -m 7 -FF -o AphisVelvet.pep.blastp.result2 &:q!:q!
QueryName HSP QueryLength SubjctLength QueryAlignment SubjctAlignment Annotation Score BitScore EValue IdentityRate QueryFrame QueryStart QueryEnd SubjectFrame SubjectStart SubjectEnd

cdhit-cluster-consensus 1.GAC.454Reads.fna.cluster.clstr 1.GAC.454Reads.fna cdhit.result &[2] 17965
…read 379519 clusters from file “1.GAC.454Reads.fna.cluster.clstr”000 lines
read 737073 sequences from file “1.GAC.454Reads.fna”
write 5000 singleton clusters
write 293650 singleton clustersCDNA$ write 6000 singleton clusters
finish 85869 clusters out of 85869 non-singleton clusters

ACYPI000002-PA RefSeq peptide NP_001119607 gi|187097094|ref|NP_001119607.1 sucrase [Acyrthosiphon pisum] 1 590 588 95.76 97.45 dme:Dmel_CG8690 CG8690 gene product from transcript CG8690-RA (EC:3.2.1.20); K01187 alpha-glucosidase [EC:3.2.1.20] 1255 488.034 7.63337e-136 46.34 1 19 583 1 15 587

######################################
Kegg results have protein name deb:DehaBAV1_0078, we can get the related information from COG.mappings.v8.3.txt
######################################

shenzy@shenzy-ubuntu:/winxp_disk2/shenzy/BGI/lla/gene_annotation$ more 11aRayScalf_all.fna.cds.faa.blastp_kegg.m8.top1
QueryName HSP QueryLength SubjctLength QueryAlignment SubjctAlignment Annotation Score BitScore EValue IdentityRate QueryFrame QueryStar
t QueryEnd SubjectFrame SubjectStart SubjectEnd
11aRayScalf10001 87 87 734 648 648 0.891375 0.891375 3 D 3 (translation) 1 215 319 100.00 67.40 deb:DehaBAV1_0078 phage integrase family protein 1
085 422.55 5.04278e-117 94.42 1 1 215 1 66 280

shenzy@shenzy-ubuntu:/winxp_disk2/shenzy/BGI/lla/gene_annotation$ grep “DehaBAV1_0078″ COG.mappings.v8.3.txt
216389.DehaBAV1_0078 32 296 COG4974 Phage integrase family protein

shenzy@shenzy-ubuntu:/winxp_disk2/shenzy/BGI/lla/gene_annotation/kegg$ ./draw_png_good.py -i anno_file_status -p /winxp_disk2/shenzy/KEGG/img/ -o test.out

#################################################################################################################

shenzy@shenzy-ubuntu:/winxp_disk2/shenzy/BGI/MB/gene_annotation/kegg$ blastall -p blastp -d kegg-Prokaryotes -i 47_acc_num_xiaoying.txt.fasta -a 15 -b 30 -v 30 -m 7 -FF -o 47_acc_num_xiaoying.txt.fasta.blastp.m7 &

shenzy@shenzy-ubuntu:/winxp_disk2/shenzy/BGI/MB/gene_annotation$ handle_KEGG_blast.py -i MBrayScalfALL.fna.cds.faa.blasp.kegg-P.m8.nohead -j ko -g anno_file2 -s anno_file_status2 > KO_list_file2

shenzy@shenzy-ubuntu:/winxp_disk2/shenzy/BGI/MB/gene_annotation/kegg$ handle_KEGG_blast_filterzero.py -i 2548N_stat.siggenes_102030min.filter.protein.fasta.blastp.m8.nohead -j ko.pep.fasta -g 162_anno_file -s 162_anno_file_status > 162_KO_list_file &
#################################################################################################################

blastall -p blastp -d kegg-Prokaryotes -i 176_protein.fasta -a 15 -b 30 -v 30 -m 7 -FF -o 176_acc_relation.fa.blastp.m7 &

handle_KEGG_blast.py -i 176_acc_relation.fa.blastp.m8.nohead -j ko -g 176_anno_file -s 176_anno_file_status > 176_KO_list_file &

draw_png_good.py -i 176_anno_file_status -p img -o 176_kegg_results &

Reordering contigs in draft genomes by MAUVE

szypanther — Wed, 18 Sep 2013 02:08:17 +0000

When to use Mauve Contig Mover (MCM)

The Mauve Contig Mover (MCM) can be used to order a draft genome relative to a related reference genome. The functionality of this software module has been described in Rissman et al. 2009 , a publication in Bioinformatics. The Mauve Contig Mover can ease a comparative study between draft and reference sequences by ordering draft contigs according to the reference genome. In many cases, true rearrangements in the draft relative to the reference can be identified. The quality of the reorder is limited by the distance between the sequences, as indicated by the amount of shared gene content among the two organisms. A more distant reference will usually yield fewer ordered draft genome contigs, and may also induce erroneous placements of draft contigs. In addition to ordering contigs, MCM also orient them in the most likely orientation, and, if annotated sequence features are specified in an input file (e.g. with GenBank format input for the draft), MCM will output adjusted coordinates ranges for the features.

Using Mauve Contig Mover

Mauve Contig Mover can be launched from the Tools->Order Contigs Menu of The Mauve Viewer.

Once the Mauve Contig Mover has been launched, it starts by requesting the user to specify an output directory. MCM will create a series of mauve alignments in the output directory, structured into several subfolders, so creating a new, empty output folder is often best to minimize clutter. The output directory selection is shown below.

Once the output directory has been set, a window similar to the Progressive Mauve Alignment Window appears. Using the Progressive Mauve alignment window is described in more detail in the section “Constructing a genome alignment “, but when used for reordering contigs, the following additional notes and constraints apply:
1.    Only two sequences should be entered. The first must always be the reference, the second the draft genome to reorder. The first may also be a draft, but only the second will be reordered.
2.    The reference genome may be in any of the allowable file formats, the draft must either be in a fasta or genbank file. If the draft is in genbank format, a unique identifier must be specified in either the LOCUS tag of each contig.
3.    The alignment parameters have already been adjusted for what is generally best for draft genomes. This may depend on the draft and reference, and can be adjusted if needed.

The Reordering Process

The reordering will begin when the start button is pressed. It is an iterative process, and may take anywhere from a half hour to several hours. It may be cancelled at any point (intermediate results will be viewable). If it is canceled after the first reorder the reordered draft genome will be available in a Multi-FastA file in the corresponding output directory, although an alignment of the canceled ordering step will not be present. If the ordering process is not manually ended, it will terminate when it finds an order has repeated. Sometimes the order will cycle through several possibilities; this indicates it cannot determine which of them is most likely. Alignment parameters may be changed before reorder starts or any time between alignments.
The following message will appear when the reorder process is complete:

The MCM Output Files

MCM will output a series of folders called alignment1-alignmentX, representing each iteration of the reorder. Each has the standard Mauve alignment files, as described in the section Mauve Output File Formats . Each folder also has an additional file called name_of_genome_contigs.tab, where “name_of_genome” is the draft genome’s name. This file is included for ease of interpreting reorder results, and also acts as an index to the fasta as the contig orders and orientations change (even if the draft was originally input as a genbank, after the first alignment, it will be converted to a fasta with annotation information preserved in a file described below). The file is divided into 3 sections, each containing a list of contigs. The data for each contig includes its label (name), its location in the genome (numbered in pseudocoordinates from the first to last contig; these coordinates can be entered into the View->Go To->Sequence Position menu option to jump to that contig using the Mauve Alignment Viewer), and whether it is oriented the same as originally input, or was complemented. The three sections are described below:

1. Contigs to reverse: This section contains contigs whose order is reversed with respect to the previous iteration. Note that contigs in this section may be oriented the same as originally input, this can be determined from the forward orcomplement designation.

2. Ordered Contings: This is a list of all the contigs in the order and orientation they appear in the fasta for the draft of this iteration of the reorder. Since these include all the contigs in the original input, those with no ordering information (no aligned region) will be clustered at the end. These will appear as contigs with no LCBs at the end of the draft genome.

3. Contigs with Conflicting Order information: This is a list of contigs containing LCBs suggesting multiple possible locations. These may be of interest to verify positioning, or to look at points of potential rearrangement or misassembly.

If the draft was input as an annotated genbank file, a second file will appear in each alignment folder called name_of_genome_features.tab. This file will contain a line for each annotation, information about its current orientation and location (which will change if the contig is inverted), coordinates from the previous iteration (indicating relative orientation), and whether it is reversed from the original input. It will also have a label field used to identify each feature. This will be gotten from the annotation, as checked in the following order: db_xref, label, gene, and locus_tag.

Thus, the folder with the highest numbered alignment contains a fasta and one (or possibly two) descriptor files representing the final order of the draft genome.

Reordering contigs from the command-line (batch mode)

In situations where it is necessary to order contigs in a large number of draft genomes it is often more desirable to automate the process using command-line interfaces and scripts. Mauve Contig Mover supports command-line operation through the Mauve Java JAR file.

Given a reference genome file called “reference.gbk” and a draft genome called “draft.fasta”, one would invoke the reorder program with the following syntax:

java -Xmx500m -cp Mauve.jar org.gel.mauve.contigs.ContigOrderer -output results_dir -ref reference.gbk -draft draft.fasta

The file Mauve.jar is part of the Mauve distribution. On windows systems it can usually be found in C:\Program Files\Mauve X\Mauve.jar where X is the version of Mauve. On Mac OS X it is located inside the Mauve application. For example, if Mauve has been placed in the OS X applications folder, Mauve.jar can be found at /Applications/Mauve.app/Contents/Resources/Java/Mauve.jar. On Linux, Mauve.jar is simply at the top level of the tar.gz archive. In the above example command, it will be necessary to specify the full path to the Mauve.jar file.

http://asap.ahabs.wisc.edu/mauve-aligner/mauve-user-guide/reording-contigs-in-draft-genomes.html

Install genometools

szypanther — Fri, 25 Jan 2013 04:27:52 +0000

the ‘new’ error message refers to a nonexistant Cairo library on your system, which is needed for the AnnotationSketch component of GenomeTools. If you do not need this, do a ‘make cleanup’ and recompile with the additional make option ‘cairo=no’, e.g. ‘make errorcheck=no cairo=no’. This will disable support for AnnotationSketch and remove the cairo dependency.

As for your other question, you can use the ‘gt suffixerator’ tool as described. However, the ‘gt’ binary is placed in the ‘bin/’ subdirectory of your GenomeTools source directory after compiling. Please keep that in mind and call ‘bin/gt’ if necessary.
You should be able to run the command line exactly as described if you install the ‘gt’ binary system-wide (‘make install’) or add its location to your PATH environment variable.

First, we should also install ruby and cairo separately !

Batch download sequences from uniprot based on protein names

szypanther — Wed, 28 Nov 2012 04:20:39 +0000

Ok, I’ll do mine in English:

go to UniProt.org.
click tab “retrieve”
Paste list into text box. Click Retrieve button.
On results page, click FASTA download [ Download (30 KB*) | Open ] (Or you could click open just to have a look).

Circos 安装和学习（一）

szypanther — Thu, 06 Sep 2012 02:33:59 +0000

http://circos.ca/documentation/

Tutorials and Course

The tutorials serve as a walkthrough through Circos. The course is a more structured set of materials that takes you through creating an image from scratch.

The tutorials act as documentation — each lesson presents a specific feature of Circos.

Example Image

Once you download and install Circos,

# install circos
> tar xvfz circos-x.xx.tgz
> cd circos-x.xx

If you get an error like

-bash: /bin/env: No such file or directory

then your 'env' binary is likely in /usr/bin (e.g. on Mac OS X) Check this by 

> which env
/usr/bin/env

To fix this, either change the first line in scripts in bin/* and tools/*/bin to 

#!/usr/bin/env perl

or make a symlink from /usr/bin/env to /bin/env

> sudo su
> cd /bin
> ln -s /usr/bin/env env

files

file	version	size	date	comment
current
circos-tools-0.16-1.tgz	0.16	13,721,840	Wed Aug 1 17:38:27 2012	circos-tools-0.16-1.tgz is a bug release.
circos-0.62-1.tgz	0.62	23,776,624	Tue Jul 3 12:59:00 2012	circos-0.62-1.tgz is a bug release.
circos-0.62.tgz	0.62	21,905,722	Mon Jun 25 15:30:13 2012
circos-tutorials-0.62.tgz	0.62	125,634,176	Mon Jun 25 14:49:15 2012
circos-course-0.61.tgz	0.61	325,718,042	Tue Jun 5 15:15:03 2012
circos-tools-0.16.tgz	0.16	11,046,936	Mon Jul 25 14:29:13 2011

下载解压后还需要安装一些Circos依赖的必要模块等：

Config::General (v2.50 or later)
GD
GD::Polyline
List::MoreUtils
Math::Bezier
Math::Round
Math::VecStat
Params::Validate
Readonly
Regexp::Common
Set::IntSpan (v1.16 or later)
Text::Format
Font::TTF::Font
利用CPAN自行安装以上模块

安装模块

perl -MCPAN -e shell
cpan> install Math::Bezier
cpan> install Regexp::Common

……

try creating the example image that ships with the Circos core distribution.

> cd example
# on UNIX systems (see README for Windows use)
> ./run

Creating a Tutorial Image

You will need to download the tutorials separately. Follow the installation instructions in the archive file.

> cd tutorials/2/2
# now try tutorial 2.2
> ../../../bin/circos -conf ./circos.conf

俺的安装路径如下：
shenzy@shenzy-ubuntu:/winxp_disk2/shenzy/circos/circos-tutorials-0.62/tutorials/2/2$ ../../../../circos-0.62-1/bin/circos -conf ./circos.conf
debuggroup conf 0.13s welcome to circos v0.62-1 25 Jun 2012
debuggroup conf 0.13s loading configuration from file ./circos.conf
debuggroup conf 0.13s looking for conf file ./circos.conf
debuggroup conf 0.13s found conf file ./circos.conf
debuggroup summary 0.32s debug will appear for these features: summary
debuggroup summary 0.32s parsing karyotype and organizing ideograms
debuggroup summary 0.49s applying global and local scaling
debuggroup summary 0.50s allocating image, colors and brushes
debuggroup summary 5.91s drawing highlights and ideograms
debuggroup summary,output 8.10s generating output
debuggroup summary,output 8.77s created PNG image ./circos.png (605 kb)
debuggroup summary,output 8.77s created SVG image ./circos.svg (371 kb)

Circos INSTALL SUCCESS!

TaxCollector: Modifying Current 16S rRNA Databases for the Rapid Classification at Six Taxonomic Levels

szypanther — Wed, 08 Aug 2012 02:50:10 +0000

Our project TaxCollector has been published in MPDI Diversity.

Abstract

The high level of conservation of 16S ribosomal RNA gene (16S rRNA) in all Prokaryotes makes this gene an ideal tool for the rapid identification and classification of these microorganisms. Databases such as the Ribosomal Database Project II (RDP-II) and the Greengenes Project offer access to sets of ribosomal RNA sequence databases useful in identification of microbes in a culture-independent analysis of microbial communities. However, these databases do not contain all of the taxonomic levels attached to the published names of the bacterial and archaeal sequences. TaxCollector is a set of scripts developed in Python language that attaches taxonomic information to all 16S rRNA sequences in the RDP-II and Greengenes databases. These modified databases are referred to as TaxCollector databases, which when used in conjunction with BLAST allow for rapid classification of sequences from any environmental or clinical source at six different taxonomic levels, from domain to species. The TaxCollector database prepared from the RDP-II database is an important component of a new 16S rRNA pipeline called PANGEA. The usefulness of TaxCollector databases is demonstrated with two very different datasets obtained using samples from a clinical setting and an agricultural soil.

TaxCollector is available at SourceForge and GitHub and is licensed under the Open Source GNU GPL v3.

Microbial Community Analysis GUI–Bioconducter

szypanther — Wed, 08 Aug 2012 02:47:48 +0000

http://www.bioconductor.org/packages/release/bioc/html/mcaGUI.html

mcaGUI

Microbial Community Analysis GUI

Bioconductor version: Release (2.10)

Microbial community analysis GUI for R using gWidgets.

Author: Wade K. Copeland, Vandhana Krishnan, Daniel Beck, Matt Settles, James Foster, Kyu-Chul Cho, Mitch Day, Roxana Hickey, Ursel M.E. Schutte, Xia Zhou, Chris Williams, Larry J. Forney, Zaid Abdo, Poor Man’s GUI (PMG) base code by John Verzani with contributions by Yvonnick Noel

Maintainer: Wade K. Copeland

To install this package, start R and enter:

    source("http://bioconductor.org/biocLite.R")
    biocLite("mcaGUI")

To cite this package in a publication, start R and enter:

    citation("mcaGUI")

Documentation

PDF	R Script	An_Introduction_and_User_Guide_for_mcaGUI.pdf
PDF		Reference Manual
Text		README

Details

biocViews	Bioinformatics, Clustering, GUI, Sequencing, Software, Visualization
Depends	lattice, MASS, proto, foreign, gWidgets(>= 0.0-36), gWidgetsRGtk2(>= 0.0-53),OTUbase, vegan, bpca
Imports
Suggests
System Requirements
License	GPL (>= 2)
URL	http://www.ibest.uidaho.edu/ibest/index.php
Depends On Me
Imports Me
Suggests Me
Version	1.4.0
Since	Bioconductor 2.8 (R-2.13)

Package Downloads

Package Source	mcaGUI_1.4.0.tar.gz
Windows Binary	mcaGUI_1.4.0.zip (32- & 64-bit)
MacOS 10.5 (Leopard) binary	mcaGUI_1.4.0.tgz
Package Downloads Report	Download Stats

Bioinformatics for personal genome interpretation

szypanther — Wed, 25 Jul 2012 09:37:15 +0000

http://bib.oxfordjournals.org/content/13/4/495.full

Key Points

Vast amounts of variation data from genome sequencing studies need to be analyzed to understand its association with various phenotypes.
Well-curated databases, reliable tools for gene prioritization and accurate methods for predicting the impact of variants will be essential for the interpretation of personal genomes.
Standard and unified protocols for testing the functional impact of genetic variations are critical for their accurate annotation.
Experimental studies and computational models describing the gene/protein interaction networks and aiming at capturing the fullcomplexity of the human genome will be key to leveraging personal genomic data for the precise diagnosis and effectivetreatment of disease.