小生这厢有礼了(BioFaceBook Personal Blog) » database

Batch download protein sequences from CMR (comprehensive microbial resource)

szypanther — Thu, 28 Feb 2013 08:11:00 +0000

NCBI 有时批量下载的protein sequence会有不一致时，可以从以下资源数据库下载（eg, eth195)

http://cmr.jcvi.org/cgi-bin/CMR/shared/MakeFrontPages.cgi?page=batchdownload

Batch download sequences from uniprot based on protein names

szypanther — Wed, 28 Nov 2012 04:20:39 +0000

Ok, I’ll do mine in English:

go to UniProt.org.
click tab “retrieve”
Paste list into text box. Click Retrieve button.
On results page, click FASTA download [ Download (30 KB*) | Open ] (Or you could click open just to have a look).

RDP Tutorials (16s Analysis)

szypanther — Wed, 12 Sep 2012 07:43:23 +0000

Workflows:

Processing 16S rRNA data using a unsupervised method

Processing 16S rRNA data using a supervised method

Processing functional gene data using a supervised method

Individual tools:

Using the Pipeline Initial Process

Align 16S rRNA sequences using Infernal Aligner

Using the RDP Classifier

Using the RDP MultiClassifier

Performing Complete Linkage Clustering

–Using the .clust File Results (for abundance stats, diversity stats, OTU matrix or rarefaction)

Performing statistical analysis (coming soon)

Align protein using HMMER3 Aligner

Frameshift-correction and closest match assignment by RDP FrameBot

Bioinformatics for personal genome interpretation

szypanther — Wed, 25 Jul 2012 09:37:15 +0000

http://bib.oxfordjournals.org/content/13/4/495.full

Key Points

Vast amounts of variation data from genome sequencing studies need to be analyzed to understand its association with various phenotypes.
Well-curated databases, reliable tools for gene prioritization and accurate methods for predicting the impact of variants will be essential for the interpretation of personal genomes.
Standard and unified protocols for testing the functional impact of genetic variations are critical for their accurate annotation.
Experimental studies and computational models describing the gene/protein interaction networks and aiming at capturing the fullcomplexity of the human genome will be key to leveraging personal genomic data for the precise diagnosis and effectivetreatment of disease.

Databases and resources for personal genome interpretation

szypanther — Wed, 25 Jul 2012 09:21:48 +0000

Database	URL	Description
Short variations—SNVs, short indels
1000 Genomes	http://www.1000genomes.org	Human short variants and inferred genotypes
dbSNP	http://www.ncbi.nlm.nih.gov/projects/SNP	Short variants from all species
HapMap	http://www.hapmap.org	Human short variants and population group haplotypes

Structural variations—structural rearrangements, CNVs, large indels
dbVar	http://www.ncbi.nlm.nih.gov/dbvar	Structural variants from all species
DGV	http://projects.tcag.ca/variation	Structural variants from healthy human controls
DGVa	http://www.ebi.ac.uk/dgva	Structural variants from all species

General variants associated with phenotypes
ClinVar	http://www.ncbi.nlm.nih.gov/clinvar	Human variant–disease associations (in development)
HGMD	http://www.hgmd.org	Human variant–disease associations (inherited diseases)
OMIM	http://www.omim.org	Human variant–disease associations (includes extensive gene and phenotype descriptions)
SwissVar	http://swissvar.expasy.org	Human variant–disease associations (non-synonymous SNVs only)

GWAS and other association studies
dbGaP	http://www.ncbi.nlm.nih.gov/gap	Controlled access to individual genotype/phenotype data from association studies
EGA	http://www.ebi.ac.uk/ega	Controlled access to individual genotype/phenotype data from association studies
GAD	http://geneticassociationdb.nih.gov	Mainly complex disease SNVs from association studies
NHGRI GWAS Catalog	http://www.genome.gov/gwastudies	Significant SNVs from GWAS

Cancer genes and variants
ICGC	http://www.icgc.org	Somatic variants from tumor sequencing projects
COSMIC	http://sanger.ac.uk/genetics/CGP/cosmic	Somatic variants from tumor sequencing and literature
Cancer Gene Census	http://sanger.ac.uk/genetics/CGP/Census	Comprehensive list of cancer-related genes
Cancer Gene Index	http://ncicb.nci.nih.gov/NCICB/projects/cgdcp	Comprehensive list of cancer-related genes, including gene–disease and gene–drug relationships
TCGA	http://cancergenome.nih.gov	Somatic variants from tumor sequencing projects

Pharmacogenomic genes and variants
DrugBank	http://drugbank.ca	Drug properties and protein amino acid target sequences
PharmGKB	http://www.pharmgkb.org	Curated and text-mined variant–drug response associations

Crowdsourced genes and variants
Gene Wiki	http://en.wikipedia.org/wiki/Portal:Gene_Wiki	Human gene/protein annotations
SNPedia	http://www.snpedia.com	Human SNV–disease associations
WikiGenes	https://www.wikigenes.org	Gene annotations from all species

Viewers of the structural and functional impact of variants
DMDM	http://bioinf.umbc.edu/dmdm	Aggregates human protein mutations at individual domain positions
LS-SNP/PDB	http://ls-snp.icm.jhu.edu/ls-snp-pdb	Variant/PDB structure viewer (includes multiple filters for selection of variants)
MutDB	http://mutdb.org	Variant/PDB structure viewer (includes SIFT predictions for nonsynonymous mutations)
SAAPdb	http://bioinf.org.uk/saap/db	Variant/PDB structure viewer (includes impact on physico-chemical and functional features)
StSNP	http://ilyinlab.org/StSNP	Variant/PDB structure viewer (includes physico-chemical impact for nonsynonymous mutations)
SNPeffect	http://snpeffect.vib.be	Variant/PDB structure viewer (includes predictions for variants to cause protein aggregation)
TopoSNP	http://gila-fw.bioengr.uic.edu/snp/toposnp	Variant/PDB structure viewer (includes location of variant on surface, in pocket or in core)

blast2go 本地化数据库安装运行以及简单在线调用

szypanther — Thu, 28 Jun 2012 05:28:57 +0000

最近需要对预测到的基因进行个注释工作，着手进行blast2go的工作：

最简单方式:运用官网的免费在线调用数据库方式，（需要安装好JAVA Java Runtime Environment (JRE) from http://www.java.com/download）

步骤如下：

（1）进入官网http://www.blast2go.com/b2glaunch/start-blast2go

选择相应大小的内存，点击here，如未能直接在线运行，则会让你保存并下载blast2go.jnlp 文件。

（2）然后直接在命令行运行 javaws blast2go.jnlp 回车即可出现界面，剩下的就是简单点击界面和运行了！

##############################################

本地化数据库命令行运行方式：

B2G4PIPE – Blast2GO without graphical interface

1.从http://www.blast2go.com/b2glaunch/resources

下载相应资源

http://www.blast2go.com/data/blast2go/b2g4pipe_v2.5.zip

http://www.blast2go.com/data/blast2go/local_b2g_db_tutorial_0809.zip

下载b2g database所需文件:

http://archive.geneontology.org/latest-full/go-assocdb-data.gz

ftp://ftp.ncbi.nlm.nih.gov/gene/DATA/gene_info.gz

ftp://ftp.ncbi.nlm.nih.gov/gene/DATA/gene2accession.gz

ftp://ftp.pir.georgetown.edu/databases/idmapping/idmapping.tb.gz

(可选,依据mysql版本)

替换b2g_db.sql 里的TYPE=MyISAM 为 ENGINE=MyISAM

同上替换go_201110-assocdb-data里:

sed -i ‘s/TYPE=MyISAM/ENGINE=MyISAM, DEFAULT CHARACTER SET latin1/’ go_201110-assocdb-data

2. 编辑后运行tutorial 里download_and_install.sh 或像下面这样手工运行:

3. 编辑并运行b2g_db.sql:

mysql -u root –password=xxx < b2g_db.sql

导入数据时可能出现Variable ‘sql_mode’ can’t be set to the value of ‘NULL’ 之类的错误, 我的方法是把那行删掉. 然后继续导入:

把剩下的写入新文件: sed -n ‘12229,$p’ go_201110-assocdb-data > tmp.txt

然后再导入tmp.txt就可以了

12229表示从这行开始(最好包含drop table 那行), $表示文件最后一行

自动建立MySQL database结构：

mysql -h localhost -P 3306 -u shenzy -p blast2go < b2g_db.sql （创建名为blast2go数据库）或用如下默认创建数据库为b2g

1	`mysql t -u root -p < b2g_db.sql`

shenzy@shenzy-ubuntu:/mnt/disk_xp/linux_shenzy$ mysql -s -u shenzy -p blast2go < go_201206-assocdb-data

mysql -s -u root -p b2g -e”LOAD DATA LOCAL INFILE ‘/database/gene2accession’ INTO TABLE gene2accession FIELDS TERMINATED BY ‘\t’ LINES TERMINATED BY ‘\n';”

shenzy@shenzy-ubuntu:/mnt/disk_xp/linux_shenzy$ mysql -s -u shenzy -p blast2go -e”LOAD DATA LOCAL INFILE ‘/mnt/disk_xp/linux_shenzy/gene_info’ INTO TABLE gene_info FIELDS TERMINATED BY ‘\t’ LINES TERMINATED BY ‘\n';”

java -Xms4G -Xmx8G -cp /home/shenzy/work/soft/blast2go/b2g4pipe/blast2go.jar:/home/shenzy/work/soft/blast2go/b2g4pipe/ext/mysql-connector-java-3.0.11-stable-bin.jar es.blast2go.prog.util.ImportPIR “idmapping.tb” “localhost” “b2g” “root” “shenzy” boolean-debug

#注意boolean-debug 仅仅是因为提示需要，否则命令行不能运行，报错如下：

Version: 17-09-2010

Usage: es.blast2go.prog.util.ImportPIR id-mapping-file db-host db-name db-user db-passwd boolean-debug

INSTALL MicrobeDB

szypanther — Thu, 17 May 2012 03:32:58 +0000

shenzy@shenzy-ubuntu:/usr/local/lib/perl/5.10.1/MicrobeDB/scripts$ load_genome.pl -d /home/shenzy/Downloads/unpublished_genomes/Pseudomonas_aeruginosa_LESB58/

2012/05/15 16:01:48> Working on /home/shenzy/Downloads/unpublished_genomes/Pseudomonas_aeruginosa_LESB58/

2012/05/15 16:01:48> Parsing file: /home/shenzy/Downloads/unpublished_genomes/Pseudomonas_aeruginosa_LESB58/Pseudomonas_aeruginoas_LESB58.gbk

2012/05/15 16:01:48> Can’t find file:/home/shenzy/Downloads/unpublished_genomes/NCBI_completegenomes.txt . GenomeProject table will not contain much organism information.

2012/05/15 16:01:48> Can’t find file:/home/shenzy/Downloads/unpublished_genomes/NCBI_completegenomes.txt . GenomeProject table will be missing a few fields of information.

DBD::mysql::st execute failed: Unknown column ‘distance_calculated’ in ‘field list’ at ../..//MicrobeDB/FullUpdate.pm line 229.

2012/05/15 16:01:48> Couldn’t add the following to microbedb: /home/shenzy/Downloads/unpublished_genomes/Pseudomonas_aeruginosa_LESB58/ ! Reason: DBD::mysql::st execute failed: Unknown column ‘distance_calculated’ in ‘field list’ at ../..//MicrobeDB/FullUpdate.pm line 229.

2012/05/15 16:01:48> at ./load_genome.pl line 92

Couldn’t add the following to microbedb: /home/shenzy/Downloads/unpublished_genomes/Pseudomonas_aeruginosa_LESB58/ ! Reason: DBD::mysql::st execute failed: Unknown column ‘distance_calculated’ in ‘field list’ at ../..//MicrobeDB/FullUpdate.pm line 229.

at ./load_genome.pl line 92

shenzy@shenzy-ubuntu:/usr/local/lib/perl/5.10.1/MicrobeDB/scripts$

where i can download the NCBI_completegenomes.txt ? could u help me, thanks!

Hello,Sorry, but we had some untested code changes to MicrobeDB just
recently that caused problems. I have fixed the bugs in the current
version of MicrobeDB. You should update MicrobeDB using the
information here:
https://github.com/mlangill/MicrobeDB/blob/master/information/UPDATE/UPDATE.mdThe NCBI_completegenomes.txt file is produced when the
download_version.pl script is run.

If you are trying to test your installation you can do so with this command:
./download_load_and_delete_old_version.pl -d ~/Downloads/ -s ‘-s
Pseudomonas_aeruginosa_LES’

Programming with the MicrobeDB API

If you know how to program in Perl you can use the MicrobeDB Perl API which allows you to retrieve data without constructing MySQL queries.

Example of a simple perl script using the MicrobeDB API that searches for all ‘recA’ genes and prints them in ‘Fasta’ format:

#Import the MicrobeDB API use lib '/your/path/to/MicrobeDB'; use MicrobeDB::Search; #intialize the search object $search_obj= MicrobeDB::Search(); #create the object that has properties that must match in the database $gene_obj= MicrobeDB::Gene(gene_name => 'recA'); #do the actual search @genes = $search_obj->object_search($gene_obj); #loop through each gene we found and print in FASTA format foreach my $gene (@genes){ print'>',$gene->gid(),"\n",$gene->gene_seq(),"\n"; }

See more examples using the MicrobeDB API in information/example_scripts.

Extending MicrobeDB

MicrobeDB can be extended to include additional types of custom information
The best way to extend MicrobeDB is to create your own tables with the fields of your choice and use a stable NCBI based identifier to “link” the tables. The reason to use these types of ids instead of the primary MicrobeDB ids is that the NCBI ids will remain the same between MicrobeDB versions. Use the following fields:
- To extend a Genomeproject, use the field “gp_id”
- To extend a Replicon, use the field “rep_accnum”
- To extend a Gene, use the field “gid” or “pid”
For example, imagine you wanted to store SNP data. You want to store the position of the SNP, an in-house experiment id, and the base at that position. Then you would want to create a new table and use the rep_accnum field to join the MicrobeDB replicon table to your in-house table. Your columns would be “your_primary_snp_id”, “rep_accnum”, “your_experiment_id”, “snp_position”, “snp_base”.