future.apply で tryCatch した時にエラーで終了する
future, future.applyは簡単な並列化にとても便利。
最近使った際に、tryCatchでラップしてるにもかかわらずエラーで処理が中止される現象に遭遇。検索すると、下記ページが。
何事も同じようなエラーに遭遇した人がいるものですね。
エラー処理内で使っていた message() が原因だそうで、これを
*1 * .9))); --theme-primary-custom-200: hsl(var(--theme-base-primary-color-h), var(--theme-base-primary-color-s), calc(var(--theme-base-primary-color-l) + *2 * .75))); --theme-primary-custom-300: hsl(var(--theme-base-primary-color-h), var(--theme-base-primary-color-s), calc(var(--theme-base-primary-color-l) + *3 * .5))); --theme-primary-custom-400: hsl(var(--theme-base-primary-color-h), var(--theme-base-primary-color-s), var(--theme-base-primary-color-l)); --theme-primary-custom-500: hsl(var(--theme-base-primary-color-h), var(--theme-base-primary-color-s), calc(var(--theme-base-primary-color-l) + (var(--theme-base-primary-color-l) * -.3))); --theme-primary-custom-600: hsl(var(--theme-base-primary-color-h), var(--theme-base-primary-color-s), calc(var(--theme-base-primary-color-l) + (var(--theme-base-primary-color-l) * -.6))); --theme-primary-custom: var(--theme-primary-custom-400); --theme-button-color: initial; --theme-button-hover-color: initial; --theme-button-hover-background-color: initial; --theme-button-active-background-color: initial; --theme-button-selected-color: initial; --theme-button-selected-background-color: initial; --theme-button-outlined-border-color: initial; --theme-button-outlined-selected-border-color: initial; --theme-button-primary-color: initial; --theme-button-primary-active-color: initial; --theme-button-primary-hover-color: initial; --theme-button-primary-selected-color: initial; --theme-button-primary-background-color: initial; --theme-button-primary-active-background-color: initial; --theme-button-primary-hover-background-color: initial; --theme-button-primary-selected-background-color: initial; --theme-background-color: var(--white); --theme-background-position: top left; --theme-background-repeat: repeat; --theme-background-size: auto; --theme-background-attachment: auto; --theme-content-background-color: var(--white); --theme-content-border-color: var(--black-225); --theme-header-background-color: var(--theme-primary); --theme-header-background-position: center left; --theme-header-background-repeat: repeat; --theme-header-background-size: auto; --theme-header-background-border-bottom: 0; --theme-header-link-color: var(--theme-primary); --theme-header-sponsored-color: hsla(0,0%,100%,0.4); --theme-header-foreground-color: transparent; --theme-header-foreground-position: bottom right; --theme-header-foreground-repeat: no-repeat; --theme-header-foreground-size: auto; --theme-footer-background-color: hsl(210,8%,5%); --theme-footer-background-position: top left; --theme-footer-background-repeat: no-repeat; --theme-footer-background-size: auto; --theme-footer-background-border-top: 0; --theme-footer-title-color: var(--black-250); --theme-footer-text-color: var(--black-300); --theme-footer-link-color: var(--black-300); --theme-footer-link-color-hover: var(--black-200); --theme-footer-divider-color: var(--black-500); --theme-footer-padding-top: 0; --theme-footer-padding-bottom: 0; --theme-link-color: initial; --theme-link-color-hover: initial; --theme-link-color-visited: initial; --theme-body-font-family: var(--ff-sans); --theme-body-font-color: var(--black-600); --theme-post-title-font-family: var(--ff-sans); --theme-post-title-color: var(--theme-link-color); --theme-post-title-color-hover: var(--theme-link-color-hover); --theme-post-title-color-visited: var(--theme-link-color-visited); --theme-post-body-font-family: var(--ff-sans); --theme-post-owner-background-color: var(--theme-secondary-100); --theme-post-owner-new-background-color: var(--theme-secondary-200); --theme-topbar-bottom-border: 1px solid var(--black-225); color: #0c0d0e; letter-spacing: normal; orphans: 2; text-align: left; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: #ffffff; text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">message(conditionMessage(e))
とするとたしかに動いた。
ggplot2 scale_* で drop=FALSE が動かない（レジェンドに空白がうまれる）
MetaPhlan --help
help一覧 (v4.1.1)
PCが重いときの参照用にリストしておく。
```
usage: metaphlan --input_type {fastq,fasta,bowtie2out,sam} [--force] [--bowtie2db METAPHLAN_BOWTIE2_DB] [-x INDEX] [--bt2_ps BowTie2 presets] [--bowtie2_exe BOWTIE2_EXE]
[--bowtie2_build BOWTIE2_BUILD] [--bowtie2out FILE_NAME] [--min_mapq_val MIN_MAPQ_VAL] [--no_map] [--tmp_dir] [--tax_lev TAXONOMIC_LEVEL] [--min_cu_len] [--min_alignment_len]
[--add_viruses] [--ignore_eukaryotes] [--ignore_bacteria] [--ignore_archaea] [--ignore_ksgbs] [--ignore_usgbs] [--stat_q] [--perc_nonzero] [--ignore_markers IGNORE_MARKERS]
[--avoid_disqm] [--stat] [-t ANALYSIS TYPE] [--nreads NUMBER_OF_READS] [--pres_th PRESENCE_THRESHOLD] [--clade] [--min_ab] [--profile_vsc] [--vsc_out VSC_OUT]
[--vsc_breadth VSC_BREADTH] [-o output file] [--sample_id_key name] [--use_group_representative] [--sample_id value] [-s sam_output_file] [--legacy-output] [--CAMI_format_output]
[--unclassified_estimation] [--mpa3] [--biom biom_output] [--mdelim mdelim] [--nproc N] [--subsampling SUBSAMPLING] [--subsampling_output SUBSAMPLING_OUTPUT]
[--subsampling_paired SUBSAMPLING_PAIRED] [-1 FORWARD_READS] [-2 REVERSE_READS] [--mapping_subsampling] [--subsampling_seed SUBSAMPLING_SEED] [--install] [--offline]
[--force_download] [--read_min_len READ_MIN_LEN] [-v] [-h]
[INPUT_FILE] [OUTPUT_FILE]
DESCRIPTION
MetaPhlAn version 4.1.1 (11 Mar 2024):
METAgenomic PHyLogenetic ANalysis for metagenomic taxonomic profiling.
AUTHORS: Aitor Blanco-Miguez (aitor.blancomiguez@unitn.it), Francesco Beghini (francesco.beghini@unitn.it), Moreno Zolfo (moreno.zolfo@unitn.it), Nicola Segata (nicola.segata@unitn.it), Duy Tin Truong, Francesco Asnicar (f.asnicar@unitn.it), Claudia Mengoni (claudia.mengoni@unitn.it)
COMMON COMMANDS
We assume here that MetaPhlAn is installed using the several options available (pip, conda, PyPi)
Also BowTie2 should be in the system path with execution and read permissions, and Perl should be installed)
========== MetaPhlAn clade-abundance estimation =================
The basic usage of MetaPhlAn consists in the identification of the clades (from phyla to species )
present in the metagenome obtained from a microbiome sample and their
relative abundance. This correspond to the default analysis type (-t rel_ab).
* Profiling a metagenome from raw reads:
$ metaphlan metagenome.fastq --input_type fastq -o profiled_metagenome.txt
* You can take advantage of multiple CPUs and save the intermediate BowTie2 output for re-running
MetaPhlAn extremely quickly:
$ metaphlan metagenome.fastq --bowtie2out metagenome.bowtie2.bz2 --nproc 5 --input_type fastq -o profiled_metagenome.txt
* If you already mapped your metagenome against the marker DB (using a previous MetaPhlAn run), you
can obtain the results in few seconds by using the previously saved --bowtie2out file and
specifying the input (--input_type bowtie2out):
$ metaphlan metagenome.bowtie2.bz2 --nproc 5 --input_type bowtie2out -o profiled_metagenome.txt
* bowtie2out files generated with MetaPhlAn versions below 3 are not compatibile.
Starting from MetaPhlAn 3.0, the BowTie2 ouput now includes the size of the profiled metagenome and the average read length.
If you want to re-run MetaPhlAn using these file you should provide the metagenome size via --nreads:
$ metaphlan metagenome.bowtie2.bz2 --nproc 5 --input_type bowtie2out --nreads 520000 -o profiled_metagenome.txt
* You can also provide an externally BowTie2-mapped SAM if you specify this format with
--input_type. Two steps: first apply BowTie2 and then feed MetaPhlAn with the obtained sam:
$ bowtie2 --sam-no-hd --sam-no-sq --no-unal --very-sensitive -S metagenome.sam -x ${mpa_dir}/metaphlan_databases/mpa_v30_CHOCOPhlAn_201901 -U metagenome.fastq
$ metaphlan metagenome.sam --input_type sam -o profiled_metagenome.txt
* We can also natively handle paired-end metagenomes, and, more generally, metagenomes stored in
multiple files (but you need to specify the --bowtie2out parameter):
$ metaphlan metagenome_1.fastq,metagenome_2.fastq --bowtie2out metagenome.bowtie2.bz2 --nproc 5 --input_type fastq
-------------------------------------------------------------------
========== Marker level analysis ============================
MetaPhlAn introduces the capability of characterizing organisms at the strain level using non
aggregated marker information. Such capability comes with several slightly different flavours and
are a way to perform strain tracking and comparison across multiple samples.
Usually, MetaPhlAn is first ran with the default -t to profile the species present in
the community, and then a strain-level profiling can be performed to zoom-in into specific species
of interest. This operation can be performed quickly as it exploits the --bowtie2out intermediate
file saved during the execution of the default analysis type.
* The following command will output the abundance of each marker with a RPK (reads per kilo-base)
higher 0.0. (we are assuming that metagenome_outfmt.bz2 has been generated before as
shown above).
$ metaphlan -t marker_ab_table metagenome_outfmt.bz2 --input_type bowtie2out -o marker_abundance_table.txt
The obtained RPK can be optionally normalized by the total number of reads in the metagenome
to guarantee fair comparisons of abundances across samples. The number of reads in the metagenome
needs to be passed with the '--nreads' argument
* The list of markers present in the sample can be obtained with '-t marker_pres_table'
$ metaphlan -t marker_pres_table metagenome_outfmt.bz2 --input_type bowtie2out -o marker_abundance_table.txt
The --pres_th argument (default 1.0) set the minimum RPK value to consider a marker present
* The list '-t clade_profiles' analysis type reports the same information of '-t marker_ab_table'
but the markers are reported on a clade-by-clade basis.
$ metaphlan -t clade_profiles metagenome_outfmt.bz2 --input_type bowtie2out -o marker_abundance_table.txt
* Finally, to obtain all markers present for a specific clade and all its subclades, the
'-t clade_specific_strain_tracker' should be used. For example, the following command
is reporting the presence/absence of the markers for the B. fragilis species and its strains
the optional argument --min_ab specifies the minimum clade abundance for reporting the markers
$ metaphlan -t clade_specific_strain_tracker --clade s__Bacteroides_fragilis metagenome_outfmt.bz2 --input_type bowtie2out -o marker_abundance_table.txt
-------------------------------------------------------------------
positional arguments:
INPUT_FILE the input file can be:
* a fastq file containing metagenomic reads
OR
* a BowTie2 produced SAM file.
OR
* an intermediary mapping file of the metagenome generated by a previous MetaPhlAn run
If the input file is missing, the script assumes that the input is provided using the standard
input, or named pipes.
IMPORTANT: the type of input needs to be specified with --input_type
OUTPUT_FILE the tab-separated output file of the predicted taxon relative abundances
[stdout if not present]
Required arguments:
--input_type {fastq,fasta,bowtie2out,sam}
set whether the input is the FASTA file of metagenomic reads or
the SAM file of the mapping of the reads against the MetaPhlAn db.
Mapping arguments:
--force Force profiling of the input file by removing the bowtie2out file
--bowtie2db METAPHLAN_BOWTIE2_DB
Folder containing the MetaPhlAn database. You can specify the location by exporting the DEFAULT_DB_FOLDER variable in the shell.[default /home/nsato2/.local/lib/python3.12/site-packages/metaphlan/metaphlan_databases]
-x INDEX, --index INDEX
Specify the id of the database version to use. If "latest", MetaPhlAn will get the latest version.
If an index name is provided, MetaPhlAn will try to use it, if available, and skip the online check.
If the database files are not found on the local MetaPhlAn installation they
will be automatically downloaded [default latest]
--bt2_ps BowTie2 presets
Presets options for BowTie2 (applied only when a FASTA file is provided)
The choices enabled in MetaPhlAn are:
* sensitive
* very-sensitive
* sensitive-local
* very-sensitive-local
[default very-sensitive]
--bowtie2_exe BOWTIE2_EXE
Full path and name of the BowTie2 executable. This option allowsMetaPhlAn to reach the executable even when it is not in the system PATH or the system PATH is unreachable
--bowtie2_build BOWTIE2_BUILD
Full path to the bowtie2-build command to use, deafult assumes that 'bowtie2-build is present in the system path
--bowtie2out FILE_NAME
The file for saving the output of BowTie2
--min_mapq_val MIN_MAPQ_VAL
Minimum mapping quality value (MAPQ) [default 5]
--no_map Avoid storing the --bowtie2out map file
--tmp_dir The folder used to store temporary files [default is the OS dependent tmp dir]
Post-mapping arguments:
--tax_lev TAXONOMIC_LEVEL
The taxonomic level for the relative abundance output:
'a' : all taxonomic levels
'k' : kingdoms
'p' : phyla only
'c' : classes only
'o' : orders only
'f' : families only
'g' : genera only
's' : species only
't' : SGBs only
[default 'a']
--min_cu_len minimum total nucleotide length for the markers in a clade for
estimating the abundance without considering sub-clade abundances
[default 2000]
--min_alignment_len The sam records for aligned reads with the longest subalignment
length smaller than this threshold will be discarded.
[default None]
--add_viruses Together with --mpa3, allow the profiling of viral organisms
--ignore_eukaryotes Do not profile eukaryotic organisms
--ignore_bacteria Do not profile bacterial organisms
--ignore_archaea Do not profile archeal organisms
--ignore_ksgbs Do not profile known SGBs (together with --sgb option)
--ignore_usgbs Do not profile unknown SGBs (together with --sgb option)
--stat_q Quantile value for the robust average
[default 0.2]
--perc_nonzero Percentage of markers with a non zero relative abundance for misidentify a species
[default 0.33]
--ignore_markers IGNORE_MARKERS
File containing a list of markers to ignore.
--avoid_disqm Deactivate the procedure of disambiguating the quasi-markers based on the
marker abundance pattern found in the sample. It is generally recommended
to keep the disambiguation procedure in order to minimize false positives
--stat Statistical approach for converting marker abundances into clade abundances
'avg_g' : clade global (i.e. normalizing all markers together) average
'avg_l' : average of length-normalized marker counts
'tavg_g' : truncated clade global average at --stat_q quantile
'tavg_l' : truncated average of length-normalized marker counts (at --stat_q)
'wavg_g' : winsorized clade global average (at --stat_q)
'wavg_l' : winsorized average of length-normalized marker counts (at --stat_q)
'med' : median of length-normalized marker counts
[default tavg_g]
Additional analysis types and arguments:
-t ANALYSIS TYPE Type of analysis to perform:
* rel_ab: profiling a metagenomes in terms of relative abundances
* rel_ab_w_read_stats: profiling a metagenomes in terms of relative abundances and estimate the number of reads coming from each clade.
* reads_map: mapping from reads to clades (only reads hitting a marker)
* clade_profiles: normalized marker counts for clades with at least a non-null marker
* marker_ab_table: normalized marker counts (only when > 0.0 and normalized by metagenome size if --nreads is specified)
* marker_counts: non-normalized marker counts [use with extreme caution]
* marker_pres_table: list of markers present in the sample (threshold at 1.0 if not differently specified with --pres_th
* clade_specific_strain_tracker: list of markers present for a specific clade, specified with --clade, and all its subclades
[default 'rel_ab']
--nreads NUMBER_OF_READS
The total number of reads in the original metagenome. It is mandatory when the --input_type is a SAM file.
--pres_th PRESENCE_THRESHOLD
Threshold for calling a marker present by the -t marker_pres_table option
--clade The clade for clade_specific_strain_tracker analysis
--min_ab The minimum percentage abundance for the clade in the clade_specific_strain_tracker analysis
Viral Sequence Clusters Analisys:
--profile_vsc Add this parameter to profile Viruses with VSCs approach.
--vsc_out VSC_OUT Path to the VSCs breadth-of-coverage output file
--vsc_breadth VSC_BREADTH
Minimum Breadth of Coverage for a Viral Group to be reported.
Default is 0.75 (at least 75 percent breadth to report)
Output arguments:
-o output file, --output_file output file
The output file (if not specified as positional argument)
--sample_id_key name Specify the sample ID key for this analysis. Defaults to 'SampleID'.
--use_group_representative
Use a species as representative for species groups.
--sample_id value Specify the sample ID for this analysis. Defaults to 'Metaphlan_Analysis'.
-s sam_output_file, --samout sam_output_file
The sam output file
--legacy-output Old MetaPhlAn2 two columns output
--CAMI_format_output Report the profiling using the CAMI output format
--unclassified_estimation
Scale relative abundances to the number of reads mapping to identified clades in order to estimate unclassified taxa
--mpa3 Perform the analysis using the MetaPhlAn 3 algorithm
--biom biom_output, --biom_output_file biom_output
If requesting biom file output: The name of the output file in biom format
--mdelim mdelim, --metadata_delimiter_char mdelim
Delimiter for bug metadata: - defaults to pipe. e.g. the pipe in k__Bacteria|p__Proteobacteria
Other arguments:
--nproc N The number of CPUs to use for parallelizing the mapping [default 4]
--subsampling SUBSAMPLING
Specify the number of reads to be considered from the input metagenomes [default None]
--subsampling_output SUBSAMPLING_OUTPUT
The output file for the subsampled reads. If --subsampling_paired is specified two files are created with suffixes R1 and R2. If not specified the subsampled reads will not be saved.
--subsampling_paired SUBSAMPLING_PAIRED
Specify the number of paired reads to be considered from the input metagenomes [default None]
-1 FORWARD_READS Specify the fastq file with forward reads of the input metagenomes. Reads are assumed to be in the same order in the forward and reverse files! [default None]
-2 REVERSE_READS Specify the fastq file with reverse reads of the input metagenomes. Reads are assumed to be in the same order in the forward and reverse files! [default None]
--mapping_subsampling
If used, the subsamping will be done on the mapping results instead of on the reads.
--subsampling_seed SUBSAMPLING_SEED
Random seed to use in the selection of the subsampled reads. Choose "random
for a random behaviour
--install Only checks if the MetaPhlAn DB is installed and installs it if not. All other parameters are ignored.
--offline If used, MetaPhlAn will not check for new database updates.
--force_download Force the re-download of the latest MetaPhlAn database.
--read_min_len READ_MIN_LEN
Specify the minimum length of the reads to be considered when parsing the input file with 'read_fastx.py' script, default value is 70
-v, --version Prints the current MetaPhlAn version and exit
-h, --help show this help message and exit
```
iMetaOmicsが発刊されていた。
iMetaについて、発刊当時記事を書いたが、さらに姉妹誌のiMetaOmicsが発刊されていた。
https://onlinelibrary.wiley.com/journal/29969514
まだインパクトファクターは付与されていないが、こちらも高くなるのだろうか。
ただ、錚々たるソフトウェアがリストされていたiMetaと比べて、現状著名なソフトウェアはパブリッシュされていないよう（そういうのはiMetaにいくのかな）。
引き続きメタゲノム×バイオインフォマティクスの新興ジャーナルとして注目である。
lefser がパブリッシュされている
マイクロバイオーム解析においてよく用いられている（た）LEfSeのRパッケージがBioinformaticsに掲載されている。
バープロットやクレイドグラムがおなじみだが、似たような図も出力できるみたい。もともとPython2系依存であったり、多重検定の補正がされていない問題があったりと現在は使いにくかったが、この実装があれば使いやすくなりそうですね。
npj viruses が発刊されている
npj viruses という npj シリーズのジャーナルが発刊されている模様。
2023/9からの論文が収載されており、新しいジャーナル。
本邦からも1報あり、クオリティの高い論文が掲載されている模様。
Attenuation of A(H7N9) influenza virus infection in mice exposed to cigarette smoke | npj Viruses
HUMAnN2 と HUMANN3 のデータベース推奨について
微生物叢の機能プロファイルとしてよく用いられてるHUMAnN。
HUMAnN2では、推奨が
Download one of the following databases (replacing $DIR with the location to store the database):
To download the UniRef90 EC filtered database (RECOMMENDED, approx. size = 846 MB):
$ humann2_databases –download uniref uniref90_ec_filtered_diamond $DIR
To download the full UniRef90 database (approx. size = 11 GB):
$ humann2_databases –download uniref uniref90_diamond $DIR
To download the UniRef50 EC filtered database (approx. size = 239 MB):
$ humann2_databases –download uniref uniref50_ec_filtered_diamond $DIR
To download the full UniRef50 database (approx. size = 4.6 GB):
$ humann2_databases –download uniref uniref50_diamond $DIR
なのが、現状版だと
Download a translated search database providing $INSTALL_LOCATION as the location to install the database:
To download the full UniRef90 database (20.7GB, recommended):
$ humann_databases --download uniref uniref90_diamond $INSTALL_LOCATION
To download the EC-filtered UniRef90 database (0.9GB):$ humann_databases --download uniref uniref90_ec_filtered_diamond $INSTALL_LOCATION
To download the full UniRef50 database (6.9GB):$ humann_databases --download uniref uniref50_diamond $INSTALL_LOCATION
To download the EC-filtered UniRef50 database (0.3GB):$ humann_databases --download uniref uniref50_ec_filtered_diamond $INSTALL_LOCATION
UniRef90のフルが推奨されている。正直 uncharacterized proteinにほぼ興味がない場合とかが殆どなので、軽いしEC-filteredが推奨でも良いような気もする。
この経緯を調べてみたがフォーラムなどでは見つからなかった。知っている方いたら教えて下さい。
