Skip to content

BU-ISCIII/buisciii-tools

Repository files navigation

buisciii-tools

python_lint Code style: black

BU-ISCIII provides a serie or services in its portfolio for supporting bioinformatics analysis to the labs in the Institute of Health Carlos III. bu-isciii tools is a set of helper tools for management of these bioinformatics analysis together with the LIMS (iSkyLIMS)

Installation

Micromamba and pip

micromamba create -n buisciii -f environment.yml micromamba activate buisciii pip install buisciii-tools

or

git checkout main conda create -n buisciii -f environment.yml conda activate pip install buisciii-tools

Dev version

If you want to install the latest code in the repository:

micromamba create -n buisciii_dev -f environment.yml micromamba activate buisciii_dev pip install --force-reinstall --upgrade git+https://github.com/bu-isciii/buisciii-tools.git@develop

or locally:

git checkout develop micromamba create -n buisciii_dev -f environment.yml micromamba activate buisciii_dev pip install .

Usage

Command-line

Run bu-isciii tools:

buisciii --help

Outputs the following:

Usage: buisciii [OPTIONS] COMMAND [ARGS]... Options: --version Show the version and exit. -v, --verbose Print verbose output to the console. -l, --log-file <filename> Save a verbose log to a file. -u, --api_user TEXT User for the API logging -p, --api_password TEXT Password for the API logging -c, --cred_file TEXT Config file with API logging credentials -D, --debug Show the full traceback on error for debugging purposes. -d, --dev Develop settings --help Show this message and exit. Commands: list List available bu-isciii services. clean Service cleaning. new-service Create new service, it will create folder and copy... scratch Copy service folder to scratch directory for execution. copy-sftp Copy resolution FOLDER to sftp, change status of... finish Service cleaning, remove big files, rename folders before... bioinfo-doc Create the folder documentation structure in bioinfo_doc... archive Archive services or retrieve services from archive autoclean-sftp Clean old sftp services fix-permissions Fix permissions

list

List available bu-isciii services:

bu-isciii list

Help:

Usage: buisciii list [OPTIONS] <service> List available bu-isciii services. Options: --help Show this message and exit.

Output:

┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ Service name ┃ Description ┃ Github ┃ ┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ IRMA │ Influenza fragment reconstruction and variant detection │ https://github.com/CDCgov/irma │ │ assembly_annotation │ nf-core/bacass: Simple bacterial assembly and annotation pipeline │ https://github.com/nf-core/bacass/tree/2.4.0 │ │ blast_nt │ Alignment of de novo assembly contigs to database │ - │ │ characterization │ Multi-Locus Sequence Typing (MLST), analysis of virulence factors, antimicrobial resistance and plasmid characterization │ - │ │ exomeeb │ Eukaria: Variant calling and annotation for a sequencing panel (e.g. epidermolysis gene panel, mouse or rat gene panel) │ - │ │ exometrio │ Human: Exome sequencing for variant calling, annotation and inheritance filtering │ - │ │ freebayes_outbreak │ Variant calling, annotation and SNP-based outbreak analysis (e.g. diploid fungal outbreak) │ - │ │ lowfreq_panel │ Low frequency variant calling from enrichment panel │ - │ │ pikavirus │ PikaVirus, a mapping-based tool for metagenome analysis of virus │ https://github.com/BU-ISCIII/PikaVirus │ │ plasmidid_assembly │ Plasmid identification tool based on mapping and assisted by assembly │ https://github.com/BU-ISCIII/plasmidID │ │ rnaseq │ RNA-Seq analysis │ https://github.com/nf-core/rnaseq │ │ seek_and_destroy │ Simple pipeline for basic quality control, host removal and exploratory analysis of samples │ https://github.com/GuilleGorines/Seek-Destroy │ │ snippy │ Rapid haploid variant calling and core genome alignment │ https://github.com/tseemann/snippy │ │ taxprofiler │ Highly parallelised multi-taxonomic profiling of shotgun short- and long-read metagenomic data │ https://github.com/nf-core/taxprofiler │ │ tbprofiler │ Mycobacterium tuberculosis variant calling and resistance prediction using TBProfiler │ https://github.com/jodyphelan/TBProfiler │ │ viralrecon │ Viral genome reconstruction analysis for SARS-COV-2 data │ https://github.com/BU-ISCIII/viralrecon │ │ wgmlst_chewbbaca │ Multilocus sequence typing (MLST) using chewBBACA │ https://github.com/B-UMMI/chewBBACA │ │ wgmlst_taranis │ Multilocus sequence typing (MLST) using Taranis │ https://github.com/BU-ISCIII/taranis │ │ wgstrio │ Human: Whole genome sequencing for SNPs variant calling, annotation and inheritance filtering │ - │ └─────────────────────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴───────────────────────────────────────────────┘

new-service

Example of usage:

buisciii new-service <resolution_id>

Help:

Usage: buisciii new-service [OPTIONS] <resolution id> Create new service, it will create folder and copy template depending on selected service. Options: -p, --path PATH Path to create the service folder -n, --no_create_folder No create service folder, only resolution -a, --ask_path Please ask for path. --help Show this message and exit.

scratch

Example of usage:

buisciii scratch --direction service_to_scratch <resolution_id>

Help:

Usage: buisciii scratch [OPTIONS] <resolution id> Copy service folder to scratch directory for execution. Options: -p, --path PATH Absolute path to the folder containing service to copy -a, --ask_path Please ask for service path. -t, --tmp_dir PATH Directory to which the files will be transfered for execution. Default: /data/ucct/bi/scratch_tmp/bi/ -d, --direction [service_to_scratch|scratch_to_service|remove_scratch] Direction of the rsync command. service_to_scratch from /data/ucct/bi/service to /data/ucct/bi/scratch_tmp/bi/.scratch_to_service: From /data/ucct/bi/scratch_tmp/bi/ to /data/ucct/bi/service --help Show this message and exit.

Finish

Example of usage:

buisciii finish <resolution_id>

Help:

Usage: buisciii finish [OPTIONS] <resolution id> Service cleaning, remove big files, rename folders before copy and copy resolution FOLDER to sftp. Options: -p, --path PATH Absolute path to the folder containg the service to reaname and copy -a, --ask_path Please ask for path, not assume pwd. -s, --sftp_folder PATH Absolute path to directory to which the files will be transfered -t, --tmp_dir PATH Absolute path to the scratch directory containing the service. --help Show this message and exit.

Finish module performs de following modules at onin this order, at once:

clean

Example of usage:

buisciii clean <resolution_id>

Help:

Usage: buisciii clean [OPTIONS] <resolution id> Service cleaning. It will either remove big files, rename folders before copy, revert this renaming, show removable files or show folders for no copy. Options: -p, --path PATH Absolute path to the folder containing service to clean -a, --ask_path Please ask for path -s, --option [full_clean|rename_nocopy|clean|revert_renaming|show_removable|show_nocopy] Select what to do inside the cleanning step: full_clean: delete files and folders to clean, rename no copy and deleted folders, rename_nocopy: just rename no copy folders, clean: delete files and folders to clean,revert_renaming: remove no_copy and delete tags,show_removable: list folders and files to remove and show_nocopy: show folders to rename with no_copy tag. --help Show this message and exit.
scratch back
buisciii scratch --direction scratch_to_service <resolution_id>
copy_sftp

Example of usage:

buisciii copy-sftp <resolution_id>

Help:

Usage: buisciii copy-sftp [OPTIONS] <resolution id> Copy resolution FOLDER to sftp, change status of resolution in iskylims and generate md, pdf, html. Options: -p, --path PATH Absolute path to directory containing files to transfer -a, --ask_path Please ask for path -s, --sftp_folder PATH Absolute path to directory to which the files will be transfered --help Show this message and exit.

bioinfo_doc

Example of usage:

buisciii bioinfo-doc <resolution_id>

Help:

Usage: buisciii bioinfo-doc [OPTIONS] <resolution id> Create the folder documentation structure in bioinfo_doc server Options: -p, --path PATH Absolute path to bioinfo_doc directory. -a, --ask_path Please ask for path, not assume /data/bioinfo_doc/. -t, --type [service_info|delivery] Select the documentation that will generate -s, --sftp_folder PATH Absolute path to sftp folfer containing service folder -r, --report_md PATH Absolute path to markdown report to use instead of the one in config file -m, --results_md PATH Absolute path to markdown report to use instead of the one in config file -e, --email_psswd TEXT Password for bioinformatica@isciii.es --help Show this message and exit.

archive

Example of usage:

buisciii archive --date_from 2022-01-01 --date_until 2023-01-01

Help:

Usage: buisciii archive [OPTIONS] Archive services or retrieve services from archive Options: -s, --service_id TEXT service id, pe SRVCNM787 -sf, --service_file TEXT file with services ids, one per line -t, --ser_type [services_and_colaborations|research] Select which folder you want to archive. -o, --option [archive|retrieve_from_archive] Select either you want to archive services or retrieve a service from archive. -sp, --skip_prompts Avoid prompts (except on service choosing) -df, --date_from TEXT The date from which start search (format 'YYYY-MM-DD') -du, --date_until TEXT The date from which end search (format 'YYYY-MM-DD') -f, --output_name TEXT Tsv output path + filename with archive stats and info --help Show this message and exit.

autoclean_sftp

Example of usage:

buisciii autoclean-sftp

Help:

Usage: buisciii autoclean-sftp [OPTIONS] Clean old sftp services Options: -s, --sftp_folder PATH Absolute path to sftp folder -d, --days INTEGER Integer, remove files older than a window of `-d  [int]` days. Default 14 days. --help Show this message and exit.

fix-permissions

Example of usage:

buisciii fix-permissions -d /data/bi

Help:

Usage: buisciii fix-permissions [OPTIONS] Fix permissions Options: -d, --input_directory PATH Input directory to fix permissions (absolute path) [required] --help Show this message and exit.

Acknowledgements

Python package idea and design is really inspired in nf-core/tools.

About

set of helper tools for bu-isciii management of its core services portfolio.

Topics

Resources

License

Contributing

Stars

Watchers

Forks

Packages

 
 
 

Contributors