🔥 Pyrodigal

Cython bindings and Python interface to Prodigal, an ORF finder for genomes and metagenomes. Now with SIMD!

🗺️ Overview

Pyrodigal is a Python module that provides bindings to Prodigal using Cython. It directly interacts with the Prodigal internals, which has the following advantages:

• single dependency: Pyrodigal is distributed as a Python package, so you
  can add it as a dependency to your project, and stop worrying about the
  Prodigal binary being present on the end-user machine.
• no intermediate files: Everything happens in memory, in a Python object
  you fully control, so you don't have to invoke the Prodigal CLI using a
  sub-process and temporary files. Sequences can be passed directly as
  strings or bytes, which avoids the overhead of formatting your input to
  FASTA for Prodigal.
• better memory usage: Pyrodigal uses more compact data structures compared
  to the original Prodigal implementation, allowing to save memory to store
  the same information. A heuristic is used to estimate the number of nodes
  to allocate based on the sequence GC% in order to minimize reallocations.
• better performance: Pyrodigal uses SIMD instructions to compute which
  dynamic programming nodes can be ignored when scoring connections. This can
  save from a third to half the runtime depending on the sequence. The Benchmarks page of the documentation contains comprehensive comparisons. See the JOSS paper
  for details about how this is achieved.
• same results: Pyrodigal is tested to make sure it produces
  exactly the same results as Prodigal v2.6.3+31b300a. This was verified
  extensively by Julian Hahnfeld and can be
  checked with his comparison repository.

📋 Features

The library now features everything from the original Prodigal CLI:

• run mode selection: Choose between single mode, using a training
  sequence to count nucleotide hexamers, or metagenomic mode, using
  pre-trained data from different organisms (prodigal -p).
• region masking: Prevent genes from being predicted across regions
  containing unknown nucleotides (prodigal -m).
• closed ends: Genes will be identified as running over edges if they
  are larger than a certain size, but this can be disabled (prodigal -c).
• training configuration: During the training process, a custom
  translation table can be given (prodigal -g), and the Shine-Dalgarno motif
  search can be forcefully bypassed (prodigal -n)
• output files: Output files can be written in a format mostly
  compatible with the Prodigal binary, including the protein translations
  in FASTA format (prodigal -a), the gene sequences in FASTA format
  (prodigal -d), or the potential gene scores in tabular format
  (prodigal -s).
• training data persistence: Getting training data from a sequence and
  using it for other sequences is supported; in addition, a training data
  file can be saved and loaded transparently (prodigal -t).

In addition, the new features are available:

• custom gene size threshold: While Prodigal uses a minimum gene size
  of 90 nucleotides (60 if on edge), Pyrodigal allows to customize this
  threshold, allowing for smaller ORFs to be identified if needed.
• custom metagenomic models: Since v3.0.0, you can use your own
  metagenomic models to run Pyrodigal in meta-mode. Check for instance
  pyrodigal-gv, which
  provides additional models for giant viruses and gut phages.

🐏 Memory

Pyrodigal makes several changes compared to the original Prodigal binary regarding memory management:

• Sequences are stored as raw bytes instead of compressed bitmaps. This means
  that the sequence itself takes 3/8th more space, but since the memory used
  for storing the sequence is often negligible compared to the memory used to
  store dynamic programming nodes, this is an acceptable trade-off for better
  performance when extracting said nodes.
• Node fields use smaller data types to fit into 128 bytes, compared to the
  176 bytes of the original Prodigal data structure.
• Node arrays are pre-allocated based on the sequence GC% to extrapolate the
  probability to find a start or stop codon.
• Genes are stored in a more compact data structure than in Prodigal (which
  reserves a buffer to store string data), saving around 1KiB per gene.

🧶 Thread-safety

pyrodigal.GeneFinder instances are thread-safe. In addition, the find_genes method is re-entrant. This means you can train an GeneFinder instance once, and then use a pool to process sequences in parallel:

import multiprocessing.pool
import pyrodigal

gene_finder = pyrodigal.GeneFinder()
gene_finder.train(training_sequence)

with multiprocessing.pool.ThreadPool() as pool:
    predictions = pool.map(orf_finder.find_genes, sequences)

🔧 Installing

Pyrodigal can be installed directly from PyPI, which hosts some pre-built wheels for the x86-64 architecture (Linux/MacOS/Windows) and the Aarch64 architecture (Linux/MacOS), as well as the code required to compile from source with Cython:

$ pip install pyrodigal

Otherwise, Pyrodigal is also available as a Bioconda package:

$ conda install -c bioconda pyrodigal

Check the install page of the documentation for other ways to install Pyrodigal on your machine.

💡 Example

Let's load a sequence from a GenBank file, use an GeneFinder to find all the genes it contains, and print the proteins in two-line FASTA format.

🔬 Biopython

To use the GeneFinder in single mode (corresponding to prodigal -p single, the default operation mode of Prodigal), you must explicitly call the train method with the sequence you want to use for training before trying to find genes, or you will get a RuntimeError:

import Bio.SeqIO
import pyrodigal

record = Bio.SeqIO.read("sequence.gbk", "genbank")

orf_finder = pyrodigal.GeneFinder()
orf_finder.train(bytes(record.seq))
genes = orf_finder.find_genes(bytes(record.seq))

However, in meta mode (corresponding to prodigal -p meta), you can find genes directly:

import Bio.SeqIO
import pyrodigal

record = Bio.SeqIO.read("sequence.gbk", "genbank")

orf_finder = pyrodigal.GeneFinder(meta=True)
for i, pred in enumerate(orf_finder.find_genes(bytes(record.seq))):
    print(f">{record.id}_{i+1}")
    print(pred.translate())

On older versions of Biopython (before 1.79) you will need to use record.seq.encode() instead of bytes(record.seq).

🧪 Scikit-bio

import skbio.io
import pyrodigal

seq = next(skbio.io.read("sequence.gbk", "genbank"))

orf_finder = pyrodigal.GeneFinder(meta=True)
for i, pred in enumerate(orf_finder.find_genes(seq.values.view('B'))):
    print(f">{record.id}_{i+1}")
    print(pred.translate())

We need to use the view method to get the sequence viewable by Cython as an array of unsigned char.

🔖 Citation

Pyrodigal is scientific software, with a published paper in the Journal of Open-Source Software. Please cite both Pyrodigal and Prodigal if you are using it in an academic work, for instance as:

Pyrodigal (Larralde, 2022), a Python library binding to Prodigal (Hyatt et al., 2010).

Detailed references are available on the Publications page of the online documentation.

💭 Feedback

⚠️ Issue Tracker

Found a bug ? Have an enhancement request ? Head over to the GitHub issue tracker if you need to report or ask something. If you are filing in on a bug, please include as much information as you can about the issue, and try to recreate the same bug in a simple, easily reproducible situation.

🏗️ Contributing

Contributions are more than welcome! See CONTRIBUTING.md for more details.

📋 Changelog

This project adheres to Semantic Versioning and provides a changelog in the Keep a Changelog format.

⚖️ License

This library is provided under the GNU General Public License v3.0. The Prodigal code was written by Doug Hyatt and is distributed under the terms of the GPLv3 as well. See vendor/Prodigal/LICENSE for more information.

This project is in no way not affiliated, sponsored, or otherwise endorsed by the original Prodigal authors. It was developed by Martin Larralde during his PhD project at the European Molecular Biology Laboratory in the Zeller team.