nextflow

• Fixed file method does not interpret correctly S3 path with globs #509
• Fixed Execution hangs when the amount of requested resources exceed the available ones #502
• Minor change in the execution report #507 #508
• Make AWS clustering compliant with billing by seconds
• Refactored nf-token AWS job definition parameter #475
• Run Batch task in a scratch directory to avoid undefined execution folder #475
• Renamed iamProfile to instanceRole #422
• Renamed ProcessNotRecoverableException to ProcessUnrecoverableException

From 0.26.0-RC2

• Fixed TMP and TMPDIR variables are not passed in Singularity environment #496
• Fixed Dynamic resources directive when defined both in a process and the config file cause an error #497
• Added Experimental support for Shifter containers #500 #501
• Improved AWS Batch task logging and task attempts
• Improved Execution report showing task realtime instead of task duration on time box plot

From 0.26.0-RC1

• Added support for AWS Batch #475
• Added support for AWS IAM role #422
• Added Experimental support for Java 9 #462
• Added transpose operator #440
• Added join operator #460
• Added html report (-with-report option)
• Added Gzip compression support to splitXxx operators #484
• Added Support PE Fastq files splitting #436
• Added Operator splitFasta should allow the creation of chunk having the same size #472
• Added HTML execution report #490 #491
• Added New scientist names #485
• Improved merge operator #458
• Improved value channel handling for reduce, filter, first, take and randomSample operators
• Prints a warning message when the config file defines setting not matching any process definition
• Fixed HEAD is detached error #389
• Fixed Include container name when calculating cache key #443
• Fixed Invalidate process cache when a referenced external script is modified #413
• Fixed Execution failure is not reported in the trace file when retry error strategy is used #404
• Fixed "when" directive should not allow staging of input files for a process, if the criteria are not met #424
• Fixed Failure during foreign files download cause execution to stop abruptly #295
• Fixed The parameter groupName cannot be used with the parameter subnet #465
• Fixed unable to download from gitlab.com public repository #457
• Fixed Print work directory when a task fails #445
• Fixed Misleading command line parameter type conversion #434
• Fixed Nextflow should not stage files that have the same name #470
• Fixed SplitCsv returns array objects instead of Lists #487
• Fixed Cannot cast object FtpURLConnection error #482
• Fixed Nested Environment Module names are not loaded properly #489
• Removed support support for Java 7 #385
• Removed deprecated Cirrus/ClusterK executor
• Removed deprecated support for DNAnexus #386

• Fixed Unable to parse the AWS price list API #444
• Added seed parameter to randomSample operator #410

• Fixed Empty channel waiting forever when viewed and mixed #428
• Fixed Error Channels cannot be used as input in more than one process or operator #429
• Fixed Script body is resolved even if the when guard evaluate to false
• Added queue process directive as option for trace file #431

• Fixed NF fails to pull images hosted in the Singularity Hub #414
• Added -y to cloud create command to skip confirmation request

• Removed invalid debug information

• Fixed NPE when parsing AWS price file #408
• Fixed Grid queue status return wrong data when a process change the submission queue dynamically #412
• Fixed MPI-like execution for SLURM batch scheduler - 13f3a091d7dc24c5712216c34b58d7ce8a947d0a
• Fixed NPE when log params key is not defined
• Fixed Optional parameter for set #399
• Added automatic pull for Singularity remote images #356
• Added London and Canada AWS regions
• Merged Pass NXF_OPTS to JVM during the java version check to allow memory definition on memory-restricted environments #411
• Improved error message in launcher file - 4f05666e8c26d5ec3255bf6d6a7448ac235b2689

• Fixed Execution hangs if a job submission fail #396
• Fixed stdout can not be sent into multiple channels #388
• Fixed Pipeline execution hangs when a process exceed the time limit and retry error strategy is used #387
• Added scientists and adjectives to run name generator
• Deprecated route operator

Fixed issue

• Duplicate mount point docker error #384.

Note

• Docker read-only mounts feature (#346) is still considered experimental, therefore mounts are writeable by default. To mount input paths as read-only set docker.writableInputMounts=false in the nextflow config file.

Improvements and fixed issue

• Fixed Dataflow threads pool get exhausted with a large number of tasks #92
• Fixed Custom function cannot be invoked in the workflow completion handler #374
• Fixed Runtime errors are not reported in the workflow.errorMessage property #369
• Fixed Warn if params.something is defined twice in a NF script #372
• Fixed Map.Entry values deserialisation process fail #373
• Fixed List active operators that may hang the script execution #315
• Fixed launcher error message #366
• Fixed AWS spot instance bootStorageSize property is ignored #368
• Fixed Implicit variable task is not accessible in cached processes #363
• Fixed Do not export user environment when submitting job to SGE-like batch scheduler #362
• Fixed Process termination can throw a ConcurrentModificationException hiding the error cause message #359
• Fixed GroovyRuntimeException during foreign file copy process #360
• Fixed Helper function to compare version numbers #319
• Fixed Docker input file mounts should be read-only for enforce container security #346
• Fixed Extend each repeater syntax to support file collections #355
• Fixed Modified task properties are not accessible in the command script during task retry #345
• Fixed PublishDir warning message
• Fixed Spread operator mishandles Iterable values provided as a dataflow variable #353
• Fixed UUID cli options are wrongly interpreted as Duration objects #349
• New toInteger converter operator
• New randomSample operator #344
• New simpleName extension method to Path/File objects #354
• New warning message when input tuple cardinality does not match declared length #348
• Improved use example into ext directive documentation #327
• Improved error handing when parsing trace file #371
• Improved error reporting for process missing input definition
• Improved launcher script to support https protocol by default #379
• Change maxErrors default to -1
• Deprecated identifier channel as variable name

Special note

• Directive maxErrors new default is -1, this means that by default this value is ignored when a task execution is retried and only maxRetries value the is taken in consideration.
• Docker input mounts are now enforced as read-only. This has been done to mitigate possible security concerns when using Docker containers. It must noted that this is coherent with the idempotent nature of nextflow processes by which a task cannot have side-effects (and therefore modify the input files).
• SGE, PBS/Torque and NGSII executors do not specify any more the -V flag when submitting a job request. If you need to export the user environment in your task you will need to add it by using the clusterOptions directive.

• Fixed PublishDir creates nested directory structures when a folder already exists #342
• Fixed NoSuchFileException is thrown when trace file path does not exist #334
• Fixed watchPath glob directory not working #335
• Fixed Channel.fromFilePairs does not handle properly multiple star wildcards #337
• Added executor for NQSII, queuing system used on NEC clusters.
• Added relative option to fromPath channel factory

• Fixed Process execution fails with not GNU version of sleep and date tools #321
• Fixed Pending processes are not cleaned up properly #322
• Fixed NPE is thrown when Singularity is enabled and a process uses no container #324
• Fixed Cached processes are not included in the timeline report #325
• Added MD5 digest file upload
• Updated docs

• Fixed Tuple array object deserialisation fails #317
• Fixed Execution time is incorrect for the local executor #313
• Use SLURM squeue --noheader instead -h command line option to support not standard implementation #316

• Fixed Improve explanation of cache DB IOException #303
• Fixed Operator ifEmpty results in a error when applied to a dataflow variable #308
• Fixed Process execution fails when the work path contains a parenthesis character #310
• Fixed Installer script fails when current path contains a blank character #311
• Fixed Include SLURM partition when checking job status #312
• Allow the usage of close operator on unbounded dataflow variable

• Added new combine operator #298
• Added new collect operator #297
• Added new CLI option -params-file to load parameters from a json/yaml file #208
• Fixed SCM user name is not set accessing private repository #300
• Fixed Infer singleton process outputs from inputs declaration #301
• Fixed Use exist status 1 for all NF errors #304
• Fixed Invalid SGE job name as defined in sge_types #299
• Added skip parameter to collectFile operator #306
• Added experimental fs command
• Improved file staging error reports #295
• Upgraded Groovy to version 2.4.10

• Fixed Show a warning message when accessing undefined parameters #293
• Fixed Staging input files in a subfolder changes their names when they are symlinks #292
• Fixed NPE accessing foreign files
• Added column char to list of path escaped chars
• Fixed Boothook script fails on Ubuntu 16.04

• Fixed Job termination is not recognised if the starter file is not created #268
• Fixed Allow NF console to resume process executions (second take) #280
• Fixed PublishDir may throw a FileAlredyExistsException #283
• Increased logger file appender buffer size to 64kb
• Updated Logback to version 1.1.10
• Updated copyright info
• Fixed NXF_DEBUG level

• Improved history lock error message
• Fixed typo in warning message
• Added cleanup to list of accepted directives
• Added NXF_OWNER to kubernetes executor
• Fixed Function nxf_kill is not invoked properly on a containerised process #275
• Fixed Pending grid jobs are not cancelled upon pipeline interruption
• Documented that the toSortedList operator can accept a comparator closure #277

• Fixed FixOwnership flag break non BASH scripts/templates #270
• Fixed Fixed PBS Executor should not have parenthesis #271
• Fixed Function nxf_kill is not invoked properly on process termination #275
• Fixed Timeline.html is not working on IE11 #272
• Fixed Merge process specific extended properties with the ones defined in the default scope #273
• Fixed Using storeDir and publishDir in the same process yields unexpected results #266

• Added support for Singularity containers #252
• Added support for HTTP/S and FTP files (read-only)
• Fixed stream close leak while copy foreign file to local file system
• Fixed Task scheduling may hang while checking for job completion #261
• Fixed Specifying the keyFile cloud config attribute result in an error #263
• Added support for multiple target channels in tap operator #255
• Added uri property to Path and File objects
• Made NXF_SCRATCH variable accessible to beforeScript handler

• Fixed Script execution terminates before all operators complete #259
• Fixed A script variable with the same name of a global variable invalidate process resume #254
• Fixed when is not recognised when used before an input and script is not declared
• Fixed count operator hung when applied to a dataflow variable
• Added warning message when an input/output set defines only one component #216 #251
• Reduced log verbosity of Ignite Scheduler classes