JobSchedulers.jl

• Feat/Optimize: Rewriting scheduler for 200~400X speed up. Scheduling 100,000 small tasks in 0.2 seconds using 24 threads.
• Deprecate: SCHEDULER_UPDATE_SECOND and set_scheduler_update_second() are no longer required. Changing them will have no effect on the scheduler.
• Feat: Now, the scheduler updates when needed, and every 0.5 second. When specific events happen, scheduler_need_action() is used to trigger update of the scheduler. SCHEDULER_REACTIVATION_TASK[] is used to trigger scheduler_need_action() every 0.5 second because a regular check is needed for future jobs (defined by j::Job.schedule_time).
• Change: Job's fields stdout_file::String and stderr_file::String is changed to stdout::Union{IO,AbstractString,Nothing} and stderr::Union{IO,AbstractString,Nothing}.
• Change: remove function format_stdxxx_file(x).
• Optimize: check whether a job needs IO redirection before wrapping in task. Also, avoid unecessary stack when wrapping a new job, avoiding recurring job's stack overflow due to creating new jobs.
• Feat: Now people can set_group_seperator(group_seperator::Regex=r": *"). A group name will be given to Job. It is useful when showing progress meters.
• Feat: New wait(j::Job) and wait(js::Vector{Job}).
• Optimize: progress bar now does not blink: now we do not clear lines before printing. Instead, printing a "erase from cursor to end of line" characters.
• Optimize: rewrite progress computing for much faster speed.

JobSchedulers v0.9.0

Full Changelog: https://github.com/cihga39871/JobSchedulers.jl/compare/v0.8.4...v0.9.0

Full Changelog: https://github.com/cihga39871/JobSchedulers.jl/compare/v0.8.1...v0.8.2

Full Changelog: https://github.com/cihga39871/JobSchedulers.jl/compare/v0.8.0...v0.8.1

Full Changelog: https://github.com/cihga39871/JobSchedulers.jl/compare/v0.7.12...v0.8.0

Full Changelog: https://github.com/cihga39871/JobSchedulers.jl/compare/v0.7.10...v0.7.12

Full Changelog: https://github.com/cihga39871/JobSchedulers.jl/compare/v0.7.9...v0.7.10

Full Changelog: https://github.com/cihga39871/JobSchedulers.jl/compare/v0.7.6...v0.7.9

What's Changed

• progress meter by @cihga39871 in https://github.com/cihga39871/JobSchedulers.jl/pull/3

New Contributors

• @cihga39871 made their first contribution in https://github.com/cihga39871/JobSchedulers.jl/pull/3

Full Changelog: https://github.com/cihga39871/JobSchedulers.jl/compare/v0.7.3...v0.7.6

• Compat: Pipelines v0.9: significant improvement on decision of re-run: considering file change.
• Fix: pretty print of Job and Vector{Job}.

v0.7.2

• Fix: unexpected output of scheduler_status() when SCHEDULER_TASK is not defined.

v0.7.1

• Compat: PrettyTables = "0.12 - 2" to satisfy DataFrames v1.3.5 which needs PrettyTables v1 but not v2.

v0.7.0

• Remove dependency DataFrames and change to PrettyTables. The loading time of DataFrames is high.

• Feature: now a Job is sticky to one thread (>1). JobSchedulers allocates and manuages it. The SCHEDULER_TASK is sticky to thread 1.

• Feature: queue(...) is rewritten.

• Feature: Better pretty print of Job and queue().

• Feature: New function: wait_queue() waits for all jobs in queue() become finished.

• Feature: New function: set_scheduler()

• Fix: set_scheduler_max_cpu(percent::Float64): use default_ncpu() if error.

• Change: SCHEDULER_UPDATE_SECOND to 0.05 from 0.6

v0.6.12

• Feature: Enchance compatibility with Pipelines v0.8.5: Program has a new field called arg_forward that is used to forward user-defined inputs/outputs to specific keyword arguments of JobSchedulers.Job(::Program, ...), including name::String, user::String, ncpu::Int, mem::Int.

v0.6.11

• Fix: running queue() when updating queue: use lock within DataFrames.DataFrame(job_queue::Vector{Job}).

v0.6.10

• Update documents.

v0.6.9

• Support Pipelines.jl v0.8.