Bot releases are hidden (Show)
Published by JinyLeeChina almost 2 years ago
Bugfix:
Published by caishunfeng about 2 years ago
not use web service
comment in alert server (#11819)common.properties
(#11910)WorkFlow Relation
page (#10836)runTask
to avoid the task group acquire failed, but the task status is in running (#11451)Published by zhuangchong about 2 years ago
runTask
to avoid the task group acquire failed, but the task status is in running (#11451) (#12011)Published by zhongjiajie about 2 years ago
WorkFlow Relation
page (#10836)Published by JinyLeeChina over 2 years ago
Published by devosend over 2 years ago
Published by zhongjiajie over 2 years ago
Published by zhuangchong over 2 years ago
[Feature]
[#6422] Add Task group feature to limit the concurrency of taskInstance
[#7332] [UI Next] A new version of the UI proposal
[#8275] [UI Next] Data Quality
[#8310] Support custom timezone
[#8373] Dependent tasks can re-run automatically in the case of complement
[#8499] Optimize workflowlieage query speed
[#8527] Add dataSource for Amazon Redshift
[#8593] Refactor cli config dict mechnism
[#9039] E2E test adapt M1 chip
[#9396] Support output parameters transfer from parent workflow to child work flow.
[#8516] [python] Add mechanism cli with simple command
[#8585] [python] Add config mechanism and cli subcommand config
[Improvements]
[#8584] Task back-end plugin optimization, new plugins only need to modify the plugin's own module
[#8874] Verify the endtime and starttime when submit/create a cron under workflow
[#9016] Dependent The global project can be selected when adding dependencies
[#9221] AlertSender optimization and gracefully close, such as MasterServer
[#9228] implement use the slot to scan the database
[#9230] Slim dist package by migrate python gatewar into apiserver
[#9372] [python] Migrate pythonGatewayServer into api server
[#9443] [python] Add missing doc about config and connect remote server
[#8719] [Master/Worker] Change the task ack to runnning callback
[#9293] [Master] add task event thread pool
[BUG]
[#7236] Failed to create tenant using S3a Minio
[#7416] Text file busy
[#7896] When the project is authorized, it will generate a duplicate authorized project
[#8089] start server failed because can't connect to postgresql
[#8183] message:datasource plugin 'spark' is not found.
[#8202] MapReduce generated command built-in parameter location is wrong
[#8751] Change param user, queue do no work in ProcessDefinition
[#8756] Process using the dependence component cannot migrate between test and prod environment
[#8760] Resource file deletion conditions
[#8791] Rectify the issue with affecting the original node's data when editing the form of the copied node.
[#8951] Worker resources are exhausted and cause downtime
[#9243] Some types of alarms can't display project name
Published by JinyLeeChina over 2 years ago
[Improvements]
[#8110] The fault-tolerant process of the worker is optimized, so that when the server is under too much pressure, the worker service is interrupted, and the task can be transferred to other workers to continue.
[#8445] Optimize the page display flag of forbidden tasks to distinguish the display of normally executed tasks
[#8441] Add a prompt on the task box to display all the long task names.
[#8187] The function of re uploading files is added in the resource center. When users need to modify the execution script, they do not need to reconfigure the task parameters to realize the function of automatically updating the execution script.
[#8169] After the modification workflow is optimized, the page remains in the DAG page and will not jump to the list page.
[#8285] In the dingtalk alarm plugin, add the markdown information type in the alarm content.
[BUG]
[#8213] Fixed the task runs incorrectly when the worker group contains uppercase letters.
[#8347] Fixed the workflow cannot be stopped when the task fails and retries
[#8135] Fixed JDBC connection parameter not input '@'
[#8367] Fixed complement may not end normally
[#8170] Fix the sub workflow that cannot be entered from the page
Published by zhuangchong over 2 years ago
Download:
https://dolphinscheduler.apache.org/en-us/download/download.html
Feature
[#7804] DingTalk alert plugin adds signature.
[#7857] Problems caused by modifying the administrator user information.
[#7675] Supports whether SQL is placed in one session
[#7571] Add dependent tips about success or failure
Improvement
[#7990] zkRoot in conf/config/install_ config does not take effect
[#7713] The data source password in the log is not encrypted
Bug:
[#8131] add delete workflow instance when delete process definition
[#8098] udf sub resource manage edit modal cant close
[#8081] process is always running: netty writeAndFlush without retry when failed leads to worker response to master failed
[#7986] After deleting the running workflow, the master keeps brushing the error log
[#8056] Edit the bug of worker grouping in environment management.
[#7512] dependent node ui dislocation
[#7913] Error in querying historical version information of workflow
[#7982] common task log in master no need to print separately
[#7962] The global parameters of the sub_process node are not passed to the associated workflow task
[#7953] query log can not show contents when task log in master on k8s
[#7815] There are duplicate processes in the process definition list
[#7698] process instance is always running: task is failure when process instance FailureStrategy.END
[#7732] field ‘is_directory’ in t_ds_resources table has error type in PostgreSQL database
[#7883] Repair JDBC connection of Oracle
[#7538] when there is a forbidden node in dag, the execution flow is abnormal
[#7789] querySimpleList return wrong projectCode
Published by lenboo almost 3 years ago
Download:
https://dolphinscheduler.apache.org/en-us/download/download.html
Feature
[#6470] Add Python API for DolphinScheduler
[#7460] Wechat alert support send to group chat
Improvement
[#7529] Init DB schema from the full sql file
Bug:
[#7661] Fix the memory leak of logger in worker
[#7750 ]Compatible historical version data source connection information
[#7748] Remove init script when gaining a new DB connection
[#7705] Because of memory limitations, upgrade from 1.3.5 to 2.0.2 error
[#7786] Server restart fail after force killed
[#7660] Wrong create time of process definition version
[#7607] Failed to execute PROCEDURE node
[#7639] Add default config of quartz and zookeeper in common config map
[#7654] Dependent node on change project code error
[#7658] workflow copy error
[#7609] Workflow is always running when worker sendResult success but master not received bug
[#7554] H2 in standalone server will auto restart after several minutes
[#7434] An error is reported when executing the MySQL table creation statement
[#7537] Dependent node retry delay did not work
[#7451] Remove '+1' (day) in the date of the complement data
[#7392] Add hive datasource failed
Published by lenboo almost 3 years ago
Download:
https://dolphinscheduler.apache.org/en-us/download/download.html
Bug fix
[#7039] fix that the hive data source connection failed when using Kerberos
[#7338] repairing standalone service startup failed
[#7215] fix the problem that the alarm group display cannot be displayed
[#7152] fix the problem of abnormal file upload
[#7083] fix the problem that the switch task failed to run
[#7070] fix the failure of workflow timeout policy
[#6809] fix the problem that the SQL task cannot send mail
[#7004] fix master still work when it lose zookeeper connection
Improvements
[#6772] optimized the operation of some APIs
[#7166] speed up the loading of workflow instance pages
[#6934] optimize the display information of the workflow relationship page
Published by lenboo almost 3 years ago
Features:
[#6597] supports dolphinscheduler upgrade from 1.x to 2.0
[#6812] speed up the site loading when project exists multi process definitions
[#6763] remove the detection of dumplicate task names
[#6655] remove the registry monitor
Bug Fix:
[#6613] fix bug in task retry mechanism
[#6707] fix error when set up the global parameters in process defintion
[#6882] fix process cannt finish and its status always be running
[#6771] fix failover worker interrupt
[#6658] fix task code in bulk definition
[#6737] fix event handle twice in two thread pool
[#6682] replace task name with task code
[#6653] fix netty client create too many channels
[#6640] fix PROCESS_HOST_UPDATE_REQUST command donot be supported.
[#6591] fix improt process definition error
Published by lenboo almost 3 years ago
New Features:
[#3961]Create registry center plug-in to support different type of registry components
[#4325]Propose for process definition json split
[#5077]Add parameter transfer function between tasks
[#4355]Propose for master refactor and scheduler module
[#6417]add 'read log' in the task instance right-click context menu
[#6499]report busy state when worker is overload
[#6471]Cache process definition in master
[#6483]upgrade the druid and mysql connector version
[#4878]Add task plug-in function to support more custom task types
[#5273]Add switch task node
[#5098]Remove deduplicate logs in controllers
[#5083]Refactored data source creation
[#5172]add timing online/offline marking color
[#3805]The implement of passing variables between tasks
[#3049]Add alarm plug-in function to support more custom alarm components
[#4463]add an option to the UI page for saving the workflow definition: whether to go online? online by default First-time contributor good first issue new feature
[#3222]Store password in ciphertext instead
[#3191]add query top n process instance order by running duration
BugFix
[#6550]The list of environments in the DAG task pop-up window is not updated
[#6506]Fix install.sh for DS 2.0 and add comment to install_config.conf
[#6497]Shell task can not use user defined environment correctly
[#6478]Missing history data in complement data mode
[#6352]override the old process definition when I use the copy workflow feature
[#6342]Task instance page date backfill bug
[#5701]When deleting a user, the accessToken associated with the user should also be deleted
[#4809]cannot get application status when kerberos authentication is enabled
[#4450]Hive/Spark data sources do not support multi-tenancy when Kerberos authentication is enabled bug
Published by CalvinKirs almost 3 years ago
Feature/Update
[Feature#6022]Add standalone server module to make it easier to develop and run
BUGFIX
[Bug#6429] [UI] sub_process node open sub_task show empty page bug
[Fix#6337][Task] sql limit param no default value
Published by CalvinKirs about 3 years ago
Feature/Update
[#5405] [Improvement]Docker & K8s Improvement Plan Round 2
[#5858][Improvement][Docker] Docker image should support multi-arch like arm64 in docker-compose
[#5706][Improvement][common] Upgrade the version of fastjson from 1.2.61 to 1.2.75
[#5577][Improvement][UI] Add Project Name in Project Page
[#5567][Improvement][UI] Add project id in web ui url for sharing
[#5475][Improvement][Api] Upload resource to remote failed, the local tmp file need to be cleared
[#5468][Improvement][Net]Optimize IP acquisition in complex network environment
[#5467][Improvement][UI] UI cannot be displayed normally in some browsers
BUGFIX
[#6007][Bug][Worker] fix Wrong complement date
[#5719][Bug][K8s] Ingress ERROR io.k8s.api.networking.v1beta1.IngressSpec.tls: got "map", expected "array" On TLS enabled
[#5701][Bug][UI][DAO]When deleting a user, the accessToken associated with the user should also be deleted
[#5699][Bug][UI] Update user error in user information
[#5596][Bug][Python] Conflict between python_home and datax_home configuration in dolphinscheduler_env.sh
[#5559][Bug][Master Server] Master Server was shutdown but the process still in system
[#5581][Bug][Mysql] Specific key was too long, max key length is 767 bytes for varchar(256) in some mysql with innodb_large_prefix=OFF
[#5578][Bug][Master] ServerNodeManager WorkerGroupListener capture data change and get data failed
[#5570][Bug][Worker] worker.groups in worker.properties is still commented after installation in 1.3.6
[#5550][Bug][Master] remove check with executePath when kill yarn job
[#5549][Bug][Worker] SqlTask NPE
[#5431][Bug][K8s] Master and worker cannot get the right address with custom DNS in 1.3.6
Published by CalvinKirs over 3 years ago
Feature/Update
[#5062][Feature][Worker Group] Create/Edit/Delete Worker Group in Web UI
[#5175][Improvement][Server] Optimize and reduce worker group queries of database and zookeeper
[#4852][Feature] Support SkyWalking agent plugin
[#4837][Improvement][Docker/K8s] Docker & K8s Improvement Plan #4843 #4732 #4871 #4881 #4804 #4886 #2687 #4897 #3124 #5155 #5068 #5100 #2584 #4905 #5351
[#5023][Improvement][Docker/K8s] Improve docker and k8s #2584 #2687 #4732 #4871 #4886 #4897
[#5158][Improvement][Docker/K8s] Support more configs, more service access, skywalking configs, improve image for python, update faq and add support matrix #5155 #5068 #5141 #402 #403
[#5068][Improvement][Docker/K8s] Task support matrix and solution on docker/k8s
[#5195][Improvement][Docker/K8s] Improve feedback #5309 #5310
[#5100][Feature][K8s] Support task scalability on K8s such as Spark, Flink and DataX
[#5224][Improvement][Docker] Add workdir for dockerfile and remove redundant container_name and dolphinscheduler-postgresql-initdb
[#5310][Improvement][Server] The log "load is too high or availablePhysicalMemorySize(G) is too low" is not clear
[#5028][Feature][MR] Support MapReduce name
[#4960][Feature][Spark] Support spark name
[#4285][Feature] Add Flink job name
[#4976][Feature][Flink] Support name and parallelism input
[#4805][Improvement][SQL] Place the upgraded schema to the correct version directory and check schema errors
[#4751][Improvement][API] Improvement the duration field modify the display to a time-based minute-second format (e.g. 1d 10h 20m 1s)
[#4740][Improvement][UI] Set the default Tenant as default or the first existing tenant in the save process
[#403][Feature]The memory of worker, master, and other services does not need to be the same
[#174][Improvement][SqlTask] Add a switch to send mail and print head logs in SqlTask
[#5187][Build] Optimize long release name and remove unused nginx release
[#5087][Feature][SqlTask] Add a switch to send mail and print head logs in SqlTask
[#4624][Improvement][Server] When the server exist in the dead server list of zk,need stop service byself
[#4969][Improvement][UI] Support more file types in file detail page
[#2619][Improvement][API] /dolphinscheduler/projects/create doesn't return project ID when create successful
[#4270][Feature]support distributed tracing
BUGFIX
[#5309][Bug][Common] memoryUsage is -33%
[#4843][Bug][Docker] 1.3.5 release compose file config error WORKER_GROUP
[#4651][Bug][Docker] Random PSQLException in docker bug
[#5351][Bug][K8s] Quartz cron task cannot take effect
[#4905][Bug][K8s] Incorrect host problem in minikube
[#5176][Bug][Server] Wrong excludeFilters cause the worker server to start the master's beans and threads like LowerWeightHostManager and RefreshResourceTask
[#5132][Bug][server] when an exception occurs in the taskExecuteThread, task cannot stop immediately
[#5115][Bug][Server] The registered address of a server is the loopback address 127.0.0.1
[#5150][Bug][Server] DB transaction failure
[#5103][Bug][Resource] The file name of File and UDF resource not changed and cannot re-upload after renaming name
[#5349][Bug][master] Manually kill the task that fails to be retried. After the parallel task succeeds, the workflow status is always running
[#5328][Bug][Bug][MasterServer] process contain depend on task is always running
[#5199][Bug][api] When sleep is included in a shell task, cannot kill the task
[#5063][SQL] Fix MySQL datasource jdbc connect parameters
[#5048][Bug][Api] Delete the stopped workflow without deleting the corresponding subprocess, the workflow instance name is not displayed in the subprocess
[#5044][Bug][Master] Fix First master fault tolerance when startup
[#5037][Bug][Server] Both the master and the worker is hanging after restarting and stopping
[#4866][Bug][Common] Not loaded hdfs-site.xml causes the client to fail to access the datanode
[#4862][Bug][Server] Kill yarn application command won't be executed when killing error bug
[#4840][Bug][Master] Master cannot fault-tolerant when multiple Masters start at the same time
[#4816][Bug][API] The API calls the workflow instance query interface and queries all the data when the time parameter is wrong
[#4738][Bug][Mail] Send mail failed instead of execute sql error
[#4760][Bug][Api] init dolphinscheduler-postgre.sql error
[#4705][Bug][API] KeeperException$NoNodeException
[#4674][Bug][UI] The tips error when login with username and without password
[#4650][Bug][Api] Process definition create, update and copy error in PostgreSQL bug
[#4716][Bug][Master,Worker] The task execution path should be calculated by the worker, not the master
[#4617][Bug][LoggerServer] task log can not refresh in time
Published by CalvinKirs over 3 years ago
Feature/Update
[K8s] Support resource storage when fs.defaultFS=file:///
[Docker] Update readme
[UI] Improve parameter name
[UI] download url resolve and security page disappear delay problem under GENERAL_USER
[API]enable response resources gzip compression
[#703]Docker image optimization
[#4439][API] Cors inteceptor problems
[#4418][login controller] Need password when open in new tab
BUG FIX
[#3237][Docker] the file requirements.yaml in helm mode has error flag
[#3298][K8s]when recreate or upgrade helm release, the host ip changed
[#4691][UDF] udf jar path not exist when fs.defaultFS starts with file:///
[#4640][UI] Fixed a 404 error when the project name contains "HTTP"
[#4648][UI] TypeError: Cannot read property 'id' of undefined in createUser
[#4566][Worker] worker-server logback partial logs are not printed
[#4526][Worker] worker server doesn't work fine in K8s
[#4472][DataSource] Hive JDBC partial permission parameter concatenation JDBC URL error
[#4247][API] When schedulertime is empty, rerun the workflow, the time of the global parameter is not the latest
[#4449][API] Update workflow instance to report null point exception
[#4455]kill task error because of shell output truncation
[#4271][Logger] IOException or NoSuchFileException in logger server
[#4162][UI]fix re-login problem in new tab and state synchronization problem in multiple tabs
[#3457][Flink] fix flink args build problem
Published by davidzollo almost 4 years ago
Feature/Update
[#2925][server] init TaskLogger in TaskExecuteProcessor
[ambari-update] change version from 1.3.3 to 1.3.4
[sqoop-update] modify sqoop task param when upgrade ds to 1.3.4
[#4143][quartz-update] upgrade quartz version to 2.3.0
BUG FIX
[#2917 #4034 #4104 #4135][sqoop] sqoop optimization and bug fix
[#3177] task time parameter parsing error
[#3457][flink] fix flink args build problem
[#4034][server] fix sqoop import fail
[#3900][server] kill multi yarn app in one job
[#4054][api] fix The last week of the month for adding/editing timing, preview and save timing will report an error
[#4054][ui] Repair the last Sunday of each month
[#4084][server] fix taskInstance state change error
[#4104] fix sqoop task jdbc string contains special char
[#4135][worker] fix sqoop import hive error
[#4114][server] fix clear task execute path is related to master.
[#4084][master] fix taskInstance state change error
[#4090][dao] execute a single task in workflow throw NullPointerException
[#4190][dao] when the amount of json data is large, the process list page display slowly.
[#4218][bug] start from the setting nodes with NODE_PRE would be NPE
[#4223][ui] fix dag node name verification
Published by lgcareer almost 4 years ago
BUG FIX
[#3615] [master] After the task is executed successfully, but the next task has not been submitted, stop the master,the workflow will fail
[#4010] [master] When the condition node fails, the process status is failed after fault tolerance
[#3929] [master/worker] After the master and worker are fault-tolerant, the process instance fails after the task instance is successfully executed
[#3994] [Sql] Failed to import SQL file of sql/dolphinscheduler_mysql.sql
[#3964] [api/ui] The timeout warning of sub_process does not take effect
[#3974] [api/worker] The child process does not inherit the worker group of the parent process
[#3966] [master] The timeout warning does not take effect in sub_process
[#3958] [api] files should not be created successfully in the directory of the authorized file
[#3618] [worker] Too many files will be opened because of task executed fininished but not release the file handle.
[#3615] [master] After the task is executed successfully, but the next task has not been submitted, stop the master,the workflow will fail.
[#3843] [api] When the update workflow definition name already exists, the prompt is not friendly.
[#3924] [ui] When saving the workflow definition, if the backend verification is not successful, the save popup will not close.
[#3789] [remote] support netty heart beat to resolve channel time out.
[#3836] [API] verifyProcessDefinitionName error message
[#3702] [api] Resource file reupload does not overwrite the original file. This button is an invalid one
[#3621] [master] After batch deleting the executing process instances, the master cannot get the worker feedback results
[#3617] [master] After subtask fault tolerance, 2 task instances are generated,The process instance status always is executing
[#3616] [master] When the master receives the result from the worker, the master just stops. After the master restarts, the workflow status is always executing
[#3573] [dao] potential horizontal unauthorized access
[#3887] [ui] The English version of the resource file list and UDF list data is not displayed
[#3702] [api] Resource file reupload does not overwrite the original file. This button is an invalid one
[#3487] [API] Creating folders with multiple threads will result in multiple identical folders
[#3493] [api] worker group manage ui could not display createtime and updatetime
[#3549] [Server][sqlTask]The alias column in the query SQL does not take effect
[#3792] [ui] The pie chart on the project homepage does not adapt to the screen size
[#3719] [api] The task resource could not be deleted
[#3548] [UI] Monitor gauge chart display blank with same pid bug
[#3707] [ui] The batch delete function in the workflow definition and workflow instance pages cannot be canceled if selected
[#3553] [ui] when edit the work flow,the selected connecting line has no style change,but the others have get bigger
[#3238] [docker] Can not create folder in docker with standalone mode
[#3713] [common] catfile method Stream not closed
[#3258] [api] Connot get create time and update time,report DateTimeParseException
Improvement
[#3720] [ui] js mailbox verification
[#3843] [api] When the update workflow definition name already exists, the prompt is not friendly
[#3131] [api] When the new tenant already exists, the prompt is incorrect