oceanbase

版本信息

项目	描述
发布日期	2024-07-31
版本号	V4.2.1_CE_BP8
Commit 号	3149c25
OBServer RPM 版本号	oceanbase-ce-4.2.1.8-108000022024072217

特性增强

• 日志流副本管理

  V4.0 之前的版本，OceanBase 数据库提供了分区副本管理的一系列运维命令，比如为分区添加一个副本、为分区删除一个副本、对分区的一个副本进行类型转换等。V4.x 系列在设计上使用日志流代替了分区的概念，对标低版本的分区运维命令。新版本重新设计实现了日志流副本级别任务的运维方式，提供了一系列语法，用于支持日志流副本的添加、删除、副本类型转换、迁移、修改日志流 Paxos 成员数量和取消容灾任务等能力，满足客户手动运维日志流副本的需求。

• DBLink 支持指定域名访问

  OceanBase 数据库支持通过 DBLink 访问远端数据库。老版本需要在创建 DBLink 时指定远端数据库的 IP 地址，新版本增加支持配置域名地址。

• 主备租户角色切换验证

  OceanBase 数据库支持租户级主备角色切换，如在数据无损情况下可以将备租户 SWITCHOVER 成主租户，将主租户 SWITCHOVER 成备租户，或在数据损失情况下将备租户 FAILOVER 成主租户。主备租户角色切换操作有失败的可能，为了降低实际执行角色切换动作的风险，新版本新增支持主备租户角色切换验证功能（SWITCHOVER/FAILOVER VERIFY），在切换命令后添加 VERIFY 关键字，可以提前验证对应操作是否可以成功执行，如对应操作不可执行，将给出报错信息。

• 主备租户事件展示

  OceanBase 数据库 V4.2.1 BP8 之前，主备租户 SWITCHOVER、FAILOVER 等事件记录在 RS 事件中，RS 事件会随时间清理，且租户级记录混在集群操作记录中不易查找。新版本将主备租户操作记录拆分到每个租户下，通过 CDB/DBA_OB_TENANT_EVENT_HISTORY 视图透出。

• 事务提交后 Cursor 访问

  V4.0 开始，事务提交后不允许 Cursor 继续 Fetch 数据。V4.2.1 BP8 开始允许 Cursor 在没读取当前事务修改的表的情况下，在事务结束后依然可以 Fetch 数据，但需要通过 _enable_enhanced_cursor_validation 配置项开启。

• 增加 SM3 加密函数

  MySQL 模式下新增 SM3 加密函数。

• OBKV 响应时间统计直方图

  在 V4.2.1 BP7 响应时间直方图功能基础上，新版本在 [G]V$OB_QUERY_RESPONSE_TIME_HISTOGRAM 系统视图中增加了 OBKV 相关的统计类别，包括 TABLEAPI SELECT、TABLEAPI INSERT、TABLEAPI DELETE、TABLEAPI UPDATE、TABLEAPI REPLACE、TABLEAPI QUERY AND MUTATE、TABLEAPI OTHER、HBASE SCAN、HBASE PUT、HBASE DELETE、HBASE APPEND、HBASE INCREMENT、HBASE CHECK AND PUT、HBASE CHECK AND DELETE、HBASE HYBRID BATCH 等。

• OBKV-HBase 列分页

  原生 HBase 支持列分页，这在大宽表场景下提供了更好的查询性能。新版本 OBKV-HBase 也提供了类似的列分页能力，支持通过 ColumnPaginationFilter 过滤器限制返回列数。

• OBKV-HBase 逆序扫描

  OBKV-HBase 的数据是按照 RowKey（行键）排序存储的，在需要有序访问数据的场景，一般会以 RowKey 正序（从小到大）扫描。但实际业务中还存在需要倒序访问数据的场景，因此新版本新增 Reverse Scan 功能，允许用户以 RowKey 逆序（从大到小）扫描表中的数据，提升查询性能。

视图变更

视图	变更类型	描述
information_schema.columns	新增列	新增 srs_id 列用于表示空间坐标系信息。
[G]V$OB_SQL_AUDIT	列内容变更	原视图 params_value 字段只会记录 SQL 参数信息。新版本增加记录 PL 入参信息，暂时仅支持普通数据类型入参记录，不支持复杂数据类型入参记录。
CDB/DBA_OB_LS_REPLICA_TASKS	新增列	新增 data_source_svr_ip、data_source_svr_port、is_manual 列，用于记录容灾任务执行时引用的数据源和容灾任务生成来源。
CDB/DBA_OB_LS_REPLICA_TASK_HISTORY	新增	新增系统视图，用于展示容灾任务执行历史。CDB 视图仅在 SYS 租户支持，DBA 视图在所有租户支持。

缺陷修复

• 修复查询改写时，表达式和常量替换判断不正确导致的查询结果异常的问题。 #1998
• 修复不同日志流间回放差距过大的情况下，可能导致备库同步超时失败的问题。
• 修复备份恢复场景下，使用 COS 作为备份介质可能引发的内存泄漏问题。
• 修复从 V4.0.0 升级到 V4.2.1 场景下，使用触发器可能存在的不兼容问题。
• 修复转储中有大量删除操作时，索引表合并慢的问题。
• 修复 PHP 驱动执行删除语句，执行结果不正确的问题。
• 修复异常场景下使用 PL 可能触发的内存泄漏和 Core 等问题。
• 修复特定机型下使用 PL 会导致 Core Dump 的问题。
• 修复 limit + offset 使用造成溢出的问题。
• 修复特定场景下未使用 Sequence 时还会默认生成 Sequence 算子导致 Insert 或 Update 耗时长的问题。
• 修复 OBKV 前缀索引扫描报错 4016 的问题。
• 修复备份清理没有删除 Table List 导致 OCP 统计错误的问题。
• 优化 PX 线程数计算，修复并行执行耗时高问题。
• 修复特定场景 Insert 数据时，sqlexecutor 内存膨胀的问题。
• 修复 OBKV 在集群迁移过程中跑流量可能导致 Core Dump 的问题。
• 修复 gb18030 字符集导出乱码的问题。
• 修复 GIS 走不到空间索引的问题。
• 修复 Insertup 在打开 ob_enable_batched_multi_statement 开关后走 Batch 优化计算结果错误的问题。
• 修复特定外表导入场景 Core Dump 的问题。
• 修复特定场景下转储合并无法调动的问题。
• 修复取消 Truncate 语句导致丢掉全局索引的问题。

版本信息

项目	描述
发布日期	2024-07-30
版本号	V4.3.2_CE_BETA
Commit 号	7f23982
RPM 版本号	oceanbase-ce-4.3.2.0-100000092024072912

版本概览

OceanBase V4.3.2 是面向 AP 业务的性能增强版本，数据读取、数据分发、数据计算、向量化处理、计划选择全方位改进，AP 类基准测试性能整体提升 10%-15%。引入 Roaringbitmap 数据类型及相关计算表达式，给大数据集合运算和去重场景提供了更优选择。丰富增量旁路导入功能、提升全量旁路导入性能，扩展支持 Parquet 文件外表，加速大规模数据的处理速度。作为一款一体化数据库，新版本也针对 express oltp、complex oltp、olap、htap、kv 等 5 类场景分别梳理了场景化参数模版，配合部署工具，让初始配置就适合每一种业务类型。此外，新版本还融合了 V4.2.x 系列前序版本的诸多兼容性增强和用户体验改进特性，并合入了TP 类 SQL 的性能改进，致力于满足各类应用场景对产品的诉求。

关键特性说明

内核增强

• RoaringBitmap

  随着大数据时代的来临，企业对于用户数据的挖掘和分析需求日益增强。RoaringBitmap 凭借着其节省空间、计算高效等特点，在用户画像、个性化推荐、精准营销等业务场景发挥了重要作用。OceanBase V4.3.2 开始在支持 RoringBitmap 数据类型，通过存储和操作一组无符号整数，提升大数据量集合计算、去重性能。为了满足多维度分析需求，该版本支持了用于基数计算、集合运算、Bitmap 判断、Bitmap 构造、Bitmap 输出、聚合运算的二十余项表达式。

• 表级覆盖写

  数据仓库中进行数据定期刷新、数据转换、数据清洗修正的场景，数据覆盖写入是较为常见的需求。OceanBase V4.3.2 新增表级覆盖写能力（INSERT OVERWRITE），原子实现表内旧数据清空和新数据写入。基于全量旁路导入能力，INSERT OVERWRITE 也体现出较高的执行性能。V4.3.2 支持 INSERT OVERWRITE tablename SELECT * FROM tablename 用法，暂不支持分区级覆盖写，分区级功能将在后续版本提供。更多详细使用信息，参见 插入数据。

• 增量旁路导入功能增强

  V4.3.1 开始支持增量旁路导入实验特性，存在一些限制，如不支持 outrow lob 的数据导入，仅支持 load_mode 为 inc_replace（即 replace 语义）的增量旁路导入等。V4.3.2 新增支持对 outrow lob 数据进行增量旁路导入的能力；并扩展 load_mode 为 inc 取值，默认为 insert 语义，在 SQL 中指定 ignore 关键字时，表现为 ignore 语义。作为正式功能发布。更多详细使用信息，参见 使用 LOAD DATA 语句旁路导入数据 和 使用 INSERT INTO SELECT 语句旁路导入数据。

• 外表功能增强

  OceanBase 较早就支持了 CSV 格式的文件外部表，但随着 AP 业务的逐渐拓展，发现一些数据湖场景中，读取 Parquet 格式的外部数据源的需求也非常普遍。V4.3.2 开始支持 Parquet 文件外表，用户可通过外表将文件内数据导入 OceanBase 内表，也可直接将外表用于跨数据源的联合查询分析。同时，为了保证外表所扫描的文件目录的实效性，新版本增加文件目录自动刷新功能，建外部表时可通过 AUTO_REFRESH 选项指定文件列表的刷新方式（手动、实时、周期），并结合 DBMS_EXTERNAL_TABLE.REFRESH_ALL_TABLE（interval int）系统包子程序管理定时刷新任务。更多详细使用信息，参见 创建外表。

• 向量化引擎增强

  V4.3.0 支持了新版向量化引擎，V4.3.2 基于新版向量化引擎，继续增加适配了一批算子及表达式，如 Hash Set 相关算子、WindowFunction 算子、MergeDistinct 算子，repeat、ceil、floor 等表达式，进一步优化 AP 场景计算性能。

• 分场景参数模版

  OceanBase 作为一款一体化数据库，支持了多种业务类型，如 express oltp、complex oltp、olap、htap、kv 等。一套默认的系统参数不能很好地适用所有场景，例如 ob_query_timeout 默认是 10s，对于 olap 业务是不合适的。所以 V4.3.2 开始，基于不同的业务类型，OBServer 区分不同场景梳理了一些关键参数的最佳配置，后续也会结合 OCP、OBD 等工具进行分业务场景初始化。更多详细信息，参见 最佳实践。

• 查询解析能力增强

  SQL 中存在个数特别多的 INLIST、AND/OR、UNION 时，往往会产生超深语法树解析，进而造成内存消耗过大、栈不足、耗时过长的问题。新版本重写这些极端场景的查询解析实现，降低了解析阶段资源消耗，增强了极端 SQL 执行的稳定性，同时提升了 SQL 解析性能。

• 查询改写能力增强

  V4.3.2 版本优化器从多个方面优化了查询改写逻辑，例如：

  • 表达式规范化强化：历史版本在 Resolver 解析阶段会对多种表达式进行规范化（canonicalize），例如将 A and (A or B) 改写为 A ，但未覆盖到 SQL 改写阶段新产生的不规范表达式。新版本引入改写阶段的表达式规范化流程，确保将表达式改写到最简化状态。

  • 条件聚合函数改写：我们将形如 select c1, sum(case when c2 = 0 then c3 else 0 end) ,sum(case when...)...,... from t1 group by c1，聚合目标不是某个确定的表达式，而是在运行时根据分支选择条件的判定结果选择出的表达式，称为条件聚合函数。新版本支持了条件聚合函数改写，支持将聚合函数压入每个分支的选择目标上，减少 case when 表达式和聚合函数的计算次数，提升相关场景的执行性能。

  • 多 MIN/MAX 改写：在 scalar group by 中，若查询中的聚合函数只有 MIN/MAX，且 MIN/MAX 的列上有索引，可以将其改写成 ORDER BY xxx LIMIT 1，利用索引序消除 ORDER BY 并将 LIMIT 1 下推到 table scan 上，从而减少对数据的读取和排序。对于 SQL 中有多个 MIN/MAX 存在的场景，历史版本采取了全表扫描方案，新版本考虑将多个 MIN/MAX 改写成多个子查询，每个子查询都利用索引直接获取到最大值或最小值，以此来避免全表扫描和排序，优化查询性能。

  • 常量 UNION 改写 Values Table：业务较常使用多个 UNION ALL 的组合形式表示一个常量表，在子查询数量过多时，会有较多 CPU 和内存消耗。新版本对常量类型的 UNION ALL/UNION 改写成对 Values Table 的查询，显著降低硬解析阶段的资源消耗。

  • SEMI JOIN 拆分优化：EXISTS 或 IN 子查询中存在多表关联且没有连接条件时，执行计划可能会出现开销较大的笛卡尔积。新版本支持对没有连接条件的表做 SEMI JOIN 拆分改写，避免笛卡尔积开销。

• 计划选择优化

  之前版本已经实现了新版 Query Range 抽取逻辑，扩展了谓词下压场景，使向量形式谓词的 Range 抽取更加精准，也解决了旧版 Query Range 的一些内存放大的场景问题。V4.3.2 在此基础上修改了 NOT IN 表达式生成 Query Range 的方式，优化了 Range 抽取性能；支持 Range Graph 裁剪，减少最终 Query Range 的抽取数量，降低内存消耗；扩展 index Hint，如指定 /*+index(t1 k1 1)*/ 时，限制只对 k1 索引的第 1 列进行 Query Range 抽取。同时，新版本也支持了 interesting ordering 索引路径按规则裁剪，LIMIT 重计算等策略优化，解决 ORDER BY LIMIT 场景因计划选择不优导致的性能问题。

• 产品行为兼容版本控制

  为了减少因为新版本行为变更导致升级后某些老版本特性使用报错的问题，OceanBase V4.3.2 版本新增产品行为兼容版本控制功能，支持通过 ob_compatibility_version、ob_security_version 系统变量，分别控制普通行为变更（不报错->报错场景）、安全行为变更（有权限->无权限）是否生效，一个租户中的兼容功能或安全功能按照哪个 OceanBase 版本来兼容，取值范围为 "4.3.2.0" 等发行版本。例如一个功能在 V4.3.3 发生了产品行为变更，当变量取值为 4.3.2.0 时，使用旧行为；取值为 4.3.3.0 时，使用新行为。通常升级场景不会自动推高该版本号，需要新版本行为时，请升级后，了解新版本行为并测试后，修改为和当前版本一致的版本号。该方案仅用于控制未来新增的产品行为变更，无法控制已经发版的存量产品行为变更。

  V4.3.2 版本通过 ob_compatibility_version 控制的产品行为有：

  • ob_compatibility_control = MYSQL5.7 时，REPLACE('abd', '', null) 行为由兼容 MySQL 8.0 修改为兼容 MySQL 5.7。
  • UPDATE/DELETE statement 在 limit 中禁用 offset。
  • 当投影项是单独的一个 null 值的时候，返回的表头会被修改为 NULL。
  • 限制用户变量最长 64 字符。

  通过 ob_security_version 控制的产品行为有：

  • Outline、Sequence 增加权限管控。
  • CREATE TABLESPACE 权限。

多模特性

• GIS 空间关系计算性能优化

  新版本优化了 ST_INTERSECTS/ST_CONTAINS/_ST_COVERS/ST_WITHIN 等空间关系计算表达式的计算性能。

  • 在全量查询窗查询场景，point 的 st_intersects 与 PG 持平，且略优于 PG；其余测试场景的平均 rt 均不到 PG 的一半，其中 linestring 的 intersect 用时只有 PG 的 1/5 ，相比 OceanBase 历史版本均有数量级的提升。
  • 在大查询窗场景，所有场景的平均 rt 均明显优于 PG，其中 linestring 的 intersect 用时仅有 PG 的 1/7，且相比 OceanBase 历史版本均有数量级的提升。
  • 在小查询窗场景，contain 计算会优于 PG，其中 point 的 contain 用时是 PG 的 1/4 左右，linestring contain 是 PG 的 4/5。但 intersect 的计算性能在 linestring 场景仅与 PG 持平，在 POINT 场景用时是 PG 的 2 倍左右。

• OUTROW 存储的 LOB 读写性能优化

  V4.3.2 版本重点对 OUTROW 存储的 LOB 进行了性能优化。

  • Table Scan 场景，相对历史版本，新版本中小 LOB 性能可以提升近 10 倍，大 LOB 可以提升至少 2 倍。但该场景相对 INROW 仍然慢一些。
  • Point Select 场景，新版本小 LOB (8K 以下)，OUTROW 的 QPS 比 INROW 低 20% 以内，中等以上大小的 LOB（32K 以上），OUTROW 的 QPS 比 INROW 低 5% 左右。在 LOB 不参与计算的场景，LOB OUTROW 存储对查询性能会更优。
  • Point Update 场景，相对历史版本，新版本 OUTROW 性能平均提升 2 倍。

MySQL 兼容

• 区分 MySQL 5.7/8.0 兼容版本

  MySQL 5.7 与 8.0 在部分场景下存在产品行为冲突，如 replace('a','',null) 在两个版本的输出结果不同。OceanBase V4.3.2 版本新增租户级初始化变量 ob_compatibility_control，用于在创建租户时指定存在产品行为冲突的情况是兼容 MySQL 5.7 版本，还是 8.0 版本。租户创建后不可修改，两种模式下均可使用不存在行为冲突的 MySQL 5.7/8.0 的特性超集。

• Union Distinct RCTE

  MySQL 8.0 支持了 CTE 的 Recursive Union All 及 Recursive Union Distinct 功能。OceanBase 从 V3.2.3 版本开始支持 Recursive Union，但仅支持 Recursive Union All，V4.3.2 版本扩展兼容了的 MySQL Recursive Union Distinct 功能，保证输出数据的唯一性。同时，新版本还加强了 Recursive Union All 功能，支持可使用内存不足时进行数据落盘。

• 自增列切主跳变优化

  OceanBase 在 V4.x 支持了创建 ORDER 模式的自增列，更好地兼容了 MySQL 自增列行为。但是 V4.3.2 之前版本在出现切主时，仍会造成自增列跳变。考虑到很多切主场景是用户主动发起的流程，并非异常场景，所以新版本优化为主动切主时仍保持自增列连续性，降低自增列跳变概率。

• 自增列改小

  V4.3.2 之前的版本，仅支持通过 ALTER TABLE 将 AUTO_INCREMENT 值改大，不支持改小，行为上和 MySQL 不完全兼容。新版本补充支持了 AUTO_INCREMENT 值改小的功能，指定新值大于自增列已存在的最大值时，可以设置成功且生效；指定新值小于或等于自增列已存在的最大值时，可以设置成功，但 AUTO_INCREMENT 会自动调整为自增列已存在的最大值的下一个值。更多详细信息，参见 定义自增列。

• SERIAL 数据类型

  新增 SERIAL 数据类型，作为 BIGINT UNSIGNED NOT NULL AUTO_INCREMENT UNIQUE 属性别名，来表示一个自动递增的大整数，适合用作表的主键。更多详细信息，参见 整数类型。

• 补充支持 MySQL 通信协议命令字

  兼容 MySQL 5.7 通信协议 COM_PROCESS_INFO、COM_PROCESS_KILL 命令字。

性能提升

• AP 典型场景性能优化

  V4.3.2 对数据块预取、向量化批处理、Filter 下压、比较计算、聚合计算、DTL shuffle、单调性 Filter 等进行了全方位策略优化，100G 数据量规格下，TPCH、TPCDS、ClickBench 等多类 AP 场景基准测试性能相比 V4.3.1 提升了约 10%～15%。

• 全量旁路导入性能优化

  全量旁路导入场景，新版本通过减少数据加载过程中的列类型转换操作、降低统计信息收集操作的 CPU 占用、INSERT INTO SELECT 去除 SELECT 端生成 pk 逻辑、列存默认关闭 sum skip index（存储层对应 Range 内指定列数据的预聚合 SUM 值）等优化，配合关闭微块校验（配置项 micro_block_merge_verify_level 设置 0），旁路导入性能提升 20% 左右。

• IN/EXISTS 子查询提升改写

  在 V4.3.2 之前，OceanBase 不支持将 SELECT 子句中的 IN/EXISTS 子查询改写成连接形态。如 select case when c1 in (select c1 from t2) then 1 else 0 end from t1; 语句，优化器不会改写连接，这导致子查询被反复扫描，性能较差。新版本支持了部分场景下 SELECT 子句中 IN/EXISTS 子查询的提升改写，尽量将子查询改写为连接形态，充分利用并行提升 SQL 执行性能。

• 存储过程 DDL 执行编译结果缓存和落盘

  V4.3.1 新增执行期存储过程编译落盘功能，解决了执行期存储过程获取 PL Cache 缓存失败，从而重编译引起的性能问题。但是存储过程自身做过 DDL 变更后，后续执行存储过程依旧需要重新编译，这个场景依然存在性能优化空间。V4.3.2 版本补充支持存储过程 DDL 执行成功后将编译结果缓存到 PL Cache 并落盘的行为，后续执行存储过程时，提升直接命中 PL Cache 缓存的概率，进一步提高存储过程执行性能。

• Show Table Status 性能优化

  老版本 SHOW TABLE STATUS FROM ... LIKE ... 场景性能较差，未利用 table_name 相关索引。新版本针对存在单表过滤条件的场景，显著提升查询性能。

• 自增列 Order 模式性能优化

  自增列 Order 模式下，需要保持数据插入的连续性。为了保证分布式架构下自增列的有序，在自增列 INSERT 时会将该值持久化到内部表，高并发情况下，响应时间会比较长。V4.3.2 对该场景做了优化，通过 cache size 预取有效降低内部表访问次数，从而显著提升性能。

• Sequence Order 模式性能优化

  OceanBase 已支持指定 order + cache 属性来创建全局连续生成的序列，但早期的实现中会忽略 cache 属性，等同于 order + nocache，在高并发场景会有明显性能问题。V4.3.2 版本对该场景进行了优化，通过选取中心节点来支持 order + cache 生成连续序列，真正意义上实现 cache 属性，显著提升高并发场景执行性能。

可靠性提升

• 备租户克隆

  OceanBase 自 V4.3.0 开始支持租户克隆功能，当用户需要对在线租户进行高资源消耗的临时数据分析或其他高风险操作时，为了避免对在线租户造成影响，可使用克隆租户完成分析或验证。同时也可将克隆租户作为容灾手段，若原始租户发生了难以恢复的误操作，可使用克隆租户进行数据回滚。但 V4.3.2 之前只限于对主租户进行克隆，主租户克隆时会阻塞备租户的日志回放，在备租户的同步与回放优先级高于克隆的场景下，主租户克隆无法达到稳定性要求。新版本增加备租户克隆能力，用户可通过克隆备租户进行业务验证或容灾，并且不会阻塞备租户的日志回放，降低系统稳定性影响。

• Transfer 事务影响优化

  V4.3.0 开始支持 Transfer 搬迁活跃事务，例如在 Transfer 源端日志流上允许非迁移 tablet 的请求继续执行并且允许开启新事务，对于已经开启的事务会短暂阻塞事务提交，Transfer start 阶段完成后再进行事务提交。在此基础上，V4.3.2 继续优化了 Transfer 对事务执行的影响，如果 Transfer 源端日志流上的事务没有操作过迁移的 tablet，变更为不阻塞事务提交，也不搬迁这些事务，尽量使得前台业务无感知，负载均衡更加透明。

• 网络备库性能优化

  新版本对 RPC 及网络框架进行优化，降低了主备库之间的日志传输耗时；同时优化了备库受控拉日志流程，减少了备租户拉日志受控的频率，提高了主备之间的日志同步性能。

• 备份性能优化

  OceanBase 备份过程中通过连续性校验确保基线版本大于转储版本以保证数据完整，但连续性检查涉及读取备份介质上的数据并执行带锁操作，影响了备份性能。新版本优化了备份过程中连续性校验操作的性能开销，支持通过 ha_low_thread_score 控制备份使用的线程数， 有效提高备份性能。

• 迁移复制源端选择优化

  历史版本选择迁移复制的源端时没有优先考虑同 Zone、同 IDC（同机房）、同 Region（同城）的副本，可能出现远距离迁移复制性能不优的问题。并且源端可能会选择 Leader 节点，业务请求量较高时，可能导致业务请求和迁移任务较慢的问题。V4.3.2 将各 Server 按照地域信息划分为同 IDC、同 Region 不同 IDC、跨 Region 三种区域关系，提供枚举配置项 choose_migration_source_policy，支持用户配置指定迁移复制选择源端的优先模式，以便优先考虑地理位置就近因素以及 Follower 副本来提升迁移效率、降低 Leader 压力。

• SET/PIECE 级物理恢复

  实际业务中存在二次备份场景，会把数据备份集或归档日志手动搬迁到新的路径。OceanBase 的物理恢复功能，限制了必须在租户的归档和备份路径中恢复，无法使用手动搬迁到新路径的备份数据，限制了恢复的灵活性。新版本增加了 SET/PIECE 级物理恢复功能，提供 ADD RESTORE SOURCE 命令来加载新路径的数据备份集 SET 或日志归档 PIECE，允许按需恢复到指定时间。

资源优化

• 分区局部索引空间优化

  后建索引流程中，通过发送内部 SQL 加载索引表数据，如果某个节点数据量很大，在排序过程中，可能引起临时空间放大。V4.2.2 版本支持了索引表补数据的渐进式执行，单机 1GB 数据量在同一批次执行，多机任务并行执行，这样在充分利用了分布式系统并行计算能力的同时，也能够解决在索引构建过程中出现的临时空间放大问题。

• 全局 CPU 前后台任务隔离

  在高性能计算环境中，资源的合理分配与隔离对于确保系统稳定性和提升效率具有决定性作用。有效的资源隔离策略可以预防任务间的资源争夺和相互干扰，从而提升资源利用效率和整体服务质量。目前，OceanBase 已实现了通过租户 Unit 规格来配置租户间的资源隔离，通过 DBMS_RESOURCE_MANAGER 系统包来配置租户内的资源隔离，涵盖 CPU 和 IO 两大重要资源。但对于租户数较多、不希望后台任务对前台请求造成严重 CPU 竞争的业务，在全局层面上对后台任务进行隔离是一个更好的选择。OceanBase V4.3.2 版本支持了全局 CPU 前后台任务隔离能力，可以在整体层面上限制后台任务的可用资源，相对租户内使用 DBMS_RESOURCE_MANAGER 单独配置更加方便易用。

安全强化

• MySQL CREATE TABLESPACE 权限

  新增 CREATE TABLESPACE 权限，仅适用于 TDE 场景。

• 增加 Outline、Sequence 权限管控（非兼容特性）

  历史版本缺少对创建维护 Outline、Sequence 的权限管控，新版本复用 MySQL CREATE、ALTER、DROP 权限，控制 Outline、Sequence 的创建、修改和删除。

易用性提升

• Buffer 表自适应合并优化

  当用户在某张表上频繁地执行插入并且同时进行批量删除，或者有大量的并发更新操作时，可能会遇到一种现象：表中的数据行数并不大，但是查询和更新的性能出现明显下降。这种现象在 OceanBase 中称为 Queuing 表（业务上有时又称 Buffer 表）效应。Buffer 表是 LSM-Tree 架构数据库都要面对的一类问题。LSM-Tree 架构下的删除操作在合并之前都只是逻辑上标记删除而非物理删除，当增量数据中存在大量标记删除的数据时，物理行数量将远多于逻辑行，从而造成严重的读放大现象，并影响优化器执行方案的生成。

  OceanBase 在 V4.1.0 做到了自动识别哪些分区在一段时间内发生比较多更改，并对这些分区进行自适应 compaction，但缺少人为控制手段。为了更灵活地解决 Buffer 表带来的性能下降问题，V4.3.2 版本继续优化了 Buffer 表自适应合并特性，提供了 5 种档位的合并策略，允许用户根据业务场景为每张表设置不同的 table_mode，来应对 Buffer 表引起的读放大现象，从而提高系统长期运行下的 QPS 等性能指标。

• 系统日志优化

  系统日志目录下，新增 ./alert/alert.log 日志文件，用于记录 DBA 关注的日志信息，初步解决 observer.log 日志量大、可读性差的问题。可通过 alert_log_level 集群级配置项设置日志级别，包括 INFO、WARN、ERROR 等。同时提供了系统外部表 sys_external_tbs.__all_external_alert_log_info 用于直接结构化查询 alert.log 日志信息。更多详细信息，参见 日志概述。

• OceanBase LogMiner

  V4.3.2 版本新增 OceanBase LogMiner（简称 ObLogMiner）工具支持。ObLogMiner 是一款用来对 OceanBase 数据库进行日志分析的命令行工具，支持在线及离线的日志分析。ObLogMiner 通过 OBCDC 来拉取并解析 CLOG 日志，并将 OBCDC 输出的逻辑日志转化成易读的格式存储到指定位置。适用于以下场景：

  • 数据误操作：数据误操作可能由于多种原因出现，比如 WHERE 子句中的范围条件错误而误删除或更新了多余的行。通过使用 ObLogMiner 可以准确了解误操作的详细信息，以便于将数据库恢复到误操作之前的状态。
  • 数据分析：ObLogMiner 会将 CLOG 日志中的各种信息(事务、表结构等) 组织并友好地展示出来，用户可以借助 ObLogMiner 的输出结果进行多样的数据分析。也可以配合外表功能读取 ObLogMiner 的输出结果在数据库中进行查询分析。

• 自增列 Cache Size 表级设定

  当前已支持通过指定租户级配置项 auto_increment_cache_size 来控制内存中自增列一次申请缓存的个数，对所有使用自增列的表生效。通常情况下，cache size 设置越大，性能会越好。但如果跳变次数很多，自增列字段取值上限又比较小，很可能导致自增列快速耗尽。V4.3.2 版本新增自增列 cache size 表级设定功能，用户可以根据列类型/业务模型/业务流量等自定义配置不同表的缓存大小，从而做到跳变和性能的均衡。

• PX 诊断能力增强

  为方便分布式计划的问题排查，V4.3.2 版本在 SQL Plan Monitor 中增加记录了算子运行时的内存使用量、磁盘使用量，整个执行过程的最大内存使用量、最大磁盘使用量等信息。并在 SQL Audit 中增加记录 QC 对应 trace_id 下的 memtable、sstable 行数信息，并支持汇总各 PX Worker 对应 trace_id 下的相关信息。同时也支持了 OBDiag 工具依据 trace_id 拉取最初报错机器日志的功能。

性能报告

基于面向不同业务类型的参数模版，该版本进行了如下性能对比测试，其中 AP 基准测试中，使用 olap 参数模版，性能相对其他模版有明显提升。

硬件规格：

• OBServer：3 台 32c256g 机器，机型为 ecs.g7.8xlarge，日志盘使用系统盘，clog 和 data 盘单独挂载两块云盘，磁盘性能级别为 PL1。

• ODP：单独部署在一台 64c128g 机器，机型为 ecs.c7.16xlarge。

操作系统：

CentOS 7.9 64 位。

租户规格：

28c180g 规格，3F 副本，primary_zone = RANDOM。

1. SYSBENCH OLTP 负载测试

基础参数设置：

ALTER system SET enable_sql_audit=false;
ALTER system SET enable_perf_event=false;
ALTER system SET syslog_level='PERF';
ALTER system SET enable_record_trace_log=false;

• table：30
• table_size：100w
• time：30s

测试结果(QPS/95%rt)：

• Point Select 性能

  Threads	express_oltp 参数模版	complex_oltp 参数模版	olap 参数模版	htap 参数模版
  32	163457.60/0.22	162747.70/0.22	161428.80/0.23	162948.95/0.20
  64	296206.41/0.25	291823.36/0.26	291583.48/0.26	293622.63/0.25
  128	505203.80/0.30	493859.95/0.31	492135.78/0.31	498132.19/0.31
  256	798005.94/0.45	794547.97/0.47	803165.10/0.49	797304.31/0.45
  512	1039286.05/0.90	1023822.11/1.14	1022666.33/1.12	1032713.76/0.90
  1024	1013992.61/2.39	1011295.14/2.39	997362.00/2.57	1004848.34/2.48

• Read Only 性能

  Threads	express_oltp 参数模版	complex_oltp 参数模版	olap 参数模版	htap 参数模版
  32	134791.19/4.10	136145.15/3.97	137486.55/3.96	137327.53/3.95
  64	244754.37/4.49	244093.17/4.57	244641.01/4.57	244586.46/4.57
  128	416929.45/5.37	420143.73/5.47	419772.35/5.28	420445.05/5.28
  256	613453.13/7.56	611436.43/8.28	603989.96/8.28	610998.14/7.43
  512	725364.76/16.12	738362.91/17.65	736059.64/15.83	720899.31/16.12
  1024	715777.22/41.10	707831.35/42.61	697077.19/44.17	706809.11/42.61

• Write Only 性能

  Threads	express_oltp 参数模版	complex_oltp 参数模版	olap 参数模版	htap 参数模版
  32	50914.06/5.00	52894.62/4.91	50589.47/5.67	52088.46/4.74
  64	90119.99/5.47	93447.67/5.37	90202.65/5.37	90264.56/5.57
  128	164488.33/5.77	166099.69/5.57	159493.96/5.99	159005.24/6.09
  256	242240.38/8.13	241749.01/8.43	232320.85/8.43	230522.31/8.74
  512	304060.67/13.70	306416.65/13.70	299155.86/13.70	289147.63/13.95
  1024	345068.37/23.52	348929.05/26.20	306096.92/29.72	327905.15/27.17

• Read Write 性能

  Threads	express_oltp 参数模版	complex_oltp 参数模版	olap 参数模版	htap 参数模版
  32	90881.38/7.84	88141.94/8.28	88216.59/8.58	89948.44/7.98
  64	159748.46/8.90	160695.31/9.06	157714.41/10.09	157230.31/9.39
  128	273142.95/10.46	275431.02/10.27	272648.28/10.27	269700.79/11.24
  256	391348.85/15.27	402154.83/15.00	382679.53/16.71	383447.47/15.27
  512	465031.62/28.67	462574.18/33.72	466465.96/26.20	461249.29/27.66
  1024	525924.96/52.89	535977.26/48.34	510540.58/58.92	522066.61/51.02

2. BMSQL TPCC 测试

基础参数设置：

#ODP
ALTER proxyconfig SET proxy_mem_limited='4G';
ALTER proxyconfig set enable_compression_protocol=false;

#OBServer SYS 租户
ALTER system SET enable_sql_audit=false;
ALTER system SET enable_perf_event=false;
ALTER system SET syslog_level='PERF';
ALTER system SET enable_record_trace_log=false;

• warehouse=1000
• terminals=800

测试结果：

	express_oltp 参数模版	complex_oltp 参数模版	olap 参数模版	htap 参数模版
tpmC (NewOrders)	292204.95	302608.69	264422.69	294316.6
tpmTOTAL	648918.9	672866.36	587580.74	654271.82
Transaction Count	3246324	3366013	2939166	3272514

3. TPCH 测试

基础参数设置：

#OBServer SYS 租户
ALTER system flush plan cache GLOBAL;
ALTER system SET enable_sql_audit=false;
ALTER system SET enable_perf_event=false;
ALTER system SET syslog_level='PERF';
ALTER system SET enable_record_trace_log=false;

#测试租户
SET GLOBAL ob_sql_work_area_percentage = 80;
SET GLOBAL ob_query_timeout = 36000000000;
SET GLOBAL ob_trx_timeout = 36000000000;
SET GLOBAL max_allowed_packet = 67108864;
SET GLOBAL parallel_servers_target = 624;

100G

测试结果（均使用列存表）：

Query	express_oltp 参数模版	complex_oltp 参数模版	olap 参数模版	htap 参数模版
Q1	32.33s	9.86s	1.83s	3.03s
Q2	1.42s	0.52s	0.22s	0.26s
Q3	7.79s	2.58s	0.52s	0.91s
Q4	9.55s	3.01s	0.27s	0.66s
Q5	15.77s	4.59s	0.73s	1.29s
Q6	0.25s	0.11s	0.06s	0.06s
Q7	9.76s	3.65s	1.19s	1.76s
Q8	5.72s	1.85s	0.39s	0.61s
Q9	25.26s	8.93s	1.88s	3.28s
Q10	3.96s	1.74s	0.46s	0.75s
Q11	2.01s	0.61s	0.14s	0.21s
Q12	6.00s	1.89s	0.33s	0.66s
Q13	8.40s	3.43s	1.64s	1.94s
Q14	0.99s	0.45s	0.19s	0.25s
Q15	1.29s	0.64s	0.31s	0.34s
Q16	2.69s	1.07s	0.53s	0.61s
Q17	3.85s	1.15s	0.21s	0.31s
Q18	17.53s	5.51s	1.33s	1.96s
Q19	1.49s	0.60s	0.24s	0.31s
Q20	6.02s	2.38s	1.33s	1.33s
Q21	26.01s	9.11s	2.70s	3.60s
Q22	6.01s	2.23s	0.79s	1.00s
Total	194.10s	65.91s	17.26s	25.13s

4. TPC-DS 测试

#OBServer SYS 租户
ALTER system flush plan cache GLOBAL;
ALTER system SET enable_sql_audit=false;
ALTER system SET enable_perf_event=false;
ALTER system SET syslog_level='PERF';
ALTER system SET enable_record_trace_log=false;

#测试租户
SET GLOBAL ob_sql_work_area_percentage = 80;
SET GLOBAL ob_query_timeout = 36000000000;
SET GLOBAL ob_trx_timeout = 36000000000;
SET GLOBAL max_allowed_packet = 67108864;
SET GLOBAL parallel_servers_target = 624;

100G

测试结果：

Query	express_oltp 参数模版	complex_oltp 参数模版	olap 参数模版	htap 参数模版
Q1	0.62	0.33	0.21	0.21
Q2	10.19	3.29	0.99	1.26
Q3	0.20	0.13	0.11	0.11
Q4	36.42	17.89	11.29	11.90
Q5	3.95	1.79	0.98	1.11
Q6	0.55	0.31	0.22	0.23
Q7	0.81	0.38	0.19	0.21
Q8	0.55	0.32	0.24	0.24
Q9	2.15	0.95	0.47	0.51
Q10	1.81	0.85	0.52	0.54
Q11	22.09	10.94	7.09	7.35
Q12	0.47	0.29	0.22	0.24
Q13	0.53	0.27	0.18	0.19
Q14	81.19	26.90	7.86	10.40
Q15	0.71	0.43	0.38	0.39
Q16	5.18	1.74	0.57	0.73
Q17	1.26	0.64	0.39	0.44
Q18	0.71	0.39	0.26	0.28
Q19	0.30	0.20	0.15	0.16
Q20	0.39	0.26	0.19	0.21
Q21	0.99	0.39	0.16	0.20
Q22	4.93	2.14	1.12	1.26
Q23	92.37	34.42	13.27	16.41
Q24	3.47	1.55	0.84	0.93
Q25	1.14	0.59	0.41	0.43
Q26	0.49	0.26	0.16	0.19
Q27	1.14	0.54	0.31	0.33
Q28	1.37	0.95	0.83	0.84
Q29	3.94	1.34	0.46	0.56
Q30	0.40	0.27	0.22	0.22
Q31	2.37	1.08	0.60	0.67
Q32	0.12	0.11	0.10	0.10
Q33	1.21	0.87	0.63	0.66
Q34	2.22	0.77	0.22	0.29
Q35	3.57	1.51	0.76	0.85
Q36	1.98	0.85	0.29	0.40
Q37	0.52	0.32	0.23	0.24
Q38	7.00	2.99	1.50	1.70
Q39	2.98	1.31	0.72	0.81
Q40	0.29	0.18	0.15	0.15
Q41	0.04	0.04	0.04	0.03
Q42	0.18	0.12	0.10	0.10
Q43	3.89	1.43	0.47	0.60
Q44	0.50	0.45	0.44	0.46
Q45	0.47	0.37	0.35	0.36
Q46	0.97	0.46	0.28	0.30
Q47	5.12	2.11	0.99	1.11
Q48	0.54	0.29	0.17	0.18
Q49	1.25	0.96	0.84	0.85
Q50	8.07	2.36	0.44	0.65
Q51	22.35	7.02	2.81	3.04
Q52	0.17	0.13	0.10	0.10
Q53	1.56	0.52	0.17	0.20
Q54	2.24	0.97	0.54	0.57
Q55	0.14	0.11	0.10	0.10
Q56	0.67	0.61	0.60	0.59
Q57	2.88	1.29	0.66	0.74
Q58	1.15	0.85	0.69	0.69
Q59	18.73	6.56	2.10	2.64
Q60	1.16	0.85	0.67	0.69
Q61	0.41	0.33	0.29	0.29
Q62	3.00	1.15	0.37	0.47
Q63	1.57	0.52	0.16	0.20
Q64	6.74	3.01	1.45	1.71
Q65	4.18	1.66	0.73	0.86
Q66	1.00	0.61	0.38	0.40
Q67	21.56	13.49	10.43	10.28
Q68	0.37	0.26	0.22	0.23
Q69	1.23	0.66	0.48	0.50
Q70	5.39	2.26	1.15	1.28
Q71	1.20	0.67	0.42	0.45
Q72	22.39	14.68	10.40	11.09
Q73	0.63	0.32	0.17	0.20
Q74	14.22	6.83	3.68	3.93
Q75	6.72	2.80	1.42	1.55
Q76	0.39	0.38	0.39	0.38
Q77	1.39	0.88	0.73	0.72
Q78	14.46	5.91	2.57	3.00
Q79	2.66	1.17	0.48	0.56
Q80	2.94	1.50	0.97	1.00
Q81	0.63	0.29	0.16	0.18
Q82	0.78	0.45	0.32	0.33
Q83	1.14	0.75	0.58	0.59
Q84	0.58	0.28	0.18	0.18
Q85	0.71	0.45	0.36	0.36
Q86	1.14	0.56	0.34	0.39
Q87	7.29	3.04	1.59	1.72
Q88	1.83	0.78	0.29	0.36
Q89	1.78	0.72	0.23	0.32
Q90	0.44	0.22	0.14	0.15
Q91	0.14	0.11	0.10	0.10
Q92	0.11	0.10	0.10	0.10
Q93	7.68	2.24	0.47	0.64
Q94	2.74	1.08	0.49	0.56
Q95	39.75	18.00	6.97	8.23
Q96	2.43	0.78	0.20	0.26
Q97	7.34	2.64	1.01	1.22
Q98	0.64	0.42	0.31	0.33
Q99	6.00	2.16	0.68	0.80
Total	570.64s	242.76s	119.88s	134.26s

兼容性变更

产品行为变更

新增如下变更：

功能	变更说明
列存表默认不会再自动创建 sum skip index	老版本列存表会自动为每一列创建 sum skip index（存储层对应 Range 内指定列数据的预聚合 SUM 值）。为了提升导入性能，V4.3.2 开始默认不会创建 sum skip index，在对表的原始列进行 sum 计算的场景，可以通过显示指定 SKIP_INDEX(SUM) 的方式来提速。如：create table t_default1(pk int primary key SKIP_INDEX(SUM), a int default 10 SKIP_INDEX(SUM));。
禁止在 delete/update 语句中的 limit 子句中使用 offset 语义	新版本在 ob_compatibility_version = 4.3.2.0 或未来指定之后的 4.3.x 版本时，禁止在 delete/update 语句中的 limit 子句中使用 offset 语义。具体而言，OceanBase 原先支持的 update/delete...limit x,x 和 update/delete...limit x offset x 写法，现在需要语法报错，与 MySQL 行为保持一致。
限制用户变量最长 64 字符	新版本在 ob_compatibility_version = 4.3.2.0 或未来指定之后的 4.3.x 版本时，用户变量长度由不设限修改为最长 64 字符。
[g]v$ob_sql_audit 视图 sql_id 字段取值变更	V4.3.2 之前的 V4.3.x 系列版本，call 语句的 sql_id 为空字符串生成的 MD5 编码；匿名块语句的 sql_id 为未经参数化的原始 PL 代码字符串生成的 MD5 编码。V4.3.2 版本将两者的 sql_id 都修改为参数化之后的语句生成的 MD5 编码。
Rename Table 变更为加锁操作	历史版本中 Rename Table 是一个不需要加锁的 Online DDL 操作，事务跨 Rename Table 操作时，可能会出现非预期的问题。V4.3.2 版本支持对 Rename Table 加表锁，并增加 Rename Table 期间的读写防御。
REPLACE('abd', '', null) 返回结果按 ob_compatibility_control 分别兼容	新版本在 ob_compatibility_version = 4.3.2.0 或未来指定之后的 4.3.x 版本时，老版本升级后或新建租户 ob_compatibility_control = MYSQL5.7 的话，REPLACE('abd', '', null) 行为由兼容 MySQL 8.0 修改为兼容 MySQL 5.7。
未指定别名的 null 值投影列名称修改为 NULL	MySQL 会将未指定别名的 null 值（\N, null, Null...）投影列的名称设置为 NULL，而出现在复合表达式中的 null 会保持原串。OceanBase 老版本不会对 null 值投影列的名称做任何修改。新版本在 ob_compatibility_version = 4.3.2.0 或未来指定之后的 4.3.x 版本时，会修改为和 MySQL 相同行为。
Outline、Sequence 新增权限管控	新版本在 ob_security_version = 4.3.2.0 或未来指定之后的 4.3.x 版本时， 需要授予 CREATE/ALTER/DROP 权限后，用户才可以创建和管理 Outline、Sequence。
创建和管理 TABLESPACE 新增 CREATE TABLESPACE 权限管控	新版本在 ob_security_version = 4.3.3.0 或未来指定之后的 4.3.x 版本时， 需要授予 CREATE TABLESPACE 权限后，用户才可以创建和管理 TABLESPACE。

视图变更

新增如下变更：

视图	变更类型	变更说明
V$OB_COMPATIBILITY_CONTROL	新增	用于展示所有可以按 OceanBase 发行版本进行产品行为兼容控制的功能。
[G]V$SQL_WORKAREA	新增列	新增 DB_ID 字段，用于描述该请求的连接所属的数据库 ID。
information_schema.events	新增	V4.3.2 仅支持表结构，暂未支持 event 功能。
sys_external_tbs.__all_external_alert_log_info	新增	sys 租户下新增系统外部表，用于结构化查看 alert.log 日志信息。

配置项变更

新增如下变更：

配置项/系统变量	变更类型	变更说明
choose_migration_source_policy	新增	新增租户级配置项，用于控制迁移源端的选择策略。提供 2 种选择：idc：为默认值。在同 idc 的机器中优先选择 follower 副本作为源端，若仅有 leader 副本，则选 leader 副本。region：在同 region 的机器中优先选择 follower 副本作为源端，若仅有 leader 副本，则选 leader 副本。
storage_rowsets_size	新增	新增租户级配置项，用于控制列存引擎一次向量化批处理行数，默认 8192。
alert_log_level	新增	新增集群级配置项，用于控制 alert.log 的日志级别，如 INFO、WARN、ERROR，默认为 INFO。
enable_global_background_resource_isolation	新增	新增集群级系统配置项，用于控制是否对全局的前后台任务进行 CPU 资源隔离。默认为 False，表示不隔离。
global_background_cpu_quota	新增	新增集群级系统配置项，用于控制 enable_global_background_resource_isolation 为 True 时，后台任务可使用的 CPU 配额。默认为 -1，表示不受 cgroup 限制。
ob_default_lob_inrow_threshold	变更默认值	该配置项用于指定 LOB INROW 存储的最大阈值，默认值由 4096 调整为 8192。

系统变量变更

新增如下变更：

系统变量	变更类型	变更说明
ob_compatibility_control	新增	新增 GLOBAL 级系统变量，用于控制存在兼容行为冲突时，OceanBase 和 MySQL 5.7 行为一致，还是和 MySQL 8.0 行为一致。默认为 MySQL5.7。创建租户时指定，租户创建后不允许修改。
ob_compatibility_version	新增	新增租户下 GLOBAL 系统变量，用于控制产品行为发生变更的功能，行为和 OceanBase 哪个发行版本兼容。新建集群时，默认为当前集群版本；版本升级时，为前一个版本配置，当前默认 4.3.2.0。可修改。
ob_security_version	新增	新增租户下 GLOBAL 系统变量，用于控制安全相关的产品行为发生变更的功能，行为和 OceanBase 哪个发行版本兼容。新建集群时，默认为当前集群版本；版本升级时，为前一个版本配置，当前默认 4.3.2.0，可修改，和 ob_compatibility_version 区别为只能推高，不能回退。

系统包变更

系统包	变更类型	变更说明
DBMS_EXTERNAL_TABLE	新增	新增 DBMS_EXTERNAL_TABLE.REFRESH_ALL_TABLE(interval int) 系统包子程序，用于管理外表文件列表定时刷新任务。

函数变更

函数名	变更类型	变更说明
PASSWORD	新增	用于加密字符串。
Roaringbitmap 相关函数	新增	支持了用于基数计算、集合运算、Bitmap 判断、Bitmap 构造、Bitmap 输出、聚合运算的二十余项表达式。

语法变更

语法	变更说明
新增 insert overwrite 覆盖写语法支持	详细信息参见 INSERT。
新增外表文件目录自动刷新相关选项	外部表创建语句新增 AUTO_REFRESH 选项指定文件列表的刷新方式（手动、实时、周期）。详细使用信息参见 CREATE EXTERNAL TABLE。
新增指定 set/piece 级恢复源命令	加载需要恢复的路径 ALTER SYSTEM ADD RESTORE SOURCE 'xxx';，详细信息参见 ADD RESTORE SOURCE。如果输入错误，可执行下述语句来撤销之前的输入 ALTER SYSTEM CLEAR RESTORE SOURCE;，详细信息参见 CLEAR RESTORE SOURCE。调用恢复命令 ALTER SYSTEM RESTORE <$restore_tenant> UNTIL '<$restore_checkpoint>' WITH 'xxx';，详细信息参见 RESTORE。
新增 auto_increment_cache_size 表选项	新增表选项语法，可以在 CREATE TABLE 或者 ALTER TABLE 时指定 auto_increment_cache_size。比如 CREATE TABLE t1 (...) auto_increment_cache_size = xxx; 和 ALTER TABLE t1 SET auto_increment_cache_size = xxx;，该值默认为 0 表示未配置，此时会使用租户级配置项作为自增列缓存大小。详细信息参见 CREATE TABLE 和 ALTER TABLE。
新增 table_mode 表选项	V4.3.2 重新启用了 V3.x 系列的表级配置项 table_mode，用户可以通过为每张表设置不同的表级配置项，以指定不同的快速冻结与自适应合并策略来应对 Buffer 表问题。如：CREATE TABLE t1(c1 INT) table_mode = 'normal/queuing/moderate/super/extreme';。详细信息参见 CREATE TABLE。
CTAS 支持并行执行	支持在 CREATE TABLE AS 语句中指定并发 Hint，例如 create /*+ parallel(N) */ table xxx as select，详细信息参见 CREATE TABLE。
创建触发器支持 IF NOT EXISTS 语法	兼容 MySQL 创建触发器 IF NOT EXISTS 语法，详细信息参见 触发器。
SQL 语句中支持使用 \N 代表 NULL	兼容 MySQL SQL 语句中指定 \N 代表 NULL 的用法，详细信息参见 NULL 值。
支持 DROP USER IF EXISTS 语法	兼容 MySQL DROP USER IF EXISTS 语法，详细信息参见 DROP USER。
MySQL 模式支持 CREATE TABLE ... [IGNORE | REPLACE] SELECT 命令	兼容 MySQL 建表指定 IGNORE | REPLACE 属性，详细信息参见 CREATE TABLE。
MySQL 模式 SELECT 语句支持 STRAIGHT_JOIN	部分兼容 MySQL STRAIGHT_JOIN 关键字，用来指示表关联顺序，详细信息参见 SELECT 语句。

周边配套

OceanBase 数据库 V4.3.2_CE 推荐使用的平台工具版本如下:

组件	版本
ODP	V4.2.3
OCP	V4.3.0_CE
OBD	V2.9.2
ODC	V4.3.0-bp1
OBCDC	V4.3.2
OMS	V4.2.4_CE
OBClient	V2.2.6
LibOBClient	V2.2.6

升级说明

• 支持 V4.3.1 Beta 在线升级到 V4.3.2 Beta。
• 暂不支持 V4.2.x 系列或更低版本升级到 V4.3.2 Beta，随着版本演进，后续会增加 V4.2.x 到 V4.3.x 的升级路径支持。
• V4.3.2 重构了多源数据的持久化格式，升级过程中需要对新旧多源数据格式进行转换，需要一定的磁盘空间和时间。因此建议在数据盘剩余容量大于 50% 时进行升级，并在分区数较多时预留相对充足的升级时间。

注意事项

• 全局前后台任务 CPU 隔离需要挂载 Cgroup 目录，且 enable_global_background_resource_isolation 打开并重启才可生效。
• 4.3.x 引入了一些 avx2 指令，在不支持该指令集 CPU 的机器上可能会 core 掉。
• 4.3.1 以上的版本，在 x86 和 arm 混布或跨 CPU 平台（支持 avx2 指令集和不支持 avx2 指令集的机器混布）场景下，产生跨机的数据（如物理迁移或恢复），可能出现报错和数据解码不正确。

Version information

Information	Description
Release date	July 12, 2024
Version number	V4.2.4_CE
Commit number	556a8f5
RPM number	oceanbase-ce-4.2.4.0-100000082024070810

Overview

OceanBase Database V4.2.4_CE is a new release designed for TP and HTAP workloads, featuring high compatibility, high performance, security, and ease of management. Building on V4.2.3_CE, the new version continues to enhance product compatibility and introduces several new MySQL features, including Event Scheduler, ASCII/TIS620 character sets, and defining column default values as expressions, thereby improving the convenience of MySQL migration. The optimizer capabilities have been enhanced to support adaptive correction after statistical information expires, optimize histogram sampling strategies, and improve row estimation accuracy to prevent statistical deviations. The new version also implements automatic routing between primary and standby databases, ensuring high service availability and seamless switching to the standby database in case of primary database failure. It improves PDML high concurrency performance, optimizes GTS overhead in DAS execution, and supports parallel synchronization of physical standby databases, thereby enhancing overall small-scale OLTP performance. Additionally, it supports REFERENCES, CREATE/DROP ROLE, and TRIGGER privilege management, strengthening enterprise-level security features of the database. In terms of operational ease, it expands the maintenance capabilities of partition balancing, optimizes deadlock detection and diagnosis mechanisms, restructures ASH report content display, and improves real-time diagnostic accuracy, thereby enhancing user operation convenience and efficiency. Furthermore, the generation method of SQL_ID for OBKV requests has been restructured, improving OBKV observability.

Key features

Feature enhancement

• Row estimation and statistical information enhancement

  The accuracy of the optimizer's cost estimation relies on accurate estimation of each operator, which in turn depends on the selection of estimation strategies and the accuracy of statistical information. In V4.2.2, the calculation method for base table row estimation and selectivity has been restructured, enhancing the accuracy of the optimizer's cost estimation. Additionally, V4.2.4_CE has been enhanced for certain complex scenarios, including:

  • Calculation of combined selectivity for multi-base-table predicates: The new version provides the cardinality_estimation_model system variable for the assumption model of predicate correlation, which determines how the optimizer estimates selectivity for predicates involving multiple base tables. The default value is PARTIAL, indicating that there is a certain level of correlation between predicates, and the system uses exponential backoff to calculate combined selectivity.

  • Adjustment in NDV calculation method: During the query plan generation process, the cardinality of the environment is maintained, and the joint NDV of multiple expressions is calculated based on this environment cardinality, thereby influencing the estimation of predicate selectivity and the number of rows in the Group By operator.

  • Adaptive correction after statistics information expiration: Every 15 minutes, the system updates the number of inserted, updated, and deleted rows in the internal table. It identifies scenarios where the statistical information expired due to significant changes in data volume (data volume changes by more than 10 times by default), and applies adaptive correction to the statistics. For example, every 15 minutes, an asynchronous background task is initiated to collect statistics for tables with significant statistics expiration, or dynamic sampling is used to prevent deviation in execution plans.

  • Dynamic sampling in more complex predicate scenarios: Dynamic sampling now can be used to calculate more accurate selectivity for common complex predicates on base tables, even in scenarios where statistical information is valid.

  In addition, the new version also optimized the histogram collection sampling strategy, decoupling the collection of histograms from the basic statistical information. It introduced new histogram collection parameters, hist_est_percent and hist_block_sample, to independently control the sampling ratio for histogram collection and determine whether block sampling is used. To optimize the histogram collection performance, the new version adaptively selects the sampling method for histogram collection based on the table size, using row sampling for small tables and block sampling for large tables.

Compatibility with MySQL

• Event Scheduler

  This version includes support for the Event Scheduler feature in MySQL mode, allowing for the scheduled execution of SQL or anonymous block statements. This is ideal for scenarios like batch processing at regular intervals and routine data maintenance. Integrating the Event Scheduler feature directly into the database reduces the need for external scheduling tools and simplifies the O&M.

• Select ... for update skip locked

  This version is now compatible with MySQL's select ... for update skip locked syntax. This means that if a lock conflict occurs, the query will not wait. Instead, it will skip rows locked by other transactions and proceed to execute the query, returning the results immediately.

• Authentication method Switch

  When a higher-version native MySQL client connects to OceanBase Database, it may use authentication methods that are not supported. To prevent connection errors, V4.2.4_CE now introduced support for Authentication method Switch. When a native client uses an unsupported authentication method to connect to OceanBase Database, the server will send an AuthSwitchRequest to the client, instructing it to restart the authentication using a method supported by the server.

• Show Create User

  The new version is compatible with the show create table syntax in MySQL, which is used to display database user information. Administrators or accounts with the SELECT privilege for the OceanBase and MySQL databases can view all user information under the tenant, while regular users can only view their own account information.

• Assigning system variables using subqueries

  In previous versions, the functionality to assign user variables using subqueries was already supported. The new version further expands this capability to assign system variables using subqueries, in order to align with MySQL behavior.

• Character set/collation expansion

  The new version includes support for the ASCII character set (with ascii_bin and ascii_general_ci collations) and the TIS620 character set (with tis620_bin and tis620_thai_ci collations). In addition, the utf8mb4 character set has been expanded to include collations such as utf8mb4_unicode_520_ci, utf8mb4_croatian_ci, utf8mb4_czech_ci, utf8mb4_0900_ai_ci, and now supports the utf8mb3 character set as an alias for utf8mb4.

• Using expressions as default values

  The new version now includes support for using independent expressions as default values for columns, such as default(DATE_FORMAT(sysdate(),'%Y%m%d')) and default(CURRENT_DATE)、default(UNIX_TIMESTAMP()).

• CHECK TABLE statement

  This version provides partial support for the syntax of the CHECK TABLE statement. This allows for verifying the existence of tables or views within the database, as well as checking the validity of referenced objects within views. However, some other features are not yet supported.

OBKV enhancements

• OBKV diagnostic enhancement

  In previous versions, OBKV's SQL_ID was generated based on the requests. That is, a unique SQL_ID will be generated for every request, making it difficult to conduct categorized diagnostics. In V4.2.4_CE, the method for generating SQL_ID of OBKV requests has been redesigned. Requests are now divided into multiple single operations, and by referencing the method of SQL parameterization, each single operation in OBKV is parameterized to generate a new SQL_ID. Based on this new method of generating SQL_ID, peripheral tools can also categorize OBKV requests, identify high-frequency operations and excessively long duration operations, thereby improving the usability of OBKV diagnostics.

Performance improvements

• Performance optimization for environments with small specifications (4C/8C)

  The system performance in clusters with 4C or 8C specifications has been improved across various scenarios, such as enhanced write performance for multi-index scenarios, reduced overhead in the rowkey_exist check during updates, minimized overhead in MemTable hash indexes, and improved plan cache hash key. As a result, the performance of OLTP-related cases has increased by approximately 20% compared with V4.2.3_CE.

• Insertup/Replace performance optimization

  In V4.x, the Insertup (insert ... values ... on duplicate) and Replace (replace into ... values ...) statements do not carry out partition pruning on a row-by-row basis. Therefore, SQL statements involving multiple tablet writes will uniformly adopt a DISTRIBUTED execution plan, obtaining a Global Transaction Timestamp (GTS) once before the execution of the statements. In scenarios involving small data volume inserts, especially when there are numerous primary key/unique key conflicts, obtaining the GTS brings about significant performance degradation. The new version refines the write scenarios, omitting the GTS retrieval operation in scenarios where the write table does not involve unique global indexes or involves unique global indexes but the written data resides in the same log stream, optimizing the execution flow, and significantly enhancing the performance of Insertup/Replace.

• PX NLJ/SPF Random Shuffle

  During PX execution, the Join operator and Subplan Filter operator can have multiple data distribution methods as alternative optimizer plan forms. However, under the current NONE_ALL distribution, there are issues with NLJ/SPF not fully utilizing parallelism, especially in cases with severe data skew, resulting in suboptimal performance. Therefore, V4.2.4_CE has added support for inserting the Random Shuffle Exchange operator at the boundary between computation and storage in NLJ/SPF, dispersing the workload of a single thread across multiple threads, and decoupling computation and data retrieval. This version provides two new data distribution methods during execution: RANDOM_ALL and HASH_ALL. These are the competitors for the NONE_ALL plan in the selection of NLJ/SPF plans. It is expected that when the PX GI operator on the left side of NLJ/SPF has fewer tasks to utilize the computation threads of NLJ/SPF, the RANDOM_ALL and HASH_ALL plans will be chosen to enhance execution performance.

• PDML transaction optimization

  The new version optimizes PDML performance in high-concurrency scenarios by supporting parallel commit and parallel log replay at the transaction layer, resulting in a significant improvement compared with earlier versions of V4.x.

• Parallel synchronization for physical standby database

  Since V4.2.0, OceanBase Database has supported a physical standby database based on direct network connectivity, enabling the transmission of logs between the primary and standby databases over the network. Following the performance optimization of the network standby database log transmission in V4.2.3, the log synchronization performance has greatly improved. However, in scenarios where the primary and standby databases are deployed in different locations, the speed of log transmission is limited by the model, and the performance improvement is constrained. In high-pressure write scenarios, the synchronization performance may not meet customer requirements. Therefore, in V4.2.4, a new synchronization model has been designed and implemented to support the parallel synchronization of logs from the primary database to the network standby database, enhancing the log synchronization capability and improving synchronization performance.

Resource usage optimization

• Memory throttling mechanism

  Prior to V4.x, there were not many modules that relied on freezes and minor compactions to release memory, with MemTable being the largest part. Therefore, in previous versions, an upper limit for memory usage was set for MemTable's memory usage, and throttling logic was used to ensure smooth operation as memory usage approached the limit, thereby avoiding sudden memory depletion that could halt the system's write operations. With the introduction of more modules that rely on freezes and minor compactions to release memory (such as the transaction data module) in V4.x, the new version provides a more detailed approach to controlling the memory usage of multiple modules. This includes the addition of memory limits control for the TxData and MDS modules, sharing the memory space with MemTable. When the accumulated memory reaches tenant memory * _tx_share_memory_limit_percentage% * writing_throttling_trigger_percentage%, overall throttling is triggered. Additionally, the new version also introduced the functionality of triggering freeze and dump of transaction data tables based on a time dimension, with a default 1800 seconds trigger interval for freezing transaction data tables, reducing the memory usage of the transaction data module.

• Memory optimization for SQL AUDIT

  SQL AUDIT records the SQL information executed by the current tenant at the machine level. A regular tenant constructs a queue of 10 million record pointers and pre-allocates memory space at machine startup, known as static memory. The actually recorded information uses dynamic memory. The new version has optimized the memory structure of SQL AUDIT, reducing the initial static memory to better adapt to smaller tenant specifications. Additionally, it supports dynamically adjusting the queue memory size based on the actual number of records for the tenant, allowing for further expansion of the number of records that can be stored for larger tenant specifications.

Reliability improvements

• Primary-standby database automatic routing

  In the V2.x and V3.x series, OceanBase Database supports cluster-level primary and standby databases, using CLUSTER_NAME as the unique identifier for a group of primary and standby clusters. You can specify the cluster name when connecting to OceanBase Database using ODP to automatically route to the primary cluster. In the V4.x series, OceanBase Database supports tenant-level primary and standby databases. The primary tenant does not record the information of the standby tenant, and the standby tenant does not record the information of the primary tenant. The primary and standby relationships are maintained by external tools, such as OceanBase Cloud Platform (OCP). In V4.x, the SERVICE_NAME concept is introduced to manage groups of primary and standby tenants. You can specify SERVICE_NAME when logging on to OceanBase Database using ODP, such as obclient -h $ip -P $port -u$user_name@SERVICE:$service_name. ODP can then route the connection to the primary tenant based on SERVICE_NAME to meet the automatic routing requirement of V4.x primary and standby databases. In addition, the system tenant provides a set of SERVICE_NAME management commands for the creation, deletion, start, and stop of SERVICE_NAME for tenants.

Security enhancements

• MySQL REFERENCES/CREATE ROLE/DROP ROLE/TRIGGER privilege

  This new version is compatible with MySQL's References, Create Role, Drop Role, and Trigger privileges. The References privilege can be specified at the Global, Database, Table, or Column level, and is required when creating or modifying tables to establish foreign keys. Additionally, the privilege can be set at the column level, offering the option to make it ineffective, aligning with MySQL compatibility. Create Role and Drop Role privileges operate at the Global level. To create a role, the Create User or Create Role privilege is required. To drop a role, the Create User or Drop Role privilege is required. The Triggers privilege can be specified at the Global, Database, or Table level, and is necessary for creating, deleting, executing, or displaying table triggers.

  To ensure that the upgrade of earlier versions does not affect online business, privilege checks are not enabled by default in upgrade scenarios. If it is necessary to use these new privileges, the security feature version can be set manually through the ob_security_version variable. This version number can only be increased and cannot be rolled back. It is enabled by default when creating a new cluster.

Usability improvements

• Scheduled/manual partition balancing

  OceanBase Database supports partition balancing based on Transfer since V4.2.0_CE. The scheduling period for balancing tasks is controlled through the tenant-level parameter partition_balance_schedule_interval. By default, the balance task is scheduled every 2 hours from the start of OBServer. The usability of this parameter is poor, as it cannot effectively control the execution time of the partition balance task. Because the Transfer action may block the read and write operations of the corresponding partition, performing partition balancing during peak hours will affect the response time. Additionally, if there are a large number of partitions that need balancing, it may temporarily increase the cluster load, affecting the overall cluster performance. In order to optimize this scenario, V4.2.4_CE introduced the DBMS_BALANCE system package, which provides users with a method to manually trigger partition balancing, DBMS_BALANCE.TRIGGER_PARTITION_BALANCE(balance_timeout). Additionally, a scheduled partition balancing task SCHEDULED_TRIGGER_PARTITION_BALANCE is built into user tenants by default, triggering partition balancing once daily at 00:00. Users can also modify the scheduling time, frequency, and other information using the DBMS_SCHEDULER subprogram based on the characteristics of their business, in order to manage partition balancing tasks in a more flexible and reliable manner.

• Deadlock detection optimization

  Prior to V4.2.4_CE, OceanBase Database provided the CDB/DBA_OB_DEADLOCK_EVENT_HISTORY view for querying deadlock events and information about the nodes involved in the deadlock event in the database. However, it lacked records of the SQL information corresponding to the holder on the deadlock loop, affecting user diagnostics. The new version enriches the deadlock detection view information by also including the session ID associated with the visitor, the machine where the resource is being requested, the log stream, tablet address, and other relevant details. Additionally, it records the SQL currently being executed, the SQL holding the lock, and the request time following a deadlock, facilitating deadlock event diagnostics.

• ASH report enhancement

  OceanBase Database is a multi-tenant distributed database. Currently, the ASH reports obtained from the system tenant contain aggregated information for multiple tenants and multiple nodes, but often, performance issues are specific to a certain tenant or node. Therefore, starting from V4.2.3, OceanBase Database provides the capability to analyze ASH reports at the node level, and V4.2.4 expands this to the tenant level, supporting the generation of ASH reports for a specific tenant ID using DBMS_WORKLOAD_REPOSITORY.ASH_REPORT.

  The new version systematically optimizes the content of the ASH report. Building on Oracle's ASH framework, the report has been adjusted to be more suitable for performance diagnostics in OceanBase. For example, it includes new analysis results for Top Active Tenants, Top Node Load, and various categories of front-end and back-end loads. Additionally, to enhance readability, the new version also supports the output of ASH reports in HTML format.

• Real-time diagnostic capability enhancement

  OceanBase Database provides comprehensive SQL performance diagnostic tools, but these are primarily based on post-execution diagnostics. In some AP scenarios, SQL executions often take a long time, resulting in poor timeliness and usability for analyzing the SQL after execution. At such times, there is a need for a real-time method to obtain information about SQL execution. In V4.2.4_CE, based on SQL PLAN MONITOR, real-time information on the number of rows and bytes scanned by the table scan operator has been added. ODC has also provided page-structured real-time diagnostic functionality on this basis, making it easier to view SQL execution plans and progress in real time.

• Resource specification estimation

  We categorize the resources in the database into two main types: logical resources and physical resources. Logical resources refer to entities corresponding to logical concepts, such as data structures, threads, locks, and sessions. Physical resources refer to hardware resources, including CPU, disk, and memory. The amount of logical resources a tenant can create may be limited by one or more physical resources. To allow users to conveniently obtain information on the current usage of logical resources corresponding to physical resources in the cluster, for more reliable planning of scaling, node replacement, and standby tenant creation operations, V4.2.4 introduces the resource specification estimation feature, providing the following dynamic views and system packages:

  • Introduces the [G]V$OB_TENANT_RESOURCE_LIMIT view, which showcases the usage, limits, effective constraints, and maximum usage after a crash and restart for each tenant on every unit.
  • Introduces the [G]V$OB_TENANT_RESOURCE_LIMIT_DETAIL view to display the limitations on a tenant's ability to create logical resources on a machine based on physical resources or parameters.
  • Introduces the DBMS_OB_LIMIT_CALCULATOR.CALCULATE_MIN_PHY_RES_NEEDED_BY_UNIT subprogram to calculate the minimum physical resources needed by a tenant on a specific node.
  • Introduces the DBMS_OB_LIMIT_CALCULATOR.CALCULATE_MIN_PHY_RES_NEEDED_BY_LOGIC_RES subprogram to calculate the physical resources required for a specific type and quantity of logical resources.
  • Introduces the DBMS_OB_LIMIT_CALCULATOR.CALCULATE_MIN_PHY_RES_NEEDED_BY_STANDBY_TENANT subprogram to calculate the minimum physical resources needed for creating a specified number of standby tenants by a specific main tenant on a designated unit.

• Backup snapshot table name

  When OCP and downstream vendors adapt table-level recovery functionality, they need to show users the tables available for recovery. The new version persistently stores the corresponding snapshot table name in the backup set and provides the capability to parse the snapshot table name through ob_admin.

• Canceling manual transfer of partitions

  The "Cancel Transfer Partition" feature is an enhancement to the "Manual Transfer Partition" functionality. After initiating a manual Transfer Partition task, the task can now be canceled before it is executed. This feature provides the following capabilities: the ability to cancel a task that has not yet started the transfer, to cancel all tasks under a tenant that have not started the transfer, and to cancel the currently executing Balance Job.

Compatibility changes

Product behavioral changes

The following table describes the changes made in this version.

Feature	Description
Changes in behavior of scheduled partition balancing task	In earlier versions, the scheduling period for the partition balancing task was controlled by the tenant-level parameterpartition_balance_schedule_interval. Starting from V4.2.4_CE, this behavior has changed to trigger the partition balancing task at a specific time using the SCHEDULED_TRIGGER_PARTITION_BALANCE task, with the default time set to 00:00 daily. Specifically: Starting from V4.2.4_CE, when you create a tenant, the SCHEDULED_TRIGGER_PARTITION_BALANCE task is enabled by default, and the value of partition_balance_schedule_interval is set to 0, indicating that the triggering of partition balancing relies entirely on scheduled tasks and manual triggers. When upgrading from V4.x prior to V4.2.4_CE to V4.2.4_CE, the default behavior of the earlier version controls the SCHEDULED_TRIGGER_PARTITION_BALANCE task is disabled. If the user desires to use the new behavior after the upgrade, they need to manually adjust the above parameter and the scheduled task. NoticeWhen setting partition_balance_schedule_interval, make sure that SCHEDULED_TRIGGER_PARTITION_BALANCE is disabled. Otherwise, an error will occur.
Manual collection of optimizer statistics requires a pre-commit transaction	There are differences between OceanBase, native Oracle, and native MySQL regarding the behavior of collecting statistics within an uncommitted transaction. With OceanBase, when collecting statistics within an uncommitted transaction using the aforementioned method, the pre-commit transaction is not submitted, whereas with native Oracle and native MySQL, a submission is performed. In the new version, this behavior is modified as follows: In a new cluster, when manually collecting statistics, the pre-commit transaction will be submitted.For clusters upgraded from an earlier version, the default behavior is to maintain the previous behavior, where manual collection of statistics does not submit a pre-commit transaction. If users wish to utilize the new behavior in the upgraded old cluster, they can do so by modifying the ob_compatibility_version system variable for control purposes.

View changes

The following table describes the changes made in this version.

View	Change type	Description
information_schema.events	New	Stores event information. This view is added in the MySQL mode.
information_schema.tables	Modified	The data_length field is adjusted to display the storage space occupied by the table, calculated in macro-block size, and measured in bytes.The index_length field is adjusted to display the storage space occupied by non-primary key indexes, calculated in macro-block size, and measured in bytes.
information_schema.partitions	Modified	The data_length field is adjusted to display the storage space occupied by table partitions, calculated in macro-block size, and measured in bytes.The index_length field is adjusted to display the storage space occupied by indexes of table partitions, calculated in macro-block size, and measured in bytes.
information_schema.role_column_grants	New	Displays the table-level privilege information for the roles that are granted and activated for the user in the current session, as well as the table-level privilege information for the roles granted to these roles. This view is added in the MySQL mode.
information_schema.role_routine_grants	New	Displays the routine-level privilege information for the roles that are granted and activated for the user in the current session, as well as the routine-level privilege information for the roles granted to these roles. This view is added in the MySQL mode.
information_schema.role_table_grants	New	Displays the column-level privilege information for the roles that are granted and activated for the user in the current session, as well as the column-level privilege information for the roles granted to these roles. This view is added in the MySQL mode.
CDB/DBA_OB_DEADLOCK_EVENT_HISTORY	Modified	MODULE: The value is fixed as transaction.VISITOR: The format is {session_id:$1}:{txid:$2}, where $1 and $2 are both numbers. (In earlier versions, it was only {txid:xx}.)OBJECT: The primary key description of the row, in the format of {addr:"$1:$2"}:{ls:$3}:{tablet:$4}:{row_key:{$5}}, where $1 is the IP address, $2 is the port (together forming the machine process address where the conflict occurred), $3 is the log stream where the conflict occurred, $4 is the tablet where the conflict occurred, and $5 is the content of the row key after stringification. (In earlier versions, it was {row_key:xxx}.)EXTRA_NAME1: Fixed as wait_sql.（In earlier versions, it was current sql.）EXTRA_VALUE1: The current SQL content of the transaction at the time of deadlock occurrence.EXTRA_NAME2: Fixed as hold_sql_request_time. (In earlier versions, it was empty.)EXTRA_VALUE2: The request time of the locking statement. (In earlier versions, it was empty.)EXTRA_NAME3: Fixed as hold_sql. (In earlier versions, it was empty.)EXTRA_VALUE3: The SQL statement of the object corresponding to the record of the line holding cycle_idx - 1. (In earlier versions, it was empty.)
CDB/DBA_BALANCE_JOBS	Modified	Added the max_end_time column to display the latest completion time of the load balancing task.
CDB/DBA_BALANCE_JOB_HISTORY	Modified	Added the max_end_time column to display the latest completion time of the load balancing task.
CDB/DBA_BALANCE_TASKS	Modified	Added the balance_strategy column to display the balancing strategy corresponding to the balance task.
CDB/DBA_BALANCE_TASK_HISTORY	Modified	Added the balance_strategy column to display the balancing strategy corresponding to the balance task.
[G]V$OB_ACTIVE_SESSION_HISTORY	Modified	Added four columns plan_hash, thread_id, stmt_type, time_model to respectively display the hash value of the execution plan corresponding to the currently executing SQL, the thread ID in which the current active session is located, the SQL type corresponding to the current active session, and the collection of sampled execution stages. Changed the column type of sample_time to TIMESTAMP.
[G]V$OB_TENANT_RESOURCE_LIMIT	New	Shows the resource status of each tenant on the OBServer node, including log streams, tablets, etc. The system tenant can view information for all tenants, while regular tenants can only view information for their own tenant.
[G]V$OB_TENANT_RESOURCE_LIMIT_DETAIL	New	Displays the detailed restrictions of various resources for each tenant on the OBServer node, such as the number of log streams limited by parameters, tenant memory size, clog disk size, etc. The system tenant can view information for all tenants, while regular tenants can only view information for their own tenant.
CDB/DBA_OB_SERVICES	New	Displays all tenants' or the current tenant's SERVICE_NAME-related information. The CDB view is only supported under the system tenant.
[G]V$OB_PROCESSLIST	Modified	Added the SERVICE_NAME column to display which SERVICE_NAME created the session. If empty, the session is not created by a SERVICE_NAME.
[G]V$SQL_PLAN_MONITOR	Modified	Changed the meaning of fields for the table scan operator. specifically: In earlier versions, otherstat_1_value, otherstat_2_value, and otherstat_3_value represent IO_READ_BYTES, TOTAL_READ_BYTES, and TOTAL_READ_ROW_COUNT, and are calculated at the end of the operator's computation.In the new version, otherstat_1_value, otherstat_2_value, otherstat_3_value, and otherstat_4_value represent IO_READ_BYTES, SSSTORE_READ_BYTES, SSSTORE_READ_ROW_COUNT, MEMSTORE_READ_ROW_COUNT, and support real-time statistics.

Parameter changes

Parameter	Change type	Description
plsql_v2_compatibility	Modified	In V4.2.3_CE, the functionality to persistently compile stored procedure DDL during the compilation phase was implemented. If the stored procedure code is large, the compilation time may be too long, delaying the execution of DDL. In V4.2.4_CE, the plsql_v2_compatibility parameter is used to control whether the functionality of persistently compiling stored procedure DDL during the compilation phase is enabled. The default is False, indicating it is not enabled. This parameter was not actually used in previous versions and does not affect other functionalities.

System variable changes

System variable	Change type	Description
event_scheduler	New	Controls whether the Event Scheduler feature is enabled. When enabled, events can be created, modified, and deleted. The default value is OFF, indicating it is not enabled. It is a global-level system variable added to the MySQL mode.
cardinality_estimation_model	New	Controls the relevance model used by the optimizer when estimating rows. It can be set to INDEPENDENT (assuming complete independence between predicates), PARTIAL (assuming a certain degree of correlation between predicates), or FULL (assuming complete correlation between predicates). The default value is PARTIAL. It is a global/session-level system variable.
enable_sql_plan_monitor	New	Controls whether the SQL statements in the current session are recorded in the SQL plan monitor. The default value is False, indicating that they are not recorded by default. When enabled, all SQL statements in the current session are recorded in the SQL plan monitor without considering other conditions. When disabled, a SQL statement in the current session will only be recorded in the SQL plan monitor if it meets one of the following conditions: the HINT is enabled, the execution duration exceeds the threshold, or the degree of parallelism is greater than 1. It is a session-level system variable.

System package changes

System package	Change type	Description
DBMS_BALANCE	New	Manages load balancing tasks. This system package is added under the user tenant.
DBMS_STATS.GATHER_TABLE_STATS	Modified	Added two parameters hist_est_percent and hist_block_sample for histogram collection. These two parameters divides the basic statistical information and histogram control strategies. The behavior after the division is as follows:estimate_percent: Controls whether basic statistical information collection is sampled. The default behavior is not to sample, and to collect statistics information with a full table scan.block_sample: Controls whether block sampling is used when collecting basic statistical information samples. The default is not to use block sampling, and to use row sampling instead.hist_est_percent: Controls the sampling ratio for histogram collection. The default behavior is for the optimizer to decide the sampling ratio based on the table size.hist_block_sample: Controls whether block sampling is used for histogram collection. The default behavior is for the optimizer to decide whether to use block sampling based on the table size. Two new Perfs, hist_est_percent and hist_block_sample have also been added.
DBMS_OB_LIMIT_CALCULATOR	New	Calculates the minimum physical resources required by the system in different scenarios.

Syntax changes

Syntax	Description
SERVICE_NAME command	Added a management command for SERVICE_NAME: ALTER SYSTEM {CREATE | DELETE | START | STOP} SERVICE $service_name [tenant = '$tenant_name'];.
Show Create User syntax	Added the show create table syntax to display database user information. This syntax is compatible with MySQL's show create table syntax.
Event-related syntax	Added commands CREATE/ALTER/DROP EVENT.
Cancel Transfer Partition-related syntax	Added the Cancel Transfer Partition syntax for canceling transfer tasks.
SET System_Variable now supports subqueries	Earlier versions already supported assigning user variables using subqueries. The new version expands the usage of subquery assignment for system variables to be compatible with MySQL behavior.
Table-level recovery command adjustment	The TENANT keyword in the syntax ALTER SYSTEM RECOVER TABLE tablename TO TENANT tenantname now is optional. This means that both ALTER SYSTEM RECOVER TABLE tablename TO TENANT tenantname and ALTER SYSTEM RECOVER TABLE tablename TO tenantname syntaxes are supported.

Recommended versions of tools

The following table lists the recommended versions of tools for OceanBase Database V4.2.4_CE.

Tool	Version
ODP	V4.2.3
OCP	V4.3.0_CE
obd	V2.9.2
All in One	V4.2.4
ODC	V4.3.0 BP1
obcdc	V4.2.4
OMS	V4.2.4_CE
obclient	V2.2.6
libobclient	V2.2.6

Upgrade notes

• OceanBase Database V4.2.1_CE_BP2 and earlier V4.x versions can be upgraded to OceanBase Database V4.2.4_CE through a valid upgrade path. V4.2.1_CE BP3 and subsequent BP versions can no longer be upgraded to V4.2.4_CE.
• OceanBase Database V4.2.3_CE can be upgraded to V4.2.4_CE.
• In cases where there are a large number of cluster tablets, the OBServer restart during the upgrade process may take longer. When the number of tablets reaches 3 million, the expected restart duration is estimated to be 20 minutes.
• When upgrading from V4.2.1_CE and V4.2.2_CE series to V4.2.4_CE, if the _nlj_batching_enabled parameter is enabled, please disable it before the upgrade. You can enable it again after the upgrade.
• Upgrade sequence for ODP and OBServer: It is recommended to upgrade OBServer first, and then upgrade ODP.
• During the upgrade, the system will automatically disable merging and DDL operations, and normal operation will be restored after the upgrade is completed.

Considerations

• To use the automatic routing feature for primary and standby databases, use ODP V4.3.1 and OCP V4.3.1 or their later versions. When ODP is started using the rs_list method, the service_name feature is not supported.
• Parallel synchronization of physical standby databases enhances the capacity limit of the standby database, but incurs I/O resource expenses for the primary tenant. When creating a network standby database, the log gap between the primary and standby tenants should be minimized, reducing the standby database's requests for a large amount of I/O resources from the primary database in the short term. When the primary tenant already has a lot of data, it is advisable to first back up the primary tenant, then create the standby tenant using backup restoration to reduce the log gap between the primary and new standby databases, and then change the log recovery source of the standby tenant to network recovery.

版本信息

项目	描述
发布日期	2024-7-12
版本号	V4.2.4_CE
Commit 号	556a8f5
OBServer RPM 版本号	oceanbase-ce-4.2.4.0-100000082024070810

版本概览

OceanBase 数据库 V4.2.4_CE 版本是面向 TP 和 HTAP 业务的高度兼容、高性能、安全且易于管理的新版本。在 V4.2.3_CE 版本基础上，新版本继续完善产品兼容性，新增多项 MySQL 特性支持，包括 Event Scheduler、ASCII/TIS620 字符集、列默认值定义为表达式等，提升 MySQL 迁移的便利性。增强优化器能力，支持统计信息过期后的自适应修正，优化直方图采样策略，提升估行准确性以防统计信息走偏。新版本也实现了主备库之间的自动路由功能，确保服务高可用性，即使在主库故障情况下也能无缝切换至备库继续提供服务。提升 PDML 高并发性能，优化 DAS 执行 GTS 开销，支持物理备库的并行同步，整体提升了小规格 OLTP 性能。另外支持 REFERENCES、CREATE/DROP ROLE、TRIGGER 权限管理，强化数据库企业级安全特性。运维易用性方面，扩展分区均衡的运维能力，优化死锁检测诊断机制，重构 ASH 报告内容展示，提升实时诊断准确性，提升了用户操作的便捷性和效率。同时重构了 OBKV 请求的 SQL_ID 生成方式，提升 OBKV 可观测性。

关键特性说明

特性增强

• 估行与统计信息增强

  优化器代价估计依赖对每个算子的准确估行，准确估行则依赖估计策略选择和统计信息的准确性。V4.2.2 版本已经重构了基表估行与选择率计算的方式，提高了优化器代价估计的准确性。针对一些复杂场景，V4.2.4_CE 版本也进行了补充完善，如：

  • 多基表谓词的联合选择率计算： 针对谓词间相关性假设模型提供 cardinality_estimation_model 系统变量，控制优化器使用哪种基表估行方案。默认为 PARTIAL，表示谓词间具有一定程度的相关性，通过指数回退计算联合选择率。
  • NDV 计算方式调整： 计划生成过程中维护环境基数，通过环境基数计算多表达式的联合 NDV，进而影响谓词选择率、GroupBy 算子行数的估算。
  • 统计信息过期自适应： 每 15 分钟将增删改行数刷新到内部表，标记统计信息过期严重的场景（默认数据量变化超过 10 倍），自适应地进行统计信息修正。如每 15 分钟对统计信息过期严重的表发起一次后台任务异步收集，或使用动态采样避免执行计划走偏。
  • 动态采样支持更多复杂谓词场景： 针对基表常见的复杂谓词，即使在统计信息有效的场景，也支持使用动态采样的方式去计算更准确的复杂谓词选择率。

  同时，新版本也优化了直方图收集采样策略，将直方图和基础统计信息收集解耦，新增统计信息收集参数 hist_est_percent 和 hist_block_sample，用于独立控制直方图收集的采样比例和是否使用块采样。为了优化直方图收集性能，新版本会根据表大小自适应选择直方图收集的采样方式，小表使用行采样，大表使用块采样。更多有关估行与统计信息增强的内容，参见统计信息和估行机制的使用。

MySQL 兼容

• Event Scheduler

  MySQL 模式下新增 Event Scheduler（事件调度器） 功能，用于定时执行 SQL 或匿名块语句，适用于定时任务执行、周期性数据维护等场景。将事件调度器集成到数据库中，减少了对外部调度工具的依赖，简化了运维操作。更多内容参见 CREATE EVENT、ALTER EVENT 和 DROP EVENT。

• Select ... for update skip locked

  新版本兼容 MySQL select ... for update skip locked 语法，表示存在锁冲突时不等待，跳过被其他事务锁定的行，立即执行查询返回结果。

• Authentication method Switch

  高版本原生 MySQL 客户端连接 OceanBase 数据库时，可能使用到未支持的 MySQL 认证方法。为了避免建连报错，V4.2.3_CE 版本支持了 Authentication method Switch，当原生客户端使用 OceanBase 数据库时不支持的认证方式创建连接时，服务端会发送 AuthSwitchRequest 给客户端，告知客户端使用服务端支持的认证方式重新开始认证。

• Show Create User

  兼容 MySQL show create table 语法，用于展示数据库用户信息。管理员或具有 OceanBase、MySQL 库 SELECT 权限的账户可以查看租户下所有用户信息，普通用户仅可查看本账户信息。

• 使用子查询为系统变量赋值

  早期版本已支持使用子查询为用户变量赋值，新版本扩展子查询赋值系统变量的用法，以兼容 MySQL 行为。

• 字符集/字符序扩展

  新增支持 ASCII 字符集（ascii_bin、ascii_general_ci 字符序），TIS620 字符集（tis620_bin、tis620_thai_ci 字符序）。另外 utf8mb4 字符集扩展了 utf8mb4_unicode_520_ci、utf8mb4_croatian_ci、 utf8mb4_czech_ci、utf8mb4_0900_ai_ci 等字符序，并支持了 utf8mb3 字符集（作为 utf8mb4 别名）。更多内容参见字符序。

• Default 值支持表达式

  新增支持不依赖其他列的表达式作为列的 Default 值，如：default(DATE_FORMAT(sysdate(),'%Y%m%d'))、default(CURRENT_DATE)、default(UNIX_TIMESTAMP()) 等。

• Check Table Statement

  部分兼容 Check Table Statement 语法，支持检查数据库中的表或视图是否存在，或检查视图引用对象是否有效。其他功能暂未支持。

OBKV 增强

• OBKV 诊断增强

  老版本中 OBKV 的 SQL_ID 根据 Request 生成，每个 Request 生成一个 SQL_ID，不便进行分类诊断。V4.2.3_CE 版本重构 OBKV 请求的 SQL_ID 生成方式，将请求拆分为多个单操作，并参考 SQL 参数化的方式，将 OBKV 单操作进行参数化后生成新的 SQL_ID。基于新的 SQL_ID 生成方式，外围工具也可对 OBKV 请求进行分类，识别高频操作和超长耗时操作等，提高 OBKV 诊断易用性。

性能提升

• 小规格（4C、8C）性能优化

  针对 4C、8C 规格集群，通过多场景优化提升系统性能，如多索引场景下的写入优化、Update 路径下 rowkey_exist 检查开销优化、Memtable hash 索引开销优化、Plan cache hash key 优化等，OLTP 相关 Case 性能相对 V4.2.3_CE 版本提升约 20%。

• Insertup/Replace 性能优化

  Insertup（指 insert ... values ... on duplicate） 和 Replace（指 replace into ... values ...）语句在 V4.x 版本中不会逐行做分区裁剪，因此涉及多 Tablet 写入的 SQL 会统一选用 DISTRIBUTED 执行计划，在语句执行前获取一次 GTS。小数据量入库场景下，尤其是存在较多主键/唯一键冲突时，GTS 获取操作会带来明显的性能损耗。新版本细化了写入场景，在写入表不涉及 unique global 索引或涉及 unique global 索引但写入数据位于同一日志流等场景，省略了 GTS 获取操作，优化了执行流程，显著提升 Insertup/Replace 性能。

• PX NLJ/SPF Random Shuffle

  在 PX 执行过程中，Join 算子以及 Subplan Filter 算子可以有多种数据分布方式作为优化器计划形态的备选，但在当前 NONE_ALL 的分布下存在 NLJ/SPF 不能充分利用并行的问题，尤其在数据偏移严重的情况下，性能不优。为此 V4.2.4_CE 版本支持了在 NLJ/SPF 的计算、存储分界插入 Random Shuffle 的 Exchange 算子，将一个线程的工作量打散到多个线程，解耦计算、取数。新增两种执行中的数据分布方式: RANDOM_ALL 和 HASH_ALL，在 NLJ/SPF 的计划选择中分别作为 NONE_ALL 计划的竞争者，预期当 NLJ/SPF 左支的 PX GI 算子划分任务较少不足以利用NLJ/SPF 的计算线程时，选择 RANDOM_ALL 和 HASH_ALL 的计划，来提升执行性能。

• PDML 事务优化

  新版本在事务层通过支持并行提交、并行回放日志等实现优化，相对 V4.x 早期版本，明显提升高并发场景下 PDML 执行性能。

• 物理备库并行同步

  Oceanbase 数据库从 V4.2.0 版本开始支持了基于网络直通的物理备库，主库与备库之间通过网络来传输日志，V4.2.3 版本支持了网络备库的日志传输性能优化后，日志同步性能相较优化前提升了很多，但是在主备库异地部署的场景下，受限于日志传输的模型，日志传输的速度提升有限。在写入压力大的场景，同步性能可能无法满足客户需求。因此在 V4.2.4_CE 版本又设计和实现了新的同步模型，支持网络备库从主库并行同步日志，增强网络备库的日志同步能力，提升同步性能。更多内容参见优化日志同步性能。

资源优化

• 内存限速机制

  在 V4.x 版本之前，需要依赖冻结转储释放内存的模块并不多，Memtable 是其中最大的一部分。因此，之前版本对 Memtable 本身设定了内存使用上限，并通过限速逻辑使其在内存使用量接近上限的过程中运行尽可能平滑，避免突然耗尽内存导致系统停写。V4.x 版本之后，随着更多依赖冻结转储释放内存的模块（如事务数据模块）被引入，新版本提供了更细致的手段来控制多个模块的内存使用，新增 TxData 和 MDS 模块内存上限控制，和 Memtable 共享内存空间，当累积内存达到 租户内存 * _tx_share_memory_limit_percentage% * writing_throttling_trigger_percentage% 时，触发整体限速。同时，新版本也增加了时间维度触发事务数据表冻结转储的功能，默认 1800 秒触发一次事务数据表的冻结，降低事务数据模块的内存占用。

• SQL AUDIT 内存优化

  SQL AUDIT 是以机器粒度去记录当前租户执行过的 SQL 信息，普通租户构造了 1000 万条记录指针队列，在机器启动时预分配内存空间，我们称之为静态内存。而实际记录的信息会使用动态内存。新版本优化了 SQL AUDIT 内存结构，降低起始静态内存，更好地适应小规格租户。同时，支持根据租户的实际记录数，动态调整队列内存大小，在高规格租户允许进一步拓展可存储记录数。

可靠性提升

• 主备库自动路由

  在 V2.x、V3.x 系列中，OceanBase 数据库支持集群级主备库，使用 CLUSTER_NAME 作为唯一表示一组主备集群的标识，用户通过 ODP 连接时，指定集群名即可自动路由到主集群。V4.x 版本变更为租户级主备库，主租户不记录备租户信息，备租户也不记录主租户信息，主备关系通过外部工具如 OCP 来维护，没办法通过 ODP 连接自动路由到主租户。新版本引入 SERVICE_NAME 概念，用于管理一组主备租户，通过 ODP 指定 SERVICE_NAME 登录，如 obclient -h $ip -P $port -u$user_name@SERVICE:$service_name，ODP 可根据 SERVICE_NAME 将连接路由至主租户，以满足 V4.x 主备库自动路由的需求。同时，系统租户下也提供了一套 SERVICE_NAME 管理命令，用于为租户创建、删除、启动、停止 SERVICE_NAME。

安全能力增强

• MySQL REFERENCES/CREATE ROLE/DROP ROLE/TRIGGER 权限

  兼容 MySQL References，Create Role，Drop Role，Trigger 权限。 References 权限可以指定 Global、Database、Table、Column 级别，建表或修改表创建 Foreign Key 时需要具备该权限，Column 级别可设置不生效，与 MySQL 兼容。Create Role 和 Drop Role 权限为 Global 级别，创建角色需要具备 Create User 或 Create Role 权限，删除角色需要具备 Create User 或 Drop Role 权限。Triggers 权限可指定 Global、Database、Table 级别，为表创建、删除、执行或显示该表的触发器时，需具备 Trigger 权限。

  为了保证旧版本升级后不影响在线业务，升级场景默认不开启以上权限校验，确认需要使用新版本权限功能时，可通过 ob_security_version 变量手动设置安全功能版本，该版本号只能推高，不能回退。新建集群时默认开启。更多内容参见MySQL 模式下的权限分类。

易用性提升

• 定时/手动分区均衡

  OceanBase 数据库 V4.2.0_CE 版本开始支持基于 Transfer 的分区均衡功能，通过租户级配置项 partition_balance_schedule_interval 控制均衡任务的调度周期，默认从 OBServer 启动时间算起，每 2 小时进行一次分区均衡。该配置项易用性较差，无法明确控制分区均衡任务的执行时间。由于 Transfer 动作可能会阻塞对应分区的读写，业务高峰期进行分区均衡会影响 RT；如果需要均衡的分区较多，也可能会暂时加大集群负载，影响集群性能。为了优化这一场景，V4.2.4_CE 版本新增 DBMS_BALANCE 系统包，为用户提供手动触发分区均衡的方法 DBMS_BALANCE.TRIGGER_PARTITION_BALANCE(balance_timeout) ，同时在用户租户创建时内置了定时分区均衡任务 SCHEDULED_TRIGGER_PARTITION_BALANCE，默认每日 00:00 触发一次分区均衡，用户也可根据业务特点通过 DBMS_SCHEDULER 子程序修改定时调度的时间、频率等信息，以便更加灵活、可靠地管理分区均衡任务。

• 死锁检测优化

  V4.2.4_CE 之前版本，OceanBase 数据库已经提供了 CDB/DBA_OB_DEADLOCK_EVENT_HISTORY 视图用于查询数据库中发生的死锁事件及参与死锁事件的环路节点信息，但缺少环路上 Holder 对应的 SQL 信息记录，影响用户诊断。新版本在此基础上丰富了死锁检测的视图信息，补充 Visitor 对应的 SESSION ID，申请资源所在的机器、日志流、Tablet 地址等信息，并记录了事务发生死锁后当前正在执行的 SQL、持锁的 SQL 及请求时间，方便死锁事件诊断。

• ASH REPORT 增强

  OceanBase 是多租户的分布式数据库，目前从系统租户获取的 ASH 报告包含了多租户多节点的汇总信息，但很多时候系统性能问题只涉及某个租户或某个节点。因此 V4.2.3 版本开始提供 ASH Report 节点级分析能力，V4.2.4 版本扩展了租户级别，支持通过 DBMS_WORKLOAD_REPOSITORY.ASH_REPORT 指定生成 ASH Report 的租户 ID 。
  新版本系统性地优化了 ASH Report 展示内容，在借鉴 Oracle ASH 框架基础上，将报告调整地更适用于 OceanBase 数据库性能诊断。例如新增 Top Active Tenants、Top Node Load 维度的分析结果，拆分各类前后台负载等。为了增强可读性，新版本还支持了输出 HTML 格式的 ASH 报告。

• 实时诊断能力增强

  OceanBase 数据库提供了较丰富的 SQL 性能诊断手段，但主要是基于 SQL 执行完的后置诊断。在一些 AP 场景，SQL 执行耗时往往较长，等待执行完后再分析的方式时效性和易用性较差。这时候需要有一种实时方式来获取 SQL 执行情况。V4.2.4_CE 版本基于 SQL PLAN MONITOR 补充了 TABLE SCAN 算子的实时行数、实时字节数等信息，在此之上 ODC 提供了页面化实时诊断功能，便于实时查看 SQL 执行计划与进度。

• 资源规格估算

  我们将数据库中的资源分为逻辑资源和物理资源两大类。逻辑资源指的是逻辑概念对应的实体，如数据结构、线程、锁、会话等；物理资源指的是硬件资源，如 CPU、磁盘、内存等。租户能够创建的逻辑资源量可能受一个或者多个物理资源限制。为了用户可以更方便地获取集群当前的逻辑资源对应的物理资源使用信息，以此更可靠地规划扩缩容、节点替换、备租户创建等操作，V4.2.4_CE 版本新增资源规格估算功能，提供以下一系列动态视图、系统包：

  • 新增 [G]V$OB_TENANT_RESOURCE_LIMIT 视图，用于展示租户在每个 Unit 上的逻辑资源使用量、上限值、生效限制条件和宕机重启后的最大占用量；
  • 新增 [G]V$OB_TENANT_RESOURCE_LIMIT_DETAIL 视图，用于展示租户在一个机器上能创建的逻辑资源所受的物理资源或配置项的限制情况；
  • 新增 DBMS_OB_LIMIT_CALCULATOR.CALCULATE_MIN_PHY_RES_NEEDED_BY_UNIT 子程序，用于计算某个租户在某个节点上所需的最小物理资源量。
  • 新增 DBMS_OB_LIMIT_CALCULATOR.CALCULATE_MIN_PHY_RES_NEEDED_BY_LOGIC_RES 子程序，用于计算特定逻辑资源类型和数量需要的物理资源量；
  • 新增 DBMS_OB_LIMIT_CALCULATOR.CALCULATE_MIN_PHY_RES_NEEDED_BY_STANDBY_TENANT 子程序，用于计算指定主租户创建指定 Unit 个数的备租户需要的最小物理资源量。

• 备份快照表名

  OCP 及下游厂商适配表级恢复功能时，需要向用户展示可供恢复的表。新版本在备份集中持久化了对应的快照表名，并提供通过 ob_admin 来解析快照表名的能力。

• 手动 Transfer Partition 功能增加 Cancel 能力

  Cancel Transfer Partition 是对手动 Transfer Partition 功能的完善。用户在发起手动 Transfer Partition 任务后，任务在没有执行完成之前可以尝试取消任务。本功能提供如下能力：可以指定取消尚未开始 TRANSFER 的任务；可以取消租户下所有没有开始 TRANSFER 的任务；可以取消当前正在执行的 Balance Job。

兼容性变更

产品行为变更

新增如下变更：

功能	变更点说明
定时分区均衡任务触发行为变更	老版本通过租户级配置项 partition_balance_schedule_interval 控制分区均衡任务的调度周期，V4.2.4_CE 版本开始通过定时分区均衡任务 SCHEDULED_TRIGGER_PARTITION_BALANCE 定点触发，默认每日 00:00。具体行为：V4.2.4_CE 版本新建租户时，SCHEDULED_TRIGGER_PARTITION_BALANCE 任务默认开启，partition_balance_schedule_interval 默认为 0，即分区均衡的触发完全依赖定时任务和手动触发。V4.2.4_CE 之前的 V4.x 版本升级到 V4.2.4_CE 版本时，默认旧版本控制行为，SCHEDULED_TRIGGER_PARTITION_BALANCE 任务默认关闭。如果希望升级后使用新行为，需要手工调整以上配置项和定时任务。注意用户设置 partition_balance_schedule_interval 时，要求 SCHEDULED_TRIGGER_PARTITION_BALANCE 关闭，否则将会产生报错。
优化器统计信息手动收集需要提交前置事务	针对在一个未提交事务内进行统计信息收集的行为，当前 OceanBase 数据库和原生 MySQL 数据库有差异，主要差异在于当前 OceanBase 数据库使用上述收集方式在一个未提交事务内进行统计信息时，不会提交前置事务，而原生 MySQL 数据库都会进行提交。新版本修改为：新集群手动收集统计信息时会提交前置事务；针对旧集群升级之后，默认保持旧行为，手动收集统计信息不提交前置事务，如果用户期望在升级上来的旧集群使用新行为，则可以通过修改 ob_compatibility_version 系统变量进行控制。

视图变更

新增如下变更：

视图	变更类型	变更说明
information_schema.events	新增	MySQL 模式下新增，用于记录 Event 信息。
information_schema.tables	字段取值变更	data_length 字段调整为展示表占据的存储空间，按宏块大小计算，单位为 Byte；index_length 字段调整为展示非主键索引占据的存储空间，按宏块大小计算，单位为 Byte。
information_schema.partitions	字段取值变更	data_length 字段调整为展示表分区占据的存储空间 ，按宏块大小计算，单位为 Byte；index_length 字段调整为表分区的索引占据的存储空间，按宏块大小计算，单位为 Byte。
information_schema.role_column_grants	新增	MySQL 模式下新增，用于展示用户在当前会话中被授予的已激活的角色的表级权限信息，以及这些角色被授予的其他角色的表级权限信息。
information_schema.role_routine_grants	新增	MySQL 模式下新增，用于展示用户在当前会话中被授予的已激活的角色的例程权限信息，以及这些角色被授予的其他角色的例程权限信息。
information_schema.role_table_grants	新增	MySQL 模式下新增，用于展示用户在当前会话中被授予的已激活的角色的列权限信息，以及这些角色被授予的其他角色的列权限信息。
CDB/DBA_OB_DEADLOCK_EVENT_HISTORY	字段取值变更	MODULE：取值固定为 transaction。VISITOR：格式为 {session_id:$1}:{txid:$2}，$1 与 $2 均为数字。（老版本仅为 {txid:xx}）OBJECT：行的主键描述，格式为 {addr:"$1:$2"}:{ls:$3}:{tablet:$4}:{row_key:{$5}}，其中 $1 为 IP，$2 为 PORT，共同组成冲突发生的机器进程地址，$3 为冲突发生时所在的日志流，$4 为冲突发生时所在的 tablet，$5 为行锁主键字符串化后的内容。(老版本为 {row_key:xxx})EXTRA_NAME1：固定为 wait_sql。（老版本为 current sql）EXTRA_VALUE1：事务发生死锁的当前正在执行的 SQL 内容。EXTRA_NAME2：固定为 hold_sql_request_time。（老版本为空）EXTRA_VALUE2：持锁语句的请求时间。（老版本为空）EXTRA_NAME3：固定为 hold_sql。（老版本为空）EXTRA_VALUE3：持有 cycle_idx - 1 行记录对应的 Object 的 SQL 语句。（老版本为空）
CDB/DBA_BALANCE_JOBS	新增列	新增 max_end_time 字段，用于展示均负载均衡任务的最大结束时间。
CDB/DBA_BALANCE_JOB_HISTORY	新增列	新增 max_end_time 字段，用于展示均负载均衡任务的最大结束时间。
CDB/DBA_BALANCE_TASKS	新增列	新增 balance_strategy 列，用于展示 balance task 对应的均衡策略。
CDB/DBA_BALANCE_TASK_HISTORY	新增列	新增 balance_strategy 列，用于展示 balance task 对应的均衡策略。
[G]V$OB_ACTIVE_SESSION_HISTORY	新增列 & 列类型变更	新增 plan_hash、thread_id、stmt_type、time_model 四列，分别展示当前执行 SQL 对应的执行计划的 hash 值、当前活跃会话所在的线程 ID、当前活跃会话对应的 SQL 类型、各采样执行阶段的数据集合。修改 sample_time 列类型为 TIMESTAMP。
[G]V$OB_TENANT_RESOURCE_LIMIT	新增	用于展示 OBServer 上每个租户的资源情况，资源包括：日志流、Tablet 等。系统租户可以查看所有租户信息，普通租户只能查看本租户信息。
[G]V$OB_TENANT_RESOURCE_LIMIT_DETAIL	新增	用于展示 OBServer 上每个租户各种资源的限制详情，如：日志流个数受限于配置项、租户内存大小、 clog 盘大小等。系统租户可以查看所有租户信息，普通租户只能查看本租户信息。
CDB/DBA_OB_SERVICES	新增	展示所有租户/本租户所有的 SERVICE_NAME 有关信息。CDB 视图仅在系统租户下支持。
[G]V$OB_PROCESSLIST	新增列	新增 SERVICE_NAME 字段，用于展示该 SESSION 是由哪个 SERVICE_NAME 创建的，为空表示非 SERVICE_NAME 创建。
[G]V$SQL_PLAN_MONITOR	列语义变更	对于 TABLE SCAN 算子有字段含义发生变化：老版本 otherstat_1_value、otherstat_2_value、otherstat_3_value 表示 IO_READ_BYTES、TOTAL_READ_BYTES、TOTAL_READ_ROW_COUNT，在算子计算结束时统计。新版本 otherstat_1_value、otherstat_2_value、otherstat_3_value、otherstat_4_value 表示 IO_READ_BYTES、SSSTORE_READ_BYTES、SSSTORE_READ_ROW_COUNT、MEMSTORE_READ_ROW_COUNT，支持实时统计。

配置项变更

配置项	变更类型	变更说明
plsql_v2_compatibility	含义变更	V4.2.3_CE 版本实现了存储过程 DDL 阶段编译落盘功能，如果存储过程代码很大时，编译时间太久会导致 DDL 很久才能执行完。V4.2.4_CE 版本使用 plsql_v2_compatibility 来控制存储过程 DDL 阶段编译落盘功能是否开启，默认为 False，表示不开启。该参数在历史版本没有被实际使用，不影响其他功能。

系统变量变更

配置项	变更类型	变更说明
event_scheduler	新增	MySQL 模式下新增 Global 级系统变量，用于控制是否开启 Event Scheduler 功能，开启后可创建、修改、删除 Event。默认为 OFF，表示不开启。
cardinality_estimation_model	新增	新增 GLOBAL/SESSION 级别系统变量，用于控制优化器估行时使用的相关性模型。可选择 INDEPENDENT（假设谓词间完全独立）、PARTIAL（假设谓词间有一定的程度的相关性）、FULL（假设谓词间是完全相关的） 三种假设。默认为 PARTIAL。
enable_sql_plan_monitor	新增	新增 SESSION 级系统变量，用于控制当前会话的 SQL 是否记入 SQL PLAN MONITOR。默认为 False，表示默认不记入。开启时，无需考虑其他条件，当前会话的 SQL 均记入 SQL PLAN MONITOR；关闭时，满足 hint 开启、执行时长超出阈值、并行度大于 1 其中一项条件后，当前会话的 SQL 才会记入 SQL PLAN MONITOR。

系统包变更

系统包	变更类型	变更说明
DBMS_BALANCE	新增	用户租户下新增系统包，用于管理负载均衡任务。
DBMS_STATS.GATHER_TABLE_STATS	修改	新增 2 个直方图收集的控制参数 hist_est_percent、hist_block_sample，将基础统计信息和直方图控制策略拆分，拆分后行为如下：estimate_percent: 控制基础统计信息收集是否采样，默认行为不采样，全表扫收集统计信息。block_sample: 控制基础统计信息采样收集时，是否使用块采样的方式，默认不使用块采样，使用行采样。hist_est_percent: 控制直方图收集采样的比例，默认行为由优化器根据表的大小自行决定采样比例。hist_block_sample: 控制直方图收集采样是否使用块采样，默认行为由优化器根据表的大小决定是否使用块采样。同时也新增了 hist_est_percent 和 hist_block_sample 两个 Perfs。
DBMS_OB_LIMIT_CALCULATOR	新增	系统租户新增 DBMS_OB_LIMIT_CALCULATOR 系统包，用于计算不同场景下系统所需的最小物理资源。

语法变更

语法	变更说明
新增 SERVICE_NAME 管理命令	新增 SERVICE_NAME 管理命令：ALTER SYSTEM {CREATE | DELETE | START | STOP} SERVICE $service_name [tenant = '$tenant_name'];
新增 Show Create User 语法	兼容 MySQL show create table 语法，用于展示数据库用户信息。
新增 Event 相关语法	新增 CREATE/ALTER/DROP EVENT 命令。
新增 Cancel Transfer Partition 相关语法	新增 Cancel Transfer Partition 相关语法，用于取消 Transfer 任务。
SET System_Variable 调整为支持子查询	之前版本已支持使用子查询为用户变量赋值，新版本扩展子查询赋值系统变量的用法，以兼容 MySQL 行为。
表级恢复命令调整	修改 ATLER SYSTEM RECOVER TABLE tablename TO TENANT tenantname 语法中 TENANT 关键字为可选。即同时支持 ATLER SYSTEM RECOVER TABLE tablename TO tenantname 语法。

周边配套

OceanBase 数据库 V4.2.4_CE 版本推荐使用的平台工具版本如下：

组件	版本
ODP	V4.2.3
OCP	V4.3.0_CE
OBD	V2.9.2
All in One	V4.2.4
ODC	V4.3.0 BP1
OBCDC	V4.2.4
OMS	V4.2.4_CE
OBClient	V2.2.6
LibOBClient	V2.2.6

升级说明

• 支持 OceanBase 数据库 V4.2.1_CE_BP2 及之前的 V4.x 版本通过有效升级路径在线升级到 OceanBase 数据库 V4.2.4_CE 版本；V4.2.1_CE_BP3 及之后的 BP 版本不再支持升级到 V4.2.4_CE 版本。
• 支持 OceanBase 数据库 V4.2.3_CE 升级到 V4.2.4_CE 版本。
• 集群 Tablet 比较多的情况下，升级过程中 OBServer 重启会比较耗时。Tablet 达到 300 万级别时，重启时长预计会达到 20 分钟。
• V4.2.1_CE 及 V4.2.2_CE 系列升级到 V4.2.4_CE 版本时，如果 _nlj_batching_enabled 配置项处于开启状态，升级前需要关闭，升级完成后可打开。
• ODP 和 OBServer 升级顺序：建议先升级 OBServer，再升级 ODP。
• 升级期间，系统会自动禁用合并和 DDL 操作，升级完成后恢复正常使用。

注意事项

• 主备库自动路由功能需要配套 ODP V4.3.1 和 OCP V4.3.1 及之后版本使用。ODP 使用 rs_list 方式启动时，不支持使用 service_name 功能。
• 物理备库并行同步提升了备库能力上限，但对主租户会有 IO 资源的开销。在主租户已有很多数据时，建议先对主租户进行一次备份，然后用备份恢复的方式创建备库租户，减少主库和新备库之间的日志差异，之后再将备租户的日志恢复源变成网络恢复源的方式。

Version information

Information	Description
Release date	June 19, 2024
Version	V4.2.3_CE_BP1
Commit number	532f46e
OBServer RPM version	oceanbase-ce-4.2.3.1-101000032024061316

System variable changes

System variable	Change type	Description
PLSQL_OPTIMIZE_LEVEL	New	Deprecated the PLSQL_OPTIMIZE_LEVEL parameter in earlier versions, as it had no actual effect. Introduced the PLSQL_OPTIMIZE_LEVEL system variable to control the optimization level during PL code compilation. The default value is 2, indicating front-end and back-end optimization are enabled.

Bug fixes

• Fixed a memory leak issue caused by inconsistency between the timezone used for generating columns or function indexes in insert or query operations and the timezone defined during table creation.
• Fixed the issue where s3_region must be specified while accessing S3 protocol-based object storage for backup and restore operations.
• Improved the PL concurrent compilation efficiency by adding the PLSQL_OPTIMIZE_LEVEL system variable to control the compilation optimization level and concurrency.
• Enhanced performance in some OLTP scenarios.

版本信息

项目	描述
发布日期	2024-06-19
版本号	V4.2.3_CE_BP1
Commit 号	532f46e
OBServer RPM 版本号	oceanbase-ce-4.2.3.1-101000032024061316

系统变量变更

系统变量	变更类型	描述
PLSQL_OPTIMIZE_LEVEL	新增	废弃早期版本 PLSQL_OPTIMIZE_LEVEL 配置项，配置项无实际作用。新增 PLSQL_OPTIMIZE_LEVEL 系统变量，用于控制 PL 代码编译时的优化等级。默认为 2，表示开启前端优化和后端优化。

缺陷修复

• 修复插入或查询时间相关生成列或函数索引时，使用的 Timezone 和创建表时使用的地区定义的 Timezone 不一致，引起的内存泄漏问题。
• 修复备份恢复访问 S3 协议的对象存储时，必须指定 s3_region 的问题。
• 优化 PL 并发编译慢的问题，增加 PLSQL_OPTIMIZE_LEVEL 系统变量支持调整编译优化级别和并发数。
• 优化部分 OLTP 场景下的性能。

Version information

Information	Description
Release date	June 6, 2024
Version	V4.2.1_CE_BP7
Commit number	69b64b8
RPM number	oceanbase-ce-4.2.1.7-107000162024060611

Enhanced features

• Operating system compatibility

  The 8U operating system is supported.

• CPU architecture compatibility

  The ppc64le architecture is supported. #1917

• Intermediate data compression enabled by default for direct load

  During direct load process, if the intermediate data consumes excessive space, it may lead to disk space exhaustion and result in import failure. The new version introduces the intermediate data compression feature for direct load, which can be controlled by the tenant-level parameter _ob_ddl_temp_file_compress_func to determine whether to enable and use the compression algorithm.

  By default, it is set to NONE, indicating no compression is enabled, suitable for scenarios with sufficient disk space and a preference for higher import performance. Based on business requirements, it can be modified to ZSTD or LZ4 to specify the corresponding compression algorithm for enabling intermediate data compression, or set to AUTO for adaptive compression enablement.

• S3, OBS, and GCS supported as the backup media

  The new version supports Amazon Simple Storage Service (S3) and S3-compatible object storage services such as Huawei Cloud Object Storage Service (OBS) and Google Cloud Storage (GCS) as the log archive and data backup destination types. You can also use backup data on S3 and S3-compatible object storage services for physical restore.

• Physical restore at the set or piece level

  In actual business scenarios, second backup is required to manually migrate data backup sets or archive log pieces to a new path. The new version provides the set- and piece-level physical restore feature. You can use the ADD RESTORE SOURCE command to load the data backup sets or archive log pieces in the new path to restore the data to a specified point in time.

• Optimization of selecting the migration source

  The new version defines the regional relationships between OBServer nodes as follows: in the same IDC, in the same region but different IDCs, and in different regions. It also provides the choose_migration_source_policy parameter for you to specify the prioritizing mode for selecting the source replica for migration. You can preferentially select a nearby follower to improve the migration efficiency and reduce the pressure on the leader.

• Physical restore progress statistics

  To allow you to learn about the running status and progress of a physical restore task, and estimate the remaining time required for the restore task, the new version provides the physical restore progress statistics feature. You can query the CDB/DBA_OB_RESTORE_PROGRESS view for the restore progress in real time.

• Optimization of minor compaction scheduling for transaction data tables in a large-sized tenant

  The new version improves the minor compaction timeliness for transaction data tables in large-sized tenants by speeding up the scheduling of minor compactions of transaction data tables, thereby further optimizing the performance of buffer tables.

• Global CPU resource isolation for foreground and background tasks

  OceanBase Database of previous versions allow you to configure different unit configs for tenants to implement CPU resource isolation between the tenants. In addition, OceanBase Database provides the DBMS_RESOURCE_MANAGER system package for you to configure CPU resource isolation within a tenant. The new version supports global CPU resource isolation for foreground and background tasks. You can limit the CPU resources available for background tasks on an overall basis, which is more convenient and easier than using the DBMS_RESOURCE_MANAGER system package to separately configure resource isolation for each tenant.

• Response time histograms

  OceanBase Database of previous versions provide statistics on the average response time and maximum response time in different SQL categories, which lack fine-grained statistics that reflect the SQL execution performance at a certain percentile. The new version supports response time histograms. You can query and monitor response time statistics of a specific SQL type at a certain percentile, such as P90 or P95, by using the [G]V$OB_QUERY_RESPONSE_TIME_HISTOGRAM view.

• Partition balancing strategy optimization

  The new version optimizes the partition balancing strategy. You can scatter continuous partitions when you create a partitioned table. For a user table that contains a LONGTEXT or LOB column or a local index, the system also calculate associated tables for partition disk balancing, contributing to more balanced disk usage.

• OBCDC startup acceleration

  The new version optimizes the startup performance of OceanBase Change Data Capture (CDC) and helps improve the synchronization performance of downstream binlog tools.

• Extension of the JSON nesting depth

  In OceanBase Database of previous versions, the maximum nesting depth of a JSON document is 100. The new version allows you to use the json_document_max_depth parameter to adjust the nesting depth limit as required.

• HBase PageFilter

  The OB-HBase model adopted by the new version supports row restriction by PageFilter in HBase and returns the specified number of rows.

• Backup performance optimization

  During backup in OceanBase Database of previous versions, a continuity check is performed to make sure that the baseline version number is greater than the minor compaction version number, which ensures data integrity. However, the continuity check involves reading data from the backup media and executing operations during which the operated data needs to be locked, compromising the backup performance. The new version reduces the performance overhead caused by a continuity check during backup. You can use the ha_low_thread_score parameter to control the number of threads used for backup, thereby effectively improving the backup performance.

Product behavioral changes

• Default value of optimizer_features_enable changed to 4.2.1.7

  OceanBase clusters created by using OceanBase Database V4.2.1 BP7 or later support new features of the optimizer. However, if you upgrade OceanBase Database from V4.2.1 BP6 or earlier to V4.2.1 BP7, the setting of the system parameter optimizer_features_enable of the optimizer remains the same as that before the upgrade. In V4.2.1 BP6 or earlier, the default value of the system parameter is empty for the optimizer. That is, the optimizer features of V4.2.1.0 are used.

• Removing limitations on major compactions upgrades

  The limitations on major compactions are removed for upgrades from minor versions of OceanBase Database V4.2 to V4.2.1 BP7.

Parameter changes

Parameter	Change type	Description
choose_migration_source_policy	New	The strategy for selecting the migration source. It is a tenant-level parameter. Two strategies are supported: idc: A follower in the same IDC is preferentially selected as the migration source. If only the leader is available, the leader is selected. region: A follower in the same region is preferentially selected as the migration source. If only the leader is available, the leader is selected. The default value is idc.
json_document_max_depth	New	The maximum nesting depth for a JSON document. It is a tenant-level parameter. The default value is 100, and the maximum value is 1024.
log_storage_warning_trigger_percentage	New	The write performance threshold in percentage for triggering a log disk fault. It is a cluster-level parameter. The default value is 0, indicating that if the write delay of a log disk is greater than the value of log_storage_warning_tolerance_time, the log disk is considered faulty. If the value is greater than 0, a log disk is considered faulty when its current write performance is lower than the value of log_storage_warning_trigger_percentage multiplied by the baseline and the duration of the situation reaches the value of log_storage_warning_tolerance_time. We recommend that you set this parameter to a value of no more than 10.
sql_plan_management_mode	New	Specifies whether to enable the SQL plan management (SPM) feature. It is a tenant-level parameter. The default value is Disable, indicating that the SPM feature is disabled. You can use either this parameter or the system variable optimizer_use_sql_plan_baselines to enable the SPM feature. However, we recommend that you use this parameter to enable SPM, which eliminates the need to restart the business application or OceanBase Database Proxy (ODP) for SPM to take effect.

System variable changes

System variable	Change type	Description
optimizer_features_enable	Default value changed	The default value is changed from empty to 4.2.1.7. In V4.2.1 BP6 and earlier, an empty default value indicates to use the optimizer features of V4.2.1.0. The new default value indicates that clusters created by using V4.2.1 BP7 and later can use the optimizer features of the new version. Clusters created by using V4.2.1 BP6 and earlier still use the optimizer version specified before the upgrade.

View changes

View	Change type	Description
CDB/DBA_OB_RESTORE_PROGRESS	Columns modified	The RECOVER_SCN, RECOVER_SCN_DISPLAY, RECOVER_PROGRESS, TABLET_COUNT, FINISH_TABLET_COUNT, and RESTORE_PROGRESS columns are added to show the physical restore progress.
[G]V$OB_QUERY_RESPONSE_TIME_HISTOGRAM	New	This view is added to display SQL response time histograms by SQL category. It is a tenant-level view.
CDB_TAB_COL_STATISTICS	Content modified	The index table information is no longer displayed.

Bug fixes

• Fixed the issue where the column specified in the select count (column) from table [where]; DDL statement, which added more than two columns to the end of a table, was not the last column in the table. If dump data was involved, the query results had more issues. #1952

• Fixed the issue where a transaction was automatically ended for a lock conflict in the first statement of the transaction.

• Fixed the issue where the parallel execution of TRUNCATE TABLE got stuck.

• Added a check on the slog disk space to the restart process.

• * Fixed the issue where incorrect passwords were occasionally passed during logons to standby tenants.

• Fixed the issue of PlTemp memory bloat in retries upon an open cursor failure.

• Fixed the issue where multiple rows were queried at a time in a pushdown scenario.

• Fixed the issue of repeated SQL retries due to the contention check on the large transaction lock.

• Fixed the issue of data disk space exhaustion caused by statistics collection for hybrid histograms.

• Fixed the issue where the outline could not be bound when an index contained a special character.

• Fixed the issue of execution plan deviation caused by inaccurate row estimation, which resulted from an error returned for plan generation in the scale-in scenario.

• Fixed the issue of a QPS or TPS drop in the migration and replication process.

• Fixed the issue of a core dump caused by direct load using OBLOADER.

• Fixed the issue where OceanBase Migration Service (OMS) subscribed to DELETE operations by mistake because CDC processed information erroneously when OBKV executed check_and_upsert.

• Fixed the issue where ERROR 4013 was reported when oversized temporary files were generated during index creation.

• Fixed the issue where too much recursion occurred due to an excessive number of connections, which resulted in a core dump.

• Fixed the issue of a high memory usage caused by MemTable occupation during backup.

• Fixed the issue of a core dump caused by the concurrence of direct load and DDL execution.

• Fixed the issue where a migration task failed due to inconsistent archive status when the archive mode was optional and data could not be written into the archive media for a long time.

• Fixed the issue of migration, replication, and rebuilding failures after the upgrade from versions earlier than V4.2.1 BP5 to V4.2.1 BP5 or later.

版本信息

项目	描述
发布日期	2024-06-06
版本号	V4.2.1_CE_BP7
Commit 号	69b64b8
OBServer RPM 版本号	oceanbase-ce-4.2.1.7-107000162024060611

特性增强

• 操作系统适配支持

  新增 8U 操作系统适配支持。

• CPU架构适配支持

  新增 ppc64le架构支持。#1917

• 旁路导入中间数据支持开启压缩
  旁路导入过程中，中间数据占用空间过大时，可能会导致磁盘空间满，进而导致导入失败。新版本增加旁路导入中间数据压缩功能，通过租户级配置项 _ob_ddl_temp_file_compress_func 控制是否开启及使用的压缩算法。

  默认为 NONE，表示不开启压缩，适用于磁盘空间足够，希望获得更高导入性能的场景。根据业务需要可修改为 ZSTD 或 LZ4，来指定对应的压缩算法开启中间数据压缩，或设置为 AUTO 自适应开启压缩。

• 备份恢复支持 S3/OBS/GCS

  新增支持 S3 协议，可将 S3/OBS/GCS 作为日志归档和数据备份的目的端，同时支持使用 S3/OBS/GCS 上的备份数据进行物理恢复。

• SET/PIECE 级物理恢复

  实际业务中存在二次备份场景，需把数据备份集或归档日志手动搬迁到新的路径。新版本增加了 SET/PIECE 级物理恢复功能，提供 ADD RESTORE SOURCE 命令来加载新路径的数据备份集 SET 或日志归档 PIECE，支持按需恢复到指定时间。更多信息请参见 执行指定路径的恢复。

• 迁移复制源端选择优化

  新版本将各 Server 按照地域信息划分为同 IDC、同 Region 不同 IDC、跨 Region 三种区域关系，同时提供 choose_migration_source_policy 配置项，用于迁移复制场景下，指定源端的选择模式，以便优先考虑地理位置就近因素以及 Follower 副本来提升迁移效率、降低 Leader 压力。更多信息请参见 choose_migration_source_policy。

• 物理恢复进度统计

  为了用户在使用物理恢复功能时可以了解恢复任务的运行状态和进度，并预估完成时间，新版本增加了物理恢复进度统计功能。用户可通过 CDB/DBA_OB_RESTORE_PROGRESS 视图实时查看恢复进度，获得更好的使用体验。更多信息请参见 查看物理恢复进度。

• 大规格租户事务数据表转储调度优化

  优化大规格租户下事务数据表转储不及时的问题，加快事务数据表的转储调度，进一步优化 Buffer 表性能。

• 全局 CPU 前后台任务隔离

  在之前版本，OceanBase 数据库已实现通过租户 Unit 规格来配置租户间的 CPU 资源隔离，提供 DBMS_RESOURCE_MANAGER 系统包来配置租户内的 CPU 资源隔离。新版本支持全局 CPU 前后台任务隔离，可以在整体层面上限制后台任务的可用资源，相对租户内使用 DBMS_RESOURCE_MANAGER 单独配置更加方便易用。更多信息请参见 使用全局 CPU 资源的前后台隔离。

• 新增响应时间统计直方图

  在之前版本，OceanBase 数据库已支持基于不同 SQL 类别的平均响应时间/最大响应时间指标，但缺乏更细粒度的反映某个分位 SQL 执行性能的指标展示。新版本增加响应时间直方图功能，可基于 [G]V$OB_QUERY_RESPONSE_TIME_HISTOGRAM 系统视图计算和监控不同类型 SQL 的 P90/P95 之类的响应时间统计。更多信息请参见 GV$OB_QUERY_RESPONSE_TIME_HISTOGRAM、V$OB_QUERY_RESPONSE_TIME_HISTOGRAM。

• 分区均衡策略优化

  新版本优化了分区均衡策略，支持在创建分区表时连续分区打散。当用户表存在 longtext/lob 列或局部索引时，分区磁盘均衡也会计算关联表，使磁盘使用更加均衡。更多信息请参见 租户内均衡。

• OBCDC 启动加速

  新版本优化了 OBCDC 启动性能，协助提升下游 Binlog 工具的同步性能。

• JSON 可嵌套层数扩展

  历史版本限制 JSON 文档的可嵌套最大深度为 100，新版本提供 json_document_max_depth 配置项，允许用户根据需求调整嵌套深度。对于超过 100 的 JSON 嵌套层数需求，用户可适当调大该配置。更多信息请参见 json_document_max_depth.

• 支持 HBase PageFilter 功能

  OB-HBase 模型下, 支持使用 HBase 中的 PageFilter 对 Row 进行限制, 返回指定数量的 Row。

• 备份性能优化

  OceanBase 数据库在备份过程中，通过连续性校验确保基线版本大于转储版本来保证数据完整。然而，连续性检查涉及读取备份介质上的数据并执行带锁操作，因此会影响备份性能。新版本优化了备份过程中连续性校验操作带来的性能开销，支持通过 ha_low_thread_score 控制备份使用的线程数，从而有效提高备份性能。更多信息请参见 ha_low_thread_score。

产品行为变更

• 优化器版本调整默认值为 "4.2.1.7"（optimizer_features_enable）

  V4.2.1 BP7 开始，新建的集群可以使用新版优化器能力，但从 V4.2.1 BP6 及以下版本升级到 V4.2.1 BP7 依然会保留之前的系统参数配置值。V4.2.1 BP6 及以下版本，默认值都为空，表示使用 V4.2.1.0 的优化器能力。

• 升级合并限制放开

  从 OceanBase 数据库 V4.2 的各版本升级到 V4.2.1 BP7 时，升级过程中放开了合并的限制。

配置项变更

配置项	变更类型	描述
choose_migration_source_policy	新增	新增租户级配置项，用于控制迁移源端的选择策略。提供 2 种选择： idc：在同 IDC 的机器中优先选择 Follower 副本作为源端。若仅有 Leader 副本，则选择 Leader 副本。region：在同 Region 的机器中优先选择 Follower 副本作为源端。若仅有 Leader 副本，则选择 Leader 副本。 默认策略为 idc。
json_document_max_depth	新增	新增租户级配置项，用于设置 JSON 文档中允许的最大嵌套层数，默认为 100，可修改嵌套层数上限至 1024。
log_storage_warning_trigger_percentage	新增	新增集群级配置项，用于设置触发日志盘故障的写入性能百分比阈值。 默认为 0，表示如果日志盘写盘延迟大于配置项 log_storage_warning_tolerance_time 的值，则认为日志盘故障。 若设置值大于 0，表示如果当前的日志盘写入性能下跌到基准性能乘以百分之 log_storage_warning_trigger_percentage 以下，并且性能下降持续 log_storage_warning_tolerance_time 秒，则认为日志盘故障。 建议设置不超过 10。
sql_plan_management_mode	新增	新增租户级配置项，用于控制是否开启 SPM 功能。默认为 Disable，表示关闭 SPM 功能。 此配置项与系统变量 optimizer_use_sql_plan_baselines 都可以控制 SPM 功能的开启，两者任一打开都可以开启 SPM 功能。推荐使用此配置项开启 SPM，可以避免重启业务应用或 ODP 才能使 SPM 生效的步骤。

系统变量变更

系统变量	变更类型	描述
optimizer_features_enable	默认值调整	V4.2.1 BP6 及之前的 V4.2.1 BP 版本，默认值为空，表示使用 V4.2.1.0 的优化器能力。 V4.2.1 BP7 版本调整默认值为 4.2.1.7，表示 V4.2.1 BP7 开始，新建的集群可以使用新版优化器能力，而从 V4.2.1 BP6 及以下版本升级依然使用升级前指定的优化器版本。

视图变更

视图	变更类型	描述
CDB/DBA_OB_RESTORE_PROGRESS	新增列	新增 RECOVER_SCN、RECOVER_SCN_DISPLAY、RECOVER_PROGRESS、TABLET_COUNT、FINISH_TABLET_COUNT、RESTORE_PROGRESS 6 列，用于展示物理恢复进度。
[G]V$OB_QUERY_RESPONSE_TIME_HISTOGRAM	新增	各租户新增视图，用于展示不同类别的 SQL 响应时间直方图信息。
CDB_TAB_COL_STATISTICS	视图内容变更	新版本不再展示索引表相关信息。

缺陷修复

• 修复通过 DDL 在表尾部加多个列 (>=2) 后，若 select count (指定列) from table [where]; 中指定列不是最后加的列，如果有转储数据，查询结果可能出现比预期多的问题。 #1952

• 修复事务第一条语句锁冲突后自动结束事务的问题。

• 修复并发 Truncate Table 可能卡住的问题。

• 增加重启时对 Slog 盘空间的检查。

• 修复登录备租户偶现密码错误的问题。

• 修复 open cursor 失败重试，导致 PlTemp 内存膨胀的问题。

• 修复特定下压场景下可能出现瞬时查询多行的问题。

• 修复大事务锁冲突检查可能导致 SQL 一直重试的问题。

• 修复混合直方图收集导致数据盘出现暴涨的问题。

• 修复索引携带特殊字符，导致 Outline 绑定不上的问题。

• 修复缩容场景下计划生成报错导致估行出错，计划走偏的问题。

• 修复迁移复制过程中，有可能触发 QPS 或 TPS 下跌的问题。

• 修复使用 obloader 旁路导入可能导致 Core Dump 的问题。

• 修复 OBKV 执行 check_and_upsert 操作时，CDC 异常处理导致 OMS 误订到 Delete 操作的问题。

• 修复建索引产生超大临时文件导致报 4013 的问题。

• 修复连接数量过多导致递归爆栈引起 Core Dump 的问题。

• 修复备份过程中占用 MemTable 导致内存消耗大的问题。

• 修复旁路导入与 DDL 并发可能导致 Core 的问题。

• 修复归档模式为 optional，且归档介质出现长时间不可写入的情况下，归档状态不一致导致迁移任务失败的问题。

• 修复 V4.2.1 BP5 之前的版本升级至 V4.2.1 BP5 及之后版本期间，可能会出现迁移、复制、Rebuild 失败的问题。

开源鸣谢

在此版本发布中，特别感谢社区伙伴的贡献：

感谢浪潮商用机器李松青 @DBres4Power 在ppc64le架构适配上的贡献。

Version information

Information	Description
Release date	May 22, 2024
Version	V4.3.1_CE_BETA
Commit number	bad90e8
RPM number	oceanbase-ce-4.3.1.0-100000032024051615
Release note	The Beta version resolved most of the issues and is becoming more and more stable. However, there may still be some minor issues or errors that need to be addressed in the final stable release, so we recommend that you use this version in a testing environment.

Overview

OceanBase Database V4.3.1, building on the foundation of V4.3.0, introduces the new feature of full-text indexing to enhance document retrieval efficiency, and expands features such as real-time materialized views, materialized view rewriting, and primary key materialized views to meet analytical needs across various scenarios. This version introduces a new partition management mechanism, including partition swapping, external table partitions, and an extension of data types for partition keys in MySQL mode, enhancing the processing capability for large-scale data. The multi-model features (including JSON, XML, and GIS) are also upgraded, adding support for multi-valued indexes for JSON data and partial updates of JSON, promoting the migration and integration of heterogeneous data. Compatibility with MySQL continues to be enhanced, supporting lateral derived tables and MySQL locking functions, facilitating integration into the ecosystem. Incremental direct load capabilities are provided, improving the performance of multiple import scenarios, while also optimizing DML performance in scenarios with multiple local indexes, improving the efficiency of basic statistical information collection, and achieving significant performance improvements in row sampling and small-scale TP scenarios. The new version further optimizes resource usage, supporting clog caching, storage compression, SQL temporary result compression, system log compression, and other features. It complements the MySQL privilege system and supports operating system configuration checks to strengthen system security. As always, OceanBase Database focuses on user experience, adding resource specification estimation capabilities, enhancing backup transparency, providing IPv6 format support, and empowering database management and operations.

V4.3.1 is positioned as a Beta version, recommended for testing in real-time data warehouses or online transaction processing businesses. A General Availability (GA) version recommended for production use will be released in the second half of the year.

Key features

Kernel enhancement

• Full-text indexing in MySQL mode (Experimental)

  In relational databases, indexes are often used to accelerate queries based on exact value matches, but ordinary B-tree indexes are not suitable for scenarios involving large amounts of text data that require fuzzy searches. In such cases, the only option is to perform a full table scan for fuzzy queries on each row, which often fails to meet performance requirements when dealing with large text and high data volumes. Additionally, some complex query scenarios, such as approximate matching and relevance sorting, are also difficult to support through SQL rewriting.

  To address the issues mentioned above, OceanBase Database introduces the full-text indexing feature in V4.3.1. By preprocessing text content to establish keyword indexes, it significantly enhances full-text search efficiency. This feature is an experimental feature in V4.3.1. Future versions are expected to enhance its functionality and mature it into a feature that is ready for production use. The current release includes the following capabilities:

  • Supports full-text indexing in MySQL mode, compatible with basic MySQL syntaxes.
  • Allows the creation of full-text indexes on CHAR, VARCHAR, and TEXT columns during table creation.
  • Applicable to partitioned tables.
  • Supports the creation of multiple full-text indexes on a primary table.
  • Includes three built-in tokenizers: SPACE (space), NGRAM, and BENG (basic English).
  • Supports using a single match against for full-text searches on multiple columns.
  • Supports NATURAL LANGUAGE MODE.

• Materialized view enhancements

  OceanBase Database V4.3.0 introduced the materialized view feature, which enhances query performance by precomputing and storing the results of view queries, thus reducing the burden of real-time computations and simplifying complex query structures to meet the demands of analytical processing (AP) scenarios. With the release of V4.3.1, the capabilities have been further extended to include support for real-time materialized views. This provides the capacity to perform immediate calculations based on materialized views and materialized log (MLOG) data, addressing the needs of businesses with critical real-time analysis requirements. The latest update also introduces the ability to define primary key constraints on materialized views, helping users to fine-tune performance for specific queries such as single-row lookups, range queries, and joins where the primary key is a determining factor. In addition, enhanced support for incremental updates on materialized views in inner join scenarios ensures more efficient refresh rates in various situations.

  Additionally, in V4.3.0, when you use materialized views, you need to manually update your business scripts, replacing operations on original tables with queries addressing the corresponding materialized views. This process introduced a significant manual rewrite burden. The new version improves this by offering automated query rewriting capabilities for materialized views in select scenarios. When you set the system variable QUERY_REWRITE_ENABLED to True, you can create materialized views and specify ENABLE QUERY REWRITE to enable the query rewrite capabilities. This means that the system can rewrite queries on the original tables to queries targeting the materialized views, thereby reducing the need for business modifications.

• Partition exchange

  As time goes by, a large amount of historical data may accumulate in the table, some of which may not require frequent access but need to be retained to meet compliance or historical data analysis requirements. To improve query performance, businesses often need to differentiate between active and inactive data and archive the inactive data. Although transferring data to a new table using SQL can address this scenario, the performance is often suboptimal for large data volumes. Therefore, OceanBase Database V4.3.1 introduces the partition exchange feature, which, by modifying the definitions of partitions and tables in the data dictionary without the need for physical data replication, can almost instantly move data from Table A to a partition in Table B, greatly enhancing the performance of data migration. This release includes the following features:

  • Support for exchanging data between a partition of a partitioned table and a normal table.
  • Support for partitioned tables with the partition type being Range (Range Columns).
  • Support for subpartitioned tables with no requirements for the partition type, and the subpartition type being Range (Range Columns).
  • Support for the INCLUDING INDEXES action, which means that during partition exchange, corresponding local indexes will also be part of the exchange and will remain usable afterward.
  • Support for WITHOUT VALIDATION mode, which requires users to ensure the data conforms to the partition key range.

• External table partitioning

  OceanBase Database introduced support for external tables since V4.2.0, though initially, this feature was confined to non-partitioned tables. In situations where a large number of files exist but a query only needs to process a fraction of them, non-partitioned external tables were less efficient since they had to scan the entire file set without the ability to trim away irrelevant data, resulting in suboptimal performance. OceanBase Database V4.3.1 adds partitioning capabilities to external tables, supporting partitioning methods similar to list partitioning on regular tables. This update offers two partitioning approaches—automatic and manual. When the automatic option is selected, the system organizes files into groups according to the predefined partition keys. If manual partitioning is selected, users are required to designate specific subpaths for data files tied to each partition. By leveraging partition pruning, which tailors data retrieval to only the necessary partitions, external table queries become far more efficient, narrowing the scope of file scanning and considerably improving query performance.

• Expansion of range columns partition key types in the MySQL mode

  To accommodate partitioning requirements of various business, OceanBase Database V4.3.1 has expanded the compatibility of range columns partition key types to include double, float, decimal, timestamp, and other types as partition keys for range columns partitioning. The specific types supported are:

  • DECIMAL and DECIMAL[(M[,D])]
  • DEC, NUMERIC, and FIXED
  • FLOAT[(M,D)] and FLOAT(p)
  • DOUBLE and DOUBLE[(M,D)]
  • DOUBLE PRECISION and REAL
  • TIMESTAMP

• PL recompilation logic optimization

  After a stored procedure is compiled into a shared library, it can be used by multiple threads. However, if dependent objects change, the shared library might become invalid and require recompilation. In the V4.3.1 release, the scenarios for recompilation have been thoroughly reviewed and refined. A series of logical optimizations have been carried out concerning temporary table matching, collection of dependency information for static SQL, and changes in table DDL. These enhancements aim to reduce the scenarios where PL CACHE cached objects become invalid, thus decreasing the need for recompilation.

• PL compilation results persisted to disk

  After a PL function or procedure is compiled, it is added to the PL cache. However, when memory pressure is excessive, it may be evicted, resulting in the loss of the compilation results after a restart, and requiring recompilation in distributed scenarios. In these situations, the PL cannot hit the PL cache, leading to triggering LLVM compilation and thus consuming some CPU resources. Starting from V4.3.1, PL functions and procedures are stored in system tables on disk, allowing the reuse of the cache after a single compilation in scenarios where DDL does not invalidate the PL cache, regardless of restart or distributed scenarios.

Multi-model features

• Multi-valued index for JSON data in MySQL mode (Experimental)

  MySQL 8.0 supports the multi-valued index feature for JSON documents and other collection data types. You can create a multi-valued index on an array or collection to efficiently retrieve elements. In OceanBase Database V4.3.1, the MySQL mode supports the multi-valued index feature for JSON data. You can create an efficient secondary index on a JSON array of multiple elements. This enhances the capabilities to query complex JSON data structures while ensuring the data model flexibility and data query performance. This feature is in the experimental stage in OceanBase Database V4.3.1, and will be enhanced for use in production environments in later versions. Take note of the following considerations when you use the multi-valued index feature:

  • Supports pre-creation of multi-valued indexes and composite multi-valued indexes.
  • Supports unique and non-unique multi-valued indexes.
  • Supports creating multi-valued indexes on JSON array element types such as INT, UINT, DOUBLE, FLOAT, and CHAR.
  • Applicable to partitioned tables.
  • Supports the use of multi-value indexes in queries with the MEMBER_OF(), JSON_CONTAINS(), and JSON_OVERLAPS() functions.

• MySQL JSON

  OceanBase Database V4.3.1 adds support for the JSON_SCHEMA_VALID, JSON_SCHEMA_VALIDATION_REPORT, and JSON_ARRAY_APPEND expressions.

• MySQL JSON partial update

  Some users store business data in JSON documents. To update a JSON document in OceanBase Database of earlier versions, all data in the document must be read and updated as a whole, which results in unsatisfactory update performance if the document is large. OceanBase Database V4.3.1 supports the JSON partial update feature to address this issue. You can use specific expressions such as JSON_SET, JSON_REPLACE, and JSON_REMOVE to update part of the fields in a JSON document, improving the update performance. You can enable this feature by using the log_row_value_options parameter.

• MySQL GIS enhancements

  OceanBase Database V4.1 supports geographic information system (GIS) data types and some spatial object-related expressions of MySQL. OceanBase Database V4.3.1 supports the storage, computing, and analysis of spatial data, and adds support for MySQL expressions such as ST_Crosses, ST_Overlaps, ST_Difference, ST_Union, ST_SymDifference, ST_Length, ST_Centroid, and ST_AsGeoJSON. As the most applied database in the GIS industry, PostgreSQL provides some common expressions different from those of MySQL. OceanBase Database V4.3.1 supports these expressions, such as _ST_Touches, _ST_Equals, _ST_MakeEnvelope, _ST_ClipByBox2D, _ST_GeometryType, _ST_IsCollection, _ST_NumInteriorRings, _ST_PointOnSurface, ST_AsMVTGeom, and _ST_AsMVT, and the storage of three-dimensional spatial objects.

• MySQL XML

  OceanBase Database V4.3.1 adds support for the ExtractValue and UpdateXML expressions.

Compatibility with MySQL

• Lateral derived tables

  A lateral derived table is a special type of derived table that can reference columns of a table that appears earlier in the same FROM clause. This makes SQL statements easier to read and write and reduces repeated data scans and calculations, thereby improving the SQL execution performance. OceanBase Database V4.3.1 supports the usage methods of lateral derived tables in MySQL 8.0.

• Communication protocol improvements

  OceanBase Database V4.3.1 supports the MySQL communication protocol command COM_SET_OPTION for specifying connection-level options, such as MYSQL_OPTION_MULTI_STATEMENTS_ON and MYSQL_OPTION_MULTI_STATEMENTS_OFF. It also supports the MySQL AuthSwitchRequest protocol for switching the authentication method, avoiding authentication errors caused by incompatible client versions.

• Show Extended Tables/Columns/Index

  OceanBase Database V4.3.1 supports new SHOW EXTENDED syntaxes of MySQL 8.0, such as SHOW EXTENDED TABLES, SHOW EXTENDED COLUMNS, and SHOW EXTENDED INDEX.

• MySQL locking functions

  OceanBase Database V4.3.1 supports the following MySQL locking functions: GET_LOCK(), IS_FREE_LOCK(), IS_USED_LOCK(), RELEASE_ALL_LOCKS(), and RELEASE_LOCK(). You can use these functions in different parts of SQL statements, for example, in the ORDER BY and HAVING clauses of a SELECT statement, in the WHERE condition of a SELECT, DELETE, or UPDATE statement, or in a SET statement. The function expression can be a literal, column value, NULL, variable, built-in function or operator, loadable function, or stored function. Locking functions are a type of built-in functions that allow you to define and use locks.

• Support for output parameters in stored procedures

  When you execute the CALL PROCEDURE statement by using the PreparedStatement protocol in MySQL mode, output parameters are supported in stored procedures.

Performance improvements

• Incremental direct load (Experimental)

  OceanBase Database supports direct load since V4.1.0. This feature simplifies the data loading path and skips the SQL, transaction, MemTable, and other modules to directly persist data into SSTables, which significantly improves the data import efficiency. However, in a scenario where the table data needs to be imported multiple times, the existing data in the table needs to be written repeatedly during each import, compromising the incremental import performance. OceanBase Database V4.3.1 provides the incremental direct load feature to optimize incremental import. Specifically, the database writes only new data rather than repeatedly writing all existing data. This ensures high import performance. You can use the /*+ direct(need_sort, max_errors_allowed, load_mode)*/ hint in the LOAD DATA or INSERT INTO SELECT statement to specify whether to enable the incremental direct load feature. You can leave load_mode unspecified or set load_mode to full to enable full direct load. You can set load_mode to inc_replace to enable incremental direct load. This feature is in the experimental stage in OceanBase Database V4.3.1, and will be enhanced for use in production environments in later versions.

• Performance improvement for the SELECT INTO OUTFILE statement

  In earlier versions, the SELECT INTO OUTFILE statement supports reading data from tables in parallel but can write data into external files only in serial, leading to a performance bottleneck in data export. OceanBase Database V4.3.1 supports parallel export. It allows you to set the data export mode by specifying the SINGLE and MAX_FILE_SIZE options in the SELECT INTO OUTFILE statement. The SINGLE option specifies to export data to a single file or multiple files. If you set the degree of parallelism (DOP) to a value greater than 1 and SINGLE to FALSE, the data is exported to multiple files in parallel. The MAX_FILE_SIZE option limits the size of an external file.

• DML performance optimization in a scenario with multiple local indexes

  The performance of DML operations on an indexed database table will decrease due to synchronous index update, especially when a large number of indexes exist. The new version takes a series of measures to reduce the maintenance overhead of local indexes by 45%, thereby improving the overall DML execution performance. Some of the measures are as follows: unlock a table with local indexes, not to record the lock holder, skip the transaction conflict check on non-unique indexes, and reduce the overhead in reporting DML statistics.

• Parallel synchronization of a single log stream

  In the log synchronization model of OceanBase Database of earlier versions, multiple log streams are synchronized and processed in parallel, and a single log stream is synchronized in pipeline mode. This log synchronization model can meet the performance requirements in a scenario where logs in the local disk are consumed in the same city. However, in a scenario where the standby database is remotely deployed or logs are read from an object storage service provided by a public cloud vendor, the performance will decrease. OceanBase Database V4.3.1 implements a model that supports parallel synchronization of a single log stream based on file data blocks, significantly improving the synchronization performance and optimizing the memory usage.

• Statistics collection performance optimization

  In earlier versions, internal SQL statements of related aggregate functions are executed to collect basic statistics. OceanBase Database V4.3.0 supports pushing down aggregate functions to the storage layer. OceanBase Database V4.3.1 supports pushing down more aggregate functions to the storage layer and avoids projecting data to the SQL layer for computing. This way, all operations for collecting basic statistics are executed at the storage layer, improving the statistics collection efficiency. Compared with V4.3.0, V4.3.1 improves the overall collection performance by about 5%.

• Row sampling performance optimization

  When the overhead for full data processing is high, you can learn the overall data distribution by observing a small part of the data. For example, the optimizer samples data to analyze data distribution and assist execution plan generation. In earlier versions, OceanBase Database uses the WHERE condition to filter a row and determines whether to sample this row. This process is time-consuming because the full data is scanned row by row. OceanBase Database V4.3.1 optimizes the sampling process. Specifically, it filters data first and then applies conditions, reducing the data reading cost. For data stored in columnar storage, it reads only required column data. This remarkably improves the sampling performance.

• Performance optimization for environments with small specifications

  To improve the performance in environments with four or eight CPU cores, OceanBase Database V4.3.1 is optimized in terms of background thread scheduling, location cache access, read/write paths, and system calls. V4.3.1 improves the performance in online transaction processing (OLTP) test scenarios by 20% to 30% compared with V4.3.0.

Reliability improvements

• Write stop upon high data disk usage

  In earlier versions, when the data disk usage is high, user write requests will still be processed until an error is returned after the memory is full due to minor compaction failures or after the clog disk is full. In this case, you need to urgently scale out the clog disk space or tenant memory to resolve this issue. In the new version, the kernel provides the feature of write stop upon high data disk usage. When the data disk usage reaches the value specified by data_disk_write_limit_percentage, an error is returned for new user write requests. After you drop tables, transfer data, or scale out the disk space to reduce the data disk usage, user writes automatically resume.

Resource usage optimization

• Clog caching

  In earlier versions of the V4.x series, OceanBase Database supports LogHotCache to cache part of the real-time logs in scenarios such as log archiving and replay. This reduces the overhead in reading logs from the disk. However, when logs are written at a high speed or multiple OceanBase Change Data Capture (CDC) instances repeatedly pull closely-associated logs, the I/O throughput of the log disk is high, resulting in full usage of the log disk bandwidth in extreme conditions. The bandwidth of a cloud disk is limited. Therefore, the overhead in reading logs from the disk must be reduced to support more log consumption scenarios. This version provides the clog caching feature, which allows consumers to directly read logs from the log cache. If the log cache is not hit, consumers can read logs from the disk and write the logs to the cache to avoid repeated disk reads, thereby reducing the usage of the log disk bandwidth. Three statistical items, 50065, 50066, and 120010, are added to the SYSSTAT views to indicate the number of log cache hits, number of log cache misses, and clog cache size at the tenant level.

• Strengthened resource management for direct load

  In OceanBase Database V4.x, the direct load feature drastically increases the data import efficiency. However, when a high DOP is specified and many direct load tasks are executed in parallel, a number of threads and memory resources are occupied due to loose resource control, thereby affecting the normal execution of other tasks. OceanBase Database V4.3.1 enhances direct load resource management from three dimensions:

  • A task-level resource management module is provided. Thread and memory resources are requested based on the execution mode of the import task and the number of partitions.
  • A tenant-level resource management module is provided to manage the thread and memory resources of a tenant that are available for direct load. The module detects resource pool changes based on scheduled tasks and reclaims resources requested by import tasks that are interrupted unexpectedly.
  • An OBServer node-level resource management module is provided for node-level resource application. The module dynamically scales the memory available for direct load tasks in the sorting phase based on the number of tasks.

• System log compression

  When the business traffic is heavy, system logs are refreshed quickly. In this case, troubleshooting will be affected due to a short retention period of system logs. The new version provides the system log compression feature. For the observer.log, rootservice.log, election.log, and trace.log log files, when the number of log files of a specific type reaches the value specified by syslog_file_uncompressed_count, the earliest logs will be compressed by using the compression method specified by syslog_compress_func. When the total occupied disk space approaches the upper limit specified by syslog_disk_size, the earliest log files are deleted to release the occupied disk space. After you enable zstd compression, the volume of logs that can be stored in the same disk space is 20 times that of logs that can be stored when compression is disabled.

• Cascading of read-only replicas by region

  OceanBase Database supports read-only replicas for weak-consistency reads and replicated tables since V4.2.0. A read-only replica synchronizes logs by registering as a full-featured replica or as a downstream replica of another read-only replica. When multiple read-only replicas and their upstream replicas are deployed in different regions, cross-region network bandwidth resources may be excessively occupied. OceanBase Database V4.3.1 enhances the capability to detect the regions of read-only replicas. During log synchronization, the database will preferentially select another replica in the same region as its upstream replica to avoid network transmission across regions, thereby saving the cross-region bandwidth.

• Compression of temporary results of SQL queries

  When an SQL query involves a large amount of data, the memory may be insufficient. In this case, the temporary intermediate results of some operators must be materialized. The execution of the SQL query fails if the disk space is fully occupied by the materialized data. OceanBase Database V4.3.1 supports compressing temporary results of SQL queries. You can use the tenant-level parameter spill_compression_codec or an SQL-level hint such as /*+opt_param('spill_compression_codec', 'lz4') */, to specify whether to compress temporary results and the compression algorithm. This feature effectively reduces the disk space occupied temporarily, so as to support query tasks with higher computing workload.

Security enhancements

• PL privilege management in MySQL mode

  OceanBase Database provides the PL privilege management feature in MySQL mode for you to manage privileges such as CREATE ROUTINE, EXECUTE, and ALTER ROUTINE. The mysql.procs_priv internal table is provided to show authorization information about stored procedures and functions. By default, the PL privilege management feature is disabled for clusters upgraded from earlier versions to V4.3.1, and is enabled for new clusters created in V4.3.1.

• Role management

  OceanBase Database V4.3.1 supports the role management feature of MySQL 8.0. You can manage and maintain a group of privileges based on roles to conveniently grant privileges to and revoke privileges from a specific type of users. You can grant privileges or other roles to or revoke privileges or other roles from a role like normal users. You can grant multiple roles to a user. However, the user has only the privileges of roles in the active state.

• Column-level privileges

  OceanBase Database V4.3.1 provides the column-level privilege management feature in MySQL mode. You can specify whether a user has the SELECT, INSERT, or UPDATE privilege on specific columns in a table.

• Operating system configuration check at startup

  Inappropriate operating system configurations can cause system issues. OceanBase Database V4.3.1 supports checking core operating system parameters when an OBServer node starts. Loose and strict check modes are supported. In loose mode, when any parameter does not meet the requirement, a warning is logged and the OBServer node can still be started. In strict mode, when any parameter does not meet the requirement, an error is reported and the OBServer node cannot be started.

Usability improvements

• Resource specification estimation

  Resources in a database are classified into logical resources and physical resources. Logical resources are entities corresponding to logical concepts, such as data structures, threads, locks, and sessions. Physical resources are hardware resources, such as CPU cores, disk space, and memory space. The amounts of logical resources that can be created for a tenant may be subject to one or more physical resources. OceanBase Database V4.3.1 provides the resource specification estimation feature for you to conveniently learn about the usage of physical resources corresponding to the current logical resources of a cluster, thereby planning scaling, node replacement, standby tenant creation, and other operations in a more reliable manner. The following dynamic views and packages are provided:

  • The [G]V$OB_TENANT_RESOURCE_LIMIT view shows the following information about the logical resources in each unit of a tenant: current usage, upper limit, effective conditions, and maximum usage after a restart upon crash.
  • The [G]V$OB_TENANT_RESOURCE_LIMIT_DETAIL view shows the details of physical resources or parameters that limit the logical resources created on a server for a tenant.
  • The DBMS_OB_LIMIT_CALCULATOR.CALCULATE_MIN_PHY_RES_NEEDED_BY_UNIT subprogram calculates the minimum amounts of physical resources required on a node for a tenant.
  • The DBMS_OB_LIMIT_CALCULATOR.CALCULATE_MIN_PHY_RES_NEEDED_BY_LOGIC_RES subprogram calculates the amounts of physical resources required for specific types and quantities of logical resources.
  • The DBMS_OB_LIMIT_CALCULATOR.CALCULATE_MIN_PHY_RES_NEEDED_BY_STANDBY_TENANT subprogram calculates the minimum amounts of physical resources required to create a standby tenant with a specified number of units for a primary tenant.

• Display of the backup progress

  In earlier versions of OceanBase Database, backup tasks are executed in black-box mode. Generally, it takes a long time to execute a backup task that involves a large amount of data. However, you cannot learn about the backup progress or estimated completion time. The new version provides the backup progress statistic feature and supports displaying the data backup progress and supplemental log backup progress. You can query the DATA_PROGRESS column in the CDB_OB_BACKUP_TASKS or DBA_OB_BACKUP_TASKS view for the macro block-level data backup progress, and query the LOG_PROGRESS column for the supplemental log backup progress.

• Backup of snapshot table names

  When OceanBase Cloud Platform (OCP) and downstream vendors adapt to the table-level restore feature, tables that can be used for restore must be displayed to users. OceanBase Database V4.3.1 persists the names of corresponding snapshot tables in the backup set and provides the ob_admin tool for parsing the snapshot table names.

• Manual partition transfer

  The existing automatic load balancing mechanism of OceanBase Database can automatically adjust the distribution of partitions to implement online scaling and partition balancing. However, you may want to aggregate or scatter certain partitions in actual business scenarios. OceanBase Database V4.3.1 provides the manual partition transfer feature. You can transfer specific partitions to a specific log stream. You can also view the status of and cancel a partition transfer task.

• Binding of both an execution plan and a throttling rule to an outline

  In earlier versions, you can bind an execution plan or a throttling rule to an outline, but not both. To cope with scenarios where an execution plan must be interfered with and an SQL statement must be throttled, OceanBase Database V4.3.1 allows you to specify MAX_CONCURRENT() and other hints in an outline creation statement. You cannot bind an execution plan by using the question mark (?) as a wildcard in sql_text. When you bind both an execution plan and a throttling rule to the same outline, the same restriction applies. The behavior of binding a specific SQL statement by using its SQL ID to an execution plan or other database objects (such as a throttling rule) remains the same.

• Allowlists and blocklists in OceanBase CDC

  OceanBase CDC supports tenant-level log synchronization in earlier versions of OceanBase Database, and also supports database-level synchronization and table-level synchronization in V4.3.1. In data consumption scenarios where you want to synchronize only part of the tables, you can configure an allowlist or a blocklist by using a simple regular expression.

• Decoupling of PL logs from SQL logs

  In earlier versions, an SQL statement executed by using PL inherits the trace ID of PL. As a result, one trace ID may be associated with a large number of logs, and it takes a long time to locate and resolve issues based on logs. OceanBase Database V4.3.1 decouples PL logs from SQL logs. It assigns independent trace IDs to SQL statements in PL procedures and adds records about external PL trace IDs to SQL AUDIT logs to improve the troubleshooting efficiency.

• PL execution time statistics

  The PL execution performance may fail to meet the expectations in business scenarios. In OceanBase Database of earlier versions, no easy method is available for you to quickly analyze whether a large amount of time is consumed in internal SQL statements or PL structures. OceanBase Database V4.3.1 supports PL structure execution time statistics. You can query the PLSQL_EXEC_TIME column in the [G]V$OB_SQL_AUDIT view for these statistics.

• Support of IPv6 for OBServer nodes

  The new version supports IPv6 addresses for OBServer nodes. SQL clients and RPC clients can connect to OBServer nodes through IPv6 addresses. It also supports hybrid deployment of OBServer nodes with an IPv4 address and those with an IPv6 address in the same cluster. You can upgrade an IPv4 cluster to the new version but cannot upgrade an IPv4 cluster to an IPv6 cluster of the new version.

Compatibility changes

Product behavioral changes

The following table describes the changes made in this version.

Feature	Description
Output of the SHOW PARAMETERS command	In earlier versions, the SHOW PARAMETERS command returns all parameters, including hidden parameters. In V4.3.1, the command returns only non-hidden parameters and hidden parameters with non-default values. The DEFAULT_VALUE and ISDEFAULT columns are added to show the default value of a parameter and whether the current value is the default value.
Log size	When the amount of data in a single row is large, the space for a single log is insufficient. To address this issue, OceanBase Database extends the size of a single log to 3.5 MB since V4.3.1. You can use the ob_admin tool to view the clogs.

View changes

The following table describes the changes made in this version.

View	Change type	Description
CDB_OB_BACKUP_TASKS	Modified	The DATA_PROGRESS, LOG_FILE_COUNT, FINISH_LOG_FILE_COUNT, and LOG_PROGRESS columns are added for recording the data backup progress and supplemental log backup progress.
DBA_OB_BACKUP_TASKS	Modified	The DATA_PROGRESS, LOG_FILE_COUNT, FINISH_LOG_FILE_COUNT, and LOG_PROGRESS columns are added for recording the data backup progress and supplemental log backup progress.
[G]V$OB_TENANT_RESOURCE_LIMIT	New	Displays the resources such as log streams and tablets on an OBServer node for each tenant. You can query the information about all tenants from the sys tenant, and the information only about the current tenant from a user tenant.
[G]V$OB_TENANT_RESOURCE_LIMIT_DETAIL	New	Displays the resource limits on an OBServer node for each tenant, such as the number of log streams, tenant memory size, and clog disk size. You can query the information about all tenants from the sys tenant, and the information only about the current tenant from a user tenant.
[G]V$OB_SESSION	New	Displays information about the current session.
[G]V$OB_PROCESSLIST	Modified	The TOTAL_CPU_TIME column is added to show the CPU time.
mysql.role_edges	New	Displays the relationships between roles and users who are granted the roles. This view is applicable in MySQL mode.
mysql.default_roles	New	Displays the roles that are enabled for users by default. This view is applicable in MySQL mode.
mysql.columns_priv	New	Displays the column-level privileges of users. This view is applicable in MySQL mode.
DBA_OB_TRANSFER_PARTITION_TASKS	New	Displays all ongoing partition transfer tasks in the current tenant.
CDB_OB_TRANSFER_PARTITION_TASKS	New	Displays the ongoing partition transfer tasks of all tenants. You can query this view only in the sys tenant.
DBA_OB_TRANSFER_PARTITION_TASK_HISTORY	New	Displays all historical partition transfer tasks executed in the current tenant.
CDB_OB_TRANSFER_PARTITION_TASK_HISTORY	New	Displays the historical partition transfer tasks executed in all tenants. You can query this view only in the sys tenant.
[G]V$OB_PARAMETERS	Modified	The DEFAULT_VALUE and ISDEFAULT columns are added to show the default value of the parameter and whether the current value is the default value.
DBA_OB_SYS_VARIABLES	New	Displays values of system variables of the current tenant.
CDB_OB_SYS_VARIABLES	Modified	The DEFAULT_VALUE and ISDEFAULT columns are added to show the default value of the system variable and whether the current value is the default value.
[GV$OB_PL_CACHE_OBJECT]	New	Displays information about cached objects in the PL cache.
[G]V$OB_SQL_AUDIT	Modified	The PL_TRACE_ID column is added to show the outer PL trace ID for association with SQL statements in PL procedures. The PLSQL_EXEC_TIME column is added to show the PL execution time in μs, excluding the SQL execution time.
[G]V$SQL_WORKAREA	Modified	The DB_ID column is added to show the ID of the database to which the request connection belongs.

Parameter changes

The following table describes the changes made in this version.

Parameter	Change type	Description
syslog_disk_size	New	The total disk space available for system logs. It is a cluster-level parameter. The default value is 0M, indicating that the entire disk is available for system logs, which is the same as in earlier versions.
syslog_compress_func	New	The compression algorithm for system log files. Valid values are none, zlib_1.0, zstd_1.0, and zstd_1.3.8. It is a cluster-level parameter. The default value is none, which specifies not to compress system log files.
syslog_file_uncompressed_count	New	The number of system log files that triggers log compression for a specific log type. It is a cluster-level parameter. The default value is 0.
json_document_max_depth	New	The maximum nesting depth allowed for a JSON document. It is a tenant-level parameter. The default value is 100.
ha_diagnose_history_recycle_interval	New	It is a new cluster-level parameter that specifies the interval for clearing historical transfer diagnostic statistics. The default value is 7 days.
data_disk_write_limit_percentage	New	The data disk usage that triggers an error for user write requests. When the specified value is reached, you can drop tables to urgently release storage space to avoid cluster faults. It is a cluster-level parameter. The value of this parameter must be greater than that of data_disk_usage_limit_percentage. We recommend that you set this parameter to the value calculated by the following formula: (1 - memstore_limit_size/data_disk_size) × 100%.The default value is 0, which specifies not to stop user write requests.
strict_check_os_params	New	It is a new startup option that specifies whether to check operating system parameters in strict mode at startup. The default value is False. When operating system parameters do not meet the requirements, a warning is displayed without affecting the normal start of the OBServer node.
log_storage_compress_all	New	Specifies whether to enable clog compression for storage. It is a tenant-level parameter. The default value is False, which specifies not to enable clog compression for storage.
log_storage_compress_func	New	The compression algorithm for clogs. Valid values are lz4_1.0, zstd_1.0, and zstd_1.3.8. It is a tenant-level parameter. The default value is lz4_1.0.
spill_compression_codec	New	The compression algorithm for operators whose temporary results need to be materialized. It is a tenant-level hidden parameter. The default value is none, which specifies not to compress the temporary results of operators.

System variable changes

The following table describes the changes made in this version.

System variable	Change type	Description
QUERY_REWRITE_ENABLED	New	Specifies whether to enable query rewrite based on materialized views. This variable is available both at the global and session levels. The default value is False, which specifies to disable the feature.
QUERY_REWRITE_INTEGRITY	New	Specifies whether to rewrite queries based on real-time or non-real-time materialized views. This variable is available both at the global and session levels. When QUERY_REWRITE_INTEGRITY is set to enforced, queries are rewritten only based on the real-time materialized view. After the rewrite, the real-time materialized view is expanded to ensure that the result of the rewritten query is the same as that of the original query. When the system variable is set to stale_tolerated, queries can be rewritten based on the non-real-time materialized view. After the rewrite, the query result may be different from that of the original query. In this case, the real-time materialized view is not expanded even if you rewrite the query based on the real-time materialized view.The default value is enforced.
activate_all_roles_on_login	New	Specifies whether to activate all roles of a user when the user logs on. It is a global-level system variable available only in MySQL tenants.
lc_time_names	New	The language in which the date and month names and their abbreviations are displayed. This variable is available both at the global and session levels. The default value is en_US.

Function/PL package changes

Function/PL package	Change type	Description
CURRENT_ROLE	New	Displays the roles activated in the current session. This function is applicable in MySQL mode.
GET_LOCK	New	Obtains the specified lock. This function is applicable in MySQL mode.
IS_FREE_LOCK	New	Checks whether the specified lock is idle. This function is applicable in MySQL mode.
IS_USED_LOCK	New	Checks whether the specified lock is in use, and returns the ID of the connection that holds the lock, if the lock is in use. This function is applicable in MySQL mode.
RELEASE_ALL_LOCKS	New	Releases all user locks. This function is applicable in MySQL mode.
RELEASE_LOCK	New	Releases the specified lock. This function is applicable in MySQL mode.
DBMS_OB_LIMIT_CALCULATOR	New	Calculates the minimum amounts of physical resources required in different scenarios. This package is available in the sys tenant.

Syntax changes

• Syntaxes related to full-text indexes are provided.
• Syntaxes related to external table partitions are provided.
• Export options are added for the SELECT INTO OUTFILE statement.
• The table-level restore command is adjusted.
• The syntax for creating a materialized view with a primary key is provided.
• Syntaxes for creating real-time materialized views or rewriteable materialized views are provided.
• The syntax for exchanging partitions is provided.
• Syntaxes for manually transferring partitions and canceling a partition transfer task are provided.
• Syntaxes related to lateral derived tables are provided.
• Syntaxes related to JSON multi-valued indexes are provided.

Recommended versions of tools

The following table lists the recommended versions of tools for OceanBase Database V4.3.1_CE.

Tool	Version
ODP	V4.2.3
OCP	V4.2.2_CE_HF1
OBD	V2.8.0
ODC	V4.2.4 BP2
OBCDC	V4.3.1_CE_BETA
OMS	V4.2.3_CE
OBClient	V2.2.3
LibOBClient	V2.2.3

Upgrade notes

• You can upgrade OceanBase Database V4.3.0_CE Beta to V4.3.1 Beta_CE.
• At present, you cannot upgrade OceanBase Database V4.2.x or earlier to V4.3.1_CE Beta. The upgrade paths from V4.2.x to V4.3.x will be supported later.

版本信息

项目	描述
发布日期	2024-05-22
版本号	V4.3.1_CE_BETA
Commit 号	bad90e8
RPM 版本号	oceanbase-ce-4.3.1.0-100000032024051615
版本说明	Beta 版本解决了大部分缺陷，并趋于稳定。推荐测试环境使用。

版本概览

OceanBase V4.3.1 在 V4.3.0 基础上，新增全文索引特性提升文档检索效率，扩展实时物化视图、物化视图改写、主键物化视图等功能以满足多场景的分析需求。引入分区管理新机制，如分区交换、外表分区、MySQL 模式分区键数据类型扩展等，提升大规模数据的处理能力。多模特性（含 JSON、XML、GIS）功能升级，增加 JSON 多值索引、JSON 部分更新的能力支持，促进异构数据的迁移与融合。MySQL 兼容性持续增强，支持 Lateral Derived Tables、 MySQL 锁函数等，以便融入生态。提供增量旁路导入能力，提升了多次导入场景的入库性能，同时优化了多局部索引场景下的 DML 性能，提高了基础统计信息收集效率，并在行采样、小规格 TP 场景取得了显著的性能提升。新版本也进一步优化了资源使用，支持了 CLOG 日志缓存、SQL 临时结果压缩、系统日志压缩等特性。补充完善了 MySQL 权限体系、支持了操作系统配置检查，加固系统安全。一如既往地关注用户体验，增加资源规格估算能力，增强备份透明度，提供 IPV6 格式支持，赋能数据库管理与运维。

V4.3.1 定位为 Beta 版本，推荐实时数仓或联机交易业务测试使用，下半年将发布推荐用于生产的 GA 版本。

关键特性说明

内核增强

• MySQL 模式全文索引（Experimental）

  关系型数据库中通常会使用索引来加速基于精准值匹配的查询，但普通的 B-Tree 索引无法应用于涉及大量文本数据需要进行模糊检索的场景，这时只能通过全表扫描来对每一行数据进行模糊查询，文本较大、数据量较多的情况下性能往往不能满足要求。另外一些复杂的查询场景，如近似匹配、相关性排序等，也难以通过改写 SQL 支撑。

  为了解决上述问题，OceanBase 在 V4.3.1 支持了全文索引功能，通过预先处理文本内容，建立关键词索引，有效提升全文检索效率。该功能在 V4.3.1 定义为实验特性，后续版本会继续扩展功能并演进为生产可用特性。本期包含以下功能：

  • MySQL 模式下支持全文索引功能，兼容 MySQL 基本语法。
  • 建表时支持对 CHAR、VARCHAR 和 TEXT 列预建全文索引。
  • 适用于分区表。
  • 支持对主表创建多个全文索引。
  • 支持 SPACE（空格）、NGRAM 和 BENG（基础英文）三种内置分词器。
  • 支持一个 match against 对多个列进行全文检索。
  • 支持 NATURAL LANGUAGE MODE 模式。

• 物化视图增强

  OceanBase V4.3.0 支持了物化视图特性，通过预计算和存储视图的查询结果，减少实时计算来提升查询性能，简化复杂查询逻辑，支撑 AP 场景需求。在此基础上，V4.3.1 版本扩展支持了实时物化视图功能，提供基于物化视图和 MLOG 两部分数据的实时计算能力，适用于实时性要求较高的分析业务。新增物化视图主键约束，支持用户为物化视图指定主键，以此优化基于主键的单行查找、范围查询或关联场景性能。同时扩展支持了内连接场景物化视图的增量更新能力，提升部分场景的物化视图刷新性能。

  另外在 V4.3.0 版本使用物化视图时，需要将业务脚本中对原始表的访问计算手工替换为查询对应的物化视图，引入了人工改写成本。新版本支持了部分场景的物化视图改写能力，在系统变量 QUERY_REWRITE_ENABLED 设置为 True 的情况下，创建物化视图时可以通过指定 ENABLE QUERY REWRITE 来使用物化视图的自动改写能力，此时系统可以将对原始表的查询改写为针对物化视图的查询，以此减少业务改造量。

• 分区交换

  随着时间的推移，表中可能会累积大量的历史数据，这些数据可能不需要频繁访问，但需要保留以满足合规性或历史数据分析需求。为了提升查询性能，业务往往需要区分活跃和非活跃数据，并将非活跃数据进行归档。通过 SQL 迁移数据到新表可以解决这个场景需求，但大数据量情况下性能往往不优。因此，OceanBase V4.3.1 版本新增分区交换功能，通过修改数据字典中分区和表的定义，而无需进行数据的物理复制，可以将 A 表数据近乎瞬时地移动到 B 表某个分区下，极大地提升了数据迁移的性能。本期包含以下功能：

  • 支持分区表中的一个分区和一个普通表进行数据互换。
  • 支持一级分区表，分区类型为 Range（Range Columns）。
  • 支持二级分区表，一级分区类型无要求，二级分区类型为 Range（Range Columns）。
  • 支持 INCLUDING INDEXES 的行为，即分区交换时，对应局部索引也会参与交换，并在交换后保持可用状态。
  • 支持 WITHOUT VALIDATION 模式，需要用户保证数据符合分区键范围。

• 外部表分区

  OceanBase V4.2.0 开始支持外部表功能，局限于非分区表。当文件数量较多，但某次查询理论上只需要扫描一部分数据文件的场景，非分区外部表只能扫描全量文件，无法进行裁剪，性能较差。V4.3.1 版本新增外部表分区功能，支持类似普通表 list 分区的分区方式，并提供了自动/手动分区的两种语法。指定自动创建分区时，系统将按照分区键的定义方式，将文件按照分区进行分组；指定手动创建分区时，需要用户指定每个分区对应的数据文件子路径。这种情况下，外表查询可以根据分区条件实现分区裁剪，减少扫描的文件数量，显著提升查询性能。

• MySQL 模式 Range Columns 分区键类型扩展

  为了满足不同业务的分区要求，OceanBase V4.3.1 在兼容 MySQL Range Columns 分区键类型的基础上，扩展支持了 double、float、decimal、timestamp 等类型作为 Range Columns 分区的分区键。具体类型包括：

  • DECIMAL、DECIMAL[(M[,D])]
  • DEC、NUMERIC、FIXED
  • FLOAT[(M,D)]、FLOAT(p)
  • DOUBLE、DOUBLE[(M,D)]
  • DOUBLE PRECISION、REAL
  • TIMESTAMP

• PL 重新编译逻辑优化

  存储过程编译为共享库后，可以提供给多个线程使用。但如果发生依赖对象变更，共享库可能失效需要重新编译。V4.3.1 版本针对重新编译场景做了梳理细化，在临时表匹配、静态 SQL 依赖对象信息收集、表 DDL 变更等方面进行一系列逻辑优化，减少因 PL CACHE 缓存对象失效导致重新编译的场景。

• PL 编译结果落盘

  当前 PL Function/Procedure 编译后会加入到 PL Cache 中，当内存压力过大时可能会被淘汰，重启后会丢失编译结果，并且在分布式场景下也需要重新编译。在这些情况下，PL 无法命中 PL Cache，进而触发 LLVM 编译，导致消耗部分 CPU 资源。V4.3.1 开始，PL Function/Procedure 被加入系统表落盘保存，未触发 DDL 使 PL 缓存失效的情况下，无论重启还是分布式场景都可以只编译一次后复用缓存。

多模特性

• MySQL 模式 JSON 多值索引（Experimental）

  MySQL 8.0 版本开始支持多值索引特性，适用于 JSON 文档和其他集合数据类型。这使得用户可以通过在数组或者集合上构建索引，来实现元素的高效检索。OceanBase V4.3.1 在 MySQL 模式下兼容了 JSON 多值索引特性，允许在包含多个元素的 JSON 数组字段上创建高效的二级索引，增强了复杂 JSON 数据结构的查询能力，兼顾了数据模型的灵活性和数据查询的高性能。该功能在 V4.3.1 定义为实验特性，后续版本会继续扩展功能并演进为生产可用特性。本期包含以下功能：

  • 支持多值索引、复合多值索引的预创建。
  • 支持唯一、非唯一的多值索引。
  • 支持创建多值索引的 JSON 数组元素类型为 INT、UINT、DOUBLE、FLOAT、CHAR 等。
  • 适用于分区表。
  • 支持 MEMBER_OF()、JSON_CONTAINS() 和 JSON_OVERLAPS() 函数使用多值索引查询。

• MySQL JSON

  新增兼容 JSON_SCHEMA_VALID、JSON_SCHEMA_VALIDATION_REPORT 和 JSON_ARRAY_APPEND 表达式。

• MySQL JSON Partial Update

  一些用户会使用 JSON 文档存储业务数据，文档需要更新时，目前只能全量读取后全量更新，文档较大时性能无法满足预期。V4.3.1 版本新增支持 JSON Partial Update 功能，当用户通过特定表达式（json_set、json_replace 和 json_remove）更新 JSON 文档的部分字段时，可以只更新要修改的部分，无需全量覆盖更新，提升更新性能。该功能需要通过 log_row_value_options 配置项开启使用。

• MySQL GIS 增强

  OceanBase V4.1 版本支持了 MySQL GIS 数据类型及部分空间对象相关的表达式，V4.3.1 版本在此基础上进一步补齐空间数据存储和计算分析的能力，新增支持 MySQL 兼容的 ST_Crosses、ST_Overlaps、ST_Difference、ST_Union、ST_SymDifference、ST_Length、ST_Centroid 和 ST_AsGeoJSON 等表达式。另外 PG 作为 GIS 行业使用最广的数据库，提供了部分区别于 MySQL 的常用表达式，OceanBase 在新版本也进行了扩展补充，包括 _ST_Touches、_ST_Equals、_ST_MakeEnvelope、_ST_ClipByBox2D、_ST_GeometryType、_ST_IsCollection、_ST_NumInteriorRings、_ST_PointOnSurface、ST_AsMVTGeom 和 _ST_AsMVT 等，同时也支持了三维空间对象的存储能力。

• MySQL XML

  新增兼容 ExtractValue、UpdateXML 表达式。

MySQL 兼容

• Lateral Derived Tables

  Lateral Derived Tables 是一种特殊的 Derived Tables，可以引用同一 FROM 子句中前面表的列，这种用法下 SQL 更易读易写，往往也可以减少数据的重复扫描计算，提升 SQL 执行性能。V4.3.1 兼容了 MySQL 8.0 的用法。

• 通信协议完善

  兼容 MySQL COM_SET_OPTION 通信协议命令，用于指定 CONNECTION 级别选项，如 MYSQL_OPTION_MULTI_STATEMENTS_ON、MYSQL_OPTION_MULTI_STATEMENTS_OFF。兼容 MySQL AuthSwitchRequest 协议，用于支持认证方式切换，解决部分客户端版本不匹配导致认证出错的问题。

• Show Extended Tables/Columns/Index

  支持 MySQL 8.0 新增的 SHOW EXTENDED 语法，如 SHOW EXTENDED TABLES、SHOW EXTENDED COLUMNS 和 SHOW EXTENDED INDEX。

• MySQL 锁函数

  V4.3.1 兼容了 MySQL GET_LOCK、IS_FREE_LOCK、IS_USED_LOCK、RELEASE_ALL_LOCKS 和 RELEASE_LOCK 等锁相关的表达式。表达式可以用在 SQL 语句的多个地方，例如 SELECT 语句的 ORDER BY 和 HAVING 子语句，SELECT、DELETE 和 UPDATE 的 WHERE 条件中以及 SET 语句中。表达式的值可以是字面值、列值、NULL、变量、内部函数和操作符、可加载函数、存储函数。锁函数是一类内部函数，允许用户自定义锁，并进行使用。

• PS 协议支持存储过程出参

  MySQL 模式下，使用 PS 协议执行 CALL PROCEDURE 语句时，新增支持存储过程带出参的场景。

性能提升

• 增量旁路导入（Experimental）

  OceanBase V4.1.0 开始支持旁路导入特性，通过精简数据加载的执行路径，跳过 SQL、事务、memtable 等模块，直接将数据持久化为 SSTable 来显著提升数据导入效率。但在表数据需要多次导入的场景，每次导入都需要将表中已有数据重写一遍，影响了增量导入的性能。V4.3.1 针对性地优化了增量导入场景，增量旁路导入无需重写原有数据，只需处理新增数据，让多次导入像第一次导入一样具有高性能。支持在 LOAD DATA 和 INSERT INTO SELECT 语句中通过 /*+ direct(need_sort, max_errors_allowed, load_mode) */ Hint 指定是否使用增量旁路导入功能。不指定 load_mode 或 load_mode 为 full 时，表示使用原有的全量旁路导入方式；指定 load_mode 为 inc_replace 时使用增量旁路导入方式。该功能在 V4.3.1 定义为实验特性，后续版本会继续扩展功能并演进为生产可用特性。

• SELECT INTO OUTFILE 性能优化

  现有的 SELECT INTO OUTFILE 功能虽然支持并行读取数据表，但只能串行写入外部文件，存在数据导出的性能瓶颈。V4.3.1 增加并行导出能力，在 SELECT INTO OUTFILE 命令基础上增加了 SINGLE、MAX_FILE_SIZE 选项，用于控制数据写入外部文件的方式。SINGLE 选项可控制将数据导出到单个文件或多个文件，指定并行度大于 1 且 SINGLE = FALSE 时，可以将数据导出到多个文件，达到并行读并行写的效果；MAX_FILE_SIZE 选项可控制导出文件的大小。

• 多局部索引场景下 DML 性能优化

  当数据库表上存在索引时，DML 会因为需要同步更新索引而产生性能下降。在索引数量特别多时，性能下降尤其明显。新版本针对表上存在多个局部索引的场景，通过局部索引去表锁、非唯一索引不记录持锁者、非唯一索引不检查事务冲突、降低 DML 统计信息上报开销等等一系列系统优化，将局部索引维护开销降低了 45%，从而提升 DML 整体执行性能。

• 单日志流并行同步

  OceanBase V4.3.1 版本之前的日志同步模型为不同日志流并行同步和处理，单日志流基于 Pipeline 方式同步。目前在同城消费本地盘日志的场景下是可以满足性能要求的，但在备库异地部署、公有云读对象存储的场景下，性能相对会差一些。V4.3.1 版本实现了基于文件数据块的单日志流并行同步模型，显著提升同步性能并优化内存使用。

• 统计信息收集性能优化

  当前基础统计信息的收集依赖通过执行相关聚合函数的内部 SQL 完成，V4.3.0 开始支持了聚合函数下压到存储层，V4.3.1 进一步扩展了收集基础统计信息可以下压到存储层的聚合函数，避免投影数据到 SQL 层再做计算，基础统计信息的收集操作全部放到了存储层，提升了基础统计信息的收集效率。新版本收集性能整体上比 V4.3.0 提升 5% 左右。

• 行采样性能优化

  在处理全量数据开销太大时，往往可以通过小部分数据观察整体的数据分布，例如优化器通过采样来分析数据分布，辅助执行计划的生成。OceanBase sample 行采样当前会先应用 WHERE 过滤条件，然后过滤出的一行数据，判断是否进行采样。这相当于逐行扫描进行判断，把数据整体都探测一遍，十分耗时。V4.3.1 优化了行采样流程，先过滤数据再应用条件，省去部分数据的读取成本，同时针对列存也优化为只读取需要的列数据，显著提升采样性能。

• 小规格性能优化

  针对 4C/8C 小规格环境，V4.3.1 版本在后台线程使用、Location Cache 访问、读写主路径、系统调用等方面进行了一系列优化，OLTP 相关测试场景，性能相对 V4.3.0 提升 20%-30%。

可靠性提升

• 数据盘满停写

  数据盘使用量过高时，当前不会停写用户请求，一直到内存因为无法转储写满，或者 clog 盘写满后才会报错。这种情况下，需要通过紧急扩 clog 盘或租户内存恢复。新版本在内核层面增加数据盘满停写功能，当用户数据盘水位线达到 data_disk_write_limit_percentage 配置后，用户写入请求会报错。通过删表、transfer 或者扩磁盘方式降低数据盘使用比例后，用户写入操作可以自动恢复。

资源使用优化

• CLOG 日志缓存

  V4.x 在归档、回放等场景已经支持了 LogHotCache，用来缓存部分实时日志，一定程度上避免了日志读盘开销。但当日志写入速度比较快或多个 OBCDC 重复拉取相近日志时，还是无法避免大量的日志读盘，极端情况下会将日志盘带宽打满。同时，云上磁盘带宽往往有限的，我们需要通过降低日志读盘的开销来支撑更多的日志消费场景。因此，该版本新增 CLOG 日志缓存功能，支持消费者直接从日志缓存读取日志进行消费，没有命中日志缓存时，依然支持从磁盘读取日志，并将日志同步写入缓存中，避免重复读取磁盘，降低日志盘带宽使用。为了方便用户监控日志缓存命中情况，SYSSTAT 新增 50065、50066 和 120010 三个统计项，用于展示租户级别的日志缓存命中次数、未命中次数和 CLOG 缓存大小。

• 旁路导入资源控制强化

  OceanBase V4.x 版本支持的旁路导入功能显著提高了数据导入速率，但在用户设定较高并行度且并发执行的旁路导入任务较多时，资源缺少严格控制，容易造成较多线程和内存占用，影响其他任务的正常执行。V4.3.1 新增三个维度的旁路导入资源管理能力：

  • 新增任务级资源申请管理模块，根据导入任务的执行模式和分区数量，申请对应的线程和内存资源。
  • 新增租户级资源管理模块，管理租户用于旁路导入的线程和内存资源，以定时任务感知资源池变化，回收异常中断的导入任务申请的资源。
  • 新增 OBServer 级资源管理模块，记录节点级资源申请，并根据任务数动态伸缩旁路导入任务排序阶段的可用内存。

• 系统日志压缩

  业务流量过大时，系统日志刷新的会比较快，保留时间较短可能影响问题排查。新版本增加系统日志压缩功能，支持对 observer.log、rootservice.log、election.log 和 trace.log 等日志文件，每类超过 syslog_file_uncompressed_count 个数时，采用 syslog_compress_func 设置的压缩方法，对最早日志进行压缩。当日志使用总空间接近 syslog_disk_size 设置的磁盘空间上限时，开始删除最早生成的日志文件，回收空间。磁盘空间不变的情况下，开启 zstd 压缩后，预计可以存储不开启压缩时 20 倍的日志量。

• R 副本按 Region 级联

  OceanBase V4.2.0 开始支持 R 副本功能，服务于弱读、复制表等场景。一个 R 副本通过注册为 F 副本或其他 R 副本的下游来同步日志，当多个 R 副本和其上游被部署在不同的 Region 时，可能占用额外的跨 Region 网络带宽。V4.3.1 增加 R 副本对 Region 的感知能力，在日志同步时会尽量选择相同 Region 的其他副本作为上游，尽量避免跨 Region 的网络传输，节省跨 Region 带宽。

• SQL 临时结果压缩

  SQL 执行涉及的数据量过大的情况下，可能出现内存不足的问题，这时部分算子需要物化临时的中间结果。当物化的数据量过大，磁盘空间写满时，会导致 SQL 执行失败。V4.3.1 新增 SQL 临时结果压缩功能，允许通过租户级配置项 spill_compression_codec 或 SQL 级 Hint（如 /*+opt_param('spill_compression_codec', 'lz4') */）指定临时结果是否压缩及压缩算法，当指定临时结果压缩时可有效降低临时磁盘空间占用，以支撑更大运算量的查询任务。

安全强化

• MySQL PL 权限管理

  MySQL 模式下新增 PL 权限管理能力，支持 CREATE ROUTINE、EXECUTE 和 ALTER ROUTINE 等权限控制。增加 mysql.procs_priv 内部表，用于展示存储过程或函数授权信息。升级集群默认不开启，新建集群会自动开启。

• MySQL 角色

  OceanBase V4.3.1 兼容了 MySQL 8.0 的角色管理功能，通过角色来管理维护一组权限，可以更方便地对某一类用户进行授权和回收。与普通用户类似，角色可以被授予或回收权限，也可以被授予或回收其他角色。用户可以被授予多个角色，但仅能使用激活状态角色中的权限。

• MySQL 列级权限

  V4.3.1 版本新增 MySQL 列级权限功能，可以用于控制用户是否有权限对某张表的某几列进行 SELECT、INSERT 或 UPDATE。

• OBServer 启动检查 OS 配置

  不合理的 OS 配置可能引发系统问题，V4.3.1 版本新增核心 OS 参数检查。提供宽松和严格检查两种模式。宽松模式下，OS 参数不符合要求时，日志报 warning ，不影响 OBServer 启动；严格模式下，OS 参数不符合要求时，报 error 且 OBServer 无法启动。

易用性提升

• 资源规格估算：

  我们将数据库中的资源分为逻辑资源和物理资源两大类。逻辑资源指的是逻辑概念对应的实体，如数据结构、线程、锁、会话等；物理资源指的是硬件资源，如：CPU、磁盘、内存等。租户能够创建的逻辑资源量可能受一个或者多个物理资源限制。为了用户可以更方便地获取集群当前的逻辑资源对应的物理资源使用信息，以此更可靠地规划扩缩容、节点替换、备租户创建等操作，V4.3.1 新增资源规格估算功能，提供以下一系列动态视图、系统包：

  • 新增 [G]V$OB_TENANT_RESOURCE_LIMIT 视图，用于展示租户在每个 Unit 上的逻辑资源使用量、上限值、生效限制条件和宕机重启后的最大占用量。
  • 新增 [G]V$OB_TENANT_RESOURCE_LIMIT_DETAIL 视图，用于展示租户在一个机器上能创建的逻辑资源所受的物理资源或配置项的限制情况。
  • 新增 DBMS_OB_LIMIT_CALCULATOR.CALCULATE_MIN_PHY_RES_NEEDED_BY_UNIT 子程序，用于计算某个租户在某个节点上所需的最小物理资源量。
  • 新增 DBMS_OB_LIMIT_CALCULATOR.CALCULATE_MIN_PHY_RES_NEEDED_BY_LOGIC_RES 子程序，用于计算特定逻辑资源类型和数量需要的物理资源量。
  • 新增 DBMS_OB_LIMIT_CALCULATOR.CALCULATE_MIN_PHY_RES_NEEDED_BY_STANDBY_TENANT 子程序，用于计算指定主租户创建指定 Unit 个数的备租户需要的最小物理资源量。

• 备份进度展示

  OceanBase 早期版本备份任务黑盒进行，大数据量的备份任务往往需要执行较长时间，用户无法了解备份进度，无法感知预计完成时间。新版本增加备份进度统计功能，支持数据备份进度和补偿日志备份进度展示。用户可通过查询 CDB/DBA_OB_BACKUP_TASKS 视图的 DATA_PROGRESS 字段获取宏块级别的数据备份进度，查询 LOG_PROGRESS 字段获取补偿日志备份进度。

• 备份快照表名

  OCP 及下游厂商适配表级恢复功能时，需要向用户展示可供恢复的表。新版本在备份集中持久化了对应的快照表名，并提供通过 ob_admin 来解析快照表名的能力。

• 手动 Transfer 分区

  OceanBase 现有的自动负载均衡机制可以自动调整分区分布，以达到在线扩缩容和分区均衡的目的。不过实际业务场景中，用户有定制数据分布的需求，希望将特定的分区进行聚合或打散。V4.3.1 版本增加手动 Transfer 分区功能，用户可以选择将特定的分区迁移到特定的日志流上，并提供任务状态查看和取消能力。

• 执行计划和限流同时绑定

  目前 OUTLINE 支持执行计划绑定和限流绑定两种功能，但不支持同时指定。考虑部分客户场景既需要干预执行计划，又需要对某条 SQL 限流的需求，V4.3.1 版本支持用户在一条创建 OUTLINE 的语句中指定 max_concurrent() 和其他 Hint。因为执行计划不支持使用 sql_text 指定通配符 ？ 绑定，执行计划和限流同时绑定时也约束为同样行为。sql_id 绑定方式未发生变化。

• OBCDC 黑白名单

  OBCDC 目前已具备租户级别的日志同步功能，V4.3.1 增加库、表级同步粒度。支持通过简单的正则表达式配置黑白名单，来满足用户只需要同步部分表的数据消费场景。

• PL & SQL 日志解耦

  当前通过 PL 执行的 SQL 语句复用了 PL 的 trace_id，导致同一个 trace_id 关联的日志量过大，通过过滤日志排查问题比较耗费时间。V4.3.1 版本将 PL 和 SQL 的日志解耦，为 PL 内部的 SQL 语句赋予独立的 trace_id，并在 SQL AUDIT 增加外层 PL trace_id 记录，有效提高问题排查效率。

• PL 执行时间统计

  业务场景中，可能会遇到 PL 执行性能不符合预期的情况，目前缺少易用的手段快速分析主要耗时在内部 SQL 还是 PL 结构本身。V4.3.1 版本增加 PL 结构执行时间统计，可直接通过 [G]V$OB_SQL_AUDIT 视图的 PLSQL_EXEC_TIME 列来获取。

• OBServer 支持 IPV6

  新版本支持 IPV6 格式的 server ip，支持 sql 客户端、rpc 客户端以 IPV6 地址连接，同时也支持了 IPV4 和 IPV6 节点的混合部署。支持 IPV4 集群升级，但不支持旧的 IPV4 类型的集群升级到 IPV6 类型。

兼容性变更

产品行为变更

新增如下变更：

功能	变更说明
SHOW PARAMETERS 展示内容变更	老版本 SHOW PARAMETERS 会展示包含隐藏配置项在内的所有参数，V4.3.1 对 SHOW 的展示规则做了调整，仅展示非隐藏参数 + 非默认值的隐藏参数。同时也增加了 default_value 和 isdefault 两列用于展示配置项默认值和当前设置是否为默认值的信息。
单条日志大小扩展为 3.5 MB	为解决单行数据量较大情况下，单条日志空间不足的问题，V4.3.1 将单条日志大小扩展为 3.5 MB。需使用配套的 ob_admin 工具来查看 CLOG 内容。

视图变更

新增如下变更：

视图	变更类型	变更说明
CDB_OB_BACKUP_TASKS	新增列	新增 DATA_PROGRESS、LOG_FILE_COUNT、FINISH_LOG_FILE_COUNT、LOG_PROGRESS 列用于记录数据和补偿日志的备份进度。
DBA_OB_BACKUP_TASKS	新增列	新增 DATA_PROGRESS、LOG_FILE_COUNT、FINISH_LOG_FILE_COUNT、LOG_PROGRESS 列用于记录数据和补偿日志的备份进度。
[G]V$OB_TENANT_RESOURCE_LIMIT	新增	用于展示 OBServer 上每个租户的资源情况，资源包括：日志流、tablet 等。系统租户可以查看所有租户信息，普通租户只能查看本租户信息。
[G]V$OB_TENANT_RESOURCE_LIMIT_DETAIL	新增	用于展示 OBServer 上每个租户各种资源的限制详情，如：日志流个数受限于配置项、租户内存大小、clog 盘大小等。系统租户可以查看所有租户信息，普通租户只能查看本租户信息。
[G]V$OB_SESSION	新增	用于记录当前 Session 信息。
[G]V$OB_PROCESSLIST	新增列	新增 total_cpu_time 字段表示 CPU 使用时间。
mysql.role_edges	新增	MySQL 模式视图，用于展示角色和用户的授予关系。
mysql.default_roles	新增	MySQL 模式视图，用于展示用户默认启用的角色。
mysql.columns_priv	新增	MySQL 模式视图，用于展示用户拥有的列级权限。
DBA_OB_TRANSFER_PARTITION_TASKS	新增	展示本租户下所有正在处理的 Transfer Partition 任务。
CDB_OB_TRANSFER_PARTITION_TASKS	新增	展示所有租户当前正在处理的 Transfer Partition 任务。仅适用于 SYS 租户。
DBA_OB_TRANSFER_PARTITION_TASK_HISTORY	新增	展示当前租户下所有执行 Transfer Partition 的任务历史。
CDB_OB_TRANSFER_PARTITION_TASK_HISTORY	新增	展示所有租户执行 Transfer Partition 的任务历史。仅适用于 SYS 租户。
[G]V$OB_PARAMETERS	新增列	新增 DEFAULT_VALUE、ISDEFAULT 字段，用于展示配置项的默认值信息。
DBA_OB_SYS_VARIABLES	新增	展示当前租户的系统变量配置。
CDB_OB_SYS_VARIABLES	新增列	新增 DEFAULT_VALUE、ISDEFAULT 列信息。记录系统变量的默认值。
[G]V$OB_PL_CACHE_OBJECT	新增	展示 PL 相关的缓存对象信息。
[G]V$OB_SQL_AUDIT	新增列	新增 PL_TRACE_ID 列，用于记录外层 PL 的 trace_id，使其可以与 PL 内部 SQL 关联。新增 PLSQL_EXEC_TIME 列，用于记录 PL 执行耗时（不包括 SQL 执行时间），单位为 us。
[G]V$SQL_WORKAREA	新增列	新增 db_id 列，用于描述该请求的连接所属的数据库 ID。

配置项变更

新增如下变更：

配置项/系统变量	变更类型	变更说明
syslog_disk_size	新增	新增集群级配置项，用于控制系统日志可使用的总磁盘空间。默认为 0M，即兼容老版本默认行为，可使用整块磁盘。
syslog_compress_func	新增	新增集群级配置项，用于控制系统日志文件压缩算法，可选 none、zlib_1.0、zstd_1.0、zstd_1.3.8。默认为 none，表示不压缩系统日志文件。
syslog_file_uncompressed_count	新增	新增集群级配置项，用于控制每种日志不压缩的系统日志文件数量。默认为 0。
json_document_max_depth	新增	新增租户级配置项，用于控制 JSON 文档中允许的最大嵌套深度。默认为 100。
ha_diagnose_history_recycle_interval	新增	新增集群级配置项，用于控制 Transfer 诊断统计历史信息的清理时间间隔。默认为 7 天。
data_disk_write_limit_percentage	新增	新增集群级配置项，用于控制数据盘达到设置的水位后，用户写入报错的问题，发现问题以后，可以通过删表的方式紧急释放空间，避免集群故障。该配置项应该大于data_disk_usage_limit_percentage，开启时建议配置：（1 - memstore_limit_size / data_disk_size) * 100%。默认值为 0，表示不开启停止用户写入的功能。
strict_check_os_params	新增	新增启动项，用于指定启动时对 OS 参数进行宽松模式还是严格模式检查。默认为 False，表示不符合 OS 参数要求时，报 warning，不影响 OBServer 正常启动。
spill_compression_codec	新增	新增租户级隐藏配置项，用于指定需要临时物化的算子选用的压缩算法。默认为 NONE，表示不进行压缩。

系统变量变更

新增如下变更：

系统变量	变更类型	变更说明
QUERY_REWRITE_ENABLED	新增	新增 Global 和 Session 级系统变量，用于控制是否开启物化视图改写能力。默认为 False，表示关闭。
QUERY_REWRITE_INTEGRITY	新增	新增 Global 和 Session 级系统变量，用于控制使用实时物化视图或非实时物化视图进行改写。当 QUERY_REWRITE_INTEGRITY 设置为 enforced 时，只使用实时物化视图进行改写，改写后的实时物化视图会进行展开，保证改写后的查询结果与原查询相同。设置为 stale_tolerated 时，允许使用非实时物化视图进行改写，改写后的查询结果可能会与原始查询不同。此时，即使使用实时物化视图进行改写，也不会执行实时物化视图展开。默认为 enforced，表示只使用实时物化视图进行改写。
activate_all_roles_on_login	新增	新增 MySQL 租户下 GLOBAL 级系统变量，用于控制用户登录时是否激活所有角色。
lc_time_names	新增	新增 Global 和 Session 级系统变量，用于控制日期/月份或缩写的名称展示使用哪种语言。默认为 en_US。

函数/PL包变更

函数/PL包	变更类型	变更说明
current_role	新增函数	适用于 MySQL 模式，用于展示当前 Session 激活的 Role。
GET_LOCK	新增函数	适用于 MySQL 模式，用于获取指定名字的锁。
IS_FREE_LOCK	新增函数	适用于 MySQL 模式，用于判断指定名字锁是否空闲。
IS_USED_LOCK	新增函数	适用于 MySQL 模式，用于判断指定名字锁是否正在使用，如果正在被使用返回持锁 connection_id。
RELEASE_ALL_LOCKS	新增函数	适用于 MySQL 模式，用于释放所有用户锁。
RELEASE_LOCK	新增函数	适用于 MySQL 模式，用于释放指定名字的锁。
DBMS_OB_LIMIT_CALCULATOR	新增系统包	系统租户新增 DBMS_OB_LIMIT_CALCULATOR 系统包，用于计算不同场景下系统所需的最小物理资源。

语法变更

• 新增全文索引相关语法。
• 新增外部表分区语法。
• SELECT INTO OUTFILE 新增导出选项。
• 表级恢复命令调整。
• 新增有主键物化视图创建语法。
• 新增实时物化视图/可改写物化视图语法。
• 新增分区交换语法。
• 新增手动 Transfer 分区和任务取消语法。
• 新增 Lateral Derived Tables 相关语法。
• 新增 JSON 多值索引相关语法。

周边配套

OceanBase 数据库 V4.3.1_CE 推荐使用的平台工具版本如下:

组件	版本
ODP	V4.2.3
OCP	V4.2.2_CE_HF1
OBD	V2.8.0
ODC	V4.2.4-bp2
OBCDC	V4.3.1_CE_BETA
OMS	V4.2.3_CE
OBClient	V2.2.3
LibOBClient	V2.2.3

升级说明

• 支持 V4.3.0_CE Beta 升级到 V4.3.1_CE Beta。
• 暂不支持 V4.2.x 系列或更低版本升级到 V4.3.1_CE Beta，随着版本演进，后续会增加 V4.2.x 到 V4.3.x 的升级路径支持。

Version information

Information	Description
Release date	May 15, 2024
Version	V4.2.1_CE_BP6
Commit number	38166dc
OBServer RPM version	oceanbase-ce-4.2.1.6-106000012024042515

Performance optimization

Optimization of show table status performance: In earlier versions, the performance of the show table status from ... like ... command is not ideal because it did not make use of indexes related to table_name. In the new version, significant improvement in query performance has been achieved for scenarios with single table filtering conditions.

Bug fixes

• Fixed the issue where after SET collation_connection = 'utf8mb4_unicode_ci' is executed, comparing/joining columns of string type in information_schema would cause an error Illegal mix of collations, which is not compatible with native MySQL.
• Fixed the issue where reading and writing LOB data during the upgrades while running both the old and new versions could lead to OBServer core dump.
• Fixed the memory leak issue caused by asynchronous query timeouts in OBKV.

版本信息

项目	描述
发布日期	2024-05-15
版本号	V4.2.1_CE_BP6
Commit 号	38166dc
OBServer RPM 版本号	oceanbase-ce-4.2.1.6-106000012024042515

性能优化

优化 show table status 性能： 老版本 show table status from ... like ... 场景性能较差，未利用 table_name 相关索引。新版本针对存在单表过滤条件的场景，显著提升查询性能。

缺陷修复

• 修复 SET collation_connection = 'utf8mb4_unicode_ci' 后，information_schema 下字符串类型的列比较/关联操作，会报错 Illegal mix of collations，和原生 MySQL 不兼容的问题。
• 修复升级混跑过程中读写 LOB 数据可能导致 OBServer Core 的问题。
• 修复 OBKV 异步查询超时后导致内存泄漏的问题。

Version information

Information	Description
Release date	April 29, 2024
Version	V4.2.3_CE_BETA
Commit number	c129d71
OBServer RPM version	oceanbase-ce-4.2.3.0-100000112024042411

Overview

OceanBase Database V4.2.3_CE_BETA is the latest version in the V4.2.x_CE series. It introduces a range of MySQL compatibility enhancements, including roles, column-level permissions, UNION DISTINCT in recursive common table expressions (RCTEs), along with several other small, but significant, functionality improvements. The new version improves the performance in various scenarios. Specifically, it improves the read/write calculation and OBKV processing efficiency for multi-model data types, extends the parallel execution (PX) capabilities for batch DML operations, improves the performance of log archiving-based and network-based physical standby databases, and resolves performance issues of auto-increment columns and sequences in ORDER mode. This version also supports Amazon Simple Storage Service (S3) and object storage services that are compatible with the S3 protocol, such as Huawei Object Storage Service (OBS) and Google Cloud Storage (GCS), as the backup destination for the backup and restore feature. In terms of resource optimization, this version optimizes the storage space for temporary results of DDL operations, improves the bypass import capabilities, and implements global CPU resource isolation in the foreground and background. Moreover, the new version implements multiple usability features expected by business, such as oblogminer for log analytics and misoperation identification, major compaction strategy selection for coping with performance issues of buffer tables, format outlines for fuzzy plan binding and throttling, and index usage monitoring for identifying useless indexes. It also provides the alert.log system log file with higher readability for the database administrator (DBA), user commands for log stream (LS) replica management, and the capability to show the physical restore progress. In the new version, more detailed PX diagnostic data and node-level analytics of ASH reports are provided to significantly improve the usability of the system.

This version is recommended for use in testing environments only and is not yet advised for deployment in production environments.

Key features

Kernel enhancements

• Enhancements in query parsing capabilities

  When an SQL statement contains an excessive number of IN lists, AND/OR, or UNION operators, it can generate extremely deep syntax trees, which in turn can cause issues like excessive memory consumption, stack shortages, and extended parsing time. In response, the new version has revamped the query parsing process for these extreme cases. This rewrite reduces the resource expenditure during the parsing phase and improves the stability of executing complex SQL statements, while also boosting SQL parsing speed. For example, in a query like select sum(c1) from t1 where c1 in (1, 2, 3, ...N) with N being 200,000, the parsing efficiency in V4.2.3_CE can be 20% to 30% faster than in V4.2.2_CE.

• Enhancements in query rewrite capabilities

  In V4.2.3_CE, the optimizer has improved the query rewriting logic in several respects, such as:

  • Expression canonicalization enhancement: In earlier versions, the resolver will canonicalize various expressions in the parsing phase. For example, it will rewrite the expression A and (A or B) as A. However, nonstandard expressions generated in the SQL rewrite phase will not be rewritten. The new version introduces the expression canonicalization process in the rewrite phase to simplify expressions to the maximum extent.

  • Rewrite of conditional aggregate functions: A conditional aggregate function does not aggregate a definite expression. Instead, it aggregates an expression selected based on the branch condition judgement result. For example, the aggregate functions in select c1, sum(case when c2 = 0 then c3 else 0 end),sum(case when...)...,... from t1 group by c1, are conditional aggregate functions. The new version supports rewriting a conditional aggregate function by pushing down the function to the selection objects of each branch to reduce the number of calculations for case when expressions and aggregate functions, thereby improving the execution performance in relevant scenarios.

  • Rewrite of queries with multiple MIN/MAX aggregate functions: For a scalar group by operator, if the query contains only MIN/MAX functions and indexes are created for the MIN/MAX columns, the query can be rewritten to order by xxx limit 1, to remove ORDER BY and push LIMIT 1 to the TABLE SCAN operator based on the index order, thereby reducing data reads and sorting. In earlier versions, an SQL query containing multiple MIN/MAX functions would result in a full table scan. However, the new version optimizes this by rewriting multiple MIN/MAX functions into separate subqueries. Each subquery directly obtains the maximum or minimum value based on the index. This avoids full table scan and sorting, thereby improving the query performance.

  • Rewrite of UNION operations on constants into a query on a VALUES table: In practice, a constant table is usually expressed by using multiple UNION ALL operators. The CPU consumption and memory consumption will be high when many subqueries are involved. In the new version, UNION ALL/UNION operations on constants are rewritten into a query on a VALUES table to remarkably reduce the resource consumption in the hard parsing phase.

  • Semi-join splitting: When an EXISTS or IN subquery involves a multi-table join without join conditions, the execution plan may produce a Cartesian product with a high overhead. In the new version, semi-join splitting is performed on tables without join conditions to avoid the overhead for producing a Cartesian product.

• Plan selection optimization

  OceanBase Database V4.2.2_CE implements new query range extraction logic, which extends predicate pushdown scenarios so that a more accurate query range is extracted for vector predicates. It also resolves the issue of memory amplification during query range extraction in some scenarios in earlier versions. On this basis, OceanBase Database V4.2.3_CE modifies the method of generating query ranges for not in expressions to optimize the range extraction performance. It supports range graph pruning to reduce the number of rows extracted in the query range, thereby decreasing memory consumption. It also allows you to specify an INDEX hint during query range extraction. For example, you can specify /*+index(t1 k1 1)*/ to extract the query range only on the first column in the k1 index. Moreover, the new version provides strategy optimization measures, such as rule-based path pruning for interesting ordering indexes and LIMIT recalculation, to resolve performance issues caused by a non-optimal plan selected in the order by limit scenario.

• Adaptive cost model

  In earlier versions of OceanBase Database, the cost model uses constant parameters measured by internal machines to represent hardware system statistics, and describes the execution overhead of each operator by using a series of formulas and constant parameters. However, in real business scenarios, different hardware environments may have different CPU clock frequencies, sequential or random read speeds, and NIC bandwidths, thereby resulting in cost estimation deviations. The optimizer cannot always generate optimal plans in different business environments because of these deviations. The new version optimizes the implementation of the cost model. It allows you to use the DBMS_STATS package to collect or set system statistics coefficients so that the cost model can automatically adapt to the hardware environment. It also provides the DBA_OB_AUX_STATISTICS view to display the system statistics coefficients of the current tenant.

• Change of the replicated table attribute

  Replicated tables are a special type of tables supported since OceanBase Database V4.2.0. A replicated table can read the latest data modifications from any healthy replica. To convert a replicated table into a normal table or convert a normal table into a replicated table, you must re-create the table and import data in versions earlier than V4.2.3_CE, which is time-consuming and complex. The new version allows you to use the ALTER TABLE statement to directly change the replicated table attribute, reducing the costs.

• Compatible version control for product behavior

  To reduce errors in using features of some earlier versions that are caused due to behavioral changes in the new version after an upgrade, OceanBase Database V4.2.3_CE supports compatible version control for product behavior. You can use system variables ob_compatibility_version and ob_security_version to respectively control whether normal behavioral changes (no errors -> errors) and security behavioral changes (with privileges -> without privileges) take effect, and the OceanBase Database version whose product behavior takes effect in a tenant after the changes. Valid values are released version numbers, such as 4.2.3.0. For example, assume that the product behavior of a feature is changed in V4.2.4. When the variable is set to 4.2.3.0, the product behavior in V4.2.3.0 takes effect. When the variable is set to 4.2.4.0, the product behavior in V4.2.4.0 takes effect. Generally, the version number that determines the product behavior does not automatically change after an upgrade. If you require the product behavior in the new version, learn about and test the product behavior after the upgrade and then change the version number to the new one. This feature applies only to product behavioral changes in the future, and cannot control existing behavioral changes in released versions.

  In OceanBase Database V4.2.3_CE, the following types of product behavior are controlled by the ob_compatibility_version system variable:

  • When ob_compatibility_control is set to MYSQL5.7, the REPLACE('abd', '', null) behavior is compatible with MySQL 5.7 rather than MySQL 8.0.
  • Offsets are prohibited in the LIMIT clause in the UPDATE/DELETE statement.
  • When the projected item is an independent null value, the returned table header is changed to NULL.
  • User variable names can contain a maximum of 64 characters in length.

  The following types of product behavior are controlled by the ob_security_version system variable:

  • Privilege control for outlines and sequences.
  • CREATE TABLESPACE privilege.

Compatibility with MySQL

• Role management

  OceanBase Database V4.2.3_CE supports the role management feature of MySQL 8.0. You can manage and maintain a group of privileges based on roles to conveniently grant privileges to and revoke privileges from a specific type of users. You can grant privileges or other roles to or revoke privileges or other roles from a role like normal users. You can grant multiple roles to a user. However, the user has only the privileges of roles in the active state.

• Column-level privileges

  OceanBase Database V4.2.3_CE supports the column-level privilege feature of MySQL. You can control whether a user has the privileges to perform SELECT, INSERT, and UPDATE operations on specific columns in a table.

• Privilege control for outlines and sequences

  In earlier versions, privilege control is absent for outlines and sequences. The new version supports the CREATE, ALTER, and DROP privileges of MySQL to control the creation, modification, and dropping of outlines and sequences.

• Optimization of auto-increment column value hopping after a leader switchover

  OceanBase Database V4.x supports creating auto-increment columns in ORDER mode for higher compatibility with the behavior of auto-increment columns in MySQL. However, in OceanBase Database of a version earlier than V4.2.3_CE, values of auto-increment columns still hop after a leader switchover is performed. Generally, leader switchover is initiated by users and is not an abnormal scenario. Therefore, in the new version, values of auto-increment columns are still continuous if a leader switchover is manually initiated. This reduces the probability of value hopping of auto-increment columns.

• Value decrease for auto-increment columns

  In OceanBase Database of a version earlier than V4.2.3_CE, you can use the ALTER TABLE statement to increase the value of an auto-increment column but cannot decrease its value, which is inconsistent with that in MySQL. In the new version, you can decrease the value of an auto-increment column. If the specified new value is greater than the maximum existing value of the auto-increment column, the setting is successful and the new value takes effect. If the specified new value is smaller than or equal to the maximum existing value of the auto-increment column, the setting is successful but the column value is automatically adjusted to the next value of the maximum existing value.

• UNION DISTINCT for RCTEs

  MySQL 8.0 supports RECURSIVE UNION ALL and RECURSIVE UNION DISTINCT for common table expressions (CTEs). OceanBase Database supports RECURSIVE UNION ALL of MySQL since V3.2.3. OceanBase Database V4.2.3_CE supports RECURSIVE UNION DISTINCT of MySQL to ensure the uniqueness of output data. The new version also enhances RECURSIVE UNION ALL so that data can be stored in the disk when the available memory is insufficient. For more information, see UNION.

• Control on the compatible version (MySQL 5.7 or MySQL 8.0)

  The product behavior of MySQL 5.7 conflicts with that of MySQL 8.0 in some scenarios. For example, the output of replace('a','',null) is different in the two versions. OceanBase Database V4.2.3_CE provides the tenant-level initialization variable ob_compatibility_control that specifies whether to use the product behavior of MySQL 5.7 or MySQL 8.0 in the case of product behavior conflicts. The variable is set during tenant creation. After a tenant is created, the variable cannot be modified. The superset of non-conflicting features of MySQL 5.7 and MySQL 8.0 can be used regardless of the compatible version specified by the variable.

• Support for syntax, variables, and views

  To enable a smooth migration of various tools and frameworks from the MySQL ecosystem to OceanBase Database without the need for alterations, OceanBase Database V4.2.3_CE and subsequent versions will gradually introduce compatibility or mock handling for the complete array of MySQL 5.7 features that are currently not fully supported by OceanBase, or for functionalities that are not directly applicable.

  The compatibility features supported in V4.2.3_CE are as follows:

  • Supports the SERIAL data type.
  • [MySQL 8.0] Supports the IF NOT EXISTS syntax during trigger creation.
  • Allows you to use \N to represent NULL in SQL statements.
  • Prohibits the offset clause in the update/delete ... limit statement.
  • Supports the password function of MySQL 5.7.
  • Supports the DROP USER IF EXISTS syntax.
  • Limits a user variable identifier to 64 characters in length.
  • Supports CREATE TABLE ... [IGNORE | REPLACE] SELECT statement.
  • Supports the STRAIGHT_JOIN syntax in SELECT statements.
  • Supports the CREATE TABLESPACE privilege.
  • Allows you to grant, revoke, and view the SHUTDOWN and RELOAD privileges of MySQL, which do not actually take effect in OceanBase Database.
  • Ignores the PRIORITY option in DML statements, with no errors returned and with the option being ineffective.
  • Ignores the LOW_PRIORITY and HIGH_PRIORITY options in INSERT statements, with no errors returned and with the options being ineffective.
  • Ignores the LOW_PRIORITY option in UPDATE statements, with no errors returned and with the option being ineffective.
  • Ignores the LOW_PRIORITY and QUICK options in DELETE statements, with no errors returned and with the options being ineffective.
  • Ignores the LOW_PRIORITY option in REPLACE statements, with no errors returned and with the options being ineffective.
  • Ignores the SQL_SMALL_RESULT, SQL_BIG_RESULT, SQL_BUFFER_RESULT, and HIGH_PRIORITY options in SELECT statements, with no errors returned and with the options being ineffective.
  • Ignores the syntax of descending indexes, with no errors returned and with the syntax being ineffective, as in MySQL 5.7.
  • Ignores the reference_definition syntax after the column definition in the CREATE TABLE statement, with no errors returned and with the syntax being ineffective, as in MySQL 5.7.
  • Ignores the KEY_BLOCK_SIZE option specified in the table/index creation syntax, with no errors returned and with the option being ineffective.
  • Returns no errors for the FLUSH PRIVILEGES syntax, which is actually inapplicable in OceanBase Database.
  • Returns no errors for the SHOW PROFILE/PROFILES syntax, which actually does not take effect in OceanBase Database.
  • Returns no errors for the SHOW FUNCTION/PROCEDURE CODE syntax, which is actually not supported in OceanBase Database.
  • Supports the INFORMATION_SCHEMA.PROFILING table schema.
  • Supports the following variables of MySQL: debug, debug_sync, innodb_change_buffering_debug, innodb_compress_debug, innodb_disable_resize_buffer_pool_debug, innodb_fil_make_page_dirty_debug, innodb_limit_optimistic_insert_debug, innodb_merge_threshold_set_all_debug, innodb_saved_page_number_debug, innodb_trx_purge_view_update_only_debug, innodb_trx_rseg_n_slots_debug, stored_program_cache, profiling, profiling_history_size, and innodb_stats_persistent. You can query and set these variables, but they do not actually take effect. After you set these variables, no error but a warning is returned.
  • Supports the COM_PROCESS_INFO and COM_PROCESS_KILL communication protocols of MySQL 5.7. Additionally, it supports the COM_REFRESH and COM_DEBUG communication protocols, which do not actually take effect.

Multimodel features

• MySQL GIS

  OceanBase Database supports geographic information system (GIS) data types and some expressions related to spatial objects since V4.1.0, and gradually supports the storage, computing, and analysis of spatial data. The new version supports the PostGis_ST_GeoHash expression in MySQL mode for calculating the geohash of a geometry. It supports the PostGis_ST_MAKEPOINT expression for creating 2D and 3D geometry points based on the coordinates. It also supports the ST_ASGEOJSON expression of MySQL for converting a geometry (WKB) into a JSON string (binary) based on a specific format.

• Memory usage optimization for JSON, XML, and GIS data types

  The new version optimizes the memory usage during the execution of JSON expressions. Specifically, the memory usage is reduced by 1/2 when you perform query operations, such as JSON_KEYS, JSON_LENGTH, and JSON_SEARCH, on JSON data. The memory usage is reduced by about 3/4 when you perform output operations, such as JSON_PRETTY and JSON_UNQUOTE, on JSON data. The new version also optimizes the memory usage in XML insert/update and output scenarios. Specifically, the memory usage is reduced by about 2/3 to 3/4 when you perform XML output operations such as GetClobVal and XmlCast, and unnecessary parsing is reduced when you perform XML insert/update operations such as UpdateXml and InsertChildXml.

  Moreover, the new version optimizes the memory usage for GIS data types in data input, output, and some query scenarios. Specifically, the memory usage is reduced by about 3/4 for GIS input scenarios such as st_geomfromtext, st_geomfromwkb, st_geogfromtext, and geometrycollection, and reduced by 1/2 for GIS output scenarios such as st_astext and st_aswkb. The memory amplification degree is remarkably decreased for query scenarios, including spatial relation query expressions such as st_crosses, st_overlaps, and _st_touches, and GIS attribute acquisition expressions such as st_geometrytype and st_iscollection.

• Optimization of the GIS spatial relation calculation performance

  The new version optimizes the performance of spatial relation calculation expressions such as ST_INTERSECTS, ST_CONTAINS, _ST_COVERS, and ST_WITHIN.

  • In a full query window, the response time (RT) of the ST_INTERSECTS expression for a point in OceanBase Database is slightly shorter than that in PostgreSQL. For other test scenarios, the average RT is less than a half of that in PostgreSQL. Specifically, the time spent on ST_INTERSECTS for a linestring is only 1/5 of that in PostgreSQL. Compared with earlier versions, the new version improves the calculation performance by orders of magnitude.

  • In a large query window, the average RT in all test scenarios is obviously shorter than that in PostgreSQL. The time spent on ST_INTERSECTS for a linestring is only 1/7 of that in PostgreSQL. Compared with earlier versions, the new version improves the calculation performance by orders of magnitude.

  • In a small query window, the calculation performance of the ST_CONTAINS expression is higher than that in PostgreSQL. Specifically, the time spent on ST_CONTAINS for a point is about 1/4 of that in PostgreSQL, and the time spent on ST_CONTAINS for a linestring is 80% of that in PostgreSQL. However, the calculation performance of the ST_INTERSECTS expression for a linestring is on a par with that in PostgreSQL. The time spent on ST_INTERSECTS for a point is about twice that in PostgreSQL.

• Read/Write performance optimization for LOB data stored in OUTROW mode

  OceanBase Database V4.2.3_CE optimizes the performance for large object (LOB) data stored in OUTROW mode.

  • In a table scan scenario, the new version can improve the performance by almost 10 times for small LOBs and by at least twice for large LOBs, compared with earlier versions. However, the performance in this scenario is still lower than that of the INROW storage mode.
  • In a point select scenario in the new version, the queries per second (QPS) of the OUTROW storage mode is less than that of the INROW storage mode by no more than 20% for small LOBs (sized less than 8 KB), and is less than that of the INROW storage mode by about 5% for medium and large LOBs (sized more than 32 KB). In a scenario where LOBs are not involved in calculations, the OUTROW storage mode can achieve a higher query performance.
  • In a point update scenario, the new version improves the performance of the OUTROW storage mode by twice, compared with earlier versions.

OBKV enhancements

• OBKV global indexes

  OceanBase Database supports local indexes for OBKV since V4.1.0. The new version also supports global indexes for OBKV. DML operations such as insert, update, delete, insert_or_update, replace, and increment/append are supported for global index tables. Moreover, global indexes can be specified in general queries and asynchronous queries.

• OBKV performance optimization

  OBKV is a key-value (KV) storage product integrated into OceanBase Database and is characterized by low delay and high throughput. In a scenario that does not require SQL for complex logic processing, you can get around the SQL module and directly call the storage API to achieve the optimal performance. Actual business scenarios and internal end-to-end testing and analysis results show that the performance overhead at the KV module layer in earlier versions can be further reduced. Therefore, the new version optimizes the performance in the following scenarios:

  • Put feature: The Put feature is intended for overwrite scenarios. An overwrite operation skips the primary key conflict check and forcibly overwrites the existing record.

  • Group commit: Group commit is a feature that allows the server to perform batch operations. After you enable group commit, the server will group Put or Get operations based on the execution plan, automatically resize the groups based on the load, and execute the operations by group. You can use the tenant-level parameter enable_kv_group_commit to enable group commit. After the feature is enabled, the operations per second (OPS) can be increased by 30% to 50% without obviously affecting the RT.

  • Batch processing: For a single-operation API, it takes much time to start and close transactions. The batch operation API of OBKV avoids this issue, improving the operation efficiency to a certain extent. However, each operation still requires the steps of constructing an operator, opening the operator, submitting a data access service (DAS) task, closing the operator, and destructing the operator, which causes execution time losses. To further improve the execution efficiency of the batch operation API, the new version implements batch operation processing at the operator level to reduce the number of operator constructions and destructions by n-1 and decrease the overhead of the DAS task, thereby effectively improving the overall performance.

  • Schema acquisition logic: In the new version, schemas are obtained at the statement level. Specifically, a simple schema is obtained only during statement execution. This avoids performance losses caused by frequent schema acquisition. In addition, frequently obtained schema information, such as table_id and column_id, is cached in the plan to form a lightweight KV schema cache.

  • Only one result returned for batch processing: In a multi-set scenario, a result is returned for each operation in earlier versions. In the new version, after batchOps.setAtomicOperation(true) and batchOps.setReturnOneResult(true) are specified, only one result, success or failure, is returned for a multi-insert, multi-put, or multi-delete scenario, shortening the interaction duration.

Performance improvements

• DML performance optimization

  OceanBase Database allows you to use the parallel DML (PDML) mechanism to improve the performance of the insert, update, and delete operations on large-sized tables and indexes. However, PDML is inapplicable in some scenarios, such as replace into ..., insert ... on duplicate key, insert all ..., merge into ..., primary key update, insertion with auto-increment columns, and multi-table update. In earlier versions, a single thread is used for writing in the preceding scenarios. When a large amount of data is involved, the response time is long, which affects the user experience. To address the performance bottleneck caused by a large amount of data in these scenarios, OceanBase Database V4.2.3_CE implements parallel write at the DAS layer. It uses a parallel control mechanism similar to PDML to significantly improve the DML execution performance.

• Backup performance optimization

  During backup in OceanBase Database, a continuity check is performed to make sure that the baseline version number is greater than the minor compaction version number, which ensures data integrity. However, the continuity check involves reading data from the backup media and executing operations during which the operated data needs to be locked, compromising the backup performance. The new version reduces the performance overhead caused by continuity check during backup. You can use the ha_low_thread_score parameter to control the number of threads used for backup, thereby effectively improving the backup performance.

• Performance optimization for network-based physical standby databases

  OceanBase Database V4.2.3_CE optimizes the RPC and network frameworks to reduce the time required for transmitting logs between the primary and standby databases. It also optimizes the controlled log pull process of the standby database to reduce the control frequency, thereby improving the log synchronization performance between the primary and standby databases.

• Caching and disk storage of the compilation result after a stored procedure undergoes a DDL change

  OceanBase Database V4.2.2_CE supports storing compiled stored procedures in the disk during execution. This resolves the performance issue caused by recompilation when a stored procedure fails to obtain the PL cache during execution. However, after a DDL change is performed on a stored procedure, the stored procedure still needs to be recompiled during execution. In this case, the performance can be further improved. In OceanBase Database V4.2.3_CE, after a DDL change is performed on a stored procedure, the compilation result can be stored in the PL cache and disk. When the stored procedure is executed later, the PL cache can be directly accessed, further improving the execution performance of the stored procedure.

• Extension of parallel DDL operations

  Parallel DDL and serial DDL are mutually exclusive. The performance is poor when parallel DDL and serial DDL are executed alternatively. As the version evolves, OceanBase Database continues to extend parallel DDL operations. OceanBase Database implements the parallel DDL framework and supports parallel TRUNCATE in V4.1.0, supports parallel CREATE TABLE in V4.2.1, and supports parallel COMMENT and CREATE INDEX in V4.2.2_CE. On this basis, OceanBase Database V4.2.3_CE supports parallel CREATE VIEW, further increasing the schema migration speed for OceanBase Migration Service (OMS).

• Performance optimization for auto-increment columns in ORDER mode

  For auto-increment columns in ORDER mode, you must ensure the continuity of inserted data. When you insert a value into an auto-increment column in a distributed architecture, the value will be persisted to the internal table to ensure the order of values of the auto-increment column. The response time will be long in the case of high concurrency. OceanBase Database V4.2.3_CE supports cache size prefetch to reduce the number of accesses to the internal table, thereby drastically improving the performance.

• Performance optimization for sequences in ORDER mode

  OceanBase Database allows you to specify the order and cache attributes to create a globally continuous sequence. In earlier versions however, the cache attribute is ignored, which means that the nocache attribute actually takes effect, leading to obvious performance issues in high concurrency scenarios. OceanBase Database V4.2.3_CE allows you to generate a continuous sequence by supporting the order and cache attributes on a central node. This greatly improves the execution performance in high concurrency scenarios since the cache attribute is realized.

• Performance optimization for the SHOW TABLE STATUS statement

  In earlier versions, the performance of the SHOW TABLE STATUS FROM ... LIKE statement is poor because indexes related to table_name are not used. The new version remarkably improves the query performance in scenarios involving single-table filter conditions.

• Support of hints in the CREATE statement

  OceanBase Database V4.2.3_CE supports the /*+ parallel(N) */ hint in the CREATE TABLE statement. You can use this hint in a CREATE TABLE ... AS SELECT ... statement to specify the degree of parallelism (DOP) for data queries and writes during table creation.

Resource optimization

• Link compression between OBServer nodes and ODP

  In actual business scenarios, applications and database services may be deployed across regions. Such a scenario is characterized by far distances and limited bandwidth resources accompanied by read and write requirements involving large amounts of data. To reduce the bandwidth usage by the transmission of large amounts of data, OceanBase Database V4.2.3_CE supports link compression between OBServer nodes and OceanBase Database Proxy (ODP). This feature is supported only when the version of ODP is V4.2.3_CE or later.

• Optimization of DDL temporary result space

  Temporary execution results of DDL operations are usually stored in a materialized structure. When an index is created, OceanBase Database needs to scan the data table and insert data into the index table. It may need to sort the scanned data during this process. If the memory space is insufficient, it will temporarily store the data in the memory in a materialized structure to release the memory space for later scan. Then, it will merge and sort the data in the materialized structure.

  In the new version, the data flow in DDL operations is optimized to address these issues. First, the new version eliminates unnecessary redundant structures to simplify the data flow. Second, the new version introduces coding compression for storing temporary results on disks. This way, the disk space consumed by temporary results during DDL operations is significantly reduced, and storage resources are thereby more efficiently utilized.

• Bypass import optimization

  • Compression of temporary files: Temporary files are generated during bypass import. When a large amount of data is imported, the generated temporary files will occupy much disk space. In this case, you need to pay attention to the disk usage and prevent the disk space from being used up by these temporary files. When the DOP for bypass import is greater than or equal to 8, the system will compress the generated temporary files to reduce the occupied disk space.
  • Import of multiple files by using wildcards: During bypass import, you can import multiple files by using wildcards. Specifically, you can use the wildcards * and ? to concurrently import multiple files whose names match a specific pattern, thereby improving the import efficiency and convenience.

• Global CPU resource isolation for foreground and background tasks

  In a high-performance computing environment, reasonable resource allocation and isolation is decisive in ensuring system stability and improving efficiency. An effective resource isolation strategy can prevent resource contention and interference between tasks, thereby improving the resource utilization efficiency and overall service quality. At present, OceanBase Database allows you to configure different unit configs for tenants to implement resource isolation between the tenants, and use the DBMS_RESOURCE_MANAGER system package to configure resource isolation within a tenant. Resource isolation is supported for CPU and I/O resources. In a business scenario where you want to prevent background tasks from contending with foreground tasks for CPU resources, resource isolation for background tasks at the global level is a better choice than tenant-level resource isolation. OceanBase Database V4.2.3_CE supports global CPU resource isolation for foreground and background tasks. You can limit the CPU resources available for background tasks on an overall basis, which is more convenient and easier than using the DBMS_RESOURCE_MANAGER system package to separately configure resource isolation.

• Partition balancing strategy optimization

  OceanBase Database supports partition balancing since V4.2.0. However, full balancing still cannot be achieved in some scenarios. OceanBase Database V4.2.3_CE optimizes the partition balancing strategy. You can scatter continuous partitions when you create a partitioned table. For a user table that contains a LONGTEXT or LOB column or a local index, the system will also calculate associated tables for partition disk balancing, contributing to more balanced disk usage.

• System log compression

  When the business traffic is heavy, system logs are refreshed quickly. In this case, troubleshooting will be affected due to a short retention period of system logs. The new version provides the system log compression feature. For the observer.log, rootservice.log, election.log, and trace.log log files, when the number of log files of a specific type reaches the value specified by syslog_file_uncompressed_count, the earliest logs will be compressed by using the compression method specified by syslog_compress_func. When the total occupied disk space approaches the upper limit specified by syslog_disk_size, the earliest log files are deleted to release the occupied disk space. After you enable zstd compression, the volume of logs that can be stored in the same disk space is 20 times that of logs that can be stored when compression is disabled.

Reliability improvements

• Optimization of selecting the migration source

  In earlier versions, replicas in the same zone, Internet Data Center (IDC), or region are not preferentially considered when a migration source is selected. This may lead to far distance migration with poor performance. In addition, if a leader is selected as the source, business requests and the migration task may be processed slowly when a large number of business requests are initiated. OceanBase Database V4.2.3_CE defines the regional relationships between the migration source and destination as follows: same IDC, same region but different IDCs, and different regions. It also provides an enumeration parameter choose_migration_source_policy for you to specify the prioritizing mode for selecting the source replica for migration. You can preferentially select a nearby follower to improve the migration efficiency and reduce the pressure on the leader.

• Write stop upon high data disk usage

  In earlier versions, when the data disk usage is high, user write requests will still be processed until an error is returned after the memory is full due to minor compaction failures or after the clog disk is full. In this case, you need to urgently scale out the clog disk space or tenant memory to resolve this issue. In the new version, the kernel provides the feature of write stop upon high data disk usage. When the data disk usage reaches the value specified by data_disk_write_limit_percentage, an error is returned for new user write requests. After you drop tables, transfer data, or scale out the disk space to reduce the data disk usage, user writes automatically resume.

High availability enhancements

• S3, OBS, and GCS supported as the backup media

  Earlier OceanBase Database versions support Network File System (NFS), Alibaba Cloud Object Storage Service (OSS), and Tencent Cloud Object Storage (COS) as the storage media for the backup and restore feature. The new version supports S3 and object storage services compatible with the S3 protocol, such as OBS and GCS, as the destination for log archiving and data backup. The backup data stored on S3 or an object storage service compatible with the S3 protocol can be used for a physical restore.

• Physical restore at the set or piece level

  In actual business scenarios, there is a need for secondary backups where data backup sets or archive logs are manually transferred to a new path. The physical restore feature of OceanBase Database requires that restore takes place within the tenant's designated archive and backup paths, which means it cannot utilize backup data that has been manually moved to a new path, thus limiting the flexibility of the restore process. The new version introduces improvements to the SET/PIECE level physical restore capabilities by providing an add restore source command. This command allows for the loading of data backup sets (SET) or log archives (PIECE) from a new path, enabling restoration to a specific point in time as needed.

Usability improvements

• OceanBase LogMiner

  OceanBase Database V4.2.3_CE supports OceanBase LogMiner (oblogminer for short). oblogminer is a command-line tool used to perform log analysis online or offline for OceanBase Database. oblogminer uses OceanBase Change Data Capture (CDC) to pull and parse clogs, converts the logical logs output by OceanBase CDC to a readable format, and stores the logs in the specified position. The tool applies to the following scenarios:

  • Data misoperations: Data misoperations may be caused by various reasons. For example, rows are mistakenly deleted or updated due to an incorrect range condition in the WHERE clause. In this case, you can use oblogminer to accurately obtain the details about the misoperation and restore the data to the state before the misoperation.
  • Data analysis: oblogminer will organize various information, such as transactions and table schemas, in clogs, and display the organized information in a user-friendly manner. You can perform data analysis based on the output results of oblogminer as needed. You can also read the output results of oblogminer in combination with external tables for query analysis in the database.

• Optimization of adaptive major compactions for buffer tables

  If you frequently insert data into a table while performing batch deletion, or if a large number of update operations are concurrently performed, the query and update performance obviously deteriorates though the table contains only a moderate number of rows. This table is defined as a queuing table in OceanBase Database. In terms of business, such a table is known as a buffer table. Buffer tables are common in databases built based on the log-structured merge-tree (LSM-tree) architecture. In the LSM-tree architecture, a delete operation is only a logical mark before a major compaction is performed so data is not physically deleted. When a large proportion of the incremental data is marked as deleted, physical rows are far more than logical rows, resulting in severe read amplification and affecting the generation of execution plans in the optimizer.

  OceanBase Database V4.1.0 can automatically identify partitions involving many changes within a specific period of time and perform adaptive major compactions for these partitions. However, no manual control means are present. To flexibly resolve the issue of performance deterioration caused by buffer tables, OceanBase Database V4.2.3_CE further optimizes adaptive major compactions for buffer tables and provides five levels of major compaction strategies. You can set different table_mode values for tables based on the business scenario to cope with read amplification caused by the buffer table issue, thereby improving metrics, such as QPS, during the long-term operating of the system.

• Format outline

  OceanBase Database of earlier versions allow you to create a normal outline to bind an execution plan or throttle specific SQL statements. This feature has strict requirements on the SQL text format. Except for the parameterized part in the outline, an extra space or tab or case inconsistency will cause a failure to hit the outline. This feature has poor usability in scenarios where the SQL text format is not fixed. OceanBase Database V4.2.3_CE provides the format outline feature with looser matching rules. When the system attempts to match a format outline for an SQL statement, it will normalize the statement into a standard format by ignoring the letter case and non-syntactic definition symbols such as spaces before the original outline matching process. SQL statements with the same format SQL text or format SQL ID after normalization will hit the same format outline. In a scenario where the number of elements in an IN list is not fixed, such as a scenario where SQL statements that contain in(1,2), in(4,5,6), and in(7,8,9,10) are delivered by business, you need to bind three outlines in(?,?), in(?,?,?), and in(?,?,?,?) to use the precise match feature of normal outlines to control the execution plan. The format outline feature in the new version requires you only to bind one outline and the system will normalize the IN list into the in(?) format. This way, multiple SQL statements containing IN lists with different numbers of elements can share the same outline. When both a normal outline and a format outline are present, the normal outline is preferentially hit.

• Index usage monitoring

  We usually create indexes to improve the query performance of the database. However, more and more indexes are created as data tables are used in more business scenarios by more operators. Unused indexes are a waste of storage space and increase the overhead of DML operations. In this case, you need to drop useless indexes to alleviate burden on the system. However, you can hardly identify all useless indexes by manual efforts. Therefore, OceanBase Database V4.2.3_CE provides the index usage monitoring feature. After you enable this feature and set the sampling method, the index usage information that meets the rules is recorded in the memory of a user tenant and refreshed to the internal table once every 15 minutes. You can then query the DBA_INDEX_USAGE view to find out whether an index is referenced and drop useless indexes to release space.

• System log optimization

  The ./alert/alert.log file is added to the system log directory to record log information concerned by DBAs. This log file can help resolve the issue where the observer.log file has poor readability because it contains a large volume of logs. You can use the cluster-level parameter alert_log_level to set the log level to INFO, WARN, or ERROR. In addition, the external system table sys_external_tbs.__all_external_alert_log_info is provided for you to view logs in the alert.log file in a structured way.

• LS replica management

  OceanBase Database of a version earlier than V4.0.0 provides a series of O&M commands for partition replica management. For example, you can use related commands to add a replica to a partition, drop a replica from a partition, and convert the type of a replica for a partition. OceanBase Database V4.x replaces the concept of partition with LS. V4.2.3_CE redesigns the partition O&M commands for LS replica-level O&M. It provides a series of syntax for adding LS replicas, dropping LS replicas, converting the type of LS replicas, migrating LS replicas, modifying the number of Paxos members of an LS replica, and canceling a disaster recovery task, to meet manual LS replica O&M requirements.

• Physical restore progress statistics

  To allow you to learn about the running status and progress of a physical restore task, and estimate the remaining time required for the restore task, the new version provides the physical restore progress statistics feature. You can query the CDB/DBA_OB_RESTORE_PROGRESS view for the restore progress in real time.

• Enhanced PX diagnostic capabilities

  To facilitate distributed plan troubleshooting, OceanBase Database V4.2.3_CE records the memory usage and disk usage during operator execution as well as the maximum memory usage and disk usage during the entire execution process in the SQL_PLAN_MONITOR view. It also records in the SQL_AUDIT view the numbers of rows in the MemTable and SSTable of trace_id corresponding to the query coordinator (QC), and supports summarizing related information of trace_id corresponding to various PX worker threads. Moreover, it allows you to use the OBDiag tool to pull the logs of the server that first reports an error based on trace_id.

• Enhanced PS diagnostic capabilities

  When a prepared statement (PS) handle leak occurs in versions earlier than V4.2.3_CE, you can only query the GV$OB_PS_ITEM_INFO view for global information, and no session-level diagnostic methods are available. OceanBase Database V4.2.3_CE provides the [G]V$OB_SESSION_PS_INFO view to display the PS reference information about each session, so as to help you accurately locate PS handle leaks.

• Node-level ASH report analysis

  Earlier versions support generating cluster-level ASH reports but lack node-level ASH statistics. If you want to view the ASH data of a specific node, you need to use an SQL script to obtain the data. This causes inconveniences in analyzing issues that occur on a single node. OceanBase Database V4.2.3_CE supports node-level ASH report analysis capabilities. Two parameters are added to the DBMS_WORKLOAD_REPOSITORY.ASH_REPORT package function for you to specify the IP address and port number of a specific node for ASH report analysis.

• Table-level cache size setting for auto-increment columns

  In earlier versions, you can use the tenant-level parameter auto_increment_cache_size to control the number of cached values requested by auto-increment columns at a time. The parameter takes effect for all tables with auto-increment columns. Generally, a larger cache size contributes to higher performance. However, if value hopping occurs frequently in an auto-increment column and the maximum value of the column is small, the value range of the column will be quickly used up. OceanBase Database V4.2.3_CE supports table-level cache size setting for auto-increment columns. You can set different cache sizes for tables based on the column type, business model, and business traffic, to keep performance stable against frequent value hopping.

• NETWORK_WAIT_TIME statistical item

  The NETWORK_WAIT_TIME statistical item is added to views SQL_AUDIT and SQLSTAT to show the wait time of wait events of the NETWORK class, such as synchronous RPC wait events and DAS asynchronous RPC lock wait events. When you troubleshoot slow SQL statements that are remotely executed, you can use this statistical item to confirm the network overhead to assist diagnostics.

• XA transaction monitoring statistical items

  Some customers use the XA protocol to ensure the atomicity of committing cross-database transactions in their business. OceanBase Database adopts a distributed architecture and consumes resources for the execution of XA statements. Therefore, more than 30 statistical items, such as XA_TRANS_START_COUNT, XA_READ_ONLY_TRANS_TOTAL_COUNT, and XA_ONE_PHASE_COMMIT_TOTAL_COUNT, are added to the SYSSTAT view for you to be aware of the changes in the XA transaction traffic.

• Improvement in SQL information recording in crash logs

  The SQL information in crash logs can be used for troubleshooting. OCP-Agent can also collect the SQL information and generate alerts when necessary. In versions earlier than V4.2.3_CE, SQL information will be recorded in crash logs when the INNER SQL main thread or user SQL main thread crashes. However, no SQL information is recorded in crash logs when the REMOTE thread, PX SQC thread, PX WORKER thread, or DAS remote execute thread crashes. For the preceding scenarios, the new version supports obtaining the sql_id or sql_string information about the execution server. When the execution server crashes, the SQL information about the control server will be recorded in crash error logs to help you locate the crash reason.

• Support of IPv6 for OBServer nodes

  The new version supports IPv6 addresses for OBServer nodes. SQL clients and RPC clients can connect to OBServer nodes through IPv6 addresses. It also supports hybrid deployment of OBServer nodes with an IPv4 address and those with an IPv6 address in the same cluster. You can upgrade an IPv4 cluster to the new version but cannot upgrade an IPv4 cluster to an IPv6 cluster of the new version. For more information about the deployment modes, see Deploy a standalone OceanBase database by using the CLI.

• Support for querying data from the GV$PLAN_CACHE_PLAN_EXPLAIN view only by plan ID

  When you query data from the [G]V$PLAN_CACHE_PLAN_EXPLAIN view in earlier versions, you must specify the IP address, port number, tenant ID, and plan ID as the filter conditions. If you specify only the plan ID, the query result set is empty, which means poor usability of the view. The new version supports scanning the underlying virtual table of this view and allows you to specify only the plan ID to accurately query data from this view.

• Trace ID parsing

  OceanBase Database uses a trace ID to mark the full process of an SQL request. The trace ID can be used to associate monitoring metrics or query logs. A trace ID contains the IP address and port number of the OBServer node that initiates the SQL request. However, no method is available for directly parsing the information contained. The new version provides the decode_trace_id function for parsing a trace ID to obtain the IP address and port number.

Compatibility changes

Product behavioral changes

The following table describes the changes made in this version.

Feature	Description
Parallel COMMENT and parallel CREATE INDEX are disabled by default after an upgrade.	Since OceanBase Database V4.2.2_CE, support for parallel COMMENT and parallel CREATE INDEX functionalities has been introduced and they are enabled by default. In V4.2.3_CE, when creating a new cluster or tenant, these features along with the newly introduced parallel CREATE VIEW are activated by default as well. However, for tenants upgraded from an earlier V4.x version to V4.2.3_CE, these three parallel DDL operations are disabled by default. If there is a need for high-performance DDL operations through these parallel functionalities, they can be enabled using the hidden parameter _parallel_ddl_control. For example, executing the command alter system set_parallel_ddl_control='SET_COMMENT:ON,CREATE_INDEX:ON,CREATE_VIEW:OFF' tenant='xxx'; will enable parallel COMMENT and parallel CREATE INDEX, while leaving parallel CREATE VIEW disabled.
In the PS protocol, an error is returned when the number of parameters exceeds 65,535.	The MySQL protocol supports a maximum of 65,535 parameters. When the number of parameters exceeds 65,535, an exception is thrown. In earlier versions of OceanBase Database, 65,535 parameters, instead of an error, are returned in this scenario. The new version is adapted for compatibility with MySQL, and an error is returned when the number of parameters exceeds 65,535.
RENAME TABLE is changed to an operation that requires a table lock.	In earlier versions, RENAME TABLE is an online DDL operation that does not require to lock the table. Unexpected issues may arise when a transaction involves other operations apart from the RENAME TABLE operation. OceanBase Database V4.2.3_CE supports locking the table when you perform a RENAME TABLE operation, and provides read/write defense during the operation.
The [g]v$plan_cache_plan_explain view supports data queries by plan ID only.	When you query data from the [g]v$plan_cache_plan_explain view in earlier versions, you must specify the IP address, port number, tenant ID, and plan ID as the filter conditions. If you specify only the plan ID, the query result set is empty, which means poor usability of the view. The new version supports scanning the underlying virtual table of this view and allows you to specify only the plan ID to accurately query data from this view.
The meaning of the tenant_id field in the [g]v$ob_sql_audit view is changed.	Some internal requests generated in a user tenant are initialized in the sys tenant. When such an internal request needs to read data, it reads data from the user tenant. In this case, the value of tenant_id is 1, and that of effective_tenant_id is the ID of the current tenant. For sequential collection requirements based on OceanBase Autonomy Service (OAS) within a tenant, OceanBase Database V4.2.3_CE adjusts the tenant_id field, which can be used as an index, to be equivalent with the effective_tenant_id field.
The value of the sql_id field in the [g]v$ob_sql_audit view is changed.	In versions earlier than V4.2.3_CE, the value of the sql_id field in a CALL statement is an MD5 value generated based on an empty string, and that of the sql_id field in an anonymous block is an MD5 value generated based on the original PL code that is not parameterized. In V4.2.3, the values of the sql_id fields in both the anonymous block and the CALL statement are changed to an MD5 value generated based on the parameterized statement.
Values of some fields in the [g]v$ob_active_session_history view are changed.	In versions earlier than V4.2.3_CE, the values of the MODULE, ACTION, and CLIENT_ID fields are always NULL. Since V4.2.3_CE, the three fields show actual user settings. Specifically, MODULE and ACTION are set by using DBMS_APPLICATION_INFO, and CLIENT_ID is set by using DBMS_SESSION.
null values in a projected column with no alias are replaced with NULL.	In MySQL, null values (\\N, null, Null...) in a projected column with no alias are replaced with NULL. However, null values in a compound expression remain the original string. In earlier versions of OceanBase Database, null values in a projected column are not modified. In the MySQL mode of OceanBase Database V4.2.3_CE, if ob_compatibility_version is set to 4.2.3.0 or a later V4.2.x_CE version, the behavior in the preceding scenario is the same as that in MySQL.
Offset semantics are prohibited in the LIMIT clause of a DELETE/UPDATE statement.	In the MySQL mode of OceanBase Database V4.2.3_CE, if ob_compatibility_version is set to 4.2.3.0 or a later V4.2.x_CE version, offset semantics are prohibited in the LIMIT clause of a DELETE/UPDATE statement. Specifically, a syntax error must be returned for the update/delete...limit x,x and update/delete...limit x offset x syntaxes supported in earlier versions, to keep consistent with MySQL.
The return result of REPLACE('abd', '', null) depends on the value of ob_compatibility_control.	In the MySQL mode of OceanBase Database V4.2.3_CE, if ob_compatibility_version is set to 4.2.3.0 or a later V4.2.x version, after an upgrade from an earlier version or if ob_compatibility_control is set to MYSQL5.7 during tenant creation, the behavior of REPLACE('abd', '', null) is changed from that of MySQL 8.0 to MySQL 5.7.
User variable names can contain a maximum of 64 characters in length.	In the MySQL mode of OceanBase Database V4.2.3_CE, if ob_compatibility_version is set to 4.2.3.0 or a later V4.2.x_CE version, the length of user variable names is limited to 64 characters, which is not limited in earlier versions.
Privilege control is supported for outlines and sequences.	In the MySQL mode of OceanBase Database V4.2.3_CE, if ob_security_version is set to 4.2.3.0 or a later V4.2.x_CE version, you can create and manage outlines and sequences only after you are granted the CREATE, ALTER, and DROP privileges.
The CREATE TABLESPACE privilege is required for creating and managing tablespaces.	In the MySQL mode of OceanBase Database V4.2.3_CE, if ob_security_version is set to 4.2.3.0 or a later V4.2.x_CE version, you can create and manage tablespaces only after you are granted the CREATE TABLESPACE privilege.
Startup of an OBServer node will not fail when the IP address does not match any NIC name.	When you start an OBServer node in the new version, if no local NIC whose name matches the IP address specified by local_ip is found, an error log is recorded and the device name specified by devname is used as the local NIC name. In earlier versions, the startup will fail in this case.

View changes

The following table describes the changes made in this version.

View	Change type	Description
CDB/DBA_OB_RESTORE_PROGRESS	Modified	Columns RECOVER_SCN, RECOVER_SCN_DISPLAY, RECOVER_PROGRESS, TABLET_COUNT, FINISH_TABLET_COUNT, and RESTORE_PROGRESS are added to show the physical restore progress.
CDB/DBA_OB_LS_REPLICA_TASKS	Modified	Columns DATA_SOURCE_SVR_IP, DATA_SOURCE_SVR_PORT, and IS_MANUAL are added to record the data source referenced during the execution of a disaster recovery task and the source that initiates the disaster recovery task.
CDB/DBA_OB_LS_REPLICA_TASK_HISTORY	New	Displays the execution history of disaster recovery tasks. You can query the CDB view only in the sys tenant, and the DBA view in all tenants.
CDB/DBA_OB_AUX_STATISTICS	New	Displays auxiliary statistics of each tenant. You can query the CDB view only in the sys tenant, and the DBA view in all tenants.
[G]V$SQL_WORKAREA	Modified	The DB_ID column is added to describe the ID of the database to which the connection of the request belongs.
[G]V$OB_SQL_AUDIT	Modified	The STMT_TYPE column is added for determining the SQL type. The NETWORK_WAIT_TIME column is added to show the wait time of wait events of the NETWORK class. The PROXY_USER column is added to show the username of the proxy user when a proxy user is used for logon. The TENANT_ID column is adjusted to be equivalent with the EFFECTIVE_TENANT_ID column. The SQL_ID column shows the actual MD5 value generated for a PL request executed in a CALL statement or an anonymous block.
[G]V$OB_PROCESSLIST	Modified	The PROXY_USER column is added to show the username of the proxy user when a proxy user is used for logon.
[G]V$OB_ACTIVE_SESSION_HISTORY	Modified	Since V4.2.3_CE, the MODULE, ACTION, and CLIENT_ID columns show the actual user settings. Specifically, MODULE and ACTION are set by using DBMS_APPLICATION_INFO, and CLIENT_ID is set by using DBMS_SESSION.
[G]V$OB_SESSION_PS_INFO	New	Displays the information about PS opened in all sessions of tenants. The V$ view displays the PS information about the current OBServer node, and the GV$ view displays the PS information about all OBServer nodes. You can query these views in all tenants.
CDB/DBA_INDEX_USAGE	New	Displays information about index usage. You can query the CDB view only in the sys tenant, and the DBA view in all tenants.
V$OB_COMPATIBILITY_CONTROL	New	Displays all features that support product behavior compatibility control based on the released OceanBase Database versions. This view is applicable in MySQL mode.
mysql.role_edges	New	Displays the relationships between roles and users who are granted the roles. This view is applicable in MySQL mode.
mysql.default_roles	New	Displays the roles that are enabled for users by default. This view is applicable in MySQL mode.
mysql.columns_priv	New	Displays the column-level privileges of users. This view is applicable in MySQL mode.
sys_external_tbs.__all_external_alert_log_info	New	Displays logs in the alert.log file in a structured way. This is a new external table in the sys tenant.

Parameter changes

Parameter	Change type	Description
enable_kv_group_commit	New	Specifies whether to enable group commit for OBKV. If this feature is enabled, the server will group operations based on the execution plan and execute the operations by group. If this feature is not enabled, the server will execute the operations one by one. It is a tenant-level parameter. The default value is False, which specifies not to enable this feature.
choose_migration_source_policy	New	The strategy for selecting the migration source. It is a tenant-level parameter. Two strategies are supported:IDC: A follower in the same IDC is preferentially selected as the migration source. If only a leader is available, the leader is selected. Region: A follower in the same region is preferentially selected as the migration source. If only a leader is available, the leader is selected.
data_disk_write_limit_percentage	New	The data disk usage that triggers an error for user write requests. When the specified value is reached, you can drop tables to urgently release storage space to avoid cluster faults. It is a cluster-level parameter. The value of this parameter must be greater than that of data_disk_usage_limit_percentage. We recommend that you set this parameter to the value of the following formula: (1 - memstore_limit_size/data_disk_size) × 100%. The default value is 0, which specifies not to stop user write requests.
alert_log_level	New	The log level of the alert.log file. Valid values are INFO, WARN, and ERROR. The default value is INFO. It is a cluster-level parameter.
syslog_disk_size	New	The total disk space available for system logs. It is a cluster-level parameter. The default value is 0M, indicating that the entire disk is available for system logs, which is the same as in earlier versions.
syslog_compress_func	New	The compression algorithm for system log files. Valid values are none, zlib_1.0, zstd_1.0, and zstd_1.3.8. It is a cluster-level parameter. The default value is none, which specifies not to compress system log files.
syslog_file_uncompressed_count	New	The number of system log files that triggers log compression for a specific log type. It is a cluster-level parameter. The default value is 0.
enable_global_background_resource_isolation	New	Specifies whether to perform global CPU resource isolation for foreground and background tasks. It is a cluster-level parameter. The default value is False, indicating no isolation.
global_background_cpu_quota	New	The CPU quota available for background tasks when enable_global_background_resource_isolation is set to True. It is a cluster-level parameter. The default value is -1, which indicates that the CPU resources available for background tasks are not subject to cgroups.
net_thread_count	Modified	The number of network I/O threads. The default value is still 0 but the adaptation strategy changes. The more the CPU cores, the more the available threads.
ddl_high_thread_score	New	The number of DAG threads available for the major compaction of KV data of each tablet during DDL data completion. It is a tenant-level parameter. The default value is 6, which is consistent with that in earlier versions.
lob_enable_block_cache_threshold	New	If the size of LOB data to be read from OUTROW storage is smaller than or equal to the specified value, the system caches the microblock in which the LOB data is located to speed up the next query. It is a tenant-level parameter. The default value is 256K.
ob_default_lob_inrow_threshold	Modified	The maximum LOB size supported by INROW storage. The default value is changed from 4096 to 8192.

System variable changes

System variable	Change type	Description
activate_all_roles_on_login	New	Specifies whether to activate all roles upon user logon. It is a global system variable applicable to MySQL tenants.
ob_compatibility_control	New	The MySQL version with which OceanBase Database keeps compatible, when a product behavior conflict occurs. Valid values are MYSQL5.7 and MYSQL8.0. It is a global system variable applicable to MySQL tenants. The default value is MYSQL5.7. This system variable is set during tenant creation and cannot be modified after tenant creation.
ob_compatibility_version	New	The OceanBase Database version whose product behavior takes effect after normal product behavioral changes. It is a global system variable that controls product behavioral changes. The default value is the current cluster version for a new cluster, and is the previous version for an upgraded cluster. In V4.2.3, the default value is 4.2.1.0. You can modify the value.
ob_security_version	New	The OceanBase Database version whose product behavior takes effect after security product behavioral changes. It is a global system variable that controls product behavioral changes. The default value is the current cluster version for a new cluster, and is the previous version for an upgraded cluster. In V4.2.3, the default value is 4.2.1.0. You can change the value of this variable only to a later version, but the limitation does not apply to the ob_compatibility_version variable.
cardinality_estimation_model	New	The correlation model used by the optimizer for row estimation. It can take effect both at the global and session level. Valid values are INDEPENDENT, PARTIAL, and FULL.INDEPENDENT: This model assumes that predicates are completely independent of each other. Such assumption is used by optimizers of V4.2.2_CE and earlier. The joint selectivity of multiple filters is the product of multiplying the selectivity of each of the filters. When many filters are involved, the row estimate is small. PARTIAL: This model assumes that predicates are partially correlated. The joint selectivity of multiple filters is calculated through exponential backoff. FULL: This model assumes that predicates are completely correlated. The joint selectivity of multiple filters is subject to the smallest selectivity of a single filter. The row estimate obtained by using this model is high, which is extreme. The default value is PARTIAL.

Function/System package changes

Function/System package	Change type	Description
DECODE_TRACE_ID	New	Parses the trace_id field to obtain the IP address and port number of the OBServer node that initiates the SQL request.
CURRENT_ROLE	New	Shows the roles activated in the current session.
PASSWORD	New	Calculates and returns the password hash. Password expressions of MySQL 5.7 are supported.

Syntax changes

Syntax	Description
Statements are provided for specifying a set- or piece-level restore source.	You can view the backup set or piece required for restore to a specific timestamp or SCN and specify the path of a backup set or piece to restore to the specific timestamp or SCN:Statement for specifying restore source paths: ALTER SYSTEM ADD RESTORE SOURCE 'xxx';Statement for revoking an incorrect input of restore source paths: ALTER SYSTEM CLEAR RESTORE SOURCE;Statement for initiating a restore: ALTER SYSTEM RESTORE <$restore_tenant> UNTIL '<$restore_checkpoint>' WITH 'xxx';Statement for parsing the original path of the backup set or piece required for restoring to the specified timestamp or SCN: ALTER SYSTEM RESTORE FROM 'uri' UNTIL { TIME='timestamp' | SCN=scn } PREVIEW;Statement for previewing the original path of the backup set or piece required for restoring to the expected timestamp or SCN: SHOW RESTORE PREVIEW;
Statements are provided for LS replica management.	Statement for adding an LS replica Statement for dropping an LS replica Statement for converting the type of an LS replica Statement for migrating an LS replica Statement for modifying the number of Paxos members of an LS replica Statement for canceling an LS replica task
The table-level option auto_increment_cache_size is provided.	You can specify the table-level option auto_increment_cache_size in a CREATE TABLE or ALTER TABLE statement. For example, in create table t1 (...) auto_increment_cache_size=xxx; or alter table t1 set auto_increment_cache_size=xxx;, the default value of auto_increment_cache_size is 0, which means that no cache size is configured for auto-increment columns. In this case, the value of the tenant-level parameter is used as the cache size for auto-increment columns.
The table-level option table_mode is provided.	The table-level option table_mode once supported in V3.x is used again in V4.2.3. You can set different table_mode values for tables to specify fast freeze and adaptive major compaction strategies triggered at different frequencies to resolve the buffer table issue. Here is an example: create table t1 (c1 int) table_mode = 'normal/queuing/moderate/super/extreme';
Hints are provided for specifying whether to enable parallel write at the DAS layer for DML statements.	The enable_parallel_das_dml and disable_parallel_das_dml hints are provided for respectively enabling and disabling parallel write at the DAS layer for DML statements. When the tenant-level hidden parameter _enable_parallel_das_dml is set to True:Scenario 1: The hint combination /+enable_parallel_das_dml enable_parallel_dml parallel(n)/ forcibly enables parallel write at the DAS layer for DML statements and adopts the DOP specified by parallel.Scenario 2: The hint combination /+enable_parallel_dml parallel(n)/ enables PDML preferentially. If PDML is not supported, parallel write at the DAS layer is enabled.Scenario 3: When a DOP is forcibly specified for a session, the behavior is the same as that in Scenario 2.
Role-related syntaxes are provided.	Syntax for creating roles: CREATE ROLE [IF NOT EXISTS] role [, role ] ...Syntax for dropping roles: DROP ROLE [IF EXISTS] role [, role ] ...Syntax for granting roles to users: GRANT role [, role] ... TO user_or_role [, user_or_role] ... [WITH ADMIN OPTION]Syntax for revoking roles from users: REVOKE [IF EXISTS] role [, role ] ... FROM user_or_role [, user_or_role ] ... [IGNORE UNKNOWN USER]Syntax for setting the roles enabled by default upon user logon:SET DEFAULT ROLE {NONE &#124; ALL &#124; role [, role ] ...} TO user [, user ] ...ALTER USER [IF EXISTS] user DEFAULT ROLE {NONE &#124; ALL &#124; role [, role ] ...}Syntax for setting the roles to be activated for the current session: SET ROLE { DEFAULT &#124; NONE &#124; ALL &#124; ALL EXCEPT role [, role ] ... &#124; role [, role ] ...}Syntax for showing the roles granted to a user: SHOW GRANTS FOR user USING role
Syntaxes are provided for granting and revoking column-level privileges.	Syntax for granting column-level privileges: GRANT [priv [(col_list)] ]+ ON [db_name.]table_name TO user [with grant option];Syntax for revoking column-level privileges: REVOKE [priv [(col_list)] ]+ ON [db_name.]table_name FROM user [with grant option];
Statements are provided for changing the replicated table attribute.	Statement for converting a normal table into a replicated table: ALTER TABLE t1 DUPLICATE_SCOPE = 'CLUSTER';Statement for converting a replicated table into a normal table: ALTER TABLE t1 DUPLICATE_SCOPE = 'NONE';

Recommended versions of tools

The following table lists the recommended versions of tools for OceanBase Database V4.2.3_CE.

Tool	Version
ODP	V4.2.3
OCP	V4.2.2_CE_HF1
OBD	V2.8.0
All in One	V4.2.3
ODC	V4.2.4-bp1
OBCDC	V4.2.3
OMS	V4.2.3_CE
OBClient	V2.2.3
LibOBClient	V2.2.3

Upgrade notes

• You can upgrade OceanBase Database V4.2.1_CE_BP2 and earlier V4.x_CE versions to V4.2.3_CE_Beta online through a valid upgrade path. You cannot upgrade OceanBase Database V4.2.1_CE_BP3 and later BP versions to V4.2.3_CE_Beta.
• You can upgrade OceanBase Database V4.2.2_CE to V4.2.2_CE_BP1 first and then to V4.2.3_CE_Beta online.
• When you upgrade OceanBase Database V4.2.1_CE and V4.2.2_CE to V4.2.3_CE_Beta, you must set the system variable _nlj_batching_enabled to False. You can set the variable to True after the upgrade.
• If your cluster contains a large number of tablets, it takes much time to restart OBServer nodes during the upgrade. It takes about 20 minutes to restart OBServer nodes when the cluster contains more than 3 million tablets. If a large number of tablets are involved, you need to reserve sufficient time for the upgrade.
• We recommend that you upgrade OBServer nodes first and then ODP.
• Major compactions and DDL operations are prohibited during the upgrade and will resume normal after the upgrade.

Acknowledgments

In the release of this version, we want to extend our acknowledgments to Qiu Yonggang @qiuyg3 from the China Unicom Software Research Institute team for his contributions to the oblogminer feature.

版本信息

项目	描述
发布日期	2024-4-29
版本号	V4.2.3_CE_BETA
Commit 号	c129d71
OBServer RPM 版本号	oceanbase-ce-4.2.3.0-100000112024042411

版本定位

V4.2.3_CE_BETA 为 V4.2.x_CE 系列最新版本，支持了一系列 MySQL 兼容特性，如角色、列级权限、Union Distinct RCTE 及大量小功能兼容。在多个场景优化了系统性能，如优化多模类型读写计算和 OBKV 处理效率，扩展 Batch DML 的并行执行能力，提升备份/网络备库性能，解决自增列/Sequence 在 Order 模式下的性能问题等。同时，备份恢复扩展支持了 S3 及兼容 S3 协议的对象存储（如：OBS，GCS）作为备份目的端。在资源优化方面，进行了一系列优化包括 DDL 临时结果空间优化、旁路导入能力提升和全局前后台 CPU 资源隔离等。新版本也实现了多个业务期待的易用性功能，如用于日志分析和误操作识别的 ObLogMiner 功能，应对 Buffer 表性能问题的合并策略选择，提供模糊绑定计划和限流方式的 Format Outline，用于识别无用索引的索引使用监控特性，提供更高可读性的面向数据库管理员的 alert.log 系统日志，增加日志流副本管理的用户命令，提供物理恢复进度展示能力。更详细的 PX 诊断数据以及 ASH Report 的节点级分析等，有效提高系统易用性。

推荐用于测试，暂不建议用于生产。

关键特性说明

内核增强

• 查询解析能力增强

  SQL 中存在个数特别多的 INLIST 、AND/OR、UNION 时，往往会产生超深语法树解析，进而造成内存消耗过大、栈不足、耗时过长的问题。新版本重写这些极端场景的查询解析实现，降低了解析阶段资源消耗，增强了极端 SQL 执行的稳定性，同时提升了 SQL 解析性能。例如 select sum(c1) from t1 where c1 in (1, 2, 3, ...N) 中 N = 20 万的查询场景，V4.2.3_CE 版本的解析性能相对 V4.2.2_CE 版本可以提升 20%~30%。

• 查询改写能力增强

  V4.2.3_CE 版本优化器从多个方面优化了查询改写逻辑，例如：

  • 表达式规范化强化：历史版本在 Resolver 解析阶段会对多种表达式进行规范化（Canonicalize），例如将 A and (A or B) 改写为 A ，但未覆盖到 SQL 改写阶段新产生的不规范表达式。新版本引入改写阶段的表达式规范化流程，确保将表达式改写到最简化状态。

  • 条件聚合函数改写：我们将形如 select c1, sum(case when c2 = 0 then c3 else 0 end),sum(case when...)...,... from t1 group by c1, 聚合目标不是某个确定的表达式，而是在运行时根据分支选择条件的判定结果选择出的表达式，称为条件聚合函数。新版本支持了条件聚合函数改写，支持将聚合函数压入每个分支的选择目标上，减少 case when 表达式和聚合函数的计算次数，提升相关场景的执行性能。

  • 多 Min/Max 改写：在 scalar group by 中，若查询中的聚合函数只有 Min/Max，且 Min/Max 的列上有索引，可以将其改写成 order by xxx limit 1，利用索引序消除 Order By 并将 Limit 1 下推到 TABLE SCAN 上，从而减少对数据的读取和排序。对于 SQL 中有多个 Min/Max 存在的场景，历史版本采取了全表扫描方案，新版本考虑将多个 Min/Max 改写成多个子查询，每个子查询都利用索引直接获取到最大值或最小值，以此来避免全表扫描和排序，优化查询性能。

  • 常量 UNION 改写 Values Table：业务较常使用多个 UNION ALL 的组合形式表示一个常量表，在子查询数量过多时，会有较多 CPU 和内存消耗。新版本对常量类型的 UNION ALL/UNION 改写成对 Values Table 的查询，显著降低硬解析阶段的资源消耗。

  • SEMI JOIN 拆分优化：EXISTS 或 IN 子查询中存在多表关联且没有连接条件时，执行计划可能会出现开销较大的笛卡尔积。新版本支持对没有连接条件的表做 SEMI JOIN 拆分改写，避免笛卡尔积开销。

• 计划选择优化

  V4.2.2_CE 版本实现了新版 Query Range 抽取逻辑，扩展了谓词下压场景，使向量形式谓词的 Range 抽取更加精准，也解决了历史版本 Query Range 的一些内存放大的场景问题。V4.2.3_CE 版本在此基础上修改了 not in 表达式生成 Query Range 的方式，优化了 Range 抽取性能；支持 Range Graph 裁剪，减少最终 Query Range 的抽取数量，降低内存消耗；扩展 INDEX Hint，如指定 /*+index(t1 k1 1)*/ 时，限制只对 k1 索引的第 1 列进行 Query Range 抽取。同时，新版本也支持了 Interesting Ordering 索引路径按规则裁剪，Limit 重计算等策略优化，解决 order by limit 场景因计划选择不优导致的性能问题。

• 自适应代价模型

  OceanBase 数据库历史版本代价模型是使用内部机器测算的常量参数来代表硬件系统统计信息，通过一系列公式与常量参数来描述每个算子的执行开销。而真实的业务场景中，不同硬件环境可能具备不同的 CPU 时钟频率、不同的顺序读或随机读的速度、不同的网卡带宽等，可能存在代价估算偏差，这些偏差会使得优化器无法在不同的业务环境总是生成最优计划。新版本优化代价模型实现，支持通过 DBMS_STATS 包来收集或设置系统统计信息系数，以达到代价模型自适应硬件的目的。同时也提供了 DBA_OB_AUX_STATISTICS 视图，用于展示当前租户的系统统计信息系数。

• 复制表属性变更

  复制表是 OceanBase 数据库 V4.2.0 版本开始支持的一种特殊表。这种表可以在任意一个“健康”的副本上读取到数据的最新修改。如果需要转换复制表为普通表，或转换普通表为复制表，V4.2.3_CE 版本之前需要重新建表导数，操作较耗时和复杂。新版本提供复制表属性变更功能，可通过 ALTER TABLE 命令实现表属性转换，降低用户操作成本。

• 产品行为兼容版本控制

  为了减少因为新版本行为变更导致升级后某些老版本特性使用报错的问题，OceanBase 数据库 V4.2.3_CE 版本新增产品行为兼容版本控制功能，支持通过 ob_compatibility_version、ob_security_version 系统变量，分别控制普通行为变更（不报错 -> 报错场景）、安全行为变更（有权限 -> 无权限）是否生效，一个租户中的兼容功能或安全功能按照哪个 OceanBase 版本来兼容，取值范围为 "4.2.3.0" 等发行版本。例如一个功能在 V4.2.4 版本发生了产品行为变更，当变量取值为 4.2.3.0 时，使用旧行为；取值为 4.2.4.0 时，使用新行为。通常升级场景不会自动推高该版本号，需要新版本行为时，请升级后，了解新版本行为并测试后，修改为和当前版本一致的版本号。该方案仅用于控制未来新增的产品行为变更，无法控制已经发版的存量产品行为变更。

  V4.2.3_CE 版本通过 ob_compatibility_version 控制的产品行为有：

  • ob_compatibility_control = MYSQL5.7 时，REPLACE('abd', '', null) 行为由兼容 MySQL 8.0 修改为兼容 MySQL 5.7。
  • UPDATE/DELETE statement 在 Limit 中禁用 Offset。
  • 当投影项是单独的一个 null 值的时候，返回的表头会被修改为 NULL。
  • 限制用户变量最长 64 字符。

  通过 ob_security_version 控制的产品行为有：

  • Outline、Sequence 增加权限管控。
  • CREATE TABLESPACE 权限。

MySQL 兼容

• 角色管理

  OceanBase 数据库 V4.2.3_CE 版本兼容了 MySQL 8.0 的角色管理功能，通过角色来管理维护一组权限，可以更方便地对某一类用户进行授权和回收。与普通用户类似，角色可以被授予或回收权限，也可以被授予或回收其他角色。用户可以被授予多个角色，但仅能使用激活状态角色中的权限。有关角色管理的更多信息，参见 角色管理。

• 列级权限

  V4.2.3_CE 版本新增 MySQL 列级权限功能，可以用于控制用户是否有权限对某张表的某几列进行 SELECT、INSERT 或 UPDATE。

• Outline、Sequence 权限管控

  历史版本缺少对创建维护 Outline、Sequence 的权限管控，新版本复用 MySQL CREATE、ALTER、DROP 权限，控制 Outline、Sequence 的创建、修改和删除。

• 自增列切主跳变优化

  OceanBase 数据库在 V4.x 版本支持了创建 ORDER 模式的自增列，更好地兼容了 MySQL 自增列行为。但是 V4.2.3_CE 之前版本在出现切主时，仍会造成自增列跳变。考虑到很多切主场景是用户主动发起的流程，并非异常场景，所以新版本优化为主动切主时仍保持自增列连续性，降低自增列跳变概率。有关自增列的更多信息，参见 自增列。

• 自增列改小

  V4.2.3_CE 之前的版本，仅支持通过 ALTER TABLE 将 AUTO_INCREMENT 值改大，不支持改小，行为上和 MySQL 不完全兼容。新版本补充支持了 AUTO_INCREMENT 值改小的功能，指定新值大于自增列已存在的最大值时，可以设置成功且生效；指定新值小于或等于自增列已存在的最大值时，也可以设置成功，但 AUTO_INCREMENT 会自动调整为自增列已存在的最大值的下一个值。

• Union Distinct RCTE

  MySQL 8.0 支持了 CTE 的 Recursive Union All 及 Recursive Union Distinct 功能。OceanBase 数据库从 V3.2.3 版本开始支持 MySQL 模式的 Recursive Union，但仅支持 Recursive Union All，V4.2.3_CE 版本扩展兼容了的 MySQL Recursive Union Distinct 功能，保证输出数据的唯一性。同时，新版本还加强了 Recursive Union All 功能，支持可使用内存不足时进行数据落盘。有关 Union Distinct RCTE 的更多信息，参见 通用表表达式。

• 区分 MySQL 5.7/8.0 兼容版本

  MySQL 5.7 与 8.0 在部分场景下存在产品行为冲突，如 replace('a','',null) 在两个版本的输出结果不同。OceanBase 数据库 V4.2.3_CE 版本新增租户级初始化变量 ob_compatibility_control，用于在创建租户时指定存在产品行为冲突的情况是兼容 MySQL 5.7 版本，还是 8.0 版本。租户创建后不可修改，两种模式下均可使用不存在行为冲突的 MySQL 5.7/8.0 的特性超集。

• 语法/变量/视图支持

  为了支持 MySQL 生态的各类工具或框架无需修改即可平滑迁移到 OceanBase 数据库，V4.2.3_CE 及之后版本会针对 MySQL 5.7 全量功能，对 OceanBase 数据库还未兼容细节的部分、不适用的能力，逐步进行兼容或 Mock 处理。

  V4.2.3_CE 版本支持的兼容能力如下：

  • 支持数据类型 SERIAL。
  • [MySQL 8.0] 支持创建触发器时指定 IF NOT EXISTS 语法。
  • SQL 语句中支持使用 \N 代表 NULL。
  • 兼容 MySQL，update/delete ... limit 语句禁用 offset 子句。
  • 兼容 MySQL 5.7 password 函数。
  • 兼容 MySQL DROP USER IF EXISTS 语法。
  • 兼容 MySQL，用户变量标识符长度限制 64 个字符。
  • 兼容 CREATE TABLE ... [IGNORE | REPLACE] SELECT statement。
  • SELECT 语句支持 STRAIGHT_JOIN 。
  • 支持 MySQL CREATE TABLESPACE 权限。
  • 支持授予、回收、查看 MySQL SHUTDOWN、RELOAD 权限，实际不生效。
  • DML 忽略 PRIORITY 不报错，不生效。
  • INSERT 忽略 LOW_PRIORITY、 HIGH_PRIORITY 不报错，不生效。
  • UPDATE 忽略 LOW_PRIORITY 不报错，不生效。
  • DELETE 忽略 LOW_PRIORITY、QUICK 不报错，不生效。
  • REPLACE 忽略 LOW_PRIORITY 不报错，不生效。
  • SELECT 忽略 SQL_SMALL_RESULT、SQL_BIG_RESULT、SQL_BUFFER_RESULT、HIGH_PRIORITY 不报错，不生效。
  • 兼容 MySQL 5.7，忽略降序索引语法不报错，不生效。
  • 兼容 MySQL 5.7，忽略 CREATE TABLE 中 Column 后的 reference_definition 语法，不报错，不生效。
  • 忽略建表建索引指定的 KEY_BLOCK_SIZE option，不报错，不生效。
  • FLUSH PRIVILEGES：语法不报错，实际不适用。
  • Show Profile/Profiles：语法不报错，实际不生效。
  • SHOW FUNCTION/PROCEDURE CODE：语法不报错，实际功能未支持。
  • 支持 INFORMATION_SCHEMA.PROFILING 表结构。
  • 新增 MySQL 变量（debug、debug_sync、innodb_change_buffering_debug、innodb_compress_debug、innodb_disable_resize_buffer_pool_debug、innodb_fil_make_page_dirty_debug、innodb_limit_optimistic_insert_debug、innodb_merge_threshold_set_all_debug、innodb_saved_page_number_debug、innodb_trx_purge_view_update_only_debug、innodb_trx_rseg_n_slots_debug、stored_program_cache、profiling、profiling_history_size、innodb_stats_persistent）。变量不产生实际效果，可查询可设置。设置后不会报错，也不会生效，产生 Warning。
  • 兼容 MySQL 5.7 通信协议 COM_PROCESS_INFO、COM_PROCESS_KILL；支持通信协议 COM_REFRESH、COM_DEBUG，实际不生效。

多模特性

• MySQL GIS

  OceanBase 数据库 V4.1.0 版本开始支持 GIS 数据类型及部分空间对象相关的表达式，后续逐渐增加了空间数据存储和计算分析的能力。新版本 MySQL 模式下扩展支持 PostGis_ST_GeoHash 表达式，用于计算 Geometry 的 Geohash 编码；扩展 PostGis_ST_MAKEPOINT 表达式，用于根据坐标创建 2D 和 3D 的 Geometry Point；新增支持 MySQL ST_ASGEOJSON 表达式，用于将 Geometry（WKB）按一定的格式转成 Json（Binary）。

• JSON/XML/GIS 内存优化

  新版本对 JSON 表达式执行过程中内存占用进行了优化，对 JSON 类型进行查询操作（JSON_KEYS、JSON_LENGTH、JSON_SEARCH 等）时，内存占用减少至 1 倍；对 JSON 类型进行输出操作（JSON_PRETTY、JSON_UNQUOTE 等）时，内存占用减少至 3 倍左右。在插入更新 XML 和输出 XML 的场景也进行了内存优化。输出 XML 操作（GetClobVal、XmlCast 等）内存占用减少到 2~3 倍左右；插入更新 XML （UpdateXml、InsertChildXml 等）时减少了不必要的解析。

  新版本同时对 GIS 类型数据输入、输出以及部分查询场景进行了内存优化。GIS 输入（st_geomfromtext/st_geomfromwkb/st_geogfromtext/geometrycollection 等）内存占用减少到 3 倍左右；GIS 输出（st_astext/st_aswkb 等）内存占用减少到 1 倍左右；查询场景（st_crosses/st_overlaps/_st_touches 等空间关系查询表达式、st_geometrytype/st_iscollection 等 GIS 属性获取表达式）也明显降低了内存放大程度。

• GIS 空间关系计算性能优化

  新版本优化了 ST_INTERSECTS/ST_CONTAINS/_ST_COVERS/ST_WITHIN 等空间关系计算表达式的计算性能。

  • 在全量查询窗查询场景，Point 的 st_intersects 与 PG 持平，且略优于 PG；其余测试场景的平均 RT 均不到 PG 的一半，其中 Linestring 的 Intersect 用时只有 PG 的 1/5 ，相比 OceanBase 数据库历史版本均有数量级的提升。

  • 在大查询窗场景，所有场景的平均 RT 均明显优于 PG，其中 Linestring 的 Intersect 用时仅有 PG 的 1/7，且相比 OceanBase 数据库历史版本均有数量级的提升。

  • 在小查询窗场景，Contain 计算会优于 PG，其中 Point 的 Contain 用时是 PG 的 1/4 左右，Linestring Contain 是 PG 的 80%。但 Intersect 的计算性能在 Linestring 场景仅与 PG 持平，在 POINT 场景用时是 PG 的 2 倍左右。

  有关空间函数的更多信息，参见 空间函数。

• OUTROW 存储的 LOB 读写性能优化

  V4.2.3_CE 版本重点对 OUTROW 存储的 LOB 进行了性能优化。

  • Table Scan 场景，相对历史版本，新版本中小 LOB 性能可以提升近 10 倍，大 LOB 可以提升至少 2 倍。但该场景相对 INROW 仍然慢一些。
  • Point Select 场景，新版本小 LOB (8K 以下)，OUTROW 的 QPS 比 INROW 低 20% 以内，中等以上大小的 LOB（32K 以上），OUTROW 的 QPS 比 INROW 低 5% 左右。在 LOB 不参与计算的场景，LOB OUTROW 存储对查询性能会更优。
  • Point Update 场景，相对历史版本，新版本 OUTROW 性能平均提升 2 倍。

  有关 OUTROW 存储的 LOB 的更多信息，参见 lob_enable_block_cache_threshold。

OBKV 增强

• OBKV 全局索引

  V4.1.0 版本开始，OBKV 支持了本地索引。新版本增加支持 OBKV 的全局索引功能，Insert、Update、Delete、insert_or_update、Replace、Increment/Append 等 DML 操作均适用于全局索引表，并支持了 Query 、异步 Query 指定全局索引查询。

• OBKV 性能优化

  OBKV 是一款集成在 OceanBase 数据库内部的低时延、高吞吐的 KV 存储类产品，在不需要通过 SQL 进行复杂逻辑处理的场景，可以跳过 SQL 模块直接调用存储接口，以获得极致性能。通过业务及内部端到端的测试分析，发现历史版本 KV 模型层性能开销还有进一步降低空间。因此新版本在以下多个场景，进行了性能优化：

  • 新增 Put 功能：Put 用于覆盖写场景，覆盖写跳过了主键冲突检查路径，强制覆盖旧记录。

  • 支持组提交：组提交是指服务端主动攒批，执行批量操作的功能。开启组提交后，服务端会将 Put 或 Get 操作按照执行计划分组，并根据负载情况自动调节组大小，按组执行。用户可通过租户级配置项 enable_kv_group_commit 开启组提交，开启后预期能够在不对 RT 造成明显影响情况下，提高 30%~50% 的 OPS。

  • 优化 Batch 批处理：OBKV 的批操作接口避免了单操作接口中每次事务开启关闭的耗时，在一定程度上提高了操作效率。但每个操作依然需要构造算子、打开算子、提交 DAS 任务、关闭算子和算子析构等步骤，存在一定的执行时间损耗。为了进一步提高批操作接口的执行效率，新版本在算子级别实现了多操作的批处理，减少 n-1 次的算子构造析构和 DAS 任务开销，有效提升整体性能。

  • 优化 Schema 获取逻辑：内部进行 Schema 获取的操作调整为语句级别，仅在语句执行时获取一次 Simple Schema，避免频繁获取 Schema 带来的性能损耗。同时将 Schema 的一些高频获取信息如 table_id、column_id 等缓存到计划中，实现了轻量级的 KV Schema Cache。

  • Batch 只返回一个结果：Multi Set 场景下，历史版本针对每个操作都返回了一个结果，新版本在设置 batchOps.setAtomicOperation(true) 和 batchOps.setReturnOneResult(true) 后，Multi Insert/Multi Put/Multi Delete 操作优化为只返回一个成功或者失败的结果，缩短交互周期。

性能提升

• DML 性能优化

  OceanBase 数据库已经支持通过 PDML 的并行执行机制来提高对大型表和索引执行插入、更新、删除等操作的性能。但是仍然存在部分场景是 PDML 暂未覆盖的，如 replace into ...、insert ... on duplicate key、insert all ...、merge into ... 更新主键、Insert 包含自增列、多表 Update 等。以上场景在之前版本会使用单线程写入方式，在数据量较大时响应时间较长，影响客户体验。为了解决大数据量的这些场景的性能瓶颈，V4.2.3_CE 版本新增支持 DAS 层并发写入能力，通过类似 PDML 的并发控制机制，显著提升 DML 执行性能。更多信息，参见 与并行执行相关的 Hint。

• 备份性能优化

  OceanBase 数据库备份过程中通过连续性校验确保基线版本大于转储版本以保证数据完整，但连续性检查涉及读取备份介质上的数据并执行带锁操作，影响了备份性能。新版本优化了备份过程中连续性校验操作的性能开销，支持通过 ha_low_thread_score 控制备份使用的线程数， 有效提高备份性能。更多信息，参见 数据备份。

• 网络备库性能优化

  V4.2.3_CE 版本优化了 RPC 及网络框架，降低了主备库之间的日志传输耗时；同时优化了备库受控拉日志流程，减少了备租户拉日志受控的频率，提高了主备之间的日志同步性能。

• 存储过程 DDL 执行编译结果缓存和落盘

  V4.2.2_CE 版本新增执行期存储过程编译落盘功能，解决了执行期存储过程获取 PL Cache 缓存失败，从而重编译引起的性能问题。但是存储过程自身做过 DDL 变更后，后续执行存储过程依旧需要重新编译，这个场景依然存在性能优化空间。V4.2.3_CE 版本补充支持存储过程 DDL 执行成功后将编译结果缓存到 PL Cache 并落盘的行为，后续执行存储过程时，提升直接命中 PL Cache 缓存的概率，进一步提高存储过程执行性能。

• 并行 DDL 扩展

  并行 DDL 与串行 DDL 互斥，在并行 DDL 与串行 DDL 交替执行时性能不优。随着版本演进，OceanBase 数据库也在不断扩展允许并行执行的 DDL 类型。V4.1.0 版本实现了并行 DDL 框架，并支持了 TRUNCATE 语句的并行执行；V4.2.1 版本支持了并行 CREATE TABLE；V4.2.2_CE 版本支持了并行 COMMENT 与并行 CREATE INDEX。在此基础上，V4.2.3_CE 版本扩展支持并行 CREATE VIEW，进一步提升 OMS 的结构迁移速度。

• 自增列 Order 模式性能优化

  自增列 Order 模式下，需要保持数据插入的连续性。为了保证分布式架构下自增列的有序，在自增列 INSERT 时会将该值持久化到内部表，高并发情况下，响应时间会比较长。V4.2.3_CE 对该场景做了优化，通过 Cache Size 预取有效降低内部表访问次数，从而显著提升性能。

• Sequence Order 模式性能优化

  OceanBase 数据库已支持指定 order + cache 属性来创建全局连续生成的序列，但早期的实现中会忽略 Cache 属性，等同于 order + nocache，在高并发场景会有明显性能问题。V4.2.3_CE 版本对该场景进行了优化，通过选取中心节点来支持 order + cache 生成连续序列，真正意义上实现 Cache 属性，显著提升高并发场景执行性能。

• Show Table Status 性能优化

  历史版本 show table status from ... like ... 场景性能较差，未利用 table_name 相关索引。新版本针对存在单表过滤条件的场景，显著提升查询性能。

• CREATE 语句支持 HINT

  在 V4.2.3_CE 版本中，新增对 CREATE TABLE 语句的 Hint 支持 /*+ parallel(N) */ 类型。该 Hint 适用于 CREATE TABLE ... AS SELECT ... 的场景，并可以控制在表创建时数据查询和写入操作的并行度。

资源优化

• OBServer 到 ODP 的链路压缩功能

  实际业务场景中，存在应用和数据库服务跨城部署的情况，距离较远、带宽有限，但仍然有大数据量读写的需求。为了降低大数据量传输带来的带宽消耗，该版本支持了 OBServer 到 ODP 的链路压缩功能，需配合 ODP V4.2.3_CE 或以上版本开启使用。

• DDL 临时结果空间优化

  在 DDL 过程中，常常会将临时结果暂存到物化结构中。在创建索引时，需要对数据表进行扫描并将数据插入到索引表中，此过程中可能需要对扫描出来的数据进行排序。如果内存空间不足，就会将当前内存中的数据暂存到物化结构中，以释放内存供后续的扫描使用。最后，对这些暂存在物化结构中的数据进行归并排序。

  针对这些问题，新版本 DDL 处理流程对数据流进行了优化。首先，它剔除了不必要的冗余结构，简化了数据流；其次，它引入了对临时结果落盘的编码压缩功能。这些改进为两个场景都带来了好处：它们显著降低了 DDL 操作期间临时结果对磁盘空间的占用，从而更加高效地利用存储资源。

• 旁路导入优化

  • 支持临时文件压缩：旁路导入过程中会产生临时文件，当导入的数据量比较大时，临时文件占用的磁盘空间比较大，需要关注磁盘的占用空间，避免旁路导入产生的临时文件将磁盘空间打爆，当旁路导入的 并发度 >= 8 时，系统会对产生临时文件做压缩，减少临时文件的空间占用。
  • 支持通配符方式多文件导入：在旁路导入过程中，新增了对通配符方式的多文件导入的支持。可以使用通配符 * 和 ? 来同时导入匹配特定模式的多个文件，从而提高导入效率和便利性。

  有关旁路导入的更多信息，参见 旁路导入。

• 全局 CPU 前后台任务隔离

  在高性能计算环境中，资源的合理分配与隔离对于确保系统稳定性和提升效率具有决定性作用。有效的资源隔离策略可以预防任务间的资源争夺和相互干扰，从而提升资源利用效率和整体服务质量。目前，OceanBase 数据库已实现了通过租户 Unit 规格来配置租户间的资源隔离，通过 DBMS_RESOURCE_MANAGER 系统包来配置租户内的资源隔离，涵盖 CPU 和 IO 两大重要资源。但对于不希望后台任务对前台请求造成严重 CPU 竞争的业务，在全局层面上对后台任务进行隔离是一个更好的选择。OceanBase 数据库 V4.2.3 版本支持了全局 CPU 前后台任务隔离能力，可以在整体层面上限制后台任务的可用资源，相对租户内使用 DBMS_RESOURCE_MANAGER 单独配置更加方便易用。有关全局 CPU 前后台任务隔离的更多信息，参见 配置 cgroup。

• 分区均衡策略优化

  OceanBase 数据库 V4.2.0 版本开始支持分区均衡功能，但在部分场景下还存在不完全均衡的问题。V4.2.3_CE 版本优化了分区均衡策略，在建分区表时支持了连续分区打散；当用户表存在 Longtext/Lob 列、局部索引时，分区磁盘均衡时也会计算关联表，使得磁盘使用更加均衡。

• 系统日志压缩

  业务流量过大时，系统日志刷新的会比较快，保留时间较短可能影响问题排查。新版本增加系统日志压缩功能，支持对 observer.log、rootservice.log、election.log、trace.log 等日志文件，每类超过 syslog_file_uncompressed_count 个数时，采用 syslog_compress_func 设置的压缩方法，对最早日志进行压缩。当日志使用总空间接近 syslog_disk_size 设置的磁盘空间上限时，开始删除最早生成的日志文件，回收空间。磁盘空间不变的情况下，开启 zstd 压缩后，预计可以存储不开启压缩时 20 倍的日志量。更多信息，参见 日志压缩与解压。

可靠性提升

• 迁移复制源端选择优化

  历史版本选择迁移复制的源端时没有优先考虑同 Zone、同 IDC (同机房)、同 Region (同城)的副本，可能出现远距离迁移复制性能不优的问题。并且源端可能会选择 Leader 节点，业务请求量较高时，可能导致业务请求和迁移任务较慢的问题。V4.2.3 版本根据地域将迁移复制的目标端和源端副本所在Server的关系划分为：同 IDC、同 Region 不同 IDC、跨 Region 三种区域，提供枚举配置项 choose_migration_source_policy，支持用户配置迁移复制选择源端副本的优先模式，以便优先考虑地理位置就近因素以及 Follower 副本来提升迁移效率、降低 Leader 压力。

• 数据盘满停写

  数据盘使用量过高时，当前不会停写用户请求，一直到内存因为无法转储写满，或者 Clog 盘写满后才会报错。这种情况下，需要通过紧急扩 Clog 盘或租户内存恢复。新版本在内核层面增加数据盘满停写功能，当用户数据盘水位线达到 data_disk_write_limit_percentage 配置后，用户写入请求会报错。通过删表、Transfer 或者扩磁盘方式降低数据盘使用比列后，用户写入操作可以自动恢复。

高可用增强

• 备份恢复支持 S3/OBS/GCS

  OceanBase 数据库的备份恢复功能支持两类存储介质：文件存储（NFS）和对象存储（OSS/COS）。新版本备份恢复扩展支持了 S3 及兼容 S3 协议的对象存储（如：OBS，GCS）作为备份介质，可以将 S3 及兼容 S3 协议的对象存储作为日志归档和数据备份的目的端，也可以使用 S3 及兼容 S3 协议的对象存储上的备份数据执行物理恢复。更多信息，参见 物理备份与恢复。

• SET/PIECE 级物理恢复

  实际业务中存在二次备份场景，会把数据备份集或归档日志手动搬迁到新的路径。OceanBase 数据库的物理恢复功能，限制了必须在租户的归档和备份路径中恢复，无法使用手动搬迁到新路径的备份数据，限制了恢复的灵活性。新版本增加了 SET/PIECE 级物理恢复功能，提供 add restore source 命令来加载新路径的数据备份集 SET 或日志归档 PIECE，允许按需恢复到指定时间。更多信息，参见 指定路径的恢复。

易用性提升

• OceanBase LogMiner

  V4.2.3_CE 版本新增 OceanBase LogMiner（简称 ObLogMiner）工具支持。ObLogMiner 是一款用来对 OceanBase 数据库进行日志分析的命令行工具，支持在线及离线的日志分析。ObLogMiner 通过 OBCDC 来拉取并解析 CLOG 日志，并将 OBCDC 输出的逻辑日志转化成易读的格式存储到指定位置。适用于以下场景：

  • 数据误操作：数据误操作可能由于多种原因出现，比如 WHERE 子句中的范围条件错误而误删除或更新了多余的行。通过使用 ObLogMiner 可以准确了解误操作的详细信息，以便于将数据库恢复到误操作之前的状态。
  • 数据分析：ObLogMiner 会将 CLOG 日志中的各种信息(事务、表结构等) 组织并友好地展示出来，用户可以借助 ObLogMiner 的输出结果进行多样的数据分析。也可以配合外表功能读取 ObLogMiner 的输出结果在数据库中进行查询分析。

  更多信息，参见 LogMiner 工具。

• Buffer 表自适应合并优化

  当用户在某张表上频繁地执行插入并且同时进行批量删除，或者有大量的并发更新操作时，可能会遇到一种现象：表中的数据行数并不大，但是查询和更新的性能出现明显下降。这种现象在 OceanBase 数据库中称为 Queuing 表(业务上有时又称 Buffer 表)效应。Buffer 表是 LSM-Tree 架构数据库都要面对的一类问题。LSM-Tree 架构下的删除操作在合并之前都只是逻辑上标记删除而非物理删除，当增量数据中存在大量标记删除的数据时，物理行数量将远多于逻辑行，从而造成严重的读放大现象，并影响优化器执行方案的生成。

  OceanBase 数据库在 V4.1.0 版本做到了自动识别哪些分区在一段时间内发生比较多更改，并对这些分区进行自适应 Compaction，但缺少人为控制手段。为了更灵活地解决 Buffer 表带来的性能下降问题，V4.2.3_CE 版本继续优化了 Buffer 表自适应合并特性，提供了 5 种档位的合并策略，允许用户根据业务场景为每张表设置不同的 table_mode，来应对 Buffer 表引起的读放大现象，从而提高系统长期运行下的 QPS 等性能指标。更多信息，参见 自适应合并。

• Format Outline

  OceanBase 比较早的版本就提供了通过创建 Outline 来绑定执行计划或指定 SQL 限流的功能（以下称为 Normal Outline），该功能对 SQL 文本的格式要求较为严格，除了可以参数化的部分之外，多一个空格、制表符或大小写不一致等情况都无法命中 Outline，SQL 格式不固定的场景易用性较差。V4.2.3_CE 版本新增 Format Outline 特性，提供了一种更为宽松的匹配规则。当用户创建 Format Outline 时，在 Outline 原有流程之前，系统会先做一次忽略大小写、空格等非语法定义符号的操作，归一化为标准格式，这使得归一化后得到同样 Format SQL Text 或 Format SQL ID 的用户请求都可以命中同一个 Format Outline。另外针对 INLIST 个数不固定的场景，如业务不固定下发 in(1,2)、 in(4,5,6)、in(7,8,9,10) 时 ，Normal Outline 精准匹配功能需要绑定 in(?,?)、in(?,?,?)、in(?,?,?,?) 三个 Outline 来控制执行计划；新版本的 Format Outline 功能只需绑定一次，系统会归一化为 in(?)，使得 INLIST 个数不同的多个 SQL 可以共享一个 Outline。当 Normal Outline 与 Format Outline 同时存在时，优先匹配 Normal Outline。更多信息，参见 CREATE FORMAT OUTLINE。

• 索引使用监控

  我们对数据库执行查询操作时，往往通过创建索引来优化查询性能。但随着数据表使用的时间增长，业务场景和操作人员不断增加，很可能会存在索引越建越多的问题。未使用的索引会浪费存储空间，也会加重 DML 操作的开销。对于这种情况，需要持续观察，删除无用的索引来给系统减负。但是仅靠人力很难识别哪些索引是无用索引，因此 OceanBase 数据库在 V4.2.3_CE 版本新增索引使用监控功能，用户可选择打开该功能并设置采样方式，在普通租户下会将符合规则的索引使用信息记录到内存，并以 15 分钟为周期刷新到内部表中，可通过 DBA_INDEX_USAGE 视图访问，以此来感知表上的索引是否有被引用，进而选择删除无用的索引表来释放空间。更多信息，参见 监控索引。

• 系统日志优化

  系统日志目录下，新增 ./alert/alert.log 日志文件，用于记录 DBA 关注的日志信息，初步解决 observer.log 日志量大、可读性差的问题。可通过 alert_log_level 集群级配置项设置日志级别，包括 INFO、WARN、ERROR 等。同时提供了系统外部表 sys_external_tbs.__all_external_alert_log_info 用于直接结构化查询 alert.log 日志信息。更多信息，参见 Alert 日志。

• 日志流副本管理

  V4.0.0 之前的版本，OceanBase 数据库提供了分区副本管理的一系列运维命令，比如为分区添加一个副本、为分区删除一个副本、对分区的一个副本进行类型转换等。V4.x 系列在设计上使用日志流代替了分区的概念，对标低版本的分区运维命令，V4.2.3_CE 版本重新设计实现了日志流副本级别任务的运维方式，提供了一系列语法，用于支持日志流副本的添加、删除、副本类型转换、迁移、修改日志流 Paxos 成员数量和取消容灾任务等能力，满足客户手动运维日志流副本的需求。更多信息，参见 副本管理。

• 物理恢复进度统计

  为了用户在使用物理恢复功能时可以了解恢复任务是否运行正常，获取恢复任务的进度情况，并预估恢复任务完成的剩余时间，在使用物理恢复功能时获得更好的体验，新版本增加物理恢复进度统计功能，可通过 CDB/DBA_OB_RESTORE_PROGRESS 视图随时查看恢复的实时进度。更多信息，参见 查看物理恢复进度。

• PX 诊断能力增强

  为方便分布式计划的问题排查，V4.2.3_CE 版本在 SQL Plan Monitor 中增加记录了算子运行时的内存使用量、磁盘使用量，整个执行过程的最大内存使用量、最大磁盘使用量等信息。并在 SQL Audit 中增加记录 QC 对应 trace_id 下的 MemTable、SSTable 行数信息，并支持汇总各 PX Worker 对应 trace_id 下的相关信息。同时也支持了 OBDiag 工具依据 trace_id 拉取最初报错机器日志的功能。

• PS 诊断能力增强

  V4.2.3_CE 之前的版本遇到 PS 句柄泄漏问题时，只能通过 GV$OB_PS_ITEM_INFO 视图查看全局信息，缺少 Session 级别的诊断方式。V4.2.3_CE 版本新增 [G]V$OB_SESSION_PS_INFO 系统视图，用于展示各个 Session 的 PS 引用信息，以便准确定位 PS 句柄泄漏等问题。

• ASH Report 节点级分析

  ASH Report 已支持生成集群级别的 ASH 分析报告，缺少节点级统计。如果想要查看某个节点上 ASH 数据，目前需要通过 SQL 脚本获取，这给单节点问题分析造成了一些不方便。V4.2.3_CE 版本增加 ASH Report 节点级分析能力，可通过 DBMS_WORKLOAD_REPOSITORY.ASH_REPORT 包函数中新增的两个参数（节点 IP 和 Port），指定单节点的 ASH Report 分析。更多信息，参见 ASH Report。

• 自增列 Cache Size 表级设定

  当前已支持通过指定租户级配置项 auto_increment_cache_size 来控制内存中自增列一次申请缓存的个数，对所有使用自增列的表生效。通常情况下，Cache Size 设置越大，性能会越好。但如果跳变次数很多，自增列字段取值上限又比较小，很可能导致自增列快速耗尽。V4.2.3_CE 版本新增自增列 Cache Size 表级设定功能，用户可以根据列类型/业务模型/业务流量等自定义配置不同表的缓存大小，从而做到跳变和性能的均衡。更多信息，参见 定义自增列。

• NETWORK_WAIT_TIME 统计项

  SQL AUDIT 和 SQLSTAT 中新增 NETWORK_WAIT_TIME 统计项，用于展示 NETWORK 类别的等待事件耗时，如同步 RPC、DAS 异步 RPC 锁等。在排查远程执行的慢 SQL 问题时，可以使用该统计项确认网络开销，辅助诊断。

• XA 事务监控统计项

  部分客户业务会采用 XA 协议来保证跨库事务提交的原子性。由于 OceanBase 数据库本身分布式架构特性，XA 语句的执行需要消耗一定的资源，该版本在 SYSSTAT 中新增 XA_TRANS_START_COUNT、XA_READ_ONLY_TRANS_TOTAL_COUNT、XA_ONE_PHASE_COMMIT_TOTAL_COUNT 等 30 余项统计项，便于在 XA 事务流量发生变化时，用户可以及时感知。更多信息，参见 监控项。

• Crash 日志中 SQL 信息打印场景完善

  Crash 日志中的 SQL 信息一方面可以用于问题排查，另一方面还可以被 OCP Agent 采集后发出告警。V4.2.3_CE 版本前，INNER SQL 主线程、用户 SQL 主线程 Crash 的情况下，Crash 日志里会打印 SQL 信息。但 REMOTE 线程、PX SQC 线程、PX WORKER 线程、DAS 远端执行线程 Crash 的时候日志里缺少 SQL 信息打印。新版本针对以上场景，支持了执行端获取 sql_id 或 sql_string 信息，如果执行端出现 Crash，将会在 Crash Error 日志中输出控制端的 SQL 信息，便于 Crash 原因定位。

• OBServer 支持 IPV6

  新版本支持 IPV6 格式的 Server IP，支持 SQL 客户端、RPC 客户端以 IPV6 地址连接，同时也支持了 IPV4 和 IPV6 节点的混合部署。支持 IPV4 集群升级，但不支持旧的 IPV4 类型的集群升级到 IPV6 类型。具体部署方式，参见 使用命令行部署单机集中式 OceanBase 数据库。

• GV$PLAN_CACHE_PLAN_EXPLAIN 视图支持仅指定 PLAN_ID 查询

  查询 [G]V$PLAN_CACHE_PLAN_EXPLAIN 视图时，必须同时指定 ip + port + tenant_id + plan_id 四个过滤条件才会返回数据，仅指定 plan_id 时返回结果为空，易用性较差。新版本支持了该视图的底层虚拟表扫描，允许仅指定 plan_id 进行数据查询，可准确返回查询结果。

• TRACE_ID 解析

  OceanBase 数据库通过 TRACE_ID 来标记 SQL 请求的全过程，用于关联监控指标或查询日志。TRACE_ID 包含了发起 SQL 请求的 OBServer 的 IP 和 Port 信息，但缺乏直接的解析方式。新版本增加 decode_trace_id 函数，用于解析 TRACE_ID 获取 IP、Port 相关信息。

兼容性变更

产品行为变更

新增如下变更：

功能	变更点说明
并行 COMMENT、并行 CREATE INDEX 升级后默认关闭	V4.2.2_CE 版本开始支持并行 COMMENT 和并行 CREATE INDEX 功能，默认开启。V4.2.3_CE 版本新建集群或租户时，也会默认开启并行 COMMENT 、并行 CREATE INDEX 以及 V4.2.3_CE 版本新增的并行 CREATE VIEW 功能。但 V4.2.2_CE 及之前的 V4.x 版本的租户升级到 V4.2.3_CE 版本后，以上三类并行 DDL 特性会变更为默认关闭状态，如有相关类型的 DDL 高性能需求，请通过设置隐藏配置项 _parallel_ddl_control 开启。例如，通过以下命令开启并行 COMMENT、并行 CREATE INDEX，关闭并行 CREATE VIEW 特性：alter system set_parallel_ddl_control='SET_COMMENT:ON,CREATE_INDEX:ON,CREATE_VIEW:OFF' tenant='xxx';
PS 协议参数数量超过 65535 时，由只返回 65535 个参数变更为报错	MySQL 协议有最多 65535 个参数的限制，如果参数数量超过 65535，会抛出错误。 OceanBase 数据库之前版本不是抛出错误，而是只返回 65535 个参数。 新版本兼容了 MySQL 行为，参数超过 65535 个时报错处理。
Rename Table 变更为加锁操作	历史版本中 Rename Table 是一个不需要加锁的 Online DDL 操作，事务跨 Rename Table 操作时，可能会出现非预期的问题。V4.2.3_CE 版本支持对 Rename Table 加表锁，并增加 Rename Table 期间的读写防御。
[g]v$plan_cache_plan_explain 支持仅指定 plan_id 进行数据查询	之前版本查询 [g]v$plan_cache_plan_explain 视图时，必须同时指定 ip + port + tenant_id + plan_id 四个过滤条件才会返回数据，仅指定 plan_id 时返回结果为空，易用性较差。新版本支持了该视图的底层虚拟表扫描，允许仅指定 plan_id 进行数据查询，可准确返回查询结果。
[g]v$ob_sql_audit 视图 tenant_id 字段含义变更	普通租户下产生的一些内部请求是基于系统租户开始初始化的，然后读数据时，切到普通租户去执行，这种情况下 tenant_id 是 1， effective_tenant_id 是当前租户 ID。 基于 OAS 的租户内有序采集需求，V4.2.3_CE 版本将可用作索引的字段 tenant_id 调整为与 effective_tenant_id 字段等价。
[g]v$ob_sql_audit 视图 sql_id 字段取值变更	V4.2.3_CE 版本之前，CALL 语句的 sql_id 为空字符串生成的 MD5 编码；匿名块语句的 sql_id 为未经参数化的原始 PL 代码字符串生成的 MD5 编码。V4.2.3_CE 版本将两者的 sql_id 都修改为参数化之后的语句生成的 MD5 编码。
[g]v$ob_active_session_history 字段取值变更	V4.2.3_CE 版本之前，MODULE、ACTION、CLIENT_ID 一直为 NULL。V4.2.3_CE 版本开始，这三列真实展示用户设置信息。（通过 DBMS_APPLICATION_INFO 设置 MODULE、ACTION，通过 DBMS_SESSION 设置 CLIENT_ID）。
未指定别名的 null 值投影列名称修改为 NULL	MySQL 会将未指定别名的 null 值（\\N, null, Null...）投影列的名称设置为 NULL，而出现在复合表达式中的 null 会保持原串。OceanBase 历史版本不会对 null 值投影列的名称做任何修改。新版本在 ob_compatibility_version = 4.2.3.0 或指定之后的 V4.2.x_CE 版本时，MySQL 模式下会修改为和 MySQL 相同行为。
禁止在 Delete/Update 语句中的 Limit 子句中使用 Offset 语义	新版本在 ob_compatibility_version = 4.2.3.0 或指定之后的 V4.2.x_CE 版本时，MySQL 模式下禁止在 Delete/Update 语句中的 Limit 子句中使用 Offset 语义。具体而言，OceanBase 原先支持的 update/delete...limit x,x 和 update/delete...limit x offset x 写法，现在需要语法报错，与 MySQL 行为保持一致。
REPLACE('abd', '', null) 返回结果按 ob_compatibility_control 分别兼容	新版本在 ob_compatibility_version = 4.2.3.0 或指定之后的 V4.2.x_CE 版本时，历史版本升级后或新建租户时取 ob_compatibility_control = MYSQL5.7 ，MySQL 模式下 REPLACE('abd', '', null) 行为由兼容 MySQL 8.0 修改为兼容 MySQL 5.7。
限制用户变量最长 64 字符	新版本在 ob_compatibility_version = 4.2.3.0 或指定之后的 V4.2.x_CE 版本时，MySQL 模式下用户变量长度由不设限修改为最长 64 字符。
Outline、Sequence 新增权限管控	新版本在 ob_security_version = 4.2.3.0 或指定之后的 V4.2.x_CE 版本时， MySQL 模式下需要授予 CREATE/ALTER/DROP 权限后，用户才可以创建和管理 Outline、Sequence。
创建和管理 TABLESPACE 新增 CREATE TABLESPACE 权限管控	新版本在 ob_security_version = 4.2.3.0 或指定之后的 V4.2.x_CE 版本时， MySQL 模式下需要授予 CREATE TABLESPACE 权限后，用户才可以创建和管理 TABLESPACE。
使用 IP 启动时移除与网卡名不匹配导致的启动失败	在启动 OBServer 时，如果无法找到与 local_ip 对应的本地网卡名称，新版本不会启动失败，而是会记录一条错误日志，并继续使用配置中的 devname 作为本地网卡名。

视图变更

新增如下变更：

视图	变更类型	变更说明
CDB/DBA_OB_RESTORE_PROGRESS	新增列	新增 RECOVER_SCN、RECOVER_SCN_DISPLAY、RECOVER_PROGRESS、TABLET_COUNT、FINISH_TABLET_COUNT、RESTORE_PROGRESS 6 列，用于展示物理恢复进度。
CDB/DBA_OB_LS_REPLICA_TASKS	新增列	新增 DATA_SOURCE_SVR_IP、DATA_SOURCE_SVR_PORT、IS_MANUAL 列，用于记录容灾任务执行时引用的数据源和容灾任务生成来源。
CDB/DBA_OB_LS_REPLICA_TASK_HISTORY	新增	新增系统视图，用于展示容灾任务执行历史。CDB 视图仅在 SYS 租户支持，DBA 视图在所有租户支持。
CDB/DBA_OB_AUX_STATISTICS	新增	用于展示每个租户的辅助统计信息。CDB 视图仅在 SYS 租户支持，DBA 视图在所有租户支持。
[G]V$SQL_WORKAREA	新增列	新增 DB_ID 字段，用于描述该请求的连接所属的数据库 ID。
[G]V$OB_SQL_AUDIT	新增列、字段含义调整、字段取值变更	新增 STMT_TYPE 字段，用于判断 SQL 类型。新增 NETWORK_WAIT_TIME 字段，用于展示网络类别等待事件的等待时间。新增 PROXY_USER 字段，用于代理用户登录场景展示代理用户名称信息。TENANT_ID 字段调整为与 EFFECTIVE_TENANT_ID 字段等价。SQL_ID 字段针对 CALL 语句执行的 PL 请求、通过匿名块执行的 PL 请求，展示语句真实生成的 MD5 编码。
[G]V$OB_PROCESSLIST	新增列	新增 PROXY_USER 字段，用于代理用户登录场景展示代理用户名称信息。
[G]V$OB_ACTIVE_SESSION_HISTORY	字段取值变更	V4.2.3_CE 版本开始，MODULE、ACTION、CLIENT_ID 三列真实展示用户设置信息。（通过 DBMS_APPLICATION_INFO 设置 MODULE、ACTION，通过 DBMS_SESSION 设置 CLIENT_ID）
[G]V$OB_SESSION_PS_INFO	新增	用于展示租户所有 Session 打开的 PS 信息。V$ 视图表示当前 OBServer，GV$ 视图表示所有 OBServer。在所有租户下支持。
CDB/DBA_INDEX_USAGE	新增	用于展示索引访问信息。CDB 视图仅在 SYS 租户支持，DBA 视图在所有租户支持。
V$OB_COMPATIBILITY_CONTROL	新增	MySQL 模式视图，用于展示所有可以按 OceanBase 数据库发行版本进行产品行为兼容控制的功能。
mysql.role_edges	新增	MySQL 模式视图，用于展示角色和用户的授予关系。
mysql.default_roles	新增	MySQL 模式视图，用于展示用户默认启用的角色。
mysql.columns_priv	新增	MySQL 模式视图，用于展示用户拥有的列级权限。
sys_external_tbs.__all_external_alert_log_info	新增	sys 租户下新增系统外部表，用于结构化查看 alert.log 日志信息。

配置项变更

配置项	变更类型	变更说明
enable_kv_group_commit	新增	新增租户级配置项，用于控制是否开启 OBKV 组提交功能，在开启的情况下，服务端会将操作按照执行计划分组，按组执行；在未开启的情况下，服务端将会一个一个执行。默认为 False，表示不开启。
choose_migration_source_policy	新增	新增租户级配置项，用于控制迁移源端的选择策略。提供 2 种选择：IDC：在同 IDC 的机器中优先选择 Follower 副本作为源端，若仅有 Leader 副本，则选 Leader 副本。Region：在同 Region 的机器中优先选择 Follower 副本作为源端，若仅有 Leader 副本，则选 Leader 副本。
data_disk_write_limit_percentage	新增	新增集群级配置项，用于控制数据盘达到设置的水位后，用户写入报错的问题，发现问题以后，可以通过删表的方式紧急释放空间，避免集群故障。该配置项应该大于 data_disk_usage_limit_percentage，开启时建议配置：(1-memstore_limit_size / data_disk_size)*100%。默认值为 0，表示不开启停止用户写入的功能。
alert_log_level	新增	新增集群级配置项，用于控制 alert.log 的日志级别，如 INFO、WARN、ERROR，默认为 INFO。
syslog_disk_size	新增	新增集群级配置项，用于控制系统日志可使用的总磁盘空间。默认为 0M，即兼容老版本默认行为，可使用整块磁盘。
syslog_compress_func	新增	新增集群级配置项，用于控制系统日志文件压缩算法，可选 none、zlib_1.0、zstd_1.0，zstd_1.3.8。默认为 none，表示不压缩系统日志文件。
syslog_file_uncompressed_count	新增	新增集群级配置项，用于控制每种日志不压缩的系统日志文件数量。默认为 0。
enable_global_background_resource_isolation	新增	新增集群级系统配置项，用于控制是否对全局的前后台任务进行 CPU 资源隔离。默认为 False，表示不隔离。
global_background_cpu_quota	新增	新增集群级系统配置项，用于控制 enable_global_background_resource_isolation 为 True 时，后台任务可使用的 CPU 配额。默认为-1，表示不受 Cgroup 限制。
net_thread_count	变更默认值	该配置项用于控制网络 I/O 线程数量。默认值仍然为 0，但自适应策略有变化。不同 CPU 数量下，可使用线程数有所提升。
ddl_high_thread_score	新增	新增租户级配置项，用于控制执行 DDL 补数据过程中，对每个 Tablet 的 KV 数据合并可使用的 DAG 线程数。默认为 6，和历史版本可用线程数一致。
lob_enable_block_cache_threshold	新增	新增租户级配置项，用于控制 OUTROW 存储的 LOB，如果长度小于等于该阈值，则进入微块缓存来加速下一次查询。默认 256K。
ob_default_lob_inrow_threshold	变更默认值	该配置项用于指定 LOB INROW 存储的最大阈值，默认值由 4096 调整为 8192。

系统变量变更

变量	变更类型	变更说明
activate_all_roles_on_login	新增	新增 MySQL 租户下 GLOBAL 级系统变量，用于控制用户登录时是否激活所有角色。
ob_compatibility_control	新增	新增 MySQL 租户 GLOBAL 级系统变量，用于控制存在兼容行为冲突时，OceanBase 数据库和 MySQL 5.7 行为一致，还是和 MySQL 8.0 行为一致。默认为 MYSQL5.7。创建租户时指定，租户创建后不允许修改。
ob_compatibility_version	新增	新增租户下 GLOBAL 系统变量，用于控制产品行为发生变更的功能，行为和 OceanBase 哪个发行版本兼容。新建集群时，默认为当前集群版本；版本升级时，为前一个版本配置，当前默认 4.2.1.0。可修改。
ob_security_version	新增	新增租户下 GLOBAL 系统变量，用于控制安全相关的产品行为发生变更的功能，行为和 OceanBase 数据库发行版本兼容。新建集群时，默认为当前集群版本；版本升级时，为前一个版本配置，当前默认 4.2.1.0。可修改，和 ob_compatibility_version 区别为该变量只能推高，不能回退。
cardinality_estimation_model	新增	新增 GLOBAL/SESSION 级系统变量，用于控制优化器估行时使用的相关性模型。可选 INDEPENDENT/PARTIAL/FULL 多种模型：INDEPENDENT：假设谓词间是完全独立的，是 V4.2.2_CE 及之前的优化器所使用的假设。多个 Filter 的联合选择率由单个 Filter 的选择率相乘得到。该假设在 Filter 较多时往往会导致估行偏小。PARTIAL: 假设谓词间有一定程度的相关性。多个 Filter 的联合选择率会通过指数回退计算。FULL: 假设谓词间是完全相关的。多个 Filter 的联合选择率直接由其中最小的 Filter 选择率决定，可以得到一个较大的估行结果。较为极端。默认为 PARTIAL。

函数/系统包变更

函数名	变更类型	变更说明
DECODE_TRACE_ID	新增	用于解析 trace_id 获取发起 SQL 请求的 OBServer IP、Port 信息。
CURRENT_ROLE	新增	用于展示当前 Session 激活的 Role。
PASSWORD	新增	支持 MySQL 5.7 Password 表达式，用于计算和返回哈希密码值。

语法变更

语法	变更说明
新增指定 SET/PIECE 级恢复源命令	新增指定位点查看恢复需要使用的 SET/PIECE 及指定 SET/PIECE 级路径恢复到指定位点：加载需要恢复的路径：ALTER SYSTEM ADD RESTORE SOURCE 'xxx';如果输入错误，可执行下述语句来撤销之前的输入：ALTER SYSTEM CLEAR RESTORE SOURCE;调用恢复命令：ALTER SYSTEM RESTORE <$restore_tenant> UNTIL '<$restore_checkpoint>' WITH 'xxx';解析恢复到指定位点所需的备份 SET/PIECE 的原始路径：ALTER SYSTEM RESTORE FROM 'uri' UNTIL { TIME='timestamp' | SCN=scn } PREVIEW;预览恢复到期望时间所需要的备份 SET/PIECE 的原始路径：SHOW RESTORE PREVIEW;
新增日志流副本管理命令	添加日志流副本。删除日志流副本。转换日志流副本类型。迁移日志流副本。修改日志流的 paxos 成员数量。取消日志流副本任务。
新增 auto_increment_cache_size 表级选项	新增表级选项语法，可以在 CREATE TABLE 或者 ALTER TABLE 时指定 auto_increment_cache_size。 比如create table t1 (...) auto_increment_cache_size=xxx; alter table t1 set auto_increment_cache_size=xxx;该值默认为 0 表示未配置，此时会使用租户级配置项作为自增列缓存大小。
新增 table_mode 表级选项	V4.2.3 重新启用了 V3.x 系列的表选项 table_mode，用户可以通过为每张表设置不同的表选项，以指定不同触发频率的快速冻结与自适应合并策略来应对 Buffer 表问题。如 create table t1 (c1 int) table_mode = 'normal/queuing/moderate/super/extreme';
新增 DAS 层 DML 是否允许多线程并行执行的 HINT	新增 enable_parallel_das_dml 和 disable_parallel_das_dml 两个 HINT，分别用于控制开启和关闭 DAS 层 DML 并发。在租户级隐藏配置项 _enable_parallel_das_dml 为 True 的前提下：场景 1 /+enable_parallel_das_dml enable_parallel_dml parallel(n)/ 中，该 HINT 组合会强制指定 DAS 并发执行，采用 parallel 定义的并发度；场景 2 /+enable_parallel_dml parallel(n)/ 中，该 HINT 组合会优先使用 PDML，如果当前场景不支持 PDML 优化，会使用 DAS 并发执行优化；场景 3 中 Session 上强制指定并行度时，行为等同于场景 2。
新增角色相关语法	创建 Role：CREATE ROLE [IF NOT EXISTS] role [, role ] ...删除 Role：DROP ROLE [IF EXISTS] role [, role ] ...授权 Role 给用户：GRANT role [, role] ... TO user_or_role [, user_or_role] ... [WITH ADMIN OPTION]撤销给用户授予的 Role：REVOKE [IF EXISTS] role [, role ] ... FROM user_or_role [, user_or_role ] ... [IGNORE UNKNOWN USER]设置用户登录时默认启用的 Role：SET DEFAULT ROLE {NONE &#124; ALL &#124; role [, role ] ...} TO user [, user ] ...ALTER USER [IF EXISTS] user DEFAULT ROLE {NONE &#124; ALL &#124; role [, role ] ...}设置当前 Session 激活哪些 Role：SET ROLE { DEFAULT &#124; NONE &#124; ALL &#124; ALL EXCEPT role [, role ] ... &#124; role [, role ] ...}show grants 展开 Role 权限：SHOW GRANTS FOR user USING role
新增列级权限授予回收相关语法	授予列级权限：GRANT [priv [(col_list)] ]+ ON [db_name.]table_name TO user [with grant option];回收列级权限：REVOKE [priv [(col_list)] ]+ ON [db_name.]table_name FROM user [with grant option];
支持复制表属性变更相关命令	普通表转复制表：ALTER TABLE t1 DUPLICATE_SCOPE = 'CLUSTER';复制表转普通表：ALTER TABLE t1 DUPLICATE_SCOPE = 'NONE';

周边配套

OceanBase 数据库 V4.2.3_CE 版本推荐使用的平台工具版本如下：

组件	版本
ODP	V4.2.3
OCP	V4.2.2_CE_HF1
OBD	V2.8.0
All in One	V4.2.3
ODC	V4.2.4-bp1
OBCDC	V4.2.3
OMS	V4.2.3_CE
OBClient	V2.2.3
LibOBClient	V2.2.3

升级说明

• 支持 OceanBase 数据库 V4.2.1_CE_BP2 及之前的 V4.x_CE 版本通过有效升级路径在线升级到 OceanBase 数据库 V4.2.3_CE_BETA 版本。V4.2.1_CE_BP3 及之后的 BP 版本目前不再支持升级到 V4.2.3_CE_BETA 版本。
• 支持 OceanBase 数据库 V4.2.2_CE 版本经停 V4.2.2_CE_BP1 版本，在线升级到 V4.2.3_CE_BETA 版本。
• V4.2.1_CE 及 V4.2.2_CE 系列的版本升级到 V4.2.3_CE_BETA 版本时，需要将系统变量 _nlj_batching_enabled 设为 False。升级完成后可以打开。
• 集群 Tablet 比较多的情况下，升级过程中 OBServer 重启会比较耗时。已知 300w+ 的 Tablet，重启时长大约在 20min，存在大量 Tablet 的场景需要特别注意预留足够的升级时间。
• ODP 和 OBServer 升级顺序：建议先升级 OBServer 版本再升级 ODP 版本。
• 升级期间，系统会自动禁用合并和 DDL 操作，升级完成后恢复正常使用。

开源鸣谢

在此版本发布中，特别感谢社区伙伴的贡献：

感谢联通软研院团队邱永刚 @qiuyg3 在 ObLogMiner 功能上的贡献。

Version information

Information	Description
Release date	April 24, 2024
Version	V4.2.1_CE_BP5
Commit number	ec03c25
OBServer RPM version	oceanbase-ce-4.2.1.5-105000032024041915

Enhanced features

• Compatibility with MySQL

  • A SET statement that contains the SET NAMES statement can also set other variables.
  • The information_schema.events table of MySQL is supported. #1194

• Other optimizations

  • The [G]V$OB_SESSION view is provided to display session information and help reduce the time required in collecting processlist statistics.
  • The NETWORK_WAIT_TIME column is added to the GV$OB_SQL_AUDIT view to indicate the total amount of time spent on events of the Network class.
  • A switch is provided for controlling parallel execution of DDL statements.
  • The slog and sstable directories now can be stored in different paths on the disk.
  • The OUTROW storage performance of large object (LOB) data is improved.
  • The issue of PL cache failure is resolved.
  • Adaptive major compaction is optimized for buffer tables.
  • The statistics collection performance is improved.

Product behavioral changes

To ensure stability, the batch rescan optimization feature was disabled by default for the NESTED-LOOP JOIN (NLJ) and SUBPLAN FILTER (SPF) operators during cluster creation in OceanBase Database since V4.2.1_CE_BP2_HF1. This feature is optimized in V4.2.1_CE_BP5 and is now enabled by default. In other words, the batch rescan optimization feature is enabled by default for new tenants. In OceanBase Database of a version earlier than V4.2.1_CE_BP5, the feature is still disabled by default. You can manually enable the feature as needed.

Bug fixes

• Fixed the issue where the query result of an SQL statement in a standalone database is inconsistent with that in a multi-node cluster. #1845
• Improved the performance of hotspot functions in an ARM environment.
• Fixed the issue where requests are accumulated in the queue of the sys tenant because the slog disk is used up.
• Fixed the issue where an additional record is displayed in some views such as information_schema.tables and all_tables after a DDL operation is performed to convert a non-partitioned table into a partitioned table.
• Fixed the issue where in the case of frequent connection and disconnection attempts, Error 4016 is returned during connection establishment when the total number of established connections reaches the specified value, because the reference count is leaked.
• Fixed the issue where the OBServer node can be hung if the obstack command is automatically executed when requests are accumulated in the queue.
• Fixed the issue where Error 4016 is returned when you execute the SHOW TRACE statement for a distributed execution plan after end-to-end tracing is enabled.
• Fixed the issue where the performance deteriorates after SQL plan management (SPM) is enabled.
• Fixed the issue where a core dump occurs if the offset exceeds INT64_MAX when the LIMIT OFFSET syntax is used in the .NET driver.
• Fixed the issue where the memory of the CommonArray memory module leaks due to bypass import.
• Fixed the issue where the memory of the PlJit module leaks due to exceptions.
• Fixed the issue where a core dump occurs during parallel execution when memory allocation fails.
• Fixed the issue of memory bloat during parallel index creation in specific scenarios.
• Fixed the issue where Error 4344 may be returned when you back up a standby database in specific scenarios.
• Fixed the issue where creating a standby tenant in the standby cluster will fail if case sensitivity is enabled for table names of tenants in the primary cluster.
• Fixed the issue where the execution time is abnormal if you execute the create table if not exists statement to create an existing table immediately after the cluster is restarted.

Considerations

OceanBase Database V4.2.1_CE_BP5 fixes the issue where connection establishment fails because the reference count of session connections is leaked. We recommend that you upgrade OceanBase Database to V4.2.1_CE_BP5 as soon as possible.

版本信息

项目	描述
发布日期	2024-04-24
版本号	V4.2.1_CE_BP5
Commit 号	ec03c25
OBServer RPM 版本号	oceanbase-ce-4.2.1.5-105000032024041915

特性增强

• MySQL 兼容性

  • 支持在同一个 SET 语句中同时包含 SET NAMES 语句和使用 SET 设置其他变量。
  • 支持兼容 MySQL 的 information_schema.events 表结构。 #1194

• 其他优化

  • 新增 [G]V$OB_SESSION 视图展示会话信息，优化 Processlist 统计时间。
  • GV$OB_SQL_AUDIT 视图新增 NETWORK_WAIT_TIME 字段，用于展示所有 Network 类事件的总时间。
  • 新增 _parallel_ddl_control 参数用于控制是否在租户级别开启各类并行 DDL 的功能。
  • 支持将 slog 和 sstable 的目录放在不同的磁盘目录下。
  • 优化 LOB 类型数据外联存储（OUTROW）性能。
  • 优化 PL Cache 缓存失效的问题。
  • 优化 Buffer 表的自适应合并。
  • 优化统计信息收集的性能。

产品行为变更

出于稳定性因素的考虑，从 V4.2.1_CE_BP2_HF1 版本开始，新建集群时，Nested Loop join（NLJ）和 SUBPLAN FILTER（SPF）算子的 BATCH RESCAN 优化默认会被关闭。我们针对该功能进行了专项提升，在 V4.2.1_CE_BP5 版本重新打开该默认开关。V4.2.1_CE_BP5 版本开始，新建的租户默认保持 BATCH RESCAN 优化开启。升级前已存在的老租户会维持原来的配置，如需打开优化，需在对应的租户下执行 SET GLOBAL _nlj_batching_enabled = true 手动开启。

缺陷修复

• 修复单节点与多节点查询结果不一致的问题。#1845
• 优化 ARM 环境下的性能热点问题。
• 修复 slog 盘满导致系统租户队列积压的问题。
• 修复执行非分区表转分区表 DDL，会导致一些视图（如 information_schema.TABLES/ALL_TABLES）多显示一条记录的问题。
• 修复频繁建连接/断连接场景下，引用计数泄漏导致建连接总数达到一定量后，建连接报错 4016 的问题。
• 修复队列积压情况下自动 obstack 可能导致 observer hang 住的问题。
• 修复开启全链路追踪后分布式执行计划 show trace 报错 4016 的问题。
• 修复打开 SPM 后性能下降的问题。
• 修复 .NET 驱动中使用 limit offset 时遇到 int64_max 溢出导致的 core 问题。
• 修复旁路导入产生 CommonArray 内存模块泄漏的问题。
• 修复异常场景下 PlJit 模块内存泄漏问题。
• 修复分配内存失败场景下，并行执行 core 的问题。
• 修复并行执行创建索引特定场景下内存膨胀的问题。
• 修复特定场景下备库备份可能出现 4344 报错的问题。
• 修复主集群租户设置了区分表名大小写，备集群同步创建备租户会失败的问题。
• 修复集群重启后，立即执行 create table if not exists 创建已经存在的表，执行时间异常的问题。

注意事项

V4.2.1_CE_BP5 版本修复了集群 session 连接数引用计数泄漏导致不能新建连接的问题，请尽快升级到 V4.2.1_CE_BP5 版本。

版本信息

项目	描述
发布日期	2024-03-28
版本号	V4.3.0_CE_BETA
Commit 号	0193a34
RPM 版本号	oceanbase-ce-4.3.0.1-100000242024032211
版本说明	Beta 版本解决了大部分缺陷，并趋于稳定。推荐测试环境使用。

版本定位

OceanBase 重磅推出 V4.3.0 版本，深入典型 AP 场景，不再局限于 TP + 轻量化 AP 的版本定位。基于 LSM-Tree 架构推出列存引擎，实现行存、列存数据存储一体化，同时推出基于 Column 数据格式描述的新版向量化引擎和基于列存的代价模型，支持高效处理大宽表，显著提升 AP 场景查询性能，并兼顾 TP 业务场景。新增物化视图功能，通过预计算存储视图的查询结果提升实时查询性能，支撑快速报表生成和数据分析场景。新版本内核也扩展了 Online DDL、支持了租户克隆等功能，优化 PDML、节点重启性能，提升 LOB 类型旁路导入效率，增加 S3 备份恢复介质支持，优化系统资源使用，并增加索引使用监控、客户端本地导入等功能提升系统易用性。推荐用于复杂分析、实时报表、实时数仓或联机交易等混合负载场景。

关键特性说明

AP 关键特性

• 列存引擎

  在大规模数据复杂分析或海量数据即席查询场景中，列式存储是 AP 数据库的关键能力之一。列式存储是一种数据文件组织方式，区别于行式存储，它将表中的数据按照列进行物理排列。数据进行列式存储时，分析场景可仅扫描用于查询计算的列数据，避免整行扫描，减少 IO 和内存等资源使用，提升计算速度。另外按列存储也天然具备更好的数据压缩条件，更易获得较高的压缩比，减少存储空间和网络传输带宽。

  不过常见的列存存储引擎在实现上往往假设不会有大量随机更新, 尽量保证列存组织数据是静态的。当真正伴随大量数据随机更新时，也会不可避免的存在系统性能问题。OceanBase LSM-Tree 架构可以将基线数据和增量数据分别处理，正好可以解决这一场景问题。因此 V4.3.0 版本在当前架构基础上继续扩展支持了列存引擎，在一套代码一个架构一个 OBServer 上，实现了列存和行存数据存储一体化，兼顾 TP 和 AP 类查询性能。

  为了方便 AP 业务迁移、方便老客户顺畅使用新版本，围绕列存引擎，从优化器到执行器、从 DDL 到事务处理等多模块都进行了适配优化。包括基于列存的新的代价模型和向量化引擎，查询下压功能的扩展和增强，Skip Index，新的列式编码算法，自适应 Compaction 等。

  业务上，用户可根据负载类型灵活设置表为行存表、列存表或行列冗余表。

• 新向量化引擎

  OceanBase 在早期版本已经实现了基于 Uniform 数据描述方式的向量化引擎，性能较非向量化引擎有了明显提升，但在深度 AP 场景，还有一些性能上的不足。V4.3.0 版本实现了向量化引擎 2.0 版本，更改为 Column 数据格式描述，避免了 ObDatum 维护带来的内存使用、序列化和读写访问开销。基于数据格式描述重构，新版本也对一批常用算子和表达式进行了重新实现，如 HashJoin、AGGR、HashGroupBy、Exchange(DTL Shuffle) 等 10 余项算子，关系运算、逻辑运算、算数运算等 20 余项 MySQL 表达式。在后续的 V4.3.x 版本也会基于新的向量化引擎，持续补充完善其他算子和表达式的实现，以便获得 AP 场景更优性能。

• 物化视图

  V4.3.0 版本新增物化视图（Materialized View）功能。物化视图是支撑 AP 业务的一个关键特性，它通过预计算和存储视图的查询结果，减少实时计算来提升查询性能，简化复杂查询逻辑，常用于快速报表生成和数据分析场景。

  因为物化视图需要存储查询结果集来优化查询性能，而物化视图与基础表之间存在数据依赖关系，每当基础表数据发生变动时，物化视图中的数据必须进行相应更新以保持同步，所以新版本也引入了物化视图刷新机制，包括全量刷新和增量刷新两种策略。全量刷新是一种较为直接的方式，每次执行刷新操作时，系统会重新执行物化视图对应的查询语句，完整地计算并覆盖原有的视图结果数据，这种方式适用于数据量相对较小的场景。相对来讲，增量刷新仅需处理自上次刷新以来发生变更的部分。为了实现精确的增量刷新，OceanBase 实现了类似 Oracle MLOG（Materialized View Log）的物化视图日志功能，通过日志详细跟踪记录基础表的增量更新数据，从而确保物化视图能够进行快速增量刷新。增量刷新方式尤其适用于数据量庞大且变更频繁的业务场景。

内核增强

• 估行系统增强

  随着 OceanBase 版本的不断演进，优化器代价估计的方式也越来越丰富。涉及到每个算子估行，目前已经支持了存储层估行、统计信息估行、动态采样、默认统计信息等算法，使用上缺少清晰的策略和完善的控制手段。V4.3.0 版本重构估行系统，根据不同场景，指定不同的估行策略优先顺序，并提供 HINT、系统变量等手工干预估行策略选择的方式。同时新版本对谓词选择率和 NDV 计算框架也做了进一步增强，以此提高优化器代价估计的准确性。

• 统计信息增强

  V4.3.0 版本进一步完善了统计信息功能、改善了统计信息收集性能、提升了统计信息兼容性和易用性。主要包括重构离线统计信息收集流程，提升统计信息收集效率；优化统计信息收集策略，默认自动收集索引直方图，使用推导统计信息收集方式；保证在线统计信息收集的事务一致性；兼容 Oracle 数据库 DBMS_STATS.COPY_TABLE_STATS 功能，用于统计信息拷贝场景；兼容 MySQL 数据库 ANALYZE TABLE 功能，提供更丰富的语法支持；新增取消统计信息收集的命令，丰富统计信息收集进度监控，增强运维易用性；提供统计信息并行删除能力等。

• 自适应代价模型

  OceanBase 历史版本代价模型是使用内部机器测算的常量参数来代表硬件系统统计信息，通过一系列公式与常量参数来描述每个算子的执行开销。而真实的业务场景中，不同硬件环境可能具备不同的 CPU 时钟频率、不同的顺序读或随机读的速度、不同的网卡带宽等，可能存在代价估算偏差，这些偏差会使得优化器无法在不同的业务环境总是生成最优计划。新版本优化代价模型实现，支持通过 DBMS_STATS 包来收集或设置系统统计信息系数，已达到代价模型自适应硬件的目的。同时也提供了 DBA_OB_AUX_STATISTICS 视图，用于展示当前租户的系统统计信息系数。

• 函数索引 SESSION 变量固化

  创建函数索引时，会向主表添加一个隐藏的虚拟生成列，定义为函数索引的索引键，然后再将虚拟生成列的值存储到索引表中。一些内置系统函数的结果会受到 SESSION 变量的影响，不同 SESSION 变量取值，即使函数入参相同，计算结果也是不同的。为提高生成列/函数索引的稳定性，新版本在创建函数索引和生成列时，会将依赖的 SESSION 变量固化至索引列和生成列的 Column Schema 中。后续计算索引列和生成列的取值时，使用固化值，不受当前 SESSION 下变量取值的影响。V4.3.0 版本支持固化的系统变量包含 timezone_info、nls_format、nls_collation、sql_mode 等。

• MySQL 模式下 Online DDL 扩充

  V4.3.0 版本对列类型变更一类的 DDL 拓展了更多 Online 场景，包括：

  • 整型转换：对于主键列/索引列/生成列/被生成列依赖的列/有 UNIQUE 或 CHECK 约束的列，列类型修改为取值范围更大的整型的场景，从 Offline DDL 变更为 Online DDL。
  • DECIMAL 类型转化：对于支持使用 DECIMAL 数据类型的列，Precision 在以下某个区间（[1,9]、[10,18]、[19,38]、[39,76]）内增长，且 Scale 不变的场景，全部为 Online DDL。
  • BIT 或 CHAR 类型转换：对于支持使用 BIT 或 CHAR 数据类型的列，宽度增长均为 Online DDL。
  • VARCHAR 或 VARBINARY 类型转换：对于支持使用 VARCHAR 或 VARBINARY 数据类型的列，宽度增长均为 Online DDL。
  • LOB 类型转换：对于支持使用 LOB 数据类型的列，除了 TINYTEXT 和 TINYBLOB 向上增长是 Offline DDL 流程，其他类型修改为取值范围更大的 LOB 类型时，均为 Online DDL 流程。
  • TINYTEXT 与 VARCHAR 类型转换：对于支持使用 TINYTEXT 数据类型的列，在 VARCHAR(x) 转换为 TINYTEXT 类型时，如果 x <= 255，为 Online DDL，否则为 Offline DDL；对于支持使用 VARCHAR 数据类型的列，在 TINYTEXT 转换为 VARCHAR(x) 类型时，如果 x >= 255，为 Online DDL，否则为 Offline DDL。
  • TINYBLOB 与 VARBINARY 类型转换：对于支持使用 TINYBLOB 数据类型的列，在 VARBINARY(x) 转换为 TINYBLOB 类型时，如果 x <= 255，为 Online DDL，否则为 Offline DDL；对于支持使用 VARBINARY 数据类型的列，在 TINYBLOB 转换为 VARBINARY(x) 类型时，如果 x >= 255，为 Online DDL，否则为 Offline DDL。

• 全局唯一 Client Session ID

  在 OBServer V4.3.0 版本和 OceanBase 数据库代理（OceanBase Database Proxy，ODP）V4.2.3 版本之前，Client 端通过 ODP 执行 SHOW PROCESSLIST 时，查询到的是 ODP 本机上的 Client 端会话 ID，通过 connection_id 等表达式或系统视图查询到的为 Server 端会话 ID。Client 端会话 ID 和 Server 端会话 ID 存在一对多的关系，难以在全链路统一，Session 信息查看容易混淆，不方便进行用户会话管理。新版本重构 Client Session ID 生成和维护流程，在 OBServer 版本不低于 V4.3.0 且 ODP 版本不低于 V4.2.3 时，通过各种渠道如 SHOW PROCESSLIST 命令、information_schema.PROCESSLIST 或 GV$OB_PROCESSLIST 等视图、connection_id、userenv('sid')/userenv('sessionid')、sys_context('userenv','sid')/sys_context('userenv','sessionid') 等表达式，查询到的会话 ID 均为 Client 端会话 ID，用户可通过指定 SQL 或 PL 中的 KILL 命令来管理用户端会话。OBServer 或 ODP 版本不满足要求时，将退化为老版本的处理方式。

• 日志流状态机改造

  新版本将日志流的状态拆分成了内存和持久化状态。持久化状态用于表示日志流的生命周期，宕机重启以后会根据持久化状态决定日志流是否应该存在，以及内存状态应该是什么。内存状态是日志流运行时状态，用于表示日志流整体状态和关键子模块状态。根据日志流明确的状态和状态 sequence，底层模块可判断日志流哪些操作是安全的，是否发生过 ABA 类型的状态变更。对于备份恢复或迁移流程，在日志流宕机重启后，也优化了工作状态表现。该特性提升了日志流相关功能的稳定性，强化了日志流并发控制能力。

• 租户克隆

  V4.3.0 版本新增租户克隆特性，用户可在 SYS 租户下执行 CREATE TENANT new_tenant_name FROM source_tenant_name WITH RESOURCE_POOL [=] resource_pool_name, UNIT [=] unit_config 命令，对指定租户快速克隆出一个新租户。租户克隆任务执行完成后，新克隆租户为备租户，用户可使用 ALTER SYSTEM ACTIVATE STANDBY TENANT new_tenant_name 将其转为主租户提供服务。新克隆租户和原始租户在初始状态下共享物理宏块，但新的数据变动和资源使用会按租户进行隔离。当用户需要对在线租户进行高资源消耗的临时数据分析或其他高风险操作时，为了避免对在线租户造成影响，可使用克隆租户完成分析或验证。同时也可将克隆租户作为容灾手段，若原始租户发生了难以恢复的误操作，可使用克隆租户进行数据回滚。

MySQL 兼容

• utf8mb4/utf16_unicode_ci 字符序

  社区版本新增支持 utf8mb4_unicode_ci 和 utf16_unicode_ci 字符序。

性能提升

• OceanBase zlib_lite_1.0 压缩算法

  OceanBase V4.3.0 版本引入了一种新的数据压缩算法：zlib_lite_1.0。该压缩算法基于 Intel 的 In-Memory Analytics Accelerator (Intel® IAA) 加速器 ，充分利用硬件功能提升数据库压缩和解压缩的性能，并且保证了对传统 zlib 库的兼容性。在具备 Intel IAA 硬件加速条件的机器上，zlib_lite_1.0 数据压缩算法可以显著提升压缩和解压缩性能；不具备硬件加速条件时，将自动回退到软件加速方式，确保跨平台的兼容性。

• PDML 事务优化

  新版本在事务层通过支持并行提交和回放日志、增加事务参与者内部分区级别回滚等实现优化，相对 V4.x 早期版本，明显提升高并发场景下 PDML 执行性能。

• Tablet 元数据加载 IO 使用优化

  OceanBase V4.x 版本支持了单机百万分区，因百万 Tablet 元数据可能无法全部放入内存之中，又支持了元数据按需加载。按需加载包括分区和分区内部子类两种级别，分区内部会拆成多种不同子类的元数据分层存储，在后台任务需要较深层次元数据的场景下，读取需要耗费较多的 IO。在本地 SSD 磁盘上，这些 IO 开销不是问题，但在机械磁盘和云盘的场景，可能影响系统性能。新版本针对需要频繁访问的元数据做了聚合存储，这类元数据访问降低为只需要 1 次 IO，大大减小空载情况下 IO 开销，不会因后台任务的 IO 开销影响到前台查询性能。同时也优化了 OBServer 重启时元数据加载流程，修改为按宏块粒度批量加载 Tablet 元数据，大大减少离散读 IO，数倍至数十倍提升重启速度。

高可用增强

• Tablet Location 主动广播/刷新

  OceanBase 早期版本已具备 Location Cache 周期性刷新机制，确保日志流 Location 信息更新的实时性和最终一致性。但 Tablet Location 只具备被动刷新能力，在 Tablet 和日志流的映射关系发生变化时，有概率触发 SQL 重试和读写报错。V4.3.0 版本增加 Tablet Location 主动广播能力，减少 Transfer 后映射关系变化导致的 SQL 重试和读写报错；并提供主动刷新能力兜底，避免不可恢复的读写报错问题。

• 备份恢复支持 S3

  OceanBase 的备份恢复功能已经支持两类存储介质：文件存储（NFS）和对象存储（OSS/COS）。新版本备份恢复增加支持 AWS S3（Simple Storage Service）存储服务，可以将 S3 作为日志归档和数据备份的目的端，也可以使用 S3 上的备份数据执行物理恢复。

• 内存规格扩缩容限制

  新版本优化了内存规格扩缩容的稳定性，避免不合理的 memory_limit 配置引起系统 OOM 问题。OBServer 上 memory_limit 设置生效需要同时满足两个条件：一是 500 租户的预留内存需要不小于实际占用内存，二是 memory_limit 取值需高于 system_memory 和 UINT 已分配内存之和。在不满足以上限制条件时，设置 memory_limit 不会报错，也不生效。

• Transfer 搬迁活跃事务

  单机日志流设计中，数据的单位是分片（tablet），日志的单位是日志流（logstream），多个分片聚合在一个日志流上使得单个日志流内部的事务避免了两阶段提交的高额代价。为了可以均衡不同日志流间的数据与流量，我们需要允许分片在不同日志流间可以灵活地迁移（即 Transfer）。然而在 Transfer 过程中，活跃事务依旧可能在数据上活动，简单的操作会破坏事务的 ACID 能力。举例来说，Transfer 源端的活跃事务数据若无法在并发过程中完整地迁移到目的端，事务的原子性就无法保证。在该版本之前，Transfer 执行过程中会主动杀掉活跃事务来避免产生事务问题，一定程度影响了事务的正常执行。为了优化这一问题，新版本支持了 Transfer 搬迁活跃事务的能力，允许活跃事务并发执行，并保证并发事务不会因为 Transfer 导致异常回滚或产生一致性问题。

资源使用优化

• 事务日志 MINIMAL 模式

  新版本重构事务日志的 MINIMAL 模式，优化和完善老版本 MINIMAL 功能实现，提升功能稳定性。MINIMAL 模式开启后，在 UPDATE、DELETE 等场景下产生的 Clog 日志量会有明显下降，可显著降低日志存储、归档、传输等资源开销，尤其适用于私有云跨城网络带宽、公有云写云盘带宽资源有限的场景。V4.3.0 Beta 版本，因 OMS 还未针对性适配，暂时默认关闭。在无需通过工具进行增量数据同步的场景，可以通过设置系统变量 binlog_row_image = MINIMAL 来开启该功能。

• 内存限速机制

  在 V4.x 版本之前，需要依赖冻结转储释放内存的模块并不多，Memtable 是其中最大的一部分。因此，之前版本对 Memtable 本身设定了内存使用上限，并通过限速逻辑使其在内存使用量接近上限的过程中运行尽可能平滑，避免突然耗尽内存导致系统停写。V4.x 版本之后，随着更多依赖冻结转储释放内存的模块（如事务数据模块）被引入，新版本提供了更细致的手段来控制多个模块的内存使用，新增 TxData 和 MDS 模块内存上限控制，和 Memtable 共享内存空间，当累积内存达到 租户内存 * _tx_share_memory_limit_percentage% * writing_throttling_trigger_percentage% 时，触发整体限速。同时，新版本也增加了时间维度触发事务数据表冻结转储的功能，默认 1800 秒触发一次事务数据表的冻结，降低事务数据模块的内存占用。

• DDL 临时结果空间优化

  在 DDL 过程中，会有很多把临时结果暂存物化结构中的流程。典型的例子有以下两个：

  1. 对于创建索引这一场景，当从数据表扫描并插入数据到索引表时，会需要对数据表扫出来的数据进行排序。排序的过程中如果内存不足，就会把当前内存中的数据暂存到物化结构中，释放内存空间以供后续的扫描，最后对这些物化结构中的数据进行归并排序。这种处理方法在内存受限时特别有效，但会带来对磁盘空间的额外需求。
  2. 在列存旁路导入场景中，系统会先把需要插入到每个 Column Group 的数据暂存在物化结构中，后续对每个 Column Group 进行插入的时候，都从物化结构中取数据。这些暂存数据的物化结构在 sort 算子中可以用来存储外部排序需要用到的中间数据，在插入 Column Group 的时候可以缓存数据从而避免重复扫表导致的额外开销。虽然这种方式能够降低重复扫描带来的性能损耗，但同样存在临时文件增大磁盘空间使用的问题。

  针对这些问题，新版本 DDL 处理流程对数据流进行了优化。首先，它剔除了不必要的冗余结构，简化了数据流；其次，它引入了对临时结果落盘的编码压缩功能。这些改进为两个场景都带来了好处：它们显著降低了 DDL 操作期间临时结果对磁盘空间的占用，从而更加高效地利用存储资源。

易用性提升

• 监控索引

  我们对数据库执行查询操作时，往往通过创建索引来优化查询性能。但随着数据表使用的时间增长，业务场景和操作人员不断增加，很可能会存在索引越建越多的问题。未使用的索引会浪费存储空间，也会加重 DML 操作的开销。对于这种情况，需要持续观察，删除无用的索引来给系统减负。但是仅靠人力很难识别哪些索引是无用索引，因此 OceanBase 在 V4.3.0 版本新增索引使用监控功能，用户可选择打开该功能并设置采样方式，在普通租户下会将符合规则的索引使用信息记录到内存，并以 15 分钟为周期刷新到内部表中，可通过 DBA_INDEX_USAGE 视图访问，以此来感知表上的索引是否有被引用，进而选择删除无用的索引表来释放空间。

• RPC 安全认证证书管理

  集群开启 RPC 认证时，如果有客户端（如仲裁、主备库、OBCDC 等）访问需求，需要先将客户端的 CA 根证书依次放入集群每台节点部署目录下，再做相关配置，操作较为复杂。V4.3.0 版本新增内部证书管理功能，SYS 租户下提供 DBMS_TRUSTED_CERTIFICATE_MANAGER 系统包，用于添加、删除、修改受 OBServer 集群信任的 CA 根证书。同时 SYS 租户下新增 DBA_OB_TRUSTED_ROOT_CERTIFICATE 视图，用于展示 OBServer 集群已添加的客户端 CA 根证书列表及证书过期时间等信息。

• 配置项 RESET

  当配置项修改后，如果希望重置为默认值，目前需要先查询配置项的默认值是什么，再手动设置，易用性较差。新版本新增 ALTER SYSTEM [RESET] parameter_name [SCOPE = {MEMORY | SPFILE | BOTH}] {TENANT [=] 'tenant_name'} 语法，用于重置配置项默认值，默认值来源于执行语句节点。SYS 租户下可重置集群级配置项或指定业务租户的配置项，业务租户可重置本租户配置项。在 OBServer 中，SCOPE 这个选项在实现上并没有差异，对于静态生效的配置项，只将配置项默认值落盘，不更新内存值。对于动态生效的配置项，更新内存值并落盘。

• 配置项数据类型细化

  新版本在配置项相关视图（如 [G]V$OB_PARAMETERS）和 SHOW PARAMETERS 命令返回结果中细化展示了配置项数据类型（data_type 列）。例如 log_disk_size 类型为 CAPACITY、rpc_port 类型为 INT 和 devname 类型为 STRING 等，以便周边工具按配置项类型进行展示和限制。

• LOB INROW 阈值配置

  当前 LOB 数据小于等于 4KB 时，会 INROW 存储（即行内存储），大于 4KB 时，会存入 LOB 辅助表。部分场景相比于存入辅助表，INROW 整存整取性能更好。因此该版本提供 LOB 存储模式动态配置能力，不超过行级存储规格限制的前提下，用户可根据业务需求动态调整 INROW 大小。

• 客户端本地导入

  V4.3.0 版本新增客户端本地导入（LOAD DATA LOCAL INFILE）功能，通过流式文件处理方式完成本地文件导入，丰富了原有的数据文件导入方式。基于该功能，开发人员无需上传文件至服务器或对象存储也可进行本地文件导入测试，提高少量数据导入的工作效率。

兼容性变更

产品行为变更

功能	变更说明
Client Session ID 由 ODP 本机唯一变更为 OceanBase 集群全局唯一	在 OBServer V4.3.0 版本和 ODP V4.2.3 版本之前，Client 端通过 ODP 执行 SHOW PROCESSLIST 时，查询到的是 ODP 本机上的 Client 端会话 ID，通过 connection_id 等表达式或系统视图查询到的为 Server 端会话 ID。Client 端会话 ID 和 Server 端会话 ID 存在一对多的关系，难以在全链路统一，Session 信息查看容易混淆，不方便进行用户会话管理。新版本重构 Client Session ID 生成和维护流程，在 OBServer 版本不低于 V4.3.0 且 ODP 版本不低于 V4.2.3 时，通过各种渠道如 SHOW PROCESSLIST 命令、information_schema.PROCESSLIST 或 GV$OB_PROCESSLIST 等视图、connection_id、userenv('sid')/userenv('sessionid')、sys_context('userenv','sid')/sys_context('userenv','sessionid') 等表达式，查询到的会话 ID 均为 Client 端会话 ID，用户可通过指定 SQL 或 PL 中的 KILL 命令来管理用户端会话。OBServer 或 ODP 版本不满足要求时，将退化为老版本的处理方式。
memory_limit 增加扩缩容限制	新版本优化了内存规格扩缩容的稳定性，避免不合理的 memory_limit 配置引起系统 OOM 问题。OBServer 上 memory_limit 设置生效需要同时满足两个条件：一是 500 租户的预留内存需要不小于实际占用内存，二是 memory_limit 取值需高于 system_memory 和 UINT 已分配内存之和。在不满足以上限制条件时，设置 memory_limit 不会报错，也不生效。
存储不再使用 zlib 压缩算法	V4.2.0 版本，存储已经禁止新建表使用 zlib 压缩，存量旧表仍可以使用 zlib 压缩。V4.3.0 版本存储层不再支持使用 zlib 压缩算法，用户升级前如果使用了 zlib 压缩算法，须将数据表的压缩算法改成其它压缩算法或不压缩；clog 传输、TableAPI 传输等也均禁止使用 zlib 压缩算法。
细化 archive_lag_target 使用限制	V4.3.0 版本细化了 archive_lag_target 配置项使用限制：如果用户未设归档介质，则在修改这个配置项时提示用户 “未设置归档介质，不允许修改这个配置项默认值”。如果用户设了归档介质为 S3，则这个配置项的最小值是 60 秒，设低于这个值则报错提示。如果用户设了归档介质为 OSS/NFS/COS 等，则配置项可以设置为取值范围内任意值。如果用户已经设了归档介质为 OSS/NFS/COS，通过命令将归档介质改成 S3，同时该配置项当前值小于 60 秒，则在修改归档介质命令时报错提示。
max_syslog_file_count 统一控制所有类型的系统日志数量	为了降低全链路诊断功能开启后日志盘用满的风险，max_syslog_file_count 从单独控制每类系统日志的数量变更为统一控制所有类型系统日志的总数量。在这种情况下，新版本会采用 FIFO（先进先出）策略进行日志文件的淘汰。
SHOW PARAMETERS 返回的 data_type 列细化配置项数据类型	SHOW PARAMETERS 返回的 data_type 字段细化了配置项数据类型，默认值也由 NULL 变更为 UNKNOWN。
UINT 中 MAX_IOPS 和 MIN_IOPS 默认值变更	老版本 MIN_IOPS 和 MAX_IOPS 均未指定时，根据 MIN_CPU 规格自动计算，1 个 Core 对应 1 万 IOPS 的值，即 MAX_IOPS = MIN_IOPS = MIN_CPU * 10000。新版本如果用户没有配置 MIN_IOPS 和 MAX_IOPS，会把默认的 IOPS 调整为 INT64_MAX，即不对 IOPS 资源进行约束。

视图变更

视图	变更类型	变更说明
DBA_OB_TRUSTED_ROOT_CERTIFICATE	新增	SYS 租户下视图，用于展示 OBServer 集群信任的客户端 CA 根证书列表及证书过期时间等信息。
CDB/DBA_MVIEW_LOGS	新增	用于描述物化视图日志信息。CDB 视图仅在 SYS 租户支持，DBA 视图在所有租户支持。
CDB/DBA_MVIEWS	新增	用于描述物化视图信息。CDB 视图仅在 SYS 租户支持，DBA 视图在所有租户支持。
CDB/DBA_MVREF_STATS_SYS_DEFAULTS	新增	用于描述物化视图刷新历史统计属性的参数的系统范围默认值。CDB 视图仅在 SYS 租户支持，DBA 视图在所有租户支持。
CDB/DBA_MVREF_STATS_PARAMS	新增	用于显示与每个物化视图关联的刷新统计信息属性。CDB 视图仅在 SYS 租户支持，DBA 视图在所有租户支持。
CDB/DBA_MVREF_RUN_STATS	新增	用于描述物化视图每次刷新运行的信息，每次运行均由 REFRESH_ID 标识。该信息包括与运行相关的计时统计信息以及运行中指定的参数。CDB 视图仅在 SYS 租户支持，DBA 视图在所有租户支持。
CDB/DBA_MVREF_STATS	新增	用于描述物化视图刷新的基本计时统计信息。CDB 视图仅在 SYS 租户支持，DBA 视图在所有租户支持。
CDB/DBA_MVREF_CHANGE_STATS	新增	用于描述所有物化视图的刷新运行关联的基表上的更改数据加载信息。CDB 视图仅在 SYS 租户支持，DBA 视图在所有租户支持。
CDB/DBA_MVREF_STMT_STATS	新增	用于描述刷新语句关联的信息。CDB 视图仅在 SYS 租户支持，DBA 视图在所有租户支持。
CDB/DBA_INDEX_USAGE	新增	用于展示索引访问信息。CDB 视图仅在 SYS 租户支持，DBA 视图在所有租户支持。
DBA_OB_CLONE_PROGRESS	新增	SYS 租户下视图，用于记录运行中的租户克隆任务信息。
DBA_OB_CLONE_HISTORY	新增	SYS 租户下视图，用于记录运行完成的克隆任务信息。
CDB/DBA_OB_AUX_STATISTICS	新增	用于展示每个租户的辅助统计信息。CDB 视图仅在 SYS 租户支持，DBA 视图在所有租户支持。
[G]V$OB_TABLET_COMPACTION_HISTORY	列变更	新增列 KEPT_SNAPSHOT 用于展示多版本保留位点信息；新增列 MERGE_LEVEL 用于展示宏块/微块重用信息；调整列 COMMENTS 的宽度。
[G]V$OB_PARAMETERS	列内容变更	DATA_TYPE 字段细化配置项数据类型，默认值也由 NULL 变更为 UNKNOWN。
[G]V$OB_PROCESSLIST	新增列	新增列 USER_CLIENT_PORT 用于展示客户端 Port。
CDB/DBA_OB_RECOVER_TABLE_JOBS	新增	用于展示表级恢复任务记录。CDB 视图仅在 SYS 租户支持，DBA 视图在所有租户支持。
CDB/DBA_OB_RECOVER_TABLE_JOB_HISTORY	新增	用于展示表级恢复任务历史记录。CDB 视图仅在 SYS 租户支持，DBA 视图在所有租户支持。
CDB/DBA_OB_IMPORT_TABLE_JOBS	新增	用于展示跨租户导入的 JOB 记录。CDB 视图仅在 SYS 租户支持，DBA 视图在所有租户支持。
CDB/DBA_OB_IMPORT_TABLE_JOB_HISTORY	新增	用于展示跨租户导入的 JOB 历史记录。CDB 视图仅在 SYS 租户支持，DBA 视图在所有租户支持。
CDB/DBA_OB_IMPORT_TABLE_TASKS	新增	用于展示表级别跨租户导入的 TASK 记录。CDB 视图仅在 SYS 租户支持，DBA 视图在所有租户支持。
CDB/DBA_OB_IMPORT_TABLE_TASK_HISTORY	新增	用于展示表级别跨租户导入的 TASK 历史记录。CDB 视图仅在 SYS 租户支持，DBA 视图在所有租户支持。
[G]V$OB_SQL_AUDIT	新增列	新增列 PLSQL_EXEC_TIME，用于展示 PL 执行耗时（不包括 SQL 执行时间），单位是 us。
[G]V$OB_LS_SNAPSHOTS	新增	用于展示 UNIT 中物理存在的日志流快照信息。

配置项变更

配置项/系统变量	变更类型	变更说明
enable_rpc_authentication_bypass	新增	新增集群级配置项，在 OBServer 开启 RPC 安全认证的场景下，用于设置是否允许 OMS 迁移服务绕过 RPC 安全认证连接集群。
default_compress_func	取值范围变更	新增 zlib_lite_1.0 取值，表示在具备硬件加速特性的环境，使用更高性能的 zlib 压缩算法。删除 zlib_1.0 取值，建表禁用 zlib_1.0 压缩算法。
large_query_threshold	取值范围变更	配置项取值范围由 [1ms, +∞) 变更为 [0ms, +∞)，取值为 0 时表示关闭大查询判定功能。
default_table_store_format	新增	新增租户级配置项，用于控制用户租户创建主表的默认格式。默认为 row，表示建表不指定 with column group 的情况下，默认为行存表。可根据需求修改为 column（默认纯列存表）或 compound（默认冗余行存列存表）。
server_cpu_quota_min	生效模式变更	生效模式由重启生效变更为立即生效。
server_cpu_quota_max	生效模式变更	生效模式由重启生效变更为立即生效。

函数/PL包变更

函数/PL包	变更类型	变更说明
ob_transaction_id	新增函数	新增 ob_transaction_id() 内置函数，用于查看当前会话的事务 ID，如果会话未处于活跃事务中，返回 0。
DBMS_TRUSTED_CERTIFICATE_MANAGER	新增包	SYS 租户下新增 PL 系统包，支持 ADD_TRUSTED_CERTIFICATE、DELETE_TRUSTED_CERTIFICATE、UPDATE_TRUSTED_CERTIFICATE 三个子程序，用于添加、删除、修改受 OBServer 集群信任的客户端 CA 根证书。在开启 RPC 认证时使用。

语法变更

• 新增 SSTable 列预聚合索引 DDL 语法。
• 新增租户克隆相关命令。
• 新增配置项重置语法。
• 新增列存与列存索引语法。
• 新增物化视图相关语法。
• 新增分区级合并相关语法。

周边配套

OceanBase 数据库 V4.3.0_CE 版本推荐使用的平台工具版本如下。

组件	版本	备注
ODP	V4.2.3	-
OCP	V4.2.2	-
OBD	V2.7.0	-
ODC	V4.2.3 BP1	-
OBCDC	V4.3.0	-
OMS	V4.2.2	仅支持 OBServer V4.3.0_CE 版本作为目标端；不支持拉取 OBServer V4.3.0_CE 版本增量数据。
OBClient	V2.2.3	-
LibOBClient	V2.2.3	-

升级说明

• 初次使用 V4.3.0 版本时，需新建集群使用。暂不支持 V1.x、V2.x、V3.x、V4.1、V4.2.x 等低版本到 V4.3.0 Beta 版本的平滑升级。如有升级需求，请使用逻辑导入导出方式。
• 后续会支持 V4.3.0 Beta 版本升级到 V4.3.x 新版本。
• 随着版本演进，后续会增加 V4.2.x 到 V4.3.x 的升级路径支持。

注意事项

• 建议 旁路导入最大并行度 = 租户内存 * 0.001 / 2M，超过此并行度，可能会存在临时文件内存不足的情况。使用旁路导入时，建议将大文件切割成多个小文件，多文件导入可提高导入效率。
• OMS 尚未适配 MINIMAL 模式。通过工具进行增量数据同步的场景，暂时禁止将 binlog_row_image 设为 MINIMAL（默认值是 FULL）。

开源鸣谢

在此版本发布中，特别感谢社区伙伴的贡献：

• 感谢 Intel 团队 @xtangxtang 和 @gangdeng-intel 在 “zlib_lite_1.0压缩算法” 功能上的贡献。

• 感谢携程团队 @Phoeniwx 和 @jiangxianfu 在开发 “索引使用监控” 功能上的贡献。