Fixes included:
- BigQuery: Bigquery conditionally support upsert if table has PK/UK.2.
- All: Add Azure Key Vault support as an option for storing credentials.
- Oracle: Support oracle rac from native reader.
- PostgreSQL: PG auto-reinit executing DDL in loop.
- DB2: DB2 incorrect node specification results in at tech.replicant.db.DBReplicationManager.a(SourceFile:3167).
- NullStorage: Improve throughput of NULLSTORAGE destination.
- All: Make sure all configurations referenced (mapper, general, etc) and all parameters within a configuration are written to the log.
- PostgreSQL: Stabilizing Postgres to Databricks.
- All: Enable KRYO format in transaction store serialization.
- Oracle: Oracle to Postgres full replication causes errors.
- SQLServer:Add a new snapshot write mode that will snapshot to a temp table and then swap it for the live table.
- DB2: Db2 IIDR Kafka - Reconcile Cursor on Kafka Rebalance.
- All: Add fetchPK/UK/FK/Indexes to per-table config in snapshot extractor config.
- All: Upsert job range recovery automatically commits the job if recovery table already has it.
- All: Extend InternalSerializer support to all relevant IOperation implementations.
- All: Merge Operation with DBOperation abstract class and IOperation with IDBOperation interface.
- CockroachDB: Enable sybase and mysql to cockroach.
- DB2: Db2 IIDR Kafka - Prevent Skipping Kafka Commit.
- DB2: Suppress Transform Error Log Message.
- All: Change the heartbeat table to allow a single table for all replication jobs.
- All: Add an option to rotate the trace file based on size as well as time.
- Oracle: Enable full replication for Oracle PDBs.
- All: Change the console to include the version and replication ID.
- PostgreSQL: Snapshot not extracting by parent table(partitioned table).
- MySQL:MySQLConnConfigForm not calling super in validate method.
- SQLServer:Replication error while testing of large datatypes i.e, VARCHAR(MAX) and NVARCHAR(MAX) from SQLServer to Postgresql.
- SQLServer: Issue with datetime2 datatype of sqlserver when tested on SQLServer to Singlestore pipeline.
- SQLServer: Use seperate extractorId for each table in sqlserver CDC mode.
- Cassandra: IllegalStateException of MVStore, due to race condition.
- Oracle: Oracle to pg devault value for timestamp.
- Oracle: Fix oracle sqlloader on 23.07.32.
- DB2: Log if XA Batch takes longer than 60 seconds.
- BigQuery: BQ default storage type.
- All: Improve security of OFFLINE licenses.
- Databricks: Minor improvement and Unit testing for DatabricksUrl parsing.
- Oracle: Fix ctor for cdb-connection.
- DB2: DB2 Kafka - User Configurable AUTO_OFFSET_RESET_CONFIG.
- DB2: DB2 IIDR Kafka - Prevent Applying Messages from Revoked Partitions.
- DB2: ClassCastException when Postgresql is used as Destination.
- SAP ASE: OFFLINE Sybase ASE licenses throw an exception.
- PostgreSQL: PostgreSQL SSL connections with host verification off should not verify the CA.
- All: Support AWS SM cross account access.
- MySQL:Mysql to oracle external tables fails on YCSB data.
- All: Improve/fix IRLobV serialization and add unit-tests for IRValue hierarchy.
- DB2: DB2 IIDR Kafka to S2 - Fix Mismatch After Skipping Messages From Revoked Partitions.
- DB2: DB2 LUW CI/CD tests are failing.
- DB2: Update all partitions during reconcile.
- DB2: Cursor Reconcile Breaks XA Catchup.
- DB2: Allow reconciling in any xa state if not in resume mode.
- Teradata: Fix Teradata offline licenses.
- Oracle: Oracle RAC Fix null pointer exception in CDB connection.
- All: Fix error propagation in AsyncCustomSerializer and support Update with different set/where column set.
- Oracle: Oracle Log Reader ASM: Handle missing file type info of ASM archive logs.
- DB2: Exit Replicant if SingleStore Online Leaf Count Changes.
- All: NPE if general config is not specified.
- All: Split the replicant-core to replicant-core project & replicant-core-common.
- Databricks: Fix Databricks DBFS staging.
- DB2: Support basic authentication for Kafka schema registry.
- MySQL: Old MySQL Server ID being grabbed for cursor movement.
- Oracle: Oracle to mysql: DUplicate PK error in TPCC benchmark run.
- MySQL: MySQL ssl validation fails even if SSL is not used.
- MySQL: Mysql SSL enable property check can throw null pointer exception if enable not provided.
â
- Pipeline source - RDS
- Organization endpoint called needlessly to many times
- Unable to save Postgresql target
- Handle firebase error on create-account page
- Not able to upload connector file
- Fix value remembered on password input
- Handle 'Unauthorized email domain' firebase error
- UI not resuming or restarting as stated
- Redesign settings page
- Clean up replication status typings in the code
- Clone signup page for Google BigQuery
- Cancel /schema request in browser on cancel
- Disable Gmail (and other disposable providers) sign-in
- Changes to support RDS PostgreSQL as source
- Pipeline destination - AlloyDB
- Pipeline destination - Google CloudSQL
- Actions are taking many times longer to execute than in past releases
- Lag - time is different than individual tables
- Forced PostgreSQL replication slot
- Forward fetch-schema trace log to UI in case of Core error
- New Connector Added to UI - AlloyDB
- Add support for new extractor field - fetchIdentityInfo
- Oracle destination support for Informix
- Mysql to MinIO snapshot with mapper results in mapper parse error
- Sybase - SSL support
- Add snapshot_summary.txt report to ZIP export
- Redis Stream - SSL support
- Support Transformation configuration YAML file upload
- MySQL as destination for Oracle
- Enable PG to BigQuery pipeline
- Changes to support CloudSQL MySQL as destination
- Changes to support CloudSQL PostgreSQL as destination
- Lazy loading schemas for filter
- Pipeline destination - AlloyDB
- Pipeline destination - Google CloudSQL
- Actions are taking many times longer to execute than in past releases
- Lag - time is different than individual tables
- Forced PostgreSQL replication slot
- Forward fetch-schema trace log to UI in case of Core error
- New Connector Added to UI - AlloyDB
- Add support for new extractor field - fetchIdentityInfo
- Oracle destination support for Informix
- Mysql to MinIO snapshot with mapper results in mapper parse error
- Sybase - SSL support
- Add snapshot_summary.txt report to ZIP export
- Redis Stream - SSL support
- Support Transformation configuration YAML file upload
- MySQL as destination for Oracle
- Enable PG to BigQuery pipeline
- Changes to support CloudSQL MySQL as destination
- Changes to support CloudSQL PostgreSQL as destination
- Lazy loading schemas for filter
- SINGLESTORE: cannot-load-null-or-0-into-auto-increment-columns-with-load-data.
- ALL: Add replicant command option to gracefully kill a job.
- Oracle: Fix bugs around Oracle upsert recovery.
- Oracle: Use data precision when creating float column.
- Sybase ASE: Allow Sybase ASE fetch internal to anything >= 1s, instead of 10s.
- Kafka: Use Committed Offsets Instead of Latest.
- Kafka: Print Kafka Offset Commit in debug log instead of verbose.
- DB2: Db2 IIDR Kafka Improve Committed Offset Handling.
- Oracle: Fix loading empty lob using Oracle Sqlldr.
- Oracle native reader: Support standby databases.
- DB2: Default to latest offset if committed unavailable.
- Oracle: Fixed a potential exception within an exception.
- CloudLogger: Enhancement during publishing pending logs.
Fixes included:
- Oracle: Oracle native reader not parsing the log properly when target isn't Oracle also2.
-Oracle: Oracle XMLType not working in native log reader3.
- Oracle: Oracle XML type not mapped properly for PostgreSQL4.
- Oracle: TIMESTAMP with time zone not mapped properly to postgreSQL timestamp with time zone5.
- Oracle: Oracle native reader shouldn't add a + for positive intervals, it's implied6.
- Oracle: Oracle null rowid columns cause snapshot to fail with an NPE7.
- DB2: Db2/IIDR/Kafka Sticky Assignor8.
- Oracle: Support IOT for Oracle native log reader9.
- Snowflake: Error Handling - Snowflake Target10.
- All: Propagate exception in Extractor and Applier call methods11.
- PostgreSQL: Support PG to PG inline DDL replication with all native data types12.
- MySQL: New feature to do idempotent transactions prior to real-time timestamp13.
- SQLServer: Enabling new extractor type for SQLServer14.
- PostgreSQL: PG to Kafka support partitioning table DDLs15.
- Kafka: Support  Azure Event Hubs for Apache Kafka16.
- CockroachDB: Fix CockroachDB connection issues17.
- DB2: Have DB2 IIDR Kafka message create timestamp as a separate column in target18.
- All: Support all cursor types in the new serialization code19.
- Oracle: Return ALL pairs value if AfterImageFormat is ALL for a source20.
- Oracle: Oracle native reader throws an exception for a row where all values are null21.
- Oracle: Fix OracleUserExtractor exception handling22.
- PostgreSQL: Postgres stream realtime table creation missing data23.
- Teradata: Issue when having special character Ă in Teradata snapshot24.
- Oracle: Fix NPE when data object id is null25.
- Teradata: Add Teradata connection config client-charset for customer user charsets26.
- DB2: Db2 iSeries ddl support27.
- Databricks: ParquetWriter: Disable Parquet Writer console logging28.
- Oracle: Fixed an issue mapping the Oracle float datatype to targets other than Oracle29.
- Oracle: Sort and filter out unwanted columns from ColumnPairs for IOT INSERT30.
- Oracle: Implement IOT updates31.
- Oracle: Implement IOT deletes32.
- Oracle: Implement IOT multi inserts33.
- Oracle: Fix positioning issue with multi insert34.
- Oracle: Fix positioning issue for opcode version 1735.
- Oracle: Fix column positioning for IOT updates36:
- Oracle: Fix IOT multi inserts37.
- All: Core obfuscation is broken on latest develop.
- Pipeline source - SAP ASE (Sybase)
- Pipeline destination - Redis Streams
- Buttons now viewable when scrolling on the webpages
- Can't click save on new Databricks connection when using DBFS
- Can't perform state update on saved-connections
- Filter page timeouts
- Filter update by file not handled properly
- When setting Pertable config for Databricks Lakehouse the schema name is not set into the per table configuration (only catalog name)
- Handle exception when /replication returns 401
- Allow schema name update when mapping to 2 level db
- Update refresh token behaviour
- UI changes to support SAP ASE (Sybase) as source
- API changes to support SAP ASE (Sybase) as source
- UI changes to support Redis Streams as destination
- API changes to support Redis Streams as destination
- Optimize filter behaviour when rendering a large dataset
- Update replication list header and make columns sortable
- Ability to download all config files
- Handle upload filter containing blocked list in the UI
- Enable PG to PG and MySQL to PG pipeline
- Type 2 CDC support
- MYSQL DDL replication support
- Informix DDL replication support
- Support applier configuration for /dev/null
- Update progress percentage for CHANGE_DATA_CAPTURE phase and TERMINATE replication status
- Reimplement trace log download within replication summary
- Pipeline source - SAP ASE (Sybase)
- Pipeline destination - Redis Streams
- Buttons now viewable when scrolling on the webpages
- Can't click save on new Databricks connection when using DBFS
- Can't perform state update on saved-connections
- Filter page timeouts
- Filter update by file not handled properly
- When setting Pertable config for Databricks Lakehouse the schema name is not set into the per table configuration (only catalog name)
- Handle exception when /replication returns 401
- Allow schema name update when mapping to 2 level db
- Update refresh token behaviour
- UI changes to support SAP ASE (Sybase) as source
- API changes to support SAP ASE (Sybase) as source
- UI changes to support Redis Streams as destination
- API changes to support Redis Streams as destination
- Optimize filter behaviour when rendering a large dataset
- Update replication list header and make columns sortable
- Ability to download all config files
- Handle upload filter containing blocked list in the UI
- Enable PG to PG and MySQL to PG pipeline
- Type 2 CDC support
- MYSQL DDL replication support
- Informix DDL replication support
- Support applier configuration for /dev/null
- Update progress percentage for CHANGE_DATA_CAPTURE phase and TERMINATE replication status
- Reimplement trace log download within replication summary
- Db2/Kafka to S2(XA) - Messages are replayed on all targets when a node is brought back online
- DB2 Kafka IIDR -> S2 XA Issue with kafka re-distributing messages
- Publish remaining logs to cloudLogger
- Make Oracle native reader multi-threaded
- MySQL split key hint should cover all values besides specified range
- For SQL Server, ensure all permissions are required for snapshot
- Log query when an error is returned from Oracle
- The encrypt-config command throws an NPE
- Support PDB Connections
- Modify replicant SQL Server log extractor to parse .TDS files directly to operations
- Test and Document SSL connection to MySQL
- Adding support for rollback and transactional store for ASE
- Notification was not sent on connection interuptions from server
- Support Kryo format for CDC opers
- Implement custom S3OutputStream and use it for CDC log segment generation
- Replication crashes if mapping used with tables which have FKs
- Improve Kerberos implementation
- Disallow retry and remove duplicate record check
- Improve applier cursor cache
- Have aggregate operations count in new relic
- Ability to have custom table properties while creating a table in Databricks
- Fix race condition in Oracle native reader
- Oracle native reader inserts are not properly formed
- DB2 realtime support for >>= <<= in transformation
- PostgreSQL Stream connection drops randomly
- Reverted a fix for S3 testConnection mode by accident
- Add thenString support to lt/gt transformation options
- Update extractor-governor config name in code to match with yaml/json config file
- Add thenString Snapshot Support for Native Transform
- Fix adding additional col in realtime mode and change default
- MySQL bin log fetching improvement
- Support KRYO serialization format for LOBs by de/serializing IRLobV instance
- Fixed a parsing error in SQL Server unicode string parsing
- Set the default for fetchExactRowCount to false
- Fix null value handling for memsql/singlestore
- Kafka as target - Fix NPE while replaying delete with shardkey
- Fix for the Oracle logminer extractor where comments in DDL were causing an exception
- Kafka offset logging improvements
- Add max-poll-records and max-poll-interval-ms to kafka config
- with the oracle native log reader extractor nchar nvarchar2 and nclob fail when sent to target
- BigQuery does not supports addconstraint oper
- Kafka as target: Ignore per table num-shards setting if replay-consistency is GLOBAL
- Â Fix NPE in test-connection for S3
- Fix Databricks partition pruning
- Kafka test connection mode support
- Null Applier throws error on realtime resume
- Add delta snapshot support for Snowflake
- Handling special values like nan,inf,-inf for floating point datatypes
- Fixing IRDecimal isNumericConvertible function
- Fixed range issue for Number(38,0) datatype for Snowflake
- Fixed crashes when running with reinit
- Fixed crash when creating table for very large varchar size
- Error when inserting row from Oracle to BigQuery.
- Npe when replicating from MySQL to BigQuery
- MySQL to Snowflake snapshot creates tables def and uploads data to stage but the actual.
- Error changing datatype
- BigQuery test connection and fetch schema do not work username password are required
- Inline ddl error for table addition
- Postgres inline created table data not getting replicated
- Inline ddl table drop throws error
- Kafka as target fixes
- Fix alter column sql for PG target
Main feature updates:
- S3 pipeline
- OAuth2 authentication support (tested with Okta)
- In-memory authentication custom password support
- Prometheus metrics exposed + Grafana dashboard
- Oracle PDB support
Detail fixes:
- Pipeline destination - S3
- Mapper not added to replication if added after the initial creation
- Errors about duplicate keys and other errors
- Need ability to fall back to non-partition table creation
- Allow the in memory authentication type to configure a different password (Docker Secret)
- Support OpenID authentication via providers like Okta (without password grant)
- Add Prometheus metric endpoint
- Add support for DDL replication
- Expose JDBC URL for connections that support it (MySQL first and foremost)
- Do not explicitly list tables in filter if the whole catalog/schema is selected (same for table and columns)
- Add support for schema validation configuration (general.yaml)
- Enable S2->GBQ and S2->S3
- Liveness monitor, schema validation and metadata connections configuration support
- Changes to support S3 as destination
- Add ability to specify custom NTP server
- PDB support
- Configure Core logs rotation
- Cleanly exit Replicant process instead of force kill
- Allow permission validation to be enabled/disabled via UI
âMain feature updates:
- S3 pipeline
- OAuth2 authentication support (tested with Okta)
- In-memory authentication custom password support
- Prometheus metrics exposed + Grafana dashboard
- Oracle PDB support
Detail fixes:
- Pipeline destination - S3
- Mapper not added to replication if added after the initial creation
- Errors about duplicate keys and other errors
- Need ability to fall back to non-partition table creation
- Allow the in memory authentication type to configure a different password (Docker Secret)
- Support OpenID authentication via providers like Okta (without password grant)
- Add Prometheus metric endpoint
- Add support for DDL replication
- Expose JDBC URL for connections that support it (MySQL first and foremost)
- Do not explicitly list tables in filter if the whole catalog/schema is selected (same for table and columns)
- Add support for schema validation configuration (general.yaml)
- Enable S2->GBQ and S2->S3
- Liveness monitor, schema validation and metadata connections configuration support
- Changes to support S3 as destination
- Add ability to specify custom NTP server
- PDB support
- Configure Core logs rotation
- Cleanly exit Replicant process instead of force kill
- Allow permission validation to be enabled/disabled via UI
Main feature updates:
- BigQuery pipeline
- Kafka pipeline
- Confluent pipeline- /dev/null destination support
- Databricks Lakehouse UI rework & Azure/S3 support
Detail fixes:
- Pipeline destination - BigQuery
- Pipeline destination - Kafka
- Pipeline destination - Confluent
- Support /dev/null destination for extractor performance testing
- Allow download of full Replicant trace log
- Allow disabling IPv6 in container
- Cannot save YugabyteSQL applier
- Disable role creation as default for all replication
- Configuration edit not working properly
- Missing restart for finished replications
- Oracle to Big Query Fails due to connection issue
- Google Big Query Applier - Replay Strategy Not Being Recorded
- Query reply-strategy not being printed
- PostgreSQL default port listed as hostname
- Oracle Applier unable to save
- Oracle roles still being fetched even when unchecked in extractor
- Cloud wording on front screen
- Catalog name undefined in the per table target configuration
- Pre-populate default port for all connectors
- Add IN_MEMORY_MERGE applier support
- Add ability to skip row counts to extractor
- Update connection with unsaved changes on test
- Databricks applier - save-file-on-error - default to false
- Add maxFilesPerBulkLoad to applier
- Add Azure and S3 support to Lakehouse
- Realtime replication mode support + MySQL cursor support
- Connector list - alphabetical sort
- MySQL extractor - expose idleBinLogTimeoutSeconds in realtime
Main feature updates:
- BigQuery pipeline
- Kafka pipeline
- Confluent pipeline- /dev/null destination support
- Databricks Lakehouse UI rework & Azure/S3 support
Detail fixes:
- Pipeline destination - BigQuery
- Pipeline destination - Kafka
- Pipeline destination - Confluent
- Support /dev/null destination for extractor performance testing
- Allow download of full Replicant trace log
- Allow disabling IPv6 in container
- Cannot save YugabyteSQL applier
- Disable role creation as default for all replication
- Configuration edit not working properly
- Missing restart for finished replications
- Oracle to Big Query Fails due to connection issue
- Google Big Query Applier - Replay Strategy Not Being Recorded
- Query reply-strategy not being printed
- PostgreSQL default port listed as hostname
- Oracle Applier unable to save
- Oracle roles still being fetched even when unchecked in extractor
- Cloud wording on front screen
- Catalog name undefined in the per table target configuration
- Pre-populate default port for all connectors
- Add IN_MEMORY_MERGE applier support
- Add ability to skip row counts to extractor
- Update connection with unsaved changes on test
- Databricks applier - save-file-on-error - default to false
- Add maxFilesPerBulkLoad to applier
- Add Azure and S3 support to Lakehouse
- Realtime replication mode support + MySQL cursor support
- Connector list - alphabetical sort
- MySQL extractor - expose idleBinLogTimeoutSeconds in realtime
Fix included:
1. Connect to oracle using local tns file
2. Changed build.gradle repository order
3. Fixing SnapshotRecovery after changes introduced by a new MR
4. Native Transform Timestamp Less Than
5. BigQuery set realtime replay strategy as AUTO
6. Do not consider blocked columns from the validation of FKs
7. SQL Server Log based extractor threading issue
8. SQServer --> RedisStream Accumulated Insert Logs during snapshot comes as Insert Oper instead of Replace
9. DB2 LUW Log Reader Handle BINARY Data Type
10. Redis Stream : Code cleanup
11. MongoDB realtime mode error when start position is mentioned
12. Add DB2 LUW as source for Debezium format messages
13. Add source extraction timestamp and operation as new columns to target tables
14. Add liveliness memory stats to metadatadb and stats file
15. getClusteringKeySql throws exception if Snowflake is used as source
16. Credential store support for Databricks
17. Support DefaultCredentialsProvider for authentication on S3_STORAGE_BROKER.
18. Db2/MQ/Kafka->Singlestore --continue-inconsistent-post-failure is not working for dst-query failures
19. Test and Document SSL connection to SAP Sybase ASE
20. Refactor fencing workflow for producer replication in the streaming storage setup
21. Error Handling - SingleStore/Mysql Target
22. Add and test SSL connection support to SingleStore
23. Sync CDC data stream through S3 with a independent snapshot migration
24. Db2 AS400 Table containing Column type as Time is throwing exception
25. Postgresql(source) DDL support
26. Oracle: Provide Alternative to Deletion Behavior Upon Resume
27. Extract exceptions used in forms to standalone classes
28. YamlUtil InputStream load methods changed to accept TriFunction that returns Composer implementation
29. FormUtils.validateNamespace() logic moved to standalone class NamespaceValidator
30. PG-DDL inline created table are having string data type for all columns
31. Oracle as a target results in exception when source isn't also Oracle
32. SQLÂ Server to Oracle DEFAULT CURRENT_TIMESTAMP cannot be translated
33. Fix Databricks connection URL, when the password contains special characters
34. Fix Oracle source CDC for inserting rowid datatype
35. Add Minio support
36. Fix NPE in MysqlRealtimeExtractorForm due to binLogIdleTimeoutS
37. Db2 luw decfloat result in parsing issue
- Oracle to Oracle pipeline
- SQL Server CDC agent support
- Databricks Deltalake UI rework & Azure/DBFS support
- Whole SaaS stack migrated to Java 17
- Computed columns support for MySQL
- Forward test-connection trace log to UI in case of Core error
- Add Azure stage support to Databricks Delta Lake
- Missing ârestartâ option for stopped replications
- Increase Replicant memory limit
- Update Sign In & Sign Up pages
- Hung on current screen (loading replication data)
- Connection details not opening
- Connection created successfully duplicated
- Replication statuses phasing in and out
- Connection test does not accurately represent what is needed for replication to succeed
- Databricks connection not working with or without stage
- Table filter search not working
- Schema not being refreshed after fetchSchema
- Update the current deadlock solution
- Fix tooltip overflow issues
- Update connection logo image files to fit to container box
- Generated columns support
- Applier BulkLoad charset support
- Increase maximum YAML schema input size
- Management service port issues on WSL2
- Snapshot applier additions - skip table failures and init indexes
- Enable APPENDING write mode in UI
- Oracle to Oracle pipeline
- SQL Server CDC agent support
- Databricks Deltalake UI rework & Azure/DBFS support
- Whole SaaS stack migrated to Java 17
- Computed columns support for MySQL
- Forward test-connection trace log to UI in case of Core error
- Add Azure stage support to Databricks Delta Lake
- Missing 'restart' option for stopped replications
- Increase Replicant memory limit
- Update Sign In & Sign Up pages
- Hung on current screen (loading replication data)
- Connection details not opening
- Connection created successfully duplicated
- Replication statuses phasing in and out
- Connection test does not accurately represent what is needed for replication to succeed
- Databricks connection not working with or without stage
- Table filter search not working
- Schema not being refreshed after fetchSchema
- Update the current deadlock solution
- Fix tooltip overflow issues
- Update connection logo image files to fit to container box
- Generated columns support
- Applier BulkLoad charset support
- Increase maximum YAML schema input size
- Management service port issues on WSL2
- Snapshot applier additions - skip table failures and init indexes
- Enable APPENDING write mode in UI
- Core 23.02.01.9
- MySQL -> MySQL pipeline
- Dashboard rework with filtering by mode and virtual list support (to allow smooth scroll with huge numbers of replications)
- Redesign of the settings menu, various UI fixes
- Support for paid cluster pause/resume/terminate
- AWS password rotation support in the Docker container (both for Metadata DB and replications)
- S3 as the shared config storage for clustered mode
- In addition to the latest Docker tag we now have the test tag - which will point to the next container version with the latest core release (for PoC etc.)
- Core 23.02.01.9
- MySQL -> MySQL pipeline
- Dashboard rework with filtering by mode and virtual list support (to allow smooth scroll with huge numbers of replications)
- Redesign of the settings menu, various UI fixes
- Support for paid cluster pause/resume/terminate
- AWS password rotation support in the Docker container (both for Metadata DB and replications)
- S3 as the shared config storage for clustered mode
- In addition to the latest Docker tag we now have the test tag - which will point to the next container version with the latest core release (for PoC etc.)
- Core 23.01.05.9- Improvements to our on-premises APIs
- Configuration YAML upload fixed
- Expose information about cluster state
- Refresh connection logos with vector ones
- Support for Oracle native log reader (ASM only on Cloud, Redo Logpath additionally for on-premises)
- Audit logging support for user actions and events
- Pass license into Docker as Base64 environment variable
- Support ARM64 architecture for on-premises container image (look for tag latest-arm64 on DockerHub)
- Show days left on license for on-premises
- Support SQL Server Azure as source
- Various UI improvements and bug fixes
- Core 23.01.05.9- Improvements to our on-premises APIs
- Configuration YAML upload fixed
- Expose information about cluster state
- Refresh connection logos with vector ones
- Support for Oracle native log reader (ASM only on Cloud, Redo Logpath additionally for on-premises)
- Audit logging support for user actions and events
- Pass license into Docker as Base64 environment variable
- Support ARM64 architecture for on-premises container image (look for tag latest-arm64 on DockerHub)
- Show days left on license for on-premises
- Support SQL Server Azure as source
- Various UI improvements and bug fixes
Fixes included:
1. Generalize the Oracle extractor throttle.
2. Replicant Crashes for Delete operation in delta mode when Applier is configured in Non-BulkLoad Mode.
3. Configure mode - run into error in new version.
4. Error in Postgresql to Postgresql table creation for tables with default values.
5. MongoDB delete not working.
6. Duplicate PK issue in certain idempotent scenario.
7. Databricks connection file with AWS secrets - failing on recent versions.
8. Test and document ssl connection to Snowflake.
9. Test and document ssl connection to BigQuery.
10. Delta-snapshot mode throws NPE when resuming replication containing compound columns.
11. SQL Server datetime columns with CURRENT_TIMESTAMP default gets created incorrectly on MySQL.
12. Allow specifying metadata config for Redis Stream as target.
13. Replicant fails to start PG->PG due to DEFAULT column value syntax error.
14. Test and Document SSL connection to Databricks.
15. Applier wrong type resolved on XA form loading resulting in parse error.
16. Db2/MQ/Kafka Blocked Columns Causing Parsing Error.
17. Introduce IRealtimeOperationSerializer interface to be used for de/serialization per different CDC format (Kryo/JSON).
18. DB2/MQ/Kafka Db2DelimitedMsgParser Error On IBM Error Messages.
19. NPE in SQL Server fetch-schmas.
20. Move create-table-without-defaults-on-failure option to applier.
21. Moved static logic from DBSchemasForm into SchemasFormUtils and DBSchemasFormParser classes.
22. In config forms, changed visibility of validate method to public and some constructors as well.
23. Moved BaseContext instance holder from ReplicationContext to BaseContext.
24. SecretManager.initializeSecretManager() changed method call to accept vars instead of GeneralConfig.
25. Decoupled parse method from FilterForm/MapperForm/ProcForm/NativeLoadConfigForm/ObjectIdForm/PathForm classes into standalone class hierarchy. Made required modifications to DBMetatataProvider/DBMetadataProviderShared classes.
26. Improve Debezium format data representation.
27. Config to avoid of overlap of tables during extraction.
28. Support bigint data type to Kafka.
29: PostgreSQL - Change default value for connection config log-reader-type.
30. Fix Databricks job-wide snapshot recovery.
31. Databricks - Fix concurrent copy failure.
1. MySQL: support  GENERATED ALWAYS AS
2. In duplicate Pk scenario (parent issue), detect and handle the error by deleting the original row
3. NPE in SQL Server
4. PostgreSQL to Kafka CDC - Null Pointer Exception
5. Additional Error Handling Options for Kafka
6. Support configuring a custom NTP server in general configuration
7. Add --terminate-post-cdc-init flag for full mode replication
8.MongoDB to BigQuery CDC
9. Add CLI dashboard for the maintenance process
10. SQL Server computed columns should be handled more intelligently
11. Testing reinit based DDL replication for CDC source Informix
12. Embed functionality of --clean-stage to the --overwrite flag
13. Renaming parts of the setup
14. Fix fetch-schema for Redis Stream and Kafka as target
15. PostgreSQL: CREATE INDEX fails when using mapper file
16. Index creation attempted when column in block: [ ] list
17. Feature Request - Yugabyte Table split clause
18. Redis Stream and Kafka  ( JSON Format )
19. DBSnapshotExtractorParser wrong class call.
20. Remove OPER_TYPE as PK from delta table
21. Resume replication failing due to replicationcontext not initialized
22. Set the default value retry-transactions to true only for ACID destinations
23. Deprecate and disable databricks _optimize-snapshot by default
24. Mongo Pipeline breaking due to version change
25. Kafka as target : Override snapshot config num-shards = 1 when...
26. Fix Azure Databricks when optimizesnapshot is disabled
27. Add SAS token support for Azure stage.
28. Oracle -> Oracle Add missing recovery key columns in case of fetch-create-sql enabled.
29. Incorporating extractor config computed-columns to support generated columns for MySQL and fixing snapshot of generated columns.
30. Set the blocking option for generated columns to false automatically when using --create-sql/fetch-create-sql.
â
1. DB2-MQ-Kafka to S2 - Duplicate entry for key 'PRIMARY'
2. Bad SQL cause ORA-00913: too many values
3. DB2-MQ-Kafka to S2 - Update failing with duplicate key error after successful insert
4. Fix parsing of connection stage config
5. Support XML
6. Support multiple replay strategy for BigQuery as target
7. Support MySQL RDS password rotation
8. Support auto-reinit if DDL fails
9. Upgrade S3 SDK version to 2.*
10. MySQL index creation error
11. Option to add prefix to topic
12. Db2 Kafka - Kafka logging improvements
13. Option to allow dot(.) as a delimiter in Kafka topic names
14. Fix system generated split-key and delta-snapshot-key handling for HANA as source
15. Initialize ASE source fields in target JSON format log ( for Kafka and Redis Stream )
16. Transforms Not Applying When Src-schemas is used
17. Log Delta Table Verbose
18. AWS SecretsManager - error parsing secretKeyPath with /
19. Setting csv-publish-method: READ as default when column transformation is used
20. DB2-MQ-Kafka - Update on timestamp column not working
21. Enable compression-type: GZIP for MemSQL/SingleStore bulk load
22. Redis Stream : Improve logging under snapshot config log-row-level-errors
23. Fix NPE in GLOBAL replay consistency validation code
24. Fix issue where we only allow connecting to a named instance by port
25. Fix for SQL Server change tracking and re-enable CI/CD tests for SQL Server
26. Fixed SQLServer fetchObjectIds() function, using schema name to filter tables from SQLServer
1. Add Voodoo LogReader as extraction method for Oracle
2. Redis Streams as a Target
3. Databricks as target : MERGE-Based Strategy ( When we don't have full before/after image )
4. Test Snowflake Iceberg Tables
5. Improve cdc log segment cache
6. Db2 As400 Add logging for commitment control
7. AS400 Support Transaction Rollback
8. Support realtime BigQuery as target using storage Write API
9. Support reinit based DDL Replication for SAPÂ ASE Sybase Source
10. Added support for delta-snapshot both src and dst for SQL Server.
â
1. Add replication ID to the transaction path for Oracle uncommitted CDC
2. Support Writing All Data to a Single Kafka Topic (From Informix and Postgres)
3. Integration with AWS Secrets Manager (or Similar Service)
4. Write to CDC log segment in chunks
5. Add stage maintenance mode
6. NPE in Snowflake initializer
7. SQL Server incorrectly blocking system schemas
8. --stop-never-slave-server-id param in MySQL is deprecated.
9. RDS PostgreSQL to RDS Postgres - replicate_io_replication_stats_summary_coreoct11_12_coreoct11_12 does not exist
10. Handle error: ORA-02149 Specified partition does not exist
11. SQLServer - fetch-schema not fetching the database specified in connection configuration
12. Test and fix source column transformation
13. MySQL as Source in real-time replication crashes when we insert into column of type year
14. Azure SQL to YugabyteSQL - Column default does not created at target
15. Databricks S3 Role arn parameter
16. Fix for cdc recovery across all pipelines
17. Ability to Surface Alerts in New Relic
18. Enable init-metadata mode for DB2
19. Fix incorrect exception when logging failed transaction
20. DB2 AS400 tables with table name length greater than 10, realtime replication is not working
21. SQL Server heartbeat doesn't work when there are multiple databases being replicated
22. Add a table level check in Oracle for full column logging when Databricks is a target
23. Table level column logging check not working
24. Sybase ASE snapshot but on release binary 22 10 28 1
Features:
1. Separate transaction files by replication ID for SQL Server CDC
2. Oracle Native Export
3. Databricks Unity catalog support
4. Allow use of Salesforce ID or Id Mod Column as Split Key
5. Add support for a credential store for the Replicant SQL Agent
6. GCP Databricks as destination
â
Fixes:
1. Db2 to Singlestore snapshot, TIMESTAMP column is shown as TIMESTMP in fetch-schema and is converted to varchar in singlestore
2. SQL Server CDC incorrectly creates multiple cursors for schemas within a database
3. Imply does not drop existing table in reinit mode if table already present
4. Change active connection logging to include TPT connections
5. Oracle to Oracle: Real-time SCN problem.
6. Add support for a credential store for the Replicant SQL Agent
7. SQL CDC should send ACKs for each file transfer
8. Decouple exit codes from custom exceptions
9. Upgrade Databricks jdbc driver
10. GCP Databricks as destination
11. Db2 LUW/As400 Losing Records When Transaction Size > 100000
12. PostgreSQL as target doesn't close files after bulk load
â
Features:
1. SAP ASEas source for realtime/full mode support.
2. OracleDBLINK based snapshot.
3. Support for Kerberos Authentication.
4. Support BigQuery as a target.
5. AddUpsert support for SingleStore.
6. EnableEmail Notifications if Lag is ABOVE a particular threshold value.
Â
Fixes:
1. MariaDB-> Google BigQuery : BigQuery not support AUTO_INCREMENT or IDENTITY.
2. Fixextractor NPE when replicating Oracle views.
3. DB2AS400 Schema with Schema name length greater than 10, realtime replication isnot working.
4. Changing Imply Implementation for their new apis.
5. Fix issues with TiDB as a target.
6. Invalidvalue when replicating table with set datatype in realtime mode.
7. ExtendDb2 XML TRANSACTION MQ message type to Kafka-Transactional log-storage.
8. Addsupport for DB2 TRANSACTION message type in DELIMITED message format.
9. Olapduplicate row insert error.
10. Snowflake target improvements.
11. MongoDBconnection needs to be established through url or connection config properties(host, port, username and password).
12. SAP ASECDC Performance testing.
13. Snowflake table cdcStageType not working for insert.
14. Number of rows mismatch while replicating updates ondatabricks using MERGE.
15. TestTargets with ASE CDC Source.
16. fixingper table replay strategy in realtime mode.
17. Snowflake bug fixing for delete query.
18. Snapshot mode not finished when trying to replicate table with SET Type.
â
Fixes included:
- BigQuery: Bigquery conditionally support upsert if table has PK/UK.2.
- All: Add Azure Key Vault support as an option for storing credentials.
- Oracle: Support oracle rac from native reader.
- PostgreSQL: PG auto-reinit executing DDL in loop.
- DB2: DB2 incorrect node specification results in at tech.replicant.db.DBReplicationManager.a(SourceFile:3167).
- NullStorage: Improve throughput of NULLSTORAGE destination.
- All: Make sure all configurations referenced (mapper, general, etc) and all parameters within a configuration are written to the log.
- PostgreSQL: Stabilizing Postgres to Databricks.
- All: Enable KRYO format in transaction store serialization.
- Oracle: Oracle to Postgres full replication causes errors.
- SQLServer:Add a new snapshot write mode that will snapshot to a temp table and then swap it for the live table.
- DB2: Db2 IIDR Kafka - Reconcile Cursor on Kafka Rebalance.
- All: Add fetchPK/UK/FK/Indexes to per-table config in snapshot extractor config.
- All: Upsert job range recovery automatically commits the job if recovery table already has it.
- All: Extend InternalSerializer support to all relevant IOperation implementations.
- All: Merge Operation with DBOperation abstract class and IOperation with IDBOperation interface.
- CockroachDB: Enable sybase and mysql to cockroach.
- DB2: Db2 IIDR Kafka - Prevent Skipping Kafka Commit.
- DB2: Suppress Transform Error Log Message.
- All: Change the heartbeat table to allow a single table for all replication jobs.
- All: Add an option to rotate the trace file based on size as well as time.
- Oracle: Enable full replication for Oracle PDBs.
- All: Change the console to include the version and replication ID.
- PostgreSQL: Snapshot not extracting by parent table(partitioned table).
- MySQL:MySQLConnConfigForm not calling super in validate method.
- SQLServer:Replication error while testing of large datatypes i.e, VARCHAR(MAX) and NVARCHAR(MAX) from SQLServer to Postgresql.
- SQLServer: Issue with datetime2 datatype of sqlserver when tested on SQLServer to Singlestore pipeline.
- SQLServer: Use seperate extractorId for each table in sqlserver CDC mode.
- Cassandra: IllegalStateException of MVStore, due to race condition.
- Oracle: Oracle to pg devault value for timestamp.
- Oracle: Fix oracle sqlloader on 23.07.32.
- DB2: Log if XA Batch takes longer than 60 seconds.
- BigQuery: BQ default storage type.
- All: Improve security of OFFLINE licenses.
- Databricks: Minor improvement and Unit testing for DatabricksUrl parsing.
- Oracle: Fix ctor for cdb-connection.
- DB2: DB2 Kafka - User Configurable AUTO_OFFSET_RESET_CONFIG.
- DB2: DB2 IIDR Kafka - Prevent Applying Messages from Revoked Partitions.
- DB2: ClassCastException when Postgresql is used as Destination.
- SAP ASE: OFFLINE Sybase ASE licenses throw an exception.
- PostgreSQL: PostgreSQL SSL connections with host verification off should not verify the CA.
- All: Support AWS SM cross account access.
- MySQL:Mysql to oracle external tables fails on YCSB data.
- All: Improve/fix IRLobV serialization and add unit-tests for IRValue hierarchy.
- DB2: DB2 IIDR Kafka to S2 - Fix Mismatch After Skipping Messages From Revoked Partitions.
- DB2: DB2 LUW CI/CD tests are failing.
- DB2: Update all partitions during reconcile.
- DB2: Cursor Reconcile Breaks XA Catchup.
- DB2: Allow reconciling in any xa state if not in resume mode.
- Teradata: Fix Teradata offline licenses.
- Oracle: Oracle RAC Fix null pointer exception in CDB connection.
- All: Fix error propagation in AsyncCustomSerializer and support Update with different set/where column set.
- Oracle: Oracle Log Reader ASM: Handle missing file type info of ASM archive logs.
- DB2: Exit Replicant if SingleStore Online Leaf Count Changes.
- All: NPE if general config is not specified.
- All: Split the replicant-core to replicant-core project & replicant-core-common.
- Databricks: Fix Databricks DBFS staging.
- DB2: Support basic authentication for Kafka schema registry.
- MySQL: Old MySQL Server ID being grabbed for cursor movement.
- Oracle: Oracle to mysql: DUplicate PK error in TPCC benchmark run.
- MySQL: MySQL ssl validation fails even if SSL is not used.
- MySQL: Mysql SSL enable property check can throw null pointer exception if enable not provided.
- SINGLESTORE: cannot-load-null-or-0-into-auto-increment-columns-with-load-data.
- ALL: Add replicant command option to gracefully kill a job.
- Oracle: Fix bugs around Oracle upsert recovery.
- Oracle: Use data precision when creating float column.
- Sybase ASE: Allow Sybase ASE fetch internal to anything >= 1s, instead of 10s.
- Kafka: Use Committed Offsets Instead of Latest.
- Kafka: Print Kafka Offset Commit in debug log instead of verbose.
- DB2: Db2 IIDR Kafka Improve Committed Offset Handling.
- Oracle: Fix loading empty lob using Oracle Sqlldr.
- Oracle native reader: Support standby databases.
- DB2: Default to latest offset if committed unavailable.
- Oracle: Fixed a potential exception within an exception.
- CloudLogger: Enhancement during publishing pending logs.
Fixes included:
- Oracle: Oracle native reader not parsing the log properly when target isn't Oracle also2.
-Oracle: Oracle XMLType not working in native log reader3.
- Oracle: Oracle XML type not mapped properly for PostgreSQL4.
- Oracle: TIMESTAMP with time zone not mapped properly to postgreSQL timestamp with time zone5.
- Oracle: Oracle native reader shouldn't add a + for positive intervals, it's implied6.
- Oracle: Oracle null rowid columns cause snapshot to fail with an NPE7.
- DB2: Db2/IIDR/Kafka Sticky Assignor8.
- Oracle: Support IOT for Oracle native log reader9.
- Snowflake: Error Handling - Snowflake Target10.
- All: Propagate exception in Extractor and Applier call methods11.
- PostgreSQL: Support PG to PG inline DDL replication with all native data types12.
- MySQL: New feature to do idempotent transactions prior to real-time timestamp13.
- SQLServer: Enabling new extractor type for SQLServer14.
- PostgreSQL: PG to Kafka support partitioning table DDLs15.
- Kafka: Support  Azure Event Hubs for Apache Kafka16.
- CockroachDB: Fix CockroachDB connection issues17.
- DB2: Have DB2 IIDR Kafka message create timestamp as a separate column in target18.
- All: Support all cursor types in the new serialization code19.
- Oracle: Return ALL pairs value if AfterImageFormat is ALL for a source20.
- Oracle: Oracle native reader throws an exception for a row where all values are null21.
- Oracle: Fix OracleUserExtractor exception handling22.
- PostgreSQL: Postgres stream realtime table creation missing data23.
- Teradata: Issue when having special character Ă in Teradata snapshot24.
- Oracle: Fix NPE when data object id is null25.
- Teradata: Add Teradata connection config client-charset for customer user charsets26.
- DB2: Db2 iSeries ddl support27.
- Databricks: ParquetWriter: Disable Parquet Writer console logging28.
- Oracle: Fixed an issue mapping the Oracle float datatype to targets other than Oracle29.
- Oracle: Sort and filter out unwanted columns from ColumnPairs for IOT INSERT30.
- Oracle: Implement IOT updates31.
- Oracle: Implement IOT deletes32.
- Oracle: Implement IOT multi inserts33.
- Oracle: Fix positioning issue with multi insert34.
- Oracle: Fix positioning issue for opcode version 1735.
- Oracle: Fix column positioning for IOT updates36:
- Oracle: Fix IOT multi inserts37.
- All: Core obfuscation is broken on latest develop.
- Db2/Kafka to S2(XA) - Messages are replayed on all targets when a node is brought back online
- DB2 Kafka IIDR -> S2 XA Issue with kafka re-distributing messages
- Publish remaining logs to cloudLogger
- Make Oracle native reader multi-threaded
- MySQL split key hint should cover all values besides specified range
- For SQL Server, ensure all permissions are required for snapshot
- Log query when an error is returned from Oracle
- The encrypt-config command throws an NPE
- Support PDB Connections
- Modify replicant SQL Server log extractor to parse .TDS files directly to operations
- Test and Document SSL connection to MySQL
- Adding support for rollback and transactional store for ASE
- Notification was not sent on connection interuptions from server
- Support Kryo format for CDC opers
- Implement custom S3OutputStream and use it for CDC log segment generation
- Replication crashes if mapping used with tables which have FKs
- Improve Kerberos implementation
- Disallow retry and remove duplicate record check
- Improve applier cursor cache
- Have aggregate operations count in new relic
- Ability to have custom table properties while creating a table in Databricks
- Fix race condition in Oracle native reader
- Oracle native reader inserts are not properly formed
- DB2 realtime support for >>= <<= in transformation
- PostgreSQL Stream connection drops randomly
- Reverted a fix for S3 testConnection mode by accident
- Add thenString support to lt/gt transformation options
- Update extractor-governor config name in code to match with yaml/json config file
- Add thenString Snapshot Support for Native Transform
- Fix adding additional col in realtime mode and change default
- MySQL bin log fetching improvement
- Support KRYO serialization format for LOBs by de/serializing IRLobV instance
- Fixed a parsing error in SQL Server unicode string parsing
- Set the default for fetchExactRowCount to false
- Fix null value handling for memsql/singlestore
- Kafka as target - Fix NPE while replaying delete with shardkey
- Fix for the Oracle logminer extractor where comments in DDL were causing an exception
- Kafka offset logging improvements
- Add max-poll-records and max-poll-interval-ms to kafka config
- with the oracle native log reader extractor nchar nvarchar2 and nclob fail when sent to target
- BigQuery does not supports addconstraint oper
- Kafka as target: Ignore per table num-shards setting if replay-consistency is GLOBAL
- Â Fix NPE in test-connection for S3
- Fix Databricks partition pruning
- Kafka test connection mode support
- Null Applier throws error on realtime resume
- Add delta snapshot support for Snowflake
- Handling special values like nan,inf,-inf for floating point datatypes
- Fixing IRDecimal isNumericConvertible function
- Fixed range issue for Number(38,0) datatype for Snowflake
- Fixed crashes when running with reinit
- Fixed crash when creating table for very large varchar size
- Error when inserting row from Oracle to BigQuery.
- Npe when replicating from MySQL to BigQuery
- MySQL to Snowflake snapshot creates tables def and uploads data to stage but the actual.
- Error changing datatype
- BigQuery test connection and fetch schema do not work username password are required
- Inline ddl error for table addition
- Postgres inline created table data not getting replicated
- Inline ddl table drop throws error
- Kafka as target fixes
- Fix alter column sql for PG target
Fix included:
1. Connect to oracle using local tns file
2. Changed build.gradle repository order
3. Fixing SnapshotRecovery after changes introduced by a new MR
4. Native Transform Timestamp Less Than
5. BigQuery set realtime replay strategy as AUTO
6. Do not consider blocked columns from the validation of FKs
7. SQL Server Log based extractor threading issue
8. SQServer --> RedisStream Accumulated Insert Logs during snapshot comes as Insert Oper instead of Replace
9. DB2 LUW Log Reader Handle BINARY Data Type
10. Redis Stream : Code cleanup
11. MongoDB realtime mode error when start position is mentioned
12. Add DB2 LUW as source for Debezium format messages
13. Add source extraction timestamp and operation as new columns to target tables
14. Add liveliness memory stats to metadatadb and stats file
15. getClusteringKeySql throws exception if Snowflake is used as source
16. Credential store support for Databricks
17. Support DefaultCredentialsProvider for authentication on S3_STORAGE_BROKER.
18. Db2/MQ/Kafka->Singlestore --continue-inconsistent-post-failure is not working for dst-query failures
19. Test and Document SSL connection to SAP Sybase ASE
20. Refactor fencing workflow for producer replication in the streaming storage setup
21. Error Handling - SingleStore/Mysql Target
22. Add and test SSL connection support to SingleStore
23. Sync CDC data stream through S3 with a independent snapshot migration
24. Db2 AS400 Table containing Column type as Time is throwing exception
25. Postgresql(source) DDL support
26. Oracle: Provide Alternative to Deletion Behavior Upon Resume
27. Extract exceptions used in forms to standalone classes
28. YamlUtil InputStream load methods changed to accept TriFunction that returns Composer implementation
29. FormUtils.validateNamespace() logic moved to standalone class NamespaceValidator
30. PG-DDL inline created table are having string data type for all columns
31. Oracle as a target results in exception when source isn't also Oracle
32. SQLÂ Server to Oracle DEFAULT CURRENT_TIMESTAMP cannot be translated
33. Fix Databricks connection URL, when the password contains special characters
34. Fix Oracle source CDC for inserting rowid datatype
35. Add Minio support
36. Fix NPE in MysqlRealtimeExtractorForm due to binLogIdleTimeoutS
37. Db2 luw decfloat result in parsing issue
Fixes included:
1. Generalize the Oracle extractor throttle.
2. Replicant Crashes for Delete operation in delta mode when Applier is configured in Non-BulkLoad Mode.
3. Configure mode - run into error in new version.
4. Error in Postgresql to Postgresql table creation for tables with default values.
5. MongoDB delete not working.
6. Duplicate PK issue in certain idempotent scenario.
7. Databricks connection file with AWS secrets - failing on recent versions.
8. Test and document ssl connection to Snowflake.
9. Test and document ssl connection to BigQuery.
10. Delta-snapshot mode throws NPE when resuming replication containing compound columns.
11. SQL Server datetime columns with CURRENT_TIMESTAMP default gets created incorrectly on MySQL.
12. Allow specifying metadata config for Redis Stream as target.
13. Replicant fails to start PG->PG due to DEFAULT column value syntax error.
14. Test and Document SSL connection to Databricks.
15. Applier wrong type resolved on XA form loading resulting in parse error.
16. Db2/MQ/Kafka Blocked Columns Causing Parsing Error.
17. Introduce IRealtimeOperationSerializer interface to be used for de/serialization per different CDC format (Kryo/JSON).
18. DB2/MQ/Kafka Db2DelimitedMsgParser Error On IBM Error Messages.
19. NPE in SQL Server fetch-schmas.
20. Move create-table-without-defaults-on-failure option to applier.
21. Moved static logic from DBSchemasForm into SchemasFormUtils and DBSchemasFormParser classes.
22. In config forms, changed visibility of validate method to public and some constructors as well.
23. Moved BaseContext instance holder from ReplicationContext to BaseContext.
24. SecretManager.initializeSecretManager() changed method call to accept vars instead of GeneralConfig.
25. Decoupled parse method from FilterForm/MapperForm/ProcForm/NativeLoadConfigForm/ObjectIdForm/PathForm classes into standalone class hierarchy. Made required modifications to DBMetatataProvider/DBMetadataProviderShared classes.
26. Improve Debezium format data representation.
27. Config to avoid of overlap of tables during extraction.
28. Support bigint data type to Kafka.
29: PostgreSQL - Change default value for connection config log-reader-type.
30. Fix Databricks job-wide snapshot recovery.
31. Databricks - Fix concurrent copy failure.
1. MySQL: support  GENERATED ALWAYS AS
2. In duplicate Pk scenario (parent issue), detect and handle the error by deleting the original row
3. NPE in SQL Server
4. PostgreSQL to Kafka CDC - Null Pointer Exception
5. Additional Error Handling Options for Kafka
6. Support configuring a custom NTP server in general configuration
7. Add --terminate-post-cdc-init flag for full mode replication
8.MongoDB to BigQuery CDC
9. Add CLI dashboard for the maintenance process
10. SQL Server computed columns should be handled more intelligently
11. Testing reinit based DDL replication for CDC source Informix
12. Embed functionality of --clean-stage to the --overwrite flag
13. Renaming parts of the setup
14. Fix fetch-schema for Redis Stream and Kafka as target
15. PostgreSQL: CREATE INDEX fails when using mapper file
16. Index creation attempted when column in block: [ ] list
17. Feature Request - Yugabyte Table split clause
18. Redis Stream and Kafka  ( JSON Format )
19. DBSnapshotExtractorParser wrong class call.
20. Remove OPER_TYPE as PK from delta table
21. Resume replication failing due to replicationcontext not initialized
22. Set the default value retry-transactions to true only for ACID destinations
23. Deprecate and disable databricks _optimize-snapshot by default
24. Mongo Pipeline breaking due to version change
25. Kafka as target : Override snapshot config num-shards = 1 when...
26. Fix Azure Databricks when optimizesnapshot is disabled
27. Add SAS token support for Azure stage.
28. Oracle -> Oracle Add missing recovery key columns in case of fetch-create-sql enabled.
29. Incorporating extractor config computed-columns to support generated columns for MySQL and fixing snapshot of generated columns.
30. Set the blocking option for generated columns to false automatically when using --create-sql/fetch-create-sql.
â
1. DB2-MQ-Kafka to S2 - Duplicate entry for key 'PRIMARY'
2. Bad SQL cause ORA-00913: too many values
3. DB2-MQ-Kafka to S2 - Update failing with duplicate key error after successful insert
4. Fix parsing of connection stage config
5. Support XML
6. Support multiple replay strategy for BigQuery as target
7. Support MySQL RDS password rotation
8. Support auto-reinit if DDL fails
9. Upgrade S3 SDK version to 2.*
10. MySQL index creation error
11. Option to add prefix to topic
12. Db2 Kafka - Kafka logging improvements
13. Option to allow dot(.) as a delimiter in Kafka topic names
14. Fix system generated split-key and delta-snapshot-key handling for HANA as source
15. Initialize ASE source fields in target JSON format log ( for Kafka and Redis Stream )
16. Transforms Not Applying When Src-schemas is used
17. Log Delta Table Verbose
18. AWS SecretsManager - error parsing secretKeyPath with /
19. Setting csv-publish-method: READ as default when column transformation is used
20. DB2-MQ-Kafka - Update on timestamp column not working
21. Enable compression-type: GZIP for MemSQL/SingleStore bulk load
22. Redis Stream : Improve logging under snapshot config log-row-level-errors
23. Fix NPE in GLOBAL replay consistency validation code
24. Fix issue where we only allow connecting to a named instance by port
25. Fix for SQL Server change tracking and re-enable CI/CD tests for SQL Server
26. Fixed SQLServer fetchObjectIds() function, using schema name to filter tables from SQLServer
1. Add Voodoo LogReader as extraction method for Oracle
2. Redis Streams as a Target
3. Databricks as target : MERGE-Based Strategy ( When we don't have full before/after image )
4. Test Snowflake Iceberg Tables
5. Improve cdc log segment cache
6. Db2 As400 Add logging for commitment control
7. AS400 Support Transaction Rollback
8. Support realtime BigQuery as target using storage Write API
9. Support reinit based DDL Replication for SAPÂ ASE Sybase Source
10. Added support for delta-snapshot both src and dst for SQL Server.
â
1. Add replication ID to the transaction path for Oracle uncommitted CDC
2. Support Writing All Data to a Single Kafka Topic (From Informix and Postgres)
3. Integration with AWS Secrets Manager (or Similar Service)
4. Write to CDC log segment in chunks
5. Add stage maintenance mode
6. NPE in Snowflake initializer
7. SQL Server incorrectly blocking system schemas
8. --stop-never-slave-server-id param in MySQL is deprecated.
9. RDS PostgreSQL to RDS Postgres - replicate_io_replication_stats_summary_coreoct11_12_coreoct11_12 does not exist
10. Handle error: ORA-02149 Specified partition does not exist
11. SQLServer - fetch-schema not fetching the database specified in connection configuration
12. Test and fix source column transformation
13. MySQL as Source in real-time replication crashes when we insert into column of type year
14. Azure SQL to YugabyteSQL - Column default does not created at target
15. Databricks S3 Role arn parameter
16. Fix for cdc recovery across all pipelines
17. Ability to Surface Alerts in New Relic
18. Enable init-metadata mode for DB2
19. Fix incorrect exception when logging failed transaction
20. DB2 AS400 tables with table name length greater than 10, realtime replication is not working
21. SQL Server heartbeat doesn't work when there are multiple databases being replicated
22. Add a table level check in Oracle for full column logging when Databricks is a target
23. Table level column logging check not working
24. Sybase ASE snapshot but on release binary 22 10 28 1
Features:
1. Separate transaction files by replication ID for SQL Server CDC
2. Oracle Native Export
3. Databricks Unity catalog support
4. Allow use of Salesforce ID or Id Mod Column as Split Key
5. Add support for a credential store for the Replicant SQL Agent
6. GCP Databricks as destination
â
Fixes:
1. Db2 to Singlestore snapshot, TIMESTAMP column is shown as TIMESTMP in fetch-schema and is converted to varchar in singlestore
2. SQL Server CDC incorrectly creates multiple cursors for schemas within a database
3. Imply does not drop existing table in reinit mode if table already present
4. Change active connection logging to include TPT connections
5. Oracle to Oracle: Real-time SCN problem.
6. Add support for a credential store for the Replicant SQL Agent
7. SQL CDC should send ACKs for each file transfer
8. Decouple exit codes from custom exceptions
9. Upgrade Databricks jdbc driver
10. GCP Databricks as destination
11. Db2 LUW/As400 Losing Records When Transaction Size > 100000
12. PostgreSQL as target doesn't close files after bulk load
â
Features:
1. SAP ASEas source for realtime/full mode support.
2. OracleDBLINK based snapshot.
3. Support for Kerberos Authentication.
4. Support BigQuery as a target.
5. AddUpsert support for SingleStore.
6. EnableEmail Notifications if Lag is ABOVE a particular threshold value.
Â
Fixes:
1. MariaDB-> Google BigQuery : BigQuery not support AUTO_INCREMENT or IDENTITY.
2. Fixextractor NPE when replicating Oracle views.
3. DB2AS400 Schema with Schema name length greater than 10, realtime replication isnot working.
4. Changing Imply Implementation for their new apis.
5. Fix issues with TiDB as a target.
6. Invalidvalue when replicating table with set datatype in realtime mode.
7. ExtendDb2 XML TRANSACTION MQ message type to Kafka-Transactional log-storage.
8. Addsupport for DB2 TRANSACTION message type in DELIMITED message format.
9. Olapduplicate row insert error.
10. Snowflake target improvements.
11. MongoDBconnection needs to be established through url or connection config properties(host, port, username and password).
12. SAP ASECDC Performance testing.
13. Snowflake table cdcStageType not working for insert.
14. Number of rows mismatch while replicating updates ondatabricks using MERGE.
15. TestTargets with ASE CDC Source.
16. fixingper table replay strategy in realtime mode.
17. Snowflake bug fixing for delete query.
18. Snapshot mode not finished when trying to replicate table with SET Type.
â
- Pipeline source - RDS
- Organization endpoint called needlessly to many times
- Unable to save Postgresql target
- Handle firebase error on create-account page
- Not able to upload connector file
- Fix value remembered on password input
- Handle 'Unauthorized email domain' firebase error
- UI not resuming or restarting as stated
- Redesign settings page
- Clean up replication status typings in the code
- Clone signup page for Google BigQuery
- Cancel /schema request in browser on cancel
- Disable Gmail (and other disposable providers) sign-in
- Changes to support RDS PostgreSQL as source
- Pipeline destination - AlloyDB
- Pipeline destination - Google CloudSQL
- Actions are taking many times longer to execute than in past releases
- Lag - time is different than individual tables
- Forced PostgreSQL replication slot
- Forward fetch-schema trace log to UI in case of Core error
- New Connector Added to UI - AlloyDB
- Add support for new extractor field - fetchIdentityInfo
- Oracle destination support for Informix
- Mysql to MinIO snapshot with mapper results in mapper parse error
- Sybase - SSL support
- Add snapshot_summary.txt report to ZIP export
- Redis Stream - SSL support
- Support Transformation configuration YAML file upload
- MySQL as destination for Oracle
- Enable PG to BigQuery pipeline
- Changes to support CloudSQL MySQL as destination
- Changes to support CloudSQL PostgreSQL as destination
- Lazy loading schemas for filter
- Pipeline source - SAP ASE (Sybase)
- Pipeline destination - Redis Streams
- Buttons now viewable when scrolling on the webpages
- Can't click save on new Databricks connection when using DBFS
- Can't perform state update on saved-connections
- Filter page timeouts
- Filter update by file not handled properly
- When setting Pertable config for Databricks Lakehouse the schema name is not set into the per table configuration (only catalog name)
- Handle exception when /replication returns 401
- Allow schema name update when mapping to 2 level db
- Update refresh token behaviour
- UI changes to support SAP ASE (Sybase) as source
- API changes to support SAP ASE (Sybase) as source
- UI changes to support Redis Streams as destination
- API changes to support Redis Streams as destination
- Optimize filter behaviour when rendering a large dataset
- Update replication list header and make columns sortable
- Ability to download all config files
- Handle upload filter containing blocked list in the UI
- Enable PG to PG and MySQL to PG pipeline
- Type 2 CDC support
- MYSQL DDL replication support
- Informix DDL replication support
- Support applier configuration for /dev/null
- Update progress percentage for CHANGE_DATA_CAPTURE phase and TERMINATE replication status
- Reimplement trace log download within replication summary
âMain feature updates:
- S3 pipeline
- OAuth2 authentication support (tested with Okta)
- In-memory authentication custom password support
- Prometheus metrics exposed + Grafana dashboard
- Oracle PDB support
Detail fixes:
- Pipeline destination - S3
- Mapper not added to replication if added after the initial creation
- Errors about duplicate keys and other errors
- Need ability to fall back to non-partition table creation
- Allow the in memory authentication type to configure a different password (Docker Secret)
- Support OpenID authentication via providers like Okta (without password grant)
- Add Prometheus metric endpoint
- Add support for DDL replication
- Expose JDBC URL for connections that support it (MySQL first and foremost)
- Do not explicitly list tables in filter if the whole catalog/schema is selected (same for table and columns)
- Add support for schema validation configuration (general.yaml)
- Enable S2->GBQ and S2->S3
- Liveness monitor, schema validation and metadata connections configuration support
- Changes to support S3 as destination
- Add ability to specify custom NTP server
- PDB support
- Configure Core logs rotation
- Cleanly exit Replicant process instead of force kill
- Allow permission validation to be enabled/disabled via UI
- Oracle to Oracle pipeline
- SQL Server CDC agent support
- Databricks Deltalake UI rework & Azure/DBFS support
- Whole SaaS stack migrated to Java 17
- Computed columns support for MySQL
- Forward test-connection trace log to UI in case of Core error
- Add Azure stage support to Databricks Delta Lake
- Missing 'restart' option for stopped replications
- Increase Replicant memory limit
- Update Sign In & Sign Up pages
- Hung on current screen (loading replication data)
- Connection details not opening
- Connection created successfully duplicated
- Replication statuses phasing in and out
- Connection test does not accurately represent what is needed for replication to succeed
- Databricks connection not working with or without stage
- Table filter search not working
- Schema not being refreshed after fetchSchema
- Update the current deadlock solution
- Fix tooltip overflow issues
- Update connection logo image files to fit to container box
- Generated columns support
- Applier BulkLoad charset support
- Increase maximum YAML schema input size
- Management service port issues on WSL2
- Snapshot applier additions - skip table failures and init indexes
- Enable APPENDING write mode in UI
- Core 23.02.01.9
- MySQL -> MySQL pipeline
- Dashboard rework with filtering by mode and virtual list support (to allow smooth scroll with huge numbers of replications)
- Redesign of the settings menu, various UI fixes
- Support for paid cluster pause/resume/terminate
- AWS password rotation support in the Docker container (both for Metadata DB and replications)
- S3 as the shared config storage for clustered mode
- In addition to the latest Docker tag we now have the test tag - which will point to the next container version with the latest core release (for PoC etc.)
- Core 23.01.05.9- Improvements to our on-premises APIs
- Configuration YAML upload fixed
- Expose information about cluster state
- Refresh connection logos with vector ones
- Support for Oracle native log reader (ASM only on Cloud, Redo Logpath additionally for on-premises)
- Audit logging support for user actions and events
- Pass license into Docker as Base64 environment variable
- Support ARM64 architecture for on-premises container image (look for tag latest-arm64 on DockerHub)
- Show days left on license for on-premises
- Support SQL Server Azure as source
- Various UI improvements and bug fixes
â
- Pipeline destination - AlloyDB
- Pipeline destination - Google CloudSQL
- Actions are taking many times longer to execute than in past releases
- Lag - time is different than individual tables
- Forced PostgreSQL replication slot
- Forward fetch-schema trace log to UI in case of Core error
- New Connector Added to UI - AlloyDB
- Add support for new extractor field - fetchIdentityInfo
- Oracle destination support for Informix
- Mysql to MinIO snapshot with mapper results in mapper parse error
- Sybase - SSL support
- Add snapshot_summary.txt report to ZIP export
- Redis Stream - SSL support
- Support Transformation configuration YAML file upload
- MySQL as destination for Oracle
- Enable PG to BigQuery pipeline
- Changes to support CloudSQL MySQL as destination
- Changes to support CloudSQL PostgreSQL as destination
- Lazy loading schemas for filter
- Pipeline source - SAP ASE (Sybase)
- Pipeline destination - Redis Streams
- Buttons now viewable when scrolling on the webpages
- Can't click save on new Databricks connection when using DBFS
- Can't perform state update on saved-connections
- Filter page timeouts
- Filter update by file not handled properly
- When setting Pertable config for Databricks Lakehouse the schema name is not set into the per table configuration (only catalog name)
- Handle exception when /replication returns 401
- Allow schema name update when mapping to 2 level db
- Update refresh token behaviour
- UI changes to support SAP ASE (Sybase) as source
- API changes to support SAP ASE (Sybase) as source
- UI changes to support Redis Streams as destination
- API changes to support Redis Streams as destination
- Optimize filter behaviour when rendering a large dataset
- Update replication list header and make columns sortable
- Ability to download all config files
- Handle upload filter containing blocked list in the UI
- Enable PG to PG and MySQL to PG pipeline
- Type 2 CDC support
- MYSQL DDL replication support
- Informix DDL replication support
- Support applier configuration for /dev/null
- Update progress percentage for CHANGE_DATA_CAPTURE phase and TERMINATE replication status
- Reimplement trace log download within replication summary
Main feature updates:
- BigQuery pipeline
- Kafka pipeline
- Confluent pipeline- /dev/null destination support
- Databricks Lakehouse UI rework & Azure/S3 support
Detail fixes:
- Pipeline destination - BigQuery
- Pipeline destination - Kafka
- Pipeline destination - Confluent
- Support /dev/null destination for extractor performance testing
- Allow download of full Replicant trace log
- Allow disabling IPv6 in container
- Cannot save YugabyteSQL applier
- Disable role creation as default for all replication
- Configuration edit not working properly
- Missing restart for finished replications
- Oracle to Big Query Fails due to connection issue
- Google Big Query Applier - Replay Strategy Not Being Recorded
- Query reply-strategy not being printed
- PostgreSQL default port listed as hostname
- Oracle Applier unable to save
- Oracle roles still being fetched even when unchecked in extractor
- Cloud wording on front screen
- Catalog name undefined in the per table target configuration
- Pre-populate default port for all connectors
- Add IN_MEMORY_MERGE applier support
- Add ability to skip row counts to extractor
- Update connection with unsaved changes on test
- Databricks applier - save-file-on-error - default to false
- Add maxFilesPerBulkLoad to applier
- Add Azure and S3 support to Lakehouse
- Realtime replication mode support + MySQL cursor support
- Connector list - alphabetical sort
- MySQL extractor - expose idleBinLogTimeoutSeconds in realtime
- Oracle to Oracle pipeline
- SQL Server CDC agent support
- Databricks Deltalake UI rework & Azure/DBFS support
- Whole SaaS stack migrated to Java 17
- Computed columns support for MySQL
- Forward test-connection trace log to UI in case of Core error
- Add Azure stage support to Databricks Delta Lake
- Missing ârestartâ option for stopped replications
- Increase Replicant memory limit
- Update Sign In & Sign Up pages
- Hung on current screen (loading replication data)
- Connection details not opening
- Connection created successfully duplicated
- Replication statuses phasing in and out
- Connection test does not accurately represent what is needed for replication to succeed
- Databricks connection not working with or without stage
- Table filter search not working
- Schema not being refreshed after fetchSchema
- Update the current deadlock solution
- Fix tooltip overflow issues
- Update connection logo image files to fit to container box
- Generated columns support
- Applier BulkLoad charset support
- Increase maximum YAML schema input size
- Management service port issues on WSL2
- Snapshot applier additions - skip table failures and init indexes
- Enable APPENDING write mode in UI
- Core 23.02.01.9
- MySQL -> MySQL pipeline
- Dashboard rework with filtering by mode and virtual list support (to allow smooth scroll with huge numbers of replications)
- Redesign of the settings menu, various UI fixes
- Support for paid cluster pause/resume/terminate
- AWS password rotation support in the Docker container (both for Metadata DB and replications)
- S3 as the shared config storage for clustered mode
- In addition to the latest Docker tag we now have the test tag - which will point to the next container version with the latest core release (for PoC etc.)
- Core 23.01.05.9- Improvements to our on-premises APIs
- Configuration YAML upload fixed
- Expose information about cluster state
- Refresh connection logos with vector ones
- Support for Oracle native log reader (ASM only on Cloud, Redo Logpath additionally for on-premises)
- Audit logging support for user actions and events
- Pass license into Docker as Base64 environment variable
- Support ARM64 architecture for on-premises container image (look for tag latest-arm64 on DockerHub)
- Show days left on license for on-premises
- Support SQL Server Azure as source
- Various UI improvements and bug fixes
Deploy the only cloud-native data replication platform youâll ever need. Get real-time, high-performance data pipelines today.
Start 14-day free trial*
8 sources & 6 targets
Pre-configured enterprise instance
Available in four US AWS regions
*No payment info required
20+ enterprise source and target connectors
Deploy on-prem or VPC
Satisfy security requirements
Fully managed, in the cloud.
