Loading...
Loading...
Archive Destroy Trigger jobs end up not finishing or finishing in a long time, with multiple jobs getting started and degrading the system. The job may terminate after 4 hours because of the query connection being closed. Additionally, "Destroy Related Records" does not remove the related records entries from the sys_archive_log, it will only remove the parent record from the log. Steps to Reproduce 1- Create a rule to archive 300M records. 2- After achieving all the records, create a rule to destroy the records. 3- Check the log of the destroy rule on the instance to verify nothing is actually getting deleted. The localhost and job thread logs will be tracing the memory issues: glide.memory.watcher SYSTEM URL= JOB: Archive Destroyer scheduled on node - f5, THREAD= glide.scheduler.worker.6, FG= false, TYPE= 4, STATE= 11, USER= system, TIME= 13,659,598, MEM= 0, ATTRIBUTES= {} worker.6 worker.6 txid=77bf93961ba9 WARNING *** WARNING *** Large Table: Table handling an extremely large result set: 384600376 Query that got us here is: TABLENAME = ar_cmdb_sam_sw_install ENCODED_QUERY = com.glide.db.QueryWarning.warn(QueryWarning.java:44) com.glide.db.DBQuery.loadResultSet(DBQuery.java:3272) com.glide.db.DBQuery.executeAndReturnTable(DBQuery.java:346) com.glide.db.DBAction.executeNormal(DBAction.java:256) com.glide.db.DBAction.executeAndReturnException(DBAction.java:210) com.glide.db.RDBMSQueryContext.executeQuery(RDBMSQueryContext.java:42) com.glide.db.DBQuery.execute(DBQuery.java:2707) com.glide.db.meta.Table.queryBasic(Table.java:339) com.glide.db.meta.Table.query(Table.java:207) com.glide.script.GlideRecordITable.query(GlideRecordITable.java:108) com.glide.script.GlideRecord.query0(GlideRecord.java:3332) com.glide.script.GlideRecord.query(GlideRecord.java:3053) com.glide.db.auxiliary.job.DestroyJob.queryDestructionTargets(DestroyJob.java:82) com.glide.db.auxiliary.job.DestroyJob.archiveDestroyBatch(DestroyJob.java:48) com.glide.db.auxiliary.job.DestroyJob.archiveDestroyRule(DestroyJob.java:38) com.glide.db.auxiliary.job.DestroyJob.execute(DestroyJob.java:28) com.glide.schedule.JobExecutor.lambda$executeJob$0(JobExecutor.java:113) com.glide.schedule.JobExecutor.executeJob(JobExecutor.java:116) com.glide.schedule.JobExecutor.execute(JobExecutor.java:100) com.glide.schedule_v2.SchedulerWorkerThread.executeJob(SchedulerWorkerThread.java:300) com.glide.schedule_v2.SchedulerWorkerThread.lambda$process$0(SchedulerWorkerThread.java:188) com.glide.worker.TransactionalWorkerThread.executeInTransaction(TransactionalWorkerThread.java:35) com.glide.schedule_v2.SchedulerWorkerThread.process(SchedulerWorkerThread.java:188) com.glide.schedule_v2.SchedulerWorkerThread.run(SchedulerWorkerThread.java:102) java.util.zip.Deflater.deflateBytes(Native Method) java.util.zip.Deflater.deflate(Deflater.java:444) java.util.zip.Deflater.deflate(Deflater.java:366) java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:251) java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:211) java.util.zip.GZIPOutputStream.write(GZIPOutputStream.java:145) java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) java.io.BufferedOutputStream.write(BufferedOutputStream.java:126) java.io.FilterOutputStream.write(FilterOutputStream.java:97) com.glide.db.meta.RowSerializer.serializeCompact(RowSerializer.java:68) com.glide.db.meta.RowSerializer.serializer(RowSerializer.java:50) com.glide.db.meta.RowBlock.compact(RowBlock.java:312) com.glide.db.meta.RowList.updateActiveBlock(RowList.java:394) com.glide.db.meta.RowList.getWriteBlock(RowList.java:331) com.glide.db.meta.RowList.add(RowList.java:178) com.glide.db.meta.Table.loadFromResultSetOrThrow(Table.java:604) com.glide.db.DBQuery.loadResultSet(DBQuery.java:3257) com.glide.db.DBQuery.executeAndReturnTable(DBQuery.java:346) com.glide.db.DBAction.executeNormal(DBAction.java:256) com.glide.db.DBAction.executeAndReturnException(DBAction.java:210) com.glide.db.RDBMSQueryContext.executeQuery(RDBMSQueryContext.java:42) com.glide.db.DBQuery.execute(DBQuery.java:2707) com.glide.db.meta.Table.queryBasic(Table.java:339) com.glide.db.meta.Table.query(Table.java:207) com.glide.script.GlideRecordITable.query(GlideRecordITable.java:108) com.glide.script.GlideRecord.query0(GlideRecord.java:3332) com.glide.script.GlideRecord.query(GlideRecord.java:3053) com.glide.db.auxiliary.job.DestroyJob.queryDestructionTargets(DestroyJob.java:82) com.glide.db.auxiliary.job.DestroyJob.archiveDestroyBatch(DestroyJob.java:48) com.glide.db.auxiliary.job.DestroyJob.archiveDestroyRule(DestroyJob.java:38) com.glide.db.auxiliary.job.DestroyJob.execute(DestroyJob.java:28) com.glide.schedule.JobExecutor.lambda$executeJob$0(JobExecutor.java:113) com.glide.schedule.JobExecutor$$Lambda$302/24351503.call(Unknown Source) com.glide.schedule.JobExecutor.executeJob(JobExecutor.java:116) com.glide.schedule.JobExecutor.execute(JobExecutor.java:100) com.glide.schedule_v2.SchedulerWorkerThread.executeJob(SchedulerWorkerThread.java:300) com.glide.schedule_v2.SchedulerWorkerThread.lambda$process$0(SchedulerWorkerThread.java:188) com.glide.schedule_v2.SchedulerWorkerThread$$Lambda$282/11999670.run(Unknown Source) com.glide.worker.TransactionalWorkerThread.executeInTransaction(TransactionalWorkerThread.java:35) com.glide.schedule_v2.SchedulerWorkerThread.process(SchedulerWorkerThread.java:188) com.glide.schedule_v2.SchedulerWorkerThread.run(SchedulerWorkerThread.java:102)
This problem has been fixed. If you are able to upgrade, review the Fixed In section to determine the latest version with a permanent fix your instance can be upgraded to. The temporary workaround would be to disable delete related records for the destroy rules.
PRB1479861
Click on a version to see all relevant bugs
ServiceNow Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.