Stray ami_ingest_ado_set_SETNUMBER entries in the queue DB

[DiegoPino]

What?

Looking at space usage i noticed that under certain circumstances, when an AMI is sent to process live but it does not process completely, and then it is sent a second time to the background, the QUEUE database of drupal will keep the oldie/data around and not clear it, even if we are calling
$queue_factory->get($queue_name)->deleteQueue(); in the batch finish process. Sometimes, the batch finish will not actually run (timeouts/failures on the client/etc)

Ideas:

• Have a cron cleaner for these queues.
• Have any "process" form submit actually delete the queue upfront *or check if it exists * before enqueueing again.

These tables can grow HUGE. So I will also write an update hook for the next release and document in slack how to clean them manually in the meantime

[DiegoPino]

Interesting. INNODB tables never actually shrink! So the queue table space keeps being used even after deleting all the entries.

I had to log into MYSQL, and use "optimize table queue" to get the desired effect. This is undocumented in Drupal and something we need to talk about (as a /from time to time practice). I just saved 150 GBytes!