Event System Overhaul & Cleanup

[jahjedtieson]

DB Update Required

ALTER TABLE JobRun ADD COLUMN Step TEXT DEFAULT NULL;

This submission replaces hard-coded Workflow and Job events with more generic events, implemented in the event system itself (instead of in the job and worklow systems). We also lay the ground work for Job events; consumers of these events can take action as event status changes, such as updating a real-time display of Cook job status in the Packrat UI.

Events:

• Simplified IEventData, removing Key generic parameter
• Renamed eAuditType.eSceneQCd to eAuditType.ePubSceneQCd, to match naming conventions elsewhere
• Implemented EventConsumerJob and EventConsumerWF
• Added eJob and eWF eEventTopic's
• Added eJob* and eWF* eEventKey's
• Modified IEventEngine, replacing createConsumer() with registerConsumer() and unregisterConsumer()
• Modified EventEngine, making it clear that we create a set of system consumers, one per event topic, and then register those via registerConsumer()
• Create a default event producer; expose it's send() method as a method of IEventEngine; this allows event publication without creation of an EventProducer.
• Add eWFGenerateSceneDownloads event key
• Remove IEventConsumer.poll, which was unused, and unnessarily complex
• Avoid attempting to create a system consumer of Job and Workflow events -- there is no system consumer for these!
• Declare EventEngine.receive in EventEngineBase abstract class

Jobs:

• Record Cook "step" during polling. When it changes from our previously recorded "Step", send an eJobUpdated event
• Replace JobPackrat.updateEngines() with new sendEvent(), using events to inform the WorkflowEngine and JobEngine of job status changes
• Replaced IJobEngine.jobCompleted by making the JobEngine an EventConsumer for Job events. Use terminating statuses (done, error, cancelled) to clean up the job map
• Declare JobEngine.sendJobEvent in JobEngineBase abstract class

Workflows:

• Replace IWorkflowEngine.jobUpdated(), .event(), and .generateSceneDownloads() with actions initiated by job events.
• Transform WorkflowEngine into a Job and Workflow Event consumer

Cache & Data:

• Removed vocabulary set and vocabulary for eWorkflowEvent

Collections:

• Use events to initiate scene download generation, instead of direct calls to the workflow engine
• Move EdanCollection.computeLicenseInfo to PublishScene.computeLicenseInfo

GraphQL:

• Use events to send the workflow ingest object notification

Storage:

• Use events to send the workflow ingest object notification from AssetStorageAdapter
• Split part of OCFLObject into OCFLObjectBase, allowing OCFLInventory to access those parts of OCFLObject when OCFLObject needs OCFLInventory ... all without a circular dependency. Yay!

DBAPI:

• Add "Step" field to JobRun, indicating the current step of a running job as received from Cook
• Extract from ObjectGraphDatabase those methods needed by ObjectGraph, in order to avoid a circular dependency between ObjectGraph and ObjectGraphDatabase

HTTP:

• Move Downloader.httpRoute to RouteBuilder.httpRoute
• Import SitemapGenerator's real definition in download.ts

Test:

• Cache test does not need to use or test DBConnection.disconnect()!
• Address new compilation issues (?)
• For now, disable jest globalTeardown's call to prisma.$disconnect().

Build:

• Used npx madge --circular --extensions ts ./ to identfy circular dependencies in our code
• Eliminated these through a combination of:

• Using interfaces, where possible
• Relocating code to avoid need for imports
• Extracting selected declarations into abstract base classes, defined in the current, existing concrete classes