Jobs run after an offset time

question

#1

I’m trying to look for a way to run jobs after a configured interval of the job enqueue time (plus or minus a minute is fine) and having trouble. Think of ActiveJob wait option.

one thought was to include a timestamp in the job payload (or use some existing data) and re-enqueue the job until the interval has passed, but I’m concerned this will create an infinite loops when processing jobs (on the minute interval or whatever).

Did anyone attempt this successfully or have pointers?


#2

One idea is to create a one-time Scheduled Event that runs and self-deletes itself afterward. Note, there are limits with CloudWatch Scheduled Events at 100 by default. It’s a soft limit that can be increased. Unsure what the hard limit is.

Wondering about the ActiveJob wait interface:

HardJob.set(wait: 1.week).perform_now(:dig, event)

Wondering if this interface is a bit more straightforward.

HardJob.perform_later(:dig, event, at: 1.week.from_now)

Anyway, Jets does not support this yet. But that’s an idea for a possible approach to take.


#3

Thanks for the feedback! I’m going to chew on this, and if I have some success with this (or another approach) I’ll followup here so others can benefit.


#4

One alternate idea I just had would be to make a Jets job that runs on a 1 minute interval, but instead of directly enqueueing the job, write to DynamoDB, and have an indexed timestamp column, which the job consults (kind of how Sidekiq uses a sorted set to store a scheduled job)


#5

Feel like it’s more moving pieces and also cost more. That also works though :+1:


#6

Aha, I think I found a better way to try after returning to the thought. SQS queues have a DelaySeconds property, which can be up to 15 minutes. I think I can use that in this instance. So excited to try.

If you’re wondering about my use case, I’m detecting events in a billing system which, I need a little grace period for, because some reconciliations are multiple steps and take a couple minutes for an operator, so I want the job to hold off on detection until a little grace window has passed.