New Beginnings

I’m a new engineer on the DigitalOcean App Platform team. I’ve spent the last 20 years writing C# code full-time. I’ve made the switch to Go and to be perfectly honest, I miss writing C#. You know that itch you have for doing something you love so much? Well, I’ve got to scratch that .NET itch somehow while learning Golang and the amazingly capable App Platform product.

I figured I’d kill two birds with one stone and build a complex distributed system on top of App Platform.

This series is going to document my journey of building a highly-available Task Scheduling system hosted on App Platform.

I’ve selected a very fancy name for the project: Web Scheduler. It’s 100% open source, and can be found in the repository at GitHub.

Scope

The Web Scheduler will be a highly-available, distributed, task-scheduling system allowing users to schedule tasks, one time, or recurring, that will use a variety of delivery mechanisms to send a signal and trigger something in a remote application.

Scheduling Options

  • One-Time: A task that will be executed once, at a specific time.
  • Recurring: A task that will be executed at a specific interval (think crontab capabilities).
    • Limit the recurrences to a specific number of times or run indefinitely.

Delivery Mechanisms

I’ve listed a few below that were top of mind.

Delivery Guarantees

The delivery of a message is guaranteed to be received by the destination, at least once. This will help keep complexity of the system low versus at most once delivery.

As for timeliness of the delivery, there are many factors involved, such as:

  • How busy the system is delivering other tasks across all users
  • Per-User Rate Limits
  • System-Level Rate Limits

Sub-second delivery guarantees would be great 😄