[Feature Request] Declarative schedules: upsert, sync, terraform, etc.

Is your feature request related to a problem? Please describe.

Currently, have to run tctl or a client sdk script to start and update Schedules. Is suboptimal for maintainability (compared to eg terraform/declarative infra).

Describe the solution you'd like

Declare what schedule a workflow should be run at, in code or markup file, and automatically upsert it.

From @dnr:

As a user, the main "higher level" thing I want in a schedule api is declarative schedules: I shouldn't have to use an imperative api to work with something that should just "be there". I want to annotate a workflow with a schedule and have it "take effect" when I deploy the worker.

This needs some design work and maybe we should do it in a meeting, but a few things to consider to start:

• The schedule that starts a workflow could be specified as an annotation on that one, or a separate object, or both options.
• The schedule id could be implicit or explicit.
• Deploying a worker with a changed schedule should be able to change the calendar spec and other stuff, but it should not reset the paused state. We'll have to think field by field what makes sense.
• Having two versions of workers deployed at once should not flip the schedule back and forth, the newer one should take priority.

Discussion items

Code location

Depending on the lang, could be in the same file as workflow. Could be next to that file, like myWorkflow.ts and myWorkflow.schedule.ts.

How it gets run

• On Worker startup:
  • Pro: easier to set up, better initial DX
  • Con: issues w/ coupling to worker startup
• By running a script in prod CI
  • tctl reading config files (wait this is temporal! we don't use those here)
  • terraform
  • script provided by SDK, or sample script with helpers provided by SDK

Provided script could be update-schedules ./workflows/*.schedule.ts, and those files could include something like:

import { myWorkflow } from './myWorkflow'

export function upsertSchedule() {
  return ({ 
    id: 'myWorkflow-schedule', 
    spec,   
    action: {
      workflowId: 'myWorkflow-schedule-workflow',
      type: myWorkflow,
      args: ['arg1', 'arg2'],
    }, 
  })
}

Start by reviewing the design questions in this issue and the proposed myWorkflow.ts and myWorkflow.schedule.ts entry points. Define the declarative schedule model, deployment or startup flow, and precedence and paused-state behavior before implementation. Done means the approach is agreed and the resulting API or tooling can upsert schedules without conflicting worker versions.