Integration: Add GitLab and Bitbucket source types to TaskSpawner for multi-platform git hosting support

🤖 Kelos Strategist Agent @gjkim42

Summary

Propose first-class gitlabIssues, gitlabMergeRequests, and bitbucketPullRequests source types in the TaskSpawner when field. Kelos today is entirely GitHub-centric for source discovery: all six source types are either GitHub-specific (githubIssues, githubPullRequests, githubWebhook) or platform-specific integrations that still assume GitHub for the workspace (jira, linearWebhook, cron). This means teams using GitLab or Bitbucket as their primary git hosting platform cannot use Kelos's core value proposition — autonomous agents that discover work items and open code changes — without building custom polling scripts and managing their own deduplication.

Adding native support for GitLab and Bitbucket would roughly double Kelos's addressable market while leveraging the platform-agnostic Source interface and WorkItem abstraction that the codebase already provides.

Motivation

1. The Source abstraction is already platform-agnostic — only the implementations are GitHub-bound

The Source interface (internal/source/source.go:36-39) returns []WorkItem, and the WorkItem struct (source.go:9-34) has no GitHub-specific fields. The TaskBuilder (internal/taskbuilder/builder.go) renders prompts and creates Tasks using source.WorkItemToTemplateVars() without any platform coupling. The spawner cycle in cmd/kelos-spawner/main.go is also generic — the only platform-specific code path is buildSourceWithProxy() which selects the source implementation.

This means the architectural investment to support new platforms is concentrated in three areas: (a) implementing Source.Discover(), (b) extending the When CRD types, and (c) adapting auth handling — not a full-stack rewrite.

2. GitLab and Bitbucket represent a large, underserved audience

GitLab has 30M+ registered users and is the primary platform for many enterprises, government agencies, and organizations that require self-hosted git (GitLab CE/EE). Bitbucket is deeply integrated with the Atlassian ecosystem — and since Kelos already has first-class Jira support, Bitbucket is the natural complement: teams already using Jira + Bitbucket could adopt Kelos with minimal friction.

3. httpPoll (#894) is not a substitute for first-class sources

The proposed generic httpPoll source provides an escape hatch for arbitrary REST APIs but cannot replicate the rich filtering that first-class sources provide: label-based filtering, assignee/author filters, state management, comment policies, priority sorting, and status reporting. Users would need to encode all of this logic in JMESPath expressions or custom scripts, defeating Kelos's declarative advantage.

4. Credential handling is the only deep coupling, and it follows an extensible pattern

The job builder (internal/controller/job_builder.go:318-331) hardcodes GITHUB_TOKEN, GH_TOKEN, and GH_ENTERPRISE_TOKEN env vars, and the git credential helper (job_builder.go:400) uses GitHub's x-access-token username. However, the Jira source already demonstrates the pattern for non-GitHub auth: dedicated secret keys, conditional env var injection in the deployment builder (taskspawner_deployment_builder.go:198-237). GitLab and Bitbucket tokens would follow the same pattern.

Use Cases

1. GitLab issue-driven development

A team using self-hosted GitLab CE labels issues with agent-ready for tasks that agents can handle autonomously. A TaskSpawner discovers these issues, spawns agents that work on a branch, and push merge requests — the same workflow Kelos provides for GitHub, but for the 30M+ GitLab user base.

apiVersion: kelos.dev/v1alpha1
kind: TaskSpawner
metadata:
  name: gitlab-issue-worker
spec:
  when:
    gitlabIssues:
      baseUrl: "https://gitlab.example.com"
      project: "mygroup/myproject"   # GitLab project path
      labels: ["agent-ready"]
      state: opened
      pollInterval: 5m
      secretRef:
        name: gitlab-token           # Secret with GITLAB_TOKEN key
  maxConcurrency: 3
  taskTemplate:
    type: claude-code
    workspaceRef:
      name: my-gitlab-repo
    credentials:
      type: api-key
      secretRef:
        name: anthropic-key
    branch: "kelos-task-{{.Number}}"
    promptTemplate: |
      Fix GitLab issue #{{.Number}}: {{.Title}}

      {{.Body}}

2. GitLab merge request review

A team wants agents to automatically review merge requests targeting their main branch. The spawner discovers MRs with the needs-review label and spawns review agents.

apiVersion: kelos.dev/v1alpha1
kind: TaskSpawner
metadata:
  name: gitlab-mr-reviewer
spec:
  when:
    gitlabMergeRequests:
      baseUrl: "https://gitlab.example.com"
      project: "mygroup/myproject"
      labels: ["needs-review"]
      state: opened
      targetBranch: "main"          # GitLab-specific: filter by target branch
      pollInterval: 3m
      secretRef:
        name: gitlab-token
  taskTemplate:
    type: claude-code
    workspaceRef:
      name: my-gitlab-repo
    branch: "{{.Branch}}"
    promptTemplate: |
      Review merge request !{{.Number}}: {{.Title}}

      {{.Body}}

      Review the changes on this branch and post review comments.

3. Bitbucket + Jira unified workflow

A team already uses Kelos with Jira for issue discovery. Adding Bitbucket PR support lets them close the loop: Jira issue → agent creates branch + PR on Bitbucket → another spawner reviews the Bitbucket PR.

apiVersion: kelos.dev/v1alpha1
kind: TaskSpawner
metadata:
  name: bitbucket-pr-reviewer
spec:
  when:
    bitbucketPullRequests:
      workspace: "myteam"           # Bitbucket workspace
      repoSlug: "myproject"
      state: "OPEN"
      pollInterval: 5m
      secretRef:
        name: bitbucket-token       # Secret with BITBUCKET_TOKEN (app password)
  taskTemplate:
    type: claude-code
    workspaceRef:
      name: my-bitbucket-repo
    branch: "{{.Branch}}"
    promptTemplate: |
      Review Bitbucket PR #{{.Number}}: {{.Title}}
      {{.Body}}

Proposal

1. New CRD types in api/v1alpha1/taskspawner_types.go

Extend the When struct with three new fields:

type When struct {
    // ... existing fields ...

    // GitLabIssues discovers issues from a GitLab project.
    // +optional
    GitLabIssues *GitLabIssues `json:"gitlabIssues,omitempty"`

    // GitLabMergeRequests discovers merge requests from a GitLab project.
    // +optional
    GitLabMergeRequests *GitLabMergeRequests `json:"gitlabMergeRequests,omitempty"`

    // BitbucketPullRequests discovers pull requests from a Bitbucket repository.
    // +optional
    BitbucketPullRequests *BitbucketPullRequests `json:"bitbucketPullRequests,omitempty"`
}

2. GitLab type definitions

// GitLabIssues discovers issues from a GitLab project via the GitLab REST API.
type GitLabIssues struct {
    // BaseURL is the GitLab instance URL (e.g., "https://gitlab.com" or
    // "https://gitlab.example.com" for self-managed).
    // +kubebuilder:validation:Required
    BaseURL string `json:"baseUrl"`

    // Project is the GitLab project path (e.g., "mygroup/myproject").
    // +kubebuilder:validation:Required
    Project string `json:"project"`

    // Labels filters issues to those with ALL of the specified labels.
    // +optional
    Labels []string `json:"labels,omitempty"`

    // ExcludeLabels skips issues that have any of the specified labels.
    // +optional
    ExcludeLabels []string `json:"excludeLabels,omitempty"`

    // State filters by issue state: "opened", "closed", or "all".
    // +kubebuilder:default="opened"
    // +optional
    State string `json:"state,omitempty"`

    // Assignee filters issues assigned to this username.
    // +optional
    Assignee string `json:"assignee,omitempty"`

    // Author filters issues created by this username.
    // +optional
    Author string `json:"author,omitempty"`

    // SecretRef references a Secret containing a GITLAB_TOKEN key
    // with a GitLab personal or project access token.
    // +kubebuilder:validation:Required
    SecretRef SecretReference `json:"secretRef"`

    // PollInterval overrides the default polling interval for this source.
    // +optional
    PollInterval *metav1.Duration `json:"pollInterval,omitempty"`
}

// GitLabMergeRequests discovers merge requests from a GitLab project.
type GitLabMergeRequests struct {
    // BaseURL is the GitLab instance URL.
    // +kubebuilder:validation:Required
    BaseURL string `json:"baseUrl"`

    // Project is the GitLab project path.
    // +kubebuilder:validation:Required
    Project string `json:"project"`

    // Labels filters MRs to those with ALL of the specified labels.
    // +optional
    Labels []string `json:"labels,omitempty"`

    // State filters by MR state: "opened", "closed", "merged", or "all".
    // +kubebuilder:default="opened"
    // +optional
    State string `json:"state,omitempty"`

    // TargetBranch filters MRs targeting this branch.
    // +optional
    TargetBranch string `json:"targetBranch,omitempty"`

    // SecretRef references a Secret containing a GITLAB_TOKEN key.
    // +kubebuilder:validation:Required
    SecretRef SecretReference `json:"secretRef"`

    // PollInterval overrides the default polling interval.
    // +optional
    PollInterval *metav1.Duration `json:"pollInterval,omitempty"`
}

3. Bitbucket type definition

// BitbucketPullRequests discovers pull requests from a Bitbucket Cloud repository.
type BitbucketPullRequests struct {
    // Workspace is the Bitbucket workspace slug.
    // +kubebuilder:validation:Required
    Workspace string `json:"workspace"`

    // RepoSlug is the Bitbucket repository slug.
    // +kubebuilder:validation:Required
    RepoSlug string `json:"repoSlug"`

    // State filters by PR state: "OPEN", "MERGED", "DECLINED", or "SUPERSEDED".
    // +kubebuilder:default="OPEN"
    // +optional
    State string `json:"state,omitempty"`

    // SecretRef references a Secret containing a BITBUCKET_TOKEN key
    // (app password or repository/workspace access token).
    // +kubebuilder:validation:Required
    SecretRef SecretReference `json:"secretRef"`

    // PollInterval overrides the default polling interval.
    // +optional
    PollInterval *metav1.Duration `json:"pollInterval,omitempty"`
}

4. Source implementations

Create internal/source/gitlab.go and internal/source/bitbucket.go implementing the Source interface. Each calls its platform's REST API:

• GitLab: GET /api/v4/projects/:id/issues?labels=...&state=... and GET /api/v4/projects/:id/merge_requests?labels=...&state=...
• Bitbucket: GET /2.0/repositories/:workspace/:repo_slug/pullrequests?state=...

Map responses to WorkItem:

WorkItem field	GitLab Issue	GitLab MR	Bitbucket PR
ID	iid (string)	iid (string)	id (string)
Number	iid	iid	id
Title	title	title	title
Body	description	description	description
URL	web_url	web_url	links.html.href
Labels	labels[]	labels[]	N/A (Bitbucket has no native labels)
Kind	"Issue"	"MR"	"PR"
Branch	N/A	source_branch	source.branch.name

5. Auth and credential injection changes

Extend the Workspace auth handling in internal/controller/job_builder.go to detect GitLab/Bitbucket secrets and inject the correct credential helper:

• GitLab: git config credential.helper '!f() { echo "username=oauth2"; echo "password=$GITLAB_TOKEN"; }; f' (GitLab uses oauth2 as the username for token auth)
• Bitbucket: git config credential.helper '!f() { echo "username=x-token-auth"; echo "password=$BITBUCKET_TOKEN"; }; f'

This follows the existing pattern at job_builder.go:400 where GitHub uses x-access-token.

6. GHProxy consideration

The current ghproxy (cmd/ghproxy/) is GitHub-specific. For the initial implementation, GitLab and Bitbucket sources would call APIs directly without a caching proxy. A gitlabproxy or generic API proxy could be added later if rate limiting becomes an issue. GitLab's rate limits (2000 req/min for authenticated users) are significantly more generous than GitHub's, making this less urgent.

Implementation phases

Phase 1 — GitLab issues (smallest useful increment):

• Add GitLabIssues CRD type and gitlabIssues When field
• Implement GitLabSource with Discover() using GitLab REST API v4
• Add source instantiation in spawner buildSourceWithProxy()
• Add GITLAB_TOKEN env var injection in deployment builder
• Add git credential helper for GitLab URLs in job builder
• Unit tests for source discovery and WorkItem mapping

Phase 2 — GitLab merge requests:

• Add GitLabMergeRequests CRD type
• Implement GitLabMergeRequestSource with branch and review state mapping
• Add optional reporting (post comments back to MRs)

Phase 3 — Bitbucket pull requests:

• Add BitbucketPullRequests CRD type
• Implement BitbucketSource using Bitbucket Cloud REST API v2
• Handle Bitbucket's OAuth2/app password auth model
• Consider Bitbucket Server (self-hosted) as a follow-up

Alternatives considered

Use httpPoll (#894) instead of first-class sources: While httpPoll provides a generic escape hatch, it cannot replicate label filtering, state management, deduplication by issue ID, comment policies, or status reporting without encoding all of this in JMESPath expressions. First-class sources provide a significantly better user experience for the most common platforms.

Wait for webhook support: GitLab and Bitbucket both support outbound webhooks. However, the existing webhook infrastructure (kelos-webhook-server) is GitHub/Linear-specific, and many self-hosted GitLab instances cannot reach external webhook endpoints due to network restrictions. Polling is the more universally applicable approach.

/kind feature

Start with internal/source/source.go, api/v1alpha1/taskspawner_types.go, and the source selection in cmd/kelos-spawner/main.go. Trace buildSourceWithProxy and the credential handling in internal/controller/job_builder.go and taskspawner_deployment_builder.go. Done means the new CRD fields, GitLab and Bitbucket Source implementations, WorkItem mappings, polling, and platform credentials work consistently with existing sources.