Azure-Samples / Azure-Samples/hve-workshop-railpen
Provide a command line tool for uploading scheme contribution files
- Dominant language
- No language data
- Stars
- 0
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
## Summary
Build a command line tool that uploads a scheme contributions file to Azure Blob Storage, so the monthly upload stops being a manual portal task.
Nothing exists yet. This is a new tool, and the repository is a skeleton.
## Context
Each month the operations team receives a contributions file from the scheme administrator and has to get it into blob storage. Today that happens by hand.
| Step | How it works today |
|------|--------------------|
| Locate the file | Someone finds it in a shared folder |
| Sign in | Azure Portal, interactive |
| Navigate | Storage account, then the correct container |
| Upload | Drag and drop |
| Record it | A note in a spreadsheet, when someone remembers |
None of that is repeatable, reviewable, or schedulable. There is no record of who uploaded what, and no way to run it unattended.
## Goal
One command that a person can run today and a scheduler can run later, with an outcome that is unambiguous to both.
## User story
As a scheme operations engineer,
I want to upload a contributions file with a single command,
so that the monthly upload is repeatable, scriptable, and leaves a record.
## Scope of the first release
- Accept a local file path and upload it to a named container
- Read the destination from configuration rather than hardcoded values
- Report the outcome so a human can read it and a script can branch on it
- Run against the local storage emulator, so no Azure subscription is needed to build or test
## Out of scope
Deliberately excluded so scope creep is visible:
- Retry and backoff policy
- Chunked or block upload for large files
- SAS token generation
- Server side encryption configuration
- Container lifecycle or retention policies
- Real Azure authentication and RBAC
- Any user interface beyond the command line
## Acceptance criteria
- [ ] A single command uploads a named local file to a named container
- [ ] The destination comes from configuration, not from a literal in the source
- [ ] Success reports the blob name, the container, and the byte count
- [ ] Failure explains what went wrong and what to do about it, rather than printing a stack trace
- [ ] The process exit code distinguishes success from failure
- [ ] No connection string, account key, or SAS query string appears in any output or log line
- [ ] The tool runs end to end against the local emulator with no cloud resources
## Design decisions still open
These are genuinely undecided. They are the agenda for refinement, not oversights.
1. **What happens when a blob of that name already exists?** Refuse, replace, or keep both by versioning the name. This is the sharp one: it changes the command surface, the operational runbook, and what "success" means. It cannot be deferred to implementation, because the first upload path has to do something.
2. **Who is the first consumer, a person or a scheduler?** A human at a terminal tolerates prompts and prose. An unattended job needs machine readable exit codes and no interactivity. The answer changes the error handling design.
3. **How is the destination configured per environment, and who owns that configuration?** Local emulator is settled. Anything beyond that is not.
## Impact and risk
| Risk | Why it matters |
|------|----------------|
| Decision 1 is deferred | The team ships an accidental default rather than a chosen one, and the runbook is written against behaviour nobody agreed |
| Credential handling is treated as a later concern | The shape chosen now determines whether real credentials can ever be introduced safely |
| The emulator hides a real difference | Behaviour verified locally may not hold against a real account, so the boundary must be explicit |
## Security considerations
Posed as questions for the review session rather than pre-answered:
| Lens | Question |
|------|----------|
| Credential handling | Where does the connection string come from, and can it ever reach a log or an error message? |
| Transport | The emulator is HTTP. What has to change for a real account? |
| Container access | Is public access asserted anywhere, or left to the account default? |
| Path handling | The file path is user input. What about traversal and symlinks? |
| Resource limits | File size is unbounded. What stops a 40 GB upload? |
| Least privilege | Does the caller need account level rights, or container scoped? |
## Testing
TBD at 3 Amigos.
## Requirements traceability
FR-01 upload a file to a container, FR-02 configurable destination, NFR-03 no credential material in output. See `docs/requirements/prd.md`.
## Tool implementation
The repository is a skeleton. This is the shape the first release builds toward.
### Solution layout
```text
SchemeFileUploader.sln
Directory.Build.props pins net8.0, Nullable, TreatWarningsAsErrors
src/
SchemeFileUploader/ library
IBlobUploader.cs the seam over the storage SDK
AzureBlobUploader.cs the real implementation
UploadService.cs orchestration and policy
UploadOutcome.cs outcome enum
UploadResult.cs result record
UploaderOptions.cs bound from configuration
SchemeFileUploader.Cli/ console entry point
Program.cs argument parsing, exit codes
appsettings.json UseDevelopmentStorage=true
tests/
SchemeFileUploader.Tests/
FakeBlobUploader.cs in-memory implementation of the seam
UploadServiceTests.cs policy tests, no emulator, run in CI
AzuriteIntegrationTests.cs one test, [Trait("Category","Integration")]
data/
contributions-2026-07.csv synthetic, no PII
```
### Key types to create
```csharp
public interface IBlobUploader
{
Task ExistsAsync(string container, string blobName, CancellationToken ct);
Task UploadAsync(string container, string blobName, Stream content,
bool overwrite, CancellationToken ct);
}
public enum UploadOutcome { Uploaded, Failed } // more members follow from decision 1
public sealed record UploadResult(
UploadOutcome Outcome,
string BlobName,
string Container,
long BytesWritten,
string Message);
public sealed class UploaderOptions
{
public string ConnectionString { get; init; } = "UseDevelopmentStorage=true";
public string DefaultContainer { get; init; } = "contributions";
}
```
`ExistsAsync` is on the interface from the start because every candidate answer to decision 1 needs it. Committing to the seam is not the same as committing to the policy.
### Orchestration shape
```csharp
public async Task UploadAsync(string path, string container, CancellationToken ct)
{
var blobName = Path.GetFileName(path);
// decision 1 lands here, before the file is opened
// a refused upload must touch neither the source stream nor the destination
await using var stream = File.OpenRead(path);
var bytes = await _uploader.UploadAsync(container, blobName, stream, overwrite: ?, ct);
return new UploadResult(UploadOutcome.Uploaded, blobName, container, bytes,
$"Uploaded {blobName} ({bytes:N0} bytes)");
}
```
The `?` is not an omission. It is what refinement has to settle.
### CLI contract
```text
Usage: uploader [--container ]
path to the file to upload
--container target container, defaults to UploaderOptions.DefaultContainer
```
| Exit code | Meaning |
|-----------|---------|
| 0 | Uploaded |
| 2 | Failed: file missing, storage unreachable, permission denied |
Exit code 1 is reserved. If decision 1 lands on "refuse", refusal takes 1, so that "I declined to act" stays distinct from "something went wrong".
### Configuration and local run
```jsonc
// src/SchemeFileUploader.Cli/appsettings.json
{ "Uploader": { "ConnectionString": "UseDevelopmentStorage=true",
"DefaultContainer": "contributions" } }
```
`UseDevelopmentStorage=true` is the published Azurite development constant, not a secret. Nothing in this repository connects to a real Azure account.
```bash
npx azurite --silent --location .azurite
dotnet run --project src/SchemeFileUploader.Cli -- data/contributions-2026-07.csv
```
### Test strategy
| Layer | Where | Runs in CI |
|-------|-------|-----------|
| Policy and outcome | `FakeBlobUploader`, an in-memory dictionary behind the seam | yes |
| Wiring against the real SDK | one Azurite test, `[Trait("Category","Integration")]` | no, opt-in |
CI runs `dotnet test --filter "Category!=Integration"`, so no emulator is needed in the pipeline. Mocking `BlobServiceClient` directly is rejected: that asserts the mock rather than the behaviour.
Cases the first slice must cover:
- container empty, upload succeeds and reports the byte count
- source file missing, fails with an actionable message and exit code 2
- no message or log line contains the connection string
### Pinned dependencies
`Azure.Storage.Blobs` 12.29.1, `xunit` 2.9.3, `Microsoft.NET.Test.Sdk` 18.9.0, `azurite` 3.36.0. Target framework `net8.0` (LTS).
## Delivery breakdown
| Slice | Scope | Status |
|-------|-------|--------|
| #2 | Walking skeleton: upload a file to a container, report the outcome, exit codes 0 and 2 | created, linked |
| S2 | Implement the answer to decision 1, with whatever command surface it needs | blocked on open decision 1 |
| S3 | Error handling shaped for the chosen consumer, human or scheduler | blocked on open decision 2 |
| S4 | Per environment configuration and credential sourcing | blocked on open decision 3 |
Only #2 exists as an issue. S2 to S4 are named but deliberately not created: each is gated on an open decision, so creating them now would commit the team to a choice it has not made.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with docs/requirements/prd.md and the three unresolved design decisions; implementation is explicitly blocked on them. Then use the listed entry points—IBlobUploader, UploadService, Program.cs, UploaderOptions, and the named tests—as the implementation map. Done means the acceptance criteria pass, including emulator coverage, actionable failures, exit codes, and no credential material in output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, csharp
- Domain
- cli, cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100