koala73 / koala73/worldmonitor
Set up fixed IMD egress proxy and restore Railway seeder
- Dominant language
- TypeScript
- Stars
- 86.6k
- Forks
- 13.1k
- Avg merge
- 8h 4m
- Merged PRs (30d)
- 825
Description
## Problem
IMD binds each production API key to one public server IPv4. The Railway service
`seed-imd-cyclone-marine` has three possible static outbound IPv4 addresses, so
direct requests cannot reliably satisfy that binding.
The previous Decodo Dedicated ISP proxy cannot be used because Decodo blocks
government targets and does not allow that restriction to be removed for this
product.
Keep the IMD integration disabled by leaving `IMD_API_KEY` absent from the
Railway seeder until every activation check below passes. Do not consume another
IMD key/IP binding before the fixed egress address is proven.
## Proposed solution
Create a small Amazon Lightsail Linux instance in Mumbai and attach a Lightsail
static IPv4. Run a dedicated Squid HTTPS forward proxy that is usable only by
the IMD Railway seeder.
Recommended starting size: Lightsail Micro, 1 GB RAM, approximately $7/month.
Research and provider comparison:
`docs/research/imd-fixed-egress-proxy-options-2026-09-05.md`
## Security requirements
- Use a dedicated DNS hostname and a publicly trusted TLS certificate.
- Require proxy authentication. Store the URL only as a Railway secret.
- Permit inbound proxy connections only from the three Railway outbound IPv4
addresses for `seed-imd-cyclone-marine`.
- Permit only `CONNECT api.imd.gov.in:443`.
- Deny all other destinations, ports, and HTTP methods.
- Do not decrypt or inspect the end-to-end IMD TLS connection.
- Do not log proxy credentials, IMD credentials, JWTs, or complete secret URLs.
- Keep SSH access restricted and apply operating-system security updates.
## Implementation tasks
- [ ] Create the Lightsail instance in the Mumbai region.
- [ ] Attach a static public IPv4 and confirm that it survives stop/start.
- [ ] Point a dedicated DNS hostname at the static IPv4.
- [ ] Install and configure Squid with an HTTPS listener and public certificate.
- [ ] Add Basic authentication and the IMD-only destination ACL.
- [ ] Restrict the Lightsail firewall to Railway's three outbound IPv4 addresses.
- [ ] Configure Railway `PROXY_URL` as an authenticated `https://` proxy URL.
- [ ] Confirm the seeder fails closed when the proxy is missing or invalid.
- [ ] Test the proxy from the Railway service before registering the IP with IMD.
- [ ] Register the proven Lightsail static IPv4 with the IMD production key.
- [ ] Restore `IMD_API_KEY` only after the registration is complete.
- [ ] Redeploy and run the IMD seeder.
- [ ] Update the operations documentation with provisioning, rotation, recovery,
and validation steps.
## Acceptance criteria
- [ ] Two egress checks, separated by an instance restart or Railway redeploy,
report the same Lightsail static IPv4.
- [ ] `CONNECT api.imd.gov.in:443` succeeds without a proxy-generated 403.
- [ ] The IMD TLS certificate validates end to end.
- [ ] Missing proxy authentication is rejected.
- [ ] A request to a non-IMD destination is rejected.
- [ ] IMD JWT minting succeeds through the proxy after the key/IP binding.
- [ ] At least one documented IMD product request returns an origin response.
- [ ] A full seeder run reports successful IMD product statuses and publishes a
non-zero record count to Redis.
- [ ] IMD health is green based on fresh data, not only a successful process exit.
- [ ] Logs and health output expose no credentials or proxy URL secrets.
## Rollback
If any acceptance check fails, remove `IMD_API_KEY` from the Railway service
again and keep the source disabled. Preserve last-good Redis data according to
the existing seeder safety contract. Do not fall back to direct IMD requests.
Contributor guide
Research direction
Start with docs/research/imd-fixed-egress-proxy-options-2026-09-05.md and the existing seeder safety contract. Provision the Mumbai Lightsail instance, configure the authenticated IMD-only Squid proxy, and test the Railway service before registering the IP or restoring IMD_API_KEY. Done means every acceptance check passes, the seeder publishes records to Redis, health is green, and secrets are absent from logs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, linux
- Domain
- cloud, devops, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100