livepeer / livepeer/go-livepeer

Should Show Error When Node Starts With Bad S3 Auth Info

Open
#597 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: icebox type: bug
Dominant language
Go
Stars
586
Forks
226
Avg merge
1d 17h
Merged PRs (30d)
19

Description

Describe the bug
The node should show an error if it's started with the wrong S3 auth info.

To Reproduce

  1. Start B with the wrong S3 auth info
  2. Start streaming to B
  3. B reports error, and stream is not playable.

Expected behavior
If wrong S3 auth is provided, we should either:

  • Exit during boot-up, or
  • Show an error during boot-up, make sure the stream playable (even when S3 upload fails), and show a more reasonable error than:
E1110 19:27:52.800041    6881 s3.go:125] Save S3 error: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>AKIAI7446PVXVE3RTGTQ</AWSAccessKeyId><StringToSign>eyAiZXhwaXJhdGlvbiI6ICIyMDE4LTExLTExVDE5OjI1OjIxLjM4WiIsCiAgICAiY29uZGl0aW9ucyI6IFsKICAgICAgeyJidWNrZXQiOiAib3MtdGVzdC1yaW5rZWJ5In0sCiAgICAgIHsiYWNsIjogInB1YmxpYy1yZWFkIn0sCiAgICAgIFsic3RhcnRzLXdpdGgiLCAiJENvbnRlbnQtVHlwZSIsICIiXSwKICAgICAgWyJzdGFydHMtd2l0aCIsICIka2V5IiwgImFjYjdjNTBkOTY4NzUxMTc1NmE0MTFhZjdmNjE2NTU3NDQ3ZmNiYTEyOWRmYzc5MjE0NWM5NDFhOWZkNDQwYzciXSwKICAgICAgeyJ4LWFtei1hbGdvcml0aG0iOiAiQVdTNC1ITUFDLVNIQTI1NiJ9LAogICAgICB7IngtYW16LWNyZWRlbnRpYWwiOiAiQUtJQUk3NDQ2UFZYVkUzUlRHVFEvMjAxODExMTAvdXMtZWFzdC0xL3MzL2F3czRfcmVxdWVzdCJ9LAogICAgICB7IngtYW16LWRhdGUiOiAiMjAxODExMTBUMDAwMDAwWiIgfQogICAgXQogIH0=</StringToSign><SignatureProvided>3afbead81ab1e0288c3bee3ca733c29002cd20abb92c03160b625ee4355e563e</SignatureProvided><StringToSignBytes>65 79 41 69 5a 58 68 77 61 58 4a 68 64 47 6c 76 62 69 49 36 49 43 49 79 4d 44 45 34 4c 54 45 78 4c 54 45 78 56 44 45 35 4f 6a 49 31 4f 6a 49 78 4c 6a 4d 34 57 69 49 73 43 69 41 67 49 43 41 69 59 32 39 75 5a 47 6c 30 61 57 39 75 63 79 49 36 49 46 73 4b 49 43 41 67 49 43 41 67 65 79 4a 69 64 57 4e 72 5a 58 51 69 4f 69 41 69 62 33 4d 74 64 47 56 7a 64 43 31 79 61 57 35 72 5a 57 4a 35 49 6e 30 73 43 69 41 67 49 43 41 67 49 48 73 69 59 57 4e 73 49 6a 6f 67 49 6e 42 31 59 6d 78 70 59 79 31 79 5a 57 46 6b 49 6e 30 73 43 69 41 67 49 43 41 67 49 46 73 69 63 33 52 68 63 6e 52 7a 4c 58 64 70 64 47 67 69 4c 43 41 69 4a 45 4e 76 62 6e 52 6c 62 6e 51 74 56 48 6c 77 5a 53 49 73 49 43 49 69 58 53 77 4b 49 43 41 67 49 43 41 67 57 79 4a 7a 64 47 46 79 64 48 4d 74 64 32 6c 30 61 43 49 73 49 43 49 6b 61 32 56 35 49 69 77 67 49 6d 46 6a 59 6a 64 6a 4e 54 42 6b 4f 54 59 34 4e 7a 55 78 4d 54 63 31 4e 6d 45 30 4d 54 46 68 5a 6a 64 6d 4e 6a 45 32 4e 54 55 33 4e 44 51 33 5a 6d 4e 69 59 54 45 79 4f 57 52 6d 59 7a 63 35 4d 6a 45 30 4e 57 4d 35 4e 44 46 68 4f 57 5a 6b 4e 44 51 77 59 7a 63 69 58 53 77 4b 49 43 41 67 49 43 41 67 65 79 4a 34 4c 57 46 74 65 69 31 68 62 47 64 76 63 6d 6c 30 61 47 30 69 4f 69 41 69 51 56 64 54 4e 43 31 49 54 55 46 44 4c 56 4e 49 51 54 49 31 4e 69 4a 39 4c 41 6f 67 49 43 41 67 49 43 42 37 49 6e 67 74 59 57 31 36 4c 57 4e 79 5a 57 52 6c 62 6e 52 70 59 57 77 69 4f 69 41 69 51 55 74 4a 51 55 6b 33 4e 44 51 32 55 46 5a 59 56 6b 55 7a 55 6c 52 48 56 46 45 76 4d 6a 41 78 4f 44 45 78 4d 54 41 76 64 58 4d 74 5a 57 46 7a 64 43 30 78 4c 33 4d 7a 4c 32 46 33 63 7a 52 66 63 6d 56 78 64 57 56 7a 64 43 4a 39 4c 41 6f 67 49 43 41 67 49 43 42 37 49 6e 67 74 59 57 31 36 4c 57 52 68 64 47 55 69 4f 69 41 69 4d 6a 41 78 4f 44 45 78 4d 54 42 55 4d 44 41 77 4d 44 41 77 57 69 49 67 66 51 6f 67 49 43 41 67 58 51 6f 67 49 48 30 3d</StringToSignBytes><RequestId>28888019B3416797</RequestId><HostId>GgeyQ+z7fSCFEXJ8K2CJkxqP9ZWNFIqHAfOCGcjPZ2EKzEBex0OklzNTbMPH76P6XMN5ewjC9H8=</HostId></Error>
E1110 19:27:52.800057    6881 mediaserver.go:360] Error saving segment 150: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>AKIAI7446PVXVE3RTGTQ</AWSAccessKeyId><StringToSign>eyAiZXhwaXJhdGlvbiI6ICIyMDE4LTExLTExVDE5OjI1OjIxLjM4WiIsCiAgICAiY29uZGl0aW9ucyI6IFsKICAgICAgeyJidWNrZXQiOiAib3MtdGVzdC1yaW5rZWJ5In0sCiAgICAgIHsiYWNsIjogInB1YmxpYy1yZWFkIn0sCiAgICAgIFsic3RhcnRzLXdpdGgiLCAiJENvbnRlbnQtVHlwZSIsICIiXSwKICAgICAgWyJzdGFydHMtd2l0aCIsICIka2V5IiwgImFjYjdjNTBkOTY4NzUxMTc1NmE0MTFhZjdmNjE2NTU3NDQ3ZmNiYTEyOWRmYzc5MjE0NWM5NDFhOWZkNDQwYzciXSwKICAgICAgeyJ4LWFtei1hbGdvcml0aG0iOiAiQVdTNC1ITUFDLVNIQTI1NiJ9LAogICAgICB7IngtYW16LWNyZWRlbnRpYWwiOiAiQUtJQUk3ND
Q2UFZYVkUzUlRHVFEvMjAxODExMTAvdXMtZWFzdC0xL3MzL2F3czRfcmVxdWVzdCJ9LAogICAgICB7IngtYW16LWRhdGUiOiAiMjAxODExMTBUMDAwMDAwWiIgfQogICAgXQogIH0=</StringToSign><SignatureProvided>3afbead81ab1e0288c3bee3ca733c29002cd20abb92c03160b625ee4355e563e</SignatureProvided><StringToSignBytes>65 79 41 69 5a 58 68 77 61 58 4a 68 64 47 6c 76 62 69 49 36 49 43 49 79 4d 44 45 34 4c 54 45 78 4c 54 45 78 56 44 45 35 4f 6a 49 31 4f 6a 49 78 4c 6a 4d 34 57 69 49 73 43 69 41 67 49 43 41 69 59 32 39 75 5a 47 6c 30 61 57 39 75 63 79 49 36 49 46 73 4b 49 43 41 67 49 43 41 67 65 79 4a 69 64 57 4e 72 5a 58 51 69 4f 69 41 69 62 33 4d 74 64 47 56 7a 64 43 31 79 61 57 35 72 5a 57 4a 35 49 6e 30 73 43 69 41 67 49 43 41 67 49 48 73 69 59 57 4e 73 49 6a 6f 67 49 6e 42 31 59 6d 78 70 59 79 31 79 5a 57 46 6b 49 6e 30 73 43 69 41 67 49 43 41 67 49 46 73 69 63 33 52 68 63 6e 52 7a 4c 58 64 70 64 47 67 69 4c 43 41 69 4a 45 4e 76 62 6e 52 6c 62 6e 51 74 56 48 6c 77 5a 53 49 73 49 43 49 69 58 53 77 4b 49 43 41 67 49 43 41 67 57 79 4a 7a 64 47 46 79 64 48 4d 74 64 32 6c 30 61 43 49 73 49 43 49 6b 61 32 56 35 49 69 77 67 49 6d 46 6a 59 6a 64 6a 4e 54 42 6b 4f 54 59 34 4e 7a 55 78 4d 54 63 31 4e 6d 45 30 4d 54 46 68 5a 6a 64 6d 4e 6a 45 32 4e 54 55 33 4e 44 51 33 5a 6d 4e 69 59 54 45 79 4f 57 52 6d 59 7a 63 35 4d 6a 45 30 4e 57 4d 35 4e 44 46 68 4f 57 5a 6b 4e 44 51 77 59 7a 63 69 58 53 77 4b 49 43 41 67 49 43 41 67 65 79 4a 34 4c 57 46 74 65 69 31 68 62 47 64 76 63 6d 6c 30 61 47 30 69 4f 69 41 69 51 56 64 54 4e 43 31 49 54 55 46 44 4c 56 4e 49 51 54 49 31 4e 69 4a 39 4c 41 6f 67 49 43 41 67 49 43 42 37 49 6e 67 74 59 57 31 36 4c 57 4e 79 5a 57 52 6c 62 6e 52 70 59 57 77 69 4f 69 41 69 51 55 74 4a 51 55 6b 33 4e 44 51 32 55 46 5a 59 56 6b 55 7a 55 6c 52 48 56 46 45 76 4d 6a 41 78 4f 44 45 78 4d 54 41 76 64 58 4d 74 5a 57 46 7a 64 43 30 78 4c 33 4d 7a 4c 32 46 33 63 7a 52 66 63 6d 56 78 64 57 56 7a 64 43 4a 39 4c 41 6f 67 49 43 41 67 49 43 42 37 49 6e 67 74 59 57 31 36 4c 57 52 68 64 47 55 69 4f 69 41 69 4d 6a 41 78 4f 44 45 78 4d 54 42 55 4d 44 41 77 4d 44 41 77 57 69 49 67 66 51 6f 67 49 43 41 67 58 51 6f 67 49 48 30 3d</StringToSignBytes><RequestId>28888019B3416797</RequestId><HostId>GgeyQ+z7fSCFEXJ8K2CJkxqP9ZWNFIqHAfOCGcjPZ2EKzEBex0OklzNTbMPH76P6XMN5ewjC9H8=</HostId></Error>

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading s3.go around line 125 and mediaserver.go around line 360, then trace how S3 authentication failures are handled during node startup and segment saving. Reproduce the bad-auth scenario and determine whether the node should fail at boot or preserve stream playback while reporting a clear error; done means the chosen behavior is observable and the misleading raw S3 response is replaced.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, go
Domain
backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.