How to set timeout properly

I've set "timeout" in the provider section of the serverless.yml file to 300, and I am calling my function with "RequestResponse". My function takes about 3 minutes or so to run. But serverless offline is retrying my function after 120 seconds, and does this a few times and finally errors out with a [TimeoutError]: Connection timed out after 120000ms. I don't expect any retries.