AWS Transfer for SFTP GetFile Error

I set up an AWS SFTP server that is backed by S3 files. I am not able to download a file because I am getting an error when it tries to read the files attributes. I gave full s3 access and it still happens. Stack trace is:

net.schmizz.sshj.sftp.SFTPException: Forbidden
at net.schmizz.sshj.sftp.Response.error(Response.java:140)
at net.schmizz.sshj.sftp.Response.ensurePacketTypeIs(Response.java:117)
at net.schmizz.sshj.sftp.SFTPEngine.stat(SFTPEngine.java:264)
at net.schmizz.sshj.sftp.SFTPEngine.stat(SFTPEngine.java:217)
at net.schmizz.sshj.sftp.SFTPFileTransfer.download(SFTPFileTransfer.java:70)
at net.schmizz.sshj.sftp.SFTPFileTransfer.download(SFTPFileTransfer.java:59)
at net.schmizz.sshj.sftp.SFTPClient.get(SFTPClient.java:229)

For uploads, I was able to get around this with setPreserveAttributes(false) but I don't see a way to disable the stat operation for downloads.