ipfs pin ls gets slower as the pinned objects increase

#### Version information:
go-ipfs version: 0.4.8-
Repo version: 5
System version: amd64/windows
Golang version: go1.8

#### Type: bug

#### Severity: medium

#### Description:
ipfs pin ls is really fast when the repo size is small, but the bigger it gets the slower it is, this affects other operations, such as calling the gc after unpinning an object. My cpu usage during this process is not very high (ipfs.exe is using around 4-5% CPU).
Is this behaviour normal or it should use more resources to finish faster?

I understand, that listing a lot of files may take a long time, this is usually not a problem since listing all the pinned files without specifying a file or at least a type is not usually done, my main worry is when deleting a file from the repo and using the repo gc command since deleting just one big file can take a long time (6.6min to delete a 20gb video), that may be too long for a regular user.