[Loki] After upgrading from 2.3.0 to 2.9.3, API is not working

**Describe the bug**
As the title says, after upgrading from 2.3.0 to 2.9.3, the API is not working (every /loki/api endpoint returns 404 page not found).

First, we though it is maybe something related to config, since it was relatively speaking big version jump.

Config:
```
auth_enabled: false
server:
http_listen_port: 3100
ingester:
lifecycler:
address: 127.0.0.1
ring:
kvstore:
store: inmemory
replication_factor: 1
final_sleep: 0s
chunk_idle_period: 15m
chunk_retain_period: 30s
max_transfer_retries: 0
wal:
dir: "/tmp/wal"
schema_config:
configs:
- from: 2019-01-01
store: boltdb
object_store: filesystem
schema: v11
index:
prefix: index_
period: 168h
storage_config:
boltdb:
directory: /loki/index
filesystem:
directory: /loki/chunks
limits_config:
enforce_metric_name: false
reject_old_samples: true
reject_old_samples_max_age: 168h
max_entries_limit_per_query: 50000
chunk_store_config:
max_look_back_period: 0s
table_manager:
retention_deletes_enabled: true
retention_period: 672h
```

After we have also tried newer config provided here, to no avail.

**Environment:**
- Infrastructure: Kubernetes
- Deployment tool: Kubernetes deployment