[Discussion] LiquidCache support for only InMemory transcoding

I was trying to see if there is a way in which we can just create inMemory Liquid format and avoid spilling to disk everytime. Wanted to check if currently what are the ways to achieve it? Didn't find it in my initial analysis. Also I found this:

/// Insert a batch into the cache, it will run cache replacement policy until the batch is inserted.
    pub(crate) fn insert_inner(&self, entry_id: EntryID, mut batch_to_cache: CachedBatch) {
        loop {
            let batch_type = CachedBatchType::from(&batch_to_cache);
            let Err(not_inserted) = self.try_insert(entry_id, batch_to_cache) else {
                self.cache_policy.notify_insert(&entry_id, batch_type);
                return;
            };

            let victims = self.cache_policy.find_victim(8);
            if victims.is_empty() {
                // no advice, because the cache is already empty
                // this can happen if the entry to be inserted is too large, in that case,
                // we write it to disk
                let on_disk_batch = self.write_in_memory_batch_to_disk(entry_id, not_inserted);
                batch_to_cache = on_disk_batch;
                continue;
            }
            self.squeeze_victims(victims);

            batch_to_cache = not_inserted;
            crate::utils::yield_now_if_shuttle();
        }
    }

This kind of write it to disk in case of insufficient memory always. @XiangpengHao let me know if I'm missing any very obvious code path..

Just checking on if there is any plan to introduce just InMemoryLiquid mode?

Start at the mentioned insert_inner entry point and trace the current LiquidCache transcoding path, especially write_in_memory_batch_to_disk. Determine whether an InMemory-only mode is intended, then define its scope and acceptance criteria before identifying the tests needed to verify that batches do not spill to disk.