Clean Architecture with Large Cursor

I have an app that has a recyclerview. The recyclerview loads data using an custom adapter. If we set in our adapter a cursor with large data (For example 3000 rows) the recyclerview scrolls smoothly. If we set in our adapter a list of objects (even a small amount, like 500 objects) the recyclerview lags on scroll. I suppose that the reason is that we consume a lot of resources by this approach and also the cursor uses a window and not a reference to all the data.

My question is how to handle this situation? Before using clean architecture i just passed the cursor to the recyclerview and everything worked ok. After using rxJava I create an object in every onNext call and keep them in a list.

Also there is another problem. The domain layer does not have reference to android classes. As a result I can not have Observable in order to pass it to presentation layer. Is the best way to keep an unchecked observable? Is there any different approach (For example any rxJava operator)?