Users' Long term and short term interests

I am trying to add users' stable long-term interests and dynamic short-term interests for next action prediction.May I know the documentation to train the two Xlnet , one transformer to capture the historical behaviour and other to extract short term( current interests according to timestamp).

Is it possible to create a custom architecture to train two transformer towers together with different inputs and then have concat at later point..?

No files, tests, or entry points are named. Start by reviewing the Transformers4Rec documentation and the existing PyTorch transformer and sequential-recommendation architecture, then determine whether dual long-term and short-term XLNet inputs with later concatenation are supported and what a complete implementation would require.