Publication for publication where documents are linked as a tree

I have 4 collections of objects growing to more than thousands of documents each. Currently I am publishing the whole collections and the application takes thus 20s to 30s to start but it works fine.

As my collections will carry on growing, I am trying to publish only the relevant part of them.

The problem is that my collections documents contain pointers pointing to others documents in the same collections and in other collections

How can I manage the publications in order to display a single document view where I need to display information from other linked documents from the same collection and from other collections ? Currently I have client side code to explore the links on the documents. Do I have to implement the same exploration code also on the server to manage the publications ?

Thanks a lot for your help