GridFS vs NoGridFS (DDP connection) for small files

I wonder which is better for the long run? to save files using GridFS and access it using files’ URLs or just store their base64 representation directly on normal mongo collection?
As I heard that meteor server caches all data published so on the long run when number of these small files increases, meteor server will cache all of them which will be a heavy wait on meteor server shoulders, and it will be good to use GridFS instead as it uses URLs and HTTP protocol. I am I wrong?