You are right! Both .pem and .crt/.key work.
The issue seems to be that a long time ago I first tried to set up with mup before doing it with mupx since mup wasn’t working. MUP created an opt/something folder, which is no longer used in mupx. mup has a known issue when deploying certificates with that folder available. The solution for me was to simply create a new app with a new appName, which creates a new docker container I guess. An alternative solution is to get rid of that specific opt folder, which I couldn’t find though, which is why I simply used a new appName in my mup.json file
Don’t forget to stop the old app before starting and deploying a new app.