So before I start talking about solutions, I’ll clarify what I meant be there is no secure way to store credentials. Firstly the reason that a token based system is preferred over credentials when providing access to an API is firstly, you don’t need to have a login flow every time, once the API user has to token that is it. Secondly you are able to put limits on tokens, so you can say, a token may access the read API but not the delete endpoint etc. So if the token is ever stolen the damage is minimized.
So now to actually trying to solve your problem. So knowing that there is no way to do this perfectly we have to focus on “what ifs”. What if someone gains access to my application server, how much information can they get? Do that have access to the whole database? Can they modify the database?. What if they download the database, what damage can they do with the stolen data, can they impersonate a person with their credentials. There are all concerns you need to think about.
So well start with the basic, assume that someone gets a copy of your database, that is a problem in itself. But if the credentials to your 3rd party are stored in clear text they can use them basically until the 3rd party disables the account. So what you need to do is encrypt them. There are many methods for this and doing it yourself is not recommended, though the commercial solutions such as Volumetric encryption are not cheap. The other concern with this scheme is that you need some service to store the encryption key. Since if it is kept on the app server or database the attacker can just take it and decrypt the data anyway. There are a few open source projects that offer key management, namely Hashicorp Vault. This way the key is only ever stored in memory and not on the app servers disks.
The problem with this is that if an attacker secretly takes control of the server, they can just sit and wait for the credentials to be decrypted and put in memory before stealing them. And the short answer is if you are storing your credentials in your main application database, there is no way to protect against this. No amount of encryption or obfuscation (obfuscation is no protection anyway).
So your final alternative is to make it as hard as possible for an attacker to get the credentials. And I talked about this a bit in the original thread. You want to have two separate systems, with a very controlled communication link.
- You have your Meteor app servers, there normal, they have a mongo database etc. You have to deal with their security in the normals ways.
- Now you also have your API scraper. This system is much more locked down than the normal app servers and they can not be accessed from the public internet, all they do is scrape data and process it. They have their own database for storing the credentials they need for scraping, that cannot be accessed by the main app servers.
- They then write this processed data into you main apps Mongo database (or to their own depending on how sensitive it is)
- The application servers only every have write access to the secure scraper database (either through a custom service or database access controls). This way if an app server is compromised they can only ever overwrite the users details, not steal them.
Now you might ask, what if my scraper servers get compromised, well that is a possibility, but this way you can harden them as much as possible and put a strict set of communication rules between them and the app servers (which face the internet and are the most likely point of first compromise by an attacker).
Hopefully this was helpful and I am happy to elaborate on any points that are confusing. Though if the 2 service set up sounds too complicated, you can always just store the raw credentials in the database and make sure that the user know that it should be a unique password to the 3rd party service and that if they or the 3rd party notice suspicious activity to notify you and change the credentials.