A couple of things here.
Firstly, the private folder is used for static assets and is created along with the rest of the collateral belonging to your app - you shouldn’t use it as an upload location. Check the Assets section of the docs.
You should avoid using any fs.xxxSync calls - they lock the nodejs event loop up, completely stopping any other nodesj code until the function completes. On a moderately busy system, that can make the server code almost unusable.
I see… thanks for the insight!
I need to read an xml and create files using data from the xml in a
directory within my app. What are the other options I should go for?
Just to be clear, there’s nothing actually preventing you from using the private folder. It’s just that the contents are packaged as part of the build. That, in turn, means that the content may be nuked on a redeployment. In fact, depending on how you choose to host your production code, you may find your local file storage is entirely disposable. For that reason, S3 is often used for permanent file storage.
However, if you want to use local storage, you can use private or any server folder outside of the application root or application build phase.
I wonder if you got your error because you didn’t wrap your fs call in a try/catch and the directory didn’t exist?
You should avoid using any fs.xxxSync calls - they lock the nodejs event loop up, completely stopping any other nodesj code until the function completes. On a moderately busy system, that can make the server code almost unusable.
The app does need to first create those files before it goes on to do anything next. Once all files are written, then only app needs to execute further. So async function here may not be helpful. Thoughts?
Assets section of the docs
Thanks for sharing the link. I also looked through the Assets api documentation. It also says there that “Note that assets are read-only.”. I reckon anything and everything inside Private folder is “assets”?
I think, using Private folder to write those files should be fine because those files will be created when app will be used by users. So when deploying on production, Private folder doesn’t need to have those files.
Despite all my extensive search, I am not able to find a way even to access Private folder from Server folder, let alone creating a folder or file there.
I am looking for few answers here:
function checkDirectorySync(directory) {
try {
fs.statSync(directory);
} catch(e) {
console.log(e);
fs.mkdirSync(directory);
}
}
checkDirectorySync("../private/xmls");
results into this error:
{ Error: ENOENT: no such file or directory, stat 'D:\D drive\Ani\Learnings\Meteor\staticsite\.meteor\local\build\programs\private\xmls'
at Object.fs.statSync (fs.js:948:11)
at checkDirectorySync (server/methods.js:94:8)
at methods.js (server/methods.js:101:1)
at fileEvaluate (packages\modules-runtime.js:343:9)
at require (packages\modules-runtime.js:238:16)
at D:\D drive\Ani\Learnings\Meteor\staticsite\.meteor\local\build\programs\server\app\app.js:453:1
at infos.forEach.info (D:\D drive\Ani\Learnings\Meteor\staticsite\.meteor\local\build\programs\server\boot.js:414:13)
at Array.forEach (<anonymous>)
at D:\D drive\Ani\Learnings\Meteor\staticsite\.meteor\local\build\programs\server\boot.js:413:9
at D:\D drive\Ani\Learnings\Meteor\staticsite\.meteor\local\build\programs\server\boot.js:463:5
errno: -4058,
code: 'ENOENT',
syscall: 'stat',
path: 'D:\\D drive\\Ani\\Learnings\\Meteor\\staticsite\\.meteor\\local\\build\\programs\\private\\xmls' }
C:\Users\animeshsharma\AppData\Local\.meteor\packages\meteor-tool\1.6.0_1\mt-os.windows.x86_64\dev_bundle\server-lib\node_modules\fibers\future.js:280
throw(ex);
Thanks for your help!!!
Basically, yes. But note that assets accessed via the Assets API are read-only, but if you use the fs commands, you can write to the files. However, bear in mind my comments about deployment. Deploying to many cloud services (including Galaxy and AWS EC2) provides a transient filesystem. While you can use EBS on EC2 to provide permanent disk storage, there is a further disadvantage of local storage: if you are running multiple instances, each has its own, independent local storage. Hence my suggestion to use S3.
I did test this myself to ensure nothing has changed, and I can create, write and read files and folders under private. I used the async version:
fs.stat(directory, err => {
console.log(err);
fs.mkdir(directory, err => {
console.log(err);
});
});
which worked as expected. The “test and create” method is not recommended, because it offers no advantages over a simple fs.mkdir or fs.mkdirSync, which will always be atomic.
Note that the private folder is not in your application root, but in the runtime root. In development, that’s D:\D drive\Ani\Learnings\Meteor\staticsite\.meteor\local\build\programs\private as reported in your original post. When you deploy into production, your runtime root is whatever you want it to be.
Yes.
Yes - but note my comments about deployment and multiple instances.
Yes and no 
Yes, if you create/update files in the private folder in your app’s root in development*. No, if you do it in the private folder in your runtime root. There is no file watching in a deployed, production app.
* Interestingly, I’ve just compared Windows with Linux and the Windows file watcher doesn’t pick up changes in the private folder. I think that’s a bug.
@robfallows, a big thanks for your response and elaborated answers to my questions!!! However, I was afraid that my post was too overwhelming to get a response.
I have been struggling on this for two days now. Considering the challenges I am facing vs. help and related stuff available on web, I am thinking to change my approach now. Rather than creating folders and files in Private directory, how about storing these files into MongoDb collection and operate from there?
What do you suggest between writing files to Private vs. storing those in DB?
I did some research on this and found your one post on it. I cloned the git repo (https://github.com/robfallows/ukgrid) and tried to run on my local but those API’s don’t seem to work anymore. What should I do to get that working?
What my app is supposed to do is:
This is the functionality of one phase. In second phase, user can go on to create/update a static version of a given site based upon these XMLs.
That’s basically GridFS (unless you’re rolling your own). You could look at ostrio:files, which integrates with GridFS among many others. Definitely much better than using the local filesystem 
Great, thank you.
I did some research on this and found your one post on it. I cloned the git repo (https://github.com/robfallows/ukgrid) and tried to run on my local but those API’s don’t seem to work anymore. What should I do to get that working?
Any clue on this? As the description suggests, your app is doing almost the same what I am looking for - getting external XML data, store that into DB, and then read from DB to display.
I’ll take a look. As long as the endpoints it’s calling are still there, I’ll update it.
Thank you so much. I am trying your suggestion for ostrio:files. I will update you with my results.
One more thing, what will it take me to be able to post new items in this forum?
New topics? You should be able to do that by now: https://forums.meteor.com/badges/1/basic?username=ani1219
Yeah, you are right. One more “thanks”. 
OK, I’ve updated the repo. Sadly, it’s not as easy to use as it was, and the frequency endpoint has been removed completely. However, what works, works
Thanks, @robfallows .
I am not sure what I am doing wrong but when I run APIKEY=“my-api-key” meteor --settings settings.json, console says ’APIKEY’ is not recognized as an internal or external command, operable program or batch file.
Do I need to do this?
Please suggest.
OK. You’re using Windows. On PowerShell you will need to do
$env:APIKEY="your-api-key"
meteor --settings settings.json
In cmd
set APIKEY="your-api-key"
meteor --settings settings.jsonThanks, @robfallows.
I got this running by setting process.env.APIKEY in ukgrid.js on server. Cheers!
Cool - just remember not to publish that code to a public repo, or your API key will be exposed!
Yeah… I understand that. Thanks.
Hi, Rob.
I need to query a collection and store all document objects into an array. I’ve been looking around for a solution on this without success. Is there any builtin way in Meteor for this?