Loading external json file into a mongo collection

help MongoDB
1

Well, that pretty much says it. Can’t seem to find a working method to do just that.

So i have created a Libs collection which is empty at the moment. Here is the schema:

libs.attachSchema(
    new SimpleSchema({
    name: {
      type: String
    },
    descript: {
      type: String
    },
    type: {
      type: String
    },
    href: {
      type: String
    },
    src: {
      type: String
    },
    dependencies: {
      type: [Object],
      blackbox: true
    },
    "dependencies.$.name": {
      type: String
    },
    "dependencies.$.descript": {
      type: String
    },
    "dependencies.$.type": {
      type: String
    },
    "dependencies.$.href": {
      type: String
    },
    "dependencies.$.src": {
      type: String
    },
    default: {
      type: Boolean
    }
  })
);

Now i have a libs.json file in my server directory. I want to load it up in my libs collection in order to fill it and the only code that i’ve found to do just that is this: ( I’ve put it in server/startup/loadLibs.js)

Meteor.startup(function () {

  var fs = Npm.require("fs");
  var Fiber = Npm.require("fibers");

  fs.readFile(Assets.getText("libs.json"), 'utf8', function (err, data) {

    data = JSON.parse(data);

    Fiber(function () {

      _.each(data, function (document) {
        libs.insert(document);
      });
    }).run();
  });
});

I guess there are a couple of things that i don’t quite get because what happens is that the collection isnt filled up with the json data AND the return of the code above is [Object, Object]…

Anybody could help?

2

You don’t need fibers or fs for this. See the example here.

2 Likes
3

it’s probably loading the whole json file as one object. you’d need to iterate its contents/children to insert.

some other options:

you can load JSON directly into mongo:
http://docs.mongodb.org/manual/reference/program/mongoimport/
but for your production env you then need a way to directly access mongo - not always the safest approach.

or you could define your JSON instead as coffeescript, which is cleaner than raw fugly JSON, and just iterate then.

4

I was just going through my bookmarks to find the link to that post of yours to paste in as a reference :smile:

5

Thanks, it gets me a little forward.

But what about nested arrays/objects within the main json object (mine is simple called “libs{}”), inside the .json file?

Should i use nested loops inside the insert command like so?:

Meteor.startup(function () {

	var libs = [];
	libs = JSON.parse(Assets.getText("libs.json"));
    _.each(libs.libs, function (data) {
  	   console.log(libs);
       Libs.insert(data);
       _.each(libs.dependencies, function(deps) {
    	Libs.dependencies.insert(deps);
    });
  });
});
6

you can stick it in mongo as is, and use deep embedded document queries to get the info you need later.
http://docs.mongodb.org/manual/tutorial/query-documents/#embedded-documents

thats why they call it a document database!

but it sounds like you need to rethink your data structure if you’re already confused about storing blobs in the DB.

OH! possibly it’s not JSON. If you were originally seeing [Object, Object] it’s probably still a string that you have. You need to JSON.parse

7

Loading external json file into a Mongo Collection using Mongoimport following command used: mongoimport --db db_name --collection coll-name --type json --file seed.json --jsonArray
If the specified collection does’t exist, It will be automatically created otherwise appended into existing collection.

1 Like
8

how can I automate this to run when my app starts?

1 Like
9

The problem with mongoimport is that the imported documents use ObjectId rather than String in their _id fields.

1 Like
10

Agree and I’m struggling with it :frowning:

11

Hi,

This is great that we can read JSON file on server side using JSON.parse(Assets.getText(…) method.

I realized that this is not the right topic for me to ask on this discussion thread, but I really needs to have the ability to read JSON file on client side, because my Meteor app will be running on iPhone and the App needs to be able to load data from json file without internet connection.

I had post message "I have a Meteor app and needs to read data, from a file “/public/_assets/results/mmresults.json”, within the file “/client/controllers.js” in this forum.

I am not looking for a way to browse and select file to upload using the browse button.
Is there a way I can store a local mongo db on an iOS device?

Please provide me suggestions as to what I need to do to read the data from my app’s json file.

Thanks in advance everyone.

-Brian-