App Structure - Monoliths & Snowflakes ❄ - How do we structure our mid-sized app?

Well hello there fellow Meteor-ians!

I’d like to get some ideas & feedback on our general architecture.

I’d be grateful for any comments, examples and ideas you have on tap.

Architecture Notes:

We have a medium-sized app, and by that I mean a fairly featureful Cordova app, (still) built with Blaze and the cool template-controller - Package.
Also, meteor-collection-helpers (a Meteor package that allows you to define helpers on your mongoDb documents) is supercool for adding functionality to all mongo documents on both client and server.
(There are many other great packages and little tricks we’re using, but it’d be too much to write it all out here. If there is any interest I can go through our favourites and list them out + write a few words to each, but i’ll just leave that for the future for now.)

Introduction: Moving from automatic imports to imports - directory and explicit imports

We’re currently in the process of restructuring our mid-sized app (I’d say) from the “old” Meteor style of “Just put everything into the filesystem & let Meteor sort it out” (with some additional import's mixed in where necessary) to the newfangled “Put everything into the imports - directory and add a caption of explicit imports everywhere” - style the kids seem to like so much these days.

“Snowflakes” - How the original architecture grew

In the beginning we started with a general structure “as usual” I’d say in the early days for small Meteor projects, something like this:

/client/
    /css/
    /templates/
        /someSpecificView/
            someSpecificViewTemplate.js
            someSpecificViewTemplate.html
        template.html
        head.html
    routes.js
/collections/
    institutions.js
    communicationChannels.js
    communicationMessages.js
    users.js
/lib/
    coolSharedHelpers.js
/methods/
    /server/
        verySecretServerMethod.js
        anotherVerySecretServerMethod.js
    aNotSoSecretMethod.js
/public/
    logo.png
    otherstuff.png
settings.json

Yadda yadda. Something like this. And it was fun and all was good in the land of the little monolith.

But as it grew, so did the number of collections, methods, library functions etc. And the little monolith knew it had to grow up and become a more structured little monolith, like a little city with different districts for different purposes. So we moved to a structure a little more like this:

Snowflakes of features

Ok, I probably this is more of a fractal-kind of thing, but snowflakes have this fractal-y structure too, right, and it’s winter, so I’ll roll with it.

As the app became larger, we started to create a services directory, which we could as well have called modules, but inspired by Domain Driven Design we wanted to kind of group related collections into “services”.

Into these service directories we moved certain areas of our business domain, each looking like a smaller version of the original structure, put into it’s own subdirectory (That’s where the “fractal” / “snowflake-y” - idea comes from):

/lib/services/
        /communication/
            /client/
                /someView/
                    someView.html
                    someView.js
            /collections/
                communicationChannels.schema.js
                communicationChannels.pub.js
                communicationChannels.helpers.js
                communicationMessages.schema.js
                communicationMessages.pub.js
                communicationMessages.helpers.js
            /lib/
                communicationRelatedUtil.js
            /methods/
                communicationRelatedMethod.js
        /appointmentBooking/
            /client/
                /someView/
                    someView.html
                    someView.js
            /collections/
                bookings.schema.js
                bookings.pub.js
                bookings.helpers.js
            /lib/
                bookingRelatedUtil.js
            /methods/
                bookingRelatedMethod.js

etc. Basically sorting stuff by relatedness into certain modules.

We never went into the direction of Meteor Packages as these always seemed a bit hairy to operate, especially as one had to list every file one wanted to include in the package (but that might have changed now since the javascript module support (import) has arrived?)

Interlude: Dude, where is your business layer?

Basically we’re doing “business” like this:

  1. Putting “Action” code into Method bodies, treating Methods as the only way “events” are triggered in the system which change the state of our App/Database
  2. Putting supporting / business code either into
    a) collection helpers (when closely related to a certain entity),
    b) in files in the /services/[service]/lib - folder of the module the code relates too.

That seems to work fine so far.

Step 3: Take all your pretty snowflakes from here: and put them all into /imports

That’s where we are now, basically. We took the entire app and put it into /imports, with additional imports for everything that needs to be imported at startup, especially on the server side, such as Publications and Method declarations of course.

Also we separated out the template / Frontend Section (mostly templates, css & template-related code) into a separate /imports/ui/ - folder.

So are you dun effin’ up yet? :smiley:

Yes… sooo… on a scale from 1 - 10, how bad is it, doctor? :smiley:

→ No, honestly, so far so good, fingers crossed, we’re quite happy with our apps structure & complexity so far. But what do you think? Do you like this basic app layout? What would you change?

Questions we have now about the apps structure for the future

To be a monolith, or to be a monolith built from separate packages?

We like it to be mostly monolithic, to be honest. The general feeling is, the more we’d mess around with multiple Meteor or NPM packages for our stuff, the more complicated just building and refactoring stuff would get. What’s the general feel / opinion on that?

We are considering breaking out certain functionality and maybe publishing it for the community. But that’d be mostly for self-containd and generally reusable functionality.

Would anyone suggest putting eg. our individual services (see above) into different (Meteor/NPM) - Packages? What would the payoffs be and what the disadvantages? How much bigger would the development overhead be?

Global import files for eg. pubs & method declarations or a fractal / snowflake - like structure?

Basically the question is this:

Should we prefer either:

/imports/
    /startup/
        /server/
            allMethods.js
                // import '/imports/services/service1/methods/method1.js'
                // import '/imports/services/service1/methods/method2.js'
                // import '/imports/services/service1/methods/method3.js'
                
                // import '/imports/services/service2/methods/method1.js'
                // (...)
            allPublications.js
                // (...)

(Which would possibly have the advantage of seeing everything the app publishes in one place)

Or something more “recursive” like this:

/imports/
    /startup/
        /server/
            startup.js
                // import '/imports/services/startup-server'
    /services/
        /communication/
            /collections/
                communicationChannels.pub.js
                communicationMessages.pub.js
            /methods/
                communicationRelatedMethod.js
            startup-server.js
                // import './collections/communicationChannels.pub'
                // import './collections/communicationMessages.pub'
                //
                // import './methods/communicationRelatedMethod'
        /appointmentBooking/
            /collections/
                bookings.pub.js
            /methods/
                bookingRelatedMethod.js
            startup-server.js
                // import './collections/bookings.pub'
                //
                // import './methods/bookingRelatedMethod'
        startup-server.js
            // import './communication/startup-server'
            // import './appointmentBooking/startup-server'

-> Which I think we’re going to prefer to keep things a bit more self-contained and hierarchical.

I also hope that this might be more efficient in the future for stuff like tree-shaking, hot module replacement etc whatever if not everything is in one big import.

What are the best practices? Are there any examples for structuring Meteor Code?

  • Should we split our code up into packages of some kind? Meteor / NPM? What would the advantages be for us?
    -> I guess an argument against Meteor packages would be that most IDEs don’t fully grasp the concept.

  • Are there examples of “Business Apps” or SAAS or something of that Caliber where we can have a look in how it’s structured?

  • Are there common “patterns” of filesystem layout you would recommend?

  • Any tipps / tricks / hacks to make working with all of these imports easier?


Sooo here I come to an end, but I think I wrote a lot already, so I’ll leave it here.

Thank you for hanging on if you did.

PLEASE if you have any idea or reaction, be it as short or incomplete a thought as it might be, just hit reply and put it out there.

We’re not looking for perfect here, we’re just looking to get some good ideas about the apps and the kind of structures being used successful “in the wild”.

I think I read most related things in the guide etc, but there weren’t really any examples of “bigger” kinds of apps there.

To round it all out, I just found / remembered this: Reusable Modules | Meteor Tutorials but that’s pretty basic too I feel

(And recommending git submodules to boot! Shudder :slight_smile: )

But yes they mean well. So, any more suggestions? What could a Meteor Enterprise (on a budget) Architecture look like in 2021?

I think this is an interested topic and hope for a lively discussion,

Thank you and all the best

Hi,

RadGrad is also a “mid-sized app” (around 30 KLOC of Typescript). We have a section of our developer guide which goes through the structure of the codebase starting at https://www.radgrad.org/docs/developers/design/overview. Keep clicking “next” at the bottom of each page.

We are now at a point where we are realizing that future extension of the system will be best supported via a kind of “plugin” system. The basic idea will be a a subdirectory of /imports called “plugins/”. Inside that directory will be a set of directories, one per plugin, (“foo/”, “bar/”, “baz/”). Plugin Foo’s directory will have an internal api/ startup/, and ui/ directories as necessary to implement its behavior).

Then, in the settings file, we will have a plugins area where we can specify which plugins we want to run at any given time.

We want to move toward this architecture because:

  • Future applications of RadGrad will have needs for different extensions of the “base” functionality.
  • We want the ability to implement “experimental” extensions to test new features and be able to easily extract them from the code base. Right now, it’s hard to remove things from the “core” of the system.

Good luck. I don’t think there’s a single correct answer to your question. I will be interested to hear what others have come up with.

Philip

1 Like

Hi Philip,

thanks for answering so quickly.

I haven’t read through your entire documentation yet, but I’ll do that ASAP.

Then I’ll get back to you here :slight_smile:

It’ll take a bit because your documentation looks like a treasure trove of wisdom to me :slight_smile: