Advices on refactoring my app and rethinking architecture

[billybobbonnet]

Hello all,

First post here, so first thing first: congratulations to all MDG devs and all contributors for the great work you did on Meteor.

I have been working on a Meteor app for 8 months or so. This is now quite a large app and I am very pleased with the result. I would like to prepare a full refactoring before I open the source code and include contributors. So far, I did things without overthinking it, so it is not shaped for long term production, especially in the current context.

I initially designed my app to migrate later to a package based architecture, such as what has been done with Telescope. Considering the recent announcements (Meteor packages are not a long term solution), I decided to review my plan. I would like to some insights on how to proceed, and what should come first.

My app is using iron-router, Blaze, and bootstrap for layout and styling. Due to deadlines constraints, I couldn’t write tests. I intend to distribute it in 3 different builds.

Here are the main points I understood about what I should ideally do for the long run:

• Split the code in parts, as I would have done with packages, but with npm
• Migrate from iron-router to flow-router in order to benefit from meteorhacks modules (e.g. sub manager) and the better performances it seems to have, not to mention it is the MDG advised choice now.
• Include a schema validation. I am thinking about astronomy.
• switch from Blaze to React and redesign all the UI (doh…)
• write tests
• My personnal addition: switch to material UI components using the NPM package (https://github.com/callemall/material-ui) with React or Materialize with Blaze.

The thing is that I don’t have time to make all these changes. I have looked into the just released Mantra architecture but learning React + Flow router + Astronomy + Material UI +Tests is too much to handle in my timing.

I would like to know what you consider as being the most important parts considering my objectives. As I see it, I absolutely must move to npm based architecture and split my app in sub-parts. It would be good to switch to Flow router asap. The schema validation along with the material UI forms would be a nice addition, requiring a lot of work. I never wrote tests (I confess) but I understand how critical it can be in a team sized project. I never used React either and the Mantra experience freaks me out a little when I consider the scale of changes required (ES6 update included).

I know this is an opinionated questions, but I am looking for opinions.

[pal]

i;d prioritise moving to flow router

Think on the Blaze to React videos at Discover Meteor blog, then decide on your view layer -
blaze/viewmodel; reaxt/mantra, angular2, vuejs…

astronomy etc - id wait on Apollo

[ffxsam]

Actually I don’t see an advantage to breaking your app into small npm packages. There were two benefits to doing that in Meteor that I can recall off the top of my head:

• Controlling load order
• Testability

But with Meteor 1.3, you have control over load order by way of ES6 import/export. I personally separate different parts of my app into feature folders:

features
├── Admin
│   ├── client
│   │   ├── components
│   │   │   ├── AddEditOrgDialog.js
│   │   │   ├── AddEditUserDialog.js
│   │   │   ├── OrgList.js
│   │   │   ├── OrgManager.js
│   │   │   ├── OrgManager.scss
│   │   │   ├── OrgMemberList.js
│   │   │   ├── OrgMemberList.scss
│   │   │   ├── OrgUserManager.js
│   │   │   ├── UserList.js
│   │   │   ├── UserManager.js
│   │   │   └── UserManager.scss
│   │   └── containers
│   │       ├── OrgManagerContainer.js
│   │       ├── OrgUserManagerContainer.js
│   │       └── UserManagerContainer.js
│   └── lib
│       ├── methods.js
│       ├── organisations.js
│       └── users.js
├── Authentication
│   └── client
│       ├── components
│       │   ├── AuthBox.js
│       │   ├── AuthBox.scss
│       │   ├── LoginForm.js
│       │   └── LoginForm.scss
│       └── containers
│           ├── AuthContainer.js
│           └── AuthContainer.scss
├── Organisations
└── Users

This is a trial though, and my first time structuring it this way. So I’ll see how it actually works out in practice.

IMO, sink most of your time and energy into React. It’s useful in other contexts outside of Meteor that will make you more employable. Not to mention React Native!

[billybobbonnet]

@pal Thank you for pointing out the Blaze to React videos. I didn’t knew it was out there and it is the first real case of migration I have seen. At least, it offers an order for things to do. I will definitely watch all these videos. Thanks. Prioritize the switch to flow router seems to be the good way to go, I can decide from there if I commit into this further.

@ffxsam One of the main benefits you didn’t mention is the capability to make different client builds (e.g. browser, mobile app1, mobile app2, etc.) depending on client roles, i.e. you add or remove local packages and build your mobile app afterwards.

Your app structure is interesting, and it isn’t a big leap from where I stand. For the moment, all my app features/modules/aspiring-packages have their own folder in clients, both, and server folders. The one exception is a “shared” client folder where I have all my components shared among features (low level components, e.g. a video component, display components who are used to display any low level component, custom components pickers, loaders, upload widgets, etc. ). I guess I will move components and pickers with the features they belong to, and attach the rest to a “core” feature available for every client.

The thing that really puzzles me is that ES6 import/exports can’t be done in the scope of a function. It means I cant do something like :

if (Users.isAdmin(Meteor.userId()){
import * from "admin";
}

I can’t then send a particular set of modules depending on my client role. imports folder allows a lazy loading of modules, but afaik, it still is a static and not data-driven way to load assets. How would you solve that?

[billybobbonnet]

I can’t then send a particular set of modules depending on my client role. imports
folder allows a lazy loading of modules, but afaik, it still is a
static and not data-driven way to load assets. How would you solve that?

About that previous question, if I use different versions of my imports/client/main.js file where I do the client imports, it should allow me to make different mobile app builds, true?

[dinos]

Maybe you want to also take a look at the proposed structure here and also the example of the todos app in that format:

[Feedback] 1.3 App Structure with Screens & Components: Get ready for A/B/Multivariate testing and continuous improvement

This is the odd idea to first publish an unfinished flow of consciousness to try to get some feedback before investing the time into a full blown medium article. Offer criticism, improvement, ideas and experience as you like. Or already use what ever you find useful now. But higher level stack planning is really falling under the table lately. So I update this post consecutively. Title: 1.3 App Structure with Screens & Components: Get ready for A/B/Multivariate testing and continuous improveme…

[billybobbonnet]

You did a very nice work on this. I didn’t knew about the gzip “factor” in the size evaluation. The rest was very instructive as well.

However, I am not sure I am connecting all the dots (apologies if I miss the obvious): how is using the router to dynamically resolve routes depending on a factor (random, roles, or whatever) prevent me from building an app with unnecessary modules?
Or (besides the files scoping by feature), is it a confirmation of my hypothesis above (1 file === 1 build) ? :

Mainly, the imports/ is now the full center of attention and provides only one single file per build target/entity for reference.

[pal]

@billybobbonnet

Thank you for pointing out the Blaze to React videos. I didn’t knew it was out there and it is the first real case of migration I have seen. At least, it offers an order for things to do. I will definitely watch all these videos. Thanks

Your welcome - but all thanks should go to SG for the videos

I’m keeping up with Apollo here: Apollo GraphQL Blog
Stunning project.

[ffxsam]

I would probably just do this:

import { somethingINeed } from 'admin';

if (Users.isAdmin(Meteor.userId()) {
  // use somethingINeed
}

It’s not the end of the world if you import something that is conditionally used. And I never use import * because it doesn’t help me keep organized as I don’t have a very clear list of what’s being used in this module. The only exception is when I’m importing Redux actions.

import * as actions from '/client/actions/auth';

I also don’t use the imports folder because I have enough levels of depth in my hierarchy as it is! Plus I can see that structure eventually going away in favor of 100% explicit imports.

[jammer]

Have you taken a look at Meteor’s v1.3 guide on app structure ?

I’m using this structure in a project at moment but I’ve replaced the ‘/imports/ui/*’ folder with the atomic design structure.

[billybobbonnet]

Have you taken a look at Meteor’s v1.3 guide on app structure ?

Yes I did.Both @dinos and @ffxsam suggestions are making more sense to me because they have a feature specific approach (mandatory imho in the case of a big app).

[ffxsam]

I find it works for how my brain works. I think, “I need to change that behavior in the billing system.” Boom, /features/Billing. And if there are shared components among different features, those just go in /client.

[billybobbonnet]

I tried to come up with an architecture suited for my needs and way of thinking. A few things about it:

• I wanted to have a kind of plug-and-play feature based architecture. I included then in every feature/module folder a client and server folders. Besides, each feature/module will have its own context.js file used as in Mantra to provide a reactive-dict object, but at a module level. The intent is to consider that one module matches one big feature/section, and one family of routes (or none, e.g. comments) and one or no collection. It means that most routes are defined at a module client folder level, including for the core module (e.g. “/” route)
• I don’t know how I will schedule things, but it is designed for flow router and react with materiallUI and FlexBox CSS/controls
• I added a schema folder for collections but I don’t plan to use it before Apollo
• Many ideas are taken from Mantra: stateless components and containers to pass data (I consider calling them controllers instead, considering they are), actions folder is renamed api, and tests folders are available for each layer
• once meteor transfer the imports folder behaviour to the rest of the app structure, I will rename it modules.
• that should eventually allow to give each feature/module its own git repo and npm package

Here is the (work in progress) folder structure:

    .
 .  ├── client //client loads the client folders of the modules and provides a client-wide context
    │   ├── main.js
    │   ├── modules.js
    │   └── context.js
    ├── imports
    │   ├── module1
    │   │   ├── client
    │   │   │   ├── api
    │   │   │   │   └── tests
    │   │   │   ├── components
    │   │   │   │   └── tests
    │   │   │   ├── containers
    │   │   │   │   └── tests
    │   │   │   ├── module1_index.js
    │   │   │   ├── lib
    │   │   │   ├── method_stubs
    │   │   │   │   └── tests
    │   │   │   ├── module1_routes.jsx
    │   │   │   └── styles
    │   │   ├── lib
    │   │   │   ├── collection
    │   │   │   ├── i18n
    │   │   │   └── schema
    │   │   ├── readme.md
    │   │   ├── server
    │   │   │   ├── module1_collections.js
    │   │   │   ├── module1_fixtures.js
    │   │   │   ├── lib
    │   │   │   ├── module1_methods.js
    │   │   │   └── module1_publications.js
    │   │   └── tests
    │   └── core
    │       ├── client
    │       │   ├── api
    │       │   │   └── tests
    │       │   ├── components
    │       │   │   └── tests
    │       │   ├── containers
    │       │   │   └── tests
    │       │   ├── core_index.js
    │       │   ├── lib
    │       │   ├── method_stubs
    │       │   │   └── tests
    │       │   ├── core_routes.jsx
    │       │   └── styles
    │       ├── lib
    │       │   ├── collection
    │       │   ├── i18n
    │       │   └── schema
    │       ├── readme.md
    │       ├── server
    │       │   ├── core_collection.js
    │       │   ├── lib
    │       │   ├── core_methods.js
    │       │   └── core_publications.js
    │       ├── startup
    │       ├── styles
    │       └── tests
    ├── both //both loads collections. It provides an app wide context.
    │   ├── context.js
    │   ├── main.js
    │   └── modules.js
    ├── package.json
    ├── public
    │   ├── fonts
    │   └── images
    └── server //server loads the server folders of the modules, and provide a server wide context
         ├── modules.js
         └── main.js

Can you see any show-stopper in what I have done so far with this folder structure? What do you think of the self-contained modular approach?

[billybobbonnet]

I realize now how overkill and useless it is to make a structure where server folder is contained in the module folder. My initial intent was to allow to make different server builds as well as different client builds using the modules.js files (in server, both and client folders) where I could activate or deactivate features/modules.

So yes, there is a big show stopper: it makes the whole structure unnecessarily complex for a bad reason (server builds). I never planned to have several servers build in the first place.

Bottom line, I am having an hard time figuring out the way to go. The Discover Meteor migration videos are advising a progressive transition. However, it looks like too much unnecessary work relatively to a from scratch refactoring, since I also plan to rewrite my UI using material design React components anyway.

[corvid]

With 1.3, it’s not terribly important to break your application down to packages.
I did that while refactoring before 1.2 simply because it was virtually impossible to control load order in any other way and it seemed to be the only feasible way to create multiple simple tests
with import, the only benefit of package-based architecture is the ability to publish it to atmosphere, and if your package isn’t public, there’s no real point.