Use Functional Programming to Simplify Your Meteor Code

Planet Earth
41

@awatson1978 Hey no worries and thanks for all the input!! With all the positive feedback i’m going to open source this soon. I’ll also be dogfooding it once I can clean up my proprietary version of this and switch. Also apologies for the long reply… so many things to digest :slight_smile: (this really helps me externalize decisions too)

I think this might even be a viable setup for a headless API only setup just because of the advantages of fibers (that’s what i’m doing… no subscriptions or DDP yet).

So… curiously interesting. Wouldn’t have necessarily occurred to me to write the REST API in a functional paradigm; but I don’t see why this couldn’t work splendidly

Same here… I didn’t think about that either until I started tinkering in other languages and their frameworks. One of my favorites is Phoenix which is where a lot of the inspiration comes from with this. One of the perks of tinkering in vary different langs is that they tend to creep back into your JS for the better :smiley:

Functional programming doesn’t mean we need to ditch namespaces. My hunch is that the general approach will be made more clear by introducing a bit of namespace

Whoops I forgot to add some of the de-structuring. Yep, totally agree! Ideally we’d have ES6 modules and could import those. I’ve been mimicking them in my non-webpack apps with a module namespace. Using Elixir modules is very much like this with the exception of how it imports them. I now tend to have a module per file and do this:

ApiUtils = {
  parseResponse(foo, bar) {
    return ...
  },
};

function myPrivateFunc(foo, bar) {
  return 'baz'
}

// then use them with or w/o the namespace depending on what the context is

const {parseResponse} = ApiUtils;
//import {parseResponse} from './utils/api';

parseResponse('blah');
ApiUtils.parseResponse('blah');

So currently most the global looking things are de-structured from the Restful (working name) namespace, though I forgot to add some to the (const {scope, get} = Restful;).
For example without the ES6 de-structuring it would look like this:

var authenticateJWT = YourMiddlewares.authenticateJWT;
var ItemController = YourContNamespace.ItemController;

Restful.applyMiddleware(
  Restful.acceptJSON(),
  authenticateJWT({except: 'post.user'}),
);

Restful.scope('/v1', () => {
  Restful.resources('/users', UserController, {except: 'delete'})
  Restful.get('/items/', ItemController, 'index')
  Restful.post('/items', ItemController, 'create')
})

Any chance you could take a look at oauth2orize and make any recommendations on how to get started with an integration?

Yep! I will look into this. So i’m just using connect under the hood so any connect middleware will plug right in (no pun intended). Then you could use the awesome Passport lib and oauth2orize will hook into that very easily. Speaking of i’m using the simple rest package to utilize basic http auth for logging into an existing meteor account :slight_smile:

So… this is a pretty standard server-side MVC approach. We’re in agreement with the controller being a controller; but the Models and Views will confuse some people accustomed to client-side MVC paradigms.

Yep I was afraid this would be kind of vague. So the model namespace would be user provided (Models.Post) and the model (and Repo) wouldn’t be in the same package as they aren’t really required. Anyhow, the repo and model are based off of the functional Ecto.Model. It’s basically just an object literal with some functions and config stuff to utilize simple-schema for validation. The Repo package would recognize the format and use it to auto validate and throw errors. I’m still trying to iron these out.

The views are tricky. Also this namespace is user defined so it’s more less by convention. So the thought was that they could choose between a JSON, XML, Text, or even an HTML view (with ES6 strings… something bare bones). Currently i’m only using JSON with my setup. I agree the name is awkward. This is also inspired from Phoenix but i’m not sure what else to call it (ideas?). This would have to be super clear in the docs and opt in of course. Here’s a brief overview of them (http://blog.rokkincat.com/json-views-in-phoenix/). The main thing I like is abstracting out the things like converting unix timestamps and other things that can be messy to test in a controller.

So, the question is whether this ought to be isomorphic. Should it be available on the client?

I was thinking about this too… My gut is to make it not crash on the client by checking for isServer under the hood so it can be loaded in a ‘both’ folder. However I can’t really see much overlap on the client. Keeping it focused (server only) is the plan so far.

So, are you familiar with monads yet?

Not well enough. My thoughts from a Haskell point of view is that a monad is just an acedemic loophole to call something thats a side effect ‘pure’… which confuses me because at the end of the day the monad is not going to undo the log/db insert if you roll back/forward in a time traveling debugger.

The biggest return-on-investment with adding functional programming to javascript apps that we’ve found is when there are clear computational pipelines that need to be created.

Yep, totally! This is what draws me into the lodash/underscore ‘chain’ functionality. Another perk i’ve noticed is that when you explicitly pass in what you’re using it’s very clear what it does/does-not do.

I’m just tossing some ideas out here, but I could imagine a method chaining pattern that connects with the Mongo collections could be extremely useful

This is super cool! One thing that might get messy though is if you have to do things like send out emails, SMS, hit a webhook, etc… then it starts to look like using picker (just a bunch of stuff in one callback). Perhaps something similar for an opt in thing where you don’t really need a controller (like a findOne or find all). Hmmmm :slight_smile:

42

does this count as functional? :smiley: (!warning coffeescript)
mapping stock and delivery subdocument from product to related eshop
would be changed to low level publication soon
and than on client applied filters and sorts :smiley:

Meteor.methods
  'eshopDepots': (masterProduct_id) ->
    
    relatedProducts = EshopProducts.find({
      masterProduct_id: masterProduct_id
    },{
      fields:
        eshop_id: 1
        item_id: 1
        stock: 1
        delivery: 1
    }).fetch()

    shopList = _.map(relatedProducts, (item) ->
      return item.eshop_id
    )

    relatedShops = Eshops.find({
      _id:
        $in:
          shopList
    },{
      fields:
        url: 1
        depots: 1
    }).fetch()

    returnShops = _.map(relatedShops, (eshop) ->
      _.extend(
        {}
        eshop
        _.pick(
          _.find(
            relatedProducts
            (item) ->
              return item.eshop_id is eshop._id
            )
          'stock'
          'delivery'
        )
      )
    )

    _.each(returnShops, (eshop) ->
      console.log JSON.stringify(eshop)
    )
43

I think it definitely uses some functional concepts!

So ‘functional’ means something different to a lot of people. Also some functions are still functional but more ‘pure’ than others (like side effects from the database). If it’s ‘pure’ it will always return the same result when called.

In general it (correct me if i’m wrong here!) uses principals like, only using the data that’s passed into it’s arguments, using functions as arguments, always returning a value, not producing side effects and in general they do one thing and ideally do it in a re-usable way.

For instance I plucked (no pun intended) one of the bits toward the bottom and used a more ‘functional approach’ (using es6)… notice these are being passed in the data they need in the params… this makes it easy to read the code without searching for an out of scope thing. Also i’m not positive this works as I didn’t double check the underscore docs and it wasnt super clear what the end goal was for some of the stuff.

function getShops(relatedShops, relatedProducts) {
  return _.map(relatedShops, findShopStuff)
}

function findShopStuff(eshop, relatedProducts) {
  // get an object of whitelisted keys
  const things = .chain(relatedProducts)
    .find(item -> item.eshop_id == eshop._id)
    .pick('stock', 'delivery')

  return {...eshop, ...things};
}

PS… this is what always drove me crazy about CoffeeScript vs Python and Ruby :laughing: It always takes me 10x longer to parse the nested hidden objects, where as in Ruby omitted braces are usually one level deep like this
do_something foo: 2, bar: 3


    relatedShops = Eshops.find({
      _id:
        $in:
          shopList
    },{
44

yes, ES6 …something can help with merging, still I would do something like

return _.merge(...eshop, things)

or I am missing some ES6 thing where it would somehow merge it :smiley:
still no side effects ES6 way, something as I did with
_merge({}, eshop, pick_results)
I am still not used to .chain yet, but with that coffee indentation, it is OK for me to read it without actual chaining

And I tried to not mutate any input, seems it was accomplished

1 Like
45

Yep so basically doing {...old, newKey:1} or {...old, ...new} is creating a new object literal and copying the keys into the object literal around it. But yea +1 for using the proceeding {} so that it doesn’t mutate!

So the babel traspiling will do this (and _extends would only be defined once per build):

const old = {foo: 1, bar: 2};

const foo = {...foo, baz: 3}

// after

var _extends = Object.assign || (/* babel polyfill */)

var old = { foo: 1, bar: 2 };

const foo = _extends({}, foo, { baz: 3 });
46

I’ll tell you what… if you could get passport and basic oauth2orize implementation working (even if it’s only a single endpoint), you’d have a bunch of people who would be super excited and would be lining up to start using it. OAuth2 server support has been an often-requested feature; and it would be a major contribution to the ecosystem.

1 Like
47

Interesting! Yea i’ll def. add support for that :thumbsup:

As a matter of fact that’s why I went the JWT route because I was having issues getting a custom OAuth server to work with IronRouter/Picker and it was too much effort at the time.

My thought to providing a drop in oauth2orize implementation would be to have a pre-filled out module/file that can easily be configured and then the module can be called, as opposed to a meteor-package that’s hard to customize (and update). Howerver it shouldn’t be harder than typing in restful gen oauth --strategy foo and pasting in the middleware init func acceptOath to insert the oauth boilerplate module

What’s bugged me about Meteor auth is that it’s not easy to reach in do something different (which is what makes it so easy to use for those 80% use cases I guess) so I would want to make it easy to change.

Meteor’s lack of a pluggable CLI makes me want to make a YO generator (or as a core of an NPM CLI tool) so that one could easily scaffold out a resource or add basic oauth boilerplate. In general scaffolds only take you so far but having an example of a ‘Comment’ resource endpoint with passing tests is a win IMHO.
Thoughts on scaffold generators?

My general gameplan is to complete this by the end of this coming weekend:

  • router
  • controller
  • controller connection helpers (like change status)
  • view layer (json for now)
  • repo (sep package)
  • model support for simple-schema (packaged with repo)
  • changeset for validation handling (tied to repo and model)
  • example app

Also dogfood it with my own app by using the package instead of my proprietary version

Then start on Oauth the following week-ish

1 Like
48

how you handle conditions while chaining?

I have few filter dropdowns which have “clear” value when they should just pass trough, but also some filtering states.
Is there any way to test it and skip computation in the “clear” state?
I can of course just _.map and test it inside it, return source object if it is in “clear”, but that would test the condition for every item in list.
I would prefer some construction where it would test just once and if it is in “clear” it would just skip whole _.map and continue on next step in chain.

1 Like
49

So there’s different ways to approach this but since chaining (or piping) is so common in Elixir this is how I tend to do it like below. Basically you can use an array as a container to pass down through each step. One element could be the actual data and another could be the sate. In other languages this might be a Tuple data structure (a fixed len array basically). We could also use an object if having the name is helpful. ES6 de-structuring makes things more readable too!

// using underscore

var inputData = [{type: 'admin', name: 'Bob'}, {}];

_.chain([inputData, true])
.foo()
.bar()
.thru(uppercaseIfAdmin)
.value()


function uppercaseIfValid([users, isValid]) {
  if (isValid) {
    const newUsers = users.map(u => ({type: u.type}));
    return [newUsers, isValid]
  }
  // skip and pass through
  return [users, isValid]
}
50

I ended up with this, now I can chain that filter

    inStockAvailability = (eshop) ->
      console.log 'filter in_stock'
      return eshop.stock?.stock_quantity? and eshop.stock.stock_quantity isnt 0

    inStoreAvailability = (eshop) ->
      console.log 'filter in_store'
      return eshop.stock?.depot? and lodash.findKey(eshop.stock.depot, (depotKey) ->
        return depotKey.stock_quantity? and depotKey.stock_quantity isnt 0
      )

    pickAvailability = ->
      console.log 'picking'
      switch Template.instance().currentAvailability.get()._id
        when 'in_stock' then return inStockAvailability
        when 'in_store' then return inStoreAvailability
        else return  -> return true

    availabilityFeed = _.filter(shopFeed, pickAvailability())

that ‘picking’ is logged just once when I change dropdown
and ‘filter *’ is logged every iteration
all working as I wanted for now :smiley:

strange that our underscore did not have _.findKey

1 Like
51

Yea meteor’s version of underscore is way old because of breaking changes. The newest lodash is better, faster, and has semantic versioning :slight_smile:

52

@awatson1978 Do you think that having nested namespaces would be difficult to work with? It would almost force one to use de-structuring.

My intent is to simulate ‘modules’ since we don’t have them in Meteor yet. It would also make it a bit easier for me when I wrap them up in a package.

For example:


Restful.Router.get('/posts', ...)
// instead of all in one namespace
Restful.get('/posts', ...)

// or ideally
const {get, post} = Restful.Router
get('/posts', ...)
post('/posts', ...)
53

Not at all. People are already accustomed to the nested Template namespacing; and what’s in Meteor is nothing compared to .Net.

Just, you know, make the namespacing meaningful. If Restful is the parent namespace, don’t make Router be the only child. Namespacing is just another form of refactoring, and so the Rule of Three applies (or Two, if you’re being aggressive about refactoring).

The times when I’ve seen namespacing be a problem tend to be when there’s a shared namespace that team members are suppose to attach objects to, and they a) have trouble loading, editing, publishing the shared object, or b) when there are different style guidelines for things like capitalization. Attaching collections to the Meteor namespace, aka Meteor.users is a good example.

1 Like
54

Thanks! My plans so far for module namespacing is:

Restful
- - Middleware (for meteor accounts auth, json, multi-part forms)
- - Router (funcs for setting up routes)
- - View (funcs for serializing JSON/XML/text/etc…)
- - Controller (helpers for transforming conn, like status code)

55

Just a heads up, I finished the Router and Controller part of the framework I mentioned earlier.

An unexpected client project popped up and I won’t be able to finish off the optional view/model/validation layers for a few more weeks but the meat of it is working!

The documentation is limited but it has a fairly simple API that can be learned by looking at the examples (not documented yet is that you need to use express body parser to parse JSON reqs).


cc @awatson1978 @serkandurusoy

1 Like
56

Nice work, looks especially clean. I was wondering if you have heard of simple:rest but then I noticed the github issue about it.

I’m fairly confident to say that simple:rest would be better for bootstrapping api projects while yours may be better for api-centric, structured projects which require flexibility.

57

Thanks!

Yea the connect core is heavily based off of simple:json-routes. It ends up being about 20-30 lines to bootstrap the actual router part.

I’m fairly confident to say that simple:rest would be better for bootstrapping api projects while yours may be better for api-centric, structured projects which require flexibility.

I’d agree. However bootstrapping an API quickly is only good for prototyping IMHO. You really want to have control over it and personally, using any quick one liner package in Meteor has cost me 10x more hours later on when it needs to be customized (hence this package I guess :laughing:)

simple:rest is great if you want a basic API with no configuration, and you don’t care about testing it so easily. It’s also great for APIs that only do basic crud. For example, If you need to send a user an email after a certain route gets hit than you’re going to have to do a lot of work.

simple:rest has lots of magic and restful has little to none (by design). Restful also focuses on maintainability and testability.

I’m using it one one microservice that is just an API, and also on two other headless Meteor servers that have webhooks. It could def. be used on a monolithic app with a client router and this as a server router too.

58

I’m using it one one microservice that is just an API

especially for a service oriented enterprise architecture where DDP and reactivity may not bring much value, one could build microservice api’s to compose/orchestrate them from a client facing app perhaps.

There’s lots of opportunity here, especially in the enterprise world. Nice!

1 Like
59

Yep! So for my use case specifically I had to choose to either user Node for an API for React Native or use Meteor and get the fibers goodness that makes things clean. The promises pollyfill makes this easier too.

The app I pulled this from is very similar but the package is more cleaned up. Soon i’ll migrate to the open source version so everyone can benefit from bugfixes/features.

1 Like
60

Please do let us know when you do a release :smile:

1 Like