Use Functional Programming to Simplify Your Meteor Code

That’s easily comprehensible. Most of the syntactic sugar in the official documentation I believe to be written by a mathematician on acid.

Taken individually most of the concepts such as how it views immutability are easily to grok. After all, it’s how we learned in high school algebra. A = 2. B = A + 1. Tuples = structs, easy to understand.

However once you throw it all together into a soup pot, then things like 6502 assembly, java or C# bytecode becomes more appealing. I just can’t see myself ever being able to scan source and immediately grok the intention.

Perhaps I’m just too old to not have a clear mnemonic representation. I wish that were not the case, as Erlang is one hella powerful VM. Perhaps someone can come up with an Erlang/Elixr transpiler for a C family language.

Was looking into the issue of immutability in reference to Redux / Immutable.js. Looks like it uses a binary tree internally so no nasty array copies. It also uses object reference not value for comparisons. That makes it memory efficient and performant.

The above video gives a nice summation.

You will obviously still kill the gc if you throw a million changes into an immutable store. So while I wouldn’t use it to hold state for a game, your average website I doubt you would notice anything without object pooling.

2 Likes

Pipes are available in Livescript, with React, I would like to suggest you to take a look at Elementary I think it looks like RedScript without Elixir syntax. Wish to see RedScript do better .

class todo-store   
  ->
    @bindActions todo-actions
    @todos =
      'id1':
        id: 'id1'
        complete: false
        text: 'Configure Jest'
      'id2':
        id: 'id2'
        complete: false
        text: 'Add unit tests'
      'id3':
        id: 'id3'
        complete: false
        text: 'Achieve 100% code coverage'

  update: (id, updates) ->
    if @todos[id] and updates then
      @todos[id] = merge(@todos[id], updates)

  update-all: (updates) ->
    self = @
    Object.keys self.todos
      |> ( .map (id) -> self.update(id, updates) )

{ section, input, label, ul } = $

module.exports = $.component do
  prop-types:
    all-todos: types.object.is-required
    are-all-complete: types.bool.is-required

  render: ->
    all-todos = @props.all-todos
    keys = Object.keys all-todos

    return null if not keys.length

    section class-name: 'main',
      input do
        class-name: 'toggle-all'
        type: 'checkbox'
        on-change: @handle-input-change
        checked: 'checked' if @props.are-all-complete
      label html-for: 'toggle-all',
        'Mark all as complete'
      ul class-name: 'todo-list',
        keys.map (key) ->
          todo = all-todos[key]
          $ todo-item, { key, todo }

  handle-input-change: ->
    todo-actions.toggle-complete-all!

Ah ok that makes sense… I imagine that would be necessary for the time traveling debugger in Redux too :smiley:

Time traveling is just no fun if it includes time dilation.

That being said, I would gladly sacrifice a few years off everyone else’s life to jump forward in time to the point where I get to see your youtube React/Redux tutorial.

1 Like

Speaking of functional programming… i’m wondering if anyone would be interested in me open sourcing a little library I wrote for a RESTful API ? I’m not sure how useful this would be.

Personally with a bit more polish I think having this on a headless Meteor server is more productive than an express API (no callbacks!)

I started out with just the Picker package and ran into maintenance problems so I made this.

@awatson1978 i’d love your opinion if you have a few spare mins!

It’s written in a functional style and heavily utilizes ES6 to create a better developer API. It was evolved from the simple:json-routes package and into something that can be used to easily maintain a large API using Meteor as the base. I’m currently using it to power a set of native and ReactNative apps.

The main features are:

  • Functional approach with ES6
  • Automatically catch errors and render proper JSON error
  • Easy to use router
  • Skinny controller to ease testing and to isolate side effects
  • Connection transformation in controller with chaining functions (like change header data)
  • Changeset integration with SimpleSchema
  • Optional JSON view layer for response data munging
  • CLI generator (with passing tests) if it would get open-sourced

Also some non-router things that I used are (and perhaps a sep package):

  • Repository pattern to abstract away database & throw errors if $ is prepended, eg. $insert
  • Model layer (works with changeset to validate insert/updates)

Here’s the gist of how it works:

Router

const {scope, get, post, put, delete_} = Restful;

applyMiddleware(
  acceptJSON(),
  authenticateJWT({except: 'post.user'}),
);

scope('/v1', () => {
  // resource saves boilerplate using conventions
  resources('/users', UserController, {except: 'delete'})
  resources('/comments', CommentController)
  resources('/posts', PostController)

  // or manually add routes
  get('/items/', ItemController, 'index')
  post('/items', ItemController, 'create')
  get('/items/:id', ItemController, 'show')
  delete_('/items/:id', ItemController, 'delete')
  put('/items/:id', ItemController, 'update')
})

Controller

const {sendJson, putStatus} = Restful;
const {Item} from Models;
const {ItemView} from Views;

ItemController = {
  debug: true, // prints params and return value
  scrubParams: [...], // pending, will reject all contrl reqs based on rules

  index(conn, params) {
    // Repository pattern abstracts persistence dep. and removes boilerplate
    const items = Repo.all(Item);
    // allows a json 'view' to munge data outside of controller (optional)
    return renderView(conn, ItemView, items);
    //or just directly send:  `sendJson(conn, items)`
  },

  create(conn, params) {
    // you could use Repo.$insert to remove error handling
    // boilerplate, see #show & #delete actions
    const [status, result] = Repo.insert(params);
    if (status === 'ok') {
      return sendJson(conn, result);
    } else {
      return sendJson(conn, 500, {error: result})
    }
  },

  update(conn, params) {
    const [status, result] = Repo.update(Item, params.id, params);
    if (status === 'ok') {
      return sendJson(conn, result);
    } else {
      return sendJson(conn, 500, {error: result});
    }
  },

  show(conn, {id}) {
    // $get will throw and return json error if non-existent
    // or get will return undefined if non-existent
    const item = Repo.$get(Item, id);
    return sendJson(conn, item);
  },

  delete(conn, {id}) {
    Repo.$delete(Item, id);
    return sendJson(conn, {});
  }
};

Repo layer (Optional)

Too much for this post, see gist:
https://gist.github.com/AdamBrodzinski/9e667d14e980df7c0fbb

JSON View layer (Optional)

const {renderMany, renderOne} = Restful;

// you can munge data as needed without concerning the database
// this makes testing the transformation much easier
// ex in controller:
//
//     return renderView(conn, ReminderView, reminders);
//
ReminderView = {
  // can use conn to determine auth/user/params/etc..
  // munge ex: add 'data: {}' wrapper, convert time to unix timestamp, etc..
  render(conn, reminder) {
    return {
      _id: reminder._id,
      scheduledAt: reminder.scheduledAt,
      userId: reminder.userId,
      userName: reminder.userName,
      userNumber: reminder.userNumber,
      message: reminder.message,
      recurFrequency: reminder.recurFrequency,
      active: reminder.active,
      recipients: reminder.recipients,
      sentLog: reminder.sentLog
    };
  }
}
3 Likes

Looks neat! Can we change the repo per route, to say alter get to accept options or match some predefined criteria, eg deleted: false etc

Thanks!

In the examples above the first param is the Model and then the collection is grabbed from that and I just noticed the repo gist is using the string collection as the first param instead.

I haven’t settled on the custom query yet but my thought is to go in two directions depending on the repetition. First you could just add a custom method to the repo like getViewerFollower and that would dry up a query.

Alternatively you could pass the query into the repo. Something like this would be nice:

query = Query.findOne(...)
query2 = Query.find(..., {fields: {_id: 1}})

Repo.get(Post, query)

query3 = Query.update({ active: false, ownerId: uid }, {multi: true})

Repo.update(Post, query3)

Then under the covers the Query.findOne just collects the arguments and passes them into the real findOne with the driver.

This kind of breaks the separation with what kind of persistence is used but seems pragmatic to me because a single custom query will need to be changed in one place regardless. Multi uses can add a custom method to stay dry.

Using this pattern has been super helpful when switching databases… it was only a few lines to switch to another DB and it still follows the database anywhere as this is just a thin wrapper around the database part :smiley:

1 Like

Hm, using a custom method would probably better in terms of security since it would not be as revealing. Also, less external input to sanitize.

1 Like

Yea that would be the best in most cases I think. Also if it’s on the server only there isn’t a security issue. On the client… ideally we’d be using ES6 modules so there wouldn’t be any globals for people to poke into… you’d have to breakpoint inside a function to get to the private method (even then webpack obfuscates the import module name to something like __Repo3).

I haven’t used the Repo pattern exactly on the client yet but have built a similar non-generic version of wrapping find, insert, etc… functions for my model Meteor methods (mainly for ease of testing). The nice thing about the Repo pattern is that (here at least) you pass in the model so it’s super generic.

Yep, I agree that a repo pattern becomes quite beneficial once app begins growing. But then you should also consider data access strategies such as paging and sorting if the data is likely to grow. You won’t want an endpoint crippling the app just because it returns thousands of items.

I suggest if you want to do functional programming in meteor ( I use it a lot! ) try Ramda. It has a package for meteor that is maintained by the authors, and its designed in a much nicer way for functional programming, where things are curried by default ( and easily allows you to curry an existing function), and the parameter order of all the functions are much better for partial application.

3 Likes

@keithnicholas cool, I need to try this out! I’ve been hesitant to do so because I use Elm and Elixir on the weekends and both of them have the data first instead of the normal way (though Elixir tends to favor composition instead of currying while Elm still does currying but backwards).

Do you find that it has extra functions that are more handy instead of lo-dash?

And is currying just to reduce the one(two(three(four(...)))) nesting issue? I’m kind of stuck in the mindset of composing with Elixir/Elm/F# type |> pipes.

Also can you give me a good example of currying that is useful? I always read about the add and add10 examples and can never seem to think… ‘oh yea I should curry this’… though i’m sure there are use cases i’m missing.

currying is more about partial application. Which is about creating building blocks.

the simplest one I use in a meteor conxtext is

idsFor = R.pluck("_id")

when doing “fizzbuzz” with Ramda I wrote

divisibleBy = R.curry((b) => R.compose(R.equals(0), R.flip(R.modulo)(b)))

which partially applys several functions in a chain/pipe, While this example isn’t too exciting, this process is where it gets more useful… setting up chains / pipes with various partially applied functions that will transform the incoming values.

I then partially applied divisibleBy(3) and divisibleBy(5)

I could of broke out a

isZero = R.equals(0)
divisibleBy = R.curry((b) => R.compose(isZero, R.flip(R.modulo)(b)))
1 Like

Ahhhh ok this makes more sense. Thanks!

Thanks for pointing out Ramda, never recognized before but it looks quite nice.

I guess you use the “original” package https://atmospherejs.com/ramda/ramda?

yep, that’s the package I use.

It introduces it as R into the project, which seems the norm.

Though there seems to be a trends for people to introduce the function names into global scope… haven’t quite made my mind up about that yet :slight_smile:

Hi, hi @SkinnyGeek1010. Just wrapped up another project, and wanted to give this some proper thought before replying. Sorry for the delay.

So… curiously interesting. Wouldn’t have necessarily occurred to me to write the REST API in a functional paradigm; but I don’t see why this couldn’t work splendidly. For high volume throughput, I suspect this could work particularly well, without worrying about the garbage collector keeping up. So, a few thoughts and a basic code review:

Namespacing

Functional programming doesn’t mean we need to ditch namespaces. My hunch is that the general approach will be made more clear by introducing a bit of namespace. I’m just making up some names up, but it could look something like this:

RestApi.scope('/v1', function() {
  // resource saves boilerplate using conventions
  this.resources('/users', UserController, {except: 'delete'})
  this.resources('/comments', CommentController)
  this.resources('/posts', PostController)

  // or manually add routes
  Endpoint.get('/items/', ItemController, 'index')
  Endpoint.post('/items', ItemController, 'create')
  Endpoint.get('/items/:id', ItemController, 'show')
  Endpoint.delete_('/items/:id', ItemController, 'delete')
  Endpoint.put('/items/:id', ItemController, 'update')
})

Middleware

Very intriguing about the middleware so far. Any chance you could take a look at oauth2orize and make any recommendations on how to get started with an integration? Not asking for a complete implementation. But how difficult would it be to register a grant and some endpoints with your approach? Something like this?

var server = oauth2orize.createServer();

RestApi.applyMiddleware(
  acceptJSON(),
  authenticateJWT({except: 'post.user'}),

  server.grant(oauth2orize.grant.code(function(client, redirectURI, user, ares, done) {
  var code = utils.uid(16);
  var ac = new AuthorizationCode(code, client.id, redirectURI, user.id, ares.scope);
  ac.save(function(err) {
    if (err) { return done(err); }
    return done(null, code);
  });
}));
);

Isomorphism & Model,View,Controller

So… this is a pretty standard server-side MVC approach. We’re in agreement with the controller being a controller; but the Models and Views will confuse some people accustomed to client-side MVC paradigms.

const {sendJson, putStatus} = Restful;
const {Item} from Models;
const {ItemView} from Views;

ItemController = { ... }

So, the question is whether this ought to be isomorphic. Should it be available on the client? If so, it’s going to run into the client-side MVC paradigm, which involves ViewPort, device media states, the CSS subsystem, etc. etc. View layer functionality that this paradigm doesn’t necessary take into account. (Not to mention the HTML subsystem and the Document Object Model.) If it’s only going to be server-side, then… yeah, maybe it can make due as-is.

Personally, I’d recommend just scrubbing any reference to Model, View, and Controller, since those terms have a half-dozen different interpretations and are therefore somewhat meaningless and a general cause of confusion.

Other Thoughts on Functional Programming

So, are you familiar with monads yets? The biggest return-on-investment with adding functional programming to javascript apps that we’ve found is when there are clear computational pipelines that need to be created. Generally, we’ve found them with data visualizations and validation testing. We use long method chain pipelines in both our D3 graphs and our Nightwatch test scripts, and they create magic at runtime.

So what’s the computational pipeline that’s involved with the Rest interface? That’s the $64,000 question here. I’m not necessarily suggesting that you create a method chaining pattern. But I also kind of am. It’s sort of the logic culmination of this approach. So what would that look like? What does a RestApi method chain look like? What syntactical verbs are used to create a coherent API syntax?

I’m just tossing some ideas out here, but I could imagine a method chaining pattern that connects with the Mongo collections could be extremely useful:

// example 1
Endpoint.route('/items/:id').collection('Items').get();

// example 2
Endpoint.route('/item/:itemId').collection('Items').findOne({_id:itemId).get();

// example 3
Endpoint.route('/item/:itemId').collection('Items').get(function(){
  return Items.findOne({_id: this.params.itemId});
});

// example 4
Endpoint.route('/item/mapreduce').collection('Items').find().fetch().get();

// example 5
Endpoint.route('/item/mapreduce').collection('Items').find().map().reduce().get();

Just some thoughts…

4 Likes

@awatson1978 Hey no worries and thanks for all the input!! With all the positive feedback i’m going to open source this soon. I’ll also be dogfooding it once I can clean up my proprietary version of this and switch. Also apologies for the long reply… so many things to digest :slight_smile: (this really helps me externalize decisions too)

I think this might even be a viable setup for a headless API only setup just because of the advantages of fibers (that’s what i’m doing… no subscriptions or DDP yet).

So… curiously interesting. Wouldn’t have necessarily occurred to me to write the REST API in a functional paradigm; but I don’t see why this couldn’t work splendidly

Same here… I didn’t think about that either until I started tinkering in other languages and their frameworks. One of my favorites is Phoenix which is where a lot of the inspiration comes from with this. One of the perks of tinkering in vary different langs is that they tend to creep back into your JS for the better :smiley:

Functional programming doesn’t mean we need to ditch namespaces. My hunch is that the general approach will be made more clear by introducing a bit of namespace

Whoops I forgot to add some of the de-structuring. Yep, totally agree! Ideally we’d have ES6 modules and could import those. I’ve been mimicking them in my non-webpack apps with a module namespace. Using Elixir modules is very much like this with the exception of how it imports them. I now tend to have a module per file and do this:

ApiUtils = {
  parseResponse(foo, bar) {
    return ...
  },
};

function myPrivateFunc(foo, bar) {
  return 'baz'
}

// then use them with or w/o the namespace depending on what the context is

const {parseResponse} = ApiUtils;
//import {parseResponse} from './utils/api';

parseResponse('blah');
ApiUtils.parseResponse('blah');

So currently most the global looking things are de-structured from the Restful (working name) namespace, though I forgot to add some to the (const {scope, get} = Restful;).
For example without the ES6 de-structuring it would look like this:

var authenticateJWT = YourMiddlewares.authenticateJWT;
var ItemController = YourContNamespace.ItemController;

Restful.applyMiddleware(
  Restful.acceptJSON(),
  authenticateJWT({except: 'post.user'}),
);

Restful.scope('/v1', () => {
  Restful.resources('/users', UserController, {except: 'delete'})
  Restful.get('/items/', ItemController, 'index')
  Restful.post('/items', ItemController, 'create')
})

Any chance you could take a look at oauth2orize and make any recommendations on how to get started with an integration?

Yep! I will look into this. So i’m just using connect under the hood so any connect middleware will plug right in (no pun intended). Then you could use the awesome Passport lib and oauth2orize will hook into that very easily. Speaking of i’m using the simple rest package to utilize basic http auth for logging into an existing meteor account :slight_smile:

So… this is a pretty standard server-side MVC approach. We’re in agreement with the controller being a controller; but the Models and Views will confuse some people accustomed to client-side MVC paradigms.

Yep I was afraid this would be kind of vague. So the model namespace would be user provided (Models.Post) and the model (and Repo) wouldn’t be in the same package as they aren’t really required. Anyhow, the repo and model are based off of the functional Ecto.Model. It’s basically just an object literal with some functions and config stuff to utilize simple-schema for validation. The Repo package would recognize the format and use it to auto validate and throw errors. I’m still trying to iron these out.

The views are tricky. Also this namespace is user defined so it’s more less by convention. So the thought was that they could choose between a JSON, XML, Text, or even an HTML view (with ES6 strings… something bare bones). Currently i’m only using JSON with my setup. I agree the name is awkward. This is also inspired from Phoenix but i’m not sure what else to call it (ideas?). This would have to be super clear in the docs and opt in of course. Here’s a brief overview of them (http://blog.rokkincat.com/json-views-in-phoenix/). The main thing I like is abstracting out the things like converting unix timestamps and other things that can be messy to test in a controller.

So, the question is whether this ought to be isomorphic. Should it be available on the client?

I was thinking about this too… My gut is to make it not crash on the client by checking for isServer under the hood so it can be loaded in a ‘both’ folder. However I can’t really see much overlap on the client. Keeping it focused (server only) is the plan so far.

So, are you familiar with monads yet?

Not well enough. My thoughts from a Haskell point of view is that a monad is just an acedemic loophole to call something thats a side effect ‘pure’… which confuses me because at the end of the day the monad is not going to undo the log/db insert if you roll back/forward in a time traveling debugger.

The biggest return-on-investment with adding functional programming to javascript apps that we’ve found is when there are clear computational pipelines that need to be created.

Yep, totally! This is what draws me into the lodash/underscore ‘chain’ functionality. Another perk i’ve noticed is that when you explicitly pass in what you’re using it’s very clear what it does/does-not do.

I’m just tossing some ideas out here, but I could imagine a method chaining pattern that connects with the Mongo collections could be extremely useful

This is super cool! One thing that might get messy though is if you have to do things like send out emails, SMS, hit a webhook, etc… then it starts to look like using picker (just a bunch of stuff in one callback). Perhaps something similar for an opt in thing where you don’t really need a controller (like a findOne or find all). Hmmmm :slight_smile:

does this count as functional? :smiley: (!warning coffeescript)
mapping stock and delivery subdocument from product to related eshop
would be changed to low level publication soon
and than on client applied filters and sorts :smiley:

Meteor.methods
  'eshopDepots': (masterProduct_id) ->
    
    relatedProducts = EshopProducts.find({
      masterProduct_id: masterProduct_id
    },{
      fields:
        eshop_id: 1
        item_id: 1
        stock: 1
        delivery: 1
    }).fetch()

    shopList = _.map(relatedProducts, (item) ->
      return item.eshop_id
    )

    relatedShops = Eshops.find({
      _id:
        $in:
          shopList
    },{
      fields:
        url: 1
        depots: 1
    }).fetch()

    returnShops = _.map(relatedShops, (eshop) ->
      _.extend(
        {}
        eshop
        _.pick(
          _.find(
            relatedProducts
            (item) ->
              return item.eshop_id is eshop._id
            )
          'stock'
          'delivery'
        )
      )
    )

    _.each(returnShops, (eshop) ->
      console.log JSON.stringify(eshop)
    )