Use Functional Programming to Simplify Your Meteor Code

If it were done in pure C, perhaps. Though I never found linked lists all that difficult to comprehend.

Generally object polling is implemented with higher level abstractions on top of the same concept. In Java it was usually ConcurrentLinkedQueue<>(). Your interface to the ugliness is merely a borrow and release method. Just had to be sure you reset the objects in the release method not the borrow to help tame even more side effects.

It merely forces you to be a better programmer. I prefered side effects and thousands of users over no side effects and a handful of users complaining on my forums about 30 second lockups.

From first hand experience with scaling apps on VMā€™s whenever I see the world ā€˜immutableā€™ this image pops into my head.

I made a subtle reference to the topic at the bottom of this thread. It seems most people using Meteor will horizontally scale long before the GC catches on fire.

As far as Erlang/Elixlr pattern matching, I looked into again after a few years from your reference to Phoenix. Still makes my eyes spin around in the sockets. Iā€™m not going to start with recursion instead of flow control. That makes me cry, and I have 3+ decades experience. :slight_smile:

I will never catch on with the ā€œCanā€™t I just meteor addā€ crowd.

2 Likes

@jacobin Yea I try not to use pattern matching if itā€™s the head spinning variety. Hereā€™s a trivial example of using it where itā€™s a little more clear (though an if/else would suffice here)

defmodule StringHelpers do
  def replace_chars("", _newChar) do
    ""   # don't do work if it's empty
  end

  def replace_chars(str, newChar) do
    String.replace(str, newChar)
  end
end

I would highly recommend picking up Dave Thomasā€™s Elixir book as a vehicle to learn functional concepts for JavaScript. Theyā€™re very transferable to JS with underscore and for better or worse takes you in a Unix pipeline mindset (composing small re-useable functions).

Hereā€™s an excerpt from the tagline that pretty much sums it upā€¦ pragmatic and non academic:

ā€¦You want to explore functional programming, but are put off by the academic feel (tell me about monads just one more time)ā€¦

http://www.amazon.com/Programming-Elixir-Functional-Concurrent-Pragmatic/dp/1937785580

2 Likes

Thatā€™s easily comprehensible. Most of the syntactic sugar in the official documentation I believe to be written by a mathematician on acid.

Taken individually most of the concepts such as how it views immutability are easily to grok. After all, itā€™s how we learned in high school algebra. A = 2. B = A + 1. Tuples = structs, easy to understand.

However once you throw it all together into a soup pot, then things like 6502 assembly, java or C# bytecode becomes more appealing. I just canā€™t see myself ever being able to scan source and immediately grok the intention.

Perhaps Iā€™m just too old to not have a clear mnemonic representation. I wish that were not the case, as Erlang is one hella powerful VM. Perhaps someone can come up with an Erlang/Elixr transpiler for a C family language.

Was looking into the issue of immutability in reference to Redux / Immutable.js. Looks like it uses a binary tree internally so no nasty array copies. It also uses object reference not value for comparisons. That makes it memory efficient and performant.

The above video gives a nice summation.

You will obviously still kill the gc if you throw a million changes into an immutable store. So while I wouldnā€™t use it to hold state for a game, your average website I doubt you would notice anything without object pooling.

2 Likes

Pipes are available in Livescript, with React, I would like to suggest you to take a look at Elementary I think it looks like RedScript without Elixir syntax. Wish to see RedScript do better .

class todo-store   
  ->
    @bindActions todo-actions
    @todos =
      'id1':
        id: 'id1'
        complete: false
        text: 'Configure Jest'
      'id2':
        id: 'id2'
        complete: false
        text: 'Add unit tests'
      'id3':
        id: 'id3'
        complete: false
        text: 'Achieve 100% code coverage'

  update: (id, updates) ->
    if @todos[id] and updates then
      @todos[id] = merge(@todos[id], updates)

  update-all: (updates) ->
    self = @
    Object.keys self.todos
      |> ( .map (id) -> self.update(id, updates) )

{ section, input, label, ul } = $

module.exports = $.component do
  prop-types:
    all-todos: types.object.is-required
    are-all-complete: types.bool.is-required

  render: ->
    all-todos = @props.all-todos
    keys = Object.keys all-todos

    return null if not keys.length

    section class-name: 'main',
      input do
        class-name: 'toggle-all'
        type: 'checkbox'
        on-change: @handle-input-change
        checked: 'checked' if @props.are-all-complete
      label html-for: 'toggle-all',
        'Mark all as complete'
      ul class-name: 'todo-list',
        keys.map (key) ->
          todo = all-todos[key]
          $ todo-item, { key, todo }

  handle-input-change: ->
    todo-actions.toggle-complete-all!

Ah ok that makes senseā€¦ I imagine that would be necessary for the time traveling debugger in Redux too :smiley:

Time traveling is just no fun if it includes time dilation.

That being said, I would gladly sacrifice a few years off everyone elseā€™s life to jump forward in time to the point where I get to see your youtube React/Redux tutorial.

1 Like

Speaking of functional programmingā€¦ iā€™m wondering if anyone would be interested in me open sourcing a little library I wrote for a RESTful API ? Iā€™m not sure how useful this would be.

Personally with a bit more polish I think having this on a headless Meteor server is more productive than an express API (no callbacks!)

I started out with just the Picker package and ran into maintenance problems so I made this.

@awatson1978 iā€™d love your opinion if you have a few spare mins!

Itā€™s written in a functional style and heavily utilizes ES6 to create a better developer API. It was evolved from the simple:json-routes package and into something that can be used to easily maintain a large API using Meteor as the base. Iā€™m currently using it to power a set of native and ReactNative apps.

The main features are:

  • Functional approach with ES6
  • Automatically catch errors and render proper JSON error
  • Easy to use router
  • Skinny controller to ease testing and to isolate side effects
  • Connection transformation in controller with chaining functions (like change header data)
  • Changeset integration with SimpleSchema
  • Optional JSON view layer for response data munging
  • CLI generator (with passing tests) if it would get open-sourced

Also some non-router things that I used are (and perhaps a sep package):

  • Repository pattern to abstract away database & throw errors if $ is prepended, eg. $insert
  • Model layer (works with changeset to validate insert/updates)

Hereā€™s the gist of how it works:

Router

const {scope, get, post, put, delete_} = Restful;

applyMiddleware(
  acceptJSON(),
  authenticateJWT({except: 'post.user'}),
);

scope('/v1', () => {
  // resource saves boilerplate using conventions
  resources('/users', UserController, {except: 'delete'})
  resources('/comments', CommentController)
  resources('/posts', PostController)

  // or manually add routes
  get('/items/', ItemController, 'index')
  post('/items', ItemController, 'create')
  get('/items/:id', ItemController, 'show')
  delete_('/items/:id', ItemController, 'delete')
  put('/items/:id', ItemController, 'update')
})

Controller

const {sendJson, putStatus} = Restful;
const {Item} from Models;
const {ItemView} from Views;

ItemController = {
  debug: true, // prints params and return value
  scrubParams: [...], // pending, will reject all contrl reqs based on rules

  index(conn, params) {
    // Repository pattern abstracts persistence dep. and removes boilerplate
    const items = Repo.all(Item);
    // allows a json 'view' to munge data outside of controller (optional)
    return renderView(conn, ItemView, items);
    //or just directly send:  `sendJson(conn, items)`
  },

  create(conn, params) {
    // you could use Repo.$insert to remove error handling
    // boilerplate, see #show & #delete actions
    const [status, result] = Repo.insert(params);
    if (status === 'ok') {
      return sendJson(conn, result);
    } else {
      return sendJson(conn, 500, {error: result})
    }
  },

  update(conn, params) {
    const [status, result] = Repo.update(Item, params.id, params);
    if (status === 'ok') {
      return sendJson(conn, result);
    } else {
      return sendJson(conn, 500, {error: result});
    }
  },

  show(conn, {id}) {
    // $get will throw and return json error if non-existent
    // or get will return undefined if non-existent
    const item = Repo.$get(Item, id);
    return sendJson(conn, item);
  },

  delete(conn, {id}) {
    Repo.$delete(Item, id);
    return sendJson(conn, {});
  }
};

Repo layer (Optional)

Too much for this post, see gist:
https://gist.github.com/AdamBrodzinski/9e667d14e980df7c0fbb

JSON View layer (Optional)

const {renderMany, renderOne} = Restful;

// you can munge data as needed without concerning the database
// this makes testing the transformation much easier
// ex in controller:
//
//     return renderView(conn, ReminderView, reminders);
//
ReminderView = {
  // can use conn to determine auth/user/params/etc..
  // munge ex: add 'data: {}' wrapper, convert time to unix timestamp, etc..
  render(conn, reminder) {
    return {
      _id: reminder._id,
      scheduledAt: reminder.scheduledAt,
      userId: reminder.userId,
      userName: reminder.userName,
      userNumber: reminder.userNumber,
      message: reminder.message,
      recurFrequency: reminder.recurFrequency,
      active: reminder.active,
      recipients: reminder.recipients,
      sentLog: reminder.sentLog
    };
  }
}
3 Likes

Looks neat! Can we change the repo per route, to say alter get to accept options or match some predefined criteria, eg deleted: false etc

Thanks!

In the examples above the first param is the Model and then the collection is grabbed from that and I just noticed the repo gist is using the string collection as the first param instead.

I havenā€™t settled on the custom query yet but my thought is to go in two directions depending on the repetition. First you could just add a custom method to the repo like getViewerFollower and that would dry up a query.

Alternatively you could pass the query into the repo. Something like this would be nice:

query = Query.findOne(...)
query2 = Query.find(..., {fields: {_id: 1}})

Repo.get(Post, query)

query3 = Query.update({ active: false, ownerId: uid }, {multi: true})

Repo.update(Post, query3)

Then under the covers the Query.findOne just collects the arguments and passes them into the real findOne with the driver.

This kind of breaks the separation with what kind of persistence is used but seems pragmatic to me because a single custom query will need to be changed in one place regardless. Multi uses can add a custom method to stay dry.

Using this pattern has been super helpful when switching databasesā€¦ it was only a few lines to switch to another DB and it still follows the database anywhere as this is just a thin wrapper around the database part :smiley:

1 Like

Hm, using a custom method would probably better in terms of security since it would not be as revealing. Also, less external input to sanitize.

1 Like

Yea that would be the best in most cases I think. Also if itā€™s on the server only there isnā€™t a security issue. On the clientā€¦ ideally weā€™d be using ES6 modules so there wouldnā€™t be any globals for people to poke intoā€¦ youā€™d have to breakpoint inside a function to get to the private method (even then webpack obfuscates the import module name to something like __Repo3).

I havenā€™t used the Repo pattern exactly on the client yet but have built a similar non-generic version of wrapping find, insert, etcā€¦ functions for my model Meteor methods (mainly for ease of testing). The nice thing about the Repo pattern is that (here at least) you pass in the model so itā€™s super generic.

Yep, I agree that a repo pattern becomes quite beneficial once app begins growing. But then you should also consider data access strategies such as paging and sorting if the data is likely to grow. You wonā€™t want an endpoint crippling the app just because it returns thousands of items.

I suggest if you want to do functional programming in meteor ( I use it a lot! ) try Ramda. It has a package for meteor that is maintained by the authors, and its designed in a much nicer way for functional programming, where things are curried by default ( and easily allows you to curry an existing function), and the parameter order of all the functions are much better for partial application.

3 Likes

@keithnicholas cool, I need to try this out! Iā€™ve been hesitant to do so because I use Elm and Elixir on the weekends and both of them have the data first instead of the normal way (though Elixir tends to favor composition instead of currying while Elm still does currying but backwards).

Do you find that it has extra functions that are more handy instead of lo-dash?

And is currying just to reduce the one(two(three(four(...)))) nesting issue? Iā€™m kind of stuck in the mindset of composing with Elixir/Elm/F# type |> pipes.

Also can you give me a good example of currying that is useful? I always read about the add and add10 examples and can never seem to thinkā€¦ ā€˜oh yea I should curry thisā€™ā€¦ though iā€™m sure there are use cases iā€™m missing.

currying is more about partial application. Which is about creating building blocks.

the simplest one I use in a meteor conxtext is

idsFor = R.pluck("_id")

when doing ā€œfizzbuzzā€ with Ramda I wrote

divisibleBy = R.curry((b) => R.compose(R.equals(0), R.flip(R.modulo)(b)))

which partially applys several functions in a chain/pipe, While this example isnā€™t too exciting, this process is where it gets more usefulā€¦ setting up chains / pipes with various partially applied functions that will transform the incoming values.

I then partially applied divisibleBy(3) and divisibleBy(5)

I could of broke out a

isZero = R.equals(0)
divisibleBy = R.curry((b) => R.compose(isZero, R.flip(R.modulo)(b)))
1 Like

Ahhhh ok this makes more sense. Thanks!

Thanks for pointing out Ramda, never recognized before but it looks quite nice.

I guess you use the ā€œoriginalā€ package https://atmospherejs.com/ramda/ramda?

yep, thatā€™s the package I use.

It introduces it as R into the project, which seems the norm.

Though there seems to be a trends for people to introduce the function names into global scopeā€¦ havenā€™t quite made my mind up about that yet :slight_smile:

Hi, hi @SkinnyGeek1010. Just wrapped up another project, and wanted to give this some proper thought before replying. Sorry for the delay.

Soā€¦ curiously interesting. Wouldnā€™t have necessarily occurred to me to write the REST API in a functional paradigm; but I donā€™t see why this couldnā€™t work splendidly. For high volume throughput, I suspect this could work particularly well, without worrying about the garbage collector keeping up. So, a few thoughts and a basic code review:

Namespacing

Functional programming doesnā€™t mean we need to ditch namespaces. My hunch is that the general approach will be made more clear by introducing a bit of namespace. Iā€™m just making up some names up, but it could look something like this:

RestApi.scope('/v1', function() {
  // resource saves boilerplate using conventions
  this.resources('/users', UserController, {except: 'delete'})
  this.resources('/comments', CommentController)
  this.resources('/posts', PostController)

  // or manually add routes
  Endpoint.get('/items/', ItemController, 'index')
  Endpoint.post('/items', ItemController, 'create')
  Endpoint.get('/items/:id', ItemController, 'show')
  Endpoint.delete_('/items/:id', ItemController, 'delete')
  Endpoint.put('/items/:id', ItemController, 'update')
})

Middleware

Very intriguing about the middleware so far. Any chance you could take a look at oauth2orize and make any recommendations on how to get started with an integration? Not asking for a complete implementation. But how difficult would it be to register a grant and some endpoints with your approach? Something like this?

var server = oauth2orize.createServer();

RestApi.applyMiddleware(
  acceptJSON(),
  authenticateJWT({except: 'post.user'}),

  server.grant(oauth2orize.grant.code(function(client, redirectURI, user, ares, done) {
  var code = utils.uid(16);
  var ac = new AuthorizationCode(code, client.id, redirectURI, user.id, ares.scope);
  ac.save(function(err) {
    if (err) { return done(err); }
    return done(null, code);
  });
}));
);

Isomorphism & Model,View,Controller

Soā€¦ this is a pretty standard server-side MVC approach. Weā€™re in agreement with the controller being a controller; but the Models and Views will confuse some people accustomed to client-side MVC paradigms.

const {sendJson, putStatus} = Restful;
const {Item} from Models;
const {ItemView} from Views;

ItemController = { ... }

So, the question is whether this ought to be isomorphic. Should it be available on the client? If so, itā€™s going to run into the client-side MVC paradigm, which involves ViewPort, device media states, the CSS subsystem, etc. etc. View layer functionality that this paradigm doesnā€™t necessary take into account. (Not to mention the HTML subsystem and the Document Object Model.) If itā€™s only going to be server-side, thenā€¦ yeah, maybe it can make due as-is.

Personally, Iā€™d recommend just scrubbing any reference to Model, View, and Controller, since those terms have a half-dozen different interpretations and are therefore somewhat meaningless and a general cause of confusion.

Other Thoughts on Functional Programming

So, are you familiar with monads yets? The biggest return-on-investment with adding functional programming to javascript apps that weā€™ve found is when there are clear computational pipelines that need to be created. Generally, weā€™ve found them with data visualizations and validation testing. We use long method chain pipelines in both our D3 graphs and our Nightwatch test scripts, and they create magic at runtime.

So whatā€™s the computational pipeline thatā€™s involved with the Rest interface? Thatā€™s the $64,000 question here. Iā€™m not necessarily suggesting that you create a method chaining pattern. But I also kind of am. Itā€™s sort of the logic culmination of this approach. So what would that look like? What does a RestApi method chain look like? What syntactical verbs are used to create a coherent API syntax?

Iā€™m just tossing some ideas out here, but I could imagine a method chaining pattern that connects with the Mongo collections could be extremely useful:

// example 1
Endpoint.route('/items/:id').collection('Items').get();

// example 2
Endpoint.route('/item/:itemId').collection('Items').findOne({_id:itemId).get();

// example 3
Endpoint.route('/item/:itemId').collection('Items').get(function(){
  return Items.findOne({_id: this.params.itemId});
});

// example 4
Endpoint.route('/item/mapreduce').collection('Items').find().fetch().get();

// example 5
Endpoint.route('/item/mapreduce').collection('Items').find().map().reduce().get();

Just some thoughtsā€¦

4 Likes