Use Functional Programming to Simplify Your Meteor Code

Yea I remember that! It’s nice to see that the JS community is heading towards that way (and I think in general it’s getting more popular). It seems like React + Redux is the gateway drug for JavaScript FP :laughing:

Being a newer programmer, learning about FP was like an ‘ah-ha’ moment but this has been realized by experienced devs for quite some time :smiley:

I agree that the flux phenomenon is overblown, though sharing what React calls ‘state’ is hard to do without some kind of pattern (as a component can’t pass to it’s sibling). I think that’s where it started and then morphed into borrowed ideas as new buzz words.

However, the nice thing is that it’s getting people thinking about decoupling the views and not keeping transient state public and scattered about.

3 Likes

Immutability everywhere is an incredibly poor fit in a generational stop the world gc world. That’s one of the problems Erlang solves beautifully with gc per process, and also why modern vm’s cache strings internally as a pool of constants.

I once had the bright idea of eliminating bugs by refactoring all Vector3f objects to be immutable in an mmo. Spent a few days and a gallon of coffee on the 90k line codebase. The gc was able to keep up during testing with the 5-10 member test team.

Once I pushed the branch into production and a thousand users logged on, things were a bit different indeed. Lesson learned. Immutable nothing, pool all objects.

In terms of FP concepts, lambdas / closures are hot for sure. However I’m just not copacetic when it comes to recursion and pattern matching. A high level language shouldn’t end up looking like something Leonard Susskind would write on his blackboard.

1 Like

It depends on your definition of high level. A pool of objects === references === pointers you have to micro manage. I find working with values only, where you can replace a value by another without bothering about distant side effects, to be more high level.

@jacobin this is really interesting, thanks for sharing! These kind of things can be tough to figure out until it’s too late :smile: I don’t think Redux is technically immutable in the way where it copies the object… instead it just enforces not mutating regular data structures.

However, I was wondering what the perf. impact would be with seamless-immutable vs immutable.js… seamless-immutable only has dev time checks to prevent you from using mutation and then in prod is using mutable structures… unless you use merge to do the trie thing.

I’ve also been wondering if an es-lint check can be made to check against mutations… it would be interesting to just enforce this rather than making slower data structures.

What do you think of Erlang/Elixir pattern matching? The ones i’ve used so far is mainly to check if something in a pipeline has an 'ok' or 'error'.

I really do agree that when you try and get too clever with FP it makes things incredibly hard for new devs or coming back a year later.

After some Reagent training and few Redux demos, I think it is nice to use functional programming and I am trying re-wire my patterns in head. Use underscore to slowly switch my habbits from Coffeescript to more widely used library so people can better understand original form of my code.

And btw, this seems like really nice tutorial - loooong, but deep http://teropa.info/blog/2015/09/10/full-stack-redux-tutorial.html

3 Likes

If it were done in pure C, perhaps. Though I never found linked lists all that difficult to comprehend.

Generally object polling is implemented with higher level abstractions on top of the same concept. In Java it was usually ConcurrentLinkedQueue<>(). Your interface to the ugliness is merely a borrow and release method. Just had to be sure you reset the objects in the release method not the borrow to help tame even more side effects.

It merely forces you to be a better programmer. I prefered side effects and thousands of users over no side effects and a handful of users complaining on my forums about 30 second lockups.

From first hand experience with scaling apps on VM’s whenever I see the world ‘immutable’ this image pops into my head.

I made a subtle reference to the topic at the bottom of this thread. It seems most people using Meteor will horizontally scale long before the GC catches on fire.

As far as Erlang/Elixlr pattern matching, I looked into again after a few years from your reference to Phoenix. Still makes my eyes spin around in the sockets. I’m not going to start with recursion instead of flow control. That makes me cry, and I have 3+ decades experience. :slight_smile:

I will never catch on with the “Can’t I just meteor add” crowd.

2 Likes

@jacobin Yea I try not to use pattern matching if it’s the head spinning variety. Here’s a trivial example of using it where it’s a little more clear (though an if/else would suffice here)

defmodule StringHelpers do
  def replace_chars("", _newChar) do
    ""   # don't do work if it's empty
  end

  def replace_chars(str, newChar) do
    String.replace(str, newChar)
  end
end

I would highly recommend picking up Dave Thomas’s Elixir book as a vehicle to learn functional concepts for JavaScript. They’re very transferable to JS with underscore and for better or worse takes you in a Unix pipeline mindset (composing small re-useable functions).

Here’s an excerpt from the tagline that pretty much sums it up… pragmatic and non academic:

…You want to explore functional programming, but are put off by the academic feel (tell me about monads just one more time)…

http://www.amazon.com/Programming-Elixir-Functional-Concurrent-Pragmatic/dp/1937785580

2 Likes

That’s easily comprehensible. Most of the syntactic sugar in the official documentation I believe to be written by a mathematician on acid.

Taken individually most of the concepts such as how it views immutability are easily to grok. After all, it’s how we learned in high school algebra. A = 2. B = A + 1. Tuples = structs, easy to understand.

However once you throw it all together into a soup pot, then things like 6502 assembly, java or C# bytecode becomes more appealing. I just can’t see myself ever being able to scan source and immediately grok the intention.

Perhaps I’m just too old to not have a clear mnemonic representation. I wish that were not the case, as Erlang is one hella powerful VM. Perhaps someone can come up with an Erlang/Elixr transpiler for a C family language.

Was looking into the issue of immutability in reference to Redux / Immutable.js. Looks like it uses a binary tree internally so no nasty array copies. It also uses object reference not value for comparisons. That makes it memory efficient and performant.

The above video gives a nice summation.

You will obviously still kill the gc if you throw a million changes into an immutable store. So while I wouldn’t use it to hold state for a game, your average website I doubt you would notice anything without object pooling.

2 Likes

Pipes are available in Livescript, with React, I would like to suggest you to take a look at Elementary I think it looks like RedScript without Elixir syntax. Wish to see RedScript do better .

class todo-store   
  ->
    @bindActions todo-actions
    @todos =
      'id1':
        id: 'id1'
        complete: false
        text: 'Configure Jest'
      'id2':
        id: 'id2'
        complete: false
        text: 'Add unit tests'
      'id3':
        id: 'id3'
        complete: false
        text: 'Achieve 100% code coverage'

  update: (id, updates) ->
    if @todos[id] and updates then
      @todos[id] = merge(@todos[id], updates)

  update-all: (updates) ->
    self = @
    Object.keys self.todos
      |> ( .map (id) -> self.update(id, updates) )

{ section, input, label, ul } = $

module.exports = $.component do
  prop-types:
    all-todos: types.object.is-required
    are-all-complete: types.bool.is-required

  render: ->
    all-todos = @props.all-todos
    keys = Object.keys all-todos

    return null if not keys.length

    section class-name: 'main',
      input do
        class-name: 'toggle-all'
        type: 'checkbox'
        on-change: @handle-input-change
        checked: 'checked' if @props.are-all-complete
      label html-for: 'toggle-all',
        'Mark all as complete'
      ul class-name: 'todo-list',
        keys.map (key) ->
          todo = all-todos[key]
          $ todo-item, { key, todo }

  handle-input-change: ->
    todo-actions.toggle-complete-all!

Ah ok that makes sense… I imagine that would be necessary for the time traveling debugger in Redux too :smiley:

Time traveling is just no fun if it includes time dilation.

That being said, I would gladly sacrifice a few years off everyone else’s life to jump forward in time to the point where I get to see your youtube React/Redux tutorial.

1 Like

Speaking of functional programming… i’m wondering if anyone would be interested in me open sourcing a little library I wrote for a RESTful API ? I’m not sure how useful this would be.

Personally with a bit more polish I think having this on a headless Meteor server is more productive than an express API (no callbacks!)

I started out with just the Picker package and ran into maintenance problems so I made this.

@awatson1978 i’d love your opinion if you have a few spare mins!

It’s written in a functional style and heavily utilizes ES6 to create a better developer API. It was evolved from the simple:json-routes package and into something that can be used to easily maintain a large API using Meteor as the base. I’m currently using it to power a set of native and ReactNative apps.

The main features are:

  • Functional approach with ES6
  • Automatically catch errors and render proper JSON error
  • Easy to use router
  • Skinny controller to ease testing and to isolate side effects
  • Connection transformation in controller with chaining functions (like change header data)
  • Changeset integration with SimpleSchema
  • Optional JSON view layer for response data munging
  • CLI generator (with passing tests) if it would get open-sourced

Also some non-router things that I used are (and perhaps a sep package):

  • Repository pattern to abstract away database & throw errors if $ is prepended, eg. $insert
  • Model layer (works with changeset to validate insert/updates)

Here’s the gist of how it works:

Router

const {scope, get, post, put, delete_} = Restful;

applyMiddleware(
  acceptJSON(),
  authenticateJWT({except: 'post.user'}),
);

scope('/v1', () => {
  // resource saves boilerplate using conventions
  resources('/users', UserController, {except: 'delete'})
  resources('/comments', CommentController)
  resources('/posts', PostController)

  // or manually add routes
  get('/items/', ItemController, 'index')
  post('/items', ItemController, 'create')
  get('/items/:id', ItemController, 'show')
  delete_('/items/:id', ItemController, 'delete')
  put('/items/:id', ItemController, 'update')
})

Controller

const {sendJson, putStatus} = Restful;
const {Item} from Models;
const {ItemView} from Views;

ItemController = {
  debug: true, // prints params and return value
  scrubParams: [...], // pending, will reject all contrl reqs based on rules

  index(conn, params) {
    // Repository pattern abstracts persistence dep. and removes boilerplate
    const items = Repo.all(Item);
    // allows a json 'view' to munge data outside of controller (optional)
    return renderView(conn, ItemView, items);
    //or just directly send:  `sendJson(conn, items)`
  },

  create(conn, params) {
    // you could use Repo.$insert to remove error handling
    // boilerplate, see #show & #delete actions
    const [status, result] = Repo.insert(params);
    if (status === 'ok') {
      return sendJson(conn, result);
    } else {
      return sendJson(conn, 500, {error: result})
    }
  },

  update(conn, params) {
    const [status, result] = Repo.update(Item, params.id, params);
    if (status === 'ok') {
      return sendJson(conn, result);
    } else {
      return sendJson(conn, 500, {error: result});
    }
  },

  show(conn, {id}) {
    // $get will throw and return json error if non-existent
    // or get will return undefined if non-existent
    const item = Repo.$get(Item, id);
    return sendJson(conn, item);
  },

  delete(conn, {id}) {
    Repo.$delete(Item, id);
    return sendJson(conn, {});
  }
};

Repo layer (Optional)

Too much for this post, see gist:
https://gist.github.com/AdamBrodzinski/9e667d14e980df7c0fbb

JSON View layer (Optional)

const {renderMany, renderOne} = Restful;

// you can munge data as needed without concerning the database
// this makes testing the transformation much easier
// ex in controller:
//
//     return renderView(conn, ReminderView, reminders);
//
ReminderView = {
  // can use conn to determine auth/user/params/etc..
  // munge ex: add 'data: {}' wrapper, convert time to unix timestamp, etc..
  render(conn, reminder) {
    return {
      _id: reminder._id,
      scheduledAt: reminder.scheduledAt,
      userId: reminder.userId,
      userName: reminder.userName,
      userNumber: reminder.userNumber,
      message: reminder.message,
      recurFrequency: reminder.recurFrequency,
      active: reminder.active,
      recipients: reminder.recipients,
      sentLog: reminder.sentLog
    };
  }
}
3 Likes

Looks neat! Can we change the repo per route, to say alter get to accept options or match some predefined criteria, eg deleted: false etc

Thanks!

In the examples above the first param is the Model and then the collection is grabbed from that and I just noticed the repo gist is using the string collection as the first param instead.

I haven’t settled on the custom query yet but my thought is to go in two directions depending on the repetition. First you could just add a custom method to the repo like getViewerFollower and that would dry up a query.

Alternatively you could pass the query into the repo. Something like this would be nice:

query = Query.findOne(...)
query2 = Query.find(..., {fields: {_id: 1}})

Repo.get(Post, query)

query3 = Query.update({ active: false, ownerId: uid }, {multi: true})

Repo.update(Post, query3)

Then under the covers the Query.findOne just collects the arguments and passes them into the real findOne with the driver.

This kind of breaks the separation with what kind of persistence is used but seems pragmatic to me because a single custom query will need to be changed in one place regardless. Multi uses can add a custom method to stay dry.

Using this pattern has been super helpful when switching databases… it was only a few lines to switch to another DB and it still follows the database anywhere as this is just a thin wrapper around the database part :smiley:

1 Like

Hm, using a custom method would probably better in terms of security since it would not be as revealing. Also, less external input to sanitize.

1 Like

Yea that would be the best in most cases I think. Also if it’s on the server only there isn’t a security issue. On the client… ideally we’d be using ES6 modules so there wouldn’t be any globals for people to poke into… you’d have to breakpoint inside a function to get to the private method (even then webpack obfuscates the import module name to something like __Repo3).

I haven’t used the Repo pattern exactly on the client yet but have built a similar non-generic version of wrapping find, insert, etc… functions for my model Meteor methods (mainly for ease of testing). The nice thing about the Repo pattern is that (here at least) you pass in the model so it’s super generic.

Yep, I agree that a repo pattern becomes quite beneficial once app begins growing. But then you should also consider data access strategies such as paging and sorting if the data is likely to grow. You won’t want an endpoint crippling the app just because it returns thousands of items.

I suggest if you want to do functional programming in meteor ( I use it a lot! ) try Ramda. It has a package for meteor that is maintained by the authors, and its designed in a much nicer way for functional programming, where things are curried by default ( and easily allows you to curry an existing function), and the parameter order of all the functions are much better for partial application.

3 Likes

@keithnicholas cool, I need to try this out! I’ve been hesitant to do so because I use Elm and Elixir on the weekends and both of them have the data first instead of the normal way (though Elixir tends to favor composition instead of currying while Elm still does currying but backwards).

Do you find that it has extra functions that are more handy instead of lo-dash?

And is currying just to reduce the one(two(three(four(...)))) nesting issue? I’m kind of stuck in the mindset of composing with Elixir/Elm/F# type |> pipes.

Also can you give me a good example of currying that is useful? I always read about the add and add10 examples and can never seem to think… ‘oh yea I should curry this’… though i’m sure there are use cases i’m missing.