Helene: A Lightweight Real-time Web Framework for Node.js and Bun

Hey guys, I would like to present a real-time framework I’ve been building for the last few years.

I believe we can create a Meteor compatibility layer or build live queries, publications and subscriptions on top of it as a ton of the groundwork has already been done. Effectively getting us some of the good parts of Meteor without being bound to any particular stack, database or legacy decision.

tldr: I am looking for anyone who would like to contribute, by being part of a working group, either by building features present in Meteor or expanding compatibility to other frameworks or helping with architectural decisions and validation.

Bear in mind that this can already be used in production as it is if you like on a small service or app. It also supports scaling with Redis.

Helene: Real-time Web Apps for Node.js and Bun :closed_book:

Documentation :blue_book:

Here is a very bad video going over the package pages:

This can potentially be even greater than my devtools :slight_smile:

I can write one easily for Helene too :wink:

Basically it works with Methods (RPC similar to Meteor) and it is completely event-driven (which abstracts things a lot, you can implement DDP on top of it for example). The client can subscribe to events, and you can manage channels and permissions.

It is completely written in TypeScript if you take it for a spin it should be easy to see what is possible.

Here is a mermaid diagram for the arch (open on github for the preview):

And if anyone is interested in using something like Meteor with Vite or Next.js, I can demo how to do it with Helene, feel free to reach out.

Helene has React hooks, like useMethod, here is a real example:

 const { result, refresh, loading } = useMethod({
    method: 'exercise.public.listBySectionId',
    params: { sectionId: _id },
    required: ['sectionId'],
    defaultValue: [],
    deps: [_id],
    lazy: true,
  })

Events:

    useRemoteEvent(
      { event: AdminEvent.USER_REGISTERED, active: user?.isAdmin },
      email => {
        client.emit(SapienzaEvent.NOTIFICATION, {
          title: t`New User Registered`,
          body: email,
        })
      },
      [user],
    )

Helene also has an equivalent to “minimongo” which works both in the server and the browser and a few hook utilities for it like:

All hooks can be seen here:

The documentation website code can be found here (Vite):

I have a private working app completely build with Helene, if you want I can do a private demo.

Hope you are interested, I hope this can help who wants more freedom in the Node.js and Bun ecosystem, it is not a replacement for Meteor or its tons of features and packages.

Perhaps you want to run a small service or job queue and want a lib that is a fraction of the size and is very simple.

Perhaps you have a use case that we still don’t support but could with your help.

@fredmaiaarantes perhaps we can collaborate on this in the future as Meteor 3 is completed. I was half afraid of publishing it here, but this can perhaps be a great opportunity for me as well as Meteor Software to expand its offerings.

The name is not set in stone either. I was thinking on something like Asteroid or “Dinosaur Killer” (haha I’m kidding)

I would deeply appreciate any feedback or criticism. :heart:

Thank you!

8 Likes

Nice work. Looks like Meteor DDP extracted into a package. If it runs on meteor back-end, with authentication, it can be a Meteor SDK.

Does it use sockets for method calls? And can you scale the pub/sub independently from the app server? I would also like to see how it compares with simpleddp (client only)

I hadn’t thought about using it with Meteor back-end but that is one possibility for sure.

For now it uses WebSockets or HTTP, shouldn’t be hard to add another transport strategy or perhaps binary transport support (for games, etc).

For HTTP I’ve an early implementation of EventSource/Server Sent Events, just needs an automatic fallback logic.

It looks to me that “simpleddp” is built from the ground up to communicate with Meteor, Helene is completely independent right now. The aim would be to have a compatibility layer or as you mentioned perhaps an SDK or extension.

We could use “simpleddp” for inspiration on creating that compatibility layer. Thank you!

3 Likes

Yes that’s what I was thinking, if you mplement the API simple DDP has on top of Helena (maybe as a plugin or another package) then I can try it in production.

1 Like

Looking more at the API provided, looks like it can be used as is to replace simple DDP, you already have most of the building blocks.

Just hooking it to meteor methods, auth and pub/sub. So I think it makes sense to compare to the two and see how thoae can be reconciled. I use simple DDP in production, it works great, but I think your approach is even cleaner so I’m interested in it.

Is it offline-first? What about Apollo GraphQL?

Looks interesting.

I don’t think it uses Apollo, if I understood it’s place correctly, I think it is replacing Apollo with a RPC APi inspired by Meteor DDP. But @leonardoventurini can keep me honest. For offline first, probably future plugin.

@storyteller Helene Data supports offline persistence in the browser (along with indexes, etc). Shouldn’t be hard to implement sync logic etc.

One reason I built Helene is because I don’t like GraphQL much, but should be very easy to implement on top of Helene too. If I remember correctly you could pass a query directly through a method call, the same would apply to Helene.

3 Likes

I took a closer look at the code and I personally liked what you did, thanks for sharing. I actually did something similar few years back but yours is more polished.

Few thoughts that comes to my mind, that might help:

  1. With Meteor 3.0, express might become a first-class citizen, thus we can piggyback on that instance and also bind to the Meteor websocket.

  2. There might be away to export something similar to meteor validated method in the compatibility package, that would basically change the method to use Helene transport instead.

  3. Could include meteor auth plug-in in the compatibility package

  4. Could use monorepo to decouple react, server and client

  5. Could decouple the collection util methods and make it a plug-in, I mean the core framework could be more focused on getting the data across and the story of nedb is not encouraging

  6. I was thinking about an example of a deployment with digital ocean apps platform with reddit, not sure what you used for deployment. An example on Galaxy with Meteor 3.0+ would be great too with “Meteor compatibility” package

  7. It seems to default to websocket however it might be better to make that client stateless and make websocket an option per page or, should the user desire, the entire client lifecycle. This is from performance perspective and a lot of apps don’t need dedicated socket connection and this would put unnecessary pressure on the server.

  8. For the plug-in system, might be useful to get aspiration from simple ddp, very well done package, another could source of aspiration would be featherjs I think.

  9. Auth plug-ins for popular identity providers will be a plus

  10. Future would be nice to have some devTool for the methods, events etc, also something like montiapm

But I think it is a great initiative, personally I think having “Meteor compatibility” with example on Galaxy etc, would help build adopters to this package since it can be used along with meteor 3.0 backend and would appeal to the same crowed. Thanks for sharing, I hope that helps, and I plan to try it in a production app at some point.

2 Likes

Thank you for the detailed feedback, I really appreciate it!

I have an app that runs on K8s on DigitalOcean, the stack is Helene, Vite and React. My deploys take 1m for production. I can setup an example and record a video on it.

I need to come up with a better architecture for the transport layer, adding better logic for fallback or optionality, I can record a video on this too.

The monorepo is a great idea, I was trying to keep it simple for now, but we can split the modules into separate packages for sure.

I use Google Auth and I have my own personal implementation of passwordless, they could easily become plugins.

I have a private devtools extension which I use directly in chrome, I can polish it a bit and release it in the chrome store.

The others I agree 100% or will take to heart. Thank you!


An open letter to anyone reading this:

I can’t do all of this alone, I would really appreciate if anyone is interested in collaborating. I am sure it is a good opportunity for a portfolio piece and to learn the internals of a framework that is simple enough to understand but complex enough to be challenging and valuable. I will support and guide anyone who is up to it.

When I get some free time I will set up a Discord server and create the issues pertaining to the project roadmap. Now is the time, if your app could benefit from Helene, tell us your issue and we can adjust the roadmap/work together.

Thank you! Great week ahead to everyone. :heart:

4 Likes

Hi @leonardoventurini - this looks awesome!

Actually, I wanted something similar to this and was trying to imagine how you’d go about building it, so now I have less to imagine :sweat_smile:

But I really felt like a lot of good ideas from Meteor had trouble filtering into the NPM ecosystem (probably because Meteor hasn’t been broken up into NPM packages yet), so I’m excited to see the RPC & data sharing layer “split out” like this.

Even more excited to see agnostic auth, middleware, and the way you’ve sort of separated out the responsibilities of events (subscriptions?) and methods but given developers ways to use both reactively (cf SWR and Tamstack). It’s a nice clean API and the simplicity is refreshing.

It would be good to have this simple API + an optional Meteor compatibility layer. It’d be very cool if people could set up Helene frontends and connect them to Meteor backends but that might be a lower priority / far off item.

I’d actually love to help but I’m a bit busy (in life in general) so apologies if that help doesn’t actually materialise!

I will also leave some (maybe annoying) questions on Github directly - I notice you don’t have discussions set up, so I will make those in Issues. Hopefully that’s OK - though I think if this gets traction you’ll want to direct more stuff to discussions (if it’s free?) so low priority questions don’t boost the “Issue” count number in a scary way.

2 Likes

I opened some issues as well, discussion is a good idea since this is early and do not want to clutter the issues count with design discussions.

1 Like

Hi @leonardoventurini, it’s great that you are sharing Helene here with the community.

For those who don’t know, Leo is a developer from Quave, but Helene is his personal project. Quave is willing to help with resources in case Leo wants to create this compatibility layer with Meteor.

Are any other companies interested?

We could create a work group, as Leo said to discuss ideas and make decisions as a group.

It could be a large project, so it’s impossible for Leo to move forward alone with it.


Side notes:

Coroutines

I’m also researching ways to run coroutines (aka Fibers) in Bun or other Javascript engine environments. Helene could also come up with a compatibility layer even supporting coroutines.

With we can support Coroutines and API of Meteor it could be a very good alternative for Meteor 2 users as well.

Why Coroutines? Coroutines is not a bad idea, it was just a problem with the approach of V8 but many languages/environments have coroutines support but of course for Node.js is now a bad idea as V8 team decided to move in another direction.

Why Coroutines & Helene? It would be easy to implement something new like Coroutines with Bun in a new environment that is just starting than migrating everything that Meteor has.

Anyway, exciting times.

5 Likes

Thank you everyone! Every feedback/idea is welcome, we can set a round table to discuss things at some point too.

I enabled discussions for the repo:

The feedback is very inspiring, thanks again!

1 Like

For sure anyone is welcome, even you Filipe :joy:

…and Quave hahaha

I love the coroutines idea, I am willing to help as soon as I get more time on my hands.

Can you share what you have found so far? Perhaps create a discussion about it somewhere?

Thank you! I really appreciate it :heart:

Exciting times… :wink:

1 Like

Started splitting into separate packages:

Please let me know if you have any suggestion.

1 Like

The separation is complete, can anyone review?

I am using yarn workspaces + lerna.

As a next step I am planning to create a mongo package and start experimenting with live queries.

tldr:

I have separated the framework into:

  • @helenejs/client
  • @helenejs/server
  • @helenejs/utils
  • @helenejs/data
  • @helenejs/react
4 Likes

Lerna is a bit dated and I think they are moving to npm workspaces mostly, I think better check before using Lerna. Other than that, I think the code is clean.

It’s still a great tool, for now it does the job as the use case is fairly simple. I do have an eye out for turborepo, but I need to study it a bit more (creating an issue about this).

Found these articles from a time ago and I also remember seeing Nx in some of the logs:

NX is good I think lerna is planning to deprecat lerna bootstrap and piggyback on NPM.

I tell you from experience it is really bad at scale and has some issues with corrupting lock files etc.

But I think it does well for small projects.