Was choosing Meteor for Android & IOS Native Apps a huge MISTAKE?

hassansardarlecodeur · October 29, 2020, 4:05pm

I build a server + web app chat application similar to WhatsApp on Meteor+Mongodb.

Seems like I did a mistake choosing Meteor.

For Web Client its all fine with the Subscriptions & Reactivity & stuff.

But when it comes to Andoid & IOS Native Applications it seems like MDG has never thought of managing official DDP libraries. Since current 3rd party DDP Native libraries are totally unstable and there are a lot of connection issues & reactivity issues.

So far I have followed these libraries but as you can see that these are 3rd Party libraries & mostly UNMAINTAINED:

So what should I do now. I am totally stuck here. My Web Application is totally Perfect but Native App are totally a mess. No stability at all.

Are there any official support by MDG itself? Or I should never have choose Meteor on the first place?

alawi · October 29, 2020, 4:22pm

I think most folks here do Cordova, react native or PWAs.

Have tried something like SimpleDDP with JavaScript? Maybe you have a bridge from native to JS? I’m not sure.

Otherwise you can go Rest route, or try fix those libraries. When you pick open source frameworks and you diverge from the common usage patterns and the community, you should be ready to fork down the road, most of those packages
are maintained by people on their spare time…

However I’m really curious about the connection being unstable and I wonder what is the underlying issue, perhaps some reconnection logic is missing from those libraries.

hassansardarlecodeur · October 29, 2020, 4:51pm

We face issues like resuming subscription after the Native App is idle or something. Also I dont know if our ideas match on this but Cordova or React Native apps can never beat proper Native Application in both areas Performance & Visuals.
I am expecting something from MDG to build for us.

alawi · October 29, 2020, 5:03pm

We do mostly PWAs but we target enterprises so we don’t need fancy animation and the stores. But I understand the need for native performance, for that you can use react native.

MDG is no longer here, a lot of the effort now is done by Open Source community members like me and you, and those libraries were never maintained by MDG to start with, they had enough scope to worry about. With regards to the connection I’m sure there is missing logic for maintaining and/or resuming the connection, I think those libraries forked from one another and never addressed this issue, but that is just a guess, I don’t have first hand experience with them.

If I were in your position, the first thing I would do is compare those libraries connection logic with something working like Meteor client or SimpleDDP and try to figure out what is missing.

hassansardarlecodeur · October 29, 2020, 5:13pm

Actually I considered Meteor as a Complete solution. Plus as most clients demand Native Apps, I would have personally went for ReactNative but thats not upto my choice. Also time is so crucial for devs like us to study Web libs and develop Mobile DDP libs based on them

alawi · October 29, 2020, 6:08pm

Like I said those are open source libraries and they are clearly marked as archived, you’ve to roll up your sleeves and take a closer look at the code.

The reconnection logic for the swift library is here

github.com

siegesmund/SwiftDDP/blob/bc0858f8b3557eeba448821968cf6949e07dc096/Sources/DDPExponentialBackoff.swift#L32


// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.

import Foundation

///Retry connection requests to the server. The retries exponentially increase the waiting time up to a certain threshold. The idea is that if the server is down temporarily, it is not overwhelmed with requests hitting at the same time when it comes back up.

class DDPExponentialBackoff {
    
    
    init(retryInterval:Double = 0.01,maxWaitInterval:Double = 5,multiplier:Double = 1.5){
        
        self.reconnectionRetryInterval = retryInterval
        self._reconnectionRetryInterval = retryInterval
        self.maxWaitInterval = maxWaitInterval
        self.multiplier = multiplier
    }
    
    //Cached original interval time
    fileprivate var _reconnectionRetryInterval:Double = 0

And for the Android

github.com

delight-im/Android-DDP/blob/4ae26e3728b5d2956ec670a0667da20f2b40b430/Source/library/src/main/java/im/delight/android/ddp/Meteor.java#L161



			@Override
			public void onDisconnected(final WebSocket websocket, final WebSocketFrame serverCloseFrame, final WebSocketFrame clientCloseFrame, final boolean closedByServer) {
				log(TAG);
				log("  onClose");

				final boolean lostConnection = mConnected;

				mConnected = false;

				if (lostConnection) {
					mReconnectAttempts++;

					if (mReconnectAttempts <= RECONNECT_ATTEMPTS_MAX) {
						// try to re-connect automatically
						reconnect();
					}
					else {
						disconnect();
					}
				}

You can see that the swift library logic is a bit more sophisticated as it try to space out the re-attempts.

SimpleDDP tries to reconnect using a configurable time interval you can see the logic here:

github.com

Gregivy/simpleddp/blob/4bd415ee52e4023787e401111d9be5ca788a333b/core/src/ddp.js#L69


    };
    if (this.sessionId) params.session = this.sessionId;
    this.socket.send(params);
});

this.socket.on("close", () => {
    let oldStatus = this.status;
    this.status = "disconnected";
    if (this.cleanQueue) this.messageQueue.empty();
    if (oldStatus != "disconnected") this.emit("disconnected");
    if (this.autoReconnect) {
        // Schedule a reconnection
        setTimeout(
            this.socket.open.bind(this.socket),
            this.reconnectInterval
        );
    }
});

this.socket.on("message:in", message => {
    if (message.msg === "connected") {

And the Meteor client, the reconnect logic is here

github.com

meteor/meteor/blob/d9db4f52f2ea6d706a25156768ea42e1fbb8f599/packages/retry/retry.js#L11


// Retry logic with an exponential backoff.
//
// options:
//  baseTimeout: time for initial reconnect attempt (ms).
//  exponent: exponential factor to increase timeout each attempt.
//  maxTimeout: maximum time between retries (ms).
//  minCount: how many times to reconnect "instantly".
//  minTimeout: time to wait for the first `minCount` retries (ms).
//  fuzz: factor to randomize retry times by (to avoid retry storms).

export class Retry {
  constructor({
    baseTimeout = 1000,
    exponent = 2.2,
    // The default is high-ish to ensure a server can recover from a
    // failure caused by load.
    maxTimeout = 5 * 60 * 1000,
    minTimeout = 10,
    minCount = 2,
    fuzz = 0.5,
  } = {}) {

It is similar to the swift library logic however it has a fuzz factor to avoid reconnection storm (i.e. many clients try to connect to the server)

So basically all of them attempt to reconnect several times and then give up. The android library seems the most primitive since it doesn’t seem to have an interval between the attempts, SimpleDDP has a fixed configurable time interval, Swift has exponential and Meteor has some random factor to distribute the reconnection attempts.

I guess what is missing from the Android/Swift library is the listener when the sockets connection opens up again that would trigger the reconnection attempt. In SimpleDDP (the browser) when the socket opens up again, the client will receive an event that triggers the reconnection.

Edit: actually the logic seems to be here for Android at least

github.com

delight-im/Android-DDP/blob/4ae26e3728b5d2956ec670a0667da20f2b40b430/Source/library/src/main/java/im/delight/android/ddp/Meteor.java#L142


		// save the context reference
		mContext = context.getApplicationContext();

		// save the data store reference
		mDataStore = dataStore;

		// create a new handler that processes the messages and events received from the WebSocket connection
		mWebSocketListener = new WebSocketAdapter() {

			@Override
			public void onConnected(final WebSocket websocket, final Map<String, List<String>> headers) {
				log(TAG);
				log("  onOpen");

				mConnected = true;
				mReconnectAttempts = 0;

				initConnection(mSessionID);
			}

			@Override

It basically reset the attempts and try to connect again, so that seems fine to me. But what SimpleDDP does better is that it has an event dispatching system that the app can handle on disconnect and reconnect, and this would allow you to display messages or refresh the page, I don’t seem to find those events in the Android library, maybe you can confirm. I would put some alerts to detect when those connection/reconnection happen to understand how the app behaves in those scenarios, that is what I did with SimpleDDP.

I hope that helps! I find this area interesting but I don’t use those libraries, I only use SimpleDDP and it works perfectly.

peterfkruger · October 29, 2020, 6:15pm

Meteor is what it is, if I may say so. What it is can be read in the documentation and the guide provided on https://meteor.com. You can consider it a “complete solution”, or a collection of resources that you can use however you wish. However, it’s an open source product that comes with no guarantee whatsoever. Following is the actual text of the Meteor license document as published on github:

The MIT License (MIT)

Copyright © 2011 - present Meteor Software Ltd.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

This license applies to all code in Meteor that is not an externally maintained library. Externally maintained libraries have their own licenses, included in the LICENSES directory.

Meteor’s intended use as a “complete solution” is a classic web application, and also as a mobile app via Cordova builds, respectively.

If I understand you correctly, you plan to use only the server part from the original product, yet you still see what you use as a “complete solution”.

I guess you should have noticed that there is no provision in Meteor to cover your particular use case, as a native Android or Swift DDP package was never part of it.

It’s like getting a car for free, which you take apart, rebuild half of it with something that the original manufacturer did not anticipate or had provisions for, and then when you don’t find manufactured parts for it to make it run the way you intended, you start complaining that it’s breaking down and someone should fix it for you.

mikkelking · October 29, 2020, 8:05pm

@hassansardarlecodeur when you are working with open source software, you have to accept a degree of risk for yourself. You are not paying for a product, you are using free software. I think the support here has been great, and there are ways to fix it. The SimpleDDP package does work well, I would recommend using it.

Most times you can work out how to fix things yourself with open source code, or someone may be able to help you. Access to the source code is a major plus for that. Your decision to use Meteor was probably a good one - just look on this as a road bump. Once you are over it, the reasons for choosing Meteor in the first place will appear more valid

hassansardarlecodeur · October 30, 2020, 11:39am

I respect Meteor community a lot…
I am using Meteor with Angular js since launch…
Build many dashboards & web application…
Learned alot, specially the community helped me with lots of issues & stuff.
Thanks for that & always will be…
As per my assumption…
When Meteor was launched no body cared about hybrid applications that much as they do now.
Native applications were the top priority those days.
So my point is MDG never thought of supporting Native applications officially at that time.
Ok i know devs should manage and work by themselves if they get stuck.
But everywhere it is not the case…
Devs mostly doesn’t get much time to manage those things.
My genuine point here which seems kind of funny in my head…
Its been many years Meteor has launched…
No body ever worked with Native Mobile Applications?
Or even if they did, they never faced issues like i did?
Seems bit confusing.
I mean community that older and experienced…
When some new devs gets to know that there is a Mature Meteor community launched many years ago, and they want to build apps quicker…
Trust me it will be a bit unfair for them to get stuck where they have to fix DDP libraries when the deadlines are on their heads.
I totally support Meteor, I will always be using and preferring Meteor for web applications & hybrid solutions like React & Ionic.
But I personally won’t recommend Meteor for Native Mobile Applications.
Thats all i have to say for now.
And thanks a lot Meteor community for the support

waldgeist · October 30, 2020, 4:33pm

@hassansardarlecodeur I understand that it is sometimes frustrating developing native frontends for Meteor. We’re kind of in the same boat, using a Unity client. We ended up using an existing DDP library and gradually adjusting it to our needs.

That library was rather “dumb”, for instance it did not care for connection break-ups, which caused us a lot of headaches, as WebSockets tend to break if the app is put into the background. If you’re used to the magic Meteor provides on the web, you get pretty spoiled. But you only learn it the hard way if the magic isn’t there anymore.

However, it turned out that the DDP protocol is so simple, that it was actually not that hard to understand and adapt things to our requirements. If I were you, I would go that route, fork one of the existing libs and extend them. Maybe you can even contribute back to these projects using PRs?

Having said this, I can still understand your expectation that stuff like that should be available out-of-the-box. But unfortunately, the mobile part of Meteor was always a bit neglected. At least, things have gotten better since Tiny took over.

alawi · October 30, 2020, 4:55pm

Yeah I think I under where @hassansardarlecodeur is coming from, he expecting that Meteor [quote=“waldgeist, post:10, topic:54390”]
If I were you, I would go that route, fork one of the existing libs and extend them. Maybe you can even contribute back to these projects using PRs?
[/quote]

Yeah I don’t think those projects are too far off for being fixed to have reliable connections, I think they need an even dispatching systems and more testing, both of their code bases are very well written and documented.

tanutapi · November 5, 2020, 12:36pm

Hi, I am an author of dart_meteor package. If you interested in create a mobile app with Flutter + Meteor. Please check my post on Medium. https://link.medium.com/mFjh4frlabb

nikos777 · November 20, 2020, 10:55am

I tried the most minimal set up with Svelte but had no luck:

github.com/meteor-svelte/meteor-svelte

ios svelte requires 3.27.0 + blank screen

opened 04:45PM - 17 Nov 20 UTC

closed 11:12AM - 21 Nov 20 UTC

quantuminformation

Done my first ios run following tutorial up to https://www.meteor.com/tutorials/…svelte/running-on-mobile ➜ simple-todos git:(master) ✗ meteor run ios ``` WARNING: this package requires svelte@3.27.0 to be installed as npm peer dependency, but your package.json specifies: "svelte": "^3.29.7" If you encounter error messages, consider running: meteor npm install --save svelte@3.27.0 => Started MongoDB. %% No simulator found for ". Falling back to the default target. %% Adding xcodebuildArg: [ '-UseModernBuildSystem=0' ] %% Adding xcodebuildArg: [ '-quiet' ] => Started your app. => App running at: http://localhost:3000/ warning: The Legacy Build System will be removed in a future release. You can configure the selected build system and this deprecation message in File ``` I was expecting app to look like ![image](https://user-images.githubusercontent.com/216566/99419378-594cb100-28fc-11eb-806d-2789019c08cf.png) but got: ![image](https://user-images.githubusercontent.com/216566/99419403-5fdb2880-28fc-11eb-8572-5f49feb664c4.png) meteor npm install --save svelte@3.27.0 didn't help fix the blank screen

ehssham · November 24, 2020, 9:08am

@alawi You have shared the codes for reconnecting WebSocket and connect meteor, however I would like to highlight the “Non-Resuming Previous Connection Issue” in DDP.

As mentioned in this link we need to send the previous connection ID to the next connection request to make use of previously subscribed data. But unfortunately, this is not working with my scenario which is tested carefully with the following test case.

Test Case:
I have a collection named “myTestCollection” and there are only 10 records (documents) we inserted against this collection. Now I subscribe to this collection for 0.2sec and disconnect the socket forcefully. It could give me 2 or 3 records in this duration and I have the previously connected session_id with me after disconnection which is known as _lastSessionId for ddp.

Now if I connect my WebSocket again, and give ddp the following request:

{"msg": "connect ", "session": "ERoZSR3R3f8zBQ6Ry", "version": "pre1","support":["pre1"]}

I would like to fetch my remaining 7 or 8 records and ddp should automatically subscribe to my previous collections since I am passing my _lastSessionId in the meteor connection request.

P.S:

I am the author of MeteorDDP library which is written in purely Swift without any bridging with WebView or javascript. I have fixed this issue by re-subscribing all previously active subscriptions but want to get the optimal solution for the Mobile library.

@hassansardarlecodeur thanks for highlighting this issue here. I am sure, Meteor folks will help me to make it better for native mobile operating systems.

alawi · November 24, 2020, 10:12am

How is that issue handled in the origin Meteor client?

Also some clients don’t even have event system or reconnect logic thus it is leaving the the developer in the dark.

ehssham · November 24, 2020, 8:38pm

Yes nobody has handled this issue specifically. Libraries like SimpleDDP are only sharing the _lastSessionId on connection request but no one guarantee about resuming the previously subscribed collections.

@alawi can you once confirm the test case in any library how that behaves? It would be great if Meteor guys consider this as feedback and mark any clue of the user what he has subscribed and how much data he got from some collection.