Is there any timeout for Meteor.call() on the client? How do we set up this timeout? The http connection has a timeout, how about the meteor methods? Suppose the server is overloaded and takes way too long to return, what is going to happen?
There’s no built-in timeout for Meteor calls. You can set up your own timeout the old fashioned way like so:
const timeout = setTimeout(() => {
throw new Error('timeout');
// OR do whatever error handling you want
}, 5000); // wait 5s
Meteor.call('someMethod', (result) => {
clearTimeout(timeout);
// do something
}
That’s basically documented here: Methods | Meteor Guide - that discusses loss of connection, although the effect is the same.
Note that timing out a call yourself won’t have any effect on the retry behaviour. To alter that, you’ll need to first prevent the default behaviour by using Meteor.apply() with the noRetry option. That will allow you to catch a timeout, but it won’t prevent the method from completing and returning, so you’ll need to allow for that, too.
2 Likes
I have a long running function, for which i’m using Meteor.apply to call the stub, with the following options
noRetry: true,
throwStubExceptions: false,
onResultReceived: (err, resp) => {
console.log("onErrorReceived", err);
console.log("onResultReceived", resp);
},
returnStubValue: true
I also have an asyncCallback to receive the response.
However, i’m getting the following error everytime
{isClientSafe: true, error: "invocation-failed", reason: "Method invocation might have failed due to dropped…ause `noRetry` option was passed to Meteor.apply.", details: undefined, message: "Method invocation might have failed due to dropped…n was passed to Meteor.apply. [invocation-failed]"}
I am using Meteor.apply, however once the error Method invocation might have failed due to dropped connection. Failing because noRetry option was passed to Meteor.apply. is thrown, it doesn’t return the response even after the stub is resolved either in onResultRecieved or asyncCallback, please suggest
Once it’s failed, it’s failed. All you can do is catch the error and retry. You may need to alter the way your method behaves and make use of an asynchronous return (to satisfy the timeout), followed by another event when the method completes. That even could be inserting a database document, or using Redis as a notification channel (look at Redis Vent in cultofcoders:redis-oplog or chatra:redpubsub).
Having said that, if your method is CPU intensive, it will likely end up blocking the Node event loop and killing performance of the application for all users. In that situation you’d be better off using a separate worker process - which could be another Meteor app on a different port. You would still use the technique above, but the worker would be sending the completion event.
1 Like
My stub is a promise, it is performing unzipping of a zip of 300 txt and 300 json files, mapping txt with json files by filename and inserting into db. Even if the stub is a promise, Meteor doesn’t have the syntax of .then() or .catch() to wait. Can Session variables help me here, or else like u said have to go for service worker. Thanks for the reply.
Yes it does - that’s just a Promise chain - or you can use async/await syntax. However, it doesn’t really matter what the implementation detail is, it’s more how to architect it for this use case.
My Stub as follows :
extractDocumentAndJSONUsingYauzl: async (data) => {
const { fileData, base_set_id } = data;
const mainPromise = new Promise((resolve, reject) => {
try {
if (Meteor.isServer && Meteor.userId()) {
const bound = Meteor.bindEnvironment(callback => {
callback();
});
const meteorUserId = Meteor.userId();
let docs = [];
let jsonFiles = {};
fs.writeFileSync("/tmp/test.zip", fileData, { encoding: "binary" });
yauzl.open("/tmp/test.zip", { lazyEntries: true }, (err, zipfile) => {
if (err) throw err;
zipfile.readEntry();
zipfile.on("entry", entry => {
let fileName = entry.fileName;
if (/\/$/.test(fileName)) {
// Directory file names end with '/'.
// Note that entires for directories themselves are optional.
// An entry's fileName implicitly requires its parent directories to exist.
zipfile.readEntry();
} else if (fileName[0] === "_" && fileName[1] === "_") {
zipfile.readEntry();
} else if (fileName.indexOf(".txt") !== -1) {
fileName =
fileName.indexOf("/") !== -1
? fileName.split("/").pop()
: fileName;
const notExist =
findIndex(docs, o => o.title === fileName) === -1;
if (notExist) {
const id = shortid.generate();
let writtableStream = fs.createWriteStream(
"assets/app/document_sets/" + id + ".txt"
);
zipfile.openReadStream(entry, (err, readStream) => {
if (err) {
console.log("error on opening read stream", err);
reject(err);
}
// readStream.on("end", () => {
// console.log("on end");
// zipfile.readEntry();
// });
readStream.pipe(writtableStream);
savingDoc(
writtableStream,
fileName,
meteorUserId,
base_set_id,
id
)
.then(resp => {
docs = [...docs, resp];
zipfile.readEntry();
})
.catch(err => {
console.log("err", err);
zipfile.readEntry();
});
});
} else {
console.log("encountered an existing file");
zipfile.readEntry();
}
} else if (fileName.indexOf(".json") !== -1) {
fileName =
fileName.indexOf("/") !== -1
? fileName.split("/").pop()
: fileName;
let jsonFile = {};
zipfile.openReadStream(entry, (err, readStream) => {
if (err) {
console.log("error on opening read stream", err);
reject(err);
}
// readStream.on("end", () => {
// zipfile.readEntry();
// });
readStream
.pipe(JSONStream.parse("$*"))
.on("data", jsonData => {
jsonFile[jsonData.key] = jsonData.value;
})
.on("end", () => {
jsonFiles[fileName] = jsonFile;
zipfile.readEntry();
})
.on("error", () => {
console.log("JSON stream error");
zipfile.readEntry();
});
});
} else {
zipfile.readEntry();
}
});
zipfile.on("close", () => {
console.log("jsonFiles", jsonFiles);
fs.unlink(
"/tmp/test.zip",
bound(err => {
if (err) reject(err);
const valid = filesValid(docs, jsonFiles);
if (!valid) {
const err = new Meteor.Error(
"files-invalid",
"Some of the files are invalid."
);
reject(err);
}
if (valid && docs.length) {
const entriesClone = docs.slice(0); // clone collection
(function processOne() {
const doc = entriesClone.splice(0, 1)[0]; // get the first record of coll and reduce coll by one
const { title, document_id } = doc;
const fileNameWithoutExtension = title.slice(0, -4);
let tempToken = {};
tempToken = jsonFiles[fileNameWithoutExtension + ".json"];
tempToken.title = title;
tempToken.user_id = meteorUserId;
tempToken.document_id = document_id;
Meteor.call(
"addDocuments",
base_set_id,
document_id,
"Labelling",
(err, resp) => {
if (err) {
console.log("err", err);
var reason = new Meteor.Error(
"add-document-error",
"Adding document failed"
);
reject(reason);
}
if (resp) {
Meteor.call(
"insertPreProcessedTokenNew",
tempToken,
(errorToken, resultToken) => {
if (errorToken) {
throw new Meteor.Error(
"insert-pre-processed-document-error",
"Inseting pre-processed document failed"
);
}
if (resultToken) {
if (entriesClone.length == 0) {
console.log("LETS RESOLVE");
resolve("Files uploaded Successfully");
} else {
processOne();
}
}
}
);
}
}
);
})();
}
})
);
});
zipfile.on("end", () => {});
zipfile.on("error", () => {
throw new Meteor.Error(
"prellabelled-error",
"Error reading Zip File"
);
});
});
}
} catch (e) {
console.log("caught in exception", e);
reject(e);
}
const savingDoc = async (
writtableStream,
fileName,
meteorUserId,
base_set_id,
id
) => {
const promise = new Promise((resolve, reject) => {
try {
writtableStream.on("finish", () => {
Documents.addFile(
"assets/app/document_sets/" + id + ".txt",
{
fileName: fileName,
type: "text/plain",
userId: meteorUserId,
fileId: id,
meta: {
createdAt: moment().format(),
documentSetId: base_set_id,
name: fileName,
status: "Labelling",
updatedAt: moment().format()
}
},
(error, fileRef) => {
if (error) {
console.log("error", error);
var reason = new Meteor.Error(
"files-invalid",
"Some of the files are invalid."
);
reject(reason);
}
if (fileRef) {
resolve({ document_id: fileRef._id, title: fileRef.name });
}
}
);
});
} catch (e) {
reject(e);
}
});
const result = await promise;
return result;
};
const filesValid = (entries, jsonFiles) => {
if (!entries.length) {
return false;
}
for (let i = 0; i < entries.length; i++) {
const fileName = entries[i].title
? entries[i].title.slice(0, -4)
: null;
if (fileName === null) {
return false;
}
if (jsonFiles[fileName + ".json"] === undefined) {
return false;
}
}
return true;
};
});
const result = await mainPromise;
return result;
},
My Method call as follows:
const reader = new FileReader();
reader.onload = fileLoadEvent => {
Meteor.apply(
"extractDocumentAndJSONUsingYauzl",
[
{
fileData: reader.result,
base_set_id: data.baseSetId
}
],
{
noRetry: true,
throwStubExceptions: false
onResultReceived: (err, resp) => {
console.log("onErrorReceived", err);
console.log("onResultReceived", resp);
},
returnStubValue: false
},
(error, result) => {
if (error) {
if (error.error === "invocation-failed") {
console.log("keep waiting", error);
} else {
console.log("error", error);
}
}
if (result) {
console.log("result", result);
}
}
);
};
reader.readAsBinaryString(data.zip);
Is there a way to do it as follows?
Meteor.apply('stubName',[args],{options}).then().catch()
Thank you so much for the replies!
You can test this out if you have running Meteor app, just you have to make sure that the zip should contain txt and json with the same name, say if you have abc.txt, it should also have abc.json
It looks like you’re calling that method from the server (given the fileReader). In which case, why use a Meteor method, when you could just use a function?
No, but you could wrap it in a Promise.
Oh, apologies for the confusion. Actually that FileReader part is written on the client i.e React side as FileReader is a browser property
1 Like