How to run a background script with `child_process.fork` so that it finds node_modules on galaxy?

help
1

I have a simple but long running task that I want to run as background process using child_process.fork. The forked process needs to import some node modules (e.g. clone).

I decided to put the script into the private directory. The dependencies are also used in the normal code, so they should be in the node_modules folder. However, if I run it on galaxy, I get an Error:

Error: Cannot find module 'clone'.

My questions:

  1. Where to best place the script (is private a good place?)?
  2. What to do so that the script finds node_modules?
2

Hi @scharf, I have the same pb. If you have found a solution, can you share it please?

3

Here is the (typescript) code that I use:

import { Meteor } from 'meteor/meteor';

export interface SpawnProcessResult {
    cmd: string;
    args: string[];
    error?: any;
    stdout: string;
    stderr: string;
    code?: number; // return code
}

export interface SpawnProcessCallback {
    (err: Meteor.Error, result?: SpawnProcessResult): void;
}

function unquote(str: string): string {
    return str.replace(/\\./g, function(s) {
        switch (s[1]) {
            case 'n':
                return '\n';
            case 't':
                return '\t';
            case 'r':
                return '\r';
        }
        return s[1];
    });
}
/**
 * Splits an argument string so that it can contain quotes etc
 * @param args
 * @returns {string[]}
 */
export function splitArgs(args: string): string[] {
    const arglist = args.split(/(\s+|"(?:[^"\\]+|\\.)*"|'(?:[^'\\]+|\\.)*')/);
    const result: string[] = [];
    for (let i = 0; i < arglist.length; i += 2) {
        const a0 = arglist[i];
        // this is a space separated arg
        if (a0) {
            result.push(a0);
        }
        // this is a quoted arg
        const a1 = (arglist[i + 1] || '').trim();
        if (a1) {
            result.push(unquote(a1.slice(1, -1)));
        }
    }
    return result;
}

export function callSpawnProcess(cmd: string, args: string[], cb: SpawnProcessCallback) {
    Meteor.call('spawnProcess', cmd, args, cb);
}

and on the server I have:

import { Meteor } from 'meteor/meteor';
import { hasPermissionTo } from 'YOUR_PERMISSION_SYSTEM';
import { SpawnProcessResult } from './SpawnProcess';
import Future = require('fibers/future');

const spawn = require('child_process').spawn;

export function spawnProcess(cmd: string, args: string[]): SpawnProcessResult {
    const future = new Future<SpawnProcessResult>();
    const result: SpawnProcessResult = {
        cmd,
        args,
        stdout: '',
        stderr: '',
    };

    const prog = spawn(cmd, args, { shell: '/bin/sh' });

    prog.stdout.on('data', (data: string) => {
        result.stdout += data;
    });

    prog.stderr.on('data', (data: string) => {
        result.stderr += data;
    });

    prog.on('error', (error: any) => {
        result.error = error;
    });

    prog.on('close', (code: number) => {
        result.code = code;
        future.return(result);
    });

    return future.wait();
}

Meteor.methods({
    spawnProcess(this: MethodThis, cmd: string, args: string[]): SpawnProcessResult {
        // VERY IMPORTANT to check permissions here using your permission system
        if (!hasPermissionTo('permToExecuteCommandsOnServer')) {
            throw new Meteor.Error(403, `User ${this.userId} cannot execute commands!`);
        }
        // see https://gist.github.com/possibilities/3443021#unblocking-meteormethods
        this.unblock();
        return spawnProcess(cmd, args);
    },
});
4

Hmm, not sure if it solves the node_modules problem, because I run only non-node processes

5

Indeed! In your first post, you were using child_process.fork() to run a NodeJS script. child_process.spawn() doesn’t fit my needs. Thank you anyway @scharf for answering me.

6

I managed to solve this, the solution is here:

1 Like