Need help with Spiderable/PhantomJS error on Modulus


#1

I run into this error when I test my server with the escaped fragment url. It works on my localhost, but when I deploy to modulus, it fails with this error:

spiderable: phantomjs failed: Error: Command failed: Can’t open '/dev/stdin’
at ChildProcess.exithandler (child_process.js:658:15)
at ChildProcess.emit (events.js:98:17)
at maybeClose (child_process.js:766:16)
at Process.ChildProcess._handle.onexit (child_process.js:833:5)
stderr: Can’t open ‘/dev/stdin’

I’ve tried contacting modulus support, but they’ve been unresponsive for the last two days. Making me wonder if it’s worth switching to another platform. I could really use help figuring out what the problem is.

I’ve tried installing the spiderable-ssl package from this google groups thread, but it hasn’t helped.

Would really appreciate any guidance!


#2

I tried out the ongoworks:spiderable package and got it to work. Thread here: https://github.com/meteor/meteor/issues/2429. Wondering if there’s a better long term solution here…


#3

This a common issue when deployed in docker env. This arise in modulus recently because now they are running apps inside docker containers.

ongoworks:spiderable is the solutions works great.

I can’t understand how it can’t be reproduced.

Well, this issue doesn’t have a clear reproduction, so it would be a fair amount of work to even figure out if it’s fixed yet or not.


#4

Arunoda - thanks for the clarification on modulus using docker now.


#5

Do you have any updates on this? I have tried using both spiderable packages on Modulus without any success.

dfischer:phantomjs + ongoworks:spiderable:

spiderable: phantomjs failed: Error: Command failed: 
    at ChildProcess.exithandler (child_process.js:658:15)
    at ChildProcess.emit (events.js:98:17)
    at maybeClose (child_process.js:766:16)
    at Process.ChildProcess._handle.onexit (child_process.js:833:5) 
stderr:

dfischer:phantomjs + spiderable:

spiderable: phantomjs failed: Error: Command failed: Can't open '/dev/stdin'
    at ChildProcess.exithandler (child_process.js:658:15)
    at ChildProcess.emit (events.js:98:17)
    at maybeClose (child_process.js:766:16)
    at Process.ChildProcess._handle.onexit (child_process.js:833:5) 
stderr: Can't open '/dev/stdin'

#6

same story;
Digital Ocean + mupx, no specific spiderable installed, my guess it is installed with mupx

spiderable: phantomjs failed: Error: Command failed:
    at ChildProcess.exithandler (child_process.js:658:15)
    at ChildProcess.emit (events.js:98:17)
    at maybeClose (child_process.js:766:16)
    at Process.ChildProcess._handle.onexit (child_process.js:833:5)
stderr:
[www.placetee.us] spiderable: phantomjs failed: Error: Command failed: Can't open '/dev/stdin'

    at ChildProcess.exithandler (child_process.js:658:15)
    at ChildProcess.emit (events.js:98:17)
    at maybeClose (child_process.js:766:16)
    at Socket.<anonymous> (child_process.js:979:11)
    at Socket.emit (events.js:95:17)
    at Pipe.close (net.js:466:12)
stderr: Can't open '/dev/stdin'

with ongoworks:spiderable

spiderable: phantomjs failed: Error: Command failed:
    at ChildProcess.exithandler (child_process.js:658:15)
    at ChildProcess.emit (events.js:98:17)
    at maybeClose (child_process.js:766:16)
    at Process.ChildProcess._handle.onexit (child_process.js:833:5)
[www.placetee.us] stderr:
[www.placetee.us] spiderable: phantomjs failed: Error: Command failed: Can't open '/dev/stdin'

    at ChildProcess.exithandler (child_process.js:658:15)
    at ChildProcess.emit (events.js:98:17)
    at maybeClose (child_process.js:766:16)
    at Socket.<anonymous> (child_process.js:979:11)
    at Socket.emit (events.js:95:17)
    at Pipe.close (net.js:466:12)
stderr: Can't open '/dev/stdin'

for some reason ongoworks:spiderable uses same /dev/stdin


#7

The error has come back up for me as well after working for the last few months. Not sure if Modulus changed some server configs again or if these packages are conflicting with some new package I installed recently.

@stanp @OminStyle @arunoda did you guys figure out what the issue was?

spiderable: phantomjs failed: Error: Command failed:
at ChildProcess.exithandler (child_process.js:658:15)
at ChildProcess.emit (events.js:98:17)
at maybeClose (child_process.js:766:16)
at Process.ChildProcess._handle.onexit (child_process.js:833:5)
stderr:

Error: Meteor code must always run within a Fiber. Try wrapping callbacks that you pass to non-Meteor libraries with Meteor.bindEnvironment.
at Object.Meteor.nodeCodeMustBeInFiber (packages/meteor/packages/meteor.js:966:1)
at [object Object].
.extend.get (packages/meteor/packages/meteor.js:978:1)
at [object Object].RouteController.lookupOption (packages/iron_router/packages/iron_router.js:122:1)
at new Controller.extend.constructor (packages/iron_router/packages/iron_router.js:82:1)
at [object Object].ctor (packages/iron_core/packages/iron_core.js:132:1)
at Function.Router.createController (packages/iron_router/packages/iron_router.js:789:1)
at Function.Router.dispatch (packages/iron_router/packages/iron_router.js:1314:1)
at Object.router (packages/iron_router/packages/iron_router.js:603:1)
at next (/mnt/app/node_modules/connect/lib/proto.js:190:15)
at packages/ongoworks_spiderable/packages/ongoworks_spiderable.js:159:1
at ChildProcess.exithandler (child_process.js:662:7)
at ChildProcess.emit (events.js:98:17)
at maybeClose (child_process.js:766:16)
at Process.ChildProcess._handle.onexit (child_process.js:833:5)


#8

my issue was with mupx, I have switched to mup instead. It looks like mupx installs the standard spiderable version and it can not run inside docker. Even if I have removed any spiderable version, I still got an error with spiderable

So mup is OK and I am using jazeee:spiderable-longer-timeout for now.
Works more or less OK, except when googlebot hits too many requests simultaneously, in that case I still got phantomjs error from time to time, but at least the previous results from cache are good.


#9

Thanks.

Turns out it was an issue with upgrading a different package (https://atmospherejs.com/okgrow/router-autoscroll) that conflicted with iron router. Once I removed that, it worked again.