Meteor with Node Packages that save local files (ie. puppeteer)

help
1

I am trying to use Meteor with puppeteer in localhost. It is a plain new Meteor application for testing.

This node package has a function that, when using node.js alone, without Meteor, saves a screenshot file to disk. In Meteor, where is this file going to be saved? Is there any conflict?

I am running a plain puppeteer example on main.js (server):

import { Meteor } from 'meteor/meteor';
import puppeteer from 'puppeteer';

Meteor.startup(() => {
 const x=1;
 const screenshot = 'booking_results.png'

 try {
  (async () => {
   const browser = await puppeteer.launch()
   const page = await browser.newPage()
   await page.goto('https://booking.com')
   await page.type('#ss', 'Belo Horizonte')
   await page.click('.sb-searchbox__button')
   await page.waitForSelector('#hotellist_inner')
   await page.screenshot({ path: screenshot })
   const hotels = await page.$$eval('span.sr-hotel__name', anchors => {
    return anchors.map(anchor => anchor.textContent.trim()).slice(0, 10)
  })
   console.log(hotels)
   await browser.close()
   console.log('See screenshot: ' + screenshot)
  })()
 } catch (err) {
 console.error(err)
 }
});

But the file is never saved. Any ideas?

1 Like
2

I have the same issue. Running an example from puppeteer’s documentation generates a screenshot only when I run the file with >>node scrape.js, but no screenshot is saved when run as part of my meteor app.

My code is in /imports/startup/server/scrape.js. I added in the console.log to make sure I wasn’t crazy - these are both printed to console successfully, but no screenshot is saved when I execute the code with >>meteor run.

const puppeteer = require("puppeteer");

console.log("file got loaded");
(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto("https://example.com");
  await page.screenshot({ path: "example.png" });

  await browser.close();
  console.log("async got executed");
})();

Any help is much appreciated!

1 Like
3

I’m not familiar with puppeteer, but I’d suggest using an absolute path to which you know you have write permissions, e.g., /tmp/example.png - this way you know where the file should end up.

4

Since you’re in dev on localhost, you might want to check under /path/to/your/app/.meteor/local/build/programs/server. That’s the working directory when running Meteor in development.

1 Like
5

Figured it out! You can use the ostrio:files package. Puppeteer’s page.screenshot returns a promise, which can resolve to a buffer you can write to your local file system (or 3rd party storage service) using ostrio:files.

/imports/api/images/images.js:

import { FilesCollection } from "meteor/ostrio:files";

// note: default storage path is '.meteor/local/build/programs/server/assets/app/uploads/Images', see:
// https://github.com/VeliovGroup/Meteor-Files/wiki/Constructor
const Images = new FilesCollection({ collectionName: "images" });

export default Images;

/imports/startup/server/scrape.js:

import Images from "../../api/images/images";

const puppeteer = require("puppeteer");

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto("https://example.com");
  bufferPromise = await page.screenshot({ path: "example.png" });
  Images.write(
    bufferPromise,
    {
      fileName: "example.png",
      type: "image/png"
    },
    function(writeError, fileRef) {
      if (writeError) {
        throw writeError;
      } else {
        console.log(
          fileRef.name + " is successfully saved to FS. _id: " + fileRef._id
        );
      }
    }
  );

  await browser.close();
})();

Happy scraping!

2 Likes