Facebook, spiders and other creepy crawlers

help
1

Hello everyone,

Recently I discovered that Google can now successfully crawl javascript apps and index them and everything. Excellent, less work for everyone!

Now I’m trying to make my app to have pages that can be shared on Facebook. Copy-paste the link on my status, and see a nice preview with the original content right bellow my post, like FB does with all Server Side Rendered apps.

I’ve been thinking of using the spiderable package in the same manner that I used to for google, but unfortunately Facebook won’t attempt to redirect to the urls with _escaped_fragment_=

I was wondering if there is a way to intercept the connection to the server and forced the FB crawler to redirect to a url with the escaped_fragment query string but I can’t seem to figure a way without hacking the platform.

This is the pseudo-code of what I’m having in mind:

if(Meteor.isServer){
   Meteor.on('clientConnect', function(){
      if(useragent === 'FBbot' && !request.uri.contains('_escaped_fragment_=')){
         res.redirectTo(res.uri + '?_escaped_fragment_=');
         DontDDP();
      }
   });
}

Is what I want to do even possible somehow?

2

I believe you can get what you want without having to worry about spiderable.

Facebook looks for Open Graph information within the <head></head> tags of the page to populate their fancy previews. You can insert the Open Graph information after a particular route/page of your app loads with javascript.

You could either use this package and get all of the other SEO goodness or rummage through and pick out the relevant bits: https://github.com/DerMambo/ms-seo

3 Likes
3

I’m assuming that you want to have custom images/descriptions/etc. per route when you share a given page to facebook. Is that correct? The images/descriptions and other things are controlled with og (opengraph) metatags. Like @tomwasd said, you can simply insert those tags when the page is rendering and facebook can pick up on that.

If you don’t want custom images/descriptions/etc. you can simply hardcode your og tags in your main template and everything else should fall into place.

4

Unfortunately my problem is not the lack of metatags but the fact that the FB bot does not render my application. All it sees is the initial html and nothing more :frowning:

Nothing can be done unless something pre-rendered is served to the FB bot or until FB starts rendering the pages it crawls

5

The package (and the technique it uses) does work - honest!

If you go to the Open Graph debugger here: https://developers.facebook.com/tools/debug/og/object/

And paste in this url: https://pedlar.co/d/KMPPSGvkJpihqAkWg/gro-anywhere-blackout-blind

And click ‘Fetch new scrape information’

It’ll show all the information you’re after.

That url is from my site which uses Meteor and the same technique as the package I linked to above.

1 Like
6

Whoops, sorry @tomwasd, I paid too much attention to the paragraph telling about the open-graph tags, and ignored the package stated at the bottom. I’ll check it out once I get home, thanks!

8

It worked! Just keep in mind that spiderable must be installed for the entire thing to work :smile:

1 Like