How to dynamically change meta tags value in every page and make it visible on view source code for scrapping?

help
1

Hi,

I’m not sure if this already an old issue that has been resolved or not. But I tried some research searching the resolved issue that is related to this one, but still I can’t find.

We are able to add meta tags on template rendering but when we scrap the site using facebook debugger or google webmaster tool the meta tags are lost or gone on the page. We already use difference packages like dochead and some other packages that supports meta tags but still no luck, I still can’t see meta tags per page when I view the source code. Does anyone here have idea on how to fix it? I am thinking about inserting a function before it’ll render UI but not sure where to insert it in the meteor.

Any suggestion are welcome. :slight_smile:

Thanks in advance.

2

This is what we do:

// server.js

import { WebApp } from 'meteor/webapp';

const serverRendering = (req, res, next) => {
  try {
    const ua = req.headers['user-agent'];
    
    // You will want to write a regex that makes sense for your app
    if (/bot/i.test(ua)) { 
      // You will need to intercept the path from `req.originalUrl` here in
      // order to know what html to build
      const html = 'Build your html here, with your meta tags';
      res.statusCode = 200;
      res.setHeader('Content-Type', 'text/html');
      res.end(html);
    } else {
      next();
    }

  } catch (err) {
    console.log(err);
  }
};


// attach the handler to webapp
WebApp.connectHandlers.use(serverRendering);
3

Hi,

Thanks for the quick reply, I just confused on how you validate that the certain request is via curl or whatever and not a request from the browser? Because I tried your suggestion but right now it override the UI when directly put the link on the browser and visit the site.

However, your suggestion is great.

Thanks

4

I’d suggest logging out ua, as you go through your different scrapers. You will need to adjust this line:

if (/bot/i.test(ua)) {

So that the bots are directed inside of the block, and so that your normal browsers are directed outside of the block to the next() call. I’m surprised your browser is passing the current regex, unless you are using some kind of user-agent modification on your browser to spoof a robot.

5

Hi, thanks. Most social media scrapper does have their own link included in the user-agent, I just tested that the facebook does this thing, so I think I can use that since we only need social media website to scrap our site and of course google seo. Thanks a lot for your time and this useful way for the meta tags solution. :slight_smile:

1 Like
6

I know almost 5 years ago…

what do you think of this?

/**
* /server/meta-tags.js
* package used : meteorhacks:inject-initial
*/

import { WebApp } from 'meteor/webapp';

WebApp.connectHandlers.use("/", function (req, res, next) {
    try {
        const url = req.url;
        const locale = url.split('/')[1] || 'fr';
        const slug = url.replace('/' + locale, "") || "/";

        const pages = Pages.findOne({ slug: slug, locale: locale });
        if (pages?.metaTags) {
            Inject.rawHead('metaTags',
                `<!-- metaTags -->
${pages.metaTags}
<!-- metaTags end -->
        `);
        }

        next();
    } catch (err) {
        console.log(err);
    }
});
7

I use webapp to do all of this and it works really good. It’s quite understated I feel, but super powerful - webapp also powers my public APIs, plug-ins and RSS. I also do crypto via it.

If you want the entire page to be spiderable you need SSR which can be a bit of a headache with Meteor but is possible. Look around the boilerplate examples, @captainn has a good one on his github

1 Like