This is more of a general SEO question, as I’ve been having a heck of a time figuring out what is and isn’t working with regard to my new website getting found by web crawlers. I’m running on Galaxy and have the mdg:seo package added, so I’m assuming that the prerender.io setup is working fine. I’ve also added the kadira:dochead package to set my per-page meta tags in my templates’ onCreate() methods.
When I load the website and inspect the <head> tag from my browser, I see these values filled in just like they should be. However, when I run “Fetch as Google” (part of the Google Webmaster Tools), the downloaded HTTP response shows the un-rendered HTML of the page (i.e. the default head tag and the one long Meteor script in the body). I also did a few "Fetch and Render"s, with mixed results. Sometimes the full web page renders in the preview window, and sometimes I get the following result:
I get the spinner icon when data is still loading normally, but this screenshot is from a page with no subscriptions. I’ve tried implementing FastRender, but whether I’m making router-level subscriptions from the server or not, the behavior of Google’s Fetch and Render seems pretty intermittent. I don’t know whether FastRender is helping, or how to actually confirm that I’m getting crawled properly.
P.S. I’m using FlowRouter and Blaze, in case that’s relevant.
I too would like to know. I thought you could just append ?_escaped_fragment_ onto your URL, then view the source and you’d see the full HTML. But that doesn’t seem to work.
Hi @danahmadi, I do also have the <meta name="fragment" content="!"> tag in my head file. I believe that accomplishes the same thing as ?_escaped_fragment_=? For what it’s worth, the facebook crawler (after quite a bit of coaxing) does show that it’s rendering the final HTML, and the fully-rendered screenshot occasionally appears on Google’s tool. Using curl on my terminal confirms that it’s being prerendered. The thing that’s throwing me off is why Google’s preview screenshot would show the webpage before waiting for subscriptionsReady (even when some pages don’t run any subscriptions), and why the “Fetch” result, in either case, would still return the unrendered HTML?
Hello! Todd from Prerender.io here. “Fetch as Google” has a known bug where they don’t check for the fragment meta tag and re-request the page with the ?_escaped_fragment_= query parameter even though Googlebot does not have that issue. So with Fetch as Google you’ll need to input the ?_escaped_fragment_= query param on the end of the URL that they are fetching and that should show your prerendered page. If that works and if you have <meta name="fragment" content="!"> in the <head> of your pages, then the real Googlebot will work just fine.
@ffxsam that sounds like the prerender.io middleware isn’t working correctly yet if it’s not proxying the request when accessing an escaped fragment URL. Do you have any other caching or packages returning your index page before the prerender middleware is executed?
I have the mdg:seo package installed and am not seeing a “Prerender” anywhere in my Galaxy logs (site is at https://www.shopte.ch)… any updates on this issue?
I had this issue in the past and received a note back from meteor that there was a “bug” - if you don’t include a settings file, then it doesn’t work. At that time, and on that site, I wasn’t using one. I submitted an object {} as my settings file and it worked. Unusual anyone would submit a site without such a file, but good to check if anyone else finds themselves here.
Hi,
since mid-june (20th) all my urls are not indexed anymore by google (which ius scary)
I try to understand what was going wrong with my site (which didnt change)
Then I noticed in prerender.io that googlebot do not query urls with ?_escaped_fragment_=
It might has something to do with google Mobile First crawling they are working on for the last years.
Some people solved this by adding googlebot to their nginx prerender script.
Now I have to figure out how to do that with Meteor Galaxy
curl -L -A “Googlebot/2.1 (+http://www.google.com/bot.html)” https://example.com
If you see a script output you know you need to update your settings or fix something with your configurations.
Regarding updating settings or fixing your configurations I will offer some advice if using Meteor Galaxy or something else.
If you are using Meteor Galaxy your best bets are as follows:
Follow the official Meteor Guide to make sure you have properly set things up and then test using the curl commands I listed above.
If things are not working after a review of the official guide and you added your Meteor mdg:seo package sometime during or before July 2018. I would then recommend removing the package using “meteor remove mdg:seo” and then re-adding the package. I would not do an update as it does not always work.
Quick note, the Meteor seo package was recently update as well as the prerender-node package. The links show that the Meteor seo and prerender-node packages being updated July 2018 to prerender version 3.2.0. The previous version per this Meteor Forum Post was 2.2.2. For reasons on what brought about this change check out this prerender-io post on Google Support.
You should be good to go after doing these things. If it does not work and you are using Meteor Galaxy it may be something else going on with your configurations.
If you are not using Meteor Galaxy, I would again review the Meteor Guide link and check out this Medium post that shows some configurations and other tips/insight that may be useful. In addition, review the advice provided by the prerender-io post on Google Support.
