Make server respond with 404 status on unmatched URLs

Like the title says, I want the server to respond with a 404 status code to requests for unidentified URLs. There must somehow be a way to do this using the WebApp API, but the documentation is very sparse and I cannot figure out how.

Most client side routers already include a 404 handler, and using them will make your life much simpler.
For example, ostrio:flow-router-extra does this like so:

// Create 404 route (catch-all)
FlowRouter.route('*', {
  action() {
    // Show 404 error page using Blaze
    this.render('notFound');

    // Can be used with BlazeLayout,
    // and ReactLayout for React-based apps
  }
});

If you really want to do this manually on the server side, there’s a lot of checking you need to do:

import { WebApp } from 'meteor/webapp';

import fs from 'fs';
import path from 'path';
import url from 'url';

const publicDirPath = path.join(
    __meteor_bootstrap__.serverDir,
    '../web.browser/app'
);
WebApp.connectHandlers.use(function(req, res, next) {
    const urlParts = url.parse(req.url);
    if (urlParts.pathname === '/') {
        // We just want to serve the app bundle
        // Use meteor default behaviour
        return next();
    }

    if (fs.existsSync(path.join(publicDirPath, urlParts.pathname))) {
        // File exists in the public dir
        // Use meteor default behaviour
        return next();
    }

   
    // Check for other routes in connect's middleware stack
    if (
        WebApp.connectHandlers.stack.some(({ route }) => route === urlParts.pathname) ||
        WebApp.rawConnectHandlers.stack.some(({ route }) => route === urlParts.pathname)
        ) {
        // If found, we just yield control to the next middleware.
        return next();
    }

    // Else we 404, if you want to serve a 404 page, add that here as well.
    res.statusCode = 404;
    res.statusMessage = 'Not found';
    res.end();
});

If you use a router, you will need to expose the routes to the server and check those routes as well.

Be really careful for things like meteor’s built in dynamic imports fetch route that you don’t overwrite it as well

4 Likes

Thank you very much for that code and the help!

Using react-router on the client, I can easily render a 404 template for unmatched routes. The problem is that the HTTP header remains the same with a 200 status code. Are you perhaps saying that there is an easier way around this?

If you’re doing server side rendering (SSR), it looks like React-router can handle this quite well: https://reacttraining.com/react-router/web/guides/server-rendering/

Integrated with Meteor’s server-render package would look something like this:

import React from 'react';

import { renderToString } from 'react-dom/server';
import { onPageLoad } from 'meteor/server-render';
import { StaticRouter } from 'react-router';

import App from './App';

onPageLoad(sink => {
    const context = {};

    sink.renderIntoElementById(
        'app', // Assuming you're rendering into an element where id="app"
        renderToString(
            <StaticRouter location={sink.request.url} context={context}>
                <App />
            </StaticRouter>
        )
    );
    if (context.status) sink.setStatusCode(context.status);
    if (context.url) sink.redirect(context.url);
});

And App would use something like the Status and NotFound components on the page linked above


Unfortunately the docs for server-render don’t explain that you can use sink.setStatusCode, but the github page has it: https://github.com/meteor/meteor/tree/master/packages/server-render

3 Likes

Thanks again for that. Still considering whether to use server-render or prerender.io, so that may well be useful!

if you have react and react-router, use server-render

also with react and flow-router its feasible to use server-render

use prerender as last resort

Why not both? I would prefer to use client-side rendering for the main functionality of my app, but SSR is probably better for certain features/pages, and I would need something like prerender.io to make the content of the client-side routes accessible to search engine web-crawlers.

Thanks you for this server side code I’m using it now.
But i don’t understand the second and third condition.

The code inside those “if” is never executed.
Even when i go to localhost:3000/myimage.jpg. (located in /public/myimage.jpg).

And i’m thinking if they are useful or no and should i remove them ? Can you give me an example that i can try that execute those conditions ?

TL,DR; Is there a difference between this :

WebApp.connectHandlers.use(function (req, res, next) {
  const urlParts = url.parse(req.url);
  if (allRoutes.includes(urlParts.pathname)) {
    console.log('test 1');
    return next();
  }

  if (fs.existsSync(path.join(publicDirPath, urlParts.pathname))) {
    console.log('test 2');
    return next();
  }

  if (
    WebApp.connectHandlers.stack.some(({ route }) => route === urlParts.pathname) ||
    WebApp.rawConnectHandlers.stack.some(({ route }) => route === urlParts.pathname)
  ) {
    console.log('test 3');
    return next();
  }

  console.log('test 4');
  res.statusCode = 404;
  res.statusMessage = 'Not found';
  return next();
});

And this :

WebApp.connectHandlers.use(function (req, res, next) {
  const urlParts = url.parse(req.url);
  if (allRoutes.includes(urlParts.pathname)) {
    console.log('test 1');
    return next();
  }

  console.log('test 4');
  res.statusCode = 404;
  res.statusMessage = 'Not found';
  return next();
});

Thanks you a lot, this is very interresting and i want to understand this !

Hey @GaetanRouzies, It’s been a while since that answer, but I’ll try to remember :upside_down_face:

Firstly, a question for you. Where does allRoutes come from?
In my answer, the third block is supposed to catch any matching routes, so we don’t accidently 404 them. Using allRoutes would do the same that but might miss internal routes or ones generated by other packages.

The second block is for normal serving of assets from the /public folder, but that might be unnecessary if Meteor catches and handles them before our middleware gets hit.
Which is likely the case if you’re never seeing the console log there.

Looking at this again, the normal way to do this in an express app is to make a 404 handler the last route on the stack. We’ve got a little less control over the middleware stack, and still want any legitimate urls to resolve, so it’s not as simple as that but it’s worth a try to add it last and see if all the normal routes are already handled.
You’ll still need a saved list of all server and client side routes (like I think you’ve done), so the client bundle is still served for legitimate pages.

Thanks you, you made me understand with your answer. I will recap with my own words, maybe it will help some people.


# Goal: Make the server respond with 404 status on unmatched URLs

## Description
When the we receive a request on our server, we check if the requested url is part of “Client side declared routes” or “Server side declared endpoints”. If not, we return a 404 status.

# In practice
## Step 1: Check the client routes
Here are for example my declared routes in Flow Router using Blaze:

// client/router.js
FlowRouter.route('/', { /* ... */ });
FlowRouter.route('/sport', { /* ... */ });
FlowRouter.route('/nutrition', { /* ... */ });
FlowRouter.route('/admin', { /* ... */ });

Server side, i created an array of my declared client routes.
And i check if the requested url is part of my client routes.

// server/webapp.js
const clientRoutes = ['/', '/sport', '/nutrition', '/admin'];

WebApp.connectHandlers.use(function (req, res, next) {
  const urlParts = url.parse(req.url);

  // Check all the client routes (['/', '/sport', '/nutrition', '/admin'])
  if (clientRoutes.includes(urlParts.pathname)) {
    return next();
  }

  res.statusCode = 404;
  res.statusMessage = 'Not found';
  return next();
});

## Step 2: Check the declared endpoints from the server
WebApp.connectHandlers.stack contains all the declared endpoint in the server. And we can have some package generating some endpoints too.

For example, i declared an endpoint.

// server/endpoints.js
WebApp.connectHandlers.use('/endpoint1', function (req, res, next) {
  res.writeHead(200);
  res.end('Hello');
});

To check if the requested url is part of my declared endpoints (and maybe part of other packages endpoints), i use WebApp.connectHandlers.stack.

So here’s the final code:

// server/webapp.js
const clientRoutes = ['/', '/sport', '/nutrition', '/admin'];

WebApp.connectHandlers.use(function (req, res, next) {
  const urlParts = url.parse(req.url);

  // Check all the client routes (['/', '/sport', '/nutrition', '/admin'])
  if (clientRoutes.includes(urlParts.pathname)) {
    return next();
  }

  // Check all the declared endpoints (['/endpoint1', '/maybe_other_endpoint_from_package'])
  if (WebApp.connectHandlers.stack.some(({ route }) => route === urlParts.pathname) ||
    WebApp.rawConnectHandlers.stack.some(({ route }) => route === urlParts.pathname)) {
    return next();
  }

  res.statusCode = 404;
  res.statusMessage = 'Not found';
  return next();
});

PS: If you have dynamic client routes (ex: /posts/:postId), you have the update the array “clientRoutes”, so it contains all the routes. (ex: Hook on a collection)

@coagmano You were right, “allRoutes” meant my client routes, i renamed it, now it’s more explicit. Does my code looks good to you ? :thinking:

1 Like

Looks great!
I’m sure this is an improvement over my old suggestion

Thanks for sharing

1 Like