Nginx : Block User Agent

In Nginx, you can block certain user agents (normally it is crawler) like this :

/etc/nginx/sites-enabled/default

server {
  listen          80;
  server_name     mysite.com;
  root            /etc/tomcat7/webapps/mysite;

  if ($http_user_agent ~* (ahrefs|wget|crawler|majestic) ) {
    return 403;
  }

  location / {
        <!-- xxx -->
  }
}

In above example, for “user agent” that contains one of this pattern : ahrefs|wget|crawler|majestic, HTTP 403 will be returned.

No comments yet. Be the first to leave a comment!

Leave a Comment

Your email address will not be published. Required fields are marked *