How to block BaiduSpider bot User-Agent?

The Baidu spider (BaiduSpider user agent) can be a real pain to block, especially since it does not respect a robots.txt as it should. This post shows you how to block BaiduSpider bot, using IIS URL Rewrite Module based on its User-Agent string.

A bot is often also called a spider. Normally you would block a bot or spider using the following robots.txt:

User-agent: BaiduSpider
Disallow: /

or

User-agent: *
Disallow: /

But this doesn’t work for blocking Baidu spider… 🙁

Using IIS URL Rewrite Module

You can use the following IIS URL Rewrite Module rule to block the BaiduSpider User-Agent on your website in Windows Server IIS. The only access allowed is to robots.txt, all other requests are blocked with a 403 Access Denied.

Expand the pattern= with multiple user agent strings, divided by a pipe (|), to block more bots. For example pattern="Baiduspider|Bing" or pattern="Googlebot|Bing".

Hint, search other IIS URL Rewrite Module related posts on Saotn.org!

<!--
  Block Baidu spider
-->
<rule name="block_BaiduSpider" stopProcessing="true">
  <match url="(.*)" />
  <conditions trackAllCaptures="true">
   <add input="{HTTP_USER_AGENT}" pattern="Baiduspider" negate="false" ignoreCase="true" />
   <add input="{URL}" pattern="^/robots\.txt" negate="true" ignoreCase="true" />
  </conditions>
  <action type="CustomResponse"
   statusCode="403"
   statusReason="Forbidden: Access is denied."
   statusDescription="Access is denied!" />
</rule>

Apache .htaccess

Block Baiduspider with this .htaccess file:

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^.*Baiduspider [NC]
RewriteRule .* - [F]

Verifying the Baiduspider block

Using Fiddler‘s Composer option, to compose an HTTP request, you can easily verify the rewrite rule, as shown in the next two images.

Ofcourse you can use wget, lwp-request or PowerShell Invoke-WebRequest too, just set the HTTP User-Agent header to Baiduspider:

$ wget --user-agent 'Baiduspider' -q -O- "https://httpbin.io/user-agent"
{
  "user-agent": "Baiduspider"
}

$ GET -H 'User-Agent: Baiduspider' -uUsSe https://httpbin.io/user-agent
GET https://httpbin.io/user-agent
User-Agent: Baiduspider

200 OK
Connection: close
Date: Sun, 14 Jun 2026 10:12:37 GMT
Content-Length: 34
Content-Type: application/json; charset=utf-8
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Client-Date: Sun, 14 Jun 2026 10:12:37 GMT
Client-Peer: 44.211.11.205:443
Client-Response-Num: 1
Client-SSL-Cert-Issuer: /C=US/O=Amazon/CN=Amazon RSA 2048 M01
Client-SSL-Cert-Subject: /CN=httpbin.io
Client-SSL-Cipher: ECDHE-RSA-AES128-GCM-SHA256
Client-SSL-Socket-Class: IO::Socket::SSL
Client-SSL-Version: TLSv1_2

{
  "user-agent": "Baiduspider"
}

$Uri = 'https://httpbin.io/user-agent'
$Headers = @{
    'User-Agent' = 'Baiduspider'
}

$content = Invoke-WebRequest -Uri $Uri -Headers $Headers
$content.RawContent
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Date: Sun, 14 Jun 2026 10:14:55 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 34

{
  "user-agent": "Baiduspider"
}

That’s it!