MyBB is not very optimized for search engines by default. One of the biggest problems is that it does not come with a robots.txt file.
Search engines will crawl all kind of stuff in your forum that makes no sense. Examples are the calendar, the archive, new reply pages and many more.
They will also find a lot of duplicated content in your forum because the same posts can be displayed in the archive, using a direct post URL and all kinds of thread URLs with lastpost or newpost.html additions. MyBB does not ship with canonical URLs by default so you’ll have issues here too.
My currently recommended robots.txt for MyBB with activated SEO-friendly-URLs looks like this:
User-agent: * Disallow: /board/archive/ Disallow: /board/private.php* Disallow: /board/usercp.php Disallow: /board/usercp2.php Disallow: /board/ratethread.php Disallow: /board/newreply.php Disallow: /board/memberlist.php Disallow: /board/printthread.php Disallow: /board/forumdisplay.php Disallow: /board/showthread.php Disallow: /board/member.php Disallow: /board/calendar.php Disallow: /board/thread-*-lastpost.html Disallow: /board/thread-*-nextnewest.html Disallow: /board/thread-*-nextoldest.html Disallow: /board/thread-*-newpost.html Disallow: /board/thread-*-post-*.html Disallow: /board/post-*.html Disallow: /board/forum-*.html?datecut=9999
Updated 09/06/2015, added one rule.
You might need to adjust the /board path if your forum is in a different subpath of your domain or remove it if your forum is not in a subfolder.
From top to bottom I’m removing the archive, because of duplicated content and to prevent search engine visitors from discovering the website in the ugly archive mode.
I do prevent access to private messages and user cp as google or any other bot don’t need that. I do prevent access to ratethread and newreply for the same reason.
I excluded the memberlist too, you might want to include it. I removed the option to print a thread. I excluded forumdisplay and showthread because we are using SEO-friendly-URLs. I excluded the calendar as mentioned before.
Now we are excluding the different writing options of a thread to minimize duplicated content.
Done – your MyBB does use a proper robots.txt now. Google does not have to crawl and index thousands of meaningless calendar years and focus on the main content of your forum without duplicating it.
MyBB for Business!