.Google's John Mueller responded to a question regarding why Google indexes web pages that are refused from creeping through robots.txt and why the it is actually secure to dismiss the relevant Look Console reports about those creeps.Bot Website Traffic To Question Specification URLs.The individual inquiring the concern documented that bots were producing hyperlinks to non-existent query parameter URLs (? q= xyz) to pages along with noindex meta tags that are additionally blocked in robots.txt. What caused the inquiry is that Google.com is actually crawling the web links to those webpages, getting blocked out through robots.txt (without watching a noindex robots meta tag) after that acquiring shown up in Google Explore Console as "Indexed, though blocked by robots.txt.".The individual talked to the observing concern:." But listed below is actually the huge question: why would certainly Google.com index pages when they can not even see the content? What is actually the perk because?".Google.com's John Mueller verified that if they can not crawl the webpage they can not see the noindex meta tag. He likewise makes a fascinating reference of the internet site: hunt operator, advising to dismiss the outcomes since the "ordinary" individuals won't see those results.He composed:." Yes, you are actually appropriate: if our company can't creep the page, our team can not find the noindex. That claimed, if our team can not creep the web pages, after that there's certainly not a lot for our company to mark. Therefore while you may view a number of those pages with a targeted website:- inquiry, the normal consumer will not view them, so I wouldn't fuss over it. Noindex is also great (without robots.txt disallow), it simply indicates the Links will definitely wind up being actually crawled (and end up in the Explore Console file for crawled/not indexed-- neither of these conditions lead to concerns to the rest of the internet site). The important part is actually that you do not produce them crawlable + indexable.".Takeaways:.1. Mueller's response affirms the constraints in operation the Website: hunt accelerated search operator for analysis main reasons. Some of those factors is given that it is actually certainly not linked to the routine search mark, it's a different point altogether.Google.com's John Mueller commented on the internet site search driver in 2021:." The short answer is that a web site: question is actually certainly not implied to be complete, nor made use of for diagnostics functions.A website query is a particular type of search that confines the results to a certain website. It is actually basically just the word web site, a digestive tract, and after that the site's domain name.This inquiry limits the outcomes to a specific site. It's not suggested to be a comprehensive selection of all the webpages coming from that web site.".2. Noindex tag without making use of a robots.txt is actually alright for these kinds of situations where a crawler is linking to non-existent web pages that are actually receiving found through Googlebot.3. URLs with the noindex tag will definitely create a "crawled/not recorded" item in Search Console and that those will not possess a damaging result on the remainder of the web site.Go through the question as well as address on LinkedIn:.Why will Google.com index pages when they can't also find the content?Featured Picture through Shutterstock/Krakenimages. com.