A new token – Google-Extended for robots.txt that will not allow the use of a website for AI training

Google introduced and new token – Google-Extended. With this token, the website will be included in the search, but the website won’t be allowed to be used for AI training. Similar tokens are used by chatbots Bard and Vertex AI.

The introduction of the token will not affect other aspects of website indexation. The main goals of this innovation according to the company, is to reach better control and transparency regarding the use of AI.

According to Danielle Romain, vice president of trust at Google, the company is planning to cooperate with web developers in searching for new machine-readable approaches to selection and controlling for web publishers.