Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.
Create a Robots.txt file to let Googlebot and other User-Agents know which directories they should and shouldn't crawl.