User-agent: NimbleCrawler Disallow: / User-agent: * Disallow: /cgi-bin/ Disallow: /abc/edit-ad89nf8e/ Disallow: /4c/admin/