This is similar to behaviour of a site: query in Google search.Ĭonfiguration > Spider > Extraction > URL Details ![]() Please note – If a crawl is started from the root, and a subdomain is not specified at the outset (for example, starting the crawl from ), then all subdomains will be crawled by default. The CDNs configuration option can be used to treat external URLs as internal. To crawl all subdomains of a root domain (such as or ), then this configuration should be enabled. These will only be crawled to a single level and shown under the External tab.įor example, if is entered as the start URL, then other subdomains discovered in the crawl such as or will be treated as ‘external’, as well as other domains such as etc. ![]() ![]() Configuration > Spider > Crawl > Crawl All Subdomainsīy default the SEO Spider will only crawl the subdomain you crawl from and treat all other subdomains encountered as external sites.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |