What are some really interesting web crawling projects?


What are some really interesting web crawling projects?

I’ve got an ongoing partially unresolved inquiry into converting sitemaps into mindmaps [1], which in ideal conditions would involve crawling a website, parsing the directory structure into a nested outline, and then inserting the crawled webpage titles into the outline. A cloud based (web) API would be the ideal form for this. I can further recommend my Meta Guide webpage, “100 Best Web Crawler Videos” [2].

[1] Mindmap Conversion – Meta-Guide.com

[2] 100 Best Web Crawler Videos – Meta-Guide.com

(Visited 361 times, 1 visits today)
Liked it? Take a second to support mendicott on Patreon!