Creepy Crawlies | Information Gather - Web Edition Module

I’m struggling with the creepy crawlies section. It seems that there should be a target to crawl but I don’t see the target button.

I’ve been having the same issue. Completed this module a while ago, then when the new content was added and I went to re-complete the module I ran into this.

At first I figured we were meant to scan a target we would’ve spawned earlier in the module, kind of assuming we’re doing the whole thing in one go, but that doesn’t seem to be the case.

1 Like

I am having a similar issue with this module. I am unable to use scrapy because HTB doesn’t allow “pip install scrapy” but they do allow “sudo apt install scrapy” (which causes DLL errors when trying to use ReconSpider with scrapy). They need to update the guide to reflect this. Anyways I had finished this module awhile ago and received the badge for It so I moved on after roughly 2 hours of trying to use a command. is weird for me aswell. https inlinefreight, HTTP inlinefreight, and www inlinefreight. Way too many imo and sometimes they don’t work.

I tried using Zapspider and was still unable to find the “reports” page the question was talking about. Was anyone able to figure this out?

“For Linux Users”
12. Virtual Environments and Packages — Python 3.12.4 documentation Go to this link and follow instructions on how to set up a virtual environment. Once you have done that run the “pip3 install scrapy” then you can bring in the ReconSpider inside the virtual environment(same dir as the V.E.) after that your good to run the command. The CLI will give you some goop, look for the json file


Thanks mate!

Bruh how are you guys even scanning a host? There is no option to spin up a target box.

For those confused about not being able to spin up a target, the target is the example used in the module, http://inlanefreight[.]com

I don’t think I’m following.

I need a host to spider, since it doesn’t give the option to spin one up what am I supposed to spider?

The module gives a few examples of using inlanefreight. This host is always up and the one the question wants you to spider.

Ohhhh is it because it’s “.com” as opposed to “.htb”

That would cause it to not go to the correct host. You can always make sure you have the right host by pinging it first and making sure you get a response.

this was super helpful, if you follow the tutorial line by line you won’t be able to run the ReconSpider. Guys, the is up and running all the time… we don’t need to spin the target! It’s also accessible without HTB VPN