How to bruteforce directories from a list of urls?

Off-topic Programming
,
1

Is there any way to bruteforce directories from a list of urls using dirb or gobuster?
url contains multiple valid urls, Ive tried using a while statement but no luck, its giving me an error after successfully scanning the first url in the file.

(!) FATAL: Invalid URL format: ual/images/
(Use: “http://host/” or “https://host/” for SSL)

#!/bin/bash
echo "starting..."
cat url | while read url;
do
dirb "$url" /usr/share/wordlists/dirb/common.txt -o output.txt
done

Thanks.

2

This error message contains the input URL: ual/images/. This URL is invalid. the URL must start with http: or with https:. This URL starts with the text ual

Your should check the content of the file url. The script looks ok. I have checked it with a simple list of 3 URLs and it works.

You can call your script like

bash -x SCRIPTNAME

The option -x instructs the bash interpreter to print out all commands and their arguments before they are executed. Then you can read the arguments of the dirb tool.

3

Thanks for the response!
The thing is with that error, the url provided was just http:///images/

So not sure why its goving me that error.

4

The URL http:///images/ contains no host name. The interpretation of this URL depends on the application. The tool curl ignores the third slash and tries to connect host “images”. The tool dirb uses internal the curl library. So also the tool dirb tries to connect the host images.

But unable to resolve the host “images” is not the error message. dirb terminates before.

I checked the source code of dirb link. The “FATAL: Invalid URL format …” error messages is printed after checking http:// or https://.

void check_url(char *url) {

  if(options.debuging>4) printf("[++++] check_url() URL: %s\n", url);

  if(strncmp(url, "http://", 7)!=0 && strncmp(url, "https://", 8)!=0) {
    printf("\n(!) FATAL: Invalid URL format: %s\n", url);
    printf("    (Use: \"http://host/\" or \"https://host/\" for SSL)\n");
    exit(-1);
    }

}

Hence the URL in the command line must be ual/images/.
I suppose checking the dirb call with bash -x scriptname.

Another option is to add the switch -d5 to the dirb call. In debug level 5 the dirb program prints out the URL before checking.

1 Like
5

Thanks for that! it has helped!