Python Module and parser library issues

Hey everyone,

Still very new with Python (all versions).
I installed a brand new kali release 2020.4.
But I am having problems with some python scripts (a CVE script that I got).

When I run it I get the following error:

  File "/usr/share/offsec-awae-wheels/beautifulsoup4-4.9.1-py2-none-any.whl/bs4/__init__.py", line 245, in __init__
bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: lxml. Do you need to install a parser library? ```

looking up the error the solutions seems to suggest to install ```lxml``` by running:  ``` sudo pip3 install lxml ``` or ``` sudo pip install lxml ``` or ``` sudo apt-get install libxml2-dev libxslt-dev python-dev ```
running the pip installs returns the following message:  ``` Requirement already satisfied: lxml in /usr/lib/python3/dist-packages (4.6.2) ```

So I tried to do the following: ``` export PYTHONPATH=/usr/lib/python3/dist-packages ``` rerun my script in the same terminal but still I get the error:  ```Couldn't find a tree builder with the features you requested: lxml. Do you need to install a parser library?```

So to say; I am lost and new to this python scripting (been new for awhile :P)

Would be grateful for a helping hand\eye to fix this up for me (its a fresh install of the distro too)

Looking at the error here:

soup = BeautifulSoup(html_content,features="lxml") File "/usr/share/offsec-awae-wheels/beautifulsoup4-4.9.1-py2-none-any.whl/bs4

especially the 4-4.9.1-py2 would IMO make me think that the script is using python2 and beautifulsoup for python version 2.

So this is what I would recommend doing:

Install pip2 (This page has moved - pip documentation v21.2.4)

and then do pip2 (or just pip) install lxml.

Let me know if it helps!

PS: you might need to also upgraded setuptools on pip2, you can do this with pip2 install --upgrade setuptools

Edit: another PS (lol!) I’d honestly recommend just learning python3, not that python2 is much different, just a few syntax differences, but it’s just that it’s EOL now and isn’t supported anymore.

Thanks @HadesSec
yeah no luck in even installing pip2
curled the file and ran it and got the following error:

  File "get-pip.py", line 24244, in <module>
    main()
  File "get-pip.py", line 199, in main
    bootstrap(tmpdir=tmpdir)
  File "get-pip.py", line 82, in bootstrap
    from pip._internal.cli.main import main as pip_entry_point
  File "/tmp/tmp2JmL1p/pip.zip/pip/_internal/cli/main.py", line 60
    sys.stderr.write(f"ERROR: {exc}") ```

But a good nose you had for pointing out the python2 (py2) ...
I guess its python3 now for me all the way :)

Wonder if there is a way to convert the script ... (google )

Type your comment> @acidbat said:

Thanks @HadesSec
yeah no luck in even installing pip2
curled the file and ran it and got the following error:

  File "get-pip.py", line 24244, in <module>
    main()
  File "get-pip.py", line 199, in main
    bootstrap(tmpdir=tmpdir)
  File "get-pip.py", line 82, in bootstrap
    from pip._internal.cli.main import main as pip_entry_point
  File "/tmp/tmp2JmL1p/pip.zip/pip/_internal/cli/main.py", line 60
    sys.stderr.write(f"ERROR: {exc}") ```

But a good nose you had for pointing out the python2 (py2) ...
I guess its python3 now for me all the way :)

Wonder if there is a way to convert the script ... (google )

Did you run get-pip.py in python2 or 3? Honestly, I’ve absolutely NEVER had any issues with that python script.

lol whoops I’m an idiot - installed it via python3 successfully.
Successfully installed pip-21.0.1
Thanks mate - will see how this goes

Type your comment> @acidbat said:

lol whoops I’m an idiot - installed it via python3 successfully.
Successfully installed pip-21.0.1
Thanks mate - will see how this goes

Happy to see it work! Was a bit nervous and stumped for a second haha.
For sure! Let me know how it goes!

@acidbat
in your error message File "/usr/share/offsec-awae-wheels/beautifulsoup4-4.9.1-py2-none-any.whl/bs4/__init__.py", especially the beautifulsoup4-4.9.1-py2-none-any.whl part, means:
this package is part of beautifulsoup version 4, 4.9.1 to be specific, but it is THE ONE THAT IS SUITABLE FOR PYTHON2, not picky about your OS, nor your processor architecture

partially disagree with @HadesSec, i don’t think this -py2 means the exploit script is written in python2, but i do think it means this beautifulsoup that comes with offsec-awae install, is only made for use with python2
edit: i see what exploit are you dealing with now haha! yeah the exploit script is in python3

then i guess you should install beautifulsoup4-4.9.1-py2-none-any.whl's brother i.e. beautifulsoup4-4.9.1-py3-none-any.whl? link here

<nonsense removed, still no idea based on what word is a lib located>

Type your comment> @bitn4b said:

@acidbat
in your error message File "/usr/share/offsec-awae-wheels/beautifulsoup4-4.9.1-py2-none-any.whl/bs4/__init__.py", especially the beautifulsoup4-4.9.1-py2-none-any.whl part, means:
this package is part of beautifulsoup version 4, 4.9.1 to be specific, but it is THE ONE THAT IS SUITABLE FOR PYTHON2, not picky about your OS, nor your processor architecture

Ok, that is good to know and thank you for explaining.

partially disagree with @HadesSec, i don’t think this -py2 means the exploit script is written in python2, but i do think it means this beautifulsoup that comes with offsec-awae install, is only made for use with python2
edit: i see what exploit are you dealing with now haha! yeah the exploit script is in python3

I admire people who can just read code and understand what needs to be done right away. Seriously think I course in python3 would do me some good.

then i guess you should install beautifulsoup4-4.9.1-py2-none-any.whl's brother i.e. beautifulsoup4-4.9.1-py3-none-any.whl? link here

Awesome, thank you - will look at this at home (where the problematic distro lives)
Any tips on how to install *.whl files?

Type your comment> @acidbat said:

Awesome, thank you - will look at this at home (where the problematic distro lives)
Any tips on how to install *.whl files?

Apparently this is the way to do it: pip install FILE.whl

@acidbat said:
I admire people who can just read code and understand what needs to be done right away. Seriously think I course in python3 would do me some good.
uh man i just see command print come with parentheses and assume it’s python3, (should be right…i guess? have you fixed your distro?)
i unfortunately don’t know shit about code >> every time i try to learn i just get overwhelmed by so many new words >>

but seriously have you fixed your python…thing?

No dice yet - I didn’t get too much time last night.
installing the .whl files did not work (said it was already installed and the python2 one did not meet my requirements of distro ‘’‘ERROR: beautifulsoup4-4.9.3-py2-none-any.whl is not a supported wheel on this platform.’’’
will download beautifulsoup4-4.9.1-py2-none-any.whl and see what happens - expecting it will be the same.

Also tried a convert from python2 to python3 script - got little further but still some modules complaints.
whats this space

bro you ALREADY HAVE beautifulsoup4-4.9.1-py2-none-any.whl, it is the reason why your exploit script errored out in the first place aaaahhhhhhhhhhhhh
in my original answer i think you need to install beautifulsoup4-4.9.1-py3-none-any.whl, the -py3 one! the -py3 one!

bro i think you need to try pip3 install beautifulsoup4-4.9.1-py3-none-any.whl
specify! 3! wherever! you! can!
including when you run your exploit python3 <exploit>

(pls work)

<i really dont know anymore>

Type your comment> @bitn4b said:

bro you ALREADY HAVE beautifulsoup4-4.9.1-py2-none-any.whl, it is the reason why your exploit script errored out in the first place aaaahhhhhhhhhhhhh
in my original answer i think you need to install beautifulsoup4-4.9.1-py3-none-any.whl, the -py3 one! the -py3 one!

bro i think you need to try pip3 install beautifulsoup4-4.9.1-py3-none-any.whl
specify! 3! wherever! you! can!
including when you run your exploit python3 <exploit>

(pls work)

Yeah did that - installed it (yay)

python3 exploit.py

Traceback (most recent call last):
  File "exploit.py", line 16, in <module>
    from random_words import RandomWords
ModuleNotFoundError: No module named 'random_words'

Module is installed

sudo pip3 install Random-Word                                                                                                                                                                                                        1 ⨯
Requirement already satisfied: Random-Word in /usr/local/lib/python3.9/dist-packages (1.0.6)
Requirement already satisfied: nose in /usr/local/lib/python3.9/dist-packages (from Random-Word) (1.3.7)
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from Random-Word) (2.25.1)

I feel like season2 of LOST

i know it now!
it should be from random_word import RandomWords!!!
there’s no s after random_word! the original script made a typo!

Type your comment> @bitn4b said:

then edit the script import part to be from Random-Word import RandomWords?

i dont feel a thing anymore

SFA mate.

Thank you for your help so far - I’ll park this crazy project for now. I did root the machine on my other VM but would be nice to align it all to work properly.
And they say python is the easiest one to learn --- yeah okay -

@acidbat
no no no pls try edit the script import part to be from random_word import RandomWords!
it is underscore (my bad) but without s followed pls try it ahhhh i tried it and it worked!

@acidbat
ahhh don’t park it man we are so close! it is not crazy it is the original writer made a typo! typo is not crazy it’s careless and stupid (or intended) it’s not your fault!

pls just try edit the script import part to be from random_word import RandomWords it just take 3 seconds and if it still wont work we will then park it!

see the original exploit said from random_words but it should be from random_word, no s, and we’re done