when i tried to solve weak rsa the error shown below is displayed what should i do

root@kaliblue:~/HTB/RsaCtfTool# openssl rsautl -decrypt -inkey private.pem -in flag.enc -out key.txt

RSA operation error
139976077754560:error:0407109F:rsa routines:RSA_padding_check_PKCS1_type_2:pkcs decoding error:…/crypto/rsa/rsa_pk1.c:241:
139976077754560:error:04065072:rsa routines:rsa_ossl_private_decrypt:padding check failed:…/crypto/rsa/rsa_ossl.c:487:

pease help me to fix this error

so close ! read the error message and perhaps check that your private was correctly created from your public one.

Tell openssl to use no padding:

$ openssl rsautl -help
Usage: rsautl [options]
Valid options are:
 -raw                     Use no padding

$ openssl rsautl -decrypt -inkey privkey.pem -in flag.enc -out flag.txt -raw

This does result in some null bytes at the beginning of the output, but it decrypts the message just fine. You can clean things up with tr:

$ openssl rsautl -decrypt -inkey privkey.pem -in flag.enc -raw | tr -d '\000' > flag.txt

Honestly though, I recommend decrypting it mathematically in Python. If you’ve managed to snag the private key you have all you need.