(first revised version, included link to URL-encoding)
After so many years, I've come up with a somewhat practicable solution
for HTML login within Xenu. Or rather, the competition did:
Submitting a FORM was never a technical problem for me; I have tested
this years ago already, but without any user interface. My mistake was
to look for a general way to catch FORMs in HTML pages, instead of just
using the login FORM url as a start.
A test version of Xenu with login FORMs can be found here:
The initial dialogbox has a new checkbox for POST and a POST query
string. Note that the query string won't be stored in the INI file.
Imagine you'd want to check the (imaginary) password protected website
Look at the HTML for the FORM:
<FORM METHOD="POST" ACTION="https://www.host.com/scripts/login.php"
<INPUT TYPE="text" NAME="User">
<INPUT TYPE="password" NAME="Password">
<INPUT TYPE="submit" NAME="Submit" VALUE="Login">
Thus, the start URL would be
and if the username is "Xenu", and the password is "Secret", the POST
query string would be
(The POST query string gets tricky if there are spaces or special
characters in the parameters. It must be "URL-encoded", see here:
You must add https://www.host.com
to the inclusion list. If you don't,
only URLs beginning with https://www.host.com/scripts/
will be checked.
Don't let Xenu hit the logout link. To avoid this, add the logout URL to
the exclusion list:
I'm not yet sure if is important to have cookies enabled. If yes, don't
let Xenu hit any URLs that delete or change something.