Automating Client Side Password Encryption For PHP/cURL Web Bot Form Submission

I know I haven’t posted anything in months and my loyal readers (hah!) have been hassling me for another installment of the Web Scraping with PHP & cURL series. I assure you, it is coming soon, though this post will be covering the solution of a web scraping and automation problem using PHP. So, you can enjoy this while you wait…

The Problem – JavaScript Client-Side Encryption Of Passwords Before Form Submission

Recently I came across something that is either becoming more common, or I am more frequently encountering due to the jobs I take, and that is JavaScript client-side encryption of passwords before submission of a login form. Now since this usually uses JavaScript it can seem a bit tricky the first time you encounter it if you’re not used to working with JS.

This kind of thing is usually easily identifiable when you see the submit button of a form calling another function, like so:

<input type="submit" onclick="validerLogin();"/>

After identifying the .js file containing this sites JavaScript, the function is found to be:

function validerLogin() {
    var doc=document.loginFrm;
    if (doc.usager.value.length==0 || doc.usager.value =='Utilisateur'){
        alert("Vous devez entrer votre nom d'usager.");
        doc.usager.focus();
        return false;
	} else if (doc.motdepasse.value.length == 0
			|| doc.motdepasse.value == 'Mot de passe') {
        alert("Vous devez entrer votre mot de passe.");
        $('#motdepasseLbl').focus();
        return false;
    } else {
        md5Pass = $.md5(doc.motdepasse.value);
        doc.pwd.value = $.md5(md5Pass+doc.salt.value);
        doc.motdepasse.value = '';
        doc.submit();
    }
}

The website, and it’s source code, being entirely in French, isn’t exactly helpful, but from a cursory glance it’s clear that, after ensuring a password has been entered, what is happening is:

  1. The value from the login forms field named motdepasse is being encrypted using an MD5 (which appears to be the jQuery MD5 Plugin by Sebastian Tschan) and is stored in the variable md5Pass.

    md5Pass = $.md5(doc.motdepasse.value);

     

  2. The encrypted password stored in the variable md5Pass is being put back through the MD5 encryption function, this time using a salt (more on this in a moment), and being injected into the value attribute of the hidden pwd input field on the sites login form.

    doc.pwd.value = $.md5(md5Pass+doc.salt.value);

     

  3. The value of motdepasse on the login form is being cleared.

    doc.motdepasse.value = '';

     

  4. The login form is being submitted.

    doc.submit();

     

Solving The Problem Of Client-Side Encryption With PHP & cURL

Now, in our web bot we need to recreate all of these steps before submitting the form using cURL. So, step by step, as above:

  1. First up, we need to encrypt our password using MD5. Luckily for us, PHP has an md5() function.

    $md5_pass = md5($password);	// md5ing the password
    

     

  2. Next up, we need to encrypt our encrypted password again, using a salt.

    It turns out that the salt being used to encrypt the password for the second time is actually a unique 5 digit number that is generated and stored in the value of a hidden form input field named salt each time the login page is refreshed.

    <input type="hidden" name="salt" value="46545" />
    

    So, before we submit the form we need to retrieve the login page and scrape the salt string.

    $salt_page = curlRequest($login_url);	// Using cURL to request the login page with the salt string on it.
    $salt = scrapeBetween($salt_page, 'salt" value="', '" />');	// Scraping the salt string.
    // You can find the curlRequest() and scrapeBetween() functions in my tutorial series on Web Scraping with PHP & cURL.
    

    With this salt string, we can then proceed to re-encrypt our password.

    $md5_pass = md5($password);	// md5ing the password.
    $salt_pass = md5($md5_pass.$salt);	// Concatenating and encrypting the md5'd pass and salt.
    

     

  3. On submission of the form using cURL we need to ensure that we pass an empty string to the motdepass field and our MD5/salted password to the pwd field, using an array like:

    $md5_pass = md5($password);	// md5ing the password.
    $salt_pass = md5($md5_pass.$salt);	// Concatenating and encrypting the md5'd pass and salt.
    $post_array = array ('usager' => $user, 'motdepass' => '', 'salt' => $salt, 'pwd' => $salt_pass);	// Building post array.
    

     

  4. Then it’s as simple as making the cURL POST request!

    $md5_pass = md5($password);	// md5ing the password
    $salt_pass = md5($md5_pass.$salt);	// Concatenating and encrypting the md5'd pass and salt.
    $post_array = array ('usager' => $user, 'motdepass' => '', 'salt' => $salt, 'pwd' => $salt_pass);	// Building post array.
    $site_login = curlPost($login_url, $post_array);	// Logging in using curlPost function. You can find this in my web scraping tutorial series, though it is just a standard POST form submission using cURL.
    

     

If all goes well, that should get us logged in and ready to do as we will!

In case anybody’s wondering, or indeed wants to give this code a try, the website in question is lespac.com

Hope you have fun with this and can use it in your own bots if and when you come across this problem.

Until next time!

20 thoughts on “Automating Client Side Password Encryption For PHP/cURL Web Bot Form Submission

  1. Hello one of my friend wrote code like this, now he is not responding
    please help me

    wrote code for password encryption

    $this-&gt;db-&gt;query("INSERT INTO `" . DB_PREFIX . "user` SET username = '" . $this-&gt;db-&gt;escape($data['username']) . "', salt = '" . $this-&gt;db-&gt;escape($salt = substr(md5(uniqid(rand(), true)), 0, 9)) . "', password = '" . $this-&gt;db-&gt;escape(sha1($salt . sha1($salt . sha1($data['password'])))) . "', firstname = '" . $this-&gt;db-&gt;escape($data['firstname']) . "', lastname = '" . $this-&gt;db-&gt;escape($data['lastname']) . "', email = '" . $this-&gt;db-&gt;escape($data['email']) . "', user_group_id = '" . (int)$data['user_group_id'] . "', status = '" . (int)$data['status'] . "', date_added = NOW()");
    

    That code stores the values in data base in this format

    INSERT INTO `oc_user` VALUES (1,1,'admin','a7008174afb39f4d72eddc4d06e1d4da','97a7dba28','','','sellerbcools@gmail.com','','122.169.212.99',1,'2013-05-31 00:18:46');
    
    INSERT INTO `oc_user` VALUES (2,10,'ProductEntry','16be018b98f6cfd7d4ba3db23b86bbfc818c4d5e','c349b9778','Prod1','Entry1','','','183.82.210.69',1,'2013-06-10 08:44:36');
    

    Now please tell me the password

  2. Hi there, I cannot find the curlPost function in any of your previous articles… 🙁

    thx a lot anyway
    D.

    1. Here you go…

      function curlPost($postUrl, $postFields) {
      	
      	$cookie = 'cookie.txt';	// Setting a cookie file to store cookie
      
      	$ch = curl_init();	// Initialising cURL session
      
      	// Setting cURL options
      	curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);	// Prevent cURL from verifying SSL certificate
      	curl_setopt($ch, CURLOPT_FAILONERROR, TRUE);	// Script should fail silently on error
      	curl_setopt($ch, CURLOPT_COOKIESESSION, TRUE);	// Use cookies
      	curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);	// Follow Location: headers
      	curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);	// Returning transfer as a string
      	curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);	// Setting cookiefile
      	curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);	// Setting cookiejar
      	curl_setopt($ch, CURLOPT_USERAGENT, $useragent);	// Setting useragent
      	curl_setopt($ch, CURLOPT_URL, $postUrl);	// Setting URL to POST to
      			
      	curl_setopt($ch, CURLOPT_POST, TRUE);	// Setting method as POST
      	curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($postFields));	// Setting POST fields as array
      			
      	$results = curl_exec($ch);	// Executing cURL session
      	curl_close($ch);	// Closing cURL session
      	
      }
      

      Where $postUrl is the URL to post to, $postFields is an array of fields to post, with the name and value, e.g.

      $postFields = array(
      	'username'	=> $username, 
      	'password'	=> $password, 
      	'email'		=> $email, 
      	// etc....
      );
      
  3. Please contact me at your earliest convenience. Looking to hire some of your time for scraping a tricky dynamic site. Hope to hear from you soon. I tried the hire me form unsuccessfully. Maybe I’ll try you on Skype.

  4. Hi Jacob

    thanks for nice article.

    I have use your code for accessing content after login.

    but it always return login page content when i print $results value

    I have use same code as you mention in comments section above.

    can you tell me what if I am doing wrong any ?

    waiting reply soon thank you for help.

  5. Hello Mr Jacob. I am in the need of your advice. I have used your tutorial to fit my needs scraping another website. However, I would like to know how to proceed if after logging in, I get another form which I must submit (it would be like repeating the same process but my problems is that I don’t know how to use the $result from the first form to repeat the process.

    Kind regards.

    1. You shouldn’t need the $result from the first page. After logging in you should have a cookie/session which will keep you logged in regardless. So just submit the second form in the same way as the first.

  6. Hello Mr. Jacob, can you give me an advice? i tried to make an automation login with curl, the site is using salt too but the encryption is using java script SHA256 hash function. can you tell me how to deal with this problem. Thanks

    1. Sure, rather than using PHPs md5() function, you can use hash() and pass the sha256 argument to it. So you would have something like:

      $encoded_password = hash('sha256', $password);
      
  7. hello, i am trying to scrapp a web page that is accessible via https, the server is nginx and i get the message “400 Bad Request The plain HTTP request was sent to HTTPS port.”, i tryed with other web pages that are using https and it worked normally.
    is it a kind of anti scrapping protection ? if yes, how can i resolve this problem
    thank you

Leave a Reply