Automate MaxMind login and DB download-CodePudding

I'm trying to use PHP CURL to login to the MaxMind account and download a specific file, I can't even get the login part to work, the following returns "false":

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://www.maxmind.com/en/account');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERPWD, "$username:$password");
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
//curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_NTLM);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
$result = curl_exec($ch);
$info = curl_getinfo($ch);
curl_close($ch);

var_dump($result);

CodePudding user response：

A much easier way to automate the download of the file would be to go to https://www.maxmind.com/en/download_files and copy the link URL for the database format you require. If you wish to always download the latest database, remove the date parameter from the URL. If you are using using wget or curl from a shell script, please be sure to put the URL within quotes.

If you are downloading the dat or mmdb formats, there is also a program to handle automatic updates, http://dev.maxmind.com/geoip/geoipupdate/

CodePudding user response：

In accordance to your statement above I'll add my comment in a re-hashed format with some additional code - however it seems you already found a solution and accepted an answer already but nevermind.

Generally a curl request to a SSL endpoint requires more options to be configured, notably in many instances the use of a valid cacert.pem file and explicit options for verifying the SSL host. The auth method chosen in the original code is more suited to logging into a restricted area protected by a .htaccess file rather than a standard web based login form where the fields used in the post request should follow the names given in the form elements. Generally when trying to emulate a web login I will copy the form in question and then find all the input elements and use them in the POST data I supply to the curl request.

/* A simple curl function */
function mmcurl( $url=NULL, $data=array(), $options=NULL ){

    /* Download cacert.pem and change path here to suit */
    $cacert='c:/wwwroot/cacert.pem';
    $cookiejar=tempnam( sys_get_temp_dir(), '_cookiejar_' );                    


    $curl=curl_init();

    if( parse_url( $url,PHP_URL_SCHEME )=='https' ){
        curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, FALSE );
        curl_setopt( $curl, CURLOPT_SSL_VERIFYHOST, 2 );
        curl_setopt( $curl, CURLOPT_CAINFO, realpath( $cacert ) );
    }

    curl_setopt( $curl, CURLOPT_URL,trim( $url ) );
    curl_setopt( $curl, CURLOPT_AUTOREFERER, TRUE );
    curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE );
    curl_setopt( $curl, CURLOPT_FRESH_CONNECT, TRUE );
    curl_setopt( $curl, CURLOPT_HEADER, FALSE );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE );
    curl_setopt( $curl, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0' );
    curl_setopt( $curl, CURLOPT_ENCODING, '' );

    if( $options->cookie ){
        curl_setopt( $curl, CURLOPT_COOKIEFILE, $cookiejar );
        curl_setopt( $curl, CURLOPT_COOKIEJAR, $cookiejar );
        curl_setopt( $curl, CURLOPT_COOKIE, $cookiejar );
    }
    if( $options->post ){
        curl_setopt( $curl, CURLOPT_POST, true );
        curl_setopt( $curl, CURLOPT_POSTFIELDS, http_build_query( $data ) );
    }

    $res=(object)array(
        'response'  =>  curl_exec( $curl ),
        'info'      =>  curl_getinfo( $curl ),
        'errors'    =>  curl_error( $curl ),
        'cookie'    =>  $cookiejar
    );
    curl_close( $curl );
    return $res;
}




/* configure and call the function - used the suggestion from @Thomas for the endpoint url  */
$url='https://www.maxmind.com/en/download_files';
$data=array(
    'login'         =>  $username,  /* where $username & $password are YOUR variables */
    'password'      =>  $password,
    'pkit_login'    =>  1,
    'pkit_done'     =>  '/en/download_files',
    'Login'         =>  'Login'
);
$options=(object)array(
    'post'  =>true,
    'cookie'=>true
);

$res=mmcurl( $url, $data, $options );

pre($res->info);

Because I do not have an account I could not test further than this but using a dummy username/password the response was a 401 error which seems about right for the wrong username and password.