How to download a csv file in Python-CodePudding

I am trying to download a csv file from the url

https://qubeshub.org/publications/1220/supportingdocs/1#supportingdocs .

the file is Elephant Morphometrics and Tusk Size-originaldata-3861.csv

I have tried using using pd.read_csv()

and

import pandas as pd
import io
import requests
url="https://qubeshub.org/publications/1220/supportingdocs/1#supportingdocs/Elephant Morphometrics and Tusk Size-originaldata-3861.csv"
s=requests.get(url).content
c=pd.read_csv(io.StringIO(s.decode('utf-8')))

CodePudding user response：

Try:

import requests

url = "https://qubeshub.org/publications/1220/serve/1/3861?el=1&download=1"

r = requests.get(url)
filename = r.headers["Content-Disposition"].split('"')[1]

with open(filename, "wb") as f_out:
    print(f"Downloading {filename}")
    f_out.write(r.content)

Prints:

Downloading Elephant Morphometrics and Tusk Size-originaldata-3861.csv

and saves the file.

CodePudding user response：

This should download the file and parse the rows and columns into a csv file

import requests
import csv
url = "https://qubeshub.org/publications/1220/serve/1/3861?el=1&download=1"
req=requests.get(url)
rows = req.content.decode('utf-8').split("\r\n")
rows.pop()
csv_local_filename = "test.csv"
with open(csv_local_filename, 'w') as fs:
    writer = csv.writer(fs, delimiter = ',')
    for row in rows:
        entries = row.split(',')
        b=writer.writerow(entries)

You'll likely want to convert those columns into the desired types before you start working with them. The example code above leaves everything as a string.

After I run the above code I see:

>tail test.csv 
2005-13,88,m,32.5,290,162.3,40
2005-13,51,m,37.5,270,113.2,40
2005-13,86,m,37.5,310,175.3,38

and

>head test.csv
Years of sample collection,Elephant ID,Sex,Estimated Age (years),shoulder Height in  cm,Tusk Length in cm,Tusk Circumference   in cm
1966-68,12,f,0.08,102,,
1966-68,34,f,0.08,89,,
1966-68,162,f,0.083,89,,
1966-68,292,f,0.083,92,,

CodePudding user response：

In Firefox after downloading file in browser you can check link to this file and it shows

https://qubeshub.org/publications/1220/serve/1/3861?el=1&download=1

and this link you should use in code

import pandas as pd

df = pd.read_csv('https://qubeshub.org/publications/1220/serve/1/3861?el=1&download=1')

print(df)