urllib.request.urlretrieve got stuck when retrieved just one image from a site

kn3l

The python code (python3)

import time
import urllib.response, requests

from config.dev import CONTENT_IMAGE_UPLOAD

directory = CONTENT_IMAGE_UPLOAD + "en_" + time.strftime('%Y%m%d')
filename =  "sample.jpg"
try:
    urllib.request.urlretrieve("https://www.miamiherald.com/latest-news/wfeh98/picture238148999/alternates/LANDSCAPE_1140/Screenshot%20(150).png", directory + "/" + filename)
    print("image is saved")
except Exception as e:
    print(e)

I am expecting to get the image in just less than one minute , but it take too long then print output as message like below.

[Errno 60] Operation timed out

I am sure the image is already exist because when I copied and pasted I got the image, but it seem the URL contains some special char in this part Screenshot%20(150).png,

How could I solve this error?

Marsilinou Zaky

You should add user agent to bypass this issue, I never use urllib directly I usually use requests as it's easier for me, you can implement the same concept using urllib if you want, but you will need to look that up, here's a sample code

import time
import urllib.response, requests

# from config.dev import CONTENT_IMAGE_UPLOAD
headers = {"user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"}
# directory = CONTENT_IMAGE_UPLOAD + "en_" + time.strftime('%Y%m%d')
filename =  "sample.jpg"
try:
    resp = requests.get("https://www.miamiherald.com/latest-news/wfeh98/picture238148999/alternates/LANDSCAPE_1140/Screenshot%20(150).png", headers=headers).content
    with open(filename, "wb") as f:
        f.write(resp)
    print("image is saved")
except Exception as e:
    print(e)

This might help you :)

Changing User Agent in Python 3 for urrlib.request.urlopen

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

request.urlretrieve in multiprocessing Python gets stuck

Image gets rotated when retrieved from file

How to send a repeat request in urllib2.urlopen in Python if the first call is just stuck?

How can I use urllib.request.urlretrieve with python 2.7

urllib.request.urlretrieve ERROR trying to download jpeg in Python

Forbiden to use urllib.request.urlretrieve to download an Excel file

Python urllib.urlretrieve gif image - loading meta information failed

Just got a server stuck in login screen!

Why it gives me the error: No module named 'urllib.request.urlretrieve'; 'urllib.request' is not a package

newb just reading from a book and copying lessons, stuck on lesson one :(

Golang http server got two remote ports from one request

How to iterate an array just retrieved from a hash?

Image not being retrieved from Dropbox

Python: urllib.request.urlretrieve saves an empty file. Writes in it "Supplied id parameter is empty."

python - urllib.request.urlretrieve throws unexpected exception unknown url type: ' '

How can I add a header to urllib.request.urlretrieve keeping my variables?

urllib.urlretrieve with custom header

urllib.urlretrieve encoding is not kept

Minikube got stuck when creating container

Hyperledger composer got stuck when upgrading the network

Got stuck when updating from 15.04 to 15.10, generating grub config file

Got stuck in scraping information from bunch of tags

I tried to retrieving data from firebase database to tableview but I just got one element

Got nil when download image link from firebase

How do I count the number of image request made by one particular site to my site

How to execute inline scripts retrieved from a JQuery Ajax request when content security is enabled in php

When posting a form from Site A to Site B, does the POST request include Site B's cookie?

Method to return value retrieved from HTTP request

Assign retrieved data from request to template