Topic: Tor and general Socks5 Proxy issues when accessing the e621 api

I'm testing proxy settings in python using the e621 api. When i query this website through a browser (which also has tor running) i can access the api just fine, but when python tries the same thing it returns with:
"403 Forbidden"

my theory regarding the issue is that I'm being blocked by a capcha that apperes sometimes when i access sites through tor

I'll leave the source and error messages here for any of my fellow python neck-beards

What can i do to prevent this 403 error from happening on tor

Source Code:

from lib.socks import monkey_patch as socks_patch
socks_patch.monkey_patch()
from lib.socks import socks_proxy_context
from urllib.request import *
import json, os, sys, time

#responce = urlopen("https://e621.net/post/show/547303.json")

with socks_proxy_context.socks_proxy_context(proxy_address=("127.0.0.1", 9150)):
	responce = urlopen("https://e621.net/post/show/547303.json")

print(responce)

Error Message:

Traceback (most recent call last):
  File "api_scrapper.py", line 10, in <module>
    responce = urlopen("https://e621.net/post/show/547303.json")
  File "C:\Python34\lib\urllib\request.py", line 153, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Python34\lib\urllib\request.py", line 461, in open
    response = meth(req, response)
  File "C:\Python34\lib\urllib\request.py", line 571, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Python34\lib\urllib\request.py", line 499, in error
    return self._call_chain(*args)
  File "C:\Python34\lib\urllib\request.py", line 433, in _call_chain
    result = func(*args)
  File "C:\Python34\lib\urllib\request.py", line 579, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

... Have you tried turning it off and back on again?

Tokaido said:
... Have you tried turning it off and back on again?

Sorry, I couldn't resist :P

You silly billy

I'm willing to bet that is just a cloudflare thing and there's basically nothing you can do about it. They are probably trying to traffic from known exit nodes because they (like many) believe that TOR has a high rate of abuse with very little legitimate traffic (read: hacking, spam, botnets, DDOS, etc.) and they are blocking it on purpose for security purposes.

Note that I'm not 100% sure of this, but it may give you a lead at least.

parasprite said:
I'm willing to bet that is just a cloudflare thing and there's basically nothing you can do about it. They are probably trying to traffic from known exit nodes because they (like many) believe that TOR has a high rate of abuse with very little legitimate traffic (read: hacking, spam, botnets, DDOS, etc.) and they are blocking it on purpose for security purposes.

Note that I'm not 100% sure of this, but it may give you a lead at least.

That seems likely, I'll just have to find a diffrent socks5 proxy to test my application with

Not sure if this is the problem (I don't remember how the API responds in this case) but your application needs to support SSL. I don't know what python uses for SSL/sockets or whether it uses it by default (unlike PHP's cURL) but that might be the problem.

TonyLemur said:
Not sure if this is the problem (I don't remember how the API responds in this case) but your application needs to support SSL. I don't know what python uses for SSL/sockets or whether it uses it by default (unlike PHP's cURL) but that might be the problem.

I tried using a non-tor socks5 proxy and it worked so it might have been server security after all