Web Scraping Rotating IP Public With Python Tor and Privoxy

In this tutorial, we will explain how to rotate the Proxy IP from Tor and Privoxy using the python stem which is responsible for rotating or rotating the proxy IP periodically. The most common use case when you use Tor and Privoxy proxies to access a web browser is to change your identity and data. Because these two programs have the ability to disguise and hide your identity and real data.


Because as we know, if we use Tor and Privoxy to open web browsing, for example we open Google Chrome, then our IP is not the original IP, but a proxy IP that has been changed by Tor and Privoxy. In other words, for example, if we use MyIndihome internet from Telkom with Public IP 36.90.67.41, then when we use TOR/Privoxy our MyIndihome IP will change. We will get foreign IP, such as Germany, the Netherlands, America and others.

OK, let's get straight to the point of discussing how to rotate Public IP with TOR, Python and Privoxy.

To make it easier for readers to configure "IP Public Rotation with Python", you should also read our previous article.




In this case we will not explain how to install Tor and Privoxy, you can read about Tor and Privoxy in the article above. Let's assume that our FreeBSD server has the Tor and Privoxy applications installed.

In this tutorial, the computer I use has Tor installed with IP/Port: 192.168.5.2:9050 and Privoxy with Port 192.168.5.2:8008. For the OS used, we rely on FreeBSD 13.2 Stable, and the Python version used is python39.

What you have to pay attention to is the script in the config file belonging to the privoxy program in the /usr/local/etc/privoxy folder. In the forward-socks5 script, you must direct it to the Tor IP and Port, namely 192.168.5.2:9050.

root@router2:~ # cd /usr/local/etc/privoxy
root@router2:/usr/local/etc/privoxy # ee config
Look for the forward-socks5 script and change the IP and Port to TOR's IP and Port.

forward-socks5 / 192.168.5.2:9050 .



1. Python and Stem Installation

The first thing you have to do is install python39 and py-stem.

root@ns1:~ # cd /usr/ports/lang/python39
root@ns1:/usr/ports/lang/python39 # make install clean

root@ns1:/usr/ports/lang/python39 # cd /usr/ports/security/py-stem
root@ns1:/usr/ports/security/py-stem # make install clean

root@ns1:/usr/ports/security/py-stem # cd /usr/ports/www/py-requests
root@ns1:/usr/ports/www/py-requests # make install clean


2. Avoid Python Errors

Because there are so many versions of Python on FreeBSD, errors often occur which cause the Python program not to run. The most common error experienced by python on FreeBSD is env python: No such file or directory. To avoid this error, follow these steps:
Delete existing python files.

root@ns1:~ # rm -R -f /usr/local/bin/python
Create a link, adapt it to the python that we will run. In this case we use python39.

root@ns1:~ # ln -s /usr/local/bin/python3.9 /usr/local/bin/python


3. Create a Python py Script

In this session we will create a py script that will rotate Proxy IPs from Tor and Privoxy. Before you create a py script, open the /usr/local/etc/tor folder, because we will edit the torrc file first. Activate the "CookieAuthentication" script by deleting the "#" sign in front of the script.

#CookieAuthentication 1

become

CookieAuthentication 1

Next we will create a password "HashedControlPassword", in this article we assume that to create this password we will use the password "12345678". After we determine the password, in the remote PUTTY console we type the script "tor --hash-password my_password" (for example, password: 12345678). The following is an example of writing in PUTTY.

root@ns1:~ # cd /usr/local/etc/tor
root@ns1:/usr/local/etc/tor # tor --hash-password 12345678
May 30 13:54:29.399 [warn] You are running Tor as root. You don't need to, and you probably shouldn't.
16:9D461F3F7FFA43C060AE6D904DD812BD51148C8CD9992D5DBBFFF930F8
root@ns1:/usr/local/etc/tor #
The script colored "blue" above is the hash generated by the script "tor --hash-password 12345678". We will enter this hash into the script in the /usr/local/etc/tor/torrc file. The next step we will edit the TOR configuration file, namely the /usr/local/etc/tor/torrc file. If you are using Windows, use PUTTY/WINSCP to make editing easier. By using PUTTY in the /usr/local/etc/tor/torrc file, type the script below.

root@ns1:~ # ee /usr/local/etc/tor/torrc
ControlPort 9051
HashedControlPassword 16:9D461F3F7FFA43C060AE6D904DD812BD51148C8CD9992D5DBBFFF930F8
In the HashedControlPassword script it must be the same as the blue script above. After everything is configured correctly, our next step is to restart the TOR application.

root@ns1:/usr/local/etc/tor # service tor restart
After we configure the TOR application, continue by configuring the Python application. This step is very important, because Python does the Public IP rotation. The first step is to create a ConnectionManager.py file which is placed in the /usr/local/etc/tor folder. Here's how to create the file.

root@ns1:/usr/local/etc/tor # touch ConnectionManager.py
root@ns1:/usr/local/etc/tor # chmod -R 755 ConnectionManager.py
After creating the file /usr/local/etc/tor/ConnectionManager.py, enter the following script in the file.

root@ns1:/usr/local/etc/tor # ee ConnectionManager.py

import time
import stem
import stem.connection
import requests
from stem import Signal
from stem.control import Controller

with Controller.from_port(port=9051) as controller:
	controller.authenticate(password="12345678")
	controller.signal(Signal.NEWNYM)
	controller.close()

proxies = {"http": "http://192.168.5.2:8008"}

headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.73.11 (KHTML, like Gecko) Version/7.0.1 Safari/537.73.11'}

r = requests.get("http://icanhazip.com", proxies=proxies, headers=headers)

print (r.text)
Pay attention to the blue script above, the password "12345678" is the password that we specified in the discussion above. Meanwhile http://192.168.5.2:8008 is the IP and port of Privoxy.


4. Test Script Python py

Now let's test the ConnectionManager.py file, if you use the putty console you must be active in the /usr/local/etc/tor folder because the py file we created is in that folder, if you are still in root, change it to the /usr/local folder /etc/tor with the script cd /usr/local/etc/tor. Below is how to test the ConnectionManager.py file script.

root@ns1:~ # cd /usr/local/etc/tor
root@ns1:/usr/local/etc/tor # python ConnectionManager.py
103.251.167.20
IP 103.251.167.20 in the script above is the Public IP which comes from the proxy server on our FreeBSD computer. Wait approximately 1 minute, and do the test again. Look at the results, the Public IP will definitely change.

root@ns1:/usr/local/etc/tor # python ConnectionManager.py
173.249.57.253
Every time we run ConnectionManager.py, our Public IP address will continue to change. You can also check each change to the Public IP on the website "https://icanhazip.com/".

After that we continue by creating a Public IP rotation file, we name it rotate.py.

root@ns1:~ # cd /usr/local/etc/tor
root@ns1:/usr/local/etc/tor # touch rotate.py
root@ns1:/usr/local/etc/tor # chmod -R 755 rotate.py
In the file /usr/local/etc/tor/rotate.py we enter the script below.

root@ns1:/usr/local/etc/tor # ee /usr/local/etc/tor/rotate.py

import time
from stem import Signal
from stem.control import Controller
def main():
    while True:
        time.sleep(20)
        print ("Merotasi IP Proxy")
        with Controller.from_port(port = 9051) as controller:
          controller.authenticate()
          controller.signal(Signal.NEWNYM)
if __name__ == '__main__':
    main()
Run the rotate.py file script.

root@ns1:~ # cd /usr/local/etc/tor
root@ns1:/usr/local/etc/tor #  python rotate.py
Merotasi IP Proxy
Merotasi IP Proxy
Merotasi IP Proxy
Merotasi IP Proxy
Merotasi IP Proxy
Merotasi IP Proxy
Merotasi IP Proxy
The Tor network can be used to send requests to different public IP addresses. In this tutorial we have used the TOR controller to change the Public IP address with the help of a Python program, and you already know how. If you follow this tutorial carefully, you should have successfully configured regular Public IP rotation.
Iwan Setiawan

I Like Adventure: Mahameru Mount, Rinjani Mount I Like Writer FreeBSD

Post a Comment

Previous Post Next Post