Re: Making Python Script Exit Completely with One Ctrl+C

On Sat, Dec 9, 2023 at 11:40 AM Ray Satiro via curl-users
<[email protected]> wrote:
>
> On 12/8/2023 8:49 PM, Hongyi Zhao via curl-users wrote:
>
> On Sat, Dec 9, 2023 at 9:37 AM Hongyi Zhao <[email protected]> wrote:
>
> On Fri, Dec 8, 2023 at 10:44 AM Dan Fandrich via curl-users
> <[email protected]> wrote:
>
> On Fri, Dec 08, 2023 at 09:56:44AM +0800, Hongyi Zhao via curl-users wrote:
>
> I tried the following method but in vain:
>
> [...]
>
>     try:
>         pass
>     except KeyboardInterrupt:
>         kb_interrupt = True
>
> This won't accomplish anything because "pass" will never raise an exception.
> I have a feeling that even putting the entire contents of the progress callback
> into a try/except block won't even help; I suspect the exception is raised the
> moment the Python interpreter is returned to by pycurl while calling that
> callback, which is before even that point.
>
> You'll probably have to block SIGINT before calling calling into pycurl, then
> unblocking in the progress callback to check it, e.g.
>
> ...
> signal.pthread_sigmask(signal.SIG_BLOCK, [signal.SIGINT])
> c.perform()
> signal.pthread_sigmask(signal.SIG_UNBLOCK, [signal.SIGINT])
> ...
>
> def progress(...):
>     try:
>         signal.pthread_sigmask(signal.SIG_UNBLOCK, [signal.SIGINT])
>     except KeyboardInterrupt:
>         print('interrupted!')
>         return 1
>     finally:
>         signal.pthread_sigmask(signal.SIG_BLOCK, [signal.SIGINT])
>
> or just set your own SIGINT handler that sets a flag (kb_interrupt), then check
> that flag in the progress callback and return an error if it's set.
>
> I tried using the first method of your tips. It works, but the
> calculated *Average Speed* is too high, as shown below:
>
> (datasci) werner@X10DAi:~/Desktop$ python callback-SIGINT.py
> Proxy: SG_ssr_futeapbquf5m.nodelist.club_1453_018d83c677e05e71f172014fc3f45e39
> Current Speed: 1371.74 kB/s
> Current Speed: 6565.90 kB/s
> Average Speed: 7552.72 kB/s
> Proxy: HK_ssr_wo8o8npg4fny.nodelist.club_1303_b5bf85111d0f51c517ec7302d3f33ce1
> Current Speed: 1252.55 kB/s
> Current Speed: 7461.93 kB/s
> Average Speed: 8807.27 kB/s
> Proxy: SG_ssr_futeapbquf5m.nodelist.club_1354_ddfba110eddfdb7037f389e9b5917477
> Current Speed: 3081.76 kB/s
> Current Speed: 9351.58 kB/s
> Average Speed: 10745.23 kB/s
> Proxy: HK_ssr_fwqvvo60u1mj.nodelist.club_1423_dba3b0996744e7b82eae7634626a5ba9
> ^Cinterrupted!
> Average Speed: 156.74 kB/s
> Proxy: SG_ssr_futeapbquf5m.nodelist.club_1356_8ce27b1301fcbcb77cc5abb28cb127a6
> Script terminated by user. Exiting.
>
> I'm not sure how to fix this problem. Attached please find the test script.
>
> I tried another method which also had the same problem:
>
> (datasci) werner@X10DAi:~/Desktop$ python rev-2.py
> Proxy: SG_ssr_futeapbquf5m.nodelist.club_1453_018d83c677e05e71f172014fc3f45e39
> Current Speed: 664.56 kB/s
> Current Speed: 7691.08 kB/s
> Average Speed: 9403.12 kB/s
> Proxy: HK_ssr_wo8o8npg4fny.nodelist.club_1303_b5bf85111d0f51c517ec7302d3f33ce1
> Current Speed: 950.39 kB/s
> Current Speed: 6146.08 kB/s
> Average Speed: 7297.46 kB/s
> Proxy: SG_ssr_futeapbquf5m.nodelist.club_1354_ddfba110eddfdb7037f389e9b5917477
> ^CTraceback (most recent call last):
>   File "/home/werner/Desktop/rev-2.py", line 60, in progress
>     def progress(download_t, download_d, upload_t, upload_d):
>
> KeyboardInterrupt
>
>
> As far as I can tell you have some issues using Python and have been receiving help in a Python discussion thread [1]. How does curl factor into this except that the average speed looks wrong to you?
>
> Your code does:
>
> average_speed = c.getinfo(pycurl.SPEED_DOWNLOAD) / 1024
>
> That should be ok. You are calculating the speed in KiB (though a lot of people will abbreviate it as KB), and not kB (notice the lowercase 'k'). If you suspect that average speed pycurl is giving you is wrong then you could compare against the curl tool.
>
> For example  (based off your python code):
>
> curl --max-time 5 --proxy socks5h://SG_ssr_futeapbquf5m.nodelist.club_1453_018d83c677e05e71f172014fc3f45e39 -o /dev/null http://ipv4.download.thinkbroadband.com/1GB.zip
>
> that would show progress information as the file is being downloaded (to /dev/null) and the average speed.
>
>
> [1]: https://discuss.python.org/t/making-python-script-exit-completely-with-one-ctrl-c/40529/1

Now, I try to compare the manually computed average speed and the one
given by pycurl in the attached script, but the latter always is 0:

(datasci) werner@X10DAi:~/Desktop$ python rev-3.py
Testing Proxy: SG_ssr_futeapbquf5m.nodelist.club_1453_018d83c677e05e71f172014fc3f45e39
Current Speed: 75.63 kB/s
Current Speed: 5395.45 kB/s
PycURL Speed: 0.00 kB/s
Average Speed: 4389.14 kB/s
Testing Proxy: HK_ssr_wo8o8npg4fny.nodelist.club_1303_b5bf85111d0f51c517ec7302d3f33ce1
Current Speed: 2138.56 kB/s
Current Speed: 7901.99 kB/s
PycURL Speed: 0.00 kB/s
Average Speed: 6323.33 kB/s
Testing Proxy: SG_ssr_futeapbquf5m.nodelist.club_1354_ddfba110eddfdb7037f389e9b5917477
Current Speed: 937.87 kB/s
Current Speed: 7540.93 kB/s
PycURL Speed: 0.00 kB/s
Average Speed: 6209.84 kB/s
Testing Proxy: HK_ssr_fwqvvo60u1mj.nodelist.club_1423_dba3b0996744e7b82eae7634626a5ba9
^CKeyboardInterrupt at line 70

I don't know how to fix this problem. Any tips will be appreciated.

Regards,
Zhao

> --
> Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-users
> Etiquette:   https://curl.se/mail/etiquette.html

-- 
Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-users
Etiquette:   https://curl.se/mail/etiquette.html

import subprocess
import time
import pycurl
from io import BytesIO
import sys
import signal

def sigint_handler(signal, frame):
    print(f"KeyboardInterrupt at line {frame.f_lineno}")
    sys.exit(0)

signal.signal(signal.SIGINT, sigint_handler)

def fetch_proxies():
    command = 'echo "show stat" | sudo socat stdio /var/run/haproxy.sock 2>/dev/null | awk -F, \'$1=="socks5" && !($2~/^(FRONTEND|BACKEND)$/) {print $2,$74}\''
    try:
        process = subprocess.Popen(command, stdout=subprocess.PIPE, shell=True)
        output, error = process.communicate()
    except Exception as e:
        print("Error occurred while executing shell command: ", e)
        return {}

    result_dict = {}

    if error is not None:
        print("Error occurred while getting proxies status: ", error.decode('utf-8'))

    lines = output.decode('utf-8').split('\n')

    for line in lines:
        if line and len(line.split(' ')) == 2:
            key_value = line.split(' ')
            result_dict[key_value[0]] = key_value[1]

    return result_dict

def test_proxy(proxy, url):
    global last_calc_time, download_start_time, total_downloaded_data
    buffer = BytesIO()

    c = pycurl.Curl()
    c.setopt(pycurl.URL, url)
    c.setopt(pycurl.WRITEDATA, buffer)
    c.setopt(pycurl.PROXY, proxy)
    c.setopt(pycurl.PROXYTYPE, pycurl.PROXYTYPE_SOCKS5_HOSTNAME)
    c.setopt(pycurl.NOPROGRESS, False)
    c.setopt(pycurl.XFERINFOFUNCTION, progress)
    c.setopt(pycurl.TIMEOUT, 5)

    download_start_time = time.time()
    last_calc_time = download_start_time
    total_downloaded_data = 0
    py_speed = 0

    try:
        c.perform()
        py_speed = c.getinfo(pycurl.SPEED_DOWNLOAD) / 1024
    except pycurl.error as e:
        pass
    finally:
        c.close()

    elapsed_time = time.time() - download_start_time
    average_speed = total_downloaded_data / elapsed_time / 1024
    print(f"PycURL Speed: {py_speed:.2f} kB/s")

    return average_speed


def progress(download_t, download_d, upload_t, upload_d):
    global last_calc_time, download_start_time, total_downloaded_data
    current_time = time.time()

    if current_time - last_calc_time >= 2:
        elapsed_time = current_time - download_start_time  # Time since download start
        current_speed = download_d / elapsed_time / 1024   # Speed since download start
        print(f"Current Speed: {current_speed:.2f} kB/s")
        last_calc_time = current_time
        total_downloaded_data = download_d

    return 0  

proxy_data = fetch_proxies()

if not proxy_data:
    print("No proxy data found.")
    sys.exit(1)

url = "http://ipv4.download.thinkbroadband.com/1GB.zip"

for key, value in proxy_data.items():
    print(f"Testing Proxy: {key}")
    average_speed = test_proxy(value, url)
    print(f"Average Speed: {average_speed:.2f} kB/s")