Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Simplify README. Make User friendly and up-to-date #152

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Simplify README. Make User friendly and up-to-date #152

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f5e4794
Update README.md
sequencerr Dec 25, 2023
788e341
Update README.md. Fix port
sequencerr Dec 25, 2023
c91cfe7
Merge branch 'master' into dev
bluet Nov 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
200 changes: 60 additions & 140 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
*Porting to Python3.10+ is painful and the progress is moving slowly.*
*We need more volunteers to join. PRs welcome! :joy:*
*We need more volunteers to join. PRs welcome!*

ProxyBroker
===========
Expand Down Expand Up @@ -31,118 +31,25 @@ Features
- Automatically removes duplicate proxies.
- Is asynchronous.

Docker
------
Docker Hub https://hub.docker.com/r/bluet/proxybroker2

```
$ docker run --rm bluet/proxybroker2 --help
usage: proxybroker [--max-conn MAX_CONN] [--max-tries MAX_TRIES]
[--timeout SECONDS] [--judge JUDGES] [--provider PROVIDERS]
[--verify-ssl]
[--log [{NOTSET,DEBUG,INFO,WARNING,ERROR,CRITICAL}]]
[--min-queue MINIMUM_PROXIES_IN_QUEUE]
[--version] [--help]
{find,grab,serve,update-geo} ...

Proxy [Finder | Checker | Server]

Commands:
These are common commands used in various situations

{find,grab,serve,update-geo}
find Find and check proxies
grab Find proxies without a check
serve Run a local proxy server
update-geo Download and use a detailed GeoIP database

Options:
--max-conn MAX_CONN The maximum number of concurrent checks of proxies
--max-tries MAX_TRIES
The maximum number of attempts to check a proxy
--timeout SECONDS, -t SECONDS
Timeout of a request in seconds. The default value is
8 seconds
--judge JUDGES Urls of pages that show HTTP headers and IP address
--provider PROVIDERS Urls of pages where to find proxies
--verify-ssl, -ssl Flag indicating whether to check the SSL certificates
--min-queue MINIMUM_PROXIES_IN_QUEUE The minimum number of proxies in the queue for checking connectivity
--log [{NOTSET,DEBUG,INFO,WARNING,ERROR,CRITICAL}]
Logging level
--version, -v Show program's version number and exit
--help, -h Show this help message and exit

Run 'proxybroker <command> --help' for more information on a command.
Suggestions and bug reports are greatly appreciated:
<https://github.com/bluet/proxybroker2/issues>

```


Requirements
------------

- Python 3.8+
- [aiohttp](https://pypi.python.org/pypi/aiohttp)
- [aiodns](https://pypi.python.org/pypi/aiodns)
- [maxminddb](https://pypi.python.org/pypi/maxminddb)

Installation
------------

### Install locally

To install last stable release from pypi:
> NOT RECOMMEND. It will install the out-dated original proxybroker package, which is no longer maintained by original maintainer. [https://github.com/constverum/ProxyBroker](https://github.com/constverum/ProxyBroker/issues/195)
> We will upload the up-to-date package under new name (proxybroker2) when the support for 3.10 is ready. [https://github.com/bluet/proxybroker2/issues/89](https://github.com/bluet/proxybroker2/issues/89)

``` {.sourceCode .bash}
$ pip install proxybroker
```

To install the latest development version from GitHub:

``` {.sourceCode .bash}
$ pip install -U git+https://github.com/bluet/proxybroker2.git
```
> Pypi package in progress. We will upload the up-to-date package under new name (proxybroker2) when the support for 3.10 is ready. https://github.com/bluet/proxybroker2/issues/89

### Use pre-built Docker image

``` {.sourceCode .bash}
$ docker pull bluet/proxybroker2
$ echo 'alias proxybroker="docker run -p 8888:8888 -v --rm bluet/proxybroker2"' >> ~/.bashrc && . ~/.bashrc # create alias
Comment on lines +39 to +45
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Docker installation instructions need improvement

The current Docker instructions have several issues:

  1. The alias command is complex and might not work on all shells
  2. The -v flag is incomplete (no volume mapping)
  3. Missing explanation of the port mapping
-$ echo 'alias proxybroker="docker run -p 8888:8888 -v --rm bluet/proxybroker2"' >> ~/.bashrc && . ~/.bashrc # create alias
+# Create an alias for easier usage (optional)
+# For bash users:
+$ echo 'alias proxybroker="docker run --rm -p 8888:8888 bluet/proxybroker2"' >> ~/.bashrc && source ~/.bashrc
+
+# For zsh users:
+$ echo 'alias proxybroker="docker run --rm -p 8888:8888 bluet/proxybroker2"' >> ~/.zshrc && source ~/.zshrc
+
+# Note: Port 8888 is used for the proxy server. You can change it if needed.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
> Pypi package in progress. We will upload the up-to-date package under new name (proxybroker2) when the support for 3.10 is ready. https://github.com/bluet/proxybroker2/issues/89
### Use pre-built Docker image
``` {.sourceCode .bash}
$ docker pull bluet/proxybroker2
$ echo 'alias proxybroker="docker run -p 8888:8888 -v --rm bluet/proxybroker2"' >> ~/.bashrc && . ~/.bashrc # create alias
> Pypi package in progress. We will upload the up-to-date package under new name (proxybroker2) when the support for 3.10 is ready. https://github.com/bluet/proxybroker2/issues/89
### Use pre-built Docker image
🧰 Tools
🪛 Markdownlint (0.35.0)

41-41: Expected: setext; Actual: atx
Heading style

(MD003, heading-style)

44-44: null
Dollar signs used before commands without showing output

(MD014, commands-show-output)

45-45: null
Dollar signs used before commands without showing output

(MD014, commands-show-output)

39-39: null
Bare URL used

(MD034, no-bare-urls)

```

### Build bundled one-file executable with pyinstaller

#### Requirements
Supported Operating System: Windows, Linux, MacOS

*On UNIX-like systems (Linux / macOSX / BSD)*

Install these tools
- upx
- objdump (this tool is usually in the binutils package)
``` {.sourceCode .bash}
$ sudo apt install -y upx-ucl binutils # On Ubuntu / Debian
```

#### Build

```
pip install pyinstaller \
&& pip install . \
&& mkdir -p build \
&& cd build \
&& pyinstaller --onefile --name proxybroker --add-data "../proxybroker/data:data" --workpath ./tmp --distpath . --clean ../py2exe_entrypoint.py \
&& rm -rf tmp *.spec
```

The executable is now in the build directory

Usage
Usage. CLI Examples
-----

### CLI Examples
Run `proxybroker --help` for more information on the options available.
Run `proxybroker <command> --help` for more information on a command.

#### Find

Expand Down Expand Up @@ -174,36 +81,10 @@ $ proxybroker serve --host 127.0.0.1 --port 8888 --types HTTP HTTPS --lvl High -

![image](https://raw.githubusercontent.com/constverum/ProxyBroker/master/docs/source/_static/cli_serve_example.gif)

Run `proxybroker --help` for more information on the options available.
Run `proxybroker <command> --help` for more information on a command.

### Basic code example

Find and show 10 working HTTP(S) proxies:

``` {.sourceCode .python}
import asyncio
from proxybroker import Broker

async def show(proxies):
while True:
proxy = await proxies.get()
if proxy is None: break
print('Found proxy: %s' % proxy)

proxies = asyncio.Queue()
broker = Broker(proxies)
tasks = asyncio.gather(
broker.find(types=['HTTP', 'HTTPS'], limit=10),
show(proxies))

loop = asyncio.get_event_loop()
loop.run_until_complete(tasks)
```

[More examples](https://proxybroker.readthedocs.io/en/latest/examples.html).

### Proxy information per requests
<details><summary>Expand Dropworn</summary>
<br>

#### HTTP
Check `X-Proxy-Info` header in response.
```
Expand Down Expand Up @@ -361,26 +242,65 @@ $ http_proxy=http://127.0.0.1:8888 https_proxy=http://127.0.0.1:8888 curl -v htt
<
* Connection #0 to host 127.0.0.1 left intact
```

Documentation
-------------

<https://proxybroker.readthedocs.io/>
</details>

TODO
----

- Check the ping, response time and speed of data transfer
- Check site access (Google, Twitter, etc) and even your own custom URL's
- Information about uptime
- Checksum of data returned
- Support for proxy authentication
- Finding outgoing IP for cascading proxy
- The ability to specify the address of the proxy without port (try to connect on defaulted ports)
- [ ] Create docs website for new proxybroker
- [ ] Check the ping, response time and speed of data transfer
- [ ] Check site access (Google, Twitter, etc) and even your own custom URL's
- [ ] Information about uptime
- [ ] Checksum of data returned
- [ ] Support for proxy authentication
- [ ] Finding outgoing IP for cascading proxy
- [ ] The ability to specify the address of the proxy without port (try to connect on defaulted ports)

Contributing
------------

To install the latest development version from GitHub:

``` {.sourceCode .bash}
$ pip install -U git+https://github.com/bluet/proxybroker2.git
```

### Build bundled one-file executable with pyinstaller

#### Requirements
Supported Operating System: Windows, Linux, MacOS

*On UNIX-like systems (Linux / macOSX / BSD)*

Install these tools
- upx
- objdump (this tool is usually in the binutils package)
``` {.sourceCode .bash}
$ sudo apt install -y upx-ucl binutils # On Ubuntu / Debian
```

#### Build

```
pip install pyinstaller \
&& pip install . \
&& mkdir -p build \
&& cd build \
&& pyinstaller --onefile --name proxybroker --add-data "../proxybroker/data:data" --workpath ./tmp --distpath . --clean ../py2exe_entrypoint.py \
&& rm -rf tmp *.spec
```

The executable is now in the build directory

### Requirements

- Python 3.8+
- [aiohttp](https://pypi.python.org/pypi/aiohttp)
- [aiodns](https://pypi.python.org/pypi/aiodns)
- [maxminddb](https://pypi.python.org/pypi/maxminddb)

### Workflow

- Fork it: <https://github.com/bluet/proxybroker2/fork>
- Create your feature branch: `git checkout -b my-new-feature`
- We use [Poetry](https://python-poetry.org/) to manage dependencies. If need, install dependencies: `poetry install`
Expand Down
Loading