Skip to content

Commit

Permalink
chore(chrome): fix ip connecting
Browse files Browse the repository at this point in the history
  • Loading branch information
j-mendez committed Sep 13, 2023
1 parent 7c60530 commit 3b5e227
Show file tree
Hide file tree
Showing 8 changed files with 22 additions and 22 deletions.
12 changes: 6 additions & 6 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ The fastest web crawler and indexer.

## Getting Started

The simplest way to get started is to use the new [hosted service](https://spiderwebai.xyz). View the [spider](./spider/README.md) or [CLI](./spider_cli/README.md) directory for local installations.
The simplest way to get started is to use our [hosted service](https://spiderwebai.xyz). View the [spider](./spider/README.md) or [CLI](./spider_cli/README.md) directory for local installations.

## Benchmarks

Expand Down
4 changes: 2 additions & 2 deletions examples/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "spider_examples"
version = "1.40.11"
version = "1.40.13"
authors = ["madeindjs <[email protected]>", "j-mendez <[email protected]>"]
description = "Multithreaded web crawler written in Rust."
repository = "https://github.com/spider-rs/spider"
Expand All @@ -22,7 +22,7 @@ htr = "0.5.27"
flexbuffers = "2.0.0"

[dependencies.spider]
version = "1.40.11"
version = "1.40.13"
path = "../spider"
features = ["serde"]

Expand Down
4 changes: 2 additions & 2 deletions spider/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "spider"
version = "1.40.11"
version = "1.40.13"
authors = ["madeindjs <[email protected]>", "j-mendez <[email protected]>"]
description = "The fastest web crawler written in Rust."
repository = "https://github.com/spider-rs/spider"
Expand Down Expand Up @@ -42,7 +42,7 @@ flexbuffers = { version = "2.0.0", optional = true }
itertools = { version = "0.10.5", optional = true }
case_insensitive_string = { version = "0.1.6", features = [ "compact", "serde" ]}
jsdom = { version = "0.0.11-alpha.1", optional = true, features = [ "hashbrown", "tokio" ] }
chromiumoxide_fork = { version = "0.5.7", optional = true, features = ["tokio-runtime", "bytes"], default-features = false }
chromiumoxide_fork = { version = "0.5.9", optional = true, features = ["tokio-runtime", "bytes"], default-features = false }

[target.'cfg(all(not(windows), not(target_os = "android"), not(target_env = "musl")))'.dependencies]
tikv-jemallocator = { version = "0.5.0", optional = true }
Expand Down
12 changes: 6 additions & 6 deletions spider/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ This is a basic async example crawling a web page, add spider to your `Cargo.tom

```toml
[dependencies]
spider = "1.40.11"
spider = "1.40.13"
```

And then the code:
Expand Down Expand Up @@ -87,7 +87,7 @@ We have a couple optional feature flags. Regex blacklisting, jemaloc backend, gl

```toml
[dependencies]
spider = { version = "1.40.11", features = ["regex", "ua_generator"] }
spider = { version = "1.40.13", features = ["regex", "ua_generator"] }
```

1. `ua_generator`: Enables auto generating a random real User-Agent.
Expand All @@ -114,7 +114,7 @@ Move processing to a worker, drastically increases performance even if worker is

```toml
[dependencies]
spider = { version = "1.40.11", features = ["decentralized"] }
spider = { version = "1.40.13", features = ["decentralized"] }
```

```sh
Expand All @@ -135,7 +135,7 @@ Use the subscribe method to get a broadcast channel.

```toml
[dependencies]
spider = { version = "1.40.11", features = ["sync"] }
spider = { version = "1.40.13", features = ["sync"] }
```

```rust,no_run
Expand Down Expand Up @@ -165,7 +165,7 @@ Allow regex for blacklisting routes

```toml
[dependencies]
spider = { version = "1.40.11", features = ["regex"] }
spider = { version = "1.40.13", features = ["regex"] }
```

```rust,no_run
Expand All @@ -192,7 +192,7 @@ If you are performing large workloads you may need to control the crawler by ena

```toml
[dependencies]
spider = { version = "1.40.11", features = ["control"] }
spider = { version = "1.40.13", features = ["control"] }
```

```rust
Expand Down
4 changes: 2 additions & 2 deletions spider_cli/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "spider_cli"
version = "1.40.11"
version = "1.40.13"
authors = ["madeindjs <[email protected]>", "j-mendez <[email protected]>"]
description = "The fastest web crawler CLI written in Rust."
repository = "https://github.com/spider-rs/spider"
Expand All @@ -26,7 +26,7 @@ quote = "1.0.18"
failure_derive = "0.1.8"

[dependencies.spider]
version = "1.40.11"
version = "1.40.13"
path = "../spider"

[[bin]]
Expand Down
2 changes: 1 addition & 1 deletion spider_cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ spider --domain http://localhost:3000 download -t _temp_spider_downloads
```

```sh
spider_cli 1.40.11
spider_cli 1.40.13
madeindjs <[email protected]>, j-mendez <[email protected]>
The fastest web crawler CLI written in Rust.

Expand Down
4 changes: 2 additions & 2 deletions spider_worker/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "spider_worker"
version = "1.40.11"
version = "1.40.13"
authors = ["madeindjs <[email protected]>", "j-mendez <[email protected]>"]
description = "The fastest web crawler CLI written in Rust."
repository = "https://github.com/spider-rs/spider"
Expand All @@ -22,7 +22,7 @@ lazy_static = "1.4.0"
env_logger = "0.10.0"

[dependencies.spider]
version = "1.40.11"
version = "1.40.13"
path = "../spider"
features = ["serde", "flexbuffers"]

Expand Down

0 comments on commit 3b5e227

Please sign in to comment.