Skip to content

Commit

Permalink
Update scraper.py
Browse files Browse the repository at this point in the history
  • Loading branch information
zGadli committed Mar 20, 2024
1 parent e1c1925 commit b9ff32b
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion lncrawl/core/scraper.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ def __process_request(self, method: str, url, **kwargs):
headers.setdefault("Referer", self.last_soup_url or self.home_url)
headers.setdefault("User-Agent", self.user_agent)
kwargs["headers"] = {
str(k).encode("ascii"): str(v).encode("ascii")
str(k).encode("utf-8"): str(v).encode("utf-8")
for k, v in headers.items()
if v
}
Expand Down

0 comments on commit b9ff32b

Please sign in to comment.