New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bug: links is not getting properly #6

Open

Shyam-Raghuwanshi opened this issue Sep 29, 2024 · 0 comments

Shyam-Raghuwanshi commented Sep 29, 2024

when i am trying to get the links of supabase.com when we start first the links is good after a time the links is comming like this bugy links

`import { Website } from '@spider-rs/spider-rs'
import fs from "fs"
const website = new Website('https://supabase.com/docs')

console.time('doSomething')
await website.crawl()
const links = website.getLinks()`

and also when i trying to do the same thing for the vercel.com
its returning me 0 links and same for neon.tech

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment