Skip to content
Sign upLog in
← Back to Community

Web scraping with Nodejs

Profile icon
HackermonDevHacker

What is web scraping and how does it work

Web scraping is when you go on a website then you find links on that website and then you can on those links to find more links and it basically keeps doing that and the links are stored in a database. This is how most search engine works. They use a web scraper to find links and then they show them to you when you need it.

Can you show me an example of a web scraper?

Sure! If you scroll down you will see a repl, the repl is an example of a web scraper. The default site is repl.it but you can change it. It uses cheerio to parse the HTML and then find links on the site.

What is cheerio?

Cheerio is a node library to parse HTML. If you ever see a web scraper chances are they are made with cheerio. Cheerio is really easy to use and you can find the docs on this page.

How do I make a search engine?

Well if you want to see me a template on that either upvote this post or tell me in the comments!

Conclusion

Web scraping is very cool

Voters
Profile icon
ViperPB
Profile icon
onionseeds
Profile icon
OriginKero
Profile icon
abdmmar
Profile icon
DiveshTheReal
Profile icon
WilliamDavidDa2
Profile icon
TheOceanFace
Profile icon
HackermonDev