COVID 19 live tracker web scraper
Ok so Im working on a web scraper that will scrape off data from a website, (This one!) and I managed to scrape off some data, but the data I want wont scrape. Heres what I mean: If you look at the website, you will see around 37M cases as of when this post was created. And when I scrape that data off with this code:
const postJobButton = data('a#valueTot.numbers').text();
it only returns 0, why is that? And ignore the goodCode.js, it does nothing. I just save my good code there that file.
FUCK
I don't do JS, but I would recommend Python for web scraping in the future. Literally takes 2 lines of code.
@DynamicSquid I have not coded in Python in so long I forgot how to do some stuff now lol. And I am also going to make a discord bot that keeps track of the live covid count. Im used to node js for networking and discord bots. But thanks! I will try it some time
python and node.js will have no effect on source code smh @DynamicSquid
@Coder100 No, Python's way simpler
simple != work @DynamicSquid
@Coder100 no python's simplicity works
@Coder100 my web scraper used python and it worked great!
but that's repl.it, they have the data in the source @DynamicSquid
@Coder100 wdym?
it's hard-coded inside in fact to prove it to you, try scraping the covid19 tracker in question, you'll see there is no difference @DynamicSquid
seee the comments are hard-coded
@Coder100 wait what's hardcoded?
the comments smh @DynamicSquid
@Coder100 but what do the comments have to do with anything?
thats what you are scraping is it not @DynamicSquid
what I am trying to say is that all the data is already hard-coded in the HTML:
<div class="comment"> oh wow python can definitely do this better than node.js </div>
@Coder100 why would you scrap comments? And yes python is better than js since it's much cleaner
@DynamicSquid yuck ruby better
oh well I will not argue with idiots
oh well I will not argue when I am losing @DynamicSquid
Here's what happened:
the source code says
0! So where does the actual number come from then? JS, how pesky >:(so basically what this means is that you can't scrape the data using your current method :( You are going to have to execute js code too (which isn't possible with your current setup)
hey hey whats this
if you could find a way to
eval()this piece of code you will be able to get covid data!psst...
use an API