Ask coding questions

← Back to all posts
YOUTUBE WEB SCRAPING
qwer1230 (0)

This is my code: When I run it I always get NONE,
from bs4 import BeautifulSoup
import lxml
import requests

htmlfile=requests.get('https://www.youtube.com/watch?v=_Bjf-iExroI&list=RD_Bjf-iExroI&start_radio=1&t=9').text
soup=BeautifulSoup(html_file,'lxml')
vid = soup.find('a',class
='yt-simple-endpoint style-scope ytd-playlist-panel-video-renderer')

print(vid)

Comments
hotnewtop
Coder100 (16869)

yes. That element is created using JAVASCRIPT. You have absolutely no way of finding that element whatsoever. Web scraping just isn't meant to do that.

What you should do instead is to use the youtube API

https://developers.google.com/youtube/v3/

much easier and more sane to use lol

Spotandjake (27)

@Coder100 the bot thought you were spam lol. also yes youtube API is best or I mean you can use ytdl or a similar module tbh as well lol, as the youtube API can be a little restricted lol. but webscraping is not the way to do it webscraping is more used for data collection like if you need results for weather and the network has no API but you should always try to find an API first as there are issues related to webscraping at the end of the day

JohnBarnhart (11)

Nice! love ur channel!

qwer1230 (0)

@JohnBarnhart what!!!, how does that help answer my question

CodeLongAndPros (1596)

@qwer1230 @JohnBarnhart
It doesn't.
John, I've warned you and am removing all of your begging. Please don't push it.

JohnBarnhart (11)

@CodeLongAndPros so sorry CodeLongAndPros. Thanks for warning me. I will not EVER do it again.