webcrawler lol
I make webcrawler I think. This is what I think a webcrawler is like. Is it? someone please tell me why it isnt.
youtube web crawler - https://repl.it/@generationXcode/youtube-logger
please suggest more websites to crawl in and if you want, fork this repl and change the url3 variable to be whichever website's URL you want to crawl and let it go.
see what you have crawled at https://regularwordydisc.generationxcode.repl.co/see
this one is safe, only shows like 100 results i think
see all that has ever been crawled at https://regularwordydisc.generationxcode.repl.co/POWER
beware they may be too many results and big boom
i get this error
@ch1ck3n hi, I'm building it to be better now. Connecting it to another service so that you all can use it as well and run it on your own repls. Rn there is a small issue with the service so I am playing around. Run it again, it works now. Also I removed the part of the program to write a file full of everything, I have a db where i store it all
@ch1ck3n haha ur using it now. I can see hehehehe. I have another service running rn that I just started up and boom so many results. Umm, I don't want to make you do any work, but if you could help out by just forking this repl and editting the url3 variable to be any other website you want this crawler to crawl thru. At this very momenbt I am working on making the results open to everyone, but I need your help here. Thanks
@ch1ck3n I wonder which website you will choose hmmmmmmmm. already have
@generationXcode oh no it is looping over one link
@generationXcode gasp how do i see
@ch1ck3n I have no idea
@ch1ck3n maybe some server issue with replit servers???
@generationXcode how do i see the links
@ch1ck3n Good point, can you wait about 5 mins. I'll quickly whip up a webpage for you to see all the links
@ch1ck3n https://regularwordydisc.generationxcode.repl.co/see
go here. the results arent properly sorted yet so I'll sort them out nicely and all. have fun!
@generationXcode my stuff arnt showing up
@ch1ck3n thats cus I have sorted it wrong and I cant have all 1000 results loaded on ur pc ur internet someday will go boom. Currently trying to find a way to sort
i give proof ur stuff there wait a sec
@generationXcode i have a good computer and my internet provider is always bragging about their products, time to show the power of developers
@generationXcode aha
you dont have to order it
@ch1ck3n ok I make special page 4 u. POWER
@generationXcode WOOOOOOO YEEEAAHHHH BOOOIII
delete 5 results please
@ch1ck3n done did
@ch1ck3n yep done
@ch1ck3n why did u want 2020 results tho???
@generationXcode the perfect number
@ch1ck3n ok... umm I'm starting the webcrawler again now. Is that alright?
@ch1ck3n ok im running it thru youtube but results arent going thru. Hmm. ig i'll work on that later now. I had a great time. Thanks!
Oh lmao u already ran it thru youtube
Yes, this is a web crawler. It works well. I have a few recommendations for you though. I sent it to you on discord.
Hi, this is unlisted as this contains less code. If you want me to relist this, write 10 to 15 more lines of code
@LegendaryWolf oof. What is the requirement? 60 lines?
lol, you got fooled! I'm not a mod. the minimum is 50 lines BTW
@LegendaryWolf umm ok... odd you would pretend to be a mod but ok lmao
OKIE, i updoot, I think you good at beautiful soup, can ya help me make a search engine? pls
@LegendaryWolf I'll see what I can do. Are you good with requests and P5js? I have a plan. Also, how many years have you programmed. I have a certain search engine I want to make that is different to the rest.
Info -
- Programing experience = 4 months 1 week
- good with requests = True / False
- if good with requests == True / False:
good with requests = medium
know JS = True
know p5.js = False
@LegendaryWolf lmao I was going to come up with an NDA but who rlly cares. Ok when I start making it all, I'll make sure you know about it. Maybe you could help out with the design part and quality of all the code.
I've made a search engine before in python - here it is -
https://replit.com/talk/share/Remastered-Alpha-Search-The-best-search-engine-add-your-own-results/125619
@LegendaryWolf idek if that counts but nice to know
the minimum is 50 lines BTW
@CodingCactus then what is the minimum
@ch1ck3n @generationXcode I don't think there is any minimum, but It's suggested to not post low effort posts.
@IntellectualGuy ig ur right
@ch1ck3n no specific minimum
koi. Also have you seen that every comment of LegendaryWolf's has one updoot? Can you check into this... @CodingCactus
@Whippingdot erm you know when you view the comments on their profiles, it shows the 'hot' ones, so of course they have a few upvotes
no, I am talking about in this comment thread @CodingCactus
@Whippingdot graphegg is probably just being nice and upvoting their replies, same is happening to me
@CodingCactus i don;t like when I see the black coloured upvote signs. The green ones calm me.
lul but you shouldn't upvote everything for that @generationXcode
k. i am just cautious cause he reached the amount of cycles I did in less than a month @CodingCactus
@Whippingdot lmao. I like doing it. I cant stop
u gotta stop. You can be banned for that @generationXcode
@Whippingdot I can upvote whatever I want. How can I be banned for that???
welp u r spam upvoting. That is giving free cycles to everyone (including me) which is the internet points for replit so... @generationXcode
cool +1 :P