Ask coding questions

← Back to all posts
Is it not possible to install scrapy on repl?
JamesXin (15)

I tried installing scrapy with import scrapy, but it says something about a recursion error...pls help

Comments
hotnewtop
YonatanBentzur (0)

I found a hacky workaround that works ok for me in case it's useful for anyone. Instead of installing the dependencies via Poetry I'm using pip.

I added a requirements.txt file to the project and put Scrapy == 2.5.0 there as the only line. Then in my main.py file I have:

import os
os.system('pip install -r requirements.txt')

And in my .replit file I added run = "python -m main" so when I hit the run button it installs scrapy successfully. After that you can change your .replit file back to whatever you want and using scrapy should just work for the rest of the session.

I've been doing that for a while now and it's a bit annoying for sure but it works and better than nothing.
I hope that helps!

KrishanuDhar (3)

All,
I am facing the same problem. I have not run into this on my PC or via pipenv install. Therefore, wondering what could be going wrong here.

Any help is appreciated.

KrishanuDhar (3)

I have removed the project.toml and retried. But the error follows, see below.

--> python3 -m poetry add Scrapy
Using version ^2.5.0 for Scrapy

Updating dependencies
Resolving dependencies...

[RecursionError]
maximum recursion depth exceeded
exit status 1

Repl.it: Package operation failed.

JamesXin (15)

@KrishanuDhar I found that if you name the project file scrapy or something it will cause packaging to fail due to how repl works. Try naming it something else.

KrishanuDhar (3)

@JamesXin None of the files I have are named scrapy. Any other recommendations?

Coder100 (18139)

Your pyproject.toml seems to be erroneous, so you should try deleting it and installing it through here:

JamesXin (15)

@Coder100 I tried that and now it gives an assertion error instead...I've tried installing scrapy on my own computer but it doesn't work either...

SUHASTADIPARTH1 (190)

Please mark my answer if it helps.

Thank you.