Problem Description

I have a project that requires validating URLs extracted from user comments and notes. (see previous bounty) I need a validation process to check the URLs by making actual HTTP requests to ensure they are accessible and return if it is valid or what sort of error if not. It should account for various failure possibilities and provide error types for each failure scenario.

Some failure cases for example: bad URL, expired domain, 404, etc.

Other cases will probably have to have specific site exceptions, like Walmart or Twitter links (bot blocking, login/paywalls) since not necessarily dead, just hard to check, so don't want them marked broken so some way to mark domains as uncheckable in results rather than invalid status which would get filtered out.

Should follow redirects, but note them in results (some links from shortener services).

Acceptance Criteria

I will provide a set of URLs extracted from comments and notes for testing the URL validation function. The function should be able to differentiate between different types of errors that can occur during the validation process.

Technical Details

I want ideally a simple 1-2 function interface to pass a list of URLs and get a matching list of statuses. Something easy to drop into my existing project.

After testing the URL validation function, I will integrate it into my project to ensure the extracted URLs are valid. Please indicate if any dependencies need to be installed or set up for the function to work correctly.