For some specific use-cases, like the car dealership example above, this can save a lot of time, and frankly, there are a lot of business models built by this advent of web scrapers. One can write such an automated script fairly easily, with less than 10 lines of code, and automatically retrieve information from the web, obviating the need to search, organize, or interact with the website manually.
This process is also known as a web crawler or bot. The act of retrieving information from, or interacting with a site hosted on the web using an automated programming script or process. This is where web scraping comes in, and is defined as such: To do this manually, again would take a very long time.
Perhaps a project to view car prices for your favorite electric car on many car dealership websites. Now imagine another scenario where you need to take different parts of the web, and organize them for a specific purpose. Even if you somehow knew the address of each page, and assuming that you are only looking at a page for about 3 seconds, it would take you nearly 500 years to view everything. With the web now hosting almost 5 billion web pages, it would be impossible to view each one of these pages personally.
0 kommentar(er)
