Search Results for "python web crawler"
Sort By:
Showing 90 open source projects for "python web crawler"
View related business solutions-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
1
SearXNG
Free internet metasearch engine which aggregates
SearXNG is a free and open-source metasearch engine designed to aggregate results from multiple search engines while prioritizing user privacy and anonymity. Instead of maintaining its own index, it queries numerous external search providers and merges the results into a single interface, increasing coverage and diversity of information. One of its core principles is privacy, as it does not track users, store personal data, or create search profiles, making it a strong alternative to... -
2
Anna’s Archive
Comprehensive search engine for books, papers, comics, magazines
Anna’s Archive is a large-scale open-source search engine and data aggregation platform designed to index and provide access to a vast collection of books, academic papers, comics, magazines, and other digital texts through a unified interface. The project includes all the infrastructure required to run a full instance locally or in production, combining web servers, databases, and search indexing systems into a scalable architecture. It relies heavily on technologies such as Elasticsearch... -
3
ddgr
DuckDuckGo from the terminal
ddgr is a cmdline utility to search DuckDuckGo from the terminal. While googler is highly popular among cmdline users, in many forums the need of a similar utility for privacy-aware DuckDuckGo came up. DuckDuckGo Bangs are super-cool too! So here's ddgr for you! Unlike the web interface, you can specify the number of search results you would like to see per page. It's more convenient than skimming through 30-odd search results per page. The default interface is carefully designed to use... -
4
dorker-py
Descubre archivos, rutas escondidas realizando busquedas avanzadas
Dorking Google - Dorker Py Descubre archivos, rutas escondidas realizando busquedas avanzadas (ES) Discover files, hidden paths by performing advanced searches (EN) -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
5
Swagbucks Web Search Bot
This is a automatic swagbucks search automator.
Basically this allows you to be able to automate searches by putting in search terms through any .txt file appropriately formatted and then the program picks that up and basically opens the link through your computer. You must login to Swagbucks for this to work. With that in mind, the title speaks for itself. To find new releases, check the version folder in each designated Operating System here. To get archived releases (ill-advised), check out the GitHub:... -
6
-
7
googler
Google Search, Google Site Search, Google News from the terminal
googler is a power tool to Google (Web & News) and Google Site Search from the command-line. It shows the title, URL and abstract for each result, which can be directly opened in a browser from the terminal. Results are fetched in pages (with page navigation). Supports sequential searches in a single googler instance. googler was initially written to cater to headless servers without X. You can integrate it with a text-based browser. However, it has grown into a very handy and flexible... -
8
SpiderFoot
Open Source Intelligence Automation.
SpiderFoot is an open source intelligence automation tool. Its goal is to automate the process of gathering intelligence about a given target, which may be an IP address, domain name, hostname or network subnet. SpiderFoot can be used offensively, i.e. as part of a black-box penetration test to gather information about the target or defensively to identify what information your organisation is freely providing for attackers to use against you. -
9
CFPanel
Web panel for keeping track of crowdfunding projects
A web panel where to keep track of crowdfunding projects in which you have taken part. -
$300 Free Credits to Build on Google CloudStart your next project with $300 in free Google Cloud credit. Spin up VMs, run containers, query petabytes in BigQuery, or build agents with Gemini Enterprise Agent Platform. Once your credits are used, keep building with 20+ always-free tier products including Compute Engine, Cloud Storage, GKE, and Cloud Run functions. No commitment required—just sign up and start building.
-
10
MovieGrabber
MovieGrabber is a fully automated way of downloading
MovieGrabber has now moved to GitHub! https://github.com/binhex/moviegrabber -
11
SE Auditor
Free SEO audit software.
SE Auditor is a program for analyzing web pages for search engines. SE Auditor is application that you can use to view statistical data about your website, in order to improve its position within the Web search results. SE Auditor is addressed to SEO professionals, website designers, developers, website testers and owners. SE Auditor enables you to check meta description, keywords, sitemap, the number of links and keyword consistency, the text/HTML ratio and many more ranking /... -
12
python-webrobot
download video from various hosters
Allows to automatically download files and video streams from web hosters. -
13
Framework (scripts, configuration, code) to build free and public services around travel and leisure data. That project makes an extensive use of already existing data sources such as Geonames and dbPedia, and adds some glue around those (eg, links).
-
14
PACS VM
ISO - Customized version of dcm4chee 2.17.3 for MySQL.
1. Add JBoss Application Server 4.2.3.GA for JDK 6. 2. Cleanup for Windows and deprecated files. 3. Off CONSOLE records - http://forums.dcm4che.org/jiveforums/thread.jspa?messageID=4787 -
15
Search Engine in python
All student and developers are invited to join this web search engine
This search engine project is created by a computer science student to challenge how search results are returned. Importance is placed on how web pages are ranked by a quick and efficient algorithm. All hands on deck is how we are calling this project. Algorithm design is really important and creative methods from better search results or web crawling is welcomed from all amateur programmers, students, web developers, software developers and computer science students from around the globe.... -
16
"Swish-e is a fast, flexible, and free open source system for indexing collections of Web pages or other files" (http://swish-e.org/ ) This module provides a Python API for this software.
-
17
MindRetrieve is a personal search engine. It helps you organize and retrieve web pages you have visited. MindRetrieve is a lightweight, cross-platform, open source application available under the BSD license. It works with all popular web browsers.
-
18
The Semantic Web Peer - Allegra is an innovative library for Semantic Web applications development. It provides a framework for asynchronous network peer programming, a simple stack of Internet standards implementations, and two new network applications:
-
19
-
20
-
21
Document summarization system. By adding document content to system, user queries will generate a summary document containing the available information to the system.
-
22
A Web application to search for files on FTP servers. Users can query files by part of the file name, the entire file name, a regular expression, or a shell pattern. To store file indexes, PostgreSQL or MySQL is used.
-
23
BTG Web Search Engine is developed for students or developers who are interested in search engine technology. The engine web will be written in C++, Java, Perl,and Python. At the beginning of the project, a simple search engine design will be used.
-
24
Voxound Extension is a daemon application used to provide additional local content access and management functionality to the voxound.com web application.
-
25
a small collection of python 3000 scripts/modules used to automate searching craigslist.org cities and categories for interesting stuff; these scripts currently use html screen scraping, since craigslist currently has no api
