Showing 88 open source projects for "python web crawler"

View related business solutions
  • Gen AI apps are built with MongoDB Atlas Icon
    Gen AI apps are built with MongoDB Atlas

    The database for AI-powered applications.

    MongoDB Atlas is the developer-friendly database used to build, scale, and run gen AI and LLM-powered apps—without needing a separate vector database. Atlas offers built-in vector search, global availability across 115+ regions, and flexible document modeling. Start building AI apps faster, all in one place.
    Start Free
  • Simple, Secure Domain Registration Icon
    Simple, Secure Domain Registration

    Get your domain at wholesale price. Cloudflare offers simple, secure registration with no markups, plus free DNS, CDN, and SSL integration.

    Register or renew your domain and pay only what we pay. No markups, hidden fees, or surprise add-ons. Choose from over 400 TLDs (.com, .ai, .dev). Every domain is integrated with Cloudflare's industry-leading DNS, CDN, and free SSL to make your site faster and more secure. Simple, secure, at-cost domain registration.
    Sign up for free
  • 1
    ddgr

    ddgr

    DuckDuckGo from the terminal

    ddgr is a cmdline utility to search DuckDuckGo from the terminal. While googler is highly popular among cmdline users, in many forums the need of a similar utility for privacy-aware DuckDuckGo came up. DuckDuckGo Bangs are super-cool too! So here's ddgr for you! Unlike the web interface, you can specify the number of search results you would like to see per page. It's more convenient than skimming through 30-odd search results per page. The default interface is carefully designed to use...
    Downloads: 4 This Week
    Last Update:
    See Project
  • 2

    dorker-py

    Descubre archivos, rutas escondidas realizando busquedas avanzadas

    Dorking Google - Dorker Py Descubre archivos, rutas escondidas realizando busquedas avanzadas (ES) Discover files, hidden paths by performing advanced searches (EN)
    Downloads: 3 This Week
    Last Update:
    See Project
  • 3
    Swagbucks Web Search Bot

    Swagbucks Web Search Bot

    This is a automatic swagbucks search automator.

    .../Swagbucks-Web-Search-Bot
    Downloads: 1 This Week
    Last Update:
    See Project
  • 4
    Software, information, data sets and documentation for the Web as Corpus community.
    Downloads: 0 This Week
    Last Update:
    See Project
  • Our Free Plans just got better! | Auth0 Icon
    Our Free Plans just got better! | Auth0

    With up to 25k MAUs and unlimited Okta connections, our Free Plan lets you focus on what you do best—building great apps.

    You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
    Try free now
  • 5
    googler

    googler

    Google Search, Google Site Search, Google News from the terminal

    googler is a power tool to Google (Web & News) and Google Site Search from the command-line. It shows the title, URL and abstract for each result, which can be directly opened in a browser from the terminal. Results are fetched in pages (with page navigation). Supports sequential searches in a single googler instance. googler was initially written to cater to headless servers without X. You can integrate it with a text-based browser. However, it has grown into a very handy and flexible...
    Downloads: 1 This Week
    Last Update:
    See Project
  • 6
    SpiderFoot

    SpiderFoot

    Open Source Intelligence Automation.

    SpiderFoot is an open source intelligence automation tool. Its goal is to automate the process of gathering intelligence about a given target, which may be an IP address, domain name, hostname or network subnet. SpiderFoot can be used offensively, i.e. as part of a black-box penetration test to gather information about the target or defensively to identify what information your organisation is freely providing for attackers to use against you.
    Leader badge
    Downloads: 138 This Week
    Last Update:
    See Project
  • 7

    CFPanel

    Web panel for keeping track of crowdfunding projects

    A web panel where to keep track of crowdfunding projects in which you have taken part.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 8
    MovieGrabber

    MovieGrabber

    MovieGrabber is a fully automated way of downloading

    MovieGrabber has now moved to GitHub! https://github.com/binhex/moviegrabber
    Downloads: 0 This Week
    Last Update:
    See Project
  • 9

    SE Auditor

    Free SEO audit software.

    SE Auditor is a program for analyzing web pages for search engines. SE Auditor is application that you can use to view statistical data about your website, in order to improve its position within the Web search results. SE Auditor is addressed to SEO professionals, website designers, developers, website testers and owners. SE Auditor enables you to check meta description, keywords, sitemap, the number of links and keyword consistency, the text/HTML ratio and many more ranking...
    Downloads: 0 This Week
    Last Update:
    See Project
  • Build Securely on AWS with Proven Frameworks Icon
    Build Securely on AWS with Proven Frameworks

    Lay a foundation for success with Tested Reference Architectures developed by Fortinet’s experts. Learn more in this white paper.

    Moving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
    Download Now
  • 10

    python-webrobot

    download video from various hosters

    Allows to automatically download files and video streams from web hosters.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 11
    Framework (scripts, configuration, code) to build free and public services around travel and leisure data. That project makes an extensive use of already existing data sources such as Geonames and dbPedia, and adds some glue around those (eg, links).
    Downloads: 0 This Week
    Last Update:
    See Project
  • 12
    PACS VM

    PACS VM

    ISO - Customized version of dcm4chee 2.17.3 for MySQL.

    1. Add JBoss Application Server 4.2.3.GA for JDK 6. 2. Cleanup for Windows and deprecated files. 3. Off CONSOLE records - http://forums.dcm4che.org/jiveforums/thread.jspa?messageID=4787
    Downloads: 0 This Week
    Last Update:
    See Project
  • 13

    Search Engine in python

    All student and developers are invited to join this web search engine

    This search engine project is created by a computer science student to challenge how search results are returned. Importance is placed on how web pages are ranked by a quick and efficient algorithm. All hands on deck is how we are calling this project. Algorithm design is really important and creative methods from better search results or web crawling is welcomed from all amateur programmers, students, web developers, software developers and computer science students from around the globe. We...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 14
    "Swish-e is a fast, flexible, and free open source system for indexing collections of Web pages or other files" (http://swish-e.org/ ) This module provides a Python API for this software.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 15
    MindRetrieve is a personal search engine. It helps you organize and retrieve web pages you have visited. MindRetrieve is a lightweight, cross-platform, open source application available under the BSD license. It works with all popular web browsers.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 16
    The Semantic Web Peer - Allegra is an innovative library for Semantic Web applications development. It provides a framework for asynchronous network peer programming, a simple stack of Internet standards implementations, and two new network applications:
    Downloads: 0 This Week
    Last Update:
    See Project
  • 17
    A tool for autonomous and virtual topical data integration using the focused web-harvesting method.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 18
    Document summarization system. By adding document content to system, user queries will generate a summary document containing the available information to the system.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 19
    Web spider and SERP scrapper
    Downloads: 0 This Week
    Last Update:
    See Project
  • 20
    A Web application to search for files on FTP servers. Users can query files by part of the file name, the entire file name, a regular expression, or a shell pattern. To store file indexes, PostgreSQL or MySQL is used.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 21
    BTG Web Search Engine is developed for students or developers who are interested in search engine technology. The engine web will be written in C++, Java, Perl,and Python. At the beginning of the project, a simple search engine design will be used.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 22
    Voxound Extension
    Voxound Extension is a daemon application used to provide additional local content access and management functionality to the voxound.com web application.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 23
    a small collection of python 3000 scripts/modules used to automate searching craigslist.org cities and categories for interesting stuff; these scripts currently use html screen scraping, since craigslist currently has no api
    Downloads: 0 This Week
    Last Update:
    See Project
  • 24
    vieW4 (vienna World Wide Web Workgroups) is a collaboration server based on Zope. This program allows a group of people to share documents, discuss them and search for previous released documents in major formats.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 25
    Starting from various input files, we parse to find a list of addresses, then display them on a map.
    Downloads: 0 This Week
    Last Update:
    See Project
  • Previous
  • You're on page 1
  • 2
  • 3
  • 4
  • Next