Showing 65 open source projects for "python web crawler"

View related business solutions
  • Our Free Plans just got better! | Auth0 Icon
    Our Free Plans just got better! | Auth0

    With up to 25k MAUs and unlimited Okta connections, our Free Plan lets you focus on what you do best—building great apps.

    You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
    Try free now
  • Simple, Secure Domain Registration Icon
    Simple, Secure Domain Registration

    Get your domain at wholesale price. Cloudflare offers simple, secure registration with no markups, plus free DNS, CDN, and SSL integration.

    Register or renew your domain and pay only what we pay. No markups, hidden fees, or surprise add-ons. Choose from over 400 TLDs (.com, .ai, .dev). Every domain is integrated with Cloudflare's industry-leading DNS, CDN, and free SSL to make your site faster and more secure. Simple, secure, at-cost domain registration.
    Sign up for free
  • 1
    Spatie Crawler

    Spatie Crawler

    An easy to use, powerful crawler implemented in PHP

    Spatie Crawler is a PHP library that allows developers to crawl websites and extract information efficiently. It can be used for web scraping, link checking, or automated testing of web pages. The library is simple to use and supports customizable crawling strategies, including controlling crawl depth and handling redirects. It’s suitable for building crawlers that navigate large or dynamically generated websites.
    Downloads: 1 This Week
    Last Update:
    See Project
  • 2
    Requests for PHP

    Requests for PHP

    Requests for PHP is a humble HTTP request library

    Requests is a HTTP library written in PHP, for human beings. It is roughly based on the API from the excellent Requests Python library. Requests is ISC Licensed (similar to the new BSD license) and has no dependencies, except for PHP 5.6+. Despite PHP’s use as a language for the web, its tools for sending HTTP requests are severely lacking. cURL has an interesting API, to say the least, and you can’t always rely on it being available. Sockets provide only low-level access and require you to build most of the HTTP response parsing yourself. ...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 3
    s3cmd

    s3cmd

    Command line tool for managing Amazon S3 and CloudFront services

    Open-source tool to access Amazon S3 file storage. S3cmd is a free command line tool and client for uploading, retrieving and managing data in Amazon S3 and other cloud storage service providers that use the S3 protocol, such as Google Cloud Storage. Lots of features and options have been added to s3cmd since its very first release in 2008.... we recently counted more than 60 command line options, including multipart uploads, encryption, incremental backup, s3 sync, ACL and Metadata...
    Leader badge
    Downloads: 974 This Week
    Last Update:
    See Project
  • 4
    LAPP package in Complete Virtual Machine

    LAPP package in Complete Virtual Machine

    Quick Apache/PHP/Postgresql Pkg for development, testing & production

    This VM is created for 2 reasons: 1. Very little initial setup work required to Develop / Test / Deploy a Dynamic Web Application live. 2. This system should keep running for Years, without requiring Updates / Breakages. If you are new to Virtual Machines, then please watch the Video below ( taken from my other PHP project called teamdocs. Just replace td with lapp wherever mentioned ) Upload your PHP Files using the free FileZilla ftp client software, to this server called lapp.local...
    Downloads: 0 This Week
    Last Update:
    See Project
  • Gen AI apps are built with MongoDB Atlas Icon
    Gen AI apps are built with MongoDB Atlas

    The database for AI-powered applications.

    MongoDB Atlas is the developer-friendly database used to build, scale, and run gen AI and LLM-powered apps—without needing a separate vector database. Atlas offers built-in vector search, global availability across 115+ regions, and flexible document modeling. Start building AI apps faster, all in one place.
    Start Free
  • 5
    LAMP package in Complete Virtual Machine

    LAMP package in Complete Virtual Machine

    A Quick LAMP/WAMP/MAMP/XAMPP Pkg for development, testing & production

    This VM is created for 2 reasons: 1. Very little initial setup work required to Develop / Test / Deploy a Dynamic Web Application live, within minutes. 2. This system should keep running for Years, without requiring Updates / Breakages. If you are new to Virtual Machines, then please watch the Video below ( taken from my other sample PHP project called teamdocs. You may cleanup this Application from the htdocs home folder link & its corresponding database, after logging into mysql. The...
    Downloads: 13 This Week
    Last Update:
    See Project
  • 6
    Goutte

    Goutte

    Goutte, a simple PHP Web Scraper

    ...The method returns a Crawler object (Symfony\Component\DomCrawler\Crawler). To use your own HTTP settings, you may create and pass an HttpClient instance to Goutte. For example, to add a 60 second request timeout. Read the documentation of the BrowserKit, DomCrawler, and HttpClient Symfony Components for more information about what you can do with Goutte. Goutte is a thin wrapper around the following Symfony Components: BrowserKit, CssSelector, DomCrawler, and HttpClient.
    Downloads: 1 This Week
    Last Update:
    See Project
  • 7
    OMLX project is a place for processing of many projects to be ready to become open source projects.
    Leader badge
    Downloads: 124 This Week
    Last Update:
    See Project
  • 8
    Ansible Examples

    Ansible Examples

    A few starter examples of ansible playbooks, to show features

    This repository collects practical, real-world examples of using Ansible to automate infrastructure, deployments, and configurations. Each directory demonstrates a specific use case—ranging from setting up web servers, load balancers, and databases to orchestrating multi-tier applications in cloud environments. The examples highlight common Ansible practices such as organizing inventories, writing reusable playbooks, using roles, and handling variables and templates. They’re designed to be...
    Downloads: 1 This Week
    Last Update:
    See Project
  • 9

    coldev

    C/C++ , PHP, PYTHON, 3D ENGINES, CHROME NATIVE CLIENT

    + C/C++ and PHP projects, Python and Others + 3D ENGINES + Videogames Projects based in Bennugd http://www.bennugd.org Web Page https://coldev.sourceforge.io/
    Leader badge
    Downloads: 13 This Week
    Last Update:
    See Project
  • Keep company data safe with Chrome Enterprise Icon
    Keep company data safe with Chrome Enterprise

    Protect your business with AI policies and data loss prevention in the browser

    Make AI work your way with Chrome Enterprise. Block unapproved sites and set custom data controls that align with your company's policies.
    Download Chrome
  • 10
    NTK RTMP SERVER

    NTK RTMP SERVER

    Naam Tamilar Web TV Live Streamer

    Naam Tamilar RTMP Server This project updated as open source for future use of Naam Tamilar Political Party. To contribute to the party and in some case if there is any possibilities if i cannot support them for long term. I thought of sharing this source code so in future it may be helpful for the community and party in which other software developers can help them to upgrade. This source is forked from - https://github.com/arut/nginx-rtmp-module and modified with multiple broadcast...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 11

    Winpweb

    Free Windows Portable Web Development Stack

    Winpweb is a Free Windows Portable Web Development Stack. It contains Nginx, MariaDB and PHP (with composer). There're also many plugins available which feel free for users to install theadditional software based on their needs. The available plugins are MongoDB, Nodejs (with npm), Ruby (with gem), Devkit, Git, Perl/Strawberryperl (with ppm), Python/Winpython (with pip) The package is very lightweight, only (36 MB), simple and easy to use by using CLI tools called 'pweb' for servers operation like (start, stop, config, etc.) ...
    Downloads: 1 This Week
    Last Update:
    See Project
  • 12

    z9

    a simple client-server ERP system

    z9 is a simple client-server ERP framework, based on a Python/PyQt modular smartclient and a PHP+SQL RESTful web service, created for the "opificio zeronove" bike shop application.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 13
    Belay Device

    Belay Device

    Automated Provisioning and Testing Framework

    Belay Device has moved! https://github.com/BelayTechnologies/Belay-Device
    Downloads: 0 This Week
    Last Update:
    See Project
  • 14
    AMPPS

    AMPPS

    AMPPS is a LAMP/MAMP/WAMP stack

    AMPPS is a stack of Apache, MySQL, PHP, Perl & Python. AMPPS enables you to focus more on using applications rather than maintaining them. With AMPPS you can create a website by installing any of the 330+ Apps, customizing it, and then simply publishing it on the internet via a wide choice of hosting service providers. A complete package on your desktop, same like the server that provides many open source web applications.
    Downloads: 16 This Week
    Last Update:
    See Project
  • 15
    The goal of this project is to make possible to access Progress database from any external program that can use sockets. The server (broker and agents) are written in Progress 4GL and made use of sockets capabilities of Progress V9.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 16
    Zoozle Search & Download Suchmaschine

    Zoozle Search & Download Suchmaschine

    Zoozle 2008 - 2010 Webpage, Tools and SQL Files

    Download search engine and directory with Rapidshare and Torrent - zoozle Download Suchmaschine All The files that run the World Leading German Download Search Engine in 2010 with 500 000 unique visitors a day - all the tools you need to set up a clone. Code Contains: - PHP Files for zoozle - Perl Crawler for gathering new content to database and all other cool tools i have...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 17

    CLUE - C Learning Undergrad Environment

    Tools to support the learning of the C programming language

    CLUE (C Learning Undergraduate Environment) is a software allowing students to work on assignments in the C language while benefiting from support for peer testing, hyper-linked tutorials to help them understand compiler error messages, detection of "novice errors" with warnings hyper-linked to tutorials. Sponsored by National Science Foundation under award CCLI #0836863.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 18
    Fast Artificial Neural Network Library is a free open source neural network library, which implements multilayer artificial neural networks in C with support for both fully connected and sparsely connected networks. Cross-platform execution in both fixed and floating point are supported. It includes a framework for easy handling of training data sets. It is easy to use, versatile, well documented, and fast. Bindings to more than 15 programming languages are available. An easy to read...
    Downloads: 34 This Week
    Last Update:
    See Project
  • 19

    JSolutions

    The solution to all your problems

    The Jsolution package is a collection of programs, scripts,libraries and documents that i have written and use in my daily life.
    Downloads: 1 This Week
    Last Update:
    See Project
  • 20
    SuperWikia Alpha Codebase 1.1

    SuperWikia Alpha Codebase 1.1

    SuperWikia: The Runtime/Realtime Environment for Codesmiths

    The SuperWikia Alpha 1.0 application is an IDE specifically developed for 3D and graphic content. Its built upon the Unity foundation and incorporates various libraries from Komodo IDE, wxDevC++, CodeBlocks, QT Creator and other applications. Its Realtime purpose is an IDE (codesmith developer access), its Runtime (U/I) function is as a space building simulator, operating in conjunction with the SuperWikia 'Navigator' user interface to simulate space economics and fleet operations. It...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 21
    Abandoned version of qbc. DOES NOT WORK properly
    Downloads: 0 This Week
    Last Update:
    See Project
  • 22
    SUIT (Scripting Using Integrated Templates) is a template framework that allows you to define your own syntax through user-defined rules. There are PHP and Python versions. This page is also the home of subprojects, Such as TIE, a template manager.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 23

    Chi's Project

    The projects of ChiNvo Studio

    The projects of ChiNvo Studio
    Downloads: 0 This Week
    Last Update:
    See Project
  • 24
    A collection of open source libraries and tools that provide solutions for common problems in processing Arabic text, especially in web applications. text normalization, phrase segmentation, text indexing, stop word lists, common spelling mistakes.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 25
    Teng is a general purpose templating engine written in C++ (i.e. library). It is also available as Python module or PHP extension. The main idea of teng is to strictly separate application logic from presentation layer. Widely used on dynamic web sites.
    Downloads: 0 This Week
    Last Update:
    See Project
  • Previous
  • You're on page 1
  • 2
  • 3
  • Next