Search Results for "python web crawler" - Page 9
Sort By:
Showing 2276 open source projects for "python web crawler"
View related business solutions-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
MongoDB Atlas | Run databases anywhereMongoDB Atlas allows you to build and run modern applications across 125+ cloud regions, spanning AWS, Azure, and Google Cloud. Its multi-cloud clusters enable seamless data distribution and automated failover between cloud providers, ensuring high availability and flexibility without added complexity.
-
1
Werkzeug
The comprehensive WSGI web application library
Werkzeug is a comprehensive WSGI web application library. It began as a simple collection of various utilities for WSGI applications and has become one of the most advanced WSGI utility libraries. Werkzeug doesn’t enforce any dependencies. It is up to the developer to choose a template engine, database adapter, and even how to handle requests. Includes an interactive debugger that allows inspecting stack traces and source code in the browser with an interactive interpreter for any frame... -
2
crawley
The unix-way web crawler
Crawls web pages and prints any link it can find. Fast HTML SAX-parser (powered by golang.org/x/net/html) Small (below 1500 SLOC), idiomatic, 100% test-covered codebase. Grabs most of useful resources URLs (pics, videos, audios, forms, etc...) Found URLs are streamed to stdout and guaranteed to be unique (with fragments omitted) Scan depth (limited by starting host and path, by default - 0) can be configured. Can crawl rules and sitemaps from robots.txt. Brute mode - scan HTML comments for URLs... -
3
CWhy
Explains and suggests fixes for compile-time errors for C, C++, C#, Go
Explains and suggests fixes for compiler error messages for a wide range of programming languages, including C, C++, C#, Go, Java, LaTeX, PHP, Python, Ruby, Rust, Swift, and TypeScript. CWhy needs to be connected to an OpenAI account or an Amazon Web Services account. Your account will need to have a positive balance for this to work (check your OpenAI balance). CWhy currently defaults to GPT-4, and falls back to GPT-3.5-turbo if a request error occurs. For the newest and best model (GPT-4... -
4
VectorVein
No-code AI workflow
Use the power of AI to build your personal knowledge base + automated workflow. No programming, just dragging to create a strong workflow and automate all tasks. Vector vein is affected LangChain as well as langflow The uncode AI workflow software developed by the inspiration aims to combine the powerful capabilities of large language models and allow users to realize the intelligibility and automation of various daily workflows through simple drag. After the software is opened normally,... -
Powering the best of the internet | FastlyEnsure your websites, applications and services can effortlessly handle the demands of your users with Fastly. Fastly’s portfolio is designed to be highly performant, personalized and secure while seamlessly scaling to support your growth.
-
5
Marvin
A batteries-included library for building AI-powered software
Meet Marvin: a batteries-included library for building AI-powered software. Marvin's job is to integrate AI directly into your codebase by making it look and feel like any other function. Marvin introduces a new concept called AI Functions. These functions differ from conventional ones in that they don’t rely on source code, but instead generate their outputs on-demand through AI. With AI functions, you don't have to write complex code for tasks like extracting entities from web pages, scoring... -
6
Archivematica
Free and open-source digital preservation system
Archivematica is a web- and standards-based, open-source application which allows your institution to preserve long-term access to trustworthy, authentic, and reliable digital content. Our target users are archivists, librarians, and anyone working to preserve digital objects. You are free to copy, modify, and distribute Archivematica with attribution under the terms of the AGPLv3 license. Archivematica is an open-source application based on recognized standards that makes it possible... -
7
iCn3D
Web-based protein structure viewer and analysis tool
"I see in 3D" (iCn3D) Structure Viewer is not only a web-based 3D viewer, but also a structure analysis tool interactively or in the batch mode using NodeJS scripts based on the npm package icn3d. iCn3D synchronizes the display of 3D structure, 2D interaction, and 1D sequences and annotations. Users' custom displays can be saved in a short URL or a PNG image. The complete package of iCn3D including Three.js and jQuery is in the directory "dist" after you get the source code with the "Code... -
8
ngx_waf
Handy, High performance, ModSecurity compatible Nginx firewall module
Handy, High-performance Nginx firewall module. Such as black and white list of IPs or IP range, uri black and white list, and request body black list, etc. Directives and rules are easy to write and readable. The IP detection is a constant-time operation. Most of the remaining inspections use caching to improve performance. Compatible with ModSecurity's rules, you can use OWASP ModSecurity Core Rule Set. Supports verifying Google, Bing, Baidu and Yandex crawlers and allowing them... -
9
quadplay
The quadplay fantasy console
quadplay✜ is a fantasy console by CasualEffects for creating and playing retro-style video games on any device. Create and play games on any laptop, desktop, tablet, or phone. Or, build your own programmable arcade machine from a Raspberry Pi, Tegra, or old computer. Create games on Windows, macOS, Linux. Play your games in any modern web browser on a laptop, desktop, tablet, phone, Raspberry Pi 4, or Jetson Nano. 60 fps @ 384 x 224 pixels = 12:7 aspect ≈ 16:9.3. 4096 sRGB (4:4:4) colors. Four... -
Test your software product anywhere in the worldGlobal App Testing is a managed pool of freelancers used by Google, Meta, Microsoft, and other world-beating software companies.
-
10
Nerves
Craft and deploy bulletproof embedded software in Elixir
Nerves is the open-source platform and infrastructure you need to build, deploy, and securely manage your fleet of IoT devices at speed and scale. Nerves is written in Elixir, but you don’t have to rewrite everything in Elixir to get the advantages of Nerves, simply bring your own code (like C, C++, Python, Rust, and more) and scale up. Nerves use the Erlang runtime system, known for being distributed, fault-tolerant, soft real-time, and highly available. Nerves has the tools you need to manage... -
11
PULSAR
Distributed pub-sub messaging system
... durability guarantees. Configurable replication between data centers across multiple geographic regions. Built from the ground up as a multi-tenant system. Supports isolation, authentication, authorization and quotas. Persistent message storage based on Apache BookKeeper. IO-level isolation between write and read operations. Flexible messaging models with high-level APIs for Java, Go, Python, C++, Node.js, WebSocket and C#. -
12
liquidpy
A port of liquid template engine for python
A port of liquid template engine for Python, on the shoulder of jinja2. -
13
uwsgi-nginx-flask
Docker image with uWSGI and Nginx for Flask applications in Python
Image with uWSGI and Nginx for Flask apps in Python running in a single container. Docker image with uWSGI and Nginx for Flask web applications in Python running in a single container. Optionally using Alpine Linux. This Docker image allows you to create Flask web applications in Python that run with uWSGI and Nginx in a single container. The combination of uWSGI with Nginx is a common way to deploy Python Flask web applications. It is widely used in the industry and would give you decent... -
14
Rio
WebApps in pure Python. No JavaScript, HTML and CSS needed
Rio is a Python framework designed to build web applications without the need for HTML, CSS, or JavaScript. Inspired by frameworks like Flutter and React, Rio offers a declarative interface and reusable components, enabling developers to create dynamic web apps entirely in Python. It streamlines the development process by managing both frontend and backend seamlessly. -
15
Sanic
Async Python 3.6+ web server/framework
Build fast, run fast with Sanic! Sanic is a Python 3.6+ web server and web framework designed to go fast. It provides a way to get a highly performant HTTP server up and running fast, while also making it easy to build, expand, and eventually scale. Sanic aspires to be as simple as possible while delivering the performance that you require. It allows the usage of the async/await syntax added in Python 3.5, so your code is guaranteed to be non-blocking and speedy. It's also ASGI compliant, so... -
16
MechanicalSoup
A Python library for automating interaction with websites
A Python library for automating interaction with websites. MechanicalSoup automatically stores and sends cookies, follows redirects, and can follow links and submit forms. It doesn't do JavaScript. MechanicalSoup was created by M Hickford, who was a fond user of the Mechanize library. Unfortunately, Mechanize was incompatible with Python 3 until 2019 and its development stalled for several years. MechanicalSoup provides a similar API, built on Python giants Requests (for HTTP sessions... -
17
WebArena
Code repo for "WebArena to build Autonomous Agents
WebArena is a realistic web environment designed for building and testing autonomous agents, providing a platform for developing web-based AI agents. -
18
pywebview
Build GUI for your Python program with JavaScript, HTML, and CSS
pywebview is a lightweight cross-platform wrapper around a webview component that allows to display HTML content in its own native GUI window. It gives you power of web technologies in your desktop application, hiding the fact that GUI is browser based. You can use pywebview either with a lightweight web framework like Flask or Bottle or on its own with a two way bridge between Python and DOM. pywebview uses native GUI for creating a web component window: WinForms on Windows, Cocoa on macOS... -
19
uvicorn-gunicorn-fastapi
Docker image with Uvicorn managed by Gunicorn
Docker image with Uvicorn managed by Gunicorn for high-performance FastAPI web applications in Python with performance auto-tuning. Optionally with Alpine Linux. -
20
Uplink
A Declarative HTTP Client for Python
A Declarative HTTP Client for Python. Inspired by Retrofit. Uplink is in beta development. The public API is still evolving, but we expect most changes to be backward compatible at this point. Uplink turns your HTTP API into a Python class. Build an instance to interact with the web service. Then, executing an HTTP request is as simply as invoking a method. Use decorators and type hints to describe each HTTP request. JSON, URL-encoded, and multipart request body and file upload. URL parameter... -
21
I hate money
A simple shared budget manager web application
I hate money is a web application made to ease shared budget management. It keeps track of who bought what, when, and for whom; and helps to settle the bills. I hate money is written in python, using the flask framework. It’s developed with ease of use in mind and is trying to keep things simple. Hope you (will) like it! The code is distributed under a BSD beerware derivative: if you meet the people in person and you want to pay them a craft beer, you are highly encouraged to do so. -
22
Transcrypt
Python 3.7 to JavaScript compiler
... prompt using stubs. Lightning-fast JavaScript 6 code: call caching, for-loop optimization, in-line JavaScript etc. Integrated static typechecking and minification at the tip of a command-line switch. Also runs on top of node.js. Extensive documentation with many code examples. Apache 2.0 license. Pip-install and go! Seamless integration with the universe of high-quality web-oriented JavaScript libraries, rather than the desktop-oriented Python ones. -
23
PyFCM
Python client for FCM - Firebase Cloud Messaging
Python client for FCM - Firebase Cloud Messaging (Android, iOS and Web) Firebase Cloud Messaging (FCM) is the new version of GCM. It inherits the reliable and scalable GCM infrastructure, plus new features. GCM users are strongly recommended to upgrade to FCM. Using FCM, you can notify a client app that new email or other data is available to sync. You can send notifications to drive user reengagement and retention. For use cases such as instant messaging, a message can transfer a payload of up... -
24
Dendrite
Tools to build web AI agents that can authenticate
Dendrite Python SDK is a toolkit for building web AI agents that can authenticate, interact with, and extract data from any website, facilitating web automation tasks. -
25
Parsera
Lightweight library for scraping web-sites with LLMs
Scrape data from any website with only a link and column descriptions. Parsera is a tool designed to scrape web content, specifically handling poorly structured or messy websites.
