Search Results for "python web crawler" - Page 31
Sort By:
Showing 3175 open source projects for "python web crawler"
View related business solutions-
Gen AI apps are built with MongoDB AtlasMongoDB Atlas provides built-in vector search and a flexible document model so developers can build, scale, and run gen AI apps without stitching together multiple databases. From LLM integration to semantic search, Atlas simplifies your AI architecture—and it’s free to get started.
-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
1
The GUIDOLib provides a powerful engine for the graphic rendering of music scores, based on the Guido Music Notation format. It supports Linux, Mac OS X, Windows, Android and iOS operating systems. A Java JNI interface is available as well as a Javascript version of the library. A Web API has also been designed, allowing to deploy the engine as a Web service.
-
2
X-RAY
The next web scraper, see through the <html> noise
Supports strings, arrays, arrays of objects, and nested object structures. The schema is not tied to the structure of the page you're scraping, allowing you to pull the data in the structure of your choosing. The API is entirely composable, giving you great flexibility in how you scrape each page. Paginate through websites, scraping each page. X-ray also supports a request delay and a pagination limit. Scraped pages can be streamed to a file, so if there's an error on one page, you won't... -
3
weatherPi
An extensible weather sensor recording network, based on Raspberry Pis
A distributed network of sensors with a central data store, to record environmental data. Project goals include: * to serve as a teaching outlet for: ** sensor programming ** distributed data collecting ** web APIs ** database ** web UI for numerical data ** graphing data sensibly It also gives me some infotainment in new (to me) technologies, and as an excuse to scatter ever more Raspberry Pis around my home. -
4
FirePlus
Complete Edition
Universal ThinKing Machine -
No-Nonsense Code-to-Cloud Security for Devs | AikidoAikido provides a unified security platform for developers, combining 12 powerful scans like SAST, DAST, and CSPM. AI-driven AutoFix and AutoTriage streamline vulnerability management, while runtime protection blocks attacks.
-
5
Offensive Web Testing Framework
Offensive Web Testing Framework (OWTF), is a framework
OWASP OWTF is a project focused on penetration testing efficiency and alignment of security tests to security standards like the OWASP Testing Guide (v3 and v4), the OWASP Top 10, PTES and NIST so that pentesters will have more time to see the big picture and think out of the box. More efficiently find, verify and combine vulnerabilities. Have time to investigate complex vulnerabilities like business logic/architectural flaws or virtual hosting sessions. Perform more tactical/targeted... -
6
WebSploit Framework
WebSploit is a high level MITM Framework
WebSploit Advanced MITM Framework [+]Autopwn - Used From Metasploit For Scan and Exploit Target Service [+]wmap - Scan,Crawler Target Used From Metasploit wmap plugin [+]format infector - inject reverse & bind payload into file format [+]phpmyadmin Scanner [+]CloudFlare resolver [+]LFI Bypasser [+]Apache Users Scanner [+]Dir Bruter [+]admin finder [+]MLITM Attack - Man Left In The Middle, XSS Phishing Attacks [+]MITM - Man In The Middle Attack [+]Java Applet Attack [+]MFOD... -
7
rest-dev-vnc-docker
Restful / SOAP API Development with common tools in VNC/noVNC Docker
... accessible ubiquitously from Your favorite smartphones, tablets, e.g., iPad, SurfacePro, Amazon Fire tablet, Chrome PC, Desktop PC, etc. (Hmmm! in theory, if you can read tiny screens, you can even use your Apple iWatch to use KNIME, Eclipse Photon, IntelliJ, etc. as long as it can display HTML-5 Web Browsers. -
8
WebExKit
An HTML/CSS/JavaScript editor with preview window
The Web Experimentation Kit allows you to enter HTML, CSS and JavaScript and see the results immediately in a browser frame side-by-side with the editor. If you've seen the W3Schools Tryit Editor, JSFiddle or CodePen then this should be familiar to you. The difference between WebExKit and these other applications is that WebExKit is a stand-alone application that runs on your desktop and it allows you to save (and reload) files to your own disk drive. The editor shows a properly formed HTML... -
9
TestRest
TestRest is a fully QA Management Tool
TestRest is Test Management offers test case authoring, reusable test cases, test execution and reporting. TestRest supports statistic and graph reports with simple modern UI interfaces. -
Deliver secure remote access with OpenVPN.OpenVPN's products provide scalable, secure remote access — giving complete freedom to your employees to work outside the office while securely accessing SaaS, the internet, and company resources.
-
10
PlatformIO Storage
An open source ecosystem for IoT development
... embedded boards and IDE Library Manager - Hundreds Popular Libraries are organized into single Web 2.0 platform: list by categories, keywords, authors, compatible platforms and frameworks; learn via examples; be up-to-date with the latest version Atmel AVR & SAM, Espressif, Freescale Kinetis, Nordic nRF51, NXP LPC, Silicon Labs EFM32, ST STM32, TI MSP430 & Tiva, Teensy, Arduino, mbed, libOpenCM3, etc. -
11
Full Stack FastAPI Couchbase
Full stack, modern web application generator
Full stack, modern web application generator. Using FastAPI, Couchbase as a database, Docker, automatic HTTPS, and more. Couchbase has a great set of features that is not easily or commonly found in alternatives. REST backend tests based on Pytest, integrated with Docker, so you can test the full API interaction, independent on the database. As it runs in Docker, it can build a new data store from scratch each time (so you can use ElasticSearch, MongoDB, or whatever you want, and just test... -
12
yz-next-apyref
Arno-Can's Python based API Reference Generator.
Multi-Syntax-Version and Multi-Implementation API reference documentation generator. Supports local and remote sources. -
13
pyindi-client
Python binding to the libindi library
... there are also bindings for node.js, Tcl (incomplete) and PHP (not useful). As application examples you will find a Python Websocket server with which you may build a web application interacting with Indi servers, and a simple PyQt application similar to the Kstars Indi Control Panel (was built as an exercise). Finally there is an equatorial mount 3D simulator written with Freecad and Python, planned to be connected with the PyIndi module. *** The pyindi-client binding has moved to github. *** -
14
Walle 2.0
Walle 2.0 web
... application, review, grayscale release, real-time websocket. Walle allows users to release code and finally can not only choose jenkins! Support various web code releases, release and rollback of php, java, python, go and other codes can be completed with one click through the web. walle is a freely configurable project, more user-friendly, high-value, an open source online deployment system that supports git, multi-user, multi-language, multi-project, and multi-environment simultaneous deployment. -
15
NeuralCoref
Fast Coreference Resolution in spaCy with Neural Networks
NeuralCoref is a pipeline extension for spaCy 2.1+ which annotates and resolves coreference clusters using a neural network. NeuralCoref is production-ready, integrated in spaCy's NLP pipeline and extensible to new training datasets. For a brief introduction to coreference resolution and NeuralCoref, please refer to our blog post. NeuralCoref is written in Python/Cython and comes with a pre-trained statistical model for English only. NeuralCoref is accompanied by a visualization client... -
16
django-rest-auth
This app makes it extremely easy to build Django powered SPA's
This app makes it extremely easy to build Django powered SPA's (Single Page App) or Mobile apps exposing all registration and authentication-related functionality as CBV's (Class Base View) and REST (JSON). Tivix rebuilt a NATO software system to organize and coordinate rescue missions for submarines in distress across the globe. The United Nations Partner Portal (UNPP) is a web application built for a group of UN agencies to simplify their business processes and streamline collaboration... -
17
ShadowSocksShare
Python ShadowSocks framework
This project obtains the shared ss(r) account from the ss(r) shared website crawler, redistributes the account and generates a subscription link by parsing and verifying the account connectivity. Since Google plus will be closed on April 2, 2019, almost all the available accounts crawled before come from Google plus. So if you are building your own website, please keep an eye on the updates of this project and redeploy using the latest source code. -
18
bristoSOFT Contacts
bristoSOFT Contacts is group oriented contact management software.
bristoSOFT Contacts is a group oriented contact management software application written in Python, PyQt and PostgreSQL based on MVC software architecture. All contacts include email, telephone, address, title, name, company and also include notes, files, integrated google maps, activities such as phone calls, appointments, messaging, calendar. Contacts is a novel project with cutting edge technology. -
19
Raven Python
Raven is the legacy Python client for Sentry
Raven is the official legacy Python client for Sentry, officially supports Python 2.6–2.7 & 3.3–3.7, and runs on PyPy and Google App Engine. It tracks errors and exceptions that happen during the execution of your application and provides instant notification with detailed information needed to prioritize, identify, reproduce, and fix each issue. It provides full out-of-the-box support for many of the popular Python frameworks, including Django, and Flask. Raven also includes drop-in support... -
20
Python Taint
Static Analysis Tool for Detecting Security Vulnerabilities in Python
Static analysis of Python web applications based on theoretical foundations (Control flow graphs, fixed point, dataflow analysis) Detect command injection, SSRF, SQL injection, XSS, directory traveral etc. A lot of customization is possible. For functions from builtins or libraries, e.g. url_for or os.path.join, use the -m option to specify whether or not they return tainted values given tainted inputs, by default this file is used. -
21
lazynlp
Library to scrape and clean web pages to create massive datasets
LazyNLP is a lightweight tool for collecting and curating large-scale text datasets for machine learning and NLP applications with minimal manual effort. -
22
Requests-HTML
Pythonic HTML Parsing for Humans
This library intends to make parsing HTML (e.g. scraping the web) as simple and intuitive as possible. When using this library you automatically get full JavaScript support! (Using Chromium, thanks to puppeteer) CSS Selectors (a.k.a jQuery-style, thanks to PyQuery). XPath Selectors, for the faint of heart. Mocked user-agent (like a real web browser). Automatic following of redirects. Connection–pooling and cookie persistence. The Requests experience you know and love, with magical parsing... -
23
django-dynamic-scraper
Creating Scrapy scrapers via the Django admin interface
Django Dynamic Scraper (DDS) is an app for Django build on top of the scraping framework Scrapy. While preserving many of the features of Scrapy it lets you dynamically create and manage spiders via the Django admin interface. With Django Dynamic Scraper (DDS) you can define your Scrapy scrapers dynamically via the Django admin interface and save your scraped items in the database you defined for your Django project. Since it simplifies things DDS is not usable for all kinds of scrapers, but... -
24
Music + Video + Animation Maker & App
Why Music + Video + Animation Maker & App is Fun?
Javaimagephp All-in-one video/film editor or maker makes sense as simple to use app for video/film or movie editing and animations/slideshows. It gets most of your imagination of an All-in-one video/film Animation editor / maker App. Video editing is made intuitive and simple. Lots of Special Effects ( > 55) ! For example this one http://ogena.ridoua.nu Formats supported are mp4, avi, flv, mov, flv and more. Description given image animation which seeks audio given a description.... -
25
Trigger Happy
Automate the exchanges of the data between applications
Automate the exchanges of the data between the applications and services you use on the web. Make Twitter talk to Mastodon, make Github talk to Mattermost, store your favorite tweets by creating notes in Evernote, follow RSS feeds and post each news in Wallabag, Pocket or Evernote. The possibilities are too numerous to name all of them, but with that project you won't have to raise your little finger at all: automate everything and make your life easier. And last but not least, as this is your...
