Search Results for "web crawler source code" - Page 9
Sort By:
Showing 374 open source projects for "web crawler source code"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Forever Free Full-Stack Observability | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
1
Opyrator
Turns your machine learning code into microservices with web API
Instantly turn your Python functions into production-ready microservices. Deploy and access your services via HTTP API or interactive UI. Seamlessly export your services into portable, shareable, and executable files or Docker images. Opyrator builds on open standards - OpenAPI, JSON Schema, and Python type hints - and is powered by FastAPI, Streamlit, and Pydantic. It cuts out all the pain for productizing and sharing your Python code - or anything you can wrap into a single Python... -
2
lxspider
Educational Python web scraping case collection for many sites
lxSpider is a collection of web scraping examples designed primarily for learning and experimentation with data extraction techniques. It gathers numerous crawler implementations that demonstrate how to collect data from a wide range of websites and online services. It focuses heavily on practical cases that illustrate how different platforms handle requests, authentication parameters, and anti-scraping protections. lxSpider includes examples targeting areas such as e-commerce platforms,... -
3
Zebrunner Community Edition
Test Automation Management Tool
Zebrunner CE (Community Edition) is a Test Automation Management Tool for continuous testing and continuous deployment. It allows you to run various kinds of tests and gain successive levels of confidence in the code quality. Zebrunner CE is integrated by default with Carina open-source TestNG framework and uses Jenkins as a CI Tool. It is built on top of popular docker solutions and includes Postgres database, Zebrunner Reporting, Jenkins Master/Slaves Nodes, Selenium Hub, Mobile... -
4
speedtest-cli
Command line interface for testing internet bandwidth using speedtest
Command line interface for testing internet bandwidth using speedtest.net. It is not a goal of this application to be a reliable latency reporting tool. Latency reported by this tool should not be relied on as a value indicative of ICMP style latency. It is a relative value used for determining the lowest latency server for performing the actual speed test against. Speedtest CLI brings the trusted technology and global server network behind Speedtest to the command line. Measure internet... -
Earn up to 16% annual interest with Nexo.Put idle assets to work with competitive interest rates, borrow without selling, and trade with precision. All in one platform. Geographic restrictions, eligibility, and terms apply.
-
5
ruia
Async Python framework for fast and flexible web scraping spiders
Ruia is an asynchronous web scraping micro-framework built for Python that focuses on simplicity, speed, and flexibility when creating web crawlers. Ruia is powered by Python’s asyncio library along with aiohttp, enabling developers to perform concurrent network requests efficiently and scrape data from websites with minimal overhead. Ruia follows a “write less, run faster” philosophy, emphasizing concise code and streamlined spider development. It provides a structured approach to building... -
6
TRACARDI - Customer Data Platform
TRACARDI free open-source customer data platform
...TRACARDI is free open-source platform which you can extend the way you want it. TRACARDI is a low-code framework you can integrate with other parts of your ecosystem. Integrate it with other open source platforms. -
7
MToolBox
A bioinformatics pipeline to analyze mtDNA from NGS data
...MToolBox provides also a Variant Call Format file (version 4.0) featuring, for the first time, allele-specific heteroplasmy. Please, check out the most recent updates of the source code with the Github repository of MToolBox: https://github.com/mitoNGS/MToolBox or visit the Web version of MToolBox @ MSeqDR: https://mseqdr.org/mtoolbox.php -
8
QPyDesk
Code editor and real-time QR code generator for QPython
QPyDesk is a code editor, and real-time QR code generator for QPython. It is a Python code editor with syntax highlighting that also generates the QR code that represents said code in real time. This application also allows you to print the generated QR code to distribute the created application. However, because QR codes have a limited storage capacity, if the code is very long, QPyDesk creates a QR code that is only valid while the application is running, that is, the QR code generated... -
9
Free Queue Manager
Web based python-flask Queue management system
A web based management system developed for the purpose of easing the process of orgnizing queues and lines. Like many other (QMS)s Queue Management Systems, FQM does provide a basic dashboard to allow the users of the system and customers alike to interact with the system via a basic yet simple user interface . Brief user guide can be found on https://fqms.github.io/images/user_guide.pdf -
Enterprise-grade ITSM, for every businessFreshservice is an intuitive, AI-powered platform that helps IT, operations, and business teams deliver exceptional service without the usual complexity. Automate repetitive tasks, resolve issues faster, and provide seamless support across the organization. From managing incidents and assets to driving smarter decisions, Freshservice makes it easy to stay efficient and scale with confidence.
-
10
Python Handout
Turn Python scripts into handouts with Markdown and figures
Handout is a lightweight library for embedding rich, interactive components such as exercises, charts, and interactive diagrams directly into static documents like Markdown, Jupyter notebooks, or static HTML pages, enabling authors to create more engaging technical handouts, tutorials, and interactive essays. It’s particularly aimed at educators, presenters, and researchers who want to make their written material come alive with runnable demonstrations and interactive problem sets without... -
11
Python4Proteomics Course
Python course for Proteomics analysis
Python course (in Spanish) for Proteomics analysis using basically Jupyter NoteBooks. For more information, you can have a look at the readme.md file in the source code tree: https://sourceforge.net/p/lp-csic-uab/p4p/code/ci/default/tree/readme.md -
12
TCellXTalk
TCellXTalk Web-App from LP CSIC/UAB
TCellXTalk is a comprehensive database of experimentally detected phosphorylation, ubiquitination and acetylation sites in human T cells. The web-app at www.TCellXTalk.org makes TCellXTalk accessible from Internet, and enables the in silico prediction of potential co-modified peptides to facilitate their experimental detection, using targeted or directed mass spectrometry, for the study of protein post-translational modification cross-talk. More detailed information on TCellXTalk and... -
13
FuzzyWuzzy
Fuzzy string matching in Python
We’ve made it our mission to pull in event tickets from every corner of the internet, showing you them all on the same screen so you can compare them and get to your game/concert/show as quickly as possible. Of course, a big problem with most corners of the internet is labeling. One of our most consistently frustrating issues is trying to figure out whether two ticket listings are for the same real-life event (that is, without enlisting the help of our army of interns). To pick an example... -
14
Flask-GraphQL
Adds GraphQL support to your Flask application
Adds GraphQL support to your Flask application. This will add /graphql endpoint to your app and enable the GraphiQL IDE. If you are using the Schema type of Graphene library, be sure to use the graphql_schema attribute to pass as schema on the GraphQLView view. Otherwise, the GraphQLSchema from graphql-core is the way to go. The GraphQLSchema object that you want the view to execute when it gets a valid request. A value to pass as the context_value to graphql execute function. By default is... -
15
WebExKit
An HTML/CSS/JavaScript editor with preview window
The Web Experimentation Kit allows you to enter HTML, CSS and JavaScript and see the results immediately in a browser frame side-by-side with the editor. If you've seen the W3Schools Tryit Editor, JSFiddle or CodePen then this should be familiar to you. The difference between WebExKit and these other applications is that WebExKit is a stand-alone application that runs on your desktop and it allows you to save (and reload) files to your own disk drive. The editor shows a properly formed... -
16
BotSlayer
BotSlayer Community Edition
BotSlayer is an application that helps track and detect potential manipulation of information spreading on Twitter. The tool is developed by the Observatory on Social Media at Indiana University --- the same lab that brought to you Botometer and Hoaxy. BotSlayer is not a tool to detect and remove likely social bots from your list of Twitter followers or friends. For that purpose, check out Botometer. If you just want to visualize the spread of some piece of information, consider Hoaxy.... -
17
ECommerceCrawlers
Collection of Python ecommerce and website crawler examples projects
ECommerceCrawlers is a collection of practical Python web crawler projects designed to gather data from a variety of ecommerce platforms, websites, and online services. It aggregates many independent crawler examples created by contributors and organized into separate subprojects that target specific sites or data sources. These examples demonstrate how to build and operate web scrapers capable of collecting structured information such as product listings, news content, job postings, social... -
18
Photon
Incredibly fast crawler designed for OSINT
Photon is an extremely fast web crawler built specifically for OSINT and reconnaissance use cases. It is designed to extract URLs, endpoints, files, and other intelligence artifacts from target websites with minimal overhead. The crawler prioritizes speed and breadth, making it suitable for mapping web attack surfaces and discovering hidden resources. Photon is commonly used during early reconnaissance phases to build a comprehensive inventory of reachable assets. Its Python implementation... -
19
Pikon-Stelarium interface
Pikon-Stelarium interface to use with azimuthal mount
This is a full project to make an software-drived azimuthal mount for the Pikon telescope, it can be use for others telescope. This project include : . Some other parts to print to adapt to the Pikon . Python code for raspberry Needed parts for "as it" use : . 1 Raspberry (model A suit well) . 3 motors steppers . 3 steppers drivers . 1 BNO055 (the one from adafruit was used) . 1 Pikon driver . 1 GPS module Sofware requirement : . raspbian . python 3 . some python libraries -
20
ShadowSocksShare
Python ShadowSocks framework
This project obtains the shared ss(r) account from the ss(r) shared website crawler, redistributes the account and generates a subscription link by parsing and verifying the account connectivity. Since Google plus will be closed on April 2, 2019, almost all the available accounts crawled before come from Google plus. So if you are building your own website, please keep an eye on the updates of this project and redeploy using the latest source code. -
21
Walle 2.0
Walle 2.0 web
...Get through server management, grayscale release. Advanced gitlab-like user management mode. Online order application, review, grayscale release, real-time websocket. Walle allows users to release code and finally can not only choose jenkins! Support various web code releases, release and rollback of php, java, python, go and other codes can be completed with one click through the web. walle is a freely configurable project, more user-friendly, high-value, an open source online deployment system that supports git, multi-user, multi-language, multi-project, and multi-environment simultaneous deployment. -
22
django-rest-auth
This app makes it extremely easy to build Django powered SPA's
This app makes it extremely easy to build Django powered SPA's (Single Page App) or Mobile apps exposing all registration and authentication-related functionality as CBV's (Class Base View) and REST (JSON). Tivix rebuilt a NATO software system to organize and coordinate rescue missions for submarines in distress across the globe. The United Nations Partner Portal (UNPP) is a web application built for a group of UN agencies to simplify their business processes and streamline collaboration... -
23
Requests-HTML
Pythonic HTML Parsing for Humans
This library intends to make parsing HTML (e.g. scraping the web) as simple and intuitive as possible. When using this library you automatically get full JavaScript support! (Using Chromium, thanks to puppeteer) CSS Selectors (a.k.a jQuery-style, thanks to PyQuery). XPath Selectors, for the faint of heart. Mocked user-agent (like a real web browser). Automatic following of redirects. Connection–pooling and cookie persistence. The Requests experience you know and love, with magical parsing... -
24
ATX
Smart phone automation tool. Support iOS, Android, WebApp and game
ATX (AutomatorX) is an open source automated testing tool that supports testing native applications, games, and Web applications on iOS and Android platforms. Use Python to write test cases, and use image recognition and control positioning technology to complete the automation of the game. Attach a dedicated IDE to complete scripting quickly. The libraries for Android and iOS native application testing have been separated and can be used alone (it is strongly recommended to use it alone, firstly, it has less dependencies and high stability, and secondly, it can be automatically completed when writing code) -
25
GeoNotebook
A Jupyter notebook extension for geospatial visualization and analysis
GeoNotebook is an open-source extension to the Jupyter Notebook ecosystem that equips users with powerful geospatial visualization and analysis capabilities directly within the notebook interface. It integrates with GeoJS and other geospatial services to enable rich, interactive map rendering, layer control, and GIS data manipulation alongside traditional code and markdown cells in a Jupyter environment.
