Search Results for "web crawler source code"
Sort By:
Showing 152 open source projects for "web crawler source code"
View related business solutions-
Veeam Data Platform v13.1 - Get Your Free TrialTry Veeam Data Platform today. Experience the unified platform that's secure by design, portable by default, and proven to recover clean, fast, and anywhere.
-
Cut Data Warehouse Costs by 54%BigQuery delivers 54% lower TCO with exabyte scale and flexible pricing. Free migration tools handle the SQL translation automatically.
-
1
claude-code-transcripts
Tools for publishing transcripts for Claude Code sessions
claude-code-transcripts is a command-line utility that takes session files exported from Claude Code (in JSON or JSONL format) and turns them into clean, navigable HTML transcripts that can be viewed in any modern web browser. It is designed to make the often dense and verbose outputs from AI coding sessions easier to read, share, and archive by breaking conversations into paginated, annotated pages with navigable timelines of prompts and responses. Users can run this tool locally or fetch... -
2
The Falcon Web Framework
The no-nonsense REST API and microservices framework
Falcon is a minimalist WSGI library for building speedy web APIs and app backends. We like to think of Falcon as the Dieter Rams of web frameworks. When it comes to building HTTP APIs, other frameworks weigh you down with tons of dependencies and unnecessary abstractions. Falcon cuts to the chase with a clean design that embraces HTTP and the REST architectural style. Highly optimized, extensible code base. Easy access to headers and bodies through request and response objects. DRY request... -
3
Tencent Cloud Code Analysis
Static code analysis
Tencent Cloud Code Analysis (TCA for short, used internally by the R&D code CodeDog ) is a cloud-native, distributed, high-performance comprehensive code analysis and tracking platform that integrates many analysis tools, including server, web and client The three components have integrated a number of self-developed tools, and also support the dynamic integration of analysis tools of various programming languages in the industry. Obtain the Tencent Cloud code analysis platform by... -
4
learn_python3_spider
Python crawler tutorial series, learning Python crawlers from 0 to 1
...Data can be stored with CSV, MySQL, or MongoDB examples. More advanced material explores anti-scraping defenses, JavaScript analysis, WebSocket data, and distributed crawlers. The repository includes practical source-code examples for scraping several real-world websites and services. -
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
5
FastAPI
FastAPI framework, high performance, easy to learn, fast to code
FastAPI is a modern, fast (high-performance), web framework for building APIs with Python 3.6+ based on standard Python type hints. Great editor support. Completion everywhere. Less time debugging. Designed to be easy to use and learn. Less time reading docs. Minimize code duplication. Multiple features from each parameter declaration. Fewer bugs. Get production-ready code. With automatic interactive documentation. Based on (and fully compatible with) the open standards for APIs: OpenAPI... -
6
H2O Wave
Realtime Web Apps and Dashboards for Python and R
No HTML, CSS, Javascript skills are required. Build rich, interactive web apps using pure Python. Broadcast live information, visualizations, and graphics using Wave's low-latency real-time server. Instant control over every connected web browser using a simple and intuitive programming model. Preview your app live as you code. Dramatically reduce the time and effort to build web apps. Easily share your apps with end-users, get feedback, improve and iterate. ~10MB static executables for... -
7
Locust
Scalable open source load testing tool
Locust is an open source user load testing tool written in Python. The idea behind Locust is to swarm your web site or other systems with attacks from simulated users during a test, with each user behavior defined by you using Python code. This swarming process is then monitored from a web UI in real-time, and will help identify any bottlenecks in your code before real users can come in. -
8
Python-Spider
Python3 web crawler practice
Python-Spider is a repository intended to teach or provide examples for writing web spiders / crawlers in Python — part of a broader learning and resource collection by its author. The code and documentation are oriented toward beginners or intermediate learners who want to learn how to fetch, parse, and extract data from websites programmatically. As part of the author’s public learning-path repositories, python-spider likely includes examples of HTTP requests, HTML parsing, maybe... -
9
TorBot
Dark Web OSINT Tool
Contributions to this project are always welcome. To add a new feature fork the dev branch and give a pull request when your new feature is tested and complete. If its a new module, it should be put inside the modules directory. The branch name should be your new feature name in the format <Feature_featurename_version(optional)>. On Linux platforms, you can make an executable for TorBot by using the install.sh script. You will need to give the script the correct permissions using chmod +x... -
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
10
Frappe
Low code web framework for real world applications
Frappe is a full-stack, low-code web framework written in Python and JavaScript, used to build scalable and modular enterprise applications. It powers ERPNext and includes tools for REST APIs, user management, document modeling, workflows, and real-time updates. Frappe uses a "model-view-controller" approach with its own ORM and frontend system, enabling rapid development without sacrificing control or performance. -
11
NiceGUI
Create web-based user interfaces with Python
NiceGUI is a Python-based UI framework that enables developers to create interactive web applications using only Python code. It abstracts away the complexities of HTML, CSS, and JavaScript, allowing for rapid development of web interfaces directly from Python scripts. NiceGUI is suitable for building dashboards, control panels, and other web-based tools, especially in contexts like robotics and data visualization. -
12
GPT All Star
AI-powered code generation tool for scratch development of web apps
AI-powered code generation tool for scratch development of web applications with a team collaboration of autonomous AI agents. This is a research project, and its primary value is to explore the possibility of autonomous AI agents. -
13
BlackSheep
Fast ASGI web framework for Python
BlackSheep is an asynchronous web framework to build event-based web applications with Python. A rich code API, based on dependency injection and inspired by Flask and ASP.NET Core. A typing-friendly codebase, which enables a comfortable development experience thanks to hints when coding with IDEs. Built-in generation of OpenAPI Documentation, supporting version 3, YAML, and JSON. A cross-platform framework, using the most modern versions of Python. BlackSheep supports automatic binding of... -
14
Dash
Build beautiful web-based analytic apps, no JavaScript required
Dash is a Python framework for building beautiful analytical web applications without any JavaScript. Built on top of Plotly.js, React and Flask, Dash easily achieves what an entire team of designers and engineers normally would. It ties modern UI controls and displays such as dropdown menus, sliders and graphs directly to your analytical Python code, and creates exceptional, interactive analytics apps. Dash apps are very lightweight, requiring only a limited number of lines of Python or... -
15
FastAPI Python
FastAPI framework, high performance, easy to learn, fast to code
FastAPI framework, high performance, easy to learn, fast to code, ready for production. FastAPI is a modern, fast (high-performance), web framework for building APIs with Python based on standard Python type hints. -
16
Full Stack FastAPI and PostgreSQL
Full stack, modern web application generator
Generate a backend and frontend stack using Python, including interactive API documentation. Production ready Python web server using Uvicorn and Gunicorn. Very high performance, on par with NodeJS and Go (thanks to Starlette and Pydantic). Great editor support. Completion everywhere. Less time debugging. Designed to be easy to use and learn. Less time reading docs. Minimize code duplication. Multiple features from each parameter declaration. Get production-ready code. With automatic... -
17
voila
Voilà turns Jupyter notebooks into standalone web applications
From notebooks to standalone web applications and dashboards. Voilà allows you to convert a Jupyter Notebook into an interactive dashboard that allows you to share your work with others. It is secure and customizable, giving you control over what your readers experience. Unlike the usual HTML-converted notebooks, each user connecting to the Voilà tornado application gets a dedicated Jupyter kernel which can execute the callbacks to changes in Jupyter interactive widgets. To render the bqplot... -
18
Transcrypt
Python 3.7 to JavaScript compiler
Python in the browser, precompiled for speed. Transcrypt started out as a personal repo, owned by Jacques de Hooge. As the project caught on and the number of people contributing issues, ideas and code grew, the repo was transferred to the QQuick organization, to be able to form a developer team on GitHub. There's a simple parallel between the Python and the JavaScript code. In combination with the use of source maps, this enables efficient debugging. Also, code can be tested from the command prompt using stubs. Lightning-fast JavaScript 6 code: call caching, for-loop optimization, in-line JavaScript etc. ... -
19
I hate money
A simple shared budget manager web application
I hate money is a web application made to ease shared budget management. It keeps track of who bought what, when, and for whom; and helps to settle the bills. I hate money is written in python, using the flask framework. It’s developed with ease of use in mind and is trying to keep things simple. Hope you (will) like it! The code is distributed under a BSD beerware derivative: if you meet the people in person and you want to pay them a craft beer, you are highly encouraged to do so. -
20
FlaskBB
A classic Forum Software in Python using Flask
A classic Forum Software in Python with a Modern and Fresh look. FlaskBB is open source and the whole source code is available on GitHub. It is being developed under the BSD 3-Clause license. Due to the plugin system, it is possible to extend FlaskBB without touching a single line of code. You just drop the plugin into the plugin's directory and activate it in the management panel. Are you sick of the standard theme or you simply want to create a customized style for your site? ... -
21
Optuna
A hyperparameter optimization framework
Optuna is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative, define-by-run style user API. Thanks to our define-by-run API, the code written with Optuna enjoys high modularity, and the user of Optuna can dynamically construct the search spaces for the hyperparameters. Optuna Dashboard is a real-time web dashboard for Optuna. You can check the optimization history, hyperparameter importances, etc. in graphs and... -
22
Streamlit
The fastest way to build data apps in Python
A faster way to build and share data apps. Streamlit turns data scripts into shareable web apps in minutes. All in pure Python. No front‑end experience is required. Build an app in a few lines of code with our magically simple API. Then see it automatically update as you iteratively save the source file. Adding a widget is the same as declaring a variable. No need to write a backend, define routes, handle HTTP requests, connect a frontend, write HTML, CSS, JavaScript, etc. ... -
23
UI UX Pro Max
AI SKILL that provide design intelligence
UI UX Pro Max is an open-source AI skill designed to provide intelligent design assistance for professional user interfaces and user experiences across web, mobile, and cross-platform frameworks. It uses an AI reasoning engine to generate complete design systems tailored to project requirements, recommending layouts, typography, colors, spacing, and component structures automatically based on natural language prompts. -
24
Sanic
Async Python 3.6+ web server/framework
Build fast, run fast with Sanic! Sanic is a Python 3.6+ web server and web framework designed to go fast. It provides a way to get a highly performant HTTP server up and running fast, while also making it easy to build, expand, and eventually scale. Sanic aspires to be as simple as possible while delivering the performance that you require. It allows the usage of the async/await syntax added in Python 3.5, so your code is guaranteed to be non-blocking and speedy. It's also ASGI compliant,... -
25
Python for OSINT
In this repository you will find sample code files for each day
Python for OSINT: 21-Day Course is a beginner-oriented training repository focused on automating repetitive open-source intelligence tasks with Python. It accompanies a free course PDF and provides sample code for lessons organized across three weeks. Topics progress from basic syntax and command-line tools to HTTP requests, APIs, JSON, CSV, databases, scraping, regular expressions, proxies, and file handling. Later lessons cover domain research, document generation, charts, maps, web archives, dates, and simple web applications. ...
