Search Results for "processing"
Sort By:
Showing 34 open source projects for "processing"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Veeam Data Platform v13.1 - Get Your Free TrialTry Veeam Data Platform today. Experience the unified platform that's secure by design, portable by default, and proven to recover clean, fast, and anywhere.
-
1
SageMaker Spark Container
Docker image used to run data processing workloads
Apache Spark™ is a unified analytics engine for large-scale data processing. It provides high-level APIs in Scala, Java, Python, and R, and an optimized engine that supports general computation graphs for data analysis. It also supports a rich set of higher-level tools including Spark SQL for SQL and DataFrames, MLlib for machine learning, GraphX for graph processing, and Structured Streaming for stream processing. -
2
Celery
Distributed task queue (development branch)
Celery is a simple, flexible, and reliable distributed system to process vast amounts of messages, while providing operations with the tools required to maintain such a system. It’s a task queue with focus on real-time processing, while also supporting task scheduling. Celery has a large and diverse community of users and contributors, you should come join us on IRC or our mailing-list. Celery is Open Source and licensed under the BSD License. A task queue’s input is a unit of work called a task. Dedicated worker processes constantly monitor task queues for new work to perform. ... -
3
NNCF
Neural Network Compression Framework for enhanced OpenVINO
NNCF (Neural Network Compression Framework) is an optimization toolkit for deep learning models, designed to apply quantization, pruning, and other techniques to improve inference efficiency. -
4
BentoML
Unified Model Serving Framework
BentoML simplifies ML model deployment and serves your models at a production scale. Support multiple ML frameworks natively: Tensorflow, PyTorch, XGBoost, Scikit-Learn and many more! Define custom serving pipeline with pre-processing, post-processing and ensemble models. Standard .bento format for packaging code, models and dependencies for easy versioning and deployment. Integrate with any training pipeline or ML experimentation platform. Parallelize compute-intense model inference workloads to scale separately from the serving logic. Adaptive batching dynamically groups inference requests for optimal performance. ... -
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
5
Smallpond
A lightweight data processing framework built on DuckDB and 3FS
smallpond is a lightweight distributed data processing framework built by DeepSeek, designed to scale DuckDB workloads over clusters using their 3FS (Fire-Flyer File System) backend. The idea is to preserve DuckDB’s fast analytics engine but lift it from single-node to multi-node settings, giving you the ability to operate on large datasets (e.g. petabyte scale) without moving to a heavyweight system like Spark. -
6
AIOHTTP
Asynchronous HTTP client/server framework for asyncio and Python
...A long awaited new feature is tracing client request life cycle to figure out when and why client request spends a time waiting for connection establishment, getting server response headers etc. Now it is possible by registering special signal handlers on every request processing stage. The main change is dropping yield from support and using async/await everywhere. Farewell, Python 3.4. You often want to send some sort of data in the URL’s query string. If you were constructing the URL by hand, this data would be given as key/value pairs in the URL after a question mark, e.g. httpbin.org/get?key=val. Requests allows you to provide these arguments as a dict, using the params keyword argument. aiohttp internally performs URL canonicalization before sending request. -
7
Ray
A unified framework for scalable computing
Modern workloads like deep learning and hyperparameter tuning are compute-intensive and require distributed or parallel execution. Ray makes it effortless to parallelize single machine code — go from a single CPU to multi-core, multi-GPU or multi-node with minimal code changes. Accelerate your PyTorch and Tensorflow workload with a more resource-efficient and flexible distributed execution framework powered by Ray. Accelerate your hyperparameter search workloads with Ray Tune. Find the best... -
8
ProtoMotions
ProtoMotions is a GPU-accelerated simulation and learning framework
...Policies can be tested across Isaac Gym, Isaac Lab, Newton, Genesis, MuJoCo, and high-fidelity Isaac Sim environments. Its deployment pipeline exports ONNX policies with observation processing included, simplifying transfers from simulation to real Unitree G1 hardware. The framework also supports procedural scene generation and motion authoring from Kimodo text-to-motion outputs. -
9
Trame
Weave various components and technologies into a Web App
...It enables the integration of various components and technologies, such as VTK and ParaView, into web applications written entirely in Python. With best-in-class platforms at its core, trame provides complete control of 3D visualizations and data processing. Developers benefit from a write-once environment from trame. trame is an open source project licensed under Apache License Version 2.0 which allows users to create open source or commercial applications without any licensing worries. By relying simply on Python and HTML, trame focuses on one's data and associated analysis and visualizations while hiding the complications of web development. -
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
10
Grab Framework Project
Web Scraping Framework
Grab is a python framework for building web scrapers. With Grab you can build web scrapers of various complexity, from simple 5-line scripts to complex asynchronous website crawlers processing millions of web pages. Grab provides an API for performing network requests and for handling the received content e.g. interacting with DOM tree of the HTML document. The single request/response API that allows you to build network request, perform it and work with the received content. The API is built on top of urllib3 and lxml libraries. ... -
11
django-rest-framework-gis
Geographic add-ons for Django REST Framework
...This field handles GeoDjango geometry fields, providing custom to_native and from_native methods for GeoJSON input/output. While precision and remove_duplicates are designed to reduce the byte size of the API response, they will also increase the processing time required to render the response. This will likely be negligible for small GeoJSON responses but may become an issue for large responses. The primary key of the model (usually the "id" attribute) is automatically used as the id field of each GeoJSON Feature Object. The GeoJSON specification allows a feature to contain a boundingbox of a feature. ... -
12
The Falcon Web Framework
The no-nonsense REST API and microservices framework
...Falcon cuts to the chase with a clean design that embraces HTTP and the REST architectural style. Highly optimized, extensible code base. Easy access to headers and bodies through request and response objects. DRY request processing via middleware components and hooks. Strict adherence to RFCs. Idiomatic HTTP error responses. Straightforward exception handling. Snappy testing with WSGI/ASGI helpers and mocks. CPython 3.5+ and PyPy 3.5+ support. No reliance on magic globals for routing and state management. Stable interfaces with an emphasis on backward compatibility. ... -
13
Zappa - Serverless Python
Serverless Python
...That means infinite scaling, zero downtime, zero maintenance - and at a fraction of the cost of your current deployments! With a traditional HTTP server, the server is online 24/7, processing requests one by one as they come in. If the queue of incoming requests grows too large, some requests will time out. With Zappa, each request is given its own virtual HTTP "server" by Amazon API Gateway. AWS handles the horizontal scaling automatically, so no requests ever time out. Each request then calls your application from a memory cache in AWS Lambda and returns the response via Python's WSGI interface. ... -
14
Pytente
Uma Ferramenta Computacional para Análise e Recuperação de Patentes
O Pytente é uma solução avançada para automatizar o processo de coleta, armazenamento e tratamento de dados bibliográficos de patentes. A ferramenta foi projetada para simplificar a coleta de grandes volumes de dados em repositórios de acesso aberto. O Pytente garante o armazenamento estruturado das informações, além da validação e eliminação de registros duplicados. Dentre as diversas funcionalidades disponibilizadas pela ferramenta, destacam-se a extração personalizada de subconjuntos de... -
15
TorchQuantum
A PyTorch-based framework for Quantum Classical Simulation
...Researchers on quantum algorithm design, parameterized quantum circuit training, quantum optimal control, quantum machine learning, and quantum neural networks. Dynamic computation graph, automatic gradient computation, fast GPU support, batch model terrorized processing. -
16
MMDeploy
OpenMMLab Model Deployment Framework
...It is a part of the OpenMMLab project. Models can be exported and run in several backends, and more will be compatible. All kinds of modules in the SDK can be extended, such as Transform for image processing, Net for Neural Network inference, Module for postprocessing and so on. Install and build your target backend. ONNX Runtime is a cross-platform inference and training accelerator compatible with many popular ML/DNN frameworks. Please read getting_started for the basic usage of MMDeploy. -
17
towhee
Framework that is dedicated to making neural data processing
...Towhee provides out-of-the-box integration with your favorite libraries, tools, and frameworks, making development quick and easy. Towhee includes a pythonic method-chaining API for describing custom data processing pipelines. We also support schemas, making processing unstructured data as easy as handling tabular data. -
18
Horovod
Distributed training framework for TensorFlow, Keras, PyTorch, etc.
...Horovod can be installed on-premise or run out-of-the-box in cloud platforms, including AWS, Azure, and Databricks. Horovod can additionally run on top of Apache Spark, making it possible to unify data processing and model training into a single pipeline. Once Horovod has been configured, the same infrastructure can be used to train models with any framework, making it easy to switch between TensorFlow, PyTorch, MXNet, and future frameworks as machine learning tech stacks continue to evolve. Start scaling your model training with just a few lines of Python code. ... -
19
The Related Values Processing Framework helps the integration of Process Control Data Historian Systems.
-
20
Wooey
A Django app that creates automatic web UIs for Python scripts
Wooey is a simple web interface to run command line Python scripts. Think of it as an easy way to get your scripts up on the web for routine data analysis, file processing, or anything else. The project was inspired by how simply and powerfully sandman could expose users to a database and by how Gooey turns ArgumentParser-based command-line scripts into WxWidgets GUIs. Originally two separate projects (Django-based djangui by Chris Mitchell and Flask-based Wooey by Martin Fitzpatrick) it has been merged to combine our efforts. ... -
21
OpenPrompt
An Open-Source Framework for Prompt-Learning
Prompt-learning is the latest paradigm to adapt pre-trained language models (PLMs) to downstream NLP tasks, which modifies the input text with a textual template and directly uses PLMs to conduct pre-trained tasks. OpenPrompt is a library built upon PyTorch and provides a standard, flexible and extensible framework to deploy the prompt-learning pipeline. OpenPrompt supports loading PLMs directly from huggingface transformers. In the future, we will also support PLMs implemented by other... -
22
PyText
A natural language modeling framework based on PyTorch
PyText is a deep-learning based NLP modeling framework built on PyTorch. PyText addresses the often-conflicting requirements of enabling rapid experimentation and of serving models at scale. It achieves this by providing simple and extensible interfaces and abstractions for model components, and by using PyTorch’s capabilities of exporting models for inference via the optimized Caffe2 execution engine. We use PyText at Facebook to iterate quickly on new modeling ideas and then seamlessly... -
23
Zappa
Serverless Python
...That means infinite scaling, zero downtime, zero maintenance - and at a fraction of the cost of your current deployments! With a traditional HTTP server, the server is online 24/7, processing requests one by one as they come in. If the queue of incoming requests grows too large, some requests will time out. With Zappa, each request is given its own virtual HTTP "server" by Amazon API Gateway. AWS handles the horizontal scaling automatically, so no requests ever time out. Each request then calls your application from a memory cache in AWS Lambda and returns the response via Python's WSGI interface. ... -
24
Twint
An advanced Twitter scraping & OSINT tool written in Python
Twint is an advanced open-source Twitter scraping and OSINT tool written in Python that extracts tweets, user data, followers, likes, and more—without relying on Twitter’s API—making it highly useful for researchers, analysts, and hobbyists who want to bypass rate limits and access public Twitter data. -
25
Django Celery
Old Celery integration project for Django
Celery is a simple, flexible, and reliable distributed system to process vast amounts of messages, while providing operations with the tools required to maintain such a system. It’s a task queue with focus on real-time processing, while also supporting task scheduling. Celery has a large and diverse community of users and contributors, you should come join us on IRC or our mailing-list. Celery is Open Source and licensed under the BSD License. A task queue’s input is a unit of work called a task. Dedicated worker processes constantly monitor task queues for new work to perform. ...
