Search Results for "data processing" - Page 3
Sort By:
Showing 74 open source projects for "data processing"
View related business solutions-
Earn up to 16% annual interest with Nexo.Generate interest, borrow against your crypto, and trade a range of cryptocurrencies — all in one platform. Geographic restrictions, eligibility, and terms apply.
-
Veeam Data Platform v13.1Try Veeam Data Platform today. Experience the unified platform that's secure by design, portable by default, and proven to recover clean, fast, and anywhere.
-
1
Albedo
A recommender system for discovering GitHub repos
...It treats repositories and developers as a graph of interactions and applies large-scale matrix factorization to model affinities, with Apache Spark providing the distributed data processing. The project focuses on implicit feedback—stars, watches, and other engagement metrics—so it can build useful recommendations without explicit ratings. A reproducible setup and Makefile-driven workflow streamline tasks like spinning up services, loading datasets, training models, and generating candidate lists. Because it’s built around Spark’s scalable primitives, Albedo can experiment on substantial snapshots of GitHub metadata rather than toy corpora. ... -
2
Python Crawler Tutorial Starts From Zero
Python crawler tutorial, taking you from zero to one
Python Crawler Tutorial Starts From Zero is a Chinese-language learning repository that teaches web crawling from introductory concepts through practical examples. Early lessons explain HTTP requests, request analysis, the Python Requests library, and common categories of extracted data. Separate chapters cover JSON processing and regular expressions for transforming responses into structured information. Practical exercises demonstrate crawlers for Douban movies, Baidu Tieba, and Baidu Translate. Broader project materials also address topics such as JavaScript reverse engineering, Selenium automation, OCR, MongoDB, and the Scrapy framework. ... -
3
Twint
An advanced Twitter scraping & OSINT tool written in Python
Twint is an advanced open-source Twitter scraping and OSINT tool written in Python that extracts tweets, user data, followers, likes, and more—without relying on Twitter’s API—making it highly useful for researchers, analysts, and hobbyists who want to bypass rate limits and access public Twitter data. -
4
Django Celery
Old Celery integration project for Django
Celery is a simple, flexible, and reliable distributed system to process vast amounts of messages, while providing operations with the tools required to maintain such a system. It’s a task queue with focus on real-time processing, while also supporting task scheduling. Celery has a large and diverse community of users and contributors, you should come join us on IRC or our mailing-list. Celery is Open Source and licensed under the BSD License. A task queue’s input is a unit of work called a... -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
5
Pipelines
An experimental programming language for data flow
Pipelines is a language and runtime for crafting massively parallel pipelines. Unlike other languages for defining data flow, the Pipeline language requires the implementation of components to be defined separately in the Python scripting language. This allows the details of implementations to be separated from the structure of the pipeline while providing access to thousands of active libraries for machine learning, data analysis, and processing. -
6
Wally
Distributed Stream Processing
...Provide high-performance & low-latency data processing. Be portable and deploy easily (i.e., run on-prem or any cloud). Manage in-memory state for the application. Allow applications to scale as needed, even when they are live and up-and-running. The primary API for Wally is written in Pony. Wally applications are written using this Pony API. -
7
cnn-text-classification-tf
Convolutional Neural Network for Text Classification in Tensorflow
The cnn-text-classification-tf repository by Denny Britz is a well-known educational implementation of convolutional neural networks for text classification using TensorFlow, aimed at helping developers and researchers understand how CNNs can be applied to natural language processing tasks. Based loosely on Kim’s influential paper on CNNs for sentence classification, this codebase demonstrates how to preprocess text data, convert words into learned embeddings, and apply multiple convolution filters to extract n-gram features that are then pooled and fed into a classifier. The project includes scripts for training, evaluation, and data handling, making it easy to run experiments on datasets such as movie reviews or other labeled text collections. ... -
8
Zhao
A compilation of "The Princely Party Relationship Network"
zhao is a repository that consolidates research, data, and insights related to Zhao, which is likely an individual’s research collection, notes, or curated resources on deep learning, AI, or computational topics (name and content context suggest specialized study). The project may include code examples, experiment results, references to academic papers, mathematical notes, and supporting scripts to explore specific ML methods, benchmarks, or theoretical findings. -
9
jsondata
Modular JSON by trees and branches, pointers and patches
The 'jsondata' package provides for the modular in-memory processing of JSON data by trees, branches, pointers, and patches. The main interface classes are: - JSONData - Core for RFC7159 based data structures. Provides modular data components. - JSONDataSerializer - Core for RFC7159 based data persistence. Provides modular data serialization. - JSONPointer - RFC6901 for addressing by pointer paths. -
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
10
pyhanlp
Chinese participle
pyhanlp is a Python interface for HanLP (Han Language Processing) that lets you use a mature Java-based NLP toolkit from Python workflows without rebuilding the underlying algorithms. It is commonly used for Chinese-language NLP tasks where you want production-grade tokenization and linguistic analysis, but still want the convenience of Python scripting. The project focuses on making HanLP’s capabilities accessible through a Python-friendly API surface, so you can integrate NLP steps into data pipelines, notebooks, and downstream ML or information-extraction code. ... -
11
pivottablejs
Drag’n’drop Pivot Tables and Charts for Jupyter/IPython Notebook
PivotTable.js is a Javascript Pivot Table and Pivot Chart library with drag-drop interactivity, and it can now be used with Jupyter/IPython Notebook via the pivottablejs module. I first built PivotTable.js with a plan to build an in-browser data analysis tool, and got as far as one where you could load up a CSV file in the browser for display. Since then, however, the Jupyter project has gathered steam and now provides a browser-based interface to some of the most powerful data processing libraries in the world, so it makes sense to interface with it. -
12
The Modular toolkit for Data Processing (MDP) is a Python data processing framework. From the user's perspective, MDP is a collection of supervised and unsupervised learning algorithms and other data processing units that can be combined into data processing sequences and more complex feed-forward network architectures. From the scientific developer's perspective, MDP is a modular framework, which can easily be expanded. ...
-
13
ElastAlert
Easy and flexible alerting with ElasticSearch
We designed ElastAlert to be reliable, highly modular, and easy to set up and configure. It works by combining Elasticsearch with two types of components, rule types and alerts. Elasticsearch is periodically queried and the data is passed to the rule type, which determines when a match is found. When a match occurs, it is given to one or more alerts, which take action based on the match. This is configured by a set of rules, each of which defines a query, a rule type, and a set of alerts.... -
14
LightProfiler
Profiler for Oracle extended SQL trace files
...It generates detailed resource profile for extended SQL trace files (10046 event), containing information about consuming of response time (by events, by cursors, etc.), data files usage, error analysis (SQL, PL/SQL) and much more. Also it contain tools for additional processing of trace files (extract session data, splitting files) and for management of database's sessions (disconnecting, tracing, monitor parameters, blocking locks, events and etc.) -
15
Importer library to import assets from different common 3D file formats such as Collada, Blend, Obj, X, 3DS, LWO, MD5, MD2, MD3, MDL, MS3D and a lot of other formats. The data is stored in an own in-memory data-format, which can be easily processed. www.open3mod.com/ is a 3D model viewer and exporter based on Assimp that is also Open Source.
-
16
TextBlob
TextBlob is a Python library for processing textual data
Simple, Pythonic, text processing, Sentiment analysis, part-of-speech tagging, noun phrase extraction, translation, and more. It provides a simple API for diving into common natural language processing (NLP) tasks such as part-of-speech tagging, noun phrase extraction, sentiment analysis, classification, translation, and more. TextBlob stands on the giant shoulders of NLTK and pattern, and plays nicely with both. Supports word inflection (pluralization and singularization) and lemmatization,... -
17
Modules for developing, configuring and running a computation based on function blocks entirely in Python. Function block based computation is a data, event and state driven approach to data processing.
-
18
Pypes is a framework which allows users to break complex data processing logic down into a series of smaller less complex tasks. These tasks, referred to as components, can then be connected so that the output of one becomes the input to another.
-
19
Banking and Retail online transaction processing platform. Software is developed utilizing ACE (Adaptive Communications Environment) which is a multi-platform middleware that handles communications, events and multi-threading. C-Tree database.
-
20
Quick reference for switching between mathematical computation environments for computer algebra, numeric processing and data visualisation. Examples are Matlab, IDL, SPlus, and their open-source counterparts Octave, Scilab, Python+NumPy and R.
-
21
UIMP(Unified Image Manipulation Program) is an OS independent Image Processing system, which provides over 1000 algorithms and supports web online, console and GUI program.
-
22
Benetech Human Rights Program Utility Library: Collection of utilities used by the Benetech Human Rights programming for processing human rights data in preparation for statistical analysis. Library handles DSV files, name canonicalization, binarray, et
-
23
SchemaDoc is a XML-based markup language for documenting XML schemas. The work products include both the vocabulary and a set of tools for combining it with the schema source (e.g. a DTD) to produce documentation in HTML, XML DocBook, LaTeX, etc.
-
24
StreamMine is a distributed event processing (streaming) infrastructure. You can create low-latency, fault-tolerant stream processing functionality with any stream-oriented operators that can be implemented in Python.
