Search Results for "python data analysis" - Page 6
Sort By:
Showing 413 open source projects for "python data analysis"
View related business solutions-
AI-generated apps that pass security reviewRetool lets you generate dashboards, admin panels, and workflows directly on your data. Type something like “Build me a revenue dashboard on my Stripe data” and get a working app with security, permissions, and compliance built in from day one. Whether on our cloud or self-hosted, create the internal software your team needs without compromising enterprise standards or control.
-
Try Google Cloud Risk-Free With $300 in CreditUse your credit across every product. Compute, storage, AI, analytics. When it runs out, 20+ products stay free. You only pay when you choose to.
-
1
ECommerceCrawlers
Collection of Python ecommerce and website crawler examples projects
ECommerceCrawlers is a collection of practical Python web crawler projects designed to gather data from a variety of ecommerce platforms, websites, and online services. It aggregates many independent crawler examples created by contributors and organized into separate subprojects that target specific sites or data sources. These examples demonstrate how to build and operate web scrapers capable of collecting structured information such as product listings, news content, job postings, social media data, and other publicly available web data. ... -
2
Requests-HTML
Pythonic HTML Parsing for Humans
This library intends to make parsing HTML (e.g. scraping the web) as simple and intuitive as possible. When using this library you automatically get full JavaScript support! (Using Chromium, thanks to puppeteer) CSS Selectors (a.k.a jQuery-style, thanks to PyQuery). XPath Selectors, for the faint of heart. Mocked user-agent (like a real web browser). Automatic following of redirects. Connection–pooling and cookie persistence. The Requests experience you know and love, with magical parsing... -
3
django-dynamic-scraper
Creating Scrapy scrapers via the Django admin interface
Django Dynamic Scraper (DDS) is an app for Django build on top of the scraping framework Scrapy. While preserving many of the features of Scrapy it lets you dynamically create and manage spiders via the Django admin interface. With Django Dynamic Scraper (DDS) you can define your Scrapy scrapers dynamically via the Django admin interface and save your scraped items in the database you defined for your Django project. Since it simplifies things DDS is not usable for all kinds of scrapers, but... -
4
Cloud Filer
S3 file management tool with encryption
Cloud Filer is a graphical cloud file management tool and S3 client. Cloud Filer allows you to upload and download files to cloud storage and can be used as part of your backup strategy. Cloud Filer includes pre-internet encryption and file name obfuscation making access to your cloud data useless without the right password. Download and install using pip install cloudfiler - see also https://pypi.org/project/cloudfiler/ -
Earn up to 16% annual interest with Nexo.Put idle assets to work with competitive interest rates, borrow without selling, and trade with precision. All in one platform. Geographic restrictions, eligibility, and terms apply.
-
5
mzitu
Python crawler that downloads image galleries and analyzes titles
...Using text segmentation and frequency analysis, the project can create a word cloud representing common keywords found in the dataset. This makes the repository both a scraping example and a small data analysis experiment built around the collected content. Overall, mzitu serves as a learning-oriented implementation of Python web scraping, data processing, and visualization techniques. -
6
Twitter Intelligence
Twitter Intelligence OSINT project performs tracking and analysis
A project written in Python for Twitter tracking and analysis without using Twitter API. This project is a Python 3.x application. The package dependencies are in the file requirements.txt. Run that command to install the dependencies. SQLite is used as the database. Tweet data is stored on the Tweet, User, Location, Hashtag, HashtagTweet tables. The database is created automatically. analysis.py performs analysis processing. -
7
gdpr
Tool to maintain gdpr data protection declaration
Admins often maintain multiple web pages, each of which under EU-GDPR requires a privacy statement. In order to keep them coherent, up-to-date and at the same time avoiding doing the same work multiple times, this project provides a tool to automatically create the appropriate statements for each page from a single source. The project is currently available in PHP, however if anyone is willing to provide a version in Python or Perl or whatever, it is more than welcome. The project is... -
8
WeChatSogou
Python library to crawl and retrieve data from WeChat accounts
WechatSogou is an open source Python library designed to retrieve data from WeChat official accounts by using the Sogou WeChat search service as its data source. It provides developers with a programmatic way to search for public accounts and collect article information without manually browsing the search interface. It functions as a crawler interface that sends requests to the search engine, retrieves results, and converts the returned pages into structured data that can be used in applications or analysis pipelines. ... -
9
pyspider
A powerful Spider(Web Crawler) system in Python
pyspider is a powerful Spider(Web Crawler) system in Python. Components are connected by message queue. Every component, including message queue, is running in their own process/thread, and replaceable. That means, when process is slow, you can have many instances of processor and make full use of multiple CPUs, or deploy to multiple machines. This architecture makes pyspider really fast. benchmarking. -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
10
gain
Asyncio-based Python framework for building fast web crawling spiders
Gain is a Python web crawling framework designed to simplify the process of building efficient and scalable web scrapers. It is built on top of asynchronous technologies such as asyncio, aiohttp, and uvloop to support high-performance crawling with concurrent network requests. It provides a structured framework for creating spiders that can navigate websites, extract structured data, and process the collected results. -
11
Toapi
Convert websites into structured APIs automatically with Python tool
Toapi is a Python library designed to transform ordinary websites into usable API services. Instead of building a traditional web crawler that collects and stores data before exposing it through an API, Toapi simplifies the process by allowing developers to define data structures that automatically generate an API layer from existing web pages. It works by parsing HTML content from a source site and mapping selected elements into structured data that can be returned as JSON through API endpoints. ... -
12
survol
RDF-based framework monitoring business systems activity
A Python agent and a web interface aiming to help the analysis and investigation of a legacy application. A set of machines, processes, databases, programs etc ... all communicating with each other, manipulating your data, and whose software architecture has become, with time, complicated, difficult to understand, and undocumented. Data are aggregated with an RDF inference engine, creating a global vision of the business information processing. -
13
haipproxy
Distributed proxy IP pool for web crawlers using Scrapy and Redis
...It automatically crawls proxy resources from the internet and aggregates them into a centralized pool that can be accessed by distributed spiders and scraping systems. It is built using Python and relies on Scrapy for high-performance crawling while Redis is used for data storage, communication, and task coordination between components. It includes crawlers that discover proxy servers, validators that test proxy availability and performance, and schedulers that manage crawling and validation tasks. HAipproxy aims to maintain a high availability proxy pool with low latency so that scraping frameworks can rotate proxies efficiently and avoid blocking during large-scale data collection. ... -
14
A comparative synteny analysis tool for target-gene SNP marker discovery: connecting genomics data to breeding in Solanaceae * The foundation of this web source is CodeIgniter. * After setting the CodeIgniter framework, unzip it under the default home directory. The database is based on Maria DB (mysql). Download the schema and import it into the DB.
-
15
Perl Web Scraping Project
Perl Web Scraping Project
...It is a form of copying, in which specific data is gathered and copied from the web, typically into a central local database or spreadsheet, for later retrieval or analysis. Web scraping a web page involves fetching it and extracting from it.[1][2] Fetching is the downloading of a page (which a browser does when you view the page). Therefore, web crawling is a main component of web scraping, to fetch pages for later processing. -
16
DSTK - DataScience ToolKit
DSTK - DataScience ToolKit for All of Us
DSTK - DataScience ToolKit is an opensource free software for statistical analysis, data visualization, text analysis, and predictive analytics. Newer version and smaller file size can be found at: https://sourceforge.net/projects/dstk3/ It is designed to be straight forward and easy to use, and familar to SPSS user. While JASP offers more statistical features, DSTK tends to be a broad solution workbench, including text analysis and predictive analytics features. ... -
17
JAWS - Just Another Web Scraper
A simple Web Scraper using Regular Expression or Html Agility
JAWS or Just Another Web Scraper, is part of the Data Scraping Softwares developed by SVbook, alongside JATI (Image to Text) and JAVT (Video to Text). JAWS offer easy interface to scrape data from the website using regular expression, text preprocessing, or HTML Agility Pack. -
18
sqliv
Massive SQL injection vulnerability scanner for automated web testing
SQLiv is a command-line security tool designed to identify SQL injection vulnerabilities in web applications through automated scanning techniques. Written primarily in Python, the project focuses on discovering potentially vulnerable web pages by analyzing URLs that contain database query parameters. It can perform large-scale scanning by using search engine queries known as SQL injection dorks to collect candidate websites and then test them for vulnerabilities. In addition to bulk scanning, SQLiv supports targeted analysis of specific domains or individual URLs, allowing security researchers to focus on particular web applications. ... -
19
Simple-Scrape is a simple web-scraping library that allows for programmatic access to HTML code. No further techniques are needed and the library is very compact and thus easy to use.
-
20
AlienVault OSSIM
Open Source SIEM
OSSIM, AlienVault’s Open Source Security Information and Event Management (SIEM) product, provides event collection, normalization and correlation. For more advanced functionality, AlienVault Unified Security Management (USM) builds on OSSIM with these additional capabilities: * Log management * Advanced threat detection with a continuously updated library of pre-built correlation rules * Actionable threat intelligence updates from AlienVault Labs Security Research Team * Rich... -
21
A simple but relatively complete DNS and DHCP implementation based on a MySQL (Or other) database with both servers drawing information from the same source thus eliminating one of the headaches of administering a small network.
-
22
Websocket Server in a Pure Data patch
a single patch for direct communications between Pd and web browsers
...download link (and notes) : https://sourceforge.net/projects/websocketserverinapatch/files/ "websocket-server in a patch.pd" single Pd file contains a reusable and fully documented websocket server, meant as an easy way to experiment with Pure Data <-> web-browsers communications through websockets, without the need of other software (like node.js or python) or any specific external to compile (just one deken library is required). A demo is included in the same patch : 'Flames', a simple cross-browser, multi-touch and multi-player graphical interface which makes Pd play creepy sounds :) Just try it : In Pd (vanilla) on Windows, Mac or Linux, install the 'mrpeach' library via 'Help->Find externals' menu, then open the patch and fire any recent web-browsers (on same machine or other computer, smartphone or tablet on same local network) with the URL provided at the top of main windows, and you're good to go ! -
23
htmlarea
Small, powerful, full featured WYSIWYG editor
HTMLArea 4 is a browser based WYSIWYG editor that easily replaces the TEXTAREA in your web pages. It is written in JavaScript, and suitable for use in any modern web browser, and any page on your web site. Current version is 4.0-2016-08-29 -
24
-
25
Trace time
Graphs Internet hop times from your PC to a web site.
The latency between your PC and a web site can sometimes be frustrating, and it can be difficult to work out if it the web site or the Internet connection. This program produces a real-time graph of the hop times between your PC and a nominated web address. You can think of it as tracert with a graphic front end.
