Search Results for "file"
Sort By:
Showing 350 open source projects for "file"
View related business solutions-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
Cut Data Warehouse Costs by 54%BigQuery delivers 54% lower TCO with exabyte scale and flexible pricing. Free migration tools handle the SQL translation automatically.
-
1
hosts
Consolidate and extend hosts files from several well-curated sources
...You manage extensions by curating this folder tree, where you will find the data for fakenews, social, gambling, and porn extension data that we maintain and provide for you. Create an optional blacklist file. The contents of this file (containing a listing of additional domains in hosts file format) are appended to the unified hosts file during the update process. A sample blacklist is included, and may be modified as you need. -
2
OnionShare
Securely and anonymously share files of any size
OnionShare is an open source tool that allows you to securely and anonymously share files of any size, host websites, and chat with friends using the Tor network. There's no need for middlemen that could very well violate the privacy and security of the things you share online. With OnionShare, you can share files directly with just an address in Tor Browser. OnionShare works because it is accessible as a Tor Onion Service. All you need to do is open it and drag and drop the files you... -
3
VCR.py
Automatically mock your HTTP interactions to simplify testing
...The first time you run code that is inside a VCR.py context manager or decorated function, VCR.py records all HTTP interactions that take place through the libraries it supports and serializes and writes them to a flat file (in yaml format by default). This flat file is called a cassette. When the relevant piece of code is executed again, VCR.py will read the serialized requests and responses from the aforementioned cassette file, and intercept any HTTP requests that it recognizes from the original test run and return the responses that corresponded to those requests. ... -
4
diskover-community
Open source file indexing & storage analytics powered by Elasticsearch
Diskover Community Edition is an open source file system indexing and storage analytics platform designed to help organizations understand and manage large volumes of file data. It crawls file systems and indexes metadata using Elasticsearch, enabling fast search, analysis, and organization of files stored across different storage systems. It allows administrators and users to explore file structures, monitor storage usage, and gain insights into how data is distributed across infrastructure. ... -
Go from Code to Production URL in SecondsSkip the Kubernetes configs. Cloud Run handles HTTPS, scaling, and infrastructure automatically. Two million requests free per month.
-
5
s3-client
Sample python script to work with Amazon S3
Example Python script to work with S3. -
6
Changelog CI
Changelog CI is a GitHub Action that enables a project
...It is able to use Markdown or reStructuredText to generate a Changelog. Finally, It writes the generated changelog at the beginning of the CHANGELOG.md/CHANGELOG.rst (or user-provided filename) file. In addition to that, if a user provides a configuration file (JSON/YAML), Changelog CI parses the user-provided configuration file and renders the changelog according to user's configuration. -
7
urllib3
Python HTTP library with thread-safe connection pooling
urllib3 is a powerful, user-friendly HTTP client for Python. Much of the Python ecosystem already uses urllib3 and you should too. Thread safety, connection pooling. Client-side TLS/SSL verification. File uploads with multipart encoding. Helpers for retrying requests and dealing with HTTP redirects. Support for gzip, deflate, brotli, and zstd encoding. Proxy support for HTTP and SOCKS. 100% test coverage. Professional support for urllib3 is available as part of the Tidelift Subscription. Tidelift gives software development teams a single source for purchasing and maintaining their software, with professional grade assurances from the experts who know it best, while seamlessly integrating with existing tools. -
8
autocrawler
Multiprocess Selenium crawler for downloading images by keywords
...AutoCrawler supports multiprocess and multithreaded downloading, which allows it to retrieve images faster by running several tasks simultaneously. Users provide search terms through a simple keyword file, and the crawler organizes downloaded images into directories for each keyword. It can download either thumbnails or full resolution images and supports multiple image formats such as JPG, GIF, and PNG. It also includes configuration options such as headless mode, download limits, proxy usage, and thread count to customize crawling behavior. -
9
ScrapeGraphAI
Python scraper based on AI
Extracting content from websites and local documents using LLM. ScrapeGraphAI is a web scraping python library that uses LLM and direct graph logic to create scraping pipelines for websites and local documents (XML, HTML, JSON, Markdown, etc.). Just say which information you want to extract and the library will do it for you. -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
10
py-multiaddr
multiaddr implementation in Python
py-multiaddr is a multiaddr implementation in Python. Every choice in computing has a tradeoff. This includes formats, algorithms, encodings, and so on. And even with a great deal of planning, decisions may lead to breaking changes down the road, or to solutions that are no longer optimal. Allowing systems to evolve and grow is important. The Multiformats Project is a collection of protocols that aim to future-proof systems, today. They do this mainly by enhancing format values with... -
11
HTTPX
A next generation HTTP client for Python
...International domains and URLs, keep-alive and connection pooling, sessions with cookie persistence, browser-style SSL verification. Basic/digest authentication, elegant key/value cookies, automatic decompression. Automatic content decoding, unicode response bodies, multipart file uploads, HTTP(S) proxy support. Connection timeouts, streaming downloads, .netrc support, and chunked requests. For more advanced topics, see the Advanced Usage section, the async support section, or the HTTP/2 section. The Developer Interface provides a comprehensive API reference. -
12
Pelican
Static site generator that supports Markdown and reST syntax
...The idea behind “pages” is that they are usually not temporal in nature and are used for content that does not change very often (e.g., “About” or “Contact” pages). Pelican tries to be smart enough to get the information it needs from the file system. -
13
tumblr-crawler
Python crawler to download photos and videos from Tumblr blogs
...It provides a script that automatically retrieves photos and videos from specified Tumblr sites and saves them locally for offline access. Users can specify one or multiple blogs to crawl by editing a configuration file or by passing parameters through the command line. Once executed, the script fetches media from the Tumblr API and stores the downloaded files in folders named after each blog. tumblr-crawler avoids re-downloading files that have already been saved, making repeated runs safe and useful for recovering missing media. It also supports optional proxy configuration, which can help when access to Tumblr content requires routing requests through a proxy server. ... -
14
node-gyp
Node.js native addon build tool
...Multiple target versions of Node.js are supported (i.e. 0.8, ..., 4, 5, 6, etc.), regardless of what version of Node.js is actually installed on your system (node-gyp downloads the necessary development files or headers for the target version). node-gyp requires that you have installed a compatible version of Python, one of: v3.6, v3.7, v3.8, or v3.9. If you have multiple Python versions installed, you can identify which Python version node-gyp should use. A binding.gyp file describes the configuration to build your module, in a JSON-like format. This file gets placed in the root of your package, alongside package.json. -
15
changedetection.io
The best free open source website change detection and restock service
...Using the browser steps configuration, add basic steps before performing change detection, such as logging into websites, adding a product to a cart, accepting cookie logins, entering dates, and refining searches. Monitor and track PDF file changes, and know when a PDF file has text changes. Know when your favourite product is on sale, or other special deals are announced before anyone else. Detect and monitor changes in JSON API responses. -
16
AWS Data Wrangler
Pandas on AWS, easy integration with Athena, Glue, Redshift, etc.
An AWS Professional Service open-source python initiative that extends the power of Pandas library to AWS connecting DataFrames and AWS data-related services. Easy integration with Athena, Glue, Redshift, Timestream, OpenSearch, Neptune, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON, and EXCEL). Built on top of other open-source projects like Pandas, Apache Arrow and Boto3, it offers abstracted functions to execute... -
17
JobFunnel
Scrape job websites into a single spreadsheet with no duplicates.
Scrape job websites into a single spreadsheet with no duplicates. Automated tool for scraping job postings into a .csv file. You can search for jobs with YAML configuration files or by passing command arguments. By performing regular scraping and reviewing, you can cut through the noise of even the busiest job markets. Run funnel with your settings YAML to populate your master CSV file with jobs from available providers. JobFunnel can be easily automated to run nightly with crontab. ... -
18
Uplink
A Declarative HTTP Client for Python
...Then, executing an HTTP request is as simply as invoking a method. Use decorators and type hints to describe each HTTP request. JSON, URL-encoded, and multipart request body and file upload. URL parameter replacement, request headers, and query parameter support. -
19
ScrapydWeb
Web app for Scrapyd cluster management
Web app for Scrapyd cluster management, with support for Scrapy log analysis & visualization. Make sure that Scrapyd has been installed and started on all of your hosts. Start ScrapydWeb via command scrapydweb. (a config file would be generated for customizing settings on the first startup.) Add your Scrapyd servers, both formats of string and tuple are supported, you can attach basic auth for accessing the Scrapyd server, as well as a string for grouping or labeling. You can select any number of Scrapyd servers by grouping and filtering, and then invoke the HTTP JSON API of Scrapyd on the cluster with just a few clicks. -
20
LinkChecker
Check links in web documents or full websites
LinkChecker is a free, GPL licensed website validator. LinkChecker checks links in web documents or full websites. It runs on Python 3 systems, requiring Python 3.8 or later. The version in the pip repository may be old, to find out how to get the latest code, plus platform-specific information and other advice see doc/install.txt in the source code archive. If you do not want to install any additional libraries/dependencies you can use the Docker image which is published on GitHub... -
21
GitGutter
A Sublime Text 2/3 plugin to see git diff in gutter
A Sublime Text plug-in to show information about files in a git repository. Gutter Icons indicating inserted, modified or deleted lines. Diff Popup with details about modified lines. Status Bar Text with information about file and repository and provides some commands like Goto Change to navigate between modified lines. Copy from Commit to copy the original content from the commit. Revert to Commit to revert a modified hunk to the original state in a commit. The diff popup shows the original content from the commit or the differences between it and the working content. ... -
22
Grab Framework Project
Web Scraping Framework
Grab is a python framework for building web scrapers. With Grab you can build web scrapers of various complexity, from simple 5-line scripts to complex asynchronous website crawlers processing millions of web pages. Grab provides an API for performing network requests and for handling the received content e.g. interacting with DOM tree of the HTML document. The single request/response API that allows you to build network request, perform it and work with the received content. The API is... -
23
SeleniumBase
A framework for browser automation and testing with Selenium
...SeleniumBase lets you customize test runs from the command line. SeleniumBase uses simple syntax for commands. pytest includes automatic test discovery. If you don't specify a specific file or folder to run, pytest will automatically search through all subdirectories for tests to run. No More Flaky Tests! SeleniumBase methods automatically wait for page elements to finish loading before interacting with them (up to a timeout limit). This means you no longer need random time.sleep() statements in your scripts. SeleniumBase includes an automated/manual hybrid solution called MasterQA, which speeds up manual testing by having automation perform all the browser actions while the manual tester handles validation. -
24
Azure SDK for Python
Active development of the Azure SDK for Python
...For your convenience, each service has a separate set of libraries that you can choose to use instead of one, large Azure package. To get started with a specific library, see the README.md (or README.rst) file located in the library's project folder. Last stable versions of packages that have been provided for usage with Azure and are production-ready. These libraries provide you with similar functionalities to the Preview ones as they allow you to use and consume existing resources and interact with them, for example: upload a blob. They might not implement the guidelines or have the same feature set as the November releases. ... -
25
FSP - File Service Protocol Suite
UDP File transfer protocol
FSP - File Service Protocol. FSP is lightweight UDP based protocol for transferring files. It is designed for anonymous transfers over unreliable networks.
