Search Results for "data file"
Sort By:
Showing 561 open source projects for "data file"
View related business solutions-
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
Compliant and Reliable File Transfers Backed by Top Security CertificationsStop relying on non-certified, legacy file transfer tools that creak under the weight of modern security demands. Get full audit trails, advanced access controls and more supported by an award-winning team of experts. Start your free 25-day trial today.
-
1
Azure Data Studio
A data management tool that enables working with other SQL tools
Azure Data Studio is a cross-platform database tool for data professionals who use on-premises and cloud data platforms on Windows, macOS, and Linux. Azure Data Studio offers a modern editor experience with IntelliSense, code snippets, source control integration, and an integrated terminal. It's engineered with the data platform user in mind, with the built-in charting of query result sets and customizable dashboards. Use Azure Data Studio to query, design, and manage your databases and data... -
2
Explorer
Series (one-dimensional) and dataframes (two-dimensional)
Explorer brings series (one-dimensional) and data frames (two-dimensional) to Elixir for fast data exploration. -
3
AWS Data Wrangler
Pandas on AWS, easy integration with Athena, Glue, Redshift, etc.
An AWS Professional Service open-source python initiative that extends the power of Pandas library to AWS connecting DataFrames and AWS data-related services. Easy integration with Athena, Glue, Redshift, Timestream, OpenSearch, Neptune, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON, and EXCEL). Built on top of other open-source projects like Pandas, Apache Arrow and Boto3, it offers abstracted functions to execute... -
4
Apache Hudi
Upserts, Deletes And Incremental Processing on Big Data
Apache Hudi (pronounced Hoodie) stands for Hadoop Upserts Deletes and Incrementals. Hudi manages the storage of large analytical datasets on DFS (Cloud stores, HDFS or any Hadoop FileSystem compatible storage). Apache Hudi is a transactional data lake platform that brings database and data warehouse capabilities to the data lake. Hudi reimagines slow old-school batch data processing with a powerful new incremental processing framework for low latency minute-level analytics. Hudi provides efficient upserts, by mapping a given hoodie key (record key + partition path) consistently to a file id, via an indexing mechanism. ... -
$300 Free Credits for Your Google Cloud ProjectsLaunch your next project with $300 in free Google Cloud credits—no strings attached. Test, build, and deploy without risk. Use your credits across the entire Google Cloud platform to find what works best for your needs. After your credits are used, continue with always-free tier services. Only pay when you're ready to scale. Sign up in minutes and start exploring.
-
5
FileTrees.jl
Parallel file processing made easy
Easy everyday parallelism with a file tree abstraction. Read a directory structure as a Julia data structure, (lazy-)load the files, apply map and reduce operations on the data while not exceeding available memory if possible. Make up a file tree in memory, create some data to go with each file (in parallel), write the tree to disk (in parallel). FileTrees is a set of tools to lazy-load, process and save file trees. -
6
JDF.jl
Julia DataFrames serialization format
JDF is a DataFrames serialization format with the following goals, fast save and load times, compressed storage on disk, enabled disk-based data manipulation (not yet achieved), and support for machine learning workloads, e.g. mini-batch, sampling (not yet achieved). JDF stores a DataFrame in a folder with each column stored as a separate file. There is also a metadata.jls file that stores metadata about the original DataFrame. Collectively, the column files, the metadata file, and the folder is called a JDF "file". ... -
7
JLD2
HDF5-compatible file format in pure Julia
...The save and load functions, provided by FileIO, provide a mechanism to read and write data from a JLD2 file. To use these functions, you may either write using FileIO or using JLD2. FileIO will determine the correct package automatically. -
8
FromFile.jl
Julia enhancement proposal (Julep) for implicit per file module
This package exports a macro @from, which can be used to import objects from files. The hope is that you will never have to write include again. FromFile is a Julia Language package. To install FromFile, please open Julia's interactive session (known as REPL) and press ] key in the REPL to use the package mode. -
9
Space Radar Electron
Disk And Memory Space Visualization App built with Electron & d3.js
Space Radar Electron is an application that offers an interactive and comprehensive visualization of disk space and memory usage of your computer. Built with Electron & d3.js, it currently offers visualizations in the form of Sunburst, Treemap and Flamegraph charts. As it scans the contents of your disk, it produces a preview visualization so you can already see what's been scanned. It allows for drilldown of directories, breadcrumbs and navigation. Space Radar works fast, and is... -
Error to trace to log to deploy. One click. No SSH.AppSignal links every error to the trace, the trace to the log, the log to the deploy that shipped it.
-
10
XLSX.jl
Excel file reader and writer for the Julia language
XLSX.jl is a Julia package to read and write Excel spreadsheet files. Internally, an Excel XLSX file is just a Zip file with a set of XML files inside. The formats for these XML files are described in the Standard ECMA-376. This package follows the EMCA-376 to parse and generate XLSX files. -
11
CSV
Utility library for working with CSV and other delimited files
Welcome to CSV.jl! A pure-Julia package for handling delimited text data, be it comma-delimited (csv), tab-delimited (tsv), or otherwise. A fast, flexible delimited file reader/writer for Julia. -
12
Parquet.jl
Julia implementation of Parquet columnar file format reader
A parquet file or dataset can be loaded using the read_parquet function. A parquet dataset is a directory with multiple parquet files, each of which is a partition belonging to the dataset. -
13
JuiceFS
JuiceFS is a distributed POSIX file system built on top of Redis
A POSIX, HDFS and S3 compatible distributed file system for cloud. JuiceFS is designed to bring back the gold-old memories and experience of file systems in local disks to the cloud. JuiceFS is POSIX compliant and is fully compatible with HDFS and S3. Cloud app building or migrating, file sharing cross-geo and cross-cloud has become easier than ever before. Whether it's a public cloud, private cloud, or hybrid cloud, JuiceFS is available on any cloud of your choice and delivers flexibility, availability, scalability and strong consistency for your data-intensive applications. ... -
14
Best-of Python
A ranked list of awesome Python open-source libraries
This curated list contains 390 awesome open-source projects with a total of 1.4M stars grouped into 28 categories. All projects are ranked by a project-quality score, which is calculated based on various metrics automatically collected from GitHub and different package managers. If you like to add or update projects, feel free to open an issue, submit a pull request, or directly edit the projects.yaml. Contributions are very welcome! Ranked list of awesome python libraries for web... -
15
UnROOT.jl
Native Julia I/O package to work with CERN ROOT files objects
UnROOT.jl is a reader for the CERN ROOT file format written entirely in Julia, without any dependence on ROOT or Python. -
16
PlutoSliderServer.jl
Web server to run just the `@bind` parts of a Pluto.jl notebook
Web server to run just the @bind parts of a Pluto.jl notebook. PlutoSliderServer can run a notebook and generate the export HTML file. This will give you the same file as the export button inside Pluto (top right), but automatically, without opening a browser. One use case is to automatically create a GitHub Pages site from a repository with notebooks. For this, take a look at our template repository that used GitHub Actions and PlutoSliderServer to generate a website on every commit. Many... -
17
Literate
Simple package for literate programming in Julia
Literate is a package for Literate Programming. The main purpose is to facilitate writing Julia examples/tutorials that can be included in your package documentation. Literate can generate markdown pages (for e.g. Documenter.jl), and Jupyter notebooks, from the same source file. There is also an option to "clean" the source from all metadata, and produce a pure Julia script. Using a single source file for multiple purposes reduces maintenance, and makes sure your different output formats are... -
18
targets
Function-oriented Make-like declarative workflows for R
The targets package is a pipeline / workflow management tool in R, designed to coordinate multi‐step computational workflows in data science / statistics. It tracks dependencies between “targets” (computational steps), skips steps whose upstream data or code hasn’t changed, supports parallel computation, branching (dynamic generation of sub‐targets), file format abstractions, and encourages reproducible and efficient analyses. It’s something like GNU Make for R, but more integrated. ... -
19
HDF5.jl
Save and load data in the HDF5 file format from Julia
HDF5 stands for Hierarchical Data Format v5 and is closely modeled on file systems. In HDF5, a "group" is analogous to a directory, a "dataset" is like a file. HDF5 also uses "attributes" to associate metadata with a particular group or dataset. HDF5 uses ASCII names for these different objects, and objects can be accessed by Unix-like pathnames, e.g., "/sample1/tempsensor/firsttrial" for a top-level group "sample1", a subgroup "tempsensor", and a dataset "firsttrial". ... -
20
Delta
A viewer for git and diff output
Code evolves, and we all spend time studying diffs. Delta aims to make this both efficient and enjoyable: it allows you to make extensive changes to the layout and styling of diffs, as well as allowing you to stay arbitrarily close to the default git/diff output. Language syntax highlighting with color themes. Within-line highlights based on a Levenshtein edit inference algorithm. Git style strings (foreground color, background color, font attributes) are supported for >20 stylable elements.... -
21
gusty
Making DAG construction easier
gusty allows you to control your Airflow DAGs, Task Groups, and Tasks with greater ease. gusty manages collections of tasks, represented as any number of YAML, Python, SQL, Jupyter Notebook, or R Markdown files. A directory of task files is instantly rendered into a DAG by passing a file path to gusty's create_dag function. gusty also manages dependencies (within one DAG) and external dependencies (dependencies on tasks in other DAGs) for each task file you define. All you have to do is... -
22
Mage.ai
Build, run, and manage data pipelines for integrating data
...That’s why we designed an easy developer experience that you’ll enjoy. Each step in your pipeline is a standalone file containing modular code that’s reusable and testable with data validations. No more DAGs with spaghetti code. Start developing locally with a single command or launch a dev environment in your cloud using Terraform. Write code in Python, SQL, or R in the same data pipeline for ultimate flexibility. -
23
VGit
Visual git plugin for Neovim
Visual Git Plugin for Neovim to enhance your git experience. -
24
Stirling-PDF
Web application that allows you to perform operations on PDF files
Stirling PDF is a powerful, locally hosted web-based PDF manipulation tool offering a wide range of editing, conversion, and utility features. It allows users to merge, split, compress, convert, OCR, and perform other operations on PDF files directly from a browser without uploading data to third-party servers. The tool is privacy-conscious, self-hostable via Docker, and built with modularity in mind to allow future expansion and integration. -
25
Arrow Julia
Official Julia implementation of Apache Arrow
This is a pure Julia implementation of the Apache Arrow data standard. This package provides Julia AbstractVector objects for referencing data that conforms to the Arrow standard. This allows users to seamlessly interface Arrow formatted data with a great deal of existing Julia code.
