Search Results for "directory"

...You may prefer this to the Git filter if your project already uses the pre-commit framework. Note that the Git filter and pre-commit hook work differently, with different effects on your working directory. The pre-commit hook operates on the notebook on disk, cleaning the copy in your working directory. The Git filter cleans notebooks as they are added to the index, leaving the copy in your working directory dirty.

An AWS Professional Service open-source python initiative that extends the power of Pandas library to AWS connecting DataFrames and AWS data-related services. Easy integration with Athena, Glue, Redshift, Timestream, OpenSearch, Neptune, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON, and EXCEL). Built on top of other open-source projects like Pandas, Apache Arrow and Boto3, it offers abstracted functions to execute...

gusty allows you to control your Airflow DAGs, Task Groups, and Tasks with greater ease. gusty manages collections of tasks, represented as any number of YAML, Python, SQL, Jupyter Notebook, or R Markdown files. A directory of task files is instantly rendered into a DAG by passing a file path to gusty's create_dag function. gusty also manages dependencies (within one DAG) and external dependencies (dependencies on tasks in other DAGs) for each task file you define. All you have to do is provide a list of dependencies or external_dependencies inside of a task file, and gusty will automatically set each task's dependencies and create external task sensors for any external dependencies listed. gusty works with both Airflow 1.x and Airflow 2.x, and has even more features, all of which aim to make the creation, management, and iteration of DAGs more fluid, so that you can intuitively design your DAG and build your tasks.

popmon is a package that allows one to check the stability of a dataset. popmon works with both pandas and spark datasets. popmon creates histograms of features binned in time-slices, and compares the stability of the profiles and distributions of those histograms using statistical tests, both over time and with respect to a reference. It works with numerical, ordinal, categorical features, and the histograms can be higher-dimensional, e.g. it can also track correlations between any two...

The SummaryWriter class provides a high-level API to create an event file in a given directory and add summaries and events to it. The class updates the file contents asynchronously. This allows a training program to call methods to add data to the file directly from the training loop, without slowing down training. TensorboardX now supports logging directly to Comet. Comet is a free cloud based solution that allows you to automatically track, compare and explain your experiments. ...

PySchool is a free and open-source Python distribution intended primarily for students who learn Python and data analysis, but it can also used by scientists, engineering, and data scientists. It includes more than 150 Python packages (full edition) including numpy, pandas, scipy, sympy, keras, scikit-learn, matplotlib, seaborn, beautifulsoup4...

...The application ensures that the latest versions of files are consistently maintained across systems while providing detailed execution logs for auditing and troubleshooting purposes. Key Features: Automated File Copying: Copies files and folders from a specified source directory to a destination directory.Supports both local and network paths. File Update Management Detects new or modified files in the source location. Updates only the changed files at the destination, reducing transfer time and bandwidth usage. CSV-Based Logging: Generates detailed execution logs in CSV format. Benefits: Reduces manual effort in file deployment and updates. ...

This is a Fortran Module that merges the performance of the computation of the Fortran with the power of a Python Package called Matplotlib plot the data computed by your fortran code. It allows you to divide you data into groups of datafiles and each group is stored on its own directory. It allows you to create different types of plotting files such as, PNG file for each datafile and PDF and MPEG files for each group of datafiles. It works with FORMATTED and UNFORMATTED fortran datafiles. When you call the main subroutine in this module which is called FORTPLOT, it creates a python coded file that you can execute it at any time to get the plots and take a decision about continue the run or halt it. ...