smart_open
Utils for streaming large files (S3, HDFS, gzip, bz2...)
...Supported backends include Amazon S3, Google Cloud Storage, Azure Blob Storage, HDFS, WebHDFS, HTTP, HTTPS, SFTP, and local filesystems. Data can be read or written without first downloading an entire remote object to disk. Transparent compression and decompression are supported for multiple common formats. Standard operations such as context managers, iteration, and supported seek behavior remain available. Optional transport dependencies let installations include only the cloud or protocol integrations an application actually requires.