smart_open is a Python library for streaming very large files from local or remote storage through a familiar file-like interface. It acts as an extended replacement for Python's built-in open function. Supported backends include Amazon S3, Google Cloud Storage, Azure Blob Storage, HDFS, WebHDFS, HTTP, HTTPS, SFTP, and local filesystems. Data can be read or written without first downloading an entire remote object to disk. Transparent compression and decompression are supported for multiple common formats. Standard operations such as context managers, iteration, and supported seek behavior remain available. Optional transport dependencies let installations include only the cloud or protocol integrations an application actually requires.

Features

  • Streaming access to very large files
  • S3, GCS, Azure, HDFS, and HTTP support
  • SSH, SCP, and SFTP file access
  • Transparent compression and decompression
  • Python open-compatible interface
  • Optional transport-specific dependencies

Project Samples

smart_open Screenshot 1

Project Activity

See All Activity >

Categories

File Sharing

License

MIT License

Follow smart_open

smart_open Web Site

Other Useful Business Software
MongoDB Atlas runs apps anywhere Icon
MongoDB Atlas runs apps anywhere

Deploy in 115+ regions with the modern database for every enterprise.

MongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
Start Free
Rate This Project
Login To Rate This Project

User Reviews

Be the first to post a review of smart_open!

Additional Project Details

Programming Language

Python

Related Categories

Python File Sharing Software

Registered

2 days ago
Report inappropriate content