Crate opendal 

Apache OpenDAL™ Rust Core: One Layer, All Storage.

Apache OpenDAL™ is an Open Data Access Layer that enables seamless interaction with diverse storage services.

Useful Links

• User guide: opendal.apache.org/docs/core — install, operations, layers, idioms, and extending the core.
• API reference: docs.rs/opendal (release) | dev
• Services & configuration: opendal.apache.org/services
• Concepts: opendal.apache.org/docs/concepts
• Upgrade Guide | Release Notes | RFCs
• Examples

Installation

cargo add opendal

Each service is a feature flag. The in-memory service is always available; enable any other service with its services-* feature, e.g. features = ["services-s3"].

Quickstart

use opendal::services;
use opendal::Operator;
use opendal::Result;

#[tokio::main]
async fn main() -> Result<()> {
    // Configure a service, then build an operator from it.
    let op = Operator::new(services::Memory::default())?;

    // The same verbs work on every service.
    op.write("hello.txt", "Hello, World!").await?;
    let bytes = op.read("hello.txt").await?;
    let meta = op.stat("hello.txt").await?;
    op.delete("hello.txt").await?;

    println!("read {} bytes", meta.content_length());
    Ok(())
}

To use a real backend, swap Memory for another service and configure it — the operations stay identical. See Getting started and Connecting to your storage.

Services

OpenDAL talks to 50+ backends through one API. A selection by category:

Type	Services
Standard Protocols	ftp, http, sftp, webdav
Object Storage	s3, gcs, azblob, oss, cos, obs, b2, vercel-blob, …
File Storage	fs, hdfs, azdls, azfile, webhdfs, ipfs, …
Consumer Cloud Storage	gdrive, onedrive, dropbox, aliyun-drive, koofr, …
Key-Value & Database	redis, etcd, tikv, rocksdb, sqlite, postgresql, mongodb, …
Cache	memcached, moka, mini-moka, ghac, vercel-artifacts

See Services for the full list and each service’s configuration keys.

Layers

Wrap an operator with layers to add retry, logging, timeout, metrics, and other cross-cutting behavior without touching your storage code. Retry, logging, timeout, and concurrency limit are built in:

use opendal::layers::RetryLayer;
use opendal::services::Memory;
use opendal::{Operator, Result};

fn build_operator() -> Result<Operator> {
    let op = Operator::new(Memory::default())?;
    Ok(op.layer(RetryLayer::new()))
}

See Going to production and the layers module.

Contributing

Check out the CONTRIBUTING guide for building, testing, and submitting changes to the core.

Used by

Check out the users list for more details on who is using OpenDAL.

Branding

The first and most prominent mentions must use the full form: Apache OpenDAL™ of the name for any individual usage (webpage, handout, slides, etc.) Depending on the context and writing style, you should use the full form of the name sufficiently often to ensure that readers clearly understand the association of both the OpenDAL project and the OpenDAL software product to the ASF as the parent organization.

For more details, see the Apache Product Name Usage Guide.

License and Trademarks

Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0

Apache OpenDAL, OpenDAL, and Apache are either registered trademarks or trademarks of the Apache Software Foundation.

Re-exports

pub extern crate opendal_testkit as tests;

Modules

blockingblocking

blocking module provides blocking APIs for OpenDAL.

executors

executors module provides implementations for the Execute trait for widely used runtimes.

layers

Layers enabled through layers-* Cargo features.

operator_futures

Futures provides the futures generated by Operator

options

Options module provides options definitions for operations.

raw

Raw modules provide raw APIs that used by underlying services

services

Service builders enabled through services-* Cargo features.

Structs

Buffer

Buffer is a wrapper of contiguous Bytes and non-contiguous [Bytes].

BufferSink

BufferSink is the adapter of [futures::Sink] generated by Writer::into_sink

BufferStream

BufferStream is a stream of buffers, created by Reader::into_stream

Capability

Capability defines the supported operations and their constraints for a storage Operator.

Copier

Copier drives a long-running copy operation one step at a time.

DeleteInput

DeleteInput is the input for delete operations.

Deleter

Deleter is designed to continuously remove content from storage.

Entry

Entry represents an entry’s path and metadata.

Error

Error is the error struct returned by all opendal functions.

Executor

Executor that runs futures in background.

FuturesAsyncReader

FuturesAsyncReader is the adapter of [AsyncRead], [AsyncBufRead] and [AsyncSeek] generated by Reader::into_futures_async_read.

FuturesAsyncWriter

FuturesIoAsyncWriter is the adapter of [AsyncWrite] for Writer.

FuturesBytesSink

FuturesBytesSink is the adapter of [futures::Sink] generated by Writer::into_bytes_sink.

FuturesBytesStream

FuturesBytesStream is the adapter of [Stream] generated by Reader::into_bytes_stream.

FuturesDeleteSink

FuturesDeleteSink is a sink that generated by Deleter

HttpBody

The streaming body returned by HttpTransporter.

HttpTransporter

Type-erased HTTP transport handle.

Lister

Lister lists entries at a given path asynchronously.

Metadata

Metadata contains all the information related to a specific path.

OperationContext

Composed resources passed from operator to services and layers.

Operator

The Operator serves as the entry point for all public asynchronous APIs.

OperatorInfo

Metadata for operator, users can use this metadata to get information of operator.

OperatorRegistry

Global registry that maps schemes to OperatorFactory functions.

OperatorUri

Parsed representation of an operator URI with normalized components.

Reader

Reader is designed to read data from given path in an asynchronous manner.

Writer

Writer is designed to write data into given path in an asynchronous manner.

Enums

BytesRange

BytesRange carries a range of content.

EntryMode

EntryMode represents the mode.

ErrorKind

OpenDAL error categories.

Traits

Builder

OpenDAL uses Builder to set up a service.

Configurator

OpenDAL uses Configurator to configure a service.

Execute

Execute trait is used to execute task in background.

HttpTransport

HTTP transport used by OpenDAL services.

IntoDeleteInput

IntoDeleteInput is a helper trait that makes it easier for users to play with Deleter.

IntoOperatorUri

Conversion trait that builds OperatorUri from various inputs.

Functions

init_default_registry

Initialize the global OperatorRegistry with enabled services.

install_default

Install the global defaults provided by the facade crate.

Type Aliases

OperatorFactory

Factory signature used to construct Operator from a URI and extra options.

Result

A specialized std::result::Result whose default error type is Error.