Struct opendal::BlockingOperator

pub struct BlockingOperator { /* private fields */ }

BlockingOperator is the entry for all public blocking APIs.

Read concepts for know more about Operator.

Examples

Init backends

Read more backend init examples in services

use opendal::services::Fs;
use opendal::BlockingOperator;
use opendal::Operator;

fn main() -> Result<()> {
    // Create fs backend builder.
    let mut builder = Fs::default();
    // Set the root for fs, all operations will happen under this root.
    //
    // NOTE: the root must be absolute path.
    builder.root("/tmp");

    // Build an `BlockingOperator` to start operating the storage.
    let _: BlockingOperator = Operator::new(builder)?.finish().blocking();

    Ok(())
}

Init backends with blocking layer

Some services like s3, gcs doesn’t have native blocking supports, we can use layers::BlockingLayer to wrap the async operator to make it blocking.

use opendal::layers::BlockingLayer;
use opendal::services::S3;
use opendal::BlockingOperator;
use opendal::Operator;

async fn test() -> Result<()> {
    // Create fs backend builder.
    let mut builder = S3::default();
    builder.bucket("test");
    builder.region("us-east-1");

    // Build an `BlockingOperator` with blocking layer to start operating the storage.
    let _: BlockingOperator = Operator::new(builder)?
        .layer(BlockingLayer::create()?)
        .finish()
        .blocking();

    Ok(())
}

Implementations

impl BlockingOperator

pub fn limit(&self) -> usize

Get current operator’s limit

pub fn with_limit(&self, limit: usize) -> Self

Specify the batch limit.

Default: 1000

pub fn info(&self) -> OperatorInfo

Get information of underlying accessor.

Examples

use opendal::BlockingOperator;

let info = op.info();

impl BlockingOperator

Operator blocking API.

pub fn stat(&self, path: &str) -> Result<Metadata>

Get given path’s metadata.

Notes

For fetch metadata of entries returned by Lister, it’s better to use [list_with] and [lister_with] with metakey query like Metakey::ContentLength | Metakey::LastModified so that we can avoid extra requests.

Behavior

Services that support create_dir

test and test/ may vary in some services such as S3. However, on a local file system, they’re identical. Therefore, the behavior of stat("test") and stat("test/") might differ in certain edge cases. Always use stat("test/") when you need to access a directory if possible.

Here are the behavior list:

Case	Path	Result
stat existing dir	abc/	Metadata with dir mode
stat existing file	abc/def_file	Metadata with file mode
stat dir without /	abc/def_dir	Error NotFound or metadata with dir mode
stat file with /	abc/def_file/	Error NotFound
stat not existing path	xyz	Error NotFound

Refer to RFC: List Prefix for more details.

Services that not support create_dir

For services that not support create_dir, stat("test/") will return NotFound even when test/abc exists since the service won’t have the concept of dir. There is nothing we can do about this.

Examples

Check if file exists

use opendal::ErrorKind;
if let Err(e) = op.stat("test") {
    if e.kind() == ErrorKind::NotFound {
        println!("file not exist")
    }
}

pub fn stat_with(&self, path: &str) -> FunctionStat

Get given path’s metadata with extra options.

Notes

For fetch metadata of entries returned by Lister, it’s better to use [list_with] and [lister_with] with metakey query like Metakey::ContentLength | Metakey::LastModified so that we can avoid extra requests.

Behavior

Services that support create_dir

test and test/ may vary in some services such as S3. However, on a local file system, they’re identical. Therefore, the behavior of stat("test") and stat("test/") might differ in certain edge cases. Always use stat("test/") when you need to access a directory if possible.

Here are the behavior list:

Case	Path	Result
stat existing dir	abc/	Metadata with dir mode
stat existing file	abc/def_file	Metadata with file mode
stat dir without /	abc/def_dir	Error NotFound or metadata with dir mode
stat file with /	abc/def_file/	Error NotFound
stat not existing path	xyz	Error NotFound

Refer to RFC: List Prefix for more details.

Services that not support create_dir

For services that not support create_dir, stat("test/") will return NotFound even when test/abc exists since the service won’t have the concept of dir. There is nothing we can do about this.

Examples

Get metadata while ETag matches

stat_with will

• return Ok(metadata) if ETag matches
• return Err(error) and error.kind() == ErrorKind::ConditionNotMatch if file exists but ETag mismatch
• return Err(err) if other errors occur, for example, NotFound.

use opendal::ErrorKind;
if let Err(e) = op.stat_with("test").if_match("<etag>").call() {
    if e.kind() == ErrorKind::ConditionNotMatch {
        println!("file exists, but etag mismatch")
    }
    if e.kind() == ErrorKind::NotFound {
        println!("file not exist")
    }
}

pub fn is_exist(&self, path: &str) -> Result<bool>

Check if this path exists or not.

Example

use anyhow::Result;
use opendal::BlockingOperator;
fn test(op: BlockingOperator) -> Result<()> {
    let _ = op.is_exist("test")?;

    Ok(())
}

pub fn create_dir(&self, path: &str) -> Result<()>

Create a dir at given path.

Notes

To indicate that a path is a directory, it is compulsory to include a trailing / in the path. Failure to do so may result in NotADirectory error being returned by OpenDAL.

Behavior

• Create on existing dir will succeed.
• Create dir is always recursive, works like mkdir -p

Examples

op.create_dir("path/to/dir/")?;

pub fn read(&self, path: &str) -> Result<Buffer>

Read the whole path into a bytes.

This function will allocate a new bytes internally. For more precise memory control or reading data lazily, please use BlockingOperator::reader

Examples

let bs = op.read("path/to/file")?;

pub fn read_with(&self, path: &str) -> FunctionRead

Read the whole path into a bytes with extra options.

This function will allocate a new bytes internally. For more precise memory control or reading data lazily, please use BlockingOperator::reader

Examples

use opendal::BlockingOperator;
use opendal::EntryMode;
use opendal::Metakey;
let bs = op.read_with("path/to/file").range(0..10).call()?;

pub fn reader(&self, path: &str) -> Result<BlockingReader>

Create a new reader which can read the whole path.

Examples

let r = op.reader("path/to/file")?;

pub fn reader_with(&self, path: &str) -> FunctionReader

Create a new reader with extra options

Examples

use opendal::BlockingOperator;
use opendal::EntryMode;
use opendal::Metakey;
let r = op
    .reader_with("path/to/file")
    .version("version_id")
    .call()?;

pub fn write(&self, path: &str, bs: impl Into<Bytes>) -> Result<()>

Write bytes into given path.

Notes

• Write will make sure all bytes has been written, or an error will be returned.

Examples

use bytes::Bytes;

op.write("path/to/file", vec![0; 4096])?;

pub fn copy(&self, from: &str, to: &str) -> Result<()>

Copy a file from from to to.

Notes

• from and to must be a file.
• to will be overwritten if it exists.
• If from and to are the same, nothing will happen.
• copy is idempotent. For same from and to input, the result will be the same.

Examples

op.copy("path/to/file", "path/to/file2")?;

pub fn rename(&self, from: &str, to: &str) -> Result<()>

Rename a file from from to to.

Notes

• from and to must be a file.
• to will be overwritten if it exists.
• If from and to are the same, a IsSameFile error will occur.

Examples

op.rename("path/to/file", "path/to/file2")?;

pub fn write_with(&self, path: &str, bs: impl Into<Bytes>) -> FunctionWrite

Write data with option described in OpenDAL RFC-0661

Notes

• Write will make sure all bytes has been written, or an error will be returned.

Examples

use bytes::Bytes;

let bs = b"hello, world!".to_vec();
let _ = op
    .write_with("hello.txt", bs)
    .content_type("text/plain")
    .call()?;

pub fn writer(&self, path: &str) -> Result<BlockingWriter>

Write multiple bytes into given path.

Notes

• Write will make sure all bytes has been written, or an error will be returned.

Examples

use bytes::Bytes;

let mut w = op.writer("path/to/file")?;
w.write(vec![0; 4096])?;
w.write(vec![1; 4096])?;
w.close()?;

pub fn writer_with(&self, path: &str) -> FunctionWriter

Create a new reader with extra options

Examples

use opendal::BlockingOperator;
use opendal::EntryMode;
use opendal::Metakey;
let mut w = op.writer_with("path/to/file").call()?;
w.write(vec![0; 4096])?;
w.write(vec![1; 4096])?;
w.close()?;

pub fn delete(&self, path: &str) -> Result<()>

Delete given path.

Notes

• Delete not existing error won’t return errors.

Examples

op.delete("path/to/file")?;

pub fn delete_with(&self, path: &str) -> FunctionDelete

Delete given path with options.

Notes

• Delete not existing error won’t return errors.

Examples

let _ = op
    .delete_with("path/to/file")
    .version("example_version")
    .call()?;

pub fn remove_via(&self, input: impl Iterator<Item = String>) -> Result<()>

remove will remove files via the given paths.

remove_via will remove files via the given vector iterators.

Notes

We don’t support batch delete now.

Examples

let stream = vec!["abc".to_string(), "def".to_string()].into_iter();
op.remove_via(stream)?;

pub fn remove(&self, paths: Vec<String>) -> Result<()>

Notes

We don’t support batch delete now.

Examples

op.remove(vec!["abc".to_string(), "def".to_string()])?;

pub fn remove_all(&self, path: &str) -> Result<()>

Remove the path and all nested dirs and files recursively.

Notes

We don’t support batch delete now.

Examples

op.remove_all("path/to/dir")?;

pub fn list(&self, path: &str) -> Result<Vec<Entry>>

List entries that starts with given path in parent dir.

Notes

Recursively List

This function only read the children of the given directory. To read all entries recursively, use BlockingOperator::list_with("path").recursive(true) instead.

Streaming List

This function will read all entries in the given directory. It could take very long time and consume a lot of memory if the directory contains a lot of entries.

In order to avoid this, you can use BlockingOperator::lister to list entries in a streaming way.

Reuse Metadata

The only metadata that is guaranteed to be available is the Mode. For fetching more metadata, please use BlockingOperator::list_with and metakey.

Examples

use opendal::BlockingOperator;
use opendal::EntryMode;
use opendal::Metakey;
let mut entries = op.list("path/to/dir/")?;
for entry in entries {
    match entry.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file")
        }
        EntryMode::DIR => {
            println!("Handling dir {}", entry.path())
        }
        EntryMode::Unknown => continue,
    }
}

pub fn list_with(&self, path: &str) -> FunctionList

List entries that starts with given path in parent dir. with options.

Notes

Streaming List

This function will read all entries in the given directory. It could take very long time and consume a lot of memory if the directory contains a lot of entries.

In order to avoid this, you can use BlockingOperator::lister to list entries in a streaming way.

Reuse Metadata

The only metadata that is guaranteed to be available is the Mode. For fetching more metadata, please specify the metakey.

Examples

List entries with prefix

This function can also be used to list entries in recursive way.

use opendal::BlockingOperator;
use opendal::EntryMode;
use opendal::Metakey;
let mut entries = op.list_with("prefix/").recursive(true).call()?;
for entry in entries {
    match entry.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file")
        }
        EntryMode::DIR => {
            println!("Handling dir like start a new list via meta.path()")
        }
        EntryMode::Unknown => continue,
    }
}

List entries with metakey for more metadata

use opendal::BlockingOperator;
use opendal::EntryMode;
use opendal::Metakey;
let mut entries = op
    .list_with("dir/")
    .metakey(Metakey::ContentLength | Metakey::LastModified)
    .call()?;
for entry in entries {
    let meta = entry.metadata();
    match meta.mode() {
        EntryMode::FILE => {
            println!(
                "Handling file {} with size {}",
                entry.path(),
                meta.content_length()
            )
        }
        EntryMode::DIR => {
            println!("Handling dir {}", entry.path())
        }
        EntryMode::Unknown => continue,
    }
}

pub fn lister(&self, path: &str) -> Result<BlockingLister>

List entries that starts with given path in parent dir.

This function will create a new BlockingLister to list entries. Users can stop listing via dropping this Lister.

Notes

Recursively List

This function only read the children of the given directory. To read all entries recursively, use BlockingOperator::lister_with and delimiter("") instead.

Metadata

The only metadata that is guaranteed to be available is the Mode. For fetching more metadata, please use BlockingOperator::lister_with and metakey.

Examples

use futures::TryStreamExt;
use opendal::BlockingOperator;
use opendal::EntryMode;
use opendal::Metakey;
let mut ds = op.lister("path/to/dir/")?;
for de in ds {
    let de = de?;
    match de.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file")
        }
        EntryMode::DIR => {
            println!("Handling dir like start a new list via meta.path()")
        }
        EntryMode::Unknown => continue,
    }
}

pub fn lister_with(&self, path: &str) -> FunctionLister

List entries within a given directory as an iterator with options.

This function will create a new handle to list entries.

An error will be returned if given path doesn’t end with /.

Examples

List current dir

use futures::TryStreamExt;
use opendal::BlockingOperator;
use opendal::EntryMode;
use opendal::Metakey;
let mut ds = op
    .lister_with("path/to/dir/")
    .limit(10)
    .start_after("start")
    .call()?;
for entry in ds {
    let entry = entry?;
    match entry.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file {}", entry.path())
        }
        EntryMode::DIR => {
            println!("Handling dir {}", entry.path())
        }
        EntryMode::Unknown => continue,
    }
}

List all files recursively

use futures::TryStreamExt;
use opendal::BlockingOperator;
use opendal::EntryMode;
use opendal::Metakey;
let mut ds = op.lister_with("path/to/dir/").recursive(true).call()?;
for entry in ds {
    let entry = entry?;
    match entry.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file {}", entry.path())
        }
        EntryMode::DIR => {
            println!("Handling dir {}", entry.path())
        }
        EntryMode::Unknown => continue,
    }
}

List files with required metadata

use futures::TryStreamExt;
use opendal::BlockingOperator;
use opendal::EntryMode;
use opendal::Metakey;
let mut ds = op
    .lister_with("path/to/dir/")
    .metakey(Metakey::ContentLength | Metakey::LastModified)
    .call()?;
for entry in ds {
    let entry = entry?;
    let meta = entry.metadata();
    match meta.mode() {
        EntryMode::FILE => {
            println!(
                "Handling file {} with size {}",
                entry.path(),
                meta.content_length()
            )
        }
        EntryMode::DIR => {
            println!("Handling dir {}", entry.path())
        }
        EntryMode::Unknown => continue,
    }
}

Trait Implementations

impl Clone for BlockingOperator

fn clone(&self) -> BlockingOperator

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for BlockingOperator

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<BlockingOperator> for Operator

fn from(v: BlockingOperator) -> Self

Converts to this type from the input type.