Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Working with Files? #133

Open
pwaller opened this issue Dec 29, 2022 · 3 comments
Open

Working with Files? #133

pwaller opened this issue Dec 29, 2022 · 3 comments
Labels
F-feature-request feature request

Comments

@pwaller
Copy link

pwaller commented Dec 29, 2022

I had a go at writing a basic program to copy the content of files. However, I found that I can't use monoio::io::copy or monoio::io::zero_copy, I understand because monoio::fs::File only implements positional I/O.

I found my way to BufReader which sounds like what I want, from the docs:

BufReader implements AsyncBufRead and AsyncReadRent, and if the inner io implements AsyncWriteRent, it will delegate the implementation.

However, I do not see an implementation unless the underlying AsyncReadRent.

I'd like to be able to copy or zero-copy file-based file descriptors in an obvious way using the functions provided by the library.

Is this possible currently, or is there some implementation missing?

@pwaller pwaller added the F-feature-request feature request label Dec 29, 2022
@Zk2u
Copy link

Zk2u commented Mar 22, 2024

I don't know if this is exactly the same issue, but I'm trying to use BufWriter with File to write to it using AsyncWriteRentExt.

BufWriter states:

BufWriter is a struct with a buffer. BufWriter implements AsyncWriteRent, and if the inner io implements AsyncReadRent, it will delegate the implementation.

However, from the impl<W: AsyncWriteRent> AsyncWriteRent for BufWriter<W>, it appears that it's also delegating AsyncWriteRent. Since File doesn't have AsyncWriteRent, we don't get it using the BufWriter, which conflicts with the statements on BufWriter.

impl<W: AsyncWriteRent> AsyncWriteRent for BufWriter<W> {
async fn write<T: IoBuf>(&mut self, buf: T) -> BufResult<usize, T> {
let owned_buf = self.buf.as_ref().unwrap();
let owned_len = owned_buf.len();
let amt = buf.bytes_init();
if self.pos + amt > owned_len {
// Buf can not be copied directly into OwnedBuf,
// we must flush OwnedBuf first.
match self.flush_buf().await {
Ok(_) => (),
Err(e) => {
return (Err(e), buf);
}
}
}
// Now there are two situations here:
// 1. OwnedBuf has data, and self.pos + amt <= owned_len,
// which means the data can be copied into OwnedBuf.
// 2. OwnedBuf is empty. If we can copy buf into OwnedBuf,
// we will copy it, otherwise we will send it directly(in
// this situation, the OwnedBuf must be already empty).
if amt > owned_len {
self.inner.write(buf).await
} else {
unsafe {
let owned_buf = self.buf.as_mut().unwrap();
owned_buf
.as_mut_ptr()
.add(self.cap)
.copy_from_nonoverlapping(buf.read_ptr(), amt);
}
self.cap += amt;
(Ok(amt), buf)
}
}
// TODO: implement it as real io_vec
async fn writev<T: IoVecBuf>(&mut self, buf: T) -> BufResult<usize, T> {
let slice = match IoVecWrapper::new(buf) {
Ok(slice) => slice,
Err(buf) => return (Ok(0), buf),
};
let (result, slice) = self.write(slice).await;
(result, slice.into_inner())
}
async fn flush(&mut self) -> std::io::Result<()> {
self.flush_buf().await?;
self.inner.flush().await
}
async fn shutdown(&mut self) -> std::io::Result<()> {
self.flush_buf().await?;
self.inner.shutdown().await
}
}

@NKID00 NKID00 mentioned this issue Jun 2, 2024
3 tasks
@Lzzzzzt
Copy link
Collaborator

Lzzzzzt commented Sep 22, 2024

Hello guys, the basic file operations should be usable, would you like to have a try? @pwaller @Zk2u

@Zk2u
Copy link

Zk2u commented Sep 22, 2024

Oh awesome thank you. I was actually doing this to try port sled 1.0.0's on disk backend to something completely async for fun using monoio. I'll check it out when I have time. Good job!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
F-feature-request feature request
Projects
None yet
Development

No branches or pull requests

3 participants