零拷贝是实现高速数据传输的一种协议。数据从源节点传送到目的节点的过程当中不通过中间缓存。java
具体提升了哪些须要咱们先来了解传统IO的方式,经过与传统IO方式对比来看。缓存
传统IO方式
在java开发中,从某台机器将一份数据经过网络传输到另一台机器,大体的代码以下:网络
Socket socket = new Socket(HOST, PORT); InputStream inputStream = new FileInputStream(FILE_PATH); OutputStream outputStream = new DataOutputStream(socket.getOutputStream()); byte[] buffer = new byte[4096]; while (inputStream.read(buffer) >= 0) { outputStream.write(buffer); } outputStream.close(); socket.close(); inputStream.close();
看起来代码很简单,但若是咱们深刻到操做系统层面,就会发现实际的微观操做更复杂。具体操做以下图:架构
1. 用户进程向OS发出read()系统调用,触发上下文切换,从用户态转换到内核态。 2. CPU发起IO请求,经过直接内存访问(DMA)从磁盘读取文件内容,复制到内核缓冲区PageCache中 3. 将内核缓冲区数据,拷贝到用户空间缓冲区,触发上下文切换,从内核态转换到用户态。 4. 用户进程向OS发起write系统调用,触发上下文切换,从用户态切换到内核态。 5. 将数据从用户缓冲区拷贝到内核中与目的地Socket关联的缓冲区。 6. 数据最终经由Socket经过DMA传送到硬件(网卡)缓冲区,write()系统调用返回,并从内核态切换回用户态。
零拷贝(Zero-copy)
如下使用FileChannel.transferTo方法,实现zero-copy:框架
SocketAddress socketAddress = new InetSocketAddress(HOST, PORT); SocketChannel socketChannel = SocketChannel.open(); socketChannel.connect(socketAddress); File file = new File(FILE_PATH); FileChannel fileChannel = new FileInputStream(file).getChannel(); fileChannel.transferTo(0, file.length(), socketChannel); fileChannel.close(); socketChannel.close();
相比传统方式,零拷贝的执行流程以下图:socket
能够看到,相比传统方式,零拷贝不走数据缓冲区减小了一些没必要要的操做。ide
零拷贝的应用
零拷贝在不少框架中获得了普遍使用,常见的好比Netty、Kafka等等。ui
在kafka中使用了不少设计思想,好比分区并行、顺序写入、页缓存、高效序列化、零拷贝等等。this
上边博客分析了Kafka的大概架构,知道了kafka中的文件都是以.log文件存储,每一个日志文件对应两个索引文件.index与.timeindex。操作系统
kafka在传输数据时利用索引,使用fileChannel.transferTo(position, count, socketChannel)指定数据位置与大小实现零拷贝。
kafka底层传输源码:(TransportLayer)
/** * Transfers bytes from `fileChannel` to this `TransportLayer`. * * This method will delegate to {@link FileChannel#transferTo(long, long, java.nio.channels.WritableByteChannel)}, * but it will unwrap the destination channel, if possible, in order to benefit from zero copy. This is required * because the fast path of `transferTo` is only executed if the destination buffer inherits from an internal JDK * class. * * @param fileChannel The source channel * @param position The position within the file at which the transfer is to begin; must be non-negative * @param count The maximum number of bytes to be transferred; must be non-negative * @return The number of bytes, possibly zero, that were actually transferred * @see FileChannel#transferTo(long, long, java.nio.channels.WritableByteChannel) */ long transferFrom(FileChannel fileChannel, long position, long count) throws IOException;
实现类(PlaintextTransportLayer):
@Override public long transferFrom(FileChannel fileChannel, long position, long count) throws IOException { return fileChannel.transferTo(position, count, socketChannel); }
该方法的功能是将FileChannel中的数据传输到TransportLayer,也就是SocketChannel。在实现类PlaintextTransportLayer的对应方法中,就是直接调用了FileChannel.transferTo()方法。