Module ethernet_tap

Userspace application TAP-like network driver

This capsule provides a userspace syscall driver designed to expose a raw IEEE 802.3 (Ethernet) compatible network interface (resembling a Linux/FreeBSD TAP virtual network device).

The driver multiplexes the underlying network interfaces across all applications. Applications can allow a receive buffer to the kernel, and all incoming frames will be placed into all applications’ receive buffers. Every application may transmit frames, which are sent on the underlying Ethernet link.

Incoming frames are sent to all processes (effectively mimicking the behavior of a MacVTap device in Virtual Ethernet Port Aggregator (VEPA) mode). This means that all processes can communicate over the Ethernet link, but processes will generally not able to talk to each other, unless the connected far-end Ethernet device supports hairpin mode and is able to send frames back on the link they came from. If processes should be able to talk to each other, we can instantiate a hairpinning bridge in the Tock kernel, or use a regular bridge and provide each process its own, dedicated tap driver.

Interface description

This driver can both deliver incoming IEEE 802.3 Ethernet frames from the backing EthernetAdapterDatapath interface to userspace applications, as well as accept frames from the application for transmission over the EthernetAdapterDatapath interface.

The driver is built on top of the StreamingProcessSlice abstraction, which allows for lossless streaming of data from the kernel to a userspace application. It places incoming Ethernet frames into the StreamingProcessSlice, prefixed by a driver-specific header per frame. This header contains each frames’ length, allowing userspace to efficiently find frame delimitors without parsing their content.

The per-frame header has the following format:

• Bytes 0 to 1 (u16, native endian): flags
  • Bit 0: receive timestamp valid
• Bytes 2 to 3 (u16, native endian): frame length in bytes, excluding this header
• Bytes 4 to 11 (u64, native endian): receive timestamp
• Bytes 12 to (12 + frame length): frame contents, starting with the Ethernet header, up to and exluding the Frame Check Sequence (FCS)

When one or more frames have been placed into the receive buffer, the driver will schedule an upcall to the application. The application must follow the regular StreamingProcessSlice semantics to swap the buffer for another buffer atomically, to ensure lossless reception of incoming frames.

Futhermore, userspace can transmit a frame by providing a buffer and issuing a command-style system call. The interface can transmit at most a single frame at any given time. Copying the to-be transmitted frame into kernel memory is a synchronous operation, while the actual transmission may be asynchronous. Thus an upcall will be issued to the application once the transmission has commenced. Userspace must not issue a transmission while a previous one has not been acknowledged by means of an upcall.

To convey further information about transmitted frames, an application can allow a “TX info” buffer into the kernel, which the driver can use to feed additional metadata back to the transmitting application, such as the frame transmission timestamp (if supported by the EthernetAdapterDatapath). The layout of this buffer is specified below.

System call interface

Command-type system calls

Implemented through EthernetTapDriver::command.

• Command system call 0: Check if the driver is installed.

  Returns the CommandReturn::success variant if the driver is installed, or CommandReturn::failure with associated ENOSUPPORT otherwise.

• Command system call 1: Query the interface RX statistics.

  Returns CommandReturn::success_u32_u32_u32 with a tuple of 32-bit counters local to the process: (rx_frames, rx_bytes, rx_frames_dropped). Neither rx_frames not rx_bytes include any dropped frames. The counters will wrap at u32::MAX.

  These counters are local to each process.

• Command system call 2: Query the interface TX statistics.

  Returns CommandReturn::success_u32_u32 with a tuple of 32-bit counters local to the process: (tx_frames, tx_bytes). The counters will wrap at u32::MAX.

  These counters are local to each process.

• Command system call 3: Transmit a frame located in the frame_transmit_buffer.

  Arguments:

  1. transmit at most n bytes of the buffer (starting at 0, limited by MAX_MTU, at most u16::MAX bytes). Supplied frames must start with the Ethernet header, and must not include the Frame Check Sequence (FCS).
  2. frame transmission identifier (u32): identifier passed back in the frame transmission upcall (upcall 2).

  Returns:

  • CommandReturn::success if the frame was queued for transmission
  • CommandReturn::failure with ErrorCode::BUSY if a frame is currently being transmitted (wait for upcall)
  • CommandReturn::failure with ErrorCode::SIZE if the TX buffer is to small to possibly contain the entire frame, or the passed frame exceeds the interface MTU.

Subscribe-type system calls

• Upcall 0: (currently not supported) Register an upcall to be informed when the driver was released by another process.

• Upcall 1: Register an upcall to be called when one or more frames have been placed into the receive StreamingProcessSlice.

• Upcall 2: Register an upcall to be called when a frame transmission has been completed.

  Upcall arguments:

  1. statuscode indicating 0 or ErrorCode

  2. If statuscode == 0, flags and length (u32, native endian):

     • bits 16..=30: flags

       • bit 16: TX timestamp valid

     • bits 0..=15: transmitted bytes

  3. frame transmission identifier (u32, native endian): identifier to associate an invocation of command system call 3 to its corresponding upcall.

Read-write allow type system calls:

• Read-write allow buffer 0: Allow a StreamingProcessSlice for received frames to be written by the driver.

• Read-write allow buffer 1: Allow a buffer for transmitted frame metadata to be written by the driver.

  The allowed buffer must be at least 8 bytes in size. Individual fields may be valid or invalid, depending on the flags of the frame transmission upcall (upcall 2) flags.

  Layout:

  • bytes 0..=7: frame transmission timestamp (u64; big endian)

    Transmission timestamp of a frame provided by the underlying EthernetAdapterDatapath implementation.

Read-only allow type system calls:

• Read-only allow buffer 0: Allow a buffer containing a frame to transmit by the driver. The frame transmission must still be scheduled by issuing the appropriate command system call (command system call 6).

Structs

App

EthernetTapDriver

Constants

DRIVER_NUM

Syscall driver number.

MAX_MTU

Maximum size of a frame which can be transmitted over the underlying EthernetAdapterDatapath device.