capsules_extra/
ethernet_tap.rs

1// Licensed under the Apache License, Version 2.0 or the MIT License.
2// SPDX-License-Identifier: Apache-2.0 OR MIT
3// Copyright Leon Schuermann <leon@is.currently.online> 2025.
4// Copyright Tock Contributors 2025.
5
6//! Userspace application TAP-like network driver
7//!
8//! This capsule provides a userspace syscall driver designed to expose a raw
9//! IEEE 802.3 (Ethernet) compatible network interface (resembling a
10//! Linux/FreeBSD TAP virtual network device).
11//!
12//! The driver multiplexes the underlying network interfaces across all
13//! applications. Applications can allow a receive buffer to the kernel, and all
14//! incoming frames will be placed into all applications' receive buffers. Every
15//! application may transmit frames, which are sent on the underlying Ethernet
16//! link.
17//!
18//! Incoming frames are sent to all processes (effectively mimicking the
19//! behavior of a MacVTap device in Virtual Ethernet Port Aggregator (VEPA)
20//! mode). This means that all processes can communicate over the Ethernet link,
21//! but processes will generally not able to talk to each other, _unless_ the
22//! connected far-end Ethernet device supports hairpin mode and is able to send
23//! frames back on the link they came from. If processes should be able to talk
24//! to each other, we can instantiate a hairpinning bridge in the Tock kernel,
25//! or use a regular bridge and provide each process its own, dedicated tap
26//! driver.
27//!
28//! ## Interface description
29//!
30//! This driver can both deliver incoming IEEE 802.3 Ethernet frames from the
31//! backing [`EthernetAdapterDatapath`] interface to userspace applications, as
32//! well as accept frames from the application for transmission over the
33//! [`EthernetAdapterDatapath`] interface.
34//!
35//! The driver is built on top of the [`StreamingProcessSlice`] abstraction,
36//! which allows for lossless streaming of data from the kernel to a userspace
37//! application. It places incoming Ethernet frames into the
38//! [`StreamingProcessSlice`], prefixed by a driver-specific header per
39//! frame. This header contains each frames' length, allowing userspace to
40//! efficiently find frame delimitors without parsing their content.
41//!
42//! The per-frame header has the following format:
43//! - Bytes 0 to 1 (`u16`, native endian): flags
44//!   - Bit 0: receive timestamp valid
45//! - Bytes 2 to 3 (`u16`, native endian): frame length in bytes, excluding this header
46//! - Bytes 4 to 11 (`u64`, native endian): receive timestamp
47//! - Bytes 12 to (12 + frame length): frame contents, starting with the
48//!   Ethernet header, up to and exluding the Frame Check Sequence (FCS)
49//!
50//! When one or more frames have been placed into the receive buffer, the driver
51//! will schedule an upcall to the application. The application must follow the
52//! regular [`StreamingProcessSlice`] semantics to swap the buffer for another
53//! buffer atomically, to ensure lossless reception of incoming frames.
54//!
55//! Futhermore, userspace can transmit a frame by providing a buffer and
56//! issuing a command-style system call. The interface can transmit at most a
57//! single frame at any given time. Copying the to-be transmitted frame into
58//! kernel memory is a synchronous operation, while the actual transmission may
59//! be asynchronous. Thus an upcall will be issued to the application once the
60//! transmission has commenced. Userspace must not issue a transmission while a
61//! previous one has not been acknowledged by means of an upcall.
62//!
63//! To convey further information about transmitted frames, an application can
64//! allow a "TX info" buffer into the kernel, which the driver can use to feed
65//! additional metadata back to the transmitting application, such as the frame
66//! transmission timestamp (if supported by the
67//! [`EthernetAdapterDatapath`]). The layout of this buffer is specified below.
68//!
69//! ## System call interface
70//!
71//! ### Command-type system calls
72//!
73//! Implemented through [`EthernetTapDriver::command`].
74//!
75//! - **Command system call `0`**: Check if the driver is installed.
76//!
77//!   Returns the [`CommandReturn::success`] variant if the driver is installed,
78//!   or [`CommandReturn::failure`] with associated ENOSUPPORT otherwise.
79//!
80//! - **Command system call `1`**: Query the interface RX statistics.
81//!
82//!   Returns [`CommandReturn::success_u32_u32_u32`] with a tuple of 32-bit
83//!   counters local to the process: `(rx_frames, rx_bytes,
84//!   rx_frames_dropped)`. Neither `rx_frames` not `rx_bytes` include any
85//!   dropped frames. The counters will wrap at `u32::MAX`.
86//!
87//!   These counters are local to each process.
88//!
89//! - **Command system call `2`**: Query the interface TX statistics.
90//!
91//!   Returns [`CommandReturn::success_u32_u32`] with a tuple of 32-bit counters
92//!   local to the process: `(tx_frames, tx_bytes)`. The counters will wrap at
93//!   `u32::MAX`.
94//!
95//!   These counters are local to each process.
96//!
97//! - **Command system call `3`**: Transmit a frame located in the
98//!   `frame_transmit_buffer`.
99//!
100//!   Arguments:
101//!   1. transmit at most `n` bytes of the buffer (starting at `0`, limited by
102//!      `MAX_MTU`, at most `u16::MAX` bytes). Supplied frames must start with
103//!      the Ethernet header, and must not include the Frame Check Sequence
104//!      (FCS).
105//!   2. frame transmission identifier (`u32`): identifier passed back in the
106//!      frame transmission upcall (upcall `2`).
107//!
108//!   Returns:
109//!   - [`CommandReturn::success`] if the frame was queued for transmission
110//!   - [`CommandReturn::failure`] with [`ErrorCode::BUSY`] if a frame is
111//!     currently being transmitted (wait for upcall)
112//!   - [`CommandReturn::failure`] with [`ErrorCode::SIZE`] if the TX buffer is
113//!     to small to possibly contain the entire frame, or the passed frame
114//!     exceeds the interface MTU.
115//!
116//! ## Subscribe-type system calls
117//!
118//! - **Upcall `0`**: _(currently not supported)_ Register an upcall to be
119//!   informed when the driver was released by another process.
120//!
121//! - **Upcall `1`**: Register an upcall to be called when one or more frames
122//!   have been placed into the receive [`StreamingProcessSlice`].
123//!
124//! - **Upcall `2`**: Register an upcall to be called when a frame transmission
125//!   has been completed.
126//!
127//!   Upcall arguments:
128//!   1. `statuscode` indicating 0 or ErrorCode
129//!
130//!   2. If `statuscode == 0`, flags and length (`u32`, native endian):
131//!
132//!      - bits `16..=30`: flags
133//!        - bit 16: TX timestamp valid
134//!
135//!      - bits `0..=15`: transmitted bytes
136//!
137//!   3. frame transmission identifier (`u32`, native endian): identifier to
138//!      associate an invocation of _command system call `3`_ to its
139//!      corresponding upcall.
140//!
141//! ## Read-write allow type system calls:
142//!
143//! - **Read-write allow buffer `0`**: Allow a [`StreamingProcessSlice`] for
144//!   received frames to be written by the driver.
145//!
146//! - **Read-write allow buffer `1`**: Allow a buffer for transmitted frame
147//!   metadata to be written by the driver.
148//!
149//!   The allowed buffer must be at least 8 bytes in size. Individual fields may
150//!   be valid or invalid, depending on the flags of the frame transmission
151//!   upcall (upcall `2`) flags.
152//!
153//!   Layout:
154//!   - bytes `0..=7`: frame transmission timestamp (u64; big endian)
155//!
156//!     Transmission timestamp of a frame provided by the underlying
157//!     [`EthernetAdapterDatapath`] implementation.
158//!
159//! ## Read-only allow type system calls:
160//!
161//! - **Read-only allow buffer `0`**: Allow a buffer containing a frame to
162//!   transmit by the driver. The frame transmission must still be scheduled by
163//!   issuing the appropriate command system call (*command system call `6`*).
164
165use kernel::errorcode::into_statuscode;
166use kernel::grant::{AllowRoCount, AllowRwCount, Grant, GrantKernelData, UpcallCount};
167use kernel::hil::ethernet::{EthernetAdapterDatapath, EthernetAdapterDatapathClient};
168use kernel::processbuffer::{ReadableProcessBuffer, WriteableProcessBuffer};
169use kernel::syscall::{CommandReturn, SyscallDriver};
170use kernel::utilities::cells::MapCell;
171use kernel::utilities::streaming_process_slice::StreamingProcessSlice;
172use kernel::ErrorCode;
173use kernel::ProcessId;
174
175/// Syscall driver number.
176pub const DRIVER_NUM: usize = capsules_core::driver::NUM::EthernetTap as usize;
177
178/// Maximum size of a frame which can be transmitted over the underlying
179/// [`EthernetAdapterDatapath`] device.
180///
181/// Currently hard-coded to `1522 - 4` bytes, for an Ethernet frame with an
182/// 802.1q VLAN tag with a 1500 byte payload MTU, excluding the 4-byte FCS.
183pub const MAX_MTU: usize = 1518;
184
185mod upcall {
186    pub const RX_FRAME: usize = 0;
187    pub const TX_FRAME: usize = 1;
188    pub const COUNT: u8 = 2;
189}
190
191mod ro_allow {
192    pub const TX_FRAME: usize = 0;
193    pub const COUNT: u8 = 1;
194}
195
196mod rw_allow {
197    pub const RX_FRAMES: usize = 0;
198    pub const TX_FRAME_INFO: usize = 1;
199    pub const COUNT: u8 = 2;
200}
201
202/// Receive streaming packet buffer frame header constants:
203mod rx_frame_header {
204    use core::ops::Range;
205
206    pub const LENGTH: usize = 12;
207    pub const FLAGS_BYTES: Range<usize> = 0_usize..2;
208    pub const FRAME_LENGTH_BYTES: Range<usize> = 2_usize..4;
209    pub const RECEIVE_TIMESTAMP_BYTES: Range<usize> = 4_usize..12;
210}
211
212#[derive(Default)]
213pub struct App {
214    // Per-process interface statistics:
215    tx_frames: u32,
216    tx_bytes: u32,
217
218    rx_frames: u32,
219    rx_bytes: u32,
220    rx_frames_dropped: u32,
221    rx_bytes_dropped: u32,
222
223    // Pending transmission state:
224    tx_pending: Option<(u16, u32)>,
225}
226
227enum TxState {
228    Active { process: ProcessId },
229    Inactive { buffer: &'static mut [u8] },
230}
231
232pub struct EthernetTapDriver<'a, E: EthernetAdapterDatapath<'a>> {
233    /// The underlying [`EthernetAdapterDatapath`] network device
234    iface: &'a E,
235
236    /// Per-process state
237    apps: Grant<
238        App,
239        UpcallCount<{ upcall::COUNT }>,
240        AllowRoCount<{ ro_allow::COUNT }>,
241        AllowRwCount<{ rw_allow::COUNT }>,
242    >,
243
244    /// Transmission buffer for frames to transmit from the application, or
245    /// indicator which application is currently transmitting.
246    tx_state: MapCell<TxState>,
247}
248
249impl<'a, E: EthernetAdapterDatapath<'a>> EthernetTapDriver<'a, E> {
250    pub fn new(
251        iface: &'a E,
252        grant: Grant<
253            App,
254            UpcallCount<{ upcall::COUNT }>,
255            AllowRoCount<{ ro_allow::COUNT }>,
256            AllowRwCount<{ rw_allow::COUNT }>,
257        >,
258        tx_buffer: &'static mut [u8],
259    ) -> Self {
260        EthernetTapDriver {
261            iface,
262            apps: grant,
263            tx_state: MapCell::new(TxState::Inactive { buffer: tx_buffer }),
264        }
265    }
266
267    pub fn initialize(&self) {
268        self.iface.enable_receive();
269    }
270
271    // This method must only be called when `tx_state` is currently `Inactive`
272    // and the `grant` indicates that there is a frame to transmit
273    // (`tx_pending.is_some()`).
274    fn transmit_frame_from_grant(
275        &self,
276        tx_state: &mut TxState,
277        process_id: ProcessId,
278        grant: &App,
279        kernel_data: &GrantKernelData<'_>,
280    ) -> Result<(), ErrorCode> {
281        // Mark this process as having an active transmission:
282        let prev_state = core::mem::replace(
283            tx_state,
284            TxState::Active {
285                process: process_id,
286            },
287        );
288
289        let transmit_buffer = match prev_state {
290            // This method must never be called during an active
291            // transmission. If this code-path is reached, this represents a
292            // driver-internal invariant violation.
293            TxState::Active { .. } => unreachable!(),
294            TxState::Inactive { buffer } => buffer,
295        };
296
297        // This method must never be called when a process does not have a
298        // pending transmission:
299        let (frame_len, transmission_identifier) = grant.tx_pending.unwrap();
300
301        // Try to copy the frame from the allowed slice into the transmission
302        // buffer. Return `ErrorCode::SIZE` if the `frame_len` exceeds the
303        // allowed slice, the interface MTU, or the transmission buffer:
304        let res = kernel_data
305            .get_readonly_processbuffer(ro_allow::TX_FRAME)
306            .and_then(|tx_frame| {
307                tx_frame.enter(|data| {
308                    if frame_len as usize
309                        > core::cmp::min(data.len(), core::cmp::min(MAX_MTU, transmit_buffer.len()))
310                    {
311                        return Err(ErrorCode::SIZE);
312                    }
313
314                    // `frame_len` fits into source slice, destination buffer, and
315                    // MTU, copy it:
316                    data[..(frame_len as usize)]
317                        .copy_to_slice(&mut transmit_buffer[..(frame_len as usize)]);
318
319                    Ok(())
320                })
321            })
322            .unwrap_or(Err(ErrorCode::FAIL));
323
324        if let Err(e) = res {
325            // We were unable to copy the frame, put the buffer back. The caller
326            // is responsible for informing the process (via a sychronous return
327            // value or upcall) and to reset the process' `tx_pending` field:
328            *tx_state = TxState::Inactive {
329                buffer: transmit_buffer,
330            };
331            return Err(e);
332        }
333
334        // Frame was copied, initiate transmission:
335        if let Err((e, returned_buffer)) =
336            self.iface
337                .transmit_frame(transmit_buffer, frame_len, transmission_identifier as usize)
338        {
339            // We were unable to initiate the transmission put the buffer
340            // back. The caller is responsible for informing the process (via a
341            // sychronous return value or upcall) and to reset the process'
342            // `tx_pending` field:
343            *tx_state = TxState::Inactive {
344                buffer: returned_buffer,
345            };
346            return Err(e);
347        }
348
349        Ok(())
350    }
351
352    // Attempt to complete a transmission requested by a process.
353    //
354    // If this process is still alive, it will reset its `tx_pending` state and
355    // enqueue an upcall. Either case, it will return the `frame_buffer` and
356    // reset the `tx_state` to `Inactive`.
357    fn complete_transmission(
358        &self,
359        err: Result<(), ErrorCode>,
360        frame_buffer: &'static mut [u8],
361        len: u16,
362        transmission_identifier: u32,
363        timestamp: Option<u64>,
364    ) {
365        let process_id = self
366            .tx_state
367            .map(|tx_state| {
368                // First, replace the `frame_buffer` and reset the `tx_state` to
369                // `Inactive`. Also extract the currently transmitting process from
370                // the `tx_state`, which must be `Active` when entering into this
371                // callback:
372                match core::mem::replace(
373                    tx_state,
374                    TxState::Inactive {
375                        buffer: frame_buffer,
376                    },
377                ) {
378                    TxState::Active { process } => process,
379                    TxState::Inactive { .. } => panic!(),
380                }
381            })
382            .unwrap();
383
384        // Now, attempt to complete this process' transmission, if its still
385        // alive and has subscribed to the transmission upcall / allowed a TX
386        // info buffer respectively:
387        let _ = self.apps.enter(process_id, |grant, kernel_data| {
388            // If the application has allowed a "TX info" buffer, write some
389            // transmission metadata to it:
390            let ts_bytes = timestamp.map_or(u64::to_be_bytes(0), |ts| u64::to_be_bytes(ts));
391            let _ = kernel_data
392                .get_readwrite_processbuffer(rw_allow::TX_FRAME_INFO)
393                .and_then(|tx_frame_info| {
394                    tx_frame_info.mut_enter(|tx_info_buf| {
395                        if tx_info_buf.len() >= 8 {
396                            tx_info_buf[0..8].copy_from_slice(&ts_bytes);
397                        }
398                    })
399                });
400
401            // Encode combined flags / length upcall parameter:
402            let flags_len: u32 = {
403                let flags_bytes = u16::to_be_bytes((timestamp.is_some() as u16) << 0);
404
405                let len_bytes = u16::to_be_bytes(len);
406
407                u32::from_be_bytes([flags_bytes[0], flags_bytes[1], len_bytes[0], len_bytes[1]])
408            };
409
410            kernel_data
411                .schedule_upcall(
412                    upcall::TX_FRAME,
413                    (
414                        into_statuscode(err),
415                        flags_len as usize,
416                        transmission_identifier as usize,
417                    ),
418                )
419                .ok();
420
421            // Reset the `tx_pending` state of this app:
422            grant.tx_pending = None;
423        });
424    }
425
426    pub fn command_transmit_frame(
427        &self,
428        process_id: ProcessId,
429        len: u16,
430        transmission_identifier: u32,
431    ) -> Result<(), ErrorCode> {
432        self.apps
433            .enter(process_id, |grant, kernel_data| {
434                // Make sure that this process does not have another pending
435                // transmission:
436                if grant.tx_pending.is_some() {
437                    return Err(ErrorCode::BUSY);
438                }
439
440                // Copy this command call's argument into the process' grant:
441                grant.tx_pending = Some((len, transmission_identifier));
442
443                // If we don't have an active transmission, try to enqueue this
444                // frame for synchronous transmission:
445                self.tx_state
446                    .map(|tx_state| {
447                        if matches!(tx_state, TxState::Inactive { .. }) {
448                            if let Err(e) = self.transmit_frame_from_grant(
449                                tx_state,
450                                process_id,
451                                grant,
452                                kernel_data,
453                            ) {
454                                // Reset the `tx_pending` field:
455                                grant.tx_pending = None;
456
457                                // Return the error:
458                                Err(e)
459                            } else {
460                                // Transmission initiated:
461                                Ok(())
462                            }
463                        } else {
464                            // Transmission is enqueued:
465                            Ok(())
466                        }
467                    })
468                    .unwrap()
469            })
470            .unwrap_or(Err(ErrorCode::FAIL))
471    }
472}
473
474/// Userspace system call driver interface implementation
475impl<'a, E: EthernetAdapterDatapath<'a>> SyscallDriver for EthernetTapDriver<'a, E> {
476    fn command(
477        &self,
478        command_num: usize,
479        arg1: usize,
480        arg2: usize,
481        process_id: ProcessId,
482    ) -> CommandReturn {
483        match command_num {
484            // Check if driver is installed
485            0 => CommandReturn::success(),
486
487            // Query process-specific RX stats
488            1 => self
489                .apps
490                .enter(process_id, |grant, _kernel_data| {
491                    CommandReturn::success_u32_u32_u32(
492                        grant.rx_frames,
493                        grant.rx_bytes,
494                        grant.rx_frames_dropped,
495                    )
496                })
497                .unwrap_or(CommandReturn::failure(ErrorCode::FAIL)),
498
499            // Query process-specific TX stats
500            2 => self
501                .apps
502                .enter(process_id, |grant, _kernel_data| {
503                    CommandReturn::success_u32_u32(grant.tx_frames, grant.tx_bytes)
504                })
505                .unwrap_or(CommandReturn::failure(ErrorCode::FAIL)),
506
507            // Transmit frame from the `allow_ro::TX_FRAME` buffer:
508            3 => match self.command_transmit_frame(process_id, arg1 as u16, arg2 as u32) {
509                Ok(()) => CommandReturn::success(),
510                Err(e) => CommandReturn::failure(e),
511            },
512
513            _ => CommandReturn::failure(ErrorCode::NOSUPPORT),
514        }
515    }
516
517    fn allocate_grant(&self, processid: ProcessId) -> Result<(), kernel::process::Error> {
518        self.apps.enter(processid, |_, _| {})
519    }
520}
521
522/// Callback client for the underlying [`EthernetAdapterDatapath`]:
523impl<'a, E: EthernetAdapterDatapath<'a>> EthernetAdapterDatapathClient
524    for EthernetTapDriver<'a, E>
525{
526    fn transmit_frame_done(
527        &self,
528        err: Result<(), ErrorCode>,
529        frame_buffer: &'static mut [u8],
530        len: u16,
531        transmission_identifier: usize,
532        timestamp: Option<u64>,
533    ) {
534        // Complete this transmission. This will reset both the process'
535        // `tx_pending` state, and the global `tx_state`:
536        self.complete_transmission(
537            err,
538            frame_buffer,
539            len,
540            transmission_identifier as u32,
541            timestamp,
542        );
543
544        // Now, check for any processes that have a pending transmission. We
545        // break out of the loop on the first successfully initiated
546        // transmission, as we only have one transmit buffer for this capsule:
547        self.tx_state
548            .map(|tx_state| {
549                for process_grant in self.apps.iter() {
550                    let process_id = process_grant.processid();
551                    let transmission_initiated = process_grant.enter(|grant, kernel_data| {
552                        let (_, transmission_identifier) = match grant.tx_pending {
553                            None => {
554                                // Skip, this process does not have a pending transmission:
555                                return false;
556                            }
557                            Some(tx_pending) => tx_pending,
558                        };
559
560                        // This process does have a pending transmission, submit it:
561                        if let Err(e) =
562                            self.transmit_frame_from_grant(tx_state, process_id, grant, kernel_data)
563                        {
564                            // Initializing the transmission failed. We must
565                            // report this error through a callback, as no other
566                            // callback will be raised for this transmission:
567                            let _ = kernel_data.schedule_upcall(
568                                upcall::TX_FRAME,
569                                (
570                                    e as usize,
571                                    0, // flags are ignored in case of an error
572                                    transmission_identifier as usize,
573                                ),
574                            );
575
576                            // Remove the pending transmission from this process:
577                            grant.tx_pending = None;
578
579                            // This error may transient and be due to the
580                            // particular enqueued transmission of this process
581                            // (like exceeding the MTU). Try scheduling a
582                            // transmission for the next pending process in the
583                            // next loop iteration:
584                            false
585                        } else {
586                            // Transmission initiated successfully:
587                            true
588                        }
589                    });
590
591                    if transmission_initiated {
592                        // Can only initate one transmission:
593                        break;
594                    }
595
596                    // Otherwise, continue:
597                }
598            })
599            .unwrap();
600    }
601
602    fn received_frame(&self, frame: &[u8], timestamp: Option<u64>) {
603        // Generate a header to prefix the frame contents placed in the
604        // processes' `TX_FRAMES` [`StreamingProcessSlice`]:
605        let mut frame_header = [0; rx_frame_header::LENGTH];
606
607        // frame_header[0..2]: flags
608        frame_header[rx_frame_header::FLAGS_BYTES]
609            .copy_from_slice(&u16::to_ne_bytes((timestamp.is_some() as u16) << 0));
610
611        let len_u16: u16 = match frame.len().try_into() {
612            Ok(len) => len,
613            Err(_) => {
614                kernel::debug!(
615                    "Incoming frame exceeds {} bytes ({} bytes), discarding.",
616                    u16::MAX,
617                    frame.len()
618                );
619                return;
620            }
621        };
622        // frame_header[2..4]: length (excluding frame_header)
623        frame_header[rx_frame_header::FRAME_LENGTH_BYTES]
624            .copy_from_slice(&u16::to_ne_bytes(len_u16));
625
626        // frame_header[4..12]: timestamp
627        frame_header[rx_frame_header::RECEIVE_TIMESTAMP_BYTES]
628            .copy_from_slice(&u64::to_ne_bytes(timestamp.unwrap_or(0)));
629
630        // For each process, try to place the new frame (with header) into its
631        // allowed streaming process slice. If at least one frame is contained
632        // in the slice, generate an upcall, but only if one is not already
633        // scheduled.
634        self.apps.iter().for_each(|process_grant| {
635            process_grant.enter(|grant, kernel_data| {
636                let rx_frames_buffer =
637                    match kernel_data.get_readwrite_processbuffer(rw_allow::RX_FRAMES) {
638                        Ok(buf) => buf,
639                        Err(_) => return,
640                    };
641                let _ = rx_frames_buffer.mut_enter(|rx_frames_slice| {
642                    let rx_frames_streaming = StreamingProcessSlice::new(rx_frames_slice);
643
644                    // The process has allowed a streaming process slice for
645                    // incoming frames, attempt to place the frame and its
646                    // header into the buffer:
647                    let append_res = rx_frames_streaming
648                        .append_chunk_from_iter(frame_header.iter().chain(frame.iter()).copied());
649
650                    // We only schedule an upcall if we were able to append this
651                    // chunk without error, and it was the first chunk we
652                    // appended to this buffer (as other upcalls will be
653                    // redundant -- the process will be able to observe
654                    // subsequent frames in the buffer). In any case, we
655                    // increment the per-process counters:
656                    let first_chunk = match append_res {
657                        Err(_) => {
658                            // We weren't able to append the chunk, increment
659                            // counters:
660                            grant.rx_frames_dropped = grant.rx_frames_dropped.wrapping_add(1);
661                            grant.rx_bytes_dropped =
662                                grant.rx_bytes_dropped.wrapping_add(len_u16 as u32);
663                            return;
664                        }
665                        Ok((first_chunk, _)) => {
666                            // We successfully appended this chunk:
667                            grant.rx_frames = grant.rx_frames.wrapping_add(1);
668                            grant.rx_bytes = grant.rx_bytes.wrapping_add(len_u16 as u32);
669                            first_chunk
670                        }
671                    };
672
673                    // Schedule an upcall if this is the first non-zero length
674                    // chunk appended to this slice:
675                    if first_chunk {
676                        let _ = kernel_data.schedule_upcall(upcall::RX_FRAME, (0, 0, 0));
677                    }
678                });
679            });
680        });
681    }
682}