capsules_extra/ethernet_tap.rs
1// Licensed under the Apache License, Version 2.0 or the MIT License.
2// SPDX-License-Identifier: Apache-2.0 OR MIT
3// Copyright Leon Schuermann <leon@is.currently.online> 2025.
4// Copyright Tock Contributors 2025.
5
6//! Userspace application TAP-like network driver
7//!
8//! This capsule provides a userspace syscall driver designed to expose a raw
9//! IEEE 802.3 (Ethernet) compatible network interface (resembling a
10//! Linux/FreeBSD TAP virtual network device).
11//!
12//! The driver multiplexes the underlying network interfaces across all
13//! applications. Applications can allow a receive buffer to the kernel, and all
14//! incoming frames will be placed into all applications' receive buffers. Every
15//! application may transmit frames, which are sent on the underlying Ethernet
16//! link.
17//!
18//! Incoming frames are sent to all processes (effectively mimicking the
19//! behavior of a MacVTap device in Virtual Ethernet Port Aggregator (VEPA)
20//! mode). This means that all processes can communicate over the Ethernet link,
21//! but processes will generally not able to talk to each other, _unless_ the
22//! connected far-end Ethernet device supports hairpin mode and is able to send
23//! frames back on the link they came from. If processes should be able to talk
24//! to each other, we can instantiate a hairpinning bridge in the Tock kernel,
25//! or use a regular bridge and provide each process its own, dedicated tap
26//! driver.
27//!
28//! ## Interface description
29//!
30//! This driver can both deliver incoming IEEE 802.3 Ethernet frames from the
31//! backing [`EthernetAdapterDatapath`] interface to userspace applications, as
32//! well as accept frames from the application for transmission over the
33//! [`EthernetAdapterDatapath`] interface.
34//!
35//! The driver is built on top of the [`StreamingProcessSlice`] abstraction,
36//! which allows for lossless streaming of data from the kernel to a userspace
37//! application. It places incoming Ethernet frames into the
38//! [`StreamingProcessSlice`], prefixed by a driver-specific header per
39//! frame. This header contains each frames' length, allowing userspace to
40//! efficiently find frame delimitors without parsing their content.
41//!
42//! The per-frame header has the following format:
43//! - Bytes 0 to 1 (`u16`, native endian): flags
44//! - Bit 0: receive timestamp valid
45//! - Bytes 2 to 3 (`u16`, native endian): frame length in bytes, excluding this header
46//! - Bytes 4 to 11 (`u64`, native endian): receive timestamp
47//! - Bytes 12 to (12 + frame length): frame contents, starting with the
48//! Ethernet header, up to and exluding the Frame Check Sequence (FCS)
49//!
50//! When one or more frames have been placed into the receive buffer, the driver
51//! will schedule an upcall to the application. The application must follow the
52//! regular [`StreamingProcessSlice`] semantics to swap the buffer for another
53//! buffer atomically, to ensure lossless reception of incoming frames.
54//!
55//! Futhermore, userspace can transmit a frame by providing a buffer and
56//! issuing a command-style system call. The interface can transmit at most a
57//! single frame at any given time. Copying the to-be transmitted frame into
58//! kernel memory is a synchronous operation, while the actual transmission may
59//! be asynchronous. Thus an upcall will be issued to the application once the
60//! transmission has commenced. Userspace must not issue a transmission while a
61//! previous one has not been acknowledged by means of an upcall.
62//!
63//! To convey further information about transmitted frames, an application can
64//! allow a "TX info" buffer into the kernel, which the driver can use to feed
65//! additional metadata back to the transmitting application, such as the frame
66//! transmission timestamp (if supported by the
67//! [`EthernetAdapterDatapath`]). The layout of this buffer is specified below.
68//!
69//! ## System call interface
70//!
71//! ### Command-type system calls
72//!
73//! Implemented through [`EthernetTapDriver::command`].
74//!
75//! - **Command system call `0`**: Check if the driver is installed.
76//!
77//! Returns the [`CommandReturn::success`] variant if the driver is installed,
78//! or [`CommandReturn::failure`] with associated ENOSUPPORT otherwise.
79//!
80//! - **Command system call `1`**: Query the interface RX statistics.
81//!
82//! Returns [`CommandReturn::success_u32_u32_u32`] with a tuple of 32-bit
83//! counters local to the process: `(rx_frames, rx_bytes,
84//! rx_frames_dropped)`. Neither `rx_frames` not `rx_bytes` include any
85//! dropped frames. The counters will wrap at `u32::MAX`.
86//!
87//! These counters are local to each process.
88//!
89//! - **Command system call `2`**: Query the interface TX statistics.
90//!
91//! Returns [`CommandReturn::success_u32_u32`] with a tuple of 32-bit counters
92//! local to the process: `(tx_frames, tx_bytes)`. The counters will wrap at
93//! `u32::MAX`.
94//!
95//! These counters are local to each process.
96//!
97//! - **Command system call `3`**: Transmit a frame located in the
98//! `frame_transmit_buffer`.
99//!
100//! Arguments:
101//! 1. transmit at most `n` bytes of the buffer (starting at `0`, limited by
102//! `MAX_MTU`, at most `u16::MAX` bytes). Supplied frames must start with
103//! the Ethernet header, and must not include the Frame Check Sequence
104//! (FCS).
105//! 2. frame transmission identifier (`u32`): identifier passed back in the
106//! frame transmission upcall (upcall `2`).
107//!
108//! Returns:
109//! - [`CommandReturn::success`] if the frame was queued for transmission
110//! - [`CommandReturn::failure`] with [`ErrorCode::BUSY`] if a frame is
111//! currently being transmitted (wait for upcall)
112//! - [`CommandReturn::failure`] with [`ErrorCode::SIZE`] if the TX buffer is
113//! to small to possibly contain the entire frame, or the passed frame
114//! exceeds the interface MTU.
115//!
116//! ## Subscribe-type system calls
117//!
118//! - **Upcall `0`**: _(currently not supported)_ Register an upcall to be
119//! informed when the driver was released by another process.
120//!
121//! - **Upcall `1`**: Register an upcall to be called when one or more frames
122//! have been placed into the receive [`StreamingProcessSlice`].
123//!
124//! - **Upcall `2`**: Register an upcall to be called when a frame transmission
125//! has been completed.
126//!
127//! Upcall arguments:
128//! 1. `statuscode` indicating 0 or ErrorCode
129//!
130//! 2. If `statuscode == 0`, flags and length (`u32`, native endian):
131//!
132//! - bits `16..=30`: flags
133//! - bit 16: TX timestamp valid
134//!
135//! - bits `0..=15`: transmitted bytes
136//!
137//! 3. frame transmission identifier (`u32`, native endian): identifier to
138//! associate an invocation of _command system call `3`_ to its
139//! corresponding upcall.
140//!
141//! ## Read-write allow type system calls:
142//!
143//! - **Read-write allow buffer `0`**: Allow a [`StreamingProcessSlice`] for
144//! received frames to be written by the driver.
145//!
146//! - **Read-write allow buffer `1`**: Allow a buffer for transmitted frame
147//! metadata to be written by the driver.
148//!
149//! The allowed buffer must be at least 8 bytes in size. Individual fields may
150//! be valid or invalid, depending on the flags of the frame transmission
151//! upcall (upcall `2`) flags.
152//!
153//! Layout:
154//! - bytes `0..=7`: frame transmission timestamp (u64; big endian)
155//!
156//! Transmission timestamp of a frame provided by the underlying
157//! [`EthernetAdapterDatapath`] implementation.
158//!
159//! ## Read-only allow type system calls:
160//!
161//! - **Read-only allow buffer `0`**: Allow a buffer containing a frame to
162//! transmit by the driver. The frame transmission must still be scheduled by
163//! issuing the appropriate command system call (*command system call `6`*).
164
165use kernel::errorcode::into_statuscode;
166use kernel::grant::{AllowRoCount, AllowRwCount, Grant, GrantKernelData, UpcallCount};
167use kernel::hil::ethernet::{EthernetAdapterDatapath, EthernetAdapterDatapathClient};
168use kernel::processbuffer::{ReadableProcessBuffer, WriteableProcessBuffer};
169use kernel::syscall::{CommandReturn, SyscallDriver};
170use kernel::utilities::cells::MapCell;
171use kernel::utilities::streaming_process_slice::StreamingProcessSlice;
172use kernel::ErrorCode;
173use kernel::ProcessId;
174
175/// Syscall driver number.
176pub const DRIVER_NUM: usize = capsules_core::driver::NUM::EthernetTap as usize;
177
178/// Maximum size of a frame which can be transmitted over the underlying
179/// [`EthernetAdapterDatapath`] device.
180///
181/// Currently hard-coded to `1522 - 4` bytes, for an Ethernet frame with an
182/// 802.1q VLAN tag with a 1500 byte payload MTU, excluding the 4-byte FCS.
183pub const MAX_MTU: usize = 1518;
184
185mod upcall {
186 pub const RX_FRAME: usize = 0;
187 pub const TX_FRAME: usize = 1;
188 pub const COUNT: u8 = 2;
189}
190
191mod ro_allow {
192 pub const TX_FRAME: usize = 0;
193 pub const COUNT: u8 = 1;
194}
195
196mod rw_allow {
197 pub const RX_FRAMES: usize = 0;
198 pub const TX_FRAME_INFO: usize = 1;
199 pub const COUNT: u8 = 2;
200}
201
202/// Receive streaming packet buffer frame header constants:
203mod rx_frame_header {
204 use core::ops::Range;
205
206 pub const LENGTH: usize = 12;
207 pub const FLAGS_BYTES: Range<usize> = 0_usize..2;
208 pub const FRAME_LENGTH_BYTES: Range<usize> = 2_usize..4;
209 pub const RECEIVE_TIMESTAMP_BYTES: Range<usize> = 4_usize..12;
210}
211
212#[derive(Default)]
213pub struct App {
214 // Per-process interface statistics:
215 tx_frames: u32,
216 tx_bytes: u32,
217
218 rx_frames: u32,
219 rx_bytes: u32,
220 rx_frames_dropped: u32,
221 rx_bytes_dropped: u32,
222
223 // Pending transmission state:
224 tx_pending: Option<(u16, u32)>,
225}
226
227enum TxState {
228 Active { process: ProcessId },
229 Inactive { buffer: &'static mut [u8] },
230}
231
232pub struct EthernetTapDriver<'a, E: EthernetAdapterDatapath<'a>> {
233 /// The underlying [`EthernetAdapterDatapath`] network device
234 iface: &'a E,
235
236 /// Per-process state
237 apps: Grant<
238 App,
239 UpcallCount<{ upcall::COUNT }>,
240 AllowRoCount<{ ro_allow::COUNT }>,
241 AllowRwCount<{ rw_allow::COUNT }>,
242 >,
243
244 /// Transmission buffer for frames to transmit from the application, or
245 /// indicator which application is currently transmitting.
246 tx_state: MapCell<TxState>,
247}
248
249impl<'a, E: EthernetAdapterDatapath<'a>> EthernetTapDriver<'a, E> {
250 pub fn new(
251 iface: &'a E,
252 grant: Grant<
253 App,
254 UpcallCount<{ upcall::COUNT }>,
255 AllowRoCount<{ ro_allow::COUNT }>,
256 AllowRwCount<{ rw_allow::COUNT }>,
257 >,
258 tx_buffer: &'static mut [u8],
259 ) -> Self {
260 EthernetTapDriver {
261 iface,
262 apps: grant,
263 tx_state: MapCell::new(TxState::Inactive { buffer: tx_buffer }),
264 }
265 }
266
267 pub fn initialize(&self) {
268 self.iface.enable_receive();
269 }
270
271 // This method must only be called when `tx_state` is currently `Inactive`
272 // and the `grant` indicates that there is a frame to transmit
273 // (`tx_pending.is_some()`).
274 fn transmit_frame_from_grant(
275 &self,
276 tx_state: &mut TxState,
277 process_id: ProcessId,
278 grant: &App,
279 kernel_data: &GrantKernelData<'_>,
280 ) -> Result<(), ErrorCode> {
281 // Mark this process as having an active transmission:
282 let prev_state = core::mem::replace(
283 tx_state,
284 TxState::Active {
285 process: process_id,
286 },
287 );
288
289 let transmit_buffer = match prev_state {
290 // This method must never be called during an active
291 // transmission. If this code-path is reached, this represents a
292 // driver-internal invariant violation.
293 TxState::Active { .. } => unreachable!(),
294 TxState::Inactive { buffer } => buffer,
295 };
296
297 // This method must never be called when a process does not have a
298 // pending transmission:
299 let (frame_len, transmission_identifier) = grant.tx_pending.unwrap();
300
301 // Try to copy the frame from the allowed slice into the transmission
302 // buffer. Return `ErrorCode::SIZE` if the `frame_len` exceeds the
303 // allowed slice, the interface MTU, or the transmission buffer:
304 let res = kernel_data
305 .get_readonly_processbuffer(ro_allow::TX_FRAME)
306 .and_then(|tx_frame| {
307 tx_frame.enter(|data| {
308 if frame_len as usize
309 > core::cmp::min(data.len(), core::cmp::min(MAX_MTU, transmit_buffer.len()))
310 {
311 return Err(ErrorCode::SIZE);
312 }
313
314 // `frame_len` fits into source slice, destination buffer, and
315 // MTU, copy it:
316 data[..(frame_len as usize)]
317 .copy_to_slice(&mut transmit_buffer[..(frame_len as usize)]);
318
319 Ok(())
320 })
321 })
322 .unwrap_or(Err(ErrorCode::FAIL));
323
324 if let Err(e) = res {
325 // We were unable to copy the frame, put the buffer back. The caller
326 // is responsible for informing the process (via a sychronous return
327 // value or upcall) and to reset the process' `tx_pending` field:
328 *tx_state = TxState::Inactive {
329 buffer: transmit_buffer,
330 };
331 return Err(e);
332 }
333
334 // Frame was copied, initiate transmission:
335 if let Err((e, returned_buffer)) =
336 self.iface
337 .transmit_frame(transmit_buffer, frame_len, transmission_identifier as usize)
338 {
339 // We were unable to initiate the transmission put the buffer
340 // back. The caller is responsible for informing the process (via a
341 // sychronous return value or upcall) and to reset the process'
342 // `tx_pending` field:
343 *tx_state = TxState::Inactive {
344 buffer: returned_buffer,
345 };
346 return Err(e);
347 }
348
349 Ok(())
350 }
351
352 // Attempt to complete a transmission requested by a process.
353 //
354 // If this process is still alive, it will reset its `tx_pending` state and
355 // enqueue an upcall. Either case, it will return the `frame_buffer` and
356 // reset the `tx_state` to `Inactive`.
357 fn complete_transmission(
358 &self,
359 err: Result<(), ErrorCode>,
360 frame_buffer: &'static mut [u8],
361 len: u16,
362 transmission_identifier: u32,
363 timestamp: Option<u64>,
364 ) {
365 let process_id = self
366 .tx_state
367 .map(|tx_state| {
368 // First, replace the `frame_buffer` and reset the `tx_state` to
369 // `Inactive`. Also extract the currently transmitting process from
370 // the `tx_state`, which must be `Active` when entering into this
371 // callback:
372 match core::mem::replace(
373 tx_state,
374 TxState::Inactive {
375 buffer: frame_buffer,
376 },
377 ) {
378 TxState::Active { process } => process,
379 TxState::Inactive { .. } => panic!(),
380 }
381 })
382 .unwrap();
383
384 // Now, attempt to complete this process' transmission, if its still
385 // alive and has subscribed to the transmission upcall / allowed a TX
386 // info buffer respectively:
387 let _ = self.apps.enter(process_id, |grant, kernel_data| {
388 // If the application has allowed a "TX info" buffer, write some
389 // transmission metadata to it:
390 let ts_bytes = timestamp.map_or(u64::to_be_bytes(0), |ts| u64::to_be_bytes(ts));
391 let _ = kernel_data
392 .get_readwrite_processbuffer(rw_allow::TX_FRAME_INFO)
393 .and_then(|tx_frame_info| {
394 tx_frame_info.mut_enter(|tx_info_buf| {
395 if tx_info_buf.len() >= 8 {
396 tx_info_buf[0..8].copy_from_slice(&ts_bytes);
397 }
398 })
399 });
400
401 // Encode combined flags / length upcall parameter:
402 let flags_len: u32 = {
403 let flags_bytes = u16::to_be_bytes((timestamp.is_some() as u16) << 0);
404
405 let len_bytes = u16::to_be_bytes(len);
406
407 u32::from_be_bytes([flags_bytes[0], flags_bytes[1], len_bytes[0], len_bytes[1]])
408 };
409
410 kernel_data
411 .schedule_upcall(
412 upcall::TX_FRAME,
413 (
414 into_statuscode(err),
415 flags_len as usize,
416 transmission_identifier as usize,
417 ),
418 )
419 .ok();
420
421 // Reset the `tx_pending` state of this app:
422 grant.tx_pending = None;
423 });
424 }
425
426 pub fn command_transmit_frame(
427 &self,
428 process_id: ProcessId,
429 len: u16,
430 transmission_identifier: u32,
431 ) -> Result<(), ErrorCode> {
432 self.apps
433 .enter(process_id, |grant, kernel_data| {
434 // Make sure that this process does not have another pending
435 // transmission:
436 if grant.tx_pending.is_some() {
437 return Err(ErrorCode::BUSY);
438 }
439
440 // Copy this command call's argument into the process' grant:
441 grant.tx_pending = Some((len, transmission_identifier));
442
443 // If we don't have an active transmission, try to enqueue this
444 // frame for synchronous transmission:
445 self.tx_state
446 .map(|tx_state| {
447 if matches!(tx_state, TxState::Inactive { .. }) {
448 if let Err(e) = self.transmit_frame_from_grant(
449 tx_state,
450 process_id,
451 grant,
452 kernel_data,
453 ) {
454 // Reset the `tx_pending` field:
455 grant.tx_pending = None;
456
457 // Return the error:
458 Err(e)
459 } else {
460 // Transmission initiated:
461 Ok(())
462 }
463 } else {
464 // Transmission is enqueued:
465 Ok(())
466 }
467 })
468 .unwrap()
469 })
470 .unwrap_or(Err(ErrorCode::FAIL))
471 }
472}
473
474/// Userspace system call driver interface implementation
475impl<'a, E: EthernetAdapterDatapath<'a>> SyscallDriver for EthernetTapDriver<'a, E> {
476 fn command(
477 &self,
478 command_num: usize,
479 arg1: usize,
480 arg2: usize,
481 process_id: ProcessId,
482 ) -> CommandReturn {
483 match command_num {
484 // Check if driver is installed
485 0 => CommandReturn::success(),
486
487 // Query process-specific RX stats
488 1 => self
489 .apps
490 .enter(process_id, |grant, _kernel_data| {
491 CommandReturn::success_u32_u32_u32(
492 grant.rx_frames,
493 grant.rx_bytes,
494 grant.rx_frames_dropped,
495 )
496 })
497 .unwrap_or(CommandReturn::failure(ErrorCode::FAIL)),
498
499 // Query process-specific TX stats
500 2 => self
501 .apps
502 .enter(process_id, |grant, _kernel_data| {
503 CommandReturn::success_u32_u32(grant.tx_frames, grant.tx_bytes)
504 })
505 .unwrap_or(CommandReturn::failure(ErrorCode::FAIL)),
506
507 // Transmit frame from the `allow_ro::TX_FRAME` buffer:
508 3 => match self.command_transmit_frame(process_id, arg1 as u16, arg2 as u32) {
509 Ok(()) => CommandReturn::success(),
510 Err(e) => CommandReturn::failure(e),
511 },
512
513 _ => CommandReturn::failure(ErrorCode::NOSUPPORT),
514 }
515 }
516
517 fn allocate_grant(&self, processid: ProcessId) -> Result<(), kernel::process::Error> {
518 self.apps.enter(processid, |_, _| {})
519 }
520}
521
522/// Callback client for the underlying [`EthernetAdapterDatapath`]:
523impl<'a, E: EthernetAdapterDatapath<'a>> EthernetAdapterDatapathClient
524 for EthernetTapDriver<'a, E>
525{
526 fn transmit_frame_done(
527 &self,
528 err: Result<(), ErrorCode>,
529 frame_buffer: &'static mut [u8],
530 len: u16,
531 transmission_identifier: usize,
532 timestamp: Option<u64>,
533 ) {
534 // Complete this transmission. This will reset both the process'
535 // `tx_pending` state, and the global `tx_state`:
536 self.complete_transmission(
537 err,
538 frame_buffer,
539 len,
540 transmission_identifier as u32,
541 timestamp,
542 );
543
544 // Now, check for any processes that have a pending transmission. We
545 // break out of the loop on the first successfully initiated
546 // transmission, as we only have one transmit buffer for this capsule:
547 self.tx_state
548 .map(|tx_state| {
549 for process_grant in self.apps.iter() {
550 let process_id = process_grant.processid();
551 let transmission_initiated = process_grant.enter(|grant, kernel_data| {
552 let (_, transmission_identifier) = match grant.tx_pending {
553 None => {
554 // Skip, this process does not have a pending transmission:
555 return false;
556 }
557 Some(tx_pending) => tx_pending,
558 };
559
560 // This process does have a pending transmission, submit it:
561 if let Err(e) =
562 self.transmit_frame_from_grant(tx_state, process_id, grant, kernel_data)
563 {
564 // Initializing the transmission failed. We must
565 // report this error through a callback, as no other
566 // callback will be raised for this transmission:
567 let _ = kernel_data.schedule_upcall(
568 upcall::TX_FRAME,
569 (
570 e as usize,
571 0, // flags are ignored in case of an error
572 transmission_identifier as usize,
573 ),
574 );
575
576 // Remove the pending transmission from this process:
577 grant.tx_pending = None;
578
579 // This error may transient and be due to the
580 // particular enqueued transmission of this process
581 // (like exceeding the MTU). Try scheduling a
582 // transmission for the next pending process in the
583 // next loop iteration:
584 false
585 } else {
586 // Transmission initiated successfully:
587 true
588 }
589 });
590
591 if transmission_initiated {
592 // Can only initate one transmission:
593 break;
594 }
595
596 // Otherwise, continue:
597 }
598 })
599 .unwrap();
600 }
601
602 fn received_frame(&self, frame: &[u8], timestamp: Option<u64>) {
603 // Generate a header to prefix the frame contents placed in the
604 // processes' `TX_FRAMES` [`StreamingProcessSlice`]:
605 let mut frame_header = [0; rx_frame_header::LENGTH];
606
607 // frame_header[0..2]: flags
608 frame_header[rx_frame_header::FLAGS_BYTES]
609 .copy_from_slice(&u16::to_ne_bytes((timestamp.is_some() as u16) << 0));
610
611 let len_u16: u16 = match frame.len().try_into() {
612 Ok(len) => len,
613 Err(_) => {
614 kernel::debug!(
615 "Incoming frame exceeds {} bytes ({} bytes), discarding.",
616 u16::MAX,
617 frame.len()
618 );
619 return;
620 }
621 };
622 // frame_header[2..4]: length (excluding frame_header)
623 frame_header[rx_frame_header::FRAME_LENGTH_BYTES]
624 .copy_from_slice(&u16::to_ne_bytes(len_u16));
625
626 // frame_header[4..12]: timestamp
627 frame_header[rx_frame_header::RECEIVE_TIMESTAMP_BYTES]
628 .copy_from_slice(&u64::to_ne_bytes(timestamp.unwrap_or(0)));
629
630 // For each process, try to place the new frame (with header) into its
631 // allowed streaming process slice. If at least one frame is contained
632 // in the slice, generate an upcall, but only if one is not already
633 // scheduled.
634 self.apps.iter().for_each(|process_grant| {
635 process_grant.enter(|grant, kernel_data| {
636 let rx_frames_buffer =
637 match kernel_data.get_readwrite_processbuffer(rw_allow::RX_FRAMES) {
638 Ok(buf) => buf,
639 Err(_) => return,
640 };
641 let _ = rx_frames_buffer.mut_enter(|rx_frames_slice| {
642 let rx_frames_streaming = StreamingProcessSlice::new(rx_frames_slice);
643
644 // The process has allowed a streaming process slice for
645 // incoming frames, attempt to place the frame and its
646 // header into the buffer:
647 let append_res = rx_frames_streaming
648 .append_chunk_from_iter(frame_header.iter().chain(frame.iter()).copied());
649
650 // We only schedule an upcall if we were able to append this
651 // chunk without error, and it was the first chunk we
652 // appended to this buffer (as other upcalls will be
653 // redundant -- the process will be able to observe
654 // subsequent frames in the buffer). In any case, we
655 // increment the per-process counters:
656 let first_chunk = match append_res {
657 Err(_) => {
658 // We weren't able to append the chunk, increment
659 // counters:
660 grant.rx_frames_dropped = grant.rx_frames_dropped.wrapping_add(1);
661 grant.rx_bytes_dropped =
662 grant.rx_bytes_dropped.wrapping_add(len_u16 as u32);
663 return;
664 }
665 Ok((first_chunk, _)) => {
666 // We successfully appended this chunk:
667 grant.rx_frames = grant.rx_frames.wrapping_add(1);
668 grant.rx_bytes = grant.rx_bytes.wrapping_add(len_u16 as u32);
669 first_chunk
670 }
671 };
672
673 // Schedule an upcall if this is the first non-zero length
674 // chunk appended to this slice:
675 if first_chunk {
676 let _ = kernel_data.schedule_upcall(upcall::RX_FRAME, (0, 0, 0));
677 }
678 });
679 });
680 });
681 }
682}