Google Summer of Code 2026

I'm

About

I am Derrick Richard, a Computer Science and Data Science student selected for Google Summer of Code 2026 with the mlverse organization. My project focuses on extending TorchVision support in R for computer vision tasks.

The core objective is to bring full-featured computer vision capabilities to the R ecosystem by implementing item transforms for detection and segmentation pipelines, adding state-of-the-art object detection models like RF-DETR, and building utility functions for rotated bounding box operations.

During the program, I contributed 21 pull requests, including item transforms for detection & segmentation pipelines, the RF-DETR object detection model, SAHI, rotated bounding box utilities, and a CUDA roi_align_rotated op — with 20 merged.

Overall, I've contributed 49 pull requests to the mlverse ecosystem across mlverse/torchvision and mlverse/torchvisionlib — spanning dataset loaders, model families, image transforms, documentation, and CUDA-level operations, with 38 merged.

My mentors and the mlverse community have been incredible. This experience has deepened my understanding of computer vision internals, R package development, and collaborative open-source engineering.

GSoC 2026

Project Overview

Extending TorchVision in R for Computer Vision Tasks

Objectives
  • Implement item transforms for detection & segmentation data augmentation pipelines
  • Implement RF-DETR object detection model using preexisting LW-DETR utilities
  • Build rotated bounding box utilities (IoU, coordinate transforms)
  • Add SAHI (Slicing Aided Hyper Inference) for small object detection
  • Implement CUDA-level operations in torchvisionlib
  • Improve reliability, documentation, and CI infrastructure
Technologies
R torch torchvision torchvisionlib C++ CUDA Rcpp roxygen2 GitHub Actions RF-DETR Object Detection SAHI

Repositories

Tensor-like toolkit for image classification, object detection, segmentation and more in R. Provides access to pre-trained models, datasets, and image/video transforms.

48 PRs contributed 77 31

Low-level C++/CUDA operations for torchvision in R. Provides compiled ops for NMS, ROI align, deformable attention, and rotated box operations.

1 PR contributed 10 5

Pull Request Timeline

All-time contributions to mlverse/torchvision and mlverse/torchvisionlib (June 2025 - August 2026)

Jun 3, 2025
mlverse/torchvision

#150 EMNIST dataset (issue #129)

Initial attempt at adding the EMNIST dataset; superseded by #152.

Closed
Jun 3, 2025
mlverse/torchvision

#151 EMNIST dataset (issue #129)

Alternative EMNIST implementation; superseded by #152.

Closed
Jun 10, 2025
mlverse/torchvision

#153 Add QMNIST Dataset (#131)

Adds the QMNIST dataset for image classification, extending the MNIST family with consistent download/caching and tests.

Merged
Jun 15, 2025
mlverse/torchvision

#152 Add EMNIST Dataset (#129)

Adds the EMNIST balanced split dataset for image classification, extending the MNIST family with consistent download/caching and tests.

Merged
Jun 15, 2025
mlverse/torchvision

#156 Add FGVC-Aircraft Dataset

Implements the FGVC-Aircraft dataset with hierarchical class annotations (variant, family, manufacturer) and comprehensive tests.

Merged
Jun 20, 2025
mlverse/torchvision

#168 Multilabel output for FGVC-Aircraft

Adds annotation_level = "all" support to FGVC-Aircraft, returning multi-level named-list annotations per image.

Merged
Jun 22, 2025
mlverse/torchvision

#154 Add FER2013 Dataset

Adds the FER2013 dataset for facial expression recognition using 48x48 grayscale images across 7 emotion categories.

Merged
Jun 24, 2025
mlverse/torchvision

#157 Add Oxford Flowers 102 Dataset

Introduces the Oxford Flowers 102 image classification dataset with consistent download/caching and test coverage.

Merged
Jun 25, 2025
mlverse/torchvision

#158 Add Caltech101 & Caltech256 Datasets

Adds both object detection datasets with flexible target_type parameters and full test suites.

Merged
Jul 2, 2025
mlverse/torchvision

#193 Polish NEWS.md using Tidyverse conventions

Reformats the package changelog to follow Tidyverse style guidelines.

Merged
Jul 3, 2025
mlverse/torchvision

#159 Add Flickr8k & Flickr30k Datasets

Adds two image captioning datasets with consistent data loaders and comprehensive test coverage.

Merged
Jul 3, 2025
mlverse/torchvision

#186 Standardize archive size display & tests

Adds archive_size fields, generalizes large-dataset test control, and cleans up existing tests.

Merged
Jul 3, 2025
mlverse/torchvision

#195 Make Flickr consistent with loaders

Aligns the Flickr caption dataset loaders with the rest of the package for consistency.

Merged
Jun 18, 2025
mlverse/torchvision

#164 Add LFW Datasets

Initial work on LFW datasets; superseded by #203.

Closed
Jul 1, 2025
mlverse/torchvision

#174 Implement cleanup in dataset tests

Cleanup work on existing dataset tests; incorporated into #186.

Closed
Jul 16, 2025
mlverse/torchvision

#194 Checks for CRAN release

Investigated CRAN release checks; no longer needed.

Closed
Jul 7, 2025
mlverse/torchvision

#162 Add Oxford-IIIT Pet Dataset

Adds the Oxford-IIIT Pet dataset supporting image classification and segmentation with multiple target types.

Merged
Jul 20, 2025
mlverse/torchvision

#202 Add VisionTransformer model family

Adds the Vision Transformer (ViT) architecture with five pretrained variants (vit_b_16, vit_b_32, vit_l_16, vit_l_32, vit_h_14).

Merged
Jul 23, 2025
mlverse/torchvision

#203 Add LFW Datasets

Adds LFW people and LFW pairs datasets for face verification and similarity tasks with comprehensive tests.

Merged
Aug 8, 2025
mlverse/torchvision

#209 Add Pascal VOC Datasets

Adds pascal_segmentation_dataset() and pascal_detection_dataset() with selective download support.

Merged
Aug 26, 2025
mlverse/torchvision

#217 Add FaceNet model family

Adds support for the FaceNet architecture for face recognition and embedding generation.

Merged
Aug 31, 2025
mlverse/torchvision

#237 Add MobileNet v3 model family

Adds model_mobilenet_v3_large() and model_mobilenet_v3_small() with pretrained weights.

Merged
Feb 1, 2026
mlverse/torchvision

#238 Add VGGFace2 Dataset

Adds support for loading the VGGFace2 dataset with proper documentation and tests.

Merged
Feb 2, 2026
mlverse/torchvision

#244 Article on model_fcn_resnet

Proposed an article on the FCN ResNet model; not merged.

Closed
Mar 28, 2026
mlverse/torchvision

#302 Fix typo and inconsistency in roxygen2 docs

Cleaned up documentation typos and inconsistencies across roxygen2 generated docs for cleaner package documentation.

Merged
Mar 28, 2026
mlverse/torchvision

#303 Add skeleton edge rendering to draw_keypoints()

Enhanced the keypoint visualization function with skeleton edge rendering for better pose estimation visualization.

Merged
Apr 2, 2026
mlverse/torchvision

#311 Polishing NEWS.md for release 0.9.0

Refined the changelog and release notes in preparation for the CRAN 0.9.0 release.

Merged
Apr 2, 2026
mlverse/torchvision

#304 Rcpp interface for element-wise array ops

Proposed an Rcpp interface for element-wise array operations; not merged.

Closed
Jun 7, 2026
mlverse/torchvision

#326 Upgrading Roxygen2 to version 8.0.0

Upgraded the roxygen2 documentation generator to v8.0.0 for improved documentation generation and newer markdown support.

Merged
Jun 16, 2026
mlverse/torchvision

#330 Add Model RF-DETR

Initial RF-DETR implementation; superseded by #333.

Closed
Jul 7, 2026
mlverse/torchvision

#329 Add SAHI (Slicing Aided Hyper Inference) Transform

Implemented the SAHI transform for small object detection, enabling sliding-window inference with automatic stitching of results.

Merged
Jul 7, 2026
mlverse/torchvision

#333 Add Model RF-DETR

Implemented the RF-DETR (Real-time Fused Detection Transformer) object detection model, leveraging preexisting LW-DETR utilities for model architecture.

Merged
Jul 7, 2026
mlverse/torchvision

#339 Bump actions/checkout from v4 to v5

Upgraded GitHub Actions checkout action to v5 in the CI pipeline for improved performance and reliability.

Merged
Jul 25, 2026
mlverse/torchvision

#342 Add transform_to_rotated_box & box_xyxy_to_xyxyr

Implemented rotated bounding box coordinate transforms and conversion utilities for rotated object detection workflows.

Merged
Jul 29, 2026
mlverse/torchvision

#366 Add item_transform_hflip

Implemented horizontal flip item transform with proper bounding box and mask adjustments for detection/segmentation pipelines.

Merged
Jul 30, 2026
mlverse/torchvision

#369 Add item_transform_vflip

Implemented vertical flip item transform for detection and segmentation data augmentation with correct bounding box updates.

Merged
Aug 1, 2026
mlverse/torchvision

#370 Add item_transform_center_crop

Implemented center crop item transform for detection and segmentation items with bounding box clipping support.

Merged
Aug 1, 2026
mlverse/torchvision

#374 Improve reliability of lfw_dataset

Fixed reliability issues in the Labeled Faces in the Wild dataset loader for more robust dataset downloading and caching.

Merged
Aug 1, 2026
mlverse/torchvision

#301 Article for Faster R-CNN

Proposed an article on Faster R-CNN; not merged.

Closed
Aug 2, 2026
mlverse/torchvisionlib

#36 Add roi_align_rotated

Implements rotated ROI Align operation in C++/CUDA for rotated object detection feature extraction in torchvisionlib.

Open
Aug 10, 2026
mlverse/torchvision

#371 Add item_transform_crop

Implemented crop item transform for detection and segmentation items, enabling precise region-of-interest extraction.

Merged
Aug 10, 2026
mlverse/torchvision

#373 Add item_transform_pad

Implemented pad item transform for detection and segmentation items with configurable padding modes and values.

Merged
Aug 10, 2026
mlverse/torchvision

#381 Add item_transform_random_horizontal_flip

Implemented randomized horizontal flip transform for detection and segmentation items with probabilistic application.

Merged
Aug 18, 2026
mlverse/torchvision

#377 Add item_transform_perspective

Implements perspective transform for detection and segmentation items with perspective distortion augmentation.

Merged
Aug 22, 2026
mlverse/torchvision

#385 Fix item_transform_rotate bounding boxes

Fixes the rotate transform to maintain tight bounding boxes after rotation, preventing box expansion artifacts.

Merged
Aug 22, 2026
mlverse/torchvision

#387 Add item_transform_random_crop

Implements random crop item transform for detection and segmentation with bounding box intersection clipping.

Merged
Aug 24, 2026
mlverse/torchvision

#389 Add item_transform_random_erasing

Implements random erasing augmentation for detection and segmentation items, randomly masking image regions for regularization.

Merged
Aug 24, 2026
mlverse/torchvision

#388 Delete downloaded archives after extraction

Cleaned up temporary archive files after dataset extraction to reduce disk usage during dataset downloads.

Closed
Aug 26, 2026
mlverse/torchvision

#393 Add item_transform_random_perspective

Implements random perspective transform for detection and segmentation items with perspective distortion augmentation.

Merged

Contribution Stats

49
Pull Requests
38
Merged
1
Open
10
Closed
2
Repositories
10
Item Transforms
4
Models Added
1
CUDA Op