Home
|
QualiCrop is a research project led by the University of Lincoln that investigates the application of hyperspectral imaging and artificial intelligence for automated tomato quality inspection. The project bridges laboratory hyperspectral imaging research with practical industrial multispectral imaging by combining laboratory image acquisition, dataset development, manual annotation, adaptive spectral band selection, deep learning and industrial deployment within a unified research framework. |
|
Figure 1. Overall QualiCrop workflow illustrating laboratory hyperspectral image acquisition, dataset development, manual annotation, adaptive spectral band selection, deep learning model development and deployment of compact multispectral imaging systems for industrial tomato quality inspection.
QualiCrop investigates how laboratory hyperspectral imaging can be translated into practical multispectral imaging systems for automated agricultural quality inspection. While hyperspectral cameras provide hundreds of contiguous spectral bands for every image pixel, their cost, acquisition speed and computational requirements make them challenging to deploy in industrial environments. The overall objective of the project is therefore to determine compact multispectral configurations capable of achieving hyperspectral-level performance using only a small number of carefully selected wavelengths.
The project spans the complete research pipeline, beginning with laboratory hyperspectral image acquisition and continuing through dataset development, semantic annotation, adaptive spectral band selection, deep learning model development and experimental validation. The resulting spectral configurations are intended to support future industrial multispectral imaging systems for high-throughput tomato quality inspection.
Research activities within QualiCrop include:
- Development of laboratory hyperspectral imaging systems.
- Large-scale hyperspectral tomato image acquisition.
- Manual pixel-level annotation and dataset curation.
- Adaptive spectral band selection.
- Semantic multiclass segmentation.
- Pixel-wise anomaly localisation.
- Instance-wise anomaly detection.
- Development of 1D and 3D residual neural network architectures.
- Evaluation of compact multispectral imaging systems.
Current research focuses on both classification-aware and reconstruction-aware spectral band selection strategies, allowing compact multispectral camera designs to be learned directly from hyperspectral data.
The QualiCrop dataset has been developed at the University of Lincoln to support research in hyperspectral imaging, semantic segmentation, anomaly detection and adaptive spectral band selection.
The primary vine tomato dataset consists of 294 tomatoes collected during Sessions 1, 2 and 4 using a laboratory hyperspectral imaging system. The dataset contains both healthy and anomalous tomatoes, including naturally occurring defects together with manually induced anomalies. Separate supervised, unsupervised and independent evaluation subsets were prepared to support multiple machine learning tasks.
Subsequent acquisition campaigns (Sessions 5 and 6) expanded the dataset to additional tomato varieties, suppliers and acquisition conditions. These sessions added approximately 450 tomatoes, bringing the complete QualiCrop collection to approximately 750 tomatoes across all acquisition campaigns.
| Category | Sub-category | Samples | Description |
|---|---|---|---|
| Acquisition | Session 1 | 102 | Initial laboratory acquisition |
| Session 2 | 57 | Second acquisition campaign | |
| Session 4 | 135 | Extended acquisition campaign | |
| Training / Test | Unsupervised Training | 102 | Representation learning |
| Supervised Training | 84 | Supervised model training | |
| Test Dataset 1 | 80 | Independent evaluation | |
| Test Dataset 2 | 98 | Independent evaluation | |
| Tomato Health | Healthy | 163 | Healthy tomatoes |
| Anomalous | 131 | Tomatoes containing visible defects | |
| Anomaly Type | Natural | 95 | Naturally occurring anomalies |
| Artificial | 36 | Manually introduced anomalies |
Note: The supervised, unsupervised and evaluation datasets represent experimental subsets prepared for different learning tasks and therefore should not be interpreted as mutually exclusive sample counts.
| Attribute | Session 5 | Session 6 |
|---|---|---|
| Source | APS Growers, Tesco | APS Growers, British & Moroccan suppliers |
| Acquisition Date | 08 May 2026 | 18 June 2026 |
| Harvest Date | — | British: 16 June 2026 Moroccan: 27 May 2026 |
| Tomato Variety | Piccolo, Ronvine NL | Salad Tomatoes |
| Condition | Healthy | Healthy & Anomalous |
| Images Acquired | 46 (Top & Bottom) | 126 (Top & Bottom) |
| Approximate Tomatoes | 220 | 233 |
Together, Sessions 5 and 6 contributed approximately 450 tomatoes, increasing the complete QualiCrop collection to approximately 750 tomatoes spanning multiple suppliers, growing conditions and acquisition campaigns.
High-quality ground truth annotations are fundamental to the QualiCrop project. Following hyperspectral image acquisition, RGB visualisations are generated from each hyperspectral cube to facilitate manual annotation. Images are then annotated using the University of Lincoln CVAT Server, where trained annotators produce detailed pixel-level semantic masks for healthy tomato tissue and multiple anomaly categories.
The annotation process follows a rigorous quality assurance procedure to ensure consistency across acquisition sessions. These annotations provide the ground truth used throughout the project for semantic segmentation, anomaly localisation, multiclass classification and adaptive spectral band selection research.
Current semantic classes include healthy tomato tissue together with multiple surface anomaly categories, vegetation and specular reflections. The annotation framework is continually refined as new acquisition sessions and research objectives are incorporated into the project.
Figure 2. Example of pixel-level semantic annotation performed using the University of Lincoln CVAT server.
One of the principal objectives of QualiCrop is to determine how laboratory hyperspectral imaging can be translated into practical multispectral imaging systems. A hyperspectral camera records hundreds of contiguous wavelength bands for every image pixel, providing detailed spectral information that enables accurate crop inspection. However, capturing the full hyperspectral spectrum is computationally expensive and often unnecessary for many vision tasks.
Our research therefore investigates adaptive spectral band selection, where the most informative wavelengths are learned automatically from the training data. Rather than manually selecting wavelengths or applying conventional feature-ranking techniques, QualiCrop learns compact multispectral camera configurations that retain the most discriminative spectral information while dramatically reducing the number of measured bands.
Two complementary optimisation strategies are currently investigated.
Classification-aware band selection directly learns spectral filters using annotated training data. During optimisation, the selected wavelengths are adjusted to maximise the performance of the downstream classification network. As a result, the learned spectral bands become highly discriminative for distinguishing healthy tissue, defects and other semantic classes.
Reconstruction-aware band selection does not require manual annotations. Instead, an autoencoder learns a compact spectral representation capable of reconstructing the original hyperspectral signal. The selected wavelengths therefore preserve the maximum amount of spectral information while using only a small number of multispectral bands.
After learning the spectral configuration, the selected bands are evaluated using two different neural network architectures.
-
1D ResNet, which analyses each pixel spectrum independently and learns purely spectral features.
-
3D ResNet, which analyses small spatial–spectral image patches, allowing both spatial texture and spectral information to be learned simultaneously.
Evaluating both architectures enables us to investigate how well the learned spectral configurations generalise across fundamentally different feature learning strategies.
The figure below presents examples of the Gaussian spectral filters learned by the proposed adaptive band selection framework. Each coloured region represents the bandwidth (FWHM) of an individual multispectral filter, while the dashed lines indicate the corresponding centre wavelengths. Together, these filters define the compact multispectral camera configuration used for downstream computer vision tasks.
Figure 3. Examples of learned Gaussian spectral filters. The upper row illustrates classification-aware optimisation, while the lower row shows reconstruction-aware optimisation.
The learned spectral configurations are evaluated using multiple computer vision tasks to investigate how well compact multispectral imaging can reproduce the performance of the original hyperspectral data.
The first evaluation investigates pixel-level multiclass semantic segmentation. The figure below compares the RGB visualisation, manually annotated ground truth and predictions generated using reconstruction-aware and classification-aware spectral band selection with the 1D ResNet architecture.
Figure 4. Representative multiclass semantic segmentation results obtained using reconstruction-aware and classification-aware spectral band selection with the 1D ResNet architecture.
The second experiment evaluates pixel-wise anomaly localisation using the selected spectral bands together with the 3D ResNet model. The results demonstrate that compact multispectral configurations remain capable of accurately identifying anomalous tomato regions despite using only a small subset of the original hyperspectral spectrum.
Figure 5. Example anomaly localisation results obtained using the 3D ResNet architecture for both reconstruction-aware and classification-aware spectral band selection.
To evaluate whether the learned spectral configurations generalise across different network architectures, the same selected wavelengths are also evaluated using a 1D ResNet, which operates directly on individual pixel spectra without explicitly modelling neighbouring spatial information.
Figure 6. Representative anomaly localisation results obtained using the 1D ResNet architecture with reconstruction-aware and classification-aware spectral band selection.
The QualiCrop project currently supports research across multiple areas of hyperspectral imaging and agricultural artificial intelligence, including:
- Hyperspectral imaging
- Multispectral imaging
- Laboratory imaging systems
- Dataset acquisition and curation
- Pixel-level semantic annotation
- Adaptive spectral band selection
- Gaussian spectral filter learning
- Semantic segmentation
- Multiclass classification
- Pixel-wise anomaly localisation
- Instance-wise anomaly detection
- Deep learning
- 1D residual neural networks
- 3D residual neural networks
- Explainable spectral learning
- Industrial computer vision
- Precision agriculture
The dataset and associated software continue to evolve as new acquisition sessions, tomato varieties, annotation tasks and learning algorithms are incorporated into the project.
The repository is organised into the following components:
- Data – Dataset documentation and acquisition sessions.
- Annotation – Annotation protocols and semantic classes.
- Band Selection – Adaptive spectral band selection methods.
- Models – 1D and 3D residual neural network architectures.
- Training – Training pipelines and experiment configurations.
- Evaluation – Quantitative and qualitative model assessment.
- Publications – Papers, reports and supplementary material.
If you use the QualiCrop dataset or associated software in your research, please cite the accompanying publication(s) once available.
University of Lincoln
QualiCrop Research Project