YieldSense: Teaching AI When Not to Be Confident
Building an AI-powered e-waste decision system taught us that making predictions is the easy part; knowing when not to trust them is the real engineering. Here’s our journey through computer vision, uncertainty-aware ML, AWS infrastructure, and the surprisingly serious business of keeping batteries away from industrial shredders.
What happens when a model’s confident mistake could be more expensive than a bad prediction on a leaderboard? For our team, the answer involved electronic waste, computer vision, a lot of engineering trade-offs, and learning to treat uncertainty as part of the product.
Electronic waste has an identity crisis. It is simultaneously a pile of discarded gadgets, a source of recoverable materials, and—occasionally—a collection of surprises nobody wants to feed into an industrial shredder.
Inside old phones, laptops, remotes, toys, and circuit boards are metals and plastics that may be recovered and reused. But some devices also contain lithium-ion batteries. If a damaged or overlooked battery reaches the wrong processing stage, it can create a serious fire risk.
That tension became the starting point for YieldSense: an e-waste decision-support system that brings together machine learning, material estimation, route optimization, and an AWS-based application architecture.
The goal was not simply to ask, “What is in this image?” It was to help answer a harder question:
Given what we know, what we do not know, and the safety and capacity constraints we must respect, what is a defensible next step?
That question shaped the ML work I contributed to—and changed how I think about building AI systems for decisions that matter outside a notebook.
1. The problem: recover value without ignoring risk
An e-waste facility has two closely related, but distinct, challenges.
Challenge | What makes it difficult? | What YieldSense aims to support |
|---|---|---|
Safety at intake | Incoming devices may contain batteries that need to be identified and reviewed before shredding. | Visual triage, hazard flags, and a workflow that can divert suspected or uncertain lots for human review. |
Material recovery | The material composition and recoverable value of a batch are not known perfectly in advance. | Composition and mass estimates, uncertainty scenarios, and comparisons between eligible recovery routes. |
A conventional pipeline might classify an image, return a confidence score, and move on with its day. The problem is that a confidence score is not a safety certificate—and a predicted material percentage is not a physical measurement.
A model can be wrong. The question is whether the rest of the system notices, communicates that uncertainty, and handles it responsibly.
That became one of our central design principles: AI should provide evidence for a decision, not silently become the authority that approves it.
2. My part of the journey: making computer vision work with limited data
My main contribution was on the machine learning side. For whole-device intake triage, we worked with the XBAT+ benchmark and a small training split of 247 images across 15 device categories—roughly 16 images per category on average.
That is not a lot of data for a visual recognition problem. It is enough to experiment; it is not enough to assume a large model will generalize just because it has a large number of parameters.
We compared approaches, including a ResNet-18 baseline and a two-stage fine-tuning approach. The small dataset made overfitting a real concern: a model can learn accidental clues from backgrounds, lighting, and camera framing instead of the features that distinguish the device itself.
So we explored a different question: rather than asking the backbone to relearn visual representations from a few hundred images, could we extract better features from a pretrained model and train a much smaller prediction layer?
A three-scale view of each device
Our
DualHeadEfficientNet uses a pretrained EfficientNet-B0 backbone and a three-scale spatial region-of-interest pyramid. The motivation was practical: small devices can occupy only a small part of the image, while blank letterbox margins can consume a surprising amount of the frame.Instead of resizing the entire image once and hoping the details survive, the model examines three views:
- Active strip: Focus on the device-bearing region and reduce the influence of large empty borders.
- Medium crop: Preserve more context around the device.
- Tight crop: Give smaller details and surface textures a closer look.
At each scale, the network combines intermediate texture features with deeper semantic features. Concatenating these representations produces a 4,176-dimensional feature vector in the multi-scale configuration.
The model then feeds that shared representation into two separate heads:
- Device-category head: predicts one of 15 device categories.
- Battery-hazard head: estimates a battery-related hazard score.
These tasks are related, but they are not interchangeable. Recognizing a phone is a category prediction; determining that a lot needs safety review is a separate decision with different consequences.
In other words: identifying the thing and deciding what should happen to it are not the same job.
The small-data plot twist: L-BFGS instead of more fine-tuning
Rather than making deep-backbone fine-tuning the default, we froze the pretrained feature extractor, cached the multi-scale representations, and trained the smaller prediction heads using L-BFGS with L2 regularization.
L-BFGS is a quasi-Newton optimization method. For the regularized linear-head objective used here, it offers a well-structured optimization problem without requiring us to backpropagate through millions of backbone parameters for every training update.
In practical terms, the workflow became:
text
Input images↓Pretrained EfficientNet-B0↓Three-scale feature extraction↓Cache feature vectors↓Train regularized prediction heads with L-BFGS↓Calibrate and evaluate on held-out data
The repository reports that this head-training stage converged in under 35 seconds on CPU. On the reported held-out split, the multi-scale model achieved 78.02% top-1 category accuracy, compared with 57.14% for the ResNet-18 baseline and 68.13% for the two-stage unfreezing baseline.
Those results were encouraging, but context matters: the held-out split contained 91 images. They support the design choice in this experiment; they do not prove it will outperform fine-tuning on every dataset or deployment environment.
The lesson I took away was straightforward: more trainable parameters are not automatically the answer to limited data. Sometimes the better move is to reuse what a pretrained model already knows, design the representation around the problem, and keep the part being learned deliberately small.
3. The bigger question: what should the system do when it is unsure?
Classification metrics are useful, but an overall accuracy number can hide the mistakes that matter most. In a safety-sensitive workflow, missing a potential battery hazard is not equivalent to confusing two harmless device categories.
We therefore treated battery-risk detection as a separate task and added two important pieces: probability calibration and a conformal thresholding procedure.
Calibration is not the same as confidence
Neural networks produce scores that can look like probabilities, but those numbers are not automatically calibrated. A model that outputs
0.90 is not necessarily correct on 90% of comparable cases.We used post-hoc temperature scaling to improve the calibration of the category and battery outputs. For battery triage, a separate held-out calibration set was then used to determine a threshold based on hazardous examples.
Conformal risk control and human review
The implementation uses a split-conformal calibration procedure. In broad terms, calibration examples help set a decision threshold, with statistical guarantees that rely on assumptions such as calibration and future data being exchangeable.
In the documented run, the calibrated battery threshold was 0.1744. Samples whose calibrated battery score met or exceeded that threshold were flagged as
suspected , allowing the backend workflow to send them toward an appropriate quarantine or human-review path.The test split caught 39 out of 39 hazardous examples in the reported evaluation. That is a promising result for this experiment—not proof of perfect real-world safety. The held-out set is small, and deployment conditions can differ from benchmark data. More representative validation, monitoring, and operational safeguards would be required before relying on a system like this in an actual facility.
The design principle: when uncertainty is material, the system should raise a flag or abstain rather than quietly convert an uncertain prediction into permission to proceed.
This is what the title of this article is really about. We are not trying to make AI timid for the sake of it. We are trying to make it better at distinguishing a useful estimate from a decision that needs more evidence.
4. The second ML track: from whole devices to shredded particles
A device photograph and a photograph of shredded material are two very different computer vision problems. Once a device has been shredded, the task is no longer to identify a product; it is to estimate the composition of the resulting particles.
YieldSense therefore contains a second ML track for post-shredder material analysis.
Track | Input | Approach | Output |
|---|---|---|---|
Track B — intake triage | Whole-device images | Multi-scale EfficientNet features, category and battery heads, calibration | Device-category probabilities and potential battery-hazard flags |
Track A — particle analysis | Shredded-particle images | 16 engineered geometry, color-moment, and color-ratio features; Extra-Trees classification and Gradient Boosting regression | Estimated material categories and particle masses |
For Track A, the feature extractor describes properties such as projected area, aspect ratio, fill ratio, foreground color statistics, and selected color ratios. A tree ensemble classifies particle material, while a Gradient Boosting regressor estimates mass from those image features and the chosen modelling assumptions.
The pipeline also reconciles aggregate visual mass estimates against the physical weighbridge total. This matters because a two-dimensional image does not directly reveal a particle's thickness or true mass. A scale reading gives the system an independent measurement for the batch-level total.
In the repository's reported experiments, Track A achieved 84.26% particle-classification accuracy in five-fold cross-validation on the paired particle dataset. Its three mixed-batch evaluations reported a mean composition MAE of 2.76%. Those numbers describe the documented datasets and evaluation setup; they should not be interpreted as guarantees for unseen industrial batches.
I like this part of the design because it resists the temptation to use deep learning everywhere. Domain-informed features and tree ensembles can be a good fit when the data and problem structure support them. Not every task needs another neural network. Sometimes a sensible feature and a well-chosen tree are already doing useful work.
5. From a prediction to a route decision
ML predictions become more useful when downstream software knows what they mean—and what their limitations are.
YieldSense connects the inference outputs to a Go backend and a constrained route-optimization engine. The two ML tracks retain different meanings: whole-device triage is not a measurement of shredded-material composition, and a material estimate is not evidence that a hazardous lot has been cleared.
For composition estimates, the system generates alternative scenarios using prediction uncertainty and a Dirichlet-based model. More ambiguous visual predictions can produce more dispersed scenarios; less ambiguous predictions can produce more concentrated ones. These scenarios give the optimizer more than a single supposedly perfect composition vector to work with.
The optimizer can then compare eligible routes while accounting for factors such as:
- facility capacity;
- material and route eligibility;
- battery-safety and review constraints;
- expected versus more conservative outcomes; and
- operational disruptions, such as a route becoming unavailable.
YieldSense supports expected and robust optimization modes, as well as simulations for price shocks and facility shutdowns. The point is not that the optimizer can predict the future. It is that operators can explore how a decision changes when the assumptions change.
That is an important connection between the ML and optimization components: uncertainty does not stop at the model output—it becomes an input to the decision process.
6. Why the AWS architecture matters
A model that works in a notebook is a useful milestone. A system that can serve predictions, persist records, manage files, and explain its decisions is a much broader engineering challenge.
The project connects a Next.js frontend to a Go backend built with Echo. The ML inference worker runs separately in Python, while AWS services provide containerized execution and persistence.
Component | Role in YieldSense |
|---|---|
Amazon ECS with AWS Fargate | Runs the containerized backend and ML inference worker without requiring us to manage the underlying servers directly. |
Amazon DynamoDB | Stores operational records such as batches, routes, and optimization runs. |
Amazon S3 | Stores intake images, model artifacts, and exported audit reports. |
Go backend | Coordinates application workflows, validation, route optimization, and communication with the inference worker. |
Python ML worker | Serves the two ML tracks through a defined inference contract. |
Next.js frontend | Provides the interface for intake, reviewing outputs, and exploring route decisions. |
Separating the Go API from the Python inference worker keeps the application's decision logic distinct from the model-serving runtime. It also makes the interface between them explicit, which is useful when models or dependencies change independently of the backend.
The project records decision artifacts and uses SHA-256 hashes for audit-related integrity checks. A hash can help detect whether an artifact has changed, but it does not by itself guarantee that an entire system is tamper-proof; access controls, storage configuration, and operational processes still matter.
This is the less glamorous part of an ML project—and one of the most important. Eventually a model will misclassify something, an API will receive an unexpected request, or an operator will ask why a particular route was recommended. At that moment, “because the AI said so” is not an acceptable debugging strategy.
7. What I learned
1. Let the data shape the model. With a small training set, frozen pretrained features and compact prediction heads were worth investigating instead of assuming full fine-tuning was the default answer.
2. Evaluate the errors that matter. Overall accuracy is useful, but hazard recall, calibration, false negatives, and uncertain examples deserve independent attention when the cost of mistakes is uneven.
3. Uncertainty should change behavior. Displaying a confidence score is not enough. When a prediction is uncertain in a consequential way, the workflow needs a clear response: review, abstention, or another source of evidence.
4. Keep responsibilities separate. Device classification, particle analysis, safety review, and route optimization answer different questions. Explicit boundaries reduce the risk of one component's output being mistaken for another kind of evidence.
5. Build for inspection, not just demonstration. Reproducible evaluations, data provenance, versioned artifacts, defined service contracts, and audit records are part of making an ML system usable—not paperwork to add after the interesting work is over.
And one more lesson: complexity is not the same thing as reliability. The most useful model is not necessarily the largest one. It is the one whose behaviour has been evaluated appropriately and whose limitations the surrounding system can handle.
8. What comes next?
YieldSense brings together computer vision, uncertainty-aware inference, material estimation, constrained optimization, and AWS-based infrastructure to explore a practical e-waste recovery problem.
The next steps are just as important as the prototype: validate on more representative data, test under different cameras and operating conditions, monitor calibration drift, evaluate difficult hazardous examples, and test the whole workflow against realistic operational constraints.
YieldSense is a decision-support system, not an autonomous recycling plant. Its benchmark results do not establish regulatory compliance, guarantee industrial safety, or demonstrate realized financial savings. Those claims would require separate evidence from real operating conditions.
But the direction is meaningful. E-waste recovery is simultaneously a materials problem, an optimization problem, a data-quality problem, and a safety problem. Addressing only one of those dimensions would leave much of the challenge untouched.
For me, the ML work reinforced a simple idea: building useful AI is not just about persuading a model to make a prediction. It is about helping the wider system understand what that prediction does—and does not—justify.
And if a project about discarded electronics teaches you to respect calibration, uncertainty, and the humble human-review queue, then the waste was not entirely wasted.
Some lessons, however, are best learned without a lithium-ion battery demonstration.
Built by combining an uncertainty-aware ML pipeline, a constrained Go optimization backend, and AWS-based container and storage infrastructure.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article