Sizing a Critical Imaging Archive: A Reusable 6-Checkpoint Plan (On-Prem, Colocation and Cloud)
Archives of large files, such as scans, images, video or recordings, are often the fastest-growing data in an organization, and they are often under-sized because of a few predictable mistakes. This guide gives you a six-checkpoint plan, a copy-and-fill template and a decision guide for placing each data tier on-premises, in colocation or in the cloud.
Archives of large files, such as scans, images, video or recordings, are often the fastest-growing data in an organization, and they are often under-sized because of a few predictable mistakes. This guide gives you a six-checkpoint plan, a copy-and-fill template and a decision guide for placing each data tier on-premises, in colocation or in the cloud.
Who this is for
Who this is for
Any team that keeps large files it cannot afford to lose, for example:
- Document and scan archives
- Photo, video or media libraries
- Surveillance and inspection footage
- Engineering, design or survey imagery
- Any dataset with a long retention period and steady growth
Why archive storage plans fail
Most under-sized archives share the same three flaws:
- Retention is never multiplied out. Annual growth × retention years is the real floor, and it is often much higher than the first estimate.
- Compression is assumed, not measured. Many image and video formats are already compressed, so advertised "data reduction" ratios rarely apply.
- Raw, usable and retained capacity get confused. RAID or erasure-coding overhead, metadata, snapshots, free-space headroom and backup copies all take their share.
The checkpoints below are designed to catch each of these before you buy anything.
The 6-checkpoint plan
Checkpoint 1: Define requirements before hardware
Agree these with business, legal and IT stakeholders first:
- Retention period for each data type
- Recovery objectives per workload: RPO (how much data you can lose) and RTO (how long you can be down)
- Acceptable retrieval time by data age (instant, minutes, hours)
- Regulations on data location, cross-border transfer and industry-specific rules
Gate: a signed retention and recovery policy exists. No sizing starts without it.
Checkpoint 2: Measure your real baseline
Collect real telemetry for at least several months, not estimates:
- Files created per day, by type
- Average file size per type
- Growth rate year over year, including planned new sites, sources or higher-resolution formats
- Current compression ratio on your own files
Gate: every number in your template is labeled measured or assumed.
Checkpoint 3: Calculate with the template
Use this worksheet. Replace the example values with yours.
| Line | Item | Formula | Example |
|---|---|---|---|
| A | Annual new data | measured | 25 TB |
| B | Retention (years) | policy | 10 |
| C | Base retained data | A × B | 250 TB |
| D | Growth factor over the period | measured / planned | e.g. 1.3 |
| E | Retained data with growth | C × D | 325 TB |
| F | Overhead factor (RAID/erasure coding, metadata, snapshots) | vendor spec | e.g. 1.3 |
| G | Headroom (free space, 20–30%) | policy | e.g. 1.25 |
| H | Primary capacity needed | E × F × G | ~530 TB |
| I | Number of independent copies (3-2-1 rule) | policy | 3 |
| J | Upper-bound total footprint | H × I | ~1,590 TB |
The example factors are illustrative. Your own measurements decide the real numbers. Line J is an upper bound that assumes every copy is full-size. In practice, tiering and cheaper media for secondary copies reduce it. The point is that the final figure is often several times the first intuition.
Gate: the calculation is reviewed by someone who did not write it.
Checkpoint 4: Tier the data by access pattern
Recent files are read often. Old files are read rarely. Design for that.
| Tier | Holds | Priority | Typical placement |
|---|---|---|---|
| Hot | Recent files, active workloads | Low latency | On-prem flash or colocation |
| Warm | Files from the past few years | Balanced cost and speed | Scalable disk or object storage |
| Cold archive | Older files kept for compliance or history | Lowest cost, slower retrieval | Object storage / cloud archive |
| Immutable backup | Independent protected copies | Cannot be deleted or encrypted | Separate location, write-once |
Remember that snapshots and replication are not backups. Only an independent, protected copy that has been restore-tested counts.
Gate: each tier has an owner, a retrieval-time target and a cost estimate.
Checkpoint 5: Choose placement (on-prem, colocation or cloud)
Use these questions as a decision guide.
| Question | If yes, lean toward |
|---|---|
| Is sub-second access to recent files mandatory? | Local or colocation hot tier |
| Is capital budget limited and demand uncertain? | Colocation or cloud (pay as you grow) |
| Do regulations restrict where data may be stored? | Local or in-country options, or approved regions only |
| Do you need an off-site copy without building a second facility? | Cloud replication |
| Is ransomware resilience a priority? | Immutable (write-once) storage |
| Will bandwidth to the cloud be limited? | Keep more data local, archive in batches |
Where AWS fits in a hybrid design:
- Amazon S3 storage classes (Intelligent-Tiering and the Glacier family) for warm and cold archives
- S3 Object Lock for write-once immutable copies
- AWS DataSync and AWS Storage Gateway to move or extend on-premises storage into S3
- Cross-Region replication for a geographically separate copy
- AWS Backup to centralize backup policies across services
Always verify current service features, Region availability and retrieval times in the AWS documentation before designing around them.
Gate: a documented placement decision for each tier, including legal sign-off on data location.
Checkpoint 6: Validate, then operate
Before go-live and on a recurring schedule:
- Network: size links for the initial migration, daily ingest, restores and failure scenarios. Use QoS so bulk transfers never starve interactive business traffic.
- Restore tests: restore real files from every tier and measure the time against your RTO.
- Retrieval costs: model realistic restore volumes, since archive tiers charge for retrieval.
- Capacity reviews: compare real growth against the template at least quarterly and update it.
- Certifications: do not treat facility ratings (such as Tier III), information-security standards (such as ISO 27001) and industry-specific accreditations as interchangeable. Each covers something different, so request the actual certificate and its scope.
Gate: a successful restore drill is documented, and the quarterly review is scheduled.
Quick checklist (copy and keep)
- Retention and recovery policy signed
- Baseline measured, with assumptions clearly labeled
- Capacity template completed and independently reviewed
- Data split into tiers with owners and targets
- Placement chosen and legally approved
- Network sized for peak and failure scenarios
- Immutable copy in place
- Restore test passed
- Quarterly capacity review scheduled
Key takeaways
- Start from policy, not products. Retention and recovery objectives drive everything else.
- Measure, then calculate. Label every number as measured or assumed.
- Raw ≠ usable ≠ retained. Include overhead, headroom and copies.
- Tier by access pattern. Match cost and speed to how often data is read.
- Hybrid is a toolbox, not a rule. Use the cloud where it solves a real problem, such as off-site copies, immutability and elastic archives.
- A backup is only real once it has been restored.
What checkpoints would you add? Share them in the comments.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article