Scratch orgs get destroyed and rebuilt on every pipeline run, so any test data habit built around a long-lived sandbox simply does not carry over. Scratch org test data generation has to happen fresh, from schema, on every single build, with no admin sitting there clicking through the setup menu. That is a fundamentally different problem than refreshing a UAT sandbox once a quarter, and treating it the same way is how CI pipelines end up stalled on a data step nobody budgeted time for.
Scratch Org Test Data Generation vs Sandbox Refresh Habits
Most admins learned data seeding on sandboxes that live for months. You refresh a full-copy or partial sandbox, maybe run a script once, then leave the data alone until the next refresh cycle. Nobody re-seeds it every morning.
Scratch orgs invert that timeline completely. A developer spins one up for a feature branch, a CI job spins up dozens per day for parallel test suites, and each instance starts with zero records. There is no data to inherit and no shortcut around generating it from nothing, every time.
This changes what "good enough" data generation looks like. A process that takes twenty minutes and needs a human to babysit the job status is fine for a quarterly sandbox refresh. It is not fine when it is supposed to run inside a build pipeline with a wall clock everyone is watching.
The Scratch Org Lifecycle Problem: Data That Has to Rebuild Itself
A scratch org's whole reason for existing is that it is disposable. It gets created from a scratch-org definition file, source gets pushed into it, tests run, and then it gets deleted, sometimes within the hour. Some teams keep them alive for a few days at most.
That lifecycle means data seeding cannot be a manual step tucked between org creation and test execution. It has to be a scripted stage in the pipeline itself, triggered automatically right after metadata deployment and before any test class or UI test suite kicks off.
It also means the seed process needs to be idempotent and fast enough to not become the bottleneck. If your org creation and metadata push take four minutes and your data seeding takes twenty-five, you have built a pipeline that nobody wants to run more than twice a day. That defeats the entire point of scratch orgs, which is fast, frequent, isolated testing.
Schema Drift Between Scratch Org Definitions and Production
Scratch org definition files describe org shape at creation time: features enabled, edition, settings. What they do not guarantee is that the metadata being pushed into that org matches whatever schema snapshot someone used to build a data seed script six months ago.
Custom fields get added. Validation rules get tightened. A picklist that used to have four values now has seven, three of which are now required for certain record types. If your seed script was hardcoded against an older schema, it either fails outright or, worse, succeeds while inserting data that no longer represents anything close to production behavior.
This is exactly why data generation for scratch orgs has to read the org's live schema after the metadata push, not before it and not from a stale export. SproutEzee's approach is to query the object and field metadata directly from the freshly deployed org, then build records against whatever exists right now, including brand-new required fields, updated picklist value sets, and any validation rules that just landed in that branch.
Static fixtures cannot do that. A live schema read can, and it is the difference between a seed step that quietly breaks every time someone ships a metadata change and one that just keeps working.
Bulk API v2 in a CI/CD Pipeline: Speed Constraints That Matter
Pipeline time budgets are unforgiving. A feature-branch build that takes fifteen minutes gets tolerated. One that takes forty-five gets skipped, deferred, or quietly disabled by a developer who just wants to merge.
Bulk API v2 fits this constraint because it handles batching and parallelization without the manual chunking that older Bulk API jobs required. You submit a job, the platform slices it into batches server-side, and you poll for completion instead of managing batch boundaries yourself.
Inside a CI job, that translates to a predictable command: submit records, poll status, exit with a code the pipeline can act on. No custom retry logic for hitting row limits mid-batch, no separate script for splitting a 50,000-row load into chunks by hand.
The practical effect is that a scratch org can go from empty to populated with tens of thousands of records in a couple of minutes rather than the better part of an hour. For a pipeline running multiple scratch orgs a day, that gap is the difference between developers trusting the automation and developers routing around it.
Designing a Repeatable Seed Script
A seed step that works once by luck is not a seed step. It needs to survive being run by a different developer, on a different branch, against a slightly different org shape, without anyone touching the configuration.
A few things matter more than they look like they should:
| Consideration | Why it matters in a scratch org context |
|---|---|
| Object load order | Parent records must exist before dependent lookups and master-detail children are inserted, or the job fails outright |
| Volume tiers | Unit tests need a handful of rows; UI regression suites testing pagination or reports need thousands |
| Idempotency | Re-running the same seed script against the same fresh org should produce the same result, not duplicate or conflicting data |
| Teardown behavior | Since the org gets deleted anyway, cleanup logic should be minimal, not a separate maintenance burden |
| Config as code | Seed rules should live in the repo alongside the metadata they depend on, versioned the same way |
The load order row deserves a second look. A scratch org spun up for a feature branch that adds a new custom object will have zero existing parent records to reference. If the seed script assumes an account already exists because that is how it worked in the old sandbox, it will fail on the first run and nobody will immediately know why.
Where Manual Data Fixtures Fall Apart at Scale
Plenty of teams check a folder of CSV fixtures into their repo and call it a data strategy. It works fine right up until someone adds a required field to Opportunity, and every single fixture file across every test suite starts throwing validation errors that have nothing to do with the code change that triggered the build.
I would call that a false negative problem, and it is more common than most engineering leads want to admit. A developer sees a failed build, assumes their code broke something, spends an hour debugging, and eventually finds out the actual issue is a two-year-old CSV file that never got updated when the schema changed.
Schema-aware generation removes that failure mode by construction. There is no fixture file to fall out of sync because there is no fixture file. The data gets built fresh against whatever schema the scratch org actually has at that moment, every time the pipeline runs.
Scratch orgs were built to make testing faster and more isolated. Test data generation that still depends on static exports or manual seeding undoes most of that benefit before the first test even runs.
Frequently Asked Questions
What is different about generating test data for scratch orgs compared to sandboxes?
Sandboxes persist for weeks or months, so data is seeded occasionally and left in place. Scratch orgs are destroyed and recreated constantly, sometimes multiple times a day per developer, so the data has to be generated fresh every single time with no manual steps involved. This makes automation and speed non-negotiable in a way they are not for a quarterly sandbox refresh.
How much test data does a scratch org actually need?
It depends entirely on the test suite running against it. Unit tests usually need only a handful of records per object to satisfy assertions, while UI regression tests covering pagination, list views, or reports need volume in the thousands to behave realistically. A good seed process supports different volume tiers rather than forcing every scratch org to load the same fixed dataset.
Can scratch org data generation run automatically inside a CI/CD pipeline?
Yes, and it needs to for scratch orgs to be practical at scale. The seed step should trigger automatically right after metadata deployment, using Bulk API v2 to load records quickly, with the pipeline polling job status and exiting based on success or failure. No developer should have to manually kick off data loading before tests can run.
Does scratch org test data need to be masked or anonymized like production clones?
No. Scratch orgs never contain a copy of production data since they start completely empty, so there is nothing to mask. Generating synthetic, schema-compliant records from scratch avoids the masking step entirely, which is one advantage scratch orgs have over full-copy sandbox refreshes.
What happens if the scratch org's schema falls out of sync with an older seed script?
The seed job typically fails on a required field that did not exist when the script was written, or it succeeds while inserting data that no longer reflects real validation rules. The fix is to have the data generation process read the org's live metadata after deployment rather than relying on a static schema snapshot from months earlier.