Salesforce caps daily API requests, batch sizes, and concurrent jobs long before most admins expect to hit those ceilings with a routine test data refresh. Bulk API v2 removes the manual batch-splitting logic that made v1 loads fragile, but it does not remove the platform limits underneath it. It just hides the mechanics until a job stalls at 4am with an opaque error code and a partially loaded object. Knowing how v2 chunks records, retries failures, and reports results separates a data generation process you trust from one you babysit every night.
This matters more for test data than for production integrations. Production loads usually move a known, bounded dataset on a schedule someone tested once. Test data generation runs constantly, against schemas that change every sprint, often creating tens of thousands of records across a dozen related objects in a single job. That volume and variability is exactly where Bulk API v2 test data workflows either hold up or quietly drop records nobody notices until QA blames a bad field mapping.
What Bulk API v2 Changes About Loading Test Data
Bulk API v1 required the calling code to split records into batches of up to 10,000 rows and submit each batch separately, tracking batch IDs and polling each one. Get the batch size wrong, or submit batches faster than Salesforce processes them, and you would see queued jobs stack up or time out. Most homegrown test data scripts got this wrong at least once.
Bulk API v2 collapses that responsibility. You submit one job with the entire CSV or record stream, and Salesforce handles internal chunking, batch sequencing, and parallelization on the server side. That is a real improvement for anyone who used to hand-tune batch sizes for different objects.
The catch is that server-side chunking is still bound by daily bulk API limits, per-job row limits, and concurrent job limits tied to your org edition. A tool that ignores those ceilings will still fail. It just fails with less visibility into why, because the chunking logic is no longer in your code where you could inspect it.
Batch Size, Chunking, and Where Limits Actually Bite
Three limits cause most Bulk API v2 test data failures in practice. First, the 150 million records per rolling 24-hour period across all bulk jobs in an org, which sounds generous until a nightly refresh across fifteen objects with related child records starts multiplying row counts fast. Second, the concurrent batch limit, which throttles how many jobs process in parallel and can queue your test data job behind other automation running in the same org. Third, per-field character limits inside the CSV payload itself, which reject rows silently unless you inspect job-level error files.
Custom objects with long text areas or rich text fields are the usual culprit for the third failure mode. A synthetic data generator that fills a long text area with 5,000 characters of placeholder content can blow past the field's actual limit if the schema was tightened after the generator's field-length assumptions were set. The job does not fail outright. It fails on a subset of rows, and those failures land in a results file most people never open.
Chunking also interacts badly with parent-child load order. If child records reference parent IDs that have not committed yet because the parent batch is still processing asynchronously, Bulk API v2 will reject those child rows with a foreign key error, even though the parent job eventually succeeds. This is a sequencing problem, not a data quality problem, and it is one of the more common ways test data loads look correct on the surface but leave orphaned child records behind.
Bulk API v1 vs v2: The Limits That Matter for Test Data
The table below covers the differences that actually change how you should structure a test data load, not every technical distinction between the two APIs.
| Factor | Bulk API v1 | Bulk API v2 |
|---|---|---|
| Batch splitting | Manual, caller-controlled | Automatic, server-controlled |
| Batch size control | Explicit (up to 10,000 rows) | Not exposed; Salesforce chunks internally |
| Job monitoring | Per-batch polling required | Single job-level status with per-record results |
| Error visibility | Per-batch error CSV | Consolidated failed-records file per job |
| Best fit for test data | Legacy scripts, tight batch control | Large, frequent, schema-varying loads |
Bulk API v2 wins on almost every axis for test data generation specifically, because test data jobs tend to be large, repetitive, and run by tools rather than hand-tuned scripts. The one place v1 still has an edge is fine-grained control over batch composition, which matters if you need to guarantee certain records land in the same transaction. For most test data seeding, that level of control is not the goal. Getting realistic, referentially sound records loaded fast, is.
Reading Job Results Instead of Assuming Success
A Bulk API v2 job returning a status of JobComplete does not mean every record loaded. It means the job finished running. Salesforce reports success and failure per record inside the results, and a job can complete with 9,800 successful rows and 200 silent failures without ever surfacing an error to whoever kicked off the load.
This is where a lot of test data pipelines quietly degrade. Someone builds a generator, runs it once, confirms record counts look roughly right, and moves on. Three months later the schema adds a required field, the generator does not know about it, and every load since then has been dropping a percentage of rows with no one checking the failed-records output.
The fix is not complicated, but it is easy to skip: pull the failed-records file after every job, not just the successful-records file. Row-level error messages tell you exactly which field, which record, and which validation or limit caused the drop. Ignoring that file is the single most common reason teams distrust their own test data without knowing why.
Building a Load Strategy for Complex Object Graphs
Object load order matters as much as batch size once you are seeding related records. Parents need to commit before children reference them, and Bulk API v2 does not sequence that for you across separate jobs. A generator that fires all objects at once, hoping the platform sorts it out, will produce orphaned lookups on any schema with more than two or three levels of relationship depth.
The practical approach is a dependency-ordered load plan: read the schema, build a directed graph of lookup and master-detail relationships, and submit jobs in topological order, waiting for parent job completion before submitting dependent child jobs. This adds latency compared to firing everything in parallel, but it is the only way to guarantee referential integrity across a Bulk API v2 test data run without post-load cleanup scripts patching broken lookups.
Org complexity also affects how aggressively you can parallelize. An org with heavy record-triggered flow automation on the objects being loaded will see those flows fire per batch, which can slow job processing and, in poorly designed automation, hit CPU time limits mid-job. Test data loads into automation-heavy orgs benefit from smaller, more frequent jobs rather than one enormous job that trips governor limits deep into a batch nobody can partially roll back.
Where SproutEzee Fits Into the Bulk API v2 Pipeline
SproutEzee reads the org's actual schema, including field-level constraints, relationship structure, and validation rules, before generating a single record. That schema-first approach is what keeps generated data inside real field-length limits and avoids the silent truncation failures described earlier. The generator does not guess at what a long text area holds; it checks.
On the load side, SproutEzee submits records through Bulk API v2 in dependency order, waits on parent job completion before submitting children, and parses the failed-records file automatically rather than treating JobComplete as success. That removes the two most common causes of Bulk API v2 test data failures: sequencing errors and unread error files.
None of this replaces understanding your own org's limits. A team running dozens of managed packages with their own automation still needs to know how those packages behave under bulk load. What it does remove is the guesswork around whether a given batch strategy will survive contact with a real, messy production-cloned schema at scale.
Frequently Asked Questions
What is Bulk API v2 in Salesforce?
Bulk API v2 is Salesforce's REST-based bulk data loading interface that handles internal batching and chunking automatically, unlike Bulk API v1 which required callers to split records into batches manually. You submit an entire dataset as a single job, and Salesforce processes it in the background, reporting results at both the job and record level. It is the recommended approach for high-volume operations like test data seeding, mass updates, and large exports.
How many records can Bulk API v2 process per batch?
Bulk API v2 does not expose a caller-controlled batch size the way v1 did; Salesforce chunks the submitted dataset internally. The practical constraint that matters is the org-wide rolling limit, currently 150 million records per 24-hour period across all bulk jobs. Individual job performance is also affected by concurrent job limits and automation running against the target objects.
Why do test data loads fail with Bulk API v2 even with correct data?
A job can return JobComplete while a subset of records fail silently, since success is reported per record, not just per job. Common causes include field-length limits exceeded by generated content, sequencing errors where child records reference parent IDs that have not committed yet, and validation rules the generator did not account for. Reviewing the failed-records file after every job is the only reliable way to catch these.
Does Bulk API v2 support relationship fields like lookups during test data creation?
Yes, but it does not automatically sequence dependent jobs for you. Parent records need to be created and committed before child records referencing them are submitted, and this ordering has to be managed by whatever tool or script is orchestrating the load. Skipping this step is one of the most common causes of orphaned lookup fields in generated test data.
How does SproutEzee use Bulk API v2 differently from a manual data load script?
SproutEzee reads the org's schema first, including field constraints and relationship structure, before generating any records, which prevents field-length and validation failures at the source. It then submits jobs in dependency order through Bulk API v2, waiting for parent jobs to complete before loading children. It also parses the failed-records output automatically rather than assuming a completed job means every record loaded successfully.