Why is the batch size in my Salesforce bulk data load different than the batch size I set?
Scenario:
We are doing bulk data load to Salesforce and the default size which we have set is 10000 but when i look at the Salesforce the number of records are processing per batch is 112 recordsHere is the Salesforce destination configurationCan you please let me know why is it processing only 100+ records at a time when i have set batch size to 10000. Because of this to just load 20000 records it is taking around 5 minutes. Can you please help us with this data load to become faster
Solution:
The size/length of your rows exceed the default SSIS buffer size. You will need to manipulate the default DefaultBufferMaxRows and DefaultBufferSize to experiment what numbers will work for your particular data set and to be able to send a batch of 10000 rows. Also, there is a AutoAdjustBufferSize that may or may not work for you if set to True, but if it doesn't work for your data set then you need to manually adjust for your needs.
Book a Presales Call
Are you new to COZYROC and evaluating our solutions?
Book a Presales Call with us now and get 5% off SSIS+.
Book a Presales Call with us now and get 5% off SSIS+.
Related Articles
Setting up Salesforce Bulk Mode with COZYROC Script Salesforce Bulk Result Transformation
When using COZYROC Salesforce components in Bulk Mode the operations are asynchronous, meaning the results are not immediately available. If needed, the status, identifiers of newly created records, and any error information can be retrieved in an ...How to increase the Data Flow Task Plus buffer size?
To increase the Data Flow Task buffer size please complete the following steps: 1) Right-click on the Data Flow Task and select Properties. 2) On the Data Flow Task's Properties screen under Misc find fields: DefaultBufferSize and ...Error Message: Invalid Batch: type is not needed for non polymorphic foreign key references: <Relationship Field>
Action: When I am executing a package that includes Salesforce Destination component in bulk mode I get the following error message: Error Message: Invalid Batch: type is not needed for non polymorphic foreign key references: <Relationship Field> ...When setting sort/order in FetchXML on Dynamics CRM Source, the component returns only the number of rows set in batch size
Issue: When setting batch size on Dynamics CRM Source, the component returns only the number of rows set in batch size and the results do not page correctly. Solution: It is possible that a sort included in the fetchXML has caused this behavior to ...Salesforce Destination: How to modify a Salesforce Destination object dynamically at runtime
The destination object is available in the data flow properties list. Follow these steps to set up an expression: Create a variable that can be loaded with the desired name of the destination object. Right-click on data flow canvas and select ...