database - Mule: Inserting 470.000 records into Salesforce, that only allows 200 records per iteration -


i hava flow in mule gets records db around 470.000 records. these records have put salesforce. salesforce allows me insert 200 records per iteration ! unfortunately have can not these records in 1 db call, overloads machine memory. idea use "foreach" component in mule keep on looping until less 200 records db call.

how can these been accomplished ?

my config flow here:

<foreach batchsize="200" doc:name="for each" countervariablename="foreachcount">         <jdbc-ee:outbound-endpoint exchange-pattern="request-response" querykey="select200records" querytimeout="-1" connector-ref="postgresconnector" doc:name="database">             <jdbc-ee:query key="select200records" value="select * parties limit 200 offset #[variable:offset]"/>         </jdbc-ee:outbound-endpoint>         <set-variable variablename="dbpayload" value="#[payload]" doc:name="variable"/>         <scripting:component doc:name="script">             <scripting:script engine="jython"><![cdata[result = len(payload)]]></scripting:script>         </scripting:component> ....

salesforce has variety of batch api limits

batches data loads can consist of single csv or xml file can no larger 10 mb.

  • a batch can contain maximum of 10,000 records.
  • a batch can contain maximum of 10,000,000 characters data in batch.
  • a field can contain maximum of 32,000 characters.
  • a record can contain maximum of 5,000 fields.
  • a record can contain maximum of 400,000 characters fields.
  • a batch must contain content or error occurs.

if using mule salesforce connector there bulk option recall allows more default batch size.


Comments

Post a Comment

Popular posts from this blog

curl - PHP fsockopen help required -

i trying work on cloudflare api , have found in documentation. unfortunately have no idea how can use in fsockopen or through curl. here example of post request. need know how can make request below data post using curl or fsockopen curl https://www.cloudflare.com/api_json.html -d 'a=cache_lvl' \ -d 'tkn=8afbe6dea02407989af4dd4c97bb6e25' \ -d 'email=sample@example.com' \ -d 'z=example.com' \ -d 'v=agg' here link cloudflare https://www.cloudflare.com/docs/client-api.html#s4.2 try one: $ch = curl_init("https://www.cloudflare.com/api_json.html"); curl_setopt($ch, curlopt_returntransfer,1); curl_setopt($ch, curlopt_followlocation, 1); curl_setopt($ch, curlopt_ssl_verifypeer,false); // handling of -d curl parameter here. $param = array( 'a' => 'cache_lvl', 'tkn' => '8afbe6dea02407989af4dd4c97bb6e25', 'email' => 'sample@example.com', 'z' =...
Read more

HTTP/1.0 407 Proxy Authentication Required PHP -

i working in restricted network/limited access internet. since have use proxy make web service call. here php code calling web service , response . $client = new soapclient("http: //www.xyz.com/abc/wsdls/login.wsdl", array("trace" => 1, "exceptions" => 1, 'proxy_host' => $proxy_host, 'proxy_port' => $proxy_port, 'proxy_login' => $proxy_login, 'proxy_password' => $proxy_password)); response is fatal error: uncaught soapfault exception: [wsdl] soap-error: parsing wsdl: couldn't load ' http://www.xyz.com/abc/wsdls/login.wsdl ' : failed load external entity if copy url error message , paste in browser(with same proxy settings) works. 2nd scenario is, if pass values instead of variables giving me diffarant response $client = new soapclient("http: //www.xyz.com/abc/wsdls/login.wsdl", array("trace" => 1, "exceptions" => 1, 'proxy_host' ...
Read more

c# - Resource not found error -

i have problem accessing method on homecontroller. show code of method : [httpget] public actionresult decriptidentifiantetredirige(string login_crypter, string mdp_crypter) { string loginacrypter = _globalmanager.protegemotdepasse(login_crypter); string mdpacrypter = _globalmanager.protegemotdepasse(mdp_crypter); user userapp = new models.user(login_crypter, mdp_crypter); if (userapp.authentificationvalidee(userapp.userlogin, userapp.password)) { session["name"] = userapp.userlogin; return redirecttoaction("accueil", "home"); } else { return redirecttoaction("validerauthentification", "home"); } } then in routeconfig.cs wrote route : routes.maproute( name: "authentificationapresdecryptage", url: "{controller}/{action}/{login_crypter}/{mdp_crypter}", defaults: new...
Read more