How do I set the default number of threads for Scala 2.10 parallel collections? -


in scala before 2.10, can set parallelism in defaultforkjoinpool (as in answer scala parallel collections degree of parallelism). in scala 2.10, api no longer exists. documented can set parallelism on single collection (http://docs.scala-lang.org/overviews/parallel-collections/configuration.html) assigning tasksupport property.

however, use parallel collections on codebase , not add 2 lines every single collection instantiation. there way configure global default thread pool size somecollection.par.map(f(_)) automatically uses default number of threads?

i know question on month old, i've had same question. googling wasn't helpful , couldn't find looked halfway sane in new api.

setting -dscala.concurrent.context.maxthreads=n suggested here: set parallelism level collections in scala 2.10? seemingly had no effect @ all, i'm not sure if used correctly (i run application 'java' in environment without 'scala' installed explicitly, might cause).

i don't know why scala-people removed essential setter appropriate package object.

however, it's possible use reflection work around incomplete/weird interface:

def setparallelismglobally(numthreads: int): unit = {   val parpkgobj = scala.collection.parallel.`package`   val defaulttasksupportfield = parpkgobj.getclass.getdeclaredfields.find{     _.getname == "defaulttasksupport"   }.get    defaulttasksupportfield.setaccessible(true)   defaulttasksupportfield.set(     parpkgobj,      new scala.collection.parallel.forkjointasksupport(       new scala.concurrent.forkjoin.forkjoinpool(numthreads)     )    ) }

for not familiar more obscure features of scala, here short explanation:

scala.collection.parallel.`package`

accesses package object defaulttasksupport variable (it looks java's static variable, it's member variable of package object). backticks required identifier, because package reserved keyword. private final field want (getfield("defaulttasksupport") didn't work reason?...), tell accessible in order able modify it, , replace it's value our own forkjointasksupport.

i don't yet understand exact mechanism of creation of parallel collections, source code of combiner trait suggests value of defaulttasksupport should percolate parallel collections somehow.

notice question qualitatively of same sort older question: "i have math.random() on codebase, how can set seed fixed number debugging purposes?" (see e.g. : set seed on math.random() ). in both cases, have sort of global "static" variable implicitly use in million different places, want change it, there no setters variable => use reflection.

ugly hell, seems work fine. if need limit total number of threads, don't forget garbage collector runs on separate thread.


Comments

Post a Comment

Popular posts from this blog

curl - PHP fsockopen help required -

i trying work on cloudflare api , have found in documentation. unfortunately have no idea how can use in fsockopen or through curl. here example of post request. need know how can make request below data post using curl or fsockopen curl https://www.cloudflare.com/api_json.html -d 'a=cache_lvl' \ -d 'tkn=8afbe6dea02407989af4dd4c97bb6e25' \ -d 'email=sample@example.com' \ -d 'z=example.com' \ -d 'v=agg' here link cloudflare https://www.cloudflare.com/docs/client-api.html#s4.2 try one: $ch = curl_init("https://www.cloudflare.com/api_json.html"); curl_setopt($ch, curlopt_returntransfer,1); curl_setopt($ch, curlopt_followlocation, 1); curl_setopt($ch, curlopt_ssl_verifypeer,false); // handling of -d curl parameter here. $param = array( 'a' => 'cache_lvl', 'tkn' => '8afbe6dea02407989af4dd4c97bb6e25', 'email' => 'sample@example.com', 'z' =...
Read more

HTTP/1.0 407 Proxy Authentication Required PHP -

i working in restricted network/limited access internet. since have use proxy make web service call. here php code calling web service , response . $client = new soapclient("http: //www.xyz.com/abc/wsdls/login.wsdl", array("trace" => 1, "exceptions" => 1, 'proxy_host' => $proxy_host, 'proxy_port' => $proxy_port, 'proxy_login' => $proxy_login, 'proxy_password' => $proxy_password)); response is fatal error: uncaught soapfault exception: [wsdl] soap-error: parsing wsdl: couldn't load ' http://www.xyz.com/abc/wsdls/login.wsdl ' : failed load external entity if copy url error message , paste in browser(with same proxy settings) works. 2nd scenario is, if pass values instead of variables giving me diffarant response $client = new soapclient("http: //www.xyz.com/abc/wsdls/login.wsdl", array("trace" => 1, "exceptions" => 1, 'proxy_host' ...
Read more

c# - Resource not found error -

i have problem accessing method on homecontroller. show code of method : [httpget] public actionresult decriptidentifiantetredirige(string login_crypter, string mdp_crypter) { string loginacrypter = _globalmanager.protegemotdepasse(login_crypter); string mdpacrypter = _globalmanager.protegemotdepasse(mdp_crypter); user userapp = new models.user(login_crypter, mdp_crypter); if (userapp.authentificationvalidee(userapp.userlogin, userapp.password)) { session["name"] = userapp.userlogin; return redirecttoaction("accueil", "home"); } else { return redirecttoaction("validerauthentification", "home"); } } then in routeconfig.cs wrote route : routes.maproute( name: "authentificationapresdecryptage", url: "{controller}/{action}/{login_crypter}/{mdp_crypter}", defaults: new...
Read more