r - CURL handle goes Stale when inside foreach() -
alright, i've figured out can query website behind login screen csv report. thought, wouldn't better concurrently? afterall reports take lot longer produce others , if querying 10 different reports @ once way more efficient. i'm on head twice here playing around https protocols , parallel processing. think frankencode there though gives me
"error in ( : task 1 failed - "stale curl handle being passed libcurl"
note "curl" current "html" variable did login successfully. happens in it's parallel chunk makes stale.
library(rcurl) library(doparallel) registerdoparallel(cores=4) agent="firefox/23.0" options(rcurloptions = list(cainfo = system.file("curlssl", "cacert.pem", package = "rcurl"))) curl = getcurlhandle() curlsetopt( cookiejar = 'cookies.txt' , useragent = agent, followlocation = true , autoreferer = true , curl = curl ) un="username@domain.com" pw="password" html = postform(paste("https://login.salesforce.com/?un=", un, "&pw=", pw, sep=""), curl=curl) urls = c("https://xyz123.salesforce.com/00o400000046ayd?export=1&enc=utf-8&xf=csv", "https://xyz123.salesforce.com/00o400000045swu?export=1&enc=utf-8&xf=csv", "https://xyz123.salesforce.com/00o400000045z3q?export=1&enc=utf-8&xf=csv") x <- foreach(i=1:4, .combine=rbind, .packages=c("rcurl")) %dopar% { xxx <- geturl(urls[i], curl=curl) }
Comments
Post a Comment