r - Speed up multi-threaded sparse-matrix multiplication -


i have 2 large sparse matrices ([700 x 400000] %*% [400000 x 700]; 98% sparse) use following code multiply, takes 8 hours, can stop calculation @ time, write existing values, , start again stopped, require. i've looked @ other prepackaged solutions (blas, native r matrix stuff), can't wrap head around them. code work's i'm trying optimize it. i'm willing go outside r if gains sufficient. i'm sure other packages can them , faster, don't know current progress or expected run-time.

  • what should focus on decrease run-time?
  • how of decrease looking at?
  • would sparse matrix library speed up?

code:

library(dosnow) library(foreach) library(data.table)  cl <- makecluster(7, outfile ="")  registerdosnow(cl)   xtw <- as.matrix(fread('net_xtw.csv', header = f, sep=','))  net_players <- as.matrix(fread('net_players_filtered.csv', header = f, sep=','))  num_column=ncol(net_players) num_row=nrow(net_players)  xtx <- matrix(0,nrow=num_column,ncol=num_column)   getk <- function(num_row,xt,x){     temp = 0     (k in 1:num_row){         temp = temp + (xt[k] * x[k])     }     return(temp) }   (i in 1:num_column) {      xt = xtw[,i]     line <- foreach (j = i:(num_column), .combine=c) %dopar% {          x = net_players[,j]          num = getk(num_row,xt,x)         return(num)      }     (k in i:num_column){         xtx[i,k] <- line[(k-(i-1))]     }  }   stopcluster(cl)   write.table(xtx, file = "net_xtwx.csv",sep=",",row.names=f,col.names=f)  rm(list=ls(all=true)) 


Comments

Popular posts from this blog

account - Script error login visual studio DefaultLogin_PCore.js -

xcode - CocoaPod Storyboard error: -