windows - Reading the last n lines from a huge text file -
i've tried
file_in <- file("myfile.log","r") x <- readlines(file_in, n=-100)
but i'm still waiting...
any appreciated
i'd use scan
this, in case know how many lines log has :
scan("foo.txt",sep="\n",what="char(0)",skip=100)
if have no clue how many need skip, have no choice move towards either
- reading in , taking last n lines (in case that's feasible),
- using
scan("foo.txt",sep="\n",what=list(null))
figure out how many records there are, or - using algorithm go through file, keeping last n lines every time
the last option :
readlastlines <- function(x,n,...){ con <- file(x) open(con) out <- scan(con,n,what="char(0)",sep="\n",quiet=true,...) while(true){ tmp <- scan(con,1,what="char(0)",sep="\n",quiet=true) if(length(tmp)==0) {close(con) ; break } out <- c(out[-1],tmp) } out }
allowing :
readlastlines("foo.txt",100)
or
readlastlines("foo.txt",100,skip=1e+7)
in case know have more 10 million lines. can save on reading time when start having extremely big logs.
edit : in fact, i'd not use r this, given size of file. on unix, can use tail command. there windows version well, somewhere in toolkit. didn't try out yet though.
Comments
Post a Comment