If we are to continue with R for the next few years, it is essential that it get faster.
There are many aspects to this. One is compiling interpreted R code into something faster.
LLVM is a toolkit that facilitates the compilation of machine code. So in the past few days
I have looked into this and developed an R package that provides R-bindings to some of
the LLVM functionality.

The package is available from http://www.omegahat.org/Rllvm, as are several examples
of its use.
I used the package to implement a compiled version of one of Luke Tierney’s compilation examples
which uses a loop in R to add 1 to each element of a vector. The compiled version gives a speedup
of a factor of 100, i.e. 100 times faster than interpreted R code. This is slower than x + 1
in R which is implemented in C and does more. But it is a promising start. The compiled version is also faster than bytecode interpreter approaches. So this is reasonably promising.

Of course, it would be nicer to leverage an existing compiler! (Think SBCL and building on top of LISP).

One could use the RGCCTranslationUnit package to obtain descriptions of routines and data
structures and then generate the interfaces to those routines via functions in Rffi.

Writing or generating C/C++ code for wrappers (see RGCCTranslationUnit) is still the way to
go in many ways, but Rffi is very convenient for dynamic invocations without any write and compile
setup costs.

As usual, you can install this from source from the Omegahat repository

install.packages(“Rffi”, repos = “http://www.omegahat.org/R”, type = “source”)

but you will need to have installed libffi.

The package allows one to use XQuery from within R and to use R functions within XQuery scripts.

There is a new version of the RKML package which handles large datasets much more rapidly.

Also, I put a new package named RJSCanvasDevice which implements and R graphics device that creates JavaScript code that can be subsequently display on a JavaScript canvas in an HTML document.

fixXMLNamespaces =
  #
  #  call as
  #    dd = fixXMLNamespaces("~/v75_step6.wsp", .namespaces = MissingNS)
  #  or
  #   dd = fixXMLNamespaces("~/v75_step6.wsp", gating = "http://www.crap.org", 'data-type' = "http://www.morecrap.org")
  #
function(doc = "~/v75_step6.wsp", ..., .namespaces = list(...))
{
    # collect the error messages
  e = xmlErrorCumulator(, FALSE)
  doc = xmlParse(doc, error = e)

  if(length(e) == 0)
     return(doc)

     # find the ones that refer to prefixes that are not defined
  ns = grep("^Namespace prefix .* not defined", unique(environment(e)$messages), val = TRUE)
  ns = unique(gsub("Namespace prefix ([^ ]+) .*", "\\1", ns))

    # now set those name spaces on the root of the document
  if(is(.namespaces, "list"))
    .namespaces = structure(as.character(unlist(.namespaces)), names = names(.namespaces))

  uris = .namespaces[ns]
  if(length(uris)) {
     mapply(function(id, uri)
              newXMLNamespace(xmlRoot(doc), uri, id),
            names(uris), uris)
     xmlParse(saveXML(doc), asText = TRUE)
  } else
     doc
}

(I’ve made some minor changes thanks to Hin-Tak’s suggestions, but haven’t tested them.)

• query information from our blog
• post blog items, append to a post, create new pages, add categories, etc.

So the RWordpress package is the result.

Correction: WordPress is taking the URL and capitalizing the p in RWordpress. This seems to happen only for words containing “wordpress”. So I have renamed the R package to RWordPress. The link on this page (even while being lower-case p in the HTML) now corresponds to the new package name. Thanks Tal.

newPost(list(description = 'This post is submitted directly from R using the RWordpress package.', title = 'Blogging directly from R - example'))

Automating the collection involves the cron facility on a Linux machine. And there she ran into some troubles which are somewhat interesting to note as a learning experience. Firstly, why use a Linux machine? Because we want a machine that is on the network all the time rather than using a laptop or home machine. Also, it is nice if that machine is backed up and reliable.

One can “google” cron and find crontab. So one needs to learn the syntax for specifying a cron job. We have one entry per line that specifies the minute, hour, day of month, month and day of week.
We can have ranges, e.g. 2-4. An asterisk * in a field means all of them possible values, i.e. first-last.

The last field is a shell command. The first question is what shell? The second question is what are the settings? In other words, is our .profile read? is the .login read? The answers to these questions are available from the documentation. But it is interesting to consider how we can derive the answers ourselves with a series of tests.

The problem the student had was that she could run R fine in an interactive shell, but in the cron job, she was getting errors. She worked through a sequence of problems and ended up with an issue
when loading the RCurl package with an error message (in the output from cron) that indicated that it couldn’t load libcurl.so.

So let’s get to work.

Probably the simplest thing to do is set a cron job that prints the environment variables. Put the following into a file, say, myCronJobs:

* * * * * env > /tmp/myEnv

and then run the shell command

crontab myCronJobs

It is always a good idea to see if the job(s) were set with

crontab -l

Now, wait a minute. Literally that is until the job runs. Alternatively, quickly create the file /tmp/myEnv and run

tail -f /tmp/myEnv

and watch the output to see the lines appear. BTW, you can run tail on the file that isn’t there!

This shows you what environment variables are set and you can compare this to your regular shell by running the env command in it. Quite a difference!

Note the value of the LD_LIBRARY_PATH environment variable when run in the regular shell and in the cron job. What directories are missing from the cron job’s version? Now let’s try to find libcurl.so. We might use the locate command. And indeed, on that machine it is in a directory that is not included in the LD_LIBRARY_PATH environment variable for the cron job. So we have to explicitly set that in our cron script with a call to

export LD_LIBRARY_PATH=/usr/local/lib:/usr/lib:/usr/local/lib64

or whatever the relevant settings are (using : to separate the directory names).

Another approach would be to update the ld.so.conf to have the system’s dynamic loader know to look in that directory. But one need’s administrator privileges to do that.

Note that the way to get the output is via shell redirection. And we can and should use redirection to put the output to standard error to standard out to appropriate files or combine them into the same output. Note that you are using the regular shell – sh which is probably a variant of bash. So you need to use

  export LD_LIBRARY_PATH=/usr/local/lib ;  /usr/local/bin/Rscript -e 'library(RCurl); print(getURLContent)' > /tmp/Routput 2>&1

• Flash
• KML
• JavaScript canvas

and to create them with minimal fuss. The RFreetype package also allows us to compute metric info for fonts.

One of the things that I have been implementing with this RGraphicsDevice package in the last week is a device that keeps a list of all the calls to the primitives and their details. This is essentially the display list that is stored in a device, but we are doing this in R and able to put our own structure on the information for each call to the device. What’s the point? Well, it allows us to do something similar to what we did in the SVGAnnotation package. There we created R plots as SVG documents and then went back and post-processed the SVG to find the SVG elements that correspond to the graphical elements in the R plot, e.g. axes, labels, data points, legend, and so on. Given those SVG elements, we can add hyperlinks, interactive commands and even animation such as the gapminder-like display.

So how does this “component device” relate to this? Well, we can collect the graphical elements and then analyze and group them to identify the different elements of the R plots. Then we can use this to understand the order in which the elements were constructed and annotate or post-process code for Flash, JavaScript canvas, KML or SVG. This is the general approach to being able to map the low-level elements to the high-level graphical components of the plot and vice verse.