Wednesday, 8 January 2014

Querying Wikipedia data with SPARQL

Wikipedia is a great and well-known resource for all kinds of information.  However this information is not readily query-able (other than by search term) on the Wikipedia site directly.  There is a community effort at to make a structured set of Wikipedia data available for querying and analysis.

The query form at is a great way to test out the kinds of data extraction possible.

As a basic example, the following SPARQL query lists software products by subject and organisation and is limited to 500 results.
SELECT ?subject ?organisation ?product
    ?organisation rdf:type <> .
    ?product <> ?organisation .
    ?product rdf:type <> .   
    ?product <> ?subject .
ORDER BY ?subject ?organisation

The following SPARQL query lists subjects for which there are at least 50 software products defined, and provides the count of products against each subject.
SELECT ?subject count(distinct ?product)
    ?organisation rdf:type <> .
    ?product <> ?organisation .
    ?product rdf:type <> .   
    ?product <> ?subject .
GROUP BY ?subject
HAVING count(distinct ?product) >= 50
ORDER BY ?subject

The queries above are very simple examples and it is possible to make more complex use of the links between data available at

I believe there are many interesting possibilities for this capability and have been using data extracted in this way in a small project.

Monday, 30 December 2013

Go and the Semantic Web: Creating Go language bindings for the Redland libRdf C library

Recently I began work on a small project that makes use of several Semantic Web Technologies (namely RDF, SPARQL and triplestores).   I chose to use this project as an opportunity to explore the Go language and have been happy with the results.

The main output of this work so far is a set of Go language bindings, tests and examples for the Redland libRDF C library.  I have made this available on github:

The Redland libRDF library provides parsers for several RDF syntaxes along with RDF storage and querying support.

The Go language standard library is fairly comprehensive, however this does not extend to RDF support and hence the need for a library such as Redland libRDF.  Go through the use of CGO has good support for calling into C code.

I was able to use the set of Redland C examples as a basis for creating equivalent Go examples in the form of tests.  Go has a built-in automated testing package and tests can be executed using the same go command used to compile.
After trying a couple of different IDEs for this work, I settled on Sublime Text 2 with the GoSublime package.

As Go relies on folder structure / file location to infer package structure, and has a simple but powerful command line; a tool like Sublime Text, which has a terminal window, makes an effective IDE for Go development.

I am currently using these bindings on small web-project which I hope to describe in more detail in the future.

Wednesday, 3 July 2013

WiseMove HTML5 Mobile App Part 2 - KnockoutJS and JQuery Mobile

In this post I describe 2 of the main libraries used in the WiseMove HTML5 mobile app introduced in my last post.

The full source for the app is available on github here:

An instance of the app tailored for desktop browsers is here:

One key decision when choosing libraries for mobile or web HTML5 development is whether to follow a declarative DOM-based approach for defining UI and model bindings, or whether to use one of the many libraries that allow you to code against a more abstracted UI model.  For the development of the WiseMove application I preferred the former approach. 

In line with this decision, I found KnockoutJS and JQuery Mobile to be an effective combination.  There are several posts that describe how these libraries can work together including this one by David Forster:

KnockoutJS allows you to decorate HTML elements with binding information that describes how values in a JavaScript model should be presented in the DOM, and how changes to DOM elements should cause updates to the JavaScript model.  This reduces the amount of event handling and DOM manipulation code that must be written.
KnockoutJS defines binding types for the standard HTML element types and events.  

When using JQuery Mobile widgets it is sometimes necessary to call methods on the elements when values change or when the DOM representation for the widget should be initialised or updated.  A widget of this type used in WiseMove is the JQuery Mobile Slider widget.

In order for the visual representation of the slider to be updated whenever the associated model value changes a KnockoutJS binding handler was needed as shown below.  The only really slider specific pieces of the code below are the call to the slider() methods.

ko.bindingHandlers.slidervalue = {
    init: function(element, valueAccessor, allBindingsAccessor, viewModel, bindingContext) {
        // This will be called when the binding is first applied to an element
        // Set up any initial state, event handlers, etc. here
        //handle the field changing
        ko.utils.registerEventHandler(element, "change", function () {
            var observable = valueAccessor();
    update: function(element, valueAccessor, allBindingsAccessor, viewModel, bindingContext) {
        // This will be called once when the binding is first applied to an element,
        // and again whenever the associated observable changes value.
        // Update the DOM element based on the supplied values here.
        var value = valueAccessor();
        var valueUnwrapped = ko.utils.unwrapObservable(value);
        if (valueUnwrapped){
 The binding is used in HTML as shown below.
<input type="number" min="-1" max="10" data-bind="slidervalue: scoreValue" />

I found KnockoutJS very easy to use and definitely a time-saver.  There are many other approaches and libraries available.  One framework that I am interested in investigating further is AngularJS, a much larger framework that  contains elements that perform some of the same functions as KnockoutJS.

I have read a number of comments concerning performance issues associated with JQuery Mobile, such as flickering and non-smooth scrolling.  I did experience some problems of this type when running the app on mobile devices however I was able to get to an acceptable level of performance after applying some of the tips and approaches found on-line.  The JQuery Mobile library is more ambitious than many others in terms of the breadth of supported platforms and browser versions.  It seems this comes with a number of challenges and a few quirks but ultimately I think the framework is an impressive one.  If you have a more controlled set of target platforms you may see advantages in a more targeted UI framework.

Thanks for reading.

Tuesday, 18 June 2013

WiseMove: a HTML5 Mobile App - full example with source - Part 1

In this small series of posts I will introduce a HTML5 Mobile App named WiseMove, and the technologies and tools used to build it.

The app is available on-line (in form suitable for desktop) at the following location:

The app can be packaged for mobile using Cordova and full source code is available at

The app is intended to serve a real, albeit small purpose.  WiseMove is aimed at assisting a person moving house to evaluate locations.  The user is able to place pins on a map that represent locations the user is considering moving to, or locations the user will visit.   The user is able to score locations by any measure they define.  The app also looks up route information from google to determine likely travel times.

The subjective scores and travel times are displayed in charts and tabular forms.

There are a very large number of frameworks and libraries in which this application could have been successfully built.  The libraries I used were:

  • JQuery Mobile
  • Knockout JS
  • Google Maps
  • Flot charts
  • QUnit JS  
The development environment included:
  • Ubuntu
  • WebStorm
  • Ripple Emulator
  • NodeJS (as static content server)
In the next post I'll dig into the way the libraries were used in detail.

Friday, 7 June 2013

NSimulate: a discrete-event simulation library for .NET

This is the first post in my blog in which I plan to talk about software development topics or announce details of some of the projects (usually open-source) that I work on from time to time.

One of my recent projects has been the development of an event simulation library for .NET.  The library is open-source (MIT style license) and available on github:

My aim for the project was to produce a flexible library that is extensible and supports simulation of processes, activities, resources and event notifications.

Some inspiration for this comes from the Simpy (python) simulation package.

There are several examples included with the library in which I have tried to demonstrate the various usage patterns.  These examples are the fairly typical set for this kind of thing, but (I hope) are implemented in a concise way.

Examples include:

  • Workshop with unreliable machines
  • Call-center with call escalation
  • Order delivery with warehouse reorder
  • Alarm Clock

Each simulation runs until an end-condition (specified as a delegate) is met.

The simulation is single-thread, and processes are implemented as iterators using the c# yield statement (so that process logic can be written cleanly and without the need for callbacks).  When a process yields a result, the process logic is paused until some condition (indicated by the yield result) is met; this may be the availability of a resource, the occurrence of a specified event, or simply the passing of a certain amount of simulation time.

One example implementation of a process is shown below.

// while the simulation is running
 // check if the queue for this machine is empty
 if (JobQueue.Count == 0){
  // if it is, wait until there is something in the queue
  yield return new WaitConditionInstruction(()=>JobQueue.Count > 0);
  // take a job from the queue
  var jobToProcess = JobQueue.Dequeue();

  // simulate processing the job
  // which takes time
  yield return new WaitInstruction(jobToProcess.ProcessingTimeRequiredByJobQueue[JobQueue]);

  // use the reliability indicator to determine if the machine is broken down
  if (CheckForRandomBreakdown()){
   // the machine has broken down
   // add the job it was processing back to the queue

   // obtain a repair person
   var allocateInstruction =  new AllocateInstruction<RepairPerson>(1);
   yield return allocateInstruction;

   // and wait for the machine to be fixed
   yield return new WaitInstruction(RepairTimeRequired);

   // then release the repair person resource
   yield return new ReleaseInstruction<RepairPerson>(allocateInstruction);
   // record the fact that the job has been processed by this machine type

   // if the job still requires other processing
   if (jobToProcess.RequiresMoreWork){
    // add it to the next queue
    // otherwise remove it from the all unprocessed jobs list
All feedback is welcome and thanks for reading.