Some Changes to OSM: Technical Details

April 1st, 2009

I haven’t updated this blog in a while because I’ve been hard at work on some very important changes to the OSM server stack. Have a look at Steve’s announcement for the high-level vision and continue on here for the low-level techniques we’ve used to speed the server up by at least two orders of magnitude.

The new server backend is MonetDB, a well-known and widely deployed open-source database with a proven track record of stability and scalability. It is definitely not an academic project.

But thats not the best part – the secret of the new blindingly fast performance is our adoption of a fixed ontology. Basically, free-form tags were slowing us down, both on the server and in the community. We’ve adopted one of the best industry standard ontologies fronted by several simple APIs . And, in order to prevent piracy of OSM data, the server now supports DRM.

The result of all this hard work by TomH and myself is that we can now run the OSM server faster on less hardware. And the fixed ontology reduces the size of the daily diffs to less than 100 bytes, so we’re distributing them via Twitter. Its what all the cool kids are doing.

:-)

Entry Filed under: OSM, Tech

1 Comment

  • 1. Administrator of OSMonetDB  |  April 1st, 2009 at 3:53 pm

    The rationale for the move to MonetDB are the reported problems in scaling up to sizes needed to support OSM as explained in this blog. It is not a secret that the man power required for database management is big, and the available man power small. We are indeed happy we could relief TomH from the firefighting[1] due to the self-organising database management MonetDB brings us.

    In the last big comparison in 2008 we already reported significant differences[2] on large scale implementations using other Open Source databases such as PostgreSQL and MySQL.

    Most of the people working on the development of MonetDB have MSc and PhD level academic education, likewise for another big search-company, while we are still actively publishing about new databases and query techniques.

    [1]http://groups.google.com/group/pgsql.performance?hl=en
    [2]http://monetdb.cwi.nl/projects/monetdb//SQL/Benchmark/TPCH/


Calendar

April 2009
M T W T F S S
« Dec   Aug »
 12345
6789101112
13141516171819
20212223242526
27282930  

Most Recent Posts