Matt's Blog

Imports and the Community

September 6th, 2009

I’ve been thinking recently on the effects of imports on the OpenStreetMap community. But there’s no real hard data — and repeatable experiments would be difficult if not impossible. So I’ve turned to that old “what if” machine; the Monte-Carlo simulation.

A caveat before reading the whole post: I’ve no way of backing any of this up, so treat it as an “informed guess”. There are assumptions about the dynamics of the community which I’ve used in the model which may, or may not, be true in reality.

The first case is looking at the effect of an initial import (i.e: before I’ve started the community simulation) on the completion of the map. What I’m simulating is thousands of contributing agents, generated based on levels of local activity on the assumption that many mappers are recruited either through friends or by local events. There’s also a background, random level of recruitment from news articles, long-range friends, etc… All contributor sign-up is linked to a population density field so that the distribution of mappers is greater in urban areas, as it is in real life. Mappers also have a “comfort radius” within which they will map, but not go beyond, which is probably the case for most casual mappers. Finally, there is a “completeness” threshold for each mapper which controls the extent to which they’ll “complete” the map. This simulates an effect I’ve seen in real life — that different people care about, and will map, different things and few people will map absolutely everything and anything.

Anyway, on to the results. The first graph is instantly informative:

Basically; the more we start with, the less you end up with and the longer it takes to get there. The worrying thing is that the asymptotic level of “completeness” (for whatever value you consider as an indicator of quality) is inversely related to the level of initial import. The asymptotic level of completion isn’t 100% because the model accounts for very rural areas, where recruitment is poor, and doesn’t account for the mobility of mappers. A simplifying assumption is that they’ll stay within their “comfort radius” of home.

But we’ve got communities everywhere. So what happens when the import is done after the community has started growing, and started mapping its area?

Hmmm… Pretty much the same thing, but less so. A word here, though, about the “import” that I’m doing into the model. It’s a best-case import which completes the map to a certain degree everywhere and doesn’t stomp on, duplicate or otherwise maul anyone’s existing data, so I don’t model the corresponding drop in the community which might result from people feeling their efforts were steam-rollered. Also, the “import” is considered to be good data, something which is rarely the case in the real world.

But what happens when the import covers only a part of the total area?

This is better still, if only because the impact of the import is localised to a part of the community. But what happens if the import isn’t contiguous, but spread randomly over a fraction of the area?

Ah… Finally, some benefit to the imports — there doesn’t seem to be any long-term damage to the community and the “completeness” levels out at roughly the same place. The “import” here was to a small fraction of the area and, again, I didn’t simulate any direct effect on the community, or any time lost in fixing-up and integrating such data into the existing map.

So what happens if we break an import up into lots of mini-imports and spread them out so that the rate of importing is roughly constant?

Interestingly, here we can see even less of a long-term effect on the rate of completion. In fact, all the curves seem to converge pretty rapidly over time. However, if you’re looking for 90% completion, it seems the approach with incremental imports makes no measurable difference to the time taken, or to the final levels of completion.

What, if any, conclusions can we draw from this? It seems that small, non-local imports aren’t detrimental, and may even be beneficial, to the growth of the community and the map. Imports of specialised features, such as bus stops or water areas, can be very useful in filling in the gaps where other contributors may not want to contribute.

However, it seems that large-scale imports, particularly of “fundamental” features such as road networks, can cause problems with the growth of a community, particularly if the import is done at a very early stage in the growth of that community.

In conclusion, there’s no evidence here that imports are all bad, but it’s my personal opinion that imports, particularly if they’re badly thought-through, too wide-ranging or don’t do enough to engage the existing community, can be bad. And often are.

Entry Filed under: OSM