Editor retention

December 4th, 2009

I’ve been playing with the changeset data for OpenStreetMap and looking to see what patterns I can find in the usage of various editors since changesets were introduced in the API 0.6 migration. We can start off just looking at the major editors by distinct users, i.e: everyone’s favourite popularity contest.

If we look at the number of changesets by editor ordered by the number of distinct users, we get a pretty blindingly obvious result:


Editor Nc Nb Nu
Potlatch 988916 739623 55941
JOSM 1092702 1043776 12945
Merkaartor 124756 117812 2100
Mapzen POI Collector 1374 1341 239
BigTinCan Upload Script 266 219 124
iLOE 1113 1019 105
osm2go 901 868 99
Osmose Raw Editor 756 358 82
Mapzen Beta 218 155 65
bulk_upload 60110 55417 61

Where Nc is the number of changesets, total. Nb is the number of changesets with bounding boxes, which usually means they represent real, geographic edits. Nu is the number of distinct users.

That’s not exactly a massive shock – Potlatch has the most users because it’s the editor on the OSM home page which requires the least effort to access. JOSM, somewhat surprisingly, has more changesets total and with bboxes, but maybe that’s just because until recently every upload was it’s own changeset. Merkaartor makes up the last of the “big three” editors with a couple of orders of magnitude more edits than than 4th place Mapzen POI collector.

Well, that didn’t tell us anything we didn’t already know. But how do editors compare at retaining their users? The following graph shows the number of users, logscale, against the length of time they’ve been editing over the past 6 months.

Editor retention over the period of 6 months

What does this tell us? Not a great deal — due to the growth of the OSM community there is always going to be a large number of users up-front and it’s not necessarily fair to assume they’re in the 1 month category, because they only joined a month ago. Correcting for this and normalising the results between the editors brings out the data much better:

Normalised editor retention over the period of 6 months

Here we can see that they all show the same basic pattern — a user is most likely to use the editor for less than a month and then lose interest. Given that the same pattern is visible in all three editors it’s hard to say whether the editor itself has much effect on that, or whether these users just lose interest in OSM altogether.

The most striking difference is that a significantly smaller number of users continue to use Potlatch and Merkaartor after the first month (33% and 39% respectively) than JOSM (64%). Of course, this admits many explanations — it could be that JOSM is an editor which attracts users after the initial drop-out stage, or it could be an editor which persuades people not to drop out. The data could be made to support either…

Entry Filed under: OSM

11 Comments

  • 1. OpenGeoData » Map E&hellip  |  December 4th, 2009 at 11:19 pm

    [...] has an interesting post comparing Map [...]

  • 2. Jim Brown  |  December 5th, 2009 at 9:25 am

    Good stuff Matt.

    It will be interesting to distinguish between overall retention (in OSM) and retention as users of the particular tool.

    Of course with some tools, like Mapzen POI collector, I would expect many user to use multiple tools concurrently. Collecting POI in the Mapzen POI Collector and then doing edits/roads etc with a web editor.

    Nice analysis….. j

  • 3. Richard Fairhurst  |  December 5th, 2009 at 10:02 am

    Very interesting.

    On the issue of retention: I think it’s pretty obvious that almost everyone (Flash haters apart) starts with Potlatch. As you say, “it’s the editor on the OSM home page which requires the least effort to access”.

    So, inevitably, its retention figure is going to be much lower than JOSM’s. Whether or not someone is a “JOSM user” or a “Potlatch user” (maybe there’s a gene or something), they will have started with Potlatch, even if just for two edits.

    One detail: Potlatch’s live mode begins a changeset immediately you enter it (whereas save mode begins a changeset when you save it). And, of course, for several months after 0.6 was introduced, live mode was all that was available in Potlatch. So I suspect the Nb-Nc discrepancy in Potlatch’s case is largely due to that.

  • 4. Christian O. Petersen  |  December 5th, 2009 at 9:50 pm

    I think it is important to remember that most tools and services that people sign-up to online has high churn. A lot of that churn is because a lot of people have good intentions about how much they will use a service, but with everything going on in their lives they forget about a lot of the services they sign up to.

    They way most successful internet sites deal with this is a combination of help to beginners, welcome emails, newsletters and social networking. I suspect that a lot of the churn from OSM is because there is very little of these activities.

  • 5. Matt  |  December 7th, 2009 at 1:25 am

    @Christian: It’s certainly one theory. Do you have any evidence or examples to back it up?

  • 6. OpenStreetMap Chile &raqu&hellip  |  December 7th, 2009 at 3:39 pm

    [...] Editor 5 de Diciembre de 2009 Publicado en Comparación, General, Otros Matt Amos escribió un artículo en su blog, en el que saca algunas conclusiones a partir de datos estadísticos de los [...]

  • 7. igorbrejc.net » Fre&hellip  |  December 8th, 2009 at 10:02 pm

    [...] Matt’s Blog » Editor retention [...]

  • 8. Russell Harrison  |  October 11th, 2010 at 9:52 pm

    I found that JOSM was the only editor that was really useful for getting significant work done. I like Merkaartor a bunch but its got some rough edges that need to be cleaned up. The flash editors are basically useless for all but the most simple edits. At least that’s been my personal experience.

  • 9. Vid the Kid  |  October 12th, 2010 at 7:30 am

    I use both JOSM and Potlatch, and I find that each is suited better to different tasks. If I’m going to do a variety of edits along a course (such as a highway) then I’ll tend to use Potlatch because the Yahoo aerial is super-convenient, and I might not know ahead of time exactly what I’ll need to edit. On the other hand, if I’m going to make a lot of adjustments in a well-defined area and/or on an easily-definable subset of OSM data over a very large area, JOSM works better.

    Examples for JOSM:
    Fixes and adjustments to boundary data (such as Ohio’s counties, or the state of Alaska)
    Systematic tagging adjustments to a city’s main road network (with help from xapi)
    Anything involving superrelations

    Examples for Potlatch:
    Applying route relations
    Cleaning up highway details (typically while applying route relations)
    Drawing natural and landuse areas
    Drawing buildings
    Merging boundaries with physical features
    Quick entry of information recently observed in the field

  • 10. Editing programs for OSM &hellip  |  October 12th, 2010 at 12:20 pm

    [...] http://www.asklater.com/matt/wordpress/2009/12/editor-retention/ [...]

  • 11. Ant  |  October 12th, 2010 at 2:10 pm

    This is fascinating, nicely done. is there any way to correlate how many nodes / ways / relations affected per editor, or perhaps an “average amount of change” per changeset, per editor?

    Another stat I think might be interesting is if it’s possible to count number of editors used per user – eg: 70% use 1 editor, 20% used 2 editors, etc


Calendar

December 2009
M T W T F S S
« Sep   Jul »
 123456
78910111213
14151617181920
21222324252627
28293031  

Most Recent Posts