Editor retention
December 4th, 2009
I’ve been playing with the changeset data for OpenStreetMap and looking to see what patterns I can find in the usage of various editors since changesets were introduced in the API 0.6 migration. We can start off just looking at the major editors by distinct users, i.e: everyone’s favourite popularity contest.
If we look at the number of changesets by editor ordered by the number of distinct users, we get a pretty blindingly obvious result:
| Editor | Nc | Nb | Nu |
|---|---|---|---|
| Potlatch | 988916 | 739623 | 55941 |
| JOSM | 1092702 | 1043776 | 12945 |
| Merkaartor | 124756 | 117812 | 2100 |
| Mapzen POI Collector | 1374 | 1341 | 239 |
| BigTinCan Upload Script | 266 | 219 | 124 |
| iLOE | 1113 | 1019 | 105 |
| osm2go | 901 | 868 | 99 |
| Osmose Raw Editor | 756 | 358 | 82 |
| Mapzen Beta | 218 | 155 | 65 |
| bulk_upload | 60110 | 55417 | 61 |
Where Nc is the number of changesets, total. Nb is the number of changesets with bounding boxes, which usually means they represent real, geographic edits. Nu is the number of distinct users.
That’s not exactly a massive shock – Potlatch has the most users because it’s the editor on the OSM home page which requires the least effort to access. JOSM, somewhat surprisingly, has more changesets total and with bboxes, but maybe that’s just because until recently every upload was it’s own changeset. Merkaartor makes up the last of the “big three” editors with a couple of orders of magnitude more edits than than 4th place Mapzen POI collector.
Well, that didn’t tell us anything we didn’t already know. But how do editors compare at retaining their users? The following graph shows the number of users, logscale, against the length of time they’ve been editing over the past 6 months.
What does this tell us? Not a great deal — due to the growth of the OSM community there is always going to be a large number of users up-front and it’s not necessarily fair to assume they’re in the 1 month category, because they only joined a month ago. Correcting for this and normalising the results between the editors brings out the data much better:
Here we can see that they all show the same basic pattern — a user is most likely to use the editor for less than a month and then lose interest. Given that the same pattern is visible in all three editors it’s hard to say whether the editor itself has much effect on that, or whether these users just lose interest in OSM altogether.
The most striking difference is that a significantly smaller number of users continue to use Potlatch and Merkaartor after the first month (33% and 39% respectively) than JOSM (64%). Of course, this admits many explanations — it could be that JOSM is an editor which attracts users after the initial drop-out stage, or it could be an editor which persuades people not to drop out. The data could be made to support either…
Entry Filed under: OSM
7 Comments Add your own
1. OpenGeoData » Map E&hellip | December 4th, 2009 at 11:19 pm
[...] has an interesting post comparing Map [...]
2. Jim Brown | December 5th, 2009 at 9:25 am
Good stuff Matt.
It will be interesting to distinguish between overall retention (in OSM) and retention as users of the particular tool.
Of course with some tools, like Mapzen POI collector, I would expect many user to use multiple tools concurrently. Collecting POI in the Mapzen POI Collector and then doing edits/roads etc with a web editor.
Nice analysis….. j
3. Richard Fairhurst | December 5th, 2009 at 10:02 am
Very interesting.
On the issue of retention: I think it’s pretty obvious that almost everyone (Flash haters apart) starts with Potlatch. As you say, “it’s the editor on the OSM home page which requires the least effort to access”.
So, inevitably, its retention figure is going to be much lower than JOSM’s. Whether or not someone is a “JOSM user” or a “Potlatch user” (maybe there’s a gene or something), they will have started with Potlatch, even if just for two edits.
One detail: Potlatch’s live mode begins a changeset immediately you enter it (whereas save mode begins a changeset when you save it). And, of course, for several months after 0.6 was introduced, live mode was all that was available in Potlatch. So I suspect the Nb-Nc discrepancy in Potlatch’s case is largely due to that.
4. Christian O. Petersen | December 5th, 2009 at 9:50 pm
I think it is important to remember that most tools and services that people sign-up to online has high churn. A lot of that churn is because a lot of people have good intentions about how much they will use a service, but with everything going on in their lives they forget about a lot of the services they sign up to.
They way most successful internet sites deal with this is a combination of help to beginners, welcome emails, newsletters and social networking. I suspect that a lot of the churn from OSM is because there is very little of these activities.
5. Matt | December 7th, 2009 at 1:25 am
@Christian: It’s certainly one theory. Do you have any evidence or examples to back it up?
6. OpenStreetMap Chile &raqu&hellip | December 7th, 2009 at 3:39 pm
[...] Editor 5 de Diciembre de 2009 Publicado en Comparación, General, Otros Matt Amos escribió un artículo en su blog, en el que saca algunas conclusiones a partir de datos estadísticos de los [...]
7. igorbrejc.net » Fre&hellip | December 8th, 2009 at 10:02 pm
[...] Matt’s Blog » Editor retention [...]
Leave a Comment
Some HTML allowed:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>
Trackback this post | Subscribe to the comments via RSS Feed