If the amount of data would be a problem, we could download the data to our servers and you can delete them. Also, we don’t need a whole year of data, several days or weeks would be enough.

Thanks!
Peter

I finally posted that AWS Public dataset. It contains the last 3 months of 2008 if that helps the other commenters:

http://developer.amazonwebservices.com/connect/entry.jspa?externalID=2596

Having the data mirrored on Amazon might also take some load off your servers ;)

I also posted source code for a Trending Topics site I built using the data:

http://www.trendingtopics.org

Thanks again for making this available. Good to know about the duplicates, I noticed that as well – will have to delete the dupes.

-Pete

they are gone? :(
Is there a way to get it back..I really need them for some research. ..

is there any other source for this?

and what about 08 data? is there any archive for it

pagecounts-20090511-120000.gz 2009-May-11 14:00:07 61.8M application/octet-stream
pagecounts-20090511-130000.gz 2009-May-11 15:00:08 65.3M application/octet-stream
pagecounts-20090511-130001.gz 2009-May-11 15:00:08 65.2M application/octet-stream
pagecounts-20090511-140000.gz 2009-May-11 16:00:09 68.5M application/octet-stream

which has one file for noon and 2PM but two for 1PM. In total, there are currently 30 files for 2009/05/11. I skimmed http://dammit.lt/wikistats/archive, and it appears to have exactly 24 files per day (I might have missed some), so I’m wondering if the recent files are groomed before archiving.

When there is more than one file per hour, are the complete counts for that hour formed by summing their corresponding counts? By using only the last file? Or something else?

Thanks for your help and for providing the excellent statistics.

Thanks – I’m going to pull together a Hadoop code example using the data and set up a mirror.

-Pete

This is great stuff. Can I mirror these stats on infochimps.org and other public dataset sites? Are these statistics under the same license as wikipedia content? I’ve been running some further analysis on the data from November that would make for a great code demo, but I don’t want to hit any licensing issues if I post the data itself.

-Pete

This does pose some questions as to the accuracy of the numbers, are they just from one squid or the whole network, is cacheing taken into account?

Anyway for my purpose its not a big deal, I’ve long been curious as too what the most read mathematics articles are and to that end took 12 hours worth to build a list http://en.wikipedia.org/wiki/User:Salix_alba/One_day_of_mathematics_page_views
of the most popular. Which prove somewhat interesting.

I just found your page-count project and it solves a big problem of mine. I’m maintaining scripts for converting the wikipedia to mobile readable eBooks ( see http://fbo.no-ip.org/wpmp ). However, some cell phones/PDAs/… just can’t take big memory cards which means some articles have to be removed – this is where your page count statistics will be very helpfull :-) . I’m currently downloading some of them – but I do have a request: Is it possible to publish i.e. daily or monthly statistics, too?
It would greatly reduce the amount of data I will have to download and parse through.

Thank you for your effords – I’ve been waiting for a long time for this kind of data.

For “no” (Norwegian WP – bokmål) I get a lot of Special:Export/PAGENAME. What’s that, a live mirror?

Other languages (da,et,fi,fo,fy,is,lt,lv,nn,pl,se,sv) are more normal.

And urldecoding was not done to ease both the export and import (whitespaces, various evil unicode bits, etc ;-)

1. Is it only Wikipedia? I only see “sv” (Swedish), but no reference to Wikipedia/Wikisource/Wikiqoute/Wikinews, etc.

2. Could you perhaps URL-decode the names so the files contains UTF-8 rather than %C3%96? (I can do this myself, no big deal, but it’s such a basic operation that it should really be done at the source. Unless there is some good reason not to do this.)