WCA website crawling

**Ron** · 08-03-2009

I added three new things to the Excel file (starting with the July 2009 version):
- Countries table
- Continents table
- SQL statements for creation of the tables

There are some additional tables that we use to generate the rankings on the pages without recalculating them all the time.
These tables are regenerated after adding new competition results.

**brunson** · 08-12-2009

Ron,

How would you feel about providing a "mysqldump --no-data" of the original schema? I imported several sheets of the workbook last night and it's not too hard to infer the types, but since it's only a single command, I thought you may not mind.

Thanks

**Ron** · 08-14-2009

How would you feel about providing a "mysqldump --no-data" of the original schema?

Please clarify your question.

**brunson** · 08-14-2009

Running the command "mysqldump --no-data schemaname" will generate a dump of all the SQL to create the schema without the data in it. That would be helpful to those of us who want to create a schema to import the data into.

I've also posted a more detailed request and an offer to assist over on the WCA site.

Thanks for all the work so far.

**Ron** · 09-02-2009

The new file for August 2009 is online now.
You need to refresh the whole database, because we needed to change some past results as well.

Brunson, I tried what you said but it does not give me more information than the table definitions I added to the first page of the file. What more information do you need?

**brunson** · 09-03-2009

Oh, I'm sorry, Ron. I didn't see that you had added that to the first page. My bad.

Thanks!

**brunson** · 09-03-2009

Originally Posted by Dave Campbell

I cannot open the file as it was created by a newer version of Excel than the one i am running (and i am running 2002). I'd personally rather see an sql dump file than an Excel file.

I just noticed that the results tab has exceeded the 65535 row limit of OpenOffice Calc and gnumeric, so I can't open it and retrieve all the result data. Bummer.

**MichaelErskine** · 09-03-2009

Originally Posted by brunson

I just noticed that the results tab has exceeded the 65535 row limit of OpenOffice Calc and gnumeric, so I can't open it and retrieve all the result data. Bummer.

I shall create a sqldump version and an ODS with two results tabs when I get into the office tomorrow. There's no good reason to get locked into non-free software

**brunson** · 09-03-2009

That would be so awesome, Michael. I was thinking of writing a python macro for OOCalc to automatically save the tabs as CSV, that's the only part that's still manual for me. I was disappointed at the 65535 row limitation of OO and ended up finding this: http://www.openoffice.org/issues/show_bug.cgi?id=30215

Normally I'd be all over complaining about proprietary formats, but Ron is doing us a huge service so I kept my mouth shut.

If you can send it to me I can put it on my web server for anyone else that wants it.

Thanks again. And no end of thanks to Ron for making the data available in the first place.