angrygoats.net

For those that haven’t heard about it, transit2012 got people together to observe the transit of Venus. Observations of the transit from different locations on the Earth can be used to calculate the distance between the Earth and the Sun. People around the globe tweeted their observations to twitter, and some software took those observations and combined them into a result.

I promoted transit2012 via twitter, Facebook and by emailing well known astronomy bloggers and podcasters. A few twitter users got excited and started promoting the project heavily themselves, and I was really lucky and got a mention in the popular Jodcast astronomy podcast.

I was concerned that people would go to the site, think “that’s cool”, then close it and forget about it. I implemented a registration system in the hopes that once people had registered – and had their very own point on a map of everyone observing – they’d feel more committed and be more likely to observe.

In the end 34 people registered. Registrations came from Germany, the United States, Canada, Australia, India, Turkey, Italy, Pakistan, Japan, Estonia, and the United Kingdom.

On the day we were cursed with bad weather. We used the Delisle’s method to calculate the parallax and thus the size of the solar system; this method requires only one observation from each observer, but observations of Venus entering or leaving the Sun’s disc can only be paired with other observations of the same event.

Europe and Western Australia were both covered with heavy cloud cover; I received some beautiful photographs of overcast Germany. The United States and Canada were both overcast but managed to get a look at it so we had several observations from there, and we got one observation from Queensland.

My software picked each compatible pair observations it could “parse” – decode the required latitude, longitude, observation type and time – and applied Delisle’s method. It then discarded any pairs which produced a negative or undefined result, and averaged those results together.

As most of the observations were from North America (either the northern US states or Canada), they paired together really badly. To observe the North/South parallax, you need pairs that are far apart on latitude. So pairs between close northern observers didn’t produce good results, but unfortunately were averaged into the result equally.

On the day some users varied from the prescribed tweet formats; my software was quite tolerant, but required particular words to appear (such as ‘entered’ or ‘left’), and required times to be converted to UTC. It was also necessary for the observer to geotag their tweet so the software knew their location. Several observations were lost because of these problems.

I’ve been through and remedied this as much as possible, discarding more aggressively close observation sites; after doing that, the result we got was 1AU = 142807811km. The accepted number for 1AU is 149597870.700 ± 0.003 km; so our answer is about 5% out from the real value. Not bad, especially considering we only had one southern hemisphere observer!

I have the observation tweets on disk, so if anyone is interested I can dump them out and share them; just get in touch.

There likely won’t be another Venus transit within my lifetime, but I’m keen to try this again with the 2016 transit of Mercury. Hopefully we’ll get a better number, and my software will improve having learnt from the problems described above.

A huge thank you to everyone that observed, discussed, promoted or got excited about this project. It was very, very rewarding to have so many people get caught up in it. In the end it was never about getting a good number, it was about re-enacting an important moment in scientific history, and getting excited about science in general. Given that we were hampered by the weather even now, it’s amazing that our 18th century counterparts achieved what they did.

Thanks everybody!

Well, I’m home. It was a nice trip.

England is a beautiful country. I stayed in Bromley, which is a little commuter town about twenty minutes away from London on the train. Each day I caught the train to Swanley – where the office is – which took about twenty minutes, travelling through the beautiful woods and fields of the green belt.

I caught up with a couple of friends while away. Had a few beers with Anil, then the next day saw him again with Alexa. Alexa and I went and saw an utterly ridiculous play – for some reason, every play we see together is surreal and yet oddly we’re the only ones laughing – The Mystery of Edwin Drood. Dickens died before finishing the book, so the end of the play was quite fun. Pretty awesome night; ended up having dinner at Pizza Express and bumping into one of the actors with his family. Spent some time amusingly failing to flag down a cab, then back to Victoria where a steam train happened to have arrived, so we went and had a look at it.

A couple of memorable evenings. It’s so strange to be back home, when it feels like only hours ago I was walking along the beautiful streets of London. I love Perth, and I’m not sure I’d love living in London, but as a place to visit it is just amazing.

I’ve just arrived in the UK on a one-week business trip. I left home at 7pm UTC+8 on Saturday, and arrived at the hotel at 2pm UTC+1 today. So accommodation-to-accommodation a 25 hour journey. So, roughly 50 hours of this week will be spent travelling. Yikes.

The longer leg of the trip is Perth to Dubai, something over twelve hours. I had an aisle seat next to a friendly older couple. Watched TV for a bit, then drifted off to sleep with headphones on; woke up to the bloke next to me thrashing around in a panic, with his wife telling him off. He managed to calm down, but would repeat this roughly every hour.

In the end I gave up on sleeping and finished off Catch-22. Brilliant book; if you’ve read it, you’ll realise why sitting in a plane with the passenger next to you spontaneously panicking and thrashing about is a absurdly appropriate environment in which to consume the book.

Dubai airport is the platonic shopping mall. Absolutely dysfunctional as an airport terminal. Displays showing gate allocations were extremely scarce, and all of the toilets had massive queues. A place so unpleasant you’d think Victor Gruen was involved.

Chap sat in my aisle on the Dubai to Gatwick leg decided to demonstrate his mountain climbing skills by nimbly hopping from seat arm to seat arm whenever he needed to get up. Very impressive, very eccentric.

Absolutely beautiful views of Razazza Lake outside of Karbala. The water was still, a deep blue against the bright white sand of the desert. The boundary was brilliantly crisp. I could also make out the Tigris threading along.

I’m now in Bromley. It appears I left all my toiletries at home (not sure how, I was sure I’d packed them!) so went out and bought replacements. I’ve treated myself to a rather nice jacket from Marks and Spencers, and am presently attempting to consume three enormous tubs of berries – Blackberries, Strawberries and Raspberries.

Life is pretty good.

I’ve got a VM over in the US; really useful to have due to good connectivity and the ability to run things on it that won’t be interrupted. One thing I run is undulatus, my twitter client. It logs all the tweets I see in a CouchDB database, and also builds a bunch of indexes so I can search them.

That takes up a lot of room. I’ve got around 4 million tweets on file; that takes up 21GB (including indexes.) Unfortunately my VM only has 25GB of disk space, so I was in trouble.

This handy article saved the day; I made a ZFS filesystem in a disk file, and then set up zfs-fuse. I set the filesystem size at 20GB, turned on transparent compression (in gzip-9 mode) and what was taking up 21GB is now taking up 5.5GB. Pretty neat; it’s actually quite fast too.

I’m not going to pretend it’s not an ugly hack, but it worked really well.

I was invited to speak to a journalism class at ECU today; I titled the lecture I gave “Covering Wikileaks”. Here are my slides.

It was fun! The students asked a lot of great questions, they seemed pretty interested – especially once I’d finished the intro and started talking about the stories I’ve written. A really fun experience!

I gave the class “everything wikileaks has released ever”; the (almost) complete download of all the torrents Wikileaks have published. About 25GB of stuff; it ought to keep them busy.

The shooting of Trayvon Martin – by all accounts because he was black, and young – reminds me of this quote from Barack Obama’s memoir, “Dreams from my Father”. At the end of chapter four, Obama’s grandmother (he was living with his grandparents) returns from a bus trip upset, having had an encounter with a man asking for money:

I returned to the kitchen. Gramps was rinsing his cup, his back turned to me. “Listen,” I said, “why don’t you just let me give her a ride. She seems pretty upset.”

“By a panhandler?”

“Yeah, I know – but it’s probably a little scary for her, seeing some big man block her way. It’s really no big deal.

He turned around and I saw now that he was shaking. “It is a big deal. It’s a big deal to me. She’s been bothered by men before. You know why she’s so scared this time? I’ll tell you why. Before you came in, she told me the fella was black.” He whispered the word. “That’s the real reason why she’s bothered. And I just don’t think that’s right.”

The words were like a fist in my stomach, and I wobbled to regain my composure.

Obama then describes a discussion of this with a friend, ending the chapter:

The earth shook under my feet, ready to crack open at any moment. I stopped, tried to steady myself, and knew for the first time that I was utterly alone.

I’ve updated my twitter client, undulatus. It no longer uses sqlite via sqlalchemy, but instead shoves all the tweets into a CouchDB database. This fixes an annoying problem with threads and locking, and also makes it much easier to do analysis on tweets.

I’ve also put all the deps into git submodules, so it’s now pretty easy to get the program running.

Now that I’ve played with it a bit, I have to say CouchDB is pretty neat. I can see why the hype – it is a better way of dealing with certain problems. I did consider using MongoDB (which seems to be pulling in more oxygen as a project) but it doesn’t run on ARM – and I want to keep this available to people running it on little ARM linux devices.

As part of this work I migrated my old tweet database, and discovered I have over three million tweets. They mostly go back to March last year – I had more, but lost them in an accidental database corruption. I’m now running CouchDB on my laptop and replicating from my server – kind of neat :-)