a different point of view

by kayeight on Wed 21 July 2010 // Posted in legacy // under

Yesterday, while becoming increasingly frustrated at the NY senate's website as I tried to scrape committee information, Nate suggested a much easier solution. I had been attempting to visit each senator's page and get their committees individually, but naturally, the website is formatted so inconsistently that no two senators list their committees in the same place or manner. I whiled away a number of hours in this direction before Nate had the idea of searching from the other direction - visiting each committee page and getting its members.

I got a good start on this track yesterday, but had to leave shortly after. Luckily, now that I'm not wrangling an enormous regular expression this should go much faster. Once we have a list of committees for each person, getting their meetings/votes is a matter of a couple loops and should be done in no time. And that will amount to the completion of this widget, at long last!