Arab Spring Data Collection
Overview:
For this project, the majority of students used preexisting data and created a timeline. However, I created a web crawler for this purpose, and automated the process using much larger amounts of data. I began crawling on http://en.wikipedia.org/wiki/Arab_Spring and
http://en.wikipedia.org/wiki/Arab_Spring_concurrent_incidents.
What is a web crawler?
A web crawler in it's most basic form is a program that visits a webpage known as the 'seed'. It views all links on the seed and visits them, these links are called 'children'. The crawler searches through the children and visits all of their links, creating children for them. This goes on until the program or a user decides enough data is gathered. To learn more about web crawlers, I've posted a youtube video below.
For this project, the majority of students used preexisting data and created a timeline. However, I created a web crawler for this purpose, and automated the process using much larger amounts of data. I began crawling on http://en.wikipedia.org/wiki/Arab_Spring and
http://en.wikipedia.org/wiki/Arab_Spring_concurrent_incidents.
What is a web crawler?
A web crawler in it's most basic form is a program that visits a webpage known as the 'seed'. It views all links on the seed and visits them, these links are called 'children'. The crawler searches through the children and visits all of their links, creating children for them. This goes on until the program or a user decides enough data is gathered. To learn more about web crawlers, I've posted a youtube video below.
Results:
Usage Instructions:
To gather data: java -jar spidor.jar
To graph data(html file in PageData): java -jar spidor.jar graph
To gather data: java -jar spidor.jar
To graph data(html file in PageData): java -jar spidor.jar graph
spidor.jar | |
File Size: | 5837 kb |
File Type: | jar |
Black Holes and The Speed of Time
Door Problem
The problem is fairly simple, 1,000 students walk by 1,000 closed locker doors. Each student inverts the state of doors based upon which student they are(student 2 inverts every second, student 3 every third, etc). I needed to calculate the number of open doors after all the students have walked past. To visualize this, I created a basic program to display a visual aid using javascript and the HTML5 canvas, to view the result, click result(courtesy of JSFiddle).