gps10's blog

Better Course Reviews Interface

The course review pages accessed through the Course Search contain information that students frequently take into consideration when planning their semester schedules. Unfortunately, these reviews are presented in plain text, with just raw numbers to read. This is where technology comes to the rescue!

To obtain all of the course reviews ever created, point your browser to https://www.cwru.edu/cgi-bin/courseevals.cgi?course=ABLE (valid network username and password required). This script takes one parameter, either a department code or a department code plus course number (e.g. MATH, MATH224, EBME, POSC109). For some reason, the department codes ABLE, CHIN, DENT, EQUI, MAND, RADI, and UNIV all return the full list of evaluations (13801 total) in the database (perhaps somebody could fix this?). It is best not to open this page directly in a browser, as you might crash your system due to the overwhelming size of the page!

Now that you have a full list of courses in a string, parse it with the following regular expression: (yes, I am a Perl regular expression newbie)


/\s+EVALUATION OF COURSES\s+(\w+),\s+(\d+)\s+\* COURSE NAME\s+:\s+(\D+)(\d+)?\s+SECTION:\s+(\d+)*\s*\* INSTRUCTOR\s+:\s+(\w+[ ]?\w+)\s+1\. HOW DOES THIS COURSE FIT INTO YOUR ACADEMIC PROGRAM\s+MAJOR REQUIRED:\s+(\d+)[%]?\s+TECH IN MAJOR:\s+(\d+)[%]?\s+CORE REQUIRED:\s+(\d+)[%]?\s+MINOR OPTION\s+:\s+(\d+)[%]?\s+OPEN ELECTIVE:\s+(\d+)[%]?\s+2\. WHAT IS YOUR CURRENT STANDING IN THE UNIVERSITY\s+FRESHMAN\s+:\s+(\d+)[%]?\s+SOPHOMORE\s+:\s+(\d+)[%]?\s+JUNIOR\s+:\s+(\d+)[%]?\s+SENIOR\s+:\s+(\d+)[%]?\s+GRADUATE\s+:\s+(\d+)[%]?\s+3\. RATE THE PACE OF THE COURSE\s+VERY FAST\s+:\s+(\d+)[%]?\s+RATHER FAST\s+:\s+(\d+)[%]?\s+MODERATE\s+:\s+(\d+)[%]?\s+RATHER SLOW\s+:\s+(\d+)[%]?\s+VERY SLOW\s+:\s+(\d+)[%]?\s+4\. RATE THE WORK LOAD OF THE COURSE\s+VERY HEAVY\s+:\s+(\d+)[%]?\s+RATHER HEAVY\s+:\s+(\d+)[%]?\s+MODERATE\s+:\s+(\d+)[%]?\s+RATHER LIGHT\s+:\s+(\d+)[%]?\s+VERY LIGHT\s+:\s+(\d+)[%]?\s+USE THE FOLLOWING SCALE TO INDICATE LEVEL OF AGREEMENT:\s+\(SA\) STRONGLY AGREE, \(A\) AGREE, \(M\) MIXED FEELINGS,\s+\(D\) DISAGREE, \(SD\) STRONGLY DISAGREE\s+% OF NO\. EVAL.\s+SA\s+A\s+M\s+D\s+SD\s+NA\s+===\s+===\s+===\s+===\s+===\s+===\s+5\. INSTRUCTOR HAS AN EFFECTIVE COMMAND\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+6\. INSTRUCTOR SPEAKS AND WRITES CLEARLY\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+7\. EXPECTATIONS OF INSTRUCTOR ARE CLEAR\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+8\. COURSE PROCEDURES CLEARLY EXPLAINED\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+9\. ABLE TO MOTIVATE STUDENTS\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+10\. ENCOURAGING QUESTIONS AND DISCUSSIONS\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+11\. COURSE STIMULATES CRITICAL THINKING\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+12\. PROPER LEARNING ATMOSPHERE PROVIDED\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+13\. STUDENTS ARE INFORMED OF THEIR PROGRESS\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+14\. GRADING IS DONE FAIRLY\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+15\. ADEQUATE ASSISTANCE IS AVAILABLE\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+16\. TEXTBOOK\/CLASS MATERIAL ARE USEFUL\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+USE THE FOLLOWING SCALE TO INDICATE THE OVERALL RATING:\s+\(E\) EXCELLENT, \(VG\) VERY GOOD, \(G\) GOOD, \(F\) FAIR, \(P\) POOR\s+% OF NO\. EVAL\.\s+E\s+VG\s+G\s+F\s+P\s+NA\s+===\s+===\s+===\s+===\s+===\s+===\s+17\.\s+THE COURSE\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+18\.\s+THE INSTRUCTOR\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+19\.\s+THE TEACHING ASSISTANT \(IF APPLICABLE\)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+20\.\s+THE LABORATORY \(IF APPLICABLE\)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+\*\* COURSE FORM COUNT\s+:\s+(\d+)\s+/


This regular expression will match all occurences for reviews in the string. To figure out what maps to what, view any course review page and look for the variables. They should map up nicely.

After the regular expression has mapped up, you have TONS of data to start mining (I have a PHP serialize() output of the array if anyone wants it-- 24MB total size). My suggestion, make some database tables, write some queries, and start looking for whatever it is you desire.

If you are lazy, my PHP 5 script for parsing the course reviews page can be found on my Personal Code Repository.

What can this data be used for? Well, for starters, individual course reviews can be formatted to look nicer. On a grander scale, the data can be analyzed on many fronts. For instance, wouldn't it be cool to specify a department or professor and return a nice graph depicting that department's or professor's averaged ratings with respect to time or against another department? Do you want to know the top 10 rated professors of all time? The worst? Has the university's average course evaluation gone up since we started striving to become the "world's most powerful learning environment?" These answers and more will be brought to you. But first, I need to get some sleep.

As a disclaimer, I realize that the data obtained through the Registrar web site requires a valid Case username and password. I understand the reasons why. I will never post this data so that it is available to the outside world. If I were to post the data or an interface to the data, I would put it behind a system which requires network authorization.

You can ping this entry by using http://blog.case.edu/gps10/mt-tb.cgi/1362 .

In a previous entry I described the procedures to parse all 14000+ course reviews from the Registrar's web site. Since...