Technorati Profile
The Problem With CORS
There is a brewing storm of dissatisfaction over the bidding system of NUS, known as CORS (petition one, petition two). As far as I know of, there has been two major disruptions in the bidding so far. The first one occurred during module bidding 2A, and the second one during tutorial balloting 1A.
CORS Introduction
Before delving into the details, let me attempt to summarize how the bidding system works. The information here is based on three years of my understanding, and should be accurate where it matters most.
First of all, understand that module allocation in NUS is unique, as it is facilitated by online bidding via CORS. Students are given bidding points in two accounts every two years. These two accounts are known as the Programme Account and General Account. Points in the programme account are used to bid for compulsory modules in each student's track (core modules). Points in the general account are used for non-core modules. Students have a limited number of points in both accounts, and can decide whether to spread them thin, or concentrate on a few select modules.
There are three major rounds of module bidding, discounting the preparatory round 0. In round 1, students place bids for their core modules, and are allocated core modules at the end of each subround 1A, 1B and 1C. Students who missed getting core modules in round 1 can continue to bid for them during all subsequent rounds. In round 2, bidding for non-core modules begin, and non-core modules are similarly allocated based on bids at the end of subrounds 2A and 2B. Finally, round 3, which has 6 subrounds A thru F, will take place. Students may bid for all modules during this round, at the end of which electronic bidding officially ends.
During each subround (1A, 1B, 2A etc), there are two phases of bidding. Open bidding, typically beginning from the start of the subround to two hours before it ends, provides information on the number of vacancies, number of bidders, the highest bid and the minimum bid required to be allocated the module. Close bidding, typically beginning from two hours before the subround ends, hides information on the highest bid and the minimum bid required to be allocated the module. The minimum bid required is usually the most important information, because no matter how high a bid is, extra points above and beyond this minimum bid are refunded to the student. Hence, if the minimum bid stands at 100, and I've invested 1000 points in this module, I'll be refunded 900 points at the end of the subround.
Tutorial balloting occur in a different mechanism. There are 2 major rounds of balloting, and 4 subrounds. Based on the modules that students are allocated, a list of available tutorials will be displayed when tutorial balloting begins. Students arrange their options according to preference, and tutorials are assigned to each student at the end of each subround.
This summary as well as timings and dates for all the various rounds are made public on the CORS website. Some details may be missing from my summary, but it should nonetheless be functionally accurate.
Excellent Premise
CORS was built for an excellent purpose: to train students to understand the opportunity costs of their actions. Because points in each account are limited, students have to make a choice as to whether play it safe and invest them in one popular module, or to spread them over many other modules. The close bidding mechanism removes perfect knowledge and entire certainty, which almost never occurs in the real world. The system hence advocates responsible decision making, careful resource allocation, rewards vigilance and even the occasional risk taking.
Such lessons are invaluable. With limited funds, a fund manager has to decide what to invest in and how much. With the proper analysis, the fund manager obtains an approximation of the return on investment, and yet still faces an element of risk when he finally makes his decision. Project managers, programmers, store keepers, indeed almost every strata of society faces this scenario at one time or the other, whether the scenario has to with which projects to initiate, which technology to learn, which product to stock up on.
The student who takes CORS seriously can hence pick up several lessons that cannot be imparted in traditional balloting or first-come-first-served systems. I for one, learnt that it is not a good idea to show my hand at the very beginning of open bidding, unless I am extremely confident of winning the bid. Otherwise, my action will only drive up the minimum bid of that module, making it more costly to obtain that module. I learnt to anticipate competitors as well, to find out the minimum and highest bid just before open bidding ends, and to make allowances for my own bids when close bidding begins.
Like I've said, those are excellent lessons, and CORS succeeds admirably as a simulation and pseudo decision support system.
So What's Wrong?
For all its noble intentions, many NUS students are unhappy with the system. Just this semester alone, CORS has twice been plagued by downtime during bidding and balloting. Support personnel have been hounded by furious students, possibly leading them to provide curt and non-commital replies when questioned on the next step to take. Based on personal experience as well as a perusal of the two petitions, I offer some bugbears here below.
a) Critical Bottlenecks
Bandwidth is important- this we all know. Combine this with a burgeoning student population, and something's got to give. A recent study of British websites found that 25% of website overloads and crashes occurred due to poor planning. Website owners failed to account for incomplete transactions, and user journeys through the websites were not properly monitored. This is more than a problem of increasing bandwidth however, for if the website was not designed properly, the doubling of hardware will not result in a doubling of performance. In particular, the study notes that "with knowledge of the journeys [of website users] and the likely load levels, sensible code refactoring and configuration tweaking can give an order of magnitude throughput gains at the critical bottlenecks".
The bottleneck in CORS presumably is two-fold. First of all, students who logon and do not log off may have their session information stored on memory, depending on the website design. Should this be a prevalent behaviour, then CORS is a disaster waiting to happen. The second bottleneck could lie in the calculation of bid transactions. While transaction volume per student may be unimpressive, the entire volume can be hefty enough to cripple the system. Students who reload the webpages to view updated information also contribute to the load, even though designers may have overlooked them during planning.
These are however merely my speculations, and will remain as such.
b) CORS World vs Real World
After the downtime during module bidding 2A, students were informed by school administrators that the system failure was caused by a massive number of students logging in at the last few hours of open bidding. After the downtime during tutorial balloting 1A, students were informed by school administrators that the failure this time was caused by a massive number of students logging in immediately once balloting began, even though tutorials were not balloted on a first-come-first-served basis.
These two events combined to paint a picture that the CORS world does not mirror the real world, no matter how much its designers want it to be.
In the CORS world, students who are vigilant enough to obtaining bidding information just before open bidding ends are rewarded. In the real world, these students crash the system, although they were doing what CORS advocated in the first place.
In the CORS world, students will stagger themselves over a normal probability distribution when tutorial balloting begins, because the system is not based on a first-come-first-served principle. In the real world, students log in early, so that this chore of preference indication is settled early, and also because the last minute rush during the previous fiasco is still fresh in their minds. Again, they crash the system.
In the CORS world, students who for some reason fail to receive a module can lodge an appeal. In the real world, many students skip the bidding, and directly lodge appeals at the end of the rounds. This allows them to obtain "free" modules directly, without sacrificing the minimum bid.
In all these cases, behaviour of the end users of CORS clearly deviated from the designers' and adminstrators' expectations. Perhaps more work is needed in tracking student behaviour, logon times, frequency and patterns, because end users seldom work in the way we expect them to.
c) Administrative Mistakes
When CORS failed during the first downtime, the CORS helpline was inundated with calls. It was almost impossible to get help via both phonecalls or emails. When the lucky few did manage to get through, the response from the support team was less than helpful. Word got around that "nothing could be done" because it was the "fault" of the students. Everyone was advised to simply "wait and see".
When you've been hitting refresh again and again for the past three hours, despite knowing you're contributing to the problem, this is the last thing you want to hear. Module selection is an important activity in a student's academic life. "Wait and see" is infinitely less preferable to reassurance that the incident will be handled, and that affected students will be attended to.
When the emails finally arrived, they came with no explanations, no apologies, and one even carried a remonstrative undertone. A PR nightmare if nothing else.
d) End User Dissatisfaction
Perhaps the most insidious threat facing CORS now is not hardware or even performance in nature. In software projects, user acceptance and satisfaction is a major hurdle. Multi-billion projects have failed because of this, and project management literature is rife with warnings.
Being an academic system, I like to think that CORS has a natural advantage over corporate systems. Unless the entire student body of NUS refuses to use the system, the few who do not go with the flow do so at their own disadvantage.
Yet dissatisfaction simmers. Comparisons with the NTU system are made and disgruntled students make caustic jibes at NUS's recent 18th international ranking. Had participation in CORS been voluntary, I dare say it'll be headed the way of website graveyard, despite its good design and purpose.
Conclusion
As I've stated before, I believe CORS is a good system. It teaches responsibility, resource allocation and opportunity costs, something few module allocation systems can boast of. Nonetheless, various performance flaws threaten its effectiveness. More importantly, NUS needs to do more to win over students where CORS is concerned. It can always choose the autocratic way by offering neither explanations nor apologies and dismissing these two incidents as isolated ones. But while that route may solve its short term woes, it is ultimately untenable in the long run.
Filed under: Singapore, Education, NUS, CORS
CORS Introduction
Before delving into the details, let me attempt to summarize how the bidding system works. The information here is based on three years of my understanding, and should be accurate where it matters most.
First of all, understand that module allocation in NUS is unique, as it is facilitated by online bidding via CORS. Students are given bidding points in two accounts every two years. These two accounts are known as the Programme Account and General Account. Points in the programme account are used to bid for compulsory modules in each student's track (core modules). Points in the general account are used for non-core modules. Students have a limited number of points in both accounts, and can decide whether to spread them thin, or concentrate on a few select modules.
There are three major rounds of module bidding, discounting the preparatory round 0. In round 1, students place bids for their core modules, and are allocated core modules at the end of each subround 1A, 1B and 1C. Students who missed getting core modules in round 1 can continue to bid for them during all subsequent rounds. In round 2, bidding for non-core modules begin, and non-core modules are similarly allocated based on bids at the end of subrounds 2A and 2B. Finally, round 3, which has 6 subrounds A thru F, will take place. Students may bid for all modules during this round, at the end of which electronic bidding officially ends.
During each subround (1A, 1B, 2A etc), there are two phases of bidding. Open bidding, typically beginning from the start of the subround to two hours before it ends, provides information on the number of vacancies, number of bidders, the highest bid and the minimum bid required to be allocated the module. Close bidding, typically beginning from two hours before the subround ends, hides information on the highest bid and the minimum bid required to be allocated the module. The minimum bid required is usually the most important information, because no matter how high a bid is, extra points above and beyond this minimum bid are refunded to the student. Hence, if the minimum bid stands at 100, and I've invested 1000 points in this module, I'll be refunded 900 points at the end of the subround.
Tutorial balloting occur in a different mechanism. There are 2 major rounds of balloting, and 4 subrounds. Based on the modules that students are allocated, a list of available tutorials will be displayed when tutorial balloting begins. Students arrange their options according to preference, and tutorials are assigned to each student at the end of each subround.
This summary as well as timings and dates for all the various rounds are made public on the CORS website. Some details may be missing from my summary, but it should nonetheless be functionally accurate.
Excellent Premise
CORS was built for an excellent purpose: to train students to understand the opportunity costs of their actions. Because points in each account are limited, students have to make a choice as to whether play it safe and invest them in one popular module, or to spread them over many other modules. The close bidding mechanism removes perfect knowledge and entire certainty, which almost never occurs in the real world. The system hence advocates responsible decision making, careful resource allocation, rewards vigilance and even the occasional risk taking.
Such lessons are invaluable. With limited funds, a fund manager has to decide what to invest in and how much. With the proper analysis, the fund manager obtains an approximation of the return on investment, and yet still faces an element of risk when he finally makes his decision. Project managers, programmers, store keepers, indeed almost every strata of society faces this scenario at one time or the other, whether the scenario has to with which projects to initiate, which technology to learn, which product to stock up on.
The student who takes CORS seriously can hence pick up several lessons that cannot be imparted in traditional balloting or first-come-first-served systems. I for one, learnt that it is not a good idea to show my hand at the very beginning of open bidding, unless I am extremely confident of winning the bid. Otherwise, my action will only drive up the minimum bid of that module, making it more costly to obtain that module. I learnt to anticipate competitors as well, to find out the minimum and highest bid just before open bidding ends, and to make allowances for my own bids when close bidding begins.
Like I've said, those are excellent lessons, and CORS succeeds admirably as a simulation and pseudo decision support system.
So What's Wrong?
For all its noble intentions, many NUS students are unhappy with the system. Just this semester alone, CORS has twice been plagued by downtime during bidding and balloting. Support personnel have been hounded by furious students, possibly leading them to provide curt and non-commital replies when questioned on the next step to take. Based on personal experience as well as a perusal of the two petitions, I offer some bugbears here below.
a) Critical Bottlenecks
Bandwidth is important- this we all know. Combine this with a burgeoning student population, and something's got to give. A recent study of British websites found that 25% of website overloads and crashes occurred due to poor planning. Website owners failed to account for incomplete transactions, and user journeys through the websites were not properly monitored. This is more than a problem of increasing bandwidth however, for if the website was not designed properly, the doubling of hardware will not result in a doubling of performance. In particular, the study notes that "with knowledge of the journeys [of website users] and the likely load levels, sensible code refactoring and configuration tweaking can give an order of magnitude throughput gains at the critical bottlenecks".
The bottleneck in CORS presumably is two-fold. First of all, students who logon and do not log off may have their session information stored on memory, depending on the website design. Should this be a prevalent behaviour, then CORS is a disaster waiting to happen. The second bottleneck could lie in the calculation of bid transactions. While transaction volume per student may be unimpressive, the entire volume can be hefty enough to cripple the system. Students who reload the webpages to view updated information also contribute to the load, even though designers may have overlooked them during planning.
These are however merely my speculations, and will remain as such.
b) CORS World vs Real World
After the downtime during module bidding 2A, students were informed by school administrators that the system failure was caused by a massive number of students logging in at the last few hours of open bidding. After the downtime during tutorial balloting 1A, students were informed by school administrators that the failure this time was caused by a massive number of students logging in immediately once balloting began, even though tutorials were not balloted on a first-come-first-served basis.
These two events combined to paint a picture that the CORS world does not mirror the real world, no matter how much its designers want it to be.
In the CORS world, students who are vigilant enough to obtaining bidding information just before open bidding ends are rewarded. In the real world, these students crash the system, although they were doing what CORS advocated in the first place.
In the CORS world, students will stagger themselves over a normal probability distribution when tutorial balloting begins, because the system is not based on a first-come-first-served principle. In the real world, students log in early, so that this chore of preference indication is settled early, and also because the last minute rush during the previous fiasco is still fresh in their minds. Again, they crash the system.
In the CORS world, students who for some reason fail to receive a module can lodge an appeal. In the real world, many students skip the bidding, and directly lodge appeals at the end of the rounds. This allows them to obtain "free" modules directly, without sacrificing the minimum bid.
In all these cases, behaviour of the end users of CORS clearly deviated from the designers' and adminstrators' expectations. Perhaps more work is needed in tracking student behaviour, logon times, frequency and patterns, because end users seldom work in the way we expect them to.
c) Administrative Mistakes
When CORS failed during the first downtime, the CORS helpline was inundated with calls. It was almost impossible to get help via both phonecalls or emails. When the lucky few did manage to get through, the response from the support team was less than helpful. Word got around that "nothing could be done" because it was the "fault" of the students. Everyone was advised to simply "wait and see".
When you've been hitting refresh again and again for the past three hours, despite knowing you're contributing to the problem, this is the last thing you want to hear. Module selection is an important activity in a student's academic life. "Wait and see" is infinitely less preferable to reassurance that the incident will be handled, and that affected students will be attended to.
When the emails finally arrived, they came with no explanations, no apologies, and one even carried a remonstrative undertone. A PR nightmare if nothing else.
d) End User Dissatisfaction
Perhaps the most insidious threat facing CORS now is not hardware or even performance in nature. In software projects, user acceptance and satisfaction is a major hurdle. Multi-billion projects have failed because of this, and project management literature is rife with warnings.
Being an academic system, I like to think that CORS has a natural advantage over corporate systems. Unless the entire student body of NUS refuses to use the system, the few who do not go with the flow do so at their own disadvantage.
Yet dissatisfaction simmers. Comparisons with the NTU system are made and disgruntled students make caustic jibes at NUS's recent 18th international ranking. Had participation in CORS been voluntary, I dare say it'll be headed the way of website graveyard, despite its good design and purpose.
Conclusion
As I've stated before, I believe CORS is a good system. It teaches responsibility, resource allocation and opportunity costs, something few module allocation systems can boast of. Nonetheless, various performance flaws threaten its effectiveness. More importantly, NUS needs to do more to win over students where CORS is concerned. It can always choose the autocratic way by offering neither explanations nor apologies and dismissing these two incidents as isolated ones. But while that route may solve its short term woes, it is ultimately untenable in the long run.
Filed under: Singapore, Education, NUS, CORS
0 Comments
Post a Comment
<< Home