Wednesday, May 21, 2008

FICO scores and faulty fly-by-wire (FBW) control systems

[draft article in progress]

An A321 aircraft fly by wire cockpit, from the Wikipedia article “Flight Control Systems”, from which the following is excerpted:

Mechanical and hydro-mechanical flight control systems are heavy and require careful routing of flight control cables through the aircraft using systems of pulleys, cranks, wires and, with hydraulically-assisted controls, hydraulic pipes. Both systems often require redundant backup to cater for failures, which again increases weight. Furthermore, both have limited ability to compensate for changing aerodynamic conditions.
Dangerous characteristics such as stalling, spinning and Pilot-induced oscillation (PIO) can still occur with these systems and depend mainly on the stability and structure of the aircraft concerned rather than the control system itself.
By using electrical control circuits combined with computers, designers can save weight, improve reliability, and use the computers to prevent the undesirable characteristics mentioned above. Advanced modern fly-by-wire systems are also used to control otherwise unstable fighter aircraft. The words "Fly-by-Wire" imply an electrically-signalled only control system. The term is generally used, however, in the sense of computer-configured controls. This is where, between the operator and the final control actuator or surface, a computer system is interposed. This modifies the inputs of the pilot in accordance with software programmes. These are carefully developed and validated in order to produce maximum operational effect without compromising safety.
So goes the Wikipedia entry. It would be unethical to do any less than to craft these systems with increased pilot safety in mind, taking the onus off the pilot to have to correct for the shortcomings of the earlier manual systems. This is all well and good for the pilot.

“Maximum operational effect without compromising safety.” This sounds like an optimization problem with constraints. It is reminiscent of the work I used to do with least cost formulation of food products. There were USDA-regulated constraints on maximum fat content, ingredient limits, etc., for the product that could be thought of as analogous to the safety constraints in an aircraft flight control system. Feed in a lot of different material parameters and constraints along with tested values for various raw materials and the computer would come up with the least cost formulation for the product while adhering to the required constraints. This was back in the 1980s. The computer was an indispensable part of that work.

Now, fast forward 30 years. The computer, instead of simply being at our service in an increasing array of domains, has curiously been assigned the role of evaluating us humans by the machine values that we built computers for to assist us with in the first place. Having programmed computers myself, I can’t tell you how sick this is! A major case in point, a poster child for this inverted state of affairs, is the FICO scoring system, where a machines attempt to evaluate humans by machine values, rather than humans utilizing machines to optimize results. It’s as if the aircraft designers in the opening part of this article decided to make a flight control system whereby the control system would rate the human by its own machine parameters, without being programmed to assist that human in any fashion. If the control system determined that the human wasn’t manually compensating well enough for changing aerodynamic conditions, rather than attempt to compensate for that failure, it would execute a command sequence to crash the plane. It would have such “gotchas” designed-in, and the designers and their colleagues, standing in the sidelines, would nervously smile whenever one of those gotchas kicked in.

Sort of like the Freddie Mac chief economist who I saw on CSPAN a few weeks back. He mentioned that not only were the shock payments of an ARM that abruptly kicked in a causal factor in the foreclosure crisis, but that actually many of the so-called subprime borrowers got into trouble even before that point in their mortgage timeline because their loans were amazingly not set up for escrow. So, he continues, the very people who would be least likely to be able to handle an irregular monthly payment would have these large insurance and tax payments sometimes come due simultaneously in the same month in addition to their regular mortgage payments. He sort of nervously chuckled at this as it wasn’t hard for him to see that such a state of affairs would lead to problems, that those problems were in some manner predictable, and that those predictions would have been born out, obviously, by the data people were now seeing.

So, in juxtaposition to these spectacular crash and burns, we have the mild amusement of the economist, an amusement which may stand in for a level of discomfort at seeing the results of these policies in human terms. In another corner we have those who flock to sites like myfico.com to try and find out how they might maximize their FICO score and thereby minimize the negative financial consequences of a low score. However, what is masked here, besides the proprietary algorithm of the scoring system itself, is the inescapable conclusion, if one stops to consider things carefully, that there really are positive (self-reinforcing) feedback loops in place, as in the imaginary “rigged” avionics system described above. Sure, an individual pilot could, with planning and forethought, be able to compensate for such a system and even make it work well under relatively normal conditions. But it is the lack of design for safety (what happens the one time the manual process is precluded during an anomalous situation) that may eventually come into play, and catch even an experienced pilot unawares.

The prime factor on which credit is determined is becoming more and more ridiculous. That is this whole concept of “paying on time”. Why do humans need to get involved at all with initiating individual monthly transactions? We have computers that can transfer money optimally “on time”, with transactions timed to the millisecond and reliably synchronized with the world’s most accurate atomic clocks. No human can exceed this level of accuracy. It’s not physically possible, just as it’s not possible to fly the Space Shuttle without the aid of its cockpit computers (computers built some 40 years ago, I might add). So, we give computers an 850 FICO score, right? So, it’s not surprising that the Space Shuttle would get a lower score for flightworthiness if you were to yank out most of its computers. It’s similarly not surprising that more people end up with lower scores for creditworthiness when you take away automated transaction systems for them such as escrow--systems which are available to people at the higher end of the economic spectrum. And then, to add insult to injury, computers are used to track these scores throughout the course of a lifetime.

Erich Fromm so writes about the human worker in The Sane Society: “He is part of the machine, rather than its master as an active agent. The machine, instead of being in his service to do work for him…has become his master.” Applied here, these words ring true more than 50 years after Fromm wrote them. However, Fromm was referring primarily to the industrial worker that passage. Fromm continues, “Instead of the machine being the substitute for human energy, man has become a substitute for the machine. His work can be defined as the performance of acts which cannot yet be performed by machines.” My thesis here is that the situation is even more disturbing than that today. Now, much repetitive work could be performed by machines, but the machines are rather employed in evaluating humans instead, modern avionics systems notwithstanding. These humans are put to the service of the machine’s scoring and classifying programs. In the book “Sorting Things Out: Classification and Its Consequences”, the authors write that “decades ago, Max Weber wrote of the iron cage of bureaucracy. Modern humans, he posited, are constrained at every juncture from true freedom of action by a set of rules of our own making…Information infrastructure adds another level of depth to that iron cage. In its layers, and in its complex interdependencies, it is a gossamer web with iron at its core.” They note that “the moral questions arise when the categories of the powerful become the taken for granted; when policy decisions are layered into inaccessible technological structures; when one group’s visibility comes at the expense of another’s suffering.”

It’s that question of power that further drives the feedback loop I referred to earlier. It is a faulty avionics system that does not correct for pilot-induced-oscillation, that does not attempt to make a death spiral less likely. A classification system that is set up to reward the powerful while further penalizing the powerless is a faulty and immoral one. Not only in a simple economic sense, but also when taken as a measure of human worth, as when people are denied employment due to their assigned credit scores. People who feel powerless often become depressed. In the following two publications on depression that is suggested in their actual titles: “Depression: The Evolution of Powerlessness” by Dr. Paul Gilbert and “Depression: A Disorder of Power” by Dr. Susan Heitler. The accelerating downward spin of the aircraft is only stopped by regaining control, by exercise of power over the situation. Likewise, the downward spin of depression is stopped by regaining control, by being able to “have one’s concerns addressed”, as Dr. Heitler states.