The literary category called ''mystery" has become an enormously subdivided phylum whose various classes, orders, families, genera, and species I will not even begin to expatiate on except to say that the germ is detective fiction. Ronald Knox, priest, satirist, writer, Sherlock Holmes expert, and translator of the Bible, brought his huge brain to bear on the matter of origins and identified the first detective mystery as Sophocles' ''Oedipus Tyrannus" (i.e., ''Oedipus Rex"). A period of some 2,300 years then elapsed until Edgar Allan Poe set forth ''The Murders in the Rue Morgue," after which the art form chugged along, picking up steam with Wilkie Collins and Arthur Conan Doyle, and positively hurtled into the 1920s, when it began to branch off in various and, to Knox, unorthodox directions.
I am only mildly interested in tracing the paths of the different forms of ''mystery" fiction or in distinguishing one group from the other with taxonomical accuracy; though, fortunately for the advance of human understanding, there are enough people who seem to do nothing else. In fact mysteries, loosely speaking, bring out the crackpot in readers, with Conan Doyle's tales of Sherlock Holmes continuing to inspire the most distinguished and fanatical. I find a bit of the latter quality in myself in that I abhor as sacrilege and abomination the plucking of Holmes from the original record to set him sleuthing in new novels and stories. But then I always bark at trespassers and am not even a true cognoscente. I only recently learned that the great Leslie Klinger had himself written a pastiche of a Sherlock Holmes tale, ''The Adventure of the Wooden Box."
And who, you may ask, is Leslie Klinger and where lies his greatness? He is the editor of the nine-volume ''Sherlock Holmes Reference Library" and the author of, among other works, the annotations that embellish ''The New Annotated Sherlock Holmes." The third volume of this magnificent work is now available (Norton, $49.95) and consists of the four titles: ''A Study in Scarlet," ''The Sign of the Four," ''The Hound of the Baskervilles," and ''The Valley of Fear.''
Two points must be made. In the first place, as Klinger notes in his preface, this is not a book for ''the serious student of Arthur Conan Doyle," by which he means that it is not for those who seek literary sources of the stories or reverberations in them of events in the life of their author. Or perhaps I should say ''author," for the other crucial point is what makes this work so utterly enchanting: ''Here," explains Klinger, ''I perpetuate the gentle fiction that Holmes and Watson really lived and that (except as noted) Dr. John H. Watson wrote the stories about Sherlock Holmes."
This conceit was first unloosed on the world by Knox in an article in which he used Watson's (and a certain ''deutero-Watson's") accounts as texts to satirize the literary criticism of his day (you can read it at: http://lachlan.bluehaze.com.au/books/knox_essays_in_satire/). The commentators quoted by Klinger throughout his annotation -- which runs down the margins and, at times, colonizes whole pages -- agree, if in little else, that we are dealing with biography and that Watson truly was Holmes's ''Boswell," as the master once called him. The inconsistencies, contradictions, and chronological irregularities that are inevitable in all serial fiction become clues here, fitting subjects for Holmes's own ''science of deduction and analysis." The alert investigator will find much revealed in the idlest detail, in the most roughly sketched background, and in similarities (plainly not inadvertent) of description. To Sherlockians, as these critics call themselves, nothing is simply the effect of authorial inattention or the hurried pen: not even the fact that Watson's wife calls him ''James." No, no -- surely she has chosen to call him by the English version of his middle name, which must be ''Hamish," though such is the complexity of the matter that there are other explanations and other possible middle names. Every detail is scrutinized, dissected, and evaluated, and many become the focus of terrific controversies whose cut and thrust, deftly summarized by Klinger, make of the margins entertaining dramatic narratives themselves.
In addition to all this piecing together of evidence, excogitation, and ingenious speculation, the margins are replete with historical detail, explanations of forgotten references, and what we shall have to call informed surmise. (''Perhaps the Master Detective's special allegiance to Leo XIII can be partly explained by the pope's awarding of a gold medal to a popular cocaine-based cocktail, Vin Mariani, also enjoyed by Queen Victoria and by Leo's successor, St. Pius X.") Additional appendices deal with such truly vexed matters as ''Was Richard Cabell 'Hugo Baskerville'?" and ''Who, Then, Is Porlock?" There are also tables and bibliographies, and, throughout the book, hundreds of illustrations from the original publications and subsequent translations, editions, and film versions of the novels.
I have just read Ruth Rendell's remarkable new novel, ''Thirteen Steps Down" (Crown, $25). Rendell is unquestionably heir to Sophocles as a mystery writer -- at least more so than Knox in her ideas of what constitutes a suitable subject for a story. But he would not approve of her novels as ''mysteries" in that they put far less emphasis on detection than on the psychological makeup of the criminal. They get just right what I take to be true: As social programs replace the ties of community and class, and as mass media homogenize and impoverish culture, society's aberrant souls become more bizarrely and brutally needy. Rendell's baddies are creepy: lonely, obsessed, monsters of self, bereft of cohesive identity. Their inner worlds are spiritually barren and aboil with morbid appetites.
In the case of the present novel, we have a character called Mix who craves celebrity, expecting to achieve it by becoming the consort of a famous model whom he stalks. He has also seized upon the murderer and necrophiliac John Reginald Christie as a hero. The novel begins with his visiting the place where Christie's notorious dwelling, 10 Rillington Place, once stood. Mix is appalled: ''Everything was new, carefully and soullessly designed. 'Soulless' -- that was the word and he was proud of himself for thinking it up." I call that brilliant. Page one, two sentences, and we already grasp the narcissism and banality of this fellow. I have no space to go on about how wonderful this novel is in character, plot, and atmosphere. It is a triumph and as I turned the last page I thought how even Knox would have loved its final denouement, a miracle of irony and compassion.
Katherine A. Powers, a writer and critic, lives in Cambridge. Her column appears on alternate Sundays. She can be reached by e-mail at email@example.com.