Writing About Espionage Secrets

This article describes the author’s experiences researching three books on espionage history in three different countries and on three different topics. The article describes the foreign intelligence arm of the Ministry for State Security; a global history of secret writing from ancient to modern times; and finally, my current project on U.S. intelligence and technology from the Cold War to the War on Terror. The article also discusses the tensions between national security and openness and reflects on the results of this research and its implications for history and for national security.


Writing about Secrets
Historians need sources -primary sources, secondary sources, interviews, archival material, and artifacts. Yet intelligence is meant to be secret, at least for a time. Even when secrets are exposed or declassified, every effort is made to keep methods and sources secret. As a result, writing about intelligence history is challenging, to say the least. Not only is it frustrating, but the finished product might not meet historical expectations regarding what a work should be in terms of completeness and narrative cohesiveness. Unfortunately, historians might produce just one small aspect of the topic resembling Swiss cheese, rather than the flowing river a narrative piece of history should be.  Macrakis (2008;2014) and Techno-Spies and the Decline of the Secret Agent, in progress.

Seduced by Secrets
Most people probably remember the storming of the "Stasi" (the East German Ministry for State Security) Headquarters building shortly after the fall of the Berlin Wall in 1989. The iconic images have been displayed again and again in the media. Even though these images show files being secured by citizens and other people, most people don't know that the foreign intelligence (like the U.S. Central Intelligence Agency, or CIA, and the UK's Secret Intelligence Service, commonly known as MI6) files of the Ministry for State Security -the Hauptverwaltung Aufklärung (HVA) -were actually legally destroyed in January of 1990 with the justification that every country has secret foreign intelligence agencies, and this information would never be made public in other countries (and as far as I know, the legality of the file destruction was never debated or contested).
The reason many people would conclude that the "Stasi Archives" would be the richest source of material for an intelligence historian is because miles and miles of files became available after the collapse of communism. And the image of miles and miles of files was also displayed on television. It is not widely known, however, that most of the files were internal domestic operation files.
And therein lies the problem for intelligence historians and for this historian as I set out to write a history of the HVA's scientific-intelligence acquisition operations. But before the reader should despair along with me, a dramatic discovery was to be made later in my research. I'll come to that in a minute.
In the meantime, when I began my research, I had to rely on using the domestic, internal files that sometimes also carried out scientifictechnical intelligence, and in the early years they often overlapped. I also started to interview former intelligence officers, defectors, and agents-all of whom were not ready to talk because of fear of imprisonment. Not surprisingly, a defector (along with a couple of agents) -Werner Stillerproved to be the best source at the beginning of my project, along with some crumbs from the archives. The most challenging, and least helpful, at first, were senior intelligence officers and leaders, who clung to their secrecy oaths and fear of prosecution. The one exception here was Markus Wolf, the legendary spy chief of the HVA, who served as the public spokesman for the HVA. I got a lucky break when an archivist alerted me to a case file about an agent -a West German businessman who spied for the Stasi, codenamed "Gorbachev" -who was run like a foreign intelligence agent, but because of a bureaucratic quirk and the overlap of internal/domestic and foreign intelligence, his file was preserved.
It was not until 1998 that the so-called "Stasi Archives" -the Federal Commissioner for the Records of the State Security Service of the former German Democratic Republic (Der Bundesbeauftragte für die Unterlagen des

Staatssicherheitsdienstes der ehemaligen Deutschen Demokratischen
Republik, or BStU) discovered a massive database called SIRA (System der Informationsrecherche) with information gathered by the HVA that another piece of the foreign intelligence puzzle could be put in place.
In 1990, the CIA performed one of the biggest coups by acquiring, through methods not totally known, the complete HVA's agent card filescalled the Rosenholz files -with code names and real names, but they did not make their find public. Only security officials could use them. Finally, the BStU acquired the German agent cards and I was able to use them in 2006.
My book finally came out in 2008. From then on, it was really more anthropology than history, as I pieced together the agent networks though court files fleshed out the facts.
In addition to the card files, information reports and court cases, I also visited the German Bundeskriminalamt's evidence collection and museums to photograph and include artifacts. 2 Even when archives are open, however, rarely can one get a case file about an agent, especially in the United States, because these sources are heavily guarded -unless, of course, the agent has been uncovered and disclosed by the enemy and an agency wants to show how successful that agent was while operational. Oleg Penkovsky is a case in point. files do not include the flesh-and-blood agent files. For that, the court case files were essential, along with several available foreign agent files.
Interviews with agents, defectors, leaders, and counter-intelligence officials added some historical detail and color not available in the files.
In the end, the sources did provide enough material to illuminate scientific-technical intelligence. In fact, together the SIRA database and the Rosenholz files are a treasure trove for certain types of research. The SIRA database confirmed the importance of computer espionage during the last decade of the German Democratic Republic's existence, and of course, the Rosenholz files revealed the agents' identity and motivations. Researchers working now will obviously not have to wait as long to obtain these rich sources discovered in the decade after the fall of the wall.
It should be noted here for readers familiar with the so-called Stasi surveillance state and the ubiquity of the informant, that foreign intelligence agents and domestic informants are different types. Domestic informants belong to the secret police arm of the Ministry for State Security. Sometimes there was some overlap when a foreign intelligence agent also spied for state security, but this distinction needs to be pointed out, because the secret police aspect of the Stasi is much more widely known.
The process of uncovering these secrets was indeed like peeling an onion. It was a process of discovery extending over many years. And, of course, revealing intentionally concealed secrets is usually not a quick process. The sad part of the story is that the foreign intelligence files were destroyed in order to protect the identity of the agents. The Rosenholz files, however, along with some defector information, made agent names available to counter-intelligence officials and prosecutors in the West. As a result, many were convicted and sent to prison. In the end, both historians and agents were losers because the archival material no longer exists, but the agents were prosecuted nevertheless.
In this case, concealing the agents' identity and keeping it secret was done to protect the services sources. This seems like a perfectly reasonable wish in the game of spy versus spy, however frustrating for the historian, who needs the files to weave together a rich, colorful, and accurate narrative. Reasonable secrecy is often at odds with the disclosure necessary for a historian, and for the appetite of the public in general. So while secrecy or concealment is sometimes used to hide illegal processes or wrongdoing and therefore seen as something negative, for intelligence agencies, secrecy is their lifeblood. Disclosure and leaks are the bane of their existence!

Global History of Secret Writing/Invisible Ink
It was fun shifting from the dark recesses of German history to writing a global history of invisible ink from ancient Greece to modern times. Here the challenges were quite different. At first I thought there was nothing, or very little, to find on the subject. There was no book on the subject, and there was very little other secondary source material. In part, this was because of the emphasis on codes and ciphers in the literature and, in part, because in earlier periods of history, it was indeed harder to research. It left fewer visible signs.
For my own research, I soon discovered some hidden articles, but the real mother lode was Google books and electronic resources for the early modern period and the eighteenth century, and the UK National Archives, which housed original manuscripts from the early modern period as well as rich agent files from MI5. The latter proved an exceptionally rich source for agent material. I chose to write the history using stories to illustrate the general history.
Of course, traditional sources like rare book and manuscript rooms were enormously helpful. In rare book rooms like Harvard University's Houghton Library, the library allowed me to examine the actual manuscripts or rare books in order to examine them for the telltale signs of invisible ink.
This was also the case for the chapter on the American Revolution, for which I used material from the Huntington Library, the U.S. National Archives and Records Administration, and rare book dealers. Although I could bring in an ultraviolet light to illuminate some types of invisible ink, I was not allowed, of course, to deface old documents by brushing them with a reagent-a chemical used to reveal invisible ink. I was quite surprised that earlier researchers actually used chemicals to reveal invisible ink in documents from the American Revolution (the papers were still in private possession and therefore not subject to federal state archives rules). The problem, of course, with researching invisible ink, is that I always had the fear that the really important stuff was still hidden and hadn't been detected yet. Even though a lot of the material was hidden, modern-day technology has allowed us to reveal previously hidden writing using UV lamps and other technologies, as pointed out. But the beauty and effectiveness of invisible ink is that unless there is suspicion that a document might contain the substance, it remains hidden, unlike codes and ciphers, which announce they are keeping a secret. In some ways, secret writing is a metaphor for the world of secrets in general. What is revealed, what is concealed? To what extent is the revealed material an accurate representation?
Using electronic sources in addition to traditional ones really helped speed up the research. I also found stories and material I would not have found using key-word searches. For example, some of the material about sympathetic ink during the Enlightenment period came from this approach by providing me with leads about material to find, though only a little of it was about espionage.
I found the UK National Archives the richest source of material about secret writing and the agents who used it. Of course, MI5 (the UK's Security Service, Military Intelligence, Section 5) is more akin to the FBI than the CIA -the latter being concerned with foreign intelligence-but they worked on catching spies, and invisible ink was part of that process because it was evidence. But as other historians like Ben Macintyre (2010) have found, the MI5 files are an extraordinary, rich source indeed.
In contrast to the very long time it took to pry out the secrets of the Stasi and its foreign intelligence arm, the invisible ink book ended up being easier and quicker to write because of a combination of traditional sources and digital sources. Most of the project was also deeper in the recesses of history. Some of the challenge of writing about modern intelligence history is that it is so recent, but of course, that is an advantage also because one can interview still-living sources.
The topic of invisible ink also touches upon the issue of protecting methods and sources. In this instance, invisible ink is a very secret method, one modern spy agencies fiercely protect. The CIA's rationale for keeping the material secret was that they could be used as building blocks for current systems. It was not until 2011 that the CIA declassified material from World War One invisible ink residing in the US National Archives.
Apparently, these comprised the oldest classified material in the US National Archives.
Despite the fact that the CIA jealously guarded those secrets, I was able to circumvent these secrecy restrictions by using material from the Stasi files. Along with chemistry colleagues, I was able to reproduce the Cold War formulas, material the CIA would never release because it was so recent. The caveat in the Freedom of Information Act (FOIA) denies material relating to methods and sources.

U.S. Intelligence History
Surprisingly, the United States also has a rich source of declassified material. Without keeping you in suspense here, I think it is because of our peculiar form of democracy and our First and Fourth Amendment rightsincluding the heavy role of the media, leading to leaks. There doesn't seem as much room for secrecy in our form of democracy as there is in the UK with its Official Secrets Act and stiff penalties for disclosure, or Germany: However, it should be noted that the United States also overclassifies a lot of material. As Peter Galison (2004) points out, overclassification leads 3 P.L. 89-487, An Act to Amend Section 3 of the Administrative Procedure Act, Chapter 324, of the Act of June 11, 1946 (60 Stat. 238), to Clarify and Protect the Right of the Public to Information, and for Other Purposes. 4 The CREST archive is located at http://www.foia.cia.gov/collection/crest-25-yearprogram-archive to the removal of of knowledge, especially scientific-technical knowledge.
When I write that the United States does not have as much room for secrecy as England, I mean that the United States has more leaks and public discussion of secrets. It also has declassified more intelligence material than those countries. England might not have classified as much material, but it also does not release much material from MI6 (in contrast to MI5), and there is a stiffer sentence for leakers (Galison, 2004).
In the case of World War II, a lot of rich material was released from OSS files as part of the Nazi documents act, and I found great material in the National Archives FBI files for the invisible ink book.
The National Security Archive at George Washington University also uses FOIA (often suing the government in the process) to declassify previously secret intelligence material with areas of foci including Cuba, the Bay of Pigs, and the Cuban Missile Crisis. Both their website and electronic source material are very rich sources for Cold War historians. Although intelligence history is not as well institutionalized at universities in the United States as it is in the United Kingdom, the CIA does have its Center for the Study of Intelligence. 5 Another rich source in the United States is a long tradition of memoir writing. It seems paradoxical for them to spill the beans, but the spy chiefs have done a lot of it along with defectors and agents. The United Kingdom and Germany don't produce as many memoirs (though some are notable, such as Markus Wolf's, but his service was dismantled). Another way to find 5 Found at https://www.cia.gov/library/center-for-the-study-of-intelligence 12 Secrecy and Society, Vol. 2, No. 1 [2018], Art. 7 https://scholarworks.sjsu.edu/secrecyandsociety/vol2/iss1/7 DOI: 10.31979/2377-6188.2018.020107 sources for research is through the Association of Former Intelligence Officers (AFIO). 6 Finally, leakers and whistle-blowers have provided explosive material relating to U.S. Intelligence. The most notable recent leaker, of course, is Edward Snowden who leaked damning and revealing slides and files to the press and to the world in 2013. The material offers a treasure trove of material documenting the National Security Agency's (NSA) domestic and global operations.
The juiciest material, of course, is the sources-the agents. Although many Soviets who spied for the United States have been uncovered and written about, the same is not true of many important agents who spied for the Soviets. I have found out through first-hand experience that the CIA will never reveal to me directly the identity of the Cuban nationals who spied for the CIA because of strict source protection. The CIA tells me that even if they are dead, they might have family alive. Revealing their names would discourage future sources because case officers assure prospective agents that their identity will remain hidden forever.
After all, once the secret material has been declassified, it is often not as exciting as the researcher would hope. Its allure is often the historian's quest to find hidden material that would illuminate the problem. Once found, the previously hidden secret is often paltry. One example of a let-down would be those invisible ink formulae finally declassified in 2011. Much to my dismay, I already knew the content of the material through other sources, some of it from the British National Archives, and some of it from published articles and books (many of which were obscure and only consulted by a historian searching for specific material).
So, that's a cursory summation of some of the types of sources available in different countries. The scene isn't quite as empty as one might expect, given the secrecy surrounding the subject. However, huge holes remain, and it is inadvisable to encourage graduate students interested in this research unless a large body of material has already been declassifiedwhich is often the case with U.S. and UK records-as it is a long road to the house of secrets.

Some Reflections on Secrecy and Historical Research
The purpose of this article was to describe the frustrations and high points of conducting research with formerly secret materials. The topic naturally lends itself to reflections on the nature of secrecy, what is kept secret, and whether the information should be kept secret. What did this material reveal that was noteworthy? Should it have been kept secret?
There is clearly a tension between an intelligence agency's need for keeping methods and sources secret and a historian's or journalist's need for openness. The compromise to this dilemma is probably the one reached in the United States, whereby intelligence material can be declassified after a period of time (usually exempting sources and methods).
The problem for a historian with keeping intelligence agency secrets locked up is that a historical period or episode can be misunderstood without the missing piece of information. One glaring example is that of the so-called VENONA files. The VENONA project was a four-decade counterintelligence The release of these files suggest that there is a right time to declassify material without jeopardizing sources and methods. The case of the Stasi is another example of this notion. The Stasi case was unprecedented, because it is unusual for a country to disappear along with its intelligence agencies! The sad part of this story is that the East German foreign intelligence unit of the Ministry for State Security was allowed to destroy its files using the protection-of-sources-and-methods "mantra" so common among intelligence professionals. In the end, as undestroyed or purloined files became slowly available, the agents were uncovered, caught, and sentenced to prison (although the sentences were not as long as spies get in the United States, for example). (And although it initially took me a while, as a researcher, to obtain this material, the wait is now not as long for others wishing to use them; however, the case files themselves have been largely destroyed.) Despite the wait, the Stasi files proved a very rich source of material. For my own subject of scientific-technical espionage, it suggested that, surprisingly, espionage did help East Germany in the short term but in the end stifled true creativity and led to the implosion of East Germany itself, because of its reliance on Western computer technology.
Finally, I should mention one more example of forcing the CIA to declassify material it would never have declassified because it revealed methods and sources. Unusually, OSS material and early Cold War intelligence material was exempt from classification because of the Nazi War Crimes Disclosure Act of 1998 (P.L. 105-246). The Act was used to find information on the Holocaust and war crimes, but it also applied to people with Nazi pasts who worked for the U.S. government. This led to a massive uncovering of the way in which the CIA used agents with Nazi pasts against the Communists. The Act also helped to declassify information about the Gehlen Organization, the German Federal Intelligence Service (Bundesnachrichtendienst [BND]), and the CIA. This information was also invaluable in illustrating the way in which intelligence agencies do unethical things under the cloak of secrecy and expediency (Brietman et al, 2005;Brietman 2001).
And even though some of the richest sources of material I found were in the British MI5 files for the secret writing topic, the more significant material historically resided in the U.S. and Stasi files. The question then arises: Did revealing the secrets damage national security? The answer is no. However, if the material had been unveiled earlier, it would likely have damaged national security. This only underscores the compromise measure of declassifying material once it has reached its expiration date.