The FBI breached its own internal rules when it spied on campaigners against the Keystone XL pipeline, failing to get approval before it cultivated informants and opened files on individuals protesting against the construction of the pipeline in Texas, documents reveal.

Internal agency documents show for the first time how FBI agents have been closely monitoring anti-Keystone activists, in violation of guidelines designed to prevent the agency from becoming unduly involved in sensitive political issues.

The hugely contentious Keystone XL pipeline, which is awaiting approval from the Obama administration, would transport tar sands oil from Canada to the Texas Gulf coast.

It has been strongly opposed for years by a coalition of environmental groups, including some involved in nonviolent civil disobedience who have been monitored by federal law enforcement agencies.

The documents reveal that one FBI investigation, run from its Houston field office, amounted to “substantial non-compliance” of Department of Justice rules that govern how the agency should handle sensitive matters.

One FBI memo, which set out the rationale for investigating campaigners in the Houston area, touted the economic advantages of the pipeline while labelling its opponents “environmental extremists”.

“Many of these extremists believe the debates over pollution, protection of wildlife, safety, and property rights have been overshadowed by the promise of jobs and cheaper oil prices,” the FBI document states. “The Keystone pipeline, as part of the oil and natural gas industry, is vital to the security and economy of the United States.”

The documents are among more than 80 pages of previously confidential FBI files obtained by the Guardian and Earth Island Journal after a request under the Freedom of Information Act.

Between November 2012 and June 2014, the documents show, the FBI collated inside knowledge about forthcoming protests, documented the identities of individuals photographing oil-related infrastructure, scrutinised police intelligence and cultivated at least one informant.

It is unclear whether the source or sources were protesters-turned-informants, private investigators or hackers. One source is referred to in the documents as having had “good access and a history of reliable reporting”.

The FBI investigation targeted Tar Sands Blockade, a direct action group that was at the time campaigning in southern Texas.

However, the partially redacted documents reveal the investigation into anti-Keystone activists occurred without prior approval of the top lawyer and senior agent in the Houston field office, a stipulation laid down in rules provided by the attorney general.

Confronted by evidence contained in the cache of documents, the agency admitted that “FBI approval levels required by internal policy were not initially obtained” for the investigation, but said the failure was remedied and later reported internally.

The FBI files appear to suggest the Houston branch of the investigation was opened in early 2013, several months after a high-level strategy meeting between the agency and TransCanada, the company building the pipeline.

For a period of time – possibly as long as eight months – agents acting beyond their authority were monitoring activists aligned with Tar Sands Blockade.

Tar Sands Blockade appeared on the FBI’s radar in late 2012, not long after the group began organising in east Houston, the end destination for Keystone’s 1,660-mile pipeline.

Environmental activists affiliated with the group were committed to peaceful civil disobedience that can involve minor infractions of law, such as trespass. But they had no history of violent or serious crime.

Ron Seifert, a key organiser at Tar Sands Blockade, said dozens of campaigners were arrested in Texas for protest-related activity around that time, but not one of them was accused of violent crime or property destruction.

The group focused on Houston’s heavily industrialised neighbourhood of Manchester, where the Valero Energy Corporation has a massive refinery capable of processing heavy crude oil.

Between early November 2012 and June 2014, the documents show, the FBI collated inside-knowledge about forthcoming protests, documented the identities of individuals photographing oil-related infrastructure, scrutinised police intelligence and cultivated at least one informant.

It is unclear whether the source or sources were protesters-turned-informants, private investigators or hackers. One source is referred to in the documents as having had “good access, and a history of reliable reporting”.

At one point, the FBI’s Houston office said it would share with TransCanada “any pertinent intelligence regarding any threats” to the company in advance of a forthcoming protest.

One of the files refers to Houston police officers who stopped two men and a woman taking photographs near the city’s industrial port, noting they were using a “large and sophisticated looking” camera.

Two of the individuals were described as having larger subject files in the FBI’s Guardian Threat Tracking System.

In another incident, the license plate belonging to a Silver Dodge was dutifully entered into the FBI’s database, after a “source” spotted the driver and another man photographing a building associated with TransCanada.

Sensitive matters

The FBI rules, laid out in the FBI’s Domestic Investigations and Operations Guide, dictate that special care should be taken over sensitive investigations such as those targeting elected officials, journalists and political organisations.

FBI work on “sensitive investigative matters” requires prior approval of both the chief division counsel (CDC), the top lawyer in the field office, and the special agent in charge (SAC).

Both are supposed to consider the severity of the threat and the consequences of “adverse impact on civil liberties and public confidence” should the investigation be made public.

However, neither Houston’s CDC or SAC were consulted in relation to the FBI’s monitoring of Tar Sands Blockade activists, the documents show.

Explaining the breach of protocols, the FBI said in a statement that it was committed to “act properly under the law”.

“While the FBI approval levels required by internal policy were not initially obtained, once discovered, corrective action was taken, non-compliance was remedied, and the oversight was properly reported through the FBI’s internal oversight mechanism,” it said.

The FBI did not deny opening an investigation into anti-Keystone campaigners, and said it was compelled to “take the initiative to secure and protect activities and entities which may be targeted for terrorism or espionage”.

But the precise nature of the FBI’s investigation, which continued for almost a year after the Houston Division acknowledged it had violated protocol, remains unclear.

The documents appear to suggest the investigation was one branch of a wider set of investigations, possibly including anti-Keystone activists elsewhere in the country.

The documents connect the investigation into anti-Keystone activists to other “domestic terrorism issues” in the agency and show there was some liaison with the local FBI “assistant weapons of mass destruction coordinator”.

Mike German, a former FBI agent, who assisted the Guardian in deciphering the bureau’s documentation, said they indicated the agency had opened a category of investigation that is known in agency parlance as an “assessment”.

Introduced as part of an expansion of FBI powers after 9/11, assessments allow agents to open intrusive investigations into individuals or groups, even if they have no reason to believe they are breaking the law.

German, now a fellow at the Brennan Center for Justice in New York, said the documents also raised questions over collusion between law enforcement and TransCanada.

“It is clearly troubling that these documents suggest the FBI interprets its national security mandate as protecting private industry from political criticism,” he said.

According to the FBI documents, the FBI concluded there were “no adverse consequences” emanating from its failure to seek approval for the sensitive investigation, noting the mistake was later “remedied”.

The investigation continued for 11 months after the mistake was spotted. It was closed after the FBI’s Houston division acknowledged its failure to find sufficient evidence of “extremist activity”.

Before closing the case, however, agents noted the existence of a file that was to be used as a repository for future intelligence “regarding the Keystone XL pipeline”.

Since then, at least a dozen anti-tar sands campaigners in Oregon, Washington, and Idaho have been contacted by the FBI. The agency has said they are not under investigation.