It is the dead of night. The day’s red-dust heat has given way to a cooling breeze. A hundred frogs chirp urgently. Tim and his crew are preparing for another stealth raid. Their mission is highly dangerous and now there’s a new threat: armed men are following them.

This is the scene repeated nightly on the eastern fringes of Amboseli national park in Kenya, close to the border with Tanzania. Tim is an elephant who, along with a group of up to 12 other males, has developed a taste for the tomatoes and maize growing on local farms on the outskirts of the park. The armed men are park rangers who have been tasked with keeping him from the crops – and saving his life.

The nocturnal game of cat and elephant is just one example of a much bigger problem playing out across Africa and Asia. It is the sharp end of an existential conflict between people and wildlife for land, food and water. It is also a departure from the traditional story of elephant conservation, which presents the big threat to the world’s largest land animal as ivory poachers and the trinket-buyers in Chinese bazaars. The ivory trade has had a significant impact, for sure, but habitat destruction caused by human population growth and development is a far more pervasive threat.

“Poaching attracts a lot of media attention, but it’s only part of a big picture,” says Julian Blanc of the UN Environment Programme in Nairobi, Kenya. “If we somehow stopped poaching tomorrow, elephants would still be in big trouble.” Habitat loss in Africa threatens many other species too, from giraffes to geckos.

Preventing habitat loss is a harder cause for celebrities and campaigning NGOs to champion (although some do). It is a more subtle narrative, full of shades of grey rather than the black-and-white certainties of the ivory trade.

It was easy for animal lovers to say “not in my name” while denouncing the buyers who fuel poaching. But consumers in the developed world and their governments also have an impact on elephant habitat through the goods that we buy and the development schemes that our governments fund.

A commercial enterprise growing fruit or flowers, for example, on land that was once elephant habitat contributes just as surely to the population’s decline as a poacher’s bullet – even if the connection may not be so immediate or so direct.

The moral argument is complex because the consumers, NGOs or governments that support such businesses are also helping to lift hundreds or thousands of people out of poverty by providing them with a livelihood. But from a conservation point of view it is often a zero-sum game: humans win, wildlife loses.

“In essence, the elephant story in Africa encompasses many aspects of your life,” says Holly Dublin, chair of the African elephant specialist group at the International Union for Conservation of Nature. “That’s how complex it is. You are implicated in this. Not just bad people who buy ivory.”

Conflict between humans and elephants is most acute in Asia. Every year, around 100 people and 40 to 50 elephants are killed as a result in India, for example.

But the problem is set to escalate sharply in Africa as development gathers pace. Governments here bear primary responsibility for wildlife stewardship, but conservation is getting harder by the year as populations increase and more infrastructure such as roads and railways is built to service the continent’s growing economy. These provide barriers to wildlife migrations, which are particularly important for elephants. Some groups have home ranges of over 3,000 sq km.

Understanding the impact of this new infrastructure will require more research. One such study has already begun in Kenya, to investigate how elephants are reacting to the newly opened (pdf) Standard Gauge Railway (SGR) between the capital Nairobi and the coastal city of Mombassa. The route runs parallel to a major road for much of its length.

The 23,000 sq km Tsavo ecosystem, comprising Tsavo East and Tsavo West national parks, which is bisected by the railway, is home to between 12,000 and 14,000 elephants – Kenya’s largest population.

Although engineers have created six 70m-wide underpasses to act as wildlife crossing points (there are other bridges and culverts where wildlife may be able to cross too), researchers are worried that if elephants don’t use them the population will be cut in two. “98% of the ecosystem connectivity has now been blocked,” says Ben Okita-Ouma, head of monitoring at the NGO Save the Elephants.

To understand the impact on the animals, Okita-Ouma’s team has fitted 10 elephants with radio collars and tracked their movements. The results are preliminary but the team found evidence the animals were wandering up and down the railway line, apparently looking for crossing locations. At times they even climbed over the railway embankment, crossing the tracks and destroying fences in the process.

Okita-Ouma believes the elephants and other animals will begin to learn where the crossing points are, but this will lead to high densities of wildlife at those points. That will potentially lead to more contact with humans, and accidents on the busy highway that runs alongside the SGR.

Okita-Ouma says illegal settlements are growing around some of these underpasses. “The government needs to come in quickly and with a lot of force to ensure that the illegal settlements are completely removed.” That will likely save human and elephant lives.

The pilot study also recommends creating wildlife overpasses on the road near the railway crossing points.

A more widespread issue as Africa’s human population grows is that the competition for land is becoming more intense. “If you need that land for people, it’s going to compromise the wildlife,” says Dublin, who asks, “what’s the plan” for these large blocks of land that both wildlife and people depend on?

According to the 2013 Elephants in the Dust report by a group of conservation NGOs, an estimated 29% of the animals’ known and possible range is heavily affected by human development. That figure is predicted to rise to 63% by 2050.

At present, much development happens without any planning oversight – often with piecemeal encroachment by arable farmers into land that was previously wild. In other cases – for example Laikipia in Kenya – there have been mass incursions into wildlife conservancies prompted by pastoralists in search of grazing for their drought-starved cattle. This brings humans and wildlife into conflict, with dangerous consequences.

“Elephants are not quite the gentle giants that many people in the west perceive them to be,” says Blanc. “They destroy crops, they kill people, often without provocation – they are one of the top killers of people in Africa.”

“One elephant is enough to destroy a year’s worth of crops for a multitude of families and create a lot of economic hardship,” he adds.

One approach is to manage that conflict where it happens. That’s where the night-time attempts to save Tim from his own appetite come in.

He has been radio-collared by scientists who, with the help of park rangers, are trying to prevent him and other males from raiding crops on farms that fringe the park.

“What we’re trying to do with this early warning system is to respond faster because we can anticipate crop raiding,” explains Ryan Wilkie of Save the Elephants. The response consists of nocturnal patrols by rangers equipped with pepper-pellet guns, thunderflash grenades and a deafening horn.

“He’s definitely incredibly smart. I mean I’m enamoured of this elephant – just trying to understand how he’s thinking … He’s certainly a very intelligent and strategic thinker,” says Wilkie.

Important though the efforts to protect Tim from himself are (he has been speared by angry farmers three times in his crop-raiding career), it would be impossible to scale this approach up to the 415,000 or so elephants on the continent. And as the pressure on habitat grows, some elephant conservationists are arguing for a more radical, continent-wide approach.

“By the time you get to the space where people and elephants are in conflict and you’re trying to mitigate there, you’re just talking about taking an aspirin for cancer,” says Dublin. Move one group of crop-raiding elephants and it will be replaced by another.

With Africa’s population set to double by 2050 from 1.2 billion now (it was 477 million in 1980), Dublin argues that countries must get ahead of human-wildlife conflict and create large-scale land-use plans to allow people and animals to coexist. The alternative will be a conservation disaster.

“Nobody’s doing any spatial planning,” Dublin complains. “That fact that you are getting elephants and humans into such close contact with each other again and again means that planners were not thinking long-term enough.”

But before they can make those longer-term plans for how land will be used, policymakers need better information about where the flashpoints are likely to be. Because elephants roam so widely, protecting the territory they need will also benefit other species.

Dublin is embarking on a major project to combine data on elephant population ranges with information about human development. Where is human population growing fastest on the continent? How are humans and elephants likely to move in response to climate change? Where are major agriculture and infrastructure schemes planned? The aim, she says, is to identify “the areas for highest probability of conflict and the areas for the highest probability of coexistence”.

The project is likely to take a couple of years to complete, but smaller-scale studies are already revealing the choices that must be made between animal welfare and human development. In Kenya, there are calls from conservationists to alter the planned Lamu Port, South Sudan, Ethiopia Transport (Lapsset) development corridor. The $29bn (£23bn) project involves a new port, highways, railways, airports, pipelines and three new resort cities, and is a key pillar in Kenya’s development plans. But consultants carrying out an environmental impact assessment of one of the proposed resort cities at Isiolo, in the centre of the country, have recommended that it be moved elsewhere to avoid disrupting elephant habitat and an important migration corridor. Investors who have already bought land in the area are likely to fiercely resist the change, so whether the plans are modified will be a major test of Lapsset’s environmental credentials.

Okita-Ouma says that across the continent, development almost always trumps conservation. Governments “have got to take into account the needs for wildlife in a much more serious manner”, he says. “At the moment very little action is taking place, and if there is any action then it isn’t what you require and want to see.”

Dublin predicts that with the pressure on habitat growing, there will be more tough choices ahead: “Sadly, I think what you are going to find is that much of the elephant range is going to be under siege. I mean, look what the plans are for Africa. Elephants live where there’s water. So does agriculture. So do humans. It will be difficult to have it all.”