Answer from Harry Potter and the Methods of Rationality

1. Magic itself is fading in the world. Not the fault of the practitioners.


2. Magical people are interbreeding with non-magical ones, thus diluting their strength in magic.


3. Knowledge to cast powerful spells is being lost.


4. Magical people are eating the wrong foods as children, or something else besides heredity is making them grow up weaker.


5. Non-magical technology is interfering with magic.


6. Strong magical people are having fewer children. Either out of preference or because magic affects their fertility.

Investigations would have to be carried out to determine which of these is the case. See HPMOR chapers 22-23 for results.

Larry Niven presented the scenario in which "mana", the energy on which magic depends, slowly ebbs and disappears (and in another story, it comes back, when the Earth passes through a region of space which is richer in mana, that rains on Earth as stardust).

A similar model is also present in The Zero Curse series by Christopher Nuttall; the ambient magic (outside of a Nexus) can be depleted by over-use, or by some clever gimmick that converts it into waste heat, not unlike Niven's anti-mana wheel.

It is also possible for the magic to actually be the "laws" of another Universe seeping into ours in a region of space where the fabric of the continuum stretches thin (see e.g. The Gods Themselves by Isaac Asimov). If that is the case, we can conflate Niven's dependency of magic from the Earth path in the Galaxy and Asimov's physical laws leaking and obtain magic that wanes semi-periodically.

The magic could also be there, but become less easily reachable (or utterly unreachable) in response to different and incompatible types of magic, or the "negative magic" of skepticism (this last in "Eyes", by David Brin, if I recall the title correctly). Or because whatever it is that empowers magic users (Merlin's gene, or blood nanomachines, or midichlorians) gets diluted with the generations.

Or it's some mumbo-jumbo quantum effect, and is being lost due to dilation of the time-space continuum. The Big Bang was actually the most magic moment of all eras. Something akin to this in David Brin's Uplift universe, with the galaxies becoming more and more isolated.

And finally, since we're talking about magic, just because. Not knowing the why's and wherefore's of magic disappearance could just be integrated in the plot.

Magic depends on Mana, a natural resource that fuels it. No mana, no magic.

Mana may or may not be a renewable resource. Whichever it is, humanity can no longer tap its source as it did in past eras.

Alternatively, the amount or mana is constant, but the amount of users has grown way too much. That means less mana per creature available for magical operations.

Total amount of magical energy in the world is constant, but because human (elvish, etc) population keeps growing the powers spread thinner and thinner. In scientific terms - the average magical flux density is decreasing, making magic more and more difficult to master for even the most skilled practitioners in subsequent generations.

Once it gets below some minimum threshold no one will be able to do magic anymore, until plague or nuclear war reduces the population back to the level of good old magical times (around 15 century, or 500 million people)

Storing Magic depletes it

Magic doesn't obey the 2nd law of thermodynamics, in fact, casting spells creates mana in the world... However a wizard can only hold so much mana within themselves at a time, so to cast more powerful spells wizards have long embued objects such as staves and wands and magic crystals with mana sinks. These kind of act like perpetually charging batteries, soaking up ambient mana in order which a wizard can later tap at will to power their spells. However these magic items are "lossy" and over time they slowly deplete the mana level in the area. As wizarding implements are lost, (or buried with ) when a wizard dies eventually there was a tipping point where more mana was being drained into the mana sinks than was being created by the magical casting ... Unfortunately the wizards didn't realize this and so started making even more magic items to bolster their powers, which caused the mana to fade even faster.

Want mana to return one day? The mana crystals aren't permanent, after hundreds (or thousands) of years the enchantments wear off and their stored mana is released back into the wild...

Simple obsolescence. Magic is just not that useful anymore.

As technology evolves, we find easier and easier ways of accomplishing the same tasks we've always had to do. And as these things are discovered, they quickly replace the older methods.

Consider walking.

Throughout history this was a primary mode of transportation for many people. Unless you had the money to own a horse, or another animal for transportation, you had to walk to get to where you needed to be. And so a lot of people walked, and they got good at walking, developing the necessary muscles and stamina needed to keep walking for a long time.

But how useful is walking in today's society? Well, depending on where you live, walking is almost a lost art. There is simply no need for it. With the ready availability of cars, or other public transportation, the most you need to walk is to and from your vehicle. Technology does the bulk of the work in getting you places, while you relax in a comfortable seat.

Magic is the same way

Unless the magical system in your world is particularly generous, most spell casting involves a level of effort proportional to the effect. Whether it be prolonged magical chanting, complex mystical runes, or simple mental/physical strain, there is a cost to casting a spell. Now, in the past, the benefits of knowing magic outweighed the costs. Knowing a haste spell meant getting places faster, telepathy sped up communications, and knowing an ice spell or two meant better food preservation, and not starving when the winter came.

But with the advent of technology? Why would I strain for hours, if not days, to create a scrying mirror to observe the surroundings of my home when I can just install a camera. The haste spell still doesn't let me outrun a car, and the ice spell just needs too much upkeep compared to a fridge. In a world such as this, magic would no longer be useful for the majority of its inhabitants. Easier to use, possibly cheaper, options via technology are widely available, and so the number of magical users declines.

But this alone is not enough to account for the prodigies, the mages who in ancient times would have single-handedly stood as a powerful deterrent to conventional armies, and shaped the elements with their will.

A genius is not always needed

There is a widely accepted view in programming, that a single highly skilled programmer is often more of a detriment than a benefit to the team where they work. If the difference in skill between them and the rest of the team is too great, then the work that they do will be too difficult for the rest of the team to follow. And consequently, no one but that lone programmer will be able to modify or improve on what they did.

As I see it, there is no greater gap in skill than magic.

As such, most large organizations would be very careful not to become dependent on a single mage. An army of golems is a fantastic work force, but if each change in design of the product they are making requires the original mage's intervention, most companies will likely settle for a less efficient technological solution. Because, don't forget, magic is hard, and there are unlikely to be an army of mages capable of operating at the scale required. Meanwhile, technicians can be trained with a mere 4 years of college.

But not all is lost!

Despite all of this, I would still argue that if nothing else, magic is cool. By that reason alone, your world will probably never fully lose its mages. Children will be interested in learning it to play tricks on their peers, or impress a crush. Adults might know a spell or two that makes their daily lives slightly easier. This low level use will allow the people of your world to find out if they have an affinity for magic, and then those with the drive to pursue it can go on to become full fledged mages.

I imagine their roles would be more like Olympic athletes, or the innovators and visionaries in your world. The difficulty in acquiring, maintaining, and using their skills will weed out all but the best, but by the same token their numbers will be tiny in comparison to the rest of the population.

And so there you have it, a waning magic in your world through no fault of its own. Just people being people.

No one suspects the Spanish Inquisition, i.e. magic has deliberately been suppressed because its too useful, if single individuals capable of using magic can match legions then they'll be hunted by governments either to use them or eliminate them. The end result is that most magical bloodlines are destroyed, and the remaining magical families' abilities are diluted by out breeding with non-magical people while in hiding.

Alternately magic requires certain materials that have been used up due to the extinct of their sources, dragon bone and the like. Or magic can only be used in the absence of a certain material, traditionally Iron, magic was destroyed by the coming of the Iron Age, this also made the European conquest of the New World an absolute certain as the European's steel and iron armour and weapons drained the native wizards and priests of arcane power by their very presence.

Distrust

CASE 1

Nonmagical beings distrust those that use magic, much like the times of Salem, Massachussetts. Over time intelligent nonmagical humans increasingly start to harness technology to counteract and eventually suppress magical beings that they distrust.

CASE 2

Humans started out nomadic and part of familial tribes. As people started to settle down and form larger societies these families kept their magical ways secret from others. Over time knowledge of the arcane was lost because of the fear of sharing that knowledge caused it to fall into legend. Similar to a trade secret, it could be knowledge of incantations, recipes, rituals, or locations (like ley lines)

Depletion of Resources

Magic is tied to specific resources those resources could be magical herbs, foods, animal parts, etc that over time become endangered or extinct. This could be due to overuse, disease, or some other outside force like natural disaster.

"The wheel of time" answer

Men and women are using different sources of magic. The source men are using is tainted, and men who use magic turn mad and very destructive. As a result men practicing magic are hunted down and killed.

Unfortunately the ability to practice magic is inherited from the parents, so with less men practicing it, the magic trait is slowly getting diluted and lost.

Magic relies on a combination of belief and inherent magical ability, you need both for it to work.

A highly technological person (scientist/engineer etc) isn't actually affected by magic that much as they don't believe in it. Likewise, a high-level mage is a being of magic and isn't affected by technological things very much. Full disclosure - this is from the old game "Arcanum".

As the level of technology available to the average person grows, they no longer need the magic so much. An injury that needed magic to heal can now be sorted out by a doctor down the road. Rocks from a cave-in can be cleared by explosives or a crane, instead of sending out for the nearest mage.

This means that less and less people believe in it across most of the world, but there are still areas of heavy magic users where it is quite effective.

A counter argument

Magic isn't declining

Everything is about perception and expectation. Magic isn't declining, it's just that with the advent of technology, the things magic is capable of just don't seem so, well, magical any more.

In a low tech society doing more complex tasks with ease appears phenomenal.

Consider the perception of a simple fire lighting magic when compared to spending 20mins with a fire drill, as opposed to an involved fire spell when compared to a cigarette lighter. Or the legend of the great mage who brought down a castle when told to a man sitting in a bulldozer.

The reason there doesn't seem to be so much magic around is simply that it's been superseded by the practicalities of technological progress. Hence the tales of great works of magic in times past are simply the folk tales of an earlier age when the baseline of comparison was much lower.

Manadynamic Second Law

Alchemists have discovered why mana is fading from the world. From our Arcanapedia, the grimoire of lasting knowledge:

The Manadynamic is the branch of Arcanes and Hidden Studies that has to do with mana and aether and their relations to energy and work. The behaviour of these quantities is governed by the Four Greatest Law of Magic and the Real of Beyond.

We already know about the First Greatest Law of Manadynamic. For non-magical people which aren't able to read the Arcane writings, this law states that mana and aether can't be created nor destroyed, it's only a process of transformation.

And the Second Greatest Law of Manadynamic states the amount of chaos or entropy in the universe increases over time.

Like every magician learns in the first year of college, in order to cast a spell, the magician needs to perform a flow of energy from an object (or space itself) with a higher state of arcane particles (mana) to another with a lesser state. This can't work in the reverse process. Each time magicians cast spells, the magical energies distribute more chaotically on the area until becoming them useless for magicians.

Currently, no mage was able to gather this dissipated mana to perform magic.

Over time, the amount of mana in the planet was decreasing, either for magicians who spend mana or for the irradiated aether from black arcane bodies and so the planet was losing all it magical power over time.

We don't know from were come to this great mana in the first place, some arcanists state that it comes from the Sun, but if that were true we would still have magical energies on Earth. Others suggest it comes from the Earth creation periods, some that it comes from the Magical Impact which produced the Wild Magical Crater also know as Chichxulub crater, or from the Teory of the Giant Arcane Body Impact.

We evolve or mana mutates

The other theory of alchemist and magicians is that there is some kind of change in the mana and body relation other the centuries.

They aren't still sure if it's part of our evolution, and so we are losing our magical genomes which let us gain access to this hidden arts or if the mana itself is mutating, like a living organism, and our bodies aren't able to adapt to this mutation, and os we are losing our affinity. Maybe, after the First Spell Plague of Earth, the magical net which covered the universe started to mutate by itself.

As humans evolved from nomadic tribes, through agrarian and industrial societies, to the current technology-driven species we are now, we have always attributed events that we don't understand to "magic" or some other metaphysical phenomena. However, human curiosity being what it is, we have always eventually discovered the real nature of what were previously "magical" events. The tides, the seasons, sunrise/sunset, etc... these were all at one time "magical". Turning stone into fire was at one time "magical". But now we know about where Earth is in the Solar System, and we know how it moves in the Solar System. We know that certain kinds of stones and minerals are, in fact, combustible, and in the right circumstances, explosive.

The point is that magic isn't necessarily disappearing, or fading, as is fabled. It's that we're slowly finding rational, physical explanations for events that were previously "magical" and metaphysical. As we do that, there are fewer and fewer events attributed to "magic". We've been doing this for so long now that we've come to the conclusion that there never was any actual magic (except the concept that we bore ourselves), and that everything (probably) has a physical explanation. These physical explanations may elude us, for now, but we are confident that we will eventually tease out these explanations. We are brave enough to put away the facade of "magic" and face the prospect that we simply don't know everything. Because of this, we are emboldened to push all the harder to learn everything that can be learned.

There's an old story I read in which wishes would all come true and thus food, shelter, etc. were never problems.

Eventually, someone wished that wishes would no longer come true.

This extremely simple idea could be built upon or complicated to suit your preferences. For example, it could be that the structure of magic was such that the "fuel" it required was the interest or belief of its practitioners. In such a wise a propaganda campaign against magic, deriding it and its practitioners, could be very effective in actually stopping magic from working. (In Madeleine L'Engle's book Many Waters, "some things have to be believed to be seen," such as unicorns, which are tied up with quantum entanglement and subatomic phenomena.)

The basic idea here is to reverse the usual cause-effect sequence with regard to modern notions of magic. That is, rather than people becoming more skeptical of magic because it stops working, you can posit that magic stopped working because people became more skeptical of it.

Or it could simply be that magic is 100% reproducible, but that it involves a mental component. And therefore anyone who merely goes through the motions as a "scientist" with the unscientific attitude that it probably won't work anyway, will only have their presupposition confirmed. In this wise, there could even be a resurgence of magic, but academics and intelligentsia who are too invested in the ordinary, material functionality of the universe, would deny that it existed and would instead comment on the rise of gullibility on the part of the general populace (since there are an increasing number of people believing in non-reproducible "unscientific" magic), not realizing the results are perfectly reproducible except by confirmed skeptics. This chapter of "Ra" is somewhat relevant to this approach, in that going through the motions isn't sufficient to repeat the experiment; one must believe and follow through mentally as well.

If you consider magic to be a type of stored energy then overuse will deplete that stored energy. Think of a high-energy capacitor. The practitioners of magic know how useful it is, but as the conduits of that energy they also suffer side effects, such as the blackouts and hemorrhaging among superhumans in the X-Men and Heroes universe so they eventually choose technology over magic. Over time some magicians become so powerful that they decide magic is too dangerous for novice magicians to use, so they block off the channels of magic which would have recharged Earth.

Plastic Reality

How about it being related to the nature of reality? Perhaps Reality is defined by what you think when you look at it.

Primitive people know very little, and make simple magical explanations for everything. So reality says "magic it is then" and everything is magical.

As time moves on we have a scientific movement. People begin studying things in detail and making scientific explanations for things. Reality says "ok science it is then" and magic falters and science is ascendant.

In other words, in Tolkien's universe the Sun is a vessel that holds the radiance of the last fruit of Laurelin, which is guided across the sky by essentially an angel. But as time goes by and eventually mankind develops the first telescope, thanks to changing beliefs reality adapts to that changing belief. By the time the telescope is ready the sun is now a ball of nuclear gas.

It's rather an agreement than some fundamental erosion of magic. A great war which saw many successful uses of magical weapons ended up devastating the entire continent with virtually no winners. The major powers signed a convention which made them destroy any magical artifacts, confine the surviving wizards to heavily guarded "magic schools", and attempt to erase any knowledge of the existence of magic.

They were largely successful. All mentions of the great war were destroyed or replaced with a hastily invented "great plague", which explained entire cities being wiped out by some unknown force. The art of magic was quickly forgotten. The later attempts to reinvent it were being actively suppressed - hence the witch hunts.

The loss of magic is a metaphor for growing up - and for progress...

In writing (yeah, yeah, I know this is Worldbuilding), it is indeed a common trope for magic to have been more powerful in the past than it is in the present. This is a natural reflection of the non-fiction literature, which takes magic fairly seriously and literally the further back in time you go. Additionally, as children grow up, they often grow more skeptical - less prone to either believe or propose magical explanations to account for things. But adults, and students of history, are not unaware that things seemed more mysterious, more mystical and strange, the farther back they probe in both recorded history and in their own personal histories.

As has been alluded to elsewhere, magic is less necessary to explain things when things are better understood, with mundane mechanisms accounting for most outcomes.

...And that's exactly the way we want you humans to think.

Honestly, when you humans find the chinks and cracks in your reality, and slip through them, you just get in the way of the rest of us, muddling things up. So now and again some mystery slips out, but we've got you properly trained lately, so your "sensible" sorts discourage the kind of sideways thinking that can deposit you in our realms, or that lets our, er, messes leak out into YOUR realm. Sure, small children haven't been trained right to avoid getting tangled up, but they're less of a problem if they slip through, usually. And what they have to say when we send them back... Nobody will believe them, so it's safer TO send them back.

I mean, most human superstitions are exactly and entirely superstitions - you have no real sense for what is actually magic, and what is your own imaginations, which is a great deal of what makes you such blundering buffoons when you get into our worlds. But that also helps in keeping you trained to be skeptical. If most mysticism wasn't bunk, you'd never let yourselves be trained to disbelieve.

Just trust me, you're better off with the separation, as are us more inherently magical creatures. Which is why most of US have been pushing for the separation for so long - with constantly improving success.

Supply and demand

• There is a constant and limited supply of mana from the Source X. Be it Astral plane, energy field, Sun, process in the core of a planet.




• It may or may not accumulate in some ores, crystals or what is needed for the story, but it is a slow process. Renewal is limited by ambient mana, but you can make them even more rare by additional factors.

Golden Age of Magic:

• Mages demand did not deplete the ambient mana supply, as there was a low population.




• There may have been massive deposits of "magic stones" and renewal was able to keep up with the consumption of few mages.

Modern Age

• Huge population( x 100-200 times) with just passive absorption can lower concentration of mana and make it less useful. And many mages and uses of mana for modern technology makes it even worse. Less useful and more dependent on talent and wealth.



• 
  

  "Magic deposits" are close to bottom and slow in renewal, even more so with low ambient mana. That makes them incredibly rare and expensive. Countries will control their flow and do all it can to find ways to make it less dependent on magic in every way possible.

  
  
  

  With time you have magic independent technology, that can rival magic on battlefield and in economics. Magic is a relic or reserved for ones at the top of pecking order.

  
  

Or go from other side.

Slowly that world created technology to rival magic. But it is more convenient and profitable.

• Everyone can use it. You do not need talent and training. No need for needed elements or a like. Even a cripple can use it.




• It is powered by cheap, external source of energy with no need for you there.




• Profits. Everyone can start a firm and sell products based on technology. And as everyone can use them, you have huge market.




• Law and order. Mages are like people with guns. You keep track of them and regulate them. Maybe, they even tried to rebel and government placed strict regulations and now there is some prejudice against them.

Passive drain of mana will only make it more easy choice to go technology path and forget magic one. At least for most.

Lots of very good answers - I just wanted to add a variation on suppression and mana limitations.

If there is a limited supply of mana, and it either doesn't replenish, or does so very slowly, an expanding population of mages could burn through it very quickly, leading to fierce competition for resources.

In response, in would in the be interests of the established to suppress the competition however they could. Less mages in the world means more magic for whoever's left. This could take several forms;

• Extremely strict controls on teaching and apprenticeships


• Taking all possible books that could help someone learn magic by themselves out of circulation


• Fierce competition amongst mages, including elimination of weaker rivals

Taken to a logical extreme, mages may in fact try to suppress all magic from the public consciousness, in order to minimise the amount of competition for mana. This could lead to a secret world / masquerade situation where the public thinks magic is just a myth.

You could also make it so the maximum strength of a mage is inversely proportional to the number of mages in the world - so for each additional magic user, every mage gets slightly weaker.

Assuming that magic exists, how can the decline of magic be explained given its huge utility - and presumably potential economic value?

I'm dubious as to the economic value of magic. Sure, it's economically positive to get your cows blessed by the village witch, but if she gets angry at you, then your cows are cursed.

Knowledge, on the other hand, is far more egalitarian. No special arcane communion with the spirit world needed be taught hygiene, nutrition, the making of charcoal, smelting metals, etc.