Btukfyl

I have watched Season 3 of Game of Thrones twice now and I still don't understand why Ramsay Snow engineers a sham escape for Theon Greyjoy.

Just to recap for anyone that does not remember, Ramsay, who is the illegitimate son of Lord Bolton, pretends to be a menial servant after Theon is betrayed and handed over to the Boltons at Winterfell. Then Ramsay, still pretending to be a servant, helps Theon escape, even killing 3 of the Bolton men pursuing him. He then leads Theon back to the Bolton Castle while pretending to lead him to freedom and Theon is imprisoned again.

What is the purpose of this sham escape? Is it explained in the book? Maybe explanatory material was edited out of the film?

What could be more torturous than thinking you are so close to freedom and safety to only realize suddenly that you are completely trapped again?

Ramsey wants to break him mentally as much as he wants to harm him physically. He is going to show that he has total control over all of Reek's life.

In the books, the situation is fairly different by this point, but Ramsay is well known for enjoying the sight of his victims thinking they have escaped, only to find out it was a sham.

From A Dance With Dragons - Reek 1 (A Theon Chapter)

He had run before. Years ago, it seemed, when he still had some strength in him, when he had still been defiant. That time it had been Kyra with the keys. She told him she had stolen them, that she knew a postern gate that was never guarded. "Take me back to Winterfell, m'lord," she begged, pale-faced and trembling. "I don't know the way. I can't escape alone. Come with me, please." And so he had. The gaoler was dead drunk in a puddle of wine, with his breeches down around his ankles. The dungeon door was open and the postern gate had been unguarded, just as she had said. They waited for the moon to go behind a cloud, then slipped from the castle and splashed across the Weeping Water, stumbling over stones, half-frozen by the icy stream. On the far side, he had kissed her. "You've saved us," he said. Fool. Fool.

It had all been a trap, a game, a jape. Lord Ramsay loved the chase and preferred to hunt two-legged prey. All night they ran through the darkling wood, but as the sun came up the sound of a distant horn came faintly through the trees, and they heard the baying of a pack of hounds. "We should split up," he told Kyra as the dogs drew closer. "They cannot track us both." The girl was crazed with fear, though, and refused to leave his side, even when he swore that he would raise a host of ironborn and come back for her if she should be the one they followed.

In addition to the answers that already explain how Ramsay used it as psychological torture to ensure that Theon wouldn't try to escape again in case it was another of Ramsay's traps, in the context of the show he also uses it as a way to interrogate Theon.

As Theon begins to trust the person who helped him escape, he confides in Ramsay at that moment that he never actually killed Bran and Rickon, but instead burned the bodies of 2 farm boys and posed them as the Stark children.

Ramsay later uses this information and shares it with Roose so that they can attempt to track down and kill the remaining Starks, as they could cause problems for them later on by potentially rallying other Northmen who support the Starks against the Boltons.

I believe one more aspect was not covered by previous answers: Ramsay killing some Bolton men during the escape. IIRC, the older Lord Bolton was wary of his bastard, especially as he was (validly) the prime suspect to have killed the legitimate son and so rise to inheritance. So Ramsay was watched and guarded, to not do any more stupidities. And he was eager to lose the tail, perhaps even putting the blame on Theon if need be.